Bright console colors don't work correctly on Linux

[bradwilson]

Given the following C# code:

using System;
using System.Linq;

namespace colors
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine();

            var colors = Enum.GetValues(typeof(ConsoleColor)).Cast<ConsoleColor>().ToList();
            foreach (var fore in colors)
            {
                foreach (var back in colors)
                {
                    Console.ForegroundColor = fore;
                    Console.BackgroundColor = back;
                    Console.Write(" *** ");

                    Console.ResetColor();
                    Console.Write("  ");
                }

                Console.WriteLine();
            }

            Console.WriteLine();
        }
    }
}

When run on Windows 10, it correctly prints out with bright foreground and background colors:

When run on Linux (in my example, Ubuntu Server 16.04), bright colors are shown dark:

I verified this with bug .NET Core 1.x and 2.0. The reason this came up is because I noticed that the color output from dotnet xunit was broken, as we rely on bright colors when writing our output. I have a temporary workaround in place which emits the correct ANSI codes when running on non-Windows machines. Here you can see the difference between the Windows and Linux outputs:

---

As an interesting postscript, the bug does not seem to affect Windows Subsystem for Linux:

So when verifying this bug, it's important to use a real Linux installation (I used VMs in Hyper-V for my testing).

/tag @karelz @terrajobst

[clairernovotny]

That powershell prompt looks pretty slick....!

[karelz]

@wfurt can you please take a look while @ianhays is out?

[stephentoub]

@bradwilson, what is the value of your TERM environment variable? Is it "xterm"? What happens if you do export TERM=xterm-256color and then try running the program again?

[stephentoub]

For reference:

[wfurt]

TERM is one thing to look for. The colors also possibly depend on profile and termcap/terminfo

[bradwilson]

@stephentoub That's worse, actually:

Neither of my failure cases (SSH or native console) have xterm as their TERM value. The former has cygwin, and the latter has linux. In both cases, xterm-256color shows the results above (and is clearly not right, because it breaks full screen apps like nano).

FWIW, I don't believe that xterm-256color should be required, since the colors supported by ConsoleColor represent the core 16 colors (8+bold) that are always available in any ANSI-compatible terminal, including xterm. This is validated for me because when I replaced the calls to Console.ForegroundColor with their ANSI equivalents, I get correct functionality all around:

[bradwilson]

@wfurt The cygwin terminal supports bold colors, as shown here with the output from msgcat --color=test:

Is .NET Core's color implementation not depending on the bold terminal feature?

[wfurt]

can you please describe what system you are connecting to and how?

What I was trying to say is that terminal handling (and colors) are somewhat more complicated.
The connecting client states it self via TERM and than the OS look into it's internal database and tries to map certain attributes into sequence of bytes. Those sequences are interpreted by your client and than turned into bold or colorful text. For example legacy VT100 could do bold but not colors.

The are parts is profile setting. Many distributions try to do all kinds of fancy stuff.
You can run "set" and see.

The other part of the puzzle may also be output setting. For example, in some cases you would get colors from 'ls'. But you would not if you pipe that to for example 'less' or 'more'.
This is because the output device is not terminal but a pipe.

If I have more details I'll try to repeat than and reproduce behavior you observed.
With that I can try to figure out if that is just some setting or real bug.

[bradwilson]

As described above, this is an Ubuntu Server 16.04 VM, running on Hyper-V. I experience the issue both via SSH (using the ssh client from Git for Windows) as well as via the native Linux console for the server (accessed via Hyper-V's client).

TERM for SSH = cygwin
TERM for Server console = linux

[stephentoub]

Is .NET Core's color implementation not depending on the bold terminal feature?

It is, but whether that's supported is dictated by the terminal, and what the terminal supports is exposed via terminfo, which .NET Core examines to determine what it can do. The terminfo specifies the max number of colors supported, and that's taken into account here when Console writes out the relevant escape string:
https://github.com/dotnet/corefx/blob/38e60f7d5b1bfa68b6dba3481fed63fad29ce315/src/System.Console/src/System/ConsolePal.Unix.cs#L539-L545
That MaxColors value is coming from your terminfo database, based on whatever is specified in the TERM environment variable. When you see the limited set of colors, what do you get when you type infocmp? On around the 4th line of output, you should see something like colors#8, colors#16, colors#256, etc. If the terminal database tells .NET Core that it only supports 8 colors, .NET Core won't try to use more than that, as it doesn't want to output escape sequences that will be gibberish.

[bradwilson]

@stephentoub It says 8 colors with bold, as I would've expected. It would appear that you're not checking for bold support to see if it's available and just assume that 8 is all you can do (rather than bolding the 8 for 16).

#       Reconstructed via infocmp from file: /lib/terminfo/c/cygwin
cygwin|ansi emulation for Cygwin,
        am, hs, mir, msgr, xon,
        colors#8, it#8, pairs#64,
        acsc=+\020\,\021-\030.^Y0\333`\004a\261f\370g\361h\260j\331k\277l\332m\300n\305o~p\304q\304r\304s_t\303u\264v\301w\302x\263y\363z\362{\343|\330}\234~\376,
        bel=^G, bold=\E[1m, clear=\E[H\E[J, cr=^M, cub=\E[%p1%dD,
        cub1=^H, cud=\E[%p1%dB, cud1=\E[B, cuf=\E[%p1%dC,
        cuf1=\E[C, cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA,
        cuu1=\E[A, dch=\E[%p1%dP, dch1=\E[P, dl=\E[%p1%dM,
        dl1=\E[M, ed=\E[J, el=\E[K, el1=\E[1K, fsl=^G, home=\E[H,
        hpa=\E[%i%p1%dG, ht=^I, ich=\E[%p1%d@, ich1=\E[@,
        il=\E[%p1%dL, il1=\E[L, ind=^J, invis=\E[8m, kb2=\E[G,
        kbs=^H, kcub1=\E[D, kcud1=\E[B, kcuf1=\E[C, kcuu1=\E[A,
        kdch1=\E[3~, kend=\E[4~, kf1=\E[[A, kf10=\E[21~,
        kf11=\E[23~, kf12=\E[24~, kf13=\E[25~, kf14=\E[26~,
        kf15=\E[28~, kf16=\E[29~, kf17=\E[31~, kf18=\E[32~,
        kf19=\E[33~, kf2=\E[[B, kf20=\E[34~, kf3=\E[[C, kf4=\E[[D,
        kf5=\E[[E, kf6=\E[17~, kf7=\E[18~, kf8=\E[19~, kf9=\E[20~,
        khome=\E[1~, kich1=\E[2~, knp=\E[6~, kpp=\E[5~, kspd=^Z,
        nel=^M^J, op=\E[39;49m, rc=\E8, rev=\E[7m, ri=\EM,
        rmacs=\E[10m, rmcup=\E[2J\E[?47l\E8, rmir=\E[4l,
        rmpch=\E[10m, rmso=\E[27m, rmul=\E[24m, rs1=\Ec\E]R,
        sc=\E7, setab=\E[4%p1%dm, setaf=\E[3%p1%dm,
        sgr=\E[0;10%?%p1%t;7%;%?%p2%t;4%;%?%p3%t;7%;%?%p6%t;1%;%?%p7%t;8%;%?%p9%t;11%;m,
        sgr0=\E[0;10m, smacs=\E[11m, smcup=\E7\E[?47h,
        smir=\E[4h, smpch=\E[11m, smso=\E[7m, smul=\E[4m, tsl=\E];,
        u6=\E[%i%d;%dR, u7=\E[6n, u8=\E[?6c, u9=\E[c,
        vpa=\E[%i%p1%dd,

[danmoseley]

My ubuntu also supports "dim" (half-bright)

[stephentoub]

It says 8 colors with bold, as I would've expected. It would appear that you're not checking for bold support

@bradwilson, are you referring to the entry bold=\E[1m that it has in addition to the setab=\E[4%p1%dm and setaf=\E[3%p1%dm entries for color? What exactly are you proposing be done... if the specified max colors is 8 rather than 16, and there's a bold entry, output both setfb and bold sequences when changing the foreground color? What about the background color?

[bradwilson]

I'm suggesting that in the face of 8-color + bold, you can at least get the 16 foreground colors (if not the 16 background colors).

[bradwilson]

FWIW, while I think it would be beneficial to the community at large to fix this, the fact that it's broken (and unlikely to be fixed) in 1.x as well as 2.x means that I'll probably leave my ANSI version in indefinitely. I can't imagine this would reach the bar of fixing it in LTS.

[wfurt]

few articles for reference

https://askubuntu.com/questions/875102/gnome-terminal-use-bright-colors-for-bold
https://en.wikipedia.org/wiki/ANSI_escape_code
https://superuser.com/questions/736873/enable-256-colors-for-cygwin-under-mintty

There seems to be general debate if bold/bright shall be used as extra colors.
This can perhaps be applied only in cases when terminal supports 8 color (or less)

[stephentoub]

Yeah, that's why we only consider the color count. But I think it'd be reasonable to augment the code to also consider bold for foreground in the case of 8 colors and bold supported.

[wfurt]

I did some testing with this and I don't think it going to work as proposed.
I tested this with xterm, iTerm2 and Terminal app from OSX with different TERM values.
The Bright/Bold setting is really treated as Bold.

I can also change iTerm2 properties and map color/bold sequences to any color I choose.
This for examples allows to use brighter color for standard 8 color palette.

This setting also has no impact on background colors.
Since this actually does not change the color, I feel it would be wrong to bolt it into color management. If anything, we should expose Bold setting and perhaps others (like underline or blinking) as separate API

[davidvontamar]

infocmp on Linux Debian with Urxvt replies with colors#0x100 which means it supports 256 colors, however .NET Core keeps showing only 8 colors and ignores all bright colors on console.
Please fix this!!

[karelz]

@david-tamar if you have idea how to fix it, we would welcome a contribution. Otherwise I am afraid this is not too high on our priority list.
Please upvote top post to help us gather the feedback on how many people want it ... thanks!

[davidvontamar]

@david-tamar if you have idea how to fix it, we would welcome a contribution. Otherwise I am afraid this is not too high on our priority list.
Please upvote top post to help us gather the feedback on how many people want it ... thanks!

I'm working right now on a solution using ANSI escape codes and testing it, but I'm going to use 256 colors, so my solution is incompatible with the Windows command prompt (I'm also not sure whether it interprets ANSI escape codes properly).

[wfurt]

I'm not sure there is good general fix. Time of monochromatic terminals is long gone. Getting more colors is easy as setting environment variable.
Note that even if you use xterm-256color, you will get only defined existing color.

[davidvontamar]

@david-tamar if you have idea how to fix it, we would welcome a contribution. Otherwise I am afraid this is not too high on our priority list.
Please upvote top post to help us gather the feedback on how many people want it ... thanks!

I'm working right now on a solution using ANSI escape codes and testing it, but I'm going to use 256 colors, so my solution is incompatible with the Windows command prompt (I'm also not sure whether it interprets ANSI escape codes properly).

I just wanted to note that if anyone still needs proper support for True Color (24-bit), 256-colors (8-bit), or just the usual 16 (4-bit) colors and ANSI escape codes (bold/italic/underlined text etc) on terminals that support it on Linux/POSIX, then I have re-implemented the System.Console class and made it fully compatible with ANSI escape codes and their associated color palettes as specified by ECMA. It detects automatically such terminals by specific environment variables or by explicit configuration. The package is available for .NET Core 2.2 and onwards on Nuget as well. Find more details in the link.

[eiriktsarpalis]

Triage: we welcome contributions towards improving this.

[dotnet-policy-service]

Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.

This process is part of our issue cleanup automation.