changeset 24028:a78888d98606

color: be more conservative about setting ANSI mode on Windows (BC)

The current color mode detection on Windows assumes the presence of the
TERM environment variable assumes ANSI is supported. However, this isn't
always true. In MSYS (commonly found as part of MinGW), TERM is set to
"cygwin" and the auto resolved color mode of "ansi" results in escape
sequences getting printed literally to the terminal. The output is
very difficult to read and results in a bad user experience. A
workaround is to activate the pager and have it attend all commands (GNU
less in MSYS can render ANSI terminal sequences properly).

In Cygwin, TERM is set to "xterm." Furthermore, Cygwin supports
displaying these terminal sequences properly (unlike MSYS).

This patch changes the mode auto-detection logic on Windows to be more
conservative about selecting the "ansi" mode. We now only select the
"ansi" mode if TERM is set and it contains the string "xterm" or if
we were unable to talk to win32 APIs to determine the settings. There is
a chance this may take away "ansi" from a terminal that actually
supports it. The recourse for this would be to patch the detection to
act appropriately and to override color.mode until that patch has
landed. However, the author believes this regression is tolerable, since
it means MSYS users won't have gibberish printed by default.

Since MSYS's common pager (less) supports display of ANSI sequences,
there is room to patch the color extensions so it can select the ANSI
color mode if a pager is activated.

Mozilla (being an active user of MSYS) would really appreciate this
being part of the stable branch. However, since I believe it is BC, I
haven't explicitly requested application to stable since I figure that
request will be denied.

--- a/hgext/color.py	Tue Feb 03 19:10:03 2015 -0600
+++ b/hgext/color.py	Tue Feb 03 16:24:32 2015 -0800
@@ -215,9 +215,23 @@
     mode = ui.config('color', 'mode', 'auto')
     realmode = mode
     if mode == 'auto':
-        if os.name == 'nt' and 'TERM' not in os.environ:
-            # looks line a cmd.exe console, use win32 API or nothing
-            realmode = 'win32'
+        if os.name == 'nt':
+            term = os.environ.get('TERM')
+            # TERM won't be defined in a vanilla cmd.exe environment.
+            if not term:
+                realmode = 'win32'
+
+            # UNIX-like environments on Windows such as Cygwin and MSYS will
+            # set TERM. They appear to make a best effort attempt at setting it
+            # to something appropriate. However, not all environments with TERM
+            # defined support ANSI. Since "ansi" could result in terminal
+            # gibberish, we error on the side of selecting "win32". However, if
+            # w32effects is not defined, we almost certainly don't support
+            # "win32", so don't even try.
+            if 'xterm' in term or not w32effects:
+                realmode = 'ansi'
+            else:
+                realmode = 'win32'
         else:
             realmode = 'ansi'