Python garbled in Windows + Git Bash environment

Comfortable bash with Git Bash and Cmder When I tried to start my life, Python was garbled.

$ python -c "print('Japanese')"
譌 ・ 譛 ャ 隱 ・

Conclusion

So add the following two lines to .bashrc.

.bashrc


chcp.com 65001
export PAGER=less

That's all for the conclusion, but I'll leave the background to that.

Problem 1: Garbled characters

$ python -c "print('Japanese')"
譌 ・ 譛 ャ 隱 ・

Investigation

I typed as many character encoding confirmation commands as I could think of.

$ echo $LANG
ja_JP.UTF-8
$ echo $PYTHONIOENCODING
utf-8
$ python -c "import sys; print(sys.stdout.encoding)"
utf-8
$ python -c "import sys; print(sys.getdefaultencoding())"
utf-8
$ python -c "import locale; print(locale.getpreferredencoding(False))"
cp932

It almost returns UTF-8. Only locale.getpreferredencoding () responded like cp932, but according to Documentation, this is:

Returns how to encode the text data based on the user's settings. This function is just a guess **, as user settings are expressed differently between different systems and may not be programmatically available on some systems.

It's not so suspicious. While doing so, I found the following description in the Document of sys.stdout:

Character encoding is platform dependent. ** On Windows, if the stream is interactive (if the isatty () method returns True), the console code page is **, otherwise the ANSI code page is used. Other platforms use locale encoding (see locale.getpreferredencoding ()).

Code page ...!

Action: Code page change

I googled because I didn't know the code page. It seems to be set with the chcp command. Even from Git Bash, if the path is in / C / windows / system32, you can call it with the extension name chcp.com.

$ chcp.com          #Code page confirmation
Current code page: 932
$ chcp.com 65001    #Code page settings(UTF-8)
$ python -c "print('Japanese')"
Japanese

It was fine and I got rid of the garbled characters.

Problem 2: Help display error

I can't see help in Python. It is displayed as Not enough memory..

>>> import sys
>>> help(sys.argv)
Not enough memory.

Investigation

It seems that Windows more.com does not work well under the environment of chcp 65001. When the content you want to display spans several pages, it displays --More (40%) -- or something that prompts you to enter a key. Such a function is called a pager.

On Linux, the less command usually plays that role. I want to use less.exe instead of more.com which is no longer available!

Action: Change pager

Python seems to use it if you specify it in the environment variable PAGER. I couldn't find any mention of the pager in the documentation. Is it self-evident, or is it a feature other than Python?

$ export PAGER=less
$ python
>>> import sys
>>> help(sys.argv)  #From here on down, the less screen is displayed, so the Python interpreter is hidden once.
Help on list object:

class list(object)
 |  list() -> new empty list
 |  list(iterable) -> new list initialized from iterable's items
 |
 |  Methods defined here:
 |
...

Congratulations help is back. This time I noticed in the Python help display, but other people using more.com are likely to run into the same problem. Even in that case, there may be something that can be solved with ʻexport PAGER = less`.

Digression

getpreferredencoding not good enough

Try running locale.getpreferredencoding () again with the resolution resolved.

$ python -c "import locale; print(locale.getpreferredencoding(False))"
cp932

After all acquisition by getpreferredencoding is not good enough.

cmd /? becomes English

If chcp 65001 is set, the result ofcmd /?will be in English. If you do start cmd /?, Japanese will be displayed.

reference: [Command Prompt / How to switch between English mode and Japanese mode-chcp --Living with Windows](http://win.just4fun.biz/%E3%82%B3%E3%83%9E%E3%83%B3%] E3% 83% 89% E3% 83% 97% E3% 83% AD% E3% 83% B3% E3% 83% 97% E3% 83% 88 /% E8% 8B% B1% E8% AA% 9E% E3 % 83% A2% E3% 83% BC% E3% 83% 89% E3% 83% BB% E6% 97% A5% E6% 9C% AC% E8% AA% 9E% E3% 83% A2% E3% 83 % BC% E3% 83% 89% E3% 81% AE% E5% 88% 87% E3% 82% 8A% E6% 9B% BF% E3% 81% 88% E6% 96% B9% E6% B3% 95 % E3% 83% BBchcp.html)

PAGER companion

It seems that the environment variable GIT_PAGER can also be used in git.

History

2016/09/27 About chcp 2016/10/09 Added about PAGER

Recommended Posts

Python garbled in Windows + Git Bash environment
python windows environment
Using venv in Windows + Docker environment [Python]
VScode environment construction (Windows10, Python, C ++, C, Git)
python windows environment construction
Development environment in Python
Python environment construction (Windows10 + Emacs)
Handle environment variables in Python
Build Python environment on Windows
Build python environment on windows
Wrapping git operations in Python
Python environment construction under Windows7 environment
Python install in 2 lines @Windows
Use httpie from windows git bash
[Python] Show multiple windows in Tkinter
Python environment construction memo on Windows 10
Hit a command in Python (Windows)
Anaconda python environment construction on Windows 10
Put MeCab in "Windows 10; Python3.5 (64bit)"
Install python2.7 on windows 32bit environment
Install scrapy in python anaconda environment
Various Anaconda settings in Windows 10 environment
[Python3] Development environment construction << Windows edition >>
Windows10: Install MeCab library in python
install tensorflow in anaconda + python3.5 environment
Install Python development environment on Windows 10
Parsing Git commit logs in Python
UnicodeDecodeError occurs in pip (Windows environment)
Create a Python environment for professionals in VS Code on Windows
Mouse operation using Windows API in Python
How to access environment variables in Python
Python 2.7, 3.4, 3.5 extension module build environment on Windows
Python project environment construction procedure (for windows)
[Python] [Windows] Serial communication in Python using DLL
[Python] [Windows] Take a screen capture in Python
Creating a python virtual environment on Windows
Set up Pipenv in Pycharm in Windows environment
Building scikit-learn in Windows 10 environment using Pycharm
To reference environment variables in Python in Blender
Launch the Python interpreter from Git bash
Windows Qt5.4 Python3.4 QProcess Japanese garbled characters
Windows + gVim + Poetry python development environment construction
Introduced sip-4.14 in python3.2.2 environment with MacOS 10.7.4
Prepare Python development environment for each project in Windows environment (VSCode + virtualEnvWrapper + Pylint)
[Python] Reason for dtype "int32" in Numpy (Windows environment) (as of September 2020)
Quadtree in Python --2
Python in optimization
CURL in python
Metaprogramming in Python
Create a Vim + Python test environment in 1 minute
Python 3.3 in Anaconda
Geocoding in python
SendKeys in Python
Ubuntu18.04.05 Creating a python virtual environment in LTS
Read a file containing garbled lines in Python
Meta-analysis in Python
Let's parse the git commit log in Python!
Building an environment that uses Python in Eclipse
Python environment construction
I want to use Python in the environment of pyenv + pipenv on Windows 10
python environment settings