Be careful of LANG for UnicodeEncodeError when printing Japanese with Python 3

Summary

When a UnicodeEncodeError occurs in Python3

phenomenon

I've been using Python 2 for about 7 years now, but I'm thinking about moving to 3 soon.

In Python3, I heard that the character strings were unified to Unicode, which made it convenient, but I stumbled upon UnicodeEncodeError.

The OS is Ubuntu 14.04.4 LTS and Python is 3.5.2 installed by pyenv. The executed code is Hello World below.

hello_ja.py


# coding: utf-8                                                                                            
print("Hello World")

result

% python hello_ja.py
Traceback (most recent call last):
  File "sample.py", line 4, in <module>
    print("\u3053\u3093\u306b\u3061\u306f\u4e16\u754c")
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-6: ordinal not in range(128)

In Python2 series, it was necessary to spend a lot of time using codecs, but in 3 series, this should be possible ...

Countermeasures

memorandum- #sys in python3.std(in|out|err)Enforce encoding of When I read, it seems that I am looking at the environment variable LANG in the character code selection of Python3 series.

When I checked the environment variables, it was Japanese utf-8.

% export | grep LANG
LANG=ja_JP.UTF-8

If LANG = C, this was the cause, but this time it seems different.

When I went back to the basics and checked if the Japanese environment was installed in the first place, language-pack-ja was not installed. I installed it with the English version image, so it seems that I forgot to put it in.

Install the package and set the default locale according to Change the default locale of Debian / Ubuntu.

% sudo apt-get install language-pack-ja
% sudo update-locale LANG=ja_JP.UTF-8

Now you can print Japanese correctly.

% python hello_ja.py
Hello World

Recommended Posts

Be careful of LANG for UnicodeEncodeError when printing Japanese with Python 3
(Note) Be careful with python argparse
[Python] Be careful when using print
UnicodeEncodeError struggle with standard output of python3
Be careful when running CakePHP3 with PHP7.2
[Python memo] Be careful when creating a two-dimensional array (list of lists)
Be careful of the type when making an image mask with Numpy
Investigation when import cannot be done with python
Be careful when working with gzip-compressed text files
When naming variables in Python, be careful not only of reserved words but also of conflicts with built-in functions.
Workaround for the problem that UTF-8 Japanese mail cannot be sent with Flask-Mail (Python3)
Japanese can be used with Python in Docker environment
Be careful when differentiating the eigenvectors of a matrix
Be careful when reading data with pandas (specify dtype)
Summary of tools for operating Windows GUI with Python
Japanese output when dealing with python in visual studio
Precautions when operating with string for TmeStampType of PySpark
Can be used with AtCoder! A collection of techniques for drawing short code in Python!
How to log with python (when No handlers could be found for logger "__main__" appears)
Solution when the image cannot be displayed with tkinter [python]
[For beginners] Summary of standard input in Python (with explanation)
Simulation of late damages for child support delinquency with python
Turn an array of strings with a for statement (Python3)
Avoid UnicodeEncodeError when throwing queries with Japanese in Python's MySQLdb
[Introduction to Udemy Python3 + Application] 51. Be careful with default arguments
Japanese morphological analysis with Python
Error when playing with python
tesseract-OCR for Python [Japanese version]
Here's a summary of things that might be useful when dealing with complex numbers in Python
How to not escape Japanese when dealing with json in python
Mechanism for automatic lint check with flake8 when committing python code
Be careful when specifying the default argument value in Python3 series
I made a lot of files for RDP connection with Python
The story of making a standard driver for db with python.
[Let's play with Python] Aiming for automatic sentence generation ~ Completion of automatic sentence generation ~
From the introduction of JUMAN ++ to morphological analysis of Japanese with Python
Memorandum of understanding when Python is run on EC2 with Apache
A memo for when pip3 is installed with python2.7 for some reason
TensorFlow 2.2 can't be installed with Python 3.8!
Be careful with easy method references
Getting Started with Python for PHPer-Classes
Getting Started with Python Basics of Python
Life game with Python! (Conway's Game of Life)
10 functions of "language with battery" python
4th night of loop with for
Speak Japanese text with OpenJTalk + python
Be careful with Python's append method
Implementation of Dijkstra's algorithm with python
Introductory table of contents for python3
When matplotlib doesn't work with python2.7
Coexistence of Python2 and 3 with CircleCI (1.0)
Getting Started with Python for PHPer-Functions
Record of Python introduction for newcomers
When using MeCab with virtualenv python
Precautions when using six with Python 2.5
[Python] Format when to_csv with pandas
Basic study of OpenCV with Python
Mathematical optimization that can be used for free work with Python + PuLP
How to deal with SSL error when connecting to S3 with boto of Python
Useful for everyday life !? Semi-automation of COSPA's strongest design of experiments with Python
Be careful when retrieving tweets at regular intervals with the Twitter API