[Python: UnicodeDecodeError] One of the error solutions when reading CSV

Introduction

Recently, there have been many opportunities to collect sensor data on an Android device, convert it from JSON format to CSV format using Python, and format the data.

So, I got stuck in a certain error, so I will leave a memo as a memorandum.

Error that occurred

UnicodeDecodeError: '****' codec can't decode byte 0x** in position **:

Looking at this error, I wondered if there was something wrong with the CSV, or if the code below was flawed. But this was not the case.

with open(path, encoding="***")

Applicable code snippet

dir = os.getcwd() + "/" + folder
files = os.listdir(dir)

This code can return the files that exist in a certain folder in list format. I was trying to use this code to handle a large number of JSON and CSV files in list format.

Cause

The cause was that files prefixed with., So-called hidden files, were included in the list. For my environment, ".DS_Store" was included in the list.

at the end

If you get stuck in a similar error (swamp), it is recommended that you make sure that you have not accidentally read a "file that you do not want to handle" such as a hidden file.

Recommended Posts

[Python: UnicodeDecodeError] One of the error solutions when reading CSV
Error due to UnicodeDecodeError when reading CSV file with Python [For beginners]
Change the length of Python csv strings
[Python] Understand the content of error messages
(One of the solutions) when pyenv install on macOS causes BUILD FAILED and Python installation fails.
[Python] Master the reading of csv files. List of main options for pandas.read_csv.
Character encoding when using csv module of python 2.7.3
The story of reading HSPICE data in Python
Example of reading and writing CSV with Python
the zen of Python
[Python] Reading CSV files
When reading a csv file with read_csv of pandas, the first column becomes index
Towards the retirement of Python2
About the ease of Python
Memorandum (Countermeasures against Unicode Decode Error when reading CSV files)
Error when playing with python
Solve the Japanese problem when using the CSV module in Python.
[Tips] Problems and solutions in the development of python + kivy
About the features of Python
The Power of Pandas: Python
[python] Error when installing library ramkan
The story of Python and the story of NaN
Check the behavior when assigning Python
Error when building mac python environment
First Python 3 ~ The beginning of repetition ~
Existence from the viewpoint of Python
pyenv-change the python version of virtualenv
Change the Python version of Homebrew
Python CSV file reading and writing
[Python] Understanding the potential_field_planning of Python Robotics
Review of the basics of Python (FizzBuzz)
Reading and writing CSV with Python
About the basics list of Python basics
Learn the basics of Python ① Beginners
Avoiding the phenomenon that blank lines are inserted when outputting Python CSV
Python Note: When you want to know the attributes of an object
[Python] Calculate the number of digits required when filling in 0s [Note]
How to deal with SSL error when connecting to S3 with boto of Python
Processing when the key input of Python pygame does not go well.
Check the behavior of destructor in Python
The story of an error in PyOCR
[Python3] Understand the basics of Beautiful Soup
Pass the path of the imported python module
Two rules when reading Python (slice notation)
The story of making Python an exe
Learning notes from the beginning of Python 1
Check the existence of the file with python
About the virtual environment of python version 3.7
What is the cause of the following error?
[Python3] Rewrite the code object of the function
I didn't know the basics of Python
The result of installing python in Anaconda
[Python] Accelerates loading of time series CSV
[Python] Try pydash of the Python version of lodash
[python] Checking the memory consumption of variables
The story of manipulating python global variables
[python] [meta] Is the type of python a type?
The basics of running NoxPlayer in Python
Pandas of the beginner, by the beginner, for the beginner [Python]
The Python project template I think of.
In search of the fastest FizzBuzz in Python