In the log file of the Web application that has not been tested in detail, some lines may be garbled (such as the error message output by the application).
If you try to read the log file with Python while trying to get statistical information from such a log file, as expected, an exception "Unable to decode" will occur on the garbled line.
Also, when suddenly an unknown system log is passed and an analysis is requested, an error "Unable to decode" may occur when reading with Pyhon even though it looks like a normal line at first glance. In this case, you can carefully investigate the character code of the log file, but if the line is not related to analysis, you may not have time to deal with the error that occurs.
Therefore, when opening the file, if the parameter "` ʻerrors ='replace'``" is specified in the argument, the garbled part (the part where decoding failed) is replaced with "?" And read. Will give you.
f = open('foo.log', mode='r', errors='replace')
I want to aggregate statistical information from the log, but please try it when you do not have time to write the processing of the decoding error.
Have a good Python life! !!
Recommended Posts