Thorough logging strategy in Python

Important points of logging

It is desirable to create a logger object with getLogger and perform logging so as not to pollute the root logging. Also, it's a good idea to devise import as shown in the good example below so that you don't accidentally pollute logging in your code.

#Bad example
import logging
logging.debug('this is bad logging')
#Good example
from logging import getLogger #Import individually so as not to accidentally pollute logging on the way
logger = getLogger(__name__)
logger.debug('this is good logging')

By setting __name__ when creating a logger, the value of __name__ can be displayed in the output.

About log level

The log level can be set for logging, and it is possible to control to which log level is displayed by setLevel. There are five log levels, DEBUG, INFO, WARNING, ERROR, and CRITICAL, which must be specified by the user. These can also be set individually for the logger and each handler described below.

from logging import getLogger, INFO

logger = getLogger(__name__)
logger.setLevel(INFO) #In this case, the log of INFO or higher is output. In other words, all but DEBUG are output

logger.debug('debug log') #In this case, this is not displayed
logger.info('info log')
logger.warning('warning log')
logger.error('error log')
logger.critical('critical log')

Output result


INFO:root:info log
WARNING:root:warning log
ERROR:root:error log
CRITICAL:root:critical log

Below is a detailed table of log levels. The user-defined log level can also be set by arbitrarily setting the numerical value given to the log level. Since the timing of use is arbitrary, it is not always necessary to comply with the table below. (Reference level)

Log level Numerical value Timing to use(Any)
DEBUG 10 If you want to output the detailed information necessary for problem search
INFO 20 When the expected processing is performed
WARNING 30 Unexpected processing or when it is likely to occur
ERROR 40 If the function cannot be performed due to a serious problem
CRITICAL 50 When a serious error occurs that makes the program unexecutable

About the format

Formats can be added to logging, and setFormatter allows you to output various detailed information such as dates, functions, and log level information to the log. Like the log level, this can be set individually for each logger and each handler.

from logging import getLogger, Formatter

logger = getLogger(__name__)
format = Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logger.setFormatter(format) #Apply format to logger

logger.info('this is format')

Output result


2021-01-13 16:40:23 - __main__ - INFO - this is format

Handler settings (StreamHandler, FileHandler)

In order to output log, it is necessary to create a handler. Set StreamHandler for console output and FileHandler for log file output.

from logging import getLogger, StreamHandler, FileHandler, Formatter, INFO, ERROR

logger = getLogger(__name__)
logger.setLevel(INFO) #If logger is set to INFO, the handler will only inherit logs above INFO

#StreamHandler settings
ch = StreamHandler()
ch.setLevel(INFO) #Log level and format can be set for each handler
ch_formatter = Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logger.addHandler(ch) #Add StreamHandler

#FileHandler settings
fh = FileHandler('log/test.log') #Specify the output file path as an argument
fh.setLevel(ERROR) #The handler cannot be set to a log level below logger(In this case, DEBUG is not possible)
fh_formatter = Formatter('%(asctime)s - %(levelname)s - %(filename)s - %(name)s - %(funcName)s - %(message)s')
logger.addHandler(fh) #Add FileHandler

Log settings using config

You can also use logging.config to load individual settings for handlers and formats as described above.

fileConfig (How to set in the config file)

from logging import getLogger
import logging.config

logging.config.fileConfig('logging_settings.ini') #Read settings from file
logger = getLogger('root') # logging_settings.ini root settings are called

logger.info('this is fileConfig from logging_settings.ini')

logging_settings.ini


[loggers] ;Register the name to be referenced by getLogger(root is__main__Obtained automatically at the time)
keys=root

[handlers] ;The following handler_Describe what was set with xxx(Separated by commas for multiple)
keys=newStreamhandler

[formatters] ;Formatter below_Describe what was set with xxx(Separated by commas for multiple)
keys=newFormatter

[logger_root] ;Settings for each logger(logger_<logger key name>To)
level=INFO
handlers=newStreamhandler

[handler_newStreamhandler]
class=Streamhandler
level=DEBUG
formatter=newFormatter
args=(sys.stderr,)

[formatter_newFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s

Output result


2021-01-13 16:40:23 - root - INFO - this is fileConfig from logging_settings.ini

dictConfig (How to set with dictionary type)

from logging import getLogger, DEBUG, INFO
import logging.config

dict_logging_settings = {
    'version': 1, #Magic
#If you want to add a new handler, add it in handlers in the same format as newStreamhandler
    'handlers': {
        'newStreamHandler': {
            'class': 'logging.StreamHandler',
            'formatter': 'newFormatter',
            'level': DEBUG
        }
    },
#If you want to add a new format, add it in formatters in the same format as newFormatter
    'formatters': {
        'newFormatter': {
            'format': '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
        },
    },
#Only root logger is exceptionally set outside loggers
    'root': {
        'handlers': ['newStreamHandler'],
        'level': INFO
    },
#For other loggers you want to add, describe them in loggers in the same way as root above.
    'loggers': {}
}

logging.config.dictConfig(dict_logging_settings) #Read settings from dictionary
logger = getLogger('root') # dict_logging_The root settings in settings are called

logger.info('this is dictConfig from dict_logging_settings')

Output result


2021-01-13 16:40:23 - root - INFO - this is dictConfig from dict_logging_settings

Reference article

Recommended Posts

Thorough logging strategy in Python
Logging properly in Python
Python Logging
Quadtree in Python --2
Python in optimization
CURL in python
Metaprogramming in Python
Python 3.3 in Anaconda
Geocoding in python
SendKeys in Python
Try logging in to qiita with Python
Meta-analysis in Python
Unittest in python
Epoch in Python
Discord in Python
Sudoku in Python
DCI in Python
quicksort in python
nCr in python
Plink in Python
Constant in python
Lifegame in Python.
FizzBuzz in Python
Sqlite in python
StepAIC in Python
Learn the design pattern "Strategy" in Python
N-gram in python
LINE-Bot [0] in Python
Csv in python
Disassemble in Python
Reflection in Python
Constant in python
nCr in Python.
format in python
Scons in Python3
Puyo Puyo in python
python in virtualenv
PPAP in Python
Get Cloud Logging available in Python in 10 minutes
Quad-tree in Python
Reflection in Python
Chemistry in Python
Hashable in python
DirectLiNGAM in Python
LiNGAM in Python
Flatten in python
flatten in python
Logging settings for daily log rotation in python
Sorted list in Python
Clustering text in Python
Daily AtCoder # 2 in Python
Implement Enigma in python
Daily AtCoder # 6 in Python
Daily AtCoder # 18 in Python
Edit fonts in Python
Singleton pattern in Python
File operations in Python
Read DXF in python
Daily AtCoder # 53 in Python
Key input in Python
Use config.ini in Python