[python3] Implement debug log output function easily with logging and Click

When writing code while looking at the debug log, it is often troublesome to change the output ON / OFF. So, here's a trick to change the log level from the command line.

First of all, normal log settings

Code example

First, I will attach a code example for using the logging module normally and outputting the log to the console. Below, I am using python3.7.

logging_test.py


import logging

from logging import DEBUG, INFO


def set_logger():
    logger = logging.getLogger('logname')
    stream = logging.StreamHandler()
    logger.addHandler(stream)

    logger.setLevel(INFO)

    return logger


def main():
    logger = set_logger()

    logger.debug('debug')
    logger.info('info')


if __name__ == "__main__":
    main()

I will explain briefly.

--The log is set with set_logger. The return value logger is the instance used for log output in the program. --stream determines the output destination of the log. This time it's console output, so I'm using logging.StreamHandler (). Use logging.FileHandler () to output to a file. --Add the output destination specified by stream to the logger instance withlogger.addHandler (stream). --Set the log level with logger.setLevel (INFO). This time, it is set to ʻINFO (information)level. --And output the log withlogger.debug ('debug')andlogger.info ('info')`.

Execution result

Here is the result of executing the above code.

>python logging_test.py
info

>

The INFO level log is output, but the DEBUG level log is not output. To output DEBUG level logs, the log level setting in the script must be logger.setLevel (DEBUG).

Subject: Change log level using Click

There is a python command line parser called Click. This makes it easy to implement the process of passing command line arguments when running a python file. Speaking of CLI creation libraries, Python Fire is very easy and convenient, but I personally recommend Click, which allows you to explicitly specify variables that can be passed to the CLI.

Code example

Use Click to tweak the code a bit.

logging_test.py


import click
import logging

from logging import DEBUG, INFO


def set_logger(debug_mode):
    logger = logging.getLogger('logname')
    stream = logging.StreamHandler()
    logger.addHandler(stream)

    if debug_mode is True:
        logger.setLevel(DEBUG)
    else:
        logger.setLevel(INFO)

    return logger


@click.command()
@click.option('--debug_mode', '-d', is_flag=True, 
              help='Show debug log')
def main(debug_mode):
    logger = set_logger(debug_mode)

    logger.debug('debug')
    logger.info('info')


if __name__ == "__main__":
    main()

The big change is that the log level settings and the main function have decorators (@ click. ~ ). Let's talk about this.

--First, the decorator @ click.option ('--debug_mode','-d', is_flag = True, help ='Show debug log') given to the main function sets the variable debug_mode. This is the part of the setting to be passed to the main function as a command line argument. The arguments are, in order from the left, the explanation when the long option, short option, flag (True / False), and help option are specified. If you add --debug_mode or -d at run time, the debug_mode variable will be True. --Next, the debug_mode variable received as a command line argument is passed to the set_logger function. If the -d option is specified, the log level will be DEBUG, and if not specified, it will be INFO.

Execution result

Here is a comparison of the execution results when executed without specifying -d and when executed with it specified.

>python logging_test.py
info

>python logging_test.py -d
debug
info

>

If you specify -d, even the DEBUG level log is output. By doing this, you can set the debug log output settings without changing the source code log level each time.

reference

Finally, here are the sites I usually refer to regarding logging and Click.

-[Introduction to Python] Let's keep a record of processing with the logging module! -Python: Click on the command line parser was too convenient

Recommended Posts

[python3] Implement debug log output function easily with logging and Click
Easily implement subcommands with python click
Output log in JSON format with Python standard logging
Unit test log output with python
Output Python log to console with GAE
Easily download mp3 / mp4 with python and youtube-dl!
Easily implement login authentication function with Laravel
Read JSON with Python and output as CSV
Python log is not output with docker-compose up
I tried function synthesis and curry with python
Output python log to both console and file
Create a Python console application easily with Click
Procedure to load MNIST with python and output to png
python input and output
Crawling with Python and Twitter API 1-Simple search function
Read json file with Python, format it, and output json
Output debug log with GAE dev_appserver.py on Eclipse + PyDev
Try hitting the Twitter API quickly and easily with Python
Python logging standard library for file output by log level
Easily write JSON and Python dataclass conversions with quicktype and dacite
Associate Python Enum with a function and make it Callable
[GCP] How to output Cloud Functions log to Cloud Logging (Stackdriver Logging) (Python)
Easily build network infrastructure and EC2 with AWS CDK Python
How to log in to AtCoder with Python and submit automatically
Programming with Python and Tkinter
Encryption and decryption with Python
Implement login function with django-allauth
Easily serverless with Python with chalice
Python and hardware-Using RS232C with Python-
Try Python output with Haxe 3.2
Output large log with discord.py
Debug Python with VS Code
Debug with PEPPER python interpreter
Python debug and test module
Works with Python and R
Extract bigquery dataset and table list with python and output as CSV
Crawling with Python and Twitter API 2-Implementation of user search function
Communicate with FX-5204PS with Python and PyUSB
Shining life with Python and OpenCV
Implement R's power.prop.test function in python
Install Python 2.7.9 and Python 3.4.x with pip.
Neural network with OpenCV 3 and Python 3
AM modulation and demodulation with python
Make apache log csv with python
[Python] font family and font with matplotlib
Scraping with Node, Ruby and Python
Easily handle lists with python + sqlite3
Output to csv file with Python
Scraping with Python and Beautiful Soup
Input / output with Python (Python learning memo ⑤)
Debug python multiprocess program with VSCode
[Code] Module and Python version output
JSON encoding and decoding with python
Hadoop introduction and MapReduce with Python
[GUI with Python] PyQt5-Drag and drop-
[Note] Hello world output with python
Reading and writing NetCDF with Python
Easily handle databases with Python (SQLite3)
I played with PyQt5 and Python3
Function synthesis and application in Python
Export and output files in Python