Please stop printing and import logging for log output --Qiita
I was reading while thinking, "Well, I see." But at the same time, I thought like this.
** So how do you write logging? ** ** ** Even for free, I'm not good at logging in Python and it takes time, but I can't do logger !! **
So I decided to write this article. The purpose is as follows.
――I want to share my own understanding that I found as a result of desperately facing logger --Don't waste your time worrying about logging when everyone makes apps in Python
Official docs and the view from reading the Qiita article above.
logging has a hierarchical structure.
└── rootLogger
├── scanLogger
│ ├── scanLogger.txt
│ ├── scanLogger.html
│ └── scanLogger.pdf
└── anotherLogger
It has (or is created) multiple child loggers with a logger called rootLogger as the parent.
For example ...
--Module called ʻurllib uses logger called ʻurllib
--ʻApscheduler.scheduler module uses ʻapscheduler.scheduler logger
The logger for each of these packages and apps is called rootLogger.
bad_sample.py
import logging
logging.info("Taking log...")
This is synonymous with playing with rootLogger directly, so let's stop.
It's similar to the concept of playing with an instance instead of playing with a class.
Even if you look at the official document or the sample, you do not know how to write it ... sweat How much fixed wording and how much can I change by myself? (Range that you can name yourself)
That's why I will present the configuration whose operation has been confirmed. If you have a problem, please copy and use it.
As much as possible, I will include a comment in the comments so that you can easily customize it.
** I am writing the logging config file in toml. ** **
You can refer to the toml package with pip install toml.
The notation method in yaml is here.
terminal
$ tree
.
├── app
│ ├── logger
│ │ ├── __init__.py
│ │ └── logger.py
│ ├── logging.config.toml
│ └── main.py
└── log
├── application.log
└── error.log
$ cd app/
$ python app.py # app/I am writing a sample assuming that this command is executed in the directory
Also put the Github sample code.
main.py
#The previous logger is a directory. The logger at the back is a file.
from logger.logger import get_logger
logger = get_logger()
def main():
logger.info("start = main()")
# do something
logger.error("oops! something wrong!!")
# do something
logger.info("end = main()")
if __name__ == "__main__":
main()
logging.config.toml is set to output logs to the console screen and file.
toml:logging.config.toml
version = 1
disable_existing_loggers = false #Do not disable loggers on other modules
[formatters]
[formatters.basic]
#Set the log format. Details of how to write will be described later
format = "%(asctime)s [%(name)s][%(levelname)s] %(message)s (%(filename)s:%(module)s:%(funcName)s:%(lineno)d)"
datefmt = "%Y-%m-%d %H:%M:%S"
[handlers]
[handlers.console]
#Set console output here
class = "logging.StreamHandler" #Fixed wording
level = "DEBUG" #log level is your choice
formatter = "basic" #Select the log format listed in formatters
stream = "ext://sys.stdout" #Fixed wording
[handlers.file]
class = "logging.handlers.TimedRotatingFileHandler" #Fixed wording. Details of TimeRotatingFileHandler will be described later. There are notes
formatter = "basic"
filename = "../log/application.log" #If specified by relative path$Write from a directory that runs python
when = 'D' #log A unit for switching files. D= day。
interval = 1 #Since day is selected above, a new file is created every day.
backupCount = 31 #Since day is selected above, the log file for 31 days is retained.
[handlers.error]
class = "logging.handlers.TimedRotatingFileHandler"
level = "ERROR"
formatter = "basic"
filename = "../log/error.log"
when = 'D'
interval = 1
backupCount = 31
[loggers]
[loggers.app_name] # app_name is the name of the logger called from python. Arbitrarily named
level = "INFO"
handlers = [
"console",
"file",
"error",
] #Set which of the handlers set above to use
propagate = false
[root]
level = "INFO"
handlers = [
"console",
"file",
"error"
] #Setting of logger's parent root Logger. I also want to keep the rootLogger in console and file
logger/__init__.py
from . import logger
logger.init_logger('logging.config.toml', 'app_name')
#First argument: configfile =The name of the config file. Relative path or absolute path of the directory executed by the python command
#Second argument: loggername =The name of the logger set in the config file
logger/logger.py
import os
import toml
from logging import getLogger
from logging.config import dictConfig
CONFIGFILE = "logging.config.toml" #Treat as a global variable. The default value is as shown on the left. Any
LOGGERNAME = "root" #Treat as a global variable. The default value is root. Any
def init_logger(configfile, loggername):
global CONFIGFILE
global LOGGERNAME
CONFIGFILE = configfile
LOGGERNAME = loggername
dictConfig(toml.load(CONFIGFILE))
def get_logger():
return getLogger(LOGGERNAME)
This is more flexible than logging in the Config file.
In the Config file, the parent directory is ../, which makes it more OS-dependent.
If you set it with python dict, you can specify it with ʻos.path.pardir`.
terminal
$ tree
.
├── app
│ ├── logger
│ │ ├── __init__.py
│ │ └── logger.py
│ └── main.py
└── log
├── application.log
└── error.log
$ cd app/
$ python main.py
Also put the Github sample code.
main.py is [main.py above](https://qiita.com/uANDi/items/9a2b980262bc43455f2e#3-1-2-%E3%83%95%E3%82%A1%E3%82 Same as% A4% E3% 83% AB).
Also, the contents of CONFIG are the same as the above logging.config.toml.
logger/__init__.py
from . import logger
logger.init_logger()
logger/logger.py
import os
from logging import getLogger
from logging.config import dictConfig
APP_NAME = os.getenv('APP_NAME', default='app_name')
CONFIG = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'basic': {
'format': '%(asctime)s [%(name)s][%(levelname)s] %(message)s (%(module)s:%(filename)s:%(funcName)s:%(lineno)d)',
'datefmt': '%Y-%m-%d %H:%M:%S'
}
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'level': 'DEBUG',
'formatter': 'basic',
'stream': 'ext://sys.stdout'
},
'file': {
'class': 'logging.handlers.TimedRotatingFileHandler',
'formatter': 'basic',
'filename': os.path.join(os.path.pardir, 'log', 'application.log'),
'when': 'D',
'interval': 1,
'backupCount': 31
},
'error': {
'class': 'logging.handlers.TimedRotatingFileHandler',
'level': 'ERROR',
'formatter': 'basic',
'filename': os.path.join(os.path.pardir, 'log', 'error.log'),
'when': 'D',
'interval': 1,
'backupCount': 31
}
},
'loggers': {
APP_NAME: {
'level': 'INFO',
'handlers': [
'console',
'file',
'error'
],
'propagate': False
}
},
'root': {
'level': 'INFO',
'handlers': [
'console',
'file',
'error'
]
}
}
def init_logger():
dictConfig(CONFIG)
def get_logger():
return getLogger(APP_NAME)
The output format is the same for both Pattern 1. and Pattern 2.
[app_name] is the logger name.
terminal
$ cd app/
$ python main.py
2019-12-21 15:43:28 [app_name][INFO] start = main() (main:main.py:main:8)
2019-12-21 15:43:28 [app_name][ERROR] oops! something wrong!! (main:main.py:main:10)
2019-12-21 15:43:28 [app_name][INFO] end = main() (main:main.py:main:12)
$ cat ../log/application.log
2019-12-21 15:43:28 [app_name][INFO] start = main() (main:main.py:main:8)
2019-12-21 15:43:28 [app_name][ERROR] oops! something wrong!! (main:main.py:main:10)
2019-12-21 15:43:28 [app_name][INFO] end = main() (main:main.py:main:12)
$ cat ../log/error.log
2019-12-21 15:43:28 [app_name][ERROR] oops! something wrong!! (main:main.py:main:10)
3-4-1. TimedRotatingFileHandler
What is set in ʻinterval and when` in Official Document TimedRotatingFileHandler Can you do it? "Is written!
Outputs log to a new file at the set time.
In the above example, 'D', that is, the file is rewritten every day.
For past log files, the date surfix is added to the file name, such as ʻapplication.log.2019-12-21`.
**Caution! ** **
TimedRotatingFileHandler will not work for applications where app.py does not start all day.
It works for web apps like Django and Flask, but may not work properly with cron.
3-4-2. Format
The Official Documentation LogRecord Attribute (https://docs.python.org/en/3/library/logging.html#id2) has a list of each format. Let's arrange the items you need!
I introduced two types of how to make logging! Let's look back on each! (If you think it's perfect, skip it !!)
By the way, Takeaways means "points, points". take away = Take away → Take this away with today's presentation! It is such a nuance.
A number of loggers have been created with rootLogger as the parent. An image like rootLogger → Class, logger → instance.
logging.info () will directly mess with the rootLogger, so let's create your own logger.
Then how do you make it concretely?
--Read logging settings from Config file -Toml sample -Yaml sample --Write the settings in the Python dictionary -Sample
--If you do not set disable_existing_loggers = False, the logger of gunicorn will be broken.
--When creating a Config file
--Note the OS dependency of the relative path
--The relative path of the log folder is from the directory where you run $ python
TimedRotatingFileHandler
--It is meaningless if the program execution time is short for the interval
--For web apps such as Django and FlaskI don't think this is the correct way to write logging. You can also add a handler to your python file with ʻaddHandler`. We would be grateful if you could teach us how to set up your recommended logging.