[LINUX] Summary of logrotate software logrotate

Very major software installed by default in recent Linux. It can be used not only for server log files but also for rotation of web application log files.

file organization

path	role
/etc/logrotate.conf	Configuration file for the entire log rotation
/etc/logrotate.d/*	Configuration file for each service
/var/lib/logrotate.status or /var/lib/logrotate/status or /var/lib/logrotate/logrotate/status or /var/lib/logrotate/logrotate.status	A file that manages the date and time of the last rotation. The path differs depending on the OS and version.

/etc/logrotate.conf contains the default settings. Since /etc/logrotate.d and below are also included, Create a configuration file for each service here.

Specifications that will be overwritten with the settings written later. /etc/logrotate.d is included after the default settings are written in /etc/logrotate.conf.

List of setting items

Settings related to rotation conditions

Directive	option	Option example	Explanation
`hourly`	-	-	Perform rotation every hour. However, if you do not set it to hourly with cron, it will be daily set with cron.
`daily`	-	-	Perform rotation daily.
`weekly`	-	-	Perform weekly rotation.
`monthly`	-	-	Perform rotation every month.
`yearly`	-	-	Perform rotation every year.
`size`	[file size]	`1M`	ログファイルが指定したfile size以上になったらローテーションする。サイズの後ろに `k`Kilobytes,`M`With megabytes,`G`If you add, it is interpreted as gigabyte. Time interval(`daily`, `weekly`, `monthly`, `yearly`) Setting is ignored`logrotate`Rotate only log files that are larger than the specified size when the command is executed.
`maxsize`	[file size]	`1M`	ログファイルが指定したfile size以上になったらローテーションする。時間間隔（`daily`, `weekly`, `monthly`, `yearly`）も設定されている場合はfile size以上になったときも時間が経過したときも両方ローテーションされる。ただし、`logrotate`If you haven't started it on an hourly basis, it's set to cron daily by default, so daily is the shortest.`logrotate`Version 3.Features implemented from 10.
`minsize`	[file size]	`1M`	ログファイルが指定したfile size以上になったらローテーションする。時間間隔（`daily`, `weekly`, `monthly`, `yearly`If) is also set, it will not be rotated until both conditions are met.
`ifempty`	-	-	Rotate even if the log file is empty.
`notifempty`	-	-	Do not rotate if the log file is empty. When this setting is made`ifempty`Override the settings of.

Settings related to log file generation

Directive	option	Option example	Explanation
`create`	[Permissions username group name]	`0644 nginx nginx`	Create a new empty log file after rotation. It is used when the program that outputs the log file cannot generate the file. You can specify file permissions, user name, and group name. For attributes that are not specified, the attributes of the original file are inherited.`postrotate`Executed before the script is executed.
`nocreate`	-	-	Do not create a new empty log file. When this setting is made`create`Override the settings of.
`copytruncate`	-	-	Copy the log file and empty the original file. In the case of a program that cannot close the log file, if the default is used, the log will continue to be written to the archive that should have been rotated. Use when you want to rotate with the log file open. * There is a slight time lag between copying and emptying, and there is a risk that the log will be lost during that time.
`nocopytruncate`	-	-	Do not empty the original file after copying the log file. When this setting is made`copytruncate`Override the settings of.
`copy`	-	-	Copy the log file, but leave the original file as is. It is used to take a snapshot of the log file, when other utilities require file truncation, or when parsing the file. When this setting is made`create`Override the settings of.
`nocopy`	-	-	Do not copy log files. When this setting is made`copy`Override the settings of.

Settings related to the file name of the log file

Directive	option	Option example	Explanation
`dateext`	-	-	8 digits date instead of numbering the suffixes of the rotated logs(-YYYYMMDD)Turn on.
`dateformat`	Format string	`_%Y-%m-%d`	Specify the date format.`%Y,%m,%d,%H,%s`Can be used. * 3.9.In versions prior to 0`%H:%M:%S`Cannot be used.
`dateyesterday`	-	-	Use yesterday's date instead of today for the date added by dateext. As a result, the time stamp of the rotated file matches the date of the file name.
`extension`	extension	`.log`	The specified extension is maintained after log rotation. If you use compression, the extension is followed by the compression extension. For example`mylog.log`I have a log file named`mylog.log.1.gz`not,`mylog.1.log.gz`You can rotate it by adding an extension after the number like.
`start`	Numbers	`5`	The beginning of the number to be added to the end of the rotation file. The default is`1`.. For example`5`If you specify`a.log` => `a.log.5` => `a.log.6` =>... and the log file name`5`start from.
`tabooext`	[+] ext[,ext,…]	`+ .test`	Files with the extension specified by this option will not be processed. The default is`.rpmsave, .rpmorig, ~, .disabled, .dpkg-old, .dpkg-dist, .dpkg-new, .cfsaved, .ucf-old, .ucf-dist, .ucf-new, .rpmnew, .swp, .cfsaved, .rhn-cfg-tmp-*`Is excluded.`+`If you write, the specified extension will be added to the existing extension list. If not, replace the existing list.

Log file directory settings

Directive	option	Option example	Explanation
`olddir`	Directory name	`/var/backup/log`	Moved to the directory specified during rotation. The destination must be on the same device as the original. Both absolute path and relative path can be specified. When this setting is made`noolddir`Override the settings of.
`noolddir`	-	-	Rotate the logs in the same directory. ※Default
`createolddir`	[Permissions username group name]	`0644 nginx nginx`	`olddir`If the directory specified in does not exist, a new directory will be created.
`nocreateolddir`	-	-	`olddir`If the directory specified in does not exist, the directory will not be created.

Settings related to log compression

Directive	option	Option example	Explanation
`compress`	-	-	Compress the rotated log with gzip.
`compresscmd`	Compression command	`/usr/bin/bzip2`	Specify the command to be used for compression when you want to compress with something other than gzip.`compresscmd`If you change the compression program with`compressext`If you do not change the value of, there is a high possibility that an error will occur.
`compressext`	extension	`.bz2`	Specifies the extension of the compressed log file. By default, it follows the set compression command.
`compressoptions`	Compression options	`--best`	Specifies command line options to pass to the compression program. With the default gzip`-6`(Compression rate priority) is specified.
`nocompress`	-	-	Do not compress rotated logs.
`delaycompress`	-	-	The first generation log is not compressed. In the case of a program that cannot close the log file, if the default is used, the log will continue to be written to the archive that should have been rotated, but if it is compressed, it cannot be written, so it is delayed assuming that case.`compress`Is ignored if is not specified.
`delaycompress`	-	-	Logs are compressed from the first generation.`compress`Is ignored if is not specified.

Settings related to the number of log file rotations and deletion

Directive	option	Option example	Explanation
`rotate`	Number of times	`7`	Reserves the specified number of logs and deletes older log files.
`maxage`	Days	`30`	Reserves logs for the specified number of days and deletes older log files.
`shred`	-	-	`unlink()`Instead of,`shred -u`Delete the log file using. *`shred`Fills the file to be deleted with random numbers so that the data cannot be recovered, and then deletes it.
`shredcycles`	Number of times	`5`	How many times to overwrite the log file before deleting`shred`Instruct. If this setting is omitted`shred`The default value of is adopted.
`noshred`	-	-	When deleting old log files`shred`Do not use. * Default setting

Settings related to email during log rotation

Directive	option	Option example	Explanation
`mail`	mail address	`[email protected]`	If the log exceeds the existence period due to rotation, it will be sent to the specified email address.
`mailfirst`	-	-	`mail`If you have set a directive, the file that has just been rotated is sent instead of the file that expires.
`maillast`	-	-	`mail`If you have set a directive, the expired file will be sent instead of the file that was just rotated. *`mail`Default settings when using directives
`nomail`	-	-	Do not email log files.

Settings related to errors during log rotation

Directive	Explanation
`missingok`	Continues processing without issuing an error even if the log file does not exist.
`nomissingok`	Gives an error if the log file does not exist. * Default setting

Script settings to be executed during log rotation

Directive	Explanation
`prerotate/endscript`	Execute the described command before log rotation.`sharedscripts`If is specified, the absolute path to all files is passed to the script.`sharedscripts`If is not specified, the command will be executed for each target log file.
`postrotate/endscript`	Execute the described command after log rotation.`sharedscripts`If is specified, the absolute path to all files is passed to the script.`sharedscripts`If is not specified, the command will be executed for each target log file.
`firstaction/endscript`	Execute the described command at the beginning of log rotation. If the log file path is a wildcard, the absolute path of the wildcard is passed to the script.
`lastaction/endscript`	Execute the described command at the end of log rotation. If the log file path is a wildcard, the absolute path of the wildcard is passed to the script.
`preremove/endscript`	Executed before deleting the log file. The name of the file to be deleted after execution is passed to the script.
`sharedscripts`	By default`prerotate`When`postrotate`The script is run for each rotated log file and the absolute path to that log file is passed as the first argument to the script.`sharedscripts`If is specified, the script will only be executed once, no matter how many files match the wildcard specification.
`nosharedscripts`	`prerotate`When`postrotate`Execute the script for each rotated log file. The absolute path to that log file is passed as the first argument to the script.

--/ Bin/sh is used as the command to be executed. --The script will not be executed if none of them require rotation.

Other settings

Directive	option	Option example	Explanation
`su`	[Username Group name]	`nginx nginx`	Rotate the log file with the specified users and groups. The default is root.`logrotate`Version 3.8.Features introduced from 0.
`include`	File or directory	-	The file given as an argument,`include`Read the directive at the location. If a directory is specified, most files in the directory (directories and named pipes,`tabooext`Files ending with the name specified in the directive are excluded) are read in alphabetical order.

Log rotation execution order

Execute first action/end script
Rotation of rotated files (eg log.1.gz-> log.2.gz)
Execute prerotate/endscript
Rotation of target file (example: log-> log.1)
Execute postrotate/endscript
File compression (eg log.1-> log.1.gz)
Execute preremove/endscript *
Delete the compression source file (example: delete log.1) *
Execute preremove/endscript *
Delete expired files (example: delete log.3.gz) *
Execute lastaction/endscript

Preremove is repeatedly executed for the number of files to be rotated.

Setting Example

apache configuration example

/var/log/httpd/*log {   #The log file path is specified here. You can use wildcards.
    missingok           #Processing continues even if the specified log file does not exist.
    notifempty          #Skip if the log file is empty.
    sharedscripts       #Execute the command described in postrotate or prerotate only once for multiple specified log files.
    delaycompress       #The first generation log is not compressed.
    postrotate          #Execute the described command after log rotation. Here, apache is forced to reload the settings.
        /sbin/service httpd reload > /dev/null 2>/dev/null || true
    endscript
}

Syslog configuration example

#Set up 4 log files at once
/var/log/cron
/var/log/messages
/var/log/secure
/var/log/spooler
{
    missingok           #Processing continues even if the specified log file does not exist.
    sharedscripts       #Execute the command described in postrotate or prerotate only once for multiple specified log files.
    postrotate          #Execute the described command after log rotation. If you move a file while syslogd is running, syslogd will lose track of the log file it should output, so it sends a HUP signal to syslogd to reload the settings.
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

WEB application log rotation setting example

#Log file path
/var/www/html/application/logs/log.php {
    compress             #Compress rotated logs with gzip
    daily                #Run daily
    dateext              #8 digits date instead of numbering the suffixes of the rotated logs(-YYYYMMDD)Turn on.
    dateformat -%Y-%m-%d #Specify the date format.
    delaycompress        #The first generation log is not compressed.
    extension .php       #Match the extension to the log.
    missingok            #Processing continues even if the specified log file does not exist.
    rotate 60            #Hold for 60 days
    su www-data www-data #Change users and groups to suit your environment.
    create 666 www-data www-data
    lastaction
        #Grant read and write permissions to the owner
        chmod u+rw /var/www/html/application/logs/*.php
    endscript
}

command

Format

logrotate [-dv] [-f|--force] [-s|--state status file]setting file..

option

option	Explanation
`-?` or `–help`	Display a help message.
`-d` or `–debug`	Turn on debug mode and-Implicitly specify v. Logs and logrotate status files are unchanged in debug mode.
`-f` or `–force`	Forcibly execute log rotation.
`-m command` or `–Mail command`	Specify the command to use when sending the log by e-mail.
`-s status file` or `–State status file`	Instruct to use an alternative status file.
`–usage`	Show concise usage.
`+-v` or `–verbose`	Turn on detailed display mode.

Command example

A command to check the operation in debug mode after setting log rotation.

Actually, log rotation is not performed.

# logrotate -d /etc/logrotate.conf

A command to check the operation in debug mode when a log file that does not meet the conditions is forcibly rotated.

Actually, log rotation is not performed.

# logrotate -fd /etc/logrotate.conf

A command to actually execute and check.

If the conditions are met, log rotation will actually be performed.

# logrotate -v /etc/logrotate.conf

A command for forcibly rotating log files that do not meet the conditions.

# logrotate -f /etc/logrotate.conf

Check the date of the rotated log file.

Note that the status file path differs depending on the OS and version (#ファイル構成).

# cat /var/lib/logrotate.status

important point

About execution authority

Since logrotate is run in cron, it will run with root privileges unless you change the user in the cron configuration and the logrotate config file.

If the parent directory of the log file is o + w or the group is other than root and is g + w, it will not rotate.

Also, if you execute a script with preremove/endscript or postremove/endscript, it will be executed with root privileges, so if it is an inappropriate script, unexpected files will be deleted, or in the worst case, it will be deleted. The server may be damaged.

In this case, change the authority with the su option or check the operation in the verification environment.

About wildcards

When the log file is specified by a wildcard, if the rotated log is included, the log file will increase in a mouse-like manner.

Wildcards should have an extension like ~/*. Log instead of~/*so that logs and archives can be distinguished.

About copytruncate

The copytruncate option has a slight time lag between copying and emptying, and there is a risk of losing logs during that time.

It is best not to use it unless you have a specific reason.

About date format

If the log file name is in date format, logrotate will sort by file name to determine which file is the oldest during rotation, so the date format will be % d-% m-% Y. If you do, you may delete new logs.

Make sure to order by date, such as -% Y-% m-% d. Well, if you're Japanese, no one would reverse it.

About hourly

logrotate is set to run in cron.daily by default. If it is left as it is, even if hourly is specified, it will be rotated daily.

If you want to rotate with hourly, make sure that logrotate is executed with cron.hourly.

Moved logrotate cron settings from daily to hourly.

# mv /etc/cron.daily/logrotate /etc/cron.hourly/

reference

https://hackers-high.com/linux/man-jp-logrotate/#nocopy https://isleofhoso.com/linux-logrotate-size/ https://www2.filewo.net/wordpress/2013/03/31/logrotate%E3%82%B9%E3%82%AF%E3%83%AA%E3%83%97%E3%83%88%E3%81%AE%E8%AA%BF%E6%9F%BB/ https://qiita.com/hdtkkj/items/85d8d3bdfc45ffeaba89 https://mogu2itachi.hatenablog.com/entry/2020/01/26/145929 http://1000k.github.io/2010/05/20/usage-of-logrotate/