Log files are essential when you need to understand what is happening on a server. Nginx records requests, applications record errors, background workers record activity, and many services maintain their own logs.
The problem is that logs rarely stop growing on their own.
Leave a busy application or web server running for long enough and a log file can grow from a few megabytes to several gigabytes. Apart from wasting disk space, extremely large log files become awkward to search, copy, archive, and inspect when something goes wrong.
Logrotate is a standard Linux utility designed to prevent that from happening. It automatically rotates, compresses, retains, and eventually removes old log files according to rules you define.
Logrotate manages the lifecycle of log files.
Instead of allowing a file such as:
/var/log/myapp/app.log
to grow indefinitely, Logrotate can periodically rename the existing file and allow a new one to take its place.
After several rotations, you might have:
app.log
app.log.1
app.log.2.gz
app.log.3.gz
app.log.4.gz
Here, app.log is the current log, while the numbered files contain older entries. Older logs can be compressed automatically to reduce the amount of disk space they consume.
You can configure Logrotate to:
One important detail is that Logrotate does not continuously monitor your log files. It is run periodically by the operating system and checks your rules each time it runs.
For example, if Logrotate only runs once per day, a maxsize 100M rule does not magically rotate a file the moment it reaches 100 MB. The size is checked the next time Logrotate runs.
The most obvious reason to rotate logs is disk space.
A badly behaved application can produce an enormous amount of logging in a surprisingly short period of time. If /var or your root filesystem runs out of space, the consequences can extend well beyond the application responsible for the logs.
Keeping a sensible number of compressed historical logs reduces this risk.
Logrotate should not be your only protection against a full disk, however. Disk usage monitoring and alerts are still important.
Searching a 50 MB log file is considerably easier than working with a 20 GB one.
Smaller logs are easier to inspect with tools such as:
tail
less
grep
zgrep
For example, you can search a compressed historical log without manually decompressing it first:
zgrep "ERROR" /var/log/myapp/app.log.3.gz
Without Logrotate, you would need to periodically rename, compress, and delete old logs yourself.
Logrotate lets you define the policy once and have the server apply it automatically.
Rotation does not have to mean immediately deleting old data.
You might decide to keep seven daily logs, four weekly logs, or several months of logs depending on your application and available storage.
This gives you historical information when investigating a problem without retaining everything indefinitely.
Logrotate is reliable and widely used, but there are a few important details to understand.
This is one of the most common problems when creating your own Logrotate configuration.
When a log file is renamed, a running application may still have the original file open. Renaming the file does not automatically make that application start writing to the newly created file.
As a result, the application can continue writing to the rotated log.
Services that support traditional file logging usually provide a way to reopen their log files. A Logrotate configuration can run that command after rotation using a postrotate block.
We will look at this later.
copytruncate is useful, but not idealYou will sometimes see configurations containing:
copytruncate
Instead of renaming the active log file, Logrotate copies its contents into the rotated file and then truncates the original file back to zero bytes.
The application therefore keeps writing to the same file.
This is useful for applications that cannot reopen their logs, but there is a small window between copying and truncating the file where new log entries can be lost.
Use the application's proper log reopening mechanism where one exists. Treat copytruncate as a fallback rather than automatically adding it to every configuration.
Logrotate manages files. It does not analyse their contents or alert you when something goes wrong.
If you want to know when an application starts producing errors, Logrotate is not the tool for that job.
Likewise, it does not provide centralised logging across multiple servers.
Logrotate can help enforce a retention policy, but simply keeping logs for a certain number of days does not necessarily satisfy legal, security, or regulatory requirements.
Some environments require centralised storage, access controls, tamper protection, backups, audit trails, or much longer retention periods.
Think of Logrotate as local log file management rather than a complete logging or compliance platform.
Logrotate is commonly already installed on Ubuntu systems.
You can check by running:
logrotate --version
If it is not installed, install it using APT:
sudo apt update
sudo apt install logrotate
You can then confirm the installation:
logrotate --version
Installing Logrotate gives you the utility itself, but something still needs to execute it periodically.
On Ubuntu systems using systemd, this is normally handled by a systemd timer.
You can inspect it with:
systemctl status logrotate.timer
You can also see when it is next scheduled to run:
systemctl list-timers logrotate.timer
On older installations, you may instead find Logrotate being executed through cron, usually via:
/etc/cron.daily/logrotate
Whichever mechanism your system uses, the important point is that the scheduler starts Logrotate and Logrotate then decides which files need rotating.
There are two places you will commonly work with when configuring Logrotate:
/etc/logrotate.conf
/etc/logrotate.d/
The first is the main configuration file.
The second is a directory containing configurations for individual applications and services.
You can inspect the main configuration with:
cat /etc/logrotate.conf
You will usually find that it includes configuration files from /etc/logrotate.d.
That allows packages and applications to maintain their own rotation rules without everything being placed in one enormous configuration file.
For example:
/etc/logrotate.d/nginx
/etc/logrotate.d/rsyslog
The exact files present depend on what you have installed.
Before creating a configuration, it helps to understand the directives you are most likely to use.
daily, weekly, and monthlyThese control how frequently a log is eligible for rotation:
daily
or:
weekly
or:
monthly
Remember that Logrotate must actually be executed at least this frequently for the rule to have the intended effect.
rotateThis controls how many old rotations are retained:
rotate 7
This tells Logrotate to keep seven rotated logs before older ones are removed.
compressCompress old logs:
compress
This significantly reduces the amount of disk space historical text logs require.
delaycompressWhen used alongside compress:
delaycompress
the most recently rotated log is left uncompressed until the next rotation.
This can be helpful for software that may briefly continue writing to the previous file.
missingokNormally, Logrotate can complain if a configured log file does not exist.
Adding:
missingok
tells it to continue without treating the missing file as an error.
notifemptyAvoid rotating an empty log:
notifempty
There is usually little value in creating archives of empty files.
createAfter moving the existing log, Logrotate can immediately create a replacement:
create 0640 www-data www-data
This creates the new log with mode 0640, owned by the www-data user and www-data group.
The correct user and group depend on the application writing the log. Do not blindly copy www-data into every configuration.
sizeYou can rotate solely according to file size:
size 100M
The log becomes eligible for rotation when it is larger than 100 MB when Logrotate checks it.
maxsizeIf you want normal time-based rotation but also want to rotate particularly large logs sooner, you can use maxsize.
For example:
daily
maxsize 100M
This keeps the daily policy while also allowing an oversized log to be rotated earlier when Logrotate runs.
Again, this is only checked when Logrotate is actually executed.
Suppose an application writes to:
/var/log/myapp/app.log
You could create:
sudo nano /etc/logrotate.d/myapp
with the following configuration:
/var/log/myapp/app.log {
daily
rotate 14
compress
delaycompress
missingok
notifempty
create 0640 www-data www-data
}
Let's break that down.
daily
makes the log eligible for rotation each day.
rotate 14
keeps 14 old rotations.
compress
compresses historical logs.
delaycompress
leaves the newest rotated log uncompressed until the following rotation.
missingok
prevents a missing log from generating an error.
notifempty
prevents empty files from being unnecessarily rotated.
Finally:
create 0640 www-data www-data
creates a fresh log file with the specified permissions and ownership.
You should change the user and group to match whichever account your application uses to write its logs.
You can apply the same rules to multiple logs.
For example:
/var/log/myapp/*.log {
daily
rotate 14
compress
delaycompress
missingok
notifempty
create 0640 www-data www-data
}
Be careful with broad wildcards. Your pattern should match active log files without accidentally matching files that Logrotate has already rotated.
If your application keeps a log file open, simply rotating the file may not be enough.
You may need to tell the application to reopen its logs after rotation.
Logrotate supports this with postrotate:
/var/log/myapp/app.log {
daily
rotate 14
compress
delaycompress
missingok
notifempty
create 0640 myapp myapp
postrotate
systemctl reload myapp >/dev/null 2>&1 || true
endscript
}
The exact command depends entirely on the application.
Some applications support a reload, some accept a particular Unix signal, and others have a dedicated command for reopening logs.
Check the documentation for the service before adding a postrotate command. Restarting an application unnecessarily just to rotate a log is usually best avoided.
copytruncateIf an application cannot reopen its log file, you may have to use:
copytruncate
For example:
/var/log/myapp/app.log {
daily
rotate 14
compress
missingok
notifempty
copytruncate
}
Logrotate copies the current contents into the rotated file and then empties the original file without replacing it.
Because the original file remains in place, the application can continue writing to it.
The downside is that a small amount of logging can theoretically be lost between the copy and truncate operations.
Use copytruncate when necessary, not simply because it appears convenient.
Do not create a configuration and assume it works.
Logrotate has a debug mode that lets you see what it intends to do without actually rotating the files.
To test the complete configuration:
sudo logrotate --debug /etc/logrotate.conf
You can also test an individual configuration:
sudo logrotate --debug /etc/logrotate.d/myapp
Debug output will show how Logrotate interprets the configuration and whether a file is currently eligible for rotation.
This is particularly useful for catching syntax errors or incorrect paths before they affect your logs.
If you want to perform an actual rotation while testing, you can force one:
sudo logrotate --force /etc/logrotate.d/myapp
Be aware that this really does rotate the log, regardless of whether it would normally be due.
Afterwards, inspect the directory:
ls -lh /var/log/myapp/
Check that:
The fourth point is particularly important.
A configuration can appear to rotate successfully while the application continues writing to the old file.
Logrotate keeps track of when logs were last rotated using a state file.
On Ubuntu, this is normally:
/var/lib/logrotate/status
You can inspect it with:
sudo cat /var/lib/logrotate/status
This can be useful when you are wondering why Logrotate says a particular file does not need rotating yet.
If a log is not rotating as expected, start with:
sudo logrotate --debug /etc/logrotate.conf
Then check a few common causes.
A daily, weekly, or monthly rule does not mean Logrotate will rotate the file every time you run it.
It remembers when the previous rotation happened.
Confirm that the configured log actually exists:
ls -lh /var/log/myapp/app.log
Check the directory, existing log and newly created log:
ls -ld /var/log/myapp
ls -l /var/log/myapp/
The application must have permission to write to the new file.
Use:
lsof
to investigate which files a process still has open.
If the application continues writing to an old rotated file, it probably needs to be told to reopen its logs using the appropriate postrotate action.
Check the timer:
systemctl status logrotate.timer
and inspect its recent activity:
journalctl -u logrotate.service
If you are on a system using cron instead, check the appropriate cron configuration and logs.
Not every log on a modern Linux server is a traditional file managed by Logrotate.
Services that write to the systemd journal are managed by systemd-journald, which has its own storage and retention settings.
You therefore should not assume every log visible through:
journalctl
needs a Logrotate configuration.
Logrotate remains particularly important for applications and services that write conventional files such as:
/var/log/example.log
There is no universal retention period.
For a small application server, keeping 7 to 30 days of local logs may be perfectly reasonable. A busy production service may generate enough data that retaining the same period locally would consume too much storage.
Consider:
Do not choose a retention period simply because an example configuration uses rotate 7.
Your logging policy should reflect how the server is actually used.
Logrotate solves a simple but important problem: log files should not be allowed to grow forever.
For many Linux servers, a small Logrotate configuration is enough to automatically rotate old logs, compress them, keep a useful amount of history, and remove files that are no longer needed.
The configuration itself is usually straightforward. The part that requires more care is understanding how the application writing the log behaves when that file is rotated.
Whenever you create a custom Logrotate configuration:
copytruncate only when a better reopening mechanism is unavailable.--debug.A few minutes spent checking those details can prevent both runaway disk usage and the much worse problem of discovering that the logs you thought you were keeping have not been written correctly.
ServerAuth provides a whole host of management tools, from controlling who can access your server, to managing your website deployments. And with an ever-growing suite of tools you'll always be one step ahead!
Start for free
