Unmanaged log files will eventually fill your disk. Applications write continuously, and standard system utilities need a predictable way to archive old entries without interrupting running processes. The standard tool for this task is logrotate, but its default configurations often hide subtle performance traps that bite production systems under heavy load.
You configure logrotate through flat text files typically placed inside the /etc/logrotate.d/ directory. A clean logrotate configuration file defines how often a file rotates, how many historical copies to keep, and how to handle compression. Here's a production-ready example for a custom application log:
/var/log/myapp/*.log {
daily
rotate 14
compress
delaycompress
missingok
notifempty
create 0640 www-data www-data
sharedscripts
postrotate
systemctl reload myapp
endscript
}Let's break down what each directive actually does. The daily option tells the daemon to check the log file every day. The rotate 14 directive ensures you keep two weeks of history before dropping the oldest logs. Using compress pairs well with delaycompress, which leaves the most recently rotated file uncompressed for twenty-four hours. This saves CPU cycles if your debugging tools still need to grep through yesterday's entries immediately.
The missingok flag prevents errors if a log file is absent during a rotation cycle, and notifempty skips rotation entirely if the log file size is zero bytes. Permissions matter immensely for security. The create 0640 www-data www-data line generates a fresh log file with strict read permissions immediately after moving the old one aside.
The Mechanics of Log Rotation and Process Signals
When logrotate runs, it renames the current log file, appends a number or date extension, and then creates a brand-new file with the original name. The critical gotcha here lies in how file descriptors work in Linux. When a long-running process keeps a log file open, its file descriptor points to the inode of that specific file. Simply renaming the file on disk doesn't change where the process writes its bytes. The application process will continue writing to the renamed file until it receives a signal to close and reopen its log handles.
This is why the postrotate script block is essential in the configuration shown earlier. By running systemctl reload myapp, you'll instruct the application to drop its old file descriptors and open the new log file created by logrotate. If you skip this step, your application will keep writing into the archived, compressed log file on disk until the service eventually restarts.
For developers running complex python services or managing background queues, understanding how system resources and file handles interact is just as important as handling asynchronous tasks, similar to the concepts covered when reviewing Async Python: asyncio Without the Confusion. If your application handles thousands of concurrent requests, dropping file descriptors abruptly without proper synchronization can lead to dropped log entries.
Understanding the Copytruncate Trap
Sometimes you can't signal an application to reopen its log files. Third-party daemons or legacy software might lack a reload signal handler entirely. Developers often reach for the copytruncate directive to solve this constraint, but it introduces a dangerous race condition.
The copytruncate strategy works by copying the contents of the active log file to the destination backup file, and then truncating the original log file down to zero bytes in place. Because the file is never renamed or closed, the application process keeps its file descriptor intact and continues writing without interruption.
The hidden danger is that copytruncate isn't atomic. Between the moment logrotate finishes reading the file contents and the moment it executes the truncation command, your application can write new log lines. Those intermediate log lines get completely lost during the truncation step because the truncate operation wipes the file from byte zero. If your application logs high-volume audit trails or financial transactions, using copytruncate will cause silent data loss.
If you must manage applications that refuse to close log files gracefully, consider managing their environments carefully. For isolating runtime dependencies and keeping your deployment footprint clean, standard containerization practices or environment management tools like those discussed in Python venv vs uv vs Poetry: Choosing for Production offer much tighter control over process lifecycles.
Testing Your Configuration
Never push a new logrotate configuration to production without testing it manually. The logrotate binary includes a verbose debug flag that simulates the entire rotation process without altering your actual log files or invoking post-rotation scripts.
Run the test command with your specific configuration file path:
logrotate -d /etc/logrotate.d/myappExamine the output carefully. The debug command prints every step the utility intends to take, showing whether files match, when rotation triggers, and what scripts execute. If you need to force an immediate rotation on a live staging server to verify permissions and compression behavior, omit the debug flag and run the force flag instead:
logrotate -f /etc/logrotate.d/myappCheck the system status file located at /var/lib/logrotate/status afterward. This file records the exact timestamp of the last rotation for every registered log target. If logrotate refuses to rotate your files, inspect this status file first. Stale entries often block unexpected rotation attempts until the scheduled interval finally rolls over.
