Traefik is an excellent reverse proxy for Linux servers, even supporting Docker. When hosting public websites, tools like CrowdSec help to secure the infrastructure from malicious attackers. However, if care is not taken, logs can often consume a significant portion of the hard disk. We will present a solution to this issue.
The Problem
Traefik has two log files: The traefik.log
and the access.log
file. When writing everything in those files by choosing a log level like TRACE
, DEBUG
or INFO
, those files can get increasingly large and fill a significant majority of the server's disk space. But how can we prevent those big log files? Just changing the log level to ERROR
or FATAL
might work, but it isn't a good idea when running tools like CrowdSec that help to secure the infrastructure from malicious attackers. The answer to the problem is easy: Logrotate!
Enable Lograte
Rotation of log files means automatically creating a new log file when a criterion is met (such as time or file size) and storing a defined number of old log files. To enable it, there are two possibilities: Changing Traefik's configuration or using the system's logrotate.
Rotate traefik.log
Rotating the traefik.log
file is easy and pretty straightforward. Based on the official documentation, the four flags maxSize
, maxBackups
, maxAge
, and compress
can be configured to personal needs. Search for your traefik.yml
file and open it in your favorite editor. When using Traefik in a Docker container, make sure the configuration file is mounted to your system. Add or change the log
part in your config to the following or something similar (YAML version):
log:
level: "INFO"
filePath: "/var/log/traefik/traefik.log"
maxSize: 100
maxBackups: 5
In this example, the log file located at /var/log/traefik/traefik.log
will be rotated if it exceeds the maximum size of 100 MB, while the system keeps a maximum of five log files. If a sixth file were created, the oldest would be deleted.
Rotate access.log
Unfortunately, the previous configuration options like maxSize
or maxBackups
are not available for the access log file. Based on the official documentation, an external USR1 signal trigger is needed to rotate the logfile. Thus, the accessLog
part of the traefik.yml
configuration file looks as follows:
accessLog:
filePath: "/var/log/traefik/access.log"
bufferingSize: 100
We use the system package logrotate
for it, described in the following for an Ubuntu server system. First, create a file /etc/logrotate.d/traefik
and open it in your favorite editor with admin permissions. We will add the following configuration for a Docker setup with CrowdSec:
/var/log/crowdsec/traefik/access.log {
weekly
rotate 13
missingok
notifempty
dateext
dateformat .%Y-%m-%d
create 0644 root root
postrotate
docker kill --signal="USR1" traefik
endscript
}
Remember to change the first line to your log location. This may differ from the previous configuration because logrotate is executed on your host system, while the configuration uses the file structure of your Docker containers. Based on the provided example configuration, it will rotate the logs every week (weekly
) and keep 13 log files (rotate 13
), which is approximately one-quarter of a year / 90 days. When a size-based log rotation is preferred, use size 100M
instead. There won't be errors if the log file is missing (missingok
) and the rotation will skip when the log file is empty (notifempty
). We will use a specific date-based format (dateext
and dateformat
). Every new log file is created with 0644 permissions, owned by the root user and group (create 0644 root root
). Since we are running Traefik inside Docker, we will send the suggested USR1
signal to the traefik
container. Remember to update this line to match your setup, e.g., if you are not using Docker or if the Traefik container has a different name. For non-Docker users, the line must be changed to kill -USR1 `pgrep traefik`
.
After creating and saving the file, test it with sudo logrotate --debug /etc/logrotate.d/traefik
. If this works, run it once with sudo logrotate /etc/logrotate.d/traefik
and manually check and verify if it really works.
Afterward, everything should work as expected. If you want to be sure, restart Traefik or your system to activate everything.
References
- Header Image: The gopher's logo of Traefik (Creative Commons 3.0 Attributions license)
- https://doc.traefik.io/traefik/observability/logs/
- https://doc.traefik.io/traefik/observability/access-logs/
- https://www.reddit.com/r/Traefik/comments/xw3lkv/traefik_log_file_gets_massive_over_time_is_there/
- https://stackoverflow.com/questions/49450422/how-to-enable-logrotation-for-traefik
- https://www.jagsiacs.co.uk/2023/09/15/traefik-log-rotation/