Logging in Nginx
One of the easiest ways to save yourself trouble with your web server is to configure appropriate logging today. Logging information on your server gives you access to the data that will help you troubleshoot and assess situations as they arise.
In this article, we will examine Nginx’s logging capabilities and discover how to configure these tools to best serve your needs. In this guide, we will be using an Ubuntu 12.04 VPS as an example, but any modern distribution should function in a similar way.
The Error_log Directive
Nginx uses a few different directives to control system logging. The one included in the core module is called “error_log”.
The error_log directive takes the following syntax:
error_log log_file [ log_level ]
The “log_file” in the example specifies the file where the logs will be written. The “log_level” specifies the lowest level of logging that you would like to record.
The error_log directive can be configured to log more or less information as required. The level of logging can be any one of the following:
- emerg: Emergency situations where the system is in an unusable state.
- alert: Severe situation where action is needed promptly.
- crit: Important problems that need to be addressed.
- error: An Error has occurred. Something was unsuccessful.
- warn: Something out of the ordinary happened, but not a cause for concern.
- notice: Something normal, but worth noting has happened.
- info: An informational message that might be nice to know.
- debug: Debugging information that can be useful to pinpoint where a problem is occurring.
The levels higher on the list are considered a higher priority. If you specify a level, the log will capture that level, and any level higher than the specified level.
For example, if you specify “error”, the log will capture messages labeled “error”, “crit”, “alert”, and “emerg”.
We can see this directive in use if we look in the main configuration file:
sudo nano /etc/nginx/nginx.conf
. . . access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; . . .
If you do not want the error_log to log anything, you must send the output into “/dev/null”:
error_log /dev/null crit;
The other logging directive that we see above, the “access_log” directive, will be discussed in the next section.
HttpLogModule Logging Directives
While the error_log directive is part of the core module, the access_log directive is part of the HttpLogModule. It provides the ability to customize logs.
There are a few other directives included with this module that assist in configuring custom logs.
The Log_format Directive
The log_format directive is used to describe the format of a log entry using plain text and variables.
There is one format that comes predefined with Nginx called “combined”. This is a common format used by many servers.
This is what the combined format would look like if it was not defined internally and needed to be specified with the log_format directive:
log_format combined '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent"';
This definition spans multiple lines until it finds the semi-colon (;).
The pieces beginning with a dollar sign ($) indicate variables, while the characters like “-“, “[“, and “]” are interpreted literally.
The general syntax of the command is:
log_format format_name string_describing_formatting;
You can use variables supported by the core module to formulate your logging strings.
The Access_log Directive
The access_log directive uses some similar syntax to the error_log directive, but is more flexible. It is used to configure custom logging.
The access_log directive uses the following syntax:
access_log /path/to/log/location [ format_of_log buffer_size ];
The default value for access_log is the “combined” format we saw in the log_format section. You can use any format defined by a log_format definition.
The buffer size is the maximum size of data that Nginx will hold before writing it all to the log. You can also specify compression of the log file by adding “gzip” into the definition:
access_log location format gzip;
Unlike the error_log directive, if you do not want logging, you can turn it off by specifying:
It is not necessary to write to “/dev/null” in this case.
As log files grow, it becomes necessary to manage the logging mechanisms to avoid filling up disk space. Log rotation is the process of switching out log files and possibly archiving old files for a set amount of time.
Nginx does not provide tools to manage log files, but it does include mechanisms that make log rotation simple.
Manual Log Rotation
If you would like to manually rotate your logs (or more likely, create a script to rotate them), you can do so by following the example in the Nginx wiki:
mv /path/to/access.log /path/to/access.log.0 kill -USR1 `cat /var/run/nginx.pid` sleep 1 [ post-rotation processing of old log file ]
First, we move the current log to a new file for archiving. A common scheme is to name the most recent log file with a suffix of “.0”, and then name older files with “.1”, and so on.
The command that actually rotates the logs is “kill -USR1 /var/run/nginx.pid”. This does not kill the Nginx process, but instead sends it a signal causing it to reload its log files. This will cause new requests to be logged to the refreshed log file.
The “/var/run/nginx.pid” file is where Nginx stores the master process’s pid. It is specified in the configuration file with a line that begins with “pid”:
sudo nano /etc/nginx/nginx.conf
. . . pid /path/to/pid/file; . . .
After the rotation, we execute “sleep 1” to allow the process to complete the transfer. We can then zip the old files or do whatever post-rotation processes we would like.
Log Rotation with logrotate
The logrotate application is a simple program to rotate logs. It is installed on Ubuntu by default, and Nginx on Ubuntu comes with a custom logrotate script.
We can see the log rotation script by typing:
sudo nano /etc/logrotate.d/nginx
The first line of the file specifies the location that the subsequent lines will apply to. Keep this in mind if you switch the location of logging in the Nginx configuration files.
The rest of the file specifies that the logs will be rotate daily and that 52 older copies will be preserved. The general configuration of logrotate is outside of the scope of this article.
We can see that the “postrotate” section contains a command similar to the manual rotation mechanisms we were employing:
postrotate [ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid` endscript
This section tells Nginx to reload the log files once the rotation is complete.
Its always better to verify if logrotate script is correct.
Just run command
It will produce debug output.
Remember, logrotate is not service which can be restarted. In case you need your logrotate script to run immediately, use:
logrotate -f -v /etc/logrotate.d/nginx
Rotating log is very important. Otherwise some day your harddisk may get full and then mysql will be the first process which will refuse to start!
Proper log configuration and management can save you time and energy in the event of a problem with your server. Having easy access to the information that will help you diagnose a problem can be the difference between a trivial fix and a persistent headache.
It is important to keep an eye on server logs in order to maintain a functional site and ensure that you are not exposing sensitive information. This guide should serve only as an introduction to your experience with logging.