Skip to content
鼓励作者:欢迎打赏犒劳

03-NGINX日志

https://bloodzer0.github.io/ossa/log_analysis/nginx_log/

Nginx配置日志格式

初始化配置

shell
log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                '$status $body_bytes_sent "$http_referer" '
                '"$http_user_agent" "$http_x_forwarded_for"';

配置为JSON格式并添加字段

shell
log_format  main escape=json  '{ "@timestamp": "$time_local", '
                        '"remote_addr": "$remote_addr",'
                        '"remote_port": "$remote_port",'
                        '"scheme": "$scheme",'
                        '"request_uri": "$request_uri",'
                        '"request_method": "$request_method",'
                        '"request_time": "$request_time",'
                        '"request_length": "$request_length",'
                        '"response_status": "$status",'
                        '"body_bytes_sent": "$body_bytes_sent",'
                        '"http_referer": "$http_referer",'
                        '"http_user_agent": "$http_user_agent",'
                        '"http_x_forwarded_for": "$http_x_forwarded_for",'
                        '"upstream_addr": "$upstream_addr",'
                        '"upstream_response_time": "$upstream_response_time"}';

Nginx日志字段含义

字段含义示例
body_bytes_sent响应body字节数3650
bytes_sent响应总字节数175
hostIP或域名(不包括端口)10.10.10.14
http_hostIP或域名(包括端口)10.10.10.14:81
http_refererreferer信息https://blog.share888.top/ 引导用户到当前请求的页面的来源URL
http_user_agentUA信息Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.90 Safari/537.36
http_x_forwarded_forXFF信息192.168.1.1 通过代理服务器的原始客户端 IP 地址。
remote_addr客户端地址10.10.10.1
remote_user客户端认证用户名admin
request请求URI和协议GET /favicon.ico HTTP/1.1
request_body请求的body
request_length请求长度571
request_method请求方法GET
request_time请求处理时间10 请求处理的总时间(秒)
response_body返回的body
response_header_data响应头数据
schema协议http
server_name虚拟主机名称
server_port服务器端口
server_protocol服务器协议
ssl_cipher交换数据中的算法
ssl_protocolSSL协议版本
status返回状态码404
time_local时间戳16/Jun/2019:23:29:50 -0400
upstream_addr后端提供服务地址
upstream_connect_time与服务器连接所花费的时间
upstream_response_time后端处理时间后端服务器响应时间(适用于反向代理)
upstream_statusupstream状态200

打印request_body

shell
# web_server配置
# The variable’s value is made available in locations processed by the proxy_pass, fastcgi_pass, uwsgi_pass, and scgi_pass directives when the request body was read to a memory buffer.
# 意味着:只有location中用到proxy_pass、fastcgi_pass、scgi_pass命令时,request_body才有值。

log_format main $request_body # 我这里是简写

location ~ \.php$ {
    root           /usr/share/nginx/html;
    fastcgi_pass   127.0.0.1:9000;
    fastcgi_index  index.php;
    fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    include        fastcgi_params;
}

打印response_body

shell
# web_server配置
log_format main $response_body # 我这里是简写
server {
    listen       80;
    server_name  localhost;
    lua_need_request_body on;
    set $response_body "";
    body_filter_by_lua '
        local response_body = string.sub(ngx.arg[1],1,1000)
        ngx.ctx.buffered = (ngx.ctx.buffered or "") ..  response_body
        if ngx.arg[2] then
            ngx.var.response_body = ngx.ctx.buffered
        end
    ';
    location ~ \.php$ {
        root           /usr/local/nginx/html;
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        include        fastcgi_params;
    }
}

完整的Nginx日志配置

shell
log_format  main escape=json  '{ "@timestamp": "$time_local", '
                    '"remote_addr": "$remote_addr",'
                    '"remote_port": "$remote_port",'
                    '"scheme": "$scheme",'
                    '"request_uri": "$request_uri",'
                    '"request_method": "$request_method",'
                    '"request_time": "$request_time",'
                    '"request_length": "$request_length",'
                    '"response_status": "$status",'
                    '"body_bytes_sent": "$body_bytes_sent",'
                    '"http_referer": "$http_referer",'
                    '"http_user_agent": "$http_user_agent",'
                    '"http_x_forwarded_for": "$http_x_forwarded_for",'
                    '"upstream_addr": "$upstream_addr",'
                    '"upstream_response_time": "$upstream_response_time",'
                    '"request_body": "$request_body", '
                    '"response_body": "$response_body" }';

注意事项

  • request_body与response_body如果在上传文件或者下载文件时,body内容会很大,需要考虑对性能和存储的影响。
  • 对于timestamp参数,可以不使用time_local而使用time_iso8601。
  • 如果运维对于网络的性能有要求,可以考虑使用$tcpinfo_rtt, $tcpinfo_rttvar, $tcpinfo_snd_cwnd, $tcpinfo_rcv_space。

Nginx配置access.log日志格式以及日志按天分割

在Nginx的使用过程中,如果不对Nginx的日志做自定义的配置的话,那么默认的access.log日志默认就只有这么一个日志文件,随着系统使用时间越来越就,日志文件就会越来越大,而且默认的日志记录的格式也不方便分析。所以我们在实际使用Nginx的过程中需要对其日志做一些配置。 在http配置块中进行配置,也可以在每个server块中配置不同的access_log用以区分

shell
http{
    # 定义一个变量来存储年月日  
    map $time_iso8601 $date {  
        default "date-not-found";  
        '~^(?<ymd>\d{4}-\d{2}-\d{2})' $ymd;
    }  
    # 定义一个变量来存储时分秒  
    map $time_iso8601 $time {  
        default "time-not-found";  
        '~T(?<hms>\d{2}:\d{2}:\d{2})' $hms;
    }
    # 定义一个变量来存储毫秒  "msec": "1731929153.581"
    map $msec $sss {  
        default "sss-not-found";  
        '~\.(?<s3>\d{3})$' $s3; 
    }
    # 定义日志格式为json格式
    log_format json_format '{  
        "timestamp": "$date $time.$sss",  
        "remote_addr": "$remote_addr",  
        "remote_user": "$remote_user",  
        "request": "$request",  
        "status": "$status",  
        "body_bytes_sent": "$body_bytes_sent",  
        "http_referer": "$http_referer",  
        "http_user_agent": "$http_user_agent",  
        "http_x_forwarded_for": "$http_x_forwarded_for"  
    },';
    # 定义一个变量来存储年月,可用来作为access.log的日志文件名,按月自动分割日志
    map $time_iso8601 $logmonth {  
        '~^(?<ym>\d{4}-\d{2})' $ym;  
        default 'date-not-found';  
    }
    # 定义一个变量来存储年月日,可用来作为access.log的日志文件名,按天自动分割日志
    map $time_iso8601 $logdate {  
        '~^(?<ymd>\d{4}-\d{2}-\d{2})' $ymd;  
        default 'date-not-found';  
    }
    # 开启access.log日志,可设置日志名变量(这里选择上面的按天)以便分割日志;可设置日志格式,以便更直观分析日志
    access_log logs/access-$logdate.log json_format;
}

配置完需重启Nginx服务,如果发现没对应日志文件生成,看是否是 logs目录没有写入的权限,可以分配权限后再看一下。

分配权限:chmod 777 logs

如有转载或 CV 的请标注本站原文地址