filebeat 获取nginx日志 发送给ElasticSearch
目的：通过filebeat获取nginx日志，发送给ElasticSearch，filebeat可以解析json格式日志，所以设置nginx日志为json格式。
1、配置nginx配置文件
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
log_format jsonTest
'{"@timestamp":"$time_iso8601",'
'"host":"$server_addr",'
'"service":"nginxTest",'
'"trace":"$upstream_http_ctx_transaction_id",'
'"log":"log",'
'"clientip":"$remote_addr",'
'"remote_user":"$remote_user",'
'"request":"$request",'
'"http_user_agent":"$http_user_agent",'
'"size":$body_bytes_sent,'
'"responsetime":$request_time,'
'"upstreamtime":"$upstream_response_time",'
'"upstreamhost":"$upstream_addr",'
'"http_host":"$host",'
'"url":"$uri",'
'"domain":"$host",'
'"xff":"$http_x_forwarded_for",'
'"referer":"$http_referer",'
'"status":"$status"}'
;
access_log /
var
/log/nginx/access.log jsonTest;
|
定义jsonTest的json格式，其中trace是页面response headers的值 ctx-transaction-id，通过upstream_http_ctx_transaction_id可以获取头文件属性。
将日志输出到/var/log/nginx/access.log
2、配置filebeat配置文件
|
1
2
3
4
5
6
7
8
9
|
filebeat.prospectors:
- type: log
paths:
-
'/root/front/logs/*.log'
json.message_key: log
json.keys_under_root:
true
output.elasticsearch:
hosts: [
"*.*.*.*:9200"
]
|
将日志传输给ElasticSearch
本文转自gaofeng36599 51CTO博客，原文链接:http://blog.51cto.com/786678398/1980222
