配置 Broker(Redis)
21.3.1.indexer /etc/logstash/conf.d/indexer.conf input { redis { host => "127.0.0.1" port => "6379" key => "logstash:demo" data_type => "list" codec => "json" type => "logstash-redis-demo" tags => ["logstashdemo"] } } output { stdout { codec => rubydebug } elasticsearch { hosts => ["127.0.0.1:9200"] } } 测试 # redis-cli 127.0.0.1:6379> RPUSH logstash:demo "{\"time\": \"2012-01-01T10:20:00\", \"message\": \"logstash demo message\"}" (integer) 1 127.0.0.1:6379> exit 如果执行成功日志如下 # cat /var/log/logstash/logstash-plain.log [2017-03-22T15:54:36,491][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://127.0.0.1:9200/]}} [2017-03-22T15:54:36,496][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://127.0.0.1:9200/, :path=>"/"} [2017-03-22T15:54:36,600][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0x20dae6aa URL:http://127.0.0.1:9200/>} [2017-03-22T15:54:36,601][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil} [2017-03-22T15:54:36,686][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}} [2017-03-22T15:54:36,693][INFO ][logstash.outputs.elasticsearch] Installing elasticsearch template to _template/logstash [2017-03-22T15:54:36,780][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::Generic:0x2f9efc89 URL://127.0.0.1>]} [2017-03-22T15:54:36,787][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000} [2017-03-22T15:54:36,792][INFO ][logstash.inputs.redis ] Registering Redis {:identity=>"redis://@127.0.0.1:6379/0 list:logstash:demo"} [2017-03-22T15:54:36,793][INFO ][logstash.pipeline ] Pipeline main started [2017-03-22T15:54:36,838][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600} [2017-03-22T15:55:10,018][WARN ][logstash.runner ] SIGTERM received. Shutting down the agent. [2017-03-22T15:55:10,024][WARN ][logstash.agent ] stopping pipeline {:id=>"main"} 21.3.2.shipper input { file { path => [ "/var/log/nginx/access.log" ] start_position => "beginning" } } filter { grok { match => { "message" => "%{NGINXACCESS}" } add_field => { "type" => "access" } } date { match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ] } geoip { source => "clientip" } } output { redis { host => "127.0.0.1" port => 6379 data_type => "list" key => "logstash:demo" } } 原文出处：Netkiller 系列 手札 本文作者：陈景峯 转载请与作者联系，同时请务必标明文章原始出处和作者信息及本声明。
