用Kibana+Logstash+Elasticsearch快速搭建实时日志查询、收集与分析系统
安装环境
先看看都需要安装什么软件包
ruby 运行Kibana 必须，
rubygems 安装ruby扩展必须
bundler 功能类似于yum
JDK 运行java程序必须
redis 用来处理日志队列
logstash 收集、过滤日志
ElasticSearch 全文搜索服务(logstash集成了一个)
kibana 页面展示
ruby 运行Kibana 必须，
rubygems 安装ruby扩展必须
bundler 功能类似于yum
JDK 运行java程序必须
redis 用来处理日志队列
logstash 收集、过滤日志
ElasticSearch 全文搜索服务(logstash集成了一个)
kibana 页面展示
192.168.18.240 logstash index，kibana，JDK
192.168.18.241 logstash agent，JDK
192.168.18.242 redis
192.168.18.243 ElasticSearch，JDK
192.168.18.241 logstash agent，JDK
192.168.18.242 redis
192.168.18.243 ElasticSearch，JDK
先安装redis (192.168.18.242)
#
wget http://redis.googlecode.com/files/redis-2.6.12.tar.gz
# tar zxvf redis-2.6.12.tar.gz
# mv redis-2.6.12 redis
# cd redis
# make -j24
# make install
# vi /root/soft/redis/redis.conf
修改level为 loglevel verbose
# redis-server /root/soft/redis/redis.conf &
看看 redis服务的状态
# lsof -i:6379
安装
elasticsearch
(192.168.18.243)
elasticsearch会依赖于java
# vi /etc/apt/sources.list
deb http:
//ftp.debian.org/debian/ squeeze main non-free
deb-src http:
//ftp.debian.org/debian/ squeeze main non-free
# apt-get update
# apt-cache search sun-java
# apt-get install sun-java6-jdk sun-java6-jre
# java -version
#
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.6.zip
#
unzip
elasticsearch-0.20.6.zip
#
mv
elasticsearch-0.20.6
/usr/local/share/elasticsearch
#
cd /usr/local/share/elasticsearch/bin/
# ./elasticsearch -f
|
在 logstash index上安装基础的软件环境：
(192.168.18.240)
# vi /etc/apt/sources.list
deb http:
//ftp.debian.org/debian/ squeeze main non-free
deb-src http:
//ftp.debian.org/debian/ squeeze main non-free
# apt-get update
# apt-cache search sun-java
# apt-get install sun-java6-jdk sun-java6-jre
# java -version
开始安装logstash (
其实logstash 就是一个java脚本，不需要安装... 下载即用
)
# wget https://logstash.objects.dreamhost.com/release/logstash-1.1.9-monolithic.jar
# vi /root/soft/redis.conf
input {
redis {
host => '192.168.18.242'
data_type => 'list'
port => "6379"
key => 'logstash:redis'
type => 'redis-input'
}
}
output {
elasticsearch {
host => '192.168.18.243'
port => "9300"
}
}
# java -jar /root/soft/logstash-1.1.9-monolithic.jar agent -f /root/soft/redis.conf -- web --backend elasticsearch:///?local
现在可以通过浏览器访问一下 http://192.168.18.240:9292 看看logstash是的页面是个什么样子
配置logstash的agent (192.168.18.241)
安装sun-java6-jre sun-java6-jdk
#
wget https://logstash.objects.dreamhost.com/release/logstash-1.1.9-monolithic.jar
# vi /root/soft/redis.conf
input {
stdin {
type => "stdin-type"
}
file {
type => "linux-syslog"
# Wildcards work, here :)
path => [ "/var/log/*.log", "/var/log/messages", "/var/log/syslog" ]
}
}
output {
redis {
host => '192.168.18.242'
data_type => 'list'
key => 'logstash:redis'
}
}
# java -jar /root/soft/logstash-1.1.9-monolithic.jar agent -f /root/soft/redis.conf &
OK，最后就是 Kibana了 ，我把Kibana装在了 logstash index上面
|
