Haproxy+keepalivd+varnish+LAMP实验
Haproxy+keepalivd+varnish+LAMP
目 录
实验要求
实验前提
实验环境设计
拓扑结构
ip网络规划
基础环境配置
一、LAMP配置
1、HTTPD配置
2、Discuz安装
二、Haproxy配置：
Haproxy日志输出配置
三、Keepalived配置
四、Varnish 配置
五、修改web服务器日志配置
六、测试
1、客户端访问测试
2、客户端访问haprxoy监控页面测试
3、varnish缓存命中查看
4、后端服务器日志查看
5、重要 假设haproxy 不设置会话保持就会出现下面的情况
七、总结
实验要求
1、 动静分离部署Discuz，动静要实现负载均衡，注意会话问题；
2、 Haproxy和后端主机间添加Vanish缓存；
3、 Hapeoxy设计要求
a、动静分离；
b、stats page ，要求仅能通过本机主机使用管理接口；
c、粉笔考虑不同服务组的调度算法；
d、Haporxy高可用；
4、 给出拓扑，写成博客；
实验前提
实验前需要关闭iptables及selinux；
实验前做好时间同步
实验环境设计
拓扑结构
根据以上要求本次实验给出以下拓扑：
ip网络规划
根据拓扑需求以下为各服务器网络配置：
注：1、因笔者笔记本性能问题dynamic-web组与static-web组分别用一台机器模拟；httpd虚拟主机实现；
2、本实验环境未做mysql的高可用集群，因此本实验环境，将mysql安装在172.16.76.30服务器仅作实验环境需求；
客户端ip：172.16.250.186；
Haproxy1：172.16.76.10；
Haproxy1：172.16.76.20；
KeepalivedVIP1:172.16.76.100；
KeepalivedVIP2:172.16.76.110;
Varnish:172.16.76.60;
dynamic-web1:172.16.76.30:80; dynamic-web1:172.16.76.30:8080;
static-web1:172.16.76.40:80；同上
static-web2:172.16.76.40:8080;
Mysql:172.16.76.30:3306;
注：以下主机名对应实验环境响应服务器
node1:172.16.76.10;
node2:172.16.76.20;
node3:172.16.76.30;
node4:172.16:76.40;
node6:172.16.76.60;
基础环境配置
node1：
[root@node1~]# yum install haproxy keepalived -y
node2:
[root@node2~]# yum install haproxy keepalived -y
node3:
[root@node3~]yum install mariadb-server httpd -y
node4:
[root@node4~]yum install httpd –y
node6:
[root@node6~]# yum install varnish -y
一、LAMP配置
1、HTTPD配置；
首先现在先配置后端web服务；
node3：
[root@node3 ~]# cd /etc/httpd/conf.d/ [root@node3 conf.d]# vim vhost.conf <VirtualHost 172.16.76.30:8080> DocumentRoot/apps/www/html/ ServerNamewww.linuxinfo.top ErrorLoglogs/bbs-8080.error_log Customloglogs/bbs-8080.access_log combined <Directory"/apps/www/html/"> Options None AllowOverrideNone Require allgranted </Directory> </VirtualHost> <VirtualHost 172.16.76.30:80> DocumentRoot/var/www/html/ ServerNamewww.linuxinfo.top ErrorLoglogs/bbs.error_log Customloglogs/bbs.access_log combined #单独记录日志 <Directory"/var/www/html/"> Options None AllowOverrideNone Require all granted </Directory> </VirtualHost> [root@node3 conf.d]# vim ../conf/httpd.conf Listen 80 Listen 8080 #设置Httpd监听端口； #DocumentRoot "/var/www/html" #将该行注释； [root@node3 conf.d]# systemctl restarthttpd.service [root@node3 ~]# mysql MariaDB [(none)]>grantall on test.* to test@'localhost' identified by "test"; MariaDB [(none)]> grant all on test.*to test@'127.0.0.1' identified by "test"; MariaDB [(none)]> grant all on test.* totest@'172.16.76.40' identified by "test"; #此处允许172.16.76.40访问。
node4:
[root@node4 ~]# cd /etc/httpd/conf.d/ [root@node4 conf.d]# vim vhost.conf <VirtualHost 172.16.76.40:8080> DocumentRoot/apps/www/html/ ServerNamewww.linuxinfo.top ErrorLoglogs/bbs-8080.error_log Customloglogs/bbs-8080.access_log combined <Directory"/apps/www/html/"> Options None AllowOverrideNone Require all granted </Directory> </VirtualHost> <VirtualHost 172.16.76.40:80> DocumentRoot/var/www/html/ ServerNamewww.linuxinfo.top ErrorLoglogs/bbs.error_log Customloglogs/bbs.access_log combined <Directory"/var/www/html/"> Options None AllowOverrideNone Require all granted </Directory> </VirtualHost> [root@node4 conf.d]# vim ../conf/httpd.conf Listen 80 Listen 8080 #设置Httpd监听端口； #DocumentRoot "/var/www/html" #将该行注释； [root@node4 conf.d]# systemctl restart httpd.service
2、Discuz安装
[root@node3 ~]#cd /var/www/html/ [root@node3 ~]#unzip Discuz_X3.3_SC_UTF8_0101.zip -d html [root@node3 ~]# chown -Rapache.apache html/
Discuz在172.16.16.30:80安装成功；因172.16.76.30还配置虚拟主机8080；
所以此处将已安装好的Discuz的html目录复制到8080虚拟主机即可；
[root@node3 ~]#cp –R /var/www/html /apps/www/html
为保证node4上static-web组网站配置相同将node3完整html目录打包传递到node4；
node3
root@node3 ~]#cd /var/www/ [root@node3 ~]#tar –cvf html.tar html/ [root@node3 ~]#scp html.tar 172.16.76.40:/var/www/ [root@node3 ~]#scp html.tar 172.16.76.40:/apps/www/
node4：
[root@node4 ~]#cd /var/www/
[root@node4 www]# tar –xvf html.tar
[root@node4 www]# cd /apps/www
[root@node4 www]# tar –xvf html.tar
注:因整个完整目录整体从172.16.76.30拷贝而来此处，所以需要修改mysql的连接地址；
[root@node4 ~]#cd /var/www/
[root@node4 upload]# vim config/config_global.php
$_config['db']['1']['dbhost'] = '172.16.76.30'; #此处设置mysql地址；
[root@node4 upload]# vim config/config_ucenter.php
define('UC_DBHOST', '172.16.76.30');
[root@node4 upload]# vim uc_server/data/config.inc.php
define('UC_DBHOST', '172.16.76.30');
即此处共计修改三个文件：
config/config_global.php
config/config_ucenter.php
uc_server/data/config.inc.php
同上: 172.16.76.40:8080 虚拟主机目录/apps/www/ 也需要修改以上三个文件。
至此LAMP全部配置完成；
写到这笔者想起来在172.16.76.30服务器上安装Discuz时其实可以配置数据库地址为172.16.76.30的这样后续复制网站目录时就不用修改那三个文件了
；
二、Haproxy配置：
node1：
[root@node1~]# cd /etc/haproxy/ root@node1 haproxy]# vim haproxy.cfg global log 127.0.0.1 local2 info #日志配置 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 4000 user haproxy group haproxy daemon # turn on stats unixsocket stats socket/var/lib/haproxy/stats defaults mode http log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive10s timeout check 10s maxconn 3000 frontend www bind *:80 mode http option forwardfor header Client-IP log global 配置:以.php结尾所有url调度至webser服务器组即:dyanamc-web组; 以.css .js等结尾与usrl中包含/static/p_w_picpathsd等字符的请求调至webser-static 即：static-web服务器组中； acl web url_end -i .php acl url_static url_beg -i /static /p_w_picpaths /javascript acl url_static url_reg -i .js.*$ .css.*$ .jpg.*$ .png.*$.gif.*$ use_backend webser if web use_backend webser-static ifurl_static default_backend webser backend webser mode http option redispatch option abortonclose option forwardfor header Client-IP #将真实ip转发至后端web；http日志设置 balance source #source调度算法；会话保持（注：看最后测试） cookie SERVERID option httpchk GET/upload/forum.php server web1 172.16.76.30:80 cookie server1 weight 6 check inter 2000 \ rise 2 fall 3 server web2 172.16.76.30:8080 cookie server1 weight 6 check inter 2000 \ rise 2 fall 3 backend webser-static server web1 172.16.76.60:80 check inter2000 rise 2 fall 3 listen admin_stats bind *:9188 mode http stats refresh 30s stats uri/haproxy-status #配置监控页面url stats realm welcomelogin\ Haproxy stats authadmin:adminpass #监控界面验证用户密码 stats hide-version stats admin if TRUE acl client src 172.16.250.186 block unless client #此处配置除172.16.250.186ip之外其余ip禁止访问监控页面； [root@node1haproxy]# systemctl restart haproxy 重启haproxy； 因Haproxy配置高可用间此配置文件拷贝至node2即可； [root@node1 haproxy]#scp haproxy.cfg172.16.76.20:/etc/haproxy
node2:
[root@node2~]#systemctl restart haproxy
Haproxy日志输出配置
1、node1
[root@node1 ~]# vim /etc/rsyslog.d/haprxoy.conf $Modload imudp $UDPServerRun 514 local2.* /var/log/haproxy/haproxy_info.log [root@node1 ~]# vim /etc/sysconfig/rsyslog SYSLOGD_OPTIONS="-c 2 -r -m 0" #配置接收远程日志
2、node2同上
三、Keepalived配置
node1：
[root@node1 ~]# cd /etc/keepalived/
[root@node1 keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_fromkeepadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node1
vrrp_mcast_group4224.0.76.100
}
vrrp_script check_haproxy {
script "killall -0haproxy"
interval 2
weight 21
}
vrrp_instance HAporxy_VIP1 {
state MASTER # haproxy2 即node2上为BACKUP
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass test1111
}
virtual_ipaddress {
172.16.76.100/16 deveth0 label eth0:0
}
notify_master"/etc/keepalived/notify.sh master VIP1 vrid100"
notify_backup"/etc/keepalived/notify.sh backup VIP1 vrid100"
notify_fault"/etc/keepalived/notify.sh fault VIP1 vrid100"
track_script {
check_haproxy
}
}
vrrp_instance HAprxoy_VIP2 {
state BACKUP # haproxy2 即node2上为MASTER
interface eth0
virtual_router_id 110
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 1111test
}
virtual_ipaddress {
172.16.76.110/16 deveth0 label eth0:1
}
notify_master"/etc/keepalived/notify.sh master VIP2 vrid110"
notify_backup"/etc/keepalived/notify.sh backup VIP2 vrid110"
notify_fault"/etc/keepalived/notify.sh fault VIP2 vrid110"
track_script {
check_haproxy
}
}
notify.sh脚本
notify.sh 脚本
#!/bin/bash
#
info="$2$3"
contact='root@localhost'
notify(){
mailsubject="$(hostname) $info tobe $1, vip floating"
mailbody="$(date +'%F %T'): vrrptransition, $(hostname) $info changed to be $1"
echo "$mailbody" | mail -s"$mailsubject" $contact
}
case$1 in
master)
notify master $2 $3
;;
backup)
notify backup $2 $3
;;
fault)
notify fault $2 $3
;;
*)
echo "Usage: $(basename$0) {master|backup|fault}"
exit 1
;;
esac
将node1Keepalive的配置文件拷贝至node2一份（注:看配置文件注释修改）
[root@node1~]# cd /etc/keepalived/
[root@node1 keepalived]# scp keepalived.conf172.16.76.20:/etc/keepalived/
node2
一次启动node1 、 node2 Keepalived
[root@node1 ~]# systemctl restart Keepalived
[root@node2 ~]# systemctl restart Keepalived
四、Varnish 配置：
node6：
[root@node6~]# cd /etc/varnish/
[root@node6 varnish]# cat default.vcl
vcl 4.0;
import directors;
probe static {
.url ="/upload/forum.php";
.interval = 5s;
.timeout = 1s;
}
backend web1 {
.host ="172.16.76.50";
.port = "8080";
.probe = static;
}
backend web2 {
.host ="172.16.76.40";
.port = "8080";
.probe = static;
}
sub vcl_init {
new webser =directors.round_robin();
webser.add_backend(web1);
webser.add_backend(web2);
}
acl purgers {
"172.16.76.30"/32;
"172.16.76.40"/32;
}
sub vcl_purge {
return(synth(200,"Purged"));
}
sub vcl_recv {
if (req.restarts == 0) {
if(req.http.x-forwarded-for) {
setreq.http.X-Forwarded-For = req.http.X-Forwarded-For +"," +client.ip;
} else {
setreq.http.X-Forwarded-For = client.ip;
}
}
set req.backend_hint =webser.backend();
if (req.url ~ "\.(html|jpg|png|bmp|jpeg|gif|js|ico|swf|css)$") {
unset req.http.cookie;
}
if (req.method == "PURGE") {
if (!client.ip ~ purgers) {
return(synth(405,"Purgingnot allowed for " + client.ip));
}
return(purge);
}
}
sub vcl_backend_response {
}
sub vcl_deliver {
if (obj.hits>0) {
set resp.http.X-Cache ="Hit Via" + " " + server.ip;
}else {
set resp.http.X-Cache ="Miss Via" + " " +server.ip;
}
}
五、修改web服务器日志配置
node3:
[root@node3~]# vim /etc/httpd/conf/httpd.conf
LogFormat"%{Client-ip}i %l %u %t \"%r\" %>s %b\"%{Referer}i\" \"%{User-Agent}i\"" combined
node4:同上
重启启动各个服务即可
六、测试
1、客户端访问测试
2、客户端访问haprxoy监控页面测试
非客户端172.16.76.186访问测试
3、varnish缓存命中查看
4、后端服务器日志查看
5、重要 假设haproxy 不设置会话保持就会出现下面的情况
验证码输入正确，但是无法登录。这是因为session会话的原因；
七、总结
本实验完成实现以下功能：
1、Haproxy负载均衡
2、Haproxy 监控页面访问控制
3、varnish缓存
4、动静分离
5、会话保持
本实验至此全部完成。实验中可能会有下瑕疵，各位注意观察出错可别怪我没有提醒哈；
若实验中真有问题欢迎指正；
