Haproxy+keepalivd+varnish+LAMP
目 录
实验要求
实验前提
实验环境设计
拓扑结构
ip网络规划
基础环境配置
一、LAMP配置
1、HTTPD配置
2、Discuz安装
二、Haproxy配置：
Haproxy日志输出配置
三、Keepalived配置
四、Varnish 配置
五、修改web服务器日志配置
六、测试
1、客户端访问测试
2、客户端访问haprxoy监控页面测试
3、varnish缓存命中查看
4、后端服务器日志查看
5、重要 假设haproxy 不设置会话保持就会出现下面的情况
七、总结
实验要求
1、 动静分离部署Discuz，动静要实现负载均衡，注意会话问题；
2、 Haproxy和后端主机间添加Vanish缓存；
3、 Hapeoxy设计要求
a、动静分离；
b、stats page ，要求仅能通过本机主机使用管理接口；
c、粉笔考虑不同服务组的调度算法；
d、Haporxy高可用；
4、 给出拓扑，写成博客；
实验前提
实验前需要关闭iptables及selinux；
实验前做好时间同步
实验环境设计
拓扑结构
根据以上要求本次实验给出以下拓扑：
![wKioL1igUDOCGsPxAACZAUz55ZM566.png]( "HAproxy+keepalived+varnish+httpd+mysql.png")
ip网络规划
根据拓扑需求以下为各服务器网络配置：
注：1、因笔者笔记本性能问题dynamic-web组与static-web组分别用一台机器模拟；httpd虚拟主机实现；
2、本实验环境未做mysql的高可用集群，因此本实验环境，将mysql安装在172.16.76.30服务器仅作实验环境需求；
客户端ip：172.16.250.186；
Haproxy1：172.16.76.10；
Haproxy1：172.16.76.20；
KeepalivedVIP1:172.16.76.100；
KeepalivedVIP2:172.16.76.110;
Varnish:172.16.76.60;
dynamic-web1:172.16.76.30:80; dynamic-web1:172.16.76.30:8080;
static-web1:172.16.76.40:80；同上
static-web2:172.16.76.40:8080;
Mysql:172.16.76.30:3306;
注：以下主机名对应实验环境响应服务器
node1:172.16.76.10;
node2:172.16.76.20;
node3:172.16.76.30;
node4:172.16:76.40;
node6:172.16.76.60;
基础环境配置
node1：
[root@node1~]# yum install haproxy keepalived -y
node2:
[root@node2~]# yum install haproxy keepalived -y
node3:
[root@node3~]yum install mariadb-server httpd -y
node4:
[root@node4~]yum install httpd –y
node6:
[root@node6~]# yum install varnish -y
一、LAMP配置
1、HTTPD配置；
首先现在先配置后端web服务；
node3：
[root@node3 ~]# cd /etc/httpd/conf.d/
[root@node3 conf.d]# vim vhost.conf
<VirtualHost 172.16.76.30:8080>
DocumentRoot/apps/www/html/
ServerNamewww.linuxinfo.top
ErrorLoglogs/bbs-8080.error_log
Customloglogs/bbs-8080.access_log combined
<Directory"/apps/www/html/">
Options None
AllowOverrideNone
Require allgranted
</Directory>
</VirtualHost>
<VirtualHost 172.16.76.30:80>
DocumentRoot/var/www/html/
ServerNamewww.linuxinfo.top
ErrorLoglogs/bbs.error_log
Customloglogs/bbs.access_log combined #单独记录日志
<Directory"/var/www/html/">
Options None
AllowOverrideNone
Require all granted
</Directory>
</VirtualHost>
[root@node3 conf.d]# vim ../conf/httpd.conf
Listen 80
Listen 8080 #设置Httpd监听端口；
#DocumentRoot "/var/www/html" #将该行注释；
[root@node3 conf.d]# systemctl restarthttpd.service
[root@node3 ~]# mysql
MariaDB [(none)]>grantall on test.* to test@'localhost' identified by "test";
MariaDB [(none)]> grant all on test.*to test@'127.0.0.1' identified by "test";
MariaDB [(none)]> grant all on test.* totest@'172.16.76.40' identified by "test";
#此处允许172.16.76.40访问。
node4:
[root@node4 ~]# cd /etc/httpd/conf.d/
[root@node4 conf.d]# vim vhost.conf
<VirtualHost 172.16.76.40:8080>
DocumentRoot/apps/www/html/
ServerNamewww.linuxinfo.top
ErrorLoglogs/bbs-8080.error_log
Customloglogs/bbs-8080.access_log combined
<Directory"/apps/www/html/">
Options None
AllowOverrideNone
Require all granted
</Directory>
</VirtualHost>
<VirtualHost 172.16.76.40:80>
DocumentRoot/var/www/html/
ServerNamewww.linuxinfo.top
ErrorLoglogs/bbs.error_log
Customloglogs/bbs.access_log combined
<Directory"/var/www/html/">
Options None
AllowOverrideNone
Require all granted
</Directory>
</VirtualHost>
[root@node4 conf.d]# vim ../conf/httpd.conf
Listen 80
Listen 8080 #设置Httpd监听端口；
#DocumentRoot "/var/www/html" #将该行注释；
[root@node4 conf.d]# systemctl restart httpd.service
2、Discuz安装
[root@node3 ~]#cd /var/www/html/
[root@node3 ~]#unzip Discuz_X3.3_SC_UTF8_0101.zip -d html
[root@node3 ~]# chown -Rapache.apache html/
![wKioL1igUmzxlaiGAADBnHVYQMo993.jpg]( "QQ截图20170212152916.jpg")
![wKioL1igUmzwmIitAACbqvCIYpo645.jpg]( "QQ截图20170212152932.jpg")
![wKiom1igUm3x9qHnAACNj-xmb00823.jpg]( "QQ截图20170212152954.jpg")
![wKiom1igUm2j_I_oAACt8Cy84BU398.jpg]( "QQ截图20170212153027.jpg")
![wKioL1igUm2SR682AADItw3PRbU482.jpg]( "QQ截图20170212153435.jpg")
Discuz在172.16.16.30:80安装成功；因172.16.76.30还配置虚拟主机8080；
所以此处将已安装好的Discuz的html目录复制到8080虚拟主机即可；
[root@node3 ~]#cp –R /var/www/html /apps/www/html
为保证node4上static-web组网站配置相同将node3完整html目录打包传递到node4；
node3
root@node3 ~]#cd /var/www/
[root@node3 ~]#tar –cvf html.tar html/
[root@node3 ~]#scp html.tar 172.16.76.40:/var/www/
[root@node3 ~]#scp html.tar 172.16.76.40:/apps/www/
node4：
[root@node4 ~]#cd /var/www/
[root@node4 www]# tar –xvf html.tar
[root@node4 www]# cd /apps/www
[root@node4 www]# tar –xvf html.tar
注:因整个完整目录整体从172.16.76.30拷贝而来此处，所以需要修改mysql的连接地址；
[root@node4 ~]#cd /var/www/
[root@node4 upload]# vim config/config_global.php
$_config['db']['1']['dbhost'] = '172.16.76.30'; #此处设置mysql地址；
[root@node4 upload]# vim config/config_ucenter.php
define('UC_DBHOST', '172.16.76.30');
[root@node4 upload]# vim uc_server/data/config.inc.php
define('UC_DBHOST', '172.16.76.30');
即此处共计修改三个文件：
config/config_global.php
config/config_ucenter.php
uc_server/data/config.inc.php
同上: 172.16.76.40:8080 虚拟主机目录/apps/www/ 也需要修改以上三个文件。
至此LAMP全部配置完成；
写到这笔者想起来在172.16.76.30服务器上安装Discuz时其实可以配置数据库地址为172.16.76.30的这样后续复制网站目录时就不用修改那三个文件了![wKioL1igUzuzzdzAAAAF24CQr4s996.png]( "1.png")
![wKioL1igUzuzzdzAAAAF24CQr4s996.png]( "1.png")
![wKioL1igUzuzzdzAAAAF24CQr4s996.png]( "1.png")
；
二、Haproxy配置：
node1：
[root@node1~]# cd /etc/haproxy/
root@node1 haproxy]# vim haproxy.cfg
global
log 127.0.0.1 local2 info #日志配置
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unixsocket
stats socket/var/lib/haproxy/stats
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive10s
timeout check 10s
maxconn 3000
frontend www
bind *:80
mode http
option forwardfor header Client-IP
log global
配置:以.php结尾所有url调度至webser服务器组即:dyanamc-web组;
以.css .js等结尾与usrl中包含/static/p_w_picpathsd等字符的请求调至webser-static
即：static-web服务器组中；
acl web url_end -i .php
acl url_static url_beg -i /static /p_w_picpaths /javascript
acl url_static url_reg -i .js.*$ .css.*$ .jpg.*$ .png.*$.gif.*$
use_backend webser if web
use_backend webser-static ifurl_static
default_backend webser
backend webser
mode http
option redispatch
option abortonclose
option forwardfor header Client-IP #将真实ip转发至后端web；http日志设置
balance source #source调度算法；会话保持（注：看最后测试）
cookie SERVERID
option httpchk GET/upload/forum.php
server web1 172.16.76.30:80 cookie server1 weight 6 check inter 2000 \
rise 2 fall 3
server web2 172.16.76.30:8080 cookie server1 weight 6 check inter 2000 \
rise 2 fall 3
backend webser-static
server web1 172.16.76.60:80 check inter2000 rise 2 fall 3
listen admin_stats
bind *:9188
mode http
stats refresh 30s
stats uri/haproxy-status #配置监控页面url
stats realm welcomelogin\ Haproxy
stats authadmin:adminpass #监控界面验证用户密码
stats hide-version
stats admin if TRUE
acl client src 172.16.250.186
block unless client
#此处配置除172.16.250.186ip之外其余ip禁止访问监控页面；
[root@node1haproxy]# systemctl restart haproxy 重启haproxy；
因Haproxy配置高可用间此配置文件拷贝至node2即可；
[root@node1 haproxy]#scp haproxy.cfg172.16.76.20:/etc/haproxy
node2:
[root@node2~]#systemctl restart haproxy
Haproxy日志输出配置
1、node1
[root@node1 ~]# vim /etc/rsyslog.d/haprxoy.conf
$Modload imudp
$UDPServerRun 514
local2.* /var/log/haproxy/haproxy_info.log
[root@node1 ~]# vim /etc/sysconfig/rsyslog
SYSLOGD_OPTIONS="-c 2 -r -m 0" #配置接收远程日志
2、node2同上
三、Keepalived配置
node1：
[root@node1 ~]# cd /etc/keepalived/
[root@node1 keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_fromkeepadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node1
vrrp_mcast_group4224.0.76.100
}
vrrp_script check_haproxy {
script "killall -0haproxy"
interval 2
weight 21
}
vrrp_instance HAporxy_VIP1 {
state MASTER # haproxy2 即node2上为BACKUP
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass test1111
}
virtual_ipaddress {
172.16.76.100/16 deveth0 label eth0:0
}
notify_master"/etc/keepalived/notify.sh master VIP1 vrid100"
notify_backup"/etc/keepalived/notify.sh backup VIP1 vrid100"
notify_fault"/etc/keepalived/notify.sh fault VIP1 vrid100"
track_script {
check_haproxy
}
}
vrrp_instance HAprxoy_VIP2 {
state BACKUP # haproxy2 即node2上为MASTER
interface eth0
virtual_router_id 110
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 1111test
}
virtual_ipaddress {
172.16.76.110/16 deveth0 label eth0:1
}
notify_master"/etc/keepalived/notify.sh master VIP2 vrid110"
notify_backup"/etc/keepalived/notify.sh backup VIP2 vrid110"
notify_fault"/etc/keepalived/notify.sh fault VIP2 vrid110"
track_script {
check_haproxy
}
}
notify.sh脚本
notify.sh 脚本
#!/bin/bash
#
info="$2$3"
contact='root@localhost'
notify(){
mailsubject="$(hostname) $info tobe $1, vip floating"
mailbody="$(date +'%F %T'): vrrptransition, $(hostname) $info changed to be $1"
echo "$mailbody" | mail -s"$mailsubject" $contact
}
case$1 in
master)
notify master $2 $3
;;
backup)
notify backup $2 $3
;;
fault)
notify fault $2 $3
;;
*)
echo "Usage: $(basename$0) {master|backup|fault}"
exit 1
;;
esac
将node1Keepalive的配置文件拷贝至node2一份（注:看配置文件注释修改）
[root@node1~]# cd /etc/keepalived/
[root@node1 keepalived]# scp keepalived.conf172.16.76.20:/etc/keepalived/
node2
一次启动node1 、 node2 Keepalived
[root@node1 ~]# systemctl restart Keepalived
[root@node2 ~]# systemctl restart Keepalived
四、Varnish 配置：
node6：
[root@node6~]# cd /etc/varnish/
[root@node6 varnish]# cat default.vcl
vcl 4.0;
import directors;
probe static {
.url ="/upload/forum.php";
.interval = 5s;
.timeout = 1s;
}
backend web1 {
.host ="172.16.76.50";
.port = "8080";
.probe = static;
}
backend web2 {
.host ="172.16.76.40";
.port = "8080";
.probe = static;
}
sub vcl_init {
new webser =directors.round_robin();
webser.add_backend(web1);
webser.add_backend(web2);
}
acl purgers {
"172.16.76.30"/32;
"172.16.76.40"/32;
}
sub vcl_purge {
return(synth(200,"Purged"));
}
sub vcl_recv {
if (req.restarts == 0) {
if(req.http.x-forwarded-for) {
setreq.http.X-Forwarded-For = req.http.X-Forwarded-For +"," +client.ip;
} else {
setreq.http.X-Forwarded-For = client.ip;
}
}
set req.backend_hint =webser.backend();
if (req.url ~ "\.(html|jpg|png|bmp|jpeg|gif|js|ico|swf|css)$") {
unset req.http.cookie;
}
if (req.method == "PURGE") {
if (!client.ip ~ purgers) {
return(synth(405,"Purgingnot allowed for " + client.ip));
}
return(purge);
}
}
sub vcl_backend_response {
}
sub vcl_deliver {
if (obj.hits>0) {
set resp.http.X-Cache ="Hit Via" + " " + server.ip;
}else {
set resp.http.X-Cache ="Miss Via" + " " +server.ip;
}
}
五、修改web服务器日志配置
node3:
[root@node3~]# vim /etc/httpd/conf/httpd.conf
LogFormat"%{Client-ip}i %l %u %t \"%r\" %>s %b\"%{Referer}i\" \"%{User-Agent}i\"" combined
node4:同上
重启启动各个服务即可
六、测试
1、客户端访问测试
![wKiom1igVleQ1-U3AAC12InO-cM062.jpg]( "QQ截图20170212173852.jpg")
![wKioL1igVlmjWlrHAAC82PRgksY489.jpg]( "QQ截图20170212173903.jpg")
2、客户端访问haprxoy监控页面测试
![wKioL1igVrzSxBXtAAEzz3s4_Ko287.jpg]( "QQ截图20170212174351.jpg")
非客户端172.16.76.186访问测试
![wKiom1igVtuiAnzcAAA4Xy5H3cE889.jpg]( "QQ截图20170211135424.jpg")
3、varnish缓存命中查看
![wKioL1igVvSBdkaGAAEDHg4BBQU356.jpg]( "QQ截图20170212172849.jpg")
4、后端服务器日志查看
![wKiom1igVxiwgaDtAAKolNsRpPE830.jpg]( "QQ截图20170212172834.jpg")
5、重要 假设haproxy 不设置会话保持就会出现下面的情况
验证码输入正确，但是无法登录。这是因为session会话的原因；
![wKioL1igV0XB-PVcAABBIzkNKLI369.jpg]( "QQ截图20170211181554.jpg")
七、总结
本实验完成实现以下功能：
1、Haproxy负载均衡
2、Haproxy 监控页面访问控制
3、varnish缓存
4、动静分离
5、会话保持
本实验至此全部完成。实验中可能会有下瑕疵，各位注意观察出错可别怪我没有提醒哈；
若实验中真有问题欢迎指正；
