拓扑图如下:
一、为什么要使用负载均衡技术?
1、系统高可用性
2、 系统可扩展性
3、 负载均衡能力
LVS+keepalived能很好的实现以上的要求,LVS提供负载均衡,keepalived提供健康检查,故障转移,提高系统的可用性!采用这样的架构以后很容易对现有系统进行扩展,只要在后端添加或者减少realserver,只要更改lvs的配置文件,并能实现无缝配置变更!
二、LVS+Keepalived介绍
1、 LVS
LVS是一个开源的软件,可以实现LINUX平台下的简单负载均衡。LVS是Linux Virtual Server的缩写,意思是Linux虚拟服务器。目前有三种IP负载均衡技术(VS/NAT、VS/TUN和VS/DR);八种调度算法(rr,wrr,lc,wlc,lblc,lblcr,dh,sh)。
2、 keepalived
Keepalived 是运行在lvs 之上,它的主要功能是实现真实机的故障隔离及负载均衡器间的失败切换,提高系统的可用性,
Keepalived的作用是检测web服务器的状态,如果有一台web服务器死机,或工作出现故障,Keepalived将检测到,并将有故障的web服务器从系统中剔除,当web服务器工作正常后Keepalived自动将web服务器加入到服务器群中。
IP配置信息:
LVS-DR-Master 172.10.2.174
LVS-DR-BACKUP 172
LVS-DR-VIP 172.10.2.201 #这个不是一台机子,是在nginx上用脚本建的,下面会有脚本代码。
WEB1-ngxserver
172.10.2.170 #这个nginx,后端我用的是apache,nginx代理apache
WEB2-ngxserver 172.10.2.206 #这个也是nginx
三、ipvsadm安装:
先安装所需要的库:
yum install zlib-devel kernel-* libnl* popt* openssl openssl-devel net-snmp-*
#ipvs安装报错主要是缺少,libnl跟popt库文件
下载地址: http://www.linuxvirtualserver.org/software/index.html
解压后:make && make install
安装好后测试下:
ipvsadm ## 必须先执行此命令才能加载ip_vs模块
没报错有输出说明安装ok了。
再输入:
lsmod | grep ip_vs
四、安装keepalived(lvs和keepalived装在一台机子上,我全用的虚拟机子)
解压后:
./configure --prefix=/usr/local/keepalived --enable-snmp && make && make install
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived/
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/ #没有这个keepalived启动后,不会检测到real server的vip
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived/
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/ #没有这个keepalived启动后,不会检测到real server的vip
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/ #没有这个keepalived无法启动,报错说找不到命令
启动:
/etc/init.d/keepalived start
4.1 /etc/keepalived/keepalived.conf配置如下
! Configuration File for keepalived
global_defs {
# notification_email {
# [email protected]
# [email protected]
# [email protected]
# }
# notification_email_from [email protected]
# smtp_server 192.168.200.1
# smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_sync_group LVS {
group {
VI_1
}
}
vrrp_instance VI_1 {
state MASTER
interface eth0
lvs_sync_daemon_interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress { #这个是vip
172.10.2.89
}
global_defs {
# notification_email {
# [email protected]
# [email protected]
# [email protected]
# }
# notification_email_from [email protected]
# smtp_server 192.168.200.1
# smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_sync_group LVS {
group {
VI_1
}
}
vrrp_instance VI_1 {
state MASTER
interface eth0
lvs_sync_daemon_interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress { #这个是vip
172.10.2.89
}
}
virtual_server 172.10.2.89 80 {
delay_loop 6
lb_algo rr
#lb_kind NAT
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
delay_loop 6
lb_algo rr
#lb_kind NAT
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 172.10.2.170 80 { #这个nginx
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_prot 80
}
}
real_server 172.10.2.206 80 { #这个也是nginx
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_prot 80
}
}
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_prot 80
}
}
real_server 172.10.2.206 80 { #这个也是nginx
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_prot 80
}
}
}
五、在两台nginx上放入脚本,(/etc/init.d/lvs_start.sh)内容如下:
#! /bin/bash
SYN_VIP=172.10.2.201
/etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $SYN_VIP netmask 255.255.255.255 broadcast $SYN_VIP
/sbin/route add -host $SYN_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start ok"
;;
stop)
ifconfig lo:0 down
route del $SYN_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stop"
;;
*)
echo "USAGE: $0 {start|stop}"
exit 1
SYN_VIP=172.10.2.201
/etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $SYN_VIP netmask 255.255.255.255 broadcast $SYN_VIP
/sbin/route add -host $SYN_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start ok"
;;
stop)
ifconfig lo:0 down
route del $SYN_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stop"
;;
*)
echo "USAGE: $0 {start|stop}"
exit 1
esac
配置权限:chmod +x /etc/init.d/lvs_start.sh
开启:/etc/init.d/lvs_start.sh start 会提示functions权限不够
加权限chmod +x /etc/rc.d/init.d/functions
/etc/init.d/lvs_start.sh start
再ifconfig看看
说明vip配置ok了。两个nginx机子都使用相同的这个脚本
六、访问测试:
在lvs keepd机子上输入命令
ip add #命令查看real server的vip是否已经被绑定到keepalived的MASTER或者BACKUP上
inet 172.10.2.201/32 scope global eth0 #说明成功绑定
ipvsadm命令检测nginx是否正常工作,
都已经检测到两台nginx在正常工作,
把170和206两台nginx开启,
接下来要清一下,由于我测试时访问ip没搞对,老访问不了,
在nginx上有配置的vip(172.10.2.201),这个ip是对外的,直接对应用户,我们在lvs_keepd机子上有绑定这个vip,那么当用户访问这个ip时,lvs_keepd会接收这个请求然后转发给170和206其中某一台机子,这里lvs也会实现一个小小的负载,
那本地windows的host配置是
172.10.2.201 www.b.com #这样访问这个域名时,直接到了lvs,再转发给了nginx,nginx再去后端apache获取数据
测试:当访问www.b.com/test235.html, lvs收到后转发给了ngx206来接收了请求,
当我把ngx206的nginx服务给关了,再次请求,lvs收到后转发给了ngx170,
当把ngx206开启时,在lvs_keepd上ipvsadm一下,会看到206重新加入进来。
这里具体怎么看,看访问日志。
#这里有一些是参照网上的一例子,但小兵新测过,可行。
car-eye 流媒体开源平台:www.car-eye.cn 有关车载流媒体和流媒体服务器开源技术咨询加群