高可用集群技术之Keepalived实现LVS高可用并负载均衡Web服务

446次阅读

共计 7959 个字符，预计需要花费 20 分钟才能阅读完成。

Keepalived 概述:
Keepalived 的作用是检测服务器的状态，如果有一台 web 服务器死机，或工作出现故障，Keepalived 将检测到，并将有故障的服务器从系统中剔除，同时使用其他服务器代替该服务器的工作，当服务器工作正常后 Keepalived 自动将服务器加入到服务器群中，这些工作全部自动完成，不需要人工干涉，需要人工做的只是修复故障的服务器.

keepalived 实现 lvs 高可用并负载均衡 web 服务：
准备工作:
2 台 keepalived 节点为: node1.samlee.com node2.samlee.com
2 台 realserver 服务为: node3.samlee.com node4.samlee.com

1、配置 RSserver 过程如下 (node3.samlee.com\node4.samlee.com):
(1) 部署 web 服务(使用 nginx 配置)
# yum -y install nginx-1.10.0-1.el6.ngx.x86_64.rpm

(2)禁用防火墙及主机名修改
# service iptables stop
# chkconfig iptables off

(3)配置路由转发及转发模块配置
# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
# echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce
# echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore

– 添加路由条目
# ifconfig lo:0 172.16.100.99 netmask 255.255.255.255 broadcast 172.16.100.99 up
# route add -host 172.16.100.99 dev lo:0

2、配置 keepalived 服务器
(1) 安装 ipvsadm 负载均衡管理器

# yum -y install ipvsadm

(2)安装 keepalived
# yum -y install keepalived
# chkconfig keepalived on

(3)配置 keepalived 默认初始状态如下:(主节点:master 状态)
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from root@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL #物理路由 id，一般指定为本机的 hostname
}
vrrp_instance VI_1 {#vrrp 实例配置段
state MASTER #指定 keepalived 服务运行状态
interface eth0 #指定 keepalived 的心跳口
virtual_router_id 51 #指定虚拟路由 id（1~255），同一 vrrp 实例的主备 keepalived 必须配置为一样
priority 101 ## 指定起始优先级，优先级高的会成为 master
advert_int 1 #vrrp 通告的发送间隔
authentication {#配置通过密码认证
auth_type PASS
auth_pass keepalivedpass
}
virtual_ipaddress {
172.16.100.99 dev eth0 label eth0:0 #定义 VIP
}
}

(4)配置 keepalived 默认初始状态如下:(备节点:backuo 状态)
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
notification_email {
root@localhost
}
notification_email_from root@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}

vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass keepalivedpass
}
virtual_ipaddress {
172.16.100.99 dev eth0 label eth0:0
}
}

(5)调用周期性检测 down 文件是否存在来控制 keepalived 服务的主备切换
在 keepalived.conf 添加如下内容:
全局配置的段下添加如下:
vrrp_script chk_downfile {#服务状态检测脚本配置段
script “[[-f /etc/keepalived/down]] && exit 1 || exit 0” #通过周期性检测 down 文件是否存在来控制 keepalived 服务的主备切换
interval 1 #指定检测间隔为 1 秒
weight -2 #指定检测失败时，优先级减 2；检测的成功或失败是由 script 后面指定的命令或脚本执行返回的状态码决定的，0 表示成功，非 0 表示失败
}

在 vrrp_instance VI_1 ##vrrp 实例配置段添加如下:
track_script {
chk_downfile #通过指定上面定义监控脚本来监控服务状态，以完成主备切换
}

重启 keepalived 服务创建 down 文件测试:
# service keepalived restart
# touch /etc/keepalived/down

— 在 /etc/keepalived/ 下建立 down 文件可实现主备模式切换

(6)调用外部脚本检测 web(nginx)健康状态
建立 /etc/keepalived/chkngx.sh 脚本
#/bin/bash
#/etc/keepalived/chkngx.sh
#
if pkill -0 nginx>/dev/null; then
[`curl -sL -w %{http_code} http://localhost -o /dev/null` -eq 200 ] && exit 0
else
service nginx start
fi
exit $?

授予脚本执行权限:
# chmod +x /etc/keepalived/chkngx.sh

配置 keepalived.conf 调用外部脚本
全局配置的段下添加如下:
vrrp_script chkngx {
script “/etc/keepalived/chkngx.sh” #指定用于检测 nginx 服务的执行脚本路径
interval 1 #监测间隔
weight -2 #失败时，优先级减 2
fall 3 # 指定 nginx 检测脚本连续执行失败次数为 3，才进行 Failover
rise 3 # 指定 nginx 检测脚本连续执行成功次数为 3，才进行 Failback
}

在 vrrp_instance VI_1 ##vrrp 实例配置段添加如下:
track_script {
chkngx #通过指定上面定义监控脚本来监控服务状态，监控 nginx 监控状态
}

(7)使用通知脚本实现如何在 vrrp 事务发生时，发送警告邮件给指定的管理员？比如：主从节点故障时
建立脚本监控脚本如下:
# vim /etc/keepalived/notify.sh
#!/bin/bash
#

declare -a vips=(172.16.100.98 172.16.100.99)
contact=’root@localhost’
Usage() {
echo “Usage:`basename $0`{master|backup|fault}”
}

Notify() {
if [“$2” = “VI_1”]; then
vip=${vips[0]}
echo $vip
elif [“$2” = “VI_2”]; then
vip=${vips[1]}
echo $vip
fi
suject=”`hostname`’s state chaged to $1″
mailbody=”`date +%F–%H:%M:%S`: `hostname`’s state chage to $1,vip floating…”
echo $mailbody | mail -s “$subject” $contact
echo $mailbody >> /tmp/keepalived.log
}

case $1 in
master)
Notify master
exit 0
;;
backup)
Notify backup
exit 0
;;
fault)
Notify fault
exit 0
;;
*)
Usage
exit 1
;;
esac

授予脚本执行权限:
# chmod +x /etc/keepalived/notify.sh

配置 keepalived.conf 调用外部脚本
在 vrrp_instance VI_1 ##vrrp 实例配置段最后添加如下:
notify_master “/etc/keepalived/notify.sh master” #状态变为 master 时，触发的通知脚本
notify_backup “/etc/keepalived/notify.sh backup” #状态变为 backup 时，触发的通知脚本
notify_fault “/etc/keepalived/notify.sh fault” #状态变为 fault 时，触发的通知脚本
notify “/etc/keepalived/notify.sh” #当发生所有的状态改变时，会先触发对应的状态通知脚本后，再触发该脚本

(8)使用脚本根据 down 文件检测实现主从状态转换
# vim /etc/keepalived/convert_state.sh
#!/bin/bash
#

usage() {
echo “./`basename $0` {master|backup}”
exit 1
}

if [$# -eq 0]; then
usage
fi

if [[“$1” = “master”]]; then
[-f /etc/keepalived/down] && rm -f /etc/keepalived/down
elif [[“$1” = “backup”]]; then
[! -f /etc/keepalived/down] && touch /etc/keepalived/down
else
usage
fi
sleep 1

# chmod +x /etc/keepalived/convert_state.sh

执行脚本 convert_state 进行备节点转换:
# /etc/keepalived/convert_state.sh backup
# ls /etc/keepalived/ | grep down
down

执行脚本 convert_state 进行主节点转换:
# /etc/keepalived/convert_state.sh master

(9)LVS+keepalived 配置 ipvs 规则实现负载均衡高可用
配置如下:
virtual_server 172.16.100.99 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.0.0
# persistence_timeout 50
protocol TCP
sorry_server 127.0.0.1 80 #如果所有节点都 down 机，将转向本地 web 服务

real_server 172.16.100.8 80 {
weight 1
HTTP_GET {#web 健康状态检测
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.16.100.9 80 {
weight 1
HTTP_GET {#web 健康状态检测
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}

最后查询 ipvs 规则，如下所示:
# ipvsadm -L -n –stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 172.16.100.99:80 3 22 0 4657 0
-> 172.16.100.8:80 0 0 0 0 0
-> 172.16.100.9:80 0 0 0 0 0

以上为 keepalived+lvs+web 服务实现所有内容。

keepalived 双主模型配置案例如下:
master1 为:
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass keepalivedpass
}
virtual_ipaddress {
172.16.100.99 dev eth0 label eth0:0
}
track_script {
chk_downfile
}
track_script {
chkngx
}
notify_master “/etc/keepalived/notify.sh master”
notify_backup “/etc/keepalived/notify.sh backup”
notify_fault “/etc/keepalived/notify.sh fault”
notify “/etc/keepalived/notify.sh”
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass keepalivedpass
}
virtual_ipaddress {
172.16.100.98 dev eth0 label eth0:1
}
track_script {
chk_downfile
}
track_script {
chkngx
}
notify_master “/etc/keepalived/notify.sh master”
notify_backup “/etc/keepalived/notify.sh backup”
notify_fault “/etc/keepalived/notify.sh fault”
notify “/etc/keepalived/notify.sh”
}

master2 为:
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass keepalivedpass
}
virtual_ipaddress {
172.16.100.99 dev eth0 label eth0:0
}
track_script {
chk_downfile
chkngx
}
track_script {
chkngx
}
notify_master “/etc/keepalived/notify.sh master”
notify_backup “/etc/keepalived/notify.sh backup”
notify_fault “/etc/keepalived/notify.sh fault”
notify “/etc/keepalived/notify.sh”
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 52
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass keepalivedpass
}
virtual_ipaddress {
172.16.100.98 dev eth0 label eth0:1
}
track_script {
chk_downfile
}
track_script {
chkngx
}
notify_master “/etc/keepalived/notify.sh master”
notify_backup “/etc/keepalived/notify.sh backup”
notify_fault “/etc/keepalived/notify.sh fault”
notify “/etc/keepalived/notify.sh”
}