Keepalived无法绑定VIP故障排查经历

308次阅读

共计 16949 个字符，预计需要花费 43 分钟才能阅读完成。

一故障描述
我在台湾合作方给定的两台虚拟机上部署 HAProxy+Keepalived 负载均衡高可用方案。在配置完 Keepalived 后，重新启动 Keepalived，Keepalived 没有绑定 VIP。

Keepalived 执行程序路径为 /data/app_platform/keepalived/sbin/keepalived
配置文件路径为 /data/app_platform/keepalived/conf/keepalived.conf
Keepalived 的启动脚本为 /etc/init.d/keepalived

keepalived.conf 的内容
LB1 Master
! Configuration File for keepalived

global_defs {
notification_email {
admin@example.com
}
notification_email_from lb1@example.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LB1_MASTER
}

vrrp_script chk_haproxy {
script “killall -0 haproxy”
interval 2
weight 2
}

vrrp_instance VI_1 {
state MASTER
interface eth1
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {

10.1.1.200/24 brd 10.1.1.255 dev eth1 label eth1:vip

}

track_script {
chk_haproxy
}
}

重新启动 Keepalived 查看日志
Mar 3 18:09:00 cv00300005248-1 Keepalived[20138]: Stopping Keepalived v1.2.15 (02/28,2015)
Mar 3 18:09:00 cv00300005248-1 Keepalived[20259]: Starting Keepalived v1.2.15 (02/28,2015)
Mar 3 18:09:00 cv00300005248-1 Keepalived[20260]: Starting Healthcheck child process, pid=20261
Mar 3 18:09:00 cv00300005248-1 Keepalived[20260]: Starting VRRP child process, pid=20262
Mar 3 18:09:00 cv00300005248-1 Keepalived_vrrp[20262]: Registering Kernel netlink reflector
Mar 3 18:09:00 cv00300005248-1 Keepalived_vrrp[20262]: Registering Kernel netlink command channel
Mar 3 18:09:00 cv00300005248-1 Keepalived_vrrp[20262]: Registering gratuitous ARP shared channel
Mar 3 18:09:00 cv00300005248-1 Keepalived_healthcheckers[20261]: Registering Kernel netlink reflector
Mar 3 18:09:00 cv00300005248-1 Keepalived_healthcheckers[20261]: Registering Kernel netlink command channel
Mar 3 18:09:00 cv00300005248-1 Keepalived_healthcheckers[20261]: Configuration is using : 3924 Bytes
Mar 3 18:09:00 cv00300005248-1 Keepalived_healthcheckers[20261]: Using LinkWatch kernel netlink reflector…
Mar 3 18:09:00 cv00300005248-1 Keepalived_vrrp[20262]: Configuration is using : 55712 Bytes
Mar 3 18:09:00 cv00300005248-1 Keepalived_vrrp[20262]: Using LinkWatch kernel netlink reflector…
Mar 3 18:09:18 cv00300005248-1 kernel: __ratelimit: 1964 callbacks suppressed
Mar 3 18:09:18 cv00300005248-1 kernel: Neighbour table overflow.
Mar 3 18:09:18 cv00300005248-1 kernel: Neighbour table overflow.
Mar 3 18:09:18 cv00300005248-1 kernel: Neighbour table overflow.
Mar 3 18:09:18 cv00300005248-1 kernel: Neighbour table overflow.
Mar 3 18:09:18 cv00300005248-1 kernel: Neighbour table overflow.
Mar 3 18:09:18 cv00300005248-1 kernel: Neighbour table overflow.
Mar 3 18:09:18 cv00300005248-1 kernel: Neighbour table overflow.
Mar 3 18:09:18 cv00300005248-1 kernel: Neighbour table overflow.
Mar 3 18:09:18 cv00300005248-1 kernel: Neighbour table overflow.
Mar 3 18:09:18 cv00300005248-1 kernel: Neighbour table overflow.

查看 VIP 绑定情况
$ ifconfig eth1:vip
eth1:vip Link encap:Ethernet HWaddr 00:16:3E:F2:37:6B
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:13

没有 VIP 绑定

二排查过程

1）检查 VIP 的配置情况
向合作方确认提供的 VIP 的详细情况

IPADDR 10.1.1.200
NETMASK 255.255.255.0
GATEWAY 10.1.1.1
Brodcast 10.1.1.255

这里设置的是

10.1.1.200/24 brd 10.1.1.255 dev eth1 label eth1:vip

2）检查 iptables 和 selinux 的设置情况
$ sudo service iptables stop
$ sudo setenforce 0
setenforce: SELinux is disabled

如果非要开启 iptables 的话，需要作些设定

iptables -I INPUT -i eth1 -d 224.0.0.0/8 -j ACCEPT
service iptables save

keepalived 使用 224.0.0.18 作为 Master 和 Backup 健康检查的通信 IP

3）检查相关的内核参数
HAProxy+Keepalived 架构需要注意的内核参数有：

# Controls IP packet forwarding
net.ipv4.ip_forward = 1
开启 IP 转发功能

net.ipv4.ip_nonlocal_bind = 1
开启允许绑定非本机的 IP

如果使用 LVS 的 DR 或者 TUN 模式结合 Keepalived 需要在后端真实服务器上特别设置两个 arp 相关的参数。这里也设置好。
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

4）检查 VRRP 的设置情况
LB1 Master
state MASTER
interface eth1
virtual_router_id 51
priority 100

LB2 Backup
state BACKUP
interface eth1
virtual_router_id 51
priority 99

Master 和 Backup 的 virtual_router_id 需要一样，priority 需要不一样，数字越大，优先级越高

5）怀疑是编译安装 Keepalived 版本出现了问题
重新下载并编译 2.1.13 的版本，并重新启动 keepalived，VIP 仍然没有被绑定。

线上有个平台的 keepalived 是通过 yum 安装的，于是打算先用 yum 安装 keepalived 后将配置文件复制过去看看是否可以绑定 VIP

rpm -ivh http://ftp.linux.ncsu.edu/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
yum -y install keepalived

cp /data/app_platform/keepalived/conf/keepalived.conf /etc/keepalived/keepalived.conf

重新启动 keepalived

然后查看日志
Mar 4 16:42:46 xxxxx Keepalived_healthcheckers[17332]: Registering Kernel netlink reflector
Mar 4 16:42:46 xxxxx Keepalived_healthcheckers[17332]: Registering Kernel netlink command channel
Mar 4 16:42:46 xxxxx Keepalived_vrrp[17333]: Opening file ‘/etc/keepalived/keepalived.conf’.
Mar 4 16:42:46 xxxxx Keepalived_vrrp[17333]: Configuration is using : 65250 Bytes
Mar 4 16:42:46 xxxxx Keepalived_vrrp[17333]: Using LinkWatch kernel netlink reflector…
Mar 4 16:42:46 xxxxx Keepalived_vrrp[17333]: VRRP sockpool: [ifindex(3), proto(112), unicast(0), fd(10,11)]
Mar 4 16:42:46 xxxxx Keepalived_healthcheckers[17332]: Opening file ‘/etc/keepalived/keepalived.conf’.
Mar 4 16:42:46 xxxxx Keepalived_healthcheckers[17332]: Configuration is using : 7557 Bytes
Mar 4 16:42:46 xxxxx Keepalived_healthcheckers[17332]: Using LinkWatch kernel netlink reflector…
Mar 4 16:42:46 xxxxx Keepalived_vrrp[17333]: VRRP_Script(chk_haproxy) succeeded
Mar 4 16:42:47 xxxxx Keepalived_vrrp[17333]: VRRP_Instance(VI_1) Transition to MASTER STATE
Mar 4 16:42:48 xxxxx Keepalived_vrrp[17333]: VRRP_Instance(VI_1) Entering MASTER STATE
Mar 4 16:42:48 xxxxx Keepalived_vrrp[17333]: VRRP_Instance(VI_1) setting protocol VIPs.
Mar 4 16:42:48 xxxxx Keepalived_vrrp[17333]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 10.1.1.200
Mar 4 16:42:48 xxxxx Keepalived_healthcheckers[17332]: Netlink reflector reports IP 10.1.1.200 added
Mar 4 16:42:53 xxxxx Keepalived_vrrp[17333]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 10.1.1.200

再查看 IP 绑定情况
$ ifconfig eth1:vip
eth1:vip Link encap:Ethernet HWaddr 00:16:3E:F2:37:6B
inet addr:10.1.1.200 Bcast:10.1.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:13

再通过 yum 将 keepalived 卸载掉
yum remove keepalived
恢复到原来的启动脚本 /etc/init.d/keepalived
重新启动 keepalived 后还是无法绑定 VIP

怀疑是 keepalived 启动脚本 /etc/init.d/keepalived 的问题
检查 /etc/init.d/keepalived
# Source function library.
. /etc/rc.d/init.d/functions

exec=”/data/app_platform/keepalived/sbin/keepalived”
prog=”keepalived”
config=”/data/app_platform/keepalived/conf/keepalived.conf”

[-e /etc/sysconfig/$prog] && . /etc/sysconfig/$prog

lockfile=/var/lock/subsys/keepalived

start() {
[-x $exec] || exit 5
[-e $config] || exit 6
echo -n $”Starting $prog: ”
daemon $exec $KEEPALIVED_OPTIONS
retval=$?
echo
[$retval -eq 0] && touch $lockfile
return $retval
}

关键是这一行
daemon $exec $KEEPALIVED_OPTIONS

由于没有复制 /etc/sysconfig/keepalived, 所以将直接执行 damon /data/app_platform/keepalived/sbin/keepalived
由于 keepalived 默认使用的是 /etc/keepalived/keepalived.conf 作为配置文件，而这里指定了不同的配置文件，所以要修改成为
daemon $exec -D -f $config

重新启动 keepalived，查看日志和 VIP 绑定情况
$ ifconfig eth1:vip
eth1:vip Link encap:Ethernet HWaddr 00:16:3E:F2:37:6B
inet addr:10.1.1.200 Bcast:10.1.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:13

6）将 LB2 Backup 的 keepalived 启动脚本也修改一下，观察 VIP 接管情况
查看 LB1 Master

$ ifconfig eth1:vip
eth1:vip Link encap:Ethernet HWaddr 00:16:3E:F2:37:6B
inet addr:10.1.1.200 Bcast:10.1.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:13

查看 LB2 Backup
$ ifconfig eth1:vip
eth1:vip Link encap:Ethernet HWaddr 00:16:3E:F2:37:6B
inet addr:10.1.1.200 Bcast:10.1.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:13

问题出现了，LB1 Master 和 LB2 Backup 都绑定了 VIP 10.1.1.200，这是不正常的！！！！

在 LB1 和 LB2 上登录 10.1.1.200 看看

[lb1 ~]$ ssh 10.1.1.200
Last login: Wed Mar 4 17:31:33 2015 from 10.1.1.200
[lb1 ~]$

[lb2 ~]$ ssh 10.1.1.200
Last login: Wed Mar 4 17:54:57 2015 from 101.95.153.246
[b2 ~]$

在 LB1 上停掉 keepalived，ping 下 10.1.1.200 这个 IP，发现无法 ping 通
在 LB2 上停掉 keepalived，ping 下 10.1.1.200 这个 IP，发现也无法 ping 通

然后开启 LB1 上的 keepalived，LB1 上可以 ping 通 10.1.1.200,LB2 上不行

开启 LB2 上的 keepalived，LB2 上可以 ping 通 10.1.1.200

由此得出，LB1 和 LB2 各自都将 VIP 10.1.1.200 绑定到本机的 eth1 网卡上。两台主机并没有 VRRP 通信，没有 VRRP 的优先级比较。

7)排查影响 VRRP 通信的原因
重新启动 LB1 Master 的 Keepalived 查看日志
Mar 5 15:45:36 gintama-taiwan-lb1 Keepalived_vrrp[32303]: Configuration is using : 65410 Bytes
Mar 5 15:45:36 gintama-taiwan-lb1 Keepalived_vrrp[32303]: Using LinkWatch kernel netlink reflector…
Mar 5 15:45:36 gintama-taiwan-lb1 Keepalived_vrrp[32303]: VRRP sockpool: [ifindex(3), proto(112), unicast(0), fd(10,11)]
Mar 5 15:45:36 gintama-taiwan-lb1 Keepalived_vrrp[32303]: VRRP_Script(chk_haproxy) succeeded
Mar 5 15:45:37 gintama-taiwan-lb1 Keepalived_vrrp[32303]: VRRP_Instance(VI_1) Transition to MASTER STATE
Mar 5 15:45:38 gintama-taiwan-lb1 Keepalived_vrrp[32303]: VRRP_Instance(VI_1) Entering MASTER STATE
Mar 5 15:45:38 gintama-taiwan-lb1 Keepalived_vrrp[32303]: VRRP_Instance(VI_1) setting protocol VIPs.
Mar 5 15:45:38 gintama-taiwan-lb1 Keepalived_vrrp[32303]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 10.1.1.200
Mar 5 15:45:38 gintama-taiwan-lb1 Keepalived_healthcheckers[32302]: Netlink reflector reports IP 10.1.1.200 added
Mar 5 15:45:43 gintama-taiwan-lb1 Keepalived_vrrp[32303]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 10.1.1.200

发现 LB1 Master 上的 Keepalived 直接进入 Master 状态，然后接管 VIP

再重新启动 LB2 Backup 上的 Keepalived，查看日志
Mar 5 15:47:42 gintama-taiwan-lb2 Keepalived_vrrp[30619]: Configuration is using : 65408 Bytes
Mar 5 15:47:42 gintama-taiwan-lb2 Keepalived_vrrp[30619]: Using LinkWatch kernel netlink reflector…
Mar 5 15:47:42 gintama-taiwan-lb2 Keepalived_vrrp[30619]: VRRP_Instance(VI_1) Entering BACKUP STATE
Mar 5 15:47:42 gintama-taiwan-lb2 Keepalived_vrrp[30619]: VRRP sockpool: [ifindex(3), proto(112), unicast(0), fd(10,11)]
Mar 5 15:47:46 gintama-taiwan-lb2 Keepalived_vrrp[30619]: VRRP_Instance(VI_1) Transition to MASTER STATE
Mar 5 15:47:47 gintama-taiwan-lb2 Keepalived_vrrp[30619]: VRRP_Instance(VI_1) Entering MASTER STATE
Mar 5 15:47:47 gintama-taiwan-lb2 Keepalived_vrrp[30619]: VRRP_Instance(VI_1) setting protocol VIPs.
Mar 5 15:47:47 gintama-taiwan-lb2 Keepalived_vrrp[30619]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 10.1.1.200
Mar 5 15:47:47 gintama-taiwan-lb2 Keepalived_healthcheckers[30618]: Netlink reflector reports IP 10.1.1.200 added
Mar 5 15:47:52 gintama-taiwan-lb2 Keepalived_vrrp[30619]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 10.1.1.200

可以看到 LB2 上的 Keepalived 先进入 BACKUP 状态，然后又转为 MASTER 状态，然后接管 VIP

这样就说明 VRRP 组播有问题。

既然 VRRP 组播有问题，就尝试使用单播发送 VRRP 报文。修改 LB1 和 LB2 的配置
LB1
添加以下配置
unicast_src_ip 10.1.1.12
unicast_peer {
10.1.1.17
}

LB2

添加以下配置
unicast_src_ip 10.1.1.17
unicast_peer {
10.1.1.12
}

unicast_src_ip 表示发送 VRRP 单播报文使用的源 IP 地址
unicast_peer 表示对端接收 VRRP 单播报文的 IP 地址

然后各自重新加载 keepalived，观察日志

LB1
Mar 5 16:13:35 gintama-taiwan-lb1 Keepalived_vrrp[2551]: VRRP_Instance(VI_1) setting protocol VIPs.
Mar 5 16:13:35 gintama-taiwan-lb1 Keepalived_vrrp[2551]: VRRP_Script(chk_haproxy) considered successful on reload
Mar 5 16:13:35 gintama-taiwan-lb1 Keepalived_vrrp[2551]: Configuration is using : 65579 Bytes
Mar 5 16:13:35 gintama-taiwan-lb1 Keepalived_vrrp[2551]: Using LinkWatch kernel netlink reflector…
Mar 5 16:13:35 gintama-taiwan-lb1 Keepalived_vrrp[2551]: VRRP sockpool: [ifindex(3), proto(112), unicast(1), fd(10,11)]
Mar 5 16:13:36 gintama-taiwan-lb1 Keepalived_vrrp[2551]: VRRP_Instance(VI_1) Transition to MASTER STATE
Mar 5 16:13:48 gintama-taiwan-lb1 Keepalived_vrrp[2551]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election
Mar 5 16:13:48 gintama-taiwan-lb1 Keepalived_vrrp[2551]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 10.1.1.200
Mar 5 16:13:48 gintama-taiwan-lb1 Keepalived_vrrp[2551]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election
Mar 5 16:13:48 gintama-taiwan-lb1 Keepalived_vrrp[2551]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 10.1.1.200

LB2
Mar 5 16:13:48 gintama-taiwan-lb2 Keepalived_vrrp[453]: VRRP_Instance(VI_1) Received higher prio advert
Mar 5 16:13:48 gintama-taiwan-lb2 Keepalived_vrrp[453]: VRRP_Instance(VI_1) Entering BACKUP STATE
Mar 5 16:13:48 gintama-taiwan-lb2 Keepalived_vrrp[453]: VRRP_Instance(VI_1) removing protocol VIPs.
Mar 5 16:13:48 gintama-taiwan-lb2 Keepalived_healthcheckers[452]: Netlink reflector reports IP 10.1.1.200 removed

查看 VIP 绑定情况，发现 LB2 上的 VIP 已经移除

在 LB1 上 LB2 上执行 ping 10.1.1.200 这个 VIP
[lb1 ~]$ ping -c 5 10.1.1.200
PING 10.1.1.200 (10.1.1.200) 56(84) bytes of data.
64 bytes from 10.1.1.200: icmp_seq=1 ttl=64 time=0.028 ms
64 bytes from 10.1.1.200: icmp_seq=2 ttl=64 time=0.020 ms
64 bytes from 10.1.1.200: icmp_seq=3 ttl=64 time=0.020 ms
64 bytes from 10.1.1.200: icmp_seq=4 ttl=64 time=0.021 ms
64 bytes from 10.1.1.200: icmp_seq=5 ttl=64 time=0.027 ms

— 10.1.1.200 ping statistics —
5 packets transmitted, 5 received, 0% packet loss, time 3999ms
rtt min/avg/max/mdev = 0.020/0.023/0.028/0.004 ms

[lb2 ~]$ ping -c 5 10.1.1.200
PING 10.1.1.200 (10.1.1.200) 56(84) bytes of data.

— 10.1.1.200 ping statistics —
5 packets transmitted, 0 received, 100% packet loss, time 14000ms

当 LB1 接管 VIP 的时候 LB2 居然无法 ping 通 VIP，同样将 LB1 的 Keepalived 停掉，LB2 可以接管 VIP，但是在 LB1 上无法 ping 通这个 VIP

在 LB1 和 LB2 上进行抓包

lb1 ~]$ sudo tcpdump -vvv -i eth1 host 10.1.1.17
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
16:46:04.827357 IP (tos 0xc0, ttl 255, id 328, offset 0, flags [none], proto VRRP (112), length 40)
10.1.1.12 > 10.1.1.17: VRRPv2, Advertisement, vrid 51, prio 102, authtype simple, intvl 1s, length 20, addrs: 10.1.1.200 auth “1111^@^@^@^@”
16:46:05.827459 IP (tos 0xc0, ttl 255, id 329, offset 0, flags [none], proto VRRP (112), length 40)
10.1.1.12 > 10.1.1.17: VRRPv2, Advertisement, vrid 51, prio 102, authtype simple, intvl 1s, length 20, addrs: 10.1.1.200 auth “1111^@^@^@^@”
16:46:06.828234 IP (tos 0xc0, ttl 255, id 330, offset 0, flags [none], proto VRRP (112), length 40)
10.1.1.12 > 10.1.1.17: VRRPv2, Advertisement, vrid 51, prio 102, authtype simple, intvl 1s, length 20, addrs: 10.1.1.200 auth “1111^@^@^@^@”
16:46:07.828338 IP (tos 0xc0, ttl 255, id 331, offset 0, flags [none], proto VRRP (112), length 40)
10.1.1.12 > 10.1.1.17: VRRPv2, Advertisement, vrid 51, prio 102, authtype simple, intvl 1s, length 20, addrs: 10.1.1.200 auth “1111^@^@^@^@”

lb2 ~]$ sudo tcpdump -vvv -i eth1 host 10.1.1.12
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
16:48:07.000029 IP (tos 0xc0, ttl 255, id 450, offset 0, flags [none], proto VRRP (112), length 40)
10.1.1.12 > 10.1.1.17: VRRPv2, Advertisement, vrid 51, prio 102, authtype simple, intvl 1s, length 20, addrs: 10.1.1.200 auth “1111^@^@^@^@”
16:48:07.999539 IP (tos 0xc0, ttl 255, id 451, offset 0, flags [none], proto VRRP (112), length 40)
10.1.1.12 > 10.1.1.17: VRRPv2, Advertisement, vrid 51, prio 102, authtype simple, intvl 1s, length 20, addrs: 10.1.1.200 auth “1111^@^@^@^@”
16:48:08.999252 IP (tos 0xc0, ttl 255, id 452, offset 0, flags [none], proto VRRP (112), length 40)
10.1.1.12 > 10.1.1.17: VRRPv2, Advertisement, vrid 51, prio 102, authtype simple, intvl 1s, length 20, addrs: 10.1.1.200 auth “1111^@^@^@^@”
16:48:09.999560 IP (tos 0xc0, ttl 255, id 453, offset 0, flags [none], proto VRRP (112), length 40)
10.1.1.12 > 10.1.1.17: VRRPv2, Advertisement, vrid 51, prio 102, authtype simple, intvl 1s, length 20, addrs: 10.1.1.200 auth “1111^@^@^@^@”

在 LB1 和 LB2 所在物理机上的其他虚拟机进行 VIP ping 测试，同样只能是 LB1 上绑定的 VIP 只能是 LB1 所在的物理机上的虚拟机可以 ping 通，LB2 所在的物理机上的虚拟机无法 ping 通，反之也是一样

有同行建议说 VRRP 和 DHCP 也有关系，经过查看对方提供的 VM 的 IP 地址居然是 DHCP 分配的，但是经过测试，VRRP 和 DHCP 没有关系。线上环境最好不要使用 DHCP 来获取 IP 地址。

8）请对方技术人员配合检查 VIP 无法 ping 通的问题
最终查明对方的内网居然使用的虚拟网络，网关是没有实际作用的。所以部分虚拟机无法通过 10.1.1.1 这个网关去访问 VIP。

让对方虚拟机提供方的技术人员到服务器调试 HAProxy+Keepalived，他们通过网络设置使得 10.1.1.200 这个 VIP 可以通过内网访问。但是当我测试时，发现当 HAProxy 挂掉后，Keepalived 无法作 VIP 的切换。

9）解决当 HAProxy 挂掉后，Keepalived 无法对 VIP 切换的问题。
经过反复测试，发现当 Keepalived 挂掉后，VIP 可以切换。但是当 HAProxy 挂掉后，VIP 无法切换。
仔细检查配置文件和查阅相关资料，最终确定是 Keepalived 的 weight 和 priority 两个参数的大小设置问题。

原来的配置文件中我设置 LB1 的 weight 为 2，priority 为 100。LB2 的 weight 为 2，priority 为 99

对方在调试的时候将 LB1 的 priority 更改为 160. 这样反复测试当 LB1 的 HAProxy 挂掉后，VIP 都无法迁移到 LB2 上。将 LB1 上的 priority 更改为 100 就可以了。

这里需要注意的是：
主 keepalived 的 priority 值与 vrrp_script 的 weight 值相减的数字小于备用 keepalived 的 priority 值即可！

vrrp_script 里的 script 返回值为 0 时认为检测成功，其它值都会当成检测失败

* weight 为正时，脚本检测成功时此 weight 会加到 priority 上，检测失败时不加。

主失败:
主 priority < 从 priority + weight 时会切换。

主成功：
主 priority + weight > 从 priority + weight 时，主依然为主

* weight 为负时，脚本检测成功时此 weight 不影响 priority，检测失败时 priority – abs(weight)

主失败:
主 priority – abs(weight) < 从 priority 时会切换主从

主成功:
主 priority > 从 priority 主依然为主。

最终的配置文件为：
! Configuration File for keepalived

global_defs {
notification_email {
admin@example.com
}
notification_email_from lb1@example.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LB1_MASTER
}

vrrp_script chk_haproxy {
script “killall -0 haproxy”
interval 2
weight 2
}

# 设置外网的 VIP
vrrp_instance eth0_VIP {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
unicast_src_ip 8.8.8.6 #使用 VRRP 的单播
unicast_peer {
8.8.8.7
}
virtual_ipaddress {
8.8.8.8/25 brd 8.8.8.255 dev eth0 label eth0:vip

}

track_script {
chk_haproxy
}
}

# 设置内网的 VIP
vrrp_instance eth1_VIP {
state MASTER
interface eth1
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
unicast_src_ip 10.1.1.12
unicast_peer {
10.1.1.17
}
virtual_ipaddress {
10.1.1.200/24 brd 10.1.1.255 dev eth1 label eth1:vip

}

track_script {
chk_haproxy
}
}

三排查总结
在配置 Keepalived 的时候，需要注意以下几点：

A. 内核开启 IP 转发和允许非本地 IP 绑定功能，如果是使用 LVS 的 DR 模式还需设置两个 arp 相关的参数。
B. 如果 Keepalived 所在网络不允许使用组播，可以使用 VRRP 单播
C. 需要注意主备的 weight 和 priority 的值，这两个值如果设置不合理可能会影响 VIP 的切换。
D. 如果使用的配置文件不是默认的配置文件，在启动 Keepalived 的时候需要使用 -f 参数指定配置文件。

CentOS 6.3 下 Haproxy+Keepalived+Apache 配置笔记 http://www.linuxidc.com/Linux/2013-06/85598.htm

Haproxy + KeepAlived 实现 WEB 群集 on CentOS 6 http://www.linuxidc.com/Linux/2012-03/55672.htm

Keepalived+Haproxy 配置高可用负载均衡 http://www.linuxidc.com/Linux/2012-03/56748.htm

Haproxy+Keepalived 构建高可用负载均衡 http://www.linuxidc.com/Linux/2012-03/55880.htm

CentOS 7 上配置 LVS + Keepalived + ipvsadm http://www.linuxidc.com/Linux/2014-11/109237.htm

Keepalived 高可用集群搭建 http://www.linuxidc.com/Linux/2014-09/106965.htm

Keepalived 的详细介绍：请点这里
Keepalived 的下载地址：请点这里

本文永久更新链接地址：http://www.linuxidc.com/Linux/2015-03/114981.htm

正文完

星哥说事-微信公众号