Lvs别样的自动部署监控shell脚本
l 脚本功能:
l 实验环境图:
l 具体脚本:
l 结果验证:
l 参考资料:
先申明,本文现在已经在我公司的测试环境和生产测试环境使用。正式环境请用keepalived+lvs.
安装ipvsadm不多说了,先说说脚本的功能,脚本分为redirect server 端和realserver 端,脚本分别为 lvs_redirector.sh 和realserver.sh脚本。另外加一个监控脚本lvs_monitor.sh(此脚本来源网友,做了一点修改,算是 取之于网络,共享给网络吧)。
脚本功能:执行 lvs_redirector.sh nat|dr|tun|stop,中的一个选项可以启动或关闭相应的lvs模式,并调用lvs_monitor.sh 监控realserver。当realserver故障,或者重新启动时,自动删除,添加相应的realserver.当realserver 全故障时,自动添加redirector server 本地127.0.0.1的web页面的故障提示。当realserver只要有一台恢复时,自动添加相应的realserver,并删除127.0.0.1。
实验环境图:Lvs具体原理可以看我的博客: 。
本文脚本的使用如下图的场景:
具体脚本:
lvs_redirector.sh 脚本如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 | #!/bin/bash # blog:http://qiaomiao.blog.51cto.com # # 20151223 v2.8 20160115 # home:http://www.qmlab.cn # # description:this script is use start lvs (nat/dr/tun),and stop lvs. #define the variable NAT_VIP= "172.16.8.11" DR_VIP= "10.0.8.20" TUN_VIP= "10.0.8.20" DIP= "10.0.8.11" RIP1= "10.0.8.21" RIP2= "10.0.8.22" RNET= "10.0.8.0/24" DPORT=80 RPORT1=80 RPORT2=80 GATEWAY01=172.16.8.254 #做nat模式时,redirector 服务器的外网卡的路由器网关地址。 GATEWAY02=10.0.8.254 #路由器网关地址,做dr 和 tun 模式时用到。 #两个GATEWAY,请根据具体网络环境做取舍。 logfile= /tmp/myself_log/lvs .log #create log dir mkdir -p /tmp/myself_log # 在这个目录可以看到启动lvs日志。 #general funiction lvs_monitor(){ if [ $1 = stop ]; then PS=` ps -ef | grep lvs_monitor.sh | awk '{printf $2" "}' ` for X in ` echo $PS` do kill -9 $X done else sh . /lvs_monitor .sh & fi } # LVS/NAT mode # start function nat_start { nat_stop VIP=$NAT_VIP echo 1 > /proc/sys/net/ipv4/ip_forward # start NAT mode ifconfig eth1 $VIP netmask 255.255.255.0 up route del -net 0.0.0.0 route add -net 0.0.0.0 netmask 0.0.0.0 gw $GATEWAY01 #ifconfig eth0:0 $VIP broadcast $VIP netmask 255.255.255.0 up ipvsadm -C ipvsadm -A -t $VIP:$DPORT -s wlc ipvsadm -a -t $VIP:$DPORT -r $RIP1:$RPORT1 -m -w 1 ipvsadm -a -t $VIP:$DPORT -r $RIP2:$RPORT2 -m -w 2 ipvsadm - ln ipvsadm -lnc } # stop function nat_stop { VIP=$NAT_VIP echo 0 > /proc/sys/net/ipv4/ip_forward # stop NAT mode ifconfig eth1 down route del -net 0.0.0.0 route add -net 0.0.0.0 netmask 0.0.0.0 gw $GATEWAY02 > /dev/null 2>&1 ipvsadm -C } # LVS/dricert routing mode # start function dr_start { dr_stop VIP=$DR_VIP echo 1 > /proc/sys/net/ipv4/ip_forward ifconfig eth0:0 $VIP broadcast $VIP netmask 255.255.255.255 up route add -host $VIP dev eth0:0 # start DR mode ipvsadm -C ipvsadm -A -t $VIP:$DPORT -s wlc ipvsadm -a -t $VIP:$DPORT -r $RIP1 -g -w 1 ipvsadm -a -t $VIP:$DPORT -r $RIP2 -g -w 2 ipvsadm - ln ipvsadm -lnc } # stop function dr_stop { VIP=$DR_VIP echo 0 > /proc/sys/net/ipv4/ip_forward ifconfig eth0:0 $VIP broadcast $VIP netmask 255.255.255.255 down route del -host $VIP dev eth0:0 > /dev/null 2>&1 # stop DR mode ipvsadm -C } # LVS/tunneling mode # start function tun_start { tun_stop VIP=$TUN_VIP # set interface #echo 1 >/proc/sys/net/ipv4/ip_forward ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up route add -host $VIP dev tunl0 # start tun mode ipvsadm -C ipvsadm -A -t $VIP:$DPORT -s wlc ipvsadm -a -t $VIP:$DPORT -r $RIP1 -i -w 1 ipvsadm -a -t $VIP:$DPORT -r $RIP2 -i -w 2 ipvsadm - ln ipvsadm -lnc } # stop function tun_stop { VIP=$TUN_VIP # set interface #echo 1 >/proc/sys/net/ipv4/ip_forward ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 down ip addr flush tunl0 route del -host $VIP dev tunl0 > /dev/null 2>&1 # stop tun mode ipvsadm -C } service iptables stop case "$1" in nat) echo -e "\n `date +%F" "%H:%M:%S` nat模式启动...." > $logfile tun_stop dr_stop nat_start lvs_monitor $1 ;; dr) echo -e "\n `date +%F" "%H:%M:%S` dr模式启动...." > $logfile tun_stop nat_stop dr_start lvs_monitor $1 ;; tun) echo -e " \n `date +%F" "%H:%M:%S ` tun模式启动...." > $logfile dr_stop nat_stop tun_start lvs_monitor $1 ;; stop) echo -e "\n `date +%F" "%H:%M:%S` 关闭lvs...." > $logfile dr_stop nat_stop tun_stop lvs_monitor $1 ;; *) echo $ "Usage: $0 {nat|dr|tun|stop}" ;; esac |
lvs_realserver.sh 脚本如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 | #!/bin/bash # write by lijing QQ 858080796, # blog:http://qiaomiao.blog.51cto.com # 20151223 v2.8 20160115 # home:http://www.qmlab.cn # # description:this script is use start lvs (nat/dr/tun). #define the variable #NAT_VIP="10.0.8.20" DR_VIP= "10.0.8.20" TUN_VIP= "10.0.8.20" DIP= "10.0.8.11" RIP1= "10.0.8.21" RIP2= "10.0.8.22" RNET= "10.0.8.0/24" DPORT=80 RPORT1=80 RPORT2=80 GATEWAY= "10.0.8.254" # # LVS/NAT mode # start function nat_start { nat_stop # set realserver gateway route del -net 0.0.0.0 route add -net 0.0.0.0 netmask 0.0.0.0 gw $DIP } # stop function nat_stop { # set realserver gateway route del -net 0.0.0.0 route add -net 0.0.0.0 netmask 0.0.0.0 gw $GATEWAY } # LVS/dricert routing mode # start function dr_start { dr_stop VIP=$DR_VIP # set interface ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up route add -host $VIP dev lo:0 # set realserver echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce } # stop function dr_stop { VIP=$DR_VIP # set interface ifconfig lo:0 down route del -host $VIP dev lo:0 > /dev/null 2>&1 # set realserver echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce } # LVS/tunneling mode # start function tun_start { tun_stop VIP=$TUN_VIP # set interface ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up route add -host $VIP dev tunl0 # set realserver echo 1 > /proc/sys/net/ipv4/conf/tunl0/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/tunl0/arp_announce echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/tunl0/rp_filter echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter } # stop function tun_stop { VIP=$TUN_VIP # set interface ip addr flush tunl0 ifconfig tunl0 down route del -host $VIP dev tunl0 > /dev/null 2>&1 # set realserver echo 0 > /proc/sys/net/ipv4/conf/tunl0/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/tunl0/arp_announce echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/tunl0/rp_filter echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter } service iptables stop case "$1" in nat) tun_stop dr_stop nat_start ;; dr) tun_stop nat_stop dr_start ;; tun) nat_stop dr_stop tun_start ;; stop) nat_stop dr_stop tun_stop ;; *) echo $ "Usage: $0 {nat|dr|tun|stop}" ;; esac |
lvs_monitor.sh 脚本如下:(注意要放在lvs_redirector.sh在同一个目录下)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | #!/bin/bash # write by http://small.blog.51cto.com/259970/1728082 # 感谢 网友 super_color rs=( "10.0.8.21" "10.0.8.22" ) vip= "10.0.8.20" port=80 logfile= "/tmp/myself_log/lvs_monitor.log" function check_alldown { #有一个rs主机能访问,就说明不是全部掉了 #检查到一个rs主机存活就退出检查 #如果全部rs不能访问,说明主机全掉了 for www in ` echo ${rs[*]}` do curl --connect-timeout 1 http: // $www &> /dev/null if [ $? - eq 0 ] then echo 0 exit 0 fi done echo 100 } function lvs_add { ipvsadm -a -t $vip:$port -r $1 echo "add rs host:$1 to lvs" } function lvs_rm { ipvsadm -d -t $vip:$port -r $1 echo "remove rs host:$1 to lvs" } function lvs_local { #如果全部rs主机掉线,并且lvs中没有127.0.0.1就添加它 #如果可以访问一个rs主机,并且lvs中有127.0.0.1就删除它 all_down=`check_alldown` rip=$(ipvsadm -L -n | gawk '/127.0.0.1/' ) if [ $all_down - eq 100 ] then if [ "$rip" = "" ] then echo "`date +%F:%H-%M-%S` all rs host is down!" >> $logfile lvs_add "127.0.0.1" fi else if [ $all_down - eq 0 ] && [ ! "$rip" = "" ] then echo "`date +%F:%H-%M-%S` one rs host is up,remove local rs host!" >> $logfile lvs_rm "127.0.0.1" fi fi } function lvs_rs { #如果可以访问一个rs主机,并且lvs中没有它就添加它 #如果不能访问一个rs主机,并且lvs中有它就删除它 lvs_local for www in ` echo ${rs[*]}` do rip=$(ipvsadm -L -n | gawk "/$www/" ) curl --connect-timeout 1 http: // $www &> /dev/null if [ $? - eq 0 ] then if [ "$rip" = "" ] then echo "`date +%F:%H-%M-%S` rs host:$www is up!" >> $logfile lvs_add "$www" fi else if [ ! "$rip" = "" ] then echo "`date +%F:%H-%M-%S` rs host:$www is down!" >> $logfile lvs_rm "$www" fi fi done } function lvs_monitor { while true do # echo "check lvs rs health!" lvs_rs sleep 1 done } lvs_monitor |
结果验证:在验证结果之前,要保证你的路由器的端口映射是正确,且生效的,上面图中:
当外网客户端192.168.20.200访问时,nat模式路由器192.168.20.14映射到172.16.8.11这个IP,
dr和 tun模式映射到 10.0.8.20这个IP。
验证方法:先测试直接内网访问两台realserver web是不是正常,以及redirector server 的本地127.0.0.1 web 是不是正常,再测试访问192.168.20.14,当其中一台故障时是不是还可以访问,到全故障时,有没有切的本地127.0.0.1(故障提示页)的web,当其中只要有一台恢复时,会不会启动添加启用,并删除127.0.0.1的web.