Load balancing with IPVS + Keepalived

IPVS implements load balancing for level transport inside Linux kernel. Running as frontend balancing the request from the clients to the backend servers using different methods of forwarding and load balance algorithms. Basically supports three methods of package forwarding:

• NAT
• Tunneling
• Direct Routing

Keepalived is a routing software, that provides a framework for load balancing and high availability. Keepalived provided high availability using the protocol VRRP to router failover and the load balancing is provided by LVS.

For this scenario I’ll use the NAT load balancing method, working with keepalived, and as health checker for the backend servers to check if one of the backends nodes fails keepalived  reports to the kernel to remove the node of the LVS topology.

Node director (IPVS + Keepalived):

• eth0: 192.168.1.54 (NAT)
• eth1: 172.16.1.1

Backend server1:

• eth0: 172.16.1.2

Backend server2:

• eth0: 172.16.1.3

Preparing the environment

– Enabling routing:

# vi /etc/sysctl.conf
net.ipv4.ip_forward=1
# sysctl -p

– Enabling NAT:

# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Installing and configuring keepalived

– Install keepalived:

#apt-get install keepalived

– Create a hash for the url that keepalived will check:

# genhash -s 172.16.1.2 -p 80 -u /index.html
MD5SUM = 01bc6b572ba171d4d3bd89abe9cb9a4c
# genhash -s 172.16.1.3 -p 80 -u /index.html
MD5SUM = ea53b3baf477a283376779a3c1985085

– Edit keepalived configuration:

# vi /etc/keepalived/keepalived.conf

global_defs {
notification_email {
root
}
notification_email_from [email protected]
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DIRECTOR
}

virtual_server 192.168.1.54 80 {
delay_loop 6
lb_algo rr
lb_kind NAT
persistence_timeout 50
protocol TCP

real_server 172.16.1.2 80 {
weight 1
HTTP_GET {
url {
path /index.html
digest 01bc6b572ba171d4d3bd89abe9cb9a4c
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}

real_server 172.16.1.3 80 {
weight 1
HTTP_GET {
url {
path /index.html
digest ea53b3baf477a283376779a3c1985085
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}

virtual_server 192.168.1.54 25 {
delay_loop 6
lb_algo rr
lb_kind NAT
persistence_timeout 50
protocol TCP

real_server 172.16.1.2 25 {
weight 1
TCP_CHECK {
connect_timeout 3
}
}

real_server 172.16.1.3 25 {
weight 1
TCP_CHECK {
connect_timeout 3
}
}
}

– virtual_server : identify a server definition block

– delay loop: specify in seconds the interval between checks.

– lb_algo: scheduler.

– lb_kind: forwarding method.

– persistence_timeout: timeout value for persistent connections.

– protocol: (TCP|UDP)

– real_server: specify a real server member.

– weight: specify weight for load balanced decisions.

– TCP_CHECK: check real servers with TCP connection.

– HTTP_GET: checking real servers using HTTP_GET request.

– connect_timeout: connect remote server using timeout.

– nb_get_retry: maximum number of retries.

– delay_before_retry: delay between two successive retries.

Starting keepalived

# /etc/init.d/keepalived start

Final checks

– ipvs code generated by keepalived for load balancing:

# ipvsadm -A -t 192.168.1.54:80 -s rr
# ipvsadm -a -t 192.168.1.54:80 -r 172.16.1.2:80 -m -w 1
# ipvsadm -a -t 192.168.1.54:80 -r 172.16.1.3:80 -m -w 1

Options:

• A : add service
• a : add server
• t : tcp service
• r : real server
• m : masquerading (packet forwarding method)
• w : weight
• s (scheduler): rr(round robin), wrr (weighted round robin),
  lc (least connections), wlc (weighted least connections),
  lblc (locality based least connection), lblcr (lblc with replication),
  dh (destination hashing), sh (source hashing), sed (shortest expected
  delay), nq (never queue).

– List of servers and services:

# ipvsadm -l

– Test connections with load balanced (Round robin):

Sources

http://www.linuxvirtualserver.org/
http://www.keepalived.org/documentation.html