Mitigating a SYN Flood Attack

Before speak about syn flood and some possible mitigation for these attacks, we’ve to remember how works a TCP communication between the server and client. Basically a connection between two hosts (A and B) is composed by three steps:

1) Establishes a communication using the tcp handshake.

2) Transfering the data

3) Releasing and closing the comunication between the two hosts.

Now we have to focus exactly in the phase 1 the tcp handshake, also is composed by three steps:

1) The host A sends a tcp segment with the SYN flag enabled and an ISN (Initial Sequence Number) generated radomly and is incremental.

2) The host B generates a new tcp segment with the flag SYN+ACK enabled, his own ISN and the ISN of the host A increased in 1.

3) The host A check that his ACK sequence number is correct, now the source sends to the host B another TCP ACK to accepting the communication and increasing the ISN sent by the host B in 1.

When the host B receives the SYN request from the A, the host allocate to the memory the TCB about the current connection. TCB (Transmission Control Block) is a complex data structure which maintains the information about the different TCP connections initiated in the machine, about all the information that can stores this data structure we can found the initial sequence number, the number of bytes received, the source and destination port number, the current window size, the state of the connection… to keep the control about all the connections established.

The host A can send to the host B continous tcp segments with the SYN flag enabled unfinished the communication and running out the memory of the host B system that is reserved for the TCP/IP stack, refusing the new connections. This is an error of the of the TCP/IP stack can be exploited by an attacker generating a DOS (Denial Of Service) attack, knowledge as syn flood.

 Syn Cookies 

One possible mitigation can be the syn cookies. Syn cookies are an implementation for the TCP/IP stack, instead of stores in the memory the request of the client, the server response the client with a cookie. When the server creates the initial sequence number, it doesn’t generates a random number, instead of it generates a “hash” composed by the timestamp, the source and destination port and IP address and the maximum segment size. With this way the server doesn’t need to store the request in memory and when the client sends the ack to the server with the cookie number increased in one, the server has only to recalculate the cookie doing the inverse operation in it to check if is a correct tcp segment and then establish the communication. The syn cookies will be only working when the syn queue is full, else the server will use the normal implementation of the TCP/IP.

To enable syn cookies in GNU/Linux based systems:

# vi /etc/sysctl.conf
net.ipv4.tcp_syncookies = 1
# sysctl -p

BSD variants:

# vi /etc/sysctl.conf
net.inet.tcp.syncookies= 1
# sysctl net.inet.tcp.syncookies=1

 Syn Cache 

A host using syn cache, has a hash table with a limited amount of space to stores a part of the data of a fully TCB. The host generates a hash with the information of the incoming SYN segment and this hash is alocated in a global hash table. When the two hosts completing the tcp handshake, this hash is moved into a full TCB. There are a limit for a hash table, if this limit is exceeded then the oldest entry is dropped. The syn cache structure took up 196 bytes for the data structure in comparison with 736 bytes for a full TCB. There are various based implementations of syn cache for the BSD variants, for Linux based systems actually there aren’t any implementation. In FreeBSD systems is enabled by default, to see the possible directives and the default values:

# sysctl -a | grep syncache
net.inet.tcp.syncache.rexmtlimit: 3
net.inet.tcp.syncache.hashsize: 512
net.inet.tcp.syncache.count: 0
net.inet.tcp.syncache.cachelimit: 15360
net.inet.tcp.syncache.bucketlimit: 30

– rexmtlimit controls the maximum number of times a SYN,ACK is retransmited before discard the connection.
– hashsize controls the size of the global hash table, must be a power of 2.
– count is a read only directive that shows how many entries are currently present in syncache.
– The cachelimit directive defines the number of syncache entries that may be allocated.
– bucketlimit controls the size for each hash.

This directives may be changed editing on the file /boot/loader.conf:

net.inet.tcp.syncache.rexmtlimit= 3
net.inet.tcp.syncache.hashsize = 512
net.inet.tcp.syncache.cachelimit = 15360
net.inet.tcp.syncache.bucketlimit = 30

 Firewall: iptables 

– Creates a new chain call syn-flood

# iptables -N syn-flood

– Jump to the chain syn-flood for the input tcp segments input on the eth0 interface:

# iptables -A INPUT -i eth0 -m -p tcp --syn -j syn-flood

– If the limit matches then return to the normal flux of iptables to continues reading the other rules

# iptables -A syn-flood -m limit --limit 1/s --limit-burst 4 -j RETURN

• –limit : limits a maximum average of packages per unit of time defined, in this case 1 package per second.
• –limit-burst: Maximum number of initial packages. This number is restarted on every time defined by –limit is not reached.

– If not matches then the package will be dropped

# iptables -A syn-flood -j DROP

Sources

http://cr.yp.to/syncookies.html
http://tools.ietf.org/html/rfc4987

– For more information about the TCP and TCB data structure you can see the RFC:
http://www.ietf.org/rfc/rfc793.txt