IPSec Suite

This document covers the IPSec protocol suite in detail. IPSec has been introduced to address the fact that the TCP/IP v4 protocol suite by desgin provides no effective security controls against a lot of imaginable security threats.


IP'sv4 original architects had no reason to provide security at the IP level. IP based network data is, therefore, wide open for tampering and eavesdropping. There are a few technologies now that exist to secure communication over the Internet. These technologies use powerful encryption technologies. However, most of them work at the application layer (PGP, S/MIME) or SSL, which work between the network- and transport layer. These technologies have their strengths and niches, but they are limited to specific uses.

There is a solution, and it isn’t restricted to a single application. Broadly speaking, you can imagine an IP based network as having four layers as following:

  • Physical / Link layer (Layer 1)
  • Network layer
  • Transport layer
  • Application layer


Each layer provides services to the level above. The significant feature of IP network is that the network layer in IP networks is entirely homogenous, and it’s the only layer that is. This means that any communication passing trough an IP network, including the Internet, has to use the IP protocol.
So, if you secure the IP layer, you secure the network.


Securing the IP layer with IPSec

An international working group under the IETF has developed a method doing exactly that. They call it the IP Security (IPSec) protocol suite. The IPSec protocol suite is based on powerful new encryption technologies, which add security services to the IP layer. It is compatible both with IPv4 and the new IPv6.
This means, if you use the IPSec suite where you normally would use IP, you secure all communications in your network for all application and for all users transparently.


Security threats in the network environment

To know you have security in your environment, you want to be confident about three things:

  • Authentication
    • that the person with whom you speaking really is that person
  • Confidentiality
    • that no one can eavesdrop on your communication)
  • Integrity
    • that the data has not been tampered with in any way during transmission

The architecture of a modern, IP based network makes all this difficult to ensure. So, the next thing is to discuss threats often used in IP based networks.



It is difficult in IP based networks to determine where a packet is really come from. A technique called spoofing takes advantage of this. To understand this, you need to know how information travels along a network. On the network layer, information is broken into small, manageable chunks of data called packets. Look at the figure below to see what such a packet contains.

ip packet


When two nodes on a IP network are communicating, the data stream between them is broken up into these packets and released into the network.

The difficulty with this from the security perspective is, that the source IP address in IP packet headers is easily changed. The attack – called spoofing - makes a packet coming from one machine appear to come from somewhere else altogether.


Session hijacking

If your TCP/IP based program trusts an source IP address to know that it’s really communicating with a server, nothing prevents someone from taking over the connection and cutting the link to the server. The Hijacker then takes the place of the server, exchanging data with the client without its knowledge.

The fact that you have identified the person with whom you are communicating once doesn’t mean that you can depend on IP to ensure that it will be the same person through the rest of the session. You need a scheme that authenticates the data’s source throughout the transmission.


Eavesdropping (LAN sniffing)

Today, Ethernet LAN’s are broadly used but it has the disadvantage of making sniffing easy. LAN sniffing is a very easy task if you don’t use a switched environment, means, in a network that uses hubs as connecting points, a packet is usually available for every node connected to that network. Conventionally, each node’s NIC only listens and responds to packets specifically addressed to it.

It’s is easy, however, to put a network card in something called “promiscuous mode” – in that mode the NIC collects every packet that passes the wire and send the packets up the IP-Stack. Usually, there is no way to detect such a NIC from elsewhere in the network. Special Sniffer programs such as Sniffer Pro from Network Associates or the Network Monitor from Microsoft give an administrator the ability to detect nodes running such a program in promiscuous mode elsewhere in the network.

A Sniffer collects all the data that passes the wire – allowing a user to detect quickly what’s going trough any segment of the network. In the hands of someone who wants to listen in on sensitive communication, a sniffer is a powerful eavesdropping tool.


Man-in-the-middle attack

To use encryption, you first have to exchange encryption keys. Once you have encrypted data with a key, you need the same key to decrypt it.

But exchanging unprotected keys through the network could easily defeat the whole purpose, since those keys could be intercepted and open up yet another type of attack – the man-in-the-middle attack.

A penetrator could plant his own key in the process very early, so that, while you believed you were communicating with one party’s key, you would actually communicating with a key know to the man-in-the-middle. IPSec offers various mechanisms to defeat the above described attacks.


Interlocking technologies from IPSec

Ok, let’s now starting with a closer look at the IPSec protocol suite. IPSec offers three interlocking technologies to defeat against traditional threats to IP-based networks.

  • Authentication Header (AH)

Ties data in each packet to a signature that can be verified from the recipient. The AH allows you to verify both the identity of the person sending the data and that the data has not been altered.

  • Encapsulation Payload (EP)

Scrambles the data with encryption so that a sniffer somewhere on the network doesn’t get something usable.

  • Internet Key Exchange (IKE)

A powerful, flexible negotiation protocol that allows users to agree on encryption and authentication methods, keys to use, hash algorithms to use and how long to use the keys before changing them and so on. The basic components of IPSec, the ESP and AH use cryptographic techniques for ensuring data confidentiality and digital signatures for authenticating the data’s source.How IPSec embeds encryption in the ESP