IPSec modes

Main mode

Main mode provides a mechanism for establishing a first phase IKE SA, which is used to negotiate future communication. The object here is to agree on enough things (authentication and encryption algorithm, hashes, and keys) to be able to communicate securely long enough to set up an SA for future communication.

1. use Main mode to bootstrap an IKE SA
2. use Quick mode within that IKE SA to negotiate a general SA
3. use that SA to communicate from now on until it expires

All associated parties do a little bit exchanging work. In the first exchange, the parties agree on basic algorithms and hashes. In the second they exchange public keys for a Diffie-Hellman exchange and pass each other random numbers the other party must sign and return to prove their identity. In the third they verify those identities.

The parties then use the generated shared Diffie-Hellmann value in the three permutations, once they derive it. All parties have to hash it three times, generating first a derivation key (used for generating additional keys later in Quick mode), then an authentication key and finally the encryption key to be used for the IKE SA.

Aggressive mode

The Aggressive mode provides the same services as the Main mode. The difference to Main mode is that it is accomplished in two exchanges rather than in three, with only one round trip and a total of three packets rather than six.

Aggressive mode exchange attains the same goal as Main mode, except than Aggressive mode does not provide integrity protection for the communicating parties. The advantage of this mode is, however, speed.

Quick mode

Once the communicating parties have established an IKE SA using Main or Aggressive mode, they can use Quick mode.

Quick mode has two purposes: negotiating general IPSec services and generating fresh keying material. Quick mode is less complex than either Main or Aggressive mode. This is because it is using an already established secure channel and can therefore be less complex. Therefore Quick mode packets are always encrypted, and always start with a hash payload – it is used to authenticate the rest of the packet.

Key refreshing can be done in two ways. If you don’t need perfect forward secrecy, Quick mode can just refresh the keying material already generated (in Main or Aggressive mode) with additional hashing. The communicating parties can exchange nonce's trough the secure channel, and use these to hash the existing keys. If you do want perfect forward secrecy you can still request an additional Diffie-Hellman exchange trough the existing SA and exchange the keys that way.
Basic Quick mode is a three packet exchange, like Aggressive mode.

Negotiating the SA

After all those exchanges to generate the IKE SA, establishing the general purpose SA is relatively simple. To generate a new SA, the initiator sends a Quick mode message, protected by the IKE SA, requesting a new SA. A single SA negotiation actually results in two SA's – one inbound, to the initiator, and one outbound. Each IPSec SA is one way and the node on the receiving end of that SA always chooses its own SPI to ensure it is the only SA using that reference.

So, using Quick mode, the initiator tells the respondent which SPI to use in future communications with it, and the responded follows up with its own selected SPI.
Each SPI, in concert with the destination IP address and the protocol, uniquely identifies a single IPSec SA. However, these SA's are always formed in pairs (inbound and outbound), and these pairs have identical parameters (keys, authentication, encryption algorithms and hashes), apart from the SPI itself.

How do you know who is who?

How do you verify that people are who they say they are in the first place? The final component of the IPSec compliant secure VPN, in most implementations, is a Certificate Authority (CA). I already covered that in a previous article. So here, again, in short, how it goes.

A CA is a trusted third party, someone whose identity you can already prove, and who can vouch for the identity of people with whom you are trying to communicate. The CA is like a public figure who you know well enough to trust, and who vouches for other people. The CA software issues certificates tying three things together:

• someone's identity
• the public key that a person uses to sign their identity to online documents
• the CA's public key (used to sign and authenticate the entire package)

The CA is the defense against the man-in-the-middle working his way into key exchanges. Whenever you initiate an exchange with someone, he has to sign it with his digital signature. You in turn can check that signature against the one on record with the CA. They have to match. You then check that certificate's signature with the CA's signature. They have to match too.

IKE provides for third party verification using the established industry-standard. X.509 format certificate.

Robust, scalable key exchange

Altogether, IPSec's IKE system offers network users a great deal that other schemes have had difficulty delivering. So, to put it all together, with an IPSec secure VPN:

• you can form SA's with a large range of attributes, layering your network
• you can build multiple domains of communication within the same secure VPN
• you can update keys as frequently as you wish
• you can do this in any IP network environment, of any size, for any size of enterprise
• you can do this with anyone using IPSec technology