One major new feature UMTS introduced when it was designed that GSM did not have was mutual authentication instead of only the device authenticating towards the network. This way, man-in-the-middle attacks can be prevented in which an attacker puts a rouge base station in place and tricks a device into using it instead of the real network. So far I always assumed that the Authentication token (AUTN) that was introduced contained all the magic. But 3G security and ciphering is a bit complex so I never dug down deep enough to actually understand how it really works. Lately, I came across the topic again and this time around I investigated a bit more. So here's how man-in-the-middle attacks are prevented in UMTS:
The story starts with the Authentication token (AUTN). This is a new parameter in UMTS that did not exist in GSM and it is computed in the Home Location Register / Authentication Center (HLR / AuC) and on the SIM card. Input parameters are a random number, which is sent during authentication from the network to the mobile device and the secret key that is only stored in the SIM card and in the Authentication Center and never sent anywhere. Another input parameter I was so far not aware about is a sequence number (SQN) that increases over time. When authentications are performed the mobile device only accepts an AUTN that was generated with a higher sequence number than what it has seen before. In practice, things are a bit complicated by the circuit switched and packet switched core network parts having an individual set of precomputed authentication vectors and each side authenticates a mobile device on its own. In other words sequence numbers increase independently on the core and packet switched side and a mechanism is in place in the mobile device to handle this. How sequence numbers are generated and increased is implementation specific but suffice it to say that the number can only increase and not decrease over time.
At this point we have the AUTN and the sequence number (SEQ) that is encoded in the AUTN to prevent replay attacks, i.e. a reuse of potentially intercepted authentication information. The next and equally vital ingredient is integrity checking of signaling messages that are exchanged between the network and the mobile device. Integrity checking is also based on the secret key and ensures that messages are not altered on the fly by an attacker that has managed to insert itself in the transmission chain. At this point an attacker can still passively eavesdrop on the signaling and user data exchange. Therefore the final ingredient is ciphering of signaling messages and user data to prevent this as well.
To quickly summarize: The following things are needed to prevent man-in-the-middle attacks and eavesdropping:
- An Authentication Token (AUTN) so the mobile knows the Authentication Center trusts the network which performs authentication
- A Sequence Number (SEQ) embedded in the authentication token to prevent replay attacks
- Integrity checking so an attacker can't act as a man in the middle
- Ciphering to prevent passive eavesdropping
For much more details see this paper from adventurous days back in 2001.