Security in Computer Networks

A message can be private but still be altered, or come from an impostor. Secure communication therefore needs several properties, each answering a different question about the exchange.

Name the property at risk

Alice and Bob are the communicating parties; Trudy is the attacker. A passive attacker captures traffic without changing it. Reading a password is eavesdropping. Inferring a busy work schedule from connection times is traffic analysis, even when the contents are encrypted. An active attacker injects, changes, deletes, or replays packets. Spoofing claims a false source identity; hijacking takes over an existing exchange; flooding tries to exhaust resources.

Internet routers generally forward packets without proving that their source addresses are genuine. Packets also cross networks outside either user's control. On Wi-Fi, another radio in range can capture transmitted frames. Protocols must protect communication even when an attacker controls the path.

Follow one exposed login

Bob connects to a café hotspot and logs in using plain HTTP. Trudy reads the password: confidentiality has failed. She changes a download link in the reply: integrity has failed. If she supplies a fake hotspot with the café's name, Bob also lacks a reliable check of the access point's identity. Encrypting just the password would not repair every one of these failures. An attacker can alter some encrypted messages without understanding them, and can replay a captured authentication message unchanged.

Keep the key secret

Kerckhoffs's principle says security must survive disclosure of the algorithm. The secret is the key. A homemade secret algorithm can be recovered from software; changing and distributing an entire algorithm is also harder than replacing a key. Replacing an exposed key protects future traffic, but does not make recordings under the old key safe.

Attacker knowledge matters too. A ciphertext-only attack starts with intercepted encrypted messages. A known-plaintext attack adds matching original messages, such as predictable HTTP/1.1 text beginning with "GET /". In a chosen-plaintext attack the attacker can obtain encryptions of messages she selects. Modern encryption is designed to resist these stronger situations.

Locate the protected path

Wi-Fi and cellular security protect a radio link. TLS protects communication between application endpoints. IPsec protects selected IP traffic between hosts or gateways. Message-level protection, such as secure email, can remain with a message during storage. A secure radio hop does not protect the rest of an internet route: after an access point removes link protection, another mechanism must protect the onward traffic.

A symmetric cipher uses the same secret key at both ends. Plaintext is the original message; ciphertext is its encrypted form. Write encryption as $c=E_{K_S}(m)$ and decryption as $m=D_{K_S}(c)$: the key is shared, but the two operations undo each other.

Key size and structural weaknesses

A Caesar shift of 3 turns "MEET AT NOON" into "PHHW DW QRRQ". Trying all 25 useful shifts breaks it. A substitution cipher has $26!\approx 4.03\times {10}^{26}$ keys, yet it preserves letter frequencies. An attacker can use repeated letters and common words to recover the mapping without trying every key. Key-space size alone does not prove security.

AES is a block cipher with 128-bit blocks and 128-, 192-, or 256-bit keys. DES had 64-bit blocks and a 56-bit key; exhaustive search broke a DES key in 1998, and DES and Triple DES are retired from standard use. Each extra key bit doubles exhaustive-search work. At ${10}^{12}$ trials per second, searching all DES keys takes about $2^{56}\mathrm{/}{10}^{12}\approx \mathrm{72,000}$ seconds, or 20 hours; average discovery takes half that. Searching all AES-128 keys at that rate takes about $1.1\times {10}^{19}$ years.

The mode changes message protection

A mode of operation handles more than one block.

CBC's first block uses an initialization vector, or IV, in place of a previous ciphertext block. The IV is sent openly; unpredictability and freshness matter, not secrecy. Randomization hides direct equality patterns across messages. It does not itself detect tampering.

Work through keystream reuse

Suppose two messages use the same keystream $k$. Their ciphertexts are $c_1=m_1\oplus k$ and $c_2=m_2\oplus k$. XORing them cancels the repeated keystream:

Knowing one plaintext now reveals the other. WEP's repeated IVs caused this kind of failure. A longer encryption key does not repair a repeated keystream.

AEAD, authenticated encryption with associated data, combines confidentiality with integrity. AES-GCM and ChaCha20-Poly1305 authenticate the ciphertext and any associated data. Associated data, such as a routing field, remains readable but cannot be changed without failing verification. TLS 1.3 uses AEAD for every protected record.

Distribute the secret

Fast bulk encryption still requires a shared key. If every pair among $n$ users needs a different key, the total is $n(n-1)\mathrm{/}2$. For 100 users, that is 4,950 keys. Public-key mechanisms let parties establish session keys without distributing a separate secret to every pair beforehand.

Bob publishes $K_B^{+}$ and keeps $K_B^{-}$ private. Alice can encrypt for Bob without first sharing a secret with him. Recovering the private key from the public key must be computationally infeasible.

Calculate a toy RSA exchange

RSA starts with primes $p$ and $q$. Compute $n=pq$ and $z=(p-1)(q-1)$. Choose a public exponent $e$ relatively prime to $z$, then find $d$ such that $ed<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>z=1$. The public key is $(n,e)$; the private key is $(n,d)$. For a message number below $n$:

Use toy primes 11 and 13. Then $n=143$ and $z=120$. With $e=7$, choose $d=103$ because $7\times 103=721=6\times 120+1$. For $m=9$, repeated squaring gives $9^2<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>143=81$ and $9^4<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>143=126$. Combining the powers gives $9^7<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>143=48$. Decryption returns ${48}^{103}<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>143=9$.

These numbers expose the arithmetic; practical RSA uses moduli of at least 2,048 bits. Factoring the modulus reveals the primes and enables key recovery. Real RSA encryption also uses randomized OAEP padding: unpadded textbook RSA repeats its ciphertext for a repeated message.

Agree on a secret with Diffie–Hellman

Diffie–Hellman establishes a shared secret without sending that secret. Alice and Bob publish a prime $p$ and generator $g$. Alice keeps $a$ secret and sends $A=g^a<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>p$. Bob keeps $b$ secret and sends $B=g^b<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>p$. Alice computes $B^a<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>p$ and Bob computes $A^b<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>p$; both obtain $g^{ab}<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>p$.

For $p=29$, $g=2$, $a=5$, and $b=12$, Alice sends 3 and Bob sends 7. Alice's calculation is $7^5<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>29=16$; Bob's is $3^{12}<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>29=16$. Their small example is easy to break; large groups make recovering the secret exponents infeasible. Elliptic-curve exchanges such as X25519 serve the same purpose with shorter keys.

Authenticate the exchange

Plain Diffie–Hellman does not identify either participant. Trudy can substitute her own public values and create two separate shared secrets, one with Alice and one with Bob. She then reads and re-encrypts traffic. An authenticated protocol binds the exchanged values to trusted identities, using signatures and certificates or an established shared secret.

Protocols use hybrid encryption: public-key work establishes a session key, and AES or ChaCha20 handles the bulk data. RSA can transport a random session key, but later theft of the RSA private key exposes recorded exchanges. Fresh ephemeral Diffie–Hellman values, deleted after use, provide forward secrecy against later theft of long-term authentication keys.

A sufficiently large quantum computer would break RSA and classical Diffie–Hellman. ML-KEM, standardized by NIST in FIPS 203 in 2024, addresses post-quantum key establishment. A hybrid such as X25519MLKEM768 combines classical and post-quantum components. This use of "hybrid" describes two key-establishment mechanisms; bulk data still uses symmetric encryption, including 256-bit keys for a strong security margin.

An integrity check must survive a deliberate attacker, not just accidental errors. A network checksum and a cryptographic hash serve different purposes, and neither authenticates a message by itself.

Understand the hash properties

A cryptographic hash function maps a message of any length to a fixed-size digest. SHA-256 produces 256 bits, usually written as 64 hexadecimal digits. Good hashes have three distinct resistance properties:

Collisions exist because a finite set of digests represents more possible messages. Security means finding a useful collision is infeasible. MD5 and SHA-1 have practical collision attacks and are unsuitable for signatures and certificates. SHA-3 supplies another modern hash family.

Small message changes spread across the digest: this is the avalanche effect. Changing "Pay Ana 100" to "Pay Ana 900" changes 124 bits of the SHA-256 output; the digests start with ECD0D31E and 21736708. By contrast, the Internet checksum of both "PAY 1290" and "PAY 9012" is 0xEC3B: swapping the relevant 16-bit words preserves their sum. A checksum designed to catch random damage is easy to preserve deliberately.

Add the secret an attacker lacks

Sending a message and its hash openly does not solve active tampering. Trudy replaces both the message and its digest, so the receiver's recalculation succeeds. A message authentication code, or MAC, adds a secret authentication key shared by Alice and Bob.

Alice sends $m$ and $\mathrm{M}\mathrm{A}\mathrm{C}(s,m)$. Bob computes a tag from the received message and his own copy of $s$, then compares tags. If Trudy changes "Pay Ana 100" to "Pay Ana 900", Bob calculates the tag for the changed text. Trudy's copied old tag fails the comparison. The message can still be readable: a MAC supplies integrity and authentication of a key holder, not confidentiality.

HMAC uses a hash with two nested, keyed passes. HMAC-SHA256 is a standard choice. Simply using $H(s\mathrm{\parallel }m)$ with SHA-256 is unsafe: length extension can let an attacker append bytes and calculate a matching tag without learning the key. HMAC's construction avoids that failure.

Separate authenticity from freshness

An old message with its original MAC still has a valid tag. To reject replay, authenticate a sequence number or a fresh challenge along with the message, and check that freshness value at the receiver. Adding an unauthenticated number is insufficient because Trudy could change it.

Both shared-key holders can make valid tags. A third party therefore cannot use a MAC to decide which holder wrote a message. Digital signatures supply that publicly verifiable evidence. When combining encryption with a separate MAC, use separate keys and encrypt-then-MAC, authenticating the ciphertext; an AEAD construction provides an integrated alternative.

A digital signature lets anyone check evidence created with one private key. The verifier needs both a sound signature check and a trustworthy link between the public key and the identity it represents.

Sign and verify a message

The signer uses a private key; a verifier uses the matching public key. Signatures bind their evidence to the message, so alteration fails verification. The private key must remain under the signer's control. This supports non-repudiation: independently verifiable evidence of signing, unlike a MAC that either shared-key holder could create.

Schemes process a digest or encoded representation of the message. In a simplified RSA model the signature is $K_B^{-}(H(m))$. The receiver computes the digest and compares it with the value recovered using Bob's public key. Actual RSA signatures add an encoding such as RSASSA-PSS; ECDSA and Ed25519 use different arithmetic. Signing is not a general-purpose method of encrypting a message with a private key.

For a toy RSA key with $n=143$, $e=7$, and $d=103$, let the digest number be 20. Bob calculates $\sigma ={20}^{103}<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>143=58$. Alice obtains ${58}^7<mrow><mi\; mathvariant="normal">m</mi><mi\; mathvariant="normal">o</mi><mi\; mathvariant="normal">d</mi></mrow>143=20$ and compares it with her own digest. If altered text has digest 21, the recovered 20 does not match. Collision resistance matters because two messages sharing a digest could otherwise share the same signing input.

Bind a key to a name

Trudy can generate a perfectly valid key pair. If she labels its public key "Bob", successful signature verification alone will not expose the substitution. A certificate binds an identity to a public key under a certification authority's signature.

An X.509 website certificate contains domain names in the subject alternative name field, the subject's public key, an issuer, validity dates, and the issuer's signature. A server normally supplies its leaf certificate and intermediate certificates. The client checks this chain to a root already trusted in its operating system or browser. A root's self-signature is not what makes it trusted; inclusion in the trust store supplies that starting trust.

The verifier checks signatures along the path, validity dates, the intended name, permitted usage, and revocation information where available. A chain ending at an unknown root does not become trusted merely because its signatures are mathematically valid.

Require evidence of key possession

Certificates are public and can be copied. A server must also show control of the associated private key by signing fresh handshake data. A stolen certificate file alone cannot produce that signature. An ordinary valid website certificate establishes control of its domain name; it does not establish that the site's content or business is honest.

End-point authentication asks who is participating now. An old valid message may prove that someone once held a key, yet fail to prove that the present sender holds it.

Break the weak versions

Alice first sends "I am Alice". Trudy can send those same words. Adding Alice's source IP address fails because a datagram can carry a forged address. Sending a password exposes it to an observer. Encrypting a fixed password hides its contents but leaves a reusable proof: Trudy records the ciphertext and sends it in another login.

This is a replay attack. The attacker does not decrypt or change the old proof. The weakness is that Bob accepts the same bytes in a new session. A stronger cipher would not fix that protocol error.

Make the proof depend on this attempt

Bob sends a fresh, unpredictable nonce, a challenge used once in this authentication context. Alice returns a value that requires her secret key, such as an HMAC of the challenge. Bob computes the expected response with his copy of the key. A response copied from an earlier challenge will not match.

For example, Monday's challenge is 0x7F3A91C2. Trudy captures its valid response. On Tuesday Bob sends 0x0B66E4D9. Replaying Monday's response fails because Tuesday's expected tag is calculated from different bytes. The short hexadecimal values illustrate the exchange; real challenges need enough random bits to avoid useful prediction or repetition.

Unpredictability blocks advance collection too. If Bob's next challenge is easy to guess, Trudy can pose as Bob, ask Alice to answer that future challenge, and later present the collected response to Bob. A challenge must be tied to the current protocol exchange, not just look different from the last one.

Authenticate with public keys

Alice can sign a fresh challenge using her private key. Bob verifies it using her genuine public key. If Bob accepts a substitute public key supplied by Trudy, Trudy can sign Bob's challenge herself. Certificates establish the identity-to-key link; the fresh signature proves current possession of that key.

In mutual authentication, both peers prove their identities. A protocol must distinguish their roles and bind proofs to the session so one side's response cannot simply be reflected as the other's. Ordinary web TLS usually authenticates the server with a certificate; the user then authenticates inside that protected channel using a password or passkey. Service connections and VPNs often authenticate both endpoints.

Protect what comes after login

Successful authentication does not protect later plaintext traffic. Trudy could wait until login finishes and then inject commands or hijack the connection. Secure handshakes establish session keys, and subsequent messages carry integrity protection tied to that session.

Challenge–response also does not strengthen a weak human password automatically. If an observer can recompute a captured response for each password guess, she can run an offline dictionary attack. The freshness check stops replay while a strong secret or suitable password-authenticated exchange addresses guessing; the two protections solve different problems.

TLS gives two application processes a confidential, authenticated channel over a reliable byte stream. In the common HTTPS case, HTTP uses TLS over TCP, normally on port 443. An application library such as OpenSSL or BoringSSL handles TLS; TCP transports its bytes without interpreting them.

Establish keys, then protect records

A handshake authenticates the peer and establishes shared keying material. Key derivation turns that material into separate secrets for handshake traffic and application data, with different keys for each direction. Alice's outgoing record cannot simply be reflected back as Bob's because the receiving direction uses different protection.

TLS splits data into records, allowing the receiver to verify one piece at a time. A plaintext record carries at most $2^{14}=\mathrm{16,384}$ bytes of content. Its short header includes a type, legacy version field, and length. Encryption adds overhead, so an encrypted record is larger than the original content. In TLS 1.3 the outer protected-record type does not directly reveal the enclosed application or handshake content type.

TLS 1.3 requires AEAD. The receiver verifies a record's authentication tag before releasing its plaintext to the application. If Trudy changes a protected HTTP request, the tag fails: the receiver terminates the connection, using bad_record_mac for a record-protection failure. TLS detects interference; it does not turn maliciously changed bytes back into a valid request.

Count records in each direction

Each sender and receiver maintains an implicit record sequence number. It is not sent in the TLS record header. TLS 1.3 XORs it with a per-direction IV to produce the AEAD nonce. Replaying an old record at a new position uses the wrong expected number and fails authentication. Deleting or reordering protected records similarly disrupts the expected sequence when later bytes are processed.

TCP's sequence numbers still handle ordinary delivery and reordering. They do not authenticate the peer: an attacker who controls the path can forge TCP headers. TLS's record count is bound to secret-key protection, so successful TCP delivery alone does not imply successful TLS verification.

Detect an incomplete ending

Suppose Bob expects a document but the TCP connection closes halfway through. A forged FIN or a broken link can end transport delivery without proving that the TLS sender finished. TLS uses the protected close_notify alert to signal an orderly end of its sending stream. Without it, the receiver must consider possible truncation and use the application's completeness rules; an unexpected TCP EOF is not an authenticated end marker.

An attacker can always discard packets and stop progress. Confidentiality and integrity do not guarantee availability. A complete HTTP message length may help an application recognize that one message arrived, but that is a separate check from orderly TLS closure.

Distinguish related protocols

TLS 1.3 is specified by RFC 9846, which replaced RFC 8446 in 2026. TLS 1.2 remains deployed; SSL 2.0/3.0 and TLS 1.0/1.1 are deprecated. DTLS adapts the security design to datagrams. QUIC carries the TLS 1.3 handshake within its UDP-based transport, then protects QUIC packets itself; it does not send ordinary TLS-over-TCP records over UDP.

A TLS handshake must establish a secret and show whose secret it is. In a full certificate-based TLS 1.3 exchange, ephemeral key agreement provides fresh keying material and signatures bind the negotiation to the server's identity.

Follow the message flights

1. ClientHello offers versions, AEAD cipher suites, key-exchange groups, signature algorithms, random data, and one or more key_share values. It usually names the server through SNI.
2. ServerHello selects parameters and supplies the server's key share. Both peers compute shared keying material and derive handshake traffic keys.
3. The server sends encrypted EncryptedExtensions, Certificate, CertificateVerify, and Finished messages. CertificateVerify signs a structured input including the handshake transcript hash; Finished authenticates the transcript with a derived MAC key.
4. The client checks the chain, server name, signature, and Finished, then sends its own Finished. Application traffic uses separately derived traffic secrets.

The server's certificate associates a name with a public key. CertificateVerify proves control of the private key in this exchange. Finished detects interference with the negotiated transcript. These are separate checks: successfully deriving a shared secret does not establish server identity.

Count round trips

At RTT 40 ms, ignoring processing and transmission delays, TCP setup takes 40 ms and a full TLS 1.3 handshake adds 40 ms. The client sends its request at 80 ms; a response can return around 120 ms. A full TLS 1.2 handshake adds two RTTs after TCP, so the corresponding times are 120 ms and 160 ms. QUIC combines transport setup with the TLS handshake, allowing a new connection to send its request after one RTT, at 40 ms.

A HelloRetryRequest adds a round trip when the server needs a different key share. Shared support for a group is not enough if the initial offer did not contain an acceptable share.

Bind negotiation and separate secrets

If Trudy removes strong cipher suites from a ClientHello in transit, the client and server record different transcripts. Authentication of those transcripts fails. TLS also includes version-downgrade signaling when a capable server negotiates an older version.

The key schedule uses HKDF, based on HMAC, to derive purpose-specific secrets. Handshake keys protect setup messages; application keys protect later data. KeyUpdate replaces application traffic keys during a long connection. TLS 1.3 removed static RSA key transport: an RSA certificate can still authenticate a handshake by signing, while ephemeral key agreement provides forward secrecy.

Resume without repeating certificate authentication

After setup, NewSessionTicket lets the peers prepare resumption. They derive a resumption PSK from the resumption secret and ticket nonce; the client later presents the ticket as the PSK's identifier and proves possession of that PSK. Resumption can include fresh Diffie–Hellman for forward secrecy.

0-RTT sends early data with the first flight, but that early data can be replayed. Use it only where repeating a request is safe, never for an unprotected repeatable payment action. The later handshake's protections do not retroactively make early data replay-resistant.

Mutual TLS adds a requested client certificate and proof of its private key. Post-quantum hybrid groups such as X25519MLKEM768, defined in RFC 10024, combine X25519 and ML-KEM-768 so secrecy survives if either component remains secure.

Encryption is useful only when it reaches the intended peer. A client must validate the identity behind the keys before sending sensitive application data.

Check the whole identity claim

A TLS client checks that the certificate chain reaches a root in its trust store, that signatures and validity periods pass, that the requested name matches the certificate, and that the certificate permits TLS server authentication. Where it checks revocation, that must pass too. CertificateVerify then demonstrates possession of the corresponding private key.

Name matching is about the host the client intended to contact, not a name the server suggests after connecting. A certificate for *.example.com covers www.example.com and mail.example.com. It covers neither example.com nor a.b.example.com: the wildcard replaces exactly one label. A perfectly valid certificate for the wrong domain still fails authentication.

For example, a client requests shop.example.net but receives a trusted, current certificate for *.example.com. The chain and dates may pass; the name check fails. The client must reject that connection. A certificate is not a general license for its holder to impersonate any server.

Understand an encrypted interception

If certificate verification is disabled, Trudy can terminate TLS from the client and create a second TLS connection to the real server. Both links are encrypted, but Trudy has the plaintext between them. Checking for encryption without checking the peer therefore misses the attack. Keep certificate validation enabled and treat a failed identity check as a failed connection.

An organization's TLS inspection proxy follows a related structure deliberately. Managed devices trust the organization's root, allowing its proxy to issue certificates they accept. The proxy terminates one secure connection and starts another. The browser-to-server path then has an additional trusted endpoint with access to plaintext.

Protect the first request

In SSL stripping, an attacker keeps the user's side on HTTP while contacting the site through HTTPS. This can begin when a browser first tries a bare address over HTTP. HSTS tells a browser to use HTTPS for later visits. A preload entry can enforce HTTPS even before the first visit; a policy learned only after a successful HTTPS visit cannot protect an earlier HTTP request retroactively.

Identify what remains visible

TLS 1.3 encrypts the server certificate and application contents, but observers still see network addresses, ports, timing, and record sizes. Ordinary ClientHello SNI exposes the requested server name. Encrypted Client Hello, standardized in RFC 9849, protects that name when deployed. A separately unprotected DNS lookup can also reveal the destination name.

TLS protects data between its endpoints while it travels. It cannot repair malware on the client, a compromised server, insecure stored data, or application flaws. It also cannot prevent packet dropping. IPsec addresses a different scope: it applies network-layer protection to selected IP traffic without requiring each application to invoke TLS. Using both can protect different stretches of the same route.

IPsec applies protection at the IP layer, where it can cover TCP, UDP, and ICMP traffic without changing their applications. Its endpoints may be individual hosts or gateways connecting whole networks.

Choose a protocol and mode

AH uses IP protocol number 51; ESP uses 50. ESP is the usual choice for modern VPN traffic. AH's protection includes address fields that NAT rewrites, making ordinary address translation incompatible with its authentication check. ESP does not authenticate the outer IP header.

In transport mode, the ESP header sits between the original IP header and the transport segment. Routers still see the original source and destination. In tunnel mode, a new IP header addresses the two tunnel endpoints, and the original IP datagram travels inside. With encrypted ESP tunnel mode, outside observers cannot read the original hosts' addresses or transport ports, but they can read the gateways' addresses needed for forwarding. AH in tunnel mode still supplies no confidentiality.

IPsec can provide confidentiality, integrity, origin authentication, and anti-replay protection. The actual services depend on protocol, algorithms, and policy; choosing AH does not supply encryption merely because it belongs to IPsec.

Keep state in each direction

A security association, or SA, defines one direction of protected traffic. It holds algorithms, keys, mode, endpoints, sequence state, replay-window state, and a lifetime. A 32-bit security parameter index, or SPI, helps the receiver select the right SA. It is an identifier, not an encryption key, and can travel openly in the ESP header.

Two-way traffic needs two IPsec SAs. A headquarters gateway serving one branch and 50 traveling laptops has 51 peers. With one ESP pair per peer it needs $2\times 51=102$ ESP SAs, plus an IKE SA per peer for management. Rekeying or separate traffic classes can add further SAs; the count here follows the stated one-pair design.

Although IP forwarding is connectionless, this cryptographic state is maintained at the protecting endpoints. Routers between them need not know the session keys or maintain the IPsec SA.

Separate policy from active protection state

The Security Policy Database (SPD) matches traffic selectors such as addresses, protocols, and ports. Its decision is to protect selected traffic, bypass protection, or discard it. The Security Association Database (SAD) stores the parameters and changing state of established SAs. A receiver uses the packet's SPI and relevant context to find that state.

The SPD answers "What should happen to this traffic?" The SAD answers "Which established keys and counters implement that protection?" A VPN policy can protect office subnets while letting another destination bypass it. Having IPsec enabled does not mean that every packet a host sends enters a tunnel.

A site-to-site VPN joins two networks through gateways; a remote-access VPN joins a user's device to a gateway. Both encapsulate traffic so that ordinary internet routers can carry it between protecting endpoints.

Establish the associations

IKEv2 sets up keys and SAs. IKE_SA_INIT exchanges algorithm proposals, Diffie–Hellman values, and nonces. The resulting keys protect IKE_AUTH, where peers authenticate their identities and create the first pair of ESP SAs. With signatures or a pre-shared key this normally takes four messages in two round trips. EAP authentication can add IKE_AUTH exchanges. Later exchanges create more SAs or replace expiring keys.

IKE initially uses UDP port 500. When NAT is detected, IKE and UDP-encapsulated ESP use port 4500. ESP itself has no TCP or UDP ports; encapsulation lets port-based NAT mappings carry it reliably, including multiple clients. The outer transport changes, while ESP still protects the inner traffic.

Send one datagram

Host 10.1.0.5 sends to 10.2.0.9. The branch gateway is 198.51.100.1; headquarters uses 203.0.113.1.

1. The branch SPD selects tunnel protection for the destination subnet, and the gateway obtains the matching SA.
2. It treats the complete original datagram as ESP's inner payload, adding a trailer with padding, pad length, and Next Header 4 for IPv4.
3. It encrypts the inner datagram and trailer, and adds the ESP header containing the SPI and next sequence number. Integrity protection covers the ESP header and protected payload; an AEAD algorithm integrates encryption and tag generation.
4. It adds the outer IP header, addressed from branch gateway to headquarters gateway. Native ESP uses protocol 50; NAT traversal carries ESP inside UDP.

Internet routers see the outer endpoint addresses. The internal addresses and ports are inside the encrypted payload.

Receive and authenticate

Headquarters uses the SPI to select an SA, preliminarily checks the sequence number, verifies integrity, and decrypts. It updates trusted replay state only after authentication succeeds. It removes the tunnel protection and forwards the recovered original datagram to 10.2.0.9. Next Header identifies the recovered payload: 4 for IPv4, 41 for IPv6, or 6 for TCP in an ESP transport-mode example.

Allow reordering but reject replay

A 64-packet anti-replay window with highest authenticated number 1000 covers 937 through 1000. Unseen packet 990 can arrive late and be accepted. Its next copy is rejected. Packet 930 is too old. A valid packet 1005 moves the range to 942 through 1005. An attacker cannot move the trusted window just by sending an unauthenticated packet with a large number.

A VPN protects only its endpoints' intervening path. After tunnel removal, internal delivery needs its own protection where required. Split tunneling sends some traffic outside the VPN. A commercial VPN makes its provider another trusted network operator. TLS/DTLS-based VPNs and WireGuard use other protocols to implement tunnels; "VPN" names the network arrangement, not one mandatory encryption protocol.

A Wi-Fi frame is available to radios in range. A nearby attacker can capture traffic or advertise an evil twin, a fake access point using a familiar network name. Wireless security must authenticate access and protect frames, not merely hide the network name.

Why WEP failed

WEP combined a shared 40- or 104-bit secret with a 24-bit public IV to initialize RC4. Its CRC-32 check detected errors but was not a keyed authenticator. The IV space has only $2^{24}=\mathrm{16,777,216}$ possibilities; randomly selected IVs reach about a 50% repeat chance after roughly 4,823 frames.

A repeated IV under the same key repeats the keystream. XORing two ciphertexts then reveals the XOR of the plaintexts. CRC-32's linear structure also lets an attacker modify bits and adjust the error check. Weak RC4 key scheduling enabled key recovery from captured frames. These failures involve nonce reuse, unauthenticated integrity, and cipher construction; simply choosing a less obvious network name fixes none of them.

Replace the frame protection

WPA2, based on IEEE 802.11i, uses CCMP: AES counter-mode encryption combined with CBC-MAC integrity. Its packet number helps construct unique nonces and reject replay. Earlier WPA/TKIP was a transitional measure for old hardware and is deprecated.

Connection proceeds through discovery, authentication that establishes a pairwise master key (PMK), the four-way handshake, and protected transfer. In WPA2-Personal, a passphrase and network name produce a 256-bit PMK through PBKDF2 with 4,096 HMAC-SHA1 iterations. Everyone using that passphrase and SSID derives the same PMK. Authentication has not yet supplied a fresh per-association traffic key.

Follow the four-way handshake

1. The access point sends ANonce.
2. The station generates SNonce and derives the PTK from the PMK, both nonces, and both MAC addresses. It sends SNonce and a message integrity code (MIC).
3. The access point derives the same PTK and checks the MIC. It supplies the group traffic key (GTK) in a protected message.
4. The station confirms receipt and installation.

The PMK is not sent in the handshake. Fresh nonces produce fresh session keys from the existing PMK. The GTK serves group traffic, while the pairwise key serves communication between that station and the access point.

Know what the password permits

A captured WPA2-Personal handshake gives an offline guessing test: derive a candidate PMK and PTK, recompute the MIC, and compare it with the capture. A user who already knows the network passphrase can also derive another user's PTK from that user's captured handshake. KRACK exploited key reinstallation and nonce reuse in implementations; patches address that defect.

WPA2-Enterprise uses 802.1X. The station is the supplicant; the access point is the authenticator and relays EAP to an authentication server, commonly through RADIUS. EAP-TLS uses client certificates; PEAP and EAP-TTLS carry credentials inside TLS protection. Users obtain individual keying material and can have access revoked separately. In either mode, Wi-Fi protection ends at the access point; TLS can continue protection to an application server.

A protected wireless link needs a sound way to establish its keys. WPA3 changes password-based Wi-Fi key establishment, while cellular networks start with a subscriber secret held by the SIM and home network.

Improve password-based Wi-Fi

WPA3-Personal uses SAE, Simultaneous Authentication of Equals, to establish the PMK. A captured SAE exchange does not provide the WPA2-style offline password test: each guess requires active interaction. SAE also provides forward secrecy, so learning the password later does not expose recorded past sessions. The four-way handshake then derives traffic keys from the PMK.

WPA3 requires Protected Management Frames (PMF), which authenticate robust management frames such as deauthentication messages. This blocks forged teardown messages; it cannot stop radio jamming. WPA3-Enterprise retains 802.1X and offers an optional 192-bit security mode using stronger algorithm choices such as GCMP-256.

Transition mode accepts WPA2 and WPA3 under one network name. Its compatibility can expose devices to the weaker WPA2 path. In 6 GHz operation, WPA3 or Enhanced Open is required, and WPA2 transition mode is not permitted; WPA3-Personal uses SAE's hash-to-element method.

Enhanced Open, based on OWE, establishes encryption without a password through Diffie–Hellman. It prevents passive listeners from reading frames, but it does not authenticate the hotspot. An evil twin can establish its own encrypted link with the user.

Authenticate a cellular subscriber

A long-term secret $K$ resides in the USIM and the home network's authentication infrastructure: HSS in 4G and UDM/ARPF in 5G. A visited network receives authentication material without learning that long-term secret.

In 4G AKA, the visited network's MME obtains a vector containing RAND, AUTN, XRES, and key material from the home HSS. The phone receives the random challenge RAND and authentication token AUTN. Its USIM checks AUTN to authenticate the network and freshness, then computes RES. The visited network accepts the subscriber when RES matches XRES. Derived keys protect later communication.

This is mutual authentication. Earlier 2G GSM authenticated the phone to the network but did not give the phone an equivalent check of the network, enabling fake-base-station attacks. Mutual authentication still needs secure algorithms and correct key handling; the long-term secret never needs to be sent to the visited network.

Conceal the permanent identity

In 5G, the home network makes the final authentication decision. The phone can also conceal the subscriber-specific part of its permanent identity, SUPI, using the home network's public key to form SUCI. Routing information remains readable so the request can reach the right home network.

Concealment requires a provisioned protection scheme. The null scheme leaves the identity unprotected. Protected SUCI reduces exposure to devices that request permanent identities; it does not make all radio activity anonymous. After registration, temporary identifiers reduce routine use of the permanent identity. Cellular access security still does not protect the full route to an app server: TLS supplies that separate application-endpoint protection.

A firewall enforces a policy between networks or network segments. It works only when the relevant traffic passes through it, and the firewall itself must resist compromise. A stateless filter makes a fresh decision for each packet using header fields.

Read a rule precisely

A rule can match source and destination IP addresses, transport protocol, ports, TCP flags, ICMP type, interface, and direction. Rules are evaluated in order: the first match decides. A final default deny drops packets that no allow rule matched.

For network 203.0.113.0/24, consider a web server at 203.0.113.10:

A SYN from 198.51.100.7:50000 to the web server's port 443 matches rule 1. A SYN to its port 22 reaches rule 3 and is dropped. A reply from the server's port 443 with ACK set matches rule 2. Every field in a rule matters: a packet using UDP port 443 does not match an allow rule for TCP.

Find shadowed rules

Suppose an earlier rule says "allow inbound TCP to any host in 203.0.113.0/24 on any port". A later rule denying SSH to 203.0.113.10 has no effect: the earlier broad allow has already decided. Check rule order along with the words of each rule. Put intended exceptions before the general rule that would otherwise match them.

Default deny is useful because adding a new service does not silently make it reachable. Default allow has the opposite consequence: any traffic not explicitly blocked passes. Neither default corrects a badly ordered rule table.

Recognize the ACK shortcut

A policy can block inbound SYN packets with ACK clear to stop new outside-initiated TCP connections. To permit replies to inside clients, a stateless filter might allow inbound TCP from source port 443 when ACK is set. Trudy can craft both that source port and that flag. The filter has no memory proving that an inside host actually opened the connection.

A TCP ACK flag is a field the sender chooses, not a certificate of an established session. Stateful filtering improves this decision by consulting connection history.

Filter forged addresses

Ingress filtering can reject internet-facing packets claiming an internal source address or a private address that should not arrive there. Egress filtering rejects outgoing packets whose source lies outside the organization's assigned ranges. These checks reduce spoofing of disallowed ranges; they do not identify an individual host or stop a machine impersonating another permitted internal address.

Stateless filters inspect headers, not application meaning. Port 443 does not prove that traffic is HTTPS or harmless. An allowed connection may carry malicious content. Packet filtering therefore works alongside application gateways, endpoint controls, and intrusion detection.

A stateful firewall adds context to packet filtering. An application gateway adds understanding of one application's requests. Network separation limits where a compromised public server can reach.

Track the actual flow

A stateful packet filter records addresses, ports, protocol, and connection state. When an inside client opens a permitted connection, the firewall creates an entry and permits appropriate returning packets. A new inbound connection still needs its own explicit permission. Entries expire when the connection closes or becomes inactive.

Suppose the table records an established TCP flow from 203.0.113.25:51000 to 198.51.100.80:443. An inbound segment reversing that address-and-port pair can match its state. An ACK from 192.0.2.66:443 to 203.0.113.30:52000 does not match the entry. Its source port and ACK flag would satisfy a loose stateless reply rule, but they do not establish membership in the tracked connection.

UDP lacks TCP connection setup. Stateful filters approximate a conversation by tracking recently used address-and-port pairs with a timeout. A DNS query to a chosen server opens a limited opportunity for the corresponding reply, not permission for every packet claiming to be DNS.

Inspect the application request

An application gateway, or proxy, receives and processes application traffic on users' behalf. A web proxy can authenticate a user and check a requested address or content against policy. To enforce that path, the packet filter must block clients from opening direct connections that bypass the proxy.

A proxy has protocol-specific logic, processing costs, and potential bottlenecks. Clients may need configuration, or the network may intercept traffic transparently. Encrypted content remains hidden from a proxy unless it terminates the TLS connection and creates another connection onward. That changes the set of endpoints trusted with plaintext.

Isolate public servers

A DMZ is a separate segment for public services. The firewall permits selected internet-to-DMZ traffic while restricting access from the DMZ to the internal network. If a web server is compromised, the attacker does not automatically obtain unrestricted internal access. Any necessary DMZ-to-internal connection should have a specific policy.

This is defense in depth: several controls reduce different paths of failure. A host firewall still matters when an infected laptop is already inside or another link bypasses the perimeter. A zero-trust design does not grant access simply because a device is on an internal subnet; it authenticates and authorizes requests. The network location can inform policy, but cannot replace identity and permission checks.

A firewall can permit a connection that carries an attack. Intrusion detection examines traffic content and patterns for signs of misuse; intrusion prevention can act directly on those findings.

Choose detection and placement

An IDS usually observes copied traffic and raises alerts. An IPS sits inline and can drop suspicious traffic. Inline placement allows immediate blocking but adds processing delay and risks interrupting legitimate use when a detection is wrong.

Snort and Suricata are examples of signature-capable systems. A rule might recognize the request path ../../etc/passwd, more than 20 new connections to SSH port 22 from one source within a minute, or SYNs to hundreds of ports. These patterns indicate suspicious behavior; a match alone does not prove a successful intrusion.

An anomaly detector might flag a large change in traffic rate or protocol mix. A new legitimate backup job can look unusual too. A false positive is an alert on harmless activity; a false negative is an attack the detector misses. Moving a sensitivity threshold usually exchanges one kind of error for the other.

Calculate what an alert means

Consider 1,000,000 harmless connections and 10 attacks per day. A detector catches 99% of attacks and falsely alerts on 0.1% of harmless connections.

1. Expected true alerts: $10\times 0.99=9.9$.
2. Expected false alerts: $\mathrm{1,000,000}\times 0.001=\mathrm{1,000}$.
3. Expected true fraction among alerts: $9.9\mathrm{/}(9.9+\mathrm{1,000})\approx 0.0098$, or about 1%.

These are expected counts across repeated days, so 9.9 need not be a whole-number daily outcome. The 99% detection rate is the share of attacks detected, not the chance that an alert is an attack. Rare attacks and a huge harmless population explain the difference: this is the base-rate problem.

Put sensors where they can see

Sensors just inside the firewall observe traffic that passed it. DMZ sensors observe public-service traffic; internal sensors can reveal attacks that never cross the perimeter. A sensor must keep up with the traffic it receives. It must also reassemble fragments and TCP segments consistently with the target; otherwise an attack split across packets may evade a pattern check.

TLS and IPsec hide encrypted payloads from a network sensor without decryption access. Addresses, ports, timing, lengths, and unencrypted handshake fields can still help. Payload inspection can occur at an endpoint where decryption happens, but an ordinary copied encrypted stream does not supply that plaintext.

A honeypot is a decoy with no intended ordinary users. Contact is therefore suspicious and can expose scanning or attack techniques with less background noise. Misconfigured clients or harmless automated traffic can still contact it, so a decoy does not make every alert conclusive.