COMP347 (Revision 10) | TrustOpen University
Upon completion of this expanded tutorial, students will be able to:
This tutorial provides a comprehensive, university‑level treatment of the transport layer’s interface to applications. We move beyond high‑level descriptions to dissect the internal machinery of TCP—its sliding windows, RTO estimators, and the evolution of its congestion‑control algorithms. We also examine UDP in detail, including its checksum calculation and how it differs from TCP in handling message boundaries.
In the socket programming section, we explore advanced patterns: non‑blocking I/O, multiplexing with select/poll/epoll, and asynchronous I/O. We then bridge transport services to application requirements through formal QoS parameters (bandwidth, latency, jitter, loss rate). Finally, we introduce QUIC as a modern alternative that combines TCP’s reliability with UDP’s low latency, and discuss its implications for application design.
The transport layer performs multiplexing at the sender (gathering data from multiple sockets) and demultiplexing at the receiver (delivering data to the correct socket).
connected UDP sockets).rwnd (receiver window) advertised in every ACK.cwnd (congestion window) which is dynamically adjusted based on loss and delay signals.| Field | Size | Purpose |
|---|---|---|
| Source/Destination Port | 16 bits each | Identify processes |
| Sequence Number | 32 bits | Byte stream position (first byte of this segment) |
| Acknowledgment Number | 32 bits | Next expected byte (cumulative ACK) |
| Data Offset | 4 bits | Length of TCP header (in 32‑bit words) |
| Flags (ACK, SYN, FIN, RST, etc.) | 9 bits | Control state transitions |
| Window | 16 bits | Advertised receive window (rwnd) – scaled if Window Scaling option is used |
| Checksum | 16 bits | Error detection for header + data + pseudo‑header |
| Urgent Pointer | 16 bits | Location of urgent data (rarely used) |
| Options | variable | MSS, Window Scale, SACK, Timestamp, etc. |
TCP uses a smoothed round‑trip time (SRTT) and the mean deviation (RTTVAR) to compute the RTO. The algorithm (Jacobson/Karels, RFC 6298):
Karn’s algorithm ensures that retransmitted segments are not used to update the RTO estimators (to avoid ambiguous ACKs).
Reno (classic):
CUBIC (default in Linux):
BBR (Bottleneck Bandwidth and RTT):
SACK (RFC 2018) allows the receiver to acknowledge non‑contiguous blocks of data, enabling the sender to retransmit only the missing segments, rather than all data from the lost point (which happens with cumulative ACKs). This improves performance when multiple packets are lost in a single window.
UDP header is fixed at 8 bytes: Source Port (16), Dest Port (16), Length (16), Checksum (16). The checksum covers the UDP header, data, and a pseudo‑header that includes source/destination IPs, protocol number, and UDP length. This ties the datagram to the IP layer, ensuring that if the packet is delivered to the wrong IP, the checksum will fail (providing a weak form of end‑to‑end integrity).
A UDP socket can be “connected” using connect(), which permanently binds the remote address. This has several benefits:
send() and recv() can be used instead of sendto()/recvfrom() (slightly simpler).| Model | Description | Scalability | Complexity |
|---|---|---|---|
| Blocking (synchronous) | Each I/O call blocks until complete | Low (requires threads for concurrency) | Low |
| Non‑blocking + poll/select | Check readiness; loop over file descriptors | Medium (O(N) scan) | Medium |
| Event‑driven (epoll/kqueue) | Kernel notifies ready descriptors | High (O(1) readiness notifications) | High |
| Asynchronous I/O (AIO) | OS handles I/O and signals completion | Very high | Very high |
SO_REUSEADDR – allows binding to a port in TIME_WAIT; essential for fast server restarts.SO_KEEPALIVE – enables periodic probes to detect dead connections (idle timeout).TCP_NODELAY – disables Nagle’s algorithm; sends small packets immediately (reduces latency for interactive apps).SO_LINGER – controls behaviour when closing a socket with unsent data; can be set to discard or block until sent.
TCP is a byte‑stream; a single send() may not transmit all data, and a single recv() may not receive all bytes. Applications must loop until all expected data is sent or received. For receiving messages with variable length, a common pattern is to prefix the message with a fixed‑length header containing the length.
BDP = bandwidth × RTT. It represents the amount of data that can be “in flight” (in the pipe) at any time. For TCP, the window size (cwnd) should be at least as large as the BDP to fully utilise the link. If the window is too small, the link is underutilised; if too large, bufferbloat may occur.
| Class | Examples | Key Requirements | Preferred Transport |
|---|---|---|---|
| Elastic (reliable) | HTTP, FTP, Email | Reliability, moderate delay tolerance | TCP |
| Real‑time (loss‑tolerant) | VoIP, gaming, live video | Low latency, low jitter, some loss acceptable | UDP (or QUIC) |
| Interactive (low‑latency) | SSH, remote desktop | Low latency, moderate reliability | TCP (optimised with Nagle disabled) |
| Bulk (high‑throughput) | File sync, backup | High throughput, high reliability | TCP (with large windows) |
QUIC is a UDP‑based transport protocol developed by Google and now standardised. It provides:
QUIC is the basis for HTTP/3. It offers a single API (similar to sockets) but with stream abstractions. For applications that need reliable, in‑order delivery per stream but want to avoid head‑of‑line blocking, QUIC is attractive. It also simplifies deployment because it uses UDP port 443, making it easier to traverse firewalls (compared to new TCP ports).
ECONNREFUSED: no process is listening on the destination port (ICMP port unreachable).ETIMEDOUT: no response after retransmissions (possibly network down or host unreachable).ECONNRESET: the remote peer sent a RST segment (connection reset).EPIPE (or SIGPIPE): writing to a closed socket; the signal can be ignored with MSG_NOSIGNAL.EAGAIN/EWOULDBLOCK: non‑blocking socket, no data ready or buffer full.
By default, writing to a socket that has been closed by the other end generates a SIGPIPE signal, which terminates the process. In servers, this is undesirable. Use MSG_NOSIGNAL flag on send(), or set SO_NOSIGPIPE socket option, or simply ignore the signal.
Q1: What is the demultiplexing key for a TCP socket?
The 4‑tuple: (source IP, source port, destination IP, destination port).
Q2: In the Jacobson/Karels RTO algorithm, which variable controls the weight of the mean deviation?
\( \beta \), typically 1/4.
Q3: What is the difference between TCP’s flow control and congestion control?
Flow control prevents sender from overwhelming the receiver (rwnd); congestion control prevents sender from overwhelming the network (cwnd).
Q4: What does the UDP checksum cover?
The UDP header, the UDP data, and a pseudo‑header containing source/destination IPs, protocol, and length.
Q5: Name two benefits of a connected UDP socket.
(1) Filters incoming datagrams to only the connected peer; (2) allows send()/recv() instead of sendto()/recvfrom(); (3) receives error notifications (ICMP).
Q6: What is the purpose of the TCP_NODELAY socket option?
It disables Nagle’s algorithm, preventing the delaying of small packets; reduces latency for interactive applications.
Q7: What is the Bandwidth‑Delay Product (BDP) and why is it important?
BDP = bandwidth × RTT. It indicates the amount of data in flight; the TCP window must be at least this size to achieve full link utilisation.
Q8: What is the main advantage of QUIC over TCP?
It provides stream multiplexing without head‑of‑line blocking, built‑in TLS 1.3, 0‑RTT connection resumption, and connection migration.
Q9: What is Karn’s algorithm used for?
It prevents retransmitted segments from affecting the RTT and RTO estimators, avoiding ambiguous ACKs.
Q10: In the TCP state diagram, what is the duration of the TIME_WAIT state?
2 MSL (Maximum Segment Lifetime), typically 60 seconds (2 × 30 seconds).
Q11: What is the difference between select() and epoll() in terms of scalability?
select() scans a fixed‑size bitmap and is O(N); epoll() uses event‑based notifications and is O(1) for ready events, scaling to thousands of file descriptors.
Q12: How does CUBIC differ from Reno in congestion control?
CUBIC uses a cubic function for window growth, making it more aggressive in high‑bandwidth, long‑distance networks, while Reno uses linear growth (additive increase).
Exercise 1 – BDP Calculation
A link has bandwidth 10 Gbps and RTT = 50 ms. What is the BDP in bytes? If TCP window is 64 KB, what is the link utilisation?
BDP = 10×10⁹ bps × 0.05 s = 5×10⁸ bits = 62.5 MB. 64 KB = 512 Kb. Utilisation = 512 Kb / 5×10⁸ = 0.001024 = 0.1%. The link is severely underutilised; window scaling is necessary.
Exercise 2 – TCP State Transition
Describe the transitions when a server closes a connection before the client.
Server sends FIN → enters FIN_WAIT_1. Client receives FIN, sends ACK → enters CLOSE_WAIT. Server receives ACK → enters FIN_WAIT_2. Client sends FIN → enters LAST_ACK. Server receives FIN, sends ACK → enters TIME_WAIT (2 MSL). Client receives ACK → CLOSED.
Exercise 3 – Socket Option Selection
You are writing a high‑frequency trading application. Which socket options would you enable and why?
Enable TCP_NODELAY to minimise latency (disable Nagle). Set SO_KEEPALIVE with short intervals to detect broken peers quickly. Increase socket receive/send buffers to handle bursts. Use non‑blocking I/O with epoll for low‑latency event handling.
Exercise 4 – UDP vs TCP for a Multiplayer Game
Compare the suitability of TCP and UDP for a fast‑paced multiplayer game.
UDP is preferred because it avoids TCP’s head‑of‑line blocking and retransmission delays. Game state updates are time‑sensitive; occasional loss is acceptable. If using UDP, the application must implement its own reliability for critical events (e.g., player death) and sequence numbers for ordering.
Exercise 5 – RTO Estimation Exercise
Given RTT samples: 100, 120, 110, 130 ms (assuming SRTT=100, RTTVAR=5 initially). Compute new SRTT, RTTVAR, RTO. α=1/8, β=1/4.
First sample: SRTT = 0.875*100 + 0.125*120 = 102.5. RTTVAR = 0.75*5 + 0.25*|102.5-120| = 3.75 + 4.375 = 8.125. RTO = 102.5 + 4*8.125 = 135 ms.
Exercise 6 – Handling SIGPIPE
Why does a server need to handle SIGPIPE? How can it be disabled in Linux?
SIGPIPE occurs when writing to a socket that the peer has closed. It can terminate the server. In Linux, use MSG_NOSIGNAL flag in send(), or ignore SIGPIPE with signal(SIGPIPE, SIG_IGN), or set SO_NOSIGPIPE on the socket.
Homework 1 – TCP Congestion Control Simulation
Simulate (conceptually or using a script) the evolution of cwnd over time for Reno and CUBIC under a scenario with occasional packet loss. Plot the differences.
Use the TCP‑friendly equations; illustrate the sawtooth pattern (Reno) vs the cubic growth (CUBIC).
Homework 2 – Socket Performance Benchmark
Write a simple echo server in your language of choice, and benchmark it with thread‑per‑connection vs event‑driven (epoll) under high concurrency (10k connections). Report the resource usage and throughput.
Use tools like `wrk` or `ab`. Measure CPU, memory, and latency percentiles.
Homework 3 – QUIC vs TCP Performance
Research the performance of QUIC compared to TCP for web traffic. Write a report on latency, throughput, and connection establishment times in different network conditions (fixed, mobile).
Look at the Google QUIC whitepaper and recent IETF drafts.
Homework 4 – UDP Reliability Layer
Design a thin reliability layer on top of UDP that provides: (a) retransmission on loss, (b) ordering, (c) flow control. Describe the packet format and state machine.
Model it after TCP’s simplified logic; use sequence numbers, ACKs, timers, and a sliding window.
Homework 5 – Socket Error Case Study
You observe `ETIMEDOUT` on a client connection. Trace the possible causes at the network and transport layers, and propose diagnostic steps.
Causes: packet loss (no ACK), firewalls dropping packets, server not responding, or asymmetric routing. Use `traceroute`, `ping`, and `tcpdump` to diagnose.
This expanded tutorial has provided a deep, university‑level understanding of transport services and their interface to applications. Key insights include:
These concepts are foundational for the remaining tutorials, where we will see how specific application‑layer protocols (HTTP, SMTP, DNS) choose and use these transport services to achieve their goals.