Theoretical Internet speeds in fixed line and mobile networks keep rising, which is a nice thing. Unfortunately, round trip delay times are not decreasing at the same rate, mostly due to physical limits these days which at some point will limit the maximum throughput of a single TCP stream no matter how fat the pipe is and how fast a server can deliver data. So where is this limit?
When transferring data with the TCP/IP protocol the receiver of a packet needs to acknowledge proper reception. This acknowledgement takes some time to reach the receiver. As the sender can’t wait for the acknowledgement before the next packet is sent there is a transmission window size that governs how much data can be unacknowledged before the sender stops transmitting. The window size is dynamic throughout the life of the TCP connection and increases over time if required. The maximum size configured by default on my Linux machine is around 5 MB. You can check on your own machine with:
cat /proc/sys/net/ipv4/tcp_wmem cat /proc/sys/net/ipv4/tcp_rmem
Think about that for a second, 5 MB of data can be “in flight” without having been acknowledged. Sounds like a lot!? Perhaps for low speeds but not once we start talking about speeds of several hundreds of megabits per second through the Internet to a server far far away.
As already stated above, the limiting factor for throughput is the round trip time required to send the data from sender to receiver and for the receiver to send back the acknowledgement to the sender. In practice, typical round trip times in LTE and DSL networks are 40 ms for traffic to/from servers somewhere in your own country. If, for example, the maximum throughput to a server is 100 Mbit/s, the required TCP window size with a 40 ms round trip delay time is (100.000.000 (bit/s) * 0.04 (seconds)) / 8 (bits) = 500.000 bytes, i.e. around 0.5 MB. Still far below the the theoretical maximum configured on my notebook.
Let’s say there is a server and a network that could deliver data at 600 Mbit/s with a round trip delay of 60 ms. The TCP window size required for that would be (600.000.000 (bit/s) * 0.06) / 8 (bits) = 4.500.000 bytes. That’s pretty close to my buffer limit already. If the round trip delay time increases only slightly my download rate will be limited below this value.
One way of fixing this is using several TCP streams simultaneously as there is one stream per buffer. But when transferring a large file, for example, that’s difficult to do in practice because most software wants to transfer a single file in a single stream.