There is a common myth that the Internet is some level platform that treats all communications platform the same when the reality is nothing of a sort.  There are built-in biases against certain types of protocols and the applications that use those protocols, but there are fair and intelligent ways of eliminating these protocol biases.  There are also biases against long distance file transfers because Transmission Control Protocol (TCP) sets a much lower “speed limit” for long distance file transfer than short distance file transfer, but the result of this TCP bias seems to be desirable from a network design standpoint.

What is latency

At the fundamental level, light travels at approximately 123 miles for every thousandths of a second in fiber optic glass which is half of its speed in free space, and the time it takes light to travel is responsible for the majority of latency experienced on the Internet.  Latency over the Internet is normally measured as Round Trip Time (RTT) in milliseconds (a thousandths of a second) which is the time it takes data to travel from one computer to the other and back.  For example, the RTT between Boston and California is 80 milliseconds.  A more detailed discussion of latency can found here.

Dealing with the impact of latency

High latency is very undesirable for online game play as it is difficult to react quickly when the interaction between players are delayed for fractions of a second.  High latency is also problematic for real-time voice or video communications as it becomes noticeable when RTT exceeds 200 milliseconds.  It is very unnatural to speak to a person where the they can’t respond for a second because of the time it takes for digitized sound to reach the other person.

Online gamers can get around the latency problem by playing on nearby gaming servers, but this only applies to ad hoc gaming where each server might have 32 players.  Massive Multiplayer Online (MMO) games which have to support hundreds of thousands of players from all over the world don’t have this option of using nearby servers because the players are spread out all over the world, so MMO gaming companies like Blizzard will partner with Network Operator like Teliasonera that can provide enhanced priority services that minimize latency.

Web applications that are poorly tuned for global access will also experience problems if they require too many round trips for each user action.  A web application that requires 200 client-server interactions when a user clicks something is going to be delayed for at least 20 seconds if the latency for each interaction is 100 milliseconds since 0.1 seconds times 200 is 20 seconds.  Fortunately for web developers, they can either optimize their applications by batching their requests into fewer round trips or they can employ remote terminal solutions like Citrix to get around the latency issue.

Latency dictates the TCP speed limit

Most applications (including the web browser) on the Internet use Transmission Control Protocol (TCP) to communicate, and latency determines the maximum throughput (bandwidth) that these applications can communicate at.  TCP is designed such that applications start their transmission of data slowly to avoid overwhelming the network and then ramp up the transmission rate when it receives confirmation (an acknowledgement) from the receiving side that its packets arrived intact.  TCP is designed such that it can transmit a maximum of 64 kilobytes (KB) of data in a single “Window” without any acknowledgement from the receiving end.  That means if the receiving side didn’t get the original transmission and needs retransmission, no more than 64 KB of transmission capacity was wasted and this is a good design characteristic of TCP.

However, this 64 KB windowing limit means that higher latency links have a smaller maximum transmission rate.  In fact if each transmission and receive acknowledgement requires 100 milliseconds (or 0.1 seconds), then we are limited to a maximum of 10 acknowledgements per second.  That limits our maximum bandwidth to ten 64 KB transmissions per second which works out to 5120 kilobits per second (Kbps).  If the round trip latency is 200 milliseconds as in the case of intercontinental communications between the United States and Europe, then our peak TCP transmission rate is cut in half to 2560 Kbps.  For close range communications such as the edge caching servers where latency is typically around 16 milliseconds, our peak TCP transmission rate is 32,000 Kbps or 32 megabits per second (Mbps).  Note that practical limits are a bit lower than these theoretical limits.

Note: I am using a simplified decimal conversion from kilo to mega by dividing by 1000, and this casual conversion method is used by most of the industry.  Computer science purists prefer to use binary-nice conversion factor of dividing by 1024 which is confusing to most people.

TCP speed limit encourages more efficient communications

TCP can potentially allow for much faster transmissions by using techniques like window scaling where multiples of the window size is used, but that only multiplies the speed limit but it does not eliminate the bias against longer distance communications.  However, this built-in bias against long distance communications is desirable in network design because it encourages short distance communications.  By definition, a data transmission that uses 1000 miles of network infrastructure takes 20 times more resources than a transmission happening over 50 miles of network infrastructure.  Encouraging content providers to build closer resources saves valuable network capacity on the long distance links which have to support millions of devices on each end.  This is precisely why Content Delivery Networks (CDN) have proliferated in recent years and it has made video on demand a reality on the Internet.

Working around the TCP speed limit

There are a few ways to bypass this TCP speed limit.  One method is to employ parallel TCP communications by using multiple TCP flows.  Each TCP flow gets the full speed limit so if there are 10 TCP flows, the speed limit is effectively raised 10-fold.  Peer-to-peer (P2P) applications are the best example of this, but they break the fairness of TCP congestion control and they cause far more network congestion than traditional single-flow TCP applications.  Furthermore, most Internet users prefer to stay with the familiarity of their web browser to download content which doesn’t support multiple flows for a single file transfer.

Like gamer developers partnering with network providers that provide premium low-latency services, content distributors get around the TCP speed limit by partnering with Content Delivery Networks (CDN) or they build their own “edge caching” networks like Google if they have the massive scale to justify it.  By bringing content closer to the end user, it vastly reduces the latency which increases the maximum TCP transmission rate.  Furthermore, short range communications is less likely to experience congestion because it goes through fewer fewer connections with fewer opportunities to get congested.  More importantly, bringing content to the edges allows content providers to negotiate cheaper short distance connectivity to a single ISP (called Paid Peering) and bypass the higher expensive and higher congestion levels of traditional Internet transit service.

Other applications get around the TCP speed limit by not using TCP at all.  User Datagram Protocol (UDP) is an alternative lighter weight protocol that lacks some of the more sophisticated features of TCP such as error correction but has the benefit of lower overhead and simplicity.  Some applications like Voice Over IP (VoIP) or video conferencing don’t need error correction because they play everything live and in real-time and they don’t have time to retransmit on errors.  They simply ignore data loss and expect the user to repeat themselves if needed.