UPDATE 1/6/2011 – Ookla data debunks FCC report – US ISPs exonerated

To simply a recent issue that has come up with an analogy, let’s imagine the following fictional scenario.

The year is 1996 and a research firm runs some studies on the performance of fax machines and found that the average fax machine operated at 3000 baud (bits per second) while the average fax machine was advertised at 6000 baud.  But based on these statistics, the research firm concludes that fax machines on average only run at half their advertised speeds.

They put out colorful graphs showing their research along with some fine print explaining that the measured speeds are actually determined by the slowest fax machine on either end of the connection and that it isn’t necessarily the fault of the fax manufacturer.  The FCC cites this alarming data and the media jumps all over it with the blaring headlines like: “Your fears confirmed: fax machine rated speeds are bogus“.  The fine print is sometimes also included in the news stories but the damage is done and people are already calling for heads to roll at Fax Manufacturing companies.

In the above scenario, newer fax machines ran at 9600 baud (9.6 kilobits per second or Kbps) but they’re often forced to connect to other older or cheaper fax machines at 2400 baud because that’s all the other fax machine can handle.  Does that mean the new fax machine is ripping off consumers by claiming 9600 baud capability while delivering an average connection rate of 4800 Kbps?  No, that would be silly.  Yet this lesson is either lost or ignored by the FCC and journalists like Nate Anderson who wrote in yesterday’s ArsTechnica headline “Your fears confirmed: “up to” broadband speeds are bogus“.  While it’s understandable that consumers might conflate broadband and Internet performance, the FCC and ArsTechnica should know better.

Anderson cites a new FCC study titled “Broadband Performance” (which really should have been titled “Internet performance”) and quotes:

“Therefore actual download speeds experienced by US consumers appear to lag advertised speeds by roughly 50 percent”.

The FCC study cites average data from ComScore and Akamai but while useful for studying “Internet Performance”, this isn’t the same thing as “broadband performance”.  Internet performance involves the lowest common denominator like our fax machine example above and either end point or any particular segment in the middle of the Internet can cause performance to drop.  Sometimes, an invisible TCP speed limit penalty (figure 1) and congestion recovery penalty that worsens proportionally with the distance between the two end points is responsible for the sub-optimal performance.  That doesn’t mean the broadband provider is giving you “bogus” performance numbers because they’re not the ones limiting your speed.

Figure 1 – Effect of latency on maximum TCP throughput

There are examples in the UK where broadband providers are at fault when they only deliver a DSL connection that is only capable of reaching 59% of the advertised performance when DSL actual data speeds should operate at around 85% of the advertised sync rate (sync rates include a bunch of protocol overhead).  The UK broadband example could actually be blamed on the broadband provider but the US study is unfairly blaming problems with the Internet on broadband providers.  To argue that consumers have a right to be confused is irresponsible and simply perpetuates ignorance.

American broadband consumers do get accurate advertised speeds for the most part and people can measure for themselves at speedtest.net which uses a nearby server.  Every test I’ve seen that actually measures broadband performance especially on higher end broadband services like Verizon FiOS (FTTH technology), AT&T U-verse (VDSL2 FTTN technology), or cable broadband do in fact peak extremely close or even exceed advertised performance.  But the speeds consumers get on the Internet are going to be significantly different from what their broadband connection is capable of.  For example, my DigitalSociety.org server on the US east coast is fully capable of delivering data at 13 to 25 Mbps to the east coast, but the TCP speed limit caps West Coast performance to just over 5 Mbps even even though the broadband connection is capable of 15+ Mbps and even when there is no congestion anywhere in the loop.

The good news is that unlike the fax machine that is stuck at a lower baud rate when being limited by a slower fax machine on the other end, broadband can be put to full use even when connecting to resources that are capped by an external non-broadband factor.  You can’t connect a fast fax machine to two slower fax machines at the same time but you can connect a fast broadband connection to multiple slower websites.  The trick is to simply download from multiple websites at the same time and the user’s broadband connection could be used to its fullest.  So for future “broadband” testing, the methodology should include concurrent download aggregate performance because that is a real-world benefit of broadband and it is more likely to show true broadband performance.

Update 8/19/2010 - In response to criticism in the comment section that this article doesn’t go into enough depth about all the issues surrounding broadband performance, I should point out that I wrote one of the more thorough discussions on broadband performance and transparency nearly a year ago.