The Akamai Speed Report Is Skewed, Too
The most recent State of the Internet report from Akamai offers some things to ponder about US broadband, but even more to ponder about the usefulness of such studies.
First, the good news. Of the 100 cities Akamai cites for having the fastest average connection speed, not one of them is in France. France, of course, is home to some onerous unbundling/open access laws and is held up as a model for the US to follow.
Now I could use this report to counter the negative perception of US broadband, and specifically push back against French style open access. That’s what Free Press would do in my shoes. There is even a case to be made for why I should.
Since Akamai uses actual connections to its servers to measure the connections people are using, the study stands in stark contrast to the “advertised” speeds most often cited in studies like the OECD’s. The difference between advertised an actual becomes important given the tendency of US companies to hew closer to the truth in their claims.
When looking at the actual speeds to gauge the cities with the fastest average, twenty-one are in the US – including six of the top ten. Again, how many of the top 100 cities are in France? Not a single one. I just can’t make that point enough.
So clearly the Akamai study makes the US look good and France look unimpressive. Right? We have 21 of the fastest cities and they have none.
The Trouble With the Numbers.
Unfortunately, like most other studies of broadband speed, the Akamai report has its own skew – just skew of a different type.
Look closely at that list of cities and you start to see a clear pattern. Of the top 10 cities, all are home to major universities. In fact, just about every city listed outside of Japan is home to a large university. In addition, the cities are relatively small cities (typically 100k to 150k population).
In other words, due to the small size and high number of students, a disproportionate percentage of the population are getting their connections… CORRECTION: their very fast connections… from campus networks. Student connections to the university networks through dorms are skewing the average speed calculations and shooting college towns to the top.
If Akamai could eliminate the bias toward colleges, it’s entirely likely the list of cities would change, and the average connection speeds would drop substantially. Or would it?
That actually raises another confounding variable. Akamai’s study fails to distinguish between wired and wireless – and that makes a difference. South Korea’s average speed, for instance, dropped 24% largely due to the rollout of the iPhone and substantially lower mobile speeds.
Underlying data can quickly muddle the results of your report. For Akamai, that makes any statistically sound assumptions about which cities are actually the fastest fairly difficult to defend. Akamai clearly demonstrates which colleges have the fastest Internet, and the people at Duke should be proud of themselves. But the report doesn’t actually give us anything we can use to compare cities internationally because the data confound their own report.
Lies, Damned Lies, and Statistics
When I was at NCTA, I wrote a series of posts about the OECD study and the multiple flaws with their methodology – including the use of advertised speeds/prices, exclusion of direct fiber/T1 lines for businesses, the use of “per 100 inhabitants” as a unit of measurement, etc.
The OECD study, often cited by doomsayers as evidence of our great broadband demise, is rife with methodological problems. Yet it gets waved about as evidence of our inferiority just as the Akamai study may be waved about as evidence of our superiority.
The truth, however, is somewhere in between. Just as their is clear evidence that Japan has fast connections (after all, just under half of the top 100 cities in the Akamai report are in Japan), there are also clear indications that the notion of $33 gbps connections that come with TV, cellular, and a new car are exaggerated.
Looking beyond that, especially as a navel-gazing exercise for the FCC, I wouldn’t pursue Title II based on the weak evidence offered for open access/common carrier. The evidence simply isn’t there for such a radical shift in policy.
If you insist on trying to take something away from these reports, here is my suggestion. There is no measure that accounts for every confounding variable, precludes ISPs from fudging the numbers, and results in a crystal clear picture of broadband speed or a global comparative framework. When we have that, we may be able to accurately compare different broadband regimes. Until then, I wouldn’t hold my breath.
Michael,
What about broadband penetration and broadband choices? It seems unfair to lob a grenade at France like that on just one metric.
I was being facetious. The point to the post was using any of these studies to say we’re good/bad and they are better/worse is folly. There are so many confounding variables that skew the studies dramatically.
To make such a simple argument (we rock and France is behind) is obviously an oversimplification. There are a wide array of issues, technologies, and other factors at play. That’s why these studies should be seen as interesting at best – but should not form the basis of policy decisions.
As an example, the Phoenix Center did a study that used the OECD formula and assumed every home and business in the world had a connection. They called it broadband nirvana. Rather than ending up in a situation where all countries were tied for first, OECD, under that formula, would actually see the US drop from 15th to 20th in the world.
Yet that report is waved about by doomsayers claiming the broadband sky is falling – knowing full well if everyone hit 100%, we would actually drop 5 places.
Akamai’s report definitely has its limitations, and I’m sure the authors of that report would freely admit to them unlike some of the more bogus reports out there. But I think it’s import to say that the Akamai report is the *least* skewed because it’s based on actual data from real applications.
That’s certainly true. It may be biased heavily toward universities, but those are definitely still connections. I’m not even sure if those connections are counted under the OECD formula. They discount direct fiber and T1 lines for business, so I suspect the university networks are left out as well.
It doesn’t matter what the OECD factors or not; their numbers are so flagrantly bad that none of it matters. They’re not doing any kind of measurements and only taking advertised data. Even then, it’s still way too inflated. Do we honestly believe that the average subscriber in Japan subscribes to an average of 90 Mbps advertised service?
Also worth pointing out: not surprising that the highest speeds in the US are in cities with big universities, because university networks are highly managed!
As the author of Akamai’s State of the Internet Report, I appreciate your feedback. As you point out, we don’t differentiate between wired and wireless data within the underlying measurements, and that can cause some skew. However, going forward, we will removing data from wireless connections from the underlying source data, which should help address the issue you raise. (And we’ll continue to increase our coverage of wireless connectivity in a separate section of the report.)
We recognize that there are a number of studies published in this area, and our goal is to contribute to the knowledge base around global Internet connectivity based on the activity we see across our globally deployed server network.
Finally, as we continue to examine the data, it is clear that there is a strong bias towards college/university locations, and we do call that out in the report. We’ll continue to work on extending our data analysis to see if we can reduce the impact of that bias.
I live in France, in a university town, 100 yards from the university; the speed of the broadband is definitely way below average. It is always a pleasure to travel to Asia where the info seems to jump on the screen without that deadly pause. It may well be anecdotal and unscientific but certainly real!
Leave your response!
Twitter Feed
About Us
Digital Society is a digital think tank that believes culture and commerce are inseparable, that the digital economy flourishes when people are free and rights are secure, and that free markets free people.
Digital Society is an independent 501(c)3 non-profit organization, funded by donations from Jon Henke and from Arts+Labs. We advocate for a pro-culture, pro-commerce digital society through research, analysis and debate on emerging technology issues.
Reply Comments
Transparency and interactivity are trademarks of the Internet era, and we aim to foster them here at Digital Society. It is inevitable that some people will disagree with the technology policy positions we take. We want to have that constructive debate.
The Reply Comments feature gives our critics a chance to respond to our viewpoints and the Digital Society audience convenient access to competing arguments. Any time we directly challenge the views of an individual or a group on this site, the party in question may substantively respond in a guest post.
Please contact executive director Jon Henke by e-mail.
Subscribe
Daily Digest Email
Recent Posts
Recent Posts
Most Commented
Most Viewed