Home » CurrentHeader, Internet, Wrong On The Internet

Division of labor between broadband and CDN

By 14 December 2010 17 Comments

Updated 12/16/2010 for clarity.


In the recent peering dispute between Level 3 Communications and Comcast, Level 3 argues that it should not have to pay Comcast anything for an additional 300,000 Mbps of private peering capacity because Level 3 provides value by delivering video content over its nationwide backbone.

(See video about how this dispute started and how the peering relationships work)

Level 3 Vice President Mark Taylor argued in a comment on my blog that Comcast is merely the equivalent of an apartment complex landlord trying to impose an unjustified fee from Level 3 to deliver the video bits a short distance to its tenants.  Taylor argues that because Level 3 handles the nationwide delivery while Comcast handles the short haul delivery, then Comcast should be happy with the free exchange.

While it’s true that Level 3 does the long distance delivery, what Taylor doesn’t tell you is that he’s only delivering original copies and then he expects to hand off 1000 replicas to Comcast for regional delivery.  Imagine the cost of shipping 10 DVDs to ten distribution centers in North America.  Even though that costs a lot per shipment, it’s only 10 shipments.  But once the DVDs reach the distribution center, 1000 copies are made and each copy has to be delivered to nearby cities and towns.  The cost of delivering each copy regionally is less, but there are 1000 times more deliveries.

Level 3 is handling the nationwide delivery of those 10 DVDs and they want to hand 10,000 replicas to Comcast for regional delivery.  Then Level 3 wants to sell a cheap door-to-door delivery service to Netflix but they don’t want to pay Comcast to do 90% of the delivery work.  They say that Comcast should charge all of its broadband customers to cover the upgrade costs imposed by Level 3′s CDN service even though most broadband customers aren’t using Netflix.  By that rationale, Level 3 should be demanding free USPS postal delivery and expect every tax payer to pick up the tab.

Quantifying the division of labor between CDN and broadband

When it comes to on-demand video delivery, Level 3 is handling the long distance service to deliver the Netflix on demand videos, but that’s only a small part of the story.  The video data delivered by Level 3 is merely the original copy of the video being sent to roughly 10 locations called Internet Exchanges spread out across the nation.  Each copy will be stored on servers at 10 replication centers and each copy will be replicated an average of thousands of times for localized delivery, but we’ll use a round 1000 replications for our estimates.

Comcast’s network in the United States is 747,000 route miles (Source: Comcast’s Network Brochure). If we remove 47,000 miles as the estimate for the long haul network because Level 3 has that covered, we’re still left with 100,000 route miles of local fiber and 600,000 route miles of Cable coax plant.  Of course the delivery distance is much shorter because each copy of video doesn’t traverse the entire 700,000 miles, but easily more than 100 miles since we’re using 10 distribution centers to reach the cities and towns.  We’ll use 100 miles for the purpose of estimating broadband delivery distance.

Level 3 has 42,000 route miles of fiber plant in the United States (source: Level 3 10K filing), but each original copy of video isn’t traversing the entire 42,000 miles.  It only needs to copy files from the nearest data center which would likely be less than 500 miles away but we’ll call it 1000 route miles (average distance from center of US mainland to US border) to deliver original copies for the purpose of our estimate.

To quantify the amount of work between Level 3 and Comcast, Figure 1 below illustrates the differences.

Figure 1 – Division of labor between broadband and CDN

Who does more work?

Note that the metric used in Figure 1 to define the “load” or the work being done by the broadband and CDN provider is Gbps * route-miles.  We can estimate that the workload for the broadband provider is easily an order of magnitude higher.  Furthermore, the upgrades required by the broadband provider is very expensive because they have to upgrade their entire broadband network.  The broadband provider has to cover the Internet Exchange sites, the backhaul, and the last-mile Cable DOCSIS broadband network.

Level 3 or any CDN provider on the other hand doesn’t need to do nearly as much to upgrade.  CDNs only need to multiply their server capacity and the number of 10-Gbps Ethernet switched ports at the 10 peering sites.  The existing backbone can already handle 300 Mbps and probably a large multiple of that since it doesn’t need to deliver the original copies in real-time.  That means CDN providers could squeeze 10 times the data at 10 times slower if necessary because it doesn’t matter if it takes over night to push out original content to the 10 distribution centers.  The broadband networks don’t have the option of delivering movies at 1/10th real-time since people aren’t going to sit around and wait 15 hours of video buffering before they can watch a 90 minute movie.  They want to watch it whenever they demand it and they expect to view the movie without a hiccup.

What did the broadband consumer already pay for?

Level 3 argues that the broadband consumer already paid for the broadband networks, but the reality is that consumers paid for fractional broad access and fractional transit access to the Internet.  Broadband consumers certainly didn’t pay to cover large companies that expect to get 300 Gbps of private peering data center connectivity on the consumer’s tab.  The reason broadband bandwidth sell at 1/20th the cost of a commercial dedicated Internet connection is because each Mbps of capacity is shared between 20 broadband users.  That means broadband only works when a small fraction of users are using the network at any given moment.

But video on demand turns that assumption of small fractional usage on its head because streaming video demands constant and long duration high capacity.  Anything beyond a small percentage of broadband users using something like Netflix at the same time will trigger the need to upgrade the backhaul and the Cable DOCSIS node which gets to be very expensive.  Since CDNs are the ones demanding all this extra capacity from the broadband provider and expecting the broadband provider to do more than 90% of the delivery work, it is understandable why a broadband provider like Comcast refuses to give away 300 Gbps (and rapidly rising) of peering capacity.

To be concise, broadband pays for occasional burst capacity while CDN services will pay for sustainable capacity.

The value of peering and transit should both be paid for

The value of broadband peering is generated by the broadband provider doing more than 90% of the work on behalf of the CDN provider and it doesn’t matter if that CDN provider also runs an Internet transit business.  If Level 3 CDN wants to benefit from the value of a high capacity peering interconnect at great expense to the broadband provider, Level 3 CDN will need to pay for it like all the other CDN providers before it.

Level 3’s argument that their Internet backbone is valuable is irrelevant to the peering debate because they’re already being compensated for the Internet transit services they provide.  Transit services invested much in the Internet’s backbone and broadband providers pay (or trade something of value) to use these transit services in addition to the fact that content/application/service providers also pay to use transit services.  This is why Comcast pays Level 3 for transit service for the value it provides but that has nothing to do with the peering service that Comcast provides to Level 3.

The economic reality of peering and transit exist because it makes sense to the market and has resulted in the rapid expansion and investment in the Internet.  Trying to turn the peering market upside down will only benefit the free loaders and hurt the Broadband companies that invest the most in our broadband networks and the most in our Internet economy.


Net Neutrality should be about stopping broadband providers from arbitrarily imposing additional fees on Internet companies that already paid their Internet transit fees through the use of expressed or implied threat, but it shouldn’t outlaw fees on voluntary services with value.  Private peering capacity offers definitive value and it is completely different and separate from the Internet.  The argument that the “open” transit Internet is insufficient so CDNs are somehow entitled to a free private fast lane at someone else’s expense through the force of government, and under the banner of “Net Neutrality” is nothing but the blatant theft of property.  Net Neutrality should stop unreasonable tolls, but it shouldn’t outlaw legitimate services while increasing broadband prices and harming investment.

Richard Bennett of ITIF.org provided the original research on the route miles operated by Level 3 and Comcast.  I highly recommend reading his article “Now Playing: Video over the Internet“.