Updated 11/22/2009 for clarity
The answer to the question in the title defines the battle over the soul of the Internet and the definition of “Net Neutrality”.  While few people can give you a concrete or consistent definition for the meaning of Net Neutrality, most people can probably agree that a “neutral network” should be a “fair network”.  Yet for some reason, many proponents of Net Neutrality want to define the “neutral network” as a “dumb network” that operates on a first come first serve basis, and they have managed to get the FCC to prohibit good network management techniques.

The dumb network proponents came to this conclusion because of a misunderstanding of the concept called end-to-end due to the misguided notion that the architectural principle of the Internet requires that the network remain minimalistic and dumb while all the intelligence and innovation (by extention business opportunities) happens at the edges of the network.  But the reality is that the first principle of the “Architectural Principles of the Internet” is that “The principle of constant change is perhaps the only principle of the Internet that should survive indefinitely”.

Does neutrality and fairness exist in an unmanaged network?

People fundamentally like the concept of “neutrality” which seems unbiased and fair.  The problem is that neutrality and fairness is never guaranteed in a dumb unmanaged network that treats the users and applications on a first come first serve or First In First Out (FIFO) basis.  I didn’t need to become a network engineer to understand that fairness isn’t a given under an unmanaged environment and it was a painful lesson that I learned as a 10 year old in summer camp being the youngest and only minority in the dorm.  When it came time to choose a bed, I was informed by the older kids that the spot on the floor was mine and that the empty bed was actually for the backpacks until the camp leader came in and settled the situation.  When it came meal time, the older kids grabbed almost everything until the camp leader reallocated the food so that everyone had something decent to eat.  Needless to say, this was an experience that I would not forget.

As it turns out, networks are very similar to that summer camp.  The difference is that there often isn’t a camp leader, and some users and applications can starve other users and applications of their equitable access to the network.  Not only do some applications and users hog 10 or more times the bandwidth of other applications, they can even take hundreds or thousands of continuous turns getting their own packet delivered while the packets of other applications sit in the router queue (think of the line at a bridge crossing) and wither on the vine.  The scientists and engineers who designed the networks that connect to and comprise the Internet have been aware of these issues from the beginning.  They created Internet standards groups that attempted to evolve the Internet and improve the efficiency, performance, and fairness of the network.

Engineers and Scientists make lousy politicians

There is a good reason why there are so few engineers and scientists who hold political office.  Marketing and politically savvy has never been an asset for the majority of them nor do their professions require these skills.  None of them ever imagined that they would have to debate the finer points of network engineering in the public.  Through habit, they gave cryptic names to their enhancements on the Internet which could ensure network efficiency and fairness, and without a thought they used politically incorrect words like “prioritization” which sounds like unfairness and bias to the layman when the opposite may be true.

Meanwhile, the activists and lawyers through a combination of technical ignorance and perhaps other self serving reasons have banded together to outlaw prioritization on networks.  They even came up with clever sounding marketable names like “Network Neutrality” under a misguided interpretation of end-to-end.  Some of them even wrote up pseudoscience papers about the dangers of prioritization lined with inaccuracies from start to finish when the data shows otherwise.  But lost in this whole debate is the true meaning of neutrality in the network, and it’s time we stepped back and ask ourselves how a neutral and fair network should really behave.

How should a network achieve fairness and neutrality?

Network Engineers know that a neutral and fair network does not come from a primitive First In First Out (FIFO) state because applications don’t behave in a neutral way.  After much consideration, I propose the following principles which I have previously described as the universal rule for reasonable application priority, and I welcome suggestions on how it could be better.  Note that I am not suggesting that these principles be mandatory for Internet Service Providers (ISPs), but that these might be considered best practice guidelines for ISPs that wish to offer something better than best effort service.

Definition of fair and neutral network

• Equitable bandwidth sharing for all applications
  In the absence of explicit instructions and preferences from the user of the network (which is almost always the case in practice), the network should strive to give each application as much capacity as it wants up to an equitable share of available capacity.  If for example four applications are in use and each wants as much capacity as possible, each application should have a quarter of the available capacity.  If the applications wants less than an equitable share, then the network should ensure that it gets what it wants and protect it against more aggressive applications while splitting its unused share among other applications.  If the application demands more than its fair share such as BitTorrent, then the network is justified in limiting it to an equitable share so that other applications aren’t deprived.
• Users can request higher preference for some applications
  In some instances, the subscriber may need to signal to the network that a particular application needs to be given priority and the network should honor these requests.  This does not mean the end user can demand prioritization beyond the service parameters of what they paid for.  For example, business grade Internet access plans limit maximum priority packets to a very small percentage of traffic and this is defined in the service contract.  The network can refuse to honor requests that exceed these contractual limits.
• Equitable bandwidth sharing between broadband subscribers
  Networks should strive to ensure equitable sharing of capacity between subscribers within the same service tier.  This should not preclude customers from buying a larger share of the network and higher bandwidth levels.
• Ensure that no application starves
  In addition to equitable bandwidth distribution, the network should also strive to eliminate short term starvation called “jitter” and minimize the harmful effects of jitter for all applications.  The easiest analogy for non-engineers is that a person can suffer long term starvation or short terms starvation.  A person can starve in the long term if he is given too little food per month, but he can also starve in the short term if he is given plenty of food for the month but it’s distributed such a way that he is given plenty to eat the first 20 days but receives nothing the following 10 days.
  Applications are like this but operate on a millisecond (one thousandths of a second) to second scale.  Some applications can’t tolerate more than 80 millisecond jitter while others can tolerate 2000 milliseconds of jitter.  A P2P file transfer application is the most tolerant to short term starvation and it is like the camel that needs to eat large quantities of food but it can go days or weeks with nothing.  A VoIP or online gaming application is like an infant that eats very little but it needs to be fed every few hours, but a P2P application can consume 100% of the network resources for hundreds or thousands of milliseconds at a time.  A network should be engineered in such a way that every application gets what it needs to operate smoothly.  That means jitter needs to be kept at an absolute minimum for VoIP or online gaming applications, and jitter should be kept within 1000 milliseconds for file transfer applications.
• Intermittent use of the network should be prioritized over persistent usage
  Prioritizing applications (by extension the users using these applications) that have very small requests over applications with very large requests is a reasonable way to manage the network.  If we allow prioritize an application like web browsing which only needs a small amount of data, we can shorten its page load time from 10 seconds to 1 second.  This slows down persistent continuous-use applications like P2P while the web browser is loading a web page, but we’re slowing P2P down for a shorter period of time because the web browser loads its webpage sooner.  The net effect is that there is no difference in the total time it takes to transfer a file using P2P if had to cede priority to short-duration applications, but the short-duration applications get a dramatic improvement in performance.  See this illustration for a visual representation, and see definition of bandwidth below.

Definitions

• Capacity – Measured in bandwidth or packet transmission opportunities over the network.  Available capacity is what the user contracted for which is frequently shared.  Broadband users always have fractional ownership of a network.  Available capacity can vary and  depends on the number of users active on the shared network and the type of applications they are using.
• Bandwidth – The rate of data transmission per unit time.  Unit time does not necessarily need to be measured by the second and it could be measured by the minute.  This is a critical distinction that would protect users or applications with short duration intermittent traffic patterns.  If we are trying to get a little more balance in the number of bits transferred per minute, that would justify giving higher packet forwarding priority to low duration users and applications over users and applications that continuously use the network.  See figure 1 here for additional details.