Home » Internet

Analysis of BitTorrent uTP congestion avoidance

By 2 November 2009 47 Comments

uTP-network-friendlyThe story that the new BitTorrent client uTorrent 2.0 is “network friendly” is making the top headlines on the Web and mailing lists.  The only problem with this story it that it has no actual data to back up its assertions.  I took the time yesterday to run some tests on the new uTorrent 2.0 beta build 16850 which supports the new “friendly” BitTorrent UTP.  Based on my initial testing, the claim that the new BitTorrent client is network friendly appears to be false.

My initial thought was that perhaps there were simply too few uTP enabled BitTorrent clients in the wild to make this test meaningful, but it turns out that previous versions of uTorrent also support the new uTP protocol.  The old clients simply won’t initiate a uTP connection while the new client will by default.  Because the vast majority of peers deployed on the Internet is running the newest stable uTorrent 1.8.4 or above due to the fact that uTorrent is the most dominant BitTorrent client in the world, the vast majority of connections I made were in fact using the new uTP mode of communications.

Methodology disclosure

  • Windows Vista with SP2
  • Tests conducted on an Intel Quad-core CPU with Intel X38 chipset and gigabit wired Ethernet
  • DSL service with 3 Mbps down and 500 Kbps up signaling speed.  Peak usable data throughput measured to 2.5 Mbps down and 410 Kbps.  Base ping measurements to the first ISP router is a stable 12 milliseconds.
  • uTorrent 2.0 build 16850 used for BitTorrent download and upload.  It is worth noting that BitTorrent corporation acquired uTorrent in 2006 and that the official BitTorrent client called “BitTorrent” is nearly identical to uTorrent although the vast majority of users use uTorrent.
  • Google Chrome build 3.0.195.6 used for HTTP download
  • Downloaded files were roughly 100 MB for both BitTorrent and HTTP
  • HTTP download source was a YouTube cache that can reliably deliver close to the peak 2.4 Mbps on downstream
  • BitTorrent file had 40 connections (5 high throughput peers and 14 moderate or low throughput peers) on downstream and 5 connections on upstream
  • Used bandwidth reported in KB/sec in uTorrent and Chrome
  • Almost every peer that connected to me during my testing were uTorrent 1.8.4 or 1.8.5.

Figure 1: uTorrent 2.0 still hogs over 90% of the network over HTTP
uTorrent 2.0 still hogs over 90% of the network over HTTP

As we can see in figure 1, the new BitTorrent uTP protocol which now runs over UDP instead of TCP still manages to hog over 90 percent of the network’s resources over HTTP (the protocol used by web browsers) when the two applications are used concurrently.  So even though we’re primarily using BitTorrent over UDP instead of TCP, the same principle of unfairness in TCP congestion control still applies.  Furthermore, it’s not just the bandwidth hogging aspect of BitTorrent that’s problematic, BitTorrent running over uTP induces just as much jitter (measured in ping) as it did before.  This jitter is illustrated in figure 2 and 3.

Figure 2: uTorrent 2.0 upload & download traffic causes massive jitter
uTorrent 2.0 upload & download traffic causes massive jitter

Figure 3: BitTorrent uTP upload traffic causes significant jitter
BitTorrent uTP upload traffic causes significant jitter

With uTorrent 2.0 running primarily over uTP with downstream running at ~90% capacity and upstream running at ~80% capacity over my DSL connection, ping times were increased 100 to 200 milliseconds with occasional 1000+ millisecond measurements where the ping measurement timed out.  These ping times are absolutely toxic to real-time applications like VoIP or online gaming.  Even with upstream only traffic, ping times still increased up to 70 milliseconds over the baseline ping measurement of 12 milliseconds.

Even a ping increase of 70 milliseconds is unbearable for online gaming.  For VoIP communications, 70 milliseconds might sound tolerable but my Lingo VoIP phone service routinely drops audio when I’m only uploading with BitTorrent much less uploading and downloading.

The user has a blunt way of throttling how much bandwidth their BitTorrent client consumes but this is a very inefficient way that over penalizes and under penalizes BitTorrent because it doesn’t allow for intelligent bandwidth sharing.  User throttling means BitTorrent bandwidth is hard capped at some arbitrary percentage when it would be much more efficient if the P2P protocol were given lower priority so it can move out of the way when necessary but  be allowed to consume 100% of the network most of the time.  Figure 4 below illustrates how intelligent networks are better.

Figure 4: Intelligent network bandwidth sharing better for everyone.

Furthermore, even capping BitTorrent to 20% of upstream capacity still generates significant amounts of jitter that would be problematic for online gaming and VoIP as shown in figure 5.  If we managed the network intelligently with smart queue management, we could permit BitTorrent or any other P2P application to operate at full speed without any kind of throttling which would be extremely beneficial to P2P users.  Upstream jitter management would ideally be performed in the DSL or Cable modem while downstream jitter management would need to be performed at the Cable or DSL headend.  This is important because a large number of P2P users are online gamers and VoIP users who deliberately shut off P2P whenever they game or use VoIP.

Figure 5: Low bandwidth BitTorrent still causes high ping times
BitTorrent capped to low bandwidth still causes high ping

In conclusion, the need to intelligently manage the network to improve the efficiency of the network and fairly allocate bandwidth between applications and users remains a top priority.  Unfortunately, the FCC’s current NPRM draft regulations prohibits good discrimination because it has been mislead to believes that all “discrimination” is bad.  But the reality is that there is a universally fair way to prioritize applications by always giving low bandwidth and low duration applications priority over high bandwidth and high duration applications.  The FCC should make exceptions for good discrimination in its NPRM.

UPDATE  5:17PM

Ernesto from Torrent Freak (the author of the original story debated here) was kind enough to link back to this article.  I realize that there’s going to be people angry with me about this but I’d like to remind people that I am one of the earliest fans of uTorrent and that I still believe that uTorrent is VERY good code.  I am also a uTorrent user myself and I want to see solutions that make every protocol work better including BitTorrent.  I don’t like hard bandwidth caps whether they’re implemented by the user because they’re afraid to bomb their own broadband connection or if they’re implemented by the ISP (typically in Canada).  I’d much rather see per-subscriber fairness at the broadband level and per-user fairness at the home level with intelligent prioritization and queue management that maximizes throughput for everything and minimizes jitter.  Also important is the fact that lower priority P2P could be given far more generous usage caps (think of cheap bulk shipping rates) especially if it causes no congestion for other subscribers.  Ultimately what is really needed is good dialog between BitTorrent users and network operators.

Update 7:24PM

I’ve contacted BitTorrent’s engineers and they have posted a response here (in comment section).  Note that they do not contradict any data in here.  They don’t even claim to address the fact that BitTorrent will hog 10 times or more bandwidth due to the multi-flow aspect of BitTorrent.  The only thing they’ve said in email to me with regard to bandwidth usage was that “use of idle bandwidth was a design objective”.  The problem that they’re not just using “idle” bandwidth; they’re taking away a lot of bandwidth from other applications and other users and taking a much bigger piece of the pie (as shown in figure 1 above) than justified.

About the only thing BitTorrent claims is that the new uTP protocol tries to keep upstream jitter to less than 100 milliseconds which is still way too high for online gaming and even my Lingo VoIP telephone service.  The downstream jitter is admittedly much greater.  At best, the new uTP protocol has minimal benefits upstream jitter and it completely ignores the problem of downstream jitter and bandwidth hogging.   If we were to be generous, uTP solves half of one congestion problem out of three.

From the looks of it, it seems that people are reading too much into the claims that the new BitTorrent protocol is “network friendly” and that it somehow obviates the need for smarter network management.  From the data that I have collected and from BitTorrent’s response, these claims appear to be greatly exaggerated.

47 Comments »

  • Digital Society » Blog Archive » Analysis of BitTorrent uTP … | FileScoop.Org said:

    [...] original here: Digital Society » Blog Archive » Analysis of BitTorrent uTP … Bookmark It Hide Sites $$('div.d850').each( function(e) { [...]

  • Debunking the hype that the new BitTorrent protocol is “network friendly” | Technology for Mortals said:

    [...] Read the rest at Digital Society. Categories: Broadband, Internet, News, P2P, Policy Tags: Comments (0) Trackbacks (0) Leave a comment Trackback [...]

  • Irish Jim said:

    Thanks for the research. I think uTP is something that will have to be improved over time.

  • fun said:

    great article, i also read your article about ‘smart queue management’.
    i have a question, do you know if there are any open source QoS applications for the end-user? (for the upload management on the application level instead of modem)

  • George Ou (author) said:

    Fun, you can do it with open source dd-wrt but its effectiveness is very limited in my testing. It is MUCH better to do this at the modem transmit queue because that is the most direct and surgical location to address the problem. I show why you need to do this here http://www.digitalsociety.org/2009/09/the-need-for-a-smarter-prioritized-internet/.

  • Arvid Norberg said:

    It’s not clear why it is unfortunate that FCC disallows providers from discriminating between applications. You don’t need to do that for “intelligent bandwidth sharing”, you just need to define a DiffServ code point and tell people about it.

  • George Ou (author) said:

    Arvid,

    How many normal people do you think understands what DiffServ is? How many applications even bother to support DiffServ? What happens when the user or application either mislabels (intentional or unintentionally) labels everything as high priority or doesn’t bother to label at all? You do realize that most applications and users don’t support DiffServ right? We deal with this in the IT world by having the company’s router classify and label packets.

    But in the consumer world, there is no IT department to do this for the user or applications. What ends up happening is that the VoIP or gaming application works lousy and the user either blames the ISP for their self induced jitter or they’re forced to do something drastic like turning their P2P file transfer application off.

    So what I am saying is that it should always be reasonable for the ISP to prioritize low-bandwidth applications over high bandwidth applications, or low duration applications over high duration applications. For example, it should always be reasonable to protect a sub 100 Kbps application like VoIP or online gaming from jitter induced by other high bandwidth applications by prioritizing it in the transmit queue of a router or switching device (modem or DSLAM or CMTS head-end).

    Furthermore, this does not limit application or user choice. We can still give them preference over the default setting so long as they behave within priority quotas so that they don’t abuse the system by labeling everything as high priority.

    The problem is that the FCC NPRM will restrict engineers and ISPs from doing this type of reasonable prioritization, and they need someone to explain to them that there is a right way for the ISP to prioritize by default.

  • fun said:

    George, I was more specifically referring to a application functioning as a virtual (bridge) layer between the Operating System and the Network Interface Card.

    This application, implementing the DiffServ QoS which you mentioned in your slides, could address the congestion problem by ‘sensing’ congestion and prioritizing applications based on the user’s needs.

    The basic idea of such a implementation can be found here:
    http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_4-1/lan_qos.html

    I am aware of the many QoS implementations that reside in most modem and or broadband routers nowadays. As you mentioned, there is a lack of knowledge with many of the end-users to make use of the possibilities of QoS.

    That is why I was wondering if a solution could be found on a higher level, like the OS. Basically, users are easily persuaded to install a application that can ‘boost’ their Internet speed, rather than configure their modem.

  • George Ou (author) said:

    Fun,

    The issue at some point has to be dealt with at the layer 2 level (the switching layer below the Internet Protocol layer) since this problem of jitter is a layer 2 phenomenon. You can manage the lower layer mechanism from the higher layers (the application), but the actual execution needs to be done at the switching layer. More specifically, the transmit queues.

  • fun said:

    George,

    It seems that the layer I was talking account actually exists. In windows xp there is a service called QoS packer scheduler which can be found under START>Run and type: gpedit.msc
    Navigate to Local Computer Policy > Administrative Templates > Network > QOS Packet Scheduler.

    To put it simply, the QoS Packet Scheduler’s primary job is traffic shaping. To accomplish this, the packet scheduler retrieves the packets from the various queues, and then marks the packets with a priority and flow rate.

    In order for QoS to work properly, the various network components located between a packet’s source and its destination must be QoS aware. Although these devices need to know how to deal with QoS, they also need to know how to process normal, non prioritized traffic as well. In order to make this possible, QoS uses a technique called marking.

    There are actually two types of marking that take place. The QoS Packet Scheduler uses Diffserv marking that is recognized by layer 3 devices, and 802.1p marking that is acknowledged by layer 2 devices.

    Layer 2 devices that are 802.1p aware are able to look at the priority markings that are assigned to packets, and then group those packets into separate classes of traffic.

    Please check the articles of Brien M. Posey for more information:
    http://www.windowsnetworking.com/articles_tutorials/Throttling-Bandwidth-QoS-Part2.html
    http://www.windowsnetworking.com/articles_tutorials/Throttling-Bandwidth-QoS-Part3.html

  • George Ou (author) said:

    Fun, I know it already exists in Windows. But do you think those mechanisms operate independently and in the absence of lower layer mechanisms in the routers and switches?

  • fun said:

    George,

    your absolutely right about the importance of lower layer mechanisms. In my view I took those mechanisms as a ‘constant’, the underlying reason was based upon the idea that, as an end-user:

    1. Practically no influence on the QoS delivered by the ISP (or internet).
    2. Difficult to adjust modem configurations: sometimes non-accessible (locked) and/or vendor specific (no universal solution).
    3. Practically speaking, the average user lives with the mentality of “if aint broken dont fix it” so there is also the lack of motivation even if there is sufficient knowledge about what is actually the source of the problem (awareness).

    Considering these implications, brought me the idea to look for solutions on a higher level, assuming that lower levels should be considered as a ‘fixed’ bit pipe.

    Thank you for your comments so far and I look forward in hearing about your views about the practicality and feasibility of solutions on a higher and/or lower level.

  • BitTorrent y su nuevo protocolo uTP » Consultorio del Dr. Ogalinski said:

    [...] que este nuevo protocolo es todo menos “amigable con las redes”, como se puede leer en este interesante artículo de Digital Society, donde se cuestiona fuertemente el nuevo [...]

  • Mohamed Abdinur said:

    Hello George before one could totally write off uTP the question of “is it better?” has to be answered. One would have to compare to another BT client or and older uTorrent not capable of both upstream or downstream uTP. In my general observations uTP is quite an improvement over the more aggressive connection handling of older versions.

    So an improvement in this department still speaks volumes of how far the BT protocol has come in reforming it’s glutton for bandwidth image.

    Is the job now done? Hell no BT as a protocol still has plenty to improve upon. My biggest gripe is that congestion is what the ISP’s tell the governments so it appears more politically correct to throttle. The real reason is bottomline profits, heavy users irrespective of congestion is lost profit. From the content they could of sold that is being pirated, to transit costs for people to interconnect with other peers across the globe. BT as a protocol has to become smarter in peer selection. Yes it selects based on fastest but that is a abusers perspective. There are ways of minimizing transit costs for ISP’s and at the same time still being the fastest by adding another test for peer selection … “Am I closer to this peer or that peer?”. Like Ono for Azureus, that is the next step or progression for BT, uTP is good but not enough to pat oneself on the back for. Finding peers on the local loop is far more effective in creating a landscape of no throttling.

    http://www.aqualab.cs.northwestern.edu/projects/Ono.html

  • Casey Banner said:

    Please be more clear on what router/networking setup you have. BitTorrent seems to have a different effect on every consumer router out there (due to the large number of connections). I would recommend running these tests through a stable router such as OpenBSD with pf or dd-wrt.

  • George Ou (author) said:

    Casey,

    I’ve run these tests on multiple occasions and gotten very similar results. This particular test is done on DD-WRT. Also, you’ll note that BitTorrent was not surprised by the results. They never claimed to do more than try to limit upstream jitter to 100ms, and they don’t do much about downstream jitter so that’s still horrendous, and they do nothing about taking 90% of the bandwidth.

    Mohamed,

    My point was that throttling (hard capping) was bad but deprioritization was good. The difference is that low priority still allows for high bandwidth, but it gets out of the way for low bandwidth applications or users.

  • George Ou (author) said:

    Mohamed,

    From what I have measured, it doesn’t appear that the jitter is better. The version of BitTorrent I measured in early 2008 didn’t have much different results from today, and it’s just as bad. Now BitTorrent is claiming that they’ve cut upstream jitter from 200 ms to 100 ms, but the version of BitTorrent I measured in 2008 was already 100 ms or less for upstream jitter. But let’s take BitTorrent’s word that they did in fact decrease upstream jitter from 200 ms to 100 ms. Is that “better”, sure. Does that mean it’s network friendly? Not even close even if we ignore the other two bigger problems.

    The other two major problems is that downstream jitter is horrendous, and BitTorrent admits this. As for bandwidth hogging, BitTorrent’s Simon Morris indicated that this was by design, and he somehow felt that BitTorrent was entitled to all the “idle” bandwidth. The fact that other applications may want a equal share of that idle bandwidth seems to be immaterial to BitTorrent, and they’ll gladly take 10 times more idle bandwidth than other applications like HTTP web browsers.

    So what I’m specifically responding to is that BitTorrent is claiming that they’re network friendly, and that network operators don’t need to do anything about the problems that BitTorrent creates. This is obviously NOT the case by any stretch of the imagination.

    My other problem with BitTorrent is that they are advocating net neutrality regulations that would prohibit ISPs from favoring low bandwidth applications like VoIP. I asked BitTorrent’s Simon Morris 5 times if he feels that BitTorrent deserves equal priority over VoIP or gaming (meaning unequal bandwidth and the ability to hog the transmit queue and cause massive jitter), and he refused to answer and got angry that I would ask such a question.

    So from this, it’s clear that BitTorrent feels entitled to crush other applications and it seems like they just want to make a few token improvements and claim that they’re not bandwidth hogs. Yet they’re advocating regulations that allow them to hog all the bandwidth they want and they want to tie the hands of ISPs who want to better support their customers by fairly allocating bandwidth between customers and applications.

  • Digital Society » Blog Archive » Knapp is right, there are apps with special requirements said:

    [...] the bandwidth and P2P gets the other 50% to prevent P2P from hogging over 90% of the traffic (and by design no less since BitTorrent feels entitled to as much idle bandwidth as they can grab), I don’t [...]

  • Digital Society » Blog Archive » Basics of packet switching network congestion said:

    [...] Figure 3 – uTorrent 2.0 download traffic causes massive jitter Source: Analysis of BitTorrent uTP congestion avoidance [...]

  • mike said:

    using Utorrent and Netlimiter Pro 2 works fine, I did have to
    degrade back to utorrent 1.8.3 as the 2.0 did bad for me

    In NetLimiter Pro 2.0 you can limit any app. and use Grants
    for important apps.

  • How not to do P2P « Procera Network's Blog said:

    [...] always agree with the sentiments of the researchers (for instance, I quite agree with “http://www.digitalsociety.org/2009/11/analysis-of-bittorrent-utp-congestion-avoidance/” George Ou on BitTorrent – for [...]

  • Rafi said:

    George, can you specify your exact uTorrent’s settings (changes to the defaults) ? Or did I just miss those .

    BTW, it will be appropriate to repeat the test after the 2.0 release will be stabilized in the form of 2.01 (now in beta).

  • George Ou (author) said:

    Rafi, they are all default settings.

    The settings aren’t the problem. The application is designed this way and BitTorrent chose to be selfish rather than polite.
    http://www.digitalsociety.org/2010/01/bittorrent-would-rather-be-selfish-than-friendly/

  • Rafi said:

    George,
    Yes, I saw this article. And your suggestion to prioritize applications probably with some QoS mechanism is a valid one for whoever have the means for that (like proper router ) . Still, the ordinary user does not have/use that.

    1. I’m sure you know that settings *cannot* remain defaults, but have to be set per your line’s max. upload (optionally with the internal setup guide)
    2. I hope you are aware that since this beta (!) you’ve tested here (and even after the release of the final build #17920), there is an on-going effort to iron some bugs/issues, and release a 2.01 follow-up version. A new protocol is not easy to implement w/o tuning it in the real environment.
    3. Your inputs here, as well as others (like my own bug-reports in the forums) will surly be taken into account in the upcoming release. If you did further tests for the current 2.01 beta- posting your finding can help them too.
    If you believe the good intentions behind the uTP development (and I do) feedbacks can surly help. As I see it, it is pre-mature of you to judge by early beta releases, and we just have to wait for the next release or two to see if this attempt to improve was successful.

    Hope you’ll find the time to test 2.01 when it’s out.

  • George Ou (author) said:

    Yes Rafi, I will continue to test (with actual usage) uTorrent, and will continue to monitor network jitter. I can report that nothing has changed (for the better), nor do I expect it to change because the design of uTP isn’t for addressing high jitter and bandwidth hogging. The application is designed to hog as much “idle” bandwidth (at the expense of other applications) as possible. The application isn’t designed to minimize jitter.

  • Rafi said:

    My comments on the latest uTorrent V 2.01 per-RC/beta here –
    http://forum.utorrent.com/viewtopic.php?id=72106

    unfortunately, there is no real change with respect to performence… :( Do you see any improvement with jitter/congestion ? I guess as long as small packets are being introduced by design, and there is no pacing of messages, it not likely to see much improvement :(

  • George Ou (author) said:

    Thanks for the link Rafi. Yes, BitTorrent’s people have told me that there is no jitter mitigation beyond keeping upstream jitter below 200 ms (which is horrific) and no real effort to contain downstream jitter. It is extremely deceptive of them to call it “network friendly”.

  • Rafi said:

    I’ve added a 5th test in that post – latency. Feel free to comment there if you like. “friendly” or not – it simply seems to be not that good compared to TCP :(

  • Digital Society » Blog Archive » Rasmussen – 63% of Internet users oppose FCC regulation of Internet said:

    [...] good friend of mine just asked me today about horrible jitter problems caused by his roommate’s BitTorrent usage and he just imposed a household ban on BitTorrent [...]

  • SiliconANGLE — Blog — Effects of BitTorrent on a Starbucks-AT&T hotspot said:

    [...] the massive degradation in performance not to mention the fact that BitTorrent has the ability to hog most of the bandwidth at the expense of other users and applications.  Even when BitTorrent is capped at a lower [...]

  • TheScynic said:

    I am one of those who does not use VoIP or do online gaming. I believe it is most fair to let the individual decide if they want to prioritize on their own computer. If you want to use VoIP, maybe you have to shut down your torrents to make a phone call. I pay for my service and do not like to have it slowed because Bubba wants to call his girl in Tucson or play some game. I pay and they pay for the pipe, and I choose and they choose what should is a priority.

    The ISPs in the US are generally viewed by people I know as greedy and unmotivated to improve the infrastructure. It is much easier (and need I add cheaper?) to point a finger at BitTorrent than to clean your own house.

  • George Ou (author) said:

    @TheScynic

    “maybe you have to shut down your torrents to make a phone call”

    Maybe you should let other people make that decision to let engineers at the Telco or at the IT department fix this for them. Most people want everything to work at the same time. They want BitTorrent to be unthrottled while VoIP and gaming work flawlessly. They don’t want to shut down BitTorrent just because it causes jitter when it’s such an easy solution to fix jitter without any performance penalties on BitTorrent.

  • uTorrent 2.0 To Eliminate The Need For ISP Throttling | TorrentFreak said:

    [...] is hard to tell if uTP really is BitTorrent’s savior (some highly doubt it), but if it lives up to the expectations it will be beneficial to both users and ISPs. The specs [...]

  • Me said:

    @George:

    Well, when I want to play a game, I turn off my torrents. Same when I want to use Skype. If I wanted all of that to work I would get two lines — one for the torrents, and another for VoIP and gaming.

    Another solution is to download on a dedicated machine overnight, use Skype and gaming during the day.

    I see plenty of options to satisfy everyone, and absolutely no need for any throttling and traffic shaping.

    If you absolutely need to have everything working at once, then by all means use cFosSpeed to traffic shape your own network:
    http://www.cfos.de/speed/cfosspeed_e.htm

    It works very well, even amongst several computers.

    What you are suggesting to be solved on the global network level would never work — as soon as ISPs start classifying traffic, people will have to start paying more for higher priorities, and BitTorrent will still be able to intentionally tag its packets as VoIP or gaming traffic and go through unharmed.

  • George Ou (author) said:

    @Me

    1. cFosSpeed doesn’t solve jitter. I’ve tested it. Even if it worked, it would only work for upstream traffic. Downstream has to be handled on the ISP end because queue management happens on the transmit end.

    2. For you to say there’s “no need for any traffic shaping” after admitting that you have to turn off BitTorrent is truly arrogant. Many people have more than one computer in the house and more than one person using the broadband connection. Those people have the right to ask their ISP for real solutions that allow all applications to work well together.

    3. I never said this needed to be solved on a global network. It only needs to be solved at the localized level where the queues build up. It doesn’t even matter if BitTorrent or any other P2P application masqueraded their port numbers and pretended to be a VoIP application (not to mention they’d get called out for bad behavior if they pulled such a stunt), the application’s behavior (e.g., shoving the router queue with hundreds of packets in a 20 ms time frame) will determine the classification behavior.

Leave your response!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.