Open source routers for WAN load balancing

Get ready for acronym overload.

A while back, I write a blog entry about pfSense, and mentioned that it was a worthwhile project.  Now I want to mention Vyatta, which is another open source router project.  While pfSense is based on BSD and has a fancy web interface, Vyatta is based on Linux, and only has CLI (command line interface).

As a general rule the CLI is not really a huge problem for me, especially since Vyatta has a PDF reference manual outlining the different commands.

The whole reason I have been looking at open source routing solutions in the first place is because I want to reconfigure our network with a load balanced solution (over two or more broadband internet connections), and I didn't find much in the commercial sector that convinced me to buy.  We also have a VOIP phone system, so whatever gets implemented needs to support SIP connections and have QOS control.  Add DMZ and firewall rules support and now you need a pretty full-featured router.

When I initially started this project, pfSense looked great.  Without too much difficulty, I got load balancing to work, but then I ran into trouble.  Configuring the DMZ was surprisingly problematic.  I never had a problem doing a basic DMZ config with any Watchguard, SonicWall or Juniper firewall...  Ultimately pfSense lost out because I found an obscure a documented limitation where SIP connections (Voip phones) have a concurrency issue so that only one voip phone can connect outbound to a remote host at the same time.  That completely takes pfSense out of the running for now.  Recent visits to their site don't show that any progress is made on this issue.  That is unfortunate.I have been informed that version 1.3 stable will address this issue

Next comes Vyatta.  I tried a setup of Vyatta vc4 running on a P4 IBM desktop, with 4 NICs in it, one for my LAN, one for my DMZ, and two for my WANs.  The initial setup was straitforward.  I configured one WAN and the DMZ and got it all running fine.  Just as I was feeling good (the moment of doom approaching, as it always does when you gain excitement about a new solution), problems struck.  I added the second WAN configuration, and started configuring WAN load balancing.  According to the docs, it should have been easy, but it was far from easy.

After many hours of painfully going over docs, and searching vyatta's forums, I came to the conclusion that Vyatta VC4 load balancing simply does not work.  This is supported by many many forum posts where people ran into odd issues trying to configure it.  One of the biggest issues I believe is that one of our WAN connections uses pppoe, and according to forum posts, to get vyatta load balancing working, you need to hard code a next hop for static routes.  For a pppoe connection where we get a different IP every time the connection comes up, this was a big problem.  I then read a forum entry about vc4.1 beta being released, with notes from one of the developers telling somebody that the load balancing issues were fixed in this new beta.  Ok, I gained some excitement again.  Now to upgrade.

I went through the process of upgrading my install to VC4.1 beta.  As with any beta, I expected things to be a little rough, maybe some glitches.  What I was really hoping for however, was working load balancing, since this was one of the biggest loudest features being advertised about vyatta since the release of VC4. 

Ultimately I was disappointed.  In VC4.1 beta 1, configuration for load balancing did NOT break connectivity for users (like in VC4 release), however the load balancing simply never worked.  On the router, I could ping hosts via both WANs, but the routes just never worked, and I couldn't figure out how to manually write routes that would function properly for load balancing.  All the traffic went through one WAN, from a kernel level route that simply picked the last WAN connection that got setup.  Instead of load balancing, all traffic was shoved down one pipe.  Unplugging that WAN resulted in loss of connectivity for users, when it should result in rerouting of all traffic through the other WAN.  All the help the developers posted regarding manual routes (in response to people having trouble with 4.1) and manually adding mangle rules yielded no result.

Some additional searches found a little feedback about some routing issues, but no real solutions yet to this issue.  So.... at the end of the day, I am disappointed, having never gotten vyatta to load balance my WANs after about 8 hours of effort.  This is a big disappointment for me since Vyatta is getting so much press, and they are talking about load balancing features so much.

So, my review/commentary/etc. on pfSense and Vyatta are that they are great concepts, but not ready for real use yet.  You can setup basic routers easily and effectively, however most $30 cheap-o routers can do all of this as well, and they are supported for that $30 price tag.

The hard core features such as wan load balancing, VOIP and QOS support (all working together) are the compelling draw, however they *don't work yet*.  Maybe 6 months from now when I can afford to waste this much time again, I'll stop in to see how pfSense and Vyatta are doing.  Maybe they will fix their killer show-stopping bugs and I can implement a working solution.

Now, to wrap up, before I get flamed, I don't mean to show lack of appreciation, after all this is open source software, and I didn't pay a penny for it (well technically my time is work some money, but no cash out of pocket).  Seriously though, these developers donate a lot of time, and they should be commended and respected for it.  My views are simply aimed at how the software solution can solve my real-world problem.  I would gladly pay a reasonable sum for a working solution using pfSense or vyatta, and once I see the product work, I will consider that.  Keep up the good work, after a while you will be able to take on the big C as long as you work out the kinks.