Problems virtualizing Nginx or Varnish ??? try this out

If you want to build a high load balanced and well cached infraestructure, and your company is over 600 searches per second ... well installing varnish and Nginx over physical hardware doesnt scale anymore, but yes, your virtual machines dont behave like you pretend, hangs, and when are overstressed, even your HOST ethernet card goes to hell , and yeah ... you've losted connectivity to all the VMs running on that host, that sucks hard of course, and youll have to write a nice report to deliver to your manager, and that sucks even harder.

So, lets get dirty, this guide applies to VMs guested over Xen HyperV, but maybe the same symptoms can help you to get this running over another HyperV

-  Symptoms :

Assuming that you have your application server and JVM tuned up to run the apps balanced or cached by nginx or varnish in a good shape, and your configurations are the necessary to handle all your traffic, here you have some of the symptoms :

-- Even you tuned out your conntrack settings, after you shot a 1700 and 6k connections to the echo server behind the virtualized nginx or varnish, the VMs that has them installed stop responding well to requests, and after a while, hangs a lose connectivity
-- The ethernet card of the Host physical server hangs, and you loose the entire box
-- You have a LOT of "Ack delayed sent" counters on your interface

Well people, here we have a point, what does the increment of Ack delayed means ? , Means that theres a constant windowing renegotiations.

Ok, so whats the deal ?  The deal is that actually packages that leaves the bridge trying to get out of your physical ethernet cards are exceeding the maximum amount of MTU setted for the interface, causing a lot of fragementation and lost of packages.

Now, the magic solution is to down a little bit the MTU on the VM ethernet interface, 1436 is a good value ( the same that you assign to an interface that is going to deal with encryption and decryption, ipsec headers, etc) .

Now you can virtualize your favorite load balancer or caching software without perfomance issues on a high connection demanding infraestructure

Regards !