Im opening this blog to help de opensource virtualization community answering some questions that we asked ourselfs while implementing a high scalable hybrid cloud, and to keep our it-tech team updated with more than 140 chars via twitter ;) .
Lets start with an issue we've faced :
- If you have a virtualized infraestructure, and you need to support high network stress over VMs running for example nginx , varnish, o caching balancing software, you NEED to tune some network kernel parameters on the physical server where the HypV runs for this to work like a charm..
-- First , you need to raise the memory available for Dom0 , why ?
---- Networking tables, metadata operations and other stuff are allocated over Dom0 memory, so we need to increase it in order to match our environment. This can be made editing /etc/xen/xend-config.sxp dom0_min_size value , and also from the kerel call value from grub.conf dom0_mem
For example in a environment on witch VMs will attend to 1300 request per second, we increased the dom0_min_size to 2048MB
NOTE: You need to disable dom0_mem_balloning !
-- Second, you need to tune up some ip.conntrack values for the phy server not to hang waiting for FYNs and orphan responses. In a environment on witch we've fixed dom0 min mem to 2gb we can raise the max conns to about 120000, and set the ip.conntrack values as this :
net.ipv4.ip_conntrack_max = 120000
net.ipv4.netfilter.ip_conntrack_generic_timeout = 10
net.ipv4.netfilter.ip_conntrack_udp_timeout = 10
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close = 10
net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait = 10
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait = 10
net.ipv4.netfilter.ip_conntrack_tcp_timeout_fin_wait = 10
net.ipv4.netfilter.ip_conntrack_max = 120000
And thats all folks ! Enjoy it
