I'm currently rolling out HAProxy on Centos 6 which will send requests to some Apache HTTPD servers and I'm having issues with performance. I've spent the last couple of days googling and still can't seem to get past 10k/sec connections consistently when benchmarking (sometimes I do get 30k/sec though).

I've pinned the IRQ's of the TX/RX queues for both the internal and external NICS to separate CPU cores and made sure HAProxy is pinned to it's own core.

I've also made the following adjustments to sysctl.conf:

   # Max open file descriptors
   fs.file-max = 331287

   # TCP Tuning
   net.ipv4.tcp_tw_reuse = 1
   net.ipv4.ip_local_port_range = 1024 65023
   net.ipv4.tcp_max_syn_backlog = 10240
   net.ipv4.tcp_max_tw_buckets = 400000
   net.ipv4.tcp_max_orphans = 60000
   net.ipv4.tcp_synack_retries = 3
   net.core.somaxconn = 40000
   net.ipv4.tcp_rmem = 4096 8192 16384
   net.ipv4.tcp_wmem = 4096 8192 16384
   net.ipv4.tcp_mem = 65536 98304 131072
   net.core.netdev_max_backlog = 40000
   net.ipv4.tcp_tw_reuse = 1

If I use AB to hit the a webserver directly I easily get 30k/s connections. If I stop the webservers and use AB to hit HAProxy then I get 30k/s connections but obviously it's useless.

I've also disabled iptables for now since I read that nf_conntrack can slow everything down, no change. I've also disabled the irqbalance service.

The fact that I can hit each individual device with 30k/s makes me believe the tuning of the servers is OK and that it must be some HAProxy config?

Here's the config which I've built from reading tuning articles, etc http://pastebin.com/zsCyAtgU

The server is a dual Xeon CPU E5-2620 (6 cores) with 32GB of RAM. Running Centos 6.2 x64. The private and public interfaces are on separate NICS.

Anyone have any ideas? Thanks.