High Tq values for HAProxy

Posted by Will on Server Fault See other posts from Server Fault or by Will
Published on 2012-09-05T14:29:29Z Indexed on 2012/09/05 15:40 UTC
Read the original article Hit count: 901

Filed under:

haproxy

I just took over administration of a new environment. A known issue is that the environment is known for high response times (20+ seconds), so I figured I'd turn on haproxy logging and see what is going on. I figured I'd see slow load times in the app servers, but I'm actually seeing high Tq values in HAProxy. The HAProxy is on EC2 and is NOT behind ELB.

Sep  5 14:22:00 haproxy-apps01 haproxy[24695]: 76.14.153.221:3371 [05/Sep/2012:14:21:49.780] http-in default_apps/fe04-c 10936/0/0/55/10991 200 488 - - ---- 111/111/0/1/0 0/0 "GET /event_times/next?callback=jQuery170189312373075111_1346854917562&_=1346854918453 HTTP/1.1"

As you can see, this one has a Tq of about 10 seconds. Not all the Tq's are high (1+ seconds), but a good percentage of them are (approx 35%). Normally when I see this behavior, I'd expect there to be network issues, but this is an incredibly high percentage of visitors to be having an issue like this, so I'm wondering if anybody has seen this or have any hints on diagnosing if the issue could possibly be on this box?

Related posts about haproxy

HA Proxy Stick-table and tcp-connection configuration

as seen on Server Fault - Search for 'Server Fault'
I am using HA Proxy HA-Proxy version 1.4.18 2011/09/16 I am trying to insert the following into /etc/init.d/haproxy.cfg file # Use General Purpose Couter (gpc) 0 in SC1 as a global abuse counter # Monitors the number of request sent by an IP over a period of 10 seconds stick-table type ip size 1m… >>> More
Rotate haproxy logs

as seen on Server Fault - Search for 'Server Fault'
I tried few things but still not able to rotate haproxy logs efficiently. I need to rotate logs when log files crosses 500 MB size. Considering haproxy is serving large no. of static tcp connections, I can not restart haproxy process though a reload is doable. Daily haproxy log file size normally… >>> More
could not bind socket while haproxy restart

as seen on Server Fault - Search for 'Server Fault'
I m restarting HAproxy by following command haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid) but i get following message [ALERT] 183/225022 (9278) : Starting proxy appli1-rewrite: cannot bind socket [ALERT] 183/225022 (9278) : Starting proxy appli2-insert:… >>> More
Restarting Haproxy Gracefully

as seen on Server Fault - Search for 'Server Fault'
As per various blogs, HAproxy can be gracefully restarted using the following command: sudo haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid) TO verify this, I had set up a apache bench script which contiguously sent message to haproxy. Ideally, whenever… >>> More
HAProxy: Display a "BADREQ" | BADREQ's by the thousands

as seen on Server Fault - Search for 'Server Fault'
My HAProxy Configuration. #HA-Proxy version 1.3.22 2009/10/14 Copyright 2000-2009 Willy Tarreau <[email protected]> global maxconn 10000 spread-checks 50 user haproxy group haproxy daemon stats socket /tmp/haproxy log localhost local0 log localhost local1 notice defaults … >>> More

Developer IT