The scenario is an API server that should handle uploads. Posting on my.host.com/api/upload should do something with the body the client sends.

However the API server has been designed to block the whole request until it fully processed the file, including some analysis which can take up to approx. 5min (...!). This has to change of course.

In the meantime I wanted to setup nginx as a load balancer in front of the API servers.

I quickly ran into a timeout issue, consulted Google and came up with this ridiculous test configuration:

user www-data;
worker_processes  4;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;

    access_log   off;

    sendfile on;
    send_timeout 3600;

    keepalive_timeout 3600 120;

    tcp_nopush on;
    tcp_nodelay on;

    gzip  off;

    client_header_timeout 3600;
    client_body_timeout 3600;

    proxy_send_timeout 3600;
    proxy_read_timeout 3600;
    proxy_connect_timeout 1800;
    proxy_next_upstream error;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

And

upstream test {
  server host1;
  server host2;
}

server {
  listen 80;
  server_name my.host.com;
  client_max_body_size 10m;

  location /api/ {
    proxy_pass http://test;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $host;
    proxy_redirect off;
  }
}

Still, when an upload happens, I get the following result in the error.log:

2010/12/22 13:36:42 [error] 5256#0: *187359 upstream timed out (110: Connection timed out) while reading response header from upstream, client: xx.xx.xx.xx, server: my.host.com, request: "POST /api/upload HTTP/1.1", upstream: "http://apiserver:80/upload", host: "my.host.com"

What else could I do? If I look at the log of the API server I still see that it is processing the request and analyzing the file. But I think 3600 seconds as a timeout should be more than enough. This happens even after a could of seconds.

And I did a reload and force-reload of the configuration as well of course.