I have an Ubuntu 10.04 box running several dozen Python Django sites using mod_wsgi (embedded mode; the faster mode, if properly configured). Performance highly fluctuates. Sometimes fast, sometimes several seconds delay. The smokeping graphs are al over the place.

Recently, I also added an nginx proxy for the static content, in the hopes it would cure the highly fluctuating performance. But, even though it reduced the number of requests Apache has to process significantly, it didn't help with the main problem.

When clicking around on websites while running htop, it can be seen that sometimes requests are almost instant, whereas sometimes it causes Apache to consume 100% CPU for a few seconds. I really don't understand where this fluctuation comes from.

I have configured the mpm_worker for Apache like this:

StartServers          1
MinSpareThreads      50
MaxSpareThreads      50
ThreadLimit          64
ThreadsPerChild      50
MaxClients           50
ServerLimit          1
MaxRequestsPerChild  0
MaxMemFree           2048

1 server with 50 threads, max 50 clients. Munin and apache2ctl -t both show a consistent presence of workers; they are not destroyed and created all the time. Yet, it behaves as such.

This tells me that once a sub interpreter is created, it should remain in memory, yet it seems sites have to reload all the time.

I also have a nginx+gunicorn box, which performs quite well. I would really like to know why Apache is so random.

This is a virtual host config:

<VirtualHost *:81>
    ServerAdmin [email protected]
    ServerName example.com

    DocumentRoot /srv/http/site/bla

    Alias /static/ /srv/http/site/static
    Alias /media/ /srv/http/site/media
    WSGIScriptAlias / /srv/http/site/passenger_wsgi.py

    <Directory />
            AllowOverride None
    </Directory>

    <Directory /srv/http/site>
            Options -Indexes FollowSymLinks MultiViews
            AllowOverride None
            Order allow,deny
            allow from all
    </Directory>

• Ubuntu 10.04
• Apache 2.2.14
• mod_wsgi 2.8
• nginx 0.7.65

Edit: I've put some code in the settings.py file of a site that writes the date to a tmp file whenever it's loaded. I can now see that the site is not randomly reloaded all the time, so Apache must be keeping it in memory. So, that's good, except it doesn't bring me closer to an answer...

Edit: I just found an error that might also be related to this:

  File "/usr/lib/python2.6/subprocess.py", line 633, in __init__
    errread, errwrite)

  File "/usr/lib/python2.6/subprocess.py", line 1049, in _execute_child
    self.pid = os.fork()

OSError: [Errno 12] Cannot allocate memory

The server has 600 of 2000 MB free, which should be plenty. Is there a limit that is set on Apache or WSGI somewhere?