Python multithreading not working on VPS server
- by Sabirul Mostofa
I am running an python multithreaded application with multiple processes which scrapes data from some websites.
While running on my localhost It works great, but on the vps server I am using( Centos 5.8, 2.6 GHZ with 4 cores) performs very slow.
From the nethogs command I get the network usage too low. I get around 8KBps with 15 threads.
On other hand, in my PC I get the usage around 100-120KBPS.
I have read about the Python GIL and threading limitations. It seems GIL never releases the lock on the VPS though it should while doing I/0
Is there any configuration in the VPS that I need to change for the threading to work properly?