Hi,

I am using postgreSQL for a scientific application (unsupervised clustering). The python program is multi-threaded so that each thread manages its own postmaster process (one per core). Hence, their is a lot of concurrency.

Each thread-process loop infinitely though two SQL queries. The first is for reading, the second is for writing. The read operation considers 500 time the amount of rows the write operation considers.

Here is the output of dstat:

----total-cpu-usage---- ------memory-usage----- -dsk/total- --paging-- --io/total-
usr sys idl wai hiq siq| used  buff  cach  free| read  writ| in   out | read writ
  4   0  32  64   0   0|3599M   63M   57G 1893M|1524k   16M|  0    0 |  98  2046 
  1   0  35  64   0   0|3599M   63M   57G 1892M|1204k   17M|  0    0 |  68  2062 
  2   0  32  66   0   0|3599M   63M   57G 1890M|1132k   17M|  0    0 |  62  2033 
  2   1  32  65   0   0|3599M   63M   57G 1904M|1236k   18M|  0    0 |  80  1994 
  2   0  31  67   0   0|3599M   63M   57G 1903M|1312k   16M|  0    0 |  70  1900 
  2   0  37  60   0   0|3599M   63M   57G 1899M|1116k   15M|  0    0 |  71  1594 
  2   1  37  60   0   0|3599M   63M   57G 1898M| 448k   17M|  0    0 |  39  2001 
  2   0  25  72   0   0|3599M   63M   57G 1896M|1192k   17M|  0    0 |  78  1946 
  1   0  40  58   0   0|3599M   63M   57G 1895M| 432k   15M|  0    0 |  38  1937 

I am pretty sure I could write more often than that for I have seen it write up to 110-140M on dstat. How can I optimize this process?