I have a python script which runs a particular script large number of times (for monte carlo purpose) and the way I have scripted it is that, I queue up the script the desired number of times it should be run then I spawn threads and each thread runs the script once and again when its done.

Once the script in a particular thread is finished, the output is written to a file by accessing a lock (so my guess was that only one thread accesses the lock at a given time). Once the lock is released by one thread, the next thread accesses it and adds its output to the previously written file and rewrites it.

I am not facing a problem when the number of iterations is small like 10 or 20 but when its large like 50 or 150, python returns a KeyError: 51 telling me element doesn't exist and the error it points out to is within the lock which puzzles me since only one thread should access the lock at once and I do not expect an error.

This is the class I use:

class errorclass(threading.Thread):

    def __init__(self, queue):
        self.__queue=queue
        threading.Thread.__init__(self)

    def run(self):
        while 1:
              item = self.__queue.get()
              if item is None: break
              result = myfunction()
              lock = threading.RLock()
              lock.acquire()
              ADD entries from current thread to entries in file and
              REWRITE FILE
              lock.release()

queue = Queue.Queue()

for i in range(threads):
    errorclass(queue).start()

for i in range(desired iterations):
    queue.put(i)
for i in range(threads):
    queue.put(None)

Python returns with KeyError: 51 for large number of desired iterations during the adding/write file operation after lock access, I am wondering if this is the correct way to use the lock since every thread has a lock operation rather than every thread accessing a shared lock? What would be the way to rectify this?