Are there some cases where Python threads can safely manipulate shared state?

Posted by erikg on Stack Overflow See other posts from Stack Overflow or by erikg
Published on 2010-04-29T20:06:43Z Indexed on 2010/04/29 20:37 UTC
Read the original article Hit count: 386

Filed under:
|
|

Some discussion in another question has encouraged me to to better understand cases where locking is required in multithreaded Python programs.

Per this article on threading in Python, I have several solid, testable examples of pitfalls that can occur when multiple threads access shared state. The example race condition provided on this page involves races between threads reading and manipulating a shared variable stored in a dictionary. I think the case for a race here is very obvious, and fortunately is eminently testable.

However, I have been unable to evoke a race condition with atomic operations such as list appends or variable increments. This test exhaustively attempts to demonstrate such a race:

from threading import Thread, Lock
import operator

def contains_all_ints(l, n):
    l.sort()
    for i in xrange(0, n):
        if l[i] != i:
            return False
    return True

def test(ntests):
    results = []
    threads = []
    def lockless_append(i):
        results.append(i)
    for i in xrange(0, ntests):
        threads.append(Thread(target=lockless_append, args=(i,)))
        threads[i].start()
    for i in xrange(0, ntests):
        threads[i].join()
    if len(results) != ntests or not contains_all_ints(results, ntests):
        return False
    else:
        return True

for i in range(0,100):
    if test(100000):
        print "OK", i
    else:
        print "appending to a list without locks *is* unsafe"
        exit()

I have run the test above without failure (100x 100k multithreaded appends). Can anyone get it to fail? Is there another class of object which can be made to misbehave via atomic, incremental, modification by threads?

Do these implicitly 'atomic' semantics apply to other operations in Python? Is this directly related to the GIL?

© Stack Overflow or respective owner

Related posts about python

Related posts about threading