Waiting for thread to finish Python
Posted
by
lunchtime
on Stack Overflow
See other posts from Stack Overflow
or by lunchtime
Published on 2014-06-08T02:54:08Z
Indexed on
2014/06/08
3:25 UTC
Read the original article
Hit count: 144
Alright, here's my problem.
I have a thread that creates another thread in a pool, applies async so I can work with the returned data, which is working GREAT. But I need the current thread to WAIT until the result is returned.
Here is the simplified code, as the current script is over 300 lines. I'm sure i've included everything for you to make sense of what I'm attempting:
from multiprocessing.pool import ThreadPool
import threading
pool = ThreadPool(processes=1)
class MyStreamer(TwythonStreamer):
#[...]
def on_success(self, data): #### Everytime data comes in, this is called
#[...]
#<Pseudocode>
if score >= limit
if list exists: Do stuff
elif list does not exist:
#</Pseudocode>
dic = []
dic.append([k1, v1])
did = dict(dic)
async_result = pool.apply_async(self.list_step, args=(did))
return_val = async_result.get()
slug = return_val[0]
idd = return_val[1]
#[...]
def list_step(self, *args):
## CREATE LIST
## RETURN 2 VALUES
class threadStream (threading.Thread):
def __init__(self, auth):
threading.Thread.__init__(self)
self.auth = auth
def run(self):
stream = MyStreamer(auth = auth[0], *auth[0])
stream.statuses.filter(track=auth[1])
t = threadStream(auth=AuthMe)
t.start()
I receive the results as intended, which is great, but how do I make it so this thread t
waits for the async_result
to come in?? My problem is everytime new data comes in, it seems that the ## CREATE LIST
function is called multiple times if similar data comes in quickly enough. So I'm ending up with many lists of the same name when I have code in place to ensure that a list will never be created if the name already exists.
So to reiterate: How do I make this thread wait on the function to complete before accepting new data / continuing. I don't think time.sleep() works because on_success
is called when data enters the stream. I don't think Thread.Join() will work either since I have to use a ThreadPool.apply_async
to receive the data I need. Is there a hack I can make in the MyStreamer class somehow? I'm kind of at a loss here. Am I over complicating things and can this be simplified to do what I want?
© Stack Overflow or respective owner