I have a pylons application where, under certain cirumstances I want to spawn multiple worker threads to process items in a queue. Right now we aren't making use of a ThreadPool (would be ideal, but we'll add that in later). The main problem is that the worker threads logging does not get written to the log files.

When I run the code outside of the pylons application the logging works fine. So I think its something to do with the pylons log handler but not sure what.

Here is a basic example of the code (trimmed down):

import logging
log = logging.getLogger(__name__)

import sys
from Queue import Queue
from threading import Thread, activeCount

def run(input, worker, args = None, simulteneousWorkerLimit = None):
    queue = Queue()
    threads = []

    if args is not None:
        if len(args) > 0:
            args = list(args)
            args = [worker, queue] + args
            args = tuple(args)
        else:
            args = (worker, queue)

    # start threads
    for i in range(4):
        t = Thread(target = __thread, args = args)
        t.daemon = True
        t.start()
        threads.append(t)

    # add ThreadTermSignal
    inputData = list(input)
    inputData.extend([ThreadTermSignal] * 4)

    # put in the queue
    for data in inputData:
        queue.put(data)

    # block until all contents are downloaded
    queue.join()
    log.critical("** A log line that appears fine **")

    del queue
    for thread in threads:
        del thread
    del threads

class ThreadTermSignal(object):
    pass

def __thread(worker, queue, *args):
    try:
        while True:
            data = queue.get()
            if data is ThreadTermSignal:
                sys.exit()
            try:
                log.critical("** I don't appear when run under pylons **")
            finally:
                queue.task_done()
    except SystemExit:
        queue.task_done()
        pass

Take note, that the log lin within the RUN method will show up in the log files, but the log line within the worker method (which is run in a spawned thread), does not appear. Any help would be appreciated. Thanks

** EDIT: I should mention that I tried passing in the "log" variable to the worker thread as well as redefining a new "log" variable within the thread and neither worked.

** EDIT: Adding the configuration used for the pylons application (which comes out of the INI file). So the snippet below is from the INI file.

[loggers]
keys = root

[handlers]
keys = wsgierrors

[formatters]
keys = generic

[logger_root]
level = WARNING
handlers = wsgierrors

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = WARNING
formatter = generic

[handler_wsgierrors]
class = pylons.log.WSGIErrorsHandler
args = ()
level = WARNING
format = generic