How can I strip Python logging calls without commenting them out?

Posted by cdleary on Stack Overflow See other posts from Stack Overflow or by cdleary
Published on 2009-02-07T00:08:11Z Indexed on 2010/05/11 3:44 UTC
Read the original article Hit count: 273

Filed under:
|
|
|

Today I was thinking about a Python project I wrote about a year back where I used logging pretty extensively. I remember having to comment out a lot of logging calls in inner-loop-like scenarios (the 90% code) because of the overhead (hotshot indicated it was one of my biggest bottlenecks).

I wonder now if there's some canonical way to programmatically strip out logging calls in Python applications without commenting and uncommenting all the time. I'd think you could use inspection/recompilation or bytecode manipulation to do something like this and target only the code objects that are causing bottlenecks. This way, you could add a manipulator as a post-compilation step and use a centralized configuration file, like so:

[Leave ERROR and above]
my_module.SomeClass.method_with_lots_of_warn_calls

[Leave WARN and above]
my_module.SomeOtherClass.method_with_lots_of_info_calls

[Leave INFO and above]
my_module.SomeWeirdClass.method_with_lots_of_debug_calls

Of course, you'd want to use it sparingly and probably with per-function granularity -- only for code objects that have shown logging to be a bottleneck. Anybody know of anything like this?

Note: There are a few things that make this more difficult to do in a performant manner because of dynamic typing and late binding. For example, any calls to a method named debug may have to be wrapped with an if not isinstance(log, Logger). In any case, I'm assuming all of the minor details can be overcome, either by a gentleman's agreement or some run-time checking. :-)

© Stack Overflow or respective owner

Related posts about python

Related posts about logging