Eventlet or gevent or Stackless + Twisted, Pylons, Django and SQL Alchemy
- by Khorkrak
We're using Twisted extensively for apps requiring a great deal of asynchronous io. There are some cases where stuff is cpu bound instead and for that we spawn a pool of processes to do the work and have a system for managing these across multiple servers as well - all done in Twisted. Works great. The problem is that it's hard to bring new team members up to speed. Writing asynchronous code in Twisted requires a near vertical learning curve. It's as if humans just don't think that way naturally.
We're considering a mixed approach perhaps. Maybe do the xmlrpc server part and process management in Twisted still but the other stuff in code that at least looks synchronous while not being as such. Then again I like explicit over implicit so hmmm. Anyway onto greenlets - how well does that stuff work? So there's Stackless and as you can see from my Gallentean avatar I'm well aware of the tremendous success in it's use for CCP's flagship EVE Online game first hand. What about Eventlet or gevent? Well for now only Eventlet works with Twisted. However gevent claims to be faster since it's not a pure python implementation it instead uses libevent. It also has fewer idiosyncrasies and defects supposedly. The documentation there is minimal in comparison to Eventlet and it's maintained by 1 guy as far as I can tell. This makes me leery but all great projects start this way so... Then there's PyPy - I haven't even finished reading about that one yet - just saw it in this thread: Drawbacks of Stackless.
So confusing - I'm wondering what the heck to do - sounds like Eventlet is probably the best bet but is it really stable enough? Anyone out there have any experience with it? Should we go with Stackless instead as it's been around and is proven technology - just like Twisted is as well - and they do work together nicely. But still I hate having to have a separate version of Python to do this. what to do....
This somewhat obnoxious blog entry hit the nail on the head for me though: Asynchronous IO for Grownups We're stuck using MySQL as well - I never knew how great PostgreSQL was until having had to work on a production OLTP system in MySQL instead - but that's another story. But if that monkey patch thing really works then wow. Just wow.
