Writing asynchronous nginx plugins with python and tornado for 1000 qps

TL;DR by returning X-Accel-Redirects and using multiple tornado processes you can use python as an async plugin language for nginx.

I've been using an architecture at work for building video streaming servers that I've not seen anybody else describe. I'm combining the ease of writing, reading and testing python with the performance of nginx. I've been pretty happy with the maintainability of the result. It's been taken from separate cache and origin to cache+origin and from Smooth Streaming to HLS; and I can hit 1000 qps and 8 Gbps of video with it (with pretty beefy hardware though).

See

http://linuxfiddle.net/f/5f96743c-3d62-40f7-89ef-f64460e904c1

This setup wouldn't serve 1000 qps, missing are:

• You'd want to use more tornado front-end processes so that python isn't the bottleneck
• If there is any cache of any kind (e.g., vfs) in the backend, you want to use content-aware backend selection
• You'd cache the mysql results rather than getting them out of the db every time
• In the front-end, you'd want to merge all simultaneous requests for the same URL into one
• Nginx should cache the service responses