Kevin's Weblog

Recently we upgraded to Rails 2.3 at work. Upon doing so, we saw our application take an aggregate performance hit of about 50%. Spending a little quality time with ruby-prof, one thing that stood out as peculiar was an awful lot of time being spent creating and tending to Ruby Thread objects. Tracing up the call stack, these were coming from inside the memcached client.

Buried in the Rails 2.3 release notes is the innocuous looking statement:

The bundled memcached client has been updated to version 1.6.4.99.

What this fails to tell you is that this upgrade added a timeout option to the client which, as the RDoc will happily inform you, “is a major performance penalty in Ruby 1.8”. Moreover, the default value for the timeout is 0.5s, rather than nil.

Our application makes heavy use of caching, so we were getting massacred by this. To add insult to injury, our most heavily optimized pages were hit the hardest, since they used memcached the most. Adding :timeout => nil to our cache layer immediately restored the previous performance.

Lessons from this story: if you’re a Rails developer using memcached, go add that option to your code; if you’re a library author adding an option with significant performance costs, default to the old behavior; if you’re on the Ruby core team, please add an interface to select so that people don’t do silly things like use Threads to implement timeouts on socket reads.