Interesting set of posts related to Twitter, Ruby on Rails, and scaling.
It starts with an online interview with Twitter developer Alex Payne, where he mentions some challenges associated with scaling and the use of RoR.
None of these scaling approaches are as fun and easy as developing
for Rails. All the convenience methods and syntactical sugar that
makes Rails such a pleasure for coders ends up being absolutely
punishing, performance-wise. Once you hit a certain threshold of
traffic, either you need to strip out all the costly neat stuff that
Rails does for you (RJS, ActiveRecord, ActiveSupport, etc.) or move
the slow parts of your application out of Rails, or both.
The 'creator' of RoR, David Heinemeier Hansson responds.
Scaling is the act of removing bottlenecks. When you remove one bottleneck (like application code execution), you tend to reveal another (like database queries). That's natural and means you're making progress. But you have to keep your marbles straight when doing this. If your bottleneck has moved to the database, you probably won't see big results by replacing pretty constructs with ugly ones. In other words, if a database query is taking 0.5 seconds, improving a loop from 0.05 to 0.01 seconds is not worth bothering with at this point.
I wasn't quite sure what DHH was implying here. It's not unusual for any infrastructure such as RoR that there is a cost tradeoff between ease of development and scalability. Luckily, I didn't have to puzzle it out too long. Mark Pilgrim provided a handy interpretive translation of DHH's post.
Rails is an ogre, and ogres have layers.
