@ Scale
Most people don’t have to deal with scale on the size of Google, Facebook, Twitter, et al. Scale for the traditional website, neonate cloud-based site, or small to medium enterprise is usually limited to optimizing the architecture on a handful of servers (instances, virtual or not) rather than tens or hundreds of thousands. Even though these architectures don’t require thinking about or dealing with @ scale issues, I feel that thinking @ scale gives us a nice toolkit which we as architects can use judiciously.
In an interview with Google’s Urz Hölzle on ZDNet, Google: At scale, everything breaks, there’s an interesting bit where he speaks about stateful services and scale and how they still use tape, and yet another where he addresses complexity when dealing with scale:
Keeping things simple and yet scalable is actually the biggest challenge. It’s really, really hard. Most things don’t work that well at scale, so you need to introduce some complexity, but you have to keep it down.
There’s also a nice article by Dotan Horovits on Resilient Cloud-Based Architecture where he speaks about lessons learned from the large Amazon Web Services US East region outage that happened late April, 2011.
These are also a good reads (and sites) for thinking and implementing @ scale:
- Evernote: Architectural Digest
- Facebook: Scaling the Messages Application Back End (Facebook Engineering)
- High Scalability: Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
- High Scalability: Did the Microsoft Stack Kill MySpace?
- Phil Whelan: The Apache Projects – The Justice League Of Scalability
- OpenStack
- Twitter Engineering Blog
Go forth and fill your toolkit!