This year, I attended O’Reilly’s Velocity conference in San Jose. I was there to present on web monitoring, and we’d just released Complete Web Monitoring, but conference organizer Jessie Robbins had a different idea. On short notice, he asked me to present something before the lunchtime break. I slapped together a presentation based on a Bitcurrent post written a while back.

Dr. Neil Gunther of Performance Dynamics, who was attending the event, approached me after the presentation. He had a number of questions about the math (or pseudo-math) in the slides, and offered to try to work out something less nonsensical than what I’d presented. Neil’s a rocket scientist (pretty much literally) and he’s been working in capacity and performance for decades, so he did a great job.

Neil just published a blog on the subject, along with a presentation analyzing the relationships involved. In the end, he concludes that:

Delay = Number of requests / (Throughput × number of machines)

The important thing to realize here is that in a cloud model, the throughput and number of machines is elastic, and may even be opaque to the IT team. There’s just the bill at the end of the month. It’s the same as buying electricity — you don’t know how many generators are providing your power, or whether it’s nuclear or solar. You just get a bill. So you have to decide where to set the fridge temperature based on what you can afford.

Now consider that there’s an optimal temperature for your fridge. Too cold, and you pay too much in electricity. Too warm, and your food spoils. Your job is to get the temperature right so you maximize food freshness and minimize cost. Of course, we know where to set our fridges, and they come clearly marked to tell us how many Kilowatt-hours they’ll consume. But apply this to a website — which is far more complex than a fridge. You have to set the delay to a place where it won’t break the bank, but will maximize conversions. So will code one day bear a “cost per user-second” certification?

Tying together the performance theory that guys like Neil understand with the analytics and conversion research coming out of Google, Microsoft and others is a fascinating, and potentially lucrative, line of work that few people seem to be playing with yet.