On Wednesday morning there was very interesting discussion of power management in datacenters and data management software
James Hamilton — formerly of Microsoft and now at Amazon — gave a killer keynote talk on power in datacenters. He traces the power from the provider through all the steps involved in powering machines and cooling. He compares all the power-related costs to the costs of the machines, networking and other costs of running a datacenter, and points out the places where efficiency can be found. A few things amazed me:
- Server costs still dominate power costs, despite recent CW driving green computing. That will shift over time, but optimizing just for power and treating servers as free is extreme and irrational.
- Enterprise systems are dominated by people (management) costs. But internet service systems have very low people costs. The enterprise systems will move that way over time. (My gloss: that will happen by evolution or revolution, more likely the latter. If you want to do an enterprise startup that seems like a good place to go hunting.)
- The best datacenters claim to run at 30% efficiency, but even that is usually bragging — it’s typically much lower.
- Since you provision for peak performance, and your servers are a majority of your budget, it does not make sense to spin down HW to save power. Instead you should find ways to turn idle time into any computing utility you can find that’s more valuable than power costs. The smart (obvious?) thing to do is to try and write software that defers work from peak times to idle times, since that both lowers your fixed costs (by shrinking the peaks) and leverages your resources. (Pat Helland noted that deferring work means on giving up on interactive ACID transactions, something he’s been pushing for some time for other reasons.)
James says all this and more better than I can. Go read his blog.
James was followed by Mehul Shah from HP Labs who gave a super overview of why and how data management software should do better in improving energy consumption. Mehul is the father of the JouleSort benchmark which pretty much kicked off green data systems research. Willis Lang and Jignesh Patel from Wisconsin also reported on their results with the effects of hardware tuning on energy consumption subject to performance SLAs.
There’s something between these 3 talks that didn’t seem quite figured out. James basically gave the advice not to power down machines (#4 above). But Pat Selinger asked him about rarely used data and whether disks with that data should spin down and he said “sure”. And in both the HP and Wisconsin work, the idea was to power down pieces of hardware in some cases. So … what’s behind the dissonance here? It’s got something to do with granularity: individual disk vs machine, modest underclocking vs. full powerdown, etc. There must be a useful analogy here to traditional performance and memory hierarchies: when you have components with very different cost regimes, you should think about managing each one differently, and about moving between levels in the hierarchy intelligently. I think there’s a power-centric analog to the traditional notion of a balanced architecture, where the utilization of the differently-powered components is balanced out to ensure that power consumption across all layers is somehow matched up properly. Mehul suggested looking for an analogy to the 5-minute rule, which may be a way to get at this. I’m not sure if the outcome will be that a “balanced” system doesn’t ever spin anything down (sort of analogous to what a sort benchmark gives you on balancing I/O paths), or instead that spin-down — like disk seek — is something you embrace, but try to budget out intelligently (a la 5-minute rule).