Found Dell's Cloud Computing Blog, figured it was a good place to look for ideas on building highly efficient servers. One interesting post was on High-Density & Co-Location.
Earlier this year I had the opportunity to tour a recently-commissioned, carrier-neutral co-location data center. Located on the outskirts of a major city, and run by one of the major co-location providers in the U.S., it was clearly a state-of-the-art facility with Tier III uptime, impressive physical security, and access to dozens of telecommunication carriers and ISP’s.
What struck me most about this particular facility was its power density – only 4kW/rack. That’s not to say the provider would turn away a customer with racks pulling 16kW of IT load. In such a situation, the provider would lease higher power circuits and “phantom racks” of white space to bridge the power and cooling gaps. Pretty soon, that one 16kW rack looks a lot like four 4kW racks in terms of cost and footprint.
With blades and high-density compute solutions pushing IT loads orders-of-magnitude beyond 4kW/rack, I was perplexed as to why a co-location provider would invest in a facility that appears to ignore all of the industry trends around the growth in IT load over time. Yet, this particular provider is not atypical. Unless a customer is willing to invest in a wholesale lease of a data center, it’s a challenge to find co-location providers capable of supporting power loads greater than 4kW/rack.
Clearly, the economics of the co-location plays a role. According to Gartner, 60-70% of a data center’s CAPEX lies in the mechanical and electrical infrastructure that defines the total supportable IT load. To build a data center capable of supporting 8kW/rack, and have customers deploy racks pulling less than 8kW is an opportunity cost the provider cannot afford. To increase NPV on the facility investment, design to the lowest common denominator of IT load, and charge for density via circuits and white space.
While co-location providers are acting rationally in building such facilities, innocuous decisions on the part of a customer can yield counterintuitive cost increases. Using quotes from some of the major co-location providers, I ran a simple case study for a customer replacing five 2kW racks with four 5kW racks as part of a refresh and consolidation effort. Between annual fee increases (which have been steeper than usual lately), setup, circuit premiums, and incremental white-space, the customer’s 20% reduction in enterprise hardware yielded a 155% Y/Y increase in co-location costs.
For companies that lack the critical mass to build-out their own data-centers, and depend upon an expanding footprint of enterprise hardware to grow their business (e.g. start-up Web 2.0 companies), these are a challenging set of market dynamics. We’d certainly like to hear from firms if they’ve experienced something similar first-hand, and what strategies can help mitigate the impact to the bottom-line.
I tried to just take a short section of the writer's post, but the background was useful to understanding the point about the costs in 2nd last paragraph. The power usage doubled, the amount of space used decreased by 20%, yet the co-lo costs went up by 155%.