Thinking like Data Centers are Plumbing for IT services, Markit is an example of Plumbers in Suits

I've joked that part of the reason I can talk to so many different companies in the data center business is because we are talking at a low level that most don't care about - the plumbing in data centers.

Competing at a low level in data centers is as if BofA and Wells Fargo said they wanted to compete on Plumbing.  That's silly.  BofA and Wells Fargo both want the best plumbing to run their business.  What connects to the plumbing and goes through the pipes we don't really talk about.

The Economist has a post on Plumbers in Suits, Markit an unknown company to many that connects much of the financial system.  This financial plumbing is the data goes through the pipes and what it is connected to.

Markit

Plumbers in suits

A private company controlled by banks connects much of the financial system

Jul 6th 2013 |From the print edition

FEW people outside finance have heard of Markit. This week afforded two examples of how the company has worked itself into the fabric of the financial system. On July 1st it released a decent set of purchasing-managers’ indices (PMIs), a closely watched series of confidence gauges it compiles each month. That sparked a day-long rally. That same day the European Commission accused 13 big investment banks of having rigged the market for credit derivatives. The complaint also cited Markit for allegedly helping to prevent exchanges from entering the business.

Plumbing may not be glamorous, but Markit has visions of high growth.

Mr Uggla thinks Markit can double in size again in the next three to five years, and then again shortly thereafter. That would put it in the same league as Bloomberg, Thomson Reuters or McGraw-Hill Financial (the owner of Standard & Poor’s, a ratings agency). Its hope is that post-crisis regulations will prove so complex as to send banks scurrying to Markit asking it to take over even more of their back-office operations.

That kind of growth rate sounds like some of the top data center companies.

Lack of Redundancy in Bridge Design causes I-5 Outage

It is amazing how there can single points of failure in data centers even though they were sold as highly available designs.  Some make the mistake that just because it hasn't failed in the past and a lot of money was paid, failure is unlikely.

My dad was a civil engineer with CalTrans (California's Department of Transportation) Bridge Division which includes overpasses (CA has way more overpasses than bridges), so whenever I read about civil engineering stuff it reminds of a possible conversation with my Dad.  Unfortunately, my dad died of colon cancer 19 years ago, so I need imagine the conversations.

In the state of Washington the I-5 has an outage, a bridge has collapsed when a truck's load hit the structure.

What is the cause of the bridge collapse, an outage of Interstate I-5 between Seattle and Vancouver, BC?  One hit from a truck and it collapses?  Sounds like a Jenga design.  Knock out this one block and the whole thing falls.

Here is a view from Google maps of what the bridge used to look like before the collapse.

The WSJ has an answer to the outage.  A single point of failure.  The lack of redundancy in the design. 

"This is not the sign of deteriorating infrastructure, this is a sign of vulnerable infrastructure," said Abolhassan Astaneh-Asl, a civil-engineering professor at the University of California, Berkeley.

"This original design in those days was fine," he said of bridges lacking redundancy, "but today we should invest in getting these…out of the system."

...

The bridge has what is known as a "fracture-critical" design, which means that if any part fails, the whole bridge could fail, said Mr. Astaneh-Asl. "A fracture critical bridge is like a chain," he said. "Any link in this chain you cut, it's going to fail."

2011 data center presentation slides

I found this presentation just sitting on Utah University's website for a 2011 data center project.

Found it interesting that this was 4,000 kW facility with 2,400 kW for critical load for a 1.7 PUE in 2011.

Here are pictures of the data center space during construction.

I can't recall where I have ever seen Generac generators at a data center site.

Generators to provide power in the event of a power outage

Brand new facility and their hot/cold aisle containment is not that impressive.  I guess that would explain the 1.7 PUE at full load.  Got to think the PUE is 2.0 during the early phases.

Once the racks were installed, airflow zones were created to channel cold air to the racks and contain and vent the hot air from the data center. Power is delivered to each row of racks from an overhead distribution system.

Detroit vs. Iowa Data Centers, GM vs. Detroit competing for the IT talent

Chris Crosby has a post on Detroit as a data center location and brings up the folks in Iowa with Facebook.

Detroit. The Data Center Capital of America

Life is good in Altoona, Iowa. With the coming of Facebook, servers will quickly outnumber the community’s 15,000 residents and the city is poised to become one of the country’s leading data center destinations. The citizenry of Altoona are, of course, ecstatic at their good fortune. The economic benefits alone are too numerous to consider. The police department is contemplating adding a second car, the country club might add nine more holes so members can play a full 18, and there’s a rumor going around that Krispy Kreme might be coming to town. Yes indeed, the gentrification of Altoona has begun—and good for them. I have nothing against the good people of Altoona. And yet I have to ask, “Why Altoona?” Why this small oasis in the Hawkeye state as opposed to say, the Motor City? That’s right, why not Detroit? I pose this as a serious question. I realize that a few of you effete data center snobs might snort in derision at the mere mention of this discussion, but really, what has Altoona got that the buckle of the Rust Belt doesn’t?

Coincidentally, GM has been making some news not nearly as widely covered as Facebook on its data centers in Detroit.

Facebook adding to its existing infrastructure is actually less transformative as GM going from its past of outsourced IT to data centers it owns and runs.  GM has recognized that IT infrastructure is critical to its success just like Facebook.

The GM renaissance of IT, canceling outsource contracts, building state of the art data centers I think is actually a cooler story to read about the transformation of IT than just another Facebook data center.  WSJ tells the history of how in 2011 a three day HP mainframe shut down woke up the CIO how ancient their IT infrastructure is. 

Hewlett-Packard Co. ’s outsourcing relationship with General Motors Co. may have been doomed regardless, but the last straw might have occurred in October 2011, early in GM CEO Dan Akerson’s tenure, when a mainframe computer that H-P operated on its  behalf went on the fritz for three days.

“All of a sudden we started having slowdowns in our manufacturing around the globe.  A bell went off in my head,” Mr. Akerson told the Wall Street Journal in an interview earlier this year. “It was so fundamental you just assumed that a company of our stature, our size, our complexity, our global reach, you had to have a 21^st century IT infrastructure. We didn’t.”

General Motors Co.CIO Randy Mott

The outage meant that GM suppliers couldn’t be sure how to fill certain expected orders and had to guess at the details, according to Jeff Liedel, executive director of infrastructure engineering at GM. He said the outage probably “couldn’t have gone on longer, without forcing us to shut plants down.”

Here is a B-roll video with no sound that shows the new data center space.  It's not as sexy as a Facebook data center and the dress shirts with slacks don't look anything like the jeans and t-shirt culture.

The enterprise data center and a companion data center at the Milford Proving Ground are part of a previously announced plan to transform GM’s global IT footprint from 23 facilities to two by 2015. Construction of the $100 million data center expansion in Milford will begin this summer.

GM IT is leveraging the Warren and Milford data centers to create a secure, private cloud that allows super-computer applications, servers and data storage to be efficiently and quickly accessed among multiple users.  

“Our data center consolidation is just one of the initiatives driving the transformation of GM’s business,” said Randy Mott, GM vice president and CIO. “It’s part of an overarching strategy to transform not only information technology but also allow GM’s business operations to be more responsive to our customers, quicker to market and deliver on our objectives to shareholders.”

GM has a 2nd data center to provide an active fail over site where data can be mirrored.

The Milford location was chosen because it is more than 25 and less than 50 miles from Warren, allowing “mirrored” data, so if one facility is off line for any reason, the other will have the same data available without interruption.

"It's all about reducing risk and making sure no one event would affect both centers at the same time," said Curt Loehr, GM Information Technology project manager. "Each Center has its own utility feed using separate paths to provide uninterrupted power.  We even checked weather data going back a half century and Warren and Milford are affected by separate weather patterns." 

GM is saving money by having the data centers on existing campuses, which have negotiated bulk utility rates, existing infrastructure and security.

I guess I kind of felt compelled to write about GM given I have a Cadillac CTS as a rental car.  no comments on the car.  The data center is way more interesting. 

Data Centers lease for Sq ft, kWh, ISP connections, Not Energy Brokers

In the old days, data centers were all about space, # of sq ft.  The power was actually hard to figure out.  The data center salesman would almost always talk in sq ft. 1,000 sq ft. 10,000 sq ft.  How much power?  Ohhh,  10,000 sq ft x 50 watts/sq ft = 500kW.  I would joke that part of why sq ft is used so much in commercial real estate is it is nice easy math.  How much does it cost to build per sq ft.  What is operating expense per sq ft.  What is rent per sq ft.  This set up a bad practice of thinking people would save money by using less space in a data center.  I am charged by space so if I go higher density, then I’ll save money.  Uh NO.  The expensive stuff in a data center is the electrical and mechanical systems.  You talk to any experienced data center operator/designer who has control over his destiny with budget for CAPeX and OPeX, he’ll choose 100 - 150 watts/sq ft.  Any higher density increases the chances of stranded power, cooling issues and a variety of things that could increase costs.  If you don’t know what stranded power is go have a talk with your electrical team and ask them how big an issue stranded power is.  

Those who lease data center space know the stranded power problem which is why they charge for the Power committed to your environment in addition to the power you consume.  If you strand 1/2 your power because you made bone headed decisions in how you designed your data center space, you’ll pay for that power as the data center operator cannot simply use that power some place else.

I read NYTimes’ James Glanz’s post on Landlords Double as Energy Brokers a few times and I am confused.  James makes the point that data centers moved from charging for space to an energy broker.

A result, an examination shows, is that the industry has evolved from a purveyor of space to an energy broker — making tremendous profits by reselling access to electrical power, and in some cases raising questions of whether the industry has become a kind of wildcat power utility.

When I hear the word Energy Broker it makes me think this is like an Enron type of deal. 

Soaring power prices have pushed the state’s utilities to the brink of bankruptcy and forced Third World-style blackouts across the world’s sixth-largest economy. Enron and other electricity marketers and generators are being investigated by the state attorney general and sued by consumers amid accusations of profiteering and market manipulation. ”Every trading company in the country has been feasting on California, and Enron is the shrewdest of them all. They are like sharks in a feeding frenzy,” says Michael Shames, executive director of the Utility Consumers’ Action Network in San Diego. Enron, an early critic of California’s deregulation plan, hotly denies those charges.

The reason why data centers charge more for power than what they pay is because of the cost of electrical systems and mechanical systems required to deliver the power.

Some data center companies, including Digital Realty Trust and DuPont Fabros Technology, charge tenants for the actual amount of electricity consumed and then add a fee calculated on capacity or square footage. Those deals, often for larger tenants, usually wind up with lower effective prices per square foot.

Regardless of the pricing model, Chris Crosby, chief executive of the Dallas-based Compass Datacenters, said that since data centers also provided protection from surges and power failures with backup generators, they could not be viewed as utilities. That backup equipment “is why people pay for our business,” Mr. Crosby said.

….

Melissa Neumann, a spokeswoman for Equinix, said that in the company’s leases, “power, cooling and space are very interrelated.” She added, “It’s simply not accurate to look at power in isolation.”

OK, data centers aren’t energy brokers.  They do a bad thing operating as a REIT to save on taxes.

Some of the biggest data center companies have won or are seeking Internal Revenue Service approval to organize themselves as real estate investment trusts, allowing them to eliminate most corporate taxes. At the same time, the companies have not drawn the scrutiny of utility regulators, who normally set prices for delivery of the power to residences and businesses.

…

Equinix is seeking a so-called private letter ruling from the I.R.S. to restructure itself, a move that has drawn criticism from tax watchdogs.

“This is an incredible example of how tax avoidance has become a major business strategy,” said Ryan Alexander, president of Taxpayers for Common Sense, a nonpartisan budget watchdog. The I.R.S., she said, “is letting people broaden these definitions in a way that they kind of create the image of a loophole.”

So, data centers shouldn’t be able to operate as a REIT because they’ll save on taxes?

I am confused on what points James was trying to make.