Facebook's next-gen data center network debuts in Altoona, Iowa
Figure A: Facebook's data center in Altoona, Iowa.
Image: Facebook
If you have ever wondered where your Facebook "likes" are stored, Brice Towns suggests there's a good chance they reside in Facebook's new $300 million, 476,000 sq ft data center in Altoona, Iowa (Figure A) -- and he should know being the site's manager. Towns, in this Facebook press release, writes, "This is the fastest we've ever completed a first building at one of our sites, and we owe a lot of that to the people of Iowa. More than 950,000 hours have already been logged in the construction of the facility, and we have an average of 450 people -- 80 percent of them from Central Iowa -- here every day, constructing a second data-center building on the site."
Construction of the second data center referred to by Towns started in June 2014 and is expected to be completed by the end of 2015. According to The Des Moines Register, cost projection for the entire complex is $1.5 billion.
Cooling and power
Those of us who write about data centers are always excited when one of the big three (Apple, Facebook, and Google) commission a new facility. We get to learn about all the new technology those scary-smart engineers have been working on.
Starting with power, Facebook mentions in this post the Altoona data center's power will be supplied by MidAmerican Energy, and be 100-percent renewable. Facebook's Data Center Energy Manager Vincent Van Son writes, "When our new data center in Altoona, Iowa, starts serving traffic, we expect it to be supplied by 100% renewable energy, as tracked by renewable energy certificates, from a new wind project in nearby Wellsburg, Iowa."
Nice, but not extraordinary, maybe Facebook is using some exotic cooling technology. Not quite, it seems Facebook engineers take advantage of our "less than balmy" upper midwest winters, using outside air to cool the computing area. Figure B shows the immense fan system used.
Figure B
The immense fan system used at Facebook's Altoona facility.
Image: Facebook
The Altoona data center does not use raised flooring. Research has proven that forcing air through a raised-floor plenum and perforated floor tiles is an inefficient use of energy. Facebook engineers, instead, pressurize the entire data hall with cool air. The hot exhaust air is funneled out of the racks into the clear enclosures (Figure C) and out of the computing area.
Figure C
Clear enclosures at Facebook's Altoona facility.
Image: Facebook
Facebook is doing all the right things to reduce the data center's power usage effectiveness (PUE) and save electricity, but there has to be something that sets this data center apart. Facebook has a history of upping the stakes with each data-center commissioning.
The new technology is...
Truth be told, Facebook is introducing something new. Rather than tweaking the physical plant, or digital components, engineers looked at Facebook's data center network topology, and they found some issues. Alexey Andreyev, the lead engineer on the project, penned: Introducing data center fabric, the next-generation Facebook data center network, a revolutionary way of moving bits around a data center.
The radical change developed by Andreyev and his team was somewhat expected. Both Google and Facebook have indicated their penchant for viewing data-centers as one giant computing device rather than an aggregate of smaller computing equipment. Google with The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines and now Facebook with Fabric. An analogy, albeit a stretch, is to compare a data center to a rack enclosure full of blade servers.
Andreyev explains why they developed Fabric, "For our next-generation data center network design we challenged ourselves to make the entire data center building one high-performance network, instead of a hierarchically oversubscribed system of clusters. We also wanted a clear and easy path for rapid network deployment and performance scalability without ripping out or customizing massive previous infrastructures."
The diagram in Figure D gives an indication of what's involved. A deep dive into Fabric is next week's assignment.
Figure D
Overview of what's involved in Facebook Fabric.
Image: Facebook
Not just Facebook
Iowa must be doing something right. Besides Facebook's $1.5 billion complex, Microsoft is investing $2 billion, and Google is building a new $1.5 billion data-center complex in Iowa.
Also see
Fastpass: Centralized network arbitration eliminates data-center latency
Photos: How Apple, Google, Facebook and Microsoft data centers are using clean energy
