HyperScale Data Centers Featured Article

Facebook Shares Secrets of New Altoona Data Center

November 24, 2014

Global businesses, in this day and age, are by nature responsible for the management of large amounts of data. There are few companies that rival the data traffic Facebook must handle. For years, it has had to innovate the data processing methods of its data centers, and as a result of that innovation, has introduced elements such as server racks, which are cooled by outside air. The company's latest announcement shows how it is using its new Fabric infrastructure to handle its growing amount of machine-to-machine traffic.

As Facebook Network Engineer Alexey Andreyev points out in a recent blog post, Facebook relies on extremely fast machine-to-machine traffic that is orders of magnitude more intensive than machine-to-user traffic—the traffic that ends up in front of Facebook end users. In order to ensure a seamless experience on the Facebook website, data centers must cooperate in real time. The largest complication of this reality is that machine-to-machine traffic is doubling approximately every year. This places pressure on the social giant to innovate because Facebook also wants to keep its infrastructure simple in addition to it being scalable.

Andreyev says Facebook used to use clusters, hundreds of server cabinets, to process data. This has several limitations—the most relevant here being that cluster-to-cluster communication is often overloaded. The best cluster hardware may be able to handle Facebook's current data load, but that hardware is only available from specific vendors, and it locks Facebook into a system where it must use platform-specific hardware and require its engineers to have specific software knowledge. Furthermore, the fact that only a few large clusters would run the show means that a failure of any one cluster could be catastrophic for end users.

The Fabric changes this structure by introducing pods. Pods are layer3 micro-clusters that are serviced by fabric switches. Each pod of four fabric switches contains 48 top of rack (TOR) switches. Server pods act like microclusters; they contain rack switches. Spine planes contain spine switches; spine switches link to fabric switches; fabric switches connect the spine planes and server pods. There are several graphics within the blog post that provide a visual reference to make the links between these elements clearer.

The benefits of this modular architecture are largely that any one rack switch can go offline without severely impacting overall data center performance. Also, data can flow from one rack switch, up through a fabric switch, up further to spine switch and back down in reverse order to any other rack switch with equal performance from one to another. There are multiple paths that data transmissions can take from rack switch to rack switch, and all those paths offer equal performance.

Facebook has created a method of handling data that is scalable to meet its needs and is simple enough for small groups of engineers to manage the entire infrastructure without knowledge of specialized hardware. Other groups may be able to learn from Facebook's example. As Adreyev puts it, Facebook has changed the rules regarding how it handles data. It has made itself a scalable infrastructure that can complete with performance only previously matched by top-tier hardware and specific vendors and has replaced that status quo with basic elements that form an integrated whole that could potentially meet Facebook's needs for years to come.

Edited by Maurice Nagle

Click here to share your opinion – Would color of equipment influence your purchasing decision, one way or another?

Featured Blog Entries

Day 4, Cisco Live! - The Wrap

Day 4 was the final day of our first ever Cisco Live! We had a great show, with many great conversations and new connections with existing and potential end users, resellers, partners and job hunters.

Day 3, Cisco Live!

Day 3 of Cisco Live is history! For Fiber Mountain, we continued to enjoy visits from decision makers and influencers who were eager to share their data center and structured cabling challenges.

Day 2, Cisco Live!

Tuesday was Day 2 of Cisco Live for Fiber Mountain and we continued to experience high levels of traffic, with many high value decision makers and influencers visiting our booth. One very interesting difference from most conferences I attend is that there are no titles on anyone's show badges. This allows open conversations without people being pretentious. I think this is a very good idea.

Day 1, Cisco Live!

Fiber Mountain is exhibiting at Cisco Live! In Las Vegas for the first time ever! Our first day was hugely successful from just about any perspective - from quantity and quality of booth visitors to successful meetings with customers.

Industry News