How Facebook Ships Code

Posted by Jean-Jacques Dubray on Jan 26, 2011

Sections

Operations & Infrastructure,

Process & Practices,

Development

Topics

Team Collaboration ,

Delivering Quality ,

Devops ,

Distributed Team ,

Collaboration ,

Quality ,

Operations ,

IT Service Management ,

Teamwork ,

Software Craftsmanship ,

Architecture ,

Infrastructure ,

Agile ,

Cloud Computing ,

PHP ,

Process ,

C++ ,

Facebook

Share |

RELATED Research

Yee Lee, a product manager at Skype, has assembled a large collection of notes detailing how code ships at Facebook. Facebook has adopted a developer driven culture. Facebook is organized around two large teams: Engineering and Ops. With a product manager-to-engineer ratio averaging between 1-7 and 1-10. All engineers go through a boot camp during which they fix bugs and attend lectures from senior engineers.

”product managers are essentially useless here.” is a quote from an engineer. engineers can modify specs mid-process, re-order work projects, and inject new feature ideas anytime [...] though, it’s apparent that Facebook’s culture has really embraced product management practices so it’s not as though the role of product management is somehow ignored or omitted.

In Yee's opinion, the culture of the company seems to be set so that everyone feels responsibility for the product.  The key to being influential is to have really good relationships with engineering managers.

Engineers handle entire features themselves, from JavaScript to Database code. Engineers are also responsible for testing, bug fixes, and post-launch maintenance of their own work. There is QA at Facebook, just not as an official team. In effect, there is mandatory code review for all changes. A Facebook employee adds:

most engineers are capable of writing bug-free code. it’s just that they don’t have an incentive to do so at most companies. when there’s a QA department, it’s easy to just throw it over to them to find the errors.

Facebook has adopted a weekly release schedule. Code is gradually roled out to the 60,000+ servers Facebook uses. The roll out is divided in 9 levels, with the first level starting with only 6 servers.

Projects are sources on a voluntary basis. Someone lobbies people to work on his idea and engineers decide to contribute or not.

Operations is obviously paramount to Facebook's success.

ops team is really well-trained, well-respected, and very business-aware. their server metrics go beyond the usual error logs, load & memory utilization stats — also include user behavior.

Facebook is one of the most scalable Web platform to date, serving more than 40 B pages / day. You certainly do not get there by chance and without a wicked smart engineering driven culture. 

RelatedVendorContent

How ALM Supports Business Processes

Visual Studio vNext: ALM features for Agile Planning, Team Collaboration

Deliver quality code quicker with "Go" Agile release management

Continuous Delivery: Anatomy of a Deployment Pipeline

Improving Software Delivery Cycles: Pre-requisites and Inhibitors

Related Sponsor

Go: Agile Release Management Solutions. Go enables predictable, defect-free and timely software releases.

1. Back to top

   Ugh

   by Cameron Purdy

   > "Facebook is probably the hottest company today"
   
   Yes, slightly hotter than MySpace.com.
   
   Peace,
   
   Cameron.

   Reply

2. Back to top

   Re: Ugh

   by Nicolas Duran

   OMG !!! LOL...

   Reply