Scaling the Social Graph: Infrastructure at Facebook
Recorded at:
- Share
-
- |
Read later
My Reading List
Bio
Jason Sobel is an engineering manager on the Infrastructure team at Facebook where he focuses on keeping the site fast and scalable. He helped develop Facebook's multiple datacenter strategy and also worked on Haystack, Facebook's custom built photo storage infrastructure. Before joining Facebook Jason was a member of the WAFL team at NetApp where he worked on file system performance.
QCon is a conference that is organized by the community, for the community.The result is a high quality conference experience where a tremendous amount of attention and investment has gone into having the best content on the most important topics presented by the leaders in our community.QCon is designed with the technical depth and enterprise focus of interest to technical team leads, architects, and project managers.
Related Sponsor
Agile Project Management That Adapts With You. Provide your global development team a shared space that adapts to the way they work.
Scribe
by
Robert Sullivan
I'm sure there are those who will say "Why reinvent the wheel, when perfectly good technologies exist", in regards to Scribe (MQ Series etc) and Thrift (CORBA (faster), REST and HipHop (phc).
However, don't we all do this? This is the classic build or buy or use OSS scenario. Sometimes you simply don't have the option, and you are challenged write, as a colleague of mine did, amazing pieces of code, such as a queueing system using SQL Server stored procedures - since we didn't have access to a commercial or open source queue application like MQ Series at the time. I remember saying to the coder that I didn't know you could do stuff (spawning off threads, etc) like that with MS SQL Server, and he replied he didn't either!
When confronted with a complex technology like CORBA, isn't that why everybody went with SOAP? I.e. the developer may run into some issue with technology X, and thus conclude it would simply be faster if they built it themselves, to their own requirements, rather than get bogged down in poor documentation and overly complex integration, with little perceived benefit. I.e. if all I need is an IoC, so that my code is "loosely coupled", do I really need the heavy-weight, xml conf burdened complexity of Spring, or simply build my own version, with the newer technology available, as Google did with Guice, using annotations?
Sure sometimes technologies start out simpler - Java as a simpler C++, SOAP as a simpler CORBA. I believe the "S" standed for "simple", at least at one point.
And I am sure there are benefits of Thrift, and Scribe, perhaps easier install, simpler than CORBA, (although this is less true with today's excellent CORBA implementations (omniORB), that are highly-efficient and easy to use.)
Now, the only downside to this is that now you have another non-standard protocol, so that you get crazy stuff like not being able to cut and paste and share info between Linux systems if they are using different UI frameworks, i.e. KDE uses kparts, a sort of COM-like framework, and GNOME uses something else, probably custom also, they did have a CORBA implementation, but didn't take off.
Now, if you are a heavyweight like Microsoft, or Google, or Facebook, who cares, because you are the 800lb gorilla and everybody else has to follow your model, but at least with Facebook it seems they are (for now) contributing back to the open source world they built their empire on - so that's cool.
Thanks again for the post!
Hello stranger!
You need to Register an InfoQ account or Login to post comments. But there's so much more behind being registered.Get the most out of the InfoQ experience.
Tell us what you think