Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Articles Key Takeaway Points and Lessons Learned from QCon San Francisco 2012

Key Takeaway Points and Lessons Learned from QCon San Francisco 2012


A large confluence of team leads, architects and project managers descended upon the Embarcadero in San Francisco this year for a technology pilgrimmage that has become an annual tradition: QCon San Francisco 2012 (QConSF).  To accomodate its 950 attendees - 30% more than last year - the conference was forced to move to a larger venue this year. 

Continuing in its tradition of providing presentations selected "by practitioners for practitioners," QConSF featured over 100 sessions, keynotes, and - for the first time this year - professionally facilitated open space sessions. Attendees this year also had the chance to attend the inaugural GrapchConnect, a co-located, 2-day conference covering important topics in the NoSQL and graph database community.  

Case studies from Facebook, Google, and Pinterest drew large crowds at QConSF, as well as a wide array of talks on continuous delivery, Big Data & NoSQL, cross-platform mobile, dynamic languages, and more.  The following article summarizes the key takeaways from QConSF 2012, including blog entries written by editors and practitioner attendees for all keynotes, tracks and sessions along with aggregated twitter feedback during the event. You can also see numerous attendee-taken photos of QCon.



Architectures You've Always Wondered About

Big Data and Analytics

Continuous Delivery

Cross Platform Mobile

Dynamic Languages for the Web

Java Renaissance

Loose Concurrency & CAP Theorem Today

Mechanical Sympathy

NoSQL Emergence

Programmable Web

Realtime Web

Tales from Silicon Valley

Taming HTML5 & JavaScript

The Agile Team Field Guide

User Experience (UX)

Solutions Track: No SQL

Opinions about QCon




Agile Retrospectives by Aino Vonge Corry

Twitter feedback on this session included:

ernando Moribe@fgmatwork :Agile Retrospectives tutorial from @apaipi was one of the best tutorials at #qconsf ever!!

Continuous Delivery by Jez Humble

Twitter feedback on this session included:

Fernando Moribe@fgmatwork : Continuous delivery with @jezhumble in the first day of #qconsf was awesome

Vlad Skvortsov@wadcom :"The reason people become engineers is so they can avoid talking to other people" @jezhumble at #qcon

Vlad Skvortsov@wadcom :Definition: legacy code is a code that doesn't have automated tests. #qcon

Cross Platform Mobile Apps with PhoneGap by Christophe Coenraets

Twitter feedback on this session included:

desbiens@desbiens : Build and deployed my first phonegap app thks to @ccoenraets #qconsf

Lou Sacco@occasl :@ccoenraets #Phonegap preso pretty impressive with regard to what's available at the JS API level with decent performance. #qconsf

Domain Driven Overview by Eric Evans

Twitter feedback on this session included:

Franklin Amorim@cyberelfo : "Reality is a distraction. A model serves some use." Eric Evans #QConSF

Hakeem Mohammed@hakeemsm : Had a good refresher on #ddd by @ericevans0 at #qconsf having read his book some yrs back it was good to hear from the man himself

Micro Services - Evolutionary Approaches For Systems Of Systems by James Lewis

Twitter feedback on this session included:

Kris Trujillo@kris_trujillo :"Tip 1 - Divide and Conquer" micro services #qconsf

Kris Trujillo@kris_trujillo :"Tip 2 - Use Conway's Law to structure teams" - Micro Services #qconsf

Kris Trujillo@kris_trujillo :"Tip 3 - The Last Responsible Moment.....don't make decisions at the moment you know the least" - Micro Services #qconsf

Kris Trujillo@kris_trujillo :"Tip 4 - Be of the web, not behind the web" - Micro Services #qconsf

Kris Trujillo@kris_trujillo :"Tip 5 - If something is important, make it an explicit part of your design" - Micro Services #qconsf

Kris Trujillo@kris_trujillo :"Tip 6 - Favor services choreography over orchestration" - Micro Services #qconsf

Kris Trujillo@kris_trujillo :"Tip 7 - Make it easy to do the right thing and hard to do the wrong thing" - Micro Services #qconsf

Kris Trujillo@kris_trujillo :Clean looking application following micro services approach #qconsf

Jeff Gothelf@jboogie :.@tsharon presenting ux research for engineers #qconsf

Kris Trujillo@kris_trujillo :"No Silver Bullets" - Micro Services #qconsf

Professional Javascript and Coffeescript by Peter Bell

Twitter feedback on this session included:

QCon San Francisco@QConSF :“@PeterBell: Thanks for everyone who took my javascript/coffeescript tutorial at @QConSF - had a blast!” - We had a blast too! #qconsf


Cool & Useless by Kevlin Henney

Twitter feedback on this session included:

Lene Mejlby@Mekkaz :Don't trust the compiler, just type it in yourself. @KevlinHenney talking about the code for a space shuttle #qconsf

Therese Hansen@qedtherese :Kevlin Henney on stage at #qconsf showing this … My Danish friends made this #CoolAndUseless

avid Whittaker@rundavidrun :@kelvinhenney introduces us to COBOL on Cogs ala Ruby on Rails...of course written in JS #AtwoodsLaw #qconsf

Fernando Moribe@fgmatwork :Funny party keynote from kevlinhenney at #qconsf

Geoff Webb@geoffnettaglich :Highlights from "Cool & Useless" keynote at #qconsf Cobol on Cogs ( ) and JS/Linux ( )

NoSQL: Past, Present, Future by Eric Brewer

Twitter feedback on this session included:

Kris Trujillo@kris_trujillo :"Data outlasts any particular implementation" - Eric Brewer #QConSF

David Whittaker@rundavidrun :@eric_brewer says: "In the 70's we moved to higher level models, from assembly to C, and from navigational to relational databases." #qconsf 

Therese Hansen@qedtherese :Eric Brewer on stage at #qconsf talking about NoSQL - and even some history of "NoSQL" before SQL

David Whittaker@rundavidrun :Want a clean model and ACID transactions? Go relational. Want elegance, modularity, and flexibility? Go NoSQL. @eric_brewer #qconsf

Erik Sowa@eriksowa :Eric Brewer at #qconsf : “ACID was sacred.” CAP that!

David Whittaker@rundavidrun :@eric_brewer developed CAP theorem to explain why he had to make the trade-offs he did when designing shared-data systems. #qconsf

Martin Thompson@mjpt777 :"Jim Gray came up with 'ACID' transactions while stoned in his hot tub. Don't get too hung up on the letters..." - Eric Brewer #qconsf

Martin Thompson@mjpt777 :"A timeout is the point at which you make the choice between consistency and availability." - Eric Brewer #qconsf

David Whittaker@rundavidrun :@eric_brewer claims "For most businesses, availability is more important than temporary inconsistency, even banking, e.g. ATMs." #qconsf

Dan Cundiff@pmotch :Eric Brewer, CAP creator says "In real world systems, availability > than consistency because being available is tied to revenue." #QconSF

Race Conditions, Distribution, Interactions--Testing the Hard Stuff and Staying Sane by John Hughes

Twitter feedback on this session included:

Geoff Webb@geoffnettaglich :John Hughes (mr haskell) says "Don't write tests, generate them" #qconsf

Ismail Elshareef@ielshareef :Don’t write tests; generate them. Use QuickCheck. Keynote by @rjmh, who is totally fired up. #qconsf #tdd #qa

David Whittaker@rundavidrun :@rjmh implements queue as circular buffer in C that is 'obviously correct' and models intended behavior in Erlang to test it. Cool! #qconsf

Therese Hansen@qedtherese :John Hughes on stage at #qconsf talking about testing for race conditions and other nasty things.

David Whittaker@rundavidrun :@rjmh models race conditions via ticket dispenser metaphor then fuzz tests, trying to interleave as sequential to check. #quickcheck #qconsf

iliketoprogram@iliketoprogram :Dear #Quickcheck - i need to research you you seem awesome! #qconsf

Real Software Engineering by Glenn Vanderburg

Twitter feedback on this session included:

Kris Trujillo@kris_trujillo :"It started to go wrong when 'software' and 'engineer' were used together" - Glenn Vanderbug #qconsf

Kris Trujillo@kris_trujillo :"Software engineering suffered from 'premature maturity' as a discipline" - Glenn Vanderburg #qconsf

Kris Trujillo@kris_trujillo :"In engineering....people design through documentation" thinking that kills us as software engineers/architects #qconsf

Kris Trujillo@kris_trujillo :"Wings on an airplane break at the exact point the engineers predicted"....if software could only be the same....#qconsf

Kris Trujillo@kris_trujillo :"Different engineering disciples are different" #qconsf

Therese Hansen@qedtherese :Software engineering is science and art #qconsf

Kris Trujillo@kris_trujillo :"Software engineering will be different from other kinds of engineering" - Glenn Vanderburg on Real Software Engineering #qconsf

Kris Trujillo@kris_trujillo :"In software building prototypes is essentially instantaneous and free"....prototypes over documents! #qconsf

Kris Trujillo@kris_trujillo :"The code is the document in the software engineering discipline" #qconsf

Johannes Carlén@johannescarlen :"The code is the model in software engineering" - Glenn Vanderburg #qconsf

adrian cockcroft@adrianco :#qconsf Software Engineering keynote uses a caricature of other engineering disciplines to claim Software is different #strawman #fail

Kris Trujillo@kris_trujillo :"What works best? Some variant of agile development" - Glenn Vanderburg #qconsf

David Whittaker@rundavidrun :@glv #QConSF asks, "When you can build a prototype quickly and free, why spend a lot of time on formal methods?"

Architectures You've Always Wondered About

Cloud Computing at Google by Randy Shoup

Twitter feedback on this session included:

Jim Constantine@jgc94131 :google's spanner - planet wide data store with globally synchronized clocks #QConSF

Hiroaki ONO@hihihiroro :KVM-hosted virtual machines using Ubunts or CentOS @Google #qconsf

Scaling Pinterest by Yashwanth Nelapati & Marty Weiner

Twitter feedback on this session included:

rama roberts@ramarob :Pinterest has 80 cc2.8XL MySQL servers. Yikes. #QconSF

CyberPunkie@CyberPunkie :#QConSF 100 Redis instances + 80 Memcache ones... #Pinterest

Big Data and Analytics

100% Big Data. 0% Hadoop. 0% Java by Pavlo Baron

Twitter feedback on this session included:

Alex Popescu@al3xandru :Years ago: lots of "free" data, very few tools. Now: more tools, more data, but most of it behind walls. /cc @pavlobaron #qcon #qconsf

Appwatch - A Big Data Application Monitoring System for Ebay by Bhaven Avalani & Yuri Finklestein

Matt O'Keefe attended this session:

This is a Big Data talk with Monitoring as the context. The problem domain includes operational management (performance, errors, anomaly detection), triaging (Root Cause Analysis), and business monitoring (customer behavior, click stream analytics). Customers of Monitoring include dev, Ops, infosec, management, research, and the business team. How much data? In 2009 it was tens of terabytes per day, now more than 500 TB/day. Drivers of this volume are business growth, SOA (many small pieces log more data), business insights, and Ops automation.

The second aspect is Data Quality. There are logs, metrics, and events with decreasing entropy in that order. Logs are free-form whereas events are well defined. Veracity increases in that order. Logs might be inaccurate.

There are tens of thousands of servers in multiple datacenters generating logs, metrics and events that feed into a data distribution system. The data is distributed to OLAP, Hadoop, and HBase for storage. Some of the data is dealt with in real-time while other activities such as OLAP for metric extraction is not. …

The hardest part of distributing such large amounts of data is fault handling. It is necessary to be able to buffer data temporarily, and handle large spikes. Their solution is similar to Scribe and Flume except the unit of work is a log entry with multiple lines. The lines must be processed in correct order. The Fault Domain Manager copies the data into downstream domains. It uses a system of queues to handle the temporary unavailability of a destination domain such as Hadoop or Messaging. Queues can indicate the pressure in the system being produced by the tens of thousands of publisher clients. The queues are implemented as circular buffers so that they can start dropping data if the pressure is too great. There are different policies such as drop head and drop tail that are applied depending on the domain’s requirements. …

* Entropy is important to look at; remove it as early as possible
* Data distribution needs to be flexible and elastic
* Storage should be optimized for access patterns

Through The Looking Glass - Applying Analytics to Development by Michael Feathers

Twitter feedback on this session included:

ames Morgan@jmrgn :Feathers is analyzing code complexity of files to number of commits. Cool stuff. Great way to identify refactoring opportunities. #qconsf

David Whittaker@rundavidrun :@mfeathers warns, "There's a bias in software towards a high change rate for some files. Keep the complexity low for these files." #qconsf

Erik Sowa@eriksowa :Thought-provoking examples of analytics on codelines by @mfeathers at #qconsf - e.g. churn vs. complexity.

Lene Mejlby@Mekkaz :Working as a team of co-makers requires mind-reading. @jeffpatton #qconsf

Continuous Delivery

Adopting Continuous Delivery by Jez Humble

Twitter feedback on this session included:

Johannes Carlén@johannescarlen :"Lean is not about reducing cost, lean is about reducing waste in the value stream" - Jez Humble #qconsf

Matt Barcomb@mattbarcomb : Continuous Delivery: your software should always be deployable...not deplorable /cc @jezhumble #qconsf

Changing Culture & Being a Force for Awesome by Jesse Robbins

Twitter feedback on this session included:

David Whittaker@rundavidrun :Continuous deployment is the paragon when all things in a large system are finally working correctly and harmoniously. @jesserobbins #qconsf

lane halley@thinknow :Small change, big impact. "No-one is in charge" --> "everyone is in charge" @jesserobbins on hacking culture change #QConSF

Lou Sacco@occasl :@jesserobbins talk was really good on how to be a good steward of change in the continuous delivery segment. #qconsf

Matt O'Keefe attended this session:

Teams that do Continuous Delivery are much happier. Seeing your code live is very gratifying. You have the freedom to experiment with new things because you aren’t stuck dealing with large releases and the challenge of getting everything right in one go. …

Tools are not enough (even really great tools like Chef!). In order to succeed you have to convince people that you can be trusted and you want to work together. The reason for this is understood, for example see Conway’s law. Teams need to work together continuously, not just at deploy time.

Choice: discourage change in the interest of stability, or allow change to happen as often as it needs to. Asking the question of which do you choose is better than just making a statement. …

Cultural change takes a long time. This is the hardest thing. Jesse’s Rule: Don’t Fight Stupid, Make More Awesome! Pick your battles and do these 5 things:

  • Start small and built on trust and safety. The machinery will resist you if you try sweeping change.
  • Create champions. Attack the least contentious thing first.
  • Use metrics to build confidence. Create something that you can point to to get people excited. Time to value is a good one.
  • Celebrate successes. This builds excitement, even for trivial accomplishments. The thing is to create arbitray points where you can look back and see progress.
  • Exploit Compelling Events. When something breaks it is a chance to do something different. “Currency to Make Change” is made available, as John Allspaw puts it.

Large-Scale Continuous Testing in the Cloud by John Penix

Twitter feedback on this session included:

desbiens@desbiens :Google runs more than 50 Million tests cases everyday, @johnpenix, #continuousdelivery #qconsf

desbiens@desbiens :3 roles and use cases : Build cop, release engineer, developer, @johnpenix, #continuousdelivery #qconsf

Dan Cundiff@pmotch :Google's John Penix CI stats: 200k test suites in code base, runs 10m test suites /day, >60m test cases / day, >4k CI builds / day. #QconSF

Jez Humble@jezhumble :“@LindroosKE: "People are not working in branches. Well, couple of teams are and they are not happy" << @johnpenix of Google #qconsf

stuartblair@stuartblair :@johnpenix: "public humiliation is usually enough to stop people checking in broken code" #qconsf

Jez Humble@jezhumble :CI and automated testing at scale at Google by @johnpenix #qconsf

Product Development with Continuous Experimentation by Frank Harris & Nell Thomas

Twitter feedback on this session included:

lane halley@thinknow :So cool. @Etsy 1-click deploy so well automated "VCs, dogs and PMs deploy." #qconsf

Ivan Linhares@ivanlinhares :Etsy at #qconsf, doing 20 deploys a day and automated A/B testing.

Dan Cundiff@pmotch :Etsy deploys to production 20+ times per day across 200+ developers. 10,000 deploys in a year. #QconSF

Hans Thunberg@hthunberg :Release small and often to get rid of scary releases. #etsy @ #qconsf

lane halley@thinknow :So excited to hear how @Etsy is using hypotheses and continuous deployment to be quick and confident. #QConSF @jboogie

Release Engineering at Facebook by Chuck Rossi

Twitter feedback on this session included:

Therese Hansen@qedtherese :Facebook developer sandbox load time. ~ David Mortenson #qconsf

Fernando Moribe@fgmatwork :"Facebook has only engineers, it does not have a QA department". test automation and engineers that write test #qconsf

Kris Trujillo@kris_trujillo :Impressed by the life of a Facebook engineer as described by David Mortenson.... sounds like a great culture #qconsf

Dante Briones@dantebriones :.@chuckr destroying the myth that #continuousdelivery means #testing in production #qconsf

Pete Hodgson@ph1 :Facebook error reports include a stack trace... with svn blame for each function so they know who to shout at. #qconsf

Pete Hodgson@ph1 :Facebook dev's khama is only visible to the dev, the release team... and the dev's manager :-/ #qconsf

Fernando Moribe@fgmatwork :"Tools alone won't save you. You need the right culture (company)" Facebook about release process #qconsf

Matt O'Keefe attended this session:

You should always be testing. People say it but don’t mean it, but Facebook takes it very seriously. Employees never go to the real because they are redirected to This is their production Facebook plus all pending changes, so the whole company is seeing what will go out. Dogfooding is important. If there’s a fatal error, you get directed to the bug report page.

File bugs when you can reproduce them. Make it easy and low friction for internal users to report an issue. The internal Facebook includes some extra chrome with a button that captures session state, then routes a bug report to the right people.

When Chuck does a push, there’s another step in that developers’ changes are not merged unless you’ve shown up. You have to reply to a message to confirm that you’re online and ready to support the push. So the actual build is which has fewer changes than latest. is not to be used as a sandbox. Developers have to resist the urge to test in prod. If you have a billion users, don’t figure things out in prod. Facebook has a separate complete and robust sandbox system.

On-call duties are serious. They make sure that they have engineers assigned as point of contact across the whole system. Facebook has a tool that allows quick lookup of on-call people. No engineer escapes this.

Facebook does everything in IRC. It scales well with up to 1000 people in a channel. Easy questions are answered by a bot. There is a command to lookup the status of any rev. They also have a browser shortcut as well. Bots are your friends and they track you like a dog. A bot will ask a developer to confirm that they want a change to go out. …

When Chuck does the final merge, he kicks off a system test immediately. They have about 3500 unit test suites and he can run one each machine. He reruns the tests after every cherrypick. …

Chuck’s job is to manage risk. When he looks at the cherry pick dashboard it shows the size of the change, and the amount of discussion in the diff tool (how controversial is the change). If both are high he looks more closely. He can also see push karma rated up to five stars for each requestor. He has an unlike button to downgrade your karma. If you get down to two stars, Chuck will just stop taking your changes. You have to come and have a talk with him to get back on track….

The main point is that you cannot tool your way out of this. The people coming on board have to be brainwashed so they buy into the cultural part. You need the right company with support from the top all the way down.

Cross Platform Mobile

Building Social Apps For All Mobile Platforms by James Pearce

Twitter feedback on this session included:

alex-and-r@alex_and_r :“@firt: Current Facebook stats Html5/web vs native traffic via @jamespearce at #qconsf” О_О

Maximiliano Firtman@firt :After Facebook has updated the infinite list from html5 to fully natively, users are consuming 2x content. Via @jamespearce at #qconsf

David Whittaker@rundavidrun :@jamespearce #QConSF says "don't ask for all permissions up front, build trust with your user over time and ask for permission as needed."

David Whittaker@rundavidrun :@jamespearce #QConSF claims key factor to creating a viral app is to give users a way share their experience via social storytelling.

David Whittaker@rundavidrun :@jamespearce #QConSF says "the mobile phone is the ultimate social connectivity device enabling a personally adapted experience."

Maximiliano Firtman@firt :Facebook on HTML5: “our challenges are not everyone’s challenges” @jamespearce #qconsf

Maximiliano Firtman@firt :Performance on mobile is so important. For Facebook, 2x faster, means 2x content consumption and 1.5 stars to 4 stars in 3 weeks #QConSF

Chris Ferdinandi attended this session:

Facebook gets traffic from 7,000 devices a day via the mobile web, compared to what looks like about a third of that from both iOS and Android native apps. The Facebook app is currently one of the top 50 most downloaded apps in the Apple App Store, and is #4 under social networking. And yet they get more traffic from their mobile HTML5 website than from their Android and iOS apps combined.

Native apps are great. Personally, I’d rather use the Facebook app than visit their site on mobile Safari. But people are still going to websites, even sites they visit regularly, on their mobile devices.

The Mobile Web Developer's Tool Belt by Pete LePage

Twitter feedback on this session included:

David Whittaker@rundavidrun :Thanks @petele for curating all these great web development tools for us! #qconsf -HammerJS, CodeKit, LawnChair, FastClick, JSConsole, etc.

desbiens@desbiens :Remote debugging w/ firefox, or chrome remote debug on android or ios w/safari, mobile tool belt #qconsf

David Whittaker@rundavidrun :@petele "You have the opportunity right now to 'make awesome.' Push the web and go make stuff!" #qconsf

Loutilities has attended this session:

Pete LePage (@petele) gives a great overview of all the different tools you should consider when embarking on mobile web application development.

Sublime, Codekit

Great sites for good design patterns for mobile dev:

  • shows app screenshots to get ideas
  • is also nice

Start with boilerplates from SenchaTouch, bootstrap, and maybe even jqueryMobile for improving performance of button clicks (usually delayed by 300ms for pinch zoom, slide, etc.)

To debug, can be used for remote debugging on other devices and will show stuff on site….coolio.

Hammer.js is cool for gestures to listen to those events good to store some transient data by abstracting away for IndexDB, WebSQL or LocalStorage

@media, cool for high density resolution or using CSS -webkit-image-set for both 1x and 2x image densities for hi-rez icons

CharlesProxy good to test network latency or Network Link Conditioner on Mac

Chrome Dev Tools have cool features to override geolocation, user agent, and device orientation

Dynamic Languages for the Web

Polyglot Web Development With Grails 2 by Jeff Brown

Twitter feedback on this session included:

Gareth Rushgrove@garethr :OH: "in dynamic languages it's important for the compiler to let you call things that don't exist" great point from @jeffscottbrown #qconsf

Using Node.js to Explore Holistic Internet Application Performance by Tom Hughes-Croucher

Twitter feedback on this session included:

Ismail Elshareef@ielshareef :“How does latency affect our servers?” asks @sh1mmer. Node.js is great for addressing mobile latency. Good talk. #qconsf

desbiens@desbiens :Event driven programming a la nodejs is well adapted to mobile architecture @sh1mmer #dynamiclanguage #qconsf

desbiens@desbiens :Discovering how mobile app architecture is so much different from desktop web @sh1mmer #dynamiclanguage #qconsf

stuartblair@stuartblair :Kudos to @sh1mmer for delivering the most compelling case for node.js I've seen yet. #qconsf

Java Renaissance

Android App Anatomy by Eric Burke

Twitter feedback on this session included:

David Whittaker@rundavidrun :In #Android push to server, add bundle to task queue, check network, remove from queue if available, repeat until empty. @burke_eric #qconsf

David Whittaker@rundavidrun :@burke_eric demos some of @square's best open source #Android libs: Tape (FIFO), Otto (bus), and Dagger (CDI) #qconsf

Loose Concurrency & CAP Theorem Today

Exploiting Loopholes in CAP by Michael T. Nygard

Twitter feedback on this session included:

rama roberts@ramarob :Nodes in diff. groups can reconstruct linear history using GPS time to get around CAP limitations. See Google Spanner. - @mtnygard #QConSF

adrian cockcroft@adrianco :#qconsf @mtnygard now talking about PACELC by @daniel_abadi Partition means trade off Avail/Consistency, Else trade off Latency/Consistency

Fear No More: Embrace Eventual Consistency by Sean Cribbs

Twitter feedback on this session included:

Alex Popescu@al3xandru :"Any sufficiently large system is in a constant state of partial failure" @justinsheehy via @seancribbs #qconsf

The CAP Balancing Act: Building Reliable Data Stores by Jeremy Edberg

Alex Handy reported for SD Times:

Jeremy Edberg, manager of cloud operations and reliability engineering at Netflix, said he learned how to store data for quick access while he was working as an engineer at the news site Reddit.

Of the Reddit stack, he said, “There was Cassandra, Postgres and memcached. The biggest way Reddit does their resiliency is through the sharding of Postgres. Reddit splits writes across four master databases, which have different types of things. Each one has at least one replica in another zone. Votes on links only get one replica, while comments have 17 replicas spread across different zones. Reddit uses Postgres like a NoSQL store. There are just two tables: a thing table and a data table.”

Edberg said one of the keys to success was using consistent key hashing in memcached. “It's important. If you're using a naive hashing algorithm, which every memcached library uses by default, [you can run into performance issues],” he said. “If you add a server, it will shift three quarters of your data to a different server, so you invalidate three quarters of your cache. Consistent key hashing is the reverse: One quarter of your data will be in the wrong place. This makes it so you don't put a huge load on your database when you add a new node. Protip: Use consistent key hashing. If you're using memcached, you're probably using a naïve one if you haven't set it manually.”

Finally, Edberg said that designing for scalability from the start is key, but the initial number of servers is also an important decision. “Going from two to three is hard,” he said. “When you have one server, going from one to two is even harder to do, which is why I recommend going for three from the beginning. Scaling from three to six is a lot easier than from one to two or two to three.”

Mechanical Sympathy

JVM Mechanics: A Peak Under The Hood by Gil Tene

Twitter feedback on this session included:

Pavlo Baron@pavlobaron :Gil Tene at #qconsf: "most of the people are measuring something that is not a problem"..."99.9% of the time your heart will keep beating"

Pavlo Baron@pavlobaron :Gil Tene at #qconsf: "backpressure doesn't let us measure bad things - they just don't get reported" - this guy is amazing!

Pavlo Baron@pavlobaron :Gil Tene at #qconsf: "percentiles are good, but always measure the maximum"

PHP on the Metal with HHVM by Keith Adams

Twitter feedback on this session included:

Martin Thompson@mjpt777 :"memcpy will fill about 1/3 of the i-cache because it is 11KB of code!" - Keith Adams #qconsf

Running with the Devil: Mechanical Sympathetic Networking by Todd Montgomery

Twitter feedback on this session included:

Martin Thompson@mjpt777 :"Batching is a key technique to making applications perform better..." - @toddlmontgomery #qconsf

Martin Thompson@mjpt777 :Apply "Application Level Framing" to greatly simplify your networking designs. @toddlmontgomery #qconsf

NoSQL Emergence

Architectural Patterns for High Availability by Adrian Cockcroft

Twitter feedback on this session included:

John Martin@tekBuddha :“Things we don’t do at Netflix: plan for capacity.” - @adrianco #QConSF

Dan Cundiff@pmotch :Listening to Netflix @adrianco talk about HA patterns. I knew about Chaos Monkey, but not Latency Monkey and Chaos Gorilla. #QconSF

Dan Cundiff@pmotch :At #QconSF I've observed several big names state they're migrating to @ubuntu. Netflix is for example. Makes sense to me.

Pavlo Baron@pavlobaron :Netflix pushes devs to open source all the code they write. This creates natural pressure for quality. I so steal this idea... #qconsf

CyberPunkie@CyberPunkie :#QConSF Netflix and massive scale: 500 Cassandra instances!

MongoDB Large-Scale Data Centric Architectures by Kenny Gorman

Twitter feedback on this session included:

Jim Constantine@jgc94131 :mongoDB jewels: your indexes must fit in memory / one write lock / ops nightmare / return code on insert doesn't tell you much #QConSF

Loutilities has attended this session:

Kenny Gorman presents on the gotchas with MongoDB and how to scale it appropriately. 

Denormalize till it works to get to true scale; split vertically and then horizontally

MongoDB is an OPS nightmare

MongoDB challenges:

  • Course lock scope; 2.2 has DB lock scope; 2.0 was global
  • Visibility
  • Schema – still need to business stuff
  • When bad things happen…it goes down fast

Design for scale:

  • Denormalize into diff. DB’s
  • Tune your workloads
  • NoORM; dump it
  • Shard early (Think about your shard key in advance)
  • Replicate (How to get availability)
  • Load test pre-prod

Programmable Web

Caching Hypermedia APIs by Tim Stokes

Twitter feedback on this session included:

Kris Trujillo@kris_trujillo :WS-* The Hypermedia Dark Ages - great way to describe it! programmable web #qconsf

Kris Trujillo@kris_trujillo :Transfer state in the body not in the URI - caching hypermedia #qconsf

Kris Trujillo@kris_trujillo :Learned a new trick with Content-Location playing into caching of POST responses - great stuff! - caching hypermedia #qconsf

Kris Trujillo@kris_trujillo :Keys to remember when caching hypermedia #qconsf

Kris Trujillo@kris_trujillo :Best approach to designing hypermedia with caching in mind #qconsf

Real-Life Stories on Building Hypermedia APIs by Mike Amundsen

Twitter feedback on this session included:

Kris Trujillo@kris_trujillo :Bash script as a client to a Hypermedia API - very cool! @mamund #qconsf

Kris Trujillo@kris_trujillo :Media Type is a pre-defined collection of Hypermedia Factors - programmable web #qconsf

Kris Trujillo@kris_trujillo :Message design, not object design - programmable web #qconsf

Kris Trujillo@kris_trujillo :People who are use to serializing objects will have a hard time with hypermedia - programmable web #qconsf

Kris Trujillo@kris_trujillo :APIs should be usable - programmable web #qconsf

Kris Trujillo@kris_trujillo :Four important aspects to consider when applying affordances to hypermedia - safety, idempotent, mutability, transclusion #qconsf

Kris Trujillo@kris_trujillo :If you get the design right your users will be able to do things you never even imagined - programmable web #qconsf

Alex Handy reported for SD Times:

Mike Amundsen, principal API architect at Layer 7, said that enterprises are extremely concerned with governance and policy enforcement on APIs. But he also said that these new focuses on SLAs and productizing APIs has resulted in some interesting shifts in how APIs are handled inside big businesses.

“More often now, I'm seeing people in sales meetings who say, 'I'm not the lead architect, I'm the product lead,’” he said. “They use that phrase. Even in the IT space, sometimes these product leads are inward-facing.”

In short, APIs are being tended to with the same care and feeding a regular software product would receive. This even extends to how companies approach API design.
“Other people showing up in the room are user-experience people,” said Amundsen. “I'm seeing people are using stories and personas for the API designs. User-centered design is a big thing for them for some of our customers.”

Unlike many of the API management themes out today, Layer 7 is not simply pushing REST, said Amundsen. In fact, one of the best practices his company suggests is to allow the API to be loosely coupled with its transport.

“You've got this compositional layer already that's your business piece,” he said. “We're telling people to actually consider the API as this thin veneer—like a UI—and a representation layer. Your components give you data, you compose that into something interesting, then you're representing it to the client, either as a SOAP message or a JSON payload, or as REST.”

Realtime Web

Integrating the Classic MVC Architecture with Real Time by Daniel Erickson

Loutilities has attended this session:

MVC is here to stay because…

  • It provides structure to a project
  • Allows people to easily jump into a project and multiple people at that
  • Easier to get started with

Real-time is great for:

  • Getting instant feedback from your users
  • Works well on mobile (Voxer)
  • WebSockets over long/short polling

How can MVC and Real Time can be mixed?

Enter Geddy…if you know Rails, NoSQL/SQL ORM layers and templating languages than Geddy is a natural progression. PostGres, Riak, MongoDB adapters; EJS and Jade TL’s. Supports RT OOTB.

Geddy gives you a reach ORM/MVC framework that you can readily work into your Node.js applications.

The Realtime Web: HTML5 WebSockets, Engine.IO, Socket.IO, SPDY, HTTP2.0 & Beyond by Guillermo Rauch

Twitter feedback on this session included:

adrian cockcroft@adrianco : #qconsf talk on websockets and Real Time Web Is a great introduction by Guillermo, author of and

Tales from Silicon Valley

Building for a Billion Users from Silicon Valley by David Mortenson

Twitter feedback on this session included:

Christian Sanz@csanz :"Facebook doesn't have QA, only engineers, they are responsible for quality" - David Mortenson, Facebook Engineering manager #QCon #spoton

Alex Handy reported for SD Times:

To get to a thousand engineers, said Mortensen, Facebook set up an internal boot camp where it trained groups of incoming engineers to get them up to speed on the development tools, environments and practices they'd be dealing with. He said this directed effort for on-boarding engineers helped to ease the pain of rapid growth over the past seven years.

In the end, all of Facebook’s boot camp efforts were directed at improving developer efficiency. As Mortensen said, any discrete task that takes over two minutes for a developer might as well be a coffee break because it interrupts productivity in a significant way.

To this end, Facebook spent a lot of time and effort to streamline its development environments. Every developer at Facebook has his or her own development machine in the data center, which contains a complete running copy of the Facebook code. Developers can log in and manipulate the code, then glean the changes, and start pushing toward integrating them into Facebook proper.

The problem, said Mortensen, is that the development environments were initially extremely responsive because Facebook is written in PHP. That means changes would be instantly useable on a development machine without a need to compile. Unfortunately, as Facebook's code became more complex, load times for development boxes became longer and longer.

To solve this, Facebook created HipHop, a PHP-to-C++ compiler. Once compiled, the C++ code would run much faster, but for the developers, every change now required a recompile, which could take up to 10 minutes. For Mortensen, 10-minute wait times were deadly to productivity.

After building an interpreter to stand in for the compiler in development machines, the wait time was cut down to around 10 seconds, but for Mortensen and the developers at Facebook, this was still too slow. The answer was to use a JIT compiler.
“We turned this on this spring, and got some awesome results,” he said. “When we went to the JIT compiler, there was cheering in the streets. But as a reminder, it's never done. The site continues to get richer. This is one of those things that they continue to keep fighting on. It's been a fun saga. With enough effort, almost anything that matters can be made efficient. You really want to make your engineers fast.”

Engineering Elegance: The Secrets of Square's Stack by Bob Lee

Twitter feedback on this session included:

Kris Trujillo@kris_trujillo :"Design is not just about pixels but the entire user experience" "To end users transactions never fail" - good stuff @crazybob #qconsf

Taming HTML5 & JavaScript

Javascript Performance Patterns by Stoyan Stefanov

Twitter feedback on this session included:

David Whittaker@rundavidrun :Load JS in iframe to avoid blocking page load but use parent doc: this is not the document you are looking for! @stoyanstefanov #qconsf

Faruk Ateş@KuraFire :“There’s lies, damn lies, and performance advice.” — @stoyanstefanov at #qconsf

David Whittaker@rundavidrun :@stoyanstefanov demonstrates how to measure performance of your JS code using #qconsf

David Whittaker@rundavidrun :Optimize JS by reducing the number of accesses to the DOM; think of it as crossing a bridge with a high toll. @stoyanstefanov #qconsf

SPDY, err.. HTTP 2.0 Is Coming! by Roberto Peon

Twitter feedback on this session included:

Ismail Elshareef@ielshareef :SPDY talk: don’t focus on Page Load Time. It’s irrelevant to users. Users care about how “fast the page becomes useful.” #qconsf

Ismail Elshareef@ielshareef :SPDY’s Server Push: server knows what resources the client needs and sends it over w/o explicit req from client. #qconsf #webperf #spdy

Dio Synodinos@synodinos :"It's not that we don't like proxies. We don't like transparent, poorly configured ones" #qconsf #spdy #tls 

The Agile Team Field Guide

Building a Learning Organization From Any Level by Matthew Barcomb

Twitter feedback on this session included:

David Whittaker@rundavidrun :Learning organizations help companies innovate, adapt, improve quality, increase employee satisfaction, and viability. @mattbarcomb #qconsf

David Whittaker@rundavidrun :When approached with any challenging activity: take a learning stance, admit ignorance, and allow failure. #agile @mattbarcomb #qconsf

itsmemacelle@itsmemacelle :Learning : action vs reflection, informal vs formal, input vs output, momentum, options, create an environment #qconsf

David Whittaker@rundavidrun :Learning Momentum (n.) - the more you learn anything, the easier and faster it is to learn other things. #KeepLearning @mattbarcomb #qconsf

David Whittaker@rundavidrun :"Come to work, do your job, and be done is a myth." To keep yourself relevant you need to constantly be learning. @mattbarcomb #qconsf

itsmemacelle@itsmemacelle :there are key times during the day that're good for learning goal is not to finish, but to practice "learning together" @mattbarcomb #qconsf

User Experience (UX)

Better Product Definition with Lean UX and Design Thinking by Jeff Gothelf

Twitter feedback on this session included:

lane halley@thinknow :Your requirements / More effectively expressed / As hypotheses @jboogie #QConSF talk intro

Mike Long@mblongii :Articulate your assumptions, define hypothesis, do lightweight tests to validate @jboogie #QConSF

Mike Long@mblongii :Measure outcome, not output :) @jboogie #QConSF

desbiens@desbiens :#qconsf value learning over growth @jboogie

Lene Mejlby@Mekkaz :#qconsf Design thinking is about observations @jboogie

Lene Mejlby@Mekkaz :#QConSF Lean UX and design thinking is not just for designers @jboogie

desbiens@desbiens :Focus on outcomes ... Lean ux @jboogie # #qconsf

esbiens@desbiens :Test the value first. @jboogie #leanux #qconsf

Mathieu Lalonde attended this session:

I had a bit of a realization ("ha ha!") when Jeff clarified that requirements should be viewed as assumptions.  Then Jeff explained how these assumptions should  be proved or disproved in the cheapest way possible.  In some cases that's using a minimum viable product (MVP) in other cases mock ups or paper prototypes might do the trick!

Another practical take away was how we could write user stories in the following style: "We believe that if we implement feature XYZ, our traders will be able to use it so they can make more trades per day."

Co-making Great Products by Jeff Patton

Twitter feedback on this session included:

lane halley@thinknow :No more "who's to blame" / Solve the problems together / Co-making products @jeffpatton #QConSF talk intro

Lene Mejlby@Mekkaz :Design is a learning experience, as well as common sense @jeffpatton #qconsf

Hans Thunberg@hthunberg :Jeff Patton - "we are all responsible for the things we create. Let's co-operate". #qconsf

Lene Mejlby@Mekkaz :Design is not a product the designers produce, it is a process designers facilitate @jeffpatton #qconsf

Lene Mejlby@Mekkaz :We are not here to build software, we are here to change the world @jeffpatton #qconsf

Tomer Sharon@tsharon :The faster you build crap, the more crap you have. - @jeffpatton #qconsf

Jeff Gothelf@jboogie :Just because it’s written on a sticky note doesn’t make it less valid. - @jeffpatton #qconsf #LeanUX

Lene Mejlby@Mekkaz :You'll need to leave the office to learn what you don't know @jeffpatton #qconsf

Lene Mejlby@Mekkaz :Telling your product's story will help you identify problems @jeffpatton #qconsf

High-quality, Impactful, Fast UX Research for Engineers by Tomer Sharon

Twitter feedback on this session included:

desbiens@desbiens :Dont listen to user, observe behavior instead @tsharon #qconsf

desbiens@desbiens :Attitude is not a predictor of behavior #leanux @tsharon #qconsf

Bhavik Joshi@joshi_bhavik :Can't help but observe parallels between #custdev #leanlaunchpad & high quality,impactful & fast lean UX design #qconsf @tsharon @thelaunch

Lene Mejlby@Mekkaz : Ask about experiences rather than predictions of the future @tsharon #qconsf

desbiens@desbiens :Ask opinion only after user experience the thing or ask about recent past experiences #leanux @tsharon #qconsf

desbiens@desbiens :Observe, observe, observe #leanux @tsharon #qconsf

Lene Mejlby@Mekkaz :Eye trackers are not mind readers! They can tell you where people are looking not what they see... @tsharon #qconsf

lane halley@thinknow :"Noticeability test" let the user use the page, then give them a cut up print out and have them recreate the page. @tsharon #QConSF

desbiens@desbiens :Noticeability test #leanux @tsharon #qconsf cut the screen in pieces and observe user assembling the puzzle

Lene Mejlby@Mekkaz :A standalone A/B test only tells you what happened, not why. Do A/B usability testing instead @tsharon #qconsf

lane halley@thinknow :User research reports wasteful, don't support team learning. Asked "where is the report?" @tsharon asks "what do you want to know?" #QConSF

Mathieu Lalonde attended this session:

The talk was very interesting and really clarified some concepts about usability analysis/research.  One of the key take away was: "Don't listen to users!  Observe their behavior."
In less than 50 min, Tomer managed to explain the psychology behind his guiding principles and then discussed three techniques that could be used:
1. High-quality noticeability test
2. Impactful A/B usability study
3. Fast, colourful collaboration tool
I thought all three techniques were very practical and seemed effective.   All in all, I found the presentation's delivery very effective and its content eye-opening.  Good job Tomer!

Quora on Mobile: A Product-Centered Approach to Multi-Platform Deployment by Anne K Halsall

Twitter feedback on this session included:

lane halley@thinknow :"An app ls like an accessory to your mobile environment, I think a lot about how to honor each viewpoint." @annekate #QConSF

lane halley@thinknow :"When you go to mobile you could probably lose 20% of the features in your product and no-one would notice." @annekate #QConSF

Lene Mejlby@Mekkaz :Really understand the platform you're working on! @annekate #qconsf

Lene Mejlby@Mekkaz :Tailoring the product for a new platform is worth it @annekate #qconsf

Mike Long@mblongii :Tailoring your product for a new platform will pay off. Don't just port it and fake it. Customers will revolt. @annekate #QConSF

Mike Long@mblongii :Build once, deploy anywhere is a myth. @annekate #QConSF #mobile

Lene Mejlby@Mekkaz :Design new products for mobile before Web @annekate #qconsf

desbiens@desbiens :The mobile deferentiator is not location @annekate, #ux #qconsf

ene Mejlby@Mekkaz :Make sure you have the data you need to make your apps better @annekate #qconsf

User Experience - More Than Just a Pretty Stick by Lane Halley

Twitter feedback on this session included:

Mike Long@mblongii :Focus on the problem, not the solution. Problems start with People ;) @thinknow #QConSF

Solutions Track: No SQL

Common Graph Database Use Cases by Emil Eifrem

Twitter feedback on this session included:

Peter Yu@ThePeterYu :Twitter is running a graph database in a JavaVM with heap size 177GB. #yam #QConSF #GraphConnect

Loutilities has attended this session:

Trends in BigData & NoSQL:

  • Increasing data size
  • Increasing connected data
  • Semi-structured data
  • Architecture – a facade over multiple services

Categories of NoSQL:

Key/Value store (heritage from Amazon Dynamo) -

  • Riak, Redis, and Voldermort are implementation examples
  • Strengths is that it’s a simple but also weakness

Columnar Stores

  • BigTable – every row can potentially have it’s
  • Examples are HBase and Cassandra, Hyper Table
  • Supports semi-structured data but weakness is nested or connections

Document Database

  • Collections of documents, JSON that could potentially be nested
  • 90% uptick on NoSQL in MongoDB but also CouchDB
  • Strength is simplicity of database, but hard to do connected data

Graph Database

  • Nodes and Relationships
  • Examples are Neo4j, InfiniteDB, OrientDB, etc.
  • Great for connectiveness of data and complexity but harder to scale to size

Sustainable Software Development by Alexander von Zitzewitz

Twitter feedback on this session included:

David Whittaker@rundavidrun :Alexander Zitzewitz: "Universities don't teach students how to maintain large complex software systems." How can we fix this? #qconsf

David Whittaker@rundavidrun :Alex Zitzewitz: "Is software 'done' when it works? Imagine building planes this're done when it looks like an airplane?!?" #qconsf

David Whittaker@rundavidrun :Alex Zitzewitz: "Technical quality manifests itself in every line of code. Highest cost impact: having a sustainable architecture." #qconsf

David Whittaker@rundavidrun :Alex Zitzewitz: "Circular dependency in code is like 'code cancer'. Reduce coupling by creating interfaces." #qconsf

David Whittaker@rundavidrun :Alex Zitzewitz points us to the essential software quality cheat sheet. Free download: … #qconsf

The Design of a SQL Interface for a NoSQL Database by Stephen Buxton & Mary Holstege

Twitter feedback on this session included:

Kris Trujillo@kris_trujillo :"Why SQL on NoSQL? Because we are in the enterprise" - NoSQL #qconsf

Kris Trujillo@kris_trujillo :"Focus on structured pieces of data" - SQL on NoSQL #qconsf

Opinions about QCon

Twitter feedback on this session included:

Woody Zuill@WoodyZuill :#QConSF staff: You are the best! Thanks for your hard work and attention to detail! What a pleasure working with you. Great Conference!

Josh Warner-Burke@hybrididentity :Food at #qconsf is solid!

Jonathan Schlaepfer@Schlaeps :This food is AMAZING. #qconsf

Kris Trujillo@kris_trujillo :Agreed! "@tsharon: Awesome little idea to rate #qconsf talks. Vote as you go out of the room: green, yellow, red.

Jeff Gothelf@jboogie :How cool is this? Instant feedback on my #qconsf talk an hour after I finished. Love it.

Jeff Gothelf@jboogie :This is how feedback is collected at #qconsf after each talk. Everyone taps on their way out. Super cool.

adrian cockcroft@adrianco :#qconsf recipe for evening keynote. First feed free beer to everyone then put on an entertaining speaker. Good job!

drian cockcroft@adrianco :Problem with attending lots of inspiring #qconsf talks is that I'm now re-writing my slides for Friday's talk, but don't have time...

Stoyan Stefanov@stoyanstefanov :Exstatic that #qconsf folks liked my js perf talk. Reviews: 0 red, 30 yellow, 87 green, pro'ly the best in my speaker "career"

kennygorman@kennygorman :Had a great time at #qconsf 2012, great to see so many brilliant people!

iliketoprogram@iliketoprogram :So much input! Feels like my head is going to explode! Thank you #qconsf and presenters! this was an awesome week!

Jussi Nieminen@jussinieminen :Thank you @QConSF #QConSF for great conference! See you next year! Over and out!

Kalle Lindroos@LindroosKE :All good things come to an end. Thx @qconsf! #qconsf


Twitter feedback on this session included:

Burak Yenier@BurakYenier :@tsharon talk on User Experience testing was stellar. The kind of talk that brings me back to #QConSF

osh Warner-Burke@hybrididentity :My mind is swimming with new ideas #qconsf

Pavlo Baron@pavlobaron :"I've put it up on GitHub" as a phrase clearly dominates #qconsf talks I've attended

adrian cockcroft@adrianco :Inspired by #qconsf and thinking about a new talk based on mining the data in Edda, which was released this week @NetflixOSS

Jonathan Calvert@orion_quotient :Companies like Google and Facebook, Continuous deployment is part of their competitive advantage -- Great quote from #qconsf

Jonathan Calvert@orion_quotient :Back in #dc after adventures at #qconsf - the real work of action from learning begins!


The success of each QCon lies in the fact that each is created virtually from scratch by the small team of experts that make up the program committee.

QConSF 2012 committee members include Jez Humble, Patrick Linskey, Steve Vinoski, Dio Synodinos, Aino Vonge Corry, Floyd Marinescu, and Nitin Bharti. The committee ensures that new tracks reflect current trends in enterprise software development, and that sessions are both educational and practical. InfoQ will be publishing video-casts of most presentations over the next few months. The publishing schedule can be found on the QCon SF web site.

QCon is co-produced by Trifork (GOTO and other Conferences) and InfoQ’s parent company, C4Media. QCon conferences include QCon New York, QCon San Francisco, QCon London, QCon Berlin, QCon Beijing,QCon Hangzhou, QCon Tokyo, and QCon Sao Paulo

Rate this Article


Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p