BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

Key Takeaway Points and Lessons Learned from QCon San Francisco 2011

Posted by Abel Avram on Dec 09, 2011 | NOTICE: The next QCon is in San Francisco Nov 3-7, Join us!

QCon San Francisco, our our practitioner-driven conference designed for team leads, architects and project managers, was held mid November. With over 650 practitioners in attendance, the conference kicked off with two days of tutorials days presented by over 80 industry experts and authors followed by three conference days with 15 tracks and over 100 speakers covering topics ranging from cross-platform mobile development to big data architectures and NoSQL.

The following takeaways include summaries and blog entries written by editors and practitioner attendees for all keynotes, tracks and sessions along with aggregated twitter feedback during the event. InfoQ will follow this by publishing video-casts of most presentations over the next few months. The publishing schedule can be found on the QCon SF web site.

Table of Contents

Keynotes

Advancing Agile to the Next stage with Lean

Agile in the Midlife

Architecture Case Studies I

Architecture Case Studies: Cloud

Beautiful Code

Big Data and NoSQL

Cross Platform Mobile Development

Functional Web

HTML5 & JavaScript, State of the Art

New Languages Changing Your World!

Software Architecture Improvements

Systems That Never Stop

Web APIs

Why is Java Still Sexy

Cloud and NoSQL Solutions

Solution Track Friday

Social Events

Opinions about QCon

Takeaways

Conclusion

Keynotes

Objects On Trial with Michael Feathers & Brian Foote & Richard P. Gabriel & Joshua Kerievsky & Eliot Miranda & Dave Ungar

Jean Lambert summarized this panel session including a picture of the judge and the jurors:

This morning, objects are being put on trial. Before us is the head Judge and a panel of jurors accusing objects of the following serious charges:

  1. Did no deliver the promise of code reuse, reusable component marketplaces and frameworks that would take the drudgery out of programming
  2. Inability to communicate intent clearly amongst themselves
  3. Insisting on intimately encapsulating behaviour and mutable state, and by having paid inadequate attention to concurrent, have left the industry vulnerable to what some have dubbed an ill-conceived "Neo-Functional Renaissance"
  4. Eschewing static type information forcing implementers to sacrifice performance in the name of linguistic simplicity
  5. High skill and education required to adequately craft state-of-the-art code to a level that is beyond what the industry is prepared to pay for

In the accused box are the following:

  1. A Penguin (Tux)
  2. A coffee mug (Java)
  3. A UML object (a.k.a. person, corporation, domain, identifier, serializable, cloneable, iterable)

Twitter feedback on this session included:

johnkwaters John K. Waters: Onstage now at #QconSF: "Objects On Trial." They're actually going for a mock trial, complete with wig and robe for the judge.

rbarcia Roland Barcia: According to #QConSF keynote, Objects are on trial for crimes against Computer Science. Java and UML being called out.

gbatcisco Gary Berger: #qconsf Objects "Guilty as Charged"

johnkwaters John K. Waters: Onstage now at #QconSF: "Objects On Trial" we failed to make things simple, says panelist. Complexity should be on trial.

johnkwaters John K. Waters: Onstage now at #QconSF: "Objects On Trial" comparing panelist's decision to get a vasectomy--no such discipline in the OO world!

johnkwaters John K. Waters: Onstage now at #QconSF: "Objects On Trial" we've evolved into "software in the age of sampling."

johnkwaters John K. Waters: Onstage now at #QconSF: "Objects On Trial" reuse was going to make code too cheap to meter, but it didn't work out that way. Copy&paste won

johnkwaters John K. Waters: Onstage now at #QconSF: "Objects On Trial" comparing the Space Shuttle program, which emphasized "reuse" of the craft to code reuse movement

johnkwaters John K. Waters: Onstage now at #QconSF: "Objects On Trial:" 1980s were the golden age of reuse.

robdaemon Rob Roland: Do you swear to tell the truth? The whole truth? So help you, Knuth? #QConSF

Things I Wish I'd Known by Rod Johnson

Jean Lambert reported from this keynote:

So, if your planning to do a start-up, you need to ask yourself: where is the opportunity? Remember that technology must come first! There must be an opportunity for disruption. Where is there a market gap? Did everyone else fail to see something? Did someone else screw up? …

Do NOT start writing code... it's addictive if you're any good at it. …

Once you've validated your business idea, believe it! Vision must matter to you at an emotional level. Align you team on the vision. Unless someone like you cares a whole awful lot, nothing is going to get better. …

Being an entrepreneur means a lot of risk. …

A great team makes a great company; This is what investors are looking for. Building a team is about complementary skills. First, you need to understand yourself and know what things you are good at and what things you suck at. Assemble individuals who possess the required skills and agree on your vision and ambition. Remember that you cannot afford to have different goals. …

Finally, a few mistakes you shouldn't do:

  • Don't piss people off, it will come back to bite you
  • Don't let customers drive your product roadmap

Twitter feedback on this session included:

rundavidrun David Whittaker: @springrod at #qconsf reminds us that often it's the questions that are complicated; the answers are usually very simple. #DrSeuss

bigballofmud Brian Foote: @springrod warns of the peril of having a Code Monkey on your Back / Green Bar Fever / Addictive qualities of TDD #qconsf no 2 code 1st

jrolstad Josh Rolstad: When beginning a startup: The technology comes first. #qconsf

Product Engineering by Mike Lee

Twitter feedback on this session included:

frubim Felipe Rubim: Funny evening keynote by @bmf - Mike Lee - on Product Engineering: "What's worse than technical debt? Social debt" #qconsf

ramrubio ramrubio: #qconsf mike lee keynote.... new things must be better than old things they replace

adrianco adrian cockcroft: Mike Lee giving #qconsf keynote on product engineering in a Three Amigos outfit, huge sombrero

CalebeSan Calebe Santos: Mike Lee @bmf keynote at #qconsf t.co/qVwujiia

 

Advancing Agile to the Next stage with Lean 

Soup To Nuts - Harnessing Lean To Break Through The Local Optimisation by Russell Miles

Steve Mercier attended this session:

Our software development actions should be rooted in values (Values lead to Beliefs which lead to Behaviors and eventually to Actions) …

Value Statement and Value Assertions as tools

He then suggested that we develop a Value Statement that should answer the why portion of any prescribed change and to then develop the Value Assertions. The idea is to validate the measurable (automatically or not) portion of the value statement (He introduced the concept of Value Tests as tests against the market, the stakeholders - this is done now, but generally outside of IT completely). The bad news is that the toolset for value assertion/feedback is not good but if we find a way to deliver smaller increment of value and measure it, we get the benefits (similar to what Facebook seems to be doing with what they call experiments);

The good news: seeing delivered value (aka progress) around us motivates us (even if it is in another team) - this can be a key to motivate teams. …

Conclusion

We should be more Value-Driven in everything we do (track it, measure it, etc.). Why is Applesuccessful? Could it be because it is essentially value-driven and not technology driven?IT (software development departments) should stop seing itself as a slave of the business but as an important part of the business instead that can use technonlogy to deliver business value... a big change!

Understanding the Magic of Lean Product Development by Don Reinertsen

Steve Mercier attended this session and took extensive notes:

Here are some ideas the speaker considered good enough to be imported from Lean Manufacturing to Software Product Development domain:

  1. queuing theory: queues sizes vs resources capacities is not linear, but exponential... the more you utilize the capacity, the more the queue size explodes (ex: traffic at rush hour: reducing capacity by one lane out of 4 at rush hour will cause more than 25% delay in the queue time)... To produce an economic output, we should take great care at selecting the right amount of capacity and use the economics of queues; we should measure queues sizes in our product development cycle and search for $ vs excess product development resources with 3 curves, a-total cost, b-cost of excess capacity and c-cost of delay. Why it works so naturally for manufacturing? Because queues are physically visible. In Software Product Development, they are invisible... however, it is really useful to know how much delaying a certain software release actually costs globally.
  2. Batch size is an very important aspect as well (ex: coffee break with 100 person at the same time vs 5 coffee breaks with 20 persons for each one) - see the economic lot size equation reference (created in 1930s). To select the right batch size, you need to measure. There are huge benefits in small batch testing for example (less debug complexity therefore cheaper debug, less open bugs, therefor cheaper testing, faster cycle time, early feedback therefore lower cost changes, ... in the end, better economics). TBD: a link here with CI small increment theory
  3. WIP (Work In Process) constraints: very powerful to deal with accumulation in states (ex: maximum 10 items in Coding, ready to test and testing states); See Little's formula that says that waiting time in the queue is function of number of items in the queue and departure rate of the items. A useful tool for this one is the visual WIP boards - those boards makes non physical/invisible items as physical tokens on the boards - the horizontal axis changes from the traditional time axis to an axis on the software state. Another useful tool is a Cumulative Flow Diagram (showing Cumulative quantity vs time). It permits to see arrivals vs departure times in the queues, therefore, both at the same time the queue sizes (by taking on the x slices) and quantities in queues (by taking y axis slices).
  4. Synchronized cadence: it makes wait time predictable (ex: dedicated support time on each day vs a % of the time of support at an unknown time); Cadence sets an upper bound for wait time and favor less context switching costs.

Conclusion

  1. The importance of Math: The underlying mechanisms behind lean methods can be used in Software Product Development; these methods affect more than one measure of performance so trade-offs are necessary; An economic model gives comparable measures with common units. (See Life Cycle profit impact (ex: one month delay can cost 500K$)).
  2. We need to calculate more economically and less with intuition! The speaker measured scientifically that any analysis (even poor ones), beats intuition in software development!

Twitter feedback on this session included:

@JoshuaKerievsky: "People are measuring cycle time and not looking at queuing time." @DReinertsen #QconSF #lean

stephen_younge Stephen Younge: I never thought an old guy in a suit could blow my mind! Don Reinersten talk on Lean Product Development at #qconsf was INCREDIBLE.

JoshuaKerievsky Joshua Kerievsky: "The queues in product development are financially invisible." @DReinertsen #QconSF

Agile in the Midlife

Lean Startup: Why It Rocks Far More Than Agile Development by Joshua Kerievsky

Twitter feedback on this session included:

JoshuaKerievsky Joshua Kerievsky: Today at #qconsf in San Francisco, I'll be talking about #leanstartup: how it rocks far more than #agile development.

 

pascal_collet Pascal Collet: @JoshuaKerievsky presented Lean Startup at #qconsf and got me all excited about the possibilities beyond Agile

scc1976 Sean Comerford: Very interesting talk on Lean Startup by @JoshuaKerievsky at #qconsf.. best session I've attended so far

stephen_younge Stephen Younge: #agile vs. #leanstartup at a glance -- from #qconsf t.co/XsuUSR2l

stephen_younge Stephen Younge: "If you think you're cutting edge for doing Agile in 2011, you're not." -- @JoshuaKerievsky on #leanstartup at #qconsf

timotta Tiago Albineli Motta: We all have to be confortable to debug systems in production #qconsf

jrolstad Josh Rolstad: Lean startup thought: I wonder how many features I've built in the enterprise that are never used once released. #qconsf

bigballofmud Brian Foote: @joshuakerievsky admonishes us to learn to Visualize Waste, so that we can better dispose of it. #leanstartup #qconsf

bigballofmud Brian Foote: @joshuakerievsky contends that #lean and a #leanstatup mentaility are much more than an Anorexic Agile rehash #qconsf

 

Retrospectives, Who Needs Them, Anyway? by Aino Vonge Corry

Steve Mercier attended this session:

What is a retro? Essentially it is a chance to reflect and learn (historically called postmortem made to learn from failures) …

What happens during retros - the 5 steps

  1. Set the stage - getting ready (ex: create safety, I'm too busy)
  2. Gather data - the past (ex: artifacts contest, Timeline)
  3. Generate insigths - now (ex: fish-bone, 5 whys, patterns and shifts)
  4. Decide what to do - the future (ex: Making the magic happen, SMART goals)
  5. Closing the retro - retro sumary (ex: what helped, what hindered, delta)

You start the next retros by asking how we are doing with regards to previously set SMART goals.

Don't "sell" retros, instead sell a way of learning how to:

  • avoid repeated mistakes
  • identify and share success

Take away:

If you leave out a step in retrospectives, you will make problems emerge that you do not understand fully nor find solutions to.

Architecture Case Studies I

Architecting Visa for Massive Scale and Continuous Innovation by John Davies

Rowan Hick attended this session:

Visa, in 18 months, launched their product V.Me, using an Agile development process, and, sitting down ? JRuby on Rails on the front end with Java on the backend. So lets repeat that back to ourselves, one of what must be amongst the largest IT shops in the world, who are probably the most paranoid in the world for security and process etc have started adopting Agile, and using rails. Cue blowing of mind.

Twitter feedback on this session included:

jtdavies John T Davies: This is the first release (at #QCon) of the secret project we've been working on at Visa (V.me) t.co/z87yWAcY Please follow @vmedev

vmedev V.me Developer: @jtdavies and @ekatzin showed the new V.me service LIVE - lots of interest still after #qcon #qconsf #vme pic.twitter.com/b1cAMOll

amoratti Andrés Moratti: Visa is launching t.co/5AIzJPVm payment method #qconsf t.co/lVpmZoKs

rowan_h Rowan Hick: #qconsf seeing Ruby on Rails in a Visa presentation - never would have predicted that a few years ago

 

Max Protect: Scalability and Caching at ESPN.com by Sean Comerford

Twitter feedback on this session included:

iaflash I, Architect: #iaflash Pretty good slides on the #technical #architecture at #ESPN.com: qconsf.com/dl/qcon-sanfra… #qcon bit.ly/vUF7zC

Scaling Social Computing - Lessons Learned at Facebook by Robert Johnson

Rowan Hick attended this session:

Robert Johnson, at Facebook said something that resonated with me profoundly (paraphrased - didn't write the exact words!) "if we push to production every day, we get 365 chances to test, vs pushing every week we get only 52 chances a year to test". That's not to test things work (ie QA), it's to test that the theory of this new widget/tweak/whatever is actually going to be adopted by your users. For me, that encapsulates the 'Lean Startup' theory on the largest scale world wide, and there was a reason the audience was engrossed listening to Robert speak.

Steve Mercier attended this session and reported:

Lead in: Scaling is about dealing with failures

Facebook culture: Move fast

Problem domain: social data database (lots of distributed data!)

Context: Facebook usage has been continuously growing for the past 7 years, every single week... that gives really no rest to the development teams

Twitter feedback on this session included:

rowan_h: The FB#qconsf presentation got me all interested in Graph databases, and how they could define/allow new ideas in Product development...

pascal_collet Pascal Collet: "goal is to reduce cost of making mistakes not to avoid mistakes" R. Johnson #qconsf

cutshot Mike Fitterman: Handling failures is about trying to keep small problems from turning into big problems. #qconsf

rbarcia Roland Barcia: In Scaling Facebook session at #QConSF, number one architectural principal: "Move Fast". 2 week sprint iteration is too slow.

cutshot Mike Fitterman: Moving fast enables the ability to try and back out (try and fix or remove) #qconsf

JBollinger Jeff Bollinger: Facebook: Use percentiles in monitoring not just averages. Averages often hide problems. #qcon

 

SimpleGeo: Staying Agile at Scale by Mike Malone

Jean Lambert summarized this session:

Failure is a powerful tool. People learn from failure but unfortunately, the first reflex most of use have is trying to prevent failure. When you think about it, we should be spending more time becoming resilient to failure instead of preventing it; This would be a much more valuable investment of our time.

  • Push code all the time - It will get less painful
  • Upgrade all the time - It will get less painful
  • Write tests all the time - It will get less painful
  • Do code reviews all the time - It will get less painful
  • Merge branches all the time - It will get less painful
  • Fail all the time, it will get less painful

So go ahead, learn from failure and enable change! Just don't forget to make sure people aren't stigmatized or afraid to fail.

Twitter feedback on this session included:

marianocortesi Mariano Cortesi: Beauty is the ultimate defense against complexity @mjmalone #qconsf

JBollinger Jeff Bollinger: Deployments - Fail fast and loud so people can quickly see casual relationships between a change and failure. - @mjmalone #qcon

Architecture Case Studies: Cloud

Netflix in the Global Cloud by Adrian Cockcroft

Jean Lambert attended this session:

Simply put, if you want to do Cloud, don't try to build one, find the best possible infrastructure web services platform in the could out there and use it! …

Why use the public Cloud you ask? Well... Netflix just could build data centres fast enough and has decided to search for a robust and scalable Cloud offering to develop their platform on. The choice was set on Amazon Web Services (AWS). Partnering with AWS has allowed Netflix to be ~100% Cloud based! Some smaller back end bits are still not moved to the Cloud however, they are aggressively pushing to change that. The simple rational behind it all was that AWS has dedicated infrastructure support teams which Netflix did not, so the move was a no brainer.

Twitter feedback on this session included:

mtnygard: My favorite #qconsf talk was definitely @adrianco talking about the Netflix Global Platform. Great detail & experience report.

jrolstad: Still basking in the glow of "Netflix in the Global Cloud" at#qconsf. Good insights on how to use EC2 effectively and reliably.

neuhausler Marcel Neuhausler: #qconsf Interesting talk by Daniel Jacobson about the platform behind the Netflix API. I use the API for new movie 'recommendations' @ntflxd

beyondWeb Michael Floyd: #qconsf Netflix 30 billion requests per month. @InfoQ

JBollinger Jeff Bollinger: AWS not so infinite. Netflix often hits AWS usage limits which requires a manual process to contact Amazon to get more capacity. #qcon

JBollinger Jeff Bollinger: Netflix does connectivity and bandwidth checks at the client level (Wii, PS3, etc..) and selects among dozens of CDN locations. #qcon

JBollinger Jeff Bollinger: Netflix AWS automated scaling is built completely on real-time CPU data. Keeps average server CPU at 50%. #qcon

stephen_younge Stephen Younge: Could this be the most beautiful graph of #qcon? Auto Scaling at Netflix. t.co/fPMis8bN

On Track For The Cloud: Building A Hybrid Cloud At Canadian Pacific by Stuart Charlton

Twitter feedback on this session included:

jinman: Excellent!!#qconsf presentation by@svrc Canadian Pacific Railway Success Story Dev/Test Solution rolled out in 4 months using#aws

Rearchitecting amazon.com for the Cloud by John Rauser

Twitter feedback on this session included:

charleshumble: In Amazon mean-time between production [to] deployments during the week is 11.6 seconds#qconsf

charleshumble: Love this idea from Amazona. Write press release first. If you can't boil it down to a page or so, maybe you shouldn't build it#qconsf

floydmarinescu: Amazon.com has been running exclusively off of AWS since Nov 2010. Awesome talk at#qconsf

JBollinger: Deployments during week days average every 11.6 seconds at Amazon. At peak over 1000 in one hour.#qconsf

mtnygard: Amazon: in May, mean # hosts receiving a deployment was 10,000.#qconsf

Beautiful Code

Beauty is in The Eye of the Beholder by Alex Papadimoulis

Twitter feedback on this session included:

dblockdotorg Daniel Doubrovkine: @apapadimoulis is saying that to prevent ugly code you need to shoot the cowboys and to step back the cleverness in our code #qconsf

How to Stop Writing Next Year's Unsustainable Piece of Code by Guilherme Silveira

Twitter feedback on this session included:

amartinsn Alexandre Martins: @guilhermecaelum did a really good presentation on writing code the right way. #qconsf pic.twitter.com/l2Encmjp

rundavidrun David Whittaker: @guilhermecaelum at #qconsf: "It's not about beauty, make yourself clear. Don't mix concise with clear."

rundavidrun David Whittaker: @guilhermecaelum at #qconsf: "Is it so hard to hit Enter?" Separate lines by control flow. #makeComplexityVisible

jrolstad Josh Rolstad: Beautiful Code: Concise = with the least amount of words, make your code as clear as possible. #qconsf

Software Naturalism - Embracing the Real Behind the Ideal by Michael Feathers

Twitter feedback on this session included:

janm67 Jan Mattsson: @mfeathers applies scientific method to software development and ties in with wabi-sabi. Nice. #qconsf

rundavidrun David Whittaker: @mfeathers at #qconsf : Plot # of commits vs complexity; high/high = runaway region - lots of bugs here but high ROI if refactored.

rundavidrun David Whittaker: @mfeathers at #qconSF : "Make metrics diagrams so simple that you don't need to be technical to see there's a problem."

unclebobmartin Uncle Bob Martin: @mfeathers #qcon (sigh) What do you call people who ship Big Balls of Mud? 99.9% are failures. 0.1% are temporarily millionaires.

mfeathers Michael Feathers: "They have name for people who ship big balls of mud: millionaires" - Brian Foote (@bigballofmud) #qcon

bigballofmud Brian Foote: Code Blindness, @mfeathers says, happens when you have no way to visualize anytihing about what is happening in the code. #qcon

bigballofmud Brian Foote: Anytime you measure something, and attach incentives to it, people will find a way to game the numbers. #qcon

 

Sufficient Design: Quality In Sync With Business Context by Joshua Kerievsky

Twitter feedback on this session included:

amartinsn Alexandre Martins: Really inspiring talk by @JoshuaKerievsky on simple design! Well done. #qconsf

jdonelan James Donelan: @joshuakerievsky talk on sufficient design and entrepreneurial engineers was the best yet today #qconsf

lorgio_jimenez Lorgio Jimenez: Programming is like mental warfare, areas of your code should be based on ROI #qconsf #SufficientDesign

Who Ever Said Code Was Supposed to be Pretty? by Brian Foote

Twitter feedback on this session included:

rundavidrun David Whittaker: @bigballofmud at #qconsf : "Big Bucket of Glue - the dominant architectural design of the 21st century." #unfortunatelyTrue

mtnygard Michael Nygard: Below average programmers create mud. So do above average programmers. #qconsf

mtnygard Michael Nygard: "Small acts of negligence can really add up." @bigballofmud #qconsf

Big Data and NoSQL

NetApp Case Study byScott Fleming & Kumar Palaniappan

Steve Mercier extensively covered this session:

Agenda

  • NetApp big data challenge
  • Technology assessment
  • Drivers

AutoSupport Family

  • catch issues before they become critical
  • secure automated call-home service
  • system monitoring and non intrusive alerting
  • RMA request without customer action
  • enables faster incident management

Takeaways

  • NetApp assessed multiple traditional DB technologies to solve its Big data problem and determine Hadoop was the best fit.
  • HW & SW short and long term costs were key in the decision to move to Hadoop on ESeries

Cross Platform Mobile Development

Mobile JavaScript Framework Bake Off! byRoland Barcia & John Bender & David Kaneda

Twitter feedback on this session included:

multiplegeorges georges: js mobile framework bake off was great this morning. went into overtime! #qconsf

Functional Web

Exploring Composition and Functional Systems in the Cloud by David Pollak

Twitter feedback on this session included:

mtnygard Michael Nygard: End user programming in Visicalc, Excel, Hypercard, etc. all got squashed when the web came along and re-empowered IT. @dpp #qconsf

mtnygard Michael Nygard: Sudden shift from @dpp: functional, different approach to cloud computing, and iPad. Being announced here. #qconsf

Webmachine: A Practical Executable Model Of Http by Steve Vinoski

Daniel Doubrovkine attended this session:

I really enjoyed @stevevinoski’s the talk on Webmachine. I’ve never written anything in Erlang. At first I sawWebmachine as a silly framework for declarative RESTful programming. But as the talked got into more complex issues a light bulb lit in my head and I think I totally got it. It’s unlikely that I’ll ever get to use this framework, but I went from being a skeptic to actually learning something.

Twitter feedback on this session included:

argv0 Andy Gross: At @stevevinoski's webmachine talk - the only piece of software cooler than Riak #qconsf pic.twitter.com/3JNDVfzy

HTML5 & JavaScript, State of the Art

HTML5 Design and Development Tooling by Christophe Coenraets & Greg Wilson

Steve Mercier attended this session:

Agenda

2 problems with what was demoed so far

  • multiple IDEs/tools for multiple target platforms (Xcode for IOS, Eclipse for Android, windows, etc.)
  • debugging remote html applications when it runs on a device
for the first problem, phonegap offers a a builder service in the cloud on EC2 (seebuild.phonegap.com) - all connectable to a Git repository

for the later problem, they developed the Web Inspector Remote tool (called also weinre), similar to web application debuggers such as the one in Chrome, Firebug, etc.)…

Conclusion

Kind of amazing how easy it was to develop animated web content applications and to deliver them to multiple devices!

Realtime Web Apps with HTML5 WebSocket and Beyond by Guillermo Rauch

Twitter feedback on this session included:

ligolnik: Socket.IO is impressive. Now I want it without node.js on the backend.#QconSF

New Languages Changing Your World!

Dart in Depth by Gilad Bracha

Jean Lambert attended this session:

So, what is Dart?

  • A simple and unsurprising OO programming language
  • Class-based single inheritance with interfaces
  • Has optional static types
  • Has real lexical scoping
  • It is single-threaded
  • Has a familiar syntax

Twitter feedback on this session included:

amacarth: Resurrected some Smalltalk, mixed in with Erlang like actors and optional typing to make something very cool: Dart.#qconsf#dart

ChrisHirsch: I think I will be keeping an eye on Dart#qconsf

amacarth: Wondering if DartVM will be part of the Chrome platform?#qconsf

charleshumble: #qconsf Dart has optional types "In other programming languages, a conventional type-checker is a lobotomized theorem prover"

jrolstad: Interesting point about Dart: types are only there for documentation. Your app will still run with mismatched (or no) types.#qconsf

Software Architecture Improvements

Refactoring to Patterns by Joshua Kerievsky

Twitter feedback on this session included:

stephen_younge: @JoshuaKerievsky (3rd from left) illustrates refactoring techniques using human objects at#qconsf.t.co/oRfNbarF

rundavidrun: @joshuakerievsky at#qconsf : to remove a method, delete the body and then refractor inline. Boom! All instances are deleted.

Software Quality – You Know It When You See It by Erik Doernenburg

Jean Lambert reported from this session:

Eric first posed the following question: "How can you see software quality or non-quality?"

Well, as it turns out, there are many ways this can be done. Simply put, the steps are fairly simple and straightforward:
  1. Collect metrics
  2. Aggregate data
  3. Render graphics

Twitter feedback on this session included:

timotta Tiago Albineli Motta: The tools that @erikdoe is showing are making me think about lot of bad examples of code #qconsf

Systems That Never Stop

And It All Went Horribly Wrong: Debugging Production Systems by Bryan Cantrill

Steve Mercier reported on this session:

Do we still need to care about failure?

single component failures still has significant costs (both economic and run-time)... but most dangerously, a single component failure puts the global system in a more vulnerable mode where further failures is more likely to happen... This is a cascading failure - and this is what induces failures in mature and reliable systems. …

Conclusion:

these challenges are real but surmountable - as in some open source systems presented below (MDB, node.js, DTrace, etc.)

Postmortem debugging: MDB

  • a debugger in illumos OS (solaris derivative)
  • extensible with custom debugger module
  • well advanced for native code but much less for dynamic environments such as Java, Python, Ruby, JS, Erlang...
  • if components going into infrastructures are developed using those languages it is critical that they support postmortem debugging.

Postmortem debugging: node.js

  • not really interesting for non JAVA...
  • debugging a dynamic environment requires a high degree of VM specificity in the debugger...
  • see all details on dtrace.org/.../nodejs-v8-postmortem-debugging

Debugging transient component failure

  • fatal failures, despite its violence, can be root-caused from a single failure
  • non fatal failure, it is more difficult to compensate for and debug

Twitter feedback on this session included:

rundavidrun David Whittaker: @bcantrill at #qconsf : most failures unfort don't cause a full stop, instead they're transient failures that slowly wreak havoc over time.

Cloud-Powered Continuous Integration and Deployment by Jinesh Varia

Steve Mercier reported from this session:

Poka yo-ke technique

See here: A poka-yoke is any mechanism in a lean manufacturing process that helps an equipment operator avoid (yokeru) mistakes (poka). Its purpose is to eliminate product defects by preventing, correcting, or drawing attention to human errors as they occur.

Ex: ATM machines with card swipers designed so you keep you card at hand instead of those that "eat" your card offering you a way to possibly forget to take it back when you leave the ATM.

Basic lessons

  1. Keep absolutely everything in version control (scripts, etc.)
  2. Commit early, commit often
  3. Always checkin to trunk, avoid branching
  4. take responsibility for check ins breaking the build
  5. automate the build, test, deploy process
  6. be prepared to stop the mainline when breaking occurs
  7. only one way to deploy and everybody uses the same way
  8. be prepared to revert to previous versions

Cloud

VCS systems in the cloud, CI servers in the cloud, distributed build in the cloud

They use Jenkins, Team city, cruise control, bamboo

Twitter feedback on this session included:

mtnygard Michael Nygard: Recommendation from @jinman: put CloudFormation template in version control, create whole stacks same way for test & prod. #qconsf

mtnygard Michael Nygard: But apparently they've got one of every CI tools out there. #qconsf

lorgio_jimenez Lorgio Jimenez: facebook doesn't do development branches. They dark launch everything. Everyday is a new experiment. #qconsf #continuosdeployment

Reliability Engineering Matters, Except When It Doesn't by Michael T. Nygard

Steve Mercier attended this session and reported:

3 Types of failures

  1. Independent failure: Failure of one unit does not make another unit more likely to fail (excellent! but is that true for software systems? not likely)
  2. Correlated failure: Failure of one unit makes another unit more likely to fail.
  3. Common mode failure: Something else, external to the measured system, makes 2 redundant units likely to fail (ex: redundant LEDs in an Apollo capsule that were both subject to overheating)

Important Note for RE when applied to software systems:

Lots of software reliability analysis make the error of assuming a perfect independence between duplicated resources. The 2 last types of failures are way more common.

Twitter feedback on this session included:

bigballofmud Brian Foote: @mtnygard points out that 50%-65% of outages are causes by human error. #qconsf

Uptime in High Volume Messaging Systems — Lessons Learned by Erik Onnen

Steve Mercier attended this session:

Architecture - General principles

  • Keep everyone moving in the same direction
  • Help discrete teams understand how they interact
  • Think in terms of small discrete services
  • Continuous capacity planning based on real data
  • Avoid local optimization decision making

Engineering for automation

3 level of testing (Unit, Functional (with mocks)and Integration). Commits are done to a single main git branch

 


Engineering for simplicity

They simplified metrics and stats capture and so they can do it everywhere!

They capture latency for Service critical operations and External service invocation

They capture counters for Service critical operations and Services faults

Engineering for Continuous Improvement

They use the 5 whys approach

Operations at UA

A team of 5 operations engineers handling more than 100 servers (Mostly bare metal, using EC2 for surge capacity)

Operations for Transparency

They measure absolutely everything, and monitor only the important things

Web APIs

Building a Great Web API by Evan Cooke

Daniel Doubrovkine attended this session:

I watched @emcooke's excellent Web API talk next. Evan came to GA a few weeks ago and gave a short version of the talk from a couch to a few techies. This is exactly the kind of talks I wanted to see more of. The guy had his hands in the code. He succeeded at building an API that people loved. He had priceless advice on each and every slide.

Techniques for Scaling the Netflix API in the Cloud by Daniel Jacobson

Daniel Doubrovkine attended this session:

@daniel_jacobson from Netflix described the recent evolution of the Netflix API. This was mostly about the internal API that thousands of devices use. They chose to create an API variation for every device a longtime ago and are now bringing scripting-like functionality to a middle-tier, server-side. Think of it as a server-side proxy for your API where you execute sequences of operations that are device-specific. Instead of performing chatty operations between a client and a server, you compile and execute them on the proxy.

Why is Java Still Sexy

JVM Performance Optimizations At Twitter's Scale by Attila Szegedi

Steve Mercier summarized this session:

Takeaway: Twitter went from 60 second of Garbage Collection (GC) per hour to 5 seconds per 3.5 days, mainly because of the internal slab allocator….

Twitter's biggest enemy: LatencyWhat are the main latency contributors (from worst to less):

  1. Garbage collector (GC)
  2. in process locking and thread scheduling
  3. I/O
  4. application inefficient algorithms but only the first one can be addressed by JVM tuning (which happens to be the expertise of the speaker)

Profile your applications, especially when 3rd parties are involved abd not well known (ex: primitive SCALA array types eating lots of memory; another ex: Guava's concurrent map eating a lot of resource - especially if you do not need concurrency)….

Writing a new memory manager is not a good idea unless you already optimized your data and your JVM... if you start doing this, it is a strong smell that you are doing something wrong!

Cloud and NoSQL Solutions

Riak Use Cases: Dissecting the Solutions to Hard Problems by Andy Gross

Twitter feedback on this session included:

jrolstad: Got sidetracked tonight with@basho's Riak database. Very enticing, want to see how it actually works for me.#qconsf

Stratos - an Open Source Cloud Platform by Paul Fremantle

Twitter feedback on this session included:

jrolstad Josh Rolstad: The Stratos presentation has good insights into how a PaaS product is built. So many things I take for granted that are hard to do. #qconsf

Performance and Scalability Solutions

Java EE 7: Developing for the Cloud by Tyler Jewell

Twitter feedback on this session included:

rbarcia Roland Barcia: At #qconsf, Java EE v7 theme is Cloud. "Cloud Services, Not Just API's." Standard REST client API, etc...

Performance and Scalability Panel byHans Jespersen & Tyler Jewell & Michael Kopp & Tom Lubinski & Floyd Marinescu &Dan Pritchett

Twitter feedback on this session included:

floydmarinescu floydmarinescu: Cloud is good for scalability because it forces you to design upfront with good patterns #qconsf performance panel

Solution Track Friday

Automating (almost) Everything using Git, Gerrit, Hudson and Mylyn by Ryan Slobojan

Steve Mercier extensively covered this session;

A presentation on how good open source tools can be integrated together to push development efficiency up….

Setting up the server (in that order)

  • Install all the softwares and then:
  • Clone GIT repository
  • Initialize Gerrit
  • Register and configure a Gerrit human user
  • Register and configure a Build user (the automating user)
  • Install Hudkins Git and Gerrit plugins
  • Configure standard Hudkins build
  • Configure Hudkins Gerrit build (integrate the review within the build)

Setting up the client (contributor - in that order)

  • Install EGit, Mylyn Builds (Hudson), and Mylyn Reviews (Gerrit)
  • Clone GIT repository (pull)
  • Configure Gerrit as remote master
  • There is no step 4! Done.

Setting up the client (committer - in that order)

  • Configure a Mylyn builds Hudson repository
  • Configure a Mylyn review Gerrit repository
  • Create a task query for Gerrit
  • There is no step 4! Done.

Building an Ecosystem for Hackers byRich Manalang

Steve Mercier attended this session:

The software life-cycle

It goes from ideas, to prototype, to code, to shipping the product and rinse and repeat until your product is bloated and the code begins to stink!…

How can we empower users

  1. Build opinionated software (if you do not like it, don't buy it - but you are thus building a fence around your product)
  2. Build a platform they can hack on (you allow you customers to get involved)…

People want to contribute to your product, but you need to give the ways to do it!

Integrating SaaS Applications with Mule iON by Ken Yagen

Steve Mercier attended this session:

The speaker wen through a discussion on APIs explosion, popular ones, the ones he thinks we should be considering in cloud application and how MuleiON can be used to properly decouple an application from all the cool cloud APIs discussed and presented….

Top 10 APIs by usage

  1. Google Maps
  2. Twitter
  3. YouTube
  4. Flicker
  5. Amazon eCommerce
  6. Facebook
  7. Twilio - Telephony service
  8. eBay
  9. Last.fm - Radio service
  10. Google search

iPaaS: Do not clutter your app

Use MuleiON (integration PaaS) for loose coupling your app to this cool stuff

Your killer app --> integration layer (ex: MuliON) --> cool stuff (Twitter, etc. all that we just listed above)

Benefits

  • Configuration approach to APIs
  • Handled for you (security, session management, streaming, http callbacks)
  • consume and invoke multiple services, retries, error handling

Essentially, decouples and keeps your application code clean

Social Events

Twitter feedback on this session included:

charleshumble: Mariachi band after the@bmf party keynote bit.ly/vAriv4#qconsf

vmedev: Great turnout at the Thirsty Bear celebrating V.me by Visa announce. Thanks devs!#qconsf#qcon#vmepic.twitter.com/neDLjbRS

signde: Visa v.me#qconsf party at thirsty bear is really nice. Free drinks and food. They even have paella! Great beers here.

floydmarinescu: Check out last night's Mariachi band after the @bmf party keynote bit.ly/vAriv4

@bmf: Loved the mariachi band march to the Metreon! t.co/htem6uM3 #qcon #qconsf

suziedwards Suzi Edwards: Time for drinks at #qconsf. Feel free to bring a bucket of white wine to me at the #thoughtworks stand. And I'll enter you into our draw!

Opinions about QCon

Daniel Doubrovkine joined QCon SF this year:

What I loved about this conference is that it was mostly technical in nature and that the best speakers were hands-on industry practitioners. I also greatly enjoyed the fact that most attendees were architect-level engineers, which made for well balanced conversations over the frequently supplied drinks.

Steve Mercier shared his impression after the first day of the conference:

Ok, day 1 of the conference is done. Now, what can be said about the experience up until now? Well, for one, I think QCon SF is awesome. The organization of such an event can be difficult as there are many people, many speakers, lots of coordination and a bunch of areas where slip-ups can occur but Trifork and InfoQ have got everything under control. The venue layout minimizes walking distance between presentations and the hotel staff is very efficient and courteous, unlike the cable car operators. So, organization-wise, I have no complaints; We are in very good hands.

Jean Lambert shared his impression after the conference:

The past three days have been quite intense and I'm on the verge of a total information overload. All that aside, the QCon San Francisco International Software Development Conference was in my eyes a great success.

Twitter opinions on QCon SF 2011 included:

jamesy829: The QCon San Francisco Conference was a blast! Time to try new stuff!#qconsf

jrolstad: Relaxing at SFO after #qconsf. Was an intense 3 days - kept us busy from 8am until late at night w/ conference and vendor parties!

argv0: #qconsf winding down. great to see old friends and meet new people. thanks @trifork, @infoq, and @basho !

ddossot:My gutfeel is that #qconsf is a refactoring away to go from good to great. Let's come back next year and see how great it'll be :)

marienborg Andreas Marienborg: And it's a wrap! My first #qconsf ever, and I loved it :)

@adrianco: Biggest problem with#qconsf is wanting to be in more than one talk at the same time. I missed some good stuff today :-(

stephen_younge: Sad to leave#qconsf early but it was probably good that I got out of there before my brain exploded.

Takeaways

Jean Lambert’s takeaways was:

Me and my work colleague came here on an information seeking mission and I can now say with great confidence that we have achieved our goal. In fact, we might have collected enough data to sift through for the next couple of weeks which will then package that in a form that is consumable in our work environment. The discussions and idea exchange will definitely continue for us but at a more local level.

Twitter feedback on QCon SF 2011 included:

stephen_younge: Back in Boulder and already experiencing#qconsf withdrawal - but at least I'm finally beginning to assimilate some of the stuff I learned.

stephen_younge: Day Two of#qconsf was awesome and it has me rethinking some#Agile and#Craftsmanship assumptions. Amazing@vmedev party last night!

emcooke: Very impressed with my first#qconsf this year. It was a hybrid of an academic and 'tech' conference with meaty talks

frubim: Flying back home soon. Exciting time at#qconsf 2011. Great discussions and sessions. Can't wait for London and New York next year!!

vmedev V.me Developer: Day 2 of #qcon about 2 wrap up. hard to tweet when I'm so engrossed in the sessions.Sid from Netflix session was interesting #vme #qconsf

platypusguy Andrew Binstock: At excellent #QConSF conference today. Heard @mfeathers @dpp @hlship and @stevevinoski among others. Good line up for one day!

ChrisHirsch ChrisHirsch: Alrighty! Back from #QCon and ready to change (or guide heh) the world at work ;-)

Conclusion

The success of each QCon lies in the fact that each is created virtually from scratch by the small team of experts that make up the program committee. This year’s committee members include Floyd Marinescu, Kresten Krab Thorup, Ryan Slobojan and Aino Vonge Corry. The committee ensures that new tracks will reflect current trends in enterprise software development, and that sessions are both educational and practical. Speakers are handpicked by track hosts who themselves are practitioners. Attendance is limited so that the experience is intimate, allowing attendees to mingle with highly respected speakers, community influencers and thought leaders. The result is a unique conference that is given by practitioners for practitioners.

QCon is co-produced by Trifork (GOTO and other Conferences) and InfoQ’s parent company, C4Media. QCon conferences include QCon London, QCon San Francisco, QCon Beijing, QCon Tokyo, QCon Sao Paolo, and our recently announced QCon New York conference.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT