InfoQ

InfoQ

Presentation

My Bookmarks

Login or Register to enable bookmarks for unlimited time.

The content has been bookmarked!

There was an error bookmarking this content! Please retry.

Recorded at:
Recorded at

Evolving the Key/Value Programming Model to a Higher Level

Presented by Billy Newport on Jan 19, 2010 Length 00:59:38
Sections
Architecture & Design,
Development,
Operations & Infrastructure
Topics
Clustering & Caching ,
Data Access ,
Architecture ,
Java
Tags
Groovy ,
QCon San Francisco 2009 ,
QCon ,
WebSphere eXtreme Scale ,
Redis ,
Memcached ,
Clojure ,
Coherence
The next QCon is in London March 5-9, Join us!
 

How would you like to view the presentation?

In case you are having issues watching this video, please follow these simple steps to help us investigate the issue:
1. Right click on the video player and select Copy log
2. Paste the copied information in an email to video-issue@infoq.com (clicking this link will fill in the default details in most email clients).
Note: in case your email client hasn't automatically picked up the email subject, please include in your email the URL of the video too.
3. Done.
We will investigate the issue and get back to you as soon as possible. Thanks for helping us improve our site!
Summary
In this presentation from QCon San Francisco 2009, Billy Newport discusses the ways that developers interact with key/value (KV) stores such as memcached and WebSphere eXtreme Scale, entity vs column-oriented approaches, synchronous and asynchronous operations, large data sets, using a DBMS as a column store, collocating closures and data, and features that could be added to increase scalability.

Bio
Billy Newport is a Distinguished Engineer working on WebSphere eXtreme Scale (ObjectGrid) and on WebSphere high availability, and has worked at IBM since Sept 2001. Besides his current activities, he helped add advanced APIs like the WorkManager APIs (JSR 236/237) and worked on the staff plugin architecture of WPS.

About the conference
QCon is a conference that is organized by the community, for the community.The result is a high quality conference experience where a tremendous amount of attention and investment has gone into having the best content on the most important topics presented by the leaders in our community. QCon is designed with the technical depth and enterprise focus of interest to technical team leads, architects, and project managers.
Uninformed IBM by Emil Vladev Posted
Re: Uninformed IBM by Emil Vladev Posted
Source code is online now by Billy Newport Posted
Only works for twitter-liked apps by Steve Tirtha Posted
Re: Only works for twitter-liked apps by Billy Newport Posted
Using Closures by Aayush Puri Posted
  1. Back to top

    Uninformed IBM

    by Emil Vladev

    I'm sorry to correct Billy here, but he misses a point in just 2 minutes of his presentation. Redis does not use thrift and never has. Also, it supports an append-only log, so no data is lost (this was probably added after the presentation).

  2. Back to top

    Re: Uninformed IBM

    by Emil Vladev

    Oh, forgot to mention that Redis DOES support replication: code.google.com/p/redis/wiki/ReplicationHowto

  3. Back to top

    Source code is online now

    by Billy Newport

    The source code for the prototype implementation is online now at github. www.github.com/bnewport

  4. Back to top

    Only works for twitter-liked apps

    by Steve Tirtha

    Hi,

    I think this kind of approach using key-value without transaction and locking only works for twitter-liked apps, where users do not accessing and updating one information in the same time.
    Even in twitter-liked apps, user do not change data from other user.
    How can this approach can be applied to business app like CRM, where 100-1000 user access and update information in the same time?

    Regards,
    Steve

  5. Back to top

    Re: Only works for twitter-liked apps

    by Billy Newport

    This approach implements atomic operations on the supported primitives like numbers, lists or sets but without transactions then you can't atomically work with multiple entries. If you want that kind of control then just using WebSphere eXtreme Scale directly with our normal APIs gives you that kind of model. Full atomic transactions, locking, serialization of changes and so on.

  6. Back to top

    Using Closures

    by Aayush Puri

    I actually found the proposal to use closures to run code on different nodes pretty neat. Looking forward to the prototype!

Educational Content

Collaboration: At the Extremities of Extreme

Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.

Yesod Web Framework

Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).

Transactions without Transactions

Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.

Attila Szegedi on JVM and GC Performance Tuning at Twitter

Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.

10 tips on how to prevent business value risk

One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.

Interview: Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives

InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.

Beauty Is in the Eye of the Beholder

Alex Papadimoulis discusses ugly code, where it comes from, how to avoid it, and how to get rid of it.

Architecting Visa for Massive Scale and Continuous Innovation

John Davies examines Visa’s architecture and shows how enterprises have architected complex integrations incorporating Hadoop, memcached, Ruby on Rails, and others to deliver innovative solutions.