New-age Transactional Systems - Not Your Grandpa's OLTP
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
The content has been bookmarked!
There was an error bookmarking this content! Please retry.

Posted by Fernand Galiana on May 05, 2007
The trend for many companies these days is to claim their entire devotion to the Agile process. We all iterate, sit or stand in SCRUM meeting often for longer than we care for and we are all sworn into delivering early and often... But how can you really measure if your application is a success or a bust? We all know that page hits and heat maps are only good in the eyes of marketeers, or some usability PHD... How do you accurately assess that the "must have" features in your application are actually being used and how are your customers actually using them ? More importantly, how can you accurately assess you are iterating down the right path for your product and company success, and not simply digging further in a rat hole ?
In today’s hyper-competitive world, later may be too late to adopt Agile development and this Roadmap for Success will help you get started. Download "Agile Development: A Manager's Roadmap for Success" now!
These very questions motivated the conception of the MOle. I was recently getting close to the delivery stage for an application. The features were abundant, the code tested and the bitchin prod machines roared. We had several bells and whistles that we all slaved long hours to implement based on our product requirements. One of the features was an nice Ajax based slider that allowed the users to filter out content based on some ranking algorithm. Then we began to wonder: 'Will customers use the slider ? If so, what settings will they use?" Sure a few thousand dollars worth of usability lab equipment and countless hours of drilling the poor souls that volunteered, would shed some light on this mystery. But who can stomach the price and time drain? So, short of babysitting, the logs and attempt to munge the one out of thousand actions scrolling by, we needed a way to intercept that particular feature and log the event, so that we could establish metrics and reporting around this valued feature. We needed a mole, an agent that would do just that in our application (i.e. intercept features and record these precious events). The MOle allowed us to gather the valuable user inputs. It provided us the necessary feedback to make the adjustments for our next iterations. As it turns out some of the features that were qualified as "must have" in our requirement documents were not being used at all. Worth noting as well, the MOle proved valuable in our test cycles to assess the coverage and efficiency of our QA team...
Now that we have hopefully whetted your appetite, here are the steps necessary to MOle your own application.
ruby script/plugin install svn://rubyforge.org/var/svn/liquidrail/plugins/mole/trunkUpon install, the plugin will create 2 tables in your db namely mole_features and mole_logs
MOLE = File.new( "#{RAILS_ROOT}/config/mole.conf" )
MOLEABLE_APP = true
MOLE_PERF_MAX_TIME = 5
All the interactions of your moled application are orchestrated via the mole.conf file. A sample mole.conf file has been dropped in the config dir during the plugin installation. You can turn the mole on/off via the MOLEABLE_APP var. A built in feature of the mole is also to record long running actions. We will cover this later but the last variable will set the default performance threshold in this case 5 seconds.
loadMOLE.path ifFile.(MOLE.path) and MOLEABLE_APP
This will load the mole configuration that instruct what/how to mole your rails application. All mole interactions occur in one single file, no need to sprinkle mole code all over your controllers. In the configuration file you simply instruct the MOle where and what to capture when a particular interaction occurs.
MyController.before( :feature => :show ) do |context, feature, args|
Mole::DbMole.mole_it( context, feature, context.session[:user_id],
:some_var => context.instance_variable_get('@myVar'))
This intercept the show action on the MyController and before, show is called, will log the controller state namely 'myVar' to the database. So not only do you persist the mere fact that this action got called but also you will record the very essence of this call via the state used to make this particular call.
or...MyController.after( :feature => :show ) do |context, feature,ret_val, args|
Mole::DbMole.mole_it( context, feature, context.session[:user_id],
:some_var => context.instance_variable_get('@myVar'), :return => ret_val))
Same idea but intercept the show action after it was called...
We have also bundle a couple of consoles to allow you to view the mole in action: The Snitch and the Yahoo! Widget MOlet...

The Snitch source can be downloaded here:
svn://rubyforge.org/var/svn/liquidrail/plugins/mole/samples/consoles/snitch
Once you've downloaded The Snitch you will need to run the following command to complete the installation:
rake setup
This will tell the application how to access your user model by specifying the users db table name and which column to use to display the user name. Also you will need to edit the database.yml to point to your db. No worries this app does not alter your db in any way.
In order to run the companion widget application, you will need to install the Yahoo! Widget Engine for your platform and then download the MOlet Widget and save it to your platform specific widgets directory ( MAC Documents/widgets -- WIN Document And Setting/user/widgets)
svn://rubyforge.org/var/svn/liquidrail/plugins/mole/samples/consoles/widgets/molet
Thank you all for stopping by and taking a fresh look at the MOle. Whether you have solid requirements, or think it would be easier to find a pizza at a WeightWatchers convention, we hope you will take it out for a spin. We are looking forward to hear all about your 'moling' experiences and will do our best to help you getting it working correctly for you. If you have comments and suggestions on how to help us improve the MOle by all means fire away...
Fernand Galiana is the owner of Imhotep Software LLC a consulting and Rails training company located in Denver, Colorado. He possesses over 15 years experience in software development mainly around user interface and middle tier. He is also the author of ZiYa a rails charting plugin. He is currently hosting the Denver Rails User Group Chapter (aka DeRailed).
step 3 should be:load MOLE.path if File.exists?(MOLE.path) and MOLEABLE_APP
The snitch app fails out of the box with: No such file or directory - script/../config/../app/views/snitch/index.rhtml
I am looking into this ...
Good catch, thanks Sven.
The Snitch code should be back online...
Loving the MOLe for sure Fernand, thanks. It has proven to have many uses, including:
1) For support issues. We are able to set the Snitch to watch a particular user interact with the application. The helps us reproduce complex issues.
2) To support sales. The Snitch provides a great interface to our sales team as they determine how engaged customers are during the trial period.
3) Real-Time performance analysis. The Snitch allows us to create a custom live dashboard in order for us to pinpoint performance problems. And sometimes they just exist for a particular user. The MOLe is flexible enough to drill down.
Thanks Again, and hooray for Ziya as well.
Dave
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.
Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.
Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).
Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
3 comments
Watch Thread Reply