New-age Transactional Systems - Not Your Grandpa's OLTP
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
The content has been bookmarked!
There was an error bookmarking this content! Please retry.
Posted by Obie Fernandez on Feb 28, 2007
Agility at scale, become as agile as you can be
Agile Development: A Manager's Roadmap for Success
SOA All-In-One Guide: KPIs & Best Practices, ESB Report
SCM best practices for multiple processes, releases & distributed teams
Thanks for a great article Andrew! ActiveMessaging really rocks (as does Rails, Stomp and ActiveMQ :). Keep up the good work!
James
LogicBlaze
Open Source SOA
Excellent article. Are there any benchmarks/performance figures yet?
good question - the straight answer is I haven't seen or or run any such test myself. If anyone else has, I would be very curious to hear about it.
So I don't know exactly what the a13g overhead is, or how it scales. I have a feeling this may depend much more on your message broker, and then what you do in the message processor. These kind of tests are a bit tricky as you are testing the network, the broker, the box, etc.
That said, it would be good to design a benchmark, perhaps based off the work done at ActiveMQ.
I can say that anecdotally, I find the processing of the message to take longer than the overhead in dispatch - we even do synchronous messages from our java app to our rails app and back.
One other nice thing is that you can run multiple instances of the poller process, so with N number of pollers all listening to the same queue, you in theory have as much scalability as you have processing power, and depending on the efficiency of the broker dispatch.
Cheers,
Andrew Kuklewicz
This work is awesome, and all we need is a way to push these messages to the browser, so that browser doesn't poll for it. any ideas on this?
I only know one non-polling based way to push to the browser, and that is the Juggernaut project.
Personally I am very tempted to mess with the juggernaut juggernaut.rubyforge.org/ solution, I just haven't had an excuse.
From what I understand it involves a separate "push server" that sits between a rails server app, and the flash apps on the browser. I see no reason a processor in a13g couldn't send a message to the juggernaut push server that would then get pushed up to the browser - if you give it a try let me know - it looks awful cool.
This work is awesome, and all we need is a way to push these messages to the browser, so that browser doesn't poll for it. any ideas on this?
We've had that part solved for a while using a cometd style integration using ActiveMQ and Jetty...
activemq.apache.org/ajax.html
In benchmarks for customers we've had a single process handling 14,000 concurrent users
James
LogicBlaze
Open Source SOA
Well Andrew, I have checked out Juggernaut inside out and actually have implemented by own push server.But the problem is, it can't bypass firewalls and worst of it NATs. This is a big hurdle in my opinion. Without solving above issues Juggernaut won't get mass adoptation.
Are we talking Java?
Would you please take a look at the table messages, I have got
mysql> select * from messages;
+----+--------------+---------------------+
| id | body | received_date |
+----+--------------+---------------------+
| 1 | Hello World | 0000-00-00 00:00:00 |
| 2 | testDateTime | 0000-00-00 00:00:00 |
+----+--------------+---------------------+
FIELD received_date, it supposed to be DateTime at which message were generated.
Please take a look at on_message
James, is that "cometd style" or api/protocol compliant cometd?
Yeah, there is a one letter typo, I sent in a fix for this before the article was published, but guess it didn't make it into the article.
def on_message(message)
logger.debug "PersistMessageProcessor received: " + message
my_message = Message.create(:body=>message, :received_date=>DateTime.new)
end
should be :received_date=>DateTime.now
not :received_date=>DateTime.new
Cheers,
-Andrew Kuklewicz
perfect, a toast !
Hello all,
I followed along with the example, however when posting the message from the view, the server outputs the following message (infinitely, it would appear):
transmit failed: undefined method `length' for #<Message:0x45f89b8>
transmit failed: undefined method `length' for #<Message:0x45f89b8>
transmit failed: undefined method `length' for #<Message:0x45f89b8>
...
My poller runs fine:
Loading D:/dev/ruby/Mess/app/processors/application.rb
Loading D:/dev/ruby/Mess/app/processors/persist_message_processor.rb
=> Subscribing to /queue/PersistMessage (processed by
PersistMessageProcessor)
I don't see a message in either the poller console, or the activemq console.
I changed the broker.yml configuration to look at denis:61613 instead of localhost because when activemq starts, it states:
...
INFO TransportServerThreadSupport - Listening for connections at: tcp://denis:61616
INFO TransportConnector - Connector openwire Started
INFO TransportServerThreadSupport - Listening for connections at: ssl://denis:61617
INFO TransportConnector - Connector ssl Started
INFO TransportServerThreadSupport - Listening for connections at: stomp://denis:61613
INFO TransportConnector - Connector stomp Started
INFO TransportServerThreadSupport - Listening for connections at: xmpp://denis:61222
INFO TransportConnector - Connector xmpp Started
...
Any ideas?
Thx,
dl
er, nevermind I overlooked 1 line in my controller, rather than
def new
@message = params[:message]
...
end
was
def new
@message = Message.new(params[:message])
...
end
For pushing message in COMET style, you may use AjaxMessaging code.google.com/p/ajaxmessaging/
Hemant:
Not true, Juggernaut can use port 443 which is open on most firewall (used for https).
Juggernaut can bypass firewalls in that it can use ports commonly open on a firewall, such as 443. I'm not sure how NAT poses a problem, the port just gets forwarded to a internal server.
I've noticed that when using the poller, there are occasional consumers lingering around which AMQ sends the msg, and hence dropped, any idea if I'm not using the poller correctly? I have a run_poller script that does:
script/poller start
sleep 30
script/poller stop
Hi Author of this Article,
I have followed all your steps to config to code.
But when i am running "script/poller run"
I am not getting desirable log in console.
Only getting as :
^C^[[Aajay-dhandes-imac:MessageMonster mrunalini$ script/poller run
/Users/mrunalini/.gem/ruby/1.8/gems/stomp-1.1.3/lib/stomp/connection.rb:156: warning: parenthesize argument(s) for future version
ActiveMessaging: adapter wmq not loaded: no such file to load -- wmq/wmq
"/Users/mrunalini/code/MessageMonster"
"/Users/mrunalini/code/MessageMonster/config/messaging.rb"
ActiveMessaging: Loading /Users/mrunalini/code/MessageMonster/app/processors/application.rb
ActiveMessaging: Loading /Users/mrunalini/code/MessageMonster/app/processors/persist_message_processor.rb
Missing the Rails 2.3.3 gem. Please `gem install -v=2.3.3 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
My development environment is Mac OSX Leapord.
Please post suitable solutions for me.
Thanks,
Ajay nayak
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.
Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.
Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).
Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
19 comments
Watch Thread Reply