Introduction
Rails has already planted its flag firmly in the ground of database driven web applications. ActiveMessaging is a Rails plugin framework that extends the borders of Rails territory by adding simplified messaging integration. With ActiveMessaging and Rails now you can loosely integrate with systems as disparate as mainframes sending MQ messages or J2EE webapps, offload processing long-running tasks, or create applications with event or message-driven architectures. As others have put it, ActiveMessaging is trying to do for messaging what ActiveRecord does for databases. This article will introduce you to ActiveMessaging and related technologies, and get you started using it in your Rails applications now.
Messaging Standards and Brokers
Three things you need to get messaging working: a communication protocol, a broker speaking this protocol, and a client library for your applications speak the protocol. Technically, the fourth thing is a system to send messages to, but in this example we'll create a system that's a bit pathetic and talks to itself. For the protocol, ActiveMessaging supports Stomp: Streaming Text-Oriented Messaging Protocol. Stomp is a messaging standard to exchange text based messages over wire connections, usually tcp/ip. The idea was to make a protocol simple enough that clients and servers for it can be created easily in almost any language (e.g. the Ruby Stomp client is about 400 lines, soaking wet with comments and blank lines), so it should be possible to integrate almost any system.
The Apache ActiveMQ message broker supports Stomp natively, though other broker options are available, with more coming all the time. In fact, the recently announced StompConnect project makes any JMS broker into a Stomp broker, which opens up Stomp, and ActiveMessaging, to practically every open source and proprietary messaging product. On the client side, ActiveMessaging supports protocols using an adapter pattern, and for Stomp it extends the Stomp RubyGem to be that adapter. More adapters are possible and in the works, but Stomp is the only one supported today.
For this article, I'll be using ActiveMQ as the broker, and working with the as yet unreleased 4.2 branch. ActiveMQ has install instructions for all platforms, but here's a simple procedure that assumes you have Java 1.4 or later:
cd /usr/local/src
#unix and cygwin
wget http://people.apache.org/repo/m2-snapshot-repository/org/apache/activemq/apache-activemq/4.2-SNAPSHOT/apache-activemq-4.2-20070221.081507-10-src.tar.gz
#os x
curl -O http://people.apache.org/repo/m2-snapshot-repository/org/apache/activemq/apache-activemq/4.2-SNAPSHOT/apache-activemq-4.2-20070221.081507-10-src.tar.gz
cd ..
tar xvfz apache-activemq-4.2-20070221.081507-10-src.tar.gz
cd apache-activemq-4.2-incubator-SNAPSHOT
./bin/activemq
That's it! By default, ActiveMQ's Stomp support should be configured to start up automatically, but if not you should add the following line to conf/activemq.xml:
<transportConnectors>
...
<!-- Add this line -->
<transportConnector name="stomp" uri="stomp://localhost:61613"/>
</transportConnectors>
That's the minimum, to configure ActiveMQ to your heart's desire, check out the ActiveMessaging wiki page on ActiveMQ, and the using ActiveMQ guides.
Rails Setup
Assuming you have Rails 1.1+ and MySql already, you'll also need 2 more RubyGems: daemons and Stomp:
sudo gem install daemons
sudo gem install stomp
Let's start with a newly minted Rails 'MessageMonster', a database for it, and install the ActiveMessaging plugin:
cd /usr/local/dev
rails MessageMonster
mysqladmin create messagemonster_development -u root
cd MessageMonster
script/plugin install http://activemessaging.googlecode.com/svn/trunk/plugins/activemessaging
All the pre-requisites are in place, so it's time to write an application.
Processing Messages
We'll start by creating a 'processor', which in ActiveMessaging lingo is a class that subscribes to destinations in order to receive and act on messages. It is the message driven equivalent of your controllers, and along with your model where you'll put most of your code.
As Rails has taught you to expect, there is a generator to create processors. Let's create one that saves messages it receives to show ActiveRecord integration:
script/generate processor PersistMessage
Running this generate for the first time outputs the following:
create app/processors
create app/processors/persist_message_processor.rb
create config/messaging.rb
create config/broker.yml
create app/processors/application.rb
create script/poller
Let's talk about each of these a moment.
app/processors
is the directory for all the processors.app/processors/persist_message_processor.rb
is the new processor, with a default implementation for you to work from.app/processors/application.rb
is the common super class from which processors extend. Use this to put shared code, such as error handling.config/broker.yml
configures how you connect to your message broker in each environment. It's the equivalent ofdatabase.yml
, but for message brokers.config/messaging.rb
is used for the rest of the messaging configuration, like destinations and message headers.script/poller
is the daemon used by ActiveMessaging to listen for messages. More on this script later.
Let's check out the broker.yml
file first, to see the options and make sure the development broker configuration is right:
development:
adapter: stomp
login: ""
passcode: ""
host: localhost
port: 61613
reliable: true
reconnectDelay: 5
...
All of these values are exactly what we need for our default ActiveMQ installation, but for production you will likely have a user and password, and if you run your broker remotely, a different host and port.
So now let's look at the persist_message_processor.rb
:
class PersistMessageProcessor < ApplicationProcessor
subscribes_to :persist_message
def on_message(message)
logger.debug "PersistMessageProcessor received: " + message
end
end
By default you'll see it is subscribed to :persist_message
and has one method, on_message
, which will be invoked by ActiveMessaging when messages for the :persist_message
subscription arrive. The body of the message is passed in as the string argument message
, but if you need to get to the Stomp message object itself, it is available in the @message
processor attribute. Processors can subscribe to more than one queue, simply add anther call to subscribes_to
.
The :persist_message
symbol is a logical name for the destination to which this processor subscribes, the definition of this destination was generated in config/messaging.rb
:
ActiveMessaging::Gateway.define do |s|
s.queue :persist_message, '/queue/PersistMessage'
end
To use a comparison to Rails, messaging.rb
is most like routes.rb
. It allows you to map a logical name (:persist_message
) to a destination on the broker (/queue/PersistMessage
). Though there is much more that can be configured here, the generated default is fine for this example.
Now we'll teach the processor to save its messages for a rainy day, starting with creating a model for the messages.
script/generate model message
Now update 'db/migrate/001_create_messages.rb' with 2 additional columns:
create_table :messages do |t|
t.column :body, :text
t.column :received_date, :datetime
end
Call migrate to create the table...
rake db:migrate
...and on to updating the processor on_message
method to use the new Message
model:
def on_message(message)
logger.debug "PersistMessageProcessor received: " + message
my_message = Message.create(:body=>message, :received_date=>DateTime.new)
end
Now, you're looking at this and thinking that's nothing new - this is just how you would write the same thing in a Rails controller. That's the point, when you use ActiveMessaging, all the convenience of the Rails application environment is available to you.
With a processor to catch them, it's time to start throwing messages (bottles sold separately).
Sending Messages
To send messages we'll use a Rails view and controller in the same application:
script/generate controller SendMessage index
Edit the view to submit message text, and to show a refreshable list of previously persisted messages:
<p style="color: green"><%= flash[:notice] %></p> <h1>Send Message</h1> <%= start_form_tag :action => 'new' %> <label>Message</label> <%=text_field_tag :message, @message %> <%= submit_tag "submit" %> <%= end_form_tag %> <%= link_to "Refresh List", :action => "index" %> <table> <% @messages.each do |m| -%> <tr><td><%= m.received_date %></td><td><%= m.body %></td></tr> <% end %> </table>
Edit the controller to send the message. To make sending from any class easier, ActiveMessaging has a mixin MessageSender
that provides a simple publish method. The publishes_to
method declares to which destination a class intends to send, but does nothing more than validate that the destination is in config/messaging.rb
. Here is how the controller uses these methods:
require 'activemessaging/processor'
class SendMessageController < ApplicationController
include ActiveMessaging::MessageSender
publishes_to :persist_message
def index
@messages = Message.find :all
end
def new
@message = params[:message]
publish :persist_message, @message
flash[:notice] = "'#{@message}' sent"
@messages = Message.find :all
render :action => 'index'
end
end
And that, as they say, is that. The controller will send a message on submit, and when next you view or refresh the page you'll see the messages saved by the processor.
Running the Example
ActiveMessaging runs in its own process, controlled by the script/poller
, which can be a gotcha – just starting the Rails server is not enough if you want to receive messages, and messages are received into a different running process than Rails. To run this example, we need 3 processes: the ActiveMQ broker, the Rails server running to send the message, and the script/poller
to receive it, each running in their own terminal:
Start ActiveMQ:
cd /usr/local/apache-activemq-4.2-incubator-SNAPSHOT
./bin/activemq
Start Rails:
cd /usr/local/dev/MessageMonster
script/server
Run ActiveMessaging:
cd /usr/local/dev/MessageMonster
script/poller run
The script/poller
follows the usual daemon semantics. Besides 'run' which keeps the processing in the foreground, you can also pass it 'start','stop','restart' and 'status'. It also uses allows you to run multiple instances simultaneously, a useful for handling higher volumes of messages.
Now navigate to the test page (i.e. http://localhost:3000/send_message ) and enter a message such as the canonical "Hello World". When the message is sent you should see the following in the poller window:
Loading /usr/local/dev/MessageMonster/app/processors/application.rb
Loading /usr/local/dev/MessageMonster/app/processors/persist_message_processor.rb
=> Subscribing to /queue/PersistMessage (processed by PersistMessageProcessor)
=> All subscribed, now polling
PersistMessageProcessor received: Hello World
The poller
always shows what processors it loads on start-up, and what subscriptions requests on the broker. If the broker is unavailable or lost, it will try to reconnect based on your broker.yml
configuration. After the poller start output, you can now also see the ActiveMessaging processor has received the message sent from Rails. Likewise, if you use the 'Refresh List' button, you'll see the message in the Rails view once it is persisted by the ActiveMessaging processor.
Where to go from here
I hope you enjoyed this intro to ActiveMessaging, hopefully you can see both its potential and ease of use. ActiveMessaging has more advanced capabilities not covered in this article, such as improved exception handling, grouping processors to be run by multiple pollers, or how to use Stomp and ActiveMQ message headers for tricks like synchronous messaging, JMS integration, and selective subscriptions. If you are thinking after seeing the basics that ActiveMessaging is too limited for your needs, I think you might be pleasantly surprised.
Thanks for your attention, and happy messaging!
Andrew Kuklewicz has worked in software development for the last 10 years, and is senior web developer at PRX (the Public Radio Exchange), a nonprofit service for distribution, peer review, and licensing of radio pieces. When not working in public media, or tap dancing, he is the current maintainer of ActiveMessaging, a committer on the Ruby Stomp project, and frequent contributer to other Ruby on Rails, Plone, and Java open source projects.
Community comments
Great article!
by James Strachan,
Performance/Throughput?
by Stefan Tilkov,
Re: Performance/Throughput?
by Andrew Kuklewicz,
Now, we can only push the messages to the views
by hemant kumar,
Re: Now, we can only push the messages to the views
by Andrew Kuklewicz,
Re: Now, we can only push the messages to the views
by hemant kumar,
Re: Now, we can only push the messages to the views
by Alex MacCaw,
Re: Now, we can only push the messages to the views
by Alex MacCaw,
Re: Now, we can only push the messages to the views
by James Strachan,
Re: Now, we can only push the messages to the views
by hemant kumar,
Re: Now, we can only push the messages to the views
by James Cook,
Re: Now, we can only push the messages to the views
by Chong Francis,
Excellent!
by Lee Zieke,
Re: Excellent!
by Andrew Kuklewicz,
Re: Excellent!
by Lee Zieke,
Errors
by Denis Labelle,
Re: Errors
by Denis Labelle,
Re: Errors
by Ajay Nayak,
Ghost consumers lingering after script/poller stop?
by Brian Balser,
Great article!
by James Strachan,
Your message is awaiting moderation. Thank you for participating in the discussion.
Thanks for a great article Andrew! ActiveMessaging really rocks (as does Rails, Stomp and ActiveMQ :). Keep up the good work!
James
LogicBlaze
Open Source SOA
Performance/Throughput?
by Stefan Tilkov,
Your message is awaiting moderation. Thank you for participating in the discussion.
Excellent article. Are there any benchmarks/performance figures yet?
Re: Performance/Throughput?
by Andrew Kuklewicz,
Your message is awaiting moderation. Thank you for participating in the discussion.
good question - the straight answer is I haven't seen or or run any such test myself. If anyone else has, I would be very curious to hear about it.
So I don't know exactly what the a13g overhead is, or how it scales. I have a feeling this may depend much more on your message broker, and then what you do in the message processor. These kind of tests are a bit tricky as you are testing the network, the broker, the box, etc.
That said, it would be good to design a benchmark, perhaps based off the work done at ActiveMQ.
I can say that anecdotally, I find the processing of the message to take longer than the overhead in dispatch - we even do synchronous messages from our java app to our rails app and back.
One other nice thing is that you can run multiple instances of the poller process, so with N number of pollers all listening to the same queue, you in theory have as much scalability as you have processing power, and depending on the efficiency of the broker dispatch.
Cheers,
Andrew Kuklewicz
Now, we can only push the messages to the views
by hemant kumar,
Your message is awaiting moderation. Thank you for participating in the discussion.
This work is awesome, and all we need is a way to push these messages to the browser, so that browser doesn't poll for it. any ideas on this?
Re: Now, we can only push the messages to the views
by Andrew Kuklewicz,
Your message is awaiting moderation. Thank you for participating in the discussion.
I only know one non-polling based way to push to the browser, and that is the Juggernaut project.
Personally I am very tempted to mess with the juggernaut juggernaut.rubyforge.org/ solution, I just haven't had an excuse.
From what I understand it involves a separate "push server" that sits between a rails server app, and the flash apps on the browser. I see no reason a processor in a13g couldn't send a message to the juggernaut push server that would then get pushed up to the browser - if you give it a try let me know - it looks awful cool.
Re: Now, we can only push the messages to the views
by James Strachan,
Your message is awaiting moderation. Thank you for participating in the discussion.
We've had that part solved for a while using a cometd style integration using ActiveMQ and Jetty...
activemq.apache.org/ajax.html
In benchmarks for customers we've had a single process handling 14,000 concurrent users
James
LogicBlaze
Open Source SOA
Re: Now, we can only push the messages to the views
by hemant kumar,
Your message is awaiting moderation. Thank you for participating in the discussion.
Well Andrew, I have checked out Juggernaut inside out and actually have implemented by own push server.But the problem is, it can't bypass firewalls and worst of it NATs. This is a big hurdle in my opinion. Without solving above issues Juggernaut won't get mass adoptation.
Re: Now, we can only push the messages to the views
by hemant kumar,
Your message is awaiting moderation. Thank you for participating in the discussion.
Are we talking Java?
Excellent!
by Lee Zieke,
Your message is awaiting moderation. Thank you for participating in the discussion.
Would you please take a look at the table messages, I have got
mysql> select * from messages;
+----+--------------+---------------------+
| id | body | received_date |
+----+--------------+---------------------+
| 1 | Hello World | 0000-00-00 00:00:00 |
| 2 | testDateTime | 0000-00-00 00:00:00 |
+----+--------------+---------------------+
FIELD received_date, it supposed to be DateTime at which message were generated.
Please take a look at on_message
Re: Now, we can only push the messages to the views
by James Cook,
Your message is awaiting moderation. Thank you for participating in the discussion.
James, is that "cometd style" or api/protocol compliant cometd?
Re: Excellent!
by Andrew Kuklewicz,
Your message is awaiting moderation. Thank you for participating in the discussion.
Yeah, there is a one letter typo, I sent in a fix for this before the article was published, but guess it didn't make it into the article.
should be
not
Cheers,
-Andrew Kuklewicz
Re: Excellent!
by Lee Zieke,
Your message is awaiting moderation. Thank you for participating in the discussion.
perfect, a toast !
Errors
by Denis Labelle,
Your message is awaiting moderation. Thank you for participating in the discussion.
Hello all,
I followed along with the example, however when posting the message from the view, the server outputs the following message (infinitely, it would appear):
My poller runs fine:
I don't see a message in either the poller console, or the activemq console.
I changed the broker.yml configuration to look at denis:61613 instead of localhost because when activemq starts, it states:
Any ideas?
Thx,
dl
Re: Errors
by Denis Labelle,
Your message is awaiting moderation. Thank you for participating in the discussion.
er, nevermind I overlooked 1 line in my controller, rather than
def new
@message = params[:message]
...
end
was
def new
@message = Message.new(params[:message])
...
end
Re: Now, we can only push the messages to the views
by Chong Francis,
Your message is awaiting moderation. Thank you for participating in the discussion.
For pushing message in COMET style, you may use AjaxMessaging code.google.com/p/ajaxmessaging/
Re: Now, we can only push the messages to the views
by Alex MacCaw,
Your message is awaiting moderation. Thank you for participating in the discussion.
Hemant:
Not true, Juggernaut can use port 443 which is open on most firewall (used for https).
Re: Now, we can only push the messages to the views
by Alex MacCaw,
Your message is awaiting moderation. Thank you for participating in the discussion.
Juggernaut can bypass firewalls in that it can use ports commonly open on a firewall, such as 443. I'm not sure how NAT poses a problem, the port just gets forwarded to a internal server.
Ghost consumers lingering after script/poller stop?
by Brian Balser,
Your message is awaiting moderation. Thank you for participating in the discussion.
I've noticed that when using the poller, there are occasional consumers lingering around which AMQ sends the msg, and hence dropped, any idea if I'm not using the poller correctly? I have a run_poller script that does:
script/poller start
sleep 30
script/poller stop
Re: Errors
by Ajay Nayak,
Your message is awaiting moderation. Thank you for participating in the discussion.
Hi Author of this Article,
I have followed all your steps to config to code.
But when i am running "script/poller run"
I am not getting desirable log in console.
Only getting as :
^C^[[Aajay-dhandes-imac:MessageMonster mrunalini$ script/poller run
/Users/mrunalini/.gem/ruby/1.8/gems/stomp-1.1.3/lib/stomp/connection.rb:156: warning: parenthesize argument(s) for future version
ActiveMessaging: adapter wmq not loaded: no such file to load -- wmq/wmq
"/Users/mrunalini/code/MessageMonster"
"/Users/mrunalini/code/MessageMonster/config/messaging.rb"
ActiveMessaging: Loading /Users/mrunalini/code/MessageMonster/app/processors/application.rb
ActiveMessaging: Loading /Users/mrunalini/code/MessageMonster/app/processors/persist_message_processor.rb
Missing the Rails 2.3.3 gem. Please `gem install -v=2.3.3 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
My development environment is Mac OSX Leapord.
Please post suitable solutions for me.
Thanks,
Ajay nayak