BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles Introduction to ActiveMessaging for Rails

Introduction to ActiveMessaging for Rails

This item in japanese

Bookmarks

Introduction

Rails has already planted its flag firmly in the ground of database driven web applications. ActiveMessaging is a Rails plugin framework that extends the borders of Rails territory by adding simplified messaging integration. With ActiveMessaging and Rails now you can loosely integrate with systems as disparate as mainframes sending MQ messages or J2EE webapps, offload processing long-running tasks, or create applications with event or message-driven architectures. As others have put it, ActiveMessaging is trying to do for messaging what ActiveRecord does for databases. This article will introduce you to ActiveMessaging and related technologies, and get you started using it in your Rails applications now.

Messaging Standards and Brokers

Three things you need to get messaging working: a communication protocol, a broker speaking this protocol, and a client library for your applications speak the protocol. Technically, the fourth thing is a system to send messages to, but in this example we'll create a system that's a bit pathetic and talks to itself. For the protocol, ActiveMessaging supports Stomp: Streaming Text-Oriented Messaging Protocol. Stomp is a messaging standard to exchange text based messages over wire connections, usually tcp/ip. The idea was to make a protocol simple enough that clients and servers for it can be created easily in almost any language (e.g. the Ruby Stomp client is about 400 lines, soaking wet with comments and blank lines), so it should be possible to integrate almost any system.

The Apache ActiveMQ message broker supports Stomp natively, though other broker options are available, with more coming all the time. In fact, the recently announced StompConnect project makes any JMS broker into a Stomp broker, which opens up Stomp, and ActiveMessaging, to practically every open source and proprietary messaging product. On the client side, ActiveMessaging supports protocols using an adapter pattern, and for Stomp it extends the Stomp RubyGem to be that adapter. More adapters are possible and in the works, but Stomp is the only one supported today.

For this article, I'll be using ActiveMQ as the broker, and working with the as yet unreleased 4.2 branch. ActiveMQ has install instructions for all platforms, but here's a simple procedure that assumes you have Java 1.4 or later:

cd /usr/local/src
#unix and cygwin
wget http://people.apache.org/repo/m2-snapshot-repository/org/apache/activemq/apache-activemq/4.2-SNAPSHOT/apache-activemq-4.2-20070221.081507-10-src.tar.gz
#os x
curl -O http://people.apache.org/repo/m2-snapshot-repository/org/apache/activemq/apache-activemq/4.2-SNAPSHOT/apache-activemq-4.2-20070221.081507-10-src.tar.gz

cd ..
tar xvfz apache-activemq-4.2-20070221.081507-10-src.tar.gz
cd apache-activemq-4.2-incubator-SNAPSHOT
./bin/activemq

That's it! By default, ActiveMQ's Stomp support should be configured to start up automatically, but if not you should add the following line to conf/activemq.xml:

<transportConnectors> 
...
<!-- Add this line -->
<transportConnector name="stomp" uri="stomp://localhost:61613"/>
</transportConnectors>

That's the minimum, to configure ActiveMQ to your heart's desire, check out the ActiveMessaging wiki page on ActiveMQ, and the using ActiveMQ guides.

Rails Setup

Assuming you have Rails 1.1+ and MySql already, you'll also need 2 more RubyGems: daemons and Stomp:

sudo gem install daemons
sudo gem install stomp

Let's start with a newly minted Rails 'MessageMonster', a database for it, and install the ActiveMessaging plugin:

cd /usr/local/dev
rails MessageMonster
mysqladmin create messagemonster_development -u root
cd MessageMonster
script/plugin install http://activemessaging.googlecode.com/svn/trunk/plugins/activemessaging

All the pre-requisites are in place, so it's time to write an application.

Processing Messages

We'll start by creating a 'processor', which in ActiveMessaging lingo is a class that subscribes to destinations in order to receive and act on messages. It is the message driven equivalent of your controllers, and along with your model where you'll put most of your code.

As Rails has taught you to expect, there is a generator to create processors. Let's create one that saves messages it receives to show ActiveRecord integration:

script/generate processor PersistMessage 

Running this generate for the first time outputs the following:

create  app/processors
create app/processors/persist_message_processor.rb
create config/messaging.rb
create config/broker.yml
create app/processors/application.rb
create script/poller

Let's talk about each of these a moment.
  • app/processors is the directory for all the processors.
  • app/processors/persist_message_processor.rb is the new processor, with a default implementation for you to work from.
  • app/processors/application.rb is the common super class from which processors extend. Use this to put shared code, such as error handling.
  • config/broker.yml configures how you connect to your message broker in each environment. It's the equivalent of database.yml, but for message brokers.
  • config/messaging.rb is used for the rest of the messaging configuration, like destinations and message headers.
  • script/poller is the daemon used by ActiveMessaging to listen for messages. More on this script later.

Let's check out the broker.yml file first, to see the options and make sure the development broker configuration is right:

development:
adapter: stomp
login: ""
passcode: ""
host: localhost
port: 61613
reliable: true
reconnectDelay: 5
...

All of these values are exactly what we need for our default ActiveMQ installation, but for production you will likely have a user and password, and if you run your broker remotely, a different host and port.

So now let's look at the persist_message_processor.rb:

class PersistMessageProcessor < ApplicationProcessor
subscribes_to :persist_message
def on_message(message)
logger.debug "PersistMessageProcessor received: " + message
end
end

By default you'll see it is subscribed to :persist_message and has one method, on_message, which will be invoked by ActiveMessaging when messages for the :persist_message subscription arrive. The body of the message is passed in as the string argument message, but if you need to get to the Stomp message object itself, it is available in the @message processor attribute. Processors can subscribe to more than one queue, simply add anther call to subscribes_to.

The :persist_message symbol is a logical name for the destination to which this processor subscribes, the definition of this destination was generated in config/messaging.rb:

ActiveMessaging::Gateway.define do |s|
s.queue :persist_message, '/queue/PersistMessage'
end

To use a comparison to Rails, messaging.rb is most like routes.rb. It allows you to map a logical name (:persist_message) to a destination on the broker (/queue/PersistMessage). Though there is much more that can be configured here, the generated default is fine for this example.

Now we'll teach the processor to save its messages for a rainy day, starting with creating a model for the messages.

script/generate model message 

Now update 'db/migrate/001_create_messages.rb' with 2 additional columns:

create_table :messages do |t|
t.column :body, :text
t.column :received_date, :datetime
end

Call migrate to create the table...

rake db:migrate 

...and on to updating the processor on_message method to use the new Message model:

def on_message(message)
logger.debug "PersistMessageProcessor received: " + message
my_message = Message.create(:body=>message, :received_date=>DateTime.new)
end

Now, you're looking at this and thinking that's nothing new - this is just how you would write the same thing in a Rails controller. That's the point, when you use ActiveMessaging, all the convenience of the Rails application environment is available to you.

With a processor to catch them, it's time to start throwing messages (bottles sold separately).

Sending Messages

To send messages we'll use a Rails view and controller in the same application:

script/generate controller SendMessage index

Edit the view to submit message text, and to show a refreshable list of previously persisted messages:

<p style="color: green"><%= flash[:notice] %></p> <h1>Send Message</h1> <%= start_form_tag :action => 'new' %> <label>Message</label> <%=text_field_tag :message, @message %>  <%= submit_tag "submit" %> <%= end_form_tag %> <%= link_to "Refresh List", :action => "index" %> <table> <% @messages.each do |m| -%>   <tr><td><%= m.received_date %></td><td><%= m.body %></td></tr> <% end %> </table> 

Edit the controller to send the message. To make sending from any class easier, ActiveMessaging has a mixin MessageSender that provides a simple publish method. The publishes_to method declares to which destination a class intends to send, but does nothing more than validate that the destination is in config/messaging.rb. Here is how the controller uses these methods:

require 'activemessaging/processor'

class SendMessageController < ApplicationController

include ActiveMessaging::MessageSender
publishes_to :persist_message

def index
@messages = Message.find :all
end

def new
@message = params[:message]
publish :persist_message, @message
flash[:notice] = "'#{@message}' sent"
@messages = Message.find :all
render :action => 'index'
end
end

And that, as they say, is that. The controller will send a message on submit, and when next you view or refresh the page you'll see the messages saved by the processor.

Running the Example

ActiveMessaging runs in its own process, controlled by the script/poller, which can be a gotcha – just starting the Rails server is not enough if you want to receive messages, and messages are received into a different running process than Rails. To run this example, we need 3 processes: the ActiveMQ broker, the Rails server running to send the message, and the script/poller to receive it, each running in their own terminal:

Start ActiveMQ:

cd /usr/local/apache-activemq-4.2-incubator-SNAPSHOT
./bin/activemq

Start Rails:

cd /usr/local/dev/MessageMonster
script/server

Run ActiveMessaging:

cd /usr/local/dev/MessageMonster
script/poller run

The script/poller follows the usual daemon semantics. Besides 'run' which keeps the processing in the foreground, you can also pass it 'start','stop','restart' and 'status'. It also uses allows you to run multiple instances simultaneously, a useful for handling higher volumes of messages.

Now navigate to the test page (i.e. http://localhost:3000/send_message ) and enter a message such as the canonical "Hello World". When the message is sent you should see the following in the poller window:

Loading /usr/local/dev/MessageMonster/app/processors/application.rb
Loading /usr/local/dev/MessageMonster/app/processors/persist_message_processor.rb
=> Subscribing to /queue/PersistMessage (processed by PersistMessageProcessor)
=> All subscribed, now polling
PersistMessageProcessor received: Hello World

The poller always shows what processors it loads on start-up, and what subscriptions requests on the broker. If the broker is unavailable or lost, it will try to reconnect based on your broker.yml configuration. After the poller start output, you can now also see the ActiveMessaging processor has received the message sent from Rails. Likewise, if you use the 'Refresh List' button, you'll see the message in the Rails view once it is persisted by the ActiveMessaging processor.

Where to go from here

I hope you enjoyed this intro to ActiveMessaging, hopefully you can see both its potential and ease of use. ActiveMessaging has more advanced capabilities not covered in this article, such as improved exception handling, grouping processors to be run by multiple pollers, or how to use Stomp and ActiveMQ message headers for tricks like synchronous messaging, JMS integration, and selective subscriptions. If you are thinking after seeing the basics that ActiveMessaging is too limited for your needs, I think you might be pleasantly surprised.

  • For more information, I recommend perusing the ActiveMessaging project site, especially the wiki and the code.
  • We are constantly improving ActiveMessaging, so to both stay up on changes (and recommend or submit your own) read or join the mailing list.
  • Thanks for your attention, and happy messaging!


    Andrew Kuklewicz has worked in software development for the last 10 years, and is senior web developer at PRX (the Public Radio Exchange), a nonprofit service for distribution, peer review, and licensing of radio pieces. When not working in public media, or tap dancing, he is the current maintainer of ActiveMessaging, a committer on the Ruby Stomp project, and frequent contributer to other Ruby on Rails, Plone, and Java open source projects.

    Rate this Article

    Adoption
    Style

    Hello stranger!

    You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

    Get the most out of the InfoQ experience.

    Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

    Community comments

    • Great article!

      by James Strachan,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      Thanks for a great article Andrew! ActiveMessaging really rocks (as does Rails, Stomp and ActiveMQ :). Keep up the good work!

      James
      LogicBlaze
      Open Source SOA

    • Performance/Throughput?

      by Stefan Tilkov,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      Excellent article. Are there any benchmarks/performance figures yet?

    • Re: Performance/Throughput?

      by Andrew Kuklewicz,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      good question - the straight answer is I haven't seen or or run any such test myself. If anyone else has, I would be very curious to hear about it.

      So I don't know exactly what the a13g overhead is, or how it scales. I have a feeling this may depend much more on your message broker, and then what you do in the message processor. These kind of tests are a bit tricky as you are testing the network, the broker, the box, etc.

      That said, it would be good to design a benchmark, perhaps based off the work done at ActiveMQ.

      I can say that anecdotally, I find the processing of the message to take longer than the overhead in dispatch - we even do synchronous messages from our java app to our rails app and back.

      One other nice thing is that you can run multiple instances of the poller process, so with N number of pollers all listening to the same queue, you in theory have as much scalability as you have processing power, and depending on the efficiency of the broker dispatch.

      Cheers,
      Andrew Kuklewicz

    • Now, we can only push the messages to the views

      by hemant kumar,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      This work is awesome, and all we need is a way to push these messages to the browser, so that browser doesn't poll for it. any ideas on this?

    • Re: Now, we can only push the messages to the views

      by Andrew Kuklewicz,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      I only know one non-polling based way to push to the browser, and that is the Juggernaut project.

      Personally I am very tempted to mess with the juggernaut juggernaut.rubyforge.org/ solution, I just haven't had an excuse.

      From what I understand it involves a separate "push server" that sits between a rails server app, and the flash apps on the browser. I see no reason a processor in a13g couldn't send a message to the juggernaut push server that would then get pushed up to the browser - if you give it a try let me know - it looks awful cool.

    • Re: Now, we can only push the messages to the views

      by James Strachan,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      This work is awesome, and all we need is a way to push these messages to the browser, so that browser doesn't poll for it. any ideas on this?


      We've had that part solved for a while using a cometd style integration using ActiveMQ and Jetty...

      activemq.apache.org/ajax.html

      In benchmarks for customers we've had a single process handling 14,000 concurrent users

      James
      LogicBlaze
      Open Source SOA

    • Re: Now, we can only push the messages to the views

      by hemant kumar,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      Well Andrew, I have checked out Juggernaut inside out and actually have implemented by own push server.But the problem is, it can't bypass firewalls and worst of it NATs. This is a big hurdle in my opinion. Without solving above issues Juggernaut won't get mass adoptation.

    • Re: Now, we can only push the messages to the views

      by hemant kumar,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      Are we talking Java?

    • Excellent!

      by Lee Zieke,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      Would you please take a look at the table messages, I have got
      mysql> select * from messages;
      +----+--------------+---------------------+
      | id | body | received_date |
      +----+--------------+---------------------+
      | 1 | Hello World | 0000-00-00 00:00:00 |
      | 2 | testDateTime | 0000-00-00 00:00:00 |
      +----+--------------+---------------------+

      FIELD received_date, it supposed to be DateTime at which message were generated.

      Please take a look at on_message

    • Re: Now, we can only push the messages to the views

      by James Cook,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      James, is that "cometd style" or api/protocol compliant cometd?

    • Re: Excellent!

      by Andrew Kuklewicz,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      Yeah, there is a one letter typo, I sent in a fix for this before the article was published, but guess it didn't make it into the article.


      def on_message(message)
      logger.debug "PersistMessageProcessor received: " + message
      my_message = Message.create(:body=>message, :received_date=>DateTime.new)
      end


      should be
      :received_date=>DateTime.now
      not
      :received_date=>DateTime.new

      Cheers,

      -Andrew Kuklewicz

    • Re: Excellent!

      by Lee Zieke,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      perfect, a toast !

    • Errors

      by Denis Labelle,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      Hello all,

      I followed along with the example, however when posting the message from the view, the server outputs the following message (infinitely, it would appear):


      transmit failed: undefined method `length' for #<Message:0x45f89b8>
      transmit failed: undefined method `length' for #<Message:0x45f89b8>
      transmit failed: undefined method `length' for #<Message:0x45f89b8>
      ...


      My poller runs fine:

      Loading D:/dev/ruby/Mess/app/processors/application.rb
      Loading D:/dev/ruby/Mess/app/processors/persist_message_processor.rb
      => Subscribing to /queue/PersistMessage (processed by
      PersistMessageProcessor)


      I don't see a message in either the poller console, or the activemq console.
      I changed the broker.yml configuration to look at denis:61613 instead of localhost because when activemq starts, it states:

      ...
      INFO TransportServerThreadSupport - Listening for connections at: tcp://denis:61616
      INFO TransportConnector - Connector openwire Started
      INFO TransportServerThreadSupport - Listening for connections at: ssl://denis:61617
      INFO TransportConnector - Connector ssl Started
      INFO TransportServerThreadSupport - Listening for connections at: stomp://denis:61613
      INFO TransportConnector - Connector stomp Started
      INFO TransportServerThreadSupport - Listening for connections at: xmpp://denis:61222
      INFO TransportConnector - Connector xmpp Started
      ...


      Any ideas?

      Thx,
      dl

    • Re: Errors

      by Denis Labelle,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      er, nevermind I overlooked 1 line in my controller, rather than
      def new
      @message = params[:message]
      ...
      end

      was
      def new
      @message = Message.new(params[:message])
      ...
      end

    • Re: Now, we can only push the messages to the views

      by Chong Francis,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      For pushing message in COMET style, you may use AjaxMessaging code.google.com/p/ajaxmessaging/

    • Re: Now, we can only push the messages to the views

      by Alex MacCaw,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      Hemant:
      Not true, Juggernaut can use port 443 which is open on most firewall (used for https).

    • Re: Now, we can only push the messages to the views

      by Alex MacCaw,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      Juggernaut can bypass firewalls in that it can use ports commonly open on a firewall, such as 443. I'm not sure how NAT poses a problem, the port just gets forwarded to a internal server.

    • Ghost consumers lingering after script/poller stop?

      by Brian Balser,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      I've noticed that when using the poller, there are occasional consumers lingering around which AMQ sends the msg, and hence dropped, any idea if I'm not using the poller correctly? I have a run_poller script that does:

      script/poller start
      sleep 30
      script/poller stop

    • Re: Errors

      by Ajay Nayak,

      Your message is awaiting moderation. Thank you for participating in the discussion.

      Hi Author of this Article,

      I have followed all your steps to config to code.
      But when i am running "script/poller run"
      I am not getting desirable log in console.

      Only getting as :
      ^C^[[Aajay-dhandes-imac:MessageMonster mrunalini$ script/poller run
      /Users/mrunalini/.gem/ruby/1.8/gems/stomp-1.1.3/lib/stomp/connection.rb:156: warning: parenthesize argument(s) for future version
      ActiveMessaging: adapter wmq not loaded: no such file to load -- wmq/wmq
      "/Users/mrunalini/code/MessageMonster"
      "/Users/mrunalini/code/MessageMonster/config/messaging.rb"
      ActiveMessaging: Loading /Users/mrunalini/code/MessageMonster/app/processors/application.rb
      ActiveMessaging: Loading /Users/mrunalini/code/MessageMonster/app/processors/persist_message_processor.rb
      Missing the Rails 2.3.3 gem. Please `gem install -v=2.3.3 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.


      My development environment is Mac OSX Leapord.

      Please post suitable solutions for me.

      Thanks,
      Ajay nayak

    Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

    Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

    BT