Introduction to ActiveMessaging for Rails

Posted by Andrew Kuklewicz on Mar 01, 2007 02:40 PM

Community

Ruby

Topics

Messaging ,

Programming ,

Ruby on Rails

Tags

ActiveMQ ,

ActiveMessaging ,

Stomp ,

ActiveRecord

Introduction

Rails has already planted its flag firmly in the ground of database driven web applications. ActiveMessaging is a Rails plugin framework that extends the borders of Rails territory by adding simplified messaging integration. With ActiveMessaging and Rails now you can loosely integrate with systems as disparate as mainframes sending MQ messages or J2EE webapps, offload processing long-running tasks, or create applications with event or message-driven architectures. As others have put it, ActiveMessaging is trying to do for messaging what ActiveRecord does for databases. This article will introduce you to ActiveMessaging and related technologies, and get you started using it in your Rails applications now.

Messaging Standards and Brokers

Three things you need to get messaging working: a communication protocol, a broker speaking this protocol, and a client library for your applications speak the protocol. Technically, the fourth thing is a system to send messages to, but in this example we'll create a system that's a bit pathetic and talks to itself. For the protocol, ActiveMessaging supports Stomp: Streaming Text-Oriented Messaging Protocol. Stomp is a messaging standard to exchange text based messages over wire connections, usually tcp/ip. The idea was to make a protocol simple enough that clients and servers for it can be created easily in almost any language (e.g. the Ruby Stomp client is about 400 lines, soaking wet with comments and blank lines), so it should be possible to integrate almost any system.

The Apache ActiveMQ message broker supports Stomp natively, though other broker options are available, with more coming all the time. In fact, the recently announced StompConnect project makes any JMS broker into a Stomp broker, which opens up Stomp, and ActiveMessaging, to practically every open source and proprietary messaging product. On the client side, ActiveMessaging supports protocols using an adapter pattern, and for Stomp it extends the Stomp RubyGem to be that adapter. More adapters are possible and in the works, but Stomp is the only one supported today.

For this article, I'll be using ActiveMQ as the broker, and working with the as yet unreleased 4.2 branch. ActiveMQ has install instructions for all platforms, but here's a simple procedure that assumes you have Java 1.4 or later:

cd /usr/local/src
 #unix and cygwin
 wget http://people.apache.org/repo/m2-snapshot-repository/org/apache/activemq/apache-activemq/4.2-SNAPSHOT/apache-activemq-4.2-20070221.081507-10-src.tar.gz
#os x
curl -O http://people.apache.org/repo/m2-snapshot-repository/org/apache/activemq/apache-activemq/4.2-SNAPSHOT/apache-activemq-4.2-20070221.081507-10-src.tar.gz
 
cd ..
tar xvfz apache-activemq-4.2-20070221.081507-10-src.tar.gz
cd apache-activemq-4.2-incubator-SNAPSHOT
 ./bin/activemq 

That's it! By default, ActiveMQ's Stomp support should be configured to start up automatically, but if not you should add the following line to conf/activemq.xml:

<transportConnectors> 
    ...
    <!-- Add this line -->
    <transportConnector name="stomp" uri="stomp://localhost:61613"/>
</transportConnectors> 

That's the minimum, to configure ActiveMQ to your heart's desire, check out the ActiveMessaging wiki page on ActiveMQ, and the using ActiveMQ guides.

Rails Setup

Assuming you have Rails 1.1+ and MySql already, you'll also need 2 more RubyGems: daemons and Stomp:

sudo gem install daemons
sudo gem install stomp 

Let's start with a newly minted Rails 'MessageMonster', a database for it, and install the ActiveMessaging plugin:

cd /usr/local/dev
rails MessageMonster
mysqladmin create messagemonster_development -u root
cd MessageMonster
script/plugin install http://activemessaging.googlecode.com/svn/trunk/plugins/activemessaging 

All the pre-requisites are in place, so it's time to write an application.

Processing Messages

We'll start by creating a 'processor', which in ActiveMessaging lingo is a class that subscribes to destinations in order to receive and act on messages. It is the message driven equivalent of your controllers, and along with your model where you'll put most of your code.

As Rails has taught you to expect, there is a generator to create processors. Let's create one that saves messages it receives to show ActiveRecord integration:

script/generate processor PersistMessage 

Running this generate for the first time outputs the following:

create  app/processors
create  app/processors/persist_message_processor.rb
create  config/messaging.rb
create  config/broker.yml
create  app/processors/application.rb
create  script/poller 

Let's talk about each of these a moment.

• app/processors is the directory for all the processors.
• app/processors/persist_message_processor.rb is the new processor, with a default implementation for you to work from.
• app/processors/application.rb is the common super class from which processors extend. Use this to put shared code, such as error handling.
• config/broker.yml configures how you connect to your message broker in each environment. It's the equivalent of database.yml, but for message brokers.
• config/messaging.rb is used for the rest of the messaging configuration, like destinations and message headers.
• script/poller is the daemon used by ActiveMessaging to listen for messages. More on this script later.

Let's check out the broker.yml file first, to see the options and make sure the development broker configuration is right:

development:
    adapter: stomp
    login: ""
    passcode: ""
    host: localhost
    port: 61613
    reliable: true
    reconnectDelay: 5
 ... 

All of these values are exactly what we need for our default ActiveMQ installation, but for production you will likely have a user and password, and if you run your broker remotely, a different host and port.

So now let's look at the persist_message_processor.rb:

class PersistMessageProcessor < ApplicationProcessor
  subscribes_to :persist_message
  def on_message(message)
    logger.debug "PersistMessageProcessor received: " + message
  end
 end

By default you'll see it is subscribed to :persist_message and has one method, on_message, which will be invoked by ActiveMessaging when messages for the :persist_message subscription arrive. The body of the message is passed in as the string argument message, but if you need to get to the Stomp message object itself, it is available in the @message processor attribute. Processors can subscribe to more than one queue, simply add anther call to subscribes_to.

The :persist_message symbol is a logical name for the destination to which this processor subscribes, the definition of this destination was generated in config/messaging.rb:

ActiveMessaging::Gateway.define do |s|
  s.queue :persist_message, '/queue/PersistMessage'
 end 

To use a comparison to Rails, messaging.rb is most like routes.rb. It allows you to map a logical name (:persist_message) to a destination on the broker (/queue/PersistMessage). Though there is much more that can be configured here, the generated default is fine for this example.

Now we'll teach the processor to save its messages for a rainy day, starting with creating a model for the messages.

script/generate model message 

Now update 'db/migrate/001_create_messages.rb' with 2 additional columns:

create_table :messages do |t|
  t.column :body, :text
  t.column :received_date, :datetime
 end 

Call migrate to create the table...

rake db:migrate 

...and on to updating the processor on_message method to use the new Message model:

def on_message(message)
  logger.debug "PersistMessageProcessor received: " + message
  my_message = Message.create(:body=>message, :received_date=>DateTime.new)
 end 

Now, you're looking at this and thinking that's nothing new - this is just how you would write the same thing in a Rails controller. That's the point, when you use ActiveMessaging, all the convenience of the Rails application environment is available to you.

With a processor to catch them, it's time to start throwing messages (bottles sold separately).

Sending Messages

To send messages we'll use a Rails view and controller in the same application:

script/generate controller SendMessage index

Edit the view to submit message text, and to show a refreshable list of previously persisted messages:

<p style="color: green"><%= flash[:notice] %></p> <h1>Send Message</h1> <%= start_form_tag :action => 'new' %> <label>Message</label> <%=text_field_tag :message, @message %>  <%= submit_tag "submit" %> <%= end_form_tag %> <%= link_to "Refresh List", :action => "index" %> <table> <% @messages.each do |m| -%>   <tr><td><%= m.received_date %></td><td><%= m.body %></td></tr> <% end %> </table> 

Edit the controller to send the message. To make sending from any class easier, ActiveMessaging has a mixin MessageSender that provides a simple publish method. The publishes_to method declares to which destination a class intends to send, but does nothing more than validate that the destination is in config/messaging.rb. Here is how the controller uses these methods:

require 'activemessaging/processor'
 
 class SendMessageController < ApplicationController
 
   include ActiveMessaging::MessageSender
  publishes_to :persist_message
 
   def index
    @messages = Message.find :all
  end
 
   def new
    @message = params[:message]
    publish :persist_message, @message
    flash[:notice] = "'#{@message}' sent"
    @messages = Message.find :all
    render :action => 'index'
  end
 end
 

And that, as they say, is that. The controller will send a message on submit, and when next you view or refresh the page you'll see the messages saved by the processor.

Running the Example

ActiveMessaging runs in its own process, controlled by the script/poller, which can be a gotcha – just starting the Rails server is not enough if you want to receive messages, and messages are received into a different running process than Rails. To run this example, we need 3 processes: the ActiveMQ broker, the Rails server running to send the message, and the script/poller to receive it, each running in their own terminal:

Start ActiveMQ:

cd /usr/local/apache-activemq-4.2-incubator-SNAPSHOT
 ./bin/activemq

Start Rails:

cd /usr/local/dev/MessageMonster
 script/server

Run ActiveMessaging:

cd /usr/local/dev/MessageMonster
script/poller run

The script/poller follows the usual daemon semantics. Besides 'run' which keeps the processing in the foreground, you can also pass it 'start','stop','restart' and 'status'. It also uses allows you to run multiple instances simultaneously, a useful for handling higher volumes of messages.

Now navigate to the test page (i.e. http://localhost:3000/send_message ) and enter a message such as the canonical "Hello World". When the message is sent you should see the following in the poller window:

Loading /usr/local/dev/MessageMonster/app/processors/application.rb
Loading /usr/local/dev/MessageMonster/app/processors/persist_message_processor.rb
=> Subscribing to /queue/PersistMessage (processed by PersistMessageProcessor)
=> All subscribed, now polling
PersistMessageProcessor received: Hello World 

The poller always shows what processors it loads on start-up, and what subscriptions requests on the broker. If the broker is unavailable or lost, it will try to reconnect based on your broker.yml configuration. After the poller start output, you can now also see the ActiveMessaging processor has received the message sent from Rails. Likewise, if you use the 'Refresh List' button, you'll see the message in the Rails view once it is persisted by the ActiveMessaging processor.

Where to go from here

I hope you enjoyed this intro to ActiveMessaging, hopefully you can see both its potential and ease of use. ActiveMessaging has more advanced capabilities not covered in this article, such as improved exception handling, grouping processors to be run by multiple pollers, or how to use Stomp and ActiveMQ message headers for tricks like synchronous messaging, JMS integration, and selective subscriptions. If you are thinking after seeing the basics that ActiveMessaging is too limited for your needs, I think you might be pleasantly surprised.

For more information, I recommend perusing the ActiveMessaging project site, especially the wiki and the code.

We are constantly improving ActiveMessaging, so to both stay up on changes (and recommend or submit your own) read or join the mailing list.

Thanks for your attention, and happy messaging!

---

Andrew Kuklewicz has worked in software development for the last 10 years, and is senior web developer at PRX (the Public Radio Exchange), a nonprofit service for distribution, peer review, and licensing of radio pieces. When not working in public media, or tap dancing, he is the current maintainer of ActiveMessaging, a committer on the Ruby Stomp project, and frequent contributer to other Ruby on Rails, Plone, and Java open source projects.

1. Back to top

   Great article!

   Mar 2, 2007 2:57 AM by James Strachan

   Thanks for a great article Andrew! ActiveMessaging really rocks (as does Rails, Stomp and ActiveMQ :). Keep up the good work! James LogicBlaze Open Source SOA

   Reply

2. Back to top

   Performance/Throughput?

   Mar 2, 2007 1:39 PM by Stefan Tilkov

   Excellent article. Are there any benchmarks/performance figures yet?

   Reply

3. Back to top

   Re: Performance/Throughput?

   Mar 2, 2007 4:35 PM by Andrew Kuklewicz

   good question - the straight answer is I haven't seen or or run any such test myself. If anyone else has, I would be very curious to hear about it. So I don't know exactly what the a13g overhead is, or how it scales. I have a feeling this may depend much more on your message broker, and then what you do in the message processor. These kind of tests are a bit tricky as you are testing the network, the broker, the box, etc. That said, it would be good to design a benchmark, perhaps based off the work done at ActiveMQ. I can say that anecdotally, I find the processing of the message to take longer than the overhead in dispatch - we even do synchronous messages from our java app to our rails app and back. One other nice thing is that you can run multiple instances of the poller process, so with N number of pollers all listening to the same queue, you in theory have as much scalability as you have processing power, and depending on the efficiency of the broker dispatch. Cheers, Andrew Kuklewicz

   Reply

4. Back to top

   Now, we can only push the messages to the views

   Mar 6, 2007 2:52 PM by hemant kumar

   This work is awesome, and all we need is a way to push these messages to the browser, so that browser doesn't poll for it. any ideas on this?

   Reply

5. Back to top

   Re: Now, we can only push the messages to the views

   Mar 6, 2007 11:06 PM by Andrew Kuklewicz

   I only know one non-polling based way to push to the browser, and that is the Juggernaut project. Personally I am very tempted to mess with the juggernaut http://juggernaut.rubyforge.org/ solution, I just haven't had an excuse. From what I understand it involves a separate "push server" that sits between a rails server app, and the flash apps on the browser. I see no reason a processor in a13g couldn't send a message to the juggernaut push server that would then get pushed up to the browser - if you give it a try let me know - it looks awful cool.

   Reply

6. Back to top

   Re: Now, we can only push the messages to the views

   Mar 7, 2007 3:24 AM by James Strachan

   This work is awesome, and all we need is a way to push these messages to the browser, so that browser doesn't poll for it. any ideas on this?

   We've had that part solved for a while using a cometd style integration using ActiveMQ and Jetty... http://activemq.apache.org/ajax.html In benchmarks for customers we've had a single process handling 14,000 concurrent users James LogicBlaze Open Source SOA

   Reply

7. Back to top

   Re: Now, we can only push the messages to the views

   Mar 18, 2007 3:56 AM by hemant kumar

   Well Andrew, I have checked out Juggernaut inside out and actually have implemented by own push server.But the problem is, it can't bypass firewalls and worst of it NATs. This is a big hurdle in my opinion. Without solving above issues Juggernaut won't get mass adoptation.

   Reply

8. Back to top

   Re: Now, we can only push the messages to the views

   Mar 18, 2007 3:56 AM by hemant kumar

   Are we talking Java?

   Reply

9. Back to top

   Excellent!

   Mar 25, 2007 6:18 AM by Zieke Lee

   Would you please take a look at the table messages, I have got mysql> select * from messages; +----+--------------+---------------------+ | id | body | received_date | +----+--------------+---------------------+ | 1 | Hello World | 0000-00-00 00:00:00 | | 2 | testDateTime | 0000-00-00 00:00:00 | +----+--------------+---------------------+ FIELD received_date, it supposed to be DateTime at which message were generated. Please take a look at on_message

   Reply

10. Back to top

    Re: Now, we can only push the messages to the views

    Mar 26, 2007 9:13 PM by James Cook

    James, is that "cometd style" or api/protocol compliant cometd?

    Reply

11. Back to top

    Re: Excellent!

    Apr 5, 2007 12:41 PM by Andrew Kuklewicz

    Yeah, there is a one letter typo, I sent in a fix for this before the article was published, but guess it didn't make it into the article.

    def on_message(message)
      logger.debug "PersistMessageProcessor received: " + message
      my_message = Message.create(:body=>message, :received_date=>DateTime.new)
    end 
    

    should be

    :received_date=>DateTime.now

    not

    :received_date=>DateTime.new

    Cheers, -Andrew Kuklewicz

    Reply

12. Back to top

    Re: Excellent!

    Apr 15, 2007 9:13 AM by Zieke Lee

    perfect, a toast !

    Reply

13. Back to top

    Errors

    Apr 26, 2007 7:36 PM by Denis Labelle

    Hello all, I followed along with the example, however when posting the message from the view, the server outputs the following message (infinitely, it would appear):

    transmit failed: undefined method `length' for # transmit failed: undefined method `length' for # transmit failed: undefined method `length' for # ...

    My poller runs fine:

    Loading D:/dev/ruby/Mess/app/processors/application.rb Loading D:/dev/ruby/Mess/app/processors/persist_message_processor.rb => Subscribing to /queue/PersistMessage (processed by PersistMessageProcessor)

    I don't see a message in either the poller console, or the activemq console. I changed the broker.yml configuration to look at denis:61613 instead of localhost because when activemq starts, it states:

    ... INFO TransportServerThreadSupport - Listening for connections at: tcp://denis:61616 INFO TransportConnector - Connector openwire Started INFO TransportServerThreadSupport - Listening for connections at: ssl://denis:61617 INFO TransportConnector - Connector ssl Started INFO TransportServerThreadSupport - Listening for connections at: stomp://denis:61613 INFO TransportConnector - Connector stomp Started INFO TransportServerThreadSupport - Listening for connections at: xmpp://denis:61222 INFO TransportConnector - Connector xmpp Started ...

    Any ideas? Thx, dl

    Reply

14. Back to top

    Re: Errors

    Apr 27, 2007 5:45 PM by Denis Labelle

    er, nevermind I overlooked 1 line in my controller, rather than def new @message = params[:message] ... end was def new @message = Message.new(params[:message]) ... end

    Reply

15. Back to top

    Re: Now, we can only push the messages to the views

    Jul 10, 2007 2:37 AM by Francis Chong

    For pushing message in COMET style, you may use AjaxMessaging http://code.google.com/p/ajaxmessaging/

    Reply

16. Back to top

    Re: Now, we can only push the messages to the views

    Jul 25, 2007 4:43 AM by Alex MacCaw

    Hemant: Not true, Juggernaut can use port 443 which is open on most firewall (used for https).

    Reply

17. Back to top

    Re: Now, we can only push the messages to the views

    Sep 7, 2007 10:49 AM by Alex MacCaw

    Juggernaut can bypass firewalls in that it can use ports commonly open on a firewall, such as 443. I'm not sure how NAT poses a problem, the port just gets forwarded to a internal server.

    Reply

18. Back to top

    Ghost consumers lingering after script/poller stop?

    Aug 13, 2008 5:14 PM by Brian Balser

    I've noticed that when using the poller, there are occasional consumers lingering around which AMQ sends the msg, and hence dropped, any idea if I'm not using the poller correctly? I have a run_poller script that does: script/poller start sleep 30 script/poller stop

    Reply

• Older >