consume web services like a ninja with eventmachine end event_utils

Lately I spent some time using eventmachine, my main interest so far has been in the deferrables and in writing clients to consume web services using them.

In the process I ended up writing event_utils a gem that makes (or at least should) more intuitive to write clients based on eventmachine deferrables.

Now let's start working at an example, first things first we need a web service to consume, our choice for this tutorial will be a dummy service named slowrand.

If you hit the url http://slowrand.elastastic.com/?delay=1 you'll be served a random number between 0 and 9, the random number will be served after a delay of at least one second.

So now that we have a web service to use, let's write a class that wraps the service.
Save the following as slow_rand.rb

class SlowRand

  attr_accessor :value

  include EM::Deferrable
  def initialize(delay = 1)
    client = EM::Protocols::HttpClient.request(
              :host => "slowrand.elastastic.com",
              :query_string => "delay=#{delay}")
    @value = nil
    client.callback do |response|
      self.value = response[:content].to_i
      puts "fetched value #{value} at #{Time.now}"
      self.succeed
    end
    client.errback { self.fail }
  end

  def +(other)
    self.value + other.value
  end

  def to_s
    value.to_s
  end

end

Let's take a look at the code above

In the initialize method we

1. use the EM HttpClient to generate a request to slowrand


2. we bind a callback to the request, in the callback we set the instance variable @value, we print some info on output and we set the deferred status of the slowrand object to succeeded


3. we bind an errback to the request to have some feedback in case the request fails

the Slowrand class defines also a + method do add one slowrand to another and for convenience defines a to_s method

ok now let's write client that fetches 2 slowrand and calculates the sum, the code will be

require 'rubygems'
require 'event_utils'
require 'slow_rand'
include EventUtils

in_deferred_loop do
  puts "started at #{Time.now}"

  a, b = SlowRand.new, SlowRand.new

  waiting_for(a, b) do
    sum = a + b
    puts "sum executed at #{Time.now}, #{a} + #{b} = #{sum}"
  end
end

Save the code above in client.rb
So, we initialize a deferred loop and print out the timestamp, after that we say to our client that he needs to wait for two slowrands to be fetched and only after that we execute the sum.

The idea behind it is very simple a and b are 2 deferrables and will return instantly but the value of a and b will be defined only after the web service will reply, that's why we ask our deferred loop to wait, specifically it will be waiting until all the deferrables listed in waiting_for will have the deferred status set.

Before running the client you need to install the eventmachine and event_utils gems

sudo gem install eventmachine
sudo gem install hungryblank-event_utils -s http://gems.github.com

and finally you can run the client

ruby client.rb

and you should see an output that looks like

started at Mon Mar 16 21:53:13 +0000 2009
fetched value 1 at Mon Mar 16 21:53:14 +0000 2009
fetched value 6 at Mon Mar 16 21:53:14 +0000 2009
sum executed at Mon Mar 16 21:53:14 +0000 2009, 1 + 6 = 7

Ok that's very little satisfaction but it takes little effort to make the client code more interesting.

Save the following code in client_multi.rb

require 'rubygems'
require 'event_utils'
require 'slow_rand'
include EventUtils

in_deferred_loop do
  puts "started at #{Time.now}"

  a, b = SlowRand.new(3), SlowRand.new(3)
  c, d = SlowRand.new(2), SlowRand.new(2)
  e, f = SlowRand.new, SlowRand.new

  waiting_for(a, b) do
    sum = a + b
    puts "== sum with delay 3 =="
    puts "sum executed at #{Time.now}, #{a} + #{b} = #{sum}"
  end

  waiting_for(c, d) do
    puts "== sum with delay 2 =="
    sum = c + d
    puts "sum executed at #{Time.now}, #{c} + #{d} = #{sum}"
  end

  waiting_for(e, f) do
    puts "== sum with delay 1 =="
    sum = e + f
    puts "sum executed at #{Time.now}, #{e} + #{f} = #{sum}"
  end
end

So in this case we execute 3 sums, as in the client seen before but with a twist, in our code we setup first a sum of slowrands with a delay of 3 seconds, after one with a delay of 2 seconds and at last one with a delay of 1 second.

After running our new client

ruby client_multi.rb

The output will look like

started at Mon Mar 16 21:57:48 +0000 2009
fetched value 0 at Mon Mar 16 21:57:50 +0000 2009
fetched value 2 at Mon Mar 16 21:57:50 +0000 2009
== sum with delay 1 ==
sum executed at Mon Mar 16 21:57:50 +0000 2009, 0 + 2 = 2
fetched value 8 at Mon Mar 16 21:57:51 +0000 2009
fetched value 6 at Mon Mar 16 21:57:51 +0000 2009
== sum with delay 2 ==
sum executed at Mon Mar 16 21:57:51 +0000 2009, 8 + 6 = 14
fetched value 8 at Mon Mar 16 21:57:52 +0000 2009
fetched value 8 at Mon Mar 16 21:57:52 +0000 2009
== sum with delay 3 ==
sum executed at Mon Mar 16 21:57:52 +0000 2009, 8 + 8 = 16

This is more interesting.
Because of the non blocking nature of eventmachine every sum has been performed as soon as possible.

The sum of the slowrands with a delay of 1 second, which was written as last in our code did actually get executed first without waiting for the code above it to be executed.

This example makes even more clear the advantages on the overall timing, the client fetched 2 slowrands with delay 1 second, 2 slowrands with delay 2 seconds and 2 slowrands with delay 3 seconds, a client that would fetch values sequentially would then spend at least 12 seconds to perform all the operations while in this case we spend about 4 seconds.

In addition of a quicker overall processing we published the first result available after a bit more than one second while a client performing actions sequentially would have spent 6 seconds waiting before publishing anything.

This example is more interesting if you think in terms of services with non predictable delays, without estimating what will happen first, you just need to specify what is needed to execute some specific code and let the events drive your code.

Hoping to have provided some interesting ground to explore I'll finish saying thanks to all the people who contributed to eventmachine or wrote tutorials that helped me in the process of learning.