Thursday, 14 May 2009

Amazon AWS SimpleDB a la carte with ruby and aws_sdb_bare

Among the Amazon Web Services or AWS if you prefer acronyms, SimpleDB is one of the less popular.

I've been using this service it for a while in some personal projects using the aws-sdb, so far the only choice to deal with SimpleDB at a low level.

Since I wanted to overcome some of the limits of aws-sdb I wrote a new gem aws_sdb_bare.
Main feature of the gem are:

* Complete implementation of SimpleDB api
* XML parsing uses Hpricot or Nokogiri

And be free of chosing your HTTP connection library and strategy!

You can use Net::HTTP or curb or go concurrent using eventmachine or typhoeus to manage your connections with sdb

So just install it


sudo gem install hungryblank-aws_sdb_bare -s http://gems.github.com


And use the code in this gist to get started, rdoc are available.

For a more complete overview of the SimpleDB ruby libraries currently available you should read this article

Monday, 11 May 2009

RailsConf 2009 the "system" stream

RailsConf 2009 finished few days ago and after a long trip back and some time to chill out I was going through the material of the conference and trying to connect some dots among the different topics which were covered.

To give some background, I started my career as sysadmin (or linux geek, if you prefer) and because of that I like when custom software, whether it's a rails app or just a small script is seen in the context of an entire stack or as glue among heterogeneous systems.

Well, this year's RailsConf made me happy including quite a few talks which had this kind of point of view, I'll list them here for my and other's reference.

The Gilt Effect: Handling 1000 Shopping Cart Updates per second in Rails
This presentation was brilliant, the guys at Gilt have a very interesting business model which translates in a really interesting technological challenge.
They use a solutions which combines ruby, rails, java and postgres to manage extreme peaks of traffic, as a note they measure their peak traffic in fractions of the amazon.com ecommerce website traffic!
Since they run time limited sales they leverage very well the EC2 pay per use scheme reducing to the minimum the size of their cluster in quiet periods and expanding it when the sale is on and customer really fights to get hold of the items Gilt offers.
It was the same time I've seen the cloud architecture leveraged in a completely spike driven business.
It's a shame that the slides of thi presentation aren't available yet.

PWN Your Infrastructure: Behind Call of Duty: World at War
These guys run several communities dedicated to popular games, they've been successful and they had to manage to scale their apps, they deploy a single app on multiple servers and they tried some different deployment and monitoring tools, and what they've found out is... the simpler tools are the best for the job they use shell scripting when shell scripting is enough, they deploy with chef to manage their deployment process and monit monit for monitoring. They developed an internal app to keep all the configuration together and aggregate monitoring infos.
I really enjoyed the "use the simpler tool for the job" philosophy they embraced, very hard core linux geek stuff. By the way the only piece of architecture they're looking to change is nfs, used to keep in sync their ruby/rubygems stack across machines.
Slides are on the railscon website

Building a Mini-Google: High-Performance Computing in Ruby Interesting talk very CS oriented, the speaker went through the publicly available docs for google page rank algorithm and in the presentations shows the theory behind it and the ruby code to actually implement it, very high quality content for a 45 mins talk.
Slides are on the railscon website

Confessions of a PackRat
I've lost this one, can't find the slides on line.

Rube Goldberg Contraptions, Building Scalable Decoupled Web Apps and Infrastructure with Ruby Really interesting talk, presenting a bunch of interesting technologies and solutions at once: deployment with chef, computer cloud coordination and administration with nanite demoed live controlling all the notebooks in the room. Presentation on slideshare but go and check out nanite on github now!

Art of the Ruby Proxy for Scale, Performance, and Monitoring
Missed this one, but slides are available on the railsconf website and some excerpt can be found on the speaker's blog. Learn how to put small ruby proxies in front of your services to achieve better performances, adding load balancing, filtering and other really neat tricks using just ruby, EventMachine and other bits.

It's Not Always Sunny In the Clouds: Lessons Learned
Interesting talk about the problems of deploying your app on AWS infrastructure, really practical tips and tricks to run your app in the cloud, slides on
railsconf website

%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs Ok this was my talk so I'll skip any comment slides are on slideshare.

This is the end of this post railsconf wrap up, I hope it's been interesting and please leave comment if you find around the missing slides