Web 2.0 Archives
Eventing Data with RabbitMQ and Riak
There's been lots of interest in my new Riak exchange type for RabbitMQ and its cousin, the Riak postcommit hook for RabbitMQ. They're two sides of the same coin but can be used independently.
I've had some questions about whether they will do this that or the other, so I thought a blog post was in order that went into a little more detail about how to use these utilities.
Installation instructions are the their respective READMEs (here and here) so I won't repeat that information in this post.
Over the last few weeks I've been mercilessly hacking on the internal Grails GORM code and Spring Data support for MongoDB to create a next-generation, rich object mapper for NoSQL stores. I've also been toying with the idea of replacing RabbitMQ's message persister with a NoSQL-based backend. Why in the world would you do that, you ask.
That's a legitimate question, so I thought I'd lay out in a little more detail my thoughts on the matter in an attempt to stimulate some debate on the relative merits of this new approach to messaging and data.
I thought I would take a few minutes and put a little meat on the bones regarding the Tomcat session manager I recently uploaded to GitHub that's backed by the Riak Key/Value datastore.
I've been playing around with Tomcat session managers for the last year or so. When I started getting involved in private cloud architectures, one of the major obstacles I ran into was that the serious session manager that would provide a sticky-session-less configuration for Tomcat or SpringSource tcServer instances was a commercial offering. Sticky sessions are the bane of a private cloud's existence because they demand certain servers be available in specific configurations. The default clustering configuration for handling failover is sorely lacking in functionality, robustness, and scalability.
Thanks to all the great help from the SpringSource and Basho folks over the last several weeks. I'm happy to start talking more in-depth about what I've been tweeting about my Grails GORM support for Riak.
The Problem
I started skipping down this garden path because I wanted to be able to easily bridge the Groovy world in which we develop our new internal, browser-based applications, and the more old-timey data sitting on our AS/400. I tried using the Java client for Riak to do some basic and low-level work and it just wasn't exactly what I was looking for. I'm a bit of a perfectionist, so I'm notoriously difficult to please. :)
I decided that working on an integration layer between Riak and Java using the great tools introduced in Spring 3.0 (and particularly in Spring 3.0.5 with the conversion abstractions) like the RestTemplate and the HttpMessageConverter would be the best approach. Not finding anything even being talked about more than pure vaporware, I simply jumped in feet first and started throwing down code as fast as I could.
Since I'm using ZooKeeper to coordinate my Groovy-based Map/Reduce framework, I've written a helper class that should be able to be used by any Groovy code to make interacting with ZooKeeper a little more Groovylicious. Here's an example of how to make working with ZooKeeper easier within a Grails app:
class MyController {
// Auto-injected by the plugin
def zooKeeper
def action = {
zooKeeper.onDataChanged = { evt ->
// We're interested in data changing while we're watching
println "data changed: ${evt}"
}
if (!zooKeeper.exists("/my/node/parent")) {
zooKeeper.createPersistentNodeAndParents("/my/node/parent")
}
node = zooKeeper.createPersistentNode("/my/node/parent/child")
node.data = 12345
}
}
The thing I like about this versus using the API directly is that I'm using Closures everywhere I can. I've implemented some core Watchers that call the closures you set on the "zooKeeper" object. All methods also take either a plain String or GString, so you can use the full power of Groovy strings (including embedding variable references for building up paths).
There is a small wrapper class for dealing with a Node directly, rather than by calling getData() and setData() (though you're certainly free to call those methods directly).
Automatic serialization and deserialization occur when setting and getting data. You don't need to worry about it yourself. Any Serializable class can be set as the data on a ZooKeeper node.
The code, as always, is Apache 2.0 licensed and available on GitHub:
http://github.com/jbrisbin/grails-zookeeper-plugin