Exposing AS/400 Service Programs via RabbitMQ and Java
One of the things we've been doing more and more at my day job (I work at the world's largest Pizza Hut franchisee, NPC International) is integrating the work we're doing in Java with the folks over on the RPG and AS/400 side of things (I don't care what IBM chooses to call it this year; to me it will always be the AS/400, and that's what I call it). The integration goes both ways: from RPG into Java and from Java back into RPG. In order to reduce complexity and implement a hillbilly version of Don't Repeat Yourself, we've decided to expose the functionality locked inside our service program functions to other code, including Java and C++. To do that, we're dispatching messages through the RabbitMQ message broker.
I realize there's no one way to do this. My way may not even be the best (I suspect its not). But I'd like to outline one way to expose RPG service programs to code written in other languages by wiring the two together with asynchronous messaging.
The RPG Side
Unfortunately, DRY can't be completely eliminated on the RPG side of things. That's just not the RPG way. :) If you have a service program that exports a function like so (maybe this header is called CALIB/CA9050H):
D get_example_value...
D PR 15P 5
D company 3P 0 VALUE
D store_number 5P 0 VALUE
D bus_date 8P 0 VALUE
...you can't just wrap a Java native method around that service program and go. The problem is in the way the export works inside the service program that implements this function. In CALIB/CA9050SRV, if you wanted to expose this function to other RPG code, you simply provide your export line like normal:
P get_example_value...
P B export
D get_example_value...
D PI 15P 5
D company 3P 0 VALUE
D store_number 5P 0 VALUE
D bus_date 8P 0 VALUE
d sql_cmd s 1028a
/FREE
sql_cmd = 'SELECT exval FROM MYLIB.MYFILE ' +
...
...and implement your function. But to call this function from Java (by declaring a "native" method in your Java class) you need a different header definition. I've not found a way to export the same function two different ways, so I had to create a parallel service program (CALIB/CA9050JSRV) that makes my function callable from Java (this header file would be CALIB/CA9050JH), and in the process, rename the function to something more Java-centric rather than the C++-influenced lowercase and underscores:
D getExampleValue...
D PR 8F ExtProc(*JAVA:
D 'com.npci.cloud.iseries.+
D CALIB.CA9050JSRV':
D 'getExampleValue')
D company 10I 0 VALUE
D store_number 10I 0 VALUE
D bus_date 10I 0 VALUE
Now I can implement my service program, EXPORTing it for use by Java by mapping this native ILE function to a Java class (here's the full member, CALIB/CA9050JSRV):
P getExampleValue...
P B EXPORT
D getExampleValue...
D PI 8F
D company 10I 0 VALUE
D store_number 10I 0 VALUE
D bus_date 10I 0 VALUE
D rate S 15P 5
/FREE
rate = get_example_value(
company : store_number : bus_date
);
return rate;
/END-FREE
P getExampleValue...
P E
All this does is delegate the actual logic to the original CALIB/CA9050SRV function, which can be used by RPG code, while this service program can be used by Java.
The Java Side
Now all that's left is to implement our Java class, declaring a native method that ties that class to this service program. This is an example which includes some extra features I've written to make working with service programs easier:
public class CA9050JSRV extends ServiceProgram {
@Procedure(srvpgm = "CA9050JSRV", name = "getexample")
public native float getExampleValue(int cono, int stono, int busdate);
@Procedure(srvpgm = "", name = "test")
public float getExampleValue(int cono, int stono, int busdate) {
return 0.8f;
}
}
I wrote an abstract base class, ServiceProgram, which is responsible for executing the System.loadLibrary() call (with the value from the annotation). Since the code that does the introspection retains this annotation during runtime, I can interrogate this annotation for the right service program to tie this class to and export that function under a shorter name, which is what is exposed to the incoming messages.
The Messaging Side
Now that we've exposed our RPG function through a Java native method, we can work on calling that Java method via introspection, triggered by a RabbitMQ message.
The code to consume messages is fairly straightforward. I simply use a QueueingConsumer (from the RabbitMQ Java API) to dump messages into a java.util.concurrent.BlockingQueue, from which various message handlers pull, based on the message type. I don't want to muddy the waters too much here--I'm saving some of this for later blog posts. But suffice it to say I have a MessageRouter class that is managed by the Spring Framework, runs on the AS/400 in its own subsytem, and listens for incoming RabbitMQ messages. If it receives a ServiceProgram invocation message, it dispatches it to the appropriate BlockingQueue and the message handler decodes the incoming JSON data. An example might look like this:
{
"program": "ca9050",
"procedure": "getexample",
"params": [ 1, 1134, 20100413 ]
}
In order to get from this JSON data to invoking the above Java method, my MessageHandler has to know what service programs and procedures it's allowed to invoke. If the method isn't annotated with a @Procedure annotation, the invoker knows nothing about it. It also doesn't respond to class names (another security feature) but arbitrary keys set up by the developer.
Maybe it's best to show the snippet of Spring XML that configures the MessageHandler:
<bean id="srvpgmMessageHandler" class="com.npci.cloud.ServiceProgramMessageHandler" scope="prototype">
<property name="servicePrograms">
<map>
<entry key="ca9050">
<bean class="com.npci.cloud.iseries.calib.CA9050JSRV"/>
</entry>
</map>
</property>
</bean>
You'll notice that the JSON data specifies the key referencing the right bean, rather than specifying a generic class name. This is intentional and security-minded. Although our internal network is secure and disconnected from the rest of the world by various routing and firewalling devices, it's still not a good idea to completely open up a service to invoking of arbitrary objects. PCI and SOX compliance auditors would have a cow, if that were the case! :)
The ServiceProgramMessageHandler calls a method on the ServiceProgram base class which is responsible for handing back the right java.lang.reflect.Method which it can invoke, passing the params from the JSON data.
The Return
Our services communicate with one another via JSON, so the output of this native method call is sent back to the requestor's queue, encoded in JSON format:
{
"result": "success",
"total": 1,
"data": [ .8 ]
}
This allows us to keep from repeating critical pieces of business logic that reside in the bowels of the AS/400 without resorting to over-priced IBM software. Who isn't for that? :)
The MessageRouter infrastructure was written on company time first, so I can't simply release it as OpenSource. I can, however, give some insight into how we're solving these problems internally and maybe inspire others to pick up the torch and write truly OpenSource solutions.
In a future post I'll discuss how we're batching messages and sending them zip-compressed to save on precious WAN bandwidth.
