Apache Tribes - Frequently Asked Questions

+ + &project; + + + Filip Hanik + Apache Tribes - Introduction + + + + + +

+ +

Apache Tribes is a group or peer-to-peer communcation framework that enables you to easily connect + your remote objects to communicate with each other. +

Import: org.apache.catalina.tribes.Channel
Import: org.apache.catalina.tribes.Member
Import: org.apache.catalina.tribes.MembershipListener
Import: org.apache.catalina.tribes.ChannelListener
Import: org.apache.catalina.tribes.group.GroupChannel
Create a class that implements: org.apache.catalina.tribes.ChannelListener
Create a class that implements: org.apache.catalina.tribes.MembershipListener
Simple class to demonstrate how to send a message: + + //create a channel + Channel myChannel = new GroupChannel(); + + //create my listeners + ChannelListener msgListener = new MyMessageListener(); + MembershipListener mbrListener = new MyMemberListener(); + + //attach the listeners to the channel + myChannel.addMembershipListener(mbrListener); + myChannel.addChannelListener(msgListener); + + //start the channel + myChannel.start(Channel.DEFAULT); + + //create a message to be sent, message must implement java.io.Serializable + //for performance reasons you probably want them to implement java.io.Externalizable + Serializable myMsg = new MyMessage(); + + //retrieve my current members + Member[] group = myChannel.getMembers(); + + //send the message + channel.send(group,myMsg,Channel.SEND_OPTIONS_DEFAULT); + +

+ Simple yeah? There is a lot more to Tribes than we have shown, hopefully the docs will be able + to explain more to you. Remember, that we are always interested in suggestions, improvements, bug fixes + and anything that you think would help this project. +

+ Note: Tribes is currently built for JDK1.5, you can run on JDK1.4 by a small modifications to locks used from the java.util.concurrent package. +

+ + +

+ Tribes is a messaging framework with group communication abilities. Tribes allows you to send and receive + messages over a network, it also allows for dynamic discovery of other nodes in the network.
+ And that is the short story, it really is as simple as that. What makes Tribes useful and unique will be + described in the section below.
+

+ The Tribes module was started early 2006 and a small part of the code base comes from the clustering module + that has been existing since 2003 or 2004. + The current cluster implementation has several short comings and many work arounds were created due + to the complexity in group communication. Long story short, what should have been two modules a long time + ago, will be now. Tribes takes out the complexity of messaging from the replication module and becomes + a fully independent and highly flexible group communication module.
+

+ In Tomcat the old modules/cluster has now become modules/groupcom(Tribes) and + modules/ha (replication). This will allow development to proceed and let the developers + focus on the issues they are actually working on rather than getting boggled down in details of a module + they are not interested in. The understanding is that both communication and replication are complex enough, + and when trying to develop them in the same module, well you know, it becomes a cluster :)
+

+ Tribes allows for guaranteed messaging, and can be customized in many ways. Why is this important?
+ Well, you as a developer want to know that the messages you are sending are reaching their destination. + More than that, if a message doesn't reach its destination, the application on top of Tribes will be notified + that the message was never sent, and what node it failed. +

+ +

+ I am a big fan of reusing code and would never dream of developing something if someone else has already + done it and it was available to me and the community I try to serve.
+ When I did my research to improve the clustering module I was constantly faced with a few obstacles:
+ 1. The framework wasn't flexible enough
+ 2. The framework was licensed in a way that neither I nor the community could use it
+ 3. Several features that I needed were missing
+ 4. Messaging was guaranteed, but no feedback was reported to me
+ 5. The semantics of my message delivery had to be configured before runtime
+ And the list continues... +

+ So I came up with Tribes, to address these issues and other issues that came along. + When designing Tribes I wanted to make sure I didn't lose any of the flexibility and + delivery semantics that the existing frameworks already delivered. The goal was to create a framework + that could do everything that the others already did, but to provide more flexibility for the application + developer. In the next section will give you the high level overview of what features tribes offers or will offer. +

+ +

+ To give you an idea of the feature set I will list it out here. + Some of the features are not yet completed, if that is the case they are marked accordingly. +

+ Pluggable modules
+ Tribes is built using interfaces. Any of the modules or components that are part of Tribes can be swapped out + to customize your own Tribes implementation. +

+ Guaranteed Messaging
+ In the default implementation of Tribes uses TCP for messaging. TCP already has guaranteed message delivery + and flow control built in. I believe that the performance of Java TCP, will outperform an implementation of + Java/UDP/flow-control/message guarantee since the logic happens further down the stack.
+ Tribes supports both non-blocking and blocking IO operations. The recommended setting is to use non blocking + as it promotes better parallelism when sending and receiving messages. The blocking implementation is available + for those platforms where NIO is still a trouble child. +

+ Different Guarantee Levels
+ There are three different levels of delivery guarantee when a message is sent.
+

IO Based send guarantee. - fastest, least reliable
+ This means that Tribes considers the message transfer to be successful + if the message was sent to the socket send buffer and accepted.
+ On blocking IO, this would be socket.getOutputStream().write(msg)
+ On non blocking IO, this would be socketChannel.write(), and the buffer byte buffer gets emptied + followed by a socketChannel.read() to ensure the channel still open. + The read() has been added since write() will succeed if the connection has been "closed" + when using NIO. +
ACK based. - recommended, guaranteed delivery
+ When the message has been received on a remote node, an ACK is sent back to the sender, + indicating that the message was received successfully. +
SYNC_ACK based. - guaranteed delivery, guaranteed processed, slowest
+ When the message has been received on a remote node, the node will process + the message and if the message was processed successfully, an ACK is sent back to the sender + indicating that the message was received and processed successfully. + If the message was received, but processing it failed, an ACK_FAIL will be sent back + to the sender. This is a unique feature that adds an incredible amount value to the application + developer. Most frameworks here will tell you that the message was delivered, and the application + developer has to build in logic on whether the message was actually processed properly by the application + on the remote node. If configured, Tribes will throw an exception when it receives an ACK_FAIL + and associate that exception with the member that didn't process the message. +

+ You can of course write even more sophisticated guarantee levels, and some of them will be mentioned later on + in the documentation. One mentionable level would be a 2-Phase-Commit, where the remote applications don't receive + the message until all nodes have received the message. Sort of like a all-or-nothing protocol. +

+ Per Message Delivery Attributes
+ Perhaps the feature that makes Tribes stand out from the crowd of group communication frameworks. + Tribes enables you to send to decide what delivery semantics a message transfer should have on a per + message basis. Meaning, that your messages are not delivered based on some static configuration + that remains fixed after the message framework has been started.
+ To give you an example of how powerful this feature is, I'll try to illustrate it with a simple example. + Imagine you need to send 10 different messsages, you could send the the following way: + + Message_1 - asynchronous and fast, no guarantee required, fire and forget + Message_2 - all-or-nothing, either all receivers get it, or none. + Message_3 - encrypted and SYNC_ACK based + Message_4 - asynchronous, SYNC_ACK and call back when the message is processed on the remote nodes + Message_5 - totally ordered, this message should be received in the same order on all nodes that have been + send totally ordered + Message_6 - asynchronous and totally ordered + Message_7 - RPC message, send a message, wait for all remote nodes to reply before returning + Message_8 - RPC message, wait for the first reply + Message_9 - RPC message, asynchronous, don't wait for a reply, collect them via a callback + Message_10- sent to a member that is not part of this group + + As you can imagine by now, these are just examples. The number of different semantics you can apply on a + per-message-basis is almost limitless. Tribes allows you to set up to 28 different on a message + and then configure Tribes to what flag results in what action on the message.
+ Imagine a shared transactional cache, probably >90% are reads, and the dirty reads should be completely + unordered and delivered as fast as possible. But transactional writes on the other hand, have to + be ordered so that no cache gets corrupted. With tribes you would send the write messages totally ordered, + while the read messages you simple fire to achieve highest throughput.
+ There are probably better examples on how this powerful feature can be used, so use your imagination and + your experience to think of how this could benefit you in your application. +

+ Interceptor based message processing
+ Tribes uses a customizable interceptor stack to process messages that are sent and received.
+ So what, all frameworks have this!
+ Yes, but in Tribes interceptors can react to a message based on the per-message-attributes + that are sent runtime. Meaning, that if you add a encryption interceptor that encrypts message + you can decide if this interceptor will encrypt all messages, or only certain messages that are decided + by the applications running on top of Tribes.
+ This is how Tribes is able to send some messages totally ordered and others fire and forget style + like the example above.
+ The number of interceptors that are available will keep growing, and we would appreciate any contributions + that you might have. +

+ Threadless Interceptor stack + The interceptor don't require any separate threads to perform their message manipulation.
+ Messages that are sent will piggy back on the thread that is sending them all the way through transmission. + The exception is the MessageDispatchInterceptor that will queue up the message + and send it on a separate thread for asynchronous message delivery. + Messages received are controlled by a thread pool in the receiver component.
+ The channel object can send a heartbeat() through the interceptor stack to allow + for timeouts, cleanup and other events.
+ The MessageDispatchInterceptor is the only interceptor that is configured by default. +

+ Parallel Delivery
+ Tribes support parallel delivery of messages. Meaning that node_A could send three messages to node_B in + parallel. This feature becomes useful when sending messages with different delivery semantics. + Otherwise if Message_1 was sent totally ordered, Message_2 would have to wait for that message to complete.
+ Through NIO, Tribes is also able to send a message to several receivers at the same time on the same thread. +

+ Silent Member Messaging
+ With Tribes you are able to send messages to members that are not in your group. + So by default, you can already send messages over a wide area network, even though the dynamic discover + module today is limited to local area networks by using multicast for dynamic node discovery. + Of course, the membership component will be expanded to support WAN memberships in the future. + But this is very useful, when you want to hide members from the rest of the group and only communicate with them +

+ +

+ I hope you have enjoyed this short introduction to Tribes. You can download Tribes here + or you can download Tribes including javadoc and this doc +

+ + +

+ + + + +