Inspirel banner

Programming Distributed Systems with YAMI4

7.2.2 Clustered Installations And Forwarding

The YAMI4 broker was intentionally build with small resource utilization in mind. This can allow lean installations on hardware with limited resources, but will be also helpful in architecting more complex systems with many brokers.

There are several reasons to set up a clustered broker - these include higher reliability, load balancing and higher throughput.

Load balancing and reliable installations can be achieved with several brokers, installed on the same or on different machines. Several messaging schemes can be considered:

Publishers and subscribers can be partitioned and use distinct brokers for distinct sets of tags - this is a natural partitioning of the whole messaging service, and the simplest way to share load across several nodes:

Publishers can send their messages to all brokers in the cluster, but subscribers can be partitioned into groups and subscribe each to only one broker. In this case the messaging load can be shared between brokers in order to serve large number of subscribers:

Publishers can send their messages to all brokers in the cluster and subscribers can connect to more than one broker as well to achieve redundancy, so that even in the case of single broker failure the message delivery is sustained. This scheme requires detection of duplicate messages at the subscriber side:

Publishers can send their messages to only one broker in the cluster, whereas subscribers can subscribe to all brokers with the same tags. In this scheme it is the publisher who should detect the broker failure and as a failover scenario repeat the message using another broker. The overall message load can be smaller than in the previous case, but at the price of potentially lost messages, as the message delivery is asynchronous and a failure in the mean time can prevent the broker from completing the request:

Clustered installations can be further elaborated with the help of automated message forwarding. The broker can have arbitrary number of permanent forward channels set up in its configuration file, which allow to push messages to another broker based on a given tag filter. The syntax and meaning of forwarding channels is described in further subsection on configuration parameters.