YAMI4 Tip #4 - Truly Non-Blocking Send

YAMI4 is a messaging solution that offers asynchronous API for its most fundamental messaging operations.

The word "asynchronous" means that the user code does not have to deal with communication-related delays on its main execution path and instead all the messaging work is done in background. This is mostly important for the send operation, which is typically performed after initialization phase, frequently and as part of some other sequence of commands. The user code can then proceed with other activities, without any need to wait for the physical message transmission, message reply, etc. (although blocking wait for subsequent events can be achieved easily if necessary, just as in example programs).

The fact that messages can be sent asynchronously has many benefits:

• Messages can be sent from GUI event handlers without any risk of blocking them, which would lead to GUI freezing.
• Messages can be sent to large number of different destinations without introducing unnecessary delays between receivers.
• Messages can be sent from callback functions that are supposed to execute rapidly - for example when the callbacks themselves result from asynchronous notifications from other YAMI4 actions or even from other asynchronous frameworks and libraries.

In all these cases it is important that the message send operation is non-blocking and does not introduce any delays other than those necessary to modify internal data structures.

The problem is - the message send operation in YAMI4 is non-blocking only when the following two conditions are met:

• The communication channel is already open - otherwise the YAMI4 agent will attempt to create the new connection and this is a blocking operation (there is a configurable timeout on the connection process, but it is still formally blocking), and
• the outgoing message queue is not filled up and there is a place in the queue to put new message.

The first problem can be solved with a separate call to open_connection - such a call can be performed during initialization phase, that is, outside of the time-critical loop or outside of asynchronous callbacks that are not supposed to block. Once the connection is created, the subsequent send operation will not have to create it and the only thing that will remain is to put new message in the existing queue.
Note that the send function should be called with auto_connect parameter set to false, so that broken communication channel is reported immediately instead of attempting any reconnection sequence.

The second issue, when the outgoing message queue is already filled up, can have several reasons, one of them being that the message receipient is not quick enough in consuming the messages and they have temporarily piled up. Without any other action on the sender side the send operation will block, waiting for the queue to be consumed first to make place for new messages. This is a natural flow-control mechanism that allows to implement stable producer-consumer scenarios, but in the case of truly non-blocking requirements it should not be relied upon. Instead, the user can inspect the size of the queue before pushing any new message. The pseudocode for this can be:

std::size_t current_level;
std::size_t high_water_mark;
std::size_t low_water_mark;

agent.get_outgoing_flow_state(current_level, high_water_mark, low_water_mark);
if (current_level < high_water_mark)
{
    // OK, there is a place in the queue,
    // this operation will not block:
    
    bool auto_connect = false;
    
    agent.send(..., auto_connect);
}
else
{
    // the queue is already full
    // ...
}

In other words, there are two features of the YAMI4 library that make it easier to write simple programs and that are appropriate most of the time:

• The send operation will try to create a new connection if it does not exist already.
• The send operation will block if the outgoing queue is full, so that the sender can slow down and adapt its average speed to be in line with the receiver.

Both features are useful, but they can become a problem if a truly non-blocking send is expected. In such a case these features have to be taken care of with separate open_connection, auto_connect and get_outgoing_flow_state facilities.

Previous tip: Shallow Parameters

Next tip: Let Me Think