Component Interactions

SMQ/S Broker Interactions

The SMQ/S Broker uses a named pipe on the local filesystem to rapidly buffer messages for relay to an SMQ/S instance.

The SMQ/S Broker uses an SQLite3 file-backed database to store a local queue consisting of message references requested by the Synapse Consumer.

The SMQ/S Broker uses the same SQLite3 database to store a table representing the pool of participating SMQ/S instances, their addresses, and their availability ratings.

Tenta Producer

Receives messages from the Tenta Producer as HTTP POST requests whose payloads consist of a JSON object, with a String field for the topic, and an array field whose elements represent messages for that topic to queue. After receipt, the SMQ/S thread processing the POST request breaks this into individual message objects and buffers each message object in a named pipe demarcated by newline.

For the duration of the SMQ/S Broker service’s lifecycle, a user-configured number of SMQ/S Broker threads pull message objects from the named pipe and relay these message objects to the SMQ/S instance in the pool with the highest availability rating. After this is done, the thread then forms a JSON object with fields representing the target SMQ/S instance ID, the message index, and topic. That object is a message reference. The message reference is added persistently to a local queue Failures are logged and raw message objects that failed to process are dumped to a separate file.

SMQ/S

Receives an HTTP POST request from SMQ/S to join the SMQ/S pool, with a payload of a JSON object consisting of a single field representing that SMQ/S instance’s instance ID. The SMQ/S Broker thread handling the request will then update a local table that contains rows representing SMQ/S instances, their network addresses, and their availability ratings.

A background thread will periodically poll the pool of SMQ/S instances and update availability ratings at user-defined intervals.

Receives an HTTP POST request from SMQ/S to drop reference messages with a payload of a JSON object consisting of an array of strings representing message indices.

SMQ/S Broker

Receives an HTTP POST request from other SMQ/S Brokers to join the SMQ/S Broker pool, with a payload of a JSON object consisting of a single field representing that SMQ/S Broker’s instance ID. The SMQ/S Broker

Receives an HTTP GET request from other SMQ/S Brokers as part of a syncing mechanism. It will respond with a queue dump of the reference messages as a JSON object consisting of an array of strings representing reference messages.

Periodically makes an HTTP GET request to all other SMQ/S Brokers in the SMQ/S Broker pool as part of a syncing mechanism. As it requests all the reference messages in each pool member’s local queue, it….

Synapse Consumer

Tenta Producer Interactions

Synapse Consumer Interactions

Transform Rules Engine Interactions

Relational Conversion Engine Interactions