Messaging consumers
Messaging consumers are responsible for connecting to the remote messaging system, listening constantly for new data, and consuming data when available. The consumers are created when the request to start “listening” to the remote topic is submitted at runtime. The underlying Kafka client consumers are created and managed by the messaging consumer implementation -KafkaMessageConsumer.
The remote topics are polled for records when the consumer thread begins.
Messaging transaction controller tasks
The messages collected in the consumers are applied to the Datastore in bulk operations. The cadence of these transactions can be tuned via theTxControllerTaskDescription.