Atoti Data Connectors Overview

This is the documentation for the Atoti Data Connectors library, with a section for users and a section for developers for both the Data Load Controller and Data Extract Engine libraries.

What is Atoti Data Connectors?

Atoti Data Connectors is a scalable, standard service to handle data loading/extraction within an Atoti Server application. This abstracts specific data loading/extraction implementations away from the client. Atoti Data Connectors sits on top of an Atoti Server application. It contains target-specific modules that each yield a JAR. Developers can pick and choose to include the artifacts they need in their POM file.

Requirements

Atoti Server Version

The Atoti Data Connectors JARs rely on local Atoti Server repositories. We achieve this by marking all Atoti Server dependencies as <scope>provided</scope> within Atoti Data Connectors POM files. This allows the Atoti Server dependencies to be inferred from the local file system and to flexibly use different minor versions. Atoti Server upgrades are easier as there is no need to rebuild/upgrade Atoti Data Connectors.

Configuration Classes

An example of including these configuration classes can be seen as follows:

    @Configuration
    @Import({
        // Required Configurations for Tracing and APM
        TracingConfig.class,
        BraveTracingConfig.class
    })
    public class ApplicationConfig {
    }

Initial Data Load

• If you are executing any DLC tasks via Spring Beans during initialization of your application, you must ensure that the TracingConfig.TRACING_BEAN bean is created first using the @DependsOn annotation. Please see code sample below as an example:

    @Bean
    @DependsOn(value = {TracingConfig.TRACING_BEAN})
    public Void initialLoad() {
        // Load data here...
    }

Module Structure

Here we can see the Atoti Data Connectors module structure with all target modules (in blue).

Currently, Atoti Data Connectors supports the following targets for the DLC and DEE components. Targets are included in the following maven modules:

Users can import the target module required for either data Loading or Extraction by including the corresponding target’s module within their POM. Then the user will have access to both the Data Load Controller and Data Extract Engine components.

Here is an example of including Atoti Data Connectors for RabbitMq:

<dependency>
    <groupId>com.activeviam.io</groupId>
    <artifactId>data-connectors-rabbitmq</artifactId>
    <version>4.0.5-AP6.0</version>
</dependency>

REST Services

Services URL

Both the DLC and DEE components are found at the following REST URLs:

/connectors/rest/dlc/<REST-version>/
/connectors/rest/dee/<REST-version>/

REST Service Version

Atoti Data Connectors REST endpoints contain a version number. This version number helps remote users ensure they are using the correct REST contract. The REST version is appended to the end of the base REST endpoint for both DLC and DEE components. More on each component’s REST service can be found here: DLC REST API Example and DEE Requests.

The REST version can be discovered through the /versions/rest endpoint. The /versions/rest endpoint allows users to retrieve the REST version the DLC and DEE are on, along with other Atoti components.

Sending a GET request to the /versions/rest endpoint will return the following structure:

{
    "version": 1,
    "serverVersion": "6.0.x",
    "apis": {
        "/connectors/rest/dlc": {
            "versions": [
                {
                    "id": "1",
                    "restPath": "/connectors/rest/dlc/v1"
                }
            ]
        },
        "/connectors/rest/dee": {
            "versions": [
                {
                    "id": "1",
                    "restPath": "/connectors/rest/dee/v1"
                }
            ]
        },
        ...
        "activeviam/pivot": {
            "versions": [
                {
                    "id": "8",
                    "restPath": "/activeviam/pivot/rest/v8",
                    "wsPath": "/activeviam/pivot/ws/v8"
                }
            ]
        },
        ...
    }
}