☰
Navigation :

Controller Configuration

Any of the source dependencies must be included in your project’s POM. You can check Data Connectors Requirements to ensure your project meets the minimum ActivePivot version requirements. You must also include one dependency for each source you wish to add. All supported sources can be found at Data Connectors Module Structure.

Here is an example of importing Data Connectors for the CSV and JDBC sources:

    <!-- Data Connectors CSV Source -->
    <dependency>
        <groupId>com.activeviam.io</groupId>
        <artifactId>data-connectors-csv</artifactId>
        <version>${dataconnectors.version}</version>
    </dependency>
    <!-- Data Connectors JDBC Source -->
    <dependency>
        <groupId>com.activeviam.io</groupId>
        <artifactId>data-connectors-jdbc</artifactId>
        <version>${dataconnectors.version}</version>
    </dependency>

Please contact your project team if you do not have the Data Connectors JAR file. It needs to be imported in your maven repository.

Next, once the required spring configurations for data source type (CSV, JDBC or Kafka/RabbitMQ) have been configured (examples further below), they need to be registered with the DataLoadController. The following snippets show you how to do that:

Create DataLoadController Bean

@Bean(destroyMethod = "close")
public IDataLoadController dataLoadController() {

    final IDataLoadController controller = new DataLoadController(datastoreConfig.datastore(), null);                
    registerJdbcSourceAndChannels(controller);    		            

    return controller;
}

Register data source and channels

Please read the DLC Data Source Configuration section to learn more about how to configure your Source.

// Example configuring and registering CSV Source and CSV Messaging Channels
private void registerCsvSourceAndChannels(final IDataLoadController controller) {
    final DlcCSVMessageChannelFactory channelFactory = new DlcCSVMessageChannelFactory<>(bulkLoadCsvSource(sourceName), datastoreConfig.datastore());
    
    // Get source from source's config.
    controller.registerSource(csvSourceConfig.csvSource());
    controller.registerChannel(
            channelFactory.createChannel(
                    CsvSourceConfig.CSV__TOPIC_PRODUCT,
                    DatastoreNames.PRODUCT_STORE_NAME),
            Arrays.asList(DatastoreNames.PRODUCT_STORE_NAME),
            new ProductScopeToRemoveWhereConditionConverter()
            );
}

// Example configuring and registering non CSV Sources and CSV Messaging Channels (in this case JDBC)
private void registerJdbcSourceAndChannels(final IDataLoadController controller) {
    final TupleMessageChannelFactory channelFactory = jdbcSourceConfig.jdbcMessageChannelFactory();

    // Get source from source's config.
    controller.registerSource(jdbcSourceConfig.jdbcSource());
    controller.registerChannel(
            channelFactory.createChannel(
                    JdbcSourceConfig.JDBC_TOPIC__PRODUCT,
                    DatastoreNames.PRODUCT_STORE_NAME),
            Arrays.asList(DatastoreNames.PRODUCT_STORE_NAME),
            new ProductScopeToRemoveWhereConditionConverter()
            );
}

Remove-where condition for UNLOAD

public class ProductScopeToRemoveWhereConditionConverter
        implements IDataLoadController.IScopeToRemoveWhereConditionConverter {

    @Override
    public ICondition apply(
            final String store,
            final Map<String, Object> scope){

        final LocalDate cobDate = LocalDate
                .parse((String) scope.get(DataLoadControllerRestService.SCOPE_KEY__COB_DATE));
        ICondition cobCond = BaseConditions.Equal(COB_DATE, cobDate);
        ICondition productCond = BaseConditions.Not(BaseConditions.Equal(BASE_STORE_PRODUCT_NAME, "N/A"));

        return BaseConditions.And(cobCond, productCond);
    }
}

Register topic alias

You may group several topics together under an alias, so that the client / data orchestrator can request to load a single topic alias instead of providing each and every topic name to load in the request. This is useful when submitting a single request to load all the topics.

For example, you may configure an alias “ALL” that groups all the topics together.

protected void registerTopicAliases(IDataLoadController dataLoadController) {
    dataLoadController.registerTopicAlias(TOPIC__ALL, Arrays.asList(TOPIC__FX_DATA, TOPIC__ORGANISATION_DATA, TOPIC__TRADE_BOOKING_DATA, TOPIC__TRADES));
    }
search.js