Configuration Flow

The following sections provide an overview of the different ways to define configurations and their order of precedence.

Configuration Flow

Configurations defined in Java (Description Spring Beans) have the lowest precedence and can be overridden by configurations defined in configuration properties.

In addition, loading Topic Configurations can be overridden at request time.

Startup Configuration

Configuration Properties

Configuration Properties get converted to Description objects on startup. These Configurations take precedence over Java Configuration.

Yaml Example

dlc:
  csv:
    local:
      sources:
        source1:
          root-base-dir: /base-dir
    topics:
      topic1:
        file-pattern: glob:trades*.csv

Java Configuration

Description Spring Beans are picked up as configuration.

Java Example

@Bean
public LocalCsvSourceDescription source1() {
    return LocalCsvSourceDescription.builder("source1", "/base-dir").build();
}

@Bean
public CsvTopicDescription topic1(NamedEntityResolverService namedEntityResolver) {
    return CsvTopicDescription.builder("topic1", "glob:trades*.csv")
            .build();
}

Request Time

Loading Topic Overrides

At request time, you can also override the configurations defined at startup for a loading topic by providing configuration properties in your request.

These override some or all of the configurations which belong to a loading topic which was configured at startup.

Example

{
  "operation": "LOAD",
  "csvTopicOverrides": {
    "trades": {
      "filePattern": "glob:alternative_trades*.csv"
    }
  }
}