Skip to main content

​
Configuration properties

Properties for a CSV Loading Topic are defined by name in the dlc.csv.topics namespace, and are picked up as configurations for the DLC.
KeyRequiredTypeDefaultDescription
file-patternStringPath matcher.
parserCsvParserPropertiesDefault Parser.CSV Parser
channelsSet<Channel>A channel is created using the topic’s name as the name of the target.Channels.
restrict-to-sourcesSet<String>Sources to which the topic is restricted. See source to topic matching.
restrict-from-sourcesSet<String>Sources from which the topic is restricted. See source to topic matching.

​
YAML example

This configuration does not specify a target for the data or the data format.
If the topic is not named the same as the store, the target must be explicitly specified in the channel of the topic.
  • The target is implicitly the Atoti table named “Trades”.
  • The format (column order and how input fields are parsed) implicitly comes from the columns and types of the targeted Atoti table, in this case “Trades”.
dlc:
  csv:
    topics:
      Trades:
        file-pattern: glob:trades*.csv

​
JSON example

CSV Topics can also be provided as an override in a DLC LOAD request. 
{
  "Trades": {
    "filePattern": "glob:different_than_trades*.csv"
  }
}

​
Java configuration

CsvTopicDescription Spring Beans are picked up as configurations for the DLC.
ParameterRequiredTypeDefaultDescription
nameYStringName of the topic.
filePatternStringPath matcher.
parserCsvParserDescriptionDefault Parser.CSV Parser.
channelsSet<ChannelDescription>A channel is created using the topic’s name as the name of the targetChannel Descriptions which belong to the topic.
restrictToSourcesSet<String>Sources to which the topic is restricted. See source to topic matching.
restrictFromSourcesSet<String>Sources from which the topic is restricted. See source to topic matching.

​
Java example

This configuration does not specify a target for the data or the data format.
If the topic is not named the same as the store, the target must be explicitly specified in the channel of the topic.
  • The target is implicitly the Atoti table named “Trades”.
  • The format (column order and how input fields are parsed) implicitly comes from the columns and types of the targeted Atoti table, in this case “Trades”.
@Bean
public CsvTopicDescription tradesTopic() {
    return CsvTopicDescription.builder("Trades", "glob:trades*.csv")
            .build();
}

​
Implicit CSV loading topics

Implicit CSV loading topics allow specific files to be loaded into a datastore table without configuring a topic. By default, implicit topics are enabled. They can be disabled by setting the dlc.csv.implicit-load-topics-enabled property to false. When using implicit CSV loading topics:
  • The topic name must match the name of an existing datastore table.
  • The files to load must be specified via the files scope key.
Example
{
  "operation": "LOAD",
  "topics": ["TradeTable"],
  "scope": {
    "files": ["path/to/file1.csv", "path/to/file2.csv"]
  }
}