CSV loading topic

Configuration properties

Properties for a CSV Loading Topic are defined by name in the dlc.csv.topics namespace, and are picked up as configurations for the DLC.

Key	Type	Default	Description
file-pattern	`String`		Path matcher.
parser	`CsvParserProperties`	Default Parser.	CSV Parser
channels	`Set<Channel>`	A channel is created using the topic’s name as the name of the target.	Channels.
restrict-to-sources	`Set<String>`		Sources to which the topic is restricted. See source to topic matching.
restrict-from-sources	`Set<String>`		Sources from which the topic is restricted. See source to topic matching.

YAML example

This configuration does not specify a target for the data or the data format.

If the topic is not named the same as the store, the target must be explicitly specified in the channel of the topic.

The target is implicitly the Atoti table named “Trades”.
The format (column order and how input fields are parsed) implicitly comes from the columns and types of the targeted Atoti table, in this case “Trades”.

dlc:
  csv:
    topics:
      Trades:
        file-pattern: glob:trades*.csv

JSON example

CSV Topics can also be provided as an override in a DLC LOAD request.

{
  "Trades": {
    "filePattern": "glob:different_than_trades*.csv"
  }
}

Java configuration

CsvTopicDescription Spring Beans are picked up as configurations for the DLC.

Parameter	Required	Type	Default	Description
name	Y	`String`		Name of the topic.
filePattern		`String`		Path matcher.
parser		`CsvParserDescription`	Default Parser.	CSV Parser.
channels		`Set<ChannelDescription>`	A channel is created using the topic’s name as the name of the target	Channel Descriptions which belong to the topic.
restrictToSources		`Set<String>`		Sources to which the topic is restricted. See source to topic matching.
restrictFromSources		`Set<String>`		Sources from which the topic is restricted. See source to topic matching.

Java example

This configuration does not specify a target for the data or the data format.

If the topic is not named the same as the store, the target must be explicitly specified in the channel of the topic.

The target is implicitly the Atoti table named “Trades”.
The format (column order and how input fields are parsed) implicitly comes from the columns and types of the targeted Atoti table, in this case “Trades”.

@Bean
public CsvTopicDescription tradesTopic() {
    return CsvTopicDescription.builder("Trades", "glob:trades*.csv")
            .build();
}

Implicit CSV loading topics

Implicit CSV loading topics allow specific files to be loaded into a datastore table without configuring a topic. By default, implicit topics are enabled. They can be disabled by setting the dlc.csv.implicit-load-topics-enabled property to false. When using implicit CSV loading topics:

The topic name must match the name of an existing datastore table.
The files to load must be specified via the files scope key.

Example

{
  "operation": "LOAD",
  "topics": ["TradeTable"],
  "scope": {
    "files": ["path/to/file1.csv", "path/to/file2.csv"]
  }
}

Operations

Request API

Configuration

Components

Functional

Extensions

Configuration properties

YAML example

JSON example

Java configuration

Java example

Implicit CSV loading topics

Example

​Configuration properties

​YAML example

​JSON example

​Java configuration

​Java example

​Implicit CSV loading topics

Example

Configuration properties

YAML example

JSON example

Java configuration

Java example

Implicit CSV loading topics