Avro Loading Topic
Configuration Properties
Properties for an Avro Loading Topic are defined by name in the dlc.avro.topics
namespace,
and are picked up as configurations for the DLC.
Key | Required | Type | Default | Description |
---|---|---|---|---|
file-pattern | String |
Path matcher. | ||
schema-path | String |
Path to schema. | ||
columns | List<String> |
Columns will implicitly come from the store which is being published to. | The order of columns in the file. If a column name matches a datastore column, its parser will be implied. For columns that do not match a datastore column, you must provide a parser in the channel as a Custom Field. Columns without a matching datastore column or parser are skipped. | |
channels | Set<Channel> |
A channel is created using the topic’s name as the name of the target. | Channels. | |
restrict-to-sources | Set<String> |
Sources to which the topic is restricted. See source to topic matching. | ||
restrict-from-sources | Set<String> |
Sources from which the topic restricted. See source to topic matching. |
YAML Example
This configuration does not specify a target for the data.
note
If you do not name your topic the same as your store you will have to explicitly specify the target in the channel of the topic.
- The target is implicitly the Atoti table named “Trades”.
- The format comes from the schema file specified in the configuration.
dlc:
avro:
topics:
Trades:
file-pattern: glob:trades*.avro
schema-path: schemas/schema.avsc
Java Configuration
AvroTopicDescription
Spring Beans are picked up as configurations for the DLC.
Parameter | Required | Type | Default | Description |
---|---|---|---|---|
name | Y | String |
Name of the topic. | |
filePattern | String |
Path matcher. | ||
schemaPath | String |
Path to schema. | ||
columns | List<String> |
Columns will implicitly come from the store which is being published to. | The order of columns in the file. If a column name matches a datastore column, its parser will be implied. For columns that do not match a datastore column, you must provide a parser in the channel as a Custom Field. Columns without a matching datastore column or parser are skipped. | |
channels | Set<ChannelDescription> |
A channel is created using the topic’s name as the name of the target | Channel Descriptions which belong to the topic. | |
restrictToSources | Set<String> |
Sources to which topic is restricted. See source to topic matching. | ||
restrictFromSources | Set<String> |
Sources from which topic is restricted. See source to topic matching. |
Java Example
This configuration does not specify a target for the data.
note
If you do not name your topic the same as your store you will have to explicitly specify the target in the channel of the topic.
- The target is implicitly the Atoti table named “Trades”.
- The format comes from the schema file specified in the configuration.
@Bean
public AvroTopicDescription tradesTopic() {
return AvroTopicDescription.builder("Trades", "glob:trades*.avro", "schemas/schema.avsc")
.build();
}