Configuration properties
Properties for the CSV Parser are provided inside the configuration of a CSV Topic.| Key | Type | Default | Description |
|---|---|---|---|
| charset | Charset | UTF-8 | Character set used to decode the file. |
| number-skipped-lines | int | 1 | The number of lines to skip at the beginning of each file (header). |
| accept-overflowing-lines | boolean | false | When set to false, a record that contains more elements is rejected, otherwise the additional elements are ignored. |
| accept-incomplete-lines | boolean | true | When set to false, a record that contains fewer elements is rejected. If set to true, null values are inserted into the missing columns. |
| separator | char | , | Character which separates fields in the file. |
| columns | List<String> | Columns will implicitly come from the store which is being published to. | The order of columns in the file. If a column name matches a datastore column, its parser will be implied. For columns that do not match a datastore column, you must provide a CsvColumnParser in the channel as a Custom Field or define a Parser Override. Columns without a matching datastore column or parser are skipped. |
| process-quotes | boolean | true | Sets the behavior of the parser regarding quotes. |
YAML example
dlc:
csv:
topics:
trades:
parser:
accept-overflowing-lines: true
separator: "\t"
Java configuration
CsvParserDescription Java object can be used for configuration in Java. The object is used in the configuration of a
CSV Topic.
CsvParserDescription
| Parameter | Type | Default | Description |
|---|---|---|---|
| charset | Charset | UTF-8 | Character set used to decode the file. |
| numberSkippedLines | int | 1 | The number of lines to skip at the beginning of each file (header). |
| acceptOverflowingLines | boolean | false | When set to false, a record that contains more elements is rejected, otherwise the additional elements are ignored. |
| acceptIncompleteLines | boolean | true | When set to false, a record that contains fewer elements is rejected. If set to true, null values are inserted into the missing columns. |
| separator | char | , | Character which separates fields in the file. |
| columns | List<String> | Columns will implicitly come from the store which is being published to. | The order of columns in the file. If a column name matches a datastore column, its parser will be implied. For columns that do not match a datastore column, you must provide a CsvColumnParser in the channel as a Custom Field or define a Parser Override. Columns without a matching datastore column or parser are skipped. |
| processQuotes | boolean | true | Sets the behavior of the parser regarding quotes. |
Java example
@Bean
public CsvTopicDescription tradesTopic() {
return CsvTopicDescription.builder("Trades", "glob:trades*.csv")
.parser(
CsvParserDescription.builder()
.separator('\t')
.acceptOverflowingLines(true)
.build()
).build();
}