Skip to main content

​
Configuration properties

Properties for the CSV Parser are provided inside the configuration of a CSV Topic.
KeyTypeDefaultDescription
charsetCharsetUTF-8Character set used to decode the file.
number-skipped-linesint1The number of lines to skip at the beginning of each file (header).
accept-overflowing-linesbooleanfalseWhen set to false, a record that contains more elements is rejected, otherwise the additional elements are ignored.
accept-incomplete-linesbooleantrueWhen set to false, a record that contains fewer elements is rejected. If set to true, null values are inserted into the missing columns.
separatorchar,Character which separates fields in the file.
columnsList<String>Columns will implicitly come from the store which is being published to.The order of columns in the file. If a column name matches a datastore column, its parser will be implied. For columns that do not match a datastore column, you must provide a CsvColumnParser in the channel as a Custom Field or define a Parser Override. Columns without a matching datastore column or parser are skipped.
process-quotesbooleantrueSets the behavior of the parser regarding quotes.

​
YAML example

dlc:
  csv:
    topics:
      trades:
        parser:
          accept-overflowing-lines: true
          separator: "\t"

​
Java configuration

CsvParserDescription Java object can be used for configuration in Java. The object is used in the configuration of a CSV Topic.
CsvParserDescription
ParameterTypeDefaultDescription
charsetCharsetUTF-8Character set used to decode the file.
numberSkippedLinesint1The number of lines to skip at the beginning of each file (header).
acceptOverflowingLinesbooleanfalseWhen set to false, a record that contains more elements is rejected, otherwise the additional elements are ignored.
acceptIncompleteLinesbooleantrueWhen set to false, a record that contains fewer elements is rejected. If set to true, null values are inserted into the missing columns.
separatorchar,Character which separates fields in the file.
columnsList<String>Columns will implicitly come from the store which is being published to.The order of columns in the file. If a column name matches a datastore column, its parser will be implied. For columns that do not match a datastore column, you must provide a CsvColumnParser in the channel as a Custom Field or define a Parser Override. Columns without a matching datastore column or parser are skipped.
processQuotesbooleantrueSets the behavior of the parser regarding quotes.

​
Java example

@Bean
public CsvTopicDescription tradesTopic() {
    return CsvTopicDescription.builder("Trades", "glob:trades*.csv")
            .parser(
                    CsvParserDescription.builder()
                            .separator('\t')
                            .acceptOverflowingLines(true)
                            .build()
            ).build();
}