Navigation :

CSV Parser Properties

Configuration Properties

Properties for the CSV Parser are provided inside the configuration of a CSV Topic.

Key	Type	Default	Description
charset	Charset	UTF-8	Character set used to decode the file.
number-skipped-lines	int	1	The number of lines to skip at the beginning of each file (header).
accept-overflowing-lines	boolean	false	When set to false, a record that contains more elements is rejected, otherwise the additional elements are ignored.
accept-incomplete-lines	boolean	true	When set to false, a record that contains fewer elements is rejected. If set to true, null values are inserted into the missing columns.
separator	char	,	Character which separates fields in the file.
column-names	List<String>	Columns will come from the store which is being published to	Column of the file. You must specify all column names in the file. The ones that do not correspond to a matching column in the datastore are skipped.
process-quotes	boolean	true	Sets the behavior of the parser regarding quotes.

Example

dlc:
  csv-topics:
    trades:
      parser:
        accept-overflowing-lines: true
        separator: "\t"

Java Configuration

CsvParserDescription java object can be used for configuration in Java. The object is used in the configuration of a CSV Topic.

CsvParserDescription

Parameter	Type	Default	Description
charset	Charset	UTF-8	Character set used to decode the file.
numberSkippedLines	int	1	The number of lines to skip at the beginning of each file (header).
acceptOverflowingLines	boolean	false	When set to false, a record that contains more elements is rejected, otherwise the additional elements are ignored.
acceptIncompleteLines	boolean	true	When set to false, a record that contains fewer elements is rejected. If set to true, null values are inserted into the missing columns.
separator	char	,	Character which separates fields in the file.
columnNames	List<String>	Columns will come from the store which is being published to	Column of the file. You must specify all column names in the file. The ones that do not correspond to a matching column in the datastore are skipped.
processQuotes	boolean	true	Sets the behavior of the parser regarding quotes.

Example

@Bean
public CsvTopicDescription tradesTopic() {
    return CsvTopicDescription.builder("Trades")
            .parser(
                    CsvParserDescription.builder()
                            .separator('\t')
                            .acceptOverflowingLines(true)
                            .build()
            ).build();
}