Skip to main content
The root directory is set through the root-base-dir property, which supports scope parameter placeholders. The DLC ensures that only files within that source can be loaded.

​
Configuration properties

Properties for a Local Parquet Source are defined by name in the dlc.parquet.local.sources namespace, and are picked up as configurations for the DLC.
KeyRequiredTypeDescription
root-base-dirYStringRoot base directory. Can include Scope Parameters.
Example #{SCOPE_PARAM}
topics-to-includeSet<String>Topics to include. See source to topic matching.
topics-to-excludeSet<String>Topics to exclude. See source to topic matching.
accepts-topic-overridesBooleanWhether this source is allowed to accept Topic Overrides in the Load Request.

​
YAML example

dlc:
  parquet:
    local:
      sources:
        localParquetSource:
          root-base-dir: data
        datedSource:
          root-base-dir: data/#{AS_OF_DATE}

​
Java configuration

LocalParquetSourceDescription Spring Beans are picked up as configurations for the DLC.
ParameterRequiredTypeDescription
nameYStringName of the source.
rootBaseDirYStringRoot base directory.
topicsToIncludeSet<String>Topics to include. See source to topic matching.
topicsToExcludeSet<String>Topics to exclude. See source to topic matching.
acceptsTopicOverridesBooleanWhether this source is allowed to accept Topic Overrides in the Load Request.

​
Java example

@Bean
LocalParquetSourceDescription source() {
    return LocalParquetSourceDescription.builder("localParquetSource", "data")
            .build();
}

@Bean
LocalParquetSourceDescription datedSource() {
    return LocalParquetSourceDescription.builder("datedSource", "data/#{AS_OF_DATE}")
            .build();
}