Skip to main content
The bucket and prefix to read from are set through the bucket and prefix properties. The DLC ensures that only files within that source can be loaded.
To use this source, an S3Client bean must be defined. A default one can be used by enabling dlc.create-default-cloud-client.aws.

​
Configuration properties

Properties for an AWS CSV Source are defined by name in the dlc.csv.aws.sources namespace, and are picked up as configurations for the DLC.
KeyRequiredTypeDescription
bucketYStringThe bucket of the directory.
prefixYStringThe directory. Can include Scope Parameters.
Example #{SCOPE_PARAM}
source-propertiesCsvSourceConfigurationPropertiesCSV Source Configuration
cloud-fetching-configCloudFetchingConfigurationPropertiesCloud Fetching Config
topics-to-includeSet<String>Topics to include. See source to topic matching.
topics-to-excludeSet<String>Topics to exclude. See source to topic matching.
accepts-topic-overridesBooleanWhether this source is allowed to accept Topic Overrides in the Load Request.

​
YAML example

dlc:
  csv:
    aws:
      sources:
        awsCsvSource:
          bucket: bucket
          prefix: dataDir

​
Java configuration

AwsCsvSourceDescription Spring Beans are picked up as configurations for the DLC.
ParameterRequiredTypeDescription
nameYStringName of the source.
bucketYStringThe bucket of the directory
prefixYStringThe directory. Can include Scope Parameters.
Example #{SCOPE_PARAM}
sourceConfigurationCsvSourceConfigurationCSV Source Configuration
cloudFetchingConfigCloudFetchingConfigCloud Fetching Config
topicsToIncludeSet<String>Topics to include. See source to topic matching.
topicsToExcludeSet<String>Topics to exclude. See source to topic matching.
acceptsTopicOverridesBooleanWhether this source is allowed to accept Topic Overrides in the Load Request.

​
Java example

@Bean
AwsCsvSourceDescription source() {
    return AwsCsvSourceDescription.builder(
                    "awsCsvSource",
                    "bucket",
                    "dataDir"
            )
            .build();
}