Skip to main content
They can be provided in a DlcRequest to filter what gets loaded or to supply default values, among other uses:
  • Filter what gets loaded
  • Provide default values

​
Filter files to load

​
Files

The files scope key allows a collection of files to be explicitly specified for loading. By default, all files are loaded for every topic in the request, regardless of the topic’s file pattern. To modify this behavior, set the apply-file-pattern key to true so that each topic in the request only loads files matching its file pattern.

​
Base directory resolution

Custom scope parameters can be used to dynamically define the base directory of a source. These parameters allow for flexible configuration by substituting placeholders in the directory path with actual values provided in the request.
The keys files and apply-file-pattern are reserved and cannot be used as custom scope parameters.
Scope parameters are inserted into the base directory path using a text replacement mechanism:
  • Placeholders in the path must start with #{ and end with }.
  • At request time, these placeholders are substituted with the corresponding values from the scope, and are blank if not provided.

​
Example

This source accepts an AS_OF_DATE scope parameter 
dlc:
  csv:
    local:
      sources:
        root-base-dir: /base-dir/#{AS_OF_DATE}
This request provides the AS_OF_DATE scope parameter to the base directory, and the data is then retrieved from /base-dir/2021-01-01. 
{
  "operation": "LOAD",
  "topics": [
    "trades"
  ],
  "scope": {
    "AS_OF_DATE": "2021-01-01"
  }
}

​
Provide default values

​
Column calculator

The scope parameters can be used in a CustomFieldDescription

​
Example

@Bean
CustomFieldDescription calculatorDescription() {
    return CustomFieldDescription.of(
            "columnCalculator",
            (DlcScope s) -> new ColumnCalculatorWhichUsesScope(s)
    );
}

​
Target

The scope parameters can be used in a TargetDescription

​
Example

@Bean
public TargetDescription targetDescription() {
    return TargetDescription.of("target", "store_name",
            (DlcScope s) -> new TuplePublisherWhichUsesScope(s)
    );
}