Scope Parameters

DLC Scope Parameters are a key-value map. They can be provided in a DlcRequest and can be used to perform the following:

• Filter what gets loaded
• Provide default values

Filter Files to Load

Base Directory of CSV Source

The scope parameters can be used with the base directory of a CSV source.

Scope parameters will be inserted using text replacement for strings in the base directory which start with “#{” and end in “}”.

Example

This source accepts an AS_OF_DATE scope parameter

dlc:
  csv:
    local:
      sources:
        root-base-dir: /base-dir/#{AS_OF_DATE}

This request provides the AS_OF_DATE scope parameter to the base directory and we then retrieve the data from /base-dir/2021-01-01

{
  "operation": "LOAD",
  "topics": [
    "trades"
  ],
  "scope": {
    "AS_OF_DATE": "2021-01-01"
  }
}

Provide default values

Column Calculator

The scope parameters can be used in a CustomFieldDescription

Example

@Bean
CustomFieldDescription calculatorDescription() {
    return CustomFieldDescription.of(
            "columnCalculator",
            (DlcScope s) -> new ColumnCalculatorWhichUsesScope(s)
    );
}

Target

The scope parameters can be used in a TargetDescription

Example

@Bean
public TargetDescription targetDescription() {
    return TargetDescription.of("target", "store_name",
            (DlcScope s) -> new TuplePublisherWhichUsesScope(s)
    );
}