SA ETL for SBM curvature

This page provides an overview of ETL for SBM Curvature, including the following breakdowns:

• Table showing you the fields included in each row of a Curvature CSV Input Data file, indicating which stores these fields are copied to during the ETL process.
• Key fields for vector creation and the fields that have the potential to be populated with vectors, as opposed to always containing scalar values.

​

Vectorization

The following table provides information about the vectors employed within the Curvature CSV Input files for each risk class:

​

PublisherUtils vectorization methods

For Curvature, two sets of vectors are created:

​

Un-Vectorization

Vectors within the input are split into several scalar lines by the RiskClassTuplePublisher. For each risk weight in the vectorized input, the corresponding index of shift up and shift down are used to create scalar tuples under the scalar risk factor. The scalar risk factor is the concatenation of the input RiskFactor and the risk weight.

​

Interpolation

​

PublisherUtils interpolation methods

Interpolation is performed for the two sets separately, using linear interpolation and a predefined set of risk weights.

​

Normalization

​

Curvature-relevant stores

The stores that are relevant for Curvature are:

​

Mapping of SBM Curvature CSV file fields onto the stores that they populate

​

Column Calculators and Tuple Publishers

​

RiskFactorColumnCalculator

If Risk Factor is not provided, RiskFactorColumnCalculator will create one based on risk class. The table below explains how Risk Factors are derived.

​

The Tuple Publisher and Publisher classes

The function of the TuplePublisher and its associated Publisher classes is to separate data in the incoming file row according to its relevance to particular stores and apply ETL logic to the incoming rows: