> ## Documentation Index
> Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt
> Use this file to discover all available pages before exploring further.
# SA ETL for SBM curvature
This page provides an overview of ETL for SBM Curvature, including the following breakdowns:
* Table showing you the fields included in each row of a Curvature CSV Input Data file, indicating which stores these fields are copied to
during the ETL process.
* Key fields for vector creation and the fields that have the potential to be populated with vectors, as opposed to always containing
scalar values.
## Vectorization
The following table provides information about the vectors employed within the Curvature CSV Input files for each risk class:
| Risk Class | Vector Information |
| ----------------- | -------------------------------------------------------------------------------- |
| - GIRR | The indices of the Shift\_Up\_PV and Shift\_Down\_PV vectors are the RiskWeight. |
| - CSR non-Sec | |
| - CSR Sec CTP | |
| - CSR Sec non-CTP | |
| - Equity | |
| - Commodity | |
| - FX | |
#### PublisherUtils vectorization methods
For Curvature, two sets of vectors are created:
| Shift Up | Shift Down |
| ----------------------------------------- | ------------------------------------------- |
| \[risk weights]+\[shift up sensitivities] | \[risk weights]+\[shift down sensitivities] |
### Un-Vectorization
Vectors within the input are split into several scalar lines by the RiskClassTuplePublisher.
For each risk weight in the vectorized input, the corresponding index of shift up and shift down are used to create scalar tuples under the scalar risk factor. The scalar risk factor is the concatenation of the input RiskFactor and the risk weight.
## Interpolation
#### PublisherUtils interpolation methods
Interpolation is performed for the two sets separately, using linear interpolation and a predefined set of risk weights.
## Normalization
#### Curvature-relevant stores
The stores that are relevant for Curvature are:
| Stores |
| --------------------- |
| SASensitivities |
| RiskFactorDescription |
| UnderlyingDescription |
### Mapping of SBM Curvature CSV file fields onto the stores that they populate
| Field from CSV file OR ColumnCalculator (if the CSV file field is null) | SASensitivities | RiskFactorDescription | UnderlyingDescription |
| ----------------------------------------------------------------------- | ----------------------------------- | --------------------- | --------------------- |
| AsOfDate | (AsOfDate) | (AsOfDate) | (AsOfDate) |
| TradeId | (TradeKey for trade level data) | | |
| Book | (part of TradeKey for summary data) | | |
| LegalEntity | (part of TradeKey for summary data) | | |
| RiskClass | (Risk Class) | (Risk Class) | (Risk Class) |
| RiskFactor (if missing `RiskFactorColumnCalculator` is used) | (RiskFactor) | (RiskFactor) | |
| Shift\_Up\_PV | | | |
| Shift\_Down\_PV | | | |
| Ccy | | | |
| RiskWeight | | | |
| PVApplied | | | |
| Type | | (RiskFactorType) | (GIRR Curve Type) |
| GIRRCcy | | | (GIRR Ccy) |
| Underlying (if missing, `UnderlyingColumnCalculator` is used) | | (Underlying) | (Underlying) |
| CSRQuality | | | (CSRQuality) |
| CSRSector | | | (CSRSector) |
| CSRTrance (Deprecated) | | | |
| EquityEconomy | | | (EquityEconomy) |
| EquityMarketCap | | | (EquityMarketCap) |
| EquitySector | | | (EquitySector) |
| CommodityLocation | | (CommodityLocation) | |
| CommodityTime (Deprecated) | | | |
| CommodityGrade (Deprecated) | | | |
| CommodityRoute (Deprecated) | | | |
| FXCounterCurrency | | (FXCounterCurrency) | |
| FxDividerEligibility | | | |
| CSRRating | | | |
| Bucket | | | |
| PresentValue | | | |
| Pool | | | |
| Attachment | | | |
| Detachment | | | |
| PVLadder | | | |
## Column Calculators and Tuple Publishers
#### RiskFactorColumnCalculator
If Risk Factor is not provided, `RiskFactorColumnCalculator` will create one based on risk class. The table below explains how Risk Factors are derived.
| Risk Class | Creation Method for RiskFactor |
| --------------- | ------------------------------------------------------------------------------------------------------------- |
| Commodity | Derived by concatenating Underlying and CommodityLocation, using ‘space’ as a delimiter |
| GIRR | Underlying is used |
| Equity | Derived by concatenating Underlying and RiskFactorType, using ‘space’ as a delimiter |
| CSR non-Sec | Derived by concatenating Underlying and RiskFactorType, using ‘space’ as a delimiter |
| CSR Sec CTP | Derived by concatenating Underlying and RiskFactorType, using ‘space’ as a delimiter |
| CSR Sec non-CTP | Derived by concatenating Underlying and RiskFactorType, using ‘space’ as a delimiter |
| FX | If FxCounterCcy is not empty, derived by concatenating Underlying and FxCounterCcy, using “\|” as a delimiter |
If `sbm.risk-factor.always-append-tenor` is set to `true`, then tenor will also be added to Risk Factor.
#### The Tuple Publisher and Publisher classes
The function of the TuplePublisher and its associated Publisher classes is to separate data in the incoming file row according to its relevance to particular
stores and apply ETL logic to the incoming rows:
| Tuple Publisher Class | Publisher Class | Function |
| ----------------------- | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| RiskClassTuplePublisher | | Splits fields according to the following criteria:
- fields relevant to DeltaPublisher OR VegaPublisher OR CurvaturePublisher
- fields relevant to StaticDataPublisher classes for RiskFactorDescription and UnderlyingDescription stores.
Note: If multiple sensitivities are held on the same line, the line is de-multiplexed here. |
| | CurvaturePublisher | CurvaturePublisher publishes to the SASensitivities store. |
| | RiskFactorPublisher (extends StaticDataPublisher) | RiskFactorPublisher publishes to the RiskFactorDescription store. |
| | UnderlyingPublisher (extends StaticDataPublisher) | UnderlyingPublisher publishes to the UnderlyingDescription store. |