Adding a Field
This page provides a description of how to add new fields to a Atoti CVA Risk Capital project. It features an example in which a ‘TestField’ is added to Atoti CVA Risk Capital. It also explores additional steps needed for adding data to stores that are populated through a Tuple Publisher. The techniques employed are generic examples that can be extended, adapted, and repeated for any use case. In all cases, we make minimal changes to the Reference Implementation.
Editing the Data
This example shows modifying files currently populating the Vega store (files which match a naming scheme of sa-cva-cva-vega-sensitivities.csv), by adding a column to them. In this case, the new column’s header is ‘TestField’. Here’s an example of the contents of one such file. (This new column was added to all files matching such naming scheme):
AsOfDate | NettingSetId | RiskClass | RiskFactorId | Sensitivity | SensitivityCcy | ReferenceName | ExplicitBucket | BucketSuffix | TestField |
---|---|---|---|---|---|---|---|---|---|
2018-12-05 | ActiveBank EU_10 | commodity | whey_Implied volatility | -12352.9 | EUR | whey | Data1 | ||
2018-12-05 | ActiveBank EU_101 | interest rate | CHF.IBOR3M_Implied volatility | 7896.9 | EUR | CHF.IBOR3M | Data2 | ||
2018-12-05 | ActiveBank EU_12 | foreign exchange | CAD_ATM Implied volatility | 5068.05 | EUR | CAD | Data3 | ||
2018-12-05 | ActiveBank EU_12 | interest rate | CHF.IBOR3M_Implied volatility | 4782.93 | EUR | CHF.IBOR3M | Data4 | ||
2018-12-05 | ActiveBank EU_12 | commodity | wheat_Implied volatility | 32805.05 | EUR | wheat | Data5 |
Step 1 - Define Customisations to Datastore via Datastore Helper
Before you can load this new column into the Cube, make sure that your datastore has a field that can accept this column.
To add a new field to an existing datastore, navigate to
the addModifications()
method in the
DatastoreCustomisationsConfig
class.
@Override
public void addModifications(IDatastoreConfigurator configurator) {
// Here we add our modification
configurator.appendField("Vega", new CustomField("TestField", ILiteralType.STRING)); // Append "Vega" store with a new field "TestField" of type STRING
}
Step 2 - Source Config / ETL Code
Modify the appropriate CsvSourceConfig
. In this case, SaCsvSourceConfig
, by creating a new class in /cvarc-starter/src/main/java/com/activeviam/cvarc/starter/cfg/impl/
, which extends SaCsvSourceConfig
.
- Modify the method which generates a list for the column names.
- If there are any tuple publishers that manipulate data before it’s published to the store, override those methods and modify them to make sure your new field is published.
@Configuration
public abstract class ExtendedSaCsvSourceConfig extends SaCsvSourceConfig {
//1. Modify column names
@Override
protected List<String> generateCvaVegaCsvFields() {
List<String> modifiedCsvFields = super.generateCvaVegaCsvFields();
modifiedCsvFields.add("TestField");
return modifiedCsvFields;
}
/**
* 2. If there is a tuple publisher for the store you are modifying then the below method is necessary so that our new tuple
* publisher which we will be used. In this case ExtendedVegaCVASensitivitiesPublisherWrapper will need to be defined.
*/
@Override
public Map<String, com.quartetfs.fwk.impl.Pair<List<String>, ITuplePublisher>> createTopicChannelsMap() {
Map<String, com.quartetfs.fwk.impl.Pair<List<String>, ITuplePublisher>> extendedTopicChannels = super.createTopicChannelsMap();
ITuplePublisher vegaCvaPublisher = new ExtendedVegaCVASensitivitiesPublisherWrapper(datastoreConfig.datastore(), generateCvaVegaToBaseStoreMapping());
extendedTopicChannels.put(CSV_TOPIC__CVA_VEGA_SENSITIVITIES, new Pair(vegaCvaPublisher.getTargetStores(),vegaCvaPublisher));
return extendedTopicChannels;
}
}
Step 3 - Source Config cont. Tuple Publisher
Below is an example of the modified publisher method ExtendedVegaCVASensitivitiesPublisherWrapper
, here we have copied the logic in VegaCVASensitivitiesPublisherWrapper
and we have added handling of ‘TestField’ by appending our subTuple.
public class ExtendedVegaCVASensitivitiesPublisherWrapper extends VegaCVASensitivitiesPublisherWrapper {
protected RiskFactorCatalogUtils riskFactorCatalogUtils = new RiskFactorCatalogUtils();
protected static Logger logger = LoggerFactory.getLogger(VegaCVASensitivitiesPublisherWrapper.class.getName());
/**
* @param dataStore {@link IDatastore}
* @param saBaseHeaderMapping Mapping of the base store field names corresponding to the index
* which they are found at inside the incoming sensitivity file.
*/
public ExtendedVegaCVASensitivitiesPublisherWrapper(IDatastore dataStore, HashMap<String, Integer> saBaseHeaderMapping) {
super(dataStore, saBaseHeaderMapping);
// Instantiate SaBaseStore Publisher
this.saBaseStorePublisher = new SaBaseStorePublisher(dataStore, SAStoreNames.SA_BASE_STORE_NAME,
SharedDataStoreNames.VEGA);
}
@Override
public void publish(List<Object[]> tuples) {
List<Object[]> storeTuples = new ArrayList<>();
tuples.stream().forEach(tuple -> {
LocalDate asOfDate = (LocalDate)tuple[0];
String riskClass = ((String)tuple[3]).toLowerCase();
Object[] subTuple = new Object[] { tuple[0], // AsOfDate
tuple[1], // NettingSetId
null, // TradeId
((String) tuple[3]).toLowerCase(), // RiskClass
tuple[4], // RiskFactorId
RiskMeasureEnum.Vega.toString().toLowerCase(), // RiskMeasure
tuple[6], // Sensitivities
tuple[7], // SensitivitiesCcy
null,
null,
null,
null,
null,
tuple[13], // NettingSetTradeId
tuple[14] // This tuple was added to the subtuple so that our TestField will be published to the store
};
// publish to SA Base store
storeTuples.add(subTuple);
});
dataStore.getTransactionManager().addAll(SAStoreNames.VEGA_STORE_NAME, storeTuples);
}
}
Step 4 - Application Config
Replace SaCsvSourceConfig
with ExtendedSaCsvSourceConfig
so that your changes can be picked up in ApplicationConfig
located in
cvarc-starter/src/main/java/com/activeviam/cvarc/starter/cfg/impl/ApplicationConfig.java
// data sourcing
SharedCsvSourceConfig.class,
SharedAzureCsvSourceConfig.class,
SharedAwsCsvSourceConfig.class,
SharedGoogleCsvSourceConfig.class,
// SaCsvSourceConfig.class, // here we comment out the original Source Config
ExtendedSaCsvSourceConfig.class, // and replace with our Extended Source Config
SaAzureCsvSourceConfig.class,
SaAwsCsvSourceConfig.class,
SaGoogleCsvSourceConfig.class,
To expose fields as hierarchies or measures, see the examples in Adding a Hierarchy and Adding a new Measure.