☰
Navigation :

Adding a Field

This page provides a description of how to add new fields to a Atoti CVA Risk Capital project. It features an example in which a ‘TestField’ is added to Atoti CVA Risk Capital. It also explores additional steps needed for adding data to stores that are populated through a Tuple Publisher. The techniques employed are generic examples that can be extended, adapted, and repeated for any use case. In all cases, we make minimal changes to the Reference Implementation.

Editing the Data

This example shows modifying files currently populating the Vega store (files which match a naming scheme of sa-cva-cva-vega-sensitivities.csv), by adding a column to them. In this case, the new column’s header is ‘TestField’. Here’s an example of the contents of one such file. (This new column was added to all files matching such naming scheme):

AsOfDate NettingSetId RiskClass RiskFactorId Sensitivity SensitivityCcy ReferenceName ExplicitBucket BucketSuffix TestField
2018-12-05 ActiveBank EU_10 commodity whey_Implied volatility -12352.9 EUR whey Data1
2018-12-05 ActiveBank EU_101 interest rate CHF.IBOR3M_Implied volatility 7896.9 EUR CHF.IBOR3M Data2
2018-12-05 ActiveBank EU_12 foreign exchange CAD_ATM Implied volatility 5068.05 EUR CAD Data3
2018-12-05 ActiveBank EU_12 interest rate CHF.IBOR3M_Implied volatility 4782.93 EUR CHF.IBOR3M Data4
2018-12-05 ActiveBank EU_12 commodity wheat_Implied volatility 32805.05 EUR wheat Data5

Step 1 - Define Customisations to Datastore via Datastore Helper

Before you can load this new column into the Cube, make sure that your datastore has a field that can accept this column.

To add a new field to an existing datastore, navigate to the addModifications() method in the DatastoreCustomisationsConfig class.

    @Override
    public void addModifications(IDatastoreConfigurator configurator) {
        // Here we add our modification
        configurator.appendField("Vega", new CustomField("TestField", ILiteralType.STRING));  // Append "Vega" store with a new field "TestField" of type STRING
    }

Step 2 - Source Config / ETL Code

Modify the appropriate CsvSourceConfig. In this case, SaCsvSourceConfig, by creating a new class in /cvarc-starter/src/main/java/com/activeviam/cvarc/starter/cfg/impl/, which extends SaCsvSourceConfig.

  1. Modify the method which generates a list for the column names.
  2. If there are any tuple publishers that manipulate data before it’s published to the store, override those methods and modify them to make sure your new field is published.
@Configuration
public abstract class ExtendedSaCsvSourceConfig extends SaCsvSourceConfig {

    //1. Modify column names
    @Override
    protected List<String> generateCvaVegaCsvFields() {
        List<String> modifiedCsvFields = super.generateCvaVegaCsvFields();
        modifiedCsvFields.add("TestField");
        return modifiedCsvFields;
    }


    /**
     * 2. If there is a tuple publisher for the store you are modifying then the below method is necessary so that our new tuple
     * publisher which we will be used. In this case ExtendedVegaCVASensitivitiesPublisherWrapper will need to be defined.
     */
    @Override
    public Map<String, com.quartetfs.fwk.impl.Pair<List<String>, ITuplePublisher>> createTopicChannelsMap() {

        Map<String, com.quartetfs.fwk.impl.Pair<List<String>, ITuplePublisher>> extendedTopicChannels = super.createTopicChannelsMap();

        ITuplePublisher vegaCvaPublisher = new ExtendedVegaCVASensitivitiesPublisherWrapper(datastoreConfig.datastore(), generateCvaVegaToBaseStoreMapping());

        extendedTopicChannels.put(CSV_TOPIC__CVA_VEGA_SENSITIVITIES, new Pair(vegaCvaPublisher.getTargetStores(),vegaCvaPublisher));

        return extendedTopicChannels;
    }
}

Step 3 - Source Config cont. Tuple Publisher

Below is an example of the modified publisher method ExtendedVegaCVASensitivitiesPublisherWrapper, here we have copied the logic in VegaCVASensitivitiesPublisherWrapper and we have added handling of ‘TestField’ by appending our subTuple.

public class ExtendedVegaCVASensitivitiesPublisherWrapper extends VegaCVASensitivitiesPublisherWrapper {
    protected RiskFactorCatalogUtils riskFactorCatalogUtils = new RiskFactorCatalogUtils();
    protected static Logger logger = LoggerFactory.getLogger(VegaCVASensitivitiesPublisherWrapper.class.getName());
    /**
     * @param dataStore           {@link IDatastore}
     * @param saBaseHeaderMapping Mapping of the base store field names corresponding to the index
     *                            which they are found at inside the incoming sensitivity file.
     */
    public ExtendedVegaCVASensitivitiesPublisherWrapper(IDatastore dataStore, HashMap<String, Integer> saBaseHeaderMapping) {
        super(dataStore, saBaseHeaderMapping);
        // Instantiate SaBaseStore Publisher
        this.saBaseStorePublisher = new SaBaseStorePublisher(dataStore, SAStoreNames.SA_BASE_STORE_NAME,
                SharedDataStoreNames.VEGA);
    }
    @Override
    public void publish(List<Object[]> tuples) {
        List<Object[]> storeTuples = new ArrayList<>();
        tuples.stream().forEach(tuple -> {
            LocalDate asOfDate = (LocalDate)tuple[0];
            String riskClass = ((String)tuple[3]).toLowerCase();

            Object[] subTuple = new Object[] { tuple[0], // AsOfDate
                    tuple[1], // NettingSetId
                    null, // TradeId
                    ((String) tuple[3]).toLowerCase(), // RiskClass
                    tuple[4], // RiskFactorId
                    RiskMeasureEnum.Vega.toString().toLowerCase(), // RiskMeasure
                    tuple[6], // Sensitivities
                    tuple[7], // SensitivitiesCcy
                    null,
                    null,
                    null,
                    null,
                    null,
                    tuple[13], // NettingSetTradeId
                    tuple[14] // This tuple was added to the subtuple so that our TestField will be published to the store
            };
            // publish to SA Base store
            storeTuples.add(subTuple);
        });
        dataStore.getTransactionManager().addAll(SAStoreNames.VEGA_STORE_NAME, storeTuples);
    }
}

Step 4 - Application Config

Replace SaCsvSourceConfig with ExtendedSaCsvSourceConfig so that your changes can be picked up in ApplicationConfig located in cvarc-starter/src/main/java/com/activeviam/cvarc/starter/cfg/impl/ApplicationConfig.java

        // data sourcing
        SharedCsvSourceConfig.class,
        SharedAzureCsvSourceConfig.class,
        SharedAwsCsvSourceConfig.class,
        SharedGoogleCsvSourceConfig.class,

//      SaCsvSourceConfig.class,                        // here we comment out the original Source Config
        ExtendedSaCsvSourceConfig.class,                // and replace with our Extended Source Config
        SaAzureCsvSourceConfig.class,
        SaAwsCsvSourceConfig.class,
        SaGoogleCsvSourceConfig.class,

To expose fields as hierarchies or measures, see the examples in Adding a Hierarchy and Adding a new Measure.