Migration notes 6.0

This page explains the changes required to migrate to the stated version of the Atoti FRTB.

Migrate to 6.0.3

Summary

  • Virtual hierarchies can be set with the property frtb.configuration.virtual-hierarchies.
  • The new Curvature CVR measure will be disabled by default in 6.0.x versions of Atoti FRTB. To enable it, set the property frtb.curvature-cvr.enabled=true in the application.yaml file.
  • Cube distribution can now handle data overlap. This means it is possible to have several data cubes with duplicated data. you need to set frtb.distribution.enable-data-duplication=true.
  • HistoricalDatesProvider no longer queries the base stores for the list of historical dates. This is achieved by using the database statistics to gather the set of unique dates. The HistoricalDatesProvider can be reverted to the previous behavior of executing distinct datastore queries by setting the property frtb.historical-dates-provider.is-using-approximate-members=false.
  • The zero risk-weight can now be specified per obligor and jurisdiction, defaulting to 0%.

Breaking changes

None.

Deprecations

  • Cube distribution by field: Distribution of data cubes across nodes by defining clustering fields is deprecated, it should be done with distribution levels.

Input file formats

Added

File Details
DRC_Public_Sector_Obligor_Risk_Weights.csv Use this file to configure specific zero risk-weights per obligor and jurisdiction.

Configuration properties

Properties added

Property Default value Description
frtb.configuration.virtual-hierarchies Empty A list of hierarchies to set as virtual.
frtb.curvature-cvr.enabled false Enables the new Curvature CVR measures.
frtb.distribution.levels.sa-cube AsOfDate@Date@Dates The levels used for the StandardisedApproachCube distribution.
frtb.distribution.levels.ima-cube AsOfDate@Date@Dates The levels used for the InternalModelApproachCube distribution.
frtb.distribution.levels.ima-drc-cube AsOfDate@Date@Dates The levels used for the IMADRCCube distribution.
frtb.distribution.levels.pl-cube AsOfDate@Date@Dates The levels used for the PLCube distribution.
frtb.distribution.levels.stress-calibration-cube AsOfDate@Date@Dates The levels used for the StressCalibrationCube distribution.
frtb.distribution.priority.sa-cube Empty The priority of the StandardisedApproachCube in cases of data overlap within a horizontal distribution.
frtb.distribution.priority.ima-cube Empty The priority of the InternalModelApproachCube in cases of data overlap within a horizontal distribution.
frtb.distribution.priority.ima-drc-cube Empty The priority of the IMADRCCube in cases of data overlap within a horizontal distribution.
frtb.distribution.priority.pl-cube Empty The priority of the PLCube in cases of data overlap within a horizontal distribution.
frtb.distribution.priority.stress-calibration-cube Empty The priority of the StressCalibrationCube in cases of data overlap within a horizontal distribution.
frtb.distribution.enable-data-duplication false Set to true to allow duplicate members in the distribution level of horizontally distributed data cubes.
directquery.incremental-refresh Empty Configures increment refresh templates.
frtb.historical-dates-provider.is-using-approximate-members true If true (default), the HistoricalDatesProvider will use the database statistics to gather the unique set of dates from the stores.
If false, it will revert to the previous behavior of executing distinct datastore queries.
frtb.historical-dates-provider.stores-to-scan [PLTrades, IMATrades, DRCIMABase, SASensitivities, StressCalibrationTrades] The list of stores to scan to collect the historical dates from.
frtb.historical-dates-provider.as-of-date-field AsOfDate The name of the date field to use when querying the stores for historical dates.

Properties deprecated

Property name Comment
sa.cube.distributing.field replaced by frtb.distribution.levels.sa-cube
ima.cube.distributing.field replaced by frtb.distribution.levels.ima-cube
ima-drc.cube.distributing.field replaced by frtb.distribution.levels.ima-drc-cube
pl.cube.distributing.field replaced by frtb.distribution.levels.pl-cube
stress-calibration.cube.distributing.field replaced by frtb.distribution.levels.stress-calibration-cube

Properties files

Files Modified

frtb-data-load.properties

New properties:

Property Name Comment Value
drc.public-sector.obligor-risk-weights.file-pattern File pattern used to load the PublicSectorObligorRiskWeights store. **/DRC_Public_Sector_Obligor_Risk_Weights*.csv{,.gz}

Datastores

Added stores

Store Details
PublicSectorObligorRiskWeights This store contains optional risk-weights for public sector obligors applied to zero risk-weight exposures in the DRC non-securitisations calculation.

Databases

Added tables

Table Details
PUBLIC_SECTOR_OBLIGOR_RISK_WEIGHTS This store contains optional risk-weights for public sector obligors applied to zero risk-weight exposures in the DRC non-securitisations calculation.

Measures

Added

Cube Measure Details
StandardisedApproach Curvature CVR This measure uses the underlying Curvature CVR Up or Curvature CVR Down measure depending on the “direction” of the Risk Position Scenario at the Bucket level. A new intermediate measure Curvature Bucket Scenario evaluates the Risk Position Scenario measure at the Book level.
This measure has been added for the following risk classes within sa-sbm:
- Commodity
- CSR non-Sec
- CSR Sec CTP
- CSR Sec non-CTP
- Equity
- FX
- GIRR
StandardisedApproach Crypto 2b Support for Cryptoassets Group 2b

Migrate to 6.0.2

No migration needed.

Migrate to 6.0.1

No migration needed.

Breaking changes

None

Summary

  • Upgraded to Data Connectors 5.0.4

Migrate to 6.0.0

Upgrading from version 5.3.0, see Atoti FRTB 6.0 Release Notes.

Atoti FRTB uses Atoti Server 6.1.7 and Atoti UI 5.2.x. For new features and fixes included in these releases, please see the Atoti UI documentation and Atoti UI Migration Notes, and the release notes for Atoti Server.

Breaking changes

  • Measure Configuration: The text replacement template for measure configuration now uses the pattern #{_name_} instead of ${_name_}. See Startup Properties.
  • frtb-application module: Some of the content of the frtb-starter module has been moved to the new frtb-application module.
  • DLC 5.0 Upgrade: With the new version of the DLC, the configuration of the ETL has been simplified.
  • What-if endpoints: The (undocumented) what-if REST endpoints used by the Switch Desk Model and Switch Book Desk what-if simulations have changed.
  • DoctorPivot endpoint: DoctorPivot is now embedded within the AdminUI. See Relocation of DoctorPivot for more.

Headline announcement

  • Upgraded to Atoti Server 6.1.7 : Upgraded to the latest Atoti Server.
  • Upgraded to JDK 21 : Upgraded from JDK 17 to JDK 21 to utilize the latest features.
  • Removed ActiveMonitor : The Active-Monitor module and related features have been removed.
  • frtb-application module: The frtb-application module replaces the frtb-starter module. All customizations should now be done in the frtb-application module, or in a new module based on the frtb-application module.
  • Sign-Off Service Availability: The Sign-Off REST service now throws ServiceUnavailableExceptions until the initial data loading phase has been completed. The Sign-Off REST service now provides a GET /enabled endpoint to check if the service is currently enabled.
  • Simplified OpenTelemetry Configuration: We now rely on OpenTelemetry’s GlobalOpenTelemetry to configure the export of logs and metrics.
  • OpenRewrite recipes As an experimental feature, we have added OpenRewrite recipes that you can use to automate some of the migration steps.

OpenRewrite recipes

note

We have included these OpenRewrite recipes as an experimental feature and would welcome your feedback on their usefulness in your migration. We can then evaluate the feature for potential inclusion in future releases. Please report any issues you have through the normal Jira channels.

To help with upgrades, we have created a set of OpenRewrite recipes that can be used to automatically perform some of the migration steps from 5.3 to 6.0.

The recipes included in this release are:

Recipe Name Description
com.activeviam.frtb.Migrate_6_0 Runs all 6.0 migration recipes.
com.activeviam.frtb.FileUpload_6_0 Updates the properties for the file-upload what-if. This includes:
  • Renames csvSource.subdirectory.dataset.whatif to file-upload.csv-source-subdirectory
  • Adds file-upload.staging-directory as the combination of input.data.root.dir.path (or csv-source.dataset) and csvSource.subdirectory.dataset.stage by referencing the old properties.
com.activeviam.frtb.DQAutoVectorizer_6_0 Deletes the old, unused activeviam.directquery.enableAutoVectorizer property.
com.activeviam.frtb.DataDirectories_6_0 Updates the properties configuring the data directories. This includes:
  • Standardizes the case of csv-source.
  • Adds csv-source.dataset-historical=${csv-source.dataset}/historical.
  • Adds csv-source.dataset-configuration=${csv-source.dataset}/configuration.

See Using OpenRewrite in Atoti FRTB for instructions on how to run the recipes.

frtb-application module

The new frtb-application Maven module has been split out of the frtb-starter Maven module.

Components moved

The following components have been moved into the new module:

  • FRTBApplication class including the main() method.
  • Spring security configuration
  • OpenTelemetry configuration
  • Logging configuration
  • Content Service configuration
  • Configuration files

Configuration classes moved

For the full list of configuration classes that have been moved into the frtb-application module, see the Moved Classes section.

Data & Test folders moved

The sample data located at /frtb-starter/src/test/resources/ has been moved out of the frtb-starter module to the /sample-data/ folder located at the root of the project.

The tests included in the frtb-starter module have been moved into the new frtb-application-tests module.

OpenTelemetry Configuration

The previous OpenTelemetry configuration has been replaced with a simple OpenTelemetry bean.

To enable the autoconfiguration of this bean, set -Dotel.java.global-autoconfigure.enabled=true on the command line.

FX Translation Risk

The field “Translation Risk Currency” has been added to support loading translation risk for different reporting currencies. It is now possible to load different translation risks for different reporting currencies.

This field is only used if FXComplexTrade=Y (otherwise the translation risk is calculated automatically).

  • If this field is empty, the sensitivity is not treated as translation risk but instead treated normally.
  • If this field is set, then the sensitivity will be interpreted as translation risk for that currency. It will be filtered and only used if the translation risk currency is the same as the reporting currency.

Alternative root parameter sets

The property parameter-set.root has been added to specify the root parameter set in the configuration and input files.

It is expected that input files are generated for the parameter set specified in this property.

Additionally, the configuration files containing parameters will need to be adapted to the new root parameter set.

  • Previously, when no parameter set was specified in the configuration it was assumed to be “BCBS”. Now, these parameters will default to the specified root parameter set.
  • Parameter configuration will need to be modified for the new root parameter set.

For example, for the bucket used for CSR non-Sec covered bonds we have in FRTBParameters.csv:

Name Value Date ParameterSet
sa.csr-nonsec.bucket.covered-bonds 8 2016-01-01
sa.csr-nonsec.bucket.covered-bonds 10 2016-01-01 CRR2

If you want to set CRR2 as the root parameter set, the above two lines can change to become:

Name Value Date ParameterSet
sa.csr-nonsec.bucket.covered-bonds 8 2016-01-01 BCBS
sa.csr-nonsec.bucket.covered-bonds 10 2016-01-01

So that the value in the CRR2 parameter set is still “10”, and the value for the BCBS parameter set is still “8”.

Summing Sensitivities

When multiple sensitivities with the same key fields are loaded, they are now summed. Previously this was only done when processing a chunk within a file, now it is done across the whole transaction.

The following configuration has been added to application.yaml and will sum sensitivities and PVs for the SASensitivities store, making sure that the currency and a few other fields are the same.

datastore-sum-duplicates:
  SASensitivities:
    matching: ["Ccy", "RiskWeight", "FXOtherCcy", "Direction"]
    summing: ["Sensitivity", "Shift_Up_PV", "Shift_Down_PV", "PresentValue", "Notional", "Adjustment"]

FRTB Modules

Removed

Module Details
frtb-activemonitor ActiveMonitor and associated configurations have been removed from Atoti FRTB.

Added

Module Details
frtb-application Provides an example Spring Boot Application for Atoti FRTB.
* Contains simple configurations required to run Atoti FRTB.
* Is intended to be used as a reference for clients to build their own applications as this module is not production-ready.
* Provides various Spring profiles outlining the properties required to run the application in different environments.
For example, the profile dist-query-node activates the application-dist-query-node.yaml configuration file which defines the properties required to run the application as a distributed query node.

tip

To help ease upgrade efforts, move all customizations into your new module.

DLC 5.0 upgrade

The upgrade to DLC 5.0 has brought some significant changes to the ETL configuration. For non-FRTB specific DLC migration, please see the DLC 5.0 Migration Guide.

DLC configurations for Atoti FRTB have now been organized into FrtbDlcConfig.

Loading Topic Configurations

Before the DLC redesign, ChannelParameters were being used to encapsulate DLC configurations into beans. Now, configurations that were previously found as ChannelParameters beans can be found as Topic Descriptions in the following classes:

Old Class New Class
AConfigurationSourceConfig ConfigurationSourceDescription
ASourceConfig SourceDescription
ASensiSourceConfig SensiSourceDescription

Column calculators are now configured as DLC CustomFieldDescription beans and have been organized into CustomFieldDescriptionConfig. The DLC implicitly adds CSVColumnParser column calculators based on the store being targeted, so those have been removed from the topic configurations. The new DLC also lets you configure parser overrides. In Atoti FRTB they can be found in ColumnParserOverridesDescriptionConfig.

Tuple publishers are now configured as DLC TargetDescription beans and have been organized into TargetDescriptionsConfig.

Unloading Topic Configurations

Unloading topics are now completely separate from loading topics. They are configured in UnloadTopicDescriptionsConfig.

Topic Aliases

Old Class New Class
SourceTopicAliases AliasesDescriptionConfig

Topic Ordering Logic

Old Class New Class
FRTBLoadDataTxControllerTask FrtbLoadOperation

ETL Extension Updates

See the following sections on how the ETL extensions have changed with the new DLC:

Removed classes

See the list of classes removed as part of the DLC 5.0 upgrade.

Input file formats

Modified

Modification File Field Optional Description
Added SBM_Delta_Sensitivities.csv Translation Risk Ccy Y FX only. Indicates the sensitivity represents translation risk; set to the reporting currency.
Added SBM_Summary_Delta_Sensitivities.csv Translation Risk Ccy Y FX only. Indicates the sensitivity represents translation risk; set to the reporting currency.
Added Trade_Attributes.csv Inclusion Y Indicates if the trade should be included (“Y”) or not by default in the calculations.
Default is ‘Y’

Startup properties

The file frtb-config.properties is now loaded into the Spring Boot environment, so the properties in this file can be overridden on the command line.

The file measures-config.json is now also loaded into the Spring Boot environment. The json file is converted to properties with the prefix “measures”.

The text replacement template in these files now uses the pattern #{_name_} instead of ${_name_}. The ${_name_} pattern can still be used for Spring Boot’s property placeholders.

Property Name Comment Value Default Definition in File
dlc.enabled Enables or disables the Data Load Controller. true or false application.yaml
file-upload.staging-directory The staging directory for the File Upload. ${csv-source.dataset}/stage application.properties
file-upload.csv-source-subdirectory The sub-directory of files on which the What-If is executed. true or false application.properties
csv-source.dataset-historical Path to the directory containing historical files to load. ${csv-source.dataset}/historical application.properties
csv-source.dataset-configuration Path to the directory containing the configuration files to load. sample-data/configuration application.properties
parameter-set.root Root parameter set. Defaults to “BCBS”
datastore-sum-duplicates._store-name_.matching List of names of fields that must match when summing duplicates for the store store-name.
If missing, all fields are matched; if empty, no fields are matched.
datastore-sum-duplicates._store-name_.summing List of names of fields that should be summed when there are duplicates for the store store-name.
directquery.aggregate-tables Configuration of the Aggregate Tables to be used when running with DirectQuery. Mapping of aggregate tables per cube application.yaml
whatif.datastore.<identifier>.* Simple datastore changes what-if simulations. See Simple Datastore Changes. inclusion-change what-if simulation definition. application.yaml

Updated properties: changed values or renamed

Changed values:

Property Name Comment New Value Old Value Default Definition in File
activeviam.jwt.expiration Value changed from an int of seconds to a Duration. Note that an integer value will be picked up as milliseconds now. 12h (Duration) 43200 (seconds) jwt.properties
csv-source.dataset Split out to csv-source.dataset-historical, csv-source.dataset-configuration and csv-source.dataset sample-data/data data application.properties
data.extraction.templates.base.dir.path Updated for new structure frtb-application/src/main/resources/DEETemplateOrders ./src/main/resources/DEETemplateOrders application.properties

Renamed

Old Property New Property Comment Default Definition in File
csv-source.parserThreads csv-source.parser-threads The naming convention has changed from camel case to kebab case, but the old property key will still work. application.properties
csv-source.bufferSize csv-source.buffer-size The naming convention has changed from camel case to kebab case, but the old property key will still work. application.properties
aws.accessKeyId aws.accessKeyId Previously managed by FRTB, now inheriting AWS SDK property aws.accessKeyId in DefaultCredentialsProvider frtb-cloud.properties
aws.secretKey aws.secretAccessKey Previously managed by FRTB, now inheriting AWS SDK property aws.secretAccessKey in DefaultCredentialsProvider frtb-cloud.properties
azure.connection.string dlc.azure.connection-string Previously managed by FRTB, now inheriting DLC property dlc.azure.connection-string for DefaultAzureClientConfig frtb-cloud.properties

Deleted properties:

Property Name Comment Deleted from File
activeviam.directquery.enableAutoVectorizer The property is no longer being used. application.properties
input.data.root.dir.path This is now a part of file-upload.staging-directory application.properties
csvSource.subdirectory.dataset.stage This is now a part of file-upload.staging-directory application.properties
csvSource.subdirectory.dataset.whatif This is now file-upload.csv-source-subdirectory application.properties
cloudSource.dataset csv-source.dataset-historical, csv-source.dataset-configuration and csv-source.dataset are now shared between cloud sources and local frtb-cloud.properties
cloud.fetch.thread No longer configurable frtb-cloud.properties
dlc.audit-logging.enabled DLC event handlers are no longer a part of the new DLC application.properties

Configuration files

Files Modified

FRTBParameters.csv
Modification Parameter Type Default Description
Deprecated (for removal) sa.drc.no-maturity-floor-when-offsetting boolean false Parameter is no longer needed ref
frtb-config.properties
Modification Parameter Type Default Description
Changed Default sa.drc.strict-long-short boolean true Changed default to true.

Datastores

Modified stores

Modification Store Field Type Description
Added Trade Mapping Inclusion String Indicates if the trade should be included (“Y”) or not by default in the calculations.
Added SaSensitivities Translation Risk Ccy STRING FX only. Indicates the sensitivity represents translation risk; set to the reporting currency.

Databases

Modified tables

Modification Table Field Type Description
Added SASENSITIVITIES TRANSLATION_RISK_CCY String FX only. Indicates the sensitivity represents translation risk; set to the reporting currency.
Added TRADE_MAPPING INCLUSION String Indicates if the trade should be included (“Y”) or not by default in the calculations.

Cube schema

Added

Cube Dimension Hierarchy Levels Datastore fields Details
All Booking Inclusion Inclusion TradeMapping.Inclusion This hierarchy is used to indicate which trades should be included in the calculations.

Modified Classes

The following classes have been added, removed or had their function replaced with new classes.

Added Classes

Added Rationale
DatastoreChangeSimulations WhatIf definitions have been restructured.
SummingDuplicateKeyHandler DuplicateKeyHandler that sums tuples.
TableSecurityConfig Configures the security roles for the actions that can be taken on the datastore.
OpenTelemetryTracingConfig Configures the OpenTelemetry service for the application.

Removed Classes

DataLoadController

The following classes related to the DLC and data loading have been removed as part of the DLC 5.0 upgrade.

Click to see the full list of removed classes.
WhatIf

The following classes have been replaced with DatastoreChangeSimulations because the configuration of WhatIfs has changed.

Removed
BookDeskSwitchSubmissionDTO
DeskModelSwitchSubmissionDTO
BookDeskSwitchRestServiceController
DeskModelSwitchRestServiceController
BookDeskSwitchSimulationDefinition
DeskModelSwitchSimulationDefinition
Services
Removed Replacement Rationale
DataAvailability Distribution no longer requires relying on checking if data is available.
JsonDataLoader Unused.
FRTBI18nConfig Atoti Server autoconfigured starters. This class is now autoconfigured by Atoti Server.

Moved Classes

Class Previous Module & Package New Module & Package
LocalContentServiceConfig frtb-starter/com.activeviam.frtb.starter.cfg.impl frtb-application/com.activeviam.frtb.application.config.content
RemoteContentServiceConfig frtb-starter/com.activeviam.frtb.starter.cfg.impl frtb-application/com.activeviam.frtb.application.config.content
BasicAuthenticationProviderConfig frtb-accelerator/com.activeviam.frtb.ref.cfg.impl.security frtb-application/com.activeviam.frtb.application.config.security
SecurityConfig frtb-accelerator/com.activeviam.frtb.ref.cfg.impl frtb-application/com.activeviam.frtb.application.config.security
UserDetailsServiceConfig frtb-starter/com.activeviam.frtb.starter.cfg.impl.security frtb-application/com.activeviam.frtb.application.config.security

Migrating to using Atoti Starters

Atoti Server now provides Spring Boot Starters to aid in application development. These starters will auto-configure some basic and required beans for the application to run. Atoti FRTB has been updated to use these starters. The following changes have been made:

Change Description
Removal of JMXEnabler Beans The JMXEnabler beans have been removed from Atoti FRTB as the Atoti Server’s Starters provide these beans automatically for us.
Removed FRTBI18nConfig Configuration Class This class is now autoconfigured by Atoti Server.
Removed SameSiteConfig The SameSite config is handled by Spring.

Historical Date Provider Service

The historical dates used for the Lookback hierarchy measures are now configurable via a IHistoricalDatesProvider bean. By default, a HistoricalDatesProvider (configured in HistoricalDatesProviderConfig) will be created that provides all AsOfDates available in all base stores.

Sign-Off REST services

The implementation of the Sign-Off REST service now has a Boolean to enable state flag. Out of the box, the Sign-Off REST services are disabled until the initial load is completed. This prevents the Sign-Off server from sending requests on those services before the end of the initial load. The status of the Sign-Off service can be retrieved through a GET request at /enabled.

Atoti Server exceptions are now used instead of the previously used Javax exceptions.

Error messages and constants have been fixed. Their prefix and/or content was previously incorrect.In particular, the constant ERROR_MESSAGE has been replaced by the constant ERROR_MESSAGE_NO_VALID_DTO_PROVIDED. The exception message prefix has been changed from "[EXPORT]" to "[SIGN-OFF]" in the error message constants.

Relocation of DoctorPivot

DoctorPivot has been moved from the endpoint /frtb-starter/doctorpivot to now being embedded within the AdminUI’s Measures dependencies tab which is located at: /frtb-starter/admin/ui/index.html#/measures-dependencies.