Migration notes 6.0
This page explains the changes required to migrate to the stated version of the Atoti FRTB.
Migrate to 6.0.3
Summary
- Virtual hierarchies can be set with the property
frtb.configuration.virtual-hierarchies
. - The new
Curvature CVR
measure will be disabled by default in 6.0.x versions of Atoti FRTB. To enable it, set the propertyfrtb.curvature-cvr.enabled=true
in theapplication.yaml
file. - Cube distribution can now handle data overlap. This means it is possible to have several data cubes with duplicated data. you need to set
frtb.distribution.enable-data-duplication=true
. HistoricalDatesProvider
no longer queries the base stores for the list of historical dates. This is achieved by using the database statistics to gather the set of unique dates. TheHistoricalDatesProvider
can be reverted to the previous behavior of executing distinct datastore queries by setting the propertyfrtb.historical-dates-provider.is-using-approximate-members=false
.- The zero risk-weight can now be specified per obligor and jurisdiction, defaulting to 0%.
Breaking changes
None.
Deprecations
- Cube distribution by field: Distribution of data cubes across nodes by defining clustering fields is deprecated, it should be done with distribution levels.
Input file formats
Added
File | Details |
---|---|
DRC_Public_Sector_Obligor_Risk_Weights.csv | Use this file to configure specific zero risk-weights per obligor and jurisdiction. |
Configuration properties
Properties added
Property | Default value | Description |
---|---|---|
frtb.configuration.virtual-hierarchies | Empty | A list of hierarchies to set as virtual. |
frtb.curvature-cvr.enabled | false | Enables the new Curvature CVR measures. |
frtb.distribution.levels.sa-cube | AsOfDate@Date@Dates | The levels used for the StandardisedApproachCube distribution. |
frtb.distribution.levels.ima-cube | AsOfDate@Date@Dates | The levels used for the InternalModelApproachCube distribution. |
frtb.distribution.levels.ima-drc-cube | AsOfDate@Date@Dates | The levels used for the IMADRCCube distribution. |
frtb.distribution.levels.pl-cube | AsOfDate@Date@Dates | The levels used for the PLCube distribution. |
frtb.distribution.levels.stress-calibration-cube | AsOfDate@Date@Dates | The levels used for the StressCalibrationCube distribution. |
frtb.distribution.priority.sa-cube | Empty | The priority of the StandardisedApproachCube in cases of data overlap within a horizontal distribution. |
frtb.distribution.priority.ima-cube | Empty | The priority of the InternalModelApproachCube in cases of data overlap within a horizontal distribution. |
frtb.distribution.priority.ima-drc-cube | Empty | The priority of the IMADRCCube in cases of data overlap within a horizontal distribution. |
frtb.distribution.priority.pl-cube | Empty | The priority of the PLCube in cases of data overlap within a horizontal distribution. |
frtb.distribution.priority.stress-calibration-cube | Empty | The priority of the StressCalibrationCube in cases of data overlap within a horizontal distribution. |
frtb.distribution.enable-data-duplication | false | Set to true to allow duplicate members in the distribution level of horizontally distributed data cubes. |
directquery.incremental-refresh | Empty | Configures increment refresh templates. |
frtb.historical-dates-provider.is-using-approximate-members | true | If true (default), the HistoricalDatesProvider will use the database statistics to gather the unique set of dates from the stores.If false, it will revert to the previous behavior of executing distinct datastore queries. |
frtb.historical-dates-provider.stores-to-scan | [PLTrades, IMATrades, DRCIMABase, SASensitivities, StressCalibrationTrades] | The list of stores to scan to collect the historical dates from. |
frtb.historical-dates-provider.as-of-date-field | AsOfDate | The name of the date field to use when querying the stores for historical dates. |
Properties deprecated
Property name | Comment |
---|---|
sa.cube.distributing.field | replaced by frtb.distribution.levels.sa-cube |
ima.cube.distributing.field | replaced by frtb.distribution.levels.ima-cube |
ima-drc.cube.distributing.field | replaced by frtb.distribution.levels.ima-drc-cube |
pl.cube.distributing.field | replaced by frtb.distribution.levels.pl-cube |
stress-calibration.cube.distributing.field | replaced by frtb.distribution.levels.stress-calibration-cube |
Properties files
Files Modified
frtb-data-load.properties
New properties:
Property Name | Comment | Value |
---|---|---|
drc.public-sector.obligor-risk-weights.file-pattern | File pattern used to load the PublicSectorObligorRiskWeights store. | **/DRC_Public_Sector_Obligor_Risk_Weights*.csv{,.gz} |
Datastores
Added stores
Store | Details |
---|---|
PublicSectorObligorRiskWeights | This store contains optional risk-weights for public sector obligors applied to zero risk-weight exposures in the DRC non-securitisations calculation. |
Databases
Added tables
Table | Details |
---|---|
PUBLIC_SECTOR_OBLIGOR_RISK_WEIGHTS | This store contains optional risk-weights for public sector obligors applied to zero risk-weight exposures in the DRC non-securitisations calculation. |
Measures
Added
Cube | Measure | Details |
---|---|---|
StandardisedApproach | Curvature CVR | This measure uses the underlying Curvature CVR Up or Curvature CVR Down measure depending on the “direction” of the Risk Position Scenario at the Bucket level. A new intermediate measure Curvature Bucket Scenario evaluates the Risk Position Scenario measure at the Book level. This measure has been added for the following risk classes within sa-sbm: - Commodity - CSR non-Sec - CSR Sec CTP - CSR Sec non-CTP - Equity - FX - GIRR |
StandardisedApproach | Crypto 2b | Support for Cryptoassets Group 2b |
Migrate to 6.0.2
No migration needed.
Migrate to 6.0.1
No migration needed.
Breaking changes
None
Summary
- Upgraded to Data Connectors 5.0.4
Migrate to 6.0.0
Upgrading from version 5.3.0, see Atoti FRTB 6.0 Release Notes.
Atoti FRTB uses Atoti Server 6.1.7 and Atoti UI 5.2.x. For new features and fixes included in these releases, please see the Atoti UI documentation and Atoti UI Migration Notes, and the release notes for Atoti Server.
Breaking changes
- Measure Configuration: The text replacement template for measure configuration now uses the pattern
#{_name_}
instead of${_name_}
. See Startup Properties. - frtb-application module: Some of the content of the
frtb-starter
module has been moved to the newfrtb-application
module. - DLC 5.0 Upgrade: With the new version of the DLC, the configuration of the ETL has been simplified.
- What-if endpoints: The (undocumented) what-if REST endpoints used by the Switch Desk Model and Switch Book Desk what-if simulations have changed.
- DoctorPivot endpoint: DoctorPivot is now embedded within the AdminUI. See Relocation of DoctorPivot for more.
Headline announcement
- Upgraded to Atoti Server 6.1.7 : Upgraded to the latest Atoti Server.
- Upgraded to JDK 21 : Upgraded from JDK 17 to JDK 21 to utilize the latest features.
- Removed ActiveMonitor : The Active-Monitor module and related features have been removed.
- frtb-application module: The
frtb-application
module replaces thefrtb-starter
module. All customizations should now be done in thefrtb-application
module, or in a new module based on thefrtb-application
module. - Sign-Off Service Availability: The Sign-Off REST service now throws
ServiceUnavailableException
s until the initial data loading phase has been completed. The Sign-Off REST service now provides a GET/enabled
endpoint to check if the service is currently enabled. - Simplified OpenTelemetry Configuration: We now rely on OpenTelemetry’s GlobalOpenTelemetry to configure the export of logs and metrics.
- OpenRewrite recipes As an experimental feature, we have added OpenRewrite recipes that you can use to automate some of the migration steps.
OpenRewrite recipes
note
We have included these OpenRewrite recipes as an experimental feature and would welcome your feedback on their usefulness in your migration. We can then evaluate the feature for potential inclusion in future releases. Please report any issues you have through the normal Jira channels.
To help with upgrades, we have created a set of OpenRewrite recipes that can be used to automatically perform some of the migration steps from 5.3 to 6.0.
The recipes included in this release are:
Recipe Name | Description |
---|---|
com.activeviam.frtb.Migrate_6_0 |
Runs all 6.0 migration recipes. |
com.activeviam.frtb.FileUpload_6_0 |
Updates the properties for the file-upload what-if. This includes:
|
com.activeviam.frtb.DQAutoVectorizer_6_0 |
Deletes the old, unused activeviam.directquery.enableAutoVectorizer property. |
com.activeviam.frtb.DataDirectories_6_0 |
Updates the properties configuring the data directories. This includes:
|
See Using OpenRewrite in Atoti FRTB for instructions on how to run the recipes.
frtb-application
module
The new frtb-application
Maven module has been split out of the frtb-starter
Maven module.
Components moved
The following components have been moved into the new module:
FRTBApplication
class including themain()
method.- Spring security configuration
- OpenTelemetry configuration
- Logging configuration
- Content Service configuration
- Configuration files
Configuration classes moved
For the full list of configuration classes that have
been moved into the frtb-application
module, see the Moved Classes section.
Data & Test folders moved
The sample data located at /frtb-starter/src/test/resources/
has been moved out of the frtb-starter
module to the /sample-data/
folder located at the root of the project.
The tests included in the frtb-starter
module have been moved into the new frtb-application-tests
module.
OpenTelemetry Configuration
The previous OpenTelemetry configuration has been replaced with a simple OpenTelemetry
bean.
To enable the autoconfiguration of this bean, set -Dotel.java.global-autoconfigure.enabled=true
on the command line.
FX Translation Risk
The field “Translation Risk Currency” has been added to support loading translation risk for different reporting currencies. It is now possible to load different translation risks for different reporting currencies.
This field is only used if FXComplexTrade=Y
(otherwise the translation risk is calculated automatically).
- If this field is empty, the sensitivity is not treated as translation risk but instead treated normally.
- If this field is set, then the sensitivity will be interpreted as translation risk for that currency. It will be filtered and only used if the translation risk currency is the same as the reporting currency.
Alternative root parameter sets
The property parameter-set.root
has been added to specify the root parameter set in the configuration and input files.
It is expected that input files are generated for the parameter set specified in this property.
Additionally, the configuration files containing parameters will need to be adapted to the new root parameter set.
- Previously, when no parameter set was specified in the configuration it was assumed to be “BCBS”. Now, these parameters will default to the specified root parameter set.
- Parameter configuration will need to be modified for the new root parameter set.
For example, for the bucket used for CSR non-Sec covered bonds we have in FRTBParameters.csv:
Name | Value | Date | ParameterSet |
---|---|---|---|
sa.csr-nonsec.bucket.covered-bonds |
8 | 2016-01-01 | |
sa.csr-nonsec.bucket.covered-bonds |
10 | 2016-01-01 | CRR2 |
If you want to set CRR2 as the root parameter set, the above two lines can change to become:
Name | Value | Date | ParameterSet |
---|---|---|---|
sa.csr-nonsec.bucket.covered-bonds |
8 | 2016-01-01 | BCBS |
sa.csr-nonsec.bucket.covered-bonds |
10 | 2016-01-01 |
So that the value in the CRR2 parameter set is still “10”, and the value for the BCBS parameter set is still “8”.
Summing Sensitivities
When multiple sensitivities with the same key fields are loaded, they are now summed. Previously this was only done when processing a chunk within a file, now it is done across the whole transaction.
The following configuration has been added to application.yaml
and will sum sensitivities and PVs for the SASensitivities
store,
making sure that the currency and a few other fields are the same.
datastore-sum-duplicates:
SASensitivities:
matching: ["Ccy", "RiskWeight", "FXOtherCcy", "Direction"]
summing: ["Sensitivity", "Shift_Up_PV", "Shift_Down_PV", "PresentValue", "Notional", "Adjustment"]
FRTB Modules
Removed
Module | Details |
---|---|
frtb-activemonitor |
ActiveMonitor and associated configurations have been removed from Atoti FRTB. |
Added
Module | Details |
---|---|
frtb-application |
Provides an example Spring Boot Application for Atoti FRTB. * Contains simple configurations required to run Atoti FRTB.* Is intended to be used as a reference for clients to build their own applications as this module is not production-ready. * Provides various Spring profiles outlining the properties required to run the application in different environments. For example, the profile dist-query-node activates the application-dist-query-node.yaml configuration file which defines the properties required to run the application as a distributed query node.tip To help ease upgrade efforts, move all customizations into your new module. |
DLC 5.0 upgrade
The upgrade to DLC 5.0 has brought some significant changes to the ETL configuration. For non-FRTB specific DLC migration, please see the DLC 5.0 Migration Guide.
DLC configurations for Atoti FRTB have now been organized into FrtbDlcConfig
.
Loading Topic Configurations
Before the DLC redesign, ChannelParameters
were being used to encapsulate DLC configurations into beans.
Now, configurations that were previously found as ChannelParameters
beans can be found as Topic Descriptions in the following classes:
Old Class | New Class |
---|---|
AConfigurationSourceConfig |
ConfigurationSourceDescription |
ASourceConfig |
SourceDescription |
ASensiSourceConfig |
SensiSourceDescription |
Column calculators are now configured as DLC CustomFieldDescription
beans and have been organized into CustomFieldDescriptionConfig
.
The DLC implicitly adds CSVColumnParser
column calculators based on the store being targeted,
so those have been removed from the topic configurations.
The new DLC also lets you configure parser overrides.
In Atoti FRTB they can be found in ColumnParserOverridesDescriptionConfig
.
Tuple publishers are now configured as DLC TargetDescription
beans and have been organized into TargetDescriptionsConfig
.
Unloading Topic Configurations
Unloading topics are now completely separate from loading topics. They are configured in UnloadTopicDescriptionsConfig
.
Topic Aliases
Old Class | New Class |
---|---|
SourceTopicAliases |
AliasesDescriptionConfig |
Topic Ordering Logic
Old Class | New Class |
---|---|
FRTBLoadDataTxControllerTask |
FrtbLoadOperation |
ETL Extension Updates
See the following sections on how the ETL extensions have changed with the new DLC:
- Adding and Populating a New Store
- Adding Columns to an Existing File and Store
- Enriching File Fields by Adding Column Calculators
Removed classes
See the list of classes removed as part of the DLC 5.0 upgrade.
Input file formats
Modified
Modification | File | Field | Optional | Description |
---|---|---|---|---|
Added | SBM_Delta_Sensitivities.csv | Translation Risk Ccy | Y | FX only. Indicates the sensitivity represents translation risk; set to the reporting currency. |
Added | SBM_Summary_Delta_Sensitivities.csv | Translation Risk Ccy | Y | FX only. Indicates the sensitivity represents translation risk; set to the reporting currency. |
Added | Trade_Attributes.csv | Inclusion | Y | Indicates if the trade should be included (“Y”) or not by default in the calculations. Default is ‘Y’ |
Startup properties
The file frtb-config.properties is now loaded into the Spring Boot environment, so the properties in this file can be overridden on the command line.
The file measures-config.json
is now also loaded into the Spring Boot environment. The json file is converted to properties with the prefix “measures”.
The text replacement template in these files now uses the pattern #{_name_}
instead of ${_name_}
.
The ${_name_}
pattern can still be used for Spring Boot’s property placeholders.
Property Name | Comment | Value | Default Definition in File |
---|---|---|---|
dlc.enabled |
Enables or disables the Data Load Controller. | true or false |
application.yaml |
file-upload.staging-directory |
The staging directory for the File Upload. | ${csv-source.dataset}/stage |
application.properties |
file-upload.csv-source-subdirectory |
The sub-directory of files on which the What-If is executed. | true or false |
application.properties |
csv-source.dataset-historical |
Path to the directory containing historical files to load. | ${csv-source.dataset}/historical | application.properties |
csv-source.dataset-configuration |
Path to the directory containing the configuration files to load. | sample-data/configuration | application.properties |
parameter-set.root |
Root parameter set. | Defaults to “BCBS” | |
datastore-sum-duplicates._store-name_.matching |
List of names of fields that must match when summing duplicates for the store store-name. If missing, all fields are matched; if empty, no fields are matched. |
||
datastore-sum-duplicates._store-name_.summing |
List of names of fields that should be summed when there are duplicates for the store store-name. | ||
directquery.aggregate-tables |
Configuration of the Aggregate Tables to be used when running with DirectQuery. | Mapping of aggregate tables per cube | application.yaml |
whatif.datastore.<identifier>.* |
Simple datastore changes what-if simulations. See Simple Datastore Changes. | inclusion-change what-if simulation definition. |
application.yaml |
Updated properties: changed values or renamed
Changed values:
Property Name | Comment | New Value | Old Value | Default Definition in File |
---|---|---|---|---|
activeviam.jwt.expiration |
Value changed from an int of seconds to a Duration . Note that an integer value will be picked up as milliseconds now. |
12h (Duration ) |
43200 (seconds) |
jwt.properties |
csv-source.dataset |
Split out to csv-source.dataset-historical , csv-source.dataset-configuration and csv-source.dataset |
sample-data/data | data | application.properties |
data.extraction.templates.base.dir.path |
Updated for new structure | frtb-application/src/main/resources/DEETemplateOrders | ./src/main/resources/DEETemplateOrders | application.properties |
Renamed
Old Property | New Property | Comment | Default Definition in File |
---|---|---|---|
csv-source.parserThreads |
csv-source.parser-threads |
The naming convention has changed from camel case to kebab case, but the old property key will still work. | application.properties |
csv-source.bufferSize |
csv-source.buffer-size |
The naming convention has changed from camel case to kebab case, but the old property key will still work. | application.properties |
aws.accessKeyId |
aws.accessKeyId |
Previously managed by FRTB, now inheriting AWS SDK property aws.accessKeyId in DefaultCredentialsProvider |
frtb-cloud.properties |
aws.secretKey |
aws.secretAccessKey |
Previously managed by FRTB, now inheriting AWS SDK property aws.secretAccessKey in DefaultCredentialsProvider |
frtb-cloud.properties |
azure.connection.string |
dlc.azure.connection-string |
Previously managed by FRTB, now inheriting DLC property dlc.azure.connection-string for DefaultAzureClientConfig |
frtb-cloud.properties |
Deleted properties:
Property Name | Comment | Deleted from File |
---|---|---|
activeviam.directquery.enableAutoVectorizer |
The property is no longer being used. | application.properties |
input.data.root.dir.path |
This is now a part of file-upload.staging-directory |
application.properties |
csvSource.subdirectory.dataset.stage |
This is now a part of file-upload.staging-directory |
application.properties |
csvSource.subdirectory.dataset.whatif |
This is now file-upload.csv-source-subdirectory |
application.properties |
cloudSource.dataset |
csv-source.dataset-historical , csv-source.dataset-configuration and csv-source.dataset are now shared between cloud sources and local |
frtb-cloud.properties |
cloud.fetch.thread |
No longer configurable | frtb-cloud.properties |
dlc.audit-logging.enabled |
DLC event handlers are no longer a part of the new DLC | application.properties |
Configuration files
Files Modified
FRTBParameters.csv
Modification | Parameter | Type | Default | Description |
---|---|---|---|---|
Deprecated (for removal) | sa.drc.no-maturity-floor-when-offsetting |
boolean | false |
Parameter is no longer needed ref |
frtb-config.properties
Modification | Parameter | Type | Default | Description |
---|---|---|---|---|
Changed Default | sa.drc.strict-long-short |
boolean | true |
Changed default to true. |
Datastores
Modified stores
Modification | Store | Field | Type | Description |
---|---|---|---|---|
Added | Trade Mapping | Inclusion | String | Indicates if the trade should be included (“Y”) or not by default in the calculations. |
Added | SaSensitivities | Translation Risk Ccy | STRING | FX only. Indicates the sensitivity represents translation risk; set to the reporting currency. |
Databases
Modified tables
Modification | Table | Field | Type | Description |
---|---|---|---|---|
Added | SASENSITIVITIES | TRANSLATION_RISK_CCY | String | FX only. Indicates the sensitivity represents translation risk; set to the reporting currency. |
Added | TRADE_MAPPING | INCLUSION | String | Indicates if the trade should be included (“Y”) or not by default in the calculations. |
Cube schema
Added
Cube | Dimension | Hierarchy | Levels | Datastore fields | Details |
---|---|---|---|---|---|
All | Booking | Inclusion | Inclusion | TradeMapping.Inclusion | This hierarchy is used to indicate which trades should be included in the calculations. |
Modified Classes
The following classes have been added, removed or had their function replaced with new classes.
Added Classes
Added | Rationale |
---|---|
DatastoreChangeSimulations |
WhatIf definitions have been restructured. |
SummingDuplicateKeyHandler |
DuplicateKeyHandler that sums tuples. |
TableSecurityConfig |
Configures the security roles for the actions that can be taken on the datastore. |
OpenTelemetryTracingConfig |
Configures the OpenTelemetry service for the application. |
Removed Classes
DataLoadController
The following classes related to the DLC and data loading have been removed as part of the DLC 5.0 upgrade.
WhatIf
The following classes have been replaced with DatastoreChangeSimulations
because the configuration of WhatIfs has changed.
Removed |
---|
BookDeskSwitchSubmissionDTO |
DeskModelSwitchSubmissionDTO |
BookDeskSwitchRestServiceController |
DeskModelSwitchRestServiceController |
BookDeskSwitchSimulationDefinition |
DeskModelSwitchSimulationDefinition |
Services
Removed | Replacement | Rationale |
---|---|---|
DataAvailability |
Distribution no longer requires relying on checking if data is available. | |
JsonDataLoader |
Unused. | |
FRTBI18nConfig |
Atoti Server autoconfigured starters. | This class is now autoconfigured by Atoti Server. |
Moved Classes
Class | Previous Module & Package | New Module & Package |
---|---|---|
LocalContentServiceConfig |
frtb-starter/com.activeviam.frtb.starter.cfg.impl |
frtb-application/com.activeviam.frtb.application.config.content |
RemoteContentServiceConfig |
frtb-starter/com.activeviam.frtb.starter.cfg.impl |
frtb-application/com.activeviam.frtb.application.config.content |
BasicAuthenticationProviderConfig |
frtb-accelerator/com.activeviam.frtb.ref.cfg.impl.security |
frtb-application/com.activeviam.frtb.application.config.security |
SecurityConfig |
frtb-accelerator/com.activeviam.frtb.ref.cfg.impl |
frtb-application/com.activeviam.frtb.application.config.security |
UserDetailsServiceConfig |
frtb-starter/com.activeviam.frtb.starter.cfg.impl.security |
frtb-application/com.activeviam.frtb.application.config.security |
Migrating to using Atoti Starters
Atoti Server now provides Spring Boot Starters to aid in application development. These starters will auto-configure some basic and required beans for the application to run. Atoti FRTB has been updated to use these starters. The following changes have been made:
Change | Description |
---|---|
Removal of JMXEnabler Beans |
The JMXEnabler beans have been removed from Atoti FRTB as the Atoti Server’s Starters provide these beans automatically for us. |
Removed FRTBI18nConfig Configuration Class |
This class is now autoconfigured by Atoti Server. |
Removed SameSiteConfig |
The SameSite config is handled by Spring. |
Historical Date Provider Service
The historical dates used for the Lookback
hierarchy measures are now configurable via a IHistoricalDatesProvider
bean. By default, a
HistoricalDatesProvider
(configured in HistoricalDatesProviderConfig
) will be created that provides all AsOfDate
s available in all base stores.
Sign-Off REST services
The implementation of the Sign-Off REST service now has a Boolean to enable state flag.
Out of the box, the Sign-Off REST services are disabled until the initial load is completed.
This prevents the Sign-Off server from sending requests on those services before the end of the initial load.
The status of the Sign-Off service can be retrieved through a GET request at /enabled
.
Atoti Server exceptions are now used instead of the previously used Javax exceptions.
Error messages and constants have been fixed. Their prefix and/or content was previously incorrect.In particular, the constant ERROR_MESSAGE
has been
replaced by the constant ERROR_MESSAGE_NO_VALID_DTO_PROVIDED
. The exception message prefix has been changed from "[EXPORT]"
to "[SIGN-OFF]"
in the
error message constants.
Relocation of DoctorPivot
DoctorPivot has been moved from the endpoint /frtb-starter/doctorpivot
to now being embedded within the AdminUI’s Measures dependencies tab which is located
at: /frtb-starter/admin/ui/index.html#/measures-dependencies
.