Migration notes 5.4

This page explains the changes required to migrate to the stated version of Atoti Market Risk.

Migrate to 5.4.0-BETA

Upgrading from version 5.3.0 - see Atoti Market Risk 5.4 Release Notes.

Headline announcement

  • Market Data API: The new Market Data API has been significantly updated with new features. It has also been moved to dedicated modules, the market-data-api library module and the market-data-api-spring-boot-starter module for easy auto-configuration. The changes include:
    • MarketDataDateShift enum: Changed the MarketDataDateShift enum value TODAY to CURRENT_DAY to clarify its meaning.
    • Interpolators: Two new interpolators are now accessible from the `IInterpolationService:
      • BilinearInterpolator lets you perform linear interpolation on a 2D surface of values.
      • TrilinearInterpolator lets you perform linear interpolation on a 3D cube of values.
    • Changed new Market Data model: The MarketDataSet and ID fields in the new Market Data data model have been swapped around.
  • Atoti Server: Upgraded to Atoti Server 6.0.11-sb3.
  • Spring Boot 3: Upgraded to Spring Boot 3.2.0, which uses Spring Framework 6.
  • Hibernate: The application has been upgraded to Hibernate 6.3.1.Final.
  • H2: The application has been upgraded to H2 2.2.224.
  • Security configuration: Some classes related to security configuration have been moved from the mr-common-config project to the mr-application project.
  • Spring Profiles: Profile-based configuration has been replaced with properties (backwards compatible). For details, see Profile-based configuration.
  • Fluent post-processors: Post-processor metrics are now set up in a Fluent style. The levels and hierarchies use strong types. For more details, see Fluent post-processor setup.
  • Specific alias classes for lambda generics: Lambda generics have been replaced with specific alias classes (e.g. MeasurePublisher for Consumer intended for publishing measures).
  • Topic Aliases: The topic aliases used by the Data Load Controller (DLC) are now defined for each topic rather than in a single place during DLC initialization.
  • New moneyness default value: The default value of the moneyness field in the sensitivities and market data has been changed from ATM to N/A.
  • Catalog name The catalog has been changed from MarketRiskAccelerator to MR.
  • Combined cube metrics: The metrics of the combined cube are now defined in the class MRCombinedCubeMeasuresConfig, their definition in pure Copper is also provided but should ne be used. For details, see The MRCombinedCube metrics.
  • Property cleanup: The unused property SENSITIVITY_TYPE has been removed in the DynamicTenorsAndMaturitiesPostProcessor class.
  • Added a new abstract tuple publisher: Introduced new abstract tuple publisher (ATableFormatTuplePublisher) for publishers where the file format does not match the table data model. For details, see New abstract tuple publisher.
  • Admin UI: The Admin UI has been upgraded to version 5.1.5.
  • PnL explain with DirectQuery: PnL Explain is now working across the transition between DirectQuery and In-Memory sensitivity cubes. For details, see PnL explain with DirectQuery.
  • UI Activation: An import of react-query is required when using mr-sdk. See UI activation for more details.

Breaking changes

  • A number of classes named ...Helper(s) have been renamed to ...Utils for naming consistency with similar classes.
  • All Utils and Constants classes, as well as other classes containing only static variables and/or methods, now have a private constructor.
  • com.activeviam.mr.common.calculations.adjustments.constants.AdjustmentsConstants has been removed and the two constants from this class added to com.activeviam.mr.common.constants.AdjustmentsConstants.
  • ASecurityConfig.PIVOT_USER_ROLES has changed from String[] to List<String>.
  • The METRIC variable on a number of measure chain classes is now a static constant in place of a final instance variable.
  • Fields on two What-If classes are now final, with corresponding getter/setter methods available.
  • The lambda generic classes have been replaced by specific classes.
  • Some classes related to security configuration have been moved from the mr-common-config project to the mr-application project.
  • In the mr-common-config project, the class UnsecuredDatabaseServiceConfig has been renamed to WhatIfAwareUnsecuredDatabaseServiceConfig.
  • The issue described in the ticket MR-1626 (trying to log in with a user that does not have admin rights throws a 403 HTTP error) has been fixed in the class MRSecurityFilterChainsConfig.
  • The NoOpPostProcessor is defined in the common lib, the package is now com.activeviam.accelerator.common.postprocessor.impl.
  • The tuple publishers that target multiple stores, or stores with a data model distinct from the input file format now extend ATableFormatTuplePublisher.
  • The unused property SENSITIVITY_TYPE has been removed in the DynamicTenorsAndMaturitiesPostProcessor class.
  • The topic aliases used by the Data Load Controller (DLC) are now defined for each topic rather than in a single place during DLC initialization. The ChannelParametersHolder::addTopic now has a 4th parameter that holds the list of all topic aliases to which the topic belongs. The function DataLoadControllerConfig::registerTopicAliases computes the aliases based on this extension and shouldn’t be modified anymore. For details, see Step 3 in Data loading and unloading.
  • Replaced the Atoti Market Risk ReferenceLevelLocationShift post-processor with the Common Library ReferenceLevelLocationShiftLight. Any usage of the post-processor will have to be replaced, as will any usage of the mr.metrics.rlshift.post-processor property referring to the old plugin key. The default value of the property has been changed to ReferenceLevelLocationShiftLight.
  • The application has been upgraded to Hibernate 6.3.1.Final and H2 2.2.224. As a consequence, the content of the content server database has changed. See Migration of the content server database.
  • The catalog name has been renamed from “MarketRiskAccelerator” to “MR”.
  • The new configuration class ApplicationJwtConfig has been added to handle the configuration used for JWT authentication.
  • The constant VALUE_CCY in StoreFieldConstants has been removed and replaced by the constant CCY.
  • The MRCombinedCube metrics are now configured by beans with the qualifier SP_QUALIFIER__COMMON_MEASURE_BUILDERS and SP_QUALIFIER__COMMON_MEASURES.
  • The interface IAsyncTask has been replaced by IRefreshTaskthat is more specific and used to perform refresh action upon DirectQuery refresh request.

Breaking changes in the Market Data API (formerly Preview Market Data API)

  • The classes that make up the new Market Data API (introduced in Atoti Market Risk 5.3.0) have been moved from the mr-common-config and mr-common-lib modules to new market-data-api and market-data-api-spring-boot-starter modules. The packages of these classes have also changed. However, the sources for the market data stores remain in the mr-sensi-config module. A description of the new modules is given below.
  • The mr.enable.preview.market-data property has been removed, together with the Spring @Conditional classes that were based on this property. To use the new Market Data API, you must now remove the following auto-configuration exclusions from MarketRiskApplication:
    • CubeMarketDataRetrievalConfig.class,
    • CurveMarketDataRetrievalConfig.class,
    • FxRateMarketDataRetrievalConfig.class,
    • InstrumentMarketDataRetrievalConfig.class,
    • SurfaceMarketDataRetrievalConfig.class,
    • AllMarketDataInterpolatorsConfig.class
    • AllInterpolationTransformationsConfig
  • The I...MarketDataHandler interfaces and their corresponding implementations have been removed.
  • The I...Retriever interfaces now return interface variations of the appropriate market data. For instance, ICubeMarketDataRetriever.getCube now returns ICubeMarketData in place of CubeMarketData.
  • The IMarketDataRetriever interface and corresponding TableSingleMarketDataRetriever implementation has been removed. Retrieval of single market data values is now handled by the relevant I...Retriever interfaces.
  • The IMarketDataService interface has been removed. This reflects the fact that the different market data types are retrieved by distinct interfaces and cannot therefore be collected in a common service. The corresponding TableMarketDataService has been removed.
  • The MarketDataRetrievalService class has been removed and is replaced by the market data retrieval interfaces described below.
  • The IMarketDataKeyTranslator interface has been renamed to IMarketDataCoordinateTranslator.
  • The DatabaseMarketDataKeyTranslator which implemented IMarketDataKeyTranslator<Object[]> has been removed. This has been replaced by DateShiftTranslator implements IMarketDataCoordinateTranslator<LocalDate> which applies the same shift to the asOfDate of market data.
  • The IDateRetrievalService has been renamed to IDateRetriever. The implementation has been renamed from TableDateRetrievalService to TableDateRetriever.
  • The IMarketDataDateService has been renamed to IContextualDateRetriever. The implementation has been renamed from MarketDataDateService to DateShiftingDateRetriever.
  • The MarketDataDateShift enum value TODAY has been renamed to CURRENT_DAY to clarify its meaning.
  • The default value of the moneyness field in the sensitivities and market data has been changed from ATM to N/A. Data in which ATM is the intended value will have to add the value explicitly.
  • The MarketDataSet and ID fields in the new Market Data API data model have been swapped around.
  • The IInterpolator interface has been split into an IInterpolatorSettings interface which is extended by the IInterpolator interface (with less responsibility) and a new IInterpolatorFactory interface.
  • The interpolator classes in com.activeviam.mr.common.services.marketdata.interpolation.interpolators.commonsmath.impl have been updated to reflect the interface changes described above. Each interpolator class now has a corresponding ...InterpolatorFactory class conforming to the IInterpolatorFactory interface.
  • The names of two interpolators have changed to more accurately reflect their intent. The LinearBicubicInterpolator is now the BicubicInterpolator and the LinearTricubicInterpolator is now the TricubicInterpolator.
  • The LinearUnivariateInterpolator class has been removed. Linear interpolation along one axis now supported with the LinearInterpolator class that works for variable numbers of axes.
  • The behavior of the existing interpolators has been changed with respect to the way they handle requests to interpolate coordinates outside their known range of coordinates. Previously this would have caused an OutOfRangeException to be thrown. Now the interpolators modify the requests to use the closest coordinate within the existing range.
  • The ...InterpolatorConfig classes in com.activeviam.mr.common.services.marketdata.interpolation now construct beans of type IInterpolatorFactory instead of IInterpolator.

Profile-based configuration

The code in Atoti Market Risk will no longer change behavior based directly on Spring profiles. Instead, the behavior will be determined by properties. The properties can be set by profile-selected configuration files. Default property files are included to preserve backwards compatibility with the use of profiles.

In particular, instead of the profiles dist-data-node, dist-query-node, and forceNetty (and the property directquery.enabled), we have the following properties:

Property Description Values
starter.deployment.type Specifies the nature of the database used for the cube. It can take three values:
  • in-memory (default): The in-memory datastore is used.
  • direct-query: An external database is used. This replaces the directquery.enabled property.

    note

    An in-memory datastore is still used for configuration data and caching.

  • query-node: No database is used.
starter.deployment.transport Specifies the messenger to use for communication between cubes. It can take two values:
  • local (default): For communication only within a single machine. It cannot be used for distribution, but is faster if distribution isn’t needed.
  • netty: For jgroups communication between machines in a distributed deployment.
  • none: For no communication bus, when no query cube is used.
mr.enable.cubes.common=false An existing flag which turns the MRCombinedCube polymorphic cube on or off.

The following configuration files have been added to match the behavior of the profiles:

Spring profile Configuration file starter.deployment.type starter.deployment.transport combined.disable
default in-memory local unset (false)
dist-data-node application-dist-data-node.yaml in-memory netty true
dist-query-node application-dist-query-node.yaml query-node netty unset (false)
forceNetty application-forceNetty.yaml unset (in-memory) netty unset (false)

See the Externalized Configuration section of the Spring boot documentation for more information on profiles and property management.

The MRCombinedCube metrics

The polymorphic metrics of the combined cube (using metrics from several underlying data cubes) are now defined by beans.

The configuration is located on MRCombinedCubeMeasuresConfig configuration class.

The metrics could be defined in the legacy way under the qualifier @Qualifier(SP_QUALIFIER__COMMON_MEASURE_BUILDERS) that will be aggregated by the cube configuration.

They can also be defined in Copper style with the qualifier @Qualifier(SP_QUALIFIER__COMMON_MEASURES). The Copper for polymorphic metrics is an alpha feature and is only present for demonstration.

The Copper for polymorphic metric is activated by the environment variable activeviam.feature.experimental.copper_in_distributed_cube.enabled=true. The spring condition @ConditionalOnExperimentalCopperInDistributedCube checks that the environment variable is set. The spring condition @ConditionalOnNotExperimentalCopperInDistributedCube checks that the environment variable is not set.

PnL explain with DirectQuery

By setting-up Atoti Market Risk using In-Memory node and DirectQuery nodes, the transition between the two nodes is now correctly handled for the PnL explain computation. As before, the dates defined in the property mr.data-load.initial-business-dates are loaded on the In-Memory node and NOT loaded in the DirectQuery node. However, on the DirectQuery node, the FXRates and MarketData stores are now loaded from both DirectQuery for the non-defined dates and from the files for the defined dates in the property. By doing this, the Day PnL Explain computation can fetch the market data for Day-1 and Day on the DirectQuery node of Day-1.

This is done by adding in memory sources on DirectQuery setup:

Condition changes

Storage source Previously used condition Common condition (DQ & In-Memory) In-Memory only condition DirectQuery only Condition
Local @ConditionalOnLocalData @ConditionalOnLocalSource @ConditionalOnLocalData @ConditionalOnDirectQueryWithLocalSource
Azure @ConditionalOnAzureData @ConditionalOnAzureSource @ConditionalOnAzureData @ConfigurationDirectQueryWithAzureSource

New Sources

A new source configuration dedicated to DirectQuery has been added by the configuration classes SensiPnLAzureCsvSourceConfig, SensiPnLLocalCsvSourceConfig, and DirectQuerySensiCsvSourceParametersProviderConfig. It loads the FX and market data on in-memory based for the dates defined by the property mr.data-load.initial-business-dates. It uses the same topics and aliases as the in-memory setup (Attributes/AllAttributes/FXRates" for FX, DailySensiConfig/SensiConfig/AllSensi/AllAttributes/AllFacts/MarketData for Market Data). A refresh issued to DirectQuery will also reload the initial data.

New qualifiers

To handle this source, several qualifiers have been added:

Constant Qualifier Type Source
SP_QUALIFIER__DQ_SENSI_TOPIC_PUBLISHERS “dq-sensi-topic-publishers” Publisher generator Function Sensi for DirectQuery
SP_QUALIFIER__DQ_SENSI_TOPIC_COLUMNS “dq-sensi-topic-columns” Columns generator Function Sensi for DirectQuery
SP_QUALIFIER__DQ_SENSI_TOPIC_TO_STORE_AND_FILE_MAP “dq-sensi-topic-to-store-and-file-map” Definition of the topics Sensi for DirectQuery

Configuration classes

Files Modified

Application configuration

Three classes have been renamed from ...Helper(s) to ...Utils for consistency with similar classes. In MR, a Utils class is one that contains only static methods. These three classes are:

Old class name New class name
DatastoreDescriptionHelper DatastoreDescriptionUtils
MeasureChainHelpers MeasureChainUtils
SchemaHelpers SchemaUtils

We have also consolidated two Utils classes. com.activeviam.mr.common.calculations.adjustments.constants.AdjustmentsConstants has been removed and the two constants in this class have been moved to com.activeviam.mr.common.constants.AdjustmentsConstants. These constants are unchanged, and you simply need to update the import statements.

Utils classes and others, such as Constants classes, that contain only static variables and/or methods are not intended to be instantiated. Therefore, for clarity, we have added private constructors to these classes. They are as follows:

  • ComponentVaRCalc
  • CookieUtil
  • DatastoreConstants
  • DatastoreDescriptionUtils
  • DimensionConstantsConfig
  • DimensionUtils
  • MeasureChainHelpers
  • MeasureConstants
  • SchemaConstants
  • SignOffConstants
  • StoreUtils
  • WhatIfConstants
  • WhatIfUtils

The type of the constant ASecurityConfig.PIVOT_USER_ROLES has changed from String[] to List<String>, where the list is an unmodifiable list. This is to ensure the immutability of this constant.

Measures configuration

A number of abstract measure chains have a METRIC field, representing the metric that the chain is responsible for creating measures for. On all these classes this is now a static constant in place of a final instance variable. These chains are:

  • AESChain
  • AETGChain
  • AHDChain
  • ASummaryESChain
  • ASummaryETGChain
  • ASummaryVaEChain
  • ASummaryVaRChain
  • ASummaryWESChain
  • ASummaryWETGChain
  • ASummaryWVaEChain
  • ASummaryWVaRChain
  • AVaEChain
  • AVaRChain
  • AWESChain
  • AWETGChain
  • AWVaEChain
  • AWVaRChain
What-If configuration

A number of instance variables have been made private on What-If configuration classes. In all cases there is a corresponding getter and setter that was already available and is now required to be used in all cases. These fields are:

Class name Field name
FileUploadPayload filePathSeparator
FileUploadPayload topics
FileUploadTopic name
FileUploadTopic files
Security configuration

The following files have been moved from the mr-common-config project to the mr-application project:

File name Old package New package
IUserLogoutSuccessHandler com.activeviam.mr.common.config.security.filter com.activeviam.mr.application.config.security
CookieUtil com.activeviam.mr.common.config.security com.activeviam.mr.application.config.security
CorsConfig com.activeviam.mr.common.config.security com.activeviam.mr.application.config.security
JwtAuthenticationConfigurer com.activeviam.mr.common.config.security com.activeviam.mr.application.config.security
SameSiteConfig com.activeviam.mr.common.config.security com.activeviam.mr.application.config.security
SecurityJwtProperties com.activeviam.mr.common.config.security com.activeviam.mr.application.config.security
UserDetailsServiceConfig com.activeviam.mr.common.config.security com.activeviam.mr.application.config.security

In the mr-common-config project, the class UnsecuredDatabaseServiceConfig has been renamed to WhatIfAwareUnsecuredDatabaseServiceConfig.

The following files, which were not used in the code, have been removed from the mr-common-config project:

  • CommonWebSecurityFiltersConfig
  • ApplicationJwtConfig
  • GlobalSecurityConfig
  • InMemoryAuthenticationSecurityConfig
  • InMemoryUserDetailsManagerBuilder
  • PasswordEncoderConfig

The class MRSecurityFilterChainsConfig has been changed to fix the known issue described in the ticket MR-1626 (trying to log in with a user that does not have admin rights throws a 403 HTTP error).

The class DispatcherServletConfig has been added in com.activeviam.mr.application.config.security to contain cookie configuration taken from the MarketRiskApplication class.

Market Data configuration

Configuration classes for the new Market Data API have been moved from the mr-common-config module to the new market-data-api-spring-boot-starter module. By adding this module as a dependency, the interpolator services will be auto-configured. The configured classes are:

Old class New class Changes
CubeMarketDataStoreConfig CubeMarketDataRetrievalConfig This configuration class now creates an IContextualCubeMarketDataRetriever in place of a `MarketDataRetrievalService.
CurveMarketDataStoreConfig CurveMarketDataRetrievalConfig This configuration class now creates an IContextualCurveMarketDataRetriever in place of a `MarketDataRetrievalService.
InstrumentMarketDataStoreConfig InstrumentMarketDataRetrievalConfig This configuration class now creates an IContextualInstrumentMarketDataRetriever in place of a `MarketDataRetrievalService.
SurfaceMarketDataStoreConfig SurfaceMarketDataRetrievalConfig This configuration class now creates an IContextualSurfaceMarketDataRetriever in place of a `MarketDataRetrievalService.
- FxRateMarketDataRetrievalConfig This new configuration class creates a store and retrievers for FX market data.
Interpolation configuration

All configuration of interpolation within the new Market Data API now resides in the new market-data-api-spring-boot-starter module. By adding this module as a dependency, the interpolator services will be auto-configured.

The ...InterpolatorConfig classes now create beans of type IInterpolatorFactory instead of IInterpolator. These factories can be used to construct the required interpolators.

In this same package, a BilinearInterpolatorConfig class and a TrilinearInterpolatorConfig class have been added which create factories for interpolators covering two and three axes respectively.

Preview Market Data property removal

The mr.enable.preview.market-data property has been removed, together with the following Spring @Conditional classes based on it:

Class Condition Replacement
@ConfigurationEnabledPreviewMarketData mr.enable.preview.market-data is true market-data-api-spring-boot-starter is a dependency of the current project.
@ConfigurationDisabledPreviewMarketData mr.enable.preview.market-data is false
@BeanEnabledPreviewMarketData mr.enable.preview.market-data is true market-data-api-spring-boot-starter is a dependency of the current project.
@BeanDisabledPreviewMarketData mr.enable.preview.market-data is false
@ConditionalOnPreviewMarketDataEnabled mr.enable.preview.market-data is true market-data-api-spring-boot-starter is a dependency of the current project.
@ConditionalOnPreviewMarketDataDisabled mr.enable.preview.market-data is false

Input file formats

Modified

Modification File Field Optional Description
Changed default MarketData.csv MoneynessLabels Y List of moneyness labels, if applicable. The default value is now N/A.
Changed default VannaSensitivities.csv, CrossGammaSensitivities.csv, CorrelationSensitivities.csv MoneynessLabels Y List of moneyness labels, if applicable. The default value is now N/A.
Changed default DeltaSensitivities.csv, GammaSensitivities.csv, VegaSensitivities.csv, VolgaSensitivities.csv, CashSensitivities.csv, ThetaSensitivities.csv MoneynessLabels Y List of moneyness labels, if applicable. The default value is now N/A.
Changed header VannaSensitivities.csv, CrossGammaSensitivities.csv, CorrelationSensitivities.csv None N/A The header ValueCCy has been replaced by Ccy.
Changed header DeltaSensitivities.csv, GammaSensitivities.csv, VegaSensitivities.csv, VolgaSensitivities.csv, CashSensitivities.csv, ThetaSensitivities.csv None N/A The header ValueCCy has been replaced by Ccy.
Changed default Instrument_Market_Data.csv MoneynessLabels Y List of moneyness labels, if applicable. The default value is now N/A.
Changed fields Instrument_Market_Data.csv InstrumentId, MarketDataSet - Changed order to MarketDataSet, InstrumentId.
Changed default Curve_Market_Data.csv MoneynessLabels Y List of moneyness labels, if applicable. The default value is now N/A.
Changed fields Curve_Market_Data.csv CurveId, MarketDataSet - Changed order to MarketDataSet, CurveId.
Changed default Surface_Market_Data.csv MoneynessLabels Y List of moneyness labels, if applicable. The default value is now N/A.
Changed fields Surface_Market_Data.csv SurfaceId, MarketDataSet - Changed order to MarketDataSet, SurfaceId.
Changed default Cube_Market_Data.csv MoneynessLabels Y List of moneyness labels, if applicable. The default value is now N/A.
Changed fields Cube_Market_Data.csv CubeId, MarketDataSet - Changed order to MarketDataSet, CubeId.

Configuration

Configuration properties

Properties added

The following properties have been created for the new market-data-api-spring-boot-starter module:

Property Default value Description
market-data-api.defaults.tenor “N/A” Default value for tenor coordinates. This is equivalent to the mr.tenors-and-maturities.default-value property of the old market data API.
market-data-api.defaults.moneyness “N/A” Default value for moneyness coordinates. This is equivalent to the mr.moneyness.default-value property of the old market data API.
market-data-api.defaults.maturity “N/A” Default value for maturity coordinates. This is equivalent to the mr.tenors-and-maturities.default-value property of the old market data API.
market-data-api.fx.pivot-currency “EUR” Where no FX rate exists between two currencies, a third (pivot) currency provides rates against both and is used to derive a rate between them. This is equivalent to the mr.fx.common-currency property of the old market data API.
Properties modified
Property Name Comment Value
mr.moneyness.default-value The default value has changed to N/A from ATM. N/A
Properties removed

The mr.enable.preview.market-data property has been removed.

Property Files

Files Modified
application.yaml

The following properties have been added to address cors issues and the removal of the distribution Spring profiles:

Key Value Description
server.servlet.session.cookie.secure true Enables/disables always marking the cookie as secure.
server.servlet.session.cookie.http-only true Enables/disables using the “HttpOnly” attribute for the cookie.
server.servlet.session.cookie.same-site none SameSite setting for the cookie.
starter.deployment.type in-memory Specifies the nature of the database used for the cube.
starter.deployment.transport local Specifies the messenger to use for communication between cubes.
Files Deleted
tracing.properties

This property file has been deleted and replaced by configuration properties. In addition, due to swapping the Brave tracing configuration for OpenTelemetry, properties have been updated.

The new properties are:

Property Name Comment Value
mr.enable.tracing.open-telemetry Enables the OpenTelemetry Configuration. false
mr.open-telemetry.zipkin-span-exporter-url URL to use for zipkin SpanProcessor bean. http://localhost:9411/api/v2/spans
mr.open-telemetry.logging-span-exporter-enabled Enables the SpanProcessor bean with LoggingSpanExporter. false

The following properties have been replaced by OpenTelemetry properties:

  • spring.zipkin.baseUrl
  • spring.zipkin.api-path
  • spring.zipkin.service.name
  • spring.zipkin.sender.type
  • spring.zipkin.enabled
  • spring.sleuth.traceId128
  • activeviam.apm.zipkin.span.level

Datastores

Added store configurations

A new store configuration is available in the market-data-api module. This is store is not currently created in Atoti Market Risk by default, which continues to use the old market data store for the present, but are available for those who wish to use the new Market Data API.

Store Class Details
FxRateMarketDataStore FxRateMarketDataStore A store that is part of the new Market Data API for FX Rate market data.

Modified store configurations

The following store configurations in the market-data-api module have been modified. These stores are not currently created in Atoti Market Risk by default, which continues to use the old market data store for the present, but are available for those who wish to use the new Market Data API.

Modification Store Class Field Type Description
Reordered CubeMarketDataStore CubeMarketDataStore MarketDataSet String This now precedes CubeId.
Reordered CubeMarketDataStore CubeMarketDataStore CubeId String This is now preceded by MarketDataSet.
Reordered CurveMarketDataStore CurveMarketDataStore MarketDataSet String This now precedes CurveId.
Reordered CurveMarketDataStore CurveMarketDataStore CurveId String This is now preceded by MarketDataSet.
Reordered InstrumentMarketDataStore InstrumentMarketDataStore MarketDataSet String This now precedes InstrumentId.
Reordered InstrumentMarketDataStore InstrumentMarketDataStore InstrumentId String This is now preceded by MarketDataSet.
Reordered SurfaceMarketDataStore SurfaceMarketDataStore MarketDataSet String This now precedes SurfaceId.
Reordered SurfaceMarketDataStore SurfaceMarketDataStore SurfaceId String This is now preceded by MarketDataSet.
Secondary index added CubeMarketDataStore CubeMarketDataStore MarketDataSet String A secondary index has been added on the AsOfDate, MarketDataSet, and CubeId fields.
Secondary index added SurfaceMarketDataStore SurfaceMarketDataStore MarketDataSet String A secondary index has been added on the AsOfDate, MarketDataSet, and SurfaceId fields.

Cube schema

No changes.

Measures

No changes.

Context values

No changes.

Other changes

Code changes in mr-application

A number of code-level changes have been made to classes in the mr-application module to improve clarity and consistency. These are:

  • LocalContentService.createHibernateConfiguration now throws an ActiveViamRuntimeException instead of a RuntimeException.
  • StaticResourcesHandler.CLASSPATH is a new private constant with the value "classpath:".
  • Nested if conditions have been removed from AdjustedCellPropertiesHandler.getForeColor\getBackColor.
  • The static variable MarketRiskApplication.context, of type ConfigurableApplicationContext, has been removed.
  • The JMXEnabler bean named JMXDatastoreEnabler, defined in MarketRiskConfig has been renamed to jmxDatastoreEnabler.
  • The JMXEnabler bean named JMXDirectMemoryMonitoringEnabler, defined in MonitorConfig has been renamed to jmxDirectMemoryMonitoringEnabler.
  • The private method performTransaction has been added to the AdjustmentExecutionConfig class to avoid the nesting of try-catch statements.
  • In a number of places, where a local variable is created and assigned just to be returned by a method, the local variable is no longer created. For instace, this has been changed for the getConfiguredStageTopics method of the FileUploadWhatIfRestfulService class.
  • The main method of the InvokeBranchRelease class now uses the Map.of method to store the JMX credentials, instead of instantiating a HashTable.
  • The new configuration class ApplicationJwtConfig has been added to handle the configuration used the properties defined for JWT authentication.
  • Deprecated description IStoreFormat is replaced by IDataTable.

Market Data API changes

The new Market Data API that was introduced as a preview in Atoti Market Risk 5.3.0 has undergone significant changes to add functionality and improve flexibility.

To reflect the independence of the API, it has been moved to a new Maven module: market-data-api. This module is not dependent on any MR modules, although it has a dependency on the Datastore Helper (DASH) and DASH is required to use the store configurations. This module is a dependency of the mr-common-lib module and so it’s classes are available throughout the MR project.

In addition to the base module, which provides interfaces and implementation classes, there is a market-data-api-spring-boot module. This contains Spring bean definitions for market data stores and retrievers, and interpolation services. As a Spring Boot starter, when you add this module as a dependency to a Spring Boot application, the beans within it will be automatically configured by Spring and made available to the application context. The following classes from the starter module will be automatically imported:

  • com.activeviam.marketdata.api.configuration.retrievers.CubeMarketDataRetrievalConfig
  • com.activeviam.marketdata.api.configuration.retrievers.CurveMarketDataRetrievalConfig
  • com.activeviam.marketdata.api.configuration.retrievers.FxRateMarketDataRetrievalConfig
  • com.activeviam.marketdata.api.configuration.retrievers.InstrumentMarketDataRetrievalConfig
  • com.activeviam.marketdata.api.configuration.retrievers.SurfaceMarketDataRetrievalConfig
  • com.activeviam.marketdata.api.configuration.interpolation.AllMarketDataInterpolatorsConfig
  • com.activeviam.marketdata.api.configuration.interpolation.AllInterpolationTransformationsConfig

The starter module is a dependency of the mr-sensi-config module. But, as Atoti Market Risk continues to use the old market data API for this release, each of the above classes is disabled from auto-configuration within the MarketRiskApplication class.

In addition to supporting cube, curve, instrument, and surface market data, FX rates are now supported and a corresponding FxRateMarketDataStore now exists.

The structure of the new Market Data API has also changed significantly from the previous release. The interfaces for retrieving market data from stores are different. Previously, a common IMarketDataRetriever interface allowed you to retrieve single values from market data stores and specific retrievers, e.g. ICubeMarketDataRetriever interfaces supported retrieval of market data collections (cubes, curves, etc.). Instead, each market data type now has a specific, corresponding retriever interface that supports retrieval from that store using the semantics relevant to it. For instance, the ICurveMarketDataRetriever stores now retrieves a single value from a curve or a whole curve, and the method parameters reflect the properties of a curve.

In addition to the base retriever interfaces, each market data type also now has a “contextual” variant, e.g. IContextualCurveMarketDataRetriever. This retriever accepts a MarketDataRetrieverContext that may modify the retrieval. Out of the box implementations apply the date shift from the context before retrieving the market data.

Beyond retrievers, the IMarketDataService interface has been renamed to IMarketDataRetrievalService. A retrieval service contains the business rules to retrieve market data for a specific purpose. It may use multiple underlying retrievers or interpolation services to do this. We provide an abstract implementation of this interface that handles caching within a query cache.

Finally, an AMarketDataPostProcessor is now provided that should provide the basis for retrieval of market data within a measure chain.

Lambda interface changes

Old interface New interface Usage
BiFunction<ICanStartBuildingSchema, IDatastoreSchemaDescription, BuildableActivePivotSchemaDescriptionBuilder> CubeBuilderFunction All the beans with @Qualifier("Cube").
DoubleBinaryOperator ShiftScalingFunction IInterpolationConfiguration::normalizationShiftOperator
BinaryOperator PrePostFunction IInterpolationConfiguration::getPreMarketDataInterpolationFunction
BinaryOperator PrePostFunction IInterpolationConfiguration::getPostMarketDataInterpolationFunction
BinaryOperator PrePostFunction IInterpolationConfiguration::getPreShiftDataInterpolationFunction
BinaryOperator PrePostFunction IInterpolationConfiguration::getPostShiftDataInterpolationFunction
DoubleBinaryOperator AbsoluteToRelativeFunction IInterpolationConfiguration::absoluteToRelativeMarketDataOperator
DoubleBinaryOperator RelativeToAbsoluteFunction IInterpolationConfiguration::relativeToAbsoluteMarketDataOperator
DoubleBinaryOperator ShiftScalingFunction IInterpolationConfiguration::normalizationShiftOperator
BiFunction<Double, ICorporateAction, Double> CorporateActionOperator IInterpolationConfiguration::corporateActionOperator
UnaryOperator<ICanBuildCommonCubeDescription> QueryCubeAggregateProviderConfig Bean with @Qualifier("MRCombinedCubeConfiguration").
UnaryOperator<ICanBuildCubeDescription> AggregateProviderConfig All beans with the corresponding return signature.
ICustomParameters.ILocationFunction LocationFunction ICustomParameters::getLeafCoordinatesFunction
BiConsumer<String, Collection<Object[]» DatastorePublisher A function with the behavior of ITransactionalWriter::addAll.
Supplier Constructor Every time a constructor without arguments needs to be provided.
Supplier NotNullCacheSupplier Supplier of function PostProcessorUtils::getInCacheOrComputeNeverNull.
Supplier NullableCacheSupplier Supplier of function PostProcessorUtils::getInCacheOrCompute.
BiFunction<ILocation, IQueryCache, List> LeafFunction Return type of ICustomLeaf::apply.
Function<Double, Pair<T, String» DataRetrieverFunction Parameter of IInterpolation::interpolation used to get the underlying data.
UnaryOperator SimpleNameConverter Simple naming convention change, used for DirectQuery.
BiPredicate<IPillar, IPillar> EqualsFunction ModifiedPillarSetOfPillar equals function.
BiFunction<LocalDate, Boolean, LocalDate> DateSearchStepper Used in PostProcessorUtils::dateSearch to go to the next date.
Function<LocalDate, T> DateSearchRetriever Used in PostProcessorUtils::dateSearch to get the value for a given date.
UnaryOperator AdjustmentConverterOperator Used to define basic operation for the adjustment.
BiFunction<Map<String, Object>, List, U> AdjustmentInputConverter Converts the user request into a pre-processed input request for the adjustment.
BiFunction<AdjustmentRequestDTO, SupportedAdjustmentDTO, Map<String, Object» AdjustmentInputParser Transforms a user request into adjustment values.
TriFunction<Object[], IStoreFormat, List, T> AdjustmentInputRetriever Pre-processing of a line before adjustment.
TriFunction<String, String, String, Map<String, Object» AdjustmentSourceTagger Fills the adjustment columns with the type of adjustment.
BiFunction<T, U, List> AdjustmentStep<T, U, V> One step of adjustment to modify a store line.
BiFunction<Map<String, Object>, List, Double> DoubleAdjustmentInputConverter Converts the user request into a pre-processed input request for the adjustment.
TriFunction<Object[], IStoreFormat, List, Double> DoubleAdjustmentInputRetriever Pre-processing of a line before adjustment.
BiFunction<Double, Double, List> DoubleAdjustmentStep One step of adjustment to modify a store line.
Consumer DatastoreConfiguratorConsumer Used for applying customizations to the IDatastoreConfigurator object.
Function<String, Set> BookRetriever Helper function for adjustment executions that retrieves a set of books belonging to a desk.
BiFunction<String, String, Set> TradeRetriever Helper function for adjustment executions that retrieves a set of trades belonging to a specific desk and book.
Supplier FileUploadAddressSupplier Used by the file upload What-If implementation to get the addresses for file upload services present in a distributed cluster.
BiFunction<IDatastoreSchemaDescription, String, T> BaseSelectionCreator Used to configure schema selection based on a datastore schema and the name of the base store.
UnaryOperator ExtraSelectionApplier Used to apply extra schema selections.
Function<IDatastoreSchemaDescription, ISelectionDescription> SelectionDescriptionCreator Used to create a selection description based on a datastore schema description.
Function<IActivePivotManagerDescriptionBuilder, IActivePivotManagerDescriptionBuilder> CatalogCreator Used to add catalogs to the AP manager description builder.
Consumer CopperCubePublisher Used to add Copper cube components to the Copper context.
Consumer MeasurePublisher Used to publish measures to the Copper context.
Function<ICanStartBuildingMeasures, IHasAtLeastOneMeasure> MeasureAdder Adds a measure through the fluent API.
Function<T, String> NameRetriever Selects a measure name from the names object (e.g. SensiMeasureNames).
Function<T, String> ParameterRetriever Selects a parameter from the parameters object (e.g. SensiMeasureParameters).
Consumer DatastorePostprocessorConfigurator Used to add datastore postprocessors to the datastore.
Consumer DataLoadControllerConfigurator Used for customizing the Data Load Controller.
BiFunction<String, Map<String, Object>, ICondition> StoreAndScopeToConditionConverter Converts a store name and a DLC scope to an ICondition.
Function<String, IFilesScopedTopic.IScopeToFileScanParametersConverter> TopicToScopeToLoadConverter Matches a topic name to a DLC scope-to-file-scan converter.
Function<String, IDataLoadController.IScopeToRemoveWhereConditionConverter> TopicToScopeToRemoveWhereConditionConverter Matches a topic name to a DLC scope-to-remove-where-condition converter.
Function<IStoreMessageChannelFactory<I, ?>, IMessageChannel<I, ?» MessageChannelCreator Creates a message channel from the given factory.
BiConsumer<AdjustmentRequestDTO, String> AdjustmentExecutor Object that contains the execution steps of the adjustments.
BiConsumer<AdjustmentDeletionRequestDTO, String> AdjustmentDeleter Object that contains the execution steps of the adjustments deletion.

Wherever the changes above have resulted in a removal of ambiguity, the associated Spring @Qualifier has been removed:

Qualifier Replacement interface alias
@Qualifier(SP_QUALIFIER__CUSTOMISATIONS) DatastoreConfiguratorConsumer
@Qualifier(“Catalog”) CatalogCreator
@Qualifier(“Cube”) CubeBuilderFunction
@Qualifier(“DLC-registration”) DataLoadControllerConfigurator
@Qualifier(“DLC-source”) Removed as ICsvSource is generic

Other interface changes

Old interface New interface Usage
Pair<U, String> Result Return type of IInterpolation::interpolation

New API to define dimension and hierarchies

You can now define a new hierarchy belonging to an existing dimension by registering an extra bean, previously this was only possible at dimension level. To be able to define a new hierarchy in a cube inside an existing dimension, the signature of the dimension beans has changed. It now returns HierarchyBuilderConsumer, replacing DimensionsAdder. This return type allows to define a hierarchy by itself.

Here’s how it works:

    // The return object will be consumed at cube creation
    @Bean
    @Qualifier("commonDimension")
    public HierarchyBuilderConsumer bookingDimension() {
        return BookingDimensionConfig::getBookingDimension;
    }

    // We can group the declaration inside the bean
    public static void getBookingDimension(@NonNull HierarchyBuilder hierarchyBuilder) {
        getDesksHierarchy(hierarchyBuilder);
        getSrcCurrencyHierarchy(hierarchyBuilder);
        getDateHierarchy(hierarchyBuilder);
    }

    // This is a vanilla hierarchy
    public static void getDesksHierarchy(@NonNull HierarchyBuilder hierarchyBuilder) {
        hierarchyBuilder.toDimension(BOOKING_DIMENSION, builder -> builder.withHierarchy(DESKS_HIERARCHY).withLevel(DESK_LEVEL));
    }

    // This is an analysis hierarchy with a shorthand compared to the previous syntax
    public static void getSrcCurrencyHierarchy(@NonNull HierarchyBuilder hierarchyBuilder) {
        hierarchyBuilder.toDimension(CURRENCY_DIMENSION, FxHierarchy.hierarchy(SRC_FX_HIERARCHY).withLevel(SRC_FX_LEVEL));
    }

    // Here you can see the set of an extra property at dimension level
    public static void getDateHierarchy(@NonNull HierarchyBuilder hierarchyBuilder) {
        hierarchyBuilder.withType(DATES_DIMENSION, TIME);
        hierarchyBuilder.toDimension(DATES_DIMENSION, b -> b.withHierarchy(DATE_HIERARCHY).withLevel(AS_OF_DATE_LEVEL).withType(ILevelInfo.LevelType.TIME));
    }

New abstract tuple publisher

We have introduced the ATableFormatTuplePublisher for sources where the input file format does not match the table data model.

Publishers extending this abstract class can define a mapping between the table fields and the input file fields. The tuples will be generated according to matching field names and this custom mapping.

Upgrade to Admin UI 5.1.5

The Admin UI has been upgraded to version 5.1.5 fromversion 5.1.1. A new property has been added to the market-risk POM file for the Admin UI version:

  <admin-ui.version>5.1.5</admin-ui.version>

and the admin-ui artifact has been added in the dependency management section of that POM file:

<dependency>
        <groupId>com.activeviam.tech</groupId>
        <artifactId>admin-ui</artifactId>
        <version>${admin-ui.version}</version>
</dependency>

and the hard-coded version number has been removed in the mr-application POM file for that artifact:

<dependency>
    <groupId>com.activeviam.tech</groupId>
    <artifactId>admin-ui</artifactId>
</dependency>

Unused property removed in AsOfDateNeighbourValuePostProcessor

The unused property IS_PARTITIONED_ON_RANGE_LEVELS_PROPERTY has been removed from the class AsOfDateNeighbourValuePostProcessor.

MarketDataDateShift enum

The MarketDataDateShift enum value TODAY has been renamed to CURRENT_DAY to clarify its meaning.

Deprecated unused column calculator

The unused SensiValueCalculator is now deprecated.

Unused property removed in DynamicTenorsAndMaturitiesPostProcessor

The unused property SENSITIVITY_TYPE has been removed from the DynamicTenorsAndMaturitiesPostProcessor class.

The signatures of the methods that used to have the sensitivityType parameter as an argument have been changed to remove that argument. As a consequence, the default implementation of the method rebucketing in the INativeCurrencyMeasures interface has been changed.

New configuration class ApplicationJwtConfig

The new configuration class ApplicationJwtConfig has been added to handle the configuration used for JWT authentication. This class can also handle the activeviam.jwt.failOnDifferentAuthorities property that was not taken into account previously. It is imported in the class MarketRiskConfig instead of the previously used JwtConfig class.

Tomcat server version bumped to 10.1.19

In the mr-application module, this version override of the spring-boot-starter-web dependency is required to customize the use of cookies to allow websocket connections.

Migration of the content server database

The application has been upgraded to Hibernate 6.3.1.Final and H2 2.2.224. As a consequence, the content of the content server database has changed. To export your existing bookmarks from MR 5.3.0 and import them into the MR 5.4.0-beta application, please follow these steps:

  1. Launch your MR 5.3.0 application.
  2. Export the bookmarks from your MR 5.3.0 application by executing the main method of the class ExportReferenceContentServer in the module mr-application-tests. You may need to change the values of the properties used for bookmark export - see mr.bookmark.properties. The bookmarks will be exported in the folder specified in the value of the contentServer.bookmarks.export-folder-name property.
  3. Launch the MR 5.4.0-beta application with the following properties:

    warning

    Please note that this will create a new content server database from scratch and erase your existing database.

 mr.application.content-server.db.hibernate.hbm2ddl.auto=create
 mr.application.content-server.reset=folder
 mr.application.content-server.factory-reset.folder-name=file:full_path_to_the folder_where_bookmarks_were_exported

Check in Atoti UI that the imported bookmarks are present.

  1. Stop the MR 5.4.0 application and copy the mr_ref_impl_content_service.mv.db file from the target folder of the mr-application module to the src/main/resources folder of the mr-application module.
  2. Launch the MR 5.4.0 application without the following properties being set:
    mr.application.content-server.db.hibernate.hbm2ddl.auto
    mr.application.content-server.reset
    mr.application.content-server.factory-reset.folder-name

You should now see your imported bookmarks in Atoti UI.

Fluent post-processor setup

The measure creation function returning a CopperMeasure is now deprecated, for instance the old definition:

    default MeasureCreator pnlDistribution() {
        return underlier -> PnLDistributionPostProcessor.measure(
                        underlier[0],
                        getMeasureParameters().getPercentileLevel(),
                        getMeasureParameters().getPnlDistributionConfigurationService(),
                        false)
                .withFormatter(getMeasureParameters().getFormatters().getPercentFormat())
                .withinFolder(getMetricParameters().getValueAtRiskFolder() + "\\" + getMetricParameters().getVarFolder())
                .as(getMetricParameters().getPnlDistribution());
    }

is now replaced by a Fluent way of defining a custom measure:

    default MeasureCreator pnlDistribution() {
        return underlier -> PnLDistributionPostProcessor.measure()
                .withUnderlyingMeasure(underlier[0])
                .withAnalysisLevels(getMeasureParameters().getPercentileLevelId())
                .withNumberOfBuckets(getMeasureParameters().getPnlDistributionConfigurationService().getNumberOfBuckets())
                .withIsAccumulate(false)
                .withFormatter(getMeasureParameters().getFormatters().getPercentFormat())
                .withinFolder(getMetricParameters().getValueAtRiskFolder() + "\\" + getMetricParameters().getVarFolder())
                .as(getMetricParameters().getPnlDistribution());
    }

The functions providing levels as string are also deprecated and replaced by a function that returns a LevelIdentifier. Those functions are suffixed with “…Id”. The Property partitionedOnRangeLevels=false has been removed from all post-processor setups.

Constant VALUE_CCY in StoreFieldConstants removed and replaced by constant CCY.

In the mr-common-lib module, the VALUE_CCY constant in StoreFieldConstants has been removed, and the constant CCY is used instead, because those two constants had the same value: Ccy.

In the mr-pnl-config module, the following classes now use the CCY constant instead of the VALUE_CCY constant:

  • SupportedAdjustmentsPnLConfig
  • PnLStoreConfig
  • PnLAggregatedStoreConfig
  • PnLFlatStoreConfig
  • SignoffSummaryPnlPublisher
  • SummaryPnlPublisher

Catalog name change

The catalog has been renamed to MR from MarketRiskAccelerator. If you are using Atoti Limits with Atoti Market Risk, you need to change your UI settings. Please see the Atoti Limits doc here: https://docs.activeviam.com/products/modules/limits/beta/online-help/dev/integration/multi-server-activate.html