Migration helper

Migration helper

The migration helper is a set of REST endpoints that automate the one-off task of seeding a DirectQuery database from an existing in-memory Atoti Market Risk deployment. It generates the SQL DDL that matches the Atoti Market Risk datastore schema and extracts the in-memory data into database-ready CSV files.

Typical workflow

1. Start Atoti Market Risk in in-memory mode and load the dataset you want to migrate.
2. Call the migration helper endpoints to produce the DDL and CSV files (see below).
3. Apply the DDL to your target database and load the CSV files.
4. Restart Atoti Market Risk with the DirectQuery profile pointing at the now-populated database (see Getting started).

Endpoints

The controller is mounted at host:port/mr-application/migrationHelper/ and exposes:

Names are mapped through MrDirectQueryNameMapper, so tables and columns in the generated DDL follow the same convention as the runtime DirectQuery schema (e.g. TradePnLs → TRADEPNLS, not TRADE_PN_LS).

The following stores are excluded from the generated schema by default (see MRMigrationHelperRestController.getDefaultExtractionBuilder()):

• Parent-child stores: BookParentChild, CounterpartyParentChild, LegalEntityParentChild — these are used to populate their hierarchy stores and are not needed in the DQ schema.
• Summary cube base stores: BaseStore, SensiBaseStore, PnLBaseStore — summary cubes are not supported under DirectQuery.
• The WhatIf simulation persistence table.
• Sign-off companion stores for TradePnLs, TradeSensitivities, PnL.

To customise the set of exported stores, subclass MRMigrationHelperRestController and override getDefaultExtractionBuilder().

Offline DDL generation

GET /migrationHelper/sqlSchema returns the DDL without opening a database connection or extracting data. It is useful to hand the schema to a DBA, to review the generated SQL in pull requests, or to bootstrap an environment where the target database is not yet provisioned.

curl "http://localhost:10010/mr-application/migrationHelper/sqlSchema?database=SNOWFLAKE&databaseName=MR_DB&schema=PUBLIC"

If databaseName or schema are omitted, the controller falls back to the Spring properties directquery.database.<type>.database and directquery.database.<type>.schema (<type> being snowflake, mssql, or databricks), then to spring.application.name, and finally to the constants returned by getDatabaseName() / getSchemaName() on the controller. For Atoti Market Risk these resolve to MR_DATABASE and MR_SCHEMA (see MRMigrationHelperRestController). MSSQL’s schema property additionally has a built-in default of dbo; for Snowflake and Databricks, the corresponding Spring properties have no Java-side default — PUBLIC (Snowflake) and default (Databricks) are JDBC connection-level defaults applied by the database server, not Spring fallbacks.

Vector emulation modes

Atoti Market Risk contains four vector tables — TradePnLs, TradeSensitivities, SensiLadders, and MarketShifts. For each table you can choose how the migrator represents the vector fields in the target database:

Modes can be selected at runtime through the REST endpoint, or programmatically on the Migrator bean:

Per-store, via REST

POST /migrationHelper/extract with body:

{
  "databaseType": "SNOWFLAKE",
  "vectorStoreBehaviours": {
    "MarketShifts": "ROW_SAME_TABLE",
    "SensiLadders": "COLUMN"
  },
  "columnVectorSizes": {
    "SensiLadders": { "Sensitivities": 1000 }
  }
}

The autoRowSameTable flag opts every key-only-plus-vector table into ROW_SAME_TABLE automatically — useful when many of your stores fit that shape. Per-store entries in vectorStoreBehaviours always win over autoRowSameTable.

Per-store, on the Migrator bean

migrator.vectorsTableBehaviour(MARKET_SHIFT_STORE_NAME, VectorType.ROW_SAME_TABLE);
migrator.vectorsTableBehaviour(SENSI_LADDER_STORE,      VectorType.COLUMN);
// Or, in one call:
migrator.autoRowSameTable(true);

To change the default used for DDL generation:

directquery:
  schema:
    default-vector-size: 500

Configuration properties

Example:

mr:
  migration:
    export:
      enabled: true
      base-dir: /data/mr-exports