Migration REST Service

The Migration REST Service can be used to assist in the migration to using with DirectQuery. The REST service is available at: host:port/frtb-starter/migrationHelper/ The REST service contains the following components:

Data Migration

Security Configuration

The security configuration is handled by the DQMigrationHelperServiceSecurityConfig bean defined in the SecurityConfig. The default security config limits the use of this service to only ADMIN use.

Database Extraction

This service allows you to export ‘s data into a format that is ready for your database of choice. This service can extract for the following databases:

ClickHouse
Snowflake

ClickHouse Export

To export data into a format ready to be loaded into ClickHouse, use the Template export. If you want to customize the stores to export, use the Custom export.

ClickHouse Template Extraction

URL: host:port/frtb-starter/migrationHelper/extractForDatabase/clickhouse Request Type: POST Body: LocalFileOutputHandler

{
    "outputDirectory": "path/to/extraction/directory",
    "compressionType": "NONE"
}

Custom ClickHouse Extraction

A custom extraction can be provided to export specific stores and datastore schemas. The following is the default configuration for the /extract REST service to extract data for a ClickHouse database:

{
    "databaseType": "CLICKHOUSE",
    "onlyIncludeStarSchemaStores": true,
    "cubeSchemasToExport": ["StandardisedApproachCube"],
    "extraStores": [
        "DeskDescription",
        "FXRates",
        "LegalEntityParentChild", "BookParentChild",
        "TrancheOverrides", "ObligorOverrides", "RiskFactorDescriptionOverrides", "RRAOOverrides", "UnderlyingDescriptionOverrides"
        ],
    "storesToIgnore": ["BookHierarchy", "LegalEntityHierarchy"],
    "localFileOutputHandler" : {
        "outputDirectory": "path/to/extraction/directory",
        "compressionType": "NONE"
    }
}

Snowflake Export

To export data into a format ready to be loaded into Snowflake, the Template export can be used. If you would like to customize the stores to export, you can use the Custom export.

Snowflake Template Extraction

URL: host:port/frtb-starter/migrationHelper/extractForDatabase/snowflake Request Type: POST Body: LocalFileOutputHandler

{
    "outputDirectory": "path/to/extraction/directory",
    "compressionType": "NONE"
}

Custom Snowflake Extraction

The following is the default configuration for the /extract REST service to extract data for a Snowflake database:

{
    "databaseType": "SNOWFLAKE",
    "onlyIncludeStarSchemaStores": true,
    "cubeSchemasToExport": ["StandardisedApproachCube"],
    "extraStores": [
        "DeskDescription",
        "FXRates",
        "LegalEntityParentChild", "BookParentChild",
        "TrancheOverrides", "ObligorOverrides", "RiskFactorDescriptionOverrides", "RRAOOverrides", "UnderlyingDescriptionOverrides"
        ],
    "storesToIgnore": ["BookHierarchy", "LegalEntityHierarchy"],
    "localFileOutputHandler" : {
        "outputDirectory": "path/to/extraction/directory",
        "compressionType": "NONE"
    }
}

Request Objects

The following objects are used in the requests to the endpoints in this REST service.

`DatabaseExtractionDTO`

Parameter	Description
`databaseType`	One of the supported databases (all uppercase). For example, “CLICKHOUSE”.
`onlyIncludeStarSchemaStores`	If you want to export only the stores that are part of the Star Schema.
`cubeSchemasToExport`	The cube schemas you want to export.
`extraStores`	Additional stores to export.
`storesToIgnore`	Stores you do not want to include in the extraction.
`localFileOutputHandler`	The output handler. This writes the results into the local file system.

`LocalFileOutputHandler`

Parameter	Description
`outputDirectory`	Where to output the CSV files. This directory must already exist because the extractor cannot create new directories.
`compressionType`	The compression to apply to the file. Either “NONE” or “GZIP”.

​Migration REST Service

​Security Configuration

​Database Extraction

​ClickHouse Export

​ClickHouse Template Extraction

​Custom ClickHouse Extraction

​Snowflake Export

​Snowflake Template Extraction

​Custom Snowflake Extraction

​Request Objects

​DatabaseExtractionDTO

​LocalFileOutputHandler