Migration REST service

Migration REST Service

The Migration REST Service can be used to assist in the migration to using a Solution with DirectQuery.

The REST service is available at: host:port/migrationHelper/

Implementing the AMigrationHelperRestController

The AMigrationHelperRestController class is the base class for the Migration REST service. Extensions will need to implement the abstract methods that are unique to the implementing Solution.

Extending getDefaultExtractionBuilder()

This method is used to create the default extraction builder for the implementing Solution which defines the schemas and store to extract from the in-memory Solution. The builder can be configured to export all stores of a specific schema, or only specific stores. Additionally isolated stores can be provided as well.

Here is an example implementation of the getDefaultExtractionBuilder() method:

@Override public DatabaseExtractionDTO.DatabaseExtractionDTOBuilder getDefaultExtractionBuilder() {
return DatabaseExtractionDTO.builder()
        .onlyIncludeStarSchemaStores(true)
        .cubeSchemasToExport(List.of("FirstCube", "SecondCube"))
        .extraStores(List.of("Isolated Store 1", "Isolated Store 2"))
        .disableVectorStores(List.of("DisableVectorsForThisStore"))
        .storesToIgnore(List.of("Dont Export This Store"));
}

Security Configuration

A security configuration is required if you need to limit the use of this service to specific users.

Here is an example of allowing only ‘ADMIN’

@Bean
public SecurityFilterChain dqMigrationHelperFilterChain(HttpSecurity http) throws Exception {
    return http
            .securityMatcher(url("/migrationHelper", WILDCARD))
            .authorizeHttpRequests(auth -> auth
                    .requestMatchers(HttpMethod.OPTIONS, url("/migrationHelper", WILDCARD))
                    .permitAll()
                    .anyRequest()
                    .hasAnyAuthority(ROLE_ADMIN)
            )
            .securityContext((sc) -> sc.requireExplicitSave(false))
            .with(jwtAuthenticationConfigurer, Customizer.withDefaults())
            .build();
}

Database Extraction

This service allows you to export Atoti Common Library’s data into a format that is ready for your database of choice.

This service can extract for the following databases:

• ClickHouse
• Databricks
• MSSQL
• Snowflake

To export data into a format ready to be loaded into ClickHouse, use the Template export. If you want to customize the stores to export, use the Custom export.

Database Template Extraction

URL: host:port/migrationHelper/extractForDatabase/xxx

Where XXX is of the following databases you want to extract for:

• clickhouse
• databricks
• mssql
• snowflake

Request Type: POST

Body: LocalFileOutputHandler

{
    "outputDirectory": "full/path/to/extraction/directory",
    "compressionType": "NONE" / "GZIP"
}

Request Body: LocalFileOutputHandler

Custom Database Extraction

A custom extraction can be provided to export specific stores and datastore schemas.

The following is the default configuration for the /extract REST service to extract data for a database:

{
  "databaseType": "XXX",
  // One of the supported databases (all uppercase). For example, "CLICKHOUSE".
  "onlyIncludeStarSchemaStores": true,
  "cubeSchemasToExport": [
    "SACube",
    "BACube"
  ],
  "extraStores": [
    "FXRates",
    "Attributes"
  ],
  "storesToIgnore": [
    "BookHierarchy"
  ],
  "localFileOutputHandler": {
    "outputDirectory": "path/to/extraction/directory",
    "compressionType": "NONE"
  }
}

Request Body: DatabaseExtractionDTO