Skip to main content

Migrate to 6.1

It is strongly recommended to begin by reviewing the what's new page, which explains some of the migration steps described here.

Important It is also strongly recommended to begin by clearing every use of deprecated methods from the Atoti library before attempting to migrate. Indeed, these methods have been, for the most part, removed in the 6.1 version. It will prove much easier to perform these small migration steps while the project still compiles, and to test that it still performs as expected, before going to the next step of this migration to 6.1.

Project configuration changes​

Atoti has been upgraded to:

  • Java 21
  • Spring 6
  • Spring Boot 3.2 These updates are necessary in order to follow the ecosystem's release cycle, particularly when it comes to security updates.

Spring Modules​

Considerable effort has gone into simplifying and streamlining the project configuration process in version 6.1.

Spring Boot Starters have been developed for Atoti, allowing the import of themed Maven modules instead of manually consolidating numerous configuration classes in the application.

The starters represent an "override or opt-out" of the configurations philosophy, rather than an "opt-in by importing" one that was previously implemented. More details can be found in the Starters documentation.

Migration steps will be provided in subsequent sections of this page. A considerable part of the Atoti modules that need to be imported may be replaced by one or two starters, increasing the maintainability of the project's pom.xml file.

Atoti Modules​

After updating the project's .pom files to use this new version of Atoti Server, some dependencies will need to be modified to align with the changes in the Atoti Server modules.

Modules com.activeviam.tech:composer-impl and com.activeviam.tech:composer-intf have been merged into a single com.activeviam.tech:composer-core module.

The relationship between the datastore and the sources (CSV, JDBC) has been inverted. These sources now depend on the datastore, and not the other way around. To utilize a specific source, the corresponding artifact must be separately imported:

  • CSV source: com.activeviam.source:csv-source
  • JDBC source: com.activeviam.source:jdbc-source
  • Parquet source: com.activeviam:parquet-source (not impacted by this change, it already had to be imported separately)

This inversion will reduce the size of the final jar by cutting out the transitive dependencies of the sources that the project does not require, and thus does not import.

Module com-activeviam-activepivot:activepivot-copper2-impl has been renamed com-activeviam-activepivot:activepivot-copper.

Module com-activeviam-activepivot:activepivot-copper2-test has been renamed com-activeviam-activepivot:activepivot-copper-test. It only contains internal and private_ packages, and should not be imported. A proper public testing module has been created for 6.1: com.activeviam:atoti-server-test, and will be discussed later on in this page.

JungSchemaPrinter has been moved to its own maven module com.activeviam.tech:datastore-schema-printer, in the package com.activeviam.database.datastore.schemaprinter.api.

Other Dependencies​

With 6.1, most of the dependencies of Atoti Server have been upgraded to their latest versions. To stay up to date with the security updates of these dependencies, this often implied major version changes. It is recommended to remove any version number that have been manually set in the project pom, and find a dependency convergence once again during this upgrade to 6.1.

Atoti Server's version lifecycle being longer than most OSS version lifecycle, dependency upgrades will continue throughout the lifetime of version 6.1.

ActiveMonitor​

ActiveMonitor no longer relies on org.apache.velocity.tools:velocity-tools-generic. This dependency allowed for dynamic message templates. Users can elect to re-add such this extension by overriding the Spring Bean returning VelocityTemplateEngine, and setting their own extensions through setExtensions(...).

Example: 

public class MessagingConfigurationOverride {
  @Bean
  public VelocityTemplateEngine velocityTemplateEngine() {
    final VelocityTemplateEngine engine = new VelocityTemplateEngine();
    final Map<String, Object> extensions = new HashMap<>();
    extensions.put("numberTool", new NumberTool());
    engine.setExtensions(extensions);
    return engine;
  }
  private void usage() {
    final VelocityTemplateEngine engine = new VelocityTemplateEngine();
    final String template = "${number} as ${numberTool.integer($number)}";
    final Map<String, Object> model = new HashMap<>();
    model.put("number", 1.2);
    engine.render(template, model); // Returns "1.2 as 1"
  }

It is recommended to replace the ActiveMonitor product by the Limits product, which concentrates most of the development efforts.

Sources​

Following the AWS end-of-support announcement for the Java SDK v1 effective December 31, 2025, the AWS cloud source has been updated to use the AWS Java SDK v2.

When using client side encryption, the metadata header x-amz-unencrypted-content-length must be updated to x-amz-meta-unencrypted-content-length when accessing S3. AWS recommends using the x-amz-meta- prefix for custom metadata headers, as it will not conflict with any other header they might add in the future. The AWS Cloud source has been updated to follow this recommendation.

Apache Arrow has been bumped from version 12 to version 17.

Distribution​

The JGroups dependency has been upgraded to the latest version available (5.3).

List of noticeable changes in JGroups XML config files:

  • max_bundle_size property does not exist anymore
  • enable_diagnostics property does not exist anymore
  • use_fork_join_pool property does not exist anymore
  • pbcast.STABLE.stability_delay property does not exist anymore
  • FD protocol does not exist anymore. One may use FD_ALL and FD_HOST instead, as described here
  • in AUTH, token parameters should now be prefixed with auth_token. Also, MD5Token does not exist anymore
  • org.jgroups.aws.s3.NATIVE_S3_PING protocol was renamed to aws.S3_PING. See more info here

Public and Internal APIs​

After upgrading the maven modules, the next step of the migration is to be able to compile the project.

Most of the classes of the Atoti Server library have been moved to new packages, following the process described in the what's new page. The Java API migration tool can be used to facilitate the migration. It will automatically update all imports with the new packages and class names.

The migration tool's output will provide a list of all the private APIs that the project relies on after this automatic migration. This list should be kept somewhere. If it still has relevant entries after manually finishing this migration to 6.1, please do contact ActiveViam's support.

REST Services​

The latest version of the Atoti REST API is 9.

Data Export Service​

A bug was found in the Data Export Service in tabular mode, causing empty locations to be incorrectly filtered out. You may need to update queries that relied on the old behavior.

To illustrate, let's take the following result of a basic SELECT as an example:

ParisNew York
January100
February350
March421

It would formerly be exported as:

January,Paris,100
February,New York,350
March,Paris,421

However, the correct export would be:

January,Paris,100
January,New York,
February,Paris,
February,New York,350
March,Paris,421
March,New York,

If one wants the first kind of result, they should use a query like:

SELECT NON EMPTY City.Members * Month.Members ON ROWS, Measures.X ON COLUMNS FROM Cube

Miscellaneous​

Renamed elements​

A lot of classes have been renamed as well as being moved to another packages. While the imports were automatically updated by the migration script, uses of these classes within the source code were not, for technical reasons.

Now that the imports compile properly, they can be used as the source of truth regarding how the code base should be migrated.

Acronyms​

The following acronyms have been modified to follow new code style standards.

Previous VersionNew Version
CSVCsv
JDBCJdbc
POJOPojo

Branding​

Most, if not all, of the references to the previous company name, QuartetFS, also abbreviated QFS, have been removed.

As previously stated, the corresponding imports should have been automatically updated, making the changes to the code base rather straightforward.

Annotations used for code injection were re-branded:

Previous VersionNew Version
QuartetExtendedPluginAtotiExtendedPlugin
QuartetExtendedPluginValueAtotiExtendedPluginValue
QuartetPluginAtotiPlugin
QuartetPluginValueAtotiPluginValue
QuartetTypeAtotiType

Moved and/or replaced properties​

ActiveViam Properties​

The class ActiveViamProperties, which references all the application properties that can be set to alter the configuration of the engine, has been updated:

KeyPrevious VersionNew VersionComment
CHUNK_ALLOCATOR_KEY_PROPERTYactiveviam.chunkAllocatorClassactiveviam.chunkAllocatorKeyThe values taken by this property were changed from class names to keys. The possible values are: "mmap", "array", "direct", "slab", "direct_buffer", "heap_buffer". "slab" is still the recommended and default.
DATA_CUBE_REST_ENDPOINT_PORT_PROPERTYactiveviam.distribution.endpoint.portSee the changes to distribution properties
DATA_CUBE_REST_ENDPOINT_SUFFIX_PROPERTYactiveviam.distribution.endpoint.suffixSee the changes to distribution properties
DATA_CUBE_REST_ENDPOINT_HOST_PROPERTYactiveviam.distribution.endpoint.hostSee the changes to distribution properties
DATA_CUBE_REST_ENDPOINT_PROTOCOL_PROPERTYactiveviam.distribution.endpoint.protocolSee the changes to distribution properties
EXTERNAL_DATABASE_QUERY_TIMEOUT_LIMITactiveviam.directquery.externalDatabaseQueryTimeoutLimitDatabase dependant. See classes com.activeviam.directquery.XXX.api.XXXClientSettings where XXX is a database product name.
ENABLE_DIRECTQUERY_AUTOVECTORIZERactiveviam.directquery.enableAutoVectorizercom.activeviam.database.api.settings.DiscovererSettingsSee also com.activeviam.directquery.api.schema.Vectorization and the how-to guide
DIRECTQUERY_AUTOVECTORIZER_DELIMITERactiveviam.directquery.autoVectorizerDelimitercom.activeviam.database.api.settings.DiscovererSettingsSee also com.activeviam.directquery.api.schema.Vectorization and the how-to guide
DIRECTQUERY_AUTOVECTORIZER_THRESHOLDactiveviam.directquery.autoVectorizerThresholdcom.activeviam.database.api.settings.DiscovererSettingsSee also com.activeviam.directquery.api.schema.Vectorization and the how-to guide
MANAGER_FEEDING_TIMEOUTactiveviam.directquery.cubeFeedingTimeoutInSecondsDatabase dependant. See classes com.activeviam.directquery.XXX.api.XXXDatabaseSettings where XXX is a database product name.
DIRECTQUERY_AUTO_ADAPTIVE_JITactiveviam.directquery.autoAdaptiveJitRemoved.
DIRECT_QUERY_SUB_QUERY_LIMITactiveviam.directquery.subQueryLimitDatabase dependant. See classes com.activeviam.directquery.XXX.api.XXXDatabaseSettings where XXX is a database product name.
DIRECT_QUERY_GET_BY_KEY_BEHAVIORactiveviam.directquery.GetByKeyBehaviorDatabase dependant. See classes com.activeviam.directquery.XXX.api.XXXDatabaseSettings where XXX is a database product name.
SNOWFLAKE_MAX_RESULTSET_SIZEactiveviam.directquery.snowflake.maxresultsetsizecom.activeviam.directquery.snowflake.api.SnowflakeClientSettings
AGGRESSIVE_AXIS_POSITION_LIMIT_CHECK_PROPERTYactiveviam.mdx.result.aggresiveAxisPositionLimitCheckMoved to the Context Value MdxContext to be configurable on a per-query basis

Application Properties​

activeviam.jwt.* properties have been renamed into atoti.jwt.* properties. Moreover, two properties have been renamed:

Previous VersionNew Version
activeviam.jwt.generateatoti.jwt.enabled
activeviam.jwt.check.user_detailsatoti.jwt.check_user_details

Static constants​

Previous VersionNew VersionComment
com.activeviam.copper.ProviderCoordinate#GLOBAL_PROVIDER_NAMEcom.activeviam.activepivot.core.intf.api.description.IAggregateProviderDefinition#GLOBAL_PROVIDER_NAME
com.quartetfs.biz.pivot.query.aggregates.impl.StoredMeasureHandler#PLUGIN_TYPEcom.activeviam.activepivot.core.intf.api.realtime.IAggregatesContinuousHandler#BASIC_HANDLER_PLUGIN_KEYHandler is now internal.
com.quartetfs.biz.pivot.query.aggregates.impl.StoredPrimitiveMeasureHandler#PLUGIN_TYPEHandler is now internal. Re-implement IAggregatesContinuousHandler if needed.
com.quartetfs.biz.pivot.query.aggregates.impl.TransactionStreamFullRefreshHandler#PLUGIN_TYPEcom.activeviam.activepivot.core.intf.api.realtime.IAggregatesContinuousHandler#FORCE_FULL_REFRESH_PLUGIN_KEYHandler is now internal.
com.quartetfs.biz.pivot.query.aggregates.impl.MultiAnalysisHierarchyMeasureHandler#PLUGIN_TYPEcom.activeviam.activepivot.core.intf.api.realtime.IAggregatesContinuousHandler#MULTI_ANALYSIS_HIERARCHY_MEASURE_HANDLER
com.quartetfs.biz.pivot.query.aggregates.impl.CommitIsolatedStoreHandler#PLUGIN_TYPEcom.activeviam.activepivot.core.intf.api.realtime.IAggregatesContinuousHandler#createCommitOnIsolatedStoreHandlerPluginKey(String)CommitIsolatedStoreHandler is a variable plugin value.
com.quartetfs.biz.pivot.query.aggregates.impl.TransactionStream#PLUGIN_TYPEcom.activeviam.activepivot.core.intf.api.realtime.IStream#ACTIVEPIVOT_PLUGIN_KEYTransactionStream is internal.
com.quartetfs.biz.pivot.query.aggregates.impl.CommitStoreStream#PLUGIN_TYPEcom.activeviam.activepivot.core.intf.api.realtime.IStream#createCommitOnStoreStreamPluginKey(String)CommitStoreStream is variable plugin value.
com.qfs.messenger.impl.LocalMessenger#PLUGIN_KEYcom.activeviam.activepivot.core.intf.api.description.IMessengerDefinition#LOCAL_PLUGIN_KEY
com.qfs.messenger.impl.NettyMessenger#PLUGIN_KEYcom.activeviam.activepivot.core.intf.api.description.IMessengerDefinition#NETTY_PLUGIN_KEY
com.quartetfs.biz.pivot.cube.hierarchy.IAnalysisHierarchy#LEVEL_TYPES_PROPERTYcom.activeviam.activepivot.core.intf.api.description.IAxisLevelDescription#ANALYSIS_LEVEL_TYPE_PROPERTYUsed on a per-level basis instead of a per-hierarchy basis
com.quartetfs.biz.pivot.cube.hierarchy.ILevel#AXIScom.activeviam.activepivot.core.intf.api.cube.hierarchy.IHierarchy#AXIS
com.quartetfs.biz.pivot.cube.hierarchy.ILevel#BRANCH_LEVEL_NAMEcom.activeviam.activepivot.core.intf.api.cube.hierarchy.IHierarchy#BRANCH_LEVEL_NAME
com.quartetfs.biz.pivot.cube.hierarchy.ILevel#EPOCH_LEVEL_NAMEcom.activeviam.activepivot.core.intf.api.cube.hierarchy.IHierarchy#EPOCH_LEVEL_NAME
com.quartetfs.biz.pivot.cube.hierarchy.ILevel#ALLMEMBERcom.activeviam.activepivot.core.intf.api.cube.hierarchy.IHierarchy#ALLMEMBER
com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AxisHierarchyBase#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_PROPERTYcom.activeviam.activepivot.core.intf.api.description.IAxisHierarchyDescription#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_PROPERTY
com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AxisHierarchyBase#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_ALWAYScom.activeviam.activepivot.core.intf.api.description.IAxisHierarchyDescription#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_ALWAYS
com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AxisHierarchyBase#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_IF_EMPTYcom.activeviam.activepivot.core.intf.api.description.IAxisHierarchyDescription#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_IF_EMPTY
com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AxisHierarchyBase#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_NEVERcom.activeviam.activepivot.core.intf.api.description.IAxisHierarchyDescription#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_NEVER

Spring Boot Starters​

As mentioned in a previous section, Version 6.1 has introduced Spring Boot Starters to ease the configuration of an Atoti project. These starters offer a default configuration for an Atoti project. A considerable part of the org.springframework.context.annotation.Configuration objects imported in a 6.0 project can be removed from the org.springframework.context.annotation.Import of the application configuration.

Services​

ActivePivotXmlaServletConfig has been completely removed. The underlying servlet has been transformed into a fully fledged REST controller. Importing the starter com.activeviam.springboot:atoti-server-starter will automatically configure the XMLA endpoint. Extensions of ActivePivotXmlaServletConfig should not be migrated. All configuration options have been ported to Spring properties.

LocalI18nConfig has been completely removed. Importing the starter com.activeviam.springboot:atoti-server-starter will automatically configure internationalization to use the file mentioned in the i18n sub folder of the resource folder.

Importing the starters com.activeviam.springboot:atoti-ui-starter and com.activeviam.springboot:atoti-admin-ui-starter will automatically setup Atoti UI and Atoti Admin UI. ActiveUIResourceServerConfig and AdminUIResourceServerConfig no longer need to be imported.

ActiveViamPropertyFromSpringConfig, used to resolve ActiveViamProperties from Spring properties, no longer needs to be imported. Importing the starter com.activeviam.springboot:atoti-server-starter will automatically forward the Spring properties to the Atoti project.

Importing the starter com.activeviam.springboot:atoti-server-starter will automatically configure ActivePivotServicesConfig. It no longer needs to be imported.

Finally, the starter will automatically import all the REST services that can be exposed with an Atoti project. Classes like ActiveViamRestServicesConfig, ActivePivotRestServicesConfig or ActivePivotWebSocketServicesConfig no longer need to be imported.

Importing the starter com.activeviam.springboot:atoti-server-starter will automatically provide a basic configuration of the IDataExportService. Builders are available for customization through DataExportServiceBuilder

Database Service​

The implementation of Database Service and its components have been reviewed to avoid exposing internal components. This resulted in changes in the way this service converts conditions and Update-Where operations.
They now operate directly on the JSON data, with the help of two side objects. One object providing some context to the operation, like the table field being processed. The other object provides convenience methods to transform data values from the JSON.

Converting a condition factory​

As mentioned above, IConditionFactory changed to receive the JSON data as well as the context of the condition and the helper object to transform data values.

As an example, the following code shows how to migrate a "greater than" condition.

Implementation in 6.0:

class ZeroCondition extends PluginValue implements IConditionFactory {

  @Override
  public String key() {
    return "$gt";
  }

  @Override
  public ICondition compile(
          final String field, final JsonNode jsonNode, final JsonConditionCompiler jsonCompiler) {
    // Extract the value from JSON using the configured parser for the table field
    final Object value = jsonCompiler.convertValue(field, jsonNode);
    // Create the equivalent database condition
    final String[] path = field.split("/", -1);
    return BaseConditions.greater(FieldPath.of(path), value);
  }
}

Implementation in 6.1:

@AtotiPluginValue(intf = IConditionFactory.class)
class GreaterCondition implements IConditionFactory {
  @Override
  public String key() {
    return "$gt";
  }
  @Override
  public ICondition compile(
      final JsonNode jsonValue,
      final IConditionContext fieldExpression,
      final IServiceJsonHelper jsonHelper) {
    final var fieldDetails = fieldExpression.getTestedFieldDetails();
    // Extract the value from JSON using the configured parser for the table field
    final Object value = jsonHelper.readValue(fieldDetails.getTableFieldReference(), jsonValue);
    // Create the equivalent database condition
    return BaseConditions.greater(fieldDetails.getFieldReference(), value);
  }
}

As seen in the conversion example, the JSON data remains.
The nullable String representing the field tested by the condition is now described in the condition context. This object conveniently reports it as a FieldPath, from the root table in the query, as well as the actual StoreField in the database.
Finally, the compiler was replaced by a helper service focusing on parsing data from JSON.

A usage not visible in this example is the case of nested condition, like a not(equal(...)). This can be achieved through the condition context, using the method IConditionContext#convertCondition. This accepts the JSON data defining the nested condition as well as an optional FieldPath, if operating on another path.

Converting an Update-Where procedure factory​

As mentioned above, IUpdateWhereProcedureFactory changed to receive the JSON data as well as the context of the procedure and the helper object to transform data values.

As an example, the following code defines a custom procedure changing the value of a table field when the value matches a given needle value.

Implementation in 6.0:

class NullIfProcedure extends PluginValue implements IUpdateWhereProcedureFactory {

  @Override
  public String key() {
    return "$nullIf";
  }

  @Override
  public UpdateWhereSelectionAndProcedure buildProcedure(
          String ignored, JsonNode value, JsonUpdateWhereCompiler updatewherebuilder) {
    final var testedField = value.get("field").asText();
    final var needle = updatewherebuilder.convertValue(testedField, value);
    return new UpdateWhereSelectionAndProcedure(
            new IUpdateWhereProcedure() {
              int position;

              @Override
              public void init(final IRecordFormat selectionFormat, final IRecordFormat recordFormat) {
                this.position = recordFormat.getFieldIndex("target");
              }

              @Override
              public void execute(final IArrayReader selectedRecord, final IArrayWriter recordWriter) {
                final var value = selectedRecord.read(this.position);
                if (needle.equals(value)) {
                  recordWriter.write(this.position, null);
                }
              }
            },
            new Selection(
                    updatewherebuilder.getTableName(),
                    List.of(AliasedField.create("target", DeprecatedDatabaseApi.toPath(field)))),
            Set.of(testedField));
  }

  @Override
  public void checkJsonNode(final JsonNode node) {}
}

Implementation in 6.1:

@AtotiPluginValue(intf = IUpdateWhereProcedureFactory.class)
class NullIfProcedure implements IUpdateWhereProcedureFactory {
  @Override
  public String key() {
    return "$nullIf";
  }
  @Override
  public UpdateWhereSelectionAndProcedure buildProcedure(
      final JsonNode value, final IUpdateWhereContext context, final IServiceJsonHelper helper) {
    final var testedField = value.get("field").asText();
    final var needle =
        helper.readValue(new StoreField(context.getTableName(), testedField), value.get("value"));
    return new UpdateWhereSelectionAndProcedure(
        new IUpdateWhereProcedure() {
          int position;
          @Override
          public void init(final IRecordFormat selectionFormat, final IRecordFormat recordFormat) {
            this.position = selectionFormat.getFieldIndex("target");
          }
          @Override
          public void execute(final IArrayReader selectedRecord, final IArrayWriter recordWriter) {
            final var value = selectedRecord.read(this.position);
            if (needle.equals(value)) {
              recordWriter.write(this.position, null);
            }
          }
        },
        new Selection(
            context.getTableName(),
            List.of(AliasedField.create("target", FieldPath.of(testedField)))),
        Set.of(testedField));
  }
  @Override
  public void checkJsonNode(final JsonNode node) {}
}

The migration mostly consists of accessing the update information from the context, like the table name or the updated field. The nullable String representing the field to update is now described in the context. This object conveniently reports it as a FieldPath, from the root table in the query, as well as the actual StoreField in the database.
And the compiler was replaced by a helper service focusing on parsing data from JSON.

Security​

Importing the starter com.activeviam.springboot:atoti-server-starter will automatically configure JwtConfig. It no longer needs to be imported.

NoSecurityDatabaseServiceConfig, is the default database rest service security when importing the starter com.activeviam.springboot:atoti-server-starter: the default implementation of IDatabaseServiceConfiguration provides No-ops and no-restrictions.

FullAccessBranchPermissionsManagerConfig has been removed from the public API. Importing the starter com.activeviam.springboot:atoti-server-starter automatically configures a branch permission manager giving full permissions to all users.

Finally, importing the starter com.activeviam.springboot:atoti-server-starter automatically configures an opinionated security for the standard Atoti REST services. This is fully designed for extension. The following packages contain the classes used to define this security:

  • com.activeviam.springboot.atoti.server.starter.private_.security, in the starter, contains the opinionated default security. It contains all the beans that can be overridden.
  • com.activeviam.web.spring.api.security.dsl contains the HttpConfigurer used to parameterize the security of each endpoint.

All the security related to Atoti's REST endpoints, and all the security related to Atoti's communication with Atoti UI, is automatically handled. This results in a lot of @Bean being obsolete in 6.1, with only the Spring related ones that need to be kept.

Defining the application​

Building an Atoti Server application operating on top of a Datastore from descriptions can now be done with the following lines of code:

// This assumes that `datastoreDescription` is defined as a `IDatastoreSchemaDescription`
// and that `managerDescription` is defined as a `IManagerDescription`.
// In this example, no permission manager is used to control access to branches
StartBuilding.application()
    .withDatastore(datastoreDescription)
    .withManager(managerDescription)
    .withoutBranchRestrictions()
    .build();

It can easily be converted to Spring beans. The following snippet provides a working example of how to expose IActivePivotManager as a Spring Bean :

@Configuration
@RequiredArgsConstructor
public class ActivePivotWithDatastoreConfig implements IDatastoreConfig, IActivePivotConfig {
  private final IActivePivotManagerDescriptionConfig apManagerConfig;
  private final IDatastoreSchemaDescriptionConfig datastoreDescriptionConfig;
  private final IActivePivotBranchPermissionsManagerConfig branchPermissionsManagerConfig;
  @Bean
  protected ApplicationWithDatastore applicationWithDatastore() {
    return StartBuilding.application()
        .withDatastore(this.datastoreDescriptionConfig.datastoreSchemaDescription())
        .withManager(this.apManagerConfig.managerDescription())
        .withEpochPolicy(this.apManagerConfig.epochManagementPolicy())
        .withBranchPermissionsManager(
            this.branchPermissionsManagerConfig.branchPermissionsManager())
        .build();
  }
  @Bean
  @Override
  public IActivePivotManager activePivotManager() {
    return applicationWithDatastore().getManager();
  }
  @Bean
  @Override
  public IDatastore database() {
    return applicationWithDatastore().getDatastore();
  }
}

This replaces the configuration class ActivePivotWithDatastoreConfig.

Atoti Components​

Registry​

In addition to the changes made to the annotations mentioned in the branding section, Registry initialization has been streamlined. There is a single entry point: Registry#initialize(RegistryContributions) that does not require any knowledge about Atoti's registry mechanism. RegistryContributions comes with a static builder that provides the exact same options as the previous ContributionProvider classes, making the migration seamless.

The return type of IPluginValue#key() is now String instead of Object.

Partitioning​

Partitioning string description no longer support the word "hash", which has been replaced with "modulo" for clarity's sake. "hashX(fieldName)" partitioning description must be changed to "moduloX(fieldName)".

Numa Node Selectors have been simplified. Implementations of INumaSelectorDescription are available for definition.

The API to retrieve number of processors available in the current machine, often used to define a partitioning, has been moved:

Previous VersionNew Version
com.qfs.platform.IPlatform#getProcessorCountcom.activeviam.tech.numalib.api.PlatformUtil#getProcessorCount

This API should be used instead of Runtime.getRuntime().availableProcessors(). See, for instance, https://bugs.openjdk.org/browse/JDK-6942632.

Content Service​

Building a Content Service​

The Content Service used to be defined as an internal detail of the ActivePivotContentService, before being extracted and published as a Bean. This was automatically performed in com.qfs.server.cfg.content.IActivePivotContentServiceConfig. With Atoti Server 6.1, the Content Service is promoted, and now represents a central Bean that must be defined by users.

Builders are available at com.activeviam.tech.contentserver.storage.api.builder.ContentServiceBuilder.

In 6.0, this might have looked like this:

public class ContentServiceConfig implements IActivePivotContentServiceConfig {

  @Override
  @Bean
  public IActivePivotContentService activePivotContentService() {
    return new ActivePivotContentServiceBuilder().build();
  }

  @Override
  @Bean
  public IContentService contentService() {
    return activePivotContentService().getContentService().getUnderlying();
  }

}

In 6.1:

@Override
@Bean
public IContentService contentService() {
  // Service defined here as a main Bean
  return ContentServiceBuilder.create().inMemory().build();
}
@Override
@Bean
public IActivePivotContentService activePivotContentService() {
  return new ActivePivotContentServiceBuilder()
      .with(contentService())
      .withCacheForEntitlements(10)
      .needInitialization("ROLE_ADMIN", "ROLE_ADMIN")
      .build();
}

Additional examples for different configuration can be found below.

In Memory: 

final var contentService = IContentService.builder().inMemory().build();

Backed with a database: 

final Properties hibernateProperties = new Properties();
hibernateProperties.setProperty(AvailableSettings.SHOW_SQL, "false");
hibernateProperties.setProperty(AvailableSettings.FORMAT_SQL, "false");
// ... any property wanted
final var config = new org.hibernate.cfg.Configuration().addProperties(hibernateProperties);
final var contentService =
    IContentService.builder().withPersistence().configuration(config).build();

Rooted in a specific directory of another Content Service: 

final var contentService = IContentService.builder().inMemory().build();
final var prefixedService = IContentService.prefixed(contentService, "root/dir");

The classes **FullAccessBranchPermissionsManagerConfig** and ContentServiceBranchPermissionsManager have been removed. A new builder is available to create the permission manager using the Content Service. This builder can be used to create the entire configuration class.
The following snippet illustrates how to do so:

/**
 * Sandbox configuration class creating the manager of branch permissions.
 *
 * @author ActiveViam
 */
@Configuration
@RequiredArgsConstructor
public class ActivePivotBranchPermissionsManagerConfig
    implements IActivePivotBranchPermissionsManagerConfig {
  private final IContentServiceConfig contentServiceConfig;
  @Bean
  @Override
  public IBranchPermissionsManager branchPermissionsManager() {
    final CachedBranchPermissionsManager manager =
        new CachedBranchPermissionsManager(
            ContentServiceBranchPermissionsManagerBuilder.create()
                .contentService(this.contentServiceConfig.contentService())
                .allowedBranchCreators(Set.of(ROLE_ADMIN, ROLE_USER))
                .defaultBranchOwners(Set.of(ROLE_ADMIN))
                .build());
    manager.setBranchPermissions(
        IEpoch.MASTER_BRANCH_NAME,
        new BranchPermissions(
            Collections.singleton(ROLE_ADMIN), IBranchPermissions.ALL_USERS_ALLOWED));
    return manager;
  }
}

Content​

It is no longer possible to interact with IContextValues from the IActivePivotContentService. They are no longer stored there by Active Pivot. Methods surrounding the use of context values have been removed, including getContextValue, setContextValue, removeContextValue, etc... An IEntitlementProvider should be used instead.

The only exceptions to this rule are KPIs and Calculated Members.

With the removed support for Context Values, the locales configured per users, previously stored inside the MdxContext, have been upgraded to dedicated content service entries: IActivePivotContentService#getUserLocale() has been added.

Moreover, the changes due to the creation of the public APIs have forced a change in the structure of KPIs and calculated members and their serialization.

To ensure that the content of an existing content service is compatible with Atoti Server 6.1, and that UI elements such as dashboards are not lost, a migration tool is available in the sandbox project to help with this task.

com.activeviam.migration.api.ContentServiceMigrator, in the sandbox application, provides a method migrate to perform this migration. To create an instance of this class, one must provide the instance of the IContentService to migrate.

Creating a backup of the content service before this database migration should be considered before running this tool.

ContentServiceMigrationApp, in the sandbox application, uses the ContentServiceMigrator and the content service configuration defined in the project to start the migration.

Databases​

DirectQuery​

A new API has been introduced in version 6.1 to define external databases and applications leveraging the DirectQuery feature. This updated API retains the core functionality of the 6.0 version, and also allows for easier transitions between the different database connectors, and for the ability to define a table without requiring a schema discovery.

com.activeviam.directquery.api.schema.SchemaDescription and com.activeviam.directquery.application.api.Application are common to all external databases. Database specific implementations of the Spring config have been replaced by a common config ADirectQueryApplicationConfig. AClickhouseConfig, for instance, has been removed.

A user guide is available on the DirectQuery Getting Started page.

ClickHouse​

The ClickHouse connector only supports ClickHouse version 24.8 LTS and above. It will also be compatible with future LTS versions that will release during the 6.1 lifecycle.

Databricks​

The Databricks connector only supports Databricks runtime version 15.4 LTS and above. It will also be compatible with future LTS versions that will release during the 6.1 lifecycle.

Datastore​

DatastoreSchemaDescription's constructors now require a List<? extends IStoreDescription> instead of a Collection<? extends IStoreDescription>.

Deprecated method ICursor.rewind() has been removed. Cursors are now considered performing one-way pass over the underlying data.

Methods using the ID of a table in the datastore transaction API have been removed. The table name should be used instead:

Previous VersionNew VersionComment
IDatastoreTransactionStatistics.getStoreTransactionStatistics(int)IDatastoreTransactionStatistics.getStoreTransactionStatistics(String)
ITransactionManager.startTransaction(int[])ITransactionManager.startTransaction(String...)
ITransactionalWriter.add(int, Object[])ITransactionalWriter.add(String, Object...)
ITransactionalWriter.addAll(int, Collection<Object[]>)ITransactionalWriter.addAll(String, Collection<Object[]>)
ITransactionManager.addRecords(int, IRecordBlock<? extends IRecordReader>)
ITransactionalWriter.remove(int, Object[])ITransactionalWriter.remove(String, Object...)
ITransactionalWriter.removeAll(int, Collection<Object[]>)ITransactionalWriter.removeAll(String, Collection<Object[]>)
ITransactionManager.removeRecords(int, IRecordBlock<? extends IRecordReader>)
IDatastoreSchemaTransactionInformation.getLockedStoreIds()
SchemaPrinter.printStoresSizes()DatabasePrinter.printTableSizes

CSV source​

Previous VersionNew VersionComment
ICsvTopic<Path> com.qfs.msg.csv.filesystem.impl.FileSystemCsvTopicFactory#createTopic(String, String, ICSVParserConfiguration)com.activeviam.source.csv.api.FileSystemCsvTopicFactory#createTopic(String, ICSVParserConfiguration, String...)The topic can be linked to multiple files at once.
ICsvTopic<Path> com.qfs.msg.csv.filesystem.impl.FileSystemCsvTopicFactory#createDirectoryTopic(String, String, String, ICSVParserConfiguration)com.activeviam.source.csv.api.FileSystemCsvTopicFactory#createDirectoryTopic(String, ICSVParserConfiguration, String, String)For consistency with the above change.
ICsvTopic<Path> com.qfs.msg.csv.filesystem.impl.FileSystemCsvTopicFactory#createPoolingDirectoryTopic(String, String, String, ICSVParserConfiguration, int)com.activeviam.source.csv.api.FileSystemCsvTopicFactory#createPoolingDirectoryTopic(String, ICSVParserConfiguration, String, String, int)For consistency with the above change.

The CsvSource class is now private. com.activeviam.source.csv.api.CsvSourceFactory offers various builders to create an ICsvSource.

ActivePivot​

Hierarchies​

Defining a filter for an entire cube is now done through the fluent builder withFactFilter, rather than using withFilter. This distinguishes the API from the other filtering methods, such as filtering a partial provider.

ISelectionDescriptionBuilder.withField(String name) only takes field names, and does not accept field descriptions.

Interface ILevel is a legacy wrapper around ILevelInfo, and is now internal. All the necessary information can be retrieved from the ILevelInfo. From this change derive other modifications

Previous VersionNew Version
List<? extends ILevel> IHierarchy#getLevelsList<ILevelInfo> IHierarchy#getLevels
ILevel#AXISIHierarchy#AXIS
ILevel#BRANCH_LEVEL_NAMEIHierarchy#BRANCH_LEVEL_NAME
ILevel#EPOCH_LEVEL_NAMEIHierarchy#EPOCH_LEVEL_NAME
ILevel#ALLMEMBERIHierarchy#ALLMEMBER
ILevel HierarchiesUtil#getLevelILevelInfo HierarchiesUtil#getLevel
ILevel AAdvancedPostProcessor#getLevelILevelInfo AAdvancedPostProcessor#getLevel

Levels, hierarchies and dimensions are respectively and uniquely identified using the LevelIdentifier, HierarchyIdentifier and DimensionIdentifier classes. Most APIs have been migrated from accepting String arguments to using the corresponding identifiers. These objects have been introduced to replace String descriptions using @ symbol, which reduced the robustness of Atoti's APIs by accepting partial descriptions.

On that note, Copper no longer supports partially defined elements anymore as well. And Copper joins no longer guess the table field to associate with a level: it is now required to properly specify the join mapping.

Previous VersionNew Version
Copper.member("LEVEL")Copper.member(Copper.level("DIMENSION", "HIERARCHY", "LEVEL"))
Copper.hierarchy("HIERARCHY")Copper.hierarchy("DIMENSION", "HIERARCHY")
Copper.level("LEVEL")Copper.level("DIMENSION", "HIERARCHY", "LEVEL")
Copper.newHierarchy(...).fromValues(...).withMembers("STORE", "FIELD")Copper.newHierarchy(...).fromValues(...).withMembers(new StoreField("STORE", "FIELD"))
Copper.newHierarchy(...).fromStore(...).withLevel("LEVEL")Copper.newHierarchy(...).fromStore(...).withLevel("LEVEL", FieldPath.of("LEVEL"))
Copper.newHierarchy(...).fromStore(...).withLevel("LEVEL", "PATH/TO/FIELD")Copper.newHierarchy(...).fromStore(...).withLevel("LEVEL", FieldPath.of("PATH", "TO", "FIELD")
Window.orderBy("HIERARCHY")Window.orderBy(Copper.hierarchy("DIMENSION", "HIERARCHY"))
CopperStore.withMapping(CopperLevel)CopperStore.withMapping(FieldPath, CopperLevel)

com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AAnalysisHierarchy has been removed. com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AAnalysisHierarchyV2 has been renamed to com.activeviam.activepivot.core.ext.api.cube.hierarchy.impl.AAnalysisHierarchy.

Analysis hierarchies should be registered using @AtotiExtendedPluginValue(intf = IAnalysisHierarchy.class, ..).

Aggregate Providers are now fully internal. Basic information is available through IActivePivotVersion#getAggregateProviderStatistics.

Measures​

User Defined Measures (Post Processors)​

Leading and trailing spaces are no longer trimmed in level descriptions. This allows to perform queries against data sources with field names containing leading and/or trailing spaces. However, this change may cause failures when passing level descriptions to post-processors in string-encoded form (e.g. "level1@hierarchy1@dimension1,level2@hierarchy2@dimension2"). This is the case for the leafLevels property ofABaseDynamicAggregationPostProcessor and its subclasses. It is necessary to ensure that no extra spaces between names and separators ('@' and ',') remain.

Examples:

  • " level @ hierarchy " is now parsed as {" level ", " hierarchy ", null} and should be rewritten to "level@hierarchy ".
  • "L1@H1, L2@H2" is now parsed as [{"L1", "H1", null}, {" L2", "H2", null}] and should be rewritten to "L1@H1,L2@H2".

IPrefetcher.name() is deprecated and will be eventually removed. Some implementations have lost their constructor without a prefetcher name. Migration requires to pass the name as first argument.

Prefetcher names were introduced to ease the writing of a post-processor. It allows calls to IAdvancedAggregatesRetriever#retrieveAggregates(String prefetcherName). Forcing a name increases the API's clarity.

On the subject of clarity, abstract class ALocationShiftPostProcessor has been updated.

Previous VersionNew Version
EVALUATE_FOR_MEASURES_PROPERTYHELPER_MEASURES_PROPERTY
UNDERLYING_PREFETCHER_NAMEHELPER_PREFETCHER_NAME
targetMeasureshelperMeasures
getTargetMeasures(...)initializeHelperMeasures(...)
createUnderlyingMeasuresPrefetcher(...)createHelperPrefetcher(...)

IScopeLocation is now internal. Some of its functionalities have been extracted to the objects that used to expose the scope location.

Previous VersionNew Version
IAdvancedAggregatesRetriever#getScope()IAdvancedAggregatesRetriever#getLocation()
IAdvancedAggregatesRetriever#getScope().createBuilder()IAdvancedAggregatesRetriever#createPointLocationBuilder()
IAggregatesRetrievalResult#getScope()IAggregatesRetrievalResult#getLocation()
IIterableAggregatesRetrievalResult#getScope()IIterableAggregatesRetrievalResult#getLocation()

Helper class LocationUtil has been simplified and many methods have been removed. For instance, LocationUtil#createRangeLocation now only has one signature, down from four different alternatives. Other notable changes include:

Previous VersionNew Version
new ModifiedLocation(ILocation, int, Object[])LocationUtil.createModifiedLocation(ILocation, int, Object[])
new ModifiedLocation(ILocation, int[], Object[][])LocationUtil.createModifiedLocation(ILocation, int[], Object[][])
new LocationBuilder(ILocation, OperationFlag...)LocationUtil.createLocationBuilder(ILocation, OperationFlag...)

Location expansion, previously done using LocationUtil#expand, has been revamped. The associated methods now provide an iterator which generates locations on the fly, rather than accumulating them all at once in a collection. This reduces the pressure that this method could put on the Garbage Collector during a query. The performance of the method was also improved.

LocationUtil#expandRangeLevels(ILocation, List) should be used to obtain an iterator that generates point locations from the given range location. A location can also be partially expanded, keeping some coordinates as ranges. In that case, LocationUtil#partialExpand(ILocation, List, List) should be used. Read these methods' documentations for more information.

Example:

// Time hierarchy: Year\Month\Date, Currency hierarchy: AllMember\Currency.
final Location location = new Location(new Object[][] {{null, null, null}, {ILevel.ALLMEMBER, null}});
final List<ILevelInfo> expansionLevels = List.of(monthLevel);
final Iterator<ILocation> iterator = LocationUtil.partialExpand(location, expansionLevels, hierarchies);
// Expands levels Year and Month, creating such locations as
// 2024\01\*|AllMember\*, 2024\02\*|AllMember\*, ... , 2023\01\*|AllMember\*, ...
iterator.forEachRemaining(location -> { ... })

IIterableAggregatesRetrievalResult#transferValues cannot use an Object[] anymore. An IWritableRecord must be created through IIterableAggregatesRetrievalResult#createRecordFormat(int... measureIds).newRecord(), and can be used as a buffer, much like the Object[] may have been before.

We fixed an issue that allowed a BasicPostProcessor to iterate over points even if all underlying measures were null. If a post processor relies on this behavior, its results will change., setting IMeasureHierarchy.COUNT_ID as an underlying measure will restore the behavior.

Use of Vectors​

Vector implementations are now private.

Previous VersionNew Version
new ArrayDoubleVector(double[])ArrayVectorUtils#doubleVector(double...)
new ArrayFloatVector(float[])ArrayVectorUtils#floatVector(float...)
new ArrayLongVector(long[])ArrayVectorUtils#longVector(long...)
new ArrayIntegerVector(int[])ArrayVectorUtils#intVector(int...)
new ArrayObjectVector(Object[])ArrayVectorUtils#objectVector(Object...)
Real Time Support​

AStoreStream only exposes one constructor: AStoreStream(IMultiVersionActivePivot). Implementations of IAggregatesContinuousHandler are internal. Their plugin keys are available in the interface.

The records sent to the AStoreStream listener are now undictionarized. It is no longer necessary to retrieve the values from dictionaries, meaning that they can be directly read from the given records.

AFullRefreshHandler has been removed. Extend AAggregatesContinuousHandler instead, and implement computeImpact(ILocation, EventT) by calling the public method Impact.fullRefresh(location).

Schema rebuild​

ScheduledActivePivotSchemaRebuilder and PeriodicActivePivotSchemaRebuilder have been removed. The scheduling must now be handled in the project. The entry point is IActivePivotManager.rebuild(String... pivotsId).

Queries​

Previous VersionNew Version
com.quartetfs.biz.pivot.query.impl.ActivePivotQueryRunner#create()com.activeviam.activepivot.core.impl.api.query.IActivePivotQueryRunner#create()
IActivePivotQueryRunner#withWildcardCoordinates(String...)
IActivePivotQueryRunner#withWildcardCoordinates(LevelIdentifier...)
IActivePivotQueryRunner#withContextValue(Class<T> class, T value)IActivePivotQueryRunner#withContextValues(T...)
DrillthroughExecutor.createLocationInterpreter(Properties)DrillthroughExecutor.createLocationInterpreter(ISelection, Properties)

In a Query Plan, Copper Joins used to be represented by nodes named External Retrieval. With the introduction of the Direct Query feature, this name was prone to confuse users, as "external" could refer to a table that is not part of the selection, but also to a table that is not at all in memory. Since these retrievals simply represent fetching data from a database, they are now called DatabaseRetrieval. These retrievals, along with JitPrimitiveRetrievals, may hit an in-memory table, or an external table. That is up to the database configuration. Because the ActivePivot instance has no knowledge of the implementation details of the underlying database, it is not possible at the moment to offer a better alternative.

These changes are reflected in the REST API for cube queries.

MdxUtil is now internal. Mdx queries can be executed using the class com.activeviam.activepivot.server.impl.api.query.MdxQueryUtil, or using the @Bean for the interface com.activeviam.activepivot.server.intf.api.webservices.IQueriesService.

Distribution​

Injections​

The setup of a distributed pivot no longer requires any explicit injections in the application. The starter com.activeviam.springboot:atoti-server-starter, and the official testing framework, automatically perform these injections. More details are available in this section.

The following calls are no longer needed (because Atoti's starter automatically performs this call):

- inject(IDistributedMessenger.class, plugin.key(), contextValueManager);
- inject(IDistributedSecurityManager.class, plugin.key(), userDetailsService);

Distribution Properties​

  • The ActiveViamProperty activeviam.distribution.endpoint.suffix is removed and replaced with DataClusterDefinitionBuilder#withEndpointSuffix(String).
  • The ActiveViamProperty activeviam.distribution.endpoint.port is removed and replaced with DataClusterDefinitionBuilder#withPortNumber(int).
  • The ActiveViamProperty activeviam.distribution.endpoint.host is removed and replaced with DataClusterDefinitionBuilder#withAddress(String).
  • The ActiveViamProperty activeviam.distribution.endpoint.protocol is removed and replaced with DataClusterDefinitionBuilder#withProtocol(String).

All can also be set through the fluent builders. When defining the IDataClusterDefinition through fluent builders, the method withUniqueIdentifierInCluster was renamed to withCubeIdentifierInCluster. The suffix and port number can be set at this time, through methods withEndpointSuffix(String) and withPort(int). These two ActiveViamProperties were redundant with Spring Boot's properties server.port and server.servlet.context-path, and have been removed. To avoid a loss of functionality, they can now be forwarded to the cluster definition, by injecting the Spring properties.

Queries​

The IClusterDefinition#EXECUTE_IN_DATA_CUBE_PROPERTY and IDistributedPostProcessor#EXECUTE_IN_DATA_CUBE_PROPERTY properties are no longer supported.

Planning a distributed query has been enhanced. Previously, queries were distributed from the leaves of the calculation chain, up to the first post-processor that would not allow distribution. This process relied on post-processor chains properly implementing IDistributedPostProcessor or IPartitionedPostProcessor.

However, the introduction of distributed Copper meant that most of these decisions would be made by the engine, rather than being taken through custom user code.

Now, as long as there is a measure that provides a way to reduce partial results coming from multiple cubes into a single result, the entire sub-chain below that measure is distributed, unless another measure strictly specifies that it cannot be distributed, by:

  • removing one of the distribution fields from the list of partitioning levels, retrieved through IPostProcessor#setPartitioningLevels.
  • implementing IDistributedPostProcessor#canBeDistributed and returning false for this query.

For instance, Copper.combine(measures) simply applies the given lambda to the underlying measures: it has no knowledge of whether it can be distributed.

  • If all the underlying measures can be distributed, this combination can also be distributed, as long as one of its ancestors in the measure chain specifies how to reduce the partial results into a single final result.
  • If one of its underlying measures cannot be distributed, then it also cannot be distributed.

This should result in queries being a lot more distributed than before.

Testing​

As previously mentioned, module com-activeviam-activepivot:activepivot-copper-test is now internal, with dedicated module com.activeviam:atoti-server-test being created instead for project testing.

The associated documentation page details how these testers should be used.

The testing utility QueryCubeSync is private. DistributionTestHelper, also in the new test module, represents the official public endpoint to create a distributed test that handles operations such as awaiting cluster stability before running a distributed query.