> ## Documentation Index > Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt > Use this file to discover all available pages before exploring further. # Migration notes # Migrate to 6.1 It is strongly recommended to begin by reviewing the [what's new page](./whats_new), which explains some of the migration steps described here. > **Important** > It is also strongly recommended to begin by clearing every use of deprecated methods from the > Atoti library before attempting to migrate. > Indeed, these methods have been, for the most part, removed in the 6.1 version. It will prove much > easier to perform these small migration steps while the project still compiles, and to test that > it still performs as expected, before going to the next step of this migration to 6.1. ## Project configuration changes Atoti has been upgraded to: * **Java 21** * **Spring 6** * **Spring Boot 3.2** These updates are necessary in order to follow the ecosystem's release cycle, particularly when it comes to security updates. ### Spring Modules Considerable effort has gone into simplifying and streamlining the project configuration process in version 6.1. Spring Boot Starters have been developed for Atoti, allowing the import of themed Maven modules instead of manually consolidating numerous configuration classes in the application. The starters represent an "override or opt-out" of the configurations philosophy, rather than an "opt-in by importing" one that was previously implemented. More details can be found in the [Starters documentation](../starters/starters_directory). Migration steps will be provided in subsequent sections of this page. A considerable part of the Atoti modules that need to be imported may be replaced by one or two starters, increasing the maintainability of the project's `pom.xml` file. ### Atoti Modules After updating the project's .pom files to use this new version of Atoti Server, some dependencies will need to be modified to align with the changes in the Atoti Server modules. Modules `com.activeviam.tech:composer-impl` and `com.activeviam.tech:composer-intf` have been merged into a single `com.activeviam.tech:composer-core` module. The relationship between the datastore and the sources (CSV, JDBC) has been inverted. These sources now depend on the datastore, and not the other way around. To utilize a specific source, the corresponding artifact must be separately imported: * CSV source: `com.activeviam.source:csv-source` * JDBC source: `com.activeviam.source:jdbc-source` * Parquet source: `com.activeviam:parquet-source` (not impacted by this change, it already had to be imported separately) This inversion will reduce the size of the final `jar` by cutting out the transitive dependencies of the sources that the project does not require, and thus does not import. Module `com-activeviam-activepivot:activepivot-copper2-impl` has been renamed `com-activeviam-activepivot:activepivot-copper`. Module `com-activeviam-activepivot:activepivot-copper2-test` has been renamed `com-activeviam-activepivot:activepivot-copper-test`. It only contains `internal` and `private_` packages, and should not be imported. A proper public testing module has been created for 6.1: `com.activeviam:atoti-server-test`, and will be discussed later on in this page. `JungSchemaPrinter` has been moved to its own maven module `com.activeviam.tech:datastore-schema-printer`, in the package `com.activeviam.database.datastore.schemaprinter.api`. ### Other Dependencies With 6.1, most of the dependencies of Atoti Server have been upgraded to their latest versions. To stay up to date with the security updates of these dependencies, this often implied major version changes. It is recommended to remove any version number that have been manually set in the project pom, and find a dependency convergence once again during this upgrade to 6.1. Atoti Server's version lifecycle being longer than most OSS version lifecycle, dependency upgrades will continue throughout the lifetime of version 6.1. #### ActiveMonitor ActiveMonitor no longer relies on `org.apache.velocity.tools:velocity-tools-generic`. This dependency allowed for dynamic message templates. Users can elect to re-add such this extension by overriding the Spring Bean returning `VelocityTemplateEngine`, and setting their own extensions through `setExtensions(...)`. Example: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} public class MessagingConfigurationOverride { @Bean public VelocityTemplateEngine velocityTemplateEngine() { final VelocityTemplateEngine engine = new VelocityTemplateEngine(); final Map extensions = new HashMap<>(); extensions.put("numberTool", new NumberTool()); engine.setExtensions(extensions); return engine; } private void usage() { final VelocityTemplateEngine engine = new VelocityTemplateEngine(); final String template = "${number} as ${numberTool.integer($number)}"; final Map model = new HashMap<>(); model.put("number", 1.2); engine.render(template, model); // Returns "1.2 as 1" } ``` > It is recommended to replace the ActiveMonitor product by the Limits product, which concentrates > most of the development efforts. #### Sources Following the [AWS end-of-support announcement for the Java SDK v1 effective December 31, 2025](https://aws.amazon.com/blogs/developer/announcing-end-of-support-for-aws-sdk-for-java-v1-x-on-december-31-2025/), the AWS cloud source has been updated to use the AWS Java SDK v2. When using client side encryption, the metadata header `x-amz-unencrypted-content-length` must be updated to `x-amz-meta-unencrypted-content-length` when accessing S3. AWS recommends using the `x-amz-meta-` prefix for custom metadata headers, as it will not conflict with any other header they might add in the future. The AWS Cloud source has been updated to follow this recommendation. Apache Arrow has been bumped from version 12 to version 17. #### Distribution The JGroups dependency has been upgraded to version 5.3 in Atoti Server 6.1 and further updated to version 5.5 in Atoti Server 6.1.14. List of noticeable changes in JGroups XML config files: * `max_bundle_size` property does not exist anymore * `enable_diagnostics` property does not exist anymore * `use_fork_join_pool` property does not exist anymore * `pbcast.STABLE.stability_delay` property does not exist anymore * `FD` protocol does not exist anymore. One may use `FD_ALL` and `FD_HOST` instead, as described [here](http://www.jgroups.org/manual5/#FD_HOST). See the updated [distributed recommendations](../distributed/distributed_recommendations#prevent-excessive-mergeview) for guidance on choosing between `FD_SOCK` and `FD_ALL` and ordering the protocols. * in `AUTH`, token parameters should now be prefixed with `auth_token.` Also, `MD5Token` does not exist anymore * `org.jgroups.aws.s3.NATIVE_S3_PING` protocol was renamed to `aws.S3_PING`. See more info [here](https://github.com/jgroups-extras/jgroups-aws) Setting a message size using the class name is now deprecated. Prefer using keys in `NettyMessageType`. ## Public and Internal APIs After upgrading the maven modules, the next step of the migration is to be able to compile the project. Most of the classes of the Atoti Server library have been moved to new packages, following the process described in the [what's new page](./whats_new). The [Java API migration tool](https://github.com/activeviam/java-api-migration-tool) can be used to facilitate the migration. It will automatically update all imports with the new packages and class names. > The migration tool's output will provide a list of all the private APIs that the project relies on > after this automatic migration. This list should be kept somewhere. If it still has relevant > entries > after manually finishing this migration to 6.1, please do contact ActiveViam's support. ## REST Services The latest version of the Atoti REST API is 9. ### Data Export Service A bug was found in the Data Export Service in tabular mode, causing empty locations to be incorrectly filtered out. You may need to update queries that relied on the old behavior. To illustrate, let's take the following result of a basic SELECT as an example: | | Paris | New York | | -------- | ----- | -------- | | January | 100 | | | February | | 350 | | March | 421 | | It would formerly be exported as: ```csv theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} January,Paris,100 February,New York,350 March,Paris,421 ``` However, the correct export would be: ```csv theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} January,Paris,100 January,New York, February,Paris, February,New York,350 March,Paris,421 March,New York, ``` If one wants the first kind of result, they should use a query like: `SELECT NON EMPTY City.Members * Month.Members ON ROWS, Measures.X ON COLUMNS FROM Cube` ## Miscellaneous ### Renamed elements A lot of classes have been renamed as well as being moved to another packages. While the imports were automatically updated by the migration script, uses of these classes within the source code were not, for technical reasons. Now that the imports compile properly, they can be used as the source of truth regarding how the code base should be migrated. #### Acronyms The following acronyms have been modified to follow new code style standards. | Previous Version | New Version | | ---------------- | ----------- | | `CSV` | `Csv` | | `JDBC` | `Jdbc` | | `POJO` | `Pojo` | #### Branding Most, if not all, of the references to the previous company name, QuartetFS, also abbreviated QFS, have been removed. As previously stated, the corresponding imports should have been automatically updated, making the changes to the code base rather straightforward. Annotations used for code injection were re-branded: | Previous Version | New Version | | ---------------------------- | -------------------------- | | `QuartetExtendedPlugin` | `AtotiExtendedPlugin` | | `QuartetExtendedPluginValue` | `AtotiExtendedPluginValue` | | `QuartetPlugin` | `AtotiPlugin` | | `QuartetPluginValue` | `AtotiPluginValue` | | `QuartetType` | `AtotiType` | ### Moved and/or replaced properties #### ActiveViam Properties The class `ActiveViamProperties`, which references all the application properties that can be set to alter the configuration of the engine, has been updated: | Key | Previous Version | New Version | Comment | | ----------------------------------------------- | ---------------------------------------------------------- | :----------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `CHUNK_ALLOCATOR_KEY_PROPERTY` | `activeviam.chunkAllocatorClass` | `activeviam.chunkAllocatorKey` | The values taken by this property were changed from class names to keys. The possible values are: "mmap", "array", "direct", "slab", "direct\_buffer", "heap\_buffer". "slab" is still the recommended and default. | | `DATA_CUBE_REST_ENDPOINT_PORT_PROPERTY` | `activeviam.distribution.endpoint.port` | | See [the changes to distribution properties](#distribution-properties) | | `DATA_CUBE_REST_ENDPOINT_SUFFIX_PROPERTY` | `activeviam.distribution.endpoint.suffix` | | See [the changes to distribution properties](#distribution-properties) | | `DATA_CUBE_REST_ENDPOINT_HOST_PROPERTY` | `activeviam.distribution.endpoint.host` | | See [the changes to distribution properties](#distribution-properties) | | `DATA_CUBE_REST_ENDPOINT_PROTOCOL_PROPERTY` | `activeviam.distribution.endpoint.protocol` | | See [the changes to distribution properties](#distribution-properties) | | `EXTERNAL_DATABASE_QUERY_TIMEOUT_LIMIT` | `activeviam.directquery.externalDatabaseQueryTimeoutLimit` | | Database dependant. See classes `com.activeviam.directquery.XXX.api.XXXClientSettings` where `XXX` is a database product name. | | `ENABLE_DIRECTQUERY_AUTOVECTORIZER` | `activeviam.directquery.enableAutoVectorizer` | `com.activeviam.database.api.settings.DiscovererSettings` | See also `com.activeviam.directquery.api.schema.Vectorization` and [the how-to guide](../directquery/how_to/use_vector_in_table) | | `DIRECTQUERY_AUTOVECTORIZER_DELIMITER` | `activeviam.directquery.autoVectorizerDelimiter` | `com.activeviam.database.api.settings.DiscovererSettings` | See also `com.activeviam.directquery.api.schema.Vectorization` and [the how-to guide](../directquery/how_to/use_vector_in_table) | | `DIRECTQUERY_AUTOVECTORIZER_THRESHOLD` | `activeviam.directquery.autoVectorizerThreshold` | `com.activeviam.database.api.settings.DiscovererSettings` | See also `com.activeviam.directquery.api.schema.Vectorization` and [the how-to guide](../directquery/how_to/use_vector_in_table) | | `MANAGER_FEEDING_TIMEOUT` | `activeviam.directquery.cubeFeedingTimeoutInSeconds` | | Database dependant. See classes `com.activeviam.directquery.XXX.api.XXXDatabaseSettings` where `XXX` is a database product name. | | `DIRECTQUERY_AUTO_ADAPTIVE_JIT` | `activeviam.directquery.autoAdaptiveJit` | | Removed. | | `DIRECT_QUERY_SUB_QUERY_LIMIT` | `activeviam.directquery.subQueryLimit` | | Database dependant. See classes `com.activeviam.directquery.XXX.api.XXXDatabaseSettings` where `XXX` is a database product name. | | `DIRECT_QUERY_GET_BY_KEY_BEHAVIOR` | `activeviam.directquery.GetByKeyBehavior` | | Database dependant. See classes `com.activeviam.directquery.XXX.api.XXXDatabaseSettings` where `XXX` is a database product name. | | `SNOWFLAKE_MAX_RESULTSET_SIZE` | `activeviam.directquery.snowflake.maxresultsetsize` | `com.activeviam.directquery.snowflake.api.SnowflakeClientSettings` | | | `AGGRESSIVE_AXIS_POSITION_LIMIT_CHECK_PROPERTY` | `activeviam.mdx.result.aggresiveAxisPositionLimitCheck` | | Moved to the Context Value `MdxContext` to be configurable on a per-query basis | #### Application Properties `activeviam.jwt.*` properties have been renamed into `atoti.jwt.*` properties. Moreover, two properties have been renamed: | Previous Version | New Version | | ----------------------------------- | :----------------------------- | | `activeviam.jwt.generate` | `atoti.jwt.enabled` | | `activeviam.jwt.check.user_details` | `atoti.jwt.check_user_details` | #### Static constants | Previous Version | New Version | Comment | | ------------------------------------------------------------------------------------------------------------ | :----------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------ | | `com.activeviam.copper.ProviderCoordinate#GLOBAL_PROVIDER_NAME` | `com.activeviam.activepivot.core.intf.api.description.IAggregateProviderDefinition#GLOBAL_PROVIDER_NAME` | | | `com.quartetfs.biz.pivot.query.aggregates.impl.StoredMeasureHandler#PLUGIN_TYPE` | `com.activeviam.activepivot.core.intf.api.realtime.IAggregatesContinuousHandler#BASIC_HANDLER_PLUGIN_KEY` | Handler is now internal. | | `com.quartetfs.biz.pivot.query.aggregates.impl.StoredPrimitiveMeasureHandler#PLUGIN_TYPE` | | Handler is now internal. Re-implement `IAggregatesContinuousHandler` if needed. | | `com.quartetfs.biz.pivot.query.aggregates.impl.TransactionStreamFullRefreshHandler#PLUGIN_TYPE` | `com.activeviam.activepivot.core.intf.api.realtime.IAggregatesContinuousHandler#FORCE_FULL_REFRESH_PLUGIN_KEY` | Handler is now internal. | | `com.quartetfs.biz.pivot.query.aggregates.impl.MultiAnalysisHierarchyMeasureHandler#PLUGIN_TYPE` | `com.activeviam.activepivot.core.intf.api.realtime.IAggregatesContinuousHandler#MULTI_ANALYSIS_HIERARCHY_MEASURE_HANDLER` | | | `com.quartetfs.biz.pivot.query.aggregates.impl.CommitIsolatedStoreHandler#PLUGIN_TYPE` | `com.activeviam.activepivot.core.intf.api.realtime.IAggregatesContinuousHandler#createCommitOnIsolatedStoreHandlerPluginKey(String)` | `CommitIsolatedStoreHandler` is a variable plugin value. | | `com.quartetfs.biz.pivot.query.aggregates.impl.TransactionStream#PLUGIN_TYPE` | `com.activeviam.activepivot.core.intf.api.realtime.IStream#ACTIVEPIVOT_PLUGIN_KEY` | `TransactionStream` is internal. | | `com.quartetfs.biz.pivot.query.aggregates.impl.CommitStoreStream#PLUGIN_TYPE` | `com.activeviam.activepivot.core.intf.api.realtime.IStream#createCommitOnStoreStreamPluginKey(String)` | `CommitStoreStream` is variable plugin value. | | `com.qfs.messenger.impl.LocalMessenger#PLUGIN_KEY` | `com.activeviam.activepivot.core.intf.api.description.IMessengerDefinition#LOCAL_PLUGIN_KEY` | | | `com.qfs.messenger.impl.NettyMessenger#PLUGIN_KEY` | `com.activeviam.activepivot.core.intf.api.description.IMessengerDefinition#NETTY_PLUGIN_KEY` | | | `com.quartetfs.biz.pivot.cube.hierarchy.IAnalysisHierarchy#LEVEL_TYPES_PROPERTY` | `com.activeviam.activepivot.core.intf.api.description.IAxisLevelDescription#ANALYSIS_LEVEL_TYPE_PROPERTY` | Used on a per-level basis instead of a per-hierarchy basis | | `com.quartetfs.biz.pivot.cube.hierarchy.ILevel#AXIS` | `com.activeviam.activepivot.core.intf.api.cube.hierarchy.IHierarchy#AXIS` | | | `com.quartetfs.biz.pivot.cube.hierarchy.ILevel#BRANCH_LEVEL_NAME` | `com.activeviam.activepivot.core.intf.api.cube.hierarchy.IHierarchy#BRANCH_LEVEL_NAME` | | | `com.quartetfs.biz.pivot.cube.hierarchy.ILevel#EPOCH_LEVEL_NAME` | `com.activeviam.activepivot.core.intf.api.cube.hierarchy.IHierarchy#EPOCH_LEVEL_NAME` | | | `com.quartetfs.biz.pivot.cube.hierarchy.ILevel#ALLMEMBER` | `com.activeviam.activepivot.core.intf.api.cube.hierarchy.IHierarchy#ALLMEMBER` | | | `com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AxisHierarchyBase#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_PROPERTY` | `com.activeviam.activepivot.core.intf.api.description.IAxisHierarchyDescription#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_PROPERTY` | | | `com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AxisHierarchyBase#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_ALWAYS` | `com.activeviam.activepivot.core.intf.api.description.IAxisHierarchyDescription#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_ALWAYS` | | | `com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AxisHierarchyBase#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_IF_EMPTY` | `com.activeviam.activepivot.core.intf.api.description.IAxisHierarchyDescription#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_IF_EMPTY` | | | `com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AxisHierarchyBase#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_NEVER` | `com.activeviam.activepivot.core.intf.api.description.IAxisHierarchyDescription#AUTO_CONTRIBUTE_UNKNOWN_MEMBER_NEVER` | | ## Spring Boot Starters As mentioned in [a previous section](#spring-modules), Version 6.1 has introduced Spring Boot Starters to ease the configuration of an Atoti project. These starters offer a default configuration for an Atoti project. A considerable part of the `org.springframework.context.annotation.Configuration` objects imported in a 6.0 project can be removed from the `org.springframework.context.annotation.Import` of the application configuration. ### Services `ActivePivotXmlaServletConfig` has been completely removed. The underlying servlet has been transformed into a fully fledged REST controller. Importing the starter `com.activeviam.springboot:atoti-server-starter` will automatically configure the XMLA endpoint. Extensions of `ActivePivotXmlaServletConfig` should not be migrated. All configuration options have been ported to Spring properties. `LocalI18nConfig` has been completely removed. Importing the starter `com.activeviam.springboot:atoti-server-starter` will automatically configure internationalization to use the file mentioned in the `i18n` sub folder of the resource folder. Importing the starters `com.activeviam.springboot:atoti-ui-starter` and `com.activeviam.springboot:atoti-admin-ui-starter` will automatically setup Atoti UI and Atoti Admin UI. `ActiveUIResourceServerConfig` and `AdminUIResourceServerConfig` no longer need to be imported. `ActiveViamPropertyFromSpringConfig`, used to resolve `ActiveViamProperties` from Spring properties, no longer needs to be imported. Importing the starter `com.activeviam.springboot:atoti-server-starter` will automatically forward the Spring properties to the Atoti project. Importing the starter `com.activeviam.springboot:atoti-server-starter` will automatically configure `ActivePivotServicesConfig`. It no longer needs to be imported. Finally, the starter will automatically import all the REST services that can be exposed with an Atoti project. Classes like `ActiveViamRestServicesConfig`, `ActivePivotRestServicesConfig` or `ActivePivotWebSocketServicesConfig` no longer need to be imported. Importing the starter `com.activeviam.springboot:atoti-server-starter` will automatically provide a basic configuration of the `IDataExportService`. Builders are available for customization through `DataExportServiceBuilder` ### Database Service The implementation of Database Service and its components have been reviewed to avoid exposing internal components. This resulted in changes in the way this service converts conditions and Update-Where operations.
They now operate directly on the JSON data, with the help of two side objects. One object providing some context to the operation, like the table field being processed. The other object provides convenience methods to transform data values from the JSON. #### Converting a condition factory As mentioned above, `IConditionFactory` changed to receive the JSON data as well as the context of the condition and the helper object to transform data values. As an example, the following code shows how to migrate a "greater than" condition. Implementation in 6.0: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} class ZeroCondition extends PluginValue implements IConditionFactory { @Override public String key() { return "$gt"; } @Override public ICondition compile( final String field, final JsonNode jsonNode, final JsonConditionCompiler jsonCompiler) { // Extract the value from JSON using the configured parser for the table field final Object value = jsonCompiler.convertValue(field, jsonNode); // Create the equivalent database condition final String[] path = field.split("/", -1); return BaseConditions.greater(FieldPath.of(path), value); } } ``` Implementation in 6.1: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} @AtotiPluginValue(intf = IConditionFactory.class) public static class GreaterCondition implements IConditionFactory { @Override public String key() { return "$gt"; } @Override public ICondition compile( final JsonNode jsonValue, final IConditionContext fieldExpression, final IServiceJsonHelper jsonHelper) { final var fieldDetails = fieldExpression.getTestedFieldDetails(); // Extract the value from JSON using the configured parser for the table field final Object value = jsonHelper.readValue(fieldDetails.getTableFieldReference(), jsonValue); // Create the equivalent database condition return BaseConditions.greater(fieldDetails.getFieldReference(), value); } } ``` As seen in the conversion example, the JSON data remains.
The nullable `String` representing the field tested by the condition is now described in the condition context. This object conveniently reports it as a `FieldPath`, from the root table in the query, as well as the actual `StoreField` in the database.
Finally, the compiler was replaced by a helper service focusing on parsing data from JSON. A usage not visible in this example is the case of nested condition, like a `not(equal(...))`. This can be achieved through the condition context, using the method `IConditionContext#convertCondition`. This accepts the JSON data defining the nested condition as well as an optional `FieldPath`, if operating on another path. #### Converting an Update-Where procedure factory As mentioned above, `IUpdateWhereProcedureFactory` changed to receive the JSON data as well as the context of the procedure and the helper object to transform data values. As an example, the following code defines a custom procedure changing the value of a table field when the value matches a given needle value. Implementation in 6.0: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} class NullIfProcedure extends PluginValue implements IUpdateWhereProcedureFactory { @Override public String key() { return "$nullIf"; } @Override public UpdateWhereSelectionAndProcedure buildProcedure( String ignored, JsonNode value, JsonUpdateWhereCompiler updatewherebuilder) { final var testedField = value.get("field").asText(); final var needle = updatewherebuilder.convertValue(testedField, value); return new UpdateWhereSelectionAndProcedure( new IUpdateWhereProcedure() { int position; @Override public void init(final IRecordFormat selectionFormat, final IRecordFormat recordFormat) { this.position = recordFormat.getFieldIndex("target"); } @Override public void execute(final IArrayReader selectedRecord, final IArrayWriter recordWriter) { final var value = selectedRecord.read(this.position); if (needle.equals(value)) { recordWriter.write(this.position, null); } } }, new Selection( updatewherebuilder.getTableName(), List.of(AliasedField.create("target", DeprecatedDatabaseApi.toPath(field)))), Set.of(testedField)); } @Override public void checkJsonNode(final JsonNode node) {} } ``` Implementation in 6.1: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} @AtotiPluginValue(intf = IUpdateWhereProcedureFactory.class) public static class NullIfProcedure implements IUpdateWhereProcedureFactory { @Override public String key() { return "$nullIf"; } @Override public UpdateWhereSelectionAndProcedure buildProcedure( final JsonNode value, final IUpdateWhereContext context, final IServiceJsonHelper helper) { final var testedField = value.get("field").asText(); final var needle = helper.readValue(new StoreField(context.getTableName(), testedField), value.get("value")); return new UpdateWhereSelectionAndProcedure( new IUpdateWhereProcedure() { int position; @Override public void init( final IRecordFormat selectionFormat, final IRecordFormat recordFormat) { this.position = selectionFormat.getFieldIndex("target"); } @Override public void execute( final IArrayReader selectedRecord, final IArrayWriter recordWriter) { final var value = selectedRecord.read(this.position); if (needle.equals(value)) { recordWriter.write(this.position, null); } } }, new Selection( context.getTableName(), List.of(AliasedField.create("target", FieldPath.of(testedField)))), Set.of(testedField)); } @Override public void checkJsonNode(final JsonNode node) {} } ``` The migration mostly consists of accessing the update information from the context, like the table name or the updated field. The nullable `String` representing the field to update is now described in the context. This object conveniently reports it as a `FieldPath`, from the root table in the query, as well as the actual `StoreField` in the database.
And the compiler was replaced by a helper service focusing on parsing data from JSON. ### Security Importing the starter `com.activeviam.springboot:atoti-server-starter` will automatically configure `JwtConfig`. It no longer needs to be imported. `NoSecurityDatabaseServiceConfig`, is the default database rest service security when importing the starter `com.activeviam.springboot:atoti-server-starter`: the default implementation of `IDatabaseServiceConfiguration` provides No-ops and no-restrictions. `FullAccessBranchPermissionsManagerConfig` has been removed from the public API. Importing the starter `com.activeviam.springboot:atoti-server-starter` automatically configures a branch permission manager giving full permissions to all users. Finally, importing the starter `com.activeviam.springboot:atoti-server-starter` automatically configures an opinionated security for the standard Atoti REST services. This is fully designed for extension. The following packages contain the classes used to define this security: * `com.activeviam.springboot.atoti.server.starter.private_.security`, in the starter, contains the opinionated default security. It contains all the beans that can be overridden. * `com.activeviam.web.spring.api.security.dsl` contains the `HttpConfigurer` used to parameterize the security of each endpoint. > All the security related to Atoti's REST endpoints, and all the security related to Atoti's > communication with Atoti UI, is automatically handled. This results in a lot of `@Bean` being > obsolete in 6.1, with only the Spring related ones that need to be kept. ### Defining the application Building an Atoti Server application operating on top of a Datastore from descriptions can now be done with the following lines of code: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} // This assumes that `datastoreDescription` is defined as a `IDatastoreSchemaDescription` // and that `managerDescription` is defined as a `IManagerDescription`. // In this example, no permission manager is used to control access to branches StartBuilding.application() .withDatastore(datastoreDescription) .withManager(managerDescription) .withoutBranchRestrictions() .build(); ``` It can easily be converted to Spring beans. The following snippet provides a working example of how to expose `IActivePivotManager` as a Spring Bean: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} @Configuration @RequiredArgsConstructor public class ActivePivotWithDatastoreConfig implements IDatastoreConfig, IActivePivotConfig { private final IActivePivotManagerDescriptionConfig apManagerConfig; private final IDatastoreSchemaDescriptionConfig datastoreDescriptionConfig; private final IBranchPermissionsManager branchPermissionsManager; @Bean protected ApplicationWithDatastore applicationWithDatastore() { return StartBuilding.application() .withDatastore(this.datastoreDescriptionConfig.datastoreSchemaDescription()) .withManager(this.apManagerConfig.managerDescription()) .withEpochPolicy(this.apManagerConfig.epochManagementPolicy()) .withBranchPermissionsManager(this.branchPermissionsManager) .build(); } @Bean @Override public IActivePivotManager activePivotManager() { return applicationWithDatastore().getManager(); } @Bean @Override public IDatastore database() { return applicationWithDatastore().getDatastore(); } } ``` This replaces the configuration class `ActivePivotWithDatastoreConfig`. ## Atoti Components ### Registry In addition to the changes made to the annotations mentioned in [the branding section](#branding), Registry initialization has been streamlined. There is a single entry point: `Registry#initialize(RegistryContributions)` that does not require any knowledge about Atoti's registry mechanism. `RegistryContributions` comes with a static builder that provides the exact same options as the previous `ContributionProvider` classes, making the migration seamless. The return type of `IPluginValue#key()` is now `String` instead of `Object`. ### Partitioning Partitioning string description no longer support the word "hash", which has been replaced with "modulo" for clarity's sake. `"hashX(fieldName)"` partitioning description must be changed to `"moduloX(fieldName)"`. Numa Node Selectors have been simplified. Implementations of `INumaSelectorDescription` are available for definition. The API to retrieve number of processors available in the current machine, often used to define a partitioning, has been moved: | Previous Version | New Version | | ---------------------------------------------- | :--------------------------------------------------------------- | | `com.qfs.platform.IPlatform#getProcessorCount` | `com.activeviam.tech.numalib.api.PlatformUtil#getProcessorCount` | This API should be used instead of `Runtime.getRuntime().availableProcessors()`. See, for instance, [https://bugs.openjdk.org/browse/JDK-6942632](https://bugs.openjdk.org/browse/JDK-6942632). ### Content Service #### Building a Content Service The Content Service used to be defined as an internal detail of the `ActivePivotContentService`, before being extracted and published as a Bean. This was automatically performed in `com.qfs.server.cfg.content.IActivePivotContentServiceConfig`. With Atoti Server 6.1, the Content Service is promoted, and now represents a central Bean that must be defined by users. Builders are available at `com.activeviam.tech.contentserver.storage.api.builder.ContentServiceBuilder`. In 6.0, this might have looked like this: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} public class ContentServiceConfig implements IActivePivotContentServiceConfig { @Override @Bean public IActivePivotContentService activePivotContentService() { return new ActivePivotContentServiceBuilder().build(); } @Override @Bean public IContentService contentService() { return activePivotContentService().getContentService().getUnderlying(); } } ``` In 6.1: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} @Override @Bean public IContentService contentService() { // Service defined here as a main Bean return ContentServiceBuilder.create().inMemory().build(); } @Override @Bean public IActivePivotContentService activePivotContentService() { return new ActivePivotContentServiceBuilder() .with(contentService()) .withCacheForEntitlements(10) .needInitialization("ROLE_ADMIN", "ROLE_ADMIN") .build(); } ``` Additional examples for different configuration can be found below. In Memory: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} final var contentService = IContentService.builder().inMemory().build(); ``` Backed with a database: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} final Properties hibernateProperties = new Properties(); hibernateProperties.setProperty(AvailableSettings.SHOW_SQL, "false"); hibernateProperties.setProperty(AvailableSettings.FORMAT_SQL, "false"); // ... any property wanted final var config = new org.hibernate.cfg.Configuration().addProperties(hibernateProperties); final var contentService = IContentService.builder().withPersistence().configuration(config).build(); ``` Rooted in a specific directory of another Content Service: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} final var contentService = IContentService.builder().inMemory().build(); final var prefixedService = IContentService.prefixed(contentService, "root/dir"); ``` The classes `**FullAccessBranchPermissionsManagerConfig**` and `ContentServiceBranchPermissionsManager` have been removed. A new builder is available to create the permission manager using the Content Service. This builder can be used to create the entire configuration class.
The following snippet illustrates how to do so: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} /** * Sandbox configuration class creating the manager of branch permissions. * * @author ActiveViam */ @Configuration @RequiredArgsConstructor public class ActivePivotBranchPermissionsManagerConfig { private final IContentService contentService; @Bean public IBranchPermissionsManager branchPermissionsManager() { final CachedBranchPermissionsManager manager = new CachedBranchPermissionsManager( ContentServiceBranchPermissionsManagerBuilder.create() .contentService(contentService) .allowedBranchCreators(Set.of(ROLE_ADMIN, ROLE_USER)) .defaultBranchOwners(Set.of(ROLE_ADMIN)) .build()); manager.setBranchPermissions( IEpoch.MASTER_BRANCH_NAME, new BranchPermissions( Collections.singleton(ROLE_ADMIN), IBranchPermissions.ALL_USERS_ALLOWED)); return manager; } } ``` #### Content It is no longer possible to interact with `IContextValues` from the `IActivePivotContentService`. They are no longer stored there by Active Pivot. Methods surrounding the use of context values have been removed, including `getContextValue`, `setContextValue`, `removeContextValue`, etc... An `IEntitlementProvider` should be used instead. The only exceptions to this rule are KPIs and Calculated Members. With the removed support for Context Values, the locales configured per users, previously stored inside the `MdxContext`, have been upgraded to dedicated content service entries: `IActivePivotContentService#getUserLocale()` has been added.
Moreover, the changes due to the creation of the public APIs have forced a change in the structure of KPIs and calculated members and their serialization. To ensure that the content of an existing content service is compatible with Atoti Server 6.1, and that UI elements such as dashboards are not lost, a migration tool is available in the sandbox project to help with this task. `com.activeviam.migration.api.ContentServiceMigrator`, in the sandbox application, provides a method `migrate` to perform this migration. To create an instance of this class, one must provide the instance of the `IContentService` to migrate. > Creating a backup of the content service before this database migration should be considered > before running this tool. `ContentServiceMigrationApp`, in the sandbox application, uses the `ContentServiceMigrator` and the content service configuration defined in the project to start the migration. ### Databases #### DirectQuery A new API has been introduced in version 6.1 to define external databases and applications leveraging the DirectQuery feature. This updated API retains the core functionality of the 6.0 version, and also allows for easier transitions between the different database connectors, and for the ability to define a table without requiring a schema discovery. `com.activeviam.directquery.api.schema.SchemaDescription` and `com.activeviam.directquery.application.api.Application` are common to all external databases. Database specific implementations of the Spring config have been replaced by a common config `ADirectQueryApplicationConfig`. `AClickhouseConfig`, for instance, has been removed. A user guide is available on [the DirectQuery Getting Started page](../directquery/getting_started). #### ClickHouse The ClickHouse connector only supports ClickHouse version 24.8 LTS and above. It will also be compatible with future LTS versions that will release during the 6.1 lifecycle. #### Databricks The Databricks connector only supports Databricks runtime version 15.4 LTS and above. It will also be compatible with future LTS versions that will release during the 6.1 lifecycle. #### Datastore `DatastoreSchemaDescription`'s constructors now require a `List` instead of a `Collection`. Deprecated method `ICursor.rewind()` has been removed. Cursors are now considered performing one-way pass over the underlying data. Methods using the ID of a table in the datastore transaction API have been removed. The table name should be used instead: | Previous Version | New Version | Comment | | ------------------------------------------------------------------------------- | :---------------------------------------------------------------------- | :------ | | `IDatastoreTransactionStatistics.getStoreTransactionStatistics(int)` | `IDatastoreTransactionStatistics.getStoreTransactionStatistics(String)` | | | `ITransactionManager.startTransaction(int[])` | `ITransactionManager.startTransaction(String...)` | | | `ITransactionalWriter.add(int, Object[])` | `ITransactionalWriter.add(String, Object...)` | | | `ITransactionalWriter.addAll(int, Collection)` | `ITransactionalWriter.addAll(String, Collection)` | | | `ITransactionManager.addRecords(int, IRecordBlock)` | | | | `ITransactionalWriter.remove(int, Object[])` | `ITransactionalWriter.remove(String, Object...)` | | | `ITransactionalWriter.removeAll(int, Collection)` | `ITransactionalWriter.removeAll(String, Collection)` | | | `ITransactionManager.removeRecords(int, IRecordBlock)` | | | | `IDatastoreSchemaTransactionInformation.getLockedStoreIds()` | | | | `SchemaPrinter.printStoresSizes()` | `DatabasePrinter.printTableSizes` | | ### CSV source | Previous Version | New Version | Comment | | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------- | | `ICsvTopic com.qfs.msg.csv.filesystem.impl.FileSystemCsvTopicFactory#createTopic(String, String, ICSVParserConfiguration)` | `com.activeviam.source.csv.api.FileSystemCsvTopicFactory#createTopic(String, ICSVParserConfiguration, String...)` | The topic can be linked to multiple files at once. | | `ICsvTopic com.qfs.msg.csv.filesystem.impl.FileSystemCsvTopicFactory#createDirectoryTopic(String, String, String, ICSVParserConfiguration)` | `com.activeviam.source.csv.api.FileSystemCsvTopicFactory#createDirectoryTopic(String, ICSVParserConfiguration, String, String)` | For consistency with the above change. | | `ICsvTopic com.qfs.msg.csv.filesystem.impl.FileSystemCsvTopicFactory#createPoolingDirectoryTopic(String, String, String, ICSVParserConfiguration, int)` | `com.activeviam.source.csv.api.FileSystemCsvTopicFactory#createPoolingDirectoryTopic(String, ICSVParserConfiguration, String, String, int)` | For consistency with the above change. | The `CsvSource` class is now private. `com.activeviam.source.csv.api.CsvSourceFactory` offers various builders to create an `ICsvSource`. ### ActivePivot #### Hierarchies Defining a filter for an entire cube is now done through the fluent builder `withFactFilter`, rather than using `withFilter`. This distinguishes the API from the other filtering methods, such as filtering a partial provider. `ISelectionDescriptionBuilder.withField(String name)` only takes field names, and does not accept field descriptions. Interface `ILevel` is a legacy wrapper around `ILevelInfo`, and is now internal. All the necessary information can be retrieved from the `ILevelInfo`. From this change derive other modifications | Previous Version | New Version | | --------------------------------------------- | :------------------------------------------- | | `List IHierarchy#getLevels` | `List IHierarchy#getLevels` | | `ILevel#AXIS` | `IHierarchy#AXIS` | | `ILevel#BRANCH_LEVEL_NAME` | `IHierarchy#BRANCH_LEVEL_NAME` | | `ILevel#EPOCH_LEVEL_NAME` | `IHierarchy#EPOCH_LEVEL_NAME` | | `ILevel#ALLMEMBER` | `IHierarchy#ALLMEMBER` | | `ILevel HierarchiesUtil#getLevel` | `ILevelInfo HierarchiesUtil#getLevel` | | `ILevel AAdvancedPostProcessor#getLevel` | `ILevelInfo AAdvancedPostProcessor#getLevel` | Levels, hierarchies and dimensions are respectively and uniquely identified using the `LevelIdentifier`, `HierarchyIdentifier` and `DimensionIdentifier` classes. Most APIs have been migrated from accepting `String` arguments to using the corresponding identifiers. These objects have been introduced to replace `String` descriptions using `@` symbol, which reduced the robustness of Atoti's APIs by accepting partial descriptions. On that note, Copper no longer supports partially defined elements anymore as well. And Copper joins no longer guess the table field to associate with a level: it is now required to properly specify the join mapping. | Previous Version | New Version | | ----------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------- | | `Copper.member("LEVEL")` | `Copper.member(Copper.level("DIMENSION", "HIERARCHY", "LEVEL"))` | | `Copper.hierarchy("HIERARCHY")` | `Copper.hierarchy("DIMENSION", "HIERARCHY")` | | `Copper.level("LEVEL")` | `Copper.level("DIMENSION", "HIERARCHY", "LEVEL")` | | `Copper.newHierarchy(...).fromValues(...).withMembers("STORE", "FIELD")` | `Copper.newHierarchy(...).fromValues(...).withMembers(new StoreField("STORE", "FIELD"))` | | `Copper.newHierarchy(...).fromStore(...).withLevel("LEVEL")` | `Copper.newHierarchy(...).fromStore(...).withLevel("LEVEL", FieldPath.of("LEVEL"))` | | `Copper.newHierarchy(...).fromStore(...).withLevel("LEVEL", "PATH/TO/FIELD")` | `Copper.newHierarchy(...).fromStore(...).withLevel("LEVEL", FieldPath.of("PATH", "TO", "FIELD")` | | `Window.orderBy("HIERARCHY")` | `Window.orderBy(Copper.hierarchy("DIMENSION", "HIERARCHY"))` | | `CopperStore.withMapping(CopperLevel)` | `CopperStore.withMapping(FieldPath, CopperLevel)` | `com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AAnalysisHierarchy` has been removed. `com.quartetfs.biz.pivot.cube.hierarchy.axis.impl.AAnalysisHierarchyV2` has been renamed to `com.activeviam.activepivot.core.ext.api.cube.hierarchy.impl.AAnalysisHierarchy`. > Analysis hierarchies should be registered > using `@AtotiExtendedPluginValue(intf = IAnalysisHierarchy.class, ..)`. Aggregate Providers are now fully internal. Basic information is available through `IActivePivotVersion#getAggregateProviderStatistics`. #### Measures ##### Copper Measure definition | Previous Version | New Version | | ----------------------------------------- | :----------------------------------------------------- | | `Copper.level("Date").atValue("Today")` | `Copper.levelAtValue(Copper.level("Date"), "Today")` | | `Copper.level("Date").at(day -> day - 1)` | `Copper.levelAt(Copper.level("Date"), day -> day - 1)` | ##### User Defined Measures (Post Processors) Leading and trailing spaces are no longer trimmed in level descriptions. This allows to perform queries against data sources with field names containing leading and/or trailing spaces. However, this change may cause failures when passing level descriptions to post-processors in string-encoded form (e.g. `"level1@hierarchy1@dimension1,level2@hierarchy2@dimension2"`). This is the case for the `leafLevels` property of`ABaseDynamicAggregationPostProcessor` and its subclasses. It is necessary to ensure that no extra spaces between names and separators (`'@'` and `','`) remain. Examples: * `" level @ hierarchy "` is now parsed as `{" level ", " hierarchy ", null}` and should be rewritten to `"level@hierarchy "`. * `"L1@H1, L2@H2"` is now parsed as `[{"L1", "H1", null}, {" L2", "H2", null}]` and should be rewritten to `"L1@H1,L2@H2"`. `IPrefetcher.name()` is deprecated and will be eventually removed. Some implementations have lost their constructor without a prefetcher name. Migration requires to pass the name as first argument. Prefetcher names were introduced to ease the writing of a post-processor. It allows calls to `IAdvancedAggregatesRetriever#retrieveAggregates(String prefetcherName)`. Forcing a name increases the API's clarity. On the subject of clarity, abstract class `ALocationShiftPostProcessor` has been updated. | Previous Version | New Version | | ----------------------------------------- | :------------------------------ | | `EVALUATE_FOR_MEASURES_PROPERTY` | `HELPER_MEASURES_PROPERTY` | | `UNDERLYING_PREFETCHER_NAME` | `HELPER_PREFETCHER_NAME` | | `targetMeasures` | `helperMeasures` | | `getTargetMeasures(...)` | `initializeHelperMeasures(...)` | | `createUnderlyingMeasuresPrefetcher(...)` | `createHelperPrefetcher(...)` | `IScopeLocation` is now internal. Some of its functionalities have been extracted to the objects that used to expose the scope location. | Previous Version | New Version | | --------------------------------------------------------- | :---------------------------------------------------------- | | `IAdvancedAggregatesRetriever#getScope()` | `IAdvancedAggregatesRetriever#getLocation()` | | `IAdvancedAggregatesRetriever#getScope().createBuilder()` | `IAdvancedAggregatesRetriever#createPointLocationBuilder()` | | `IAggregatesRetrievalResult#getScope()` | `IAggregatesRetrievalResult#getLocation()` | | `IIterableAggregatesRetrievalResult#getScope()` | `IIterableAggregatesRetrievalResult#getLocation()` | Helper class `LocationUtil` has been simplified and many methods have been removed. For instance, `LocationUtil#createRangeLocation` now only has one signature, down from four different alternatives. Other notable changes include: | Previous Version | New Version | | ---------------------------------------------------- | :------------------------------------------------------------------ | | `new ModifiedLocation(ILocation, int, Object[])` | `LocationUtil.createModifiedLocation(ILocation, int, Object[])` | | `new ModifiedLocation(ILocation, int[], Object[][])` | `LocationUtil.createModifiedLocation(ILocation, int[], Object[][])` | | `new LocationBuilder(ILocation, OperationFlag...)` | `LocationUtil.createLocationBuilder(ILocation, OperationFlag...)` | Location expansion, previously done using `LocationUtil#expand`, has been revamped. The associated methods now provide an iterator which generates locations on the fly, rather than accumulating them all at once in a collection. This reduces the pressure that this method could put on the Garbage Collector during a query. The performance of the method was also improved. `LocationUtil#expandRangeLevels(ILocation, List)` should be used to obtain an iterator that generates point locations from the given range location. A location can also be partially expanded, keeping some coordinates as ranges. In that case, `LocationUtil#partialExpand(ILocation, List, List)` should be used. Read these methods' documentations for more information. Example: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} // Time hierarchy: Year\Month\Date, Currency hierarchy: AllMember\Currency. final Location location = new Location(new Object[][] {{null, null, null}, {ILevel.ALLMEMBER, null}}); final List expansionLevels = List.of(monthLevel); final Iterator iterator = LocationUtil.partialExpand(location, expansionLevels, hierarchies); // Expands levels Year and Month, creating such locations as // 2024\01\*|AllMember\*, 2024\02\*|AllMember\*, ... , 2023\01\*|AllMember\*, ... iterator.forEachRemaining(location -> { ... }) ``` `IIterableAggregatesRetrievalResult#transferValues` cannot use an `Object[]` anymore. An `IWritableRecord` must be created through `IIterableAggregatesRetrievalResult#createRecordFormat(int... measureIds).newRecord()`, and can be used as a buffer, much like the `Object[]` may have been before. `IIterableAggregatesRetrievalResult#forEachPoint(IPointProcedure procedure)` is now deprecated. Passing measure ids allow great optimizations and ensure no procedure is applied on empty point. That's why `IIterableAggregatesRetrievalResult#forEachPoint(int[] underlyingMeasures, IPointProcedure procedure)` must be used instead. We fixed an issue that allowed a `BasicPostProcessor` to iterate over points even if all underlying measures were null. If a post processor relies on this behavior, its results will change., setting `IMeasureHierarchy.COUNT_ID` as an underlying measure will restore the behavior. ##### Aggregation functions with history (`IHistoryAggregationFunction`) To help reduce boxing, `IHistory` now extends `IReadableCell`. For compatibility reasons `IHistory` has default implementations of the methods of `IReadableCell`; they must be overridden to reduce the boxing. Histories with return types that are primitive numbers should extend one of the following abstract classes: `ADoubleHistory`, `AFloatHistory`, `ALongHistory`, or `AIntegerHistory` to benefit from this improvement. ##### Use of Vectors Vector implementations are now private. | Previous Version | New Version | | --------------------------------- | :----------------------------------------- | | `new ArrayDoubleVector(double[])` | `ArrayVectorUtils#doubleVector(double...)` | | `new ArrayFloatVector(float[])` | `ArrayVectorUtils#floatVector(float...)` | | `new ArrayLongVector(long[])` | `ArrayVectorUtils#longVector(long...)` | | `new ArrayIntegerVector(int[])` | `ArrayVectorUtils#intVector(int...)` | | `new ArrayObjectVector(Object[])` | `ArrayVectorUtils#objectVector(Object...)` | ##### Real Time Support `AStoreStream` only exposes one constructor: `AStoreStream(IMultiVersionActivePivot)`. Implementations of `IAggregatesContinuousHandler` are internal. Their plugin keys are available in the interface. The records sent to the `AStoreStream` listener are now undictionarized. It is no longer necessary to retrieve the values from dictionaries, meaning that they can be directly read from the given records. `AFullRefreshHandler` has been removed. Extend `AAggregatesContinuousHandler` instead, and implement `computeImpact(ILocation, EventT)` by calling the public method `Impact.fullRefresh(location)`. #### Schema rebuild `ScheduledActivePivotSchemaRebuilder` and `PeriodicActivePivotSchemaRebuilder` have been removed. The scheduling must now be handled in the project. The entry point is `IActivePivotManager.rebuild(String... pivotsId)`. #### Queries | Previous Version | New Version | | -------------------------------------------------------------------- | :-------------------------------------------------------------------------------- | | `com.quartetfs.biz.pivot.query.impl.ActivePivotQueryRunner#create()` | `com.activeviam.activepivot.core.impl.api.query.IActivePivotQueryRunner#create()` | | | `IActivePivotQueryRunner#withWildcardCoordinates(String...)` | | | `IActivePivotQueryRunner#withWildcardCoordinates(LevelIdentifier...)` | | `IActivePivotQueryRunner#withContextValue(Class class, T value)` | `IActivePivotQueryRunner#withContextValues(T...)` | | `DrillthroughExecutor.createLocationInterpreter(Properties)` | `DrillthroughExecutor.createLocationInterpreter(ISelection, Properties)` | In a Query Plan, Copper Joins used to be represented by nodes named `External Retrieval`. With the introduction of the DirectQuery feature, this name was prone to confuse users, as "external" could refer to a table that is not part of the selection, but also to a table that is not at all in memory. Since these retrievals simply represent fetching data from a database, they are now called `DatabaseRetrieval`. These retrievals, along with `JitPrimitiveRetrievals`, may hit an in-memory table, or an external table. That is up to the database configuration. Because the ActivePivot instance has no knowledge of the implementation details of the underlying database, it is not possible at the moment to offer a better alternative. > These changes are reflected in the REST API for cube queries. `MdxUtil` is now internal. Mdx queries can be executed using the class `com.activeviam.activepivot.server.impl.api.query.MdxQueryUtil`, or using the `@Bean` for the interface `com.activeviam.activepivot.server.intf.api.webservices.IQueriesService`. ### Distribution #### Injections The setup of a distributed pivot no longer requires any explicit injections in the application. The starter `com.activeviam.springboot:atoti-server-starter`, and the official testing framework, automatically perform these injections. The following calls are no longer needed (because Atoti's starter automatically performs this call): ```diff theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} - inject(IDistributedMessenger.class, plugin.key(), contextValueManager); - inject(IDistributedSecurityManager.class, plugin.key(), userDetailsService); ``` #### Distribution Properties * The `ActiveViamProperty` `activeviam.distribution.endpoint.suffix` is removed and replaced with `DataClusterDefinitionBuilder#withEndpointSuffix(String)`. * The `ActiveViamProperty` `activeviam.distribution.endpoint.port` is removed and replaced with `DataClusterDefinitionBuilder#withPortNumber(int)`. * The `ActiveViamProperty` `activeviam.distribution.endpoint.host` is removed and replaced with `DataClusterDefinitionBuilder#withAddress(String)`. * The `ActiveViamProperty` `activeviam.distribution.endpoint.protocol` is removed and replaced with `DataClusterDefinitionBuilder#withProtocol(String)`. All can also be set through the fluent builders. When defining the `IDataClusterDefinition` through fluent builders, the method `withUniqueIdentifierInCluster` was renamed to `withCubeIdentifierInCluster`. The suffix and port number can be set at this time, through methods `withEndpointSuffix(String)` and `withPort(int)`. These two `ActiveViamProperties` were redundant with Spring Boot's properties `server.port` and `server.servlet.context-path`, and have been removed. To avoid a loss of functionality, they can now be forwarded to the cluster definition, by injecting the Spring properties. #### Queries The `IClusterDefinition#EXECUTE_IN_DATA_CUBE_PROPERTY` and `IDistributedPostProcessor#EXECUTE_IN_DATA_CUBE_PROPERTY` properties are no longer supported. Planning a distributed query has been enhanced. Previously, queries were distributed from the leaves of the calculation chain, up to the first post-processor that would not allow distribution. This process relied on post-processor chains properly implementing `IDistributedPostProcessor` or `IPartitionedPostProcessor`. However, the introduction of distributed Copper meant that most of these decisions would be made by the engine, rather than being taken through custom user code. Now, as long as there is a measure that provides a way to reduce partial results coming from multiple cubes into a single result, the entire sub-chain below that measure is distributed, unless another measure strictly specifies that it cannot be distributed, by: * removing one of the distribution fields from the list of partitioning levels, retrieved through `IPostProcessor#setPartitioningLevels`. * implementing `IDistributedPostProcessor#canBeDistributed` and returning false for this query. For instance, `Copper.combine(measures)` simply applies the given lambda to the underlying measures: it has no knowledge of whether it can be distributed. * If all the underlying measures can be distributed, this combination can also be distributed, as long as one of its ancestors in the measure chain specifies how to reduce the partial results into a single final result. * If one of its underlying measures cannot be distributed, then it also cannot be distributed. This should result in queries being a lot more distributed than before. ## Testing As previously mentioned, module `com-activeviam-activepivot:activepivot-copper-test` is now internal, with dedicated module `com.activeviam:atoti-server-test` being created instead for project testing. [The associated documentation page](../copper/copper_advanced#defining-a-test-environment) details how these testers should be used. The testing utility `QueryCubeSync` is private. `DistributionTestHelper`, also in the new test module, represents the official public endpoint to create a distributed test that handles operations such as awaiting cluster stability before running a distributed query.