Skip to main content

Changelog

For user-facing changes, refer to the What's New? page.

The format is based on Keep a Changelog and this project adheres to Romantic Versioning.

6.0.12-sb3​

2024-03-01

Added​

  • PIVOT-8437 Add experimental API to access the database in Copper: Copper.experimental().databaseMeasure(...).
  • PIVOT-8761 Added a new Copper measure: Copper.experimental().switchOn(...).
  • PIVOT-8787 DirectQuery: Support for inequality operators (>, >=, <, <=) in queries conditions.

Deprecated​

  • PIVOT-8660 com.activeviam.source.jdbc.api.calculator.JdbcCalculatorUtils is deprecated. Use com.activeviam.tech.jdbc.internal.read.JdbcReaderUtils instead.

Fixed​

  • PIVOT-8411 Fixed the deltas sent to update the values of continuous queries involving location shifting post-processor measures (such as CopperMeasure.parentValueOn etc).
  • PIVOT-8419 Fixed equal and in condition on insertion time update where trigger early registration.
  • PIVOT-8750 Fixed PartitionStatistics.getMaxRowId().
  • PIVOT-8782 Fixed issues when exporting Mdx query in Arrow files without COLUMNS.
  • PIVOT-8132 POJOColumnCalculator is now thread safe.

Security​

  • PIVOT-8671 DirectQuery: Upgraded com.clickhouse:clickhouse-http-client to version 0.6.0 to fix CVE-2024-23689. This might change a few Clickhouse imports.
  • PIVOT-8672 Upgrade Spring Boot dependencies to fix CVE-2024-22243 and CVE-2024-22234.

6.0.11-sb3​

2024-01-16

Added​

  • PIVOT-4994 Added new property IDataClusterDefinition.CONCEAL_BRANCHES to hide the branches of the data cube from the query cubes.
  • PIVOT-6525 DirectQuery: introduce mechanism to incrementally refresh cube with data from external database. See com.activeviam.directquery.api.IRefreshable#refresh(ChangeDescription) and associated documentation.

Changed​

  • PIVOT-8515 MIN and MAX aggregation functions now support objects that implement the Comparable interface.
  • PIVOT-8536 breaking upgraded the minimal Java version to 17. Support for JDK 21 is not supported in this version.
  • PIVOT-8536 breaking The underlying class of com.qfs.snl.persistence.type.impl.AJsonConverter changed to become an attribute converter and became a private class.
  • PIVOT-8536 breaking com.qfs.msg.csv.impl.ChannelTask#threadPool changed from ParserThreadPool to IExecutorService.
  • PIVOT-8536 breaking com.activeviam.tracing.impl.Tracing now deals with OpenTelemetry classes, like io.opentelemetry.api.trace.Tracer or io.opentelemetry.api.trace.Span.

Deprecated​

  • PIVOT-8376 MdxDrillthroughToCsvConverter(MdxDrillthroughRowset, IJsonOutputConfiguration) is deprecated in favor of MdxDrillthroughToCsvConverter(MdxDrillthroughRowset, IJsonCsvOutputConfiguration).

Removed​

  • PIVOT-8536 breaking Removed the classes com.qfs.repository.persistence.type.impl.StringCoordinateType, com.qfs.snl.persistence.type.impl.StringListType, com.qfs.snl.persistence.type.impl.StringStringMapType and com.qfs.snl.persistence.type.impl.StateListType since the underlying concept was removed from Hibernate 6.

Fixed​

  • PIVOT-5110 Fixed DrillthroughProperties deep clone method.
  • PIVOT-5283 Fixed QueryResultsProcessingTask statistics on aggregation duration.
  • PIVOT-6475 Fixed a bug in aggregates cache which might lead to producing empty results on subsequent queries involving the same location.
  • PIVOT-7838 Fixed class cast in ShiftMeasure.LevelShift#equals.
  • PIVOT-8316 Improved execution speed of queries with filters on partial providers.
  • PIVOT-8343 Improved record query performance when run on current thread by not creating empty completion tasks for non-matching partitions.
  • PIVOT-8427 Fixed null value handling in Copper UDAF .outputFromBuffer().
  • PIVOT-8480 Fixed bug in GetAggregatesResultCellsetTester that caused an error while checking integer measures with .hasValues().
  • PIVOT-8517 Fixed thread starvation issue when the threads of each NUMA node wait for the threads from another NUMA nodes to finish.

Security​

  • PIVOT-8536 breaking Upgraded Atoti Server to depend on spring-boot-dependencies:3.2.0.
  • PIVOT-8536 breaking Upgraded Atoti Server to depend on Spring Framework 6.1.1.
  • PIVOT-8536 breaking Upgraded Atoti Server to depend on Hibernate 6.3.1.Final.

6.0.10​

2023-12-07

Added​

  • PIVOT-8128 Added specific calculators for vectors in JDBC Source: DoubleArrayJdbcColumnCalculator, IntegerArrayJdbcColumnCalculator, FloatArrayJdbcColumnCalculator and LongArrayJdbcColumnCalculator.
  • PIVOT-8137 IReadableDatastore#getEntityResolver() now returns an IDatastoreSchemaEntityResolver to be able to get specific metadata about a Datastore.
  • PIVOT-8169 Added IJdbcSourceBuilder to create an IJDBCSource.
  • PIVOT-8211 Added KeyValueSourceFactory to create an IKeyValueSource.
  • PIVOT-8274 Added CsvSourceFactory to create an ICSVSource.

Changed​

  • PIVOT-8127 Jdbc source: convert automatically array column to datastore vector column.

Deprecated​

  • PIVOT-8027 DirectQuery: Static method com.activeviam.directquery.YOUR_DATABASE.api.Session#registerUdaf() has been deprecated in favor of com.activeviam.directquery.YOUR_DATABASE.api.Session#registerSqlUdaf().
  • PIVOT-8137 Deprecated methods IDatastoreSchema#getPartitioningDescription(String) and IDatastoreSchema#getNumaSelectorDescription(String).
  • PIVOT-8169 Deprecated AJDBCSource, ArrayJDBCSource, MapJDBCSource and NativeJDBCSource, use IJDBCSource.builder() instead.
  • PIVOT-8211 Deprecated KeyValueSource, use KeyValueSourceFactory instead.
  • PIVOT-8247 IDatastoreSchemaDescriptionPostProcessor and all the implementations are deprecated. Datastore's description post-processors are no longer needed. Most of the transformation done in the post processors can be done directly at the schema's description creation.
  • PIVOT-8274 Deprecated CSVSource, use CsvSourceFactory instead.

Fixed​

  • PIVOT-7960 Fixed the deadlock in the ConflationEventEngine caused by the asynchronous processing of events of different types.
  • PIVOT-8182 In a distributed setup, unexpected errors that happened in one cube while sending a message should no longer cause off-heap memory leaks after failing to properly receive this message in another cube. These leaks are believed to be Netty buffers that were not properly released on error handling.
  • PIVOT-8322 Fixed NotSerializableException when using Copper.userDefinedAgg on a query cube.
  • PIVOT-8318 Fixed NullPointerException when registering a stream for post-processed measure depending on a store lookup in a Query Cube.
  • PIVOT-8359 Reduce memory footprint for hash code computation of datastore conditions.
  • PIVOT-8359 Speed up hash code computation for cube conditions.

Security​

  • PIVOT-7805 Migrated the products to depend on Spring Boot 2.7.17, to address CVE-2023-44487.
  • PIVOT-7805 Upgraded com.google.cloud:google-cloud-bigquery to 2.34.0 to fix CVE-2023-5072
  • PIVOT-7805 Upgraded com.google.cloud:google-cloud-bigquerystorage to 2.44.1, com.google.cloud:google-cloud-bigquerystorage to 2.29.0 and com.google.guava:guava to 32.1.3-jre for dependencies convergence.
  • PIVOT-8319 Migrated the products to depend on Spring Boot 2.7.18, to address CVE-2023-4586 and CVE-2023-34062.
  • PIVOT-8319 Upgrade ch.qos.logback:logback-core, ch.qos.logback:logback-classic and ch.qos.logback:logback-access to version 1.2.13 to fix CVE-2023-6378.

6.0.9​

2023-10-27

Added​

  • PIVOT-5894 Experimental: Copper can now be used with distributed cubes. See documentation.
  • PIVOT-7769 Added CopperPostProcessor.executeInDataCube() method.
  • PIVOT-7236 PIVOT-7893 Improved the performance of cycle detection in the planning of a pivot query. When printing the query plan timings, this should improve the part that falls into Planning finalization duration, which happens before the actual execution of the query (associated with execution time). This development impacts queries with a large query plan (several hundreds of nodes)
  • PIVOT-8008 Added readiness endpoint on atoti server /actuator/health/readiness

Changed​

  • PIVOT-7246 DirectQuery: upgrade com.amazon.redshift:redshift-jdbc42 to 2.1.0.19.

Fixed​

  • PIVOT-7580 Fix continuous MDX query initialization issue caused by uninitialized post-processor on a query cube.
  • PIVOT-7111 Fix missing maxLevelPerHierarchy field on cube/query/mdx endpoint.

6.0.8​

2023-10-18

danger

Version 6.0.8 introduced a major error in the Leaf Aggregate Provider. Version 6.0.9 is released early to remediate this issue.

Added​

  • PIVOT-5875 Added the PercentileFunction aggregation function. See the Percentile documentation page.
  • PIVOT-7176 The cube filter is now used in distributed cubes to limit the number of data nodes the query is sent to.
  • PIVOT-7202 Add withTableFields(String...fieldNames) to simplify the creation of database queries. For instance withFields(FieldPath.of("fieldName1"), FieldPath.of("fieldName2")) can now be replaced by withTableFields("fieldName1", "fieldName2").
  • PIVOT-7441 Support all signatures of InStr in MDX queries.
  • PIVOT-7737 Support fields beginning by $ in update-where procedures of the database service.

Changed​

  • PIVOT-7737 Do not require procedures to start by $ in update-where of the database service.
  • PIVOT-7737 Make the database service update-where considered keys as procedures first and then as field names.
  • PIVOT-7878 CopperLevel.isin(Collection<Object>) was replaced by isin(Collection<?>) to reduce ambiguities with CopperLevel.isin(Object...).
  • PIVOT-7931 Improved the performance of LeafAggregateProvider when the majority of members of the hierarchy are included in the level filter.
  • PIVOT-8031 Analysis hierarchies extending AAnalysisHierarchyV2 no longer need to define an IAnalysisHierarchyDescriptionProvider if their full description is given to the new method added to the description builder: withAnalysisHierarchy(IAnalysisHierarchyDescription).

Deprecated​

  • PIVOT-7807 DirectQuery: Method com.activeviam.directquery.YOUR_DATABASE.api.Database#getSchema() has been deprecated in favor of com.activeviam.directquery.YOUR_DATABASE.api.Database#getUserSchema().
  • PIVOT-7754 Methods withFields(String...fieldNames) have been deprecated in favor of withFieldsWithoutAlias(String...fieldNames).

Fixed​

  • PIVOT-7632 The XMLA Servlet now properly handles Excel requests to cancel a query. Excel primarily sends cancellation requests when a user types in a search bar, stops typing (a query is sent to Atoti Server), and then resumes typing.
  • PIVOT-7640 Fix issue with CurrentOrdinal mdx function not being able to operate on aliased sets.
  • PIVOT-7682 Fix NPE when a file was not found in content service.
  • PIVOT-7732 CopperPostProcessor.withType(...) now returns a CopperPostProcessor instead of a CopperMeasure.
  • PIVOT-7785 Fix issue with queries where a group-by field has an alias conflicting with the path of a field used in a measure.
  • PIVOT-7822 Fix issue with datastore aggregate queries when querying for duplicate field paths and some specific conditions.
  • PIVOT-7910 NeighborValuePostProcessor no longer throws an AssertionError if queried on a non-existing member of the shifting hierarchy.
  • PIVOT-8031 Fix clone method of hierarchy description.

Security​

  • PIVOT-7805 Upgraded org.springframework.security to 5.8.17.
  • PIVOT-7805 Migrated the products to depend on Spring Boot 2.7.16, to address as many CVEs as possible.
  • PIVOT-7805 Upgraded Google libraries (com.google.cloud:google-cloud-bigquery to 2.33.1, com.google.cloud:google-cloud-bigquerystorage to 2.43.0, google-cloud-storage to 2.27.1, com.google.guava:guava to 32.1.2-jre) to mitigate CVE-2022-3171 and CVE-2023-32731.
  • PIVOT-7805 Upgraded org.apache.arrow:arrow-vector and org.apache.arrow:arrow-memory-unsafe to 12.0.1 for dependency convergence with Google ecosystem.
  • PIVOT-7805 Upgraded com.nimbusds:nimbus-jose-jwt to 9.35 to mitigate CVE-2023-1370.
  • PIVOT-7805 Upgraded org.xerial.snappy:snappy-java to 1.1.10.5 to mitigate CVE-2023-43642.
  • PIVOT-7805 Upgraded org.mybatis:mybatis to 3.5.13 to mitigate XRAY-127508 and CVE-2020-26945.
  • PIVOT-7805 Upgraded org.yaml:snakeyaml to 2.0 to mitigate CVE-2022-1471.
  • PIVOT-7805 Upgraded org.mozilla:rhino to 1.7.14 to mitigate XRAY-178804.
  • PIVOT-7888 Upgraded libraries to connect to Microsoft Azure (azure-sdk-bom to 1.2.17, azure-resourcemanager to 2.31.0).
  • PIVOT-7888 Replaced org.apache.velocity:velocity-tools with org.apache.velocity.tools:velocity-tools-generic 3.1 and upgraded org.apache.velocity:velocity-engine-core to mitigate CVE-2015-0899, CVE-2016-1181, CVE-2018-1000632, CVE-2020-10683, CVE-2020-13936.

6.0.7​

2023-08-31

Added​

  • PIVOT-5714 Add a DistinctCountHistoryFunction function: a distinct count that properly supports removal.
  • PIVOT-7295 Add a CubeFilter translator to allow the cube filters to be added to the map of context values in REST queries.
  • PIVOT-7563 Add support for emulated vectors (multi-row and multi-column) in DirectQuery Databricks connector.

Changed​

  • PIVOT-6776 Mapper functions in post-processors are no longer obliged to write null values int result cells explicitly.
  • PIVOT-7382 Packages can now be excluded from the ClasspathContributionProvider's scan.
  • PIVOT-7554 Copper measures that have name are now visible by default.
  • PIVOT-7611 Default lifetime of session context values is increased from 10 minutes to 10 hours.

Removed​

  • PIVOT-7553 Maven module composer-test now imports its own test dependencies using <scope>test</test>. Projects relying on this module to transitively import junit-jupiter-params, awaitility and mockito-core will need to add these dependencies to their own dependencies.
  • PIVOT-7553 Maven module composer-test no longer imports spring-cloud-starter-sleuth.

Fixed​

  • PIVOT-5714 Changed the DistinctCountFunction to accept non-comparable types.
  • PIVOT-7377 Fixed a parallelism issue and improved the speed of an aggregate provider transaction in a very specific case: multiple database partitions only contribute to a unique partition of an aggregate provider.
  • PIVOT-7402 The MdxBenchmark class has been replaced by the now working RestClientMDXBenchmark.
  • PIVOT-7570 Fix issue in SQL generation for large multi row vectors in MsSql.
  • PIVOT-7644 Fix issue with Aggregate Table not being used for some databases.
  • PIVOT-7693 DirectQuery: Increase timeout for creating a connection on Redshift connection pool to avoid failure at initial loading.

Security​

  • PIVOT-7568 Upgrade spring-boot to 2.7.14 to patch CVE-2023-20883 and CVE-2023-33008.
  • PIVOT-7568 Upgrade snappy-java to 1.1.10.3 to patch CVE-2023-34453, CVE-2023-34454 and CVE-2023-34455.

6.0.6​

2023-07-26

Added​

  • PIVOT-5860 In the MemoryAnalysisService MBean, added an operation Dump full application memory statistics exporting memory statistics for all versions.
  • PIVOT-6356 DirectQuery: add support for Databricks database.
  • PIVOT-7429 DirectQuery: introduction of settings POJO for configuration purpose:BigqueryDirectQuerySettings, ClickhouseDirectQuerySettings, DatabricksDirectQuerySettings, MsSqlDirectQuerySettings, RedshiftDirectQuerySettings, SnowflakeDirectQuerySettings, SynapseDirectQuerySettings.

Changed​

  • PIVOT-5860 In the MemoryAnalysisService MBean, renamed the operation Dump memory statistics to Dump memory statistics for most recent version since this operation only exports the most recent version.
  • PIVOT-7349 RecordFormat#getFieldNames no longer creates a new list at each method call, thus improving its footprint.

Deprecated​

  • PIVOT-7349 PropertiesRecordFormat is no longer used internally and is now deprecated, for removal in 6.1.0.
  • PIVOT-7429 DirectQuery: Configuration parameters for DirectQuery that were passed via ActiveViamProperty (either JVM arguments or environment variables) have been deprecated in favor of the associated POJO (BigqueryDirectQuerySettings, ClickhouseDirectQuerySettings, DatabricksDirectQuerySettings, MsSqlDirectQuerySettings, RedshiftDirectQuerySettings, SnowflakeDirectQuerySettings, SynapseDirectQuerySettings).

Fixed​

  • PIVOT-7236 Fix performance issue in CycleDetectingGraphVisitor.
  • PIVOT-7269 Updated IPostProcessor#IS_PARTITIONED_ON_RANGE_LEVELS_PROPERTY javadoc to correctly link to its default value.
  • PIVOT-7308 Fix unnecessary cloning of swapped vector blocks.
  • PIVOT-7315 Fix overriding epoch dimension visibility with the mdx context.
  • PIVOT-7316 Fix query issue when distributing field is null or "N/A". The query engine used to skip cubes with only null distributing fields values.
  • PIVOT-7321 When running ListQueries against a Datastore, the option runCurrentThread no longer forks an additional task.
  • PIVOT-7336 Copper.storeLookup could use a too recent Datastore version when a transaction would create a Datastore version but not an ActivePivot one.
  • PIVOT-7359 When asked to output the K top/bottom elements of the vector, the ZeroVector class no longer throws IndexArrayOutOfBoundException when K = vector.size().
  • PIVOT-7367 Properly propagate context value dependencies when optimizing calculated measures with Copper.
  • PIVOT-7398 Fix issue with slicing introspection analysis hierarchy and aggregate provider on DirectQuery.
  • PIVOT-7413 Fix SQL for queries with Multi Row Vector tables and incomplete joins in DirectQuery.
  • PIVOT-7419 Fix bug in DrilldownMember causing members to be missing in the result.
  • PIVOT-7433 Filters on Member captions through XMLA were not properly taken into account.

6.0.5​

2023-06-08

Added​

  • PIVOT-6176 Add a WebSocketSnippet to test the WebSocket server with a client call.
  • PIVOT-6804 Add possibility to enable aggressive check (using ActiveVIamProperty#AGGRESSIVE_CELLSET_SIZE_CHECK_PROPERTY) on MDX CellSet axis positions during axis materialization (see and IMdxContext#getAxisPositionLimit()).
  • PIVOT-6953 Atoti Python API: Add configuration options for Hibernate
  • PIVOT-6980 Allow python API to specify custom distribution protocol, include S3Ping JAR in atoti-aws plugin.
  • PIVOT-7046 Added getHierarchicalMapping() on IActivePivot.
  • PIVOT-7272 Allow python API to specify distributing fields.

Changed​

  • PIVOT-6957 All Basic authentication mechanisms are replaced by Login Form authentication. Atoti Standard Runtime users may now specify path to their own HTML login pages when using Ldap, Kerberos or Basic authentication mechanisms.

Deprecated​

caution

From version 6.0.5, Atoti Server annotates some classes as @Deprecated, for removal in 6.1. These deprecations are accompanied by another custom annotation @InternalApi. The @InternalAPI annotation indicates that these classes were designed to be used by our internal teams at ActiveViam only, and not as part of the public API. It is for this reason that they are being deprecated. Atoti Server does not provide any replacement for them, as these classes are not features.
See deprecations page for more details.

  • PIVOT-7046 IActivePivot.getAggregateProvider() is deprecated in favor of IActivePivot.getHierarchicalMapping()

Fixed​

  • PIVOT-5978 LimitedByteArrayOutputStream now handles specific edge cases that could cause the XMLA Servlet to still generate Out of Memory Errors.
  • PIVOT-6593 Improved read/write and serialization/deserialization performances on snappy-compressed files in memory statistics exports.
  • PIVOT-6923 Datastore implementation of IQueryRunner#getByKeyQuery() now accepts key fields in the order specified by IDataTable#getKeyFieldNames().
  • PIVOT-7020 Correctly propagate exception to partitions of a JITPrimitiveAggregatesRetrieval.
  • PIVOT-7044 Reduce the time needed to create Copper measures.
  • PIVOT-7050 Inserting data into an initially empty joined store now correctly updates corresponding analysis hierarchies.
  • PIVOT-7062 Fix data loading for JDBC drivers that do not support fetch size hints.
  • PIVOT-7081 Fix implementation of FastLogicalQFSBitmap#interruptibleNaiveOr(..). Previous implementation didn't account for the append-only constraint of the QFSBitmap.
  • PIVOT-7078 Fix a performance regression for Vector-based Aggregations.
  • PIVOT-7089 Fix data race on drop of empty results in ExecutionPlan.
  • PIVOT-7145 DirectQuery: Supports a wider range of special characters in ClickHouse column names.
  • PIVOT-7160 DirectQuery: Fix generated MAC dump when using an AdaptiveJit as full provider with DirectQuery.
  • PIVOT-7188 DirectQuery: Fix issue during validation of DirectQuery when using distributed cube.
  • PIVOT-7209 Fix NullPointerException in ActivePivot Monitoring when retrieving attributes of Query Cube MBean in distributed setup.
  • PIVOT-7240 DirectQuery: Fix issue on streamed list queries which could lead to record loss.
  • PIVOT-7260 Fix bug preventing java code being reused to define Copper level filtering measures.

Security​

  • PIVOT-7057 Upgrade snowflake-jdbc to 3.13.30 to patch a vulnerability to command injection via SSO URL authentication.
  • PIVOT-7230 Upgrade spring-boot to 2.7.11 to patch CVE-2023-20873.

6.0.4​

2023-04-17

Added​

  • PIVOT-5523 The context value expiration time can now be configured by ActiveViamProperty.CONTEXT_VALUE_MANAGER_VALUE_LIFETIME_PROPERTY. This will affect the calculated members created by Excel.
  • PIVOT-5964 Added builder for GetByKeyQuery.
  • PIVOT-6840 Added query tags on GetByKeyQuery.
  • PIVOT-7072 Added IDatabase.getMasterHead(). It will replace IDatastore.getHead().

Changed​

  • PIVOT-5446 Error messages that are printed when accessing the reserved position for null values in a NullableDictionary are now clearer.
  • PIVOT-6984 DirectQuery for ClickHouse correctly reports unsupported table Engines. This was previously unchecked, causing to unknown behavior when used with incorrect engines.

Deprecated​

  • PIVOT-3121 Deprecating the following datastore classes as part of internal cleaning of unused classes:
    • com.qfs.store.query.IFieldCriterion
    • com.qfs.store.query.IRecordMatcher
    • com.qfs.store.query.plan.IStoreQueryPlanToBe
    • com.qfs.store.query.plan.impl.QueryPlanToBe
    • com.qfs.store.record.IFieldMatchRecordProcedure
    • com.qfs.store.record.flatten.IRecordFlattenerFactory
    • com.qfs.store.transaction.ILogAction
    • com.qfs.store.transaction.impl.MoveLogAction
  • PIVOT-6175 com.qfs.store.part.IPartitioning#getMaxPartitionId() is deprecated in favor of PartitioningSnapshot#getMaxPartitionId().
  • PIVOT-6175 com.qfs.store.part.impl.PartitioningUtil#initializePartitioning(IPartitioning, IPartitioning, TIntIntMap) is deprecated in favor of PartitioningSnapshot#from(IPartitioning, TIntIntMap, PartitioningSnapshot).

Removed​

  • PIVOT-6051 DirectQuery: the property com.activeviam.properties.impl.ActiveViamProperty.MANAGER_FEEDING_TIMEOUT has been deprecated to avoid the misconception that the query was cancelled by ActivePivot.

Fixed​

  • PIVOT-5288 Fix TimeLineHandler impact computation for distribution.
  • PIVOT-5316 Fix issue with not seeing all processor cores for Windows machines with 64+ CPUs.
  • PIVOT-6213 DatastoreTransactionQueryRunner no longer creates a transaction object on a partition when there is a transaction on the store.
  • PIVOT-6506 Now stopping the application will wait for the cache to be discarded.
  • PIVOT-6640 Atoti Python API: Ignore non-JSON files when loading translations.
  • PIVOT-6642 Improve thread synchronization in the MDX engine.
  • PIVOT-6669 Speed up query using the Mdx range operator in subselects.
  • PIVOT-6792 Fix race condition in distribution. The issue appeared intermittently on consecutive branch creation and deletion.
  • PIVOT-6796 Fix performance regression when feeding copper defined analysis hierarchies.
  • PIVOT-6817 DirectQuery: fix issue with count in some query using multi row vector tables.
  • PIVOT-6834 Fix ClassCastException when collecting query plan retrieval timings.
  • PIVOT-6917 Distributed drillthrough queries properly function when one of the application's data nodes declares concealed hierarchies.
  • PIVOT-6987 and PIVOT-7068 DirectQuery: fix issue with SQL generation when querying multi row vector tables while filtering on another table.
  • PIVOT-7034 DirectQuery: fix possible deadlock when managing update-where triggers in the datastores bundled in DirectQuery applications.
  • PIVOT-7053 Fix Unknown coordinate exception on Analysis hierarchy
  • PIVOT-7054 DirectQuery: fix issue with SQL generation in case of joins towards multi row vector tables when the local and external table names of the target table are different.

6.0.3​

2023-03-07

Added​

  • PIVOT-5565 IJoinMeasureDescription#setMapping(Collection) is added to allow to use a level in a copper join more than once. Its corresponding getter is IJoinMeasureDescription#getMappingToCube().
  • PIVOT-6168 Add execution context compute time per retrieval to query plan, only available for bitmap retrievals.
  • PIVOT-5875 Added the MedianFunction aggregation function.
  • PIVOT-5929 Allow to add a prefix to the names generated in the content server with ActiveViamProperty.CONTENT_SERVICE_NAME_GENERATOR_PREFIX. This is done to easily differentiate between artifacts and assets created on different content server instances.
  • PIVOT-6644 DirectQuery: Add validator on joins for external databases;
  • PIVOT-6645 DirectQuery: Add validator on null values in multi-column primitive vectors.
  • PIVOT-6646 DirectQuery: Add validator on null values in multi-rows primitive vectors.

Deprecated​

  • PIVOT-5565 IJoinMeasureDescription#getMapping() is deprecated in favor of IJoinMeasureDescription#getMappingToCube() returning a collection of mapping between levels and fields. This allows to map the same levels multiple times to different fields. The setter IJoinMeasureDescription#setMapping(Map) has been deprecated as well.
  • PIVOT-6672 com.qfs.distribution.impl.QueryCubeSync#view in favor of com.qfs.distribution.impl.QueryCubeSync#getView().
  • PIVOT-6687 com.qfs.pivot.agg.IAggregateStore#getMeasureColumnIndex(String) is deprecated, in favor of com.qfs.pivot.agg.IAggregateStore#getColumnIndex(String).

Fixed​

  • PIVOT-5744 Wait for result size task to be done before discarding retrieval's result.
  • PIVOT-6581 Fix issue when using DirectQuery on Snowflake with a proxy.
  • PIVOT-6596 Fix NullPointerException when querying a database with a non null condition and a null additional condition.
  • PIVOT-6730 MDX Calculated Members properly work again in polymorphic distributed applications.
  • PIVOT-6745 Clustering fields are detected in join tables to trigger multi step queries.

6.0.2​

2023-02-10

Added​

  • PIVOT-5874 Add the Copper#argmax and Copper#argmin functions.
  • PIVOT-5981 Support Microsoft SQL Server (mssql) version 2017, 2019 & 2022

Changed​

  • PIVOT-5031 Restore printNumaConfiguration() for Aggregate Providers.
  • PIVOT-5693 Reduce transient memory footprint of the JIT aggregate provider. This is particularly crucial for use cases including a large number of small transactions.
  • PIVOT-5317 The property ActiveviamProperty#DATA_CUBE_REST_ENDPOINT_PROPERTY has now been split into two properties, ActiveviamProperty#DATA_CUBE_REST_ENDPOINT_PORT_PROPERTY and ActiveviamProperty#DATA_CUBE_REST_ENDPOINT_SUFFIX_PROPERTY.
  • PIVOT-6036 IStatisticsQueryRunner.estimateColumnCardinality now throws CannotEstimateColumnCardinalityException if the estimation is not possible (instead of various Exceptions).
  • PIVOT-6046 Now the query engine checks for interruptions during the planning phase. This is done to avoid cases where the query involving a lot of filtering blocks the session.
  • PIVOT-6204 Allow users to disable the initialization of an application in DirectQuery beans. The bean method #startApplication from base configuration classes can be disabled using the property activeviam.directquery.spring.enableLegacyInitialization. Users should instead do their own initialization using tools like CommandLineRunner.
    This applies to ABigQueryConfig, AClickhouseConfig, ARedshiftConfig, ASnowflakeConfig and ASynapseConfig.
  • PIVOT-6210 internal IActivePivotSession have been removed from all distributed messengers and messages.
  • PIVOT-6210 internal the constructor parameter Id (referring to the session id) has been removed from all messenger constructors.
  • PIVOT-6210 Getting/Setting the range sharing limit using JMX is now located in the pivot MBean instead of the QueryPlan MBean.
  • PIVOT-6210 Distributed messages implementing the IDatabaseAware interface will be injected a database.
  • PIVOT-6211 Database queries correctly fail when having several fields with the same alias.
  • PIVOT-6436 Change packages of internal Database query objects erroneously located in public packages.
  • PIVOT-6521 internal IActivePivotSession and its subclasses have been removed.
  • PIVOT-6521 The class MdxQueryUtil provides utility classes to execute MDX queries.

Deprecated​

  • PIVOT-6204 The method #apManagerInitPrerequisitePluginInjections from DirectQuery base configuration classes has been deprecated. Users should instead do their own initialization using tools like CommandLineRunner.
    This applies to ABigQueryConfig, AClickhouseConfig, AMsSqlConfig, ARedshiftConfig, ASnowflakeConfig and ASynapseConfig.

Removed​

  • PIVOT-5317 Remove the syntax allowing to define multiple endpoint within a single application.

Fixed​

  • PIVOT-4134 Fix NullPointerException in the query plan caused by invalid locations or measures in Prefetchers.
  • PIVOT-5541 In a distributed application, a restart of a messenger will now properly free the port it was previously bound to. The freed port will be reusable, meaning that a messenger will no longer be able to use its entire dedicated range of ports.
  • PIVOT-5767 Wait for the datastore to close before stopping QfsPools.
  • PIVOT-5994 Children of range-sharing retrievals are correctly printed as dependency in the query plan.
  • PIVOT-6009 Correctly name Spans after their creation.
  • PIVOT-6015 Fix copper first objects comparator being ignored
  • PIVOT-6133 Fix deadlock when allocating partition content.
  • PIVOT-6196 Fix an issue where we couldn't restart a DistributedMessenger when it was already paused.
  • PIVOT-6203 Fix an issue with StoreFieldMonitor#histogram() in JMX console producing ClassCastException on dictionarized fields.
  • PIVOT-6633 Fix NullPointerException during continuous queries registration in distributed setup.
  • PIVOT-6500 Support KPIs in Data Export CSV tabular format.
  • PIVOT-6577 Fix ClassCastException occurring when aggregating the contributors.COUNT measure.

6.0.1​

2022-12-07

Added​

  • PIVOT-6113 Add a dictionarize method on Field in DirectQuery databases.

Changed​

  • PIVOT-5852 The WatcherService will now create its WatcherThread with the same tracing context as the parent task (for example a file listener).
  • PIVOT-5989 Performance improvement in DirectQuery's generated SQL queries by pushing some conditions in the join clause.
  • PIVOT-6122 internal ClickHouse DataValidator is now package-private and final.
  • PIVOT-6122 internal IStreamListenerRegistration, returned by IDataStreamer#registerListStreamView, is now correctly public. It migrated from com.activeviam.database.internal.view into com.activeviam.database.api.query.
  • PIVOT-6122 internal The listener type com.activeviam.database.internal.view.IListViewListener, required by IDataStreamer#registerListStreamView has been made public as com.activeviam.database.api.query.IStreamListViewListener.
    The internal method #getPriority() has been removed. The internal method #onSelfFailure(Throwable) has changed not to return any value, preventing it from being able to rollback a transaction.
  • PIVOT-6122 internal DatabaseId entities, returned by various methods like IDatabaseSchema#getDatabaseId() is now public, moving from the package com.activeviam.database.internal.schema to com.activeviam.database.api.schema.
  • PIVOT-6122 internal the following public classes have been final, to avoid internal tempering:
    • com.activeviam.database.api.query.PointAggregateStore
    • com.activeviam.database.api.query.ResultAcceptorWithoutPartitioningWrapper
    • com.qfs.store.selection.impl.Selection

Deprecated​

  • PIVOT-5877 The ILocalityIdentifier#canBeLocalized is deprecated. Use ILocalityIdentifier#localMeasurePredicate instead.

Removed​

  • PIVOT-6081 The PartialProviderDefinition.NO_DIMENSION_DEFINED flag for unspecified dimension names has been removed

Fixed​

  • PIVOT-5867 AHibernateContentService's locking strategy is improved to lock smaller unit of works. This will prevent potential deadlocks.
  • PIVOT-5978 Prevent OutOfMemoryError crashes in the XMLA Servlet when attempting to send a message that does not fit in a byte array of size Integer.MAX_VALUE. RuntimeException will be thrown instead.
  • PIVOT-5997 Adaptive JIT is now always used instead of JIT with DirectQuery.
  • PIVOT-6027 Fix APM Spring configuration.
  • PIVOT-6053 Fix performance issue when reducing the number of allowed concurrent queries in the MdxQueryBouncer.
  • PIVOT-6112 Fix issue in DirectQuery filtering on the wrong dates when pushing queries to Snowflake.

6.0.0​

2022-11-08

Added​

  • PIVOT-2930 Add ICubeFormatter.getMeasureCaption(String) that will replace getMeasureCaptions().
  • PIVOT-4747 New API: IDatabase. The IDatastore is now a specialization of IDatabase.
  • PIVOT-4887 New Rest Bulk API for Content Server. See RestBulkApiServerController.
  • PIVOT-5077 Replication is disabled by default for the contributors.COUNT measure on distributed cubes. It can be re-enabled using the cube description builder, right after stating asQueryCube().
  • PIVOT-5177 IPartialAggregateProviders can be filtered on certain levels (whether they are handled by the provider).
  • PIVOT-5180 Added springdoc dependency in the sandbox to showcase how to list all rest endpoints.
  • PIVOT-5181 Lead/Lag/First/Last measures can be configured to shift to the exact location or the first non-null element. New builders are available in the class com.activeviam.copper.api.Copper.
  • PIVOT-5183 Add rebuild operations for indexes to the datastore API.
  • PIVOT-5207 New endpoint in the Database Rest API to retrieve actual number of records in the store. See IDatabaseRestService. It replaces the IDatastoreRestService.
  • PIVOT-5317 Data cube now can have a unique identifier using the fluent builder API IDataClusterDescriptionBuilderWithIdAndMessengerDefinition#withUniqueIdentifierInCluster.
  • PIVOT-5317 Expose remote data cube physical addresses in the query cube.
  • PIVOT-5429 Introduced new interface IAnalysisHierarchyDescriptionProvider to describe the properties of the Java-based analysis hierarchies.
  • PIVOT-5561 Introduce a new API to write aggregation bindings on objects and vectors. See [this migration note](/products/atoti/server/6.0.12-sb3/docs/release/migration_notes#Aggregation module changes) for details.
  • PIVOT-5609 Serialization of null values: introduce a new method ConditionToJsonNodeHelper.convertMapIncludeAll that includes keys with null values in the JsonNode. It relies on a dedicated ObjectMapper (see JacksonSerializer#getObjectMapper(true))
  • PIVOT-5631 ActiveUIResourceServerConfig and AdminUIResourceServerConfig are now core classes. You no longer need to copy them from the Sandbox to your projects.
  • PIVOT-5734 Copper#constant now supports Object Vectors.
  • PIVOT-5871 New aggregation functions were added to the core product. See the what's new page
  • ADMINUI-10 In the sandbox, a /ui folder is created in the Content Server at startup if it does not exist yet. It contains the file structure necessary for ActiveUI to work correctly.

Changed​

  • PIVOT-3121 IPostProcessedMeasure no longer has a generic parameter.
  • PIVOT-4747 The Datastore API is no longer accessible from an ActivePivot. It was replaced by the Database API. See the migration notes for details.
  • PIVOT-5116 In the StoreDescriptionBuilder all operations on fields should be done before defining the partitioning. For example, you cannot call addField() after withPartitioning().
  • PIVOT-5135 Creating a sub-second QueriesTimeLimit will now throw an IllegalArgumentException stating that a time limit must have a duration over 1 second.
  • PIVOT-5145 IPluginValue and IExtendedPluginValue are not Serializable by default anymore.
  • PIVOT-5295 MDXCellSet now always returns the list of default members, even if the cell set itself is empty.
  • PIVOT-5298 All ActivePivot named threads are now prefixed by activeviam and suffixed by worker-i where i is the thread count number.
  • PIVOT-5333 Increase severity for full reset and pending initial discovery messages to WARNING in distributed clusters.
  • PIVOT-5349 The method CSVSource#setFileIdentifierComparator was moved to ICSVSourceConfiguration, which is now created through the builder CSVSourceConfigurationBuilder.
  • PIVOT-5373 ICursor is now auto-closeable. See migration notes.
  • PIVOT-5378 The INumaNodeSelector interface has been separated into INumaNodeSelector and IDatastoreNumaNodeSelector, which live in different modules.
  • PIVOT-5383 Tracing has been removed from APM and is now part of the core product.
  • PIVOT-5417 All rest endpoints have been migrated to Spring MVC and the JsonResponse wrapper removed.
  • PIVOT-5422 IDatastoreSchemaDescription.getDictionaryGroups now uses sets of StoreField.
  • PIVOT-5429 Numerous ActiveViamProperty and JwtConfig property values were renamed to uniformize their naming conventions (see the migration notes).
  • PIVOT-5429 Pivot hierarchy descriptions now use the IHierarchyDescription interface.
  • PIVOT-5429 Changed how the Analysis hierarchies are defined. See migration notes.
  • PIVOT-5468 The Datastore REST API is now called Database REST API. Queries will behave like if the autoDetectOuterReferences parameter was set to true in 5.11.
  • PIVOT-5486 Sub-interfaces and classes of conditions are now internal.
  • PIVOT-5486 The IDictionaryCursor interface was removed. By default, IListQueryRunner#run and similar methods will return plain values. Use the option IListQueryRunner#withDictionarizedValues() to retrieve dictionarized values.
  • PIVOT-5486 DatastoreQueryHelper.selectDistinct(...) was changed to only support querying distinct values of a field belonging to a given table.
  • PIVOT-5486 Renamed PARTIAL_PROVIDER_SELECTOR_PLUGIN_KEY_PROPSERTY to PARTIAL_PROVIDER_SELECTOR_PLUGIN_KEY_PROPERTY (typo in PROPERTY).
  • PIVOT-5490 The methods in HierarchiesUtil no longer throw checked exceptions.
  • PIVOT-5528 Change the behavior of Copper Join Hierarchies that come from a field joined with a datastore reference on the Copper store. In the past the results could be empty when the reference failed.
  • PIVOT-5548 Aggregate provider's ProviderCoordinate now contains the provider's name instead of the index. Renamed MemoryStatisticConstants#ATTR_NAME_PROVIDER_INDEX to ATTR_NAME_PROVIDER_NAME.
  • PIVOT-5556 Provided a clearer exception message when attempting to create a hierarchy misusing MultiVersionAxisHierarchy#FIELD_PATH_PROPERTY. This property replaces the FIELD_EXPRESSION_PROPERTY.
  • PIVOT-5582 Namespace and version are by default deduced from namespace prefix in ASpringVersionedRestServerConfig.
  • PIVOT-5637 Partial providers can no longer be filtered on levels they do not handle.
  • PIVOT-5683 Constructors of AAggregation and its implementations were changed. Passing the type of the source is now mandatory. Implementations that can enforce their own aggregated type now do so.
  • PIVOT-5723 All post-processor output types are now nullable by default.
  • PIVOT-5732 Changed the implementation of the or operator on multiple bitmaps.
  • PIVOT-5792 Changed the API to create Dynamic conditions on the Datastore/Database. It is now mandatory to name a parameter using the as(String) method.
  • PIVOT-5820 Unify IChunkSetCursor and IArrayReaderWithCursor into a single concept, ITableCursor.
  • PIVOT-5834 Static methods in BaseConditions have been renamed. They are now all starting with a lower case. Lesser and LessOrEqual have been changed to less and lessOrEqual.
  • PIVOT-5839 Chunks#DEFAULT_CHUNK_SIZE now cannot exceed 65536.
  • PIVOT-5839 ActiveViam Property NATIVE_MEMORY_CACHE_RATIO_PROPERTY (-Dactiveviam.nativeMemoryCacheRatio) now defaults to 0.4, instead of 0.8.
  • PIVOT-5840 IAxisLevelDescription's propertyName field is now named selectionField to clarify what it represents. The getPropertyName() and setPropertyName(String) methods have been renamed accordingly.
  • PIVOT-5914 Sandbox applications use AdminUI version 5.0.7.

Deprecated​

  • PIVOT-4747 Deprecated IDatastoreVersion in favor or IDatabaseVersion
  • PIVOT-4747 Deprecated IReadableDatastore#getHead() in favor of getHead(String) and IReadableDatastore#getBranches() in favor of getHeads().
  • PIVOT-4747 Deprecated several of the signatures ofDatastoreQueryHelper#createGetByKeyQuery.
  • PIVOT-5429 Several methods from AAnalysisHierarchy have been deprecated, use IAnalysisHierarchyDescriptionProvider instead (see migration notes).
  • PIVOT-5429 Deprecated the ability to have introspecting levels in AAnalysisHierarchy.
  • PIVOT-5486 DistinctAcceptor is now considered in internal class. Its signature changed to work on plain records without dictionary values and don't require the number of partitions.
  • PIVOT-5536 The name of the ActivePivotManager's monitoring bean is now a constant: ActivePivotManager.
  • PIVOT-5561 The following abstract classes are deprecated: AGenericBaseAggregationFunction, AGenericAggregationFunction, AGenericVectorAggregationFunction, AVectorAggregationFunction.
  • PIVOT-5905 The method hasNext in ICursor has been deprecated, use next directly.
  • PIVOT-5905 The method rewind of ICursor will become an internal method.

Removed​

  • PIVOT-4437 Remove ActivePivotDatastorePostProcessor, NumaSelectorPostProcessor and WeightMaximizingNumaSelectorPostProcessor.
  • PIVOT-5175 Dropped SOAP services.
  • PIVOT-5180 Dropped CXF.
  • PIVOT-5320 Removed legacy post-processors. See the migration notes for details.
  • PIVOT-5590 Removed realtime drillthrough feature from Excel add-in.
  • PIVOT-5590 Removed native output type "Excel" from the list of supported drillthrough output types.
  • PIVOT-5378 Removed getDatastore methods from ISchemaSelectionSession and IActivePivotSchemaSelectionSession
  • PIVOT-5413 Removed IActivePivotDescription.IDescriptionPostProcessor. Description post-processing does not need to be handled by the core product.
  • PIVOT-5413 Removed the ability to define an Analysis aggregation procedures in a hierarchy description.
  • PIVOT-5468 Removed the autoDetectOuterReferences parameter in Database service.
  • PIVOT-5561 Removed internal attributes and methods to use when writing an aggregation binding for Objects. For example, it is not required nor possible to manually call #write(int, Object) and writeBoolean(int, boolean) when doing vector aggregation using AAggregationBindingCloneable, AVectorAggregationBinding or derivatives of SumVectorAggregationBinding. See [this migration note](./migration_notes#Aggregation module changes) for details.
  • PIVOT-5561 Removed ABasicAggregationBindingCloneable, replaced by a refactored AAggregationBindingCloneable.
  • PIVOT-5574 Removed Reporting and Scheduling Service.
  • PIVOT-5605 Removed class SecurityFromCubeToDatabaseFilterHook.
  • PIVOT-5668 Removed IStreamPublisher form IStreamSnapshotService. The snapshot service now uses the same publisher as the streaming service.
  • PIVOT-5680 Removed KeepLastBranchEpochPolicy.
  • PIVOT-5785 Removed getName method from ICursor, use getFieldName on IRecordFormat.
  • PIVOT-5792 Removed the ability to give an index to a Dynamic condition. One should now give a name.
  • PIVOT-5834 Removed BaseConditions.True() and BaseConditions.False(), use BaseConditions.TRUE and BaseConditions.FALSE instead.
  • PIVOT-5840 AxisLevelDescription(String levelName) was removed. A selectionField (formerly propertyName) must now be specified to avoid unexpected behavior.

Fixed​

  • PIVOT-5416 Fix computeAggregationProcedureTiming for cached retrievals.
  • PIVOT-5622 Correctly define constructors for generated Bean property accessors.
  • PIVOT-5809 Fix ArrayIndexOutOfBounds exceptions in ActivePivotAggregatesRetriever.
  • PIVOT-5774 Fix a DiscardedDataException potentially occurring in Vector aggregation operations
  • PIVOT-5783 Fix the toString() method for CsvCharacterCodingError.
  • PIVOT-5813 Fix NullPointerException that could occur with Copper's window.
  • PIVOT-5873 Fix thread vector allocator un-consistency. Previously, a thread requesting a transient allocator could end-up with a direct memory allocator.
  • PIVOT-5887 Arrange for a spare thread for continuous MDX updates. Previous implementation could cause live locks in case of work-stealing.

5.11.4​

2022-09-26

Added​

  • PIVOT-4365 Add an MdxQueryBouncer limiting the number of concurrently running mdx queries. See What's New? page for configuration details.
  • PIVOT-5027 Empty partitions are automatically dropped at the end of transactions.
  • PIVOT-5077 Add IDistributedActivePivotInstanceDescription.NO_REPLICATION_MEASURES property to disable replication for certain measures on distributed cubes.
  • PIVOT-5631 ActiveUIResourceServerConfig and AdminUIResourceServerConfig are now core classes. You no more need to copy them from the Sandbox to your projects.

Changed​

  • PIVOT-4934 Adjust Logging levels for received statements on the MDX REST endpoint to INFO.
  • PIVOT-5030 Disable RangeSharing for Post-Processors when an Analysis Hierarchy is expressed in the queried location.
  • PIVOT-5304 Improve the performance of the code handling the generation of the locations when analysis hierarchies are expressed.
  • PIVOT-5626 Skip initialization of previously failed post-processors
  • PIVOT-5657 Switch outer reference policy to auto-detection mode for the queries executed from Datastore MBeans.
  • PIVOT-5671 Log the complete APM monitor result's only when detailed logging is enabled.
  • PIVOT-5677 Activiti dependencies upgraded from 5.22.0 to 5.23.0.
  • PIVOT-5728 Improve JIT retrievals completion performance in some applications with a high partition count.

Fixed​

  • PIVOT-5041 Clarify the behavior and error message of Licensing.reload() and its underlying MBean.
  • PIVOT-5365 Fix ActivePivotVersion releasing its reference to the datastore too early. In some cases, a race condition could lead to wrong results when using a KeepLastEpochPolicy and querying an old not marked ActivePivotVersion with a JustInTime provider.
  • PIVOT-5479 Fix ClassCastException when querying the drillthrough headers on a distributed cube.
  • PIVOT-5458 Correctly propagate exception in ExternalDatastoreRetrieval.
  • PIVOT-5502 Fix cached data mutated in DatastoreService.
  • PIVOT-5508 Fix the creation of the ColumnIntegerRowMapping memory statistics for MAC.
  • PIVOT-5515 Fix ArrayIndexOutOfBoundsException in ChunkedBitmap when adding two bitmaps.
  • PIVOT-5539 Fix NullPointerException when discarding transient results from a distributed measure with no value.
  • PIVOT-5502 Fix cached data mutated in DatastoreService.
  • PIVOT-5575 Fix a performance issue impacting queries on cubes having partial providers and a large amount of hierarchies.
  • PIVOT-5639 Fix an issue with nested combine operations with context value dependencies in Copper.
  • PIVOT-5640 Fix an issue where unrelated store partitions could generate unnecessary transient objects. This reduces the memory footprint of small transactions.
  • PIVOT-5661 Fixed potential deadlock in JDBCTask when all JDBCAppendRunnable crashed.
  • PIVOT-5671 Hide sensitive information in logs.
  • PIVOT-5713 Fix ClassCastException during the recording of distributed messages statistics.
  • PIVOT-5759 Fix Deadlock when allocating partition content.
  • PIVOT-5777 Fix wrong parameter order in RemoteContentService.

5.11.3​

2022-05-06

Added​

  • PIVOT-5301 BottomCountFunction and TopCountFunction should accept a string expression as a comparator.
  • PIVOT-5297 Support all arrow TimeStampVector types: TimeStampNanoTZVector, TimeStampMicroTZVector, TimeStampMilliTZVector, TimeStampSecTZVector.

Added​

  • PIVOT-5301 BottomCountFunction and TopCountFunction should accept a string expression as a comparator.

Changed​

  • PIVOT-5202 Improve the Copper doc on map methods to specify that the cells must be written into at each iteration.

Fixed​

  • PIVOT-5318 Fix ClassCastException in the PlanningGraph
  • PIVOT-5333 Increase severity for full reset and pending initial discovery messages to WARNING in distributed clusters.
  • PIVOT-5395 Fix the creation of the HashMapRowMapping memory statistics for MAC.
  • PIVOT-5407 Fix Frequent Value compression not properly handling null values.
  • PIVOT-5428 Fix ArrayIndexOutOfBoundsException when using withBatchSize in parquet source.
  • PIVOT-5443 Fix SimpleCubeDescriptionBuilder not properly setting shared context values.
  • PIVOT-5479 Fix ClassCastException when querying the drillthrough headers on a distributed cube.

5.11.2​

2022-04-01

Added​

  • PIVOT-5232 A UI to browse the Datastore. See AdminUIResourceServerConfig in the Sandbox.
  • PIVOT-5246 Add ChunkSingleBoolean to compress Boolean chunks containing the same value for all positions.

Changed​

  • PIVOT-5232 Breaking Moved JungSchemaPrinter and associated classes to datastore test artifact. spring-boot, netty, google-guava, azure-identity, azure-security-keyvault-keys and azure-storage-blob are updated to the latest version.
  • PIVOT-5239 Improve DirectedGraph.getEdgesTo.
  • PIVOT-5275 Record reading and writing operations now strictly follow the JVM's automatic type widening described in the JVM's spec. The previous behavior was to only allow the operations corresponding to the field type.
  • PIVOT-5322 Renaming of exposed properties related to consul service configuration within APM.
  • PIVOT-5367 Renamed properties intermediateSize and transientSize in QueryResultLimit to intermediateLimit and transientLimit, respectively.
  • PIVOT-5411 Update Spring Boot to 2.6.6 which includes a vulnerability fix.

Removed​

  • PIVOT-5232 BREAKING ContentServerResourceServerConfig was removed. It is replaced by AdminUIResourceServerConfig (See the Sandbox).

Fixed​

  • PIVOT-5098 Fixed an IllegalStateException when performing queries containing cached aggregates for primitive measures and primitive-aggregated measures.
  • PIVOT-5188 Fix potential bug in AnalyticFunctionPostProcessor where partitioning levels were not properly set.
  • PIVOT-5218 Improve performance of the Mdx NonEmpty function
  • PIVOT-5259 Correctly propagate custom aggregation function used for result reduction in distributed post-processors.
  • PIVOT-5280 Using the flexible datastore no longer removes the insertion time and commit time update procedures.
  • PIVOT-5291 Correctly group monitored JMX APM components and fix some variable visibility issues.
  • PIVOT-5308 Clarify the javadoc of Copper.StoreLookup when the specified store has nullable key fields.
  • PIVOT-5309 Correctly set elapsed time in aggregate retrievals.
  • PIVOT-5310 Fix retrieval result size figures when printing the query plan. In some cases, results were discarded before their result size computed.
  • PIVOT-5311 Fix bug where hierarchy visibility was set to false but was still visible.
  • PIVOT-5328 Fix an NullPointerException when having multiple threads discarding the same transient result at the same time.
  • PIVOT-5330 Fix issue with store size and cursor not being printed when using SchemaPrinter.
  • PIVOT-5335 Fix aggregates cache not being updated in Query Cubes after transactions without any impact on the pivot's structure.
  • PIVOT-5342 Fix bug where we could not query joined store's lookup measure on grand total.
  • PIVOT-5396 The reversed comparators did not return their plugin key.

5.11.1​

2022-01-12

Added​

  • PIVOT-5170 Add result size reporting for distributed query plan print.

Changed​

  • PIVOT-5009 Prevent users from creating Copper join hierarchies based on a store field included in the mapping.
  • PIVOT-5238 Parallelize ActivePivotManagerBuilder's postProcess to reduce cubes creation time.

Fixed​

  • PIVOT-5095 Fix real-time update on query with epoch hierarchy. Previously, real-time updates on such queries resulted in branch disappearing from the updated result. See Real-Time What-If Queries section.
  • PIVOT-5189 Fix IIterableAggregatesRetrievalResult#transferValues that could copy the wrong aggregates.
  • PIVOT-5195 Fix bug in the revert to version. Some records were not reverted in some edge cases.
  • PIVOT-5208 Enforce PostProcessor initialization when requesting the dataType of a post-processed measure.
  • PIVOT-5210 Fix a bug in lead, lag, first, last and window Copper functionalities when relying on a Context Value measure.
  • PIVOT-5224 License is now redacted when logged.

5.11.0​

2021-11-26

Added​

  • PIVOT-4449 Added a detailed reporting of CSV parsing errors. Use the CSV_PARSING_REPORT_ENABLED property to enable it, and CSV_PARSING_REPORT_MAX_ANOMALIES to limit its size.
  • PIVOT-4639 Add parameter to auto-detect outer references in DatastoreService.
  • PIVOT-4642 Integrate APM (Application Performance Monitoring) library into ActivePivot.
  • PIVOT-4653 ActivePivot's custom data structures following com.qfs.chunk.IArray and com.qfs.chunk.IWritableArray can now spawn a cursor. This cursor can be moved up and down the array and perform read/write operations at its current position.
  • PIVOT-4655 Add a data export service to download or export MDX queries.
  • PIVOT-4676 The query plan now includes aggregation procedures time and external datastore retrievals for PrimitiveAnalysisAggregationRetrieval. It also includes the name of the used partial provider for PartialPrimitiveAggregatesRetrieval.
  • PIVOT-4677 Integrate the Excel Add-in library into ActivePivot.
  • PIVOT-4708 Added virtual hierarchies experimental feature. Those are hierarchies whose members are not materialized. A hierarchy can be defined as virtual using the hierarchy builder API .virtual().
  • PIVOT-4715 Added an interface ICanUseOtherReferenceOrBuild in the selection fluent builders for ease of use with Spring Beans.
  • PIVOT-4733 Allow to override measure aggregator xmla property with CopperMeasure#withXmlaAggregator(int).
  • PIVOT-4804 Throw an exception when creating measures names with commas.
  • PIVOT-4806 License path is now printed in the log.
  • PIVOT-4815 Ability to name (give a alias) when selecting field in a RecordQuery or a GetByKeyQuery. Before the name was the path of the field.
  • PIVOT-4824 Include epochId of the max and latest continuous query statistics.
  • PIVOT-4847 Add mark() and unmark() on IDatastoreVersion.
  • PIVOT-4913 The JDBC Source will now log the exceptions that make its workers exit the parallel processing of records.
  • PIVOT-5037 The Copper tester now supports hasValues() in every case and a new method withDelta() to set the allowed delta between the values given by the user in hasValues() and the actual values returned by the measures.
  • PIVOT-5059 Partial providers can now have a name using the fluent builder API PartialBuilder#withName(..).
  • PIVOT-5060 Support encoding and decoding NaN and +/-Infinity in JSON.
  • PIVOT-5069 Add the currently written record in case of type error in IRecordFormat#newRecord().
  • PIVOT-5118 Add a toString() to CloudCsvDataProvider() for better error message.
  • PIVOT-5170 Log query results size by default when printing the query plan. Note that DistributedAggregatesRetrieval doesn't support such feature yet.

Changed​

  • PIVOT-3978 Partitioning fields are now automatically dictionarized.
  • PIVOT-4315 The Copper Typing API is now simpler, only allowing to call withType after operations like plus, minus, etc.
  • PIVOT-4498 The ActiveViamPropertyFromSpringConfig.resolve() bean is now named ActiveViamPropertyFromSpringConfig.SPRING_PROPERTIES_RESOLVER.
  • PIVOT-4484 Deprecate usage of OutputStream in TableBuilder and add usage of StringBuilder
  • PIVOT-4653 IVector and IBlock no longer extend IWritableArray.
  • PIVOT-4653 IPostProcessor abstract implementations are given a version 2, such as ABasicPostProcessorV2, ADynamicAggregationPostProcessorV2 and AAdvancedPostProcessorV2. See the migration notes for details.
  • PIVOT-4747 ReferencedField was replaced by ReachableField.
  • PIVOT-4781 StreamSnapshotService#snapShotStreams(String) is now compatible with StreamSnapshotService#loadStreams(String). Now, both operation ignore STOPPED streams. Use StreamSnapshotService#snapShotAllStreams(String) to snapshot all streams including the stopped ones.
  • PIVOT-4789 Provide possible solutions in error message when we failed to initialize the SaferNative class in LinuxPlatform.
  • PIVOT-4802 Reduce the memory consumption of ModifiedLocation, primarily used in cumulated aggregates and in shifted measures.
  • PIVOT-4808 Report error if the ActivePivotSchema fails to attach.
  • PIVOT-4813 Update ActiveMonitor REST API documentation to correctly illustrate how a new monitor can be added.
  • PIVOT-4820 Change constructors of KeepLastEpochPolicy and FlexibleDatastoreEpochPolicy to use Duration instead of long.
  • PIVOT-4833 AVectorAggregationFunction.aggregate method will now receive a vector allocator.
  • PIVOT-4834 Chunks reading and writing operations now strictly follow the JVM's automatic type widening described in the JVM's spec. The previous behavior was to only allow the operations corresponding to the chunk type.
  • PIVOT-4893 Vectors reading and writing operations now strictly follow the JVM's automatic type widening described in the JVM's spec. The previous behavior was to only allow the operations corresponding to the vector type.
  • PIVOT-4912 Add default implementation of CancellableCountedCompleter#afterCompute(Throwable) calling CancellableCountedCompleter#tryComplete().
  • PIVOT-4947 Return a lighter ActivePivot description in SOAP webservices called SoapActivePivotDiscovery.
  • PIVOT-4959 Reorganize the IVectorBinding API. Some methods have seen their two IVector parameters inverted, but they now all carry the less confusing names of left and right (rather than source and destination). The left vector will always receive the result of the vector operation.
  • PIVOT-4973 IReadableDatastore now implements IBranchedMultiVersion.
  • PIVOT-5003 Reorganize AMultiVersionAnalysisHierarchy and AAnalysisHierarchy to clearly state the methods to implement. The notable changes is that now, users must override AMultiVersionAnalysisHierarchy#processIntrospectedMember to complete paths of introspected members, instead of overriding #contributeMember directly. This change removed a lot of internal methods from the interface IAnalysisHierarchy.
  • PIVOT-5003 Introduce AAnalysisHierarchyV2. This class will replace AAnalysisHierarchy in ActivePivot 6.1.
  • PIVOT-5003 Correct typo by renaming IAnalysisHierarchyInfo#getUseIntrospection into IAnalysisHierarchyInfo#getUseIntrospection.
  • PIVOT-5032 Upgrade org.apache.hadoop to 3.3.1.
  • PIVOT-5021 Upgrade Spring Boot version to 2.5.4 and update dependencies.
  • PIVOT-5024 Use ChunkedIntegerArray instead of IntegerArray in AMultiVersionUniqueRecordIndex so that we can now put more than 300M records per partition.
  • PIVOT-5058 Implements the correct equals() and hashcode() for StoreDescription and VectorFieldDescription.
  • PIVOT-5123 Partitioning modulo 1 now throws an exception.
  • PIVOT-5129 Renamed AnalysigAggregationProcedureFactory to AnalysisAggregationProcedureFactory.
  • PIVOT-5168 Bump Spring version to 2.6
  • PIVOT-5168 Bump Jackson version to 2.13.0
  • PIVOT-5168 Bump Apache Arrow version to 6.0.1
  • PIVOT-5168 Bump Apache CXF version to 3.4.5

Removed​

  • APS-12776 Remove IEpochManager.mark(...) and IEpochManager.unmark(...). One should call them on the underlying IEpochHistory.
  • PIVOT-4244 Remove the XML descriptions of the cubes.
  • PIVOT-4747 ISelectionField was removed. Use SelectionField instead.
  • PIVOT-4748 StreamUtil.negate has been removed; we use Predicate.not instead
  • PIVOT-4776 IMultiMap was removed. Use a Map whose values are collections instead.
  • PIVOT-4986 Release more resources after dropping a partition (dictionaries of indexes that are not used by a reference can now be released every time).
  • PIVOT-5011 Remove the provided strings from exception messages in RestContentServer and JsonQueryService.
  • PIVOT-5029 IAggregatedMeasureAware was removed.
  • PIVOT-5084 Removed an invalid warning that was triggered when building Copper Hierarchies backed by a store. Said warning was stating these hierarchies shouldn't be factless.
  • PIVOT-5094 IMultiVersionStorePartition#drop(IDroppedPartitionInformation) does not take any argument anymore.
  • PIVOT-5117 Remove TransactionWrapper.

Fixed​

  • PIVOT-4487 Fix Copper post-processor bug where if the only underlying measures of a calculated measure were null or constant measures, the post-processor wouldn't be evaluated at all.
  • PIVOT-4735 Fix continuous handler registration issue. The issue happened in distribution when some underlying measures have not yet appeared thus causing a duplication in handler key of the parent measure.
  • PIVOT-4780 Fix unnecessary query plan logging when queryPlanSummary is enabled.
  • PIVOT-4800 Enhance query plan timing reports. This particularly fixes two man issues:
    • JITPrimitiveAggregatesRetrieval Partitioned retrievals for non-existing partitions used to display the largest execution time of their siblings. Now, only partitioned retrievals that performed some work at least and whose result is empty will be reported in the timing section, along with the other retrievals with result
    • In some cases, for a group of related retrievals (siblings), timing were not printed if one of the sibling has no result. This edge case has been fixed
  • PIVOT-4811 Fix ClassCastException in SingleValueFunction when working with arrays.
  • PIVOT-4814 Fix an issue where exporting the memory statistic of a chunk containing non-direct vector blocks would throw a NPE.
  • PIVOT-4816 Fix an issue occurring when merging two AggregateStores. This caused the merged result to include null values instead of the correct value for some measures.
  • PIVOT-4818 Fix an issue leading to incorrect transitions between IntegerArrayVersion.
  • PIVOT-4819 Fix UnsupportedOperationException in ActivePivotStatistics. The issue happened when computing transaction fact statistics on an incompatible type of transaction.
  • PIVOT-4862 Fix an export error in the IMemoryStatistic generation for TombStone classes.
  • PIVOT-4866 Fix NullPointerException in FilterTranslator. The issue could happen when translating a wildcard, e.g. null, to the data cube.
  • PIVOT-4881 Fix an issue where the resizing of IntegerArray and LongArray generated non-transient chunks regardless of its isTransient parameter.
  • PIVOT-4901 Prevent aggregates computation (i.e. GetAggregatesQuery) when mdx CELL PROPERTIES is equal to CELL_ORDINAL.
  • PIVOT-4923 Fix hiding subtotals issue that could not hide all the measures.
  • PIVOT-4935 Fix NPE when accessing an uninitialized datastore via MBean.
  • PIVOT-4944 Fix the cleaning method of the prefetch scope location when executing more than one aggregation procedure in a single PrimitiveAnalysisAggregationRetrieval.
  • PIVOT-4946 Fix an issue with Copper storeLookup that could fail to find its underlying measures, in the case where these measures were previously re-optimized.
  • PIVOT-4949 Fix thread safety for SharedDelegateDictionary#mapFromCell.
  • PIVOT-4951 Fix an edge case issue with IChunkInteger sparse compression.
  • PIVOT-4956 Fix GenericVectorBinding#applyAs.. operations for non-commutative operators.
  • PIVOT-4958 Fix the update of the ContentServer entries for branches containing characters that are illegal for FileSystem-specific paths.
  • PIVOT-4964 Fix an ArrayIndexOutOfBoundException happening when compressing chunks with freed rows as FrequentValueChunk.
  • PIVOT-4972 Fix branches not visible sometimes with continuous queries. The ISecurityDetails was not always propagated when registering queries.
  • PIVOT-4979 Fix missing fields issue in the analysis aggregation procedures when a cube filter is done on a Copper join.
  • PIVOT-4992 Fix dependency issue in content-server-spring.
  • PIVOT-4998 Reduce time spent at startup optimizing Copper measures
  • PIVOT-5010 Properly set the level type of a slicing Copper hierarchy
  • PIVOT-5016 Fix ArrayIndexOutOfBoundsException in the SetExpander of the Mdx engine.
  • PIVOT-5038 Fixed errors in usage counting caused by decrementing the usage too many times.
  • PIVOT-5050 Fix distributed post-processors initialization. This issue happened when chaining at least three post-processors where the bottom one in the chain is not initialized and caused missing dependencies propagation in some cases.
  • PIVOT-5062 Fix NullPointerException in PointListRetrievalResult in the case of a polymorphic distributed setup.
  • PIVOT-5068 Fix NullPointerException in the AggregatesResultCopier used by postprocessors.
  • PIVOT-5074 Fix SubCubeProperties#removeRestriction(dimension, hierarchy). The issue happened in distributed polymorphic setup where the same dimension from two data nodes includes different hierarchies and one of the dimension hierarchies cardinality is greater than 1.
  • PIVOT-5078 The Parquet Source correctly parses boolean vectors.
  • PIVOT-5097 Fix NullPointerException when registering a GetAggregatesQuery build using a location Collections not supporting null elements.
  • PIVOT-5115 Fix IllegalArgumentException during query plan logging when executionTimingPrint is enabled .
  • PIVOT-5119 Fix rollback process of store transactions.
  • PIVOT-5128 Fix a memory leak that could happen upon query failure/cancellation if an AggregatesCache was used. * PIVOT-5137 Fix LevelIdentifier.fromDescription and HierarchyIdentifier.fromDescription. They did not handle correctly the escaping of @.
  • PIVOT-5137 Fix LevelIdentifier.fromDescription and HierarchyIdentifier.fromDescription. They did not handle correctly the escaping of @.
  • PIVOT-5151 Fix QfsStrings.escape with $ as escape symbol.
  • PIVOT-5156 Fixed missing query plan retrievals. The issue happened when underlying retrievals with different partitioning were merged. Note that this does not affect query results nor the query plan export feature, but only the query plan printing.
  • PIVOT-5162 Fix potential NPE in PrimitiveAnalysisAggregationRetrieval when joining results.