Skip to main content

Changelog

For user-facing changes, refer to the What's New? page.

The format is based on Keep a Changelog and this project adheres to Romantic Versioning.

5.10.20​

2023-12-14

Changed​

  • PIVOT-8220 Remove Maven scopes compile defined in Atoti BOM. This avoids defining by mistake strange default scopes in user projects.

Fixed​

  • PIVOT-8182 In a distributed setup, unexpected errors that happened in one cube while sending a message should no longer cause off-heap memory leaks after failing to properly receive this message in another cube. These leaks are believed to be Netty buffers that were not properly released on error handling.
  • PIVOT-8220 Restore the Maven scope compile for the dependency antlr-runtime. This was causing NoClassDefFound errors at startup and errors when running any MDX query or operations on cubes.
  • PIVOT-8343 Improved record query performance when run on current thread by not creating empty completion tasks for non-matching partitions.
  • PIVOT-7838 Fixed class cast in ShiftMeasure.LevelShift#equals.

Security​

  • Upgrade project to SpringBoot 2.7.18 to fix CVEs. This now removes the dependency to netty-all:4.1.100, containing an unfixed CVE.

5.10.19​

2023-11-03

Added​

  • PIVOT-7176 The cube filter is now used in distributed cubes to limit the number of data nodes the query is sent to.

Fixed​

  • PIVOT-6669 Speed up query using the Mdx range operator in subselects.
  • PIVOT-7367 PIVOT-8013 Context values are now properly propagated along a chain of measures after going through Copper's measure optimization.
  • PIVOT-7580 Fix continuous MDX query initialization issue caused by uninitialized post-processor on a query cube.
  • PIVOT-7910 NeighborValuePostProcessor no longer throws an AssertionError if queried on a non-existing member of the shifting hierarchy.

Security​

  • PIVOT-8079 Migrate snakeyaml to 2.0 to address CVEs.
  • PIVOT-8079 Migrate netty-codec-http2 to 4.1.100.Final to address CVEs.
  • PIVOT-8079 Exclude dependency from org.apache.velocity:velocity to address CVEs.

5.10.18​

2023-09-27

Added​

  • PIVOT-6657 Add a warning when the license expires in less that 30 days. This is checked whenever a license is loaded.

Changed​

  • PIVOT-5622 Refactor BeanPropertyCompiler to be able to call parent constructors for beans.

Fixed​

  • PIVOT-7377 Fixed a parallelism issue, and improved the speed of an aggregate provider transaction in a very specific case: multiple database partitions only contribute to a unique partition of an aggregate provider.
  • PIVOT-8322 Fixed NotSerializableException when using Copper.userDefinedAgg on a query cube.
  • PIVOT-7023 Fixed "A result already existed for the scope" error when running a full refresh on continuous queries

Security​

  • PIVOT-7695 Migrate the products to depend on SpringBoot 2.7.15, to address as many CVEs as possible.
  • PIVOT-7695 Upgrade AWS SDK to 1.12.469.
  • PIVOT-7695 Upgrade Azure dependencies to be compatible with SpringBoot versions.
  • PIVOT-7695 Upgrade CXF to 3.4.10 to address multiple CVEs.
  • PIVOT-7695 Migrate Jackson to 2.13.5 to address CVEs.
  • PIVOT-7695 Pull guava 32.1.2 to mitigate reported CVEs. Guava is not immediately used in Atoti Server but the optional Datastore Schema Printer and cloud sources.
  • PIVOT-7782 Upgraded libraries to connect to Microsoft Azure (azure-identity to 1.10.1, azure-core-http-netty to 1.13.7, azure-keyvault-keys to 4.6.5, azure-storage-blob to 12.24.0, azure-storage-blob-cryptography to 12.23.0, azure-resourcemanager to 2.30.0)

5.10.17​

2023-08-10

Added​

  • PIVOT-5860 In the MemoryAnalysisService MBean, added an operation Dump full application memory statistics exporting memory statistics for all versions.
  • PIVOT-7170 Specify in documentation that the DataExport service is experimental
  • PIVOT-7295 Add a CubeFilter translator to allow the cube filters to be added to the map of context values in REST queries.

Changed​

  • PIVOT-5860 In the MemoryAnalysisService MBean, renamed the operation Dump memory statistics to Dump memory statistics for most recent version since this operation only exports the most recent version.

Fixed​

  • PIVOT-7050 Inserting data into an initially empty joined store now correctly updates corresponding analysis hierarchies.
  • PIVOT-7349 RecordFormat#getFieldNames no longer creates a new list at each method call, thus improving its footprint.
  • PIVOT-7359 When asked to output the K top/bottom elements of the vector, the ZeroVector class no longer throws IndexArrayOutOfBoundException when K = vector.size().
  • PIVOT-7419 Fix bug in DrilldownMember causing members to be missing in the result.

5.10.16​

2023-06-22

Fixed​

  • PIVOT-7336 Copper.storeLookup could use a too recent Datastore version when a transaction would create a Datastore version but not an ActivePivot one.

5.10.15​

2023-05-11

Fixed​

  • PIVOT-5316 Fix issue with not seeing all processor cores for Windows machines with 64+ CPUs.
  • PIVOT-5978 LimitedByteArrayOutputStream now handles specific edge cases that could cause the XMLA Servlet to still generate Out of Memory Errors.
  • PIVOT-6213 TransactionQueryRunner no longer creates a transaction object on a partition when there is a transaction on the store.
  • PIVOT-7020 Correctly propagate exception to partitions of a JITPrimitiveAggregatesRetrieval.
  • PIVOT-7062 Fix data loading for JDBC drivers that do not support fetch size hints.

5.10.14​

2023-04-03

Added​

  • PIVOT-5734 Copper#constant now supports Object Vectors.
  • PIVOT-5929 Allow to add a prefix to the names generated in the content server with ActiveViamProperty.CONTENT_SERVICE_NAME_GENERATOR_PREFIX. This is done to easily differentiate between artifacts and assets created on different content server instances.
  • PIVOT-6642 Improve thread synchronization in the MDX engine.

Changed​

  • PIVOT-5693 Reduce transient memory footprint of the JIT aggregate provider. This is particularly crucial for use cases including a large number of small transactions.
  • PIVOT-6046 Now the query engine checks for interruptions during the planning phase. This is done to avoid cases where the query involving a lot of filtering blocks the session.

Fixed​

  • PIVOT-6175 Fix partitioning race conditions during query execution. The query engine now operates with immutable snapshots of partitioning state, which prevents wrong retrieval sizing and ArrayIndexOutOfBoundsException in particular.
  • PIVOT-6196 Fix an issue where we couldn't restart a DistributedMessenger when it was already paused.
  • PIVOT-6917 Distributed drillthrough queries properly function when one of the application's data nodes declares concealed hierarchies.
  • PIVOT-7089 Fix data race when discarding retrieval results in the query engine.

Security​

  • PIVOT-6740 Fix CRLF vulnerability in SpringCorsFilter.

5.10.13​

2022-12-23

Changed​

  • PIVOT-6046 Now the query engine checks for interruptions during the planning phase. This is done to avoid cases where the query involving a lot of filtering blocks the session.

Fixed​

  • PIVOT-5541 In a distributed application, a restart of a messenger will now properly free the port it was previously bound to. The freed port will be reusable, meaning that a messenger will no longer be able to use its entire dedicated range of ports.
  • PIVOT-6159 Fix partitioning race condition during query execution. The issue could happen during a query execution where concurrent store transaction trigger the creation of new partitions.
  • PIVOT-5873 Fix thread vector allocator un-consistency. Previously, a thread requesting a transient allocator could end-up with a direct memory allocator.
  • PIVOT-5925 Prevent NullPointerException for non selected query partitions result cursor.
  • PIVOT-5978 Prevent OutOfMemoryError crashes in the XMLA Servlet when attempting to send a message that does not fit in a byte array of size Integer.MAX_VALUE. RuntimeException will be thrown instead.
  • PIVOT-5994 Children of range-sharing retrievals are correctly printed as dependency in the query plan.
  • PIVOT-6133 Fix deadlock when allocating partition content.
  • PIVOT-6792 Fix race condition in distribution. The issue appeared intermittently on consecutive branch creation and deletion.

5.10.12​

2022-10-17

Fixed​

  • PIVOT-5639 Fix an issue with nested combine operations with context value dependencies in Copper.
  • PIVOT-5809 Fix ArrayIndexOutOfBounds exceptions in ActivePivotAggregatesRetriever.
  • PIVOT-5777 Fix wrong parameter order in RemoteContentService.
  • PIVOT-5825 Fix IllegalArgumentException occurring when defining a Copper bucketing hierarchy using a custom-named Copper join level as one of its underlying sources.
  • PIVOT-5867 AHibernateContentService's locking strategy is improved to lock smaller unit of works. This will prevent potential deadlocks.
  • PIVOT-5887 Arrange for a spare thread for continuous MDX updates. Previous implementation could cause live locks in case of work-stealing.

Security​

  • PIVOT-5751 Upgraded vulnerable dependencies of commons-compress, commons-io, cxf-rt-transports and pdfbox.

5.10.11​

2022-09-05

Changed​

  • PIVOT-5626 Skip initialization of previously failed post-processors
  • PIVOT-5657 Change outer reference policy to auto-detection mode for the queries executed from Datastore MBeans.
  • PIVOT-5677 Activiti dependencies upgraded from 5.22.0 to 5.23.0.
  • PIVOT-5728 Improve JIT retrievals completion performance in some applications with a high partition count.
  • PIVOT-5304 Improve the performance of the code handling the generation of the locations when analysis hierarchies are expressed.

Fixed​

  • PIVOT-5210 PIVOT-5639 Fix a bug in lead, lag, first, last and window Copper functionalities when relying on a Context Value measure.
  • PIVOT-5640 Fix an issue where unrelated store partitions could generate unnecessary transient objects. This reduces the memory footprint of small transactions.
  • PIVOT-5661 Fixed potential deadlock in JDBCTask when all JDBCAppendRunnable crashed.
  • PIVOT-5716 Fix serialization issue in QueriesRestService#mdxQueryPlan endpoints.
  • PIVOT-5713 Fix ClassCastException during the recording of distributed messages statistics.
  • PIVOT-5759 Fix Deadlock when allocating partition content.

5.10.10​

2022-07-11

Changed​

  • PIVOT-4934 Adjust Logging levels for received statements on the MDX REST endpoint to INFO.
  • PIVOT-5365 Change internal methods' signatures in EpochManager and VersionHistory.

Fixed​

  • PIVOT-5030 Disable RangeSharing for Post-Processors when an Analysis Hierarchy is expressed in the queried location.
  • PIVOT-5041 Clarify the behaviour and error message of Licensing.reload() and its underlying MBean.
  • PIVOT-5365 Fix ActivePivotVersion releasing its reference to the datastore too early. In some cases, a race condition could lead to wrong results when using a KeepLastEpochPolicy and querying an old not marked ActivePivotVersion with a JustInTime provider.
  • PIVOT-5458 Correctly propagate exception in ExternalDatastoreRetrieval.
  • PIVOT-5502 Fix cached data mutated in DatastoreService.
  • PIVOT-5508 Fix the creation of the ColumnIntegerRowMapping memory statistics for MAC.
  • PIVOT-5515 Fix ArrayIndexOutOfBoundsException in ChunkedBitmap when adding two bitmaps.
  • PIVOT-5539 Fix NullPointerException when discarding transient results from a distributed measure with no value.

5.10.9​

2022-05-06

Added​

  • PIVOT-5301 BottomCountFunction and TopCountFunction should accept a string expression as a comparator.

Changed​

  • PIVOT-5367 Renamed properties intermediateSize and transientSize in QueryResultLimit to intermediateLimit and transientLimit, respectively.

Fixed​

  • PIVOT-5333 Increase severity for full reset and pending initial discovery messages to WARNING in distributed clusters.
  • PIVOT-5342 Fix bug where we could not query joined store's lookup measure on grand total.
  • PIVOT-5395 Fix the creation of the HashMapRowMapping memory statistics for MAC.
  • PIVOT-5407 Fix Frequent Value compression not properly handling null values.
  • PIVOT-5428 Fix ArrayIndexOutOfBoundsException when using withBatchSize in parquet source.
  • PIVOT-5443 Fix SimpleCubeDescriptionBuilder not properly setting shared context values.

5.10.8​

2022-03-09

Added​

  • PIVOT-5246 Add ChunkSingleBoolean to compress Boolean chunks containing the same value for all positions.
  • PIVOT-5260 Parallelize hierarchies contribution within their dimension.

Changed​

  • PIVOT-5258 Return the original SubCubeProperties when intersecting ISubCubeProperties granting full access to the pivot.

Fixed​

  • PIVOT-5098 Fixed an IllegalStateException when performing queries containing cached aggregates for primitive measures and primitive-aggregated measures.
  • PIVOT-5188 Fix potential bug in AnalyticFunctionPostProcessor where partitioning levels were not properly set.
  • PIVOT-5218 Improve performance of the Mdx NonEmpty function.
  • PIVOT-5259 Correctly propagate custom aggregation function used for result reduction in distributed post-processors.
  • PIVOT-5308 Clarify the javadoc of Copper.StoreLookup when the specified store has nullable key fields.
  • PIVOT-5310 Fix retrieval result size figures when printing the query plan. In some cases, results were discarded before their result size computed.
  • PIVOT-5311 Fix bug where hierarchy visibility was set to false but was still visible.
  • PIVOT-5328 Fix an NullPointerException when having multiple threads discarding the same transient result at the same time.
  • PIVOT-5335 Fix aggregates cache not being invalidated in Query Cubes after transactions without any impact on the pivot's structure.

5.10.7​

2022-01-12

Added​

  • PIVOT-5170 Log query results size by default when printing the query plan.

Fixed​

  • PIVOT-4901 Prevent aggregates computation (i.e. GetAggregatesQuery) when mdx CELL PROPERTIES is equal to CELL_ORDINAL.
  • PIVOT-5095 Fix real-time update on query with epoch hierarchy. Previously, real-time updates on such queries resulted in branch disappearing from the updated result. See Real-Time What-If Queries section.
  • PIVOT-5162 Fix potential NPE in PrimitiveAnalysisAggregationRetrieval when joining results.
  • PIVOT-5195 Fix bug in the revert to version. Some records were not reverted in some edge cases.

5.10.6​

2021-11-19

Added​

  • PIVOT-5037 The Copper tester now supports hasValues() in every case and a new method withDelta() to set the allowed delta between the values given by the user in hasValues() and the actual values returned by the measures.
  • PIVOT-5118 Add a toString() to CloudCsvDataProvider() for better error message.

Deprecated​

  • PIVOT-5117 Deprecate TransactionWrapper. It will be removed in ActivePivot 5.11.0.

Fixed​

  • PIVOT-4800 Enhance query plan timing reports. This particularly fixes two man issues:
    • JITPrimitiveAggregatesRetrieval Partitioned retrievals for non-existing partitions used to display the largest execution time of their siblings. Now, only partitioned retrievals that performed some work at least and whose result is empty will be reported in the timing section, along with the other retrievals with result
    • In some cases, for a group of related retrievals (siblings), timing were not printed if one of the sibling has no result. This edge case has been fixed
  • PIVOT-5038 Fixed errors in usage counting caused by decrementing the usage too many times.
  • PIVOT-5050 Fix distributed post-processors initialization. This issue happened when chaining at least three post-processors where the bottom one in the chain is not initialized and caused missing dependencies propagation in some cases.
  • PIVOT-5074 Fix SubCubeProperties#removeRestriction(dimension, hierarchy). The issue happened in distributed polymorphic setup where the same dimension from two data nodes includes different hierarchies and one of the dimension hierarchies cardinality is greater than 1.
  • PIVOT-5097 Fix NullPointerException when registering a GetAggregatesQuery build using a location Collections not supporting null elements.
  • PIVOT-5115 Fix IllegalArgumentexception during query plan logging when executionTimingPrint is enabled .
  • PIVOT-5119 Fix rollback process of store transactions.
  • PIVOT-5128 Fix a memory leak that could happen upon query failure/cancellation if an AggregatesCache was used.

5.10.5​

2021-10-19

Added​

  • PIVOT-4941 Added two methods in IPointValueDao to permanently delete Parameters in ActiveMonitor.
  • PIVOT-4973 Added a warning to IReadableDatastore#getMostRecentVersion, stating that #getHead should be used instead.
  • PIVOT-4913 The JDBC Source will now log the exceptions that make its workers exit the parallel processing of records.
  • PIVOT-5069 Add the currently written record in case of type error in IRecordFormat#newRecord().

Changed​

  • PIVOT-5024 Use ChunkedIntegerArray instead of IntegerArray in AMultiVersionUniqueRecordIndex so that we can now put more than 300M records per partition.
  • PIVOT-5048 Upgrade Junit version to 5.8.

Removed​

  • PIVOT-5011 Removed the provided strings from exception messages in RestContentServer and JsonQueryService.
  • PIVOT-5084 Removed an invalid warning that was triggered when building Copper Hierarchies backed by a store. Said warning was stating these hierarchies shouldn't be factless.

Fixed​

  • PIVOT-4951 Fix an edge case issue with IChunkInteger sparse compression.
  • PIVOT-4972 Fix branches not visible sometimes with continuous queries. The ISecurityDetails was not always propagated when registering queries.
  • PIVOT-4998 Reduce time spent at startup optimizing Copper measures
  • PIVOT-5010 Properly set the level type of a slicing Copper hierarchy.
  • PIVOT-5016 Fix ArrayIndexOutOfBoundsException in the SetExpander of the Mdx engine.
  • PIVOT-5038 Fix measure dependencies being improperly attributed in the PlanningGraph.
  • PIVOT-5062 Fix NullPointerException in PointListRetrievalResult in the case of a polymorphic distributed setup.
  • PIVOT-5068 Fix NullPointerException in the AggregatesResultCopier used by postprocessors.
  • PIVOT-5156 Fixed missing query plan retrievals. The issue happened when underlying retrievals with different partitioning were merged. Note that this does not affect query results nor the query plan export feature, but only the query plan printing.

5.10.4​

2021-09-03

Added​

  • PIVOT-4733 Allow to override measure aggregator xmla property with CopperMeasure#withXmlaAggregator(int).
  • PIVOT-4824 Include epochId of the max and latest continuous query statistics.

Fixed​

  • PIVOT-4866 Fix NullPointerException in FilterTranslator. The issue could happen when translating a wildcard, e.g. null, to the data cube.
  • PIVOT-4881 Fix an issue where the resizing of IntegerArray and LongArray generated non-transient chunks regardless of its isTransient parameter.
  • PIVOT-4923 Fix hiding subtotals issue that could not hide all the measures.
  • PIVOT-4935 Fix NPE when accessing an uninitialized datastore via MBean.
  • PIVOT-4964 Fix an ArrayIndexOutOfBoundException happening when compressing chunks with freed rows as FrequentValueChunk.
  • PIVOT-4944 Fix the cleaning method of the prefetch scope location when executing more than one aggregation procedure in a single PrimitiveAnalysisAggregationRetrieval.
  • PIVOT-4946 Fix an issue with Copper storeLookup that could fail to find its underlying measures, in the case where these measures were previously re-optimized.
  • PIVOT-4949 Fix thread safety for SharedDelegateDictionary#mapFromCell.
  • PIVOT-4956 Fix GenericVectorBinding#applyAs.. operations for non-commutative operators.
  • PIVOT-4958 Fix the update of the ContentServer entries for branches containing characters that are illegal for FileSystem-specific paths.
  • PIVOT-4979 Fix missing fields issue in the analysis aggregation procedures when a cube filter is done on a Copper join.
  • PIVOT-4992 Fix dependency issue in content-server-spring.
  • PIVOT-5137 Fix LevelIdentifier.fromDescription and HierachyIdentifier.fromDescription. They did not handle correctly the escaping of @.

5.10.3​

2021-06-18

Added​

  • PIVOT-4804 Alert the user when creating measures names with commas.
  • PIVOT-4806 License path is now printed in the log.
  • PIVOT-4820 Add constructor to KeepLastEpochPolicy using Duration.

Changed​

  • PIVOT-4498 The ActiveViamPropertyFromSpringConfig.resolve() bean is now named ActiveViamPropertyFromSpringConfig.SPRING_PROPERTIES_RESOLVER.
  • PIVOT-4781 StreamSnapshotService#snapShotStreams(String) is now compatible with StreamSnapshotService#loadStreams(String). Now, both operation ignore STOPPED streams. Use StreamSnapshotService#snapShotAllStreams(String) to snapshot all streams including the stopped ones.
  • PIVOT-4789 Provide possible solutions in error message when we failed to initialize the SaferNative class in LinuxPlatform.
  • PIVOT-4802 Reduce the memory consumption of ModifiedLocation, primarily used in cumulated aggregates and in shifted measures.
  • PIVOT-4808 Report error if the ActivePivotSchema fails to attach.
  • PIVOT-4813 Update ActiveMonitor REST API documentation to correctly illustrate how a new monitor can be added.

Deprecated​

  • PIVOT-4847 Deprecate methods on the IEpochManager that will be removed in 5.11.0. One should rather use the same methods on IVersionHistory.

Fixed​

  • PIVOT-4735 Fix continuous handler registration issue. The issue happened in distribution when some underlying measures have not yet appeared thus causing a duplication in handler key of the parent measure.
  • PIVOT-4780 Fix unnecessary query plan logging when queryPlanSummary is enabled.
  • PIVOT-4807 Fix an issue where using continuous query handler inheritance could lead to the failure of the publication of measures.
  • PIVOT-4811 Fix ClassCastException in SingleValueFunction when working with arrays.
  • PIVOT-4814 Fix an issue where exporting the memory statistic of a chunk containing non-direct vector blocks would throw a NPE.
  • PIVOT-4816 Fix an issue occurring when merging two AggregateStores. This caused the merged result to include null values instead of the correct value for some measures.
  • PIVOT-4818 Fix an issue leading to incorrect transitions between IntegerArrayVersion.
  • PIVOT-4819 Fix UnsupportedOperationException in ActivePivotStatistics. The issue happened when computing transaction fact statistics on an incompatible type of transaction.
  • PIVOT-4862 Fix an export error in the IMemoryStatistic generation for TombStone classes.

5.10.2​

2021-04-30

Added​

  • PIVOT-4486 Add the possibility to apply query cube filter on the prefetch of Copper analytic functions using the API CopperAnalyticMeasure#withFilter(AnalyticFilter).
  • PIVOT-4713 Add the method IDistributedHierarchyManager#waitForAllMessages allowing to wait for all the recorded messages to be processed.

Changed​

  • PIVOT-4300 Post-processor dependency graph cycle detection is now enabled by default.
  • PIVOT-4780 Change MBeans in QueryPlanMonitoring to print execution plan and timing full or as summary, e.g. QueryPlanMonitoring#enableExecutionTimingPrintFull() or QueryPlanMonitoring#enableExecutionTimingPrintAsSummary().

Fixed​

  • APS-13016 Better handle network errors which were causing issues with license checks.
  • PIVOT-4717 Fix UDAF error that could occur when totally emptying the underlying store.
  • PIVOT-4725 Some datastore versions were not unmarked after stopping the cubes.
  • PIVOT-4728 Measure names with several parentheses were not correctly supported by the Formula PostProcessor.
  • PIVOT-4730 Prevent Real-Time statistics errors from crashing continuous queries update. The issue happened when statistics computations tried to fetch retrievals that had been dismissed by the query engine (usually because of a non-existent location).
  • PIVOT-4739 Fix publishing of WindowReduceMeasure (description equality).
  • PIVOT-4749 Clarify the documentation of the cube/export/mdx/download REST service endpoint.
  • PIVOT-4760 Fix an issue where IStoreToParquetMapping would map fields from the datastore to the wrong fields in the parquet schema.
  • PIVOT-4766 Fix Frequent Chunk Compression bug where nested FrequentNullDoubleChunk could be created.
  • PIVOT-4771 Fix NPE cause by CopperPostProcessor#withInheritedContinuousQueryHandlers. The issue happened while inheriting the handlers of a post-processor that hasn't any.
  • PIVOT-4772 Fix a query issue where wildcards on slicing analysis hierarchies based on isolated store fields would conflict with Copper bucketing hierarchies.
  • PIVOT-4775 Fix an issue with AAdvancedPostProcessor performing an eager modification of the prefetched cube filter.
  • PIVOT-4775 Fix an issue with the query engine where some retrievals handing analysis hierarchies, were wrongly declaring their dependencies.
  • PIVOT-4791 All datastore versions are now synchronously unmarked when stopping the pivot.

5.10.1​

2021-03-26

Added​

  • PIVOT-4473 Add loggers for CREATE, UPDATE and DROP MDX statements
  • PIVOT-4638 Add the maskAtOrBelow() method to the CopperMeasureToAggregateAbove interface, allowing to define levels below which a measure will not return aggregated values when queried. The behavior is the complementary of the doNotAggregateAbove() method.
  • PIVOT-4702 Add support for DATEPART function in MDX with the simple signature DatePart(interval, date).

Changed​

  • PIVOT-4705 Make Records#getGlobalDefaultValue use int instead of ILiteralType.
  • PIVOT-4707 PointIndexedRecordBlock no longer implements IChunkSet.
  • PIVOT-4707 Renamed ImmutableRecordList in ImmutableRecordBlock to match the pattern established in sibling classes.

Deprecated​

  • PIVOT-4705 Deprecate SchemaHelper#undictionarize in favour of Records#undictionarize.
  • PIVOT-4724 Deprecate com.qfs.pivot.impl.ActivePivotSchemaExtendedPlugin in favour of com.quartetfs.biz.pivot.impl.ActivePivotSchemaExtendedPlugin located in the conventional package.

Removed​

  • PIVOT-4707 Removed PointIndexedRecordBlock destroyer methods.
  • PIVOT-4707 Removed IAggregateSource interface.
  • PIVOT-4707 Removed a constructor in AWritableFixedSizeRecordBlock.
  • PIVOT-4707 Removed transferFrom(IArrayReader source, int sourceIndex, IWritableByteRecord record) from class ARecordFieldHandler.
  • PIVOT-4718 Drop legacy methods from IByteRecordFormat.

Fixed​

  • PIVOT-3790 Fix the partitioning levels of the legacy post-processor NeighbourValuePostProcessor.
  • PIVOT-4177 Prevent two Store Lookups from colliding in the query cache if they ask for different fields on the same store.
  • PIVOT-4344 Fix SingleValueFunction on Arrays type.
  • PIVOT-4364 Fix Real-Time issues with Copper. This includes the following corrections:
    • In addition to the CommitIsolatedStoreHandler handler, join measures now include the StoreMeasureHandler in their handlers' list
    • Copper#newPostProcessor(..) allows now to propagate the underlying measures handlers using CopperPostProcessor#withInheritedContinuousQueryHandlers()
    • Delegate continuous handler initialization to post-processors
  • PIVOT-4452 Fix an issue where exported memory statistics could not be read by MAC.
  • PIVOT-4690 Restore allocators that where removed in the previous release: HeapBufferChunkAllocator, DirectBufferChunkAllocator and DirectChunkAllocator.
  • PIVOT-4691 Fix possible casting exception when using CopperMeasure.withType(ILiteralType.BOOLEAN) with an aggregation function returning Boolean.
  • PIVOT-4710 Copper.storeLookup(...) (i.e. StoreLookupMeasure) now handles real time updates.
  • PIVOT-4719 Fix lazy initialization and double check idiom in RecordFormat#getFieldIndex(String).

Security​

  • APS-12594 BREAKING The XML parsers used throughout ActivePivot (JAXB, XMLA Servlet...) are now given a restrictive configuration to better prevent XXE injection attacks:
    • Automatic DDoS prevention. This may limit XML constructs, such as deep recursive XML elements (XML entities calling more XML entities, and so on).
    • Disables !DOCTYPE annotation to prevent XXE injections through this annotation.
    • Disables the XInclude annotation to prevent construction of sub-documents that could be retrieved from a distant server.
    • Disables the use of external Document Type Definitions, for the same reason.

5.10.0​

2021-02-26

Added​

  • APS-12558 Add new MdxContext parameters that can override the visibility of hierarchies and dimensions, forcing them to be shown or hidden regardless of the cube's configuration.
  • APS-12866 Rename NotificationMember#AWAIT_NOTIFICATIONS to IDataClusterDefinition#AWAIT_NOTIFICATIONS_TIME. The property can now be declared in data cluster definitions using ICanBuildDataClusterDescriptionBuilder#withProperty.
  • APS-12879 Expressions like Copper.sum("pnl").publish(context) have to be replaced with Copper.sum("pnl").withName("pnl.SUM").publish(context).
  • APS-12908 Add a selectDistinct() method to DatastoreQueryHelper that accepts a condition.
  • PIVOT-3352 Add fencing on AUniqueRecordIndexTransaction methods.
  • PIVOT-3978 Changed when the 'force discard' was called in KeepLastEpochPolicy.
  • PIVOT-3978 IPrimaryRecordIndexVersion#getKeyFields() moved to IPrimaryRecordIndex#getFields.
  • PIVOT-3978 Rename ARemoveUnknowKeyListener to ARemoveUnknownKeyListener to fix typo.
  • PIVOT-4177 Add the possibility to define a Store Lookup in Copper. See Copper.storeLookup(storeName).
  • PIVOT-4180 Properly shutdown DataNodes in a distributed cluster. When a data node is being shutdown, it will send a GoodbyeMessage to the query node which will immediately trigger an asynchronous task responsible of removing all of its subsequent contributions.
  • PIVOT-4220 Add on-the-fly discard of transient results during query execution.
  • PIVOT-4280 Support copper join on any level. It was not possible to do a join on a bucketing level for instance.
  • PIVOT-4356 Join Hierarchies with multiple levels can now be published with Copper, using the unified API Copper.newHierarchy(...).
  • PIVOT-4357 User-defined aggregate functions can now perform dynamic aggregation on measures.
  • PIVOT-4409 Allow to define an Analysis hierarchy on the base store (withStoreName method)
  • PIVOT-4417 Add a new compression for chunks, storing only values different from a frequent value.
  • PIVOT-4438 Update MDX engine error messages to use hierarchy unique names instead of simple names.
  • PIVOT-4476 Add the possibility to specify the fields of the records given to the StoreTransactionBuilder in Copper testing with the method withFields().
  • PIVOT-4480 Add a JMX operation in the store monitoring MBean that produces a general chunk memory usage report. MAC is still recommended for detailed memory analysis.
  • PIVOT-4490 Add health events to the JDBC source notifying the success or the failure of a JDBC source fetch.
  • PIVOT-4491 Added support for primitive aggregated measures in the Aggregates Cache.
  • PIVOT-4546 All licenses must have an end date.
  • PIVOT-4549 Add truncate method to ITransactionManager for deleting all records from a store.
  • PIVOT-4566 Add a fluent builder withSharedContextValues(values...) allowing the definition of multiple shared context values at once avoiding repetitive calls to withSharedContextValue(value).
  • PIVOT-4569 Add Copper.timestamp() to access timestamp measure.
  • PIVOT-4589 Add support in the parquet source for filtering parsed files using a java.util.regex.Pattern, through IParquetParserBuilder.withPattern(java.util.regex.Pattern). These health events can be handled using the interface IJdbcSourceHealthEventHandler.
  • PIVOT-4599 Add IClearableBitmap interface for non-incremental add and clear operations.
  • PIVOT-4600 Add MultiVersionLongArray.

Changed​

  • APS-12788 AxisHierarchy now reports that a field used to create a level should be dictionarized, instead of stating that this operation is only available in unit-tests.
  • APS-12918 Add more parser-policy-related interruption checks to the CSV source's ParserTasks. This can considerably improve performance when using the sampling parser policy.
  • APS-12932 Bitmap operations now catch CancellationException early to prevent logging bitmaps that are valid.
  • APS-12943 Change the default behavior of QueriesResultLimit context value from #withoutLimit() to #defaultLimit(). Now, we also allow -1 value for a limit to represent no-limit on QueriesResultLimit context value.
  • APS-12946 The DistributedEpochManager added in PIVOT-4501 is visible in the Manager MBean under the corresponding distributed cube's MBean.
  • APS-12975 Reduce DiscardOnGcTask timeout duration from one hour to one minute. In very rare cases, this will let a version be discarded more rapidly after being released.
  • PIVOT-3914 Change AAdvancedPostProcessor.computeOutputType() to fall back to the return type of its evaluator, if it has one.
  • PIVOT-4191 Rename methods in QueriesResultLimit.
  • PIVOT-4356 Bucketing Hierarchies must now be published using the unified API Copper.newHierarchy(...).fromValues(...).
  • PIVOT-4375 The Azure Cloud Source has been migrated from Azure Blob Storage SDK v8 to v12. Please refer to the migration notes to see what has changed.
  • PIVOT-4402 Remove static getters from JoinMeasureDescription.
  • PIVOT-4410 BREAKING: Changes were made on the hierarchy builder. hierarchyBuilder.factless().withStoreName("myStore") was replaced by hierarchyBuilder.fromStore("myStore").
  • PIVOT-4464 Remove ActiveMonitor from the sandbox projects.
  • PIVOT-4501 Enhance version management in a distributed setup. Versions are now handled in the Query Cube using a dedicated DistributedEpochManager that keeps track of versions history and trigger discard tasks on GC.
  • PIVOT-4502 Optimize number of operations in chunk scan methods by computing '~epoch' once before iterating over the rows.
  • PIVOT-4509 Register standard Drillthrough calculators through an extended plugin.
  • PIVOT-4522 Change the signature to create a UDAF to allow allocating vectors off-heap and lazy cloning. One should now call IWritableBuffer.allocateVector instead of IVector.cloneOnHeap(), and IWritableBuffer.readWritableVector(int) instead of IWritableArray.readVector(int).
  • PIVOT-4558 Change the behavior of IHierarchy#isEmpty(). Now, the method checks whether the corresponding hierarchy is empty or not instead of checking for leaf members. This enhances the performance of queries with high cardinality hierarchies.
  • PIVOT-4564 Levels from hidden dimensions and hierarchies are now omitted in XMLA discoveries. This caused an incompatibility with some XMLA clients such as ADOMD.NET.
  • PIVOT-4579 The severity of the log message notifying that a property is missing has been changed from WARNING to INFO.
  • PIVOT-4599 Rename FastLogical to FastLogicalQFSBitmap.
  • PIVOT-4608 Lift limitation on Datastore queries with respect to the combination of indexes. Previously, the Datastore could combine up to 3 indexes when looking for range patterns like a in (...) and b in (...). Now, the maximal limit is configured by ActiveViamProperty.MAX_RANGE_OPERANDS_WITH_MULTIPLE_VALUES and set by default to 10.
  • PIVOT-4620 Fix issue with Hibernate where a high number of input parameters causes server exceptions.
  • PIVOT-4622 Fix ActiveMonitor PointValue commit logic. This adds a consistency check when committing an update on an existing point value. Now, we make sure that the "from" point value and its active value in the database have the same key. Otherwise, the commit is not allowed and an exception is thrown.
  • PIVOT-4626 Fix a bug in the parquet source where an invalid index exception would be thrown if a parquet vector field was not mapped to any field in the fed datastore.

Deprecated​

  • PIVOT-4244 Our web services exposed with SOAP are deprecated. Your are encourage to start using the REST services.
  • PIVOT-4244 XML representation of the cube configuration and our DTOs.
  • PIVOT-4679 Ability to filter the facts sent to a cube (ActivePivotInstanceDescription.setFilterDescription).

Removed​

  • APS-12686 Remove JGroupsMessenger.
  • PIVOT-3978 Remove the Transaction Log.
  • PIVOT-3978 Remove legacy TransactionWrapper.
  • PIVOT-4428 Remove following methods:
    • IChunk.findRowsInTransactionEqualTo(Object, IIntIterator, int, IRowMapping, int, IBitmap)
    • IChunkSet.findRowsInTransactionEqualTo(int, Object, IIntIterator, int, int, IBitmap)
    • IChunk.findRowsInTransactionInSet(Set, IIntIterator, int, IRowMapping, int, IBitmap)
    • IChunkSet.findRowsInTransactionInSet(int, Set, IIntIterator, int, int, IBitmap) as duplicates of their counterpart outside of a transaction (findRowsEqualTo, ...)
  • PIVOT-4560 Remove duplicate javax.xml.stream dependency in serialization-xstream (this dependency is included in the JRE).
  • PIVOT-4615 Remove heavy-to-compute logging in ImpactComputationTask.
  • PIVOT-4669 Remove deprecated MapAggregatesProvider.

Fixed​

  • APS-12090 Fix issue with branch permission manager preventing to see new branches on real time Mdx queries.
  • APS-12909 Fix a memory leak within the catch up listener's mechanism.
  • APS-12911 Fix postprocessors auto-partitioning in the the planning phase.
  • APS-12912 Fix a bug where spring boot would occasionally fail with message BeanCurrentlyInCreationException: Error creating bean with name 'resourceMapping': Requested bean is currently in creation. when importing multiple configuration classes inheriting ASpringResourceServerConfig.
  • APS-12915 Correctly process Cube Filters in a distributed application.
  • APS-12931 Fix an erroneous check that would throw when trying to define user-defined aggregate functions in partial aggregate providers.
  • APS-12937 The FullRefreshEvent will now include the set of all impacted measures. This allows to distinguish between measures not impacted by the real-time event and measures with null update.
  • APS-12962 Fix bug preventing measure publication when using withType(...) on a calculated measure, and reusing this measure in another calculated measure.
  • APS-12965 Fix Copper Join producing incorrect aggregation results when using a Join Measure based on the key fields of the joined Store.
  • APS-12966 Fix an issue where formatters would not be taken into account for UDAF measures.
  • APS-12985 Correct overflow error in dictionaries occurring when inserted more than 2 ^ 30 values in a field dictionary.
  • PIVOT-4241 Fix the reserved memory tracking of the SlabMemoryAllocator when the cache is evicted.
  • PIVOT-4243 Fix the reserved memory tracking of the INativeMemoryAllocator classes to track the reserved memory and not only the used memory. The tracking is done at the lowest level for more accuracy.
  • PIVOT-4370 The Parquet Source now correctly handles empty strings when writing into a store field that should not support empty strings, replacing them with the store field's default value.
  • PIVOT-4388 Changed the return type of the Store Builder method withVectorField when chained after a dictionarized or indexed vector field. It gives access to a much fuller API to configure this second vector field, or finish configuring the store with methods like withoutKey().
  • PIVOT-4427 Fix ChunkedBitmap#equals implementation.
  • PIVOT-4436 Fix ClassCastException occurring when a join measure was defined on a dictionarized field.
  • PIVOT-4453 ActivePivot now correctly unregister all its datastore continuous selections when stopped.
  • PIVOT-4455 Correctly compute vector average, ignoring null vectors instead of counting them as actual values.
  • PIVOT-4456 PIVOT-4457 Correct Datastore conditions testing for equality or greater-than/less-than on nullable fields. Previously, this could:
    • throw an exception for a non-dictionarized field with long series of null values;
    • return incorrect rows from the store when looking for the primitive value 0 on a non-dictionarized field;
    • return rows with null for greater-than/less-than conditions.
  • PIVOT-4460 Fix a bug where MemoryAnalysisService would not produce report files for query cubes in distributed application.
  • PIVOT-4461 Fix NullPointerException during Datastore queries looking for null in store fields after compression. This only happened in the case of offset compression, for int and long values or dictionarized fields.
  • PIVOT-4463 Fix IllegalStateException (No free or removed slots available) in MergingStorePartitionUpdate.
  • PIVOT-4500 Fix filter translation edge case in distribution. This caused AND filtering operation on the same hierarchy to be wrongfully translated into an OR operation.
  • PIVOT-4503 Fix a bug where the batched parquet parser could throw an end-of-file exception when reading a vector column.
  • PIVOT-4503 Fix a bug where the batched parquet parser would not properly read row group and page metadata, potentially throwing an end-of-file exception.
  • PIVOT-4507 Fix issue with mdx drillthrough query involving a member whose name contains brackets.
  • PIVOT-4516 UDAF used in a aggregate provider always stored its vectors on heap.
  • PIVOT-4517 Fix a bug where an ArrayIndexOutOfBoundsException would be thrown when generating the memory statistic of an empty target store referenced by a non-empty owner store.
  • PIVOT-4525 CopperMeasure#parentValueOn(CopperHierarchy) now properly handles an analysis hierarchy being passed as an argument.
  • PIVOT-4527 Fix XMLA discovery of UDAF measures.
  • PIVOT-4532 Fix the computation of retrieval results size in the query plan. The total result size for partitioned retrievals were not accumulated together correctly and an empty result for a partition was causing a NPE.
  • PIVOT-4533 Fix a bug in Copper where an UnsupportedOperationException would be thrown when subtracting a float, long, or int vector to a null vector aggregate.
  • PIVOT-4534 Prevent PrimitiveAnalysisAggregationRetrieval from inheriting not required measures from its underlying retrieval. This prevents mismatch binding when a parent PrimitiveAnalysisAggregationRetrieval depends on a child PrimitiveAnalysisAggregationRetrieval including additional join measures. It also improves the query times and memory usage for such cases.
  • PIVOT-4542 Improve error message when QueriesResultLimits are exceeded.
  • PIVOT-4550 Fix a Copper bug where measures could not be published.
  • PIVOT-4563 Fix a bug where the Left and Right MDX functions would throw a NullPointerException for null values.
  • PIVOT-4565 Fix a bug where enforcing a parallel computation of a Copper join could make a query timeout.
  • PIVOT-4572 Fix ClassCastException during the execution of MDX queries relying on previously cached aggregates. The issue is due to a bad partitioning of a post-processor depending on a cached node and a non-partitioned post-processor node. Now, cached nodes forbid all partitioning levels and bubble them up to their respective parents.
  • PIVOT-4586 Fix NullPointerException in DriltthoughResultDTO.
  • PIVOT-4590 Fix ArrayIndexOutOfBoundsException occuring when merging two retrieval results. Particularly, the issue was due to the fact that some post-processors ignore null values and don't write them into the final result. As a consequence, the capacity of the underlying post-processor column won't be aligned with the retrievals points, causing such exception when merging the actual result with another one.
  • PIVOT-4592 Fix Copper errors when reusing code to build complex measures.
  • PIVOT-4593 Fix a bug where the batched parquet parser would read incorrect values for all subsequent vectors if a column contained a null or empty vector.
  • PIVOT-4633 Fix ClassCastException in DataCubeAdapter.
  • PIVOT-4633 BREAKING: The method Location#getArrayLocation() has been removed. Please use the method ILocation#getCoordinate(hierarchyIdx, levelIndex) instead.
  • PIVOT-4649 Fix issue with DrillthroughQuery headers incorrectly returning the type of a column.
  • PIVOT-4671 The Query cube will now always reject messages and contributions from unknown applications.
  • PIVOT-4678 Fix a bug where similar Copper measures using different constants were confused with each other.

5.9.3​

2020-09-29

Added​

  • PIVOT-4191 Limit intermediate and transient retrievals result size within a GetAggregatesQuery using QueriesResultLimit. Intermediate result size referes to the size of a single retrieval whereas the transient result size is the result size cross retrievals belonging to a single GetAggregatesQuery. Note that this limit is set with respect to the number of points i.e. retrieval locations.
  • PIVOT-4341 Add IArrayReader.readVector method.
  • PIVOT-4387 Add a validation step for StoreToParquetMappings before parsing a parquet file.

Changed​

  • APS-12854 Important: Remove Where operation now throws if the given condition reads in or through a store locked in the transaction (a store which will be written into concurrently).
  • APS-12878 Improve the optimization of copper measures depending on other copper measures. In some cases the number of intermediary results will be reduced: this will reduce the memory consumed by queries.
  • PIVOT-4391 Memory statistic exports do not include chunks of records for future versions anymore.

Deprecated​

  • APS-12854 deprecated SchemaHelper methods followReferences and getStoresInReferencesPath. The class now has methods with specific implementations whether the path given ends with a field name or not.

Fixed​

  • APS-12731 Fix QueriesTimeLimit heuristic overflow in the MDX engine. This led to cases where the actual query was still running, yet because of the overflow, the query engine was continuously timing out, creating erroneous TimeoutExceptions that never bubbled up because the query didn't time out.
  • APS-12861 Fix the planning of ExternalDatastoreRetrieval to remove duplicate retrieval executions.
  • APS-12863 Fix UnsupportedOperationException occurring when applying branch security in BranchUtils.grantBranch. It happened when the SubCube Properties of the current user were an intersection.
  • APS-12871 Fix an edge-case issue with the RemoteContentServer that would throw if two paths in the Content Server were equal up to a / in the first, and a special character in the second.
  • APS-12873 Fix a bug in LocationBuilder#setCoordinate(). The method was skiping levels during path iteration.
  • APS-12876 Fix an NPE occurring with history aggregation functions in the case of a Copper Inner Join with no matching facts in joined store.
  • APS-12885 Fix an edge-case where datastore query yields additional unexpected results. The issue happens when specifying a condition on a reference field and the target store includes a ValuePartitioning on that field.
  • APS-12889 Fix a wrong error message displayed when attempting to perform arithmetic operations between measures with incompatible types in Copper.
  • PIVOT-4383 Fix faulty behavior in MemoryAnalysisService regarding the exporting of multi-versioned datastores.
  • APS-12891 Fix potential ClassCastException when using the square sum or multiply aggregation function on int values.
  • PIVOT-4382 Fix a bug where MemoryAnalysisService's Dump memory statistics for some epochs bean operation would produce statistics that were incompatible with MAC.
  • PIVOT-4384 Fix a bug where calling AConcreteReferenceVersion::getMemoryStatistic() on a reference version whose owner or target partition was dropped in a subsequent epoch would produce a NullPointerException.
  • PIVOT-4392 Improve the performance of the batched parquet parser on binary fields.
  • PIVOT-4400 Support compiling ActivePivot projects with recent JDKs.
  • PIVOT-4401 Fix a bug in the completion of JIT retrievals that could lead to threads not being properly stopped when encountering an exception in underlying retrieval operations.

5.9.2​

2020-08-14

Added​

  • APS-12708 Add a new method CopperMeasure.marginalTotalOn(hierarchies...) to the Copper API to create marginal totals over an existing measure. Added CopperMeasure.totalOn(hierarchies...) and CopperMeasure.grandTotal().
  • APS-12724 Add a new ActiveViamProperty, set through -Dactiveviam.contentServer.restrictedListing=<boolean>, which will change the permission behavior of the Content Server. It defaults to false. When set to true, a user will not be able to list a file without having read access to the said file, even if he has read access to the parent directory.
  • PIVOT-4304 Add indexed aggregate providers support for user-defined aggregate functions.
  • PIVOT-4342 Add automatic support for String vectors in the parquet source.
  • PIVOT-4355 Add IBroadcastMessage#getMessageClass() and IBroadcastResult#getAnswerClass() methods allowing to retrieve respectively the class representing a message type and an answer type.
  • PIVOT-4324 Add IParquetParserBuilder to build parquet source parsers. Add a new parquet parser implementation that reads records in batches, leading to significant performance improvements.
  • PIVOT-4354 Add JUnit5 extensions equivalent of test rules JUnitCubeTesterBuilder, DistributedRule, ResourceRule and ActiveViamPropertyRule. Migrate ATestFencing to be JUnit 5-compliant.
  • DOC-273 Add a new property activeviam.i18nconfig.file.encoding, used by LocalI18nConfig to control which encoding to decode internationalization files with.

Changed​

  • APS-12730 Improve query engine to keep the maximal partitioning for operations on primitive measures.
  • PIVOT-4233 Performing a filtering operation with an OR condition is now only possible in Copper if both operands are on the same level.
  • PIVOT-4324 Parquet source parsers should now be constructed using a IParquetParserBuilder.
  • PIVOT-4371 Improve the performance of BasicPostProcessor evaluation.
  • DOC-273 Simplify LocalI18nConfig to avoid needing to use the content service, and instead use local resource files directly for internationalization.

Deprecated​

  • APS-12708 Deprecating Copper.total(measure, hierarchy) in favor of CopperMeasure.totalOn(hierarchy).
  • PIVOT-4324 Deprecating ParquetParser public constructors in favor of IParquetParserBuilder.
  • PIVOT-4354 Deprecating JUnitCubeTesterBuilder, DistributedRule, ResourceRule and ActiveViamPropertyRule. For removal in 5.10.
  • PIVOT-4367 Deprecating com.qfs.func functional interfaces as well as com.qfs.func.IEither, com.qfs.func.impl.Either, com.qfs.func.IOptionalEither, com.qfs.func.impl.OptionalEither and com.quartetfs.fwk.util.FloatBinaryOperator in favor of their new counterparts in package com.activeviam.util.function.

Fixed​

  • APS-12709 Fix an issue with Copper's bucketing procedure producing wrong results when performed on top of a Join operation in a many-to-many scenario.
  • APS-12710 Speed up the building of Copper-based bucketing Analysis Hierarchies depending on multiple levels.
  • APS-12746 Fix an issue that could lead to a wrong order of dependencies of Copper measures.
  • APS-12769 Prevent NPE in AAdvancedPostProcessor#retrieveNamedPrefetchAggregatesWithAnalysisLevels
  • APS-12777 Allow StreamingService to garbage collect and dispose of already stopped streams.
  • APS-12782 Fix a bug where the default measure defined in the pivot description was ignored.
  • APS-12796 Fix query returning incomplete results. This could happen when there were IN conditions on indexed fields used in the partitioning.
  • APS-12802 Fix a bug where Copper measures based on analytic functions would return wrong results when the hierarchy prior to the hierarchy used in the Window was slicing and queried with a range coordinate.
  • APS-12805 The Mdx engine now supports VBA functions used with the VBA! prefix.
  • APS-12823 Fix Sandbox CORS configuration.
  • PIVOT-4233 Performing a filtering operation with conditions on multiple levels of the same hierarchy now performs an AND operation for each filtered level.
  • PIVOT-4312 Fix IllegalArgumentException happening when querying a Join measure from a isolated store joined with Copper without mapping on a location where join-related Analysis hierarchies are not expressed.
  • PIVOT-4326 Fix Copper Window functions interaction with the QueryCache.
  • PIVOT-4363 Fix a memory leak caused by User-Defined Aggregate Functions when running queries.

5.9.1​

2020-06-25

Added​

  • APS-12715 IPostProcessedRetrievalResultWriter.aggregateAndContribute(IAggregationFunction, int, List<? extends IIterableAggregatesRetrievalResult>). The extra argument provides the type of data consumed by the aggregation function. While this type is often the same as the measure type, functions converting longs to vectors of longs break this assumption.
  • APS-12740 Add support for a flexible ISecurityWebSocketHandlerFactory allowing to override the default SecurityContextWebSocketHandle handler.

Changed​

  • APS-12753 Throw a clearer intermediate exception if an exception occurs during row parsing, when feeding rows to a datastore through DatastoreService.
  • APS-12757 Reducing network call of ICloudEntityPath#getLastModified() for Azure implementation.

Deprecated​

  • APS-12715 Deprecating IPostProcessedRetrievalResultWriter.aggregateAndContribute(IAggregationFunction, List<? extends IIterableAggregatesRetrievalResult>) to replace it with IPostProcessedRetrievalResultWriter.aggregateAndContribute(IAggregationFunction, int, List<? extends IIterableAggregatesRetrievalResult>).
  • APS-12718 Deprecating DrillthroughExecutor.doExecuteHeaders(Collection<ILocation> locations, Collection<String> measures, boolean isDrillthroughContinuousQuery, boolean log) in favor of its overloaded sibling without the log boolean. For removal in 5.10.0.

Fixed​

  • APS-12080 Fix Content Service MBean registration.
  • APS-12588 Fix a bug with dropped partitions. The drop was not correctly reverted when the transaction was rolled back. This could cause multiple exceptions in the Bitmap aggregate provider.
  • APS-12656 Fix a bug that could occur when downloading large files from Azure.
  • APS-12700 Fix IndexOutOfBoundsException appearing randomly when using rangesharing. The computation was failing in presence of factless locations.
  • APS-12715 Pass the correct input type to IAggregationFunction when creating a reduction binding in Post-Processors.
  • APS-12716 Fix aggregation issues with Copper Left join that could occur when a filter was on a joined hierarchy.
  • APS-12719 Fix IllegalArgumentException with Copper window functions occurring when querying locations with wildcards on ALL levels.
  • APS-12725 Copper join measures now have their MDX formatters properly applied.
  • APS-12739 Fix a bug affecting branches for distribution, where hierarchies could be seen as not empty even if they were.
  • APS-12751 Cube measures are now planned correctly when queried together with a Copper join measure.
  • APS-12752 Fix PrimaryIndexRangeLookup for datastore query on non-existent member.
  • APS-12763 Fix a bug in GetAggregatesStream, causing UnsupportedOperationExceptions.
  • PIVOT-4307 Mismatches for an aggregation procedure now only impact the join measures of said procedure, and a mismatch on a procedure will no longer lead to the failure of the following aggregation procedures on the same location.
  • PIVOT-4309 Correct the health messages for distribution.
  • PIVOT-4310 Records the actual start time for DistributedPostProcessedRetrieval instead of using the start time of the underlying task.
  • PIVOT-4313 Fix a bug that could occur when merging planned datastore retrieval operations on the same store.
  • PIVOT-4321 It was not possible to change the Aggregate Cache's size at runtime, if the cache was disabled in the description.
  • PIVOT-4331 Fix a bug when no risk data was loaded in the sandbox when ran in non-distributed mode.

5.9.0​

2020-05-07

Added​

  • APS-9446 Better usage of primary and secondary indexes in Datastore queries.
  • APS-11620 Adding additional IHealthEvent events for cluster monitoring, namely for cluster view change and cluster member view change notification.
  • APS-11620 Adds a REST endpoint to forward ActiveUI traces to a tracing server. It also add support for tracing capabilities by intercepting trace information in websockets.
  • APS-11629 Allow compiledGetByKey Datastore queries to support dictionarized values of keys as input.
  • APS-11905 Introduce ActiveViamException, ActiveViamRuntimeException and ActiveViamSecurityException to replace Quartet exceptions. The new classes are now located in the package com.activeviam.fwk.
  • APS-12021 Allow the aggregation of all siblings except current member
  • APS-12052 Added AggregationLevels option to Copper doNotAggregateAbove() method to specify the required levels of aggregation of a Copper measure.
  • APS-12120 Fix Vector casting in Copper
  • APS-12141 Support for plus & minus between vector and scalar in Copper
  • APS-12229 Added nullAtTopLevels method for ParentValue measures in Copper to specify the measure behavior when the shift operation is performed above the top level.
  • APS-12329 Added toDescription method to add a description to a measure in Copper
  • APS-12304 BREAKING: Adding a check for partial provider measures that verifies that the included/excluded measures are valid aggregated measures. Any project not respecting this constraint will throw a QuartetRunitimeException at start-up.
  • APS-12458 Fix an issue that caused, in distributed environment, one object streaming failure to cascade over every subsequent object deserialization.
  • APS-12498 Fix an issue in QfsFiles that caused the getResourceAsStream method to delete first character of the given path when running with OpenJdk11.0.6 (or later) and trying to access resources inside a jar.
  • APS-12511 The ActivePivot Version of the project is now printed at project initialization, and is accessible through the ActivePivot Manager MBean.
  • APS-12532 Added the ToStringEqualCondition class to the extended plugin for matching conditions.
  • APS-12534 Fix an ArrayIndexOutOfBoundsException happening when publishing Copper join measures based on a Join Operation without a specified mapping.
  • APS-12656 Adding two ActiveViamProperties regarding the Azure Cloud Source to configure the downloading retry policy.
  • PIVOT-3669 The epoch level of the epoch dimension can now be disabled. See IEpochDimensionDescription. It is disabled by default. The epoch dimension can be renamed Branch with the following property activeviam.mdx.epoch.dimension.legacyName=false.
  • PIVOT-4032 Introduced User-defined aggregation functions that can be defined through the new Copper method userDefinedAgg([...]) or by implementing IUserDefinedAggregateFunction. Those aggregation functions can be used in aggregated measures by using the IAdvancedAggregatedMeasure interface. Read the dedicated Copper documentation for more details.
  • PIVOT-4036 Endpoint to get the query plan of an MDX query
  • PIVOT-4085 Added support for LEFT joins in Copper.
  • PIVOT-4093 Improve the deserialization of Memory Statistics.
  • PIVOT-4094 Hierarchies and measures of a data cube can now be concealed from the distributed application. More details in the what's new section
  • PIVOT-4117 Support dependencies between Aggregation Procedures. The dependencies are defined through the attribute IAnalysisAggregationProcedureDescription#getUnderlyingLevels() of the description.
  • PIVOT-4117 It is now possible to define Aggregation Procedures not depending on any Hierarchy.
  • PIVOT-4119 Support all Vector and Scalar primitive types in Copper
  • PIVOT-4122 Adding support for dynamic visualisation and change of netty messages max size limits using JMX in the DistributedMessenger MBean. Users can use now see the message max size limits as an attribute in the DistributedMessenger MBean and can change this configuration using the operation setMaxSizeForMessage(messageType, size).
  • PIVOT-4132 Allow Copper to work with non-Copper analysis levels, by using the IAnalysisHierarchy#LEVEL_TYPES_PROPERTY property.
  • PIVOT-4151 Add getter to DiscriminatorComparator to access the wrapped custom comparator.
  • PIVOT-4154 Support measure groups for BuildablePostProcessorBuilder in ICanStartBuildingMeasures
  • PIVOT-4181 Add JmxOperation to a Bitmap's aggregate provider that changes the level of the verifications on Bitmaps during a bitmap logical operation, for debugging purposes. Default value is NONE, as this will add a significant overhead in production.
  • PIVOT-4263 SimpleTransactionBuilder only starts a transaction on stores with added/removed tuples.
  • PIVOT-4264 Added CopperMeasure.withName. This is an alias for CopperMeasure.as.
  • PIVOT-4300 Added an optional cycle detection check in the dependency graph of query plans.

Changed​

  • APS-11799 ContentServerResourceServerConfigV2 was renamed ContentServerResourceServerConfig.
  • APS-11931 Use standard configuration for CORS in Spring.
  • APS-11967 BREAKING: consider the cube filter when replicating results from Data Cubes. Like with locations, if the Cube Filter contains hierarchies absent from the Data Cube, replication won't apply, according to the distribution property dataReplication.
  • APS-12025 Do not print Datastore schema in headless mode.
  • APS-12026 Remove the Bean annotation from custom filters.
  • APS-12125 Restored default behavior of ALocationShiftPostProcessor.getTargetMeasures() to getting contributors.COUNT (see PIVOT-3976 in 5.8.4)
  • APS-12169 Changed Java Native Access dependency's version from 5.2.0 to 5.5.0 for projects requiring newer versions of the project.
  • APS-12281 Make the configuration I18nConfig close the formatters it creates, instead of delegating that to ActivePivotManager.
  • APS-12370 Improve the error message when a measure is declared with an invalid formatter.
  • APS-12383 Change the Datastore-related ICondition's equals() method so that two instances are now considered equal if their sub-conditions are permutations of each other.
  • APS-12386 Providing multiple IContextValueTranslator implementations for the same context value interface will now throw an IllegalArgumentException as this case is not supported.
  • APS-12388 Change distributed data node addresses generation to process id based instead of random.
  • APS-12399 The AggregatesEvent of the ws-client module now contains a branch field containing the name of the branch on which the event occured.
  • APS-12400 Improve MDX boolean functions evaluation. Instead of an eager evaluation on both arguments we double-check if the first argument absorbe the second one, in which case the evaluation of the latter is ignored.
  • APS-12416 Hierarchies created by a Copper join on an empty store made query result empty. Now a 'default' member is contributed to prevent these hierarchies to be empty.
  • APS-12443 Changed CSVParserConfiguration full constructor. It now takes one more boolean argument, acceptIncompleteLines, which states whether the parser should take into account a line whose number of columns is smaller than the expected, or not.
  • APS-12490 Upgrade versions of various dependencies like Spring and CXF.
  • APS-12468 Preventing unnecessary catalog creation for distributed cube when using the fluent API.
  • APS-12514 Changing the default companion when the expression of the calculated member is a member of the same hierarchy and not a calculated member. This will improve the performances.
  • APS-12549 Fix postprocessors auto-partitioning in the the finalization of planning phase.
  • APS-12605 The Copper shift measures creation methods have changed : CopperMeasure#at([...]) methods have been changed to a unified CopperMeasure#shift(CopperLevelsAt...) method. Find more information in our user guide for updated examples.
  • APS-12647 BREAKING: Changed CopperMeasure's method cast() to withType().
  • PIVOT-3434 Add metadata to the DTOs of the ContentServer WS API after changes. This avoids additional REST calls to retrieve them. This changed the structure of ContentUpdate and introduce an new DTO ContentUpdateDto.
  • PIVOT-3669 Parameters of the epoch dimension were renamed (see IEpochDimensionDescription):
    • branchesLevelComparator becomes branchLevelComparator
    • branchesLevelFormatter becomes branchLevelFormatter
    • epochsLevelComparator becomes epochLevelComparator
    • epochsLevelFormatter becomes epochLevelFormatter
  • PIVOT-3669 MdxCubeUtil.countEpochsBetween was replaced by IMdxEpochManager.countEpochsBetween.
  • PIVOT-3996 Reduced dependencies of parquet-source by excluding packages and importing smaller ones.
  • PIVOT-4029 Change the Min and Max aggregation functions to append only by default. The Min/Max functions supporting suppression, i.e., disaggregation (history functions) will be either called by the bitmap or specified by the user.
  • PIVOT-4037 Convert the sandbox to Spring Boot.
  • PIVOT-4037 Update Activiti configuration to be SpringBoot compatible.
  • PIVOT-4037 Interrupt daemon threads on closing.
  • PIVOT-4058 Migrate MDX documentation to Docusaurus.
  • PIVOT-4069 Change the signature of IPlugin.valueOf. The plugin keys can no more be arrays.
  • PIVOT-4074 The method IAnalysisAggregationProcedure.createContext() takes a new argument: IScopeLocation prefetchLocation that corresponds to the location processed by the aggregation procedures in IAnalysisAggregationProcedure.buildRequestScope().
  • PIVOT-4117 Aggregation Procedures cannot depend on the hierarchies they handle. This is enforced at validation time.
  • PIVOT-4120 CopperMeasure.at now takes fully defined CopperLevel instead of the level's name as arguments.
  • PIVOT-4122 Changed the default distributed message size limits.
  • PIVOT-4120 If you use Copper.level(levelName) then Copper APIs receiving it will now throw if several levels match instead of picking the first one. You should use Copper.level(dimensionName, hierarchyName, levelName) instead. Same for Copper.hierarchy(hierarchyName).
  • PIVOT-4136 Fix MeasuresDetail attribute in the partial provider MBean.
  • PIVOT-4137 Better Object to primitive conversion in Records.
  • PIVOT-4137 Do not push idempotent locale changes in the Content Server.
  • PIVOT-4162 Detect line ending at byte level in the CSV Source.
  • PIVOT-4171 Reduced memory consumption of HierarchiesUtil#getDefaultMember (avoid creating a list).
  • PIVOT-4208 Change how the descriptions are post processed. Read the migration notes for more details.
  • PIVOT-4219 Distributed Cube no longer accept the epochLevel to be enabled in the epoch dimension, all queries being run against the pivot's head. Read the distributed what if page for more details.
  • PIVOT-4224 Removed unnecessary copy of array in CartesianProductValuesSupplier.
  • PIVOT-4273 JustInTime is now the default AggregateProvider instead of Bitmap.
  • PIVOT-4281 Alligning core dependencies version with spring-boot 2.2.6.RELEASE.

Deprecated​

  • APS-11905 Deprecate QuartetException, QuartetRuntimeException and QuartetSecurityException into ActiveviamException, ActiveviamRuntimeException and ActiveviamSecurityException. The new classes are now located in the package com.activeviam.fwk.
  • APS-12686 The JgroupsMessenger is deprecated and will be removed in 5.10.0. NettyMessenger and LocalMessenger are the most efficient messengers and the ones users should rely on.
  • PIVOT-4070 The transaction log and the datastore replication will be removed in a future version.
  • PIVOT-4217 Configuring your application with Spring XML configuration files is deprecated.

Removed​

  • APS-11931 Delete SpringCorsFilter in favour of Spring implementation.
  • APS-12215 Internal methods potentially visible to users have been deprecated:
    • IInternalAggregatesRetrievalResult#join(..) without ICubeFilter
  • APS-12281 Replace ICubeFormatterFactory#stop by ICubeFormatterFactory#close from AutoCloseable.
  • DOC-273 Remove deprecated class com.qfs.msg.csv.translator.impl.ConstantCalculator, which was moved to the composer module in 5.6.1.
  • PIVOT-3121 Remove deprecated constructor InMemoryReportStorage(Duration, ITimeSupplier), defining an unused attribute.
  • PIVOT-4081 Remove the ability to load the ActivePivot descriptions from the IActivePivotContentService
  • PIVOT-4141 Remove unused internal dependency from ActivePivotServicesConfig to IDatastoreConfig.
  • PIVOT-4166 Remove IntegerArrayVersion.largestSize.
  • PIVOT-4169 Remove unused property ICSVSourceConfiguration.FOLLOW_SYMBOLIC_LINKS_PROPERTY, because it is already specified when defining the FileSystemCSVTopicFactory.
  • PIVOT-4217 Remove Spring XML configuration files. You are encouraged to use the Spring Java configuration.
  • PIVOT-4270 Removing deprecated methods across the repository. Removed MinimumDepthPostProcessor
  • PIVOT-4270 Removed MinimumDepthPostProcessor in favor of MinimumLevelsPostProcessor
  • PIVOT-4270 Removed TimeBucketerPostProcessor in favor of using CoPPer
  • PIVOT-4270 Removed ALegacyPostProcessor in favor of AAdvancedPostProcessor
  • PIVOT-4270 Removed SimplePostProcessorLegacy

Fixed​

  • APS-9409 Improve Mdx queries speed by reducing lock contention on the internal results
  • APS-11728 Fix a bug in the QFSBitmap.not operator, that could produce malformed bitmaps that could make other operations throw when run on the malformed bitmap.
  • APS-12081 Fixing NullPointerException triggered by the initialization of the FilteringPostProcessorOnTheFlyPattern.
  • APS-12082 In some cases the server could send a transient empty drillthrough result when the actual result was computed. This is no more the case.
  • APS-12109 Fix a NullPointerException that could occur in the VersionHistory when releasing a version.
  • APS-12132 Fix ArrayIndexOutOfBoundsException in Copper which could happen in measures using several time the same underlying measure
  • APS-12134 Do not fail when lookup for a location/filter without result.
  • APS-12136 Fix publication of implicit hierarchies due to joins made in Copper
  • APS-12147 Fixed the DatastoreService that could throw an ArrayOutOfBoundException when having more fields on the concerned target store than on the main store, an offset greater than the amount of fields in the main store, and when trying to access a field with an index greater that the offset.
  • APS-12175 Fix bug in the Mdx Iif function when used with tuples
  • APS-12215 Correctly set retrieval results filters when Analysis Hierarchies are involved. It previously resulted in missing results.
  • APS-12217 Fix name generation when using cube as datacube
  • APS-12227 Fixed JSON queries that wrongly locked all stores
  • APS-12240 Fix the partitioning levels of the Copper shift operations when data is partitioned on a level of the shifted hierarchy.
  • APS-12443 acceptIncompleteLines set to false properly thrown when the last line of a csv file is incomplete.
  • APS-12299 Fix a NoSuchElementException that could happen while using a Copper shift operation defined with a lambda along a cube filter.
  • APS-12305 Fixed an issue with branch deletion in a distributed cluster, causing a small memory leak for each deleted branch.
  • APS-12306 Fix an UnsupportedOperationException occurring when committing on a branch after dropping and recreating a partition on another branch. Also fix disappearing records on a branch when, on another branch, their partition is dropped and not re-created.
  • APS-12361 Fix an IllegalArgumentException when using a defined CopperMeasure instance as underlyingMeasure of another CopperMeasure.
  • APS-12374 Fix an issue with the publication of optimized measures in Copper.
  • APS-12387 Fix a NullPointerException that occured when removing a data node and readding it to the cluster within removalDelay seconds.
  • APS-12393 Fix a measure publication issue in Copper when calling filter() on an already filtered measure.
  • APS-12399 Fix the XML serialization of the FullRefreshEvent.
  • APS-12466 Fix a RuntimeException about non shared-dictionaries that could happen when querying a Join-defined measure defined in Copper 2 in a distributed scenario with multiple join operations.
  • APS-12477 Fix the declared output type of the AverageFunction aggregation function to match the type of the returned aggregates.
  • APS-12499 Fix a ClassCastException that could happen using Copper NamedMeasures originated from a CopperMeasure instance that was previously cast with CopperMeasure#cast() .
  • APS-12510 Fix the IllegalArgumentException: Empty list of fields to retrieve from the store happening with JoinAnalysisAggregationProcedure when doing a Copper query with no hierarchy in the location.
  • APS-12546 CSVSourceConfiguration constructor correctly uses the defaults of the given Properties, if present.
  • APS-12568 Fix an issue with DirectChunkIntegerNullable, which could be converted to a DirectChunkInteger while still having null values
  • APS-12586 Changed a NullPointerException into an explicit exception when starting a cube containing a measure with an invalid formatter.
  • APS-12641 Fix ranges update on registered query.
  • APS-12647 Fix an issue with .cast() which was ignored when used prior to a .doNotAggregateAbove() and other methods.
  • PIVOT-3927 Fix the visibility of Distributed epochs, not to see random past epochs.
  • PIVOT-3976 Correctly handle the publication of measures already defined outside of Copper calculations.
  • PIVOT-4035 Fixed some QFSBitmap exceptions that were caused by logical OR operations.
  • PIVOT-4133 Fix the multiply operator between a scalar measure and a vector measure in Copper
  • PIVOT-4139 Allow .excludeCurrent() with MIN and MAX aggregation functions in cumulative Copper measures
  • PIVOT-4140 Fix a NullPointerException happening when merging partitionned records from a datastore prefetch request.
  • PIVOT-4152 APS-12387 PointListRetrievalResult methods now regularly checks for query interruption, preventing certain queries to continue running in the background while they have responded with a timeout.
  • PIVOT-4161 Fix an issue happening with Java 11: the first submitted records in a partition could disappear
  • PIVOT-4168 Fix an issue with aggregation analytic functions in Copper returning unexpected results using vector fields.
  • PIVOT-4170 Fix a Copper bug in which an error was thrown when a user redefined the dimension or hierarchy name of a join level and then reused that level.
  • PIVOT-4185 Fix a NullPointerException that could happen when optimizing measures with null values in Copper 2.
  • PIVOT-4212 Correctly check the field of CSV Column Calculator when passed to the Parquet Source.
  • PIVOT-4229 Fix an issue with the Copper Shift measure that would not be evaluated when it's underlying measure is null, regardless of the shift operation.
  • PIVOT-4248 Fix an issue in Copper join operations leading to unexpected results when the levels were mapped through the join to fields that were not in the key of the joined store.
  • PIVOT-4272 Fix an issue where having multiple join procedures could lead to a wrong execution logic for said procedures.
  • PIVOT-4278 Fix an issue with Copper join operations not working when the mapping fields of the join were on different stores.
  • PIVOT-4288 Fix query planning producing wrong aggregated values when the datastore is partitioned on a field absent in the pivot.
  • SENTINEL-606 Correct listing of stopped alerts, possibly reporting alerts stopped by a concurrent operation.