5.8.12

Changelog

For user-facing changes, refer to the What's New? page.

The format is based on Keep a Changelog and this project adheres to Romantic Versioning.

5.8.12

2020-07-31

Added

  • APS-12724 Add a new ActiveViamProperty, set through -Dactiveviam.contentServer.restrictedListing=<boolean>, which will change the permission behavior of the Content Server. It defaults to false. When set to true, a user will not be able to list a file without having read access to the said file, even if he has read access to the parent directory.

Changed

  • APS-12730 Improve query engine to keep the maximal partitioning for operations on primitive measures.
  • APS-12757 Reducing network call of ICloudEntityPath#getLastModified() for Azure implementation.

Fixed

  • APS-12641 Fix ranges update on registered query.
  • APS-12710 Speed up the building of Copper-based bucketing Analysis Hierarchies depending on multiple levels.
  • APS-12725 Copper join measures now have their MDX formatters properly applied.
  • APS-12746 Fix an issue that could lead to a wrong order of dependencies of Copper measures.
  • APS-12751 Cube measures are now planned correctly when queried together with a Copper join measure.
  • APS-12763 Fix a bug in GetAggregatesStream, causing UnsupportedOperationExceptions.
  • APS-12769 Prevent NPE in AAdvancedPostProcessor#retrieveNamedPrefetchAggregatesWithAnalysisLevels
  • APS-12777 Allow StreamingService to garbage collect and dispose of already stopped streams.
  • APS-12790 Improve the performance of ChunkHistory.isNull().
  • APS-12802 Fix a bug where Copper measures based on analytic functions would return wrong results when the hierarchy prior to the hierarchy used in the Window was slicing and queried with a range coordinate.
  • APS-12805 The Mdx engine now supports VBA functions used with the VBA! prefix.
  • PIVOT-4312 Fix IllegalArgumentException happening when querying a Join measure from a isolated store joined with Copper without mapping on a location where join-related Analysis hierarchies are not expressed.
  • PIVOT-4313 Fix a bug that could occur when merging planned datastore retrieval operations on the same store.
  • PIVOT-4321 It was not possible to change the Aggregate Cache's size at runtime, if the cache was disabled in the description.

5.8.11

2020-06-04

Added

  • PIVOT-4264 Add CopperMeasure.withName(name) and CopperMeasure.parentValueOn(hierarchy)

Deprecated

  • APS-12718 Deprecating DrillthroughExecutor.doExecuteHeaders(Collection<ILocation> locations, Collection<String> measures, boolean isDrillthroughContinuousQuery, boolean log) in favor of its overloaded sibling without the log boolean. For removal in 5.10.0.
  • PIVOT-4264 Copper.parentValue(measure, hierarchy) will be replaced by CopperMeasure.parentValueOn(hierarchy)

Fixed

  • APS-12588 Fix a bug with dropped partitions. The drop was not correctly reverted when the transaction was rolled back. This could cause multiple exceptions in the Bitmap aggregate provider.
  • APS-12656 Fix a bug that could occur when downloading large files from Azure.
  • APS-12700 Fix IndexOutOfBoundsException appearing randomly when using rangesharing. The computation was failing in presence of factless locations.
  • APS-12716 Fix aggregation issue with join in Copper2. The results could be wrong when creating a filter on a hierarchy defined on the joined store.
  • APS-12719 Fix IllegalArgumentException with Copper window functions occurring when querying locations with wildcards on ALL levels.
  • APS-12739 Fix a bug affecting branches for distribution, where hierarchies could be seen as not empty even if they were.
  • APS-12782 Fix a bug where the default measure defined in the pivot description was ignored.

5.8.10

2020-05-05

Added

  • APS-12652 Adding two ActiveViamProperties regarding the Azure Cloud Source to configure the downloading retry policy.

Fixed

  • APS-12549 Fix postprocessors auto-partitioning in the the finalization of planning phase.
  • APS-12647 Fix an issue with .cast() in Copper 2 which was ignored when used prior to a .doNotAggregateAbove() and other methods
  • PIVOT-4229 Fix an issue with the Copper Shift measure that would not be evaluated when it's underlying measure is null, regardless of the shift operation.
  • PIVOT-4272 Fix an issue where having multiple join procedures could lead to a wrong execution logic for said procedures.
  • PIVOT-4278 Fix an issue with Copper join operations not working when the mapping fields of the join were on different stores.
  • PIVOT-4288 Fix query planning producing wrong aggregated values when the datastore is partitioned on a field absent in the pivot.

5.8.9

2020-04-07

Added

  • APS-11620 Adding additional IHealthEvent events for cluster monitoring, namely for cluster view change and cluster member view change notification.
  • APS-12352 Added the ToStringEqualCondition class to the extended plugin for matching conditions.
  • APS-12511 The ActivePivot Version of the project is now printed at project initialization, and is accessible through the ActivePivot Manager MBean.

Changed

  • APS-12514 Changing the default companion when the expression of the calculated member is a member of the same hierarchy and not a calculated member. This will improve the performances.
  • PIVOT-3927 Rewrite the implementation of DistributedVersionHistory, not to extend VersionHistory anymore.
  • PIVOT-4224 Removed unnecessary copy of array in CartesianProductValuesSupplier.
  • PIVOT-4263 SimpleTransactionBuilder only starts a transaction on stores with added/removed tuples.

Deprecated

  • APS-12352 The CubeFilter#ToStringEqualCondition inner class have been deprecated. It has been replaced by com.quartetfs.fwk.filtering.impl.ToStringEqualCondition. The previous class will be removed in version 5.9.0.
  • APS-12647 Deprecating cast() in Copper 2 in favor of withType().
  • PIVOT-4081 Deprecate the usage of the cube descriptions in the ActivePivotContentService and ActivePivotManagerDescriptionConfigFromContentServer.

Fixed

  • APS-12361 Fix an IllegalArgumentException when using a defined CopperMeasure instance as underlyingMeasure of another CopperMeasure.
  • APS-12534 Fix an ArrayIndexOutOfBoundsException happening when publishing Copper join measures based on a Join Operation without a specified mapping.
  • APS-12499 Fix a ClassCastException that could happen using Copper NamedMeasures originated from a CopperMeasure instance that was previously cast with CopperMeasure#cast() .
  • APS-12510 Fix the IllegalArgumentException: Empty list of fields to retrieve from the store happening with JoinAnalysisAggregationProcedure when doing a Copper query with no hierarchy in the location.
  • APS-12546 CSVSourceConfiguration constructor correctly uses the defaults of the given Properties, if present.
  • APS-12568 Fix an issue with DirectChunkIntegerNullable, which could be converted to a DirectChunkInteger while still having null values
  • APS-12586 Changed a NullPointerException into an explicit exception when starting a cube containing a measure with an invalid formatter.
  • PIVOT-3927 Fix the visibility of Distributed epochs, not to see random past epochs.
  • PIVOT-4248 Fix an issue in Copper join operations leading to unexpected results when the levels were mapped through the join to fields that were not in the key of the joined store.

5.8.8

2020-02-18

Changed

  • APS-12399 The AggregatesEvent of the ws-client module now contains a branch field containing the name of the branch on which the event occured.
  • APS-12443 Changed CSVParserConfiguration full constructor. It now takes one more boolean argument, acceptIncompleteLines, which states whether the parser should take into account a line whose number of columns is smaller than the expected, or not.

Deprecated

  • APS-12443 CSVParserConfiguration full constructor now takes one more boolean argument, acceptIncompleteLines. The previous constructor will be removed in version 5.9.0.

Fixed

  • APS-12391 Fix a NullPointerException that occured when removing a data node and readding it to the cluster within removalDelay seconds.
  • APS-12399 Fix the XML serialization of the FullRefreshEvent.
  • APS-12443 acceptIncompleteLines set to false properly throws when the last line of a csv file is incomplete.
  • APS-12458 Fix a StreamCorruptedError that could happen in NettyInputStream after receiving an exception message, and wrongly resetting the stream's reading index.
  • APS-12466 Fix a RuntimeException about non shared-dictionaries that could happen when querying a Join-defined measure defined in Copper 2 in a distributed scenario with multiple join operations.
  • APS-12477 Fix the declared output type of the AverageFunction aggregation function to match the type of the returned aggregates.
  • APS-12498 Fix an issue in QfsFiles that caused the getResourceAsStream method to delete first character of the given path when running with OpenJdk11.0.6 (or later) and trying to access resources inside a jar.
  • PIVOT-4212 Correctly check the field of CSV Column Calculator when passed to the Parquet Source.

5.8.7

2020-01-17

Added

  • APS-12141 Support for plus & minus between vector and scalar in Copper 2
  • APS-12329 Added method withDescription to set a description of your measure in Copper 2

Changed

  • APS-12304 BREAKING: Adding a check for partial provider measures that verifies that the included/excluded measures are valid aggregated measures. Any project not respecting this constraint will throw a QuartetRuntimeException at start-up.
  • APS-12370 Improve the error message when a measure is declared with an invalid formatter.
  • APS-12400 Improve MDX boolean functions evaluation. Instead of an eager evaluation on both arguments we double-check if the first argument absorbe the second one, in which case the evaluation of the latter is ignored.
  • APS-12468 Preventing unnecessary catalog creation for distributed cube when using the fluent API.

Fixed

  • APS-11728 Fix an bug in the QFSBitmap.not operator, that could produce malformed bitmaps that could make other operations throw when run on the malformed bitmap.
  • APS-12175 Fix bug in the Mdx Iif function when used with tuples
  • APS-12305 Fixed an issue with branch deletion in a distributed cluster, causing a small memory leak for each deleted branch.
  • APS-12374 Fix an issue with the publication of optimized measures in Copper 2.
  • APS-12383 Change the Datastore-related ICondition's equals() method so that two instances are now considered equal if their sub-conditions are permutations of each other.
  • APS-12387 PointListRetrievalResult methods now regularly checks for query interruption, preventing certain queries to continue running in the background while they have responded with a timeout.
  • APS-12393 Fix a measure publication issue in Copper 2 when calling filter() on an already filtered measure.
  • PIVOT-4136 Fix MeasuresDetail attribute in the partial provider MBean.
  • PIVOT-4161 Fix an issue happening with Java 11: the first submitted records in a partition could disappear
  • PIVOT-4168 Fix an issue with aggregation analytic functions in Copper 2 returning unexpected results using vector fields.
  • PIVOT-4170 Fix a Copper bug in which an error was thrown when a user redefined the dimension or hierarchy name of a join level and then reused that level.
  • PIVOT-4185 Fix a NullPointerException that could happend when optimizing measures with null values in Copper 2.

5.8.6

2019-12-13

Added

  • APS-11629 Allow compiledGetByKey Datastore queries to support dictionarized values of keys as input.
  • APS-12052 Added AggregationLevels option to Copper2 doNotAggregateAbove() method to specify the required levels of aggregation of a Copper measure.
  • APS-12021 Allow the aggregation of all siblings except current member
  • APS-12120 Fix Vector casting in Copper
  • APS-12229 Added nullAtTopLevels() method for ParentValue measures in Copper2 to specify the measure behavior when the shift operation is performed above the top level.
  • PIVOT-4117 Support dependencies between Aggregation Procedures. The dependencies are defined through the attribute IAnalysisAggregationProcedureDescription#getUnderlyingLevels() of the description.
  • PIVOT-4117 It is now possible to define Aggregation Procedures not dependending on any Hierarchy.
  • PIVOT-4119 Support all Vector and Scalar primitive types in Copper 2
  • PIVOT-4132 Allow Copper to work with non-Copper analysis levels, by using the IAnalysisHierarchy#LEVE L_TYPES_PROPERTY property.

Changed

  • APS-11967 BREAKING: consider the cube filter when replicating results from Data Cubes. Like with locations, if the Cube Filter contains hierarchies absent from the Data Cube, replication won't apply, according to the distribution property dataReplication. Not to break existing projects, a temporary new replication policy was introduced REPLICATE_ON_SLICER_IGNORING_FILTER. It provides the behaviour before the fix, and will be removed in 5.9.
  • APS-12179 Updated nimbus-jose dependency version to 8.2 to avoid a potential vulnerability.
  • PIVOT-4120 BREAKING: CopperMeasure.at now takes fully defined CopperLevel instead of the level's name as arguments.
  • PIVOT-4120 BREAKING: If you use Copper.level(levelName) then Copper APIs receiving it will now throw if several levels match instead of picking the first one. You should use Copper.level(dimensionName, hierarchyName, levelName) instead. Same for Copper.hierarchy(hierarchyName).
  • PIVOT-4137 Better Object to primitive conversion in Records.
  • PIVOT-4137 Do not push idempotent locale changes in the content server.
  • PIVOT-4137 Made the datastore configuration optional in ActivePivotServicesConfig.
  • PIVOT-4171 Reduced memory consumption of HierarchiesUtil#getDefaultMember (avoid creating a list).

Deprecated

  • APS-11967 The following were only defined for support reasons and are already deprecated.
    • Enum value DataReplication#REPLICATE_ON_SLICER_IGNORING_FILTER
    • Value "REPLICATE_ON_SLICER_IGNORING_FILTER" for the Distribution property dataReplication.
  • APS-12215 Internal methods potentially visible to users have been deprecated:
    • IInternalAggregatesRetrievalResult#join(..) without ICubeFilter
    • IInternalAggregatesRetrievalResult#expandResult(List<? extends IHierarchy>, List<? extends ILevelInfo>)

Fixed

  • APS-12082 In some cases the server could send a transient empty drillthrough result when the actual result was computed. This is no more the case.
  • APS-12132 Fix ArrayIndexOutOfBoundsException in Copper 2 which could happen in measures using several time the same underlying measure
  • APS-12136 Fix publication of implicit hierarchies due to joins made in Copper
  • APS-12215 Correctly set retrieval results filters when Analysis Hierarchies are involved. It previously resulted in missing results.
  • APS-12217 Fix name generation when using cube as datacube
  • APS-12227 Fixed JSON queries that wrongly locked all stores
  • APS-12240 Fix the partitioning levels of the Copper shift operations when data is partitioned on a level of the shifted hierarchy.
  • APS-12299 Fix a NoSuchElementException that could happen while using a Copper 2 shift operation defined with a lambda along a cube filter.
  • APS-12306 Fix an UnsupportedOperationException occurring when committing on a branch after dropping and recreating a partition on another branch. Also fix disappearing records on a branch when, on another branch, their partition is dropped and not re-created.
  • PIVOT-4133 Fix the multiply operator between a scalar measure and a vector measure in Copper
  • PIVOT-4139 Allow .excludeCurrent() with MIN and MAX aggregation functions in cumulative Copper measures
  • PIVOT-4140 Fix a NullPointerException happening when merging partitionned records from a datastore prefetch request.

5.8.5

2019-11-08

Added

  • APS-11629 Allow compiledGetByKey Datastore queries to support dictionarized values of keys as input.
  • APS-12052 Added AggregationLevels option to Copper2 doNotAggregateAbove() method to specify the required levels of aggregation of a Copper measure.
  • APS-12021 Allow the aggregation of all siblings except current member
  • PIVOT-4132 Allow Copper to work with non-Copper analysis levels, by using the IAnalysisHierarchy#LEVEL_TYPES_PROPERTY property.

Changed

  • APS-12125 Restored default behavior of ALocationShiftPostProcessor.getTargetMeasures() to getting contributors.COUNT (see PIVOT-3976 in 5.8.4)
  • PIVOT-4058 Migrate MDX documentation to Docusaurus.
  • PIVOT-4093 Improve the deserialization of Memory Statistics.

Fixed

  • APS-12081 Fixing NullPointerException triggered by the initialization of the FilteringPostProcessorOnTheFlyPattern.
  • PIVOT-3976 Correctly handle the publication of measures already defined outside of Copper calculations.
  • APS-12109 Fix a NullPointerException that could occur in the VersionHistory when releasing a version.
  • APS-12120 fix Vector casting in Copper.
  • APS-12134 Do not fail when lookup for a location/filter without result.
  • SENTINEL-606 Correct listing of stopped alerts, possibly reporting alerts stopped by a concurrent operation.
  • APS-12147 Fixed the DatastoreService that could throw an ArrayOutOfBoundException when having more fields on the concerned target store than on the main store, an offset greater than the amount of fields in the main store, and when trying to access a field with an index greater that the offset.
  • PIVOT-4035 Fixed some QFSBitmap exceptions that were caused by logical OR operations

5.8.4

2019-10-03

Added

  • APS-11965 Add ContentServerSnapshotter.eraseAndImport with InputStream as argument.
  • APS-11987 Ignore empty files when reading parquet folder.

Changed

  • PIVOT-4013 Throw an exception if a schema contains cubes with and without the epoch dimension.
  • PIVOT-3275 Throw an exception if a file in the content server is moved to the same target.
  • PIVOT-11450 Enforcing non-vector fields in cube levels.
  • PIVOT-4057 Update all REST documentation to end a @RequestParameter with an equal sign.`
  • PIVOT-4064 Filter context values sent to ActiveUI to not expose user-authentication and subcube context values.
  • PIVOT-4038 Improved the documentation about ActiveViamProperties
  • APS-11933 Some methods of ActivePivotSchemaTransactionManager are no more synchronized.
  • PIVOT-4041 Only reference block vectors under chunks of Vectors, instead of Vectors with their blocks.
  • APS-11950 Defining Copper calculations with the withCalculations([...]) method can only be called once per cube constructor.
  • APS-12002 Update the sandbox project to the latest ContentServer UI and ActiveUI app.
  • APS-12009 Allow CopperMeasure#mapToDouble() to use typed lambdas.
  • APS-11932 Throw an exception when trying to create a CSVParserConfiguration containing empty column names"
  • PIVOT-3713 ChunkPrimitiveInteger was renamed AChunkPrimitiveInteger.
  • APS-12035 Revert optimization when relinking indexes. It seems to generate DiscardedEpochException for still unknown reasons.
  • PIVOT-3976 API Change (rolled back with version 5.8.5): Changed default behavior of ALocationShiftPostProcessor.getTargetMeasures() when getting the underlying measures. If there are none, the method defaults to contributors.COUNT

Fixed

  • PIVOT-4020 Fixed an issue in secondary indexes where indexes were not relinked entirely when an updateWhere triggered on a commit.
  • APS-11955 Fixed a bug in PrefixedContentServer where a ConflictException could happen when starting concurrent Pivot instances on the same Content Server.
  • APS-11940 Check for overflow before requesting direct chunk allocation
  • PIVOT-4013 In some cases continuous selection listeners on a single branch could see other branches.
  • APS-11933 Fixed an ActivePivotTransactionException that could be thrown by ActivePivotSchemaTransactionManager.startTransaction when an isolated store had an analysis hierarchy.
  • PIVOT-4041 Add missing chunks attributes for JDK11 chunk classes.
  • PIVOT-4048 Copper measures named with .as() will always be published and available for queries by their name.
  • APS-12056 PIVOT-4024 Fix issue related to measure publication in Copper 2
  • APS-12057 Fixed a bug in the Parquet Source that initialized null values with the previously read value.
  • PIVOT-4063 Fixed a ClassCastException when performing a DistinctCount aggregation.

5.8.3

2019-08-12

Added

Changed

  • APS-11863 Changed log level from Warning to Fine for task.printPendingCompleters() in ActivePivotVersion.printExpiredQueryWarning.

Deprecated

  • APS-11865 The empty constructor of RangeSetUnionProcedure is deprecated.

Fixed

  • APS-11801 Fixed a bug in LocationUtil.createRangeLocation(List<? extends IHierarchy> hierarchies, List<IHierarchyInfo> hierInfos, List<ILevelInfo> levelInfos, ILocation originalLocation) that could cause an IndexOutOfBoundException when using a FormulaPostProcessor with the dynamicAggregation operand.
  • APS-11871 Correct code not to look for Range-Sharing optimizations when it is configured to 0. It only saves some CPU time, as it would not have found any Range-Sharing candidates.
  • APS-11865 Speedup Planning finalization time by reducing the number of retrievals. It will impact queries containing a list with tens of members, including non existing ones.
  • PIVOT-3800, APS-11292 Fix the way postprocessors are auto-partitioned. ClassCastException in PostProcessedResultsMerger does not happen anymore.
  • APS-11886 Fixed a bug in a reordering operation of secondary indexes that could induce a DiscardedEpochException
  • APS-11849 Fixed an IllegalStateException while filtering on the members of a Analysis Hierarchy configured with an Aggregation Procedure.
  • APS-11922 Fixed ActivePivotWebServicesConfig to Autowire the IActivePivotServicesConfig interface rather than its provided implementation.
  • APS-11693 Prevent query cube from incorrectly removing data cube contribution.

5.8.2

2019-07-03

Added

  • APS-11817 Setting the new flag acceptIncompleteLines to false will throw when parsing a CSV file and encountering a number of columns smaller than expected. The default value is true (i.e. log but do not throw).
  • APS-11426 MDX Order function now supports stable sort for simple statements where call to Order are directly nested. This means that executing Order1(Order2(set)) will sort the set according to Order1, tie-breaking with Order2.

Changed

  • APS-11722 Exclude Analysis Hierarchies from Partial Aggregate Providers.
  • APS-11730 Update aws-sdk version from 1.11.240 to 1.11.368, and update jgroups version from 4.0.15.Final to 4.0.19.Final so that it is aligned with that of aws-sdk.
  • PIVOT-3602 Throw when in SUBSELECT statements, an MDX expression cannot be written as a CrossJoin of single hierarchy sets.
  • PIVOT-3968 Improve the performance of relinking modified records in secondary indices.
  • APS-11799 ContentServerResourceServerConfigV2 changes the content server UI namespace from content to content/ui because of a conflict which broke the scheduling service.
  • PIVOT-3966 Review the registration of default Types and Plugins from the core product, not to collide with user-defined Types and Plugins. As part of the change, internal classes AttachedQueryExtendedPlugin and DistributedAgentExtendedPlugin have been moved, and ReturnFunctionPlugin has been renamed into ReturnFunctionExtendedPlugin.

Deprecated

  • APS-11415 StreamEventUtil.getEventHeads(Object event, long epoch) was replaced by StreamEventUtil.getEventHeads(Object event).
  • APS-11799 com.qfs.content.cfg.impl.ContentServerResourceServerConfig should no longer be used. It is replaced by com.qfs.content.cfg.impl.ContentServerResourceServerConfigV2, and the "V2" will be removed in 5.9.0.
  • APS-11817 The constructor of CSVParserConfiguration that defines the acceptIncompleteLines flag should be used instead of the one that sets the flag to its default value.

Fixed

  • APS-11744 Partial providers were not used when there were custom permissions on branches.
  • PIVOT-3624 Correctly take the cube filter into account when performing a drillthrough on the query cube.
  • APS-11415 Fix IllegalStateException: Query should not be executed on shadow version logs when using branches.
  • APS-11779 Correctly return the size of the stores
  • PIVOT-3969 Fix resource not found (e.g. i18n) when running the application as a jar with SpringBoot.
  • APS-11798 Correctly create PostProcessors when mixing fluent builders and standard builders
  • PIVOT-3988 Correct #equals(..) for compressed entries ActiveViam sources. It was previously throwing ClassCastExceptions.
  • PIVOT-3997 Correct duplicate handling in the list of stores passed to a Datastore transaction. Previously, some stores could be erroneously deleted from the list.
  • APS-11793 Fix invalid cast in Repository Hibernate layer, causing ClassCastException to Coordinates.
  • APS-11768 Fix ArrayIndexOutOfBoundException in AAdvancedPostProcessor#removeAnalysisLevelsFromFilter, which was encountered when using a FilteringPostProcessor.
  • PIVOT-3991 Fixed missing data in the statistics exported by MemoryAnalysisService.

5.8.1

2019-06-01

Added

  • APS-10824 Allow an IActivePivotManagerDescription to have a single distributed cube with fluent builders.
  • PIVOT-3963 Added a MBean attribute for Partial Providers listing included measures.
  • APS-11178 A gross sum aggregation function (sum of the absolute values).

Changed

  • PIVOT-3920 Reuse existing record format for the result if available.
  • PIVOT-3919 Use a code fast-path when splitting paths of one char.
  • PIVOT-3818 Improve the registration of MDX streams to consume less CPU resource.
  • APS-11601 Update the READMEs of the Sandbox project, to detail and explain how to customize the configuration of the packaged UIs.
  • APS-11702 Do not restart every monitor when one monitor fails to start. Instead, this monitor start will be retried later.

Fixed

  • PIVOT-3942 Correctly use IPartitionedPostProcessor#DEFAULT_PARTITIONING_ON_RANGE_LEVELS as the default value auto range partitioning for Post-Processors.
  • PIVOT-3540 Correct re-initialization of the MdxRegistration state when creating new Listeners.
  • APS-11657 Fix a NullPointerException in the CSVExportDrillthroughExecutor which occurred when one of the records had a null field.
  • PIVOT-3957 Fix BadArgumentException which occurred when a range property was set and the result of an Mdx query contained an empty axis.
  • APS-10884 Precise documentation on StoreDescription.shouldCompressIndexes() and CompressIndexesPostProcessor. Improve memory allocation for dictionaries.
  • PIVOT-3957 Add documentation on NumaSelectorPostProcessor.
  • APS-11563 Include all measures NoOp retrievals are retrieving into their result.
  • PIVOT-3960 Fix a bug in QFSBitmap#or that could result in extra members appearing in a result.
  • APS-11728 Fix a bug in QFSBitmap#or that could produce wrong results when using filters in a query.
  • APS-11719 Fix a bug in Store builders where a Duplicate key handler could not be defined when there was no pending taggable field.
  • PIVOT-3965 Fix an IllegalArgumentException that could be raised when using SpringBoot > 1.3 and giving resources with absolute paths on Windows.
  • PIVOT-3565 Correctly retrieve the timestamp of a file last modification.
  • APS-11722 Correctly pick the best Partial Aggregate Provider even with locations or subcubes including members of Analysis Hierarchies.
  • PIVOT-3864 Forbid to start a transaction on stores indirectly involved (i.e. through a reference) in an insertion-time update-where trigger.

5.8.0

2019-04-15

Added

  • PIVOT-3622 Integrate the Cloud Source to the core product.
  • APS-11272 Add a new post-processor FilteringPostProcessorOnTheFlyPattern as an alternative to the MDX function MemberCaptionSearch. It allows to search for a given pattern in member captions for a given hierarchy. It will be used by ActiveUI when available.
  • APS-11323 Provide a context value translator for every core Context Value.
  • APS-10617 Add a way to manually handle field name conflicts when building the datastore selection.
  • APS-11533 Expose the range sharing in IAggregateProviderDefinition
  • PIVOT-3804 Support the lazy loading of Mdx results with any number of axes.

Changed

  • PIVOT-3839 Datastore and Copper tests have been moved to their own modules (com.activeviam.tech:datastore-test and com.activeviam.activepivot:activepivot-copper-test)

  • ABasicPostProcessor (respectively ABaseDynamicPostProcessors) now uses named prefetchers. They will use the constant ABasicPostProcessor.BASIC_POST_PROCESSOR_PREFETCHER (resp. ABaseDynamicAggregationPostProcessor.DYNAMIC_AGGREGATION_POST_PROCESSOR_PREFETCHER) as the name of the prefetcher to retrieve. If you define your own prefetcher you need to name it correctly :

IPrefetcher<?> namedPrefetcher = IPrefetcher.name(ABasicPostProcessor.BASIC_POST_PROCESSOR_PREFETCHER, prefetcher);

  • APS-11227 The driverClassName argument for the JDBCSource constructor has been moved to SimpleConnectionSupplier. It is now only required when no IConnectionSupplier is provided.

    APS-10692 The appendQueueSize attribute is no longer part of the AJDBCSource, but rather the JDBCTopic.

    Read the migration notes for more details.

  • APS-10998 The ICubeFilterBuilder.includeMembers method accepting wildcards for members path name (eg. List<?>) has been moved to includeMembersWithConditions. The new ICubeFilterBuilder.includeMembers now only accepts List<String> The ICubeFilterBuilder.excludeMembers method accepting wildcards for members path name (eg. List<?>) has been moved to excludeMembersWithConditions. The new ICubeFilterBuilder.excludeMembers now only accepts List<String>

  • PIVOT-3670 When a listener listening on several branches (e.g. pivot) is attached on a non empty datastore, it will now catch up with the current state in a maintenance operation (i.e. blocking all transactions until it catches up).

    For listeners listening on a single branch, the catch-up mechanism was improved. As previously, it does not block transactions, but its memory footprint was improved. The property qfs.selection.listener.catchUpMaxTime defines the maximum time (in s) in which the listener will try to catch-up while not blocking transactions: when it is exceeded, the listener will run a maintenance operation to finish catching up. The default value is 3min, and setting a non positive duration sets an infinite maximum time.

  • APS-9927 The ActivePivot Sandbox now has a 4-Level Hierarchy (the 'LegalEntity' and 'BusinessUnit' levels have been added on top of the already existing Booking hierarchy)

  • APS-10705 The sandbox project properly configures its firewall to support special chars in file names for the Content Server.

  • APS-11634 StoreUtils.getPartitionSizes() now returns -1 as a value for dropped or non existing partitions

  • APS-10848 The StreamingMonitorConfig configuration dependency to IUserDetailsService is no longer mandatory.

  • PIVOT-3866 IEpochHistory.markHead() and IEpochHistory.markHead(String branch) now return the IEpoch that was marked instead of void.

  • APS-11427 The metadata fetching methods of the JDBCSource have been modified to not rely on a non-standard behavior of JDBC Driver implementation of ResultSetMetaData.getColumnName(), but instead use the standardized ResultSetMetaData.getColumnLabel()

  • APS-11423 Improve the logs for notifications generated by real-time queries.

  • PIVOT-3906 Rewrite the tasks performing Range-Sharing to avoid potential assertion failures.

  • PIVOT-3943 Perform the hierarchy contributions (during a Pivot commit) in parallel; this can speed up commit times in configurations with many hierarchies.

  • {@jira: APS-11920, PIVOT-3005} ActivePivot queries now return "NaN" for Double.NaN values instead of "\uFFFD" (�).

Deprecated

  • PIVOT-3670 An ISelectionListener does not record changes anymore, so it has no reason to be cleared anymore. The functions cleared() and isClearable() will be removed in 5.9.

Removed

  • PIVOT-3834 The changes introduced in Post Processors V2 in 5.7.3 are now merged into the normal implementation of the post processors. Read the migration notes for more details. The following post processors V2 have been removed :

    • ABaseDynamicAggregationPostProcessorV2
    • ABasicPostProcessorV2
    • ADynamicAggregationPostProcessorV2
    • AFilteringPostProcessorV2

  • PIVOT-3833 The following methods and fields have been removed :

    • The MDXQuery.QUERY_NAME_PROPERTY constant has been removed
    • IMdxStream.viewUpdated has been removed => use IMdxStream.publishCellDtos
    • ContextUtils.ensureContext has been removed => use IContext.apply(IContextSnapshot)
    • ContextUtils.restoreContext has been removed => use IContext.apply(IContextSnapshot)
    • In JDBCSource the constructors without an appendBatchSize has been removed
    • In ClientPool the constructors with vararg that may lead to undetected type errors => use constructors taking collections

  • The AJDBCSource constructors with no appendBatchSize arguments have been removed.

  • PIVOT-3855: Remove the LINE storage type. The only possible configuration is now StorageType.COLUMN. Read the migration notes to see how to adapt your code.

  • APS-11272 Remove InterruptionUtil#checkInterruption(int) and InterruptionChecker#checkInterruption(int) in favour of PeriodicInterruptionChecker to avoid static variables in the code.

  • APS-11552 Sets.isEmpty is replaced by ISet.isEmpty.

Fixed

  • APS-11552 Speed up the evaluation of CrossJoin on axes with NON EMPTY
  • APS-10889 Detect when native library cannot be loaded because the native layer failed to initialize.
  • APS-11308 Using real-time when new branches are created no longer produces infinite loops.
  • APS-11446 Fix race condition on PointListRetrievalResult additional columns that resulted in an ArrayIndexOutOfBounds on some queries
  • APS-10905 CoPPer measures created with a .join() on an isolated store will now be properly updated during continuous queries when the store is updated.
  • PIVOT-3862 Registering a continuous query with a listener on a datastore selection (such as com.quartetfs.biz.pivot.postprocessing.streams.impl.AStoreStream) no longer triggers an initial view, harmonizing the behavior across all continuous query listeners.
  • APS-11305 PIVOT-3761 Improve CoPPer error messages
  • PIVOT-2924 Correct the MBean on top of ActivePivotContentService to clean the local and root caches.
  • APS-11309 Remove an unnecessary warning when manually stopping a query task.
  • APS-11439 Fix an issue in Copper when using a legacy postprocessor in calculations.
  • APS-11351 Allow the same CoPPer column to be used in a calculation multiple times.
  • APS-10914 Correct query planning combining range sharing and joins from Copper.
  • APS-11438 Fix order of transaction units (add, remove, removeWhere...). The order is now respected.
  • APS-11022 Fix the ContentServiceBranchPermissionManager to allow for full server eraseAndImport instead of having to update subtrees only.
  • APS-11526 Correctly export Monitor during field in DTOs.
  • APS-11547 The FilteringPostProcessorV2 now returns an empty result when no record matches its filtered member(s); the method OrCondition#of has also been removed.
  • APS-11060 Partial provider was not used when the Epoch dimension was disabled.
  • DS-272 Fix deadlock that could appear when submitting records in parallel on the same partition.
  • APS-11469 Allow to set the default member of the Epoch hierarchy like any other hierarchy.
  • APS-11511 Union function used in SubSelect could wrongly return an empty result.
  • APS-11587 Fix the Datastore Rest API, incorrectly failing when finding a reference that was not an expected one
  • APS-11586 Fix the Datastore Rest API, incorrectly mixing the number of fields and the depth of their reference paths.
  • PIVOT-3926 Fix concurrency issue in on-demand licensing
  • PIVOT-3924 Avoid building unnecessary objects when calling Objects.requireNonNull.
  • APS-11634 SchemaPrinter no longer prints discarded partitions as existing with a record size of 0
  • PIVOT-3922 Performance improvements for join made in Copper
  • APS-11619 Fix an occasional NullPointerException raised in the sandbox.

5.7.7

Added

  • PIVOT-3778 Measures that have already been published in Copper are available as regular columns in the following createDatasetFromFacts datasets.
  • PIVOT-3762 New mode in the query engine that sends only notification events for continuous queries. This allows to explore a current result without running the risk of seeing it change and at the same time know when the current view is not up-to-date anymore. See WebSocket documentation to enable it.

Fixed

  • APS-11326 Fix CubeFilter serialization problem related to ICubeFilterIntersectionWithSubCube
  • APS-11280/APS-11274 Fix several issues regarding EmptyAggregateStore and ReplicatedAggregateStore.
  • APS-11283 AVG aggregation function could prevent IVector Objects from being garbage collected.
  • APS-11306 Fix serialization/deserialization in RemoteDatastoreService
  • PIVOT-3808 chunk compression can be disabled (-Dqfs.compression.chunkset.disable=true)

5.7.6

Added

  • APS-11246 Add interruption exception in MembersCaptionSearch and MembersNameSearch.
  • APS-11250 Always provide a record format for ICursor, even for null results.
  • APS-11110 ConditionToJsonNodeHelper now uses JacksonSerializer as ObjectMapper, which let your register your own (de)serializers.
  • APS-11249 Add a method to split the datastore/schema path in a clean way: DatastoreSchemaDescriptionUtil.splitPath. The mirror of DatastoreSchemaDescriptionUtil.createPath which creates a path with the correct separator from an array.

Changed

  • APS-11294 Replace the micro letter by "us" as the unit for time values in micro-seconds.

Fixed

  • APS-11273 fix wrong query results due to chunk compaction & garbage collection.
  • APS-11087 fix issue in distributed architecture when there are more than 64 members in the cluster.
  • PIVOT-3788 Return a null retrieval result when a datastore external request has no result.
  • APS-11278 Fix the number of cores detected on a Linux machine when numactl is used.
  • APS-11298 Add translation files in sandbox for Spanish and German (es-ES and de-DE) so the locale can be changed in ActiveUI without error.
  • APS-11274 APS-11280 Correctly clone empty aggregate stores.

5.7.5

Added

  • APS-11121 Improve the error message in case of incompatible partitionings. Now, the message includes involved store names, the considered reference as well as general suggestions of changes.
  • APS-11242 Improve the error message in case of unknown underlying measure. Now, the message prints only the 10 measure names that are the closest (according to the Levenshtein algorithm) to the the provided name. The Levenshtein distance is now available for use in QfsStrings.
  • APS-11090 Add a IBaseCondition to JSON Node converter, it can be found in ConditionToJsonNodeHelper class.

Changed

Fixed

  • APS-11086 Add additional checks to monitor execution stream to support empty slicer. This fixes a IndexOutOfBoundException in SentinelMdxStream.
  • APS-11092 The warning in ClusterLocalityIdentifierFactory about having post-processors with COPY aggregation function and no slicing distributing field now shows when it is supposed to.
  • APS-11154 AFilteringPostProcessorV2 now extends ABasicPostProcessorV2 and correctly handles analysisLevel property.
  • APS-11154 AFilteringPostProcessorV2 and FilteringPostProcessorV2 are now parametrized with OutputType.
  • PIVOT-3765 fixes an edge case in ChunkedBitmap.equals().
  • PIVOT-3785 Correct QFSBitmap IndexOutOfBoundException.
  • APS-11186 Rewrite condition in remove where to match more easily the drop partitions criteria.
  • APS-11081 Add a cache to cube filters intersections to improve performance when intersecting a lot with the same subcube properties.
  • APS-11212 Allow to reset a branch to its parent even if its forkpoint is a shadow commit.

5.7.4

Added

  • APS-10942 The appendBatchSize in AJDBCSource (indicating how many records are drained from the queue at once) is now configurable in the constructor.

Changed

  • PIVOT-3715 A failure during a transaction in ActivePivot triggers a rollback of the underlying datastore transaction when possible.
  • APS-10990 Added Column#notEqualTo operator
  • PIVOT-3224 Added Columns#shortSum
  • PIVOT-3764 Improve the logic for impact computation in presence of Aggregation Procedures. This avoids to always trigger a full-refresh.
  • APS-11068 Improve the exception in case any operation performed on a store during a commit fails. It always provides the store name, operation type and modified store partition.

Fixed

  • PIVOT-3744 CoPPer can now drillup on analysis hierarchy levels
  • PIVOT-3760 Correct replanning of aggregation procedure dependencies
  • APS-10971 Ensures that retrievals computing the logic of Aggregation Procedures returns a true retrieval result. This fixes IllegalStateException complaining about "an unplanned retrieval (no primitive result)".
  • APS-10846 Remove the incorrect check in memory monitoring report, causing UnsupportedOperationException.
  • PIVOT-3663 Releasing a branch in a data cube does not throw an exception during the planning of a query in the query cube
  • APS-11032 Use an appropriate object to merge contribution deltas in query cubes. This fixes an UnsupportedOperationException.
  • APS-11063 fix translation with NO_REPLICATION
  • PIVOT-3768 Support calling #next() without previous calls to #hasNext() for QfsBitmap iterators.
  • APS-11075 Fix a bug when cloning a partitioning description with modulo on several fields.
  • APS-10912 Checks for empty cube are now done on all the head versions and not only the last one.

5.7.3

Changed

  • PIVOT-3720 Reduce the impact (memory consumption is reduced, local results are more efficiently translated <=> queries should be quicker) of Data Replication in polymorphic distribution
  • APS-10729 - Fetch & Branch : Update the Fetch and ITransactionManager classes to handle data fetch into specific branch
  • PIVOT-3714 Datastore fields used by levels are automatically converted to the default value when inserted in the datastore. This prevents errors in ActivePivot at commit time, when creating new Mdx Members.
  • PIVOT-3741 Added AFilteringPostProcessorV2 and its concrete class FilteringPostProcessorV2, which are better implementations than AFilteringPostProcessor and FilteringPostProcessorWithEvaluator. The old ones will be removed in 5.8.0 and the new ones will take their name.

Fixed

  • APS-10706 fix Aggregates Cache Statistics; the MBean also displays pivot version and branch name for the statistics.
  • PIVOT-3676 The DistinctCountFunction now has an aggregated datatype of LONG instead of the same type as its inputType.
  • PIVOT-3659 CoPPer now supports using aliases for fields in the base store
  • PIVOT-3734 Fix incorrect exception raised by the sandbox on single-core machines due to ConstantPartitioning.
  • APS-10875 Improve SinglePartitioning performance by removing a TIntSet of partition ids which causes thread contention
  • APS-10843 Fix faulty nano-to-milli conversion in CSV source logging
  • APS-10685 Fix NullPointerException in AHibernateContentService by allowing nested read actions
  • APS-10920 Fix issue when joining with a dataset creates more than 1 analysis hierarchy
  • APS-10897 Correct the handler of the Lookup Post-Processor to refresh a query for deleted point values.
  • APS-10897 Correct the computation of changes for effective parameters. Previously, deleted point values with future expiration were returned as deleted multiple times.
  • APS-10757 Improve ClientPool constructors to avoid confusion between properties and extensions. This correct usages of client pools to correctly pass the properties.
  • APS-10856 Fix an ArrayOutOfBoundsException when a new partition is created in the middle of a query planning
  • APS-10630 Queries on a query cube with an empty data cube no longer throw a NullPointerException.
  • APS-10929 Fix NullPointerException which sometimes happened when conflating transactions.
  • APS-10880 Fix NullPointerException when expanding query scope into a location associated with no facts.
  • APS-10813 Fix legacy post processor visibility
  • APS-10881 Analysis hierarchies were only partially supported by basic and dynamic post processors. The analysis levels were removed on the location but not the cube filter. As fixing this behavior might introduce breaking changes in how the post processors work we introduced three new classes that must be extended instead of the previous version if the post processor uses the analysisLevels property:
    • ABasicPostProcessorV2
    • ABaseDynamicAggregationPostProcessorV2
    • ADynamicAggregationPostProcessorV2 These classes will be merged back into there classic implementation in 5.8, potentially introducing a breaking change for subclasses that overwrite the compute function or the prefetchers Post Processors that try to use analysisLevels without handling it correctly will now log a warning during the init. You can learn how to correctly implement AAdvancedPostProcessor with analysis levels in this confluence article.
  • PIVOT-3737 Correct ConcurrentModificationException when using aggregation procedures.
  • PIVOT-3738 Correctly consider partitionings of underlying retrievals of aggregation procedures.

5.7.2

Added

  • APS-10427 Hadoop Configuration can be given to the parquet parser
  • APS-10618 Support for user defined datastore schema separators with the system property activeviam.datastore.schema.separator

Changed

  • APS-10627 The Epoch dimension is now visible by default when enabled.
  • PIVOT-3685 Detects usage of analysis hierarchies within the cube filters or the security subcubes.
  • APS-10691 Added QFSPools.getMixedWorkloadThreadCount() to allow knowing how many threads can be used at max for a transaction (or for a query, since transaction and query pools have the same size).
  • SENTINEL-624 Improve performance of the Repository during insertion of new point values and while computing differential updates involving a lot of deleted point values.
  • APS-10708 Fail when registering a workflow relying on an undefined Activiti workflow process.
  • PIVOT-3707 Cube filters are now available to AggregationProcedures from #createContext through the ActivePivot context.
  • APS-10615 It is now possible to exclude a specific field from the selection when adding all fields or all reachable fields of the base store or a reference (using .except()), rather than always excluding all fields with the same name (which is what .excluding() does)
  • PIVOT-3675 Default implementations exist to configure branch and store permissions. They are designed for development, not apply any security.
    See FullAccessBranchPermissionsManagerConfig and NoSecurityDatastoreServiceConfig.
  • APS-10711 Formatting for thousand/million rescaling and multisection formatting are now supported.
  • PIVOT-3224 Join between datasets now uses Analysis Aggregation Procedures.

Removed

  • The following classes have been removed from the sandbox:
    • ForexPostProcessor
    • ForexHandler
    • ForexStream
    • ReferenceCurrency
    • ReferenceCurrencyTranslator.
  • The measure pnl.FOREX is defined in copper. The currency can now be chosen via a new slicing hierarchy named TargetCurrency (automatically created by copper).

Fixed

  • APS-10699 Avoid exception when new branch is created without subcube
  • PIVOT-3699 A multimensional single element set in the MDX WHERE clause now works (like a tuple).
  • APS-10740 Correctly compute modulo on multiple fields, not to double the number of produced values.
  • APS-10766 unmark epochs that are released because they corresponded to shadow commits
  • PIVOT-3580 Implementation of a filtered list, FilteredList, to manage different measure description (sub)types for a single parent list.
  • PIVOT-3673 Fix NullPointerException when an ExpandLocationResult overflows during large transactions
  • APS-10724 Correct formatting of alert messages in the sandbox project.

5.7.1

Added

  • APS-6362 The Aggregate Store size was added in JMX for both the Bitmap and Leaf aggregate providers.
  • PIVOT-3548 The Content Server UI now has a title.

Changed

  • APS-10622 APS-10624 PIVOT-3608 Fluent Builder API was improved (slicing, factless, memberProperty).
  • SENTINEL-622 Improve the parameter store partitioning. This store is now partitioned by parameter name and ActivePivot name for which it defined, so any lookup for a given parameter within a given cube hits a given partition. Then, the partitioning distributes the point values on a multi-modulo of the erasure and the wildcard erasure using the number of cores of the machine.
  • APS-10426 Null values in parquet source are now converted to default field value.

Fixed

  • APS-10614 Correct query planning for query cubes with respect to analysis hierarchies.
  • APS-10184 Measures from different applications and with the same aggregation id (even if they have different names) are no longer merged in the the query cube, giving wrong results
  • APS-10591 Correct merging of retrieval results not containing any aggregated measures.
  • PIVOT-3537 Correctly update permissions of folders from the Content Server UI.
  • APS-10580 Prevent from automatically adding levels to analysis hierarchies declared using an xml file, and add check that analysis hierarchies do not have levels.
  • APS-10528 Fix MemberNamePathStyle context value in intersected context
  • APS-10545 Fix issue when a data cube leaves its cluster while planning a query
  • PIVOT-3224 (CoPPer) Use actual cube name in tests

5.7.0

Added

  • PIVOT-3612 Drop dedicated Angular UI of ActiveMonitor. ActiveUI becomes the only UI for this component, providing all features of the old UI. See the migration notes to see how to update your existing post-processors.
  • PIVOT-3609 Support prepared statements in fetch-only JDBC Topics
  • DS-293 Add API to create CompiledGetByKey query and run them, either on a particular version (runInVersion) or on the current transaction (runInTransaction). See the documentation for more details and examples.
  • PIVOT-3655 The Content Server / Service supports multiple owners and readers for each entry.
  • APS-10493 Create AFilteringPostProcessor to increase flexibility when implementing a filtering post-processor.

Changed

  • PIVOT-3584 Replace IPartitionedPostProcessor#getPartitioningLevels(...) called IPostProcessor#setPartitioningLevels(..) to configure the levels on which a post-processor is partitioned.
  • PIVOT-3647 Revamped Spring configuration classes to either define beans or import other classes.
  • PIVOT-2879 Do not notify Content Server listener when a non-readable child entry is updated.
  • APS-9893 Keep an aggregates cache for the latest version of each branch. The branches on which aggregates will be cached can be configured.
  • PIVOT-3371 FileSystemCSVTopicFactory now implements AutoCloseable. Usage in the sandbox has been modified (Spring Bean that auto-closes instead of the callback in the csv source close method)
  • PIVOT-3625 Update dependencies to their latest versions, notably Spring (→ 5.0.6) and Spring-security (→ 5.0.5).

Fixed

  • PIVOT-3626 Correctly publish retrieval additional measure across threads.
  • APS-10465 Fix native measure computation in distributed setup when one of the data cubes is empty
  • DS-294 Fix "scan in transaction" methods for chunks having a single value
  • PIVOT-3641 fix incorrect subcube compilation when using condition on analysis level
  • APS-10485 fix realtime drillthrough when member contains "[" and "]"
← What's new in ActivePivotMigration notes →