Changelog

The format is based on Keep a Changelog and this project adheres to Romantic Versioning.

5.7.9

2019-05-23

Added

  • APS-10824 Allow an IActivePivotManagerDescription to have a single distributed cube with fluent builders.

Changed

  • APS-11427 The metadata fetching methods of the JDBCSource have been modified to not rely on a non-standard behavior of JDBC Driver implementation of ResultSetMetaData.getColumnName(), but instead use the standardized ResultSetMetaData.getColumnLabel()
  • PIVOT-3906 Rewrite the tasks performing Range-Sharing to avoid potential assertion failures.
  • PIVOT-3920 Reuse existing record format for the result if available.
  • PIVOT-3919 Use a code fast-path when splitting paths of one char.
  • PIVOT-3818 Improve the registration of MDX streams to consume less CPU resource.
  • PIVOT-3943 Perform the hierarchy contributions (during a Pivot commit) in parallel; this can speed up commit times in configurations with many hierarchies.
  • APS-11601 Update the READMEs of the Sandbox project, to detail and explain how to customize the configuration of the packaged UIs.
  • PIVOT-3974 Upgrade Sandbox ActiveUI version to 4.2.11

Fixed

  • APS-11511 Union function used in SubSelect could wrongly return an empty result.
  • APS-11586 Fix the Datastore Rest API, incorrectly mixing the number of fields and the depth of their reference paths.
  • APS-11587 Fix the Datastore Rest API, incorrectly failing when finding a reference that was not an expected one
  • PIVOT-3926 Fix concurrency issue in on-demand licensing
  • APS-11619 Fix an occasional NullPointerException raised in the sandbox.
  • PIVOT-3942 Correctly use IPartitionedPostProcessor#DEFAULT_PARTITIONING_ON_RANGE_LEVELS as the default value auto range partitioning for Post-Processors.
  • PIVOT-3540 Correct re-initialization of the MdxRegistration state when creating new Listeners.
  • APS-11657 Fix a NullPointerException in the CSVExportDrillthroughExecutor which occured when one of the records had a null field.
  • APS-11563 Include all measures NoOp retrievals are retrieving into their result.
  • PIVOT-3960 Fix a bug in QFSBitmap#or that could result in extra members appearing in a result.
  • PIVOT-3938 Fix two bugs in the ReorderListener: 1) The delegate listener could miss an add, delete, or remove. 2) Could erronously send a transactionRolledback to the delegate listener.
  • APS-11728 Fix a bug in QFSBitmap#or that could produce wrong results when using filters in a query.
  • PIVOT-3565 Correctly retrieve the timestamp of a file last modification.

5.7.8

2019-03-08

Added

  • APS-10710 Provide a default implementation for the Parameter store configuration. If one is explicitly provided by developers, the default implementation is not used.
  • APS-10733 Add logs when daemons must wait for a stable distribution to start.
  • APS-11149 Add possibility to define properties to pre-aggregated measures in fluent builders
  • APS-11272 Add a new post-processor FilteringPostProcessorOnTheFlyPattern as an alternative to the MDX function MemberCaptionSearch. It allows to search for a given pattern in member captions for a given hierarchy. It will be used by ActiveUI when available.
  • APS-11323 Provide a context value translator for every core Context Value.

Changed

  • PIVOT-3837 Migrate the sandbox project to ActiveUI application.
  • APS-10705 The sandbox project properly configures its firewall to support special chars in file names for the Content Server.

Deprecated

  • APS-11552 Sets.isEmpty is replaced by ISet.isEmpty

Fixed

  • APS-11552 Speed up the evaluation of CrossJoin on axes with NON EMPTY
  • APS-10889 Correctly detect that the Native Layer itself could not be initialized.
  • APS-10505 Remove undesired log when initializing the formatter of a measure that is not granted.
  • APS-10914 Correct query planning combining range sharing and joins from Copper.
  • APS-11439 Fix an issue in Copper when using a legacy postprocessor in calculations.
  • APS-11351 Allow the same CoPPer column to be used in a calculation multiple times.
  • APS-11022 Fix the ContentServiceBranchPermissionManager to allow for full server eraseAndImport instead of having to update subtrees only.
  • APS-11547 The FilteringPostProcessorV2 now returns an empty result when no record matches its filtered member(s).
  • APS-11526 Correctly export Monitor during field in DTOs.
  • PIVOT-2924 Correct the MBean on top of ActivePivotContentService to clean the local and root caches.
  • APS-11060 Partial provider was not used when the Epoch dimension was disabled.
  • DS-272 Fix deadlock that could appear when submitting records in parallel on the same partition.
  • APS-11469 Allow to set the default member of the Epoch hierarchy like any other hierarchy.
  • APS-11438 Fix order of transaction units (add, remove, removeWhere...). The order is now respected.
  • PIVOT-3862 Registering a continuous query with a listener on a datastore selection (such as com.quartetfs.biz.pivot.postprocessing.streams.impl.AStoreStream) no longer triggers an initial view, harmonizing the behavior across all continuous query listeners.
  • PIVOT-3924 Avoid building unnecessary objects when calling Objects.requireNonNull.
  • APS-11634 SchemaPrinter no longer prints discared partitions as existing with a record size of 0

Security

  • APS-11145 Migrate the sandbox project to ContentServer UI application, fixing a possible CSRF and XSS issue with the old UI.

5.7.7

Added

  • PIVOT-3778 Measures that have already been published in Copper are available as regular columns in the following createDatasetFromFacts datasets.
  • PIVOT-3762 New mode in the query engine that sends only notification events for continuous queries. This allows to explore a current result without running the risk of seeing it change and at the same time know when the current view is not up-to-date anymore. See WebSocket documentation to enable it.

Fixed

  • APS-11326 Fix CubeFilter serialization problem related to ICubeFilterIntersectionWithSubCube
  • APS-11280/APS-11274 Fix several issues regarding EmptyAggregateStore and ReplicatedAggregateStore.
  • APS-11283 AVG aggregation function could prevent IVector Objects from being garbage collected.
  • APS-11306 Fix serialization/deserialization in RemoteDatastoreService
  • PIVOT-3808 chunk compression can be disabled (-Dqfs.compression.chunkset.disable=true)

5.7.6

Added

  • APS-11246 Add interruption exception in MembersCaptionSearch and MembersNameSearch.
  • APS-11250 Always provide a record format for ICursor, even for null results.
  • APS-11110 ConditionToJsonNodeHelper now uses JacksonSerializer as ObjectMapper, which let your register your own (de)serializers.
  • APS-11249 Add a method to split the datastore/schema path in a clean way: DatastoreSchemaDescriptionUtil.splitPath. The mirror of DatastoreSchemaDescriptionUtil.createPath which creates a path with the correct separator from an array.

Changed

  • APS-11294 Replace the micro letter by "us" as the unit for time values in micro-seconds.

Fixed

  • APS-11273 fix wrong query results due to chunk compaction & garbage collection.
  • APS-11087 fix issue in distributed architecture when there are more than 64 members in the cluster.
  • PIVOT-3788 Return a null retrieval result when a datastore external request has no result.
  • APS-11278 Fix the number of cores detected on a Linux machine when numactl is used.
  • APS-11298 Add translation files in sandbox for Spanish and German (es-ES and de-DE) so the locale can be changed in ActiveUI without error.
  • APS-11274 APS-11280 Correctly clone empty aggregate stores.

5.7.5

Added

  • APS-11121 Improve the error message in case of incompatible partitionings. Now, the message includes involved store names, the considered reference as well as general suggestions of changes.
  • APS-11242 Improve the error message in case of unknown underlying measure. Now, the message prints only the 10 measure names that are the closest (according to the Levenshtein algorithm) to the the provided name. The Levenshtein distance is now available for use in QfsStrings.
  • APS-11090 Add a IBaseCondition to JSON Node converter, it can be found in ConditionToJsonNodeHelper class.

Changed

Fixed

  • APS-11086 Add additional checks to monitor execution stream to support empty slicer. This fixes a IndexOutOfBoundException in SentinelMdxStream.
  • APS-11092 The warning in ClusterLocalityIdentifierFactory about having post-processors with COPY aggregation function and no slicing distributing field now shows when it is supposed to.
  • APS-11154 AFilteringPostProcessorV2 now extends ABasicPostProcessorV2 and correctly handles analysisLevel property.
  • APS-11154 AFilteringPostProcessorV2 and FilteringPostProcessorV2 are now parametrized with OutputType.
  • PIVOT-3765 fixes an edge case in ChunkedBitmap.equals().
  • PIVOT-3785 Correct QFSBitmap IndexOutOfBoundException.
  • APS-11186 Rewrite condition in remove where to match more easily the drop partitions criteria.
  • APS-11081 Add a cache to cube filters intersections to improve performance when intersecting a lot with the same subcube properties.
  • APS-11212 Allow to reset a branch to its parent even if its forkpoint is a shadow commit.

5.7.4

Added

  • APS-10942 The appendBatchSize in AJDBCSource (indicating how many records are drained from the queue at once) is now configurable in the constructor.

Changed

  • PIVOT-3715 A failure during a transaction in ActivePivot triggers a rollback of the underlying datastore transaction when possible.
  • APS-10990 Added Column#notEqualTo operator
  • PIVOT-3224 Added Columns#shortSum
  • PIVOT-3764 Improve the logic for impact computation in presence of Aggregation Procedures. This avoids to always trigger a full-refresh.
  • APS-11068 Improve the exception in case any operation performed on a store during a commit fails. It always provides the store name, operation type and modified store partition.

Fixed

  • PIVOT-3744 CoPPer can now drillup on analysis hierarchy levels
  • PIVOT-3760 Correct replanning of aggregation procedure dependencies
  • APS-10971 Ensures that retrievals computing the logic of Aggregation Procedures returns a true retrieval result. This fixes IllegalStateException complaining about "an unplanned retrieval (no primitive result)".
  • APS-10846 Remove the incorrect check in memory monitoring report, causing UnsupportedOperationException.
  • PIVOT-3663 Releasing a branch in a data cube does not throw an exception during the planning of a query in the query cube
  • APS-11032 Use an appropriate object to merge contribution deltas in query cubes. This fixes an UnsupportedOperationException.
  • APS-11063 fix translation with NO_REPLICATION
  • PIVOT-3768 Support calling #next() without previous calls to #hasNext() for QfsBitmap iterators.
  • APS-11075 Fix a bug when cloning a partitioning description with modulo on several fields.
  • APS-10912 Checks for empty cube are now done on all the head versions and not only the last one.

5.7.3

Changed

  • PIVOT-3720 Reduce the impact (memory consumption is reduced, local results are more efficiently translated <=> queries should be quicker) of Data Replication in polymorphic distribution
  • APS-10729 - Fetch & Branch : Update the Fetch and ITransactionManager classes to handle data fetch into specific branch
  • PIVOT-3714 Datastore fields used by levels are automatically converted to the default value when inserted in the datastore. This prevents errors in ActivePivot at commit time, when creating new Mdx Members.
  • PIVOT-3741 Added AFilteringPostProcessorV2 and its concrete class FilteringPostProcessorV2, which are better implementations than AFilteringPostProcessor and FilteringPostProcessorWithEvaluator. The old ones will be removed in 5.8.0 and the new ones will take their name.

Fixed

  • APS-10706 fix Aggregates Cache Statistics; the MBean also displays pivot version and branch name for the statistics.
  • PIVOT-3676 The DistinctCountFunction now has an aggregated datatype of LONG instead of the same type as its inputType.
  • PIVOT-3659 CoPPer now supports using aliases for fields in the base store
  • PIVOT-3734 Fix incorrect exception raised by the sandbox on single-core machines due to ConstantPartitioning.
  • APS-10875 Improve SinglePartitioning performance by removing a TIntSet of partion ids which causes thread contension
  • APS-10843 Fix faulty nano-to-milli conversion in CSV source logging
  • APS-10685 Fix NullPointerException in AHibernateContentService by allowing nested read actions
  • APS-10920 Fix issue when joining with a dataset creates more than 1 analysis hierarchy
  • APS-10897 Correct the handler of the Lookup Post-Processor to refresh a query for deleted point values.
  • APS-10897 Correct the computation of changes for effective parameters. Previously, deleted point values with future expiration were returned as deleted multiple times.
  • APS-10757 Improve ClientPool constructors to avoid confusion between properties and extensions. This correct usages of client pools to correctly pass the properties.
  • APS-10856 Fix an ArrayOutOfBoundsException when a new partition is created in the middle of a query planning
  • APS-10630 Queries on a query cube with an empty data cube no longer throw a NullPointerException.
  • APS-10929 Fix NullPointerException which sometimes happened when conflating transactions.
  • APS-10880 Fix NullPointerException when expanding query scope into a location associated with no facts.
  • APS-10813 Fix legacy post processor visibility
  • APS-10881 Analysis hierarchies were only partially supported by basic and dynamic post processors. The analysis levels were removed on the location but not the cube filter. As fixing this behavior might introduce breaking changes in how the post processors work we introduced three new classes that must be extended instead of the previous version if the post processor uses the analysisLevels property:
    • ABasicPostProcessorV2
    • ABaseDynamicAggregationPostProcessorV2
    • ADynamicAggregationPostProcessorV2 These classes will be merged back into there classic implementation in 5.8, potentially introducing a breaking change for subclasses that overwrite the compute function or the prefetchers Post Processors that try to use analysisLevels without handling it correctly will now log a warning during the init. You can learn how to correctly implement AAdvancedPostProcessor with analysis levels in this confluence article.
  • PIVOT-3737 Correct ConcurrentModificationException when using aggregation procedures.
  • PIVOT-3738 Correctly consider partitionings of underlying retrievals of aggregation procedures.

5.7.2

Added

  • APS-10427 Hadoop Configuration can be given to the parquet parser
  • APS-10618 Support for user defined datastore schema separators with the system property activeviam.datastore.schema.separator

Changed

  • APS-10627 The Epoch dimension is now visible by default when enabled.
  • PIVOT-3685 Detects usage of analysis hierarchies within the cube filters or the security subcubes.
  • APS-10691 Added QFSPools.getMixedWorkloadThreadCount() to allow knowing how many threads can be used at max for a transaction (or for a query, since transaction and query pools have the same size).
  • SENTINEL-624 Improve performance of the Repository during insertion of new point values and while computing differential updates involving a lot of deleted point values.
  • APS-10708 Fail when registering a workflow relying on an undefined Activiti workflow process.
  • PIVOT-3707 Cube filters are now available to AggregationProcedures from #createContext through the ActivePivot context.
  • APS-10615 It is now possible to exclude a specific field from the selection when adding all fields or all reachable fields of the base store or a reference (using .except()), rather than always excluding all fields with the same name (which is what .excluding() does)
  • PIVOT-3675 Default implementations exist to configure branch and store permissions. They are designed for development, not apply any security.
    See FullAccessBranchPermissionsManagerConfig and NoSecurityDatastoreServiceConfig.
  • APS-10711 Formatting for thousand/million rescaling and multisection formatting are now supported.
  • PIVOT-3224 Join between datasets now uses Analysis Aggregation Procedures.

Removed

  • The following classes have been removed from the sandbox:
    • ForexPostProcessor
    • ForexHandler
    • ForexStream
    • ReferenceCurrency
    • ReferenceCurrencyTranslator.
  • The measure pnl.FOREX is defined in copper. The currency can now be chosen via a new slicing hierarchy named TargetCurrency (automatically created by copper).

Fixed

  • APS-10699 Avoid exception when new branch is created without subcube
  • PIVOT-3699 A multimensional single element set in the MDX WHERE clause now works (like a tuple).
  • APS-10740 Correctly compute modulo on multiple fields, not to double the number of produced values.
  • APS-10766 unmark epochs that are released because they corresponded to shadow commits
  • PIVOT-3580 Implementation of a filtered list, FilteredList, to manage different measure description (sub)types for a single parent list.
  • PIVOT-3673 Fix NullPointerException when an ExpandLocationResult overflows during large transactions
  • APS-10724 Correct formatting of alert messages in the sandbox project.

5.7.1

Added

  • APS-6362 The Aggregate Store size was added in JMX for both the Bitmap and Leaf aggregate providers.
  • PIVOT-3548 The Content Server UI now has a title.

Changed

  • APS-10622 APS-10624 PIVOT-3608 Fluent Builder API was improved (slicing, factless, memberProperty).
  • SENTINEL-622 Improve the parameter store partitioning. This store is now partitioned by parameter name and ActivePivot name for which it defined, so any lookup for a given parameter within a given cube hits a given partition. Then, the partitioning distributes the point values on a multi-modulo of the erasure and the wildcard erasure using the number of cores of the machine.
  • APS-10426 Null values in parquet source are now converted to default field value.

Fixed

  • APS-10614 Correct query planning for query cubes with respect to analysis hierarchies.
  • APS-10184 Measures from different applications and with the same aggregation id (even if they have different names) are no longer merged in the the query cube, giving wrong results
  • APS-10591 Correct merging of retrieval results not containing any aggregated measures.
  • PIVOT-3537 Correctly update permissions of folders from the Content Server UI.
  • APS-10580 Prevent from automatically adding levels to analysis hierarchies declared using an xml file, and add check that analysis hierarchies do not have levels.
  • APS-10528 Fix MemberNamePathStyle context value in intersected context
  • APS-10545 Fix issue when a data cube leaves its cluster while planning a query
  • PIVOT-3224 (CoPPer) Use actual cube name in tests

5.7.0

Added

  • PIVOT-3612 Drop dedicated Angular UI of ActiveMonitor. ActiveUI becomes the only UI for this component, providing all features of the old UI. See the migration notes to see how to update your existing post-processors.
  • PIVOT-3609 Support prepared statements in fetch-only JDBC Topics
  • DS-293 Add API to create CompiledGetByKey query and run them, either on a particular version (runInVersion) or on the current transaction (runInTransaction). See the documentation for more details and examples.
  • PIVOT-3655 The Content Server / Service supports multiple owners and readers for each entry.
  • APS-10493 Create AFilteringPostProcessor to increase flexibility when implementing a filtering post-processor.

Changed

  • PIVOT-3584 Replace IPartitionedPostProcessor#getPartitioningLevels(...) called IPostProcessor#setPartitioningLevels(..) to configure the levels on which a post-processor is partitioned.
  • PIVOT-3647 Revamped Spring configuration classes to either define beans or import other classes.
  • PIVOT-2879 Do not notify Content Server listener when a non-readable child entry is updated.
  • APS-9893 Keep an aggregates cache for the latest version of each branch. The branches on which aggregates will be cached can be configured.
  • PIVOT-3371 FileSystemCSVTopicFactory now implements AutoCloseable. Usage in the sandbox has been modified (Spring Bean that auto-closes instead of the callback in the csv source close method)
  • PIVOT-3625 Update dependencies to their latest versions, notably Spring (→ 5.0.6) and Spring-security (→ 5.0.5).

Fixed

  • PIVOT-3626 Correctly publish retrieval additional measure across threads.
  • APS-10465 Fix native measure computation in distributed setup when one of the data cubes is empty
  • DS-294 Fix "scan in transaction" methods for chunks having a single value
  • PIVOT-3641 fix incorrect subcube compilation when using condition on analysis level
  • APS-10485 fix realtime drillthrough when member contains "[" and "]"
Operation description →