Changelog
For user-facing changes, refer to the What's New? page.
The format is based on Keep a Changelog and this project adheres to Romantic Versioning.
5.10.6
2021-11-19
Added
- PIVOT-5037 The Copper tester now supports
hasValues()
in every case and a new methodwithDelta()
to set the allowed delta between the values given by the user inhasValues()
and the actual values returned by the measures. - PIVOT-5118 Add a toString() to
CloudCsvDataProvider()
for better error message.
Deprecated
- PIVOT-5117 Deprecate
TransactionWrapper
. It will be removed in ActivePivot 5.11.0.
Fixed
- PIVOT-4800 Enhance query plan timing reports. This particularly fixes two man issues:
JITPrimitiveAggregatesRetrieval
Partitioned retrievals for non-existing partitions used to display the largest execution time of their siblings. Now, only partitioned retrievals that performed some work at least and whose result is empty will be reported in the timing section, along with the other retrievals with result- In some cases, for a group of related retrievals (siblings), timing were not printed if one of the sibling has no result. This edge case has been fixed
- PIVOT-5038 Fixed errors in usage counting caused by decrementing the usage too many times.
- PIVOT-5050 Fix distributed post-processors initialization. This issue happened when chaining at least three post-processors where the bottom one in the chain is not initialized and caused missing dependencies propagation in some cases.
- PIVOT-5074 Fix
SubCubeProperties#removeRestriction(dimension, hierarchy)
. The issue happened in distributed polymorphic setup where the same dimension from two data nodes includes different hierarchies and one of the dimension hierarchies cardinality is greater than 1. - PIVOT-5097 Fix
NullPointerException
when registering aGetAggregatesQuery
build using a location Collections not supportingnull
elements. - PIVOT-5119 Fix rollback process of store transactions.
- PIVOT-5128 Fix a memory leak that could happen upon query failure/cancellation if an
AggregatesCache
was used.
5.10.5
2021-10-19
Added
- PIVOT-4941 Added two methods in
IPointValueDao
to permanently delete Parameters in ActiveMonitor. - PIVOT-4973 Added a warning to
IReadableDatastore#getMostRecentVersion
, stating that#getHead
should be used instead. - PIVOT-4913 The JDBC Source will now log the exceptions that make its workers exit the parallel processing of records.
- PIVOT-5069 Add the currently written record in case of type error in
IRecordFormat#newRecord()
.
Changed
- PIVOT-5024 Use
ChunkedIntegerArray
instead ofIntegerArray
inAMultiVersionUniqueRecordIndex
so that we can now put more than 300M records per partition. - PIVOT-5048 Upgrade Junit version to 5.8.
Removed
- PIVOT-5011 Removed the provided strings from exception messages in
RestContentServer
andJsonQueryService
. - PIVOT-5084 Removed an invalid warning that was triggered when building Copper Hierarchies backed by a store. Said warning was stating these hierarchies shouldn't be factless.
Fixed
- PIVOT-4951 Fix an edge case issue with
IChunkInteger
sparse compression. - PIVOT-4972 Fix branches not visible sometimes with continuous queries. The
ISecurityDetails
was not always propagated when registering queries. - PIVOT-4998 Reduce time spent at startup optimizing Copper measures
- PIVOT-5010 Properly set the level type of a slicing Copper hierarchy.
- PIVOT-5016 Fix
ArrayIndexOutOfBoundsException
in theSetExpander
of the Mdx engine. - PIVOT-5038 Fix measure dependencies being improperly attributed in the
PlanningGraph
. - PIVOT-5062 Fix
NullPointerException
inPointListRetrievalResult
in the case of a polymorphic distributed setup. - PIVOT-5068 Fix
NullPointerException
in the AggregatesResultCopier used by postprocessors.
5.10.4
2021-09-03
Added
- PIVOT-4733 Allow to override measure aggregator xmla property with
CopperMeasure#withXmlaAggregator(int)
. - PIVOT-4824 Include epochId of the max and latest continuous query statistics.
Fixed
- PIVOT-4866 Fix
NullPointerException
inFilterTranslator
. The issue could happen when translating a wildcard, e.g.null
, to the data cube. - PIVOT-4881 Fix an issue where the resizing of
IntegerArray
andLongArray
generated non-transient chunks regardless of itsisTransient
parameter. - PIVOT-4923 Fix hiding subtotals issue that could not hide all the measures.
- PIVOT-4935 Fix NPE when accessing an uninitialized datastore via MBean.
- PIVOT-4964 Fix an
ArrayIndexOutOfBoundException
happening when compressing chunks with freed rows asFrequentValueChunk
. - PIVOT-4944 Fix the cleaning method of the prefetch scope location when executing more than one aggregation procedure in a single
PrimitiveAnalysisAggregationRetrieval
. - PIVOT-4946 Fix an issue with Copper
storeLookup
that could fail to find its underlying measures, in the case where these measures were previously re-optimized. - PIVOT-4949 Fix thread safety for
SharedDelegateDictionary#mapFromCell
. - PIVOT-4956 Fix
GenericVectorBinding#applyAs..
operations for non-commutative operators. - PIVOT-4958 Fix the update of the ContentServer entries for branches containing characters that are illegal for FileSystem-specific paths.
- PIVOT-4979 Fix missing fields issue in the analysis aggregation procedures when a cube filter is done on a Copper join.
- PIVOT-4992 Fix dependency issue in content-server-spring.
- PIVOT-5137 Fix
LevelIdentifier.fromDescription
andHierachyIdentifier.fromDescription
. They did not handle correctly the escaping of@
.
5.10.3
2021-06-18
Added
- PIVOT-4804 Alert the user when creating measures names with commas.
- PIVOT-4806 License path is now printed in the log.
- PIVOT-4820 Add constructor to
KeepLastEpochPolicy
usingDuration
.
Changed
- PIVOT-4498 The
ActiveViamPropertyFromSpringConfig.resolve()
bean is now namedActiveViamPropertyFromSpringConfig.SPRING_PROPERTIES_RESOLVER
. - PIVOT-4781
StreamSnapshotService#snapShotStreams(String)
is now compatible withStreamSnapshotService#loadStreams(String)
. Now, both operation ignoreSTOPPED
streams. UseStreamSnapshotService#snapShotAllStreams(String)
to snapshot all streams including the stopped ones. - PIVOT-4789 Provide possible solutions in error message when we failed to initialize the
SaferNative
class inLinuxPlatform
. - PIVOT-4802 Reduce the memory consumption of
ModifiedLocation
, primarily used in cumulated aggregates and in shifted measures. - PIVOT-4808 Report error if the ActivePivotSchema fails to attach.
- PIVOT-4813 Update ActiveMonitor REST API documentation to correctly illustrate how a new monitor can be added.
Deprecated
- PIVOT-4847 Deprecate methods on the
IEpochManager
that will be removed in 5.11.0. One should rather use the same methods onIVersionHistory
.
Fixed
- PIVOT-4735 Fix continuous handler registration issue. The issue happened in distribution when some underlying measures have not yet appeared thus causing a duplication in handler key of the parent measure.
- PIVOT-4780 Fix unnecessary query plan logging when
queryPlanSummary
is enabled. - PIVOT-4807 Fix an issue where using continuous query handler inheritance could lead to the failure of the publication of measures.
- PIVOT-4811 Fix
ClassCastException
inSingleValueFunction
when working with arrays. - PIVOT-4814 Fix an issue where exporting the memory statistic of a chunk containing non-direct vector blocks would throw a NPE.
- PIVOT-4816 Fix an issue occurring when merging two
AggregateStore
s. This caused the merged result to includenull
values instead of the correct value for some measures. - PIVOT-4818 Fix an issue leading to incorrect transitions between
IntegerArrayVersion
. - PIVOT-4819 Fix
UnsupportedOperationException
inActivePivotStatistics
. The issue happened when computing transaction fact statistics on an incompatible type of transaction. - PIVOT-4862 Fix an export error in the
IMemoryStatistic
generation for TombStone classes.
5.10.2
2021-04-30
Added
- PIVOT-4486 Add the possibility to apply query cube filter on the prefetch of Copper analytic functions using the API
CopperAnalyticMeasure#withFilter(AnalyticFilter)
. - PIVOT-4713 Add the method
IDistributedHierarchyManager#waitForAllMessages
allowing to wait for all the recorded messages to be processed.
Changed
- PIVOT-4300 Post-processor dependency graph cycle detection is now enabled by default.
- PIVOT-4780 Change MBeans in
QueryPlanMonitoring
to print execution plan and timing full or as summary, e.g.QueryPlanMonitoring#enableExecutionTimingPrintFull()
orQueryPlanMonitoring#enableExecutionTimingPrintAsSummary()
.
Fixed
- APS-13016 Better handle network errors which were causing issues with license checks.
- PIVOT-4717 Fix UDAF error that could occur when totally emptying the underlying store.
- PIVOT-4725 Some datastore versions were not unmarked after stopping the cubes.
- PIVOT-4728 Measure names with several parentheses were not correctly supported by the Formula PostProcessor.
- PIVOT-4730 Prevent Real-Time statistics errors from crashing continuous queries update. The issue happened when statistics computations tried to fetch retrievals that had been dismissed by the query engine (usually because of a non-existent location).
- PIVOT-4739 Fix publishing of WindowReduceMeasure (description equality).
- PIVOT-4749 Clarify the documentation of the
cube/export/mdx/download
REST service endpoint. - PIVOT-4760 Fix an issue where
IStoreToParquetMapping
would map fields from the datastore to the wrong fields in the parquet schema. - PIVOT-4766 Fix Frequent Chunk Compression bug where nested
FrequentNullDoubleChunk
could be created. - PIVOT-4771 Fix NPE cause by
CopperPostProcessor#withInheritedContinuousQueryHandlers
. The issue happened while inheriting the handlers of a post-processor that hasn't any. - PIVOT-4772 Fix a query issue where wildcards on slicing analysis hierarchies based on isolated store fields would conflict with Copper bucketing hierarchies.
- PIVOT-4775 Fix an issue with
AAdvancedPostProcessor
performing an eager modification of the prefetched cube filter. - PIVOT-4775 Fix an issue with the query engine where some retrievals handing analysis hierarchies, were wrongly declaring their dependencies.
- PIVOT-4791 All datastore versions are now synchronously unmarked when stopping the pivot.
5.10.1
2021-03-26
Added
- PIVOT-4473 Add loggers for CREATE, UPDATE and DROP MDX statements
- PIVOT-4638 Add the
maskAtOrBelow()
method to theCopperMeasureToAggregateAbove
interface, allowing to define levels below which a measure will not return aggregated values when queried. The behavior is the complementary of thedoNotAggregateAbove()
method. - PIVOT-4702 Add support for DATEPART function in MDX with the simple signature DatePart(interval, date).
Changed
- PIVOT-4705 Make
Records#getGlobalDefaultValue
useint
instead ofILiteralType
. - PIVOT-4707
PointIndexedRecordBlock
no longer implementsIChunkSet
. - PIVOT-4707 Renamed
ImmutableRecordList
inImmutableRecordBlock
to match the pattern established in sibling classes.
Deprecated
- PIVOT-4705 Deprecate
SchemaHelper#undictionarize
in favour ofRecords#undictionarize
. - PIVOT-4724 Deprecate
com.qfs.pivot.impl.ActivePivotSchemaExtendedPlugin
in favour ofcom.quartetfs.biz.pivot.impl.ActivePivotSchemaExtendedPlugin
located in the conventional package.
Removed
- PIVOT-4707 Removed
PointIndexedRecordBlock
destroyer methods. - PIVOT-4707 Removed
IAggregateSource
interface. - PIVOT-4707 Removed a constructor in
AWritableFixedSizeRecordBlock
. - PIVOT-4707 Removed
transferFrom(IArrayReader source, int sourceIndex, IWritableByteRecord record)
from classARecordFieldHandler
. - PIVOT-4718 Drop legacy methods from
IByteRecordFormat
.
Fixed
- PIVOT-3790 Fix the partitioning levels of the legacy post-processor
NeighbourValuePostProcessor
. - PIVOT-4177 Prevent two Store Lookups from colliding in the query cache if they ask for different fields on the same store.
- PIVOT-4344 Fix SingleValueFunction on Arrays type.
- PIVOT-4364 Fix Real-Time issues with Copper. This includes the following corrections:
- In addition to the
CommitIsolatedStoreHandler
handler, join measures now include theStoreMeasureHandler
in their handlers' list Copper#newPostProcessor(..)
allows now to propagate the underlying measures handlers usingCopperPostProcessor#withInheritedContinuousQueryHandlers()
- Delegate continuous handler initialization to post-processors
- In addition to the
- PIVOT-4452 Fix an issue where exported memory statistics could not be read by MAC.
- PIVOT-4690 Restore allocators that where removed in the previous release:
HeapBufferChunkAllocator
,DirectBufferChunkAllocator
andDirectChunkAllocator
. - PIVOT-4691 Fix possible casting exception when using
CopperMeasure.withType(ILiteralType.BOOLEAN)
with an aggregation function returningBoolean
. - PIVOT-4710
Copper.storeLookup(...)
(i.e.StoreLookupMeasure
) now handles real time updates. - PIVOT-4719 Fix lazy initialization and double check idiom in
RecordFormat#getFieldIndex(String)
.
Security
- APS-12594 BREAKING The XML parsers used throughout ActivePivot
(JAXB, XMLA Servlet...) are now given a restrictive configuration to better prevent XXE injection attacks:
- Automatic DDoS prevention. This may limit XML constructs, such as deep recursive XML elements (XML entities calling more XML entities, and so on).
- Disables
!DOCTYPE
annotation to prevent XXE injections through this annotation. - Disables the
XInclude
annotation to prevent construction of sub-documents that could be retrieved from a distant server. - Disables the use of external Document Type Definitions, for the same reason.
5.10.0
2021-02-26
Added
- APS-12558 Add new
MdxContext
parameters that can override the visibility of hierarchies and dimensions, forcing them to be shown or hidden regardless of the cube's configuration. - APS-12866 Rename
NotificationMember#AWAIT_NOTIFICATIONS
toIDataClusterDefinition#AWAIT_NOTIFICATIONS_TIME
. The property can now be declared in data cluster definitions usingICanBuildDataClusterDescriptionBuilder#withProperty
. - APS-12879 Expressions like
Copper.sum("pnl").publish(context)
have to be replaced withCopper.sum("pnl").withName("pnl.SUM").publish(context)
. - APS-12908 Add a
selectDistinct()
method toDatastoreQueryHelper
that accepts a condition. - PIVOT-3352 Add fencing on
AUniqueRecordIndexTransaction
methods. - PIVOT-3978 Changed when the 'force discard' was called in
KeepLastEpochPolicy
. - PIVOT-3978
IPrimaryRecordIndexVersion#getKeyFields()
moved toIPrimaryRecordIndex#getFields
. - PIVOT-3978 Rename
ARemoveUnknowKeyListener
toARemoveUnknownKeyListener
to fix typo. - PIVOT-4177 Add the possibility to define a Store Lookup in Copper. See
Copper.storeLookup(storeName)
. - PIVOT-4180 Properly shutdown
DataNodes
in a distributed cluster. When a data node is being shutdown, it will send aGoodbyeMessage
to the query node which will immediately trigger an asynchronous task responsible of removing all of its subsequent contributions. - PIVOT-4220 Add on-the-fly discard of transient results during query execution.
- PIVOT-4280 Support copper join on any level. It was not possible to do a join on a bucketing level for instance.
- PIVOT-4356 Join Hierarchies with multiple levels can now be published with Copper, using the unified API
Copper.newHierarchy(...)
. - PIVOT-4357 User-defined aggregate functions can now perform dynamic aggregation on measures.
- PIVOT-4409 Allow to define an Analysis hierarchy on the base store (
withStoreName
method) - PIVOT-4417 Add a new compression for chunks, storing only values different from a frequent value.
- PIVOT-4438 Update MDX engine error messages to use hierarchy unique names instead of simple names.
- PIVOT-4476 Add the possibility to specify the fields of the records given to the
StoreTransactionBuilder
in Copper testing with the methodwithFields()
. - PIVOT-4480 Add a JMX operation in the store monitoring MBean that produces a general chunk memory usage report. MAC is still recommended for detailed memory analysis.
- PIVOT-4490 Add health events to the JDBC source notifying the success or the failure of a JDBC source fetch.
- PIVOT-4491 Added support for primitive aggregated measures in the Aggregates Cache.
- PIVOT-4546 All licenses must have an end date.
- PIVOT-4549 Add
truncate
method toITransactionManager
for deleting all records from a store. - PIVOT-4566 Add a fluent builder
withSharedContextValues(values...)
allowing the definition of multiple shared context values at once avoiding repetitive calls towithSharedContextValue(value)
. - PIVOT-4569 Add
Copper.timestamp()
to access timestamp measure. - PIVOT-4589 Add support in the parquet source for filtering parsed files using a
java.util.regex.Pattern
, throughIParquetParserBuilder.withPattern(java.util.regex.Pattern)
. These health events can be handled using the interfaceIJdbcSourceHealthEventHandler
. - PIVOT-4599 Add
IClearableBitmap
interface for non-incremental add and clear operations. - PIVOT-4600 Add
MultiVersionLongArray
.
Changed
- APS-12788
AxisHierarchy
now reports that a field used to create a level should be dictionarized, instead of stating that this operation is only available in unit-tests. - APS-12918 Add more parser-policy-related interruption checks to the CSV source's
ParserTask
s. This can considerably improve performance when using the sampling parser policy. - APS-12932 Bitmap operations now catch
CancellationException
early to prevent logging bitmaps that are valid. - APS-12943 Change the default behavior of
QueriesResultLimit
context value from#withoutLimit()
to#defaultLimit()
. Now, we also allow -1 value for a limit to represent no-limit onQueriesResultLimit
context value. - APS-12946 The
DistributedEpochManager
added in PIVOT-4501 is visible in the Manager MBean under the corresponding distributed cube's MBean. - APS-12975 Reduce
DiscardOnGcTask
timeout duration from one hour to one minute. In very rare cases, this will let a version be discarded more rapidly after being released. - PIVOT-3914 Change
AAdvancedPostProcessor.computeOutputType()
to fall back to the return type of its evaluator, if it has one. - PIVOT-4191 Rename methods in
QueriesResultLimit
. - PIVOT-4356 Bucketing Hierarchies must now be published using the unified API
Copper.newHierarchy(...).fromValues(...)
. - PIVOT-4375 The Azure Cloud Source has been migrated from Azure Blob Storage SDK v8 to v12. Please refer to the migration notes to see what has changed.
- PIVOT-4402 Remove static getters from
JoinMeasureDescription
. - PIVOT-4410 BREAKING: Changes were made on the hierarchy builder.
hierarchyBuilder.factless().withStoreName("myStore")
was replaced byhierarchyBuilder.fromStore("myStore")
. - PIVOT-4464 Remove ActiveMonitor from the sandbox projects.
- PIVOT-4501 Enhance version management in a distributed setup. Versions are now handled in the Query Cube using a dedicated
DistributedEpochManager
that keeps track of versions history and trigger discard tasks on GC. - PIVOT-4502 Optimize number of operations in chunk scan methods by computing '~epoch' once before iterating over the rows.
- PIVOT-4509 Register standard Drillthrough calculators through an extended plugin.
- PIVOT-4522 Change the signature to create a UDAF to allow allocating vectors off-heap and lazy cloning. One should now call
IWritableBuffer.allocateVector
instead ofIVector.cloneOnHeap()
, andIWritableBuffer.readWritableVector(int)
instead ofIWritableArray.readVector(int)
. - PIVOT-4558 Change the behavior of
IHierarchy#isEmpty()
. Now, the method checks whether the corresponding hierarchy is empty or not instead of checking for leaf members. This enhances the performance of queries with high cardinality hierarchies. - PIVOT-4564 Levels from hidden dimensions and hierarchies are now omitted in XMLA discoveries. This caused an incompatibility with some XMLA clients such as ADOMD.NET.
- PIVOT-4579 The severity of the log message notifying that a property is missing has been changed from WARNING to INFO.
- PIVOT-4599 Rename
FastLogical
toFastLogicalQFSBitmap
. - PIVOT-4608 Lift limitation on Datastore queries with respect to the combination of indexes. Previously, the Datastore could combine up to 3 indexes when looking for range patterns like
a in (...) and b in (...)
. Now, the maximal limit is configured byActiveViamProperty.MAX_RANGE_OPERANDS_WITH_MULTIPLE_VALUES
and set by default to 10. - PIVOT-4620 Fix issue with Hibernate where a high number of input parameters causes server exceptions.
- PIVOT-4622 Fix ActiveMonitor PointValue commit logic. This adds a consistency check when committing an update on an existing point value. Now, we make sure that the "from" point value and its active value in the database have the same key. Otherwise, the commit is not allowed and an exception is thrown.
- PIVOT-4626 Fix a bug in the parquet source where an invalid index exception would be thrown if a parquet vector field was not mapped to any field in the fed datastore.
Deprecated
- PIVOT-4244 Our web services exposed with SOAP are deprecated. Your are encourage to start using the REST services.
- PIVOT-4244 XML representation of the cube configuration and our DTOs.
- PIVOT-4679 Ability to filter the facts sent to a cube (
ActivePivotInstanceDescription.setFilterDescription
).
Removed
- APS-12686 Remove
JGroupsMessenger
. - PIVOT-3978 Remove the Transaction Log.
- PIVOT-3978 Remove legacy
TransactionWrapper
. - PIVOT-4428 Remove following methods:
IChunk.findRowsInTransactionEqualTo(Object, IIntIterator, int, IRowMapping, int, IBitmap)
IChunkSet.findRowsInTransactionEqualTo(int, Object, IIntIterator, int, int, IBitmap)
IChunk.findRowsInTransactionInSet(Set, IIntIterator, int, IRowMapping, int, IBitmap)
IChunkSet.findRowsInTransactionInSet(int, Set, IIntIterator, int, int, IBitmap)
as duplicates of their counterpart outside of a transaction (findRowsEqualTo
, ...)
- PIVOT-4560 Remove duplicate
javax.xml.stream
dependency inserialization-xstream
(this dependency is included in the JRE). - PIVOT-4615 Remove heavy-to-compute logging in
ImpactComputationTask
. - PIVOT-4669 Remove deprecated
MapAggregatesProvider
.
Fixed
- APS-12090 Fix issue with branch permission manager preventing to see new branches on real time Mdx queries.
- APS-12909 Fix a memory leak within the catch up listener's mechanism.
- APS-12911 Fix postprocessors auto-partitioning in the the planning phase.
- APS-12912 Fix a bug where spring boot would occasionally fail with message
BeanCurrentlyInCreationException: Error creating bean with name 'resourceMapping': Requested bean is currently in creation.
when importing multiple configuration classes inheritingASpringResourceServerConfig
. - APS-12915 Correctly process Cube Filters in a distributed application.
- APS-12931 Fix an erroneous check that would throw when trying to define user-defined aggregate functions in partial aggregate providers.
- APS-12937 The
FullRefreshEvent
will now include the set of all impacted measures. This allows to distinguish between measures not impacted by the real-time event and measures withnull
update. - APS-12962 Fix bug preventing measure publication when using
withType(...)
on a calculated measure, and reusing this measure in another calculated measure. - APS-12965 Fix Copper Join producing incorrect aggregation results when using a Join Measure based on the key fields of the joined Store.
- APS-12966 Fix an issue where formatters would not be taken into account for UDAF measures.
- APS-12985 Correct overflow error in dictionaries occurring when inserted more than
2 ^ 30
values in a field dictionary. - PIVOT-4241 Fix the reserved memory tracking of the
SlabMemoryAllocator
when the cache is evicted. - PIVOT-4243 Fix the reserved memory tracking of the
INativeMemoryAllocator
classes to track the reserved memory and not only the used memory. The tracking is done at the lowest level for more accuracy. - PIVOT-4370 The Parquet Source now correctly handles empty strings when writing into a store field that should not support empty strings, replacing them with the store field's default value.
- PIVOT-4388 Changed the return type of the Store Builder method
withVectorField
when chained after a dictionarized or indexed vector field. It gives access to a much fuller API to configure this second vector field, or finish configuring the store with methods likewithoutKey()
. - PIVOT-4427 Fix
ChunkedBitmap#equals
implementation. - PIVOT-4436 Fix
ClassCastException
occurring when a join measure was defined on a dictionarized field. - PIVOT-4453 ActivePivot now correctly unregister all its datastore continuous selections when stopped.
- PIVOT-4455 Correctly compute vector average, ignoring null vectors instead of counting them as actual values.
- PIVOT-4456 PIVOT-4457 Correct Datastore conditions testing for equality or greater-than/less-than on nullable fields. Previously, this could:
- throw an exception for a non-dictionarized field with long series of
null
values; - return incorrect rows from the store when looking for the primitive value 0 on a non-dictionarized field;
- return rows with
null
for greater-than/less-than conditions.
- throw an exception for a non-dictionarized field with long series of
- PIVOT-4460 Fix a bug where
MemoryAnalysisService
would not produce report files for query cubes in distributed application. - PIVOT-4461 Fix
NullPointerException
during Datastore queries looking fornull
in store fields after compression. This only happened in the case of offset compression, for int and long values or dictionarized fields. - PIVOT-4463 Fix
IllegalStateException
(No free or removed slots available) inMergingStorePartitionUpdate
. - PIVOT-4500 Fix filter translation edge case in distribution. This caused
AND
filtering operation on the same hierarchy to be wrongfully translated into anOR
operation. - PIVOT-4503 Fix a bug where the batched parquet parser could throw an end-of-file exception when reading a vector column.
- PIVOT-4503 Fix a bug where the batched parquet parser would not properly read row group and page metadata, potentially throwing an end-of-file exception.
- PIVOT-4507 Fix issue with mdx drillthrough query involving a member whose name contains brackets.
- PIVOT-4516 UDAF used in a aggregate provider always stored its vectors on heap.
- PIVOT-4517 Fix a bug where an
ArrayIndexOutOfBoundsException
would be thrown when generating the memory statistic of an empty target store referenced by a non-empty owner store. - PIVOT-4525
CopperMeasure#parentValueOn(CopperHierarchy)
now properly handles an analysis hierarchy being passed as an argument. - PIVOT-4527 Fix XMLA discovery of UDAF measures.
- PIVOT-4532 Fix the computation of retrieval results size in the query plan. The total result size for partitioned retrievals were not accumulated together correctly and an empty result for a partition was causing a NPE.
- PIVOT-4533 Fix a bug in Copper where an
UnsupportedOperationException
would be thrown when subtracting a float, long, or int vector to a null vector aggregate. - PIVOT-4534 Prevent
PrimitiveAnalysisAggregationRetrieval
from inheriting not required measures from its underlying retrieval. This prevents mismatch binding when a parentPrimitiveAnalysisAggregationRetrieval
depends on a childPrimitiveAnalysisAggregationRetrieval
including additional join measures. It also improves the query times and memory usage for such cases. - PIVOT-4542 Improve error message when
QueriesResultLimit
s are exceeded. - PIVOT-4550 Fix a Copper bug where measures could not be published.
- PIVOT-4563 Fix a bug where the
Left
andRight
MDX functions would throw aNullPointerException
fornull
values. - PIVOT-4565 Fix a bug where enforcing a parallel computation of a Copper join could make a query timeout.
- PIVOT-4572 Fix
ClassCastException
during the execution of MDX queries relying on previously cached aggregates. The issue is due to a bad partitioning of a post-processor depending on a cached node and a non-partitioned post-processor node. Now, cached nodes forbid all partitioning levels and bubble them up to their respective parents. - PIVOT-4586 Fix
NullPointerException
in DriltthoughResultDTO. - PIVOT-4590 Fix
ArrayIndexOutOfBoundsException
occuring when merging two retrieval results. Particularly, the issue was due to the fact that some post-processors ignorenull
values and don't write them into the final result. As a consequence, the capacity of the underlying post-processor column won't be aligned with the retrievals points, causing such exception when merging the actual result with another one. - PIVOT-4592 Fix Copper errors when reusing code to build complex measures.
- PIVOT-4593 Fix a bug where the batched parquet parser would read incorrect values for all subsequent vectors if a column contained a
null
or empty vector. - PIVOT-4633 Fix
ClassCastException
in DataCubeAdapter. - PIVOT-4633 BREAKING: The method
Location#getArrayLocation()
has been removed. Please use the methodILocation#getCoordinate(hierarchyIdx, levelIndex)
instead. - PIVOT-4649 Fix issue with DrillthroughQuery headers incorrectly returning the type of a column.
- PIVOT-4671 The Query cube will now always reject messages and contributions from unknown applications.
- PIVOT-4678 Fix a bug where similar Copper measures using different constants were confused with each other.
5.9.3
2020-09-29
Added
- PIVOT-4191 Limit intermediate and transient retrievals result size within a
GetAggregatesQuery
usingQueriesResultLimit
. Intermediate result size referes to the size of a single retrieval whereas the transient result size is the result size cross retrievals belonging to a singleGetAggregatesQuery
. Note that this limit is set with respect to the number of points i.e. retrieval locations. - PIVOT-4341 Add
IArrayReader.readVector
method. - PIVOT-4387 Add a validation step for
StoreToParquetMapping
s before parsing a parquet file.
Changed
- APS-12854 Important: Remove Where operation now throws if the given condition reads in or through a store locked in the transaction (a store which will be written into concurrently).
- APS-12878 Improve the optimization of copper measures depending on other copper measures. In some cases the number of intermediary results will be reduced: this will reduce the memory consumed by queries.
- PIVOT-4391 Memory statistic exports do not include chunks of records for future versions anymore.
Deprecated
- APS-12854 deprecated
SchemaHelper
methodsfollowReferences
andgetStoresInReferencesPath
. The class now has methods with specific implementations whether the path given ends with a field name or not.
Fixed
- APS-12731 Fix
QueriesTimeLimit
heuristic overflow in the MDX engine. This led to cases where the actual query was still running, yet because of the overflow, the query engine was continuously timing out, creating erroneousTimeoutException
s that never bubbled up because the query didn't time out. - APS-12861 Fix the planning of
ExternalDatastoreRetrieval
to remove duplicate retrieval executions. - APS-12863 Fix
UnsupportedOperationException
occurring when applying branch security inBranchUtils.grantBranch
. It happened when the SubCube Properties of the current user were an intersection. - APS-12871 Fix an edge-case issue with the
RemoteContentServer
that would throw if two paths in the Content Server were equal up to a/
in the first, and a special character in the second. - APS-12873 Fix a bug in
LocationBuilder#setCoordinate()
. The method was skiping levels during path iteration. - APS-12876 Fix an NPE occurring with history aggregation functions in the case of a Copper Inner Join with no matching facts in joined store.
- APS-12885 Fix an edge-case where datastore query yields additional unexpected results. The issue happens when specifying a condition on a reference field and the target store includes a ValuePartitioning on that field.
- APS-12889 Fix a wrong error message displayed when attempting to perform arithmetic operations between measures with incompatible types in Copper.
- PIVOT-4383 Fix faulty behavior in
MemoryAnalysisService
regarding the exporting of multi-versioned datastores. - APS-12891 Fix potential
ClassCastException
when using the square sum or multiply aggregation function on int values. - PIVOT-4382 Fix a bug where
MemoryAnalysisService
's Dump memory statistics for some epochs bean operation would produce statistics that were incompatible with MAC. - PIVOT-4384 Fix a bug where calling
AConcreteReferenceVersion::getMemoryStatistic()
on a reference version whose owner or target partition was dropped in a subsequent epoch would produce aNullPointerException
. - PIVOT-4392 Improve the performance of the batched parquet parser on binary fields.
- PIVOT-4400 Support compiling ActivePivot projects with recent JDKs.
- PIVOT-4401 Fix a bug in the completion of JIT retrievals that could lead to threads not being properly stopped when encountering an exception in underlying retrieval operations.
5.9.2
2020-08-14
Added
- APS-12708 Add a new method
CopperMeasure.marginalTotalOn(hierarchies...)
to the Copper API to create marginal totals over an existing measure. AddedCopperMeasure.totalOn(hierarchies...)
andCopperMeasure.grandTotal()
. - APS-12724 Add a new ActiveViamProperty, set through
-Dactiveviam.contentServer.restrictedListing=<boolean>
, which will change the permission behavior of the Content Server. It defaults to false. When set to true, a user will not be able to list a file without having read access to the said file, even if he has read access to the parent directory. - PIVOT-4304 Add indexed aggregate providers support for user-defined aggregate functions.
- PIVOT-4342 Add automatic support for String vectors in the parquet source.
- PIVOT-4355 Add
IBroadcastMessage#getMessageClass()
andIBroadcastResult#getAnswerClass()
methods allowing to retrieve respectively the class representing a message type and an answer type. - PIVOT-4324 Add
IParquetParserBuilder
to build parquet source parsers. Add a new parquet parser implementation that reads records in batches, leading to significant performance improvements. - PIVOT-4354 Add JUnit5 extensions equivalent of test rules
JUnitCubeTesterBuilder
,DistributedRule
,ResourceRule
andActiveViamPropertyRule
. MigrateATestFencing
to be JUnit 5-compliant. - DOC-273 Add a new property
activeviam.i18nconfig.file.encoding
, used byLocalI18nConfig
to control which encoding to decode internationalization files with.
Changed
- APS-12730 Improve query engine to keep the maximal partitioning for operations on primitive measures.
- PIVOT-4233 Performing a filtering operation with an OR condition is now only possible in Copper if both operands are on the same level.
- PIVOT-4324 Parquet source parsers should now be constructed using a
IParquetParserBuilder
. - PIVOT-4371 Improve the performance of
BasicPostProcessor
evaluation. - DOC-273 Simplify
LocalI18nConfig
to avoid needing to use the content service, and instead use local resource files directly for internationalization.
Deprecated
- APS-12708 Deprecating
Copper.total(measure, hierarchy)
in favor ofCopperMeasure.totalOn(hierarchy)
. - PIVOT-4324 Deprecating
ParquetParser
public constructors in favor ofIParquetParserBuilder
. - PIVOT-4354 Deprecating
JUnitCubeTesterBuilder
,DistributedRule
,ResourceRule
andActiveViamPropertyRule
. For removal in 5.10. - PIVOT-4367 Deprecating
com.qfs.func
functional interfaces as well ascom.qfs.func.IEither
,com.qfs.func.impl.Either
,com.qfs.func.IOptionalEither
,com.qfs.func.impl.OptionalEither
andcom.quartetfs.fwk.util.FloatBinaryOperator
in favor of their new counterparts in packagecom.activeviam.util.function
.
Fixed
- APS-12709 Fix an issue with Copper's bucketing procedure producing wrong results when performed on top of a Join operation in a many-to-many scenario.
- APS-12710 Speed up the building of Copper-based bucketing Analysis Hierarchies depending on multiple levels.
- APS-12746 Fix an issue that could lead to a wrong order of dependencies of Copper measures.
- APS-12769 Prevent NPE in
AAdvancedPostProcessor#retrieveNamedPrefetchAggregatesWithAnalysisLevels
- APS-12777 Allow
StreamingService
to garbage collect and dispose of already stopped streams. - APS-12782 Fix a bug where the default measure defined in the pivot description was ignored.
- APS-12796 Fix query returning incomplete results. This could happen when there were IN conditions on indexed fields used in the partitioning.
- APS-12802 Fix a bug where Copper measures based on analytic functions would return wrong results when the hierarchy prior to the hierarchy used in the
Window
was slicing and queried with a range coordinate. - APS-12805 The Mdx engine now supports VBA functions used with the
VBA!
prefix. - APS-12823 Fix Sandbox CORS configuration.
- PIVOT-4233 Performing a filtering operation with conditions on multiple levels of the same hierarchy now performs an AND operation for each filtered level.
- PIVOT-4312 Fix
IllegalArgumentException
happening when querying a Join measure from a isolated store joined with Copper without mapping on a location where join-related Analysis hierarchies are not expressed. - PIVOT-4326 Fix Copper
Window
functions interaction with the QueryCache. - PIVOT-4363 Fix a memory leak caused by User-Defined Aggregate Functions when running queries.
5.9.1
2020-06-25
Added
- APS-12715
IPostProcessedRetrievalResultWriter.aggregateAndContribute(IAggregationFunction, int, List<? extends IIterableAggregatesRetrievalResult>)
. The extra argument provides the type of data consumed by the aggregation function. While this type is often the same as the measure type, functions converting longs to vectors of longs break this assumption. - APS-12740 Add support for a flexible
ISecurityWebSocketHandlerFactory
allowing to override the defaultSecurityContextWebSocketHandle
handler.
Changed
- APS-12753 Throw a clearer intermediate exception if an exception occurs during row parsing, when feeding rows to a datastore through
DatastoreService
. - APS-12757 Reducing network call of ICloudEntityPath#getLastModified() for Azure implementation.
Deprecated
- APS-12715 Deprecating
IPostProcessedRetrievalResultWriter.aggregateAndContribute(IAggregationFunction, List<? extends IIterableAggregatesRetrievalResult>)
to replace it withIPostProcessedRetrievalResultWriter.aggregateAndContribute(IAggregationFunction, int, List<? extends IIterableAggregatesRetrievalResult>)
. - APS-12718 Deprecating
DrillthroughExecutor.doExecuteHeaders(Collection<ILocation> locations, Collection<String> measures, boolean isDrillthroughContinuousQuery, boolean log)
in favor of its overloaded sibling without the log boolean. For removal in 5.10.0.
Fixed
- APS-12080 Fix Content Service MBean registration.
- APS-12588 Fix a bug with dropped partitions. The drop was not correctly reverted when the transaction was rolled back. This could cause multiple exceptions in the Bitmap aggregate provider.
- APS-12656 Fix a bug that could occur when downloading large files from Azure.
- APS-12700 Fix
IndexOutOfBoundsException
appearing randomly when using rangesharing. The computation was failing in presence of factless locations. - APS-12715 Pass the correct input type to
IAggregationFunction
when creating a reduction binding in Post-Processors. - APS-12716 Fix aggregation issues with Copper Left join that could occur when a filter was on a joined hierarchy.
- APS-12719 Fix
IllegalArgumentException
with Copper window functions occurring when querying locations with wildcards on ALL levels. - APS-12725 Copper join measures now have their MDX formatters properly applied.
- APS-12739 Fix a bug affecting branches for distribution, where hierarchies could be seen as not empty even if they were.
- APS-12751 Cube measures are now planned correctly when queried together with a Copper join measure.
- APS-12752 Fix PrimaryIndexRangeLookup for datastore query on non-existent member.
- APS-12763 Fix a bug in
GetAggregatesStream
, causing UnsupportedOperationExceptions. - PIVOT-4307 Mismatches for an aggregation procedure now only impact the join measures of said procedure, and a mismatch on a procedure will no longer lead to the failure of the following aggregation procedures on the same location.
- PIVOT-4309 Correct the health messages for distribution.
- PIVOT-4310 Records the actual start time for
DistributedPostProcessedRetrieval
instead of using the start time of the underlying task. - PIVOT-4313 Fix a bug that could occur when merging planned datastore retrieval operations on the same store.
- PIVOT-4321 It was not possible to change the Aggregate Cache's size at runtime, if the cache was disabled in the description.
- PIVOT-4331 Fix a bug when no risk data was loaded in the sandbox when ran in non-distributed mode.
5.9.0
2020-05-07
Added
- APS-9446 Better usage of primary and secondary indexes in Datastore queries.
- APS-11620 Adding additional
IHealthEvent
events for cluster monitoring, namely for cluster view change and cluster member view change notification. - APS-11620 Adds a REST endpoint to forward ActiveUI traces to a tracing server. It also add support for tracing capabilities by intercepting trace information in websockets.
- APS-11629 Allow
compiledGetByKey
Datastore queries to support dictionarized values of keys as input. - APS-11905 Introduce
ActiveViamException
,ActiveViamRuntimeException
andActiveViamSecurityException
to replace Quartet exceptions. The new classes are now located in the packagecom.activeviam.fwk
. - APS-12021 Allow the aggregation of all siblings except current member
- APS-12052 Added
AggregationLevels
option to CopperdoNotAggregateAbove()
method to specify the required levels of aggregation of a Copper measure. - APS-12120 Fix Vector casting in Copper
- APS-12141 Support for plus & minus between vector and scalar in Copper
- APS-12229 Added
nullAtTopLevels
method for ParentValue measures in Copper to specify the measure behavior when the shift operation is performed above the top level. - APS-12329 Added
toDescription
method to add a description to a measure in Copper - APS-12304 BREAKING: Adding a check for partial provider measures that verifies that the included/excluded measures are valid aggregated measures. Any project
not respecting this constraint will throw a
QuartetRunitimeException
at start-up. - APS-12458 Fix an issue that caused, in distributed environment, one object streaming failure to cascade over every subsequent object deserialization.
- APS-12498 Fix an issue in
QfsFiles
that caused thegetResourceAsStream
method to delete first character of the given path when running with OpenJdk11.0.6 (or later) and trying to access resources inside a jar. - APS-12511 The ActivePivot Version of the project is now printed at project initialization, and is accessible through the ActivePivot Manager MBean.
- APS-12532 Added the
ToStringEqualCondition
class to the extended plugin for matching conditions. - APS-12534 Fix an
ArrayIndexOutOfBoundsException
happening when publishing Copper join measures based on a Join Operation without a specified mapping. - APS-12656 Adding two ActiveViamProperties regarding the Azure Cloud Source to configure the downloading retry policy.
- PIVOT-3669 The epoch level of the epoch dimension can now be disabled. See
IEpochDimensionDescription
. It is disabled by default. The epoch dimension can be renamedBranch
with the following propertyactiveviam.mdx.epoch.dimension.legacyName=false
. - PIVOT-4032 Introduced User-defined aggregation functions that can be defined through the new Copper method
userDefinedAgg([...])
or by implementingIUserDefinedAggregateFunction
. Those aggregation functions can be used in aggregated measures by using theIAdvancedAggregatedMeasure
interface. Read the dedicated Copper documentation for more details. - PIVOT-4036 Endpoint to get the query plan of an MDX query
- PIVOT-4085 Added support for LEFT joins in Copper.
- PIVOT-4093 Improve the deserialization of Memory Statistics.
- PIVOT-4094 Hierarchies and measures of a data cube can now be concealed from the distributed application. More details in the what's new section
- PIVOT-4117 Support dependencies between Aggregation Procedures. The dependencies are defined through the attribute
IAnalysisAggregationProcedureDescription#getUnderlyingLevels()
of the description. - PIVOT-4117 It is now possible to define Aggregation Procedures not depending on any Hierarchy.
- PIVOT-4119 Support all Vector and Scalar primitive types in Copper
- PIVOT-4122 Adding support for dynamic visualisation and change of netty messages max size limits using JMX in the
DistributedMessenger
MBean. Users can use now see the message max size limits as an attribute in theDistributedMessenger
MBean and can change this configuration using the operationsetMaxSizeForMessage(messageType, size)
. - PIVOT-4132 Allow Copper to work with non-Copper analysis levels, by using the
IAnalysisHierarchy#LEVEL_TYPES_PROPERTY
property. - PIVOT-4151 Add getter to
DiscriminatorComparator
to access the wrapped custom comparator. - PIVOT-4154 Support measure groups for BuildablePostProcessorBuilder in ICanStartBuildingMeasures
- PIVOT-4181 Add JmxOperation to a Bitmap's aggregate provider that changes the level of the verifications on Bitmaps during a bitmap logical operation, for debugging purposes. Default value is
NONE
, as this will add a significant overhead in production. - PIVOT-4263
SimpleTransactionBuilder
only starts a transaction on stores with added/removed tuples. - PIVOT-4264 Added
CopperMeasure.withName
. This is an alias forCopperMeasure.as
. - PIVOT-4300 Added an optional cycle detection check in the dependency graph of query plans.
Changed
- APS-11799
ContentServerResourceServerConfigV2
was renamedContentServerResourceServerConfig
. - APS-11931 Use standard configuration for CORS in Spring.
- APS-11967 BREAKING: consider the cube filter when replicating results from Data Cubes. Like with
locations, if the Cube Filter contains hierarchies absent from the Data Cube, replication won't apply, according
to the distribution property
dataReplication
. - APS-12025 Do not print Datastore schema in headless mode.
- APS-12026 Remove the Bean annotation from custom filters.
- APS-12125 Restored default behavior of
ALocationShiftPostProcessor.getTargetMeasures()
to gettingcontributors.COUNT
(see PIVOT-3976 in 5.8.4) - APS-12169 Changed Java Native Access dependency's version from 5.2.0 to 5.5.0 for projects requiring newer versions of the project.
- APS-12281 Make the configuration
I18nConfig
close the formatters it creates, instead of delegating that toActivePivotManager
. - APS-12370 Improve the error message when a measure is declared with an invalid formatter.
- APS-12383 Change the Datastore-related
ICondition
'sequals()
method so that two instances are now considered equal if their sub-conditions are permutations of each other. - APS-12386 Providing multiple
IContextValueTranslator
implementations for the same context value interface will now throw anIllegalArgumentException
as this case is not supported. - APS-12388 Change distributed data node addresses generation to process id based instead of random.
- APS-12399 The
AggregatesEvent
of the ws-client module now contains abranch
field containing the name of the branch on which the event occured. - APS-12400 Improve MDX boolean functions evaluation. Instead of an eager evaluation on both arguments we double-check if the first argument absorbe the second one, in which case the evaluation of the latter is ignored.
- APS-12416 Hierarchies created by a Copper join on an empty store made query result empty. Now a 'default' member is contributed to prevent these hierarchies to be empty.
- APS-12443 Changed
CSVParserConfiguration
full constructor. It now takes one more boolean argument,acceptIncompleteLines
, which states whether the parser should take into account a line whose number of columns is smaller than the expected, or not. - APS-12490 Upgrade versions of various dependencies like Spring and CXF.
- APS-12468 Preventing unnecessary catalog creation for distributed cube when using the fluent API.
- APS-12514 Changing the default companion when the expression of the calculated member is a member of the same hierarchy and not a calculated member. This will improve the performances.
- APS-12549 Fix postprocessors auto-partitioning in the the finalization of planning phase.
- APS-12605 The Copper shift measures creation methods have changed :
CopperMeasure#at([...])
methods have been changed to a unifiedCopperMeasure#shift(CopperLevelsAt...)
method. Find more information in our user guide for updated examples. - APS-12647 BREAKING: Changed CopperMeasure's method
cast()
towithType()
. - PIVOT-3434 Add metadata to the DTOs of the ContentServer WS API after changes. This avoids additional REST
calls to retrieve them. This changed the structure of
ContentUpdate
and introduce an new DTOContentUpdateDto
. - PIVOT-3669 Parameters of the epoch dimension were renamed (see
IEpochDimensionDescription
):branchesLevelComparator
becomesbranchLevelComparator
branchesLevelFormatter
becomesbranchLevelFormatter
epochsLevelComparator
becomesepochLevelComparator
epochsLevelFormatter
becomesepochLevelFormatter
- PIVOT-3669
MdxCubeUtil.countEpochsBetween
was replaced byIMdxEpochManager.countEpochsBetween
. - PIVOT-3996 Reduced dependencies of
parquet-source
by excluding packages and importing smaller ones. - PIVOT-4029 Change the Min and Max aggregation functions to append only by default. The Min/Max functions supporting suppression, i.e., disaggregation (history functions) will be either called by the bitmap or specified by the user.
- PIVOT-4037 Convert the sandbox to Spring Boot.
- PIVOT-4037 Update Activiti configuration to be SpringBoot compatible.
- PIVOT-4037 Interrupt daemon threads on closing.
- PIVOT-4058 Migrate MDX documentation to Docusaurus.
- PIVOT-4069 Change the signature of
IPlugin.valueOf
. The plugin keys can no more be arrays. - PIVOT-4074 The method
IAnalysisAggregationProcedure.createContext()
takes a new argument:IScopeLocation prefetchLocation
that corresponds to the location processed by the aggregation procedures inIAnalysisAggregationProcedure.buildRequestScope()
. - PIVOT-4117 Aggregation Procedures cannot depend on the hierarchies they handle. This is enforced at validation time.
- PIVOT-4120
CopperMeasure.at
now takes fully definedCopperLevel
instead of the level's name as arguments. - PIVOT-4122 Changed the default distributed message size limits.
- PIVOT-4120 If you use
Copper.level(levelName)
then Copper APIs receiving it will now throw if several levels match instead of picking the first one. You should useCopper.level(dimensionName, hierarchyName, levelName)
instead. Same forCopper.hierarchy(hierarchyName)
. - PIVOT-4136 Fix
MeasuresDetail
attribute in the partial provider MBean. - PIVOT-4137 Better Object to primitive conversion in Records.
- PIVOT-4137 Do not push idempotent locale changes in the Content Server.
- PIVOT-4162 Detect line ending at byte level in the CSV Source.
- PIVOT-4171 Reduced memory consumption of
HierarchiesUtil#getDefaultMember
(avoid creating a list). - PIVOT-4208 Change how the descriptions are post processed. Read the migration notes for more details.
- PIVOT-4219 Distributed Cube no longer accept the epochLevel to be enabled in the epoch dimension, all queries being run against the pivot's head. Read the distributed what if page for more details.
- PIVOT-4224 Removed unnecessary copy of array in
CartesianProductValuesSupplier
. - PIVOT-4273 JustInTime is now the default AggregateProvider instead of Bitmap.
- PIVOT-4281 Alligning core dependencies version with spring-boot 2.2.6.RELEASE.
Deprecated
- APS-11905 Deprecate
QuartetException
,QuartetRuntimeException
andQuartetSecurityException
intoActiveviamException
,ActiveviamRuntimeException
andActiveviamSecurityException
. The new classes are now located in the packagecom.activeviam.fwk
. - APS-12686 The
JgroupsMessenger
is deprecated and will be removed in 5.10.0.NettyMessenger
andLocalMessenger
are the most efficient messengers and the ones users should rely on. - PIVOT-4070 The transaction log and the datastore replication will be removed in a future version.
- PIVOT-4217 Configuring your application with Spring XML configuration files is deprecated.
Removed
- APS-11931 Delete
SpringCorsFilter
in favour of Spring implementation. - APS-12215 Internal methods potentially visible to users have been deprecated:
IInternalAggregatesRetrievalResult#join(..)
withoutICubeFilter
- APS-12281 Replace
ICubeFormatterFactory#stop
byICubeFormatterFactory#close
fromAutoCloseable
. - DOC-273 Remove deprecated class
com.qfs.msg.csv.translator.impl.ConstantCalculator
, which was moved to thecomposer
module in 5.6.1. - PIVOT-3121 Remove deprecated constructor
InMemoryReportStorage(Duration, ITimeSupplier)
, defining an unused attribute. - PIVOT-4081 Remove the ability to load the ActivePivot descriptions from the
IActivePivotContentService
- PIVOT-4141 Remove unused internal dependency from
ActivePivotServicesConfig
toIDatastoreConfig
. - PIVOT-4166 Remove
IntegerArrayVersion.largestSize
. - PIVOT-4169 Remove unused property
ICSVSourceConfiguration.FOLLOW_SYMBOLIC_LINKS_PROPERTY
, because it is already specified when defining theFileSystemCSVTopicFactory
. - PIVOT-4217 Remove Spring XML configuration files. You are encouraged to use the Spring Java configuration.
- PIVOT-4270 Removing deprecated methods across the repository. Removed
MinimumDepthPostProcessor
- PIVOT-4270 Removed
MinimumDepthPostProcessor
in favor ofMinimumLevelsPostProcessor
- PIVOT-4270 Removed
TimeBucketerPostProcessor
in favor of using CoPPer - PIVOT-4270 Removed
ALegacyPostProcessor
in favor ofAAdvancedPostProcessor
- PIVOT-4270 Removed
SimplePostProcessorLegacy
Fixed
- APS-9409 Improve Mdx queries speed by reducing lock contention on the internal results
- APS-11728 Fix a bug in the
QFSBitmap.not
operator, that could produce malformed bitmaps that could make other operations throw when run on the malformed bitmap. - APS-12081 Fixing
NullPointerException
triggered by the initialization of theFilteringPostProcessorOnTheFlyPattern
. - APS-12082 In some cases the server could send a transient empty drillthrough result when the actual result was computed. This is no more the case.
- APS-12109 Fix a
NullPointerException
that could occur in the VersionHistory when releasing a version. - APS-12132 Fix ArrayIndexOutOfBoundsException in Copper which could happen in measures using several time the same underlying measure
- APS-12134 Do not fail when lookup for a location/filter without result.
- APS-12136 Fix publication of implicit hierarchies due to joins made in Copper
- APS-12147 Fixed the DatastoreService that could throw an ArrayOutOfBoundException when having more fields on the concerned target store than on the main store, an offset greater than the amount of fields in the main store, and when trying to access a field with an index greater that the offset.
- APS-12175 Fix bug in the Mdx Iif function when used with tuples
- APS-12215 Correctly set retrieval results filters when Analysis Hierarchies are involved. It previously resulted in missing results.
- APS-12217 Fix name generation when using cube as datacube
- APS-12227 Fixed JSON queries that wrongly locked all stores
- APS-12240 Fix the partitioning levels of the Copper shift operations when data is partitioned on a level of the shifted hierarchy.
- APS-12443
acceptIncompleteLines
set to false properly thrown when the last line of a csv file is incomplete. - APS-12299 Fix a
NoSuchElementException
that could happen while using a Copper shift operation defined with a lambda along a cube filter. - APS-12305 Fixed an issue with branch deletion in a distributed cluster, causing a small memory leak for each deleted branch.
- APS-12306 Fix an
UnsupportedOperationException
occurring when committing on a branch after dropping and recreating a partition on another branch. Also fix disappearing records on a branch when, on another branch, their partition is dropped and not re-created. - APS-12361 Fix an
IllegalArgumentException
when using a defined CopperMeasure instance as underlyingMeasure of another CopperMeasure. - APS-12374 Fix an issue with the publication of optimized measures in Copper.
- APS-12387 Fix a
NullPointerException
that occured when removing a data node and readding it to the cluster withinremovalDelay
seconds. - APS-12393 Fix a measure publication issue in Copper when calling
filter()
on an already filtered measure. - APS-12399 Fix the XML serialization of the
FullRefreshEvent
. - APS-12466 Fix a
RuntimeException
about non shared-dictionaries that could happen when querying a Join-defined measure defined in Copper 2 in a distributed scenario with multiple join operations. - APS-12477 Fix the declared output type of the
AverageFunction
aggregation function to match the type of the returned aggregates. - APS-12499 Fix a
ClassCastException
that could happen using CopperNamedMeasures
originated from aCopperMeasure
instance that was previously cast withCopperMeasure#cast()
. - APS-12510 Fix the
IllegalArgumentException: Empty list of fields to retrieve from the store
happening withJoinAnalysisAggregationProcedure
when doing a Copper query with no hierarchy in the location. - APS-12546
CSVSourceConfiguration
constructor correctly uses the defaults of the givenProperties
, if present. - APS-12568 Fix an issue with
DirectChunkIntegerNullable
, which could be converted to aDirectChunkInteger
while still having null values - APS-12586 Changed a
NullPointerException
into an explicit exception when starting a cube containing a measure with an invalid formatter. - APS-12641 Fix ranges update on registered query.
- APS-12647 Fix an issue with
.cast()
which was ignored when used prior to a.doNotAggregateAbove()
and other methods. - PIVOT-3927 Fix the visibility of Distributed epochs, not to see random past epochs.
- PIVOT-3976 Correctly handle the publication of measures already defined outside of Copper calculations.
- PIVOT-4035 Fixed some QFSBitmap exceptions that were caused by logical OR operations.
- PIVOT-4133 Fix the multiply operator between a scalar measure and a vector measure in Copper
- PIVOT-4139 Allow
.excludeCurrent()
with MIN and MAX aggregation functions in cumulative Copper measures - PIVOT-4140 Fix a
NullPointerException
happening when merging partitionned records from a datastore prefetch request. - PIVOT-4152 APS-12387
PointListRetrievalResult
methods now regularly checks for query interruption, preventing certain queries to continue running in the background while they have responded with a timeout. - PIVOT-4161 Fix an issue happening with Java 11: the first submitted records in a partition could disappear
- PIVOT-4168 Fix an issue with aggregation analytic functions in Copper returning unexpected results using vector fields.
- PIVOT-4170 Fix a Copper bug in which an error was thrown when a user redefined the dimension or hierarchy name of a join level and then reused that level.
- PIVOT-4185 Fix a
NullPointerException
that could happen when optimizing measures with null values in Copper 2. - PIVOT-4212 Correctly check the field of CSV Column Calculator when passed to the Parquet Source.
- PIVOT-4229 Fix an issue with the Copper Shift measure that would not be evaluated when it's underlying measure is null, regardless of the shift operation.
- PIVOT-4248 Fix an issue in Copper join operations leading to unexpected results when the levels were mapped through the join to fields that were not in the key of the joined store.
- PIVOT-4272 Fix an issue where having multiple join procedures could lead to a wrong execution logic for said procedures.
- PIVOT-4278 Fix an issue with Copper join operations not working when the mapping fields of the join were on different stores.
- PIVOT-4288 Fix query planning producing wrong aggregated values when the datastore is partitioned on a field absent in the pivot.
- SENTINEL-606 Correct listing of stopped alerts, possibly reporting alerts stopped by a concurrent operation.
5.8.4
2019-10-03
Added
- APS-11965 Add
ContentServerSnapshotter.eraseAndImport
withInputStream
as argument. - APS-11987 Ignore empty files when reading parquet folder.
Changed
- PIVOT-4013 Throw an exception if a schema contains cubes with and without the epoch dimension.
- PIVOT-3275 Throw an exception if a file in the content server is moved to the same target.
- PIVOT-11450 Enforcing non-vector fields in cube levels.
- PIVOT-4057 Update all REST documentation to end a
@RequestParameter
with an equal sign.` - PIVOT-4064 Filter context values sent to ActiveUI to not expose
user-authentication
andsubcube
context values. - PIVOT-4038 Improved the documentation about
ActiveViamProperties
- APS-11933 Some methods of
ActivePivotSchemaTransactionManager
are no moresynchronized
. - PIVOT-4041 Only reference block vectors under chunks of Vectors, instead of Vectors with their blocks.
- APS-11950 Defining Copper calculations with the
withCalculations([...])
method can only be called once per cube constructor. - APS-12002 Update the sandbox project to the latest ContentServer UI and ActiveUI app.
- APS-12009 Allow
CopperMeasure#mapToDouble()
to use typed lambdas. - APS-11932 Throw an exception when trying to create a CSVParserConfiguration containing empty column names"
- PIVOT-3713
ChunkPrimitiveInteger
was renamedAChunkPrimitiveInteger
. - APS-12035 Revert optimization when relinking indexes. It seems to generate
DiscardedEpochException
for still unknown reasons. - PIVOT-3976 API Change (rolled back with version 5.9.0): Changed default behavior of
ALocationShiftPostProcessor.getTargetMeasures()
when getting the underlying measures. If there are none, the method defaults tocontributors.COUNT
Fixed
- PIVOT-4020 Fixed an issue in secondary indexes where indexes were not relinked entirely when an updateWhere triggered on a commit.
- APS-11955 Fixed a bug in
PrefixedContentServer
where aConflictException
could happen when starting concurrent Pivot instances on the same Content Server. - APS-11940 Check for overflow before requesting direct chunk allocation
- PIVOT-4013 In some cases continuous selection listeners on a single branch could see other branches.
- APS-11933 Fixed an ActivePivotTransactionException that could be thrown by
ActivePivotSchemaTransactionManager.startTransaction
when an isolated store had an analysis hierarchy. - PIVOT-4041 Add missing chunks attributes for JDK11 chunk classes.
- PIVOT-4048 Copper measures named with
.as()
will always be published and available for queries by their name. - APS-12056 PIVOT-4024 Fix issue related to measure publication in Copper 2
- APS-12057 Fixed a bug in the Parquet Source that initialized null values with the previously read value.
- PIVOT-4063 Fixed a
ClassCastException
when performing aDistinctCount
aggregation.
5.8.3
2019-08-12
Added
- PIVOT-3976 Added experimental module Copper2, replacing the current Copper module in 5.9.0. See new documentation.
Changed
- APS-11863 Changed log level from
Warning
toFine
fortask.printPendingCompleters()
inActivePivotVersion.printExpiredQueryWarning
.
Deprecated
- APS-11865 The empty constructor of
RangeSetUnionProcedure
is deprecated.
Fixed
- APS-11801 Fixed a bug in
LocationUtil.createRangeLocation(List<? extends IHierarchy> hierarchies, List<IHierarchyInfo> hierInfos, List<ILevelInfo> levelInfos, ILocation originalLocation)
that could cause anIndexOutOfBoundException
when using aFormulaPostProcessor
with thedynamicAggregation
operand. - APS-11871 Correct code not to look for Range-Sharing optimizations when it is configured to 0. It only saves some CPU time, as it would not have found any Range-Sharing candidates.
- APS-11865 Speedup
Planning finalization time
by reducing the number of retrievals. It will impact queries containing a list with tens of members, including non existing ones. - PIVOT-3800, APS-11292 Fix the way postprocessors are auto-partitioned.
ClassCastException
inPostProcessedResultsMerger
does not happen anymore. - APS-11886 Fixed a bug in a reordering operation of secondary indexes that could induce a
DiscardedEpochException
- APS-11849 Fixed an
IllegalStateException
while filtering on the members of a Analysis Hierarchy configured with an Aggregation Procedure. - APS-11922 Fixed
ActivePivotWebServicesConfig
to Autowire the IActivePivotServicesConfig interface rather than its provided implementation. - APS-11693 Prevent query cube from incorrectly removing data cube contribution.
5.8.2
2019-07-03
Added
- APS-11817 Setting the new flag
acceptIncompleteLines
tofalse
will throw when parsing a CSV file and encountering a number of columns smaller than expected. The default value istrue
(i.e. log but do not throw). - APS-11426 MDX Order function now supports stable sort for simple statements where call to
Order
are directly nested. This means that executingOrder1(Order2(set))
will sort theset
according toOrder1
, tie-breaking withOrder2
.
Changed
- APS-11722 Exclude Analysis Hierarchies from Partial Aggregate Providers.
- APS-11730 Update aws-sdk version from 1.11.240 to 1.11.368, and update jgroups version from 4.0.15.Final to 4.0.19.Final so that it is aligned with that of aws-sdk.
- PIVOT-3602 Throw when in SUBSELECT statements, an MDX expression cannot be written as a CrossJoin of single hierarchy sets.
- PIVOT-3968 Improve the performance of relinking modified records in secondary indices.
- APS-11799
ContentServerResourceServerConfigV2
changes the content server UI namespace fromcontent
tocontent/ui
because of a conflict which broke the scheduling service. - PIVOT-3966 Review the registration of default Types and Plugins from the core product, not to collide with
user-defined Types and Plugins. As part of the change, internal classes
AttachedQueryExtendedPlugin
andDistributedAgentExtendedPlugin
have been moved, andReturnFunctionPlugin
has been renamed intoReturnFunctionExtendedPlugin
.
Deprecated
- APS-11415
StreamEventUtil.getEventHeads(Object event, long epoch)
was replaced byStreamEventUtil.getEventHeads(Object event)
. - APS-11799
com.qfs.content.cfg.impl.ContentServerResourceServerConfig
should no longer be used. It is replaced bycom.qfs.content.cfg.impl.ContentServerResourceServerConfigV2
, and the "V2" will be removed in 5.9.0. - APS-11817 The constructor of
CSVParserConfiguration
that defines theacceptIncompleteLines
flag should be used instead of the one that sets the flag to its default value.
Fixed
- APS-11744 Partial providers were not used when there were custom permissions on branches.
- PIVOT-3624 Correctly take the cube filter into account when performing a drillthrough on the query cube.
- APS-11415 Fix
IllegalStateException: Query should not be executed on shadow version
logs when using branches. - APS-11779 Correctly return the size of the stores
- PIVOT-3969 Fix resource not found (e.g. i18n) when running the application as a jar with SpringBoot.
- APS-11798 Correctly create PostProcessors when mixing fluent builders and standard builders
- PIVOT-3988 Correct
#equals(..)
for compressed entries ActiveViam sources. It was previously throwingClassCastException
s. - PIVOT-3997 Correct duplicate handling in the list of stores passed to a Datastore transaction. Previously, some stores could be erroneously deleted from the list.
- APS-11793 Fix invalid cast in Repository Hibernate layer, causing
ClassCastException
toCoordinates
. - APS-11768 Fix ArrayIndexOutOfBoundException in AAdvancedPostProcessor#removeAnalysisLevelsFromFilter, which was encountered when using a FilteringPostProcessor.
- PIVOT-3991 Fixed missing data in the statistics exported by
MemoryAnalysisService
.
5.8.1
2019-06-01
Added
- APS-10824 Allow an
IActivePivotManagerDescription
to have a single distributed cube with fluent builders. - PIVOT-3963 Added a MBean attribute for Partial Providers listing included measures.
- APS-11178 A gross sum aggregation function (sum of the absolute values).
Changed
- PIVOT-3920 Reuse existing record format for the result if available.
- PIVOT-3919 Use a code fast-path when splitting paths of one char.
- PIVOT-3818 Improve the registration of MDX streams to consume less CPU resource.
- APS-11601 Update the READMEs of the Sandbox project, to detail and explain how to customize the configuration of the packaged UIs.
- APS-11702 Do not restart every monitor when one monitor fails to start. Instead, this monitor start will be retried later.
Fixed
- PIVOT-3942 Correctly use
IPartitionedPostProcessor#DEFAULT_PARTITIONING_ON_RANGE_LEVELS
as the default value auto range partitioning for Post-Processors. - PIVOT-3540 Correct re-initialization of the
MdxRegistration
state when creating new Listeners. - APS-11657 Fix a NullPointerException in the CSVExportDrillthroughExecutor which occurred when one of the records had a null field.
- PIVOT-3957 Fix BadArgumentException which occurred when a range property was set and the result of an Mdx query contained an empty axis.
- APS-10884 Precise documentation on
StoreDescription.shouldCompressIndexes()
andCompressIndexesPostProcessor
. Improve memory allocation for dictionaries. - PIVOT-3957 Add documentation on
NumaSelectorPostProcessor
. - APS-11563 Include all measures NoOp retrievals are retrieving into their result.
- PIVOT-3960 Fix a bug in
QFSBitmap#or
that could result in extra members appearing in a result. - APS-11728 Fix a bug in
QFSBitmap#or
that could produce wrong results when using filters in a query. - APS-11719 Fix a bug in Store builders where a Duplicate key handler could not be defined when there was no pending taggable field.
- PIVOT-3965 Fix an IllegalArgumentException that could be raised when using SpringBoot > 1.3 and giving resources with absolute paths on Windows.
- PIVOT-3565 Correctly retrieve the timestamp of a file last modification.
- APS-11722 Correctly pick the best Partial Aggregate Provider even with locations or subcubes including members of Analysis Hierarchies.
- PIVOT-3864 Forbid to start a transaction on stores indirectly involved (i.e. through a reference) in an insertion-time update-where trigger.
5.8.0
2019-04-15
Added
- PIVOT-3622 Integrate the Cloud Source to the core product.
- APS-11272 Add a new post-processor
FilteringPostProcessorOnTheFlyPattern
as an alternative to the MDX functionMemberCaptionSearch
. It allows to search for a given pattern in member captions for a given hierarchy. It will be used by ActiveUI when available. - APS-11323 Provide a context value translator for every core Context Value.
- APS-10617 Add a way to manually handle field name conflicts when building the datastore selection.
- APS-11533 Expose the range sharing in IAggregateProviderDefinition
- PIVOT-3804 Support the lazy loading of Mdx results with any number of axes.
Changed
PIVOT-3839 Datastore and Copper tests have been moved to their own modules (
com.activeviam.tech:datastore-test
andcom.activeviam.activepivot:activepivot-copper-test
)ABasicPostProcessor (respectively ABaseDynamicPostProcessors) now uses named prefetchers. They will use the constant ABasicPostProcessor.BASIC_POST_PROCESSOR_PREFETCHER (resp. ABaseDynamicAggregationPostProcessor.DYNAMIC_AGGREGATION_POST_PROCESSOR_PREFETCHER) as the name of the prefetcher to retrieve. If you define your own prefetcher you need to name it correctly :
IPrefetcher<?> namedPrefetcher = IPrefetcher.name(ABasicPostProcessor.BASIC_POST_PROCESSOR_PREFETCHER, prefetcher);
APS-11227 The
driverClassName
argument for theJDBCSource
constructor has been moved toSimpleConnectionSupplier
. It is now only required when noIConnectionSupplier
is provided.APS-10692 The appendQueueSize attribute is no longer part of the
AJDBCSource
, but rather theJDBCTopic
.Read the migration notes for more details.
APS-10998 The
ICubeFilterBuilder.includeMembers
method accepting wildcards for members path name (eg.List<?>
) has been moved toincludeMembersWithConditions
. The newICubeFilterBuilder.includeMembers
now only acceptsList<String>
TheICubeFilterBuilder.excludeMembers
method accepting wildcards for members path name (eg.List<?>
) has been moved toexcludeMembersWithConditions
. The newICubeFilterBuilder.excludeMembers
now only acceptsList<String>
PIVOT-3670 When a listener listening on several branches (e.g. pivot) is attached on a non empty datastore, it will now catch up with the current state in a maintenance operation (i.e. blocking all transactions until it catches up).
For listeners listening on a single branch, the catch-up mechanism was improved. As previously, it does not block transactions, but its memory footprint was improved. The property
qfs.selection.listener.catchUpMaxTime
defines the maximum time (in s) in which the listener will try to catch-up while not blocking transactions: when it is exceeded, the listener will run a maintenance operation to finish catching up. The default value is 3min, and setting a non positive duration sets an infinite maximum time.APS-9927 The ActivePivot Sandbox now has a 4-Level Hierarchy (the 'LegalEntity' and 'BusinessUnit' levels have been added on top of the already existing
Booking
hierarchy)APS-10705 The sandbox project properly configures its firewall to support special chars in file names for the Content Server.
APS-11634
StoreUtils.getPartitionSizes()
now returns -1 as a value for dropped or non existing partitionsAPS-10848 The
StreamingMonitorConfig
configuration dependency toIUserDetailsService
is no longer mandatory.PIVOT-3866
IEpochHistory.markHead()
andIEpochHistory.markHead(String branch)
now return theIEpoch
that was marked instead ofvoid
.APS-11427 The metadata fetching methods of the
JDBCSource
have been modified to not rely on a non-standard behavior of JDBC Driver implementation ofResultSetMetaData.getColumnName()
, but instead use the standardizedResultSetMetaData.getColumnLabel()
APS-11423 Improve the logs for notifications generated by real-time queries.
PIVOT-3906 Rewrite the tasks performing Range-Sharing to avoid potential assertion failures.
PIVOT-3943 Perform the hierarchy contributions (during a Pivot commit) in parallel; this can speed up commit times in configurations with many hierarchies.
{@jira: APS-11920, PIVOT-3005} ActivePivot queries now return "NaN" for
Double.NaN
values instead of "\uFFFD" (�).
Deprecated
- PIVOT-3670 An
ISelectionListener
does not record changes anymore, so it has no reason to becleared
anymore. The functionscleared()
andisClearable()
will be removed in 5.9.
Removed
PIVOT-3834 The changes introduced in Post Processors V2 in 5.7.3 are now merged into the normal implementation of the post processors. Read the migration notes for more details. The following post processors V2 have been removed :
- ABaseDynamicAggregationPostProcessorV2
- ABasicPostProcessorV2
- ADynamicAggregationPostProcessorV2
- AFilteringPostProcessorV2
PIVOT-3833 The following methods and fields have been removed :
- The
MDXQuery.QUERY_NAME_PROPERTY
constant has been removed IMdxStream.viewUpdated
has been removed => useIMdxStream.publishCellDtos
ContextUtils.ensureContext
has been removed => useIContext.apply(IContextSnapshot)
ContextUtils.restoreContext
has been removed => useIContext.apply(IContextSnapshot)
- In JDBCSource the constructors without an appendBatchSize has been removed
- In ClientPool the constructors with vararg that may lead to undetected type errors => use constructors taking collections
- The
The
AJDBCSource
constructors with noappendBatchSize
arguments have been removed.PIVOT-3855: Remove the LINE storage type. The only possible configuration is now
StorageType.COLUMN
. Read the migration notes to see how to adapt your code.APS-11272 Remove
InterruptionUtil#checkInterruption(int)
andInterruptionChecker#checkInterruption(int)
in favour ofPeriodicInterruptionChecker
to avoid static variables in the code.APS-11552
Sets.isEmpty
is replaced byISet.isEmpty
.
Fixed
- APS-11552 Speed up the evaluation of CrossJoin on axes with NON EMPTY
- APS-10889 Detect when native library cannot be loaded because the native layer failed to initialize.
- APS-11308 Using real-time when new branches are created no longer produces infinite loops.
- APS-11446 Fix race condition on PointListRetrievalResult additional columns that resulted in an ArrayIndexOutOfBounds on some queries
- APS-10905 CoPPer measures created with a
.join()
on an isolated store will now be properly updated during continuous queries when the store is updated. - PIVOT-3862 Registering a continuous query with a listener on a datastore selection (such as com.quartetfs.biz.pivot.postprocessing.streams.impl.AStoreStream) no longer triggers an initial view, harmonizing the behavior across all continuous query listeners.
- APS-11305 PIVOT-3761 Improve CoPPer error messages
- PIVOT-2924 Correct the MBean on top of
ActivePivotContentService
to clean the local and root caches. - APS-11309 Remove an unnecessary warning when manually stopping a query task.
- APS-11439 Fix an issue in Copper when using a legacy postprocessor in calculations.
- APS-11351 Allow the same CoPPer column to be used in a calculation multiple times.
- APS-10914 Correct query planning combining range sharing and joins from Copper.
- APS-11438 Fix order of transaction units (add, remove, removeWhere...). The order is now respected.
- APS-11022 Fix the ContentServiceBranchPermissionManager to allow for full server eraseAndImport instead of having to update subtrees only.
- APS-11526 Correctly export
Monitor
during field in DTOs. - APS-11547 The FilteringPostProcessorV2 now returns an empty result when no record matches its filtered member(s); the method
OrCondition#of
has also been removed. - APS-11060 Partial provider was not used when the Epoch dimension was disabled.
- DS-272 Fix deadlock that could appear when submitting records in parallel on the same partition.
- APS-11469 Allow to set the default member of the Epoch hierarchy like any other hierarchy.
- APS-11511 Union function used in SubSelect could wrongly return an empty result.
- APS-11587 Fix the Datastore Rest API, incorrectly failing when finding a reference that was not an expected one
- APS-11586 Fix the Datastore Rest API, incorrectly mixing the number of fields and the depth of their reference paths.
- PIVOT-3926 Fix concurrency issue in on-demand licensing
- PIVOT-3924 Avoid building unnecessary objects when calling
Objects.requireNonNull
. - APS-11634 SchemaPrinter no longer prints discarded partitions as existing with a record size of 0
- PIVOT-3922 Performance improvements for join made in Copper
- APS-11619 Fix an occasional NullPointerException raised in the sandbox.
5.7.7
Added
- PIVOT-3778 Measures that have already been published in Copper are available as regular columns in the following createDatasetFromFacts datasets.
- PIVOT-3762 New mode in the query engine that sends only notification events for continuous queries. This allows to explore a current result without running the risk of seeing it change and at the same time know when the current view is not up-to-date anymore. See WebSocket documentation to enable it.
Fixed
- APS-11326 Fix CubeFilter serialization problem related to ICubeFilterIntersectionWithSubCube
- APS-11280/APS-11274 Fix several issues regarding EmptyAggregateStore and ReplicatedAggregateStore.
- APS-11283 AVG aggregation function could prevent IVector Objects from being garbage collected.
- APS-11306 Fix serialization/deserialization in RemoteDatastoreService
- PIVOT-3808 chunk compression can be disabled (
-Dqfs.compression.chunkset.disable=true
)
5.7.6
Added
- APS-11246 Add interruption exception in MembersCaptionSearch and MembersNameSearch.
- APS-11250 Always provide a record format for ICursor, even for null results.
- APS-11110 ConditionToJsonNodeHelper now uses JacksonSerializer as ObjectMapper, which let your register your own (de)serializers.
- APS-11249 Add a method to split the datastore/schema path in a clean way: DatastoreSchemaDescriptionUtil.splitPath. The mirror of DatastoreSchemaDescriptionUtil.createPath which creates a path with the correct separator from an array.
Changed
- APS-11294 Replace the micro letter by "us" as the unit for time values in micro-seconds.
Fixed
- APS-11273 fix wrong query results due to chunk compaction & garbage collection.
- APS-11087 fix issue in distributed architecture when there are more than 64 members in the cluster.
- PIVOT-3788 Return a null retrieval result when a datastore external request has no result.
- APS-11278 Fix the number of cores detected on a Linux machine when numactl is used.
- APS-11298 Add translation files in sandbox for Spanish and German (es-ES and de-DE) so the locale can be changed in ActiveUI without error.
- APS-11274 APS-11280 Correctly clone empty aggregate stores.
5.7.5
Added
- APS-11121 Improve the error message in case of incompatible partitionings. Now, the message includes involved store names, the considered reference as well as general suggestions of changes.
- APS-11242 Improve the error message in case of unknown underlying measure. Now, the message prints only the 10 measure names that are the closest (according to the Levenshtein algorithm) to the the provided name. The Levenshtein distance is now available for use in QfsStrings.
- APS-11090 Add a IBaseCondition to JSON Node converter, it can be found in ConditionToJsonNodeHelper class.
Changed
- APS-11103 post-processor properties (such as XMLA description) from the data cube are now sent to the associated measure in the query cube (see auto-distributed post-processors for more details).
- APS-11104, PIVOT-3188 formatters defined in a data cube are now propagated to a query cube Distributed Cube Description (v5.5 and After).
- PIVOT-3780 Records in the datastore query result cursor are now ordered by partition.
Fixed
- APS-11086 Add additional checks to monitor execution stream to support empty slicer. This fixes a
IndexOutOfBoundException
inSentinelMdxStream
. - APS-11092 The warning in ClusterLocalityIdentifierFactory about having post-processors with COPY aggregation function and no slicing distributing field now shows when it is supposed to.
- APS-11154
AFilteringPostProcessorV2
now extendsABasicPostProcessorV2
and correctly handles analysisLevel property. - APS-11154
AFilteringPostProcessorV2
andFilteringPostProcessorV2
are now parametrized with OutputType. - PIVOT-3765 fixes an edge case in ChunkedBitmap.equals().
- PIVOT-3785 Correct QFSBitmap
IndexOutOfBoundException
. - APS-11186 Rewrite condition in remove where to match more easily the drop partitions criteria.
- APS-11081 Add a cache to cube filters intersections to improve performance when intersecting a lot with the same subcube properties.
- APS-11212 Allow to reset a branch to its parent even if its forkpoint is a shadow commit.
5.7.4
Added
- APS-10942 The appendBatchSize in AJDBCSource (indicating how many records are drained from the queue at once) is now configurable in the constructor.
Changed
- PIVOT-3715 A failure during a transaction in ActivePivot triggers a rollback of the underlying datastore transaction when possible.
- APS-10990 Added Column#notEqualTo operator
- PIVOT-3224 Added Columns#shortSum
- PIVOT-3764 Improve the logic for impact computation in presence of Aggregation Procedures. This avoids to always trigger a full-refresh.
- APS-11068 Improve the exception in case any operation performed on a store during a commit fails. It always provides the store name, operation type and modified store partition.
Fixed
- PIVOT-3744 CoPPer can now drillup on analysis hierarchy levels
- PIVOT-3760 Correct replanning of aggregation procedure dependencies
- APS-10971 Ensures that retrievals computing the logic of Aggregation Procedures returns a true retrieval result. This fixes IllegalStateException complaining about "an unplanned retrieval (no primitive result)".
- APS-10846 Remove the incorrect check in memory monitoring report, causing UnsupportedOperationException.
- PIVOT-3663 Releasing a branch in a data cube does not throw an exception during the planning of a query in the query cube
- APS-11032 Use an appropriate object to merge contribution deltas in query cubes. This fixes an UnsupportedOperationException.
- APS-11063 fix translation with NO_REPLICATION
- PIVOT-3768 Support calling #next() without previous calls to #hasNext() for QfsBitmap iterators.
- APS-11075 Fix a bug when cloning a partitioning description with modulo on several fields.
- APS-10912 Checks for empty cube are now done on all the head versions and not only the last one.
5.7.3
Changed
- PIVOT-3720 Reduce the impact (memory consumption is reduced, local results are more efficiently translated <=> queries should be quicker) of Data Replication in polymorphic distribution
- APS-10729 - Fetch & Branch : Update the Fetch and ITransactionManager classes to handle data fetch into specific branch
- PIVOT-3714 Datastore fields used by levels are automatically converted to the default value when inserted in the datastore. This prevents errors in ActivePivot at commit time, when creating new Mdx Members.
- PIVOT-3741 Added AFilteringPostProcessorV2 and its concrete class FilteringPostProcessorV2, which are better implementations than AFilteringPostProcessor and FilteringPostProcessorWithEvaluator. The old ones will be removed in 5.8.0 and the new ones will take their name.
Fixed
- APS-10706 fix Aggregates Cache Statistics; the MBean also displays pivot version and branch name for the statistics.
- PIVOT-3676 The DistinctCountFunction now has an aggregated datatype of LONG instead of the same type as its inputType.
- PIVOT-3659 CoPPer now supports using aliases for fields in the base store
- PIVOT-3734 Fix incorrect exception raised by the sandbox on single-core machines due to ConstantPartitioning.
- APS-10875 Improve SinglePartitioning performance by removing a TIntSet of partition ids which causes thread contention
- APS-10843 Fix faulty nano-to-milli conversion in CSV source logging
- APS-10685 Fix NullPointerException in AHibernateContentService by allowing nested read actions
- APS-10920 Fix issue when joining with a dataset creates more than 1 analysis hierarchy
- APS-10897 Correct the handler of the Lookup Post-Processor to refresh a query for deleted point values.
- APS-10897 Correct the computation of changes for effective parameters. Previously, deleted point values with future expiration were returned as deleted multiple times.
- APS-10757 Improve ClientPool constructors to avoid confusion between properties and extensions. This correct usages of client pools to correctly pass the properties.
- APS-10856 Fix an ArrayOutOfBoundsException when a new partition is created in the middle of a query planning
- APS-10630 Queries on a query cube with an empty data cube no longer throw a NullPointerException.
- APS-10929 Fix NullPointerException which sometimes happened when conflating transactions.
- APS-10880 Fix NullPointerException when expanding query scope into a location associated with no facts.
- APS-10813 Fix legacy post processor visibility
- APS-10881 Analysis hierarchies were only partially supported by basic and dynamic post processors. The
analysis levels were removed on the location but not the cube filter. As fixing this behavior might introduce breaking
changes in how the post processors work we introduced three new classes that must be extended instead of the previous
version if the post processor uses the analysisLevels property:
ABasicPostProcessorV2
ABaseDynamicAggregationPostProcessorV2
ADynamicAggregationPostProcessorV2
These classes will be merged back into there classic implementation in 5.8, potentially introducing a breaking change for subclasses that overwrite the compute function or the prefetchers Post Processors that try to use analysisLevels without handling it correctly will now log a warning during the init. You can learn how to correctly implement AAdvancedPostProcessor with analysis levels in this confluence article.
- PIVOT-3737 Correct ConcurrentModificationException when using aggregation procedures.
- PIVOT-3738 Correctly consider partitionings of underlying retrievals of aggregation procedures.
5.7.2
Added
- APS-10427 Hadoop Configuration can be given to the parquet parser
- APS-10618 Support for user defined datastore schema separators with the system property
activeviam.datastore.schema.separator
Changed
- APS-10627 The Epoch dimension is now visible by default when enabled.
- PIVOT-3685 Detects usage of analysis hierarchies within the cube filters or the security subcubes.
- APS-10691 Added
QFSPools.getMixedWorkloadThreadCount()
to allow knowing how many threads can be used at max for a transaction (or for a query, since transaction and query pools have the same size). - SENTINEL-624 Improve performance of the Repository during insertion of new point values and while computing differential updates involving a lot of deleted point values.
- APS-10708 Fail when registering a workflow relying on an undefined Activiti workflow process.
- PIVOT-3707 Cube filters are now available to AggregationProcedures from #createContext through the ActivePivot context.
- APS-10615 It is now possible to exclude a specific field from the selection when adding all fields or all reachable fields of the base store or a reference (using .except()), rather than always excluding all fields with the same name (which is what .excluding() does)
- PIVOT-3675 Default implementations exist to configure branch and store permissions. They are designed for
development, not apply any security.
SeeFullAccessBranchPermissionsManagerConfig
andNoSecurityDatastoreServiceConfig
. - APS-10711 Formatting for thousand/million rescaling and multisection formatting are now supported.
- PIVOT-3224 Join between datasets now uses Analysis Aggregation Procedures.
Removed
- The following classes have been removed from the sandbox:
- ForexPostProcessor
- ForexHandler
- ForexStream
- ReferenceCurrency
- ReferenceCurrencyTranslator.
- The measure pnl.FOREX is defined in copper. The currency can now be chosen via a new slicing hierarchy named TargetCurrency (automatically created by copper).
Fixed
- APS-10699 Avoid exception when new branch is created without subcube
- PIVOT-3699 A multimensional single element set in the MDX WHERE clause now works (like a tuple).
- APS-10740 Correctly compute modulo on multiple fields, not to double the number of produced values.
- APS-10766 unmark epochs that are released because they corresponded to shadow commits
- PIVOT-3580 Implementation of a filtered list, FilteredList, to manage different measure description (sub)types for a single parent list.
- PIVOT-3673 Fix NullPointerException when an ExpandLocationResult overflows during large transactions
- APS-10724 Correct formatting of alert messages in the sandbox project.
5.7.1
Added
- APS-6362 The Aggregate Store size was added in JMX for both the Bitmap and Leaf aggregate providers.
- PIVOT-3548 The Content Server UI now has a title.
Changed
- APS-10622 APS-10624 PIVOT-3608 Fluent Builder API was improved (slicing, factless, memberProperty).
- SENTINEL-622 Improve the parameter store partitioning. This store is now partitioned by parameter name and ActivePivot name for which it defined, so any lookup for a given parameter within a given cube hits a given partition. Then, the partitioning distributes the point values on a multi-modulo of the erasure and the wildcard erasure using the number of cores of the machine.
- APS-10426 Null values in parquet source are now converted to default field value.
Fixed
- APS-10614 Correct query planning for query cubes with respect to analysis hierarchies.
- APS-10184 Measures from different applications and with the same aggregation id (even if they have different names) are no longer merged in the the query cube, giving wrong results
- APS-10591 Correct merging of retrieval results not containing any aggregated measures.
- PIVOT-3537 Correctly update permissions of folders from the Content Server UI.
- APS-10580 Prevent from automatically adding levels to analysis hierarchies declared using an xml file, and add check that analysis hierarchies do not have levels.
- APS-10528 Fix MemberNamePathStyle context value in intersected context
- APS-10545 Fix issue when a data cube leaves its cluster while planning a query
- PIVOT-3224 (CoPPer) Use actual cube name in tests
5.7.0
Added
- PIVOT-3612 Drop dedicated Angular UI of ActiveMonitor. ActiveUI becomes the only UI for this component, providing all features of the old UI. See the migration notes to see how to update your existing post-processors.
- PIVOT-3609 Support prepared statements in fetch-only JDBC Topics
- DS-293 Add API to create CompiledGetByKey query and run them, either on a particular version (runInVersion) or on the current transaction (runInTransaction). See the documentation for more details and examples.
- PIVOT-3655 The Content Server / Service supports multiple owners and readers for each entry.
- APS-10493 Create AFilteringPostProcessor to increase flexibility when implementing a filtering post-processor.
Changed
- PIVOT-3584 Replace
IPartitionedPostProcessor#getPartitioningLevels(...)
calledIPostProcessor#setPartitioningLevels(..)
to configure the levels on which a post-processor is partitioned. - PIVOT-3647 Revamped Spring configuration classes to either define beans or import other classes.
- PIVOT-2879 Do not notify Content Server listener when a non-readable child entry is updated.
- APS-9893 Keep an aggregates cache for the latest version of each branch. The branches on which aggregates will be cached can be configured.
- PIVOT-3371 FileSystemCSVTopicFactory now implements AutoCloseable. Usage in the sandbox has been modified (Spring Bean that auto-closes instead of the callback in the csv source close method)
- PIVOT-3625 Update dependencies to their latest versions, notably Spring (→ 5.0.6) and Spring-security (→ 5.0.5).
Fixed
- PIVOT-3626 Correctly publish retrieval additional measure across threads.
- APS-10465 Fix native measure computation in distributed setup when one of the data cubes is empty
- DS-294 Fix "scan in transaction" methods for chunks having a single value
- PIVOT-3641 fix incorrect subcube compilation when using condition on analysis level
- APS-10485 fix realtime drillthrough when member contains "[" and "]"