Removing data from a distributed data cube with overlapping data can lead to query inconsistencies.
A query may be planned before data removal but executed after the removal, causing missing or incorrect results.
This happens even if the removed members exist in another data cube because the query node already decided where to retrieve them.
The target level provided in the LevelMembers parameter is a distributing level of a query cube that has multiple distributing levels and horizontal data duplication enabled
That level is not the distributing level of the application associated with the data cube
The cluster must be stable during the masking operation. In particular, a query cube joining in the middle of a masking may keep trying to retrieve the removed members from the data node because it is not aware of the masking.
The masking operations are forbidden in query cubes where the deprecated IMultiVersionDistributedActivePivot.unloadMembersFromDataNode has already been invoked.
The IMultiVersionDataActivePivot.unmaskMembers method reverses a masking operation, re-enabling the query cube to retrieve the unmasked members from the data cube.
Unmasking should not be part of a usual data removal workflow.
It should only be used to roll back a masking operation that was performed in error or is no longer needed.Note: Unmasking members that are not currently masked has no effect.
Both maskMembers and unmaskMembers methods return a CompletableFuture<IMaskingOperationReport> that completes once the operation has been processed across all query cubes. The IMaskingOperationReport provides detailed information about the operation outcome:
Success status: Whether the operation succeeded across all query cubes
Successful query cubes: A list of query cube addresses where the operation was successfully applied
Failed query cubes with reasons: A map of query cube addresses to failure reasons for those where the operation failed. In case of repeated failures due to the retry mechanism, only the failure reason from the last attempt is provided.
A failure does not mean the operation failed for all query cubes - the masking or unmasking may be successful for some query cubes but not for others. If masking or unmasking is not successful, the cube continues to operate, but members that are masked cannot be retrieved from the data cube where they are masked.To observe which members are actually masked from which data cubes, see the Monitoring section.
// Create LevelMembers recordsfinal LevelMembers countryMemberToMask = new LevelMembers(LevelIdentifier.simple("Country"), List.of("US", "France"));final LevelMembers currencyMemberToMask = new LevelMembers(LevelIdentifier.simple("Currency"), List.of("USD", "EUR"));// Mask members on both data cubes (with 3 retries)final CompletableFuture<IMaskingOperationReport> maskingA = dataCubeA.maskMembers(countryMemberToMask, IEpoch.MASTER_BRANCH_NAME, 3);final CompletableFuture<IMaskingOperationReport> maskingB = dataCubeB.maskMembers(currencyMemberToMask, IEpoch.MASTER_BRANCH_NAME, 3);// Wait for both masking operations to completeCompletableFuture.allOf(maskingA, maskingB).join();// Check that both masking operations were successfulfinal IMaskingOperationReport maskingReportA = maskingA.join();final IMaskingOperationReport maskingReportB = maskingB.join();if (!maskingReportA.isSuccessful() || !maskingReportB.isSuccessful()) { throw new IllegalStateException( "Masking failed. Report A: " + maskingReportA + ", Report B: " + maskingReportB);}// Now it is safe to remove the data from the database:try { transactionManager.startTransaction("baseStore"); try { transactionManager.removeWhere( "baseStore", BaseConditions.in(FieldPath.of("country"), "US", "France")); } catch (final Exception exception) { transactionManager.rollbackTransaction(); throw new RuntimeException("Problem while removing data in transaction", exception); } transactionManager.commitTransaction();} catch (final Exception exception) { // Unmask members on both data cubes (with 3 retries) final CompletableFuture<IMaskingOperationReport> unmaskingA = dataCubeA.unmaskMembers(countryMemberToMask, IEpoch.MASTER_BRANCH_NAME, 3); final CompletableFuture<IMaskingOperationReport> unmaskingB = dataCubeB.unmaskMembers(currencyMemberToMask, IEpoch.MASTER_BRANCH_NAME, 3); // Wait for both unmasking operations to complete CompletableFuture.allOf(unmaskingA, unmaskingB).join(); // Check that both unmasking operations were successful final IMaskingOperationReport unmaskingReportA = unmaskingA.join(); final IMaskingOperationReport unmaskingReportB = unmaskingB.join(); if (!unmaskingReportA.isSuccessful() || !unmaskingReportB.isSuccessful()) { throw new IllegalStateException( "Unmasking failed. Report A: " + unmaskingReportA + ", Report B: " + unmaskingReportB); }}
This approach ensures that:
Both data cubes mask their respective members before any data is removed
All query cubes are aware of the masking before the database removal
Query results remain consistent throughout the operation
The actual data removal only happens after all masking operations complete successfully
The same information can be accessed through JMX using the getDistributingLevelMemberMapping operation in the MBean
com.activeviam:node0=ActivePivotManager,node1=<SchemaName>,node2=<CubeName>.
Was this page helpful?
Assistant
Responses are generated using AI and may contain mistakes.