Skip to main content

Aggregate Provider

Introduction​

To provide better performance, Atoti's aggregation engine can store pre-aggregated data for some selected cube locations, in the AggregateProvider component. The aggregation functions will use the aggregate provider to access pre-aggregated data rather than computing everything from scratch, leading to faster query results.

A provider can use three main approaches to store pre-aggregated data:

  • the Just-in-Time approach where nothing is stored and everything is computed at run time
  • the bitmap approach where the pre-aggregated data is stored in a table and which uses a bitmap index to match the locations asked in the query to the locations in the table
  • the leaf approach where the pre-aggregated data is stored in a table like the bitmap approach but compute the bitmap at run time.

The first approach is light-weight but slower. The bitmap is the fastest but is memory-intensive. The leaf is a compromise.

For data-intensive application, not all data can be pre-aggregated so we compose an aggregate provider with:

  • a Just-in-Time global provider to be used by default
  • some partial providers to cover the most used locations for the most used measures in the queries run by the users. They can be a mix of bitmap and leaf providers.

To find the best configuration of partial providers, try using the ai-based optimizer!

Defining an aggregate provider using the fluent builder​

It is possible to define the aggregate provider configuration using the fluent builder:

pivotDescription =
    StartBuilding.cube("MyPivot")
        .withAggregatedMeasure()
        .sum("value")
        .withSingleLevelDimension("id")
        .withAggregateProvider()
        .jit()
        .withPartialProvider()
        .bitmap()
        .includingOnlyLevel(LevelIdentifier.simple("id"))
        .end()
        .build();

Direct definition of aggregate providers​

It is also possible to define the aggregate provider definition directly.

First we define the partial providers:

final PartialProviderDefinition partialProvider =
    new PartialProviderDefinition(
        "MyProvider",
        IAggregateProviderDefinition.BITMAP_PLUGIN_TYPE,
        List.of(LevelIdentifier.simple("id")),
        List.of("value.SUM"),
        PartialProviderFilters.noFilter(),
        new Properties(),
        null);

Then we define the global provider which will hold the partial providers:

final AggregateProviderDefinition globalProviderDefinition = new AggregateProviderDefinition();
globalProviderDefinition.setPartialProviders(List.of(partialProvider));

Finally, we add the aggregate provider definition to the cube:

IActivePivotInstanceDescription pivotDescription =
    StartBuilding.cube("MyPivot")
        .withAggregatedMeasure()
        .sum("value")
        .withSingleLevelDimension("id")
        .withAggregateProvider(globalProviderDefinition)
        .build();