ActivePivot

ActivePivot

  • 5.10.3
  • Other Versions
  • User Guide
  • Technical Documentation
  • Support

›Aggregation & Analytics

Introduction

  • Overview
  • What's new in ActivePivot

Getting Started

  • Overview
  • AP in a Nutshell
  • Development Environment
  • Download
  • Sandbox Project

Concepts

  • Overview
  • AP Concepts in a Nutshell
  • Data Versioning (MVCC)
  • Dimensions and Hierarchies
  • Partitioning and NUMA
  • Other Concepts

Data Loading

  • Overview
  • Datastore

    • Datastore Configuration
    • Datastore Transactions
    • Store Indexing

    ETL

    • Overview
    • CSV Source
    • JDBC Source
    • Parquet Source

    Loading data from the cloud

    • Cloud Source
    • Amazon S3 Cloud Source
    • Azure Cloud Source
    • Google Cloud Source

    Specific data types

    • Handling Dates
    • Vectors

Aggregation & Analytics

  • Overview
  • Cube Configuration
  • Copper API

    • Introduction
    • API
    • Measures
    • Hierarchies
    • Publication
    • Join operations
    • Advanced topics

    Streaming API

    • Continuous Queries Overview
    • Streaming Overview
    • Continuous Query Engine
    • Continuous Handlers

    Advanced APIs

    • Cube Locations
    • Post-Processors
    • Cube Filters
    • Member Properties
    • Context Values

Data Querying

  • Overview
  • Business Frontends
  • Server Endpoints

    • XMLA
    • Datastore REST API
    • Cube REST API
    • Cube Websocket API

    MDX

    • MDX Engine Configuration
    • MDX Functions
    • MDX Operators
    • MDX Formats
    • MDX Filtering
    • MDX Snippets
    • MDX Cellsets
  • Datastore Queries
  • Location-Based Queries
  • Drillthrough Extensions

Configuration

  • Overview
  • ContentServer

    • Content Server
    • ContentServer REST API
    • CS Websocket API
  • ActivePivot Properties
  • Internationalization

Security

  • Overview
  • Client/Server Communication

    • Authentication
    • Authorization & Entitlements

    Data Access Control

    • Datastore Access Control
    • ActivePivot Access Control
    • Branch Permission Manager

Distributed Architecture

  • Overview
  • Communication Flows
  • Post-Processors
  • Security
  • What-If
  • Recommendations
  • Distribution Properties

Operations

  • Overview
  • Monitoring

    • Health Dispatcher
    • Query Execution Plan
    • Monitoring Query Execution
    • JMX monitoring
    • Off-Heap Memory Export
    • Tracing REST API
  • Troubleshooting
  • Performance
  • High Availability

Release & Migration Notes

  • Changelog
  • Migration notes
  • Deprecated features

Reference

  • Javadoc
  • REST APIs

Cube Configuration

ActivePivot managers, schemas and cubes can be described by using fluent builders, which allows you to configure everything with concise and readable Java code.

These builders follow the same principles as the Datastore fluent builders:

  • Multiple interfaces guide (e.g. code completion in IDE) the developer through the configuration steps.
  • There is a de-facto compile-time consistency check of what the developer does.

The StartBuilding class contains all the methods to start building any object with the fluent builder interfaces, including the datastore builders.

A given cube description contains several elements, either mandatory (e.g. cube dimensions & hierarchies) or optional (e.g. some of the shared context elements).

Manager, Schemas and Cubes

Any ActivePivot application needs an ActivePivot Manager that hold a reference to an OLAP "catalog" of the cubes exposed to remote clients, and (for internal purpose) also holds references to one or multiple "schemas" (fact model definition) and underlying "cubes" (where those facts contribute).

From a high-level perspective, here is a very minimal example of ActivePivot manager configuration...

ISelectionDescription selection = StartBuilding.selection()
        .fromBaseStore("store")
        .withField("field")
        .build();
IActivePivotManagerDescription desc = StartBuilding.managerDescription()
        .withSchema()
        .withSelection(selection)
        .withCube("Cube", b -> b.withSingleLevelDimension("field")).build();

You may have noticed that in the above description neither the manager nor the schema have names, and we didn't even explicitly configure a catalog... There are methods to choose names for managers, schemas and define catalogs, but if one doesn't choose to use them then the builder will use some default names (i.e. "Manager", "Catalog" and "Schema") and include in the catalog all the cubes that have been configured.

In the above example we chose to explicitly define the fields of the selection (a.k.a. the fact model)... But the fluent builder comes with convenient shortcut methods to help create your selections much faster. Hence, you can now simply create a selection that consist of dozens of fields automatically "denormalized" from a given Datastore mode, with for instance:

StartBuilding.selection(datastoreDescription)
        .fromBaseStore("risk")
        .withAllReachableFields()
        .withAlias("UnderlierCurrency", "Currency")
        .withAlias("ProductId", "productId")
        .build();

The javadoc of com.activeviam.desc.build.ISelectionDescriptionBuilder provides an API that can be used to simply create your selections.

For the catalog, if you don't declare one, one will automatically be created, containing all cubes and called "Catalog". Additionally, the containingAllCubes() method of the catalog builder is an easy way to just change the name of the single Catalog containing all the cubes (this one appears in UIs as opposed to the manager and schemas names).

As another example, here is a slightly more comprehensive configuration of an ActivePivot manager:

StartBuilding.managerDescription()
        .withSchema("A")
            .withSelection(StartBuilding.selection().fromBaseStore("a").withField("f").build())
            .withCube("C", b -> b.withSingleLevelDimension("f"))
        .withSchema("B")
            .withSelection(StartBuilding.selection().fromBaseStore("a").withField("f").build())
            .withCube("C2", b -> b.withSingleLevelDimension("f"))
        .withCatalog("C")
            .containingCubes("C")
        .withCatalog("Main")
            .containingAllCubes()
        .build();

Cube Configuration

A cube description can be simply built with the following code:

final IActivePivotInstanceDescription desc = StartBuilding.cube()
        .withName("Cube")
        .withSingleLevelDimension("d")
        .build();

This is the shortest cube description one can have. This cube will have a single dimension with a single hierarchy containing a single level and only have the native measures.

Of course your project will usually have multiple measures and dimensions, customize the aggregate provider, and will more look like:

ICanStartBuildingDimensions.DimensionsAdder dimensions = b -> b
        .withDimension("Underlyings")
            .withHierarchyOfSameName()
                .withLevels("UnderlierType", "UnderlierCode")
            .withHierarchy("Products")
                .asDefaultHierarchy()
                .withLevels("ProductType", "ProductName")
        .withSingleLevelDimension("HostName")
        .withDimension("Currency")
            .withHierarchyOfSameName()
                .withLevelOfSameName()
                    .withFirstObjects("EUR", "GBP", "USD", "JPY")
        .withDimension("CounterParty")
            .withHierarchyOfSameName()
                .withLevel("CouterPartyGroup")
                .withLevel("CounterParty")
        .withDimension("Geography")
            .withHierarchy("City")
                .withLevelOfSameName()
                    .withMemberProperties(
                            new PropertyInfo("Latitude", "latitude"),
                            new PropertyInfo("Longitude", "longitude"))
        .withDimension("Trades")
            .withHierarchy("Trades")
                .withLevel("TradeId")
        .withEpochDimension()
            .withEpochLevel()
                .withComparator("REVERSE_EPOCH")
                .withFormatter("EPOCH[HH:mm:ss]")
                .end();
Function<ICanStartBuildingMeasures, IHasAtLeastOneMeasure> measures = b -> b
        .withContributorsCount()
            .withAlias("Count")
            .withFormatter("INT[#,###]")
        .withAggregatedMeasure()
            .sum("pnl")
            .withinFolder("pnl")
            .withFormatter("DOUBLE[#,###.00;-#,###.00]")
        .withAggregatedMeasure()
            .sum("delta")
            .withinFolder("sensitivities")
            .withFormatter("DOUBLE[#,###.00;-#,###.00]");
StartBuilding.cube()
        .withName("MyCube")
        .withMeasures(measures)
        .withDimensions(dimensions)
        .withAggregateProvider()
            .jit()
            .withProperty("rangeSharing", "true")
            .withPartialProvider()
                .bitmap()
                .excludingHierarchy("Trades", "Trades").end()
                .includingOnlyMeasures("pnlDelta.SUM", "pnl.SUM")
            .withPartialProvider()
                .bitmap()
                .includingOnlyHierarchy("Underlyings", "Underlyings")
                    .and("HistoricalDates", "HistoricalDates").end()
            .withPartialProvider()
                .excludingMeasures("pnlDelta.SUM", "pnl.SUM")
        .withDrillthroughExecutor()
            .withKey("TIME_BUCKET_DT_EXECUTOR")
            .withProperty("bucketHierarchy", "TimeBucketDynamic")
            .withProperty("bucketedLevel", "Value Date")
            .end()
        .withAggregatesCache()
            .withSize(10_000)
            .cachingOnlyMeasures("contributors.COUNT", "pnl.SUM")
        .withSharedContextValue(new DrillthroughProperties(
                new CustomComparator<String>(List.of("Desk", "Currency"), null),
                List.of("BookId", "City"),
                List.of(new CalculatedDrillthroughColumnDescription(
                        "DoubleAdder",
                        "delta + gamma",
                        "delta,gamma")),
                List.of(new CalculatedDrillthroughColumnSetDescription(
                        "PnlCurrencyColumnSet",
                        Immutable.map(
                                "prefix",
                                "pnl in",
                                "currencies",
                                "EUR,USD,GBP,JPY,CHF,ZAR").toProperties()))))
        .withSharedContextValue(new MdxContext())
        .build();

You will note that in this description the list of dimensions and measures are given by a separate function. That allows you to split it in multiple blocks/beans when you have hundreds of dimensions and measures in your cube.

These functions can also call more functions, adding blocks of dimensions and measures, to allow measures and dimensions declarations to be grouped or split at will.

Aggregation Functions

There are several ways to create aggregated measures based on raw values as shown below.

The first method uses core aggregation functions (sum, min, max, avg) as specific method names on a named schema field.

.withAggregatedMeasure()
    .sum("SchemaField")
    .withinFolder("pnl")
    .withFormatter("DOUBLE[#,###.00;-#,###.00]")

The second method uses core aggregation functions (longSum, shortSum, squareSum) as specific method names after the schema field is given.

.withAggregatedMeasure()
    .withField("SchemaField")
    .shortSum()

The third method uses the plugin key for the aggregation function, whether it's a custom aggregation function or a core aggregation function.

.withAggregatedMeasure()
    .withField("SchemaField")
    .withAggregationFunction(CustomAggregationFunction.PLUGIN_KEY)
.withAggregatedMeasure()
    .withField("SchemaField")
    .withAggregationFunction(SumFunction.PLUGIN_KEY)

Copper

Additional measures and hierarchies can be defined through the Copper API.

← OverviewIntroduction →
  • Manager, Schemas and Cubes
  • Cube Configuration
  • Aggregation Functions
  • Copper
ActivePivot
Community
Stack OverflowLinkedinTwitter
More
Blog
Copyright © 2021 ActiveViam