Context Values

Introduction

The context values allow to change the behavior/results of the user's queries. The concept has been originally introduced in ActivePivot to handle security (through sub-cube properties) and user-related properties.

For example: users in Asia want all amounts (expressed in different currencies) to be converted in USD whereas users in Europe want them converted in EUR.

The context allows defining a set of context values that are particular to the execution of a particular query, on a particular cube, by a particular user.

The context values effectively assigned for each execution context depend on what has been configured/assigned in various layers of ActivePivot, from the ones defined for the whole application to those specific to a query.

In the standard MDX/OLAP world there is no notion of context value but ActivePivot brings in this extra feature.

There are standard context values shipped with ActivePivot, but one can also implement one's own custom custom context values.

Standard Context Values

The following context values are shipped with the core product.

IQueriesTimeLimit

In ActivePivot, any query is interrupted if it exceeds a certain execution time, which can be defined via the IQueriesTimeLimit context value.

IQueriesResultLimit

This context value allows to limit the number of locations that are retrieved during a query.

This is different from the resultLimit in the IMdxContext. While the resultLimit indicates the maximal number of cells in the result of queries, that limit is applied after query evaluation.

The IQueriesResultLimit context value, on the other hand, is meant to limit the memory impact of computing a query:

• intermediateLimit allows to set the limit number of locations for each single intermediate result of the query
• transientLimit allows to set the transient limit resulting from the accumulation of all the intermediate results within a single query

ISubCubeProperties

This context value is used to forbid the access to data and/or measures. In ActivePivot, a sub-cube view is a restriction of a real hypercube where some members are hidden and sub-totals aggregated "on-the-fly".

There are 3 major types of restriction one can fine tune:

• Give an access to the cube: If not granted, the cube will be seen as entirely empty.
• Grant several axis members: If nothing done, all members are granted by default. If some members are granted, only those granted members are visible, all the siblings are excluded.
• Grant several measure members: When no measure is explicitly granted, then all available measures are granted. If some measures are explicitly granted, then all the remaining measures are hidden. When you want to hide a measure you must actually grant access to all the remaining measures.

Here is an example from the sandbox project that grants only one Desk member:

StartBuilding.subCubeProperties()
        .grantMembers()
        .onDimension(BOOKING_DIMENSION)
        .onHierarchy(TRADE__DESK)
        .memberPath(ILevel.ALLMEMBER)
        .memberPath("LegalEntityA")
        .memberPath("BusinessUnitA")
        .memberPath("DeskA")
        .build();

This subcube property is then put in the context of users with role "ROLE_DESK_A" so they can only see the desk they are working on.

It is also possible to use advanced conditions to grant members, allowing easier write of "all but..." conditions for example.

Here is an example to filter out a single member:

StartBuilding.subCubeProperties()
        .grantMembers()
        .onDimension("Employees")
        .onHierarchy("Employees")
        .memberPath(ILevel.ALLMEMBER)
        .memberPath(new NotCondition(new EqualCondition("IAmInvisible")))
        .build();

And another example to filter out several members:

StartBuilding.subCubeProperties()
        .grantMembers()
        .onDimension("Employees")
        .onHierarchy("Employees")
        .memberPath(ILevel.ALLMEMBER)
        .memberPath(
                new NotCondition(
                        new com.quartetfs.fwk.filtering.impl.InCondition(
                                "Andrew Fuller",
                                "Anne Dodsworth")))
        .build();

In a distributed context, subcube properties from a query cube are intersected with the subcube properties of data cubes at query time.

As a result, if the query cube's context grants access to members A and B, and the data cube's context grants access only to member A, the query will be executed with only member A as granted member.

IMdxContext

The IMdxContext context value allows configuring the MDX Engine.

IDrillthroughProperties

Drill-through properties are used when processing a drill-through query. They allow hiding and ordering columns, and possibly excluding some objects if they are undesirable into users' blotters.

By default, ActivePivot considers that all the fields used in the cube levels and measures have to be displayed as drill-through result columns.

The default behavior is unlikely to be universally relevant for every end-user, so it is very common that one must configure drill-through context values to fine tune the drill-through behavior on a per-cube basis and/or per-user basis.

The drill-through properties can be defined using the fluent builder (example taken from the sandbox project):

StartBuilding.drillthroughProperties()
        // Setting some columns first in order based on their names
        .withHeaderComparator(
                new CustomComparator<>(Arrays.asList(TRADE__DESK, CURRENCY), Collections.emptyList()))
        .hideColumn(TRADE__BOOK_ID)
        .hideColumn(CITY_OBJECT)
        .withCalculatedColumn()
        .withName("delta + gamma")
        .withPluginKey(DoubleAdderColumn.PLUGIN_KEY)
        .withUnderlyingFields("delta", "gamma")
        .end()
        .withCalculatedColumn()
        .withName("Book ID")
        .withPluginKey(BookIdColumn.PLUGIN_KEY)
        .withUnderlyingFields(TRADE__BOOK_ID)
        .end()
        .withCalculatedColumn()
        .withName("City name")
        .withPluginKey(CityNameColumn.PLUGIN_KEY)
        .withUnderlyingFields(CITY_OBJECT)
        .end()
        .withCalculatedColumnSet()
        .withPluginKey(PnlCurrencyColumnSet.PLUGIN_KEY)
        .withProperty("prefix", "pnl in")
        .withProperty("currencies", "EUR,USD,GBP,JPY,CHF,ZAR")
        .end()
        // Hard limit for the number of rows returned by drillthrough queries
        .withMaxRows(10000)
        .build();

• withHeaderComparator allows ordering columns in a drillthrough.
• hideColumn allows to hide non-relevant data.
• withCalculatedColumn and withCalculatedColumnSet allow adding columns in the drillthrough that do not exist in the datastore and are computed on-the-fly during a drillthrough query.
• withMaxRows specifies the maximum number of rows that drillthrough queries are authorized to return. If this context value is set and the query engine realizes a drillthrough query will produce more rows than allowed, an exception is thrown.

It is also possible to define the error behavior, when an error occurs in a calculated column, with setBehavior. setUnknownColumnBehavior defines the error behavior in another error case, when a column is asked in the drillthrough but does not exist. For both error behavior settings, there exist three values: silent, warning or throw an exception.

IQueryCache

The query cache is a map that stores custom data during queries.

Never set this context value! (API or configuration).
The query cache differs from other context values as it is completely handled by ActivePivot. The query cache is a place holder that you can use for caching data during the lifespan of a query.

Note - The query cache is specific to each query / not shared across queries (however, it is shared across threads processing parts of the same query - typically, during post-processing).

In case of a stream event triggering computations, the query cache's lifespan extends from the computation of the impact by the handlers to the subsequent computations of update values.

Because of the multi-threaded nature of those steps, the query cache is thread safe. In fact, it is a java.util.concurrent.ConcurrentMap. As such, it lets you perform operations like pufIfAbsent.

There are two typical use cases for the query cache:

• ensuring consistency in the query when fetching external data. Suppose for example that a post-processor uses a market data. This market data might change while the query computes. The query cache gives you the possibility to ensure that all calls to that post-processor (within the execution of a same query) use the same market data by caching it (using the putIfAbsent method).
• avoiding redundant processing. Suppose for example that each call to a post-processor (whatever the location it's evaluated on) requires a same computation to be run. Using the query cache, the result of that calculation can be cached to avoid redundant processing in subsequent calls to that post-processor (within the same overall query).

The query cache can be retrieved from within a post-processor by calling :

final IQueryCache queryCache = pivot.getContext().get(IQueryCache.class);

IMissedPrefetchBehavior

This context value describes the behavior of the ActivePivot query engine when it detects that an IPostProcessor does not correctly expose its prefetchers.

If set to SILENT, the measure needed for post-processing that was not exposed by the post-processor's prefetchers will be retrieved at compute time, resulting in poor performance. If set to WARN, the computation will be the same but a warning will be printed in the logs so that the performance hit does not get completely ignored. If set to THROW (recommended setting while developing), the query will throw, allowing to immediately spot the problem and fix the postprocessor.

How to Implement a Custom Context Value

In customer projects, customized context values are often used to give to the end-users the ability to remotely configure the computation done by particular post-processors. For example, for Market Risk analysis in finance, an end-user would be given the ability to change (via some UI action) the confidence level taken into account by the post-processor computing the VaR (Value-At-Risk) measure requested via usual MDX query.

Fully implementing custom context values requires several tasks that are not necessarily described in detail here. However, we aim at giving you a pretty good overview of all the different bits of customized code one would need to write. Hence, we mostly detail the starting point: defining and implementing the custom context value on ActivePivot server side.

So, sticking to our example, we show how to implement a customized confidence level context value. Then, we also give an overview of all the further customizations you can base on your new context value (accessing it from within a post-processor evaluation, configuring a default value for it, having a web service method for remotely interacting with it, ...).

Define a Dedicated Interface

Any context value in ActivePivot must implement the com.quartetfs.biz.pivot.context.IContextValue interface because this is how ActivePivot generically manipulates context value objects internally. Moreover, your customized context value must be defined via a proper dedicated interface, which constitutes a unique identifier of what your context value relates to. Typically, this is what allows ActivePivot to make the difference between different types of IContextValue objects.

In our example, we want to store the confidence level as a double value (e.g. "95.0" for meaning a 95% confidence level):

public interface IVarConfidenceLevel extends IContextValue {
    double getConfidenceLevel();
}

Start the Actual Implementation

You have to derive your customized implementation from the abstract class com.quartetfs.biz.pivot.context.impl.AContextValue (shipped with core product), and obviously, to implement also the interface you defined for it (IVarConfidenceLevel in our example).

Note that here we said "you have to", instead of "we strongly recommend you to..." as we do usually for other abstract classes we provide for helping customization. This is because of some constraints related to JAXB serialization. No need to say much more here: we elaborate further about this in a dedicated section.

Extending AContextValue also helps you remember important things that must be in your implementation besides what you already know (e.g. getConfidenceLevel() in our example), and especially...

When implementing a custom context value you must implement consistently the equals(...) and hashCode() methods

This is crucial for ensuring that internal ActivePivot management of your context values is correct/safe. For example, in the real-time/streaming layer: 2 streams might be attached to the same stream node because their associated contexts seem the same while they are actually different! Say, if you implement equals/hashCode wrongly it can significantly mess up with the internal logic responsible for efficient sharing of computation load between 'similar' queries. These issues are quite advanced and very tough to debug, so it's critical that the implementation is careful about that in the first place.

Remember that your favorite IDE can probably assist you in building proper equals and hashCode methods.

Finally, as IContextValue extends IClone, your implementation shall take care of having a proper clone() method too. However, you would only need to consider overriding AContextValue.clone() if dealing with mutable fields: see the section about immutability concerns for more details.

So, back to our confidence level example, we end up with the following implementation:

public static final class VarConfidenceLevel extends AContextValue implements IVarConfidenceLevel {
    private static final long serialVersionUID = 5987321732634610588L;
    private final double confidenceLevel;
    public VarConfidenceLevel(double confidenceLevel) {
        this.confidenceLevel = confidenceLevel;
    }
    @Override
    public Class<? extends IContextValue> getContextInterface() {
        return IVarConfidenceLevel.class;
    }
    @Override
    public double getConfidenceLevel() {
        return confidenceLevel;
    }
    @Override
    public int hashCode() {
        final int prime = 31;
        int result = 1;
        long temp;
        temp = Double.doubleToLongBits(confidenceLevel);
        result = prime * result + (int) (temp ^ (temp >>> 32));
        return result;
    }
    @Override
    public boolean equals(Object obj) {
        if (this == obj) {
            return true;
        }
        if (obj == null) {
            return false;
        }
        if (getClass() != obj.getClass()) {
            return false;
        }
        VarConfidenceLevel other = (VarConfidenceLevel) obj;
        if (Double.doubleToLongBits(confidenceLevel) != Double.doubleToLongBits(other.confidenceLevel)) {
            return false;
        }
        return true;
    }
}

Ensure Immutability

Context values must be immutable.

This is something very important to enforce, otherwise you may experience unexpected (and hard to troubleshoot) behaviors in ActivePivot.

Usually, the way to ensure that a given object is immutable is (taken from http://download.oracle.com/javase/tutorial/essential/concurrency/imstrat.html):

• Do not provide "setter" methods — methods that modify fields or objects referred to by fields.
• Make all fields final and private.
• Do not allow subclasses to override methods. The simplest way to do this is to declare the class as final. A more sophisticated approach is to make the constructor private and construct instances in factory methods.
• If the instance fields include references to mutable objects, do not allow those objects to be changed:
  • Do not provide methods that modify the mutable objects.
  • Do not share references to the mutable objects. Never store references to external, mutable objects passed to the constructor; if necessary, create copies, and store references to the copies. Similarly, create copies of your internal mutable objects when necessary to avoid returning the originals in your methods.

The last point listed above might be the more subtle to check, but in short it means: be very careful if your custom implementation has some field(s) whose type(s) is(are) other than a Java primitive type or String.

Typically,

• When doing the construction of your custom context value object, you shall take care of not keeping any reference to some mutable object passed to the constructor.
• Don't forget that your custom context value implementation must support a proper cloning... AContextValue already implements a default clone() behavior, but it is only sufficient for supporting cases of custom context value with obvious immutable fields (primitive, String).

Our example of the confidence level context value is relatively easy to manage in that respect: all we need to ensure is that the class is final and that the confidenceLevel field is final.

Add JAXB support

JAXB allows for XML serialization of the context value. It is used in several features : saving the context values attached to roles in the content server, saving the whole cube description in an XML file, modifying context values via SOAP services...

In ActivePivot we use a particular "enhanced" JAXB implementation, which is relying on JAXBIntroductions. It enables configuring JAXB serialization through XML configuration files instead of the traditional way using annotations in the Java code. Thus, one can really decouple the Java implementation from the serialization logic.

JAXB implementation requirements

Unfortunately, JAXB serialization introduces some requirements that are not 100% compatible with the immutability topic we discussed earlier in this article. Indeed, JAXB is designed for serializing Java "beans", not really immutable objects. Consequently, each class that you want to equip with support for JAXB serialization requires (at least):

• a private no-arg constructor
• a private setter for each field that shall be mapped with an XML attribute

So, in the VarConfidenceLevel example, you would remove the final modifier for the confidenceLevel field, add a private no-arg constructor, and add a private setter for the confidenceLevel field.

public static final class VarConfidenceLevel extends AContextValue implements IVarConfidenceLevel {
    private static final long serialVersionUID = 5987321732634610588L;
    private double confidenceLevel;
    public VarConfidenceLevel(double confidenceLevel) {
        this.confidenceLevel = confidenceLevel;
    }
    /**
     * Private empty constructor for JAXB compatibility.
     */
    private VarConfidenceLevel() {
    }
    /**
     * Private setter for JAXB compatibility
     *
     * @param confidenceLevel the confidence level to set for VaR computation.
     */
    private void setConfidenceLevel(double confidenceLevel) {
        this.confidenceLevel = confidenceLevel;
    }
    // ... the rest of the implementation

Theoretically speaking, we don't respect anymore the contract for immutability (the best proof of that is that JAXB serialization can first create the object with default constructor, and then call the setter to change its state). That said, because of the field is still private and - most importantly - because our setter remains private, we can consider it is quite a small breach of the traditional immutability contract. The object is not mutated after its first complete initialization.

In short, our implementation is both JAXB-compatible and also preventing third-party developers to have any obvious write access to the object state once instantiated.

JAXB configuration

You need to place a serializer.jaxb.cfg.xml file at the root of the project resources/ folder. In this file you can define the paths to your different JAXB custom configurations (you can use any path defined relatively to the root of the resources/ folder). There is one such file in the sandbox project, which references the standard ActivePivot context values JAXB configurations.

In our example, we define only the mapping related to the custom confidence level context value object.

The resources/serializer.jaxb.cfg.xml file would look like:

<properties>
    <entry key="ActivePivot-Custom-ConfidenceLevelContext">JAXB-INF/pivot.varconfidencelevel-jaxb-mapping.xml</entry>
</properties>

And the resources/JAXB-INF/pivot.varconfidencelevel-jaxb-mapping.xml file would look like:

<jaxb-intros xmlns="http://www.jboss.org/xsd/jaxb/intros" xsi:noNamespaceSchemaLocation="http://anonsvn.jboss.org/repos/jbossws/projects/jaxbintros/trunk/src/main/resources/jaxb-intros.xsd">
    <Class name="com.abc.mypackage.VaRConfidenceLevel">
        <XmlRootElement name="VarConfidenceLevel" />
        <XmlType />
        <Method name="getConfidenceLevel">
            <XmlAttribute name="confidenceLevel" required="true" />
        </Method>
    </Class>
</jaxb-intros>

Plug the ContextValue in ActiveUI

For a ContextValue to appear in ActiveUI, a plugin implementing IContextValueTranslator needs to be created on the server side.

When the context value has only a single property, its translator can simply extend the SimpleContextValueTranslator:

@QuartetPluginValue(intf = IContextValueTranslator.class)
public static class VarConfidenceLevelTranslator
        extends SimpleContextValueTranslator<String, IVarConfidenceLevel> {
    /** Translator key. */
    public static final String KEY = "VarConfidenceLevelTranslator";
    // ... implementation of format, parse, createInstance,
    // getContent and other abstract methods

If the context value has several properties, its translator must extend MultipleContextValueTranslator: the method computeAvailableProperties() exposes the properties of the context value that can be seen and changed via ActiveUI.

If user wants to hide one or some properties, he can simply omit those properties from the result of this method.

Define the default value for any context value in ActivePivot

We can choose one or several (in combination) of the following configuration options:

• Define a default value for a given cube, via the "shared context". This is configured when configuring the cube:

  StartBuilding.cube("CUBE")
          .withDimension("myDimension")
          .withHierarchyOfSameName()
          .withLevel("myLevel")
          .withSharedContextValue(new VarConfidenceLevel(0.95))
          .build();

• Define a global (understand: applying to all cubes) default value for a given user role. This is configured via the entitlements:

  new SimpleEntitlementsProviderBuilder().withGlobalEntitlement()
          .forRole("ROLE_HIGH_CONFIDENCE")
          .withContextValue(new VarConfidenceLevel(0.95))
          .build();

• Define a default value for a given user role and a particular cube. This is configured via the entitlements:

  new SimpleEntitlementsProviderBuilder().withPivotEntitlement()
          .forRole("ROLE_LOW_CONFIDENCE_ON_A")
          .onPivot("CUBE_A")
          .withContextValue(new VarConfidenceLevel(0.9))
          .build();

Read a Context Value From Within a Post-Processor Evaluation

Before accessing a context value, a post-processor must declare its dependency to it, which can be done via calling com.quartetfs.biz.pivot.postprocessing.impl.APostProcessor.addContextDependency method. If this is not done properly, one will trigger some specific error message when trying to execute the post-processing.

The context values that are used in the post-processor are declared because of a performance enhancement feature that allows sharing measures across multiple queries. That is only possible if the measure has the same value in each query: declaring the context values used in each post-processor allows the query engine to know that it can't share measures when their relevant context values are not equal (thus also the need to be careful about equals and hashcode implementations).

public class VaRPostProcessor extends ABasicPostProcessor<Double> {
    // ... some code...
    @Override
    public void init(Properties properties) throws ActiveViamException {
        super.init(properties);
        // ... some code...
        addContextDependency(IVarConfidenceLevel.class);
    }
    // ... some code...

Once the context value used is declared as a dependency, it can be read during the post-pro evaluation, by retrieving it from the actual context values seen by the current cube proxy (IActivePivot pivot instance) dedicated to the query:

double getConfidenceLevel() throws ActiveViamException {
    IVarConfidenceLevel confidenceLevelContext = pivot.getContext().get(IVarConfidenceLevel.class);
    if (confidenceLevelContext == null) {
        // Do what you want here (throw or return a default value)
    }
    return confidenceLevelContext.getConfidenceLevel();
}