Context Values

Introduction

The context values allow to change the behavior/results of the user's queries. The concept has been originally introduced in ActivePivot to handle security (through sub-cube properties) and user-related properties.

For example: users in Asia want all amounts (expressed in different currencies) to be converted in USD whereas users in Europe want them converted in EUR.

The context allows defining a set of context values that are particular to the execution of a particular query, on a particular cube, by a particular user.

The context values effectively assigned for each execution context depend on what has been configured/assigned in various layers of ActivePivot, from the ones defined for the whole application to those specific to a query.

In the standard MDX/OLAP world there is no notion of context value but ActivePivot brings in this extra feature.

There are standard context values shipped with ActivePivot, but one can also implement one's own custom custom context values.

Standard Context Values

The following context values are shipped with the core product.

IQueriesTimeLimit

In ActivePivot, any query is interrupted if it exceeds a certain execution time, which can be defined via the IQueriesTimeLimit context value.

IQueriesResultLimit

This context value allows to limit the number of locations that are retrieved during a query.

This is different from the resultLimit in the IMdxContext. While the resultLimit indicates the maximal number of cells in the result of queries, that limit is applied after query evaluation.

The IQueriesResultLimit context value, on the other hand, is meant to limit the memory impact of computing a query:

• intermediateLimit allows to set the limit number of locations for each single intermediate result of the query
• transientLimit allows to set the transient limit resulting from the accumulation of all the intermediate results within a single query

ISubCubeProperties

This context value is used to forbid the access to data and/or measures. In ActivePivot, a sub-cube view is a restriction of a real hypercube where some members are hidden and sub-totals aggregated "on-the-fly".

There are 3 major types of restriction one can fine tune:

• Give an access to the cube: If not granted, the cube will be seen as entirely empty.
• Grant several axis members: If nothing done, all members are granted by default. If some members are granted, only those granted members are visible, all the siblings are excluded.
• Grant several measure members: When no measure is explicitly granted, then all available measures are granted. If some measures are explicitly granted, then all the remaining measures are hidden. When you want to hide a measure you must actually grant access to all the remaining measures.

Here is an example from the sandbox project that grants only one Desk member:

final var subCubeProperties = new SubCubeProperties(true);
subCubeProperties.grantMembers(
    BOOKING_DIMENSION,
    TRADE__DESK,
    List.of(ILevel.ALLMEMBER, "LegalEntityA", "BusinessUnitA", "DeskA"));

This subcube property is then put in the context of users with role "ROLE_DESK_A" so they can only see the desk they are working on.

It is also possible to use advanced conditions to grant members, allowing easier write of "all but..." conditions for example.

Here is an example to filter out a single member:

final var subCubeProperties = new SubCubeProperties(true);
subCubeProperties.grantMembers(
    "Employees",
    "Employees",
    List.of(ILevel.ALLMEMBER, new NotCondition(new EqualCondition("IAmInvisible"))));

And another example to filter out several members:

subCubeProperties.grantMembers(
    "Employees",
    "Employees",
    List.of(
        ILevel.ALLMEMBER,
        new NotCondition(new InCondition("Andrew Fuller", "Anne Dodsworth"))));

In a distributed context, subcube properties from a query cube are intersected with the subcube properties of data cubes at query time.

As a result, if the query cube's context grants access to members A and B, and the data cube's context grants access only to member A, the query will be executed with only member A as granted member.

IMdxContext

The IMdxContext context value allows configuring the MDX Engine.

IDrillthroughProperties

Drill-through properties are used when processing a drill-through query. They allow hiding and ordering columns, and possibly excluding some objects if they are undesirable into users' blotters.

By default, ActivePivot considers that all the fields used in the cube levels and measures have to be displayed as drill-through result columns.

The default behavior is unlikely to be universally relevant for every end-user, so it is very common that one must configure drill-through context values to fine tune the drill-through behavior on a per-cube basis and/or per-user basis.

The drill-through properties can be defined using the fluent builder (example taken from the sandbox project):

StartBuilding.drillthroughProperties()
    // Setting some columns first in order based on their names
    .withHeaderComparator(
        new CustomComparator<>(Arrays.asList(TRADE__DESK, CURRENCY), Collections.emptyList()))
    .hideColumn(TRADE__BOOK_ID)
    .hideColumn(CITY_OBJECT)
    .withCalculatedColumn()
    .withName("delta + gamma")
    .withPluginKey(DoubleAdderColumn.PLUGIN_KEY)
    .withUnderlyingFields("delta", "gamma")
    .end()
    .withCalculatedColumn()
    .withName("Book ID")
    .withPluginKey(BookIdColumn.PLUGIN_KEY)
    .withUnderlyingFields(TRADE__BOOK_ID)
    .end()
    .withCalculatedColumn()
    .withName("City name")
    .withPluginKey(CityNameColumn.PLUGIN_KEY)
    .withUnderlyingFields(CITY_OBJECT)
    .end()
    .withCalculatedColumnSet()
    .withPluginKey(PnlCurrencyColumnSet.PLUGIN_KEY)
    .withProperty("prefix", "pnl in")
    .withProperty("currencies", "EUR,USD,GBP,JPY,CHF,ZAR")
    .end()
    // Hard limit for the number of rows returned by drillthrough queries
    .withMaxRows(10000)
    .build();

• withHeaderComparator allows ordering columns in a drillthrough.
• hideColumn allows to hide non-relevant data.
• withCalculatedColumn and withCalculatedColumnSet allow adding columns in the drillthrough that do not exist in the datastore and are computed on-the-fly during a drillthrough query.
• withMaxRows specifies the maximum number of rows that drillthrough queries are authorized to return. If this context value is set and the query engine realizes a drillthrough query will produce more rows than allowed, an exception is thrown.

It is also possible to define the error behavior, when an error occurs in a calculated column, with setBehavior. setUnknownColumnBehavior defines the error behavior in another error case, when a column is asked in the drillthrough but does not exist. For both error behavior settings, there exist three values: silent, warning or throw an exception.

IQueryCache

The query cache is a map that stores custom data during queries.

Never set this context value! (API or configuration).
The query cache differs from other context values as it is completely handled by ActivePivot. The query cache is a placeholder that you can use for caching data during the lifespan of a query.

Note - The query cache is specific to each query / not shared across queries (however, it is shared across threads processing parts of the same query - typically, during post-processing).

In case of a stream event triggering computations, the query cache's lifespan extends from the computation of the impact by the handlers to the subsequent computations of update values.

Because of the multithreaded nature of those steps, the query cache is thread safe. In fact, it is a java.util.concurrent.ConcurrentMap. As such, it lets you perform operations like pufIfAbsent.

There are two typical use cases for the query cache:

• ensuring consistency in the query when fetching external data. Suppose for example that a post-processor uses a market data. This market data might change while the query computes. The query cache gives you the possibility to ensure that all calls to that post-processor (within the execution of a same query) use the same market data by caching it (using the putIfAbsent method).
• avoiding redundant processing. Suppose for example that each call to a post-processor (whatever the location it's evaluated on) requires a same computation to be run. Using the query cache, the result of that calculation can be cached to avoid redundant processing in subsequent calls to that post-processor (within the same overall query).

The query cache can be retrieved from within a post-processor by calling :

final IQueryCache queryCache = pivot.getContext().get(IQueryCache.class);

IMissedPrefetchBehavior

This context value describes the behavior of the ActivePivot query engine when it detects that an IPostProcessor does not correctly expose its prefetchers.

If set to SILENT, the measure needed for post-processing that was not exposed by the post-processor's prefetchers will be retrieved at compute time, resulting in poor performance. If set to WARN, the computation will be the same but a warning will be printed in the logs so that the performance hit does not get completely ignored. If set to THROW (recommended setting while developing), the query will throw, allowing to immediately spot the problem and fix the postprocessor.

How to Implement a Custom Context Value

In customer projects, customized context values are often used to give to the end-users the ability to remotely configure the computation done by particular post-processors. For example, for Market Risk analysis in finance, an end-user would be given the ability to change (via some UI action) the confidence level taken into account by the post-processor computing the VaR (Value-At-Risk) measure requested via usual MDX query.

Fully implementing custom context values requires several tasks that are not necessarily described in detail here. However, we aim at giving you a pretty good overview of all the different bits of customized code one would need to write. Hence, we mostly detail the starting point: defining and implementing the custom context value on ActivePivot server side.

So, sticking to our example, we show how to implement a customized confidence level context value. Then, we also give an overview of all the further customizations you can base on your new context value (accessing it from within a post-processor evaluation, configuring a default value for it, having a web service method for remotely interacting with it, ...).

Define a Dedicated Interface

Any context value in ActivePivot must implement the com.quartetfs.biz.pivot.context.IContextValue interface because this is how ActivePivot generically manipulates context value objects internally. Moreover, your customized context value must be defined via a proper dedicated interface, which constitutes a unique identifier of what your context value relates to. Typically, this is what allows ActivePivot to make the difference between different types of IContextValue objects.

In our example, we want to store the confidence level as a double value (e.g. "95.0" for meaning a 95% confidence level):

public interface IVarConfidenceLevel extends IContextValue {
  double getConfidenceLevel();
}

Start the Actual Implementation

You have to derive your customized implementation from the abstract class com.quartetfs.biz.pivot.context.impl.AContextValue (shipped with core product), and obviously, to implement also the interface you defined for it (IVarConfidenceLevel in our example).

Note that here we said "you have to", instead of "we strongly recommend you to..." as we do usually for other abstract classes we provide for helping customization. This is because of some constraints related to JAXB serialization. No need to say much more here: we elaborate further about this in a dedicated section.

Extending AContextValue also helps you remember important things that must be in your implementation besides what you already know (e.g. getConfidenceLevel() in our example), and especially...

When implementing a custom context value you must implement consistently the equals(...) and hashCode() methods

This is crucial for ensuring that internal ActivePivot management of your context values is correct/safe. For example, in the real-time/streaming layer: 2 streams might be attached to the same stream node because their associated contexts seem the same while they are actually different! Say, if you implement equals/hashCode wrongly it can significantly mess up with the internal logic responsible for efficient sharing of computation load between 'similar' queries. These issues are quite advanced and very tough to debug, so it's critical that the implementation is careful about that in the first place.

Remember that your favorite IDE can probably assist you in building proper equals and hashCode methods.

Finally, as IContextValue extends IClone, your implementation shall take care of having a proper clone() method too. However, you would only need to consider overriding AContextValue.clone() if dealing with mutable fields: see the section about immutability concerns for more details.

So, back to our confidence level example, we end up with the following implementation:

public static final class VarConfidenceLevel extends AContextValue
    implements IVarConfidenceLevel {
  private static final long serialVersionUID = 5987321732634610588L;
  private final double confidenceLevel;
  public VarConfidenceLevel(double confidenceLevel) {
    this.confidenceLevel = confidenceLevel;
  }
  @Override
  public Class<? extends IContextValue> getContextInterface() {
    return IVarConfidenceLevel.class;
  }
  @Override
  public double getConfidenceLevel() {
    return confidenceLevel;
  }
  @Override
  public int hashCode() {
    final int prime = 31;
    int result = 1;
    final long temp;
    temp = Double.doubleToLongBits(confidenceLevel);
    result = prime * result + (int) (temp ^ (temp >>> 32));
    return result;
  }
  @Override
  public boolean equals(Object obj) {
    if (this == obj) {
      return true;
    }
    if (obj == null) {
      return false;
    }
    if (getClass() != obj.getClass()) {
      return false;
    }
    final VarConfidenceLevel other = (VarConfidenceLevel) obj;
    if (Double.doubleToLongBits(confidenceLevel)
        != Double.doubleToLongBits(other.confidenceLevel)) {
      return false;
    }
    return true;
  }
}

Ensure Immutability

Context values must be immutable.

This is something very important to enforce, otherwise you may experience unexpected (and hard to troubleshoot) behaviors in ActivePivot.

Usually, the way to ensure that a given object is immutable is (taken from http://download.oracle.com/javase/tutorial/essential/concurrency/imstrat.html):

• Do not provide "setter" methods — methods that modify fields or objects referred to by fields.
• Make all fields final and private.
• Do not allow subclasses to override methods. The simplest way to do this is to declare the class as final. A more sophisticated approach is to make the constructor private and construct instances in factory methods.
• If the instance fields include references to mutable objects, do not allow those objects to be changed:
  • Do not provide methods that modify the mutable objects.
  • Do not share references to the mutable objects. Never store references to external, mutable objects passed to the constructor; if necessary, create copies, and store references to the copies. Similarly, create copies of your internal mutable objects when necessary to avoid returning the originals in your methods.

The last point listed above might be the more subtle to check, but in short it means: be very careful if your custom implementation has some field(s) whose type(s) is(are) other than a Java primitive type or String.

Typically,

• When doing the construction of your custom context value object, you shall take care of not keeping any reference to some mutable object passed to the constructor.
• Don't forget that your custom context value implementation must support a proper cloning... AContextValue already implements a default clone() behavior, but it is only sufficient for supporting cases of custom context value with obvious immutable fields (primitive, String).

Our example of the confidence level context value is relatively easy to manage in that respect: all we need to ensure is that the class is final and that the confidenceLevel field is final.

Serialization

In order to be supported by ActivePivot's Content Server, context values must support serialization in JSON format. More specifically, each context value should be serializable with Jackson library mappers provided by the class JacksonSerializer. One may refer to the source code of the class to get full information about JSON serialization API in ActivePivot and chosen configuration of Jackson properties.

The IContextValue interface is annotated with the following line:

@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, property = "className")

This means that every object implementing IContextValue interface will be serialized with a dedicated JSON property called "className". This property saves the information about the type of the serialized object. This information is then required by the code that will deserialize and process context values.

For example, let us consider a simple boolean context value with a single boolean attribute value. Its serialized version will have 2 properties:

{
  "className" : "com.activeviam.contextservice.bool.BooleanContextValue",
  "value" : true
}

The text above can be put in the string str and deserialized like this:

IContextValue cv = JacksonSerializer.deserialize(IContextValue.class, str);

However, it would be impossible to execute the code above if the "className" property was omitted. It is thus not recommended to override the "className" property by custom properties with such name in order to avoid deserialization problems.

Most of the classes from Java standard library are Jackson-serializable by default. Custom classes having only Jackson-serializable attributes are Jackson-serializable as well. In practice, this means that for the vast majority of the use-cases no additional code is required for context value serialization.

However, there are still some constructs in java that are not Jackson-serializable out of the box. One has to fulfill the requirements of Jackson in order to properly serialize such constructs. For example, classes with attributes having wildcard types need custom Jackson serializers to be implemented. For details, please refer to the documentation of the Jackson project.

Plug the ContextValue in Atoti UI

For a ContextValue to appear in Atoti UI, a plugin implementing IContextValueTranslator needs to be created on the server side.

When the context value has only a single property, its translator can simply extend the SimpleContextValueTranslator:

@QuartetPluginValue(intf = IContextValueTranslator.class)
public static class VarConfidenceLevelTranslator
    extends SimpleContextValueTranslator<String, IVarConfidenceLevel> {
  /** Translator key. */
  public static final String KEY = "VarConfidenceLevelTranslator";
  // ... implementation of format, parse, createInstance,
  // getContent and other abstract methods

If the context value has several properties, its translator must extend MultipleContextValueTranslator: the method computeAvailableProperties() exposes the properties of the context value that can be seen and changed via Atoti UI.

If user wants to hide one or some properties, he can simply omit those properties from the result of this method.

Define the default value for any context value in ActivePivot

We can choose one or several (in combination) of the following configuration options:

• Define a default value for a given cube, via the "shared context". This is configured when configuring the cube:

  StartBuilding.cube("CUBE")
      .withDimension("myDimension")
      .withHierarchyOfSameName()
      .withLevel("myLevel")
      .withSharedContextValue(new VarConfidenceLevel(0.95))
      .build();

• Define a global (understand: applying to all cubes) default value for a given user role. This is configured via the entitlements:

  new SimpleEntitlementsProviderBuilder()
      .withGlobalEntitlement()
      .forRole("ROLE_HIGH_CONFIDENCE")
      .withContextValue(new VarConfidenceLevel(0.95))
      .build();

• Define a default value for a given user role and a particular cube. This is configured via the entitlements:

  new SimpleEntitlementsProviderBuilder()
      .withPivotEntitlement()
      .forRole("ROLE_LOW_CONFIDENCE_ON_A")
      .onPivot("CUBE_A")
      .withContextValue(new VarConfidenceLevel(0.9))
      .build();

Read a Context Value From Within a Post-Processor Evaluation

Before accessing a context value, a post-processor must declare its dependency to it, which can be done by overriding com.activeviam.pivot.postprocessing.impl.AAdvancedPostProcessor.initializeContextDependencies method. If this is not done properly, one will trigger some specific error message when trying to execute the post-processing.

The context values that are used in the post-processor are declared because of a performance enhancement feature that allows sharing measures across multiple queries. That is only possible if the measure has the same value in each query: declaring the context values used in each post-processor allows the query engine to know that it can't share measures when their relevant context values are not equal (thus also the need to be careful about equals and hashcode implementations).

public class VaRPostProcessor extends ABasicPostProcessor {
  // ... some code...
  @Override
  protected Set<Class<? extends IContextValue>> initializeContextDependencies(
      Properties properties) {
    final var contextDependencies = super.initializeContextDependencies(properties);
    contextDependencies.add(IVarConfidenceLevel.class);
    return contextDependencies;
  }
  // ... some code...

Once the context value used is declared as a dependency, it can be read during the post-pro evaluation, by retrieving it from the actual context values seen by the current cube proxy (IActivePivot pivot instance) dedicated to the query:

double getConfidenceLevel() {
  final IVarConfidenceLevel confidenceLevelContext =
      pivot.getContext().get(IVarConfidenceLevel.class);
  if (confidenceLevelContext == null) {
    // Do what you want here (throw or return a default value)
  }
  return confidenceLevelContext.getConfidenceLevel();
}