Skip to main content
In business analysis, drillthrough is a term used to describe the act of exploring data from a condensed view by examining an exploded view of the underlying data. In the context of data aggregation, this means it is possible to list the facts at a given location, helping explain a specific aggregate result. Atoti gives the user the ability to explore and extract sets of facts stored in our database, the Atoti Datastore. To achieve this, Atoti offers a bespoke API based on locations (see Database Queries). However, experience has shown us that customers sometimes need to extract some very particular set of facts, that do not rely on location conditions, but rather on some specific pre-computation (e.g. filters on measure values). The MDX drillthrough allows querying for facts in the datastore, based on an MDX query that returns a single cell. The facts returned by the drillthrough are the ones that add to the contributors.COUNT on the location of the cell, with the same filters as the MDX query.

MDX Drillthrough Query

Basic Drillthrough Statement

The MDX drillthrough statement standard is defined by Microsoft. An MDX drillthrough query looks very similar to an MDX query. It starts with the DRILLTHROUGH keyword, followed by an MDX select statement. Any select statement can be used, with as complex filter as needed, provided that the statement is syntactically valid and it returns a single cell. Here is an example of an MDX drillthrough with a complex filter:

Real-time

The MDX drillthrough supports real time updates. However, those updates are limited to the set and subset of data determined when the query is launched (i.e. the location and the subcubes are evaluated only once, when the query is registered). Atoti will deliver the related updates for this location, and those subcubes, as and when they occur. For example, the query above contains a TopCount filter: the real time updates will concern data contributing to the desk that has the best delta.SUM at the moment the query was launched, even if further data updates change which desk is now the best.

Return Clause

It is possible to add a RETURN clause to the MDX drillthrough query to select which columns are to be retrieved, or in other words which fields of the datastore are wanted. The following example extracts all the facts regarding DeskA, but only the Date, Desk and pnl attributes are returned:
The RETURN clause:
  • replaces the use of the context value drillthrough.hiddencolumns (this context value still exists though). Instead of indicating which columns need to be hidden in the context value, the user indicates which columns need to be seen directly in the query.
  • defines the order of the columns of an MDX drillthrough query.
  • is not mandatory. If not present, all columns will be retrieved.
  • should contain a list of quoted column header names (i.e. names of fields defined in the Atoti Schema) separated by a coma.

Paging

It is possible to cut the result of a drillthrough query into conveniently smaller pieces. This reduces the memory overhead resulting from a drillthrough query. For example, let us suppose that a query returns 10 000 000 rows. It may be more convenient to receive the results in packets of 1 000 000 rows. This can be achieved by using additional keywords in the query:
  • MAXROWS: indicates the maximum number of rows to be returned by the resulting rowset (cf. MSDN specification).
  • FIRSTROW: indicates the index of the first row to be retrieved (custom ActiveViam keyword).
So for the first million rows, we could have a query like:
while the second million rows would be extracted with:
and so on for all 10 groups of one million. It is possible to use each of these keywords on its own, without the other. If MAXROWS is used on its own, the default value of FIRSTROW is 0. If FIRSTROW is used on its own, the default value for MAXROWS is -1, which means “no limit except the one defined as a safeguard in the cube configuration”. The maxrows safeguard defined in the cube configuration prevents the server from crashing (with OutOfMemory exception) because too many rows are returned via a drillthrough query. Customization to address this limitation is available through the context value drillthrough.maxrows. It is important to be aware that keyword MAXROWS does not override this context value.
FIRSTROW is a custom keyword of Atoti and not part of the MSDN specification. A similar keyword, FIRSTROWSET, does exist, but does not mean the same thing: FIRSTROWSET identifies the partition whose rowset is returned first.

MDX Drillthrough Function

As stated in the MSDN specification, several functions applicable to the RETURN clause elements exist officially. These functions are used to format the values returned by an MDX drillthrough query, column by column.
By default, the elements of the columns returned by a query are Java objects. For example, in the sandbox, the risk AsOfDate field is a LocalDate (see DatastoreDescriptionConfig). Asking for [AsOfDate] is equivalent to MemberValue([AsOfDate]) and will return the LocalDate object, while asking for Caption([AsOfDate]) will return the DATE[yyyy-MM-dd] formatted string, since this is the formatter for the AsOfDate field in EquityDerivativesCubeDimensionsConfig.

Calculated Columns

In broad terms, a calculated column is an additional column, derived and calculated from other column values and then added to the drillthrough result. The configuration of a calculated column is required to declare all its dependencies (other columns on which the calculated value depends). Doing this ensures that Atoti is able to retrieve the minimum set of columns needed to compute the calculated columns. A calculated drillthrough column is an extended plugin value. An example is given in the sandbox project:
The calculated column is then added to the drillthrough properties in the cube configuration by referencing the plugin key and defining the underlying fields (see EquityDerivativesCubeConfig#drillthroughProperties()).

Calculated Column Sets

Calculated column sets allow to dynamically create calculated columns “on-the-fly” (i.e. when a query is launched), depending on various parameters contained in an IDrillthroughQueryContext. A calculated drillthrough column set is an extended plugin value. An example is given in the sandbox project:
Note that a calculated column defines the evaluate method that allows to return the column value based on the underlying fields. By contrast, a calculated column set defines the generate method, that does not explain directly how to evaluate each column but that generates calculated column descriptions which, themselves, refer to calculated columns through their plugin key. It’s the calculated columns that do define the evaluate method.

Calculated Members

When doing an MDX query, each cell corresponds to a combination of a location (eventually filtered) and a measure. When doing a drillthrough on a cell in an MDX query, we list the facts that contributed to that cell. In the case of a simple measure, the facts correspond to the (eventually filtered) location, and we don’t need to take the measure into account. However, things get a bit more complicated if the measure causes a change in the location (for example, yesterday’s data, or the parent’s total…). In that case, it becomes more complex: does the drillthrough user want to see the data for the selected location, or the (shifted) data that was used to compute the measure at the selected location? And what if we use several locations to compute the measure (percentage of parent for example, where we use both the queried location and its parent)? Atoti partially supports the use of calculated members in an MDX drillthrough query, but with restrictions so that deciding which data should be displayed in the drillthrough is straightforward. We allow simple calculated members that do not modify the location, some very specific structures for percentage of parent and percentage of total (where we assume the data that the user wants for the drillthrough are the facts contributing to the queried location, not the parent or the total), and tuples (where we assume the data the user wants for the drillthrough are the facts contributing to the tuple, rather than the queried location). The following table provides a list of supported calculated members in an MDX Drillthrough query:

Limitations

  1. The keywords MAXROWS and FIRSTROW must not be used when executing an MDX drillthrough query in real time.
  2. Real time is only possible on the latest epoch of the cube.
  3. Only the declared calculated members are supported.
  4. The location and subcubes are evaluated once and once only, at the initialization of the query. See real time section.

How to Make Drillthrough Queries Compatible with Analysis Hierarchy Business Logic

The internal query engine knows which set of facts needs to be retrieved from the datastore, based on the MDX drillthrough query. However, some hierarchies, like analysis hierarchies, are defined by some customized analytical algorithms rather than the usual data introspection mechanism. This is why some projects will need to tailor the MDX drillthrough query to datastore query conversion by adding their own business logic during that process. One can implement their own ILocationInterpreter to do so, or extend the DefaultLocationInterpreter. This object is in charge of converting drillthrough query locations (IDrillthroughQuery#getLocations()) into a datastore condition (ICondition). It can be used to change the set of facts returned by a drillthrough query, to filter the result for instance. By default (DefaultLocationInterpreter), the location interpreter ignores the coordinates of the analysis hierarchies. An example on how to extend DefaultLocationInterpreter to see an analysis hierarchy in the drillthrough can be found in the sandbox:
Then an IDrillthroughExecutor must be defined, that makes use of the created ILocationInterpreter. In the sandbox, you can look at TimeBucketDrillthroughExecutor:
Then the executor must be referenced when configuring the cube, using withDrillthroughExecutor.