In business analysis, drillthrough is a term used to describe the act of exploring data from a condensed view by examining an exploded view of the underlying data. In the context of data aggregation, this means it is possible to list the facts at a given location, helping explain a specific aggregate result. Atoti gives the user the ability to explore and extract sets of facts stored in our database, the Atoti Datastore. To achieve this, Atoti offers a bespoke API based on locations (see Database Queries). However, experience has shown us that customers sometimes need to extract some very particular set of facts, that do not rely on location conditions, but rather on some specific pre-computation (e.g. filters on measure values). The MDX drillthrough allows querying for facts in the datastore, based on an MDX query that returns a single cell. The facts returned by the drillthrough are the ones that add to the contributors.COUNT on the location of the cell, with the same filters as the MDX query.Documentation Index
Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt
Use this file to discover all available pages before exploring further.
MDX Drillthrough Query
Basic Drillthrough Statement
The MDX drillthrough statement standard is defined by Microsoft. An MDX drillthrough query looks very similar to an MDX query. It starts with theDRILLTHROUGH keyword,
followed by an MDX select statement.
Any select statement can be used, with as complex filter as needed, provided that the statement is syntactically valid
and it returns a single cell.
Here is an example of an MDX drillthrough with a complex filter:
Real-time
The MDX drillthrough supports real time updates. However, those updates are limited to the set and subset of data determined when the query is launched (i.e. the location and the subcubes are evaluated only once, when the query is registered). Atoti will deliver the related updates for this location, and those subcubes, as and when they occur. For example, the query above contains aTopCount filter:
the real time updates will concern data contributing to the desk that has the best delta.SUM
at the moment the query was launched, even if further data updates change which desk is now the best.
Return Clause
It is possible to add aRETURN clause to the MDX drillthrough query to select which columns are to be retrieved,
or in other words which fields of the datastore are wanted.
The following example extracts all the facts regarding DeskA, but only the Date, Desk and pnl attributes are returned:
RETURN clause:
- replaces the use of the context value
drillthrough.hiddencolumns(this context value still exists though). Instead of indicating which columns need to be hidden in the context value, the user indicates which columns need to be seen directly in the query. - defines the order of the columns of an MDX drillthrough query.
- is not mandatory. If not present, all columns will be retrieved.
- should contain a list of quoted column header names (i.e. names of fields defined in the Atoti Schema) separated by a coma.
Paging
It is possible to cut the result of a drillthrough query into conveniently smaller pieces. This reduces the memory overhead resulting from a drillthrough query. For example, let us suppose that a query returns 10 000 000 rows. It may be more convenient to receive the results in packets of 1 000 000 rows. This can be achieved by using additional keywords in the query:MAXROWS: indicates the maximum number of rows to be returned by the resulting rowset (cf. MSDN specification).FIRSTROW: indicates the index of the first row to be retrieved (custom ActiveViam keyword).
MAXROWS is used on its own, the default value of FIRSTROW is 0.
If FIRSTROW is used on its own, the default value for MAXROWS is -1,
which means “no limit except the one defined as a safeguard in the cube configuration”.
The maxrows safeguard defined in the cube configuration prevents the server from crashing (with OutOfMemory exception)
because too many rows are returned via a drillthrough query.
Customization to address this limitation is available through the context value drillthrough.maxrows.
It is important to be aware that keyword MAXROWS does not override this context value.
FIRSTROWis a custom keyword of Atoti and not part of the MSDN specification. A similar keyword,FIRSTROWSET, does exist, but does not mean the same thing: FIRSTROWSET identifies the partition whose rowset is returned first.
MDX Drillthrough Function
As stated in the MSDN specification, several functions applicable to theRETURN clause elements exist officially.
These functions are used to format the values returned by an MDX drillthrough query, column by column.
| Symbol | Meaning |
|---|---|
| fully supported | |
| will not be supported |
| Function Name | Support | Comments |
|---|---|---|
| Name | The elements of the specified column are formatted using the ToString formatter (Object.toString()). | |
| Caption | The elements of the specified column are formatted using the formatter declared in the cube description. If none is defined, the ToString formatter is used. | |
| MemberValue | The column is not formatted (default value) | |
| UniqueName | ||
| Key | ||
| CustomRollup | There is no custom rollup in Atoti | |
| CustomRollupProperties | ||
| UnaryOperator |
By default, the elements of the columns returned by a query are Java objects. For example, in the sandbox, the risk AsOfDate field is aLocalDate(seeDatastoreDescriptionConfig). Asking for[AsOfDate]is equivalent toMemberValue([AsOfDate])and will return theLocalDateobject, while asking forCaption([AsOfDate])will return theDATE[yyyy-MM-dd]formatted string, since this is the formatter for the AsOfDate field inEquityDerivativesCubeDimensionsConfig.
Calculated Columns
In broad terms, a calculated column is an additional column, derived and calculated from other column values and then added to the drillthrough result. The configuration of a calculated column is required to declare all its dependencies (other columns on which the calculated value depends). Doing this ensures that Atoti is able to retrieve the minimum set of columns needed to compute the calculated columns. A calculated drillthrough column is an extended plugin value. An example is given in the sandbox project:EquityDerivativesCubeConfig#drillthroughProperties()).
Calculated Column Sets
Calculated column sets allow to dynamically create calculated columns “on-the-fly” (i.e. when a query is launched), depending on various parameters contained in anIDrillthroughQueryContext.
A calculated drillthrough column set is an extended plugin value. An example is given in the sandbox project:
evaluate method that allows
to return the column value based on the underlying fields.
By contrast, a calculated column set defines the generate method, that does not explain directly how to evaluate
each column but that generates calculated column descriptions which, themselves, refer to calculated columns
through their plugin key. It’s the calculated columns that do define the evaluate method.
Calculated Members
When doing an MDX query, each cell corresponds to a combination of a location (eventually filtered) and a measure. When doing a drillthrough on a cell in an MDX query, we list the facts that contributed to that cell. In the case of a simple measure, the facts correspond to the (eventually filtered) location, and we don’t need to take the measure into account. However, things get a bit more complicated if the measure causes a change in the location (for example, yesterday’s data, or the parent’s total…). In that case, it becomes more complex: does the drillthrough user want to see the data for the selected location, or the (shifted) data that was used to compute the measure at the selected location? And what if we use several locations to compute the measure (percentage of parent for example, where we use both the queried location and its parent)? Atoti partially supports the use of calculated members in an MDX drillthrough query, but with restrictions so that deciding which data should be displayed in the drillthrough is straightforward. We allow simple calculated members that do not modify the location, some very specific structures for percentage of parent and percentage of total (where we assume the data that the user wants for the drillthrough are the facts contributing to the queried location, not the parent or the total), and tuples (where we assume the data the user wants for the drillthrough are the facts contributing to the tuple, rather than the queried location). The following table provides a list of supported calculated members in an MDX Drillthrough query:| Example | Support | Comment |
|---|---|---|
[Measures].[custom] AS [Measures].[pnl.SUM] $ 10 | Basic operation with a measure.$ could be one of the following operators: +, -, *, / | |
[Measures].[custom] AS [Measures].[gamma.SUM] $ [Measures].[delta.SUM] | Operation between two (or more) measures. Not supported in polymorphic distribution.$ could be one of the following operators: +, -, *, / | |
[Measures].[custom] AS [Measures].[pnl.SUM], FORE_COLOR = IIf([Measures].[dummy] >= 0, RGB(0,0,0), RGB(255,255,255)) | Calculated member for dynamic formatting. Simple formatting is also supported. | |
[Measures].[custom] AS ([Measures].[pnl.SUM], [Bookings].[Desk].[ALL].[AllMember].[DeskA], [Time].[AsOfDate].[Tue Jan 27 01:00:00 CET 2015]) | The calculated member is a simple tuple. Be careful when using such formula, as it might change the location where the drillthrough will be performed. | |
[Measures].[custom] AS IIf(IsEmpty(([Bookings].[Bookings].CurrentMember.Parent,[Measures].[contributors.COUNT])), NULL, [Measures].[contributors.COUNT]/([Bookings].[Bookings].CurrentMember.Parent, [Measures].[contributors.COUNT])) | Percentage of parent: the location is not modified. | |
[Measures].[custom] AS IIf(IsEmpty(([Bookings].[Bookings].[ALL].[AllMember],[Measures].[contributors.COUNT])), NULL, [Measures].[contributors.COUNT]/([Bookings].[Bookings].[ALL].[AllMember], [Measures].[contributors.COUNT])) | Percentage of total: the location is not modified. | |
[Measures].[custom] AS ([Measures].[contributors.COUNT], [Date].[Date].[Calendar Year].[2012]) - ([Measures].[contributors.COUNT], [Date].[Date].[Calendar Year].[2011]) | ||
[Bookings].[Desk].[custom] AS [Bookings].[Desk].[ALL].[AllMember].[DeskA] | Not supported because the dimension of the calculated member is not Measures. | |
[Measures].[custom] AS Count(Descendants([Bookings].[Desk].CurrentMember,[Bookings].[Desk].[Desk]), EXCLUDEEMPTY) | Distinct count formula. |
Limitations
- The keywords
MAXROWSandFIRSTROWmust not be used when executing an MDX drillthrough query in real time. - Real time is only possible on the latest epoch of the cube.
- Only the declared calculated members are supported.
- The location and subcubes are evaluated once and once only, at the initialization of the query. See real time section.
How to Make Drillthrough Queries Compatible with Analysis Hierarchy Business Logic
The internal query engine knows which set of facts needs to be retrieved from the datastore, based on the MDX drillthrough query. However, some hierarchies, like analysis hierarchies, are defined by some customized analytical algorithms rather than the usual data introspection mechanism. This is why some projects will need to tailor the MDX drillthrough query to datastore query conversion by adding their own business logic during that process. One can implement their ownILocationInterpreter to do so, or extend the DefaultLocationInterpreter.
This object is in charge of converting drillthrough query locations (IDrillthroughQuery#getLocations())
into a datastore condition (ICondition).
It can be used to change the set of facts returned by a drillthrough query, to filter the result for instance.
By default (DefaultLocationInterpreter), the location interpreter ignores the coordinates of the analysis hierarchies.
An example on how to extend DefaultLocationInterpreter to see an analysis hierarchy in the drillthrough
can be found in the sandbox:
IDrillthroughExecutor must be defined, that makes use of the created ILocationInterpreter.
In the sandbox, you can look at TimeBucketDrillthroughExecutor:
withDrillthroughExecutor.