> ## Documentation Index
> Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt
> Use this file to discover all available pages before exploring further.

# What's new in 5.9

## 5.9

### New features

* **Java version**: ActivePivot 5.9 requires Java 11.
* **New Copper API**: The Copper API, allowing you to easily define measures and hierarchies, has
  been overhauled. Documentation for the API is available [here](../../../copper/copper_intro).
* **User Defined Aggregate Functions**: These new functions can access several fields and store
  several intermediate values. They can be defined using the Copper API or by
  extending `AUserDefinedAggregateFunction`. The documentation is
  available [here](../../../copper/copper_measures#aggregated-measures-with-user-defined-aggregate-functions-udaf).
* **Option to conceal hierarchies and measures from the distributed application**: A concealed
  hierarchy is only known in the data cube where it's located. The query node has no knowledge of
  this hierarchy. Concealing a hierarchy improves the stability of the cluster and shortens the
  discovery phase, by significantly reducing its impact on the network's bandwidth. We advise
  concealing hierarchies that have a very high cardinality. For more information, see
  the [distributed documentation](../../../distributed/distributed_recommendations#avoid-high-cardinality-hierarchies).

  Measures can also be concealed. A concealed measure is only known in the data cube where it's
  located. We recommend concealing measures that are not relevant to the query node, such as
  intermediate measures for calculations that do not need to be visible in the User Interface.
* **Option to disable the epoch level**: The epoch level of the epoch dimension can now be enabled
  or disabled. By default, the epoch level is now disabled, only the branch level is enabled. To
  enable the epoch level, use `IEpochDimensionDescription.setEpochLevelEnabled(boolean)`.
* **Export query plans**: A new REST service allows you to perform a query and export its execution
  plan. The plan now contains the result of each part of the query for each Data Cube (if any), and
  each pass of the MDX query.
* **Limit the result size of a single GetAggregatesQuery** (since 5.9.3): The result size for every
  retrieval as well as the transient result size cross retrievals within a given GetAggregatesQuery
  can be limited in terms of number of points i.e. locations. These limits can be set in the project
  configuration as a shared context value in the cube description or dynamically at query time
  using `QueriesResultLimit` helper methods. If not specified, the default behavior is without
  limit. Note that the transient result size also includes intermediate retrievals result size,
  often required to compute the top retrievals (Retrievals appearing in the query plan) in complex
  queries.

### Improvements

* The CSV Source is now capable of detecting line ending at the byte level. This improves the
  parallelism and speed of the CSV Source when reading one large file, at the cost of not supporting
  a few uncommon character sets.
* The performances of the MDX Engine has been improved by using fine-grained locking instead of
  coarse-grained locking.
* Indexes are now more intensively used for Datastore queries.
  In previous versions, only two cases were covered:

  * an Index was used when the conditions on the index fields were set on a single value.
    Eg: Index on fields A, B, C was used for a condition `And(a = 1, b = 2, c = 3)`
  * an Index was used for a single condition with multiple values. Eg: Index on field F can be
    used for the condition `b IN [1, 2, 3]`
    For other cases, Indexes were ignored. For example, despite an Index on fields A and B, the
    query with conditions
    `And(a IN [1, 2], b = 3)` could not be planned using Index lookup. It could eventually rely on
    an Index lookup for A or B, provided such an Index existed, and field scans for the rest.

  The new logic adds the following support:

  * an Index on any number of fields can be used if only one field is subject to a condition with
    multiple values. Such an Index will be used by pre-compiled queries and ad-hoc queries.
    Eg: Index on fields A, B, C, D can be used for
    conditions `And(a = 1, b = 2, c IN [3, 4], d = 5)`.
  * an Index on any number of fields can be used if no more than three of its fields are involved
    in conditions with multiple values. Such an Index can **only be used by ad-hoc queries**, for
    the query engine must be able to estimate the number of points generated by the conditions.
    Eg: Index on fields A, B, C, D can be used for
    conditions `And(a IN [1, 2], b = 3, c = 4, d IN [5, 6, 7])`.

  The maximum limit for cross joins are configured by the properties
  `ActiveViamProperty#MAX_LOOKUPS_ON_PRIMARY_INDEX`
  and `ActiveViamProperty#MAX_LOOKUPS_ON_SECONDARY_INDEX`.

### Miscellaneous

* The Sandbox has been converted to Spring Boot. Additionally, many features have been transformed
  into unit tests and are presented in the online documentation.