> ## Documentation Index
> Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Configuration properties

export const productName = "Atoti FRTB";

export const coreProductName = "Atoti Server";

This section documents the different configuration properties that can be used when running {productName} with DirectQuery enabled.

By default, the properties are defined in the file **frtb-application/src/main/resources/application.yaml** for optimal readability. You can also define these properties in another `.yaml` or
`.properties` files. If defining in a `.properties` file, you must specify the full property path.

# Properties

Here is a list of all DirectQuery configuration properties available for all Databases. Note that the properties can be defined with `kebab-case` or `camelCase` in the `.properties`,`.yaml` or as command line arguments.

* [`starter.deployment.type`](#deployment-type)
* [`directquery`](#common-directquery-properties)
  * [`incremental-refresh`](#incremental-refresh)
  * [`dates-filter`](#dates-filter)
    * [`dates-to-include`](#dates-to-include)
    * [`dates-to-exclude`](#dates-to-exclude)
  * [`schema`](#schema-validation)
    * [`validation`](#schema-validation)
      * [`validate-on-startup`](#validate-on-startup)
      * [`fail-startup-on-bad-schema`](#fail-startup-on-bad-schema)
  * [`clustering`](#clustering)
    * [`enabled`](#enabled)
    * [`clustering-per-table`](#clustering-per-table)
  * [`aggregate-tables`](#aggregate-tables)
    * [`enabled`](#enabled-1)
    * [`cubes`](#cubes)
      * [`Map: cubeName -> aggregate-providers`](#aggregate-providers)
        * [`Map: providerName -> remote-table-name`](#remote-table-name)
  * [`database`](#database-specific-properties)
    * [`type`](#type)
    * [`snowflake`](#snowflake-database-properties)
      * [`connection-string`](#connection-string)
      * [`username`](#username)
      * [`password`](#password)
      * [`warehouse`](#warehouse)
      * [`feeding-warehouse`](#feeding-warehouse)
      * [`database`](#database)
      * [`schema`](#schema)
    * [`clickhouse`](#clickhouse-database-properties)
      * [`username`](#username-1)
      * [`password`](#password-1)
      * [`port`](#port)
      * [`host-name`](#host-name)
      * [`database`](#database-1)
      * [`schema`](#schema-1)
    * [`databricks`](#databricks-database-properties)
      * [`timeTravelPolicy`](#timetravelpolicy)
      * [`connectionString`](#connectionstring)
      * [`heavyLoadConnectionString`](#heavyloadingconnectionstring)
      * [`database`](#database-2)
      * [`schema`](#schema-2)

## Deployment Type

***Base path:*** `starter.deployment.type`

This property is used to specify the deployment type of the application. To enable the use of direct query, the deployment type should be set to `direct-query`.

## Common DirectQuery Properties

***Base path:*** `directquery`

These properties are applied to any database the application is connecting to.

## Incremental refresh

***Base path:*** `directquery.incremental-refresh.templates`<br />
***Default Value:*** `Empty List`
Defines the templates that can be used when performing an incremental refresh operation.

Each template defines the tables and fields that can be updated when performing an incremental refresh operation. This allows for a more structured and reusable approach to incremental refresh operations.

The templates contain **tables** and **fields** to include in the incremental refresh operation. Fields can also be given an `alias` to simplify the request body when performing the incremental refresh operation. Aliases can exist across multiple tables, but must be unique per table. Aliases allow users to specify one field-value pair as the values are reused across the multiple tables, simplifying the request body.

The template definition is as follows for a template named `MyIncrementalRefreshTemplate` on the `BaseTable` and `TradeTable` tables:

```yaml theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
templates:
  MyIncrementalRefreshTemplate: # The name of the template
    tables:
      BaseTable: # The local table name
        fields:
          - name: TradeId # The local field name
          - name: AsOfDate # The local field name
            alias: Date # Optional alias for the field
      TradeTable:
        fields:
          - name: TradeAttribute
          - name: AsOfDate
            alias: Date
```

## Dates Filter

***Base path:*** `directquery.dates-filter`<br />
***Default Behavior:*** The DirectQuery data node will include all dates in the remote database.

These properties are used to specify the dates that the DirectQuery data node has access to. This is important as the DirectQuery data node should not contain
any dates included in any other data node as when using distribution, two nodes of the same cube cannot contain the same date.

The dates filter can be customized to handle custom filtering use cases. Please see the [Dates Filter Bean](../customization-and-internals/dates-filter-bean)
documentation for more information.

By default, the DirectQuery data node will include all dates in the remote database. If you want to include only a subset of dates, you can specify them here.

### `dates-to-include`

***Default Value:*** `Empty List`

Only the specified dates will be included in the DirectQuery data cube, all other dates will be excluded.

This property cannot be set along with [`dates-to-exclude`](#dates-to-exclude).

### `dates-to-exclude`

***Default Value:*** `Empty List`

The specified list od dates will be excluded from the DirectQuery data cube and all other dates on the database will be included.

This property cannot be set along with [`dates-to-include`](#dates-to-include).

## Schema Validation

***Base path:*** `directquery.schema.validation`

These properties are used to specify when and how to validate the remote database schema. If the remote database schema is bad or invalid, this can result in
significant performance issues. A schema with duplicate keys can result in longer than normal aggregation and application refresh times.

### `validate-on-startup`

***Default Value:*** `true`

This will initiate a validation check of the remote database when the application starts. This validation will check that all remote tables are readable and
that they do not contain any duplicate keys. If there are any errors they will be logged and, if configured with the `fail-startup-on-bad-schema` property, an
exception will be thrown.

### `fail-startup-on-bad-schema`

***Default Value:*** `true`

This property sets what you want the system to do when a bad schema is encountered when the application is starting: either fail or just log warnings.

## Clustering

***Base path:*** `directquery.clustering`

Clustering properties are specified per database, for example, when specifying the clustering fields for the database in use:

```yaml theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
directquery:
  clustering:
    enabled: true
    clustering-per-table:
      SASensitivities: AsOfDate
```

Clustering allows {coreProductName} to break up queries into smaller ones that can be run in parallel. This improves the initial aggregation time on the remote database
as the aggregation query can be run in parallel. See the {coreProductName} documentation on
[clustering fields](/engine/java-sdk/6.1/directquery/directquery-clustering-fields/) for more information.

### `enabled`

***Default Value:*** `false`

If we want to enable clustering for this database.

### `clustering-per-table`

***Default Value:*** `empty list`

Specifies a mapping of tables to a list of clustering fields.

Here is an example of how to specify the key values:

```yaml theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
clustering-per-table:
  TableName1: Field_1, Field_2
  TableName2: Field_1
  etc...
```

## Aggregates Tables

***Base path:*** `directquery.aggregate-tables`

These properties specify the remote database table to use for each aggregate provider.

Here is an example of how to set the key values:

### `enabled`

***Default Value:*** `false`

Enables / disables the use of aggregate tables:

```yaml theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
directquery:
  aggregate-tables:
    enabled: true
    cubes:
      InternalModelApproachCube:
        aggregate-providers:
          LEAF-Provider-0:
            remote-table-name: INTERNAL_MODEL_APPROACH_CUBE_LEAF_PROVIDER_0_AGGREGATE_TABLE
          LEAF-Provider-1:
            remote-table-name: INTERNAL_MODEL_APPROACH_CUBE_LEAF_PROVIDER_1_AGGREGATE_TABLE
      StandardisedApproachCube:
        aggregate-providers:
          LEAF-Provider-2:
            remote-table-name: STANDARDISED_APPROACH_CUBE_LEAF_PROVIDER_2_AGGREGATE_TABLE
```

### `cubes`

***Default Value:*** `null`

***Type*** `Map<String, AggregateProvidersProperties>`

Contains a mapping of CubeName -> [`aggregate-providers`](#aggregate-providers). This specifies the configuration of aggregate tables for each cube.

### `aggregate-providers`

***Default Value:*** `null`

***Type*** `Map<String, DqAggregateTableProperties>`

Contains a mapping of `AggregateProviderName` -> [`remote-table-name`](#remote-table-name). This specifies the aggregate tables for each aggregate provider.

### `remote-table-name`

***Default Value:*** `null`

The name of the remote database table to use as the aggregate table.

## Database Specific Properties

***Base path:*** `directquery.database.xxx`

These properties are applied to the specific database the application is connecting to and may be unique per database.

### `type`

***Default Value:*** `clickhouse`

The database to use in the current instance of the application. This value must be the name of the database in lowercase.

## Snowflake Database Properties

***Base path:*** `directquery.database.snowflake`

These properties only apply to the Snowflake database.

### `connection-string`

***Default Value:*** `null`

The connection string used to connect to the remote Snowflake instance.

### `username`

***Default Value:*** `null`

The username to use to authenticate the connection to Snowflake.

### `password`

***Default Value:*** `null`

The password to use to authenticate the connection to Snowflake.

### `warehouse`

***Default Value:*** `null`

The warehouse (compute) to use when executing queries on Snowflake.

### `feeding-warehouse`

***Default Value:*** `directquery.database.snowflake.warehouse` value

The warehouse (compute) to use when computing the initial aggregates on Snowflake. This will also be the warehouse to use when performing an Application Refresh.

The idea is to use a larger instance so the initial aggregation time can be faster. This warehouse will only be used for the initial aggregation and refreshes,
any other user query that hits Snowflake will be executed using the `directquery.database.warehouse` warehouse.

### `database`

***Default Value:*** `null`

The Snowflake Database to connect to and use for queries.

### `schema`

***Default Value:*** `null`

The Snowflake Schema to connect to and use for queries.

## ClickHouse Database Properties

***Base path:*** `directquery.database.clickhouse`

These properties only apply to the ClickHouse database.

### `username`

***Default Value:*** `null`

The optional username to use to authenticate the connection to ClickHouse. If no authentication is defined on your ClickHouse database, these properties can
be ignored.

### `password`

***Default Value:*** `null`

The optional password to use to authenticate the connection to ClickHouse. If no authentication is defined on your ClickHouse database, these properties can
be ignored.

### `port`

***Default Value:*** `null`

The port the ClickHouse database is exposed on.

### `host-name`

***Default Value:*** `null`

The hostname the ClickHouse database is hosted from. For example, when running locally this would be `localhost`

### `database`

***Default Value:*** `null`

The ClickHouse Database to connect to and use for queries.

### `schema`

***Default Value:*** `null`

The ClickHouse Schema to connect to and use for queries.

## Databricks Database Properties

***Base path:*** `directquery.database.databricks`

These properties only apply to the Databricks database.

### `timeTravelPolicy`

***Default Value:*** `DISABLED`

The property that allows for time-travel based on the policy specified.

### `connectionString`

***Default Value:*** `null`

The connection string used to connect to the remote Databricks cluster.

### `heavyLoadingConnectionString`

***Default Value:*** `null`

The heavy loading connection string computes the aggregates and is only used when the application starts or is refreshed.

### `database`

***Default Value:*** `null`

The Databricks Database to connect to and use for queries.

### `schema`

***Default Value:*** `null`

The Databricks Schema to connect to and use for queries.