> ## Documentation Index
> Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Monitoring on the Datastore

## Tracing

The Datastore leverages tracing to provides insights into the queries executed on the external database, such as the executed SQL query or the query id in the external database.

Check [tracing](../monitoring/tracing) to configure it.

## Logging

Without tracing activated, the Datastore provides some logging of the API and external queries.

### Logger naming

To retrieve all the Datastore logs, the logger name should be "atoti.server.datastore".

As can be seen in the class, there are also some more granular loggers, to focus on one particular aspect of
Datastore such as "atoti.server.datastore.transaction", ...

Here is the exhaustive list:

* atoti.server.database
  * atoti.server.database.condition\_optimization
  * atoti.server.database.query\_statistics
* atoti.server.datastore
  * atoti.server.datastore.configuration
  * atoti.server.datastore.data\_streaming
  * atoti.server.datastore.query
  * atoti.server.datastore.structural\_transaction
  * atoti.server.datastore.transaction

The logger names can be found in the `com.activeviam.database.api.DatabaseLoggerConstants.class`.

### Logging Tags

SQL Queries are also augmented with tags in the logs, allowing to get more information about the query itself.
In the log of a query, there is a map associating a tag with its value.

Here is a list of existing tags (note that only "category" will always be present)

* "category": explains which type of query it is; it could be for feeding a hierarchy, an aggregate provider,
  or a drillthrough query.
* "pivot\_name": the name of the pivot which originated the query.
* "provider\_name": the name of the aggregate provider, needing data.
* "aggregate\_table\_name": the name of the aggregate table being queried.

## Transaction timings

The transaction timings are reported in both logs and traces.

```mermaid theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
sequenceDiagram
    participant User
    participant Datastore
    participant Listeners
    note right of User: Interactions are done through the ITransactionManager

    User->>Datastore: Call to startTransaction
    User->>Datastore: Add records
    Datastore-->>Datastore: Processing
    User->>Datastore: Delete records
    Datastore-->>Datastore: Processing
    User->>Datastore: Update records
    Datastore-->>Datastore: Processing
    User->>Datastore: Call to ITransactionManager.commitTransaction
    Datastore-->>Datastore: Wait for the end of the processing
    critical 2 phase commit
        Datastore-->>Listeners: Notify of commit intent
        Listeners-->>Datastore: Acknowledge the possibility to commit
        Datastore-->>Datastore: Commits the transaction
    end
    Datastore->>User: ITransactionManager.commitTransaction returns
    Datastore-->>Listeners: Notify of transaction commit
```

* **Transaction duration**: from calling `ITransactionManager#startTransaction()` to the return of the call to `ITransactionManager#commitTransaction()`
* **Datastore processing duration**: from calling `ITransactionManager#startTransaction()` to the end of the processing of the updates by the datastore (after the call to `ITransactionManager#commitTransaction()`, but before the 2-phase commit).
* **Datastore commit duration**: time of the 2-phase commit protocol (from the end fo the datastore processing to the datastore commit). This includes some work done in the listeners (the cube and the pivot being among those).
* **User interaction duration**: from calling `ITransactionManager#startTransaction()` to calling to `ITransactionManager#commitTransaction()`

We have the following relation `Transaction duration = Datastore processing duration + Datastore commit duration`

The datastore processing duration can be equal to the user interaction duration if there is no more work to do when the user calls `ITransactionManager#commitTransaction()`.
However, it is usually greater than the user interaction duration, as the user usually does not wait for the end of the processing before calling `ITransactionManager#commitTransaction()`.