Metrics Overview
Atoti Server provides metrics through the OpenTelemetry API, which can be used with a variety of observability backends. See how to set up observability with OpenTelemetry SDK here.
By default, Atoti provides metrics for a variety of components.
Metrics Types
Atoti use the OpenTelemetry API to provide metrics. The metrics are divided into 3 types:
- Gauge: A metric that represents a sampled value at a given time.
- Sum: A cumulative metric that represents a sum of individual measurements made over a period of time.
- Histogram: A metric that samples observations (usually things like request durations or response sizes) and counts them in buckets.
See the OpenTelemetry documentation for more information.
Metrics Tags
Metrics are augmented with tags, allowing to get more information about the metric itself. That information can be used to filter or group the metrics.
Metrics Available
The following metrics are provided by Atoti:
| Metric Name | Type | Tags | Description |
|---|---|---|---|
| atoti.agg_provider.bitmap.data.memory_footprint | Gauge | cube.name, agg_provider.name | Measures in MB the total memory usage of data stored by the bitmap index. That could be helpful to identify performance bottlenecks. |
| atoti.agg_provider.bitmap.memory_footprint | Gauge | cube.name, agg_provider.name | Measures in MB the total memory usage of the bitmap index including the data size and other information. |
| atoti.agg_provider.index.memory_footprint | Gauge | cube.name, agg_provider.name, agg_provider.type | Tracks the memory size in MB of the index of the aggregate provider. |
| atoti.agg_provider.index.size | Gauge | name, index.name, type | Measures the size of each index within an aggregate provider. |
| atoti.agg_provider.off_heap.size | Gauge | name, type | Tracks the size of an aggregate provider's off-heap memory usage. |
| atoti.agg_provider.on_heap.size | Gauge | name, type | Tracks the size of an aggregate provider's on-heap memory usage. |
| atoti.agg_provider.partition.bitmap.data.memory_footprint | Gauge | cube.name, agg_provider.partition.id, agg_provider.name, agg_provider.type | Measures in MB the memory usage of data stored in one partition of the bitmap index. This is useful for comparing the amount of data between partitions. |
| atoti.agg_provider.partition.bitmap.memory_footprint | Gauge | cube.name, agg_provider.partition.id, agg_provider.name, agg_provider.type | Measures in MB the memory usage of a partition of the bitmap index including data size and other information. This is useful for comparing partitions of the same bitmap aggregate provider. |
| atoti.agg_provider.partition.index.memory_footprint | Gauge | cube.name, agg_provider.partition.id, agg_provider.name, agg_provider.type | Tracks the total memory size of an index. |
| atoti.agg_provider.partition.index.size | Gauge | cube.name, agg_provider.partition.id, agg_provider.name, agg_provider.type | Provides the number of lines of the index in the aggregate provider partition. |
| atoti.agg_provider.partition.off_heap.size | Gauge | cube.name, agg_provider.partition.id, agg_provider.name, agg_provider.type | Tracks the off-heap memory usage in MB of an aggregate provider. |
| atoti.agg_provider.partition.on_heap.size | Gauge | cube.name, agg_provider.partition.id, agg_provider.name, agg_provider.type | Tracks the on-heap memory usage in MB of an aggregate provider. |
| atoti.agg_provider.partition.size | Gauge | name, type, partition.id, date:value (for each partition key) | Indicates the size of each partition within an aggregate provider. |
| atoti.agg_provider.size | Gauge | name, type | Measures the overall size of an aggregate provider. |
| atoti.application.off_heap.allocate.hits | Sum | node.id | Counts the number of successful off-heap memory allocation attempts. |
| atoti.application.off_heap.allocate.size | Histogram | node.id | Measures the size of each off-heap memory allocation. |
| atoti.application.off_heap.committed | Gauge | Tracks the amount of off-heap memory that the Atoti Server has committed to use. | |
| atoti.application.off_heap.free.hits | Sum | node.id | Counts the number of successful off-heap memory free operations. |
| atoti.application.off_heap.free.size | Histogram | node.id | Measures the size of each off-heap memory free operation. |
| atoti.application.off_heap.used | Gauge | Measures the off-heap memory currently used by the Atoti Server. | |
| atoti.cube.epoch.added | Sum | cube.name | Tracks the number of epochs that have been added to the cube. |
| atoti.cube.epoch.current | Gauge | cube.name | Displays the most recent epoch visible in the cube. |
| atoti.cube.epoch.deleted | Sum | cube.name | Counts the number of epochs deleted through garbage collection in the cube. |
| atoti.cube.epoch.live | Gauge | cube.name | Shows the count of live epochs in the cube, with versions maintained by a strong reference. |
| atoti.cube.epoch.oldest | Gauge | cube.name | Indicates the oldest epoch still live within the cube. |
| atoti.cube.epoch.released | Sum | cube.name | Counts the number of epochs that have been released in the cube, setting the strong reference to null. |
| atoti.cube.rebuild | Sum | cube.name | Tracks the number of times the cube has been rebuilt. |
| atoti.cube.start | Sum | cube.name | Counts the number of times a cube has been started. |
| atoti.cube.stop | Sum | cube.name | Counts the number of times a cube has been stopped. |
| atoti.database.epoch.added | Sum | database.id | Counts the number of new epochs created in an in-memory or external database. |
| atoti.database.data_model_transaction.hit | Sum | database.id, success | Count of structural transactions received by the database (schema updates). |
| atoti.database.data_model_transaction.time | Histogram | database.id, success | Time to process structural transactions received on the database (schema updates). |
| atoti.datastore.branch.count | Gauge | datastore.name | Measures the number of branches of the datastore. |
| atoti.datastore.epoch.current | Gauge | datastore.name | Shows the most recent epoch that is currently visible in the datastore. |
| atoti.datastore.epoch.deleted | Sum | datastore.name | Counts the number of epochs that have been deleted via garbage collection. |
| atoti.datastore.epoch.live | Gauge | datastore.name | Indicates the number of live epochs in the datastore, where versions are kept by a strong reference. |
| atoti.datastore.epoch.oldest | Gauge | datastore.name | Displays the oldest epoch that is still active in the datastore. |
| atoti.datastore.epoch.released | Sum | datastore.name | Tracks the number of epochs that have been released, where the strong reference is set to null. |
| atoti.loading.files | Sum | source.type, mode (for CSV source), store.name (for Parquet Source), topic.name (for CSV source). | Count of files loaded by the sources into the datastore. |
| atoti.loading.files.lines.errored | Sum | source.type, mode (for CSV source), store.name (for Parquet Source), topic.name (for CSV source). | Count of lines ingested but errored because not correctly parsed. This does not include headers. For a CSV source, parsingReportEnabled must be set to true in its configuration. |
| atoti.loading.files.lines.ingested | Sum | source.type, mode (for CSV source), store.name (for Parquet Source), topic.name (for CSV source). | Count of lines loaded into the datastore. The tags for a CSV source are source.type, mode and topic.name. The tags for a parquet source are only source.type and store.name. For a CSV source, parsingReportEnabled must be set to true in its configuration. |
| atoti.loading.sql | Sum | source.type, topic.name | Tracks the number of SQL queries run to load data. |
| atoti.loading.sql.errored | Sum | source.type, topic.name | Tracks the number of SQL queries run to load data, which failed. |
| atoti.loading.sql.lines.errored | Sum | source.type, topic.name | Tracks the number of rows, which were ingested but could not be loaded into the datastore. |
| atoti.loading.sql.lines.ingested | Sum | source.type, topic.name | Tracks the number of rows ingested. |
| atoti.loading.sql.time | Histogram | source.type, topic.name | Measures the time (in ms) required to run the query and feed the datastore. |
| atoti.query.hit | Sum | type, success, cube.name | Counts the total number of queries executed in the cube. |
| atoti.query.time | Histogram | type, success, cube.name | Measures the time taken for cube queries, with the ability to filter by type, success, and cube name. |
| atoti.table.chunk.size | Gauge | table.name | Measures the size of an allocated chunk of a table. |
| atoti.table.epoch.current | Gauge | table.name | Latest epoch visible in a table. |
| atoti.table.lines.added | Sum | table.name | Counts the number of lines added to a table. |
| atoti.table.lines.deleted | Sum | table.name | Counts the number of lines deleted from each table. |
| atoti.table.lines.freed | Sum | table.name | Counts the number of line freed from each table. |
| atoti.table.lines.updated | Sum | table.name | Counts the number of lines updated in each table. |
| atoti.table.partition.created | Sum | table.name | Counts the number of table partitions that have been created. |
| atoti.table.partition.current_epoch | Gauge | table.name, partition.id | Latest epoch of a table partition in the datastore. |
| atoti.table.partition.dropped | Sum | table.name | Counts the number of table partitions that have been dropped. |
| atoti.table.partition.memory.footprint | Gauge | table.name, partition.id | Measures the memory usage of a partition of a table. |
| atoti.table.partition.size | Gauge | table.name, partition.id, date:value (for each partition key) | Provides the size of each table partition. Tags help track partition keys and values, identifying significant differences in partition sizes. |
| atoti.table.size | Gauge | table.name | Indicates the current size of the table, reflecting the difference between lines added and deleted over the table's lifetime. |
| atoti.table.size.memory.footprint | Gauge | table.name | Measures the memory footprint of a table, including all versions. This should equal lines added plus lines updated minus lines freed over the table's lifetime. |
| atoti.transaction.commit.hit | Sum | success | Tracks the total number of database transactions, incrementing by one for each transaction. The success tag helps identify abnormal rollback rates. |
| atoti.transaction.commit.time | Histogram | success | Measures the time taken for database commits. The success tag allows filtering between successful and rollback transactions, providing insights into the performance of each. |
| atoti.query.continuous.registered | Sum | cube.name | Counts the number of continuous queries currently registered in each cube. |
| atoti.query.continuous.total | Gauge | cube.name | Counts the number of continuous queries currently registered in each cube. |