> ## Documentation Index
> Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Getting started with the Data Load Controller

The Data Load Controller lets you load and unload data without worrying
about the underlying implementation details.

<Note>
  For the Data Load Controller documentation, click [here](https://docs.activeviam.com/products/modules/data-connectors/latest/online-help/).
</Note>

## Overview

The Data Load Controller supports managing data within Atoti Market Risk.
It provides an interface between the ETL (Extract, Transform Load) layer
and the management of what data is loaded into Atoti Market Risk.

There are three main parameters in this interface (all of which are
configurable):

* **Operation**: the operation the DLC performs. Typically, this is
  load or unload.
* **Topics**: the list of data sources defined in the ETL for which to
  perform the operation. Topic aliases are used to easily specify
  groups of topics.
* **Scope**: a filter on the data to be loaded. For example, to
  load/unload data for a particular as-of date.

## Initial load

Starting up the application triggers the initial loading of data. This
is managed programmatically within Atoti Market Risk, with Spring beans
initiating the loading of topics and scopes at startup.

For more information, see [Initial Load](./dlc-mr-config#initial-load).

## Data Orchestrator

Beyond the initial data load, the data orchestrator is a custom external
component that is responsible for managing (loading and unloading) data
in Atoti Market Risk.

For example, the same process that copies the data ready for Atoti Market Risk to load can also use the REST service to tell the Data Load
Controller to trigger the loading of the data into Atoti Market Risk.

## REST Service

The DLC provides a REST service which the data orchestrator (or other
external component) can use to manage the loading and unload of data in
Atoti Market Risk.

The DLC REST API has two endpoints:

* **execute**: for loading and unloading data.
* **status**: to check on a previous command.

### Execute

For `execute`, requests use POST and send JSON objects with the
following fields:

**Operation**: ’load’ or ‘unload’.

* **load**: load the data according to topic and scope.
* **unload**: delete data from the datastore according to topic and
  scope. This doesn’t depend on how or when the data was loaded.

<Note>
  This doesn’t delete reference data that was added as part of the data normalization. For example, if you delete sensitivities, the corresponding risk-factors are not removed.
</Note>

**Topics**: a list of topics to load or unload. In addition to the topic
for each file type (see [Input file formats](../../../input-files)),
aliases are defined to load sets of files. For a list of MR-specific topic
aliases, see [Topic Aliases](./dlc-mr-config#topic-aliases).

**Scope**: three scopes are defined in Atoti Market Risk: as-of date,
configuration, and historical. For more information on these, see
[Scopes](./dlc-mr-config#scopes).

This example snippet loads FXData for the 27th of September, 2018.

```json theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
{
"operation": "LOAD",
"topics": ["FXData"],
"scope": {
"AsOfDate": "2018-09-27"
}
}
```

This example snippet removes the 27th of September, 2018 from the Sensitivity cube:

```json theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
{
"operation":"UNLOAD",
"topics":[
"Sensitivities"
],
"scope":{
"AsOfDate":"2018-09-27"
}
}
```

#### Loading data into a branch

You can add the ‘Branch’ keyword to any scope section to specify a
branch in the cube for data update. On the ’load’ feature, it is used to
find the resource to load (mainly the path for the file). On the
‘unload’ feature, it is used to build the remove criteria (Condition).

For example, here’s how to load new FXData on the “test” branch:

```json theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
{
"operation": "LOAD",
"topics":[
"FXData"
],
"scope": {
"AsOfDate": "2018-09-28",
"Branch": "test"
}
}
```

### Status

The status endpoint can be used to query the status of a request.

The Processing ID (also called Transaction ID) for a particular load is
returned as part of the response. The status is already part of the
load/unload response, but if you want to check it again, you can make
the following status call through **GET**:

```html theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
REST URL:
http://localhost:10010/mr-application/services/rest/dlc/status?processingId=\<Processing
ID/>
```