> ## Documentation Index
> Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Getting started with the data load controller

export const productName = "Atoti FRTB";

The Data Load Controller lets you load and unload data without worrying
about the underlying implementation details.

<Note>
  For the Data Load Controller documentation, click [here](https://docs.activeviam.com/products/tools/data-connectors/latest/online-help/dlc-overview).
</Note>

## Overview

The Data Load Controller supports managing data within {productName}.
It provides an interface between the ETL (Extract, Transform Load) layer
and the management of what data is loaded into {productName}.

There are three main parameters in this interface (all of which are
configurable):

* **Operation**: the operation the DLC performs. Typically, this is
  load or unload.
* **Topics**: the list of data sources defined in the ETL for which to
  perform the operation. Topic aliases are used to easily specify
  groups of topics.
* **Scope**: a filter on the data to be loaded. For example, to
  load/unload data for a particular as-of date.

 

## Initial load

Starting up the application triggers the initial loading of data. This
is managed programmatically within the Solution, with Spring beans
initiating the loading of topics and scopes at startup.

For more information, see [Initial Load](./dlc-frtb-config#initial-load).

## Data Orchestrator

Beyond the initial data load, the data orchestrator is a custom external
component that is responsible for managing (loading and unloading) data
in {productName}.

For example, the same process that copies the data ready for the
Solution to load can also use the REST service to tell the Data Load
Controller to trigger the loading of the data into the Solution.

## REST Service

The DLC provides a RESTservice, which the data orchestrator or other
external components can use to manage the loading and unloading of data in
the Solution.

The DLC REST API has two endpoints:

* **execute**: for loading and unloading data.
* **status**: to check on a previous command.

REST URL for execute:

```html theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
http://<hostname>:<port>/frtb-starter/connectors/rest/dlc/execute
```

<Note>
  The `FRTBDataLoadControllerRestController` and its rest service `/frtb-starter/services/rest/dlc/execute` are deprecated and will be removed in future versions. Use the new REST service `/frtb-starter/connectors/rest/dlc/v1/execute` instead.
</Note>

### Execute

For `execute`, requests use POST and send JSON objects with the
following fields:

**Operation**: 'load' or 'unload'.

* **load**: load the data according to topic and scope.
* **unload**: delete data from the datastore according to topic and
  scope. This doesn't depend on how or when the data was loaded.

<Note>
  This doesn't delete reference data that was added as part of the data normalization. For example, if you delete sensitivities, the corresponding risk-factors are not removed.
</Note>

**Topics**: a list of topics to load or unload. In addition to the topic
for each file type (see [Input file formats](../../../input-files/index)),
aliases are defined to load sets of files. For a list of FRTB-specific topic
aliases, see [Topic Aliases](./dlc-frtb-config#topic-aliases).

**Scope**: three scopes are defined in {productName}: as-of date,
configuration, and historical. For more information on these, see
[Scopes](./dlc-frtb-config#scopes).

This example snippet loads FXData for the 27th of September, 2018.

```json theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
{  
"operation": "LOAD",  
"topics": ["FXData"],  
"scope": {  
"AsOfDate": "2018-09-27",  
}  
}
```

This example snippet removes the 27th of September, 2018 from the SA and
IMA cubes:

```json theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
{  
"operation":"unload",  
"topics":[  
"ALLSA", "ALLIMADaily"  
],  
"scope":{  
"AsOfDate":"2018-09-27"  
}
}
```

#### Loading data into a branch

You can add the 'Branch' keyword to any scope section to specify a
branch in the cube for data update. On the 'load' feature, it is used to
find the resource to load (mainly the path for the file). On the
'unload' feature, it is used to build the remove criteria (Condition).

For example, here's how to load new FXData on the "test" branch:

```json theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
{  
"operation": "LOAD",  
"topics":[  
"FXData"  
],  
"scope": {  
"AsOfDate": "2018-09-28",  
"Branch": "test"  
}  
}
```

### Status

The status endpoint can be used to query the status of a request.

The Processing ID (also called Transaction ID) for a particular load is
returned as part of the response. The status is already part of the
load/unload response, but if you want to check it again, you can make
the following status call through **GET**:

```html theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
REST URL:
http://<hostname>:<port>/frtb-starter/connectors/rest/dlc/v1/status?processingId=\<Processing ID/>
```