The data load controller is designed to manage the loading (and
unloading) of data in all accelerators.
Loading of data is expected to be triggered on start-up (“initial load”)
and via a REST service by the external orchestration process used to
provide the data to the accelerator–for example, the same process that
copies files into a directory for the accelerator to load will also use
the REST service to tell the Data Load Controller to load the data into
the accelerator.
Scope
The scope describes which data is loaded (and unloaded). For example,
the has an as-of date scope for all daily data; it would also be
possible to add a “desk” parameter to the scope if you wanted to load
data by desk.
Each request to the Data Load Controller contains a scope. In the
reference implementation, the as-of date scope is turned into a
directory and all the files in that directory are loaded. When unloading
data, the scope is turned into a RemoveWhere clause to be run on the
datastore.