LOAD - Atoti Hub

Overview

The following request fields control which topics load, into which branch, and with what scope.

Request

Java
JSON

`DlcLoadRequest`

Key	Required	Type	Description
topics	Y	`Set<String>`	Set of topics or aliases.
topicOverrides		`Set<ILoadingTopicDescription>`	Topic overrides.
scope		`IDlcScope`	Scope Parameters
branch		`String`	The branch to operate on.
performGcOnStart		`Boolean`	Perform a Garbage collection before the datastore transaction.
performGcOnCompletion		`Boolean`	Perform a Garbage collection after the datastore transaction.
sourceName		`String`	The source name to operate on. See source to topic matching.
sourceType		`DlcSourceType`	The source type to operate on. See source to topic matching.

Example:

DlcLoadRequest.builder()
    .topics("Topic1", "Topic2") // Required
    .topicOverrides(Set.of(CsvTopicDescription.builder("TopicToOverride", "glob:file*.csv")...build()))
    .scope(DlcScope.of("ScopeKey1", "ScopeValue1"))
    .branch("BranchToLoadInto")
    .sourceName("NameOfSourceToUse")
    .sourceType(DlcSourceType.LOCAL_SOURCE)
    .build();

`DlcLoadRequestDTO`

Key	Required	Type	Description
topics	Y	`Set<String>`	Set of topics or aliases.
csvTopicOverrides		`Map<String, CsvTopicProperties>`	CSV Topic overrides.
jdbcTopicOverrides		`Map<String, JdbcTopicProperties>`	JDBC Topic overrides.
scope		`Map<String, Object>`	Scope Parameters
branch		`String`	The branch to operate on.
performGcOnStart		`Boolean`	Perform a Garbage collection before the datastore transaction.
performGcOnCompletion		`Boolean`	Perform a Garbage collection after the datastore transaction.
sourceName		`String`	The source name to operate on. See source to topic matching.
sourceType		`DlcSourceType`	The source type to operate on. See source to topic matching.

Example:

{
  "operation": "LOAD",
  "topics": ["Topic1", "Topic2"],
  "csvTopicOverrides": {
    "TopicToOverride": {
      "path": "glob:file*.csv"
    }
  },
  "scope": {
    "ScopeKey1": "ScopeValue1"
  },
  "branch": "BranchToLoadInto",
  "performGcOnStart": true,
  "performGcOnCompletion": true,
  "sourceName": "NameOfSourceToUse",
  "sourceType": "LOCAL_SOURCE"
}

Response

Java
JSON

`DlcLoadResponse`

Key	Type	Description
report	`LoadReport`	Report of what occurred during the loading operation.
status	`DlcStatus`	Status of the DLC Operation.

`LoadReport`

Key	Type	Description
parsingReport	`Map<String, Map<String, ITopicParsingReport>>`	Per source and per topic parsing report provided by the Atoti API. Will contain fine-grain details. Needs to be enabled either in the Source Configuration or through global property `activeviam.msg.reporting.enabled`
errors	`Map<String, List<DlcMessageCollector.OnError>>`	Errors that occurred while loading.
messagesPublished	`Map<String, List<DlcMessageCollector.OnMessagePublished>>`	The Atoti message when the records are published to the datastore.
messageStarted	`Map<String, List<DlcMessageCollector.OnMessageStarted>>`	The Atoti message when the records are sent to the Tuple Publisher.

`DlcStatus`

A simple enum to capture if the DLC request was executed.

Key	Description
OK	Request was handled successfully.
ERROR	Request was not able to be performed correctly.

`DlcLoadResponseDTO`

Key	Type	Description
parsingReport	`Map<String, Map<String, ITopicParsingReport>>`	Per source and per topic parsing report provided by the Atoti API. Will contain fine-grain details. Needs to be enabled either in the Source Configuration or through global property `activeviam.msg.reporting.enabled`
errors	`Map<String, List<DlcMessageCollector.OnError>>`	Errors that occurred while loading.
messagesPublished	`Map<String, List<DlcMessageCollector.OnMessagePublished>>`	The Atoti message when the records are published to the datastore.
messageStarted	`Map<String, List<DlcMessageCollector.OnMessageStarted>>`	The Atoti message when the records are sent to the Tuple Publisher.
status	`DlcStatus`	Status of the DLC Operation.

Example:

{
  "parsingReport": {},
  "errors": {},
  "messagesPublished": {
    "sourceName": [
      {
        "key": "FileSystemFileInfo [fullName=root/file.csv, lastModifiedTime=Tue Feb 25 16:14:22 EST 2025]",
        "info": {
          "publishedObjectCount": 30,
          "characterCount": 6342,
          "lineCount": 31,
          "byteCount": 6342,
          "elapsedTime": 0.021108000,
          "aggregatedTime": 15301750,
          "aggregatedReadingTime": 3976791,
          "aggregatedDecodingTime": 78834,
          "aggregatedParsingTime": 702750,
          "aggregatedPublicationTime": 10543375,
          "exceptions": [],
          "creationTime": "2025-03-18T00:59:18.449823Z",
          "numberSkippedLines": 1
        }
      }
    ]
  },
  "messageStarted": {
    "sourceName": [
      {
        "key": "FileSystemFileInfo [fullName=root/file.csv, lastModifiedTime=Tue Feb 25 16:14:22 EST 2025]"
      }
    ]
  },
  "status": "OK"
}

​Overview

​Request

​DlcLoadRequest

​Example:

​DlcLoadRequestDTO

​Example:

​Response

​DlcLoadResponse

​LoadReport

​DlcStatus

​DlcLoadResponseDTO

​Example:

Overview

Request

`DlcLoadRequest`

Example:

`DlcLoadRequestDTO`

Example:

Response

`DlcLoadResponse`

`LoadReport`

`DlcStatus`

`DlcLoadResponseDTO`

Example: