> ## Documentation Index > Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt > Use this file to discover all available pages before exploring further. # Unload topic > How to configure `UnloadTopicDescription` to remove facts from a datastore table using a removal condition factory, including custom and implicit unload topics. Custom unload topics are defined with a `removalConditionFactory`, while implicit unload topics are created automatically from the request scope. # Configuration ## YAML configuration The unload topics can be defined via YAML configuration. The topics name must match a datastore table's name. The list of field names per topic name must all be present in the scope when performing the unload operation for that topic. The unload operation will remove all facts from the store where the field equals the value provided in the scope. The DLC also has a feature to create [implicit unload topics](#implicit-unload-topics). This feature can be enabled or disabled. ```yaml theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} dlc: unload: implicit-topics-enabled: true topics: BaseStore: - CobDate - ParameterSet ``` ## Java configuration The unload topics can be configured by defining `UnloadTopicDescription` beans in the application. ## `UnloadTopicDescription` The `UnloadTopicDescription` is used to define the configuration of an unload topic. The following properties are available: | Parameter | Required | Type | Default | Description | | ----------------------- | :------: | ----------------------------------------------------- | :---------------------------------------- | --------------------------------------------------------------------- | | name | Y | `String` | | Name of the topic. | | stores | Y | `Set` | Empty `Set` | Set of stores this topics will unload from. | | removalConditionFactory | Y | `BiFunction` | Factory that creates a `False` condition. | Factory that creates the condition to filter the facts to be removed. | ### Removal condition factory The `removalConditionFactory` is a `BiFunction` that takes an `IStoreDescription` and a `DlcScope` and returns an `ICondition`. This condition is used to filter the facts to be removed from the store. The `UnloadTopicDescription.of(String name, Map> fieldNamesPerStore)` will create a `removalConditionFactory` that will, for each store (`fieldNamesPerStore` keys), create a condition where each field matches the provided field value in the scope. This factory will only work if all field names are provided in the scope. #### Example All facts that have an `AsOfDate` of `2021-01-01` can be removed from the store by specifying a scope of: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} DlcScope.of( "AsOfDate", "2021-01-01" ); ``` All facts that have an `AsOfDate` of `2021-01-01` and that have a `Currency` of `usd` can be removed from the store by specifying a scope of: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} DlcScope.of( "AsOfDate", "2021-01-01", "Currency", "USD" ); ``` #### Data type considerations It is important to note the following: * The `removalConditionFactory` should be able to ensure the `DlcScope` keys are properly typed. Since `ICondition`s are strict filters, if the datatype of the scope value does not align with the datatype of the store column, then no data will be unloaded. * When unloading a `DlcScope` that was provided via a REST request, the scope values may be of type `String`. ## Custom unload topics Custom unload topics can be created by defining `UnloadTopicDescription` beans in the application. The removal condition factory can be customized, or the `DlcUnloadOperationHelperUtil` can be used to create removal conditions. The following example creates an unload topic that removes facts from the "StoreName" store where "FiledName\_1" and "FieldName\_2" field values come from the scope. ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} @Bean public UnloadTopicDescription simpleUnloadTopic(){ return UnloadTopicDescription.of( "CustomUnloadTopic", Map.of("StoreName", Set.of("FieldName_1", "FieldName_2")) ); } ``` The following example creates an unload topic that removes all trades that contain a specific trade id: ```java theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} @Bean public UnloadTopicDescription fuzzyUnloadTopic(){ return UnloadTopicDescription.builder("UnloadFuzzyTradeTopic") .stores(Set.of("BaseStore")) .removalConditionFactory( (store, scope) -> { // Verify scope contains the expected keys assert(scope.containsKey("tradeId")); // Scope value may end with a wildcard String tradeSearchValue = (String) scope.get("tradeId"); tradeSearchValue = tradeSearchValue.replace("*", ""); // Remove all trades that contain the provided search value return BaseConditions.containsIgnoreCase( FieldPath.of("tradeId"), tradeSearchValue ); } ) .build(); } ``` This topic can be used with a scope of `("tradeId", "Trade_123*")` to remove all trades that have a trade id containing the substring `"Trade_123"`. ## Implicit unload topics Implicit unload topics allow data to be unloaded from a datastore table without explicitly defining an unload topic in the configuration. Please be aware of the following when using implicit topics: * The usage of implicit topics exposes the application to requests that may be overly flexible, allowing users to unload data from any table for any field/value pair. * The request must know the details of the datastore schema. The usage of implicit unload topics can be enabled via the `implicit-topics-enabled` property ```yaml theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} dlc: unload: implicit-topics-enabled: true ``` When using implicit unload topics: * The topic name **must** match the name of an existing datastore table. * The removal condition will be resolved from the scope of the incoming request. ##### Example ```json theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}} { "operation": "UNLOAD", "topics": ["TradeTable"], "scope": { "tradeID": "IRS_GBP_CPI 35a38b80" } } ``` The DLC will create an implicit `UnloadTopicDescription` at request time when a topic is requested that both matches the name of a datastore table and no `UnloadTopicDescription` exists in the DLC configuration. ### Implicit `UnloadTopicDescription`s | Parameter | Value | | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | `name` | Name of the store.
Example: `"TradeBaseStore"`. | | `stores` | As above, the name of the store in question.
Note: The value is just the one store.
Example: `List ["TradeBaseStore"]` | | `removalConditionFactory` | Default `IUnloadTopicDescription.SCOPE_TO_REMOVAL_CONDITION` | ### Garbage collection It may be required to perform a Garbage collection after an unload operation. Please see the [Garbage collection](/functional/scope-parameters#garbage-collection) section for details.