Implementing Atoti What-If simulations

This section provides a detailed example of using Atoti What-If to implement what-if scenarios through simulations.

Importing the library in your project

To use Atoti What-If, import the Spring Boot Starter in your Maven configuration:

<dependency>
    <groupId>com.activeviam.apps</groupId>
    <artifactId>atoti-what-if-spring-boot-starter</artifactId>
    <version>5.0.0-AS6.1</version>
</dependency>

Released versions of Atoti What-If are built using a specific version of Atoti Server but will work with any non-breaking patch version of the core library.

Using the Spring Boot Starter (recommended)

The Spring Boot Starter auto-configures all What-If components. You only need to provide two beans:

IDatabaseService — provided by your Atoti Server application
IWhatIfPersistenceProperties — provides Hibernate configuration for persistence

@Bean
public IWhatIfPersistenceProperties whatIfPersistenceProperties(
        @Value("classpath:hibernate.properties") Resource resource) throws IOException {
    Properties props = new Properties();
    props.load(resource.getInputStream());
    return () -> props.entrySet().stream()
        .collect(Collectors.toMap(
            e -> String.valueOf(e.getKey()),
            e -> String.valueOf(e.getValue())
        ));
}

The starter automatically creates and wires:

DatabaseSimulationEngine
IUniqueIdGenerator
ISimulationPersistenceManager
IDatabaseSimulationsSecurityManager
DatabaseSimulationsWorkflow
DatabaseSimulationsRestService

For distributed environments, enable distributed mode and specify the query node name:

atoti:
  what-if:
    distribution:
      enabled: true
      query-node-name: "MyQueryNodeName"

See the configuration reference for all available properties, and how to set up the Spring Boot Starter for more details.

Manual configuration (advanced)

To enable the creation of Atoti What-If simulations within your project, you need to create several objects.

The `IDatabaseService` implementation

All Atoti Server applications will contain an IDatabaseService implementation, which will be suitable for Atoti What-If in most cases. Distributed environments are a special case, where the query node requires a custom IDatabaseService with access to the data nodes. We provide a RestDistributedDatabaseService implementation for this scenario, which would typically be used as follows:

@Bean
@ConditionalOnQueryNode
@ConditionalOnMissingBean
public AddressSupplier addressSupplier() {
    return () -> mvPivot.getClusterMembersRestAddresses().values().stream()
            // for backwards compatibility, add protocol if missing
            .map(addr -> addr.regionMatches(true, 0, "http", 0, 4) ? addr : ("http://" + addr))
            .collect(Collectors.toSet());
}

@Bean
@ConditionalOnQueryNode
@ConditionalOnMissingBean
public AuthenticatorSupplier authenticatorSupplier(IJwtService jwtService) {
    return () -> new JwtAuthenticator(jwtService);
}

@Bean
@Primary
@ConditionalOnQueryNode
public IDatabaseService restRemoteService(AddressSupplier addressSupplier, AuthenticatorSupplier authenticatorSupplier) {
    return new RestDistributedDatabaseService(addressSupplier, authenticatorSupplier, () -> mvPivot);
}

In this example, we rely on an IMultiVersionDistributedActivePivot object to retrieve the addresses of the cluster members (data nodes). These addresses will be used internally to forward simulation executions and merge the results. We also need to create an AAuthenticator supplier which will provide the security context of the thread executing a request to the RemoteDatabaseService connection. This ensures access to the data node databases is secured through built-in authentication and authorization.

The persistence manager

Simulation persistence is enabled through the creation of an ISimulationPersistenceManager object:

@Bean
public ISimulationPersistenceManager persistenceManager(WhatIfHibernateProperties whatIfHibernateProperties) {
    var configuration = new org.hibernate.cfg.Configuration();
    var properties = new Properties();
    properties.putAll(whatIfHibernateProperties.getPersistence());
    configuration.addProperties(properties);
    configuration.addAnnotatedClass(DatabaseSimulationJPA.class);
    var sessionFactory = configuration.buildSessionFactory();

    HibernateSimulationPersistenceManager<DatabaseSimulationJPA> persistenceManager = HibernateSimulationPersistenceManager.of(DatabaseSimulationJPA.class);
    persistenceManager.setSessionFactory(sessionFactory);
    return persistenceManager;
}

This example uses a non-auditable Hibernate persistence manager. The following implementations of a persistence manager are available in the Atoti What-If library:

Manager	Description
HibernateSimulationPersistenceManager	Persists simulations in an SQL database, using Hibernate.
HibernateAuditablePersistenceManager	Persists simulations in an SQL database, using Hibernate. Adds audit capabilities to the persistence layer.

The security manager

To handle simulation permissioning, the following implementations of IDatabaseSimulationsSecurityManager are provided:

Manager	Description
SpringDatabaseSimulationsSecurityManager	A security manager that uses Spring security to retrieve user authentication and authorization.
NoOpDatabaseSimulationsSecurityManager	A security manager that allows all actions to be performed by all users.

The configuration class should use the appropriate implementation:

@Bean
public IDatabaseSimulationsSecurityManager securityManager(ISimulationPersistenceManager persistenceManager, IBranchPermissionsManager branchPermissionsManager) {
    return new SpringDatabaseSimulationsSecurityManager(persistenceManager, branchPermissionsManager);
}

The simulation engine

An instance of DatabaseSimulationEngine has to be created, and the associated DatabaseSimulationsUtils class should use the same IDatabaseService instance:

@Bean
public DatabaseSimulationEngine simulationEngine(IDatabaseService databaseService) {
    DatabaseSimulationEngine engine = new DatabaseSimulationEngine();
    engine.setDatabaseService(databaseService);
    DatabaseSimulationsUtils.setDatabaseService(databaseService);
    return engine;
}

The unique ID generator

An implementation of IUniqueIdGenerator is required for the workflow to correctly assign IDs to simulation executions. The Atoti What-If library contains the IncrementalUniqueIdGenerator, providing an incremental counter starting from the Epoch second. Alternatively, we can use a custom implementation:

private static class UniqueIdGenerator implements IUniqueIdGenerator {
    private final AtomicLong generator = new AtomicLong(0);

    @Override
    public Long generateId() {
        synchronized (generator) {
            return generator.getAndIncrement();
        }
    }
}

@Bean
public IUniqueIdGenerator simulationIdGenerator() {
    return new UniqueIdGenerator();
}

The simulation workflow

An instance of DatabaseSimulationsWorkflow will then use the objects defined in our configuration:

@Bean
public DatabaseSimulationsWorkflow simulationWorkflow(
        DatabaseSimulationEngine engine,
        ISimulationPersistenceManager manager,
        IDatabaseSimulationsSecurityManager securityManager,
        IDatabaseService databaseService,
        IUniqueIdGenerator idGenerator) {
    return new DatabaseSimulationsWorkflow(
            engine,
            manager,
            securityManager,
            databaseService,
            idGenerator);
}

The simulation REST service

Finally, to enable UI interactions with the workflow and persistence, create a DatabaseSimulationsRestService:

@Bean
public DatabaseSimulationsRestService simulationsService(
        ISimulationPersistenceManager manager,
        DatabaseSimulationsWorkflow workflow,
        IDatabaseService databaseService) {
    return new DatabaseSimulationsRestService(manager, workflow, databaseService);
}

Creating definition implementations

Definition implementations are use-case specific implementations of the IDatabaseSimulationDefinition interface, with specific required parameters, logic for creating JsonDatabaseAction objects and a method for the retrieval of before/after values for a specific instance of the definition.

The IDatabaseSimulationDefinition interface and ADatabaseSimulationDefinition abstract class

The IDatabaseSimulationDefinition interface is an extension of the core IExtendedPluginValue containing the following methods:

Method	Return type	Description
`getParameters()`	`Map<String, String>`	The implementation should return a map of the definition instance parameters as Name -> Serialized value.
`toJsonDatabaseEdit()`	`JsonDatabaseEdit`	The implementation should transform the parameters of the definition instance into a `JsonDatabaseEdit` to be sent to the `IDatabaseService`. This method will be called by the `DatabaseSimulationEngine`.
`getDescription()`	`String`	The implementation should return the description of the definition.
`getDiffs(IDatabaseService databaseService)`	`List<DatabaseSimulationDiffDTO>`	The implementation should provide a mechanism to retrieve a list of `DatabaseSimulationDiffDTO` from a database service.

The ADatabaseSimulationDefinition class is an abstract class providing a skeleton implementation of the IDatabaseSimulationDefinition interface. Abstract methods are:

Method	Return type	Description
`getAllParameterNames()`	`Set<String>`	A method for retrieving all the relevant parameter names, for instantiating the parameters `Map<String, String>`.
`generateDatabaseActions()`	`JsonDatabaseAction[]`	A method for generating an array of `JsonDatabaseAction` to use in a `JsonDatabaseEdit`.

Implemented or stubbed methods are:

Method	Description
`getParameters()`	Returns a stored `Map<String, String>` of parameters.
`addParameter(String key, Object value)`	Adds a parameter to the parameters `Map<String, String>`.
`getParameter(String key)`	Returns the `String` representation of the requested parameter, from the parameters `Map<String, String`>.
`getParameter(String key, TypeReference<T> type)`	Returns a deserialized parameter object from the parameters `Map<String, String>`, using a Jackson `TypeReference`. Intended to be used with `Collections` types.
`getParameter(String key, Class<T> class)`	Returns a deserialized parameter object from the parameters `Map<String, String>` using the class object passed in.
`toJsonDatabaseEdit()`	Implementation of the interface method that wraps the `generateDatabaseActions()` result in a `JsonDatabaseEdit` object.
`getDiffs(IDatabaseService databaseService)`	Stub implementation of the interface method which returns a list containing a `DatabaseSimulationDiffDTO` with all fields set to “N/A”.

Required constructors

As all IDatabaseSimulationDefinition instances are also IExtendedPluginValue objects that are instantiated by the Atoti Server Registry, a specialized constructor is provided in the ADatabaseSimulationDefinition class.

While completely custom implementations of IDatabaseSimulationDefinition can be created, it is strongly recommended to extend the ADatabaseSimulationDefinition class. The specialized constructor of the custom implementation can then be:

public MyCustomSimulationDefinition(Map<String, Object> parameters) {
    super(parameters);
}

Without a constructor accepting Map<String, Object> parameters as an input, the Registry will not be able to instantiate the definitions.

Custom constructors can then be implemented to take any input and be passed through to the getParameters() method.

Creating concrete definitions for a given use-case

Starting from the ADatabaseSimulationDefinition abstract class detailed above, and taking into account the required constructors, a typical definition implementation would be:

Class annotations

The concrete class is annotated to be an extended plugin value, with an associated plugin key:

@QuartetExtendedPluginValue(intf = IDatabaseSimulationDefinition.class, key = PnLArithmeticSimulationDefinition.PLUGIN_KEY)
public class PnLArithmeticSimulationDefinition extends ADatabaseSimulationDefinition {
    public static final String PLUGIN_KEY = "PnLBookScalingSimulationDefinition";

    ...

    @Override
    public String getType() {
        return PLUGIN_KEY;
    }
}

Constructors

The two constructors for this implementation are:

public PnLArithmeticSimulationDefinition(Map<String, Object> parameters) {
    super(parameters);
}

public PnLArithmeticSimulationDefinition(BranchAwareAdjustmentRequestDTO dto, boolean isAbsolute) {
    super(Map.of(
        DTO, dto,
        IS_ABSOLUTE, isAbsolute
    ));
}

In this example, the BranchAwareAdjustmentRequestDTO is a DTO containing all the relevant information related to the simulation parameters. The second constructor isn’t required, as the Map<String, Object> can be created externally and passed to the mandatory constructor.

The JsonDatabaseAction generation method

The definition in this example will use the input parameters to create a single action that applies an arithmetic operation to a field in a database table:

@Override
protected JsonDatabaseAction[] generateDatabaseActions() {
    BranchAwareAdjustmentRequestDTO dto = getParameter(DTO, BranchAwareAdjustmentRequestDTO.class);
    boolean isAbsolute = getParameter(IS_ABSOLUTE, Boolean.class);

    JsonNode conditionNode = DatabaseSimulationsUtils.baseConditionToJsonNode((IBaseCondition) convertToCondition(dto));

    JsonDatabaseAction[] actions = List.of(
            DatabaseSimulationsUtils.newUpdateDatabaseAction(
                    TRADE_PNL_STORE_NAME,
                    conditionNode,
                    ArithmeticUpdateProcedureFactory.generateProcedureAsJsonNode(
                            PNL_VECTOR,
                            isAbsolute ? ArithmeticUpdateProcedureFactory.Operation.PLUS : ArithmeticUpdateProcedureFactory.Operation.SCALE,
                            Double.valueOf(dto.valueOfInput(StoreFieldConstants.SENSITIVITY_VALUES))
                    )
            )
    ).toArray(JsonDatabaseAction[]::new);

    return actions;
}

Any JsonDatabaseAction type can be implemented within this method.

Helper methods for creating JsonDatabaseAction objects

To simplify the creation of JsonDatabaseAction objects, several helper methods are provided in the DatabaseSimulationsUtils utility class.

General helper methods

We include methods for converting to and from JsonNode objects as used by the database service APIs.

Method	Return type	Description
`baseConditionToJsonNode(IBaseCondition condition)`	`JsonNode`	Converts an `IBaseCondition` object into a `JsonNode`.
`stringToJsonNode(String string)`	`JsonNode`	Converts a `String` to a `JsonNode`.
`jsonNodeToString(JsonNode node)`	`String`	Converts a `JsonNode` to a `String`.
`mapToJsonNode(Map<String, Object> map)`	`JsonNode`	Converts a `Map<String, Object>` to a `JsonNode`.

Actions by type

Type-specific methods are included for all JsonDatabaseAction types:

Method	Type	Description
`newAddDatabaseAction(String baseTable, String[][] tuples)`	`JsonDatabaseAction` for inserting new tuples.	Creates a `JsonDatabaseAction` for the insertion of the given tuples (as an array of `String[]`) into the specified table.
`newRemoveDatabaseAction(String baseTable, JsonNode condition)`	`JsonDatabaseAction` for deleting rows from a table.	Creates a `JsonDatabaseAction` that removes rows matching the condition from the specified table.
`newUpdateDatabaseAction(String baseTable, JsonNodeCondition condition, JsonNode updateProcedure)`	`JsonDatabaseAction` for executing an update procedure on rows in a table.	Creates a `JsonDatabaseAction` that executes an update procedure on the rows in the specified table that match the given condition.
`newDuplicateDatabaseAction(String baseTable, JsonNode condition, String branch, Map<String, String> suffixes)`	`JsonDatabaseAction` for duplicating rows in a table.	Retrieves rows in the given table for the branch, adds the specified suffixes to the appropriate fields and generates a `JsonDatabaseAction` that inserts the resulting tuples into the table.
`newDuplicateDatabaseAction(String baseTable, JsonNode condition, String branch, Map<String, String> suffixes, Map<String, Function<Object, Object>> overrides)`	`JsonDatabaseAction` for duplicating rows in a table.	Retrieves rows in the given table for the branch, applies any defined overrides, adds the specified suffixes to the appropriate fields and generates a `JsonDatabaseAction` that inserts the resulting tuples into the table.

IUpdateWhereProcedureFactory implementations

For the newUpdateDatabaseAction(String baseTable, JsonNodeCondition condition, JsonNode updateProcedure) method, we provide several implementations of the IUpdateWhereProcedureFactory interface, for use as the update procedure, through the following methods:

Factory	Method	Description
`ArithmeticUpdateProcedureFactory`	`generateProcedureAsJsonNode(String field, Operation operation, Double operator)`	Generates a `JsonNode` update-where procedure that applies the given operation (`PLUS`, `MINUS` or `SCALE`) to a field, with the given value.
`ArithmeticUpdateProcedureFactory`	`generateProcedureAsJsonNode(String field, Operation operation, Double operator, List<Integer> indicesInVector)`	Generates a `JsonNode` update-where procedure that applies the given operation (`PLUS`, `MINUS` or `SCALE`) to a field, with the given value. Optionally, a list of vector indices for which the operation applies can be supplied.
`UpdateWithFieldValuesProcedureFactory`	`generateProcedureAsJsonNode(Map<String, Object> fieldsToUpdate)`	Generates a `JsonNode` update-where procedure that changes the value of a field to a set value.
`UpdateWithListOfValuesProcedureFactory`	`generateProcedureAsJsonNode(Set<String> keys, List<Map<String, Object>> rowsToUpdate)`	Generates a `JsonNode` update-where procedure that selects records based on keys and fills in the remaining fields from the matched entry in the list of rows expressed as `Map<String, Object>`. Allows for a larger degree of flexibility than the `UpdateWithFieldValuesProcedureFactory`, as the field changes can be different for each key.

The DTOs

External data transfer is handled through DTO objects wrapping the definitions, simulations and diff objects within the simulation workflow. All DTO objects provide getters and setters for the instance parameters within the object.

The DatabaseSimulationDiffDTO

Field	Type	Description
store	`String`	The store for which the before and after values are retrieved.
filters	`Map<String, String>`	A map of field values used for the conditional retrieval of before and after values.
beforeValue	`String`	The value before the simulation has been executed, serialized into a `String`.
afterValue	`String`	The value after the simulation has been executed, serialized into a `String`.

The DatabaseSimulationDefinitionDTO

Field	Type	Description
definitionType	`String`	The plugin type of the definition instance.
definitionParameters	`Map<String, String>`	The parameters used in the definition instance, as a serialized `Map`.
definitionDescription	`String`	A description of the definition instance.

The DatabaseSimulationDefinitionDTO also offers conversion methods:

Method	Return type	Description
`from(String serialized)`	`DatabaseSimulationDefinitionDTO`	Conversion method from a serialized `String` into a concrete object.
`from(Map<String, Object> serialized)`	`DatabaseSimulationDefinitionDTO`	Conversion method from a serialized `Map` into a concrete object.
`toDefinition()`	`IDatabaseSimulationDefinition`	Conversion from the DTO into the exact simulation definition instance.

The DatabaseSimulationDTO

Field	Type	Description
simulationId	`Long`	The ID of the simulation.
simulationName	`String`	The name of the simulation.
createdBy	`String`	The user who created the simulation instance.
creationDate	`String`	The date of the creation of the simulation instance.
executedBy	`String`	The user who executed the simulation instance.
executionDate	`String`	The date of the execution of the simulation instance.
status	`String`	The status of the simulation.
branchName	`String`	The name of the branch on which the simulation has been executed.
parentBranchName	`String`	The parent branch of the simulation execution.
definitionType	`String`	The plugin type of the definition instance used in this simulation.
definitionParameters	`Map<String, String>`	The parameters used in the definition instance used in this simulation, as a serialized `Map`.
definitionDescription	`String`	A description of the definition instance used in this simulation.

The DatabaseSimulationDTO also offers a conversion method:

Method	Return type	Description
`toDatabaseSimulation()`	`IDatabaseSimulation`	Conversion from the DTO into the exact simulation instance.

Custom REST services

If the generic endpoints available in DatabaseSimulationsRestService are not enough, a REST service handling specific simulation types can be created:

@RestController
@RequestMapping(REST_API_URL_PREFIX)
public class TradeRescaleRestServiceController {

	private static final String NAMESPACE = "services/"+ RestPrefixExtractor.REST_NAMESPACE + "/whatif";
	public static final String REST_API_URL_PREFIX = "/" + NAMESPACE + "/tradescale";

	@Autowired
	private DatabaseSimulationsWorkflow workflow;

	protected static final Logger LOGGER = LoggerFactory.getLogger(TradeRescaleRestServiceController.class);

	@PostMapping(value = "/scaleTrade")
	public DatabaseSimulationStatus scaleTrade(@RequestBody TradeRescaleDTO dto) {
		LOGGER.info("[TRADE_RESCALING] Attempting to rescale trades: {}.", dto.getTradeID());
		IDatabaseSimulationDefinition definition = new TradeDuplicateAndRescaleSimulationDefinition(
				List.of(TradeDuplicateAndRescaleSimulationDefinition.RESCALE_SA, TradeDuplicateAndRescaleSimulationDefinition.RESCALE_IMA), dto);

		IDatabaseSimulation simulation = workflow.execute(new DatabaseSimulation(dto.getTitle(), definition, dto.getUserName(), dto.getBranch()));

		DatabaseSimulationStatus status = simulation.getStatus();

		if (DatabaseSimulationStatus.FAILED.equals(status)) {
			LOGGER.warn("[TRADE_RESCALING] Execution failed for Simulation ID: {}", simulation.getSimulationId());
		}
		if (DatabaseSimulationStatus.UNAUTHORISED.equals(status)) {
			LOGGER.warn("[TRADE_RESCALING] Execution unauthorized for user: {}", simulation.getCreatedBy());
		}
		LOGGER.info("[TRADE_RESCALING] Successfully rescaled trades: {}.", dto.getTradeID());
		return status;
	}
}

In this example, we expose a single endpoint for the creation and execution of an IDatabaseSimulation. The service creates an IDatabaseSimulationDefinition from a custom implementation, executes it through the DatabaseSimulationsWorkflow and returns the status object.

​Implementing Atoti What-If simulations

​Importing the library in your project

​Using the Spring Boot Starter (recommended)

​Manual configuration (advanced)

​The IDatabaseService implementation

​The persistence manager

​The security manager

​The simulation engine

​The unique ID generator

​The simulation workflow

​The simulation REST service

​Creating definition implementations

​The IDatabaseSimulationDefinition interface and ADatabaseSimulationDefinition abstract class

​Required constructors

​Creating concrete definitions for a given use-case

​Class annotations

​Constructors

​The JsonDatabaseAction generation method

​Helper methods for creating JsonDatabaseAction objects

​General helper methods

​Actions by type

​IUpdateWhereProcedureFactory implementations

​The DTOs

​The DatabaseSimulationDiffDTO

​The DatabaseSimulationDefinitionDTO

​The DatabaseSimulationDTO

​Custom REST services

Implementing Atoti What-If simulations

Importing the library in your project

Using the Spring Boot Starter (recommended)

Manual configuration (advanced)

The `IDatabaseService` implementation

The persistence manager

The security manager

The simulation engine

The unique ID generator

The simulation workflow

The simulation REST service

Creating definition implementations

The IDatabaseSimulationDefinition interface and ADatabaseSimulationDefinition abstract class

Required constructors

Creating concrete definitions for a given use-case

Class annotations

Constructors

The JsonDatabaseAction generation method

Helper methods for creating JsonDatabaseAction objects

General helper methods

Actions by type

IUpdateWhereProcedureFactory implementations

The DTOs

The DatabaseSimulationDiffDTO

The DatabaseSimulationDefinitionDTO

The DatabaseSimulationDTO

Custom REST services