Add custom data loading

The initial load of Atoti Limits data allows you to bulk load limits and incidents when the application first starts and connects to your Atoti Server instance. The loading process can be customized to fit your needs through configuration properties or by implementing your own custom service.

Initial load configuration properties

How to customize the initial load data location

By default, Atoti Limits will attempt to load the data from CSV files located in the directory specified by the limits.initial-load.root-dir property. When Atoti Limits starts and connects to your Atoti Server instance, it will look in the location specified by this property for a subdirectory with the name of your Atoti Server instance which is specified by the limits.autoconfiguration.server-name property.

The limits.autoconfiguration.server-name property is autoconfigured by default, but you can set it explicitly in the configuration for your Atoti Server if needed.

The default directory for Atoti Limits data is ./src/main/resources/data. If you need to change this location, set the limits.initial-load.root-dir property to the desired path in your application.yml file.

How to customize the initial load data file patterns

Within the directory specified by the limits.initial-load.root-dir property, Atoti Limits will look for files that match the patterns specified by the limits.initial-load.file-path-matchers.* properties. The default patterns are:

Property Name	Default Value	Description
`limits.initial-load.file-path-matchers.limits`	`glob:*/limits_approve*.csv`	Pattern to match limit files
`limits.initial-load.file-path-matchers.limit-structures`	`glob:*/limit_structures*.csv`	Pattern to match limit structure files
`limits.initial-load.file-path-matchers.incidents`	`glob:/incident//incident.csv`	Pattern to match incident files
`limits.initial-load.file-path-matchers.as-of-date`	`glob:*/as_of_date*.csv`	Pattern to match AsOfDate files

To customize these patterns, set these properties accordingly in your application.yml file. If you only wish to customize the pattern for some of the file types, you can set only those specific properties, and the others will use the default values.

How to choose the initial as-of-date

The as-of-date file is optional. Atoti Limits can take the initial as-of-date from an ILimitsInitialAsOfDateProvider bean instead. The limits.as-of-date.initial-provider property selects the default provider bean. It also decides whether Atoti Limits reads the as-of-date file matching limits.initial-load.file-path-matchers.as-of-date:

file (default): reads the as-of-date file. At startup, if there is no as_of_date.csv file in limits.initial-load.root-dir, writes one with the current date in the default time zone of the server. Later restarts read this file.
utc-time: ignores the as-of-date file and starts with the current UTC date at each startup. Writes no file.
none: ignores the as-of-date file and provides no default bean. The as-of-date must come from a declared ILimitsInitialAsOfDateProvider bean. Without a declared bean and without an as-of-date already set, the data load fails with a LimitsRuntimeException. The Atoti Limits server stays up: only the data load of the connected server fails.

In every mode, when the as-of-date file is not read or contains no date, an as-of-date already set, for example through the /asOfDate/set REST endpoint, is kept. Otherwise, the provider bean, if any, gives the initial as-of-date.

The as-of-date does not advance on its own. If the Atoti Limits server stays up overnight or longer, the as-of-date keeps its old value until Atoti Limits reconnects to the Atoti Server instance or a new date is set.The utc-time value only sets the as-of-date when Atoti Limits connects to a server and loads its data. With file, a restart does not change the as-of-date either, because Atoti Limits reads the generated file.To move to a new as-of-date, do one of the following:

Reconnect Atoti Limits to the Atoti Server instance (with utc-time, this loads the current UTC date).
Call the /asOfDate/set REST endpoint.
With file, provide a new as-of-date file.

To choose another date, declare an ILimitsInitialAsOfDateProvider bean. A declared bean replaces the default one, whatever the value of limits.as-of-date.initial-provider. With a declared bean, Atoti Limits ignores the as-of-date file, so an existing as-of-date file does not need to be deleted. The interface has a single method, which must return a date. If it returns null, the data load fails with a LimitsRuntimeException.

@FunctionalInterface
public interface ILimitsInitialAsOfDateProvider {

  LocalDate getInitialAsOfDate();
}

For example, the following bean starts Atoti Limits with the previous calendar day in UTC:

@Bean
public ILimitsInitialAsOfDateProvider limitsInitialAsOfDateProvider() {
  return () -> LocalDate.now(ZoneOffset.UTC).minusDays(1);
}

How to customize the file loading service

If you need to customize the initial load process beyond what is possible with the configuration properties, you can implement your own file loading service by creating a custom ILimitsFileLoadingService implementation to replace the default DefaultLimitsFileLoadingService.

ILimitsFileLoadingService

/**
 * <b>ILimitsFileLoadingService</b>
 *
 * <p>This interface defines the methods for loading limits data from CSV files.
 *
 * <p>Implementations of this interface should ensure that the data is loaded in the correct order
 * as described in the {@link #loadLimitsData(String)} method.
 *
 * @author ActiveViam
 */
@FunctionalInterface
public interface ILimitsFileLoadingService {

  /**
   * Loads all limits data, including asOfDate, limit structures, limits, and incidents. This method
   * is intended to be called when Limits first connects to a server. Besides the AsOfDate, the
   * ordering of the calls is important to ensure that the data is loaded correctly:
   *
   * <ol>
   *   <li>1. Load the AsOfDate (this could be done at any point in this sequence)
   *   <li>2. Load the limit structures before the limits because creating limits requires
   *       information from the structures.
   *   <li>3. Load the limits before the incidents because creating incidents requires information
   *       from the limits
   *   <li>4. Load the incidents last because they are dependent on the limits.
   * </ol>
   *
   * @param serverName the name of the server for which to load the limits data
   */
  void loadLimitsData(String serverName);
}

Once you have created your custom implementation, expose it as a Spring bean in your project and Spring will use it instead of the default implementation.

​Initial load configuration properties

​How to customize the initial load data location

​How to customize the initial load data file patterns

​How to choose the initial as-of-date

​How to customize the file loading service

Initial load configuration properties

How to customize the initial load data location

How to customize the initial load data file patterns

How to choose the initial as-of-date

How to customize the file loading service