Skip to main content

Datastore Configuration

The datastore consists of a collection of stores that can be linked together through references. This page introduces the concepts you need to know when working with the datastore and walks you through setting up a basic datastore from a datastore description.

Stores

A store is a collection of records that share the same attributes, called the fields (or columns) of the store.

In a standard database, a store is similar to a table, and records similar to rows.

Store Description

All stores are built according to a store description, represented by the interface IStoreDescription.

When building a reference description, we recommend using the builder interface available through StartBuilding.store(). The builder provides a fluent API to specify the field structure of a store.

The following snippet defines the description of a Product store including information about products sold by a retailer:

IStoreDescription productStoreDescription = StartBuilding.store()
.withStoreName("Product")
.withField("id", ILiteralType.LONG)
.asKeyField()
.withField("category", ILiteralType.STRING, "uncategorized")
.withNullableField("name", ILiteralType.STRING)
.withVectorField("priceHistory", ILiteralType.DOUBLE)
.build();

Field Specification

Store fields are specified by:

  • a name
  • a data type, from the values contained in the ILiteralType interface

Both scalar types and vector types are supported as field types.

.withField("id", ILiteralType.LONG)
.withVectorField("priceHistory", ILiteralType.DOUBLE)

Default Values

A default value may be specified during field declaration. Otherwise, a default value is selected based on the data type: 0 for numerical types and the string "N/A" for most object types.

A field may also be declared nullable, meaning that no default value is used for the field, accepting null as a value instead.

.withField("category", ILiteralType.STRING, "uncategorized")
.withNullableField("name", ILiteralType.STRING)

Key Fields

Any number of fields in a store can be flagged as key fields through:

.withField("id", ILiteralType.LONG)
.asKeyField()

The set of key fields in a store should uniquely identify each record in the store. A key field can't be made nullable: nullability flags on key fields are ignored when creating the datastore.

You can build a keyless store using .withoutKey() after all fields have been specified, although such stores are restricted regarding possible operations, due to the lack of key.

References

The different stores within a datastore can be linked together through references.

A reference is a mapping from one store (the owner store) to another store (the target store), associating one or more fields of the former to fields of the latter. Each reference defines a many-to-one relationship between two stores. They are useful at query time to relate the information from different stores together.

A datastore generally has a multitude of references, forming a directed graph of stores. The store graph cannot contain any cycle: the schema of stores can also be referred to as a star schema, as it has a tree-like structure.

A store can reference another one multiple times, as long as the referenced fields are different for the multiple references.

Constraints

When a reference is created between two stores, a uniqueness constraint is added to the group of fields used by the reference in the target store. As such, a line of the base store cannot point to multiple lines of the target store. This is a consequence of the many-to-one relationship represented by the reference.

For example, given two stores A and B:

  • Store A

    TradeIdValue
    Trade110
    Trade220
  • Store B

    TradeIdLegTypeBook
    Trade1N/ABook1
    Trade2N/ABook2
    Trade3PayBook3
    Trade3RecvBook3

If we define a reference from Store A's TradeId field to Store B's TradeId field, a KeyConstraintViolationException is raised when Store B is loaded, because multiple lines in Store B share the same TradeId.

Reference Description

Similarly to stores, datastore references are built according to a reference description, represented by the interface IReferenceDescription.

When building a reference description, we recommend using the fluent builder interface available through StartBuilding.reference().

A fully defined reference description is created like so:

IReferenceDescription referenceDescription = StartBuilding.reference()
.fromStore("store A")
.toStore("targetStore B")
.withName("AToB")
.withMapping("field from A", "field from B")
.withMapping("another field from A", " another field from B")
.build();

Example

In this example, the required datastore consists of two stores:

  • Sale: records correspond to a product sale.
  • PriceHistory: records define the product prices at certain dates.

IStoreDescription saleStoreDescription = StartBuilding.store()
.withStoreName("Sale")
.withField("id", ILiteralType.LONG)
.asKeyField()
.withField("productId", ILiteralType.LONG)
.asKeyField()
.withField("date", ILiteralType.DATE)
.asKeyField()
.withField("quantity", ILiteralType.INT)
.withField("unitPrice", ILiteralType.DOUBLE)
.build();
IStoreDescription priceHistoryStoreDescription = StartBuilding.store()
.withStoreName("Product")
.withField("id", ILiteralType.LONG)
.asKeyField()
.withField("name", ILiteralType.STRING)
.withField("category", ILiteralType.STRING, "uncategorized")
.build();

The following reference associates the product information from the Product store to the sales of the Sale store, using the product ID as mapping.

IReferenceDescription referenceDescription = StartBuilding.reference()
.fromStore("Sale")
.toStore("Product")
.withName("SaleToProduct")
.withMapping("productId", "id")
.build();

Datastore Schema

A datastore is built according to a datastore schema, which gathers the store descriptions and the reference descriptions that compose the datastore.

When building a datastore schema, we recommend using the fluent builder interface available through StartBuilding.datastoreSchema():

IDatastoreSchemaDescription datastoreSchemaDescription = StartBuilding.datastoreSchema()
.withStore(saleStoreDescription)
.withStore(priceHistoryStoreDescription)
.withReference(referenceDescription)
.build();

The .withStore() and .withReference() methods can be chained as many times as needed.

Creating a Datastore

Once the datastore schema has been created, the datastore can be built using StartBuilding.application():

var application = StartBuilding.application()
.withDatastore(datastoreSchemaDescription)
.withoutBranchRestrictions()
.build();
IDatastore datastore = application.getDatastore();

The IDatastore interface implements AutoCloseable, and can be torn down using the close() method.