Skip to main content

Datastore Configuration

The datastore consists of a collection of stores that can be linked together through references. This page introduces the concepts you need to know when working with the datastore and walks you through setting up a basic datastore from a datastore description.

Stores

A store is a collection of records that share the same attributes, called the fields (or columns) of the store.

In a standard database, a store is similar to a table, and records similar to rows.

Store Description

All stores are built according to a store description, represented by the interface IStoreDescription.

When building a reference description, we recommend using the builder interface available through StartBuilding.store(). The builder provides a fluent API to specify the field structure of a store.

The following snippet defines the description of a Product store including information about products sold by a retailer:

IStoreDescription productStoreDescription = StartBuilding.store()
        .withStoreName("Product")
        .withField("id", ILiteralType.LONG).asKeyField()
        .withField("category", ILiteralType.STRING, "uncategorized")
        .withNullableField("name", ILiteralType.STRING)
        .withVectorField("priceHistory", ILiteralType.DOUBLE)
        .build();

Field Specification

Store fields are specified by:

  • a name
  • a data type, from the values contained in the ILiteralType interface

Both scalar types and vector types are supported as field types.

.withField("id", ILiteralType.LONG)
.withVectorField("priceHistory", ILiteralType.DOUBLE)

Default Values

A default value may be specified during field declaration. Otherwise, a default value is selected based on the data type: 0 for numerical types and the string "N/A" for most object types.

A field may also be declared nullable, meaning that no default value is used for the field, accepting null as a value instead.

.withField("category", ILiteralType.STRING, "uncategorized")
.withNullableField("name", ILiteralType.STRING)

Key Fields

Any number of fields in a store can be flagged as key fields through:

.withField("id", ILiteralType.LONG)
.asKeyField()

The set of key fields in a store should uniquely identify each record in the store. A key field can't be made nullable: nullability flags on key fields are ignored when creating the datastore.

You can build a keyless store using .withoutKey() after all fields have been specified, although such stores are restricted regarding possible operations, due to the lack of key.

References

The different stores within a datastore can be linked together through references.

A reference is a mapping from one store (the owner store) to another store (the target store), associating one or more fields of the former to fields of the latter. Each reference defines a many-to-one relationship between two stores. They are useful at query time to relate the information from different stores together.

A datastore generally has a multitude of references, forming a directed graph of stores. The store graph cannot contain any cycle: the schema of stores can also be referred to as a star schema, as it has a tree-like structure.

A store can reference another one multiple times, as long as the referenced fields are different for the multiple references.

Constraints

When a reference is created between two stores, a uniqueness constraint is added to the group of fields used by the reference in the target store. As such, a line of the base store cannot point to multiple lines of the target store. This is a consequence of the many-to-one relationship represented by the reference.

For example, given two stores A and B:

  • Store A

    TradeIdValue
    Trade110
    Trade220

  • Store B

    TradeIdLegTypeBook
    Trade1N/ABook1
    Trade2N/ABook2
    Trade3PayBook3
    Trade3RecvBook3

If we define a reference from Store A's TradeId field to Store B's TradeId field, a KeyConstraintViolationException is raised when Store B is loaded, because multiple lines in Store B share the same TradeId.

Reference Description

Similarly to stores, datastore references are built according to a reference description, represented by the interface IReferenceDescription.

When building a reference description, we recommend using the fluent builder interface available through StartBuilding.reference().

A fully defined reference description is created like so:

IReferenceDescription referenceDescription = StartBuilding.reference()
        .fromStore("store A")
        .toStore("targetStore B")
        .withName("AToB")
        .withMapping("field from A", "field from B")
        .withMapping("another field from A", " another field from B")
        .build();

Example

In this example, the required datastore consists of two stores:

  • Sale: records correspond to a product sale.
  • PriceHistory: records define the product prices at certain dates.

IStoreDescription saleStoreDescription = StartBuilding.store()
        .withStoreName("Sale")
        .withField("id", ILiteralType.LONG).asKeyField()
        .withField("productId", ILiteralType.INT).asKeyField()
        .withField("date", ILiteralType.DATE).asKeyField()
        .withField("quantity", ILiteralType.INT)
        .withField("unitPrice", ILiteralType.DOUBLE)
        .build();
IStoreDescription priceHistoryStoreDescription = StartBuilding.store()
        .withStoreName("Product")
        .withField("id", ILiteralType.LONG).asKeyField()
        .withField("name", ILiteralType.STRING)
        .withField("category", ILiteralType.STRING, "uncategorized")
        .build();

The following reference associates the product information from the Product store to the sales of the Sale store, using the product ID as mapping.

IReferenceDescription referenceDescription = StartBuilding.reference()
        .fromStore("Sale")
        .toStore("Product")
        .withName("SaleToProduct")
        .withMapping("productId", "id")
        .build();

Datastore Schema

A datastore is built according to a datastore schema, which gathers the store descriptions and the reference descriptions that compose the datastore.

When building a datastore schema, we recommend using the fluent builder interface available through StartBuilding.datastoreSchema():

IDatastoreSchemaDescription datastoreSchemaDescription = StartBuilding.datastoreSchema()
        .withStore(saleStoreDescription)
        .withStore(priceHistoryStoreDescription)
        .withReference(referenceDescription)
        .build();

The .withStore() and .withReference() methods can be chained as many times as needed.

Creating a Datastore

Once the datastore schema has been created, the datastore can be built using StartBuilding.datastore():

IDatastore datastore = StartBuilding.datastore()
        .setSchemaDescription(datastoreSchemaDescription)
        .build();

The IDatastore interface implements AutoCloseable, and can be torn down using the close() method.