Skip to main content

External data validation

DirectQuery is making various assumptions about how the data stored is in the database and how the database is configured. If these assumptions are invalid then the result of the queries might also be invalid, so these requirements must be respected.

DirectQuery provide ways to run the validations on the external data. They are not run by default as they can consume a lot of resources, but it is possible to run these validations manually.

Table validation​

Unique key fields​

It is possible to define key fields on the table in the data model. DirectQuery assumes that each key is unique. This is necessary for instance when the key fields are used as join fields to ensure that the join is a many-to-one.

final ITableConstraintsValidator validator = session.getDataValidator().forTable(table);
final ITableValidatorErrorReportWithRowInfo primaryKeyErrorReport =
    validator.checkDuplicatedKeys();
final boolean isValid = primaryKeyErrorReport.isValid();

Multi columns vectors​

When storing a vector with one value per column, it is necessary that each primitive value to be non null.

Multi row vector table validation​

When storing vector with one index and one value per row, it it necessary to follow these constraints:

  • Indexes must start at 0 or 1
  • Indexes must start with the same index
  • No index must be missing (between first and last)
  • Values for primitive types can not be null

final IMultiRowVectorTableValidator validator =
    session.getDataValidator().forMultiRowVectorTable(table);
final ITableValidatorErrorReportWithRowInfo primaryKeyErrorReport =
    validator.checkDuplicatedKeys();
final ITableValidatorErrorReportWithRowInfo missingIndexErrorReport =
    validator.checkMissingIndex();
final ITableValidatorErrorReportWithRowInfo nullValuesErrorReport = validator.checkNullValues();
final ITableValidatorErrorReportForStartIndex startIndexReport =
    validator.checkIdenticalStartIndex();
final boolean isValid =
    primaryKeyErrorReport.isValid()
        && missingIndexErrorReport.isValid()
        && nullValuesErrorReport.isValid()
        && startIndexReport.isValid();

See vectors documentation for more details about this type of vector.