Incremental Refresh

Incremental refresh is called through the Application API. It is important to note that all table names and field names are those from the DirectQuery local schema.

final TableUpdateDetail tableUpdateDetail =
    TableUpdateDetail.create(
        "SALES",
        ChangeType.ADD_ROWS,
        ConditionFactory.equal("DATE", LocalDate.of(2022, 4, 16)));
app.refresh(ChangeDescription.create(List.of(tableUpdateDetail)));

Here some change examples with their change description.

Multi table add rows

In this case, we have added some new sales for the seller google.
We have also added a new product. sales has a many-to-one relationship with product. Table sales changes

date	product	seller	quantity
2024-01-01	P1	microsoft	20
2024-01-01	P2	amazon	30
2024-01-01	P2	amazon	20
2024-01-01	P1	google	20
2024-01-01	P2	google	30
2024-01-01	P3	google	20

Table products changes

date	product	quantity
2024-01-01	P1	10
2024-01-01	P2	20
2024-01-01	P3	30.0

Change description

Table Update Detail	sales	products
Change type	ADD_ROWS	ADD_ROWS
Change Scope	sales.seller = ‘google’	product.product = ‘P3’

final List<TableUpdateDetail> updateDetails =
    List.of(
        TableUpdateDetail.create(
            "SALES", ChangeType.ADD_ROWS, ConditionFactory.equal("SELLER", "google")),
        TableUpdateDetail.create(
            "PRODUCTS", ChangeType.ADD_ROWS, ConditionFactory.equal("PRODUCT", "P3")));
app.refresh(ChangeDescription.create(updateDetails));

Multi table add rows using an unknown condition

In this case, we have added new rows in sales table.
All these new sales has a new seller which has been added in users table. sales has a many-to-one relationship with users. Table sales changes

date	product	seller	quantity
2024-01-01	P1	microsoft	20
2024-01-01	P2	amazon	30
2024-01-01	P2	amazon	20
2024-01-02	P1	google	20
2024-01-02	P2	alibaba	30
2024-01-02	P3	ibm	20

Table users changes

id	warehouse
microsoft	nyo-1
amazon	chi-1
google	tor-1
alibaba	bei-1
ibm	par-1

This change can be expressed with this description which could be cumbersome to build because of the IN condition. Change description

Table Update Detail	sales	users
Change type	ADD_ROWS	ADD_ROWS
Change Scope	sales.date = ‘2024-01-02’	users.id in (‘google’, ‘alibaba’,‘ibm’)

Here we managed to resolve a cumbersome condition looking at the data, but sometimes it is not possible to come up with a convenient condition.
In this case you can use the unknown condition. Change description with an unknown condition

Table Update Detail	sales	users
Change type	ADD_ROWS	ADD_ROWS
Change Scope	sales.date = ‘2024-01-02’	Unknown condition

This solution could have adverse consequences.
If you have defined a factless hierarchy on user table, the missing link with the fact table forbid us to use the sales table scope.
Connector will full refresh on this hierarchy stream.

final List<TableUpdateDetail> updateDetails =
    List.of(
        TableUpdateDetail.create(
            "SALES",
            ChangeType.ADD_ROWS,
            ConditionFactory.equal("DATE", LocalDate.of(2024, 1, 2))),
        TableUpdateDetail.create("PRODUCTS", ChangeType.ADD_ROWS, ConditionFactory.unknown()));
app.refresh(ChangeDescription.create(updateDetails));

​Multi table add rows

​Multi table add rows using an unknown condition

Multi table add rows

Multi table add rows using an unknown condition