☰
Navigation :

Dates To Include filter configuration

Dates to Include Filter

When using DirectQuery, you may want to only include a subset of data available on the remote database. You can configure this through a custom bean.

The SP_QUALIFIER__DIRECT_QUERY_DATES_INITIAL_LOAD_FILTER_CONDITION and SP_QUALIFIER__DIRECT_QUERY_DATES_ACTIVEPIVOT_INSTANCE_DESCRIPTION_FILTER_CONDITION beans are used to tell Atoti Market Risk which dates to include (or exclude) from the DirectQuery data node cube. Two beans are necessary because one bean is an instance of com.qfs.condition.ICondition and the other is an instance of com.quartetfs.fwk.filtering.ICondition. The filtering logic of both beans need to be identical. These beans can be entirely customized to create a condition on which dates to include in the DirectQuery data node.

Default Implementation

The default implementation of the SP_QUALIFIER__DIRECT_QUERY_DATES_INITIAL_LOAD_FILTER_CONDITION and SP_QUALIFIER__DIRECT_QUERY_DATES_ACTIVEPIVOT_INSTANCE_DESCRIPTION_FILTER_CONDITION beans excludes all dates that were provided to the mr.data-load.initial-business-dates property. This is so that when running with an InMemory data node, the DirectQuery and InMemory node do not contain any of the same AsOfDates.

The default beans are defined in InitialDataDirectQueryConfig as:

@Bean
@Qualifier(SP_QUALIFIER__DIRECT_QUERY_DATES_INITIAL_LOAD_FILTER_CONDITION)
public com.qfs.condition.ICondition filterAsOfDates(){

    if(initialLoadAsOfDates == null){
    return null;
    }

    return not(in(FieldPath.of(StoreFieldConstants.AS_OF_DATE), initialLoadAsOfDates));
}

@Bean
@Qualifier(SP_QUALIFIER__DIRECT_QUERY_DATES_ACTIVEPIVOT_INSTANCE_DESCRIPTION_FILTER_CONDITION)
public com.quartetfs.fwk.filtering.ICondition filterAsOfDatesFwk(){

    if(initialLoadAsOfDates == null){
    return null;
    }

    List<com.quartetfs.fwk.filtering.ICondition> conditions = initialLoadAsOfDates.stream()
    .map(date -> new SubCondition(
        "AsOfDate",
        new NotCondition(new EqualCondition(date))
        )
    ).collect(Collectors.toList());

    return new AndCondition(conditions);
}

Custom Implementations

To create a custom implementation, you must create your own @Primary @Bean with a @Qualifier of SP_QUALIFIER__DIRECT_QUERY_DATES_INITIAL_LOAD_FILTER_CONDITION and SP_QUALIFIER__DIRECT_QUERY_DATES_ACTIVEPIVOT_INSTANCE_DESCRIPTION_FILTER_CONDITION that returns an instance of com.qfs.condition.ICondition and an instance of com.quartetfs.fwk.filtering.ICondition respectively. This conditions can be thought of as what to include in the DirectQuery data node.

In the instance of com.quartetfs.fwk.filtering.ICondition, each condition included must be a SubCondition, where the AsOfDate is used as the matching field.

Examples

Here are some simple examples of how to configure this bean.

Include All Available Data in DirectQuery

If we wanted to include all data available on our remote database, we simply return null and there will be no condition on what data can be loaded into the DirectQuery data node:

    @Bean
    @Qualifier(SP_QUALIFIER__DIRECT_QUERY_DATES_INITIAL_LOAD_FILTER_CONDITION)
    public com.qfs.condition.ICondition filterAsOfDates(){
        return null;
    }

    @Bean
    @Qualifier(SP_QUALIFIER__DIRECT_QUERY_DATES_ACTIVEPIVOT_INSTANCE_DESCRIPTION_FILTER_CONDITION)
    public com.quartetfs.fwk.filtering.ICondition filterAsOfDatesFwk(){
        return null;
    }

Only Include Specific Dates

    @Bean
    @Primary
    @Qualifier(SP_QUALIFIER__DIRECT_QUERY_DATES_INITIAL_LOAD_FILTER_CONDITION)
    public com.qfs.condition.ICondition filterAsOfDates(){
        // Restrict DirectQuery node to INCLUDE the specific dates:
        List<LocalDate> asOfDatesToLoad = Arrays.asList(LocalDate.of(2020, 1, 1),
            LocalDate.of(2020, 1, 2),
            LocalDate.of(2020, 1, 3));

        return not(in(FieldPath.of(StoreFieldConstants.AS_OF_DATE), asOfDatesToLoad));
    }
        
	@Bean
	@Primary
    @Qualifier(SP_QUALIFIER__DIRECT_QUERY_DATES_ACTIVEPIVOT_INSTANCE_DESCRIPTION_FILTER_CONDITION)
	public com.quartetfs.fwk.filtering.ICondition customFilterCondition(){
		// Restrict DirectQuery node to INCLUDE the specific dates:
		List<SubCondition> includeConditions = new ArrayList<>();

        includeConditions.add(new SubCondition(AS_OF_DATE, new EqualCondition(LocalDate.of(2020, 1, 1))));
        includeConditions.add(new SubCondition(AS_OF_DATE, new EqualCondition(LocalDate.of(2020, 1, 2))));
        includeConditions.add(new SubCondition(AS_OF_DATE, new EqualCondition(LocalDate.of(2020, 1, 3))));

		return new OrCondition(includeConditions);
	}