Dates To Include filter configuration

Dates to Include Filter

When using DirectQuery you may want to only include a subset of data available on the remote database. You can configure this through a custom bean.

The directQueryDatesToIncludeFilter bean is used to tell Atoti FRTB which dates to include (or exclude) from the DirectQuery data node cube. This bean can be entirely customized and creates a condition on which dates to include in the DirectQuery data node.

Default Implementation

The default implementation of the directQueryDatesToIncludeFilter bean will exclude all dates that were provided to the initial-load.business-dates property. This is so that when running with an InMemory data node, the DirectQuery and InMemory node do not contain any of the same AsOfDates.

The default bean is defined in InitialDataDirectQueryConfig as:

	@Bean
	@Qualifier("directQueryDatesToIncludeFilter")
	public ICondition directQueryDatesToIncludeFilter(@Qualifier("initialLoadAsOfDates") List<LocalDate> initialLoadAsOfDates){
		// If no initialLoadAsOfDates are specified, then there is no date filter to apply:
		if(initialLoadAsOfDates == null || initialLoadAsOfDates.isEmpty()){
			return null;
		}

		// Restrict DirectQuery node to NOT include any of the initialLoadAsOfDates dates:
		List<SubCondition> subConditions = new ArrayList<>();

		for(var initialLoadDate : initialLoadAsOfDates){
			// Condition: AsOfDate field != initialLoadDate
			subConditions.add(
					new SubCondition(AS_OF_DATE, new NotCondition(new EqualCondition(initialLoadDate)))
			);
		}
		return new AndCondition(subConditions);
	}

Custom Implementations

To create a custom implementation you must create your own @Primary @Bean with a @Qualifier of directQueryDatesToIncludeFilter that returns an ICondition. This condition can be thought as what to include in the DirectQuery data node.

Each condition included must be a SubCondition where the AsOfDate is used as the matching field.

Examples

Here are some simple examples of how to configure this bean.

Include All Available Data in DirectQuery

If we wanted to include all data available on our remote database, we simply return null and there will be no condition on what data can be loaded into the DirectQuery data node:

	@Bean
	@Primary
	@Qualifier("directQueryDatesToIncludeFilter")
	public ICondition customFilterCondition(){
	    retun null;
	}

Only Include Specific Dates

	@Bean
	@Primary
	@Qualifier("directQueryDatesToIncludeFilter")
	public ICondition customFilterCondition(){
		// Restrict DirectQuery node to INCLUDE the specific dates:
		List<SubCondition> excludeConditions = new ArrayList<>();

		excludeConditions.add(new SubCondition(AS_OF_DATE, new EqualCondition(LocalDate.of(2020, 1, 1))));
		excludeConditions.add(new SubCondition(AS_OF_DATE, new EqualCondition(LocalDate.of(2020, 1, 2))));
		excludeConditions.add(new SubCondition(AS_OF_DATE, new EqualCondition(LocalDate.of(2020, 1, 3))));

		return new OrCondition(excludeConditions);
	}