CSV Sources
This section describes how to configure the different CSV sources for the Local File System or one of the supported Cloud Sources: AWS, Azure, Google Cloud Platform on a Fetching or Listening data source.
CSV Source Channel Configuration
A Channel defines how to move data from a Source to a Datastore.
/**
* This method will create and register a CSV Channel with the DataLoadController.
*
* @param dataLoadController The DataLoadController to register a CSV Channel with.
* @param csvSource The CSV Source the Channel is to execute on.
*/
public void channelConfiguration(IDataLoadController dataLoadController, ICSVSource<I> csvSource){
/*
Create and register a channel
*/
final DlcCSVMessageChannelFactory<I> channelFactory = new DlcCSVMessageChannelFactory<I>(csvSource, datastore);
String storeName = "Products_Store";
String topicName = "Products Topic";
var channel = channelFactory.createChannel(topicName, storeName);
var targetStores = List.of(storeName);
// Anonymous implementation of a full remove where condition
var removeWhereConditionConverter = new IDataLoadController.IScopeToRemoveWhereConditionConverter() {
@Override
public ICondition apply(final String store, final Map<String, Object> scope) {
// Remove everything from the store:
return BaseConditions.TRUE;
}
};
// Register the channel
dataLoadController.registerChannel(channel, targetStores, removeWhereConditionConverter);
}