CSV Sources

This section describes how to configure the different CSV sources for the Local File System or one of the supported Cloud Sources: AWS, Azure, Google Cloud Platform on a Fetching or Listening data source.

CSV Source Channel Configuration

A Channel defines how to move data from a Source to a Datastore.

/**
 * This method will create and register a CSV Channel with the DataLoadController.
 * 
 * @param dataLoadController The DataLoadController to register a CSV Channel with.
 * @param csvSource The CSV Source the Channel is to execute on.
 */
public void channelConfiguration(IDataLoadController dataLoadController, ICSVSource<I> csvSource){
	/*
	Create and register a channel
 	*/

	final var channelFactory = new DlcCSVMessageChannelFactory<I>(csvSource, datastore);

	var storeName = "Products_Store";
	var topicName = "Products Topic";

	var channel = channelFactory.createChannel(topicName, storeName);
	var targetStores = List.of(storeName);

	// Anonymous implementation of a full remove where condition
	var removeWhereConditionConverter = new IDataLoadController.IScopeToRemoveWhereConditionConverter() {
		@Override
		public ICondition apply(final String store, final Map<String, Object> scope) {
			// Remove everything from the store:
			return BaseConditions.TRUE;
		}
	};

	// Register the channel
	dataLoadController.registerChannel(channel, targetStores, removeWhereConditionConverter);
}