Package com.activeviam.risk.ref.cfg.impl

Class ACSVSourceConfig

    • Field Detail

      • LOGGER

        protected static Logger LOGGER

      • env

        @Autowired
protected org.springframework.core.env.Environment env

      • datastoreConfig

        @Autowired
protected com.qfs.server.cfg.IDatastoreConfig datastoreConfig

      • folder

        @Autowired
@Qualifier("folder")
protected String folder

    • Constructor Detail

      • ACSVSourceConfig

        public ACSVSourceConfig()

    • Method Detail

      • getTopicToStoreAndFilePatternMap

        protected abstract Map<String,​com.quartetfs.fwk.IPair<String,​String>> getTopicToStoreAndFilePatternMap()

      • csvSource

        @Bean
public abstract com.activeviam.tools.dataloadcontroller.core.source.csv.CsvScopedFetchSource<Path> csvSource()

      • defineTopic

        public void defineTopic​(com.qfs.msg.csv.impl.CSVSource<Path> source,
                        String filePath,
                        String topicName)

      • createTopicConfig

        protected <I> SimpleSourceConfig.ITopicConfig<com.qfs.msg.csv.IFileInfo<Path>> createTopicConfig​(String topic,
                                                                                                 String store,
                                                                                                 com.qfs.source.ITuplePublisher<com.qfs.msg.csv.IFileInfo<Path>> tuplePublisher)
        Create an SimpleSourceConfig.ITopicConfig object.
        Type Parameters:
        I - type of identifier of the files
        Parameters:
        topic - the topic name
        store - the topic's store name.
        tuplePublisher - The channel's tuple puslisher
        Returns:
        the topic config

      • getTuplePublisher

        protected com.qfs.source.ITuplePublisher<com.qfs.msg.csv.IFileInfo<Path>> getTuplePublisher​(String topic)
        This function is intended to be overridden in order to specify some specific tuple publishers
        Parameters:
        topic - The current topic
        Returns:
        A tuple publisher if any or null

      • createCsvSource

        protected com.activeviam.tools.dataloadcontroller.core.source.csv.CsvScopedFetchSource<Path> createCsvSource​(String sourceName)
        Create a CSV source object suitable for use with the IDataLoadController.
        Parameters:
        sourceName - name of the CSV source.
        Returns:
        the CSV source

      • csvChannelFactory

        @Bean
protected com.qfs.source.impl.CSVMessageChannelFactory<Path> csvChannelFactory()
        This function must be overridden in order to create a specific bean
        Returns:
        the CSVMessageChannelFactory

      • csvSourceConfiguration

        @Bean
public Properties csvSourceConfiguration()
        The CSVSource properties
        Returns:
        properties

      • csvSource

        public com.activeviam.tools.dataloadcontroller.core.source.csv.CsvScopedFetchSource<Path> csvSource​(String sourceName)
        Generates a specific source with topics
        Parameters:
        sourceName - name of the source
        Returns:
        the fetch source

      • verifyFolder

        public void verifyFolder​(String dataSet)
        Verify that a folder is valid
        Parameters:
        dataSet - folder

      • getLinesToSkip

        protected int getLinesToSkip​(String topic)

      • getFileColumns

        protected List<String> getFileColumns​(String topicName,
                                      Map<String,​com.quartetfs.fwk.IPair<String,​String>> topicToStoreAndFilePatternMap,
                                      com.qfs.store.IDatastoreSchemaMetadata schemaMetadata)
        Retrieves the list of file columns for a given store, either from a custom list where configured, or from the store configuration.
        Parameters:
        topicName - The name of the destination store.
        topicToStoreAndFilePatternMap - resouce used to find-out the store name from a specific topic
        schemaMetadata - The schema metadata.
        Returns:
        The list of columns in the source file.