Package com.activeviam.io.data.source
-
Interface Summary Interface Description IAvroFileSource Contract for a source that can load data from Avro files in ActivePivot.IAvroFileSource.IChannelFetchFuture Contract for aFuture
that represents the pending completion of an asynchronous fetch operation for a givenscope
that wassubmitted
to anAvro file source
.IAvroFileSource.IChannelFetchScope Contract for a set of parameters that defines a fetch operation to be executed by anAvro file source
for a given data scope in a given Avrochannel
.IAvroFilesScopedFetchTopic Contract for a topic of a source that can fetch Avro files to load data in ActivePivot.IDataSource<I,E> Contract for a source that can fetch or listen to data for a specific scope that is specified at runtime for each fetch or listen operation.IDlcTopic Topic which can contain extra properties that can be used by all DLC topics.IFetchingDataSource<I,E> Contract for a source that can fetch data for a specific scope that is specified at runtime for each fetch iteration.IFilesScopedFetchTopic IFilesScopedTopic Contract for a topic of a source that can fetch files and listen to directories to load data in ActivePivot.IFilesScopedTopic.IFileScanParameters Contract for file scan parametersIFilesScopedTopic.IScopeToFileScanParametersConverter Contract for a converter of a fetch scope into file scanparameters
IFileTask<I,E> Contract for a task that a files source may execute to load data in ActivePivot.IFileTask.IResult Contract for a result produced by aIFileTask
.IJdbcScopedFetchSource Contract for a source that can load data from a Database into ActivePivot, based on SQL statements (defined via source topics) and fetch business scopes, which are defined at fetch time to derive the SQL input parameters required for injection into those statements.IJdbcScopedFetchTopic Contract for a topic of a source that can fetch a DB to load data in ActivePivot.IJdbcScopedFetchTopic.IScopeToSqlParametersConverter Contract for a converter of a business scope (for a particular statement task execution) into the relevant SQL input parameters (given the JDBC statement topic that this tasks relates to).IJdbcStatementTask<I,E> Contract for a task that a JDBC source may execute to process a given Statement SQL statement to retrieve data from a Database and load them in ActivePivot.IJdbcStatementTask.IResult Contract for a result produced by aJDBC statement task
.IListeningDataSource<I,E> Contract for a source that can consume data from a messaging source like Apache Kafka, RabbitMQ, and can stay connected and "listen" for real-time data.IParquetTopic<I> Contract for a topic of the Parquet Source.ITupleSource Contract for asource
that producesObject[]
tuples. -
Class Summary Class Description AAvroFileTask<E> Base (abstract) implementation ofIFileTask
.AAvroFileTask.Result Base implementation forIFileTask.IResult
.ADatastoreTransactionTask An abstract implementation of a task that executes a Datastore transaction with an unspecified 'content'.AExecutionTask Abstract execution task.AJdbcPreparedStatementTask<E> Base implementation of aIJdbcStatementTask
that executes aprepared statement
.AJdbcPreparedStatementTask.Result Default implementation ofIJdbcStatementTask.IResult
.AParquetFileTask<E> Base (abstract) implementation ofIFileTask
.AParquetFileTask.Result Base implementation forIFileTask.IResult
.ArrayAsTroveListGenericDatumReader AEnhancedGenericDatumReader
that usesTIntArrayList
,TLongArrayList
,TFloatArrayList
andTDoubleArrayList
Java types to materialize values when reading schema type array for element type int, long, float and double (respectively).AvroCompactArrayUtils Collection of utility methods to handle 'SparseVector' in ActivePivot based on Avro 'CompactArray' records.AvroFilesScopedFetchSource Default implementation ofIAvroFileSource
.AvroFilesScopedFetchSource.ChannelFetchFuture Standard implementation ofIAvroFileSource.IChannelFetchFuture
.AvroFilesScopedFetchSource.ChannelFetchScope Standard implementation ofIAvroFileSource.IChannelFetchScope
.AvroFilesScopedFetchSource.LogMessages AvroFilesScopedFetchTopic Default implementation of anIAvroFilesScopedFetchTopic
.AvroFileTask Generic blank implementation of an Avro TaskAzureCsvScopedFetchSource<F> Fetches from an Azure CSV Blob file.AzureCsvScopedListenSource Listens to an Azure CSV Blob file.CustomKafkaAvroDeserializerGenericRecord Customize the deserialize method to reuse GenericDatumReader.CustomKafkaAvroDeserializerObjectTuple Customize the deserialize method to reuse GenericDatumReader and GenericObject.DlcParquetHealthEventHandler EnhancedGenericData Enhanced implementation of Avro coreGenericData
specifically for usage withGenericDatumReader
.EnhancedGenericData.Record Enhanced generic record implementation.EnhancedGenericDatumReader Extend Avro core implementation of generic datum reader with leverage ofEnhancedGenericData
(by default), and optional collection/logging of detailed statistics (read objects count) about reader activity.FilesScopedFetchTopic Default implementation of aIFilesScopedFetchTopic
.FilesScopedFetchTopic.FetchScopeToFileScanParametersConverter Default implementation ofIFilesScopedTopic.IScopeToFileScanParametersConverter
.FilesScopedFetchTopic.FileScanParameters POJO default implementation ofIFilesScopedTopic.IFileScanParameters
which contains a relative path and whether or not to recursively scan the directory.GcpCsvScopedFetchSource<F> Fetches from a S3 CSV file.GcpCsvScopedListenSource<F> Listens to a Google Cloud Storage CSV bucket.JdbcPreparedStatementTask Standard implementation of aIJdbcStatementTask
that executes aprepared statement
.JdbcScopedFetchSource Default implementation of aIJdbcScopedFetchSource
.JdbcStatementTopic Base implementation of aIJdbcScopedFetchTopic
.JdbcStatementTopic.ScopeToSqlParametersConverter Default implementation forIJdbcScopedFetchTopic.IScopeToSqlParametersConverter
.KafkaJsonDeserializer Deserialize bytes from kafka to aJsonNode
and form an Object[] tuple from the JsonNode.LoadDataTxControllerTask AIDataLoadController.IExecutionTask
that loads data related to one or multiple topics and a particular business scope, within the boundaries of a single Datastore transaction.MutableArrayElementGenericDatumReader AEnhancedGenericDatumReader
that usesMutableInt
,MutableLong
,MutableFloat
,MutableDouble
andMutableBoolean
Java types to materialize values when reading schema type int, long, float, double and boolean (respectively) for array elements.ParquetFileListener<I> Listen for created and / or modified parquet files, parsing them for submitting the records to datastore.ParquetFileTask Generic blank implementation of an Parquet Task for Topic implementationParquetSource<I> Parquet implementation of ITupleSource and IFetchingDataSourceParquetSource.ChannelFetchFuture Contract for aFuture
that represents the pending completion of an asynchronous fetch operation for a givenscope
ParquetSource.ChannelFetchScope Standard implementation ofParquetSource.ChannelFetchScope
.ParquetTopic<I> Contract for a topic of the Parquet Source.RabbitMqMessageConsumer Consume messages from a RabbitMQ queue.ReadTupleFieldColumnCalculator Column calculator that extracts values for a particular column in the given (source)Object[]
tuple, based on the column index that has been configured at construction time.S3CsvScopedFetchSource<F> Fetches from a S3 CSV file.S3CsvScopedListenSource Listens to a S3 CSV bucket.SparseVectorUtils Collection of utility methods to handle 'SparseVector' storage in ActivePivot.StartListeningControllerTask AIDataLoadController.IExecutionTask
that starts consumers on theMessagingTopic
/s specified.StopListeningControllerTask AIDataLoadController.IExecutionTask
that stops the consumers/listeners for givenMessagingTopic
.UnloadDataTxControllerTask AIDataLoadController.IExecutionTask
that unloads data related to one or multiple topics and a particular business scope, within the boundaries of a single Datastore transaction. -
Enum Summary Enum Description AAvroFileTask.AvroReaderVersion Supported versions of Avro reader implementation.SparseVectorUtils.SparseVectorRepresentation Possible representation type for a'SparseVector'
in primitive form.