Migration notes from 5.0 pre-releases

This page lists the changes between pre-releases and explains any changes required to migrate from these early releases to the stated version of Atoti Data Connectors:

Changelogs:

• Changes between 5.0.0-M2 and 5.0.0
• Changes between 5.0.0-M1 and M2

Migration notes:

• Migrate from 5.0.0-M2 to 5.0.0
• Migrate from 5.0.0-M1 to M2

Changes between 5.0.0-M2 and 5.0.0

Added

Issue Key	Details
BAST-1152	Added support for messaging source (Kafka)

Changed

Issue Key	Details
BAST-1239	Refactoring changes including renaming classes and reorganizing packages
BAST-1297	Externalized glob to regex logic used in cloud topic factories so it can be overridden if needed
BAST-1269	Improved how default configurations can be overridden
BAST-1282	Flattened module structure
BAST-1283	rollbackOnTransactionOnError property is now true by default

Fixed

Issue Key	Details
BAST-1272	Removed trailing slash in ScopeTemplateResolverService which was causing issues for cloud topics
BAST-1285	Fixed dlc.enabled not being true by default
BAST-1287	Fixed serialization and deserialization of all DTOs
BAST-1291	Fixed bug in exception handler causing stack trace to not be present
BAST-1298	Fixed bug where implicit unload topics were being created for topics which do not match a table name
BAST-1283	Fixed bug where rollbackOnTransactionOnError property was being ignored (and default behavior is now true)
BAST-1292	Fixed vulnerable dependencies being brought in

Migrate from 5.0.0-M2 to 5.0.0

Upgrading from version 5.0.0-M2, see Atoti Data Connectors 5.0 Release Notes.

Atoti Data Connectors uses Atoti Server 6.1.1. For new features and fixes included in that release, please see the release notes for Atoti Server.

Breaking Changes

None

Headline announcement

• Flattened Module Structure: To simplify imports, we have flattened the module structure.
• Renamed and reorganized packages: We’ve reoganized packages and renamed a few classes and methods to better align the properties and the description objects.
• Added support for Kafka Messaging Source: Support for streaming from a Kafka source has been added. See Messaging Configuration.

Flattened Module Structure

We’ve flattened the module structure to simplify imports. See Getting Started for the updated dependencies.

Default rollbackTransactionOnError behavior

The default behavior has changed to roll back the transaction on error. You can override this behavior by setting the dlc.rollback-transaction-on-error property to false.

Renamed & Reorganized Packages

Change Type	OLD Class Name	NEW Class Name
Renamed & moved	com.activeviam.io.dlc.impl.ParserOverrides	com.activeviam.io.dlc.impl.description.ParserOverridesDescription
Renamed & moved	com.activeviam.io.dlc.impl.DlcAliases	com.activeviam.io.dlc.impl.description.AliasesDescription
Renamed & moved	com.activeviam.io.dlc.api.description.sink.AnonymousSinkDescription	com.activeviam.io.dlc.impl.description.topic.channel.publish.AnonymousTargetDescription
Renamed & moved	com.activeviam.io.dlc.api.description.sink.SinkDescription	com.activeviam.io.dlc.impl.description.topic.channel.publish.TargetDescription
Renamed & moved	com.activeviam.io.dlc.api.description.column.calc.AnonymousColumnCalculatorDescription	com.activeviam.io.dlc.impl.description.topic.channel.column.calc.AnonymousCustomFieldDescription
Renamed & moved	com.activeviam.io.dlc.api.description.column.calc.ColumnCalculatorDescription	com.activeviam.io.dlc.impl.description.topic.channel.column.calc.CustomFieldDescription
Renamed & moved	com.activeviam.io.dlc.api.request.DlcOperationScope	com.activeviam.io.dlc.impl.operations.request.scope.DlcScope
Moved	com.activeviam.io.dlc.api.description.DlcDescription	com.activeviam.io.dlc.impl.description.DlcDescription
Moved	com.activeviam.io.dlc.impl.NamedEntityResolverService	com.activeviam.io.dlc.impl.utils.NamedEntityResolverService
Moved	com.activeviam.io.dlc.api.request.DlcLoadRequest	com.activeviam.io.dlc.impl.operations.request.DlcLoadRequest
Moved	com.activeviam.io.dlc.api.annotations.ConditionalOnDlcEnabled	com.activeviam.io.dlc.impl.annotations.ConditionalOnDlcEnabled
Moved	com.activeviam.io.dlc.description.AwsCsvSourceDescription	com.activeviam.io.dlc.impl.description.source.AwsCsvSourceDescription
Moved	com.activeviam.io.dlc.description.AzureCsvSourceDescription	com.activeviam.io.dlc.impl.description.source.AzureCsvSourceDescription
Moved	com.activeviam.io.dlc.description.GcpCsvSourceDescription	com.activeviam.io.dlc.impl.description.source.GcpCsvSourceDescription
Moved	com.activeviam.io.dlc.description.JdbcSourceDescription	com.activeviam.io.dlc.impl.description.source.JdbcSourceDescription
Moved	com.activeviam.io.dlc.description.LocalAvroSourceDescription	com.activeviam.io.dlc.impl.description.source.LocalAvroSourceDescription
Moved	com.activeviam.io.dlc.description.LocalCsvSourceDescription	com.activeviam.io.dlc.impl.description.source.LocalCsvSourceDescription
Moved	com.activeviam.io.dlc.description.LocalParquetSourceDescription	com.activeviam.io.dlc.impl.description.source.LocalParquetSourceDescription
Moved	com.activeviam.io.dlc.description.TupleSourceDescription	com.activeviam.io.dlc.impl.description.source.TupleSourceDescription
Moved	com.activeviam.io.dlc.description.AvroTopicDescription	com.activeviam.io.dlc.impl.description.topic.AvroTopicDescription
Moved	com.activeviam.io.dlc.description.CsvTopicDescription	com.activeviam.io.dlc.impl.description.topic.CsvTopicDescription
Moved	com.activeviam.io.dlc.description.JdbcTopicDescription	com.activeviam.io.dlc.impl.description.topic.JdbcTopicDescription
Moved	com.activeviam.io.dlc.description.ParquetTopicDescription	com.activeviam.io.dlc.impl.description.topic.ParquetTopicDescription
Moved	com.activeviam.io.dlc.description.TupleTopicDescription	com.activeviam.io.dlc.impl.description.topic.TupleTopicDescription
Moved	com.activeviam.io.dlc.description.UnloadTopicDescription	com.activeviam.io.dlc.impl.description.topic.UnloadTopicDescription
Moved	com.activeviam.io.dlc.api.description.ChannelDescription	com.activeviam.io.dlc.impl.description.topic.channel.ChannelDescription
Moved	com.activeviam.io.dlc.description.CsvParserDescription	com.activeviam.io.dlc.impl.description.topic.parser.CsvParserDescription
Moved	com.activeviam.io.dlc.impl.operations.request.DlcListenStatusRequest	com.activeviam.io.dlc.api.request.DlcListenStatusRequest
Moved	com.activeviam.io.dlc.api.request.DlcLoadRequest	com.activeviam.io.dlc.impl.operations.request.DlcLoadRequest
Moved	com.activeviam.io.dlc.api.request.DlcStartListenRequest	com.activeviam.io.dlc.impl.operations.request.DlcStartListenRequest
Moved	com.activeviam.io.dlc.api.request.DlcListenStatusRequest	com.activeviam.io.dlc.impl.operations.request.DlcListenStatusRequest
Moved	com.activeviam.io.dlc.api.request.DlcStopListenRequest	com.activeviam.io.dlc.impl.operations.request.DlcStopListenRequest
Moved	com.activeviam.io.dlc.api.request.DlcUnloadRequest	com.activeviam.io.dlc.impl.operations.request.DlcUnloadRequest
Moved	com.activeviam.io.dlc.impl.operations.DlcListenStatusOperation	com.activeviam.io.dlc.impl.operations.DlcListenStatusOperation
Moved	com.activeviam.io.dlc.impl.operations.DlcLoadOperation	com.activeviam.io.dlc.impl.operations.DlcLoadOperation
Moved	com.activeviam.io.dlc.impl.operations.DlcStartListenOperation	com.activeviam.io.dlc.impl.operations.DlcStartListenOperation
Moved	com.activeviam.io.dlc.impl.operations.DlcStopListenOperation	com.activeviam.io.dlc.impl.operations.DlcStopListenOperation
Moved	com.activeviam.io.dlc.impl.operations.DlcUnloadOperation	com.activeviam.io.dlc.impl.operations.DlcUnloadOperation
Moved	com.activeviam.io.dlc.api.response.DlcStatus	com.activeviam.io.dlc.api.operations.response.DlcStatus

NamedEntityResolverService

OLD Method	NEW Method
columnCalculatorDescriptions	customFields
getDlcSinkDescription	getTarget

CSV Parser

OLD Method	NEW Method
columnNames	columns

OLD Property	NEW Property
column-names	columns

Channel

OLD Property	NEW Property
target-name	target

JDBC Source

OLD Method	NEW Method
driverClassName	driverClass

OLD Property	NEW Property
driver-class-name	driver-class

See Atoti Data Connectors 5.0.0 Migration Notes for details on:

• Completely re-written DLC
• Our step guide to migrating to the new DLC API

Changes between 5.0.0-M1 and M2

Added

Issue Key	Details
BAST-1246	Added implicit parsing column calculators which can be overridden via ParserOverrides bean
BAST-1249	Added cloud (Azure, AWS, GCP) support for CSV.
BAST-1153	Added support for JDBC source type.
BAST-1177	Added support for UNLOAD operation.
BAST-1154	Added support for listening operations (START_LISTEN, LISTEN_STATUS, STOP_LISTEN).
BAST-1151	Added support for Avro and Parquet.

Changed

Issue Key	Details
BAST-1162	Requests now enforce that every topic belongs to only one source. You can specify the source / source type in the request.
BAST-1262	Changed the structure of the configuration properties.

Fixed issues introduced in 5.0.0-M1

Issue Key	Details
BAST-1238	Correctly propagate errors to the IDlcResponse object & type requests and responses.

Migrate from 5.0.0-M1 to 5.0.0-M2

Upgrading from version 5.0.0-M1, see Atoti Data Connectors 5.0.0-M2 Release Notes.

Atoti Data Connectors uses Atoti Server 6.1.1 . For new features and fixes included in that release, please see the release notes for Atoti Server.

Breaking Changes

• Configuration Properties : The structure of the configuration properties has changed.
• Single Source per Topic : Requests now enforce that every topic belongs to only one source. You can specify the source / source type in the request.

Headline announcement

See Atoti Data Connectors 5.0.0 Migration Notes for details on:

• Implicit Parsing Column Calculators: Added implicit parsing column calculators which can be overridden via ParserOverrides bean. This means you no longer need to keep CsvColumnParsers in your channel description.
• CSV Cloud support: Added support for loading CSV files from Azure, AWS, and GCP.
• UNLOAD Operation: Added support for UNLOAD operation.
• Listening Operations: Added support for listening operations (START_LISTEN, LISTEN_STATUS, STOP_LISTEN).
• Avro & Parquet: Added support for Avro and Parquet.

Dependency Changes

None.

Configuration files

The format of the configuration properties has changed.

Previous	New	Additional Notes
dlc.local-csv-sources	dlc.csv.local.sources	Sources were previously a list, now they are a map from source name to description
dlc.csv-topics	dlc.csv.topics

Services

No changes.

Other Changes

Operations can now be extended directly rather than via the IDlcOperation interface. DlcLoadRequest.builder().operation("LOAD") should be replaced with DlcLoadRequest.builder()