Changelog

For information on upgrading from previous versions, see the Atoti Data Connectors Migration Notes

4.2.1

2024-10-04

Summary

  • Removed CVE-2024-28752 coming from a cxf-rt-rs-client import: This import has been removed from the Data-Connectors dependencies.

Dependency versions

Component Version
Atoti Server 6.0.11.sb3+
Java JDK17

Removed

Issue Key Details
BAST-1163 Removed org.apache.cxf.cxf-rt-rs-client dependency with CVE-2024-28752.

4.2.0-AS6.1

2024-10-25

Summary

  • Upgraded to Atoti Server 6.1: This release of Atoti Data Connectors is intended for you to use as a bridge while upgrading to Atoti Server 6.1 and should be replaced with the soon-to-be released Atoti Data Connectors 5.0.
  • Removed CVE-2024-28752 coming from a cxf-rt-rs-client import: This import has been removed from the Data-Connectors dependencies.
  • Upgraded to Java 21: We have upgraded from Java 17 to Java 21.

Upgrade to AS6.1

Atoti Server 6.1 introduces a new API structure that finalizes the Sources. The DLC has extended the Sources so we can pick and choose which files to load based on the DLC’s Scope. Atoti Server 6.1 has added new Topic features to allow us to load specific files without the need to extend the core Sources.

Consequently, the DLC Sources no longer extend the core Source, but instead delegate all API calls to a local delegate instance of the Source.

note

The DLC APIs have not changed and can still be used as they are.

However, we encourage you to move to Atoti Data Connectors 5.0 and implement the re-designed DLC API when it becomes available.

Known issues

None.

Dependency versions

Component Version
Atoti Server 6.1.1
Java JDK21

Changed

Issue Key Details
BAST-1077 Upgraded to Atoti Server 6.1.1. The minimum supported Atoti Server version is now 6.1.1.
BAST-1077 Upgraded to Java 21 for Atoti Server 6.1.

Removed

Issue Key Details
BAST-1163 Removed org.apache.cxf.cxf-rt-rs-client dependency with CVE-2024-28752.

4.2.0

2024-06-20

Summary

  • Allow Listening on Fetch Topics: The DLC now allows Listening on defined Fetch Topics.

Dependency versions

Component Version
Atoti Server 6.0.11.sb3+
Java JDK17

Added

Issue Key Details
BAST-1069 Allow Listening on Fetch Topics/Sources.

Listening on Fetch Topics

The DLC now allows Listening on defined Fetch Topics. With this you can start listening on a Fetch Topic. There is no need to register a new Listen Source nor define Listen Topics.

When Listening on a Fetch Topic, a hybrid ListenOnFetchSource and topics will be created that will listen for file updates and process them in a batch Fetch operation.

For more on how the Listen-on-Fetch mechanism works and what can be configured, see the Listen-on-Fetch section.

4.1.1-AP6.0-sb3

2024-04-22

Summary

  • Fixed DEE CSV Search Facts Queries: The CSV Search Facts Queries have been fixed in this version.

Known issues

None.

Dependency versions

Component Version
Atoti Server 6.0.11.sb3+
Java JDK17

Fixed

Issue Key Details
PST-1066 Fixed issue of using the incorrect constructor for CSV Search Facts Queries in the DEE.

4.1.0

2024-04-18

Summary

  • Removed DEE Remoting Services : The remoting services have been removed in response to vulnerability CVE-2016-1000027 to prevent potential remote code execution. See the migration notes for list of removed services.
  • Upgrade to Java 17: Upgraded to use Java 17.
  • Upgraded to Spring Boot 3 : Upgraded to use Spring Boot 3. With this, we have also updated to use Open Telemetry for tracing.

Known issues

  • DEE CSV Search Facts Queries: The CSV Search Facts Queires fail in the DEE.

Dependency versions

Component Version
Atoti Server 6.0.11.sb3+
Java JDK17

Changed

Issue Key Details
PST-1048 Upgrade to Atoti 6.0.11-SB3 to support Spring Boot 3. The minimum supported Atoti version is now Atoti 6.0.11-SB3.

Removed

Issue Key Details
PST-991 Removed remoting services: DataExtractionRemotingServicesConfig, DataExtractionRemotingServicesConfig in response to vulnerability CVE-2016-1000027 as well as the dependencies on the package javax.ws.rs.*

4.0.5

2024-05-09

Summary

  • Fixed DEE CSV Search Facts Queries: The CSV Search Facts Queries have been fixed in this version.

Known issues

Issue Key Details
PST-953 Depending on your project’s javax/jakarta dependency, if an exception is raised in the DEE, then a java.lang.NoClassDefFoundError: jakarta/ws/rs/ext/RuntimeDelegate exception may be thrown.
See workaround below.

Dependency versions

Component Version
Atoti Server 6.0.2+
Java JDK11

Fixed

Issue Key Details
PST-1074 Fixed DEE SearchFactsQueries failing due to missing constructor runtime error.

4.0.4

2024-04-18

Summary

Known issues

Issue Key Details
PST-953 Depending on your project’s javax/jakarta dependency, if an exception is raised in the DEE, then a java.lang.NoClassDefFoundError: jakarta/ws/rs/ext/RuntimeDelegate exception may be thrown.
See workaround below.

Workaround for mishandled exceptions in DEE (PST-953)

The javax library has been replaced with jakarta. The RuntimeDelegate that javax/jakarta use may not be available depending on the dependencies in your project. This can cause exceptions in the DEE to be consumed when the DEE tries to wrap them into a Response object.

To resolve this, add the following dependencies to your application:

<dependency>
	<groupId>javax.ws.rs</groupId>
	<artifactId>javax.ws.rs-api</artifactId>
	<version>2.1.1</version>
</dependency>
<dependency>
	<groupId>org.glassfish.jersey.core</groupId>
	<artifactId>jersey-common</artifactId>
</dependency>

note

In the future, the DEE will move away from relying on javax/jakarta and instead of its REST services returning Response objects, we will use Spring to return DTO response objects.

Dependency versions

Component Version
Atoti Server 6.0.2+
Java JDK11

Fixed

Issue Key Details
PST-1066 Configuration property: dee.get-aggregates-query.use-default-members always uses value of false.

4.0.3

2023-10-30

Summary

  • GAQ Slicer Level Handling: Configuration property has been added to adjust how members of slicer levels are handled in Get Aggregates Queries DEE orders.

Known issues

Issue Key Details
PST-953 Depending on your project’s javax/jakarta dependency, if an exception is raised in the DEE, then a java.lang.NoClassDefFoundError: jakarta/ws/rs/ext/RuntimeDelegate exception may be thrown.
See workaround below.
PST-1066 Configuration property: dee.get-aggregates-query.use-default-members always uses value of false.

Workaround for mishandled exceptions in DEE (PST-953)

The javax library has been replaced with jakarta. The RuntimeDelegate that javax/jakarta use may not be available depending on the dependencies in your project. This can cause exceptions in the DEE to be consumed when the DEE tries to wrap them into a Response object.

A resolution may be to add the following dependencies to your application:

<dependency>
	<groupId>javax.ws.rs</groupId>
	<artifactId>javax.ws.rs-api</artifactId>
	<version>2.1.1</version>
</dependency>
<dependency>
	<groupId>org.glassfish.jersey.core</groupId>
	<artifactId>jersey-common</artifactId>
</dependency>

note

In future, the DEE will move away from relying on javax/jakarta and instead of its REST services returning Response objects, we will use Spring to return DTO response objects.

Dependency versions

Component Version
Atoti Server 6.0.2+
Java JDK11

Added

Issue Key Details
PST-962 Added configuration property for handling Slicer Levels with DEE GAQ. For details, see dee.get-aggregates-query.use-default-members.

4.0.2

2023-08-09

Summary

  • DLC and DEE services discoverable in /versions/rest endpoint : The DLC and DEE services are discoverable in the /versions/rest REST service. See REST Service Version for more information.
  • Deprecated internal REST services: Some internal REST services have been deprecated. See the Migration Notes for more information.

Fixed

Known Issues

None.

4.0.1

2023-02-17

Summary

ActivePivot 6.0.2 Support : Support for ActivePivot 6.0.2 and up.

Fixed

  • PIVOT-5950: Datastore Sizes will be printed when a DLC Load / Unload operation is performed.

Known Issues

No known issues at this time.

4.0.0

2022-12-13

Summary

ActivePivot 6.0 Support : Support for ActivePivot 6.0 and up.

Known issues

ActivePivot 6.0.1 will cause issues with the DLC Listen topics.

Added

Changed

Changed some of the Rest Services to better align with Spring MVC and removed exposing Rest Services through ActivePivot’s ASpringRestServerConfig. Instead, we now expose Rest services by extending Spring’s ASpringVersionedRestServerConfig and WebMvcConfigurer classes. You can find a complete list of classes that have been renamed or refactored in the Renamed Classes section of the Migration notes.

Removed

The following classes and features have been removed:

Prerequisites

The following requirements are no longer required as of Data Connectors 4.0.0:

  • No longer need a ServicesConfigBeanPostProcessor.class in your ApplicationConfig.
  • No longer need to specify the "com.activeviam.apm" path in your Registry as the APM has been integrated into the core ActivePivot.

Features

  • CFX support
  • XML support for DEE order definitions
  • DEE Remoting Service

3.0.2

2023-02-02

Summary

3.0.1

2022-07-07

Summary

  • Small fix to fix reading DEE Template Orders located within JAR files.

Known Issues

  • Updated 2023-02-02 Data Connectors 3.0.1-AP5.11 does not work with AP 5.11.5+. You will need to upgrade to Data Connectors 3.0.2-AP5.11.

3.0.0

2022-02-04

Support for ActivePivot 5.11.

Summary

  • Support for ActivePivot 5.11.0 : Data-Connectors has been upgraded to support ActivePivot 5.11+.
  • Migrated to using ActivePivot’s APM : Migrated to use ActivePivot 5.11’s included APM.

Known Issues

The name of the bean used to initialize the APM’s Tracing APMTracingConfig.APM_TRACING has private access in AP5.11.0 & 5.11.1. For now, you can use the value of the String in your @DependsOn statement for your initial load.

Here is an example:

    @Bean
    @DependsOn(value = {"APM_TRACING"})
    public Void initialLoad() {
        // Load data here...
    }

Dependency Version Updates

Component Version
ActivePivot 5.11.0
Spring 2.6.1
Kafka 3.1.0
Avro 1.11.0

Fixed

Issue Details
TransactionTaskUtil.getExceptionEventsForTask() returned empty list getExceptionEventsForTask(taskName) now returns a collection of failure events.

2.2.2

2022-12-10

Summary

  • Added support for template place holders in GetAggregatesQuery
  • Added new DlcMessageHandler to catch exceptions thrown during CSV tuple publishing. Usage is explained in CSV Configuration and examples of creating custom IMessageHandlers can be found in Creating Custom CSV Message Handlers section. For migrating to using the new DlcMessageHandler please read the Migration Notes

Added

Issue Key Details
PST-757 Added support for template place holders in GetAggregatesQuery

Fixed

Issue Key Details
PST-747 Fixed issue where DLC would throw exception when Stop Listening on a Topic-Alias or on multiple Topics.
PST-717 Fixed issue where DLC was not aware of errors thrown during CSV Tuple Publishing.
PST-749 Allow DEE parameter formatNAToEmpty to work with all extraction queries (ie: SearchFactsQueries).
Class Cast Exception in DEE on failing orders Removed ClassCastException in ADataExtractionRestService.getTaskIds that caused DEE error messages to get eaten.

2.2.1

2021-11-23

Summary

  • Removed unused dependency that broke compatibility with Excel.

Fixed

Issue Key Details
PST-725 Removed unused dependency that broke compatibility with Excel.

2.2.0

2021-10-19

Summary

  • Add APM Monitoring to DEE : Added JMX properties to allow for additional logging of dee tasks.

Dependency versions

Component Version
ActivePivot 5.9.6
Java JDK11

Known Issues

  • APM needs to be upgraded to 10.0.1 in order to prevent issue where parsing of large files will cause null SpanContext value for HealthEvents, leading to the DLC not properly adding events to the Cache. This can be avoided by excluding APM from Data-Connectors then re-importing the correct 5.10.1 version. Example below:
<dependency>
  <groupId>com.activeviam.io</groupId>
  <artifactId>data-connectors-csv</artifactId>
  <version>2.2.0-AP5.10</version>
  <exclusions>
    <exclusion>
      <groupId>com.activeviam.lib</groupId>
      <artifactId>activepivot-apm</artifactId>
    </exclusion>
  </exclusions>
</dependency>

<dependency>
    <groupId>com.activeviam.lib</groupId>
    <artifactId>activepivot-apm</artifactId>
    <version>5.10.1</version>
</dependency>

Added

Issue Key Details
PST-733 Added APM Monitoring to DEE

Changed

Issue Key Details
PST-737 Upgraded APM Version to 5.10

Removed

  • com.activeviam.io.monitoring.LoggingUtils
    • Replaced by com.activeviam.apm.logging.impl.LoggingUtils

Fixed

Issue Key Details
PST-744 Fixed time units used in ADatastoreTransactionTask as the Nanosecond throughput time was in Milliseconds.

2.1.0

Fixed

Issue Key Details
PST-706 Removed NPE when checking if events have errors during Loading or Unloading of data
PST-711 Added safety checks when accessing DLC cache for expired events. Added proper closing of TraceContexts used to trace DLC IExecutionTask

Added

Key Details
PST-667 Added support for Listening on CSV files on the Local and Cloud file system. More can be found on the DLC Real-Time Data Sources page.
PST-712 Added ability to specify Get Aggregate Query exports using Level@Hierarchy@Dimension=Value Location strings. More can be found on Get aggregates query DEE order

Changed

Issue Key Details
PST-703 Removed Deprecation of Enhanced Drillthroughs as we will be keeping the enhanced drillthrough functionality

2.0.1

Fixed

Issue Key Details
PST-708 Updated APM dependency version that was causing the SpanContext of DLC events to be null in the DLC Cache

2.0.0

Known Issues

  • Parsing of large files will cause null SpanContext value for HealthEvents, leading to the DLC not properly adding events to the Cache.

Changed

  • AP version has been updated to 5.10.1
  • The Azure Cloud Source has migrated its dependency to the Azure Blob Storage SDK from v8 to v12

1.2.1

Fixed

Issue Key Details
PST-706 Removed NPE when checking if events have errors during Loading or Unloading of data

1.2.0

Added

Issue Key Details
PST-688 Integrated APM tracing for DLC health events

Fixed

Issue Key Details
PST-685 Fixed DLC response timeTakenMs to show ms instead of nanoseconds
PST-690 Fixed DEE issue in AEnhancedDrillthroughService in case of entries with the same underlying datastore field and entries without dictionaries. Fixed export of vector

Changed

Issue Key Details
PST-684 Removed IScopedFetchSource interface as it causes confusion, and is functionally identical to IFetchingDataSource
PST-670 Added CustomKafkaAvroDeserializerGenericRecord and CustomKafkaAvroDeserializerObjectTuple classes

1.1.0

Fixed

Issue Key Details
BAS-670 Removed NPE thrown when REST endpoint getTaskStatus() called after executing a DEE Order containing a bad MDX

Added

Issue Key Details
PST-662 Product Center’s ParquetParserBuilder has been implemented. This implementation requires AP version 5.9.2+
PST-668 Added ability to customize ordering of Topics within a given request. For more information, see Custom Ordering for Loading Topics