ActivePivot

ActivePivot

  • 5.9.15
  • Other Versions
  • User Guide
  • Technical Documentation
  • Support

›Loading data from the cloud

Introduction

  • Overview
  • What's new in ActivePivot

Getting Started

  • Overview
  • AP in a Nutshell
  • Development Environment
  • Download
  • Sandbox Project

Concepts

  • Overview
  • AP Concepts in a Nutshell
  • Data Versioning (MVCC)
  • Dimensions and Hierarchies
  • Partitioning and NUMA
  • Other Concepts

Data Loading

  • Overview
  • Datastore

    • Datastore Configuration
    • Datastore Transactions
    • Store Indexing

    ETL

    • Overview
    • CSV Source
    • JDBC Source
    • Parquet Source

    Loading data from the cloud

    • Cloud Source
    • Amazon S3 Cloud Source
    • Azure Cloud Source
    • Google Cloud Source

Aggregation & Analytics

  • Overview
  • Cube Configuration
  • Copper API

    • Introduction
    • API
    • Measures
    • Hierarchies
    • Publication
    • Join operations
    • Advanced topics

    Streaming API

    • Continuous Queries Overview
    • Streaming Overview
    • Continuous Query Engine
    • Continuous Handlers

    Advanced APIs

    • Cube Locations
    • Post-Processors
    • Cube Filters
    • Member Properties
    • Context Values

Data Querying

  • Overview
  • Business Frontends
  • Server Endpoints

    • XMLA
    • Datastore REST API
    • Cube REST API
    • Cube Websocket API

    MDX

    • MDX Engine Configuration
    • MDX Functions
    • MDX Operators
    • MDX Formats
    • MDX Filtering
    • MDX Snippets
    • MDX Cellsets
  • Datastore Queries
  • Location-Based Queries
  • Drillthrough Extensions

Configuration

  • Overview
  • ContentServer

    • Content Server
    • ContentServer REST API
    • CS Websocket API
  • ActivePivot Properties
  • Internationalization

Security

  • Overview
  • Client/Server Communication

    • Authentication
    • Authorization & Entitlements

    Data Access Control

    • Datastore Access Control
    • ActivePivot Access Control
    • Branch Permission Manager

Distributed Architecture

  • Overview
  • Communication Flows
  • Post-Processors
  • Security
  • What-If
  • Recommendations
  • Distribution Properties

Operations

  • Overview
  • Monitoring

    • Health Dispatcher
    • Query Execution Plan
    • Monitoring Query Execution
    • JMX monitoring
    • Off-Heap Memory Export
    • Tracing REST API
  • Troubleshooting
  • Performance
  • High Availability

Release & Migration Notes

  • Changelog
  • Migration notes

Reference

  • Javadoc
  • REST APIs

Google Cloud Source

This documentation page assumes you are already familiar with the general structure of Cloud Sources in ActivePivot as well as with Google Cloud Storage.

The Google Cloud Source relies on Google Cloud Storage SDK for Java. Make sure you are familiar with this SDK when using the Google Cloud Source.

Cloud Source to Google Cloud Storage concepts

Entities

The Google implementation of ICloudEntity is GoogleEntity. It is essentially a wrapper around an object from the Google Cloud Storage SDK.

Locating entities

Entity paths

IGoogleEntityPath implements ICloudEntityPath. It is a reference to an object and its metadata.

Directories

The Google Cloud Storage implementation for ICloudDirectory is represented by the GoogleCloudDirectory implementation.

A directory is tied to a bucket. It contains all objects whose names start with a certain prefix. For example, a directory on a certain container with the prefix directory1/subdirectory2 would contain the first three of the following objects:

inside:
  directory1/subdirectory2/object1.txt
  directory1/subdirectory2/object2.txt
  directory1/subdirectory2/subdirectory3/object3.txt

not inside:
  object4.txt
  other_directory/object5.txt
  directory1/object6.txt

A directory with an empty prefix corresponds to the root of the container.

A Google Cloud Storage directory object can be constructed by specifying the Storage client, a bucket name and a prefix.

The Storage client is the configuration of the connection to Google Cloud Storage. It can be configured as follows using the Google Cloud SDK.

Storage storage = StorageOptions.newBuilder().setProjectId("myProject").build().getService();

CSVDataProviderFactory

To configure the CSV source to read Google object, you can use the GoogleCsvDataProviderFactory class to configure how the files are downloaded.

← Azure Cloud SourceNext →
  • Cloud Source to Google Cloud Storage concepts
    • Entities
    • Locating entities
  • CSVDataProviderFactory
ActivePivot
Community
Stack OverflowLinkedinTwitter
More
Blog
Copyright © 2023 ActiveViam