> ## Documentation Index
> Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt
> Use this file to discover all available pages before exploring further.

# ParquetLoad

<span id="atoti_parquet.ParquetLoad" />

> atoti\_parquet.ParquetLoad(<br />
>     *path*: [Path](https://docs.python.org/3/library/pathlib.html#pathlib.Path) | [str](https://docs.python.org/3/library/stdtypes.html#str),<br />
>     \*,<br />
>     *client\_side\_encryption*: [ClientSideEncryptionConfig](./atoti.ClientSideEncryptionConfig#atoti.ClientSideEncryptionConfig) | [None](https://docs.python.org/3/library/constants.html#None) = `None`,<br />
>     *columns*: [Mapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping)\[[str](https://docs.python.org/3/library/stdtypes.html#str), [str](https://docs.python.org/3/library/stdtypes.html#str)] = `frozendict({})`,<br />
> )

The definition of an Apache Parquet file load.

The [`load()`](./atoti.Table.load#atoti.Table.load) outcome depends on the *error\_handling* argument.
For instance, the following Parquet file has a **Count** column stored as `float64`, but `table` expects `int`.

<Note>
  Parquet is strongly typed, so the type mismatch is caught up front and no rows load regardless of *error\_handling*.
  This argument only controls whether an error or warning is raised and what is logged.
  See [`CsvLoad`](./atoti.data_load.CsvLoad#atoti.CsvLoad) for an example where invalid rows are skipped individually.
</Note>

```pycon theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
>>> import pyarrow as pa
>>> import pyarrow.parquet as pq
>>> file_path = tmp_path / "data.parquet"
>>> pq.write_table(
...     pa.table(
...         {
...             "City": ["Paris", "London"],
...             "Count": pa.array([3.14, 2.72]),
...         }
...     ),
...     file_path,
... )
>>> table = session.create_table(
...     "Error handling",
...     data_types={"City": "String", "Count": "int"},
...     keys={"City"},
... )
```

* `"fail"` raises an error and interrupts the load:
  ```pycon theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
  >>> table += ("Rome", 42)
  >>> table.head()
        Count
  City
  Rome     42
  >>> from atoti_parquet import ParquetLoad
  >>> parquet_load = ParquetLoad(file_path)
  >>> with session.tables.data_transaction():  
  ...     table.load(parquet_load, error_handling="fail")
  Traceback (most recent call last):
      ...
  py4j.protocol.Py4JError: ...Caused by: com.activeviam.tech.core.api.exceptions.ConfigurationException: Incompatible Type: datastore type 'int' doesn't match parquet type 'double' for the store field 'Count' and parquet field 'Count'...
  >>> table.head()
        Count
  City
  Rome     42
  >>> exception_message = "Caused by: com.activeviam.tech.core.api.exceptions.ConfigurationException: Incompatible Type: datastore type 'int' doesn't match parquet type 'double' for the store field 'Count' and parquet field 'Count'"
  >>> _count_transaction_exceptions(exception_message) > 0
  True
  ```
* `"warn"` skips the incompatible data and raises a Python warning:
  ```pycon theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
  >>> from warnings import catch_warnings, simplefilter
  >>> with (
  ...     session.tables.data_transaction(),
  ...     catch_warnings(record=True) as warnings,
  ... ):
  ...     simplefilter("always")
  ...     table.load(parquet_load, error_handling="warn")
  >>> [warning.message for warning in warnings]
  [UserWarning("Some errors occurred while loading data into t['Error handling'].")]
  >>> table.head()
        Count
  City
  Rome     42
  >>> _count_transaction_exceptions(exception_message) > 0
  True
  ```
* `"log"` skips the incompatible data and logs the errors:
  ```pycon theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
  >>> with session.tables.data_transaction():
  ...     table.load(parquet_load, error_handling="log")
  >>> table.head()
        Count
  City
  Rome     42
  >>> _count_transaction_exceptions(exception_message) > 0
  True
  ```
* `"skip"` skips the incompatible data without logging:
  ```pycon theme={"languages":{"custom":["/engine/python-sdk/0.9/languages/pycon.tmLanguage.json"]}}
  >>> with session.tables.data_transaction():
  ...     table.load(parquet_load, error_handling="skip")
  >>> table.head()
        Count
  City
  Rome     42
  >>> _count_transaction_exceptions(exception_message)
  0
  ```

<Callout icon="link">
  **See also**:
  The other [`DataLoad`](./atoti.data_load.DataLoad#atoti.data_load.DataLoad) implementations.
</Callout>

### Attributes

<h4 id="atoti_parquet.ParquetLoad.path">
  *path*
</h4>

The path to the Parquet file.

If a path pointing to a directory is provided, all of the files with the `.parquet` extension in the directory will be loaded into the same table and, as such, they are all expected to share the same schema.

The path can also be a glob pattern (e.g. `"path/to/directory/*.parquet"`).

<h4 id="atoti_parquet.ParquetLoad.client_side_encryption">
  *client\_side\_encryption*
</h4>

> [ClientSideEncryptionConfig](./atoti.ClientSideEncryptionConfig#atoti.ClientSideEncryptionConfig) | [None](https://docs.python.org/3/library/constants.html#None)

<h4 id="atoti_parquet.ParquetLoad.columns">
  *columns*
</h4>

Mapping from file column names to table column names.

When the mapping is not empty, columns of the file absent from the mapping keys will not be loaded.
Other parameters accepting column names expect to be passed table column names (i.e. values of this mapping) and not file column names.