Parquet S3 FDW 1.1.0 released

Posted on 2023-12-26 by Toshiba
Related Open Source

We have just released version 1.1.0 of the Foreign Data Wrapper for Parquet file on Amazon S3.

This release can work with PostgreSQL 13, 14, 15 and 16.

This release improves following item (from 1.0.0):

  • Support PosgreSQL 16.0
  • Support DDL feature for PGSpider
  • Apply Parquet FDW updates

The FDW supports following features :

  • Support SELECT of parquet file on local file system or Amazon S3
  • Support INSERT, DELETE, UPDATE (Foreign modification).
  • Support MinIO access instead of Amazon S3.
  • Allow control over whether foreign servers keep connections open after transaction completion. This is controlled by keep_connections and defaults to on.
  • Support parquet_s3_fdw function parquet_s3_fdw_get_connections() to report open foreign server connections.
  • Support to create a foreign table for multiple files in a directory by specifying a directory path.
  • Support schemaless mode that allows access to elements in the data file without changing the table definition by using jsonb typed column.

This is developed by Toshiba Digital Innovation Technology Center. Please see the repository for details.
Source repository : https://github.com/pgspider/parquet_s3_fdw