Skip to main content

Query Engines

An engine's Iceberg connector can read Paimon's published Iceberg representation if it supports the catalog, data file format, column types, and Iceberg features used by the table. Configure the reader's access to both metadata and the original Paimon data files.

For Flink and Spark, start with the append-table walkthrough. The examples below use Iceberg connectors; all writes and maintenance remain in Paimon.

Trino Iceberg​

First publish the animals table using the Hive catalog example. Configure Trino to connect to the same Hive metastore.

Create etc/catalog/iceberg.properties on the Trino server:

connector.name=iceberg
iceberg.catalog.type=hive_metastore
hive.metastore.uri=thrift://<metastore-host>:9083

Add the filesystem configuration and credentials required by your storage. Trino requires a configured filesystem in addition to the metastore connection; see the Iceberg connector configuration.

Query the registered table with the fully qualified catalog, schema, and table name:

SELECT kind, name
FROM iceberg.default.animals
WHERE kind = 'mammal'
ORDER BY name;
kind name
mammal cat
mammal dog

If you configured metadata.iceberg.database or metadata.iceberg.table, use those registration names in the Trino query.

AWS Athena​

Athena discovers Iceberg tables through AWS Glue. Publish the table using a properly configured Glue Hive client, and check Athena's Iceberg limitations. Athena documents support for Iceberg v2 tables. Use the full-compaction mode for primary key tables when querying with a reader that cannot consume v3 deletion vectors.

For readers that require the legacy manifest field naming, enable this Paimon table option:

'metadata.iceberg.manifest-legacy-version' = 'true'

This controls the manifest representation; it does not add support for otherwise unsupported Iceberg versions or column types.

DuckDB​

Install and load DuckDB's Iceberg extension and choose the catalog or metadata-file access method supported by your extension version. Check its support for the format version and deletion information used by the table.

When troubleshooting a read, verify that the data paths referenced by the manifests are reachable. Paimon's data files remain in the Paimon table directory even when the Iceberg metadata is in a separate warehouse. Do not treat <warehouse>/iceberg as a copy of the data.

For integrations that require data files under a data subdirectory, Paimon supports the following option for new writes:

'data-file.path-directory' = 'data'

Changing this setting does not relocate existing data files. Use it only when required by the reader or integration you deploy. Keep metadata.iceberg.manifest-compression at its default snappy unless the reader supports the alternative codec.

Troubleshooting​

SymptomWhat to check
Table not foundPublication mode, Hadoop warehouse path, Hive/REST registration name, and writer-side catalog dependencies
New rows are missingCompleted Paimon commit, primary key compaction mode, metadata publication, and reader cache
Data files cannot be openedReader filesystem support, credentials, and access to the original Paimon data paths
Schema or manifest parsing failsType restrictions, format version, manifest codec, and connector version
Tag not foundWhether its snapshot exists in Iceberg metadata, reader refresh, and REST tag limitations

For Hive or REST tables missing from the service, inspect the writer logs as well as the local metadata directory. A missing metadata committer dependency can leave local Iceberg metadata files present without registering the table in the external catalog.