Skip to main content

Configuration Reference

Set these options on the Paimon table to control Iceberg metadata publication. Reader-side Iceberg catalog properties are configured separately.

Find the Right Options​

TaskOptions and guide
Enable publication and choose its locationmetadata.iceberg.storage, metadata.iceberg.storage-location; catalogs and metadata layout
Register in Hivemetadata.iceberg.uri, metadata.iceberg.hive-*, metadata.iceberg.database, metadata.iceberg.table; Hive Catalog
Register in RESTmetadata.iceberg.rest.*; REST Catalog
Publish deletion vectorsmetadata.iceberg.format-version plus deletion-vectors.*; primary key tables
Adjust manifest compatibilitymetadata.iceberg.manifest-compression, metadata.iceberg.manifest-legacy-version; query engines
Control metadata cleanupmetadata.iceberg.delete-after-commit.enabled, metadata.iceberg.previous-versions-max

Metadata retention options control generated metadata files, not Paimon data-file retention. Use Paimon's snapshot management and tag management for table history.

Iceberg Table Options​

Key Default Type Description
metadata.iceberg.compaction.max.file-num
 50 Integer If number of small Iceberg manifest metadata files exceeds this limit, always trigger manifest metadata compaction regardless of their total size.
metadata.iceberg.compaction.min.file-num
 10 Integer Minimum number of Iceberg manifest metadata files to trigger manifest metadata compaction.
metadata.iceberg.database
 (none) String Metastore database name for Iceberg Catalog. Set this as an iceberg database alias if using a centralized Catalog. Multiple databases can be specified with semicolons, e.g. 'db1;db2'. The table will be registered in each database.
metadata.iceberg.delete-after-commit.enabled
 true Boolean Whether to delete old metadata files after each table commit
metadata.iceberg.format-version
 2 Integer The format version of iceberg table, the value can be 2 or 3. Note that only version 3 supports deletion vector.
metadata.iceberg.glue.skip-archive
 false Boolean Skip archive for AWS Glue catalog.
metadata.iceberg.hadoop-conf-dir
 (none) String hadoop-conf-dir for Iceberg Hive catalog.
metadata.iceberg.hive-client-class
 "org.apache.hadoop.hive.metastore.HiveMetaStoreClient" String Hive client class name for Iceberg Hive Catalog.
metadata.iceberg.hive-conf-dir
 (none) String hive-conf-dir for Iceberg Hive catalog.
metadata.iceberg.hive-skip-update-stats
 false Boolean Skip updating Hive stats.
metadata.iceberg.manifest-compression
 "snappy" String Compression for Iceberg manifest files.
metadata.iceberg.manifest-legacy-version
 false Boolean Should use the legacy manifest version to generate Iceberg's 1.4 manifest files.
metadata.iceberg.previous-versions-max
 0 Integer The number of old metadata files to keep after each table commit. For rest-catalog, it will keep 1 old metadata at least.
metadata.iceberg.storage
 disabled

Enum

 When set, produce Iceberg metadata after a snapshot is committed, so that Iceberg readers can read Paimon's raw data files.

Possible values:
  • "disabled": Disable Iceberg compatibility support.
  • "table-location": Store Iceberg metadata in each table's directory.
  • "hadoop-catalog": Store Iceberg metadata in a separate directory. This directory can be specified as the warehouse directory of an Iceberg Hadoop catalog.
  • "hive-catalog": Not only store Iceberg metadata like hadoop-catalog, but also create Iceberg external table in Hive.
  • "rest-catalog": Store Iceberg metadata in a REST catalog. This allows integration with Iceberg REST catalog services.
metadata.iceberg.storage-location
 (none)

Enum

 To store Iceberg metadata in a separate directory or under table location

Possible values:
  • "table-location": Store Iceberg metadata in each table's directory. Useful for standalone Iceberg tables or Java API access. Can also be used with Hive Catalog
  • "catalog-location": Store Iceberg metadata in a separate directory. Allows integration with Hive Catalog or Hadoop Catalog.
metadata.iceberg.table
 (none) String Metastore table name for Iceberg Catalog.Set this as an iceberg table alias if using a centralized Catalog.
metadata.iceberg.unknown-host-retry.initial-delay-ms
 1000 Long Initial delay in milliseconds before retrying an Iceberg REST catalog DNS lookup failure. The delay doubles after each failure.
metadata.iceberg.unknown-host-retry.max-retries
 5 Integer Maximum number of retries after an Iceberg REST catalog DNS lookup failure. Set to 0 to disable retries.
metadata.iceberg.uri
 (none) String Hive metastore uri for Iceberg Hive catalog.

Prefixed Properties​

metadata.iceberg.rest.<key> passes <key> to the REST catalog client. It is a property prefix, so the individual service-specific keys are not listed in the generated table above.

metadata.iceberg.table-properties.<key> supplies custom table properties during REST publication. Iceberg-reserved properties are ignored. Use the dedicated Paimon options for format version and publication behavior.