Configurations

This documentation is for an unreleased version of Apache Paimon. We recommend you use the latest stable version.

Configurations #

Options for Iceberg Compatibility.

Key	Default	Type	Description
metadata.iceberg.compaction.max.file-num	50	Integer	If number of small Iceberg manifest metadata files exceeds this limit, always trigger manifest metadata compaction regardless of their total size.
metadata.iceberg.compaction.min.file-num	10	Integer	Minimum number of Iceberg manifest metadata files to trigger manifest metadata compaction.
metadata.iceberg.database	(none)	String	Metastore database name for Iceberg Catalog. Set this as an iceberg database alias if using a centralized Catalog.
metadata.iceberg.delete-after-commit.enabled	true	Boolean	Whether to delete old metadata files after each table commit
metadata.iceberg.format-version	2	Integer	The format version of iceberg table, the value can be 2 or 3. Note that only version 3 supports deletion vector.
metadata.iceberg.glue.skip-archive	false	Boolean	Skip archive for AWS Glue catalog.
metadata.iceberg.hadoop-conf-dir	(none)	String	hadoop-conf-dir for Iceberg Hive catalog.
metadata.iceberg.hive-client-class	"org.apache.hadoop.hive.metastore.HiveMetaStoreClient"	String	Hive client class name for Iceberg Hive Catalog.
metadata.iceberg.hive-conf-dir	(none)	String	hive-conf-dir for Iceberg Hive catalog.
metadata.iceberg.hive-skip-update-stats	false	Boolean	Skip updating Hive stats.
metadata.iceberg.manifest-compression	"snappy"	String	Compression for Iceberg manifest files.
metadata.iceberg.manifest-legacy-version	false	Boolean	Should use the legacy manifest version to generate Iceberg's 1.4 manifest files.
metadata.iceberg.previous-versions-max	0	Integer	The number of old metadata files to keep after each table commit
metadata.iceberg.storage	disabled	Enum	When set, produce Iceberg metadata after a snapshot is committed, so that Iceberg readers can read Paimon's raw data files. Possible values: "disabled": Disable Iceberg compatibility support. "table-location": Store Iceberg metadata in each table's directory. "hadoop-catalog": Store Iceberg metadata in a separate directory. This directory can be specified as the warehouse directory of an Iceberg Hadoop catalog. "hive-catalog": Not only store Iceberg metadata like hadoop-catalog, but also create Iceberg external table in Hive.
metadata.iceberg.storage-location	(none)	Enum	To store Iceberg metadata in a separate directory or under table location Possible values: "table-location": Store Iceberg metadata in each table's directory. Useful for standalone Iceberg tables or Java API access. Can also be used with Hive Catalog "catalog-location": Store Iceberg metadata in a separate directory. Allows integration with Hive Catalog or Hadoop Catalog.
metadata.iceberg.table	(none)	String	Metastore table name for Iceberg Catalog.Set this as an iceberg table alias if using a centralized Catalog.
metadata.iceberg.uri	(none)	String	Hive metastore uri for Iceberg Hive catalog.