Ecosystem
Use Paimon tables across ingestion pipelines, SQL engines, and lakehouse management tools. Start with an integration below, then follow Connecting Engines to configure catalog discovery and storage access.
Choose an Integration
| What you want to do | Start here |
|---|---|
| Ingest CDC events or build a streaming pipeline | Flink Quick Start, CDC Ingestion |
| Run batch transformations or Spark SQL | Spark Quick Start |
| Process streams with Spark micro-batches | Spark Structured Streaming |
| Query Paimon from an OLAP engine | StarRocks, Doris |
| Query or write tables with distributed SQL | Trino |
| Access tables from Hive | Hive |
| Inspect tables in a lakehouse management service | Amoro |
Compatibility Matrix
For bundled connectors, match the engine version to the connector artifact for your Paimon release. The Flink, Spark, and Hive versions below describe connector modules in this branch; artifact availability depends on the release. Externally maintained integrations have their own release cycle, embedded Paimon version, and feature limits.
| Integration | Version selection | Access to Paimon tables |
|---|---|---|
| Flink | 1.16–1.20 and 2.0–2.2 | Batch and streaming reads/writes; DDL and row changes have version-specific requirements. |
| Spark | 3.2–3.5, 4.0, and 4.1; match the Scala binary version | Batch reads/writes, DDL, and row changes; streaming requires Spark 3.3+. |
| Hive | 2.1, 2.2, 2.3, 3.1, and 2.1-cdh-6.3 | Batch reads, table creation, and INSERT INTO; writes require MapReduce. |
| Trino | Match the independently released Paimon connector to Trino | Batch reads; supported connectors also provide DDL, inserts, and time travel. See the guide's table-layout limits. |
| Presto | Follow the separate connector's version requirements | See the connector repository for installation and supported operations. |
| StarRocks | Paimon catalogs available from 3.1 | Query existing tables through an external catalog; check the engine release for individual features. |
| Doris | Select a release with the required catalog and reader features | Query existing tables through an external catalog; REST catalog access requires Doris 3.1+. |
A connector's ability to read a table also depends on its data types, file format, merge engine, and enabled features, such as deletion vectors. Check the relevant engine guide before enabling a new table feature in a warehouse shared by several engines.
Streaming Engines
Use Flink for continuous ingestion, change processing, and lookup joins. Use Spark Structured Streaming for micro-batch pipelines. Configure the table's changelog producer for the changes that downstream readers need.
Batch Engines
Use Spark SQL or Flink batch SQL to read a snapshot
and run transformations. Consult the write guides for Spark and
Flink before using overwrite, DELETE, UPDATE, or MERGE INTO:
SQL support and table requirements differ by engine and version.
OLAP Engines
StarRocks and Doris query Paimon through their own external catalogs. Trino and Presto use separately distributed connectors. Configure access to both the catalog and the underlying files, and choose the table read mode for your freshness requirements.
Download
Use the engine downloads for Paimon artifacts and each integration guide for installation. For Trino and Presto, use the connector project's release instructions; its version does not necessarily match this documentation's Paimon version.