This documentation is for an unreleased version of Apache Paimon. We recommend you use the latest stable version.

Overview #

Compatibility Matrix #

Engine Version Batch Read Batch Write Create Table Alter Table Streaming Write Streaming Read Batch Overwrite DELETE & UPDATE MERGE INTO
Flink 1.15 - 1.19 ✅(1.17+) ✅(1.17+)
Spark 3.1 - 3.5 ✅(3.3+) ✅(3.3+) ✅(3.3+) ✅(3.2+) ✅(3.2+) ✅(3.2+)
Hive 2.1 - 3.1
Trino 420 - 439 ✅(427+) ✅(427+) ✅(427+)
Presto 0.236 - 0.280
StarRocks 3.1+
Doris 2.0.6+

Streaming Engines #

Flink is the most comprehensive streaming computing engine that is widely used for data CDC ingestion and the construction of streaming pipelines.

Recommended version is Flink 1.17.2.

Spark Streaming #

You can also use Spark Streaming to build a streaming pipeline. Spark’s schema evolution capability will be better implemented, but you must accept the mechanism of mini-batch.

Batch Engines #

Spark Batch #

Spark Batch is the most widely used batch computing engine.

Recommended version is Spark 3.4.3.

Flink Batch is also available, which can make your pipeline more integrated with streaming and batch unified.

OLAP Engines #

StarRocks #

StarRocks is the most recommended OLAP engine with the most advanced integration.

Recommended version is StarRocks 3.2.6.

Other OLAP #

You can also use Doris and Trino and Presto, or, you can just use Spark, Flink and Hive to query Paimon tables.

Download #

Download Link

Edit This Page
Copyright © 2024 The Apache Software Foundation. Apache Paimon, Paimon, and its feather logo are trademarks of The Apache Software Foundation.