This documentation is for an unreleased version of Apache Paimon. We recommend you use the latest stable version.

Overview #

Compatibility Matrix #

Engine	Version	Batch Read	Batch Write	Create Table	Alter Table	Streaming Write	Streaming Read	Batch Overwrite	DELETE & UPDATE	MERGE INTO	Time Travel
Flink	1.15 - 1.19	✅	✅	✅	✅(1.17+)	✅	✅	✅	✅(1.17+)	❌	✅
Spark	3.1 - 3.5	✅	✅(3.2+)	✅	✅	✅(3.3+)	✅(3.3+)	✅(3.2+)	✅(3.2+)	✅(3.2+)	✅(3.3+)
Hive	2.1 - 3.1	✅	✅	✅	❌	❌	❌	❌	❌	❌	✅
Trino	420 - 439	✅	✅(427+)	✅(427+)	✅(427+)	❌	❌	❌	❌	❌	✅
Presto	0.236 - 0.280	✅	❌	✅	✅	❌	❌	❌	❌	❌	❌
StarRocks	3.1+	✅	❌	❌	❌	❌	❌	❌	❌	❌	✅
Doris	2.0.6+	✅	❌	❌	❌	❌	❌	❌	❌	❌	✅

Streaming Engines #

Flink Streaming #

Flink is the most comprehensive streaming computing engine that is widely used for data CDC ingestion and the construction of streaming pipelines.

Recommended version is Flink 1.17.2.

Spark Streaming #

You can also use Spark Streaming to build a streaming pipeline. Spark’s schema evolution capability will be better implemented, but you must accept the mechanism of mini-batch.

Batch Engines #

Spark Batch #

Spark Batch is the most widely used batch computing engine.

Recommended version is Spark 3.4.3.

Flink Batch #

Flink Batch is also available, which can make your pipeline more integrated with streaming and batch unified.

OLAP Engines #

StarRocks #

StarRocks is the most recommended OLAP engine with the most advanced integration.

Recommended version is StarRocks 3.2.6.

Other OLAP #

You can also use Doris and Trino and Presto, or, you can just use Spark, Flink and Hive to query Paimon tables.

Download #

Download Link