PyPaimon

PyPaimon is the Python implementation of Apache Paimon. It connects to catalogs, reads and writes lakehouse tables, and makes text, vectors, images, and video available to Python data and training workflows. The core Python API does not require a JDK.

Start with Installation, then run the Quick Start to create a local table, commit a batch, and read it back.

Choose an API

You want to…	Start here	What you control
Work with ordinary Paimon tables	Python API	Catalogs, schemas, predicates, scan splits, and explicit commits
Build a text, vector, or media application	Multimodal API	Table operations, payload reads, search, and row IDs through a compact interface
Process data across workers	Ray Data or Daft	Distributed reads, transforms, and writes
Feed a training loop	PyTorch	Iterable reads, frame decoding, and contiguous windows
Query or inspect a table interactively	SQL or CLI	SQL results, table metadata, tags, and branches

The multimodal API wraps the same catalog and table implementation. It creates data-evolution tables without primary keys and supplies defaults for row tracking, deletion vectors, and BLOB descriptors. Use the Python API when you need to work with other table types or control scan and commit steps directly.

Learn by task

Task	Guides
Connect and define data	Catalogs and tables, data types
Read and write batches	Batch writes, batch reads
Follow new data	Streaming reads and consumers
Retain a dataset version	Tags, branches and rollback
Store and update media	BLOB storage, BLOB object API, data evolution, video frames
Match new data to existing rows	Upsert and merge, Ray joins
Backfill derived features	Ray row IDs and backfills
Retrieve candidates	Vector and full-text search, row IDs
Import robot data	HDF5 and ROSBag, LeRobot, RoboMIND AgileX
Compare training storage	RoboMIND ACT benchmark
Configure storage access	FUSE, PyJindoSDK
Inspect metadata	System tables, CLI query and inspect

Environment Settings

See Installation for virtual environments and package installation.

Build From Source

See Install from source to use this checkout.

Optional Dependencies

See Optional dependencies for file formats, compute frameworks, dataset importers, and their Python version requirements.