Learn Paimon
Learn how to choose a table design, follow changes from a writer to storage, and diagnose the files that remain after an update or compaction. These guides connect Paimon's concepts through small examples; the linked reference pages cover individual features in detail.
Choose a Learning Path
| Your question | Start here | What you will learn |
|---|---|---|
| Which table fits my workload? | Scenario Guide | Choose row semantics, distribution, and streaming output separately. |
| How do I store payloads and update AI features? | AI Data Pipelines | Combine BLOBs, column updates, vector indexes, and Python processing. |
| What does a write actually change on disk? | Understand Files | Trace schema creation, inserts, deletes, compaction, and expiration. |
| Why do I have so many small files? | Streaming Writes and Small Files | Separate write fragmentation, compaction backlog, and retained history. |
For a first read, follow the pages in that order. If you already operate a table, start with the file walkthrough or the small-file diagnosis table.
Before You Begin
Read Basic Concepts for snapshots, manifests, partitions, and buckets. To run examples, configure a Paimon catalog using the Flink quick start, Spark quick start, or PyPaimon quick start. Each guide identifies the engine and assumptions for its examples.
Keep These Distinctions in Mind
- Logical rows and physical records: a primary-key query can return one row while several versions of that key remain in data files.
- Commit and compaction: a commit publishes a snapshot; compaction reorganizes files and publishes its result through a commit.
- Compaction and expiration: retiring a file from the current snapshot does not immediately remove it from storage. Retained history can still need it.
- Storage and indexes: storing a vector or text value does not automatically build a search index. Index coverage has its own lifecycle.
Continue with the References
Use Primary-Key Tables for merge and changelog behavior, Append Tables for batch and ordered append layouts, and Multimodal Tables for column storage and search. For ongoing operations, see Maintenance and System Tables.