Command Line Interface #

PyPaimon provides a command-line interface (CLI) for interacting with Paimon catalogs and tables. The CLI allows you to read data from Paimon tables directly from the command line.

Installation #

The CLI is installed automatically when you install PyPaimon:

pip install pypaimon

After installation, the paimon command will be available in your terminal.

Basic Usage #

Before using the CLI, you need to create a catalog configuration file. By default, the CLI looks for a paimon.yaml file in the current directory.

Create a paimon.yaml file with your catalog settings:

Filesystem Catalog:

metastore: filesystem
warehouse: /path/to/warehouse

REST Catalog:

metastore: rest
uri: http://localhost:8080
warehouse: catalog_name

Usage:

paimon [OPTIONS] COMMAND [ARGS]...

-c, --config PATH: Path to catalog configuration file (default: paimon.yaml)
--help: Show help message and exit

Table Commands #

Table Read #

Read data from a Paimon table and display it in a tabular format.

paimon table read mydb.users

Options:

--select, -s: Select specific columns to read (comma-separated)
--where, -w: Filter condition in SQL-like syntax
--limit, -l: Maximum number of results to display (default: 100)

Examples:

# Read with limit
paimon table read mydb.users -l 50

# Read specific columns
paimon table read mydb.users -s id,name,age

# Filter with WHERE clause
paimon table read mydb.users --where "age > 18"

# Combine select, where, and limit
paimon table read mydb.users -s id,name -w "age >= 20 AND city = 'Beijing'" -l 50

WHERE Operators

The --where option supports SQL-like filter expressions:

Operator	Example
`=`, `!=`, `<>`	`name = 'Alice'`
`<`, `<=`, `>`, `>=`	`age > 18`
`IS NULL`, `IS NOT NULL`	`deleted_at IS NULL`
`IN (...)`, `NOT IN (...)`	`status IN ('active', 'pending')`
`BETWEEN ... AND ...`	`age BETWEEN 20 AND 30`
`LIKE`	`name LIKE 'A%'`

Multiple conditions can be combined with AND and OR (AND has higher precedence). Parentheses are supported for grouping:

# AND condition
paimon table read mydb.users -w "age >= 20 AND age <= 30"

# OR condition
paimon table read mydb.users -w "city = 'Beijing' OR city = 'Shanghai'"

# Parenthesized grouping
paimon table read mydb.users -w "(age > 18 OR name = 'Bob') AND city = 'Beijing'"

# IN list
paimon table read mydb.users -w "city IN ('Beijing', 'Shanghai', 'Hangzhou')"

# BETWEEN
paimon table read mydb.users -w "age BETWEEN 25 AND 35"

# LIKE pattern
paimon table read mydb.users -w "name LIKE 'A%'"

# IS NULL / IS NOT NULL
paimon table read mydb.users -w "email IS NOT NULL"

Literal values are automatically cast to the appropriate Python type based on the table schema (e.g., INT fields cast to int, DOUBLE to float).

Output:

 id    name  age      city
  1   Alice   25   Beijing
  2     Bob   30  Shanghai
  3 Charlie   35 Guangzhou
  4   David   28  Shenzhen
  5     Eve   32  Hangzhou

Table Get #

Get and display table schema information in JSON format. The output format is the same as the schema JSON format used in table create, making it easy to export and reuse table schemas.

paimon table get mydb.users

Output:

{
  "fields": [
    {"id": 0, "name": "user_id", "type": "BIGINT"},
    {"id": 1, "name": "username", "type": "STRING"},
    {"id": 2, "name": "email", "type": "STRING"},
    {"id": 3, "name": "age", "type": "INT"},
    {"id": 4, "name": "city", "type": "STRING"},
    {"id": 5, "name": "created_at", "type": "TIMESTAMP"},
    {"id": 6, "name": "is_active", "type": "BOOLEAN"}
  ],
  "partitionKeys": ["city"],
  "primaryKeys": ["user_id"],
  "options": {
    "bucket": "4",
    "changelog-producer": "input"
  },
  "comment": "User information table"
}

Note: The output JSON can be saved to a file and used directly with the table create command to recreate the table structure.

Table Snapshot #

Get and display the latest snapshot information of a Paimon table in JSON format. The snapshot contains metadata about the current state of the table.

paimon table snapshot mydb.users

Output:

{
  "version": 3,
  "id": 5,
  "schemaId": 1,
  "baseManifestList": "manifest-list-5-base-...",
  "deltaManifestList": "manifest-list-5-delta-...",
  "changelogManifestList": null,
  "totalRecordCount": 1000,
  "deltaRecordCount": 100,
  "changelogRecordCount": null,
  "commitUser": "user-123",
  "commitIdentifier": 1709123456789,
  "commitKind": "APPEND",
  "timeMillis": 1709123456789,
  "watermark": null,
  "statistics": null,
  "nextRowId": null
}

Table Create #

Create a new Paimon table with a schema defined in a JSON file. The schema JSON format is the same as the output from table get, ensuring consistency and easy schema reuse.

Options:

--schema, -s: Path to schema JSON file - Required
--ignore-if-exists, -i: Do not raise error if table already exists

The schema JSON file follows the same format as output by table get:

Field Properties:

id: Field ID (integer, typically starts from 0) - Required
name: Field name - Required
type: Field data type (e.g., INT, BIGINT, STRING, TIMESTAMP, DECIMAL(10,2)) - Required
description: Optional field description

Schema Properties:

fields: List of field definitions - Required
partitionKeys: List of partition key column names
primaryKeys: List of primary key column names
options: Table options as key-value pairs
comment: Table comment

Example Workflow:

Export schema from an existing table:

paimon table get mydb.users > users_schema.json

Create a new table with the same schema:

paimon table create mydb.users_copy --schema users_schema.json

Table Import #

Import data from CSV or JSON files into an existing Paimon table. This is useful for bulk loading data from external sources.

Options:

--input, -i: Path to input file (CSV or JSON format) - Required

Supported Formats:

CSV (.csv): Comma-separated values file
JSON (.json): JSON file with array of objects format

Import from CSV #

The CSV file should have:

A header row with column names matching the table schema
Data types compatible with the table columns

id,name,age,city
1,Alice,25,Beijing
2,Bob,30,Shanghai
3,Charlie,35,Guangzhou

Output:

Successfully imported 3 rows into 'mydb.users'.

Import from JSON #

The JSON file should be an array of objects with keys matching the table column names.

[
  {"id": 1, "name": "Alice", "age": 25, "city": "Beijing"},
  {"id": 2, "name": "Bob", "age": 30, "city": "Shanghai"},
  {"id": 3, "name": "Charlie", "age": 35, "city": "Guangzhou"}
]

Output:

Successfully imported 3 rows into 'mydb.users'.

Important Notes #

The target table must exist before importing data
Column names in the file must match the table schema
Data types should be compatible with the table schema
The import operation appends data to the existing table

Table Rename #

Rename a table in the catalog. Both source and target must be specified in database.table format.

paimon table rename mydb.old_name mydb.new_name

Output:

Table 'mydb.old_name' renamed to 'mydb.new_name' successfully.

Note: Both filesystem and REST catalogs support table rename. For filesystem catalogs, the rename is performed by renaming the underlying table directory.

Table Drop #

Drop a table from the catalog. This will permanently delete the table and all its data.

Options:

--ignore-if-not-exists, -i: Do not raise error if table does not exist

paimon table drop mydb.old_table

Output:

Table 'mydb.old_table' dropped successfully.

Warning: This operation cannot be undone. All data in the table will be permanently deleted.

Table Alter #

Alter a table’s schema or options. This command supports multiple sub-commands for different types of schema changes.

Basic Syntax #

paimon table alter DATABASE.TABLE [--ignore-if-not-exists] SUBCOMMAND [OPTIONS]

Global Options:

--ignore-if-not-exists, -i: Do not raise error if table does not exist

Set Option #

Set a table option (key-value pair):

paimon table alter mydb.users set-option -k snapshot.num-retained-max -v 10

Remove Option #

Remove a table option:

paimon table alter mydb.users remove-option -k snapshot.num-retained-max

Add Column #

Add a new column to the table:

Example:

paimon table alter mydb.users add-column -n email -t STRING -c "User email address"

Example with position (first):

paimon table alter mydb.users add-column -n row_id -t BIGINT --first

Example with position (after):

paimon table alter mydb.users add-column -n email -t STRING --after name

Drop Column #

Drop a column from the table:

paimon table alter mydb.users drop-column -n email

Rename Column #

Rename an existing column:

paimon table alter mydb.users rename-column -n username -m user_name

Alter Column #

Alter an existing column’s type, comment, or position. Multiple changes can be specified in a single command.

Change Column Type:

paimon table alter mydb.users alter-column -n age -t BIGINT

Change Column Comment:

paimon table alter mydb.users alter-column -n age -c 'User age in years'

Change Column Position:

paimon table alter mydb.users alter-column -n age --first

paimon table alter mydb.users alter-column -n age --after name

Multiple changes in one command:

paimon table alter mydb.users alter-column -n age -t BIGINT -c 'User age in years'

Update Comment #

paimon table alter mydb.users update-comment -c "Updated user information table"

Database Commands #

DB Get #

Get and display database information in JSON format.

paimon db get mydb

Output:

{
  "name": "mydb",
  "options": {}
}

DB Create #

Create a new database.

# Create a simple database
paimon db create mydb

# Create with properties
paimon db create mydb -p '{"key1": "value1", "key2": "value2"}'

# Create and ignore if already exists
paimon db create mydb -i

DB Drop #

Drop an existing database.

# Drop a database
paimon db drop mydb

# Drop and ignore if not exists
paimon db drop mydb -i

# Drop with all tables (cascade)
paimon db drop mydb --cascade

DB Alter #

Alter database properties by setting or removing properties.

# Set properties
paimon db alter mydb --set '{"key1": "value1", "key2": "value2"}'

# Remove properties
paimon db alter mydb --remove key1 key2

# Set and remove properties in one command
paimon db alter mydb --set '{"key1": "new_value"}' --remove key2

DB List Tables #

List all tables in a database.

paimon db list-tables mydb

Output:

orders
products
users

Catalog Commands #

Catalog List DBs #

List all databases in the catalog.

paimon catalog list-dbs

Output:

default
mydb
analytics