org.apache.paimon.table.sink

Interface BatchTableCommit

All Superinterfaces:

AutoCloseable, TableCommit

All Known Subinterfaces:

InnerTableCommit

All Known Implementing Classes:

TableCommitImpl

@Public
public interface BatchTableCommit
extends TableCommit

A TableCommit for batch processing. Recommended for one-time committing.

Since:

0.4.0

Method Summary

Modifier and Type	Method and Description
void	commit(List<CommitMessage> commitMessages) Create a new commit.
void	compactManifests() Compact the manifest entries.
void	truncatePartitions(List<Map<String,String>> partitionSpecs) Truncate partitions, like normal commit(java.util.List<org.apache.paimon.table.sink.CommitMessage>), files are not immediately deleted, they are only logically deleted and will be deleted after the snapshot expires.
void	truncateTable() Truncate table, like normal commit(java.util.List<org.apache.paimon.table.sink.CommitMessage>), files are not immediately deleted, they are only logically deleted and will be deleted after the snapshot expires.
void	updateStatistics(Statistics statistics) Commit new statistics.

Methods inherited from interface org.apache.paimon.table.sink.TableCommit

abort

Methods inherited from interface java.lang.AutoCloseable

close

Method Detail

commit

void commit(List<CommitMessage> commitMessages)

Create a new commit. One commit may generate up to two snapshots, one for adding new files and the other for compaction. There will be some expiration policies after commit:

1. Snapshot expiration may occur according to three options:

• 'snapshot.time-retained': The maximum time of completed snapshots to retain.
• 'snapshot.num-retained.min': The minimum number of completed snapshots to retain.
• 'snapshot.num-retained.max': The maximum number of completed snapshots to retain.

2. Partition expiration may occur according to 'partition.expiration-time'. The partition check is expensive, so all partitions are not checked every time when invoking this method. The check frequency is controlled by 'partition.expiration-check-interval'. Partition expiration will create an 'OVERWRITE' snapshot.

Parameters:

commitMessages - commit messages from table write

truncateTable

void truncateTable()

Truncate table, like normal commit(java.util.List<org.apache.paimon.table.sink.CommitMessage>), files are not immediately deleted, they are only logically deleted and will be deleted after the snapshot expires.

truncatePartitions

void truncatePartitions(List<Map<String,String>> partitionSpecs)

Truncate partitions, like normal commit(java.util.List<org.apache.paimon.table.sink.CommitMessage>), files are not immediately deleted, they are only logically deleted and will be deleted after the snapshot expires.

updateStatistics

void updateStatistics(Statistics statistics)

Commit new statistics. Generates a snapshot with Snapshot.CommitKind.ANALYZE.

compactManifests

void compactManifests()

Compact the manifest entries. Generates a snapshot with Snapshot.CommitKind.COMPACT.