org.apache.paimon.table.sink

Interface BatchWriteBuilder

All Superinterfaces:

Serializable, WriteBuilder

All Known Implementing Classes:

BatchWriteBuilderImpl

@Public
public interface BatchWriteBuilder
extends WriteBuilder

An interface for building the BatchTableWrite and BatchTableCommit.

Example of distributed batch writing:

 // 1. Create a WriteBuilder (Serializable)
 Table table = catalog.getTable(...);
 WriteBuilder builder = table.newWriteBuilder();

 // 2. Write records in distributed tasks
 BatchTableWrite write = builder.newWrite();
 write.write(...);
 write.write(...);
 write.write(...);
 List<CommitMessage> messages = write.prepareCommit();

 // 3. Collect all CommitMessages to a global node and commit
 BatchTableCommit commit = builder.newCommit();
 commit.commit(allCommitMessages());
 

Since:

0.4.0

Field Summary

Fields

Modifier and Type	Field and Description
static long	COMMIT_IDENTIFIER

Method Summary

Modifier and Type	Method and Description
BatchTableCommit	newCommit() Create a TableCommit to commit CommitMessages.
BatchTableWrite	newWrite() Create a TableWrite to write InternalRows.
default BatchWriteBuilder	withOverwrite() Overwrite writing, same as the 'INSERT OVERWRITE' semantics of SQL.
BatchWriteBuilder	withOverwrite(Map<String,String> staticPartition) Overwrite writing, same as the 'INSERT OVERWRITE T PARTITION (...)' semantics of SQL.

Methods inherited from interface org.apache.paimon.table.sink.WriteBuilder

newWriteSelector, rowType, tableName

Field Detail

COMMIT_IDENTIFIER

static final long COMMIT_IDENTIFIER

See Also:

Constant Field Values

Method Detail

withOverwrite

default BatchWriteBuilder withOverwrite()

Overwrite writing, same as the 'INSERT OVERWRITE' semantics of SQL.

withOverwrite

BatchWriteBuilder withOverwrite(@Nullable
                                Map<String,String> staticPartition)

Overwrite writing, same as the 'INSERT OVERWRITE T PARTITION (...)' semantics of SQL.

newWrite

BatchTableWrite newWrite()

Create a TableWrite to write InternalRows.

Specified by:

newWrite in interface WriteBuilder

newCommit

BatchTableCommit newCommit()

Create a TableCommit to commit CommitMessages.

Specified by:

newCommit in interface WriteBuilder