org.apache.paimon.table.sink

Interface StreamWriteBuilder

All Superinterfaces:

Serializable, WriteBuilder

All Known Implementing Classes:

StreamWriteBuilderImpl

@Public
public interface StreamWriteBuilder
extends WriteBuilder

An interface for building the StreamTableWrite and StreamTableCommit.

Key points to achieve exactly-once consistency:

• CommitUser represents a user. A user can commit multiple times. In distributed processing, you are expected to use the same commitUser.
• Different applications need to use different commitUsers.
• The commitIdentifier of StreamTableWrite and StreamTableCommit needs to be consistent, and the id needs to be incremented for the next committing.
• When a failure occurs, if you still have uncommitted CommitMessages, please retry with StreamTableCommit.filterAndCommit(java.util.Map<java.lang.Long, java.util.List<org.apache.paimon.table.sink.CommitMessage>>) to exclude the committed messages by commitIdentifier.

Since:

0.4.0

Method Summary

Modifier and Type	Method and Description
String	commitUser() Get commit user, set by withCommitUser(java.lang.String).
StreamTableCommit	newCommit() Create a TableCommit to commit CommitMessages.
StreamTableWrite	newWrite() Create a TableWrite to write InternalRows.
StreamWriteBuilder	withCommitUser(String commitUser) Set commit user, the default value is a random UUID.

Methods inherited from interface org.apache.paimon.table.sink.WriteBuilder

newWriteSelector, rowType, tableName

Method Detail

commitUser

String commitUser()

Get commit user, set by withCommitUser(java.lang.String).

withCommitUser

StreamWriteBuilder withCommitUser(String commitUser)

Set commit user, the default value is a random UUID. The commit user used by TableWrite and TableCommit must be the same, otherwise there will be some conflicts.

newWrite

StreamTableWrite newWrite()

Create a TableWrite to write InternalRows.

Specified by:

newWrite in interface WriteBuilder

newCommit

StreamTableCommit newCommit()

Create a TableCommit to commit CommitMessages.

Specified by:

newCommit in interface WriteBuilder