@Public public interface StreamTableCommit extends TableCommit
TableCommit
for stream processing. You can use this class to commit multiple times.StreamWriteBuilder
Modifier and Type | Method and Description |
---|---|
void |
commit(long commitIdentifier,
List<CommitMessage> commitMessages)
Create a new commit.
|
int |
filterAndCommit(Map<Long,List<CommitMessage>> commitIdentifiersAndMessages)
Filter out all
List<CommitMessage> which have been committed and commit the remaining
ones. |
abort
close
void commit(long commitIdentifier, List<CommitMessage> commitMessages)
1. Snapshot expiration may occur according to three options:
2. Partition expiration may occur according to 'partition.expiration-time'. The partition check is expensive, so all partitions are not checked every time when invoking this method. The check frequency is controlled by 'partition.expiration-check-interval'. Partition expiration will create an 'OVERWRITE' snapshot.
Compared to filterAndCommit(java.util.Map<java.lang.Long, java.util.List<org.apache.paimon.table.sink.CommitMessage>>)
, this method does not check if
commitIdentifier
has been committed, so this method might be faster. Please only use
this method if you can make sure that the commitIdentifier
hasn't been committed
before.
commitIdentifier
- Committed transaction ID, can start from 0. If there are multiple
commits, please increment this ID.commitMessages
- commit messages from table writeStreamTableWrite.prepareCommit(boolean, long)
int filterAndCommit(Map<Long,List<CommitMessage>> commitIdentifiersAndMessages)
List<CommitMessage>
which have been committed and commit the remaining
ones.
Compared to commit(long, java.util.List<org.apache.paimon.table.sink.CommitMessage>)
, this method will first check if a commitIdentifier
has been committed, so this method might be slower. A common usage of this
method is to retry the commit process after a failure.
commitIdentifiersAndMessages
- a map containing all CommitMessage
s in question.
The key is the commitIdentifier
.List<CommitMessage>
committed.Copyright © 2023–2024 The Apache Software Foundation. All rights reserved.