org.apache.paimon.flink.sink

Class StoreCommitter

java.lang.Object

org.apache.paimon.flink.sink.StoreCommitter

All Implemented Interfaces:

AutoCloseable, Committer<Committable,ManifestCommittable>

public class StoreCommitter
extends Object
implements Committer<Committable,ManifestCommittable>

Committer for dynamic store.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.paimon.flink.sink.Committer

Committer.Context, Committer.Factory<CommitT,GlobalCommitT>

Constructor Summary

Constructors

Constructor and Description
StoreCommitter(FileStoreTable table, TableCommit commit, Committer.Context context)

Method Summary

Modifier and Type	Method and Description
boolean	allowLogOffsetDuplicate()
void	close()
ManifestCommittable	combine(long checkpointId, long watermark, List<Committable> committables) Compute an aggregated committable from a list of committables.
ManifestCommittable	combine(long checkpointId, long watermark, ManifestCommittable manifestCommittable, List<Committable> committables)
void	commit(List<ManifestCommittable> committables) Commits the given GlobalCommitT.
int	filterAndCommit(List<ManifestCommittable> globalCommittables, boolean checkAppendFiles) Filter out all GlobalCommitT which have committed, and commit the remaining GlobalCommitT.
boolean	forceCreatingSnapshot()
CommitterMetrics	getCommitterMetrics()
Map<Long,List<Committable>>	groupByCheckpoint(Collection<Committable> committables)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.paimon.flink.sink.Committer

createContext, filterAndCommit

Constructor Detail

StoreCommitter

public StoreCommitter(FileStoreTable table,
                      TableCommit commit,
                      Committer.Context context)

Method Detail

getCommitterMetrics

@VisibleForTesting
public CommitterMetrics getCommitterMetrics()

forceCreatingSnapshot

public boolean forceCreatingSnapshot()

Specified by:

forceCreatingSnapshot in interface Committer<Committable,ManifestCommittable>

combine

public ManifestCommittable combine(long checkpointId,
                                   long watermark,
                                   List<Committable> committables)

Description copied from interface: Committer

Compute an aggregated committable from a list of committables.

Specified by:

combine in interface Committer<Committable,ManifestCommittable>

combine

public ManifestCommittable combine(long checkpointId,
                                   long watermark,
                                   ManifestCommittable manifestCommittable,
                                   List<Committable> committables)

Specified by:

combine in interface Committer<Committable,ManifestCommittable>

commit

public void commit(List<ManifestCommittable> committables)
            throws IOException,
                   InterruptedException

Description copied from interface: Committer

Commits the given GlobalCommitT.

Specified by:

commit in interface Committer<Committable,ManifestCommittable>

Throws:

IOException

InterruptedException

filterAndCommit

public int filterAndCommit(List<ManifestCommittable> globalCommittables,
                           boolean checkAppendFiles)

Description copied from interface: Committer

Filter out all GlobalCommitT which have committed, and commit the remaining GlobalCommitT.

Specified by:

filterAndCommit in interface Committer<Committable,ManifestCommittable>

groupByCheckpoint

public Map<Long,List<Committable>> groupByCheckpoint(Collection<Committable> committables)

Specified by:

groupByCheckpoint in interface Committer<Committable,ManifestCommittable>

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception

allowLogOffsetDuplicate

public boolean allowLogOffsetDuplicate()