org.apache.paimon.flink.source.assigners

Class AlignedSplitAssigner

java.lang.Object

org.apache.paimon.flink.source.assigners.AlignedSplitAssigner

All Implemented Interfaces:

SplitAssigner

public class AlignedSplitAssigner
extends Object
implements SplitAssigner

Splits are allocated at the granularity of snapshots. When the splits of the current snapshot are not fully allocated and checkpoint are not triggered, the next snapshot will not be allocated.

Constructor Summary

Constructors

Constructor and Description
AlignedSplitAssigner()

Method Summary

Modifier and Type	Method and Description
void	addSplit(int subtask, FileStoreSourceSplit splits) Add one split of a specified subtask to the assigner.
void	addSplitsBack(int suggestedTask, List<FileStoreSourceSplit> splits) Adds a set of splits to this assigner.
List<FileStoreSourceSplit>	getNext(int subtask, String hostname) Gets the next split.
Optional<Long>	getNextSnapshotId(int subtask) Gets the snapshot id of the next split.
boolean	isAligned()
int	numberOfRemainingSplits() Gets the current number of remaining splits.
int	remainingSnapshots()
Collection<FileStoreSourceSplit>	remainingSplits() Gets the remaining splits that this assigner has pending.
void	removeFirst()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AlignedSplitAssigner

public AlignedSplitAssigner()

Method Detail

getNext

public List<FileStoreSourceSplit> getNext(int subtask,
                                          @Nullable
                                          String hostname)

Description copied from interface: SplitAssigner

Gets the next split.

When this method returns an empty Optional, then the set of splits is assumed to be done and the source will finish once the readers finished their current splits.

Specified by:

getNext in interface SplitAssigner

addSplit

public void addSplit(int subtask,
                     FileStoreSourceSplit splits)

Description copied from interface: SplitAssigner

Add one split of a specified subtask to the assigner.

Specified by:

addSplit in interface SplitAssigner

addSplitsBack

public void addSplitsBack(int suggestedTask,
                          List<FileStoreSourceSplit> splits)

Description copied from interface: SplitAssigner

Adds a set of splits to this assigner. This happens for example when some split processing failed and the splits need to be re-added, or when new splits got discovered.

Specified by:

addSplitsBack in interface SplitAssigner

remainingSplits

public Collection<FileStoreSourceSplit> remainingSplits()

Description copied from interface: SplitAssigner

Gets the remaining splits that this assigner has pending.

Specified by:

remainingSplits in interface SplitAssigner

getNextSnapshotId

public Optional<Long> getNextSnapshotId(int subtask)

Description copied from interface: SplitAssigner

Gets the snapshot id of the next split.

Specified by:

getNextSnapshotId in interface SplitAssigner

numberOfRemainingSplits

public int numberOfRemainingSplits()

Description copied from interface: SplitAssigner

Gets the current number of remaining splits. This method should be guaranteed to be thread-safe.

Specified by:

numberOfRemainingSplits in interface SplitAssigner

isAligned

public boolean isAligned()

remainingSnapshots

public int remainingSnapshots()

removeFirst

public void removeFirst()