org.apache.paimon.flink.source

Class BucketUnawareCompactSource

java.lang.Object

org.apache.paimon.flink.source.AbstractNonCoordinatedSource<UnawareAppendCompactionTask>

org.apache.paimon.flink.source.BucketUnawareCompactSource

All Implemented Interfaces:

Serializable, org.apache.flink.api.connector.source.Source<UnawareAppendCompactionTask,SimpleSourceSplit,NoOpEnumState>, org.apache.flink.api.connector.source.SourceReaderFactory<UnawareAppendCompactionTask,SimpleSourceSplit>

public class BucketUnawareCompactSource
extends AbstractNonCoordinatedSource<UnawareAppendCompactionTask>

Source for unaware-bucket Compaction.

Note: The function is the source of unaware-bucket compactor coordinator. It will read the latest snapshot continuously by compactionCoordinator, and generate new compaction tasks. The source is used in unaware-bucket compaction job (both stand-alone and write-combined). Besides, we don't need to save state in this source, it will invoke a full scan when starting up, and scan continuously for the following snapshot.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BucketUnawareCompactSource.BucketUnawareCompactSourceReader BucketUnawareCompactSourceReader.

Constructor Summary

Constructors

Constructor and Description
BucketUnawareCompactSource(FileStoreTable table, boolean isStreaming, long scanInterval, Predicate filter)

Method Summary

Modifier and Type	Method and Description
static org.apache.flink.streaming.api.datastream.DataStreamSource<UnawareAppendCompactionTask>	buildSource(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env, BucketUnawareCompactSource source, String tableIdentifier)
org.apache.flink.api.connector.source.SourceReader<UnawareAppendCompactionTask,SimpleSourceSplit>	createReader(org.apache.flink.api.connector.source.SourceReaderContext readerContext)
org.apache.flink.api.connector.source.Boundedness	getBoundedness()

Methods inherited from class org.apache.paimon.flink.source.AbstractNonCoordinatedSource

createEnumerator, getEnumeratorCheckpointSerializer, getSplitSerializer, restoreEnumerator

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BucketUnawareCompactSource

public BucketUnawareCompactSource(FileStoreTable table,
                                  boolean isStreaming,
                                  long scanInterval,
                                  @Nullable
                                  Predicate filter)

Method Detail

getBoundedness

public org.apache.flink.api.connector.source.Boundedness getBoundedness()

createReader

public org.apache.flink.api.connector.source.SourceReader<UnawareAppendCompactionTask,SimpleSourceSplit> createReader(org.apache.flink.api.connector.source.SourceReaderContext readerContext)
                                                                                                               throws Exception

Throws:

Exception

buildSource

public static org.apache.flink.streaming.api.datastream.DataStreamSource<UnawareAppendCompactionTask> buildSource(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env,
                                                                                                                  BucketUnawareCompactSource source,
                                                                                                                  String tableIdentifier)