public class BucketUnawareCompactSource extends org.apache.flink.streaming.api.functions.source.RichSourceFunction<UnawareAppendCompactionTask>
Note: The function is the source function of unaware-bucket compactor coordinator. It will read the latest snapshot continuously by compactionCoordinator, and generate new compaction tasks. The source function is used in unaware-bucket compaction job (both stand-alone and write-combined). Besides, we don't need to save state in this function, it will invoke a full scan when starting up, and scan continuously for the following snapshot.
Constructor and Description |
---|
BucketUnawareCompactSource(FileStoreTable table,
boolean isStreaming,
long scanInterval,
Predicate filter) |
Modifier and Type | Method and Description |
---|---|
static org.apache.flink.streaming.api.datastream.DataStreamSource<UnawareAppendCompactionTask> |
buildSource(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env,
BucketUnawareCompactSource source,
boolean streaming,
String tableIdentifier) |
void |
cancel() |
void |
open(org.apache.flink.configuration.Configuration parameters) |
void |
run(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<UnawareAppendCompactionTask> sourceContext) |
close, getIterationRuntimeContext, getRuntimeContext, setRuntimeContext
public BucketUnawareCompactSource(FileStoreTable table, boolean isStreaming, long scanInterval, @Nullable Predicate filter)
public void open(org.apache.flink.configuration.Configuration parameters) throws Exception
open
in interface org.apache.flink.api.common.functions.RichFunction
open
in class org.apache.flink.api.common.functions.AbstractRichFunction
Exception
public void run(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<UnawareAppendCompactionTask> sourceContext) throws Exception
Exception
public void cancel()
public static org.apache.flink.streaming.api.datastream.DataStreamSource<UnawareAppendCompactionTask> buildSource(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env, BucketUnawareCompactSource source, boolean streaming, String tableIdentifier)
Copyright © 2023–2024 The Apache Software Foundation. All rights reserved.