org.apache.paimon.flink.sink

Class CommitterOperatorFactory<CommitT,GlobalCommitT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperatorFactory<CommitT>

org.apache.paimon.flink.sink.CommitterOperatorFactory<CommitT,GlobalCommitT>

All Implemented Interfaces:

Serializable, org.apache.flink.streaming.api.operators.OneInputStreamOperatorFactory<CommitT,CommitT>, org.apache.flink.streaming.api.operators.StreamOperatorFactory<CommitT>, org.apache.flink.streaming.runtime.tasks.ProcessingTimeServiceAware

public class CommitterOperatorFactory<CommitT,GlobalCommitT>
extends org.apache.flink.streaming.api.operators.AbstractStreamOperatorFactory<CommitT>
implements org.apache.flink.streaming.api.operators.OneInputStreamOperatorFactory<CommitT,CommitT>

StreamOperatorFactory for CommitterOperator.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected NavigableMap<Long,GlobalCommitT>	committablesPerCheckpoint Group the committable by the checkpoint id.
protected CommittableStateManager<GlobalCommitT>	committableStateManager
protected Committer<CommitT,GlobalCommitT>	committer Aggregate committables to global committables and commit the global committables to the external system.
protected Committer.Factory<CommitT,GlobalCommitT>	committerFactory
protected Long	endInputWatermark
protected boolean	forceSingleParallelism Whether to check the parallelism while runtime.
protected String	initialCommitUser This commitUser is valid only for new jobs.
protected boolean	streamingCheckpointEnabled

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperatorFactory

chainingStrategy, processingTimeService

Constructor Summary

Constructors

Constructor and Description
CommitterOperatorFactory(boolean streamingCheckpointEnabled, boolean forceSingleParallelism, String initialCommitUser, Committer.Factory<CommitT,GlobalCommitT> committerFactory, CommittableStateManager<GlobalCommitT> committableStateManager)
CommitterOperatorFactory(boolean streamingCheckpointEnabled, boolean forceSingleParallelism, String initialCommitUser, Committer.Factory<CommitT,GlobalCommitT> committerFactory, CommittableStateManager<GlobalCommitT> committableStateManager, Long endInputWatermark)

Method Summary

Modifier and Type	Method and Description
<T extends org.apache.flink.streaming.api.operators.StreamOperator<CommitT>> T	createStreamOperator(org.apache.flink.streaming.api.operators.StreamOperatorParameters<CommitT> parameters)
Class<? extends org.apache.flink.streaming.api.operators.StreamOperator>	getStreamOperatorClass(ClassLoader classLoader)

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperatorFactory

getChainingStrategy, getMailboxExecutor, setChainingStrategy, setMailboxExecutor, setProcessingTimeService

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperatorFactory

getChainingStrategy, getOperatorAttributes, isInputTypeConfigurable, isLegacySource, isOutputTypeConfigurable, isStreamSource, setChainingStrategy, setInputType, setOutputType

Field Detail

streamingCheckpointEnabled

protected final boolean streamingCheckpointEnabled

forceSingleParallelism

protected final boolean forceSingleParallelism

Whether to check the parallelism while runtime.

initialCommitUser

protected final String initialCommitUser

This commitUser is valid only for new jobs. After the job starts, this commitUser will be recorded into the states of write and commit operators. When the job restarts, commitUser will be recovered from states and this value is ignored.

committablesPerCheckpoint

protected final NavigableMap<Long,GlobalCommitT> committablesPerCheckpoint

Group the committable by the checkpoint id.

committerFactory

protected final Committer.Factory<CommitT,GlobalCommitT> committerFactory

committableStateManager

protected final CommittableStateManager<GlobalCommitT> committableStateManager

committer

protected Committer<CommitT,GlobalCommitT> committer

Aggregate committables to global committables and commit the global committables to the external system.

endInputWatermark

protected final Long endInputWatermark

Constructor Detail

CommitterOperatorFactory

public CommitterOperatorFactory(boolean streamingCheckpointEnabled,
                                boolean forceSingleParallelism,
                                String initialCommitUser,
                                Committer.Factory<CommitT,GlobalCommitT> committerFactory,
                                CommittableStateManager<GlobalCommitT> committableStateManager)

CommitterOperatorFactory

public CommitterOperatorFactory(boolean streamingCheckpointEnabled,
                                boolean forceSingleParallelism,
                                String initialCommitUser,
                                Committer.Factory<CommitT,GlobalCommitT> committerFactory,
                                CommittableStateManager<GlobalCommitT> committableStateManager,
                                Long endInputWatermark)

Method Detail

createStreamOperator

public <T extends org.apache.flink.streaming.api.operators.StreamOperator<CommitT>> T createStreamOperator(org.apache.flink.streaming.api.operators.StreamOperatorParameters<CommitT> parameters)

Specified by:

createStreamOperator in interface org.apache.flink.streaming.api.operators.StreamOperatorFactory<CommitT>

getStreamOperatorClass

public Class<? extends org.apache.flink.streaming.api.operators.StreamOperator> getStreamOperatorClass(ClassLoader classLoader)

Specified by:

getStreamOperatorClass in interface org.apache.flink.streaming.api.operators.StreamOperatorFactory<CommitT>