org.apache.paimon.flink.sink

Class TableWriteOperator<IN>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.paimon.flink.sink.PrepareCommitOperator<IN,Committable>

org.apache.paimon.flink.sink.TableWriteOperator<IN>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<IN>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OneInputStreamOperator<IN,Committable>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<Committable>, org.apache.flink.streaming.api.operators.StreamOperator<Committable>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.YieldingOperator<Committable>

Direct Known Subclasses:

CdcDynamicBucketWriteOperator, CdcRecordStoreWriteOperator, DynamicBucketRowWriteOperator, RowDataStoreWriteOperator

public abstract class TableWriteOperator<IN>
extends PrepareCommitOperator<IN,Committable>

An abstract class for table write operator.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected FileStoreTable	table
protected StoreSinkWrite	write

Fields inherited from class org.apache.paimon.flink.sink.PrepareCommitOperator

memoryPool

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Constructor and Description
TableWriteOperator(FileStoreTable table, StoreSinkWrite.Provider storeSinkWriteProvider, String initialCommitUser)

Method Summary

Modifier and Type	Method and Description
void	close()
protected abstract boolean	containLogSystem()
protected StoreSinkWriteState	createState(org.apache.flink.runtime.state.StateInitializationContext context, StoreSinkWriteState.StateValueFilter stateFilter)
protected String	getCommitUser(org.apache.flink.runtime.state.StateInitializationContext context)
StoreSinkWrite	getWrite()
void	initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
protected List<Committable>	prepareCommit(boolean waitCompaction, long checkpointId)
void	processRecordAttributes(RecordAttributes recordAttributes)
void	snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)

Methods inherited from class org.apache.paimon.flink.sink.PrepareCommitOperator

endInput, prepareSnapshotPreBarrier, setup

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

finish, getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, open, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, snapshotState, useSplittableTimers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, open, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.Input

processElement, processLatencyMarker, processWatermark, processWatermarkStatus

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Field Detail

table

protected FileStoreTable table

write

protected transient StoreSinkWrite write

Constructor Detail

TableWriteOperator

public TableWriteOperator(FileStoreTable table,
                          StoreSinkWrite.Provider storeSinkWriteProvider,
                          String initialCommitUser)

Method Detail

initializeState

public void initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
                     throws Exception

Specified by:

initializeState in interface org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator

Overrides:

initializeState in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<Committable>

Throws:

Exception

createState

protected StoreSinkWriteState createState(org.apache.flink.runtime.state.StateInitializationContext context,
                                          StoreSinkWriteState.StateValueFilter stateFilter)
                                   throws Exception

Throws:

Exception

getCommitUser

protected String getCommitUser(org.apache.flink.runtime.state.StateInitializationContext context)
                        throws Exception

Throws:

Exception

processRecordAttributes

public void processRecordAttributes(RecordAttributes recordAttributes)
                             throws Exception

Specified by:

processRecordAttributes in interface org.apache.flink.streaming.api.operators.Input<IN>

Overrides:

processRecordAttributes in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<Committable>

Throws:

Exception

containLogSystem

protected abstract boolean containLogSystem()

snapshotState

public void snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)
                   throws Exception

Specified by:

snapshotState in interface org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator

Overrides:

snapshotState in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<Committable>

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.streaming.api.operators.StreamOperator<Committable>

Overrides:

close in class PrepareCommitOperator<IN,Committable>

Throws:

Exception

prepareCommit

protected List<Committable> prepareCommit(boolean waitCompaction,
                                          long checkpointId)
                                   throws IOException

Specified by:

prepareCommit in class PrepareCommitOperator<IN,Committable>

Throws:

IOException

getWrite

@VisibleForTesting
public StoreSinkWrite getWrite()