org.apache.paimon.flink.sink

Class StoreSinkWriteImpl

java.lang.Object

org.apache.paimon.flink.sink.StoreSinkWriteImpl

All Implemented Interfaces:

StoreSinkWrite

Direct Known Subclasses:

AsyncLookupSinkWrite, GlobalFullCompactionSinkWrite

public class StoreSinkWriteImpl
extends Object
implements StoreSinkWrite

Default implementation of StoreSinkWrite. This writer does not have states.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.paimon.flink.sink.StoreSinkWrite

StoreSinkWrite.Provider, StoreSinkWrite.WithWriteBufferProvider

Field Summary

Fields

Modifier and Type	Field and Description
protected String	commitUser
protected StoreSinkWriteState	state
protected TableWriteImpl<?>	write

Constructor Summary

Constructors

Constructor and Description
StoreSinkWriteImpl(FileStoreTable table, String commitUser, StoreSinkWriteState state, org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager, boolean ignorePreviousFiles, boolean waitCompaction, boolean isStreamingMode, MemoryPoolFactory memoryPoolFactory, org.apache.flink.metrics.MetricGroup metricGroup)
StoreSinkWriteImpl(FileStoreTable table, String commitUser, StoreSinkWriteState state, org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager, boolean ignorePreviousFiles, boolean waitCompaction, boolean isStreamingMode, MemorySegmentPool memoryPool, org.apache.flink.metrics.MetricGroup metricGroup)

Method Summary

Modifier and Type	Method and Description
void	close()
void	compact(BinaryRow partition, int bucket, boolean fullCompaction)
TableWriteImpl<?>	getWrite()
void	notifyNewFiles(long snapshotId, BinaryRow partition, int bucket, List<DataFileMeta> files)
List<Committable>	prepareCommit(boolean waitCompaction, long checkpointId)
void	replace(FileStoreTable newTable) Replace the internal TableWriteImpl with the one provided by newWriteProvider.
void	snapshotState()
boolean	streamingMode()
SinkRecord	toLogRecord(SinkRecord record)
void	withCompactExecutor(ExecutorService compactExecutor)
void	withInsertOnly(boolean insertOnly) This method is called when the insert only status of the records changes.
SinkRecord	write(InternalRow rowData)
SinkRecord	write(InternalRow rowData, int bucket)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

commitUser

protected final String commitUser

state

protected final StoreSinkWriteState state

write

protected TableWriteImpl<?> write

Constructor Detail

StoreSinkWriteImpl

public StoreSinkWriteImpl(FileStoreTable table,
                          String commitUser,
                          StoreSinkWriteState state,
                          org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
                          boolean ignorePreviousFiles,
                          boolean waitCompaction,
                          boolean isStreamingMode,
                          @Nullable
                          MemorySegmentPool memoryPool,
                          @Nullable
                          org.apache.flink.metrics.MetricGroup metricGroup)

StoreSinkWriteImpl

public StoreSinkWriteImpl(FileStoreTable table,
                          String commitUser,
                          StoreSinkWriteState state,
                          org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
                          boolean ignorePreviousFiles,
                          boolean waitCompaction,
                          boolean isStreamingMode,
                          MemoryPoolFactory memoryPoolFactory,
                          @Nullable
                          org.apache.flink.metrics.MetricGroup metricGroup)

Method Detail

withCompactExecutor

public void withCompactExecutor(ExecutorService compactExecutor)

withInsertOnly

public void withInsertOnly(boolean insertOnly)

Description copied from interface: StoreSinkWrite

This method is called when the insert only status of the records changes.

Specified by:

withInsertOnly in interface StoreSinkWrite

Parameters:

insertOnly - If true, all the following records would be of RowKind.INSERT, and no two records would have the same primary key.

write

@Nullable
public SinkRecord write(InternalRow rowData)
                           throws Exception

Specified by:

write in interface StoreSinkWrite

Throws:

Exception

write

@Nullable
public SinkRecord write(InternalRow rowData,
                                  int bucket)
                           throws Exception

Specified by:

write in interface StoreSinkWrite

Throws:

Exception

toLogRecord

public SinkRecord toLogRecord(SinkRecord record)

Specified by:

toLogRecord in interface StoreSinkWrite

compact

public void compact(BinaryRow partition,
                    int bucket,
                    boolean fullCompaction)
             throws Exception

Specified by:

compact in interface StoreSinkWrite

Throws:

Exception

notifyNewFiles

public void notifyNewFiles(long snapshotId,
                           BinaryRow partition,
                           int bucket,
                           List<DataFileMeta> files)

Specified by:

notifyNewFiles in interface StoreSinkWrite

prepareCommit

public List<Committable> prepareCommit(boolean waitCompaction,
                                       long checkpointId)
                                throws IOException

Specified by:

prepareCommit in interface StoreSinkWrite

Throws:

IOException

snapshotState

public void snapshotState()
                   throws Exception

Specified by:

snapshotState in interface StoreSinkWrite

Throws:

Exception

streamingMode

public boolean streamingMode()

Specified by:

streamingMode in interface StoreSinkWrite

close

public void close()
           throws Exception

Specified by:

close in interface StoreSinkWrite

Throws:

Exception

replace

public void replace(FileStoreTable newTable)
             throws Exception

Description copied from interface: StoreSinkWrite

Replace the internal TableWriteImpl with the one provided by newWriteProvider. The state of the old TableWriteImpl will also be transferred to the new TableWriteImpl by TableWriteImpl.checkpoint() and TableWriteImpl.restore(List).

Currently, this method is only used by CDC sinks because they need to deal with schema changes. TableWriteImpl with the new schema will be provided by newWriteProvider.

Specified by:

replace in interface StoreSinkWrite

Throws:

Exception

getWrite

@VisibleForTesting
public TableWriteImpl<?> getWrite()