org.apache.paimon.operation

Class MemoryFileStoreWrite<T>

java.lang.Object

org.apache.paimon.operation.AbstractFileStoreWrite<T>

org.apache.paimon.operation.MemoryFileStoreWrite<T>

Type Parameters:

T - type of record to write.

All Implemented Interfaces:

FileStoreWrite<T>, Restorable<List<FileStoreWrite.State<T>>>

Direct Known Subclasses:

AppendOnlyFileStoreWrite, KeyValueFileStoreWrite

public abstract class MemoryFileStoreWrite<T>
extends AbstractFileStoreWrite<T>

Base FileStoreWrite implementation which supports using shared memory and preempting memory from other writers.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.paimon.operation.AbstractFileStoreWrite

AbstractFileStoreWrite.WriterContainer<T>

Nested classes/interfaces inherited from interface org.apache.paimon.operation.FileStoreWrite

FileStoreWrite.State<T>

Field Summary

Fields

Modifier and Type	Field and Description
protected CacheManager	cacheManager
protected CoreOptions	options

Fields inherited from class org.apache.paimon.operation.AbstractFileStoreWrite

compactionMetrics, ioManager, isStreamingMode, snapshotManager, tableName, writers

Constructor Summary

Constructors

Constructor and Description
MemoryFileStoreWrite(SnapshotManager snapshotManager, FileStoreScan scan, CoreOptions options, RowType partitionType, IndexMaintainer.Factory<T> indexFactory, DeletionVectorsMaintainer.Factory dvMaintainerFactory, String tableName)

Method Summary

Modifier and Type	Method and Description
void	close() Close the writer.
protected void	notifyNewWriter(RecordWriter<T> writer)
List<CommitMessage>	prepareCommit(boolean waitCompaction, long commitIdentifier) Prepare commit in the write.
FileStoreWrite<T>	withMemoryPoolFactory(MemoryPoolFactory memoryPoolFactory) With memory pool factory for the current file store write.
FileStoreWrite<T>	withMetricRegistry(MetricRegistry metricRegistry) With metrics to measure compaction.

Methods inherited from class org.apache.paimon.operation.AbstractFileStoreWrite

checkpoint, compact, createConflictAwareWriterCleanChecker, createNoConflictAwareWriterCleanChecker, createWriter, createWriterCleanChecker, createWriterContainer, forceBufferSpill, getActiveBuckets, getCompactExecutor, getWriterWrapper, notifyNewFiles, restore, withCompactExecutor, withExecutionMode, withIgnorePreviousFiles, withInsertOnly, withIOManager, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.paimon.operation.FileStoreWrite

withMemoryPool

Field Detail

options

protected final CoreOptions options

cacheManager

protected final CacheManager cacheManager

Constructor Detail

MemoryFileStoreWrite

public MemoryFileStoreWrite(SnapshotManager snapshotManager,
                            FileStoreScan scan,
                            CoreOptions options,
                            RowType partitionType,
                            @Nullable
                            IndexMaintainer.Factory<T> indexFactory,
                            @Nullable
                            DeletionVectorsMaintainer.Factory dvMaintainerFactory,
                            String tableName)

Method Detail

withMemoryPoolFactory

public FileStoreWrite<T> withMemoryPoolFactory(MemoryPoolFactory memoryPoolFactory)

Description copied from interface: FileStoreWrite

With memory pool factory for the current file store write.

Specified by:

withMemoryPoolFactory in interface FileStoreWrite<T>

Overrides:

withMemoryPoolFactory in class AbstractFileStoreWrite<T>

Parameters:

memoryPoolFactory - the given memory pool factory.

notifyNewWriter

protected void notifyNewWriter(RecordWriter<T> writer)

Overrides:

notifyNewWriter in class AbstractFileStoreWrite<T>

withMetricRegistry

public FileStoreWrite<T> withMetricRegistry(MetricRegistry metricRegistry)

Description copied from interface: FileStoreWrite

With metrics to measure compaction.

Specified by:

withMetricRegistry in interface FileStoreWrite<T>

Overrides:

withMetricRegistry in class AbstractFileStoreWrite<T>

prepareCommit

public List<CommitMessage> prepareCommit(boolean waitCompaction,
                                         long commitIdentifier)
                                  throws Exception

Description copied from interface: FileStoreWrite

Prepare commit in the write.

Specified by:

prepareCommit in interface FileStoreWrite<T>

Overrides:

prepareCommit in class AbstractFileStoreWrite<T>

Parameters:

waitCompaction - if this method need to wait for current compaction to complete

commitIdentifier - identifier of the commit being prepared

Returns:

the file committable list

Throws:

Exception - the thrown exception

close

public void close()
           throws Exception

Description copied from interface: FileStoreWrite

Close the writer.

Specified by:

close in interface FileStoreWrite<T>

Overrides:

close in class AbstractFileStoreWrite<T>

Throws:

Exception - the thrown exception