org.apache.paimon.operation

Class AppendOnlyFileStoreWrite

java.lang.Object

org.apache.paimon.operation.AbstractFileStoreWrite<T>

org.apache.paimon.operation.MemoryFileStoreWrite<InternalRow>

org.apache.paimon.operation.AppendOnlyFileStoreWrite

All Implemented Interfaces:

BundleFileStoreWriter, FileStoreWrite<InternalRow>, Restorable<List<FileStoreWrite.State<InternalRow>>>

public class AppendOnlyFileStoreWrite
extends MemoryFileStoreWrite<InternalRow>
implements BundleFileStoreWriter

FileStoreWrite for AppendOnlyFileStore.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.paimon.operation.AbstractFileStoreWrite

AbstractFileStoreWrite.WriterContainer<T>

Nested classes/interfaces inherited from interface org.apache.paimon.operation.FileStoreWrite

FileStoreWrite.State<T>

Field Summary

Fields inherited from class org.apache.paimon.operation.MemoryFileStoreWrite

cacheManager, options

Fields inherited from class org.apache.paimon.operation.AbstractFileStoreWrite

compactionMetrics, ioManager, isStreamingMode, snapshotManager, tableName, writers

Constructor Summary

Constructors

Constructor and Description
AppendOnlyFileStoreWrite(FileIO fileIO, RawFileSplitRead read, long schemaId, String commitUser, RowType rowType, FileStorePathFactory pathFactory, SnapshotManager snapshotManager, FileStoreScan scan, CoreOptions options, BucketMode bucketMode, DeletionVectorsMaintainer.Factory dvMaintainerFactory, String tableName)

Method Summary

Modifier and Type	Method and Description
List<DataFileMeta>	compactRewrite(BinaryRow partition, int bucket, java.util.function.Function<String,DeletionVector> dvFactory, List<DataFileMeta> toCompact)
protected RecordWriter<InternalRow>	createWriter(Long snapshotId, BinaryRow partition, int bucket, List<DataFileMeta> restoredFiles, long restoredMaxSeqNumber, CommitIncrement restoreIncrement, ExecutorService compactExecutor, DeletionVectorsMaintainer dvMaintainer)
protected void	forceBufferSpill()
void	withIgnorePreviousFiles(boolean ignorePrevious) Set whether the write operation should ignore previously stored files.
void	writeBundle(BinaryRow partition, int bucket, BundleRecords bundle) Write the batch data to the store according to the partition and bucket.

Methods inherited from class org.apache.paimon.operation.MemoryFileStoreWrite

close, notifyNewWriter, prepareCommit, withMemoryPoolFactory, withMetricRegistry

Methods inherited from class org.apache.paimon.operation.AbstractFileStoreWrite

checkpoint, compact, createWriterContainer, getActiveBuckets, getCompactExecutor, getWriterWrapper, notifyNewFiles, restore, withCompactExecutor, withExecutionMode, withInsertOnly, withIOManager, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.paimon.operation.FileStoreWrite

close, compact, notifyNewFiles, prepareCommit, withCompactExecutor, withExecutionMode, withInsertOnly, withIOManager, withMemoryPool, withMemoryPoolFactory, withMetricRegistry, write

Methods inherited from interface org.apache.paimon.utils.Restorable

checkpoint, restore

Constructor Detail

AppendOnlyFileStoreWrite

public AppendOnlyFileStoreWrite(FileIO fileIO,
                                RawFileSplitRead read,
                                long schemaId,
                                String commitUser,
                                RowType rowType,
                                FileStorePathFactory pathFactory,
                                SnapshotManager snapshotManager,
                                FileStoreScan scan,
                                CoreOptions options,
                                BucketMode bucketMode,
                                @Nullable
                                DeletionVectorsMaintainer.Factory dvMaintainerFactory,
                                String tableName)

Method Detail

createWriter

protected RecordWriter<InternalRow> createWriter(@Nullable
                                                 Long snapshotId,
                                                 BinaryRow partition,
                                                 int bucket,
                                                 List<DataFileMeta> restoredFiles,
                                                 long restoredMaxSeqNumber,
                                                 @Nullable
                                                 CommitIncrement restoreIncrement,
                                                 ExecutorService compactExecutor,
                                                 @Nullable
                                                 DeletionVectorsMaintainer dvMaintainer)

Specified by:

createWriter in class AbstractFileStoreWrite<InternalRow>

compactRewrite

public List<DataFileMeta> compactRewrite(BinaryRow partition,
                                         int bucket,
                                         @Nullable
                                         java.util.function.Function<String,DeletionVector> dvFactory,
                                         List<DataFileMeta> toCompact)
                                  throws Exception

Throws:

Exception

withIgnorePreviousFiles

public void withIgnorePreviousFiles(boolean ignorePrevious)

Description copied from interface: FileStoreWrite

Set whether the write operation should ignore previously stored files.

Specified by:

withIgnorePreviousFiles in interface FileStoreWrite<InternalRow>

Overrides:

withIgnorePreviousFiles in class AbstractFileStoreWrite<InternalRow>

Parameters:

ignorePrevious - whether the write operation should ignore previously stored files.

forceBufferSpill

protected void forceBufferSpill()
                         throws Exception

Overrides:

forceBufferSpill in class AbstractFileStoreWrite<InternalRow>

Throws:

Exception

writeBundle

public void writeBundle(BinaryRow partition,
                        int bucket,
                        BundleRecords bundle)
                 throws Exception

Description copied from interface: BundleFileStoreWriter

Write the batch data to the store according to the partition and bucket.

Specified by:

writeBundle in interface BundleFileStoreWriter

Parameters:

partition - the partition of the data

bucket - the bucket id of the data

bundle - the given data

Throws:

Exception - the thrown exception when writing the record