org.apache.paimon.operation

Class AppendOnlyFileStoreWrite

java.lang.Object

org.apache.paimon.operation.AbstractFileStoreWrite<T>

org.apache.paimon.operation.MemoryFileStoreWrite<InternalRow>

org.apache.paimon.operation.AppendOnlyFileStoreWrite

All Implemented Interfaces:

BundleFileStoreWriter, FileStoreWrite<InternalRow>, Restorable<List<FileStoreWrite.State<InternalRow>>>

Direct Known Subclasses:

AppendOnlyFixedBucketFileStoreWrite, AppendOnlyUnawareBucketFileStoreWrite

public abstract class AppendOnlyFileStoreWrite
extends MemoryFileStoreWrite<InternalRow>
implements BundleFileStoreWriter

FileStoreWrite for AppendOnlyFileStore.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.paimon.operation.AbstractFileStoreWrite

AbstractFileStoreWrite.WriterContainer<T>

Nested classes/interfaces inherited from interface org.apache.paimon.operation.FileStoreWrite

FileStoreWrite.State<T>

Field Summary

Fields inherited from class org.apache.paimon.operation.MemoryFileStoreWrite

cacheManager, options

Fields inherited from class org.apache.paimon.operation.AbstractFileStoreWrite

compactionMetrics, ioManager, isStreamingMode, snapshotManager, tableName, writers

Constructor Summary

Constructors

Constructor and Description
AppendOnlyFileStoreWrite(FileIO fileIO, RawFileSplitRead read, long schemaId, RowType rowType, RowType partitionType, FileStorePathFactory pathFactory, SnapshotManager snapshotManager, FileStoreScan scan, CoreOptions options, DeletionVectorsMaintainer.Factory dvMaintainerFactory, String tableName)

Method Summary

Modifier and Type	Method and Description
List<DataFileMeta>	compactRewrite(BinaryRow partition, int bucket, java.util.function.Function<String,DeletionVector> dvFactory, List<DataFileMeta> toCompact)
protected RecordWriter<InternalRow>	createWriter(Long snapshotId, BinaryRow partition, int bucket, List<DataFileMeta> restoredFiles, long restoredMaxSeqNumber, CommitIncrement restoreIncrement, ExecutorService compactExecutor, DeletionVectorsMaintainer dvMaintainer)
protected void	forceBufferSpill()
protected abstract CompactManager	getCompactManager(BinaryRow partition, int bucket, List<DataFileMeta> restoredFiles, ExecutorService compactExecutor, DeletionVectorsMaintainer dvMaintainer)
void	writeBundle(BinaryRow partition, int bucket, BundleRecords bundle) Write the batch data to the store according to the partition and bucket.

Methods inherited from class org.apache.paimon.operation.MemoryFileStoreWrite

close, notifyNewWriter, prepareCommit, withMemoryPoolFactory, withMetricRegistry

Methods inherited from class org.apache.paimon.operation.AbstractFileStoreWrite

checkpoint, compact, createConflictAwareWriterCleanChecker, createNoConflictAwareWriterCleanChecker, createWriterCleanChecker, createWriterContainer, getActiveBuckets, getCompactExecutor, getWriterWrapper, notifyNewFiles, restore, withCompactExecutor, withExecutionMode, withIgnorePreviousFiles, withInsertOnly, withIOManager, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.paimon.operation.FileStoreWrite

close, compact, notifyNewFiles, prepareCommit, withCompactExecutor, withExecutionMode, withIgnorePreviousFiles, withInsertOnly, withIOManager, withMemoryPool, withMemoryPoolFactory, withMetricRegistry, write

Methods inherited from interface org.apache.paimon.utils.Restorable

checkpoint, restore

Constructor Detail

AppendOnlyFileStoreWrite

public AppendOnlyFileStoreWrite(FileIO fileIO,
                                RawFileSplitRead read,
                                long schemaId,
                                RowType rowType,
                                RowType partitionType,
                                FileStorePathFactory pathFactory,
                                SnapshotManager snapshotManager,
                                FileStoreScan scan,
                                CoreOptions options,
                                @Nullable
                                DeletionVectorsMaintainer.Factory dvMaintainerFactory,
                                String tableName)

Method Detail

createWriter

protected RecordWriter<InternalRow> createWriter(@Nullable
                                                 Long snapshotId,
                                                 BinaryRow partition,
                                                 int bucket,
                                                 List<DataFileMeta> restoredFiles,
                                                 long restoredMaxSeqNumber,
                                                 @Nullable
                                                 CommitIncrement restoreIncrement,
                                                 ExecutorService compactExecutor,
                                                 @Nullable
                                                 DeletionVectorsMaintainer dvMaintainer)

Specified by:

createWriter in class AbstractFileStoreWrite<InternalRow>

getCompactManager

protected abstract CompactManager getCompactManager(BinaryRow partition,
                                                    int bucket,
                                                    List<DataFileMeta> restoredFiles,
                                                    ExecutorService compactExecutor,
                                                    @Nullable
                                                    DeletionVectorsMaintainer dvMaintainer)

compactRewrite

public List<DataFileMeta> compactRewrite(BinaryRow partition,
                                         int bucket,
                                         @Nullable
                                         java.util.function.Function<String,DeletionVector> dvFactory,
                                         List<DataFileMeta> toCompact)
                                  throws Exception

Throws:

Exception

forceBufferSpill

protected void forceBufferSpill()
                         throws Exception

Overrides:

forceBufferSpill in class AbstractFileStoreWrite<InternalRow>

Throws:

Exception

writeBundle

public void writeBundle(BinaryRow partition,
                        int bucket,
                        BundleRecords bundle)
                 throws Exception

Description copied from interface: BundleFileStoreWriter

Write the batch data to the store according to the partition and bucket.

Specified by:

writeBundle in interface BundleFileStoreWriter

Parameters:

partition - the partition of the data

bucket - the bucket id of the data

bundle - the given data

Throws:

Exception - the thrown exception when writing the record