org.apache.paimon.io

Class KeyValueDataFileWriter

java.lang.Object

org.apache.paimon.io.SingleFileWriter<T,R>

org.apache.paimon.io.StatsCollectingSingleFileWriter<KeyValue,DataFileMeta>

org.apache.paimon.io.KeyValueDataFileWriter

All Implemented Interfaces:

Closeable, AutoCloseable, FileWriter<KeyValue,DataFileMeta>

Direct Known Subclasses:

KeyValueDataFileWriterImpl, KeyValueThinDataFileWriterImpl

public abstract class KeyValueDataFileWriter
extends StatsCollectingSingleFileWriter<KeyValue,DataFileMeta>

A StatsCollectingSingleFileWriter to write data files containing KeyValues. Also produces DataFileMeta after writing a file.

NOTE: records given to the writer must be sorted because it does not compare the min max keys to produce DataFileMeta.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.paimon.io.SingleFileWriter

SingleFileWriter.AbortExecutor

Field Summary

Fields

Modifier and Type	Field and Description
protected RowType	keyType
protected RowType	valueType

Fields inherited from class org.apache.paimon.io.SingleFileWriter

closed, fileIO, path

Constructor Summary

Constructors

Constructor and Description
KeyValueDataFileWriter(FileIO fileIO, FormatWriterFactory factory, Path path, java.util.function.Function<KeyValue,InternalRow> converter, RowType keyType, RowType valueType, RowType writeRowType, SimpleStatsExtractor simpleStatsExtractor, long schemaId, int level, String compression, CoreOptions options, FileSource fileSource, FileIndexOptions fileIndexOptions)

Method Summary

Modifier and Type	Method and Description
void	close()
DataFileMeta	result()
void	write(KeyValue kv) Add only one record to this file writer.

Methods inherited from class org.apache.paimon.io.StatsCollectingSingleFileWriter

fieldStats, writeBundle

Methods inherited from class org.apache.paimon.io.SingleFileWriter

abort, abortExecutor, path, reachTargetSize, recordCount, writeImpl

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.paimon.io.FileWriter

write, write, write

Field Detail

keyType

protected final RowType keyType

valueType

protected final RowType valueType

Constructor Detail

KeyValueDataFileWriter

public KeyValueDataFileWriter(FileIO fileIO,
                              FormatWriterFactory factory,
                              Path path,
                              java.util.function.Function<KeyValue,InternalRow> converter,
                              RowType keyType,
                              RowType valueType,
                              RowType writeRowType,
                              @Nullable
                              SimpleStatsExtractor simpleStatsExtractor,
                              long schemaId,
                              int level,
                              String compression,
                              CoreOptions options,
                              FileSource fileSource,
                              FileIndexOptions fileIndexOptions)

Method Detail

write

public void write(KeyValue kv)
           throws IOException

Description copied from interface: FileWriter

Add only one record to this file writer.

NOTE: If any exception occurs during writing, the writer should clean up useless files for the user.

Specified by:

write in interface FileWriter<KeyValue,DataFileMeta>

Overrides:

write in class StatsCollectingSingleFileWriter<KeyValue,DataFileMeta>

Parameters:

kv - to write.

Throws:

IOException - if encounter any IO error.

result

@Nullable
public DataFileMeta result()
                              throws IOException

Returns:

the result for this closed file writer.

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class SingleFileWriter<KeyValue,DataFileMeta>

Throws:

IOException