org.apache.paimon.arrow.converter

Class ArrowBatchConverter

java.lang.Object

org.apache.paimon.arrow.converter.ArrowBatchConverter

Direct Known Subclasses:

ArrowPerRowBatchConverter, ArrowVectorizedBatchConverter

public abstract class ArrowBatchConverter
extends Object

A reusable converter to convert Paimon data in RecordReader.RecordIterator into Arrow format.

Field Summary

Fields

Modifier and Type	Field and Description
protected ArrowFieldWriter[]	fieldWriters
protected RecordReader.RecordIterator<InternalRow>	iterator
protected org.apache.arrow.vector.VectorSchemaRoot	root

Method Summary

Modifier and Type	Method and Description
void	close()
protected abstract void	doWrite(int maxBatchRows)
org.apache.arrow.vector.VectorSchemaRoot	next(int maxBatchRows) Write and get at most maxBatchRows data.
protected void	releaseIterator()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

root

protected final org.apache.arrow.vector.VectorSchemaRoot root

fieldWriters

protected final ArrowFieldWriter[] fieldWriters

iterator

protected RecordReader.RecordIterator<InternalRow> iterator

Method Detail

next

@Nullable
public org.apache.arrow.vector.VectorSchemaRoot next(int maxBatchRows)

Write and get at most maxBatchRows data. Return null when finishing writing current iterator.

NOTE: the returned value will be reused, and it's lifecycle is managed by this writer.

doWrite

protected abstract void doWrite(int maxBatchRows)

close

public void close()

releaseIterator

protected void releaseIterator()