org.apache.paimon.hive.mapred

Class PaimonRecordReader

java.lang.Object

org.apache.paimon.hive.mapred.PaimonRecordReader

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.mapred.RecordReader<Void,RowDataContainer>

public class PaimonRecordReader
extends Object
implements org.apache.hadoop.mapred.RecordReader<Void,RowDataContainer>

Base RecordReader for paimon. Reads KeyValues from data files and picks out InternalRow for Hive to consume.

NOTE: To support projection push down, when selectedColumns does not match columnNames this reader will still produce records of the original schema. However, columns not in selectedColumns will be null.

Constructor Summary

Constructors

Constructor and Description
PaimonRecordReader(ReadBuilder readBuilder, PaimonInputSplit split, List<String> paimonColumns, List<String> hiveColumns, List<String> selectedColumns, String tagToPartField)

Method Summary

Modifier and Type	Method and Description
void	close()
Void	createKey()
static org.apache.hadoop.mapred.RecordReader<Void,RowDataContainer>	createRecordReader(PaimonInputSplit split, org.apache.hadoop.mapred.JobConf jobConf)
RowDataContainer	createValue()
long	getPos()
float	getProgress()
boolean	next(Void key, RowDataContainer value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PaimonRecordReader

public PaimonRecordReader(ReadBuilder readBuilder,
                          PaimonInputSplit split,
                          List<String> paimonColumns,
                          List<String> hiveColumns,
                          List<String> selectedColumns,
                          @Nullable
                          String tagToPartField)
                   throws IOException

Parameters:

paimonColumns - columns stored in Paimon table

hiveColumns - columns Hive expects

selectedColumns - columns we really have to read

Throws:

IOException

Method Detail

next

public boolean next(Void key,
                    RowDataContainer value)
             throws IOException

Specified by:

next in interface org.apache.hadoop.mapred.RecordReader<Void,RowDataContainer>

Throws:

IOException

createKey

public Void createKey()

Specified by:

createKey in interface org.apache.hadoop.mapred.RecordReader<Void,RowDataContainer>

createValue

public RowDataContainer createValue()

Specified by:

createValue in interface org.apache.hadoop.mapred.RecordReader<Void,RowDataContainer>

getPos

public long getPos()
            throws IOException

Specified by:

getPos in interface org.apache.hadoop.mapred.RecordReader<Void,RowDataContainer>

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface org.apache.hadoop.mapred.RecordReader<Void,RowDataContainer>

Throws:

IOException

getProgress

public float getProgress()
                  throws IOException

Specified by:

getProgress in interface org.apache.hadoop.mapred.RecordReader<Void,RowDataContainer>

Throws:

IOException

createRecordReader

public static org.apache.hadoop.mapred.RecordReader<Void,RowDataContainer> createRecordReader(PaimonInputSplit split,
                                                                                              org.apache.hadoop.mapred.JobConf jobConf)
                                                                                       throws IOException

Throws:

IOException