org.apache.paimon.format.parquet.reader

Class ParquetSplitReaderUtil

java.lang.Object

org.apache.paimon.format.parquet.reader.ParquetSplitReaderUtil

public class ParquetSplitReaderUtil
extends Object

Util for generating ColumnReader.

Constructor Summary

Constructors

Constructor and Description
ParquetSplitReaderUtil()

Method Summary

Modifier and Type	Method and Description
static List<ParquetField>	buildFieldsList(List<DataField> children, List<String> fieldNames, org.apache.parquet.io.MessageColumnIO columnIO)
static ColumnReader	createColumnReader(DataType fieldType, org.apache.parquet.schema.Type type, List<org.apache.parquet.column.ColumnDescriptor> columnDescriptors, org.apache.parquet.column.page.PageReadStore pages, ParquetField field, int depth)
static WritableColumnVector	createWritableColumnVector(int batchSize, DataType fieldType, org.apache.parquet.schema.Type type, List<org.apache.parquet.column.ColumnDescriptor> columnDescriptors, int depth)
static org.apache.parquet.io.ColumnIO	getArrayElementColumn(org.apache.parquet.io.ColumnIO columnIO)
static org.apache.parquet.io.GroupColumnIO	getMapKeyValueColumn(org.apache.parquet.io.GroupColumnIO groupColumnIO)
static org.apache.parquet.io.ColumnIO	lookupColumnByName(org.apache.parquet.io.GroupColumnIO groupColumnIO, String columnName) Parquet's column names are case in sensitive.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ParquetSplitReaderUtil

public ParquetSplitReaderUtil()

Method Detail

createColumnReader

public static ColumnReader createColumnReader(DataType fieldType,
                                              org.apache.parquet.schema.Type type,
                                              List<org.apache.parquet.column.ColumnDescriptor> columnDescriptors,
                                              org.apache.parquet.column.page.PageReadStore pages,
                                              ParquetField field,
                                              int depth)
                                       throws IOException

Throws:

IOException

createWritableColumnVector

public static WritableColumnVector createWritableColumnVector(int batchSize,
                                                              DataType fieldType,
                                                              org.apache.parquet.schema.Type type,
                                                              List<org.apache.parquet.column.ColumnDescriptor> columnDescriptors,
                                                              int depth)

buildFieldsList

public static List<ParquetField> buildFieldsList(List<DataField> children,
                                                 List<String> fieldNames,
                                                 org.apache.parquet.io.MessageColumnIO columnIO)

lookupColumnByName

public static org.apache.parquet.io.ColumnIO lookupColumnByName(org.apache.parquet.io.GroupColumnIO groupColumnIO,
                                                                String columnName)

Parquet's column names are case in sensitive. So when we look up columns we first check for exact match, and if that can not find we look for a case-insensitive match.

getMapKeyValueColumn

public static org.apache.parquet.io.GroupColumnIO getMapKeyValueColumn(org.apache.parquet.io.GroupColumnIO groupColumnIO)

getArrayElementColumn

public static org.apache.parquet.io.ColumnIO getArrayElementColumn(org.apache.parquet.io.ColumnIO columnIO)