org.apache.orc.impl

Class RecordReaderImpl

java.lang.Object

org.apache.orc.impl.RecordReaderImpl

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.orc.RecordReader

public class RecordReaderImpl
extends Object
implements org.apache.orc.RecordReader

An orc RecordReaderImpl.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RecordReaderImpl.PositionProviderImpl An orc PositionProvider impl.
static class	RecordReaderImpl.SargApplier search argument applier.
static class	RecordReaderImpl.ZeroPositionProvider An orc PositionProvider impl.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.orc.OrcProto.ColumnStatistics	EMPTY_COLUMN_STATISTICS
protected org.apache.hadoop.fs.Path	path
protected org.apache.orc.TypeDescription	schema

Constructor Summary

Constructors

Constructor and Description
RecordReaderImpl(org.apache.orc.impl.ReaderImpl fileReader, org.apache.orc.Reader.Options options, FileIndexResult fileIndexResult)

Method Summary

Modifier and Type	Method and Description
void	close()
static String	encodeTranslatedSargColumn(int rootColumn, Integer indexInSourceTable)
static org.apache.hadoop.hive.ql.io.sarg.SearchArgument.TruthValue	evaluatePredicate(org.apache.orc.ColumnStatistics stats, org.apache.hadoop.hive.ql.io.sarg.PredicateLeaf predicate, org.apache.orc.util.BloomFilter bloomFilter) Evaluate a predicate with respect to the statistics from the column that is referenced in the predicate.
static org.apache.hadoop.hive.ql.io.sarg.SearchArgument.TruthValue	evaluatePredicate(org.apache.orc.ColumnStatistics stats, org.apache.hadoop.hive.ql.io.sarg.PredicateLeaf predicate, org.apache.orc.util.BloomFilter bloomFilter, boolean useUTCTimestamp) Evaluate a predicate with respect to the statistics from the column that is referenced in the predicate.
org.apache.orc.CompressionCodec	getCompressionCodec()
int	getMaxDiskRangeChunkLimit()
float	getProgress() Return the fraction of rows that have been read from the selected.
long	getRowNumber()
static int[]	mapSargColumnsToOrcInternalColIdx(List<org.apache.hadoop.hive.ql.io.sarg.PredicateLeaf> sargLeaves, org.apache.orc.impl.SchemaEvolution evolution) Find the mapping from predicate leaves to columns.
static int[]	mapTranslatedSargColumns(List<org.apache.orc.OrcProto.Type> types, List<org.apache.hadoop.hive.ql.io.sarg.PredicateLeaf> sargLeaves)
boolean	nextBatch(org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch batch)
protected boolean[]	pickRowGroups() Pick the row groups that we need to load from the current stripe.
org.apache.orc.impl.OrcIndex	readRowIndex(int stripeIndex, boolean[] included, boolean[] readCols)
org.apache.orc.OrcProto.StripeFooter	readStripeFooter(org.apache.orc.StripeInformation stripe)
void	seekToRow(long rowNumber)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EMPTY_COLUMN_STATISTICS

public static final org.apache.orc.OrcProto.ColumnStatistics EMPTY_COLUMN_STATISTICS

path

protected final org.apache.hadoop.fs.Path path

schema

protected final org.apache.orc.TypeDescription schema

Constructor Detail

RecordReaderImpl

public RecordReaderImpl(org.apache.orc.impl.ReaderImpl fileReader,
                        org.apache.orc.Reader.Options options,
                        FileIndexResult fileIndexResult)
                 throws IOException

Throws:

IOException

Method Detail

mapSargColumnsToOrcInternalColIdx

public static int[] mapSargColumnsToOrcInternalColIdx(List<org.apache.hadoop.hive.ql.io.sarg.PredicateLeaf> sargLeaves,
                                                      org.apache.orc.impl.SchemaEvolution evolution)

Find the mapping from predicate leaves to columns.

Parameters:

sargLeaves - the search argument that we need to map

evolution - the mapping from reader to file schema

Returns:

an array mapping the sarg leaves to concrete column numbers in the file

readStripeFooter

public org.apache.orc.OrcProto.StripeFooter readStripeFooter(org.apache.orc.StripeInformation stripe)
                                                      throws IOException

Throws:

IOException

evaluatePredicate

public static org.apache.hadoop.hive.ql.io.sarg.SearchArgument.TruthValue evaluatePredicate(org.apache.orc.ColumnStatistics stats,
                                                                                            org.apache.hadoop.hive.ql.io.sarg.PredicateLeaf predicate,
                                                                                            org.apache.orc.util.BloomFilter bloomFilter)

Evaluate a predicate with respect to the statistics from the column that is referenced in the predicate.

Parameters:

stats - the statistics for the column mentioned in the predicate

predicate - the leaf predicate we need to evaluation

Returns:

the set of truth values that may be returned for the given predicate.

evaluatePredicate

public static org.apache.hadoop.hive.ql.io.sarg.SearchArgument.TruthValue evaluatePredicate(org.apache.orc.ColumnStatistics stats,
                                                                                            org.apache.hadoop.hive.ql.io.sarg.PredicateLeaf predicate,
                                                                                            org.apache.orc.util.BloomFilter bloomFilter,
                                                                                            boolean useUTCTimestamp)

Evaluate a predicate with respect to the statistics from the column that is referenced in the predicate. Includes option to specify if timestamp column stats values should be in UTC.

Parameters:

stats - the statistics for the column mentioned in the predicate

predicate - the leaf predicate we need to evaluation

bloomFilter -

useUTCTimestamp -

Returns:

the set of truth values that may be returned for the given predicate.

pickRowGroups

protected boolean[] pickRowGroups()
                           throws IOException

Pick the row groups that we need to load from the current stripe.

Returns:

an array with a boolean for each row group or null if all of the row groups must be read.

Throws:

IOException

nextBatch

public boolean nextBatch(org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch batch)
                  throws IOException

Specified by:

nextBatch in interface org.apache.orc.RecordReader

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface org.apache.orc.RecordReader

Throws:

IOException

getRowNumber

public long getRowNumber()

Specified by:

getRowNumber in interface org.apache.orc.RecordReader

getProgress

public float getProgress()

Return the fraction of rows that have been read from the selected. section of the file

Specified by:

getProgress in interface org.apache.orc.RecordReader

Returns:

fraction between 0.0 and 1.0 of rows consumed

readRowIndex

public org.apache.orc.impl.OrcIndex readRowIndex(int stripeIndex,
                                                 boolean[] included,
                                                 boolean[] readCols)
                                          throws IOException

Throws:

IOException

seekToRow

public void seekToRow(long rowNumber)
               throws IOException

Specified by:

seekToRow in interface org.apache.orc.RecordReader

Throws:

IOException

encodeTranslatedSargColumn

public static String encodeTranslatedSargColumn(int rootColumn,
                                                Integer indexInSourceTable)

mapTranslatedSargColumns

public static int[] mapTranslatedSargColumns(List<org.apache.orc.OrcProto.Type> types,
                                             List<org.apache.hadoop.hive.ql.io.sarg.PredicateLeaf> sargLeaves)

getCompressionCodec

public org.apache.orc.CompressionCodec getCompressionCodec()

getMaxDiskRangeChunkLimit

public int getMaxDiskRangeChunkLimit()