@ThreadSafe public class OrcFileFormat extends FileFormat
FileFormat
.Modifier and Type | Field and Description |
---|---|
static String |
IDENTIFIER |
formatIdentifier
Constructor and Description |
---|
OrcFileFormat(FileFormatFactory.FormatContext formatContext) |
Modifier and Type | Method and Description |
---|---|
FormatReaderFactory |
createReaderFactory(RowType projectedRowType,
List<Predicate> filters)
Create a
FormatReaderFactory from the type, with projection pushed down. |
Optional<SimpleStatsExtractor> |
createStatsExtractor(RowType type,
SimpleColStatsCollector.Factory[] statsCollectors) |
FormatWriterFactory |
createWriterFactory(RowType type)
The
OrcWriterFactory will create ThreadLocalClassLoaderConfiguration from the
input writer config to avoid classloader leaks. |
Properties |
orcProperties() |
int |
readBatchSize() |
static DataType |
refineDataType(DataType type) |
void |
validateDataFields(RowType rowType)
Validate data field type supported or not.
|
createReaderFactory, fromIdentifier, fromIdentifier, getFormatIdentifier, getIdentifierPrefixOptions
public static final String IDENTIFIER
public OrcFileFormat(FileFormatFactory.FormatContext formatContext)
@VisibleForTesting public Properties orcProperties()
@VisibleForTesting public int readBatchSize()
public Optional<SimpleStatsExtractor> createStatsExtractor(RowType type, SimpleColStatsCollector.Factory[] statsCollectors)
createStatsExtractor
in class FileFormat
public FormatReaderFactory createReaderFactory(RowType projectedRowType, @Nullable List<Predicate> filters)
FileFormat
FormatReaderFactory
from the type, with projection pushed down.createReaderFactory
in class FileFormat
projectedRowType
- Type with projection.filters
- A list of filters in conjunctive form for filtering on a best-effort basis.public void validateDataFields(RowType rowType)
FileFormat
validateDataFields
in class FileFormat
public FormatWriterFactory createWriterFactory(RowType type)
OrcWriterFactory
will create ThreadLocalClassLoaderConfiguration
from the
input writer config to avoid classloader leaks.
TODO: The ThreadLocalClassLoaderConfiguration
in OrcWriterFactory
should
be removed after https://issues.apache.org/jira/browse/ORC-653 is fixed.
createWriterFactory
in class FileFormat
type
- The data type for the writerCopyright © 2023–2024 The Apache Software Foundation. All rights reserved.