org.apache.orc

Class OrcFile

java.lang.Object

org.apache.orc.OrcFile

public class OrcFile
extends Object

Contains factory methods to read or write ORC files.

NOTE: The file was copied and modified to support zstd-jni. This feature is only supported in ORC 2.0, but 2.0 only supports JDK17. We need to support JDK8.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	OrcFile.BloomFilterVersion BloomFilterVersion.
static class	OrcFile.CompressionStrategy CompressionStrategy Enum.
static class	OrcFile.EncodingStrategy EncodingStrategy Enum.
static class	OrcFile.ReaderOptions Orc ReaderOptions.
static class	OrcFile.Version Create a version number for the ORC file format, so that we can add non-forward compatible changes in the future.
static interface	OrcFile.WriterCallback WriterCallback.
static interface	OrcFile.WriterContext WriterContext.
static class	OrcFile.WriterImplementation WriterImplementation Enum.
static class	OrcFile.WriterOptions Options for creating ORC file writers.
static class	OrcFile.WriterVersion Records the version of the writer in terms of which bugs have been fixed.
static class	OrcFile.ZstdCompressOptions ZstdCompressOptions.

Field Summary

Fields

Modifier and Type	Field and Description
static OrcFile.WriterVersion	CURRENT_WRITER The WriterVersion for this version of the software.
static String	MAGIC

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	OrcFile()

Method Summary

Modifier and Type	Method and Description
static org.apache.orc.Reader	createReader(org.apache.hadoop.fs.Path path, OrcFile.ReaderOptions options)
static org.apache.orc.Writer	createWriter(org.apache.hadoop.fs.Path path, OrcFile.WriterOptions opts) Create an ORC file writer.
static List<org.apache.hadoop.fs.Path>	mergeFiles(org.apache.hadoop.fs.Path outputPath, OrcFile.WriterOptions options, List<org.apache.hadoop.fs.Path> inputFiles) Merges multiple ORC files that all have the same schema to produce a single ORC file.
static OrcFile.ReaderOptions	readerOptions(org.apache.hadoop.conf.Configuration conf)
static OrcFile.WriterOptions	writerOptions(org.apache.hadoop.conf.Configuration conf) Create a set of writer options based on a configuration.
static OrcFile.WriterOptions	writerOptions(Properties tableProperties, org.apache.hadoop.conf.Configuration conf) Create a set of write options based on a set of table properties and configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAGIC

public static final String MAGIC

See Also:

Constant Field Values

CURRENT_WRITER

public static final OrcFile.WriterVersion CURRENT_WRITER

The WriterVersion for this version of the software.

Constructor Detail

OrcFile

protected OrcFile()

Method Detail

readerOptions

public static OrcFile.ReaderOptions readerOptions(org.apache.hadoop.conf.Configuration conf)

createReader

public static org.apache.orc.Reader createReader(org.apache.hadoop.fs.Path path,
                                                 OrcFile.ReaderOptions options)
                                          throws IOException

Throws:

IOException

writerOptions

public static OrcFile.WriterOptions writerOptions(org.apache.hadoop.conf.Configuration conf)

Create a set of writer options based on a configuration.

Parameters:

conf - the configuration to use for values

Returns:

A WriterOptions object that can be modified

writerOptions

public static OrcFile.WriterOptions writerOptions(Properties tableProperties,
                                                  org.apache.hadoop.conf.Configuration conf)

Create a set of write options based on a set of table properties and configuration.

Parameters:

tableProperties - the properties of the table

conf - the configuration of the query

Returns:

a WriterOptions object that can be modified

createWriter

public static org.apache.orc.Writer createWriter(org.apache.hadoop.fs.Path path,
                                                 OrcFile.WriterOptions opts)
                                          throws IOException

Create an ORC file writer. This is the public interface for creating writers going forward and new options will only be added to this method.

Parameters:

path - filename to write to

opts - the options

Returns:

a new ORC file writer

Throws:

IOException

mergeFiles

public static List<org.apache.hadoop.fs.Path> mergeFiles(org.apache.hadoop.fs.Path outputPath,
                                                         OrcFile.WriterOptions options,
                                                         List<org.apache.hadoop.fs.Path> inputFiles)
                                                  throws IOException

Merges multiple ORC files that all have the same schema to produce a single ORC file. The merge will reject files that aren't compatible with the merged file so the output list may be shorter than the input list. The stripes are copied as serialized byte buffers. The user metadata are merged and files that disagree on the value associated with a key will be rejected.

Parameters:

outputPath - the output file

options - the options for writing with although the options related to the input files' encodings are overridden

inputFiles - the list of files to merge

Returns:

the list of files that were successfully merged

Throws:

IOException