org.apache.paimon.format

Class FileFormat

java.lang.Object

org.apache.paimon.format.FileFormat

Direct Known Subclasses:

AvroFileFormat, OrcFileFormat, ParquetFileFormat

public abstract class FileFormat
extends Object

Factory class which creates reader and writer factories for specific file format.

NOTE: This class must be thread safe.

Field Summary

Fields

Modifier and Type	Field and Description
protected String	formatIdentifier

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	FileFormat(String formatIdentifier)

Method Summary

Modifier and Type	Method and Description
FormatReaderFactory	createReaderFactory(RowType rowType)
abstract FormatReaderFactory	createReaderFactory(RowType projectedRowType, List<Predicate> filters) Create a FormatReaderFactory from the type, with projection pushed down.
Optional<SimpleStatsExtractor>	createStatsExtractor(RowType type, SimpleColStatsCollector.Factory[] statsCollectors)
abstract FormatWriterFactory	createWriterFactory(RowType type) Create a FormatWriterFactory from the type.
static FileFormat	fromIdentifier(String identifier, FileFormatFactory.FormatContext context) Create a FileFormat from format identifier and format options.
static FileFormat	fromIdentifier(String identifier, Options options)
static FileFormat	getFileFormat(Options options, String formatIdentifier)
String	getFormatIdentifier()
abstract void	validateDataFields(RowType rowType) Validate data field type supported or not.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

formatIdentifier

protected String formatIdentifier

Constructor Detail

FileFormat

protected FileFormat(String formatIdentifier)

Method Detail

getFormatIdentifier

public String getFormatIdentifier()

createReaderFactory

public abstract FormatReaderFactory createReaderFactory(RowType projectedRowType,
                                                        @Nullable
                                                        List<Predicate> filters)

Create a FormatReaderFactory from the type, with projection pushed down.

Parameters:

projectedRowType - Type with projection.

filters - A list of filters in conjunctive form for filtering on a best-effort basis.

createWriterFactory

public abstract FormatWriterFactory createWriterFactory(RowType type)

Create a FormatWriterFactory from the type.

validateDataFields

public abstract void validateDataFields(RowType rowType)

Validate data field type supported or not.

createReaderFactory

public FormatReaderFactory createReaderFactory(RowType rowType)

createStatsExtractor

public Optional<SimpleStatsExtractor> createStatsExtractor(RowType type,
                                                           SimpleColStatsCollector.Factory[] statsCollectors)

fromIdentifier

@VisibleForTesting
public static FileFormat fromIdentifier(String identifier,
                                                           Options options)

fromIdentifier

public static FileFormat fromIdentifier(String identifier,
                                        FileFormatFactory.FormatContext context)

Create a FileFormat from format identifier and format options.

getFileFormat

public static FileFormat getFileFormat(Options options,
                                       String formatIdentifier)