org.apache.paimon.schema

Class SchemaEvolutionUtil

java.lang.Object

org.apache.paimon.schema.SchemaEvolutionUtil

public class SchemaEvolutionUtil
extends Object

Utils for schema evolution.

Constructor Summary

Constructors

Constructor and Description
SchemaEvolutionUtil()

Method Summary

Modifier and Type	Method and Description
static IndexCastMapping	createIndexCastMapping(List<DataField> tableFields, List<DataField> dataFields) Create index mapping from table fields to underlying data fields.
static int[]	createIndexMapping(List<DataField> tableFields, List<DataField> dataFields) Create index mapping from table fields to underlying data fields.
static List<Predicate>	devolveDataFilters(List<DataField> tableFields, List<DataField> dataFields, List<Predicate> filters) When pushing down filters after schema evolution, we should devolve the literals from new types (in dataFields) to original types (in tableFields).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SchemaEvolutionUtil

public SchemaEvolutionUtil()

Method Detail

createIndexMapping

@Nullable
public static int[] createIndexMapping(List<DataField> tableFields,
                                                 List<DataField> dataFields)

Create index mapping from table fields to underlying data fields. For example, the table and data fields are as follows

• table fields: 1->c, 6->b, 3->a
• data fields: 1->a, 3->c

We can get the index mapping [0, -1, 1], in which 0 is the index of table field 1->c in data fields, -1 is the index of 6->b in data fields and 1 is the index of 3->a in data fields.

Parameters:

tableFields - the fields of table

dataFields - the fields of underlying data

Returns:

the index mapping

createIndexCastMapping

public static IndexCastMapping createIndexCastMapping(List<DataField> tableFields,
                                                      List<DataField> dataFields)

Create index mapping from table fields to underlying data fields.

devolveDataFilters

@Nullable
public static List<Predicate> devolveDataFilters(List<DataField> tableFields,
                                                           List<DataField> dataFields,
                                                           List<Predicate> filters)

When pushing down filters after schema evolution, we should devolve the literals from new types (in dataFields) to original types (in tableFields). We will visit all predicate in filters, reset its field index, name and type, and ignore predicate if the field is not exist.

Parameters:

tableFields - the table fields

dataFields - the underlying data fields

filters - the filters

Returns:

the data filters