org.apache.paimon.utils

Class SnapshotManager

java.lang.Object

org.apache.paimon.utils.SnapshotManager

All Implemented Interfaces:

Serializable

public class SnapshotManager
extends Object
implements Serializable

Manager for Snapshot, providing utility methods related to paths and snapshot hints.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static int	EARLIEST_SNAPSHOT_DEFAULT_RETRY_NUM
static String	SNAPSHOT_PREFIX

Constructor Summary

Constructors

Constructor and Description
SnapshotManager(FileIO fileIO, Path tablePath, String branchName, SnapshotLoader snapshotLoader, org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Path,Snapshot> cache)

Method Summary

Modifier and Type	Method and Description
String	branch()
void	commitEarliestHint(long snapshotId)
void	commitLatestHint(long snapshotId)
SnapshotManager	copyWithBranch(String branchName)
void	deleteLatestHint()
void	deleteSnapshot(long snapshotId)
Snapshot	earlierOrEqualTimeMills(long timestampMills) Returns a Snapshot whoes commit time is earlier than or equal to given timestamp mills.
Snapshot	earlierOrEqualWatermark(long watermark)
boolean	earliestFileNotExists()
Snapshot	earliestSnapshot()
Long	earliestSnapshotId()
FileIO	fileIO()
static int	findPreviousOrEqualSnapshot(List<Snapshot> sortedSnapshots, long targetSnapshotId)
static int	findPreviousSnapshot(List<Snapshot> sortedSnapshots, long targetSnapshotId)
List<Snapshot>	findSnapshotsForIdentifiers(String user, List<Long> identifiers) Find the snapshot of the specified identifiers written by the specified user.
void	invalidateCache()
Snapshot	laterOrEqualTimeMills(long timestampMills) Returns a Snapshot whoes commit time is later than or equal to given timestamp mills.
Snapshot	laterOrEqualWatermark(long watermark)
Snapshot	latestSnapshot()
Long	latestSnapshotId()
Optional<Snapshot>	latestSnapshotOfUser(String user)
Long	pickOrLatest(java.util.function.Predicate<Snapshot> predicate)
List<Snapshot>	safelyGetAllSnapshots() If FileNotFoundException is thrown when reading the snapshot file, this snapshot may be deleted by other processes, so just skip this snapshot.
Snapshot	snapshot(long snapshotId)
long	snapshotCount()
Path	snapshotDirectory()
boolean	snapshotExists(long snapshotId)
Path	snapshotPath(long snapshotId)
List<Path>	snapshotPaths(java.util.function.Predicate<Long> predicate)
Iterator<Snapshot>	snapshots()
Iterator<Snapshot>	snapshotsWithId(List<Long> snapshotIds)
Iterator<Snapshot>	snapshotsWithinRange(Optional<Long> optionalMaxSnapshotId, Optional<Long> optionalMinSnapshotId)
Path	tablePath()
Snapshot	traversalSnapshotsFromLatestSafely(Filter<Snapshot> checker) Traversal snapshots from latest to earliest safely, this is applied on the writer side because the committer may delete obsolete snapshots, which may cause the writer to encounter unreadable snapshots.
Snapshot	tryGetSnapshot(long snapshotId)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SNAPSHOT_PREFIX

public static final String SNAPSHOT_PREFIX

See Also:

Constant Field Values

EARLIEST_SNAPSHOT_DEFAULT_RETRY_NUM

public static final int EARLIEST_SNAPSHOT_DEFAULT_RETRY_NUM

See Also:

Constant Field Values

Constructor Detail

SnapshotManager

public SnapshotManager(FileIO fileIO,
                       Path tablePath,
                       @Nullable
                       String branchName,
                       @Nullable
                       SnapshotLoader snapshotLoader,
                       @Nullable
                       org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Path,Snapshot> cache)

Method Detail

copyWithBranch

public SnapshotManager copyWithBranch(String branchName)

fileIO

public FileIO fileIO()

tablePath

public Path tablePath()

branch

public String branch()

snapshotPath

public Path snapshotPath(long snapshotId)

snapshotDirectory

public Path snapshotDirectory()

invalidateCache

public void invalidateCache()

snapshot

public Snapshot snapshot(long snapshotId)

tryGetSnapshot

public Snapshot tryGetSnapshot(long snapshotId)
                        throws FileNotFoundException

Throws:

FileNotFoundException

snapshotExists

public boolean snapshotExists(long snapshotId)

deleteSnapshot

public void deleteSnapshot(long snapshotId)

latestSnapshot

@Nullable
public Snapshot latestSnapshot()

latestSnapshotId

@Nullable
public Long latestSnapshotId()

earliestSnapshot

@Nullable
public Snapshot earliestSnapshot()

earliestFileNotExists

public boolean earliestFileNotExists()

earliestSnapshotId

@Nullable
public Long earliestSnapshotId()

pickOrLatest

@Nullable
public Long pickOrLatest(java.util.function.Predicate<Snapshot> predicate)

earlierOrEqualTimeMills

@Nullable
public Snapshot earlierOrEqualTimeMills(long timestampMills)

Returns a Snapshot whoes commit time is earlier than or equal to given timestamp mills. If there is no such a snapshot, returns null.

laterOrEqualTimeMills

@Nullable
public Snapshot laterOrEqualTimeMills(long timestampMills)

Returns a Snapshot whoes commit time is later than or equal to given timestamp mills. If there is no such a snapshot, returns null.

earlierOrEqualWatermark

@Nullable
public Snapshot earlierOrEqualWatermark(long watermark)

laterOrEqualWatermark

@Nullable
public Snapshot laterOrEqualWatermark(long watermark)

snapshotCount

public long snapshotCount()
                   throws IOException

Throws:

IOException

snapshots

public Iterator<Snapshot> snapshots()
                             throws IOException

Throws:

IOException

snapshotPaths

public List<Path> snapshotPaths(java.util.function.Predicate<Long> predicate)
                         throws IOException

Throws:

IOException

snapshotsWithId

public Iterator<Snapshot> snapshotsWithId(List<Long> snapshotIds)

snapshotsWithinRange

public Iterator<Snapshot> snapshotsWithinRange(Optional<Long> optionalMaxSnapshotId,
                                               Optional<Long> optionalMinSnapshotId)

safelyGetAllSnapshots

public List<Snapshot> safelyGetAllSnapshots()
                                     throws IOException

If FileNotFoundException is thrown when reading the snapshot file, this snapshot may be deleted by other processes, so just skip this snapshot.

Throws:

IOException

latestSnapshotOfUser

public Optional<Snapshot> latestSnapshotOfUser(String user)

findSnapshotsForIdentifiers

public List<Snapshot> findSnapshotsForIdentifiers(@Nonnull
                                                  String user,
                                                  List<Long> identifiers)

Find the snapshot of the specified identifiers written by the specified user.

traversalSnapshotsFromLatestSafely

@Nullable
public Snapshot traversalSnapshotsFromLatestSafely(Filter<Snapshot> checker)

Traversal snapshots from latest to earliest safely, this is applied on the writer side because the committer may delete obsolete snapshots, which may cause the writer to encounter unreadable snapshots.

findPreviousSnapshot

public static int findPreviousSnapshot(List<Snapshot> sortedSnapshots,
                                       long targetSnapshotId)

findPreviousOrEqualSnapshot

public static int findPreviousOrEqualSnapshot(List<Snapshot> sortedSnapshots,
                                              long targetSnapshotId)

deleteLatestHint

public void deleteLatestHint()
                      throws IOException

Throws:

IOException

commitLatestHint

public void commitLatestHint(long snapshotId)
                      throws IOException

Throws:

IOException

commitEarliestHint

public void commitEarliestHint(long snapshotId)
                        throws IOException

Throws:

IOException