Modifier and Type | Class and Description |
---|---|
class |
Changelog
The metadata of changelog.
|
Modifier and Type | Method and Description |
---|---|
static Snapshot |
Snapshot.fromJson(String json) |
static Snapshot |
Snapshot.fromPath(FileIO fileIO,
Path path) |
static Snapshot |
Snapshot.tryFromPath(FileIO fileIO,
Path path) |
Modifier and Type | Method and Description |
---|---|
void |
FileStore.setSnapshotCache(org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Path,Snapshot> cache) |
Constructor and Description |
---|
Changelog(Snapshot snapshot) |
Modifier and Type | Method and Description |
---|---|
protected SnapshotReader.Plan |
LookupDataTableScan.handleOverwriteSnapshot(Snapshot snapshot) |
SnapshotReader.Plan |
CompactionDiffFollowUpScanner.scan(Snapshot snapshot,
SnapshotReader snapshotReader) |
boolean |
CompactionDiffFollowUpScanner.shouldScanSnapshot(Snapshot snapshot) |
Modifier and Type | Method and Description |
---|---|
Snapshot |
StaticFileStoreSplitEnumerator.snapshot() |
Constructor and Description |
---|
StaticFileStoreSplitEnumerator(org.apache.flink.api.connector.source.SplitEnumeratorContext<FileStoreSourceSplit> context,
Snapshot snapshot,
SplitAssigner splitAssigner) |
StaticFileStoreSplitEnumerator(org.apache.flink.api.connector.source.SplitEnumeratorContext<FileStoreSourceSplit> context,
Snapshot snapshot,
SplitAssigner splitAssigner,
DynamicPartitionFilteringInfo dynamicPartitionFilteringInfo) |
Modifier and Type | Method and Description |
---|---|
void |
AbstractIcebergCommitCallback.call(List<ManifestEntry> committedEntries,
Snapshot snapshot) |
Modifier and Type | Method and Description |
---|---|
Map<Pair<BinaryRow,Integer>,List<IndexFileMeta>> |
IndexFileHandler.scan(Snapshot snapshot,
String indexType,
Set<BinaryRow> partitions) |
List<IndexManifestEntry> |
IndexFileHandler.scanEntries(Snapshot snapshot,
String indexType,
Set<BinaryRow> partitions) |
Modifier and Type | Method and Description |
---|---|
List<ManifestFileMeta> |
ManifestList.readAllManifests(Snapshot snapshot)
Return all
ManifestFileMeta instances for either data or changelog manifests in this
snapshot. |
List<ManifestFileMeta> |
ManifestList.readChangelogManifests(Snapshot snapshot)
Return a
ManifestFileMeta for each changelog manifest in this snapshot. |
List<ManifestFileMeta> |
ManifestList.readDataManifests(Snapshot snapshot)
Return a
ManifestFileMeta for each data manifest in this snapshot. |
List<ManifestFileMeta> |
ManifestList.readDeltaManifests(Snapshot snapshot)
Return a
ManifestFileMeta for each delta manifest in this snapshot. |
Modifier and Type | Method and Description |
---|---|
void |
TagPreviewCommitCallback.call(List<ManifestEntry> committedEntries,
Snapshot snapshot) |
void |
AddPartitionCommitCallback.call(List<ManifestEntry> committedEntries,
Snapshot snapshot) |
Modifier and Type | Class and Description |
---|---|
class |
FileDeletionBase<T extends Snapshot>
Base class for file deletion including methods for clean data files, manifest files and empty
data directories.
|
Modifier and Type | Field and Description |
---|---|
Snapshot |
ManifestsReader.Result.snapshot |
Modifier and Type | Method and Description |
---|---|
Snapshot |
FileStoreScan.Plan.snapshot()
Snapshot of this plan, return null if the table is empty or the manifest list is
specified.
|
Modifier and Type | Method and Description |
---|---|
protected Set<Snapshot> |
OrphanFilesClean.safelyGetAllSnapshots(String branch) |
Modifier and Type | Method and Description |
---|---|
protected void |
FileDeletionBase.addMergedDataFiles(Map<BinaryRow,Map<Integer,Set<String>>> dataFiles,
Snapshot snapshot)
NOTE: This method is used for building data file skipping set.
|
void |
SnapshotDeletion.cleanUnusedDataFiles(Snapshot snapshot,
java.util.function.Predicate<ManifestEntry> skipper) |
void |
TagDeletion.cleanUnusedDataFiles(Snapshot taggedSnapshot,
java.util.function.Predicate<ManifestEntry> skipper) |
void |
FileDeletionBase.cleanUnusedIndexManifests(Snapshot snapshot,
Set<String> skippingSet) |
void |
SnapshotDeletion.cleanUnusedManifests(Snapshot snapshot,
Set<String> skippingSet) |
void |
TagDeletion.cleanUnusedManifests(Snapshot taggedSnapshot,
Set<String> skippingSet) |
protected void |
FileDeletionBase.cleanUnusedManifests(Snapshot snapshot,
Set<String> skippingSet,
boolean deleteDataManifestLists,
boolean deleteChangelog) |
void |
FileDeletionBase.cleanUnusedStatisticsManifests(Snapshot snapshot,
Set<String> skippingSet) |
protected void |
OrphanFilesClean.collectWithoutDataFile(String branch,
Snapshot snapshot,
java.util.function.Consumer<String> usedFileConsumer,
java.util.function.Consumer<String> manifestConsumer) |
protected void |
OrphanFilesClean.collectWithoutDataFileWithManifestFlag(String branch,
Snapshot snapshot,
java.util.function.Consumer<Pair<String,Boolean>> usedFileWithFlagConsumer) |
java.util.function.Predicate<ManifestEntry> |
TagDeletion.dataFileSkipper(Snapshot fromSnapshot) |
Set<String> |
FileDeletionBase.manifestSkippingSet(Snapshot skippingSnapshot)
Changelogs were not checked.
|
ManifestsReader.Result |
ManifestsReader.read(Snapshot specifiedSnapshot,
ScanMode scanMode) |
protected Collection<ManifestEntry> |
FileDeletionBase.readMergedDataFiles(Snapshot snapshot) |
default Long |
FileStoreScan.totalRecordCount(Snapshot snapshot)
Return record count of all changes occurred in this snapshot given the scan.
|
protected List<String> |
FileDeletionBase.tryReadDataManifests(Snapshot snapshot) |
FileStoreScan |
FileStoreScan.withSnapshot(Snapshot snapshot) |
FileStoreScan |
AbstractFileStoreScan.withSnapshot(Snapshot snapshot) |
Modifier and Type | Method and Description |
---|---|
java.util.function.Predicate<ManifestEntry> |
FileDeletionBase.createDataFileSkipperForTags(List<Snapshot> taggedSnapshots,
long expiringSnapshotId) |
java.util.function.Predicate<ManifestEntry> |
TagDeletion.dataFileSkipper(List<Snapshot> fromSnapshots) |
Set<String> |
ChangelogDeletion.manifestSkippingSet(List<Snapshot> skippingSnapshots) |
Set<String> |
FileDeletionBase.manifestSkippingSet(List<Snapshot> skippingSnapshots) |
Constructor and Description |
---|
Result(Snapshot snapshot,
List<ManifestFileMeta> allManifests,
List<ManifestFileMeta> filteredManifests) |
Modifier and Type | Method and Description |
---|---|
Snapshot |
PrivilegedFileStoreTable.snapshot(long snapshotId) |
Modifier and Type | Method and Description |
---|---|
void |
PrivilegedFileStore.setSnapshotCache(org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Path,Snapshot> cache) |
Modifier and Type | Method and Description |
---|---|
Optional<Statistics> |
StatsFileHandler.readStats(Snapshot snapshot) |
Modifier and Type | Method and Description |
---|---|
Snapshot |
Table.snapshot(long snapshotId)
Get the
Snapshot from snapshot id. |
default Snapshot |
FormatTable.snapshot(long snapshotId) |
default Snapshot |
ReadonlyTable.snapshot(long snapshotId) |
Snapshot |
DelegatedFileStoreTable.snapshot(long snapshotId) |
Modifier and Type | Method and Description |
---|---|
static List<Snapshot> |
ExpireSnapshotsImpl.findSkippingTags(List<Snapshot> sortedTags,
long beginInclusive,
long endExclusive)
Find the skipping tags in sortedTags for range of [beginInclusive, endExclusive).
|
Modifier and Type | Method and Description |
---|---|
void |
RollbackHelper.cleanLargerThan(Snapshot retainedSnapshot)
Clean snapshots and tags whose id is larger than given snapshot's.
|
Modifier and Type | Method and Description |
---|---|
static List<Snapshot> |
ExpireSnapshotsImpl.findSkippingTags(List<Snapshot> sortedTags,
long beginInclusive,
long endExclusive)
Find the skipping tags in sortedTags for range of [beginInclusive, endExclusive).
|
void |
FileStoreTable.setSnapshotCache(org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Path,Snapshot> cache) |
void |
DelegatedFileStoreTable.setSnapshotCache(org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Path,Snapshot> cache) |
Modifier and Type | Method and Description |
---|---|
void |
CommitCallback.call(List<ManifestEntry> committedEntries,
Snapshot snapshot) |
Modifier and Type | Method and Description |
---|---|
protected SnapshotReader.Plan |
DataTableStreamScan.handleOverwriteSnapshot(Snapshot snapshot) |
Modifier and Type | Method and Description |
---|---|
static Snapshot |
TimeTravelUtil.resolveSnapshot(FileStoreTable table) |
static Snapshot |
TimeTravelUtil.resolveSnapshotFromOptions(CoreOptions options,
SnapshotManager snapshotManager) |
static Snapshot |
StaticFromTimestampStartingScanner.timeTravelToTimestamp(SnapshotManager snapshotManager,
long timestamp) |
static Snapshot |
StaticFromWatermarkStartingScanner.timeTravelToWatermark(SnapshotManager snapshotManager,
long watermark) |
Modifier and Type | Method and Description |
---|---|
default SnapshotReader.Plan |
FollowUpScanner.getOverwriteChangesPlan(Snapshot snapshot,
SnapshotReader snapshotReader) |
SnapshotReader.Plan |
SnapshotReader.readIncrementalDiff(Snapshot before) |
SnapshotReader.Plan |
SnapshotReaderImpl.readIncrementalDiff(Snapshot before) |
SnapshotReader.Plan |
FollowUpScanner.scan(Snapshot snapshot,
SnapshotReader snapshotReader) |
SnapshotReader.Plan |
InputChangelogFollowUpScanner.scan(Snapshot snapshot,
SnapshotReader snapshotReader) |
SnapshotReader.Plan |
CompactionChangelogFollowUpScanner.scan(Snapshot snapshot,
SnapshotReader snapshotReader) |
SnapshotReader.Plan |
AllDeltaFollowUpScanner.scan(Snapshot snapshot,
SnapshotReader snapshotReader) |
SnapshotReader.Plan |
DeltaFollowUpScanner.scan(Snapshot snapshot,
SnapshotReader snapshotReader) |
boolean |
BoundedChecker.shouldEndInput(Snapshot snapshot) |
boolean |
FollowUpScanner.shouldScanSnapshot(Snapshot snapshot) |
boolean |
InputChangelogFollowUpScanner.shouldScanSnapshot(Snapshot snapshot) |
boolean |
CompactionChangelogFollowUpScanner.shouldScanSnapshot(Snapshot snapshot) |
boolean |
AllDeltaFollowUpScanner.shouldScanSnapshot(Snapshot snapshot) |
boolean |
DeltaFollowUpScanner.shouldScanSnapshot(Snapshot snapshot) |
SnapshotReader |
SnapshotReader.withSnapshot(Snapshot snapshot) |
SnapshotReader |
SnapshotReaderImpl.withSnapshot(Snapshot snapshot) |
Modifier and Type | Method and Description |
---|---|
Snapshot |
FileMonitorTable.snapshot(long snapshotId) |
Snapshot |
CompactBucketsTable.snapshot(long snapshotId) |
Snapshot |
AuditLogTable.snapshot(long snapshotId) |
Snapshot |
ReadOptimizedTable.snapshot(long snapshotId) |
Modifier and Type | Class and Description |
---|---|
class |
Tag
Snapshot with tagCreateTime and tagTimeRetained.
|
Modifier and Type | Method and Description |
---|---|
Snapshot |
Tag.trimToSnapshot() |
Modifier and Type | Method and Description |
---|---|
static Tag |
Tag.fromSnapshotAndTagTtl(Snapshot snapshot,
java.time.Duration tagTimeRetained,
java.time.LocalDateTime tagCreateTime) |
Modifier and Type | Method and Description |
---|---|
Snapshot |
SnapshotManager.earlierOrEqualTimeMills(long timestampMills)
Returns a
Snapshot whoes commit time is earlier than or equal to given timestamp
mills. |
Snapshot |
SnapshotManager.earliestSnapshot() |
Snapshot |
NextSnapshotFetcher.getNextSnapshot(long nextSnapshotId) |
Snapshot |
SnapshotManager.laterOrEqualTimeMills(long timestampMills)
Returns a
Snapshot whoes commit time is later than or equal to given timestamp mills. |
Snapshot |
SnapshotManager.laterOrEqualWatermark(long watermark) |
Snapshot |
SnapshotManager.latestSnapshot() |
Snapshot |
SnapshotManager.snapshot(long snapshotId) |
Snapshot |
TagManager.taggedSnapshot(String tagName)
Get the tagged snapshot by name.
|
Snapshot |
SnapshotManager.traversalSnapshotsFromLatestSafely(Filter<Snapshot> checker)
Traversal snapshots from latest to earliest safely, this is applied on the writer side
because the committer may delete obsolete snapshots, which may cause the writer to encounter
unreadable snapshots.
|
Snapshot |
SnapshotManager.tryGetSnapshot(long snapshotId) |
Modifier and Type | Method and Description |
---|---|
List<Snapshot> |
SnapshotManager.findSnapshotsForIdentifiers(String user,
List<Long> identifiers)
Find the snapshot of the specified identifiers written by the specified user.
|
Optional<Snapshot> |
SnapshotManager.latestSnapshotOfUser(String user) |
List<Snapshot> |
SnapshotManager.safelyGetAllSnapshots()
If
FileNotFoundException is thrown when reading the snapshot file, this snapshot may
be deleted by other processes, so just skip this snapshot. |
Iterator<Snapshot> |
SnapshotManager.snapshots() |
Iterator<Snapshot> |
SnapshotManager.snapshotsWithId(List<Long> snapshotIds) |
Iterator<Snapshot> |
SnapshotManager.snapshotsWithinRange(Optional<Long> optionalMaxSnapshotId,
Optional<Long> optionalMinSnapshotId) |
List<Snapshot> |
TagManager.taggedSnapshots()
Get all tagged snapshots sorted by snapshot id.
|
SortedMap<Snapshot,List<String>> |
TagManager.tags()
Get all tagged snapshots with names sorted by snapshot id.
|
SortedMap<Snapshot,List<String>> |
TagManager.tags(java.util.function.Predicate<String> filter)
Retrieves a sorted map of snapshots filtered based on a provided predicate.
|
Modifier and Type | Method and Description |
---|---|
static void |
SnapshotNotExistException.checkNotNull(Snapshot snapshotId,
String errMsg) |
void |
TagManager.createOrReplaceTag(Snapshot snapshot,
String tagName,
java.time.Duration timeRetained,
List<TagCallback> callbacks) |
void |
TagManager.createTag(Snapshot snapshot,
String tagName,
java.time.Duration timeRetained,
List<TagCallback> callbacks)
Create a tag from given snapshot and save it in the storage.
|
void |
TagManager.replaceTag(Snapshot snapshot,
String tagName,
java.time.Duration timeRetained)
Replace a tag from given snapshot and save it in the storage.
|
Modifier and Type | Method and Description |
---|---|
static int |
SnapshotManager.findPreviousOrEqualSnapshot(List<Snapshot> sortedSnapshots,
long targetSnapshotId) |
static int |
SnapshotManager.findPreviousSnapshot(List<Snapshot> sortedSnapshots,
long targetSnapshotId) |
Long |
SnapshotManager.pickOrLatest(java.util.function.Predicate<Snapshot> predicate) |
Snapshot |
SnapshotManager.traversalSnapshotsFromLatestSafely(Filter<Snapshot> checker)
Traversal snapshots from latest to earliest safely, this is applied on the writer side
because the committer may delete obsolete snapshots, which may cause the writer to encounter
unreadable snapshots.
|
Constructor and Description |
---|
SnapshotManager(FileIO fileIO,
Path tablePath,
String branchName,
org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Path,Snapshot> cache) |
Copyright © 2023–2024 The Apache Software Foundation. All rights reserved.