You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Remove temporary band-aid introduced via #1781. Once we upload state checkpoints to the object store, they are accessible to all partition processor replicas. Hence, it is theoretically possible to trim the log once the checkpoint was stored. However, if some replicas are lagging behind, this can lead to them encountering trim gaps. So either, we wait until all replicas have reached the lsn of the checkpoint before trimming or we add support to handle trim gaps by downloading the latest checkpoint.
The text was updated successfully, but these errors were encountered:
tillrohrmann
changed the title
Reenable proper trimming based on completed snapshots
Reenable proper trimming based on completed checkpoints
Nov 1, 2024
What the partition processor needs to report to the CC is the last snapshotted lsn. Based on this information, the CC can make a decision when it is safe to trim the log.
Remove temporary band-aid introduced via #1781. Once we upload state checkpoints to the object store, they are accessible to all partition processor replicas. Hence, it is theoretically possible to trim the log once the checkpoint was stored. However, if some replicas are lagging behind, this can lead to them encountering trim gaps. So either, we wait until all replicas have reached the lsn of the checkpoint before trimming or we add support to handle trim gaps by downloading the latest checkpoint.
The text was updated successfully, but these errors were encountered: