engine

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 3, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AppendSnapshotCatalog added in v1.4.8 

func AppendSnapshotCatalog(s store.ObjectStore, summary core.SnapshotSummary)

AppendSnapshotCatalog loads the current catalog, appends a new summary, and persists it. This is best-effort; errors are logged but not propagated.

func CheckRepoLock added in v1.3.0 

func CheckRepoLock(ctx context.Context, s store.ObjectStore) error

CheckRepoLock returns an error if the repository is currently locked by another operation (either exclusive or shared). Callers that only need to detect conflicts use this instead of acquiring their own lock.

func RemoveFromSnapshotCatalog added in v1.4.8 

func RemoveFromSnapshotCatalog(s store.ObjectStore, refs ...string)

RemoveFromSnapshotCatalog loads the current catalog, removes entries whose refs match, and persists the result. This is best-effort.

func SaveSnapshotCatalog added in v1.4.8 

func SaveSnapshotCatalog(s store.ObjectStore, catalog []core.SnapshotSummary) error

SaveSnapshotCatalog persists the full catalog to the store.

Types

type BackupManager 

type BackupManager struct {
	// contains filtered or unexported fields
}

BackupManager orchestrates a backup: scanning a source for changes, uploading new or modified files, and persisting a snapshot backed by a Merkle-HAMT.

func NewBackupManager 

func NewBackupManager(src store.Source, dest store.ObjectStore, reporter ui.Reporter, hmacKey []byte, opts ...BackupOption) *BackupManager

func (*BackupManager) Run 

func (bm *BackupManager) Run(ctx context.Context) (*RunResult, error)

Run executes a full backup: scan the source for changes, upload new/modified files, build a new HAMT root, and persist a snapshot.

type BackupOption 

type BackupOption func(*backupConfig)

BackupOption configures a backup operation.

func WithBackupDryRun 

func WithBackupDryRun() BackupOption

WithBackupDryRun scans the source and reports what would change without writing to the store.

func WithGenerator 

func WithGenerator(name string) BackupOption

WithGenerator overrides the default generator name in snapshot metadata.

func WithMeta 

func WithMeta(key, value string) BackupOption

WithMeta adds a key-value pair to the snapshot metadata.

func WithTags 

func WithTags(tags ...string) BackupOption

WithTags adds tags to the backup snapshot.

func WithVerbose 

func WithVerbose() BackupOption

WithVerbose enables verbose output during backup.

type ChangeType 

type ChangeType string

ChangeType describes how a file differs between two snapshots. 

const (
	ChangeAdded    ChangeType = "A"
	ChangeRemoved  ChangeType = "D"
	ChangeModified ChangeType = "M"
)

type CheckError added in v1.4.7 

type CheckError struct {
	Key     string // Object key (e.g. "chunk/abc123")
	Type    string // Error category: "missing", "read_error", "corrupt", "parse_error"
	Message string
}

CheckError describes a single integrity error found during a check.

func (CheckError) String added in v1.4.7 

func (e CheckError) String() string

type CheckManager added in v1.4.7 

type CheckManager struct {
	// contains filtered or unexported fields
}

CheckManager verifies the integrity of a repository by walking the full reference chain and checking that every referenced object can be read.

func NewCheckManager added in v1.4.7 

func NewCheckManager(s store.ObjectStore, reporter ui.Reporter) *CheckManager

NewCheckManager creates a CheckManager.

func (*CheckManager) Run added in v1.4.7 

func (cm *CheckManager) Run(ctx context.Context, opts ...CheckOption) (*CheckResult, error)

Run verifies the repository integrity.

type CheckOption added in v1.4.7 

type CheckOption func(*checkConfig)

CheckOption configures a check operation.

func WithCheckVerbose added in v1.4.7 

func WithCheckVerbose() CheckOption

WithCheckVerbose logs each verified object.

func WithReadData added in v1.4.7 

func WithReadData() CheckOption

WithReadData enables full byte-level verification: re-hash all chunk data and verify content manifests match their referenced chunks.

func WithSnapshotRef added in v1.4.7 

func WithSnapshotRef(ref string) CheckOption

WithSnapshotRef limits the check to a single snapshot instead of all.

type CheckResult added in v1.4.7 

type CheckResult struct {
	SnapshotsChecked int
	ObjectsVerified  int
	Errors           []CheckError
}

CheckResult holds the outcome of a check operation.

type Chunker 

type Chunker struct {
	// contains filtered or unexported fields
}

Chunker splits a byte stream into content-defined chunks, deduplicates them, and persists the resulting Content object.

func NewChunker 

func NewChunker(s store.ObjectStore, hmacKey []byte) *Chunker

func (*Chunker) CreateContentObject 

func (c *Chunker) CreateContentObject(chunkRefs []string, size int64, contentHash string) (string, error)

CreateContentObject persists a Content object keyed by contentHash and returns its store ref.

func (*Chunker) ProcessStream 

func (c *Chunker) ProcessStream(r io.Reader, onProgress func(int64)) (refs []string, size int64, hash string, err error)

ProcessStream splits r into content-defined chunks and stores each one (skipping duplicates). It returns the ordered chunk refs, total byte count, and the SHA-256 content hash over the raw stream.

onProgress is called after each chunk with the number of raw bytes consumed.

type DiffManager 

type DiffManager struct {
	// contains filtered or unexported fields
}

DiffManager compares two snapshots and reports file-level changes.

func NewDiffManager 

func NewDiffManager(s store.ObjectStore) *DiffManager

func (*DiffManager) Run 

func (dm *DiffManager) Run(ctx context.Context, snapID1, snapID2 string, opts ...DiffOption) (*DiffResult, error)

Run resolves two snapshot IDs and computes the diff.

type DiffOption 

type DiffOption func(*diffConfig)

DiffOption configures a diff operation.

type DiffResult 

type DiffResult struct {
	Ref1    string
	Ref2    string
	Changes []FileChange
}

DiffResult holds the outcome of a diff operation.

type FileChange 

type FileChange struct {
	Type ChangeType
	Path string
	Meta core.FileMeta
}

FileChange is a single entry in a diff report.

type ForgetManager 

type ForgetManager struct {
	// contains filtered or unexported fields
}

ForgetManager removes a snapshot and its index pointers, optionally pruning unreachable objects afterwards.

func NewForgetManager 

func NewForgetManager(s store.ObjectStore, reporter ui.Reporter) *ForgetManager

func (*ForgetManager) Run 

func (fm *ForgetManager) Run(ctx context.Context, snapshotID string, opts ...ForgetOption) (*ForgetResult, error)

Run removes the snapshot identified by snapshotID.

func (*ForgetManager) RunPolicy 

func (fm *ForgetManager) RunPolicy(ctx context.Context, opts ...ForgetOption) (*PolicyResult, error)

RunPolicy applies a retention policy to all snapshots and removes those not matched by any keep rule. Use WithKeepLast, WithKeepDaily, etc. to configure.

type ForgetOption 

type ForgetOption func(*forgetConfig)

ForgetOption configures a forget operation.

func WithDryRun 

func WithDryRun() ForgetOption

WithDryRun shows what would be removed without actually removing anything.

func WithFilterAccount 

func WithFilterAccount(account string) ForgetOption

WithFilterAccount restricts the policy to snapshots from this account.

func WithFilterPath 

func WithFilterPath(path string) ForgetOption

WithFilterPath restricts the policy to snapshots from this path.

func WithFilterSource 

func WithFilterSource(source string) ForgetOption

WithFilterSource restricts the policy to snapshots from this source type.

func WithFilterTag 

func WithFilterTag(tag string) ForgetOption

WithFilterTag restricts the policy to snapshots that have this tag.

func WithGroupBy 

func WithGroupBy(fields string) ForgetOption

WithGroupBy sets the fields used to group snapshots for policy application. Comma-separated list of: source, account, path, tags. Empty string disables grouping.

func WithKeepDaily 

func WithKeepDaily(n int) ForgetOption

WithKeepDaily keeps one snapshot per day for the last n days that have snapshots.

func WithKeepHourly 

func WithKeepHourly(n int) ForgetOption

WithKeepHourly keeps one snapshot per hour for the last n hours that have snapshots.

func WithKeepLast 

func WithKeepLast(n int) ForgetOption

WithKeepLast keeps the n most recent snapshots.

func WithKeepMonthly 

func WithKeepMonthly(n int) ForgetOption

WithKeepMonthly keeps one snapshot per month for the last n months that have snapshots.

func WithKeepWeekly 

func WithKeepWeekly(n int) ForgetOption

WithKeepWeekly keeps one snapshot per ISO week for the last n weeks that have snapshots.

func WithKeepYearly 

func WithKeepYearly(n int) ForgetOption

WithKeepYearly keeps one snapshot per year for the last n years that have snapshots.

func WithPrune 

func WithPrune() ForgetOption

WithPrune runs a prune pass after forgetting snapshots.

type ForgetPolicy 

type ForgetPolicy struct {
	KeepLast    int
	KeepHourly  int
	KeepDaily   int
	KeepWeekly  int
	KeepMonthly int
	KeepYearly  int
}

ForgetPolicy describes which snapshots to keep.

func (ForgetPolicy) IsEmpty 

func (p ForgetPolicy) IsEmpty() bool

func (ForgetPolicy) String 

func (p ForgetPolicy) String() string

type ForgetResult 

type ForgetResult struct {
	Prune *PruneResult // nil when prune was not requested
}

ForgetResult holds the outcome of a forget operation.

type GroupKey 

type GroupKey struct {
	Source  string
	Account string
	Path    string
	Tags    string // sorted, comma-joined
}

GroupKey identifies a group of snapshots for policy application.

func (GroupKey) String 

func (k GroupKey) String() string

type KeepReason 

type KeepReason struct {
	Entry   SnapshotEntry
	Reasons []string
}

KeepReason pairs a snapshot with the reasons it was kept.

type ListManager 

type ListManager struct {
	// contains filtered or unexported fields
}

ListManager enumerates all available snapshots.

func NewListManager 

func NewListManager(s store.ObjectStore) *ListManager

func (*ListManager) Run 

func (lm *ListManager) Run(ctx context.Context, opts ...ListOption) (*ListResult, error)

Run lists every snapshot in the store.

type ListOption 

type ListOption func(*listConfig)

ListOption configures a list operation.

type ListResult 

type ListResult struct {
	Snapshots []SnapshotEntry
}

ListResult holds the snapshots returned by a list operation.

type LockHandle added in v1.3.0 

type LockHandle struct {
	// contains filtered or unexported fields
}

LockHandle is returned by AcquireRepoLock and must be released when the operation completes. A background goroutine refreshes the lock every refreshRate so that the TTL stays short (fast recovery on crash) while supporting arbitrarily long operations.

func AcquireRepoLock added in v1.3.0 

func AcquireRepoLock(ctx context.Context, s store.ObjectStore, operation string) (*LockHandle, error)

AcquireRepoLock creates an exclusive lock for operation. If another non-expired lock exists (exclusive or shared), the call returns an error.

To mitigate TOCTOU races on stores without conditional writes, the lock is written and then immediately re-read to verify this process still owns it.

func AcquireSharedLock added in v1.3.0 

func AcquireSharedLock(ctx context.Context, s store.ObjectStore, operation string) (*LockHandle, error)

AcquireSharedLock creates a shared lock for an operation (like backup or restore). If an exclusive lock exists, the call returns an error. Multiple shared locks can exist simultaneously.

func (*LockHandle) Release added in v1.3.0 

func (h *LockHandle) Release()

Release stops the refresh goroutine and deletes the lock only if this handle still owns it (prevents deleting a lock acquired by another process after ours expired).

type LsSnapshotManager 

type LsSnapshotManager struct {
	// contains filtered or unexported fields
}

LsSnapshotManager lists the file tree of a single snapshot.

func NewLsSnapshotManager 

func NewLsSnapshotManager(s store.ObjectStore) *LsSnapshotManager

func (*LsSnapshotManager) Run 

func (lm *LsSnapshotManager) Run(ctx context.Context, snapshotID string, opts ...LsSnapshotOption) (*LsSnapshotResult, error)

Run resolves the snapshot, collects metadata, and returns the tree structure.

type LsSnapshotOption 

type LsSnapshotOption func(*lsSnapshotConfig)

LsSnapshotOption configures an ls-snapshot operation.

type LsSnapshotResult 

type LsSnapshotResult struct {
	Ref       string
	Snapshot  core.Snapshot
	RootRefs  []string
	RefToMeta map[string]core.FileMeta
	ChildRefs map[string][]string
}

LsSnapshotResult holds the data returned by an ls-snapshot operation.

type PolicyGroupResult 

type PolicyGroupResult struct {
	Key    GroupKey
	Keep   []KeepReason
	Remove []SnapshotEntry
}

PolicyGroupResult holds the policy evaluation result for a single group.

type PolicyResult 

type PolicyResult struct {
	Groups []PolicyGroupResult
	Prune  *PruneResult
}

PolicyResult holds the outcome of a policy-based forget operation.

type PruneManager 

type PruneManager struct {
	// contains filtered or unexported fields
}

PruneManager implements mark-and-sweep garbage collection over the object store.

func NewPruneManager 

func NewPruneManager(s store.ObjectStore, reporter ui.Reporter) *PruneManager

func (*PruneManager) Run 

func (pm *PruneManager) Run(ctx context.Context, opts ...PruneOption) (*PruneResult, error)

type PruneOption 

type PruneOption func(*pruneConfig)

func WithPruneDryRun 

func WithPruneDryRun() PruneOption

func WithPruneVerbose 

func WithPruneVerbose() PruneOption

type PruneResult 

type PruneResult struct {
	BytesReclaimed int64
	ObjectsDeleted int
	ObjectsScanned int
	DryRun         bool
}

type RepoLock added in v1.3.0 

type RepoLock struct {
	Operation  string `json:"operation"`
	Holder     string `json:"holder"`
	AcquiredAt string `json:"acquired_at"`
	ExpiresAt  string `json:"expires_at"`
	IsShared   bool   `json:"is_shared,omitempty"`
}

RepoLock is the JSON payload stored at index/lock (or index/lock/shared/<uuid>).

func BreakRepoLock added in v1.3.0 

func BreakRepoLock(ctx context.Context, s store.ObjectStore) ([]*RepoLock, error)

BreakRepoLock forcibly removes the repository lock (exclusive and shared) regardless of who holds it. Returns the locks that were removed.

type RestoreManager 

type RestoreManager struct {
	// contains filtered or unexported fields
}

RestoreManager recreates a snapshot's file tree as a ZIP archive.

func NewRestoreManager 

func NewRestoreManager(s store.ObjectStore, reporter ui.Reporter) *RestoreManager

func (*RestoreManager) Run 

func (rm *RestoreManager) Run(ctx context.Context, w io.Writer, snapshotRef string, opts ...RestoreOption) (*RestoreResult, error)

Run writes the snapshot's file tree as a ZIP archive to w. snapshotRef can be "", "latest", a bare hash, or "snapshot/<hash>".

type RestoreOption 

type RestoreOption func(*restoreConfig)

RestoreOption configures a restore operation.

func WithRestoreDryRun 

func WithRestoreDryRun() RestoreOption

WithRestoreDryRun resolves the snapshot and reports what would be restored without writing the archive.

func WithRestorePath added in v1.4.7 

func WithRestorePath(p string) RestoreOption

WithRestorePath limits the restore to files matching the given path. If the path ends with "/", all files under that subtree are included. Otherwise, only the file with the exact path is restored.

func WithRestoreVerbose 

func WithRestoreVerbose() RestoreOption

WithRestoreVerbose logs each file/dir being written.

type RestoreResult 

type RestoreResult struct {
	SnapshotRef  string
	Root         string
	FilesWritten int
	DirsWritten  int
	BytesWritten int64
	Errors       int
	DryRun       bool
}

RestoreResult holds the outcome of a restore operation.

type RunResult 

type RunResult struct {
	SnapshotHash     string
	SnapshotRef      string
	Root             string
	FilesNew         int64
	FilesChanged     int64
	FilesUnmodified  int64
	FilesRemoved     int64
	DirsNew          int64
	DirsChanged      int64
	DirsUnmodified   int64
	DirsRemoved      int64
	BytesAddedRaw    int64
	BytesAddedStored int64
	Duration         time.Duration
	DryRun           bool
}

RunResult holds the outcome of a successful backup run.

type SnapshotEntry 

type SnapshotEntry struct {
	Ref     string
	Snap    core.Snapshot
	Created time.Time
}

SnapshotEntry is a snapshot loaded for policy evaluation.

func LoadSnapshotCatalog added in v1.4.8 

func LoadSnapshotCatalog(s store.ObjectStore) ([]SnapshotEntry, error)

LoadSnapshotCatalog returns all snapshots, using the catalog index when available and falling back to individual GETs only for snapshots that are missing from the catalog. The catalog is automatically rebuilt/updated whenever a mismatch with the live snapshot keys is detected. Results are sorted newest-first by Created time.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.