perf: build lower layer squashes only when a scan reads them (#5492)

* perf: keep only the image squash when scanning with the squashed scope

- stereoscope builds a squashed tree for every layer, each a full copy of the one below, so memory
  grows with layers x files. The squashed scope only ever reads the top one.
- new `GetSourceConfig.WithImageSquashOnly(bool)` (and `sourceproviders.Config` field) reads images
  with stereoscope's `image.WithImageSquashOnly`, scoped to that one read
- the CLI turns it on when `--scope squashed` (the default)
- an image source read this way returns an error from `FileResolver` for `all-layers` and
  `deep-squashed` instead of panicking on the missing layer squashes

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(deps): temporarily pin stereoscope to anchore/stereoscope#731

Replaces stereoscope with the PR head on the contributor fork so this builds. Drop the replace and
bump to a released version once that PR merges.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(deps): bump stereoscope to main for WithImageSquashOnly

Drops the temporary replace to the contributor fork now that anchore/stereoscope#731 has merged.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* revert: drop the squash-only image read and bump stereoscope

stereoscope is replacing `image.WithImageSquashOnly` with lazy layer squashes, where every layer keeps a usable squash. That removes the need for `GetSourceConfig.WithImageSquashOnly` and the scope guard in `FileResolver`, so both go.

- bumps stereoscope to `feat/lazy-layer-squashes` (anchore/stereoscope `43c8f7e`), pending that PR merging

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* perf: build lower layer squashes only when a scan reads them

The image source provider now reads images with `image.WithLazyLayerSquashes`: the image squash is built during the read, and each lower layer's squash only when something reads it. A squashed scan never does, so it skips a full file tree copy per layer.

- squashed scans: faster, much less memory on images with many layers
- `scan`/`attest` with all-layers or deep-squashed stay eager (an internal ctx flag), since eager squashing overlaps with fetching and indexing and is faster when every layer gets read
- library callers going through `syft.GetSource` get lazy for every scope, since the scope isn't known when the image is read. All-layers is still correct, just slower
- no public API change

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* pull in stereoscope@main

Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
This commit is contained in:
Alex Goodman
2026-11-04 15:12:25 -05:00
committed by GitHub
co-authored by Will Murphy
parent 551442b1bc
commit fd712ee71a
5 changed files with 108 additions and 7 deletions
+5
View File
@@ -260,6 +260,11 @@ func getSource(ctx context.Context, opts *options.Catalog, userInput string, sou
cfg = cfg.WithDigestAlgorithms(hashers...)
}
if source.ParseScope(opts.Scope) != source.SquashedScope {
// every other scope reads each layer's squash, which is cheaper to build while the image is read
ctx = internal.WithEagerLayerSquashes(ctx)
}
src, err := syft.GetSource(ctx, userInput, cfg)
if err != nil {
return nil, fmt.Errorf("could not determine source: %w", err)