perf: build lower layer squashes only when a scan reads them (#5492)

* perf: keep only the image squash when scanning with the squashed scope

- stereoscope builds a squashed tree for every layer, each a full copy of the one below, so memory
  grows with layers x files. The squashed scope only ever reads the top one.
- new `GetSourceConfig.WithImageSquashOnly(bool)` (and `sourceproviders.Config` field) reads images
  with stereoscope's `image.WithImageSquashOnly`, scoped to that one read
- the CLI turns it on when `--scope squashed` (the default)
- an image source read this way returns an error from `FileResolver` for `all-layers` and
  `deep-squashed` instead of panicking on the missing layer squashes

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(deps): temporarily pin stereoscope to anchore/stereoscope#731

Replaces stereoscope with the PR head on the contributor fork so this builds. Drop the replace and
bump to a released version once that PR merges.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(deps): bump stereoscope to main for WithImageSquashOnly

Drops the temporary replace to the contributor fork now that anchore/stereoscope#731 has merged.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* revert: drop the squash-only image read and bump stereoscope

stereoscope is replacing `image.WithImageSquashOnly` with lazy layer squashes, where every layer keeps a usable squash. That removes the need for `GetSourceConfig.WithImageSquashOnly` and the scope guard in `FileResolver`, so both go.

- bumps stereoscope to `feat/lazy-layer-squashes` (anchore/stereoscope `43c8f7e`), pending that PR merging

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* perf: build lower layer squashes only when a scan reads them

The image source provider now reads images with `image.WithLazyLayerSquashes`: the image squash is built during the read, and each lower layer's squash only when something reads it. A squashed scan never does, so it skips a full file tree copy per layer.

- squashed scans: faster, much less memory on images with many layers
- `scan`/`attest` with all-layers or deep-squashed stay eager (an internal ctx flag), since eager squashing overlaps with fetching and indexing and is faster when every layer gets read
- library callers going through `syft.GetSource` get lazy for every scope, since the scope isn't known when the image is read. All-layers is still correct, just slower
- no public API change

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* pull in stereoscope@main

Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
This commit is contained in:
Alex Goodman
2026-11-04 15:12:25 -05:00
committed by GitHub
co-authored by Will Murphy
parent 551442b1bc
commit fd712ee71a
5 changed files with 108 additions and 7 deletions
+5 -7
View File
@@ -39,7 +39,6 @@ require (
github.com/dustin/go-humanize v1.1.0
github.com/elliotchance/phpserialize v1.4.0
github.com/facebookincubator/nvdtools v0.1.5
github.com/github/go-spdx/v2 v2.8.0
github.com/gkampitakis/go-snaps v0.5.24
github.com/go-git/go-billy/v5 v5.9.2
github.com/go-git/go-git/v5 v5.19.3
@@ -69,6 +68,7 @@ require (
github.com/nix-community/go-nix v0.0.0-20250101154619-4bdde671e0a1
github.com/olekukonko/tablewriter v1.1.5
github.com/opencontainers/go-digest v1.0.0
github.com/pb33f/ordered-map/v2 v2.3.2
github.com/pelletier/go-toml v1.9.5
github.com/quasilyte/go-ruleguard/dsl v0.3.23
github.com/rust-secure-code/go-rustaudit v0.0.0-20250226111315-e20ec32e963c
@@ -83,6 +83,7 @@ require (
github.com/spf13/afero v1.15.0
github.com/spf13/cobra v1.10.2
github.com/stretchr/testify v1.12.1
github.com/tailscale/hujson v0.0.0-20260302212456-ecc657c15afd
github.com/ulikunitz/xz v0.5.17
github.com/vbatts/go-mtree v0.7.0
github.com/vifraa/gopom v1.0.0
@@ -95,17 +96,14 @@ require (
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f
golang.org/x/mod v0.42.0
golang.org/x/net v0.61.0
golang.org/x/sys v0.49.0
golang.org/x/time v0.16.0
golang.org/x/tools v0.52.0
howett.net/plist v1.0.1
modernc.org/sqlite v1.61.0
)
require (
github.com/pb33f/ordered-map/v2 v2.3.2
github.com/tailscale/hujson v0.0.0-20260302212456-ecc657c15afd
golang.org/x/sys v0.49.0
howett.net/plist v1.0.1
)
require github.com/github/go-spdx/v2 v2.8.0
require (
cel.dev/expr v0.25.2 // indirect