mirror of
https://github.com/anchore/syft.git
synced 2026-11-05 08:56:41 +01:00
perf: build lower layer squashes only when a scan reads them (#5492)
* perf: keep only the image squash when scanning with the squashed scope - stereoscope builds a squashed tree for every layer, each a full copy of the one below, so memory grows with layers x files. The squashed scope only ever reads the top one. - new `GetSourceConfig.WithImageSquashOnly(bool)` (and `sourceproviders.Config` field) reads images with stereoscope's `image.WithImageSquashOnly`, scoped to that one read - the CLI turns it on when `--scope squashed` (the default) - an image source read this way returns an error from `FileResolver` for `all-layers` and `deep-squashed` instead of panicking on the missing layer squashes Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * chore(deps): temporarily pin stereoscope to anchore/stereoscope#731 Replaces stereoscope with the PR head on the contributor fork so this builds. Drop the replace and bump to a released version once that PR merges. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * chore(deps): bump stereoscope to main for WithImageSquashOnly Drops the temporary replace to the contributor fork now that anchore/stereoscope#731 has merged. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * revert: drop the squash-only image read and bump stereoscope stereoscope is replacing `image.WithImageSquashOnly` with lazy layer squashes, where every layer keeps a usable squash. That removes the need for `GetSourceConfig.WithImageSquashOnly` and the scope guard in `FileResolver`, so both go. - bumps stereoscope to `feat/lazy-layer-squashes` (anchore/stereoscope `43c8f7e`), pending that PR merging Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * perf: build lower layer squashes only when a scan reads them The image source provider now reads images with `image.WithLazyLayerSquashes`: the image squash is built during the read, and each lower layer's squash only when something reads it. A squashed scan never does, so it skips a full file tree copy per layer. - squashed scans: faster, much less memory on images with many layers - `scan`/`attest` with all-layers or deep-squashed stay eager (an internal ctx flag), since eager squashing overlaps with fetching and indexing and is faster when every layer gets read - library callers going through `syft.GetSource` get lazy for every scope, since the scope isn't known when the image is read. All-layers is still correct, just slower - no public API change Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * pull in stereoscope@main Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com> --------- Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com> Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
This commit is contained in:
co-authored by
Will Murphy
parent
551442b1bc
commit
fd712ee71a
@@ -260,6 +260,11 @@ func getSource(ctx context.Context, opts *options.Catalog, userInput string, sou
|
||||
cfg = cfg.WithDigestAlgorithms(hashers...)
|
||||
}
|
||||
|
||||
if source.ParseScope(opts.Scope) != source.SquashedScope {
|
||||
// every other scope reads each layer's squash, which is cheaper to build while the image is read
|
||||
ctx = internal.WithEagerLayerSquashes(ctx)
|
||||
}
|
||||
|
||||
src, err := syft.GetSource(ctx, userInput, cfg)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("could not determine source: %w", err)
|
||||
|
||||
Reference in New Issue
Block a user