Compare commits

...
77 Commits
Author SHA1 Message Date
anchore-oss-update-botandanchore-oss-update-bot 30f4bce762 chore(deps): update anchore dependencies (#5513)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-11-05 20:27:50 +00:00
Pujitha PaladuguandPujitha Paladugu c581c84ff1 fix(installer): repair OpenSSL fallback in hash_sha256 (#5521)
Motivation: when gsha256sum, sha256sum and shasum are all absent but openssl
is available, install.sh fails checksum verification. The fallback ran
`openssl -dst openssl dgst -sha256`, which is an invalid command, and would
also have failed on `cut -d ' ' -f a`, an invalid field argument. The
installer exits 1 and installs nothing; checksum verification is not bypassed.

Approach: call `openssl dgst -sha256 "$TARGET"` and take the digest as the
last space-separated token of its "SHA256(file)= <digest>" output
(also "SHA2-256(...)" on OpenSSL 3.x), using POSIX parameter expansion.

Validation: added test_hash_sha256_openssl in test/install/0_checksums_test.sh,
which overrides is_command so only openssl is found and checks the digest of
the fixture archive (file and stdin). Running `sh 0_checksums_test.sh` in
test/install fails on the test before the change ("Invalid command '-dst'")
and passes after. Run on macOS with the system openssl only; the full
installer run from the issue and the container-based make targets were not run.

Report: https://github.com/anchore/syft/issues/5520

Assisted-by: claude-sonnet-5-5 (via Claude Code)

Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
2026-11-05 20:17:31 +00:00
fa02ff2d26 feat(executable): capture GNU ELF build IDs (#5426)
* feat(executable): capture GNU ELF build IDs

Capture GNU ELF build IDs in Syft JSON so consumers can correlate
otherwise unidentified binaries with known artifacts or build records.

Read GNU build-ID notes from .note.gnu.build-id, with a 64 KiB read limit.
Validate the note owner and type before recording the identifier.

Add focused extraction and JSON round-trip tests and update the schema.

Signed-off-by: Mario Corchero <mcorcherojim@bloomberg.net>

* chore: bump JSON schema to 16.1.13 for executable identifiers

16.1.12 landed on main for coordinate fullPath, so the identifiers
addition moves to the next version.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* feat(executable): capture go build IDs from .note.go.buildid

The go linker only writes a GNU build ID with -B or when linking
externally, so most pure-Go binaries had no identifier. Every go ELF
carries .note.go.buildid, captured as kind `go-build-id` in the form
`go tool buildid` prints.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* docs(executable): describe identifiers generically

Identifier field docs (and the published schema) now give GNU hex and
Go build IDs as examples instead of stating every value is a GNU build
ID in lowercase hex, so the contract holds as more kinds are added.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(executable): keep identifier notes inside elfutil bounds

Register `.note.gnu.build-id` and `.note.go.buildid` in elfutil's
sectionsReadByName as that list asks, and never open a compressed
identifier note. Real ones are SHF_ALLOC and cannot be compressed, so
this drops the only decompression path into these reads.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test(executable): check identifiers from real linker output

Cross-compiles a tiny go program with `-B` so the test sees a GNU
build ID and a go build ID written by an actual linker and read through
the cataloger entrypoint.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Mario Corchero <mcorcherojim@bloomberg.net>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Mario Corchero <mcorcherojim@bloomberg.net>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-11-04 22:42:03 +00:00
Alex GoodmanandWill Murphy fd712ee71a perf: build lower layer squashes only when a scan reads them (#5492)
* perf: keep only the image squash when scanning with the squashed scope

- stereoscope builds a squashed tree for every layer, each a full copy of the one below, so memory
  grows with layers x files. The squashed scope only ever reads the top one.
- new `GetSourceConfig.WithImageSquashOnly(bool)` (and `sourceproviders.Config` field) reads images
  with stereoscope's `image.WithImageSquashOnly`, scoped to that one read
- the CLI turns it on when `--scope squashed` (the default)
- an image source read this way returns an error from `FileResolver` for `all-layers` and
  `deep-squashed` instead of panicking on the missing layer squashes

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(deps): temporarily pin stereoscope to anchore/stereoscope#731

Replaces stereoscope with the PR head on the contributor fork so this builds. Drop the replace and
bump to a released version once that PR merges.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(deps): bump stereoscope to main for WithImageSquashOnly

Drops the temporary replace to the contributor fork now that anchore/stereoscope#731 has merged.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* revert: drop the squash-only image read and bump stereoscope

stereoscope is replacing `image.WithImageSquashOnly` with lazy layer squashes, where every layer keeps a usable squash. That removes the need for `GetSourceConfig.WithImageSquashOnly` and the scope guard in `FileResolver`, so both go.

- bumps stereoscope to `feat/lazy-layer-squashes` (anchore/stereoscope `43c8f7e`), pending that PR merging

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* perf: build lower layer squashes only when a scan reads them

The image source provider now reads images with `image.WithLazyLayerSquashes`: the image squash is built during the read, and each lower layer's squash only when something reads it. A squashed scan never does, so it skips a full file tree copy per layer.

- squashed scans: faster, much less memory on images with many layers
- `scan`/`attest` with all-layers or deep-squashed stay eager (an internal ctx flag), since eager squashing overlaps with fetching and indexing and is faster when every layer gets read
- library callers going through `syft.GetSource` get lazy for every scope, since the scope isn't known when the image is read. All-layers is still correct, just slower
- no public API change

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* pull in stereoscope@main

Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
2026-11-04 15:12:25 -05:00
Tim Olshansky 551442b1bc fix(terraform): generate NVD-style terraform_provider CPEs for terraform lock packages (#5261)
* fix: generate terraform-provider-specific CPEs for terraform lock packages

The terraform-lock-cataloger sets Language to Go, which causes the Go CPE
heuristic to generate application-level CPEs for terraform providers. For
example, registry.terraform.io/grafana/grafana produces the same CPE as
the Grafana application itself, leading grype to flag providers with
unrelated CVEs.

This adds terraform-specific vendor/product candidate functions that
generate CPEs matching the NVD convention for terraform providers
(e.g. terraform-provider-grafana instead of grafana), with terraform
as the target software qualifier.

Signed-off-by: Tim Olshansky <456103+timols@users.noreply.github.com>

* fix: align terraform provider CPEs with NVD terraform_provider convention

NVD records terraform providers with a fixed "terraform_provider" product
and the provider type in target_sw, e.g.
cpe:2.3:a:hashicorp:terraform_provider:*:*:*:*:*:vault:*:*

Generate <namespace>:terraform_provider:<version>:...:<type> instead of
<namespace>:terraform-provider-<type>:<version>:...:terraform, which never
appears in NVD.

Signed-off-by: Tim Olshansky <456103+timols@users.noreply.github.com>

---------

Signed-off-by: Tim Olshansky <456103+timols@users.noreply.github.com>
2026-11-03 22:01:20 -05:00
Guillaume MartyandGuillaume Marty 639de5db64 fix: keep dot-separated pre-release identifiers in Go ldflags versions (#5422)
The ldflags version regexes stopped at the first dot after the patch
number, so v1.28.0-rc.1 was reported as v1.28.0-rc.

Signed-off-by: Guillaume Marty <279286709+LemonCondor@users.noreply.github.com>
Co-authored-by: Guillaume Marty <279286709+LemonCondor@users.noreply.github.com>
2026-11-03 10:20:09 -05:00
Juan 5ec416cb24 fix: do not generate CPEs for msrc-kb packages (#5510)
A msrc-kb package's name is an MSRC product ID and its version a KB
number, so the generic generator produced CPEs such as
cpe:2.3:a:12437:12437:935509:*:*:*:*:*:*:*. A KB is a patch rather than
a product and has no CPE representation; these packages are matched
against MSRC data by name and version. Skip generation for them, as is
already done for ML model packages. The dictionary lookup already
returns nothing for this type.

Signed-off-by: juanjsebgarcia <github@juangarcia.co.uk>
2026-11-02 15:52:38 -05:00
anchore-oss-update-botandanchore-oss-update-bot 173055c39e chore(deps): update anchore dependencies (#5479)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-11-02 16:30:25 +00:00
anchore-oss-update-botandanchore-oss-update-bot f974ef598c chore(deps): update tool versions (#5509)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-11-01 10:19:06 +00:00
dependabot[bot] e6813ba7b6 chore(deps): bump github.com/github/go-spdx/v2 from 2.7.1 to 2.8.0 (#5502)
Bumps [github.com/github/go-spdx/v2](https://github.com/github/go-spdx) from 2.7.1 to 2.8.0.
- [Release notes](https://github.com/github/go-spdx/releases)
- [Commits](https://github.com/github/go-spdx/compare/v2.7.1...v2.8.0)

---
updated-dependencies:
- dependency-name: github.com/github/go-spdx/v2
  dependency-version: 2.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 13:45:16 +00:00
dependabot[bot] 1238ace393 chore(deps): bump github.com/go-jose/go-jose/v4 from 4.1.5 to 4.1.6 (#5504)
Bumps [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose) from 4.1.5 to 4.1.6.
- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Commits](https://github.com/go-jose/go-jose/compare/v4.1.5...v4.1.6)

---
updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v4
  dependency-version: 4.1.6
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 13:43:40 +00:00
dependabot[bot] 80ecdfe63f chore(deps): bump github.com/deitch/magic (#5503)
Bumps [github.com/deitch/magic](https://github.com/deitch/magic) from 0.0.0-20230404182410-1ff89d7342da to 1.0.0.
- [Commits](https://github.com/deitch/magic/commits/v1.0.0)

---
updated-dependencies:
- dependency-name: github.com/deitch/magic
  dependency-version: 1.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 13:35:51 +00:00
dependabot[bot] 1fa6bf6848 chore(deps): bump github.com/gpustack/gguf-parser-go (#5500)
Bumps [github.com/gpustack/gguf-parser-go](https://github.com/gpustack/gguf-parser-go) from 0.26.4 to 0.27.0.
- [Release notes](https://github.com/gpustack/gguf-parser-go/releases)
- [Commits](https://github.com/gpustack/gguf-parser-go/compare/v0.26.4...v0.27.0)

---
updated-dependencies:
- dependency-name: github.com/gpustack/gguf-parser-go
  dependency-version: 0.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 13:29:22 +00:00
dependabot[bot] b0bc371473 chore(deps): bump modernc.org/sqlite from 1.60.1 to 1.61.0 (#5499)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.60.1 to 1.61.0.
- [Changelog](https://gitlab.com/cznic/sqlite/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.60.1...v1.61.0)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-version: 1.61.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 13:29:00 +00:00
dependabot[bot] d91462b824 chore(deps): bump github.com/gkampitakis/go-snaps from 0.5.23 to 0.5.24 (#5501)
Bumps [github.com/gkampitakis/go-snaps](https://github.com/gkampitakis/go-snaps) from 0.5.23 to 0.5.24.
- [Release notes](https://github.com/gkampitakis/go-snaps/releases)
- [Commits](https://github.com/gkampitakis/go-snaps/compare/v0.5.23...v0.5.24)

---
updated-dependencies:
- dependency-name: github.com/gkampitakis/go-snaps
  dependency-version: 0.5.24
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 13:28:32 +00:00
dependabot[bot] 181b9198c0 chore(deps): bump zizmorcore/zizmor-action from 0.7.1 to 0.8.0 (#5497)
Bumps [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) from 0.7.1 to 0.8.0.
- [Release notes](https://github.com/zizmorcore/zizmor-action/releases)
- [Commits](https://github.com/zizmorcore/zizmor-action/compare/85b2b9fb8e3325e7a23d7d0a9d7ba5ae0f999e3e...53290badfa369cb0bf7ad16342cb62c2fa47ac7b)

---
updated-dependencies:
- dependency-name: zizmorcore/zizmor-action
  dependency-version: 0.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 13:27:35 +00:00
dependabot[bot] 4614cf1d85 chore(deps): bump docker/login-action from 4.6.0 to 4.7.0 (#5498)
Bumps [docker/login-action](https://github.com/docker/login-action) from 4.6.0 to 4.7.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/dbcb813823bdd20940b903addbd779551569679f...81c6f93d4bc991e510b342a00b419b3bb4efc5c6)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 4.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 13:27:21 +00:00
Alex Goodman f46acb6e4c fix(cyclonedx): tighten source-distribution external refs (#5493)
* build source PURL from metadata and gate it to CycloneDX 1.6+

- the source PURL is built from package metadata instead of editing the binary PURL: only `distro` (plus `epoch` and `rpmmod` for rpm) is carried over, subpath is dropped, and the ref is skipped when the PURL type does not match the metadata ecosystem
- one emission rule for all ecosystems: only when the source name or version differs from the binary, so self-sourced rpm and alpm packages no longer get a redundant ref
- source RPM names must end in `.src.rpm` or `.nosrc.rpm`, and `nosrc` maps to `arch=nosrc`
- `source-distribution` refs are dropped when encoding below spec 1.6, since cyclonedx-go does not downgrade them (CycloneDX/cyclonedx-go#291)

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* limit source-distribution refs to deb and rpm

- apk origins and alpm pkgbases are build recipes, not published source packages, and their names are usually also real binary packages, so a `source-distribution` ref there would point at a binary package. They keep relying on the `upstream` PURL qualifier.
- document why this is an external reference rather than `pedigree.ancestors`, with links to the CycloneDX spec discussion (CycloneDX/specification#612)

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-29 18:54:49 -04:00
swadaptive 168722d880 feat(cyclonedx): output source PURL as external reference (#5246)
* Add tests for source packages in external references

Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>

* Source distribution external reference utilities and test coverage

Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>

* Add source-distribution external reference to components

Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>

* Golden test fixtures

Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>

* Adjust arch qualifier for source PURLs

Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>

---------

Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>
2026-10-29 17:42:00 -04:00
4b8a3db5f3 Add nested archive cataloging (#5260)
Signed-off-by: Kudryavcev Nikolay <kydry.nikolau@gmail.com>
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Kudryavcev Nikolay <kydry.nikolau@gmail.com>
Co-authored-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-29 17:33:07 -04:00
Vamshidher Reddy Jannapu Reddy 14a0beb93c Fix unbounded memory use decoding AI model metadata from CycloneDX (#5404)
* fix: stop unbounded recursion decoding self-referencing CycloneDX metadata

SafeTensorsModelInfo and GGUFFileHeader both carry a Parts field that is a
slice of their own type. When a CycloneDX component with that metadata is
decoded from properties, the slice decoder tries element 0, which decodes a
struct whose Parts slice tries element 0 one level deeper, and so on. The
recursion never finds a base case, building longer property-name prefixes
on every level, so memory grows until the process is killed (seen in
grype on any machine-learning-model component written by syft).

Return early from struct decoding when no property is named by the prefix
or nested below it: none of the fields could be decoded anyway, and the
recursion is now bounded by the depth of the properties that exist.

Refs anchore/grype#3767

Signed-off-by: Vamshidher Reddy Jannapu Reddy <v.jannapu@gmail.com>

* perf: look up nested CycloneDX property names in a sorted index

The struct check added in the previous commit scanned every property of a
component for every struct it decoded. Decoding a slice of structs, such as
a package's locations, does that once per element, so a component with n
properties took O(n^2): 20,000 location properties took 5.3s to decode
instead of 18ms, a cost that an untrusted SBOM can choose.

Sort the property names once per Decode/DecodeInto call and binary-search
them instead. The result of each check is unchanged; decoding is back to
O(n log n) (27ms for the same 20,000 properties).

Refs anchore/grype#3767

Signed-off-by: Vamshidher Reddy Jannapu Reddy <v.jannapu@gmail.com>

---------

Signed-off-by: Vamshidher Reddy Jannapu Reddy <v.jannapu@gmail.com>
2026-10-29 15:02:36 -04:00
Alex Goodman 1998095808 fix: hash each file once in the file digest cataloger (#5491)
Symlinks resolve to their target, so the same coordinates could show up
once per path that reaches them and get hashed again each time. The
results map hid this but the work was repeated. Drop repeated
coordinates before digesting.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-29 12:07:01 -04:00
Pujitha PaladuguandPujitha Paladugu 12f0e2d837 fix(binary): detect envoy 1.37.1 and 1.40.x binaries (#5476)
Motivation: the envoy binary classifier returns no package for envoy 1.37.1,
1.40.0 and 1.40.1 (the official envoyproxy/envoy images), while neighbouring
versions such as 1.37.0 and 1.38.0 are detected.

Approach:
- the main matcher (NUL-delimited version followed by
  envoy_reloadable_features) only accepted 1.3x; widen it to 1.[34]x, which
  covers 1.40.0 and 1.40.1.
- in the 1.37.1 binary the version string sits between different neighbouring
  strings, so the existing matchers do not fire. Add a matcher anchored on the
  preceding "DeltaDiscoveryResponse.removed_resources" string.
- add snippets, config.yaml entries and test cases for 1.37.1, 1.40.0, 1.40.1
  (linux/amd64 images, pinned by digest).

Validation:
- cd syft/pkg/cataloger/binary && go test . -run
  'Test_Cataloger_PositiveCases/envoy' : the three new fixtures fail without
  the classifiers.go change and all envoy cases pass with it.
- golangci-lint run ./syft/pkg/cataloger/binary/... : 0 issues.
- Built syft before/after and scanned the full linux/arm64 envoy binaries from
  the official images: 1.37.1, 1.40.0 and 1.40.1 were not reported before and
  are reported with the right version after; 1.37.0 and 1.38.0 are unchanged.
- Not run: the rest of the package test suite needs docker, which is not
  available here (Test_PEPackageCataloger etc.), and snippets for other
  binaries are not downloaded locally.

The contrib-build fallback matchers still only accept 1.3x; they are untouched.

Report: https://github.com/anchore/syft/issues/5470

Assisted-by: claude-sonnet-5-5 (via Claude Code)

Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
2026-10-28 11:01:40 -04:00
hsdfat 1636e1119a fix(binary): match istio snapshot and older pre-release versions (#5121)
The istio pilot-agent/pilot-discovery classifiers missed every
-snapshot.N build and the older -alpha.N/-beta.N/-rc.N layouts.

Two defects:

1. The doubled-version matcher anchored on a bare x.y.z followed by
   NULs, but pre-release builds repeat the full version including the
   suffix (1.15.0-beta.0 NUL NUL NUL 1.15.0-beta.0), so the anchor never
   matched. Only the capture group had been widened previously.

2. -snapshot.N was absent from every alternation, and snapshot builds
   store the version only once with no leading copy, so no existing
   matcher could reach it.

Widen the leading anchor to accept the same optional pre-release suffix
it already accepts in the capture group, add -snapshot.N to each
alternation, and add a matcher for the standalone snapshot layout. The
new matcher is second so it can never shadow a normal build.

Adds real captured snippets for pilot-agent and pilot-discovery at
1.0.0-snapshot.0, 1.5.0-alpha.0, 1.15.0-beta.0 and 1.19.0-rc.0.

Fixes #5060

Signed-off-by: phatlc <phatle.hsd@gmail.com>
2026-10-28 11:00:16 -04:00
anchore-oss-update-botandanchore-oss-update-bot fb56f1eef1 chore(deps): update anchore dependencies (#5467)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-10-26 18:35:31 +00:00
Vignesh 842d2b3deb Fix Bitnami CPE application part (#5127)
Signed-off-by: vigneshakaviki <kumarvignesh295@gmail.com>
2026-10-26 15:33:13 +00:00
idhyt 1401107138 fix(go-module-cataloger): use correct error variable in package load loop (#4983)
In the loop over `rootPkgs`, the error appended to `unknownErr` was incorrectly
using the outer `err` (from `packages.Load`) instead of the individual package
error `e`. This could cause:
- nil or stale error values to be recorded when package-specific errors existed.
- Loss of detailed per-package error information in the final error report.

Changed `unknown.Append(..., err)` to `unknown.Append(..., e)` to properly
capture each package's load error.

Signed-off-by: idhyt <idhyt3r@gmail.com>
2026-10-26 15:05:27 +00:00
Pujitha PaladuguandPujitha Paladugu 722df28ed1 fix(java): skip comment lines in gradle.lockfile (#5372)
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
2026-10-26 10:46:54 -04:00
dependabot[bot] 4c500520d0 chore(deps): bump github.com/bmatcuk/doublestar/v4 from 4.10.2 to 4.10.3 (#5460)
Bumps [github.com/bmatcuk/doublestar/v4](https://github.com/bmatcuk/doublestar) from 4.10.2 to 4.10.3.
- [Release notes](https://github.com/bmatcuk/doublestar/releases)
- [Commits](https://github.com/bmatcuk/doublestar/compare/v4.10.2...v4.10.3)

---
updated-dependencies:
- dependency-name: github.com/bmatcuk/doublestar/v4
  dependency-version: 4.10.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-26 13:29:29 +00:00
anchore-oss-update-botandanchore-oss-update-bot 9f9751aade chore(deps): update tool versions (#5475)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-10-26 10:26:38 +00:00
anchore-oss-update-botandanchore-oss-update-bot 447ee20bdc chore(deps): update CPE dictionary index (#5473)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-10-26 01:43:08 +00:00
anchore-oss-update-botandanchore-oss-update-bot eec916df6c chore(deps): update anchore dependencies (#5420)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-10-23 23:15:47 -04:00
Alex Goodman 1c4d672b28 chore: let go-make own the go version (#5464)
* chore: let go-make own the go version

Drops syft's own Go version pin so CI and releases use go-make setup's default. The pin predates the go-make port (#4773, carried through #4923) and had to be kept in sync in four places.

- removes the `go-version` input from `.github/actions/bootstrap` and the three `go-version: "1.26.2"` overrides in `validations.yaml`
- builds move from Go 1.26 to go-make's default (currently `1.27.2`). `go.mod` stays at `go 1.26.8`, so nothing changes for library consumers
- this is groundwork for the emSign/eMudhra CA fix: Go `1.27.3` distrusts the compromised intermediates in `crypto/x509`. anchore/go-make#170 bumps the default, and the go-make pin here gets bumped once that is released. The `v1.55.0` images also still carry `emSign ECC Root CA - G3`, so a release is needed after the bump

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore: reformat generated license list with go 1.27 gofmt

go 1.27 gofmt aligns long map keys differently, so the json schema drift check rewrote this file.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-23 11:22:18 -04:00
dependabot[bot] 44c215450b chore(deps): bump golang.org/x/tools from 0.51.0 to 0.52.0 (#5463)
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.51.0 to 0.52.0.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.51.0...v0.52.0)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-version: 0.52.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-23 13:53:07 +00:00
dependabot[bot] e1dcbd89df chore(deps): bump github.com/gookit/color from 1.6.2 to 1.6.3 (#5459)
Bumps [github.com/gookit/color](https://github.com/gookit/color) from 1.6.2 to 1.6.3.
- [Release notes](https://github.com/gookit/color/releases)
- [Commits](https://github.com/gookit/color/compare/v1.6.2...v1.6.3)

---
updated-dependencies:
- dependency-name: github.com/gookit/color
  dependency-version: 1.6.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-23 13:36:23 +00:00
dependabot[bot] e61fb72e46 chore(deps): bump github.com/go-test/deep from 1.1.1 to 1.1.2 (#5462)
Bumps [github.com/go-test/deep](https://github.com/go-test/deep) from 1.1.1 to 1.1.2.
- [Release notes](https://github.com/go-test/deep/releases)
- [Changelog](https://github.com/go-test/deep/blob/main/CHANGES.md)
- [Commits](https://github.com/go-test/deep/compare/v1.1.1...v1.1.2)

---
updated-dependencies:
- dependency-name: github.com/go-test/deep
  dependency-version: 1.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-23 13:36:03 +00:00
dependabot[bot] 71fe033ab5 chore(deps): bump github.com/klauspost/compress from 1.20.1 to 1.20.3 (#5461)
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.20.1 to 1.20.3.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Commits](https://github.com/klauspost/compress/compare/v1.20.1...v1.20.3)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-version: 1.20.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-23 13:31:44 +00:00
dependabot[bot] a5426c6642 chore(deps): bump zizmorcore/zizmor-action from 0.6.4 to 0.7.1 (#5457)
Bumps [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) from 0.6.4 to 0.7.1.
- [Release notes](https://github.com/zizmorcore/zizmor-action/releases)
- [Commits](https://github.com/zizmorcore/zizmor-action/compare/cc914d7f3750a2d13d75c7f184a1060aa0e9d482...85b2b9fb8e3325e7a23d7d0a9d7ba5ae0f999e3e)

---
updated-dependencies:
- dependency-name: zizmorcore/zizmor-action
  dependency-version: 0.7.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-23 13:30:56 +00:00
dependabot[bot] 816106f0b2 chore(deps): bump golang.org/x/mod from 0.41.0 to 0.42.0 (#5458)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.41.0 to 0.42.0.
- [Commits](https://github.com/golang/mod/compare/v0.41.0...v0.42.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-version: 0.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-23 13:30:11 +00:00
akshaver b55707f60f fix(binary): preserve semver build metadata in ffmpeg version (#5452)
Signed-off-by: Adam Shaver <ashaver@nvidia.com>
2026-10-21 17:10:43 -04:00
Christopher Angelo PhillipsandAlex Goodman f15c3343e2 Pin Gguff lib to prevent dropping packages (#5447)
* test: catalog GGUF models larger than the header

gguf-parser-go v0.26.3+ rejects a header whose tensors declare more data than follows it. Syft parses only a 50 MB copy of each model, so any larger model is dropped.

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>

* fix: fix parser with rollback and verify with test

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>

* fix: bound GGUF tensor info before the parser reads it

The header pre-scan stopped after the metadata KVs and left tensor infos to gguf-parser-go. Older releases of the library allocate whatever length a tensor name declares, so a tiny file could force a multi-GB allocation, or a fatal out of memory that panic recovery can't catch.

- tensor names are bounded like metadata keys (non-empty, length limit, shared string budget)
- dimension count is capped at `GGML_MAX_DIMS`
- a tensor count larger than the header now fails at EOF in the pre-scan

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: give the GGUF header copy the model's real size

The cataloger copies at most 50 MB of a model and parses that copy. Newer gguf-parser-go releases check the tensor data the header declares against the file size, so the truncated copy of any larger model was rejected.

- after the header is validated, the copy is extended to the size the resolver reports for the model, leaving the tail as a hole the parser never reads
- sources that report no size (OCI model artifacts today) keep the old behavior

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(deps): restore gguf-parser-go v0.26.4

The header copy now carries the model's real size, so the pin to v0.26.0 and its dependabot ignore are no longer needed. This also brings back upstream's string and array length bounds, and the library's own `IsNumeric`.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: keep the extended GGUF header copy sparse on Windows

NTFS reserves disk for the full length of a file extended with truncate unless the file is marked sparse, so cataloging a large model on Windows reserved the model's size in the temp dir until cleanup.

- the header copy is marked sparse (`FSCTL_SET_SPARSE`) before it is extended
- if the filesystem can't make it sparse (FAT32, exFAT), the copy is not extended, and models larger than it fail to parse as before
- `golang.org/x/sys` moves from an indirect to a direct dependency

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-19 14:40:43 -04:00
dependabot[bot] a81193226f chore(deps): bump github.com/hashicorp/go-getter from 1.8.9 to 1.8.10 (#5436)
Bumps [github.com/hashicorp/go-getter](https://github.com/hashicorp/go-getter) from 1.8.9 to 1.8.10.
- [Release notes](https://github.com/hashicorp/go-getter/releases)
- [Changelog](https://github.com/hashicorp/go-getter/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/go-getter/compare/v1.8.9...v1.8.10)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-getter
  dependency-version: 1.8.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-16 13:28:00 +00:00
dependabot[bot] 50ea42989e chore(deps): bump actions/download-artifact from 8.0.1 to 8.0.2 (#5433)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 8.0.1 to 8.0.2.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c...9000827ccba6bdab643e8b6fd33ac0654aef8333)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: 8.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-16 13:27:46 +00:00
dependabot[bot] 83ba8304c1 chore(deps): bump github.com/gookit/color from 1.6.1 to 1.6.2 (#5435)
Bumps [github.com/gookit/color](https://github.com/gookit/color) from 1.6.1 to 1.6.2.
- [Release notes](https://github.com/gookit/color/releases)
- [Commits](https://github.com/gookit/color/compare/v1.6.1...v1.6.2)

---
updated-dependencies:
- dependency-name: github.com/gookit/color
  dependency-version: 1.6.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-16 13:27:06 +00:00
dependabot[bot] 0f1cd9c6f2 chore(deps): bump actions/upload-artifact from 7.0.1 to 7.0.2 (#5432)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 7.0.1 to 7.0.2.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/043fb46d1a93c77aae656e7c1c64a875d1fc6a0a...cf430e030ddbb5b0abf93d22962f4752f3646cd9)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: 7.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-16 13:26:10 +00:00
dependabot[bot] 758c232716 chore(deps): bump github.com/github/go-spdx/v2 from 2.7.0 to 2.7.1 (#5434)
Bumps [github.com/github/go-spdx/v2](https://github.com/github/go-spdx) from 2.7.0 to 2.7.1.
- [Release notes](https://github.com/github/go-spdx/releases)
- [Commits](https://github.com/github/go-spdx/compare/v2.7.0...v2.7.1)

---
updated-dependencies:
- dependency-name: github.com/github/go-spdx/v2
  dependency-version: 2.7.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-16 13:24:44 +00:00
Weston Steimel 30200f1d69 fix: upgrade to golang.org/x/net v0.61.0 (#5427)
Upgrade to golang.org/x/net v0.61.0 to resolve reporting of GO-2026-6612

Signed-off-by: Weston Steimel <author@code.w.steimel.me.uk>
2026-10-15 07:20:55 -04:00
anchore-oss-update-botandanchore-oss-update-bot ddc490c5c5 chore(deps): update anchore dependencies (#5387)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-10-12 16:29:27 -04:00
dependabot[bot] 46bac23392 chore(deps): bump golang.org/x/tools from 0.50.0 to 0.51.0 (#5417)
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.50.0 to 0.51.0.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.50.0...v0.51.0)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-version: 0.51.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-12 20:23:06 +00:00
dependabot[bot] 07752f0741 chore(deps): bump github.com/go-git/go-git/v5 from 5.19.2 to 5.19.3 (#5416)
Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) from 5.19.2 to 5.19.3.
- [Release notes](https://github.com/go-git/go-git/releases)
- [Changelog](https://github.com/go-git/go-git/blob/main/HISTORY.md)
- [Commits](https://github.com/go-git/go-git/compare/v5.19.2...v5.19.3)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-git/v5
  dependency-version: 5.19.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-12 20:21:35 +00:00
Alex Goodman f8ffa7382e chore: bump go-make module and action together (#5415)
The go-make module and its action need to move in lockstep, but dependabot bumps them in separate PRs that drift apart. Automation now bumps both together, so dependabot ignores go-make in both ecosystems.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-12 15:57:20 -04:00
Alex Goodman c2ea35ef32 chore: add COPYRIGHT file and update CONTRIBUTORS.md (#5414)
Fixes #5394

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-12 15:00:02 -04:00
anchore-oss-update-botandanchore-oss-update-bot c0109a7ad1 chore(deps): update CPE dictionary index (#5409)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-10-12 01:56:57 +00:00
dependabot[bot] 33419c8f39 chore(deps): bump github.com/pb33f/ordered-map/v2 from 2.3.1 to 2.3.2 (#5400)
Bumps [github.com/pb33f/ordered-map/v2](https://github.com/pb33f/ordered-map) from 2.3.1 to 2.3.2.
- [Changelog](https://github.com/pb33f/ordered-map/blob/master/CHANGELOG.md)
- [Commits](https://github.com/pb33f/ordered-map/compare/v2.3.1...v2.3.2)

---
updated-dependencies:
- dependency-name: github.com/pb33f/ordered-map/v2
  dependency-version: 2.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-09 13:42:52 +00:00
dependabot[bot] 6790272518 chore(deps): bump github.com/anchore/go-make in /.make (#5402)
Bumps [github.com/anchore/go-make](https://github.com/anchore/go-make) from 0.8.1 to 0.9.1.
- [Release notes](https://github.com/anchore/go-make/releases)
- [Commits](https://github.com/anchore/go-make/compare/v0.8.1...v0.9.1)

---
updated-dependencies:
- dependency-name: github.com/anchore/go-make
  dependency-version: 0.9.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-09 13:37:22 +00:00
Peter McConnell 740beb6dc4 fix: search for the .NET bundle marker without holding the file (#5393)
Signed-off-by: Peter McConnell <peter.mcconnell@upwind.io>
2026-10-09 09:36:52 -04:00
dependabot[bot] 3e41abadc4 chore(deps): bump github.com/go-git/go-billy/v5 from 5.9.1 to 5.9.2 (#5399)
Bumps [github.com/go-git/go-billy/v5](https://github.com/go-git/go-billy) from 5.9.1 to 5.9.2.
- [Release notes](https://github.com/go-git/go-billy/releases)
- [Commits](https://github.com/go-git/go-billy/compare/v5.9.1...v5.9.2)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-billy/v5
  dependency-version: 5.9.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-09 13:35:51 +00:00
dependabot[bot] 0a050b0e27 chore(deps): bump github.com/gpustack/gguf-parser-go (#5401)
Bumps [github.com/gpustack/gguf-parser-go](https://github.com/gpustack/gguf-parser-go) from 0.26.3 to 0.26.4.
- [Release notes](https://github.com/gpustack/gguf-parser-go/releases)
- [Commits](https://github.com/gpustack/gguf-parser-go/compare/v0.26.3...v0.26.4)

---
updated-dependencies:
- dependency-name: github.com/gpustack/gguf-parser-go
  dependency-version: 0.26.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-09 13:35:03 +00:00
dependabot[bot] fc8bd6c9d8 chore(deps): bump anchore/sbom-action from 0.24.2 to 0.24.3 (#5395)
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.24.2 to 0.24.3.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/3ad7283483fc7af8ff2b4ea19663c2d5ca935e26...66cbf4bc1f1c0d2edc94016e65bc221b6bb0ad6c)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.24.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-09 13:31:27 +00:00
dependabot[bot] 7e713d106e chore(deps): bump modernc.org/sqlite from 1.59.0 to 1.60.1 (#5398)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.59.0 to 1.60.1.
- [Changelog](https://gitlab.com/cznic/sqlite/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.59.0...v1.60.1)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-version: 1.60.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-09 13:29:55 +00:00
dependabot[bot] 1f5ac47990 chore(deps): bump anchore/go-make/.github/actions/setup (#5397)
Bumps [anchore/go-make/.github/actions/setup](https://github.com/anchore/go-make) from 0.8.1 to 0.9.1.
- [Release notes](https://github.com/anchore/go-make/releases)
- [Commits](https://github.com/anchore/go-make/compare/fa5421b3bf24c9d18f11a736c5eb4172f2b7d683...27a9f35f41b5fe86d68815844efd2ba816e9ba6b)

---
updated-dependencies:
- dependency-name: anchore/go-make/.github/actions/setup
  dependency-version: 0.9.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-09 13:29:42 +00:00
dependabot[bot] 28e38c29f3 chore(deps): bump anchore/go-make/.github/actions/setup (#5396)
Bumps [anchore/go-make/.github/actions/setup](https://github.com/anchore/go-make) from 0.8.1 to 0.9.1.
- [Release notes](https://github.com/anchore/go-make/releases)
- [Commits](https://github.com/anchore/go-make/compare/fa5421b3bf24c9d18f11a736c5eb4172f2b7d683...27a9f35f41b5fe86d68815844efd2ba816e9ba6b)

---
updated-dependencies:
- dependency-name: anchore/go-make/.github/actions/setup
  dependency-version: 0.9.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-09 13:27:40 +00:00
Silas ParkerandAlex Goodman 546c872c9d Add support for scanning IPK packages using the deb-archive-cataloger (#5111)
* Add support for scanning IPK packages using the deb-archive-cataloger

Signed-off-by: Silas Parker <silas@srp.me.uk>

* fix(debian): share License field parsing between opkg status and archive control files

- the control-file `License:` fallback now goes through the same helper as the opkg status DB path, so a package reports the same licenses whether found installed or as an archive
- bitbake `&` / `|` are rewritten to SPDX `AND` / `OR` and the whole value is validated as one expression instead of split by hand, which kept grouping like `(A & B) | C` intact
- non-SPDX values fall back to a deduped list of names with operators and parens dropped (previously the status DB path emitted `&`, `(BSD`, etc as licenses)
- fields over 4KB are ignored, bounding work per archive and keeping deep paren nesting away from the recursive expression parser

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(debian): split Yocto recipe filenames in the Source field

Yocto writes the bitbake recipe filename (`Source: zlib_1.3.2.bb`) where dpkg expects a source package name. This is now split into `Source: zlib` / `SourceVersion: 1.3.2` for both ipk archives and opkg status DB entries, instead of surfacing the `.bb` filename as the upstream package.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(debian): do not label ipk archives as Debian packages

- `.ipk` archives no longer get a `pkg:deb` PURL from the assumed debian distro; opkg-based distros (Yocto, OpenWrt) have no purl type, so these now match packages found in an opkg status DB, which already get no PURL
- docs now cover `.ipk` and note that only ar-framed ipks are supported (OpenWrt's older gzipped-tar framing is rejected as an invalid archive)

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test(debian): assemble the ipk fixture at build time instead of committing it

The fixture Dockerfile now builds a Yocto-shaped `.ipk` from a checked-in `control/` directory (copied from the original poky-tiny build) using `tar`, `zstd` and binutils `ar`, the same way bitbake's `package_ipk` packs one. This drops the prebuilt binary and the 20-30 minute Yocto `Dockerfile.build` while keeping the same archive layout and control metadata.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Silas Parker <silas@srp.me.uk>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-07 20:41:13 +00:00
MetalandAlex Goodman b0a8de7bf6 fix(redact): reset redaction store between command runs (#5386)
* fix(redact): reset redaction store between command runs

cli.Command() may be invoked more than once in the same process (e.g.
when syft is embedded as a library). The clio initializer calls
internal/redact.Set on every command execution, but the redact store is
process-global and Set panics when a store already exists, so the second
invocation dies with "replace existing redaction store (probably
unintentional)".

Add redact.Reset() to clear the previous run's store and call it in the
initializer before Set. The double-Set guard in Set is kept: an
unexpected second Set within a single run still panics.

Add TestAppClioSetupConfigInitializerCanRunMultipleTimes which runs the
initializer twice; it panics with the exact reported message on the old
code and passes with the fix.

Fixes #2285

Signed-off-by: JasonMetal <935216773@qq.com>

* fix(redact): release the redact store at the end of each run

Clear the global redact store in the post-run hook instead of resetting it
before every `Set`. The double-`Set` panic keeps its meaning: a store is only
present while a run is in flight, so a second run starting mid-run still
panics rather than splitting secrets across two stores and leaking the ones
in the dropped store. Sequential runs in the same process work since the
previous run releases its store on the way out.

A run only releases the store it set, so it never clears a store owned by
someone else.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: JasonMetal <935216773@qq.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-07 20:28:15 +00:00
anchore-oss-update-botandanchore-oss-update-bot d9489c2230 chore(deps): update tool versions (#5381)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-10-07 10:27:20 +00:00
anchore-oss-update-botandanchore-oss-update-bot b254e6d92f chore(deps): update anchore dependencies (#5367)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-10-06 10:36:33 -04:00
29b92a58c1 Add container-storage support for Podman and Buildah (#5368)
* feat: add containers-storage support for Podman and Buildah, update tests and documentation

Signed-off-by: Bruce Clark <bruce.clark@mandg.com>

* test: update containers-storage tests to validate package.json and handle missing images

Signed-off-by: Bruce Clark <bruce.clark@mandg.com>

* test: fix rootless containers-storage test setup

Install uidmap for Podman and Buildah CI jobs.
Align temporary rootful and rootless storage paths.
Clean up rootless stores within the builder's user namespace.

Signed-off-by: Bruce Clark <bruce.clark@mandg.com>

* fix: only use containers-storage when explicitly requested

- containers-storage is no longer part of automatic image resolution; it's used only for `--from containers-storage`, the `containers-storage:` scheme, or `default-pull-source: containers-storage` (now an accepted value). Opening a store runs full graph driver init (mounts, locks, store writes), which is too invasive as a probe on every plain image reference.
- rootless stores must be read from inside the builder's user namespace (`podman unshare syft ...`), documented in the README.
- the `containers_image_openpgp` tag is now applied to unit/integration tests and lint, so dev tooling compiles the same provider that ships.
- containers-storage CI runs through `make containers-storage-test` and covers both vfs and overlay stores.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: exclude cgo btrfs driver and unmount overlay store in cleanup

- cgo-enabled linux builds (`-race` tests, lint) compile go.podman.io's btrfs graph driver, which needs libbtrfs headers. Add `exclude_graphdriver_btrfs` alongside `containers_image_openpgp` everywhere the tag is set; release builds are `CGO_ENABLED=0` and never included it.
- the overlay driver bind-mounts its graphroot inside the rootless user namespace, so the containers-storage test unmounts it before removing the store, and cleanup no longer fails the run.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Bruce Clark <bruce.clark@mandg.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Bruce Clark <bruce.clark@mandg.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-05 15:02:37 +00:00
Alex Goodman 6c9bb732dd ci: run the full unit suite on windows (#5370)
* ci: run the full unit suite on windows via make unit

Drops the curated `unit:windows` subset; the windows job now bootstraps like linux and runs `make unit`. Exceptions get added as failures show up.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* ci: skip fixture refresh for windows unit tests

binny has no windows release, so the bootstrap action can't install the tools the fixture refresh needs.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: skip linux container fixtures when docker can't run them

Windows CI runners only run windows containers, so image fixtures go through `testutils.SkipWithoutLinuxContainers`. No behavior change on linux/mac.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: close leaked file handles and posix squashfs paths on windows

- output writers are now closed on early returns in `scan`/`convert` (open handles block cleanup on windows)
- `newSBOMMultiWriter` used `path.Dir` on host paths, so output subdirectories were never created on windows
- squashfs walk paths were built with `filepath.Join`
- skip a few posix-only test cases (dir perms, xdg fallback, go-rpmdb ndb handle leak upstream)

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(golang): resolve go source module dir natively on windows

- `parseGoModFile` passed the resolver's posix (`/c/...`) path to the go toolchain, so go source resolution silently found nothing on windows
- skip container-built fixtures (go archs, java builds) and golden image tarballs on windows; stereoscope's layer cache filenames contain `:`, which windows rejects

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: keep posix paths posix on windows

- `ChrootContext.ToChrootPath` nested volumes (`D:\cwd\C:\root`) when the scan root was on another drive, so the prefix was never trimmed
- bitnami, python wheel/egg, and alpm parsers used `filepath` on posix location paths
- alpm backup-file lookups silently missed on windows

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* ci: pull the test fixture cache for windows unit tests

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: don't map virtual file tree paths through the host on windows

`FiletreeResolver` without a chroot (snaps) ran request paths through `filepath.Abs`, rooting them on the current volume (`/payload.txt` -> `/d/payload.txt`), so snap scans on windows found nothing.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: match tar entry globs as posix on windows

- `matchesAnyGlob` used `doublestar.PathMatch` (host separator) on tar entry names, so tar-wrapped java archives silently produced nothing on windows
- test helpers now treat binary snippet keys and `FromFile` locations as posix

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* incorporate windows path fix

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* incorporate go-rpmdb close fix

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* close squashfs fixture writers before finalizing

- windows dir listings can report a zero size for files still open, so the manifest was packed empty

(also adds a temporary windows diagnostic for the binary cataloger)

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* temp: surface windows binary cataloger diagnostics

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: keep directory symlink targets posix on windows

- the directory indexer joined symlink targets with native separators, so links in the posix file tree pointed nowhere and globs on link names found nothing
- binary fixture logical keys used native separators, so nested snippets were picked as the fixture dir on windows

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* keep addSymlinkToIndex under the funlen limit

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* drop windows skips covered by the stereoscope and go-rpmdb bumps

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-05 10:41:03 -04:00
anchore-oss-update-botandanchore-oss-update-bot 6d2493bae4 chore(deps): update CPE dictionary index (#5375)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-10-05 01:52:49 +00:00
dependabot[bot] bc68299565 chore(deps): bump github.com/klauspost/compress from 1.20.0 to 1.20.1 (#5369)
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.20.0 to 1.20.1.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Commits](https://github.com/klauspost/compress/compare/v1.20.0...v1.20.1)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-version: 1.20.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-02 13:30:02 +00:00
Christopher Angelo PhillipsandAlex Goodman c33fa32cf1 fix(dotnet): resolve packages.lock dependencies deterministically (#5210)
* fix(dotnet): resolve packages.lock dependencies within their target framework

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>

* fix(dotnet): make packages.lock package metadata deterministic across target frameworks

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>

* fix: parse packages lock tests

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>

* fix: update lockfile parse to read into stable structure

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>

* fix: remove stale comments

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>

* test: new test for project entry case

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>

* fix(dotnet): key packages.lock packages case-insensitively

NuGet IDs are case-insensitive, and edge lookup already treated them that way, but package identity used the exact-case name. Two spellings of one ID and version across target frameworks became two packages, and type precedence never ran between them.

- packages merge on the lowercased name + version, display name comes from the winning entry
- a package listing itself (under any casing) no longer produces a self edge

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-01 21:21:09 +00:00
Alex Goodman cc326e45a6 fix: preserve quoted args when deferring to tagged install script (#5363)
The installer word-split the original arguments when piping into a tagged release's `install.sh`, so `-b '/tmp/syft install path'` arrived as three words and the trailing word was treated as a release tag.

- argument parsing moved into `parse_install_args` so `main` keeps its original args
- tagged script is invoked with `"$@"` instead of the unquoted `PROGRAM_ARGS`

Port of anchore/grype#3738

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-01 15:00:50 -04:00
Alex GoodmanandKeith Zantow ef356eccea fix: sign release checksums with a sigstore bundle (#5362)
* fix: sign release checksums with a sigstore bundle

The cosign update rejects the legacy `--output-certificate` / `--output-signature` flags alongside the new bundle format, which broke signing during the v1.53.0 release.

- checksums are now signed into a single `checksums.txt.sigstore.json` bundle (signature, certificate, and tlog proof)
- `install.sh -v` verifies with `--bundle` for releases >= v1.53.0 and keeps the `.sig` / `.pem` flow for older releases

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore: smooth the cutover to sigstore bundle verification

- `install.sh -v` now hints that bundle verification needs cosign v2.5.0 or newer when verification of a bundle-signed release fails
- install tests keep exercising the legacy `.sig` / `.pem` path against v1.52.0 once the latest release is bundle-signed
- touch the install test environment Dockerfiles so the cached images rebuild with a current cosign

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: only hint at the cosign minimum when cosign is actually too old

- `install.sh -v` used to print the cosign v2.5.0 note on any failed bundle verification, including a missing bundle or a bad signature. It now only prints it when cosign reports it couldn't read the bundle (`bundle does not contain cert`), which is how v2.2.4 through v2.4.1 fail on a sigstore bundle.
- `verify_sign` is now covered with a stub cosign, so the bundle path is exercised before any bundle-signed release is latest

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: stop the install when a release asset is missing or fails its checksum

- `install.sh` now stops when the checksums file, the signature bundle, or the asset can't be downloaded, instead of handing an empty or error-page file to the next step
- a sha256 mismatch between the asset and the checksums file now stops the install. Before this the mismatch was logged and the asset was installed anyway, so `-v` only ever vouched for `checksums.txt`, not the binary

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore: pin cosign in the install test images

The install test images pulled whatever cosign was latest, unverified. They now install cosign v3.1.3 checked against its published sha256, so the install tests always run with a cosign that can verify sigstore bundles (releases >= v1.53.0 need v2.5.0 or newer).

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
2026-10-01 15:00:37 -04:00
dependabot[bot] db7e445c54 chore(deps): bump github.com/ulikunitz/xz from 0.5.16 to 0.5.17 (#5366)
Bumps [github.com/ulikunitz/xz](https://github.com/ulikunitz/xz) from 0.5.16 to 0.5.17.
- [Commits](https://github.com/ulikunitz/xz/compare/v0.5.16...v0.5.17)

---
updated-dependencies:
- dependency-name: github.com/ulikunitz/xz
  dependency-version: 0.5.17
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 18:03:47 +00:00
dependabot[bot] d654c00bd6 chore(deps): bump github.com/bmatcuk/doublestar/v4 from 4.10.0 to 4.10.2 (#5365)
Bumps [github.com/bmatcuk/doublestar/v4](https://github.com/bmatcuk/doublestar) from 4.10.0 to 4.10.2.
- [Release notes](https://github.com/bmatcuk/doublestar/releases)
- [Commits](https://github.com/bmatcuk/doublestar/compare/v4.10.0...v4.10.2)

---
updated-dependencies:
- dependency-name: github.com/bmatcuk/doublestar/v4
  dependency-version: 4.10.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 18:00:41 +00:00
Alex Goodman c00be184b5 fix: revert hashstructure to v0.6.0 to keep artifact IDs stable (#5364)
* fix: revert hashstructure to v0.6.0 to keep artifact IDs stable

hashstructure v1 changed `SlicesAsSets` hashing (every slice now gets a finalizing hash pass, and duplicate elements are deduplicated), which changed nearly every artifact ID syft produces.

- pin `github.com/gohugoio/hashstructure` back to v0.6.0 and have dependabot ignore major bumps
- make the aggregate `Unit tests` job fail when a dependency fails; previously it was skipped, which satisfies the required check and let #5277 auto-merge with failing unit tests

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* ci: fail unit test gate on any failed, cancelled, or skipped dependency

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* ci: require every unit test dependency to succeed

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-01 17:40:34 +00:00
dependabot[bot] 29d2de72c8 chore(deps): bump github.com/gohugoio/hashstructure from 0.6.0 to 1.1.0 (#5277)
Bumps [github.com/gohugoio/hashstructure](https://github.com/gohugoio/hashstructure) from 0.6.0 to 1.1.0.
- [Release notes](https://github.com/gohugoio/hashstructure/releases)
- [Commits](https://github.com/gohugoio/hashstructure/compare/v0.6.0...v1.1.0)

---
updated-dependencies:
- dependency-name: github.com/gohugoio/hashstructure
  dependency-version: 1.1.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 15:35:21 +00:00
365 changed files with 26284 additions and 1969 deletions
+2 -2
View File
@@ -16,7 +16,7 @@ tools:
# used to upload test fixture cache
- name: oras
version:
want: v1.3.4
want: v1.3.6
method: github-release
with:
repo: oras-project/oras
@@ -24,7 +24,7 @@ tools:
# used to parse JSON/YAML annotations on the fixture cache image
- name: yq
version:
want: v4.53.6
want: v4.54.1
method: github-release
with:
repo: mikefarah/yq
+1 -6
View File
@@ -6,10 +6,6 @@ description: "Bootstrap all syft tools and dependencies on top of go-make's setu
# We add the syft-specific extras here: apt packages and the test fixture cache.
inputs:
go-version:
description: "Go version to install (passed to go-make/setup)"
required: true
default: "1.26.2"
cache-key-prefix:
description: "Prefix all cache keys with this value (passed to go-make/setup)"
required: true
@@ -30,9 +26,8 @@ runs:
using: "composite"
steps:
- name: Setup go + go-make tooling
uses: anchore/go-make/.github/actions/setup@fa5421b3bf24c9d18f11a736c5eb4172f2b7d683 # v0.8.1
uses: anchore/go-make/.github/actions/setup@cf465259f4c9f0b0b0bcb35164ede11f7dc6d9f6 # v0.9.6
with:
go-version: ${{ inputs.go-version }}
cache-key-prefix: ${{ inputs.cache-key-prefix }}
cache-enabled: ${{ inputs.cache-enabled }}
+8
View File
@@ -21,6 +21,11 @@ updates:
- dependency-name: "github.com/aquasecurity/go-version"
- dependency-name: "github.com/knqyf263/go-apk-version"
- dependency-name: "github.com/knqyf263/go-deb-version"
# v1 changed SlicesAsSets hashing, which changes every artifact ID syft produces
- dependency-name: "github.com/gohugoio/hashstructure"
update-types: ["version-update:semver-major"]
# go-make module and action refs are bumped together by automation
- dependency-name: "github.com/anchore/go-make"
- package-ecosystem: "github-actions"
directories:
@@ -34,3 +39,6 @@ updates:
open-pull-requests-limit: 10
labels:
- "dependencies"
ignore:
# go-make module and action refs are bumped together by automation
- dependency-name: "anchore/go-make/*"
+4 -4
View File
@@ -46,7 +46,7 @@ jobs:
# these are checks that should be run on pull-request and merges to main.
# we do NOT want to kick off a release if these have not been verified on main.
# Please see the validations.yaml workflow for the names that should be used here.
checks: '["Acceptance tests (Linux)", "Acceptance tests (Mac)", "Build snapshot artifacts", "CLI tests (Linux)", "Integration tests", "Static analysis", "Unit tests"]'
checks: '["Acceptance tests (Linux)", "Acceptance tests (Mac)", "Build snapshot artifacts", "CLI tests (Linux)", "Containers-storage tests (Podman)", "Containers-storage tests (Buildah)", "Integration tests", "Static analysis", "Unit tests"]'
release:
needs: [check-gate, version-available]
@@ -89,13 +89,13 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Hub
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f #v4.6.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 #v4.7.0
with:
username: ${{ secrets.ANCHOREOSSWRITE_DH_USERNAME }}
password: ${{ secrets.ANCHOREOSSWRITE_DH_PAT }}
- name: Login to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f #v4.6.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 #v4.7.0
with:
registry: ghcr.io
username: ${{ github.actor }}
@@ -118,7 +118,7 @@ jobs:
# for updating brew formula in anchore/homebrew-syft
GITHUB_BREW_TOKEN: ${{ secrets.ANCHOREOPS_GITHUB_OSS_WRITE_TOKEN }}
- uses: anchore/sbom-action@3ad7283483fc7af8ff2b4ea19663c2d5ca935e26 #v0.24.2
- uses: anchore/sbom-action@66cbf4bc1f1c0d2edc94016e65bc221b6bb0ad6c #v0.24.3
continue-on-error: true
with:
file: go.mod
@@ -25,7 +25,7 @@ jobs:
persist-credentials: false
- name: "Run zizmor"
uses: zizmorcore/zizmor-action@cc914d7f3750a2d13d75c7f184a1060aa0e9d482 # v0.6.4
uses: zizmorcore/zizmor-action@53290badfa369cb0bf7ad16342cb62c2fa47ac7b # v0.8.0
with:
# there is a pass/fail gate as a repo ruleset (if there is no ruleset configured then the action will pass by default)
advanced-security: true
+88 -12
View File
@@ -51,8 +51,19 @@ jobs:
name: "Unit tests"
runs-on: ubuntu-latest
needs: [Unit-Test-Linux, Unit-Test-Windows]
# a skipped job satisfies a required check, so this job must always run and fail explicitly
# unless every dependency succeeded (otherwise a failing PR can still be auto-merged)
if: ${{ always() }}
steps:
- run: echo All unit tests passed!
- name: Require all unit test jobs to succeed
env:
NEEDS: ${{ toJSON(needs) }}
run: |
echo "$NEEDS" | jq -e 'length > 0 and all(.[]; .result == "success")' > /dev/null || {
echo "$NEEDS" | jq -r 'to_entries[] | "\(.key): \(.value.result)"'
exit 1
}
echo All unit tests passed!
Unit-Test-Linux:
name: "Unit tests (Linux)"
@@ -85,16 +96,21 @@ jobs:
with:
persist-credentials: false
# go-make's setup only (not ./.github/actions/bootstrap): no apt, binny tools, or fixture cache are needed
# go-make's setup only (not ./.github/actions/bootstrap): binny ships no windows release, so binny tools
# are unavailable here (`make unit` skips the fixture refresh on windows)
- name: Setup go + go-make tooling
uses: anchore/go-make/.github/actions/setup@fa5421b3bf24c9d18f11a736c5eb4172f2b7d683 # v0.8.1
with:
# keep in sync with the go-version default in ./.github/actions/bootstrap
go-version: "1.26.2"
uses: anchore/go-make/.github/actions/setup@cf465259f4c9f0b0b0bcb35164ede11f7dc6d9f6 # v0.9.6
# the binny-free version of `make download-test-fixture-cache` (keep the oras version in sync with .binny.yaml)
- name: Download test fixture cache
shell: bash
run: |
curl -sSfL https://github.com/oras-project/oras/releases/download/v1.3.4/oras_1.3.4_windows_amd64.zip -o "$RUNNER_TEMP/oras.zip"
unzip -q "$RUNNER_TEMP/oras.zip" oras.exe -d "$RUNNER_TEMP"
"$RUNNER_TEMP/oras.exe" pull ghcr.io/anchore/oss-cache/syft-test-fixture-cache:v2
# the full unit suite leans on linux docker fixtures, so this runs the windows-relevant subset
- name: Run unit tests
run: make unit:windows
run: make unit
Integration-Test:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
@@ -152,12 +168,72 @@ jobs:
run: make snapshot-smoke-test
- name: Upload snapshot artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a #v7.0.1
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 #v7.0.2
with:
name: snapshot
path: snapshot/
retention-days: 30
Containers-Storage-Podman:
name: "Containers-storage tests (Podman)"
needs: [Build-Snapshot-Artifacts]
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 #v7.0.1
with:
persist-credentials: false
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 #v8.0.2
with:
name: snapshot
path: snapshot
# go-make's setup only (not ./.github/actions/bootstrap): the test only needs make + the snapshot binary
- name: Setup go + go-make tooling
uses: anchore/go-make/.github/actions/setup@cf465259f4c9f0b0b0bcb35164ede11f7dc6d9f6 # v0.9.6
- name: Install Podman
run: |
sudo apt-get update
sudo apt-get install --no-install-recommends -y jq podman uidmap
- name: Scan image from Podman's local store
run: make containers-storage-test
env:
BUILDER: podman
Containers-Storage-Buildah:
name: "Containers-storage tests (Buildah)"
needs: [Build-Snapshot-Artifacts]
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 #v7.0.1
with:
persist-credentials: false
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 #v8.0.2
with:
name: snapshot
path: snapshot
# go-make's setup only (not ./.github/actions/bootstrap): the test only needs make + the snapshot binary
- name: Setup go + go-make tooling
uses: anchore/go-make/.github/actions/setup@cf465259f4c9f0b0b0bcb35164ede11f7dc6d9f6 # v0.9.6
- name: Install Buildah
run: |
sudo apt-get update
sudo apt-get install --no-install-recommends -y buildah jq uidmap
- name: Scan image from Buildah's local store
run: make containers-storage-test
env:
BUILDER: buildah
Acceptance-Linux:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
name: "Acceptance tests (Linux)"
@@ -176,7 +252,7 @@ jobs:
download-test-fixture-cache: true
- name: Download snapshot artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c #v8.0.1
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 #v8.0.2
with:
name: snapshot
path: snapshot
@@ -222,7 +298,7 @@ jobs:
download-test-fixture-cache: true
- name: Download snapshot artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c #v8.0.1
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 #v8.0.2
with:
name: snapshot
path: snapshot
@@ -254,7 +330,7 @@ jobs:
download-test-fixture-cache: true
- name: Download snapshot artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c #v8.0.1
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 #v8.0.2
with:
name: snapshot
path: snapshot
+4
View File
@@ -1,6 +1,10 @@
version: "2"
run:
tests: false
# keep in sync with buildTags in .make/main.go
build-tags:
- containers_image_openpgp
- exclude_graphdriver_btrfs
linters:
default: none
enable:
+4 -6
View File
@@ -15,6 +15,7 @@ builds:
binary: syft
goos: [linux]
goarch: [amd64, arm64, ppc64le, riscv64, s390x]
tags: [containers_image_openpgp, exclude_graphdriver_btrfs]
mod_timestamp: &build-timestamp '{{ .CommitTimestamp }}'
ldflags: &build-ldflags |
-w
@@ -167,16 +168,13 @@ sboms:
- "--output"
- "json=$document"
# the signature, certificate, and transparency log proof all land in a single sigstore bundle.
signs:
- cmd: .tool/cosign
signature: "${artifact}.sig"
certificate: "${artifact}.pem"
signature: "${artifact}.sigstore.json"
args:
- "sign-blob"
- "--use-signing-config=false"
- "--oidc-issuer=https://token.actions.githubusercontent.com"
- "--output-certificate=${certificate}"
- "--output-signature=${signature}"
- "--bundle=${signature}"
- "${artifact}"
- "--yes"
artifacts: checksum
+5 -5
View File
@@ -1,12 +1,12 @@
module github.com/anchore/syft/.make
go 1.25.8
go 1.26.0
require github.com/anchore/go-make v0.8.1
require github.com/anchore/go-make v0.9.6
require (
github.com/bmatcuk/doublestar/v4 v4.10.0 // indirect
github.com/bmatcuk/doublestar/v4 v4.10.3 // indirect
github.com/goccy/go-yaml v1.19.2 // indirect
golang.org/x/mod v0.40.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/mod v0.42.0 // indirect
golang.org/x/sys v0.49.0 // indirect
)
+8 -8
View File
@@ -1,10 +1,10 @@
github.com/anchore/go-make v0.8.1 h1:GAhPXR+9qVQhFB1pfVlHFOzxAt3dTDTUIA7Tp7nx9tg=
github.com/anchore/go-make v0.8.1/go.mod h1:xALbVsPsDoJnJxdkjPlxOLZV7GbAdT5Pv3gd0kP1Lz0=
github.com/bmatcuk/doublestar/v4 v4.10.0 h1:zU9WiOla1YA122oLM6i4EXvGW62DvKZVxIe6TYWexEs=
github.com/bmatcuk/doublestar/v4 v4.10.0/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/anchore/go-make v0.9.6 h1:HjO7H/M/WZ4v3B73/EuShuQHHblQJGwHDrl/MqCavWs=
github.com/anchore/go-make v0.9.6/go.mod h1:zE6KeB+nY+sMSWHYx2uaiOznLAq3CD9JvFlzWDDpKw4=
github.com/bmatcuk/doublestar/v4 v4.10.3 h1:1UIv1+fqH3++ViVkpnaPTB+Pi1UHycnPFAq14zgK+gI=
github.com/bmatcuk/doublestar/v4 v4.10.3/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
golang.org/x/mod v0.40.0 h1:hUv+3cXcdRHz08UmSiOob7sadHig73uo5bkXxQ/tvUs=
golang.org/x/mod v0.40.0/go.mod h1:0/weTWkPWGBikyTWAX3dkjVztMmBA5hM0DH6BElSupE=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/mod v0.42.0 h1:ICeL3t2L30ljSPHHxDnR4fqVcBt/zR2zJAb9juavmQg=
golang.org/x/mod v0.42.0/go.mod h1:u5fzIrRInkL8MaO1cbRkUHT64v303YIT5KEqpNr7jRs=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
+31 -15
View File
@@ -1,7 +1,6 @@
package main
import (
"os"
"path/filepath"
"runtime"
"strconv"
@@ -17,6 +16,12 @@ import (
"github.com/anchore/go-make/tasks/gotest"
)
// buildTags must match the linux build in .goreleaser.yaml so tests and lint compile the same code that ships
// (containers_image_openpgp compiles in stereoscope's real containers-storage provider instead of its stub).
// exclude_graphdriver_btrfs drops the cgo-only btrfs driver, which needs libbtrfs headers whenever cgo is on (e.g.
// under -race); release builds are CGO_ENABLED=0 and never include it anyway.
const buildTags = "containers_image_openpgp,exclude_graphdriver_btrfs"
func main() {
Makefile(
// shared anchore tasks
@@ -30,6 +35,7 @@ func main() {
gotest.Name("unit"),
gotest.ExcludeGlob("**/test/**"),
gotest.CoverageThreshold(62),
gotest.Tags(buildTags),
race(),
),
@@ -53,7 +59,7 @@ func main() {
raceFlag = " -race"
}
Run(
"go test -count=1 -timeout=30m"+raceFlag+" ./cmd/syft/internal/test/integration/...",
"go test -count=1 -timeout=30m -tags="+buildTags+raceFlag+" ./cmd/syft/internal/test/integration/...",
run.Env("GODEBUG", "dontfreezetheworld=1"),
)
},
@@ -67,7 +73,7 @@ func main() {
Log("race detector disabled (RACE=false); skipping race smoke")
return
}
Run("go run -race cmd/syft/main.go anchore/test_images:grype-quality-dotnet-69f15d2")
Run("go run -race -tags="+buildTags+" cmd/syft/main.go anchore/test_images:grype-quality-dotnet-69f15d2")
},
},
@@ -97,19 +103,20 @@ func main() {
},
},
// windows unit tests: deliberately not hooked into "test" (no RunsOn). The full unit suite leans on
// linux docker fixture images and posix-only assumptions, so this covers the packages that own
// host path handling (where windows bugs live) and skips the tests that need docker image fixtures.
// ponytail: curated package list, widen it as more of the suite is made windows-safe.
// containers-storage tests: build a fixture image with BUILDER (podman or buildah) into an isolated local store
// and scan it with the snapshot binary. Linux only, and not hooked into "test" since it needs the builder installed.
Task{
Name: "unit:windows",
Description: "run the windows-relevant subset of unit tests",
Name: "containers-storage-test",
Description: "Run containers-storage tests (BUILDER=podman|buildah)",
Run: func() {
Run("go test -count=1"+
` -skip "Image|Squash|AllLayers|MixFileTypes|GivenCoordinates"`+
" ./syft/internal/windows/... ./syft/internal/fileresolver/... ./syft/source/directorysource/... ./syft/source/filesource/..."+
" ./syft/file ./syft/file/cataloger/filedigest/... ./syft/file/cataloger/filecontent/...",
run.Stdout(os.Stderr),
bin := snapshotBinPath()
if !file.Exists(bin) {
Log("snapshot binary not found at %s; building single-target snapshot", bin)
Run("make snapshot:single-target")
}
Run(
"bash test/containers-storage/source-test.sh "+config.Env("BUILDER", "podman"),
run.Env("SYFT_BINARY_LOCATION", bin),
)
},
},
@@ -157,7 +164,7 @@ func main() {
Task{
Name: "unit:syft",
RunsOn: lang.List("unit"),
Dependencies: Deps("refresh-fixtures"),
Dependencies: unitFixtureDeps(),
},
Task{
Name: "clean:syft",
@@ -185,6 +192,15 @@ func raceEnabled() bool {
return config.CI && !config.Windows
}
// unitFixtureDeps skips the fixture cache refresh on windows: it needs binny-managed tools (binny ships
// no windows release) and linux docker, so fixture-backed tests there have to skip themselves instead.
func unitFixtureDeps() []string {
if config.Windows {
return nil
}
return Deps("refresh-fixtures")
}
// race applies raceEnabled() to a gotest suite. gotest exposes no functional option
// for the race detector, but gotest.Config.Race is exported.
func race() gotest.Option {
+1 -1
View File
@@ -1,6 +1,6 @@
# Syft Contributors
The following Syft components were contributed by external authors/organizations.
This file credits notable contributions from external authors and organizations; see `git log` for everyone who has contributed to Syft.
## GraalVM Native Image
+4
View File
@@ -0,0 +1,4 @@
Copyright (c) Anchore, Inc.
See LICENSE for the full license text.
See CONTRIBUTORS.md and git log for a full list of contributors.
+12
View File
@@ -61,6 +61,18 @@ syft <image> -o cyclonedx-json
syft <image> -o spdx-json=./spdx.json -o cyclonedx-json=./cdx.json
```
### Local containers-storage images
On Linux, syft can scan images from a local containers-storage store (e.g. built with Buildah or Podman) when asked explicitly; plain image references never look there:
```bash
syft --from containers-storage localhost/myimage:latest
# rootless stores must be read from inside the builder's user namespace
podman unshare syft --from containers-storage localhost/myimage:latest
```
This is included in the Linux release binaries. When building from source, add `-tags containers_image_openpgp,exclude_graphdriver_btrfs`.
> [!TIP]
> **Check out the [Getting Started guide](https://oss.anchore.com/docs/guides/sbom/getting-started/)** to explore all of the capabilities and features.
+25
View File
@@ -0,0 +1,25 @@
package cli
import (
"io"
"testing"
"github.com/stretchr/testify/require"
"github.com/anchore/clio"
"github.com/anchore/syft/internal/redact"
)
func TestCommandCanRunMultipleTimes(t *testing.T) {
// https://github.com/anchore/syft/issues/2285
t.Cleanup(redact.Reset)
for i := 0; i < 2; i++ {
_, cmd := create(clio.Identification{Name: "syft"}, io.Discard)
cmd.SetArgs([]string{"cataloger", "list", "-o", "json"})
require.NoError(t, cmd.Execute())
// the store is released at the end of each run so the next one can set its own
require.Nil(t, redact.Get())
}
}
+9 -1
View File
@@ -41,6 +41,8 @@ func AppClioSetupConfig(id clio.Identification, out io.Writer) *clio.SetupConfig
stereoscope.SetBus(state.Bus)
bus.Set(state.Bus)
// the redact store is process-global and only cleared when a run ends (see the post-run below), so
// setting it while another run is still in flight panics rather than splitting secrets across stores.
redact.Set(state.RedactStore)
log.Set(state.Logger)
@@ -48,8 +50,14 @@ func AppClioSetupConfig(id clio.Identification, out io.Writer) *clio.SetupConfig
return nil
},
).
WithPostRuns(func(_ *clio.State, _ error) {
WithPostRuns(func(state *clio.State, _ error) {
stereoscope.Cleanup() //nolint:staticcheck // we don't have access to the image object here
// release the redact store now that nothing in this run can report anymore, so a later run in the same
// process (e.g. syft embedded as a library) can set its own. Only release the store this run set.
if redact.Get() == state.RedactStore {
redact.Reset()
}
})
return clioCfg
}
@@ -0,0 +1,32 @@
package internal
import (
"io"
"testing"
"github.com/stretchr/testify/require"
"github.com/anchore/clio"
"github.com/anchore/go-logger/adapter/discard"
gologgerredact "github.com/anchore/go-logger/adapter/redact"
"github.com/anchore/syft/internal/redact"
)
func TestAppClioSetupConfigInitializerPanicsWhileRunInFlight(t *testing.T) {
t.Cleanup(redact.Reset)
cfg := AppClioSetupConfig(clio.Identification{Name: "syft"}, io.Discard)
require.Len(t, cfg.Initializers, 1)
newState := func() *clio.State {
return &clio.State{Logger: discard.New(), RedactStore: gologgerredact.NewStore()}
}
require.NoError(t, cfg.Initializers[0](newState()))
// a second run starting before the first has finished must not silently replace the store, otherwise
// secrets added to the first store would no longer be redacted
require.PanicsWithValue(t, "replace existing redaction store (probably unintentional)", func() {
_ = cfg.Initializers[0](newState())
})
}
+4
View File
@@ -114,6 +114,10 @@ func RunConvert(ctx context.Context, opts *ConvertOptions, userInput string) err
if err != nil {
return err
}
// output files are opened up front; release them if we bail out before writing
if c, ok := writer.(io.Closer); ok {
defer func() { _ = c.Close() }()
}
}
for _, output := range unchanged {
+10
View File
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
"io"
"os"
"reflect"
"strings"
@@ -181,6 +182,10 @@ func runScan(ctx context.Context, id clio.Identification, opts *scanOptions, use
if err != nil {
return err
}
// output files are opened up front; release them if we bail out before writing
if c, ok := writer.(io.Closer); ok {
defer func() { _ = c.Close() }()
}
sources := opts.From
if len(sources) == 0 {
@@ -255,6 +260,11 @@ func getSource(ctx context.Context, opts *options.Catalog, userInput string, sou
cfg = cfg.WithDigestAlgorithms(hashers...)
}
if source.ParseScope(opts.Scope) != source.SquashedScope {
// every other scope reads each layer's squash, which is cheaper to build while the image is read
ctx = internal.WithEagerLayerSquashes(ctx)
}
src, err := syft.GetSource(ctx, userInput, cfg)
if err != nil {
return nil, fmt.Errorf("could not determine source: %w", err)
+10 -3
View File
@@ -4,6 +4,7 @@ import (
"io"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"time"
@@ -22,9 +23,10 @@ func Test_defaultDir(t *testing.T) {
homeDir := filepath.Join(tmpDir, "fake-home")
tests := []struct {
name string
env map[string]string
expected string
name string
env map[string]string
expected string
skipOnWindows bool
}{
{
name: "no-xdg",
@@ -32,6 +34,8 @@ func Test_defaultDir(t *testing.T) {
"HOME": homeDir,
},
expected: homeDir,
// xdg falls back to LOCALAPPDATA on windows, not HOME
skipOnWindows: true,
},
{
name: "xdg-cache",
@@ -59,6 +63,9 @@ func Test_defaultDir(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if test.skipOnWindows && runtime.GOOS == "windows" {
t.Skip("posix-only cache dir fallback")
}
restoreCache(t)
for k, v := range test.env {
+12 -5
View File
@@ -102,6 +102,7 @@ func (cfg Catalog) ToSBOMConfig(id clio.Identification) *syft.CreateSBOMConfig {
WithComplianceConfig(cfg.ToComplianceConfig()).
WithUnknownsConfig(cfg.ToUnknownsConfig()).
WithSearchConfig(cfg.ToSearchConfig()).
WithArchiveConfig(cfg.ToArchiveConfig()).
WithPackagesConfig(cfg.ToPackagesConfig()).
WithLicenseConfig(cfg.ToLicenseConfig()).
WithFilesConfig(cfg.ToFilesConfig()).
@@ -118,6 +119,15 @@ func (cfg Catalog) ToSearchConfig() cataloging.SearchConfig {
}
}
func (cfg Catalog) ToArchiveConfig() cataloging.ArchiveSearchConfig {
return cataloging.DefaultArchiveSearchConfig().
WithIncludeIndexedArchives(cfg.Package.SearchIndexedArchives).
WithIncludeUnindexedArchives(cfg.Package.SearchUnindexedArchives).
WithMaxDepth(cfg.Package.NestedArchiveMaxDepth).
WithMaxMemoryBytes(cfg.Package.NestedArchiveMaxMemoryBytes).
WithMaxDiskBytes(cfg.Package.NestedArchiveMaxDiskBytes)
}
func (cfg Catalog) ToRelationshipsConfig() cataloging.RelationshipsConfig {
return cataloging.RelationshipsConfig{
PackageFileOwnership: cfg.Relationships.PackageFileOwnership,
@@ -136,6 +146,7 @@ func (cfg Catalog) ToComplianceConfig() cataloging.ComplianceConfig {
func (cfg Catalog) ToUnknownsConfig() cataloging.UnknownsConfig {
return cataloging.UnknownsConfig{
RemoveWhenPackagesDefined: cfg.Unknowns.RemoveWhenPackagesDefined,
IncludeExecutablesWithoutPackages: cfg.Unknowns.ExecutablesWithoutPackages,
IncludeUnexpandedArchives: cfg.Unknowns.UnexpandedArchives,
}
@@ -169,10 +180,6 @@ func (cfg Catalog) ToLicenseConfig() cataloging.LicenseConfig {
}
func (cfg Catalog) ToPackagesConfig() pkgcataloging.Config {
archiveSearch := cataloging.ArchiveSearchConfig{
IncludeIndexedArchives: cfg.Package.SearchIndexedArchives,
IncludeUnindexedArchives: cfg.Package.SearchUnindexedArchives,
}
return pkgcataloging.Config{
Binary: binary.DefaultClassifierCatalogerConfig(),
Cpp: cpp.DefaultCatalogerConfig().
@@ -219,7 +226,7 @@ func (cfg Catalog) ToPackagesConfig() pkgcataloging.Config {
WithMavenLocalRepositoryDir(cfg.Java.MavenLocalRepositoryDir).
WithUseNetwork(*multiLevelOption(false, enrichmentEnabled(cfg.Enrich, task.Java, task.Maven), cfg.Java.UseNetwork)).
WithMavenBaseURL(cfg.Java.MavenURL).
WithArchiveTraversal(archiveSearch, cfg.Java.MaxParentRecursiveDepth).
WithMaxParentRecursiveDepth(cfg.Java.MaxParentRecursiveDepth).
WithResolveTransitiveDependencies(cfg.Java.ResolveTransitiveDependencies),
}
}
+16 -5
View File
@@ -6,9 +6,12 @@ import (
)
type packageConfig struct {
SearchUnindexedArchives bool `yaml:"search-unindexed-archives" json:"search-unindexed-archives" mapstructure:"search-unindexed-archives"`
SearchIndexedArchives bool `yaml:"search-indexed-archives" json:"search-indexed-archives" mapstructure:"search-indexed-archives"`
ExcludeBinaryOverlapByOwnership bool `yaml:"exclude-binary-overlap-by-ownership" json:"exclude-binary-overlap-by-ownership" mapstructure:"exclude-binary-overlap-by-ownership"` // exclude synthetic binary packages owned by os package files
SearchUnindexedArchives bool `yaml:"search-unindexed-archives" json:"search-unindexed-archives" mapstructure:"search-unindexed-archives"`
SearchIndexedArchives bool `yaml:"search-indexed-archives" json:"search-indexed-archives" mapstructure:"search-indexed-archives"`
ExcludeBinaryOverlapByOwnership bool `yaml:"exclude-binary-overlap-by-ownership" json:"exclude-binary-overlap-by-ownership" mapstructure:"exclude-binary-overlap-by-ownership"` // exclude synthetic binary packages owned by os package files
NestedArchiveMaxDepth int `yaml:"nested-archive-max-depth" json:"nested-archive-max-depth" mapstructure:"nested-archive-max-depth"`
NestedArchiveMaxMemoryBytes int64 `yaml:"nested-archive-max-memory-bytes" json:"nested-archive-max-memory-bytes" mapstructure:"nested-archive-max-memory-bytes"`
NestedArchiveMaxDiskBytes int64 `yaml:"nested-archive-max-disk-bytes" json:"nested-archive-max-disk-bytes" mapstructure:"nested-archive-max-disk-bytes"`
}
var _ interface {
@@ -17,12 +20,17 @@ var _ interface {
func (o *packageConfig) DescribeFields(descriptions clio.FieldDescriptionSet) {
descriptions.Add(&o.SearchIndexedArchives, `search within archives that do contain a file index to search against (zip)
note: for now this only applies to the java package cataloger`)
note: this only applies to the java package cataloger, and is ignored when nested-archive-max-depth is not 0 (every archive is searched)`)
descriptions.Add(&o.SearchUnindexedArchives, `search within archives that do not contain a file index to search against (tar, tar.gz, tar.bz2, etc)
note: enabling this may result in a performance impact since all discovered compressed tars will be decompressed
note: for now this only applies to the java package cataloger`)
note: this only applies to the java package cataloger, and is ignored when nested-archive-max-depth is not 0 (every archive is searched)`)
descriptions.Add(&o.ExcludeBinaryOverlapByOwnership, `allows users to exclude synthetic binary packages from the sbom
these packages are removed if an overlap with a non-synthetic package is found`)
descriptions.Add(&o.NestedArchiveMaxDepth, `maximum depth to recursively catalog nested archives (0 = disabled, -1 = as deep as 16 levels, the most allowed)
note: when enabled this replaces the java cataloger's own unbounded jar-in-jar recursion, so archives deeper than this are reported as unknowns rather than cataloged`)
descriptions.Add(&o.NestedArchiveMaxMemoryBytes, `maximum bytes of nested archive content held in memory at once (0 = none, -1 = unbounded)`)
descriptions.Add(&o.NestedArchiveMaxDiskBytes, `maximum bytes of nested archive content on disk at once (0 = none, -1 = unbounded)
note: this and nested-archive-max-memory-bytes cannot both be 0 when nested-archive-max-depth is not 0`)
}
func defaultPackageConfig() packageConfig {
@@ -31,5 +39,8 @@ func defaultPackageConfig() packageConfig {
SearchIndexedArchives: c.IncludeIndexedArchives,
SearchUnindexedArchives: c.IncludeUnindexedArchives,
ExcludeBinaryOverlapByOwnership: true,
NestedArchiveMaxDepth: c.MaxDepth,
NestedArchiveMaxMemoryBytes: c.MaxMemoryBytes,
NestedArchiveMaxDiskBytes: c.MaxDiskBytes,
}
}
+53
View File
@@ -0,0 +1,53 @@
package options
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/cataloging"
)
func Test_packageConfigDefaultsMatchArchiveSearchConfig(t *testing.T) {
want := cataloging.DefaultArchiveSearchConfig()
got := defaultPackageConfig()
assert.Equal(t, want.MaxDepth, got.NestedArchiveMaxDepth)
assert.Equal(t, want.MaxMemoryBytes, got.NestedArchiveMaxMemoryBytes)
assert.Equal(t, want.MaxDiskBytes, got.NestedArchiveMaxDiskBytes)
require.Zero(t, got.NestedArchiveMaxDepth, "nested archive cataloging must be off by default")
}
func Test_ToArchiveConfig(t *testing.T) {
defaults := cataloging.DefaultArchiveSearchConfig()
t.Run("defaults round trip unchanged", func(t *testing.T) {
cfg := Catalog{Package: defaultPackageConfig()}
got := cfg.ToArchiveConfig()
assert.Equal(t, defaults.MaxDepth, got.MaxDepth)
assert.Equal(t, defaults.MaxMemoryBytes, got.MaxMemoryBytes)
assert.Equal(t, defaults.MaxDiskBytes, got.MaxDiskBytes)
})
t.Run("every setting is threaded through", func(t *testing.T) {
// a key that parses but is not carried into the archive config is invisible in a config
// dump, so assert each one arrives
cfg := Catalog{Package: packageConfig{
NestedArchiveMaxDepth: 3,
NestedArchiveMaxMemoryBytes: 111,
NestedArchiveMaxDiskBytes: 3333,
SearchIndexedArchives: true,
SearchUnindexedArchives: true,
}}
got := cfg.ToArchiveConfig()
assert.Equal(t, 3, got.MaxDepth)
assert.Equal(t, int64(111), got.MaxMemoryBytes)
assert.Equal(t, int64(3333), got.MaxDiskBytes)
assert.True(t, got.IncludeIndexedArchives)
assert.True(t, got.IncludeUnindexedArchives)
})
}
+2 -2
View File
@@ -36,7 +36,7 @@ var _ clio.PostLoader = (*imageSource)(nil)
func (o *sourceConfig) DescribeFields(descriptions clio.FieldDescriptionSet) {
descriptions.Add(&o.File.Digests, `the file digest algorithms to use on the scanned file (options: "md5", "sha1", "sha224", "sha256", "sha384", "sha512")`)
descriptions.Add(&o.Image.DefaultPullSource, `allows users to specify which image source should be used to generate the sbom
valid values are: registry, docker, podman`)
valid values are: registry, docker, podman, containers-storage`)
}
type imageSource struct {
@@ -74,7 +74,7 @@ func (c *imageSource) PostLoad() error {
return checkDefaultSourceValues(c.DefaultPullSource)
}
var validDefaultSourceValues = []string{"registry", "docker", "podman", ""}
var validDefaultSourceValues = []string{"registry", "docker", "podman", "containers-storage", ""}
func checkDefaultSourceValues(source string) error {
validValues := strset.New(validDefaultSourceValues...)
+28 -4
View File
@@ -2,10 +2,11 @@ package options
import (
"bytes"
"errors"
"fmt"
"io"
"os"
"path"
"path/filepath"
"sort"
"strings"
@@ -20,7 +21,10 @@ import (
"github.com/anchore/syft/syft/sbom"
)
var _ sbom.Writer = (*sbomMultiWriter)(nil)
var _ interface {
io.Closer
sbom.Writer
} = (*sbomMultiWriter)(nil)
var _ interface {
io.Closer
@@ -164,6 +168,12 @@ func newSBOMMultiWriter(options ...sbomWriterDescription) (_ *sbomMultiWriter, e
}
out := &sbomMultiWriter{}
defer func() {
// don't leak the files already opened when a later output fails
if err != nil {
_ = out.Close()
}
}()
for _, option := range options {
switch len(option.Path) {
@@ -172,8 +182,8 @@ func newSBOMMultiWriter(options ...sbomWriterDescription) (_ *sbomMultiWriter, e
format: option.Format,
})
default:
// create any missing subdirectories
dir := path.Dir(option.Path)
// create any missing subdirectories (option.Path is a host path, so this must be filepath)
dir := filepath.Dir(option.Path)
if dir != "" {
s, err := os.Stat(dir)
if err != nil {
@@ -210,6 +220,20 @@ func (m *sbomMultiWriter) Write(s sbom.SBOM) (errs error) {
return errs
}
// Close releases any files held by the writers. Write already closes each file it writes to, so this matters
// when the SBOM is never written (e.g. an error after the writer was created).
func (m *sbomMultiWriter) Close() error {
var errs error
for _, w := range m.writers {
if c, ok := w.(io.Closer); ok {
if err := c.Close(); err != nil && !errors.Is(err, os.ErrClosed) {
errs = multierror.Append(errs, err)
}
}
}
return errs
}
// sbomStreamWriter implements sbom.Writer for a given format and io.Writer, also providing a close function for cleanup
type sbomStreamWriter struct {
format sbom.FormatEncoder
+2 -2
View File
@@ -202,9 +202,9 @@ func Test_newSBOMMultiWriter(t *testing.T) {
if test.err {
assert.Error(t, err)
return
} else {
assert.NoError(t, err)
}
require.NoError(t, err)
t.Cleanup(func() { _ = mw.Close() })
assert.Len(t, mw.writers, len(test.expected))
+7 -1
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"io"
"os"
"strings"
"sync"
"time"
@@ -186,7 +187,12 @@ func (m *UI) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
}
func (m UI) View() string {
return m.frame.View()
v := m.frame.View()
// bubbletea erases the line the cursor rests on at exit, which would drop the frame's last line
if !strings.HasSuffix(v, "\n") {
return v + "\n"
}
return v
}
var (
+31
View File
@@ -0,0 +1,31 @@
package ui
import (
"io"
"testing"
tea "github.com/charmbracelet/bubbletea"
"github.com/stretchr/testify/require"
"github.com/anchore/bubbly/bubbles/frame"
)
var _ tea.Model = (*staticModel)(nil)
type staticModel string
func Test_UI_viewEndsOnAnEmptyLine(t *testing.T) {
// bubbletea erases the line the cursor rests on when the program stops, so a view whose last line
// carries content loses it. Unless a log line happened to be in the footer, that dropped the last row
// of the cataloging tree on any source that logs nothing.
subject := New(io.Discard, false)
subject.frame.(*frame.Frame).AppendModel(staticModel("first row\nlast row"))
require.Equal(t, "first row\nlast row\n", subject.View())
}
func (m staticModel) Init() tea.Cmd { return nil }
func (m staticModel) Update(tea.Msg) (tea.Model, tea.Cmd) { return m, nil }
func (m staticModel) View() string { return string(m) }
+83 -86
View File
@@ -12,38 +12,37 @@ require (
github.com/acobaugh/osrelease v0.1.0
github.com/adrg/xdg v0.5.3
github.com/anchore/bubbly v0.2.2
github.com/anchore/clio v0.1.1
github.com/anchore/clio v0.1.2
github.com/anchore/fangs v0.1.1
github.com/anchore/go-collections v0.1.1
github.com/anchore/go-homedir v0.1.1
github.com/anchore/go-logger v0.2.0
github.com/anchore/go-macholibre v0.1.1
github.com/anchore/go-rpmdb v0.2.0
github.com/anchore/go-rpmdb v0.2.2
github.com/anchore/go-sync v0.1.2
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.2.0
github.com/anchore/stereoscope v0.3.3
github.com/anchore/stereoscope v0.4.1
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be
github.com/aquasecurity/go-pep440-version v0.0.1
github.com/bitnami/go-version v0.0.0-20250131085805-b1f57a8634ef
github.com/blakesmith/ar v0.0.0-20190502131153-809d4375e1fb
github.com/bmatcuk/doublestar/v4 v4.10.0
github.com/bmatcuk/doublestar/v4 v4.10.3
github.com/cespare/xxhash/v2 v2.3.0
github.com/charmbracelet/bubbles v1.0.0
github.com/charmbracelet/bubbletea v1.3.10
github.com/charmbracelet/lipgloss v1.1.0
github.com/dave/jennifer v1.7.1
github.com/deitch/magic v0.0.0-20230404182410-1ff89d7342da
github.com/deitch/magic v1.0.0
github.com/diskfs/go-diskfs v1.9.4
github.com/distribution/reference v0.6.0
github.com/dustin/go-humanize v1.1.0
github.com/elliotchance/phpserialize v1.4.0
github.com/facebookincubator/nvdtools v0.1.5
github.com/github/go-spdx/v2 v2.7.0
github.com/gkampitakis/go-snaps v0.5.23
github.com/go-git/go-billy/v5 v5.9.1
github.com/go-git/go-git/v5 v5.19.2
github.com/go-test/deep v1.1.1
github.com/gkampitakis/go-snaps v0.5.24
github.com/go-git/go-billy/v5 v5.9.2
github.com/go-git/go-git/v5 v5.19.3
github.com/go-test/deep v1.1.2
github.com/go-viper/mapstructure/v2 v2.5.0
github.com/goccy/go-yaml v1.19.2
github.com/gohugoio/hashstructure v0.6.0
@@ -51,10 +50,10 @@ require (
github.com/google/go-containerregistry v0.22.1
github.com/google/licensecheck v0.3.1
github.com/google/uuid v1.6.0
github.com/gookit/color v1.6.1
github.com/gpustack/gguf-parser-go v0.26.3
github.com/gookit/color v1.6.3
github.com/gpustack/gguf-parser-go v0.27.0
github.com/hashicorp/go-cleanhttp v0.5.2
github.com/hashicorp/go-getter v1.8.9
github.com/hashicorp/go-getter v1.8.10
github.com/hashicorp/go-multierror v1.1.1
github.com/hashicorp/hcl/v2 v2.25.0
github.com/iancoleman/strcase v0.3.0
@@ -62,13 +61,14 @@ require (
github.com/jedib0t/go-pretty/v6 v6.8.3
github.com/jinzhu/copier v0.4.0
github.com/kastenhq/goversion v0.0.0-20230811215019-93b2f8823953
github.com/klauspost/compress v1.20.0
github.com/klauspost/compress v1.20.3
github.com/magiconair/properties v1.18.12
github.com/mholt/archives v0.1.5
github.com/moby/sys/mountinfo v0.7.2
github.com/nix-community/go-nix v0.0.0-20250101154619-4bdde671e0a1
github.com/olekukonko/tablewriter v1.1.5
github.com/opencontainers/go-digest v1.0.0
github.com/pb33f/ordered-map/v2 v2.3.2
github.com/pelletier/go-toml v1.9.5
github.com/quasilyte/go-ruleguard/dsl v0.3.23
github.com/rust-secure-code/go-rustaudit v0.0.0-20250226111315-e20ec32e963c
@@ -83,7 +83,8 @@ require (
github.com/spf13/afero v1.15.0
github.com/spf13/cobra v1.10.2
github.com/stretchr/testify v1.12.1
github.com/ulikunitz/xz v0.5.16
github.com/tailscale/hujson v0.0.0-20260302212456-ecc657c15afd
github.com/ulikunitz/xz v0.5.17
github.com/vbatts/go-mtree v0.7.0
github.com/vifraa/gopom v1.0.0
github.com/wagoodman/go-partybus v0.0.0-20230516145632-8ccac152c651
@@ -93,38 +94,36 @@ require (
go.uber.org/goleak v1.3.0
go.yaml.in/yaml/v3 v3.0.5
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f
golang.org/x/mod v0.41.0
golang.org/x/net v0.59.0
golang.org/x/mod v0.42.0
golang.org/x/net v0.61.0
golang.org/x/sys v0.49.0
golang.org/x/time v0.16.0
golang.org/x/tools v0.50.0
modernc.org/sqlite v1.59.0
golang.org/x/tools v0.52.0
howett.net/plist v1.0.1
modernc.org/sqlite v1.61.0
)
require (
github.com/pb33f/ordered-map/v2 v2.3.1
github.com/tailscale/hujson v0.0.0-20260302212456-ecc657c15afd
howett.net/plist v1.0.1
)
require github.com/github/go-spdx/v2 v2.8.0
require (
cel.dev/expr v0.25.2 // indirect
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.23.2 // indirect
cloud.google.com/go/auth v0.23.3 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
cloud.google.com/go/compute/metadata v0.9.1 // indirect
cloud.google.com/go/iam v1.12.0 // indirect
cloud.google.com/go/monitoring v1.30.0 // indirect
cloud.google.com/go/storage v1.65.1 // indirect
cloud.google.com/go/storage v1.68.0 // indirect
cyphar.com/go-pathrs v0.2.5 // indirect
dario.cat/mergo v1.0.2 // indirect
github.com/DataDog/zstd v1.5.5 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.33.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.34.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.57.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.57.0 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29 // indirect
github.com/Microsoft/go-winio v0.6.3-0.20260930231756-f19d9717deb0 // indirect
github.com/Microsoft/hcsshim v0.15.0-rc.4 // indirect
github.com/ProtonMail/go-crypto v1.4.1 // indirect
github.com/ProtonMail/go-crypto v1.5.2 // indirect
github.com/STARRY-S/zip v0.2.3 // indirect
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/agext/levenshtein v1.2.1 // indirect
@@ -135,27 +134,26 @@ require (
github.com/apparentlymart/go-textseg/v17 v17.0.1 // indirect
github.com/aquasecurity/go-version v0.0.1 // indirect
github.com/atotto/clipboard v0.1.4 // indirect
github.com/aws/aws-sdk-go-v2 v1.44.0 // indirect
github.com/aws/aws-sdk-go-v2 v1.47.1 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.20 // indirect
github.com/aws/aws-sdk-go-v2/config v1.32.40 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.19.39 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.40 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.40 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.40 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.41 // indirect
github.com/aws/aws-sdk-go-v2/config v1.33.6 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.20.6 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.20.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.5.4 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.8.4 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.5.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.19 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.10.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.40 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.41 // indirect
github.com/aws/aws-sdk-go-v2/service/s3 v1.108.0 // indirect
github.com/aws/aws-sdk-go-v2/service/signin v1.6.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.34.0 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.39.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.46.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.11.5 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.14.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.20.4 // indirect
github.com/aws/aws-sdk-go-v2/service/s3 v1.114.0 // indirect
github.com/aws/aws-sdk-go-v2/service/signin v1.10.1 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.38.1 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.43.1 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.51.1 // indirect
github.com/aws/smithy-go v1.28.1 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/becheran/wildmatch-go v1.0.0 // indirect
github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d // indirect
github.com/bodgit/plumbing v1.3.0 // indirect
github.com/bodgit/sevenzip v1.6.1 // indirect
@@ -172,7 +170,7 @@ require (
github.com/cncf/xds/go v0.0.0-20260202195803-dba9d589def2 // indirect
github.com/containerd/cgroups/v3 v3.1.3 // indirect
github.com/containerd/containerd/api v1.12.0 // indirect
github.com/containerd/containerd/v2 v2.4.1 // indirect
github.com/containerd/containerd/v2 v2.4.2 // indirect
github.com/containerd/continuity v0.5.0 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
@@ -181,16 +179,16 @@ require (
github.com/containerd/log/otel v0.1.0 // indirect
github.com/containerd/platforms v1.0.0-rc.5 // indirect
github.com/containerd/plugin v1.1.0 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.18.2 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.19.0 // indirect
github.com/containerd/ttrpc v1.2.9 // indirect
github.com/containerd/typeurl/v2 v2.3.0 // indirect
github.com/containers/libtrust v0.0.0-20230121012942-c1716e8a8d01 // indirect
github.com/containers/ocicrypt v1.3.2 // indirect
github.com/cyberphone/json-canonicalization v0.0.0-20241213102144-19d51d7fe467 // indirect
github.com/cyphar/filepath-securejoin v0.7.0 // indirect
github.com/docker/cli v29.8.1+incompatible // indirect
github.com/docker/docker-credential-helpers v0.9.8 // indirect
github.com/docker/go-connections v0.8.1 // indirect
github.com/docker/cli v29.9.2+incompatible // indirect
github.com/docker/docker-credential-helpers v0.9.10 // indirect
github.com/docker/go-connections v0.8.2 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
@@ -201,19 +199,19 @@ require (
github.com/felixge/fgprof v0.9.5 // indirect
github.com/felixge/httpsnoop v1.1.0 // indirect
github.com/fsnotify/fsnotify v1.10.1 // indirect
github.com/gabriel-vasile/mimetype v1.4.15 // indirect
github.com/gabriel-vasile/mimetype v1.4.16 // indirect
github.com/gkampitakis/ciinfo v0.3.4 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-jose/go-jose/v4 v4.1.5 // indirect
github.com/go-jose/go-jose/v4 v4.1.6 // indirect
github.com/go-logr/logr v1.4.4 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-restruct/restruct v1.2.0-alpha // indirect
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
github.com/google/go-intervals v0.0.2 // indirect
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.20 // indirect
github.com/googleapis/gax-go/v2 v2.24.0 // indirect
github.com/google/pprof v0.0.0-20261016150608-1facf8f91b50 // indirect
github.com/google/s2a-go v0.1.10 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.22 // indirect
github.com/googleapis/gax-go/v2 v2.24.1 // indirect
github.com/hashicorp/aws-sdk-go-base/v2 v2.0.0-beta.74 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-version v1.9.0 // indirect
@@ -224,8 +222,7 @@ require (
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/klauspost/pgzip v1.2.6 // indirect
github.com/klauspost/pgzip v1.2.7 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/lucasb-eyer/go-colorful v1.3.0 // indirect
@@ -233,8 +230,8 @@ require (
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/mattn/go-localereader v0.0.2-0.20220822084749-2491eb6c1c75 // indirect
github.com/mattn/go-runewidth v0.0.27 // indirect
github.com/mattn/go-sqlite3 v1.14.48 // indirect
github.com/mattn/go-runewidth v0.0.31 // indirect
github.com/mattn/go-sqlite3 v1.14.52 // indirect
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/miekg/pkcs11 v1.1.2 // indirect
github.com/mikelolasagasti/xz v1.0.1 // indirect
@@ -246,8 +243,8 @@ require (
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/locker v1.0.1 // indirect
github.com/moby/moby/api v1.56.0 // indirect
github.com/moby/moby/client v0.6.0 // indirect
github.com/moby/moby/api v1.56.1 // indirect
github.com/moby/moby/client v0.6.2 // indirect
github.com/moby/sys/capability v0.4.0 // indirect
github.com/moby/sys/signal v0.7.1 // indirect
github.com/moby/sys/user v0.4.1 // indirect
@@ -257,7 +254,7 @@ require (
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.16.0 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/ncruces/go-strftime v1.1.0 // indirect
github.com/nwaples/rardecode/v2 v2.4.1 // indirect
github.com/olekukonko/cat v0.0.0-20250911104152-50322a0618f6 // indirect
github.com/olekukonko/errors v1.2.0 // indirect
@@ -265,11 +262,12 @@ require (
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/opencontainers/runtime-spec v1.3.0 // indirect
github.com/opencontainers/selinux v1.15.1 // indirect
github.com/pb33f/go-yaml v0.1.0 // indirect
github.com/pborman/indent v1.2.1 // indirect
github.com/pelletier/go-toml/v2 v2.4.3 // indirect
github.com/pierrec/lz4/v4 v4.1.26 // indirect
github.com/piprate/json-gold v0.7.0 // indirect
github.com/pjbgf/sha1cd v0.6.0 // indirect
github.com/pjbgf/sha1cd v0.7.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pkg/profile v1.7.0 // indirect
github.com/pkg/xattr v0.4.12 // indirect
@@ -281,11 +279,11 @@ require (
github.com/rogpeppe/go-internal v1.14.1 // indirect
github.com/sagikazarmark/locafero v0.11.0 // indirect
github.com/sahilm/fuzzy v0.1.1 // indirect
github.com/secure-systems-lab/go-securesystemslib v0.11.0 // indirect
github.com/secure-systems-lab/go-securesystemslib v0.11.1 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/sigstore/fulcio v1.8.7 // indirect
github.com/sigstore/protobuf-specs v0.5.1 // indirect
github.com/sigstore/sigstore v1.10.8 // indirect
github.com/sigstore/fulcio v1.8.8 // indirect
github.com/sigstore/protobuf-specs v0.5.2 // indirect
github.com/sigstore/sigstore v1.10.11 // indirect
github.com/sirupsen/logrus v1.10.2 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/smallnest/ringbuffer v0.0.0-20241116012123-461381446e3d // indirect
@@ -295,11 +293,11 @@ require (
github.com/spf13/cast v1.10.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/spf13/viper v1.21.0 // indirect
github.com/spiffe/go-spiffe/v2 v2.7.0 // indirect
github.com/spiffe/go-spiffe/v2 v2.8.1 // indirect
github.com/stefanberger/go-pkcs11uri v0.0.0-20230803200340-78284954bff6 // indirect
github.com/stretchr/objx v0.5.3 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/sylabs/sif/v2 v2.24.1 // indirect
github.com/sylabs/sif/v2 v2.24.2 // indirect
github.com/sylabs/squashfs v1.0.6 // indirect
github.com/tchap/go-patricia/v2 v2.3.3 // indirect
github.com/therootcompany/xz v1.0.1 // indirect
@@ -307,9 +305,10 @@ require (
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/varlink/go v0.4.1-0.20261015173642-c935be0c53f7 // indirect
github.com/vbatts/tar-split v0.12.3 // indirect
github.com/vbauerster/cupwriter v0.0.4 // indirect
github.com/vbauerster/mpb/v8 v8.14.0 // indirect
github.com/vbauerster/cupwriter v0.0.5 // indirect
github.com/vbauerster/mpb/v8 v8.16.2 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
@@ -326,26 +325,24 @@ require (
go.opentelemetry.io/otel/sdk v1.46.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.46.0 // indirect
go.opentelemetry.io/otel/trace v1.46.0 // indirect
go.podman.io/image/v5 v5.41.2 // indirect
go.podman.io/storage v1.64.1 // indirect
go.yaml.in/yaml/v4 v4.0.0-rc.2 // indirect
go.podman.io/image/v5 v5.42.0 // indirect
go.podman.io/storage v1.65.0 // indirect
go4.org v0.0.0-20230225012048-214862532bf5 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/oauth2 v0.36.0 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/term v0.46.0 // indirect
golang.org/x/text v0.42.0 // indirect
golang.org/x/crypto v0.58.0 // indirect
golang.org/x/oauth2 v0.37.0 // indirect
golang.org/x/sync v0.24.0 // indirect
golang.org/x/term v0.47.0 // indirect
golang.org/x/text v0.43.0 // indirect
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
gonum.org/v1/gonum v0.17.0 // indirect
google.golang.org/api v0.294.0 // indirect
google.golang.org/api v0.299.0 // indirect
google.golang.org/genproto v0.0.0-20260715232425-e75dac1f907d // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260825221802-da73d73af1c5 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260825221802-da73d73af1c5 // indirect
google.golang.org/grpc v1.83.2 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260921155816-b14227669459 // indirect
google.golang.org/grpc v1.84.0 // indirect
google.golang.org/protobuf v1.36.12 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
modernc.org/libc v1.75.7 // indirect
modernc.org/libc v1.79.0 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.12.1 // indirect
)
+172 -174
View File
@@ -31,8 +31,8 @@ cloud.google.com/go v0.98.0/go.mod h1:ua6Ush4NALrHk5QXDWnjvZHN93OuF0HfuEPq9I1X0c
cloud.google.com/go v0.99.0/go.mod h1:w0Xx2nLzqWJPuozYQX+hFfCSI8WioryfRDzkoI/Y2ZA=
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
cloud.google.com/go/auth v0.23.2 h1:pxSCpfiji41hpzpPdMCftEUCezpgpqmmDdYiAjCKXxo=
cloud.google.com/go/auth v0.23.2/go.mod h1:4DhBRcqvtljQN3dJ57qtqbib5ZGCYE5f2crfiiC2EM0=
cloud.google.com/go/auth v0.23.3 h1:UMK+oBtuNGMCR/6i6mmySUItqjOazpJrbmZyhGbGBWo=
cloud.google.com/go/auth v0.23.3/go.mod h1:fClbry28fo7XkxhSeT6AQtAVAp6Jy0fW9N99PoPNPFM=
cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc=
cloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
@@ -41,8 +41,8 @@ cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvf
cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
cloud.google.com/go/compute/metadata v0.9.1 h1:CTE1OWBQ0vnF5uHwdFAQJvMQ0Fi/KRcqqKTo9V0F8Ik=
cloud.google.com/go/compute/metadata v0.9.1/go.mod h1:NtnlvB6X3t4R6xSWyVX/ZWk493PCxGQlhI/iqxh4M8I=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/firestore v1.6.1/go.mod h1:asNXNOzBdyVQmEU+ggO8UPodTkEVFW5Qx+rwHnAz+EY=
@@ -63,8 +63,8 @@ cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0Zeo
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
cloud.google.com/go/storage v1.65.1 h1:LRRpBJUTf+OXDPX9jZUKZ3mSLIsz3htG+qUpeNZovyA=
cloud.google.com/go/storage v1.65.1/go.mod h1:UsS9OgFg/XHOSYakQ8ZtLWWeyGkk1WnmD/GsGfN0BHM=
cloud.google.com/go/storage v1.68.0 h1:gqrAMJ51OZjYgU6AJ2U60um90YQhSjq8HEIQNtJ4C/8=
cloud.google.com/go/storage v1.68.0/go.mod h1:UsS9OgFg/XHOSYakQ8ZtLWWeyGkk1WnmD/GsGfN0BHM=
cloud.google.com/go/trace v1.16.0 h1:GmQovzFc5F0CNfl0VLgL64aoTtu7xsM0YajW2GlG9+E=
cloud.google.com/go/trace v1.16.0/go.mod h1:r+bdAn16dKLSV1G2D5v3e58IlQlizfxWrUfjx7kM7X0=
cyphar.com/go-pathrs v0.2.5 h1:SnX9FBvnoyn3lUs1dkMgZ52bAETpirNu3FTRh5HlRik=
@@ -84,8 +84,8 @@ github.com/CycloneDX/cyclonedx-go v0.12.0/go.mod h1:V2577HhxDDCDLYfkm55WJrz16nHT
github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
github.com/DataDog/zstd v1.5.5 h1:oWf5W7GtOLgp6bciQYDmhHHjdhYkALu6S/5Ni9ZgSvQ=
github.com/DataDog/zstd v1.5.5/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.33.0 h1:l7+6kwRMJNwdCvYdDl7Eax+wzEYHSnNY7zrrfbhDdTA=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.33.0/go.mod h1:pJTkW8hEUIIi3Pf65lPZOnn4Y81yCllX6IWk2jNXdkM=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.34.0 h1:yzIYdwuro811Z27D3T80Wkd3rqZzb0K43nner7Eh1yE=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.34.0/go.mod h1:pJTkW8hEUIIi3Pf65lPZOnn4Y81yCllX6IWk2jNXdkM=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.57.0 h1:jLdiS1vO+XJFyDSWRHBx56r4s/NNtcl5J6KyCcWUX/w=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.57.0/go.mod h1:8lmpHY+1VRoteiOwyrQMDt1YGXOrFKCz+1wJW7n3ODY=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/cloudmock v0.57.0 h1:cSjUzZ7KU8hicTgzaSv9NmSyM9fTVK3y5lsBUl3wOis=
@@ -99,15 +99,15 @@ github.com/Masterminds/semver/v3 v3.5.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lpr
github.com/Masterminds/sprig/v3 v3.3.0 h1:mQh0Yrg1XPo6vjYXgtf5OtijNAKJRNcTdOOGZe3tPhs=
github.com/Masterminds/sprig/v3 v3.3.0/go.mod h1:Zy1iXRYNqNLUolqCpL4uhk6SHUMAOSCzdgBfDb35Lz0=
github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29 h1:0kQAzHq8vLs7Pptv+7TxjdETLf/nIqJpIB4oC6Ba4vY=
github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29/go.mod h1:ZWa7ssZJT30CCDGJ7fk/2SBTq9BIQrrVjrcss0UW2s0=
github.com/Microsoft/go-winio v0.6.3-0.20260930231756-f19d9717deb0 h1:71VU8kW/LCxJkx41+YYys1EgQO+AUnPa1TrxTSyYUCU=
github.com/Microsoft/go-winio v0.6.3-0.20260930231756-f19d9717deb0/go.mod h1:ma0QpxizD4fyyzJ1C7MIblymTD2Nxz5Mho/3CepwIZg=
github.com/Microsoft/hcsshim v0.15.0-rc.4 h1:aZFX4LH0S20Lgjq0wG61StIClj7im4yzrxIClkaR8Z8=
github.com/Microsoft/hcsshim v0.15.0-rc.4/go.mod h1:BA9CBztgu4h/6Jsvo1O1M4qjWw09PoYpaEYgexPE578=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/OneOfOne/xxhash v1.2.8 h1:31czK/TI9sNkxIKfaUfGlU47BAxQ0ztGgd9vPyqimf8=
github.com/OneOfOne/xxhash v1.2.8/go.mod h1:eZbhyaAYD41SGSSsnmcpxVoRiQ/MPUTjUdIIOT9Um7Q=
github.com/ProtonMail/go-crypto v1.4.1 h1:9RfcZHqEQUvP8RzecWEUafnZVtEvrBVL9BiF67IQOfM=
github.com/ProtonMail/go-crypto v1.4.1/go.mod h1:e1OaTyu5SYVrO9gKOEhTc+5UcXtTUa+P3uLudwcgPqo=
github.com/ProtonMail/go-crypto v1.5.2 h1:cucYnvqcY7UOXVD//mSyjeaPY0SSN3v5cDkYPxumINk=
github.com/ProtonMail/go-crypto v1.5.2/go.mod h1:/RaSu30DaKO4RY+XdV/ACcCcZkGr7AhUIduq5sjzzCo=
github.com/STARRY-S/zip v0.2.3 h1:luE4dMvRPDOWQdeDdUxUoZkzUIpTccdKdhHHsQJ1fm4=
github.com/STARRY-S/zip v0.2.3/go.mod h1:lqJ9JdeRipyOQJrYSOtpNAiaesFO6zVDsE8GIGFaoSk=
github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1ow=
@@ -126,8 +126,8 @@ github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRF
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/anchore/bubbly v0.2.2 h1:53JHRlpaNZjMcUYdz7n1U7+kOhy0zsJv9gpGEXvTcz8=
github.com/anchore/bubbly v0.2.2/go.mod h1:M4twoFkczlfuNdU8QV7oQjVJwPbxdUWRMfKydcLba0U=
github.com/anchore/clio v0.1.1 h1:9bkYnaFbpT0J3G0qSHyQqa/lpK5zV0C9Gtgh82H/jj0=
github.com/anchore/clio v0.1.1/go.mod h1:CFPIopJlakJbBpKtAL/EPpj4W9hgBO7PfFc8N2FAtlE=
github.com/anchore/clio v0.1.2 h1:kCWNUkJsRJ5A103fFSLG+CSqIP9bOIFX2X5GEnup0HM=
github.com/anchore/clio v0.1.2/go.mod h1:LqoM9TUMDolUMaPV4hslLvuooqIjpTN8J7V/rBqk/Zc=
github.com/anchore/fangs v0.1.1 h1:NBTHocmWM2/0gNhebpoE/wEB0Z0nxh2lESww2o/Uf5M=
github.com/anchore/fangs v0.1.1/go.mod h1:82oZuk4+RynOvAoxnXKmI1zO5xk/kbOw+5t4fAAPSsk=
github.com/anchore/go-collections v0.1.1 h1:kGlmSehs+ctTBZr+98F2+taHeolHy+XDx1Nczl8GZc8=
@@ -140,8 +140,8 @@ github.com/anchore/go-lzo v0.1.1 h1:IwL/fvkdtlIrYIXck6WxZ3nb8WjjHziYYmGxlooyOnM=
github.com/anchore/go-lzo v0.1.1/go.mod h1:3kLx0bve2oN1iDwgM1U5zGku1Tfbdb0No5qp1eL1fIk=
github.com/anchore/go-macholibre v0.1.1 h1:sTBHi3S1WVIOCamc6NyT8Sn9mBmD30DCTfjfVm4k6/M=
github.com/anchore/go-macholibre v0.1.1/go.mod h1:YNq2610RlvGCK0Za+Klz/OCMtUaEnwwAGA/VfUAQaro=
github.com/anchore/go-rpmdb v0.2.0 h1:ixcyMg/guhBL9XjBVwhTv9CG48xO07Qkg2uDX2+zx/c=
github.com/anchore/go-rpmdb v0.2.0/go.mod h1:ATsRlpCXstnoYfzqBfhwGw0U2dolx1BBQ9rAU8jWBAw=
github.com/anchore/go-rpmdb v0.2.2 h1:44VC7mvFAxV9t5JUZYg3S9sqG+E94iZvyWACMDq9hSk=
github.com/anchore/go-rpmdb v0.2.2/go.mod h1:Y64EbetdqhryMjtk8IzXkdCQw/aNDx+feZCT9aj6rjs=
github.com/anchore/go-struct-converter v0.2.0-rc2 h1:q+859fW2/jbHJHB2etbNfRlFwYpknyvbqqk1hUdamQ4=
github.com/anchore/go-struct-converter v0.2.0-rc2/go.mod h1:cDBA5vhcR62nXWo8QH9/Kk2807o65ISaHPNPX66L+Uw=
github.com/anchore/go-sync v0.1.2 h1:RDD5RCanrWqhbtBulqPSZpERadUlgcYjpybYzYd3Uk8=
@@ -150,8 +150,8 @@ github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZV
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/packageurl-go v0.2.0 h1:CkrM4RMUwrEGAiE1OVlxaZNzWj0TuHRey7o4T/EAErk=
github.com/anchore/packageurl-go v0.2.0/go.mod h1:2JCgOQMIsqZ7TmliXG4PnUthPJAKE3mWQbsW2XHjAOE=
github.com/anchore/stereoscope v0.3.3 h1:sur8G2THAOnOrSXDjIc1SXzQwco8hjFBtGFq1jFvNVA=
github.com/anchore/stereoscope v0.3.3/go.mod h1:LxL4iOFtB8XMbA1MykUUTkmtQmDuYdbr4+TkBkCfljU=
github.com/anchore/stereoscope v0.4.1 h1:7mxw24JCabbJIAcyvQDjBd2pbzo+0HSFVgjYlE8Zm+k=
github.com/anchore/stereoscope v0.4.1/go.mod h1:hYJsi3M8a8N1O66u+rMQSgv5TREGM7QIjKBjPPnwiD8=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ=
github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
@@ -175,40 +175,40 @@ github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPd
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4=
github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI=
github.com/aws/aws-sdk-go-v2 v1.44.0 h1:4IbaHhtzy+4h37z4JQyO9a2QsiCml3CNYHtq5hIHigo=
github.com/aws/aws-sdk-go-v2 v1.44.0/go.mod h1:bttEH6JqnUL8LepvDVfdrds/fZ5bCIxzpe3abyUrhDU=
github.com/aws/aws-sdk-go-v2 v1.47.1 h1:uOIZnp4PK3ZhKI0dNrJrhTEsLxbpXHTAJlwoS1pvAtw=
github.com/aws/aws-sdk-go-v2 v1.47.1/go.mod h1:bttEH6JqnUL8LepvDVfdrds/fZ5bCIxzpe3abyUrhDU=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.20 h1:GPRlPwz40I2B2VrBEASOA3Bi77NyeqejNLkifosX0rs=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.20/go.mod h1:g7PNzKcsOKWb4fkSRBA7BZVAS6Y8IcxzN+nRohhQ1Q8=
github.com/aws/aws-sdk-go-v2/config v1.32.40 h1:lAVC9gMmKusmqDRe32dPtgKl/BWvJmMJoWELKHCAObw=
github.com/aws/aws-sdk-go-v2/config v1.32.40/go.mod h1:8xOJLbe/hOj1g4PVsfJYV7O2byq+UGET1onDdUgbwqc=
github.com/aws/aws-sdk-go-v2/credentials v1.19.39 h1:XOg8LC3Kgnsa3WiPQjc7Bi8k5IBN92cPYfIV9XMFss0=
github.com/aws/aws-sdk-go-v2/credentials v1.19.39/go.mod h1:GonTDBQ+mTpCVNwaHjj0PagspfrYYMEqOx7FehoEP/I=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.40 h1:r5aGipEVgI9aT/tAGjdrPbDQvIAKdTrS3rUPQtG4Rmo=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.40/go.mod h1:vOD3CnPxAdkL6MWZeROkZsTlskklMFfgVFkHzx/oZpY=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.40 h1:UIXlbijuB2XK1Kr57fo8iIxCuaSHJzwZ1uo+2tbEYIk=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.40/go.mod h1:wcEsL6jscjZjVUinb0Q5qD/GXOG1yT3GNfmT9HuDwzU=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.40 h1:xLQVRDs2NddDmK9BEyh5KSlJ1Gpy5/GIJXrV6WcVGAE=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.40/go.mod h1:XRXnpFVFGLaEVK+olDdFIM1vNa04ETW452oFGEPUxAo=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.41 h1:nv/ILuCY0yXACzMQwvtt/HbqDDjemZiI0AeDbxGQlnU=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.41/go.mod h1:dzvOSpxaPqQ3j0xS6Lc1vyVuWW0RBj7s/QqYpzu3Q/0=
github.com/aws/aws-sdk-go-v2/config v1.33.6 h1:MBjkSTLczek/UgiK+EYPIoRTqE7gP8vtW3OFbFo7Nug=
github.com/aws/aws-sdk-go-v2/config v1.33.6/go.mod h1:grRAFzdAZJrwcbasJRg2MPvIrVjtlfXllHssN6+E1JE=
github.com/aws/aws-sdk-go-v2/credentials v1.20.6 h1:NpAFXCU7NzXNkdGK3zQTtsRJ+3v9tZQV0xcdRw8uBdw=
github.com/aws/aws-sdk-go-v2/credentials v1.20.6/go.mod h1:mcZCoiPnyMvP8VMNbygNX5lLqSlkYJIMPODylQMurOk=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.20.1 h1:8gALAAmacnIXh+z6VkdDanv4/IkG5APdg4DZLDTmLog=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.20.1/go.mod h1:Z7IJhJU+poOdJjUR2wpyY21ossQ1XS/R3Lk9Msq5kM4=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.5.4 h1:CLq4+8UHCI+ZZYl/EuJxXovaIVN2xeeT8JV+dsApQ5E=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.5.4/go.mod h1:Wv4q5sAM04xAMkoOedxLx2inVf6K5FdxYp+A61L+q/0=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.8.4 h1:dD4MR81I7YkpEBRk6UP9rocC2QnT3qVuXwzlYTtfGEs=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.8.4/go.mod h1:EcXV1kAFd5XwSkDHlj94gnF3q5CkJyYiIJfH8N0VmrE=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.5.4 h1:7Wo47d/xn/7KttCSBd8EGYeZ7ULRFRkUHr6vkZPBzVQ=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.5.4/go.mod h1:tDB2IVC1xC3vX8o+6uRlzhTxP3g1b77CZXFX/oD2FnQ=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.19 h1:bAdDl/HkGCcGPoe25ToSHEw23VIxt6CT5fLcg111BKg=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.19/go.mod h1:KaUzbLxv4CeSxh6ZCl9B4m7CuFenS8kUEaDs+f/DQr4=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.10.0 h1:U8/A0RRBaEspzH1uul3JHLbypXwEGUkRkvoT9f0ATcM=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.10.0/go.mod h1:UELStX5KwtJNtQxa+UuF8dc3z4UYc40e8yHYJSozNwY=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.40 h1:gr3Fw1cxZXNCdeo/lQ7isHEHzvHVM7z75qb2zW9aMjw=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.40/go.mod h1:8z/9CmfnQhiuXD7Ykbcg4a/whSWsniE0ODSx9uwVzfk=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.41 h1:Q9DIKDuJix/oJnQxFpQ26L0EwVa/YNo4k2kbktrjQjE=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.41/go.mod h1:x+TuqkOIG1SZS0+yN54sExGA9ZpjhPO6vPdYnpTFX1M=
github.com/aws/aws-sdk-go-v2/service/s3 v1.108.0 h1:Yp+x5PKXEmoqHsgP/pAkBy5Tyq1UlXAzM0OInh0vxWw=
github.com/aws/aws-sdk-go-v2/service/s3 v1.108.0/go.mod h1:locV6DtXyp7Xzr2BG6jtsbeBi3YAWJ/CY4xUThYmIwQ=
github.com/aws/aws-sdk-go-v2/service/signin v1.6.0 h1:agcr0j8YeFEzdXNo17Rg9MbbjLRjrimabwNtji4e+lU=
github.com/aws/aws-sdk-go-v2/service/signin v1.6.0/go.mod h1:qU5PxgQ4JiUOOMotzfO3+5oUda5W+8JDVKyLQqlrJik=
github.com/aws/aws-sdk-go-v2/service/sso v1.34.0 h1:FxaN8/sn61DTXNI6Gt678tFJUY8iUsCchm6Y/F/RjaA=
github.com/aws/aws-sdk-go-v2/service/sso v1.34.0/go.mod h1:vu4OY6s8LJtT8BtYG2LD6BGSZMptkYn3o5hvCPB22jc=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.39.0 h1:crWKPeGYTBTuBxQ3p73kjfJvt4brUIsr+Fuypko8FxY=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.39.0/go.mod h1:HjjZVhaBz0JBR/kbWKThmNDhFKS7y6EURuk493tJk9Y=
github.com/aws/aws-sdk-go-v2/service/sts v1.46.0 h1:IZ63JdogSNNjex/jsODNv7jGDcO/xJYd9FsgyfCsp1g=
github.com/aws/aws-sdk-go-v2/service/sts v1.46.0/go.mod h1:I+rwAf3spG5dITBaAo3xXRowk8kiOhtU1kYxfvCTC44=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.11.5 h1:/TYsZXdA8UTa+WCtCYSAJIr1vwl0+eho6TUgJGwFFO8=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.11.5/go.mod h1:qPqp1Uwd/BqdhPufv6oem9j5J7HNsgc2V22dUiDPn+s=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.14.4 h1:29SvnfGhXjTl8ONxFwbj2rs6lbhiFXD2CgFQmbT/bXY=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.14.4/go.mod h1:wm04I5DMuNVvZHFe/dHnUxincvNbbK7AiNBbYsQivek=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.20.4 h1:pPiWfgeNxqluKEph7hvU88kuGKBPOWzO+Dk9t2zqqNs=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.20.4/go.mod h1:YlwGoIUDG/3kBQbdNOVs/xKZ9J01G8e/6D1mRBj9uTk=
github.com/aws/aws-sdk-go-v2/service/s3 v1.114.0 h1:VMAdYqr4Jn/8ATs9BHC5riwrs0d6m1Z2ohFriSwZwm0=
github.com/aws/aws-sdk-go-v2/service/s3 v1.114.0/go.mod h1:9APRWGLFITKD+xzWSIyT9V7QV4bNlEuIieWlzXgGFlI=
github.com/aws/aws-sdk-go-v2/service/signin v1.10.1 h1:DzCCWLzcIRQ77F3DEUljud7bEjTgFOIKXP52NmVRyhU=
github.com/aws/aws-sdk-go-v2/service/signin v1.10.1/go.mod h1:xpo/geVldu8payT375WekctUzopG/hBU7miiqItMUlw=
github.com/aws/aws-sdk-go-v2/service/sso v1.38.1 h1:Umtl/0YZhng4xndfW3lKJrYYP7NLEjI6bGXVomwLcs0=
github.com/aws/aws-sdk-go-v2/service/sso v1.38.1/go.mod h1:rRD/dnm7q0HYE/I5TMaPgkWyyUGLcwuxHLABsLnQ3e0=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.43.1 h1:orIWdNiLgzrhu/11RcPPKO/SBzUUymbUQuZbSPImghg=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.43.1/go.mod h1:skwM/xsbR/1ReUTesv9BhpJp1VjajR7DWQnuVLwiXsQ=
github.com/aws/aws-sdk-go-v2/service/sts v1.51.1 h1:0HOqZXRvMytH6bFHVIc0oJX07sZjfhz0zXtjs6gdE8s=
github.com/aws/aws-sdk-go-v2/service/sts v1.51.1/go.mod h1:26zA0GhDrLo+yiLI2yXWxqB1PdsShfLikoI7GOEgugM=
github.com/aws/smithy-go v1.28.1 h1:R/nXH00c8qcfCzQVELtRw+eLQWtzv+VAIEFJ1/xxXlQ=
github.com/aws/smithy-go v1.28.1/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
@@ -217,8 +217,6 @@ github.com/aymanbagabas/go-udiff v0.3.1 h1:LV+qyBQ2pqe0u42ZsUEtPiCaUoqgA9gYRDs3v
github.com/aymanbagabas/go-udiff v0.3.1/go.mod h1:G0fsKmG+P6ylD0r6N/KgQD/nWzgfnl8ZBcNLgcbrw8E=
github.com/bahlo/generic-list-go v0.2.0 h1:5sz/EEAK+ls5wF+NeqDpk5+iNdMDXrh3z3nPnH1Wvgk=
github.com/bahlo/generic-list-go v0.2.0/go.mod h1:2KvAjgMlE5NNynlg/5iLrrCCZ2+5xWbdbCW3pNTGyYg=
github.com/becheran/wildmatch-go v1.0.0 h1:mE3dGGkTmpKtT4Z+88t8RStG40yN9T+kFEGj2PZFSzA=
github.com/becheran/wildmatch-go v1.0.0/go.mod h1:gbMvj0NtVdJ15Mg/mH9uxk2R1QCistMyU7d9KFzroX4=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
@@ -229,8 +227,8 @@ github.com/bitnami/go-version v0.0.0-20250131085805-b1f57a8634ef h1:TSFnfbbu2oAO
github.com/bitnami/go-version v0.0.0-20250131085805-b1f57a8634ef/go.mod h1:9iglf1GG4oNRJ39bZ5AZrjgAFD2RwQbXw6Qf7Cs47wo=
github.com/blakesmith/ar v0.0.0-20190502131153-809d4375e1fb h1:m935MPodAbYS46DG4pJSv7WO+VECIWUQ7OJYSoTrMh4=
github.com/blakesmith/ar v0.0.0-20190502131153-809d4375e1fb/go.mod h1:PkYb9DJNAwrSvRx5DYA+gUcOIgTGVMNkfSCbZM8cWpI=
github.com/bmatcuk/doublestar/v4 v4.10.0 h1:zU9WiOla1YA122oLM6i4EXvGW62DvKZVxIe6TYWexEs=
github.com/bmatcuk/doublestar/v4 v4.10.0/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/bmatcuk/doublestar/v4 v4.10.3 h1:1UIv1+fqH3++ViVkpnaPTB+Pi1UHycnPFAq14zgK+gI=
github.com/bmatcuk/doublestar/v4 v4.10.3/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/bodgit/plumbing v1.3.0 h1:pf9Itz1JOQgn7vEOE7v7nlEfBykYqvUYioC61TwWCFU=
github.com/bodgit/plumbing v1.3.0/go.mod h1:JOTb4XiRu5xfnmdnDJo6GmSbSbtSyufrsyZFByMtKEs=
github.com/bodgit/sevenzip v1.6.1 h1:kikg2pUMYC9ljU7W9SaqHXhym5HyKm8/M/jd31fYan4=
@@ -300,8 +298,8 @@ github.com/containerd/cgroups/v3 v3.1.3 h1:eUNflyMddm18+yrDmZPn3jI7C5hJ9ahABE5q6
github.com/containerd/cgroups/v3 v3.1.3/go.mod h1:PKZ2AcWmSBsY/tJUVhtS/rluX0b1uq1GmPO1ElCmbOw=
github.com/containerd/containerd/api v1.12.0 h1:kuQm82SbDrCuO4n7hf2L8zsBtZLuympyq5X/VotfX2A=
github.com/containerd/containerd/api v1.12.0/go.mod h1:EBcSzoi9Vl18cdODaXUCskf3D2NT8lsSXeZJnU5jIUc=
github.com/containerd/containerd/v2 v2.4.1 h1:DUx/ZJN7cEu0WuzHClDB+68H/bqMEH5pWoEjf0ae4hc=
github.com/containerd/containerd/v2 v2.4.1/go.mod h1:vgLdtvl3prFk1d3ZVqQcsDD5Q9IKM+vAIdU54U85w+I=
github.com/containerd/containerd/v2 v2.4.2 h1:p6Qg+xbEz8lvwdhHDx0U469xENvp4LgGBx+E7djXkGw=
github.com/containerd/containerd/v2 v2.4.2/go.mod h1:vgLdtvl3prFk1d3ZVqQcsDD5Q9IKM+vAIdU54U85w+I=
github.com/containerd/continuity v0.5.0 h1:7a85HZpCSs+1Zps0Ee3DPSuAWY+0SJM1JNM51nlEVDg=
github.com/containerd/continuity v0.5.0/go.mod h1:/lNJvtJKUQStBzpVQ1+rasXO1LAWtUQssk28EZvJ3nE=
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
@@ -318,8 +316,8 @@ github.com/containerd/platforms v1.0.0-rc.5 h1:vXd569rDrz8LeMXzAnBsy6LADV5YtsD8o
github.com/containerd/platforms v1.0.0-rc.5/go.mod h1:lKlMXyLybmBedS/JJm11uDofzI8L2v0J2ZbYvNsbq1A=
github.com/containerd/plugin v1.1.0 h1:O+7lczNJVMy8rz0YNx3xGB8tTf5qY4i5abF041Ew19U=
github.com/containerd/plugin v1.1.0/go.mod h1:qBTum+A8lJ6lO44A19Eo7y1OlcLj4OWFH1DA/vnHmcc=
github.com/containerd/stargz-snapshotter/estargz v0.18.2 h1:yXkZFYIzz3eoLwlTUZKz2iQ4MrckBxJjkmD16ynUTrw=
github.com/containerd/stargz-snapshotter/estargz v0.18.2/go.mod h1:XyVU5tcJ3PRpkA9XS2T5us6Eg35yM0214Y+wvrZTBrY=
github.com/containerd/stargz-snapshotter/estargz v0.19.0 h1:tak4qAx648jBAgZmywkd+6kaJ5pqSOVzKYB5DcVSHxk=
github.com/containerd/stargz-snapshotter/estargz v0.19.0/go.mod h1:Cr0pxOkuN7Y61+Sik6X6/nPtePW7hsKWdBt+MoHQRr0=
github.com/containerd/ttrpc v1.2.9 h1:ha0ak962T0s3CA/RoZ6S6xiWZQF24GrBaEpiGX1uihg=
github.com/containerd/ttrpc v1.2.9/go.mod h1:jjtQRwXm4DL3KsHKW8vDiUOV6wO0hi6IPhmJhxU7aEs=
github.com/containerd/typeurl/v2 v2.3.0 h1:HZHPhRWo5XMy3QGQoPrUzbW/2ckwjfweHmOwlkIrPAQ=
@@ -344,8 +342,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/deitch/magic v0.0.0-20230404182410-1ff89d7342da h1:ZOjWpVsFZ06eIhnh4mkaceTiVoktdU67+M7KDHJ268M=
github.com/deitch/magic v0.0.0-20230404182410-1ff89d7342da/go.mod h1:B3tI9iGHi4imdLi4Asdha1Sc6feLMTfPLXh9IUYmysk=
github.com/deitch/magic v1.0.0 h1:Nma1YAj7EB8Mlv4pVhbPpUp0CV1luYj2NOnE2+2xPNE=
github.com/deitch/magic v1.0.0/go.mod h1:B3tI9iGHi4imdLi4Asdha1Sc6feLMTfPLXh9IUYmysk=
github.com/dgrijalva/jwt-go/v4 v4.0.0-preview1/go.mod h1:+hnT3ywWDTAFrW5aE+u2Sa/wT555ZqwoCS+pk3p6ry4=
github.com/diskfs/go-diskfs v1.9.4 h1:0j2d7eG4IjyxL6+ChWbDPocdBCF6HQ4HBWU2WDYWVnc=
github.com/diskfs/go-diskfs v1.9.4/go.mod h1:TePJORO83Adh5pb2SqsxAwaP0fofFxKLkxctiS/9OQc=
@@ -353,14 +351,14 @@ github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5Qvfr
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/djherbis/times v1.6.0 h1:w2ctJ92J8fBvWPxugmXIv7Nz7Q3iDMKNx9v5ocVH20c=
github.com/djherbis/times v1.6.0/go.mod h1:gOHeRAz2h+VJNZ5Gmc/o7iD9k4wW7NMVqieYCY99oc0=
github.com/docker/cli v29.8.1+incompatible h1:qYL1bCp6cRw2SB1xmLlIOPyV171dilw9W2Jew38vy9c=
github.com/docker/cli v29.8.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/cli v29.9.2+incompatible h1:trup/vUxx5uUo2t+87Cd9AmITR+DvKUe4H2tTtZN8FY=
github.com/docker/cli v29.9.2+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk=
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker-credential-helpers v0.9.8 h1:bIREROb7So6PRlq6KTtdS9MPEjC29OQRkFNlvK2OX8Q=
github.com/docker/docker-credential-helpers v0.9.8/go.mod h1:v1S+hepowrQXITkEfw6o4+BMbGot02wiKpzWhGUZK6c=
github.com/docker/go-connections v0.8.1 h1:JibmG5hULs5qXSr/cp/w3Pw5fZuStt4MOHMUExb29/M=
github.com/docker/go-connections v0.8.1/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/docker-credential-helpers v0.9.10 h1:LCWwv7UmS7fQVmdKDWeMuureOeY5kkbC5kZfkzxkS6k=
github.com/docker/docker-credential-helpers v0.9.10/go.mod h1:v1S+hepowrQXITkEfw6o4+BMbGot02wiKpzWhGUZK6c=
github.com/docker/go-connections v0.8.2 h1:9d9DEokx5W7qlr0shkFXIxEA7+TTD+llwzcykSlxMNc=
github.com/docker/go-connections v0.8.2/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 h1:2tV76y6Q9BB+NEBasnqvs7e49aEBFI8ejC89PSnWH+4=
@@ -419,32 +417,32 @@ github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7z
github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU=
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
github.com/gabriel-vasile/mimetype v1.4.15 h1:05iP/CYtZ/w455R/KZM6rZ5ieAdh99UPtd+d3YzLmaI=
github.com/gabriel-vasile/mimetype v1.4.15/go.mod h1:azpTcoLcDZRNgFou5j+APrqQx9HqVPWa6ijYQIIVswQ=
github.com/gabriel-vasile/mimetype v1.4.16 h1:5izNFuvA37st1dDRAcA7TtmngBmRgiIPDmTtiEvjx1E=
github.com/gabriel-vasile/mimetype v1.4.16/go.mod h1:3oFFwxX8kTklZwzIl2aQEDZT0lKY8xH92ST88bAwba4=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/github/go-spdx/v2 v2.7.0 h1:GzfXx4wFdlilARxmFRXW/mgUy3A4vSqZocCMFV6XFdQ=
github.com/github/go-spdx/v2 v2.7.0/go.mod h1:Ftc45YYG1WzpzwEPKRVm9Jv8vDqOrN4gWoCkK+bHer0=
github.com/github/go-spdx/v2 v2.8.0 h1:WKCnF2Ee/x+zwSIEeQdm6+be0H6PiCNLQmSZ5CFmxMI=
github.com/github/go-spdx/v2 v2.8.0/go.mod h1:Ftc45YYG1WzpzwEPKRVm9Jv8vDqOrN4gWoCkK+bHer0=
github.com/gkampitakis/ciinfo v0.3.4 h1:5eBSibVuSMbb/H6Elc0IIEFbkzCJi3lm94n0+U7Z0KY=
github.com/gkampitakis/ciinfo v0.3.4/go.mod h1:1NIwaOcFChN4fa/B0hEBdAb6npDlFL8Bwx4dfRLRqAo=
github.com/gkampitakis/go-snaps v0.5.23 h1:okh5QR48zpUjpWtu65AtqxdCY8huJq+dEDuUzd1PuKg=
github.com/gkampitakis/go-snaps v0.5.23/go.mod h1:uy3lVzCCRRsAwYqSocyw5fY8xRLCYEfqoOJNxr8HonM=
github.com/glebarez/go-sqlite v1.20.3 h1:89BkqGOXR9oRmG58ZrzgoY/Fhy5x0M+/WV48U5zVrZ4=
github.com/glebarez/go-sqlite v1.20.3/go.mod h1:u3N6D/wftiAzIOJtZl6BmedqxmmkDfH3q+ihjqxC9u0=
github.com/gkampitakis/go-snaps v0.5.24 h1:oyQ/CQA4s0nkVm6CLlM2GJSU8e4p6KGPZbrSCnTNBzE=
github.com/gkampitakis/go-snaps v0.5.24/go.mod h1:uy3lVzCCRRsAwYqSocyw5fY8xRLCYEfqoOJNxr8HonM=
github.com/glebarez/go-sqlite v1.23.0 h1:FyhIq4jqmgphQAUlY79zPldYGwISEZikaDfhiGWkkaI=
github.com/glebarez/go-sqlite v1.23.0/go.mod h1:IIYrOH3L0rHY3jb4IXOHoWdklNajSGUN2eJcvK8WrnI=
github.com/gliderlabs/ssh v0.3.8 h1:a4YXD1V7xMF9g5nTkdfnja3Sxy1PVDCj1Zg4Wb8vY6c=
github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU=
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic=
github.com/go-git/go-billy/v5 v5.9.1 h1:8U73XiOTfINdItHVa6z4Gv7ToObcZ6grkqQbLryLCdA=
github.com/go-git/go-billy/v5 v5.9.1/go.mod h1:ExsU+jcGwXTBOnyilvAnEM1wug1IxHr4yP2ZXsNRtV0=
github.com/go-git/go-billy/v5 v5.9.2 h1:OXFSRyz4g20upsGDJgQG9Bak1l/ZEv8GHVYB52O71sE=
github.com/go-git/go-billy/v5 v5.9.2/go.mod h1:ExsU+jcGwXTBOnyilvAnEM1wug1IxHr4yP2ZXsNRtV0=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399 h1:eMje31YglSBqCdIqdhKBW8lokaMrL3uTkpGYlE2OOT4=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399/go.mod h1:1OCfN199q1Jm3HZlxleg+Dw/mwps2Wbk9frAWm+4FII=
github.com/go-git/go-git/v5 v5.19.2 h1:wkfn7vOlUBu8ivAWKBWisTiwJK4jYHzTF8Ndv1LyGqY=
github.com/go-git/go-git/v5 v5.19.2/go.mod h1:QqCBE1EFN5ddFmrliLQ3/ntRCUjZU3EJuwuB/jWEHjk=
github.com/go-git/go-git/v5 v5.19.3 h1:qHttbZ+Am7wEjdTpR1XMQ4rl42FK9SIXzZ4o9x7s6M4=
github.com/go-git/go-git/v5 v5.19.3/go.mod h1:Ye4C8dVigkqrv9UsB4NMdSLV4yAIkLiIhW61gcOyhl4=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-jose/go-jose/v4 v4.1.5 h1:RjgjO2LOtWOJKUC5wpwY9LR3B3vwVAz6JS2YHfYU6eA=
github.com/go-jose/go-jose/v4 v4.1.5/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/go-jose/go-jose/v4 v4.1.6 h1:sNRkftxFJxwsILiQukqp5z83QBSmntVwmgRY7AqBDO4=
github.com/go-jose/go-jose/v4 v4.1.6/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
@@ -458,8 +456,8 @@ github.com/go-restruct/restruct v1.2.0-alpha h1:2Lp474S/9660+SJjpVxoKuWX09JsXHSr
github.com/go-restruct/restruct v1.2.0-alpha/go.mod h1:KqrpKpn4M8OLznErihXTGLlsXFGeLxHUrLRRI/1YjGk=
github.com/go-sql-driver/mysql v1.6.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/go-test/deep v1.1.1 h1:0r/53hagsehfO4bzD2Pgr/+RgHqhmf+k1Bpse2cTu1U=
github.com/go-test/deep v1.1.1/go.mod h1:5C2ZWiW0ErCdrYzpqxLbTX7MG14M9iiw8DgHncVwcsE=
github.com/go-test/deep v1.1.2 h1:JUuyv17rKrqYfLFnP9A4Yh6hw0635ugB93bMg2bwuDQ=
github.com/go-test/deep v1.1.2/go.mod h1:lwVFiFFWPjydTsV3xGJ/YTQsnWwi8zxBFM+gOVcwINE=
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM=
@@ -555,32 +553,32 @@ github.com/google/pprof v0.0.0-20210609004039-a478d1d731e9/go.mod h1:kpwsk12EmLe
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20211214055906-6f57359322fd/go.mod h1:KgnwoLYCZ8IQu3XUZ8Nc/bM9CCZFOyjUNOSygVozoDg=
github.com/google/pprof v0.0.0-20240227163752-401108e1b7e7/go.mod h1:czg5+yv1E0ZGTi6S6vVK1mke0fV+FaUhNGcd6VRS9Ik=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
github.com/google/pprof v0.0.0-20261016150608-1facf8f91b50 h1:PlNWJRxu5APSV0pWqfIsZ0qt9OOGHSWfZQ4zvx+WgWs=
github.com/google/pprof v0.0.0-20261016150608-1facf8f91b50/go.mod h1:yUo1AMMpsPIDFGuwDK8nLcf8+vvAd5l2O6JDZ6hpwrM=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/s2a-go v0.1.10 h1:EMp+aOuXN6l8cE/gjF5Bt+vyZxsUuyCWe9chDWR/+uU=
github.com/google/s2a-go v0.1.10/go.mod h1:pz4tyvwXvJLLbyrkh6FW1eS2zPUXMaTmyNhYtyP2tNw=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.2.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.20 h1:t/xL64VUoN69MuMRQuJETqYGOw4Z9mSRJK9epIEtwFk=
github.com/googleapis/enterprise-certificate-proxy v0.3.20/go.mod h1:L3D/IQExI6LqEjBdXcZQ1WluSgigQmSwBboFstVPM4w=
github.com/googleapis/enterprise-certificate-proxy v0.3.22 h1:NU4XpII6jD+Dxcot94fqjE+AfJoE/lQP9q3faYGzC/c=
github.com/googleapis/enterprise-certificate-proxy v0.3.22/go.mod h1:L3D/IQExI6LqEjBdXcZQ1WluSgigQmSwBboFstVPM4w=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pfu6SK+H1/DsU0=
github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM=
github.com/googleapis/gax-go/v2 v2.24.0 h1:myMaPYyF9MecEmvQqMqomIwn9t/4KCZN9qnwsS76wlg=
github.com/googleapis/gax-go/v2 v2.24.0/go.mod h1:IaTHBDd7NHxSCiu0vEs8pQZu4dGZrWwuSoxCnk16OFM=
github.com/googleapis/gax-go/v2 v2.24.1 h1:AtqTN21IXMMWo99LiEVAiBfNNQmO40d8xUfZI640mc0=
github.com/googleapis/gax-go/v2 v2.24.1/go.mod h1:bWeBei0NVwaNZKb2y1HUBS7gLXIF3/Tu3pq7j8D2Tb0=
github.com/gookit/assert v0.1.1 h1:lh3GcawXe/p+cU7ESTZ5Ui3Sm/x8JWpIis4/1aF0mY0=
github.com/gookit/assert v0.1.1/go.mod h1:jS5bmIVQZTIwk42uXl4lyj4iaaxx32tqH16CFj0VX2E=
github.com/gookit/color v1.2.5/go.mod h1:AhIE+pS6D4Ql0SQWbBeXPHw7gY0/sjHoA4s/n1KB7xg=
github.com/gookit/color v1.6.1 h1:KoTnDxJPRgrL0SoX0f8rCFg2zI0t4E3GZZBMo2nN8LU=
github.com/gookit/color v1.6.1/go.mod h1:9ACFc7/1IpHGBW8RwuDm/0YEnhg3dwwXpoMsmtyHfjs=
github.com/gookit/color v1.6.3 h1:m6E49x+KftGaspcLtPLMsyci2ARhWIUW0PUV1owMlhs=
github.com/gookit/color v1.6.3/go.mod h1:9ACFc7/1IpHGBW8RwuDm/0YEnhg3dwwXpoMsmtyHfjs=
github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
github.com/gpustack/gguf-parser-go v0.26.3 h1:F63PlUPIW56HGU5k4Crv0JAG9ojtz/TuwzgMYlb82Ec=
github.com/gpustack/gguf-parser-go v0.26.3/go.mod h1:y4TwTtDqFWTK+xvprOjRUh+dowgU2TKCX37vRKvGiZ0=
github.com/gpustack/gguf-parser-go v0.27.0 h1:eXFU9pmJEMQHOTL9qI742lXHWSa1CtlMWUU3gFce7nM=
github.com/gpustack/gguf-parser-go v0.27.0/go.mod h1:y4TwTtDqFWTK+xvprOjRUh+dowgU2TKCX37vRKvGiZ0=
github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
github.com/hashicorp/aws-sdk-go-base/v2 v2.0.0-beta.74 h1:mymLUKThnV9wFvogOK8NnsMP9/vlhnjXY98gr2QIGW8=
github.com/hashicorp/aws-sdk-go-base/v2 v2.0.0-beta.74/go.mod h1:Bh9qYL8ehmDxSg14Tk8oxFCP90XHfs6NxV1D84884xA=
@@ -593,8 +591,8 @@ github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtng
github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
github.com/hashicorp/go-getter v1.8.9 h1:1AOTMUmz/S/GuqOTE6+bg6nwPXTEbKr3Tc/T/lVS7is=
github.com/hashicorp/go-getter v1.8.9/go.mod h1:qI7vH/m552bXutKe4UkWptOJl4bo2KeO/CSOoh4s0ps=
github.com/hashicorp/go-getter v1.8.10 h1:uufJp+uacA0fSDRxclsHkiE6yUQTKgwv5b25QexgVM8=
github.com/hashicorp/go-getter v1.8.10/go.mod h1:5HyKGCQmT+I3MGAM76vk/fWpktjwxn7xruCUr9CrVuo=
github.com/hashicorp/go-hclog v0.12.0/go.mod h1:whpDNt7SSdeAju8AWKIWsul05p54N/39EeqMAyrmvFQ=
github.com/hashicorp/go-hclog v1.0.0/go.mod h1:whpDNt7SSdeAju8AWKIWsul05p54N/39EeqMAyrmvFQ=
github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=
@@ -666,13 +664,11 @@ github.com/kevinburke/ssh_config v1.2.0/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/klauspost/compress v1.20.3 h1:zWNoFqq/lSuYQRim3dUXlXeA0IqwuZGicdG5ZTfY5j4=
github.com/klauspost/compress v1.20.3/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU=
github.com/klauspost/pgzip v1.2.6/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
github.com/klauspost/pgzip v1.2.7 h1:02QB3Ttao6zOWDnSsv3bIvjN24bX0eGjWniQ8vuBfkA=
github.com/klauspost/pgzip v1.2.7/go.mod h1:g7E6NrOKHOzah4QwK6Ue1tNCJs8IDiNOfjiXTr85U2E=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
@@ -713,10 +709,10 @@ github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsRe
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-localereader v0.0.2-0.20220822084749-2491eb6c1c75 h1:P8UmIzZMYDR+NGImiFvErt6VWfIRPuGM+vyjiEdkmIw=
github.com/mattn/go-localereader v0.0.2-0.20220822084749-2491eb6c1c75/go.mod h1:8fBrzywKY7BI3czFoHkuzRoWE9C+EiG4R1k4Cjx5p88=
github.com/mattn/go-runewidth v0.0.27 h1:Feg/Oou5zI/wnpgDF6omIU0OokC9GxLC/WRknhVlIR0=
github.com/mattn/go-runewidth v0.0.27/go.mod h1:3qAiGCV4Koz/yuveO58qUefmUTRm8r0IGEXZ9jeHp/8=
github.com/mattn/go-sqlite3 v1.14.48 h1:7XHIgl0a8HwOaiK4E47ozLkST78rR9+OtNGx27D/TFs=
github.com/mattn/go-sqlite3 v1.14.48/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
github.com/mattn/go-runewidth v0.0.31 h1:1oCV4ReTlosDBc6rYRw5cL8S/4y8iv5oRl4SJ5IBgwo=
github.com/mattn/go-runewidth v0.0.31/go.mod h1:3qAiGCV4Koz/yuveO58qUefmUTRm8r0IGEXZ9jeHp/8=
github.com/mattn/go-sqlite3 v1.14.52 h1:wVbm2Qnf4OXkqhBTSPuCRZDRnxfbVrrmiCEroVdog8U=
github.com/mattn/go-sqlite3 v1.14.52/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d h1:5PJl274Y63IEHC+7izoQE9x6ikvDFZS2mDVS3drnohI=
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
@@ -750,10 +746,10 @@ github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3N
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg=
github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc=
github.com/moby/moby/api v1.56.0 h1:GQzua3NA599ASSIICx0iFgiJeO9YkdDARvQsm23ZZuQ=
github.com/moby/moby/api v1.56.0/go.mod h1:sZ+THbVWkjOmBPPfbnzdD/G1LuIexWhqlSHHPTDQ1Uk=
github.com/moby/moby/client v0.6.0 h1:AJjEB21QPbXSXjDsZorFBoDZPhMrfbpaPLgSMAW9Bgs=
github.com/moby/moby/client v0.6.0/go.mod h1:OCo00wNRyA3m4lmJ228W3JbyCN4ZNNYjpOXiJydBdcQ=
github.com/moby/moby/api v1.56.1 h1:PpWkvVPB7Fr/No8w+TfyJ/I6rWZ2YPZhT3JorIg+06c=
github.com/moby/moby/api v1.56.1/go.mod h1:sZ+THbVWkjOmBPPfbnzdD/G1LuIexWhqlSHHPTDQ1Uk=
github.com/moby/moby/client v0.6.2 h1:ekSPlMccrZGcw6Scb8T9D39qE0mB+XupU/yS0zNw9N4=
github.com/moby/moby/client v0.6.2/go.mod h1:DGcnO7qmPJrqobx/ggQgQ/AUk+2wgVdQv7uxB0/mEsc=
github.com/moby/sys/capability v0.4.0 h1:4D4mI6KlNtWMCM1Z/K0i7RV1FkX+DBDHKVJpCndZoHk=
github.com/moby/sys/capability v0.4.0/go.mod h1:4g9IK291rVkms3LKCDOoYlnV8xKwoDTpIrNEE35Wq0I=
github.com/moby/sys/mountinfo v0.7.2 h1:1shs6aH5s4o5H2zQLn796ADW1wMrIwHsyJ2v9KouLrg=
@@ -779,8 +775,8 @@ github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIf
github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc=
github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/ncruces/go-strftime v1.1.0 h1:vtPmPWMkb539RGV9WpWT9EwJGvrOCKvMNI4obWL47Hk=
github.com/ncruces/go-strftime v1.1.0/go.mod h1:yVaq7iwCN09vHRrKBrrflXT0M9tTMG2KmZ9Fj7xeG/g=
github.com/nix-community/go-nix v0.0.0-20250101154619-4bdde671e0a1 h1:kpt9ZfKcm+EDG4s40hMwE//d5SBgDjUOrITReV2u4aA=
github.com/nix-community/go-nix v0.0.0-20250101154619-4bdde671e0a1/go.mod h1:qgCw4bBKZX8qMgGeEZzGFVT3notl42dBjNqO2jut0M0=
github.com/nsf/jsondiff v0.0.0-20210926074059-1e845ec5d249 h1:NHrXEjTNQY7P0Zfx1aMrNhpgxHmow66XQtm0aQLY0AE=
@@ -808,8 +804,10 @@ github.com/opencontainers/selinux v1.15.1/go.mod h1:LenyElirjUHszfxrjuFqC85HIeXZ
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/pb33f/ordered-map/v2 v2.3.1 h1:5319HDO0aw4DA4gzi+zv4FXU9UlSs3xGZ40wcP1nBjY=
github.com/pb33f/ordered-map/v2 v2.3.1/go.mod h1:qxFQgd0PkVUtOMCkTapqotNgzRhMPL7VvaHKbd1HnmQ=
github.com/pb33f/go-yaml v0.1.0 h1:zZ6Gg+ATbumXlXZNLMHIe7+zTkVqe20lx+TSYRrXEuY=
github.com/pb33f/go-yaml v0.1.0/go.mod h1:QvWdkHP5VjFYAeHMQUkaPQqREaQnBNOXmWzAjNrnSdo=
github.com/pb33f/ordered-map/v2 v2.3.2 h1:wDyaZ2Pv9QLh64X4utCeD5Zoi9nIv2aW3lwv172O5PQ=
github.com/pb33f/ordered-map/v2 v2.3.2/go.mod h1:1OhFrXu3OYw3kM+FXF+Ug3ugmmZ9LE8z0JfQMLvtV0w=
github.com/pborman/indent v1.2.1 h1:lFiviAbISHv3Rf0jcuh489bi06hj98JsVMtIDZQb9yM=
github.com/pborman/indent v1.2.1/go.mod h1:FitS+t35kIYtB5xWTZAPhnmrxcciEEOdbyrrpz5K6Vw=
github.com/pelletier/go-toml v1.9.4/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=
@@ -821,8 +819,8 @@ github.com/pierrec/lz4/v4 v4.1.26 h1:GrpZw1gZttORinvzBdXPUXATeqlJjqUG/D87TKMnhjY
github.com/pierrec/lz4/v4 v4.1.26/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
github.com/piprate/json-gold v0.7.0 h1:bEMirgA5y8Z2loTQfxyIFfY+EflxH1CTP6r/KIlcJNw=
github.com/piprate/json-gold v0.7.0/go.mod h1:RVhE35veDX19r5gfUAR+IYHkAUuPwJO8Ie/qVeFaIzw=
github.com/pjbgf/sha1cd v0.6.0 h1:3WJ8Wz8gvDz29quX1OcEmkAlUg9diU4GxJHqs0/XiwU=
github.com/pjbgf/sha1cd v0.6.0/go.mod h1:lhpGlyHLpQZoxMv8HcgXvZEhcGs0PG/vsZnEJ7H0iCM=
github.com/pjbgf/sha1cd v0.7.0 h1:ZRNPKHj+gfkLBf0KJv/p1Hmz+7bqCT5o311YX0nq+DA=
github.com/pjbgf/sha1cd v0.7.0/go.mod h1:pKR5Li+qTCo+ebqITZfwPlJGoCFCvSO00qxjbNtTUFQ=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -884,8 +882,8 @@ github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d h1:hrujxIzL1woJ7
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d/go.mod h1:uugorj2VCxiV1x+LzaIdVa9b4S4qGAcH6cbhh4qVxOU=
github.com/sanity-io/litter v1.5.8 h1:uM/2lKrWdGbRXDrIq08Lh9XtVYoeGtcQxk9rtQ7+rYg=
github.com/sanity-io/litter v1.5.8/go.mod h1:9gzJgR2i4ZpjZHsKvUXIRQVk7P+yM3e+jAF7bU2UI5U=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.3 h1:1EYB5IzjZawrrnELUi78f9fPu57HuXjmddZPjrls/28=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.3/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/sassoftware/go-rpmutils v0.4.0 h1:ojND82NYBxgwrV+mX1CWsd5QJvvEZTKddtCdFLPWhpg=
github.com/sassoftware/go-rpmutils v0.4.0/go.mod h1:3goNWi7PGAT3/dlql2lv3+MSN5jNYPjT5mVcQcIsYzI=
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e h1:7q6NSFZDeGfvvtIRwBrU/aegEYJYmvev0cHAwo17zZQ=
@@ -893,20 +891,20 @@ github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e/go.mod h1:DkpGd7
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
github.com/sebdah/goldie/v2 v2.8.0 h1:dZb9wR8q5++oplmEiJT+U/5KyotVD+HNGCAc5gNr8rc=
github.com/sebdah/goldie/v2 v2.8.0/go.mod h1:oZ9fp0+se1eapSRjfYbsV/0Hqhbuu3bJVvKI/NNtssI=
github.com/secure-systems-lab/go-securesystemslib v0.11.0 h1:iuCR9kcMFD4QurdKrGvPLoKZLv9YvwPYVr0473BdtFs=
github.com/secure-systems-lab/go-securesystemslib v0.11.0/go.mod h1:+PMOTjUGwHj2vcZ+TFKlb1tXRbrdWE1LYDT5i9JC80Q=
github.com/secure-systems-lab/go-securesystemslib v0.11.1 h1:ayahDPjfSIqKegyt5YVGEvQ7SAi72GHXTy+b2YjUd5w=
github.com/secure-systems-lab/go-securesystemslib v0.11.1/go.mod h1:UyOjhoZLi76ir63u1ptubEevVe6dlNPmNxwgVvKqFFY=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sergi/go-diff v1.4.0 h1:n/SP9D5ad1fORl+llWyN+D6qoUETXNZARKjyY2/KVCw=
github.com/sergi/go-diff v1.4.0/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
github.com/shopspring/decimal v1.4.0 h1:bxl37RwXBklmTi0C79JfXCEBD1cqqHt0bbgBAGFp81k=
github.com/shopspring/decimal v1.4.0/go.mod h1:gawqmDU56v4yIKSwfBSFip1HdCCXN8/+DMd9qYNcwME=
github.com/sigstore/fulcio v1.8.7 h1:d7QJQxukgXarqPCT9uDNeyzt1Bbt/biw58/VfOnwI/o=
github.com/sigstore/fulcio v1.8.7/go.mod h1:7fX+QyigZxRGKTgHvTfOfxwwniCDxK02PVloHy5/vg4=
github.com/sigstore/protobuf-specs v0.5.1 h1:/5OPaNuolRJmQfeZLayJGFXMpsRJEdgC6ah1/+7Px7U=
github.com/sigstore/protobuf-specs v0.5.1/go.mod h1:DRBzpFuE+LnvQMN10/dU6nBeKwVLGEQ6o2FovN2Rats=
github.com/sigstore/sigstore v1.10.8 h1:1Mgkxvkw4AXMfIP1DOjc6kw0GkUgA8pGVpveN/EfOq4=
github.com/sigstore/sigstore v1.10.8/go.mod h1:f9+B/4iaYimvUkySyb2mvc73n3RLqNn24grHZM/ET8M=
github.com/sigstore/fulcio v1.8.8 h1:AtVoVQ4R7YaP6RHIH9puHyB398CHS/9WYc6d+gKf5Y0=
github.com/sigstore/fulcio v1.8.8/go.mod h1:RaFomAGGM7VZKAR9ZGqQhcf1Peryc9E1020/kbX8Uto=
github.com/sigstore/protobuf-specs v0.5.2 h1:RSWWUY8QrVTxbYH00jY/jg2e7YnjzrpwP+PeHTMll0E=
github.com/sigstore/protobuf-specs v0.5.2/go.mod h1:DRBzpFuE+LnvQMN10/dU6nBeKwVLGEQ6o2FovN2Rats=
github.com/sigstore/sigstore v1.10.11 h1:fjrwDfR0AWmRONYV/hKGJyqbT0osMtaBz9FTZ6MH9XI=
github.com/sigstore/sigstore v1.10.11/go.mod h1:ecPk+ftLqP8/TmzQFJmn0+QrL2yUQIJvyzVKNmDiw6U=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
@@ -945,8 +943,8 @@ github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3A
github.com/spf13/viper v1.10.0/go.mod h1:SoyBPwAtKDzypXNDFKN5kzH7ppppbGZtls1UpIy5AsM=
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
github.com/spiffe/go-spiffe/v2 v2.7.0 h1:uXe1MflJoHw58wAUvxVlcM7WpKtijWG7I1UidcGh6g4=
github.com/spiffe/go-spiffe/v2 v2.7.0/go.mod h1:47Q0Q9/AqGha8QLHp+kxpH4Wca7X7EnOtlIJy3mxZ3U=
github.com/spiffe/go-spiffe/v2 v2.8.1 h1:eXZMLsu+3MLEPJyGJkolqtVrteZfQdUpOWj6LTiDl/E=
github.com/spiffe/go-spiffe/v2 v2.8.1/go.mod h1:47Q0Q9/AqGha8QLHp+kxpH4Wca7X7EnOtlIJy3mxZ3U=
github.com/stefanberger/go-pkcs11uri v0.0.0-20230803200340-78284954bff6 h1:pnnLyeX7o/5aX8qUQ69P/mLojDqwda8hFOCBTmP/6hw=
github.com/stefanberger/go-pkcs11uri v0.0.0-20230803200340-78284954bff6/go.mod h1:39R/xuhNgVhi+K0/zst4TLrJrVmbm6LVgl4A0+ZFS5M=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -970,8 +968,8 @@ github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUm
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/sylabs/sif/v2 v2.24.1 h1:OhTOfTwBaGXfbWYWXK9C6Pojma3A1bTaNJ6CEhyuKok=
github.com/sylabs/sif/v2 v2.24.1/go.mod h1:PRq9MoP0g+p0qE5ZRGrOhyVAUdHrPsAVmXHgoiW6VU0=
github.com/sylabs/sif/v2 v2.24.2 h1:rdkFf6z3xLC6yKweU7S9marlKfkS91Ht0ruIri6YWFY=
github.com/sylabs/sif/v2 v2.24.2/go.mod h1:bbrkk0JhTX+xNJcnUtmDkivoO2r4DGWM3LxNR0U2ooA=
github.com/sylabs/squashfs v1.0.6 h1:PvJcDzxr+vIm2kH56mEMbaOzvGu79gK7P7IX+R7BDZI=
github.com/sylabs/squashfs v1.0.6/go.mod h1:DlDeUawVXLWAsSRa085Eo0ZenGzAB32JdAUFaB0LZfE=
github.com/tailscale/hujson v0.0.0-20260302212456-ecc657c15afd h1:Rf9uhF1+VJ7ZHqxrG8pJ6YacmHvVCmByDmGbAWCc/gA=
@@ -994,16 +992,18 @@ github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM=
github.com/ulikunitz/xz v0.5.8/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/ulikunitz/xz v0.5.16 h1:ld6NyySjx5lowVKwJvMRLnW5nxKX/xnpSiFYZ/Lxur0=
github.com/ulikunitz/xz v0.5.16/go.mod h1:H9Rt/W6/Qj27PGauhQc6nfCDy7vHpzsOThBSaYDoEhw=
github.com/ulikunitz/xz v0.5.17 h1:flR0y/x1hgM8EGV1AW3Xll6T413G0glV8UfBwR617V4=
github.com/ulikunitz/xz v0.5.17/go.mod h1:H9Rt/W6/Qj27PGauhQc6nfCDy7vHpzsOThBSaYDoEhw=
github.com/varlink/go v0.4.1-0.20261015173642-c935be0c53f7 h1:pl46d1ZU17758S4EQhns0po6ZPtX+EdHgMPd1uvjia4=
github.com/varlink/go v0.4.1-0.20261015173642-c935be0c53f7/go.mod h1:Cqta0IQ13zfx9ZFugzjWDKUhyMryjOfMKoPrC3TY8fc=
github.com/vbatts/go-mtree v0.7.0 h1:ytmOc3MTRidZiBi9VBCyZ2BHe4fZS47L5v7BVXDWW4E=
github.com/vbatts/go-mtree v0.7.0/go.mod h1:EjdpFC+LZy1TXbRGNa1MKKgjQ+7ew3foMFJK8o4/TdY=
github.com/vbatts/tar-split v0.12.3 h1:Cd46rkGXI3Td4yrVNwU8ripbxFaQbmesqhjBUUYAJSw=
github.com/vbatts/tar-split v0.12.3/go.mod h1:sQOc6OlqGCr7HkGx/IDBeKiTIvqhmj8KffNhEXG4Nq0=
github.com/vbauerster/cupwriter v0.0.4 h1:9sBPe0uXWLZuWQU5lqVbhyFlxX6c09asST/YfatFAys=
github.com/vbauerster/cupwriter v0.0.4/go.mod h1:IFyzS6Xis5dnBH/rdAhrnuzg3c+KkUqEN6yE8lhJlDw=
github.com/vbauerster/mpb/v8 v8.14.0 h1:55SR80dptMfASxIG/oCEkBXgBhxeSu4GrVsjl16oKmA=
github.com/vbauerster/mpb/v8 v8.14.0/go.mod h1:HgpQPKfcWe3kbuGGPmi+jatHreMase5C3Fp5dpdAy0Q=
github.com/vbauerster/cupwriter v0.0.5 h1:pxlPgxPLIebXh8lJPOZmn2TmjPE8+PqJIFkKkr7hI9I=
github.com/vbauerster/cupwriter v0.0.5/go.mod h1:ptJDgOtLtGZp1gLzycD0TFnZztDhrQdsikG6EAA4xYQ=
github.com/vbauerster/mpb/v8 v8.16.2 h1:+UH2Nhn6asKkdX58hSgN+5Jqg/JcFSqCcU+8z0IiYYo=
github.com/vbauerster/mpb/v8 v8.16.2/go.mod h1:PFkxR+jl7Eew9ukpdUcodHBGq9ow74b6cXI7xL0t3mA=
github.com/vifraa/gopom v1.0.0 h1:L9XlKbyvid8PAIK8nr0lihMApJQg/12OBvMA28BcWh0=
github.com/vifraa/gopom v1.0.0/go.mod h1:oPa1dcrGrtlO37WPDBm5SqHAT+wTgF8An1Q71Z6Vv4o=
github.com/wagoodman/go-partybus v0.0.0-20230516145632-8ccac152c651 h1:jIVmlAFIqV3d+DOxazTR9v+zgj8+VYuQBzPgBZvWBHA=
@@ -1072,10 +1072,10 @@ go.opentelemetry.io/otel/sdk/metric v1.46.0/go.mod h1:I1PbKrdVc8Qu8HYVDNtqVIwLwj
go.opentelemetry.io/otel/trace v1.46.0 h1:OULy7ccdJnZtJ0UDYFOIGaCmiWzJ8Vi2G/Rsu60qs1c=
go.opentelemetry.io/otel/trace v1.46.0/go.mod h1:J7GAXweO77XSFkB/rmAqk9D6ihszhFjLU+d9WuUxDLI=
go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
go.podman.io/image/v5 v5.41.2 h1:YnVaFrGLPWSBmipKfNvlOsGcBtk5bUxlPxQ+A6CS/uo=
go.podman.io/image/v5 v5.41.2/go.mod h1:T/bzxEFpdpAzyHabPmcq5K+my3wBsht22sFkFjJSKT4=
go.podman.io/storage v1.64.1 h1:DqbG8scW/bD3xHiSRzV7O5fzn46qVC+SwmUN6C6Fr5w=
go.podman.io/storage v1.64.1/go.mod h1:ft0DzCRMZs1yn0rszmLYXEEwgd7IIc66JRwSRPUDwwc=
go.podman.io/image/v5 v5.42.0 h1:C//Ix0Thj2LYOQO6++NjU3McLFLQbf4czgUyH98R9gc=
go.podman.io/image/v5 v5.42.0/go.mod h1:mmIgf9EdpF68j8vdX1/OxhOSIknU1PO3G6/ClRkrwm4=
go.podman.io/storage v1.65.0 h1:GgY5VibEw6coVhOGYWuxZ4F7H2AaTfPiwL7E1CJ2dnM=
go.podman.io/storage v1.65.0/go.mod h1:ZDXsVSzdho+p4fDky+iMVCPwRtjS1kSiHcyhBwUB6M0=
go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
@@ -1084,8 +1084,6 @@ go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
go.yaml.in/yaml/v4 v4.0.0-rc.2 h1:/FrI8D64VSr4HtGIlUtlFMGsm7H7pWTbj6vOLVZcA6s=
go.yaml.in/yaml/v4 v4.0.0-rc.2/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
go4.org v0.0.0-20230225012048-214862532bf5 h1:nifaUDeh+rPaBCMPMQHZmvJf+QdpLFnuQPwx+LxVmtc=
go4.org v0.0.0-20230225012048-214862532bf5/go.mod h1:F57wTi5Lrj6WLyswp5EYV1ncrEbFGHD4hhz6S1ZYeaU=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
@@ -1104,8 +1102,8 @@ golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliY
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.33.0/go.mod h1:bVdXmD7IV/4GdElGPozy6U7lWdRXA4qyRVGJV57uQ5M=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54=
golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -1149,8 +1147,8 @@ golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
golang.org/x/mod v0.42.0 h1:ICeL3t2L30ljSPHHxDnR4fqVcBt/zR2zJAb9juavmQg=
golang.org/x/mod v0.42.0/go.mod h1:u5fzIrRInkL8MaO1cbRkUHT64v303YIT5KEqpNr7jRs=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -1200,8 +1198,8 @@ golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.59.0 h1:5zfYln+w5XCxwrnMMJPufRgNoXEaGxl0wo5GqPXyues=
golang.org/x/net v0.59.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw=
golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -1219,8 +1217,8 @@ golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a/go.mod h1:KelEdhl1UZF7XfJ
golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20211005180243-6b3c2da341f1/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/oauth2 v0.37.0 h1:JUlcxA8oAtauLfiH8FX2/FkAWHAdi0QtGCGc+hofE98=
golang.org/x/oauth2 v0.37.0/go.mod h1:IxwZNxUULJmpBFf9K/9NTMSIfZZuvuTy1gGxhigP/58=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -1238,8 +1236,8 @@ golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA=
golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -1319,8 +1317,8 @@ golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
@@ -1330,8 +1328,8 @@ golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/term v0.29.0/go.mod h1:6bl4lRlvVuDgSf3179VpIxBF0o10JUpXWOnI7nErv7s=
golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE=
golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc=
golang.org/x/term v0.47.0 h1:cpyOwQkr2Nzxe2p6KnrTG7Ei2wtzjsuSjhpXXepZ/BM=
golang.org/x/term v0.47.0/go.mod h1:efAfAuG6RKABvg1ArGebRkvX6LXul14SM67o2e4mUCY=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -1347,8 +1345,8 @@ golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY=
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -1412,8 +1410,8 @@ golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/tools v0.50.0 h1:c2ifzfcuY7L90lZ2aKd8S4K2NpASF08SZx9ZuJkHmSU=
golang.org/x/tools v0.50.0/go.mod h1:7ulVMw3831Mwi5EZD6RomGyffr4VFjuNYXf2BbCEAV0=
golang.org/x/tools v0.52.0 h1:ryxg4U63oaUYk7uOLTF37n43f4qgtGvqV4Ioit1BaEs=
golang.org/x/tools v0.52.0/go.mod h1:ItcP5UNaVGG+Yc4eIYmaFLNJ4soEopkx5Y2d4lcEurs=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -1454,8 +1452,8 @@ google.golang.org/api v0.57.0/go.mod h1:dVPlbZyBo2/OjBpmvNdpn2GRm6rPy75jyU7bmhdr
google.golang.org/api v0.59.0/go.mod h1:sT2boj7M9YJxZzgeZqXogmhfmRWDtPzT31xkieUbuZU=
google.golang.org/api v0.61.0/go.mod h1:xQRti5UdCmoCEqFxcz93fTl338AVqDgyaDRuOZ3hg9I=
google.golang.org/api v0.62.0/go.mod h1:dKmwPCydfsad4qCH08MSdgWjfHOyfpd4VtDGgRFdavw=
google.golang.org/api v0.294.0 h1:8gASjJxdtcIieB3OqbkLcF0FfbXVNqKtU5iozD1ssvA=
google.golang.org/api v0.294.0/go.mod h1:02qB8+Ox1ZFzcaKFMguy1nQLJmSIyvV6Ff4txJEXtl4=
google.golang.org/api v0.299.0 h1:b3K+ydSMd0kh6TQI6bJyApRQfqQX2MfSOaVkpM59mJw=
google.golang.org/api v0.299.0/go.mod h1:zlR3GVA8b2R5nv5Ij9UWe37StVB3cxDD7DBFi4ZFsHw=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
@@ -1530,8 +1528,8 @@ google.golang.org/genproto v0.0.0-20260715232425-e75dac1f907d h1:C9v1o0/4quuhOAf
google.golang.org/genproto v0.0.0-20260715232425-e75dac1f907d/go.mod h1:Wz2wFJntZFmLGo7pLDXZ3wYk5hyc0Mb+SkHhDDXT+lU=
google.golang.org/genproto/googleapis/api v0.0.0-20260825221802-da73d73af1c5 h1:izFU9hz7aeLI/Mi1J0991ae+xcwRLr7hTqWnB/9aIIU=
google.golang.org/genproto/googleapis/api v0.0.0-20260825221802-da73d73af1c5/go.mod h1:3LhxRw4YYkf+ylAfgaY9JlVLFKhokkCV8duhLLe7+t0=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260825221802-da73d73af1c5 h1:1VUiZAXyC+zmiFYi+WLtBzr68Cj8wOofHjjrA/kkizc=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260825221802-da73d73af1c5/go.mod h1:DjtHYE8FKJLivXcBEjGwndXfIC23G0VpXiXKqG179uA=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260921155816-b14227669459 h1:b0xCahf3FK2m2Cv0p4vTozGPWncCvLfwV86UNg8xWU8=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260921155816-b14227669459/go.mod h1:OaIUM3+LpYcK2GXM4FTmhWoIq371Owdr+Cc7/BsYHHc=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
@@ -1559,8 +1557,8 @@ google.golang.org/grpc v1.39.1/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnD
google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34=
google.golang.org/grpc v1.40.1/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34=
google.golang.org/grpc v1.42.0/go.mod h1:k+4IHHFw41K8+bbowsex27ge2rCb65oeWqe4jJ590SU=
google.golang.org/grpc v1.83.2 h1:EManeRomTObA0BU7I8vXgg/78uE5MJ9M8B39EX2WscU=
google.golang.org/grpc v1.83.2/go.mod h1:YPI1hK3kDked6iHvgX3tR0y+nX/qpMFKhPgFsokw1S8=
google.golang.org/grpc v1.84.0 h1:soMyaPJ8pAak5PIQ0DGBUir0XRo2fRoMqhNWMLlLxO0=
google.golang.org/grpc v1.84.0/go.mod h1:ljCht0DrxQrXBDRTZp52Qxh3Ffk8CdYm2sj4O2QN2C0=
google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
@@ -1610,20 +1608,20 @@ honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
howett.net/plist v1.0.1 h1:37GdZ8tP09Q35o9ych3ehygcsL+HqKSwzctveSlarvM=
howett.net/plist v1.0.1/go.mod h1:lqaXoTrLY4hg8tnEzNru53gicrbv7rrk+2xJA/7hw9g=
modernc.org/cc/v4 v4.29.2 h1:h6+9ciCnPKutf4I03CvheAvDLX7+IHlqR6Iy6J+cgd8=
modernc.org/cc/v4 v4.29.2/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.35.0 h1:F+TUsmw09QxLzmi3aeYYGxjAXarmZaKgj3mKQHNaA8w=
modernc.org/ccgo/v4 v4.35.0/go.mod h1:qrVGs9S3Sr2Ztcg9ve+kTAYMp5a3YvWjo+SoN06kJ5I=
modernc.org/cc/v4 v4.30.3 h1:aDq8gIX1X7+VFgO/JYYji6pLWL2H4tv3g8WatT3KCys=
modernc.org/cc/v4 v4.30.3/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.37.0 h1:BAuJCUtIHIxFIYaFhqXIw5YaoMBlNCUlFovejHfMip4=
modernc.org/ccgo/v4 v4.37.0/go.mod h1:/8627VhwWtAbqUXmPORxFQU5SgLrAx2jl9seLw9GBhY=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/gc/v3 v3.2.0 h1:/Y3LGDMQaoYf55PebP1qsZ5FDGgAZyG1RYKEHeQzh1o=
modernc.org/gc/v3 v3.2.0/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.75.7 h1:o3DTP9/0p9pKmY2WCKQaySW6wIiZhNM7wc2lUoyhfew=
modernc.org/libc v1.75.7/go.mod h1:bO5o2ztHxBb2rjz0PgdHN0sSMw57CgxGFLZ3Qd/QpVQ=
modernc.org/libc v1.79.0 h1:9Dvvv++bAMSkSuBzuOFAatbTV5OIoq6Y6FohmD3e7q8=
modernc.org/libc v1.79.0/go.mod h1:uT3kR61NsbeKJSLqq83GJwSnZMVLlSjyAz4DfTaBy1M=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
@@ -1632,8 +1630,8 @@ modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.59.0 h1:X1es1GpqBlS/5T+vbM4HLUdaa8OtQx468DF2vrx+38A=
modernc.org/sqlite v1.59.0/go.mod h1:+paeT2A3iPRHkQDwG7oA6Tk0zQd5woMEI8q7orfry8k=
modernc.org/sqlite v1.61.0 h1:sQi63oLQ9uquAUA4CzOisH35Q3pEszkPBuHM0tFL8CY=
modernc.org/sqlite v1.61.0/go.mod h1:f4gr5tbpx2umklaT/QVtbi/50hNdM+4ng+2zMo8KXJ0=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
+52 -22
View File
@@ -8,7 +8,6 @@ REPO="${PROJECT_NAME}"
GITHUB_DOWNLOAD_PREFIX=https://github.com/${OWNER}/${REPO}/releases/download
INSTALL_SH_BASE_URL=https://get.anchore.io/${PROJECT_NAME}
LEGACY_INSTALL_SH_BASE_URL=https://raw.githubusercontent.com/${OWNER}/${PROJECT_NAME}
PROGRAM_ARGS=$@
# signature verification options
@@ -19,6 +18,8 @@ VERIFY_SIGN=false
VERIFY_SIGN_SUPPORTED_VERSION=v0.104.0
# this is the earliest tag in the repo where the -v flag was introduced to this install.sh script
VERIFY_SIGN_FLAG_VERSION=v1.6.0
# releases at or after this version are signed with a sigstore bundle (.sigstore.json); earlier releases ship a separate .sig and .pem
VERIFY_SIGN_BUNDLE_VERSION=v1.53.0
# do not change the name of this parameter (this must always be backwards compatible)
DOWNLOAD_TAG_INSTALL_SCRIPT=${DOWNLOAD_TAG_INSTALL_SCRIPT:-true}
@@ -256,8 +257,9 @@ hash_sha256() (
hash=$(shasum -a 256 "$TARGET" 2>/dev/null) || return 1
echo "$hash" | cut -d ' ' -f 1
elif is_command openssl; then
hash=$(openssl -dst openssl dgst -sha256 "$TARGET") || return 1
echo "$hash" | cut -d ' ' -f a
hash=$(openssl dgst -sha256 "$TARGET") || return 1
# output is "SHA256(<file>)= <digest>" (or "SHA2-256(...)" on OpenSSL 3.x)
echo "${hash##* }"
else
log_err "hash_sha256 unable to find command to compute sha-256 hash"
return 1
@@ -386,8 +388,8 @@ download_github_release_checksums_files() (
complete_url=$(github_release_asset_url "${download_url}" "${name}" "${version}" "${filename}")
output_path="${output_dir}/${complete_filename}"
http_download "${output_path}" "${complete_url}" ""
asset_file_exists "${output_path}"
http_download "${output_path}" "${complete_url}" "" || return 1
asset_file_exists "${output_path}" || return 1
log_trace "download_github_release_checksums_files() returned '${output_path}' for file '${complete_filename}'"
@@ -585,28 +587,31 @@ download_and_install_asset() (
install_asset "${asset_filepath}" "${install_path}" "${binary}"
)
# verify_sign [checksums-file-path] [certificate-reference] [signature-reference] [version]
# verify_sign [checksums-file-path] [verification-material-flags...]
#
# attempts verify the signature of the checksums file from the release workflow in Github Actions run against the main branch.
# the verification material is either "--bundle <path>" or (for older releases) "--certificate <ref> --signature <ref>".
#
verify_sign() {
checksums_file=$1
cert_reference=$2
sig_reference=$3
shift
log_trace "verifying artifact $1"
log_trace "verifying artifact ${checksums_file}"
log_file=$(mktemp)
${COSIGN_BINARY} \
verify-blob "$checksums_file" \
--certificate "$cert_reference" \
--signature "$sig_reference" \
"$@" \
--certificate-identity "https://github.com/${OWNER}/${REPO}/.github/workflows/release.yaml@refs/heads/main" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" > "${log_file}" 2>&1
if [ $? -ne 0 ]; then
log_err "$(cat "${log_file}")"
# cosign older than v2.4.2 can't read the bundle's verification material and reports a missing cert
if [ "$1" = "--bundle" ] && grep -q "bundle does not contain cert" "${log_file}"; then
log_err "note: releases >= ${VERIFY_SIGN_BUNDLE_VERSION} are verified with a sigstore bundle, which requires cosign v2.5.0 or newer (check '${COSIGN_BINARY} version')"
fi
rm -f "${log_file}"
return 1
fi
@@ -630,7 +635,10 @@ download_asset() (
log_trace "download_asset(url=${download_url}, destination=${destination}, name=${name}, os=${os}, arch=${arch}, version=${version}, format=${format})"
checksums_filepath=$(download_github_release_checksums "${download_url}" "${name}" "${version}" "${destination}")
if ! checksums_filepath=$(download_github_release_checksums "${download_url}" "${name}" "${version}" "${destination}"); then
log_err "unable to download checksums file"
return 1
fi
log_trace "checksums content:\n$(cat ${checksums_filepath})"
@@ -642,13 +650,25 @@ download_asset() (
fi
if [ "$VERIFY_SIGN" = true ]; then
checksum_sig_file_url=$(github_release_checksums_sig_url "${download_url}" "${name}" "${version}")
log_trace "checksums signature url: ${checksum_sig_file_url}"
if compare_semver "${version}" "${VERIFY_SIGN_BUNDLE_VERSION}"; then
if ! checksums_bundle_filepath=$(download_github_release_checksums_files "${download_url}" "${name}" "${version}" "${destination}" "checksums.txt.sigstore.json"); then
log_err "unable to download checksums signature bundle"
return 1
fi
log_trace "checksums bundle: ${checksums_bundle_filepath}"
checksums_cert_file_url=$(github_release_checksums_cert_url "${download_url}" "${name}" "${version}")
log_trace "checksums certificate url: ${checksums_cert_file_url}"
set -- --bundle "${checksums_bundle_filepath}"
else
checksum_sig_file_url=$(github_release_checksums_sig_url "${download_url}" "${name}" "${version}")
log_trace "checksums signature url: ${checksum_sig_file_url}"
if ! verify_sign "${checksums_filepath}" "${checksums_cert_file_url}" "${checksum_sig_file_url}"; then
checksums_cert_file_url=$(github_release_checksums_cert_url "${download_url}" "${name}" "${version}")
log_trace "checksums certificate url: ${checksums_cert_file_url}"
set -- --certificate "${checksums_cert_file_url}" --signature "${checksum_sig_file_url}"
fi
if ! verify_sign "${checksums_filepath}" "$@"; then
log_err "signature verification failed"
return 1
fi
@@ -657,9 +677,15 @@ download_asset() (
asset_url="${download_url}/${asset_filename}"
asset_filepath="${destination}/${asset_filename}"
http_download "${asset_filepath}" "${asset_url}" ""
if ! http_download "${asset_filepath}" "${asset_url}" ""; then
log_err "unable to download asset '${asset_url}'"
return 1
fi
hash_sha256_verify "${asset_filepath}" "${checksums_filepath}"
# this is what ties the asset to the (optionally signature verified) checksums file, so a mismatch must stop the install
if ! hash_sha256_verify "${asset_filepath}" "${checksums_filepath}"; then
return 1
fi
log_trace "download_asset_by_checksums_file() returned '${asset_filepath}'"
@@ -765,7 +791,7 @@ prep_signature_verification() {
fi
}
main() (
parse_install_args() {
# parse arguments
# note: never change default install directory (this must always be backwards compatible)
@@ -805,6 +831,11 @@ EOF
set +u
tag=$1
set -u
}
main() (
parse_install_args "$@"
if [ -z "${tag}" ]; then
log_info "checking github for the current release tag"
@@ -812,7 +843,6 @@ EOF
else
log_info "checking github for release tag='${tag}'"
fi
set -u
if ! tag=$(get_release_tag "${OWNER}" "${REPO}" "${tag}"); then
log_err "unable to find tag='${tag}'"
@@ -843,7 +873,7 @@ EOF
log_warn "failed to fetch from ${INSTALL_SH_BASE_URL}, trying fallback URL"
install_script=$(http_copy "${LEGACY_INSTALL_SH_BASE_URL}/${tag}/install.sh" "")
fi
echo "${install_script}" | sh -s -- ${PROGRAM_ARGS}
echo "${install_script}" | sh -s -- "$@"
exit $?
fi
+106
View File
@@ -0,0 +1,106 @@
# internal/archive
Opens one archive as its own filesystem so the ordinary catalogers can run against its contents.
The archive cataloger task (`internal/task/archive_tasks.go`) drives the recursion; this package
does everything for a single archive.
## Flow
```
Extract(ctx, reader, archiveCoordinates, limiter, exclusions) -> *Resolver
acquire read in place if the reader is already seekable, else hold in memory / spill to disk
identifyFormat internal/file.IdentifyArchive first, end-of-central-directory fallback for stub-prefixed zips
Resolver.extract every entry -> entryHeader -> Resolver.add (excluded entries are never stored)
digestsOf SHA-1 of the archive itself
```
`Extract` returns `nil` for content that is not an archive (a bare compression format such as a
gzipped file that is not a tar counts as not an archive), `ErrDiskLimitReached` when the archive's
own bytes cannot be placed, and otherwise a `Resolver` whose `Cleanup` releases everything.
## Resolver
One type does the work: `Resolver` is fed entries by `Extract`, holds their content, indexes them,
and implements `file.Resolver` over the result. Each entry is a `node` carrying its header and a
`blob`; the archive's own bytes, when they have to be held at all, are one more blob.
Every location it returns extends the archive's coordinates: `RealPath` and `FileSystemID` stay
those of the outermost archive on the scanned filesystem, and `FullPath` is the `!/`-delimited
chain down to the entry (`/app.war!/WEB-INF/lib/dep.jar!/META-INF/MANIFEST.MF`). `LocalPath` is the
entry's path within this archive.
## Where bytes live
Blobs are held in memory while the memory limit admits them and in one spill file per archive
(`archive-spill-*`) once it does not; when a chunk is refused, everything held in memory moves to
the file first. The file is created on first use, so most archives never touch the filesystem, and
blobs are located in it by offset with no framing of their own. It goes under the scan's temp root
when the context carries one (`internal/tmpdir`), otherwise the system temp directory.
Anything already random-access (an `*os.File`, or a nested archive read out of its parent's
resolver) is used in place and costs nothing, so only a top-level stream is ever copied. Its memory
is refunded once it is extracted and digested; bytes it spilled wait for `Cleanup` with the entries.
## Limits
`Limits` bound what one scan holds *at once*; each falls as archives are released. For both limits,
positive is the bound, zero forbids the resource, negative is unbounded.
- `Limiter` is scan-wide; a `charge` is one archive's draw on it. Bytes are charged as they land,
never from declared sizes.
- Memory refused → content moves to disk. Disk refused → the archive's own bytes cannot be placed
(`ErrDiskLimitReached`, archive skipped) or an entry cannot be stored (`Resolver.Truncated`, the
entries stored so far are still cataloged).
- Index bookkeeping is approximated as `approxIndexBytesPerEntry` (2 KiB) per node, listed or
implied by an entry's path, plus name length. It lives in memory, so it is charged to the memory
limit, spilling held content to make room; only a zero memory limit lets it fall back to the disk
budget so that configuration still indexes. An entry whose index cost is refused is dropped like
one whose content is, and marks the resolver `Truncated`.
- `Budget` bounds the bytes decompressed under one top-level archive, summed over everything nested
in it. Unlike `Limits` it never refills, so a recursive bomb whose every level fits the limits
still ends (`errBudgetSpent`, the resolver is `Truncated`).
- `Limiter.Peak` reports the most held at once over the scan, for reporting.
## Hostile archives
Nothing an archive declares is trusted. Content is charged chunk by chunk as it decompresses, so a
deflate or gzip bomb stops at the disk limit with the entry dropped whole. Every node is charged
against the memory limit, including directories implied by deep paths, so an archive of millions of
tiny entries or of a few entries with 4 KiB names stops at the memory limit. A recursive bomb stops
at the `Budget`. Names longer than PATH_MAX are refused, and nesting stops at the task's `MaxDepth`.
See the bomb tests in `extract_test.go`.
## Lookup
Entries are indexed in a stereoscope `filetree.FileTree` + `filetree.Index` and searched with
`filetree.SearchContext`, the same index the directory and image resolvers use. A query may name an
entry by its full path or by its `LocalPath` (leading slash optional); a full path into another
archive in the scan names nothing. Directories implied by entry paths exist for `HasPath` but are
never files. The last entry at a path wins over earlier
duplicates, as it does on extraction; a file and a directory at one path keep the first.
Links resolve inside the archive only, as the file tree resolves them: a hard link names its target
from the archive root. A lookup matching a link and its target returns one location, with
`AccessPath` the full path matched and `FullPath` that of the file read. Dangling and cyclic links are not
files. `AllLocations` enumerates every non-directory path, links included. Answers are sorted by path.
## Other pieces
- `Exclusions`: the scan's exclusion patterns, of which only the `**/` shape means the same thing
re-rooted at an archive. Matched against each entry path and its parent directories at extraction
time. `syft.CreateSBOM` passes the source's patterns to the task, which builds one `Exclusions`
for the scan.
- `Traversal`: put on the context by the task so a cataloger (java) knows which archive it is inside,
what its digests are, and which packages describe its parent. `VirtualPath(location)` converts a
location's `FullPath` to the java cataloger's colon-delimited `app.war:WEB-INF/lib/inner.jar`
form, escaping `%` and `:` in entry names. `Container` and `EntryCoordinates` split and build the
coordinates of files within archives (the SPDX and CycloneDX formats use them too); `OwnerContainers` and
`OwnedCoordinates` map package-owned file paths to coordinates in the right archive.
- `WithNestedCataloging` / `NestedCatalogingEnabled`: `syft.CreateSBOM` marks the context when the
archive task is running, so the java cataloger leaves nested archives to the task instead of
opening them itself.
- `entryHeader`: keeps regular files, directories and links only (no devices or fifos), cleans entry names to
archive-relative paths, refuses names longer than PATH_MAX or containing control characters, line
separators or the `!/` archive path separator (marking the resolver truncated), and drops header
fields nothing reads (PAX records, user and group names, sub-second and access times) since the
header is held for the life of the archive.
+38
View File
@@ -0,0 +1,38 @@
package archive
import (
"strings"
"github.com/bmatcuk/doublestar/v4"
)
// Exclusions are the scan's exclusion patterns that apply inside archives.
//
// A scan accepts three pattern shapes: `./x` anchored at the scan root, `*/x` exactly one level below
// it, and `**/x` at any depth. Only the last means the same thing when re-rooted at an archive, so
// only it reaches inside one.
type Exclusions []string
func NewExclusions(scanPatterns []string) Exclusions {
var out Exclusions
for _, pattern := range scanPatterns {
if strings.HasPrefix(pattern, "**/") {
// a trailing slash reads as "a directory" but makes doublestar match nothing (issue #4839)
out = append(out, strings.TrimSuffix(pattern, "/"))
}
}
return out
}
// Excludes reports whether an archive-relative path, or any directory above it, matches a pattern.
// Directories are checked because an archive need not list a directory before the files in it.
func (e Exclusions) Excludes(entryPath string) bool {
for _, pattern := range e {
for end := len(entryPath); end > 0; end = strings.LastIndexByte(entryPath[:end], '/') {
if matched, _ := doublestar.Match(pattern, entryPath[:end]); matched {
return true
}
}
}
return false
}
+56
View File
@@ -0,0 +1,56 @@
package archive
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestNewExclusions(t *testing.T) {
tests := []struct {
name string
given []string
want Exclusions
}{
{name: "nothing configured"},
{name: "root-anchored does not reach inside", given: []string{"./nested/marker.txt"}},
{name: "one level down is anchored too", given: []string{"*/vendor"}},
{name: "any depth reaches inside", given: []string{"**/*.rpm"}, want: Exclusions{"**/*.rpm"}},
{name: "mixed keeps only the any-depth patterns", given: []string{"./a", "**/b", "*/c", "**/d"}, want: Exclusions{"**/b", "**/d"}},
// a trailing slash makes doublestar match nothing (issue #4839)
{name: "a trailing slash is trimmed", given: []string{"**/vendor/"}, want: Exclusions{"**/vendor"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, NewExclusions(tt.given))
})
}
}
func TestExclusions_Excludes(t *testing.T) {
exclusions := NewExclusions([]string{"**/vendor", "**/*.rpm"})
tests := []struct {
path string
want bool
}{
{"vendor", true},
{"a/vendor", true},
{"vendor/x/y.go", true},
{"a/vendor/y.go", true},
{"pkg.rpm", true},
{"deep/down/pkg.rpm", true},
{"keep/vendored.txt", false},
{"vendor-ish", false},
{"keep.txt", false},
{"PKG.RPM", false},
}
for _, tt := range tests {
t.Run(tt.path, func(t *testing.T) {
assert.Equal(t, tt.want, exclusions.Excludes(tt.path))
})
}
assert.False(t, NewExclusions(nil).Excludes("anything"))
assert.False(t, Exclusions{"["}.Excludes("keep.txt"), "a malformed pattern excludes nothing")
}
+263
View File
@@ -0,0 +1,263 @@
package archive
import (
"bytes"
"context"
"crypto"
"errors"
"fmt"
"io"
"path"
"github.com/mholt/archives"
"github.com/anchore/syft/internal"
intFile "github.com/anchore/syft/internal/file"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/file"
)
// ErrUnsupportedFormat is returned for an archive in a format that is not extracted (see supportedExtractor).
var ErrUnsupportedFormat = errors.New("archive format is not supported")
// Extract opens the archive read from r as its own filesystem, holding its bytes and entries within
// the limiter's bounds. It returns nil when the content is not an archive, ErrUnsupportedFormat when it
// is one that is not extracted, and ErrDiskLimitReached when the archive's own bytes cannot be placed
// within the disk limit. The caller must Cleanup the result.
//
// archiveCoordinates are those of the archive file itself; every location the resolver returns
// extends them with its entry path (see file.Coordinates).
func Extract(ctx context.Context, r io.Reader, archiveCoordinates file.Coordinates, limiter *Limiter, exclusions Exclusions) (*Resolver, error) {
charge := limiter.charge()
resolver := newResolver(ctx, archiveCoordinates, charge)
archivePath := archiveCoordinates.FullNestedPath()
// deferred so that a panic in a decoder still gives back the charge and removes any spill file
succeeded := false
defer func() {
if !succeeded {
resolver.Cleanup()
}
}()
content, release, err := resolver.acquire(r)
if err != nil {
return nil, err
}
// the archive's own bytes are needed only until they are extracted and digested
defer release()
format, err := identifyFormat(ctx, path.Base(archiveCoordinates.LocalPath()), content)
if format == nil || err != nil {
return nil, err
}
if err := resolver.extract(ctx, format, content, exclusions); err != nil {
return nil, fmt.Errorf("unable to extract archive %q: %w", archivePath, err)
}
resolver.Digests = digestsOf(ctx, content, archivePath)
succeeded = true
return resolver, nil
}
// acquire makes the archive's own bytes random-access, which identifying, extracting and digesting
// them needs. Bytes that already are, such as a file on disk or an entry of a parent archive, are
// read in place and cost nothing. Anything else is held within the limits like an entry, and the
// returned release gives back what memory it takes; bytes spilled to disk stay until Cleanup.
// Reaching the disk limit yields ErrDiskLimitReached.
func (r *Resolver) acquire(content io.Reader) (ReaderAtSeeker, func(), error) {
if ras, ok := content.(ReaderAtSeeker); ok {
return ras, func() {}, nil
}
var b blob
if err := r.put(&b, content); err != nil {
if errors.Is(err, ErrDiskLimitReached) {
return nil, nil, err
}
return nil, nil, fmt.Errorf("unable to read archive content: %w", err)
}
return r.open(&b), func() { r.discard(&b) }, nil
}
// identifyFormat returns the format able to extract this content, or nil when it is not an archive.
//
// Identification from the head of the stream wins: a tar holding one jar ends in that jar's central
// directory, and reading the tar as a zip would lose the rest. The end of the stream is consulted
// only when the head identifies nothing supported, which is how a zip appended to a launcher script (a
// Spring Boot executable jar) is found, and how a java resource adapter (a zip named .rar) is read.
func identifyFormat(ctx context.Context, name string, content ReaderAtSeeker) (archives.Extractor, error) {
if _, err := content.Seek(0, io.SeekStart); err != nil {
return nil, nil
}
format, _, err := intFile.IdentifyArchive(ctx, name, content)
if err == nil {
if extractor := supportedExtractor(format); extractor != nil {
return extractor, nil
}
}
if HasZipEndOfCentralDirectory(content) {
return archives.Zip{}, nil
}
if err == nil && holdsFiles(format) {
return nil, fmt.Errorf("%w: %s", ErrUnsupportedFormat, format.MediaType())
}
return nil, nil
}
// holdsFiles reports whether the format is an archive of files rather than a bare compression format,
// such as a gzipped man page, which holds one stream and is not cataloged as an archive.
func holdsFiles(format archives.Format) bool {
switch f := format.(type) {
case archives.CompressedArchive:
return f.Extraction != nil
case archives.Extractor:
return true
}
return false
}
// supportedExtractor admits only the zip and tar families, the tar family optionally compressed. Other
// formats mholt identifies, such as 7z and rar, pull in decoders that have not been exercised against
// untrusted input here, so they are not opened until they are deliberately supported. A bare
// compression format, such as a gzipped file that is not a tar, is not extractable either.
func supportedExtractor(format archives.Format) archives.Extractor {
switch f := format.(type) {
case archives.Zip:
return f
case archives.Tar:
return f
case archives.CompressedArchive:
if _, ok := f.Extraction.(archives.Tar); ok {
return f
}
}
return nil
}
// extract adds every entry of the archive. Reaching the disk limit stops the walk and marks the
// resolver truncated rather than failing.
func (r *Resolver) extract(ctx context.Context, format archives.Extractor, content ReaderAtSeeker, exclusions Exclusions) error {
if _, err := content.Seek(0, io.SeekStart); err != nil {
return err
}
var source io.Reader = content
if ca, ok := format.(archives.CompressedArchive); ok && ca.Compression != nil {
// a compressed tar decompresses everything to reach each header, including entries skipped whole
// and whatever an entry leaves unread, so the budget is charged for every byte the decoder
// produces rather than for what is kept. A zip or a plain tar seeks past what it skips
rc, err := ca.OpenReader(content)
if err != nil {
return err
}
defer internal.CloseAndLogError(rc, "decompressor")
source, format, r.metered = &meteredReader{ctx: ctx, reader: rc, budget: r.budget}, ca.Extraction, true
}
err := format.Extract(ctx, source, func(_ context.Context, f archives.FileInfo) error {
hdr, ok := r.entryHeader(f)
if !ok || exclusions.Excludes(hdr.Name) {
return nil
}
if !f.Mode().IsRegular() {
return r.add(hdr, nil)
}
entry, err := f.Open()
if err != nil {
log.WithFields("entry", hdr.Name, "error", err).Trace("unable to open archive entry, skipping it")
return nil
}
defer func() {
if err := entry.Close(); err != nil {
log.WithFields("entry", hdr.Name, "error", err).Trace("unable to close archive entry")
}
}()
return r.add(hdr, entry)
})
r.finish()
if errors.Is(err, ErrDiskLimitReached) {
r.truncate(fmt.Errorf("extraction stopped: %w", err))
return nil
}
if errors.Is(err, errBudgetSpent) {
r.truncate(fmt.Errorf("extraction stopped, possible decompression bomb: %w", err))
return nil
}
if err != nil && ctx.Err() == nil && r.entries > 0 {
// the stream broke partway (a truncated .tar.gz); what was read before that is still good
r.truncate(fmt.Errorf("extraction stopped early: %w", err))
return nil
}
return err
}
// archiveDigestHashes are the digests taken of an archive as a whole: SHA-1, matching what the java
// cataloger reports for a jar.
var archiveDigestHashes = []crypto.Hash{crypto.SHA1}
// digestsOf digests the archive's own bytes while they are in hand. A failure is logged and returns
// nothing rather than discarding an archive that extracted fine.
func digestsOf(ctx context.Context, content ReaderAtSeeker, archivePath string) []file.Digest {
if _, err := content.Seek(0, io.SeekStart); err != nil {
log.WithFields("archive", archivePath, "error", err).Trace("unable to rewind archive content for digests")
return nil
}
digests, err := intFile.NewDigestsFromFile(ctx, io.NopCloser(content), archiveDigestHashes)
if err != nil {
log.WithFields("archive", archivePath, "error", err).Trace("unable to digest archive")
return nil
}
return digests
}
var (
// zipLocalFileHeaderSignature starts every entry in a zip.
zipLocalFileHeaderSignature = []byte{'P', 'K', 0x03, 0x04}
// zipEOCDSignature starts the end-of-central-directory record that closes every zip. Readers
// locate the central directory from it, which is why a zip with data prepended still opens.
zipEOCDSignature = []byte{'P', 'K', 0x05, 0x06}
)
const (
// maxAppendedArchiveStubBytes is how far into a file an appended zip's first entry is looked for. A
// Spring Boot launcher stub is about 9 KB.
maxAppendedArchiveStubBytes = 64 * 1024
// maxZipEOCDSearch is how far from the end the end-of-central-directory record can start: the 22
// byte record plus a comment of up to 65535 bytes.
maxZipEOCDSearch = 22 + 65535
)
// MayHideAnAppendedArchive reports whether a zip entry signature appears in the head of the stream,
// which a zip appended to a launcher stub does and a file that merely has an archive's extension
// almost never does. The returned reader replays what was consumed.
func MayHideAnAppendedArchive(r io.Reader) (io.Reader, bool) {
head := make([]byte, maxAppendedArchiveStubBytes)
n, err := io.ReadFull(r, head)
if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) {
return r, false
}
head = head[:n]
found := bytes.Contains(head, zipLocalFileHeaderSignature)
if seeker, ok := r.(io.Seeker); ok {
if _, err := seeker.Seek(0, io.SeekStart); err == nil {
return r, found
}
}
return io.MultiReader(bytes.NewReader(head), r), found
}
// HasZipEndOfCentralDirectory reports whether the content ends in a zip end-of-central-directory
// record, the one part of a zip whose position is known without parsing it.
func HasZipEndOfCentralDirectory(r ReaderAtSeeker) bool {
size, err := r.Seek(0, io.SeekEnd)
if err != nil || size < int64(len(zipEOCDSignature)) {
return false
}
window := min(size, int64(maxZipEOCDSearch))
buf := make([]byte, window)
if _, err := r.ReadAt(buf, size-window); err != nil && !errors.Is(err, io.EOF) {
return false
}
return bytes.Contains(buf, zipEOCDSignature)
}
+706
View File
@@ -0,0 +1,706 @@
package archive
import (
"archive/tar"
"archive/zip"
"bytes"
"compress/gzip"
"context"
"fmt"
"io"
"io/fs"
"math/rand"
"os"
"path/filepath"
"strings"
"testing"
"github.com/mholt/archives"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
stereoscopeFile "github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/syft/file"
)
var sampleFiles = map[string]string{
"file1.txt": "content1",
"dir/file2.txt": "content2",
}
func Test_identifyFormat(t *testing.T) {
ctx := context.Background()
launcher := append([]byte("#!/bin/bash\nexec java -jar \"$0\" \"$@\"\nexit 0\n"), zipBytes(t, sampleFiles)...)
tests := []struct {
name string
data []byte
want archives.Extractor
unsupported bool
}{
{"test.zip", zipBytes(t, sampleFiles), archives.Zip{}, false},
{"test.tar.gz", tarGzBytes(t, sampleFiles), archives.CompressedArchive{}, false},
{"plain.txt", []byte("just text"), nil, false},
// identified by mholt, but not opened until their decoders are deliberately supported
{"test.7z", []byte("7z\xbc\xaf\x27\x1c\x00\x04"), nil, true},
{"test.rar", []byte("Rar!\x1a\x07\x01\x00"), nil, true},
// a bare compression format holds one stream, not files
{"page.1.gz", gzBytes(t, "a man page"), nil, false},
// a java resource adapter archive is a zip that shares rar's extension
{"adapter.rar", zipBytes(t, sampleFiles), archives.Zip{}, false},
// the name can be claimed by identification, so the probe alone must not admit prose
{"app.jar", []byte("plain text, no central directory anywhere in it"), nil, false},
{"bundle", []byte("plain text, no central directory anywhere in it"), nil, false},
// content sniffing types this as a shell script; only its tail says zip
{"app.jar", launcher, archives.Zip{}, false},
}
for _, tt := range tests {
t.Run(fmt.Sprintf("%s/%d bytes", tt.name, len(tt.data)), func(t *testing.T) {
got, err := identifyFormat(ctx, tt.name, bytes.NewReader(tt.data))
if tt.unsupported {
assert.ErrorIs(t, err, ErrUnsupportedFormat)
} else {
assert.NoError(t, err)
}
if tt.want == nil {
assert.Nil(t, got)
return
}
assert.IsType(t, tt.want, got)
})
}
}
func TestResolver_extract_storesEveryEntryInMemoryWhenUnbounded(t *testing.T) {
for name, data := range map[string][]byte{
"test.zip": zipBytes(t, sampleFiles),
"test.tar.gz": tarGzBytes(t, sampleFiles),
} {
t.Run(name, func(t *testing.T) {
r, root := extractBytes(t, data, name, nil)
assert.NoError(t, r.Truncated)
assert.Zero(t, r.written)
assert.Empty(t, filesIn(t, root))
entries := readStore(t, r)
assert.Equal(t, "content1", entries["file1.txt"].body)
assert.Equal(t, "content2", entries["dir/file2.txt"].body)
})
}
}
func TestResolver_extract_diskLimitTruncatesAtTheEntryThatDoesNotFit(t *testing.T) {
// each entry costs its index estimate plus its content on disk
record := approxIndexBytes(tar.Header{Name: "file1.txt"})
data := zipBytes(t, map[string]string{"file1.txt": "a", "file2.txt": "b", "file3.txt": "c"})
r, _ := extractBytes(t, data, "test.zip", diskCharge(2*(record+1)+record))
assert.Error(t, r.Truncated)
assert.Equal(t, []string{"file1.txt", "file2.txt"}, storedNames(t, r), "the refused entry is not stored")
entries := readStore(t, r)
assert.Equal(t, "a", entries["file1.txt"].body)
assert.Equal(t, "b", entries["file2.txt"].body)
}
func TestResolver_extract_diskLimitBitesMidEntry(t *testing.T) {
// a tar is walked entry by entry, so the limit can bite part way through an entry
data := tarGzBytes(t, map[string]string{
"a/first.txt": strings.Repeat("a", 40*1024),
"b/second.txt": strings.Repeat("b", 40*1024),
})
const budget = 80 * 1024
charge := diskCharge(budget)
r, root := extractBytes(t, data, "test.tar.gz", charge)
assert.Error(t, r.Truncated)
assert.Equal(t, []string{"a", "a/first.txt", "b"}, storedNames(t, r), "the directory a refused entry implied stays")
assert.Greater(t, r.written, int64(40*1024), "the first entry landed whole and the second part way")
_, disk := charge.held()
assert.LessOrEqual(t, disk, int64(budget))
assert.Greater(t, disk, r.written, "the index estimate is charged to disk too")
info, err := os.Stat(filepath.Join(root, spillFile(t, root)))
require.NoError(t, err)
assert.Equal(t, r.written, info.Size())
}
func TestResolver_extract_directoryOnlyArchivesAreBoundedByTheIndexEstimate(t *testing.T) {
var entries []testEntry
for i := range 50 {
entries = append(entries, testEntry{name: fmt.Sprintf("d%04d/", i), dir: true})
}
room := approxIndexBytes(tar.Header{Name: "d0000"}) * 10
for name, data := range map[string][]byte{
"dirs.zip": zipFromEntries(t, entries),
"dirs.tar.gz": tarGzFromEntries(t, entries),
} {
t.Run(name, func(t *testing.T) {
r, _ := extractBytes(t, data, name, diskCharge(room))
assert.Error(t, r.Truncated)
assert.Len(t, storedNames(t, r), 10)
})
}
}
func TestResolver_extract_entryNamesAreCleanedAndNothingIsWrittenUnderThem(t *testing.T) {
entries := []testEntry{
{name: "../../etc/passwd", body: "evil"},
{name: "/etc/hosts", body: "pwned"},
{name: "META-INF/", dir: true, mode: 0o555},
{name: "META-INF/MANIFEST.MF", body: "Manifest-Version: 1.0"},
{name: "passwd", link: "../../../../etc/passwd"},
{name: "shadow", link: "/etc/shadow"},
{name: "bin/link.txt", link: "../lib/real.txt"},
}
for name, data := range map[string][]byte{
"evil.zip": zipFromEntries(t, entries),
"evil.tar.gz": tarGzFromEntries(t, entries),
} {
t.Run(name, func(t *testing.T) {
r, root := extractBytes(t, data, name, nil)
require.NoError(t, r.Truncated)
assert.Equal(t, []string{"META-INF", "META-INF/MANIFEST.MF", "bin", "bin/link.txt", "etc", "etc/hosts", "etc/passwd", "passwd", "shadow"},
storedNames(t, r))
assert.Empty(t, filesIn(t, root), "entries live in the store, never under their own names")
assert.NoFileExists(t, filepath.Join(t.TempDir(), "..", "etc", "passwd"))
got := readStore(t, r)
assert.Equal(t, "evil", got["etc/passwd"].body)
assert.Equal(t, "pwned", got["etc/hosts"].body)
assert.Equal(t, stereoscopeFile.TypeDirectory, got["META-INF"].kind)
assert.Equal(t, fs.FileMode(0o555), got["META-INF"].mode.Perm())
assert.Equal(t, "Manifest-Version: 1.0", got["META-INF/MANIFEST.MF"].body)
// links are headers carrying their target verbatim; the index resolves them inside the archive
for _, link := range []string{"passwd", "shadow", "bin/link.txt"} {
assert.Equal(t, stereoscopeFile.TypeSymLink, got[link].kind)
assert.Empty(t, got[link].body)
}
assert.Equal(t, "../../../../etc/passwd", got["passwd"].linkTarget)
assert.Equal(t, "/etc/shadow", got["shadow"].linkTarget)
assert.Equal(t, "../lib/real.txt", got["bin/link.txt"].linkTarget)
})
}
}
func TestResolver_extract_dataEdgeCases(t *testing.T) {
t.Run("an empty archive is not an error", func(t *testing.T) {
r, _ := extractBytes(t, zipBytes(t, map[string]string{}), "test.zip", nil)
assert.Zero(t, r.entries)
assert.NoError(t, r.Truncated)
})
t.Run("a zero-byte entry costs no disk", func(t *testing.T) {
r, _ := extractBytes(t, zipBytes(t, map[string]string{"empty.txt": ""}), "test.zip", memCharge(0))
assert.Equal(t, []string{"empty.txt"}, storedNames(t, r))
assert.Zero(t, r.written)
assert.Empty(t, readStore(t, r)["empty.txt"].body)
})
t.Run("unicode entry names survive", func(t *testing.T) {
names := map[string]string{
"café/naïve.txt": "accents",
"日本語/ファイル.txt": "japanese",
"emoji-🎉.txt": "emoji",
}
r, _ := extractBytes(t, zipBytes(t, names), "test.zip", nil)
entries := readStore(t, r)
for name, want := range names {
assert.Equal(t, want, entries[name].body, name)
}
})
t.Run("an entry name with too many path components is skipped and reported", func(t *testing.T) {
atCap := strings.Repeat("a/", maxEntryNameDepth-1) + "f"
data := zipBytes(t, map[string]string{
atCap: "at the cap",
strings.Repeat("b/", maxEntryNameDepth) + "f": "over the cap",
})
r, _ := extractBytes(t, data, "test.zip", nil)
names := storedNames(t, r)
assert.Contains(t, names, atCap)
assert.Len(t, names, maxEntryNameDepth, "only the entry at the cap and its implied directories")
assert.ErrorContains(t, r.Truncated, "path components")
})
t.Run("an entry whose chain is longer than a path is skipped and reported", func(t *testing.T) {
// the chain is the archive's own path, the separator, then the name
fits := strings.Repeat("a", MaxPathBytes-len("test-archive!/"))
data := zipBytes(t, map[string]string{
fits: "at the cap",
fits + "b": "over the cap",
})
r, _ := extractBytes(t, data, "test.zip", nil)
assert.Equal(t, []string{fits}, storedNames(t, r))
assert.ErrorContains(t, r.Truncated, "archive chain")
})
t.Run("an over-long entry name is skipped", func(t *testing.T) {
data := zipBytes(t, map[string]string{
strings.Repeat("a/", maxEntryNameBytes) + "f": "deep",
"ok.txt": "fine",
})
r, _ := extractBytes(t, data, "test.zip", nil)
assert.Equal(t, []string{"ok.txt"}, storedNames(t, r))
})
}
func TestResolver_extract_unsafeEntryNamesAreRefusedAndReported(t *testing.T) {
jar := string(zipBytes(t, map[string]string{"META-INF/MANIFEST.MF": "real"}))
tests := []struct {
name string
forged string
}{
{name: "archive path separator", forged: "a.jar!/META-INF/MANIFEST.MF"},
{name: "line break", forged: "x\nlayerID: sha256:forged"},
{name: "unicode line separator", forged: "x\u2028y"},
{name: "C1 control", forged: "x\u0085y"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
data := zipBytes(t, map[string]string{"a.jar": jar, tt.forged: "forged"})
r, _ := extractBytes(t, data, "outer.zip", nil)
assert.Equal(t, []string{"a.jar"}, storedNames(t, r))
require.Error(t, r.Truncated)
assert.Contains(t, r.Truncated.Error(), fmt.Sprintf("%q", tt.forged))
})
}
}
func TestResolver_extract_excludedEntriesAreNeverStored(t *testing.T) {
data := zipBytes(t, map[string]string{
"keep.txt": "kept",
"vendor/lib.go": "excluded by directory",
"deep/down/pkg.rpm": "excluded by name",
})
limiter := NewLimiter(Limits{MaxMemoryBytes: 1 << 20, MaxDiskBytes: -1})
r, _ := resolverIn(t, limiter.charge())
content := bytes.NewReader(data)
format, err := identifyFormat(context.Background(), "test.zip", content)
require.NoError(t, err)
err = r.extract(context.Background(), format, content,
NewExclusions([]string{"**/vendor", "**/*.rpm"}))
require.NoError(t, err)
require.NoError(t, r.Truncated)
assert.Equal(t, []string{"keep.txt"}, storedNames(t, r))
mem, _ := limiter.InUse()
assert.Equal(t, approxIndexBytes(tar.Header{Name: "keep.txt"})+4, mem, "an excluded entry costs nothing")
}
// a compressed tar reader decompresses a skipped entry to reach the next header, so skipping must still
// spend the budget or an excluded or oversized entry would decompress without bound
func TestResolver_extract_skippedCompressedTarBytesSpendTheBudget(t *testing.T) {
big := strings.Repeat("\x00", 4<<20)
tests := []struct {
name string
exclusions []string
entryCap int64
}{
{name: "excluded entry", exclusions: []string{"**/*.bin"}, entryCap: maxEntryBytes},
{name: "entry over the size cap", entryCap: 1 << 10},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
defer func(prev int64) { maxEntryBytes = prev }(maxEntryBytes)
maxEntryBytes = tt.entryCap
data := tarGzFromEntries(t, []testEntry{{name: "bomb.bin", body: big}, {name: "after.txt", body: "after"}})
r, _ := resolverIn(t, memCharge(1<<30))
r.budget = NewBudget(1 << 20)
content := bytes.NewReader(data)
format, err := identifyFormat(context.Background(), "test.tar.gz", content)
require.NoError(t, err)
require.NoError(t, r.extract(context.Background(), format, content, NewExclusions(tt.exclusions)))
require.Error(t, r.Truncated)
assert.Empty(t, storedNames(t, r), "the walk stopped at the entry that spent the budget")
assert.Zero(t, r.budget.remaining)
})
}
}
// a body behind a header that doesn't read as a regular file (here a socket per its mode bits) is never
// opened, but the tar reader still decompresses it to reach the next header
func TestResolver_extract_bodiesOfNonRegularTarEntriesSpendTheBudget(t *testing.T) {
var buf bytes.Buffer
gw := gzip.NewWriter(&buf)
tw := tar.NewWriter(gw)
body := strings.Repeat("\x00", 4<<20)
require.NoError(t, tw.WriteHeader(&tar.Header{Name: "sock", Typeflag: tar.TypeReg, Mode: 0o140644, Size: int64(len(body))}))
_, err := tw.Write([]byte(body))
require.NoError(t, err)
require.NoError(t, tw.WriteHeader(&tar.Header{Name: "after.txt", Typeflag: tar.TypeReg, Mode: 0o644, Size: 5}))
_, err = tw.Write([]byte("after"))
require.NoError(t, err)
require.NoError(t, tw.Close())
require.NoError(t, gw.Close())
r, _ := resolverIn(t, memCharge(1<<30))
r.budget = NewBudget(1 << 20)
content := bytes.NewReader(buf.Bytes())
format, err := identifyFormat(context.Background(), "test.tar.gz", content)
require.NoError(t, err)
require.NoError(t, r.extract(context.Background(), format, content, nil))
assert.ErrorIs(t, r.Truncated, errBudgetSpent)
assert.Empty(t, storedNames(t, r))
assert.Zero(t, r.budget.remaining)
}
// a zip or a plain tar seeks past what it skips, so an excluded entry there costs nothing
func TestResolver_extract_skippedSeekableBytesAreFree(t *testing.T) {
entries := []testEntry{{name: "bomb.bin", body: strings.Repeat("\x00", 4<<20)}, {name: "after.txt", body: "after"}}
var plainTar bytes.Buffer
tw := tar.NewWriter(&plainTar)
for _, e := range entries {
hdr := e.header()
require.NoError(t, tw.WriteHeader(&hdr))
_, err := tw.Write([]byte(e.body))
require.NoError(t, err)
}
require.NoError(t, tw.Close())
for name, data := range map[string][]byte{"test.zip": zipFromEntries(t, entries), "test.tar": plainTar.Bytes()} {
t.Run(name, func(t *testing.T) {
r, _ := resolverIn(t, memCharge(1<<30))
r.budget = NewBudget(1 << 20)
content := bytes.NewReader(data)
format, err := identifyFormat(context.Background(), name, content)
require.NoError(t, err)
require.NoError(t, r.extract(context.Background(), format, content, NewExclusions([]string{"**/*.bin"})))
require.NoError(t, r.Truncated)
assert.Equal(t, []string{"after.txt"}, storedNames(t, r))
})
}
}
func TestExtract_readsEntriesAndDigestsTheArchive(t *testing.T) {
data := zipBytes(t, map[string]string{"dir/hello.txt": "hello world"})
extracted, err := Extract(context.Background(), bytes.NewReader(data), file.Coordinates{RealPath: "app.war", FileSystemID: "parentFS", FullPath: "app.war!/some/path/app.zip"}, nil, nil)
require.NoError(t, err)
require.NotNil(t, extracted)
t.Cleanup(extracted.Cleanup)
assert.NoError(t, extracted.Truncated)
require.Len(t, extracted.Digests, 1)
assert.Equal(t, "sha1", extracted.Digests[0].Algorithm)
locations, err := extracted.FilesByGlob("**/*.txt")
require.NoError(t, err)
require.Len(t, locations, 1)
assert.Equal(t, file.Coordinates{RealPath: "app.war", FileSystemID: "parentFS", FullPath: "app.war!/some/path/app.zip!/dir/hello.txt"},
locations[0].Coordinates)
reader, err := extracted.FileContentsByLocation(locations[0])
require.NoError(t, err)
body, err := io.ReadAll(reader)
require.NoError(t, err)
assert.Equal(t, "hello world", string(body))
}
func TestExtract_heldContentIsReleasedOnceExtracted(t *testing.T) {
// the archive's own bytes are only needed to extract and digest it; afterwards the limiter holds
// the entries and the index estimate alone
data := zipBytes(t, map[string]string{"hello.txt": "hi"})
limiter := NewLimiter(Limits{MaxMemoryBytes: -1, MaxDiskBytes: -1})
extracted, err := Extract(context.Background(), unseekable{bytes.NewReader(data)}, file.NewCoordinates("app.zip", ""), limiter, nil)
require.NoError(t, err)
t.Cleanup(extracted.Cleanup)
mem, disk := limiter.InUse()
assert.Equal(t, approxIndexBytes(tar.Header{Name: "hello.txt"})+2, mem)
assert.Zero(t, disk)
}
func TestExtract_heldContentWrittenToDiskStaysUntilCleanup(t *testing.T) {
// only a top-level stream is ever held; a nested archive is read out of its parent, which is held
// for the whole descent anyway, so spilled bytes wait for Cleanup with the entries
ctx, root := scanContext(t)
data := zipBytes(t, map[string]string{"hello.txt": "hi"})
limiter := NewLimiter(Limits{MaxMemoryBytes: 0, MaxDiskBytes: -1})
extracted, err := Extract(ctx, unseekable{bytes.NewReader(data)}, file.NewCoordinates("app.zip", ""), limiter, nil)
require.NoError(t, err)
t.Cleanup(extracted.Cleanup)
require.Len(t, filesIn(t, root), 1, "the archive's bytes and its entries share one file")
_, disk := limiter.InUse()
assert.Equal(t, approxIndexBytes(tar.Header{Name: "hello.txt"})+2+int64(len(data)), disk)
extracted.Cleanup()
assert.Empty(t, filesIn(t, root))
_, disk = limiter.InUse()
assert.Zero(t, disk)
}
func TestExtract_aTruncatedStreamKeepsWhatWasRead(t *testing.T) {
// a .tar.gz cut off partway breaks the stream for every entry after the cut; the ones before it are
// still good and must not be thrown away
noise := make([]byte, 64*1024)
rand.New(rand.NewSource(1)).Read(noise)
data := tarGzBytes(t, map[string]string{"a.txt": "first", "b.bin": string(noise)})
cut := data[:len(data)/2]
extracted, err := Extract(context.Background(), bytes.NewReader(cut), file.NewCoordinates("app.tar.gz", ""), NewLimiter(unboundedLimits), nil)
require.NoError(t, err)
require.NotNil(t, extracted)
t.Cleanup(extracted.Cleanup)
assert.Error(t, extracted.Truncated)
locations, err := extracted.FilesByGlob("**")
require.NoError(t, err)
assert.Equal(t, []string{"/a.txt"}, localPaths(locations))
}
func TestExtract_aPanicStillReleasesEverything(t *testing.T) {
// a decoder panicking on hostile bytes must not leak the charge or the spill file
ctx, root := scanContext(t)
limiter := NewLimiter(Limits{MaxMemoryBytes: 0, MaxDiskBytes: -1})
require.Panics(t, func() {
_, _ = Extract(ctx, &panicsAfter{n: 4096}, file.NewCoordinates("app.zip", ""), limiter, nil)
})
mem, disk := limiter.InUse()
assert.Zero(t, mem)
assert.Zero(t, disk)
assert.Empty(t, filesIn(t, root))
}
// panicsAfter hands out n zero bytes, then panics.
type panicsAfter struct{ n int }
func (p *panicsAfter) Read(b []byte) (int, error) {
if p.n <= 0 {
panic("decoder blew up")
}
n := min(len(b), p.n)
clear(b[:n])
p.n -= n
return n, nil
}
func TestExtract_archiveBytesThatCannotBePlacedAreSkipped(t *testing.T) {
data := zipBytes(t, sampleFiles)
limiter := NewLimiter(Limits{MaxMemoryBytes: 0, MaxDiskBytes: 0})
_, err := Extract(context.Background(), unseekable{bytes.NewReader(data)}, file.NewCoordinates("app.zip", ""), limiter, nil)
assert.ErrorIs(t, err, ErrDiskLimitReached)
// read in place, the same bytes cost nothing; only the entries are refused
extracted, err := Extract(context.Background(), bytes.NewReader(data), file.NewCoordinates("app.zip", ""), limiter, nil)
require.NoError(t, err)
require.NotNil(t, extracted)
t.Cleanup(extracted.Cleanup)
assert.Error(t, extracted.Truncated)
locations, err := extracted.FilesByGlob("**/*")
require.NoError(t, err)
assert.Empty(t, locations)
}
func Test_HasZipEndOfCentralDirectory(t *testing.T) {
tests := []struct {
name string
data []byte
want bool
}{
{"a plain zip", zipBytes(t, sampleFiles), true},
{"a zip behind a launcher script", append([]byte("#!/bin/bash\nexit 0\n"), zipBytes(t, sampleFiles)...), true},
{"prose", []byte("this is not an archive, it is a sentence"), false},
{"a tar.gz", tarGzBytes(t, sampleFiles), false},
{"empty", nil, false},
{"shorter than the signature", []byte("PK"), false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, HasZipEndOfCentralDirectory(bytes.NewReader(tt.data)))
})
}
}
func Test_MayHideAnAppendedArchive(t *testing.T) {
launcher := append([]byte("#!/bin/bash\nexit 0\n"), zipBytes(t, sampleFiles)...)
prose := []byte("plain text, whatever the extension claims")
t.Run("a seekable reader is rewound and returned as is", func(t *testing.T) {
reader := bytes.NewReader(launcher)
got, found := MayHideAnAppendedArchive(reader)
assert.True(t, found)
assert.Same(t, reader, got)
rest, err := io.ReadAll(got)
require.NoError(t, err)
assert.Equal(t, launcher, rest)
})
t.Run("an unseekable reader replays what was consumed", func(t *testing.T) {
got, found := MayHideAnAppendedArchive(unseekable{bytes.NewReader(launcher)})
assert.True(t, found)
rest, err := io.ReadAll(got)
require.NoError(t, err)
assert.Equal(t, launcher, rest)
})
t.Run("prose hides nothing", func(t *testing.T) {
got, found := MayHideAnAppendedArchive(unseekable{bytes.NewReader(prose)})
assert.False(t, found)
rest, err := io.ReadAll(got)
require.NoError(t, err)
assert.Equal(t, prose, rest)
})
}
// zeros is a reader of n zero bytes: the most compressible content there is.
func zeros(n int64) io.Reader {
return io.LimitReader(zeroSource{}, n)
}
type zeroSource struct{}
func (zeroSource) Read(p []byte) (int, error) {
clear(p)
return len(p), nil
}
func TestExtract_decompressionBombIsBoundedByTheLimits(t *testing.T) {
// 64 MiB of zeros compresses to a few hundred KiB. The declared size is never trusted: bytes are
// charged as they land, so the limits hold whatever the archive claims
const inflated = 64 << 20
limits := Limits{MaxMemoryBytes: 1 << 20, MaxDiskBytes: 4 << 20}
var zipBomb bytes.Buffer
zw := zip.NewWriter(&zipBomb)
w, err := zw.Create("bomb.bin")
require.NoError(t, err)
_, err = io.Copy(w, zeros(inflated))
require.NoError(t, err)
require.NoError(t, zw.Close())
var tarBomb bytes.Buffer
gw := gzip.NewWriter(&tarBomb)
tw := tar.NewWriter(gw)
require.NoError(t, tw.WriteHeader(&tar.Header{Name: "bomb.bin", Mode: 0o644, Size: inflated, Typeflag: tar.TypeReg}))
_, err = io.Copy(tw, zeros(inflated))
require.NoError(t, err)
require.NoError(t, tw.Close())
require.NoError(t, gw.Close())
for name, data := range map[string][]byte{"bomb.zip": zipBomb.Bytes(), "bomb.tar.gz": tarBomb.Bytes()} {
t.Run(name, func(t *testing.T) {
require.Less(t, len(data), 1<<20, "the fixture must be a bomb: small on disk, huge inflated")
ctx, root := scanContext(t)
limiter := NewLimiter(limits)
r, err := Extract(ctx, bytes.NewReader(data), file.NewCoordinates(name, ""), limiter, nil)
require.NoError(t, err)
require.NotNil(t, r)
t.Cleanup(r.Cleanup)
assert.Error(t, r.Truncated)
locations, err := r.FilesByGlob("**/*")
require.NoError(t, err)
assert.Empty(t, locations, "the entry that did not fit is not cataloged in part")
peakMemory, peakDisk := limiter.Peak()
assert.LessOrEqual(t, peakMemory, limits.MaxMemoryBytes)
assert.LessOrEqual(t, peakDisk, limits.MaxDiskBytes)
r.Cleanup()
assert.Empty(t, filesIn(t, root))
})
}
}
func TestExtract_entryCountBombIsBoundedByTheMemoryLimit(t *testing.T) {
// every entry is a node in memory whatever the disk limit says, so a tiny archive of many entries
// is bounded by the memory limit alone
limits := Limits{MaxMemoryBytes: 1 << 20, MaxDiskBytes: -1}
limiter := NewLimiter(limits)
r, err := Extract(context.Background(), bytes.NewReader(manyEntryZip(t, 20_000)), file.NewCoordinates("many.zip", ""), limiter, nil)
require.NoError(t, err)
t.Cleanup(r.Cleanup)
assert.Error(t, r.Truncated)
assert.Less(t, r.entries, 20_000)
assert.LessOrEqual(t, int64(nodeCount(r))*approxIndexBytesPerEntry, limits.MaxMemoryBytes+approxIndexBytesPerEntry,
"nodes held must not exceed what the memory limit paid for")
peakMemory, _ := limiter.Peak()
assert.LessOrEqual(t, peakMemory, limits.MaxMemoryBytes)
}
func TestExtract_deepPathBombIsBoundedByTheMemoryLimit(t *testing.T) {
// 100 entries whose names each imply 250 distinct directories would be 25,000 nodes; the implied
// directories are charged as nodes, so the memory limit bounds them too
files := map[string]string{}
for i := range 100 {
var b strings.Builder
for j := range maxEntryNameDepth - 6 {
fmt.Fprintf(&b, "%03d-%03d/", i, j)
}
b.WriteString("f")
require.LessOrEqual(t, b.Len(), maxEntryNameBytes)
files[b.String()] = "x"
}
limits := Limits{MaxMemoryBytes: 8 << 20, MaxDiskBytes: -1}
limiter := NewLimiter(limits)
r, err := Extract(context.Background(), bytes.NewReader(zipBytes(t, files)), file.NewCoordinates("deep.zip", ""), limiter, nil)
require.NoError(t, err)
t.Cleanup(r.Cleanup)
assert.Error(t, r.Truncated)
assert.LessOrEqual(t, int64(nodeCount(r))*approxIndexBytesPerEntry, limits.MaxMemoryBytes+approxIndexBytesPerEntry)
peakMemory, _ := limiter.Peak()
assert.LessOrEqual(t, peakMemory, limits.MaxMemoryBytes)
}
func TestExtract_locationsKeepTheTopLevelRealPath(t *testing.T) {
data := zipBytes(t, map[string]string{"dir/hello.txt": "hello world"})
// a top-level archive's coordinates, with FullPath unset or set to RealPath, name entries alike
for _, archive := range []file.Coordinates{
{RealPath: "/app.zip"},
{RealPath: "/app.zip", FullPath: "/app.zip"},
} {
extracted, err := Extract(context.Background(), bytes.NewReader(data), archive, nil, nil)
require.NoError(t, err)
require.NotNil(t, extracted)
t.Cleanup(extracted.Cleanup)
locations, err := extracted.FilesByPath("dir/hello.txt")
require.NoError(t, err)
require.Len(t, locations, 1)
assert.Equal(t, file.Coordinates{RealPath: "/app.zip", FullPath: "/app.zip!/dir/hello.txt"}, locations[0].Coordinates)
assert.Equal(t, "/dir/hello.txt", locations[0].LocalPath())
for _, loc := range []file.Location{
locations[0],
file.NewLocation("dir/hello.txt"),
file.NewLocation("/dir/hello.txt"),
file.NewLocationFromCoordinates(file.Coordinates{RealPath: "/app.zip", FullPath: "/app.zip!/dir/hello.txt"}),
} {
reader, err := extracted.FileContentsByLocation(loc)
require.NoError(t, err, "location %s", loc)
body, err := io.ReadAll(reader)
require.NoError(t, err)
assert.Equal(t, "hello world", string(body))
metadata, err := extracted.FileMetadataByLocation(loc)
require.NoError(t, err, "location %s", loc)
assert.Equal(t, "/app.zip!/dir/hello.txt", metadata.Path)
}
_, err = extracted.FileContentsByLocation(file.NewLocationFromCoordinates(file.Coordinates{RealPath: "/other.zip", FullPath: "/other.zip!/dir/hello.txt"}))
assert.Error(t, err, "an entry of another archive is not read from this one")
}
}
+344
View File
@@ -0,0 +1,344 @@
package archive
import (
"archive/tar"
"archive/zip"
"bytes"
"compress/gzip"
"context"
"fmt"
"io"
"io/fs"
"os"
"sort"
"testing"
"github.com/stretchr/testify/require"
stereoscopeFile "github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/internal/tmpdir"
"github.com/anchore/syft/syft/file"
)
// held reports what this charge is currently holding, in memory and on disk.
func (c *charge) held() (memory, disk int64) {
if c == nil {
return 0, 0
}
c.limiter.mu.Lock()
defer c.limiter.mu.Unlock()
return c.inMemory, c.onDisk
}
// heldInMemory reports the content the resolver is still holding in memory.
func (r *Resolver) heldInMemory() int64 {
var total int64
for _, b := range r.held {
total += int64(len(b.mem))
}
return total
}
// unseekable hides a reader's Seek and ReadAt, forcing content through the held path.
type unseekable struct{ io.Reader }
// testEntry is one entry of a fixture archive: a directory when dir is set, a link when link is set
// (hard when hard is set, otherwise symbolic), otherwise a regular file.
type testEntry struct {
name string
body string
link string
hard bool
dir bool
mode fs.FileMode
}
// header is the tar header this entry would carry.
func (e testEntry) header() tar.Header {
switch {
case e.dir:
return tar.Header{Name: e.name, Mode: int64(e.modeOr(0o755)), Typeflag: tar.TypeDir}
case e.link != "" && e.hard:
return tar.Header{Name: e.name, Mode: 0o644, Typeflag: tar.TypeLink, Linkname: e.link}
case e.link != "":
return tar.Header{Name: e.name, Mode: 0o777, Typeflag: tar.TypeSymlink, Linkname: e.link}
}
return tar.Header{Name: e.name, Mode: int64(e.modeOr(0o644)), Size: int64(len(e.body)), Typeflag: tar.TypeReg}
}
func fileEntries(files map[string]string) []testEntry {
names := make([]string, 0, len(files))
for name := range files {
names = append(names, name)
}
// limits bite as the walk proceeds, so entry order decides what lands before a truncation
sort.Strings(names)
entries := make([]testEntry, 0, len(names))
for _, name := range names {
entries = append(entries, testEntry{name: name, body: files[name]})
}
return entries
}
func zipBytes(t testing.TB, files map[string]string) []byte {
return zipFromEntries(t, fileEntries(files))
}
func zipFromEntries(t testing.TB, entries []testEntry) []byte {
t.Helper()
var buf bytes.Buffer
w := zip.NewWriter(&buf)
for _, e := range entries {
hdr := &zip.FileHeader{Name: e.name}
body := e.body
switch {
case e.dir:
hdr.SetMode(fs.ModeDir | e.modeOr(0o755))
case e.link != "":
hdr.SetMode(fs.ModeSymlink | 0o777)
body = e.link
default:
hdr.SetMode(e.modeOr(0o644))
}
fw, err := w.CreateHeader(hdr)
require.NoError(t, err)
_, err = fw.Write([]byte(body))
require.NoError(t, err)
}
require.NoError(t, w.Close())
return buf.Bytes()
}
func gzBytes(t testing.TB, body string) []byte {
t.Helper()
var buf bytes.Buffer
gw := gzip.NewWriter(&buf)
_, err := gw.Write([]byte(body))
require.NoError(t, err)
require.NoError(t, gw.Close())
return buf.Bytes()
}
func tarGzBytes(t testing.TB, files map[string]string) []byte {
return tarGzFromEntries(t, fileEntries(files))
}
func tarGzFromEntries(t testing.TB, entries []testEntry) []byte {
t.Helper()
var buf bytes.Buffer
gw := gzip.NewWriter(&buf)
tw := tar.NewWriter(gw)
for _, e := range entries {
hdr := e.header()
require.NoError(t, tw.WriteHeader(&hdr))
_, err := tw.Write([]byte(e.body))
require.NoError(t, err)
}
require.NoError(t, tw.Close())
require.NoError(t, gw.Close())
return buf.Bytes()
}
func (e testEntry) modeOr(fallback fs.FileMode) fs.FileMode {
if e.mode != 0 {
return e.mode
}
return fallback
}
func regularHeader(name string, size int64) tar.Header {
return tar.Header{Name: name, Size: size, Mode: 0o600, Typeflag: tar.TypeReg}
}
// scanContext returns a context whose temp root is a fresh directory, and that directory.
func scanContext(t testing.TB) (context.Context, string) {
t.Helper()
root := t.TempDir()
return tmpdir.WithValue(context.Background(), tmpdir.FromPath(root)), root
}
// filesIn lists the regular files directly in dir.
func filesIn(t testing.TB, dir string) []string {
t.Helper()
entries, err := os.ReadDir(dir)
require.NoError(t, err)
var names []string
for _, e := range entries {
if !e.IsDir() {
names = append(names, e.Name())
}
}
return names
}
// spillFile is the one file a resolver has written, or "" when it has not.
func spillFile(t testing.TB, dir string) string {
t.Helper()
files := filesIn(t, dir)
require.LessOrEqual(t, len(files), 1)
if len(files) == 0 {
return ""
}
return files[0]
}
// resolverIn makes an empty resolver over the given charge that spills into dir, along with that
// directory. Its entries are added by the test.
func resolverIn(t testing.TB, charge *charge) (*Resolver, string) {
t.Helper()
root := t.TempDir()
ctx := tmpdir.WithValue(context.Background(), tmpdir.FromPath(root))
r := newResolver(ctx, file.NewCoordinates("test-archive", ""), charge)
t.Cleanup(r.Cleanup)
return r, root
}
// addAll adds entries in the order given and finishes the resolver.
func addAll(t testing.TB, r *Resolver, entries []testEntry) {
t.Helper()
for _, e := range entries {
require.NoError(t, r.add(e.header(), bytes.NewReader([]byte(e.body))))
}
r.finish()
}
// resolverOver builds a resolver holding the given files within a memory bound.
func resolverOver(t testing.TB, maxInMemory int64, files map[string]string) *Resolver {
t.Helper()
r, _ := resolverIn(t, memCharge(maxInMemory))
r.archiveCoordinates = file.NewCoordinates("outer.jar", "")
addAll(t, r, fileEntries(files))
return r
}
// resolverFrom builds a resolver holding the given entries, links and directories included, with
// ample memory.
func resolverFrom(t testing.TB, entries ...testEntry) *Resolver {
t.Helper()
r, _ := resolverIn(t, memCharge(1<<20))
r.archiveCoordinates = file.NewCoordinates("outer.jar", "")
addAll(t, r, entries)
return r
}
// extractBytes extracts an archive into a resolver built over the given charge.
func extractBytes(t testing.TB, data []byte, name string, charge *charge) (*Resolver, string) {
t.Helper()
r, root := resolverIn(t, charge)
content := bytes.NewReader(data)
format, err := identifyFormat(context.Background(), name, content)
require.NoError(t, err)
require.NotNil(t, format, "fixture must identify as an archive")
require.NoError(t, r.extract(context.Background(), format, content, nil))
return r, root
}
func readEntry(t testing.TB, r ReaderAtSeeker) string {
t.Helper()
_, err := r.Seek(0, io.SeekStart)
require.NoError(t, err)
b, err := io.ReadAll(r)
require.NoError(t, err)
return string(b)
}
// storedEntry is one entry read back out of a resolver.
type storedEntry struct {
body string
linkTarget string
mode fs.FileMode
kind stereoscopeFile.Type
}
// readStore reads every stored entry by name: files, links, and directories listed or implied.
func readStore(t testing.TB, r *Resolver) map[string]storedEntry {
t.Helper()
out := map[string]storedEntry{}
for _, ref := range r.tree.AllFiles(stereoscopeFile.AllTypes()...) {
entry, err := r.index.Get(ref)
require.NoError(t, err)
out[entry.Metadata.Path] = storedEntry{
body: readEntry(t, r.open(r.content[ref.ID()])),
linkTarget: entry.Metadata.LinkDestination,
mode: entry.Metadata.FileInfo.Mode(),
kind: entry.Metadata.Type,
}
}
return out
}
// storedNames lists every listed entry's name, files and directories alike.
func storedNames(t testing.TB, r *Resolver) []string {
t.Helper()
var names []string
for name := range readStore(t, r) {
names = append(names, name)
}
sort.Strings(names)
return names
}
// nodeCount is every path the tree holds, including directories the archive never listed.
func nodeCount(r *Resolver) int {
return len(r.tree.AllRealPaths())
}
// contentAt is the blob held for the entry at a rooted path.
func contentAt(t testing.TB, r *Resolver, p string) *blob {
t.Helper()
_, res, err := r.tree.File(stereoscopeFile.Path(p))
require.NoError(t, err)
require.NotNil(t, res)
return r.content[res.Reference.ID()]
}
// memCharge is a charge against a memory bound with disk unbounded.
func memCharge(maxMemory int64) *charge {
return NewLimiter(Limits{MaxMemoryBytes: maxMemory, MaxDiskBytes: -1}).charge()
}
// diskCharge is a charge against a disk bound with memory refusing everything.
func diskCharge(maxDisk int64) *charge {
return NewLimiter(Limits{MaxDiskBytes: maxDisk}).charge()
}
// localPaths lists the local entry path of every location (rather than full nested archive path), sorted.
func localPaths(locations []file.Location) []string {
var out []string
for _, loc := range locations {
out = append(out, loc.LocalPath())
}
sort.Strings(out)
return out
}
// manyEntryZip is a zip of count one-byte files.
func manyEntryZip(t testing.TB, count int) []byte {
t.Helper()
files := make(map[string]string, count)
for i := range count {
files[fmt.Sprintf("e%06d.txt", i)] = "x"
}
return zipBytes(t, files)
}
// extractedResolver extracts one archive through Extract and returns the resolver, along with the scan
// temp root its spill file would appear under.
func extractedResolver(t testing.TB, data []byte, limits Limits) (*Resolver, string) {
t.Helper()
ctx, root := scanContext(t)
r, err := Extract(ctx, bytes.NewReader(data), file.NewCoordinates("app.zip", ""), NewLimiter(limits), nil)
require.NoError(t, err)
require.NotNil(t, r)
t.Cleanup(r.Cleanup)
return r, root
}
var (
unboundedLimits = Limits{MaxMemoryBytes: -1, MaxDiskBytes: -1}
// spillingLimits hold nothing in memory, so every entry lands in the spill file
spillingLimits = Limits{MaxMemoryBytes: 0, MaxDiskBytes: -1}
)
+202
View File
@@ -0,0 +1,202 @@
package archive
import (
"errors"
"sync"
)
// Budget bounds the bytes decompressed under one top-level archive, summed over every archive nested in
// it. Unlike Limits it never refills as archives are released: it bounds work, not what is held at once,
// which is what stops a recursive bomb (42.zip) whose every level fits the limits on its own. A nil
// *Budget admits everything.
//
// ponytail: no lock, since the archive walk is sequential; guard it if the walk ever runs concurrently.
type Budget struct {
remaining int64
}
func NewBudget(bytes int64) *Budget {
return &Budget{remaining: bytes}
}
// take spends n bytes, or reports false (and spends the rest) when fewer than n remain.
func (b *Budget) take(n int64) bool {
if b == nil {
return true
}
if n > b.remaining {
b.remaining = 0
return false
}
b.remaining -= n
return true
}
var errBudgetSpent = errors.New("archive tree decompression budget spent")
// ErrDiskLimitReached reports that placing an archive's content would exceed the disk limit.
var ErrDiskLimitReached = errors.New("archive content would exceed the disk limit")
// Limits bound how much archive content one scan holds at once. Each limit falls as archives are
// released, so it bounds peak usage rather than the total ever written.
//
// For both limits: positive is the bound, zero forbids the resource, negative is unbounded.
type Limits struct {
// MaxMemoryBytes bounds archive content held in memory. Content that does not fit is written to
// disk instead.
MaxMemoryBytes int64
// MaxDiskBytes bounds archive content written to disk. Content that does not fit is dropped and
// the archive is skipped or truncated.
MaxDiskBytes int64
}
// Limiter tracks how much archive content a scan holds right now against its Limits. A nil *Limiter
// enforces nothing.
type Limiter struct {
limits Limits
mu sync.Mutex
memory int64
disk int64
peakMemory int64
peakDisk int64
}
func NewLimiter(limits Limits) *Limiter {
return &Limiter{limits: limits}
}
// Peak reports the most memory and disk held at any one moment.
func (l *Limiter) Peak() (memory, disk int64) {
if l == nil {
return 0, 0
}
l.mu.Lock()
defer l.mu.Unlock()
return l.peakMemory, l.peakDisk
}
// InUse reports the memory and disk held right now.
func (l *Limiter) InUse() (memory, disk int64) {
if l == nil {
return 0, 0
}
l.mu.Lock()
defer l.mu.Unlock()
return l.memory, l.disk
}
// charge starts accounting for one archive. Releasing the charge gives back everything it took.
func (l *Limiter) charge() *charge {
if l == nil {
return nil
}
return &charge{limiter: l}
}
// charge is one archive's draw on a Limiter. A nil *charge admits everything.
//
// Bytes are charged as they land, never from the sizes an archive declares, since those can lie.
type charge struct {
limiter *Limiter
inMemory int64
onDisk int64
}
func admits(limit, held, n int64) bool {
switch {
case limit < 0:
return true
case limit == 0:
return false
default:
return held+n <= limit
}
}
// memory charges n bytes held in memory, or reports false (charging nothing) when the memory limit
// does not admit them.
func (c *charge) memory(n int64) bool {
if c == nil || n <= 0 {
return true
}
l := c.limiter
l.mu.Lock()
defer l.mu.Unlock()
if !admits(l.limits.MaxMemoryBytes, l.memory, n) {
return false
}
l.memory += n
c.inMemory += n
l.peakMemory = max(l.peakMemory, l.memory)
return true
}
// disk charges n bytes written to disk, or reports false (charging nothing) when the disk limit does
// not admit them.
func (c *charge) disk(n int64) bool {
if c == nil || n <= 0 {
return true
}
l := c.limiter
l.mu.Lock()
defer l.mu.Unlock()
if !admits(l.limits.MaxDiskBytes, l.disk, n) {
return false
}
l.disk += n
c.onDisk += n
l.peakDisk = max(l.peakDisk, l.disk)
return true
}
// index charges n bytes of index bookkeeping, which lives in memory whatever the limits say. A zero
// memory limit is a valid configuration that must still index archives, so only then does the cost
// fall back to the disk budget; a bounded memory limit bounds the index too.
func (c *charge) index(n int64) bool {
if c.memory(n) {
return true
}
return c != nil && c.limiter.limits.MaxMemoryBytes == 0 && c.disk(n)
}
// refundMemory gives back n bytes no longer held in memory.
func (c *charge) refundMemory(n int64) {
if c == nil || n <= 0 {
return
}
l := c.limiter
l.mu.Lock()
defer l.mu.Unlock()
n = min(n, c.inMemory)
l.memory -= n
c.inMemory -= n
}
// refundDisk gives back n bytes no longer on disk.
func (c *charge) refundDisk(n int64) {
if c == nil || n <= 0 {
return
}
l := c.limiter
l.mu.Lock()
defer l.mu.Unlock()
n = min(n, c.onDisk)
l.disk -= n
c.onDisk -= n
}
// release gives back everything this charge took. Safe to call more than once.
func (c *charge) release() {
if c == nil {
return
}
l := c.limiter
l.mu.Lock()
defer l.mu.Unlock()
l.memory -= c.inMemory
l.disk -= c.onDisk
c.inMemory, c.onDisk = 0, 0
}
+178
View File
@@ -0,0 +1,178 @@
package archive
import (
"sync"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestLimiter_chargeAndRelease(t *testing.T) {
limiter := NewLimiter(Limits{MaxMemoryBytes: 100, MaxDiskBytes: 200})
first := limiter.charge()
require.True(t, first.memory(60))
require.True(t, first.disk(150))
mem, disk := limiter.InUse()
assert.Equal(t, int64(60), mem)
assert.Equal(t, int64(150), disk)
second := limiter.charge()
assert.False(t, second.memory(50), "60 + 50 is over the memory limit")
assert.False(t, second.disk(60), "150 + 60 is over the disk limit")
assert.True(t, second.memory(40), "and exactly at the limit is admitted")
mem, _ = limiter.InUse()
assert.Equal(t, int64(100), mem, "a refused charge takes nothing")
first.release()
mem, disk = limiter.InUse()
assert.Equal(t, int64(40), mem, "release gives back exactly what that charge took")
assert.Zero(t, disk)
first.release()
mem, _ = limiter.InUse()
assert.Equal(t, int64(40), mem, "releasing twice must not give back what was never taken")
}
func TestLimiter_refund(t *testing.T) {
limiter := NewLimiter(Limits{MaxDiskBytes: 100})
charge := limiter.charge()
require.True(t, charge.disk(80))
charge.refundDisk(80)
_, disk := limiter.InUse()
assert.Zero(t, disk)
assert.True(t, charge.disk(100), "the refunded room is available again")
charge.refundDisk(500)
_, disk = limiter.InUse()
assert.Zero(t, disk, "a refund larger than what was charged cannot drive the limiter negative")
}
func TestLimiter_threeStateReadingIsUniform(t *testing.T) {
t.Run("positive is the limit", func(t *testing.T) {
c := NewLimiter(Limits{MaxMemoryBytes: 100, MaxDiskBytes: 100}).charge()
assert.True(t, c.memory(100))
assert.False(t, c.memory(1))
assert.True(t, c.disk(100))
assert.False(t, c.disk(1))
})
t.Run("zero refuses unconditionally on both", func(t *testing.T) {
c := NewLimiter(Limits{MaxMemoryBytes: 0, MaxDiskBytes: 0}).charge()
assert.False(t, c.memory(1))
assert.False(t, c.disk(1))
})
t.Run("negative admits unconditionally on both", func(t *testing.T) {
c := NewLimiter(Limits{MaxMemoryBytes: -1, MaxDiskBytes: -1}).charge()
assert.True(t, c.memory(1_000_000_000))
assert.True(t, c.disk(1_000_000_000))
})
}
func TestLimiter_nilIsUnbounded(t *testing.T) {
var limiter *Limiter
charge := limiter.charge()
assert.Nil(t, charge)
assert.True(t, charge.memory(1_000_000))
assert.True(t, charge.disk(1_000_000))
charge.refundDisk(10)
charge.release()
mem, disk := charge.held()
assert.Zero(t, mem)
assert.Zero(t, disk)
mem, disk = limiter.InUse()
assert.Zero(t, mem)
assert.Zero(t, disk)
mem, disk = limiter.Peak()
assert.Zero(t, mem)
assert.Zero(t, disk)
}
func TestLimiter_concurrentChargesAgree(t *testing.T) {
limiter := NewLimiter(Limits{MaxMemoryBytes: 1_000_000, MaxDiskBytes: -1})
var wg sync.WaitGroup
charges := make([]*charge, 50)
for i := range charges {
charges[i] = limiter.charge()
wg.Add(1)
go func(c *charge) {
defer wg.Done()
for range 100 {
c.memory(1)
c.disk(2)
}
}(charges[i])
}
wg.Wait()
mem, disk := limiter.InUse()
assert.Equal(t, int64(50*100), mem)
assert.Equal(t, int64(50*100*2), disk)
for _, c := range charges {
c.release()
}
mem, disk = limiter.InUse()
assert.Zero(t, mem)
assert.Zero(t, disk)
}
func TestLimiter_peak(t *testing.T) {
t.Run("is a high-water mark", func(t *testing.T) {
l := NewLimiter(Limits{MaxMemoryBytes: -1, MaxDiskBytes: -1})
first := l.charge()
require.True(t, first.memory(100))
require.True(t, first.disk(300))
first.release()
second := l.charge()
require.True(t, second.memory(40))
require.True(t, second.disk(50))
memory, disk := l.InUse()
assert.Equal(t, int64(40), memory)
assert.Equal(t, int64(50), disk)
peakMemory, peakDisk := l.Peak()
assert.Equal(t, int64(100), peakMemory, "the peak is the most held at once, not the most recently held")
assert.Equal(t, int64(300), peakDisk)
})
t.Run("counts concurrent holders together", func(t *testing.T) {
l := NewLimiter(Limits{MaxMemoryBytes: -1, MaxDiskBytes: -1})
outer := l.charge()
require.True(t, outer.memory(100))
inner := l.charge()
require.True(t, inner.memory(60))
inner.release()
outer.release()
peakMemory, _ := l.Peak()
assert.Equal(t, int64(160), peakMemory)
})
t.Run("a refused charge does not move it", func(t *testing.T) {
l := NewLimiter(Limits{MaxMemoryBytes: 50, MaxDiskBytes: 50})
c := l.charge()
assert.False(t, c.memory(500))
assert.False(t, c.disk(500))
peakMemory, peakDisk := l.Peak()
assert.Zero(t, peakMemory)
assert.Zero(t, peakDisk)
})
}
+575
View File
@@ -0,0 +1,575 @@
package archive
import (
"archive/tar"
"context"
"errors"
"fmt"
"io"
"io/fs"
"os"
"path"
"slices"
"sort"
"strings"
"time"
"unicode"
"github.com/dustin/go-humanize"
"github.com/mholt/archives"
stereoscopeFile "github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/stereoscope/pkg/filetree"
intFile "github.com/anchore/syft/internal/file"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/tmpdir"
"github.com/anchore/syft/syft/file"
)
// Resolver is one archive opened as its own filesystem: a file.Resolver over the entries Extract fed
// into it, holding their content within the scan's limits. Entries are indexed in a stereoscope file
// tree, the same index the directory and image resolvers search.
//
// Locations it returns keep the outermost archive's RealPath and carry the entry's FullPath. Queries
// may name an entry either by its full path or by its path within the archive (see rootedPath).
type Resolver struct {
// Digests are of the archive file itself.
Digests []file.Digest
// Truncated is non-nil when the resolver covers only part of the archive, and says why: the disk limit
// or the decompression budget stopped extraction early, or an entry was too large or unreadable.
Truncated error
archiveCoordinates file.Coordinates
charge *charge
budget *Budget
metered bool // the budget is charged as the archive is decoded (see meteredReader), not in put
// content is held in memory while the memory limit admits it and in one spill file once it does
// not. The file is created on first use, so an archive that stays in memory never touches the
// filesystem, and blobs are located in it by offset with no framing of their own.
tempDir *tmpdir.TempDir
held []*blob // blobs in memory, in the order they were added
file *os.File
remove func()
written int64
chunk []byte
tree *filetree.FileTree
index filetree.Index
search filetree.Searcher // built by finish, once every entry is in the tree
content map[stereoscopeFile.ID]*blob
entries int // non-directory entries added
}
var _ file.Resolver = (*Resolver)(nil)
// newResolver returns an empty resolver whose locations extend the archive's coordinates, spilling into
// the scan's temp root when ctx carries one (see internal/tmpdir).
func newResolver(ctx context.Context, archiveCoordinates file.Coordinates, charge *charge) *Resolver {
return &Resolver{
archiveCoordinates: archiveCoordinates,
charge: charge,
budget: budgetFromContext(ctx),
tempDir: tmpdir.FromContext(ctx),
tree: filetree.New(),
index: filetree.NewIndex(),
content: map[stereoscopeFile.ID]*blob{},
}
}
// Cleanup removes everything the archive holds on disk and releases its charge against the limiter.
// Readers handed out before are no longer valid. Safe to call more than once.
func (r *Resolver) Cleanup() {
if r == nil {
return
}
r.releaseStorage()
r.charge.release()
}
// approxIndexBytesPerEntry estimates what holding one tree node costs beyond its content: the node, its
// header and metadata, and its index slots. It is charged for every node an entry creates, including
// the directories its path implies, so an archive of many tiny entries or of a few entries with very
// deep paths is still bounded by the limits.
const approxIndexBytesPerEntry = 2 * 1024
func approxIndexBytes(hdr tar.Header) int64 {
return approxIndexBytesPerEntry + int64(len(hdr.Name)+len(hdr.Linkname))
}
// add stores and indexes one entry, reading its content in full now since archive readers are
// sequential. Reaching a limit stores nothing for this entry and returns ErrDiskLimitReached; entries
// added before it remain usable. The last entry at a path wins, as it does when the archive is
// extracted (tar appends updates, an OCI layer applies in order, and java's legacy zip reading
// overwrites too), so a stale earlier copy, such as an older pom.properties, can't stand in for what is
// really there.
func (r *Resolver) add(hdr tar.Header, content io.Reader) error {
entryPath := path.Clean("/" + hdr.Name)
if entryPath == "/" {
return nil
}
replacing, ok := r.replace(entryPath, hdr)
if !ok {
return nil
}
// directories the path implies that do not exist yet are nodes this entry pays for. Walking from the
// deepest parent up stops at the first that exists, since its ancestors exist too
var newDirs []string
for i := strings.LastIndexByte(entryPath, '/'); i > 0; i = strings.LastIndexByte(entryPath[:i], '/') {
if r.tree.HasPath(stereoscopeFile.Path(entryPath[:i])) {
break
}
newDirs = append(newDirs, entryPath[:i])
}
slices.Reverse(newDirs)
if !r.chargeIndex(approxIndexBytes(hdr) + int64(len(newDirs))*approxIndexBytesPerEntry) {
return ErrDiskLimitReached
}
// indexed as if listed, since a search for a directory's children looks the directory up by name
for _, dir := range newDirs {
if err := r.store(dir, tar.Header{Name: dir, Typeflag: tar.TypeDir, Mode: 0o755}, &blob{}); err != nil {
log.WithFields("entry", hdr.Name, "error", err).Trace("skipping archive entry")
return nil
}
}
b := &blob{}
if hdr.Typeflag == tar.TypeReg && content != nil {
// reading one byte past the cap is how an entry over it is told apart from one exactly at it
err := r.put(b, io.LimitReader(content, maxEntryBytes+1))
if err == nil && b.size > maxEntryBytes {
err = errEntryTooLarge
}
if err != nil {
r.discard(b)
if errors.Is(err, errEntryTooLarge) {
r.truncate(fmt.Errorf("an entry was skipped: %w of %s", err, humanize.IBytes(uint64(maxEntryBytes))))
return nil
}
if errors.Is(err, ErrDiskLimitReached) || errors.Is(err, errBudgetSpent) {
return err
}
// a damaged entry (a CRC mismatch, a corrupt deflate stream) costs only that entry
r.truncate(fmt.Errorf("entry %q could not be read: %w", hdr.Name, err))
return nil
}
}
if err := r.store(entryPath, hdr, b); err != nil {
// a path the tree cannot hold, such as a file beneath another file's name
r.discard(b)
log.WithFields("entry", hdr.Name, "error", err).Trace("skipping archive entry")
return nil
}
if hdr.Typeflag != tar.TypeDir && !replacing {
r.entries++
}
return nil
}
// replace clears the way for an entry at a path that is already held, reporting whether it did and
// false when the entry should be dropped. A later non-directory entry replaces an earlier one, refunding
// what it held. A file and a directory at one path keep the first, since swapping kinds would mean
// re-parenting what is already beneath it.
func (r *Resolver) replace(entryPath string, hdr tar.Header) (replacing, ok bool) {
exists, res, _ := r.tree.File(stereoscopeFile.Path(entryPath))
if !exists || !res.HasReference() {
return false, true
}
earlier, err := r.index.Get(*res.Reference)
if err != nil || earlier.IsDir() || hdr.Typeflag == tar.TypeDir {
return false, false
}
// a removed node's index entry is left behind, but searches only answer with references in the tree
if err := r.tree.RemovePath(stereoscopeFile.Path(entryPath)); err != nil {
log.WithFields("entry", hdr.Name, "error", err).Trace("unable to replace archive entry")
return false, false
}
if b := r.content[res.ID()]; b != nil {
r.discard(b)
}
delete(r.content, res.ID())
return true, true
}
func (r *Resolver) store(rooted string, hdr tar.Header, b *blob) error {
ref, err := r.addToTree(rooted, hdr)
if err != nil {
return err
}
metadata := stereoscopeFile.NewMetadata(hdr, r.open(b))
metadata.Path = entryPath(rooted)
r.index.Add(*ref, metadata)
r.content[ref.ID()] = b
return nil
}
func (r *Resolver) addToTree(entryPath string, hdr tar.Header) (*stereoscopeFile.Reference, error) {
p := stereoscopeFile.Path(entryPath)
switch hdr.Typeflag {
case tar.TypeDir:
return r.tree.AddDir(p)
case tar.TypeSymlink:
return r.tree.AddSymLink(p, stereoscopeFile.Path(hdr.Linkname))
case tar.TypeLink:
// a hard link names its target from the archive root
return r.tree.AddHardLink(p, stereoscopeFile.Path(path.Join("/", hdr.Linkname)))
default:
return r.tree.AddFile(p)
}
}
// maxEntryBytes caps any one entry, as the legacy java extraction did (see intFile.SafeCopy): without it
// a single entry of an overlapping-entry zip bomb can decompress up to the disk limit.
var maxEntryBytes int64 = intFile.PerFileReadLimit
var errEntryTooLarge = errors.New("archive entry exceeds the per-entry size cap")
// truncate marks the resolver as covering part of the archive, keeping the first reason given.
func (r *Resolver) truncate(err error) {
if r.Truncated == nil {
r.Truncated = err
}
}
// chargeIndex charges index bookkeeping, which lives in memory. Content held in memory can move to
// disk to make room for it; the index cannot, so when it is refused the entries are.
func (r *Resolver) chargeIndex(n int64) bool {
if r.charge.index(n) {
return true
}
if len(r.held) == 0 || r.spill() != nil {
return false
}
return r.charge.index(n)
}
// finish builds the search context once every entry is added.
func (r *Resolver) finish() {
r.search = filetree.NewSearchContext(r.tree, r.index)
}
// linkDestination is the rooted path a link points at inside the archive. A hard link names its
// target from the archive root, as does an absolute symlink, so "/etc/passwd" is an entry of this
// archive and never the host's. A relative symlink is joined onto the link's own directory, and
// cleaning against the root keeps "../../x" inside the archive.
func linkDestination(linkPath, linkname string, hard bool) string {
if hard || path.IsAbs(linkname) {
return path.Join("/", linkname)
}
return path.Join(path.Dir(linkPath), linkname)
}
// entryPath is a node's path within the archive: archive-relative with no leading slash.
func entryPath(rooted string) string {
return strings.TrimPrefix(rooted, "/")
}
// fullPath is how paths leave the resolver: the archive's own full path, then the entry path.
func (r *Resolver) fullPath(rooted string) string {
return r.archiveCoordinates.FullNestedPath() + file.ArchivePathSeparator + entryPath(rooted)
}
// rootedPath is the tree path a query names, which is either form a path takes: a full path is trimmed to
// the entry path within this archive, and anything else (a LocalPath, with or without its leading slash)
// is taken as archive-relative already, since the tree looks a relative path up from its root. The archive's full path with a trailing "!" (what path.Dir gives for an entry at the root)
// names the root. A full path into another archive in the scan names nothing here and reports false.
func (r *Resolver) rootedPath(userPath string) (string, bool) {
if rest, ok := strings.CutPrefix(userPath, r.archiveCoordinates.FullNestedPath()); ok {
if rest == file.ArchivePathSeparator[:1] {
return "/", true
}
if strings.HasPrefix(rest, file.ArchivePathSeparator) {
// keep the separator's slash as the rooted path's
return path.Clean(rest[len(file.ArchivePathSeparator)-1:]), true
}
}
if r.outside(userPath) {
return "", false
}
return path.Clean(userPath), true
}
// outside reports whether a path not under this archive's full path is a full path from the scan
// nonetheless: every full path in this archive tree starts with the outermost archive's real path and a
// separator. The real path alone is taken as an entry path, so an entry named like the outermost archive
// is found by its LocalPath.
func (r *Resolver) outside(userPath string) bool {
rest, ok := strings.CutPrefix(userPath, r.archiveCoordinates.RealPath)
return ok && strings.HasPrefix(rest, file.ArchivePathSeparator)
}
// coordinates name an entry by the outermost archive's real path, which never includes a path inside an
// archive, and its full path through every archive.
func (r *Resolver) coordinates(rooted string) file.Coordinates {
return file.Coordinates{
RealPath: r.archiveCoordinates.RealPath,
FileSystemID: r.archiveCoordinates.FileSystemID,
FullPath: r.fullPath(rooted),
}
}
// location names a resolution: a link is reported at its target's path with the link's own full path
// as the access path.
func (r *Resolver) location(res stereoscopeFile.Resolution) file.Location {
coordinates := r.coordinates(string(res.RealPath))
if res.RequestPath == "" || res.RequestPath == res.RealPath {
return file.NewLocationFromCoordinates(coordinates)
}
return file.NewVirtualLocationFromCoordinates(coordinates, r.fullPath(string(res.RequestPath)))
}
// resolve finds the file a path names, following links, and reports false for directories and
// paths the archive never listed.
func (r *Resolver) resolve(userPath string) (*stereoscopeFile.Resolution, bool) {
rooted, ok := r.rootedPath(userPath)
if !ok {
return nil, false
}
res, err := r.search.SearchByPath(rooted, filetree.FollowBasenameLinks)
if err != nil || res == nil || !res.HasReference() {
return nil, false
}
entry, err := r.index.Get(*res.Reference)
if err != nil || entry.IsDir() {
return nil, false
}
return res, true
}
func (r *Resolver) FileContentsByLocation(location file.Location) (io.ReadCloser, error) {
res, ok := r.resolve(location.FullNestedPath())
if !ok {
return nil, fmt.Errorf("no archive entry for path=%q", location.FullNestedPath())
}
return entryReader{r.open(r.content[res.ID()])}, nil
}
func (r *Resolver) FileMetadataByLocation(location file.Location) (file.Metadata, error) {
rooted, ok := r.rootedPath(location.FullNestedPath())
if !ok {
return file.Metadata{}, fmt.Errorf("no archive entry for path=%q", location.FullNestedPath())
}
exists, res, err := r.tree.File(stereoscopeFile.Path(rooted))
if err != nil || !exists || !res.HasReference() {
return file.Metadata{}, fmt.Errorf("no archive entry for path=%q", location.FullNestedPath())
}
entry, err := r.index.Get(*res.Reference)
if err != nil {
return file.Metadata{}, err
}
// the index holds entry paths; callers get full paths, as with locations
metadata := entry.Metadata
metadata.Path = r.fullPath(rooted)
if metadata.LinkDestination != "" {
metadata.LinkDestination = r.fullPath(linkDestination(rooted, metadata.LinkDestination, metadata.Type == stereoscopeFile.TypeHardLink))
}
return metadata, nil
}
func (r *Resolver) HasPath(userPath string) bool {
rooted, ok := r.rootedPath(userPath)
return ok && r.tree.HasPath(stereoscopeFile.Path(rooted))
}
func (r *Resolver) FilesByPath(paths ...string) ([]file.Location, error) {
var out []file.Location
var seen []stereoscopeFile.ID // few paths are asked for at once
for _, userPath := range paths {
res, ok := r.resolve(userPath)
if !ok || slices.Contains(seen, res.ID()) {
continue
}
seen = append(seen, res.ID())
out = append(out, r.location(*res))
}
return out, nil
}
func (r *Resolver) FilesByGlob(patterns ...string) ([]file.Location, error) {
var out []file.Location
seen := stereoscopeFile.NewFileReferenceSet()
for _, pattern := range patterns {
rooted, ok := r.rootedPath(pattern)
if !ok {
continue
}
results, err := r.searchByGlob(rooted)
if err != nil {
return nil, fmt.Errorf("invalid glob %q: %w", pattern, err)
}
for _, res := range results {
if !res.HasReference() || seen.Contains(*res.Reference) {
continue
}
entry, err := r.index.Get(*res.Reference)
if err != nil || entry.IsDir() {
continue
}
seen.Add(*res.Reference)
out = append(out, r.location(res))
}
}
sortLocations(out)
return out, nil
}
// searchByGlob is the index search, except that stereoscope answers "<dir>/*" by looking the directory
// up by name and the root has none, so "/*" (or "*") walks the tree instead.
func (r *Resolver) searchByGlob(pattern string) ([]stereoscopeFile.Resolution, error) {
if strings.TrimPrefix(pattern, "/") == "*" {
return r.tree.FilesByGlob("/*", filetree.FollowBasenameLinks)
}
return r.search.SearchByGlob(pattern, filetree.FollowBasenameLinks)
}
// sortLocations orders answers by path, which decides how locations group into packages when results
// merge into the SBOM.
func sortLocations(locations []file.Location) {
sort.Slice(locations, func(i, j int) bool {
if a, b := locations[i].FullNestedPath(), locations[j].FullNestedPath(); a != b {
return a < b
}
return locations[i].AccessPath < locations[j].AccessPath
})
}
func (r *Resolver) FilesByMIMEType(types ...string) ([]file.Location, error) {
results, err := r.search.SearchByMIMEType(types...)
if err != nil {
return nil, err
}
var out []file.Location
for _, res := range results {
if res.HasReference() {
out = append(out, r.location(res))
}
}
sortLocations(out)
return out, nil
}
// RelativeFileByPath resolves against the archive root: an archive has no layers to be relative to.
func (r *Resolver) RelativeFileByPath(_ file.Location, userPath string) *file.Location {
locations, _ := r.FilesByPath(userPath)
if len(locations) == 0 {
return nil
}
return &locations[0]
}
func (r *Resolver) AllLocations(ctx context.Context) <-chan file.Location {
out := make(chan file.Location)
go func() {
defer close(out)
refs := r.tree.AllFiles(stereoscopeFile.TypeRegular, stereoscopeFile.TypeSymLink, stereoscopeFile.TypeHardLink)
sort.Slice(refs, func(i, j int) bool { return refs[i].RealPath < refs[j].RealPath })
for _, ref := range refs {
select {
case out <- r.location(stereoscopeFile.Resolution{RequestPath: ref.RealPath, Reference: &ref}):
case <-ctx.Done():
return
}
}
}()
return out
}
// entryReader gives an entry's reader the Close the resolver interface requires. The resolver owns
// the bytes, so there is nothing to close.
type entryReader struct {
ReaderAtSeeker
}
func (entryReader) Close() error { return nil }
// maxEntryNameBytes is the longest entry name stored: PATH_MAX. Every path component becomes a node,
// so a longer name is a cheap way to inflate the index and no filesystem could hold it anyway.
const maxEntryNameBytes = MaxPathBytes
// MaxPathBytes is PATH_MAX, the longest whole archive chain a scan stores or a decoded document may build.
// Each nested file copies its container's chain, so without it a long name shared by many small entries
// costs memory quadratic in the document.
const MaxPathBytes = 4096
// MaxDepth is the deepest archives nest: a scan stops there whatever depth is configured, and a decoded
// document is held to it. No real nesting comes close, and a hostile archive (a zip quine, a recursive bomb)
// must still end.
const MaxDepth = 16
// maxEntryNameDepth is the most path components an entry name may have. Adding an entry costs work per
// component, and no real archive nests anywhere near this deep.
const maxEntryNameDepth = 256
// entryHeader builds the header stored for one archive entry, or reports false for an entry not worth
// carrying: device nodes, fifos and sockets, which no cataloger reads, and names that cannot be a path
// inside the archive. A name refused as unsafe marks the resolver truncated.
//
// The name is cleaned to an archive-relative path with no leading slash, so "../x" names x at the
// archive root rather than anything outside it. Symlinks are stored as headers, never followed on the
// host; the resolver resolves their targets inside the archive.
func (r *Resolver) entryHeader(f archives.FileInfo) (tar.Header, bool) {
mode := f.Mode()
if !mode.IsRegular() && !mode.IsDir() && mode.Type()&fs.ModeSymlink == 0 {
log.WithFields("entry", f.NameInArchive, "mode", mode).Debug("skipping non-regular archive entry")
return tar.Header{}, false
}
name, err := EntryName(f.NameInArchive)
if err != nil {
r.truncate(fmt.Errorf("an entry was skipped: %w", err))
return tar.Header{}, false
}
if name == "" {
return tar.Header{}, false
}
// held to the chain length a decoder accepts back, so the SBOM round-trips
if chain := len(r.archiveCoordinates.FullNestedPath()) + len(file.ArchivePathSeparator) + len(name); chain > MaxPathBytes {
r.truncate(fmt.Errorf("an entry was skipped: its path of %d bytes within the archive chain exceeds the %d byte cap", chain, MaxPathBytes))
return tar.Header{}, false
}
hdr, err := tar.FileInfoHeader(f.FileInfo, f.LinkTarget)
if err != nil {
log.WithFields("entry", f.NameInArchive, "error", err).Debug("skipping archive entry with an unusable header")
return tar.Header{}, false
}
hdr.Name = name
hdr.Linkname = f.LinkTarget
// nothing reads these, and the header is held for the life of the archive
hdr.PAXRecords = nil
hdr.Uname, hdr.Gname = "", ""
hdr.AccessTime, hdr.ChangeTime = time.Time{}, time.Time{}
hdr.ModTime = hdr.ModTime.Truncate(time.Second)
return *hdr, true
}
// EntryName cleans an archive entry's name to a path relative to the archive root, returning "" when it
// names nothing inside the archive. This is the trust boundary for names: every one ends up in FullPath
// and the encoders, so a name that could forge an archive boundary ("!/") or a line in a line-oriented
// format (control characters, unicode line separators) is refused. Decoders building nested coordinates
// from a document hold its names to the same rules.
func EntryName(name string) (string, error) {
if len(name) > maxEntryNameBytes {
return "", fmt.Errorf("entry name of %d bytes exceeds the %d byte cap", len(name), maxEntryNameBytes)
}
if strings.IndexFunc(name, isUnsafeNameRune) >= 0 {
return "", fmt.Errorf("entry name %q contains a control character", name)
}
cleaned := strings.TrimPrefix(path.Clean("/"+name), "/")
if depth := strings.Count(cleaned, "/") + 1; depth > maxEntryNameDepth {
return "", fmt.Errorf("entry name of %d path components exceeds the %d component cap", depth, maxEntryNameDepth)
}
if strings.Contains(cleaned, file.ArchivePathSeparator) {
return "", fmt.Errorf("entry name %q contains the archive path separator %q", name, file.ArchivePathSeparator)
}
return cleaned, nil
}
func isUnsafeNameRune(c rune) bool {
return unicode.IsControl(c) || c == '\u2028' || c == '\u2029'
}
+506
View File
@@ -0,0 +1,506 @@
package archive
import (
"archive/tar"
"bytes"
"context"
"io"
"path"
"sort"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
stereoscopeFile "github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/syft/file"
)
var jarEntriesFixture = map[string]string{
"META-INF/MANIFEST.MF": "Manifest-Version: 1.0\n",
"META-INF/maven/com.example/lib/pom.properties": "groupId=com.example\n",
"META-INF/maven/com.example/lib/pom.xml": "<project/>",
"LICENSE": "Apache License 2.0",
"BOOT-INF/lib/inner.jar": "PK\x03\x04 not really",
"com/example/Thing.class": "bytecode",
}
func TestResolver_add_refusedEntryIsNotStored(t *testing.T) {
// a half-stored pom a cataloger parses is worse than an absent one
r, _ := resolverIn(t, NewLimiter(Limits{MaxMemoryBytes: 0, MaxDiskBytes: 0}).charge())
err := r.add(regularHeader("a.txt", 3), bytes.NewReader([]byte("abc")))
assert.ErrorIs(t, err, ErrDiskLimitReached)
assert.Empty(t, storedNames(t, r))
assert.False(t, r.HasPath("a.txt"))
}
func TestResolver_add_refusedContentLeavesNoNodeBehind(t *testing.T) {
// the index estimate is admitted, then the content is refused
record := approxIndexBytes(regularHeader("dir/a.txt", 0)) + approxIndexBytesPerEntry // the entry and the directory it implies
r, _ := resolverIn(t, diskCharge(record+2))
err := r.add(regularHeader("dir/a.txt", 3), bytes.NewReader([]byte("abc")))
assert.ErrorIs(t, err, ErrDiskLimitReached)
assert.False(t, r.HasPath("dir/a.txt"))
assert.True(t, r.HasPath("dir"), "the directories it implied stay")
}
func TestResolver_add_directoryEntriesHoldNoContent(t *testing.T) {
r, _ := resolverIn(t, memCharge(1<<20))
require.NoError(t, r.add(tar.Header{Name: "lib", Typeflag: tar.TypeDir, Mode: 0o755}, nil))
assert.Zero(t, r.heldInMemory())
assert.Empty(t, readStore(t, r)["lib"].body)
}
func TestResolver_add_indexDoesNotFallBackToDiskWhenMemoryIsBounded(t *testing.T) {
// the index lives in memory, so a bounded memory limit bounds it; only a zero limit defers to disk
limiter := NewLimiter(Limits{MaxMemoryBytes: 1024, MaxDiskBytes: -1})
r, _ := resolverIn(t, limiter.charge())
err := r.add(regularHeader("a.txt", 0), bytes.NewReader(nil))
assert.ErrorIs(t, err, ErrDiskLimitReached)
_, disk := limiter.InUse()
assert.Zero(t, disk)
}
func TestResolver_add_indexMakesRoomBySpillingHeldContent(t *testing.T) {
record := approxIndexBytes(regularHeader("a.txt", 0))
r, dir := resolverIn(t, memCharge(2*record+50))
require.NoError(t, r.add(regularHeader("a.txt", 100), bytes.NewReader(bytes.Repeat([]byte("a"), 100))))
require.Equal(t, int64(100), r.heldInMemory())
// b's index does not fit beside a's content, so a's content moves to disk and b is indexed
require.NoError(t, r.add(regularHeader("b.txt", 3), bytes.NewReader([]byte("bbb"))))
r.finish()
assert.Equal(t, int64(3), r.heldInMemory())
assert.NotEmpty(t, spillFile(t, dir))
assert.Equal(t, []string{"a.txt", "b.txt"}, storedNames(t, r))
assert.Equal(t, strings.Repeat("a", 100), readStore(t, r)["a.txt"].body)
}
func TestResolver_add_lastEntryAtAPathWins(t *testing.T) {
r, _ := resolverIn(t, memCharge(1<<20))
require.NoError(t, r.add(regularHeader("a.txt", 5), bytes.NewReader([]byte("first"))))
require.NoError(t, r.add(regularHeader("a.txt", 6), bytes.NewReader([]byte("second"))))
require.NoError(t, r.add(regularHeader("b.txt", 1), bytes.NewReader([]byte("b"))))
require.NoError(t, r.add(tar.Header{Name: "b.txt", Typeflag: tar.TypeSymlink, Linkname: "a.txt"}, nil))
r.finish()
assert.Equal(t, "second", readStore(t, r)["a.txt"].body)
assert.Equal(t, "a.txt", readStore(t, r)["b.txt"].linkTarget, "a later entry may change kind")
assert.Equal(t, 2, r.entries)
assert.Equal(t, int64(len("second")), r.heldInMemory(), "the earlier copy is refunded")
locations, err := r.FilesByGlob("**/a.txt")
require.NoError(t, err)
require.Len(t, locations, 1)
reader, err := r.FileContentsByLocation(locations[0])
require.NoError(t, err)
assert.Equal(t, "second", readEntry(t, reader.(entryReader).ReaderAtSeeker))
}
func TestResolver_add_fileAndDirectoryAtOnePathKeepsTheFirst(t *testing.T) {
r, _ := resolverIn(t, memCharge(1<<20))
require.NoError(t, r.add(regularHeader("a/b", 1), bytes.NewReader([]byte("x"))))
require.NoError(t, r.add(regularHeader("a", 4), bytes.NewReader([]byte("file"))))
require.NoError(t, r.add(regularHeader("c", 4), bytes.NewReader([]byte("file"))))
require.NoError(t, r.add(tar.Header{Name: "c", Typeflag: tar.TypeDir, Mode: 0o755}, nil))
r.finish()
got := readStore(t, r)
assert.Equal(t, stereoscopeFile.TypeDirectory, got["a"].kind)
assert.Equal(t, "file", got["c"].body)
}
func TestResolver_pathsAndContents(t *testing.T) {
r := resolverOver(t, 1<<20, jarEntriesFixture)
locations, err := r.FilesByPath("/META-INF/MANIFEST.MF")
require.NoError(t, err)
require.Len(t, locations, 1)
assert.True(t, r.HasPath("/META-INF/MANIFEST.MF"))
assert.True(t, r.HasPath("META-INF/MANIFEST.MF"), "a path may arrive with or without the leading slash")
assert.False(t, r.HasPath("/nope"))
reader, err := r.FileContentsByLocation(locations[0])
require.NoError(t, err)
body, err := io.ReadAll(reader)
require.NoError(t, err)
require.NoError(t, reader.Close())
assert.Equal(t, "Manifest-Version: 1.0\n", string(body))
}
func TestResolver_fullPathsOutsideTheArchiveNameNothing(t *testing.T) {
r, _ := resolverIn(t, memCharge(1<<20))
r.archiveCoordinates = file.Coordinates{RealPath: "/app.war", FullPath: "/app.war!/WEB-INF/lib/dep.jar"}
addAll(t, r, fileEntries(map[string]string{"/app.war": "same name", "/META-INF/MANIFEST.MF": "m"}))
locations, err := r.FilesByPath("/app.war!/WEB-INF/lib/dep.jar!/META-INF/MANIFEST.MF")
require.NoError(t, err)
assert.Len(t, locations, 1, "a full path in this archive")
for _, p := range []string{"/app.war!/WEB-INF/lib/dep.jar", "/app.war!/WEB-INF/lib/other.jar!/META-INF/MANIFEST.MF"} {
locations, err = r.FilesByPath(p)
require.NoError(t, err)
assert.Empty(t, locations, "%s is outside the archive", p)
assert.False(t, r.HasPath(p), p)
}
for _, p := range []string{"app.war", "/app.war"} {
locations, err = r.FilesByPath(p)
require.NoError(t, err)
require.Len(t, locations, 1, "%s is an entry path named like the outermost archive", p)
assert.Equal(t, "/app.war!/WEB-INF/lib/dep.jar!/app.war", locations[0].FullPath, p)
}
again, err := r.FilesByPath(locations[0].LocalPath())
require.NoError(t, err)
assert.Equal(t, locations, again, "a LocalPath finds its own entry")
}
func TestResolver_mimeTypesAreSniffedFromContent(t *testing.T) {
r := resolverOver(t, 1<<20, map[string]string{
"notes.txt": "just some text",
"run.sh": "#!/bin/bash\necho hello\n",
})
locations, err := r.FilesByMIMEType("text/x-shellscript")
require.NoError(t, err)
assert.Equal(t, []string{"/run.sh"}, localPaths(locations))
}
func TestResolver_answersAreOrdered(t *testing.T) {
// order decides which locations group into which package when results merge into the SBOM
entries := map[string]string{}
for _, name := range []string{
"b/z.jar", "a/y.jar", "c/x.jar", "a/b/c/w.jar", "z.jar", "a.jar",
} {
entries[name] = "PK\x03\x04 not really"
}
r := resolverOver(t, 1<<20, entries)
first, err := r.FilesByGlob("**/*.jar")
require.NoError(t, err)
require.Len(t, first, len(entries))
byMIME, err := r.FilesByMIMEType("application/octet-stream")
require.NoError(t, err)
for range 8 {
again, err := r.FilesByGlob("**/*.jar")
require.NoError(t, err)
assert.Equal(t, first, again)
mimeAgain, err := r.FilesByMIMEType("application/octet-stream")
require.NoError(t, err)
assert.Equal(t, byMIME, mimeAgain)
var all []string
for loc := range r.AllLocations(context.Background()) {
all = append(all, loc.LocalPath())
}
assert.True(t, sort.StringsAreSorted(all))
}
var paths []string
for _, loc := range first {
paths = append(paths, loc.LocalPath())
}
assert.True(t, sort.StringsAreSorted(paths), "ordered by path, not merely stable")
}
func TestResolver_impliedDirectoriesArePathsButNotFiles(t *testing.T) {
r := resolverOver(t, 1<<20, jarEntriesFixture)
assert.True(t, r.HasPath("META-INF"))
assert.True(t, r.HasPath("/META-INF/maven"))
locations, err := r.FilesByPath("META-INF")
require.NoError(t, err)
assert.Empty(t, locations)
locations, err = r.FilesByGlob("**/maven")
require.NoError(t, err)
assert.Empty(t, locations)
}
// stereoscope finds a directory's children by looking the directory up by name: directories the archive
// never listed must be indexed, and the root, which has no name, answered another way
func TestResolver_globsForADirectorysChildren(t *testing.T) {
r := resolverOver(t, 1<<20, map[string]string{
"LICENSE": "root",
"META-INF/LICENSE.txt": "nested",
"META-INF/maven/g/a/pom.xml": "deeper",
})
assert.Equal(t, []string{"/LICENSE"}, globPaths(t, r, "/*"))
assert.Equal(t, []string{"/LICENSE"}, globPaths(t, r, "*"))
assert.Equal(t, []string{"/META-INF/LICENSE.txt"}, globPaths(t, r, "/META-INF/*"))
assert.Equal(t, []string{"/META-INF/LICENSE.txt"}, globPaths(t, r, "META-INF/*"))
assert.Equal(t, []string{"/META-INF/maven/g/a/pom.xml"}, globPaths(t, r, "**/a/*"))
}
func TestResolver_invalidGlobIsAnError(t *testing.T) {
r := resolverOver(t, 1<<20, jarEntriesFixture)
_, err := r.FilesByGlob("**/[")
assert.Error(t, err)
}
func globPaths(t *testing.T, r *Resolver, pattern string) []string {
t.Helper()
locations, err := r.FilesByGlob(pattern)
require.NoError(t, err)
return localPaths(locations)
}
// several paths to one file answer as one location: the file's own path when it matched, otherwise
// the lowest-sorting link
func TestResolver_linksCollapseToOnePath(t *testing.T) {
tests := []struct {
name string
entries []testEntry
pattern string
want []string
reason string
}{
{
name: "the file itself beats every link to it",
entries: []testEntry{
{name: "opt/real.jar", body: "PK\x03\x04"},
{name: "opt/a-sym.jar", link: "real.jar"},
{name: "opt/z-sym.jar", link: "real.jar"},
{name: "opt/hard.jar", link: "opt/real.jar", hard: true},
},
pattern: "**/*.jar",
want: []string{"/opt/real.jar"},
reason: "one file, so one location - and it is named by the path that holds it",
},
{
name: "with no candidate naming the file, the lowest-sorting link wins",
entries: []testEntry{
{name: "opt/real.bin", body: "PK\x03\x04"},
{name: "opt/z-sym.jar", link: "real.bin"},
{name: "opt/a-sym.jar", link: "real.bin"},
{name: "opt/m-sym.jar", link: "real.bin"},
},
pattern: "**/*.jar",
want: []string{"/opt/real.bin"},
reason: "the content is the target's, so that is the real path the answer carries",
},
{
name: "distinct files stay distinct",
entries: []testEntry{
{name: "opt/one.jar", body: "one"},
{name: "opt/two.jar", body: "two"},
{name: "opt/one-sym.jar", link: "one.jar"},
},
pattern: "**/*.jar",
want: []string{"/opt/one.jar", "/opt/two.jar"},
reason: "collapsing is per file, not per pattern",
},
{
name: "a dangling link is not a file",
entries: []testEntry{
{name: "opt/real.jar", body: "PK\x03\x04"},
{name: "opt/gone.jar", link: "missing.jar"},
},
pattern: "**/*.jar",
want: []string{"/opt/real.jar"},
reason: "it names no content",
},
{
name: "two dangling links are two paths, not one",
entries: []testEntry{
{name: "opt/a.jar", link: "missing.jar"},
{name: "opt/b.jar", link: "missing.jar"},
},
pattern: "**/*.jar",
want: nil,
reason: "a link to nothing is not a file",
},
{
name: "a chain of links collapses onto the file at its end",
entries: []testEntry{
{name: "opt/real.jar", body: "PK\x03\x04"},
{name: "opt/mid.jar", link: "real.jar"},
{name: "opt/outer.jar", link: "mid.jar"},
},
pattern: "**/*.jar",
want: []string{"/opt/real.jar"},
reason: "following one hop at a time reaches the same file from every path",
},
{
name: "a cycle resolves to nothing rather than looping",
entries: []testEntry{
{name: "opt/a.jar", link: "b.jar"},
{name: "opt/b.jar", link: "a.jar"},
},
pattern: "**/*.jar",
want: nil,
reason: "neither reaches a file",
},
{
name: "a hard link resolves from the archive root, not from its own directory",
entries: []testEntry{
{name: "opt/real.jar", body: "PK\x03\x04"},
{name: "opt/nested/hard.jar", link: "opt/real.jar", hard: true},
},
pattern: "**/*.jar",
want: []string{"/opt/real.jar"},
reason: "read as relative it would resolve to opt/nested/opt/real.jar and dangle",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
r := resolverFrom(t, tt.entries...)
assert.Equal(t, tt.want, globPaths(t, r, tt.pattern), tt.reason)
})
}
}
func TestResolver_collapsedLocationReadsTheTarget(t *testing.T) {
r := resolverFrom(t,
testEntry{name: "opt/real.bin", body: "PK\x03\x04 real contents"},
testEntry{name: "opt/a-sym.jar", link: "real.bin"},
testEntry{name: "opt/z-sym.jar", link: "real.bin"},
)
locations, err := r.FilesByGlob("**/*.jar")
require.NoError(t, err)
require.Len(t, locations, 1)
assert.Equal(t, "/opt/real.bin", locations[0].LocalPath(), "content comes from the file")
assert.Equal(t, "outer.jar!/opt/a-sym.jar", locations[0].AccessPath, "the path searched by is the one matched")
reader, err := r.FileContentsByLocation(locations[0])
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, reader.Close()) })
body, err := io.ReadAll(reader)
require.NoError(t, err)
assert.Equal(t, "PK\x03\x04 real contents", string(body),
"a location standing for a link must still read the file's bytes, not the link's none")
}
func TestResolver_fullPathsAreTrimmedToEntryPaths(t *testing.T) {
// catalogers build queries from the full paths the resolver hands out, e.g. path.Dir plus a sibling
r := resolverOver(t, 1<<20, map[string]string{"pom.xml": "a", "lib/dep.jar": "b"})
locations, err := r.FilesByPath("outer.jar!/lib/dep.jar")
require.NoError(t, err)
require.Len(t, locations, 1)
assert.Equal(t, "outer.jar!/lib/dep.jar", locations[0].AccessPath)
locations, err = r.FilesByPath(path.Join(path.Dir(locations[0].AccessPath), "../pom.xml"))
require.NoError(t, err)
require.Len(t, locations, 1, "a sibling at the archive root is reached through the full path")
assert.Equal(t, "outer.jar!/pom.xml", locations[0].FullNestedPath())
assert.True(t, r.HasPath(path.Dir("outer.jar!/pom.xml")), "the directory of a root entry is the root")
assert.True(t, r.HasPath("lib/dep.jar"), "an entry path is still accepted")
locations, err = r.FilesByGlob("outer.jar!/**/*.jar")
require.NoError(t, err)
require.Len(t, locations, 1)
}
func TestResolver_danglingLinkHasNoContent(t *testing.T) {
r := resolverFrom(t, testEntry{name: "opt/gone.jar", link: "missing.jar"})
var locations []file.Location
for l := range r.AllLocations(context.Background()) {
locations = append(locations, l)
}
require.Len(t, locations, 1)
assert.Equal(t, "/opt/gone.jar", locations[0].LocalPath())
_, err := r.FileContentsByLocation(locations[0])
assert.Error(t, err, "a link to nothing has no content to read")
metadata, err := r.FileMetadataByLocation(locations[0])
require.NoError(t, err)
assert.Equal(t, "outer.jar!/opt/missing.jar", metadata.LinkDestination,
"where it pointed is still reported, resolved against the link's own directory")
}
func TestResolver_mimeTypeAnswersOnlyWithFiles(t *testing.T) {
r := resolverFrom(t,
testEntry{name: "opt/real.jar", body: "PK\x03\x04 zip contents here"},
testEntry{name: "opt/sym.jar", link: "real.jar"},
testEntry{name: "opt/hard.jar", link: "opt/real.jar", hard: true},
testEntry{name: "opt/gone.jar", link: "missing.jar"},
testEntry{name: "opt/notes.txt", body: "hello world"},
)
byMIME := func(types ...string) []string {
t.Helper()
locations, err := r.FilesByMIMEType(types...)
require.NoError(t, err)
return localPaths(locations)
}
assert.Equal(t, []string{"/opt/real.jar"}, byMIME("application/zip"),
"the file answers, not the two links to it")
assert.Equal(t, []string{"/opt/notes.txt"}, byMIME("text/plain"))
assert.Empty(t, byMIME(""),
"a link sniffs to no type, and must not be reachable by asking for that")
assert.Equal(t, []string{"/opt/notes.txt", "/opt/real.jar"}, byMIME("application/zip", "text/plain"))
}
// AllLocations enumerates what the archive holds rather than searching for files: the file metadata
// cataloger records a row per path, links included
func TestResolver_allLocationsKeepsEveryPath(t *testing.T) {
r := resolverFrom(t,
testEntry{name: "opt/real.jar", body: "PK\x03\x04"},
testEntry{name: "opt/sym.jar", link: "real.jar"},
testEntry{name: "opt/gone.jar", link: "missing.jar"},
)
var got []string
for l := range r.AllLocations(context.Background()) {
got = append(got, l.LocalPath())
}
assert.Equal(t, []string{"/opt/gone.jar", "/opt/real.jar", "/opt/sym.jar"}, got,
"every path the archive names, links included")
}
func TestResolver_add_anEntryOverTheCapIsSkippedAndReadNoFurther(t *testing.T) {
// one entry of an overlapping-entry bomb can inflate without bound; it must stop at the cap, cost
// the archive only that entry, and say so
orig := maxEntryBytes
maxEntryBytes = 1024
t.Cleanup(func() { maxEntryBytes = orig })
r, _ := resolverIn(t, memCharge(-1))
endless := &countingReader{r: zeroSource{}}
require.NoError(t, r.add(tar.Header{Name: "bomb.bin", Typeflag: tar.TypeReg, Mode: 0o644}, endless))
require.NoError(t, r.add(tar.Header{Name: "ok.txt", Typeflag: tar.TypeReg, Mode: 0o644}, strings.NewReader("ok")))
r.finish()
assert.LessOrEqual(t, endless.n, 1024+copyChunkSize, "reading stops just past the cap")
assert.ErrorIs(t, r.Truncated, errEntryTooLarge)
assert.ErrorContains(t, r.Truncated, "per-entry size cap of 1.0 KiB")
locations, err := r.FilesByGlob("**")
require.NoError(t, err)
assert.Equal(t, []string{"/ok.txt"}, localPaths(locations))
}
type countingReader struct {
r io.Reader
n int
}
func (c *countingReader) Read(p []byte) (int, error) {
n, err := c.r.Read(p)
c.n += n
return n, err
}
+217
View File
@@ -0,0 +1,217 @@
package archive
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"os"
"github.com/anchore/syft/internal/log"
)
// copyChunkSize is how many bytes are read and charged at a time, bounding how far a charge runs
// ahead of what is actually held.
const copyChunkSize = 32 * 1024
// spillFilePattern names the one file an archive's content spills into, as in os.CreateTemp.
const spillFilePattern = "archive-spill-*"
// ReaderAtSeeker is the random access an archive format needs: a zip is read from its central
// directory at the end of the stream.
type ReaderAtSeeker interface {
io.Reader
io.ReaderAt
io.Seeker
}
// blob is one piece of content the resolver holds: in memory, or at an offset in its spill file once
// memory refused it. Moving changes nothing that points at the blob.
type blob struct {
mem []byte // content while held in memory
off int64 // where content starts in the spill file once written there
size int64
}
// put reads content in full into b, since archive readers are sequential. Each chunk is charged before it
// is held, so a stream larger than the memory limit is never held in full. Reaching the disk limit
// returns ErrDiskLimitReached; what landed in the file before that stays there and stays charged.
func (r *Resolver) put(b *blob, content io.Reader) error {
if r.chunk == nil {
r.chunk = make([]byte, copyChunkSize)
}
for {
n, readErr := content.Read(r.chunk)
if n > 0 {
// content read through a meteredReader was charged as it was decoded
if !r.metered && !r.budget.take(int64(n)) {
return errBudgetSpent
}
if err := r.write(b, r.chunk[:n]); err != nil {
return err
}
}
if readErr == nil {
continue
}
if errors.Is(readErr, io.EOF) {
break
}
return readErr
}
return nil
}
// meteredReader charges the decompression budget for every byte read through it and stops once ctx is
// done, so nothing a decoder produces gets around the budget.
type meteredReader struct {
ctx context.Context
reader io.Reader
budget *Budget
}
func (m *meteredReader) Read(p []byte) (int, error) {
if err := m.ctx.Err(); err != nil {
return 0, err
}
n, err := m.reader.Read(p)
if n > 0 && !m.budget.take(int64(n)) {
return 0, errBudgetSpent
}
return n, err
}
// write appends p to b: into memory while the memory limit admits it, otherwise into the spill file
// after everything held in memory is moved there first.
func (r *Resolver) write(b *blob, p []byte) error {
n := int64(len(p))
onDisk := b.mem == nil && b.size > 0
if !onDisk {
if r.charge.memory(n) {
if b.mem == nil {
r.held = append(r.held, b)
}
b.mem = append(b.mem, p...)
b.size += n
return nil
}
if err := r.spill(); err != nil {
return err
}
}
if err := r.openFile(); err != nil {
return err
}
if !r.charge.disk(n) {
return ErrDiskLimitReached
}
if b.size == 0 {
b.off = r.written
}
// b is always the blob most recently written, so it ends where the file does
w, err := r.file.WriteAt(p, r.written)
r.written += int64(w)
b.size += int64(w)
r.charge.refundDisk(n - int64(w))
if err != nil {
return fmt.Errorf("unable to write archive content to disk: %w", err)
}
return nil
}
// spill moves every blob held in memory into the spill file, charging the disk limit as bytes land
// and refunding the memory they leave.
func (r *Resolver) spill() error {
if len(r.held) == 0 {
return nil
}
if err := r.openFile(); err != nil {
return err
}
log.WithFields("archive", r.archiveCoordinates.FullNestedPath(), "held", len(r.held), "bytes", r.written).
Debug("archive content does not fit in memory; writing it to disk")
for len(r.held) > 0 {
b := r.held[0]
n := int64(len(b.mem))
if !r.charge.disk(n) {
return ErrDiskLimitReached
}
if _, err := r.file.WriteAt(b.mem, r.written); err != nil {
return fmt.Errorf("unable to write archive content to disk: %w", err)
}
b.off, b.mem = r.written, nil
r.written += n
r.charge.refundMemory(n)
r.held = r.held[1:]
}
return nil
}
func (r *Resolver) openFile() error {
if r.file != nil {
return nil
}
var err error
if r.tempDir != nil {
r.file, r.remove, err = r.tempDir.NewFile(spillFilePattern)
} else {
r.file, err = os.CreateTemp("", spillFilePattern)
r.remove = func() {
if err := os.Remove(r.file.Name()); err != nil {
log.WithFields("path", r.file.Name(), "error", err).Trace("unable to remove archive temp file")
}
}
}
if err != nil {
return fmt.Errorf("unable to create archive temp file: %w", err)
}
return nil
}
// discard drops a blob's bytes held in memory and refunds them. Bytes it holds on disk stay until
// Cleanup.
func (r *Resolver) discard(b *blob) {
for i, held := range r.held {
if held == b {
r.held = append(r.held[:i], r.held[i+1:]...)
break
}
}
r.charge.refundMemory(int64(len(b.mem)))
b.mem, b.size = nil, 0
}
// open returns a reader over a blob's content, wherever it is. The reader supports Read, Seek and
// ReadAt, so a nested archive can be read where it lies.
func (r *Resolver) open(b *blob) ReaderAtSeeker {
switch {
case b.mem != nil:
return bytes.NewReader(b.mem)
case b.size == 0:
return bytes.NewReader(nil)
default:
return io.NewSectionReader(r.file, b.off, b.size)
}
}
// releaseStorage refunds every blob held in memory and closes, removes and refunds the spill file.
// Readers from open must not be used after this. Safe to call more than once.
func (r *Resolver) releaseStorage() {
for _, b := range r.held {
r.charge.refundMemory(int64(len(b.mem)))
b.mem = nil
}
r.held = nil
if r.file == nil {
return
}
if err := r.file.Close(); err != nil {
log.WithFields("archive", r.archiveCoordinates.FullNestedPath(), "error", err).Trace("unable to close archive temp file")
}
r.remove()
r.charge.refundDisk(r.written)
r.file, r.remove, r.written = nil, nil, 0
}
+276
View File
@@ -0,0 +1,276 @@
package archive
import (
"archive/zip"
"bytes"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestResolver_put_holdsSmallContentInMemory(t *testing.T) {
s, dir := resolverIn(t, memCharge(1024))
var b blob
require.NoError(t, s.put(&b, bytes.NewReader([]byte("hello"))))
assert.Equal(t, int64(5), s.heldInMemory())
assert.Empty(t, filesIn(t, dir))
assert.Equal(t, "hello", readEntry(t, s.open(&b)))
}
func TestResolver_put_spillsToDiskAndKeepsTheSameBlobs(t *testing.T) {
// nodes pointing at a blob are not rebuilt when its content moves
s, dir := resolverIn(t, memCharge(8))
var first, second blob
require.NoError(t, s.put(&first, bytes.NewReader([]byte("aaaa"))))
require.Equal(t, int64(4), s.heldInMemory())
require.NoError(t, s.put(&second, bytes.NewReader([]byte("bbbbbb"))))
assert.Zero(t, s.heldInMemory(), "everything moves out, not just the blob that did not fit")
assert.NotEmpty(t, spillFile(t, dir))
assert.Equal(t, int64(10), s.written)
assert.Equal(t, "aaaa", readEntry(t, s.open(&first)))
assert.Equal(t, "bbbbbb", readEntry(t, s.open(&second)))
}
func TestResolver_put_threeStateLimits(t *testing.T) {
body := []byte(strings.Repeat("e", 500))
t.Run("a zero memory limit sends everything to disk", func(t *testing.T) {
charge := NewLimiter(Limits{MaxMemoryBytes: 0, MaxDiskBytes: 10_000}).charge()
s, dir := resolverIn(t, charge)
var b blob
require.NoError(t, s.put(&b, bytes.NewReader(body)))
assert.Zero(t, s.heldInMemory())
assert.NotEmpty(t, spillFile(t, dir))
assert.Equal(t, body, []byte(readEntry(t, s.open(&b))))
mem, disk := charge.held()
assert.Zero(t, mem)
assert.Equal(t, int64(500), disk)
})
t.Run("a negative memory limit holds content regardless of how much is already held", func(t *testing.T) {
charge := NewLimiter(Limits{MaxMemoryBytes: -1, MaxDiskBytes: 10_000}).charge()
require.True(t, charge.memory(1_000_000))
s, dir := resolverIn(t, charge)
var b blob
require.NoError(t, s.put(&b, bytes.NewReader(body)))
assert.Empty(t, filesIn(t, dir))
mem, disk := charge.held()
assert.Equal(t, int64(1_000_000+len(body)), mem)
assert.Zero(t, disk)
})
t.Run("a zero disk limit still holds what memory admits", func(t *testing.T) {
charge := NewLimiter(Limits{MaxMemoryBytes: 10_000, MaxDiskBytes: 0}).charge()
s, dir := resolverIn(t, charge)
var b blob
require.NoError(t, s.put(&b, bytes.NewReader(body)))
assert.Empty(t, filesIn(t, dir))
mem, disk := charge.held()
assert.Equal(t, int64(len(body)), mem)
assert.Zero(t, disk)
})
t.Run("a negative disk limit writes with no ceiling", func(t *testing.T) {
s, dir := resolverIn(t, NewLimiter(Limits{MaxMemoryBytes: 0, MaxDiskBytes: -1}).charge())
var b blob
require.NoError(t, s.put(&b, bytes.NewReader(body)))
assert.NotEmpty(t, spillFile(t, dir))
assert.Equal(t, body, []byte(readEntry(t, s.open(&b))))
})
t.Run("a nil charge holds everything in memory", func(t *testing.T) {
s, dir := resolverIn(t, nil)
var b blob
require.NoError(t, s.put(&b, bytes.NewReader(body)))
assert.Empty(t, filesIn(t, dir))
assert.Equal(t, body, []byte(readEntry(t, s.open(&b))))
})
}
func TestResolver_put_bytesReadBeforeTheLimitRefusedAreNotLost(t *testing.T) {
body := []byte(strings.Repeat("b", 250))
charge := NewLimiter(Limits{MaxMemoryBytes: 100, MaxDiskBytes: 10_000}).charge()
s, _ := resolverIn(t, charge)
var b blob
require.NoError(t, s.put(&b, bytes.NewReader(body)))
assert.Equal(t, body, []byte(readEntry(t, s.open(&b))))
mem, disk := charge.held()
assert.Zero(t, mem, "nothing stays charged to a limit that refused")
assert.Equal(t, int64(250), disk)
}
// peakTrackingReader records the highest memory charge observed while being read.
type peakTrackingReader struct {
r io.Reader
charge *charge
peak int64
}
func (p *peakTrackingReader) Read(b []byte) (int, error) {
n, err := p.r.Read(b)
if mem, _ := p.charge.held(); mem > p.peak {
p.peak = mem
}
return n, err
}
func TestResolver_put_neverBuffersBeyondTheMemoryLimit(t *testing.T) {
const limit = int64(3 * copyChunkSize)
body := []byte(strings.Repeat("c", 10*copyChunkSize))
charge := NewLimiter(Limits{MaxMemoryBytes: limit, MaxDiskBytes: int64(len(body)) + 1}).charge()
tracked := &peakTrackingReader{r: bytes.NewReader(body), charge: charge}
s, _ := resolverIn(t, charge)
var b blob
require.NoError(t, s.put(&b, tracked))
assert.Equal(t, body, []byte(readEntry(t, s.open(&b))))
assert.LessOrEqual(t, tracked.peak, limit, "memory charged must never exceed the limit, even transiently")
mem, disk := charge.held()
assert.Zero(t, mem)
assert.Equal(t, int64(len(body)), disk)
}
func TestResolver_put_diskLimitIsTerminal(t *testing.T) {
tests := []struct {
name string
limits Limits
}{
{"a disk limit that is exceeded", Limits{MaxDiskBytes: 100}},
{"a zero disk limit", Limits{MaxMemoryBytes: 10, MaxDiskBytes: 0}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
charge := NewLimiter(tt.limits).charge()
s, dir := resolverIn(t, charge)
var b blob
err := s.put(&b, bytes.NewReader([]byte(strings.Repeat("d", 500))))
require.ErrorIs(t, err, ErrDiskLimitReached)
s.Cleanup()
assert.Empty(t, filesIn(t, dir), "cleanup removes the partial file")
mem, disk := charge.held()
assert.Zero(t, mem)
assert.Zero(t, disk, "and refunds it")
})
}
}
func TestResolver_cleanupRefundsEverythingAndIsIdempotent(t *testing.T) {
charge := NewLimiter(Limits{MaxMemoryBytes: 8, MaxDiskBytes: -1}).charge()
s, dir := resolverIn(t, charge)
var spilled, held blob
require.NoError(t, s.put(&spilled, bytes.NewReader([]byte("spilled to disk"))))
require.NoError(t, s.put(&held, bytes.NewReader([]byte("held"))))
mem, disk := charge.held()
require.Equal(t, int64(4), mem)
require.Equal(t, int64(15), disk)
require.Equal(t, "held", readEntry(t, s.open(&held)))
s.Cleanup()
assert.Empty(t, filesIn(t, dir))
mem, disk = charge.held()
assert.Zero(t, mem)
assert.Zero(t, disk)
_, err := io.ReadAll(s.open(&spilled))
assert.Error(t, err, "readers over the spill file are no longer valid")
s.Cleanup()
mem, disk = charge.held()
assert.Zero(t, mem, "a second cleanup refunds nothing twice")
assert.Zero(t, disk)
}
func TestResolver_put_discardRefundsWhatIsHeldInMemory(t *testing.T) {
charge := memCharge(1024)
s, _ := resolverIn(t, charge)
var keep, drop blob
require.NoError(t, s.put(&keep, bytes.NewReader([]byte("keep"))))
require.NoError(t, s.put(&drop, bytes.NewReader([]byte("dropped"))))
s.discard(&drop)
mem, _ := charge.held()
assert.Equal(t, int64(4), mem)
assert.Equal(t, int64(4), s.heldInMemory())
assert.Empty(t, readEntry(t, s.open(&drop)))
assert.Equal(t, "keep", readEntry(t, s.open(&keep)))
}
func TestResolver_filesystemCostDoesNotTrackEntryCount(t *testing.T) {
small, smallRoot := extractedResolver(t, manyEntryZip(t, 2), spillingLimits)
large, largeRoot := extractedResolver(t, manyEntryZip(t, 5000), spillingLimits)
assert.Len(t, filesIn(t, smallRoot), 1)
assert.Len(t, filesIn(t, largeRoot), 1, "5000 entries cost the same filesystem entries as 2")
assert.Equal(t, 2, small.entries)
assert.Equal(t, 5000, large.entries)
// entries cost their content and nothing else on disk: no headers, no padding
assert.Equal(t, int64(2), small.written)
assert.Equal(t, int64(5000), large.written)
}
func TestResolver_anEntryAtTheEndIsReadBySeeking(t *testing.T) {
// the reader over an entry is a bounded window at its offset, not a re-read of what precedes it
const filler = 64 * 1024
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
for i := range 8 {
w, err := zw.Create(fmt.Sprintf("filler%d.bin", i))
require.NoError(t, err)
_, err = w.Write(bytes.Repeat([]byte{byte(i)}, filler))
require.NoError(t, err)
}
last, err := zw.Create("zz-last.txt")
require.NoError(t, err)
wanted := strings.Repeat("last", 64)
_, err = last.Write([]byte(wanted))
require.NoError(t, err)
require.NoError(t, zw.Close())
r, root := extractedResolver(t, buf.Bytes(), spillingLimits)
body, err := os.ReadFile(filepath.Join(root, spillFile(t, root)))
require.NoError(t, err)
offset := int64(bytes.Index(body, []byte(wanted)))
require.Greater(t, offset, int64(len(body))*9/10, "the last entry must really be at the end of the file")
reader := r.open(contentAt(t, r, "/zz-last.txt"))
end, err := reader.Seek(0, io.SeekEnd)
require.NoError(t, err)
assert.Equal(t, int64(len(wanted)), end)
head := make([]byte, 8)
_, err = reader.ReadAt(head, 0)
require.NoError(t, err)
assert.Equal(t, wanted[:8], string(head))
}
+194
View File
@@ -0,0 +1,194 @@
package archive
import (
"context"
"path"
"slices"
"strings"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
// Traversal describes the archive whose contents a cataloger is running against. The archive
// cataloger task puts one on the context before running catalogers inside an archive, so a cataloger
// such as java can describe the archive it is inside without owning the recursion.
type Traversal struct {
// Location is the archive file's location within its parent filesystem, including the access path
// it was reached by, so VirtualPath(Location) is the chain from the scan root as seen through that
// access path. The files inside it extend its real coordinates instead (see ContainerPath).
Location file.Location
// Digests are of the archive file itself, taken during extraction; a cataloger running inside the
// archive cannot compute them since the archive is not inside itself.
Digests []file.Digest
// Parent is the traversal of the archive this one is nested in, nil for an archive at the scan root.
Parent *Traversal
// Packages are those the catalogers found describing this archive (located at it), filled in once
// they have run. Catalogers inside a nested archive read them through Parent, as java does to link a
// nested jar to the jar that holds it.
Packages []pkg.Package
}
// FullPathIn is the full path of the entry at entryPath inside the archive whose full path is
// containerPath, or entryPath itself for a blank containerPath (the scan root).
func FullPathIn(containerPath, entryPath string) string {
if containerPath == "" {
return entryPath
}
// cleaning an already clean path allocates nothing
return containerPath + file.ArchivePathSeparator + strings.TrimPrefix(path.Clean(entryPath), "/")
}
// OwnerContainers are the containers a package owns files within: the container of each of its
// locations, each once, as coordinates whose FullPath is the container's full path (blank at the scan
// root). A package without locations owns files at the scan root. See OwnedCoordinates.
func OwnerContainers(p pkg.Package) []file.Coordinates {
var out []file.Coordinates
for _, l := range p.Locations.ToUnorderedSlice() {
c := file.Coordinates{FileSystemID: l.FileSystemID}
if container := ContainerPath(l.Coordinates); container != "" {
c.RealPath, c.FullPath = l.RealPath, container
}
if !slices.Contains(out, c) {
out = append(out, c)
}
}
if len(out) == 0 {
out = append(out, file.Coordinates{})
}
return out
}
// OwnedCoordinates are the coordinates of the file at ownedPath within container, one of
// OwnerContainers. At the scan root the path may still need resolving (it may pass through links).
func OwnedCoordinates(container file.Coordinates, ownedPath string) file.Coordinates {
if container.FullPath == "" {
return file.Coordinates{RealPath: ownedPath, FileSystemID: container.FileSystemID}
}
return file.Coordinates{
RealPath: container.RealPath,
FileSystemID: container.FileSystemID,
FullPath: FullPathIn(container.FullPath, ownedPath),
}
}
// FullTraversalArchivePath is the archive's own full path, which is the ContainerPath of every file inside it. Outside
// any archive (a nil Traversal) it is blank, as for files on the scanned filesystem.
func (t *Traversal) FullTraversalArchivePath() string {
if t == nil {
return ""
}
return t.Location.FullNestedPath()
}
// ContainerPath returns the full path of the archive directly holding the file at c, or blank for a
// file on the scanned filesystem, whether or not its FullPath is set.
func ContainerPath(c file.Coordinates) string {
entries, ok := archiveEntries(c)
if !ok {
return ""
}
if i := strings.LastIndex(entries, file.ArchivePathSeparator); i >= 0 {
return c.RealPath + file.ArchivePathSeparator + entries[:i]
}
return c.RealPath
}
// ContainingArchive returns the coordinates of the archive directly holding the file at c, reporting false
// for a file on the scanned filesystem.
func ContainingArchive(c file.Coordinates) (file.Coordinates, bool) {
containerPath := ContainerPath(c)
if containerPath == "" {
return file.Coordinates{}, false
}
container := file.Coordinates{RealPath: c.RealPath, FileSystemID: c.FileSystemID}
if containerPath != c.RealPath {
container.FullPath = containerPath
}
return container, true
}
// archiveEntries is the "!/"-delimited chain of entry paths after RealPath, reporting false for a file
// not inside an archive. RealPath is split off first, so a "!/" in it is not taken for a boundary.
func archiveEntries(c file.Coordinates) (string, bool) {
remain, ok := strings.CutPrefix(c.FullNestedPath(), c.RealPath)
if !ok {
return "", false
}
return strings.CutPrefix(remain, file.ArchivePathSeparator)
}
// VirtualPath returns the colon-delimited chain of archives from the scan root to loc, ending in loc
// itself as accessed: "app.war:WEB-INF/lib/dep.jar". Outside any archive it is the location's path
// unchanged. In an entry name '%' is escaped as "%25" and then ':' as "%3A", so splitting on ':'
// recovers the archive boundaries and no two entry names share an escaped form. Leading slashes are
// dropped to match the java cataloger, which joins slash-less zip entry names onto the containing
// archive's path.
func VirtualPath(loc file.Location) string {
accessPath := loc.Path()
entries, ok := archiveEntries(loc.Coordinates)
if !ok {
return accessPath
}
segments := strings.Split(entries, file.ArchivePathSeparator)
if rest, ok := strings.CutPrefix(accessPath, ContainerPath(loc.Coordinates)); ok && strings.HasPrefix(rest, file.ArchivePathSeparator) {
accessPath = rest[len(file.ArchivePathSeparator):]
}
segments[len(segments)-1] = accessPath
for i, s := range segments {
segments[i] = strings.TrimPrefix(escapeEntryPath(s), "/")
}
return loc.RealPath + ":" + strings.Join(segments, ":")
}
var entryPathEscaper = strings.NewReplacer("%", "%25", ":", "%3A")
func escapeEntryPath(p string) string {
if !strings.ContainsAny(p, "%:") {
return p
}
return entryPathEscaper.Replace(p)
}
type traversalCtxKey struct{}
func WithTraversal(ctx context.Context, t *Traversal) context.Context {
return context.WithValue(ctx, traversalCtxKey{}, t)
}
// TraversalFromContext returns the archive traversal on the context, or nil outside an archive.
func TraversalFromContext(ctx context.Context) *Traversal {
t, _ := ctx.Value(traversalCtxKey{}).(*Traversal)
return t
}
type budgetCtxKey struct{}
// WithBudget sets the decompression budget that Extract spends from, for this archive and every archive
// extracted beneath it.
func WithBudget(ctx context.Context, b *Budget) context.Context {
return context.WithValue(ctx, budgetCtxKey{}, b)
}
func budgetFromContext(ctx context.Context) *Budget {
b, _ := ctx.Value(budgetCtxKey{}).(*Budget)
return b
}
type nestedCatalogingCtxKey struct{}
// WithNestedCataloging marks the context of a scan whose archive cataloger task will extract every
// archive and run the catalogers inside it. syft.CreateSBOM sets it before any task runs, so a
// cataloger that would otherwise open archives itself can leave them to the task instead.
func WithNestedCataloging(ctx context.Context) context.Context {
return context.WithValue(ctx, nestedCatalogingCtxKey{}, true)
}
// NestedCatalogingEnabled reports whether the archive cataloger task is extracting archives in this scan.
func NestedCatalogingEnabled(ctx context.Context) bool {
enabled, _ := ctx.Value(nestedCatalogingCtxKey{}).(bool)
return enabled
}
+113
View File
@@ -0,0 +1,113 @@
package archive
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/file"
)
func TestTraversalContextRoundTrip(t *testing.T) {
assert.Nil(t, TraversalFromContext(context.Background()))
trav := &Traversal{Location: file.NewLocation("app.war")}
ctx := WithTraversal(context.Background(), trav)
require.Same(t, trav, TraversalFromContext(ctx))
child := &Traversal{Location: locationIn("app.war", "WEB-INF/lib/dep.jar")}
ctx = WithTraversal(ctx, child)
require.Same(t, child, TraversalFromContext(ctx))
}
// locationIn is a location as the Resolver returns it: the outermost archive's real path, and the chain
// of entry paths within it.
func locationIn(realPath, archivePath string) file.Location {
return file.NewLocationFromCoordinates(file.Coordinates{RealPath: realPath, FullPath: realPath + file.ArchivePathSeparator + archivePath})
}
func TestContainerPath(t *testing.T) {
assert.Equal(t, "", ContainerPath(file.NewCoordinates("/app.war", "")))
assert.Equal(t, "", ContainerPath(file.Coordinates{RealPath: "/app.war", FullPath: "/app.war"}), "a top-level file whose FullPath is set")
assert.Equal(t, "", ContainerPath(file.Coordinates{RealPath: "/odd!/name"}), "a separator in a top-level real path is not an archive boundary")
assert.Equal(t, "", ContainerPath(file.Coordinates{RealPath: "/odd!/name", FullPath: "/odd!/name"}), "nor when FullPath is set")
assert.Equal(t, "/odd!/name.zip", ContainerPath(locationIn("/odd!/name.zip", "a.txt").Coordinates))
assert.Equal(t, "/odd!/name.zip!/b.jar", ContainerPath(locationIn("/odd!/name.zip", "b.jar!/a.txt").Coordinates))
assert.Equal(t, "/app.war", ContainerPath(locationIn("/app.war", "WEB-INF/lib/dep.jar").Coordinates))
assert.Equal(t, "/app.war!/WEB-INF/lib/dep.jar",
ContainerPath(locationIn("/app.war", "WEB-INF/lib/dep.jar!/META-INF/MANIFEST.MF").Coordinates))
}
func TestVirtualPath(t *testing.T) {
tests := []struct {
name string
location file.Location
expected string
}{
{
name: "outside an archive returns the path unchanged",
location: file.NewLocation("/WEB-INF/lib/dep.jar"),
expected: "/WEB-INF/lib/dep.jar",
},
{
name: "outside an archive preserves a relative path",
location: file.NewLocation("some/dir/example.jar"),
expected: "some/dir/example.jar",
},
{
name: "outside an archive with FullPath set returns the path unchanged",
location: file.NewLocationFromCoordinates(file.Coordinates{RealPath: "/odd!/name.jar", FullPath: "/odd!/name.jar"}),
expected: "/odd!/name.jar",
},
{
name: "single level trims leading slash",
location: locationIn("app.war", "/WEB-INF/lib/dep.jar"),
expected: "app.war:WEB-INF/lib/dep.jar",
},
{
name: "single level with slash-less entry",
location: locationIn("app.war", "WEB-INF/lib/dep.jar"),
expected: "app.war:WEB-INF/lib/dep.jar",
},
{
name: "multi-level chain",
location: locationIn("app.war", "WEB-INF/lib/dep.jar!/inner/most.jar"),
expected: "app.war:WEB-INF/lib/dep.jar:inner/most.jar",
},
{
// a colon in the entry name must not be mistaken for a chain delimiter
name: "colon in entry path is escaped",
location: locationIn("app.war", "WEB-INF/lib/dep:special.jar"),
expected: "app.war:WEB-INF/lib/dep%3Aspecial.jar",
},
{
name: "every colon in the entry path is escaped",
location: locationIn("app.war", "lib/a:b:c.jar"),
expected: "app.war:lib/a%3Ab%3Ac.jar",
},
{
// or the literal text %3A would collide with an escaped colon
name: "the escape character is itself escaped",
location: locationIn("outer.zip", "a%3Ab.jar"),
expected: "outer.zip:a%253Ab.jar",
},
{
name: "access path is preferred over real path",
location: file.NewVirtualLocation("/real/dep.jar", "/link/dep.jar"),
expected: "/link/dep.jar",
},
{
name: "access path within an archive replaces the last entry",
location: file.NewVirtualLocationFromCoordinates(
file.Coordinates{RealPath: "app.war", FullPath: "app.war!/WEB-INF/lib/real.jar"}, "app.war!/WEB-INF/lib/link.jar"),
expected: "app.war:WEB-INF/lib/link.jar",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.expected, VirtualPath(tt.location))
})
}
}
+79 -1
View File
@@ -1,6 +1,9 @@
package bus
import (
"context"
"sync"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
@@ -36,7 +39,82 @@ func Notify(message string) {
})
}
func StartCatalogerTask(info monitor.GenericTask, size int64, initialStage string) *monitor.TaskProgress {
type catalogerTaskRegistryKey struct{}
// catalogerTaskRegistry remembers the progress row each cataloger publishes. A cataloger runs once
// over the scan root and again inside every archive the archive cataloger walks; a re-run reports
// into the row it already owns rather than publishing another, and republishing an ID would replace
// the live row with one the UI never sees completed.
type catalogerTaskRegistry struct {
mu sync.Mutex
rows map[string]*monitor.TaskProgress
}
// WithCatalogerTaskRegistry returns a context under which each cataloger publishes one progress row
// however many times it runs. A context that already carries a registry is returned unchanged.
func WithCatalogerTaskRegistry(ctx context.Context) context.Context {
if catalogerTaskRegistryFromContext(ctx) != nil {
return ctx
}
return context.WithValue(ctx, catalogerTaskRegistryKey{}, &catalogerTaskRegistry{
rows: make(map[string]*monitor.TaskProgress),
})
}
func catalogerTaskRegistryFromContext(ctx context.Context) *catalogerTaskRegistry {
reg, _ := ctx.Value(catalogerTaskRegistryKey{}).(*catalogerTaskRegistry)
return reg
}
// catalogerTaskKey identifies the row a task owns: its ID when it has one, otherwise its parent and
// title (file catalogers publish no ID).
func catalogerTaskKey(info monitor.GenericTask) string {
if info.ID != "" {
return "id:" + info.ID
}
return "title:" + info.ParentID + "/" + info.Title.Default
}
func StartCatalogerTask(ctx context.Context, info monitor.GenericTask, size int64, initialStage string) *monitor.TaskProgress {
reg := catalogerTaskRegistryFromContext(ctx)
if reg == nil {
return publishCatalogerTask(info, size, initialStage)
}
reg.mu.Lock()
defer reg.mu.Unlock()
key := catalogerTaskKey(info)
if row, ok := reg.rows[key]; ok {
resumeCatalogerTask(row, size, initialStage)
return row
}
t := publishCatalogerTask(info, size, initialStage)
reg.rows[key] = t
return t
}
// resumeCatalogerTask reopens a row an earlier run completed and grows its total by what this run
// adds, so the row does not read as finished mid-walk. A row carrying a real failure keeps it.
func resumeCatalogerTask(row *monitor.TaskProgress, size int64, initialStage string) {
if size < 0 || row.Size() < 0 {
row.SetTotal(-1)
} else {
row.SetTotal(row.Size() + size)
}
if progress.IsErrCompleted(row.Error()) {
row.SetError(nil)
}
if initialStage != "" {
row.AtomicStage.Set(initialStage)
}
}
func publishCatalogerTask(info monitor.GenericTask, size int64, initialStage string) *monitor.TaskProgress {
t := &monitor.TaskProgress{
AtomicStage: progress.NewAtomicStage(initialStage),
Manual: progress.NewManual(size),
+152
View File
@@ -0,0 +1,152 @@
package bus
import (
"context"
"errors"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/wagoodman/go-progress"
"github.com/anchore/syft/syft/event/monitor"
)
func digestsTask() monitor.GenericTask {
return monitor.GenericTask{
Title: monitor.Title{Default: "File digests"},
ParentID: monitor.TopLevelCatalogingTaskID,
}
}
func TestStartCatalogerTask_withoutARegistryEveryRunPublishes(t *testing.T) {
// the right answer for a caller driving a single pass: nothing re-runs, so nothing should be reused
ctx := context.Background()
first := StartCatalogerTask(ctx, digestsTask(), 2, "")
second := StartCatalogerTask(ctx, digestsTask(), 3, "")
assert.NotSame(t, first, second)
assert.Equal(t, int64(2), first.Size())
assert.Equal(t, int64(3), second.Size())
}
func TestStartCatalogerTask_reRunReportsIntoTheRowItAlreadyOwns(t *testing.T) {
ctx := WithCatalogerTaskRegistry(context.Background())
first := StartCatalogerTask(ctx, digestsTask(), 2, "")
first.Increment()
first.Increment()
first.SetCompleted()
// the archive walk reaches the same cataloger again, over an archive's contents this time
second := StartCatalogerTask(ctx, digestsTask(), 3, "")
require.Same(t, first, second, "a re-run must find the row it already published")
assert.Equal(t, int64(5), second.Size(), "the total must cover both runs, not just the first")
assert.False(t, progress.IsCompleted(second), "a row with work still to do must not read as complete")
for i := 0; i < 3; i++ {
second.Increment()
}
second.SetCompleted()
assert.Equal(t, int64(5), second.Current())
assert.Equal(t, int64(5), second.Size(), "the count must not run past the total")
assert.True(t, progress.IsCompleted(second))
}
func TestStartCatalogerTask_aTotalResolvedOnCompletionIsAddable(t *testing.T) {
// a run that could not say how much it would do says so when it signs off (Manual.SetCompleted
// backfills the total from the count), so by the time a re-run arrives there is a real number to
// add to. This is the file metadata cataloger's shape: it publishes -1 every run.
ctx := WithCatalogerTaskRegistry(context.Background())
row := StartCatalogerTask(ctx, digestsTask(), -1, "")
row.Increment()
row.SetCompleted()
require.Equal(t, int64(1), row.Size(), "signing off must resolve the total to what the run did")
require.Same(t, row, StartCatalogerTask(ctx, digestsTask(), -1, ""))
assert.Equal(t, int64(-1), row.Size(), "a re-run that cannot say its own size makes the row indeterminate again")
row.Increment()
row.SetCompleted()
assert.Equal(t, int64(2), row.Current())
assert.Equal(t, int64(2), row.Size(), "and signing off resolves it again, over both runs")
}
func TestStartCatalogerTask_indeterminateTotalsStayIndeterminate(t *testing.T) {
// a run whose size is unknown cannot be added to one that is known
tests := []struct {
name string
first, next int64
complete func(row *monitor.TaskProgress)
}{
{
name: "the re-run cannot say its own size",
first: 2, next: -1,
complete: func(row *monitor.TaskProgress) { row.Increment(); row.SetCompleted() },
},
{
name: "the first run did nothing, so completing resolved nothing",
first: -1, next: 3,
complete: func(row *monitor.TaskProgress) { row.SetCompleted() },
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ctx := WithCatalogerTaskRegistry(context.Background())
row := StartCatalogerTask(ctx, digestsTask(), tt.first, "")
tt.complete(row)
require.Same(t, row, StartCatalogerTask(ctx, digestsTask(), tt.next, ""))
assert.Equal(t, int64(-1), row.Size())
assert.False(t, progress.IsCompleted(row), "an indeterminate row is not complete until it says so")
})
}
}
func TestStartCatalogerTask_aFailedRowKeepsItsFailure(t *testing.T) {
// a later run finding nothing wrong does not mean the earlier one succeeded
ctx := WithCatalogerTaskRegistry(context.Background())
failure := errors.New("unable to read the thing")
row := StartCatalogerTask(ctx, digestsTask(), 2, "")
row.SetError(failure)
require.Same(t, row, StartCatalogerTask(ctx, digestsTask(), 3, ""))
assert.ErrorIs(t, row.Error(), failure)
}
func TestStartCatalogerTask_rowsAreKeyedByTaskIdentity(t *testing.T) {
ctx := WithCatalogerTaskRegistry(context.Background())
byID := func(id string) monitor.GenericTask {
return monitor.GenericTask{ID: id, Title: monitor.Title{Default: "shared title"}}
}
assert.Same(t, StartCatalogerTask(ctx, byID("a"), -1, ""), StartCatalogerTask(ctx, byID("a"), -1, ""))
assert.NotSame(t, StartCatalogerTask(ctx, byID("a"), -1, ""), StartCatalogerTask(ctx, byID("b"), -1, ""),
"an ID is the identity where there is one, even when two tasks share a title")
// a task publishing without an ID, as every file cataloger does, is keyed by where it sits and what
// it is called
underCataloging := monitor.GenericTask{Title: monitor.Title{Default: "Files"}, ParentID: "cataloging"}
underSomethingElse := monitor.GenericTask{Title: monitor.Title{Default: "Files"}, ParentID: "other"}
assert.Same(t, StartCatalogerTask(ctx, underCataloging, -1, ""), StartCatalogerTask(ctx, underCataloging, -1, ""))
assert.NotSame(t, StartCatalogerTask(ctx, underCataloging, -1, ""), StartCatalogerTask(ctx, underSomethingElse, -1, ""))
}
func TestWithCatalogerTaskRegistry_isIdempotent(t *testing.T) {
// anything that re-runs catalogers may ask for a registry; a second one would hold none of the rows
// already published and every re-run would publish again
ctx := WithCatalogerTaskRegistry(context.Background())
row := StartCatalogerTask(ctx, digestsTask(), 2, "")
nested := WithCatalogerTaskRegistry(ctx)
assert.Same(t, row, StartCatalogerTask(nested, digestsTask(), 2, ""))
}
+4
View File
@@ -6,6 +6,7 @@ import (
"net/url"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"time"
@@ -95,6 +96,9 @@ func Test_makeDiskKey(t *testing.T) {
}
func Test_errors(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("windows ignores unix permission bits on directories")
}
tmp := t.TempDir()
cache := filepath.Join(tmp, "cache")
// make a non-writable directory
@@ -9,6 +9,7 @@ import (
"os"
"path/filepath"
"regexp"
"slices"
"strings"
"github.com/anchore/syft/internal/capabilities/internal"
@@ -31,6 +32,10 @@ type ConfigInfo struct {
var appConfigAnnotationPattern = regexp.MustCompile(`^//\s*app-config:\s*(.+)$`)
// embeddedFieldsAnnotationPattern matches "// capabilities-fields: A, B" on an embedded struct, which limits
// the embedded fields listed to the ones named (for a shared type the cataloger reads only part of).
var embeddedFieldsAnnotationPattern = regexp.MustCompile(`^//\s*capabilities-fields:\s*(.+)$`)
// DiscoverConfigs walks the cataloger directory and discovers all configuration structs
// Returns map where key is "packageName.StructName" (e.g., "golang.CatalogerConfig")
func DiscoverConfigs(repoRoot string) (map[string]ConfigInfo, error) {
@@ -160,6 +165,9 @@ func extractCatalogerConfigFieldsRecursive(structType *ast.StructType, visitedTy
if len(field.Names) == 0 {
// this is an embedded field - resolve and extract its fields
embeddedFields := resolveEmbeddedStructFields(field.Type, visitedTypes)
if only := embeddedFieldAllowlist(field.Doc); only != nil {
embeddedFields = slices.DeleteFunc(embeddedFields, func(f ConfigField) bool { return !only[f.Name] })
}
fields = append(fields, embeddedFields...)
continue
}
@@ -195,6 +203,26 @@ func extractCatalogerConfigFieldsRecursive(structType *ast.StructType, visitedTy
return fields
}
// embeddedFieldAllowlist returns the field names from a "capabilities-fields:" annotation, or nil when there
// is none (every exported embedded field is listed).
func embeddedFieldAllowlist(commentGroup *ast.CommentGroup) map[string]bool {
if commentGroup == nil {
return nil
}
for _, comment := range commentGroup.List {
matches := embeddedFieldsAnnotationPattern.FindStringSubmatch(comment.Text)
if len(matches) < 2 {
continue
}
only := make(map[string]bool)
for _, name := range strings.Split(matches[1], ",") {
only[strings.TrimSpace(name)] = true
}
return only
}
return nil
}
// resolveEmbeddedStructFields resolves an embedded struct type and extracts its fields recursively
func resolveEmbeddedStructFields(fieldType ast.Expr, visitedTypes map[string]bool) []ConfigField {
// extract the type name from the expression
@@ -457,3 +457,32 @@ func TestDiscoverAllowedConfigStructs(t *testing.T) {
t.Logf(" - %s", configType)
}
}
func TestEmbeddedFieldAllowlist(t *testing.T) {
comments := func(lines ...string) *ast.CommentGroup {
g := &ast.CommentGroup{}
for _, l := range lines {
g.List = append(g.List, &ast.Comment{Text: l})
}
return g
}
tests := []struct {
name string
doc *ast.CommentGroup
want map[string]bool
}{
{name: "no doc lists every field", doc: nil, want: nil},
{name: "no annotation lists every field", doc: comments("// Deprecated: use something else."), want: nil},
{
name: "annotation limits to the named fields",
doc: comments("// Deprecated: use something else.", "// capabilities-fields: IncludeIndexedArchives, IncludeUnindexedArchives"),
want: map[string]bool{"IncludeIndexedArchives": true, "IncludeUnindexedArchives": true},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
require.Equal(t, tt.want, embeddedFieldAllowlist(tt.doc))
})
}
}
@@ -212,7 +212,7 @@ func parseGenericCatalogerFunction(funcDecl *ast.FuncDecl, filePath, repoRoot st
// extractPackageNameFromPath extracts the package name from a cataloger source file path
// e.g., "syft/pkg/cataloger/swift/cataloger.go" -> "swift"
func extractPackageNameFromPath(filePath string) string {
parts := strings.Split(filePath, string(filepath.Separator))
parts := strings.Split(filepath.ToSlash(filePath), "/")
// find the LAST occurrence of "cataloger" in the path
// (to handle test fixtures with multiple "cataloger" segments)
@@ -4,6 +4,7 @@ import (
"go/ast"
"go/parser"
"go/token"
"path/filepath"
"testing"
"github.com/stretchr/testify/require"
@@ -337,7 +338,7 @@ func TestResolveImportPath(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := resolveImportPath(tt.importPath, testRepoRoot)
require.Equal(t, tt.want, got)
require.Equal(t, filepath.FromSlash(tt.want), got)
})
}
}
+1 -1
View File
@@ -69,5 +69,5 @@ func locationSorter(a, b file.Location) int {
return strings.Compare(a.AccessPath, b.AccessPath)
}
return strings.Compare(a.RealPath, b.RealPath)
return strings.Compare(a.FullNestedPath(), b.FullNestedPath())
}
+1 -1
View File
@@ -14,5 +14,5 @@ func DefaultLocationComparer(x, y file.Location) bool {
}
func LocationComparerWithoutLayer(x, y file.Location) bool {
return cmp.Equal(x.RealPath, y.RealPath) && cmp.Equal(x.AccessPath, y.AccessPath)
return cmp.Equal(x.FullNestedPath(), y.FullNestedPath()) && cmp.Equal(x.AccessPath, y.AccessPath)
}
+3 -1
View File
@@ -3,7 +3,7 @@ package internal
const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "16.1.11"
JSONSchemaVersion = "16.1.13"
// Changelog
// 16.1.0 - reformulated the python pdm fields (added "URL" and removed the unused "path" field).
@@ -18,4 +18,6 @@ const (
// 16.1.9 - add Symbols (grouped by owning package import path) to GolangBinaryBuildinfoEntry metadata
// 16.1.10 - add SafeTensorsModelInfo metadata type for the safetensors AI model cataloger
// 16.1.11 - add CpanDistribution and CpanUnpackedRelease metadata types for the perl catalogers
// 16.1.12 - add fullPath to file coordinates, naming the nested archive chain a file was found in (blank for files not within an archive)
// 16.1.13 - add executable identifiers (ELF GNU and Go build IDs)
)
+6 -6
View File
@@ -6,13 +6,13 @@ import (
"io"
)
const perFileReadLimit = 2 * GB
// PerFileReadLimit is the most read from any one archive entry, guarding against decompression bombs.
const PerFileReadLimit = 2 * GB
// safeCopy limits the copy from the reader. This is useful when extracting files from archives to
// protect against decompression bomb attacks.
func safeCopy(writer io.Writer, reader io.Reader) error {
numBytes, err := io.Copy(writer, io.LimitReader(reader, perFileReadLimit))
if numBytes >= perFileReadLimit {
// SafeCopy bounds the copy to guard against decompression bombs when extracting from archives.
func SafeCopy(writer io.Writer, reader io.Reader) error {
numBytes, err := io.Copy(writer, io.LimitReader(reader, PerFileReadLimit))
if numBytes >= PerFileReadLimit {
return fmt.Errorf("zip read limit hit (potential decompression bomb attack)")
}
// Propagate decompression / read errors up to the caller. io.Copy
+4 -4
View File
@@ -29,7 +29,7 @@ func (r *errReader) Read(p []byte) (int, error) {
func TestSafeCopy(t *testing.T) {
t.Run("clean copy returns nil", func(t *testing.T) {
var buf bytes.Buffer
if err := safeCopy(&buf, strings.NewReader("hello")); err != nil {
if err := SafeCopy(&buf, strings.NewReader("hello")); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got := buf.String(); got != "hello" {
@@ -38,12 +38,12 @@ func TestSafeCopy(t *testing.T) {
})
t.Run("propagates decompression error", func(t *testing.T) {
// #4806: safeCopy used to drop non-EOF errors, so the caller
// #4806: SafeCopy used to drop non-EOF errors, so the caller
// would persist a partial buffer as a successful extract and
// downstream catalogers silently read empty manifests.
sentinel := errors.New("flate: corrupt input before offset 42")
var buf bytes.Buffer
err := safeCopy(&buf, &errReader{data: []byte("partial"), err: sentinel})
err := SafeCopy(&buf, &errReader{data: []byte("partial"), err: sentinel})
if err == nil {
t.Fatalf("expected error to be returned, got nil")
}
@@ -56,7 +56,7 @@ func TestSafeCopy(t *testing.T) {
// The old code had a dead io.EOF branch that labelled clean
// reads as decompression bombs; keep the happy path clean.
var buf bytes.Buffer
err := safeCopy(&buf, io.LimitReader(strings.NewReader("abc"), 3))
err := SafeCopy(&buf, io.LimitReader(strings.NewReader("abc"), 3))
if err != nil {
t.Fatalf("unexpected error on clean copy: %v", err)
}
+4 -3
View File
@@ -4,7 +4,7 @@ import (
"errors"
"io/fs"
"os"
"path/filepath"
"path"
"strings"
"github.com/diskfs/go-diskfs/filesystem"
@@ -25,7 +25,8 @@ func WalkDiskDir(fsys filesystem.FileSystem, root string, fn WalkDiskDirFunc) er
}
for _, info := range infos {
p := filepath.Join(root, info.Name())
// squashfs paths are posix regardless of host os
p := path.Join(root, info.Name())
err = walkDiskDir(fsys, p, info, fn)
if err != nil {
if errors.Is(err, fs.SkipDir) {
@@ -74,7 +75,7 @@ func walkDiskDir(fsys filesystem.FileSystem, name string, d os.FileInfo, walkDir
}
for _, d1 := range dirs {
name1 := filepath.Join(name, d1.Name())
name1 := path.Join(name, d1.Name())
if err := walkDiskDir(fsys, name1, d1, walkDirFn); err != nil {
if errors.Is(err, fs.SkipDir) {
break
+2
View File
@@ -19,6 +19,8 @@ func createTestFS(t *testing.T) filesystem.FileSystem {
filename := "test.squashfs"
f, err := os.Create(filepath.Join(dir, filename))
require.NoError(t, err)
// windows can't remove the temp dir while the backing file is open
t.Cleanup(func() { _ = f.Close() })
b := file.New(f, false)
fsys, err := squashfs.Create(b, 0, 0, 4096)
+3 -2
View File
@@ -70,7 +70,7 @@ func ExtractGlobsFromTarToUniqueTempFile(ctx context.Context, archivePath, dir s
}
defer internal.CloseAndLogError(packedFile, archivePath)
if err := safeCopy(tempFile, packedFile); err != nil {
if err := SafeCopy(tempFile, packedFile); err != nil {
return fmt.Errorf("unable to copy source=%q for tar=%q: %w", file.Name(), archivePath, err)
}
@@ -84,7 +84,8 @@ func ExtractGlobsFromTarToUniqueTempFile(ctx context.Context, archivePath, dir s
func matchesAnyGlob(name string, globs ...string) bool {
for _, glob := range globs {
if matches, err := doublestar.PathMatch(glob, name); err == nil && matches {
// tar entry names are always posix, so don't match with the host separator (PathMatch)
if matches, err := doublestar.Match(glob, name); err == nil && matches {
return true
}
}
+3 -3
View File
@@ -101,7 +101,7 @@ func ExtractFromZipToUniqueTempFile(ctx context.Context, archivePath, dir string
return fmt.Errorf("unable to extract directories, only files: %s", file.NameInArchive)
}
if err := safeCopy(tempFile, zippedFile); err != nil {
if err := SafeCopy(tempFile, zippedFile); err != nil {
return fmt.Errorf("unable to copy source=%q for zip=%q: %w", file.NameInArchive, archivePath, err)
}
@@ -138,7 +138,7 @@ func ContentsFromZip(ctx context.Context, archivePath string, paths ...string) (
}
var buffer bytes.Buffer
if err := safeCopy(&buffer, zippedFile); err != nil {
if err := SafeCopy(&buffer, zippedFile); err != nil {
return fmt.Errorf("unable to copy source=%q for zip=%q: %w", file.NameInArchive, archivePath, err)
}
@@ -210,7 +210,7 @@ func extractSingleFile(file archives.FileInfo, expandedFilePath, archivePath str
}
}()
if err := safeCopy(outputFile, zippedFile); err != nil {
if err := SafeCopy(outputFile, zippedFile); err != nil {
return fmt.Errorf("unable to copy source=%q to dest=%q for zip=%q: %w", file.NameInArchive, outputFile.Name(), archivePath, err)
}
}
+17
View File
@@ -0,0 +1,17 @@
package internal
import "context"
type eagerLayerSquashesKey struct{}
// WithEagerLayerSquashes has the image source provider build every layer's squashed tree while it reads the
// image, instead of the first time each one is used. Set it when the scan will read lower layers (any scope but
// squashed): eager squashing overlaps with fetching and indexing the layers, so those scans finish sooner.
func WithEagerLayerSquashes(ctx context.Context) context.Context {
return context.WithValue(ctx, eagerLayerSquashesKey{}, true)
}
func EagerLayerSquashes(ctx context.Context) bool {
v, _ := ctx.Value(eagerLayerSquashesKey{}).(bool)
return v
}
+6 -6
View File
@@ -49,7 +49,7 @@ func findRhelFeatures(resolver file.Resolver, release *linux.Release) error {
for _, contentManifestFile := range contentManifestFiles {
found, err := fileMatchesAny(resolver, contentManifestFile, rhelEUSPatterns...)
if err != nil {
return fmt.Errorf("unable to read content manifest from %s: %w", contentManifestFile.RealPath, err)
return fmt.Errorf("unable to read content manifest from %s: %w", contentManifestFile.FullNestedPath(), err)
}
if found {
release.ExtendedSupport = true
@@ -163,9 +163,9 @@ func hasActiveUbuntuESMService(resolver file.Resolver) (bool, error) {
func hasEnabledESMServiceInStatus(resolver file.Resolver, location file.Location) (bool, error) {
contents, err := resolver.FileContentsByLocation(location)
if err != nil {
return false, fmt.Errorf("unable to read ubuntu-advantage status from %s: %w", location.RealPath, err)
return false, fmt.Errorf("unable to read ubuntu-advantage status from %s: %w", location.FullNestedPath(), err)
}
defer internal.CloseAndLogError(contents, location.RealPath)
defer internal.CloseAndLogError(contents, location.FullNestedPath())
var status struct {
Services []struct {
@@ -174,7 +174,7 @@ func hasEnabledESMServiceInStatus(resolver file.Resolver, location file.Location
} `json:"services"`
}
if err := json.NewDecoder(contents).Decode(&status); err != nil {
return false, fmt.Errorf("unable to parse ubuntu-advantage status from %s: %w", location.RealPath, err)
return false, fmt.Errorf("unable to parse ubuntu-advantage status from %s: %w", location.FullNestedPath(), err)
}
// pro status values are enabled/disabled/n-a/warning/—; only "enabled" proves the stream is active.
@@ -200,8 +200,8 @@ func hasInstalledESMPackage(packages *pkg.Collection) bool {
func fileMatchesAny(resolver file.Resolver, location file.Location, patterns ...*regexp.Regexp) (bool, error) {
contents, err := resolver.FileContentsByLocation(location)
if err != nil {
return false, fmt.Errorf("unable to read %s: %w", location.RealPath, err)
return false, fmt.Errorf("unable to read %s: %w", location.FullNestedPath(), err)
}
defer internal.CloseAndLogError(contents, location.RealPath)
defer internal.CloseAndLogError(contents, location.FullNestedPath())
return internal.MatchAnyFromReader(contents, patterns...)
}
+2
View File
@@ -16,6 +16,7 @@ import (
"github.com/anchore/syft/internal/os"
"github.com/anchore/syft/internal/sbomsync"
"github.com/anchore/syft/internal/task"
"github.com/anchore/syft/internal/testutils"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/linux"
@@ -84,6 +85,7 @@ func Test_EnvironmentTask(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
testutils.SkipWithoutLinuxContainers(t)
tarPath := imagetest.GetFixtureImageTarPath(t, test.name)
// get the source
+8
View File
@@ -13,6 +13,14 @@ func Set(s redact.Store) {
store = s
}
// Reset clears the global redaction store. It must only be called once a command
// run has fully finished (nothing left to Add or Apply), so that a later run can
// Set its own store. Calling it while a run is in flight would let a second Set
// split secrets across two stores and leak the ones in the dropped store.
func Reset() {
store = nil
}
func Get() redact.Store {
return store
}
@@ -71,7 +71,7 @@ func compareElfBinaryPackages(s *sbom.SBOM) []artifact.ID {
pkgsToDelete := make([]artifact.ID, 0)
for _, elfPkg := range allElfPackages(s) {
for _, loc := range onlyPrimaryEvidenceLocations(elfPkg) {
for _, otherPkg := range s.Artifacts.Packages.PackagesByPath(loc.RealPath) {
for _, otherPkg := range s.Artifacts.Packages.PackagesByPath(loc.FullNestedPath()) {
// we only care about comparing binary packages to each other (not other types)
if otherPkg.Type != pkg.BinaryPkg {
continue
@@ -130,7 +130,7 @@ func populateRelationships(exec file.Executable, parentPkg pkg.Package, resolver
for _, loc := range libLocations {
// are you in our index?
realBaseName := path.Base(loc.RealPath)
realBaseName := path.Base(loc.FullNestedPath())
pkgCollection := index.owningLibraryPackage(realBaseName)
if pkgCollection.PackageCount() < 1 {
addRelationship(
@@ -3,6 +3,7 @@ package binary
import (
"path"
"github.com/anchore/syft/internal/archive"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/sbomsync"
"github.com/anchore/syft/syft/file"
@@ -63,7 +64,7 @@ func locationsThatProvideLibraries(accessor sbomsync.Accessor) (map[string]file.
continue
}
basename := path.Base(coord.RealPath)
basename := path.Base(coord.FullNestedPath())
set := libLocationsByBasename[basename]
set.Add(coord)
allLibLocations.Add(coord)
@@ -82,6 +83,11 @@ func packagesWithLibraryOwnership(resolver file.Resolver, accessor sbomsync.Acce
var ownedFilePaths []string
if p.Type == pkg.BinaryPkg {
for _, loc := range p.Locations.ToSlice() {
if archive.ContainerPath(loc.Coordinates) != "" {
// a path within an archive cannot be resolved from the scan root; the location is the library
addLibraryOwner(packagesByLibraryPath, allLibLocations, loc.Coordinates, p)
continue
}
ownedFilePaths = append(ownedFilePaths, loc.Path())
}
} else {
@@ -114,18 +120,19 @@ func populatePackagesByLibraryPath(
}
for _, loc := range ownedLocation {
// if the location is a library, add the package to the set of packages that own the library
if !allLibLocations.Contains(loc.Coordinates) {
continue
}
if _, ok := packagesByLibraryPath[loc.Coordinates]; !ok {
packagesByLibraryPath[loc.Coordinates] = pkg.NewCollection()
}
// we have a library path, add the package to the set of packages that own the library
packagesByLibraryPath[loc.Coordinates].Add(p)
addLibraryOwner(packagesByLibraryPath, allLibLocations, loc.Coordinates, p)
}
}
return packagesByLibraryPath
}
// addLibraryOwner records p as an owner of the file at coordinates when that file is a library.
func addLibraryOwner(packagesByLibraryPath map[file.Coordinates]*pkg.Collection, allLibLocations file.CoordinateSet, coordinates file.Coordinates, p pkg.Package) {
if !allLibLocations.Contains(coordinates) {
return
}
if _, ok := packagesByLibraryPath[coordinates]; !ok {
packagesByLibraryPath[coordinates] = pkg.NewCollection()
}
packagesByLibraryPath[coordinates].Add(p)
}
+20 -2
View File
@@ -6,6 +6,7 @@ import (
"github.com/bmatcuk/doublestar/v4"
"github.com/scylladb/go-set/strset"
"github.com/anchore/syft/internal/archive"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/sbomsync"
"github.com/anchore/syft/syft/artifact"
@@ -100,7 +101,7 @@ func findOwnershipByFilesRelationships(resolver file.Resolver, catalog *pkg.Coll
// Now establish relationships, considering symlinks
for _, candidateOwnerPkg := range catalog.Sorted() {
id := candidateOwnerPkg.ID()
if candidateOwnerPkg.Metadata == nil {
if candidateOwnerPkg.Metadata == nil || withinArchive(candidateOwnerPkg) {
continue
}
@@ -164,7 +165,7 @@ func directOwnersByPath(catalog *pkg.Collection) map[pkg.Type]*strset.Set {
// First, identify direct ownership of all files
for _, p := range catalog.Sorted() {
if p.Metadata == nil {
if p.Metadata == nil || withinArchive(p) {
continue
}
@@ -192,6 +193,21 @@ func directOwnersByPath(catalog *pkg.Collection) map[pkg.Type]*strset.Set {
return directOwnership
}
// withinArchive reports whether every location of p is inside an archive. Such a package's owned files
// are paths within its archive, not the scan root, so it is not treated as a file owner.
func withinArchive(p pkg.Package) bool {
locations := p.Locations.ToSlice()
if len(locations) == 0 {
return false
}
for _, l := range locations {
if archive.ContainerPath(l.Coordinates) == "" {
return false
}
}
return true
}
func resolvePaths(ownedFilePath string, resolver file.Resolver) []string {
// though we have a string path, we need to resolve symlinks and other filesystem oddities since we cannot assume this is a real path
var locs []file.Location
@@ -206,6 +222,8 @@ func resolvePaths(ownedFilePath string, resolver file.Resolver) []string {
ownedFilePaths := strset.New(ownedFilePath)
for _, loc := range locs {
// Owned file paths are currently based on the top-level resolver, which is always the RealPath and includes
// all files found within archives at that location
ownedFilePaths.Add(loc.RealPath)
}
return ownedFilePaths.List()
@@ -78,6 +78,56 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
return []pkg.Package{parent, child}, []artifact.Relationship{relationship}
},
},
{
name: "package-within-archive-is-not-a-file-owner",
setup: func(t testing.TB) ([]pkg.Package, []artifact.Relationship) {
nestedOwner := pkg.Package{
Locations: file.NewLocationSet(file.NewLocationFromCoordinates(file.Coordinates{
RealPath: "/rootfs.tar",
FullPath: "/rootfs.tar!/var/lib/rpm/Packages",
})),
Type: pkg.RpmPkg,
Metadata: pkg.RpmDBEntry{
Files: []pkg.RpmFileRecord{{Path: "/d/path"}},
},
}
nestedOwner.SetID()
child := pkg.Package{
Locations: file.NewLocationSet(file.NewLocation("/d/path")),
Type: pkg.BinaryPkg,
}
child.SetID()
return []pkg.Package{nestedOwner, child}, nil
},
},
{
// the archive's own package (when it has one) is found by its path; what is inside it is
// linked to that package by CONTAINS, not owned by whoever owns the archive file
name: "top-level-owner-of-an-archive-does-not-own-packages-within-it",
setup: func(t testing.TB) ([]pkg.Package, []artifact.Relationship) {
parent := pkg.Package{
Locations: file.NewLocationSet(file.NewLocation("/var/lib/rpm/Packages")),
Type: pkg.RpmPkg,
Metadata: pkg.RpmDBEntry{
Files: []pkg.RpmFileRecord{{Path: "/opt/bundle.tar"}},
},
}
parent.SetID()
child := pkg.Package{
Locations: file.NewLocationSet(file.NewLocationFromCoordinates(file.Coordinates{
RealPath: "/opt/bundle.tar",
FullPath: "/opt/bundle.tar!/lib/thing.jar",
})),
Type: pkg.JavaPkg,
}
child.SetID()
return []pkg.Package{parent, child}, nil
},
},
{
name: "only-real-file-owner-gets-ownership-relationship-not-symlink-owner",
resolver: mockFR{
+4 -16
View File
@@ -15,18 +15,9 @@ func TestRelationshipsEvidentBy(t *testing.T) {
c := pkg.NewCollection()
coordA := file.Coordinates{
RealPath: "/somewhere/real",
FileSystemID: "abc",
}
coordC := file.Coordinates{
RealPath: "/somewhere/real",
FileSystemID: "abc",
}
coordD := file.Coordinates{
RealPath: "/somewhere/real",
FileSystemID: "abc",
}
coordA := file.NewCoordinates("/somewhere/real", "abc")
coordC := file.NewCoordinates("/somewhere/real", "abc")
coordD := file.NewCoordinates("/somewhere/real", "abc")
pkgA := pkg.Package{
Locations: file.NewLocationSet(
// added!
@@ -39,10 +30,7 @@ func TestRelationshipsEvidentBy(t *testing.T) {
pkgA.SetID()
c.Add(pkgA)
coordB := file.Coordinates{
RealPath: "/somewhere-else/real",
FileSystemID: "def",
}
coordB := file.NewCoordinates("/somewhere-else/real", "def")
pkgB := pkg.Package{
Locations: file.NewLocationSet(
// added!
+10 -22
View File
@@ -21,12 +21,8 @@ func Test_Index(t *testing.T) {
p3 := pkg.Package{
Name: "pkg-3",
}
c1 := file.Coordinates{
RealPath: "/coords/1",
}
c2 := file.Coordinates{
RealPath: "/coords/2",
}
c1 := file.NewCoordinates("/coords/1", "")
c2 := file.NewCoordinates("/coords/2", "")
for _, p := range []*pkg.Package{&p1, &p2, &p3} {
p.SetID()
@@ -156,18 +152,10 @@ func Test_Coordinates(t *testing.T) {
p3 := pkg.Package{
Name: "pkg-3",
}
c1 := file.Coordinates{
RealPath: "/coords/1",
}
c2 := file.Coordinates{
RealPath: "/coords/2",
}
c3 := file.Coordinates{
RealPath: "/coords/3",
}
c4 := file.Coordinates{
RealPath: "/coords/4",
}
c1 := file.NewCoordinates("/coords/1", "")
c2 := file.NewCoordinates("/coords/2", "")
c3 := file.NewCoordinates("/coords/3", "")
c4 := file.NewCoordinates("/coords/4", "")
for _, p := range []*pkg.Package{&p1, &p2, &p3} {
p.SetID()
@@ -237,10 +225,10 @@ func TestRemove(t *testing.T) {
p1 := pkg.Package{Name: "pkg-1"}
p2 := pkg.Package{Name: "pkg-2"}
p3 := pkg.Package{Name: "pkg-3"}
c1 := file.Coordinates{RealPath: "/coords/1"}
c2 := file.Coordinates{RealPath: "/coords/2"}
c3 := file.Coordinates{RealPath: "/coords/3"}
c4 := file.Coordinates{RealPath: "/coords/4"}
c1 := file.NewCoordinates("/coords/1", "")
c2 := file.NewCoordinates("/coords/2", "")
c3 := file.NewCoordinates("/coords/3", "")
c4 := file.NewCoordinates("/coords/4", "")
for _, p := range []*pkg.Package{&p1, &p2, &p3} {
p.SetID()
+34 -34
View File
@@ -2131,40 +2131,40 @@ var urlToLicense = map[string]string{
"raw.githubusercontent.com/NTIA/scos-sensor/refs/heads/master/LICENSE.md": "NTIA-PD",
"raw.githubusercontent.com/OpenMDW/OpenMDW/refs/heads/main/1.0/LICENSE.openmdw": "OpenMDW-1.0",
"raw.githubusercontent.com/blackhole89/notekit/master/data/fonts/Charter%20license.txt": "Bitstream-Charter",
"raw.githubusercontent.com/landtuna/open-mtools/6c04c2882eca7c4313ec267766ea53ac9bfe129b/mdump.c": "Informatica",
"raw.githubusercontent.com/landtuna/open-mtools/6c04c2882eca7c4313ec267766ea53ac9bfe129b/msend.c": "Informatica",
"rc.quest.com/topics/openssh/license.php#tcpwrappers": "TCP-wrappers",
"research.cs.wisc.edu/condor/license.html#condor": "Condor-1.1",
"research.scea.com/scea_shared_source_license.html": "SCEA",
"romanrm.net/mit-zero": "MIT-0",
"sam.zoy.org/wtfpl/COPYING": "WTFPL",
"scancode-licensedb.aboutcode.org/bsd-unchanged": "BSD-2-Clause-pos-unchanged",
"scripts.sil.org/cms/scripts/page.php?item_id=OFL10_web": "OFL-1.0-no-RFN",
"scripts.sil.org/cms/scripts/page.php?item_id=OFL_web": "OFL-1.1-no-RFN",
"slicer.org/LICENSE": "3D-Slicer-1.0",
"solderpad.org/licenses/SHL-0.5/": "SHL-0.5",
"solderpad.org/licenses/SHL-0.51/": "SHL-0.51",
"source.icu-project.org/repos/icu/icu/trunk/license.html": "ICU",
"sourceforge.net/p/buddy/gitcode/ci/master/tree/README": "Buddy",
"sourceforge.net/p/ctrio/git/ci/master/tree/README": "ISC-Veillard",
"sourceforge.net/p/mcj/xfig/ci/master/tree/src/Makefile.am": "Xfig",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/other/pnmtorle.c": "URT-RLE",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/other/rletopnm.c": "URT-RLE",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/ppm/ppmtompeg/jrevdct.c#l1189": "MPEG-SSG",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/editor/pnmstitch.c#l2": "pnmstitch",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/netpbm.c#l8": "HPND-Pbmplus",
"sourceforge.net/p/sbcl/sbcl/ci/master/tree/src/code/loop.lisp": "LOOP",
"sourceforge.net/p/trustedqsl/tqsl/ci/master/tree/LICENSE.txt": "TrustedQSL",
"sourceforge.net/p/xmedcon/code/ci/master/tree/libs/dicom/README": "OFFIS",
"sourceforge.net/p/xmedcon/code/ci/master/tree/libs/ljpg/": "IJG-short",
"sourceforge.net/projects/sevenzip/files/LZMA%20SDK/": "LZMA-SDK-9.22",
"sources.debian.org/copyright/license/debianutils/4.11.2/": "SMAIL-GPL",
"sources.debian.org/src/openmpi/4.1.0-10/ompi/debuggers/msgq_interface.h/?hl=19#L19": "mpi-permissive",
"sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/licenses/BitTorrent?r1=1.1&r2=1.1.1.1&diff_format=s": "BitTorrent-1.0",
"sourceware.org/bzip2/1.0.5/bzip2-manual-1.0.5.html": "bzip2-1.0.6",
"sourceware.org/cgit/binutils-gdb/tree/include/coff/sym.h#n11": "MIPS",
"sourceware.org/cgit/binutils-gdb/tree/include/coff/symconst.h#n11": "ThirdEye",
"sourceware.org/cgit/valgrind/tree/mpi/libmpiwrap.c": "bzip2-1.0.6",
"raw.githubusercontent.com/landtuna/open-mtools/6c04c2882eca7c4313ec267766ea53ac9bfe129b/mdump.c": "Informatica",
"raw.githubusercontent.com/landtuna/open-mtools/6c04c2882eca7c4313ec267766ea53ac9bfe129b/msend.c": "Informatica",
"rc.quest.com/topics/openssh/license.php#tcpwrappers": "TCP-wrappers",
"research.cs.wisc.edu/condor/license.html#condor": "Condor-1.1",
"research.scea.com/scea_shared_source_license.html": "SCEA",
"romanrm.net/mit-zero": "MIT-0",
"sam.zoy.org/wtfpl/COPYING": "WTFPL",
"scancode-licensedb.aboutcode.org/bsd-unchanged": "BSD-2-Clause-pos-unchanged",
"scripts.sil.org/cms/scripts/page.php?item_id=OFL10_web": "OFL-1.0-no-RFN",
"scripts.sil.org/cms/scripts/page.php?item_id=OFL_web": "OFL-1.1-no-RFN",
"slicer.org/LICENSE": "3D-Slicer-1.0",
"solderpad.org/licenses/SHL-0.5/": "SHL-0.5",
"solderpad.org/licenses/SHL-0.51/": "SHL-0.51",
"source.icu-project.org/repos/icu/icu/trunk/license.html": "ICU",
"sourceforge.net/p/buddy/gitcode/ci/master/tree/README": "Buddy",
"sourceforge.net/p/ctrio/git/ci/master/tree/README": "ISC-Veillard",
"sourceforge.net/p/mcj/xfig/ci/master/tree/src/Makefile.am": "Xfig",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/other/pnmtorle.c": "URT-RLE",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/other/rletopnm.c": "URT-RLE",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/ppm/ppmtompeg/jrevdct.c#l1189": "MPEG-SSG",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/editor/pnmstitch.c#l2": "pnmstitch",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/netpbm.c#l8": "HPND-Pbmplus",
"sourceforge.net/p/sbcl/sbcl/ci/master/tree/src/code/loop.lisp": "LOOP",
"sourceforge.net/p/trustedqsl/tqsl/ci/master/tree/LICENSE.txt": "TrustedQSL",
"sourceforge.net/p/xmedcon/code/ci/master/tree/libs/dicom/README": "OFFIS",
"sourceforge.net/p/xmedcon/code/ci/master/tree/libs/ljpg/": "IJG-short",
"sourceforge.net/projects/sevenzip/files/LZMA%20SDK/": "LZMA-SDK-9.22",
"sources.debian.org/copyright/license/debianutils/4.11.2/": "SMAIL-GPL",
"sources.debian.org/src/openmpi/4.1.0-10/ompi/debuggers/msgq_interface.h/?hl=19#L19": "mpi-permissive",
"sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/licenses/BitTorrent?r1=1.1&r2=1.1.1.1&diff_format=s": "BitTorrent-1.0",
"sourceware.org/bzip2/1.0.5/bzip2-manual-1.0.5.html": "bzip2-1.0.6",
"sourceware.org/cgit/binutils-gdb/tree/include/coff/sym.h#n11": "MIPS",
"sourceware.org/cgit/binutils-gdb/tree/include/coff/symconst.h#n11": "ThirdEye",
"sourceware.org/cgit/valgrind/tree/mpi/libmpiwrap.c": "bzip2-1.0.6",
"sourceware.org/git/?p=binutils-gdb.git;a=blob;f=libiberty/strcasecmp.c;h=131d81c2ce7881fa48c363dc5bf5fb302c61ce0b;hb=HEAD": "BSD-4.3RENO",
"sourceware.org/git/?p=binutils-gdb.git;a=blob;f=readline/readline/support/wcwidth.c;h=0f5ec995796f4813abbcf4972aec0378ab74722a;hb=HEAD#l55": "HPND-Markus-Kuhn",
"sourceware.org/git/?p=bzip2.git;a=blob;f=LICENSE;hb=bzip2-1.0.6": "bzip2-1.0.6",
+699
View File
@@ -0,0 +1,699 @@
package task
import (
"context"
"fmt"
"io/fs"
"os"
"path/filepath"
"sort"
"strings"
"sync"
"testing"
"time"
"github.com/scylladb/go-set/strset"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal/archive"
"github.com/anchore/syft/internal/sbomsync"
"github.com/anchore/syft/internal/testutils"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cataloging"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/java"
"github.com/anchore/syft/syft/sbom"
)
// This file covers nesting across archive families. The other nested-archive fixtures are all zips,
// so a zip-specific mechanism would pass them; the requirement here is that an archive of one family
// inside another behaves as it would inside its own family - same packages, same container-path chain,
// same CONTAINS edges, same depth accounting.
//
// Chains are built in the test rather than committed as binaries, so each case can vary which family
// sits at which level.
// archiveFamily is one archive format buildNestedArchive can produce.
//
// ".jar" is its own family rather than a flavour of zip: the java cataloger meets it as a jar wherever
// it sits, while the archive task descends into it like any other zip. That differs mid-chain from at
// the leaf, so the matrix below uses a jar as container and as payload.
type archiveFamily string
const (
familyZip archiveFamily = ".zip"
familyJar archiveFamily = ".jar"
familyTar archiveFamily = ".tar"
familyTarGz archiveFamily = ".tar.gz"
familyTgz archiveFamily = ".tgz"
)
// nestPlan describes one chain to build: a family per level (outermost first), the entries for the
// innermost archive, and extras to add beside the nested archive at a given level. The extras grade
// the levels' sizes or entry counts, which is how the limit cases place a bound between two levels.
type nestPlan struct {
families []archiveFamily
leaf map[string][]byte
extra map[int]map[string][]byte
}
// nestedArchive is one built chain: the outermost archive's bytes, and the addresses that chain
// composes to at each level.
type nestedArchive struct {
// name is the outermost archive's file name, as written into the scanned directory.
name string
// bytes is the outermost archive.
bytes []byte
// sizes is each archive's size in bytes, outermost first. A container is never smaller than what it
// holds, so these decrease, and the limit cases place a bound between two of them.
sizes []int
// paths is each archive's path within its parent, outermost first; paths[0] is the outermost archive
// as the scan's resolver reports it.
paths []string
// containerPaths is each archive's full path, the container path its own contents carry, outermost first.
containerPaths []string
// virtualPaths is the same chain in java's colon-joined form, outermost first.
virtualPaths []string
}
// nestingRow is one line of the depth-3 matrix.
type nestingRow struct {
name string
families []archiveFamily
why string
}
// nestingVisit is one filesystem the archive cataloger's sub-pipeline was run against.
type nestingVisit struct {
virtualPath string
fullPath string
depth int
// heldFiles are the spill files live at this moment, sampled only when the probe was given a temp
// dir to look in. A nested archive's own bytes are read where they lie, so one appears per archive
// whose entries did not fit in memory.
heldFiles []string
}
// nestingProbe records one visit per filesystem the archive cataloger runs its sub-pipeline against,
// reading the chain off the traversal on the context.
type nestingProbe struct {
// tempDir, when set, is where extraction work directories live
tempDir string
mu sync.Mutex
visits []nestingVisit
}
func Test_mixedFamilyNesting_leafIsCatalogedOnceWithTheFullChain(t *testing.T) {
// family order must not change the result: the same package, addressed by a chain naming its own
// containers in its own order
for _, row := range nestingRows() {
t.Run(row.name, func(t *testing.T) {
scanDir := t.TempDir()
nested := writeNestedArchive(t, scanDir, nestPlan{families: row.families, leaf: markerLeaf()})
probe := &nestingProbe{}
s := runNesting(t, scanDir, 3, defaultLimits(), markerTask(), probe.task())
assert.Equal(t, 1, packageCount(s, "marker-pkg"), "%s: the leaf package must be cataloged exactly once", row.why)
locs := leafLocations(s)
require.Len(t, locs, 1, "the leaf must be reached by exactly one path")
// exact, never a substring: a partially composed chain passes a substring check
assert.Equal(t, nested.leafContainerPath(), archive.ContainerPath(locs[0].Coordinates),
"the chain must name all three archives outer-to-inner")
assert.Equal(t, nested.leafContainerPath()+"!/nested/marker.txt", locs[0].FullNestedPath(),
"the path stays relative to the archive the file came from")
assert.Equal(t, nested.containerPaths, probe.containerPaths(),
"every level must be cataloged, outer to inner, whatever families they are")
})
}
}
func Test_mixedFamilyNesting_containsEdgesChainAtEveryLevel(t *testing.T) {
// the edges make the chain navigable in the SBOM. The archive-to-archive edges must not depend on
// the file catalogers having recorded the inner archives, since the default file selection does not
for _, row := range nestingRows() {
t.Run(row.name, func(t *testing.T) {
scanDir := t.TempDir()
nested := writeNestedArchive(t, scanDir, nestPlan{families: row.families, leaf: markerLeaf()})
s := withArchiveContains(t, runNesting(t, scanDir, 3, defaultLimits(), markerTask()))
// each archive's coordinates as seen in its parent; the outermost sits in the scanned
// directory, so it carries no filesystem id of its own
archiveCoords := make([]file.Coordinates, len(row.families))
for i := range row.families {
archiveCoords[i] = nested.coordinates(i)
}
fileEdges := strset.New()
pkgEdges := strset.New()
for _, rel := range s.Relationships {
if rel.Type != artifact.ContainsRelationship {
continue
}
from, ok := rel.From.(file.Coordinates)
if !ok {
continue
}
switch to := rel.To.(type) {
case file.Coordinates:
fileEdges.Add(coordKey(from) + " -> " + coordKey(to))
case pkg.Package:
pkgEdges.Add(coordKey(from) + " -> " + to.Name)
}
}
for i := 0; i < len(archiveCoords)-1; i++ {
assert.True(t, fileEdges.Has(coordKey(archiveCoords[i])+" -> "+coordKey(archiveCoords[i+1])),
"expected a CONTAINS edge from %s to %s", nested.paths[i], nested.paths[i+1])
}
innermost := archiveCoords[len(archiveCoords)-1]
assert.True(t, pkgEdges.Has(coordKey(innermost)+" -> marker-pkg"),
"expected a CONTAINS edge from %s to the leaf package", nested.paths[len(nested.paths)-1])
// with file cataloging on, files inside an archive get an edge from it too
s = withArchiveContains(t, runNesting(t, scanDir, 3, defaultLimits(), markerTask(), fileMetadataTask()))
var leafEdge bool
for _, rel := range s.Relationships {
from, _ := rel.From.(file.Coordinates)
to, _ := rel.To.(file.Coordinates)
leafEdge = leafEdge || (from == innermost && to == file.Coordinates{RealPath: innermost.RealPath, FullPath: innermost.FullNestedPath() + "!/nested/marker.txt"})
}
assert.True(t, leafEdge, "expected a CONTAINS edge from the innermost archive to the leaf file")
})
}
}
func Test_mixedFamilyNesting_everyJarIsCatalogedOnceUnderItsChain(t *testing.T) {
// two mechanisms meet a jar: the java cataloger catalogs it as a package, the archive task descends
// into it. A jar at any level must do both, exactly once each; the failure mode is a duplicate, not an
// error. The colon-joined virtual paths are in already-published SBOMs, so they must name the chain.
for _, row := range nestingRows() {
var jarLevels []int
for i, family := range row.families {
if family == familyJar {
jarLevels = append(jarLevels, i)
}
}
if len(jarLevels) == 0 {
continue
}
t.Run(row.name, func(t *testing.T) {
scanDir := t.TempDir()
nested := writeNestedArchive(t, scanDir, nestPlan{families: row.families, leaf: markerLeaf()})
probe := &nestingProbe{}
s := runNesting(t, scanDir, 3, defaultLimits(), markerTask(), javaTask(), probe.task())
for _, level := range jarLevels {
name := fmt.Sprintf("level%d", level)
assert.Equal(t, 1, packageCount(s, name), "the jar's own package must be cataloged exactly once")
assert.Equal(t, nested.virtualPaths[level], javaVirtualPaths(s)[name],
"the jar's virtual path must be its own place in the chain")
}
// and the archive task still descended through every level
assert.Equal(t, nested.containerPaths, probe.containerPaths())
locs := leafLocations(s)
require.Len(t, locs, 1)
assert.Equal(t, nested.leafContainerPath(), archive.ContainerPath(locs[0].Coordinates))
})
}
}
func Test_mixedFamilyNesting_depthCountsLevelsNotFamilies(t *testing.T) {
// two rows with different families at every level, so the bound is shown not to turn on which
// families are involved
for _, row := range nestingRows()[:2] {
t.Run(row.name, func(t *testing.T) {
scanDir := t.TempDir()
nested := writeNestedArchive(t, scanDir, nestPlan{families: row.families, leaf: markerLeaf()})
probe := &nestingProbe{}
s := runNesting(t, scanDir, 2, defaultLimits(), markerTask(), fileMetadataTask(), probe.task())
assert.Equal(t, nested.containerPaths[:2], probe.containerPaths(),
"at depth 2 the outer and middle archives are cataloged and the innermost is not")
assert.Zero(t, packageCount(s, "marker-pkg"),
"the leaf is one level past the bound, so its package must be absent")
// the innermost archive is still cataloged as a file, inside the middle archive
innermost := nested.coordinates(2)
_, ok := s.Artifacts.FileMetadata[innermost]
assert.True(t, ok, "the undescended archive must still be recorded as a file at %s", coordKey(innermost))
// and what was left unexplored is said out loud, not silently dropped
reasons := s.Artifacts.Unknowns[innermost]
require.Len(t, reasons, 1, "unknowns: %v", s.Artifacts.Unknowns)
assert.Contains(t, reasons[0], "depth limit was reached")
})
}
}
func Test_mixedFamilyNesting_aNestedArchiveIsReadWhereItLies(t *testing.T) {
// an archive's own bytes are read where they lie: a scanned file from disk, a nested archive from
// its parent's entries. Nothing copies them, so at most one spill file per level appears whatever
// the memory limit; only entries are held, and only they can be written out.
scanDir := t.TempDir()
tempDir := isolatedTempDir(t)
nested := writeNestedArchive(t, scanDir, gradedChain())
for _, tc := range []struct {
name string
memoryBytes int64
wantHeldFiles []string
}{
{
name: "with room in memory for every level's entries",
memoryBytes: int64(2 * (nested.sizes[0] + nested.sizes[1] + nested.sizes[2])),
},
{
name: "with a memory limit of zero, so every level's entries are written out",
memoryBytes: 0,
wantHeldFiles: []string{"archive-spill", "archive-spill", "archive-spill"},
},
} {
t.Run(tc.name, func(t *testing.T) {
probe := &nestingProbe{tempDir: tempDir}
bounds := archive.Limits{MaxMemoryBytes: tc.memoryBytes, MaxDiskBytes: -1}
s := runNesting(t, scanDir, 3, bounds, markerTask(), probe.task())
deepest := probe.deepestVisit()
require.Equal(t, 3, deepest.depth, "the sample must be taken with all three levels still held")
assert.Equal(t, tc.wantHeldFiles, deepest.heldFiles)
assert.Equal(t, nested.containerPaths, probe.containerPaths())
locs := leafLocations(s)
require.Len(t, locs, 1)
assert.Equal(t, nested.leafContainerPath(), archive.ContainerPath(locs[0].Coordinates))
})
}
}
func Test_mixedFamilyNesting_diskLimitTruncatesTheLevelItCannotAdmit(t *testing.T) {
// the walk descends before it unwinds, so the outer and middle archives still hold their entries
// when the innermost level's do not fit in memory. With nothing admitted on disk that level is
// cataloged from nothing, the shallower ones keep what they found, and the scan finishes. An
// implementation that waited for capacity would deadlock, hence the timeout.
scanDir := t.TempDir()
nested := writeNestedArchive(t, scanDir, deepChain())
// a sibling of a different family, processed once the chain has been released, so cataloged in full
require.NoError(t, os.WriteFile(
filepath.Join(scanDir, "sibling.tgz"),
testutils.TarGz(t, map[string][]byte{"nested/marker.txt": []byte("sibling")}),
0o644,
))
// room for the outer two levels' entries (each about the size of the archive it holds) but not the
// innermost's payload, which is half an archive larger than the slack
memoryLimit := int64(nested.sizes[0] + nested.sizes[1] + deepChainPayloadBytes/2)
probe := &nestingProbe{}
bounds := archive.Limits{MaxMemoryBytes: memoryLimit, MaxDiskBytes: 0}
s := runNesting(t, scanDir, 3, bounds, markerTask(), probe.task())
want := append(append([]string{}, nested.containerPaths...), "/sibling.tgz")
assert.ElementsMatch(t, want, probe.containerPaths(),
"every level is visited, the truncated one with nothing inside it")
containers := strset.New()
for _, loc := range leafLocations(s) {
containers.Add(archive.ContainerPath(loc.Coordinates))
}
assert.False(t, containers.Has(nested.leafContainerPath()), "the truncated level's contents must be absent")
assert.True(t, containers.Has("/sibling.tgz"), "the sibling of a different family must be cataloged fully")
var truncated []string
for coords, reasons := range s.Artifacts.Unknowns {
for _, reason := range reasons {
if strings.Contains(reason, "disk limit") {
truncated = append(truncated, coords.FullNestedPath())
}
}
}
assert.Equal(t, []string{nested.coordinates(2).FullNestedPath()}, truncated,
"the truncation is recorded against the archive that reached the limit")
}
func defaultLimits() archive.Limits {
return archiveLimits(cataloging.DefaultArchiveSearchConfig())
}
// runNesting runs the archive cataloger task over scanDir with the given sub-pipeline and returns the
// SBOM it built. It runs on its own goroutine and fails if the task has not finished in time.
//
// The depth-3 limit cases need the timeout: the walk descends before it unwinds, so the outer and
// middle archives still hold their content when the inner one is refused. An implementation that
// waited for capacity would deadlock, and a test asserting only the result would hang rather than fail.
func runNesting(t *testing.T, scanDir string, depth int, bounds archive.Limits, subPipeline ...Task) *sbom.SBOM {
t.Helper()
tsk := newLimitedTestTask(t, depth, bounds, subPipeline...)
s := &sbom.SBOM{Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(),
FileMetadata: map[file.Coordinates]file.Metadata{},
}}
done := make(chan error, 1)
go func() {
done <- tsk.Execute(context.Background(), dirTestResolver{dir: scanDir}, sbomsync.NewBuilder(s))
}()
select {
case err := <-done:
require.NoError(t, err, "reaching a bound must not fail the scan")
return s
case <-time.After(60 * time.Second):
t.Fatal("the scan did not finish in time: reaching a limit must skip the archive that " +
"reached it, never wait for capacity that nothing in the chain will release")
return nil
}
}
// withArchiveContains runs the task that follows every scan with archive cataloging on.
func withArchiveContains(t *testing.T, s *sbom.SBOM) *sbom.SBOM {
t.Helper()
require.NoError(t, NewArchiveContainsTask().Execute(context.Background(), nil, sbomsync.NewBuilder(s)))
return s
}
// markerTask emits one package per "marker.txt" found, so the matrix asserts behavior without
// depending on any real cataloger's fixture format.
func markerTask() Task {
return NewTask("marker-cataloger", func(_ context.Context, resolver file.Resolver, builder sbomsync.Builder) error {
locations, err := resolver.FilesByGlob("**/marker.txt")
if err != nil {
return err
}
for _, loc := range locations {
p := pkg.Package{
Name: "marker-pkg",
Version: "1.0.0",
Type: pkg.BinaryPkg,
Locations: file.NewLocationSet(loc),
}
p.SetID()
builder.AddPackages(p)
}
return nil
})
}
// javaTask is the java archive cataloger as the cataloging factory builds it, so a jar is met as a jar
// wherever it sits in a chain.
func javaTask() Task {
return NewPackageTask(CatalogingFactoryConfig{}, java.NewArchiveCataloger(java.DefaultArchiveCatalogerConfig()))
}
// fileMetadataTask records every file, so each file inside an archive gets an archive-to-file edge.
func fileMetadataTask() Task {
return newFileMetadataCatalogerTask(file.AllFilesSelection)
}
// pack builds one archive of this family holding the given entries.
func (f archiveFamily) pack(t *testing.T, base string, entries map[string][]byte) []byte {
t.Helper()
switch f {
case familyZip:
return testutils.Zip(t, entries)
case familyJar:
// a manifest makes it a real jar to the java cataloger at any level; the package java reports is
// named after the file, not the manifest title
return testutils.Jar(t, base, "1.0", entries)
case familyTar:
return testutils.Tar(t, entries)
case familyTarGz, familyTgz:
return testutils.TarGz(t, entries)
}
t.Fatalf("unknown archive family %q", f)
return nil
}
// leafContainerPath is the container path the innermost archive's contents carry: every archive, outer to
// inner. Assert it exactly, never as a substring: a partially composed chain passes a substring check.
func (n nestedArchive) leafContainerPath() string {
return n.containerPaths[len(n.containerPaths)-1]
}
// coordinates are those of the archive at the given level: the outermost archive's path, and the chain
// of entries within it.
func (n nestedArchive) coordinates(level int) file.Coordinates {
return file.Coordinates{RealPath: n.paths[0], FullPath: strings.Join(n.paths[:level+1], "!/")}
}
// buildNestedArchive builds one archive per family given, each holding the next, with leaf inside
// the innermost.
func buildNestedArchive(t *testing.T, plan nestPlan) nestedArchive {
t.Helper()
require.NotEmpty(t, plan.families, "a chain needs at least one level")
out := nestedArchive{
sizes: make([]int, len(plan.families)),
paths: make([]string, len(plan.families)),
containerPaths: make([]string, len(plan.families)),
virtualPaths: make([]string, len(plan.families)),
}
// the level index goes in every name and path segment, so no two levels share a path and a chain
// assertion cannot pass on a value composed from the wrong level
for i, family := range plan.families {
base := fmt.Sprintf("level%d", i)
if i == 0 {
// the outermost archive is a file in the scanned directory, which the resolver reports rooted
out.name = base + string(family)
out.paths[i] = "/" + out.name
continue
}
out.paths[i] = fmt.Sprintf("nest%d/%s%s", i, base, family)
}
for i := range plan.families {
if i == 0 {
out.containerPaths[i] = out.paths[0]
out.virtualPaths[i] = out.paths[0]
continue
}
out.containerPaths[i] = out.containerPaths[i-1] + "!/" + out.paths[i]
out.virtualPaths[i] = out.virtualPaths[i-1] + ":" + out.paths[i]
}
// built innermost first, because each level's bytes are an entry of the level above it
var packed []byte
for i := len(plan.families) - 1; i >= 0; i-- {
entries := map[string][]byte{}
if i == len(plan.families)-1 {
for name, body := range plan.leaf {
entries[name] = body
}
} else {
entries[out.paths[i+1]] = packed
}
for name, body := range plan.extra[i] {
entries[name] = body
}
packed = plan.families[i].pack(t, fmt.Sprintf("level%d", i), entries)
out.sizes[i] = len(packed)
}
out.bytes = packed
return out
}
// writeNestedArchive writes the chain's outermost archive into dir and returns the chain.
func writeNestedArchive(t *testing.T, dir string, plan nestPlan) nestedArchive {
t.Helper()
nested := buildNestedArchive(t, plan)
require.NoError(t, os.WriteFile(filepath.Join(dir, nested.name), nested.bytes, 0o644))
return nested
}
// markerLeaf is the family-agnostic leaf payload: one file no archive format has an opinion about,
// found by markerTask. It keeps the rows comparable - the same package must come out whichever
// families the chain is made of.
func markerLeaf() map[string][]byte {
return map[string][]byte{"nested/marker.txt": []byte("leaf")}
}
// nestingRows is the depth-3 matrix. Each row covers a case the others do not.
func nestingRows() []nestingRow {
return []nestingRow{
{
name: "tar.gz/zip/jar",
families: []archiveFamily{familyTarGz, familyZip, familyJar},
why: "the headline case: compressed tar outermost, jar innermost",
},
{
name: "zip/tar.gz/jar",
families: []archiveFamily{familyZip, familyTarGz, familyJar},
why: "compressed tar in the middle, where the stream-only format is not the entry point",
},
{
name: "jar/tar/zip",
families: []archiveFamily{familyJar, familyTar, familyZip},
why: "jar as a container rather than a payload",
},
{
name: "tgz/tar/jar",
families: []archiveFamily{familyTgz, familyTar, familyJar},
why: "the .tgz alias resolving at a non-outermost boundary",
},
{
name: "tar/jar/tar.gz",
families: []archiveFamily{familyTar, familyJar, familyTarGz},
why: "compressed tar as the innermost, which nothing else covers",
},
}
}
func (p *nestingProbe) task() Task {
return NewTask("nesting-probe", func(ctx context.Context, _ file.Resolver, _ sbomsync.Builder) error {
trav := archive.TraversalFromContext(ctx)
visit := nestingVisit{}
if trav != nil {
visit.virtualPath = archive.VirtualPath(trav.Location)
visit.fullPath = trav.FullTraversalArchivePath()
visit.depth = strings.Count(archive.VirtualPath(trav.Location), ":") + 1
}
if p.tempDir != "" {
visit.heldFiles = heldFiles(p.tempDir)
}
p.mu.Lock()
defer p.mu.Unlock()
p.visits = append(p.visits, visit)
return nil
})
}
// containerPaths is the chain of every archive whose contents were cataloged, in visit order.
func (p *nestingProbe) containerPaths() []string {
p.mu.Lock()
defer p.mu.Unlock()
var out []string
for _, v := range p.visits {
if v.fullPath == "" {
continue
}
out = append(out, v.fullPath)
}
return out
}
// deepestVisit is the visit at the innermost level reached, the only moment at which every level of a
// chain is still held and its routing observable. It returns the whole visit so a case can also assert
// it got as deep as it meant to: an empty overflow sample from a visit that never left the scan root
// would otherwise read as "nothing overflowed".
func (p *nestingProbe) deepestVisit() nestingVisit {
p.mu.Lock()
defer p.mu.Unlock()
var best nestingVisit
for _, v := range p.visits {
if v.depth > best.depth {
best = v
}
}
return best
}
// heldFiles names the spill files live under tempDir, one "archive-spill" per archive holding content
// that did not fit in memory. Meaningful only while the archives are still held, so it is sampled from
// inside the sub-pipeline rather than after the scan.
func heldFiles(tempDir string) (names []string) {
_ = filepath.WalkDir(tempDir, func(_ string, d fs.DirEntry, err error) error {
if err != nil || d.IsDir() || !strings.HasPrefix(d.Name(), "archive-") {
return nil //nolint:nilerr // an unreadable directory holds nothing this probe can report
}
// the name is the pattern with os.CreateTemp's random suffix in place of the "*"
names = append(names, d.Name()[:strings.LastIndexByte(d.Name(), '-')])
return nil
})
sort.Strings(names)
return names
}
// isolatedTempDir points archive extraction at a directory of this test's own, so an overflow sample
// sees only this scan's work directories.
func isolatedTempDir(t *testing.T) string {
t.Helper()
dir := t.TempDir()
t.Setenv("TMPDIR", dir) // unix
t.Setenv("TMP", dir) // windows
t.Setenv("TEMP", dir) // windows fallback
require.Equal(t, dir, os.TempDir(), "extraction must be writing where the probe is looking")
return dir
}
func leafLocations(s *sbom.SBOM) []file.Location {
var out []file.Location
for _, p := range s.Artifacts.Packages.Sorted() {
if p.Name != "marker-pkg" {
continue
}
out = append(out, p.Locations.ToSlice()...)
}
return out
}
func javaVirtualPaths(s *sbom.SBOM) map[string]string {
out := map[string]string{}
for _, p := range s.Artifacts.Packages.Sorted() {
if metadata, ok := p.Metadata.(pkg.JavaArchive); ok {
out[p.Name] = metadata.VirtualPath
}
}
return out
}
func packageCount(s *sbom.SBOM, name string) int {
var n int
for _, p := range s.Artifacts.Packages.Sorted() {
if p.Name == name {
n++
}
}
return n
}
func coordKey(c file.Coordinates) string {
return c.FullNestedPath()
}
// gradedChain is the headline row with each level padded so the three archives have clearly different
// sizes: a container is never smaller than what it holds, so a bound can only be placed between two
// levels if they are graded on purpose.
func gradedChain() nestPlan {
return nestPlan{
families: []archiveFamily{familyTarGz, familyZip, familyJar},
leaf: markerLeaf(),
extra: map[int]map[string][]byte{
0: {"pad0.bin": incompressible(24 * 1024)},
1: {"pad1.bin": incompressible(24 * 1024)},
2: {"pad2.bin": incompressible(8 * 1024)},
},
}
}
// deepChainPayloadBytes is the size of deepChain's leaf payload.
const deepChainPayloadBytes = 64 * 1024
// deepChain is the headline row with one large, incompressible payload at the leaf and no other
// padding. The payload dominates every level, so the three archives are about the same size. It sorts
// before the marker so it is the first entry refused when the leaf archive's entries do not fit.
func deepChain() nestPlan {
return nestPlan{
families: []archiveFamily{familyTarGz, familyZip, familyJar},
leaf: map[string][]byte{
"0-payload.bin": incompressible(deepChainPayloadBytes),
"nested/marker.txt": []byte("leaf"),
},
}
}
+501
View File
@@ -0,0 +1,501 @@
package task
import (
"context"
"errors"
"fmt"
"io"
"maps"
"runtime/debug"
"slices"
"time"
"github.com/dustin/go-humanize"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/archive"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/mimetype"
"github.com/anchore/syft/internal/sbomsync"
"github.com/anchore/syft/internal/unknown"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cataloging"
"github.com/anchore/syft/syft/cataloging/pkgcataloging"
"github.com/anchore/syft/syft/event/monitor"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
)
const ArchiveCatalogerTaskName = "archive-cataloger"
// NewArchiveCatalogerTask returns a task that extracts every archive it finds, runs the same tasks
// against the archive's contents as a standalone filesystem, and records
// CONTAINS relationships from the archive to the packages found directly inside it. It recurses into the archives
// it extracts up to cfg.MaxDepth; the task drops itself from the tasks so that is the only recursion.
//
// Each archive is handled as the scan root is: pkgTasks, then the archives nested in it, then fileTasks,
// which see the packages found in the nested archives (see Traversal.PackageFiles).
//
// The task is the authority on which archives were not cataloged and why; with
// unknowns.IncludeUnexpandedArchives it also reports archives in formats it does not extract.
//
// exclusions are the scan's exclusion patterns, applied inside each archive. Returns nil when archive
// cataloging is disabled (MaxDepth == 0; negative or beyond archive.MaxDepth means as deep as archive.MaxDepth).
func NewArchiveCatalogerTask(cfg cataloging.ArchiveSearchConfig, unknowns cataloging.UnknownsConfig, pkgTasks, fileTasks []Task, exclusions []string) Task {
return newArchiveCatalogerTask(cfg.MaxDepth, archiveLimits(cfg), unknowns.IncludeUnexpandedArchives, pkgTasks, fileTasks, exclusions)
}
func archiveLimits(cfg cataloging.ArchiveSearchConfig) archive.Limits {
return archive.Limits{MaxMemoryBytes: cfg.MaxMemoryBytes, MaxDiskBytes: cfg.MaxDiskBytes}
}
func newArchiveCatalogerTask(maxDepth int, limits archive.Limits, reportUnsupportedUnknowns bool, pkgTasks, fileTasks []Task, exclusions []string) Task {
if maxDepth == 0 {
return nil
}
pkgTasks, fileTasks = withoutArchiveCataloger(pkgTasks), withoutArchiveCataloger(fileTasks)
if len(pkgTasks)+len(fileTasks) == 0 {
return nil
}
fn := func(ctx context.Context, resolver file.Resolver, builder sbomsync.Builder) error {
// catalogers run once per archive, and each must report into the one progress row it owns
ctx = bus.WithCatalogerTaskRegistry(ctx)
c := &archiveCataloger{
maxDepth: maxDepth,
reportUnsupportedUnknowns: reportUnsupportedUnknowns,
pkgTasks: pkgTasks,
fileTasks: fileTasks,
limiter: archive.NewLimiter(limits),
exclusions: archive.NewExclusions(exclusions),
progress: bus.StartCatalogerTask(ctx, archiveCatalogerProgressInfo(), -1, ""),
}
err := c.catalog(ctx, resolver, 0, builder)
c.progress.AtomicStage.Set(fmt.Sprintf("%s archives", humanize.Comma(c.progress.Current())))
c.progress.SetCompleted()
c.logStats()
return err
}
return NewTask(ArchiveCatalogerTaskName, fn, pkgcataloging.PackageTag, "archive")
}
func archiveCatalogerProgressInfo() monitor.GenericTask {
return monitor.GenericTask{
Title: monitor.Title{
Default: "Archives",
},
ID: ArchiveCatalogerTaskName,
ParentID: monitor.TopLevelCatalogingTaskID,
}
}
func withoutArchiveCataloger(tasks []Task) []Task {
var out []Task
for _, t := range tasks {
if t == nil || t.Name() == ArchiveCatalogerTaskName {
continue
}
out = append(out, t)
}
return out
}
type archiveCataloger struct {
maxDepth int
reportUnsupportedUnknowns bool
pkgTasks []Task
fileTasks []Task
limiter *archive.Limiter
exclusions archive.Exclusions
// progress is the one row for the whole walk: its count is how many archives have been entered
progress *monitor.TaskProgress
// ancestors are the digests of the archives the walk is currently inside. The walk is depth-first
// and sequential, so this is a stack.
ancestors []file.Digest
// the archive that spent the longest on its own extraction and cataloging, excluding nested archives
slowest time.Duration
slowestPath string
}
// logStats reports what the walk cost, once. The peaks are the most held against each limit at any
// one moment, comparable to the configured bounds.
func (c *archiveCataloger) logStats() {
peakMemory, peakDisk := c.limiter.Peak()
fields := []any{
"archives", c.progress.Current(),
"peak-memory", humanize.Bytes(uint64(peakMemory)),
"peak-disk", humanize.Bytes(uint64(peakDisk)),
}
if c.slowestPath != "" {
fields = append(fields, "slowest", c.slowestPath, "slowest-took", c.slowest.Round(time.Millisecond))
}
log.WithFields(fields...).Info("nested archive cataloging complete")
}
// catalog processes every archive in the resolver, recursing into each up to maxDepth.
func (c *archiveCataloger) catalog(ctx context.Context, resolver file.Resolver, depth int, builder sbomsync.Builder) error {
return c.catalogCandidates(ctx, resolver, c.discoverArchives(resolver), depth, builder)
}
func (c *archiveCataloger) catalogCandidates(ctx context.Context, resolver file.Resolver, candidates []archiveCandidate, depth int, builder sbomsync.Builder) error {
maxDepth := c.maxDepth
// a deeper scan would write chains no decoder accepts back
if maxDepth < 0 || maxDepth > archive.MaxDepth {
maxDepth = archive.MaxDepth
}
if depth >= maxDepth {
// java's own recursion is off while this task runs, so an archive left here is otherwise lost
// without a trace
for _, candidate := range candidates {
if candidate.sniffedAsArchive {
recordArchiveUnknown(builder, candidate.location.Coordinates, "nested archive not cataloged: the nested archive depth limit was reached")
}
}
return nil
}
var errs error
for _, candidate := range candidates {
if err := ctx.Err(); err != nil {
return err
}
if err := c.processArchive(ctx, resolver, candidate, depth, builder); err != nil {
errs = unknown.Append(errs, candidate.location, err)
}
}
return errs
}
// processArchive processes the given candidate archive, catching panics
//
//nolint:funlen
func (c *archiveCataloger) processArchive(ctx context.Context, parentResolver file.Resolver, candidate archiveCandidate, depth int, builder sbomsync.Builder) (err error) {
// the decoders run on untrusted bytes; a panic in one costs this archive, not the scan
defer func() {
if r := recover(); r != nil {
// the stack goes to the log, not the reason: the reason lands in the published SBOM's unknowns
log.WithFields("path", candidate.location.FullNestedPath()).Debugf("archive cataloging panic stack:\n%s", debug.Stack())
err = fmt.Errorf("panic while cataloging archive: %v", r)
}
}()
location := candidate.location
content, err := parentResolver.FileContentsByLocation(location)
if err != nil {
return err
}
defer internal.CloseAndLogError(content, location.FullNestedPath())
var archiveContent io.Reader = content
if !candidate.sniffedAsArchive {
var mayHide bool
if archiveContent, mayHide = archive.MayHideAnAppendedArchive(content); !mayHide {
return nil
}
}
started := time.Now()
if depth == 0 {
// one budget for this archive and everything nested in it, however deep
ctx = archive.WithBudget(ctx, archive.NewBudget(archiveTreeBudget(parentResolver, location)))
}
// extracted files keep the real path and filesystem ID of the outermost archive; the chain of entries
// to each is carried as its archive path. The chain names real paths, not the link the archive was
// reached by, so it is stable and joins to the archive's package location by prefix.
traversal := &archive.Traversal{Location: location, Parent: archive.TraversalFromContext(ctx)}
archivePath := location.FullNestedPath()
extracted, err := archive.Extract(ctx, archiveContent, location.Coordinates, c.limiter, c.exclusions)
if errors.Is(err, archive.ErrUnsupportedFormat) {
if c.reportUnsupportedUnknowns {
recordArchiveUnknown(builder, location.Coordinates, "archive not cataloged: "+err.Error())
}
return nil
}
if errors.Is(err, archive.ErrDiskLimitReached) {
// nothing is released while this archive waits, so skip it rather than block the scan
recordArchiveUnknown(builder, location.Coordinates, "archive skipped: its content would exceed the disk limit")
return nil
}
if err != nil {
return err
}
if extracted == nil {
// a format with no extractor, or a candidate whose head only looked like an appended archive.
// Logged rather than returned: an image can hold thousands of these
log.WithFields("archive", archivePath, "sniffed-as-archive", candidate.sniffedAsArchive).
Trace("no extractor could read this candidate; skipping")
return nil
}
defer extracted.Cleanup()
// an archive whose bytes match one it is nested in (a zip quine) would recurse forever
if c.isAncestor(extracted.Digests) {
recordArchiveUnknown(builder, location.Coordinates, "archive not cataloged: it is identical to an archive that contains it")
return nil
}
c.ancestors = append(c.ancestors, extracted.Digests...)
defer func() { c.ancestors = c.ancestors[:len(c.ancestors)-len(extracted.Digests)] }()
if extracted.Truncated != nil {
recordArchiveUnknown(builder, location.Coordinates, "archive cataloged from part of its contents: "+extracted.Truncated.Error())
}
traversal.Digests = extracted.Digests
ctx = archive.WithTraversal(ctx, traversal)
c.progress.Increment()
// everything found in this archive and the archives nested in it, merged into the parent's SBOM once complete
found := &sbom.SBOM{Artifacts: sbom.Artifacts{Packages: pkg.NewCollection()}}
foundBuilder := sbomsync.NewBuilder(found)
runArchiveTasks(ctx, c.pkgTasks, extracted, location.Coordinates, foundBuilder)
direct := found.Artifacts.Packages.Sorted()
traversal.Packages = packagesAt(direct, location.Coordinates)
// self time excludes descending, so a containing archive is not charged for what it contains
selfTime := time.Since(started)
// as at the scan root, nested archives are cataloged before this archive's files, so the packages
// describing them (located in this archive) select their files here
errs := c.catalogCandidates(ctx, extracted, c.discoverArchives(extracted), depth+1, foundBuilder)
started = time.Now()
runArchiveTasks(ctx, c.fileTasks, extracted, location.Coordinates, foundBuilder)
// the files within it are related once the scan has decided which it describes (see NewArchiveContainsTask)
for _, p := range direct {
found.Relationships = append(found.Relationships, artifact.Relationship{From: location.Coordinates, To: p, Type: artifact.ContainsRelationship})
}
mergeArchiveResults(found, builder)
c.progress.AtomicStage.Set(humanize.Comma(c.progress.Current()) + " archives (" + archivePath + ")")
if took := selfTime + time.Since(started); took > c.slowest {
c.slowest, c.slowestPath = took, archivePath
}
return errs
}
// a top-level archive and everything nested in it may decompress to archiveTreeRatio times its size, or
// archiveTreeFloorBytes if that is more. Real trees (a war of jars, a layer of tarballs) expand a few
// times over; a recursive bomb expands by orders of magnitude at every level.
var (
archiveTreeRatio int64 = 100
archiveTreeFloorBytes int64 = 1 << 30
)
func archiveTreeBudget(resolver file.Resolver, location file.Location) int64 {
var size int64
if m, err := resolver.FileMetadataByLocation(location); err == nil && m.FileInfo != nil {
size = m.Size()
}
return max(archiveTreeRatio*size, archiveTreeFloorBytes)
}
func (c *archiveCataloger) isAncestor(digests []file.Digest) bool {
for _, d := range digests {
if slices.Contains(c.ancestors, d) {
return true
}
}
return false
}
// packagesAt returns the packages located at the given coordinates, i.e. those describing the file there.
func packagesAt(pkgs []pkg.Package, coordinates file.Coordinates) []pkg.Package {
var out []pkg.Package
for _, p := range pkgs {
if locatedAt(p, coordinates) {
out = append(out, p)
}
}
return out
}
// locatedAt reports whether one of p's locations is at the given coordinates.
func locatedAt(p pkg.Package, coordinates file.Coordinates) bool {
for _, l := range p.Locations.ToUnorderedSlice() {
if l.Coordinates == coordinates {
return true
}
}
return false
}
func recordArchiveUnknown(builder sbomsync.Builder, coordinates file.Coordinates, reason string) {
appendUnknowns(builder, ArchiveCatalogerTaskName, []unknown.CoordinateError{{Coordinates: coordinates, Reason: errors.New(reason)}})
}
// runArchiveTasks runs each task against the archive's contents.
//
// Failures are recorded as RunTask records them at the top level: errors carrying coordinates keep
// them, anything else is attributed to the archive. A failure never fails the archive or the scan.
func runArchiveTasks(ctx context.Context, tasks []Task, resolver file.Resolver, archiveCoordinates file.Coordinates, builder sbomsync.Builder) {
for _, t := range tasks {
err := runSubTaskSafely(ctx, t, resolver, builder)
if err == nil {
continue
}
log.WithFields("task", t.Name(), "error", err).Trace("archive sub-cataloger task reported an error")
unknowns, remaining := unknown.ExtractCoordinateErrors(err)
if remaining != nil {
unknowns = append(unknowns, unknown.CoordinateError{Coordinates: archiveCoordinates, Reason: remaining})
}
appendUnknowns(builder, t.Name(), unknowns)
}
}
// runSubTaskSafely is runTaskSafely without the task name in a panic's error, since appendUnknowns
// already prefixes it.
func runSubTaskSafely(ctx context.Context, t Task, resolver file.Resolver, builder sbomsync.Builder) (err error) {
defer func() {
if r := recover(); r != nil {
log.WithFields("task", t.Name()).Debugf("archive sub-cataloger task panic stack:\n%s", debug.Stack())
err = fmt.Errorf("panic: %v", r)
}
}()
return t.Execute(ctx, resolver, builder)
}
// archiveCandidate is one file the walk will try to open.
type archiveCandidate struct {
location file.Location
// sniffedAsArchive is true when the file's content types it as an archive. Such a file that cannot
// be extracted is a broken archive and is reported; a file that is a candidate only by name is
// passed over silently.
sniffedAsArchive bool
}
// zipArchiveGlobs find files named like zip archives whose content did not sniff as one, such as a
// Spring Boot executable jar: a launcher script with a zip appended. The name only decides what gets
// opened; the end-of-central-directory record decides whether it is an archive.
//
// Globs rather than a MIME lookup, because asking for every executable, script and unrecognized file
// returned thousands of locations that were all discarded.
var zipArchiveGlobs = []string{
"**/*.jar", "**/*.war", "**/*.ear", "**/*.par", "**/*.sar", "**/*.nar", "**/*.kar",
"**/*.hpi", "**/*.jpi", "**/*.far", "**/*.rar", "**/*.zip", "**/*.apk", "**/*.aar",
"**/*.egg", "**/*.whl", "**/*.lpkg", "**/*.zap", "**/*.exe",
}
var archiveMIMETypes = slices.Sorted(slices.Values(mimetype.ArchiveMIMETypeSet.List()))
// discoverArchives returns every file worth opening: those whose content is an archive, plus those
// named like a zip whose tail may still hold one.
func (c *archiveCataloger) discoverArchives(resolver file.Resolver) []archiveCandidate {
archives, err := resolver.FilesByMIMEType(archiveMIMETypes...)
if err != nil {
log.WithFields("error", err).Debug("unable to list archive files for recursive cataloging")
return nil
}
var candidates []archiveCandidate
seen := map[file.Coordinates]int{}
for _, loc := range archives {
seen[loc.Coordinates] = len(candidates)
candidates = append(candidates, archiveCandidate{location: loc, sniffedAsArchive: true})
}
namedLikeZips, err := resolver.FilesByGlob(zipArchiveGlobs...)
if err != nil {
log.WithFields("error", err).Debug("unable to list files that may carry an appended archive")
return candidates
}
for _, loc := range namedLikeZips {
if i, dup := seen[loc.Coordinates]; dup {
// a MIME hit is the bare real path; the name it is reached by (a link such as
// lib/dep.jar -> ../store/3f9a1c) is what name-based catalogers like java key off
if prev := candidates[i].location; prev.AccessPath == "" || prev.AccessPath == prev.FullNestedPath() {
candidates[i].location = loc
}
continue
}
seen[loc.Coordinates] = len(candidates)
candidates = append(candidates, archiveCandidate{location: loc})
}
return candidates
}
// mergeArchiveResults copies everything found in an archive into its parent's SBOM.
func mergeArchiveResults(found *sbom.SBOM, builder sbomsync.Builder) {
if pkgs := found.Artifacts.Packages.Sorted(); len(pkgs) > 0 {
builder.AddPackages(pkgs...)
}
if accessor, ok := builder.(sbomsync.Accessor); ok {
accessor.WriteToSBOM(func(s *sbom.SBOM) {
mergeFileArtifacts(&s.Artifacts, &found.Artifacts)
})
}
if len(found.Relationships) > 0 {
builder.AddRelationships(found.Relationships...)
}
}
// NewArchiveContainsTask relates each archive to every file the SBOM describes directly within it. It must
// run after every other task, so the files within archives are those the scan root's configuration
// selects (file selection, unknowns, package evidence) and no others.
func NewArchiveContainsTask() Task {
fn := func(_ context.Context, _ file.Resolver, builder sbomsync.Builder) error {
var rels []artifact.Relationship
builder.(sbomsync.Accessor).ReadFromSBOM(func(s *sbom.SBOM) {
rels = archiveContains(s.AllCoordinates())
})
builder.AddRelationships(rels...)
return nil
}
return NewTask("archive-contains", fn)
}
// archiveContains relates the archive holding each file within one to that file, and each such archive to
// the archive holding it, so an archive the SBOM does not otherwise describe still links what is within it.
func archiveContains(coordinates []file.Coordinates) []artifact.Relationship {
related := file.NewCoordinateSet(coordinates...)
var rels []artifact.Relationship
for _, c := range coordinates {
for {
container, ok := archive.ContainingArchive(c)
if !ok {
break
}
rels = append(rels, artifact.Relationship{From: container, To: c, Type: artifact.ContainsRelationship})
if related.Contains(container) {
break
}
related.Add(container)
c = container
}
}
return rels
}
func mergeFileArtifacts(dst, src *sbom.Artifacts) {
mergeInto(&dst.FileMetadata, src.FileMetadata)
mergeInto(&dst.FileDigests, src.FileDigests)
mergeInto(&dst.FileContents, src.FileContents)
mergeInto(&dst.FileLicenses, src.FileLicenses)
mergeInto(&dst.Executables, src.Executables)
if len(src.Unknowns) > 0 && dst.Unknowns == nil {
dst.Unknowns = map[file.Coordinates][]string{}
}
for coordinates, reasons := range src.Unknowns {
dst.Unknowns[coordinates] = append(dst.Unknowns[coordinates], reasons...)
}
}
func mergeInto[K comparable, V any](dst *map[K]V, src map[K]V) {
if len(src) == 0 {
return
}
if *dst == nil {
*dst = make(map[K]V, len(src))
}
maps.Copy(*dst, src)
}
+957
View File
@@ -0,0 +1,957 @@
package task
import (
"bytes"
"context"
"crypto/sha1"
"encoding/hex"
"errors"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/bmatcuk/doublestar/v4"
"github.com/scylladb/go-set/strset"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
stereoscopeFile "github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/internal/archive"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/mimetype"
"github.com/anchore/syft/internal/sbomsync"
"github.com/anchore/syft/internal/testutils"
"github.com/anchore/syft/internal/unknown"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cataloging"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/event/monitor"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
)
// dirTestResolver is a minimal file.Resolver over a directory. FilesByMIMEType sniffs each file's
// content as a real resolver does; FileContentsByLocation opens the file, which is random-access.
type dirTestResolver struct {
file.Resolver
dir string
fsid string
}
// streamingDirResolver is dirTestResolver handing out plain streams, so an archive's own bytes must be
// held in memory or written to disk rather than read in place.
type streamingDirResolver struct {
dirTestResolver
}
func (d streamingDirResolver) FileContentsByLocation(loc file.Location) (io.ReadCloser, error) {
f, err := d.dirTestResolver.FileContentsByLocation(loc)
if err != nil {
return nil, err
}
return struct {
io.Reader
io.Closer
}{f, f}, nil
}
// panickingDirResolver is dirTestResolver whose reads of one file panic, standing in for a decoder
// that panics on hostile bytes.
type panickingDirResolver struct {
dirTestResolver
path string
}
func (d panickingDirResolver) FileContentsByLocation(loc file.Location) (io.ReadCloser, error) {
if loc.RealPath == d.path {
return io.NopCloser(panicReader{}), nil
}
return d.dirTestResolver.FileContentsByLocation(loc)
}
type panicReader struct{}
func (panicReader) Read([]byte) (int, error) { panic("decoder blew up") }
func Test_archiveCataloger_aPanicCostsOnlyThatArchive(t *testing.T) {
rootDir := t.TempDir()
for _, name := range []string{"bad.zip", "good.zip"} {
require.NoError(t, os.WriteFile(filepath.Join(rootDir, name), testutils.Zip(t, map[string][]byte{"a.txt": []byte("a")}), 0o600))
}
var seen []string
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1), capturingTask(t, &seen, nil))
resolver := panickingDirResolver{dirTestResolver: dirTestResolver{dir: rootDir}, path: "/bad.zip"}
err := tsk.Execute(context.Background(), resolver, sbomsync.NewBuilder(newTestSBOM()))
// a coordinate error becomes an unknown; a plain error (or an escaped panic) would fail the scan
coordErrs, remaining := unknown.ExtractCoordinateErrors(err)
require.NoError(t, remaining)
require.Len(t, coordErrs, 1)
assert.Equal(t, "/bad.zip", coordErrs[0].Coordinates.RealPath)
assert.EqualError(t, coordErrs[0].Reason, "panic while cataloging archive: decoder blew up", "no stack trace in the SBOM-bound reason")
assert.Equal(t, []string{"/good.zip"}, seen, "the sibling is still cataloged")
}
func newTestTask(t *testing.T, cfg cataloging.ArchiveSearchConfig, subPipeline ...Task) Task {
t.Helper()
tsk := NewArchiveCatalogerTask(cfg, cataloging.UnknownsConfig{}, subPipeline, nil, nil)
require.NotNil(t, tsk)
return tsk
}
// newLimitedTestTask builds the task with the limiter's own limits, where zero forbids the resource.
func newLimitedTestTask(t *testing.T, maxDepth int, limits archive.Limits, subPipeline ...Task) Task {
t.Helper()
tsk := newArchiveCatalogerTask(maxDepth, limits, false, subPipeline, nil, nil)
require.NotNil(t, tsk)
return tsk
}
func newTestSBOM() *sbom.SBOM {
return &sbom.SBOM{Artifacts: sbom.Artifacts{Packages: pkg.NewCollection()}}
}
func countingTask(ran *int) Task {
return NewTask("count-runs", func(_ context.Context, _ file.Resolver, _ sbomsync.Builder) error {
*ran++
return nil
})
}
// capturingTask records the virtual path of every archive the sub-pipeline is run against and how many
// files its resolver holds.
func capturingTask(t *testing.T, seen *[]string, fileCounts *map[string]int) Task {
return NewTask("capture", func(ctx context.Context, r file.Resolver, _ sbomsync.Builder) error {
trav := archive.TraversalFromContext(ctx)
require.NotNil(t, trav)
*seen = append(*seen, archive.VirtualPath(trav.Location))
if fileCounts != nil {
locs, err := r.FilesByGlob("**")
require.NoError(t, err)
(*fileCounts)[archive.VirtualPath(trav.Location)] = len(locs)
}
return nil
})
}
func Test_archiveCataloger_truncationStillCatalogs(t *testing.T) {
// reaching a limit is a truncation, not a failure: the sub-pipeline still runs over what was stored
big := bytes.Repeat([]byte("x"), 4096)
outerZip := testutils.Zip(t, map[string][]byte{
"a/small.txt": []byte("small"),
"b/big.txt": big,
})
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "outer.zip"), outerZip, 0o600))
// nothing is held in memory, so each entry is charged to disk: small.txt lands, big.txt is refused
limits := archive.Limits{MaxMemoryBytes: 0, MaxDiskBytes: 6000}
var seen []string
fileCounts := map[string]int{}
tsk := newLimitedTestTask(t, 1, limits, capturingTask(t, &seen, &fileCounts))
s := newTestSBOM()
err := tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(s))
require.NoError(t, err, "a truncated extraction must not fail the scan")
assert.Equal(t, []string{"/outer.zip"}, seen, "the sub-pipeline must run over the partial contents")
assert.Equal(t, 1, fileCounts["/outer.zip"], "only the entry stored before the limit is visible")
reasons, ok := s.Artifacts.Unknowns[file.NewCoordinates("/outer.zip", "")]
require.True(t, ok, "the truncation must be recorded against the archive; got %v", s.Artifacts.Unknowns)
require.Len(t, reasons, 1)
assert.Contains(t, reasons[0], "archive cataloged from part of its contents")
assert.Contains(t, reasons[0], ArchiveCatalogerTaskName, "the unknown names the task that recorded it")
}
func Test_archiveCataloger_aDamagedEntryCostsOnlyThatEntry(t *testing.T) {
rootDir := t.TempDir()
// entry data overwritten but headers and end-of-central-directory intact: detected as an archive,
// and data.bin fails when it is read. ok.txt comes after it and is intact.
corrupt := testutils.Zip(t, map[string][]byte{"data.bin": incompressible(4096), "ok.txt": []byte("ok")})
require.Greater(t, len(corrupt), 1024)
for i := 100; i < 600; i++ {
corrupt[i] ^= 0xff
}
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "corrupt.zip"), corrupt, 0o600))
var seen []string
fileCounts := map[string]int{}
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1), capturingTask(t, &seen, &fileCounts))
s := newTestSBOM()
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(s)))
assert.Equal(t, []string{"/corrupt.zip"}, seen, "the archive is still cataloged")
assert.Equal(t, 1, fileCounts["/corrupt.zip"], "from the entries that could be read")
reasons := s.Artifacts.Unknowns[file.NewCoordinates("/corrupt.zip", "")]
require.Len(t, reasons, 1)
assert.Contains(t, reasons[0], `entry "data.bin" could not be read`)
}
func Test_NewArchiveCatalogerTask_gating(t *testing.T) {
someTask := NewTask("noop", func(_ context.Context, _ file.Resolver, _ sbomsync.Builder) error { return nil })
tests := []struct {
name string
cfg cataloging.ArchiveSearchConfig
subPipeline []Task
wantTask bool
}{
{
name: "disabled at depth 0",
cfg: cataloging.DefaultArchiveSearchConfig(),
subPipeline: []Task{someTask},
},
{
name: "no sub-pipeline",
cfg: cataloging.DefaultArchiveSearchConfig().WithMaxDepth(2),
subPipeline: nil,
},
{
name: "enabled",
cfg: cataloging.DefaultArchiveSearchConfig().WithMaxDepth(2),
subPipeline: []Task{someTask},
wantTask: true,
},
{
name: "enabled at negative depth",
cfg: cataloging.DefaultArchiveSearchConfig().WithMaxDepth(-1),
subPipeline: []Task{someTask},
wantTask: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := NewArchiveCatalogerTask(tt.cfg, cataloging.UnknownsConfig{}, tt.subPipeline, nil, nil)
if tt.wantTask {
require.NotNil(t, got)
assert.Equal(t, ArchiveCatalogerTaskName, got.Name())
return
}
assert.Nil(t, got)
})
}
}
func Test_archiveCataloger_depthStopsAtTheCeiling(t *testing.T) {
// "unbounded" must still end, or a zip quine or recursive bomb runs forever, and a configured depth
// past the ceiling must too, or the SBOM holds chains no decoder accepts back
chain := testutils.Zip(t, map[string][]byte{"deep.txt": []byte("deep")})
for range archive.MaxDepth + 4 {
chain = testutils.Zip(t, map[string][]byte{"n.zip": chain})
}
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "outer.zip"), chain, 0o600))
for _, depth := range []int{-1, archive.MaxDepth + 2} {
t.Run(fmt.Sprint(depth), func(t *testing.T) {
var seen []string
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(depth), capturingTask(t, &seen, nil))
_ = tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(newTestSBOM()))
assert.Len(t, seen, archive.MaxDepth)
})
}
}
func Test_archiveCataloger_anArchiveIdenticalToAnAncestorIsNotEntered(t *testing.T) {
// a zip quine contains itself. Building one is impractical, so the walk is started as if it were
// already inside an archive with these exact bytes, which is the state a quine reaches one level down
outerZip := testutils.Zip(t, map[string][]byte{"a.txt": []byte("a")})
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "outer.zip"), outerZip, 0o600))
var ran int
c := newTestCataloger(-1, archive.NewLimiter(archive.Limits{MaxMemoryBytes: -1, MaxDiskBytes: -1}), countingTask(&ran))
sum := sha1.Sum(outerZip)
c.ancestors = []file.Digest{{Algorithm: "sha1", Value: hex.EncodeToString(sum[:])}}
s := newTestSBOM()
require.NoError(t, c.catalog(context.Background(), dirTestResolver{dir: rootDir}, 0, sbomsync.NewBuilder(s)))
assert.Zero(t, ran, "the sub-pipeline must not run inside a repeat of an ancestor")
reasons := s.Artifacts.Unknowns[file.NewCoordinates("/outer.zip", "")]
require.Len(t, reasons, 1)
assert.Contains(t, reasons[0], "identical to an archive that contains it")
assert.Len(t, c.ancestors, 1, "the walk leaves the ancestor stack as it found it")
}
func Test_archiveCataloger_stopsWhenTheContextIsCancelled(t *testing.T) {
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "outer.zip"), testutils.Zip(t, map[string][]byte{"a.txt": []byte("a")}), 0o600))
ctx, cancel := context.WithCancel(context.Background())
cancel()
var ran int
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1), countingTask(&ran))
err := tsk.Execute(ctx, dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(newTestSBOM()))
require.ErrorIs(t, err, context.Canceled)
assert.Zero(t, ran)
}
func Test_archiveCataloger_decompressionBudgetIsSharedByATreeAndFreshPerTopLevelArchive(t *testing.T) {
// a recursive bomb fits the limits one level at a time, so what bounds it is the total decompressed
// under one top-level archive
origRatio, origFloor := archiveTreeRatio, archiveTreeFloorBytes
archiveTreeRatio, archiveTreeFloorBytes = 0, 100_000
t.Cleanup(func() { archiveTreeRatio, archiveTreeFloorBytes = origRatio, origFloor })
zeros := make([]byte, 60_000)
nested := testutils.Zip(t, map[string][]byte{"zeros.bin": zeros})
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "bomb.zip"), testutils.Zip(t, map[string][]byte{"a.zip": nested, "b.zip": nested}), 0o600))
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "sibling.zip"), testutils.Zip(t, map[string][]byte{"zeros.bin": zeros}), 0o600))
var seen []string
fileCounts := map[string]int{}
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(2), capturingTask(t, &seen, &fileCounts))
s := newTestSBOM()
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(s)))
assert.Equal(t, 1, fileCounts["/bomb.zip:a.zip"], "the first nested archive fits the tree's budget")
assert.Equal(t, 0, fileCounts["/bomb.zip:b.zip"], "the second does not: the budget is shared, not per archive")
reasons := s.Artifacts.Unknowns[file.Coordinates{RealPath: "/bomb.zip", FullPath: "/bomb.zip!/b.zip"}]
require.Len(t, reasons, 1)
assert.Contains(t, reasons[0], "possible decompression bomb")
assert.Equal(t, 1, fileCounts["/sibling.zip"], "another top-level archive starts with a fresh budget")
}
func Test_archiveCataloger_noSelfContainsEdge(t *testing.T) {
// a jar java cannot identify records an unknown at its own coordinates; that must not turn into
// "x.jar CONTAINS x.jar", which a graph walker would loop on
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "x.jar"), testutils.Zip(t, map[string][]byte{"readme.txt": []byte("no manifest")}), 0o600))
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1), javaTask(), fileMetadataTask())
s := newTestSBOM()
s.Artifacts.FileMetadata = map[file.Coordinates]file.Metadata{}
_ = tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(s))
withArchiveContains(t, s)
require.Contains(t, s.Artifacts.Unknowns, file.NewCoordinates("/x.jar", ""), "the fixture must reproduce the unknown")
var contains int
for _, rel := range s.Relationships {
if rel.Type != artifact.ContainsRelationship {
continue
}
contains++
assert.NotEqual(t, rel.From.ID(), rel.To.ID(), "self-referential CONTAINS edge: %+v", rel)
}
assert.Positive(t, contains, "the archive still contains its files")
}
func Test_NewArchiveContainsTask(t *testing.T) {
// a file within an archive is contained only when the SBOM describes it, as a file at the scan root is
// listed only then; an executable no package owns is recorded whatever the file selection
in := func(fullPath string) file.Coordinates {
return file.Coordinates{RealPath: "/outer.tar", FileSystemID: "layer", FullPath: fullPath}
}
outer := file.NewCoordinates("/outer.tar", "layer")
jar := in("/outer.tar!/lib/app.jar")
manifest := in("/outer.tar!/lib/app.jar!/META-INF/MANIFEST.MF")
unknownExe := in("/outer.tar!/bin/setup.exe")
s := newTestSBOM()
s.Artifacts.FileMetadata = map[file.Coordinates]file.Metadata{manifest: {}, file.NewCoordinates("/loose.txt", "layer"): {}}
s.Artifacts.Executables = map[file.Coordinates]file.Executable{in("/outer.tar!/bin/tool"): {}}
s.Artifacts.Unknowns = map[file.Coordinates][]string{unknownExe: {"no package identified in executable file"}}
withArchiveContains(t, s)
type edge struct{ from, to file.Coordinates }
var edges []edge
for _, r := range s.Relationships {
assert.Equal(t, artifact.ContainsRelationship, r.Type)
edges = append(edges, edge{r.From.(file.Coordinates), r.To.(file.Coordinates)})
}
assert.ElementsMatch(t, []edge{
{jar, manifest},
{outer, jar}, // the jar is not described itself, but is on the way to what is
{outer, unknownExe},
}, edges)
}
func Test_NewArchiveCatalogerTask_dropsItselfFromSubPipeline(t *testing.T) {
// a copy of itself in the sub-pipeline would process every nesting level twice
cfg := cataloging.DefaultArchiveSearchConfig().WithMaxDepth(2)
innerZip := testutils.Zip(t, map[string][]byte{"leaf.txt": []byte("leaf")})
outerZip := testutils.Zip(t, map[string][]byte{"nested/inner.zip": innerZip})
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "outer.zip"), outerZip, 0o600))
var ran int
countTask := countingTask(&ran)
poisoned := []Task{countTask, NewArchiveCatalogerTask(cfg, cataloging.UnknownsConfig{}, []Task{countTask}, nil, nil)}
require.NotNil(t, poisoned[1])
tsk := newTestTask(t, cfg, poisoned...)
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(newTestSBOM())))
assert.Equal(t, 2, ran, "two archives, one sub-pipeline task each")
t.Run("a sub-pipeline of nothing but itself yields no task", func(t *testing.T) {
only := []Task{NewArchiveCatalogerTask(cfg, cataloging.UnknownsConfig{}, []Task{countTask}, nil, nil)}
assert.Nil(t, NewArchiveCatalogerTask(cfg, cataloging.UnknownsConfig{}, only, nil, nil))
})
}
func Test_archiveCataloger_nestedArchivesKeepTheLayerFileSystemID(t *testing.T) {
// extracted files keep the layer digest as their FileSystemID at every level; the nesting chain
// rides on the traversal's VirtualPath and the coordinate's FullPath
innerZip := testutils.Zip(t, map[string][]byte{"leaf.txt": []byte("leaf")})
outerZip := testutils.Zip(t, map[string][]byte{"nested/inner.zip": innerZip})
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "outer.zip"), outerZip, 0o600))
const layerDigest = "sha256:1111111111111111111111111111111111111111111111111111111111111111"
var chains, fsids []string
captureTask := NewTask("capture-fsid", func(ctx context.Context, _ file.Resolver, _ sbomsync.Builder) error {
trav := archive.TraversalFromContext(ctx)
require.NotNil(t, trav)
chains = append(chains, archive.VirtualPath(trav.Location))
fsids = append(fsids, trav.Location.FileSystemID)
return nil
})
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(2), captureTask)
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: rootDir, fsid: layerDigest}, sbomsync.NewBuilder(newTestSBOM())))
assert.Equal(t, []string{layerDigest, layerDigest}, fsids)
assert.ElementsMatch(t, []string{"/outer.zip", "/outer.zip:nested/inner.zip"}, chains)
}
// twoLayerResolver reports the same archive path under two filesystem ids with different content, as an
// all-layers image scan does for an archive rewritten in a later layer.
type twoLayerResolver struct {
file.Resolver
path string
contentBy map[string][]byte
}
func Test_archiveCataloger_sameArchivePathInTwoLayersDoesNotCollide(t *testing.T) {
resolver := twoLayerResolver{
path: "/app/bundle.zip",
contentBy: map[string][]byte{
"layer-one": testutils.Zip(t, map[string][]byte{"lib/config.json": []byte(`{"v":1}`)}),
"layer-two": testutils.Zip(t, map[string][]byte{"lib/config.json": []byte(`{"v":2}`)}),
},
}
var chains []string
fileTask := NewTask("record-files", func(ctx context.Context, r file.Resolver, builder sbomsync.Builder) error {
trav := archive.TraversalFromContext(ctx)
require.NotNil(t, trav)
chains = append(chains, trav.Location.FileSystemID)
locs, err := r.FilesByGlob("**")
require.NoError(t, err)
metadata := map[file.Coordinates]file.Metadata{}
for _, loc := range locs {
metadata[loc.Coordinates] = file.Metadata{Path: loc.RealPath}
}
builder.(sbomsync.Accessor).WriteToSBOM(func(s *sbom.SBOM) {
s.Artifacts.FileMetadata = metadata
})
return nil
})
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1), fileTask)
s := &sbom.SBOM{Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(),
FileMetadata: map[file.Coordinates]file.Metadata{},
}}
require.NoError(t, tsk.Execute(context.Background(), resolver, sbomsync.NewBuilder(s)))
assert.ElementsMatch(t, []string{"layer-one", "layer-two"}, chains)
// the tables are keyed by Coordinates, so a collision is a silently missing entry
assert.Len(t, s.Artifacts.FileMetadata, 2)
}
func Test_archiveCataloger_subPipelineFailureIsRecordedAsAnUnknown(t *testing.T) {
// runSubPipeline bypasses RunTask, where the coordinate-error to unknown conversion lives
outerZip := testutils.Zip(t, map[string][]byte{"lib/broken.json": []byte("{")})
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "outer.zip"), outerZip, 0o600))
cfg := cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1)
t.Run("a coordinate error keeps the coordinates the cataloger named", func(t *testing.T) {
failing := NewTask("failing-cataloger", func(_ context.Context, r file.Resolver, _ sbomsync.Builder) error {
locs, err := r.FilesByGlob("**")
require.NoError(t, err)
require.NotEmpty(t, locs)
var errs error
for _, loc := range locs {
errs = unknown.Append(errs, loc, errors.New("unable to parse"))
}
return errs
})
tsk := newTestTask(t, cfg, failing)
s := newTestSBOM()
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(s)),
"a cataloger failing inside an archive must not fail the archive or the scan")
require.Len(t, s.Artifacts.Unknowns, 1)
for coords, reasons := range s.Artifacts.Unknowns {
assert.Equal(t, "/outer.zip!/lib/broken.json", coords.FullNestedPath(), "the inner coordinates must survive the merge")
assert.Equal(t, "/outer.zip", coords.RealPath, "and must be addressable per archive")
assert.Equal(t, []string{"failing-cataloger: unable to parse"}, reasons)
}
})
t.Run("an error with no coordinates is attributed to the archive", func(t *testing.T) {
failing := NewTask("bare-error-cataloger", func(_ context.Context, _ file.Resolver, _ sbomsync.Builder) error {
return errors.New("cataloger blew up")
})
tsk := newTestTask(t, cfg, failing)
s := newTestSBOM()
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(s)))
require.Len(t, s.Artifacts.Unknowns, 1)
reasons, ok := s.Artifacts.Unknowns[file.NewCoordinates("/outer.zip", "")]
require.True(t, ok, "an error with no location of its own belongs to the containing archive")
assert.Equal(t, []string{"bare-error-cataloger: cataloger blew up"}, reasons)
})
t.Run("a panicking cataloger is recovered and recorded", func(t *testing.T) {
panicking := NewTask("panicking-cataloger", func(_ context.Context, _ file.Resolver, _ sbomsync.Builder) error {
panic("nope")
})
tsk := newTestTask(t, cfg, panicking)
s := newTestSBOM()
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(s)),
"a panic inside an archive must not fail the scan")
reasons, ok := s.Artifacts.Unknowns[file.NewCoordinates("/outer.zip", "")]
require.True(t, ok)
require.Len(t, reasons, 1)
assert.Equal(t, "panicking-cataloger: panic: nope", reasons[0])
})
}
func Test_archiveCataloger_subPipelineUnknownsAreAddedNotReplaced(t *testing.T) {
outerZip := testutils.Zip(t, map[string][]byte{"lib/thing.json": []byte("{}")})
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "outer.zip"), outerZip, 0o600))
preexisting := file.Coordinates{RealPath: "/outer.zip", FullPath: "/outer.zip!/lib/thing.json"}
writer := NewTask("scratch-writer", func(_ context.Context, _ file.Resolver, builder sbomsync.Builder) error {
builder.(sbomsync.Accessor).WriteToSBOM(func(s *sbom.SBOM) {
if s.Artifacts.Unknowns == nil {
s.Artifacts.Unknowns = map[file.Coordinates][]string{}
}
s.Artifacts.Unknowns[preexisting] = append(s.Artifacts.Unknowns[preexisting], "recorded by the cataloger itself")
})
return nil
})
failing := NewTask("failing-cataloger", func(_ context.Context, _ file.Resolver, _ sbomsync.Builder) error {
return errors.New("boom")
})
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1), writer, failing)
s := newTestSBOM()
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(s)))
assert.Equal(t, []string{"recorded by the cataloger itself"}, s.Artifacts.Unknowns[preexisting])
assert.Equal(t, []string{"failing-cataloger: boom"}, s.Artifacts.Unknowns[file.NewCoordinates("/outer.zip", "")])
}
func Test_archiveCataloger_limitsFallWhenAnArchiveIsReleased(t *testing.T) {
entry := bytes.Repeat([]byte("z"), 400)
entries := map[string][]byte{}
for i := range 5 {
entries[fmt.Sprintf("e%d.bin", i)] = entry
}
rootDir := t.TempDir()
for _, name := range []string{"one.zip", "two.zip"} {
require.NoError(t, os.WriteFile(filepath.Join(rootDir, name), testutils.Zip(t, entries), 0o600))
}
// room in memory for one archive's entries and their index cost, not for two at once
limiter := archive.NewLimiter(archive.Limits{MaxMemoryBytes: 20000, MaxDiskBytes: 0})
// sampled while the archive is still held
var extracted []int
var peakMemory int64
sample := NewTask("sample", func(_ context.Context, r file.Resolver, _ sbomsync.Builder) error {
locs, err := r.FilesByGlob("**")
require.NoError(t, err)
extracted = append(extracted, len(locs))
if mem, _ := limiter.InUse(); mem > peakMemory {
peakMemory = mem
}
return nil
})
c := newTestCataloger(1, limiter, sample)
require.NoError(t, c.catalog(context.Background(), dirTestResolver{dir: rootDir}, 0, sbomsync.NewBuilder(newTestSBOM())))
assert.Equal(t, []int{5, 5}, extracted,
"both archives must extract in full: the first is released before the second is admitted")
assert.Positive(t, peakMemory, "entries within the memory limit are held in memory")
mem, disk := limiter.InUse()
assert.Zero(t, mem, "the memory limit must fall back to nothing once the walk unwinds")
assert.Zero(t, disk, "and so must the disk limit")
}
func newTestCataloger(maxDepth int, limiter *archive.Limiter, subPipeline ...Task) *archiveCataloger {
return &archiveCataloger{
maxDepth: maxDepth,
pkgTasks: subPipeline,
limiter: limiter,
progress: bus.StartCatalogerTask(context.Background(), archiveCatalogerProgressInfo(), -1, ""),
}
}
func Test_archiveCataloger_archiveWhoseOwnBytesExceedTheDiskLimitIsSkipped(t *testing.T) {
// a streamed archive's own bytes must be placed before it can be opened; when they do not fit on
// disk the archive is skipped rather than waited for, and a sibling that fits is cataloged in full
oversized := testutils.Zip(t, map[string][]byte{"payload.bin": incompressible(8000)})
small := testutils.Zip(t, map[string][]byte{"ok.txt": []byte("ok")})
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "oversized.zip"), oversized, 0o600))
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "small.zip"), small, 0o600))
// room for the small archive's bytes, its one entry and that entry's index cost
diskLimit := int64(len(small)) + 3000
require.Greater(t, int64(len(oversized)), diskLimit, "the fixture must actually exceed the limit")
limits := archive.Limits{MaxMemoryBytes: 0, MaxDiskBytes: diskLimit}
var seen []string
fileCounts := map[string]int{}
tsk := newLimitedTestTask(t, 1, limits, capturingTask(t, &seen, &fileCounts))
s := newTestSBOM()
require.NoError(t, tsk.Execute(context.Background(), streamingDirResolver{dirTestResolver{dir: rootDir}}, sbomsync.NewBuilder(s)),
"reaching a limit must not fail the scan")
assert.Equal(t, []string{"/small.zip"}, seen, "the oversized archive is skipped and the sibling is still cataloged")
assert.Equal(t, 1, fileCounts["/small.zip"])
reasons := s.Artifacts.Unknowns[file.NewCoordinates("/oversized.zip", "")]
require.Len(t, reasons, 1, "the skip must be recorded against the archive; got %v", s.Artifacts.Unknowns)
assert.Contains(t, reasons[0], "archive skipped")
}
func Test_archiveCataloger_discoverArchivesSelectsTarFamily(t *testing.T) {
marker := map[string][]byte{"nested/marker.txt": []byte("hello")}
scanDir := t.TempDir()
for name, body := range map[string][]byte{
"bundle.tar.gz": testutils.TarGz(t, marker),
"bundle.tgz": testutils.TarGz(t, marker),
"bundle.tar": testutils.Tar(t, marker),
"bundle.zip": testutils.Zip(t, marker),
"notes.txt": []byte("not an archive at all"),
} {
require.NoError(t, os.WriteFile(filepath.Join(scanDir, name), body, 0o600))
}
var seen []string
fileCounts := map[string]int{}
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1), capturingTask(t, &seen, &fileCounts))
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: scanDir}, sbomsync.NewBuilder(newTestSBOM())))
assert.ElementsMatch(t, []string{"/bundle.tar.gz", "/bundle.tgz", "/bundle.tar", "/bundle.zip"}, seen,
"every tar-family archive must be reached, and the plain text file must not be")
for _, archivePath := range seen {
assert.Equal(t, 1, fileCounts[archivePath], "%s must have been extracted", archivePath)
}
t.Run("the MIME types the tar family is sniffed as are in the set discovery asks for", func(t *testing.T) {
for _, mt := range []string{"application/x-tar", "application/gzip", "application/x-gtar", "application/zip"} {
assert.True(t, mimetype.ArchiveMIMETypeSet.Has(mt), "%s must be a candidate archive type", mt)
}
})
}
type recordingPublisher struct {
events []partybus.Event
}
func Test_archiveCatalogerTask_publishesOneProgressRowPerCataloger(t *testing.T) {
// a consumer keying rows by ID (as the syft CLI does) replaces a republished row and never sees it
// complete, so each cataloger keeps to the one row it already owns
inner := testutils.Zip(t, map[string][]byte{"inner/file.txt": []byte("hello")})
scanDir := t.TempDir()
for _, name := range []string{"one.zip", "two.zip"} {
require.NoError(t, os.WriteFile(filepath.Join(scanDir, name), inner, 0o600))
}
publisher := &recordingPublisher{}
bus.Set(publisher)
t.Cleanup(func() { bus.Set(nil) })
// the archive row's stage as each archive's sub-pipeline starts: the previous archive is done by then,
// so the row must read the walk's own count rather than anything a nested cataloger left behind
var rows []*monitor.TaskProgress
var stageAtEntry []string
noisy := NewTask("noisy", func(ctx context.Context, _ file.Resolver, _ sbomsync.Builder) error {
stageAtEntry = append(stageAtEntry, publisher.events[0].Value.(progress.StagedProgressable).Stage())
p := bus.StartCatalogerTask(ctx, monitor.GenericTask{ID: "noisy"}, 3, "")
rows = append(rows, p)
p.AtomicStage.Set("working on something")
p.Add(3)
p.SetCompleted()
return nil
})
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1), noisy)
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: scanDir}, sbomsync.NewBuilder(newTestSBOM())))
var started []string
for _, e := range publisher.events {
if e.Type != event.CatalogerTaskStarted {
continue
}
info, ok := e.Source.(monitor.GenericTask)
require.True(t, ok, "expected a generic task as the event source")
started = append(started, info.ID)
}
assert.Equal(t, []string{ArchiveCatalogerTaskName, "noisy"}, started,
"each cataloger publishes one row, however many archives the walk enters")
require.Len(t, rows, 2, "the sub-pipeline ran once per archive")
assert.Same(t, rows[0], rows[1], "both runs must report into the one row that cataloger owns")
assert.Equal(t, int64(6), rows[0].Current(), "what every run found is counted onto that row")
require.Len(t, publisher.events, 2)
prog, ok := publisher.events[0].Value.(progress.StagedProgressable)
require.True(t, ok)
assert.ErrorIs(t, prog.Error(), progress.ErrCompleted, "the row must be completed, or a waiting consumer never finishes")
assert.Equal(t, int64(2), prog.Current(), "the archive row counts every archive the walk entered")
assert.Equal(t, "2 archives", prog.Stage(), "and says so once the walk is done")
// one.zip sorts before two.zip, and the resolver walks the scan directory in lexical order
assert.Equal(t, []string{"", "1 archives (/one.zip)"}, stageAtEntry,
"the row must take its stage back from the sub-pipeline once an archive is done")
}
func Test_archiveCataloger_reportsPeakUsageAndTheSlowestArchive(t *testing.T) {
// self time excludes nested archives, or the outer archive would be named every scan
innerJar := testutils.Zip(t, map[string][]byte{"README.txt": []byte("inner")})
outerZip := testutils.Zip(t, map[string][]byte{"lib/inner.jar": innerJar})
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "outer.zip"), outerZip, 0o600))
const innerWork = 40 * time.Millisecond
slow := NewTask("slow-inside-the-inner-archive", func(ctx context.Context, _ file.Resolver, _ sbomsync.Builder) error {
if trav := archive.TraversalFromContext(ctx); trav != nil && strings.HasSuffix(archive.VirtualPath(trav.Location), "inner.jar") {
time.Sleep(innerWork)
}
return nil
})
limiter := archive.NewLimiter(archive.Limits{MaxMemoryBytes: -1, MaxDiskBytes: -1})
c := newTestCataloger(3, limiter, slow)
require.NoError(t, c.catalog(context.Background(), dirTestResolver{dir: rootDir}, 0, sbomsync.NewBuilder(newTestSBOM())))
assert.Equal(t, "/outer.zip!/lib/inner.jar", c.slowestPath, "the slowest archive is named by its full chain")
assert.GreaterOrEqual(t, c.slowest, innerWork)
assert.Equal(t, int64(2), c.progress.Current())
// the peaks outlive the walk that set them
peakMemory, _ := limiter.Peak()
assert.Positive(t, peakMemory, "entries held in memory must show on the peak")
memory, disk := limiter.InUse()
assert.Zero(t, memory)
assert.Zero(t, disk)
}
func Test_archiveCataloger_slowestIsUnsetWhenNoArchiveIsCataloged(t *testing.T) {
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "notes.txt"), []byte("not an archive"), 0o600))
noop := NewTask("noop", func(context.Context, file.Resolver, sbomsync.Builder) error { return nil })
c := newTestCataloger(2, archive.NewLimiter(archive.Limits{MaxMemoryBytes: -1, MaxDiskBytes: -1}), noop)
require.NoError(t, c.catalog(context.Background(), dirTestResolver{dir: rootDir}, 0, sbomsync.NewBuilder(newTestSBOM())))
assert.Empty(t, c.slowestPath)
assert.Zero(t, c.progress.Current())
c.logStats() // must not panic with nothing to report
}
func Test_archiveCataloger_aMisnamedNonArchiveIsNotAnError(t *testing.T) {
// the name says archive, the content does not: it is never opened
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "notes.zip"),
[]byte("this is prose, not an archive, whatever the extension claims\n"), 0o600))
var ran int
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1), countingTask(&ran))
err := tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(newTestSBOM()))
assert.NoError(t, err, "a file that is not an archive is not a failure to report")
assert.Zero(t, ran, "and nothing is cataloged inside it")
}
func Test_archiveCataloger_catalogsAZipBehindALauncherScript(t *testing.T) {
// the Spring Boot executable jar shape: content sniffing types it as a shell script, so it reaches
// the walk only by name and qualifies only by its end-of-central-directory record
inner := testutils.Zip(t, map[string][]byte{"nested/marker.txt": []byte("found me")})
prefixed := append([]byte("#!/bin/bash\nexec java -jar \"$0\" \"$@\"\nexit 0\n"),
testutils.Zip(t, map[string][]byte{"BOOT-INF/lib/inner.jar": inner})...)
rootDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(rootDir, "app.jar"), prefixed, 0o600))
var seen []string
tsk := newTestTask(t, cataloging.DefaultArchiveSearchConfig().WithMaxDepth(3), capturingTask(t, &seen, nil))
require.NoError(t, tsk.Execute(context.Background(), dirTestResolver{dir: rootDir}, sbomsync.NewBuilder(newTestSBOM())))
assert.Equal(t, []string{"/app.jar", "/app.jar:BOOT-INF/lib/inner.jar"}, seen,
"the stub must not hide the archive, and the walk must carry on into what it holds")
}
func (d dirTestResolver) FilesByMIMEType(types ...string) ([]file.Location, error) {
return filesByMIMEType(d.dir, d.fsid, types...)
}
func filesByMIMEType(dir, fsid string, types ...string) ([]file.Location, error) {
wanted := strset.New(types...)
var locs []file.Location
err := filepath.WalkDir(dir, func(path string, entry fs.DirEntry, err error) error {
if err != nil || entry.IsDir() {
return err
}
f, err := os.Open(path)
if err != nil {
return err
}
mimeType := stereoscopeFile.MIMEType(f)
if closeErr := f.Close(); closeErr != nil {
return closeErr
}
if !wanted.Has(mimeType) {
return nil
}
rel, err := filepath.Rel(dir, path)
if err != nil {
return err
}
coords := file.Coordinates{RealPath: "/" + filepath.ToSlash(rel), FileSystemID: fsid}
locs = append(locs, file.NewLocationFromCoordinates(coords))
return nil
})
return locs, err
}
func (d dirTestResolver) FileContentsByLocation(loc file.Location) (io.ReadCloser, error) {
return os.Open(filepath.Join(d.dir, filepath.FromSlash(loc.RealPath)))
}
func (d dirTestResolver) FileMetadataByLocation(loc file.Location) (file.Metadata, error) {
info, err := os.Stat(filepath.Join(d.dir, filepath.FromSlash(loc.RealPath)))
return file.Metadata{FileInfo: info}, err
}
// matchesAnyGlob matches the resolver-relative path with a leading slash, the form a real resolver
// globs over. No patterns matches everything.
func matchesAnyGlob(root, path string, patterns []string) bool {
if len(patterns) == 0 {
return true
}
rel, err := filepath.Rel(root, path)
if err != nil {
return false
}
rel = "/" + filepath.ToSlash(rel)
for _, pattern := range patterns {
if ok, err := doublestar.Match(pattern, rel); err == nil && ok {
return true
}
}
return false
}
func (r twoLayerResolver) FilesByGlob(_ ...string) ([]file.Location, error) {
return nil, nil
}
func (d dirTestResolver) FilesByGlob(patterns ...string) ([]file.Location, error) {
var locs []file.Location
err := filepath.WalkDir(d.dir, func(path string, entry fs.DirEntry, err error) error {
if err != nil || entry.IsDir() {
return err
}
if !matchesAnyGlob(d.dir, path, patterns) {
return nil
}
rel, err := filepath.Rel(d.dir, path)
if err != nil {
return err
}
coords := file.Coordinates{RealPath: "/" + filepath.ToSlash(rel), FileSystemID: d.fsid}
locs = append(locs, file.NewLocationFromCoordinates(coords))
return nil
})
return locs, err
}
func (r twoLayerResolver) FilesByMIMEType(_ ...string) ([]file.Location, error) {
var locs []file.Location
for _, fsid := range []string{"layer-one", "layer-two"} {
locs = append(locs, file.NewLocationFromCoordinates(file.NewCoordinates(r.path, fsid)))
}
return locs, nil
}
func (r twoLayerResolver) FileContentsByLocation(loc file.Location) (io.ReadCloser, error) {
body, ok := r.contentBy[loc.FileSystemID]
if !ok {
return nil, os.ErrNotExist
}
return io.NopCloser(bytes.NewReader(body)), nil
}
// FileMetadataByLocation reports no metadata, so the archive falls back to the minimum budget.
func (r twoLayerResolver) FileMetadataByLocation(file.Location) (file.Metadata, error) {
return file.Metadata{}, os.ErrNotExist
}
// incompressible returns n bytes deflate cannot shrink, so an archive built from them has a
// predictable size to set a limit against.
func incompressible(n int) []byte {
b := make([]byte, n)
x := uint32(12345)
for i := range b {
x = x*1664525 + 1013904223
b[i] = byte(x >> 24)
}
return b
}
func (p *recordingPublisher) Publish(e partybus.Event) {
p.events = append(p.events, e)
}
+4 -4
View File
@@ -47,7 +47,7 @@ func newFileDigestCatalogerTask(selection file.Selection, hashers []crypto.Hash,
result, err := filedigest.NewCataloger(hashers).Catalog(ctx, resolver, coordinates...)
accessor.WriteToSBOM(func(sbom *sbom.SBOM) {
sbom.Artifacts.FileDigests = result
mergeInto(&sbom.Artifacts.FileDigests, result)
})
return err
@@ -78,7 +78,7 @@ func newFileMetadataCatalogerTask(selection file.Selection, tags ...string) Task
result, err := filemetadata.NewCataloger().Catalog(ctx, resolver, coordinates...)
accessor.WriteToSBOM(func(sbom *sbom.SBOM) {
sbom.Artifacts.FileMetadata = result
mergeInto(&sbom.Artifacts.FileMetadata, result)
})
return err
@@ -104,7 +104,7 @@ func newFileContentCatalogerTask(cfg filecontent.Config, tags ...string) Task {
result, err := filecontent.NewCataloger(cfg).Catalog(ctx, resolver)
accessor.WriteToSBOM(func(sbom *sbom.SBOM) {
sbom.Artifacts.FileContents = result
mergeInto(&sbom.Artifacts.FileContents, result)
})
return err
@@ -130,7 +130,7 @@ func newExecutableCatalogerTask(selection file.Selection, cfg executable.Config,
result, err := executable.NewCataloger(cfg).CatalogCtx(ctx, resolver)
accessor.WriteToSBOM(func(sbom *sbom.SBOM) {
sbom.Artifacts.Executables = result
mergeInto(&sbom.Artifacts.Executables, result)
})
return err
+4 -16
View File
@@ -40,19 +40,13 @@ func Test_coordinatesForSelection(t *testing.T) {
Relationships: []artifact.Relationship{
{
From: pkg.Package{},
To: file.Coordinates{
RealPath: "path",
FileSystemID: "fs",
},
To: file.NewCoordinates("path", "fs"),
Type: artifact.ContainsRelationship,
},
},
},
files: []file.Coordinates{
{
RealPath: "path",
FileSystemID: "fs",
},
file.NewCoordinates("path", "fs"),
},
ok: true,
},
@@ -63,10 +57,7 @@ func Test_coordinatesForSelection(t *testing.T) {
Relationships: []artifact.Relationship{
{
From: pkg.Package{},
To: file.Coordinates{
RealPath: "path",
FileSystemID: "fs",
},
To: file.NewCoordinates("path", "fs"),
// wrong type
Type: artifact.DependencyOfRelationship,
},
@@ -82,10 +73,7 @@ func Test_coordinatesForSelection(t *testing.T) {
Relationships: []artifact.Relationship{
{
From: file.Coordinates{}, // wrong type
To: file.Coordinates{
RealPath: "path",
FileSystemID: "fs",
},
To: file.NewCoordinates("path", "fs"),
Type: artifact.ContainsRelationship,
},
},
+13 -6
View File
@@ -3,9 +3,11 @@ package task
import (
"context"
"fmt"
"path"
"strings"
"unicode"
"github.com/anchore/syft/internal/archive"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/relationship"
@@ -48,7 +50,7 @@ func NewPackageTask(cfg CatalogingFactoryConfig, c pkg.Cataloger, tags ...string
HideOnSuccess: true,
}
t := bus.StartCatalogerTask(info, -1, "")
t := bus.StartCatalogerTask(ctx, info, -1, "")
pkgs, relationships, err := c.Catalog(ctx, resolver)
@@ -167,7 +169,7 @@ func applyComplianceRules(p *pkg.Package, cfg cataloging.ComplianceConfig) (bool
loc := "unknown"
locs := p.Locations.ToSlice()
if len(locs) > 0 {
loc = locs[0].Path()
loc = locs[0].FullNestedPath()
}
switch action {
case cataloging.ComplianceActionDrop:
@@ -236,10 +238,10 @@ func packageFileOwnershipRelationships(p pkg.Package, resolver file.PathResolver
locations := map[artifact.ID]file.Location{}
for _, path := range fileOwner.OwnedFiles() {
pathRefs, err := resolver.FilesByPath(path)
for _, ownedPath := range fileOwner.OwnedFiles() {
pathRefs, err := resolver.FilesByPath(ownedPath)
if err != nil {
return nil, fmt.Errorf("unable to find path for path=%q: %w", path, err)
return nil, fmt.Errorf("unable to find path for path=%q: %w", ownedPath, err)
}
if len(pathRefs) == 0 {
@@ -250,8 +252,13 @@ func packageFileOwnershipRelationships(p pkg.Package, resolver file.PathResolver
}
for _, ref := range pathRefs {
// an archive looks a relative path up from its root, but an owned path may be relative to
// anything (a python RECORD is relative to site-packages), so only an absolute one owns a file in one
if archive.ContainerPath(ref.Coordinates) != "" && !path.IsAbs(ownedPath) {
continue
}
if oldRef, ok := locations[ref.ID()]; ok {
log.Debugf("found path duplicate of %s", oldRef.RealPath)
log.Debugf("found path duplicate of %s", oldRef.FullNestedPath())
}
locations[ref.ID()] = ref
}
+31 -2
View File
@@ -1,11 +1,15 @@
package task
import (
"bytes"
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal/archive"
"github.com/anchore/syft/internal/testutils"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cataloging"
"github.com/anchore/syft/syft/cpe"
@@ -64,8 +68,8 @@ func TestApplyCompliance(t *testing.T) {
p2 := pkg.Package{Name: "", Version: "1.0"} // missing name
p3 := pkg.Package{Name: "pkg-3", Version: ""} // missing version
p4 := pkg.Package{Name: "pkg-4", Version: ""} // missing version
c1 := file.Coordinates{RealPath: "/coords/1"}
c2 := file.Coordinates{RealPath: "/coords/2"}
c1 := file.NewCoordinates("/coords/1", "")
c2 := file.NewCoordinates("/coords/2", "")
for _, p := range []*pkg.Package{&p1, &p2, &p3, &p4} {
p.SetID()
@@ -281,3 +285,28 @@ func TestApplyComplianceRules_DropAndStub(t *testing.T) {
assert.False(t, isCompliant)
assert.Equal(t, cataloging.UnknownStubValue, p.Version)
}
func Test_packageFileOwnershipRelationships_exactPathsWithinAnArchive(t *testing.T) {
// a wheel's RECORD is relative to site-packages: looked up from the wheel's root it finds the file, but
// only an exact path owns one, as at the scan root
data := testutils.Zip(t, map[string]string{"pip/__init__.py": "", "bin/tool": ""})
whl, err := archive.Extract(context.Background(), bytes.NewReader(data), file.NewCoordinates("/pip.whl", ""),
archive.NewLimiter(archive.Limits{MaxMemoryBytes: 1 << 20}), archive.NewExclusions(nil))
require.NoError(t, err)
t.Cleanup(whl.Cleanup)
found, err := whl.FilesByPath("pip/__init__.py")
require.NoError(t, err)
require.Len(t, found, 1, "a relative path still finds the file")
p := pkg.Package{Name: "pip", Metadata: pkg.PythonPackage{Files: []pkg.PythonFileRecord{{Path: "pip/__init__.py"}, {Path: "/bin/tool"}}}}
p.SetID()
rels, err := packageFileOwnershipRelationships(p, whl)
require.NoError(t, err)
var owned []string
for _, r := range rels {
owned = append(owned, r.To.(file.Coordinates).FullNestedPath())
}
assert.Equal(t, []string{"/pip.whl!/bin/tool"}, owned)
}
+4 -3
View File
@@ -40,10 +40,11 @@ func packagesToRemove(accessor sbomsync.Accessor) []artifact.ID {
}
case "":
if isPrimaryEvidence {
if exists := filterDuplicates[getKey(p, l)]; exists {
k := getKey(p, l)
if exists := filterDuplicates[k]; exists {
break
}
filterDuplicates[getKey(p, l)] = true
filterDuplicates[k] = true
noPrimary = false
break
}
@@ -59,5 +60,5 @@ func packagesToRemove(accessor sbomsync.Accessor) []artifact.ID {
}
func getKey(pkg pkg.Package, loc file.Location) string {
return fmt.Sprintf("%s-%s-%s-%s", pkg.Name, pkg.Version, loc.RealPath, loc.AccessPath)
return fmt.Sprintf("%s-%s-%s-%s", pkg.Name, pkg.Version, loc.FullNestedPath(), loc.AccessPath)
}
+37 -3
View File
@@ -2,8 +2,11 @@ package task
import (
"context"
"path"
"slices"
"strings"
"github.com/anchore/syft/internal/archive"
intFile "github.com/anchore/syft/internal/file"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/sbomsync"
@@ -36,11 +39,20 @@ func (c unknownsLabelerTask) finalize(resolver file.Resolver, s *sbom.SBOM) {
hasPackageReference := coordinateReferenceLookup(resolver, s)
if c.RemoveWhenPackagesDefined {
for coords := range s.Artifacts.Unknowns {
for coords, reasons := range s.Artifacts.Unknowns {
if !hasPackageReference(coords) {
continue
}
delete(s.Artifacts.Unknowns, coords)
// what the archive cataloger reports is about the archive's contents (left out, cut short),
// which a package describing the archive does not account for
kept := slices.DeleteFunc(reasons, func(r string) bool {
return !strings.HasPrefix(r, formatUnknown("", ArchiveCatalogerTaskName))
})
if len(kept) == 0 {
delete(s.Artifacts.Unknowns, coords)
continue
}
s.Artifacts.Unknowns[coords] = kept
}
}
@@ -59,7 +71,7 @@ func (c unknownsLabelerTask) finalize(resolver file.Resolver, s *sbom.SBOM) {
if c.IncludeUnexpandedArchives {
ctx := context.Background()
for coords := range s.Artifacts.FileMetadata {
format, _, notArchiveErr := intFile.IdentifyArchive(ctx, coords.RealPath, nil)
format, _, notArchiveErr := intFile.IdentifyArchive(ctx, coords.FullNestedPath(), nil)
if format != nil && notArchiveErr == nil && !hasPackageReference(coords) {
s.Artifacts.Unknowns[coords] = append(s.Artifacts.Unknowns[coords], "archive not cataloged")
}
@@ -71,6 +83,7 @@ func formatUnknown(err string, task ...string) string {
return strings.Join(task, "/") + ": " + err
}
//nolint:gocognit
func coordinateReferenceLookup(resolver file.Resolver, s *sbom.SBOM) func(coords file.Coordinates) bool {
allPackageCoords := file.NewCoordinateSet()
@@ -83,7 +96,28 @@ func coordinateReferenceLookup(resolver file.Resolver, s *sbom.SBOM) func(coords
// relationships for these owned files may be disabled, but we always want to include them
for p := range s.Artifacts.Packages.Enumerate() {
if f, ok := p.Metadata.(pkg.FileOwner); ok {
// owned paths are within the containers the package was found in
var archives []file.Coordinates
atRoot := false
for _, container := range archive.OwnerContainers(p) {
if container.FullPath == "" {
atRoot = true
} else {
archives = append(archives, container)
}
}
for _, ownedFilePath := range f.OwnedFiles() {
// an archive's contents are not in this resolver to follow links in, and only an absolute owned
// path names a file in one (see packageFileOwnershipRelationships)
if path.IsAbs(ownedFilePath) {
for _, container := range archives {
allPackageCoords.Add(archive.OwnedCoordinates(container, ownedFilePath))
}
}
if !atRoot {
continue
}
// resolve these owned files, as they may have symlinks
// but coordinates we will test against are always absolute paths
locations, err := resolver.FilesByPath(ownedFilePath)
+120
View File
@@ -0,0 +1,120 @@
package task
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cataloging"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
)
func Test_unknownsLabeler_archiveReasonsSurvivePackages(t *testing.T) {
jar := file.NewCoordinates("/app.jar", "")
p := pkg.Package{Name: "app", Locations: file.NewLocationSet(file.NewLocationFromCoordinates(jar))}
p.SetID()
s := &sbom.SBOM{Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(p),
Unknowns: map[file.Coordinates][]string{jar: {
formatUnknown("archive cataloged from part of its contents: too big", ArchiveCatalogerTaskName),
formatUnknown("unable to read manifest", "java-archive-cataloger"),
}},
}}
unknownsLabelerTask{cataloging.UnknownsConfig{RemoveWhenPackagesDefined: true}}.finalize(file.NewMockResolverForPaths(), s)
assert.Equal(t, []string{formatUnknown("archive cataloged from part of its contents: too big", ArchiveCatalogerTaskName)}, s.Artifacts.Unknowns[jar],
"a package describing the archive explains a parse failure, not contents left out")
}
func Test_unknownsLabeler_archiveContentsAreReportedByFullPath(t *testing.T) {
// a package describing an archive does not account for the files within it
war := file.NewCoordinates("/app.war", "layer")
in := func(fullPath string) file.Coordinates {
return file.Coordinates{RealPath: "/app.war", FileSystemID: "layer", FullPath: fullPath}
}
nativeLib := in("/app.war!/WEB-INF/lib/jna.jar!/linux-x86-64/libjnidispatch.so")
unidentifiedJar := in("/app.war!/WEB-INF/lib/javax.inject-1.jar")
p := pkg.Package{Name: "app", Locations: file.NewLocationSet(file.NewLocationFromCoordinates(war))}
p.SetID()
noPackage := formatUnknown("no package identified in archive", "java-archive-cataloger")
s := &sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(p),
Executables: map[file.Coordinates]file.Executable{nativeLib: {}},
Unknowns: map[file.Coordinates][]string{unidentifiedJar: {noPackage}},
},
Relationships: []artifact.Relationship{{From: war, To: unidentifiedJar, Type: artifact.ContainsRelationship}},
}
unknownsLabelerTask{cataloging.UnknownsConfig{RemoveWhenPackagesDefined: true, IncludeExecutablesWithoutPackages: true}}.finalize(file.NewMockResolverForPaths(), s)
assert.Equal(t, map[file.Coordinates][]string{
nativeLib: {formatUnknown("no package identified in executable file", unknownsLabelerTaskName)},
unidentifiedJar: {noPackage},
}, s.Artifacts.Unknowns)
}
func Test_unknownsLabeler_relativeOwnedPathsNameNothingInAnArchive(t *testing.T) {
// a python RECORD lists paths relative to site-packages, which resolve nowhere at the scan root either
whl := func(fullPath string) file.Coordinates {
return file.Coordinates{RealPath: "/pip.whl", FullPath: fullPath}
}
tool := whl("/pip.whl!/pip/tool")
p := pkg.Package{
Name: "pip",
Locations: file.NewLocationSet(file.NewLocationFromCoordinates(whl("/pip.whl!/pip.dist-info/METADATA"))),
Metadata: pkg.PythonPackage{Files: []pkg.PythonFileRecord{{Path: "pip/tool"}}},
}
p.SetID()
s := &sbom.SBOM{Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(p),
Executables: map[file.Coordinates]file.Executable{tool: {}},
}}
unknownsLabelerTask{cataloging.UnknownsConfig{IncludeExecutablesWithoutPackages: true}}.finalize(file.NewMockResolverForPaths(), s)
assert.NotEmpty(t, s.Artifacts.Unknowns[tool])
}
func Test_unknownsLabeler_ownedFilesAreWithinTheOwnersContainer(t *testing.T) {
atRoot := file.NewCoordinates("/usr/bin/foo", "")
inArchive := file.Coordinates{RealPath: "/rootfs.tar", FullPath: "/rootfs.tar!/usr/bin/foo"}
owner := func(at file.Coordinates) pkg.Package {
p := pkg.Package{
Name: "owner",
Type: pkg.RpmPkg,
Locations: file.NewLocationSet(file.NewLocationFromCoordinates(at)),
Metadata: pkg.RpmDBEntry{Files: []pkg.RpmFileRecord{{Path: "/usr/bin/foo"}}},
}
p.SetID()
return p
}
tests := []struct {
name string
owner file.Coordinates
referenced file.Coordinates
}{
{name: "an owner at the scan root", owner: file.NewCoordinates("/var/lib/rpm/Packages", ""), referenced: atRoot},
{name: "an owner inside an archive", owner: file.Coordinates{RealPath: "/rootfs.tar", FullPath: "/rootfs.tar!/var/lib/rpm/Packages"}, referenced: inArchive},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := &sbom.SBOM{Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(owner(tt.owner)),
Executables: map[file.Coordinates]file.Executable{atRoot: {}, inArchive: {}},
}}
unknownsLabelerTask{cataloging.UnknownsConfig{IncludeExecutablesWithoutPackages: true}}.finalize(file.NewMockResolverForPaths("/usr/bin/foo"), s)
for _, exe := range []file.Coordinates{atRoot, inArchive} {
assert.Equal(t, exe != tt.referenced, len(s.Artifacts.Unknowns[exe]) > 0, exe.FullNestedPath())
}
})
}
}
+84
View File
@@ -0,0 +1,84 @@
package testutils
import (
"archive/tar"
"archive/zip"
"bytes"
"compress/gzip"
"fmt"
"io"
"sort"
"testing"
"github.com/stretchr/testify/require"
)
// Zip builds a zip of the entries in sorted name order: archive limits bite as entries are read, so
// order decides which land before a truncation.
func Zip[T ~string | ~[]byte](t testing.TB, entries map[string]T) []byte {
t.Helper()
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
for _, name := range sortedNames(entries) {
w, err := zw.Create(name)
require.NoError(t, err)
_, err = w.Write([]byte(entries[name]))
require.NoError(t, err)
}
require.NoError(t, zw.Close())
return buf.Bytes()
}
// Tar builds a tar of the entries in sorted name order.
func Tar[T ~string | ~[]byte](t testing.TB, entries map[string]T) []byte {
t.Helper()
var buf bytes.Buffer
writeTar(t, &buf, entries)
return buf.Bytes()
}
// TarGz builds a gzipped tar of the entries in sorted name order.
func TarGz[T ~string | ~[]byte](t testing.TB, entries map[string]T) []byte {
t.Helper()
var buf bytes.Buffer
gw := gzip.NewWriter(&buf)
writeTar(t, gw, entries)
require.NoError(t, gw.Close())
return buf.Bytes()
}
// Jar is the smallest jar the java cataloger identifies, a manifest naming the artifact, plus extra.
func Jar(t testing.TB, title, version string, extra map[string][]byte) []byte {
t.Helper()
entries := map[string][]byte{"META-INF/MANIFEST.MF": Manifest(title, version)}
for name, body := range extra {
entries[name] = body
}
return Zip(t, entries)
}
// Manifest is a jar manifest naming the artifact.
func Manifest(title, version string) []byte {
return fmt.Appendf(nil, "Manifest-Version: 1.0\nImplementation-Title: %s\nImplementation-Version: %s\n", title, version)
}
func writeTar[T ~string | ~[]byte](t testing.TB, w io.Writer, entries map[string]T) {
t.Helper()
tw := tar.NewWriter(w)
for _, name := range sortedNames(entries) {
body := []byte(entries[name])
require.NoError(t, tw.WriteHeader(&tar.Header{Name: name, Mode: 0o644, Size: int64(len(body)), Typeflag: tar.TypeReg}))
_, err := tw.Write(body)
require.NoError(t, err)
}
require.NoError(t, tw.Close())
}
func sortedNames[T any](entries map[string]T) []string {
names := make([]string, 0, len(entries))
for name := range entries {
names = append(names, name)
}
sort.Strings(names)
return names
}
+30
View File
@@ -0,0 +1,30 @@
package testutils
import (
"os/exec"
"runtime"
"strings"
"sync"
"testing"
)
var dockerServerOS = sync.OnceValue(func() string {
out, err := exec.Command("docker", "version", "--format", "{{.Server.Os}}").Output()
if err != nil {
return ""
}
return strings.TrimSpace(string(out))
})
// SkipWithoutLinuxContainers skips tests whose fixtures are linux container images (built or run via docker).
// This only kicks in on windows, where the docker daemon is often running windows containers (or is absent),
// so the linux fixture images can't be built. Everywhere else the fixture tooling is left to fail loudly.
func SkipWithoutLinuxContainers(t testing.TB) {
t.Helper()
if runtime.GOOS != "windows" {
return
}
if os := dockerServerOS(); os != "linux" {
t.Skipf("linux container fixtures unavailable (docker server os=%q)", os)
}
}
+2 -1
View File
@@ -265,8 +265,9 @@ func TestLazyCreation(t *testing.T) {
// root dir is not created until needed
assert.Equal(t, "", td.root)
_, _, err := td.NewFile("trigger-*")
f, _, err := td.NewFile("trigger-*")
require.NoError(t, err)
require.NoError(t, f.Close())
assert.NotEmpty(t, td.root)
+2 -2
View File
@@ -23,9 +23,9 @@ var _ error = (*CoordinateError)(nil)
func (u *CoordinateError) Error() string {
if u.Coordinates.FileSystemID == "" {
return fmt.Sprintf("%s: %v", u.Coordinates.RealPath, u.Reason)
return fmt.Sprintf("%s: %v", u.Coordinates.FullNestedPath(), u.Reason)
}
return fmt.Sprintf("%s (%s): %v", u.Coordinates.RealPath, u.Coordinates.FileSystemID, u.Reason)
return fmt.Sprintf("%s (%s): %v", u.Coordinates.FullNestedPath(), u.Coordinates.FileSystemID, u.Reason)
}
// New returns a new CoordinateError unless the reason is a CoordinateError itself, in which case
+1 -3
View File
@@ -169,9 +169,7 @@ func Test_Join(t *testing.T) {
}
func Test_flatten(t *testing.T) {
coords := file.Coordinates{
RealPath: "real/path",
}
coords := file.NewCoordinates("real/path", "")
e1 := fmt.Errorf("e1")
e2 := fmt.Errorf("e2")
c1 := New(coords, fmt.Errorf("c1"))
+2
View File
@@ -19,6 +19,8 @@ func ProcessPathErrors(err error) error {
if pathErrorRegex.MatchString(errText) {
foundPath := pathErrorRegex.ReplaceAllString(err.Error(), "$1")
if foundPath != "" {
// a path in an error text is taken as is: nesting is never inferred from "!/" in it, since a real
// path can hold one too. A caller with the location in hand should use New instead
return New(file.NewLocation(foundPath), err)
}
}
+13 -12
View File
@@ -17,28 +17,29 @@ func Test_ProcessPathErrors(t *testing.T) {
{
errorText: `prefix path="/var/lib/thing" suffix`,
expected: &CoordinateError{
Coordinates: file.Coordinates{
RealPath: "/var/lib/thing",
},
Reason: fmt.Errorf(`prefix path="/var/lib/thing" suffix`),
Coordinates: file.NewCoordinates("/var/lib/thing", ""),
Reason: fmt.Errorf(`prefix path="/var/lib/thing" suffix`),
},
},
{
errorText: `prefix path="/var/lib/thing"`,
expected: &CoordinateError{
Coordinates: file.Coordinates{
RealPath: "/var/lib/thing",
},
Reason: fmt.Errorf(`prefix path="/var/lib/thing"`),
Coordinates: file.NewCoordinates("/var/lib/thing", ""),
Reason: fmt.Errorf(`prefix path="/var/lib/thing"`),
},
},
{
errorText: `path="/var/lib/thing" suffix`,
expected: &CoordinateError{
Coordinates: file.Coordinates{
RealPath: "/var/lib/thing",
},
Reason: fmt.Errorf(`path="/var/lib/thing" suffix`),
Coordinates: file.NewCoordinates("/var/lib/thing", ""),
Reason: fmt.Errorf(`path="/var/lib/thing" suffix`),
},
},
{
errorText: `path="/data/Hello!/world.txt"`,
expected: &CoordinateError{
Coordinates: file.NewCoordinates("/data/Hello!/world.txt", ""),
Reason: fmt.Errorf(`path="/data/Hello!/world.txt"`),
},
},
{
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+39 -6
View File
@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "anchore.io/schema/syft/json/16.1.11/document",
"$id": "anchore.io/schema/syft/json/16.1.13/document",
"$ref": "#/$defs/Document",
"$defs": {
"AlpmDbEntry": {
@@ -716,11 +716,15 @@
"properties": {
"path": {
"type": "string",
"description": "RealPath is the canonical absolute form of the path accessed (all symbolic links have been followed and relative path components like '.' and '..' have been removed)."
"description": "RealPath is the canonical absolute form of the path accessed (all symbolic links have been followed and relative path components like '.' and '..' have been removed). For a file found within an archive, it is the real path of the outermost archive on the scanned filesystem."
},
"layerID": {
"type": "string",
"description": "FileSystemID is an ID representing and entire filesystem. For container images, this is a layer digest. For directories or a root filesystem, this is blank."
"description": "FileSystemID is an ID representing an entire filesystem. For container images, this is a layer digest. For directories or a root filesystem, this is blank. A file inside an archive carries the FileSystemID of the filesystem the archive was found in."
},
"fullPath": {
"type": "string",
"description": "FullPath is the full path to the file: RealPath, then for a file within an archive a \"!/\"-delimited chain of entry paths through any nested archives, ending with the file's own entry path (e.g. \"/app.war!/WEB-INF/lib/dep.jar!/META-INF/MANIFEST.MF\"). Each segment is a real path (links resolved) relative to the archive before it. For a file not within an archive it is blank, so coordinates outside archives are unchanged by the field (in JSON, equality and ID)."
}
},
"type": "object",
@@ -1464,6 +1468,13 @@
"elfSecurityFeatures": {
"$ref": "#/$defs/ELFSecurityFeatures",
"description": "ELFSecurityFeatures contains ELF-specific security hardening information when Format is ELF."
},
"identifiers": {
"items": {
"$ref": "#/$defs/ExecutableIdentifier"
},
"type": "array",
"description": "Identifiers lists self-declared build identifiers found in the binary (e.g. ELF GNU build IDs or Go build IDs)."
}
},
"type": "object",
@@ -1475,6 +1486,24 @@
],
"description": "Executable contains metadata about binary files and their security features."
},
"ExecutableIdentifier": {
"properties": {
"kind": {
"type": "string",
"description": "Kind identifies the type and source of the identifier."
},
"value": {
"type": "string",
"description": "Value is the identifier in the conventional text form for its Kind (e.g. lowercase hex for an ELF GNU build ID)."
}
},
"type": "object",
"required": [
"kind",
"value"
],
"description": "ExecutableIdentifier is a self-declared build identifier used to correlate a binary with its build or debug information."
},
"File": {
"properties": {
"id": {
@@ -2544,15 +2573,19 @@
"properties": {
"path": {
"type": "string",
"description": "RealPath is the canonical absolute form of the path accessed (all symbolic links have been followed and relative path components like '.' and '..' have been removed)."
"description": "RealPath is the canonical absolute form of the path accessed (all symbolic links have been followed and relative path components like '.' and '..' have been removed). For a file found within an archive, it is the real path of the outermost archive on the scanned filesystem."
},
"layerID": {
"type": "string",
"description": "FileSystemID is an ID representing and entire filesystem. For container images, this is a layer digest. For directories or a root filesystem, this is blank."
"description": "FileSystemID is an ID representing an entire filesystem. For container images, this is a layer digest. For directories or a root filesystem, this is blank. A file inside an archive carries the FileSystemID of the filesystem the archive was found in."
},
"fullPath": {
"type": "string",
"description": "FullPath is the full path to the file: RealPath, then for a file within an archive a \"!/\"-delimited chain of entry paths through any nested archives, ending with the file's own entry path (e.g. \"/app.war!/WEB-INF/lib/dep.jar!/META-INF/MANIFEST.MF\"). Each segment is a real path (links resolved) relative to the archive before it. For a file not within an archive it is blank, so coordinates outside archives are unchanged by the field (in JSON, equality and ID)."
},
"accessPath": {
"type": "string",
"description": "AccessPath is the path used to retrieve file contents (which may or may not have hardlinks / symlinks in the path)"
"description": "AccessPath is the path used to retrieve file contents (which may or may not have hardlinks / symlinks in the path). For a file within an archive, it is a full path (see Coordinates.FullPath)."
},
"annotations": {
"additionalProperties": {
+678
View File
@@ -0,0 +1,678 @@
package syft
import (
"archive/tar"
"bytes"
"compress/gzip"
"context"
"os"
"path/filepath"
"sort"
"sync"
"testing"
"unicode/utf8"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
"github.com/anchore/syft/internal/archive"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/task"
"github.com/anchore/syft/internal/testutils"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cataloging"
"github.com/anchore/syft/syft/cataloging/filecataloging"
"github.com/anchore/syft/syft/cataloging/pkgcataloging"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/event/monitor"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/java"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/syft/source/directorysource"
)
// Every CreateSBOM call pays to compile the license scanner, so the end-to-end behavior is asserted
// over one scan of one fixture directory rather than a scan per claim. The fixture holds every shape
// the claims need; the subtests read the one SBOM.
//
// scan root
//
// loose.txt an ordinary file, counted by the file catalogers
// nested/marker.txt excluded by the root-anchored pattern ./nested/marker.txt
// app.zip nested/marker.txt, nested/keep.txt, nested/excluded.txt (**/excluded.txt)
// app/data.zip nested/marker.txt, under a directory
// outer.zip outer/marker.txt
// deeper/inner.zip nested/marker.txt, two levels down
// skip/inner-excluded.zip nested/marker.txt, excluded by **/inner-excluded.zip
// 日本語/café-🎉/unicode.zip nested/marker.txt, under a non-ASCII path
// pkg.rpm a zip holding a marker, excluded by **/*.rpm
// KEPT.RPM the same zip, kept: the pattern is lower-case (a distinct base
// name, so the two coexist on a case-insensitive filesystem)
// app.war WEB-INF/lib/war-lib-2.0.jar
// bundle.zip lib/zip-lib-1.2.jar
// bundle.tar.gz lib/tgz-lib-1.2.jar
var fixtureExclusions = []string{"./nested/marker.txt", "**/excluded.txt", "**/inner-excluded.zip", "**/*.rpm"}
func writeFixture(t *testing.T) string {
t.Helper()
dir := t.TempDir()
write := func(name string, data []byte) {
require.NoError(t, os.MkdirAll(filepath.Dir(filepath.Join(dir, name)), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(dir, name), data, 0o644))
}
markerZip := testutils.Zip(t, map[string]string{"nested/marker.txt": "hello"})
write("loose.txt", []byte("outside"))
write("nested/marker.txt", []byte("at the root"))
write("app.zip", testutils.Zip(t, map[string]string{
"nested/marker.txt": "hello",
"nested/keep.txt": "hello",
"nested/excluded.txt": "hello",
}))
write("app/data.zip", markerZip)
write("outer.zip", testutils.Zip(t, map[string][]byte{
"outer/marker.txt": []byte("hello"),
"deeper/inner.zip": markerZip,
"skip/inner-excluded.zip": markerZip,
"日本語/café-🎉/unicode.zip": markerZip,
}))
write("pkg.rpm", markerZip)
write("KEPT.RPM", markerZip)
write("app.war", testutils.Zip(t, map[string][]byte{
"WEB-INF/lib/war-lib-2.0.jar": testutils.Jar(t, "war-lib", "2.0", nil),
}))
write("bundle.zip", testutils.Zip(t, map[string][]byte{
"lib/zip-lib-1.2.jar": testutils.Jar(t, "zip-lib", "1.2", nil),
}))
write("bundle.tar.gz", testutils.TarGz(t, map[string][]byte{
"lib/tgz-lib-1.2.jar": testutils.Jar(t, "tgz-lib", "1.2", nil),
}))
return dir
}
func TestArchiveCataloger(t *testing.T) {
scanDir := writeFixture(t)
resolvedScanDir, err := filepath.EvalSymlinks(scanDir)
require.NoError(t, err)
resolvedScanDir = filepath.ToSlash(resolvedScanDir)
recorder := &recordingBus{}
bus.Set(recorder)
t.Cleanup(func() { bus.Set(nil) })
var (
probeMu sync.Mutex
seen []string // every .txt and .zip path any resolver handed a cataloger could see
)
probe := resolverProbeCataloger{globs: []string{"**/*.txt", "**/*.zip"}, record: func(paths []string) {
probeMu.Lock()
defer probeMu.Unlock()
seen = append(seen, paths...)
}}
// the java catalogers come from the factory, as the CLI builds them: the flag handing recursion
// to the archive task is derived onto the factory's config. Both legacy search settings are off,
// the cell of the matrix that looks like a bug: with the feature on they no longer gate anything
packages := pkgcataloging.DefaultConfig()
packages.JavaArchive.IncludeIndexedArchives = false
packages.JavaArchive.IncludeUnindexedArchives = false
cfg := DefaultCreateSBOMConfig().
WithCatalogerSelection(cataloging.NewSelectionRequest().WithDefaults("java")).
WithCatalogers(
pkgcataloging.NewAlwaysEnabledCatalogerReference(markerCataloger{}),
pkgcataloging.NewAlwaysEnabledCatalogerReference(probe),
).
WithPackagesConfig(packages).
WithFilesConfig(filecataloging.DefaultConfig().WithSelection(file.AllFilesSelection)).
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(3))
s := scanDirWithExclusions(t, scanDir, cfg, fixtureExclusions...)
t.Run("markers are found at every depth under archive-relative paths", func(t *testing.T) {
// the location's RealPath is the outermost archive; FullPath adds the chain of entries within it
assert.Equal(t, []string{
"KEPT.RPM!/nested/marker.txt",
"app.zip!/nested/marker.txt",
"app/data.zip!/nested/marker.txt",
"outer.zip!/deeper/inner.zip!/nested/marker.txt",
"outer.zip!/outer/marker.txt",
"outer.zip!/日本語/café-🎉/unicode.zip!/nested/marker.txt",
}, packageLocations(s, "marker-pkg"))
})
t.Run("archive contents do not leak into the parent tree", func(t *testing.T) {
for _, loc := range packageLocations(s, "marker-pkg") {
assert.NotContains(t, loc, ".zip/", "a joined path smears the archive's contents into the parent's tree")
assert.NotContains(t, loc, "!//", "a nested path must not be rooted")
}
})
t.Run("unicode entry names survive to output", func(t *testing.T) {
for _, loc := range packageLocations(s, "marker-pkg") {
assert.True(t, utf8.ValidString(loc))
}
})
t.Run("a CONTAINS relationship links each archive to what was found inside", func(t *testing.T) {
var containers []string
for _, p := range s.Artifacts.Packages.Sorted() {
if p.Name != "marker-pkg" {
continue
}
for _, rel := range s.Relationships {
if rel.Type != artifact.ContainsRelationship || rel.To.ID() != p.ID() {
continue
}
if coords, ok := rel.From.(file.Coordinates); ok {
containers = append(containers, coords.FullNestedPath())
}
}
}
sort.Strings(containers)
assert.Equal(t, []string{
"KEPT.RPM",
"app.zip",
"app/data.zip",
"outer.zip",
"outer.zip!/deeper/inner.zip",
"outer.zip!/日本語/café-🎉/unicode.zip",
}, containers, "the archive file, named in its own parent, contains the package")
})
t.Run("each file within an archive is contained by that archive", func(t *testing.T) {
containers := fileContainers(s)
assert.Equal(t, "app.zip", containers["app.zip!/nested/keep.txt"])
assert.Equal(t, "outer.zip!/deeper/inner.zip", containers["outer.zip!/deeper/inner.zip!/nested/marker.txt"])
assert.Equal(t, "outer.zip", containers["outer.zip!/deeper/inner.zip"])
})
t.Run("exclusions are the scan's own", func(t *testing.T) {
// there is no archive-specific exclusion setting: a scan pattern's shape decides whether it
// reaches inside an archive, and an excluded entry is absent from the archive's filesystem
// rather than present and skipped
markers := packageLocations(s, "marker-pkg")
assert.NotContains(t, markers, "nested/marker.txt", "a root-anchored pattern excludes the file at the scan root")
assert.Contains(t, markers, "app.zip!/nested/marker.txt", "and says nothing about the same path inside an archive")
assert.NotContains(t, markers, "pkg.rpm!/nested/marker.txt", "an archive whose own path is excluded is not extracted")
assert.Contains(t, markers, "KEPT.RPM!/nested/marker.txt", "a lower-case pattern does not match an upper-case name")
assert.NotContains(t, markers, "outer.zip!/skip/inner-excluded.zip!/nested/marker.txt", "a nested archive is excluded at its own level")
assert.Contains(t, seen, "app.zip!/nested/keep.txt")
assert.NotContains(t, seen, "app.zip!/nested/excluded.txt", "a **/ pattern excludes inside an archive, from the index itself")
assert.Contains(t, seen, "outer.zip!/deeper/inner.zip", "the probe ran inside outer.zip")
assert.NotContains(t, seen, "outer.zip!/skip/inner-excluded.zip", "the excluded inner archive is absent from outer's filesystem")
})
t.Run("a jar inside an archive is cataloged exactly once, under the virtual path java gives it", func(t *testing.T) {
// the failure mode is a duplicate: a jar inside a zip can be reached by the task's walk and by
// the java cataloger's own unarchiving. The colon-delimited virtual path appears in published
// SBOMs, so it must not change with the mechanism; for a directory source the archive's path
// is relative, so the chain has no leading separator
assert.Equal(t, map[string][]string{
"war-lib": {"app.war:WEB-INF/lib/war-lib-2.0.jar"},
"zip-lib": {"bundle.zip:lib/zip-lib-1.2.jar"},
"tgz-lib": {"bundle.tar.gz:lib/tgz-lib-1.2.jar"},
}, javaVirtualPaths(s), "with both legacy search settings off, every container is still descended")
})
t.Run("no extraction path reaches the SBOM", func(t *testing.T) {
// covers syft/archive-content-identity#nested-paths-are-archive-relative: file cataloging is
// the one path that enumerates every location rather than searching for them, so a resolver
// root that leaked would land in FileMetadata and gain a CONTAINS edge
var paths []string
for coords := range s.Artifacts.FileMetadata {
paths = append(paths, coords.RealPath)
}
for _, p := range s.Artifacts.Packages.Sorted() {
for _, loc := range p.Locations.ToSlice() {
paths = append(paths, loc.RealPath, loc.AccessPath)
}
}
for _, rel := range s.Relationships {
for _, end := range []artifact.Identifiable{rel.From, rel.To} {
if coords, ok := end.(file.Coordinates); ok {
paths = append(paths, coords.RealPath)
}
}
}
require.NotEmpty(t, paths)
for _, p := range paths {
assert.NotContains(t, p, "archive-spill-", "the spill file reached the SBOM at %q", p)
if filepath.IsAbs(p) {
// the directory resolver has always reported the scan root absolute: it is the path the
// user asked for. Anything else absolute is a filesystem this feature invented
assert.Equal(t, resolvedScanDir, p, "only the scanned directory itself may be reported absolute")
}
}
})
t.Run("file metadata covers every entry of every archive, and nothing else", func(t *testing.T) {
// an exact set is also the reproducibility check: anything of a per-run temp directory reaching
// output would fail it
var got []string
for coords := range s.Artifacts.FileMetadata {
if archive.ContainerPath(coords) == "" {
continue
}
got = append(got, coords.FullNestedPath())
}
sort.Strings(got)
assert.Equal(t, []string{
"KEPT.RPM!/nested/marker.txt",
"app.war!/WEB-INF/lib/war-lib-2.0.jar",
"app.war!/WEB-INF/lib/war-lib-2.0.jar!/META-INF/MANIFEST.MF",
"app.zip!/nested/keep.txt",
"app.zip!/nested/marker.txt",
"app/data.zip!/nested/marker.txt",
"bundle.tar.gz!/lib/tgz-lib-1.2.jar",
"bundle.tar.gz!/lib/tgz-lib-1.2.jar!/META-INF/MANIFEST.MF",
"bundle.zip!/lib/zip-lib-1.2.jar",
"bundle.zip!/lib/zip-lib-1.2.jar!/META-INF/MANIFEST.MF",
"outer.zip!/deeper/inner.zip",
"outer.zip!/deeper/inner.zip!/nested/marker.txt",
"outer.zip!/outer/marker.txt",
"outer.zip!/日本語/café-🎉/unicode.zip",
"outer.zip!/日本語/café-🎉/unicode.zip!/nested/marker.txt",
}, got)
})
t.Run("file catalogers publish one progress row counting archive contents", func(t *testing.T) {
// the file catalogers run once over the scan root and again over every archive; each must
// report into one row carrying the total of every run, or the count runs past the total and
// renders as an overshot bar that reads as complete mid-walk
rows := map[string][]*progress.Manual{}
for _, e := range recorder.events {
if e.Type != event.CatalogerTaskStarted {
continue
}
info, ok := e.Source.(monitor.GenericTask)
require.True(t, ok)
tp, ok := e.Value.(*monitor.TaskProgress)
require.True(t, ok)
rows[info.Title.Default] = append(rows[info.Title.Default], tp.Manual)
}
for _, title := range []string{"File digests", "File metadata"} {
published := rows[title]
require.Len(t, published, 1, "%s must publish exactly one row for the whole scan", title)
row := published[0]
assert.Greater(t, row.Current(), int64(len(s.Artifacts.FileMetadata))/2,
"%s must count the files inside the archives, got %d", title, row.Current())
if row.Size() >= 0 {
assert.Equal(t, row.Current(), row.Size(), "%s total must cover every run that reported into it", title)
}
}
})
}
// TestArchiveCataloger_featureOff is the one comparison that needs a second scan: with the task
// disabled, java's own unarchiving reaches the jars its legacy settings admit, under the same virtual
// path the task gives them.
func TestArchiveCataloger_featureOff(t *testing.T) {
scanDir := writeFixture(t)
cfg := DefaultCreateSBOMConfig().
WithCatalogerSelection(cataloging.NewSelectionRequest().WithDefaults("java"))
s := scanDirWithExclusions(t, scanDir, cfg, fixtureExclusions...)
// indexed archives (zip, war) are searched by default and unindexed ones (tar.gz) are not, so
// what a user gets without the feature depends on settings the feature makes irrelevant
assert.Equal(t, map[string][]string{
"war-lib": {"app.war:WEB-INF/lib/war-lib-2.0.jar"},
"zip-lib": {"bundle.zip:lib/zip-lib-1.2.jar"},
}, javaVirtualPaths(s), "java's own recursion finds each once, under the virtual path the task also reports")
t.Run("the top-level archive search booleans reach java", func(t *testing.T) {
cfg := DefaultCreateSBOMConfig().
WithCatalogerSelection(cataloging.NewSelectionRequest().WithDefaults("java")).
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithIncludeUnindexedArchives(true))
s := scanDirWithExclusions(t, scanDir, cfg, fixtureExclusions...)
assert.Contains(t, javaVirtualPaths(s), "tgz-lib")
assert.False(t, cfg.Packages.JavaArchive.IncludeUnindexedArchives, "the caller's config is not written to")
})
}
func TestArchiveCataloger_jarReachedThroughALinkIsIdentified(t *testing.T) {
// a content-addressed layout (pnpm, nix, bazel): the jar is stored under a hash and only the link
// to it is named like a jar
var buf bytes.Buffer
gw := gzip.NewWriter(&buf)
tw := tar.NewWriter(gw)
jar := testutils.Jar(t, "json-simple", "1.1.1", nil)
require.NoError(t, tw.WriteHeader(&tar.Header{Name: "store/3f9a1c", Mode: 0o644, Size: int64(len(jar))}))
_, err := tw.Write(jar)
require.NoError(t, err)
require.NoError(t, tw.WriteHeader(&tar.Header{Name: "app/lib/json-simple-1.1.1.jar", Typeflag: tar.TypeSymlink, Linkname: "../../store/3f9a1c"}))
require.NoError(t, tw.Close())
require.NoError(t, gw.Close())
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "bundle.tar.gz"), buf.Bytes(), 0o644))
cfg := DefaultCreateSBOMConfig().
WithCatalogerSelection(cataloging.NewSelectionRequest().WithDefaults("java")).
WithFilesConfig(filecataloging.DefaultConfig().WithSelection(file.AllFilesSelection)).
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(2))
s := scanDirWithExclusions(t, dir, cfg)
assert.Equal(t, []string{"bundle.tar.gz!/store/3f9a1c"}, packageLocations(s, "json-simple"))
// the jar's own files are addressed by its real path, the same one its package location names
manifest := file.Coordinates{RealPath: "bundle.tar.gz", FullPath: "bundle.tar.gz!/store/3f9a1c!/META-INF/MANIFEST.MF"}
assert.Contains(t, s.Artifacts.FileMetadata, manifest)
}
func TestArchiveCataloger_defaultFileSelectionCoversRootJars(t *testing.T) {
// with the default owned-by-package selection, a jar at the root is described by the archive task,
// so the root file catalogers must run after it or the jar gets no file entries at all
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "app.jar"), testutils.Jar(t, "app", "1.0", nil), 0o644))
for name, depth := range map[string]int{"feature off": 0, "feature on": 1} {
t.Run(name, func(t *testing.T) {
cfg := DefaultCreateSBOMConfig().
WithCatalogerSelection(cataloging.NewSelectionRequest().WithDefaults("java")).
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(depth))
s := scanDirWithExclusions(t, dir, cfg)
require.Equal(t, []string{"app.jar"}, packageLocations(s, "app"))
jar := file.NewCoordinates("app.jar", "")
assert.Contains(t, s.Artifacts.FileMetadata, jar)
assert.Contains(t, s.Artifacts.FileDigests, jar)
})
}
}
func TestArchiveCataloger_defaultFileSelectionCoversNestedJars(t *testing.T) {
// a nested jar's package is found inside the jar but located in the archive holding it, so that
// archive's file catalogers must run after it, as the root's do
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "outer.zip"), testutils.Zip(t, map[string][]byte{
"lib/inner.jar": testutils.Jar(t, "inner", "1.0", nil),
"readme.txt": []byte("hello"),
"empty.zip": testutils.Zip(t, map[string]string{"a.txt": "hello"}),
}), 0o644))
cfg := DefaultCreateSBOMConfig().
WithCatalogerSelection(cataloging.NewSelectionRequest().WithDefaults("java")).
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(2))
s := scanDirWithExclusions(t, dir, cfg)
require.Equal(t, []string{"outer.zip!/lib/inner.jar"}, packageLocations(s, "inner"))
jar := file.Coordinates{RealPath: "outer.zip", FullPath: "outer.zip!/lib/inner.jar"}
assert.Contains(t, s.Artifacts.FileMetadata, jar)
assert.Contains(t, s.Artifacts.FileDigests, jar)
assert.Equal(t, "outer.zip", fileContainers(s)["outer.zip!/lib/inner.jar"])
// as at the root, a file no package owns is not described, nor is an archive in which nothing was found
var described []string
for _, c := range s.AllCoordinates() {
described = append(described, c.FullNestedPath())
}
assert.NotContains(t, described, "outer.zip!/readme.txt")
assert.NotContains(t, described, "outer.zip!/empty.zip")
assert.NotContains(t, described, "outer.zip!/empty.zip!/a.txt")
}
func TestArchiveCataloger_archiveUnknowns(t *testing.T) {
dir := t.TempDir()
plain := testutils.Zip(t, map[string]string{"readme.txt": "hi"})
require.NoError(t, os.WriteFile(filepath.Join(dir, "outer.zip"), testutils.Zip(t, map[string][]byte{
"lib/inner.jar": testutils.Jar(t, "inner", "1.0", nil),
"lib/plain.zip": plain,
}), 0o644))
require.NoError(t, os.WriteFile(filepath.Join(dir, "plain.zip"), plain, 0o644))
scan := func(depth int) map[string][]string {
cfg := DefaultCreateSBOMConfig().
WithCatalogerSelection(cataloging.NewSelectionRequest().WithDefaults("java")).
WithUnknownsConfig(cataloging.DefaultUnknownsConfig()).
WithFilesConfig(filecataloging.DefaultConfig().WithSelection(file.AllFilesSelection)).
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(depth))
out := map[string][]string{}
for c, reasons := range scanDirWithExclusions(t, dir, cfg).Artifacts.Unknowns {
out[c.FullNestedPath()] = reasons
}
return out
}
t.Run("an expanded archive is not reported as not cataloged", func(t *testing.T) {
assert.Empty(t, scan(3))
})
t.Run("an archive past the depth limit is reported once, by the archive cataloger", func(t *testing.T) {
unknowns := scan(1)
assert.NotContains(t, unknowns, "outer.zip")
for _, nested := range []string{"outer.zip!/lib/inner.jar", "outer.zip!/lib/plain.zip"} {
require.Len(t, unknowns[nested], 1, nested)
assert.Contains(t, unknowns[nested][0], "depth limit was reached")
}
})
t.Run("an archive is reported as not cataloged when nesting is off", func(t *testing.T) {
assert.Equal(t, []string{"archive not cataloged"}, scan(0)["plain.zip"])
})
}
func TestArchiveCataloger_unsupportedArchiveFormatsAreReported(t *testing.T) {
dir := t.TempDir()
var page bytes.Buffer
gw := gzip.NewWriter(&page)
_, err := gw.Write([]byte("a man page"))
require.NoError(t, err)
require.NoError(t, gw.Close())
require.NoError(t, os.WriteFile(filepath.Join(dir, "outer.zip"), testutils.Zip(t, map[string][]byte{
"legacy.7z": []byte("7z\xbc\xaf\x27\x1c\x00\x04"),
"page.1.gz": page.Bytes(),
}), 0o644))
cfg := DefaultCreateSBOMConfig().
WithCatalogerSelection(cataloging.NewSelectionRequest().WithDefaults("java")).
WithUnknownsConfig(cataloging.DefaultUnknownsConfig()).
WithFilesConfig(filecataloging.DefaultConfig().WithSelection(file.AllFilesSelection)).
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(3))
unknowns := map[string][]string{}
for c, reasons := range scanDirWithExclusions(t, dir, cfg).Artifacts.Unknowns {
unknowns[c.FullNestedPath()] = reasons
}
assert.Equal(t, map[string][]string{
"outer.zip!/legacy.7z": {"archive-cataloger: archive not cataloged: archive format is not supported: application/x-7z-compressed"},
}, unknowns, "a bare compressed file holds no files, so it is not an archive left uncataloged")
}
func TestArchiveCataloger_javaParentLinkageMatchesTheLegacyPath(t *testing.T) {
// a consumer walking java DEPENDENCY_OF edges (the only structure SPDX and CycloneDX keep) must see
// the same war-to-jar linkage whether or not the archive task does the recursion
dir := t.TempDir()
war := testutils.Jar(t, "app", "1.0", map[string][]byte{
"WEB-INF/lib/lib-2.0.jar": testutils.Jar(t, "lib", "2.0", nil),
})
require.NoError(t, os.WriteFile(filepath.Join(dir, "app.war"), war, 0o644))
for name, depth := range map[string]int{"feature off": 0, "feature on": 2} {
t.Run(name, func(t *testing.T) {
cfg := DefaultCreateSBOMConfig().
WithCatalogerSelection(cataloging.NewSelectionRequest().WithDefaults("java")).
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(depth))
s := scanDirWithExclusions(t, dir, cfg)
var edges []string
for _, rel := range s.Relationships {
from, fromOK := rel.From.(pkg.Package)
to, toOK := rel.To.(pkg.Package)
if fromOK && toOK && rel.Type == artifact.DependencyOfRelationship {
edges = append(edges, from.Name+" -> "+to.Name)
}
}
assert.Equal(t, []string{"lib -> app"}, edges)
var parent string
for _, p := range s.Artifacts.Packages.Sorted() {
if m, ok := p.Metadata.(pkg.JavaArchive); ok && p.Name == "lib" && m.Parent != nil {
parent = m.Parent.Name
}
}
assert.Equal(t, "app", parent)
})
}
}
func TestArchiveCataloger_descriptorRecordsArchiveConfig(t *testing.T) {
// a consumer must be able to tell from the SBOM alone how deep archive contents were searched
cfg := DefaultCreateSBOMConfig().
WithCatalogerSelection(cataloging.NewSelectionRequest().WithDefaults("java")).
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(3))
s := scanDirWithExclusions(t, t.TempDir(), cfg)
trail, ok := s.Descriptor.Configuration.(configurationAuditTrail)
require.True(t, ok)
assert.Equal(t, 3, trail.Archive.MaxDepth)
assert.Contains(t, trail.Catalogers.Used, task.ArchiveCatalogerTaskName)
}
func TestArchiveCataloger_javaArchiveDepthIsAConfigError(t *testing.T) {
// depth and limits are not java settings; setting them on java's deprecated copy must fail fast
// rather than be silently ignored
for name, set := range map[string]func(*cataloging.ArchiveSearchConfig){
"max depth": func(c *cataloging.ArchiveSearchConfig) { c.MaxDepth = 3 },
"max memory bytes": func(c *cataloging.ArchiveSearchConfig) { c.MaxMemoryBytes = 1 },
"max disk bytes": func(c *cataloging.ArchiveSearchConfig) { c.MaxDiskBytes = 1 },
} {
t.Run(name, func(t *testing.T) {
supplied := java.DefaultArchiveCatalogerConfig()
supplied.ArchiveSearchConfig = cataloging.ArchiveSearchConfig{}
set(&supplied.ArchiveSearchConfig)
cfg := DefaultCreateSBOMConfig().WithPackagesConfig(pkgcataloging.Config{JavaArchive: supplied})
src, err := directorysource.New(directorysource.Config{Path: t.TempDir()})
require.NoError(t, err)
t.Cleanup(func() { _ = src.Close() })
_, err = cfg.Create(context.Background(), src)
require.ErrorContains(t, err, "set them with CreateSBOMConfig.Archive")
})
}
}
// packageLocations lists the full path of every location of every package with the name.
func packageLocations(s *sbom.SBOM, name string) []string {
var out []string
for _, p := range s.Artifacts.Packages.Sorted() {
if p.Name != name {
continue
}
for _, loc := range p.Locations.ToSlice() {
out = append(out, loc.FullNestedPath())
}
}
sort.Strings(out)
return out
}
// javaVirtualPaths maps each java package name to the virtual paths it was reported under; a name
// with two entries was cataloged twice.
func javaVirtualPaths(s *sbom.SBOM) map[string][]string {
out := map[string][]string{}
for _, p := range s.Artifacts.Packages.Sorted() {
if metadata, ok := p.Metadata.(pkg.JavaArchive); ok {
out[p.Name] = append(out[p.Name], metadata.VirtualPath)
}
}
return out
}
// markerCataloger is a stub package cataloger emitting one package per "marker.txt" it finds, so the
// end-to-end tests assert behavior without depending on any real cataloger's fixture format.
type markerCataloger struct{}
func (markerCataloger) Name() string { return "marker-cataloger" }
func (markerCataloger) Catalog(_ context.Context, resolver file.Resolver) ([]pkg.Package, []artifact.Relationship, error) {
locations, err := resolver.FilesByGlob("**/marker.txt")
if err != nil {
return nil, nil, err
}
var pkgs []pkg.Package
for _, loc := range locations {
p := pkg.Package{
Name: "marker-pkg",
Version: "1.0.0",
Type: pkg.BinaryPkg,
Locations: file.NewLocationSet(loc),
}
p.SetID()
pkgs = append(pkgs, p)
}
return pkgs, nil, nil
}
// resolverProbeCataloger reports what the resolver it was handed can see, which is how "absent from
// the index" is told from "present and skipped" within a scan.
type resolverProbeCataloger struct {
globs []string
record func([]string)
}
func (resolverProbeCataloger) Name() string { return "resolver-probe-cataloger" }
func (c resolverProbeCataloger) Catalog(_ context.Context, resolver file.Resolver) ([]pkg.Package, []artifact.Relationship, error) {
locations, err := resolver.FilesByGlob(c.globs...)
if err != nil {
return nil, nil, err
}
var paths []string
for _, loc := range locations {
paths = append(paths, loc.FullNestedPath())
}
c.record(paths)
return nil, nil, nil
}
// recordingBus captures the progress rows a scan publishes.
type recordingBus struct {
mu sync.Mutex
events []partybus.Event
}
func (p *recordingBus) Publish(e partybus.Event) {
p.mu.Lock()
defer p.mu.Unlock()
p.events = append(p.events, e)
}
// fileContainers maps the full path of each file an archive CONTAINS to the archive's full path.
func fileContainers(s *sbom.SBOM) map[string]string {
out := map[string]string{}
for _, r := range s.Relationships {
from, fromFile := r.From.(file.Coordinates)
to, toFile := r.To.(file.Coordinates)
if r.Type == artifact.ContainsRelationship && fromFile && toFile {
out[to.FullNestedPath()] = from.FullNestedPath()
}
}
return out
}
// scanDirWithExclusions scans with the given exclusion patterns configured on the source, where
// exclusions live; archive cataloging has no exclusion setting of its own.
func scanDirWithExclusions(t *testing.T, dir string, cfg *CreateSBOMConfig, exclude ...string) *sbom.SBOM {
t.Helper()
src, err := directorysource.New(directorysource.Config{
Path: dir,
Exclude: source.ExcludeConfig{Paths: exclude},
})
require.NoError(t, err)
t.Cleanup(func() { _ = src.Close() })
s, err := cfg.Create(context.Background(), src)
require.NoError(t, err)
return s
}
+47
View File
@@ -1,17 +1,49 @@
package cataloging
import "github.com/anchore/stereoscope/pkg/file"
const (
// DefaultArchiveMaxDepth is 0: nested archives are not cataloged unless asked for.
DefaultArchiveMaxDepth = 0
// DefaultArchiveMaxMemoryBytes bounds the archive content held in memory at once. It decides only
// whether an archive costs a write: content that does not fit goes to disk and is cataloged anyway.
DefaultArchiveMaxMemoryBytes = 2 * file.GB
// DefaultArchiveMaxDiskBytes bounds the archive content on disk at once. Content that does not fit
// has nowhere to go, so the archive holding it is skipped or truncated. Set high as a backstop
// against a pathological archive filling the disk.
DefaultArchiveMaxDiskBytes = 100 * file.GB
)
type ArchiveSearchConfig struct {
// IncludeIndexedArchives indicates whether to search within indexed archive files (e.g., .zip).
IncludeIndexedArchives bool `yaml:"include-indexed-archives" json:"include-indexed-archives" mapstructure:"include-indexed-archives"`
// IncludeUnindexedArchives indicates whether to search within unindexed archive files (e.g., .tar*).
IncludeUnindexedArchives bool `yaml:"include-unindexed-archives" json:"include-unindexed-archives" mapstructure:"include-unindexed-archives"`
// MaxDepth is how many levels of nested archives to catalog: 0 disables it, and no setting goes deeper than 16 levels
// (negative goes that deep).
MaxDepth int `yaml:"max-depth" json:"max-depth" mapstructure:"max-depth"`
// MaxMemoryBytes bounds the archive content held in memory at once; content that does not fit is
// written to disk. Zero holds nothing in memory, negative is unbounded.
MaxMemoryBytes int64 `yaml:"max-memory-bytes" json:"max-memory-bytes" mapstructure:"max-memory-bytes"`
// MaxDiskBytes bounds the archive content written to disk at once; an archive whose content does
// not fit is skipped or truncated. Zero writes nothing to disk, negative is unbounded. MaxMemoryBytes
// and MaxDiskBytes cannot both be zero while MaxDepth is not.
MaxDiskBytes int64 `yaml:"max-disk-bytes" json:"max-disk-bytes" mapstructure:"max-disk-bytes"`
}
func DefaultArchiveSearchConfig() ArchiveSearchConfig {
return ArchiveSearchConfig{
IncludeIndexedArchives: true,
IncludeUnindexedArchives: false,
MaxDepth: DefaultArchiveMaxDepth,
MaxMemoryBytes: DefaultArchiveMaxMemoryBytes,
MaxDiskBytes: DefaultArchiveMaxDiskBytes,
}
}
@@ -24,3 +56,18 @@ func (c ArchiveSearchConfig) WithIncludeUnindexedArchives(include bool) ArchiveS
c.IncludeUnindexedArchives = include
return c
}
func (c ArchiveSearchConfig) WithMaxDepth(depth int) ArchiveSearchConfig {
c.MaxDepth = depth
return c
}
func (c ArchiveSearchConfig) WithMaxMemoryBytes(size int64) ArchiveSearchConfig {
c.MaxMemoryBytes = size
return c
}
func (c ArchiveSearchConfig) WithMaxDiskBytes(size int64) ArchiveSearchConfig {
c.MaxDiskBytes = size
return c
}
+36
View File
@@ -0,0 +1,36 @@
package cataloging
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestDefaultArchiveSearchConfig(t *testing.T) {
cfg := DefaultArchiveSearchConfig()
assert.True(t, cfg.IncludeIndexedArchives)
assert.False(t, cfg.IncludeUnindexedArchives)
assert.Equal(t, DefaultArchiveMaxDepth, cfg.MaxDepth)
assert.Equal(t, int64(DefaultArchiveMaxMemoryBytes), cfg.MaxMemoryBytes)
assert.Equal(t, int64(DefaultArchiveMaxDiskBytes), cfg.MaxDiskBytes)
}
func TestArchiveSearchConfig_WithMethods(t *testing.T) {
cfg := DefaultArchiveSearchConfig()
cfg = cfg.WithMaxDepth(5)
assert.Equal(t, 5, cfg.MaxDepth)
cfg = cfg.WithMaxMemoryBytes(1024)
assert.Equal(t, int64(1024), cfg.MaxMemoryBytes)
cfg = cfg.WithMaxDiskBytes(2048)
assert.Equal(t, int64(2048), cfg.MaxDiskBytes)
cfg = cfg.WithIncludeIndexedArchives(false)
assert.False(t, cfg.IncludeIndexedArchives)
cfg = cfg.WithIncludeUnindexedArchives(true)
assert.True(t, cfg.IncludeUnindexedArchives)
}
+1
View File
@@ -16,6 +16,7 @@ type configurationAuditTrail struct {
DataGeneration cataloging.DataGenerationConfig `json:"data-generation" yaml:"data-generation" mapstructure:"data-generation"`
Packages pkgcataloging.Config `json:"packages" yaml:"packages" mapstructure:"packages"`
Files filecataloging.Config `json:"files" yaml:"files" mapstructure:"files"`
Archive cataloging.ArchiveSearchConfig `json:"archive" yaml:"archive" mapstructure:"archive"`
Licenses cataloging.LicenseConfig `json:"licenses" yaml:"licenses" mapstructure:"licenses"`
Catalogers catalogerManifest `json:"catalogers" yaml:"catalogers" mapstructure:"catalogers"`
ExtraConfigs any `json:"extra,omitempty" yaml:"extra" mapstructure:"extra"`
+27 -8
View File
@@ -10,6 +10,7 @@ import (
"github.com/scylladb/go-set/strset"
"github.com/anchore/go-sync"
"github.com/anchore/syft/internal/archive"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/licenses"
"github.com/anchore/syft/internal/log"
@@ -38,7 +39,7 @@ func CreateSBOM(ctx context.Context, src source.Source, cfg *CreateSBOMConfig) (
srcMetadata := src.Describe()
taskGroups, audit, err := cfg.makeTaskGroups(srcMetadata)
taskGroups, audit, err := cfg.makeTaskGroups(srcMetadata, sourceExclusions(src))
if err != nil {
return nil, err
}
@@ -57,8 +58,9 @@ func CreateSBOM(ctx context.Context, src source.Source, cfg *CreateSBOMConfig) (
Search: cfg.Search,
Relationships: cfg.Relationships,
DataGeneration: cfg.DataGeneration,
Packages: cfg.Packages,
Packages: cfg.packagesWithArchiveSearch(),
Files: cfg.Files,
Archive: cfg.Archive,
Licenses: cfg.Licenses,
Catalogers: *audit,
ExtraConfigs: cfg.ToolConfiguration,
@@ -88,8 +90,8 @@ func CreateSBOM(ctx context.Context, src source.Source, cfg *CreateSBOMConfig) (
}
}
catalogingProgress := monitorCatalogingTask(src.ID(), taskGroups)
packageCatalogingProgress := monitorPackageCatalogingTask()
catalogingProgress := monitorCatalogingTask(ctx, src.ID(), taskGroups)
packageCatalogingProgress := monitorPackageCatalogingTask(ctx)
builder := sbomsync.NewBuilder(&s, monitorPackageCount(packageCatalogingProgress))
for i := range taskGroups {
@@ -112,6 +114,14 @@ func setupContext(ctx context.Context, cfg *CreateSBOMConfig) (context.Context,
// configure parallel executors
ctx = setContextExecutors(ctx, cfg)
// one progress row per cataloger for the whole scan, however many archives it is run against
ctx = bus.WithCatalogerTaskRegistry(ctx)
// the archive cataloger task recurses into jars too, so the java cataloger must not unarchive them itself
if cfg.Archive.MaxDepth != 0 {
ctx = archive.WithNestedCataloging(ctx)
}
// configure temp dir factory for catalogers (if not already set)
if tmpdir.FromContext(ctx) == nil {
ctx, _ = tmpdir.Root(ctx, "syft-cataloger")
@@ -173,7 +183,7 @@ func monitorPackageCount(prog *monitor.TaskProgress) func(s *sbom.SBOM) {
}
}
func monitorPackageCatalogingTask() *monitor.TaskProgress {
func monitorPackageCatalogingTask(ctx context.Context) *monitor.TaskProgress {
info := monitor.GenericTask{
Title: monitor.Title{
Default: "Packages",
@@ -183,10 +193,10 @@ func monitorPackageCatalogingTask() *monitor.TaskProgress {
ParentID: monitor.TopLevelCatalogingTaskID,
}
return bus.StartCatalogerTask(info, -1, "")
return bus.StartCatalogerTask(ctx, info, -1, "")
}
func monitorCatalogingTask(srcID artifact.ID, tasks [][]task.Task) *monitor.TaskProgress {
func monitorCatalogingTask(ctx context.Context, srcID artifact.ID, tasks [][]task.Task) *monitor.TaskProgress {
info := monitor.GenericTask{
Title: monitor.Title{
Default: "Catalog contents",
@@ -203,7 +213,7 @@ func monitorCatalogingTask(srcID artifact.ID, tasks [][]task.Task) *monitor.Task
length += int64(len(tg))
}
return bus.StartCatalogerTask(info, length, "")
return bus.StartCatalogerTask(ctx, info, length, "")
}
func formatTaskNames(tasks []task.Task) []string {
@@ -218,3 +228,12 @@ func formatTaskNames(tasks []task.Task) []string {
sort.Strings(list)
return list
}
// sourceExclusions returns the exclusion patterns the source was configured with, so the same
// exclusions apply inside the archives found in it.
func sourceExclusions(src source.Source) []string {
if excluder, ok := src.(source.PathExcluder); ok {
return excluder.ExcludedPaths()
}
return nil
}
+67 -7
View File
@@ -28,6 +28,7 @@ type CreateSBOMConfig struct {
Packages pkgcataloging.Config
Licenses cataloging.LicenseConfig
Files filecataloging.Config
Archive cataloging.ArchiveSearchConfig
Parallelism int
CatalogerSelection cataloging.SelectionRequest
@@ -49,6 +50,7 @@ func DefaultCreateSBOMConfig() *CreateSBOMConfig {
Packages: pkgcataloging.DefaultConfig(),
Licenses: cataloging.DefaultLicenseConfig(),
Files: filecataloging.DefaultConfig(),
Archive: cataloging.DefaultArchiveSearchConfig(),
Parallelism: 0, // use default: run in parallel based on number of CPUs
packageTaskFactories: task.DefaultPackageTaskFactories(),
@@ -144,6 +146,13 @@ func (c *CreateSBOMConfig) WithFilesConfig(cfg filecataloging.Config) *CreateSBO
return c
}
// WithArchiveConfig sets how archives are cataloged: how deep to recurse into nested archives (MaxDepth
// 0, the default, does not recurse) and how much extracted content the scan may hold at once.
func (c *CreateSBOMConfig) WithArchiveConfig(cfg cataloging.ArchiveSearchConfig) *CreateSBOMConfig {
c.Archive = cfg
return c
}
// WithoutFiles allows for disabling file cataloging altogether.
func (c *CreateSBOMConfig) WithoutFiles() *CreateSBOMConfig {
c.Files = filecataloging.Config{
@@ -183,14 +192,15 @@ func (c *CreateSBOMConfig) WithCatalogers(catalogerRefs ...pkgcataloging.Catalog
// groups, where each task in a group can be run concurrently, while tasks in different groups must be run serially.
// The final set of task groups is returned along with a cataloger manifest that describes the catalogers that were
// selected and the tokens that were sensitive to this selection (both for adding and removing from the final set).
func (c *CreateSBOMConfig) makeTaskGroups(src source.Description) ([][]task.Task, *catalogerManifest, error) {
//
// exclusions are the source's exclusion patterns, which the archive cataloger applies inside each archive.
func (c *CreateSBOMConfig) makeTaskGroups(src source.Description, exclusions []string) ([][]task.Task, *catalogerManifest, error) {
var taskGroups [][]task.Task
// generate package and file tasks based on the configuration
environmentTasks := c.environmentTasks()
scopeTasks := c.scopeTasks()
relationshipsTasks := c.relationshipTasks(src)
unknownTasks := c.unknownsTasks()
osFeatureDetectionTasks := c.osFeatureDetectionTasks()
pkgTasks, fileTasks, selectionEvidence, err := c.selectTasks(src)
@@ -198,11 +208,29 @@ func (c *CreateSBOMConfig) makeTaskGroups(src source.Description) ([][]task.Task
return nil, nil, err
}
// the same package and file catalogers run against the contents of every archive, after the scan
// root and before relationship and unknowns post-processing
archiveTask := task.NewArchiveCatalogerTask(c.Archive, c.Unknowns, pkgTasks, fileTasks, exclusions)
// the archive cataloger reports the archives it does not expand, which the labeler can only guess at
unknowns := c.Unknowns
if archiveTask != nil {
unknowns.IncludeUnexpandedArchives = false
}
unknownTasks := unknownsTasks(unknowns)
// combine the user-provided and configured tasks
if c.Files.Selection == file.FilesOwnedByPackageSelection {
switch {
case c.Files.Selection == file.FilesOwnedByPackageSelection && archiveTask != nil:
// the archive task describes archives at the root too (java leaves them to it), so the root file
// catalogers must run after it to see those packages
taskGroups = append(taskGroups, pkgTasks, []task.Task{archiveTask}, fileTasks)
case c.Files.Selection == file.FilesOwnedByPackageSelection:
// special case: we need the package info when we are cataloging files owned by packages
taskGroups = append(taskGroups, pkgTasks, fileTasks)
} else {
case archiveTask != nil:
taskGroups = append(taskGroups, append(pkgTasks, fileTasks...), []task.Task{archiveTask})
default:
taskGroups = append(taskGroups, append(pkgTasks, fileTasks...))
}
@@ -226,6 +254,11 @@ func (c *CreateSBOMConfig) makeTaskGroups(src source.Description) ([][]task.Task
taskGroups = append(taskGroups, osFeatureDetectionTasks)
}
// archives are related to the files within them only once every task has decided which files are described
if archiveTask != nil {
taskGroups = append(taskGroups, []task.Task{task.NewArchiveContainsTask()})
}
// identifying the environment (i.e. the linux release) must be done first as this is required for package cataloging
taskGroups = append(
[][]task.Task{
@@ -237,6 +270,9 @@ func (c *CreateSBOMConfig) makeTaskGroups(src source.Description) ([][]task.Task
var allTasks []task.Task
allTasks = append(allTasks, pkgTasks...)
allTasks = append(allTasks, fileTasks...)
if archiveTask != nil {
allTasks = append(allTasks, archiveTask)
}
return taskGroups, &catalogerManifest{
Requested: selectionEvidence.Request,
@@ -260,7 +296,7 @@ func (c *CreateSBOMConfig) selectTasks(src source.Description) ([]task.Task, []t
SearchConfig: c.Search,
RelationshipsConfig: c.Relationships,
DataGenerationConfig: c.DataGeneration,
PackagesConfig: c.Packages,
PackagesConfig: c.packagesWithArchiveSearch(),
LicenseConfig: c.Licenses,
ComplianceConfig: c.Compliance,
FilesConfig: c.Files,
@@ -440,10 +476,10 @@ func (c *CreateSBOMConfig) environmentTasks() []task.Task {
// unknownsTasks returns a set of tasks that perform any necessary post-processing
// to identify SBOM elements as unknowns
func (c *CreateSBOMConfig) unknownsTasks() []task.Task {
func unknownsTasks(cfg cataloging.UnknownsConfig) []task.Task {
var tasks []task.Task
if t := task.NewUnknownsLabelerTask(c.Unknowns); t != nil {
if t := task.NewUnknownsLabelerTask(cfg); t != nil {
tasks = append(tasks, t)
}
@@ -461,12 +497,36 @@ func (c *CreateSBOMConfig) osFeatureDetectionTasks() []task.Task {
return tasks
}
// packagesWithArchiveSearch returns a copy of the packages config with the top-level archive search
// booleans applied to java, which is still the only cataloger that reads them.
//
// ponytail: a top-level boolean wins only when it differs from the default, so callers who set just
// java's deprecated copy see no change. Delete this along with that field.
func (c *CreateSBOMConfig) packagesWithArchiveSearch() pkgcataloging.Config {
pkgs := c.Packages
def := cataloging.DefaultArchiveSearchConfig()
java := &pkgs.JavaArchive.ArchiveSearchConfig
if c.Archive.IncludeIndexedArchives != def.IncludeIndexedArchives {
java.IncludeIndexedArchives = c.Archive.IncludeIndexedArchives
}
if c.Archive.IncludeUnindexedArchives != def.IncludeUnindexedArchives {
java.IncludeUnindexedArchives = c.Archive.IncludeUnindexedArchives
}
return pkgs
}
func (c *CreateSBOMConfig) validate() error {
if c.Relationships.ExcludeBinaryPackagesWithFileOwnershipOverlap {
if !c.Relationships.PackageFileOwnershipOverlap {
return fmt.Errorf("invalid configuration: to exclude binary packages based on file ownership overlap relationships, cataloging file ownership overlap relationships must be enabled")
}
}
if j := c.Packages.JavaArchive.ArchiveSearchConfig; j.MaxDepth != 0 || j.MaxMemoryBytes != 0 || j.MaxDiskBytes != 0 {
return fmt.Errorf("invalid configuration: nested archive depth and limits are not java cataloger settings, set them with CreateSBOMConfig.Archive")
}
if a := c.Archive; a.MaxDepth != 0 && a.MaxMemoryBytes == 0 && a.MaxDiskBytes == 0 {
return fmt.Errorf("invalid configuration: nested archive cataloging is enabled but its memory and disk limits are both 0, leaving nowhere to extract archives")
}
return nil
}
+88 -1
View File
@@ -18,6 +18,7 @@ import (
"github.com/anchore/syft/syft/cataloging/pkgcataloging"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/java"
"github.com/anchore/syft/syft/source"
)
@@ -329,7 +330,7 @@ func TestCreateSBOMConfig_makeTaskGroups(t *testing.T) {
require.NotEmpty(t, tt.wantTaskNames)
// test the subject
gotTasks, gotManifest, err := tt.cfg.makeTaskGroups(tt.src)
gotTasks, gotManifest, err := tt.cfg.makeTaskGroups(tt.src, nil)
tt.wantErr(t, err)
if err != nil {
return
@@ -554,6 +555,31 @@ func TestCreateSBOMConfig_validate(t *testing.T) {
),
wantErr: assert.Error,
},
{
name: "nested archives enabled with no memory or disk",
cfg: DefaultCreateSBOMConfig().WithArchiveConfig(
cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1).WithMaxMemoryBytes(0).WithMaxDiskBytes(0),
),
wantErr: assert.Error,
},
{
name: "nested archives enabled with memory only",
cfg: DefaultCreateSBOMConfig().WithArchiveConfig(
cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1).WithMaxDiskBytes(0),
),
},
{
name: "nested archives enabled with disk only",
cfg: DefaultCreateSBOMConfig().WithArchiveConfig(
cataloging.DefaultArchiveSearchConfig().WithMaxDepth(1).WithMaxMemoryBytes(0),
),
},
{
name: "nested archives disabled with no memory or disk",
cfg: DefaultCreateSBOMConfig().WithArchiveConfig(
cataloging.DefaultArchiveSearchConfig().WithMaxMemoryBytes(0).WithMaxDiskBytes(0),
),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
@@ -564,3 +590,64 @@ func TestCreateSBOMConfig_validate(t *testing.T) {
})
}
}
func TestCreateSBOMConfig_archiveTaskGroup(t *testing.T) {
src := source.Description{Metadata: source.DirectoryMetadata{}}
archiveTaskIn := func(groups [][]task.Task) bool {
for _, group := range groups {
for _, tsk := range group {
if tsk.Name() == task.ArchiveCatalogerTaskName {
return true
}
}
}
return false
}
t.Run("no archive task by default", func(t *testing.T) {
// the feature is off by default, and off must mean no task at all rather than a registered
// task that does nothing: a registered task is reported as a selected cataloger
groups, manifest, err := DefaultCreateSBOMConfig().makeTaskGroups(src, nil)
require.NoError(t, err)
assert.False(t, archiveTaskIn(groups))
assert.NotContains(t, manifest.Used, task.ArchiveCatalogerTaskName)
})
t.Run("archive task is present at negative depth", func(t *testing.T) {
cfg := DefaultCreateSBOMConfig().
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(-1))
groups, _, err := cfg.makeTaskGroups(src, nil)
require.NoError(t, err)
assert.True(t, archiveTaskIn(groups))
})
t.Run("archive task is present when enabled", func(t *testing.T) {
cfg := DefaultCreateSBOMConfig().
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(2))
groups, manifest, err := cfg.makeTaskGroups(src, nil)
require.NoError(t, err)
assert.True(t, archiveTaskIn(groups))
assert.Contains(t, manifest.Used, task.ArchiveCatalogerTaskName, "the manifest must say archive contents were searched")
})
t.Run("enabling the feature does not mutate the caller's package config", func(t *testing.T) {
// the java side learns the archive task is running from the context, not from its config: a
// consumer who handed us a package config gets it back exactly as supplied.
supplied := java.DefaultArchiveCatalogerConfig().WithUseNetwork(true)
cfg := DefaultCreateSBOMConfig().
WithArchiveConfig(cataloging.DefaultArchiveSearchConfig().WithMaxDepth(2)).
WithPackagesConfig(pkgcataloging.Config{JavaArchive: supplied})
_, _, err := cfg.makeTaskGroups(src, nil)
require.NoError(t, err)
assert.Equal(t, supplied, cfg.Packages.JavaArchive,
"the caller's java config must come back byte-identical")
assert.True(t, cfg.Packages.JavaArchive.UseNetwork,
"a caller-supplied field must survive untouched")
assert.Zero(t, cfg.Packages.JavaArchive.MaxDepth,
"java's own squash-inlined depth must not be written to either")
})
}
+14 -14
View File
@@ -64,11 +64,11 @@ func (i *Cataloger) CatalogCtx(ctx context.Context, resolver file.Resolver) (map
return nil, err
}
prog := catalogingProgress(int64(len(locs)))
prog := catalogingProgress(ctx, int64(len(locs)))
results := make(map[file.Coordinates]file.Executable)
errs := sync.Collect(&ctx, cataloging.ExecutorFile, sync.ToSeq(locs), func(loc file.Location) (*file.Executable, error) {
prog.AtomicStage.Set(loc.Path())
prog.AtomicStage.Set(loc.FullNestedPath())
exec, err := processExecutableLocation(loc, resolver)
if err != nil {
@@ -93,15 +93,15 @@ func (i *Cataloger) CatalogCtx(ctx context.Context, resolver file.Resolver) (map
func processExecutableLocation(loc file.Location, resolver file.Resolver) (exec *file.Executable, err error) {
reader, err := resolver.FileContentsByLocation(loc)
if err != nil {
log.WithFields("error", err, "path", loc.RealPath).Debug("unable to get file contents")
log.WithFields("error", err, "path", loc.FullNestedPath()).Debug("unable to get file contents")
return nil, fmt.Errorf("unable to get file contents: %w", err)
}
defer internal.CloseAndLogError(reader, loc.RealPath)
defer internal.CloseAndLogError(reader, loc.FullNestedPath())
defer func() {
if r := recover(); r != nil {
// a malformed binary must not take down the whole SBOM; the caller records this as an unknown
log.WithFields("path", loc.RealPath, "panic", r).Debug("recovered from panic while reading executable")
log.WithFields("path", loc.FullNestedPath(), "panic", r).Debug("recovered from panic while reading executable")
log.Tracef("executable panic stack:\n%s", debug.Stack())
exec, err = nil, fmt.Errorf("recovered from panic while reading executable: %v", r)
}
@@ -109,14 +109,14 @@ func processExecutableLocation(loc file.Location, resolver file.Resolver) (exec
uReader, err := unionreader.GetUnionReader(reader)
if err != nil {
log.WithFields("error", err, "path", loc.RealPath).Debug("unable to get union reader")
log.WithFields("error", err, "path", loc.FullNestedPath()).Debug("unable to get union reader")
return nil, fmt.Errorf("unable to get union reader: %w", err)
}
return processExecutable(loc, uReader)
}
func catalogingProgress(locations int64) *monitor.TaskProgress {
func catalogingProgress(ctx context.Context, locations int64) *monitor.TaskProgress {
info := monitor.GenericTask{
Title: monitor.Title{
Default: "Executables",
@@ -124,7 +124,7 @@ func catalogingProgress(locations int64) *monitor.TaskProgress {
ParentID: monitor.TopLevelCatalogingTaskID,
}
return bus.StartCatalogerTask(info, locations, "")
return bus.StartCatalogerTask(ctx, info, locations, "")
}
func filterByGlobs(locs []file.Location, globs []string) ([]file.Location, error) {
@@ -146,7 +146,7 @@ func filterByGlobs(locs []file.Location, globs []string) ([]file.Location, error
func locationMatchesGlob(loc file.Location, globs []string) (bool, error) {
for _, glob := range globs {
for _, path := range []string{loc.RealPath, loc.AccessPath} {
for _, path := range []string{loc.LocalPath(), loc.AccessPath} {
if path == "" {
continue
}
@@ -169,13 +169,13 @@ func processExecutable(loc file.Location, reader unionreader.UnionReader) (*file
format, err := findExecutableFormat(reader)
if err != nil {
log.Debugf("unable to determine executable kind for %v: %v", loc.RealPath, err)
log.Debugf("unable to determine executable kind for %v: %v", loc.FullNestedPath(), err)
return nil, fmt.Errorf("unable to determine executable kind: %w", err)
}
if format == "" {
// this is not an "unknown", so just log -- this binary does not have parseable data in it
log.Debugf("unable to determine executable format for %q", loc.RealPath)
log.Debugf("unable to determine executable format for %q", loc.FullNestedPath())
return nil, nil
}
@@ -184,17 +184,17 @@ func processExecutable(loc file.Location, reader unionreader.UnionReader) (*file
switch format {
case file.ELF:
if err = findELFFeatures(&data, reader); err != nil {
log.WithFields("error", err, "path", loc.RealPath).Trace("unable to determine ELF features")
log.WithFields("error", err, "path", loc.FullNestedPath()).Trace("unable to determine ELF features")
err = fmt.Errorf("unable to determine ELF features: %w", err)
}
case file.PE:
if err = findPEFeatures(&data, reader); err != nil {
log.WithFields("error", err, "path", loc.RealPath).Trace("unable to determine PE features")
log.WithFields("error", err, "path", loc.FullNestedPath()).Trace("unable to determine PE features")
err = fmt.Errorf("unable to determine PE features: %w", err)
}
case file.MachO:
if err = findMachoFeatures(&data, reader); err != nil {
log.WithFields("error", err, "path", loc.RealPath).Trace("unable to determine Macho features")
log.WithFields("error", err, "path", loc.FullNestedPath()).Trace("unable to determine Macho features")
err = fmt.Errorf("unable to determine Macho features: %w", err)
}
}

Some files were not shown because too many files have changed in this diff Show More