The image source provider now reads images with `image.WithLazyLayerSquashes`: the image squash is built during the read, and each lower layer's squash only when something reads it. A squashed scan never does, so it skips a full file tree copy per layer.
- squashed scans: faster, much less memory on images with many layers
- `scan`/`attest` with all-layers or deep-squashed stay eager (an internal ctx flag), since eager squashing overlaps with fetching and indexing and is faster when every layer gets read
- library callers going through `syft.GetSource` get lazy for every scope, since the scope isn't known when the image is read. All-layers is still correct, just slower
- no public API change
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
stereoscope is replacing `image.WithImageSquashOnly` with lazy layer squashes, where every layer keeps a usable squash. That removes the need for `GetSourceConfig.WithImageSquashOnly` and the scope guard in `FileResolver`, so both go.
- bumps stereoscope to `feat/lazy-layer-squashes` (anchore/stereoscope `43c8f7e`), pending that PR merging
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
- stereoscope builds a squashed tree for every layer, each a full copy of the one below, so memory
grows with layers x files. The squashed scope only ever reads the top one.
- new `GetSourceConfig.WithImageSquashOnly(bool)` (and `sourceproviders.Config` field) reads images
with stereoscope's `image.WithImageSquashOnly`, scoped to that one read
- the CLI turns it on when `--scope squashed` (the default)
- an image source read this way returns an error from `FileResolver` for `all-layers` and
`deep-squashed` instead of panicking on the missing layer squashes
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Motivation: the envoy binary classifier returns no package for envoy 1.37.1,
1.40.0 and 1.40.1 (the official envoyproxy/envoy images), while neighbouring
versions such as 1.37.0 and 1.38.0 are detected.
Approach:
- the main matcher (NUL-delimited version followed by
envoy_reloadable_features) only accepted 1.3x; widen it to 1.[34]x, which
covers 1.40.0 and 1.40.1.
- in the 1.37.1 binary the version string sits between different neighbouring
strings, so the existing matchers do not fire. Add a matcher anchored on the
preceding "DeltaDiscoveryResponse.removed_resources" string.
- add snippets, config.yaml entries and test cases for 1.37.1, 1.40.0, 1.40.1
(linux/amd64 images, pinned by digest).
Validation:
- cd syft/pkg/cataloger/binary && go test . -run
'Test_Cataloger_PositiveCases/envoy' : the three new fixtures fail without
the classifiers.go change and all envoy cases pass with it.
- golangci-lint run ./syft/pkg/cataloger/binary/... : 0 issues.
- Built syft before/after and scanned the full linux/arm64 envoy binaries from
the official images: 1.37.1, 1.40.0 and 1.40.1 were not reported before and
are reported with the right version after; 1.37.0 and 1.38.0 are unchanged.
- Not run: the rest of the package test suite needs docker, which is not
available here (Test_PEPackageCataloger etc.), and snippets for other
binaries are not downloaded locally.
The contrib-build fallback matchers still only accept 1.3x; they are untouched.
Report: https://github.com/anchore/syft/issues/5470
Assisted-by: claude-sonnet-5-5 (via Claude Code)
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
The istio pilot-agent/pilot-discovery classifiers missed every
-snapshot.N build and the older -alpha.N/-beta.N/-rc.N layouts.
Two defects:
1. The doubled-version matcher anchored on a bare x.y.z followed by
NULs, but pre-release builds repeat the full version including the
suffix (1.15.0-beta.0 NUL NUL NUL 1.15.0-beta.0), so the anchor never
matched. Only the capture group had been widened previously.
2. -snapshot.N was absent from every alternation, and snapshot builds
store the version only once with no leading copy, so no existing
matcher could reach it.
Widen the leading anchor to accept the same optional pre-release suffix
it already accepts in the capture group, add -snapshot.N to each
alternation, and add a matcher for the standalone snapshot layout. The
new matcher is second so it can never shadow a normal build.
Adds real captured snippets for pilot-agent and pilot-discovery at
1.0.0-snapshot.0, 1.5.0-alpha.0, 1.15.0-beta.0 and 1.19.0-rc.0.
Fixes#5060
Signed-off-by: phatlc <phatle.hsd@gmail.com>
In the loop over `rootPkgs`, the error appended to `unknownErr` was incorrectly
using the outer `err` (from `packages.Load`) instead of the individual package
error `e`. This could cause:
- nil or stale error values to be recorded when package-specific errors existed.
- Loss of detailed per-package error information in the final error report.
Changed `unknown.Append(..., err)` to `unknown.Append(..., e)` to properly
capture each package's load error.
Signed-off-by: idhyt <idhyt3r@gmail.com>
* test: catalog GGUF models larger than the header
gguf-parser-go v0.26.3+ rejects a header whose tensors declare more data than follows it. Syft parses only a 50 MB copy of each model, so any larger model is dropped.
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix: fix parser with rollback and verify with test
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix: bound GGUF tensor info before the parser reads it
The header pre-scan stopped after the metadata KVs and left tensor infos to gguf-parser-go. Older releases of the library allocate whatever length a tensor name declares, so a tiny file could force a multi-GB allocation, or a fatal out of memory that panic recovery can't catch.
- tensor names are bounded like metadata keys (non-empty, length limit, shared string budget)
- dimension count is capped at `GGML_MAX_DIMS`
- a tensor count larger than the header now fails at EOF in the pre-scan
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: give the GGUF header copy the model's real size
The cataloger copies at most 50 MB of a model and parses that copy. Newer gguf-parser-go releases check the tensor data the header declares against the file size, so the truncated copy of any larger model was rejected.
- after the header is validated, the copy is extended to the size the resolver reports for the model, leaving the tail as a hole the parser never reads
- sources that report no size (OCI model artifacts today) keep the old behavior
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore(deps): restore gguf-parser-go v0.26.4
The header copy now carries the model's real size, so the pin to v0.26.0 and its dependabot ignore are no longer needed. This also brings back upstream's string and array length bounds, and the library's own `IsNumeric`.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: keep the extended GGUF header copy sparse on Windows
NTFS reserves disk for the full length of a file extended with truncate unless the file is marked sparse, so cataloging a large model on Windows reserved the model's size in the temp dir until cleanup.
- the header copy is marked sparse (`FSCTL_SET_SPARSE`) before it is extended
- if the filesystem can't make it sparse (FAT32, exFAT), the copy is not extended, and models larger than it fail to parse as before
- `golang.org/x/sys` moves from an indirect to a direct dependency
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* Add support for scanning IPK packages using the deb-archive-cataloger
Signed-off-by: Silas Parker <silas@srp.me.uk>
* fix(debian): share License field parsing between opkg status and archive control files
- the control-file `License:` fallback now goes through the same helper as the opkg status DB path, so a package reports the same licenses whether found installed or as an archive
- bitbake `&` / `|` are rewritten to SPDX `AND` / `OR` and the whole value is validated as one expression instead of split by hand, which kept grouping like `(A & B) | C` intact
- non-SPDX values fall back to a deduped list of names with operators and parens dropped (previously the status DB path emitted `&`, `(BSD`, etc as licenses)
- fields over 4KB are ignored, bounding work per archive and keeping deep paren nesting away from the recursive expression parser
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(debian): split Yocto recipe filenames in the Source field
Yocto writes the bitbake recipe filename (`Source: zlib_1.3.2.bb`) where dpkg expects a source package name. This is now split into `Source: zlib` / `SourceVersion: 1.3.2` for both ipk archives and opkg status DB entries, instead of surfacing the `.bb` filename as the upstream package.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(debian): do not label ipk archives as Debian packages
- `.ipk` archives no longer get a `pkg:deb` PURL from the assumed debian distro; opkg-based distros (Yocto, OpenWrt) have no purl type, so these now match packages found in an opkg status DB, which already get no PURL
- docs now cover `.ipk` and note that only ar-framed ipks are supported (OpenWrt's older gzipped-tar framing is rejected as an invalid archive)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test(debian): assemble the ipk fixture at build time instead of committing it
The fixture Dockerfile now builds a Yocto-shaped `.ipk` from a checked-in `control/` directory (copied from the original poky-tiny build) using `tar`, `zstd` and binutils `ar`, the same way bitbake's `package_ipk` packs one. This drops the prebuilt binary and the 20-30 minute Yocto `Dockerfile.build` while keeping the same archive layout and control metadata.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Silas Parker <silas@srp.me.uk>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* feat: add containers-storage support for Podman and Buildah, update tests and documentation
Signed-off-by: Bruce Clark <bruce.clark@mandg.com>
* test: update containers-storage tests to validate package.json and handle missing images
Signed-off-by: Bruce Clark <bruce.clark@mandg.com>
* test: fix rootless containers-storage test setup
Install uidmap for Podman and Buildah CI jobs.
Align temporary rootful and rootless storage paths.
Clean up rootless stores within the builder's user namespace.
Signed-off-by: Bruce Clark <bruce.clark@mandg.com>
* fix: only use containers-storage when explicitly requested
- containers-storage is no longer part of automatic image resolution; it's used only for `--from containers-storage`, the `containers-storage:` scheme, or `default-pull-source: containers-storage` (now an accepted value). Opening a store runs full graph driver init (mounts, locks, store writes), which is too invasive as a probe on every plain image reference.
- rootless stores must be read from inside the builder's user namespace (`podman unshare syft ...`), documented in the README.
- the `containers_image_openpgp` tag is now applied to unit/integration tests and lint, so dev tooling compiles the same provider that ships.
- containers-storage CI runs through `make containers-storage-test` and covers both vfs and overlay stores.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: exclude cgo btrfs driver and unmount overlay store in cleanup
- cgo-enabled linux builds (`-race` tests, lint) compile go.podman.io's btrfs graph driver, which needs libbtrfs headers. Add `exclude_graphdriver_btrfs` alongside `containers_image_openpgp` everywhere the tag is set; release builds are `CGO_ENABLED=0` and never included it.
- the overlay driver bind-mounts its graphroot inside the rootless user namespace, so the containers-storage test unmounts it before removing the store, and cleanup no longer fails the run.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Bruce Clark <bruce.clark@mandg.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Bruce Clark <bruce.clark@mandg.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ci: run the full unit suite on windows via make unit
Drops the curated `unit:windows` subset; the windows job now bootstraps like linux and runs `make unit`. Exceptions get added as failures show up.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ci: skip fixture refresh for windows unit tests
binny has no windows release, so the bootstrap action can't install the tools the fixture refresh needs.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: skip linux container fixtures when docker can't run them
Windows CI runners only run windows containers, so image fixtures go through `testutils.SkipWithoutLinuxContainers`. No behavior change on linux/mac.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: close leaked file handles and posix squashfs paths on windows
- output writers are now closed on early returns in `scan`/`convert` (open handles block cleanup on windows)
- `newSBOMMultiWriter` used `path.Dir` on host paths, so output subdirectories were never created on windows
- squashfs walk paths were built with `filepath.Join`
- skip a few posix-only test cases (dir perms, xdg fallback, go-rpmdb ndb handle leak upstream)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(golang): resolve go source module dir natively on windows
- `parseGoModFile` passed the resolver's posix (`/c/...`) path to the go toolchain, so go source resolution silently found nothing on windows
- skip container-built fixtures (go archs, java builds) and golden image tarballs on windows; stereoscope's layer cache filenames contain `:`, which windows rejects
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: keep posix paths posix on windows
- `ChrootContext.ToChrootPath` nested volumes (`D:\cwd\C:\root`) when the scan root was on another drive, so the prefix was never trimmed
- bitnami, python wheel/egg, and alpm parsers used `filepath` on posix location paths
- alpm backup-file lookups silently missed on windows
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ci: pull the test fixture cache for windows unit tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: don't map virtual file tree paths through the host on windows
`FiletreeResolver` without a chroot (snaps) ran request paths through `filepath.Abs`, rooting them on the current volume (`/payload.txt` -> `/d/payload.txt`), so snap scans on windows found nothing.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: match tar entry globs as posix on windows
- `matchesAnyGlob` used `doublestar.PathMatch` (host separator) on tar entry names, so tar-wrapped java archives silently produced nothing on windows
- test helpers now treat binary snippet keys and `FromFile` locations as posix
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* incorporate windows path fix
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* incorporate go-rpmdb close fix
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* close squashfs fixture writers before finalizing
- windows dir listings can report a zero size for files still open, so the manifest was packed empty
(also adds a temporary windows diagnostic for the binary cataloger)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* temp: surface windows binary cataloger diagnostics
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: keep directory symlink targets posix on windows
- the directory indexer joined symlink targets with native separators, so links in the posix file tree pointed nowhere and globs on link names found nothing
- binary fixture logical keys used native separators, so nested snippets were picked as the fixture dir on windows
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* keep addSymlinkToIndex under the funlen limit
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* drop windows skips covered by the stereoscope and go-rpmdb bumps
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(dotnet): resolve packages.lock dependencies within their target framework
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix(dotnet): make packages.lock package metadata deterministic across target frameworks
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix: parse packages lock tests
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix: update lockfile parse to read into stable structure
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix: remove stale comments
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* test: new test for project entry case
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix(dotnet): key packages.lock packages case-insensitively
NuGet IDs are case-insensitive, and edge lookup already treated them that way, but package identity used the exact-case name. Two spellings of one ID and version across target frameworks became two packages, and type precedence never ran between them.
- packages merge on the lowercased name + version, display name comes from the winning entry
- a package listing itself (under any casing) no longer produces a self edge
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
The `perl-binary` classifier now emits `pkg:cpan/perl@<version>` instead of `pkg:generic/perl@<version>`.
`perl` is itself a CPAN distribution and carries advisories under that name, so typing it as `generic` left every one of them unreachable.
This changes an existing purl. Anything keyed on `pkg:generic/perl` (allowlists, policy, SBOM diffs) needs updating.
Builds on the cpan cataloging branch, which adds the package type this uses.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* feat: catalog CPAN distributions installed by perl clients
Adds `perl` as a language and `cpan` as a package type, with two catalogers behind them.
- `perl-cpan-installed-cataloger` reads `.meta/*/install.json` (cpanm, cpm, carton) and `auto/**/.packlist` (anything installed through `ExtUtils::MakeMaker` or `Module::Build`, including CPAN.pm). Both globs are unanchored, so a local-lib or carton application tree is found the same as a system install.
- `perl-cpan-meta-cataloger` reads an unpacked release's own `META.json` or `META.yml`, gated on a sibling `MANIFEST` so source checkouts are ignored.
Packages are keyed on the **distribution**, not the module, because that is what CPAN, MetaCPAN and the advisory data all use. `LWP.pm` belongs to `libwww-perl` and reporting it as `LWP` would make every advisory for it unreachable. The distribution name comes from `install.json`'s `dist` field, and from the `auto/` path for packlists.
purls are `pkg:cpan/<distribution>@<version>`, with the PAUSE author added as an `author` qualifier when the evidence carries it. Metadata comes in two types: `cpan-distribution` for installed evidence and `cpan-unpacked-release` for a release sitting on disk. The tiers differ in more than a name, so a consumer should not have to string-match a cataloger name to tell them apart: an unpacked release has no PAUSE path and therefore no author and no file list, and "installed and loadable by the interpreter" is a materially stronger claim than "a source tree exists here".
A packlist's version comes from `perllocal.pod`. EUMM writes it and the packlist from the same variables in the same install target, so the `auto/` path segments and the perllocal `Module` name are the same key, and the recorded `VERSION` is what was evaluated at build time rather than what can be read back statically. Scraping `$VERSION` out of the main `.pm` is the fallback, since `NO_PERLLOCAL` suppresses the file and Module::Build never writes one. Three rules pick the stanza: dashed module name, longest `installed into` libdir that prefixes the packlist path, and last match wins because the file is append-only. Without the libdir rule a second perl on the image silently supplies the version.
Where scraping is the fallback, it reads more than a plain `our $VERSION = '...'`. A version declared in the package statement (`package Foo::Bar v1.0.0;`) counts, which matters because a distribution can declare it that way and carry no `$VERSION` at all: `CPAN::02Packages::Search` is one, and without this it reports no version and matches nothing. Fully qualified `$Foo::Bar::VERSION` counts too, which is what older Dist::Zilla emitted and what real `JSON::PP` 2.27300 still carries. `qv('1.2.3')` is handled. A version computed at runtime is not, and those are reported without one rather than guessed at.
Packlist entries are parsed the way `ExtUtils::Packlist` writes them: a line can carry space-separated metadata after the path (`/path/to/File.pm type=file`), which is what `installperl` produces, so the suffix is stripped rather than the line being cut at its first space.
`META.yml` is read alongside `META.json`, because about 43% of current CPAN releases ship no `META.json` and they skew old, which is where the advisories are. Where both sit in one directory the `META.json` wins. A `META.yml` a strict parser rejects skips that directory rather than failing the scan, which is routine rather than defensive for pre-spec releases.
A packlist is literally a file list, so its paths are surfaced through `pkg.FileOwner`. They are reported as recorded, unfiltered: a path the packlist claims and the filesystem lacks means the file was removed or overwritten out from under the installer, which is worth seeing rather than hiding.
Build leftovers under `~/.cpanm/work` and `~/.cpan/build` are skipped. They genuinely are unpacked release tarballs, `MANIFEST` included, so the `MANIFEST` gate admits them and a path exclusion is the only signal available. Without it every distribution on an image that did not clean up is reported twice. The cost is that a tarball deliberately kept under `~/.cpan/build` stops being reported.
Two behaviors that look wrong but are not:
- a distribution can be reported twice at different versions. `libwww-perl` 5.836 bundles `HTTP-Date`, `HTTP-Message` and `LWP-MediaTypes`, and installing it over the modern standalone releases overwrites their `.pm` files. Both versions are genuinely present, so the merge refuses to pair disagreeing versions rather than hiding one.
- a distribution with no readable version is reported without one rather than dropped, so it stays visible.
The coverage boundary is stated in full in the `package perl` doc comment. In short: modules installed from distro packages are out of scope, since packagers strip CPAN metadata with `NO_PACKLIST` and deb, rpm and apk already report them. Core and dual-life distributions bundled with the interpreter have no coverage, because nothing on disk carries their versions; the interpreter itself is reported separately. Vendored trees, `App::FatPacker` output and PAR archives are invisible, because what they carry is module identity with no offline map to a distribution. A packlist-derived name is the installer's `NAME` and may not be the distribution name, which is left to the vulnerability data to resolve.
One correction worth calling out, since it is easy to arrive at twice: a sibling `MANIFEST` is the only thing separating an unpacked release from a source checkout. An earlier version of the guard also rejected any tree containing a `dist.ini`, on the theory that it marked a Dist::Zilla source tree. dzil ships `dist.ini` *inside* the tarballs it builds and lists it in the generated `MANIFEST`, so that exclusion was silently skipping real releases, `URI` among them.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test(cli): bump coverage image package count for the new perl fixture
The CPAN fixture added to `image-pkg-coverage` (`URI-5.35`) is shared with the CLI suite through a symlink, so the squashed package count goes from 42 to 43.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(perl): take CPAN distribution name and version from install.json dist
cpanm writes the main module's `$VERSION` into `version` and the release distvname into `dist`, and the two often disagree (`Carp-Assert-More-2.9.0` records `2.009000`). Trimming `version` off `dist` failed on those, so the whole distvname became the package name, and the module version never paired with the packlist record for the same install, which reported the distribution twice.
- name and version are now split out of `dist` at the last `-` followed by a version, as CPAN::DistnameInfo does
- a `-TRIAL` suffix is dropped, so `Try-Tiny-0.26-TRIAL` reports as `Try-Tiny` `0.26`
- the PAUSE path fallback uses the same split
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
The traefik-binary classifier regex required a NUL byte or replacement
character immediately before the version string. On some architectures
(e.g. s390x), the real version string in the binary has no such prefix,
so the classifier instead matched an earlier, unrelated NUL-prefixed
numeric string elsewhere in the binary (e.g. '2.7.0' from an unrelated
printf-style format constant), producing an incorrect version.
This adds an additional pattern, tried first via MatchAny, that matches
version strings terminated by two NUL bytes without requiring a NUL
prefix. The original pattern remains as a fallback for older traefik
binaries (e.g. v1.x) that use single-NUL termination.
Fixes#4980
Signed-off-by: Mayank Sekhar <mayankshekharsingh@gmail.com>
* fix: scan a Java resource adapter (.rar) given as a file source
The file source picks an extractor for the scan target from its file name
alone. A Java resource adapter archive (JCA) uses the .rar extension but is
a zip file, so it was handed to the RAR extractor. With rardecode v2.2.0 the
RAR extractor does not return on non-RAR input (an infinite loop in its
signature search), so `syft scan --from file adapter.rar` never finished.
The same bytes named .jar scan in under a second.
When the name says RAR but the content is a zip archive, leave the file
unextracted, the same way .jar, .war and .ear scan targets are handled. The
java cataloger already reads .rar files (anchore/syft#4136) and now finds the
resource adapter and its nested jars. Real RAR archives are still extracted.
Signed-off-by: Ismael Donmez <ismael.donmez@chainguard.dev>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore(deps): bump github.com/nwaples/rardecode/v2 from 2.2.0 to 2.4.1
rardecode v2.2.0 loops forever in its RAR signature search when the input
does not start with a RAR signature (fixed upstream in nwaples/rardecode
f501ac8, "fix infinite loop in findSig", released in v2.2.2). syft reaches
that search for any file source whose name contains ".rar", so a scan of a
self-extracting RAR, or of any other file with such a name, never finished.
With v2.4.1 a self-extracting RAR is extracted, and a file that is not a RAR
archive fails fast with "rardecode: RAR signature not found", the same way a
file named .zip that is not a zip archive fails today.
github.com/mholt/archives v0.1.5 (the latest release) still requires v2.2.0,
so the newer version is selected here.
Signed-off-by: Ismael Donmez <ismael.donmez@chainguard.dev>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix: clarify the .rar zip log line and cover self-extracting RAR sources
- the debug line no longer calls every `.rar`-named zip a resource adapter
- a self-extracting RAR (stub before the signature) given as a file source is
still extracted
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Ismael Donmez <ismael.donmez@chainguard.dev>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ci: drop gen-5 instance families from the test runner
m5a runners run the unit suite ~2.3x slower than m8id (15m vs 6.5m for `go test`), and when spot capacity for newer families dries up runs-on falls back to them. Keeping gen-6 through gen-8 leaves a wide spot pool without the slow tail.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: shrink decompression bomb fixtures
The bomb and heap-bound tests pushed 100MB-16GB through gzip and parsers, which under `-race` made them the slowest tests in the unit suite. Each now uses the smallest fixture that still separates the bounded path from the unbounded one.
- arch mtree caps are measured at a small cap and scaled to the shipped constants, so raising a cap still fails the budget
- snap and debian bombs expand to 32MB instead of 512MB (4x their 8MB budgets)
- golang ELF bombs still declare 256MB but deliver 64MB
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: add windows unit test job and capture dir resolver path bug
- add a `Unit tests (Windows)` job running a new `make unit:windows` task, scoped to the packages that own host path handling (tests needing linux docker fixtures are skipped)
- `Unit tests` is now a gate job over the Linux and Windows unit jobs
- add a resolver test asserting that locations from `AllLocations` resolve again via `FilesByPath` and use posix paths. This is expected to fail on windows today, which is why file digests come back empty for `dir:` scans (#5325)
- fix windows-only test files that no longer compiled
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore: drop verbose output from integration tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: windows path handling outside of the file lookup bug
- chroot paths are always posix (`ToChrootPath` no longer returns backslash paths on windows), same for the directory source name derived from `base`
- `windows.FromPosix` no longer panics on input that isn't volume-encoded (e.g. relative or already native paths)
- skip tests on windows that rely on symlink fixtures, shell script fixtures, or emulate windows paths on a posix host
- disable `core.autocrlf` on the windows CI job so fixture digests match linux
This leaves only the `FilesByPath` lookup failures from #5325.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: skip posix-only fileresolver tests on windows
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: split contained paths on posix separators on windows
- `allContainedPaths` cleaned with the native separator but split on `/`, so a root reached through a symlink never had its ancestors indexed on windows
- index assertions in tests now key on the posix path the index actually stores
- skip symlink fixture tests and a posix-only test helper on windows
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore: revert unrelated go.sum churn and allContainedPaths change
- restore `go.sum` and `.make/go.sum` (accidentally populated by a local `-mod=mod` invocation)
- revert the `allContainedPaths` separator change: production passes native windows paths there, and splitting them on `/` produces drive-relative ancestors, so skip the posix-only test on windows instead
- annotate the remaining `filepath` usages that operate on native paths
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: search the file tree with posix paths on windows (#5325)
The directory and file resolvers key their file tree on posix paths (`/c/some/path`), but `FilesByPath`, `HasPath` and root-anchored `FilesByGlob` searched it with the native path from the chroot context (`C:\some\path`). On windows every lookup silently missed, so anything that resolves files by path found nothing. This is why file digests were empty for `dir:` scans.
Requests are now converted to posix before searching the tree.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(spdx): write a name for the root package when the source has none
A source read from another SBOM (a purl list, a CycloneDX document
without metadata.component, an SPDX document without a DESCRIBES
relationship) can have no name. The SPDX encoder then wrote the document
root package with an empty PackageName, which SPDX requires. In
tag-value output the PackageName line is omitted, so the root package's
fields land in the previous section and syft cannot read its own output
back ("received unknown tag SPDXID in CreationInfo section").
Fall back to the document name for the root package, and make the
document name fall back to "unknown" for every source type rather than
only for unrecognized ones.
Signed-off-by: Tony <tjagodka@gmail.com>
* fix(spdx): drop syft's root package on decode for every SPDX version
- the root package name now comes from one fallback chain (source name, image repo name, source ID, document name), so the root purl no longer gets an empty name for a source decoded from CycloneDX
- decoding strips the root syft writes for an unknown source, so `syft convert` round trips stop adding an `unknown` package each pass
- for SPDX 2.1 and 2.2, which have no `primaryPackagePurpose`, the purpose is taken from the source type in syft's `DocumentRoot-<type>-` ID
**behavior change:** decoding a syft-written SPDX 2.1/2.2 document no longer returns the root as a package or its `CONTAINS` relationships, matching 2.3.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Tony <tjagodka@gmail.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(python): link uv.lock dependents to the locked version they name
A forked uv.lock can lock several versions of one package, and uv records the version on each dependency entry in that case. The parser dropped that field, so every pandas version depended on every numpy version. Carry the locked version through provides and requires as name@version (bare name stays for unique locks) and keep marker text unchanged.
Fixes#5340
Signed-off-by: Dev M <devtechedge@gmail.com>
* fix(python): pair uv.lock versions without widening public metadata
- the locked version on a uv dependency entry is only needed to pair relationships, and the relationships already carry it, so it stays on the internal toml struct instead of `pkg.PythonUvLockDependencyEntry`. The previous approach hid it from JSON but still hashed it into package IDs.
- dependency specs are now built from the raw lock entries at parse time rather than read back off package metadata
- optional-dependencies get the same version pairing, so an extra that names one locked version no longer links to all of them
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Dev M <devtechedge@gmail.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
Motivation:
syft's PHP interpreter cataloger reports each bundled PHP C extension
(openssl, ldap, sqlite3, etc.) as its own package, using the PHP
interpreter's own version number as the package version (since these
extensions are compiled into a specific PHP release and have no
independent version of their own). The classifier already attaches an
explicit CPE like cpe:2.3:a:php-openssl:php-openssl:8.5.7:... to make
clear this refers to the bundled PHP extension, not upstream OpenSSL.
However, that CPE was marked with Source: cpe.GeneratedSource. Syft's
package-finalization pipeline (internal/task/package_task_factory.go)
only skips its generic, name-based CPE generation when a package
already has an "authoritative" CPE (any Source other than
GeneratedSource) - so it also generated and appended a second CPE,
cpe:2.3:a:openssl:openssl:8.5.7:..., derived from the bare package
name. That second CPE is indistinguishable from a real standalone
OpenSSL/LDAP/SQLite installation and can cause vulnerability scanners
that consume this SBOM to falsely flag PHP hosts as running vulnerable
upstream libraries at version "8.5.7" (a PHP version, not a real
OpenSSL/LDAP/SQLite release). See anchore/syft#5014 for a full repro.
Approach:
Mark the classifier's explicit CPE with Source: cpe.DeclaredSource
instead of cpe.GeneratedSource, so it is treated as authoritative and
the extra name-derived CPE is no longer generated. This mirrors the
existing pattern in
syft/pkg/cataloger/java/parse_jvm_release.go's newJvmCpe, which uses
the same DeclaredSource + comment convention to mark a
syft-synthesized (not literally file-declared) CPE as trustworthy and
suppress further generation. It also brings this classifier in line
with its sibling php-cli/php-fpm/php-apache classifiers in the same
file, which already mark their CPEs as authoritative (via
NVDDictionaryLookupSource, not applicable here since php-<ext> is not
a real NVD dictionary entry).
This change only suppresses the extra generated CPE; it does not
change the package name, type, version, or PURL, and the existing
php-<ext>:php-<ext> CPE is unaffected.
Validation:
go build ./...
go test ./syft/pkg/cataloger/php/... ./internal/task/... ./syft/pkg/cataloger/internal/cpegenerate/...
All pass except the two pre-existing docker-image-fixture subtests of
Test_InterpreterCataloger, which fail in this sandbox only because the
`docker` binary isn't available (exec: "docker": executable file not
found in $PATH); confirmed via `git stash` that these two subtests
fail identically on main without this change, i.e. this is a sandbox
limitation and not a regression. Added a new unit test,
Test_getClassifier_declaredCPESource, that directly asserts the
classifier's CPE now carries Source: cpe.DeclaredSource.
Fixes#5014
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
* fix: report scanner errors instead of returning a partial parse as complete
A line longer than `bufio.Scanner`'s 64KB buffer ends the scan with `ErrTooLong`, which only `Err()` reports. Parsers now return what they found along with the error (recorded as an unknown for the file), or log it where there is no error to return.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: allow lines up to 1MB in line-oriented package metadata parsers
Adds `internal.NewLineScanner`, a `bufio.Scanner` capped at 1MB per line rather than the 64KB default, the same cap the alpm parser already uses. Long single-line fields (descriptions, license text, dependency lists) now parse instead of ending the scan early.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: use io.ReadFull where a short read was silently accepted
- dotnet bundle 7-bit int decoding could read a zero-byte result as a zero byte
- wordpress plugin header parsing included the unread tail of its buffer
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: finish moving line scanners to the 1MB cap and checking scanner errors
- java manifest, pom.properties, Gemfile.lock, requirements.txt, homebrew formula, alpm and the purl/cpe decoders now use `internal.NewLineScanner`
- snap kernel changelog reports a scan error instead of silently dropping the base kernel package
- the attestation output reader keeps draining the pipe after a scan error so cosign can't block on a full pipe
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: stop logging the full setup.py line once per dependency match
The debug log ran once per regex match and included the whole line, so log volume grew with the square of line length. With 1MB lines that is hundreds of GB of debug output from a single crafted file.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(gentoo): don't dereference a nil location when a LICENSE scan fails
`extractLicenses` accepts a nil location, but the scan error log added earlier on this branch dereferenced it.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(gentoo): bound license group expansion on hostile license_groups
- dedup within a group uses a set instead of `slices.Contains`, which was quadratic in tokens per line and got much worse under the 1MB line cap
- each group is expanded once and memoized, so a chain of groups that reference each other repeatedly no longer expands exponentially
- repeated references to the same group within one line are skipped
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(javascript): don't panic on a yarn v1 lockfile line missing its value
A v1 `yarn.lock` dependency line without a version constraint, or a
`version`/`resolved`/`integrity` line without a value, indexed past the end of
the split line. Such dependencies are now skipped and empty fields left empty.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(alpine): don't panic on an apk db ACL or checksum line before its entry
An `M:`, `a:` or `Z:` line that comes before any directory or file entry in
`lib/apk/db/installed` indexed `files[-1]`. Such a line is now skipped.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(erlang): don't panic on a truncated rebar.lock or .app file
The term parser read past the end of the input when a file ended mid-term:
whitespace only, an unclosed list, a lone `<`, or a comment ending in `\r`.
These now return an "unexpected end of input" error.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(lua): don't panic on a truncated .rockspec
The rockspec parser read past the end of the input when a file ended mid-statement:
a lone `[`, an unclosed `[[...]]` value or `[...]` index, or a `local` assignment cut off
before its `=`, value, or line end. These now return an "unexpected end of input" error.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(swift): don't panic on a Podfile.lock pod without a version
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(gentoo): don't panic on a short obj line in portage CONTENTS
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(spdx): don't panic on an opam checksum without an algorithm
The opam cataloger stores checksums verbatim, so a malformed one reached the SPDX encoder and panicked there, outside the cataloger's recover. Checksums without an `=` are now skipped.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* refactor: use strings.Cut where a split was only indexed after a Contains check
Also guards `windows.FromPosix` against a path with no separator.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: guard untyped package metadata fields
- new `any`-typed metadata fields must be allowlisted with a reason
- allowlisted untyped fields are pushed through `SetID` and every encoder with
non-finite, deeply nested and large values
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: fuzz every generic cataloger parser
One fuzz target over every glob-selected parser in the capabilities data, run
through its cataloger task. Each input must not panic, must finish in time and
must keep allocations proportional to its size. Seeds come from each parser's
testdata plus the string literals in its package.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: scan malformed inputs under memory and time limits
A directory of small malformed files (deep nesting, oversized declared counts,
truncation, type mismatches, non-finite floats) is scanned by the CLI with a
memory limit and an RSS ceiling. The scan must exit 0 and write an SBOM.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: stop a fuzz input once its live heap passes the allocation limit
The allocation check only ran after the parser returned, so an input on its way to exhausting memory ran the machine into swap first. The harness now polls the live heap while the parser runs and panics past the same limit, which the fuzz engine saves as a crasher.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: bound nesting depth and error context in the erlang and lua parsers
- a deeply nested rebar.lock or .rockspec overflowed the goroutine stack, which is fatal rather than a recoverable panic, so one file could take down the whole scan. Both parsers now stop at 1000 levels with an error.
- parse errors echoed whole input lines into the SBOM as unknowns, and lua appended another copy at every nesting level. `PrintError` now clips to a window around the error column, and lua decorates once at the top.
- the Podfile.lock error no longer echoes the pod entry.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: close ways the malformed-input checks could pass quietly
- a ctx-aware parser hitting the fuzz timeout now fails the input instead of racing the hard deadline
- the heap watchdog measures growth from a post-GC baseline rather than an absolute process-wide number
- a saved fuzz input naming a target that no longer exists fails instead of being skipped
- the CLI malformed-input scan fails on a recovered parser panic, not only on a non-zero exit
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(cpe): cap sub-selection prefixes to bound candidate cross-product
a name with thousands of `-`/`_` separators produced one vendor and product candidate per segment, and the vendor x product cross-product made CPE generation roughly cubic in time and memory (8KB name: ~250s, 4.8GB RSS). `generateSubSelections` now stops after 10 prefixes. The full name is still a candidate, so real package names are unaffected.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(cpe): bound candidate count and field length before the cross-product
the prefix cap only covered one way to inflate the vendor x product cross-product in `FromPackageAttributes`. Java groupIDs add a vendor (and, with no artifactID, a product) per dot segment, and .NET executables add one of each per file, so a 1000-segment `Bundle-SymbolicName` still produced ~1M CPEs and several GB.
- vendor and product candidates are each capped at 64 (shortest first, so truncation is deterministic)
- names, versions, and candidates over 512 bytes are skipped, since every byte is paid once per CPE in key building and regex validation
real packages produce around a dozen candidates with names well under 100 bytes, so output for real inputs is unchanged. Also corrects the earlier comment: prefixes only become vendor candidates and the cost was quadratic, not cubic.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
UPX pads to a 4 byte boundary before it writes the loader stub. The
image-small-upx build happened to need no padding, so the cataloger
tests over it passed with or without the fix in #5347.
- build the fixture with -X main.Version=1.0.11, which needs 2 bytes of
padding, so the existing cataloger tests fail without the fix
- add TestImageSmallUPXNeedsLoaderPadding, which fails if a change to the
fixture's inputs means it no longer needs padding
- run the crafted loader test on both sides of the boundary, so a skip
that always rounds up is caught too
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
UPX pads its output to a 4 byte boundary before it writes the loader
stub, and l_lsize counts from that boundary. The block chain resumed at
the end of the last PT_LOAD extent plus l_lsize, so whenever the
compressed extents ended off a 4 byte boundary it landed 1 to 3 bytes
early, read a misaligned b_info, and stopped. The padding between the
segments was never filled in, the reconstruction ended at that first
hole, and .go.buildinfo in the data segment went with it, so the binary
contributed no packages.
The loader skip now rounds up to 4 before adding l_lsize, which matches
the funpad4 in UPX's own unpacker. The crafted fixture for the tail
extents put its stub straight after the head extent, the layout the bug
assumed, so it now pads the way UPX does.
Signed-off-by: huuyafwww <huuya.yamauchi@3-shake.com>
Same issue as #5350: go1.27's compress/flate encodes this 512MB fixture to just over 1MB, so the fixture sanity check tripped before the bound was exercised. The line bound itself is fine.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
go1.27's compress/flate encodes 512MB of zeros to ~1MB (1052690 bytes) where go1.26 gives ~510KB, so the fixture sanity check tripped on newer toolchains before the bound was ever exercised. The decompression bound itself is fine (bounded read allocates ~300KB on both).
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
- pre-scan the GGUF metadata KV section and reject values gguf-parser-go would panic on, recurse on without bound, or overallocate for: wrong typed well-known keys, zero alignment, nested or oversized fully decoded arrays, an unreasonable `split.tensors.count`, and empty strings the parser would misread. Keys are matched the way the parser sees them. The KV count, fully read string bytes, and skipped string counts are bounded too. The file becomes an unknown instead of taking down the scan.
- `copyHeader` now bounds its own read to the max header size.
- non-finite floats in `header` (including inside arrays) become `"NaN"`, `"+Inf"` or `"-Inf"` so JSON encoding can't fail. `header` and `metadataHash` are otherwise unchanged.
- a `metadataHash` failure is now an error instead of an empty hash.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(ai): keep merging GGUF layers when one fails to parse
a single unparseable layer in an OCI GGUF artifact skipped the merge, so the
named model lost its parts and nameless layers were dropped. The error is
still returned.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(ai): only send GGUF layers to the GGUF cataloger
the media type glob `application/vnd.docker.ai*` also matched config,
model.file, license and safetensors layers, so every safetensors OCI model
reported spurious "invalid GGUF magic number" unknowns. Match
`application/vnd.docker.ai.gguf.v3` exactly.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(ai): don't wrap safetensors parameter counts
- tensors whose shape product overflows uint64 are skipped instead of wrapping
- per-file and per-shard parameter totals saturate instead of wrapping
- parameter count and dominant dtype are computed in one pass
- GGUF licenses are created with the scan context, and drop no-op unknown checks
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* refactor(ai): derive safetensors model info from the header in one place
the file and OCI layer parsers built identical metadata from a parsed header.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(ai): keep the GGUF media type with the GGUF cataloger
also corrects stale comments in the GGUF merge processor.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Companion files next to AI models (`config.json`, `README.md`, and the OCI model.file, license and model config layers) are now read with named size limits, and are only used when they look like what they claim to be.
- JSON companions are read whole up to a limit and skipped when larger, rather than being fully buffered or parsed truncated. OCI companion layers are fetched one byte past the cap so oversize is detectable.
- README and license layers read a prefix. The frontmatter block is capped at 256 KB and list values at 32.
- dir and OCI scans share one 8 MB safetensors header cap. Directory scans used to allow 100 MB.
- `config.json` is only used when it has `architectures` or `_name_or_path`, and is looked up beside the model or in its parent directory, no longer all the way up to the scan root
- OCI model.file layers are processed in a stable order, so the chosen name no longer changes between runs
- README frontmatter:
- a list-form `license:` is accepted and no longer drops `base_model`
- `null` values are ignored
- delimiters must be exact `---` lines
- model names from `_name_or_path` / `base_model`:
- Windows paths are handled
- `.`, `..` and `/` fall back to the next name source
- names are capped at 256 characters
Behavior changes worth noting:
- a `config.json` two or more directories above the model no longer names it
- JSON `config.json` files without `architectures` or `_name_or_path` (e.g. `generation_config.json`, app configs) are ignored
- safetensors headers over 8 MB in directory scans are now rejected, matching OCI scans (real shard headers are well under 1 MB)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(generic): recover from parser and processor panics
- a panic while parsing one file becomes an unknown for that location instead of failing the whole SBOM
- a panic in a package processor is logged as a warning and the unprocessed results are kept
- stacks go to the trace log, not into the unknown text
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(unknown): keep errors that unwrap to nil
`sync.PanicError` with a non-error panic value (e.g. `panic("boom")`) unwraps to nil, and was being dropped entirely from the error graph instead of being kept as a leaf.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(task): don't put the stack trace in task panic errors
Task panics are still fatal, but the error now reads `panic in task "<name>": <value>` and the stack goes to the debug log.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test(generic): cover chained processors when a later one panics
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(file): recover from panics in the executable and file digest catalogers
A panic while reading one file is now recorded as an unknown for that file instead of failing the whole SBOM, the same as generic catalogers.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(ai): use go.yaml.in/yaml/v3 to parse model card frontmatter
gopkg.in/yaml.v3 panics on some malformed merge-key payloads (for
example "<<:\r? 0:"), which crashed the gguf and safetensors catalogers
when a model README contained such frontmatter. Switch the Hugging Face
frontmatter parser to the maintained go.yaml.in/yaml/v3 fork, which is
already used by the other catalogers and returns an error instead of
panicking. The existing error path logs and skips the frontmatter.
Fixes#5327
Signed-off-by: tk1475 <118763009+tk1475@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore: finish moving off gopkg.in/yaml.v3
- move the snap catalogers and `internal/capabilities` to `go.yaml.in/yaml/v3`. The snap parsers decode yaml from scanned sources and hit the same merge-key panic as #5327
- enable `depguard` with a deny rule for `gopkg.in/yaml.v3` so it can't be reintroduced
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: tk1475 <118763009+tk1475@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
The package supplier is optional in SPDX, but fileSource dereferenced it
without the nil check containerSource already has, so a valid SPDX 2.3
document whose described package has primaryPackagePurpose FILE and no
supplier crashed syft convert and grype sbom: input.
A JSON null in files, a package's files or externalRefs (or an SPDX 2.1
package the version converter cannot map) decodes to a nil pointer that
the conversion then dereferences. Drop nil entries once at the start of
ToSyftModel instead of guarding each loop.
Signed-off-by: Tony <tjagodka@gmail.com>
* fix(java): resolve Groovy 4+ group IDs to org.apache.groovy
Groovy 4.0 moved from the Codehaus coordinates (org.codehaus.groovy) to the
Apache Software Foundation (org.apache.groovy). JARs that ship no pom
metadata fall through to the static artifact-to-group map, which carries the
3.x-era org.codehaus.groovy entries for the whole groovy family, so 4.x
artifacts were attributed to an obsolete group even though their manifest
identifies the Apache vendor.
Where the static map attributes an artifact to the historical group, a
major-version migration resolves groovy 4.x and newer to org.apache.groovy,
while 3.x and earlier keep their historical group. The Eclipse-hosted
groovy-eclipse-* artifacts, which share the prefix but kept their Codehaus
coordinates, are excluded.
Fixes#5311
Signed-off-by: texasich <101962694+texasich@users.noreply.github.com>
* refactor(java): share version-aware groovy group ID lookup with CPE generation
- move the artifact-to-group lookup behind `cpegenerate.ArtifactIDToGroupID(artifactID, version)` so the PURL and CPE paths agree; the exported mutable map is now unexported
- version-dependent entries are expressed as a `groupIDMigration{before, after, atMajor}` (groovy moved to `org.apache.groovy` at 4.0); unconditional entries, including `groovy-eclipse-*` and apache-only 4.0+ modules, stay in the plain map
- add apache-only modules (`groovy-ginq`, `groovy-contracts`, `groovy-callsite`, ...) that otherwise took the group from their per-module `Automatic-Module-Name`
This changes CPEs for pom-less groovy 4+ jars (all real ones): vendors are now `apache` (e.g. `cpe:2.3:a:apache:groovy:4.0.33`) instead of `codehaus`, matching the PURL, jars with pom metadata, and NVD.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: texasich <101962694+texasich@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
closes#5299: docker:// references are OCI images, not GitHub repos.
Returning empty string produces a package with no PURL, matching the
treatment for local composite actions.
Signed-off-by: Yunare Maia <yunare@gmail.com>
An extra-dep can be pinned as "@sha256:<hash>,<size>" or as "@rev:<n>". The hash
slice assumed the first form, so a rev pin left the end index at -1 and the
parser panicked with slice bounds out of range. A rev pin carries no package
hash, so only the sha256 form now yields one.
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
* build(deps): bump go-make to v0.8.1
Picks up golangci-lint 2.13.2 (built with go1.27) in place of 2.11.4 (go1.26).
The old build panics outright on a go1.27 GOROOT with "file requires newer Go
version go1.27", and an older mise-shim build fails the quieter way: it loses
type info and reports zero findings with exit 0, so a ruleguard run comes back
green without having evaluated anything.
Also folds in what the newer linter flags:
- `reflect.Ptr` -> `reflect.Pointer` in three files, applied by --fix
- `tw.CellFormatting{MergeMode: ...}` -> `tw.CellMerging{Mode: ...}` in the
cataloger info table, for a staticcheck SA1019 deprecation
Note the deprecation text names `CellConfig.CellMerging.Mode`, but the field is
`CellConfig.Merging`; `CellMerging` is the type. That command path has no test
coverage, so the swap was checked by rendering the same hierarchical-merge table
both ways and diffing: byte identical.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore(lint): tune goconst for data tables
goconst got stricter in golangci-lint 2.13 and now counts strings in map
literals and struct fields, which lights up every lookup table in the repo. It
was already enabled, so this is main going red on a linter bump rather than
anything new in the tree: 101 findings, and the default threshold of 3 means a
vendor name appearing in three rows of a table reads as a magic constant.
Raising `min-occurrences` to 5 takes that to 14. The rest split by whether the
repeated string is data or an actual constant:
- `cpegenerate` is excluded by path. It maps artifact names to vendors,
products, and maven group ids, and repeats strings 35, 83, even 293 times, so
no threshold reaches it. Hoisting those would trade a readable table for
indirection.
- the two SBOM fixture builders in `format/internal/testutil` get a
function-level nolint each. The repeated names and versions are what the
golden files assert against.
- `mysql-cluster` and `php` were genuine repeated literals and are now
constants.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore(test): extract measureAlloc into a shared helper
The allocation-budget tests added during the unbounded allocation work grew a
copy of the same helper in every package they touched: four identical
`measureAlloc` definitions plus two more spelled out inline against
`runtime.MemStats`.
Now `testutils.MeasureAlloc`, at the repo root so `internal/spillbuf` can reach
it too. The doc comment carries the two things that were only recorded in some
of the copies: TotalAlloc is process-wide so callers must not use t.Parallel,
and a budget assertion only means something if the fixture would really blow
past it unguarded, so measure the unguarded path as well.
No behavior change, the body was identical in all six.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore(lint): add ruleguard rules for overflow and swallowed EOF
Two rules out of the recent unbounded allocation work (#5187, #5195, #5267,
#5291, #5292, #5293, #5294).
`noOverflowingBoundsCheck` flags `offset + size > limit` (and <, <=, >=) where
an operand is unsigned, neither addend is constant, and the operands are at
least 32 bits. Unsigned addition wraps, so an offset out of a file header can
carry the sum past the wrap point and pass a check it should have failed. The
fix is to subtract against the limit instead. Constant addends are excluded
because `pos+4 <= uint32(len(buf))` is the normal way to step a cursor, and the
32-bit floor drops nibbles widened for a comparison. Would have caught four
sites across #5195 and #5291.
`noSwallowedEOF` flags `return nil` / `return nil, nil` on an io.EOF check. The
caller gets a zero value it cannot tell from real data, and anything decoding a
fixed-size struct off it parses the padding as content. That is the shape behind
#5186.
Both decide from the expression alone, and writing the fix deletes the pattern,
so neither needs a suppression anywhere in the tree. That was the bar: rules for
the allocation and decompression classes were tried and dropped, because the
bound they look for lives in a preceding statement (or, for a decompressor
constructor, always downstream), so no correct implementation can silence them.
That would have meant ~15 permanent nolints on code that is already right. The
rules here that do work don't judge whether a bound exists, they route to a
wrapper that takes one: noDirectTempFiles, noDirectELFOpen.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: pin allocation budgets on the alpm, deb, and changelog bounds
The bounds added for #5292, #5293 and #5294 are asserted by the error they
return. That keeps passing if the bound is removed and something else rejects
the input later, which is the failure these were written for. Each of these
measures bytes instead, and proves the fixture is really a bomb first so a
fixture that quietly stops being one fails rather than passing.
- alpm mtree: 16MB of newlines, well inside the 64MB byte cap, costs 16.5GB
uncapped against 238MB with maxMtreeLines. That 16.5GB is the measurement the
line cap was added for and it was recorded only in the commit message. Note
what the cap actually buys: 300k entries still cost a few hundred MB, per
concurrent cataloger.
- deb members: 1.28GB against 392KB. Both arms read with io.ReadAll on purpose,
since the stream itself never holds much whatever the cap says; what the cap
protects is a consumer that buffers, and the real one is a tar reader building
entries.
- snap changelog: 1.28GB against 175KB. This one has no explicit cap at all. It
is held by bufio.Scanner refusing a token past MaxScanTokenSize, which is
incidental, so switching to io.ReadAll or a bufio.Reader to pick up long lines
would remove it silently.
Nothing added for the kernel module reader: its copy is already
io.Copy(tempFile, io.LimitReader(rc, max+1)), so peak disk is bounded by
construction and heap is streaming. A heap budget would assert the wrong
resource, and the three existing subtests cover the cap, at-cap fidelity, and
spill cleanup.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: pin allocation budgets on the ai and alpm byte caps
Follows the same pattern as the alpm line cap: measure bytes, and prove the
fixture is really a bomb first so one that quietly stops being a bomb fails
rather than passes.
- safetensors: 252MB against 2.6KB. The existing over-cap case deliberately
omits the body, since the guard rejects on the declared length before reading
anything. That is the right test for the guard but it would keep passing if
the check moved after the read, so this one delivers the bytes.
- gguf: 268MB against 168MB. copyHeader io.Copy's whatever it is handed, so
maxHeaderSize holds only because every caller wraps the reader first. This
pins that the limit is where the bound lives, so dropping the wrapper at a
call site fails here.
- alpm byte cap: 740MB against 132MB, the counterpart to the line cap budget.
Worth knowing for anyone writing the next one of these: a single enormous line
does not exercise the alpm byte cap. go-mtree runs its own bufio.Scanner and
refuses a token past 64KB, so that fixture fails as "token too long" before the
byte cap is consulted. The fixture here uses ordinary 1KB lines, enough of them
to pass 64MB while staying under the 300k line cap, so the byte cap is actually
the thing under test.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: assert bytes retained instead of allocated for gguf copyHeader
the bounded arm writes into a bytes.Buffer, whose doubling growth costs a
multiple of what it holds and lands on a different power of two depending on
the platform. it measured 168MB locally and 320MB on CI, past the budget.
buf.Len() states the same property deterministically.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* Update steroscope to include OCI platform metadata
Signed-off-by: Jason Paulos <jasonpaulos@users.noreply.github.com>
* chore(deps): point stereoscope at anchore/main
drops the fork `replace` directive and moves to a pseudo-version off stereoscope
main (`a22e5ae`, anchore/stereoscope#657). no tag carries it yet, so this should
move to a real version once one is cut.
`readImageMetadata` now fills os/architecture/variant for every provider, not
just oci. the docker tarball provider set neither before, so image metadata and
the image purl change for docker-archive input too:
- `"architecture": ""` -> `"amd64"`
- `"os": ""` -> `"linux"`
- `?arch=` -> `?arch=amd64`
the empty `arch=` qualifier was never right, but it is user-visible output.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: cover single-platform oci layouts
the pr only added assertions to the multiplatform fixture, which already
reported the right platform. the behavior that actually changed is single-image
layouts: they used to report empty os/arch and silently ignore `--platform`, and
now report what the image config says and fail with a platform mismatch.
also drops the non-linux skip on the default-platform test. the oci providers
default to `linux/<GOARCH>` regardless of host os, so it holds on macos too.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Jason Paulos <jasonpaulos@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
Alex Goodman
Pujitha PaladuguandPujitha Paladugu
hsdfat
Vignesh
idhyt
anchore-oss-update-botandanchore-oss-update-bot
akshaver
Christopher Angelo PhillipsandAlex Goodman
Peter McConnell
Silas ParkerandAlex Goodman
King StarandWill Murphy
Mayank Sekhar
Darragh McGurk
KR RavindraandWill Murphy
Antoni (Tony) JagodkaandAlex Goodman
Dev MandAlex Goodman
huuya
texasichandAlex Goodman
Yunare Maia
Arpit Jain
somaz
Jason PaulosandAlex Goodman