Compare commits

...
Author SHA1 Message Date
Alex Goodman ab69742d25 limit source-distribution refs to deb and rpm
- apk origins and alpm pkgbases are build recipes, not published source packages, and their names are usually also real binary packages, so a `source-distribution` ref there would point at a binary package. They keep relying on the `upstream` PURL qualifier.
- document why this is an external reference rather than `pedigree.ancestors`, with links to the CycloneDX spec discussion (CycloneDX/specification#612)

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-30 12:56:05 -04:00
Alex Goodman bf8ded3cf8 build source PURL from metadata and gate it to CycloneDX 1.6+
- the source PURL is built from package metadata instead of editing the binary PURL: only `distro` (plus `epoch` and `rpmmod` for rpm) is carried over, subpath is dropped, and the ref is skipped when the PURL type does not match the metadata ecosystem
- one emission rule for all ecosystems: only when the source name or version differs from the binary, so self-sourced rpm and alpm packages no longer get a redundant ref
- source RPM names must end in `.src.rpm` or `.nosrc.rpm`, and `nosrc` maps to `arch=nosrc`
- `source-distribution` refs are dropped when encoding below spec 1.6, since cyclonedx-go does not downgrade them (CycloneDX/cyclonedx-go#291)

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-30 12:56:05 -04:00
swadaptive d53a8d283f Adjust arch qualifier for source PURLs
Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>
2026-09-30 12:56:05 -04:00
swadaptive 13c6591cb2 Golden test fixtures
Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>
2026-09-30 12:56:05 -04:00
swadaptive 4ffde9f891 Add source-distribution external reference to components
Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>
2026-09-30 12:56:05 -04:00
swadaptive 06451adc43 Source distribution external reference utilities and test coverage
Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>
2026-09-30 12:56:05 -04:00
swadaptive ecf6b9eff5 Add tests for source packages in external references
Signed-off-by: swadaptive <268603322+swadaptive@users.noreply.github.com>
2026-09-30 12:56:05 -04:00
Mayank Sekhar d61c2cada3 fix(binary): correct traefik version detection on big-endian arches (#5281)
The traefik-binary classifier regex required a NUL byte or replacement
character immediately before the version string. On some architectures
(e.g. s390x), the real version string in the binary has no such prefix,
so the classifier instead matched an earlier, unrelated NUL-prefixed
numeric string elsewhere in the binary (e.g. '2.7.0' from an unrelated
printf-style format constant), producing an incorrect version.

This adds an additional pattern, tried first via MatchAny, that matches
version strings terminated by two NUL bytes without requiring a NUL
prefix. The original pattern remains as a fallback for older traefik
binaries (e.g. v1.x) that use single-NUL termination.

Fixes #4980

Signed-off-by: Mayank Sekhar <mayankshekharsingh@gmail.com>
2026-09-30 16:51:03 +00:00
0bbcff1ae3 Fix hang when scanning a Java resource adapter (.rar) file (#5348)
* fix: scan a Java resource adapter (.rar) given as a file source

The file source picks an extractor for the scan target from its file name
alone. A Java resource adapter archive (JCA) uses the .rar extension but is
a zip file, so it was handed to the RAR extractor. With rardecode v2.2.0 the
RAR extractor does not return on non-RAR input (an infinite loop in its
signature search), so `syft scan --from file adapter.rar` never finished.
The same bytes named .jar scan in under a second.

When the name says RAR but the content is a zip archive, leave the file
unextracted, the same way .jar, .war and .ear scan targets are handled. The
java cataloger already reads .rar files (anchore/syft#4136) and now finds the
resource adapter and its nested jars. Real RAR archives are still extracted.

Signed-off-by: Ismael Donmez <ismael.donmez@chainguard.dev>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore(deps): bump github.com/nwaples/rardecode/v2 from 2.2.0 to 2.4.1

rardecode v2.2.0 loops forever in its RAR signature search when the input
does not start with a RAR signature (fixed upstream in nwaples/rardecode
f501ac8, "fix infinite loop in findSig", released in v2.2.2). syft reaches
that search for any file source whose name contains ".rar", so a scan of a
self-extracting RAR, or of any other file with such a name, never finished.
With v2.4.1 a self-extracting RAR is extracted, and a file that is not a RAR
archive fails fast with "rardecode: RAR signature not found", the same way a
file named .zip that is not a zip archive fails today.

github.com/mholt/archives v0.1.5 (the latest release) still requires v2.2.0,
so the newer version is selected here.

Signed-off-by: Ismael Donmez <ismael.donmez@chainguard.dev>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: clarify the .rar zip log line and cover self-extracting RAR sources

- the debug line no longer calls every `.rar`-named zip a resource adapter
- a self-extracting RAR (stub before the signature) given as a file source is
  still extracted

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Ismael Donmez <ismael.donmez@chainguard.dev>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-30 15:54:36 +00:00
anchore-oss-update-botandanchore-oss-update-bot f21e439a63 chore(deps): update SPDX license list (#5315)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-09-30 15:50:47 +00:00
Will Murphy c511d509ef fix: match opensource.org license URLs across SPDX URL forms (#5361)
SPDX license list 3.29.0 rewrote every opensource.org seeAlso URL from
opensource.org/licenses/<ID> to opensource.org/license/<ID> (and e.g.
opensource.org/license/mit/ to opensource.org/license/MIT). Package metadata
in the wild (Maven POM <license><url>, npm, etc.) still uses the older forms,
so regenerating the list would stop LicenseByURL from resolving them.

Fall back to a normalized index of the opensource.org entries that ignores the
www. prefix, /licenses/ vs /license/, case, and a trailing slash. The index is
built from the generated urlToLicense map, so it works with both the current
and the updated license list without changes to the generator. URL forms that
would normalize to different licenses are left out rather than guessed.

Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
2026-09-30 15:07:12 +00:00
Alex Goodman fdeb78fc19 Speed up unit tests in CI (#5360)
* ci: drop gen-5 instance families from the test runner

m5a runners run the unit suite ~2.3x slower than m8id (15m vs 6.5m for `go test`), and when spot capacity for newer families dries up runs-on falls back to them. Keeping gen-6 through gen-8 leaves a wide spot pool without the slow tail.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: shrink decompression bomb fixtures

The bomb and heap-bound tests pushed 100MB-16GB through gzip and parsers, which under `-race` made them the slowest tests in the unit suite. Each now uses the smallest fixture that still separates the bounded path from the unbounded one.

- arch mtree caps are measured at a small cap and scaled to the shipped constants, so raising a cap still fails the budget
- snap and debian bombs expand to 32MB instead of 512MB (4x their 8MB budgets)
- golang ELF bombs still declare 256MB but deliver 64MB

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-30 10:58:11 -04:00
Alex Goodman 6ac7afb439 Fix missing file hashes when running on windows system (#5341)
* test: add windows unit test job and capture dir resolver path bug

- add a `Unit tests (Windows)` job running a new `make unit:windows` task, scoped to the packages that own host path handling (tests needing linux docker fixtures are skipped)
- `Unit tests` is now a gate job over the Linux and Windows unit jobs
- add a resolver test asserting that locations from `AllLocations` resolve again via `FilesByPath` and use posix paths. This is expected to fail on windows today, which is why file digests come back empty for `dir:` scans (#5325)
- fix windows-only test files that no longer compiled

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore: drop verbose output from integration tests

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: windows path handling outside of the file lookup bug

- chroot paths are always posix (`ToChrootPath` no longer returns backslash paths on windows), same for the directory source name derived from `base`
- `windows.FromPosix` no longer panics on input that isn't volume-encoded (e.g. relative or already native paths)
- skip tests on windows that rely on symlink fixtures, shell script fixtures, or emulate windows paths on a posix host
- disable `core.autocrlf` on the windows CI job so fixture digests match linux

This leaves only the `FilesByPath` lookup failures from #5325.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: skip posix-only fileresolver tests on windows

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: split contained paths on posix separators on windows

- `allContainedPaths` cleaned with the native separator but split on `/`, so a root reached through a symlink never had its ancestors indexed on windows
- index assertions in tests now key on the posix path the index actually stores
- skip symlink fixture tests and a posix-only test helper on windows

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore: revert unrelated go.sum churn and allContainedPaths change

- restore `go.sum` and `.make/go.sum` (accidentally populated by a local `-mod=mod` invocation)
- revert the `allContainedPaths` separator change: production passes native windows paths there, and splitting them on `/` produces drive-relative ancestors, so skip the posix-only test on windows instead
- annotate the remaining `filepath` usages that operate on native paths

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: search the file tree with posix paths on windows (#5325)

The directory and file resolvers key their file tree on posix paths (`/c/some/path`), but `FilesByPath`, `HasPath` and root-anchored `FilesByGlob` searched it with the native path from the chroot context (`C:\some\path`). On windows every lookup silently missed, so anything that resolves files by path found nothing. This is why file digests were empty for `dir:` scans.

Requests are now converted to posix before searching the tree.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-30 09:08:13 -04:00
Darragh McGurk b23e461731 Stop reading the mount table for every single-file source (#5257)
Signed-off-by: Darragh McGurk <120114728+dmcg310@users.noreply.github.com>
2026-09-29 21:44:47 +00:00
KR RavindraandWill Murphy 8c63a2403c fix: conan expat CPE (#5259)
Add libexpat_project vendor and libexpat product candidates for Conan
packages named expat, so generated CPEs match the NVD entries used for
libexpat vulnerabilities.

Signed-off-by: KR Ravindra <42912207+KR-Ravindra@users.noreply.github.com>
Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
2026-09-29 21:43:53 +00:00
Antoni (Tony) JagodkaandAlex Goodman e4ff7f5973 fix(spdx): write a name for the root package when the source has none (#5349)
* fix(spdx): write a name for the root package when the source has none

A source read from another SBOM (a purl list, a CycloneDX document
without metadata.component, an SPDX document without a DESCRIBES
relationship) can have no name. The SPDX encoder then wrote the document
root package with an empty PackageName, which SPDX requires. In
tag-value output the PackageName line is omitted, so the root package's
fields land in the previous section and syft cannot read its own output
back ("received unknown tag SPDXID in CreationInfo section").

Fall back to the document name for the root package, and make the
document name fall back to "unknown" for every source type rather than
only for unrecognized ones.

Signed-off-by: Tony <tjagodka@gmail.com>

* fix(spdx): drop syft's root package on decode for every SPDX version

- the root package name now comes from one fallback chain (source name, image repo name, source ID, document name), so the root purl no longer gets an empty name for a source decoded from CycloneDX
- decoding strips the root syft writes for an unknown source, so `syft convert` round trips stop adding an `unknown` package each pass
- for SPDX 2.1 and 2.2, which have no `primaryPackagePurpose`, the purpose is taken from the source type in syft's `DocumentRoot-<type>-` ID

**behavior change:** decoding a syft-written SPDX 2.1/2.2 document no longer returns the root as a package or its `CONTAINS` relationships, matching 2.3.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Tony <tjagodka@gmail.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 21:30:52 +00:00
Dev MandAlex Goodman 37d45de9ff fix(python): link uv.lock dependents to the locked version they name (#5351)
* fix(python): link uv.lock dependents to the locked version they name

A forked uv.lock can lock several versions of one package, and uv records the version on each dependency entry in that case. The parser dropped that field, so every pandas version depended on every numpy version. Carry the locked version through provides and requires as name@version (bare name stays for unique locks) and keep marker text unchanged.

Fixes #5340

Signed-off-by: Dev M <devtechedge@gmail.com>

* fix(python): pair uv.lock versions without widening public metadata

- the locked version on a uv dependency entry is only needed to pair relationships, and the relationships already carry it, so it stays on the internal toml struct instead of `pkg.PythonUvLockDependencyEntry`. The previous approach hid it from JSON but still hashed it into package IDs.
- dependency specs are now built from the raw lock entries at parse time rather than read back off package metadata
- optional-dependencies get the same version pairing, so an extra that names one locked version no longer links to all of them

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Dev M <devtechedge@gmail.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 17:23:22 -04:00
Pujitha PaladuguandPujitha Paladugu 26927eab02 fix: suppress misleading upstream CPE for PHP extension binaries (#5102)
Motivation:
syft's PHP interpreter cataloger reports each bundled PHP C extension
(openssl, ldap, sqlite3, etc.) as its own package, using the PHP
interpreter's own version number as the package version (since these
extensions are compiled into a specific PHP release and have no
independent version of their own). The classifier already attaches an
explicit CPE like cpe:2.3:a:php-openssl:php-openssl:8.5.7:... to make
clear this refers to the bundled PHP extension, not upstream OpenSSL.

However, that CPE was marked with Source: cpe.GeneratedSource. Syft's
package-finalization pipeline (internal/task/package_task_factory.go)
only skips its generic, name-based CPE generation when a package
already has an "authoritative" CPE (any Source other than
GeneratedSource) - so it also generated and appended a second CPE,
cpe:2.3:a:openssl:openssl:8.5.7:..., derived from the bare package
name. That second CPE is indistinguishable from a real standalone
OpenSSL/LDAP/SQLite installation and can cause vulnerability scanners
that consume this SBOM to falsely flag PHP hosts as running vulnerable
upstream libraries at version "8.5.7" (a PHP version, not a real
OpenSSL/LDAP/SQLite release). See anchore/syft#5014 for a full repro.

Approach:
Mark the classifier's explicit CPE with Source: cpe.DeclaredSource
instead of cpe.GeneratedSource, so it is treated as authoritative and
the extra name-derived CPE is no longer generated. This mirrors the
existing pattern in
syft/pkg/cataloger/java/parse_jvm_release.go's newJvmCpe, which uses
the same DeclaredSource + comment convention to mark a
syft-synthesized (not literally file-declared) CPE as trustworthy and
suppress further generation. It also brings this classifier in line
with its sibling php-cli/php-fpm/php-apache classifiers in the same
file, which already mark their CPEs as authoritative (via
NVDDictionaryLookupSource, not applicable here since php-<ext> is not
a real NVD dictionary entry).

This change only suppresses the extra generated CPE; it does not
change the package name, type, version, or PURL, and the existing
php-<ext>:php-<ext> CPE is unaffected.

Validation:
  go build ./...
  go test ./syft/pkg/cataloger/php/... ./internal/task/... ./syft/pkg/cataloger/internal/cpegenerate/...
All pass except the two pre-existing docker-image-fixture subtests of
Test_InterpreterCataloger, which fail in this sandbox only because the
`docker` binary isn't available (exec: "docker": executable file not
found in $PATH); confirmed via `git stash` that these two subtests
fail identically on main without this change, i.e. this is a sandbox
limitation and not a regression. Added a new unit test,
Test_getClassifier_declaredCPESource, that directly asserts the
classifier's CPE now carries Source: cpe.DeclaredSource.

Fixes #5014

Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
2026-09-29 17:15:36 -04:00
Alex Goodman 1dc16fbac4 fix: report scanner errors and raise the line cap in metadata parsers (#5353)
* fix: report scanner errors instead of returning a partial parse as complete

A line longer than `bufio.Scanner`'s 64KB buffer ends the scan with `ErrTooLong`, which only `Err()` reports. Parsers now return what they found along with the error (recorded as an unknown for the file), or log it where there is no error to return.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: allow lines up to 1MB in line-oriented package metadata parsers

Adds `internal.NewLineScanner`, a `bufio.Scanner` capped at 1MB per line rather than the 64KB default, the same cap the alpm parser already uses. Long single-line fields (descriptions, license text, dependency lists) now parse instead of ending the scan early.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: use io.ReadFull where a short read was silently accepted

- dotnet bundle 7-bit int decoding could read a zero-byte result as a zero byte
- wordpress plugin header parsing included the unread tail of its buffer

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: finish moving line scanners to the 1MB cap and checking scanner errors

- java manifest, pom.properties, Gemfile.lock, requirements.txt, homebrew formula, alpm and the purl/cpe decoders now use `internal.NewLineScanner`
- snap kernel changelog reports a scan error instead of silently dropping the base kernel package
- the attestation output reader keeps draining the pipe after a scan error so cosign can't block on a full pipe

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: stop logging the full setup.py line once per dependency match

The debug log ran once per regex match and included the whole line, so log volume grew with the square of line length. With 1MB lines that is hundreds of GB of debug output from a single crafted file.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(gentoo): don't dereference a nil location when a LICENSE scan fails

`extractLicenses` accepts a nil location, but the scan error log added earlier on this branch dereferenced it.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(gentoo): bound license group expansion on hostile license_groups

- dedup within a group uses a set instead of `slices.Contains`, which was quadratic in tokens per line and got much worse under the 1MB line cap
- each group is expanded once and memoized, so a chain of groups that reference each other repeatedly no longer expands exponentially
- repeated references to the same group within one line are skipped

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 16:52:58 -04:00
Alex Goodman d0e1e7be05 Harden cataloger parsers against truncated input (do not panic) (#5355)
* fix(javascript): don't panic on a yarn v1 lockfile line missing its value

A v1 `yarn.lock` dependency line without a version constraint, or a
`version`/`resolved`/`integrity` line without a value, indexed past the end of
the split line. Such dependencies are now skipped and empty fields left empty.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(alpine): don't panic on an apk db ACL or checksum line before its entry

An `M:`, `a:` or `Z:` line that comes before any directory or file entry in
`lib/apk/db/installed` indexed `files[-1]`. Such a line is now skipped.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(erlang): don't panic on a truncated rebar.lock or .app file

The term parser read past the end of the input when a file ended mid-term:
whitespace only, an unclosed list, a lone `<`, or a comment ending in `\r`.
These now return an "unexpected end of input" error.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(lua): don't panic on a truncated .rockspec

The rockspec parser read past the end of the input when a file ended mid-statement:
a lone `[`, an unclosed `[[...]]` value or `[...]` index, or a `local` assignment cut off
before its `=`, value, or line end. These now return an "unexpected end of input" error.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(swift): don't panic on a Podfile.lock pod without a version

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(gentoo): don't panic on a short obj line in portage CONTENTS

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(spdx): don't panic on an opam checksum without an algorithm

The opam cataloger stores checksums verbatim, so a malformed one reached the SPDX encoder and panicked there, outside the cataloger's recover. Checksums without an `=` are now skipped.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* refactor: use strings.Cut where a split was only indexed after a Contains check

Also guards `windows.FromPosix` against a path with no separator.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: guard untyped package metadata fields

- new `any`-typed metadata fields must be allowlisted with a reason
- allowlisted untyped fields are pushed through `SetID` and every encoder with
  non-finite, deeply nested and large values

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: fuzz every generic cataloger parser

One fuzz target over every glob-selected parser in the capabilities data, run
through its cataloger task. Each input must not panic, must finish in time and
must keep allocations proportional to its size. Seeds come from each parser's
testdata plus the string literals in its package.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: scan malformed inputs under memory and time limits

A directory of small malformed files (deep nesting, oversized declared counts,
truncation, type mismatches, non-finite floats) is scanned by the CLI with a
memory limit and an RSS ceiling. The scan must exit 0 and write an SBOM.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: stop a fuzz input once its live heap passes the allocation limit

The allocation check only ran after the parser returned, so an input on its way to exhausting memory ran the machine into swap first. The harness now polls the live heap while the parser runs and panics past the same limit, which the fuzz engine saves as a crasher.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: bound nesting depth and error context in the erlang and lua parsers

- a deeply nested rebar.lock or .rockspec overflowed the goroutine stack, which is fatal rather than a recoverable panic, so one file could take down the whole scan. Both parsers now stop at 1000 levels with an error.
- parse errors echoed whole input lines into the SBOM as unknowns, and lua appended another copy at every nesting level. `PrintError` now clips to a window around the error column, and lua decorates once at the top.
- the Podfile.lock error no longer echoes the pod entry.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: close ways the malformed-input checks could pass quietly

- a ctx-aware parser hitting the fuzz timeout now fails the input instead of racing the hard deadline
- the heap watchdog measures growth from a post-GC baseline rather than an absolute process-wide number
- a saved fuzz input naming a target that no longer exists fails instead of being skipped
- the CLI malformed-input scan fails on a recovered parser panic, not only on a non-zero exit

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 16:26:42 -04:00
Alex Goodman 57bd6a5f9c fix(cpe): bound CPE candidate generation for pathological package metadata (#5356)
* fix(cpe): cap sub-selection prefixes to bound candidate cross-product

a name with thousands of `-`/`_` separators produced one vendor and product candidate per segment, and the vendor x product cross-product made CPE generation roughly cubic in time and memory (8KB name: ~250s, 4.8GB RSS). `generateSubSelections` now stops after 10 prefixes. The full name is still a candidate, so real package names are unaffected.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(cpe): bound candidate count and field length before the cross-product

the prefix cap only covered one way to inflate the vendor x product cross-product in `FromPackageAttributes`. Java groupIDs add a vendor (and, with no artifactID, a product) per dot segment, and .NET executables add one of each per file, so a 1000-segment `Bundle-SymbolicName` still produced ~1M CPEs and several GB.

- vendor and product candidates are each capped at 64 (shortest first, so truncation is deterministic)
- names, versions, and candidates over 512 bytes are skipped, since every byte is paid once per CPE in key building and regex validation

real packages produce around a dozen candidates with names well under 100 bytes, so output for real inputs is unchanged. Also corrects the earlier comment: prefixes only become vendor candidates and the cost was quadratic, not cubic.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 16:01:21 -04:00
Will Murphy c60aabad37 test(golang): make the UPX fixture exercise loader padding (#5354)
UPX pads to a 4 byte boundary before it writes the loader stub. The
image-small-upx build happened to need no padding, so the cataloger
tests over it passed with or without the fix in #5347.

- build the fixture with -X main.Version=1.0.11, which needs 2 bytes of
  padding, so the existing cataloger tests fail without the fix
- add TestImageSmallUPXNeedsLoaderPadding, which fails if a change to the
  fixture's inputs means it no longer needs padding
- run the crafted loader test on both sides of the boundary, so a skip
  that always rounds up is caught too

Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
2026-09-29 19:00:42 +00:00
huuya c0072873c9 fix(golang): account for UPX's loader padding when unpacking Go binaries (#5347)
UPX pads its output to a 4 byte boundary before it writes the loader
stub, and l_lsize counts from that boundary. The block chain resumed at
the end of the last PT_LOAD extent plus l_lsize, so whenever the
compressed extents ended off a 4 byte boundary it landed 1 to 3 bytes
early, read a misaligned b_info, and stopped. The padding between the
segments was never filled in, the reconstruction ended at that first
hole, and .go.buildinfo in the data segment went with it, so the binary
contributed no packages.

The loader skip now rounds up to 4 before adding l_lsize, which matches
the funpad4 in UPX's own unpacker. The crafted fixture for the tail
extents put its stub straight after the head extent, the layout the bug
assumed, so it now pads the way UPX does.

Signed-off-by: huuyafwww <huuya.yamauchi@3-shake.com>
2026-09-29 14:25:04 -04:00
Alex Goodman 236ab25393 test(snap): bound the kernel changelog bomb fixture by ratio, not a fixed 1MB (#5352)
Same issue as #5350: go1.27's compress/flate encodes this 512MB fixture to just over 1MB, so the fixture sanity check tripped before the bound was exercised. The line bound itself is fine.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 11:19:17 -04:00
Alex Goodman fa6ea7b10c test(debian): bound the bomb fixture by ratio, not a fixed 1MB (#5350)
go1.27's compress/flate encodes 512MB of zeros to ~1MB (1052690 bytes) where go1.26 gives ~510KB, so the fixture sanity check tripped on newer toolchains before the bound was ever exercised. The decompression bound itself is fine (bounded read allocates ~300KB on both).

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 10:25:30 -04:00
Alex Goodman ae7843e17c fix(ai): handle malformed GGUF headers without panicking (#5345)
- pre-scan the GGUF metadata KV section and reject values gguf-parser-go would panic on, recurse on without bound, or overallocate for: wrong typed well-known keys, zero alignment, nested or oversized fully decoded arrays, an unreasonable `split.tensors.count`, and empty strings the parser would misread. Keys are matched the way the parser sees them. The KV count, fully read string bytes, and skipped string counts are bounded too. The file becomes an unknown instead of taking down the scan.
- `copyHeader` now bounds its own read to the max header size.
- non-finite floats in `header` (including inside arrays) become `"NaN"`, `"+Inf"` or `"-Inf"` so JSON encoding can't fail. `header` and `metadataHash` are otherwise unchanged.
- a `metadataHash` failure is now an error instead of an empty hash.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 14:03:31 +00:00
Alex Goodman a4d3aedf4e fix(ai): GGUF layer handling and safetensors parameter counts (#5343)
* fix(ai): keep merging GGUF layers when one fails to parse

a single unparseable layer in an OCI GGUF artifact skipped the merge, so the
named model lost its parts and nameless layers were dropped. The error is
still returned.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(ai): only send GGUF layers to the GGUF cataloger

the media type glob `application/vnd.docker.ai*` also matched config,
model.file, license and safetensors layers, so every safetensors OCI model
reported spurious "invalid GGUF magic number" unknowns. Match
`application/vnd.docker.ai.gguf.v3` exactly.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(ai): don't wrap safetensors parameter counts

- tensors whose shape product overflows uint64 are skipped instead of wrapping
- per-file and per-shard parameter totals saturate instead of wrapping
- parameter count and dominant dtype are computed in one pass
- GGUF licenses are created with the scan context, and drop no-op unknown checks

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* refactor(ai): derive safetensors model info from the header in one place

the file and OCI layer parsers built identical metadata from a parsed header.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(ai): keep the GGUF media type with the GGUF cataloger

also corrects stale comments in the GGUF merge processor.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 09:40:17 -04:00
Alex Goodman 50dbb33e66 fix(ai): bound model companion file reads and validate their contents (#5344)
Companion files next to AI models (`config.json`, `README.md`, and the OCI model.file, license and model config layers) are now read with named size limits, and are only used when they look like what they claim to be.

- JSON companions are read whole up to a limit and skipped when larger, rather than being fully buffered or parsed truncated. OCI companion layers are fetched one byte past the cap so oversize is detectable.
- README and license layers read a prefix. The frontmatter block is capped at 256 KB and list values at 32.
- dir and OCI scans share one 8 MB safetensors header cap. Directory scans used to allow 100 MB.
- `config.json` is only used when it has `architectures` or `_name_or_path`, and is looked up beside the model or in its parent directory, no longer all the way up to the scan root
- OCI model.file layers are processed in a stable order, so the chosen name no longer changes between runs
- README frontmatter:
  - a list-form `license:` is accepted and no longer drops `base_model`
  - `null` values are ignored
  - delimiters must be exact `---` lines
- model names from `_name_or_path` / `base_model`:
  - Windows paths are handled
  - `.`, `..` and `/` fall back to the next name source
  - names are capped at 256 characters

Behavior changes worth noting:
- a `config.json` two or more directories above the model no longer names it
- JSON `config.json` files without `architectures` or `_name_or_path` (e.g. `generation_config.json`, app configs) are ignored
- safetensors headers over 8 MB in directory scans are now rejected, matching OCI scans (real shard headers are well under 1 MB)

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 09:33:57 -04:00
Alex Goodman 10af8342c0 fix: don't fail the whole SBOM when a cataloger panics on one file (#5342)
* fix(generic): recover from parser and processor panics

- a panic while parsing one file becomes an unknown for that location instead of failing the whole SBOM
- a panic in a package processor is logged as a warning and the unprocessed results are kept
- stacks go to the trace log, not into the unknown text

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(unknown): keep errors that unwrap to nil

`sync.PanicError` with a non-error panic value (e.g. `panic("boom")`) unwraps to nil, and was being dropped entirely from the error graph instead of being kept as a leaf.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(task): don't put the stack trace in task panic errors

Task panics are still fatal, but the error now reads `panic in task "<name>": <value>` and the stack goes to the debug log.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test(generic): cover chained processors when a later one panics

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(file): recover from panics in the executable and file digest catalogers

A panic while reading one file is now recorded as an unknown for that file instead of failing the whole SBOM, the same as generic catalogers.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-29 09:21:03 -04:00
f7f3c6efc3 fix(ai): use go.yaml.in/yaml/v3 to parse model card frontmatter (#5328)
* fix(ai): use go.yaml.in/yaml/v3 to parse model card frontmatter

gopkg.in/yaml.v3 panics on some malformed merge-key payloads (for
example "<<:\r? 0:"), which crashed the gguf and safetensors catalogers
when a model README contained such frontmatter. Switch the Hugging Face
frontmatter parser to the maintained go.yaml.in/yaml/v3 fork, which is
already used by the other catalogers and returns an error instead of
panicking. The existing error path logs and skips the frontmatter.

Fixes #5327

Signed-off-by: tk1475 <118763009+tk1475@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore: finish moving off gopkg.in/yaml.v3

- move the snap catalogers and `internal/capabilities` to `go.yaml.in/yaml/v3`. The snap parsers decode yaml from scanned sources and hit the same merge-key panic as #5327
- enable `depguard` with a deny rule for `gopkg.in/yaml.v3` so it can't be reintroduced

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: tk1475 <118763009+tk1475@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-28 19:22:45 +00:00
Antoni (Tony) Jagodka a8d4fb845f fix(spdx): don't panic on a root file without a supplier or null entries (#5326)
The package supplier is optional in SPDX, but fileSource dereferenced it
without the nil check containerSource already has, so a valid SPDX 2.3
document whose described package has primaryPackagePurpose FILE and no
supplier crashed syft convert and grype sbom: input.

A JSON null in files, a package's files or externalRefs (or an SPDX 2.1
package the version converter cannot map) decodes to a nil pointer that
the conversion then dereferences. Drop nil entries once at the start of
ToSyftModel instead of guarding each loop.

Signed-off-by: Tony <tjagodka@gmail.com>
2026-09-28 15:01:13 -04:00
Sverrir Sigurdsson ff45b9ea63 fix: release package collection lock in Ubuntu ESM detection (#5336)
Signed-off-by: Sverrir Sigurðsson <svesig@gmail.com>
2026-09-28 17:10:55 +00:00
anchore-oss-update-botandanchore-oss-update-bot fb55856e33 chore(deps): update CPE dictionary index (#5312)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-09-28 01:50:42 +00:00
anchore-oss-update-botandanchore-oss-update-bot a76b5fa36b chore(deps): update tool versions (#5337)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com>
2026-09-27 10:26:41 +00:00
dependabot[bot] 643bf2c24d chore(deps): bump github.com/containerd/containerd/v2 (#5335)
Bumps [github.com/containerd/containerd/v2](https://github.com/containerd/containerd) from 2.3.5 to 2.3.6.
- [Release notes](https://github.com/containerd/containerd/releases)
- [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md)
- [Commits](https://github.com/containerd/containerd/compare/v2.3.5...v2.3.6)

---
updated-dependencies:
- dependency-name: github.com/containerd/containerd/v2
  dependency-version: 2.3.6
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-25 19:53:30 +00:00
dependabot[bot] c5baac0e94 chore(deps): bump modernc.org/sqlite from 1.58.0 to 1.59.0 (#5330)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.58.0 to 1.59.0.
- [Changelog](https://gitlab.com/cznic/sqlite/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.58.0...v1.59.0)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-version: 1.59.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-25 13:35:26 +00:00
dependabot[bot] 19341fd0a2 chore(deps): bump github.com/olekukonko/tablewriter from 1.1.4 to 1.1.5 (#5331)
Bumps [github.com/olekukonko/tablewriter](https://github.com/olekukonko/tablewriter) from 1.1.4 to 1.1.5.
- [Release notes](https://github.com/olekukonko/tablewriter/releases)
- [Commits](https://github.com/olekukonko/tablewriter/compare/v1.1.4...v1.1.5)

---
updated-dependencies:
- dependency-name: github.com/olekukonko/tablewriter
  dependency-version: 1.1.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-25 13:29:56 +00:00
dependabot[bot] ec8f035584 chore(deps): bump docker/setup-buildx-action from 4.3.0 to 4.4.1 (#5329)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 4.3.0 to 4.4.1.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/37fe631027851001ddb9b187196cc803df7f5f0e...f87e5991a6d7451dcb8d9637bfbc97413f497069)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: 4.4.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-25 13:29:46 +00:00
dependabot[bot] 9d39701d50 chore(deps): bump github.com/magiconair/properties (#5333)
Bumps [github.com/magiconair/properties](https://github.com/magiconair/properties) from 1.18.11 to 1.18.12.
- [Release notes](https://github.com/magiconair/properties/releases)
- [Commits](https://github.com/magiconair/properties/compare/v1.18.11...v1.18.12)

---
updated-dependencies:
- dependency-name: github.com/magiconair/properties
  dependency-version: 1.18.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-25 13:29:27 +00:00
dependabot[bot] 82718d36ca chore(deps): bump github.com/dustin/go-humanize from 1.0.1 to 1.1.0 (#5334)
Bumps [github.com/dustin/go-humanize](https://github.com/dustin/go-humanize) from 1.0.1 to 1.1.0.
- [Commits](https://github.com/dustin/go-humanize/compare/v1.0.1...v1.1.0)

---
updated-dependencies:
- dependency-name: github.com/dustin/go-humanize
  dependency-version: 1.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-25 13:29:03 +00:00
dependabot[bot] cc1bb1f28c chore(deps): bump github.com/hashicorp/hcl/v2 from 2.24.0 to 2.25.0 (#5332)
Bumps [github.com/hashicorp/hcl/v2](https://github.com/hashicorp/hcl) from 2.24.0 to 2.25.0.
- [Release notes](https://github.com/hashicorp/hcl/releases)
- [Changelog](https://github.com/hashicorp/hcl/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/hcl/compare/v2.24.0...v2.25.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/hcl/v2
  dependency-version: 2.25.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-25 13:28:33 +00:00
Alex Goodman 14e96fcbf8 fix(ui): render TUI log lines on their own rows, with colors and a small header (#5324)
- log lines shown under the TUI were glued onto the last UI row, and the last row could get erased (and overwritten by the report) when the TUI exits. Fixed in `bubbly` v0.2.2
- log colors were lost because the logger latched its TTY check on the first entry, which went to the TUI footer buffer. `go-logger` v0.2.0 checks per entry and supports `TerminalWriter`, which the footer writer now implements (`NO_COLOR` is still honored)
- log lines under the TUI get a dim `Logs (non-fatal, see -v for more):` title with a branch pointing at the first entry, so it's clear what the trailing lines are

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-24 10:40:52 -04:00
Alex Goodman af508550e6 test: pin multiplatform OCI fixture by digest (#5323)
Docker Hub rebuilt and re-pushed `busybox:1.38.0`, which changed the per-platform config digests and broke `TestMultiPlatformOCIImageSelection` on uncached runs.

- pin the fixture to the busybox 1.38.0 index digest so future re-pushes can't shift it
- update the expected per-platform digests to match

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-24 10:05:59 -04:00
texasichandAlex Goodman 409ac78f62 fix(java): resolve Groovy 4+ group IDs to org.apache.groovy (#5313)
* fix(java): resolve Groovy 4+ group IDs to org.apache.groovy

Groovy 4.0 moved from the Codehaus coordinates (org.codehaus.groovy) to the
Apache Software Foundation (org.apache.groovy). JARs that ship no pom
metadata fall through to the static artifact-to-group map, which carries the
3.x-era org.codehaus.groovy entries for the whole groovy family, so 4.x
artifacts were attributed to an obsolete group even though their manifest
identifies the Apache vendor.

Where the static map attributes an artifact to the historical group, a
major-version migration resolves groovy 4.x and newer to org.apache.groovy,
while 3.x and earlier keep their historical group. The Eclipse-hosted
groovy-eclipse-* artifacts, which share the prefix but kept their Codehaus
coordinates, are excluded.

Fixes #5311

Signed-off-by: texasich <101962694+texasich@users.noreply.github.com>

* refactor(java): share version-aware groovy group ID lookup with CPE generation

- move the artifact-to-group lookup behind `cpegenerate.ArtifactIDToGroupID(artifactID, version)` so the PURL and CPE paths agree; the exported mutable map is now unexported
- version-dependent entries are expressed as a `groupIDMigration{before, after, atMajor}` (groovy moved to `org.apache.groovy` at 4.0); unconditional entries, including `groovy-eclipse-*` and apache-only 4.0+ modules, stay in the plain map
- add apache-only modules (`groovy-ginq`, `groovy-contracts`, `groovy-callsite`, ...) that otherwise took the group from their per-module `Automatic-Module-Name`

This changes CPEs for pom-less groovy 4+ jars (all real ones): vendors are now `apache` (e.g. `cpe:2.3:a:apache:groovy:4.0.33`) instead of `codehaus`, matching the PURL, jars with pom metadata, and NVD.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: texasich <101962694+texasich@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-23 17:20:28 +00:00
Yunare Maia 41870fec7f fix: skip docker:// references in github-actions PURL generation (#5302)
closes #5299: docker:// references are OCI images, not GitHub repos.
Returning empty string produces a package with no PURL, matching the
treatment for local composite actions.

Signed-off-by: Yunare Maia <yunare@gmail.com>
2026-09-23 14:13:22 +00:00
Arpit Jain 97fd939355 fix(haskell): do not slice past the end of a stack.yaml rev pin (#5310)
An extra-dep can be pinned as "@sha256:<hash>,<size>" or as "@rev:<n>". The hash
slice assumed the first form, so a rev pin left the end index at -1 and the
parser panicked with slice bounds out of range. A rev pin carries no package
hash, so only the sha256 form now yields one.

Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
2026-09-23 14:07:57 +00:00
somaz 3f4105a657 fix(elf): avoid a panic on a truncated dynamic section (#5321)
Signed-off-by: somaz <genius5711@gmail.com>
2026-09-23 14:02:12 +00:00
dependabot[bot] b49f0172f6 chore(deps): bump golang.org/x/net from 0.58.0 to 0.59.0 (#5309)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.58.0 to 0.59.0.
- [Commits](https://github.com/golang/net/compare/v0.58.0...v0.59.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.59.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-18 13:30:41 +00:00
dependabot[bot] 46fd83817f chore(deps): bump golang.org/x/tools from 0.49.0 to 0.50.0 (#5307)
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.49.0 to 0.50.0.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.49.0...v0.50.0)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-version: 0.50.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-18 13:30:01 +00:00
dependabot[bot] a2ed08dfdd chore(deps): bump zizmorcore/zizmor-action from 0.6.3 to 0.6.4 (#5306)
Bumps [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) from 0.6.3 to 0.6.4.
- [Release notes](https://github.com/zizmorcore/zizmor-action/releases)
- [Commits](https://github.com/zizmorcore/zizmor-action/compare/70fb788f84895a7701f5643d103d587e460b5c99...cc914d7f3750a2d13d75c7f184a1060aa0e9d482)

---
updated-dependencies:
- dependency-name: zizmorcore/zizmor-action
  dependency-version: 0.6.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-18 13:29:50 +00:00
dependabot[bot] 34e7b0df98 chore(deps): bump anchore/go-make/.github/actions/setup (#5308)
Bumps [anchore/go-make/.github/actions/setup](https://github.com/anchore/go-make) from 0.8.0 to 0.8.1.
- [Release notes](https://github.com/anchore/go-make/releases)
- [Commits](https://github.com/anchore/go-make/compare/430e2175bb166bfd138ef75466adbcc8ddab875f...fa5421b3bf24c9d18f11a736c5eb4172f2b7d683)

---
updated-dependencies:
- dependency-name: anchore/go-make/.github/actions/setup
  dependency-version: 0.8.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-18 13:29:37 +00:00
dependabot[bot] 3b41acb0ff chore(deps): bump go.opentelemetry.io/otel/sdk from 1.44.0 to 1.45.0 (#5301)
Bumps [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go) from 1.44.0 to 1.45.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.44.0...v1.45.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/sdk
  dependency-version: 1.45.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-18 00:39:32 +00:00
Alex Goodman 3325493831 chore: pin the bounds from the unbounded allocation work (#5298)
* build(deps): bump go-make to v0.8.1

Picks up golangci-lint 2.13.2 (built with go1.27) in place of 2.11.4 (go1.26).
The old build panics outright on a go1.27 GOROOT with "file requires newer Go
version go1.27", and an older mise-shim build fails the quieter way: it loses
type info and reports zero findings with exit 0, so a ruleguard run comes back
green without having evaluated anything.

Also folds in what the newer linter flags:

- `reflect.Ptr` -> `reflect.Pointer` in three files, applied by --fix
- `tw.CellFormatting{MergeMode: ...}` -> `tw.CellMerging{Mode: ...}` in the
  cataloger info table, for a staticcheck SA1019 deprecation

Note the deprecation text names `CellConfig.CellMerging.Mode`, but the field is
`CellConfig.Merging`; `CellMerging` is the type. That command path has no test
coverage, so the swap was checked by rendering the same hierarchical-merge table
both ways and diffing: byte identical.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(lint): tune goconst for data tables

goconst got stricter in golangci-lint 2.13 and now counts strings in map
literals and struct fields, which lights up every lookup table in the repo. It
was already enabled, so this is main going red on a linter bump rather than
anything new in the tree: 101 findings, and the default threshold of 3 means a
vendor name appearing in three rows of a table reads as a magic constant.

Raising `min-occurrences` to 5 takes that to 14. The rest split by whether the
repeated string is data or an actual constant:

- `cpegenerate` is excluded by path. It maps artifact names to vendors,
  products, and maven group ids, and repeats strings 35, 83, even 293 times, so
  no threshold reaches it. Hoisting those would trade a readable table for
  indirection.
- the two SBOM fixture builders in `format/internal/testutil` get a
  function-level nolint each. The repeated names and versions are what the
  golden files assert against.
- `mysql-cluster` and `php` were genuine repeated literals and are now
  constants.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(test): extract measureAlloc into a shared helper

The allocation-budget tests added during the unbounded allocation work grew a
copy of the same helper in every package they touched: four identical
`measureAlloc` definitions plus two more spelled out inline against
`runtime.MemStats`.

Now `testutils.MeasureAlloc`, at the repo root so `internal/spillbuf` can reach
it too. The doc comment carries the two things that were only recorded in some
of the copies: TotalAlloc is process-wide so callers must not use t.Parallel,
and a budget assertion only means something if the fixture would really blow
past it unguarded, so measure the unguarded path as well.

No behavior change, the body was identical in all six.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(lint): add ruleguard rules for overflow and swallowed EOF

Two rules out of the recent unbounded allocation work (#5187, #5195, #5267,
#5291, #5292, #5293, #5294).

`noOverflowingBoundsCheck` flags `offset + size > limit` (and <, <=, >=) where
an operand is unsigned, neither addend is constant, and the operands are at
least 32 bits. Unsigned addition wraps, so an offset out of a file header can
carry the sum past the wrap point and pass a check it should have failed. The
fix is to subtract against the limit instead. Constant addends are excluded
because `pos+4 <= uint32(len(buf))` is the normal way to step a cursor, and the
32-bit floor drops nibbles widened for a comparison. Would have caught four
sites across #5195 and #5291.

`noSwallowedEOF` flags `return nil` / `return nil, nil` on an io.EOF check. The
caller gets a zero value it cannot tell from real data, and anything decoding a
fixed-size struct off it parses the padding as content. That is the shape behind
#5186.

Both decide from the expression alone, and writing the fix deletes the pattern,
so neither needs a suppression anywhere in the tree. That was the bar: rules for
the allocation and decompression classes were tried and dropped, because the
bound they look for lives in a preceding statement (or, for a decompressor
constructor, always downstream), so no correct implementation can silence them.
That would have meant ~15 permanent nolints on code that is already right. The
rules here that do work don't judge whether a bound exists, they route to a
wrapper that takes one: noDirectTempFiles, noDirectELFOpen.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: pin allocation budgets on the alpm, deb, and changelog bounds

The bounds added for #5292, #5293 and #5294 are asserted by the error they
return. That keeps passing if the bound is removed and something else rejects
the input later, which is the failure these were written for. Each of these
measures bytes instead, and proves the fixture is really a bomb first so a
fixture that quietly stops being one fails rather than passing.

- alpm mtree: 16MB of newlines, well inside the 64MB byte cap, costs 16.5GB
  uncapped against 238MB with maxMtreeLines. That 16.5GB is the measurement the
  line cap was added for and it was recorded only in the commit message. Note
  what the cap actually buys: 300k entries still cost a few hundred MB, per
  concurrent cataloger.
- deb members: 1.28GB against 392KB. Both arms read with io.ReadAll on purpose,
  since the stream itself never holds much whatever the cap says; what the cap
  protects is a consumer that buffers, and the real one is a tar reader building
  entries.
- snap changelog: 1.28GB against 175KB. This one has no explicit cap at all. It
  is held by bufio.Scanner refusing a token past MaxScanTokenSize, which is
  incidental, so switching to io.ReadAll or a bufio.Reader to pick up long lines
  would remove it silently.

Nothing added for the kernel module reader: its copy is already
io.Copy(tempFile, io.LimitReader(rc, max+1)), so peak disk is bounded by
construction and heap is streaming. A heap budget would assert the wrong
resource, and the three existing subtests cover the cap, at-cap fidelity, and
spill cleanup.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: pin allocation budgets on the ai and alpm byte caps

Follows the same pattern as the alpm line cap: measure bytes, and prove the
fixture is really a bomb first so one that quietly stops being a bomb fails
rather than passes.

- safetensors: 252MB against 2.6KB. The existing over-cap case deliberately
  omits the body, since the guard rejects on the declared length before reading
  anything. That is the right test for the guard but it would keep passing if
  the check moved after the read, so this one delivers the bytes.
- gguf: 268MB against 168MB. copyHeader io.Copy's whatever it is handed, so
  maxHeaderSize holds only because every caller wraps the reader first. This
  pins that the limit is where the bound lives, so dropping the wrapper at a
  call site fails here.
- alpm byte cap: 740MB against 132MB, the counterpart to the line cap budget.

Worth knowing for anyone writing the next one of these: a single enormous line
does not exercise the alpm byte cap. go-mtree runs its own bufio.Scanner and
refuses a token past 64KB, so that fixture fails as "token too long" before the
byte cap is consulted. The fixture here uses ordinary 1KB lines, enough of them
to pass 64MB while staying under the 300k line cap, so the byte cap is actually
the thing under test.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: assert bytes retained instead of allocated for gguf copyHeader

the bounded arm writes into a bytes.Buffer, whose doubling growth costs a
multiple of what it holds and lands on a different power of two depending on
the platform. it measured 168MB locally and 320MB on CI, past the budget.
buf.Len() states the same property deterministically.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-09-17 14:11:16 -04:00
216 changed files with 6813 additions and 1370 deletions
+1 -1
View File
@@ -16,7 +16,7 @@ tools:
# used to upload test fixture cache
- name: oras
version:
want: v1.3.3
want: v1.3.4
method: github-release
with:
repo: oras-project/oras
+7
View File
@@ -0,0 +1,7 @@
# check out text files with lf on every platform (regardless of core.autocrlf), letting git detect binary files
* text=auto eol=lf
# fixtures that must keep their crlf line endings byte for byte
syft/linux/testdata/unprintable -text
syft/pkg/cataloger/bitnami/testdata/no-rel/opt/bitnami/redis/.spdx-redis.spdx -text
syft/pkg/cataloger/java/testdata/jar-metadata/**/META-INF/MANIFEST.MF -text
+1 -1
View File
@@ -30,7 +30,7 @@ runs:
using: "composite"
steps:
- name: Setup go + go-make tooling
uses: anchore/go-make/.github/actions/setup@430e2175bb166bfd138ef75466adbcc8ddab875f # v0.8.0
uses: anchore/go-make/.github/actions/setup@fa5421b3bf24c9d18f11a736c5eb4172f2b7d683 # v0.8.1
with:
go-version: ${{ inputs.go-version }}
cache-key-prefix: ${{ inputs.cache-key-prefix }}
+1 -1
View File
@@ -86,7 +86,7 @@ jobs:
# dockers_v2 builds multi-platform manifests in a single buildx invocation, which requires a
# docker-container driver builder (the default builder on the runner cannot build multi-platform).
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Hub
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f #v4.6.0
@@ -25,7 +25,7 @@ jobs:
persist-credentials: false
- name: "Run zizmor"
uses: zizmorcore/zizmor-action@70fb788f84895a7701f5643d103d587e460b5c99 # v0.6.3
uses: zizmorcore/zizmor-action@cc914d7f3750a2d13d75c7f184a1060aa0e9d482 # v0.6.4
with:
# there is a pass/fail gate as a repo ruleset (if there is no ruleset configured then the action will pass by default)
advanced-security: true
+29 -1
View File
@@ -27,7 +27,7 @@ jobs:
# enough; a 4 vCPU instance can't pull much more than that anyway.
# note: s3-cache intentionally omitted -- PR runs are untrusted and must not write to the
# shared cache backend that the trusted release workflow reads from (cache poisoning).
runs-on: &test-runner "runs-on=${{ github.run_id }}/cpu=4+8/ram=16+32/family=m5+m6+m7+m8+c5+c6+c7+c8/spot=price-capacity-optimized/volume=100gb"
runs-on: &test-runner "runs-on=${{ github.run_id }}/cpu=4+8/ram=16+32/family=m6+m7+m8+c6+c7+c8/spot=price-capacity-optimized/volume=100gb"
permissions:
contents: read
steps:
@@ -49,6 +49,13 @@ jobs:
Unit-Test:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
name: "Unit tests"
runs-on: ubuntu-latest
needs: [Unit-Test-Linux, Unit-Test-Windows]
steps:
- run: echo All unit tests passed!
Unit-Test-Linux:
name: "Unit tests (Linux)"
runs-on: *test-runner
permissions:
contents: read
@@ -68,6 +75,27 @@ jobs:
- name: Check for capability drift
run: make check-capability-drift
Unit-Test-Windows:
name: "Unit tests (Windows)"
runs-on: windows-latest
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 #v7.0.1
with:
persist-credentials: false
# go-make's setup only (not ./.github/actions/bootstrap): no apt, binny tools, or fixture cache are needed
- name: Setup go + go-make tooling
uses: anchore/go-make/.github/actions/setup@fa5421b3bf24c9d18f11a736c5eb4172f2b7d683 # v0.8.1
with:
# keep in sync with the go-version default in ./.github/actions/bootstrap
go-version: "1.26.2"
# the full unit suite leans on linux docker fixtures, so this runs the windows-relevant subset
- name: Run unit tests
run: make unit:windows
Integration-Test:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
name: "Integration tests"
+18 -1
View File
@@ -7,6 +7,7 @@ linters:
- asciicheck
- bodyclose
- copyloopvar
- depguard
- dogsled
- dupl
- errcheck
@@ -29,9 +30,20 @@ linters:
- unused
- whitespace
settings:
depguard:
rules:
yaml:
deny:
# gopkg.in/yaml.v3 is unmaintained and panics on some malformed input (see #5327)
- pkg: gopkg.in/yaml.v3
desc: use go.yaml.in/yaml/v3 instead
funlen:
lines: 70
statements: 50
goconst:
# the default of 3 turns every repeated literal in a lookup table or a test fixture into a finding.
# 5 keeps the ones worth hoisting without flagging data that reads better written out.
min-occurrences: 5
gocritic:
enabled-checks:
- deferInLoop
@@ -54,6 +66,12 @@ linters:
- linters:
- revive
text: "var-naming: avoid package names that conflict"
# cpegenerate is almost entirely lookup tables mapping artifact names to vendors, products, and
# maven group ids. The repeated strings are the data itself, so hoisting them to constants would
# replace a readable table with indirection for no gain.
- linters:
- goconst
path: syft/pkg/cataloger/internal/cpegenerate/
paths:
- third_party$
- builtin$
@@ -61,7 +79,6 @@ linters:
# do not enable...
# - deadcode # The owner seems to have abandoned the linter. Replaced by "unused".
# - depguard # We don't have a configuration for this yet
# - goprintffuncname # does not catch all cases and there are exceptions
# - nakedret # does not catch all cases and should not fail a build
# - gochecknoglobals
+3 -3
View File
@@ -2,11 +2,11 @@ module github.com/anchore/syft/.make
go 1.25.8
require github.com/anchore/go-make v0.8.0
require github.com/anchore/go-make v0.8.1
require (
github.com/bmatcuk/doublestar/v4 v4.10.0 // indirect
github.com/goccy/go-yaml v1.19.2 // indirect
golang.org/x/mod v0.37.0 // indirect
golang.org/x/sys v0.46.0 // indirect
golang.org/x/mod v0.40.0 // indirect
golang.org/x/sys v0.47.0 // indirect
)
+6 -6
View File
@@ -1,10 +1,10 @@
github.com/anchore/go-make v0.8.0 h1:3XAD4WNHIbqY4kwFyCQJ7kKiYLzSzjB0UwzDVwRqlUY=
github.com/anchore/go-make v0.8.0/go.mod h1:4M6TnArb5w693VyWsgr5dCWrk2BLNu/ed4JUcsrzS34=
github.com/anchore/go-make v0.8.1 h1:GAhPXR+9qVQhFB1pfVlHFOzxAt3dTDTUIA7Tp7nx9tg=
github.com/anchore/go-make v0.8.1/go.mod h1:xALbVsPsDoJnJxdkjPlxOLZV7GbAdT5Pv3gd0kP1Lz0=
github.com/bmatcuk/doublestar/v4 v4.10.0 h1:zU9WiOla1YA122oLM6i4EXvGW62DvKZVxIe6TYWexEs=
github.com/bmatcuk/doublestar/v4 v4.10.0/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/mod v0.40.0 h1:hUv+3cXcdRHz08UmSiOob7sadHig73uo5bkXxQ/tvUs=
golang.org/x/mod v0.40.0/go.mod h1:0/weTWkPWGBikyTWAX3dkjVztMmBA5hM0DH6BElSupE=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
+19 -1
View File
@@ -1,6 +1,7 @@
package main
import (
"os"
"path/filepath"
"runtime"
"strconv"
@@ -52,7 +53,7 @@ func main() {
raceFlag = " -race"
}
Run(
"go test -count=1 -timeout=30m -v"+raceFlag+" ./cmd/syft/internal/test/integration/...",
"go test -count=1 -timeout=30m"+raceFlag+" ./cmd/syft/internal/test/integration/...",
run.Env("GODEBUG", "dontfreezetheworld=1"),
)
},
@@ -96,6 +97,23 @@ func main() {
},
},
// windows unit tests: deliberately not hooked into "test" (no RunsOn). The full unit suite leans on
// linux docker fixture images and posix-only assumptions, so this covers the packages that own
// host path handling (where windows bugs live) and skips the tests that need docker image fixtures.
// ponytail: curated package list, widen it as more of the suite is made windows-safe.
Task{
Name: "unit:windows",
Description: "run the windows-relevant subset of unit tests",
Run: func() {
Run("go test -count=1"+
` -skip "Image|Squash|AllLayers|MixFileTypes|GivenCoordinates"`+
" ./syft/internal/windows/... ./syft/internal/fileresolver/... ./syft/source/directorysource/... ./syft/source/filesource/..."+
" ./syft/file ./syft/file/cataloger/filedigest/... ./syft/file/cataloger/filecontent/...",
run.Stdout(os.Stderr),
)
},
},
// default validation pipeline (replaces Taskfile `default`/`pr-validations`/`validations`).
Task{
Name: "default",
+5
View File
@@ -158,6 +158,11 @@ func (l *backgroundLineReader) read(reader io.Reader, stage *progress.Stage) {
l.lock.Unlock()
}
if err := s.Err(); err != nil {
log.WithFields("error", err).Debug("unable to read attestation output")
// keep draining so the writer never blocks on a full pipe
_, _ = io.Copy(io.Discard, reader)
}
}
func (l backgroundLineReader) Lines() []string {
+2 -2
View File
@@ -349,8 +349,8 @@ func renderCatalogerInfoTable(_ *capabilities.Document, catalogers []capabilitie
})),
tablewriter.WithConfig(tablewriter.Config{
Row: tw.CellConfig{
Formatting: tw.CellFormatting{MergeMode: tw.MergeHierarchical},
Alignment: tw.CellAlignment{Global: tw.AlignLeft},
Merging: tw.CellMerging{Mode: tw.MergeHierarchical},
Alignment: tw.CellAlignment{Global: tw.AlignLeft},
},
}),
)
@@ -16,19 +16,21 @@ import (
"github.com/anchore/syft/syft/source"
)
// multiplatformImage is busybox:1.38.0 pinned by index digest, since docker hub periodically rebuilds and
// re-pushes official image tags, which changes the per-platform digests asserted below.
const multiplatformImage = "docker.io/library/busybox@sha256:fd7dc98638c8e305f4dc34e979f1c0fdfdcaeb0fbf8fcff77ae834b6da3d7e6e"
// TestMultiPlatformOCIImageSelection verifies that syft can load a single platform out of a
// multi-platform OCI image on disk (both --from oci-dir and --from oci-archive) by selecting the
// image that matches the requested platform.
func TestMultiPlatformOCIImageSelection(t *testing.T) {
remoteImage := "docker.io/library/busybox:1.38.0"
// Per-platform image config digests within the multi-platform index, obtained from the OCI layout
// (see anchore/stereoscope integration tests: TestPlatformSelectionWithOciLocalSources).
expectedDigest := map[string]string{
"arm64": "sha256:e0e8b3cbfed68a90084781e2962f9c0deead51c5a3f11a488eef0283a4284bc2",
"s390x": "sha256:0cf160e720a8e4f20883b72276a6eaded83b79539f3f1d39e35a3336154b9960",
"amd64": "sha256:c6348fa86ba0fb2108c9334f5fe913ddc6d853313e655891f133a0127c30099f",
"ppc64le": "sha256:b144fc0e06537d07956cde340b878a81a717e394edb736d3110858a12a6635cb",
"arm64": "sha256:d2482869a6b838d3b4c9cad0c8085c06277909482771eb525d298fc3a7d07927",
"s390x": "sha256:e55a0fa55032cceac2d983f4b4a25b28e42303b46a2f1915fc0fa45051e6ca70",
"amd64": "sha256:aaef90e065235eb0b2d9938be85d1b81add2902cf89affe2cdfa19be33476ac0",
"ppc64le": "sha256:8158829b015d1c5285d63d54d37e95998ef0af4177e3f416678decbcaff3170c",
}
// syft --from source tag -> stereoscope OCI source used to prepare the local fixture
@@ -39,7 +41,7 @@ func TestMultiPlatformOCIImageSelection(t *testing.T) {
for from, imageSource := range sources {
t.Run(from, func(t *testing.T) {
localPath := imagetest.PrepareMultiplatformFixtureImage(t, imageSource, remoteImage)
localPath := imagetest.PrepareMultiplatformFixtureImage(t, imageSource, multiplatformImage)
for _, arch := range []string{"amd64", "arm64", "s390x", "ppc64le"} {
t.Run(fmt.Sprintf("linux/%s", arch), func(t *testing.T) {
platform, err := image.NewPlatform("linux/" + arch)
@@ -73,8 +75,6 @@ func TestMultiPlatformOCIImageSelection(t *testing.T) {
// TestMultiPlatformOCIImageSelection_UnavailablePlatform verifies that requesting a platform not present
// in the multi-platform OCI image results in an error rather than silently selecting the wrong image.
func TestMultiPlatformOCIImageSelection_UnavailablePlatform(t *testing.T) {
remoteImage := "docker.io/library/busybox:1.38.0"
// windows/amd64 is not present in this linux-only multi-platform image
platform, err := image.NewPlatform("windows/amd64")
require.NoError(t, err)
@@ -86,7 +86,7 @@ func TestMultiPlatformOCIImageSelection_UnavailablePlatform(t *testing.T) {
for from, imageSource := range sources {
t.Run(from, func(t *testing.T) {
localPath := imagetest.PrepareMultiplatformFixtureImage(t, imageSource, remoteImage)
localPath := imagetest.PrepareMultiplatformFixtureImage(t, imageSource, multiplatformImage)
_, err := syft.GetSource(
context.Background(),
@@ -101,8 +101,6 @@ func TestMultiPlatformOCIImageSelection_UnavailablePlatform(t *testing.T) {
// TestMultiPlatformOCIImageSelection_DefaultPlatform verifies that not specifying a platform results
// in the current platform being selected.
func TestMultiPlatformOCIImageSelection_DefaultPlatform(t *testing.T) {
remoteImage := "docker.io/library/busybox:1.38.0"
sources := map[string]image.Source{
"oci-dir": image.OciDirectorySource,
"oci-archive": image.OciTarballSource,
@@ -110,7 +108,7 @@ func TestMultiPlatformOCIImageSelection_DefaultPlatform(t *testing.T) {
for from, imageSource := range sources {
t.Run(from, func(t *testing.T) {
localPath := imagetest.PrepareMultiplatformFixtureImage(t, imageSource, remoteImage)
localPath := imagetest.PrepareMultiplatformFixtureImage(t, imageSource, multiplatformImage)
src, err := syft.GetSource(
context.Background(),
+45 -1
View File
@@ -1,6 +1,7 @@
package ui
import (
"bytes"
"fmt"
"io"
"os"
@@ -8,6 +9,7 @@ import (
"time"
tea "github.com/charmbracelet/bubbletea"
"github.com/charmbracelet/lipgloss"
"github.com/wagoodman/go-partybus"
"github.com/anchore/bubbly"
@@ -52,7 +54,7 @@ func New(out io.Writer, quiet bool, handlers ...bubbly.EventHandler) *UI {
func (m *UI) Setup(subscription partybus.Unsubscribable) error {
// we still want to collect log messages, however, we also the logger shouldn't write to the screen directly
if logWrapper, ok := log.Get().(logger.Controller); ok {
logWrapper.SetOutput(m.frame.(*frame.Frame).Footer())
logWrapper.SetOutput(&logFooter{w: m.frame.(*frame.Frame).Footer()})
}
m.subscription = subscription
@@ -187,6 +189,48 @@ func (m UI) View() string {
return m.frame.View()
}
var (
logTitleStyle = lipgloss.NewStyle().Foreground(lipgloss.Color("8")) // dark grey
// the first log line gets a branch pointing at it from the title, the rest are indented to match
logBranch = logTitleStyle.Render("└─▶") + " "
logIndent = " "
)
// logFooter decorates log entries drawn below the TUI with a one-time title (written when the first entry arrives)
// and a branch from the title to the first log line. Writes are serialized by the logger.
type logFooter struct {
w io.Writer
started bool
}
func (l *logFooter) Write(p []byte) (int, error) {
var buf bytes.Buffer
first := !l.started
if first {
buf.WriteString(logTitleStyle.Render("Logs (non-fatal, see -v for more):") + "\n")
l.started = true
}
for i, line := range bytes.SplitAfter(p, []byte("\n")) {
if len(line) == 0 {
continue
}
if first && i == 0 {
buf.WriteString(logBranch)
} else {
buf.WriteString(logIndent)
}
buf.Write(line)
}
if _, err := l.w.Write(buf.Bytes()); err != nil {
return 0, err
}
return len(p), nil
}
// IsTerminal marks the footer as terminal-bound (it is drawn by the TUI) so the logger keeps colors. NO_COLOR is
// still honored by the formatter.
func (*logFooter) IsTerminal() bool { return true }
func runWithTimeout(timeout time.Duration, fn func() error) (err error) {
c := make(chan struct{}, 1)
go func() {
+26 -26
View File
@@ -1,6 +1,6 @@
module github.com/anchore/syft
go 1.26.3
go 1.26.8
require (
github.com/BurntSushi/toml v1.6.0
@@ -11,12 +11,12 @@ require (
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
github.com/acobaugh/osrelease v0.1.0
github.com/adrg/xdg v0.5.3
github.com/anchore/bubbly v0.2.1
github.com/anchore/bubbly v0.2.2
github.com/anchore/clio v0.1.1
github.com/anchore/fangs v0.1.1
github.com/anchore/go-collections v0.1.1
github.com/anchore/go-homedir v0.1.1
github.com/anchore/go-logger v0.1.1
github.com/anchore/go-logger v0.2.0
github.com/anchore/go-macholibre v0.1.1
github.com/anchore/go-rpmdb v0.2.0
github.com/anchore/go-sync v0.1.2
@@ -36,7 +36,7 @@ require (
github.com/deitch/magic v0.0.0-20230404182410-1ff89d7342da
github.com/diskfs/go-diskfs v1.9.4
github.com/distribution/reference v0.6.0
github.com/dustin/go-humanize v1.0.1
github.com/dustin/go-humanize v1.1.0
github.com/elliotchance/phpserialize v1.4.0
github.com/facebookincubator/nvdtools v0.1.5
github.com/github/go-spdx/v2 v2.7.0
@@ -56,18 +56,18 @@ require (
github.com/hashicorp/go-cleanhttp v0.5.2
github.com/hashicorp/go-getter v1.8.9
github.com/hashicorp/go-multierror v1.1.1
github.com/hashicorp/hcl/v2 v2.24.0
github.com/hashicorp/hcl/v2 v2.25.0
github.com/iancoleman/strcase v0.3.0
github.com/invopop/jsonschema v0.14.0
github.com/jedib0t/go-pretty/v6 v6.8.3
github.com/jinzhu/copier v0.4.0
github.com/kastenhq/goversion v0.0.0-20230811215019-93b2f8823953
github.com/klauspost/compress v1.20.0
github.com/magiconair/properties v1.18.11
github.com/magiconair/properties v1.18.12
github.com/mholt/archives v0.1.5
github.com/moby/sys/mountinfo v0.7.2
github.com/nix-community/go-nix v0.0.0-20250101154619-4bdde671e0a1
github.com/olekukonko/tablewriter v1.1.4
github.com/olekukonko/tablewriter v1.1.5
github.com/opencontainers/go-digest v1.0.0
github.com/pelletier/go-toml v1.9.5
github.com/quasilyte/go-ruleguard/dsl v0.3.23
@@ -94,11 +94,10 @@ require (
go.yaml.in/yaml/v3 v3.0.5
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f
golang.org/x/mod v0.41.0
golang.org/x/net v0.58.0
golang.org/x/net v0.59.0
golang.org/x/time v0.16.0
golang.org/x/tools v0.49.0
gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v1.58.0
golang.org/x/tools v0.50.0
modernc.org/sqlite v1.59.0
)
require (
@@ -131,6 +130,7 @@ require (
github.com/anchore/go-struct-converter v0.2.0-rc2 // indirect
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
github.com/apparentlymart/go-textseg/v17 v17.0.1 // indirect
github.com/aquasecurity/go-version v0.0.1 // indirect
github.com/atotto/clipboard v0.1.4 // indirect
github.com/aws/aws-sdk-go-v2 v1.44.0 // indirect
@@ -170,7 +170,7 @@ require (
github.com/cncf/xds/go v0.0.0-20260202195803-dba9d589def2 // indirect
github.com/containerd/cgroups/v3 v3.1.3 // indirect
github.com/containerd/containerd/api v1.11.1 // indirect
github.com/containerd/containerd/v2 v2.3.5 // indirect
github.com/containerd/containerd/v2 v2.3.6 // indirect
github.com/containerd/continuity v0.5.0 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
@@ -198,7 +198,7 @@ require (
github.com/gkampitakis/ciinfo v0.3.4 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-jose/go-jose/v4 v4.1.4 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/logr v1.4.4 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-restruct/restruct v1.2.0-alpha // indirect
github.com/gogo/protobuf v1.3.2 // indirect
@@ -248,7 +248,7 @@ require (
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.16.0 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/nwaples/rardecode/v2 v2.2.0 // indirect
github.com/nwaples/rardecode/v2 v2.4.1 // indirect
github.com/olekukonko/cat v0.0.0-20250911104152-50322a0618f6 // indirect
github.com/olekukonko/errors v1.2.0 // indirect
github.com/olekukonko/ll v0.1.6 // indirect
@@ -293,25 +293,25 @@ require (
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
github.com/zclconf/go-cty v1.16.3 // indirect
github.com/zclconf/go-cty v1.19.0 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.44.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.68.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 // indirect
go.opentelemetry.io/otel v1.44.0 // indirect
go.opentelemetry.io/otel/metric v1.44.0 // indirect
go.opentelemetry.io/otel/sdk v1.44.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.44.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.opentelemetry.io/otel v1.45.0 // indirect
go.opentelemetry.io/otel/metric v1.45.0 // indirect
go.opentelemetry.io/otel/sdk v1.45.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.45.0 // indirect
go.opentelemetry.io/otel/trace v1.45.0 // indirect
go.yaml.in/yaml/v4 v4.0.0-rc.2 // indirect
go4.org v0.0.0-20230225012048-214862532bf5 // indirect
golang.org/x/crypto v0.56.0 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/oauth2 v0.36.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/term v0.45.0 // indirect
golang.org/x/text v0.41.0 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/term v0.46.0 // indirect
golang.org/x/text v0.42.0 // indirect
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
gonum.org/v1/gonum v0.17.0 // indirect
google.golang.org/api v0.294.0 // indirect
@@ -321,7 +321,7 @@ require (
google.golang.org/grpc v1.83.2 // indirect
google.golang.org/protobuf v1.36.12 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
modernc.org/libc v1.75.6 // indirect
modernc.org/libc v1.75.7 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.12.1 // indirect
)
+52 -50
View File
@@ -120,8 +120,8 @@ github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuy
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/anchore/bubbly v0.2.1 h1:b3ZNBefY8ks5DI4SQ7BXSbBIsN6eoaTD6s1su7NuCBo=
github.com/anchore/bubbly v0.2.1/go.mod h1:o6a9aH3/fCn+mb5g1T1tBbQZ4hGR0yO5/uD1d2Vtago=
github.com/anchore/bubbly v0.2.2 h1:53JHRlpaNZjMcUYdz7n1U7+kOhy0zsJv9gpGEXvTcz8=
github.com/anchore/bubbly v0.2.2/go.mod h1:M4twoFkczlfuNdU8QV7oQjVJwPbxdUWRMfKydcLba0U=
github.com/anchore/clio v0.1.1 h1:9bkYnaFbpT0J3G0qSHyQqa/lpK5zV0C9Gtgh82H/jj0=
github.com/anchore/clio v0.1.1/go.mod h1:CFPIopJlakJbBpKtAL/EPpj4W9hgBO7PfFc8N2FAtlE=
github.com/anchore/fangs v0.1.1 h1:NBTHocmWM2/0gNhebpoE/wEB0Z0nxh2lESww2o/Uf5M=
@@ -130,8 +130,8 @@ github.com/anchore/go-collections v0.1.1 h1:kGlmSehs+ctTBZr+98F2+taHeolHy+XDx1Nc
github.com/anchore/go-collections v0.1.1/go.mod h1:zgDbPSNDpfU47tVo7aW+2suG7+vXB1Sq1qtDhRJknQ0=
github.com/anchore/go-homedir v0.1.1 h1:KpaO5BPrVT7cVMcEZ9KfFfEi0iuo3DtagPGl0C744X0=
github.com/anchore/go-homedir v0.1.1/go.mod h1:9B0DGhbmMAJVGGEbrlb+PkORM5eDFCEOtZ3xQ22qaLA=
github.com/anchore/go-logger v0.1.1 h1:HPBNmQVBdYliwaFVUL7oJ3UvSlaoXAMj/ZagAIYaCpA=
github.com/anchore/go-logger v0.1.1/go.mod h1:ekWuh5BkZVwyXnyEd4fwL29N4pNLssgRsqh2+kFN/b8=
github.com/anchore/go-logger v0.2.0 h1:V8Gazz+F8xwWEWz+whqt8/5163IkO4XrAie+1qbK10k=
github.com/anchore/go-logger v0.2.0/go.mod h1:omqLWBwT/B5/aTwIH6pl5WBylBzeH3BGHUTS6OFeFvY=
github.com/anchore/go-lzo v0.1.1 h1:IwL/fvkdtlIrYIXck6WxZ3nb8WjjHziYYmGxlooyOnM=
github.com/anchore/go-lzo v0.1.1/go.mod h1:3kLx0bve2oN1iDwgM1U5zGku1Tfbdb0No5qp1eL1fIk=
github.com/anchore/go-macholibre v0.1.1 h1:sTBHi3S1WVIOCamc6NyT8Sn9mBmD30DCTfjfVm4k6/M=
@@ -156,6 +156,8 @@ github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuW
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/apparentlymart/go-textseg/v15 v15.0.0 h1:uYvfpb3DyLSCGWnctWKGj857c6ew1u1fNQOlOtuGxQY=
github.com/apparentlymart/go-textseg/v15 v15.0.0/go.mod h1:K8XmNZdhEBkdlyDdvbmmsvpAG721bKi0joRfFdHIWJ4=
github.com/apparentlymart/go-textseg/v17 v17.0.1 h1:bpMXRgQ5cEoRNuQke1a80/Nl6w3G5eoIbWo9f3gXkAs=
github.com/apparentlymart/go-textseg/v17 v17.0.1/go.mod h1:fa8X4jgGeevslICIY6LcdjkSecWnXmYd9Lk34z/VxZs=
github.com/aquasecurity/go-pep440-version v0.0.1 h1:8VKKQtH2aV61+0hovZS3T//rUF+6GDn18paFTVS0h0M=
github.com/aquasecurity/go-pep440-version v0.0.1/go.mod h1:3naPe+Bp6wi3n4l5iBFCZgS0JG8vY6FT0H4NGhFJ+i4=
github.com/aquasecurity/go-version v0.0.1 h1:4cNl516agK0TCn5F7mmYN+xVs1E3S45LkgZk3cbaW2E=
@@ -294,8 +296,8 @@ github.com/containerd/cgroups/v3 v3.1.3 h1:eUNflyMddm18+yrDmZPn3jI7C5hJ9ahABE5q6
github.com/containerd/cgroups/v3 v3.1.3/go.mod h1:PKZ2AcWmSBsY/tJUVhtS/rluX0b1uq1GmPO1ElCmbOw=
github.com/containerd/containerd/api v1.11.1 h1:h8nfoDW9+fNsC/9TwiAHj8B1GzXKtR4eFtkhi/X5RLU=
github.com/containerd/containerd/api v1.11.1/go.mod h1:CaQFRu+N1MtbgL6JDOJLUB1hCKESU1lD6MuTJhgtdlw=
github.com/containerd/containerd/v2 v2.3.5 h1:9MYlI81gUcOZ0WsCkSMtvOU7rTR3hqAoa2eCzhoLlkA=
github.com/containerd/containerd/v2 v2.3.5/go.mod h1:RXDyLPaI3zoO7dFdAW9/54W4cix+z3A6larieufC9mg=
github.com/containerd/containerd/v2 v2.3.6 h1:huoqZXaDW1x1kgPvFC4rMwxv92vUJnqgq7I5b1rSddM=
github.com/containerd/containerd/v2 v2.3.6/go.mod h1:qaUSWKk9EchqbudSbhv1MxI8dXKGYHLpZtMQdkEkVZ4=
github.com/containerd/continuity v0.5.0 h1:7a85HZpCSs+1Zps0Ee3DPSuAWY+0SJM1JNM51nlEVDg=
github.com/containerd/continuity v0.5.0/go.mod h1:/lNJvtJKUQStBzpVQ1+rasXO1LAWtUQssk28EZvJ3nE=
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
@@ -348,8 +350,8 @@ github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDD
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 h1:2tV76y6Q9BB+NEBasnqvs7e49aEBFI8ejC89PSnWH+4=
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707/go.mod h1:qssHWj60/X5sZFNxpG4HBPDHVqxNm4DfnCKgrbZOT+s=
github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdfkVLjJ8T6VcRQv3SXugXy999NBtR9aFY=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/dustin/go-humanize v1.1.0 h1:dbKTrvD0klcbBV/h4AWJdMuZogJACoMlvWIWZ5b2xWg=
github.com/dustin/go-humanize v1.1.0/go.mod h1:hc1CvRkJMsgxqjmjMQF3QNRAZBwY8AXBAzKYoSX9sFI=
github.com/elazarl/goproxy v1.7.2 h1:Y2o6urb7Eule09PjlhQRGNsqRfPmYI3KKQLFpCAV3+o=
github.com/elazarl/goproxy v1.7.2/go.mod h1:82vkLNir0ALaW14Rc399OTTjyNREgmdL2cVoIbS6XaE=
github.com/elliotchance/phpserialize v1.4.0 h1:cAp/9+KSnEbUC8oYCE32n2n84BeW8HOY3HMDI8hG2OY=
@@ -432,8 +434,8 @@ github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/logr v1.4.4 h1:tG4xh9yMsRCAiodLVTxyrkzSZ9+o0L1Kg/+cPVcbP/8=
github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-restruct/restruct v1.2.0-alpha h1:2Lp474S/9660+SJjpVxoKuWX09JsXHSrdV7Nv3/gkvc=
@@ -595,8 +597,8 @@ github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uG
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
github.com/hashicorp/hcl/v2 v2.24.0 h1:2QJdZ454DSsYGoaE6QheQZjtKZSUs9Nh2izTWiwQxvE=
github.com/hashicorp/hcl/v2 v2.24.0/go.mod h1:oGoO1FIQYfn/AgyOhlg9qLC6/nOJPX3qGbkZpYAcqfM=
github.com/hashicorp/hcl/v2 v2.25.0 h1:HmmQVYRny4MaBo4b20TjmL46wyuUxpnMWkPZ4+NTbWk=
github.com/hashicorp/hcl/v2 v2.25.0/go.mod h1:vR+FKETxoZAmRlHgFfKmuqivj+C4Izm/c66XkmZ3r7M=
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
github.com/hashicorp/mdns v1.0.1/go.mod h1:4gW7WsVCke5TE7EPeYliwHlRUyBtfCwuFwuMg2DmyNY=
github.com/hashicorp/mdns v1.0.4/go.mod h1:mtBihi+LeNXGtG8L9dX59gAEa12BDtBQSp4v/YAJqrc=
@@ -668,8 +670,8 @@ github.com/lucasb-eyer/go-colorful v1.3.0 h1:2/yBRLdWBZKrf7gB40FoiKfAWYQ0lqNcbuQ
github.com/lucasb-eyer/go-colorful v1.3.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/lyft/protoc-gen-star v0.5.3/go.mod h1:V0xaHgaf5oCCqmcxYcWiDfTiKsZsRc87/1qhoTACD8w=
github.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60=
github.com/magiconair/properties v1.18.11 h1:j5ozYZl0zCjG7ahMDH0GWIobOvvUzT0BdAguG0ViKy0=
github.com/magiconair/properties v1.18.11/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/magiconair/properties v1.18.12 h1:sT9zQpvTB3B4gzrX0tmZNTEaGyg8Zw55MFYRE32Mr9I=
github.com/magiconair/properties v1.18.12/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/maruel/natural v1.3.0 h1:VsmCsBmEyrR46RomtgHs5hbKADGRVtliHTyCOLFBpsg=
github.com/maruel/natural v1.3.0/go.mod h1:v+Rfd79xlw1AgVBjbO0BEQmptqb5HvL/k9GRHB7ZKEg=
@@ -756,16 +758,16 @@ github.com/nix-community/go-nix v0.0.0-20250101154619-4bdde671e0a1 h1:kpt9ZfKcm+
github.com/nix-community/go-nix v0.0.0-20250101154619-4bdde671e0a1/go.mod h1:qgCw4bBKZX8qMgGeEZzGFVT3notl42dBjNqO2jut0M0=
github.com/nsf/jsondiff v0.0.0-20210926074059-1e845ec5d249 h1:NHrXEjTNQY7P0Zfx1aMrNhpgxHmow66XQtm0aQLY0AE=
github.com/nsf/jsondiff v0.0.0-20210926074059-1e845ec5d249/go.mod h1:mpRZBD8SJ55OIICQ3iWH0Yz3cjzA61JdqMLoWXeB2+8=
github.com/nwaples/rardecode/v2 v2.2.0 h1:4ufPGHiNe1rYJxYfehALLjup4Ls3ck42CWwjKiOqu0A=
github.com/nwaples/rardecode/v2 v2.2.0/go.mod h1:7uz379lSxPe6j9nvzxUZ+n7mnJNgjsRNb6IbvGVHRmw=
github.com/nwaples/rardecode/v2 v2.4.1 h1:F7zNW2LdAuuBThHWXQaiFUGVD/sef299NfWSB1nHAl4=
github.com/nwaples/rardecode/v2 v2.4.1/go.mod h1:7uz379lSxPe6j9nvzxUZ+n7mnJNgjsRNb6IbvGVHRmw=
github.com/olekukonko/cat v0.0.0-20250911104152-50322a0618f6 h1:zrbMGy9YXpIeTnGj4EljqMiZsIcE09mmF8XsD5AYOJc=
github.com/olekukonko/cat v0.0.0-20250911104152-50322a0618f6/go.mod h1:rEKTHC9roVVicUIfZK7DYrdIoM0EOr8mK1Hj5s3JjH0=
github.com/olekukonko/errors v1.2.0 h1:10Zcn4GeV59t/EGqJc8fUjtFT/FuUh5bTMzZ1XwmCRo=
github.com/olekukonko/errors v1.2.0/go.mod h1:ppzxA5jBKcO1vIpCXQ9ZqgDh8iwODz6OXIGKU8r5m4Y=
github.com/olekukonko/ll v0.1.6 h1:lGVTHO+Qc4Qm+fce/2h2m5y9LvqaW+DCN7xW9hsU3uA=
github.com/olekukonko/ll v0.1.6/go.mod h1:NVUmjBb/aCtUpjKk75BhWrOlARz3dqsM+OtszpY4o88=
github.com/olekukonko/tablewriter v1.1.4 h1:ORUMI3dXbMnRlRggJX3+q7OzQFDdvgbN9nVWj1drm6I=
github.com/olekukonko/tablewriter v1.1.4/go.mod h1:+kedxuyTtgoZLwif3P1Em4hARJs+mVnzKxmsCL/C5RY=
github.com/olekukonko/tablewriter v1.1.5 h1:4LoZSfMySpMQY3PT8RWJsJeuEuMIoo9xGRgvmqjg6IQ=
github.com/olekukonko/tablewriter v1.1.5/go.mod h1:+kedxuyTtgoZLwif3P1Em4hARJs+mVnzKxmsCL/C5RY=
github.com/onsi/gomega v1.34.1 h1:EUMJIKUjM8sKjYbtxQI9A4z2o+rruxnzNvpknOXie6k=
github.com/onsi/gomega v1.34.1/go.mod h1:kU1QgUvBDLXBJq618Xvm2LUX6rSAfRaFRTcdOeDLwwY=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
@@ -976,8 +978,8 @@ github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/zclconf/go-cty v1.16.3 h1:osr++gw2T61A8KVYHoQiFbFd1Lh3JOCXc/jFLJXKTxk=
github.com/zclconf/go-cty v1.16.3/go.mod h1:VvMs5i0vgZdhYawQNq5kePSpLAoz8u1xvZgrPIxfnZE=
github.com/zclconf/go-cty v1.19.0 h1:IV8WdqYZc2c5rLX9bEoLNXKojBAp0MZPBHMIrCoa/s4=
github.com/zclconf/go-cty v1.19.0/go.mod h1:12W89jGn3JCOIQi7infWr9m80rOkb5RNYJqXMZcN4c8=
github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940 h1:4r45xpDWB6ZMSMNJFMOjqrGHynW3DIBuR2H9j0ug+Mo=
github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940/go.mod h1:CmBdvvj3nqzfzJ6nTCIwDTPZ56aVGvDrmztiO5g3qrM=
github.com/zyedidia/generic v1.2.2-0.20230320175451-4410d2372cb1 h1:V+UsotZpAVvfj3X/LMoEytoLzSiP6Lg0F7wdVyu9gGg=
@@ -1002,20 +1004,20 @@ go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.6
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.68.0/go.mod h1:Sje3i3MjSPKTSPvVWCaL8ugBzJwik3u4smCjUeuupqg=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 h1:CqXxU8VOmDefoh0+ztfGaymYbhdB/tT3zs79QaZTNGY=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0/go.mod h1:BuhAPThV8PBHBvg8ZzZ/Ok3idOdhWIodywz2xEcRbJo=
go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
go.opentelemetry.io/otel v1.45.0 h1:pdrWmLHofpubmArBv1LgFSv1Z0Ie/ppdZzu+kUN5EeU=
go.opentelemetry.io/otel v1.45.0/go.mod h1:XZxIqPapzEYnhNSScF5DIqXhm/rYi0FzCe2XddAwZfQ=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.44.0 h1:hqxVTu/GtBF+vJ8d1fzW7fRxZFvgoDjWcxwwCaFDYpU=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.44.0/go.mod h1:z5fVEF4X5v0ESvlJqBrrFlBVoj5EQuefZpzsu7R+x5Q=
go.opentelemetry.io/otel/metric v1.44.0 h1:1w0gILTcHdr3YI+ixLyjemwrVnsMURbTZFrSYCdDdmc=
go.opentelemetry.io/otel/metric v1.44.0/go.mod h1:8O7hanEPBNgEMmybD3s2VBKcgWOCsA6tzHBPODAiquo=
go.opentelemetry.io/otel/metric/x v0.66.0 h1:YkCrx1zLOChi9ZcZ6euupOcsgzbVlec7D/xoEU1+cTA=
go.opentelemetry.io/otel/metric/x v0.66.0/go.mod h1:d1+BDj9t96do0/1LoU1ayfCv79ZgNE41qbhBvnMOBZk=
go.opentelemetry.io/otel/sdk v1.44.0 h1:nHYwb9lK+fJPU/dnT6s7W7Z8itMWyqrnVfbheVYrZ58=
go.opentelemetry.io/otel/sdk v1.44.0/go.mod h1:Osuydd3Se74nqjAKxid74N5eC+jfEqfTegHRnq58oK0=
go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRks6si09iEfI=
go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
go.opentelemetry.io/otel/metric v1.45.0 h1:7Eg1uH7CJ5cXv9is6tnBe1FI6rj1nwUdbFypRm3br/M=
go.opentelemetry.io/otel/metric v1.45.0/go.mod h1:HAPbm1nd3p1PmFH7v2dR+6BjXxw+Lq4a2+pndMAm08s=
go.opentelemetry.io/otel/metric/x v0.67.0 h1:PcicCNZFkZ4bXfSooXdo3WN7RBOVOtjVdo1wD358Uns=
go.opentelemetry.io/otel/metric/x v0.67.0/go.mod h1:FBjCWZe6wgcqxcMtjdGiClDKXb2YxxXii0CXftE4QtI=
go.opentelemetry.io/otel/sdk v1.45.0 h1:4VVSMgQ83dUgW2aoX5f6JgLvHwIvzcuLnF9lUdCSpCw=
go.opentelemetry.io/otel/sdk v1.45.0/go.mod h1:Sr40LgXV7DsKMMJMKOhUWOgMWTfAaqvm2kF0g7ilwuA=
go.opentelemetry.io/otel/sdk/metric v1.45.0 h1:oVFszMfyj1Am6s24Vtc7wBb8BKLcwepJjNEYILuiE3o=
go.opentelemetry.io/otel/sdk/metric v1.45.0/go.mod h1:vUWUxDZvu1WVRj8JA8S0AdhsPrZoDpA2DdZauIh4mDA=
go.opentelemetry.io/otel/trace v1.45.0 h1:l/mP6Uv7oNO7/TblbhpbgMidxhq1uO/rPsikOyVhxag=
go.opentelemetry.io/otel/trace v1.45.0/go.mod h1:qoJJA2xNMnxRrdISU/kLtfUH2wNeQbiv+jhs/CxI8bc=
go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
@@ -1041,8 +1043,8 @@ golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.56.0 h1:GUh5Ii4J5jtcseSMiRqr1jXCNHoxjeV9Fmekc2oLy6Y=
golang.org/x/crypto v0.56.0/go.mod h1:OMW5y6CY9l38uPLmxU6l6pwcXp1obtLo3e6gT7gQR2I=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -1128,8 +1130,8 @@ golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qx
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
golang.org/x/net v0.59.0 h1:5zfYln+w5XCxwrnMMJPufRgNoXEaGxl0wo5GqPXyues=
golang.org/x/net v0.59.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -1161,8 +1163,8 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -1237,13 +1239,13 @@ golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0=
golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=
golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE=
golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -1254,8 +1256,8 @@ golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -1316,8 +1318,8 @@ golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
golang.org/x/tools v0.50.0 h1:c2ifzfcuY7L90lZ2aKd8S4K2NpASF08SZx9ZuJkHmSU=
golang.org/x/tools v0.50.0/go.mod h1:7ulVMw3831Mwi5EZD6RomGyffr4VFjuNYXf2BbCEAV0=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -1526,8 +1528,8 @@ modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.75.6 h1:yKk8qo+Di4gkmvRboK8ocCqH22FiUCR6jRy2OwtCRus=
modernc.org/libc v1.75.6/go.mod h1:bO5o2ztHxBb2rjz0PgdHN0sSMw57CgxGFLZ3Qd/QpVQ=
modernc.org/libc v1.75.7 h1:o3DTP9/0p9pKmY2WCKQaySW6wIiZhNM7wc2lUoyhfew=
modernc.org/libc v1.75.7/go.mod h1:bO5o2ztHxBb2rjz0PgdHN0sSMw57CgxGFLZ3Qd/QpVQ=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
@@ -1536,8 +1538,8 @@ modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.58.0 h1:38u40/bwkfM7f0Myhosl+SEMltSDxnGdQf8o6Kjmys0=
modernc.org/sqlite v1.58.0/go.mod h1:rsD2CckafgObKC4DhBlGBf+RiHxkc3hINGt1Xw32tVY=
modernc.org/sqlite v1.59.0 h1:X1es1GpqBlS/5T+vbM4HLUdaa8OtQx468DF2vrx+38A=
modernc.org/sqlite v1.59.0/go.mod h1:+paeT2A3iPRHkQDwG7oA6Tk0zQd5woMEI8q7orfry8k=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"sort"
"github.com/scylladb/go-set/strset"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
"github.com/anchore/syft/internal/task"
)
+2 -2
View File
@@ -1,4 +1,4 @@
// this file handles YAML file reading and writing with comment preservation, using gopkg.in/yaml.v3's node tree to maintain all existing comments during regeneration.
// this file handles YAML file reading and writing with comment preservation, using go.yaml.in/yaml/v3's node tree to maintain all existing comments during regeneration.
package main
import (
@@ -6,7 +6,7 @@ import (
"os"
"regexp"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
"github.com/anchore/syft/internal/capabilities"
"github.com/anchore/syft/internal/capabilities/internal"
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"testing"
"github.com/stretchr/testify/require"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
)
func TestFindFieldValue(t *testing.T) {
@@ -5,7 +5,7 @@ import (
"os"
"path/filepath"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
"github.com/anchore/syft/internal/capabilities"
)
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"github.com/google/go-cmp/cmp"
"github.com/stretchr/testify/require"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
)
func TestConfigSchemaExtensions(t *testing.T) {
+19
View File
@@ -0,0 +1,19 @@
package internal
import (
"bufio"
"io"
)
// maxScannedLineSize is the longest line NewLineScanner will return before Scan stops with bufio.ErrTooLong.
// The buffer only grows to this size when a line needs it.
const maxScannedLineSize = 1024 * 1024
// NewLineScanner returns a bufio.Scanner that accepts lines up to 1MB rather than the 64KB default, since
// package metadata files carry long single-line fields (descriptions, license text, dependency lists). Callers
// must still check Err() after scanning.
func NewLineScanner(r io.Reader) *bufio.Scanner {
s := bufio.NewScanner(r)
s.Buffer(nil, maxScannedLineSize)
return s
}
+24
View File
@@ -0,0 +1,24 @@
package internal
import (
"bufio"
"strings"
"testing"
"github.com/stretchr/testify/require"
)
func TestNewLineScanner(t *testing.T) {
long := strings.Repeat("a", 100*1024)
s := NewLineScanner(strings.NewReader(long + "\nb\n"))
require.True(t, s.Scan())
require.Equal(t, long, s.Text())
require.True(t, s.Scan())
require.Equal(t, "b", s.Text())
require.False(t, s.Scan())
require.NoError(t, s.Err())
s = NewLineScanner(strings.NewReader(strings.Repeat("a", maxScannedLineSize+1)))
require.False(t, s.Scan())
require.ErrorIs(t, s.Err(), bufio.ErrTooLong)
}
+4 -6
View File
@@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"regexp"
"slices"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
@@ -191,12 +192,9 @@ func hasInstalledESMPackage(packages *pkg.Collection) bool {
if packages == nil {
return false
}
for p := range packages.Enumerate(pkg.DebPkg) {
if esmVersionPattern.MatchString(p.Version) {
return true
}
}
return false
return slices.ContainsFunc(packages.Sorted(pkg.DebPkg), func(p pkg.Package) bool {
return esmVersionPattern.MatchString(p.Version)
})
}
func fileMatchesAny(resolver file.Resolver, location file.Location, patterns ...*regexp.Regexp) (bool, error) {
+28
View File
@@ -8,6 +8,7 @@ import (
"path/filepath"
"slices"
"testing"
"time"
"github.com/stretchr/testify/require"
@@ -232,6 +233,33 @@ func Test_findUbuntuFeatures(t *testing.T) {
}
}
// returning early from Collection.Enumerate leaks its goroutine and read lock, so any later write blocks forever.
func Test_findUbuntuFeatures_releasesPackageCollectionLock(t *testing.T) {
resolver := fixtureResolverForDir(t, "testdata/ubuntu_plain")
s := sbom.SBOM{}
s.Artifacts.LinuxDistribution = &linux.Release{ID: "ubuntu"}
s.Artifacts.Packages = pkg.NewCollection(
pkg.Package{Name: "openssl", Version: "1.1.1f-1ubuntu2.19+esm1", Type: pkg.DebPkg},
pkg.Package{Name: "libcap2", Version: "1:2.32-1ubuntu0.1~esm1", Type: pkg.DebPkg},
)
require.NoError(t, os.DetectFeatures(context.Background(), resolver, sbomsync.NewBuilder(&s)))
require.True(t, s.Artifacts.LinuxDistribution.ExtendedSupport)
added := make(chan struct{})
go func() {
s.Artifacts.Packages.Add(pkg.Package{Name: "zlib1g", Version: "1:1.2.11.dfsg-2ubuntu1", Type: pkg.DebPkg})
close(added)
}()
select {
case <-added:
case <-time.After(5 * time.Second):
t.Fatal("package collection is still read-locked after feature detection")
}
}
// a read error in the apt-source signal must not suppress the ubuntu-advantage status signal; the three
// signals are meant to be independent fallbacks.
func Test_findUbuntuFeatures_signalErrorDoesNotSuppressOthers(t *testing.T) {
@@ -0,0 +1,74 @@
package packagemetadata
import (
"reflect"
"strings"
"testing"
)
// untypedFieldAllowlist holds package metadata fields that are allowed to carry untyped (interface) values.
// Values in these fields come straight from whatever a parser decoded, so they are not shaped by syft
// and can hold anything an encoder or hasher chokes on. Every entry is part of existing SBOM output and
// cannot be retyped without breaking it; new entries need a reason and a review.
var untypedFieldAllowlist = map[string]string{
"GGUFFileHeader.RemainingKeyValues": "raw GGUF header key-values; retyping changes the header output and metadataHash",
}
// TestNoUntypedMetadataFields fails when a package metadata type gains a field typed any, []any,
// map[string]any (or any other interface) that is not on the allowlist.
func TestNoUntypedMetadataFields(t *testing.T) {
var hits []string
for _, ty := range AllTypes() {
rt := reflect.TypeOf(ty)
collectUntyped(rt, rt.Name(), map[reflect.Type]bool{}, &hits)
}
seen := map[string]bool{}
for _, h := range hits {
seen[h] = true
if _, ok := untypedFieldAllowlist[h]; !ok {
t.Errorf("untyped field %s: use a concrete type, or add it to untypedFieldAllowlist with a reason", h)
}
}
for k := range untypedFieldAllowlist {
if !seen[k] {
t.Errorf("stale untypedFieldAllowlist entry %s: field no longer exists or is now typed", k)
}
}
}
// collectUntyped records "Struct.Field" for every field whose type reaches an interface, descending
// through pointers, slices, arrays, maps and nested syft-owned structs.
func collectUntyped(rt reflect.Type, path string, visited map[reflect.Type]bool, hits *[]string) {
switch rt.Kind() {
case reflect.Interface:
*hits = append(*hits, path)
case reflect.Pointer, reflect.Slice, reflect.Array:
collectUntyped(rt.Elem(), path, visited, hits)
case reflect.Map:
collectUntyped(rt.Key(), path, visited, hits)
collectUntyped(rt.Elem(), path, visited, hits)
case reflect.Struct:
if visited[rt] {
return
}
if rt.Name() != "" && !strings.HasPrefix(rt.PkgPath(), "github.com/anchore/syft/") {
// a third-party type in the model carries whatever shape the library gives it (the same leak as an
// interface field), so it counts as untyped. The standard library is left alone.
if strings.Contains(strings.Split(rt.PkgPath(), "/")[0], ".") {
*hits = append(*hits, path)
}
return
}
visited[rt] = true
for i := 0; i < rt.NumField(); i++ {
f := rt.Field(i)
// unexported embedded structs still promote their exported fields to the encoders and the hasher. A
// field is only out of reach when it is neither encoded nor hashed by SetID.
if (!f.IsExported() && !f.Anonymous) || (f.Tag.Get("json") == "-" && f.Tag.Get("hash") == "ignore") {
continue
}
collectUntyped(f.Type, rt.Name()+"."+f.Name, visited, hits)
}
}
}
+7
View File
@@ -0,0 +1,7 @@
package internal
// MaxSafeTensorsHeaderSize caps a safetensors header's JSON (excluding its
// 8-byte length prefix). Shared by the ai cataloger and the OCI model source so
// dir and OCI scans accept the same headers. Real-world shard headers are well
// under 1 MB.
const MaxSafeTensorsHeaderSize = 8 * 1024 * 1024
+43
View File
@@ -44,6 +44,7 @@ type LicenseInfo struct {
// LicenseByURL returns the license ID and name for a given URL from the SPDX license list.
// The URL should match one of the URLs in the seeAlso field of an SPDX license.
// The scheme (http:// or https://) is stripped before lookup, so both schemes match.
// opensource.org URLs additionally match regardless of the form used (see normalizeOpenSourceOrgURL).
func LicenseByURL(url string) (LicenseInfo, bool) {
url = strings.TrimSpace(url)
url = stripScheme(url)
@@ -52,9 +53,51 @@ func LicenseByURL(url string) (LicenseInfo, bool) {
ID: id,
}, true
}
if normalized, ok := normalizeOpenSourceOrgURL(url); ok {
if id, exists := openSourceOrgURLToLicense[normalized]; exists {
return LicenseInfo{
ID: id,
}, true
}
}
return LicenseInfo{}, false
}
// openSourceOrgURLToLicense indexes the opensource.org entries of urlToLicense by normalized URL.
var openSourceOrgURLToLicense = buildOpenSourceOrgURLToLicense(urlToLicense)
func buildOpenSourceOrgURLToLicense(urls map[string]string) map[string]string {
index := make(map[string]string)
ambiguous := make(map[string]bool)
for url, id := range urls {
normalized, ok := normalizeOpenSourceOrgURL(url)
if !ok || ambiguous[normalized] {
continue
}
if existing, exists := index[normalized]; exists && existing != id {
// don't guess between licenses that only differ by URL form
delete(index, normalized)
ambiguous[normalized] = true
continue
}
index[normalized] = id
}
return index
}
// normalizeOpenSourceOrgURL reduces a scheme-less opensource.org URL to a canonical form.
// SPDX has changed these URLs over time (e.g. 3.29.0 replaced opensource.org/licenses/MIT and
// opensource.org/license/mit/ with opensource.org/license/MIT), while package metadata in the wild
// still uses the older forms. The www. prefix, /licenses/ vs /license/, case, and trailing slash are ignored.
func normalizeOpenSourceOrgURL(url string) (string, bool) {
url = strings.TrimPrefix(strings.ToLower(url), "www.")
if !strings.HasPrefix(url, "opensource.org/") {
return "", false
}
url = strings.Replace(url, "opensource.org/licenses/", "opensource.org/license/", 1)
return strings.TrimSuffix(url, "/"), true
}
// stripScheme removes http:// or https:// prefix from a URL.
// This allows a single map entry to match both schemes.
func stripScheme(url string) string {
+254 -199
View File
@@ -2,7 +2,7 @@
// This file was generated using data from https://spdx.org/licenses/licenses.json
package spdxlicense
const Version = "3.28.0"
const Version = "3.29.0"
var licenseIDs = map[string]string{
"0bsd": "0BSD",
@@ -116,6 +116,7 @@ var licenseIDs = map[string]string{
"aswfdigitalassets1.0.0": "ASWF-Digital-Assets-1.0",
"aswfdigitalassets1.1": "ASWF-Digital-Assets-1.1",
"aswfdigitalassets1.1.0": "ASWF-Digital-Assets-1.1",
"atcgame": "atc-game",
"baekmuk": "Baekmuk",
"bahyph": "Bahyph",
"barr": "Barr",
@@ -142,8 +143,11 @@ var licenseIDs = map[string]string{
"briangladman2.0clause": "Brian-Gladman-2-Clause",
"briangladman2clause": "Brian-Gladman-2-Clause",
"briangladman3.0.0clause": "Brian-Gladman-3-Clause",
"briangladman3.0.0clausenoconversion": "Brian-Gladman-3-Clause-no-conversion",
"briangladman3.0clause": "Brian-Gladman-3-Clause",
"briangladman3.0clausenoconversion": "Brian-Gladman-3-Clause-no-conversion",
"briangladman3clause": "Brian-Gladman-3-Clause",
"briangladman3clausenoconversion": "Brian-Gladman-3-Clause-no-conversion",
"bsd1.0.0clause": "BSD-1-Clause",
"bsd1.0clause": "BSD-1-Clause",
"bsd1clause": "BSD-1-Clause",
@@ -154,6 +158,7 @@ var licenseIDs = map[string]string{
"bsd2.0.0clausenetbsd": "BSD-2-Clause",
"bsd2.0.0clausepatent": "BSD-2-Clause-Patent",
"bsd2.0.0clausepkgconfdisclaimer": "BSD-2-Clause-pkgconf-disclaimer",
"bsd2.0.0clauseposunchanged": "BSD-2-Clause-pos-unchanged",
"bsd2.0.0clauseviews": "BSD-2-Clause-Views",
"bsd2.0clause": "BSD-2-Clause",
"bsd2.0clausedarwin": "BSD-2-Clause-Darwin",
@@ -162,6 +167,7 @@ var licenseIDs = map[string]string{
"bsd2.0clausenetbsd": "BSD-2-Clause",
"bsd2.0clausepatent": "BSD-2-Clause-Patent",
"bsd2.0clausepkgconfdisclaimer": "BSD-2-Clause-pkgconf-disclaimer",
"bsd2.0clauseposunchanged": "BSD-2-Clause-pos-unchanged",
"bsd2.0clauseviews": "BSD-2-Clause-Views",
"bsd2clause": "BSD-2-Clause",
"bsd2clausedarwin": "BSD-2-Clause-Darwin",
@@ -170,6 +176,7 @@ var licenseIDs = map[string]string{
"bsd2clausenetbsd": "BSD-2-Clause",
"bsd2clausepatent": "BSD-2-Clause-Patent",
"bsd2clausepkgconfdisclaimer": "BSD-2-Clause-pkgconf-disclaimer",
"bsd2clauseposunchanged": "BSD-2-Clause-pos-unchanged",
"bsd2clauseviews": "BSD-2-Clause-Views",
"bsd3.0.0clause": "BSD-3-Clause",
"bsd3.0.0clauseacpica": "BSD-3-Clause-acpica",
@@ -184,6 +191,7 @@ var licenseIDs = map[string]string{
"bsd3.0.0clausenonuclearlicense2014": "BSD-3-Clause-No-Nuclear-License-2014",
"bsd3.0.0clausenonuclearwarranty": "BSD-3-Clause-No-Nuclear-Warranty",
"bsd3.0.0clauseopenmpi": "BSD-3-Clause-Open-MPI",
"bsd3.0.0clauseopenwebui": "BSD-3-Clause-OpenWebUI",
"bsd3.0.0clausesun": "BSD-3-Clause-Sun",
"bsd3.0.0clausetso": "BSD-3-Clause-Tso",
"bsd3.0clause": "BSD-3-Clause",
@@ -199,6 +207,7 @@ var licenseIDs = map[string]string{
"bsd3.0clausenonuclearlicense2014": "BSD-3-Clause-No-Nuclear-License-2014",
"bsd3.0clausenonuclearwarranty": "BSD-3-Clause-No-Nuclear-Warranty",
"bsd3.0clauseopenmpi": "BSD-3-Clause-Open-MPI",
"bsd3.0clauseopenwebui": "BSD-3-Clause-OpenWebUI",
"bsd3.0clausesun": "BSD-3-Clause-Sun",
"bsd3.0clausetso": "BSD-3-Clause-Tso",
"bsd3clause": "BSD-3-Clause",
@@ -214,6 +223,7 @@ var licenseIDs = map[string]string{
"bsd3clausenonuclearlicense2014": "BSD-3-Clause-No-Nuclear-License-2014",
"bsd3clausenonuclearwarranty": "BSD-3-Clause-No-Nuclear-Warranty",
"bsd3clauseopenmpi": "BSD-3-Clause-Open-MPI",
"bsd3clauseopenwebui": "BSD-3-Clause-OpenWebUI",
"bsd3clausesun": "BSD-3-Clause-Sun",
"bsd3clausetso": "BSD-3-Clause-Tso",
"bsd4.0.0clause": "BSD-4-Clause",
@@ -232,12 +242,15 @@ var licenseIDs = map[string]string{
"bsd4reno": "BSD-4.3RENO",
"bsd4tahoe": "BSD-4.3TAHOE",
"bsdadvertisingacknowledgement": "BSD-Advertising-Acknowledgement",
"bsdasktoendorse": "BSD-ask-to-endorse",
"bsdattributionhpnddisclaimer": "BSD-Attribution-HPND-disclaimer",
"bsdinfernonettverk": "BSD-Inferno-Nettverk",
"bsdmarkmodifications": "BSD-Mark-Modifications",
"bsdprotection": "BSD-Protection",
"bsdsourcealtgpl": "BSD-Source-alt-GPL",
"bsdsourcebeginningfile": "BSD-Source-beginning-file",
"bsdsourcecode": "BSD-Source-Code",
"bsdsourcecodenodisclaimer": "BSD-Source-Code-no-disclaimer",
"bsdsystemics": "BSD-Systemics",
"bsdsystemicsw3.0.0works": "BSD-Systemics-W3Works",
"bsdsystemicsw3.0works": "BSD-Systemics-W3Works",
@@ -246,6 +259,7 @@ var licenseIDs = map[string]string{
"bsl1.0": "BSL-1.0",
"bsl1.0.0": "BSL-1.0",
"buddy": "Buddy",
"bugroff": "Bugroff",
"busl1": "BUSL-1.1",
"busl1.1": "BUSL-1.1",
"busl1.1.0": "BUSL-1.1",
@@ -316,8 +330,11 @@ var licenseIDs = map[string]string{
"ccbync3.0": "CC-BY-NC-3.0",
"ccbync3.0.0": "CC-BY-NC-3.0",
"ccbync3.0.0de": "CC-BY-NC-3.0-DE",
"ccbync3.0.0igo": "CC-BY-NC-3.0-IGO",
"ccbync3.0de": "CC-BY-NC-3.0-DE",
"ccbync3.0igo": "CC-BY-NC-3.0-IGO",
"ccbync3de": "CC-BY-NC-3.0-DE",
"ccbync3igo": "CC-BY-NC-3.0-IGO",
"ccbync4": "CC-BY-NC-4.0",
"ccbync4.0": "CC-BY-NC-4.0",
"ccbync4.0.0": "CC-BY-NC-4.0",
@@ -592,6 +609,7 @@ var licenseIDs = map[string]string{
"fair": "Fair",
"fbm": "FBM",
"fdkaac": "FDK-AAC",
"fdkmpegh": "FDK-MPEG-H",
"fergusontwofish": "Ferguson-Twofish",
"frameworx1": "Frameworx-1.0",
"frameworx1.0": "Frameworx-1.0",
@@ -767,6 +785,9 @@ var licenseIDs = map[string]string{
"hippocratic2": "Hippocratic-2.1",
"hippocratic2.1": "Hippocratic-2.1",
"hippocratic2.1.0": "Hippocratic-2.1",
"hippocratic3.0.0core": "Hippocratic-3.0-core",
"hippocratic3.0core": "Hippocratic-3.0-core",
"hippocratic3core": "Hippocratic-3.0-core",
"hp1986": "HP-1986",
"hp1986.0": "HP-1986",
"hp1986.0.0": "HP-1986",
@@ -813,6 +834,7 @@ var licenseIDs = map[string]string{
"imlib2": "Imlib2",
"imlib2.0": "Imlib2",
"imlib2.0.0": "Imlib2",
"informatica": "Informatica",
"infozip": "Info-ZIP",
"innernet2": "Inner-Net-2.0",
"innernet2.0": "Inner-Net-2.0",
@@ -1011,6 +1033,9 @@ var licenseIDs = map[string]string{
"mulanpsl2.0.0": "MulanPSL-2.0",
"multics": "Multics",
"mup": "Mup",
"mvt1": "MVT-1.1",
"mvt1.1": "MVT-1.1",
"mvt1.1.0": "MVT-1.1",
"naist2003": "NAIST-2003",
"naist2003.0": "NAIST-2003",
"naist2003.0.0": "NAIST-2003",
@@ -1489,6 +1514,8 @@ var urlToLicense = map[string]string{
"cdla.io/permissive-1-0": "CDLA-Permissive-1.0",
"cdla.io/sharing-1-0": "CDLA-Sharing-1.0",
"celestrak.org/publications/AIAA/2006-6753/faq.php": "SGP4",
"cgit.freebsd.org/src/plain/sys/dev/aic7xxx/aic7xxx_osm.c": "BSD-Source-alt-GPL",
"cgit.freebsd.org/src/tree/usr.bin/sockstat/sockstat.c": "BSD-2-Clause-pos-unchanged",
"cgit.freedesktop.org/libbsd/tree/man/setproctitle.3bsd": "libutil-David-Nugent",
"coil.apotheon.org/plaintext/01.0.txt": "COIL-1.0",
"core.tcl-lang.org/tk/file?name=compat/unistd.h": "HPND-UC",
@@ -1512,6 +1539,7 @@ var urlToLicense = map[string]string{
"creativecommons.org/licenses/by-nc/2.0/legalcode": "CC-BY-NC-2.0",
"creativecommons.org/licenses/by-nc/2.5/legalcode": "CC-BY-NC-2.5",
"creativecommons.org/licenses/by-nc/3.0/de/legalcode": "CC-BY-NC-3.0-DE",
"creativecommons.org/licenses/by-nc/3.0/igo/legalcode": "CC-BY-NC-3.0-IGO",
"creativecommons.org/licenses/by-nc/3.0/legalcode": "CC-BY-NC-3.0",
"creativecommons.org/licenses/by-nc/4.0/legalcode": "CC-BY-NC-4.0",
"creativecommons.org/licenses/by-nd-nc/1.0/legalcode": "CC-BY-NC-ND-1.0",
@@ -1564,6 +1592,7 @@ var urlToLicense = map[string]string{
"directory.fsf.org/wiki/License:Fdk": "FDK-AAC",
"doc.qt.io/archives/3.3/license.html": "QPL-1.0",
"doc.qt.nokia.com/3.3/license.html": "QPL-1.0",
"docs.mvt.re/en/latest/license/": "MVT-1.1",
"docs.omniverse.nvidia.com/install-guide/latest/common/licenses.html": "AML-glslang",
"docs.oracle.com/cd/E81115_01/html/E81116/licenses.html": "man2html",
"docs.python.org/3/license.html": "Python-2.0.1",
@@ -1669,6 +1698,7 @@ var urlToLicense = map[string]string{
"fedoraproject.org/wiki/Licensing:Mplus?rd=Licensing/mplus": "mplus",
"fedoraproject.org/wiki/Licensing:ThorPublicLicense": "TPL-1.0",
"firstdonoharm.dev/version/2/1/license.html": "Hippocratic-2.1",
"firstdonoharm.dev/version/3/0/core.txt": "Hippocratic-3.0-core",
"forge.gouv.qc.ca/licence/fr/liliq-v1-1/": "LiLiQ-P-1.1",
"forge.gouv.qc.ca/licence/liliq-p": "LiLiQ-R-1.1",
"forge.gouv.qc.ca/licence/liliq-p/": "LiLiQ-P-1.1",
@@ -1686,68 +1716,70 @@ var urlToLicense = map[string]string{
"ftp://ftp.tin.org/pub/news/utils/newsx/newsx-1.6.tar.gz": "Zeeff",
"gianluca.dellavedova.org/2011/01/03/clarified-artistic-license/": "ClArtistic",
"git.denx.de/?p=u-boot.git;a=blob;f=arch/powerpc/cpu/ppc4xx/miiphy.c;h=297155fdafa064b955e53e9832de93bfb0cfb85b;hb=9fab4bf4cc077c21e43941866f3f2c196f28670d": "IBM-pibs",
"git.enlightenment.org/legacy/imlib2.git/tree/COPYING": "Imlib2",
"git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man2/getcpu.2#n4": "Linux-man-pages-1-para",
"git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man2/migrate_pages.2#n8": "Linux-man-pages-copyleft-2-para",
"git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man2/move_pages.2#n5": "Linux-man-pages-copyleft-2-para",
"git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man2/set_mempolicy.2#n5": "Linux-man-pages-copyleft-var",
"git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/infiniband/core/sa.h": "Linux-OpenIB",
"git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/net/sunrpc/auth_gss/gss_generic_token.c?h=v4.19": "HPND-sell-variant",
"git.openldap.org/openldap/openldap/-/blob/master/COPYRIGHT#L55-63": "BSD-4.3RENO",
"git.openldap.org/openldap/openldap/-/blob/master/COPYRIGHT?ref_type=heads#L39-51": "Furuseth",
"git.openldap.org/openldap/openldap/-/blob/master/libraries/libldap/os-local.c?ref_type=heads#L19-23": "PADL",
"git.savannah.gnu.org/cgit/freetype/freetype2.git/tree/docs/FTL.TXT": "FTL",
"git.savannah.gnu.org/cgit/gcl.git/tree/gcl/lsp/gcl_loop.lsp?h=Version_2_6_13pre": "LOOP",
"git.savannah.gnu.org/cgit/gnulib.git/tree/modules/COPYING?id=7b08932179d0d6b017f7df01a2ddf6e096b038e3": "FSFULLRSD",
"git.savannah.gnu.org/cgit/indent.git/tree/doc/indent.texi?id=a74c6b4ee49397cf330b333da1042bffa60ed14f#n1788": "BSD-4.3TAHOE",
"git.savannah.gnu.org/cgit/indent.git/tree/doc/indent.texi?id=a74c6b4ee49397cf330b333da1042bffa60ed14f#n74": "Latex2e-translated-notice",
"git.savannah.gnu.org/cgit/kazlib.git/tree/except.c?id=0062df360c2d17d57f6af19b0e444c51feb99036": "Kazlib",
"git.savannah.gnu.org/cgit/wget.git/tree/util/trunc.c?h=v1.21.3&id=40747a11e44ced5a8ac628a41f879ced3e2ebce9#n6": "FSFAP-no-warranty-disclaimer",
"git.sr.ht/~nabijaczleweli/ossp-var": "OSSP",
"github.com/389ds/389-ds-base/blob/main/ldap/include/sysexits-compat.h#L15": "BSD-4.3TAHOE",
"github.com/AcademySoftwareFoundation/foundation/blob/main/digital_assets/aswf_digital_assets_license_v1.0.txt": "ASWF-Digital-Assets-1.0",
"github.com/AcademySoftwareFoundation/foundation/blob/main/digital_assets/aswf_digital_assets_license_v1.1.txt": "ASWF-Digital-Assets-1.1",
"github.com/AdaCore/gnatcoll-core/blob/master/docs/index.rst": "AdaCore-doc",
"github.com/AdaCore/gnatcoll-db/blob/master/docs/index.rst": "AdaCore-doc",
"github.com/AdaCore/xmlada/blob/master/docs/index.rst": "AdaCore-doc",
"github.com/BuildCraft/BuildCraft/blob/623d323b1868712f29f4a8b0979a02e8d1835131/LICENSE": "MMPL-1.0.1",
"github.com/CommunitySpecification/1.0/blob/master/1._Community_Specification_License-v1.md": "Community-Spec-1.0",
"github.com/CorsixTH/deps/blob/fd339a9f526d1d9c9f01ccf39e438a015da50035/licences/libgsm.txt": "TU-Berlin-2.0",
"github.com/Cyan4973/xxHash/blob/dev/doc/xxhash_spec.md": "LPD-document",
"github.com/DISIC/politique-de-contribution-open-source/blob/master/LICENSE.pdf": "etalab-2.0",
"github.com/Distrotech/hdparm/blob/4517550db29a91420fb2b020349523b1b4512df2/LICENSE.TXT": "hdparm",
"github.com/Distrotech/transfig/blob/master/transfig/transfig.c": "Xfig",
"github.com/DrItanium/maya/blob/master/LICENSE.CLIPS": "Clips",
"github.com/Dual-Life/mime-base64/blob/master/Base64.xs#L12": "metamail",
"github.com/EthicalSource/hippocratic-license/blob/58c0e646d64ff6fbee275bfe2b9492f914e3ab2a/LICENSE.txt": "Hippocratic-2.1",
"github.com/FreeRADIUS/freeradius-client/blob/master/COPYRIGHT#L32": "HPND-Fenneberg-Livingston",
"github.com/GNOME/gcr/blob/master/docs/COPYING": "GCR-docs",
"github.com/GNOME/libxml2/blob/master/dict.c": "ISC-Veillard",
"github.com/HDFGroup/hdf5/?tab=License-1-ov-file#readme": "HDF5",
"github.com/KhronosGroup/SPIRV-Cross/blob/main/LICENSES/LicenseRef-KhronosFreeUse.txt": "MIT-Khronos-old",
"github.com/KhronosGroup/glslang/blob/main/LICENSE.txt#L949": "AML-glslang",
"github.com/MariaDB/server/blob/11.6/libmysqld/lib_sql.cc": "Boehm-GC-without-fee",
"github.com/Neo23x0/sigma/blob/master/LICENSE.Detection.Rules.md": "DRL-1.0",
"github.com/OGRECave/ogre/blob/master/OgreMain/include/OgreSingleton.h#L28C3-L35C46": "Game-Programming-Gems",
"github.com/Perl/perl5/blob/blead/util.c#L6136": "Martin-Birgmeier",
"github.com/PixarAnimationStudios/OpenSubdiv/blob/v3_5_0/opensubdiv/version.cpp#L2-L22": "Pixar",
"github.com/PixarAnimationStudios/OpenSubdiv/raw/v3_5_0/LICENSE.txt": "Pixar",
"github.com/RTimothyEdwards/magic/blob/master/LICENSE": "HPND-UC-export-US",
"github.com/SELinuxProject/selinux/blob/master/libselinux/LICENSE": "libselinux-1.0",
"github.com/SWI-Prolog/packages-clib/blob/master/sha1/brg_endian.h": "Brian-Gladman-3-Clause",
"github.com/SWI-Prolog/packages-xpce/blob/161a40cd82004f731ba48024f9d30af388a7edf5/src/img/gifwrite.c#L21-L26": "FBM",
"github.com/SWI-Prolog/swipl-devel/blob/master/src/os/dtoa.c": "dtoa",
"github.com/SigmaHQ/Detection-Rule-License/blob/6ec7fbde6101d101b5b5d1fcb8f9b69fbc76c04a/LICENSE.Detection.Rules.md": "DRL-1.1",
"github.com/Slicer/Slicer/blob/main/License.txt": "3D-Slicer-1.0",
"github.com/Unidata/UDUNITS-2/blob/master/COPYRIGHT": "UCAR",
"github.com/acpica/acpica/blob/master/source/common/acfileio.c#L119": "BSD-3-Clause-acpica",
"github.com/adaptivecomputing/torque/blob/master/PBS_License.txt": "OpenPBS-2.3",
"github.com/adversary-org/wtfnmf/raw/3f2cd8235a64350a57a51b9739715edaea63ec1a/COPYING.WTFNMFPL-utf8": "WTFNMFPL",
"github.com/adversary-org/wtfnmf/raw/refs/tags/1.0/COPYING.WTFNMFPL": "WTFNMFPL",
"github.com/ap/Text-Tabs/blob/master/lib.modern/Text/Tabs.pm#L148": "TTWL",
"github.com/apache/apr/blob/trunk/LICENSE#L298C6-L298C29": "Caldera-no-preamble",
"github.com/audacious-media-player/audacious/blob/master/COPYING": "BSD-2-Clause-pkgconf-disclaimer",
"github.com/audacious-media-player/audacious/blob/master/src/audacious/main.cc": "BSD-2-Clause-pkgconf-disclaimer",
"git.enlightenment.org/legacy/imlib2.git/tree/COPYING": "Imlib2",
"git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man2/getcpu.2#n4": "Linux-man-pages-1-para",
"git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man2/migrate_pages.2#n8": "Linux-man-pages-copyleft-2-para",
"git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man2/move_pages.2#n5": "Linux-man-pages-copyleft-2-para",
"git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man2/set_mempolicy.2#n5": "Linux-man-pages-copyleft-var",
"git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/infiniband/core/sa.h": "Linux-OpenIB",
"git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/net/sunrpc/auth_gss/gss_generic_token.c?h=v4.19": "HPND-sell-variant",
"git.openldap.org/openldap/openldap/-/blob/master/COPYRIGHT#L55-63": "BSD-4.3RENO",
"git.openldap.org/openldap/openldap/-/blob/master/COPYRIGHT?ref_type=heads#L39-51": "Furuseth",
"git.openldap.org/openldap/openldap/-/blob/master/libraries/libldap/os-local.c?ref_type=heads#L19-23": "PADL",
"git.savannah.gnu.org/cgit/freetype/freetype2.git/tree/docs/FTL.TXT": "FTL",
"git.savannah.gnu.org/cgit/gcl.git/tree/gcl/lsp/gcl_loop.lsp?h=Version_2_6_13pre": "LOOP",
"git.savannah.gnu.org/cgit/gnulib.git/tree/modules/COPYING?id=7b08932179d0d6b017f7df01a2ddf6e096b038e3": "FSFULLRSD",
"git.savannah.gnu.org/cgit/indent.git/tree/doc/indent.texi?id=a74c6b4ee49397cf330b333da1042bffa60ed14f#n1788": "BSD-4.3TAHOE",
"git.savannah.gnu.org/cgit/indent.git/tree/doc/indent.texi?id=a74c6b4ee49397cf330b333da1042bffa60ed14f#n74": "Latex2e-translated-notice",
"git.savannah.gnu.org/cgit/kazlib.git/tree/except.c?id=0062df360c2d17d57f6af19b0e444c51feb99036": "Kazlib",
"git.savannah.gnu.org/cgit/wget.git/tree/util/trunc.c?h=v1.21.3&id=40747a11e44ced5a8ac628a41f879ced3e2ebce9#n6": "FSFAP-no-warranty-disclaimer",
"git.sr.ht/~nabijaczleweli/ossp-var": "OSSP",
"github.com/389ds/389-ds-base/blob/main/ldap/include/sysexits-compat.h#L15": "BSD-4.3TAHOE",
"github.com/AcademySoftwareFoundation/foundation/blob/main/digital_assets/aswf_digital_assets_license_v1.0.txt": "ASWF-Digital-Assets-1.0",
"github.com/AcademySoftwareFoundation/foundation/blob/main/digital_assets/aswf_digital_assets_license_v1.1.txt": "ASWF-Digital-Assets-1.1",
"github.com/AdaCore/gnatcoll-core/blob/master/docs/index.rst": "AdaCore-doc",
"github.com/AdaCore/gnatcoll-db/blob/master/docs/index.rst": "AdaCore-doc",
"github.com/AdaCore/xmlada/blob/master/docs/index.rst": "AdaCore-doc",
"github.com/BuildCraft/BuildCraft/blob/623d323b1868712f29f4a8b0979a02e8d1835131/LICENSE": "MMPL-1.0.1",
"github.com/CommunitySpecification/1.0/blob/master/1._Community_Specification_License-v1.md": "Community-Spec-1.0",
"github.com/CorsixTH/deps/blob/fd339a9f526d1d9c9f01ccf39e438a015da50035/licences/libgsm.txt": "TU-Berlin-2.0",
"github.com/Cyan4973/xxHash/blob/dev/doc/xxhash_spec.md": "LPD-document",
"github.com/DISIC/politique-de-contribution-open-source/blob/master/LICENSE.pdf": "etalab-2.0",
"github.com/Distrotech/hdparm/blob/4517550db29a91420fb2b020349523b1b4512df2/LICENSE.TXT": "hdparm",
"github.com/Distrotech/transfig/blob/master/transfig/transfig.c": "Xfig",
"github.com/DrItanium/maya/blob/master/LICENSE.CLIPS": "Clips",
"github.com/Dual-Life/mime-base64/blob/master/Base64.xs#L12": "metamail",
"github.com/EthicalSource/hippocratic-license/blob/58c0e646d64ff6fbee275bfe2b9492f914e3ab2a/LICENSE.txt": "Hippocratic-2.1",
"github.com/Fraunhofer-IIS/mpeghdec/blob/main/LICENSE.txt": "FDK-MPEG-H",
"github.com/FreeRADIUS/freeradius-client/blob/master/COPYRIGHT#L32": "HPND-Fenneberg-Livingston",
"github.com/GNOME/gcr/blob/master/docs/COPYING": "GCR-docs",
"github.com/GNOME/libxml2/blob/master/dict.c": "ISC-Veillard",
"github.com/HDFGroup/hdf5/?tab=License-1-ov-file#readme": "HDF5",
"github.com/KhronosGroup/SPIRV-Cross/blob/main/LICENSES/LicenseRef-KhronosFreeUse.txt": "MIT-Khronos-old",
"github.com/KhronosGroup/glslang/blob/main/LICENSE.txt#L949": "AML-glslang",
"github.com/MariaDB/server/blob/11.6/libmysqld/lib_sql.cc": "Boehm-GC-without-fee",
"github.com/Neo23x0/sigma/blob/master/LICENSE.Detection.Rules.md": "DRL-1.0",
"github.com/OGRECave/ogre/blob/master/OgreMain/include/OgreSingleton.h#L28C3-L35C46": "Game-Programming-Gems",
"github.com/Perl/perl5/blob/blead/util.c#L6136": "Martin-Birgmeier",
"github.com/PicoEFI/PicoEFI/blob/2263ac1a2bc98e5534d64a4896d3cfbbae00745a/LICENSES/LicenseRef-scancode-bsd-no-disclaimer-unmodified.txt": "BSD-Source-Code-no-disclaimer",
"github.com/PixarAnimationStudios/OpenSubdiv/blob/v3_5_0/opensubdiv/version.cpp#L2-L22": "Pixar",
"github.com/PixarAnimationStudios/OpenSubdiv/raw/v3_5_0/LICENSE.txt": "Pixar",
"github.com/RTimothyEdwards/magic/blob/master/LICENSE": "HPND-UC-export-US",
"github.com/SELinuxProject/selinux/blob/master/libselinux/LICENSE": "libselinux-1.0",
"github.com/SWI-Prolog/packages-clib/blob/master/sha1/brg_endian.h": "Brian-Gladman-3-Clause",
"github.com/SWI-Prolog/packages-xpce/blob/161a40cd82004f731ba48024f9d30af388a7edf5/src/img/gifwrite.c#L21-L26": "FBM",
"github.com/SWI-Prolog/swipl-devel/blob/master/src/os/dtoa.c": "dtoa",
"github.com/SigmaHQ/Detection-Rule-License/blob/6ec7fbde6101d101b5b5d1fcb8f9b69fbc76c04a/LICENSE.Detection.Rules.md": "DRL-1.1",
"github.com/Slicer/Slicer/blob/main/License.txt": "3D-Slicer-1.0",
"github.com/Unidata/UDUNITS-2/blob/master/COPYRIGHT": "UCAR",
"github.com/acpica/acpica/blob/master/source/common/acfileio.c#L119": "BSD-3-Clause-acpica",
"github.com/adaptivecomputing/torque/blob/master/PBS_License.txt": "OpenPBS-2.3",
"github.com/adversary-org/wtfnmf/raw/3f2cd8235a64350a57a51b9739715edaea63ec1a/COPYING.WTFNMFPL-utf8": "WTFNMFPL",
"github.com/adversary-org/wtfnmf/raw/refs/tags/1.0/COPYING.WTFNMFPL": "WTFNMFPL",
"github.com/ap/Text-Tabs/blob/master/lib.modern/Text/Tabs.pm#L148": "TTWL",
"github.com/apache/apr/blob/trunk/LICENSE#L298C6-L298C29": "Caldera-no-preamble",
"github.com/audacious-media-player/audacious/blob/master/COPYING": "BSD-2-Clause-pkgconf-disclaimer",
"github.com/audacious-media-player/audacious/blob/master/src/audacious/main.cc": "BSD-2-Clause-pkgconf-disclaimer",
"github.com/aws/mit-0": "MIT-0",
"github.com/awsdocs/aws-cloud9-user-guide/blob/master/LICENSE-SAMPLECODE": "MIT-0",
"github.com/bagder/curl/blob/master/COPYING": "curl",
@@ -1772,6 +1804,7 @@ var urlToLicense = map[string]string{
"github.com/file/file/blob/master/COPYING": "BSD-2-Clause-Darwin",
"github.com/freebsd/freebsd-src/blob/main/lib/msun/src/e_acosh.c": "SunPro",
"github.com/freebsd/freebsd-src/blob/main/lib/msun/src/e_lgammal.c": "SunPro",
"github.com/freebsd/pkg/blob/main/src/add.c": "BSD-2-Clause-pos-unchanged",
"github.com/gabegundy/atlserver/blob/master/License.txt": "MS-LPL",
"github.com/greymass/swift-eosio/blob/master/LICENSE": "BSD-3-Clause-No-Military-License",
"github.com/hamano/man2html/blob/master/man2html.c": "man2html",
@@ -1784,6 +1817,7 @@ var urlToLicense = map[string]string{
"github.com/jsommers/pytricia/blob/master/patricia.c#L33-L67": "BSD-4-Clause",
"github.com/kfish/xsel/blob/master/COPYING": "HPND-sell-variant",
"github.com/kohler/t1utils/blob/master/LICENSE": "MIT-Click",
"github.com/krb5/krb5/blob/kfw-4.0.1-beta1/src/lib/crypto/builtin/aes/aesopt.h": "Brian-Gladman-3-Clause-no-conversion",
"github.com/krb5/krb5/blob/krb5-1.21.2-final/NOTICE#L111-L133": "HPND-export2-US",
"github.com/krb5/krb5/blob/krb5-1.21.2-final/NOTICE#L1157-L1182": "HPND-export-US-modify",
"github.com/krb5/krb5/blob/krb5-1.21.2-final/NOTICE#L140-L156": "Brian-Gladman-2-Clause",
@@ -1793,6 +1827,7 @@ var urlToLicense = map[string]string{
"github.com/krb5/krb5/blob/krb5-1.21.2-final/NOTICE#L831-L852": "HPND-export-US-acknowledgement",
"github.com/krzyzanowskim/CryptoSwift/blob/main/LICENSE": "CryptoSwift",
"github.com/lattera/freebsd/blob/master/sys/cam/cam.c#L4": "BSD-Source-beginning-file",
"github.com/libarchive/libarchive/blob/master/libarchive_fe/line_reader.c": "BSD-2-Clause-pos-unchanged",
"github.com/libcheck/check/blob/master/checkmk/checkmk.in": "checkmk",
"github.com/libjpeg-turbo/libjpeg-turbo/blob/main/README.ijg#L117-L161": "IJG",
"github.com/lsof-org/lsof/blob/master/COPYING": "lsof",
@@ -1805,6 +1840,7 @@ var urlToLicense = map[string]string{
"github.com/mm2/Little-CMS/blob/master/src/cmssm.c#L207": "softSurfer",
"github.com/mruby/mruby/blob/83d12f8d52522cdb7c8cc46fad34821359f453e6/mrbgems/mruby-dir/src/Win/dirent.c#L127-L140": "HPND-Kevlin-Henney",
"github.com/mtoyoda/sl/blob/master/LICENSE": "SL",
"github.com/mvt-project/mvt/blob/main/LICENSE": "MVT-1.1",
"github.com/n8n-io/n8n/blob/master/LICENSE.md": "SUL-1.0",
"github.com/ncbi/datasets/blob/0ea4cd16b61e5b799d9cc55aecfa016d6c9bd2bf/LICENSE.md": "NCBI-PD",
"github.com/ncbi/datasets/blob/master/LICENSE.md": "NCBI-PD",
@@ -1815,6 +1851,7 @@ var urlToLicense = map[string]string{
"github.com/novnc/noVNC/blob/master/core/crypto/des.js#L24": "Widget-Workshop",
"github.com/oetiker/rrdtool-1.x/blob/master/src/plbasename.c#L8-L11": "gtkbook",
"github.com/open-quantum-safe/liboqs/blob/40b01fdbb270f8614fde30e65d30e9da18c02393/src/common/rand/rand_nist.c#L1-L15": "NIST-Software",
"github.com/open-webui/open-webui/blob/main/LICENSE": "BSD-3-Clause-OpenWebUI",
"github.com/openjdk/jdk/blob/6162e2c5213c5dd7c1127fd9616b543efa898962/LICENSE": "GPL-2.0-or-later",
"github.com/openssh/openssh-portable/blob/1b11ea7c58cd5c59838b5fa574cd456d6047b2d4/LICENCE#L10": "SSH-OpenSSH",
"github.com/openssh/openssh-portable/blob/1b11ea7c58cd5c59838b5fa574cd456d6047b2d4/pathnames.h": "SSH-short",
@@ -1846,6 +1883,7 @@ var urlToLicense = map[string]string{
"github.com/sigmavirus24/x11-ssh-askpass/blob/master/dynlist.c": "HPND-sell-variant-MIT-disclaimer-rev",
"github.com/signal11/hidapi/blob/master/LICENSE-orig.txt": "HIDAPI",
"github.com/slogan621/gtkbook": "gtkbook",
"github.com/sudo-project/sudo/blob/c1307ea9ff340ce0538779f8e456501461fc44b7/plugins/sudoers/redblack.c#L24-L43": "BSD-ask-to-endorse",
"github.com/swh/ladspa/blob/7bf6f3799fdba70fda297c2d8fd9f526803d9680/gsm/COPYRIGHT": "TU-Berlin-1.0",
"github.com/tats/w3m/blob/master/COPYING": "w3m",
"github.com/tcheneau/Routing/blob/f09f46fcfe636107f22f2c98348188a65a135d98/README.md": "NIST-PD",
@@ -1856,6 +1894,7 @@ var urlToLicense = map[string]string{
"github.com/uim/libgcroots/blob/master/COPYING": "Boehm-GC",
"github.com/usnistgov/fipy/blob/86aaa5c2ba2c6f1be19593c5986071cf6568cc34/LICENSE.rst": "NIST-PD-fallback",
"github.com/usnistgov/jsip/blob/59700e6926cbe96c5cdae897d9a7d2656b42abe3/LICENSE": "NIST-PD-fallback",
"github.com/vattam/BSDGames/blob/master/atc/extern.c#L36": "atc-game",
"github.com/vixie/cron/tree/545b3f5246824a9cda5905eeb7cf019c95e66995": "Vixie-Cron",
"github.com/vstroebel/jpeg-encoder/blob/main/src/fdct.rs#L1-L72": "IJG",
"github.com/wernerd/ZRTPCPP/blob/6b3cd8e6783642292bad0c21e3e5e5ce45ff3e03/cryptcommon/twofish.c#L113C3-L127": "Ferguson-Twofish",
@@ -1869,6 +1908,7 @@ var urlToLicense = map[string]string{
"gitlab.com/embeddable-common-lisp/ecl/-/blob/develop/src/lsp/loop.lsp": "LOOP",
"gitlab.com/libeigen/eigen/-/blob/master/COPYING.MINPACK": "Minpack",
"gitlab.common-lisp.net/cmucl/cmucl/-/blob/master/src/code/loop.lisp": "LOOP",
"gitlab.freedesktop.org/dbus/dbus/-/blob/789d97ad53044f94351a958e785c2923643b346a/COPYING#L11-207": "AFL-2.1",
"gitlab.freedesktop.org/freetype/freetype/-/raw/master/docs/FTL.TXT": "FTL",
"gitlab.freedesktop.org/libopenraw/libopenraw/blob/master/lib/ljpegdecompressor.cpp#L32": "Cornell-Lossless-JPEG",
"gitlab.freedesktop.org/mesa/glw/-/blob/master/README?ref_type=heads": "SGI-OpenGL",
@@ -1892,6 +1932,7 @@ var urlToLicense = map[string]string{
"gitlab.freedesktop.org/xorg/lib/libxinerama/-/blob/cc22c2f60c3862482562955116d5455263b443dc/COPYING#L44-66": "X11-no-permit-persons",
"gitlab.freedesktop.org/xorg/lib/libxtst/-/blob/master/COPYING?ref_type=heads#L108-117": "HPND-doc-sell",
"gitlab.freedesktop.org/xorg/lib/libxtst/-/blob/master/COPYING?ref_type=heads#L70-77": "HPND-doc",
"gitlab.freedesktop.org/xorg/xserver/-/blob/dd5c2595a42d3ff0c4f18d9b53d1f6c3fd934fd4/COPYING#L365-389": "MIT",
"gitlab.freedesktop.org/xorg/xserver/-/blob/master/COPYING?ref_type=heads#L1781": "HPND-sell-MIT-disclaimer-xserver",
"gitlab.freedesktop.org/xorg/xserver/-/blob/master/COPYING?ref_type=heads#L239": "DEC-3-Clause",
"gitlab.freedesktop.org/xorg/xserver/-/blob/master/COPYING?ref_type=heads#L752": "Adobe-Display-PostScript",
@@ -1906,6 +1947,7 @@ var urlToLicense = map[string]string{
"heasarc.gsfc.nasa.gov/docs/software/ftools/fv/doc/license.html": "CFITSIO",
"helixcommunity.org/content/rpsl": "RPSL-1.0",
"hstechdocs.helpsystems.com/manuals/globalscape/archive/cuteftp6/open_ssl_license_agreement.htm": "OpenSSL-standalone",
"interoperable-europe.ec.europa.eu/sites/default/files/inline-files/EUPL%20v1_2%20EN(1).txt": "EUPL-1.2",
"java.net/projects/javaeetutorial/pages/BerkeleyLicense": "BSD-3-Clause-No-Nuclear-License-2014",
"javaee.github.io/glassfish/LICENSE": "CDDL-1.1",
"jogamp.org/git/?p=gluegen.git;a=blob_plain;f=LICENSE.txt": "BSD-3-Clause-No-Nuclear-Warranty",
@@ -1924,6 +1966,7 @@ var urlToLicense = map[string]string{
"library.netapp.com/ecm/ecm_download_file/ECMP1196395": "OpenSSL-standalone",
"license.coscl.org.cn/MulanPSL/": "MulanPSL-1.0",
"license.coscl.org.cn/MulanPSL2": "MulanPSL-2.0",
"license.mvt.re/1.1/": "MVT-1.1",
"lists.gnu.org/archive/html/autoconf/2012-04/msg00061.html": "FSFULLRWD",
"lists.opensource.org/pipermail/license-discuss_lists.opensource.org/2002-November/006304.html": "HPND",
"mariadb.com/bsl11/": "BUSL-1.1",
@@ -1965,99 +2008,105 @@ var urlToLicense = map[string]string{
"opensource.linux-mirror.org/licenses/afl-1.1.txt": "AFL-1.1",
"opensource.linux-mirror.org/licenses/afl-1.2.txt": "AFL-1.2",
"opensource.linux-mirror.org/licenses/afl-2.1.txt": "AFL-2.1",
"opensource.org/license/apache-2-0": "Apache-2.0",
"opensource.org/license/mit/": "MIT",
"opensource.org/license/olfl-1-3/": "OLFL-1.3",
"opensource.org/license/osc-license-1-0": "OSC-1.0",
"opensource.org/licenses/0BSD": "0BSD",
"opensource.org/licenses/AGPL-3.0": "AGPL-3.0-or-later",
"opensource.org/licenses/APL-1.0": "APL-1.0",
"opensource.org/licenses/Apache-1.1": "Apache-1.1",
"opensource.org/licenses/Apache-2.0": "Apache-2.0",
"opensource.org/licenses/Artistic-1.0": "Artistic-1.0-cl8",
"opensource.org/licenses/BSD-2-Clause": "BSD-2-Clause",
"opensource.org/licenses/BSD-3-Clause": "BSD-3-Clause",
"opensource.org/licenses/BSDplusPatent": "BSD-2-Clause-Patent",
"opensource.org/licenses/BSL-1.0": "BSL-1.0",
"opensource.org/licenses/CAL-1.0": "CAL-1.0-Combined-Work-Exception",
"opensource.org/licenses/CATOSL-1.1": "CATOSL-1.1",
"opensource.org/licenses/CNRI-Python": "CNRI-Python",
"opensource.org/licenses/CPAL-1.0": "CPAL-1.0",
"opensource.org/licenses/CPL-1.0": "CPL-1.0",
"opensource.org/licenses/CUA-OPL-1.0": "CUA-OPL-1.0",
"opensource.org/licenses/ECL-1.0": "ECL-1.0",
"opensource.org/licenses/ECL-2.0": "ECL-2.0",
"opensource.org/licenses/EFL-1.0": "EFL-1.0",
"opensource.org/licenses/EFL-2.0": "EFL-2.0",
"opensource.org/licenses/EPL-1.0": "EPL-1.0",
"opensource.org/licenses/EUDatagrid": "EUDatagrid",
"opensource.org/licenses/EUPL-1.1": "EUPL-1.1",
"opensource.org/licenses/EUPL-1.2": "EUPL-1.2",
"opensource.org/licenses/Entessa": "Entessa",
"opensource.org/licenses/Fair": "Fair",
"opensource.org/licenses/Frameworx-1.0": "Frameworx-1.0",
"opensource.org/licenses/GPL-2.0": "GPL-2.0-or-later",
"opensource.org/licenses/GPL-3.0": "GPL-3.0-or-later",
"opensource.org/licenses/HPND": "HPND",
"opensource.org/licenses/IPA": "IPA",
"opensource.org/licenses/IPL-1.0": "IPL-1.0",
"opensource.org/licenses/ISC": "ISC",
"opensource.org/licenses/Intel": "Intel",
"opensource.org/licenses/LGPL-2.1": "LGPL-2.1-or-later",
"opensource.org/licenses/LGPL-3.0": "LGPL-3.0-or-later",
"opensource.org/licenses/LPL-1.0": "LPL-1.0",
"opensource.org/licenses/LPL-1.02": "LPL-1.02",
"opensource.org/licenses/LPPL-1.3c": "LPPL-1.3c",
"opensource.org/licenses/LiLiQ-P-1.1": "LiLiQ-P-1.1",
"opensource.org/licenses/LiLiQ-R-1.1": "LiLiQ-R-1.1",
"opensource.org/licenses/LiLiQ-Rplus-1.1": "LiLiQ-Rplus-1.1",
"opensource.org/licenses/MIT": "MIT",
"opensource.org/licenses/MPL-1.0": "MPL-1.0",
"opensource.org/licenses/MPL-1.1": "MPL-1.1",
"opensource.org/licenses/MPL-2.0": "MPL-2.0-no-copyleft-exception",
"opensource.org/licenses/MS-PL": "MS-PL",
"opensource.org/licenses/MS-RL": "MS-RL",
"opensource.org/licenses/MirOS": "MirOS",
"opensource.org/licenses/Motosoto": "Motosoto",
"opensource.org/licenses/Multics": "Multics",
"opensource.org/licenses/NASA-1.3": "NASA-1.3",
"opensource.org/licenses/NCSA": "NCSA",
"opensource.org/licenses/NGPL": "NGPL",
"opensource.org/licenses/NOSL3.0": "NPOSL-3.0",
"opensource.org/licenses/NTP": "NTP",
"opensource.org/licenses/Naumen": "Naumen",
"opensource.org/licenses/OCLC-2.0": "OCLC-2.0",
"opensource.org/licenses/OFL-1.1": "OFL-1.1-no-RFN",
"opensource.org/licenses/OGTSL": "OGTSL",
"opensource.org/licenses/OPL-2.1": "OSET-PL-2.1",
"opensource.org/licenses/OSL-1.0": "OSL-1.0",
"opensource.org/licenses/OSL-2.1": "OSL-2.1",
"opensource.org/licenses/OSL-3.0": "OSL-3.0",
"opensource.org/licenses/PHP-3.0": "PHP-3.0",
"opensource.org/licenses/PostgreSQL": "PostgreSQL",
"opensource.org/licenses/Python-2.0": "Python-2.0",
"opensource.org/licenses/QPL-1.0": "QPL-1.0",
"opensource.org/licenses/RPL-1.1": "RPL-1.1",
"opensource.org/licenses/RPL-1.5": "RPL-1.5",
"opensource.org/licenses/RPSL-1.0": "RPSL-1.0",
"opensource.org/licenses/RSCPL": "RSCPL",
"opensource.org/licenses/SISSL": "SISSL",
"opensource.org/licenses/SPL-1.0": "SPL-1.0",
"opensource.org/licenses/SimPL-2.0": "SimPL-2.0",
"opensource.org/licenses/Sleepycat": "Sleepycat",
"opensource.org/licenses/UCL-1.0": "UCL-1.0",
"opensource.org/licenses/UPL": "UPL-1.0",
"opensource.org/licenses/VSL-1.0": "VSL-1.0",
"opensource.org/licenses/W3C": "W3C",
"opensource.org/licenses/Watcom-1.0": "Watcom-1.0",
"opensource.org/licenses/Xnet": "Xnet",
"opensource.org/licenses/ZPL-2.0": "ZPL-2.0",
"opensource.org/licenses/Zlib": "Zlib",
"opensource.org/licenses/afl-3.0": "AFL-3.0",
"opensource.org/licenses/artistic-license-2.0": "Artistic-2.0",
"opensource.org/licenses/attribution": "AAL",
"opensource.org/licenses/cddl1": "CDDL-1.0",
"opensource.org/licenses/nokia": "Nokia",
"opensource.org/license/0BSD": "0BSD",
"opensource.org/license/AAL": "AAL",
"opensource.org/license/AFL-3.0": "AFL-3.0",
"opensource.org/license/AGPL-3.0": "AGPL-3.0-or-later",
"opensource.org/license/AGPL-3.0-only": "AGPL-3.0-only",
"opensource.org/license/APL-1.0": "APL-1.0",
"opensource.org/license/Artistic-1.0": "Artistic-1.0-cl8",
"opensource.org/license/Artistic-2.0": "Artistic-2.0",
"opensource.org/license/BSD-2-Clause": "BSD-2-Clause",
"opensource.org/license/BSD-2-Clause-Patent": "BSD-2-Clause-Patent",
"opensource.org/license/BSD-3-Clause": "BSD-3-Clause",
"opensource.org/license/BSL-1.0": "BSL-1.0",
"opensource.org/license/CAL-1.0": "CAL-1.0-Combined-Work-Exception",
"opensource.org/license/CATOSL-1.1": "CATOSL-1.1",
"opensource.org/license/CDDL-1.0": "CDDL-1.0",
"opensource.org/license/CNRI-Python": "CNRI-Python",
"opensource.org/license/CNRI-Python-GPL-Compatible": "CNRI-Python-GPL-Compatible",
"opensource.org/license/CPAL-1.0": "CPAL-1.0",
"opensource.org/license/CPL-1.0": "CPL-1.0",
"opensource.org/license/CUA-OPL-1.0": "CUA-OPL-1.0",
"opensource.org/license/ECL-1.0": "ECL-1.0",
"opensource.org/license/ECL-2.0": "ECL-2.0",
"opensource.org/license/EFL-1.0": "EFL-1.0",
"opensource.org/license/EFL-2.0": "EFL-2.0",
"opensource.org/license/EPL-1.0": "EPL-1.0",
"opensource.org/license/EUDatagrid": "EUDatagrid",
"opensource.org/license/EUPL-1.1": "EUPL-1.1",
"opensource.org/license/EUPL-1.2": "EUPL-1.2",
"opensource.org/license/Entessa": "Entessa",
"opensource.org/license/Fair": "Fair",
"opensource.org/license/Frameworx-1.0": "Frameworx-1.0",
"opensource.org/license/GPL-2.0": "GPL-2.0-or-later",
"opensource.org/license/GPL-2.0-only": "GPL-2.0-only",
"opensource.org/license/GPL-3.0": "GPL-3.0-or-later",
"opensource.org/license/GPL-3.0-only": "GPL-3.0-only",
"opensource.org/license/HPND": "HPND",
"opensource.org/license/IPA": "IPA",
"opensource.org/license/IPL-1.0": "IPL-1.0",
"opensource.org/license/ISC": "ISC",
"opensource.org/license/Intel": "Intel",
"opensource.org/license/LGPL-2.1": "LGPL-2.1-or-later",
"opensource.org/license/LGPL-2.1-only": "LGPL-2.1-only",
"opensource.org/license/LGPL-3.0": "LGPL-3.0-or-later",
"opensource.org/license/LPL-1.0": "LPL-1.0",
"opensource.org/license/LPL-1.02": "LPL-1.02",
"opensource.org/license/LPPL-1.3c": "LPPL-1.3c",
"opensource.org/license/LiLiQ-P-1.1": "LiLiQ-P-1.1",
"opensource.org/license/LiLiQ-R-1.1": "LiLiQ-R-1.1",
"opensource.org/license/LiLiQ-Rplus-1.1": "LiLiQ-Rplus-1.1",
"opensource.org/license/MIT": "MIT",
"opensource.org/license/MPL-1.0": "MPL-1.0",
"opensource.org/license/MPL-1.1": "MPL-1.1",
"opensource.org/license/MPL-2.0": "MPL-2.0-no-copyleft-exception",
"opensource.org/license/MS-PL": "MS-PL",
"opensource.org/license/MS-RL": "MS-RL",
"opensource.org/license/MirOS": "MirOS",
"opensource.org/license/Motosoto": "Motosoto",
"opensource.org/license/Multics": "Multics",
"opensource.org/license/NASA-1.3": "NASA-1.3",
"opensource.org/license/NCSA": "NCSA",
"opensource.org/license/NGPL": "NGPL",
"opensource.org/license/NPOSL-3.0": "NPOSL-3.0",
"opensource.org/license/NTP": "NTP",
"opensource.org/license/Naumen": "Naumen",
"opensource.org/license/Nokia": "Nokia",
"opensource.org/license/OCLC-2.0": "OCLC-2.0",
"opensource.org/license/OFL-1.1": "OFL-1.1-no-RFN",
"opensource.org/license/OGTSL": "OGTSL",
"opensource.org/license/OLFL-1.3": "OLFL-1.3",
"opensource.org/license/OSC-1.0": "OSC-1.0",
"opensource.org/license/OSET-PL-2.1": "OSET-PL-2.1",
"opensource.org/license/OSL-1.0": "OSL-1.0",
"opensource.org/license/OSL-2.1": "OSL-2.1",
"opensource.org/license/OSL-3.0": "OSL-3.0",
"opensource.org/license/PHP-3.0": "PHP-3.0",
"opensource.org/license/PSF-2-0": "PSF-2.0",
"opensource.org/license/PostgreSQL": "PostgreSQL",
"opensource.org/license/Python-2.0": "Python-2.0",
"opensource.org/license/Python-2.0.1": "Python-2.0.1",
"opensource.org/license/QPL-1.0": "QPL-1.0",
"opensource.org/license/RPL-1.1": "RPL-1.1",
"opensource.org/license/RPL-1.5": "RPL-1.5",
"opensource.org/license/RPSL-1.0": "RPSL-1.0",
"opensource.org/license/RSCPL": "RSCPL",
"opensource.org/license/SISSL": "SISSL",
"opensource.org/license/SPL-1.0": "SPL-1.0",
"opensource.org/license/SiMPL-2.0": "SimPL-2.0",
"opensource.org/license/Sleepycat": "Sleepycat",
"opensource.org/license/UCL-1.0": "UCL-1.0",
"opensource.org/license/UPL-1.0": "UPL-1.0",
"opensource.org/license/VSL-1.0": "VSL-1.0",
"opensource.org/license/W3C": "W3C",
"opensource.org/license/Watcom-1.0": "Watcom-1.0",
"opensource.org/license/Xnet": "Xnet",
"opensource.org/license/ZPL-2.0": "ZPL-2.0",
"opensource.org/license/Zlib": "Zlib",
"opensource.org/license/apache-1.1": "Apache-1.1",
"opensource.org/license/apache-2.0": "Apache-2.0",
"opensource.org/license/curl": "curl",
"oss.sgi.com/projects/FreeB/": "SGI-B-1.1",
"oss.sgi.com/projects/FreeB/SGIFreeSWLicB.1.0.html": "SGI-B-1.0",
"oss.sgi.com/projects/FreeB/SGIFreeSWLicB.2.0.pdf": "SGI-B-2.0",
@@ -2082,37 +2131,40 @@ var urlToLicense = map[string]string{
"raw.githubusercontent.com/NTIA/scos-sensor/refs/heads/master/LICENSE.md": "NTIA-PD",
"raw.githubusercontent.com/OpenMDW/OpenMDW/refs/heads/main/1.0/LICENSE.openmdw": "OpenMDW-1.0",
"raw.githubusercontent.com/blackhole89/notekit/master/data/fonts/Charter%20license.txt": "Bitstream-Charter",
"rc.quest.com/topics/openssh/license.php#tcpwrappers": "TCP-wrappers",
"research.cs.wisc.edu/condor/license.html#condor": "Condor-1.1",
"research.scea.com/scea_shared_source_license.html": "SCEA",
"romanrm.net/mit-zero": "MIT-0",
"sam.zoy.org/wtfpl/COPYING": "WTFPL",
"scripts.sil.org/cms/scripts/page.php?item_id=OFL10_web": "OFL-1.0-no-RFN",
"scripts.sil.org/cms/scripts/page.php?item_id=OFL_web": "OFL-1.1-no-RFN",
"slicer.org/LICENSE": "3D-Slicer-1.0",
"solderpad.org/licenses/SHL-0.5/": "SHL-0.5",
"solderpad.org/licenses/SHL-0.51/": "SHL-0.51",
"source.icu-project.org/repos/icu/icu/trunk/license.html": "ICU",
"sourceforge.net/p/buddy/gitcode/ci/master/tree/README": "Buddy",
"sourceforge.net/p/ctrio/git/ci/master/tree/README": "ISC-Veillard",
"sourceforge.net/p/mcj/xfig/ci/master/tree/src/Makefile.am": "Xfig",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/other/pnmtorle.c": "URT-RLE",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/other/rletopnm.c": "URT-RLE",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/ppm/ppmtompeg/jrevdct.c#l1189": "MPEG-SSG",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/editor/pnmstitch.c#l2": "pnmstitch",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/netpbm.c#l8": "HPND-Pbmplus",
"sourceforge.net/p/sbcl/sbcl/ci/master/tree/src/code/loop.lisp": "LOOP",
"sourceforge.net/p/trustedqsl/tqsl/ci/master/tree/LICENSE.txt": "TrustedQSL",
"sourceforge.net/p/xmedcon/code/ci/master/tree/libs/dicom/README": "OFFIS",
"sourceforge.net/p/xmedcon/code/ci/master/tree/libs/ljpg/": "IJG-short",
"sourceforge.net/projects/sevenzip/files/LZMA%20SDK/": "LZMA-SDK-9.22",
"sources.debian.org/copyright/license/debianutils/4.11.2/": "SMAIL-GPL",
"sources.debian.org/src/openmpi/4.1.0-10/ompi/debuggers/msgq_interface.h/?hl=19#L19": "mpi-permissive",
"sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/licenses/BitTorrent?r1=1.1&r2=1.1.1.1&diff_format=s": "BitTorrent-1.0",
"sourceware.org/bzip2/1.0.5/bzip2-manual-1.0.5.html": "bzip2-1.0.6",
"sourceware.org/cgit/binutils-gdb/tree/include/coff/sym.h#n11": "MIPS",
"sourceware.org/cgit/binutils-gdb/tree/include/coff/symconst.h#n11": "ThirdEye",
"sourceware.org/cgit/valgrind/tree/mpi/libmpiwrap.c": "bzip2-1.0.6",
"raw.githubusercontent.com/landtuna/open-mtools/6c04c2882eca7c4313ec267766ea53ac9bfe129b/mdump.c": "Informatica",
"raw.githubusercontent.com/landtuna/open-mtools/6c04c2882eca7c4313ec267766ea53ac9bfe129b/msend.c": "Informatica",
"rc.quest.com/topics/openssh/license.php#tcpwrappers": "TCP-wrappers",
"research.cs.wisc.edu/condor/license.html#condor": "Condor-1.1",
"research.scea.com/scea_shared_source_license.html": "SCEA",
"romanrm.net/mit-zero": "MIT-0",
"sam.zoy.org/wtfpl/COPYING": "WTFPL",
"scancode-licensedb.aboutcode.org/bsd-unchanged": "BSD-2-Clause-pos-unchanged",
"scripts.sil.org/cms/scripts/page.php?item_id=OFL10_web": "OFL-1.0-no-RFN",
"scripts.sil.org/cms/scripts/page.php?item_id=OFL_web": "OFL-1.1-no-RFN",
"slicer.org/LICENSE": "3D-Slicer-1.0",
"solderpad.org/licenses/SHL-0.5/": "SHL-0.5",
"solderpad.org/licenses/SHL-0.51/": "SHL-0.51",
"source.icu-project.org/repos/icu/icu/trunk/license.html": "ICU",
"sourceforge.net/p/buddy/gitcode/ci/master/tree/README": "Buddy",
"sourceforge.net/p/ctrio/git/ci/master/tree/README": "ISC-Veillard",
"sourceforge.net/p/mcj/xfig/ci/master/tree/src/Makefile.am": "Xfig",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/other/pnmtorle.c": "URT-RLE",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/other/rletopnm.c": "URT-RLE",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/converter/ppm/ppmtompeg/jrevdct.c#l1189": "MPEG-SSG",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/editor/pnmstitch.c#l2": "pnmstitch",
"sourceforge.net/p/netpbm/code/HEAD/tree/super_stable/netpbm.c#l8": "HPND-Pbmplus",
"sourceforge.net/p/sbcl/sbcl/ci/master/tree/src/code/loop.lisp": "LOOP",
"sourceforge.net/p/trustedqsl/tqsl/ci/master/tree/LICENSE.txt": "TrustedQSL",
"sourceforge.net/p/xmedcon/code/ci/master/tree/libs/dicom/README": "OFFIS",
"sourceforge.net/p/xmedcon/code/ci/master/tree/libs/ljpg/": "IJG-short",
"sourceforge.net/projects/sevenzip/files/LZMA%20SDK/": "LZMA-SDK-9.22",
"sources.debian.org/copyright/license/debianutils/4.11.2/": "SMAIL-GPL",
"sources.debian.org/src/openmpi/4.1.0-10/ompi/debuggers/msgq_interface.h/?hl=19#L19": "mpi-permissive",
"sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/licenses/BitTorrent?r1=1.1&r2=1.1.1.1&diff_format=s": "BitTorrent-1.0",
"sourceware.org/bzip2/1.0.5/bzip2-manual-1.0.5.html": "bzip2-1.0.6",
"sourceware.org/cgit/binutils-gdb/tree/include/coff/sym.h#n11": "MIPS",
"sourceware.org/cgit/binutils-gdb/tree/include/coff/symconst.h#n11": "ThirdEye",
"sourceware.org/cgit/valgrind/tree/mpi/libmpiwrap.c": "bzip2-1.0.6",
"sourceware.org/git/?p=binutils-gdb.git;a=blob;f=libiberty/strcasecmp.c;h=131d81c2ce7881fa48c363dc5bf5fb302c61ce0b;hb=HEAD": "BSD-4.3RENO",
"sourceware.org/git/?p=binutils-gdb.git;a=blob;f=readline/readline/support/wcwidth.c;h=0f5ec995796f4813abbcf4972aec0378ab74722a;hb=HEAD#l55": "HPND-Markus-Kuhn",
"sourceware.org/git/?p=bzip2.git;a=blob;f=LICENSE;hb=bzip2-1.0.6": "bzip2-1.0.6",
@@ -2129,6 +2181,7 @@ var urlToLicense = map[string]string{
"tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/COPYING.TGPPL.rst": "TGPPL-1.0",
"ti.arc.nasa.gov/opensource/nosa/": "NASA-1.3",
"trac.enlightenment.org/e/browser/trunk/imlib2/COPYING": "Imlib2",
"tunes.org/legalese/bugroff.html": "Bugroff",
"ubuntu.com/legal/font-licence": "Ubuntu-font-1.0",
"unlicense.org/": "Unlicense",
"vimdoc.sourceforge.net/htmldoc/uganda.html": "Vim",
@@ -2139,6 +2192,7 @@ var urlToLicense = map[string]string{
"web.archive.org/web/20041020171434/http://www.rosenlaw.com/osl2.0.html": "OSL-2.0",
"web.archive.org/web/20050212003940/http://www.rosenlaw.com/osl21.htm": "OSL-2.1",
"web.archive.org/web/20060319014854/http://info.borland.com/devsupport/interbase/opensource/IPL.html": "Interbase-1.0",
"web.archive.org/web/20061013013632/http://opensource.linux-mirror.org/licenses/afl-2.1.txt": "AFL-2.1",
"web.archive.org/web/20080207013128/http://www.gnome.org/fonts/": "Bitstream-Vera",
"web.archive.org/web/20100302225219/http://www.zimbra.com/license/zimbra-public-license-1-3.html": "Zimbra-1.3",
"web.archive.org/web/20111123062036/http://research.cs.wisc.edu/condor/license.html#condor": "Condor-1.1",
@@ -2243,11 +2297,11 @@ var urlToLicense = map[string]string{
"www.jython.org/license.html": "CNRI-Jython",
"www.kermitproject.org/ck90.html#source": "HPND-export-US",
"www.kernel.org/doc/man-pages/licenses.html": "Linux-man-pages-copyleft",
"www.latex-project.org/lppl/LPPL-1.3c.txt": "LPPL-1.3c",
"www.latex-project.org/lppl/lppl-1-0.txt": "LPPL-1.0",
"www.latex-project.org/lppl/lppl-1-1.txt": "LPPL-1.1",
"www.latex-project.org/lppl/lppl-1-2.txt": "LPPL-1.2",
"www.latex-project.org/lppl/lppl-1-3a.txt": "LPPL-1.3a",
"www.latex-project.org/lppl/lppl-1-3c.txt": "LPPL-1.3c",
"www.lemis.com/grog/UNIX/ancient-source-all.pdf": "Caldera",
"www.libpng.org/pub/png/src/libpng-LICENSE.txt": "libpng-2.0",
"www.mail-archive.com/debian-bugs-rc@lists.debian.org/msg11494.html": "Symlinks",
@@ -2269,16 +2323,17 @@ var urlToLicense = map[string]string{
"www.netlib.org/lapack/LICENSE.txt": "BSD-3-Clause-Open-MPI",
"www.netlib.org/minpack/disclaimer": "Minpack",
"www.networkcultures.org/_uploads/%233notebook_telekommunist.pdf": "PPL",
"www.oclc.org/research/activities/software/license/v2final.htm": "OCLC-2.0",
"www.ogc.org/ogc/software/1.0": "OGC-1.0",
"www.ohwr.org/project/cernohl/wikis/Documents/CERN-OHL-version-2": "CERN-OHL-W-2.0",
"www.ohwr.org/project/licenses/wikis/cern-ohl-v1.1": "CERN-OHL-1.1",
"www.ohwr.org/project/licenses/wikis/cern-ohl-v1.2": "CERN-OHL-1.2",
"www.open-mpi.org/community/license.php": "BSD-3-Clause-Open-MPI",
"www.opencascade.com/content/occt-public-license": "OCCT-PL",
"www.opendatacommons.org/licenses/odbl/1.0/": "ODbL-1.0",
"www.opengroup.org/testing/downloads/The_Open_Group_TSL.txt": "OGTSL",
"www.openhub.net/licenses/mslpl": "MS-LPL",
"www.nist.gov/open/copyright-fair-use-and-licensing-statements-srd-data-software-and-technical-series-publications": "NIST-Software",
"www.oclc.org/research/activities/software/license/v2final.htm": "OCLC-2.0",
"www.ogc.org/ogc/software/1.0": "OGC-1.0",
"www.ohwr.org/project/cernohl/wikis/Documents/CERN-OHL-version-2": "CERN-OHL-W-2.0",
"www.ohwr.org/project/licenses/wikis/cern-ohl-v1.1": "CERN-OHL-1.1",
"www.ohwr.org/project/licenses/wikis/cern-ohl-v1.2": "CERN-OHL-1.2",
"www.open-mpi.org/community/license.php": "BSD-3-Clause-Open-MPI",
"www.opencascade.com/content/occt-public-license": "OCCT-PL",
"www.opendatacommons.org/licenses/odbl/1.0/": "ODbL-1.0",
"www.opengroup.org/testing/downloads/The_Open_Group_TSL.txt": "OGTSL",
"www.openhub.net/licenses/mslpl": "MS-LPL",
"www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=LICENSE;hb=1cae062821881f41b73012ba816434897abf4205": "OLDAP-2.6",
"www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=LICENSE;hb=37b4b3f6cc4bf34e1d3dec61e69914b9819d8894": "NBPL-1.0",
"www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=LICENSE;hb=42b0383c50c299977b5893ee695cf4e486fb0dc7": "OLDAP-1.2",
@@ -2301,7 +2356,7 @@ var urlToLicense = map[string]string{
"www.opensource.apple.com/license/apsl/": "APSL-2.0",
"www.opensource.apple.com/source/IOSerialFamily/IOSerialFamily-7/APPLE_LICENSE": "APSL-1.1",
"www.opensource.apple.com/source/tcl/tcl-5/tcl/generic/regfronts.c": "Spencer-99",
"www.opensource.org/licenses/EPL-2.0": "EPL-2.0",
"www.opensource.org/license/EPL-2.0": "EPL-2.0",
"www.openssl.org/source/license.html": "OpenSSL",
"www.osetfoundation.org/public-license": "OSET-PL-2.1",
"www.parliament.uk/site-information/copyright-parliament/open-parliament-licence/": "OPL-UK-3.0",
+90
View File
@@ -123,6 +123,96 @@ func TestLicenseByURL_AlternateScheme(t *testing.T) {
}
}
func TestLicenseByURL_OpenSourceOrgVariants(t *testing.T) {
// SPDX has changed the form of opensource.org URLs across list versions (e.g. 3.29.0 moved
// opensource.org/licenses/MIT to opensource.org/license/MIT); every form should keep resolving.
tests := []struct {
name string
url string
wantID string
}{
{name: "old plural path", url: "http://opensource.org/licenses/MIT", wantID: "MIT"},
{name: "new singular path", url: "https://opensource.org/license/MIT", wantID: "MIT"},
{name: "lowercase with trailing slash", url: "https://opensource.org/license/mit/", wantID: "MIT"},
{name: "plural path with trailing slash", url: "https://opensource.org/licenses/MIT/", wantID: "MIT"},
{name: "www prefix", url: "https://www.opensource.org/licenses/MIT", wantID: "MIT"},
{name: "uppercase host", url: "https://WWW.OpenSource.org/licenses/MIT", wantID: "MIT"},
{name: "old plural path BSD-3-Clause", url: "https://opensource.org/licenses/BSD-3-Clause", wantID: "BSD-3-Clause"},
{name: "new singular path BSD-3-Clause", url: "https://opensource.org/license/BSD-3-Clause", wantID: "BSD-3-Clause"},
{name: "lowercase BSD-3-Clause", url: "https://opensource.org/licenses/bsd-3-clause", wantID: "BSD-3-Clause"},
{name: "old plural path Apache-2.0", url: "http://opensource.org/licenses/Apache-2.0", wantID: "Apache-2.0"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
info, found := LicenseByURL(tt.url)
if !found {
t.Fatalf("LicenseByURL(%q) found = false, want true", tt.url)
}
if info.ID != tt.wantID {
t.Errorf("LicenseByURL(%q) ID = %v, want %v", tt.url, info.ID, tt.wantID)
}
})
}
}
func TestLicenseByURL_NonOpenSourceOrgStaysExact(t *testing.T) {
// the relaxed matching is scoped to opensource.org; other hosts may have case-sensitive paths
if _, found := LicenseByURL("https://www.apache.org/licenses/license-2.0"); found {
t.Errorf("LicenseByURL() found = true for case-altered non-opensource.org URL, want false")
}
if _, found := LicenseByURL("https://opensource.org/licenses/not-a-real-license"); found {
t.Errorf("LicenseByURL() found = true for unknown opensource.org URL, want false")
}
}
func TestNormalizeOpenSourceOrgURL(t *testing.T) {
tests := []struct {
url string
want string
wantOK bool
}{
{url: "opensource.org/licenses/MIT", want: "opensource.org/license/mit", wantOK: true},
{url: "opensource.org/license/mit/", want: "opensource.org/license/mit", wantOK: true},
{url: "www.opensource.org/licenses/EPL-2.0", want: "opensource.org/license/epl-2.0", wantOK: true},
{url: "lists.opensource.org/pipermail/license-discuss", wantOK: false},
{url: "www.apache.org/licenses/LICENSE-2.0", wantOK: false},
}
for _, tt := range tests {
t.Run(tt.url, func(t *testing.T) {
got, ok := normalizeOpenSourceOrgURL(tt.url)
if ok != tt.wantOK {
t.Fatalf("normalizeOpenSourceOrgURL(%q) ok = %v, want %v", tt.url, ok, tt.wantOK)
}
if got != tt.want {
t.Errorf("normalizeOpenSourceOrgURL(%q) = %q, want %q", tt.url, got, tt.want)
}
})
}
}
func TestBuildOpenSourceOrgURLToLicense(t *testing.T) {
index := buildOpenSourceOrgURLToLicense(map[string]string{
"opensource.org/licenses/MIT": "MIT",
"opensource.org/license/mit/": "MIT",
"opensource.org/licenses/Foo": "Foo-1.0",
"opensource.org/license/foo": "Foo-2.0",
"opensource.org/license/FOO/": "Foo-1.0",
"www.apache.org/licenses/Bar": "Bar",
})
if got := index["opensource.org/license/mit"]; got != "MIT" {
t.Errorf("index[mit] = %q, want MIT", got)
}
if got, exists := index["opensource.org/license/foo"]; exists {
t.Errorf("index[foo] = %q, want no entry for URL forms mapping to different licenses", got)
}
if len(index) != 1 {
t.Errorf("len(index) = %d, want 1 (non-opensource.org URLs excluded): %v", len(index), index)
}
}
func TestStripScheme(t *testing.T) {
tests := []struct {
name string
+9 -17
View File
@@ -8,12 +8,12 @@ import (
"math/rand"
"os"
"path/filepath"
"runtime"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal/testutils"
"github.com/anchore/syft/internal/tmpdir"
)
@@ -161,13 +161,9 @@ func TestSparsePlacementIsNotAnAllocationKnob(t *testing.T) {
b := newTestBuffer(t, WithMemLimit(1<<20))
mustWrite(t, b, 0, repeat('A', 64))
var before, after runtime.MemStats
runtime.GC()
runtime.ReadMemStats(&before)
mustWrite(t, b, far, repeat('B', 64))
runtime.ReadMemStats(&after)
allocated := after.TotalAlloc - before.TotalAlloc
allocated := testutils.MeasureAlloc(t, func() {
mustWrite(t, b, far, repeat('B', 64))
})
t.Logf("writing 64 bytes at offset %d allocated %d bytes", far, allocated)
assert.Less(t, allocated, uint64(1<<20),
"a far placement must cost its own bytes, not its offset")
@@ -714,15 +710,11 @@ func TestMemoryTierGrowsAmortized(t *testing.T) {
)
b := newTestBuffer(t, WithMemLimit(limit))
var before, after runtime.MemStats
runtime.GC()
runtime.ReadMemStats(&before)
for off := int64(0); off < limit; off += chunk {
mustWrite(t, b, off, repeat('A', chunk))
}
runtime.ReadMemStats(&after)
allocated := after.TotalAlloc - before.TotalAlloc
allocated := testutils.MeasureAlloc(t, func() {
for off := int64(0); off < limit; off += chunk {
mustWrite(t, b, off, repeat('A', chunk))
}
})
t.Logf("filling a %d byte tier in %d byte chunks allocated %d bytes", limit, chunk, allocated)
assert.Less(t, allocated, uint64(4*limit),
"growth has to be amortized, not a fresh copy of the whole tier per write")
+2 -1
View File
@@ -51,7 +51,8 @@ func runTaskSafely(ctx context.Context, t Task, resolver file.Resolver, s sbomsy
// handle individual cataloger panics
defer func() {
if e := recover(); e != nil {
err = fmt.Errorf("%v at:\n%s", e, string(debug.Stack()))
log.WithFields("task", t.Name()).Debugf("task panic stack:\n%s", debug.Stack())
err = fmt.Errorf("panic in task %q: %v", t.Name(), e)
}
}()
+20 -2
View File
@@ -8,8 +8,12 @@ import (
"github.com/wagoodman/go-progress"
"github.com/anchore/syft/internal/sbomsync"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/event/monitor"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
"github.com/anchore/syft/syft/sbom"
)
func Test_TaskExecutor_PanicHandling(t *testing.T) {
@@ -21,6 +25,20 @@ func Test_TaskExecutor_PanicHandling(t *testing.T) {
Manual: progress.NewManual(-1),
})
require.Error(t, err)
require.Contains(t, err.Error(), "executor_test.go")
require.EqualError(t, err, `panic in task "panicking-cataloger": something bad happened`)
}
func Test_RunTask_GenericParserPanicBecomesUnknown(t *testing.T) {
c := generic.NewCataloger("panicking-cataloger").
WithParserByPath(func(context.Context, file.Resolver, *generic.Environment, file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
panic("boom")
}, "executor_test.go")
s := &sbom.SBOM{}
err := RunTask(context.Background(), NewPackageTask(DefaultCatalogingFactoryConfig(), c), file.NewMockResolverForPaths("executor_test.go"), sbomsync.NewBuilder(s), &monitor.TaskProgress{
Manual: progress.NewManual(-1),
})
require.NoError(t, err)
require.Len(t, s.Artifacts.Unknowns, 1)
}
+31
View File
@@ -0,0 +1,31 @@
package testutils
import (
"runtime"
"testing"
)
// MeasureAlloc reports the bytes allocated while fn ran.
//
// This exists for guards whose whole point is "a small input cannot make us reserve a large buffer".
// Only a byte count states that property: asserting that an error comes back would keep passing if the
// allocation were hoisted above the check, which is the bug these guards were written for.
//
// Two things to know when using it:
//
// - TotalAlloc is process-wide, so a test calling this must not call t.Parallel. Another test's
// allocations would land in the measurement.
// - a budget assertion is only meaningful if the fixture would really blow past it unguarded. Prefer
// measuring the unguarded path too and asserting it exceeds the declared size, so a fixture that
// quietly stops being a bomb fails instead of passing.
func MeasureAlloc(t testing.TB, fn func()) uint64 {
t.Helper()
var before, after runtime.MemStats
runtime.GC()
runtime.ReadMemStats(&before)
fn()
runtime.ReadMemStats(&after)
return after.TotalAlloc - before.TotalAlloc
}
+4
View File
@@ -183,6 +183,10 @@ func visitErrors(err error, fn func(error) error) error {
// unwrap singly wrapped errors
if e, ok := err.(interface{ Unwrap() error }); ok {
wrapped := e.Unwrap()
if wrapped == nil {
// nothing is wrapped (e.g. a sync.PanicError with a non-error value), so this is a leaf
return fn(err)
}
got := visitErrors(wrapped, fn)
if got == nil {
return nil
@@ -8,6 +8,7 @@ import (
"github.com/stretchr/testify/require"
"github.com/anchore/go-sync"
"github.com/anchore/syft/syft/file"
)
@@ -234,3 +235,11 @@ func Test_Append(t *testing.T) {
e3 = Appendf(e2, file.NewLocation("l3"), "%s", "e3")
require.Equal(t, "l1: e1\nl3: e3", e3.Error())
}
func Test_ExtractCoordinateErrors_keepsNilUnwrap(t *testing.T) {
// sync.PanicError unwraps to nil when the panic value is not an error
err := sync.PanicError{Value: "x"}
unknowns, remaining := ExtractCoordinateErrors(err)
require.Empty(t, unknowns)
require.Equal(t, err, remaining)
}
+11 -1
View File
@@ -7,6 +7,7 @@ import (
"debug/macho"
"encoding/binary"
"fmt"
"runtime/debug"
"sort"
"github.com/bmatcuk/doublestar/v4"
@@ -89,7 +90,7 @@ func (i *Cataloger) CatalogCtx(ctx context.Context, resolver file.Resolver) (map
return results, errs
}
func processExecutableLocation(loc file.Location, resolver file.Resolver) (*file.Executable, error) {
func processExecutableLocation(loc file.Location, resolver file.Resolver) (exec *file.Executable, err error) {
reader, err := resolver.FileContentsByLocation(loc)
if err != nil {
log.WithFields("error", err, "path", loc.RealPath).Debug("unable to get file contents")
@@ -97,6 +98,15 @@ func processExecutableLocation(loc file.Location, resolver file.Resolver) (*file
}
defer internal.CloseAndLogError(reader, loc.RealPath)
defer func() {
if r := recover(); r != nil {
// a malformed binary must not take down the whole SBOM; the caller records this as an unknown
log.WithFields("path", loc.RealPath, "panic", r).Debug("recovered from panic while reading executable")
log.Tracef("executable panic stack:\n%s", debug.Stack())
exec, err = nil, fmt.Errorf("recovered from panic while reading executable: %v", r)
}
}()
uReader, err := unionreader.GetUnionReader(reader)
if err != nil {
log.WithFields("error", err, "path", loc.RealPath).Debug("unable to get union reader")
@@ -0,0 +1,40 @@
package executable
import (
"context"
"io"
"testing"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal/unknown"
"github.com/anchore/syft/syft/file"
)
type panicReader struct{}
func (panicReader) Read([]byte) (int, error) { panic("boom") }
type panickingContentsResolver struct {
*file.MockResolver
}
func (r panickingContentsResolver) FilesByMIMEType(...string) ([]file.Location, error) {
return r.FilesByPath("cataloger_test.go")
}
func (r panickingContentsResolver) FileContentsByLocation(file.Location) (io.ReadCloser, error) {
return io.NopCloser(panicReader{}), nil
}
func TestCatalogCtx_panicBecomesUnknown(t *testing.T) {
resolver := panickingContentsResolver{file.NewMockResolverForPaths("cataloger_test.go")}
results, err := NewCataloger(DefaultConfig()).CatalogCtx(context.Background(), resolver)
require.Empty(t, results)
unknowns, remaining := unknown.ExtractCoordinateErrors(err)
require.NoError(t, remaining)
require.Len(t, unknowns, 1)
require.ErrorContains(t, unknowns[0].Reason, "recovered from panic while reading executable: boom")
}
+3 -23
View File
@@ -156,32 +156,12 @@ func hasElfDynFlag1(f *elf.File, flag elf.DynFlag1) bool {
}
func hasElfDynTag(f *elf.File, tag elf.DynTag) bool {
// source https://github.com/golang/go/blob/9b4b3e5acca2dabe107fa2c3ed963097d78a4562/src/cmd/cgo/internal/testshared/shared_test.go#L280
ds := f.SectionByType(elf.SHT_DYNAMIC)
if ds == nil {
return false
}
d, err := ds.Data()
vals, err := f.DynValue(tag)
if err != nil {
log.WithFields("error", err).Trace("unable to read dynamic section from elf file")
return false
}
for len(d) > 0 {
var t elf.DynTag
switch f.Class {
case elf.ELFCLASS32:
t = elf.DynTag(f.ByteOrder.Uint32(d[0:4]))
d = d[8:]
case elf.ELFCLASS64:
t = elf.DynTag(f.ByteOrder.Uint64(d[0:8]))
d = d[16:]
}
if t == tag {
return true
}
}
return false
return len(vals) > 0
}
func isELFPIE(f *elf.File) bool {
@@ -1,7 +1,9 @@
package executable
import (
"bytes"
"debug/elf"
"encoding/binary"
"os"
"path/filepath"
"testing"
@@ -226,3 +228,89 @@ func Test_elfHasExports(t *testing.T) {
})
}
}
func Test_hasElfDynTag(t *testing.T) {
dynEntry := func(class elf.Class, tag elf.DynTag) []byte {
if class == elf.ELFCLASS32 {
return binary.LittleEndian.AppendUint32(binary.LittleEndian.AppendUint32(nil, uint32(tag)), 0)
}
return binary.LittleEndian.AppendUint64(binary.LittleEndian.AppendUint64(nil, uint64(tag)), 0)
}
tests := []struct {
name string
class elf.Class
dynamic []byte
want bool
}{
{
name: "32-bit tag present",
class: elf.ELFCLASS32,
dynamic: dynEntry(elf.ELFCLASS32, elf.DT_BIND_NOW),
want: true,
},
{
name: "64-bit tag present",
class: elf.ELFCLASS64,
dynamic: dynEntry(elf.ELFCLASS64, elf.DT_BIND_NOW),
want: true,
},
{
name: "64-bit tag absent",
class: elf.ELFCLASS64,
dynamic: dynEntry(elf.ELFCLASS64, elf.DT_NULL),
want: false,
},
{
name: "32-bit truncated dynamic section",
class: elf.ELFCLASS32,
dynamic: dynEntry(elf.ELFCLASS32, elf.DT_BIND_NOW)[:4],
want: false,
},
{
name: "64-bit truncated dynamic section",
class: elf.ELFCLASS64,
dynamic: dynEntry(elf.ELFCLASS64, elf.DT_BIND_NOW)[:12],
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
f, err := elf.NewFile(bytes.NewReader(elfWithDynamicSection(t, tt.class, tt.dynamic)))
require.NoError(t, err)
assert.Equal(t, tt.want, hasElfDynTag(f, elf.DT_BIND_NOW))
})
}
}
// elfWithDynamicSection builds a little-endian ELF holding only a null section and a SHT_DYNAMIC section.
func elfWithDynamicSection(t *testing.T, class elf.Class, dynamic []byte) []byte {
t.Helper()
ident := [elf.EI_NIDENT]byte{0x7f, 'E', 'L', 'F', byte(class), byte(elf.ELFDATA2LSB), byte(elf.EV_CURRENT)}
var buf bytes.Buffer
write := func(v any) {
require.NoError(t, binary.Write(&buf, binary.LittleEndian, v))
}
if class == elf.ELFCLASS32 {
ehsize, shentsize := binary.Size(elf.Header32{}), binary.Size(elf.Section32{})
dataOff := ehsize + 2*shentsize
write(elf.Header32{
Ident: ident, Type: uint16(elf.ET_DYN), Machine: uint16(elf.EM_386), Version: uint32(elf.EV_CURRENT),
Shoff: uint32(ehsize), Ehsize: uint16(ehsize), Shentsize: uint16(shentsize), Shnum: 2,
})
write(elf.Section32{})
write(elf.Section32{Type: uint32(elf.SHT_DYNAMIC), Off: uint32(dataOff), Size: uint32(len(dynamic)), Addralign: 1})
} else {
ehsize, shentsize := binary.Size(elf.Header64{}), binary.Size(elf.Section64{})
dataOff := ehsize + 2*shentsize
write(elf.Header64{
Ident: ident, Type: uint16(elf.ET_DYN), Machine: uint16(elf.EM_X86_64), Version: uint32(elf.EV_CURRENT),
Shoff: uint64(ehsize), Ehsize: uint16(ehsize), Shentsize: uint16(shentsize), Shnum: 2,
})
write(elf.Section64{})
write(elf.Section64{Type: uint32(elf.SHT_DYNAMIC), Off: uint64(dataOff), Size: uint64(len(dynamic)), Addralign: 1})
}
buf.Write(dynamic)
return buf.Bytes()
}
+12 -2
View File
@@ -5,6 +5,7 @@ import (
"crypto"
"errors"
"fmt"
"runtime/debug"
"github.com/dustin/go-humanize"
@@ -87,7 +88,7 @@ func (i *Cataloger) Catalog(ctx context.Context, resolver file.Resolver, coordin
return results, err
}
func (i *Cataloger) catalogLocation(ctx context.Context, resolver file.Resolver, location file.Location) ([]file.Digest, error) {
func (i *Cataloger) catalogLocation(ctx context.Context, resolver file.Resolver, location file.Location) (digests []file.Digest, err error) {
meta, err := resolver.FileMetadataByLocation(location)
if err != nil {
return nil, err
@@ -104,7 +105,16 @@ func (i *Cataloger) catalogLocation(ctx context.Context, resolver file.Resolver,
}
defer internal.CloseAndLogError(contentReader, location.AccessPath)
digests, err := intFile.NewDigestsFromFile(ctx, contentReader, i.hashes)
defer func() {
if r := recover(); r != nil {
// the caller records this as an unknown for the location instead of failing the whole SBOM
log.WithFields("path", location.RealPath, "panic", r).Debug("recovered from panic while digesting file")
log.Tracef("file digest panic stack:\n%s", debug.Stack())
digests, err = nil, fmt.Errorf("recovered from panic while digesting file: %v", r)
}
}()
digests, err = intFile.NewDigestsFromFile(ctx, contentReader, i.hashes)
if err != nil {
return nil, internal.ErrPath{Context: "digests-cataloger", Path: location.RealPath, Err: err}
}
@@ -15,6 +15,7 @@ import (
stereoscopeFile "github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/stereoscope/pkg/imagetest"
intFile "github.com/anchore/syft/internal/file"
"github.com/anchore/syft/internal/unknown"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/syft/source/directorysource"
@@ -202,3 +203,27 @@ func TestFileDigestCataloger_GivenCoordinates(t *testing.T) {
}
}
type panicReader struct{}
func (panicReader) Read([]byte) (int, error) { panic("boom") }
type panickingContentsResolver struct {
*file.MockResolver
}
func (r panickingContentsResolver) FileContentsByLocation(file.Location) (io.ReadCloser, error) {
return io.NopCloser(panicReader{}), nil
}
func TestDigestsCataloger_panicBecomesUnknown(t *testing.T) {
resolver := panickingContentsResolver{file.NewMockResolverForPaths("cataloger_test.go")}
results, err := NewCataloger([]crypto.Hash{crypto.SHA256}).Catalog(context.Background(), resolver, file.NewLocation("cataloger_test.go").Coordinates)
require.Empty(t, results)
unknowns, remaining := unknown.ExtractCoordinateErrors(err)
require.NoError(t, remaining)
require.Len(t, unknowns, 1)
require.ErrorContains(t, unknowns[0].Reason, "recovered from panic while digesting file: boom")
}
@@ -2,6 +2,7 @@
package spdxhelpers
import (
"cmp"
"crypto/sha1"
"fmt"
"path"
@@ -179,7 +180,8 @@ func toRootRelationships(rootPackage *spdx.Package, packages []*spdx.Package) (o
func toRootPackage(s source.Description) *spdx.Package {
var prefix string
name := s.Name
// the SPDX package name is mandatory, and a source decoded from another SBOM may not carry one
name := cmp.Or(s.Name, imageRepoName(s.Metadata), s.ID, helpers.DocumentName(s))
version := s.Version
var purl *packageurl.PackageURL
@@ -210,7 +212,7 @@ func toRootPackage(s source.Description) *spdx.Package {
checksums = append(checksums, *c)
purl = &packageurl.PackageURL{
Type: "oci",
Name: s.Name,
Name: name,
Version: m.ManifestDigest,
Qualifiers: qualifiers,
}
@@ -240,7 +242,7 @@ func toRootPackage(s source.Description) *spdx.Package {
checksums = append(checksums, *c)
purl = &packageurl.PackageURL{
Type: "oci",
Name: s.Name,
Name: name,
Version: m.ManifestDigest,
Qualifiers: qualifiers,
}
@@ -275,10 +277,6 @@ func toRootPackage(s source.Description) *spdx.Package {
default:
prefix = prefixUnknown
purpose = spdxPrimaryPurposeOther
if name == "" {
name = s.ID
}
}
p := &spdx.Package{
@@ -308,6 +306,25 @@ func toRootPackage(s source.Description) *spdx.Package {
return p
}
// imageRepoName returns the repository name from an image or model user input, the same way the image source names
// itself, so a source decoded from another SBOM (which may only carry the user input) gets a name without the tag.
func imageRepoName(metadata any) string {
var userInput string
switch m := metadata.(type) {
case source.ImageMetadata:
userInput = m.UserInput
case source.OCIModelMetadata:
userInput = m.UserInput
default:
return ""
}
ref, _ := reference.Parse(userInput)
if named, ok := ref.(reference.Named); ok {
return named.Name()
}
return ""
}
func toSPDXID(identifiable artifact.Identifiable) spdx.ElementID {
id := string(identifiable.ID())
if after, ok := strings.CutPrefix(id, "SPDXRef-"); ok {
@@ -581,10 +598,13 @@ func toPackageChecksums(p pkg.Package) ([]spdx.Checksum, bool) {
})
case pkg.OpamPackage:
for _, checksum := range meta.Checksums {
parts := strings.Split(checksum, "=")
algorithm, value, ok := strings.Cut(checksum, "=")
if !ok {
continue
}
checksums = append(checksums, spdx.Checksum{
Algorithm: spdx.ChecksumAlgorithm(strings.ToUpper(parts[0])),
Value: parts[1],
Algorithm: spdx.ChecksumAlgorithm(strings.ToUpper(algorithm)),
Value: value,
})
}
}
@@ -516,6 +516,20 @@ func Test_toPackageChecksums(t *testing.T) {
},
filesAnalyzed: false,
},
{
name: "Opam Package with a checksum missing its algorithm separator",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Metadata: pkg.OpamPackage{
Checksums: []string{"f5f1c0b4", "md5=abc"},
},
},
expected: []spdx.Checksum{
{Algorithm: "MD5", Value: "abc"},
},
filesAnalyzed: false,
},
{
name: "Package with no metadata type",
pkg: pkg.Package{
@@ -1131,3 +1145,49 @@ func Test_otherLicenses(t *testing.T) {
})
}
}
func Test_toRootPackage_name(t *testing.T) {
tests := []struct {
name string
src source.Description
wantName string
wantPurl string
}{
{
name: "source name wins",
src: source.Description{Name: "named", ID: "id", Metadata: source.ImageMetadata{UserInput: "alpine:3.18"}},
wantName: "named",
},
{
// a CycloneDX container component decodes to an image source with only the user input set
name: "image user input without a source name",
src: source.Description{Metadata: source.ImageMetadata{
UserInput: "alpine:3.18",
ManifestDigest: "sha256:0e4b2e0d6b2f3c2b1d3bb3a1c8f1c4d9e6a9b1e8c6f4d2b3a1f0e9d8c7b6a5f4",
}},
wantName: "alpine",
wantPurl: "pkg:oci/alpine@sha256%3A0e4b2e0d6b2f3c2b1d3bb3a1c8f1c4d9e6a9b1e8c6f4d2b3a1f0e9d8c7b6a5f4?arch=&tag=3.18",
},
{
name: "id without a source name",
src: source.Description{ID: "some-id"},
wantName: "some-id",
},
{
name: "nothing to name it by",
src: source.Description{Metadata: source.DirectoryMetadata{}},
wantName: "unknown",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
p := toRootPackage(tt.src)
assert.Equal(t, tt.wantName, p.PackageName)
var purl string
if len(p.PackageExternalReferences) > 0 {
purl = p.PackageExternalReferences[0].Locator
}
assert.Equal(t, tt.wantPurl, purl)
})
}
}
@@ -34,6 +34,8 @@ func ToSyftModel(doc *spdx.Document) (*sbom.SBOM, error) {
return nil, errors.New("cannot convert SPDX document to Syft model because document is nil")
}
removeNilElements(doc)
spdxIDMap := make(map[string]any)
s := &sbom.SBOM{
@@ -55,6 +57,28 @@ func ToSyftModel(doc *spdx.Document) (*sbom.SBOM, error) {
return s, nil
}
// removeNilElements drops null entries from the document's lists. A JSON null (or an element the SPDX version
// converter cannot map) decodes to a nil pointer, and the rest of the conversion dereferences these elements.
func removeNilElements(doc *spdx.Document) {
doc.Packages = withoutNil(doc.Packages)
for _, p := range doc.Packages {
p.Files = withoutNil(p.Files)
p.PackageExternalReferences = withoutNil(p.PackageExternalReferences)
}
doc.Files = withoutNil(doc.Files)
doc.Relationships = withoutNil(doc.Relationships)
}
func withoutNil[T any](items []*T) []*T {
var out []*T
for _, item := range items {
if item != nil {
out = append(out, item)
}
}
return out
}
func isDirectory(name string) bool {
if name == "." || name == ".." || strings.HasSuffix(name, "/") || !strings.Contains(path.Base(name), ".") {
return true
@@ -123,12 +147,24 @@ func extractSource(spdxIDMap map[string]any, doc *spdx.Document) source.Descript
}
p := rootPackages[0]
rootType := syftRootType(p.PackageSPDXIdentifier)
switch p.PrimaryPackagePurpose {
purpose := p.PrimaryPackagePurpose
if purpose == "" {
// SPDX 2.1 and 2.2 have no primary package purpose, so fall back to the source type syft puts in the root's ID
purpose = syftRootPurpose(rootType)
}
switch purpose {
case spdxPrimaryPurposeContainer:
src = containerSource(p)
case spdxPrimaryPurposeFile:
src = fileSource(p)
case spdxPrimaryPurposeOther:
// syft writes a root package even for a source it knows nothing about, which is not a package in the SBOM
if rootType != prefixUnknown {
return src
}
default:
return src
}
@@ -141,6 +177,30 @@ func extractSource(spdxIDMap map[string]any, doc *spdx.Document) source.Descript
return src
}
// syftRootType returns the source type syft encodes in a root package ID ("DocumentRoot-<type>-<name>"), or "" for
// any other ID.
func syftRootType(id spdx.ElementID) string {
rest, ok := strings.CutPrefix(string(id), "DocumentRoot-")
if !ok {
return ""
}
rootType, _, _ := strings.Cut(rest, "-")
return rootType
}
// syftRootPurpose returns the primary package purpose syft writes for a root package of the given source type.
func syftRootPurpose(rootType string) string {
switch rootType {
case prefixImage, prefixOCIModel, prefixSnap:
return spdxPrimaryPurposeContainer
case prefixDirectory, prefixFile:
return spdxPrimaryPurposeFile
case prefixUnknown:
return spdxPrimaryPurposeOther
}
return ""
}
func containerSource(p *spdx.Package) source.Description {
id := string(p.PackageSPDXIdentifier)
@@ -201,7 +261,7 @@ func fileSource(p *spdx.Package) source.Description {
}
supplier := ""
if p.PackageSupplier.Supplier != helpers.NOASSERTION {
if p.PackageSupplier != nil && p.PackageSupplier.Supplier != helpers.NOASSERTION {
supplier = p.PackageSupplier.Supplier
}
+5 -1
View File
@@ -7,6 +7,7 @@ import (
"io"
"strings"
syftinternal "github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/format/internal"
@@ -61,7 +62,7 @@ func toSyftModel(r io.Reader) (*sbom.SBOM, error) {
var errs []error
pkgs := pkg.NewCollection()
scanner := bufio.NewScanner(r)
scanner := syftinternal.NewLineScanner(r)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
@@ -86,6 +87,9 @@ func toSyftModel(r io.Reader) (*sbom.SBOM, error) {
p.SetID()
pkgs.Add(p)
}
if err := scanner.Err(); err != nil {
errs = append(errs, err)
}
return &sbom.SBOM{
Artifacts: sbom.Artifacts{
+39
View File
@@ -114,6 +114,45 @@ func TestCycloneDxImageEncoder(t *testing.T) {
)
}
func TestCycloneDxSourcePackageEncoder(t *testing.T) {
dir := t.TempDir()
testutil.AssertEncoderAgainstGoldenSnapshot(t,
testutil.EncoderSnapshotTestConfig{
Subject: testutil.SourcePackagesInput(t, dir),
Format: getEncoder(t),
UpdateSnapshot: *updateSnapshot,
PersistRedactionsInSnapshot: true,
IsJSON: true,
Redactor: redactor(dir),
},
)
}
func TestCycloneDxSourcePackageEncoder_SpecVersions(t *testing.T) {
s := testutil.SourcePackagesInput(t, t.TempDir())
tests := []struct {
version string
want bool
}{
{version: "1.4", want: false},
{version: "1.5", want: false},
{version: "1.6", want: true},
{version: cyclonedxutil.DefaultVersion, want: true},
}
for _, tt := range tests {
t.Run(tt.version, func(t *testing.T) {
enc, err := NewFormatEncoderWithConfig(EncoderConfig{Version: tt.version})
require.NoError(t, err)
var buf bytes.Buffer
require.NoError(t, enc.Encode(&buf, s))
// the source-distribution reference type only exists from spec 1.6
assert.Equal(t, tt.want, strings.Contains(buf.String(), `"source-distribution"`))
})
}
}
func redactor(values ...string) testutil.Redactor {
return testutil.NewRedactions().
WithValuesRedacted(values...).
@@ -0,0 +1,275 @@
{
"$schema": "http://cyclonedx.org/schema/bom-1.7.schema.json",
"bomFormat": "CycloneDX",
"specVersion": "1.7",
"serialNumber": "urn:uuid:redacted",
"version": 1,
"metadata": {
"timestamp": "timestamp:redacted",
"tools": {
"components": [
{
"type": "application",
"author": "anchore",
"name": "syft",
"version": "v0.42.0-bogus"
}
]
},
"component": {
"bom-ref": "redacted",
"type": "file",
"name": "some/path"
}
},
"components": [
{
"bom-ref": "pkg:deb/debian/attr@1%3A2.4.47-2%2Bb1?arch=amd64&distro=debian-12&package-id=5cc0e8979d7f5115&upstream=attr%401%3A2.4.47-2",
"type": "library",
"name": "attr",
"version": "1:2.4.47-2+b1",
"purl": "pkg:deb/debian/attr@1%3A2.4.47-2%2Bb1?arch=amd64&distro=debian-12&upstream=attr%401%3A2.4.47-2",
"externalReferences": [
{
"url": "pkg:deb/debian/attr@1%3A2.4.47-2?arch=source&distro=debian-12",
"type": "source-distribution"
}
],
"properties": [
{
"name": "syft:package:foundBy",
"value": "dpkg-db-cataloger"
},
{
"name": "syft:package:type",
"value": "deb"
},
{
"name": "syft:package:metadataType",
"value": "dpkg-db-entry"
},
{
"name": "syft:location:0:path",
"value": "/var/lib/dpkg/status"
},
{
"name": "syft:metadata:installedSize",
"value": "0"
},
{
"name": "syft:metadata:source",
"value": "attr"
},
{
"name": "syft:metadata:sourceVersion",
"value": "1:2.4.47-2"
}
]
},
{
"bom-ref": "pkg:rpm/centos/bind-export-libs@9.11.13-3.el8?arch=x86_64&distro=centos-8&epoch=32&package-id=6008fca05ca7b418&upstream=bind-9.11.13-3.el8.src.rpm",
"type": "library",
"name": "bind-export-libs",
"version": "32:9.11.13-3.el8",
"purl": "pkg:rpm/centos/bind-export-libs@9.11.13-3.el8?arch=x86_64&distro=centos-8&epoch=32&upstream=bind-9.11.13-3.el8.src.rpm",
"externalReferences": [
{
"url": "pkg:rpm/centos/bind@9.11.13-3.el8?arch=src&distro=centos-8&epoch=32",
"type": "source-distribution"
}
],
"properties": [
{
"name": "syft:package:foundBy",
"value": "rpm-db-cataloger"
},
{
"name": "syft:package:type",
"value": "rpm"
},
{
"name": "syft:package:metadataType",
"value": "rpm-db-entry"
},
{
"name": "syft:location:0:path",
"value": "/var/lib/rpm/Packages"
},
{
"name": "syft:metadata:epoch",
"value": "32"
},
{
"name": "syft:metadata:release",
"value": "3.el8"
},
{
"name": "syft:metadata:size",
"value": "0"
},
{
"name": "syft:metadata:sourceRpm",
"value": "bind-9.11.13-3.el8.src.rpm"
}
]
},
{
"bom-ref": "pkg:apk/alpine/busybox@1.35.0-r17?arch=x86_64&distro=alpine-3.16.3&package-id=aa74af1190de1c40",
"type": "library",
"name": "busybox",
"version": "1.35.0-r17",
"purl": "pkg:apk/alpine/busybox@1.35.0-r17?arch=x86_64&distro=alpine-3.16.3",
"properties": [
{
"name": "syft:package:foundBy",
"value": "apk-db-cataloger"
},
{
"name": "syft:package:type",
"value": "apk"
},
{
"name": "syft:package:metadataType",
"value": "apk-db-entry"
},
{
"name": "syft:location:0:path",
"value": "/lib/apk/db/installed"
},
{
"name": "syft:metadata:installedSize",
"value": "0"
},
{
"name": "syft:metadata:originPackage",
"value": "busybox"
},
{
"name": "syft:metadata:size",
"value": "0"
}
]
},
{
"bom-ref": "pkg:alpm/arch/gcc-libs@13.2.1-3?arch=x86_64&distro=arch-rolling&package-id=58a2fcf749a9a3cc&upstream=gcc",
"type": "library",
"name": "gcc-libs",
"version": "13.2.1-3",
"purl": "pkg:alpm/arch/gcc-libs@13.2.1-3?arch=x86_64&distro=arch-rolling&upstream=gcc",
"properties": [
{
"name": "syft:package:foundBy",
"value": "alpm-db-cataloger"
},
{
"name": "syft:package:type",
"value": "alpm"
},
{
"name": "syft:package:metadataType",
"value": "alpm-db-entry"
},
{
"name": "syft:location:0:path",
"value": "/var/lib/pacman/local/gcc-libs-13.2.1-3/desc"
},
{
"name": "syft:metadata:architecture",
"value": "x86_64"
},
{
"name": "syft:metadata:basepackage",
"value": "gcc"
},
{
"name": "syft:metadata:package",
"value": "gcc-libs"
},
{
"name": "syft:metadata:size",
"value": "0"
},
{
"name": "syft:metadata:version",
"value": "13.2.1-3"
}
]
},
{
"bom-ref": "pkg:apk/alpine/libc-utils@0.7.2-r3?arch=x86_64&distro=alpine-3.16.3&package-id=16a8c7b19fde6f4f&upstream=libc-dev",
"type": "library",
"name": "libc-utils",
"version": "0.7.2-r3",
"purl": "pkg:apk/alpine/libc-utils@0.7.2-r3?arch=x86_64&distro=alpine-3.16.3&upstream=libc-dev",
"properties": [
{
"name": "syft:package:foundBy",
"value": "apk-db-cataloger"
},
{
"name": "syft:package:type",
"value": "apk"
},
{
"name": "syft:package:metadataType",
"value": "apk-db-entry"
},
{
"name": "syft:location:0:path",
"value": "/lib/apk/db/installed"
},
{
"name": "syft:metadata:installedSize",
"value": "0"
},
{
"name": "syft:metadata:originPackage",
"value": "libc-dev"
},
{
"name": "syft:metadata:size",
"value": "0"
}
]
},
{
"bom-ref": "pkg:deb/debian/libpam-runtime@1.5.2-6%2Bdeb12u1?arch=all&distro=debian-12&package-id=7b4ddc97df57aec4&upstream=pam",
"type": "library",
"name": "libpam-runtime",
"version": "1.5.2-6+deb12u1",
"purl": "pkg:deb/debian/libpam-runtime@1.5.2-6%2Bdeb12u1?arch=all&distro=debian-12&upstream=pam",
"externalReferences": [
{
"url": "pkg:deb/debian/pam@1.5.2-6%2Bdeb12u1?arch=source&distro=debian-12",
"type": "source-distribution"
}
],
"properties": [
{
"name": "syft:package:foundBy",
"value": "dpkg-db-cataloger"
},
{
"name": "syft:package:type",
"value": "deb"
},
{
"name": "syft:package:metadataType",
"value": "dpkg-db-entry"
},
{
"name": "syft:location:0:path",
"value": "/var/lib/dpkg/status"
},
{
"name": "syft:metadata:installedSize",
"value": "0"
},
{
"name": "syft:metadata:source",
"value": "pam"
}
]
}
]
}
+14
View File
@@ -87,6 +87,20 @@ func TestCycloneDxImageEncoder(t *testing.T) {
)
}
func TestCycloneDxSourcePackageEncoder(t *testing.T) {
dir := t.TempDir()
testutil.AssertEncoderAgainstGoldenSnapshot(t,
testutil.EncoderSnapshotTestConfig{
Subject: testutil.SourcePackagesInput(t, dir),
Format: getEncoder(t),
UpdateSnapshot: *updateSnapshot,
PersistRedactionsInSnapshot: true,
IsJSON: false,
Redactor: redactor(dir),
},
)
}
func redactor(values ...string) testutil.Redactor {
return testutil.NewRedactions().
WithValuesRedacted(values...).
@@ -0,0 +1,121 @@
<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.7" serialNumber="redacted" version="1">
<metadata>
<timestamp>redacted</timestamp>
<tools>
<components>
<component type="application">
<author>anchore</author>
<name>syft</name>
<version>v0.42.0-bogus</version>
</component>
</components>
</tools>
<component bom-ref="redacted" type="file">
<name>some/path</name>
</component>
</metadata>
<components>
<component bom-ref="pkg:deb/debian/attr@1%3A2.4.47-2%2Bb1?arch=amd64&amp;distro=debian-12&amp;package-id=5cc0e8979d7f5115&amp;upstream=attr%401%3A2.4.47-2" type="library">
<name>attr</name>
<version>1:2.4.47-2+b1</version>
<purl>pkg:deb/debian/attr@1%3A2.4.47-2%2Bb1?arch=amd64&amp;distro=debian-12&amp;upstream=attr%401%3A2.4.47-2</purl>
<externalReferences>
<reference type="source-distribution">
<url>pkg:deb/debian/attr@1%3A2.4.47-2?arch=source&amp;distro=debian-12</url>
</reference>
</externalReferences>
<properties>
<property name="syft:package:foundBy">dpkg-db-cataloger</property>
<property name="syft:package:type">deb</property>
<property name="syft:package:metadataType">dpkg-db-entry</property>
<property name="syft:location:0:path">/var/lib/dpkg/status</property>
<property name="syft:metadata:installedSize">0</property>
<property name="syft:metadata:source">attr</property>
<property name="syft:metadata:sourceVersion">1:2.4.47-2</property>
</properties>
</component>
<component bom-ref="pkg:rpm/centos/bind-export-libs@9.11.13-3.el8?arch=x86_64&amp;distro=centos-8&amp;epoch=32&amp;package-id=6008fca05ca7b418&amp;upstream=bind-9.11.13-3.el8.src.rpm" type="library">
<name>bind-export-libs</name>
<version>32:9.11.13-3.el8</version>
<purl>pkg:rpm/centos/bind-export-libs@9.11.13-3.el8?arch=x86_64&amp;distro=centos-8&amp;epoch=32&amp;upstream=bind-9.11.13-3.el8.src.rpm</purl>
<externalReferences>
<reference type="source-distribution">
<url>pkg:rpm/centos/bind@9.11.13-3.el8?arch=src&amp;distro=centos-8&amp;epoch=32</url>
</reference>
</externalReferences>
<properties>
<property name="syft:package:foundBy">rpm-db-cataloger</property>
<property name="syft:package:type">rpm</property>
<property name="syft:package:metadataType">rpm-db-entry</property>
<property name="syft:location:0:path">/var/lib/rpm/Packages</property>
<property name="syft:metadata:epoch">32</property>
<property name="syft:metadata:release">3.el8</property>
<property name="syft:metadata:size">0</property>
<property name="syft:metadata:sourceRpm">bind-9.11.13-3.el8.src.rpm</property>
</properties>
</component>
<component bom-ref="pkg:apk/alpine/busybox@1.35.0-r17?arch=x86_64&amp;distro=alpine-3.16.3&amp;package-id=aa74af1190de1c40" type="library">
<name>busybox</name>
<version>1.35.0-r17</version>
<purl>pkg:apk/alpine/busybox@1.35.0-r17?arch=x86_64&amp;distro=alpine-3.16.3</purl>
<properties>
<property name="syft:package:foundBy">apk-db-cataloger</property>
<property name="syft:package:type">apk</property>
<property name="syft:package:metadataType">apk-db-entry</property>
<property name="syft:location:0:path">/lib/apk/db/installed</property>
<property name="syft:metadata:installedSize">0</property>
<property name="syft:metadata:originPackage">busybox</property>
<property name="syft:metadata:size">0</property>
</properties>
</component>
<component bom-ref="pkg:alpm/arch/gcc-libs@13.2.1-3?arch=x86_64&amp;distro=arch-rolling&amp;package-id=58a2fcf749a9a3cc&amp;upstream=gcc" type="library">
<name>gcc-libs</name>
<version>13.2.1-3</version>
<purl>pkg:alpm/arch/gcc-libs@13.2.1-3?arch=x86_64&amp;distro=arch-rolling&amp;upstream=gcc</purl>
<properties>
<property name="syft:package:foundBy">alpm-db-cataloger</property>
<property name="syft:package:type">alpm</property>
<property name="syft:package:metadataType">alpm-db-entry</property>
<property name="syft:location:0:path">/var/lib/pacman/local/gcc-libs-13.2.1-3/desc</property>
<property name="syft:metadata:architecture">x86_64</property>
<property name="syft:metadata:basepackage">gcc</property>
<property name="syft:metadata:package">gcc-libs</property>
<property name="syft:metadata:size">0</property>
<property name="syft:metadata:version">13.2.1-3</property>
</properties>
</component>
<component bom-ref="pkg:apk/alpine/libc-utils@0.7.2-r3?arch=x86_64&amp;distro=alpine-3.16.3&amp;package-id=16a8c7b19fde6f4f&amp;upstream=libc-dev" type="library">
<name>libc-utils</name>
<version>0.7.2-r3</version>
<purl>pkg:apk/alpine/libc-utils@0.7.2-r3?arch=x86_64&amp;distro=alpine-3.16.3&amp;upstream=libc-dev</purl>
<properties>
<property name="syft:package:foundBy">apk-db-cataloger</property>
<property name="syft:package:type">apk</property>
<property name="syft:package:metadataType">apk-db-entry</property>
<property name="syft:location:0:path">/lib/apk/db/installed</property>
<property name="syft:metadata:installedSize">0</property>
<property name="syft:metadata:originPackage">libc-dev</property>
<property name="syft:metadata:size">0</property>
</properties>
</component>
<component bom-ref="pkg:deb/debian/libpam-runtime@1.5.2-6%2Bdeb12u1?arch=all&amp;distro=debian-12&amp;package-id=7b4ddc97df57aec4&amp;upstream=pam" type="library">
<name>libpam-runtime</name>
<version>1.5.2-6+deb12u1</version>
<purl>pkg:deb/debian/libpam-runtime@1.5.2-6%2Bdeb12u1?arch=all&amp;distro=debian-12&amp;upstream=pam</purl>
<externalReferences>
<reference type="source-distribution">
<url>pkg:deb/debian/pam@1.5.2-6%2Bdeb12u1?arch=source&amp;distro=debian-12</url>
</reference>
</externalReferences>
<properties>
<property name="syft:package:foundBy">dpkg-db-cataloger</property>
<property name="syft:package:type">deb</property>
<property name="syft:package:metadataType">dpkg-db-entry</property>
<property name="syft:location:0:path">/var/lib/dpkg/status</property>
<property name="syft:metadata:installedSize">0</property>
<property name="syft:metadata:source">pam</property>
</properties>
</component>
</components>
</bom>
@@ -2,6 +2,7 @@ package cyclonedxutil
import (
"io"
"slices"
"github.com/CycloneDX/cyclonedx-go"
@@ -29,9 +30,35 @@ func NewEncoder(version string, format cyclonedx.BOMFileFormat, pretty bool) (En
func (e Encoder) Encode(writer io.Writer, s sbom.SBOM) error {
bom := cyclonedxhelpers.ToFormatModel(s)
if e.version < cyclonedx.SpecVersion1_6 {
// cyclonedx-go does not downgrade this 1.6+ reference type for older spec versions
// (see https://github.com/CycloneDX/cyclonedx-go/issues/291), so drop it rather than emit an invalid document
dropExternalReferenceType(bom.Components, cyclonedx.ERTypeSourceDistribution)
}
enc := cyclonedx.NewBOMEncoder(writer, e.format)
enc.SetPretty(e.pretty)
enc.SetEscapeHTML(false)
return enc.EncodeVersion(bom, e.version)
}
func dropExternalReferenceType(components *[]cyclonedx.Component, refType cyclonedx.ExternalReferenceType) {
if components == nil {
return
}
for i := range *components {
c := &(*components)[i]
dropExternalReferenceType(c.Components, refType)
if c.ExternalReferences == nil {
continue
}
refs := slices.DeleteFunc(*c.ExternalReferences, func(r cyclonedx.ExternalReference) bool {
return r.Type == refType
})
if len(refs) == 0 {
c.ExternalReferences = nil
continue
}
c.ExternalReferences = &refs
}
}
@@ -7,6 +7,7 @@ import (
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal/file"
syftFile "github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
@@ -79,12 +80,162 @@ func encodeExternalReferences(p pkg.Package) *[]cyclonedx.ExternalReference {
}
}
}
if srcRef := encodeSourcePackageExternalReference(p); srcRef != nil {
refs = append(refs, *srcRef)
}
if len(refs) > 0 {
return &refs
}
return nil
}
// encodeSourcePackageExternalReference returns a "source-distribution" reference holding the PURL of the distro
// source package that a binary OS package was built from (e.g. libpam-runtime -> pkg:deb/debian/pam@...?arch=source).
// It is only emitted when the source differs from the binary package (by name or version) and when the package PURL
// is of the type implied by the metadata. The main consumer is vulnerability matching, since distros tend to report
// vulnerabilities against the source package rather than each binary package built from it.
//
// why an external reference and not pedigree: `component.pedigree.ancestors` looks like the obvious spot for "this
// binary came from that source", but pedigree describes code lineage (forks, patched variants, and the patches that
// separate them). A binary built unmodified from its own distro source package is the same code in a different form,
// not a fork, and CycloneDX components are deliberately agnostic to source vs binary form (see the comments from the
// CycloneDX maintainers in https://github.com/CycloneDX/specification/issues/612#issuecomment-2958800363). The
// ancestor slot is better left for the upstream project a distro patched (e.g. Linux-PAM for Debian's pam), which
// syft does not know today.
//
// why "source-distribution": the spec defines it as "the location where the source code distributable can be
// obtained", and a deb/rpm source package is exactly that distributable (a .dsc and tarballs in the Debian source
// pool, a .src.rpm in an SRPMS repo). The value is a PURL (an identifier) rather than a download URL, which the
// schema allows since external reference URLs are URIs of any scheme, and a CycloneDX maintainer confirmed this exact
// shape is valid in https://github.com/CycloneDX/specification/issues/612#issuecomment-5428057874. A first-class
// "source" component type was discussed in that issue and deferred to CycloneDX 2.0.
func encodeSourcePackageExternalReference(p pkg.Package) *cyclonedx.ExternalReference {
if p.PURL == "" {
return nil
}
src := sourcePackageOf(p)
if src == nil {
return nil
}
// the binary PURL is only consulted for the namespace and a small set of qualifiers (e.g. distro), everything
// else describes the binary package and must not be restated as a fact about the source package.
binPurl, err := packageurl.FromString(p.PURL)
if err != nil || binPurl.Type != src.purlType {
return nil
}
qualifiers := map[string]string{}
for _, q := range binPurl.Qualifiers {
if src.keepQualifier(q.Key) {
qualifiers[q.Key] = q.Value
}
}
if src.arch != "" {
qualifiers[pkg.PURLQualifierArch] = src.arch
}
srcPurl := packageurl.NewPackageURL(
src.purlType,
binPurl.Namespace,
src.name,
src.version,
packageurl.QualifiersFromMap(qualifiers),
"",
)
return &cyclonedx.ExternalReference{
Type: cyclonedx.ERTypeSourceDistribution,
URL: srcPurl.ToString(),
}
}
// sourcePackage is the identity of the source package a binary OS package was built from.
type sourcePackage struct {
purlType string
name string
version string
// arch is the ecosystem's source architecture marker ("source" for deb, "src" or "nosrc" for rpm)
arch string
}
func (s sourcePackage) keepQualifier(key string) bool {
switch key {
case pkg.PURLQualifierDistro:
return true
case pkg.PURLQualifierEpoch, pkg.PURLQualifierRpmModularity:
return s.purlType == pkg.RpmPkg.PackageURLType()
}
return false
}
// sourcePackageOf returns the source package for p, or nil when there is none or it is the same as the binary.
//
// only deb and rpm are supported, since those are the ecosystems that publish real source packages with their own
// PURL identity (arch=source / arch=src). apk (origin) and alpm (pkgbase) are intentionally left out: there the
// "source" is a build recipe rather than a published source distributable, and its name is usually also the name of
// a real binary package (e.g. pkg:apk/alpine/libc-dev@0.7.2-r3 is an installable package, not a source archive), so
// a "source-distribution" reference would point at a binary package. Whether those ecosystems have a meaningful
// source PURL is still an open question, and until then the `upstream` PURL qualifier continues to carry that info.
func sourcePackageOf(p pkg.Package) *sourcePackage {
var src sourcePackage
var binName, binVersion string
switch m := p.Metadata.(type) {
case pkg.DpkgDBEntry:
src = dpkgSourcePackage(m)
binName, binVersion = m.Package, m.Version
case pkg.DpkgArchiveEntry:
src = dpkgSourcePackage(pkg.DpkgDBEntry(m))
binName, binVersion = m.Package, m.Version
case pkg.RpmDBEntry:
src = rpmSourcePackage(m.SourceRpm)
binName, binVersion = m.Name, m.Version+"-"+m.Release
case pkg.RpmArchive:
src = rpmSourcePackage(m.SourceRpm)
binName, binVersion = m.Name, m.Version+"-"+m.Release
default:
return nil
}
if src.name == "" || src.version == "" {
return nil
}
if src.name == binName && src.version == binVersion {
return nil
}
return &src
}
func dpkgSourcePackage(entry pkg.DpkgDBEntry) sourcePackage {
version := entry.SourceVersion
if version == "" {
version = entry.Version
}
return sourcePackage{purlType: pkg.DebPkg.PackageURLType(), name: entry.Source, version: version, arch: "source"}
}
// rpmSourcePackage parses a source RPM filename of the form <name>-<version>-<release>.(no)src.rpm.
func rpmSourcePackage(sourceRpm string) sourcePackage {
var arch string
switch {
case strings.HasSuffix(sourceRpm, ".src.rpm"):
arch = "src"
case strings.HasSuffix(sourceRpm, ".nosrc.rpm"):
arch = "nosrc"
default:
return sourcePackage{}
}
nvr := strings.TrimSuffix(sourceRpm, "."+arch+".rpm")
release := strings.LastIndex(nvr, "-")
if release < 0 {
return sourcePackage{}
}
ver := strings.LastIndex(nvr[:release], "-")
if ver < 0 {
return sourcePackage{}
}
return sourcePackage{purlType: pkg.RpmPkg.PackageURLType(), name: nvr[:ver], version: nvr[ver+1:], arch: arch}
}
// supported algorithm in cycloneDX as of 1.4
// "MD5", "SHA-1", "SHA-256", "SHA-384", "SHA-512",
// "SHA3-256", "SHA3-384", "SHA3-512", "BLAKE2b-256", "BLAKE2b-384", "BLAKE2b-512", "BLAKE3"
@@ -144,6 +144,342 @@ func Test_encodeExternalReferences(t *testing.T) {
}
}
func Test_encodeSourcePackage(t *testing.T) {
rpmEpoch := 32
tests := []struct {
name string
input pkg.Package
expected *cyclonedx.ExternalReference
}{
{
name: "no metadata",
input: pkg.Package{},
expected: nil,
},
{
name: "no PURL",
input: pkg.Package{
Name: "libpam-runtime",
Version: "1.5.2-6+deb12u1",
Type: pkg.DebPkg,
Metadata: pkg.DpkgDBEntry{
Package: "libpam-runtime",
Version: "1.5.2-6+deb12u1",
Source: "pam",
},
},
expected: nil,
},
{
// apk origins are build recipes, not published source packages, so they are not supported
name: "from apk is not supported",
input: pkg.Package{
Name: "libc-utils",
Version: "0.7.2-r3",
Type: pkg.ApkPkg,
PURL: "pkg:apk/alpine/libc-utils@0.7.2-r3?arch=x86_64&distro=alpine-3.16.3&upstream=libc-dev",
Metadata: pkg.ApkDBEntry{
Package: "libc-utils",
OriginPackage: "libc-dev",
Version: "0.7.2-r3",
Architecture: "x86_64",
},
},
expected: nil,
},
{
name: "from deb",
input: pkg.Package{
Name: "libpam-runtime",
Version: "1.5.2-6+deb12u1",
Type: pkg.DebPkg,
PURL: "pkg:deb/debian/libpam-runtime@1.5.2-6%2Bdeb12u1?arch=all&distro=debian-12&upstream=pam",
Metadata: pkg.DpkgDBEntry{
Package: "libpam-runtime",
Version: "1.5.2-6+deb12u1",
Source: "pam",
Architecture: "all",
},
},
expected: &cyclonedx.ExternalReference{
Type: cyclonedx.ERTypeSourceDistribution, URL: "pkg:deb/debian/pam@1.5.2-6%2Bdeb12u1?arch=source&distro=debian-12",
},
},
{
name: "from deb with source version",
input: pkg.Package{
Name: "attr",
Version: "1:2.4.47-2+b1",
Type: pkg.DebPkg,
PURL: "pkg:deb/debian/attr@1%3A2.4.47-2%2Bb1?arch=amd64&distro=debian-12&upstream=attr%401%3A2.4.47-2",
Metadata: pkg.DpkgDBEntry{
Package: "attr",
Version: "1:2.4.47-2+b1",
Source: "attr",
SourceVersion: "1:2.4.47-2",
Architecture: "amd64",
},
},
expected: &cyclonedx.ExternalReference{
Type: cyclonedx.ERTypeSourceDistribution, URL: "pkg:deb/debian/attr@1%3A2.4.47-2?arch=source&distro=debian-12",
},
},
{
name: "from deb with empty source",
input: pkg.Package{
Name: "some-package",
Version: "1.0.0",
Type: pkg.DebPkg,
PURL: "pkg:deb/debian/some-package@1.0.0?arch=all&distro=debian-12",
Metadata: pkg.DpkgDBEntry{
Package: "some-package",
Version: "1.0.0",
Architecture: "all",
},
},
expected: nil,
},
{
name: "from deb archive",
input: pkg.Package{
Name: "libpam-runtime",
Version: "1.5.2-6+deb12u1",
Type: pkg.DebPkg,
PURL: "pkg:deb/libpam-runtime@1.5.2-6%2Bdeb12u1?arch=all&upstream=pam",
Metadata: pkg.DpkgArchiveEntry{
Package: "libpam-runtime",
Version: "1.5.2-6+deb12u1",
Source: "pam",
Architecture: "all",
},
},
expected: &cyclonedx.ExternalReference{
Type: cyclonedx.ERTypeSourceDistribution, URL: "pkg:deb/pam@1.5.2-6%2Bdeb12u1?arch=source",
},
},
{
name: "from rpm",
input: pkg.Package{
Name: "bind-export-libs",
Version: "32:9.11.13-3.el8",
Type: pkg.RpmPkg,
PURL: "pkg:rpm/centos/bind-export-libs@9.11.13-3.el8?arch=x86_64&distro=centos-8&epoch=32&upstream=bind-9.11.13-3.el8.src.rpm",
Metadata: pkg.RpmDBEntry{
Name: "bind-export-libs",
Version: "9.11.13",
Release: "3.el8",
Epoch: &rpmEpoch,
Arch: "x86_64",
SourceRpm: "bind-9.11.13-3.el8.src.rpm",
},
},
expected: &cyclonedx.ExternalReference{
Type: cyclonedx.ERTypeSourceDistribution, URL: "pkg:rpm/centos/bind@9.11.13-3.el8?arch=src&distro=centos-8&epoch=32",
},
},
{
name: "from rpm with source = bin",
input: pkg.Package{
Name: "some-package",
Version: "32:1.0.0-1.el8",
Type: pkg.RpmPkg,
PURL: "pkg:rpm/centos/some-package@1.0.0-1.el8?arch=x86_64&distro=centos-8&epoch=32&upstream=some-package-1.0.0-1.el8.src.rpm",
Metadata: pkg.RpmDBEntry{
Name: "some-package",
Version: "1.0.0",
Release: "1.el8",
Epoch: &rpmEpoch,
Arch: "x86_64",
SourceRpm: "some-package-1.0.0-1.el8.src.rpm",
},
},
expected: nil,
},
{
name: "from rpm archive",
input: pkg.Package{
Name: "bind-export-libs",
Version: "32:9.11.13-3.el8",
Type: pkg.RpmPkg,
PURL: "pkg:rpm/bind-export-libs@9.11.13-3.el8?arch=x86_64&epoch=32&upstream=bind-9.11.13-3.el8.src.rpm",
Metadata: pkg.RpmArchive{
Name: "bind-export-libs",
Version: "9.11.13",
Release: "3.el8",
Epoch: &rpmEpoch,
Arch: "x86_64",
SourceRpm: "bind-9.11.13-3.el8.src.rpm",
},
},
expected: &cyclonedx.ExternalReference{
Type: cyclonedx.ERTypeSourceDistribution, URL: "pkg:rpm/bind@9.11.13-3.el8?arch=src&epoch=32",
},
},
{
// alpm pkgbases are build recipes, not published source packages, so they are not supported
name: "from alpm is not supported",
input: pkg.Package{
Name: "gcc-libs",
Version: "13.2.1-3",
Type: pkg.AlpmPkg,
PURL: "pkg:alpm/arch/gcc-libs@13.2.1-3?arch=x86_64&distro=arch-rolling&upstream=gcc",
Metadata: pkg.AlpmDBEntry{
Package: "gcc-libs",
Version: "13.2.1-3",
BasePackage: "gcc",
Architecture: "x86_64",
},
},
expected: nil,
},
{
name: "from rpm with dashes in the source name",
input: pkg.Package{
Name: "perl-Foo-Bar-libs",
Type: pkg.RpmPkg,
PURL: "pkg:rpm/redhat/perl-Foo-Bar-libs@1.0-1.el8?arch=x86_64&distro=rhel-8",
Metadata: pkg.RpmDBEntry{
Name: "perl-Foo-Bar-libs",
Version: "1.0",
Release: "1.el8",
SourceRpm: "perl-Foo-Bar-1.0-1.el8.src.rpm",
},
},
expected: &cyclonedx.ExternalReference{
Type: cyclonedx.ERTypeSourceDistribution, URL: "pkg:rpm/redhat/perl-Foo-Bar@1.0-1.el8?arch=src&distro=rhel-8",
},
},
{
name: "from rpm nosrc",
input: pkg.Package{
Name: "some-firmware",
Type: pkg.RpmPkg,
PURL: "pkg:rpm/redhat/some-firmware@1.0-1.el8?arch=noarch&distro=rhel-8",
Metadata: pkg.RpmDBEntry{
Name: "some-firmware",
Version: "1.0",
Release: "1.el8",
SourceRpm: "firmware-1.0-1.el8.nosrc.rpm",
},
},
expected: &cyclonedx.ExternalReference{
Type: cyclonedx.ERTypeSourceDistribution, URL: "pkg:rpm/redhat/firmware@1.0-1.el8?arch=nosrc&distro=rhel-8",
},
},
{
name: "from rpm keeps rpmmod",
input: pkg.Package{
Name: "nodejs-libs",
Type: pkg.RpmPkg,
PURL: "pkg:rpm/redhat/nodejs-libs@18.0-1.el8?arch=x86_64&distro=rhel-8&rpmmod=nodejs%3A18",
Metadata: pkg.RpmDBEntry{
Name: "nodejs-libs",
Version: "18.0",
Release: "1.el8",
SourceRpm: "nodejs-18.0-1.el8.src.rpm",
},
},
expected: &cyclonedx.ExternalReference{
Type: cyclonedx.ERTypeSourceDistribution, URL: "pkg:rpm/redhat/nodejs@18.0-1.el8?arch=src&distro=rhel-8&rpmmod=nodejs%3A18",
},
},
{
name: "from rpm with malformed source rpm",
input: pkg.Package{
Name: "gpg-pubkey",
Type: pkg.RpmPkg,
PURL: "pkg:rpm/redhat/gpg-pubkey@1-1?distro=rhel-8",
Metadata: pkg.RpmDBEntry{
Name: "gpg-pubkey",
Version: "1",
Release: "1",
SourceRpm: "(none)",
},
},
expected: nil,
},
{
name: "from rpm with non-source rpm filename",
input: pkg.Package{
Name: "bash",
Type: pkg.RpmPkg,
PURL: "pkg:rpm/redhat/bash@5.1-2?arch=x86_64&distro=rhel-8",
Metadata: pkg.RpmDBEntry{
Name: "bash",
Version: "5.1",
Release: "2",
SourceRpm: "bash-5.1-2.x86_64.rpm",
},
},
expected: nil,
},
{
name: "from rpm with too few dashes",
input: pkg.Package{
Name: "bash",
Type: pkg.RpmPkg,
PURL: "pkg:rpm/redhat/bash@5.1-2?arch=x86_64&distro=rhel-8",
Metadata: pkg.RpmDBEntry{
Name: "bash",
Version: "5.1",
Release: "2",
SourceRpm: "bash-5.1.src.rpm",
},
},
expected: nil,
},
{
name: "binary qualifiers and subpath are not carried over",
input: pkg.Package{
Name: "libssl3",
Type: pkg.DebPkg,
PURL: "pkg:deb/debian/libssl3@3.0.0-1?arch=amd64&distro=debian-12&checksum=sha256:dead&download_url=https://example.com/x.deb&cpes=cpe:2.3:a:x:x:1:*:*:*:*:*:*:*#sub/dir",
Metadata: pkg.DpkgDBEntry{
Package: "libssl3",
Version: "3.0.0-1",
Source: "openssl",
},
},
expected: &cyclonedx.ExternalReference{
Type: cyclonedx.ERTypeSourceDistribution, URL: "pkg:deb/debian/openssl@3.0.0-1?arch=source&distro=debian-12",
},
},
{
name: "PURL type does not match metadata",
input: pkg.Package{
Name: "x",
Type: pkg.DebPkg,
PURL: "pkg:npm/x@1",
Metadata: pkg.DpkgDBEntry{
Package: "x",
Version: "1",
Source: "lodash",
},
},
expected: nil,
},
{
name: "invalid PURL",
input: pkg.Package{
Name: "x",
Type: pkg.DebPkg,
PURL: "not-a-purl",
Metadata: pkg.DpkgDBEntry{
Package: "x",
Version: "1",
Source: "y",
},
},
expected: nil,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, encodeSourcePackageExternalReference(test.input))
})
}
}
func Test_isValidExternalRef(t *testing.T) {
tests := []struct {
name string
@@ -93,7 +93,7 @@ func encode(out map[string]string, value reflect.Value, prefix string, fn FieldN
typ := value.Type()
switch typ.Kind() {
case reflect.Ptr:
case reflect.Pointer:
if value.IsNil() {
return
}
@@ -158,7 +158,7 @@ func fieldName(f reflect.StructField, prefix string, fn FieldName) (string, bool
// Decode based on the given type, applies all values to hydrate a new instance
func Decode(typ reflect.Type, values map[string]string, prefix string, fn FieldName) any {
isPtr := false
for typ.Kind() == reflect.Ptr {
for typ.Kind() == reflect.Pointer {
typ = typ.Elem()
isPtr = true
}
@@ -188,7 +188,7 @@ func Decode(typ reflect.Type, values map[string]string, prefix string, fn FieldN
func DecodeInto(obj any, values map[string]string, prefix string, fn FieldName) {
value := reflect.ValueOf(obj)
for value.Type().Kind() == reflect.Ptr {
for value.Type().Kind() == reflect.Pointer {
value = value.Elem()
}
@@ -205,7 +205,7 @@ func decode(vals map[string]string, value reflect.Value, prefix string, fn Field
incoming, valid := vals[prefix]
switch typ.Kind() {
case reflect.Ptr:
case reflect.Pointer:
t := typ.Elem()
v := value
if v.IsNil() {
@@ -269,13 +269,13 @@ func decode(vals map[string]string, value reflect.Value, prefix string, fn Field
str := fmt.Sprintf("%s:%d", prefix, idx)
// create new placeholder and decode values
newType := t
if t.Kind() == reflect.Ptr {
if t.Kind() == reflect.Pointer {
newType = t.Elem()
}
v := reflect.New(newType)
if decode(vals, v.Elem(), str, fn) {
// append to slice
if t.Kind() != reflect.Ptr {
if t.Kind() != reflect.Pointer {
v = v.Elem()
}
slice = reflect.Append(slice, v)
@@ -304,7 +304,7 @@ func decode(vals map[string]string, value reflect.Value, prefix string, fn Field
keyVals[key] = after
// create new placeholder and decode key
newKeyType := keyType
if keyType.Kind() == reflect.Ptr {
if keyType.Kind() == reflect.Pointer {
newKeyType = keyType.Elem()
}
k := reflect.New(newKeyType)
@@ -312,18 +312,18 @@ func decode(vals map[string]string, value reflect.Value, prefix string, fn Field
log.Debugf("unable to decode key for: %s", key)
continue
}
if keyType.Kind() != reflect.Ptr {
if keyType.Kind() != reflect.Pointer {
k = k.Elem()
}
// create new placeholder and decode value
newValueType := valueType
if valueType.Kind() == reflect.Ptr {
if valueType.Kind() == reflect.Pointer {
newValueType = valueType.Elem()
}
v := reflect.New(newValueType)
if decode(vals, v.Elem(), key, fn) {
if valueType.Kind() != reflect.Ptr {
if valueType.Kind() != reflect.Pointer {
v = v.Elem()
}
@@ -367,7 +367,7 @@ func PtrToStruct(ptr any) any {
return nil
}
switch v.Type().Kind() {
case reflect.Ptr:
case reflect.Pointer:
return PtrToStruct(v.Elem().Interface())
case reflect.Interface:
return PtrToStruct(v.Elem().Interface())
@@ -9,16 +9,21 @@ func DocumentName(src source.Description) string {
return src.Name
}
var name string
switch metadata := src.Metadata.(type) {
case source.ImageMetadata:
return metadata.UserInput
name = metadata.UserInput
case source.OCIModelMetadata:
return metadata.UserInput
name = metadata.UserInput
case source.DirectoryMetadata:
return metadata.Path
name = metadata.Path
case source.FileMetadata:
return metadata.Path
default:
name = metadata.Path
}
// the SPDX document name is mandatory, and a source read from another SBOM may not have one
if name == "" {
return "unknown"
}
return name
}
@@ -1,8 +1,6 @@
package helpers
import (
"fmt"
"strings"
"testing"
"github.com/stretchr/testify/assert"
@@ -73,11 +71,23 @@ func Test_DocumentName(t *testing.T) {
},
expected: "some/name",
},
{
name: "no name or metadata",
srcMetadata: source.Description{},
expected: "unknown",
},
{
name: "metadata without a name",
srcMetadata: source.Description{
Metadata: source.ImageMetadata{},
},
expected: "unknown",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
actual := DocumentName(test.srcMetadata)
assert.True(t, strings.HasPrefix(actual, test.expected), fmt.Sprintf("actual name %q", actual))
assert.Equal(t, test.expected, actual)
// track each scheme tested (passed or not)
tracker.Tested(t, test.srcMetadata.Metadata)
@@ -16,6 +16,7 @@ import (
"github.com/anchore/syft/syft/source/directorysource"
)
//nolint:goconst // fixture data: the repeated names and versions are what the golden files assert against, so they read better spelled out than behind constants
func DirectoryInput(t testing.TB, dir string) sbom.SBOM {
catalog := newDirectoryCatalog()
@@ -97,6 +98,7 @@ func DirectoryInputWithAuthorField(t testing.TB) sbom.SBOM {
}
}
//nolint:goconst // fixture data: the repeated names and versions are what the golden files assert against, so they read better spelled out than behind constants
func newDirectoryCatalog() *pkg.Collection {
catalog := pkg.NewCollection()
ctx := context.TODO()
@@ -19,6 +19,7 @@ import (
"github.com/anchore/syft/syft/source/stereoscopesource"
)
//nolint:goconst // fixture data: the repeated names and versions are what the golden files assert against, so they read better spelled out than behind constants
func ImageInput(t testing.TB, testImage string, options ...ImageOption) sbom.SBOM {
t.Helper()
catalog := pkg.NewCollection()
@@ -95,6 +96,7 @@ func changeToDirectoryWithGoldenFixture(t testing.TB, testImage string) func() {
return fn
}
//nolint:goconst // fixture data: the repeated names and versions are what the golden files assert against, so they read better spelled out than behind constants
func populateImageCatalog(catalog *pkg.Collection, img *image.Image) {
// TODO: this helper function is coupled to the image-simple fixture, which seems like a bad idea
_, ref1, _ := img.SquashedTree().File("/somefile-1.txt", filetree.FollowBasenameLinks)
@@ -0,0 +1,146 @@
package testutil
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source/directorysource"
)
func SourcePackagesInput(t testing.TB, dir string) sbom.SBOM {
path := filepath.Join(dir, "some", "path")
require.NoError(t, os.MkdirAll(path, 0755))
src, err := directorysource.New(
directorysource.Config{
Path: path,
Base: dir,
},
)
require.NoError(t, err)
return sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: newSourcePackageCatalog(),
},
Source: src.Describe(),
Descriptor: sbom.Descriptor{
Name: "syft",
Version: "v0.42.0-bogus",
},
}
}
//nolint:funlen
func newSourcePackageCatalog() *pkg.Collection {
epoch := 32
catalog := pkg.NewCollection()
// apk: the origin differs from the package name, but apk is not supported so no source reference is expected
catalog.Add(pkg.Package{
Name: "libc-utils",
Version: "0.7.2-r3",
Type: pkg.ApkPkg,
FoundBy: "apk-db-cataloger",
Locations: file.NewLocationSet(file.NewLocation("/lib/apk/db/installed")),
PURL: "pkg:apk/alpine/libc-utils@0.7.2-r3?arch=x86_64&distro=alpine-3.16.3&upstream=libc-dev",
Metadata: pkg.ApkDBEntry{
Package: "libc-utils",
OriginPackage: "libc-dev",
Version: "0.7.2-r3",
Architecture: "x86_64",
},
})
// apk: negative control -- the package is its own origin
catalog.Add(pkg.Package{
Name: "busybox",
Version: "1.35.0-r17",
Type: pkg.ApkPkg,
FoundBy: "apk-db-cataloger",
Locations: file.NewLocationSet(file.NewLocation("/lib/apk/db/installed")),
PURL: "pkg:apk/alpine/busybox@1.35.0-r17?arch=x86_64&distro=alpine-3.16.3",
Metadata: pkg.ApkDBEntry{
Package: "busybox",
OriginPackage: "busybox",
Version: "1.35.0-r17",
Architecture: "x86_64",
},
})
// deb: source name differs, and the source version is implied by the binary version
catalog.Add(pkg.Package{
Name: "libpam-runtime",
Version: "1.5.2-6+deb12u1",
Type: pkg.DebPkg,
FoundBy: "dpkg-db-cataloger",
Locations: file.NewLocationSet(file.NewLocation("/var/lib/dpkg/status")),
PURL: "pkg:deb/debian/libpam-runtime@1.5.2-6%2Bdeb12u1?arch=all&distro=debian-12&upstream=pam",
Metadata: pkg.DpkgDBEntry{
Package: "libpam-runtime",
Version: "1.5.2-6+deb12u1",
Source: "pam",
Architecture: "all",
},
})
// deb: the source version differs from the binary version
catalog.Add(pkg.Package{
Name: "attr",
Version: "1:2.4.47-2+b1",
Type: pkg.DebPkg,
FoundBy: "dpkg-db-cataloger",
Locations: file.NewLocationSet(file.NewLocation("/var/lib/dpkg/status")),
PURL: "pkg:deb/debian/attr@1%3A2.4.47-2%2Bb1?arch=amd64&distro=debian-12&upstream=attr%401%3A2.4.47-2",
Metadata: pkg.DpkgDBEntry{
Package: "attr",
Version: "1:2.4.47-2+b1",
Source: "attr",
SourceVersion: "1:2.4.47-2",
Architecture: "amd64",
},
})
// rpm: name and version both come out of the source RPM filename, and the epoch is carried over
catalog.Add(pkg.Package{
Name: "bind-export-libs",
Version: "32:9.11.13-3.el8",
Type: pkg.RpmPkg,
FoundBy: "rpm-db-cataloger",
Locations: file.NewLocationSet(file.NewLocation("/var/lib/rpm/Packages")),
PURL: "pkg:rpm/centos/bind-export-libs@9.11.13-3.el8?arch=x86_64&distro=centos-8&epoch=32&upstream=bind-9.11.13-3.el8.src.rpm",
Metadata: pkg.RpmDBEntry{
Name: "bind-export-libs",
Version: "9.11.13",
Release: "3.el8",
Epoch: &epoch,
Arch: "x86_64",
SourceRpm: "bind-9.11.13-3.el8.src.rpm",
},
})
// alpm: the base package differs from the package name, but alpm is not supported so no source reference is expected
catalog.Add(pkg.Package{
Name: "gcc-libs",
Version: "13.2.1-3",
Type: pkg.AlpmPkg,
FoundBy: "alpm-db-cataloger",
Locations: file.NewLocationSet(file.NewLocation("/var/lib/pacman/local/gcc-libs-13.2.1-3/desc")),
PURL: "pkg:alpm/arch/gcc-libs@13.2.1-3?arch=x86_64&distro=arch-rolling&upstream=gcc",
Metadata: pkg.AlpmDBEntry{
Package: "gcc-libs",
Version: "13.2.1-3",
BasePackage: "gcc",
Architecture: "x86_64",
},
})
return catalog
}
+5 -1
View File
@@ -8,6 +8,7 @@ import (
"strings"
"github.com/anchore/packageurl-go"
syftinternal "github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/format/internal"
"github.com/anchore/syft/syft/pkg"
@@ -61,7 +62,7 @@ func toSyftModel(r io.Reader) (*sbom.SBOM, error) {
var errs []error
pkgs := pkg.NewCollection()
scanner := bufio.NewScanner(r)
scanner := syftinternal.NewLineScanner(r)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
@@ -84,6 +85,9 @@ func toSyftModel(r io.Reader) (*sbom.SBOM, error) {
p.SetID()
pkgs.Add(p)
}
if err := scanner.Err(); err != nil {
errs = append(errs, err)
}
return &sbom.SBOM{
Artifacts: sbom.Artifacts{
+35
View File
@@ -5,6 +5,7 @@ import (
"io"
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
@@ -183,3 +184,37 @@ func TestDecoder_Identify(t *testing.T) {
})
}
}
func TestDecoder_DecodeNullElements(t *testing.T) {
// a JSON null in these lists (or an SPDX 2.1 package the version converter cannot map) decodes to a nil pointer.
// a null entry directly in "packages" is not covered: for SPDX 2.2 and 2.3 it panics inside spdx/tools-golang
for _, doc := range []string{
`{"spdxVersion":"SPDX-2.1","packages":[{}]}`,
`{"spdxVersion":"SPDX-2.3","files":[null]}`,
`{"spdxVersion":"SPDX-2.3","packages":[{"SPDXID":"SPDXRef-a","name":"a","files":[null]}]}`,
`{"spdxVersion":"SPDX-2.3","packages":[{"SPDXID":"SPDXRef-a","name":"a","externalRefs":[null]}]}`,
} {
t.Run(doc, func(t *testing.T) {
assert.NotPanics(t, func() {
_, _, _, _ = NewFormatDecoder().Decode(strings.NewReader(doc))
})
})
}
}
func TestDecoder_DecodeFileRootWithoutSupplier(t *testing.T) {
// the package supplier is optional in SPDX, including on the package the document describes
f, err := os.Open("testdata/spdx/file-root-without-supplier.spdx.json")
require.NoError(t, err)
defer f.Close()
s, _, _, err := NewFormatDecoder().Decode(f)
require.NoError(t, err)
assert.Equal(t, "app.tar.gz", s.Source.Name)
var names []string
for _, p := range s.Artifacts.Packages.Sorted() {
names = append(names, p.Name)
}
assert.Equal(t, []string{"requests"}, names)
}
+2 -23
View File
@@ -236,23 +236,6 @@ func TestSupportedVersions(t *testing.T) {
subject := testutil.DirectoryInput(t, t.TempDir())
dec := NewFormatDecoder()
relationshipOffsetPerVersion := map[string]int{
// the package representing the source gets a relationship from the source package to all other packages found
// these relationships cannot be removed until the primaryPackagePurpose info is available in 2.3
"2.1": 2,
"2.2": 2,
// the source-to-package relationships can be removed since the primaryPackagePurpose info is available in 2.3
"2.3": 0,
"3.0": 0,
}
pkgCountOffsetPerVersion := map[string]int{
"2.1": 1, // the source is mapped as a package, but cannot distinguish it since the primaryPackagePurpose info is not available until 2.3
"2.2": 1, // the source is mapped as a package, but cannot distinguish it since the primaryPackagePurpose info is not available until 2.3
"2.3": 0, // the source package can be removed since the primaryPackagePurpose info is available
"3.0": 0,
}
for _, enc := range encs {
t.Run(enc.Version(), func(t *testing.T) {
require.Contains(t, versions, enc.Version())
@@ -274,13 +257,9 @@ func TestSupportedVersions(t *testing.T) {
require.NotEmpty(t, s.Artifacts.Packages.PackageCount())
offset := relationshipOffsetPerVersion[enc.Version()]
assert.Equal(t, len(subject.Relationships), len(s.Relationships), "mismatched relationship count")
assert.Equal(t, len(subject.Relationships)+offset, len(s.Relationships), "mismatched relationship count")
offset = pkgCountOffsetPerVersion[enc.Version()]
if !assert.Equal(t, subject.Artifacts.Packages.PackageCount()+offset, s.Artifacts.Packages.PackageCount(), "mismatched package count") {
if !assert.Equal(t, subject.Artifacts.Packages.PackageCount(), s.Artifacts.Packages.PackageCount(), "mismatched package count") {
t.Logf("expected: %d", subject.Artifacts.Packages.PackageCount())
for _, p := range subject.Artifacts.Packages.Sorted() {
t.Logf(" - %s", p.String())
@@ -47,13 +47,13 @@
"copyrightText": "NOASSERTION"
},
{
"SPDXID": "SPDXRef-DocumentRoot-Unknown-",
"SPDXID": "SPDXRef-DocumentRoot-Unknown-unknown",
"copyrightText": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "NOASSERTION",
"name": "",
"name": "unknown",
"supplier": "NOASSERTION"
}
],
@@ -82,18 +82,18 @@
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Unknown-",
"spdxElementId": "SPDXRef-DocumentRoot-Unknown-unknown",
"relatedSpdxElement": "SPDXRef-Package-files-analyzed-false-7d37ba9d2f7c574b",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Unknown-",
"spdxElementId": "SPDXRef-DocumentRoot-Unknown-unknown",
"relatedSpdxElement": "SPDXRef-Package-files-analyzed-true-035066c2086b8bb4",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DocumentRoot-Unknown-",
"relatedSpdxElement": "SPDXRef-DocumentRoot-Unknown-unknown",
"relationshipType": "DESCRIBES"
}
]
@@ -0,0 +1,29 @@
{
"spdxVersion": "SPDX-2.3",
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "app.tar.gz",
"documentNamespace": "https://example.com/spdx/app-1.0",
"creationInfo": { "created": "2026-09-24T00:00:00Z", "creators": ["Tool: example-1.0"] },
"packages": [
{
"SPDXID": "SPDXRef-app",
"name": "app.tar.gz",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"primaryPackagePurpose": "FILE"
},
{
"SPDXID": "SPDXRef-requests",
"name": "requests",
"versionInfo": "2.31.0",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"externalRefs": [{ "referenceCategory": "PACKAGE-MANAGER", "referenceType": "purl", "referenceLocator": "pkg:pypi/requests@2.31.0" }]
}
],
"relationships": [
{ "spdxElementId": "SPDXRef-DOCUMENT", "relationshipType": "DESCRIBES", "relatedSpdxElement": "SPDXRef-app" },
{ "spdxElementId": "SPDXRef-app", "relationshipType": "CONTAINS", "relatedSpdxElement": "SPDXRef-requests" }
]
}
+39 -21
View File
@@ -3,6 +3,7 @@ package spdxtagvalue
import (
"bytes"
"flag"
"fmt"
"testing"
"github.com/stretchr/testify/assert"
@@ -143,21 +144,6 @@ func TestSupportedVersions(t *testing.T) {
subject := testutil.DirectoryInput(t, t.TempDir())
dec := NewFormatDecoder()
relationshipOffsetPerVersion := map[string]int{
// the package representing the source gets a relationship from the source package to all other packages found
// these relationships cannot be removed until the primaryPackagePurpose info is available in 2.3
"2.1": 2,
"2.2": 2,
// the source-to-package relationships can be removed since the primaryPackagePurpose info is available in 2.3
"2.3": 0,
}
pkgCountOffsetPerVersion := map[string]int{
"2.1": 1, // the source is mapped as a package, but cannot distinguish it since the primaryPackagePurpose info is not available until 2.3
"2.2": 1, // the source is mapped as a package, but cannot distinguish it since the primaryPackagePurpose info is not available until 2.3
"2.3": 0, // the source package can be removed since the primaryPackagePurpose info is available
}
for _, enc := range encs {
t.Run(enc.Version(), func(t *testing.T) {
require.Contains(t, versions, enc.Version())
@@ -178,13 +164,9 @@ func TestSupportedVersions(t *testing.T) {
require.NotEmpty(t, s.Artifacts.Packages.PackageCount())
offset := relationshipOffsetPerVersion[enc.Version()]
assert.Equal(t, len(subject.Relationships), len(s.Relationships), "mismatched relationship count")
assert.Equal(t, len(subject.Relationships)+offset, len(s.Relationships), "mismatched relationship count")
offset = pkgCountOffsetPerVersion[enc.Version()]
if !assert.Equal(t, subject.Artifacts.Packages.PackageCount()+offset, s.Artifacts.Packages.PackageCount(), "mismatched package count") {
if !assert.Equal(t, subject.Artifacts.Packages.PackageCount(), s.Artifacts.Packages.PackageCount(), "mismatched package count") {
t.Logf("expected: %d", subject.Artifacts.Packages.PackageCount())
for _, p := range subject.Artifacts.Packages.Sorted() {
t.Logf(" - %s", p.String())
@@ -209,3 +191,39 @@ func defaultFormatEncoders() []sbom.FormatEncoder {
}
return encs
}
func TestEncodeDecodeSourceWithoutName(t *testing.T) {
// a source read from another SBOM (a purl list, a CycloneDX document without metadata.component) may have no name,
// but the SPDX package name is mandatory and the tag-value decoder needs it to start the root package section
p := pkg.Package{Name: "left-pad", Version: "1.3.0", Type: pkg.NpmPkg}
p.SetID()
for _, src := range []source.Description{
{},
{Metadata: source.FileMetadata{}},
{Metadata: source.DirectoryMetadata{}},
{Metadata: source.ImageMetadata{}},
{Metadata: source.OCIModelMetadata{}},
{Metadata: source.SnapMetadata{}},
} {
for _, enc := range defaultFormatEncoders() {
t.Run(fmt.Sprintf("%T/%s", src.Metadata, enc.Version()), func(t *testing.T) {
var buf bytes.Buffer
require.NoError(t, enc.Encode(&buf, sbom.SBOM{
Source: src,
Artifacts: sbom.Artifacts{Packages: pkg.NewCollection(p)},
}))
s, _, _, err := NewFormatDecoder().Decode(bytes.NewReader(buf.Bytes()))
require.NoError(t, err, "encoded document:\n%s", buf.String())
var names []string
for _, decoded := range s.Artifacts.Packages.Sorted() {
names = append(names, decoded.Name)
}
// the root package describes the source, so it must not come back as a package of the SBOM
assert.Equal(t, []string{"left-pad"}, names)
})
}
}
}
+94
View File
@@ -0,0 +1,94 @@
package format
import (
"bytes"
"fmt"
"math"
"testing"
"github.com/anchore/syft/syft/format/syftjson"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
// TestEncodersHandleUntypedMetadataValues pushes awkward values through the untyped metadata fields
// allowlisted in internal/packagemetadata (see untypedFieldAllowlist) and asserts that SetID and every
// encoder neither panic nor fail. A new allowlist entry should get a case here too.
func TestEncodersHandleUntypedMetadataValues(t *testing.T) {
cases := []struct {
name string
value any
// encoding/json rejects non-finite floats, so syft-json returns an error for them. Producers
// have to replace these before they reach the model (the GGUF cataloger does); the case stays
// here so the expectation flips visibly if syft-json ever learns to encode them.
syftJSONRejects bool
}{
{name: "NaN", value: math.NaN(), syftJSONRejects: true},
{name: "+Inf float32", value: float32(math.Inf(1)), syftJSONRejects: true},
{name: "-Inf", value: math.Inf(-1), syftJSONRejects: true},
{name: "NaN in array", value: []any{1.0, math.NaN()}, syftJSONRejects: true},
{name: "invalid utf-8", value: "\xff\xfe"},
{name: "NUL and control characters", value: "a\x00b\x1b"},
{name: "max uint64", value: uint64(math.MaxUint64)},
{name: "deep array", value: nested(2000, func(v any) any { return []any{v} })},
{name: "deep map", value: nested(2000, func(v any) any { return map[string]any{"k": v} })},
{name: "large array", value: make([]any, 100_000)},
}
for _, tt := range cases {
t.Run(tt.name, func(t *testing.T) {
p := pkg.Package{
Name: "model",
Version: "1.0",
Type: pkg.ModelPkg,
Metadata: pkg.GGUFFileHeader{
RemainingKeyValues: map[string]any{"general.value": tt.value},
},
}
if !noPanic(t, "SetID", p.SetID) {
return
}
s := sbom.SBOM{
Artifacts: sbom.Artifacts{Packages: pkg.NewCollection(p)},
Source: source.Description{Metadata: source.DirectoryMetadata{Path: "/models"}},
}
for _, enc := range Encoders() {
id := fmt.Sprintf("%s@%s", enc.ID(), enc.Version())
var buf bytes.Buffer
noPanic(t, id, func() {
err := enc.Encode(&buf, s)
switch {
case tt.syftJSONRejects && enc.ID() == syftjson.ID:
if err == nil {
t.Errorf("%s: expected an encode error for a non-finite float", id)
}
case err != nil:
t.Errorf("%s: encode failed: %v", id, err)
}
})
}
})
}
}
func nested(depth int, wrap func(any) any) any {
var v any = "leaf"
for range depth {
v = wrap(v)
}
return v
}
func noPanic(t *testing.T, what string, fn func()) (ok bool) {
t.Helper()
defer func() {
if r := recover(); r != nil {
t.Errorf("%s: panic: %v", what, r)
ok = false
}
}()
fn()
return true
}
+7 -9
View File
@@ -7,11 +7,12 @@ import (
"encoding/binary"
"fmt"
"io"
"runtime"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal/testutils"
)
// section describes one section to place in a fixture. The compression headers are deliberately
@@ -543,13 +544,10 @@ func TestNewFile_BombDoesNotAllocate(t *testing.T) {
{name: ".strtab", typ: elf.SHT_STRTAB},
}, buildOpts{})
var before, after runtime.MemStats
runtime.GC()
runtime.ReadMemStats(&before)
_, err := NewFile(bytes.NewReader(data))
runtime.ReadMemStats(&after)
var err error
allocated := testutils.MeasureAlloc(t, func() {
_, err = NewFile(bytes.NewReader(data))
})
require.Error(t, err)
assert.Contains(t, err.Error(), "over the")
@@ -557,7 +555,7 @@ func TestNewFile_BombDoesNotAllocate(t *testing.T) {
// the guard has to reject before the allocation, not after it. The margin is loose on purpose: this
// is asserting "nothing near the declared size was allocated", not a precise budget.
const margin = 32 * 1024 * 1024
assert.Less(t, after.TotalAlloc-before.TotalAlloc, uint64(margin),
assert.Less(t, allocated, uint64(margin),
"parsing allocated far more than the input warrants")
}
+4 -1
View File
@@ -216,5 +216,8 @@ func (r ChrootContext) ToChrootPath(nativePath string) string {
responsePath = strings.TrimPrefix(responsePath, prefix)
}
return responsePath
// chroot paths are posix regardless of host (these end up as location paths in the SBOM). This uses
// filepath (not path) since responsePath is native here (FromPosix above, then trimmed of a native prefix),
// and only filepath.ToSlash knows the host separator to convert.
return filepath.ToSlash(responsePath)
}
@@ -3,6 +3,7 @@ package fileresolver
import (
"os"
"path/filepath"
"runtime"
"testing"
"github.com/stretchr/testify/assert"
@@ -10,6 +11,9 @@ import (
)
func Test_ChrootContext_RequestResponse(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("fixture relies on symlinks, which git does not check out as links on windows by default")
}
// /
// somewhere/
// outside.txt
@@ -563,6 +567,9 @@ func TestNewChrootContext_BaseValidation(t *testing.T) {
}
func TestToNativeGlob(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("chroot context and expectations are posix paths")
}
tests := []struct {
name string
chrootContext ChrootContext
@@ -4,7 +4,6 @@ import (
"fmt"
"io/fs"
"os"
"path"
"path/filepath"
"runtime"
"sort"
@@ -18,6 +17,7 @@ import (
"github.com/wagoodman/go-progress"
"github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/syft/internal/windows"
)
type indexerMock struct {
@@ -138,6 +138,9 @@ func TestDirectoryIndexer_handleFileAccessErr(t *testing.T) {
}
func TestDirectoryIndexer_IncludeRootPathInIndex(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("indexes the posix filesystem root")
}
filterFn := func(_, path string, _ os.FileInfo, _ error) error {
if path != "/" {
return fs.SkipDir
@@ -206,11 +209,7 @@ func TestDirectoryIndexer_index(t *testing.T) {
info, err := os.Stat(test.path)
assert.NoError(t, err)
// note: the index uses absolute paths, so assertions MUST keep this in mind
cwd, err := os.Getwd()
require.NoError(t, err)
p := file.Path(path.Join(cwd, test.path))
p := indexedPath(t, test.path)
assert.Equal(t, true, tree.HasPath(p))
exists, ref, err := tree.File(p)
assert.Equal(t, true, exists)
@@ -226,6 +225,9 @@ func TestDirectoryIndexer_index(t *testing.T) {
}
func TestDirectoryIndexer_index_for_AncestorSymlinks(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("fixture relies on symlinks, which git does not check out as links on windows by default")
}
// note: this test is testing the effects from NewFromDirectory, indexTree, and addPathToIndex
_, filename, _, ok := runtime.Caller(0)
require.True(t, ok)
@@ -256,11 +258,7 @@ func TestDirectoryIndexer_index_for_AncestorSymlinks(t *testing.T) {
info, err := os.Stat(test.path)
assert.NoError(t, err)
// note: the index uses absolute paths, so assertions MUST keep this in mind
cwd, err := os.Getwd()
require.NoError(t, err)
p := file.Path(path.Join(cwd, test.path))
p := indexedPath(t, test.path)
assert.Equal(t, true, tree.HasPath(p))
exists, ref, err := tree.File(p)
assert.Equal(t, true, exists)
@@ -287,6 +285,9 @@ func TestDirectoryIndexer_index_survive_badSymlink(t *testing.T) {
}
func TestDirectoryIndexer_index_survive_inaccessibleSymlinkTarget(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("relies on symlinks and posix permissions")
}
// a symlink that resolves into a directory we don't have permission to traverse should be skipped
// with a warning like any other inaccessible path, not abort the entire scan (see #3286).
if os.Geteuid() == 0 {
@@ -323,6 +324,9 @@ func TestDirectoryIndexer_index_survive_inaccessibleSymlinkTarget(t *testing.T)
}
func TestDirectoryIndexer_SkipsAlreadyVisitedLinkDestinations(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("fixture relies on symlinks, which git does not check out as links on windows by default")
}
var observedPaths []string
pathObserver := func(_, p string, _ os.FileInfo, _ error) error {
fields := strings.Split(p, "testdata/symlinks-prune-indexing")
@@ -430,6 +434,11 @@ func TestDirectoryIndexer_IndexesAllTypes(t *testing.T) {
}
func Test_allContainedPaths(t *testing.T) {
if runtime.GOOS == "windows" {
// ponytail: production passes native windows paths here, which these posix cases don't represent.
// Ancestor indexing for a root reached through a symlink is likely broken on windows; fix separately.
t.Skip("cases are posix paths")
}
tests := []struct {
name string
@@ -485,6 +494,9 @@ func Test_allContainedPaths(t *testing.T) {
}
func Test_relativePath(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("exercises a posix-only test helper")
}
tests := []struct {
name string
basePath string
@@ -548,3 +560,17 @@ func relativePath(basePath, givenPath string) string {
return relPath
}
// indexedPath returns how the given cwd-relative path is keyed in the index (absolute and always posix)
func indexedPath(t *testing.T, rel string) file.Path {
t.Helper()
cwd, err := os.Getwd()
require.NoError(t, err)
// filepath (not path): cwd is a native host path, so it must be joined with the host separator before
// being converted to the posix form the index is keyed on
p := filepath.Join(cwd, rel)
if windows.HostRunningOnWindows() {
p = windows.ToPosix(p)
}
return file.Path(p)
}
@@ -0,0 +1,48 @@
package fileresolver
import (
"context"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
stereoscopeFile "github.com/anchore/stereoscope/pkg/file"
)
// a location handed out by the resolver should be resolvable again by its own path. This is what the file
// digest cataloger relies on, and it silently found nothing on windows (see #5325).
func TestDirectory_LocationPathsRoundTrip(t *testing.T) {
// resolve symlinks up front (e.g. /var -> /private/var on macOS) so the cwd prefix matches the indexed paths
root, err := filepath.EvalSymlinks(t.TempDir())
require.NoError(t, err)
require.NoError(t, os.MkdirAll(filepath.Join(root, "sub", "dir"), 0o755))
for _, p := range []string{"top.txt", filepath.Join("sub", "mid.txt"), filepath.Join("sub", "dir", "deep.txt")} {
require.NoError(t, os.WriteFile(filepath.Join(root, p), []byte(p), 0o600))
}
// mirror `syft dir:.` from within the scanned directory
t.Chdir(root)
resolver, err := NewFromDirectory(".", "")
require.NoError(t, err)
var regular int
for loc := range resolver.AllLocations(context.Background()) {
md, err := resolver.FileMetadataByLocation(loc)
require.NoError(t, err)
if md.Type != stereoscopeFile.TypeRegular {
continue
}
regular++
assert.NotContains(t, loc.RealPath, `\`, "location paths should be posix")
found, err := resolver.FilesByPath(loc.RealPath)
require.NoError(t, err)
assert.Len(t, found, 1, "unable to resolve %q by its own path", loc.RealPath)
}
assert.Equal(t, 3, regular)
}
@@ -1,126 +0,0 @@
package fileresolver
import "testing"
func Test_windowsToPosix(t *testing.T) {
type args struct {
windowsPath string
}
tests := []struct {
name string
args args
wantPosixPath string
}{
{
name: "basic case",
args: args{
windowsPath: `C:\some\windows\place`,
},
wantPosixPath: "/c/some/windows/place",
},
{
name: "escaped case",
args: args{
windowsPath: `C:\\some\\windows\\place`,
},
wantPosixPath: "/c/some/windows/place",
},
{
name: "forward slash",
args: args{
windowsPath: `C:/foo/bar`,
},
wantPosixPath: "/c/foo/bar",
},
{
name: "mix slash",
args: args{
windowsPath: `C:\foo/bar\`,
},
wantPosixPath: "/c/foo/bar",
},
{
name: "case sensitive case",
args: args{
windowsPath: `C:\Foo/bAr\`,
},
wantPosixPath: "/c/Foo/bAr",
},
{
name: "special char case",
args: args{
windowsPath: `C:\ふー\バー`,
},
wantPosixPath: "/c/ふー/バー",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if gotPosixPath := windowsToPosix(tt.args.windowsPath); gotPosixPath != tt.wantPosixPath {
t.Errorf("windowsToPosix() = %v, want %v", gotPosixPath, tt.wantPosixPath)
}
})
}
}
func Test_posixToWindows(t *testing.T) {
type args struct {
posixPath string
}
tests := []struct {
name string
args args
wantWindowsPath string
}{
{
name: "basic case",
args: args{
posixPath: "/c/some/windows/place",
},
wantWindowsPath: `C:\some\windows\place`,
},
{
name: "escaped case",
args: args{
posixPath: "/c/some/windows/place",
},
wantWindowsPath: `C:\\some\\windows\\place`,
},
{
name: "forward slash",
args: args{
posixPath: "/c/foo/bar",
},
wantWindowsPath: `C:/foo/bar`,
},
{
name: "mix slash",
args: args{
posixPath: "/c/foo/bar",
},
wantWindowsPath: `C:\foo/bar\`,
},
{
name: "case sensitive case",
args: args{
posixPath: "/c/Foo/bAr",
},
wantWindowsPath: `C:\Foo/bAr\`,
},
{
name: "special char case",
args: args{
posixPath: "/c/ふー/バー",
},
wantWindowsPath: `C:\ふー\バー`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if gotWindowsPath := posixToWindows(tt.args.posixPath); gotWindowsPath != tt.wantWindowsPath {
t.Errorf("posixToWindows() = %v, want %v", gotWindowsPath, tt.wantWindowsPath)
}
})
}
}
+5 -1
View File
@@ -29,11 +29,15 @@ func newFileIndexer(path, base string, visitors ...PathIndexVisitor) *fileIndexe
base: base,
tree: filetree.New(),
index: filetree.NewIndex(),
// Unlike the directory indexer there is no skipPathsByMountTypeAndName here: it exists
// to keep a directory walk out of /proc, /sys and similar, and to do so it reads the
// whole mount table on construction. A file source indexes exactly one file and its
// parent, so there is nothing to walk into, and the mount table read is paid for
// every file source with nothing in return.
pathIndexVisitors: append(
[]PathIndexVisitor{
requireFileInfo,
disallowByFileType,
skipPathsByMountTypeAndName(path),
},
visitors...,
),
@@ -0,0 +1,51 @@
package fileresolver
import (
"os"
"path/filepath"
"testing"
"github.com/moby/sys/mountinfo"
)
// BenchmarkNewFromFile is the cost of building a resolver for a single-file source, which is
// paid once per file by callers that catalog files individually. The default case is
// NewFromFile as shipped. The with-mount-skipper case attaches skipPathsByMountTypeAndName
// the way the file indexer used to, so the difference is what reading the mount table for
// every file source cost. That cost scales with the mount table, so its size is reported as
// the mounts metric: expect a bigger gap on a container host than on a laptop.
//
// go test ./syft/internal/fileresolver/ -run '^$' -bench NewFromFile -benchmem -count 6 | tee bench.txt
// benchstat -col /variant bench.txt
func BenchmarkNewFromFile(b *testing.B) {
path := filepath.Join(b.TempDir(), "file.txt")
if err := os.WriteFile(path, []byte("hello\n"), 0o644); err != nil {
b.Fatal(err)
}
mounts, err := mountinfo.GetMounts(nil)
if err != nil {
b.Fatal(err)
}
b.Run("variant=default", func(b *testing.B) {
for b.Loop() {
if _, err := NewFromFile(path); err != nil {
b.Fatal(err)
}
}
b.ReportMetric(float64(len(mounts)), "mounts")
})
b.Run("variant=with-mount-skipper", func(b *testing.B) {
for b.Loop() {
r, err := newFromFileWithoutIndex(path, skipPathsByMountTypeAndName(path))
if err != nil {
b.Fatal(err)
}
if err := r.buildIndex(); err != nil {
b.Fatal(err)
}
}
b.ReportMetric(float64(len(mounts)), "mounts")
})
}
@@ -3,13 +3,11 @@ package fileresolver
import (
"io/fs"
"os"
"path"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/stereoscope/pkg/file"
)
// - Verify that both the parent and the path are indexed
@@ -38,11 +36,7 @@ func Test_index(t *testing.T) {
info, err := os.Stat(test.path)
assert.NoError(t, err)
// note: the index uses absolute paths, so assertions MUST keep this in mind
cwd, err := os.Getwd()
require.NoError(t, err)
p := file.Path(path.Join(cwd, test.path))
p := indexedPath(t, test.path)
assert.Equal(t, true, tree.HasPath(p))
exists, ref, err := tree.File(p)
assert.Equal(t, true, exists)
@@ -71,7 +65,9 @@ func Test_ignoresPathIfFiltered(t *testing.T) {
testPath := "testdata/system_paths/target/home/place"
cwd, cwdErr := os.Getwd()
require.NoError(t, cwdErr)
ignorePath := path.Join(cwd, testPath)
// filepath (not path): filters see the native absolute path (before posix normalization), so the
// expected value must be joined with the host separator to match on windows
ignorePath := filepath.Join(cwd, testPath)
filterFn := func(_, path string, _ os.FileInfo, _ error) error {
if path == ignorePath {
return ErrSkipPath
@@ -91,7 +87,8 @@ func Test_ignoresPathIfParentFiltered(t *testing.T) {
cwd, cwdErr := os.Getwd()
require.NoError(t, cwdErr)
ignorePath := path.Join(cwd, parentPath)
// filepath (not path): see Test_ignoresPathIfFiltered
ignorePath := filepath.Join(cwd, parentPath)
filterFn := func(_, path string, _ os.FileInfo, _ error) error {
if path == ignorePath {
return fs.SkipDir
@@ -6,6 +6,7 @@ import (
"fmt"
"io"
"os"
"path/filepath"
stereoscopeFile "github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/stereoscope/pkg/filetree"
@@ -34,8 +35,17 @@ func nativeOSFileOpener(ref stereoscopeFile.Reference) (io.ReadCloser, error) {
return stereoscopeFile.NewLazyReadCloser(filePath), nil
}
// requestPath converts a user (chroot) path into the path the file tree is keyed on. The tree is always posix
// (windows paths are volume-encoded, e.g. /c/some/path), while ToNativePath yields a native host path.
func (r *FiletreeResolver) requestPath(userPath string) (string, error) {
return r.Chroot.ToNativePath(userPath)
nativePath, err := r.Chroot.ToNativePath(userPath)
if err != nil {
return "", err
}
if windows.HostRunningOnWindows() {
return windows.ToPosix(nativePath), nil
}
return nativePath, nil
}
// responsePath takes a path from the underlying fs domain and converts it to a path that is relative to the root of the file resolver.
@@ -85,10 +95,6 @@ func (r FiletreeResolver) FilesByPath(userPaths ...string) ([]file.Location, err
continue
}
if windows.HostRunningOnWindows() {
userStrPath = windows.ToPosix(userStrPath)
}
if ref.HasReference() {
references = append(references,
file.NewVirtualLocationFromDirectory(
@@ -103,8 +109,19 @@ func (r FiletreeResolver) FilesByPath(userPaths ...string) ([]file.Location, err
return references, nil
}
// requestGlob is the glob equivalent of requestPath.
func (r FiletreeResolver) requestGlob(pattern string) (string, error) {
return r.Chroot.ToNativeGlob(pattern)
glob, err := r.Chroot.ToNativeGlob(pattern)
if err != nil {
return "", err
}
// only a glob that was anchored to the root comes back native. Globs that start with a wildcard
// (e.g. **/foo) are returned untouched and are already posix, and ToPosix would root them at "/".
// This uses filepath (not path) since we are asking whether this is a native absolute path.
if windows.HostRunningOnWindows() && filepath.IsAbs(glob) {
return windows.ToPosix(glob), nil
}
return glob, nil
}
// FilesByGlob returns all file.References that match the given path glob pattern from any layer in the image.
+24 -7
View File
@@ -22,6 +22,15 @@ func SkipWhitespace(data []byte, i *int) {
}
}
// MaxDepth bounds nesting in the recursive hand-written parsers. Real files nest a handful of levels; without a
// bound, deep input overflows the goroutine stack, which is fatal rather than a recoverable panic.
const MaxDepth = 1000
// errorContext is how many bytes of each echoed line PrintError keeps around the error column, so an error in a
// huge single-line input stays small (and does not copy the file into the SBOM as an unknown).
const errorContext = 80
// PrintError describes where in data offset i is, echoing up to two lines of context clipped around the column.
func PrintError(data []byte, i int) string {
line := 1
char := 1
@@ -57,15 +66,23 @@ func PrintError(data []byte, i int) string {
sep := ": "
lines := ""
if len(prev) > 1 {
lines += fmt.Sprintf("%s%s%s\n", l1, sep, prev[len(prev)-2])
}
if len(prev) > 0 {
lines += fmt.Sprintf("%s%s%s\n", l2, sep, prev[len(prev)-1])
start := max(0, char-1-errorContext/2)
clip := func(s string) string {
if start >= len(s) {
return ""
}
return s[start:min(len(s), start+errorContext)]
}
pointer := strings.Repeat(" ", len(l2)+len(sep)+char-1) + "^"
lines := ""
if len(prev) > 1 {
lines += fmt.Sprintf("%s%s%s\n", l1, sep, clip(prev[len(prev)-2]))
}
if len(prev) > 0 {
lines += fmt.Sprintf("%s%s%s\n", l2, sep, clip(prev[len(prev)-1]))
}
pointer := strings.Repeat(" ", len(l2)+len(sep)+char-1-start) + "^"
return fmt.Sprintf("line: %v, char: %v\n%s%s", line, char, lines, pointer)
}
+8
View File
@@ -35,7 +35,15 @@ func ToPosix(windowsPath string) (posixPath string) {
func FromPosix(posixPath string) (windowsPath string) {
// decode the volume (e.g. /c/<path> --> C:\\) - There should always be a volume name.
// The volume may be a UNC path (e.g. /\\localhost\C$\ --> \\localhost\C$\)
if !strings.HasPrefix(posixPath, "/") {
// not a volume-encoded posix path (e.g. relative or already native), so there is nothing to decode
return posixPath
}
pathFields := strings.Split(posixPath, "/")
if len(pathFields) < 2 {
// no separator means no volume to decode
return filepath.Clean(posixPath)
}
rootPath := strings.ToUpper(pathFields[1])
volumeName := AppendRootTerminator(rootPath)
@@ -0,0 +1,35 @@
package windows
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestToPosix(t *testing.T) {
tests := map[string]string{
`C:\some\windows\place`: "/c/some/windows/place",
`C:\\some\\windows\\place`: "/c/some/windows/place",
`C:/foo/bar`: "/c/foo/bar",
`C:\foo/bar\`: "/c/foo/bar",
`C:\Foo/bAr\`: "/c/Foo/bAr",
`C:\ふー\バー`: "/c/ふー/バー",
}
for in, want := range tests {
assert.Equal(t, want, ToPosix(in), in)
}
}
func TestFromPosix(t *testing.T) {
tests := map[string]string{
"/c/some/windows/place": `C:\some\windows\place`,
"/c/Foo/bAr": `C:\Foo\bAr`,
"/c/ふー/バー": `C:\ふー\バー`,
// non volume-encoded input is left alone
"relative/path": "relative/path",
`D:\already\native`: `D:\already\native`,
}
for in, want := range tests {
assert.Equal(t, want, FromPosix(in), in)
}
}
+2 -2
View File
@@ -156,8 +156,8 @@ var busyboxVersionMatcher = regexp.MustCompile(`BusyBox v[\d.]+`)
func parseBusyBox(contents string) (*Release, error) {
matches := busyboxVersionMatcher.FindAllString(contents, -1)
for _, match := range matches {
parts := strings.Split(match, " ")
version := strings.ReplaceAll(parts[1], "v", "")
_, version, _ := strings.Cut(match, " ")
version = strings.ReplaceAll(version, "v", "")
return simpleRelease(match, "busybox", version, ""), nil
}
+6 -2
View File
@@ -11,8 +11,12 @@ import (
const (
catalogerName = "gguf-cataloger"
ggufLayerMediaType = "application/vnd.docker.ai*"
safeTensorsCatalogerName = "safetensors-cataloger"
// dockerAIGGUFMediaType mirrors ggufLayerMediaType in
// syft/source/ocimodelsource/registry_client.go. It is matched exactly: the
// other application/vnd.docker.ai.* layers are not GGUF.
dockerAIGGUFMediaType = "application/vnd.docker.ai.gguf.v3"
)
// NewGGUFCataloger returns a new cataloger instance for GGUF model files.
@@ -21,7 +25,7 @@ const (
func NewGGUFCataloger() pkg.Cataloger {
return generic.NewCataloger(catalogerName).
WithParserByGlobs(parseGGUFModel, "**/*.gguf").
WithParserByMediaType(parseGGUFModel, ggufLayerMediaType).
WithParserByMediaType(parseGGUFModel, dockerAIGGUFMediaType).
WithProcessors(ggufMergeProcessor)
}
+46
View File
@@ -1,10 +1,17 @@
package ai
import (
"context"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal/tmpdir"
"github.com/anchore/syft/syft/internal/fileresolver"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
@@ -138,3 +145,42 @@ func TestGGUFCataloger(t *testing.T) {
})
}
}
func TestGGUFCataloger_ociLayers(t *testing.T) {
// a GGUF OCI artifact is merged into one named package, while the other
// application/vnd.docker.ai.* layers (config, model.file, license and
// safetensors weights) never reach the GGUF parser.
named := newTestGGUFBuilder().
withStringKV("general.architecture", "llama").
withStringKV("general.name", "llama3-8b").
build()
part := newTestGGUFBuilder().withStringKV("general.architecture", "llama").build()
dir := t.TempDir()
layers := map[string]fileresolver.LayerInfo{}
for digest, l := range map[string]struct {
mediaType string
content []byte
}{
"sha256:named": {dockerAIGGUFMediaType, named},
"sha256:part": {dockerAIGGUFMediaType, part},
"sha256:config": {dockerAIModelConfigMediaTypes[0], []byte("not a gguf file")},
"sha256:file": {dockerAIModelFileMediaType, []byte("not a gguf file")},
"sha256:license": {dockerAILicenseMediaType, []byte("not a gguf file")},
"sha256:weights": {dockerAISafeTensorsMediaType, []byte("not a gguf file")},
} {
p := filepath.Join(dir, digest[len("sha256:"):])
require.NoError(t, os.WriteFile(p, l.content, 0o600))
layers[digest] = fileresolver.LayerInfo{TempPath: p, MediaType: l.mediaType}
}
resolver := fileresolver.NewContainerImageModel(dir, layers, "example/model:latest")
ctx := tmpdir.WithValue(context.Background(), tmpdir.FromPath(t.TempDir()))
pkgs, _, err := NewGGUFCataloger().Catalog(ctx, resolver)
require.NoError(t, err)
require.Len(t, pkgs, 1)
assert.Equal(t, "llama3-8b", pkgs[0].Name)
header, ok := pkgs[0].Metadata.(pkg.GGUFFileHeader)
require.True(t, ok)
assert.Len(t, header.Parts, 1)
}
+7 -7
View File
@@ -8,17 +8,17 @@ import (
// ggufMergeProcessor consolidates multiple GGUF packages into a single package
// representing the AI model. When scanning OCI images with multiple layers,
// each layer may produce a separate package. This processor finds the package
// with a name and merges metadata from nameless packages into its GGUFFileParts field.
// with a name and merges metadata from nameless packages into its Parts field.
// Only packages with a non-empty name are returned in the final result.
func ggufMergeProcessor(pkgs []pkg.Package, rels []artifact.Relationship, err error) ([]pkg.Package, []artifact.Relationship, error) {
if err != nil {
return pkgs, rels, err
}
// don't early-return on err: it aggregates per-file failures, and skipping the
// merge would drop every nameless layer and the named package's Parts along
// with it (see safeTensorsMergeProcessor).
if len(pkgs) == 0 {
return pkgs, rels, err
}
// Separate packages with names from those without
// separate packages with names from those without
var namedPkgs []pkg.Package
var namelessHeaders []pkg.GGUFFileHeader
@@ -27,14 +27,14 @@ func ggufMergeProcessor(pkgs []pkg.Package, rels []artifact.Relationship, err er
namedPkgs = append(namedPkgs, p)
} else {
if header, ok := p.Metadata.(pkg.GGUFFileHeader); ok {
// We do not want a kv hash for nameless headers
// we do not want a kv hash for nameless headers
header.MetadataKeyValuesHash = ""
namelessHeaders = append(namelessHeaders, header)
}
}
}
// If there are no named packages, return nothing
// if there are no named packages, return nothing
if len(namedPkgs) == 0 {
return nil, rels, err
}
+60 -20
View File
@@ -3,7 +3,7 @@ package ai
import (
"bytes"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/pkg"
@@ -15,6 +15,12 @@ type hfConfig struct {
NameOrPath string `json:"_name_or_path"`
}
// looksLikeHF reports whether the config carries any field we use. Other JSON
// (generation_config.json, unrelated app configs) is ignored.
func (c hfConfig) looksLikeHF() bool {
return len(c.Architectures) > 0 || c.NameOrPath != ""
}
func applyHFConfig(md *pkg.SafeTensorsModelInfo, cfg *hfConfig) {
if md.Architecture == "" && len(cfg.Architectures) > 0 {
md.Architecture = cfg.Architectures[0]
@@ -23,8 +29,8 @@ func applyHFConfig(md *pkg.SafeTensorsModelInfo, cfg *hfConfig) {
// readmeFrontmatter holds the subset of YAML frontmatter fields we extract.
type readmeFrontmatter struct {
License string `yaml:"license"`
BaseModel []string `yaml:"base_model"`
Licenses []string
BaseModel []string
}
type licenseFrontmatter struct {
@@ -32,21 +38,33 @@ type licenseFrontmatter struct {
}
// extractFrontmatterBlock returns the YAML bytes between the first and second
// "---" delimiters of a file
// "---" delimiter lines of a file
func extractFrontmatterBlock(buf []byte) []byte {
trimmed := bytes.TrimLeft(buf, "\xef\xbb\xbf \t\r\n")
if !bytes.HasPrefix(trimmed, []byte("---")) {
first, rest, _ := bytes.Cut(trimmed, []byte("\n"))
if !isFrontmatterDelimiter(first) {
return nil
}
rest := trimmed[3:]
if i := bytes.IndexByte(rest, '\n'); i >= 0 {
rest = rest[i+1:]
for off := 0; off < len(rest); {
line, _, found := bytes.Cut(rest[off:], []byte("\n"))
if isFrontmatterDelimiter(line) {
if off == 0 || off > maxFrontmatterSize {
return nil
}
return rest[:off]
}
if !found {
break
}
off += len(line) + 1
}
block, _, found := bytes.Cut(rest, []byte("\n---"))
if !found {
return nil
}
return block
return nil
}
// isFrontmatterDelimiter reports whether a line is exactly "---", ignoring
// trailing whitespace and \r.
func isFrontmatterDelimiter(line []byte) bool {
return bytes.Equal(bytes.TrimRight(line, " \t\r"), []byte("---"))
}
// parseFrontmatter decodes a Hugging Face model card YAML frontmatter block
@@ -57,8 +75,9 @@ func parseFrontmatter(buf []byte) *readmeFrontmatter {
return nil
}
// both fields may be a scalar or a list, so decode them as nodes
var raw struct {
License string `yaml:"license"`
License yaml.Node `yaml:"license"`
BaseModel yaml.Node `yaml:"base_model"`
}
if err := yaml.Unmarshal(block, &raw); err != nil {
@@ -66,16 +85,37 @@ func parseFrontmatter(buf []byte) *readmeFrontmatter {
return nil
}
fm := readmeFrontmatter{License: raw.License}
switch raw.BaseModel.Kind {
return &readmeFrontmatter{
Licenses: yamlStrings(raw.License),
BaseModel: yamlStrings(raw.BaseModel),
}
}
// yamlStrings returns the non-empty, non-null values of a scalar or a sequence
// of scalars, keeping at most maxFrontmatterValues. Anything else yields nil.
func yamlStrings(n yaml.Node) []string {
switch n.Kind {
case yaml.ScalarNode:
if raw.BaseModel.Value != "" {
fm.BaseModel = []string{raw.BaseModel.Value}
if isYAMLString(&n) {
return []string{n.Value}
}
case yaml.SequenceNode:
_ = raw.BaseModel.Decode(&fm.BaseModel)
var out []string
for _, c := range n.Content {
if len(out) == maxFrontmatterValues {
break
}
if c.Kind == yaml.ScalarNode && isYAMLString(c) {
out = append(out, c.Value)
}
}
return out
}
return &fm
return nil
}
func isYAMLString(n *yaml.Node) bool {
return n.Value != "" && n.ShortTag() != "!!null"
}
// parseLicenseFrontmatter returns the producer-declared SPDX identifier
+28 -20
View File
@@ -3,7 +3,6 @@ package ai
import (
"context"
"encoding/json"
"io"
"path"
"github.com/anchore/syft/internal"
@@ -13,42 +12,42 @@ import (
)
// resolveSafeTensorsDirIdentity handles the directory-scan case for safe tensors
// find config.json beside the model files (walking up parent directories to the
// scanned source root if no sibling exists) and a sibling README.md. It returns
// find config.json beside the model files (or one directory up, for the shard
// subdirectory layout) and a sibling README.md. It returns
// the group's name candidates, resolved licenses, and supporting evidence.
func resolveSafeTensorsDirIdentity(ctx context.Context, resolver file.Resolver, dir string, md *pkg.SafeTensorsModelInfo) safeTensorsIdentity {
id := safeTensorsIdentity{fallbackName: safeTensorsDirName(dir)}
if loc, cfg := findDirHFConfig(resolver, dir); cfg != nil {
applyHFConfig(md, cfg)
id.nameOrPath = cfg.NameOrPath
id.nameOrPath = sanitizeModelName(cfg.NameOrPath)
id.supporting = append(id.supporting, *loc)
}
if loc, fm := readDirReadmeFrontmatter(resolver, path.Join(dir, "README.md")); fm != nil {
if fm.License != "" {
id.licenses = pkg.NewLicensesFromValuesWithContext(ctx, fm.License)
if len(fm.Licenses) > 0 {
id.licenses = pkg.NewLicensesFromValuesWithContext(ctx, fm.Licenses...)
}
if id.nameOrPath == "" && len(fm.BaseModel) > 0 {
id.nameOrPath = fm.BaseModel[0]
if id.nameOrPath == "" {
id.nameOrPath = firstModelName(fm.BaseModel)
}
id.supporting = append(id.supporting, *loc)
}
return id
}
// findDirHFConfig looks for a config.json beside the model files
// findDirHFConfig looks for an HF config.json beside the model files, then in
// the parent directory (covers weights kept in a subdirectory of the model). It
// does not walk further up, so an unrelated config.json higher in the tree can't
// name every model below it.
func findDirHFConfig(resolver file.Resolver, dir string) (*file.Location, *hfConfig) {
for {
if loc, cfg := readDirHFConfig(resolver, path.Join(dir, "config.json")); cfg != nil {
return loc, cfg
}
parent := path.Dir(dir)
if parent == dir {
return nil, nil // reached the source root
}
dir = parent
if loc, cfg := readDirHFConfig(resolver, path.Join(dir, "config.json")); cfg != nil {
return loc, cfg
}
if parent := path.Dir(dir); parent != dir {
return readDirHFConfig(resolver, path.Join(parent, "config.json"))
}
return nil, nil
}
func readDirHFConfig(resolver file.Resolver, p string) (*file.Location, *hfConfig) {
@@ -62,11 +61,19 @@ func readDirHFConfig(resolver file.Resolver, p string) (*file.Location, *hfConfi
}
defer internal.CloseAndLogError(rc, p)
buf, err := readBounded(rc, maxHFConfigSize)
if err != nil {
log.Debugf("failed to read %s: %v", p, err)
return nil, nil
}
var cfg hfConfig
if err := json.NewDecoder(rc).Decode(&cfg); err != nil {
if err := json.Unmarshal(buf, &cfg); err != nil {
log.Debugf("failed to decode %s: %v", p, err)
return nil, nil
}
if !cfg.looksLikeHF() {
return nil, nil
}
return &locations[0], &cfg
}
@@ -81,8 +88,9 @@ func readDirReadmeFrontmatter(resolver file.Resolver, p string) (*file.Location,
}
defer internal.CloseAndLogError(rc, p)
buf, err := io.ReadAll(io.LimitReader(rc, 1024*1024))
buf, err := readPrefix(rc, maxReadmePrefixSize)
if err != nil {
log.Debugf("failed to read %s: %v", p, err)
return nil, nil
}
fm := parseFrontmatter(buf)
+32 -20
View File
@@ -4,8 +4,8 @@ import (
"bytes"
"context"
"encoding/json"
"io"
"path"
"sort"
gcrname "github.com/google/go-containerregistry/pkg/name"
@@ -28,9 +28,18 @@ func resolveSafeTensorsOCIIdentity(ctx context.Context, resolver file.Resolver,
log.Debugf("failed to list docker AI model-file layers: %v", err)
}
// Collect config / readme candidates separately so the layer-iteration order
// returned by the resolver doesn't decide the precedence.
var configName, readmeName, readmeLicense string
// the resolver returns layers in map order, and the first config / README
// seen wins below, so sort for a stable result across runs. For OCI
// artifacts every RealPath is "/", so in practice this is digest order.
sort.Slice(modelFileLocs, func(i, j int) bool {
if modelFileLocs[i].RealPath != modelFileLocs[j].RealPath {
return modelFileLocs[i].RealPath < modelFileLocs[j].RealPath
}
return modelFileLocs[i].FileSystemID < modelFileLocs[j].FileSystemID
})
var configName, readmeName string
var readmeLicenses []string
var supporting []file.Location
for _, loc := range modelFileLocs {
cfg, fm := classifyOCIModelFileLayer(resolver, loc)
@@ -38,15 +47,15 @@ func resolveSafeTensorsOCIIdentity(ctx context.Context, resolver file.Resolver,
case cfg != nil:
applyHFConfig(md, cfg)
if configName == "" {
configName = cfg.NameOrPath
configName = sanitizeModelName(cfg.NameOrPath)
}
supporting = append(supporting, loc)
case fm != nil:
if readmeLicense == "" {
readmeLicense = fm.License
if len(readmeLicenses) == 0 {
readmeLicenses = fm.Licenses
}
if readmeName == "" && len(fm.BaseModel) > 0 {
readmeName = fm.BaseModel[0]
if readmeName == "" {
readmeName = firstModelName(fm.BaseModel)
}
supporting = append(supporting, loc)
}
@@ -64,7 +73,7 @@ func resolveSafeTensorsOCIIdentity(ctx context.Context, resolver file.Resolver,
supporting: supporting,
}
// License precedence: a dedicated vnd.docker.ai.license layer is a
// License precedence: a dedicated vnd.docker.ai.license layer
// outranks the free-text license field in a model card's README frontmatter.
licLocs, err := ociResolver.FilesByMediaType(dockerAILicenseMediaType)
if err != nil {
@@ -74,8 +83,8 @@ func resolveSafeTensorsOCIIdentity(ctx context.Context, resolver file.Resolver,
case len(licLocs) > 0:
id.licenses = identifyLicenseLayers(ctx, resolver, licLocs)
id.supporting = append(id.supporting, licLocs...)
case readmeLicense != "":
id.licenses = pkg.NewLicensesFromValuesWithContext(ctx, readmeLicense)
case len(readmeLicenses) > 0:
id.licenses = pkg.NewLicensesFromValuesWithContext(ctx, readmeLicenses...)
}
return id
@@ -108,10 +117,11 @@ func ociImageRefBasename(resolver file.Resolver) string {
return path.Base(parsed.Context().RepositoryStr())
}
// classifyOCIModelFileLayer reads up to 4 MiB of a model.file layer and decodes
// it as either an HF config.json or a README model card's YAML frontmatter,
// based on its leading bytes. It returns whichever it recognized; both are nil
// when the layer is neither (or fails to decode). The caller owns precedence and
// classifyOCIModelFileLayer reads a model.file layer (up to maxModelFileLayerSize)
// and decodes it as either an HF config.json or a README model card's YAML
// frontmatter, based on its leading bytes. It returns whichever it recognized;
// both are nil when the layer is neither, fails to decode, or is JSON that
// doesn't look like an HF config (e.g. generation_config.json). The caller owns precedence and
// metadata enrichment.
func classifyOCIModelFileLayer(resolver file.Resolver, loc file.Location) (*hfConfig, *readmeFrontmatter) {
rc, err := resolver.FileContentsByLocation(loc)
@@ -120,17 +130,18 @@ func classifyOCIModelFileLayer(resolver file.Resolver, loc file.Location) (*hfCo
}
defer internal.CloseAndLogError(rc, loc.RealPath)
buf, err := io.ReadAll(io.LimitReader(rc, 4*1024*1024))
buf, err := readBounded(rc, maxModelFileLayerSize)
if err != nil {
log.Debugf("failed to read docker AI model-file layer: %v", err)
return nil, nil
}
trimmed := bytes.TrimLeft(buf, "\xef\xbb\xbf \t\r\n")
switch {
case bytes.HasPrefix(trimmed, []byte("---")):
return nil, parseFrontmatter(buf)
return nil, parseFrontmatter(trimmed)
case bytes.HasPrefix(trimmed, []byte("{")):
var cfg hfConfig
if err := json.Unmarshal(buf, &cfg); err != nil {
if err := json.Unmarshal(trimmed, &cfg); err != nil || !cfg.looksLikeHF() {
return nil, nil
}
return &cfg, nil
@@ -165,8 +176,9 @@ func readLicenseSPDXIDFromFrontmatter(resolver file.Resolver, loc file.Location)
}
defer internal.CloseAndLogError(rc, loc.RealPath)
buf, err := io.ReadAll(io.LimitReader(rc, 64*1024))
buf, err := readPrefix(rc, maxLicensePrefixSize)
if err != nil {
log.Debugf("failed to read docker AI license layer: %v", err)
return ""
}
return parseLicenseFrontmatter(buf)
+46
View File
@@ -0,0 +1,46 @@
package ai
import (
"fmt"
"io"
"github.com/anchore/syft/internal"
)
// size limits for model companion files and headers. JSON reads that hit a
// limit fail (see readBounded) instead of parsing a truncated document.
// Frontmatter-only reads (README, license) take a prefix instead, since the
// frontmatter sits at the top and is itself capped by maxFrontmatterSize.
const (
maxHFConfigSize = 4 * 1024 * 1024
maxReadmePrefixSize = 1 * 1024 * 1024 // dir scans; OCI README layers use maxModelFileLayerSize
maxModelFileLayerSize = 4 * 1024 * 1024
maxLicensePrefixSize = 64 * 1024
maxModelConfigBlobSize = 1 * 1024 * 1024
maxFrontmatterSize = 256 * 1024
maxFrontmatterValues = 32
maxModelNameLength = 256
// maxSafeTensorsHeaderSize caps the header JSON (excluding its 8-byte length
// prefix). It is shared with the OCI source so dir and OCI scans accept the
// same headers.
maxSafeTensorsHeaderSize = internal.MaxSafeTensorsHeaderSize
)
// readPrefix reads at most limit bytes. It is for frontmatter-only reads, where
// the rest of the file is never needed.
func readPrefix(r io.Reader, limit int64) ([]byte, error) {
return io.ReadAll(io.LimitReader(r, limit))
}
// readBounded reads at most limit bytes and reports overflow instead of silently truncating.
func readBounded(r io.Reader, limit int64) ([]byte, error) {
b, err := io.ReadAll(io.LimitReader(r, limit+1))
if err != nil {
return nil, err
}
if int64(len(b)) > limit {
return nil, fmt.Errorf("content exceeds %d bytes", limit)
}
return b, nil
}
+1 -1
View File
@@ -75,7 +75,7 @@ func mergeShardsInto(merged *pkg.SafeTensorsModelInfo, shards []pkg.SafeTensorsM
seenKV := map[string]bool{}
for _, s := range shards {
shardTensorTotal += s.TensorCount
merged.Parameters += s.Parameters
merged.Parameters = saturatingAdd(merged.Parameters, s.Parameters)
firstNonEmpty(&merged.Quantization, s.Quantization)
for _, kv := range s.UserMetadata {
if seenKV[kv.Key] {
+8
View File
@@ -1,6 +1,7 @@
package ai
import (
"math"
"testing"
"github.com/stretchr/testify/assert"
@@ -164,3 +165,10 @@ func TestRollupHash(t *testing.T) {
assert.NotEqual(t, "a", ab)
assert.NotEqual(t, "b", ab)
}
func TestMergeShardsInto_parametersSaturate(t *testing.T) {
var merged pkg.SafeTensorsModelInfo
shards := []pkg.SafeTensorsModelInfo{{Parameters: math.MaxUint64 - 1}, {Parameters: 2}}
mergeShardsInto(&merged, shards)
assert.Equal(t, uint64(math.MaxUint64), merged.Parameters)
}
+32 -3
View File
@@ -1,15 +1,44 @@
package ai
import "path"
import (
"path"
"strings"
"unicode/utf8"
)
// pickSafeTensorsName implements the documented naming precedence chain:
func pickSafeTensorsName(nameOrPath, fallbackName string) string {
if nameOrPath != "" {
return path.Base(nameOrPath)
if name := sanitizeModelName(nameOrPath); name != "" {
return name
}
return fallbackName
}
// firstModelName returns the first value that sanitizes to a usable name.
func firstModelName(values []string) string {
for _, v := range values {
if name := sanitizeModelName(v); name != "" {
return name
}
}
return ""
}
// sanitizeModelName reduces a producer-supplied name or path (_name_or_path,
// base_model) to its last path element, capped at maxModelNameLength runes. It
// returns "" when nothing usable is left.
func sanitizeModelName(nameOrPath string) string {
base := path.Base(strings.ReplaceAll(nameOrPath, `\`, "/"))
switch base {
case "/", ".", "..", "":
return ""
}
if utf8.RuneCountInString(base) > maxModelNameLength {
base = string([]rune(base)[:maxModelNameLength])
}
return base
}
// safeTensorsDirName returns the directory-scan naming fallback: the base name
// of the group's parent directory (the group key is already that directory).
func safeTensorsDirName(directory string) string {
+4 -2
View File
@@ -1,17 +1,19 @@
package ai
import (
"context"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
func newGGUFPackage(metadata *pkg.GGUFFileHeader, modelName, version, license string, locations ...file.Location) pkg.Package {
func newGGUFPackage(ctx context.Context, metadata *pkg.GGUFFileHeader, modelName, version, license string, locations ...file.Location) pkg.Package {
p := pkg.Package{
Name: modelName,
Version: version,
Locations: file.NewLocationSet(locations...),
Type: pkg.ModelPkg,
Licenses: pkg.NewLicenseSet(pkg.NewLicensesFromValues(license)...),
Licenses: pkg.NewLicenseSet(pkg.NewLicensesFromValuesWithContext(ctx, license)...),
Metadata: *metadata,
// NOTE: PURL is intentionally not set as the package-url spec
// has not yet finalized support for ML model packages
+2
View File
@@ -1,6 +1,7 @@
package ai
import (
"context"
"testing"
"github.com/stretchr/testify/require"
@@ -103,6 +104,7 @@ func TestNewGGUFPackage(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
actual := newGGUFPackage(
context.Background(),
tt.metadata,
tt.input.modelName,
tt.input.version,
+300 -5
View File
@@ -1,9 +1,14 @@
package ai
import (
"bufio"
"encoding/binary"
"errors"
"fmt"
"io"
"math"
"slices"
"strings"
gguf_parser "github.com/gpustack/gguf-parser-go"
)
@@ -12,12 +17,41 @@ import (
const (
ggufMagicNumber = 0x46554747 // "GGUF" in little-endian
maxHeaderSize = 50 * 1024 * 1024 // 50MB for large tokenizer vocabularies
// maxGGUFArrayDepth allows arrays of arrays but nothing deeper.
maxGGUFArrayDepth = 2
// maxGGUFDecodedItems bounds the items, across the whole header, of arrays that gguf-parser-go
// always decodes in full. Real files hold a few per-layer arrays.
maxGGUFDecodedItems = 65536
// maxGGUFSplitTensors bounds split.tensors.count, which the library preallocates tensor infos for.
// Shards declare the total across all splits, so this can't be the header tensor count.
maxGGUFSplitTensors = 1 << 17
// maxGGUFKeyLength is the key length limit from the GGUF spec.
maxGGUFKeyLength = 65535
// maxGGUFKVCount bounds the metadata KV count. Real files hold hundreds; vocabularies live in arrays.
maxGGUFKVCount = 65536
// maxGGUFReadStringBytes bounds, across the header, the key and string bytes the library reads in
// full and syft then hashes and encodes (chat templates are the largest, at tens of KB).
maxGGUFReadStringBytes = 8 << 20
// maxGGUFSkippedStrings bounds, across the header, the strings the library skips one at a time
// inside arrays (vocabularies run to a few hundred thousand tokens and merges).
maxGGUFSkippedStrings = 1 << 22
)
// copyHeader copies the GGUF header from the reader to the writer.
// ggufDecodedArraySuffixes are the keys gguf-parser-go decodes in full even with SkipLargeMetadata
// (hardcoded in its ReadArray as of v0.26.3; re-check on upgrade).
var ggufDecodedArraySuffixes = []string{
".feed_forward_length", ".attention.head_count", ".attention.head_count_kv", ".attention.sliding_window_pattern",
}
// ggufScalarSizes is the encoded size of each value type, indexed by type (0 for strings and arrays).
var ggufScalarSizes = [...]int{1, 1, 2, 2, 4, 4, 4, 1, 0, 0, 8, 8, 8}
// copyHeader copies at most maxHeaderSize bytes of the GGUF header from the reader to the writer.
// It validates the magic number first, then copies the rest of the data.
// The reader should be wrapped with io.LimitedReader to prevent OOM issues.
func copyHeader(w io.Writer, r io.Reader) error {
r = io.LimitReader(r, maxHeaderSize)
// Read initial chunk to validate magic number
// GGUF format: magic(4) + version(4) + tensor_count(8) + metadata_kv_count(8) + metadata_kvs + tensors_info
initialBuf := make([]byte, 24) // Enough for magic, version, tensor count, and kv count
@@ -37,7 +71,6 @@ func copyHeader(w io.Writer, r io.Reader) error {
}
// Copy the rest of the header from reader to writer
// The LimitedReader will return EOF once maxHeaderSize is reached
if _, err := io.Copy(w, r); err != nil {
return fmt.Errorf("failed to copy GGUF header: %w", err)
}
@@ -45,7 +78,269 @@ func copyHeader(w io.Writer, r io.Reader) error {
return nil
}
// Helper to convert gguf_parser metadata to simpler types
// validateGGUFHeader walks the metadata KV section and rejects values that gguf-parser-go would panic
// on, recurse on without bound, or overallocate for. It reads without storing anything.
// ponytail: tensor infos are not checked here (zero block size types, dimension overflow); those
// panics are recoverable and are left to the per-file panic recovery in the generic cataloger.
func validateGGUFHeader(r io.Reader) error {
v := &ggufHeaderValidator{
r: bufio.NewReader(r),
decodedBudget: maxGGUFDecodedItems,
readStringBudget: maxGGUFReadStringBytes,
skippedStringBudget: maxGGUFSkippedStrings,
}
var prefix struct {
Magic, Version uint32
}
if err := binary.Read(v.r, binary.LittleEndian, &prefix); err != nil {
return fmt.Errorf("reading header prefix: %w", err)
}
v.v1 = prefix.Version <= 1
// the tensor count is validated by the library
if _, err := v.readLen(); err != nil {
return fmt.Errorf("reading tensor count: %w", err)
}
kvCount, err := v.readLen()
if err != nil {
return fmt.Errorf("reading metadata count: %w", err)
}
if kvCount > maxGGUFKVCount {
return fmt.Errorf("metadata count %d exceeds %d", kvCount, maxGGUFKVCount)
}
for i := uint64(0); i < kvCount; i++ {
key, err := v.readKey()
if err != nil {
return fmt.Errorf("metadata key %d: %w", i, err)
}
var vt uint32
if err := binary.Read(v.r, binary.LittleEndian, &vt); err != nil {
return fmt.Errorf("metadata key %q: %w", key, err)
}
if err := v.checkKey(key, gguf_parser.GGUFMetadataValueType(vt)); err != nil {
return fmt.Errorf("metadata key %q: %w", key, err)
}
}
return nil
}
type ggufHeaderValidator struct {
r *bufio.Reader
v1 bool // v1 uses 32-bit lengths
decodedBudget uint64
readStringBudget uint64
skippedStringBudget uint64
}
func (v *ggufHeaderValidator) checkKey(key string, vt gguf_parser.GGUFMetadataValueType) error {
decoded := slices.ContainsFunc(ggufDecodedArraySuffixes, func(s string) bool { return strings.HasSuffix(key, s) })
n, err := v.readValue(vt, decoded, 1)
if err != nil {
return err
}
// these are read with typed accessors that panic on a type mismatch
switch key {
case "general.architecture", "general.name", "general.license", "general.type", "general.author",
"general.url", "general.description", "controlvector.model_hint":
if vt != gguf_parser.GGUFMetadataValueTypeString {
return fmt.Errorf("expected string, got %v", vt)
}
case "general.quantization_version", "general.file_type":
if !vt.IsNumeric() {
return fmt.Errorf("expected number, got %v", vt)
}
case "general.alignment":
if vt != gguf_parser.GGUFMetadataValueTypeUint32 || n == 0 {
return fmt.Errorf("expected non-zero uint32, got %v %v", vt, n)
}
case "split.tensors.count":
if !vt.IsNumeric() {
return fmt.Errorf("expected number, got %v", vt)
}
if math.IsNaN(n) || n < 0 || n > maxGGUFSplitTensors {
return fmt.Errorf("unreasonable split tensor count %.0f", n)
}
}
return nil
}
// readValue consumes one value, returning scalar numbers as float64 for the key checks.
func (v *ggufHeaderValidator) readValue(vt gguf_parser.GGUFMetadataValueType, decoded bool, depth int) (float64, error) {
if uint32(vt) >= uint32(len(ggufScalarSizes)) {
return 0, fmt.Errorf("invalid value type %d", vt)
}
switch vt {
case gguf_parser.GGUFMetadataValueTypeString:
// only strings inside skipped arrays are skipped by the library, the rest are read in full
return 0, v.skipString(depth > 1 && !decoded)
case gguf_parser.GGUFMetadataValueTypeArray:
return 0, v.readArray(decoded, depth)
}
size := ggufScalarSizes[vt]
b, err := v.r.Peek(size)
if err != nil {
return 0, err
}
var buf [8]byte
copy(buf[:], b)
if _, err := v.r.Discard(size); err != nil {
return 0, err
}
u := binary.LittleEndian.Uint64(buf[:])
switch vt {
case gguf_parser.GGUFMetadataValueTypeInt8:
return float64(int8(u)), nil
case gguf_parser.GGUFMetadataValueTypeInt16:
return float64(int16(u)), nil
case gguf_parser.GGUFMetadataValueTypeInt32:
return float64(int32(u)), nil
case gguf_parser.GGUFMetadataValueTypeInt64:
return float64(int64(u)), nil
case gguf_parser.GGUFMetadataValueTypeFloat32:
return float64(math.Float32frombits(uint32(u))), nil
case gguf_parser.GGUFMetadataValueTypeFloat64:
return math.Float64frombits(u), nil
}
return float64(u), nil
}
func (v *ggufHeaderValidator) readArray(decoded bool, depth int) error {
var itemType uint32
if err := binary.Read(v.r, binary.LittleEndian, &itemType); err != nil {
return err
}
n, err := v.readLen()
if err != nil {
return err
}
it := gguf_parser.GGUFMetadataValueType(itemType)
switch {
case itemType >= uint32(len(ggufScalarSizes)):
return fmt.Errorf("invalid array item type %d", itemType)
case decoded && n > v.decodedBudget:
return fmt.Errorf("decoded arrays exceed %d items", maxGGUFDecodedItems)
case it == gguf_parser.GGUFMetadataValueTypeArray && (!decoded || depth >= maxGGUFArrayDepth):
// the library can only skip over arrays of scalars
return errors.New("unsupported nested array")
case n > maxHeaderSize:
return fmt.Errorf("array length %d exceeds header size", n)
case !decoded && ggufScalarSizes[itemType] > 0:
// skipped arrays of fixed size items can be passed over in one go
_, err := v.r.Discard(int(n) * ggufScalarSizes[itemType])
return err
}
if decoded {
v.decodedBudget -= n
}
for i := uint64(0); i < n; i++ {
if _, err := v.readValue(it, decoded, depth+1); err != nil {
return err
}
}
return nil
}
func (v *ggufHeaderValidator) readLen() (uint64, error) {
if v.v1 {
var n uint32
err := binary.Read(v.r, binary.LittleEndian, &n)
return uint64(n), err
}
var n uint64
err := binary.Read(v.r, binary.LittleEndian, &n)
return n, err
}
// readKey reads a key the way gguf-parser-go will see it, whitespace trimmed.
func (v *ggufHeaderValidator) readKey() (string, error) {
n, err := v.readLen()
if err != nil {
return "", err
}
if n == 0 || n > maxGGUFKeyLength {
return "", fmt.Errorf("invalid key length %d", n)
}
if err := v.spendReadString(n); err != nil {
return "", err
}
buf := make([]byte, n)
if _, err := io.ReadFull(v.r, buf); err != nil {
return "", err
}
return strings.TrimSpace(string(buf)), nil
}
func (v *ggufHeaderValidator) skipString(allowEmpty bool) error {
n, err := v.readLen()
if err != nil {
return err
}
// gguf-parser-go reads an empty string as a full pool buffer, so everything after it would be
// parsed out of step with this walk
if n == 0 && !allowEmpty {
return errors.New("empty string")
}
if n > maxHeaderSize {
return fmt.Errorf("string length %d exceeds header size", n)
}
if allowEmpty {
if v.skippedStringBudget == 0 {
return fmt.Errorf("skipped strings exceed %d", maxGGUFSkippedStrings)
}
v.skippedStringBudget--
} else if err := v.spendReadString(n); err != nil {
return err
}
_, err = v.r.Discard(int(n))
return err
}
func (v *ggufHeaderValidator) spendReadString(n uint64) error {
if n > v.readStringBudget {
return fmt.Errorf("strings exceed %d bytes", maxGGUFReadStringBytes)
}
v.readStringBudget -= n
return nil
}
// sanitizeGGUFValue replaces non-finite floats, which JSON can not encode, leaving everything else as is.
func sanitizeGGUFValue(v any) any {
switch t := v.(type) {
case float32:
if f := float64(t); math.IsNaN(f) || math.IsInf(f, 0) {
return nonFiniteString(f)
}
case float64:
if math.IsNaN(t) || math.IsInf(t, 0) {
return nonFiniteString(t)
}
case gguf_parser.GGUFMetadataKVArrayValue:
// keep the library shape as is; items are only present for the few keys the library
// decodes in full, which the pre-scan keeps small and shallow
if len(t.Array) > 0 {
items := make([]any, len(t.Array))
for i, item := range t.Array {
items[i] = sanitizeGGUFValue(item)
}
t.Array = items
}
return t
}
return v
}
func nonFiniteString(f float64) string {
switch {
case math.IsInf(f, 1):
return "+Inf"
case math.IsInf(f, -1):
return "-Inf"
}
return "NaN"
}
// helper to convert gguf_parser metadata to simpler types
func convertGGUFMetadataKVs(kvs gguf_parser.GGUFMetadataKVs) map[string]any {
result := make(map[string]any)
@@ -56,7 +351,7 @@ func convertGGUFMetadataKVs(kvs gguf_parser.GGUFMetadataKVs) map[string]any {
"general.version", "general.parameter_count", "general.quantization":
continue
}
result[kv.Key] = kv.Value
result[kv.Key] = sanitizeGGUFValue(kv.Value)
}
return result
+26 -14
View File
@@ -13,9 +13,7 @@ import (
gguf_parser "github.com/gpustack/gguf-parser-go"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/tmpdir"
"github.com/anchore/syft/internal/unknown"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
@@ -38,14 +36,20 @@ func parseGGUFModel(ctx context.Context, _ file.Resolver, _ *generic.Environment
defer cleanup()
tempPath := tempFile.Name()
// Copy and validate the GGUF file header using LimitedReader to prevent OOM
// We use LimitedReader to cap reads at maxHeaderSize (50MB)
limitedReader := &io.LimitedReader{R: reader, N: maxHeaderSize}
if err := copyHeader(tempFile, limitedReader); err != nil {
// copy and validate the GGUF file header (bounded to maxHeaderSize)
if err := copyHeader(tempFile, reader); err != nil {
tempFile.Close()
return nil, nil, fmt.Errorf("failed to copy GGUF header: %w", err)
}
if _, err := tempFile.Seek(0, io.SeekStart); err != nil {
tempFile.Close()
return nil, nil, fmt.Errorf("failed to rewind GGUF header: %w", err)
}
err = validateGGUFHeader(tempFile)
tempFile.Close()
if err != nil {
return nil, nil, fmt.Errorf("invalid GGUF header: %w", err)
}
// Parse using gguf-parser-go with options to skip unnecessary data
ggufFile, err := gguf_parser.ParseGGUFFile(tempPath,
@@ -61,6 +65,11 @@ func parseGGUFModel(ctx context.Context, _ file.Resolver, _ *generic.Environment
// Extract version separately (will be set on Package.Version)
modelVersion := extractVersion(ggufFile.Header.MetadataKV)
metadataHash, err := computeKVMetadataHash(ggufFile.Header.MetadataKV)
if err != nil {
return nil, nil, err
}
// Convert to syft metadata structure
syftMetadata := &pkg.GGUFFileHeader{
Architecture: metadata.Architecture,
@@ -69,7 +78,7 @@ func parseGGUFModel(ctx context.Context, _ file.Resolver, _ *generic.Environment
GGUFVersion: uint32(ggufFile.Header.Version),
TensorCount: ggufFile.Header.TensorCount,
RemainingKeyValues: convertGGUFMetadataKVs(ggufFile.Header.MetadataKV),
MetadataKeyValuesHash: computeKVMetadataHash(ggufFile.Header.MetadataKV),
MetadataKeyValuesHash: metadataHash,
}
// If model name is not in metadata, use filename
@@ -79,6 +88,7 @@ func parseGGUFModel(ctx context.Context, _ file.Resolver, _ *generic.Environment
// Create package from metadata
p := newGGUFPackage(
ctx,
syftMetadata,
metadata.Name,
modelVersion,
@@ -86,14 +96,17 @@ func parseGGUFModel(ctx context.Context, _ file.Resolver, _ *generic.Environment
reader.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
)
return []pkg.Package{p}, nil, unknown.IfEmptyf([]pkg.Package{p}, "unable to parse GGUF file")
return []pkg.Package{p}, nil, nil
}
// computeKVMetadataHash computes a stable hash of the KV metadata for use as a global identifier
func computeKVMetadataHash(metadata gguf_parser.GGUFMetadataKVs) string {
// computeKVMetadataHash computes a stable hash of the sanitized KV metadata for use as a global identifier
func computeKVMetadataHash(metadata gguf_parser.GGUFMetadataKVs) (string, error) {
// Sort the KV pairs by key for stable hashing
sortedKVs := make([]gguf_parser.GGUFMetadataKV, len(metadata))
copy(sortedKVs, metadata)
for i, kv := range metadata {
kv.Value = sanitizeGGUFValue(kv.Value)
sortedKVs[i] = kv
}
sort.Slice(sortedKVs, func(i, j int) bool {
return sortedKVs[i].Key < sortedKVs[j].Key
})
@@ -101,13 +114,12 @@ func computeKVMetadataHash(metadata gguf_parser.GGUFMetadataKVs) string {
// Marshal sorted KVs to JSON for stable hashing
jsonBytes, err := json.Marshal(sortedKVs)
if err != nil {
log.Debugf("failed to marshal metadata for hashing: %v", err)
return ""
return "", fmt.Errorf("failed to marshal GGUF metadata for hashing: %w", err)
}
// Compute xxhash
hash := xxhash.Sum64(jsonBytes)
return fmt.Sprintf("%016x", hash) // 16 hex chars (64 bits)
return fmt.Sprintf("%016x", hash), nil // 16 hex chars (64 bits)
}
// extractVersion attempts to extract version from metadata KV pairs
+251
View File
@@ -0,0 +1,251 @@
package ai
import (
"bytes"
"context"
"encoding/binary"
"encoding/json"
"io"
"math"
"strings"
"testing"
gguf_parser "github.com/gpustack/gguf-parser-go"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal/tmpdir"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/format/syftjson"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
)
func le(v any) []byte {
buf := new(bytes.Buffer)
binary.Write(buf, binary.LittleEndian, v)
return buf.Bytes()
}
func ggufString(s string) []byte {
return append(le(uint64(len(s))), s...)
}
func parseGGUFBytes(t *testing.T, data []byte) ([]pkg.Package, error) {
t.Helper()
ctx, td := tmpdir.Root(context.Background(), "gguf-test")
t.Cleanup(func() { _ = td.Cleanup() })
reader := file.NewLocationReadCloser(file.NewLocation("/model.gguf"), io.NopCloser(bytes.NewReader(data)))
var pkgs []pkg.Package
var err error
require.NotPanics(t, func() {
pkgs, _, err = parseGGUFModel(ctx, nil, nil, reader)
})
return pkgs, err
}
func TestParseGGUFModel_malformedHeader(t *testing.T) {
// nested arrays of arrays, each one level deeper, ending in an empty uint8 array
nested := func(depth int) []byte {
v := ggufArray(ggufTypeUint8, 0, nil)
for i := 1; i < depth; i++ {
v = ggufArray(ggufTypeArray, 1, v)
}
return v
}
// a header that declares more KVs than the cap, with none following
tooManyKVs := append(append([]byte("GGUF"), le(uint32(3))...), le([]uint64{0, maxGGUFKVCount + 1})...)
tests := []struct {
name string
builder *testGGUFBuilder
data []byte // used instead of builder when set
wantErr bool
}{
{name: "too many KVs", data: tooManyKVs, wantErr: true},
{name: "split tensor count is a string", builder: newTestGGUFBuilder().withStringKV("split.tensors.count", "1"), wantErr: true},
{name: "split tensor count over the cap", builder: newTestGGUFBuilder().withUint32KV("split.tensors.count", maxGGUFSplitTensors+1), wantErr: true},
{name: "read strings too long in total", builder: newTestGGUFBuilder().withStringKV("x", strings.Repeat("a", maxGGUFReadStringBytes)), wantErr: true},
{
name: "skipped strings too many in total",
builder: newTestGGUFBuilder().withRawKV("tokenizer.ggml.tokens", ggufTypeArray, ggufArray(ggufTypeString, maxGGUFSkippedStrings+1, make([]byte, 8*(maxGGUFSkippedStrings+1)))),
wantErr: true,
},
{name: "v1 header", builder: newTestGGUFBuilder().withVersion(1).withStringKV("general.name", "m").withUint32KV("llama.context_length", 8)},
{name: "architecture not a string", builder: newTestGGUFBuilder().withUint32KV("general.architecture", 1), wantErr: true},
{name: "name not a string", builder: newTestGGUFBuilder().withUint32KV("general.name", 1), wantErr: true},
{name: "license not a string", builder: newTestGGUFBuilder().withUint32KV("general.license", 1), wantErr: true},
{name: "type not a string", builder: newTestGGUFBuilder().withUint32KV("general.type", 1), wantErr: true},
{name: "author not a string", builder: newTestGGUFBuilder().withUint32KV("general.author", 1), wantErr: true},
{name: "url not a string", builder: newTestGGUFBuilder().withUint32KV("general.url", 1), wantErr: true},
{name: "description not a string", builder: newTestGGUFBuilder().withUint32KV("general.description", 1), wantErr: true},
{name: "model hint not a string", builder: newTestGGUFBuilder().withUint32KV("controlvector.model_hint", 1), wantErr: true},
{name: "quantization version is a string", builder: newTestGGUFBuilder().withStringKV("general.quantization_version", "2"), wantErr: true},
{name: "alignment is a string", builder: newTestGGUFBuilder().withStringKV("general.alignment", "32"), wantErr: true},
{name: "alignment is uint64", builder: newTestGGUFBuilder().withUint64KV("general.alignment", 32), wantErr: true},
{name: "alignment is zero", builder: newTestGGUFBuilder().withUint32KV("general.alignment", 0), wantErr: true},
{
name: "array of arrays on a skipped key",
builder: newTestGGUFBuilder().withRawKV("tokenizer.ggml.merges", ggufTypeArray, ggufArray(ggufTypeArray, 1, ggufArray(ggufTypeUint8, 0, nil))),
wantErr: true,
},
{
name: "invalid array item type",
builder: newTestGGUFBuilder().withRawKV("tokenizer.ggml.scores", ggufTypeArray, ggufArray(99, 0, nil)),
wantErr: true,
},
{name: "split tensor count out of range", builder: newTestGGUFBuilder().withUint64KV("split.tensors.count", 1<<62), wantErr: true},
{name: "split tensor count huge", builder: newTestGGUFBuilder().withUint64KV("split.tensors.count", 1<<40), wantErr: true},
{name: "split tensor count negative", builder: newTestGGUFBuilder().withRawKV("split.tensors.count", ggufTypeInt64, le(int64(-1))), wantErr: true},
{
name: "nesting too deep on a decoded key",
builder: newTestGGUFBuilder().withRawKV("llama.attention.head_count", ggufTypeArray, nested(3)),
wantErr: true,
},
{
name: "decoded array too long",
builder: newTestGGUFBuilder().withRawKV("llama.feed_forward_length", ggufTypeArray, ggufArray(ggufTypeUint8, maxGGUFDecodedItems+1, make([]byte, maxGGUFDecodedItems+1))),
wantErr: true,
},
{name: "empty string value", builder: newTestGGUFBuilder().withStringKV("general.author", ""), wantErr: true},
{name: "padded key", builder: newTestGGUFBuilder().withUint32KV(" general.alignment\t", 0), wantErr: true},
{name: "long padded key", builder: newTestGGUFBuilder().withUint32KV(strings.Repeat(" ", 300)+"general.alignment", 0), wantErr: true},
{name: "file type is a string", builder: newTestGGUFBuilder().withUint32KV("general.file_type", 1).withStringKV("general.file_type", "x"), wantErr: true},
{
name: "decoded arrays too long in total",
builder: newTestGGUFBuilder().
withRawKV("a.feed_forward_length", ggufTypeArray, ggufArray(ggufTypeUint8, maxGGUFDecodedItems, make([]byte, maxGGUFDecodedItems))).
withRawKV("b.feed_forward_length", ggufTypeArray, ggufArray(ggufTypeUint8, 1, make([]byte, 1))),
wantErr: true,
},
{name: "invalid value type", builder: newTestGGUFBuilder().withRawKV("x", 0xFFFFFFFF, nil), wantErr: true},
{name: "empty key", builder: newTestGGUFBuilder().withUint32KV("", 1), wantErr: true},
{
name: "empty string in a decoded array",
builder: newTestGGUFBuilder().withRawKV("llama.attention.head_count", ggufTypeArray, ggufArray(ggufTypeString, 1, le(uint64(0)))),
wantErr: true,
},
{
name: "empty string in a skipped array",
builder: newTestGGUFBuilder().withRawKV("tokenizer.ggml.tokens", ggufTypeArray, ggufArray(ggufTypeString, 1, le(uint64(0)))),
},
{
name: "array of arrays on a decoded key",
builder: newTestGGUFBuilder().withRawKV("llama.attention.head_count", ggufTypeArray, nested(2)),
},
{name: "alignment is a uint32", builder: newTestGGUFBuilder().withUint32KV("general.alignment", 32)},
{
name: "split keys as written by llama.cpp",
builder: newTestGGUFBuilder().
withRawKV("split.no", ggufTypeUint16, le(uint16(0))).
withRawKV("split.count", ggufTypeUint16, le(uint16(3))).
withRawKV("split.tensors.count", ggufTypeInt32, le(int32(3000))),
},
{
name: "split shard declares the total tensor count",
builder: newTestGGUFBuilder().withRawKV("split.tensors.count", ggufTypeInt32, le(int32(3000))),
},
{
name: "skipped scalar array",
builder: newTestGGUFBuilder().withRawKV("tokenizer.ggml.scores", ggufTypeArray, ggufArray(ggufTypeFloat32, 3, make([]byte, 12))),
},
// ponytail: zero block size tensor types and dimension overflow panic inside tensor info parsing;
// those are left to the generic cataloger's panic handling rather than checked here.
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
data := tt.data
if data == nil {
data = tt.builder.withStringKV("general.architecture", "llama").build()
}
pkgs, err := parseGGUFBytes(t, data)
if tt.wantErr {
// the pre-scan has to be what rejects these, before the library sees them
require.ErrorContains(t, err, "invalid GGUF header")
assert.Empty(t, pkgs)
return
}
require.NoError(t, err)
require.Len(t, pkgs, 1)
})
}
}
func TestParseGGUFModel_nonFiniteFloat(t *testing.T) {
data := newTestGGUFBuilder().
withStringKV("general.architecture", "llama").
withRawKV("llama.rope.freq_base", ggufTypeFloat32, le(float32(math.Inf(1)))).
withRawKV("llama.rope.scale", ggufTypeFloat64, le(math.Inf(-1))).
withRawKV("llama.attention.head_count", ggufTypeArray, ggufArray(ggufTypeFloat32, 2, le([]float32{8, float32(math.NaN())}))).
withRawKV("tokenizer.ggml.scores", ggufTypeArray, ggufArray(ggufTypeFloat32, 1, le(float32(0)))).
build()
pkgs, err := parseGGUFBytes(t, data)
require.NoError(t, err)
require.Len(t, pkgs, 1)
meta := pkgs[0].Metadata.(pkg.GGUFFileHeader)
assert.NotEmpty(t, meta.MetadataKeyValuesHash)
assert.Equal(t, "+Inf", meta.RemainingKeyValues["llama.rope.freq_base"])
assert.Equal(t, "-Inf", meta.RemainingKeyValues["llama.rope.scale"])
headCount := meta.RemainingKeyValues["llama.attention.head_count"].(gguf_parser.GGUFMetadataKVArrayValue)
assert.Equal(t, []any{float32(8), "NaN"}, headCount.Array)
var buf bytes.Buffer
s := sbom.SBOM{Artifacts: sbom.Artifacts{Packages: pkg.NewCollection(pkgs...)}}
require.NoError(t, syftjson.NewFormatEncoder().Encode(&buf, s))
assert.NotEmpty(t, buf.Bytes())
}
// TestParseGGUFModel_outputUnchanged pins the header output and hash for ordinary values to what syft
// produced as of f7f3c6efc, before malformed header handling. The expected values were captured by
// running this same fixture on that commit. float32 0.1 catches any widening to float64.
func TestParseGGUFModel_outputUnchanged(t *testing.T) {
data := newTestGGUFBuilder().
withStringKV("general.architecture", "llama").
withRawKV("tokenizer.ggml.tokens", ggufTypeArray, ggufArray(ggufTypeString, 2, append(ggufString("a"), ggufString("b")...))).
withRawKV("llama.attention.head_count", ggufTypeArray, ggufArray(ggufTypeFloat32, 2, le([]float32{0.1, 0.2}))).
withRawKV("llama.feed_forward_length", ggufTypeArray, ggufArray(ggufTypeArray, 2, append(
ggufArray(ggufTypeUint32, 2, le([]uint32{1, 2})),
ggufArray(ggufTypeUint32, 1, le([]uint32{3}))...))).
withRawKV("llama.rope.freq_base", ggufTypeFloat32, le(float32(0.1))).
build()
pkgs, err := parseGGUFBytes(t, data)
require.NoError(t, err)
require.Len(t, pkgs, 1)
meta := pkgs[0].Metadata.(pkg.GGUFFileHeader)
assert.Equal(t, "4f28d55f0985a985", meta.MetadataKeyValuesHash)
header, err := json.Marshal(meta.RemainingKeyValues)
require.NoError(t, err)
assert.JSONEq(t, `{
"llama.attention.head_count": {"type": 6, "len": 2, "array": [0.1, 0.2], "startOffset": 170, "size": 8},
"llama.feed_forward_length": {"type": 9, "len": 2, "array": [
{"type": 4, "len": 2, "array": [1, 2], "startOffset": 239, "size": 8},
{"type": 4, "len": 1, "array": [3], "startOffset": 259, "size": 4}
], "startOffset": 227, "size": 36},
"llama.rope.freq_base": 0.1,
"tokenizer.ggml.tokens": {"type": 8, "len": 2, "startOffset": 102, "size": 18}
}`, string(header))
}
// TestCopyHeader_appliesItsOwnBound pins that copyHeader caps the copy at maxHeaderSize itself,
// whatever reader it is handed.
//
// Bytes retained is asserted directly rather than through testutils.MeasureAlloc: the destination is a
// bytes.Buffer, whose doubling growth costs a multiple of what it holds and lands on a different power
// of two depending on the platform. buf.Len() is the same property without that noise.
func TestCopyHeader_appliesItsOwnBound(t *testing.T) {
const payload = 2*maxHeaderSize + 1
file := make([]byte, 24, 24+payload)
binary.LittleEndian.PutUint32(file[0:4], ggufMagicNumber)
file = append(file, bytes.Repeat([]byte{'x'}, payload)...)
var buf bytes.Buffer
require.NoError(t, copyHeader(&buf, bytes.NewReader(file)))
assert.Equal(t, maxHeaderSize, buf.Len(), "copyHeader must cap the copy at maxHeaderSize")
}
+52 -32
View File
@@ -5,6 +5,8 @@ import (
"encoding/json"
"fmt"
"io"
"math"
"math/bits"
"sort"
"strings"
@@ -15,9 +17,7 @@ import (
// SafeTensors file format: [8 bytes u64 LE header size] [N bytes JSON header] [tensor data].
// Reference: https://github.com/huggingface/safetensors#format
const (
maxSafeTensorsHeaderSize = 100 * 1024 * 1024 // 100MB ceiling on header JSON to prevent OOM
)
// The header JSON is capped at maxSafeTensorsHeaderSize (see limits.go).
// safeTensorsHeader is the decoded JSON header. Tensor entries live alongside a
// reserved "__metadata__" key holding a string-to-string producer map. We decode
@@ -84,47 +84,67 @@ func readSafeTensorsHeader(r io.Reader) (*safeTensorsHeader, error) {
return h, nil
}
// parameterCount sums the element counts across all tensors in the header.
func (h *safeTensorsHeader) parameterCount() uint64 {
var total uint64
for _, t := range h.tensors {
count := uint64(1)
for _, dim := range t.Shape {
if dim <= 0 {
count = 0
break
}
count *= uint64(dim)
}
total += count
// modelInfo returns the metadata derivable from the header bytes alone. Naming,
// licenses and ShardCount are left to the merge processor.
func (h *safeTensorsHeader) modelInfo() pkg.SafeTensorsModelInfo {
params, dtype := h.parameterStats()
return pkg.SafeTensorsModelInfo{
Format: "safetensors",
TensorCount: uint64(len(h.tensors)),
Parameters: params,
Quantization: normalizeDType(dtype),
UserMetadata: userMetadataKeyValues(h.metadata),
MetadataHash: h.metadataHash(),
}
return total
}
// dominantDType returns the dtype that accounts for the largest fraction of parameters.
// For mixed-precision models the "dominant" dtype is still a useful summary.
func (h *safeTensorsHeader) dominantDType() string {
// parameterStats sums the element counts across all tensors and returns the
// dtype that accounts for the largest share of them. For mixed-precision models
// the "dominant" dtype is still a useful summary. Totals saturate rather than wrap.
func (h *safeTensorsHeader) parameterStats() (total uint64, dominantDType string) {
sizeByDType := make(map[string]uint64)
for _, t := range h.tensors {
count := uint64(1)
for _, dim := range t.Shape {
if dim <= 0 {
count = 0
break
}
count *= uint64(dim)
count, ok := tensorElements(t.Shape)
if !ok {
continue
}
sizeByDType[t.DType] += count
total = saturatingAdd(total, count)
sizeByDType[t.DType] = saturatingAdd(sizeByDType[t.DType], count)
}
var best string
var bestSize uint64
for dtype, size := range sizeByDType {
if size > bestSize || (size == bestSize && dtype < best) {
best = dtype
if size > bestSize || (size == bestSize && dtype < dominantDType) {
dominantDType = dtype
bestSize = size
}
}
return best
return total, dominantDType
}
// tensorElements returns the element count for a shape, or ok=false on a
// non-positive dim or when the product overflows uint64.
func tensorElements(shape []int64) (uint64, bool) {
n := uint64(1)
for _, d := range shape {
if d <= 0 {
return 0, false
}
hi, lo := bits.Mul64(n, uint64(d))
if hi != 0 {
return 0, false
}
n = lo
}
return n, true
}
// saturatingAdd returns a+b, clamped to math.MaxUint64 instead of wrapping.
func saturatingAdd(a, b uint64) uint64 {
sum, carry := bits.Add64(a, b, 0)
if carry != 0 {
return math.MaxUint64
}
return sum
}
// metadataHash returns a stable xxhash64 over the logical tensor content
@@ -6,7 +6,6 @@ import (
"io"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/unknown"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
@@ -26,22 +25,13 @@ func parseSafeTensorsFile(_ context.Context, _ file.Resolver, _ *generic.Environ
return nil, nil, fmt.Errorf("failed to read safetensors header: %w", err)
}
// ShardCount is intentionally not set here: the merge processor is the single
// owner of ShardCount and derives it from the number of shards in the group.
md := pkg.SafeTensorsModelInfo{
Format: "safetensors",
TensorCount: uint64(len(header.tensors)),
Parameters: header.parameterCount(),
Quantization: normalizeDType(header.dominantDType()),
UserMetadata: userMetadataKeyValues(header.metadata),
MetadataHash: header.metadataHash(),
}
md := header.modelInfo()
p := newSafeTensorsPackage(
&md,
reader.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
)
return []pkg.Package{p}, nil, unknown.IfEmptyf([]pkg.Package{p}, "unable to parse safetensors file")
return []pkg.Package{p}, nil, nil
}
// integrity check
+3 -11
View File
@@ -8,7 +8,6 @@ import (
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/unknown"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
@@ -50,7 +49,7 @@ type dockerAIModelConfig struct {
func parseSafeTensorsOCIConfig(_ context.Context, _ file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
defer internal.CloseAndLogError(reader, reader.Path())
body, err := io.ReadAll(io.LimitReader(reader, 1024*1024))
body, err := readBounded(reader, maxModelConfigBlobSize)
if err != nil {
return nil, nil, fmt.Errorf("failed to read docker AI model config: %w", err)
}
@@ -81,7 +80,7 @@ func parseSafeTensorsOCIConfig(_ context.Context, _ file.Resolver, _ *generic.En
&md,
reader.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
)
return []pkg.Package{p}, nil, unknown.IfEmptyf([]pkg.Package{p}, "unable to parse docker AI safetensors config")
return []pkg.Package{p}, nil, nil
}
// parseSafeTensorsOCILayer decodes the JSON header of a SafeTensors weight
@@ -94,14 +93,7 @@ func parseSafeTensorsOCILayer(_ context.Context, _ file.Resolver, _ *generic.Env
return nil, nil, fmt.Errorf("failed to read safetensors layer header: %w", err)
}
md := pkg.SafeTensorsModelInfo{
Format: "safetensors",
TensorCount: uint64(len(header.tensors)),
Parameters: header.parameterCount(),
Quantization: normalizeDType(header.dominantDType()),
UserMetadata: userMetadataKeyValues(header.metadata),
MetadataHash: header.metadataHash(),
}
md := header.modelInfo()
p := newSafeTensorsPackage(
&md,
+301 -16
View File
@@ -6,13 +6,17 @@ import (
"encoding/binary"
"encoding/json"
"io"
"math"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal/testutils"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
@@ -47,7 +51,7 @@ func buildSafeTensorsFile(t *testing.T, metadata map[string]string, tensors map[
//
// The naming precedence (owned by the merge processor's pickSafeTensorsName) is:
// 1. config.json _name_or_path (path.Base applied), found beside the model or
// by walking up parent directories to the scan root
// in its parent directory
// 2. otherwise the model's immediate parent directory base name
// → drop (no package) when neither yields a usable name
//
@@ -230,9 +234,9 @@ func TestSafeTensorsCataloger(t *testing.T) {
expectedPackages: nil,
},
{
// rung 1 via parent-walk: findDirHFConfig walks up from the model
// directory, so a config.json in an ancestor names a nested model.
name: "config.json in an ancestor directory names a nested model",
// rung 1 via the parent: findDirHFConfig also checks the model
// directory's parent, so a config.json there names a nested model.
name: "config.json in the parent directory names a nested model",
setup: func(t *testing.T) string {
dir := t.TempDir()
model(t, filepath.Join(dir, "dir", "someothertensor"))
@@ -247,6 +251,87 @@ func TestSafeTensorsCataloger(t *testing.T) {
},
},
},
{
// the walk stops at the parent: a config.json two levels up is not used.
name: "config.json two levels up is not used",
setup: func(t *testing.T) string {
dir := t.TempDir()
model(t, filepath.Join(dir, "a", "b", "someothertensor"))
writeFile(t, filepath.Join(dir, "a", "config.json"), `{"_name_or_path":"org/TooFar"}`)
return dir
},
expectedPackages: []pkg.Package{
{
Name: "someothertensor",
Type: pkg.ModelPkg,
Metadata: wantMetadata(""),
},
},
},
{
// JSON without any HF field is not treated as a model config, so the
// lookup keeps going to the parent.
name: "non-HF config.json is skipped",
setup: func(t *testing.T) string {
dir := t.TempDir()
modelDir := filepath.Join(dir, "app", "sometensor")
model(t, modelDir)
writeFile(t, filepath.Join(modelDir, "config.json"), `{"port":8080}`)
writeFile(t, filepath.Join(dir, "app", "config.json"), `{"_name_or_path":"org/Parent"}`)
return dir
},
expectedPackages: []pkg.Package{
{
Name: "Parent",
Type: pkg.ModelPkg,
Metadata: wantMetadata(""),
},
},
},
{
// an oversized config.json is skipped entirely, while a large README
// still yields its frontmatter (only a prefix is read).
name: "oversized config.json is ignored and a large README still names the model",
setup: func(t *testing.T) string {
dir := t.TempDir()
modelDir := filepath.Join(dir, "sometensor")
model(t, modelDir)
writeFile(t, filepath.Join(modelDir, "config.json"),
`{"architectures":["X"],"_name_or_path":"`+strings.Repeat("A", maxHFConfigSize)+`"}`)
writeFile(t, filepath.Join(modelDir, "README.md"),
"---\nlicense: MIT\nbase_model: org/Readme\n---\n"+strings.Repeat("A", maxReadmePrefixSize))
return dir
},
expectedPackages: []pkg.Package{
{
Name: "Readme",
Type: pkg.ModelPkg,
Licenses: pkg.NewLicenseSet(
pkg.NewLicenseFromFields("MIT", "", nil),
),
Metadata: wantMetadata(""),
},
},
},
{
// a _name_or_path that sanitizes to nothing doesn't block README base_model.
name: "degenerate _name_or_path falls through to README base_model",
setup: func(t *testing.T) string {
dir := t.TempDir()
modelDir := filepath.Join(dir, "sometensor")
model(t, modelDir)
writeFile(t, filepath.Join(modelDir, "config.json"), `{"_name_or_path":"."}`)
writeFile(t, filepath.Join(modelDir, "README.md"), "---\nbase_model: org/Readme\n---\n")
return dir
},
expectedPackages: []pkg.Package{
{
Name: "Readme",
Type: pkg.ModelPkg,
Metadata: wantMetadata(""),
},
},
},
{
// grouping: independent models in one scan are grouped by their own
// parent directory and each named from it.
@@ -600,6 +685,60 @@ spdx-id: Apache-2.0
assertHasLicense(t, out[0], "Apache-2.0")
})
t.Run("OCI: config precedence is stable across layer order", func(t *testing.T) {
dir := t.TempDir()
a := filepath.Join(dir, "a.json")
b := filepath.Join(dir, "b.json")
gen := filepath.Join(dir, "generation_config.json")
require.NoError(t, os.WriteFile(a, []byte(`{"_name_or_path":"org/first"}`), 0o644))
require.NoError(t, os.WriteFile(b, []byte(`{"_name_or_path":"org/second"}`), 0o644))
require.NoError(t, os.WriteFile(gen, []byte(`{"max_new_tokens":10}`), 0o644))
configMd := pkg.SafeTensorsModelInfo{Format: "safetensors", TensorCount: 1}
for _, order := range [][]string{{a, b, gen}, {gen, b, a}} {
var locs []file.Location
for _, p := range order {
locs = append(locs, file.NewLocation(p))
}
resolver := file.NewMockResolverForMediaTypes(map[string][]file.Location{
dockerAIModelFileMediaType: locs,
})
out, _, err := safeTensorsMergeProcessor(context.Background(), resolver, []pkg.Package{ociPkg(configMd)}, nil, nil)
require.NoError(t, err)
require.Len(t, out, 1)
assert.Equal(t, "first", out[0].Name)
// the non-HF generation config is not recorded as supporting evidence
assert.Len(t, out[0].Locations.ToSlice(), 3)
}
})
t.Run("dir scan: non-HF config.json in the parent is not supporting evidence", func(t *testing.T) {
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.json")
require.NoError(t, os.WriteFile(cfgPath, []byte(`{"port":8080}`), 0o644))
loc, cfg := findDirHFConfig(file.NewMockResolverForPaths(cfgPath), filepath.Join(dir, "model"))
assert.Nil(t, loc)
assert.Nil(t, cfg)
})
t.Run("OCI: a large license layer still yields its frontmatter spdx-id", func(t *testing.T) {
dir := t.TempDir()
licensePath := filepath.Join(dir, "LICENSE")
require.NoError(t, os.WriteFile(licensePath,
[]byte("---\nspdx-id: Apache-2.0\n---\n"+strings.Repeat("A", maxLicensePrefixSize)), 0o644))
assert.Equal(t, "Apache-2.0", readLicenseSPDXIDFromFrontmatter(file.NewMockResolverForPaths(licensePath), file.NewLocation(licensePath)))
})
t.Run("OCI: oversized model.file layer is rejected", func(t *testing.T) {
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.json")
require.NoError(t, os.WriteFile(cfgPath,
[]byte(`{"_name_or_path":"org/x","pad":"`+strings.Repeat("A", maxModelFileLayerSize)+`"}`), 0o644))
cfg, fm := classifyOCIModelFileLayer(file.NewMockResolverForPaths(cfgPath), file.NewLocation(cfgPath))
assert.Nil(t, cfg)
assert.Nil(t, fm)
})
t.Run("names packages and still propagates an upstream error", func(t *testing.T) {
// A non-nil upstream error (e.g. another file in the run failed to parse)
// must NOT short-circuit naming. The package is still nameless at this
@@ -621,7 +760,7 @@ spdx-id: Apache-2.0
// (or neither, asserting the drop path).
//
// Precedence (highest → lowest):
// 1. config.json _name_or_path (path.Base applied; both dir-scan and OCI)
// 1. config.json _name_or_path (sanitized to its last path element; both dir-scan and OCI)
// 2. fallback name — OCI image-ref last segment, or dir-scan parent directory
// base name (the merge processor computes the right one per group)
// → drop (empty name) when nothing matches
@@ -662,6 +801,31 @@ func TestSafeTensorsNamingPrecedence(t *testing.T) {
want: "tiny-llama",
},
// sanitizing
{
name: "windows-style path uses the last element",
nameOrPath: `C:\models\win-model`,
fallbackName: "fallback-name",
want: "win-model",
},
{
name: "degenerate path falls back",
nameOrPath: "..",
fallbackName: "fallback-name",
want: "fallback-name",
},
{
name: "root path falls back",
nameOrPath: "/",
fallbackName: "fallback-name",
want: "fallback-name",
},
{
name: "long names are truncated",
nameOrPath: "org/" + strings.Repeat("x", maxModelNameLength+10),
want: strings.Repeat("x", maxModelNameLength),
},
// drops
{
name: "drop: both rungs empty",
@@ -983,8 +1147,9 @@ func TestSafeTensorsHeader_parameterCountAndDType(t *testing.T) {
"small": {DType: "F32", Shape: []int64{16, 16}},
"scalar": {DType: "F32", Shape: []int64{}}, // empty shape contributes 1
}}
assert.Equal(t, uint64(1000*16+16*16+1), h.parameterCount())
assert.Equal(t, "BF16", h.dominantDType())
params, dtype := h.parameterStats()
assert.Equal(t, uint64(1000*16+16*16+1), params)
assert.Equal(t, "BF16", dtype)
}
func TestSafeTensorsHeader_parameterCount_nonPositiveDims(t *testing.T) {
@@ -996,7 +1161,29 @@ func TestSafeTensorsHeader_parameterCount_nonPositiveDims(t *testing.T) {
"zero": {DType: "F32", Shape: []int64{4, 0}},
"negative": {DType: "F32", Shape: []int64{-1, 8}},
}}
assert.Equal(t, uint64(16), h.parameterCount())
params, _ := h.parameterStats()
assert.Equal(t, uint64(16), params)
}
func TestSafeTensorsHeader_parameterStats_overflow(t *testing.T) {
// a shape whose product overflows uint64 is skipped instead of wrapping (the
// first tensor would wrap to 0) and can't flip the dominant dtype; the total
// saturates instead of wrapping.
h := &safeTensorsHeader{tensors: map[string]safeTensorsEntry{
"good": {DType: "BF16", Shape: []int64{4, 4}},
"wraps": {DType: "F32", Shape: []int64{4294967296, 4294967296}},
"overflow": {DType: "F32", Shape: []int64{math.MaxInt64, 3}},
}}
params, dtype := h.parameterStats()
assert.Equal(t, uint64(16), params)
assert.Equal(t, "BF16", dtype)
h = &safeTensorsHeader{tensors: map[string]safeTensorsEntry{
"a": {DType: "F32", Shape: []int64{math.MaxInt64, 2}},
"b": {DType: "F32", Shape: []int64{math.MaxInt64, 2}},
}}
params, _ = h.parameterStats()
assert.Equal(t, uint64(math.MaxUint64), params)
}
func TestSafeTensorsHeader_dominantDType_tieBreak(t *testing.T) {
@@ -1006,7 +1193,8 @@ func TestSafeTensorsHeader_dominantDType_tieBreak(t *testing.T) {
"a": {DType: "F32", Shape: []int64{4, 4}},
"b": {DType: "BF16", Shape: []int64{4, 4}},
}}
assert.Equal(t, "BF16", h.dominantDType())
_, dtype := h.parameterStats()
assert.Equal(t, "BF16", dtype)
}
func TestNormalizeDType(t *testing.T) {
@@ -1035,25 +1223,78 @@ func TestParseFrontmatter(t *testing.T) {
name string
input string
wantNil bool
wantLicense string
wantLicenses []string
wantBaseModel []string
}{
{
name: "list base_model",
input: "---\nlicense: mit\nbase_model:\n - org/Model\n---\nbody",
wantLicense: "mit",
wantLicenses: []string{"mit"},
wantBaseModel: []string{"org/Model"},
},
{
name: "scalar base_model",
input: "---\nlicense: apache-2.0\nbase_model: org/Model\n---\n",
wantLicense: "apache-2.0",
wantLicenses: []string{"apache-2.0"},
wantBaseModel: []string{"org/Model"},
},
{
name: "leading BOM",
input: "\xef\xbb\xbf---\nlicense: mit\n---\n",
wantLicense: "mit",
name: "list license keeps base_model",
input: "---\nlicense: [mit, apache-2.0]\nbase_model: org/Model\n---\n",
wantLicenses: []string{"mit", "apache-2.0"},
wantBaseModel: []string{"org/Model"},
},
{
name: "mapping-valued fields are ignored, not fatal",
input: "---\nlicense: {name: mit}\nbase_model: org/Model\n---\n",
wantBaseModel: []string{"org/Model"},
},
{
name: "leading BOM",
input: "\xef\xbb\xbf---\nlicense: mit\n---\n",
wantLicenses: []string{"mit"},
},
{
name: "null license is ignored",
input: "---\nlicense: null\nbase_model: ~\n---\n",
wantLicenses: nil,
wantBaseModel: nil,
},
{
name: "empty block",
input: "---\n---\n",
wantNil: true,
},
{
name: "oversized block is rejected",
input: "---\nlicense: mit\n" + strings.Repeat("# pad\n", maxFrontmatterSize/6+1) + "---\n",
wantNil: true,
},
{
name: "closing delimiter at EOF with trailing whitespace",
input: "---\nlicense: mit\n--- ",
wantLicenses: []string{"mit"},
},
{
name: "list values are capped",
input: "---\nlicense: [" + strings.Repeat("mit,", maxFrontmatterValues+5) + "mit]\n---\n",
wantLicenses: slices.Repeat([]string{"mit"}, maxFrontmatterValues),
},
{
name: "CRLF delimiters",
input: "---\r\nlicense: mit\r\n---\r\nbody",
wantLicenses: []string{"mit"},
},
{
name: "opening delimiter must be exactly ---",
input: "----\nlicense: mit\n---\n",
wantNil: true,
},
{
name: "a ---x line inside the block does not close it",
input: "---\nbase_model: org/Model\n---x: 1\nlicense: mit\n---\n",
wantLicenses: []string{"mit"},
wantBaseModel: []string{"org/Model"},
},
{
name: "no frontmatter",
@@ -1065,6 +1306,12 @@ func TestParseFrontmatter(t *testing.T) {
input: "---\nlicense: mit\n",
wantNil: true,
},
{
// https://github.com/anchore/syft/issues/5327
name: "malformed frontmatter that panics the yaml decoder",
input: "---\n<<:\r? 0:\n---\nbody",
wantNil: true,
},
}
for _, tt := range tests {
@@ -1075,7 +1322,7 @@ func TestParseFrontmatter(t *testing.T) {
return
}
require.NotNil(t, fm)
assert.Equal(t, tt.wantLicense, fm.License)
assert.Equal(t, tt.wantLicenses, fm.Licenses)
if tt.wantBaseModel != nil {
assert.Equal(t, tt.wantBaseModel, fm.BaseModel)
}
@@ -1156,6 +1403,12 @@ limitations:
input: "---\nspdx-id: MIT\n(never closes)\n",
want: "",
},
{
// https://github.com/anchore/syft/issues/5327
name: "malformed frontmatter that panics the yaml decoder",
input: "---\n<<:\r? 0:\n---\nbody\n",
want: "",
},
}
for _, tt := range tests {
@@ -1200,3 +1453,35 @@ func TestDockerAIModelConfigMediaTypes(t *testing.T) {
})
}
}
// TestReadSafeTensorsHeader_capBoundsHeap pins maxSafeTensorsHeaderSize as bytes rather than as an
// error string. The existing over-cap case omits the body precisely because the guard rejects before
// allocating, which means it would keep passing if the guard moved after the read. This one delivers
// the bytes, so moving the check shows up as heap.
func TestReadSafeTensorsHeader_capBoundsHeap(t *testing.T) {
const declared = maxSafeTensorsHeaderSize + 1
body := bytes.Repeat([]byte{'x'}, declared)
file := make([]byte, 8, 8+len(body))
binary.LittleEndian.PutUint64(file, uint64(declared))
file = append(file, body...)
// the unguarded shape: honor the declared length without weighing it against a ceiling first.
// This is what readSafeTensorsHeader does once the cap is passed, so it is what the cap prevents.
unbounded := testutils.MeasureAlloc(t, func() {
b, err := io.ReadAll(io.LimitReader(bytes.NewReader(file[8:]), declared))
require.NoError(t, err)
require.Len(t, b, declared)
})
require.Greater(t, unbounded, uint64(declared),
"fixture did not actually cost more than its own size; it is no longer a bomb")
bounded := testutils.MeasureAlloc(t, func() {
_, err := readSafeTensorsHeader(bytes.NewReader(file))
require.ErrorContains(t, err, "exceeds maximum")
})
t.Logf("unbounded allocated %d bytes, bounded allocated %d bytes", unbounded, bounded)
assert.Less(t, bounded, uint64(8*1024*1024),
"the declared-size check has to reject before the body is read, not after")
}
+2 -2
View File
@@ -104,10 +104,10 @@ func mergeDirModels(ctx context.Context, resolver file.Resolver, pkgs []pkg.Pack
// groupByParentDir buckets filesystem-scanned models by the directory their
// primary-evidence file lives in.
//
// This encodes a deliberate assumption: a directory holds one logical
// model, so every .safetensors file in a directory is treated as a shard of the
// same modeland merged into one package.
// same model and merged into one package.
// The trade-off is that if a directory happens to contain several unrelated models,
// they are merged into one package rather than reported separately.
// We accept that because the conventional on-disk layout gives each model

Some files were not shown because too many files have changed in this diff Show More