* chore: centralize temp files and prefer streaming IO
Catalogers that create temp files ad-hoc can easily forget cleanup,
leaking files on disk. Similarly, io.ReadAll is convenient but risks
OOM on large or malicious inputs.
Introduce internal/tmpdir to manage all cataloger temp storage under
a single root directory with automatic cleanup. Prefer streaming
parsers (bufio.Scanner, json/yaml.NewDecoder, io.LimitReader) over
buffering entire inputs into memory. Add ruleguard rules to enforce
both practices going forward.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: go back to old release parsing
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* simplify to limit reader in version check
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: regex change postponed
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* simplify supplement release to limitreader
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
---------
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
BUILDPLATFORM is automatically set to the host's platform in new Docker,
so having it defined as an arg results in it being overridden by this
automatic value. Since it was always assigned to a literal string in the
test files, just use that string.
Additionally, image platform is better pulled from the manifest, not the
image config, in containerd store, so try that first.
Additionally, python3 is on PATH on new macs by default, but not python.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore(deps): update tools to latest versions
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* chore(lint): fix errors in new golangci-lint
Two fixes:
First, replace sb.WriteString(fmt.Sprintf(...)) with fmt.Fprintf(&sb, ...)
Second, suppress errors where we read from the local file system at a
user provided path. This is a CLI tool, and reads from user provided
paths on the local file system by design.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Co-authored-by: spiffcs <32073428+spiffcs@users.noreply.github.com>
Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* migrate fixtures to testdata
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: correct broken symlinks after testdata migration
The migration from test-fixtures to testdata broke several symlinks:
- elf-test-fixtures symlinks pointed to old test-fixtures paths
- elf-test-fixtures needed to be renamed to elf-testdata
- image-pkg-coverage symlink pointed to test-fixtures instead of testdata
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: handle missing classifiers/bin directory in Makefile
The clean-fingerprint target was failing when classifiers/bin doesn't
exist (e.g., on fresh clone without downloaded binaries).
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add gitignore negation for jar/zip fixtures in test/cli
The jar and zip files in test/cli/testdata/image-unknowns were being
gitignored by the root .gitignore patterns. This caused them to be
untracked and not included when building docker images in CI, resulting
in Test_Unknowns failures since the test expects errors from corrupt
archive files that weren't present.
Add a .gitignore in test/cli/testdata to negate the exclusions for
these specific test fixture files.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* switch fixture cache to v2
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: update expected versions for rebuilt fixtures
Update test expectations for packages that have been updated in
upstream repositories when docker images are rebuilt:
- glibc: 2.42-r4 → 2.43-r1 (wolfi)
- php: 8.2.29 → 8.2.30 (ubuntu/apache)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* upgrade go
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add go-shlex dependency for testdata manager tool
The manager tool in syft/pkg/cataloger/binary/testdata/ imports
go-shlex, but since it's in a testdata directory, Go doesn't track
its dependencies. This caused CI failures when go.mod didn't
explicitly list the dependency.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* refactor: move binary classifier manager to internal/
Move the manager tool from testdata/manager to internal/manager so
that Go properly tracks its dependencies. Code in testdata directories
is ignored by Go for dependency tracking, which caused CI failures
when go.mod didn't explicitly list transitive dependencies.
This is a cleaner solution than manually adding dependencies to go.mod
for code that happens to live in testdata.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add gitignore negations for test fixtures blocked by root patterns
Multiple test fixtures were being blocked by root-level gitignore patterns
like bin/, *.jar, *.tar, and *.exe. This adds targeted .gitignore files with
negation patterns to allow these specific test fixtures to be tracked:
- syft/linux/testdata/os/busybox/bin/busybox (blocked by bin/)
- syft/pkg/cataloger/java/testdata/corrupt/example.{jar,tar} (blocked by *.jar, *.tar)
- syft/pkg/cataloger/binary/testdata/classifiers/snippets/go-version-hint/**/bin/go (blocked by bin/)
- syft/pkg/cataloger/bitnami/testdata/no-rel/.../bin/redis-server (blocked by bin/)
Also updates the bitnami test expectation to include the newly required
.gitignore files in the test fixture.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: update glibc version expectation (2.43-r1 -> 2.43-r2)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add capability drift check as unit step
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* dont clear test observations before drift detection
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* bump stereoscope commit to main
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@surgut.co.uk>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
* fix(deb and snaps): prevent excess reads
Previously, Syft could allocate excess memory or tempfile space if there
were highly compressed objects in deb archives, or at paths where the
kernel changelog was expected by the snap cataloger. Use io.LimitReaders
for extracting parts of deb archives, and refactor the snap cataloger's
reading of the kernel changelog to use a streaming parsing, eliminating
the possibility of excess allocation.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* fix: always cleanup temp file from file source
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* use streaming strategy for deb archives
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
---------
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(cpe-generation): set start and end date
Previously, the update job was silently failing because the NVD API
returns a 404 with no body if a start date is specified but not an end
date. Further, the API returns an error if more than 120 days are in
range of the start and end date.
Update the API client to:
1. Return a non-nil error on http 404
2. Chunk the date range into 120 day chunks
3. Pass start and end date to avoid errors.
Also add more tolerant timestamp parsing since the previous update job
would fail with timestamp format errors.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* refactor(cpe-generator): remove callbacks
Previously, this job had callbacks that were there to make sure that
incremental progress could be written to disk. However, incremental
progress was not being written to disk, and there were issues related to
the callbacks like double logging. Therefore, just remove the callbacks
and do simple imperative code to page through the API results.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
---------
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
NVD uses "aomedia" as the vendor for libavif CVEs. This change adds
libavif to the APK package CPE candidate additions with "aomedia" as
an additional vendor, enabling Syft/Grype to match CVEs like
CVE-2025-48174 and CVE-2025-48175.
Signed-off-by: Peter Bücker <peter.buecker@gmail.com>
Signed-off-by: Alan Pope <alan@popey.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Co-authored-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* add info command from generated capabilities
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* correct gentoo and arch ecosystems
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* rename os pkg types
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* better binary cataloger description
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* expose metadata and pacakge types in json
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* expose json schema types
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add completeness tests for metadata types
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* latest generation
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix linting
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* improve testing a docs
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix tests and linting
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* restore goreleaser config
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* tweak diagram
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix pdm
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore: java binary data
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* new capability descriptions for gguf and python
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* correct poetry lock integrity hash claim
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix compile error
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: remove purl version from overrides
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* fix lua deps ref
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* keep gguf as ai ecosystem
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* split packages.yaml to multiple files by go package
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ensure tests do not use go test cache
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* sort json output for info command
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* docs: fix ocaml, php, and portage capabilities yaml
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: update erlang capabilities
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: update java capabilities
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: update javascript capabilities
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: update linux kernel capabilities
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* remove missing tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix package.yaml references
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* revert license list change
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* check for drift in capability descriptions
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* regenerate capabilities
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test cleanup
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* use fixture cache in static analysis
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* claim fixtures pre-req for cap generation
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* update documentation with correct regeneration procedure
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore: ruby-gemspec-cataloger finds no dependencies
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: fix python docs and config comment
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: commit re-generated java yaml
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* add cataloger selection to caps command
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* re-generate cap yamls
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix tests for cataloger selection
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix cli test
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add missing tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix linting
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* rename cmd to `cataloger info`
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* [wip] change capability description locations
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* [wip] continued
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* [wip] adjust for import cycles
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* correct docs
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix linting
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* decompress upx packed binaries
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix linting and remove dead code
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* syft detect embedded deps.json,dotnet , fixed#4344
Signed-off-by: Rez Moss <hi@rezmoss.com>
* [wip] have pe utils process embedded dep.json
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* [wip] add PoC bundler processing
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* [wip] search for bundle marker within pe sections
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* put bundle parsing for multiple .net versions under test
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Rez Moss <hi@rezmoss.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fixed#4316 go mod with ver purl
Signed-off-by: Rez Moss <hi@rezmoss.com>
* go mod purl fixed, added func to handle go.mod
Signed-off-by: Rez Moss <hi@rezmoss.com>
* fix: use module name in PURL string everywhere
Signed-off-by: Keith Zantow <kzantow@gmail.com>
---------
Signed-off-by: Rez Moss <hi@rezmoss.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
* fix:the os of an elf binary should be detected even when the os version is empty
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* chore:revoke the update of appCpe
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* chore:resume the testcase
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* fix:revoke the possible compromise to the json schema
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* fix:align with the json schema
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* add a json schema(pre-relase,may be in conflict with others')
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* chore:add a json schema
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* chore:revert the accidental change to 16.1.0
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* regression/fix:best effort to get the os info
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* chore:resume the previous json file
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* update the schema ver to 16.2.0
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* chore:no breaking behavior
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* chore: follow the guide of the README.md
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* appCpe is temporarily unused
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
* preserve json field for osCPE
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
Removes an accidental `fmt.Println("error", err)` that was left in
the javascript dependency parser. This causes noisy output to stdout
when parsing npm package-lock.json files that contain dependency
specifiers that aren't valid PURLs.
Signed-off-by: Chris Greeno <chris@fresha.com>
---------
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Co-authored-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Otherwise, we get test failures on macOS if macOS has decided to put
.DS_Store entries in the test fixtures.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Because package names in METADATA files may have upper case like
Werkzeug or Jinja2, but Syft artifacts have normalized names and are
lower case, like werkzeug or jinja2, Syft would miss emitting dependency
relationships. Therefore, normalize dependency names before comparing
with existing artifacts.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
