* chore: centralize temp files and prefer streaming IO
Catalogers that create temp files ad-hoc can easily forget cleanup,
leaking files on disk. Similarly, io.ReadAll is convenient but risks
OOM on large or malicious inputs.
Introduce internal/tmpdir to manage all cataloger temp storage under
a single root directory with automatic cleanup. Prefer streaming
parsers (bufio.Scanner, json/yaml.NewDecoder, io.LimitReader) over
buffering entire inputs into memory. Add ruleguard rules to enforce
both practices going forward.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: go back to old release parsing
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* simplify to limit reader in version check
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: regex change postponed
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* simplify supplement release to limitreader
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
---------
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore(deps): update tools to latest versions
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* chore(lint): fix errors in new golangci-lint
Two fixes:
First, replace sb.WriteString(fmt.Sprintf(...)) with fmt.Fprintf(&sb, ...)
Second, suppress errors where we read from the local file system at a
user provided path. This is a CLI tool, and reads from user provided
paths on the local file system by design.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Co-authored-by: spiffcs <32073428+spiffcs@users.noreply.github.com>
Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* migrate fixtures to testdata
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: correct broken symlinks after testdata migration
The migration from test-fixtures to testdata broke several symlinks:
- elf-test-fixtures symlinks pointed to old test-fixtures paths
- elf-test-fixtures needed to be renamed to elf-testdata
- image-pkg-coverage symlink pointed to test-fixtures instead of testdata
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: handle missing classifiers/bin directory in Makefile
The clean-fingerprint target was failing when classifiers/bin doesn't
exist (e.g., on fresh clone without downloaded binaries).
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add gitignore negation for jar/zip fixtures in test/cli
The jar and zip files in test/cli/testdata/image-unknowns were being
gitignored by the root .gitignore patterns. This caused them to be
untracked and not included when building docker images in CI, resulting
in Test_Unknowns failures since the test expects errors from corrupt
archive files that weren't present.
Add a .gitignore in test/cli/testdata to negate the exclusions for
these specific test fixture files.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* switch fixture cache to v2
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: update expected versions for rebuilt fixtures
Update test expectations for packages that have been updated in
upstream repositories when docker images are rebuilt:
- glibc: 2.42-r4 → 2.43-r1 (wolfi)
- php: 8.2.29 → 8.2.30 (ubuntu/apache)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* upgrade go
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add go-shlex dependency for testdata manager tool
The manager tool in syft/pkg/cataloger/binary/testdata/ imports
go-shlex, but since it's in a testdata directory, Go doesn't track
its dependencies. This caused CI failures when go.mod didn't
explicitly list the dependency.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* refactor: move binary classifier manager to internal/
Move the manager tool from testdata/manager to internal/manager so
that Go properly tracks its dependencies. Code in testdata directories
is ignored by Go for dependency tracking, which caused CI failures
when go.mod didn't explicitly list transitive dependencies.
This is a cleaner solution than manually adding dependencies to go.mod
for code that happens to live in testdata.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add gitignore negations for test fixtures blocked by root patterns
Multiple test fixtures were being blocked by root-level gitignore patterns
like bin/, *.jar, *.tar, and *.exe. This adds targeted .gitignore files with
negation patterns to allow these specific test fixtures to be tracked:
- syft/linux/testdata/os/busybox/bin/busybox (blocked by bin/)
- syft/pkg/cataloger/java/testdata/corrupt/example.{jar,tar} (blocked by *.jar, *.tar)
- syft/pkg/cataloger/binary/testdata/classifiers/snippets/go-version-hint/**/bin/go (blocked by bin/)
- syft/pkg/cataloger/bitnami/testdata/no-rel/.../bin/redis-server (blocked by bin/)
Also updates the bitnami test expectation to include the newly required
.gitignore files in the test fixture.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: update glibc version expectation (2.43-r1 -> 2.43-r2)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add capability drift check as unit step
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* dont clear test observations before drift detection
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* bump stereoscope commit to main
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add info command from generated capabilities
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* correct gentoo and arch ecosystems
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* rename os pkg types
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* better binary cataloger description
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* expose metadata and pacakge types in json
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* expose json schema types
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add completeness tests for metadata types
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* latest generation
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix linting
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* improve testing a docs
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix tests and linting
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* restore goreleaser config
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* tweak diagram
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix pdm
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore: java binary data
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* new capability descriptions for gguf and python
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* correct poetry lock integrity hash claim
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix compile error
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: remove purl version from overrides
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* fix lua deps ref
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* keep gguf as ai ecosystem
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* split packages.yaml to multiple files by go package
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ensure tests do not use go test cache
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* sort json output for info command
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* docs: fix ocaml, php, and portage capabilities yaml
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: update erlang capabilities
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: update java capabilities
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: update javascript capabilities
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: update linux kernel capabilities
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* remove missing tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix package.yaml references
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* revert license list change
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* check for drift in capability descriptions
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* regenerate capabilities
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test cleanup
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* use fixture cache in static analysis
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* claim fixtures pre-req for cap generation
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* update documentation with correct regeneration procedure
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore: ruby-gemspec-cataloger finds no dependencies
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: fix python docs and config comment
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: commit re-generated java yaml
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* add cataloger selection to caps command
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* re-generate cap yamls
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix tests for cataloger selection
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix cli test
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add missing tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix linting
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* rename cmd to `cataloger info`
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* [wip] change capability description locations
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* [wip] continued
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* [wip] adjust for import cycles
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* correct docs
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix linting
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* unpin dependencies for package dedup case
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* prevent make from blocking tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add check
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fixed#4316 go mod with ver purl
Signed-off-by: Rez Moss <hi@rezmoss.com>
* go mod purl fixed, added func to handle go.mod
Signed-off-by: Rez Moss <hi@rezmoss.com>
* fix: use module name in PURL string everywhere
Signed-off-by: Keith Zantow <kzantow@gmail.com>
---------
Signed-off-by: Rez Moss <hi@rezmoss.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
* feat: Add license enrichment from pypi to python packages
* Implement license caching and improve test coverage
---------
Signed-off-by: Tim Olshansky <456103+timols@users.noreply.github.com>
* ref: close source in test and examples
Signed-off-by: Kudryavcev Nikolay <kydry.nikolau@gmail.com>
* ref: pretty file/directory source resolver (make them more similar)
Signed-off-by: Kudryavcev Nikolay <kydry.nikolau@gmail.com>
* ref: move absoluteSymlinkFreePathToParent to file resolver
Signed-off-by: Kudryavcev Nikolay <kydry.nikolau@gmail.com>
* revert breaking change
Signed-off-by: Kudryavcev Nikolay <kydry.nikolau@gmail.com>
---------
Signed-off-by: Kudryavcev Nikolay <kydry.nikolau@gmail.com>
* feat: update integration test with correct package for httpd
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* chore: update integration and cli tests with new upstream expectations
- php interpreter 8.3.21 => 8.3.22
- runCycloneDXInDocker update for local arm64 qemu emulation CycloneDX
- getSyftBinaryLocationByOS update to detect arm64 v8.0 artifact path
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* chore: add snalshot to test command for fixture builds
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* chore: update cdx in docker for all GOOS
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* feat: expose rpm signature information
This helps with more confident identification of an rpm.
In theory, two rpms can be built that have the same purl string, and
otherwise look identical in syft's output, but the PGP information
would distinguish them as signed either by different keys, or signed at
different times.
In practice, this usually makes no difference since rpms tend to have
unique name/version/release strings. This just gives increased
confidence about the identity of the rpm found in the db.
Signed-off-by: Ralph Bean <rbean@redhat.com>
* chore: generate json schema
Signed-off-by: Ralph Bean <rbean@redhat.com>
* re-generate json schema
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* rename to a more generic signature field
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* rename rpm.pgp to rpm.signatures
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* split out signature fields
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* bump json schema
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* include RPM archives
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* update json schema
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* dont fail on unknown signature type
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Ralph Bean <rbean@redhat.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add nix DB cataloger
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add derivation path to nix store pkg metadata
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* go mod tidy
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* allow for derivation path to be optional
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* repin build image and disable syscall filtering
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* bump storage capacity
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* track nix derivation details on packages
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* image fixture should have derivation examples
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* address comments
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* Add support for PHP Pear and unify PECL with it
Signed-off-by: Laurent Goderre <laurent.goderre@docker.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* remove log statements
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix struct comment
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Laurent Goderre <laurent.goderre@docker.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* consider child dll claims for .NET packages from deps.json
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* make dll claim propagation configurable
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add combined deps.json + pe binary cataloger
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* deprecate pe and deps standalone catalogers
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* parse resource names + add tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix integration and CLI tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add some helpful code comments
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* allow for dropping Dep packages that are missing DLLs
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* migrate json schema changes to 24
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* keep application configuration
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* correct config help
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* [wip] detect claims of dlls within deps.json
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* [wip] fix tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add assembly repack detection
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* .net package count is lower due to dll claim requirement
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* prototype: start bitnami cataloger
Bitnami images have spdx SBOMs at predictable paths, and Syft could more
accurately identify the software in these images by scanning those
SBOMs. Start work on this by forking the sbom-cataloger as a new
bitnami-cataloger.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* wire up bitnami cataloger to run on images by default
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* feat: add support for Bitnami cataloguer
Signed-off-by: juan131 <jariza@vmware.com>
* feat: use a better SPDX sample for unit tests
Signed-off-by: juan131 <jariza@vmware.com>
* bugfix: only report bitnami pkgs
Signed-off-by: juan131 <jariza@vmware.com>
* feat: adapt JSON schema, spdxutil and packagemetadata
Signed-off-by: juan131 <jariza@vmware.com>
* bugfix: integration tests
Signed-off-by: juan131 <jariza@vmware.com>
* feat: implement FileOwner interface
Signed-off-by: juan131 <jariza@vmware.com>
* bugfix: update json schema
Signed-off-by: juan131 <jariza@vmware.com>
* [wip] add bitnami owned files and fix binary package ownership filtering
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* feat: obtain bitnami pkg files based on SPDX relationships tree
Signed-off-by: juan131 <jariza@vmware.com>
* preserve type switches
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* rename bitnami entry metadata type
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* restrict find main pkg logic
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add missing graalvm source info
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* bugfix: integration tests
Signed-off-by: juan131 <jariza@vmware.com>
* bugfix: mod tidy
Signed-off-by: juan131 <jariza@vmware.com>
---------
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Signed-off-by: juan131 <jariza@vmware.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add file catalogers to selection configuration
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix typos
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* warn when there is conflicting file cataloging configuration
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* allow for explicit removal of all package and file tasks
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* address PR feedback
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
