Adds `perl` as a language and `cpan` as a package type, with two catalogers behind them.
- `perl-cpan-installed-cataloger` reads `.meta/*/install.json` (cpanm, cpm, carton) and `auto/**/.packlist` (anything installed through `ExtUtils::MakeMaker` or `Module::Build`, including CPAN.pm). Both globs are unanchored, so a local-lib or carton application tree is found the same as a system install.
- `perl-cpan-meta-cataloger` reads an unpacked release's own `META.json` or `META.yml`, gated on a sibling `MANIFEST` so source checkouts are ignored.
Packages are keyed on the **distribution**, not the module, because that is what CPAN, MetaCPAN and the advisory data all use. `LWP.pm` belongs to `libwww-perl` and reporting it as `LWP` would make every advisory for it unreachable. The distribution name comes from `install.json`'s `dist` field, and from the `auto/` path for packlists.
purls are `pkg:cpan/<distribution>@<version>`, with the PAUSE author added as an `author` qualifier when the evidence carries it. Metadata comes in two types: `cpan-distribution` for installed evidence and `cpan-unpacked-release` for a release sitting on disk. The tiers differ in more than a name, so a consumer should not have to string-match a cataloger name to tell them apart: an unpacked release has no PAUSE path and therefore no author and no file list, and "installed and loadable by the interpreter" is a materially stronger claim than "a source tree exists here".
A packlist's version comes from `perllocal.pod`. EUMM writes it and the packlist from the same variables in the same install target, so the `auto/` path segments and the perllocal `Module` name are the same key, and the recorded `VERSION` is what was evaluated at build time rather than what can be read back statically. Scraping `$VERSION` out of the main `.pm` is the fallback, since `NO_PERLLOCAL` suppresses the file and Module::Build never writes one. Three rules pick the stanza: dashed module name, longest `installed into` libdir that prefixes the packlist path, and last match wins because the file is append-only. Without the libdir rule a second perl on the image silently supplies the version.
Where scraping is the fallback, it reads more than a plain `our $VERSION = '...'`. A version declared in the package statement (`package Foo::Bar v1.0.0;`) counts, which matters because a distribution can declare it that way and carry no `$VERSION` at all: `CPAN::02Packages::Search` is one, and without this it reports no version and matches nothing. Fully qualified `$Foo::Bar::VERSION` counts too, which is what older Dist::Zilla emitted and what real `JSON::PP` 2.27300 still carries. `qv('1.2.3')` is handled. A version computed at runtime is not, and those are reported without one rather than guessed at.
Packlist entries are parsed the way `ExtUtils::Packlist` writes them: a line can carry space-separated metadata after the path (`/path/to/File.pm type=file`), which is what `installperl` produces, so the suffix is stripped rather than the line being cut at its first space.
`META.yml` is read alongside `META.json`, because about 43% of current CPAN releases ship no `META.json` and they skew old, which is where the advisories are. Where both sit in one directory the `META.json` wins. A `META.yml` a strict parser rejects skips that directory rather than failing the scan, which is routine rather than defensive for pre-spec releases.
A packlist is literally a file list, so its paths are surfaced through `pkg.FileOwner`. They are reported as recorded, unfiltered: a path the packlist claims and the filesystem lacks means the file was removed or overwritten out from under the installer, which is worth seeing rather than hiding.
Build leftovers under `~/.cpanm/work` and `~/.cpan/build` are skipped. They genuinely are unpacked release tarballs, `MANIFEST` included, so the `MANIFEST` gate admits them and a path exclusion is the only signal available. Without it every distribution on an image that did not clean up is reported twice. The cost is that a tarball deliberately kept under `~/.cpan/build` stops being reported.
Two behaviors that look wrong but are not:
- a distribution can be reported twice at different versions. `libwww-perl` 5.836 bundles `HTTP-Date`, `HTTP-Message` and `LWP-MediaTypes`, and installing it over the modern standalone releases overwrites their `.pm` files. Both versions are genuinely present, so the merge refuses to pair disagreeing versions rather than hiding one.
- a distribution with no readable version is reported without one rather than dropped, so it stays visible.
The coverage boundary is stated in full in the `package perl` doc comment. In short: modules installed from distro packages are out of scope, since packagers strip CPAN metadata with `NO_PACKLIST` and deb, rpm and apk already report them. Core and dual-life distributions bundled with the interpreter have no coverage, because nothing on disk carries their versions; the interpreter itself is reported separately. Vendored trees, `App::FatPacker` output and PAR archives are invisible, because what they carry is module identity with no offline map to a distribution. A packlist-derived name is the installer's `NAME` and may not be the distribution name, which is left to the vulnerability data to resolve.
One correction worth calling out, since it is easy to arrive at twice: a sibling `MANIFEST` is the only thing separating an unpacked release from a source checkout. An earlier version of the guard also rejected any tree containing a `dist.ini`, on the theory that it marked a Dist::Zilla source tree. dzil ships `dist.ini` *inside* the tarballs it builds and lists it in the generated `MANIFEST`, so that exclusion was silently skipping real releases, `URI` among them.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* migrate fixtures to testdata
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: correct broken symlinks after testdata migration
The migration from test-fixtures to testdata broke several symlinks:
- elf-test-fixtures symlinks pointed to old test-fixtures paths
- elf-test-fixtures needed to be renamed to elf-testdata
- image-pkg-coverage symlink pointed to test-fixtures instead of testdata
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: handle missing classifiers/bin directory in Makefile
The clean-fingerprint target was failing when classifiers/bin doesn't
exist (e.g., on fresh clone without downloaded binaries).
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add gitignore negation for jar/zip fixtures in test/cli
The jar and zip files in test/cli/testdata/image-unknowns were being
gitignored by the root .gitignore patterns. This caused them to be
untracked and not included when building docker images in CI, resulting
in Test_Unknowns failures since the test expects errors from corrupt
archive files that weren't present.
Add a .gitignore in test/cli/testdata to negate the exclusions for
these specific test fixture files.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* switch fixture cache to v2
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: update expected versions for rebuilt fixtures
Update test expectations for packages that have been updated in
upstream repositories when docker images are rebuilt:
- glibc: 2.42-r4 → 2.43-r1 (wolfi)
- php: 8.2.29 → 8.2.30 (ubuntu/apache)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* upgrade go
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add go-shlex dependency for testdata manager tool
The manager tool in syft/pkg/cataloger/binary/testdata/ imports
go-shlex, but since it's in a testdata directory, Go doesn't track
its dependencies. This caused CI failures when go.mod didn't
explicitly list the dependency.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* refactor: move binary classifier manager to internal/
Move the manager tool from testdata/manager to internal/manager so
that Go properly tracks its dependencies. Code in testdata directories
is ignored by Go for dependency tracking, which caused CI failures
when go.mod didn't explicitly list transitive dependencies.
This is a cleaner solution than manually adding dependencies to go.mod
for code that happens to live in testdata.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add gitignore negations for test fixtures blocked by root patterns
Multiple test fixtures were being blocked by root-level gitignore patterns
like bin/, *.jar, *.tar, and *.exe. This adds targeted .gitignore files with
negation patterns to allow these specific test fixtures to be tracked:
- syft/linux/testdata/os/busybox/bin/busybox (blocked by bin/)
- syft/pkg/cataloger/java/testdata/corrupt/example.{jar,tar} (blocked by *.jar, *.tar)
- syft/pkg/cataloger/binary/testdata/classifiers/snippets/go-version-hint/**/bin/go (blocked by bin/)
- syft/pkg/cataloger/bitnami/testdata/no-rel/.../bin/redis-server (blocked by bin/)
Also updates the bitnami test expectation to include the newly required
.gitignore files in the test fixture.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: update glibc version expectation (2.43-r1 -> 2.43-r2)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add capability drift check as unit step
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* dont clear test observations before drift detection
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* bump stereoscope commit to main
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* resolve owned file paths when searching for overlaps
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* dont remove empty paths
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: sorting locations should consider pkg evidence
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* simplify location test options for comparison
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* prototype: start bitnami cataloger
Bitnami images have spdx SBOMs at predictable paths, and Syft could more
accurately identify the software in these images by scanning those
SBOMs. Start work on this by forking the sbom-cataloger as a new
bitnami-cataloger.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* wire up bitnami cataloger to run on images by default
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* feat: add support for Bitnami cataloguer
Signed-off-by: juan131 <jariza@vmware.com>
* feat: use a better SPDX sample for unit tests
Signed-off-by: juan131 <jariza@vmware.com>
* bugfix: only report bitnami pkgs
Signed-off-by: juan131 <jariza@vmware.com>
* feat: adapt JSON schema, spdxutil and packagemetadata
Signed-off-by: juan131 <jariza@vmware.com>
* bugfix: integration tests
Signed-off-by: juan131 <jariza@vmware.com>
* feat: implement FileOwner interface
Signed-off-by: juan131 <jariza@vmware.com>
* bugfix: update json schema
Signed-off-by: juan131 <jariza@vmware.com>
* [wip] add bitnami owned files and fix binary package ownership filtering
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* feat: obtain bitnami pkg files based on SPDX relationships tree
Signed-off-by: juan131 <jariza@vmware.com>
* preserve type switches
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* rename bitnami entry metadata type
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* restrict find main pkg logic
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add missing graalvm source info
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* bugfix: integration tests
Signed-off-by: juan131 <jariza@vmware.com>
* bugfix: mod tidy
Signed-off-by: juan131 <jariza@vmware.com>
---------
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Signed-off-by: juan131 <jariza@vmware.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add dpkg evidence support
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* use path over filepath
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add jvm cataloger
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* simplify version selection
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* CPEs from JVM cataloger should be declared
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ensure package overlap is enabled for sensitive use cases
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* more permissive glob
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add policy for empty name and version
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* default stub version
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* modifying ids requires augmenting relationships
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix ELF package types to be honored
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* prefer OS packages over binary packages when there are duplicates
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add python package relationships
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* nil for empty relationships collections
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* new json schema for optional python requiremenets
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* update format snapshots for python packages
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* decompose python parsers more + add tests around plural fields
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* update JSON schema with python dep refs
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* use pkg values in relationship fields
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add linter rule for using values in relationships
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* use new cmptest package for comparing relationships
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* create cmptest for common cmp.Diff options in test
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* condense matches for relationship ruleguard
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* remove relationship type from rules
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* restore build tag
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* suggest using values
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* nil check pkgs
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
This PR adds DependencyOf relationships when ELF packages have been discovered by the binary cataloger. The discovered file.Executable type has a []ImportedLibraries that's read from the file when discovered by syft. By mapping these imported libraries back to the package collection, syft is able to create relationships showing which packages are dependencies of other packages by just reading metadata from the ELF executable.
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Brian Ebarb <ebarb.brian@sers.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* remove existing cataloging API
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add file cataloging config
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add package cataloging config
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add configs for cross-cutting concerns
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* rename CLI option configs to not require import aliases later
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* update all nested structs for the Catalog struct
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* update Catalog cli options
- add new cataloger selection options (selection and default)
- remove the excludeBinaryOverlapByOwnership
- deprecate "catalogers" flag
- add new javascript configuration
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* migrate relationship capabilities to separate internal package
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* refactor golang cataloger to use configuration options when creating packages
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* create internal object to facilitate reading from and writing to an SBOM
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* create a command-like object (task) to facilitate partial SBOM creation
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add cataloger selection capability
- be able to parse string expressions into a set of resolved actions against sets
- be able to use expressions to select/add/remove tasks to/from the final set of tasks to run
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add package, file, and environment related tasks
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* update existing file catalogers to use nested UI elements
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add CreateSBOMConfig that drives the SBOM creation process
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* capture SBOM creation info as a struct
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add CreateSBOM() function
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* update docs with SBOM selection help + breaking changes
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix multiple override default inputs
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix deprecation flag printing to stdout
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* refactor cataloger selection description to separate object
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* address review comments
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* keep expression errors and show specific suggestions only
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* address additional review feedback
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* address more review comments
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* addressed additional PR review feedback
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix file selection references
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* remove guess language data generation option
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add tests for coordinatesForSelection
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* rename relationship attributes
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add descriptions to relationships config fields
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* improve documentation around configuration options
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add explicit errors around legacy config entries
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
