According to maven central, the package called "xalan" should just have
the group ID xalan, but currently syft isn't able to find that.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Previously, building the index of the filesystem when source was file
would fail if part of the path syft was passed to the file included a
symlinked directory, resulting in cataloging misses.
---------
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Allow existing matcher to match host identifiers longer than 12
characters. The binaries distributed by redis have the version before
payload, so add a matcher for that. Add test fixtures covering these
scenarios.
Signed-off-by: David Dooling <david.dooling@docker.com>
* fix: update parsing logic to remove empty space
* tests: update with test cases provided by community
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
* Add ruby.NewGemSpecCataloger to DirectoryCatalogers.
Signed-off-by: Evan <chaol@vmware.com>
* fixed tests
Signed-off-by: Evan <chaol@vmware.com>
* Addressed review comment
Signed-off-by: Evan <chaol@vmware.com>
* Remove NewInstalledGemSpecCataloger from default dir catalogers
Because the files that the installed gemspec cataloger work off of are a
subset of the files that the more general gemspec cataloger will work
off of, we shouldn't have both of them on by default, since this could
result in finding the same package twice.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
---------
Signed-off-by: Evan <chaol@vmware.com>
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Co-authored-by: Will Murphy <will.murphy@anchore.com>
adds a unique synthetic package to the SBOM output that represents the go compiler when it is detected as a part of a package discovered by the go binary cataloger.
When using an SBOM generated by syft - downstream vulnerability scanners now have the opportunity to detect/report on the PURL/CPEs attached to the new stdlib package.
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Previously, which PURL was generated depended on the order of key iteration
in maps. Also update an integ test that was apparently only passing because
of the previous issue.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Now that the test fixture pins to a particular digest, there's no need
for platform specific architecture switches in this test.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Many of these images have a slightly different sets of packages when the
arm64 variant is pulled, so that leaving this digest unpinned causes the
tests to fail on arm64 hosts. Pin the FROM lines to force stable
platform values regardless of host architecture.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Previously, there were some data races in syft. Right now, none are
detected, so check for data races on the overall command, and on unit
tests. (Checking for races on integration tests triples the time needed
for those tests, from ~1 minute to ~3 minutes on my workstation, so that
was not done at this time.)
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Otherwise, small renames like 'hudson-war-2.2.1.war' to 'hudson.war', would cause
syft to incorrectly catolog the archive.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
If crypto settings or arch cannot be determined, still attempt to catalog packages from
the build info, rather than panicking.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
A recent update to gopom changed many fields to be omitted when empty,
resulting in a number of nil pointers inside the nested structure of the
pom that previously didn't exist. Defend against these in the search for
the property value.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
This reflect code occasionally throws an obscure panic, but not enough
information is logged before the panic to know why it panicked. Log
enough to tell what property and package are being analyzed when the
panic occurs.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Separate the logic for CPE and PURL generation.
PURL generation needs a single answer for groupID based on a priority of discovering the field.
CPE generation still uses multiple potential groupID to populate the candidate cpe.
Improve GroupID detection.
Currently syft does not use any hierarchy for GroupID detection and treats all sources as equal.
It treats fields from the manifest file with priority. This change adds a hierarchy to the fields and returns a single answer based on that hierarchy.
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
Fixes#931
PR #1948 introduces a new implicit exclusion for binary packages that overlap by file ownership and have certain characteristics:
1) the relationship between packages is OwnershipByFileOverlap
2) the parent package is an "os" package - see changelog for included catalogers
3) the child is a synthetic package generated by the binary cataloger - see changelog for included catalogers
4) the package names are identical
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
* Add opkg info directory and status file to deb cataloger
opkg uses the same or nearly the same metadata and structure as Debian:
**/lib/opkg/status lists status information for all packages
**/lib/opkg/info/opkg.conffiles is a list of configuration files
**/lib/opkg/info/*.list contains files and directories installed by the package
**/lib/opkg/info/*.preinst are scripts to run before installation
**/lib/opkg/info/*.postinst are scripts to run after installation
**/lib/opkg/info/*.postrm are scripts to run after package removal
**/lib/opkg/info/*.control provides package metadata
Signed-off-by: Nicholas R. Smith <nicholas_smith@selinc.com>
---------
Signed-off-by: Nicholas R. Smith <nicholas_smith@selinc.com>
Co-authored-by: Nicholas R. Smith <nicholas_smith@selinc.com>
* fix: default image source name to user input
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: add test
Signed-off-by: Keith Zantow <kzantow@gmail.com>
---------
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* Add support for parsing .NET assemblies
Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Former-commit-id: 69c33fe4d77357d843c11590f3b07825bc6249ac
* Add dll and exe files
Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Former-commit-id: b9d204efa6d2ef385b5fbb7a59a3474ecabea641
* Add PE cataloger to directory catalogers
Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Former-commit-id: 9711c00d9da92e2887e0c1f92edd740ea5345849
* Don't set language to dotnet for PEs
Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Former-commit-id: 368313fddac9160d8a06a01ebe8c5ac7990232f5
* Fix spelling of cataloger in constructor
Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Former-commit-id: e42fd77b2f8b6d42e076a84f6cce386861260941
* Adjust which cases in PE parsing return errors
Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Former-commit-id: 95b25f8fc3a7d4e18fe30e489b09851f316795ff
* remove build binary from branch
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Former-commit-id: fa54c0d0aef0998d5520e9f44cae51f5f9cd38a2
* Fix failing CLI tests
Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
---------
Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* docs: update docs with new docker specific instructions
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Former-commit-id: c67c76e84df84e3e24aa307637d884ca8b7e3eea
* fix: or-later suffix has been updated to consider deprecated +
If a given license has the suffix "or-later" it previously could have
been considered or represented with a "+". Example "GFDL-1.0-or-later"
could have been represented as "GFDL-1.0+". This PR allows the license
list generation to consider "or-later" as == to "+" when generating
permutations for upgrading deprecated licenses.
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Add some explanation around why there are different default sets of catalogers for image scans versus directory scans. Hopefully clarify questions related to #1776.
Signed-off-by: Timothy Gerla <tim@gerla.net>
Track a chronicle config file that causes chronicle to bump minor
version instead of major version in response to the "breaking-change"
label for pre-1.0 releases.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
* [wip] try to reflect metadata types... probably wont work
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* refactor to add unit test to ensure there is coverage in the schema
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* [wip] generate metadata container
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add generation of metadata container struct for JSON schema generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update linter script to account for code generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
---------
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
to keep things consistent across the schema we want Locations and URLs to be plural fields now that they are fields on the License struct
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
this PR makes the following changes to update the underlying license model to have more expressive capabilities
it also provides some guarantee's surrounding the license values themselves
- Licenses are updated from string -> pkg.LicenseSet which contain pkg.License with the following fields:
- original `Value` read by syft
- If it's possible to construct licenses will always have a valid SPDX expression for downstream consumption
- the above is run against a generated list of SPDX license ID to try and find the correct ID
- SPDX concluded vs declared is added to the new struct
- URL source for license is added to the new struct
- Location source is added to the new struct to show where the expression was pulled from
If syft is asked to parse an SBOM that was written by a newer version of
syft, emit a warning, since the current version of syft doesn't know about
fields that may be added in the future.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Add a cataloger that detects installed R packages by looking for DESCRIPTION
files. The base R package is now picked up in coverageImage tests in
test/cli/packages_cmd_test.go, so increment expected package counts for the
tests that use that image.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Rather than log a warning for every instance of an unknown relationship type,
or similar error, log a count of how many times each of these errors is
raised.
---------
Signed-off-by: Will Murphy <will.murphy@anchore.com>
The tests in test/cli/packages_cmd_test.go are hard to debug when different
packages are found in different environments. For example, CI runs and M1 macs
have been observed to have different package counts. Therefore, if the test is
about to fail, log a sorted list of the packages that were found, so that it is
easy to compare failures of these tests.
Signed-off-by: Will Murphy <will.murphy@anchore.com>
- Create new metadata struct and type for python requirements.
- Update parsing of python requirements to use python requirements metadata.
- Remove extras and url from line. Add them to metadata instead.
- Add unit test to test that extras are removed from package name.
- Update test to look at requirements metadata.
- Will need updated in future to support more than just == for the version constraint.
- Update JSON schema data
Closesanchore/grype#1246Closesanchore/grype#1251
Signed-off-by: Shane Dell <shanedell100@gmail.com>
* Create consul binary classifier
Closes#1590
Signed-off-by: Shane Dell <shanedell100@gmail.com>
* Create test for consul binary classifier
Signed-off-by: Shane Dell <shanedell100@gmail.com>
* Update version for consul. Add note that about consul version matcher is brittle
Signed-off-by: Shane Dell <shanedell100@gmail.com>
---------
Signed-off-by: Shane Dell <shanedell100@gmail.com>
- Update README.md to show that nar is now supported.
- Created a java-archives/example.nar so that the tests wouldn't break.
- Add nar glob and as an option for pkgType.
Closes#1701
Signed-off-by: Shane Dell <shanedell100@gmail.com>
Update the license_list.go to have more permissible inputs for greater SPDXID matching.
EX:
GPL3 gpl3 gpl-3 and GPL-3 can all map to GPL-3.0-only
By moving all strings to lower and removing the "-" we're able to return valid SPDX license ID for a greater diversity of input strings.
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
* add argocd, helm, kustomize and kubectl binary classifiers
* update golang PURL
* address PR faceback about binary/test-fixtures/Makefile
* remove the /v[n] suffix from the PURL in both argocd and helm
---------
Signed-off-by: y12studio <y12studio@gmail.com>
Adding APK OriginPackage CPE candidates to the child package
results in false positives in grype because it can't associate
CPE-based findings to the corresponding OriginPackage APK fixes.
This reverts changing the `upstream` in the PURL for APK packages
as the logic in Grype that uses it expects it to be an APK package
name. This also allows refactoring to unexport and move the APK
CPE candidate generation logic closer to where CPE generation occurs
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
This fixes some instances where the improved APK CPE generation
logic caused regressions for older alpine package APK metadata.
It now generates multiple "upstream" candidates with both name
and package type which reduces the amount of duplicated code in
the apk cpe gen logic. This also improves the handling of stream
version packages, so now we can correctly identify packages such
as ruby3.2-rexml as the rexml ruby gem.
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
The apk purl spec allows for vendor-specific namespace. I noticed
in the embedded SBOMs from wolfi that the purls are of the form
`pkg:apk/wolfi/curl@7.83.0-r0?arch=x86`, but the current logic in
syft actually prevents purl generation entirely if the distro isn't
alpine, so this corrects that behaviour.
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
* refactor: move apk upstream logic to apk metadata
Export the logic for parsing upstream APK package names
so it can be accessed from apk metadata objects directly.
This also tightens the upstream regex pattern as several
edge cases were being missed.
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
* fix: ensure correct handling for apk packages beginning with digits
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
* fix: upstream generation for ruby
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
---------
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
* replace raw globs with index equivelent operations
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add cataloger test for alpm cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix import sorting for binary cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting for mock resolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* separate portage cataloger parser impl from cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* enhance cataloger pkgtest utils to account for resolver responses
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for alpm cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for apkdb cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for dpkg cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for cpp cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for dart cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for dotnet cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for elixir cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for erlang cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for golang cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for haskell cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for java cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for javascript cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for php cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for portage cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for python cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for rpm cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for rust cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for sbom cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for swift cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* allow generic catloger to run all mimetype searches at once
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove stutter from php and javascript cataloger constructors
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bump stereoscope
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add tests for generic.Search
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add exceptions for java archive git ignore entries
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* enhance basename and extension resolver methods to be variadic
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* dont allow * prefix on extension searches
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add glob-based cataloger tests for ruby cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove unnecessary string casting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* incorporate surfacing of leaf link resolitions from stereoscope results
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* [wip] switch to stereoscope file metadata
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* [wip + failing] revert to old globs but keep new resolvers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* index files, links, and dirs within the directory resolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix several resolver bugs and inconsistencies
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* move format testutils to internal package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update syft json to account for file type string normalization
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split up directory resolver from indexing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update docs to include details about searching
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* [wip] bump stereoscope to development version
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust symlinks fixture to be fixed to digest
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix all-locations resolver tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix test fixture reference
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename file.Type
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bump stereoscope
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix PR comment to exclude extra *
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bump to dev version of stereoscope
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bump to final version of stereoscope
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* move observing resolver to pkgtest
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
---------
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add release trigger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* deduplicate version and changelog calls + add gh checks
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add more chronicle verbosity, but not when triggering releases
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bump chronicle version to get --version-file feature
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update bootstrap tool workflow to include glow
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add version prefix check on tags in release quality gate
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
---------
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
When base is set, it should appear identically to when we scan the root
filesystem - and as a result, the path should begin with the path
separator.
E.g. when scanning the root `./target/` with the same base,
`target/bin/busybox` should appear in the output as `/bin/busybox`, not
as previously as `bin/busybox`.
Signed-off-by: Justin Chadwell <me@jedevc.com>
* fix: update config struct to not decode password/key
* test: update tests to confirm no secrets in output
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
* source: avoid second-step of symlink resolution in directory resolver
We can use the already existing file tree to peform symlink resolution
for FilesByPath, instead of traversing the symlinks again.
This moves all of the symlink logic into the indexing code, and then we
can rely on syft's resolution algorithm over the index in this part of
the codebase.
Signed-off-by: Justin Chadwell <me@jedevc.com>
* source: add base parameter to directory resolver
The new base parameter is an optional parameter for the directory
resolver that resolves all symlinks relative to this root. There are two
intended use cases:
- base = "/". The previous behavior, symlinks are resolved relative to
the root filesystem.
- base = path. Symlinks are resolved relative to the target filesystem,
allowing correct behavior when scanning unpacked container filesystems
on disk.
Signed-off-by: Justin Chadwell <me@jedevc.com>
* source: add tests for new base parameter
Signed-off-by: Justin Chadwell <me@jedevc.com>
---------
Signed-off-by: Justin Chadwell <me@jedevc.com>
* fix AssertEncoderAgainstGoldenSnapshot calls to conditionally update
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* chore: redact schema versions from syftjson comparison
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: does not need a multiline expression
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
* feat: prefer known CPE vendors over other candidates
All ASF projects will be under the `apache` vendor in CPE, and
indeed this is already one of the candidates, but the logic
for selecting the 'most specific' CPE string would select for
example `apache_software_foundation` or `commons-text`.
This is not necessarily 'wrong' in the CPE candidate selection
logic: there is no way to reliably determine the right candidate.
I think it makes sense to use specific data around the vendor
candidate generation, somewhat similar to
'defaultCandidateAdditions'.
Unfortunately there are still a few CVE's for old (pre-5.x,
long unsupported) tomcat versions that are actually tagged with
`apache_software_foundation`, but I'm not sure those are worth
spending time on.
Signed-off-by: Arnout Engelen <arnout@bzzt.net>
* chore: swap out array of vendors for set data structure
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Arnout Engelen <arnout@bzzt.net>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
The original fix b125ea83ba didn't catch
all the excessive warnings, it seems like getArches can also be called
on binaries that aren't neccessarily go binaries, so the messages from
this should also be Trace instead of Warn.
Signed-off-by: Justin Chadwell <me@jedevc.com>
This PR adds support for npm lockfile version 3, which drops the
"dependencies" key and uses "packages" instead. I've refactored the
lockfile parser to make the distinction between the versions explicit
rather than the implicit behaviour before. It _might_ be worth splitting
into separate files at some point, but the logic is so minimal that I
haven't done it.
Fixes#1203
Signed-off-by: Rob Cresswell <robcresswell@users.noreply.github.com>
* add second generation of generic cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* upgrade aplm cataloger to use generic.Cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove pacakge found-by attribute from the definition of a package ID
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Previously, extracting relationships between packages and files was not
completing correctly, as SPDXRef- ElementIDs were being compared to raw
IDs, and so never matched. This patch ensures that we always compare
ElementIDs, to ensure that the hasFiles field is correctly populated.
Signed-off-by: Justin Chadwell <me@jedevc.com>
add debug distroless image to published release
Debian was chosen based on the fact that it is the smallest available distroless image
The new tag is `anchore/syft:debug`
Closes#833
* add main module field to go bin metadata
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* udpate json ouput schema to 3.2.4
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* clean up fixture
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* Add filters to package cataloger
This PR adds filters so a package without name or version doesn't go in
the list of all discovered packages.
Integration and cli tests were added to validate the feature.
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* add nolint:funlen to cataloger/catalog.go
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* don't require package version
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* add package filtering to generic and python cataloger
also removes cli tests in favor of integration and unit tests
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* drop nolint:funlen
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* check for no-removal operation
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* remove unused fixtures
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* rename no-version file to hide semantic version
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* drop integration tests and add pkg func for validation
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* python cataloger use global pkg validation
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* check for valid packages on deb/go/rpm catalogers
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* update rpm cataloger after rebase
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* nit with pointers
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* simpler use of package validation
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* remmove double pkg validations
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* rename func param to artifactsToExclude
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* add test for relationships and bug fix
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* feedback changes
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* Support RPM distros with newer db formats
Recent RPM distros (Fedora 33+, CBL-Mariner 2.0+, amazonlinux 2022+)
use an sqlite package database in /var/lib/rpm/rpmdb.sqlite, or
"ndb" format (SUSE).
Remove anchore's fork in favour of the upstream,
https://github.com/knqyf263/go-rpmdb, to gain support for
these formats.
Signed-off-by: Tom Fay <tomfay@microsoft.com>
* add exception for modernc.org repos
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* shorten rpmdb helper function
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
* Use SBOM descriptor version
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* Update tests
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* CycloneDX extract tools metadata in decoding stage
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* add descriptor to spdx tag-value test
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* remove comment
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* golang module CPE with full path
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* add note on longer Golang CPEs
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* bump stereoscope version to include source path fix
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* go mod tidy
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* add convert command
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* mvp
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* fix hanging bug
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* validate SBOM formats for conversion
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* move convert cmd to new structure
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* remove bin
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* drop event loop from convert cmd
extract SBOM type from document namespace
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* validate SPDX in tests
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* documenting convert cmd
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* support output format=file.json notation
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* test convertible formats
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* fix typo
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* clean up
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* more clean up and docs
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* feedback changes
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* nit
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* feedback changes
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* re-use more code
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* undo encode-decode cycle test
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* remove unnecessary test constraint
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* fix readme
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* try verbose
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* cleaner README and no table conversion
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* simpler conversion
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* feedback changes and cleanup
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* nit space fix
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* use defer
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* feedback changes
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
* Fix "bad output format" for `github-json` output
Signed-off-by: Steven Maude <git@stevenmaude.co.uk>
* Update formats in README
Signed-off-by: Steven Maude <git@stevenmaude.co.uk>
* Run `make lint-fix`
Signed-off-by: Steven Maude <git@stevenmaude.co.uk>
* read Go main module version as is - (devel)
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
* fix package test with default (devel) main module
Signed-off-by: Jonas Galvão Xavier <jonasx@anchore.com>
The Official CPE dictionary currently contains entries for springframework with three different vendors: springsource, vmware, and pivotal_software. This appears to be because ownership has changed over time.
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
* Implement fmt.Stringer with format.ID
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Add failing test for formats processing empty SBOMs
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Account for nil SPDX document during Syft model conversion
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Less verbose logging in Golang Cataloger
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* debug for known gray errors
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* only show warnings when a binary is not a go executable
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
Since syft now depends on debug/buildinfo go 1.18 is required to build
syft and as such go.mod needs updating
Signed-off-by: 06kellyjac <jack@control-plane.io>
* initial working version
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* added build settings to pkg metadata
wip - unit tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* handle mach-O FatFiles
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add support to mod replace
fixed golang catalger tests
trying GH Actions with go 1.18rc1
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* log error
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* use go-macholibre for extraction
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* cleaner tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add version to main module
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* check macho file with macholibre
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* run golangci in its own workflow
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* wip - golangci workflow
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix golangci wf yml
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix golangci wf yml
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* wip - golangci wf
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* wip - golangci wf
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* get arch from bin file headers
upgrade macholibre
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go mod tidy
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* test new stereoscope lazy reader interface
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go mod tidy
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* remove devel version from golang cataloger
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go mod tidy
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* switch github workflows to go1.18 stable
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add union reader interface in golang cataloger
update stereoscope
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go mod tidy
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* simpler golangci validation
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix makefile
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* get archs refactor
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* nolint for golang version
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix go bin tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* feedback changes
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* golangci nolint needs a \n before package
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* cleanup
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* move golangci-lint to its own jobs again
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix ci yaml
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add support for xcoff files
add arch assets to test bin file types
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* clean up golangci-lint config
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* nolint for xcoff
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* explain nolints
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* remove unused xcoff testdata assets
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* make go bin test-fixtures in docker
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix make clean with -f
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* update json output schema
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* update schema version in test fixture
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* feedback changes
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* explain possible empty main module
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add test cases for yarn parser regex
Signed-off-by: Patrick Glass <patrickglass@gmail.com>
* update yarn.lock parser to support yarn berry
Add support for Yarn v3 (berry) which changes the output
Collapse regex for parsing scoped and non-scoped packages
Add tests for the regex to ensure backwards compatability
and to catch issues with future changes.
Signed-off-by: Patrick Glass <patrickglass@gmail.com>
* simplify yarn test expressions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Patrick Glass <patrickglass@gmail.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
Add source.NewFromRegistry function so that the syft attest command can always explicitly ask for an OCIRegistry provider rather than rely on local daemon detection for image sources.
Attestation can not be used where local images loaded in a daemon are the source. Digest values for the layer identification step in attestation can sometimes vary across workstations.
This fix makes it so that attest is generating an SBOM for, and attesting to, a source that exists in an OCI registry. It should never load a source from a local user docker/podman daemon.
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
add syft attest command to produce an attestation as application/vnd.in-toto+json to standard out using on disk PKI
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
* ignore minor parsing error when reading dpkg status files
helps with https://github.com/anchore/syft/issues/733
Question: should we add a smarter parser to guess approximate installed-size
value?
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add datasize lib to help dpkg parsing
added unit tests to expand coverage of dpkg parsing
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* drop parse error
added unit tests to handleNewKeyValue
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* don't return parsing errors from dpkg
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go mod tidy
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* test higher level functions
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* return parsing err to let cataloger handle it
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* feedback changes
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* ignore key parsing error
log warning with relevant context
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go mod tidy
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add context info to log lines
simpler error assertion
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* use error.As to assert error in chain
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* update stereoscope
fetches latest fixes for UI
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* use context when getting image
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* use SYFT_LOG_FILE
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* enable debug logs when SYFT_LOG_FILE is set
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* set log.file and add tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* test log file in temp directory
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add note on binding refactor
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* remove unused function
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* reduce parallelism of builds and increase install.sh test setup buffer
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* change logging mechanism for signing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* restore automatic parallelism determination for goreleaser
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rm logging goreleaser version
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* refactor signing steps in release/snapshot workflows
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* show signing logs on snapshot or release failure
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update install.sh + tests to account for new goreleaser changes
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update cli tests to account for new goreleaser build names
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix acceptance test to use new snapshot bin path
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add notarization
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* address review comments
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update stereoscope
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go mod tidy
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix FilesByMIMEType tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* change expected mime types in unit tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* test stereoscope fix
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* remove mod replace and use latest stereoscope
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* Update Syft formats for SyftJson
This change will introduce omitempty struct tag to PackageCustomData.
This struct tag will cause null and empty values to be dropped on serialization
for consumers downstream.
Signed-off-by: Toure Dunnon <toure.dunnon@anchore.com>
* Updated the golden files for syftjson to allow for proper
test coverage.
Signed-off-by: Toure Dunnon <toure.dunnon@anchore.com>
* Add tests for image and directory syftjson source
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Add failing test case for file source unmarshaling
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Fix file source unmarshaling
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Add test case for unknown source type
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* remove strong distro type
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bump json schema to v3 (breaking distro shape)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* allow for v2 decoding of distro idLikes field in v3 json decoder
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix casing in simple linux release name
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use discovered name as pretty name in simple linux release
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Support Windows Directory Resolver
Add function that converts windows to posix functionality
Add function that converts posix to windows
Add build tags to remove windows developer environment errors
redact carriage return specific windows issues
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
* Fix CPE generation when the generated CPE contains invalid characters
Currently syft seems to generate invalid CPEs which do not
conform with the official CPE spec. This is because the underlying
nvdtools library is not a completely spec compliant implementation
and has some interesting bugs/issues.
The following are the list of issues I have encountered with nvdtools:
1. It parses strings which are not CPEs incorrectly as valid CPEs. This
messes up our filter function which is supposed to filter out any
incorrect CPEs we generate. In order to fix this, I have introduced
a new regex in the NewCPE function which follows the upstream spec and
filters out any incorrect CPEs.
2. Introduce wfn.WFNize for any cpe attributes we infer from packages.
This ensures that we are escaping and quoting any special characters
before putting them into CPEs. Note that nvdtools has yet another bug
in the WFNize function, specifically the "addSlashesAt" part of the
function which stops the loop as soon as it encounters ":" a valid
character for a WFN attribute after quoting, but the way nvdtools
handles it causes it to truncate strings that container ":". As a result
strings like "prefix:1.2" which would have been quoted as "prefix\:1.2"
end up becoming "prefix" instead causing loss of information and
incorrect CPEs being generated. As a result in such cases, we remove out
strings containing ":" in any part entirely for now. This is similar
to the way we were handling CPE filtering in the past with http urls as
vendor strings
3. Add special handling for version which contain ":" due to epochs in
debian and rpm. In this case, we strip out the parts before ":" i.e.
the epoch and only output the actual function. This ensures we are not
discarding valid version strings due to pt #.2.
In the future we should look at moving to a more spec compliant cpe
parsing library to avoid such shenanigans.
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Remove WFNize for input strings
WFNize seems to not be part of the standard as per
https://pkg.go.dev/github.com/facebookincubator/nvdtools@v0.1.4/wfn#WFNize
and seems to have bugs/issues with encode/decode cycles, so I am
just removing it at this point and relying on the CPE regex to filter
out invalid CPEs for now.
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Quote the string on decode to ensure consistent CPE string generation
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add test cases for round-tripping the CPE and fix strip slashes
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add comprehensive tests for cpe parsing
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Use strings.Builder instead of byte buffer
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* add lpkg support to java cataloger
linter clean up
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix comment formatting
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add filename test for lpkg
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* commment on lpkg file extension tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix comment typo
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix import format
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* simpler test validation
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add direct_url.json fields to python metadata
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* rename DirectURLOrigin struct; add stub for file
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* add detection for direct_url.json
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* Add tests for direct-url information and add it to the output purl
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Update golden snapshot ids after adding new python package metadata field
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add test names for packageurl tests
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* set package ID in catalogers and improve hashing performance
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update setting ID + tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Add failing test for extra empty lines in manifest
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Handle extra empty lines in Java manifests
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* ignore target link files based on path
log when files are actually indexed
add test for sym link resolution
golang test nits
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* nil catalog should act like an empty catalog
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove dir path filtering in favor of file type filtering
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split out addPathToIndex into specialized functions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add test for nul catalog enumeration
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* conditionally discover MIME types for file based on file resolver index
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* change logging around cataloging
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add tests to cover possible infinite symlink loop for resolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
* Add failing test for missing versions
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Look through all named sections for version
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Consistent installation of yajsv
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Adjust output text for test assertion
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* recover from panics in stdlib binary parsing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add CLI test to cover regression case
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add cataloging within universal binaries
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update json test fixtures
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add comments + correct 32 bit multi arch magic check
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add cyclone json format
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* adapt format to sbom.SBOM structure
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* cycloneDX json output with official lib
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add cycloneDX 1.3 schema output in xml
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix lints errors
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* tidying go mod
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* remove cycloneDX 1.2 format
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* update cycloneDX xml schema
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix cyclone according to schema
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* use RFC 2141 URN form of uuid for serial number
add schema validation for cycloneDX 1.3 JSON output
add yajsv cli for JSON schema validation during tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* tidying go mod up
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go get json schema validator
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* install yajsv without mess with go mod
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* reuse code between cycloneDX json & xml encoders
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add output options for cyclone XML
add bom.json to .gitignore
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add cyclone json format
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* adapt format to sbom.SBOM structure
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* cycloneDX json output with official lib
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add cycloneDX 1.3 schema output in xml
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix lints errors
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* tidying go mod
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* remove cycloneDX 1.2 format
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* update cycloneDX xml schema
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix cyclone according to schema
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* use RFC 2141 URN form of uuid for serial number
add schema validation for cycloneDX 1.3 JSON output
add yajsv cli for JSON schema validation during tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* tidying go mod up
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go get json schema validator
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* install yajsv without mess with go mod
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* reuse code between cycloneDX json & xml encoders
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add output options for cyclone XML
add bom.json to .gitignore
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix cyclone12xml removal
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* feedback changes
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go mod tidy
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix spdx namespace and add scheme range assertions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* validate SPDX document name from source metadata
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* comment why namespace tests only check prefix
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add first-level archive processing when input is a file
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add license exception for github.com/xi2/xz
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* always return cleanup function
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* change source.NewFromFile log entry to warn
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* ensure file source always has cleanup function
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* ensure we are always preferring the unarchive cleanup function for source
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split source.Location and create source.Coordinates for minimal path addressing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* move coordinates into separate file
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Update syft/source/coordinates.go
Co-authored-by: Dan Luhring <luhring@users.noreply.github.com>
* migrate pkg.ID and pkg.Relationship to artifact package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* return relationships from tasks
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix more tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add artifact.Identifiable by Identity() method
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove catalog ID assignment
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust spdx helpers to use copy of packages
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* stabilize package ID relative to encode-decode format cycles
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename Identity() to ID()
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use zero value for nils in ID generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* enable source.Location to be identifiable
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* hoist up package relationship discovery to analysis stage
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update ownership-by-file-overlap relationship description
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add test reminders to put new relationships under test
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust PHP composer.lock parser function to return relationships
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* stop hidding command from help doc
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* do not index irregular files
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix fixture dir and err name
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* improve the description of irregular files
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* explicitly check indexed file name
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add new cyclonedx format object
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove cyclonedx presenter
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove cyclonedx presenter call
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove dependence on golden images for format tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* wire up new formt + rename all-presenters ref
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add CLI test to ensure that all formats can be expressed as report output
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add cyclonedx version and encoding format to package name
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* optionally preserve format snapshot images
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting + text unit tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update build tags, ui support, and stereoscope, and release for windows support
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* show help text when no args are given
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* repurpose the input args validation function
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* ensure app does not check for update in cli tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Don't check the Built-By flag
Signed-off-by: Josh Bressers <josh@bress.net>
* Remove alpine pinning to resolve conflict with main
Signed-off-by: Josh Bressers <josh@bress.net>
* add query by MIME type to source.FileResolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* import stereoscope lib changes to find mime type
- add bin cataloger
- add bin parser
- add mime type go utils
- import new resolver
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* add go std library code to unpack bin
- keep them in their own (original) files
- add note for "this code was copied from"
- comment the lines the required changing
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
* Fix CPE set comparison mismatch
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Add failing test to assert CPE generation excludes URLs
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Add removeByCondition method to fieldCandidateSet
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Prevent invalid CPE values for products and vendors
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Introduce removeWhere and rename filter to condition
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Refactor fieldCandidateSet and condition logic
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Move CPE parsing filter to end of CPE generation
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* add Type conversion to remove strong distro type limit
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* update signatures to be correct variable from os-release
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* consider additional vendor candidates for ruby, python, rpm, npm, and java
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add java pom.xml processing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* allow for downstream transform control in cpe generation processing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* migrate CPE generation logic to dedicated package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split java manifest groupID extraction into two tiers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* extract groupID from pom parent project during CPE generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update java groupID processing tests to cover multi-tier approach
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix constructor names for cpe.fieldCandidate
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename helper function to startsWithTopLevelDomain
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add nil changes for java manifest sections
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update comment to reflect parsing maven files
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split out java description parsing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split out pom parent processing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* simplify vendorsFromGroupIDs and associated tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* simplify test type for vendorsFromGroupIDs
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* copy candidate varidations to new instances
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename CPE generation string util functions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add an explanation around fieldCandidate
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* simplify type for the cpe.fieldCandidateSet
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* make CPE filter function names more readable
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update groupIDsFromJavaManifest to use a guard clause
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* extract groupID extraction from artifactID fields into a separate function
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bump goreleaser version to combat failure
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust CPE specificity sorting to include field length and bias certain fields
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove * vendor values from CPE generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* re-enable generating CPEs for jenkins and jira plugins
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* improve CPE generation logic based on java artifactID and groupID
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add ruby-lang as target software candidate for gems in CPE generation logic
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename filterCpes to filterCPEs
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* refactor CPE filters and groupID processing (for linting)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use ruby-lang as vendor candidate not target software
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* address PR comments for CPE generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update SPDX license list from 3.13 to 3.14
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove license list version from spdx snapshot unit tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* change directory resolver to ignore system runtime paths + drive by index
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add event/etui support for filesystem indexing (for dir resolver)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add warnings for path indexing problems
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add directory resolver index tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* improve testing around directory resolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* renamed p var to path when not conflicting with import
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* pull docker image in CLI dir scan timeout test
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* ensure file not exist errors do not stop directory resolver indexing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split UI from event handling
Signed-off-by: Alex Goodman <wagoodman@gmail.com>
* add event loop tests
Signed-off-by: Alex Goodman <wagoodman@gmail.com>
* use stereoscope cleanup function during signal handling
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* correct error wrapping in packages cmd
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* migrate ui event handlers to ui package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* clarify command worker input var + remove dead comments
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Allow registry auth config without authority value
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Update CLI tests for new stereoscope log output
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* enhance cpe generation for group id and filtering
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename group id const + add doc comment for HasAnyOfPrefixes
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add registry image source
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use explicit source for fetching image + add scheme and registry tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust test variable name and add credential helper function
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add initial secrets cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update ETUI elements with new catalogers (file metadata, digests, and secrets)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update secrets cataloger to read full contents into memory for searching
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* quick prototype of parallelization secret regex search
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* quick prototype with single aggregated regex
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* quick prototype for secret search line-by-line
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* quick prototype hybrid secrets search
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add secrets cataloger with line strategy
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust verbiage towards SearchResults instead of Secrets + add tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update json schema with secrets cataloger results
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* address PR comments
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update readme with secrets config options
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* ensure file catalogers call AllLocations once
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Also adds artifact location to sort key for Sorted() to ensure
consistent sorts when artifacts of same name, version, and type are
found in different locations in the image. Location should be sufficient
since we assume only one package of a given name and version can exist
in one location, even if that location is an package-db like rpmdb.
Signed-off-by: Zach Hill <zach@anchore.com>
* Create independent build targets for Mac and Linux
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Create targets for macOS signing and notarization
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Create target for Linux packaging
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Update release workflow and leverage new make targets
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Add release assets to release draft
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Add homebrew formula release follow-up and improve Makefile
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Add follow-up workflow for updating version check file
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Get rid of fetch depth 0 for checkout action
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Add follow-up workflow for Docker images
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Restore wait-for-checks job
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Replace make functions with shell functions
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Account for envsubst command in bootstrap-ci-linux
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* move homebrew generation into script
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add release approval step; remove goreleaser; add docker image smoke testing in acceptance step
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* replace homebrew formula template file with heredoc template
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update release documentation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
Some debian-based variants (such as Google's Distroless images)
don't write a single file to `/var/lib/dpkg/status`, but rather write
a file per package to `/var/lib/dpkg/status.d/`
related to #44
Signed-off-by: Weston Steimel <weston.steimel@gmail.com>
* add marking package relations by file ownership
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* correct json schema version; ensure fileOwners dont return dups; pin test pkg versions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* extract package relationships into separate section
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* pull in client-go features for import of PackageRelationships
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* move unit test for ownership by files relationship further down
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename relationship to "ownership-by-file-overlap"
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
printf"WARNING! It looks like this isn't the CI environment. This script modifies the macOS Keychain setup in ways you probably wouldn't want for your own machine. It also requires an Apple Developer ID Certificate that you shouldn't have outside of the CI environment.\n\nExiting early to make sure nothing bad happens.\n"
exit1
fi
# Install gon (see https://github.com/mitchellh/gon for details).
brew tap mitchellh/gon
brew install mitchellh/gon/gon
# Write signing certificate to disk from environment variable.
# In order to have all keychain interactions avoid an interactive user prompt, we need to control the password for the keychain in question, which means we need to create a new keychain into which we'll import the signing certificate and from which we'll later access this certificate during code signing.
# Setting the partition list for this certificate's private key to include "apple-tool:" and "apple:" allows the codesign command to access this keychain item without an interactive user prompt. (codesign is invoked by gon.)
runs-on:macos-latest# We're creating these snapshot builds on macOS to be consistent with our release workflow's build process, which also takes place on macOS (due to code signing requirements).
- format:zip# This is a hack! We don't actually intend to use _this_ ZIP file, we just need goreleaser to consider the ZIP file produced by gon (which will have the same file name) to be an artifact so we can use it downstream in publishing (e.g. to a homebrew tap)
id:syft-zip
builds:
- syft-macos
- linux-build
signs:
- artifacts:checksum
cmd:sh
args:
- '-c'
# we should not include the zip artifact, as the artifact is mutated throughout the next macOS notarization step
signature:"syft_${VERSION}_darwin_amd64.dmg"# This is somewhat unintuitive. This gets the DMG file recognized as an artifact. In fact, both a DMG and a ZIP file are being produced by this signing step.
args:
- "{{ .IsSnapshot }}"
- "gon.hcl"
- "./dist/syft_{{ .Version }}_darwin_amd64"
artifacts:all
# note: the signing process is depending on tar.gz archives. If this format changes then .github/scripts/apple-signing/*.sh will need to be adjusted
If you are looking to contribute to this project and want to open a Github pull request ("PR"), there are a few guidelines of what we are looking for in patches. Make sure you go through this document and ensure that your code proposal is aligned.
If you are looking to contribute to this project and want to open a GitHub pull request ("PR"), there are a few guidelines of what we are looking for in patches. Make sure you go through this document and ensure that your code proposal is aligned.
## Sign your work
## Setting up your environment
Before you can contribute to Syft, you need to configure your development environment.
### Debian setup
You will need to install Go. The version on https://go.dev works best, using the system golang doesn't always work the way you might expect.
Refer to the go.mod file in the root of this repo for the recommended version of Go to install.
You will also need Docker. There's no reason the system packages shouldn't work, but we used the official Docker package. You can find instructions for installing Docker in Debian [here](https://docs.docker.com/engine/install/debian/).
You also need to install some Debian packages
```sh
sudo apt-get install build-essential zip bc libxml2-utils git
```
## Configuring Git
You will need to configure your git client with your name and email address. This is easily done from the command line.
This username and email address will matter later in this guide.
## Fork the repo
You should fork the Syft repo using the "Fork" button at the top right of the Syft GitHub [site](https://github.com/anchore/syft/). You will be doing your development in your fork, then submit a pull request to Syft. There are many resources how to use GitHub effectively, we will not cover those here.
## Adding a feature or fix
If you look at the Syft [Issue](https://github.com/anchore/syft/issues) there are plenty of bugs and feature requests. Maybe look at the [good first issue](https://github.com/anchore/syft/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22) list if you're not sure where to start.
## Commit guidelines
In the Syft project we like commits and pull requests (PR) to be easy to understand and review. Open source thrives best when everything happening is over documented and small enough to be understood.
### Granular commits
Please try to make every commit as simple as possible, but no simpler. The idea is that each commit should be a logical unit of code. Try not to commit too many tiny changes, for example every line changed in a file as a separate commit. And also try not to make a commit enormous, for example committing all your work at the end of the day.
Rather than try to follow a strict guide on what is or is not best, we try to be flexible and simple in this space. Do what makes the most sense for the changes you are trying to include.
### Commit title and description
Remember that the message you leave for a commit is for the reviewer in the present, and for someone (maybe you) changing something in the future. Please make sure the title and description used is easy to understand and explains what was done. Jokes and clever comments generally don't age well in commit messages. Just the facts please.
## Sign off your work
The `sign-off` is an added line at the end of the explanation for the commit, certifying that you wrote it or otherwise have the right to submit it as an open-source patch. By submitting a contribution, you agree to be bound by the terms of the DCO Version 1.1 and Apache License Version 2.0.
Signing a commit certifies the below Developer's Certificate of Origin (DCO):
Signing off a commit certifies the below Developer's Certificate of Origin (DCO):
```text
Developer's Certificate of Origin 1.1
@@ -44,20 +95,13 @@ When committing your change, you can add the required line manually so that it l
Signed-off-by: John Doe <john.doe@example.com>
```
Alternatively, configure your Git client with your name and email to use the`-s`flag when creating a commit:
[//]: # (TODO: Commit guidelines, commit title, extra body description)
[//]: # (TODO: PR title and description)
## Test your changes
This project has a `Makefile` which includes many helpers running both unit and integration tests. Although PRs will have automatic checks for these, it is useful to run them locally, ensuring they pass before submitting changes. Ensure you've bootstrapped once before running tests:
This project has a `Makefile` which includes many helpers running both unit and integration tests. You can run `make help` to see all the options. Although PRs will have automatic checks for these, it is useful to run them locally, ensuring they pass before submitting changes. Ensure you've bootstrapped once before running tests:
```text
$ make bootstrap
@@ -98,6 +129,26 @@ $ make unit
$ make integration
```
You can also run `make all` to run a more extensive test suite, but there is additional configuration that will be needed for those tests to run correctly. We will not cover the extra steps here.
## Pull Request
If you made it this far and all the tests are passing, it's time to submit a Pull Request (PR) for Syft. Submitting a PR is always a scary moment as what happens next can be an unknown. The Syft project strives to be easy to work with, we appreciate all contributions. Nobody is going to yell at you or try to make you feel bad. We love contributions and know how scary that first PR can be.
### PR Title and Description
Just like the commit title and description mentioned above, the PR title and description is very important for letting others know what's happening. Please include any details you think a reviewer will need to more properly review your PR.
A PR that is very large or poorly described has a higher likelihood of being pushed to the end of the list. Reviewers like PRs they can understand and quickly review.
### What to expect next
Please be patient with the project. We try to review PRs in a timely manner, but this is highly dependent on all the other tasks we have going on. It's OK to ask for a status update every week or two, it's not OK to ask for a status update every day.
It's very likely the reviewer will have questions and suggestions for changes to your PR. If your changes don't match the current style and flow of the other code, expect a request to change what you've done.
## Document your changes
When proposed changes are modifying user-facing functionality or output, it is expected the PR will include updates to the documentation as well.
And lastly, when proposed changes are modifying user-facing functionality or output, it is expected the PR will include updates to the documentation as well. Syft is not a project that is heavy on documentation. This will mostly be updating the README and help for the tool.
If nobody knows new features exist, they can't use them!
In order to test and develop in this repo you will need the following dependencies installed:
- Golang
- docker
- make
- Python (>= 3.9)
### Docker settings for getting started
Make sure you've updated your docker settings so the default docker socket path is available.
Go to:
docker -> settings -> advanced
Make sure:
```
Allow the default Docker socket to be used
```
is checked.
Also double check that the docker context being used is the default context. If it is not, run:
`docker context use default`
After cloning the following step can help you get setup:
1. run `make bootstrap` to download go mod dependencies, create the `/.tmp` dir, and download helper utilities.
2. run `make` to view the selection of developer commands in the Makefile
3. run `make build` to build the release snapshot binaries and packages
4. for an even quicker start you can run `go run cmd/syft/main.go` to print the syft help.
- this command `go run cmd/syft/main.go alpine:latest` will compile and run syft against `alpine:latest`
5. view the README or syft help output for more output options
The main make tasks for common static analysis and testing are `lint`, `format`, `lint-fix`, `unit`, `integration`, and `cli`.
See `make help` for all the current make tasks.
### Internal Artifactory Settings
**Not always applicable**
Some companies have Artifactory setup internally as a solution for sourcing secure dependencies.
If you're seeing an issue where the unit tests won't run because of the below error then this section might be relevant for your use case.
```
[ERROR] [ERROR] Some problems were encountered while processing the POMs
```
If you're dealing with an issue where the unit tests will not pull/build certain java fixtures check some of these settings:
- a `settings.xml` file should be available to help you communicate with your internal artifactory deployment
- this can be moved to `syft/pkg/cataloger/java/test-fixtures/java-builds/example-jenkins-plugin/` to help build the unit test-fixtures
- you'll also want to modify the `build-example-jenkins-plugin.sh` to use `settings.xml`
For more information on this setup and troubleshooting see [issue 1895](https://github.com/anchore/syft/issues/1895#issuecomment-1610085319)
## Architecture
At a high level, this is the package structure of syft:
```
./cmd/syft/
│ ├── cli/
│ │ ├── cli.go // where all commands are wired up
│ │ ├── commands/ // all command implementations
│ │ ├── options/ // all command flags and configuration options
│ │ └── ui/ // all handlers for events that are shown on the UI
│ └── main.go // entrypoint for the application
└── syft/ // the "core" syft library
├── format/ // contains code to encode or decode to and from SBOM formats
├── pkg/ // contains code to catalog packages from a source
├── sbom/ // contains the definition of an SBOM
└── source/ // contains code to create a source object for some input type (e.g. container image, directory, etc)
```
Syft's core library is implemented in the `syft` package and subpackages, where the major packages are:
- the `syft/source` package produces a `source.Source` object that can be used to catalog a directory, container, and other source types.
- the `syft` package contains a single function that can take a `source.Source` object and catalog it, producing an `sbom.SBOM` object
- the `syft/format` package contains the ability to encode and decode SBOMs to and from different SBOM formats (such as SPDX and CycloneDX)
The `cmd` pacakge at the highest level execution flow wires up [`spf13/cobra`](https://github.com/spf13/cobra) commands for execution in the main application:
```mermaid
sequenceDiagram
participant main as cmd/syft/main
participant cli as cli.New()
participant root as root.Execute()
participant cmd as <command>.Execute()
main->>+cli:
Note right of cli: wire ALL CLI commands
Note right of cli: add flags for ALL commands
cli-->>-main: root command
main->>+root:
root->>+cmd:
cmd-->>-root: (error)
root-->>-main: (error)
Note right of cmd: Execute SINGLE command from USER
```
The `packages` command uses the core library to generate an SBOM for the given user input:
```mermaid
sequenceDiagram
participant source as source.New(ubuntu:latest)
participant sbom as sbom.SBOM
participant catalog as syft.CatalogPackages(src)
participant encoder as syft.Encode(sbom, format)
Note right of source: use "ubuntu:latest" as SBOM input
source-->>+sbom: add source to SBOM struct
source-->>+catalog: pass src to generate catalog
catalog-->-sbom: add cataloging results onto SBOM
sbom-->>encoder: pass SBOM and format desiered to syft encoder
encoder-->>source: return bytes that are the SBOM of the original input
Note right of catalog: cataloger configuration is done based on src
```
Additionally, here is a [gist of using syft as a library](https://gist.github.com/spiffcs/3027638b7ba904d07e482a712bc00d3d) to generate a SBOM for a docker image.
### `pkg.Package` object
The `pkg.Package` object is a core data structure that represents a software package. Fields like `name` and `version` probably don't need
a detailed explanation, but some of the other fields are worth a quick overview:
-`FoundBy`: the name of the cataloger that discovered this package (e.g. `python-pip-cataloger`).
-`Locations`: these are the set of paths and layer ids that were parsed to discover this package (e.g. `python-pip-cataloger`).
-`Language`: the language of the package (e.g. `python`).
-`Type`: this is a high-level categorization of the ecosystem the package resides in. For instance, even if the package is a egg, wheel, or requirements.txt reference, it is still logically a "python" package. Not all package types align with a language (e.g. `rpm`) but it is common.
-`Metadata`: specialized data for specific location(s) parsed. We should try and raise up as much raw information that seems useful. As a rule of thumb the object here should be as flat as possible and use the raw names and values from the underlying source material parsed.
When `pkg.Package` is serialized an additional `MetadataType` is shown. This is a label that helps consumers understand the datashape of the `Metadata` field.
By convention the `MetadataType` value should follow these rules of thumb:
- Only use lowercase letters, numbers, and hyphens. Use hyphens to separate words.
- **Try to anchor the name in the ecosystem, language, or packaging tooling it belongs to**. For a package manager for a language ecosystem the language, framework or runtime should be used as a prefix. For instance `pubspec-lock` is an OK name, but `dart-pubspec-lock` is better. For an OS package manager this is not necessary (e.g. `apk-db-entry` is a good name, but `alpine-apk-db-entry` is not since `alpine` and the `a` in `apk` is redundant).
- **Be as specific as possible to what the data represents**. For instance `ruby-gem` is NOT a good `MetadataType` value, but `ruby-gemspec` is. Why? Ruby gem information can come from a gemspec file or a Gemfile.lock, which are very different. The latter name provides more context as to what to expect.
- **Should describe WHAT the data is, NOT HOW it's used**. For instance `r-description-installed-file` is NOT a good `MetadataType` value since it's trying to convey that we use the DESCRIPTION file in the R ecosystem to detect installed packages. Instead simply describe what the DESCRIPTION file is itself without context of how it's used: `r-description`.
- **Use the `lock` suffix** to distinct between manifest files that loosely describe package version requirements vs files that strongly specify one and only one version of a package ("lock" files). These should only be used with respect to package managers that have the guide and lock distinction, but would not be appropriate otherwise (e.g. `rpm` does not have a guide vs lock, so `lock` should NOT be used to describe a db entry).
- **Use the `archive` suffix to indicate a package archive** (e.g. rpm file, apk file, etc) that describes the contents of the package. For example an RPM file that was cataloged would have a `rpm-archive` metadata type (not to be confused with an RPM DB record entry which would be `rpm-db-entry`).
- **Use the `entry` suffix** to indicate information about a package that was found as a single entry within file that has multiple package entries. If the entry was found within a DB or a flat-file store for an OS package manager, you should use `db-entry`.
- **Should NOT contain the phrase `package`**, though exceptions are allowed (say if the canonical name literally has the phrase package in it).
- **Should NOT contain have a `file` suffix** unless the canonical name has the term "file", such as a `pipfile` or `gemfile`. An example of a bad name for this rule is`ruby-gemspec-file`; a better name would be `ruby-gemspec`.
- **Should NOT contain the exact filename+extensions**. For instance `pipfile.lock` shouldn't really be in the name, instead try and describe what the file is: `python-pipfile-lock` (but shouldn't this be `python-pip-lock` you might ask? No, since the `pip` package manger is not related to the `pipfile` project).
- **Should NOT contain the phrase `metadata`**, unless the canonical name has this term.
- **Should represent a single use case**. For example, trying to describe Hackage metadata with a single `HackageMetadata` struct (and thus `MetadataType`) is not allowed since it represents 3 mutually exclusive use cases: representing a `stack.yaml`, `stack.lock`, or `cabal.project` file. Instead, each of these should have their own struct types and `MetadataType` values.
There are other cases that are not covered by these rules... and that's ok! The goal is to provide a consistent naming scheme that is easy to understand and use when it's applicable. If the rules do not exactly apply in your situation then just use your best judgement (or amend these rules as needed whe new common cases come up).
What if the underlying parsed data represents multiple files? There are two approaches to this:
- use the primary file to represent all the data. For instance, though the `dpkg-cataloger` looks at multiple files to get all information about a package, it's the `status` file that gets represented.
- nest each individual file's data under the `Metadata` field. For instance, the `java-archive-cataloger` may find information from on or all of the files: `pom.xml`, `pom.properties`, and `MANIFEST.MF`. However, the metadata is simply `java-metadata' with each possibility as a nested optional field.
### Syft Catalogers
Catalogers are the way in which syft is able to identify and construct packages given a set a targeted list of files.
For example, a cataloger can ask syft for all `package-lock.json` files in order to parse and raise up javascript packages
(see [how file globs](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/javascript/cataloger.go#L16-L21) and
[file parser functions](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/javascript/cataloger.go#L16-L21) are used
for a quick example).
From a high level catalogers have the following properties:
- _They are independent from one another_. The java cataloger has no idea of the processes, assumptions, or results of the python cataloger, for example.
- _They do not know what source is being analyzed_. Are we analyzing a local directory? an image? if so, the squashed representation or all layers? The catalogers do not know the answers to these questions. Only that there is an interface to query for file paths and contents from an underlying "source" being scanned.
- _Packages created by the cataloger should not be mutated after they are created_. There is one exception made for adding CPEs to a package after the cataloging phase, but that will most likely be moved back into the cataloger in the future.
Cataloger names should be unique and named with the following rules of thumb in mind:
- Must end with `-cataloger`
- Use lowercase letters, numbers, and hyphens only
- Use hyphens to separate words
- Catalogers for language ecosystems should start with the language name (e.g. `python-` for a cataloger that raises up python packages)
- Distinct between when the cataloger is searching for evidence of installed packages vs declared packages. For example, there are currently two different gemspec-based catalogers, the `ruby-gemspec-cataloger` and `ruby-installed-gemspec-cataloger`, where the latter requires that the gemspec is found within a `specifications` directory (which means it was installed, not just at the root of a source repo).
#### Building a new Cataloger
Catalogers must fulfill the [`pkg.Cataloger` interface](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger.go) in order to add packages to the SBOM.
All catalogers should be added to:
- the [global list of catalogers](https://github.com/anchore/syft/blob/9995950c70e849f9921919faffbfcf46401f71f3/syft/pkg/cataloger/cataloger.go#L92-L125)
- at least one source-specific list, today the two lists are [directory catalogers and image catalogers](https://github.com/anchore/syft/blob/9995950c70e849f9921919faffbfcf46401f71f3/syft/pkg/cataloger/cataloger.go#L39-L89)
For reference, catalogers are [invoked within syft](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/catalog.go#L41-L100) one after the other, and can be invoked in parallel.
`generic.NewCataloger` is an abstraction syft used to make writing common components easier (see the [apkdb cataloger](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/apkdb/cataloger.go) for example usage).
It takes the following information as input:
- A `catalogerName` to identify the cataloger uniquely among all other catalogers.
- Pairs of file globs as well as parser functions to parse those files. These parser functions return a slice of [`pkg.Package`](https://github.com/anchore/syft/blob/9995950c70e849f9921919faffbfcf46401f71f3/syft/pkg/package.go#L19) as well as a slice of [`artifact.Relationship`](https://github.com/anchore/syft/blob/9995950c70e849f9921919faffbfcf46401f71f3/syft/artifact/relationship.go#L31) to describe how the returned packages are related. See this [the apkdb cataloger parser function](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/apkdb/parse_apk_db.go#L22-L102) as an example.
Identified packages share a common `pkg.Package` struct so be sure that when the new cataloger is constructing a new package it is using the [`Package` struct](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/package.go#L16-L31).
If you want to return more information than what is available on the `pkg.Package` struct then you can do so in the `pkg.Package.Metadata` section of the struct, which is unique for each [`pkg.Type`](https://github.com/anchore/syft/blob/v0.70.0/syft/pkg/type.go).
See [the `pkg` package](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg) for examples of the different metadata types that are supported today.
These are plugged into the `MetadataType` and `Metadata` fields in the above struct. `MetadataType` informs which type is being used. `Metadata` is an interface converted to that type.
Finally, here is an example of where the package construction is done within the apk cataloger:
- [Calling the APK package constructor from the parser function](https://github.com/anchore/syft/blob/v0.70.0/syft/pkg/cataloger/apkdb/parse_apk_db.go#L106)
Interested in building a new cataloger? Checkout the [list of issues with the `new-cataloger` label](https://github.com/anchore/syft/issues?q=is%3Aopen+is%3Aissue+label%3Anew-cataloger+no%3Aassignee)!
If you have questions about implementing a cataloger feel free to file an issue or reach out to us [on slack](https://anchore.com/slack)!
#### Searching for files
All catalogers are provided an instance of the [`file.Resolver`](https://github.com/anchore/syft/blob/v0.70.0/syft/source/file_resolver.go#L8) to interface with the image and search for files. The implementations for these
abstractions leverage [`stereoscope`](https://github.com/anchore/stereoscope) in order to perform searching. Here is a
rough outline how that works:
1. a stereoscope `file.Index` is searched based on the input given (a path, glob, or MIME type). The index is relatively fast to search, but requires results to be filtered down to the files that exist in the specific layer(s) of interest. This is done automatically by the `filetree.Searcher` abstraction. This abstraction will fallback to searching directly against the raw `filetree.FileTree` if the index does not contain the file(s) of interest. Note: the `filetree.Searcher` is used by the `file.Resolver` abstraction.
2. Once the set of files are returned from the `filetree.Searcher` the results are filtered down further to return the most unique file results. For example, you may have requested for files by a glob that returns multiple results. These results are filtered down to deduplicate by real files, so if a result contains two references to the same file, say one accessed via symlink and one accessed via the real path, then the real path reference is returned and the symlink reference is filtered out. If both were accessed by symlink then the first (by lexical order) is returned. This is done automatically by the `file.Resolver` abstraction.
3. By the time results reach the `pkg.Cataloger` you are guaranteed to have a set of unique files that exist in the layer(s) of interest (relative to what the resolver supports).
## Testing
### Levels of testing
-`unit`: The default level of test which is distributed throughout the repo are unit tests. Any `_test.go` file that
does not reside somewhere within the `/test` directory is a unit test. Other forms of testing should be organized in
the `/test` directory. These tests should focus on correctness of functionality in depth. % test coverage metrics
only considers unit tests and no other forms of testing.
-`integration`: located within `test/integration`, these tests focus on the behavior surfaced by the common library
entrypoints from the `syft` package and make light assertions about the results surfaced. Additionally, these tests
tend to make diversity assertions for enum-like objects, ensuring that as enum values are added to a definition
that integration tests will automatically fail if no test attempts to use that enum value. For more details see
the "Data diversity and freshness assertions" section below.
-`cli`: located with in `test/cli`, these are tests that test the correctness of application behavior from a
snapshot build. This should be used in cases where a unit or integration test will not do or if you are looking
for in-depth testing of code in the `cmd/` package (such as testing the proper behavior of application configuration,
CLI switches, and glue code before syft library calls).
-`acceptance`: located within `test/compare` and `test/install`, these are smoke-like tests that ensure that application
packaging and installation works as expected. For example, during release we provide RPM packages as a download
artifact. We also have an accompanying RPM acceptance test that installs the RPM from a snapshot build and ensures the
output of a syft invocation matches canned expected output. New acceptance tests should be added for each release artifact
and architecture supported (when possible).
### Data diversity and freshness assertions
It is important that tests against the codebase are flexible enough to begin failing when they do not cover "enough"
of the objects under test. "Cover" in this case does not mean that some percentage of the code has been executed
during testing, but instead that there is enough diversity of data input reflected in testing relative to the
definitions available.
For instance, consider an enum-like value like so:
```go
typeLanguagestring
const(
JavaLanguage="java"
JavaScriptLanguage="javascript"
PythonLanguage="python"
RubyLanguage="ruby"
GoLanguage="go"
)
```
Say we have a test that exercises all the languages defined today:
```go
funcTestCatalogPackages(t*testing.T){
testTable:=[]struct{
// ... the set of test cases that test all languages
}
for_,test:=rangecases{
t.Run(test.name,func(t*testing.T){
// use inputFixturePath and assert that syft.CatalogPackages() returns the set of expected Package objects
// ...
})
}
}
```
Where each test case has a `inputFixturePath` that would result with packages from each language. This test is
brittle since it does not assert that all languages were exercised directly and future modifications (such as
adding a new language) won't be covered by any test cases.
To address this the enum-like object should have a definition of all objects that can be used in testing:
```go
typeLanguagestring
// const( Java Language = ..., ... )
varAllLanguages=[]Language{
Java,
JavaScript,
Python,
Ruby,
Go,
Rust,
}
```
Allowing testing to automatically fail when adding a new language:
```go
funcTestCatalogPackages(t*testing.T){
testTable:=[]struct{
// ... the set of test cases that (hopefully) covers all languages
}
// new stuff...
observedLanguages:=strset.New()
for_,test:=rangecases{
t.Run(test.name,func(t*testing.T){
// use inputFixturePath and assert that syft.CatalogPackages() returns the set of expected Package objects
// ...
// new stuff...
for_,actualPkg:=rangeactual{
observedLanguages.Add(string(actualPkg.Language))
}
})
}
// new stuff...
for_,expectedLanguage:=rangepkg.AllLanguages{
if!observedLanguages.Contains(expectedLanguage){
t.Errorf("failed to test language=%q",expectedLanguage)
}
}
}
```
This is a better test since it will fail when someone adds a new language but fails to write a test case that should
exercise that new language. This method is ideal for integration-level testing, where testing correctness in depth
is not needed (that is what unit tests are for) but instead testing in breadth to ensure that units are well integrated.
A similar case can be made for data freshness; if the quality of the results will be diminished if the input data
is not kept up to date then a test should be written (when possible) to assert any input data is not stale.
An example of this is the static list of licenses that is stored in `internal/spdxlicense` for use by the SPDX
presenters. This list is updated and published periodically by an external group and syft can grab and update this
list by running `go generate ./...` from the root of the repo.
An integration test has been written to grabs the latest license list version externally and compares that version
with the version generated in the codebase. If they differ, the test fails, indicating to someone that there is an
action needed to update it.
**_The key takeaway is to try and write tests that fail when data assumptions change and not just when code changes.**_
### Snapshot tests
The format objects make a lot of use of "snapshot" testing, where you save the expected output bytes from a call into the
git repository and during testing make a comparison of the actual bytes from the subject under test with the golden
copy saved in the repo. The "golden" files are stored in the `test-fixtures/snapshot` directory relative to the go
package under test and should always be updated by invoking `go test` on the specific test file with a specific CLI
update flag provided.
Many of the `Format` tests make use of this approach, where the raw SBOM report is saved in the repo and the test
compares that SBOM with what is generated from the latest presenter code. For instance, at the time of this writing
the CycloneDX presenter snapshots can be updated by running:
```bash
go test ./internal/formats -update-cyclonedx
```
These flags are defined at the top of the test files that have tests that use the snapshot files.
Snapshot testing is only as good as the manual verification of the golden snapshot file saved to the repo! Be careful
# note: we cannot clean the snapshot directory since the pipeline builds the snapshot separately
.PHONY:acceptance-mac
acceptance-mac:$(SNAPSHOTDIR)## Run acceptance tests on build snapshot binaries and packages (Mac)
$(call title,Running acceptance test: Run on Mac)
$(ACC_DIR)/mac.sh \
$(SNAPSHOTDIR)\
$(ACC_DIR)\
$(ACC_TEST_IMAGE)\
$(RESULTSDIR)
# note: we cannot clean the snapshot directory since the pipeline builds the snapshot separately
.PHONY:acceptance-linux
acceptance-linux:acceptance-test-deb-package-installacceptance-test-rpm-package-install## Run acceptance tests on build snapshot binaries and packages (Linux)
# note: this is used by CI to determine if the inline-scan report cache should be busted for the inline-compare tests
A CLI tool and go library for generating a Software Bill of Materials (SBOM) from container images and filesystems.
A CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner like [Grype](https://github.com/anchore/grype).
-Catalog container images and filesystems to discover packages and libraries.
- Supports packages and libraries from various ecosystems (APK, DEB, RPM, Ruby Bundles, Python Wheel/Egg/requirements.txt, JavaScript NPM/Yarn, Java JAR/EAR/WAR, Jenkins plugins JPI/HPI, Go modules)
- Linux distribution identification (supports Alpine, BusyBox, CentOS/RedHat, Debian/Ubuntu flavored distributions)
-Supports Docker and OCI image formats
## Features
-Generates SBOMs for container images, filesystems, archives, and more to discover packages and libraries
- Supports OCI, Docker and [Singularity](https://github.com/sylabs/singularity) image formats
- Linux distribution identification
-Works seamlessly with [Grype](https://github.com/anchore/grype) (a fast, modern vulnerability scanner)
- Able to create signed SBOM attestations using the [in-toto specification](https://github.com/in-toto/attestation/blob/main/spec/README.md)
- Convert between SBOM formats, such as CycloneDX, SPDX, and Syft's own format.
If you encounter an issue, please [let us know using the issue tracker](https://github.com/anchore/syft/issues).
### Supported Ecosystems
- Alpine (apk)
- C (conan)
- C++ (conan)
- Dart (pubs)
- Debian (dpkg)
- Dotnet (deps.json)
- Objective-C (cocoapods)
- Elixir (mix)
- Erlang (rebar3)
- Go (go.mod, Go binaries)
- Haskell (cabal, stack)
- Java (jar, ear, war, par, sar, nar, native-image)
- JavaScript (npm, yarn)
- Jenkins Plugins (jpi, hpi)
- Linux kernel archives (vmlinz)
- Linux kernel modules (ko)
- Nix (outputs in /nix/store)
- PHP (composer)
- Python (wheel, egg, poetry, requirements.txt)
- Red Hat (rpm)
- Ruby (gem)
- Rust (cargo.lock)
- Swift (cocoapods, swift-package-manager)
## Installation
**Note**: Currently, Syft is built only for Linux, macOS and Windows.
### Recommended
```bash
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
```
... or, you can specify a release version and destination directory for the installation:
```
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b <DESTINATION_DIR> <RELEASE_VERSION>
```
### Chocolatey
The chocolatey distribution of syft is community maintained and not distributed by the anchore team
```powershell
chocoinstallsyft-y
```
### Scoop
```powershell
scoopinstallsyft
```
### Homebrew
```bash
brew install syft
```
### Nix
**Note**: Nix packaging of Syft is [community maintained](https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/syft/default.nix). Syft is available in the [stable channel](https://nixos.wiki/wiki/Nix_channels#The_official_channels) since NixOS `22.05`.
```bash
nix-env -i syft
```
... or, just try it out in an ephemeral nix shell:
```bash
nix-shell -p syft
```
## Getting started
To generate an SBOM for a Docker or OCI image:
### SBOM
To generate an SBOM for a container image:
```
syft <image>
```
The above output includes only software that is visible in the container (i.e., the squashed representation of the image).
To include software from all image layers in the SBOM, regardless of its presence in the final image, provide `--scope all-layers`:
The above output includes only software that is visible in the container (i.e., the squashed representation of the image). To include software from all image layers in the SBOM, regardless of its presence in the final image, provide `--scope all-layers`:
```
syft <image> --scope all-layers
```
### Supported sources
Syft can generate a SBOM from a variety of sources:
```
# catalog a container image archive (from the result of `docker image save ...`, `podman save ...`, or `skopeo copy` commands)
syft path/to/image.tar
# catalog a Singularity Image Format (SIF) container
syft path/to/image.sif
# catalog a directory
syft path/to/dir
```
The output format for Syft is configurable as well:
Sources can be explicitly provided with a scheme:
```
docker:yourrepo/yourimage:tag use images from the Docker daemon
podman:yourrepo/yourimage:tag use images from the Podman daemon
containerd:yourrepo/yourimage:tag use images from the Containerd daemon
docker-archive:path/to/yourimage.tar use a tarball from disk for archives created from "docker save"
oci-archive:path/to/yourimage.tar use a tarball from disk for OCI archives (from Skopeo or otherwise)
oci-dir:path/to/yourimage read directly from a path on disk for OCI layout directories (from Skopeo or otherwise)
singularity:path/to/yourimage.sif read directly from a Singularity Image Format (SIF) container on disk
dir:path/to/yourproject read directly from a path on disk (any directory)
file:path/to/yourproject/file read directly from a path on disk (any single file)
registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
```
If an image source is not provided and cannot be detected from the given reference it is assumed the image should be pulled from the Docker daemon.
If docker is not present, then the Podman daemon is attempted next, followed by reaching out directly to the image registry last.
This default behavior can be overridden with the `default-image-pull-source` configuration option (See [Configuration](https://github.com/anchore/syft#configuration) for more details).
### Default Cataloger Configuration by scan type
Syft uses different default sets of catalogers depending on what it is scanning: a container image or a directory on disk. The default catalogers for an image scan assumes that package installation steps have already been completed. For example, Syft will identify Python packages that have egg or wheel metadata files under a site-packages directory, since this indicates software actually installed on an image.
However, if you are scanning a directory, Syft doesn't assume that all relevant software is installed, and will use catalogers that can identify declared dependencies that may not yet be installed on the final system: for example, dependencies listed in a Python requirements.txt.
You can override the list of enabled/disabled catalogers by using the "catalogers" keyword in the [Syft configuration file](https://github.com/anchore/syft#configuration).
##### Image Scanning:
- alpmdb
- apkdb
- binary
- dotnet-deps
- dotnet-portable-executable
- dpkgdb
- go-module-binary
- graalvm-native-image
- java
- javascript-package
- linux-kernel
- nix-store
- php-composer-installed
- portage
- python-package
- rpm-db
- ruby-gemspec
- sbom
##### Directory Scanning:
- alpmdb
- apkdb
- binary
- cocoapods
- conan
- dartlang-lock
- dotnet-deps
- dotnet-portable-executable
- dpkgdb
- elixir-mix-lock
- erlang-rebar-lock
- go-mod-file
- go-module-binary
- graalvm-native-image
- haskell
- java
- java-gradle-lockfile
- java-pom
- javascript-lock
- linux-kernel
- nix-store
- php-composer-lock
- portage
- python-index
- python-package
- rpm-db
- rpm-file
- ruby-gemfile
- rust-cargo-lock
- sbom
- swift-package-manager
##### Non Default:
- cargo-auditable-binary
### Excluding file paths
Syft can exclude files and paths from being scanned within a source by using glob expressions
**Note:** in the case of _image scanning_, since the entire filesystem is scanned it is
possible to use absolute paths like `/etc` or `/usr/**/*.txt` whereas _directory scans_
exclude files _relative to the specified directory_. For example: scanning `/usr/foo` with
`--exclude ./package.json` would exclude `/usr/foo/package.json` and `--exclude '**/package.json'`
would exclude all `package.json` files under `/usr/foo`. For _directory scans_,
it is required to begin path expressions with `./`, `*/`, or `**/`, all of which
will be resolved _relative to the specified scan directory_. Keep in mind, your shell
may attempt to expand wildcards, so put those parameters in single quotes, like:
`'**/*.json'`.
### Output formats
The output format for Syft is configurable as well using the
`-o` (or `--output`) option:
```
syft <image> -o <format>
```
Where the `format`s available are:
-`json`: Use this to get as much information out of Syft as possible!
-`text`: A row-oriented, human-and-machine-friendly output.
-`cyclonedx`: A XML report conforming to the [CycloneDX 1.2](https://cyclonedx.org/) specification.
-`table`: A columnar summary (default).
Where the `formats` available are:
-`syft-json`: Use this to get as much information out of Syft as possible!
-`syft-text`: A row-oriented, human-and-machine-friendly output.
-`cyclonedx-xml`: A XML report conforming to the [CycloneDX 1.4 specification](https://cyclonedx.org/specification/overview/).
-`cyclonedx-json`: A JSON report conforming to the [CycloneDX 1.4 specification](https://cyclonedx.org/specification/overview/).
-`spdx-tag-value`: A tag-value formatted report conforming to the [SPDX 2.3 specification](https://spdx.github.io/spdx-spec/v2.3/).
-`spdx-tag-value@2.2`: A tag-value formatted report conforming to the [SPDX 2.2 specification](https://spdx.github.io/spdx-spec/v2.2.2/).
-`spdx-json`: A JSON report conforming to the [SPDX 2.3 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.3/schemas/spdx-schema.json).
-`spdx-json@2.2`: A JSON report conforming to the [SPDX 2.2 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.2/schemas/spdx-schema.json).
-`github-json`: A JSON report conforming to GitHub's dependency snapshot format.
-`syft-table`: A columnar summary (default).
-`template`: Lets the user specify the output format. See ["Using templates"](#using-templates) below.
## Installation
## Using templates
**Recommended (macOS and Linux)**
```bash
# install the latest version to /usr/local/bin
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
Syft lets you define custom output formats, using [Go templates](https://pkg.go.dev/text/template). Here's how it works:
# install a specific version into a specific dir
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b <SOME_BIN_PATH> <RELEASE_VERSION>
- Define your format as a Go template, and save this template as a file.
- Set the output format to "template" (`-o template`).
- Specify the path to the template file (`-t ./path/to/custom.template`).
- Syft's template processing uses the same data models as the `syft-json` output format — so if you're wondering what data is available as you author a template, you can use the output from `syft <image> -o syft-json` as a reference.
**Example:** You could make Syft output data in CSV format by writing a Go template that renders CSV data and then running `syft <image> -o template -t ~/path/to/csv.tmpl`.
Syft also includes a vast array of utility templating functions from [sprig](http://masterminds.github.io/sprig/) apart from the default Golang [text/template](https://pkg.go.dev/text/template#hdr-Functions) to allow users to customize the output format.
Lastly, Syft has custom templating functions defined in `./syft/format/template/encoder.go` to help parse the passed-in JSON structs.
## Multiple outputs
Syft can also output _multiple_ files in differing formats by appending
`=<file>` to the option, for example to output Syft JSON and SPDX JSON:
When a container runtime is not present, Syft can still utilize credentials configured in common credential sources (such as `~/.docker/config.json`). It will pull images from private registries using these credentials. The config file is where your credentials are stored when authenticating with private registries via some command like `docker login`. For more information see the `go-containerregistry` [documentation](https://github.com/google/go-containerregistry/tree/main/pkg/authn).
An example `config.json` looks something like this:
```json
{
"auths":{
"registry.example.com":{
"username":"AzureDiamond",
"password":"hunter2"
}
}
}
```
You can run the following command as an example. It details the mount/environment configuration a container needs to access a private registry:
```
docker run -v ./config.json:/config/config.json -e "DOCKER_CONFIG=/config" anchore/syft:latest <private_image>
```
### Docker Credentials in Kubernetes
Here's a simple workflow to mount this config file as a secret into a container on Kubernetes.
1. Create a secret. The value of `config.json` is important. It refers to the specification detailed [here](https://github.com/google/go-containerregistry/tree/main/pkg/authn#the-config-file). Below this section is the `secret.yaml` file that the pod configuration will consume as a volume. The key `config.json` is important. It will end up being the name of the file when mounted into the pod.
```yaml
# secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: registry-config
namespace: syft
data:
config.json: <base64 encoded config.json>
```
`kubectl apply -f secret.yaml`
2. Create your pod running syft. The env `DOCKER_CONFIG` is important because it advertises where to look for the credential file. In the below example, setting `DOCKER_CONFIG=/config` informs syft that credentials can be found at `/config/config.json`. This is why we used `config.json` as the key for our secret. When mounted into containers the secrets' key is used as the filename. The `volumeMounts` section mounts our secret to `/config`. The `volumes` section names our volume and leverages the secret we created in step one.
```yaml
# pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: syft-k8s-usage
spec:
containers:
- image: anchore/syft:latest
name: syft-private-registry-demo
env:
- name: DOCKER_CONFIG
value: /config
volumeMounts:
- mountPath: /config
name: registry-config
readOnly: true
args:
- <private_image>
volumes:
- name: registry-config
secret:
secretName: registry-config
```
`kubectl apply -f pod.yaml`
3. The user can now run `kubectl logs syft-private-registry-demo`. The logs should show the Syft analysis for the `<private_image>` provided in the pod configuration.
Using the above information, users should be able to configure private registry access without having to do so in the `grype` or `syft` configuration files. They will also not be dependent on a Docker daemon, (or some other runtime software) for registry configuration and access.
## Format conversion (experimental)
The ability to convert existing SBOMs means you can create SBOMs in different formats quickly, without the need to regenerate the SBOM from scratch, which may take significantly more time.
This feature is experimental and data might be lost when converting formats. Packages are the main SBOM component easily transferable across formats, whereas files and relationships, as well as other information Syft doesn't support, are more likely to be lost.
We support formats with wide community usage AND good encode/decode support by Syft. The supported formats are:
- Syft JSON (```-o syft-json```)
- SPDX 2.2 JSON (```-o spdx-json```)
- SPDX 2.2 tag-value (```-o spdx-tag-value```)
- CycloneDX 1.4 JSON (```-o cyclonedx-json```)
- CycloneDX 1.4 XML (```-o cyclonedx-xml```)
Conversion example:
```sh
syft alpine:latest -o syft-json=sbom.syft.json # generate a syft SBOM
syft convert sbom.syft.json -o cyclonedx-json=sbom.cdx.json # convert it to CycloneDX
```
## Attestation (experimental)
### Keyless support
Syft supports generating attestations using cosign's [keyless](https://github.com/sigstore/cosign/blob/main/KEYLESS.md) signatures.
Note: users need to have >= v1.12.0 of cosign installed for this command to function
To use this feature with a format like CycloneDX json simply run:
```
syft attest --output cyclonedx-json <IMAGE WITH OCI WRITE ACCESS>
```
This command will open a web browser and allow the user to authenticate their OIDC identity as the root of trust for the attestation (Github, Google, Microsoft).
After authenticating, Syft will upload the attestation to the OCI registry specified by the image that the user has write access to.
You will need to make sure your credentials are configured for the OCI registry you are uploading to so that the attestation can write successfully.
Users can then verify the attestation(or any image with attestations) by running:
Users should see that the uploaded attestation claims are validated, the claims exist within the transparency log, and certificates on the attestations were verified against [fulcio](https://github.com/SigStore/fulcio).
There will also be a printout of the certificates subject `<user identity>` and the certificate issuer URL: `<provider of user identity (Github, Google, Microsoft)>`:
The above output is in the form of the [DSSE envelope](https://github.com/secure-systems-lab/dsse/blob/master/envelope.md#dsse-envelope).
The payload is a base64 encoded `in-toto` statement with the generated SBOM as the predicate. For details on workflows using this command see [here](#adding-an-sbom-to-an-image-as-an-attestation-using-syft).
## Configuration
Configuration search paths:
@@ -81,22 +456,306 @@ Configuration search paths:
Configuration options (example values are the default):
```yaml
# the output format of the SBOM report (options: table, text, json)
# same as -o ; SYFT_OUTPUT env var
output:"table"
# the search space to look for packages (options: all-layers, squashed)
# same as -s ; SYFT_SCOPE env var
scope:"squashed"
# the output format(s) of the SBOM report (options: syft-table, syft-text, syft-json, spdx-json, ...)
# same as -o, --output, and SYFT_OUTPUT env var
# to specify multiple output files in differing formats, use a list:
# output:
# - "syft-json=<syft-json-output-file>"
# - "spdx-json=<spdx-json-output-file>"
output: "syft-table"
# suppress all output (except for the SBOM report)
# same as -q ; SYFT_QUIET env var
quiet: false
# same as --file; write output report to a file (default is to write to stdout)
file: ""
# enable/disable checking for application updates on startup
# same as SYFT_CHECK_FOR_APP_UPDATE env var
check-for-app-update: true
# allows users to specify which image source should be used to generate the sbom
# valid values are: registry, docker, podman
default-image-pull-source: ""
# a list of globs to exclude from scanning. same as --exclude ; for example:
# exclude:
# - "/etc/**"
# - "./out/**/*.json"
exclude: []
# allows users to exclude synthetic binary packages from the sbom
# these packages are removed if an overlap with a non-synthetic package is found
exclude-binary-overlap-by-ownership: true
# os and/or architecture to use when referencing container images (e.g. "windows/armv6" or "arm64")
# same as --platform; SYFT_PLATFORM env var
platform: ""
# set the list of package catalogers to use when generating the SBOM
# default = empty (cataloger set determined automatically by the source type [image or file/directory])
# catalogers:
# - alpm-db-cataloger
# - apkdb-cataloger
# - binary-cataloger
# - cargo-auditable-binary-cataloger
# - cocoapods-cataloger
# - conan-cataloger
# - dartlang-lock-cataloger
# - dotnet-deps-cataloger
# - dotnet-portable-executable-cataloger
# - dpkg-db-cataloger
# - elixir-mix-lock-cataloger
# - erlang-rebar-lock-cataloger
# - go-module-file-cataloger
# - go-module-binary-cataloger
# - graalvm-native-image-cataloger
# - haskell-cataloger
# - java-cataloger
# - java-gradle-lockfile-cataloger
# - java-pom-cataloger
# - javascript-lock-cataloger
# - javascript-package-cataloger
# - linux-kernel-cataloger
# - nix-store-cataloger
# - php-composer-installed-cataloger
# - php-composer-lock-cataloger
# - portage-cataloger
# - python-package-cataloger
# - python-installed-package-cataloger
# - rpm-db-cataloger
# - rpm-archive-cataloger
# - ruby-gemfile-cataloger
# - ruby-installed-gemspec-cataloger
# - rust-cargo-lock-cataloger
# - sbom-cataloger
# - spm-cataloger
catalogers:
# all format configuration
format:
# default value for all formats that support the "pretty" option (default is unset)
pretty:
# all syft-json format options
json:
# include space indention and newlines (inherits default value from 'format.pretty' or 'false' if parent is unset)
# note: inherits default value from 'format.pretty' or 'false' if parent is unset
pretty: false
# transform any syft-json output to conform to an approximation of the v11.0.1 schema. This includes:
# - using the package metadata type names from before v12 of the JSON schema (changed in https://github.com/anchore/syft/pull/1983)
#
# Note: this will still include package types and fields that were added at or after json schema v12. This means
# that output might not strictly be json schema v11 compliant, however, for consumers that require time to port
# over to the final syft 1.0 json output this option can be used to ease the transition.
#
# Note: long term support for this option is not guaranteed.
legacy: false
# all template format options
template:
# path to the template file to use when rendering the output with the `template` output format.
# Note that all template paths are based on the current syft-json schema.
# same as -t ; SYFT_TEMPLATE_PATH env var
path: ""
# all spdx-json format options
spdx-json:
# include space indention and newlines
# note: inherits default value from 'format.pretty' or 'false' if parent is unset
pretty: false
# all cyclonedx-json format options
cyclonedx-json:
# include space indention and newlines
# note: inherits default value from 'format.pretty' or 'false' if parent is unset
pretty: false
# all cyclonedx-xml format options
cyclonedx-xml:
# include space indention
# note: inherits default value from 'format.pretty' or 'false' if parent is unset
pretty: false
# cataloging packages is exposed through the packages and power-user subcommands
package:
# search within archives that do contain a file index to search against (zip)
# note: for now this only applies to the java package cataloger
# SYFT_PACKAGE_SEARCH_INDEXED_ARCHIVES env var
search-indexed-archives: true
# search within archives that do not contain a file index to search against (tar, tar.gz, tar.bz2, etc)
# note: enabling this may result in a performance impact since all discovered compressed tars will be decompressed
# note: for now this only applies to the java package cataloger
# SYFT_PACKAGE_SEARCH_UNINDEXED_ARCHIVES env var
search-unindexed-archives: false
cataloger:
# enable/disable cataloging of packages
# SYFT_PACKAGE_CATALOGER_ENABLED env var
enabled: true
# the search space to look for packages (options: all-layers, squashed)
# same as -s ; SYFT_PACKAGE_CATALOGER_SCOPE env var
scope: "squashed"
golang:
# search for go package licences in the GOPATH of the system running Syft, note that this is outside the
# container filesystem and potentially outside the root of a local directory scan
# SYFT_GOLANG_SEARCH_LOCAL_MOD_CACHE_LICENSES env var
search-local-mod-cache-licenses: false
# specify an explicit go mod cache directory, if unset this defaults to $GOPATH/pkg/mod or $HOME/go/pkg/mod
# SYFT_GOLANG_LOCAL_MOD_CACHE_DIR env var
local-mod-cache-dir: ""
# search for go package licences by retrieving the package from a network proxy
# SYFT_GOLANG_SEARCH_REMOTE_LICENSES env var
search-remote-licenses: false
# remote proxy to use when retrieving go packages from the network,
# if unset this defaults to $GOPROXY followed by https://proxy.golang.org
# SYFT_GOLANG_PROXY env var
proxy: ""
# specifies packages which should not be fetched by proxy
# if unset this defaults to $GONOPROXY
# SYFT_GOLANG_NOPROXY env var
no-proxy: ""
java:
maven-url: "https://repo1.maven.org/maven2"
max-parent-recursive-depth: 5
# enables Syft to use the network to fill in more detailed information about artifacts
# currently this enables searching maven-url for license data
# when running across pom.xml files that could have more information, syft will
# explicitly search maven for license information by querying the online pom when this is true
# this option is helpful for when the parent pom has more data,
# that is not accessible from within the final built artifact
use-network: false
linux-kernel:
# whether to catalog linux kernel modules found within lib/modules/** directories
# SYFT_LINUX_KERNEL_CATALOG_MODULES env var
catalog-modules: true
python:
# when running across entries in requirements.txt that do not specify a specific version
# (e.g. "sqlalchemy >= 1.0.0, <= 2.0.0, != 3.0.0, <= 3.0.0"), attempt to guess what the version could
# be based on the version requirements specified (e.g. "1.0.0"). When enabled the lowest expressible version
# when given an arbitrary constraint will be used (even if that version may not be available/published).
guess-unpinned-requirements: false
file-contents:
cataloger:
# enable/disable cataloging of file contents
# SYFT_FILE_CONTENTS_CATALOGER_ENABLED env var
enabled: true
# the search space to look for file contents (options: all-layers, squashed)
# SYFT_FILE_CONTENTS_CATALOGER_SCOPE env var
scope: "squashed"
# skip searching a file entirely if it is above the given size (default = 1MB; unit = bytes)
# SYFT_FILE_CONTENTS_SKIP_FILES_ABOVE_SIZE env var
skip-files-above-size: 1048576
# file globs for the cataloger to match on
# SYFT_FILE_CONTENTS_GLOBS env var
globs: []
file-metadata:
cataloger:
# enable/disable cataloging of file metadata
# SYFT_FILE_METADATA_CATALOGER_ENABLED env var
enabled: true
# the search space to look for file metadata (options: all-layers, squashed)
# SYFT_FILE_METADATA_CATALOGER_SCOPE env var
scope: "squashed"
# the file digest algorithms to use when cataloging files (options: "md5", "sha1", "sha224", "sha256", "sha384", "sha512")
# SYFT_FILE_METADATA_DIGESTS env var
digests: ["sha256"]
# maximum number of workers used to process the list of package catalogers in parallel
parallelism: 1
# options that apply to all scan sources
source:
# alias name for the source
# SYFT_SOURCE_NAME env var; --source-name flag
name: ""
# alias version for the source
# SYFT_SOURCE_VERSION env var; --source-version flag
version: ""
# options affecting the file source type
file:
# the file digest algorithms to use on the scanned file (options: "md5", "sha1", "sha224", "sha256", "sha384", "sha512")
digests: ["sha256"]
# options when pulling directly from a registry via the "registry:" or "containerd:" scheme
registry:
# skip TLS verification when communicating with the registry
# SYFT_REGISTRY_INSECURE_SKIP_TLS_VERIFY env var
insecure-skip-tls-verify: false
# use http instead of https when connecting to the registry
# SYFT_REGISTRY_INSECURE_USE_HTTP env var
insecure-use-http: false
# filepath to a CA certificate (or directory containing *.crt, *.cert, *.pem) used to generate the client certificate
# SYFT_REGISTRY_CA_CERT env var
ca-cert: ""
# credentials for specific registries
auth:
# the URL to the registry (e.g. "docker.io", "localhost:5000", etc.)
# SYFT_REGISTRY_AUTH_AUTHORITY env var
- authority: ""
# SYFT_REGISTRY_AUTH_USERNAME env var
username: ""
# SYFT_REGISTRY_AUTH_PASSWORD env var
password: ""
# note: token and username/password are mutually exclusive
# SYFT_REGISTRY_AUTH_TOKEN env var
token: ""
# filepath to the client certificate used for TLS authentication to the registry
# SYFT_REGISTRY_AUTH_TLS_CERT env var
tls-cert: ""
# filepath to the client key used for TLS authentication to the registry
# SYFT_REGISTRY_AUTH_TLS_KEY env var
tls-key: ""
# - ... # note, more credentials can be provided via config file only (not env vars)
# generate an attested SBOM
attest:
# path to the private key file to use for attestation
# SYFT_ATTEST_KEY env var
key: "cosign.key"
# password to decrypt to given private key
# SYFT_ATTEST_PASSWORD env var, additionally responds to COSIGN_PASSWORD
password: ""
log:
# use structured logging
# same as SYFT_LOG_STRUCTURED env var
@@ -109,30 +768,44 @@ log:
# location to write the log file (default is not to have a log file)
# same as SYFT_LOG_FILE env var
file: ""
```
anchore:
# (feature-preview) enable uploading of results to Anchore Enterprise automatically (supported on Enterprise 3.0+)
# same as SYFT_ANCHORE_UPLOAD_ENABLED env var
upload-enabled:false
### Adding an SBOM to an image as an attestation using Syft
# (feature-preview) the Anchore Enterprise Host or URL to upload results to (supported on Enterprise 3.0+)
# (feature-preview) the path after the host to the Anchore External API (supported on Enterprise 3.0+)
# same as SYFT_ANCHORE_PATH env var
path:""
SBOMs themselves can serve as input to different analysis tools. [Grype](https://github.com/anchore/grype), a vulnerability scanner CLI tool from Anchore, is one such tool. Publishers of container images can use attestations to enable their consumers to trust Syft-generated SBOM descriptions of those container images. To create and provide these attestations, image publishers can run `syft attest` in conjunction with the [cosign](https://github.com/sigstore/cosign) tool to attach SBOM attestations to their images.
# (feature-preview) the username to authenticate against Anchore Enterprise (supported on Enterprise 3.0+)
# same as -u ; SYFT_ANCHORE_USERNAME env var
username:""
#### Example attestation
Note for the following example replace `docker.io/image:latest` with an image you own. You should also have push access to
its remote reference. Replace `$MY_PRIVATE_KEY` with a private key you own or have generated with cosign.
# (feature-preview) the password to authenticate against Anchore Enterprise (supported on Enterprise 3.0+)
attestExample=`{{.appName}}{{.command}} --output [FORMAT] alpine:latest defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry
Short:"Generate an SBOM as an attestation for the given [SOURCE] container image",
Long:"Generate a packaged-based Software Bill Of Materials (SBOM) from a container image as the predicate of an in-toto attestation that will be uploaded to the image registry",
Long:"[Experimental] Convert SBOM files to, and from, SPDX, CycloneDX and Syft's format. For more info about data loss between formats see https://github.com/anchore/syft#format-conversion-experimental",
packagesExample=`{{.appName}}{{.command}} alpine:latest a summary of discovered packages
{{.appName}}{{.command}} alpine:latest -o json show all possible cataloging details
{{.appName}}{{.command}} alpine:latest -o cyclonedx show a CycloneDX formatted SBOM
{{.appName}}{{.command}} alpine:latest -o cyclonedx-json show a CycloneDX JSON formatted SBOM
{{.appName}}{{.command}} alpine:latest -o spdx show a SPDX 2.3 Tag-Value formatted SBOM
{{.appName}}{{.command}} alpine:latest -o spdx@2.2 show a SPDX 2.2 Tag-Value formatted SBOM
{{.appName}}{{.command}} alpine:latest -o spdx-json show a SPDX 2.3 JSON formatted SBOM
{{.appName}}{{.command}} alpine:latest -o spdx-json@2.2 show a SPDX 2.2 JSON formatted SBOM
{{.appName}}{{.command}} alpine:latest -vv show verbose debug information
{{.appName}}{{.command}} alpine:latest -o template -t my_format.tmpl show a SBOM formatted according to given template file
Supports the following image sources:
{{.appName}}{{.command}} yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}}{{.command}} path/to/a/file/or/dir a Docker tar, OCI tar, OCI directory, SIF container, or generic filesystem directory
`
schemeHelpHeader="You can also explicitly specify the scheme to use:"
imageSchemeHelp=`{{.appName}}{{.command}} docker:yourrepo/yourimage:tag explicitly use the Docker daemon
{{.appName}}{{.command}} podman:yourrepo/yourimage:tag explicitly use the Podman daemon
{{.appName}}{{.command}} registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
{{.appName}}{{.command}} docker-archive:path/to/yourimage.tar use a tarball from disk for archives created from "docker save"
{{.appName}}{{.command}} oci-archive:path/to/yourimage.tar use a tarball from disk for OCI archives (from Skopeo or otherwise)
{{.appName}}{{.command}} oci-dir:path/to/yourimage read directly from a path on disk for OCI layout directories (from Skopeo or otherwise)
{{.appName}}{{.command}} singularity:path/to/yourimage.sif read directly from a Singularity Image Format (SIF) container on disk
`
nonImageSchemeHelp=`{{.appName}}{{.command}} dir:path/to/yourproject read directly from a path on disk (any directory)
{{.appName}}{{.command}} file:path/to/yourproject/file read directly from a path on disk (any single file)
response:"this is really long this is really long this is really long this is really long this is really long this is really long this is really long this is really long ",
code:200,
expected:nil,
err:true,
name:"too long",
response:"this is really long this is really long this is really long this is really long this is really long this is really long this is really long this is really long ",
CheckForAppUpdatebool`yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"`// whether to check for an application update on start up or not
log.WithFields("error",err).Warn("unable to parse event")
returnnil
}
tsk:=m.newTaskProgress(
taskprogress.Title{
Default:"Load image",
Running:"Loading image",
Success:"Loaded image",
},
taskprogress.WithStagedProgressable(prog),
)
ifimgName!=""{
tsk.Context=[]string{imgName}
}
return[]tea.Model{tsk}
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.