Compare commits

...
90 Commits
Author SHA1 Message Date
anchore-actions-token-generator[bot]andspiffcs dd30c99bc2 chore(deps): update bootstrap tools to latest versions (#1728)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2023-04-11 10:13:14 -04:00
Shane Dell 16ebcb2455 Add support for nar files. (#1727)
- Update README.md to show that nar is now supported.
- Created a java-archives/example.nar so that the tests wouldn't break.
- Add nar glob and as an option for pkgType.

Closes #1701

Signed-off-by: Shane Dell <shanedell100@gmail.com>
2023-04-11 09:41:49 -04:00
Alex Goodman 305838582b add highlevel details about catalogers (#1726)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-04-10 13:11:22 -04:00
dependabot[bot] a260fb2774 chore(deps): bump golang.org/x/net from 0.8.0 to 0.9.0 (#1722)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.8.0 to 0.9.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.8.0...v0.9.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-07 15:58:21 -04:00
anchore-actions-token-generator[bot]andkzantow f83cae35f2 chore(deps): update stereoscope to e95d60a265e384df29b7a139f5c5402d6ad72e06 (#1721)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: kzantow <kzantow@users.noreply.github.com>
2023-04-07 08:48:17 -04:00
Henry Sachs 0fed17f1c8 feat: gradle lockfile support (#1719)
Signed-off-by: Henry Sachs <Henry.Sachs@deutschebahn.com>
2023-04-06 14:58:28 -04:00
dependabot[bot] da44db92e9 chore(deps): bump github.com/docker/docker (#1715) 2023-04-06 13:44:51 +00:00
dependabot[bot] 4a499c946e chore(deps): bump golang.org/x/mod from 0.9.0 to 0.10.0 (#1713) 2023-04-06 13:44:41 +00:00
dependabot[bot] 99c28a94a4 chore(deps): bump golang.org/x/term from 0.6.0 to 0.7.0 (#1714) 2023-04-06 13:36:16 +00:00
dependabot[bot] f7ac4e98af chore(deps): bump github.com/spf13/cobra from 1.6.1 to 1.7.0 (#1716)
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.6.1...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-06 09:34:59 -04:00
dependabot[bot] 394ec8d215 chore(deps): bump peter-evans/create-pull-request from 4 to 5 (#1712)
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4 to 5.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v4...v5)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-05 19:04:26 -04:00
Keith Zantow 7845381331 chore: update tools-golang to v0.5.0 (#1717)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-04-05 13:59:52 -04:00
7464079a09 Add Nix cataloger (#1696)
* Add Basic Nix Cataloger

Signed-off-by: Julio Tain Sueiras <juliosueiras@gmail.com>

* Update nix def for the latest syft definition

Signed-off-by: Julio Tain Sueiras <juliosueiras@gmail.com>

* capture nix package files on pkg.NixStoreMetadata

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix unit tests and linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update JSON schema

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address review comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* Update syft/pkg/cataloger/nix/parse_nix_store_path_test.go

Co-authored-by: Florian Klink <flokli@flokli.de>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* support unstable version conventions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update json schema relative to main branch

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update syft json with v7.1.1 schema

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix CLI tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove extra continue statement

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add Nix to list of supported ecosystems

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Julio Tain Sueiras <juliosueiras@gmail.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Julio Tain Sueiras <juliosueiras@gmail.com>
Co-authored-by: Florian Klink <flokli@flokli.de>
2023-04-04 10:53:56 -04:00
Alex Goodman 8a574c9ed9 refactor spdx tooling test to reduce intermittent failures (#1707)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-04-03 18:43:28 +00:00
Alex Goodman 681d250fdc Capture file ownership relationships from portage ecosystem (#1702)
* add portage as file owners

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update json schema with NPM files

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-04-03 09:46:18 -04:00
Keith Zantow 2022ffa0e5 chore: update deprecated set-output calls (#1705)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-04-03 09:36:11 -04:00
Christopher Angelo Phillips dfcc07e512 feat: Add config option to allow user to select the default image source location
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-03-31 10:04:10 -04:00
dependabot[bot] 2fa238af7c chore(deps): bump github.com/docker/docker (#1699)
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 23.0.1+incompatible to 23.0.2+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v23.0.1...v23.0.2)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-29 10:00:37 -04:00
anchore-actions-token-generator[bot]andspiffcs 63bbd1e3ed chore(deps): update bootstrap tools to latest versions (#1697)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2023-03-27 09:17:34 -04:00
anchore-actions-token-generator[bot]andwagoodman 81b87dd108 chore(deps): update stereoscope to d7551b7f46f53179922d6229709d3d1602881080 (#1693)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: wagoodman <wagoodman@users.noreply.github.com>
2023-03-23 16:30:08 +00:00
Christopher Angelo Phillips f473bb75a8 1577 spdxlicense generate (#1691)
Update the license_list.go to have more permissible inputs for greater SPDXID matching.
EX:
GPL3 gpl3 gpl-3 and GPL-3 can all map to GPL-3.0-only

By moving all strings to lower and removing the "-" we're able to return valid SPDX license ID for a greater diversity of input strings.
---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-03-23 11:48:24 -04:00
dependabot[bot] 539bc2afcb chore(deps): bump github.com/vbatts/go-mtree from 0.5.2 to 0.5.3 (#1692)
Bumps [github.com/vbatts/go-mtree](https://github.com/vbatts/go-mtree) from 0.5.2 to 0.5.3.
- [Release notes](https://github.com/vbatts/go-mtree/releases)
- [Changelog](https://github.com/vbatts/go-mtree/blob/main/releases.md)
- [Commits](https://github.com/vbatts/go-mtree/compare/v0.5.2...v0.5.3)

---
updated-dependencies:
- dependency-name: github.com/vbatts/go-mtree
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-23 11:09:32 -04:00
Avi DeitcherandKeith Zantow 9fd532246a feat: scan local go mod cache for licenses of golang packages (#1645)
Signed-off-by: Avi Deitcher <avi@deitcher.net>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
2023-03-23 10:38:15 -04:00
Keith Zantow 11e926ab2f chore: fix flaky license sorting (#1690)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-03-22 14:41:49 -04:00
dependabot[bot] 168c5aed51 chore(deps): bump github.com/gookit/color from 1.5.2 to 1.5.3 (#1689) 2023-03-22 14:26:58 -04:00
Dan d02c56aa5f fix: shell completion by adding missing usage message required by spf13/cobra (#1688)
Signed-off-by: DanHam <DanHam@users.noreply.github.com>
2023-03-22 13:45:09 -04:00
anchore-actions-token-generator[bot] 829a71cd92 chore(deps): update bootstrap tools to latest versions (#1686) 2023-03-22 09:01:24 -04:00
Keith Zantow 34ace36a9e chore: tweak some workflow text (#1685)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-03-21 11:08:49 -04:00
Alex Goodman 100cf1003d Remove more side effects from application config testing (#1684)
* remove a few side effects from config testing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix xdg config app name prefix

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* account for restoring and protecting xdg state throughout testing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-03-20 20:53:45 +00:00
Aidan DelaneyandChristopher Phillips f11a7b5e9f Deprecate config.yaml as valid config source; Add unit regression for correct config paths (#1640)
Warn user of future deprecation of ./config.yaml for v1.0.0 release

---------

Signed-off-by: Aidan Delaney <adelaney21@bloomberg.net>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-03-20 15:13:35 -04:00
anchore-actions-token-generator[bot] 434aa7fd46 chore: Update syft bootstrap tools to latest versions. (#1682)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-03-20 13:20:48 -04:00
Marc-Etienne Vargenau 5fb0423b72 Update documentation: (#1680)
- Syft is now outputing SPDX 2.3 by default
- Give syntax to get SPDX 2.2

Signed-off-by: Marc-Etienne Vargenau <marc-etienne.vargenau@nokia.com>
2023-03-20 10:10:35 -04:00
anchore-actions-token-generator[bot] 7998520848 chore: Update Stereoscope to 7928713c391e20abaede6a029f4ce37b628a4c8b (#1681) 2023-03-18 10:32:39 -04:00
Keith Zantow d05000ff21 fix: reduce logging for bad dpkg lines (#1675)
* fix: reduce logging for bad dpkg lines to Trace level
---------

Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-03-17 13:08:51 -04:00
witchcraze f66e77e2c6 fix ruby classifier (#1678)
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-03-17 09:42:20 -04:00
Christopher Angelo Phillips 928c4a55ff feat: add shared dir for easier cleanup (#1676)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-03-16 16:05:34 -04:00
dependabot[bot] 1899eb50d0 chore(deps): bump github.com/google/go-containerregistry (#1672)
Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/google/go-containerregistry/releases)
- [Changelog](https://github.com/google/go-containerregistry/blob/main/.goreleaser.yml)
- [Commits](https://github.com/google/go-containerregistry/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: github.com/google/go-containerregistry
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-16 12:07:47 -04:00
dependabot[bot] b5ec4d4f08 chore(deps): bump actions/setup-go from 3 to 4 (#1671)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-16 12:02:07 -04:00
Christopher Angelo Phillips 61362c04fa fix: move defer after error to protect panic case (#1670)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-03-15 15:29:10 -04:00
Joye Lin e3140063d4 feat: add argocd, helm, kustomize and kubectl binary classifiers (#1663)
* add argocd, helm, kustomize and kubectl binary classifiers
* update golang PURL
* address PR faceback about binary/test-fixtures/Makefile
* remove the /v[n] suffix from the PURL in both argocd and helm

---------

Signed-off-by: y12studio <y12studio@gmail.com>
2023-03-15 14:53:22 -04:00
razzle 1d9ef34ec7 defer closing file (#1668)
Signed-off-by: razzle <harry@razzle.cloud>
2023-03-15 14:50:42 -04:00
Keith Zantow 302735097e fix: remove author contributing to javascript CPEs (#1669) 2023-03-14 14:10:24 +00:00
Keith Zantow cc0a376aba fix: more python matching support (#1667) 2023-03-13 13:26:43 -04:00
anchore-actions-token-generator[bot]andspiffcs b379dd9f27 Update syft bootstrap tools to latest versions. (#1666)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2023-03-13 10:40:13 -04:00
witchcraze a81e0c8008 feat: add ruby classifier (#1665)
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-03-10 08:29:40 -05:00
anchore-actions-token-generator[bot] 41cbbe09b2 Update syft bootstrap tools to latest versions. (#1658) 2023-03-07 12:54:32 -05:00
Keith Zantow 7714bc0521 fix: improved Python binary detection (#1648) 2023-03-07 10:52:29 -05:00
Weston Steimel 096d2b7bff fix: suppress some known incorrect vendor candidates for npm CPEs (#1659)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-07 10:18:44 -05:00
Keith Zantow 7cfdffab5f fix: sanitize SPDX LicenseRefs (#1657)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-03-06 10:55:23 -05:00
dependabot[bot] f43953d225 chore(deps): bump golang.org/x/mod from 0.8.0 to 0.9.0 (#1655) 2023-03-06 15:49:34 +00:00
dependabot[bot] eea1b48cbb chore(deps): bump golang.org/x/net from 0.7.0 to 0.8.0 (#1653) 2023-03-06 15:38:34 +00:00
dependabot[bot] a063cf300b chore(deps): bump github.com/spf13/afero from 1.9.4 to 1.9.5 (#1654) 2023-03-06 15:21:35 +00:00
dependabot[bot] b73903519c chore(deps): bump golang.org/x/term from 0.5.0 to 0.6.0 (#1656) 2023-03-06 15:20:43 +00:00
Keith Zantow 304be4a5a1 fix: dotnet PURL types are invalid (#1649)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-03-03 16:45:20 -05:00
Weston Steimel c4cbe211a3 feat: disable cpe vendor wildcards to reduce false positives (#1647)
* improved parsing of vendor from github url

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* stop generating wildcard vendors

Add logic for parsing javascript and ruby package vendor candidates from
url and author fields and stop generating wildcard vendor candidates

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

---------

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-03 17:26:46 +00:00
Avi Deitcher 01230aa766 read relative etc/apk/repositories for alpine version when no OS provided (#1615)
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2023-03-02 13:04:56 -05:00
Keith Zantow 5f90d03718 fix: possible race condition (#1639) 2023-03-01 15:35:01 -05:00
Weston Steimel e2ebc9769f fix: remove APK OriginPackage cpe candidates (#1637)
Adding APK OriginPackage CPE candidates to the child package
results in false positives in grype because it can't associate
CPE-based findings to the corresponding OriginPackage APK fixes.

This reverts changing the `upstream` in the PURL for APK packages
as the logic in Grype that uses it expects it to be an APK package
name.  This also allows refactoring to unexport and move the APK
CPE candidate generation logic closer to where CPE generation occurs

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 17:24:43 +00:00
Keith Zantow 2e6e3b0c74 fix: rebar lock file decoding panic (#1628) 2023-03-01 10:08:29 -05:00
Keith Zantow 24584a4d27 fix: handle individual cataloger panics (#1636) 2023-03-01 10:03:34 -05:00
Weston Steimel 8e1205f7ab fix: apk product/vendor generation for old metadata (#1635)
This fixes some instances where the improved APK CPE generation
logic caused regressions for older alpine package APK metadata.
It now generates multiple "upstream" candidates with both name
and package type which reduces the amount of duplicated code in
the apk cpe gen logic.  This also improves the handling of stream
version packages, so now we can correctly identify packages such
as ruby3.2-rexml as the rexml ruby gem.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 14:58:35 +00:00
Weston Steimel e92b0fa629 feat: rust toolchain binary cataloger (#1601)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 14:53:37 +00:00
Weston Steimel bcc0751a40 feat: retain go package info when no module declared (#1632)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 14:26:44 +00:00
Weston Steimel f1169e56fc fix: improved CPE-generation for several more APK packages (#1631)
* fix: correct vendor for musl

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor for firefox and thunderbird

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor/product for chromium

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct product for apache http server

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct product for tiff

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor for ghostscript

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor for openjpeg

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor/product for xorg-server

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor for podofo

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor for wpa_supplicant

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

---------

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 08:55:40 -05:00
Christopher Angelo Phillips 98e737fc27 chore: update deprecated release flag (#1629)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-02-27 15:57:56 -05:00
dependabot[bot] ff34594284 chore(deps): bump actions/upload-artifact from 2 to 3 (#1627)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-27 14:17:29 -05:00
Benji Visser 9e953b1da3 feat: add support for SUPPORT_END in /etc/os-release (#1612)
Signed-off-by: Benji Visser <benji@093b.org>
2023-02-27 13:43:19 -05:00
Weston Steimel fbda21f4f4 fix: further improvements to CPE generation for apk packages (#1623)
* fix: consider upstream logic during apk cpe gen
* fix: correct apk CPE for go
* fix: correct apk CPE for ruby
* fix: correct apk CPE for bazel
* fix: correct apk CPE for clang
* fix: correct apk CPE for openjdk
* fix: correct apk CPE for glibc
* fix: correct apk CPE for gli
* fix: correct apk CPE for bas
* fix: correct apk CPE for alsa-lib
* fix: correct apk CPE for alsa
* fix: determine apk cpe vendor from known URLs
* fix: add more url prefix->vendor mappings for apk
* refactor: allow reuse of vendor by url prefix logic
* feat: extract username as vendor candidate from github/gitlab

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-27 13:16:04 -05:00
dependabot[bot] d23b4d4cbd chore(deps): bump github.com/stretchr/testify from 1.8.1 to 1.8.2 (#1625)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-27 13:14:20 -05:00
dependabot[bot] f3acff81f3 chore(deps): bump actions/checkout from 2 to 3 (#1626)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-27 13:14:03 -05:00
Nils Hanke fa0a9fe8f9 feat: set cosign attest predicate type based on Syft output type (#1598)
Signed-off-by: Nils Hanke <nils.hanke@outlook.de>
2023-02-24 15:08:40 -05:00
dependabot[bot] 284bae9d5f chore(deps): bump github.com/spf13/afero from 1.9.3 to 1.9.4 (#1609)
Bumps [github.com/spf13/afero](https://github.com/spf13/afero) from 1.9.3 to 1.9.4.
- [Release notes](https://github.com/spf13/afero/releases)
- [Commits](https://github.com/spf13/afero/compare/v1.9.3...v1.9.4)

---
updated-dependencies:
- dependency-name: github.com/spf13/afero
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-24 15:07:52 -05:00
Weston Steimel 3ee1af0dc6 fix: correct apk purls for other distros (#1620)
The apk purl spec allows for vendor-specific namespace.  I noticed
in the embedded SBOMs from wolfi that the purls are of the form
`pkg:apk/wolfi/curl@7.83.0-r0?arch=x86`, but the current logic in
syft actually prevents purl generation entirely if the distro isn't
alpine, so this corrects that behaviour.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-24 15:07:07 -05:00
Weston Steimel 0c5f03235e refactor: move apk upstream logic to apk metadata (#1619)
* refactor: move apk upstream logic to apk metadata

Export the logic for parsing upstream APK package names
so it can be accessed from apk metadata objects directly.

This also tightens the upstream regex pattern as several
edge cases were being missed.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: ensure correct handling for apk packages beginning with digits

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: upstream generation for ruby

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

---------

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-24 15:59:19 +00:00
Keith Zantow 5e8aa4da5e fix: decoding null apk metadata pullDependencies (#1614)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-02-23 19:55:49 +00:00
Benji Visser abfec62219 feat: haproxy binary matcher (#1591)
Signed-off-by: Benji Visser <benji@093b.org>
2023-02-23 14:39:08 -05:00
Weston Steimel 0c05855131 fix: determine upstream for apk version streams (#1610)
Determines better upstream package name for version-stream apk packages:

Examples:

- postgresql-13 -> postgresql
- postgresql15 -> postgresql
- go-1.19 -> go
- perl100.55 -> perl

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-23 17:32:34 +00:00
Weston Steimel 1150772d06 fix: improve CPE generation for curl APK (#1608)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-23 17:32:12 +00:00
Alex Goodman 669fee84d5 Revert "add workaround for macos github actions cache issue (#1584)" (#1605)
This reverts commit 0076b19893.

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-22 15:03:12 -05:00
anchore-actions-token-generator[bot]andwagoodman aa151da5fe Update Stereoscope to fab1c9638abc2c21cd53dca1f205f37d71148ee0 (#1604)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: wagoodman <wagoodman@users.noreply.github.com>
2023-02-22 19:08:35 +00:00
Keith Zantow 4bf677d555 chore: fix cataloger_test (#1603)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-02-22 17:30:41 +00:00
Keith Zantow f5e20521e0 fix: merging of binary packages (#1583) 2023-02-22 12:03:15 -05:00
Keith Zantow 8f6a317fef fix: issue when matching format versions (#1585)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-02-22 10:32:05 -05:00
anchore-actions-token-generator[bot] d339ffdcb5 chore: update syft bootstrap tools to latest versions. (#1593) 2023-02-22 10:16:14 -05:00
Benji Visser e84ffc6003 feat: add perl binary classifier (#1592)
Signed-off-by: Benji Visser <benji@093b.org>
2023-02-22 09:58:05 -05:00
anchore-actions-token-generator[bot] bb52a25c8a Update Stereoscope to 529924d6d5aa6c708cceffc651883b6e1e27f5df (#1602)
Signed-off-by: GitHub <noreply@github.com>
2023-02-22 08:49:04 +00:00
Marc-Etienne Vargenau 4b7b709473 Update SPDX license list to 3.20 (#1600)
Signed-off-by: Marc-Etienne Vargenau <marc-etienne.vargenau@nokia.com>
2023-02-21 15:12:28 -05:00
Keith Zantow 2ef2eed382 chore: update SPDX license list (#1599) 2023-02-21 17:31:24 +00:00
Alex Goodman f6716092af fix cataloger selection to be more specific (#1582)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-17 15:35:54 +00:00
Alex Goodman 0076b19893 add workaround for macos github actions cache issue (#1584)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-17 10:29:33 -05:00
202 changed files with 251558 additions and 8809 deletions
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
OUTPUT="${OUTPUT//'%'/'%25'}" # URL encode all '%' characters
OUTPUT="${OUTPUT//$'\n'/'%0A'}" # URL encode all '\n' characters
OUTPUT="${OUTPUT//$'\r'/'%0D'}" # URL encode all '\r' characters
echo "::set-output name=result::$OUTPUT"
echo "result=$OUTPUT" >> $GITHUB_OUTPUT
- uses: actions/upload-artifact@v3
with:
+1 -1
View File
@@ -30,7 +30,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v3
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
+2 -2
View File
@@ -151,12 +151,12 @@ jobs:
with:
status: ${{ job.status }}
fields: repo,workflow,action,eventName
text: "A new Syft release has been published: https://github.com/anchore/syft/releases/tag/${{ github.ref_name }}"
text: "A new Syft release has been published: https://github.com/anchore/syft/releases/tag/${{ github.event.inputs.version }}"
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_TOOLBOX_WEBHOOK_URL }}
if: ${{ success() }}
- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v3
with:
name: artifacts
path: dist/**/*
+4 -4
View File
@@ -16,7 +16,7 @@ jobs:
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v3
- uses: actions/setup-go@v4
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
@@ -58,14 +58,14 @@ jobs:
app_id: ${{ secrets.TOKEN_APP_ID }}
private_key: ${{ secrets.TOKEN_APP_PRIVATE_KEY }}
- uses: peter-evans/create-pull-request@v4
- uses: peter-evans/create-pull-request@v5
with:
signoff: true
delete-branch: true
branch: auto/latest-bootstrap-tools
labels: dependencies
commit-message: 'Update syft bootstrap tools to latest versions.'
title: 'Update syft bootstrap tools to latest versions.'
commit-message: 'chore(deps): update bootstrap tools to latest versions'
title: 'chore(deps): update bootstrap tools to latest versions'
body: |
- [golangci-lint ${{ steps.latest-versions.outputs.GOLANGCILINT }}](https://github.com/golangci/golangci-lint/releases/tag/${{ steps.latest-versions.outputs.GOLANGCILINT }})
- [bouncer ${{ steps.latest-versions.outputs.BOUNCER }}](https://github.com/wagoodman/go-bouncer/releases/tag/${{ steps.latest-versions.outputs.BOUNCER }})
@@ -16,7 +16,7 @@ jobs:
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v3
- uses: actions/setup-go@v4
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
@@ -29,7 +29,7 @@ jobs:
go mod tidy
# export the version for use with create-pull-request
echo "::set-output name=LATEST_VERSION::$LATEST_VERSION"
echo "LATEST_VERSION=$LATEST_VERSION" >> $GITHUB_OUTPUT
id: latest-version
- uses: tibdex/github-app-token@v1
@@ -38,14 +38,14 @@ jobs:
app_id: ${{ secrets.TOKEN_APP_ID }}
private_key: ${{ secrets.TOKEN_APP_PRIVATE_KEY }}
- uses: peter-evans/create-pull-request@v4
- uses: peter-evans/create-pull-request@v5
with:
signoff: true
delete-branch: true
branch: auto/latest
branch: auto/latest-stereoscope
labels: dependencies
commit-message: "Update Stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}"
title: "Update Stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}"
commit-message: "chore(deps): update stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}"
title: "chore(deps): update stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}"
body: |
Update Stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}
token: ${{ steps.generate-token.outputs.token }}
+7
View File
@@ -54,6 +54,13 @@ jobs:
path: syft/pkg/cataloger/golang/test-fixtures/archs/binaries
key: ${{ runner.os }}-unit-go-binaries-cache-${{ hashFiles( 'syft/pkg/cataloger/golang/test-fixtures/archs/binaries.fingerprint' ) }}
- name: Restore binary cataloger test-fixture cache
id: unit-binary-cataloger-cache
uses: actions/cache@v3
with:
path: syft/pkg/cataloger/binary/test-fixtures/classifiers/dynamic
key: ${{ runner.os }}-unit-binary-cataloger-cache-${{ hashFiles( 'syft/pkg/cataloger/binary/test-fixtures/cache.fingerprint' ) }}
- name: Run unit tests
run: make unit
+9 -1
View File
@@ -119,7 +119,15 @@ sequenceDiagram
Catalogers are the way in which syft is able to identify and construct packages given some amount of source metadata.
For example, Syft can locate and process `package-lock.json` files when performing filesystem scans.
See: [how to specify file globs](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/javascript/cataloger.go#L16-L21)
and an implementation of the [package-lock.json parser](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/javascript/cataloger.go#L16-L21) fora quick review.
and an implementation of the [package-lock.json parser](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/javascript/cataloger.go#L16-L21) for a quick review.
From a high level catalogers have the following properties:
- They are independent from one another. The java cataloger has no idea of the processes, assumptions, or results of the python cataloger, for example.
- They do not know what source is being analyzed. Are we analyzing a local directory? an image? if so, the squashed representation or all layers? The catalogers do not know the answers to these questions. Only that there is an interface to query for file paths and contents from an underlying "source" being scanned.
- Packages created by the cataloger should not be mutated after they are created. There is one exception made for adding CPEs to a package after the cataloging phase, but that will most likely be moved back into the cataloger in the future.
#### Building a new Cataloger
+14 -4
View File
@@ -4,17 +4,17 @@ TEMP_DIR := ./.tmp
# Command templates #################################
LINT_CMD := $(TEMP_DIR)/golangci-lint run --tests=false
GOIMPORTS_CMD := $(TEMP_DIR)/gosimports -local github.com/anchore
RELEASE_CMD := $(TEMP_DIR)/goreleaser release --rm-dist
RELEASE_CMD := $(TEMP_DIR)/goreleaser release --clean
SNAPSHOT_CMD := $(RELEASE_CMD) --skip-publish --skip-sign --snapshot
CHRONICLE_CMD = $(TEMP_DIR)/chronicle
GLOW_CMD = $(TEMP_DIR)/glow
# Tool versions #################################
GOLANGCILINT_VERSION := v1.51.1
GOSIMPORTS_VERSION := v0.3.5
GOLANGCILINT_VERSION := v1.52.2
GOSIMPORTS_VERSION := v0.3.8
BOUNCER_VERSION := v0.4.0
CHRONICLE_VERSION := v0.6.0
GORELEASER_VERSION := v1.15.2
GORELEASER_VERSION := v1.17.0
YAJSV_VERSION := v1.4.1
COSIGN_VERSION := v1.13.1
QUILL_VERSION := v0.2.0
@@ -189,6 +189,10 @@ fingerprints:
cd test/integration/test-fixtures && \
make cache.fingerprint
# for BINARY test fixtures
cd syft/pkg/cataloger/binary/test-fixtures && \
make cache.fingerprint
# for JAVA BUILD test fixtures
cd syft/pkg/cataloger/java/test-fixtures/java-builds && \
make packages.fingerprint
@@ -214,6 +218,7 @@ fixtures:
$(call title,Generating test fixtures)
cd syft/pkg/cataloger/java/test-fixtures/java-builds && make
cd syft/pkg/cataloger/rpm/test-fixtures && make
cd syft/pkg/cataloger/binary/test-fixtures && make
.PHONY: show-test-image-cache
show-test-image-cache: ## Show all docker and image tar cache
@@ -247,6 +252,11 @@ install-test-ci-mac: $(SNAPSHOT_DIR)
cd test/install && \
make ci-test-mac
.PHONY: generate-compare-file
generate-compare-file:
$(call title,Generating compare test file)
go run ./cmd/syft $(COMPARE_TEST_IMAGE) -o json > $(COMPARE_DIR)/test-fixtures/acceptance-centos-8.2.2004.json
# note: we cannot clean the snapshot directory since the pipeline builds the snapshot separately
.PHONY: compare-mac
compare-mac: $(TEMP_DIR) $(SNAPSHOT_DIR) ## Run compare tests on build snapshot binaries and packages (Mac)
+30 -9
View File
@@ -42,9 +42,10 @@ For commercial support options with Syft or Grype, please [contact Anchore](http
- Erlang (rebar3)
- Go (go.mod, Go binaries)
- Haskell (cabal, stack)
- Java (jar, ear, war, par, sar, native-image)
- Java (jar, ear, war, par, sar, nar, native-image)
- JavaScript (npm, yarn)
- Jenkins Plugins (jpi, hpi)
- Nix (outputs in /nix/store)
- PHP (composer)
- Python (wheel, egg, poetry, requirements.txt)
- Red Hat (rpm)
@@ -110,9 +111,7 @@ The above output includes only software that is visible in the container (i.e.,
syft <image> --scope all-layers
```
## Supported sources
### Supported sources
Syft can generate a SBOM from a variety of sources:
@@ -141,7 +140,13 @@ file:path/to/yourproject/file read directly from a path on disk (any
registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
```
#### Default Cataloger Configuration by scan type
If an image source is not provided and cannot be detected from the given reference it is assumed the image should be pulled from the Docker daemon.
If docker is not present, then the Podman daemon is attempted next, followed by reaching out directly to the image registry last.
This default behavior can be overridden with the `default-image-pull-source` configuration option (See [Configuration](https://github.com/anchore/syft#configuration) for more details).
### Default Cataloger Configuration by scan type
##### Image Scanning:
- alpmdb
@@ -179,7 +184,7 @@ registry:yourrepo/yourimage:tag pull image directly from a registry (no
- conan
- hackage
#### Non Default:
##### Non Default:
- cargo-auditable-binary
### Excluding file paths
@@ -213,8 +218,10 @@ Where the `formats` available are:
- `text`: A row-oriented, human-and-machine-friendly output.
- `cyclonedx-xml`: A XML report conforming to the [CycloneDX 1.4 specification](https://cyclonedx.org/specification/overview/).
- `cyclonedx-json`: A JSON report conforming to the [CycloneDX 1.4 specification](https://cyclonedx.org/specification/overview/).
- `spdx-tag-value`: A tag-value formatted report conforming to the [SPDX 2.2 specification](https://spdx.github.io/spdx-spec/).
- `spdx-json`: A JSON report conforming to the [SPDX 2.2 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.2/schemas/spdx-schema.json).
- `spdx-tag-value`: A tag-value formatted report conforming to the [SPDX 2.3 specification](https://spdx.github.io/spdx-spec/v2.3/).
- `spdx-tag-value@2.2`: A tag-value formatted report conforming to the [SPDX 2.2 specification](https://spdx.github.io/spdx-spec/v2.2.2/).
- `spdx-json`: A JSON report conforming to the [SPDX 2.3 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.3/schemas/spdx-schema.json).
- `spdx-json@2.2`: A JSON report conforming to the [SPDX 2.2 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.2/schemas/spdx-schema.json).
- `github`: A JSON report conforming to GitHub's dependency snapshot format.
- `table`: A columnar summary (default).
- `template`: Lets the user specify the output format. See ["Using templates"](#using-templates) below.
@@ -391,7 +398,7 @@ Certificate subject: test.email@testdomain.com
Certificate issuer URL: https://accounts.google.com
```
#### Local private key support
### Local private key support
To generate an SBOM attestation for a container image using a local private key:
```
@@ -434,6 +441,10 @@ file: ""
# same as SYFT_CHECK_FOR_APP_UPDATE env var
check-for-app-update: true
# allows users to specify which image source should be used to generate the sbom
# valid values are: registry, docker, podman
default-image-pull-source: ""
# a list of globs to exclude from scanning. same as --exclude ; for example:
# exclude:
# - "/etc/**"
@@ -492,6 +503,16 @@ package:
# same as -s ; SYFT_PACKAGE_CATALOGER_SCOPE env var
scope: "squashed"
golang:
# search for go package licences in the GOPATH of the system running Syft, note that this is outside the
# container filesystem and potentially outside the root of a local directory scan
# SYFT_GOLANG_SEARCH_LOCAL_MOD_CACHE_LICENSES env var
search-local-mod-cache-licenses: false
# specify an explicit go mod cache directory, if unset this defaults to $GOPATH/pkg/mod or $HOME/go/pkg/mod
# SYFT_GOLANG_LOCAL_MOD_CACHE_DIR env var
local-mod-cache-dir: ""
# cataloging file contents is exposed through the power-user subcommand
file-contents:
cataloger:
+20 -5
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"os"
"os/exec"
"strings"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
@@ -26,7 +27,7 @@ import (
"github.com/anchore/syft/syft/source"
)
func Run(ctx context.Context, app *config.Application, args []string) error {
func Run(_ context.Context, app *config.Application, args []string) error {
err := ValidateOutputOptions(app)
if err != nil {
return err
@@ -46,7 +47,7 @@ func Run(ctx context.Context, app *config.Application, args []string) error {
// could be an image or a directory, with or without a scheme
// TODO: validate that source is image
userInput := args[0]
si, err := source.ParseInputWithName(userInput, app.Platform, true, app.Name)
si, err := source.ParseInputWithName(userInput, app.Platform, app.Name, app.DefaultImagePullSource)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
@@ -130,7 +131,21 @@ func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-
return
}
args := []string{"attest", si.UserInput, "--predicate", f.Name()}
// Select Cosign predicate type based on defined output type
// As orientation, check: https://github.com/sigstore/cosign/blob/main/pkg/cosign/attestation/attestation.go
var predicateType string
switch strings.ToLower(o) {
case "cyclonedx-json":
predicateType = "cyclonedx"
case "spdx-tag-value":
predicateType = "spdx"
case "spdx-json":
predicateType = "spdxjson"
default:
predicateType = "custom"
}
args := []string{"attest", si.UserInput, "--predicate", f.Name(), "--type", predicateType}
if app.Attest.Key != "" {
args = append(args, "--key", app.Attest.Key)
}
@@ -152,7 +167,7 @@ func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-
}
defer w.Close()
b := &busWriter{r: r, w: w, mon: &progress.Manual{N: -1}}
b := &busWriter{r: r, w: w, mon: progress.NewManual(-1)}
execCmd.Stdout = b
execCmd.Stderr = b
defer b.mon.SetCompleted()
@@ -160,7 +175,7 @@ func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-
// attest the SBOM
err = execCmd.Run()
if err != nil {
b.mon.Err = err
b.mon.SetError(err)
errs <- fmt.Errorf("unable to attest SBOM: %w", err)
return
}
+1
View File
@@ -55,6 +55,7 @@ func New() (*cobra.Command, error) {
// rootCmd is currently an alias for the packages command
rootCmd := &cobra.Command{
Use: fmt.Sprintf("%s [SOURCE]", internal.ApplicationName),
Short: packagesCmd.Short,
Long: packagesCmd.Long,
Args: packagesCmd.Args,
+1 -4
View File
@@ -46,14 +46,11 @@ func generateCatalogPackagesTask(app *config.Application) (Task, error) {
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
packageCatalog, relationships, theDistro, err := syft.CatalogPackages(src, app.ToCatalogerConfig())
if err != nil {
return nil, err
}
results.PackageCatalog = packageCatalog
results.LinuxDistribution = theDistro
return relationships, nil
return relationships, err
}
return task, nil
+1
View File
@@ -17,6 +17,7 @@ func (o AttestOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
return bindAttestConfigOptions(cmd.Flags(), v)
}
//nolint:revive
func bindAttestConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
if err := v.BindPFlag("attest.key", flags.Lookup("key")); err != nil {
return err
+1
View File
@@ -31,6 +31,7 @@ func (o *FulcioOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
return bindFulcioConfigOptions(cmd.Flags(), v)
}
//nolint:revive
func bindFulcioConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
if err := v.BindPFlag("attest.fulcio-url", flags.Lookup("fulcio-url")); err != nil {
return err
+1
View File
@@ -31,6 +31,7 @@ func (o *OIDCOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
return bindOIDCConfigOptions(cmd.Flags(), v)
}
//nolint:revive
func bindOIDCConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
if err := v.BindPFlag("attest.oidc-issuer", flags.Lookup("oidc-issuer")); err != nil {
return err
+1
View File
@@ -54,6 +54,7 @@ func (o *PackagesOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
return bindPackageConfigOptions(cmd.Flags(), v)
}
//nolint:revive
func bindPackageConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
// Formatting & Input options //////////////////////////////////////////////
+1
View File
@@ -22,6 +22,7 @@ func (o *RekorOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
return bindRekorConfigOptions(cmd.Flags(), v)
}
//nolint:revive
func bindRekorConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
// TODO: config re-design
if err := v.BindPFlag("attest.rekor-url", flags.Lookup("rekor-url")); err != nil {
+1
View File
@@ -22,6 +22,7 @@ func (o *RootOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
return bindRootConfigOptions(cmd.PersistentFlags(), v)
}
//nolint:revive
func bindRootConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
if err := v.BindPFlag("config", flags.Lookup("config")); err != nil {
return err
+1 -1
View File
@@ -11,7 +11,7 @@ type VersionOptions struct {
var _ Interface = (*VersionOptions)(nil)
func (o *VersionOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
func (o *VersionOptions) AddFlags(cmd *cobra.Command, _ *viper.Viper) error {
cmd.Flags().StringVarP(&o.Output, "output", "o", "text", "format to show version information (available=[text, json])")
return nil
}
+4 -2
View File
@@ -18,8 +18,10 @@ const (
{{.appName}} {{.command}} alpine:latest -o json show all possible cataloging details
{{.appName}} {{.command}} alpine:latest -o cyclonedx show a CycloneDX formatted SBOM
{{.appName}} {{.command}} alpine:latest -o cyclonedx-json show a CycloneDX JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx show a SPDX 2.2 Tag-Value formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx-json show a SPDX 2.2 JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx show a SPDX 2.3 Tag-Value formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx@2.2 show a SPDX 2.2 Tag-Value formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx-json show a SPDX 2.3 JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx-json@2.2 show a SPDX 2.2 JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -vv show verbose debug information
{{.appName}} {{.command}} alpine:latest -o template -t my_format.tmpl show a SBOM formatted according to given template file
+2 -2
View File
@@ -23,7 +23,7 @@ import (
"github.com/anchore/syft/syft/source"
)
func Run(ctx context.Context, app *config.Application, args []string) error {
func Run(_ context.Context, app *config.Application, args []string) error {
err := ValidateOutputOptions(app)
if err != nil {
return err
@@ -42,7 +42,7 @@ func Run(ctx context.Context, app *config.Application, args []string) error {
// could be an image or a directory, with or without a scheme
userInput := args[0]
si, err := source.ParseInputWithName(userInput, app.Platform, true, app.Name)
si, err := source.ParseInputWithName(userInput, app.Platform, app.Name, app.DefaultImagePullSource)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
+1 -1
View File
@@ -47,7 +47,7 @@ func Run(_ context.Context, app *config.Application, args []string) error {
}()
userInput := args[0]
si, err := source.ParseInputWithName(userInput, app.Platform, true, app.Name)
si, err := source.ParseInputWithName(userInput, app.Platform, app.Name, app.DefaultImagePullSource)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
+1 -1
View File
@@ -15,7 +15,7 @@ import (
"github.com/anchore/syft/internal/version"
)
func Version(v *viper.Viper, app *config.Application) *cobra.Command {
func Version(v *viper.Viper, _ *config.Application) *cobra.Command {
o := &options.VersionOptions{}
cmd := &cobra.Command{
Use: "version",
+23 -22
View File
@@ -16,7 +16,7 @@ require (
github.com/go-test/deep v1.1.0
github.com/google/go-cmp v0.5.9
github.com/google/uuid v1.3.0
github.com/gookit/color v1.5.2
github.com/gookit/color v1.5.3
github.com/hashicorp/go-multierror v1.1.1
github.com/jinzhu/copier v0.3.5
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
@@ -31,20 +31,20 @@ require (
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e
github.com/sergi/go-diff v1.3.1
github.com/sirupsen/logrus v1.9.0
github.com/spdx/tools-golang v0.5.0-rc1
github.com/spf13/afero v1.9.3
github.com/spf13/cobra v1.6.1
github.com/spdx/tools-golang v0.5.0
github.com/spf13/afero v1.9.5
github.com/spf13/cobra v1.7.0
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.15.0
github.com/stretchr/testify v1.8.1
github.com/stretchr/testify v1.8.2
github.com/vifraa/gopom v0.2.1
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/mod v0.8.0
golang.org/x/net v0.7.0
golang.org/x/term v0.5.0
golang.org/x/mod v0.10.0
golang.org/x/net v0.9.0
golang.org/x/term v0.7.0
gopkg.in/yaml.v2 v2.4.0
)
@@ -52,14 +52,15 @@ require (
github.com/CycloneDX/cyclonedx-go v0.7.1-0.20221222100750-41a1ac565cce
github.com/Masterminds/sprig/v3 v3.2.3
github.com/anchore/go-logger v0.0.0-20220728155337-03b66a5207d8
github.com/anchore/stereoscope v0.0.0-20230216143338-4b5ebf8c7f4b
github.com/docker/docker v23.0.1+incompatible
github.com/google/go-containerregistry v0.13.0
github.com/anchore/stereoscope v0.0.0-20230406143206-e95d60a265e3
github.com/docker/docker v23.0.3+incompatible
github.com/google/go-containerregistry v0.14.0
github.com/google/licensecheck v0.3.1
github.com/invopop/jsonschema v0.7.0
github.com/knqyf263/go-rpmdb v0.0.0-20221030135625-4082a22221ce
github.com/opencontainers/go-digest v1.0.0
github.com/sassoftware/go-rpmutils v0.2.0
github.com/vbatts/go-mtree v0.5.2
github.com/vbatts/go-mtree v0.5.3
golang.org/x/exp v0.0.0-20230202163644-54bba9f4231b
gopkg.in/yaml.v3 v3.0.1
)
@@ -72,9 +73,9 @@ require (
github.com/anchore/go-struct-converter v0.0.0-20221118182256-c68fdcfa2092 // indirect
github.com/becheran/wildmatch-go v1.0.0 // indirect
github.com/containerd/containerd v1.6.18 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.12.1 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/docker/cli v20.10.20+incompatible // indirect
github.com/docker/cli v23.0.1+incompatible // indirect
github.com/docker/distribution v2.8.1+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/docker/go-connections v0.4.0 // indirect
@@ -84,16 +85,16 @@ require (
github.com/gabriel-vasile/mimetype v1.4.0 // indirect
github.com/go-restruct/restruct v1.2.0-alpha // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/huandu/xstrings v1.3.3 // indirect
github.com/iancoleman/orderedmap v0.0.0-20190318233801-ac98e3ecb4b0 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/klauspost/compress v1.15.11 // indirect
github.com/klauspost/compress v1.16.0 // indirect
github.com/klauspost/pgzip v1.2.5 // indirect
github.com/logrusorgru/aurora v0.0.0-20200102142835-e9ef32dff381 // indirect
github.com/magiconair/properties v1.8.7 // indirect
@@ -126,13 +127,13 @@ require (
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.5.0 // indirect
golang.org/x/text v0.7.0 // indirect
golang.org/x/tools v0.2.0 // indirect
golang.org/x/sys v0.7.0 // indirect
golang.org/x/text v0.9.0 // indirect
golang.org/x/tools v0.7.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/genproto v0.0.0-20221227171554-f9683d7f8bef // indirect
google.golang.org/grpc v1.52.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
google.golang.org/protobuf v1.29.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
lukechampine.com/uint128 v1.1.1 // indirect
modernc.org/cc/v3 v3.36.0 // indirect
+48 -44
View File
@@ -90,8 +90,8 @@ github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZV
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501 h1:AV7qjwMcM4r8wFhJq3jLRztew3ywIyPTRapl2T1s9o8=
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501/go.mod h1:Blo6OgJNiYF41ufcgHKkbCKF2MDOMlrqhXv/ij6ocR4=
github.com/anchore/stereoscope v0.0.0-20230216143338-4b5ebf8c7f4b h1:vMEAfz91QLjJq2W8JPxpIC4dG4OeynTY4MisHnZ19F0=
github.com/anchore/stereoscope v0.0.0-20230216143338-4b5ebf8c7f4b/go.mod h1:6oSG43mzahqiktzXZDctqi1o66fwU2wDk3xki0KlnbA=
github.com/anchore/stereoscope v0.0.0-20230406143206-e95d60a265e3 h1:F662TNGcpBfB1ZmCJJUot4EBHGxCKvTPvub01qaWxX0=
github.com/anchore/stereoscope v0.0.0-20230406143206-e95d60a265e3/go.mod h1:u44RadbOJB103GwtYzJ2Kdad9J+SmZMW98jCYBJkSzs=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
github.com/andybalholm/brotli v1.0.4 h1:V7DdXeJtZscaqfNuAdSRuRFzuiKlHSC/Zh3zl9qY3JY=
@@ -134,8 +134,8 @@ github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWH
github.com/cncf/xds/go v0.0.0-20211130200136-a8f946100490/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/containerd/containerd v1.6.18 h1:qZbsLvmyu+Vlty0/Ex5xc0z2YtKpIsb5n45mAMI+2Ns=
github.com/containerd/containerd v1.6.18/go.mod h1:1RdCUu95+gc2v9t3IL+zIlpClSmew7/0YS8O5eQZrOw=
github.com/containerd/stargz-snapshotter/estargz v0.12.1 h1:+7nYmHJb0tEkcRaAW+MHqoKaJYZmkikupxCqVtmPuY0=
github.com/containerd/stargz-snapshotter/estargz v0.12.1/go.mod h1:12VUuCq3qPq4y8yUW+l5w3+oXV3cx2Po3KSe/SmPGqw=
github.com/containerd/stargz-snapshotter/estargz v0.14.3 h1:OqlDCK3ZVUO6C3B/5FSkDwbkEETK84kQgEeFwDC+62k=
github.com/containerd/stargz-snapshotter/estargz v0.14.3/go.mod h1:KY//uOCIkSuNAHhJogcZtrNHdKrA99/FCCRjE3HD36o=
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
@@ -146,12 +146,12 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgrijalva/jwt-go/v4 v4.0.0-preview1/go.mod h1:+hnT3ywWDTAFrW5aE+u2Sa/wT555ZqwoCS+pk3p6ry4=
github.com/docker/cli v20.10.20+incompatible h1:lWQbHSHUFs7KraSN2jOJK7zbMS2jNCHI4mt4xUFUVQ4=
github.com/docker/cli v20.10.20+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/cli v23.0.1+incompatible h1:LRyWITpGzl2C9e9uGxzisptnxAn1zfZKXy13Ul2Q5oM=
github.com/docker/cli v23.0.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.1+incompatible h1:Q50tZOPR6T/hjNsyc9g8/syEs6bk8XXApsHjKukMl68=
github.com/docker/distribution v2.8.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v23.0.1+incompatible h1:vjgvJZxprTTE1A37nm+CLNAdwu6xZekyoiVlUZEINcY=
github.com/docker/docker v23.0.1+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v23.0.3+incompatible h1:9GhVsShNWz1hO//9BNg/dpMnZW25KydO4wtVxWAIbho=
github.com/docker/docker v23.0.3+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.7.0 h1:xtCHsjxogADNZcdv1pKUHXryefjlVRqWqIhk/uXJp0A=
github.com/docker/docker-credential-helpers v0.7.0/go.mod h1:rETQfLdHNT3foU5kuNkFR1R1V12OJRRO5lzt2D1b5X0=
github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
@@ -237,8 +237,9 @@ github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM=
github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/snappy v0.0.2/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM=
@@ -257,12 +258,13 @@ github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-containerregistry v0.13.0 h1:y1C7Z3e149OJbOPDBxLYR8ITPz8dTKqQwjErKVHJC8k=
github.com/google/go-containerregistry v0.13.0/go.mod h1:J9FQ+eSS4a1aC2GNZxvNpbWhgp0487v+cgiilB4FqDo=
github.com/google/go-containerregistry v0.14.0 h1:z58vMqHxuwvAsVwvKEkmVBz2TlgBgH5k6koEXBtlYkw=
github.com/google/go-containerregistry v0.14.0/go.mod h1:aiJ2fp/SXvkWgmYHioXnbMdlgB8eXiiYOY55gfN91Wk=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/licensecheck v0.3.1 h1:QoxgoDkaeC4nFrtGN1jV7IPmDCHFNIVh54e5hSt6sPs=
github.com/google/licensecheck v0.3.1/go.mod h1:ORkR35t/JjW+emNKtfJDII0zlciG9JgbT7SmsohlHmY=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
@@ -293,8 +295,8 @@ github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pf
github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM=
github.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g=
github.com/gookit/color v1.2.5/go.mod h1:AhIE+pS6D4Ql0SQWbBeXPHw7gY0/sjHoA4s/n1KB7xg=
github.com/gookit/color v1.5.2 h1:uLnfXcaFjlrDnQDT+NCBcfhrXqYTx/rcCa6xn01Y8yI=
github.com/gookit/color v1.5.2/go.mod h1:w8h4bGiHeeBpvQVePTutdbERIUf3oJE5lZ8HM0UgXyg=
github.com/gookit/color v1.5.3 h1:twfIhZs4QLCtimkP7MOxlF3A0U/5cDPseRT9M/+2SCE=
github.com/gookit/color v1.5.3/go.mod h1:NUzwzeehUfl7GIb36pqId+UGmRfQcU/WiiyTTeNjHtE=
github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
github.com/hashicorp/consul/api v1.11.0/go.mod h1:XjsvQN+RJGWI2TWy1/kqaE16HrR2J/FWgkYjdZQsX9M=
github.com/hashicorp/consul/sdk v0.8.0/go.mod h1:GBvyrGALthsZObzUGsfgHZQDXjg4lOjagTIwIR1vPms=
@@ -342,8 +344,8 @@ github.com/imdario/mergo v0.3.11/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH
github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU=
github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc=
github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/invopop/jsonschema v0.7.0 h1:2vgQcBz1n256N+FpX3Jq7Y17AjYt46Ig3zIWyy770So=
github.com/invopop/jsonschema v0.7.0/go.mod h1:O9uiLokuu0+MGFlyiaqtWxwqJm41/+8Nj0lD7A36YH0=
github.com/jinzhu/copier v0.3.5 h1:GlvfUwHk62RokgqVNvYsku0TATCF7bAHVwEXoBh3iJg=
@@ -363,8 +365,8 @@ github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+o
github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.11.4/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.11.7/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.15.11 h1:Lcadnb3RKGin4FYM/orgq0qde+nc15E5Cbqg4B9Sx9c=
github.com/klauspost/compress v1.15.11/go.mod h1:QPwzmACJjUTFsnSHH934V6woptycfrDDJnH7hvFVbGM=
github.com/klauspost/compress v1.16.0 h1:iULayQNOReoYUe+1qtKOqw9CwJv3aNQu8ivo7lw1HU4=
github.com/klauspost/compress v1.16.0/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/klauspost/pgzip v1.2.5 h1:qnWYvvKqedOF2ulHpMG72XQol4ILEJ8k2wwRl/Km8oE=
github.com/klauspost/pgzip v1.2.5/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
@@ -515,19 +517,19 @@ github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spdx/gordf v0.0.0-20201111095634-7098f93598fb/go.mod h1:uKWaldnbMnjsSAXRurWqqrdyZen1R7kxl8TkmWk2OyM=
github.com/spdx/tools-golang v0.5.0-rc1 h1:ooCSe48QatlidqEFd+nSI308tyeNTR6NJvauUj3ApX8=
github.com/spdx/tools-golang v0.5.0-rc1/go.mod h1:LI6onw172PdO57Ob/hgnLDD4Y2PMnroeNT3wO/2WJJI=
github.com/spdx/tools-golang v0.5.0 h1:/fqihV2Jna7fmow65dHpgKNsilgLK7ICpd2tkCnPEyY=
github.com/spdx/tools-golang v0.5.0/go.mod h1:kkGlrSXXfHwuSzHQZJRV3aKu9ZXCq/MSf2+xyiJH1lM=
github.com/spf13/afero v1.3.3/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4=
github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I=
github.com/spf13/afero v1.9.3 h1:41FoI0fD7OR7mGcKE/aOiLkGreyf8ifIOQmJANWogMk=
github.com/spf13/afero v1.9.3/go.mod h1:iUV7ddyEEZPO5gA3zD4fJt6iStLlL+Lg4m2cihcDf8Y=
github.com/spf13/afero v1.9.5 h1:stMpOSZFs//0Lv29HduCmli3GUfpFoF3Y1Q/aXj/wVM=
github.com/spf13/afero v1.9.5/go.mod h1:UBogFpq8E9Hx+xc5CNTTEpTnuHVmXDwZcZcE1eb/UhQ=
github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cast v1.4.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cast v1.5.0 h1:rj3WzYc11XZaIZMPKmwP96zkFEnnAmV8s6XbB2aY32w=
github.com/spf13/cast v1.5.0/go.mod h1:SpXXQ5YoyJw6s3/6cMTQuxvgRl3PCJiyaX9p6b155UU=
github.com/spf13/cobra v1.3.0/go.mod h1:BrRVncBjOJa/eUcVVm9CE+oC6as8k+VYr4NY7WCi9V4=
github.com/spf13/cobra v1.6.1 h1:o94oiPyS4KD1mPy2fmcYYHHfCxLqYjJOhGsCHFZtEzA=
github.com/spf13/cobra v1.6.1/go.mod h1:IOw/AERYS7UzyrGinqmz6HLUo219MORXGxhbaJUqzrY=
github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I=
github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0=
github.com/spf13/jwalterweatherman v1.1.0 h1:ue6voC5bR5F8YxI5S67j9i582FU4Qvo2bmqnqMYADFk=
github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
@@ -550,8 +552,9 @@ github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/subosito/gotenv v1.4.2 h1:X1TuBLAMDFbaTAChgCBLu3DU3UPyELpnF2jjJ2cz/S8=
github.com/subosito/gotenv v1.4.2/go.mod h1:ayKnFf/c6rvx/2iiLrJUk1e6plDbT3edrFNGqEflhK0=
@@ -567,16 +570,16 @@ github.com/ulikunitz/xz v0.5.9/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oW
github.com/ulikunitz/xz v0.5.10 h1:t92gobL9l3HE202wg3rlk19F6X+JOxl9BBrCCMYEYd8=
github.com/ulikunitz/xz v0.5.10/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/urfave/cli v1.22.4/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/vbatts/go-mtree v0.5.2 h1:d8SAbLJiR1cR3pe1J+FBaalRkCQw95gP12/P+a9PUcA=
github.com/vbatts/go-mtree v0.5.2/go.mod h1:e0NDJ+bT3jG7ZINeB9HR5AxTvjskCsOR54+9KoaXyDc=
github.com/vbatts/go-mtree v0.5.3 h1:S/jYlfG8rZ+a0bhZd+RANXejy7M4Js8fq9U+XoWTd5w=
github.com/vbatts/go-mtree v0.5.3/go.mod h1:eXsdoPMdL2jcJx6HweWi9lYQxBsTp4lNhqqAjgkZUg8=
github.com/vbatts/tar-split v0.11.2 h1:Via6XqJr0hceW4wff3QRzD5gAk/tatMw/4ZA7cTlIME=
github.com/vbatts/tar-split v0.11.2/go.mod h1:vV3ZuO2yWSVsz+pfFzDG/upWH1JhjOiEaWq6kXyQ3VI=
github.com/vifraa/gopom v0.2.1 h1:MYVMAMyiGzXPPy10EwojzKIL670kl5Zbae+o3fFvQEM=
github.com/vifraa/gopom v0.2.1/go.mod h1:oPa1dcrGrtlO37WPDBm5SqHAT+wTgF8An1Q71Z6Vv4o=
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5 h1:phTLPgMRDYTizrBSKsNSOa2zthoC2KsJsaY/8sg3rD8=
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5/go.mod h1:JPirS5jde/CF5qIjcK4WX+eQmKXdPc6vcZkJ/P0hfPw=
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240 h1:r6BlIP7CVZtMlxUQhT40h1IE1TzEgKVqwmsVGuscvdk=
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240/go.mod h1:jLXFoL31zFaHKAAyZUh+sxiTDFe1L1ZHrcK2T1itVKA=
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5 h1:lwgTsTy18nYqASnH58qyfRW/ldj7Gt2zzBvgYPzdA4s=
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5/go.mod h1:jLXFoL31zFaHKAAyZUh+sxiTDFe1L1ZHrcK2T1itVKA=
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb h1:Yz6VVOcLuWLAHYlJzTw7JKnWxdV/WXpug2X0quEzRnY=
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb/go.mod h1:nDi3BAC5nEbVbg+WSJDHLbjHv0ZToq8nMPA97XMxF3E=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
@@ -626,7 +629,7 @@ golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWP
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.3.0 h1:a06MkbcxBrEFc0w0QIZWXrH/9cCX6KJyWbBOIwAn+7A=
golang.org/x/crypto v0.3.0/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
@@ -669,8 +672,8 @@ golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.5.0/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0 h1:LUYupSeNrTNCGzR/hVBk2NHZO4hXcVaW1k4Qx7rjPx8=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.10.0 h1:lFO9qtOdlre5W1jxS3r/4szv2/6iXxScdzjoBMXNhYk=
golang.org/x/mod v0.10.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -714,10 +717,11 @@ golang.org/x/net v0.0.0-20210410081132-afb366fc7cd1/go.mod h1:9tjilg8BloeKEkVJvy
golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210505024714-0287a6fb4125/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.9.0 h1:aWJ/m6xSmxWBx+V0XRHTlrYrPG56jKsLdTFmsSsCzOM=
golang.org/x/net v0.9.0/go.mod h1:d48xBJpPfHeWQsugry2m+kC02ZBRGRgulfHnEXEuWns=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -823,15 +827,15 @@ golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.7.0 h1:3jlCCIQZPdOYu1h8BkNvLz8Kgwtae2cagcG/VamtZRU=
golang.org/x/sys v0.7.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
golang.org/x/term v0.5.0 h1:n2a8QNdAb0sZNpU9R1ALUXBbY+w51fCQDN+7EdxNBsY=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.7.0 h1:BEvjmm5fURWqcfbSKTdpkDXYBrUS1c0m8agp14W48vQ=
golang.org/x/term v0.7.0/go.mod h1:P32HKFT3hSsZrRxla30E9HqToFYAQPCMs/zFMBUFqPY=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -842,8 +846,8 @@ golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0 h1:2sjJmO8cDvYveuX97RDLsxlyUxLl+GHoLxBiRdHllBE=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -907,8 +911,8 @@ golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.2.0 h1:G6AHpWxTMGY1KyEYoAQ5WTtIekUUvDNjan3ugu60JvE=
golang.org/x/tools v0.2.0/go.mod h1:y4OqIKeOV/fWJetJ8bXPU1sEVniLMIyDAZWeHdV+NTA=
golang.org/x/tools v0.7.0 h1:W4OVu8VVOaIO0yzWMNdepAulS7YfoS3Zabrm8DOXXU4=
golang.org/x/tools v0.7.0/go.mod h1:4pg6aUX35JBAogB10C9AtvVL+qowtN4pT3CGSQex14s=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -1064,8 +1068,8 @@ google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlba
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w=
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.29.0 h1:44S3JjaKmLEE4YIkjzexaP+NzZsudE3Zin5Njn/pYX0=
google.golang.org/protobuf v1.29.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+69 -27
View File
@@ -3,6 +3,7 @@ package config
import (
"errors"
"fmt"
"os"
"path"
"reflect"
"sort"
@@ -17,6 +18,7 @@ import (
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/pkg/cataloger"
golangCataloger "github.com/anchore/syft/syft/pkg/cataloger/golang"
)
var (
@@ -38,25 +40,27 @@ type Application struct {
ConfigPath string `yaml:"configPath,omitempty" json:"configPath" mapstructure:"config"`
Verbosity uint `yaml:"verbosity,omitempty" json:"verbosity" mapstructure:"verbosity"`
// -q, indicates to not show any status output to stderr (ETUI or logging UI)
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"`
Outputs []string `yaml:"output" json:"output" mapstructure:"output"` // -o, the format to use for output
OutputTemplatePath string `yaml:"output-template-path" json:"output-template-path" mapstructure:"output-template-path"` // -t template file to use for output
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Dev development `yaml:"dev" json:"dev" mapstructure:"dev"`
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Catalogers []string `yaml:"catalogers" json:"catalogers" mapstructure:"catalogers"`
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
FileMetadata FileMetadata `yaml:"file-metadata" json:"file-metadata" mapstructure:"file-metadata"`
FileClassification fileClassification `yaml:"file-classification" json:"file-classification" mapstructure:"file-classification"`
FileContents fileContents `yaml:"file-contents" json:"file-contents" mapstructure:"file-contents"`
Secrets secrets `yaml:"secrets" json:"secrets" mapstructure:"secrets"`
Registry registry `yaml:"registry" json:"registry" mapstructure:"registry"`
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
Name string `yaml:"name" json:"name" mapstructure:"name"`
Parallelism int `yaml:"parallelism" json:"parallelism" mapstructure:"parallelism"` // the number of catalog workers to run in parallel
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"`
Outputs []string `yaml:"output" json:"output" mapstructure:"output"` // -o, the format to use for output
OutputTemplatePath string `yaml:"output-template-path" json:"output-template-path" mapstructure:"output-template-path"` // -t template file to use for output
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Dev development `yaml:"dev" json:"dev" mapstructure:"dev"`
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Catalogers []string `yaml:"catalogers" json:"catalogers" mapstructure:"catalogers"`
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
Golang golang `yaml:"golang" json:"golang" mapstructure:"golang"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
FileMetadata FileMetadata `yaml:"file-metadata" json:"file-metadata" mapstructure:"file-metadata"`
FileClassification fileClassification `yaml:"file-classification" json:"file-classification" mapstructure:"file-classification"`
FileContents fileContents `yaml:"file-contents" json:"file-contents" mapstructure:"file-contents"`
Secrets secrets `yaml:"secrets" json:"secrets" mapstructure:"secrets"`
Registry registry `yaml:"registry" json:"registry" mapstructure:"registry"`
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
Name string `yaml:"name" json:"name" mapstructure:"name"`
Parallelism int `yaml:"parallelism" json:"parallelism" mapstructure:"parallelism"` // the number of catalog workers to run in parallel
DefaultImagePullSource string `yaml:"default-image-pull-source" json:"default-image-pull-source" mapstructure:"default-image-pull-source"` // specify default image pull source
}
func (cfg Application) ToCatalogerConfig() cataloger.Config {
@@ -68,6 +72,10 @@ func (cfg Application) ToCatalogerConfig() cataloger.Config {
},
Catalogers: cfg.Catalogers,
Parallelism: cfg.Parallelism,
Golang: golangCataloger.GoCatalogerOpts{
SearchLocalModCacheLicenses: cfg.Golang.SearchLocalModCacheLicenses,
LocalModCacheDir: cfg.Golang.LocalModCacheDir,
},
}
}
@@ -123,6 +131,12 @@ func (cfg *Application) parseConfigValues() error {
return err
}
}
if err := checkDefaultSourceValues(cfg.DefaultImagePullSource); err != nil {
return err
}
// check for valid default source options
// parse nested config options
// for each field in the configuration struct, see if the field implements the parser interface
// note: the app config is a pointer, so we need to grab the elements explicitly (to traverse the address)
@@ -185,6 +199,7 @@ func loadDefaultValues(v *viper.Viper) {
v.SetDefault("check-for-app-update", true)
v.SetDefault("catalogers", nil)
v.SetDefault("parallelism", 1)
v.SetDefault("default-image-pull-source", "")
// for each field in the configuration struct, see if the field implements the defaultValueLoader interface and invoke it if it does
value := reflect.ValueOf(Application{})
@@ -208,6 +223,7 @@ func (cfg Application) String() string {
return string(appaStr)
}
// nolint:funlen
func loadConfig(v *viper.Viper, configPath string) error {
var err error
// use explicitly the given user config
@@ -223,13 +239,26 @@ func loadConfig(v *viper.Viper, configPath string) error {
// start searching for valid configs in order...
// 1. look for .<appname>.yaml (in the current directory)
confFilePath := "." + internal.ApplicationName
// TODO: Remove this before v1.0.0
// See syft #1634
v.AddConfigPath(".")
v.SetConfigName("." + internal.ApplicationName)
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
} else if !errors.As(err, &viper.ConfigFileNotFoundError{}) {
return fmt.Errorf("unable to parse config=%q: %w", v.ConfigFileUsed(), err)
v.SetConfigName(confFilePath)
// check if config.yaml exists in the current directory
// DEPRECATED: this will be removed in v1.0.0
if _, err := os.Stat("config.yaml"); err == nil {
log.Warn("DEPRECATED: ./config.yaml as a configuration file is deprecated and will be removed as an option in v1.0.0, please rename to .syft.yaml")
}
if _, err := os.Stat(confFilePath + ".yaml"); err == nil {
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
} else if !errors.As(err, &viper.ConfigFileNotFoundError{}) {
return fmt.Errorf("unable to parse config=%q: %w", v.ConfigFileUsed(), err)
}
}
// 2. look for .<appname>/config.yaml (in the current directory)
@@ -256,11 +285,12 @@ func loadConfig(v *viper.Viper, configPath string) error {
}
// 4. look for <appname>/config.yaml in xdg locations (starting with xdg home config dir, then moving upwards)
v.AddConfigPath(path.Join(xdg.ConfigHome, internal.ApplicationName))
v.SetConfigName("config")
configPath = path.Join(xdg.ConfigHome, internal.ApplicationName)
v.AddConfigPath(configPath)
for _, dir := range xdg.ConfigDirs {
v.AddConfigPath(path.Join(dir, internal.ApplicationName))
}
v.SetConfigName("config")
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
@@ -269,3 +299,15 @@ func loadConfig(v *viper.Viper, configPath string) error {
}
return nil
}
var validDefaultSourceValues = []string{"registry", "docker", "podman", ""}
func checkDefaultSourceValues(source string) error {
validValues := internal.NewStringSet(validDefaultSourceValues...)
if !validValues.Contains(source) {
validValuesString := strings.Join(validDefaultSourceValues, ", ")
return fmt.Errorf("%s is not a valid default source; please use one of the following: %s''", source, validValuesString)
}
return nil
}
+129
View File
@@ -0,0 +1,129 @@
package config
import (
"fmt"
"os"
"path"
"testing"
"github.com/adrg/xdg"
"github.com/mitchellh/go-homedir"
"github.com/spf13/viper"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TODO: set negative case when config.yaml is no longer a valid option
func TestApplicationConfig(t *testing.T) {
// disable homedir package cache for testing
originalCacheOpt := homedir.DisableCache
homedir.DisableCache = true
t.Cleanup(func() {
homedir.DisableCache = originalCacheOpt
})
// ensure we have no side effects for xdg package for future tests
originalXDG := os.Getenv("XDG_CONFIG_HOME")
t.Cleanup(func() {
// note: we're not using t.Setenv since the effect we're trying to eliminate is within the xdg package
require.NoError(t, os.Setenv("XDG_CONFIG_HOME", originalXDG))
xdg.Reload()
})
// config is picked up at desired configuration paths
// VALID: .syft.yaml, .syft/config.yaml, ~/.syft.yaml, <XDG_CONFIG_HOME>/syft/config.yaml
// DEPRECATED: config.yaml is currently supported by
tests := []struct {
name string
setup func(t *testing.T) string
assertions func(t *testing.T, app *Application)
cleanup func()
}{
{
name: "explicit config",
setup: func(t *testing.T) string {
return "./test-fixtures/.syft.yaml"
}, // no-op for explicit config
assertions: func(t *testing.T, app *Application) {
assert.Equal(t, "test-explicit-config", app.File)
},
},
{
name: "current working directory named config",
setup: func(t *testing.T) string {
err := os.Chdir("./test-fixtures/config-wd-file") // change application cwd to test-fixtures
require.NoError(t, err)
return ""
},
assertions: func(t *testing.T, app *Application) {
assert.Equal(t, "test-wd-named-config", app.File)
},
},
{
name: "current working directory syft dir config",
setup: func(t *testing.T) string {
err := os.Chdir("./test-fixtures/config-dir-test") // change application cwd to test-fixtures
require.NoError(t, err)
return ""
},
assertions: func(t *testing.T, app *Application) {
assert.Equal(t, "test-dir-config", app.File)
},
},
{
name: "home directory file config",
setup: func(t *testing.T) string {
// Because Setenv affects the whole process, it cannot be used in parallel tests or
// tests with parallel ancestors: see separate XDG test for consequence of this
t.Setenv("HOME", "./test-fixtures/config-home-test/config-file")
return ""
},
assertions: func(t *testing.T, app *Application) {
assert.Equal(t, "test-home-config", app.File)
},
},
{
name: "XDG file config",
setup: func(t *testing.T) string {
wd, err := os.Getwd()
require.NoError(t, err)
configDir := path.Join(wd, "./test-fixtures/config-home-test") // set HOME to testdata
// note: this explicitly has multiple XDG paths, make certain we use the first VALID one (not the first one)
t.Setenv("XDG_CONFIG_DIRS", fmt.Sprintf("/another/foo/bar:%s", configDir))
xdg.Reload()
return ""
},
assertions: func(t *testing.T, app *Application) {
assert.Equal(t, "test-home-XDG-config", app.File)
},
cleanup: func() {
require.NoError(t, os.Unsetenv("XDG_CONFIG_DIRS"))
xdg.Reload()
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if test.cleanup != nil {
t.Cleanup(test.cleanup)
}
wd, err := os.Getwd()
require.NoError(t, err)
defer os.Chdir(wd) // reset working directory after test
application := &Application{}
viperInstance := viper.New()
// this will override home in case you are running this test locally and DO have a syft config
// in your home directory... now it will be ignored. Same for XDG_CONFIG_DIRS.
t.Setenv("HOME", "/foo/bar")
t.Setenv("XDG_CONFIG_DIRS", "/foo/bar")
xdg.Reload()
configPath := test.setup(t)
err = application.LoadAllValues(viperInstance, configPath)
require.NoError(t, err)
test.assertions(t, application)
})
}
}
+13
View File
@@ -0,0 +1,13 @@
package config
import "github.com/spf13/viper"
type golang struct {
SearchLocalModCacheLicenses bool `json:"search-local-mod-cache-licenses" yaml:"search-local-mod-cache-licenses" mapstructure:"search-local-mod-cache-licenses"`
LocalModCacheDir string `json:"local-mod-cache-dir" yaml:"local-mod-cache-dir" mapstructure:"local-mod-cache-dir"`
}
func (cfg golang) loadDefaultValues(v *viper.Viper) {
v.SetDefault("golang.search-local-mod-cache-licenses", false)
v.SetDefault("golang.local-mod-cache-dir", "")
}
+7
View File
@@ -0,0 +1,7 @@
# same as --file; write output report to a file (default is to write to stdout)
file: "test-explicit-config"
package:
cataloger:
scope: "squashed"
# same as --scope; limit the scope of the cataloger to only the specified types
@@ -0,0 +1,5 @@
# same as --file; write output report to a file (default is to write to stdout)
file: "test-dir-config"
package:
cataloger:
scope: "squashed"
@@ -0,0 +1,5 @@
# same as --file; write output report to a file (default is to write to stdout)
file: "test-home-config"
package:
cataloger:
scope: "squashed"
@@ -0,0 +1,5 @@
# same as --file; write output report to a file (default is to write to stdout)
file: "test-home-XDG-config"
package:
cataloger:
scope: "squashed"
@@ -0,0 +1,5 @@
# same as --file; write output report to a file (default is to write to stdout)
file: "test-wd-named-config"
package:
cataloger:
scope: "squashed"
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "6.2.0"
JSONSchemaVersion = "7.1.1"
)
+1 -4
View File
@@ -161,10 +161,7 @@ func UnzipToDir(archivePath, targetDir string) error {
return err
}
if err = extractSingleFile(file, joinedPath, archivePath); err != nil {
return err
}
return nil
return extractSingleFile(file, joinedPath, archivePath)
}
return TraverseFilesInZip(archivePath, visitor)
+53
View File
@@ -0,0 +1,53 @@
package licenses
import "github.com/anchore/syft/internal"
// all of these taken from https://github.com/golang/pkgsite/blob/8996ff632abee854aef1b764ca0501f262f8f523/internal/licenses/licenses.go#L338
// which unfortunately is not exported. But fortunately is under BSD-style license.
var (
FileNames = []string{
"COPYING",
"COPYING.md",
"COPYING.markdown",
"COPYING.txt",
"LICENCE",
"LICENCE.md",
"LICENCE.markdown",
"LICENCE.txt",
"LICENSE",
"LICENSE.md",
"LICENSE.markdown",
"LICENSE.txt",
"LICENSE-2.0.txt",
"LICENCE-2.0.txt",
"LICENSE-APACHE",
"LICENCE-APACHE",
"LICENSE-APACHE-2.0.txt",
"LICENCE-APACHE-2.0.txt",
"LICENSE-MIT",
"LICENCE-MIT",
"LICENSE.MIT",
"LICENCE.MIT",
"LICENSE.code",
"LICENCE.code",
"LICENSE.docs",
"LICENCE.docs",
"LICENSE.rst",
"LICENCE.rst",
"MIT-LICENSE",
"MIT-LICENCE",
"MIT-LICENSE.md",
"MIT-LICENCE.md",
"MIT-LICENSE.markdown",
"MIT-LICENCE.markdown",
"MIT-LICENSE.txt",
"MIT-LICENCE.txt",
"MIT_LICENSE",
"MIT_LICENCE",
"UNLICENSE",
"UNLICENCE",
}
FileNameSet = internal.NewStringSet(FileNames...)
)
+33
View File
@@ -0,0 +1,33 @@
package licenses
import (
"io"
"github.com/google/licensecheck"
"golang.org/x/exp/slices"
)
const (
coverageThreshold = 75
unknownLicenseType = "UNKNOWN"
)
// Parse scans the contents of a license file to attempt to determine the type of license it is
func Parse(reader io.Reader) (licenses []string, err error) {
contents, err := io.ReadAll(reader)
if err != nil {
return nil, err
}
cov := licensecheck.Scan(contents)
if cov.Percent < float64(coverageThreshold) {
licenses = append(licenses, unknownLicenseType)
}
for _, m := range cov.Match {
if slices.Contains(licenses, m.ID) {
continue
}
licenses = append(licenses, m.ID)
}
return
}
@@ -11,8 +11,6 @@ import (
"strings"
"text/template"
"time"
"github.com/scylladb/go-set/strset"
)
// This program generates license_list.go.
@@ -35,7 +33,7 @@ var licenseIDs = map[string]string{
}
`))
var versionMatch = regexp.MustCompile(`-([0-9]+)\.?([0-9]+)?\.?([0-9]+)?\.?`)
var versionMatch = regexp.MustCompile(`([0-9]+)\.?([0-9]+)?\.?([0-9]+)?\.?`)
func main() {
if err := run(); err != nil {
@@ -49,7 +47,6 @@ func run() error {
if err != nil {
return fmt.Errorf("unable to get licenses list: %w", err)
}
var result LicenseList
if err = json.NewDecoder(resp.Body).Decode(&result); err != nil {
return fmt.Errorf("unable to decode license list: %w", err)
@@ -103,58 +100,59 @@ func run() error {
// The third pass is for overwriting deprecated licenses with replacements, for example GPL-2.0+ is deprecated
// and now maps to GPL-2.0-or-later.
func processSPDXLicense(result LicenseList) map[string]string {
// first pass build map
var licenseIDs = make(map[string]string)
for _, l := range result.Licenses {
cleanID := strings.ToLower(l.ID)
if _, exists := licenseIDs[cleanID]; exists {
log.Fatalf("duplicate license ID found: %q", cleanID)
}
licenseIDs[cleanID] = l.ID
}
// The order of variations/permutations of a license ID matters because of we how shuffle its digits,
// that is because the permutation code can generate the same value for two difference licenses,
// for example: The licenses `ABC-1.0` and `ABC-1.1` can both map to `ABC-1`,
// so we need to guarantee the order they are created to avoid mapping them wrongly. So we use a sorted list.
// To overwrite deprecated licenses during the first pass we would later on rely on map order,
// [which in go is not consistent by design](https://stackoverflow.com/a/55925880).
// The order of variations/permutations of a license ID matter.
// The permutation code can generate the same value for two difference licenses,
// for example: The licenses `ABC-1.0` and `ABC-1.1` can both map to `ABC1`,
// we need to guarantee the order they are created to avoid mapping them incorrectly.
// To do this we use a sorted list.
sort.Slice(result.Licenses, func(i, j int) bool {
return result.Licenses[i].ID < result.Licenses[j].ID
})
// second pass to build exceptions and replacements
replaced := strset.New()
// keys are simplified by removing dashes and lowercasing ID
// this is so license declarations in the wild like: LGPL3 LGPL-3 lgpl3 and lgpl-3 can all match
licenseIDs := make(map[string]string)
for _, l := range result.Licenses {
var multipleID []string
cleanID := strings.ToLower(l.ID)
// licensePerms includes the cleanID in return slice
cleanID := cleanLicenseID(l.ID)
licensePerms := buildLicenseIDPermutations(cleanID)
var replacement *License
// if license is deprecated, find its replacement and add to licenseIDs
if l.Deprecated {
replacement = result.findReplacementLicense(l)
idToMap := l.ID
replacement := result.findReplacementLicense(l)
if replacement != nil {
licenseIDs[cleanID] = replacement.ID
idToMap = replacement.ID
}
// it's important to use the original licensePerms here so that the deprecated license
// can now point to the new correct license
for _, id := range licensePerms {
if _, exists := licenseIDs[id]; exists {
// can be used to debug duplicate license permutations and confirm that examples like GPL1
// do not point to GPL-1.1
// log.Println("duplicate license list permutation found when mapping deprecated license to replacement")
// log.Printf("already have key: %q for SPDX ID: %q; attempted to map replacement ID: %q for deprecated ID: %q\n", id, value, replacement.ID, l.ID)
continue
}
licenseIDs[id] = idToMap
}
}
multipleID = append(multipleID, buildLicensePermutations(cleanID)...)
for _, id := range multipleID {
// don't make replacements for IDs that have already been replaced. Since we have a sorted license list
// the earliest replacement is correct (any future replacements are not.
// e.g. replace lgpl-2 with LGPL-2.1-only is wrong, but with LGPL-2.0-only is correct)
if replacement == nil || replaced.Has(id) {
if _, exists := licenseIDs[id]; !exists {
licenseIDs[id] = l.ID
}
} else {
// a useful debugging line during builds
log.Printf("replacing %s with %s\n", id, replacement.ID)
licenseIDs[id] = replacement.ID
replaced.Add(id)
// if license is not deprecated, add all permutations to licenseIDs
for _, id := range licensePerms {
if _, exists := licenseIDs[id]; exists {
// log.Println("found duplicate license permutation key for non deprecated license")
// log.Printf("already have key: %q for SPDX ID: %q; tried to insert as SPDX ID:%q\n", id, value, l.ID)
continue
}
licenseIDs[id] = l.ID
}
}
return licenseIDs
}
func cleanLicenseID(id string) string {
cleanID := strings.ToLower(id)
return strings.ReplaceAll(cleanID, "-", "")
}
File diff suppressed because it is too large Load Diff
+55 -53
View File
@@ -20,51 +20,7 @@ type License struct {
SeeAlso []string `json:"seeAlso"`
}
func (l License) canReplace(other License) bool {
if l.Deprecated {
return false
}
// We want to replace deprecated licenses with non-deprecated counterparts
// For more information, see: https://github.com/spdx/license-list-XML/issues/1676
if other.Deprecated {
switch {
case strings.ReplaceAll(l.ID, "-only", "") == other.ID:
return true
case strings.ReplaceAll(l.ID, "-or-later", "+") == other.ID:
return true
case l.ID == "BSD-2-Clause" && other.ID == "BSD-2-Clause-NetBSD":
return true
case l.ID == "BSD-2-Clause-Views" && other.ID == "BSD-2-Clause-FreeBSD":
return true
case l.ID == "bzip2-1.0.6" && other.ID == "bzip2-1.0.5":
return true
case l.ID == "SMLNJ" && other.ID == "StandardML-NJ":
return true
}
}
if l.Name != other.Name {
return false
}
if l.OSIApproved != other.OSIApproved {
return false
}
if len(l.SeeAlso) != len(other.SeeAlso) {
return false
}
for i, sa := range l.SeeAlso {
if sa != other.SeeAlso[i] {
return false
}
}
return l.ID == other.ID
}
// findReplacementLicense returns a replacement license for a deprecated license
func (ll LicenseList) findReplacementLicense(deprecated License) *License {
for _, l := range ll.Licenses {
if l.canReplace(deprecated) {
@@ -75,16 +31,62 @@ func (ll LicenseList) findReplacementLicense(deprecated License) *License {
return nil
}
func buildLicensePermutations(license string) (perms []string) {
lv := findLicenseVersion(license)
vp := versionPermutations(lv)
version := strings.Join(lv, ".")
for _, p := range vp {
perms = append(perms, strings.Replace(license, version, p, 1))
func (l License) canReplace(deprecated License) bool {
// don't replace a license with a deprecated license
if l.Deprecated {
return false
}
return perms
// We want to replace deprecated licenses with non-deprecated counterparts
// For more information, see: https://github.com/spdx/license-list-XML/issues/1676
switch {
case strings.ReplaceAll(l.ID, "-only", "") == deprecated.ID:
return true
case strings.ReplaceAll(l.ID, "-or-later", "+") == deprecated.ID:
return true
case l.ID == "BSD-2-Clause" && deprecated.ID == "BSD-2-Clause-NetBSD":
return true
case l.ID == "BSD-2-Clause-Views" && deprecated.ID == "BSD-2-Clause-FreeBSD":
return true
case l.ID == "bzip2-1.0.6" && deprecated.ID == "bzip2-1.0.5":
return true
case l.ID == "SMLNJ" && deprecated.ID == "StandardML-NJ":
return true
}
if l.Name != deprecated.Name {
return false
}
if l.OSIApproved != deprecated.OSIApproved {
return false
}
if len(l.SeeAlso) != len(deprecated.SeeAlso) {
return false
}
for i, sa := range l.SeeAlso {
if sa != deprecated.SeeAlso[i] {
return false
}
}
return l.ID == deprecated.ID
}
func buildLicenseIDPermutations(cleanID string) (perms []string) {
lv := findLicenseVersion(cleanID)
vp := versionPermutations(lv)
permSet := strset.New()
version := strings.Join(lv, ".")
for _, p := range vp {
permSet.Add(strings.Replace(cleanID, version, p, 1))
}
permSet.Add(cleanID)
return permSet.List()
}
func findLicenseVersion(license string) (version []string) {
+20 -23
View File
@@ -67,56 +67,57 @@ func TestLicensePermutations(t *testing.T) {
{
"GPL-1-only",
[]string{
"GPL-1-only",
"GPL-1.0-only",
"GPL-1.0.0-only",
"gpl1only",
"gpl1.0only",
"gpl1.0.0only",
},
},
{
"GPL-2",
[]string{
"GPL-2",
"GPL-2.0",
"GPL-2.0.0",
"gpl2",
"gpl2.0",
"gpl2.0.0",
},
},
{
"GPL-2.0+",
[]string{
"GPL-2+",
"GPL-2.0+",
"GPL-2.0.0+",
"gpl2+",
"gpl2.0+",
"gpl2.0.0+",
},
},
{
"GPL-3.0.0-or-later",
[]string{
"GPL-3-or-later",
"GPL-3.0-or-later",
"GPL-3.0.0-or-later",
"gpl3orlater",
"gpl3.0orlater",
"gpl3.0.0orlater",
},
},
{
"abc-1.1",
[]string{
"abc-1",
"abc-1.1",
"abc-1.1.0",
"abc1",
"abc1.1",
"abc1.1.0",
},
},
{
"oldap-2.0",
[]string{
"oldap-2",
"oldap-2.0",
"oldap-2.0.0",
"oldap2",
"oldap2.0",
"oldap2.0.0",
},
},
}
for _, test := range tests {
t.Run(test.shortName, func(t *testing.T) {
perms := buildLicensePermutations(test.shortName)
cleanID := cleanLicenseID(test.shortName)
perms := buildLicenseIDPermutations(cleanID)
assert.ElementsMatch(t, test.permutations, perms)
})
}
@@ -183,10 +184,6 @@ func TestFindLicenseVersion(t *testing.T) {
"GPL-2",
[]string{"2"},
},
{
"bzip2-1",
[]string{"1"},
},
{
"php-3.01",
[]string{"3", "01"},
File diff suppressed because it is too large Load Diff
+13 -9
View File
@@ -18,16 +18,20 @@ const (
//go:generate go run ./generate
func ID(id string) (value, other string, exists bool) {
id = strings.TrimSpace(id)
// ignore blank strings or the joiner
if id == "" || id == "AND" {
return "", "", false
}
// ID returns the canonical license ID for the given license ID
// Note: this function is only concerned with returning a best match of an SPDX license ID
// SPDX Expressions will be handled by a parent package which will call this function
func ID(id string) (value string, exists bool) {
// first look for a canonical license
if value, exists := licenseIDs[strings.ToLower(id)]; exists {
return value, "", exists
if value, exists := licenseIDs[cleanLicenseID(id)]; exists {
return value, exists
}
// we did not find, so treat it as a separate license
return "", id, true
return "", false
}
func cleanLicenseID(id string) string {
id = strings.TrimSpace(id)
id = strings.ToLower(id)
return strings.ReplaceAll(id, "-", "")
}
File diff suppressed because it is too large Load Diff
+3 -3
View File
@@ -9,8 +9,8 @@ import (
func TestLicenceListIDs(t *testing.T) {
// do a sanity check on the generated data
assert.Equal(t, "0BSD", licenseIDs["0bsd"])
assert.Equal(t, "ZPL-2.1", licenseIDs["zpl-2.1"])
assert.Equal(t, "GPL-2.0-only", licenseIDs["gpl-2"])
assert.Equal(t, "GPL-2.0-or-later", licenseIDs["gpl-2+"])
assert.Equal(t, "ZPL-2.1", licenseIDs["zpl2.1"])
assert.Equal(t, "GPL-2.0-only", licenseIDs["gpl2"])
assert.Equal(t, "GPL-2.0-or-later", licenseIDs["gpl2+"])
assert.NotEmpty(t, Version)
}
+13 -27
View File
@@ -6,83 +6,70 @@ import (
"github.com/stretchr/testify/assert"
)
func TestIDParse(t *testing.T) {
func TestSPDXIDRecognition(t *testing.T) {
var tests = []struct {
shortName string
id string
other string
found bool
}{
{
"GPL-1-only",
"GPL-1.0-only",
"",
true,
},
{
"gpl1",
"GPL-1.0-only",
true,
},
{
"gpl-1",
"GPL-1.0-only",
true,
},
{
"GPL-2",
"GPL-2.0-only",
"",
true,
},
{
"GPL-2+",
"GPL-2.0-or-later",
"",
true,
},
{
"GPL-3.0.0-or-later",
"GPL-3.0-or-later",
"",
true,
},
{
"GPL-3-with-autoconf-exception",
"GPL-3.0-with-autoconf-exception",
"",
true,
},
{
"CC-by-nc-3-de",
"CC-BY-NC-3.0-DE",
"",
true,
},
// the below few cases are NOT expected, however, seem unavoidable given the current approach
{
"w3c-20150513.0.0",
"W3C-20150513",
"",
true,
},
{
"spencer-86.0.0",
"Spencer-86",
"",
true,
},
{
"unicode-dfs-2015.0.0",
"Unicode-DFS-2015",
"",
true,
},
{
"Unknown",
"",
"Unknown",
true,
},
{
" ",
"",
"",
false,
},
{
"AND",
"",
" ",
"",
false,
},
@@ -90,10 +77,9 @@ func TestIDParse(t *testing.T) {
for _, test := range tests {
t.Run(test.shortName, func(t *testing.T) {
value, other, exists := ID(test.shortName)
value, exists := ID(test.shortName)
assert.Equal(t, test.found, exists)
assert.Equal(t, test.id, value)
assert.Equal(t, test.other, other)
})
}
}
+1
View File
@@ -45,6 +45,7 @@ type artifactMetadataContainer struct {
Hackage pkg.HackageMetadata
Java pkg.JavaMetadata
KbPackage pkg.KbPackageMetadata
Nix pkg.NixStoreMetadata
NpmPackage pkg.NpmPackageJSONMetadata
NpmPackageLock pkg.NpmPackageLockJSONMetadata
MixLock pkg.MixLockMetadata
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+1 -2
View File
@@ -106,9 +106,8 @@ func stripSlashes(s string) string {
for i, c := range s {
if c == '\\' && i+1 < len(s) && strings.ContainsRune(allowedCPEPunctuation, rune(s[i+1])) {
continue
} else {
sb.WriteRune(c)
}
sb.WriteRune(c)
}
return sb.String()
}
+3 -5
View File
@@ -51,10 +51,10 @@ func (i *DigestsCataloger) Catalog(resolver source.FileResolver) (map[source.Coo
if err != nil {
return nil, err
}
prog.N++
prog.Increment()
results[location.Coordinates] = result
}
log.Debugf("file digests cataloger processed %d files", prog.N)
log.Debugf("file digests cataloger processed %d files", prog.Current())
prog.SetCompleted()
return results, nil
}
@@ -123,9 +123,7 @@ func CleanDigestAlgorithmName(name string) string {
func digestsCatalogingProgress(locations int64) (*progress.Stage, *progress.Manual) {
stage := &progress.Stage{}
prog := &progress.Manual{
Total: locations,
}
prog := progress.NewManual(locations)
bus.Publish(partybus.Event{
Type: event.FileDigestsCatalogerStarted,
+3 -5
View File
@@ -32,18 +32,16 @@ func (i *MetadataCataloger) Catalog(resolver source.FileResolver) (map[source.Co
}
results[location.Coordinates] = metadata
prog.N++
prog.Increment()
}
log.Debugf("file metadata cataloger processed %d files", prog.N)
log.Debugf("file metadata cataloger processed %d files", prog.Current())
prog.SetCompleted()
return results, nil
}
func metadataCatalogingProgress(locations int64) (*progress.Stage, *progress.Manual) {
stage := &progress.Stage{}
prog := &progress.Manual{
Total: locations,
}
prog := progress.NewManual(locations)
bus.Publish(partybus.Event{
Type: event.FileMetadataCatalogerStarted,
+4 -6
View File
@@ -55,12 +55,12 @@ func (i *SecretsCataloger) Catalog(resolver source.FileResolver) (map[source.Coo
return nil, err
}
if len(result) > 0 {
secretsDiscovered.N += int64(len(result))
secretsDiscovered.Add(int64(len(result)))
results[location.Coordinates] = result
}
prog.N++
prog.Increment()
}
log.Debugf("secrets cataloger discovered %d secrets", secretsDiscovered.N)
log.Debugf("secrets cataloger discovered %d secrets", secretsDiscovered.Current())
prog.SetCompleted()
return results, nil
}
@@ -139,9 +139,7 @@ type SecretsMonitor struct {
func secretsCatalogingProgress(locations int64) (*progress.Stage, *progress.Manual, *progress.Manual) {
stage := &progress.Stage{}
secretsDiscovered := &progress.Manual{}
prog := &progress.Manual{
Total: locations,
}
prog := progress.NewManual(locations)
bus.Publish(partybus.Event{
Type: event.SecretsCatalogerStarted,
@@ -10,14 +10,21 @@ import (
func encodeLicenses(p pkg.Package) *cyclonedx.Licenses {
lc := cyclonedx.Licenses{}
for _, licenseName := range p.Licenses {
if value, other, exists := spdxlicense.ID(licenseName); exists {
if value, exists := spdxlicense.ID(licenseName); exists {
lc = append(lc, cyclonedx.LicenseChoice{
License: &cyclonedx.License{
ID: value,
Name: other,
ID: value,
},
})
continue
}
// not found so append the licenseName as is
lc = append(lc, cyclonedx.LicenseChoice{
License: &cyclonedx.License{
Name: licenseName,
},
})
}
if len(lc) > 0 {
return &lc
+12 -6
View File
@@ -24,6 +24,12 @@ func License(p pkg.Package) string {
// take all licenses and assume an AND expression; for information about license expressions see https://spdx.github.io/spdx-spec/appendix-IV-SPDX-license-expressions/
parsedLicenses := parseLicenses(p.Licenses)
for i, v := range parsedLicenses {
if strings.HasPrefix(v, spdxlicense.LicenseRefPrefix) {
parsedLicenses[i] = SanitizeElementID(v)
}
}
if len(parsedLicenses) == 0 {
return NOASSERTION
}
@@ -33,12 +39,12 @@ func License(p pkg.Package) string {
func parseLicenses(raw []string) (parsedLicenses []string) {
for _, l := range raw {
if value, other, exists := spdxlicense.ID(l); exists {
parsed := value
if other != "" {
parsed = spdxlicense.LicenseRefPrefix + other
}
parsedLicenses = append(parsedLicenses, parsed)
if value, exists := spdxlicense.ID(l); exists {
parsedLicenses = append(parsedLicenses, value)
} else {
// we did not find a valid SPDX license ID so treat as separate license
otherLicense := spdxlicense.LicenseRefPrefix + l
parsedLicenses = append(parsedLicenses, otherLicense)
}
}
return
@@ -65,6 +65,17 @@ func Test_License(t *testing.T) {
},
expected: "GPL-2.0-only",
},
{
name: "includes valid LicenseRef-",
input: pkg.Package{
Licenses: []string{
"one thing first",
"two things/#$^second",
"MIT",
},
},
expected: "LicenseRef-one-thing-first AND LicenseRef-two-things----second AND MIT",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
@@ -45,6 +45,8 @@ func SourceInfo(p pkg.Package) string {
answer = "acquired package info from cabal or stack manifest files"
case pkg.HexPkg:
answer = "acquired package info from rebar3 or mix manifest file"
case pkg.NixPkg:
answer = "acquired package info from nix store path"
default:
answer = "acquired package info from the following paths"
}
@@ -199,6 +199,14 @@ func Test_SourceInfo(t *testing.T) {
"from rebar3 or mix manifest file",
},
},
{
input: pkg.Package{
Type: pkg.NixPkg,
},
expected: []string{
"from nix store path",
},
},
}
var pkgTypes []pkg.Type
for _, test := range tests {
@@ -9,6 +9,8 @@ import (
"time"
"github.com/spdx/tools-golang/spdx"
"golang.org/x/exp/maps"
"golang.org/x/exp/slices"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
@@ -514,19 +516,23 @@ func toFileTypes(metadata *source.FileMetadata) (ty []string) {
func toOtherLicenses(catalog *pkg.Catalog) []*spdx.OtherLicense {
licenses := map[string]bool{}
for _, pkg := range catalog.Sorted() {
for _, license := range parseLicenses(pkg.Licenses) {
for _, p := range catalog.Sorted() {
for _, license := range parseLicenses(p.Licenses) {
if strings.HasPrefix(license, spdxlicense.LicenseRefPrefix) {
licenses[license] = true
}
}
}
var result []*spdx.OtherLicense
for license := range licenses {
sorted := maps.Keys(licenses)
slices.Sort(sorted)
for _, license := range sorted {
// separate the actual ID from the prefix
name := strings.TrimPrefix(license, spdxlicense.LicenseRefPrefix)
result = append(result, &spdx.OtherLicense{
LicenseIdentifier: license,
LicenseIdentifier: SanitizeElementID(license),
LicenseName: name,
ExtractedText: NONE, // we probably should have some extracted text here, but this is good enough for now
})
@@ -369,7 +369,7 @@ func Test_fileIDsForPackage(t *testing.T) {
}
func Test_H1Digest(t *testing.T) {
sbom := sbom.SBOM{}
s := sbom.SBOM{}
tests := []struct {
name string
pkg pkg.Package
@@ -416,7 +416,7 @@ func Test_H1Digest(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
catalog := pkg.NewCatalog(test.pkg)
pkgs := toPackages(catalog, sbom)
pkgs := toPackages(catalog, s)
require.Len(t, pkgs, 1)
for _, p := range pkgs {
if test.expectedDigest == "" {
@@ -431,3 +431,67 @@ func Test_H1Digest(t *testing.T) {
})
}
}
func Test_OtherLicenses(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expected []*spdx.OtherLicense
}{
{
name: "no licenseRef",
pkg: pkg.Package{
Licenses: []string{
"MIT",
},
},
expected: nil,
},
{
name: "single licenseRef",
pkg: pkg.Package{
Licenses: []string{
"un known",
},
},
expected: []*spdx.OtherLicense{
{
LicenseIdentifier: "LicenseRef-un-known",
LicenseName: "un known",
ExtractedText: NONE,
},
},
},
{
name: "multiple licenseRef",
pkg: pkg.Package{
Licenses: []string{
"un known",
"not known %s",
"MIT",
},
},
expected: []*spdx.OtherLicense{
{
LicenseIdentifier: "LicenseRef-not-known--s",
LicenseName: "not known %s",
ExtractedText: NONE,
},
{
LicenseIdentifier: "LicenseRef-un-known",
LicenseName: "un known",
ExtractedText: NONE,
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
catalog := pkg.NewCatalog(test.pkg)
otherLicenses := toOtherLicenses(catalog)
require.Len(t, otherLicenses, len(test.expected))
require.Equal(t, test.expected, otherLicenses)
})
}
}
+1 -6
View File
@@ -85,14 +85,9 @@ func versionMatches(version string, match string) bool {
return true
}
dots := strings.Count(match, ".")
if dots == 0 {
match += ".*"
}
match = strings.ReplaceAll(match, ".", "\\.")
match = strings.ReplaceAll(match, "*", ".*")
match = strings.ReplaceAll(match, "\\..*", "(\\..*)*")
match = fmt.Sprintf("^%s$", match)
match = fmt.Sprintf("^%s(\\..*)*$", match)
matcher, err := regexp.Compile(match)
if err != nil {
return false
+49
View File
@@ -227,12 +227,61 @@ func Test_versionMatches(t *testing.T) {
match: "3.1",
matches: true,
},
{
name: "wildcard-version matches minor",
version: "7.1.3",
match: "7.*",
matches: true,
},
{
name: "wildcard-version matches patch",
version: "7.4.8",
match: "7.4.*",
matches: true,
},
{
name: "sub-version matches major",
version: "7.19.11",
match: "7",
matches: true,
},
{
name: "sub-version matches minor",
version: "7.55.2",
match: "7.55",
matches: true,
},
{
name: "sub-version matches patch",
version: "7.32.6",
match: "7.32.6",
matches: true,
},
// negative tests
{
name: "different number does not match",
version: "3",
match: "4",
matches: false,
},
{
name: "sub-version doesn't match major",
version: "7.2.5",
match: "8.2.5",
matches: false,
},
{
name: "sub-version doesn't match minor",
version: "7.2.9",
match: "7.1",
matches: false,
},
{
name: "sub-version doesn't match patch",
version: "7.32.6",
match: "7.32.5",
matches: false,
},
}
for _, test := range tests {
+1
View File
@@ -213,5 +213,6 @@ func TestEncodeFullJSONDocument(t *testing.T) {
s,
*updateJson,
true,
schemaVersionRedactor,
)
}
+2 -1
View File
@@ -1,6 +1,7 @@
package syftjson
import (
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/sbom"
)
@@ -8,7 +9,7 @@ const ID sbom.FormatID = "syft-json"
func Format() sbom.Format {
return sbom.NewFormat(
"6",
internal.JSONSchemaVersion,
encoder,
decoder,
validator,
@@ -24,6 +24,7 @@ type LinuxRelease struct {
BugReportURL string `json:"bugReportURL,omitempty"`
PrivacyPolicyURL string `json:"privacyPolicyURL,omitempty"`
CPEName string `json:"cpeName,omitempty"`
SupportEnd string `json:"supportEnd,omitempty"`
}
func (s *IDLikes) UnmarshalJSON(data []byte) error {
@@ -89,7 +89,7 @@
}
},
"schema": {
"version": "6.2.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.2.0.json"
"version": "7.1.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-7.1.1.json"
}
}
@@ -185,7 +185,7 @@
}
},
"schema": {
"version": "6.2.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.2.0.json"
"version": "7.1.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-7.1.1.json"
}
}
@@ -112,7 +112,7 @@
}
},
"schema": {
"version": "6.2.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.2.0.json"
"version": "7.1.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-7.1.1.json"
}
}
+1
View File
@@ -62,6 +62,7 @@ func toLinuxReleaser(d *linux.Release) model.LinuxRelease {
BugReportURL: d.BugReportURL,
PrivacyPolicyURL: d.PrivacyPolicyURL,
CPEName: d.CPEName,
SupportEnd: d.SupportEnd,
}
}
@@ -1,7 +1,6 @@
package syftjson
import (
"strings"
"testing"
"github.com/scylladb/go-set/strset"
@@ -9,16 +8,10 @@ import (
"github.com/stretchr/testify/require"
"github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/formats/syftjson/model"
"github.com/anchore/syft/syft/source"
)
func Test_SyftJsonID_Compatibility(t *testing.T) {
jsonMajorVersion := strings.Split(internal.JSONSchemaVersion, ".")[0]
assert.Equal(t, jsonMajorVersion, string(Format().Version()))
}
func Test_toSourceModel(t *testing.T) {
allSchemes := strset.New()
for _, s := range source.AllSchemes {
+1
View File
@@ -53,6 +53,7 @@ func toSyftLinuxRelease(d model.LinuxRelease) *linux.Release {
BugReportURL: d.BugReportURL,
PrivacyPolicyURL: d.PrivacyPolicyURL,
CPEName: d.CPEName,
SupportEnd: d.SupportEnd,
}
}
+1 -4
View File
@@ -70,13 +70,10 @@ func CatalogPackages(src *source.Source, cfg cataloger.Config) (*pkg.Catalog, []
}
catalog, relationships, err := cataloger.Catalog(resolver, release, cfg.Parallelism, catalogers...)
if err != nil {
return nil, nil, nil, err
}
relationships = append(relationships, newSourceRelationshipsFromCatalog(src, catalog)...)
return catalog, relationships, release, nil
return catalog, relationships, release, err
}
func newSourceRelationshipsFromCatalog(src *source.Source, c *pkg.Catalog) []artifact.Relationship {
+1
View File
@@ -125,6 +125,7 @@ func parseOsRelease(contents string) (*Release, error) {
BugReportURL: values["BUG_REPORT_URL"],
PrivacyPolicyURL: values["PRIVACY_POLICY_URL"],
CPEName: values["CPE_NAME"],
SupportEnd: values["SUPPORT_END"],
}
// don't allow for empty contents to result in a Release object being created
+7 -6
View File
@@ -89,19 +89,20 @@ func TestIdentifyRelease(t *testing.T) {
{
fixture: "test-fixtures/os/fedora",
release: &Release{
PrettyName: "Fedora 31 (Container Image)",
Name: "Fedora",
PrettyName: "Fedora Linux 36 (Container Image)",
Name: "Fedora Linux",
ID: "fedora",
IDLike: nil,
Version: "31 (Container Image)",
VersionID: "31",
Version: "36 (Container Image)",
VersionID: "36",
Variant: "Container Image",
VariantID: "container",
HomeURL: "https://fedoraproject.org/",
SupportURL: "https://fedoraproject.org/wiki/Communicating_and_getting_help",
SupportURL: "https://ask.fedoraproject.org/",
BugReportURL: "https://bugzilla.redhat.com/",
PrivacyPolicyURL: "https://fedoraproject.org/wiki/Legal:PrivacyPolicy",
CPEName: "cpe:/o:fedoraproject:fedora:31",
CPEName: "cpe:/o:fedoraproject:fedora:36",
SupportEnd: "2023-05-16",
},
},
{
+1
View File
@@ -19,6 +19,7 @@ type Release struct {
BugReportURL string
PrivacyPolicyURL string
CPEName string // A CPE name for the operating system, in URI binding syntax
SupportEnd string // The date at which support for this version of the OS ends.
}
func (r *Release) String() string {
@@ -1,21 +1,22 @@
NAME=Fedora
VERSION="31 (Container Image)"
NAME="Fedora Linux"
VERSION="36 (Container Image)"
ID=fedora
VERSION_ID=31
VERSION_ID=36
VERSION_CODENAME=""
PLATFORM_ID="platform:f31"
PRETTY_NAME="Fedora 31 (Container Image)"
ANSI_COLOR="0;34"
PLATFORM_ID="platform:f36"
PRETTY_NAME="Fedora Linux 36 (Container Image)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:31"
CPE_NAME="cpe:/o:fedoraproject:fedora:36"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f31/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f36/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=31
REDHAT_BUGZILLA_PRODUCT_VERSION=36
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=31
REDHAT_SUPPORT_PRODUCT_VERSION=36
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
SUPPORT_END=2023-05-16
VARIANT="Container Image"
VARIANT_ID=container
+2
View File
@@ -87,6 +87,8 @@ func (a *spaceDelimitedStringSlice) UnmarshalJSON(data []byte) error {
}
*a = s
return nil
case nil:
return nil
default:
return fmt.Errorf("invalid type for string array: %T", obj)
}
+9
View File
@@ -98,6 +98,15 @@ func TestApkMetadata_UnmarshalJSON(t *testing.T) {
Files: []ApkFileRecord{{Path: "/usr"}},
},
},
{
name: "null pullDependencies",
input: `{
"pullDependencies": null
}`,
want: ApkMetadata{
Dependencies: nil,
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
+8 -3
View File
@@ -1,7 +1,12 @@
package pkg
import "github.com/anchore/syft/syft/source"
type BinaryMetadata struct {
Classifier string `mapstructure:"Classifier" json:"classifier"`
RealPath string `mapstructure:"RealPath" json:"realPath"`
VirtualPath string `mapstructure:"VirtualPath" json:"virtualPath"`
Matches []ClassifierMatch `mapstructure:"Matches" json:"matches"`
}
type ClassifierMatch struct {
Classifier string `mapstructure:"Classifier" json:"classifier"`
Location source.Location `mapstructure:"Location" json:"location"`
}
@@ -24,6 +24,7 @@ func TestCataloger_Globs(t *testing.T) {
pkgtest.NewCatalogTester().
FromDirectory(t, test.fixture).
ExpectsResolverContentQueries(test.expected).
IgnoreUnfulfilledPathResponses("etc/apk/repositories").
TestCataloger(t, NewApkdbCataloger())
})
}
+4 -23
View File
@@ -9,10 +9,6 @@ import (
"github.com/anchore/syft/syft/source"
)
var (
prefixes = []string{"py-", "py2-", "py3-", "ruby-"}
)
func newPackage(d pkg.ApkMetadata, release *linux.Release, locations ...source.Location) pkg.Package {
p := pkg.Package{
Name: d.Package,
@@ -30,24 +26,9 @@ func newPackage(d pkg.ApkMetadata, release *linux.Release, locations ...source.L
return p
}
func generateUpstream(m pkg.ApkMetadata) string {
if m.OriginPackage != "" && m.OriginPackage != m.Package {
return m.OriginPackage
}
for _, p := range prefixes {
if strings.HasPrefix(m.Package, p) {
return strings.TrimPrefix(m.Package, p)
}
}
return m.Package
}
// packageURL returns the PURL for the specific Alpine package (see https://github.com/package-url/purl-spec)
func packageURL(m pkg.ApkMetadata, distro *linux.Release) string {
if distro == nil || distro.ID != "alpine" {
// note: there is no namespace variation (like with debian ID_LIKE for ubuntu ID, for example)
if distro == nil {
return ""
}
@@ -55,13 +36,13 @@ func packageURL(m pkg.ApkMetadata, distro *linux.Release) string {
pkg.PURLQualifierArch: m.Architecture,
}
if m.OriginPackage != "" {
qualifiers[pkg.PURLQualifierUpstream] = generateUpstream(m)
if m.OriginPackage != m.Package {
qualifiers[pkg.PURLQualifierUpstream] = m.OriginPackage
}
return packageurl.NewPackageURL(
packageurl.TypeAlpine,
"alpine",
strings.ToLower(distro.ID),
m.Package,
m.Version,
pkg.PURLQualifiers(
+9 -24
View File
@@ -20,7 +20,7 @@ func Test_PackageURL(t *testing.T) {
expected string
}{
{
name: "bad distro",
name: "non-alpine distro",
metadata: pkg.ApkMetadata{
Package: "p",
Version: "v",
@@ -30,7 +30,7 @@ func Test_PackageURL(t *testing.T) {
ID: "something else",
VersionID: "3.4.6",
},
expected: "",
expected: "pkg:apk/something%20else/p@v?arch=a&distro=something%20else-3.4.6",
},
{
name: "gocase",
@@ -97,32 +97,17 @@ func Test_PackageURL(t *testing.T) {
expected: "pkg:apk/alpine/p@v?arch=a&upstream=origin&distro=alpine-3.4.6",
},
{
name: "upstream python package information as qualifier",
name: "wolfi distro",
metadata: pkg.ApkMetadata{
Package: "py3-potatoes",
Version: "v",
Architecture: "a",
OriginPackage: "py3-potatoes",
Package: "p",
Version: "v",
Architecture: "a",
},
distro: linux.Release{
ID: "alpine",
VersionID: "3.4.6",
ID: "wolfi",
VersionID: "20221230",
},
expected: "pkg:apk/alpine/py3-potatoes@v?arch=a&upstream=potatoes&distro=alpine-3.4.6",
},
{
name: "python package with origin package as upstream",
metadata: pkg.ApkMetadata{
Package: "py3-non-existant",
Version: "v",
Architecture: "a",
OriginPackage: "abcdefg",
},
distro: linux.Release{
ID: "alpine",
VersionID: "3.4.6",
},
expected: "pkg:apk/alpine/py3-non-existant@v?arch=a&upstream=abcdefg&distro=alpine-3.4.6",
expected: "pkg:apk/wolfi/p@v?arch=a&distro=wolfi-20221230",
},
}
+73 -2
View File
@@ -3,7 +3,9 @@ package apkdb
import (
"bufio"
"fmt"
"io"
"path"
"regexp"
"strconv"
"strings"
@@ -20,11 +22,15 @@ import (
// integrity check
var _ generic.Parser = parseApkDB
var (
repoRegex = regexp.MustCompile(`(?m)^https://.*\.alpinelinux\.org/alpine/v([^/]+)/([a-zA-Z0-9_]+)$`)
)
// parseApkDB parses packages from a given APK installed DB file. For more
// information on specific fields, see https://wiki.alpinelinux.org/wiki/Apk_spec.
//
//nolint:funlen
func parseApkDB(_ source.FileResolver, env *generic.Environment, reader source.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
//nolint:funlen,gocognit
func parseApkDB(resolver source.FileResolver, env *generic.Environment, reader source.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
scanner := bufio.NewScanner(reader)
var apks []pkg.ApkMetadata
@@ -101,6 +107,19 @@ func parseApkDB(_ source.FileResolver, env *generic.Environment, reader source.L
if env != nil {
r = env.LinuxRelease
}
// this is somewhat ugly, but better than completely failing when we can't find the release,
// e.g. embedded deeper in the tree, like containers or chroots.
// but we now have no way of handling different repository sources. On the other hand,
// we never could before this. At least now, we can handle some.
// This should get fixed with https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10875
if r == nil {
// find the repositories file from the relative directory of the DB file
releases := findReleases(resolver, reader.Location.RealPath)
if len(releases) > 0 {
r = &releases[0]
}
}
pkgs := make([]pkg.Package, 0, len(apks))
for _, apk := range apks {
@@ -110,6 +129,58 @@ func parseApkDB(_ source.FileResolver, env *generic.Environment, reader source.L
return pkgs, discoverPackageDependencies(pkgs), nil
}
func findReleases(resolver source.FileResolver, dbPath string) []linux.Release {
if resolver == nil {
return nil
}
reposLocation := path.Clean(path.Join(path.Dir(dbPath), "../../../etc/apk/repositories"))
locations, err := resolver.FilesByPath(reposLocation)
if err != nil {
log.Tracef("unable to find APK repositories file %q: %+v", reposLocation, err)
return nil
}
if len(locations) == 0 {
return nil
}
location := locations[0]
reposReader, err := resolver.FileContentsByLocation(location)
if err != nil {
log.Tracef("unable to fetch contents for APK repositories file %q: %+v", reposLocation, err)
return nil
}
return parseReleasesFromAPKRepository(source.LocationReadCloser{
Location: location,
ReadCloser: reposReader,
})
}
func parseReleasesFromAPKRepository(reader source.LocationReadCloser) []linux.Release {
var releases []linux.Release
reposB, err := io.ReadAll(reader)
if err != nil {
log.Tracef("unable to read APK repositories file %q: %+v", reader.Location.RealPath, err)
return nil
}
parts := repoRegex.FindAllStringSubmatch(string(reposB), -1)
for _, part := range parts {
if len(part) >= 3 {
releases = append(releases, linux.Release{
Name: "Alpine Linux",
ID: "alpine",
VersionID: part[1],
})
}
}
return releases
}
func parseApkField(line string) *apkField {
parts := strings.SplitN(line, ":", 2)
if len(parts) != 2 {
@@ -1,8 +1,10 @@
package apkdb
import (
"io"
"os"
"path/filepath"
"strings"
"testing"
"github.com/google/go-cmp/cmp"
@@ -1186,3 +1188,49 @@ func Test_stripVersionSpecifier(t *testing.T) {
})
}
}
func TestParseReleasesFromAPKRepository(t *testing.T) {
tests := []struct {
repos string
want []linux.Release
desc string
}{
{
"https://foo.alpinelinux.org/alpine/v3.14/main",
[]linux.Release{
{Name: "Alpine Linux", ID: "alpine", VersionID: "3.14"},
},
"single repo",
},
{
`https://foo.alpinelinux.org/alpine/v3.14/main
https://foo.alpinelinux.org/alpine/v3.14/community`,
[]linux.Release{
{Name: "Alpine Linux", ID: "alpine", VersionID: "3.14"},
{Name: "Alpine Linux", ID: "alpine", VersionID: "3.14"},
},
"multiple repos",
},
{
``,
nil,
"empty",
},
{
`https://foo.bar.org/alpine/v3.14/main
https://foo.them.org/alpine/v3.14/community`,
nil,
"invalid repos",
},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
reposReader := io.NopCloser(strings.NewReader(tt.repos))
got := parseReleasesFromAPKRepository(source.LocationReadCloser{
Location: source.NewLocation("test"),
ReadCloser: reposReader,
})
assert.Equal(t, tt.want, got)
})
}
}
+30 -25
View File
@@ -33,49 +33,54 @@ func (c Cataloger) Catalog(resolver source.FileResolver) ([]pkg.Package, []artif
for _, cls := range defaultClassifiers {
log.WithFields("classifier", cls.Class).Trace("cataloging binaries")
pkgs, err := catalog(resolver, cls)
newPkgs, err := catalog(resolver, cls)
if err != nil {
log.WithFields("error", err, "classifier", cls.Class).Warn("unable to catalog binary package: %w", err)
continue
}
packages = append(packages, pkgs...)
newPackages:
for i := range newPkgs {
newPkg := &newPkgs[i]
for j := range packages {
p := &packages[j]
// consolidate identical packages found in different locations or by different classifiers
if packagesMatch(p, newPkg) {
mergePackages(p, newPkg)
continue newPackages
}
}
packages = append(packages, *newPkg)
}
}
return packages, relationships, nil
}
func catalog(resolver source.FileResolver, cls classifier) ([]pkg.Package, error) {
var pkgs []pkg.Package
// mergePackages merges information from the extra package into the target package
func mergePackages(target *pkg.Package, extra *pkg.Package) {
// add the locations
target.Locations.Add(extra.Locations.ToSlice()...)
// update the metadata to indicate which classifiers were used
meta, _ := target.Metadata.(pkg.BinaryMetadata)
if m, ok := extra.Metadata.(pkg.BinaryMetadata); ok {
meta.Matches = append(meta.Matches, m.Matches...)
}
target.Metadata = meta
}
func catalog(resolver source.FileResolver, cls classifier) (packages []pkg.Package, err error) {
locations, err := resolver.FilesByGlob(cls.FileGlob)
if err != nil {
return nil, err
}
for _, location := range locations {
reader, err := resolver.FileContentsByLocation(location)
pkgs, err := cls.EvidenceMatcher(resolver, cls, location)
if err != nil {
return nil, err
}
locationReader := source.NewLocationReadCloser(location, reader)
newPkgs, err := cls.EvidenceMatcher(cls, locationReader)
if err != nil {
return nil, err
}
newPackages:
for i := range newPkgs {
newPkg := &newPkgs[i]
for j := range pkgs {
p := &pkgs[j]
// consolidate identical packages found in different locations,
// but continue to track each location
if packagesMatch(p, newPkg) {
p.Locations.Add(newPkg.Locations.ToSlice()...)
continue newPackages
}
}
pkgs = append(pkgs, *newPkg)
}
packages = append(packages, pkgs...)
}
return pkgs, nil
return packages, nil
}
// packagesMatch returns true if the binary packages "match" based on basic criteria
+538 -158
View File
@@ -2,9 +2,13 @@ package binary
import (
"errors"
"fmt"
"io"
"strings"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -27,10 +31,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "15beta4",
Type: "binary",
PURL: "pkg:generic/postgresql@15beta4",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
Locations: locations("postgres"),
Metadata: metadata("postgresql-binary"),
},
},
{
@@ -41,10 +43,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "15.1",
Type: "binary",
PURL: "pkg:generic/postgresql@15.1",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
Locations: locations("postgres"),
Metadata: metadata("postgresql-binary"),
},
},
{
@@ -55,10 +55,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "9.6.24",
Type: "binary",
PURL: "pkg:generic/postgresql@9.6.24",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
Locations: locations("postgres"),
Metadata: metadata("postgresql-binary"),
},
},
{
@@ -69,10 +67,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "9.5alpha1",
Type: "binary",
PURL: "pkg:generic/postgresql@9.5alpha1",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
Locations: locations("postgres"),
Metadata: metadata("postgresql-binary"),
},
},
{
@@ -83,10 +79,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "2.9.6",
Type: "binary",
PURL: "pkg:generic/traefik@2.9.6",
Locations: singleLocation("traefik"),
Metadata: pkg.BinaryMetadata{
Classifier: "traefik-binary",
},
Locations: locations("traefik"),
Metadata: metadata("traefik-binary"),
},
},
{
@@ -97,10 +91,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "1.7.34",
Type: "binary",
PURL: "pkg:generic/traefik@1.7.34",
Locations: singleLocation("traefik"),
Metadata: pkg.BinaryMetadata{
Classifier: "traefik-binary",
},
Locations: locations("traefik"),
Metadata: metadata("traefik-binary"),
},
},
{
@@ -111,10 +103,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "1.6.18",
Type: "binary",
PURL: "pkg:generic/memcached@1.6.18",
Locations: singleLocation("memcached"),
Metadata: pkg.BinaryMetadata{
Classifier: "memcached-binary",
},
Locations: locations("memcached"),
Metadata: metadata("memcached-binary"),
},
},
{
@@ -125,10 +115,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "2.4.54",
Type: "binary",
PURL: "pkg:generic/httpd@2.4.54",
Locations: singleLocation("httpd"),
Metadata: pkg.BinaryMetadata{
Classifier: "httpd-binary",
},
Locations: locations("httpd"),
Metadata: metadata("httpd-binary"),
},
},
{
@@ -139,10 +127,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "8.2.1",
Type: "binary",
PURL: "pkg:generic/php-cli@8.2.1",
Locations: singleLocation("php"),
Metadata: pkg.BinaryMetadata{
Classifier: "php-cli-binary",
},
Locations: locations("php"),
Metadata: metadata("php-cli-binary"),
},
},
{
@@ -153,10 +139,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "8.2.1",
Type: "binary",
PURL: "pkg:generic/php-fpm@8.2.1",
Locations: singleLocation("php-fpm"),
Metadata: pkg.BinaryMetadata{
Classifier: "php-fpm-binary",
},
Locations: locations("php-fpm"),
Metadata: metadata("php-fpm-binary"),
},
},
{
@@ -167,10 +151,80 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "8.2.1",
Type: "binary",
PURL: "pkg:generic/php@8.2.1",
Locations: singleLocation("libphp.so"),
Metadata: pkg.BinaryMetadata{
Classifier: "php-apache-binary",
},
Locations: locations("libphp.so"),
Metadata: metadata("php-apache-binary"),
},
},
{
name: "positive-perl-5.12.5",
fixtureDir: "test-fixtures/classifiers/positive/perl-5.12.5",
expected: pkg.Package{
Name: "perl",
Version: "5.12.5",
Type: "binary",
PURL: "pkg:generic/perl@5.12.5",
Locations: locations("perl"),
Metadata: metadata("perl-binary"),
},
},
{
name: "positive-perl-5.20.0",
fixtureDir: "test-fixtures/classifiers/positive/perl-5.20.0",
expected: pkg.Package{
Name: "perl",
Version: "5.20.0",
Type: "binary",
PURL: "pkg:generic/perl@5.20.0",
Locations: locations("perl"),
Metadata: metadata("perl-binary"),
},
},
{
name: "positive-perl-5.37.8",
fixtureDir: "test-fixtures/classifiers/positive/perl-5.37.8",
expected: pkg.Package{
Name: "perl",
Version: "5.37.8",
Type: "binary",
PURL: "pkg:generic/perl@5.37.8",
Locations: locations("perl"),
Metadata: metadata("perl-binary"),
},
},
{
name: "positive-haproxy-1.5.14",
fixtureDir: "test-fixtures/classifiers/positive/haproxy-1.5.14",
expected: pkg.Package{
Name: "haproxy",
Version: "1.5.14",
Type: "binary",
PURL: "pkg:generic/haproxy@1.5.14",
Locations: locations("haproxy"),
Metadata: metadata("haproxy-binary"),
},
},
{
name: "positive-haproxy-1.8.22",
fixtureDir: "test-fixtures/classifiers/positive/haproxy-1.8.22",
expected: pkg.Package{
Name: "haproxy",
Version: "1.8.22",
Type: "binary",
PURL: "pkg:generic/haproxy@1.8.22",
Locations: locations("haproxy"),
Metadata: metadata("haproxy-binary"),
},
},
{
name: "positive-haproxy-2.7.3",
fixtureDir: "test-fixtures/classifiers/positive/haproxy-2.7.3",
expected: pkg.Package{
Name: "haproxy",
Version: "2.7.3",
Type: "binary",
PURL: "pkg:generic/haproxy@2.7.3",
Locations: locations("haproxy"),
Metadata: metadata("haproxy-binary"),
},
},
{
@@ -181,10 +235,116 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "2.8.23",
Type: "binary",
PURL: "pkg:generic/redis@2.8.23",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
Locations: locations("redis-server"),
Metadata: metadata("redis-binary"),
},
},
{
name: "positive-argocd-2.5.11",
fixtureDir: "test-fixtures/classifiers/dynamic/argocd-2.5.11",
expected: pkg.Package{
Name: "argocd",
Version: "2.5.11",
Type: "binary",
PURL: "pkg:golang/github.com/argoproj/argo-cd@2.5.11",
Locations: locations("argocd"),
Metadata: metadata("argocd"),
},
},
{
name: "positive-argocd-2.6.4",
fixtureDir: "test-fixtures/classifiers/dynamic/argocd-2.6.4",
expected: pkg.Package{
Name: "argocd",
Version: "2.6.4",
Type: "binary",
PURL: "pkg:golang/github.com/argoproj/argo-cd@2.6.4",
Locations: locations("argocd"),
Metadata: metadata("argocd"),
},
},
{
name: "positive-helm-3.11.1",
fixtureDir: "test-fixtures/classifiers/dynamic/helm-3.11.1",
expected: pkg.Package{
Name: "helm",
Version: "3.11.1",
Type: "binary",
PURL: "pkg:golang/helm.sh/helm@3.11.1",
Locations: locations("helm"),
Metadata: metadata("helm"),
},
},
{
name: "positive-helm-3.10.3",
fixtureDir: "test-fixtures/classifiers/dynamic/helm-3.10.3",
expected: pkg.Package{
Name: "helm",
Version: "3.10.3",
Type: "binary",
PURL: "pkg:golang/helm.sh/helm@3.10.3",
Locations: locations("helm"),
Metadata: metadata("helm"),
},
},
{
name: "positive-kubectl-1.24.11",
fixtureDir: "test-fixtures/classifiers/dynamic/kubectl-1.24.11",
expected: pkg.Package{
Name: "kubectl",
Version: "1.24.11",
Type: "binary",
PURL: "pkg:golang/k8s.io/kubectl@1.24.11",
Locations: locations("kubectl"),
Metadata: metadata("kubectl"),
},
},
{
name: "positive-kubectl-1.25.7",
fixtureDir: "test-fixtures/classifiers/dynamic/kubectl-1.25.7",
expected: pkg.Package{
Name: "kubectl",
Version: "1.25.7",
Type: "binary",
PURL: "pkg:golang/k8s.io/kubectl@1.25.7",
Locations: locations("kubectl"),
Metadata: metadata("kubectl"),
},
},
{
name: "positive-kubectl-1.26.2",
fixtureDir: "test-fixtures/classifiers/dynamic/kubectl-1.26.2",
expected: pkg.Package{
Name: "kubectl",
Version: "1.26.2",
Type: "binary",
PURL: "pkg:golang/k8s.io/kubectl@1.26.2",
Locations: locations("kubectl"),
Metadata: metadata("kubectl"),
},
},
{
name: "positive-kustomize-4.5.7",
fixtureDir: "test-fixtures/classifiers/dynamic/kustomize-4.5.7",
expected: pkg.Package{
Name: "kustomize",
Version: "4.5.7",
Type: "binary",
PURL: "pkg:golang/sigs.k8s.io/kustomize@4.5.7",
Locations: locations("kustomize"),
Metadata: metadata("kustomize"),
},
},
{
name: "positive-kustomize-5.0.0",
fixtureDir: "test-fixtures/classifiers/dynamic/kustomize-5.0.0",
expected: pkg.Package{
Name: "kustomize",
Version: "5.0.0",
Type: "binary",
PURL: "pkg:golang/sigs.k8s.io/kustomize@5.0.0",
Locations: locations("kustomize"),
Metadata: metadata("kustomize"),
},
},
{
@@ -195,10 +355,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "4.0.11",
Type: "binary",
PURL: "pkg:generic/redis@4.0.11",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
Locations: locations("redis-server"),
Metadata: metadata("redis-binary"),
},
},
{
@@ -209,10 +367,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "5.0.0",
Type: "binary",
PURL: "pkg:generic/redis@5.0.0",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
Locations: locations("redis-server"),
Metadata: metadata("redis-binary"),
},
},
{
@@ -223,10 +379,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "6.0.16",
Type: "binary",
PURL: "pkg:generic/redis@6.0.16",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
Locations: locations("redis-server"),
Metadata: metadata("redis-binary"),
},
},
{
@@ -237,101 +391,168 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "7.0.0",
Type: "binary",
PURL: "pkg:generic/redis@7.0.0",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
Locations: locations("redis-server"),
Metadata: metadata("redis-binary"),
},
},
{
name: "positive-libpython3.7.so",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/python-binary-lib-3.7",
expected: pkg.Package{
Name: "python",
Version: "3.7.4a-vZ9",
PURL: "pkg:generic/python@3.7.4a-vZ9",
Locations: singleLocation("libpython3.7.so"),
Version: "3.7.4",
PURL: "pkg:generic/python@3.7.4",
Locations: locations("libpython3.7.so"),
Metadata: metadata("python-binary-lib"),
},
},
{
name: "positive-python-3.11.2-from-shared-lib",
fixtureDir: "test-fixtures/classifiers/dynamic/python-binary-shared-lib-3.11",
expected: pkg.Package{
Name: "python",
Version: "3.11.2",
PURL: "pkg:generic/python@3.11.2",
Locations: locations("python3", "libpython3.11.so.1.0"),
Metadata: pkg.BinaryMetadata{
Classifier: "python-binary-lib",
Matches: []pkg.ClassifierMatch{
match("python-binary", "python3"),
match("python-binary", "libpython3.11.so.1.0"),
match("python-binary-lib", "libpython3.11.so.1.0"),
},
},
},
},
{
name: "positive-python-3.9-from-shared-redhat-lib",
fixtureDir: "test-fixtures/classifiers/dynamic/python-binary-shared-lib-redhat-3.9",
expected: pkg.Package{
Name: "python",
Version: "3.9.13",
PURL: "pkg:generic/python@3.9.13",
Locations: locations("python3.9", "libpython3.9.so.1.0"),
Metadata: pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match("python-binary", "python3.9"),
match("python-binary", "libpython3.9.so.1.0"),
match("python-binary-lib", "libpython3.9.so.1.0"),
},
},
},
},
{
name: "positive-python-binary-with-version-3.9",
fixtureDir: "test-fixtures/classifiers/dynamic/python-binary-with-version-3.9",
expected: pkg.Package{
Name: "python",
Version: "3.9.2",
PURL: "pkg:generic/python@3.9.2",
Locations: locations("python3.9"),
Metadata: pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match("python-binary", "python3.9"),
},
},
},
},
{
name: "positive-python-binary-3.4-alpine",
fixtureDir: "test-fixtures/classifiers/dynamic/python-binary-3.4-alpine",
expected: pkg.Package{
Name: "python",
Version: "3.4.10",
PURL: "pkg:generic/python@3.4.10",
Locations: locations("python3.4", "libpython3.4m.so.1.0"),
Metadata: pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match("python-binary", "python3.4"),
match("python-binary", "libpython3.4m.so.1.0"),
match("python-binary-lib", "libpython3.4m.so.1.0"),
},
},
},
},
{
name: "positive-python-3.5-with-incorrect-match",
fixtureDir: "test-fixtures/classifiers/positive/python-3.5-with-incorrect-match",
expected: pkg.Package{
Name: "python",
Version: "3.5.3",
PURL: "pkg:generic/python@3.5.3",
Locations: locations("python3.5"),
Metadata: metadata("python-binary"),
},
},
{
name: "positive-python3.6",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/python-binary-3.6",
expected: pkg.Package{
Name: "python",
Version: "3.6.3a-vZ9",
PURL: "pkg:generic/python@3.6.3a-vZ9",
Locations: singleLocation("python3.6"),
Metadata: pkg.BinaryMetadata{
Classifier: "python-binary",
},
Version: "3.6.3",
PURL: "pkg:generic/python@3.6.3",
Locations: locations("python3.6"),
Metadata: metadata("python-binary"),
},
},
{
name: "positive-patchlevel.h",
fixtureDir: "test-fixtures/classifiers/positive",
name: "positive-python-duplicates",
fixtureDir: "test-fixtures/classifiers/positive/python-duplicates",
expected: pkg.Package{
Name: "python",
Version: "3.9-aZ5",
PURL: "pkg:generic/python@3.9-aZ5",
Locations: singleLocation("patchlevel.h"),
Version: "3.8.16",
Type: "binary",
PURL: "pkg:generic/python@3.8.16",
Locations: locations("dir/python3.8", "python3.8", "libpython3.8.so"),
Metadata: pkg.BinaryMetadata{
Classifier: "cpython-source",
Matches: []pkg.ClassifierMatch{
match("python-binary", "dir/python3.8"),
match("python-binary", "python3.8"),
match("python-binary-lib", "libpython3.8.so"),
},
},
},
},
{
name: "positive-go",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/go-1.14",
expected: pkg.Package{
Name: "go",
Version: "1.14",
PURL: "pkg:generic/go@1.14",
Locations: singleLocation("go"),
Metadata: pkg.BinaryMetadata{
Classifier: "go-binary",
},
Locations: locations("go"),
Metadata: metadata("go-binary"),
},
},
{
name: "positive-node",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/node-19.2.1",
expected: pkg.Package{
Name: "node",
Version: "19.2.1",
PURL: "pkg:generic/node@19.2.1",
Locations: singleLocation("node"),
Metadata: pkg.BinaryMetadata{
Classifier: "nodejs-binary",
},
Locations: locations("node"),
Metadata: metadata("nodejs-binary"),
},
},
{
name: "positive-go-hint",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/go-hint-1.15",
expected: pkg.Package{
Name: "go",
Version: "1.15",
PURL: "pkg:generic/go@1.15",
Locations: singleLocation("VERSION"),
Metadata: pkg.BinaryMetadata{
Classifier: "go-binary-hint",
},
Locations: locations("VERSION"),
Metadata: metadata("go-binary-hint"),
},
},
{
name: "positive-busybox",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/busybox-3.33.3",
expected: pkg.Package{
Name: "busybox",
Version: "3.33.3",
Locations: singleLocation("["), // note: busybox is a link to [
Metadata: pkg.BinaryMetadata{
Classifier: "busybox-binary",
VirtualPath: "busybox",
},
Locations: locations("["), // note: busybox is a link to [
Metadata: metadata("busybox-binary", "[", "busybox"),
},
},
{
@@ -342,11 +563,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "1.8.0_352-b08",
Type: "binary",
PURL: "pkg:generic/java@1.8.0_352-b08",
Locations: singleLocation("java"),
Metadata: pkg.BinaryMetadata{
Classifier: "java-binary-openjdk",
VirtualPath: "java",
},
Locations: locations("java"),
Metadata: metadata("java-binary-openjdk", "java"),
},
},
{
@@ -357,11 +575,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "11.0.17+8-LTS",
Type: "binary",
PURL: "pkg:generic/java@11.0.17+8-LTS",
Locations: singleLocation("java"),
Metadata: pkg.BinaryMetadata{
Classifier: "java-binary-openjdk",
VirtualPath: "java",
},
Locations: locations("java"),
Metadata: metadata("java-binary-openjdk", "java"),
},
},
{
@@ -372,11 +587,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "19.0.1+10-21",
Type: "binary",
PURL: "pkg:generic/java@19.0.1+10-21",
Locations: singleLocation("java"),
Metadata: pkg.BinaryMetadata{
Classifier: "java-binary-oracle",
VirtualPath: "java",
},
Locations: locations("java"),
Metadata: metadata("java-binary-oracle", "java"),
},
},
{
@@ -387,11 +599,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "19.0.1+10-21",
Type: "binary",
PURL: "pkg:generic/java@19.0.1+10-21",
Locations: singleLocation("java"),
Metadata: pkg.BinaryMetadata{
Classifier: "java-binary-oracle",
VirtualPath: "java",
},
Locations: locations("java"),
Metadata: metadata("java-binary-oracle", "java"),
},
},
{
@@ -402,13 +611,109 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "1.8.0-foreman_2022_09_22_15_30-b00",
Type: "binary",
PURL: "pkg:generic/java@1.8.0-foreman_2022_09_22_15_30-b00",
Locations: singleLocation("java"),
Locations: locations("java"),
Metadata: metadata("java-binary-ibm", "java"),
},
},
{
name: "positive-rust-1.50.0-macos",
fixtureDir: "test-fixtures/classifiers/positive/rust-1.50.0",
expected: pkg.Package{
Name: "rust",
Version: "1.50.0",
Type: "binary",
PURL: "pkg:generic/rust@1.50.0",
Locations: locations("lib/rustlib/aarch64-apple-darwin/lib/libstd-f6f9eec1635e636a.dylib"),
Metadata: metadata("rust-standard-library-macos"),
},
},
{
name: "positive-rust-1.67.1-macos",
fixtureDir: "test-fixtures/classifiers/positive/rust-1.67.1/toolchains/stable-aarch64-apple-darwin",
expected: pkg.Package{
Name: "rust",
Version: "1.67.1",
Type: "binary",
PURL: "pkg:generic/rust@1.67.1",
Locations: locations("lib/libstd-16f2b65e77054c42.dylib"),
Metadata: metadata("rust-standard-library-macos"),
},
},
{
name: "positive-rust-1.67.1-linux",
fixtureDir: "test-fixtures/classifiers/positive/rust-1.67.1/toolchains/stable-x86_64-unknown-linux-musl",
expected: pkg.Package{
Name: "rust",
Version: "1.67.1",
Type: "binary",
PURL: "pkg:generic/rust@1.67.1",
Locations: locations("lib/libstd-86aefecbddda356d.so"),
Metadata: metadata("rust-standard-library-linux"),
},
},
{
name: "positive-ruby-3.2.1",
fixtureDir: "test-fixtures/classifiers/dynamic/ruby-library-3.2.1",
expected: pkg.Package{
Name: "ruby",
Version: "3.2.1",
Type: "binary",
PURL: "pkg:generic/ruby@3.2.1",
Locations: locations("ruby", "libruby.so.3.2.1"),
Metadata: pkg.BinaryMetadata{
Classifier: "java-binary-ibm",
VirtualPath: "java",
Matches: []pkg.ClassifierMatch{
match("ruby-binary", "ruby"),
match("ruby-binary", "libruby.so.3.2.1"),
},
},
},
},
{
name: "positive-ruby-2.7.7",
fixtureDir: "test-fixtures/classifiers/dynamic/ruby-library-2.7.7",
expected: pkg.Package{
Name: "ruby",
Version: "2.7.7p221",
Type: "binary",
PURL: "pkg:generic/ruby@2.7.7p221",
Locations: locations("ruby", "libruby.so.2.7.7"),
Metadata: pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match("ruby-binary", "ruby"),
match("ruby-binary", "libruby.so.2.7.7"),
},
},
},
},
{
name: "positive-ruby-2.6.10",
fixtureDir: "test-fixtures/classifiers/dynamic/ruby-library-2.6.10",
expected: pkg.Package{
Name: "ruby",
Version: "2.6.10p210",
Type: "binary",
PURL: "pkg:generic/ruby@2.6.10p210",
Locations: locations("ruby", "libruby.so.2.6.10"),
Metadata: pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match("ruby-binary", "ruby"),
match("ruby-binary", "libruby.so.2.6.10"),
},
},
},
},
{
name: "positive-ruby-1.9.3p551",
fixtureDir: "test-fixtures/classifiers/positive/ruby-1.9.3p551",
expected: pkg.Package{
Name: "ruby",
Version: "1.9.3p551",
Type: "binary",
PURL: "pkg:generic/ruby@1.9.3p551",
Locations: locations("ruby"),
Metadata: metadata("ruby-binary"),
},
},
}
for _, test := range tests {
@@ -424,18 +729,9 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
packages, _, err := c.Catalog(resolver)
require.NoError(t, err)
ok := false
for _, p := range packages {
if test.expected.Locations.ToSlice()[0].RealPath == p.Locations.ToSlice()[0].RealPath {
ok = true
assertPackagesAreEqual(t, test.expected, p)
}
}
if !ok {
t.Fatalf("could not find test location=%q", test.expected.Locations.ToSlice()[0].RealPath)
}
require.Len(t, packages, 1)
assertPackagesAreEqual(t, test.expected, packages[0])
})
}
}
@@ -452,11 +748,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases_Image(t *testing.T) {
expected: pkg.Package{
Name: "busybox",
Version: "1.35.0",
Locations: singleLocation("/bin/["),
Metadata: pkg.BinaryMetadata{
Classifier: "busybox-binary",
VirtualPath: "/bin/busybox",
},
Locations: locations("/bin/["),
Metadata: metadata("busybox-binary", "/bin/[", "/bin/busybox"),
},
},
}
@@ -475,18 +768,20 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases_Image(t *testing.T) {
packages, _, err := c.Catalog(resolver)
require.NoError(t, err)
ok := false
for _, p := range packages {
if test.expected.Locations.ToSlice()[0].RealPath == p.Locations.ToSlice()[0].RealPath {
ok = true
assertPackagesAreEqual(t, test.expected, p)
expectedLocations := test.expected.Locations.ToSlice()
gotLocations := p.Locations.ToSlice()
require.Len(t, gotLocations, len(expectedLocations))
for i, expectedLocation := range expectedLocations {
gotLocation := gotLocations[i]
if expectedLocation.RealPath != gotLocation.RealPath {
t.Fatalf("locations do not match; expected: %v got: %v", expectedLocations, gotLocations)
}
}
}
if !ok {
t.Fatalf("could not find test location=%q", test.expected.Locations.ToSlice()[0].RealPath)
assertPackagesAreEqual(t, test.expected, p)
}
})
}
}
@@ -505,18 +800,101 @@ func TestClassifierCataloger_DefaultClassifiers_NegativeCases(t *testing.T) {
assert.Equal(t, 0, len(actualResults))
}
func singleLocation(s string) source.LocationSet {
return source.NewLocationSet(source.NewLocation(s))
func locations(locations ...string) source.LocationSet {
var locs []source.Location
for _, s := range locations {
locs = append(locs, source.NewLocation(s))
}
return source.NewLocationSet(locs...)
}
// metadata paths are: realPath, virtualPath
func metadata(classifier string, paths ...string) pkg.BinaryMetadata {
return pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match(classifier, paths...),
},
}
}
// match paths are: realPath, virtualPath
func match(classifier string, paths ...string) pkg.ClassifierMatch {
realPath := ""
if len(paths) > 0 {
realPath = paths[0]
}
virtualPath := ""
if len(paths) > 1 {
virtualPath = paths[1]
}
return pkg.ClassifierMatch{
Classifier: classifier,
Location: source.Location{
Coordinates: source.Coordinates{
RealPath: realPath,
},
VirtualPath: virtualPath,
},
}
}
func assertPackagesAreEqual(t *testing.T, expected pkg.Package, p pkg.Package) {
meta1 := expected.Metadata.(pkg.BinaryMetadata)
meta2 := p.Metadata.(pkg.BinaryMetadata)
var failMessages []string
expectedLocations := expected.Locations.ToSlice()
gotLocations := p.Locations.ToSlice()
if len(expectedLocations) != len(gotLocations) {
failMessages = append(failMessages, "locations are not equal length")
} else {
for i, expectedLocation := range expectedLocations {
gotLocation := gotLocations[i]
if expectedLocation.RealPath != gotLocation.RealPath {
failMessages = append(failMessages, fmt.Sprintf("locations do not match; expected: %v got: %v", expectedLocation.RealPath, gotLocation.RealPath))
}
}
}
m1 := expected.Metadata.(pkg.BinaryMetadata).Matches
m2 := p.Metadata.(pkg.BinaryMetadata).Matches
matches := true
if len(m1) == len(m2) {
for i, m1 := range m1 {
m2 := m2[i]
if m1.Classifier != m2.Classifier {
matches = false
break
}
if m1.Location.RealPath != "" && m1.Location.RealPath != m2.Location.RealPath {
matches = false
break
}
if m1.Location.VirtualPath != "" && m1.Location.VirtualPath != m2.Location.VirtualPath {
matches = false
break
}
}
} else {
matches = false
}
if !matches {
failMessages = append(failMessages, "classifier matches not equal")
}
if expected.Name != p.Name ||
expected.Version != p.Version ||
expected.PURL != p.PURL ||
meta1.Classifier != meta2.Classifier {
assert.Failf(t, "packages not equal", "%v != %v", expected, p)
expected.PURL != p.PURL {
failMessages = append(failMessages, "packages do not match")
}
if len(failMessages) > 0 {
assert.Failf(t, strings.Join(failMessages, "; "), "diff: %s",
cmp.Diff(expected, p,
cmp.Transformer("Locations", func(l source.LocationSet) []source.Location {
return l.ToSlice()
}),
cmpopts.IgnoreUnexported(pkg.Package{}, source.Location{}),
cmpopts.IgnoreFields(pkg.Package{}, "CPEs", "FoundBy", "MetadataType", "Type"),
))
}
}
@@ -544,7 +922,7 @@ func (p *panicyResolver) FileContentsByLocation(_ source.Location) (io.ReadClose
return nil, errors.New("not implemented")
}
func (p *panicyResolver) HasPath(s string) bool {
func (p *panicyResolver) HasPath(_ string) bool {
return true
}
@@ -575,6 +953,8 @@ func (p *panicyResolver) FileMetadataByLocation(_ source.Location) (source.FileM
return source.FileMetadata{}, errors.New("not implemented")
}
var _ source.FileResolver = (*panicyResolver)(nil)
func Test_Cataloger_ResilientToErrors(t *testing.T) {
c := NewCataloger()
+133 -16
View File
@@ -2,14 +2,19 @@ package binary
import (
"bytes"
"debug/elf"
"debug/macho"
"debug/pe"
"fmt"
"io"
"reflect"
"regexp"
"strings"
"text/template"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/unionreader"
@@ -49,16 +54,36 @@ type classifier struct {
}
// evidenceMatcher is a function called to catalog Packages that match some sort of evidence
type evidenceMatcher func(classifier classifier, reader source.LocationReadCloser) ([]pkg.Package, error)
type evidenceMatcher func(resolver source.FileResolver, classifier classifier, location source.Location) ([]pkg.Package, error)
func evidenceMatchers(matchers ...evidenceMatcher) evidenceMatcher {
return func(resolver source.FileResolver, classifier classifier, location source.Location) ([]pkg.Package, error) {
for _, matcher := range matchers {
match, err := matcher(resolver, classifier, location)
if err != nil {
return nil, err
}
if match != nil {
return match, nil
}
}
return nil, nil
}
}
func fileNameTemplateVersionMatcher(fileNamePattern string, contentTemplate string) evidenceMatcher {
pat := regexp.MustCompile(fileNamePattern)
return func(classifier classifier, reader source.LocationReadCloser) ([]pkg.Package, error) {
if !pat.MatchString(reader.RealPath) {
return func(resolver source.FileResolver, classifier classifier, location source.Location) ([]pkg.Package, error) {
if !pat.MatchString(location.RealPath) {
return nil, nil
}
filepathNamedGroupValues := internal.MatchNamedCaptureGroups(pat, reader.RealPath)
filepathNamedGroupValues := internal.MatchNamedCaptureGroups(pat, location.RealPath)
// versions like 3.5 should not match any character, but explicit dot
for k, v := range filepathNamedGroupValues {
filepathNamedGroupValues[k] = strings.ReplaceAll(v, ".", "\\.")
}
tmpl, err := template.New("").Parse(contentTemplate)
if err != nil {
@@ -76,26 +101,70 @@ func fileNameTemplateVersionMatcher(fileNamePattern string, contentTemplate stri
return nil, fmt.Errorf("unable to compile rendered regex=%q: %w", patternBuf.String(), err)
}
contents, err := getContents(reader)
contents, err := getContents(resolver, location)
if err != nil {
return nil, fmt.Errorf("unable to get read contents for file: %w", err)
}
matchMetadata := internal.MatchNamedCaptureGroups(tmplPattern, string(contents))
return singlePackage(classifier, reader, matchMetadata), nil
return singlePackage(classifier, location, matchMetadata), nil
}
}
func fileContentsVersionMatcher(pattern string) evidenceMatcher {
pat := regexp.MustCompile(pattern)
return func(classifier classifier, reader source.LocationReadCloser) ([]pkg.Package, error) {
contents, err := getContents(reader)
return func(resolver source.FileResolver, classifier classifier, location source.Location) ([]pkg.Package, error) {
contents, err := getContents(resolver, location)
if err != nil {
return nil, fmt.Errorf("unable to get read contents for file: %w", err)
}
matchMetadata := internal.MatchNamedCaptureGroups(pat, string(contents))
return singlePackage(classifier, reader, matchMetadata), nil
return singlePackage(classifier, location, matchMetadata), nil
}
}
//nolint:gocognit
func sharedLibraryLookup(sharedLibraryPattern string, sharedLibraryMatcher evidenceMatcher) evidenceMatcher {
pat := regexp.MustCompile(sharedLibraryPattern)
return func(resolver source.FileResolver, classifier classifier, location source.Location) (packages []pkg.Package, _ error) {
libs, err := sharedLibraries(resolver, location)
if err != nil {
return nil, err
}
for _, lib := range libs {
if !pat.MatchString(lib) {
continue
}
locations, err := resolver.FilesByGlob("**/" + lib)
if err != nil {
return nil, err
}
for _, libraryLication := range locations {
pkgs, err := sharedLibraryMatcher(resolver, classifier, libraryLication)
if err != nil {
return nil, err
}
for _, p := range pkgs {
// set the source binary as the first location
locationSet := source.NewLocationSet(location)
locationSet.Add(p.Locations.ToSlice()...)
p.Locations = locationSet
meta, _ := p.Metadata.(pkg.BinaryMetadata)
p.Metadata = pkg.BinaryMetadata{
Matches: append([]pkg.ClassifierMatch{
{
Classifier: classifier.Class,
Location: location,
},
}, meta.Matches...),
}
packages = append(packages, p)
}
}
}
return packages, nil
}
}
@@ -107,7 +176,7 @@ func mustPURL(purl string) packageurl.PackageURL {
return p
}
func singlePackage(classifier classifier, reader source.LocationReadCloser, matchMetadata map[string]string) []pkg.Package {
func singlePackage(classifier classifier, location source.Location, matchMetadata map[string]string) []pkg.Package {
version, ok := matchMetadata["version"]
if !ok {
return nil
@@ -125,15 +194,18 @@ func singlePackage(classifier classifier, reader source.LocationReadCloser, matc
p := pkg.Package{
Name: classifier.Package,
Version: version,
Locations: source.NewLocationSet(reader.Location),
Locations: source.NewLocationSet(location),
Type: pkg.BinaryPkg,
CPEs: cpes,
FoundBy: catalogerName,
MetadataType: pkg.BinaryMetadataType,
Metadata: pkg.BinaryMetadata{
Classifier: classifier.Class,
RealPath: reader.RealPath,
VirtualPath: reader.VirtualPath,
Matches: []pkg.ClassifierMatch{
{
Classifier: classifier.Class,
Location: location,
},
},
},
}
@@ -156,8 +228,13 @@ func singlePackage(classifier classifier, reader source.LocationReadCloser, matc
return []pkg.Package{p}
}
func getContents(reader source.LocationReadCloser) ([]byte, error) {
unionReader, err := unionreader.GetUnionReader(reader.ReadCloser)
func getContents(resolver source.FileResolver, location source.Location) ([]byte, error) {
reader, err := resolver.FileContentsByLocation(location)
if err != nil {
return nil, err
}
unionReader, err := unionreader.GetUnionReader(reader)
if err != nil {
return nil, fmt.Errorf("unable to get union reader for file: %w", err)
}
@@ -177,3 +254,43 @@ func singleCPE(cpeString string) []cpe.CPE {
cpe.Must(cpeString),
}
}
// sharedLibraries returns a list of all shared libraries found within a binary, currently
// supporting: elf, macho, and windows pe
func sharedLibraries(resolver source.FileResolver, location source.Location) ([]string, error) {
contents, err := getContents(resolver, location)
if err != nil {
return nil, err
}
r := bytes.NewReader(contents)
e, _ := elf.NewFile(r)
if e != nil {
symbols, err := e.ImportedLibraries()
if err != nil {
log.Debugf("unable to read elf binary at: %s -- %s", location.RealPath, err)
}
return symbols, nil
}
m, _ := macho.NewFile(r)
if m != nil {
symbols, err := m.ImportedLibraries()
if err != nil {
log.Debugf("unable to read macho binary at: %s -- %s", location.RealPath, err)
}
return symbols, nil
}
p, _ := pe.NewFile(r)
if p != nil {
symbols, err := p.ImportedLibraries()
if err != nil {
log.Debugf("unable to read pe binary at: %s -- %s", location.RealPath, err)
}
return symbols, nil
}
return nil, nil
}
+2 -4
View File
@@ -67,10 +67,8 @@ func Test_ClassifierCPEs(t *testing.T) {
locations, err := resolver.FilesByPath(test.fixture)
require.NoError(t, err)
require.Len(t, locations, 1)
location := locations[0]
readCloser, err := resolver.FileContentsByLocation(location)
require.NoError(t, err)
pkgs, err := test.classifier.EvidenceMatcher(test.classifier, source.NewLocationReadCloser(location, readCloser))
pkgs, err := test.classifier.EvidenceMatcher(resolver, test.classifier, locations[0])
require.NoError(t, err)
require.Len(t, pkgs, 1)
+118 -22
View File
@@ -9,9 +9,16 @@ var defaultClassifiers = []classifier{
{
Class: "python-binary",
FileGlob: "**/python*",
EvidenceMatcher: fileNameTemplateVersionMatcher(
`(.*/|^)python(?P<version>[0-9]+\.[0-9]+)$`,
`(?m)(?P<version>{{ .version }}\.[0-9]+[-_a-zA-Z0-9]*)`),
EvidenceMatcher: evidenceMatchers(
// try to find version information from libpython shared libraries
sharedLibraryLookup(
`^libpython[0-9]+(?:\.[0-9]+)+[a-z]?\.so.*$`,
libpythonMatcher),
// check for version information in the binary
fileNameTemplateVersionMatcher(
`(?:.*/|^)python(?P<version>[0-9]+(?:\.[0-9]+)+)$`,
pythonVersionTemplate),
),
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
CPEs: []cpe.CPE{
@@ -20,25 +27,11 @@ var defaultClassifiers = []classifier{
},
},
{
Class: "python-binary-lib",
FileGlob: "**/libpython*.so*",
EvidenceMatcher: fileNameTemplateVersionMatcher(
`(.*/|^)libpython(?P<version>[0-9]+\.[0-9]+).so.*$`,
`(?m)(?P<version>{{ .version }}\.[0-9]+[-_a-zA-Z0-9]*)`),
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:a:python_software_foundation:python:*:*:*:*:*:*:*:*"),
cpe.Must("cpe:2.3:a:python:python:*:*:*:*:*:*:*:*"),
},
},
{
Class: "cpython-source",
FileGlob: "**/patchlevel.h",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)#define\s+PY_VERSION\s+"?(?P<version>[0-9\.\-_a-zA-Z]+)"?`),
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
Class: "python-binary-lib",
FileGlob: "**/libpython*.so*",
EvidenceMatcher: libpythonMatcher,
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:a:python_software_foundation:python:*:*:*:*:*:*:*:*"),
cpe.Must("cpe:2.3:a:python:python:*:*:*:*:*:*:*:*"),
@@ -53,6 +46,42 @@ var defaultClassifiers = []classifier{
PURL: mustPURL("pkg:generic/go@version"),
CPEs: singleCPE("cpe:2.3:a:golang:go:*:*:*:*:*:*:*:*"),
},
{
Class: "argocd",
FileGlob: "**/argocd",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)common\.version=(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`),
Package: "argocd",
PURL: mustPURL("pkg:golang/github.com/argoproj/argo-cd@version"),
CPEs: singleCPE("cpe:2.3:a:argoproj:argocd:*:*:*:*:*:*:*"),
},
{
Class: "helm",
FileGlob: "**/helm",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)\x00v(?P<version>[0-9]+\.[0-9]+\.[0-9]+)\x00`),
Package: "helm",
PURL: mustPURL("pkg:golang/helm.sh/helm@version"),
CPEs: singleCPE("cpe:2.3:a:helm:helm:*:*:*:*:*:*:*"),
},
{
Class: "kustomize",
FileGlob: "**/kustomize",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)version=kustomize/v(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`),
Package: "kustomize",
PURL: mustPURL("pkg:golang/sigs.k8s.io/kustomize@version"),
CPEs: singleCPE("cpe:2.3:a:kustomize:kustomize:*:*:*:*:*:*:*"),
},
{
Class: "kubectl",
FileGlob: "**/kubectl",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)\x00v(?P<version>[0-9]+\.[0-9]+\.[0-9]+)\x00`),
Package: "kubectl",
PURL: mustPURL("pkg:golang/k8s.io/kubectl@version"),
CPEs: singleCPE("cpe:2.3:a:kubectl:kubectl:*:*:*:*:*:*:*"),
},
{
Class: "redis-binary",
FileGlob: "**/redis-server",
@@ -120,6 +149,26 @@ var defaultClassifiers = []classifier{
Package: "busybox",
CPEs: singleCPE("cpe:2.3:a:busybox:busybox:*:*:*:*:*:*:*:*"),
},
{
Class: "haproxy-binary",
FileGlob: "**/haproxy",
EvidenceMatcher: evidenceMatchers(
fileContentsVersionMatcher(`(?m)HA-Proxy version (?P<version>[0-9]+\.[0-9]+\.[0-9]+)`),
fileContentsVersionMatcher(`(?m)(?P<version>[0-9]+\.[0-9]+\.[0-9]+)-[0-9a-zA-Z]{7}.+HAProxy version`),
),
Package: "haproxy",
PURL: mustPURL("pkg:generic/haproxy@version"),
CPEs: singleCPE("cpe:2.3:a:haproxy:haproxy:*:*:*:*:*:*:*:*"),
},
{
Class: "perl-binary",
FileGlob: "**/perl",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)\/usr\/local\/lib\/perl\d\/(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`),
Package: "perl",
PURL: mustPURL("pkg:generic/perl@version"),
CPEs: singleCPE("cpe:2.3:a:perl:perl:*:*:*:*:*:*:*:*"),
},
{
Class: "php-cli-binary",
FileGlob: "**/php*",
@@ -187,4 +236,51 @@ var defaultClassifiers = []classifier{
Package: "postgresql",
PURL: mustPURL("pkg:generic/postgresql@version"),
},
{
Class: "rust-standard-library-linux",
FileGlob: "**/libstd-????????????????.so",
EvidenceMatcher: fileContentsVersionMatcher(
// clang LLVM (rustc version 1.48.0 (7eac88abb 2020-11-16))
`(?m)(\x00)clang LLVM \(rustc version (?P<version>[0-9]+(\.[0-9]+)?(\.[0-9]+)) \(\w+ \d{4}\-\d{2}\-\d{2}\)`),
Package: "rust",
PURL: mustPURL("pkg:generic/rust@version"),
CPEs: singleCPE("cpe:2.3:a:rust-lang:rust:*:*:*:*:*:*:*:*"),
},
{
Class: "rust-standard-library-macos",
FileGlob: "**/libstd-????????????????.dylib",
EvidenceMatcher: fileContentsVersionMatcher(
// c 1.48.0 (7eac88abb 2020-11-16)
`(?m)c (?P<version>[0-9]+(\.[0-9]+)?(\.[0-9]+)) \(\w+ \d{4}\-\d{2}\-\d{2}\)`),
Package: "rust",
PURL: mustPURL("pkg:generic/rust@version"),
CPEs: singleCPE("cpe:2.3:a:rust-lang:rust:*:*:*:*:*:*:*:*"),
},
{
Class: "ruby-binary",
FileGlob: "**/ruby",
EvidenceMatcher: evidenceMatchers(
rubyMatcher,
sharedLibraryLookup(
// try to find version information from libruby shared libraries
`^libruby\.so.*$`,
rubyMatcher),
),
Package: "ruby",
PURL: mustPURL("pkg:generic/ruby@version"),
CPEs: singleCPE("cpe:2.3:a:ruby-lang:ruby:*:*:*:*:*:*:*:*"),
},
}
// in both binaries and shared libraries, the version pattern is [NUL]3.11.2[NUL]
var pythonVersionTemplate = `(?m)\x00(?P<version>{{ .version }}[-._a-zA-Z0-9]*)\x00`
var libpythonMatcher = fileNameTemplateVersionMatcher(
`(?:.*/|^)libpython(?P<version>[0-9]+(?:\.[0-9]+)+)[a-z]?\.so.*$`,
pythonVersionTemplate,
)
var rubyMatcher = fileContentsVersionMatcher(
// ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux]
// ruby 2.7.7p221 (2022-11-24 revision 168ec2b1e5) [x86_64-linux]
`(?m)ruby (?P<version>[0-9]+\.[0-9]+\.[0-9]+(p[0-9]+)?) `)
@@ -0,0 +1 @@
classifiers/dynamic
@@ -0,0 +1,151 @@
.PHONY: all
all: \
classifiers/dynamic/python-binary-shared-lib-3.11 \
classifiers/dynamic/python-binary-shared-lib-redhat-3.9 \
classifiers/dynamic/python-binary-with-version-3.9 \
classifiers/dynamic/python-binary-3.4-alpine \
classifiers/dynamic/ruby-library-3.2.1 \
classifiers/dynamic/ruby-library-2.7.7 \
classifiers/dynamic/ruby-library-2.6.10 \
classifiers/dynamic/argocd-2.5.11 \
classifiers/dynamic/argocd-2.6.4 \
classifiers/dynamic/helm-3.11.1 \
classifiers/dynamic/helm-3.10.3 \
classifiers/dynamic/kubectl-1.24.11 \
classifiers/dynamic/kubectl-1.25.7 \
classifiers/dynamic/kubectl-1.26.2 \
classifiers/dynamic/kustomize-4.5.7 \
classifiers/dynamic/kustomize-5.0.0
classifiers/dynamic/python-binary-shared-lib-3.11:
$(eval $@_image := "python:3.11-slim@sha256:0b106e1d2bf485c2a41474bc9cd5103e9eea4e179f40f10741b53b127059221e")
./get-image-file.sh $($@_image) \
/usr/local/bin/python3.11 \
$@/python3
./get-image-file.sh $($@_image) \
/usr/local/lib/libpython3.11.so.1.0 \
$@/libpython3.11.so.1.0
classifiers/dynamic/python-binary-shared-lib-redhat-3.9:
$(eval $@_image := "registry.access.redhat.com/ubi8/python-39@sha256:f3cf958b96ce016b63e3e163e488f52e42891304dafef5a0811563f22e3cbad0")
./get-image-file.sh $($@_image) \
/usr/bin/python3.9 \
$@/python3.9
./get-image-file.sh $($@_image) \
/usr/lib64/libpython3.9.so.1.0 \
$@/libpython3.9.so.1.0
classifiers/dynamic/python-binary-with-version-3.9:
$(eval $@_image := "python:3.9.16-bullseye@sha256:93fb93c461a2e47a2176706fad1f39eaacd5dd40e19c0b018699a28c03eb2e2a")
./get-image-file.sh $($@_image) \
/usr/bin/python3.9 \
$@/python3.9
classifiers/dynamic/python-binary-3.4-alpine:
$(eval $@_image := "python:3.4-alpine@sha256:c210b660e2ea553a7afa23b41a6ed112f85dbce25cbcb567c75dfe05342a4c4b")
./get-image-file.sh $($@_image) \
/usr/local/bin/python3.4 \
$@/python3.4
./get-image-file.sh $($@_image) \
/usr/local/lib/libpython3.4m.so.1.0 \
$@/libpython3.4m.so.1.0
classifiers/dynamic/ruby-library-3.2.1:
$(eval $@_image := "ruby:3.2.1-bullseye@sha256:b4a140656b0c5d26c0a80559b228b4d343f3fdbf56682fcbe88f6db1fa9afa6b")
./get-image-file.sh $($@_image) \
/usr/local/bin/ruby \
$@/ruby
./get-image-file.sh $($@_image) \
/usr/local/lib/libruby.so.3.2.1 \
$@/libruby.so.3.2.1
./get-image-file.sh $($@_image) \
/usr/local/lib/libruby.so.3.2 \
$@/libruby.so.3.2
classifiers/dynamic/ruby-library-2.7.7:
$(eval $@_image := "ruby:2.7.7-bullseye@sha256:055191740a063f33fef1f09423e5ed8f91143aae62a3772a90910118464c5120")
./get-image-file.sh $($@_image) \
/usr/local/bin/ruby \
$@/ruby
./get-image-file.sh $($@_image) \
/usr/local/lib/libruby.so.2.7.7 \
$@/libruby.so.2.7.7
./get-image-file.sh $($@_image) \
/usr/local/lib/libruby.so.2.7 \
$@/libruby.so.2.7
classifiers/dynamic/ruby-library-2.6.10:
$(eval $@_image := "ruby:2.6.10@sha256:771a810704167e55da8a19970c5dfa6eb795dfee32547adffa29ea72703f7243")
./get-image-file.sh $($@_image) \
/usr/local/bin/ruby \
$@/ruby
./get-image-file.sh $($@_image) \
/usr/local/lib/libruby.so.2.6.10 \
$@/libruby.so.2.6.10
./get-image-file.sh $($@_image) \
/usr/local/lib/libruby.so.2.6 \
$@/libruby.so.2.6
classifiers/dynamic/argocd-2.5.11:
$(eval $@_image := "argoproj/argocd:v2.5.11@sha256:d1062935b3256ec69422843ebcb50debb54fd389436961586000c8ce6ee7f249")
./get-image-file.sh $($@_image) \
/usr/local/bin/argocd \
$@/argocd
classifiers/dynamic/argocd-2.6.4:
$(eval $@_image := "argoproj/argocd:v2.6.4@sha256:61fcbba187ff53c00696cb580edf70cada59c45cf399d8477631acf43cf522ee")
./get-image-file.sh $($@_image) \
/usr/local/bin/argocd \
$@/argocd
classifiers/dynamic/helm-3.11.1:
$(eval $@_image := "alpine/helm:3.11.1@sha256:8628e3695fb743a8b9de89626f1b7a221280c2152c0e288c2504e59b68233e8b")
./get-image-file.sh $($@_image) \
/usr/bin/helm \
$@/helm
classifiers/dynamic/helm-3.10.3:
$(eval $@_image := "argoproj/argocd:v2.6.4@sha256:61fcbba187ff53c00696cb580edf70cada59c45cf399d8477631acf43cf522ee")
./get-image-file.sh $($@_image) \
/usr/local/bin/helm \
$@/helm
classifiers/dynamic/kubectl-1.24.11:
$(eval $@_image := "bitnami/kubectl:1.24.11@sha256:79d60c5ac8a1dc84e2c39f56d8e8cc0053159b5ed88f283bdf8fbda1ee86c8bc")
./get-image-file.sh $($@_image) \
/opt/bitnami/kubectl/bin/kubectl \
$@/kubectl
classifiers/dynamic/kubectl-1.25.7:
$(eval $@_image := "bitnami/kubectl:1.25.7@sha256:d7b00dbfdc6d8890aefe40edfb6c1d4c90cbb6c978794bb51a21744edc34ba7a")
./get-image-file.sh $($@_image) \
/opt/bitnami/kubectl/bin/kubectl \
$@/kubectl
classifiers/dynamic/kubectl-1.26.2:
$(eval $@_image := "line/kubectl-kustomize:1.26.2-5.0.0@sha256:9ee3b4a9a21f0777fc1d8c64208290f818a2e68c5e9e892e931621bda089bf06")
./get-image-file.sh $($@_image) \
/usr/local/bin/kubectl \
$@/kubectl
classifiers/dynamic/kustomize-4.5.7:
$(eval $@_image := "argoproj/argocd:v2.6.4@sha256:61fcbba187ff53c00696cb580edf70cada59c45cf399d8477631acf43cf522ee")
./get-image-file.sh $($@_image) \
/usr/local/bin/kustomize \
$@/kustomize
classifiers/dynamic/kustomize-5.0.0:
$(eval $@_image := "line/kubectl-kustomize:1.26.2-5.0.0@sha256:9ee3b4a9a21f0777fc1d8c64208290f818a2e68c5e9e892e931621bda089bf06")
./get-image-file.sh $($@_image) \
/usr/local/bin/kustomize \
$@/kustomize
.PHONY: clean
clean:
rm -rf classifiers/dynamic
.PHONY: cache.fingerprint
cache.fingerprint: # for CI
$(title,Install test fixture fingerprint)
@find ./classifiers/dynamic/* -type f -exec md5sum {} + | awk '{print $1}' | sort | tee /dev/stderr | md5sum | tee cache.fingerprint >> cache.fingerprint
@@ -1,2 +1,2 @@
!libpython3.7.so
!libphp.so
!*.so
!VERSION
@@ -0,0 +1 @@
HA-Proxy version 1.5.14 2015/07/02Copyright 2000-2015 Willy Tarreau <willy@haproxy.org>

Some files were not shown because too many files have changed in this diff Show More