Compare commits

...
Author SHA1 Message Date
Keith Zantow 4c3e49957c chore: more lenient java groupID lookups
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-08-17 14:38:28 -04:00
Keith Zantow 4762ba0943 feat: use java package names to determine known groupids (#2032)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-08-17 12:55:25 -04:00
Keith Zantow d1635971a1 fix: inconsistent removal of binaries by overlap (#2036)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-08-17 15:27:31 +00:00
9467bd66c2 fix: CycloneDX relationships not output or decoded properly (#1974)
Signed-off-by: Mark Galpin <mark@tidelift.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Mark Galpin <mark@tidelift.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
2023-08-17 11:02:12 -04:00
Keith Zantow 59107324ce chore: restore cataloger.DefaultConfig (#2028)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-08-14 20:28:07 +00:00
Keith Zantow b3d7ba569b fix: read direct package files when decoding SPDX tag-value (#2014)
* fix: read direct package files when decoding SPDX tag-value

---------

Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-08-14 11:37:24 -04:00
anchore-actions-token-generator[bot]andspiffcs c7fe58683d chore(deps): update bootstrap tools to latest versions (#2022)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2023-08-14 11:36:15 -04:00
anchore-actions-token-generator[bot]andwagoodman 28b06dae25 chore(deps): update CPE dictionary index (#2025)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: wagoodman <wagoodman@users.noreply.github.com>
2023-08-14 11:35:57 -04:00
anchore-actions-token-generator[bot]andspiffcs a90cff1cd2 chore(deps): update bootstrap tools to latest versions (#2012)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2023-08-10 13:20:09 -04:00
dependabot[bot]andChristopher Phillips 82eafeaf4a chore(deps): bump github.com/vifraa/gopom from 0.2.2 to 1.0.0 (#2008)
* chore(deps): bump github.com/vifraa/gopom from 0.2.2 to 1.0.0
* refactor: update consumer code to use new optional values

Bumps [github.com/vifraa/gopom](https://github.com/vifraa/gopom) from 0.2.2 to 1.0.0.
- [Release notes](https://github.com/vifraa/gopom/releases)
- [Commits](https://github.com/vifraa/gopom/compare/v0.2.2...v1.0.0)

---
updated-dependencies:
- dependency-name: github.com/vifraa/gopom
  dependency-type: direct:production
  update-type: version-update:semver-major
...
---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-08-09 17:22:51 -04:00
Christopher Angelo Phillips 541c8d339b 1948-filter-pkg-by-type (#2011)
---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-08-09 16:05:52 -04:00
dependabot[bot] 6bf6f85584 chore(deps): bump github.com/dave/jennifer from 1.6.1 to 1.7.0 (#2009)
Bumps [github.com/dave/jennifer](https://github.com/dave/jennifer) from 1.6.1 to 1.7.0.
- [Commits](https://github.com/dave/jennifer/compare/v1.6.1...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/dave/jennifer
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-09 14:46:11 -04:00
Keith Zantow c7272fd6a5 fix: SPDX license values and download location (#2007)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-08-08 15:55:50 -04:00
Christopher Angelo Phillips 466da7cbda 931: binary cataloger exclusion defaults for ownership by overlap (#1948)
Fixes #931

PR #1948 introduces a new implicit exclusion for binary packages that overlap by file ownership and have certain characteristics:

1) the relationship between packages is OwnershipByFileOverlap
2) the parent package is an "os" package - see changelog for included catalogers
3) the child is a synthetic package generated by the binary cataloger - see changelog for included catalogers
4) the package names are identical

---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-08-08 13:00:52 -04:00
dependabot[bot] 2fc65094b7 chore(deps): bump golang.org/x/net from 0.13.0 to 0.14.0 (#2004)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.13.0 to 0.14.0.
- [Commits](https://github.com/golang/net/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-07 10:34:00 -04:00
dependabot[bot] d7ff77072a chore(deps): bump modernc.org/sqlite from 1.24.0 to 1.25.0 (#1998)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.24.0 to 1.25.0.
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.24.0...v1.25.0)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-04 14:24:31 -04:00
Christopher Angelo Phillips 78660022bf test: add coverage for new rpmdb paths (#1999)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-08-04 13:04:36 -04:00
Keith Zantow aaf767f8d3 chore: improve spdx purl decoding (#1996)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-08-04 11:43:21 -04:00
Keith Zantow 79014ed8c8 fix: gradle lockfile parser groupId handling (#1995)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-08-04 11:42:26 -04:00
Christopher Angelo Phillips e774006052 fix: update glob to use newer usr/lib/sysimage path (#1997)
See this link for details on the path migration for the rpmdb
https://fedoraproject.org/wiki/Changes/RelocateRPMToUsr

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-08-03 16:23:50 -07:00
Nicholas R. Smith 1d6d5f7f5f fix: opkg search glob (#1994)
Signed-off-by: Nicholas R. Smith <nicholas_smith@selinc.com>
2023-08-03 19:33:11 +00:00
Sem Provoost 433a7b8a42 feat: nginx binary classifier (#1988)
Signed-off-by: SemProvoost <27961543+SemProvoost@users.noreply.github.com>
2023-08-03 13:09:31 -04:00
Nicholas R. SmithandNicholas R. Smith e55277f26d Expand deb cataloger to include opkg (#1985)
* Add opkg info directory and status file to deb cataloger

opkg uses the same or nearly the same metadata and structure as Debian:
**/lib/opkg/status lists status information for all packages
**/lib/opkg/info/opkg.conffiles is a list of configuration files
**/lib/opkg/info/*.list contains files and directories installed by the package
**/lib/opkg/info/*.preinst are scripts to run before installation
**/lib/opkg/info/*.postinst are scripts to run after installation
**/lib/opkg/info/*.postrm are scripts to run after package removal
**/lib/opkg/info/*.control provides package metadata

Signed-off-by: Nicholas R. Smith <nicholas_smith@selinc.com>

---------

Signed-off-by: Nicholas R. Smith <nicholas_smith@selinc.com>
Co-authored-by: Nicholas R. Smith <nicholas_smith@selinc.com>
2023-08-03 12:33:14 -04:00
anchore-actions-token-generator[bot]andspiffcs c2b4231cc3 chore(deps): update bootstrap tools to latest versions (#1991)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2023-08-03 10:53:29 -04:00
dependabot[bot] c150b4e358 chore(deps): bump github.com/google/go-containerregistry (#1993)
Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.15.2 to 0.16.1.
- [Release notes](https://github.com/google/go-containerregistry/releases)
- [Changelog](https://github.com/google/go-containerregistry/blob/main/.goreleaser.yml)
- [Commits](https://github.com/google/go-containerregistry/compare/v0.15.2...v0.16.1)

---
updated-dependencies:
- dependency-name: github.com/google/go-containerregistry
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-03 10:53:09 -04:00
Keith Zantow 3f0475efb7 chore: update bubbly to fix hanging (#1990)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-08-02 10:28:35 -04:00
dependabot[bot] 2e376d067f chore(deps): bump golang.org/x/net from 0.12.0 to 0.13.0 (#1989) 2023-08-02 14:16:49 +00:00
Christopher Angelo Phillips 8e893dfc20 feat: use originator logic to fill supplier (#1980)
* feat: use Originator to fill supplier for NTIA minimum
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-08-01 17:19:49 -04:00
Alex Goodman 756d0f29af add metadata types to all cpe test fixtures (#1982)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-31 16:35:09 -04:00
Keith Zantow e2f7befbfb fix: default image source name to user input (#1979)
* fix: default image source name to user input

Signed-off-by: Keith Zantow <kzantow@gmail.com>

* chore: add test

Signed-off-by: Keith Zantow <kzantow@gmail.com>

---------

Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-07-31 17:29:18 +00:00
anchore-actions-token-generator[bot]andkzantow f14742b3f3 chore(deps): update stereoscope to d1f3d766295ed3c8362ac1be68070e2a1dba4d03 (#1975)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: kzantow <kzantow@users.noreply.github.com>
2023-07-31 12:02:33 -04:00
Christopher Angelo Phillips 3aae316456 chore: update to latest commit in tools-golang (#1969)
* chore: update to latest commit in tools-golang

---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-07-27 15:29:22 -04:00
Alex Goodmanandmanifestori 063e9da65d Guess unpinned versions in python requirements.txt (#1966)
* feat: python requirements.txt parsing inclusive

Signed-off-by: manifestori <ori@manifestcyber.com>

* refactor: parseVersion

Signed-off-by: manifestori <ori@manifestcyber.com>

* add python config for optional requirements version constraint resolution

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix tests

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* allow for python requirements metadata to be optional

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* restore cyclonedx dependency

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: manifestori <ori@manifestcyber.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: manifestori <ori@manifestcyber.com>
2023-07-27 14:26:59 -04:00
dependabot[bot] bf1102c3f1 chore(deps): bump github.com/vifraa/gopom from 0.2.1 to 0.2.2 (#1965)
Bumps [github.com/vifraa/gopom](https://github.com/vifraa/gopom) from 0.2.1 to 0.2.2.
- [Release notes](https://github.com/vifraa/gopom/releases)
- [Commits](https://github.com/vifraa/gopom/compare/v0.2.1...v0.2.2)

---
updated-dependencies:
- dependency-name: github.com/vifraa/gopom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-27 13:28:42 -04:00
Alex Goodman bbd2d42dbb Fix panic condition on docker pull failure (#1968)
* [wip] add image pull error handlers

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix panic and ui hang on docker pull failure

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* linter fix

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-27 11:32:02 -04:00
Alex Goodman d84120f499 bump JSON schema to account for simplified python env markers (#1967)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-27 14:13:17 +00:00
Keith Zantow 9480f10ccd feat: support top-level SPDX package and graph (#1934)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-07-26 13:54:32 -04:00
dependabot[bot] 1e4d26f526 chore(deps): bump github.com/go-git/go-git/v5 from 5.8.0 to 5.8.1 (#1959)
Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) from 5.8.0 to 5.8.1.
- [Release notes](https://github.com/go-git/go-git/releases)
- [Commits](https://github.com/go-git/go-git/compare/v5.8.0...v5.8.1)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-git/v5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-26 13:34:03 +00:00
Tristan Farkas e1c1832f84 Add cataloger for Swift Package Manager. (#1919)
Signed-off-by: Tristan Farkas <Tristan.Farkas@axis.com>
2023-07-25 14:35:21 -04:00
anchore-actions-token-generator[bot]andkzantow 9a73380f29 chore(deps): update stereoscope to d515761c6ca2743a67d7d08053db69235ae76d1d (#1953)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: kzantow <kzantow@users.noreply.github.com>
2023-07-25 10:49:21 -04:00
dependabot[bot] 2e718cf865 chore(deps): bump github.com/docker/docker (#1955)
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.2+incompatible to 24.0.5+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v24.0.2...v24.0.5)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-25 10:37:16 -04:00
dependabot[bot] 4000a84624 chore(deps): bump github.com/go-git/go-git/v5 from 5.7.0 to 5.8.0 (#1951)
Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) from 5.7.0 to 5.8.0.
- [Release notes](https://github.com/go-git/go-git/releases)
- [Commits](https://github.com/go-git/go-git/compare/v5.7.0...v5.8.0)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-git/v5
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-24 11:28:54 -04:00
Dan LuhringandAlex Goodman 99d172f0d1 Introduce indexed embedded CPE dictionary (#1897)
* Introduce indexed embedded CPE dictionary

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Don't generate cpe-index on make snapshot

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Add unit tests for individual addEntry funcs

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* migrate CPE index build to go generate and add periodic workflow

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add test to ensure generated cpe index is wired up to function that uses it

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-21 13:54:19 +00:00
dependabot[bot] 3f5c601620 chore(deps): bump github.com/gookit/color from 1.5.3 to 1.5.4 (#1949)
Bumps [github.com/gookit/color](https://github.com/gookit/color) from 1.5.3 to 1.5.4.
- [Release notes](https://github.com/gookit/color/releases)
- [Commits](https://github.com/gookit/color/compare/v1.5.3...v1.5.4)

---
updated-dependencies:
- dependency-name: github.com/gookit/color
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-21 08:50:47 -04:00
Dan LuhringandAlex Goodman 8478e0bef7 Add support for parsing .NET assemblies (#1943)
* Add support for parsing .NET assemblies

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: 69c33fe4d77357d843c11590f3b07825bc6249ac

* Add dll and exe files

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: b9d204efa6d2ef385b5fbb7a59a3474ecabea641

* Add PE cataloger to directory catalogers

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: 9711c00d9da92e2887e0c1f92edd740ea5345849

* Don't set language to dotnet for PEs

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: 368313fddac9160d8a06a01ebe8c5ac7990232f5

* Fix spelling of cataloger in constructor

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: e42fd77b2f8b6d42e076a84f6cce386861260941

* Adjust which cases in PE parsing return errors

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: 95b25f8fc3a7d4e18fe30e489b09851f316795ff

* remove build binary from branch

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

Former-commit-id: fa54c0d0aef0998d5520e9f44cae51f5f9cd38a2

* Fix failing CLI tests

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

---------

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-19 15:34:07 -04:00
Christopher Angelo Phillips 0327fdc88a docs: capture artifactory dev settings from 1895 (#1947)
* docs: capture artifactory dev settings from 1895

---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-07-19 16:54:18 +00:00
Alex Goodman 88b3d1e9bb remove build binary and add explicit git ignore
Former-commit-id: 6455f2d8a5f1910b4a8b681ddef79919886d638d
2023-07-18 14:06:34 -04:00
Christopher Angelo Phillips 204b790012 docs: update docs with new docker specific instructions (#1941)
* docs: update docs with new docker specific instructions
---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Former-commit-id: c67c76e84df84e3e24aa307637d884ca8b7e3eea
2023-07-17 18:19:21 +00:00
Alex Goodman 35699f6fdc remove jotframe UI (#1932)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-13 13:21:52 -04:00
Christopher Angelo Phillips 2e7fd031d4 fix: remove indirect dependency of circl v1.1.0 (#1940)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-07-13 12:30:37 -04:00
Christopher Angelo Phillips 32296f5943 chore: move wait before iteration to guarantee read before tea (#1931)
* chore: move wait before iteration to guarantee read before tea

---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-07-12 13:59:31 -04:00
134 changed files with 36176 additions and 1502 deletions
+1 -1
View File
@@ -6,7 +6,7 @@ on:
workflow_dispatch:
env:
GO_VERSION: "1.19.x"
GO_VERSION: "1.20.x"
GO_STABLE_VERSION: true
jobs:
@@ -0,0 +1,43 @@
name: PR to update CPE dictionary index
on:
schedule:
- cron: "0 1 * * 1" # every monday at 1 AM
workflow_dispatch:
env:
GO_VERSION: "1.20.x"
GO_STABLE_VERSION: true
jobs:
upgrade-cpe-dictionary-index:
runs-on: ubuntu-latest
if: github.repository == 'anchore/syft' # only run for main repo
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- run: |
make generate-cpe-dictionary-index
- uses: tibdex/github-app-token@v1
id: generate-token
with:
app_id: ${{ secrets.TOKEN_APP_ID }}
private_key: ${{ secrets.TOKEN_APP_PRIVATE_KEY }}
- uses: peter-evans/create-pull-request@v5
with:
signoff: true
delete-branch: true
branch: auto/latest-cpe-dictionary-index
labels: dependencies
commit-message: "chore(deps): update CPE dictionary index"
title: "chore(deps): update CPE dictionary index"
body: |
Update CPE dictionary index based on the latest available CPE dictionary
token: ${{ steps.generate-token.outputs.token }}
@@ -6,7 +6,7 @@ on:
workflow_dispatch:
env:
GO_VERSION: "1.19.x"
GO_VERSION: "1.20.x"
GO_STABLE_VERSION: true
jobs:
+2
View File
@@ -2,6 +2,7 @@ go.work
go.work.sum
/bin
/.bin
/build
CHANGELOG.md
VERSION
/test/results
@@ -19,6 +20,7 @@ VERSION
*.hpi
*.zip
.idea/
*.iml
*.log
.images
.tmp/
+4
View File
@@ -5,3 +5,7 @@ The following Syft components were contributed by external authors/organizations
## GraalVM Native Image
A cataloger contributed by Oracle Corporation that extracts packages given within GraalVM Native Image SBOMs.
## Swift Package Manager
A cataloger contributed by Axis Communications that catalogs packages resolved by Swift Package Manager.
+39
View File
@@ -7,6 +7,25 @@ In order to test and develop in this repo you will need the following dependenci
- docker
- make
### Docker settings for getting started
Make sure you've updated your docker settings so the default docker socket path is available.
Go to:
docker -> settings -> advanced
Make sure:
```
Allow the default Docker socket to be used
```
is checked.
Also double check that the docker context being used is the default context. If it is not, run:
`docker context use default`
After cloning the following step can help you get setup:
1. run `make bootstrap` to download go mod dependencies, create the `/.tmp` dir, and download helper utilities.
2. run `make` to view the selection of developer commands in the Makefile
@@ -19,6 +38,26 @@ The main make tasks for common static analysis and testing are `lint`, `format`,
See `make help` for all the current make tasks.
### Internal Artifactory Settings
**Not always applicable**
Some companies have Artifactory setup internally as a solution for sourcing secure dependencies.
If you're seeing an issue where the unit tests won't run because of the below error then this section might be relevant for your use case.
```
[ERROR] [ERROR] Some problems were encountered while processing the POMs
```
If you're dealing with an issue where the unit tests will not pull/build certain java fixtures check some of these settings:
- a `settings.xml` file should be available to help you communicate with your internal artifactory deployment
- this can be moved to `syft/pkg/cataloger/java/test-fixtures/java-builds/example-jenkins-plugin/` to help build the unit test-fixtures
- you'll also want to modify the `build-example-jenkins-plugin.sh` to use `settings.xml`
For more information on this setup and troubleshooting see [issue 1895](https://github.com/anchore/syft/issues/1895#issuecomment-1610085319)
## Architecture
Syft is used to generate a Software Bill of Materials (SBOM) from different kinds of input.
+11 -5
View File
@@ -10,11 +10,11 @@ CHRONICLE_CMD = $(TEMP_DIR)/chronicle
GLOW_CMD = $(TEMP_DIR)/glow
# Tool versions #################################
GOLANGCILINT_VERSION := v1.53.3
GOLANGCILINT_VERSION := v1.54.1
GOSIMPORTS_VERSION := v0.3.8
BOUNCER_VERSION := v0.4.0
CHRONICLE_VERSION := v0.6.0
GORELEASER_VERSION := v1.19.2
CHRONICLE_VERSION := v0.7.0
GORELEASER_VERSION := v1.20.0
YAJSV_VERSION := v1.4.1
COSIGN_VERSION := v2.1.1
QUILL_VERSION := v0.2.0
@@ -298,7 +298,7 @@ compare-test-rpm-package-install: $(TEMP_DIR) $(SNAPSHOT_DIR)
$(TEMP_DIR)
## Code generation targets #################################
## Code and data generation targets #################################
.PHONY: generate-json-schema
generate-json-schema: ## Generate a new json schema
@@ -309,11 +309,17 @@ generate-license-list: ## Generate an updated spdx license list
go generate ./internal/spdxlicense/...
gofmt -s -w ./internal/spdxlicense
.PHONY: generate-cpe-dictionary-index
generate-cpe-dictionary-index: ## Build the CPE index based off of the latest available CPE dictionary
$(call title,Building CPE index)
go generate ./syft/pkg/cataloger/common/cpe/dictionary
## Build-related targets #################################
.PHONY: build
build: $(SNAPSHOT_DIR) ## Build release snapshot binaries and packages
build:
CGO_ENABLED=0 go build -trimpath -ldflags "$(LDFLAGS)" -o $@ ./cmd/syft
$(SNAPSHOT_DIR): ## Build snapshot release binaries and packages
$(call title,Building snapshot artifacts)
+14 -1
View File
@@ -53,7 +53,7 @@ For commercial support options with Syft or Grype, please [contact Anchore](http
- Red Hat (rpm)
- Ruby (gem)
- Rust (cargo.lock)
- Swift (cocoapods)
- Swift (cocoapods, swift-package-manager)
## Installation
@@ -211,6 +211,7 @@ You can override the list of enabled/disabled catalogers by using the "cataloger
- ruby-gemfile
- rust-cargo-lock
- sbom
- swift-package-manager
##### Non Default:
- cargo-auditable-binary
@@ -481,6 +482,10 @@ default-image-pull-source: ""
# - "./out/**/*.json"
exclude: []
# allows users to exclude synthetic binary packages from the sbom
# these packages are removed if an overlap with a non-synthetic package is found
exclude-binary-overlap-by-ownership: true
# os and/or architecture to use when referencing container images (e.g. "windows/armv6" or "arm64")
# same as --platform; SYFT_PLATFORM env var
platform: ""
@@ -521,6 +526,7 @@ platform: ""
# - ruby-gemspec-cataloger
# - rust-cargo-lock-cataloger
# - sbom-cataloger
# - spm-cataloger
catalogers:
# cataloging packages is exposed through the packages and power-user subcommands
@@ -575,6 +581,13 @@ linux-kernel:
# SYFT_LINUX_KERNEL_CATALOG_MODULES env var
catalog-modules: true
python:
# when running across entries in requirements.txt that do not specify a specific version
# (e.g. "sqlalchemy >= 1.0.0, <= 2.0.0, != 3.0.0, <= 3.0.0"), attempt to guess what the version could
# be based on the version requirements specified (e.g. "1.0.0"). When enabled the lowest expressible version
# when given an arbitrary constraint will be used (even if that version may not be available/published).
guess-unpinned-requirements: false
# cataloging file contents is exposed through the power-user subcommand
file-contents:
cataloger:
+9 -3
View File
@@ -15,6 +15,7 @@ import (
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/file"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
@@ -101,14 +102,19 @@ func execWorker(app *config.Application, userInput string, writer sbom.Writer) <
},
)
if src != nil {
defer src.Close()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", userInput, err)
return
}
defer func() {
if src != nil {
if err := src.Close(); err != nil {
log.Tracef("unable to close source: %+v", err)
}
}
}()
s, err := GenerateSBOM(src, errs, app)
if err != nil {
errs <- err
+6 -5
View File
@@ -19,6 +19,12 @@ func runModel(t testing.TB, m tea.Model, iterations int, message tea.Msg, h ...*
return message
}
for _, each := range h {
if each != nil {
each.Wait()
}
}
for i := 0; cmd != nil && i < iterations; i++ {
msgs := flatten(cmd())
var nextCmds []tea.Cmd
@@ -31,11 +37,6 @@ func runModel(t testing.TB, m tea.Model, iterations int, message tea.Msg, h ...*
cmd = tea.Batch(nextCmds...)
}
for _, each := range h {
if each != nil {
each.Wait()
}
}
return m.View()
}
+5 -2
View File
@@ -82,12 +82,15 @@ func (m *UI) Teardown(force bool) error {
if !force {
m.handler.Running.Wait()
m.program.Quit()
// typically in all cases we would want to wait for the UI to finish. However there are still error cases
// that are not accounted for, resulting in hangs. For now, we'll just wait for the UI to finish in the
// happy path only. There will always be an indication of the problem to the user via reporting the error
// string from the worker (outside of the UI after teardown).
m.running.Wait()
} else {
m.program.Kill()
}
m.running.Wait()
// TODO: allow for writing out the full log output to the screen (only a partial log is shown currently)
// this needs coordination to know what the last frame event is to change the state accordingly (which isn't possible now)
+30 -25
View File
@@ -16,7 +16,7 @@ require (
github.com/go-test/deep v1.1.0
github.com/google/go-cmp v0.5.9
github.com/google/uuid v1.3.0
github.com/gookit/color v1.5.3
github.com/gookit/color v1.5.4
github.com/hashicorp/go-multierror v1.1.1
github.com/jinzhu/copier v0.3.5
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
@@ -31,20 +31,19 @@ require (
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e
github.com/sergi/go-diff v1.3.1
github.com/sirupsen/logrus v1.9.3
github.com/spdx/tools-golang v0.5.2
github.com/spdx/tools-golang v0.5.3
github.com/spf13/afero v1.9.5
github.com/spf13/cobra v1.7.0
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.16.0
github.com/stretchr/testify v1.8.4
github.com/vifraa/gopom v0.2.1
github.com/vifraa/gopom v1.0.0
github.com/wagoodman/go-partybus v0.0.0-20230516145632-8ccac152c651
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/mod v0.12.0
golang.org/x/net v0.12.0
golang.org/x/term v0.10.0
golang.org/x/net v0.14.0
golang.org/x/term v0.11.0
gopkg.in/yaml.v2 v2.4.0
)
@@ -52,42 +51,47 @@ require (
github.com/CycloneDX/cyclonedx-go v0.7.1
github.com/Masterminds/semver v1.5.0
github.com/Masterminds/sprig/v3 v3.2.3
github.com/anchore/bubbly v0.0.0-20230712165553-812110ab0a10
github.com/anchore/bubbly v0.0.0-20230801194016-acdb4981b461
github.com/anchore/clio v0.0.0-20230602170917-e747e60c4aa0
github.com/anchore/go-logger v0.0.0-20230531193951-db5ae83e7dbe
github.com/anchore/stereoscope v0.0.0-20230627195312-cd49355d934e
github.com/anchore/stereoscope v0.0.0-20230727211946-d1f3d766295e
github.com/aquasecurity/go-pep440-version v0.0.0-20210121094942-22b2f8951d46
github.com/charmbracelet/bubbletea v0.24.2
github.com/charmbracelet/lipgloss v0.7.1
github.com/dave/jennifer v1.6.1
github.com/dave/jennifer v1.7.0
github.com/deitch/magic v0.0.0-20230404182410-1ff89d7342da
github.com/docker/docker v24.0.2+incompatible
github.com/docker/distribution v2.8.2+incompatible
github.com/docker/docker v24.0.5+incompatible
github.com/github/go-spdx/v2 v2.1.2
github.com/gkampitakis/go-snaps v0.4.7
github.com/go-git/go-billy/v5 v5.4.1
github.com/go-git/go-git/v5 v5.7.0
github.com/google/go-containerregistry v0.15.2
github.com/go-git/go-git/v5 v5.8.1
github.com/google/go-containerregistry v0.16.1
github.com/google/licensecheck v0.3.1
github.com/invopop/jsonschema v0.7.0
github.com/knqyf263/go-rpmdb v0.0.0-20230301153543-ba94b245509b
github.com/opencontainers/go-digest v1.0.0
github.com/saferwall/pe v1.4.4
github.com/sassoftware/go-rpmutils v0.2.0
github.com/vbatts/go-mtree v0.5.3
github.com/zyedidia/generic v1.2.2-0.20230320175451-4410d2372cb1
golang.org/x/exp v0.0.0-20230202163644-54bba9f4231b
gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v1.24.0
modernc.org/sqlite v1.25.0
)
require (
dario.cat/mergo v1.0.0 // indirect
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
github.com/DataDog/zstd v1.4.5 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.2.0 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20230518184743-7afd39499903 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95 // indirect
github.com/acomagu/bufpipe v1.0.4 // indirect
github.com/anchore/fangs v0.0.0-20230531202914-48a718c6b4ba // indirect
github.com/anchore/go-struct-converter v0.0.0-20221118182256-c68fdcfa2092 // indirect
github.com/aquasecurity/go-version v0.0.0-20210121072130-637058cfe492 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/becheran/wildmatch-go v1.0.0 // indirect
github.com/charmbracelet/bubbles v0.16.1 // indirect
@@ -97,12 +101,12 @@ require (
github.com/containerd/containerd v1.7.0 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/docker/cli v23.0.5+incompatible // indirect
github.com/docker/distribution v2.8.2+incompatible // indirect
github.com/docker/cli v24.0.0+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
github.com/edsrzf/mmap-go v1.1.0 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/felixge/fgprof v0.9.3 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
@@ -154,12 +158,12 @@ require (
github.com/rivo/uniseg v0.2.0 // indirect
github.com/rogpeppe/go-internal v1.9.0 // indirect
github.com/shopspring/decimal v1.2.0 // indirect
github.com/skeema/knownhosts v1.1.1 // indirect
github.com/skeema/knownhosts v1.2.0 // indirect
github.com/spf13/cast v1.5.1 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/stretchr/objx v0.5.0 // indirect
github.com/subosito/gotenv v1.4.2 // indirect
github.com/sylabs/sif/v2 v2.8.1 // indirect
github.com/sylabs/sif/v2 v2.11.5 // indirect
github.com/sylabs/squashfs v0.6.1 // indirect
github.com/therootcompany/xz v1.0.1 // indirect
github.com/tidwall/gjson v1.14.4 // indirect
@@ -173,10 +177,11 @@ require (
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.10.0 // indirect
golang.org/x/text v0.11.0 // indirect
golang.org/x/tools v0.8.0 // indirect
go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352 // indirect
golang.org/x/sync v0.2.0 // indirect
golang.org/x/sys v0.11.0 // indirect
golang.org/x/text v0.12.0 // indirect
golang.org/x/tools v0.9.1 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
google.golang.org/grpc v1.55.0 // indirect
@@ -186,9 +191,9 @@ require (
lukechampine.com/uint128 v1.2.0 // indirect
modernc.org/cc/v3 v3.40.0 // indirect
modernc.org/ccgo/v3 v3.16.13 // indirect
modernc.org/libc v1.22.5 // indirect
modernc.org/libc v1.24.1 // indirect
modernc.org/mathutil v1.5.0 // indirect
modernc.org/memory v1.5.0 // indirect
modernc.org/memory v1.6.0 // indirect
modernc.org/opt v0.1.3 // indirect
modernc.org/strutil v1.1.3 // indirect
modernc.org/token v1.0.1 // indirect
@@ -199,7 +204,7 @@ require (
// go: warning: github.com/andybalholm/brotli@v1.0.1: retracted by module author: occasional panics and data corruption
github.com/andybalholm/brotli v1.0.4 // indirect
github.com/pkg/errors v0.9.1 // indirect
golang.org/x/crypto v0.11.0 // indirect
golang.org/x/crypto v0.12.0 // indirect
)
retract (
+65 -53
View File
@@ -48,6 +48,8 @@ cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohl
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
cloud.google.com/go/storage v1.14.0/go.mod h1:GrKmX003DSIwi9o29oFT7YDnHYwZoctc3fOKtUw0Xmo=
dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 h1:UQHMgLO+TxOElx5B5HZ4hJQsoJ/PvUvKRhJHDQXO8P8=
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
@@ -72,8 +74,8 @@ github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v
github.com/Microsoft/go-winio v0.6.1 h1:9/kr64B9VUZrLm5YYwbGtUJnMgqWVOdUAXu6Migciow=
github.com/Microsoft/go-winio v0.6.1/go.mod h1:LRdKpFKfdobln8UmuiYcKPot9D2v6svN5+sAH+4kjUM=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/ProtonMail/go-crypto v0.0.0-20230518184743-7afd39499903 h1:ZK3C5DtzV2nVAQTx5S5jQvMeDqWtD1By5mOoyY/xJek=
github.com/ProtonMail/go-crypto v0.0.0-20230518184743-7afd39499903/go.mod h1:8TI4H3IbrackdNgv+92dI+rhpCaLqM0IfpgCgenFvRE=
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95 h1:KLq8BE0KwCL+mmXnjLWEAOYO+2l2AE4YMmqG1ZpZHBs=
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95/go.mod h1:EjAoLdwvbIOoOQr3ihjnSoLZRtE8azugULFRteWMNc0=
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d h1:licZJFw2RwpHMqeKTCYkitsPqHNxTmd4SNR5r94FGM8=
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d/go.mod h1:asat636LX7Bqt5lYEZ27JNDcqxfjdBQuJ/MM4CN/Lzo=
github.com/acobaugh/osrelease v0.1.0 h1:Yb59HQDGGNhCj4suHaFQQfBps5wyoKLSSX/J/+UifRE=
@@ -86,8 +88,8 @@ github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuy
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/anchore/bubbly v0.0.0-20230712165553-812110ab0a10 h1:Wrqt9fd8ygEMyFtxncZU7RgW2qBu5CL1x876xIyUlPU=
github.com/anchore/bubbly v0.0.0-20230712165553-812110ab0a10/go.mod h1:Ger02eh5NpPm2IqkPAy396HU1KlK3BhOeCljDYXySSk=
github.com/anchore/bubbly v0.0.0-20230801194016-acdb4981b461 h1:xGu4/uMWucwWV0YV3fpFIQZ6KVfS/Wfhmma8t0s0vRo=
github.com/anchore/bubbly v0.0.0-20230801194016-acdb4981b461/go.mod h1:Ger02eh5NpPm2IqkPAy396HU1KlK3BhOeCljDYXySSk=
github.com/anchore/clio v0.0.0-20230602170917-e747e60c4aa0 h1:g0UqRW60JDrf5fb40RUyIwwcfQ3nAJqGj4aUCVTwFE4=
github.com/anchore/clio v0.0.0-20230602170917-e747e60c4aa0/go.mod h1:0IQVIROfgRX4WZFMfgsbNZmMgLKqW/KgByyJDYvWiDE=
github.com/anchore/fangs v0.0.0-20230531202914-48a718c6b4ba h1:tJ186HK8e0Lf+hhNWX4fJrq14yj3mw8JQkkLhA0nFhE=
@@ -104,14 +106,18 @@ github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZV
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501 h1:AV7qjwMcM4r8wFhJq3jLRztew3ywIyPTRapl2T1s9o8=
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501/go.mod h1:Blo6OgJNiYF41ufcgHKkbCKF2MDOMlrqhXv/ij6ocR4=
github.com/anchore/stereoscope v0.0.0-20230627195312-cd49355d934e h1:zhk3ZLtomMJ750nNCE+c24PonMzoO/SeL/4uTr1L9kM=
github.com/anchore/stereoscope v0.0.0-20230627195312-cd49355d934e/go.mod h1:0LsgHgXO4QFnk2hsYwtqd3fR18PIZXlFLIl2qb9tu3g=
github.com/anchore/stereoscope v0.0.0-20230727211946-d1f3d766295e h1:S6IhYpsBCpvphlHA1tN0glSG/kjVvFzC6OJuU2qW5Pc=
github.com/anchore/stereoscope v0.0.0-20230727211946-d1f3d766295e/go.mod h1:0LsgHgXO4QFnk2hsYwtqd3fR18PIZXlFLIl2qb9tu3g=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
github.com/andybalholm/brotli v1.0.4 h1:V7DdXeJtZscaqfNuAdSRuRFzuiKlHSC/Zh3zl9qY3JY=
github.com/andybalholm/brotli v1.0.4/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/aquasecurity/go-pep440-version v0.0.0-20210121094942-22b2f8951d46 h1:vmXNl+HDfqqXgr0uY1UgK1GAhps8nbAAtqHNBcgyf+4=
github.com/aquasecurity/go-pep440-version v0.0.0-20210121094942-22b2f8951d46/go.mod h1:olhPNdiiAAMiSujemd1O/sc6GcyePr23f/6uGKtthNg=
github.com/aquasecurity/go-version v0.0.0-20210121072130-637058cfe492 h1:rcEG5HI490FF0a7zuvxOxen52ddygCfNVjP0XOCMl+M=
github.com/aquasecurity/go-version v0.0.0-20210121072130-637058cfe492/go.mod h1:9Beu8XsUNNfzml7WBf3QmyPToP1wm1Gj/Vc5UJKqTzU=
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=
github.com/armon/go-metrics v0.3.10/go.mod h1:4O98XIr/9W0sxpJ8UaYkvjk10Iff7SnFrb4QAOwNTFc=
@@ -129,7 +135,7 @@ github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kB
github.com/bmatcuk/doublestar/v4 v4.6.0 h1:HTuxyug8GyFbRkrffIpzNCSK4luc0TY3wzXvzIZhEXc=
github.com/bmatcuk/doublestar/v4 v4.6.0/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/bradleyjkemp/cupaloy/v2 v2.8.0 h1:any4BmKE+jGIaMpnU8YgH/I2LPiLBufr6oMMlVBbn9M=
github.com/bwesterb/go-ristretto v1.2.0/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
github.com/bwesterb/go-ristretto v1.2.3/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/census-instrumentation/opencensus-proto v0.3.0/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
@@ -149,7 +155,6 @@ github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMn
github.com/circonus-labs/circonus-gometrics v2.3.1+incompatible/go.mod h1:nmEj6Dob7S7YxXgwXpfOuvO54S+tGdZdw9fuRZt25Ag=
github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp5jckzBHf4XRpQvBOLI+I=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cloudflare/circl v1.1.0/go.mod h1:prBCrKB9DV4poKZY1l9zBXg2QJY7mvgRvtMxxK7fi4I=
github.com/cloudflare/circl v1.3.3 h1:fE/Qz0QdIGqeWfnwq0RE0R7MI51s0M2E4Ga9kq5AEMs=
github.com/cloudflare/circl v1.3.3/go.mod h1:5XYMA4rFBvNIrhs50XuiBJ15vF2pZn4nnUKZrLbUZFA=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
@@ -170,23 +175,24 @@ github.com/containerd/stargz-snapshotter/estargz v0.14.3 h1:OqlDCK3ZVUO6C3B/5FSk
github.com/containerd/stargz-snapshotter/estargz v0.14.3/go.mod h1:KY//uOCIkSuNAHhJogcZtrNHdKrA99/FCCRjE3HD36o=
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/cpuguy83/go-md2man/v2 v2.0.1/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/dave/jennifer v1.6.1 h1:T4T/67t6RAA5AIV6+NP8Uk/BIsXgDoqEowgycdQQLuk=
github.com/dave/jennifer v1.6.1/go.mod h1:nXbxhEmQfOZhWml3D1cDK5M1FLnMSozpbFN/m3RmGZc=
github.com/dave/jennifer v1.7.0 h1:uRbSBH9UTS64yXbh4FrMHfgfY762RD+C7bUPKODpSJE=
github.com/dave/jennifer v1.7.0/go.mod h1:nXbxhEmQfOZhWml3D1cDK5M1FLnMSozpbFN/m3RmGZc=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/deitch/magic v0.0.0-20230404182410-1ff89d7342da h1:ZOjWpVsFZ06eIhnh4mkaceTiVoktdU67+M7KDHJ268M=
github.com/deitch/magic v0.0.0-20230404182410-1ff89d7342da/go.mod h1:B3tI9iGHi4imdLi4Asdha1Sc6feLMTfPLXh9IUYmysk=
github.com/dgrijalva/jwt-go/v4 v4.0.0-preview1/go.mod h1:+hnT3ywWDTAFrW5aE+u2Sa/wT555ZqwoCS+pk3p6ry4=
github.com/docker/cli v23.0.5+incompatible h1:ufWmAOuD3Vmr7JP2G5K3cyuNC4YZWiAsuDEvFVVDafE=
github.com/docker/cli v23.0.5+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/cli v24.0.0+incompatible h1:0+1VshNwBQzQAx9lOl+OYCTCEAD8fKs/qeXMx3O0wqM=
github.com/docker/cli v24.0.0+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.2+incompatible h1:T3de5rq0dB1j30rp0sA2rER+m322EBzniBPB6ZIzuh8=
github.com/docker/distribution v2.8.2+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v24.0.2+incompatible h1:eATx+oLz9WdNVkQrr0qjQ8HvRJ4bOOxfzEo8R+dA3cg=
github.com/docker/docker v24.0.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v24.0.5+incompatible h1:WmgcE4fxyI6EEXxBRxsHnZXrO1pQ3smi0k/jho4HLeY=
github.com/docker/docker v24.0.5+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.7.0 h1:xtCHsjxogADNZcdv1pKUHXryefjlVRqWqIhk/uXJp0A=
github.com/docker/docker-credential-helpers v0.7.0/go.mod h1:rETQfLdHNT3foU5kuNkFR1R1V12OJRRO5lzt2D1b5X0=
github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
@@ -198,6 +204,8 @@ github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5/go.mod h1:qssHWj6
github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdfkVLjJ8T6VcRQv3SXugXy999NBtR9aFY=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/edsrzf/mmap-go v1.1.0 h1:6EUwBLQ/Mcr1EYLE4Tn1VdW1A4ckqCQWZBw8Hr0kjpQ=
github.com/edsrzf/mmap-go v1.1.0/go.mod h1:19H/e8pUPLicwkyNgOykDXkJ9F0MHE+Z52B8EIth78Q=
github.com/elazarl/goproxy v0.0.0-20221015165544-a0805db90819 h1:RIB4cRk+lBqKK3Oy0r2gRX4ui7tuhiZq2SuTtTCi0/0=
github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ=
@@ -244,8 +252,8 @@ github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmS
github.com/go-git/go-billy/v5 v5.4.1 h1:Uwp5tDRkPr+l/TnbHOQzp+tmJfLceOlbVucgpTz8ix4=
github.com/go-git/go-billy/v5 v5.4.1/go.mod h1:vjbugF6Fz7JIflbVpl1hJsGjSHNltrSw45YK/ukIvQg=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20230305113008-0c11038e723f h1:Pz0DHeFij3XFhoBRGUDPzSJ+w2UcK5/0JvF8DRI58r8=
github.com/go-git/go-git/v5 v5.7.0 h1:t9AudWVLmqzlo+4bqdf7GY+46SUuRsx59SboFxkq2aE=
github.com/go-git/go-git/v5 v5.7.0/go.mod h1:coJHKEOk5kUClpsNlXrUvPrDxY3w3gjHvhcZd8Fodw8=
github.com/go-git/go-git/v5 v5.8.1 h1:Zo79E4p7TRk0xoRgMq0RShiTHGKcKI4+DI6BfJc/Q+A=
github.com/go-git/go-git/v5 v5.8.1/go.mod h1:FHFuoD6yGz5OSKEBK+aWN9Oah0q54Jxl0abmj6GnqAo=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
@@ -317,8 +325,8 @@ github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-containerregistry v0.15.2 h1:MMkSh+tjSdnmJZO7ljvEqV1DjfekB6VUEAZgy3a+TQE=
github.com/google/go-containerregistry v0.15.2/go.mod h1:wWK+LnOv4jXMM23IT/F1wdYftGWGr47Is8CG+pmHK1Q=
github.com/google/go-containerregistry v0.16.1 h1:rUEt426sR6nyrL3gt+18ibRcvYpKYdpsa5ZW7MA08dQ=
github.com/google/go-containerregistry v0.16.1/go.mod h1:u0qB2l7mvtWVR5kNcbFIhFY1hLbf8eeGapA+vbFDCtQ=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/licensecheck v0.3.1 h1:QoxgoDkaeC4nFrtGN1jV7IPmDCHFNIVh54e5hSt6sPs=
github.com/google/licensecheck v0.3.1/go.mod h1:ORkR35t/JjW+emNKtfJDII0zlciG9JgbT7SmsohlHmY=
@@ -355,8 +363,8 @@ github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pf
github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM=
github.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g=
github.com/gookit/color v1.2.5/go.mod h1:AhIE+pS6D4Ql0SQWbBeXPHw7gY0/sjHoA4s/n1KB7xg=
github.com/gookit/color v1.5.3 h1:twfIhZs4QLCtimkP7MOxlF3A0U/5cDPseRT9M/+2SCE=
github.com/gookit/color v1.5.3/go.mod h1:NUzwzeehUfl7GIb36pqId+UGmRfQcU/WiiyTTeNjHtE=
github.com/gookit/color v1.5.4 h1:FZmqs7XOyGgCAxmWyPslpiok1k05wmY3SJTytgvYFs0=
github.com/gookit/color v1.5.4/go.mod h1:pZJOeOS8DM43rXbp4AZo1n9zCU2qjpcRko0b6/QJi9w=
github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
github.com/hashicorp/consul/api v1.11.0/go.mod h1:XjsvQN+RJGWI2TWy1/kqaE16HrR2J/FWgkYjdZQsX9M=
github.com/hashicorp/consul/sdk v0.8.0/go.mod h1:GBvyrGALthsZObzUGsfgHZQDXjg4lOjagTIwIR1vPms=
@@ -420,7 +428,6 @@ github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHm
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213/go.mod h1:vNUNkEQ1e29fT/6vq2aBdFsgNPmy8qMdSay1npru+Sw=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/kevinburke/ssh_config v1.2.0 h1:x584FjTGwHzMwvHx18PXxbBVzfnxogHaAReU4gf13a4=
@@ -467,7 +474,6 @@ github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.6/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84=
github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
@@ -584,8 +590,11 @@ github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6L
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
github.com/saferwall/pe v1.4.4 h1:Ml++7/2/Z1iKwV4zCsd1nIqTEAdUQKAetwbbcCarhOg=
github.com/saferwall/pe v1.4.4/go.mod h1:SNzv3cdgk8SBI0UwHfyTcdjawfdnN+nbydnEL7GZ25s=
github.com/sagikazarmark/crypt v0.3.0/go.mod h1:uD/D+6UF4SrIR1uGEv7bBNkNqLGqUr43MRiaGWX1Nig=
github.com/sassoftware/go-rpmutils v0.2.0 h1:pKW0HDYMFWQ5b4JQPiI3WI12hGsVoW0V8+GMoZiI/JE=
github.com/sassoftware/go-rpmutils v0.2.0/go.mod h1:TJJQYtLe/BeEmEjelI3b7xNZjzAukEkeWKmoakvaOoI=
@@ -599,18 +608,19 @@ github.com/sergi/go-diff v1.3.1 h1:xkr+Oxo4BOQKmkn/B9eMK0g5Kg/983T9DqqPHwYqD+8=
github.com/sergi/go-diff v1.3.1/go.mod h1:aMJSSKb2lpPvRNec0+w3fl7LP9IOFzdc9Pa4NFbPK1I=
github.com/shopspring/decimal v1.2.0 h1:abSATXmQEYyShuxI4/vyW3tV1MrKAJzCZ/0zLUXYbsQ=
github.com/shopspring/decimal v1.2.0/go.mod h1:DKyhrW/HYNuLGql+MJL6WCR6knT2jwCFRcu2hWCYk4o=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/skeema/knownhosts v1.1.1 h1:MTk78x9FPgDFVFkDLTrsnnfCJl7g1C/nnKvePgrIngE=
github.com/skeema/knownhosts v1.1.1/go.mod h1:g4fPeYpque7P0xefxtGzV81ihjC8sX2IqpAoNkjxbMo=
github.com/skeema/knownhosts v1.2.0 h1:h9r9cf0+u7wSE+M183ZtMGgOJKiL96brpaz5ekfJCpM=
github.com/skeema/knownhosts v1.2.0/go.mod h1:g4fPeYpque7P0xefxtGzV81ihjC8sX2IqpAoNkjxbMo=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spdx/gordf v0.0.0-20201111095634-7098f93598fb/go.mod h1:uKWaldnbMnjsSAXRurWqqrdyZen1R7kxl8TkmWk2OyM=
github.com/spdx/tools-golang v0.5.2 h1:dtMNjJreWPe37584ajk7m/rQtfJaLpRMk7pUGgvekOg=
github.com/spdx/tools-golang v0.5.2/go.mod h1:/ETOahiAo96Ob0/RAIBmFZw6XN0yTnyr/uFZm2NTMhI=
github.com/spdx/tools-golang v0.5.3 h1:ialnHeEYUC4+hkm5vJm4qz2x+oEJbS0mAMFrNXdQraY=
github.com/spdx/tools-golang v0.5.3/go.mod h1:/ETOahiAo96Ob0/RAIBmFZw6XN0yTnyr/uFZm2NTMhI=
github.com/spf13/afero v1.3.3/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4=
github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I=
github.com/spf13/afero v1.9.5 h1:stMpOSZFs//0Lv29HduCmli3GUfpFoF3Y1Q/aXj/wVM=
@@ -650,8 +660,8 @@ github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXl
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/subosito/gotenv v1.4.2 h1:X1TuBLAMDFbaTAChgCBLu3DU3UPyELpnF2jjJ2cz/S8=
github.com/subosito/gotenv v1.4.2/go.mod h1:ayKnFf/c6rvx/2iiLrJUk1e6plDbT3edrFNGqEflhK0=
github.com/sylabs/sif/v2 v2.8.1 h1:whr4Vz12RXfLnYyVGHoD/rD/hbF2g9OW7BJHa+WIqW8=
github.com/sylabs/sif/v2 v2.8.1/go.mod h1:LQOdYXC9a8i7BleTKRw9lohi0rTbXkJOeS9u0ebvgyM=
github.com/sylabs/sif/v2 v2.11.5 h1:7ssPH3epSonsTrzbS1YxeJ9KuqAN7ISlSM61a7j/mQM=
github.com/sylabs/sif/v2 v2.11.5/go.mod h1:GBoZs9LU3e4yJH1dcZ3Akf/jsqYgy5SeguJQC+zd75Y=
github.com/sylabs/squashfs v0.6.1 h1:4hgvHnD9JGlYWwT0bPYNt9zaz23mAV3Js+VEgQoRGYQ=
github.com/sylabs/squashfs v0.6.1/go.mod h1:ZwpbPCj0ocIvMy2br6KZmix6Gzh6fsGQcCnydMF+Kx8=
github.com/therootcompany/xz v1.0.1 h1:CmOtsn1CbtmyYiusbfmhmkpAAETj0wBIH6kCYaX+xzw=
@@ -672,18 +682,17 @@ github.com/ulikunitz/xz v0.5.9/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oW
github.com/ulikunitz/xz v0.5.10 h1:t92gobL9l3HE202wg3rlk19F6X+JOxl9BBrCCMYEYd8=
github.com/ulikunitz/xz v0.5.10/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/urfave/cli v1.22.12/go.mod h1:sSBEIC79qR6OvcmsD4U3KABeOTxDqQtdDnaFuUN30b8=
github.com/urfave/cli/v2 v2.3.0/go.mod h1:LJmUH05zAU44vOAcrfzZQKsZbVcdbOG8rtL3/XcUArI=
github.com/vbatts/go-mtree v0.5.3 h1:S/jYlfG8rZ+a0bhZd+RANXejy7M4Js8fq9U+XoWTd5w=
github.com/vbatts/go-mtree v0.5.3/go.mod h1:eXsdoPMdL2jcJx6HweWi9lYQxBsTp4lNhqqAjgkZUg8=
github.com/vbatts/tar-split v0.11.3 h1:hLFqsOLQ1SsppQNTMpkpPXClLDfC2A3Zgy9OUU+RVck=
github.com/vbatts/tar-split v0.11.3/go.mod h1:9QlHN18E+fEH7RdG+QAJJcuya3rqT7eXSTY7wGrAokY=
github.com/vifraa/gopom v0.2.1 h1:MYVMAMyiGzXPPy10EwojzKIL670kl5Zbae+o3fFvQEM=
github.com/vifraa/gopom v0.2.1/go.mod h1:oPa1dcrGrtlO37WPDBm5SqHAT+wTgF8An1Q71Z6Vv4o=
github.com/vifraa/gopom v1.0.0 h1:L9XlKbyvid8PAIK8nr0lihMApJQg/12OBvMA28BcWh0=
github.com/vifraa/gopom v1.0.0/go.mod h1:oPa1dcrGrtlO37WPDBm5SqHAT+wTgF8An1Q71Z6Vv4o=
github.com/wagoodman/go-partybus v0.0.0-20230516145632-8ccac152c651 h1:jIVmlAFIqV3d+DOxazTR9v+zgj8+VYuQBzPgBZvWBHA=
github.com/wagoodman/go-partybus v0.0.0-20230516145632-8ccac152c651/go.mod h1:b26F2tHLqaoRQf8DywqzVaV1MQ9yvjb0OMcNl7Nxu20=
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5 h1:lwgTsTy18nYqASnH58qyfRW/ldj7Gt2zzBvgYPzdA4s=
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5/go.mod h1:jLXFoL31zFaHKAAyZUh+sxiTDFe1L1ZHrcK2T1itVKA=
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb h1:Yz6VVOcLuWLAHYlJzTw7JKnWxdV/WXpug2X0quEzRnY=
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb/go.mod h1:nDi3BAC5nEbVbg+WSJDHLbjHv0ZToq8nMPA97XMxF3E=
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
@@ -708,6 +717,8 @@ github.com/zyedidia/generic v1.2.2-0.20230320175451-4410d2372cb1/go.mod h1:ly2RB
go.etcd.io/etcd/api/v3 v3.5.1/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs=
go.etcd.io/etcd/client/pkg/v3 v3.5.1/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g=
go.etcd.io/etcd/client/v2 v2.305.1/go.mod h1:pMEacxZW7o8pg4CrFE7pquyCJJzZvkvdD2RibOCCCGs=
go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352 h1:CCriYyAfq1Br1aIYettdHZTy8mBTIPo7We18TuO/bak=
go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352/go.mod h1:SNgMg+EgDFwmvSmLRTNKC5fegJjB7v23qTQ0XLGUNHk=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
@@ -723,7 +734,6 @@ go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9i
go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190228161510-8dd112bcdc25/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
@@ -738,9 +748,10 @@ golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5y
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.3.0/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/crypto v0.3.1-0.20221117191849-2c476679df9a/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
golang.org/x/crypto v0.11.0 h1:6Ewdq3tDic1mg5xRO4milcWCfMVQhI4NkqWWvqejpuA=
golang.org/x/crypto v0.11.0/go.mod h1:xgJhtzW8F9jGdVFWZESrid1U1bjeNy4zgy5cRr/CIio=
golang.org/x/crypto v0.12.0 h1:tFM/ta59kqch6LlvYnPa0yx5a83cL2nHflFhYKvv9Yk=
golang.org/x/crypto v0.12.0/go.mod h1:NF0Gs7EO5K4qLn+Ylc+fih8BSTeIjAP05siRnAh98yw=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -832,8 +843,8 @@ golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/net v0.12.0 h1:cfawfvKITfUsFCeJIHJrbSxpeu/E81khclypR0GVT50=
golang.org/x/net v0.12.0/go.mod h1:zEVYFnQC7m/vmpQFELhcD1EWkZlX69l4oqgmer6hfKA=
golang.org/x/net v0.14.0 h1:BONx9s002vGdD9umnlX1Po8vOZmrgH34qlHcD1MfK14=
golang.org/x/net v0.14.0/go.mod h1:PpSgVXXLK0OxS0F31C1/tv6XNguvCrnXIDrFMspZIUI=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -863,8 +874,9 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.2.0 h1:PUR+T4wwASmuSTYdKjYHI5TD22Wy5ogLU5qZCOLxBrI=
golang.org/x/sync v0.2.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -934,6 +946,7 @@ golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211205182925-97ca703d548d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -945,17 +958,16 @@ golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.10.0 h1:SqMFp9UcQJZa+pmYuAKjd9xq1f0j5rLcDIk0mj4qAsA=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.11.0 h1:eG7RXZHdqOJ1i+0lgLgCpSXAp6M3LYlAo6osgSi0xOM=
golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
golang.org/x/term v0.10.0 h1:3R7pNqamzBraeqj/Tj8qt1aQ2HpmlC+Cx/qL/7hn4/c=
golang.org/x/term v0.10.0/go.mod h1:lpqdcUyK/oCiQxvxVrppt5ggO2KCZ5QblwqPnfZ6d5o=
golang.org/x/term v0.11.0 h1:F9tnn/DA/Im8nCwm+fX+1/eBwi4qFjRT++MhtVC4ZX0=
golang.org/x/term v0.11.0/go.mod h1:zC9APTIj3jG3FdV/Ons+XE1riIZXG4aZ4GTHiPZJPIU=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -968,8 +980,8 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.11.0 h1:LAntKIrcmeSKERyiOh0XMV39LXS8IE9UL2yP7+f5ij4=
golang.org/x/text v0.11.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.12.0 h1:k+n5B8goJNdU7hSvEtMUz3d1Q6D/XW4COJSJR6fN0mc=
golang.org/x/text v0.12.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -1032,8 +1044,8 @@ golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.8.0 h1:vSDcovVPld282ceKgDimkRSC8kpaH1dgyc9UMzlt84Y=
golang.org/x/tools v0.8.0/go.mod h1:JxBZ99ISMI5ViVkT1tr6tdNmXeTrcpVSD3vZ1RsRdN4=
golang.org/x/tools v0.9.1 h1:8WMNJAz3zrtPmnYC7ISf5dEn3MT0gY7jBJfw27yrrLo=
golang.org/x/tools v0.9.1/go.mod h1:owI94Op576fPu3cIGQeHs3joujW/2Oc6MtlxbF5dfNc=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -1232,16 +1244,16 @@ modernc.org/ccgo/v3 v3.16.13 h1:Mkgdzl46i5F/CNR/Kj80Ri59hC8TKAhZrYSaqvkwzUw=
modernc.org/ccgo/v3 v3.16.13/go.mod h1:2Quk+5YgpImhPjv2Qsob1DnZ/4som1lJTodubIcoUkY=
modernc.org/ccorpus v1.11.6 h1:J16RXiiqiCgua6+ZvQot4yUuUy8zxgqbqEEUuGPlISk=
modernc.org/httpfs v1.0.6 h1:AAgIpFZRXuYnkjftxTAZwMIiwEqAfk8aVB2/oA6nAeM=
modernc.org/libc v1.22.5 h1:91BNch/e5B0uPbJFgqbxXuOnxBQjlS//icfQEGmvyjE=
modernc.org/libc v1.22.5/go.mod h1:jj+Z7dTNX8fBScMVNRAYZ/jF91K8fdT2hYMThc3YjBY=
modernc.org/libc v1.24.1 h1:uvJSeCKL/AgzBo2yYIPPTy82v21KgGnizcGYfBHaNuM=
modernc.org/libc v1.24.1/go.mod h1:FmfO1RLrU3MHJfyi9eYYmZBfi/R+tqZ6+hQ3yQQUkak=
modernc.org/mathutil v1.5.0 h1:rV0Ko/6SfM+8G+yKiyI830l3Wuz1zRutdslNoQ0kfiQ=
modernc.org/mathutil v1.5.0/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
modernc.org/memory v1.5.0 h1:N+/8c5rE6EqugZwHii4IFsaJ7MUhoWX07J5tC/iI5Ds=
modernc.org/memory v1.5.0/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU=
modernc.org/memory v1.6.0 h1:i6mzavxrE9a30whzMfwf7XWVODx2r5OYXvU46cirX7o=
modernc.org/memory v1.6.0/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU=
modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sqlite v1.24.0 h1:EsClRIWHGhLTCX44p+Ri/JLD+vFGo0QGjasg2/F9TlI=
modernc.org/sqlite v1.24.0/go.mod h1:OrDj17Mggn6MhE+iPbBNf7RGKODDE9NFT0f3EwDzJqk=
modernc.org/sqlite v1.25.0 h1:AFweiwPNd/b3BoKnBOfFm+Y260guGMF+0UFk0savqeA=
modernc.org/sqlite v1.25.0/go.mod h1:FL3pVXie73rg3Rii6V/u5BoHlSoyeZeIgKZEgHARyCU=
modernc.org/strutil v1.1.3 h1:fNMm+oJklMGYfU9Ylcywl0CO5O6nTfaowNsh2wpPjzY=
modernc.org/strutil v1.1.3/go.mod h1:MEHNA7PdEnEwLvspRMtWTNnp2nnyvMfkimT1NKNAGbw=
modernc.org/tcl v1.15.2 h1:C4ybAYCGJw968e+Me18oW55kD/FexcHbqH2xak1ROSY=
+34 -26
View File
@@ -20,6 +20,7 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger"
golangCataloger "github.com/anchore/syft/syft/pkg/cataloger/golang"
"github.com/anchore/syft/syft/pkg/cataloger/kernel"
pythonCataloger "github.com/anchore/syft/syft/pkg/cataloger/python"
)
var (
@@ -41,30 +42,32 @@ type Application struct {
ConfigPath string `yaml:"configPath,omitempty" json:"configPath" mapstructure:"config"`
Verbosity uint `yaml:"verbosity,omitempty" json:"verbosity" mapstructure:"verbosity"`
// -q, indicates to not show any status output to stderr (ETUI or logging UI)
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"`
Outputs []string `yaml:"output" json:"output" mapstructure:"output"` // -o, the format to use for output
OutputTemplatePath string `yaml:"output-template-path" json:"output-template-path" mapstructure:"output-template-path"` // -t template file to use for output
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Dev development `yaml:"dev" json:"dev" mapstructure:"dev"`
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Catalogers []string `yaml:"catalogers" json:"catalogers" mapstructure:"catalogers"`
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
Golang golang `yaml:"golang" json:"golang" mapstructure:"golang"`
LinuxKernel linuxKernel `yaml:"linux-kernel" json:"linux-kernel" mapstructure:"linux-kernel"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
FileMetadata FileMetadata `yaml:"file-metadata" json:"file-metadata" mapstructure:"file-metadata"`
FileClassification fileClassification `yaml:"file-classification" json:"file-classification" mapstructure:"file-classification"`
FileContents fileContents `yaml:"file-contents" json:"file-contents" mapstructure:"file-contents"`
Secrets secrets `yaml:"secrets" json:"secrets" mapstructure:"secrets"`
Registry registry `yaml:"registry" json:"registry" mapstructure:"registry"`
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
Name string `yaml:"name" json:"name" mapstructure:"name"`
Source sourceCfg `yaml:"source" json:"source" mapstructure:"source"`
Parallelism int `yaml:"parallelism" json:"parallelism" mapstructure:"parallelism"` // the number of catalog workers to run in parallel
DefaultImagePullSource string `yaml:"default-image-pull-source" json:"default-image-pull-source" mapstructure:"default-image-pull-source"` // specify default image pull source
BasePath string `yaml:"base-path" json:"base-path" mapstructure:"base-path"` // specify base path for all file paths
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"`
Outputs []string `yaml:"output" json:"output" mapstructure:"output"` // -o, the format to use for output
OutputTemplatePath string `yaml:"output-template-path" json:"output-template-path" mapstructure:"output-template-path"` // -t template file to use for output
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Dev development `yaml:"dev" json:"dev" mapstructure:"dev"`
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Catalogers []string `yaml:"catalogers" json:"catalogers" mapstructure:"catalogers"`
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
Golang golang `yaml:"golang" json:"golang" mapstructure:"golang"`
LinuxKernel linuxKernel `yaml:"linux-kernel" json:"linux-kernel" mapstructure:"linux-kernel"`
Python python `yaml:"python" json:"python" mapstructure:"python"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
FileMetadata FileMetadata `yaml:"file-metadata" json:"file-metadata" mapstructure:"file-metadata"`
FileClassification fileClassification `yaml:"file-classification" json:"file-classification" mapstructure:"file-classification"`
FileContents fileContents `yaml:"file-contents" json:"file-contents" mapstructure:"file-contents"`
Secrets secrets `yaml:"secrets" json:"secrets" mapstructure:"secrets"`
Registry registry `yaml:"registry" json:"registry" mapstructure:"registry"`
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
Name string `yaml:"name" json:"name" mapstructure:"name"`
Source sourceCfg `yaml:"source" json:"source" mapstructure:"source"`
Parallelism int `yaml:"parallelism" json:"parallelism" mapstructure:"parallelism"` // the number of catalog workers to run in parallel
DefaultImagePullSource string `yaml:"default-image-pull-source" json:"default-image-pull-source" mapstructure:"default-image-pull-source"` // specify default image pull source
BasePath string `yaml:"base-path" json:"base-path" mapstructure:"base-path"` // specify base path for all file paths
ExcludeBinaryOverlapByOwnership bool `yaml:"exclude-binary-overlap-by-ownership" json:"exclude-binary-overlap-by-ownership" mapstructure:"exclude-binary-overlap-by-ownership"` // exclude synthetic binary packages owned by os package files
}
func (cfg Application) ToCatalogerConfig() cataloger.Config {
@@ -74,8 +77,9 @@ func (cfg Application) ToCatalogerConfig() cataloger.Config {
IncludeUnindexedArchives: cfg.Package.SearchUnindexedArchives,
Scope: cfg.Package.Cataloger.ScopeOpt,
},
Catalogers: cfg.Catalogers,
Parallelism: cfg.Parallelism,
Catalogers: cfg.Catalogers,
Parallelism: cfg.Parallelism,
ExcludeBinaryOverlapByOwnership: cfg.ExcludeBinaryOverlapByOwnership,
Golang: golangCataloger.NewGoCatalogerOpts().
WithSearchLocalModCacheLicenses(cfg.Golang.SearchLocalModCacheLicenses).
WithLocalModCacheDir(cfg.Golang.LocalModCacheDir).
@@ -85,6 +89,9 @@ func (cfg Application) ToCatalogerConfig() cataloger.Config {
LinuxKernel: kernel.LinuxCatalogerConfig{
CatalogModules: cfg.LinuxKernel.CatalogModules,
},
Python: pythonCataloger.CatalogerConfig{
GuessUnpinnedRequirements: cfg.Python.GuessUnpinnedRequirements,
},
}
}
@@ -216,6 +223,7 @@ func loadDefaultValues(v *viper.Viper) {
v.SetDefault("catalogers", nil)
v.SetDefault("parallelism", 1)
v.SetDefault("default-image-pull-source", "")
v.SetDefault("exclude-binary-overlap-by-ownership", true)
// for each field in the configuration struct, see if the field implements the defaultValueLoader interface and invoke it if it does
value := reflect.ValueOf(Application{})
+13
View File
@@ -0,0 +1,13 @@
package config
import (
"github.com/spf13/viper"
)
type python struct {
GuessUnpinnedRequirements bool `json:"guess-unpinned-requirements" yaml:"guess-unpinned-requirements" mapstructure:"guess-unpinned-requirements"`
}
func (cfg python) loadDefaultValues(v *viper.Viper) {
v.SetDefault("python.guess-unpinned-requirements", false)
}
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "9.0.0"
JSONSchemaVersion = "10.0.0"
)
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -210,24 +210,33 @@ func collectRelationships(bom *cyclonedx.BOM, s *sbom.SBOM, idMap map[string]int
return
}
for _, d := range *bom.Dependencies {
to, fromExists := idMap[d.Ref].(artifact.Identifiable)
if !fromExists {
continue
}
if d.Dependencies == nil {
continue
}
toPtr, toExists := idMap[d.Ref]
if !toExists {
continue
}
to, ok := common.PtrToStruct(toPtr).(artifact.Identifiable)
if !ok {
continue
}
for _, t := range *d.Dependencies {
from, toExists := idMap[t].(artifact.Identifiable)
if !toExists {
fromPtr, fromExists := idMap[t]
if !fromExists {
continue
}
from, ok := common.PtrToStruct(fromPtr).(artifact.Identifiable)
if !ok {
continue
}
s.Relationships = append(s.Relationships, artifact.Relationship{
From: from,
To: to,
Type: artifact.DependencyOfRelationship, // FIXME this information is lost
// match assumptions in encoding, that this is the only type of relationship captured:
Type: artifact.DependencyOfRelationship,
})
}
}
@@ -8,8 +8,10 @@ import (
"github.com/CycloneDX/cyclonedx-go"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
)
@@ -336,3 +338,101 @@ func Test_missingComponentsDecode(t *testing.T) {
assert.NoError(t, err)
}
func Test_decodeDependencies(t *testing.T) {
c1 := cyclonedx.Component{
Name: "c1",
}
c2 := cyclonedx.Component{
Name: "c2",
}
c3 := cyclonedx.Component{
Name: "c3",
}
for _, c := range []*cyclonedx.Component{&c1, &c2, &c3} {
c.BOMRef = c.Name
}
setTypes := func(typ cyclonedx.ComponentType, components ...cyclonedx.Component) *[]cyclonedx.Component {
var out []cyclonedx.Component
for _, c := range components {
c.Type = typ
out = append(out, c)
}
return &out
}
tests := []struct {
name string
sbom cyclonedx.BOM
expected []string
}{
{
name: "dependencies decoded as dependencyOf relationships",
sbom: cyclonedx.BOM{
Components: setTypes(cyclonedx.ComponentTypeLibrary,
c1,
c2,
c3,
),
Dependencies: &[]cyclonedx.Dependency{
{
Ref: c1.BOMRef,
Dependencies: &[]string{
c2.BOMRef,
c3.BOMRef,
},
},
},
},
expected: []string{c2.Name, c3.Name},
},
{
name: "dependencies skipped with unhandled components",
sbom: cyclonedx.BOM{
Components: setTypes("",
c1,
c2,
c3,
),
Dependencies: &[]cyclonedx.Dependency{
{
Ref: c1.BOMRef,
Dependencies: &[]string{
c2.BOMRef,
c3.BOMRef,
},
},
},
},
expected: nil,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
s, err := ToSyftModel(&test.sbom)
require.NoError(t, err)
require.NotNil(t, s)
var deps []string
if s != nil {
for _, r := range s.Relationships {
if r.Type != artifact.DependencyOfRelationship {
continue
}
if p, ok := r.To.(pkg.Package); !ok || p.Name != c1.Name {
continue
}
if p, ok := r.From.(pkg.Package); ok {
deps = append(deps, p.Name)
}
}
}
require.Equal(t, test.expected, deps)
})
}
}
@@ -143,27 +143,29 @@ func toDependencies(relationships []artifact.Relationship) []cyclonedx.Dependenc
for _, r := range relationships {
exists := isExpressiblePackageRelationship(r.Type)
if !exists {
log.Debugf("unable to convert relationship from CycloneDX 1.4 JSON, dropping: %+v", r)
log.Debugf("unable to convert relationship type to CycloneDX JSON, dropping: %#v", r)
continue
}
// we only capture package-to-package relationships for now
fromPkg, ok := r.From.(*pkg.Package)
fromPkg, ok := r.From.(pkg.Package)
if !ok {
log.Tracef("unable to convert relationship fromPkg to CycloneDX JSON, dropping: %#v", r)
continue
}
toPkg, ok := r.To.(*pkg.Package)
toPkg, ok := r.To.(pkg.Package)
if !ok {
log.Tracef("unable to convert relationship toPkg to CycloneDX JSON, dropping: %#v", r)
continue
}
// ind dep
innerDeps := []string{}
innerDeps = append(innerDeps, deriveBomRef(*fromPkg))
innerDeps = append(innerDeps, deriveBomRef(fromPkg))
result = append(result, cyclonedx.Dependency{
Ref: deriveBomRef(*toPkg),
Ref: deriveBomRef(toPkg),
Dependencies: &innerDeps,
})
}
@@ -3,7 +3,13 @@ package cyclonedxhelpers
import (
"testing"
"github.com/CycloneDX/cyclonedx-go"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
)
func Test_formatCPE(t *testing.T) {
@@ -32,3 +38,98 @@ func Test_formatCPE(t *testing.T) {
})
}
}
func Test_relationships(t *testing.T) {
p1 := pkg.Package{
Name: "p1",
}
p1.SetID()
p2 := pkg.Package{
Name: "p2",
}
p2.SetID()
p3 := pkg.Package{
Name: "p3",
}
p3.SetID()
tests := []struct {
name string
sbom sbom.SBOM
expected []string
}{
{
name: "package dependencyOf relationships output as dependencies",
sbom: sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(p1, p2, p3),
},
Relationships: []artifact.Relationship{
{
From: p2,
To: p1,
Type: artifact.DependencyOfRelationship,
},
{
From: p3,
To: p1,
Type: artifact.DependencyOfRelationship,
},
},
},
expected: []string{p2.Name, p3.Name},
},
{
name: "package contains relationships not output",
sbom: sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(p1, p2, p3),
},
Relationships: []artifact.Relationship{
{
From: p2,
To: p1,
Type: artifact.ContainsRelationship,
},
{
From: p3,
To: p1,
Type: artifact.ContainsRelationship,
},
},
},
expected: nil,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
cdx := ToFormatModel(test.sbom)
got := cdx.Dependencies
var deps []string
if got != nil {
for _, r := range *got {
for _, d := range *r.Dependencies {
c := findComponent(cdx, d)
require.NotNil(t, c)
deps = append(deps, c.Name)
}
}
}
require.Equal(t, test.expected, deps)
})
}
}
func findComponent(cdx *cyclonedx.BOM, bomRef string) *cyclonedx.Component {
for _, c := range *cdx.Components {
if c.BOMRef == bomRef {
return &c
}
}
return nil
}
@@ -4,12 +4,12 @@ import (
"github.com/anchore/syft/syft/source"
)
func DocumentName(srcMetadata source.Description) string {
if srcMetadata.Name != "" {
return srcMetadata.Name
func DocumentName(src source.Description) string {
if src.Name != "" {
return src.Name
}
switch metadata := srcMetadata.Metadata.(type) {
switch metadata := src.Metadata.(type) {
case source.StereoscopeImageSourceMetadata:
return metadata.UserInput
case source.DirectorySourceMetadata:
@@ -17,13 +17,11 @@ func Test_DocumentName(t *testing.T) {
tests := []struct {
name string
inputName string
srcMetadata source.Description
expected string
}{
{
name: "image",
inputName: "my-name",
name: "image",
srcMetadata: source.Description{
Metadata: source.StereoscopeImageSourceMetadata{
UserInput: "image-repo/name:tag",
@@ -34,21 +32,27 @@ func Test_DocumentName(t *testing.T) {
expected: "image-repo/name:tag",
},
{
name: "directory",
inputName: "my-name",
name: "directory",
srcMetadata: source.Description{
Metadata: source.DirectorySourceMetadata{Path: "some/path/to/place"},
},
expected: "some/path/to/place",
},
{
name: "file",
inputName: "my-name",
name: "file",
srcMetadata: source.Description{
Metadata: source.FileSourceMetadata{Path: "some/path/to/place"},
},
expected: "some/path/to/place",
},
{
name: "named",
srcMetadata: source.Description{
Name: "some/name",
Metadata: source.FileSourceMetadata{Path: "some/path/to/place"},
},
expected: "some/name",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
+27 -18
View File
@@ -1,6 +1,8 @@
package spdxhelpers
import (
"crypto/sha256"
"fmt"
"strings"
"github.com/anchore/syft/internal/spdxlicense"
@@ -27,29 +29,18 @@ func License(p pkg.Package) (concluded, declared string) {
// https://spdx.github.io/spdx-spec/v2.3/SPDX-license-expressions/
pc, pd := parseLicenses(p.Licenses.ToSlice())
for i, v := range pc {
if strings.HasPrefix(v, spdxlicense.LicenseRefPrefix) {
pc[i] = SanitizeElementID(v)
}
}
for i, v := range pd {
if strings.HasPrefix(v, spdxlicense.LicenseRefPrefix) {
pd[i] = SanitizeElementID(v)
}
}
return joinLicenses(pc), joinLicenses(pd)
}
func joinLicenses(licenses []string) string {
func joinLicenses(licenses []spdxLicense) string {
if len(licenses) == 0 {
return NOASSERTION
}
var newLicenses []string
for _, v := range licenses {
for _, l := range licenses {
v := l.id
// check if license does not start or end with parens
if !strings.HasPrefix(v, "(") && !strings.HasSuffix(v, ")") {
// if license contains AND, OR, or WITH, then wrap in parens
@@ -66,14 +57,31 @@ func joinLicenses(licenses []string) string {
return strings.Join(newLicenses, " AND ")
}
func parseLicenses(raw []pkg.License) (concluded, declared []string) {
type spdxLicense struct {
id string
value string
}
func parseLicenses(raw []pkg.License) (concluded, declared []spdxLicense) {
for _, l := range raw {
var candidate string
if l.Value == "" {
continue
}
candidate := spdxLicense{}
if l.SPDXExpression != "" {
candidate = l.SPDXExpression
candidate.id = l.SPDXExpression
} else {
// we did not find a valid SPDX license ID so treat as separate license
candidate = spdxlicense.LicenseRefPrefix + l.Value
if len(l.Value) <= 64 {
// if the license text is less than the size of the hash,
// just use it directly so the id is more readable
candidate.id = spdxlicense.LicenseRefPrefix + SanitizeElementID(l.Value)
} else {
hash := sha256.Sum256([]byte(l.Value))
candidate.id = fmt.Sprintf("%s%x", spdxlicense.LicenseRefPrefix, hash)
}
candidate.value = l.Value
}
switch l.Type {
@@ -83,5 +91,6 @@ func parseLicenses(raw []pkg.License) (concluded, declared []string) {
declared = append(declared, candidate)
}
}
return concluded, declared
}
@@ -1,11 +1,16 @@
package spdxhelpers
import (
"strings"
"testing"
"github.com/spdx/tools-golang/spdx"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
)
func Test_License(t *testing.T) {
@@ -103,6 +108,77 @@ func Test_License(t *testing.T) {
}
}
func Test_otherLicenses(t *testing.T) {
pkg1 := pkg.Package{
Name: "first-pkg",
Version: "1.1",
Licenses: pkg.NewLicenseSet(
pkg.NewLicense("MIT"),
),
}
pkg2 := pkg.Package{
Name: "second-pkg",
Version: "2.2",
Licenses: pkg.NewLicenseSet(
pkg.NewLicense("non spdx license"),
),
}
bigText := `
Apache License
Version 2.0, January 2004`
pkg3 := pkg.Package{
Name: "third-pkg",
Version: "3.3",
Licenses: pkg.NewLicenseSet(
pkg.NewLicense(bigText),
),
}
tests := []struct {
name string
packages []pkg.Package
expected []*spdx.OtherLicense
}{
{
name: "no other licenses when all valid spdx expressions",
packages: []pkg.Package{pkg1},
expected: nil,
},
{
name: "other licenses includes original text",
packages: []pkg.Package{pkg2},
expected: []*spdx.OtherLicense{
{
LicenseIdentifier: "LicenseRef-non-spdx-license",
ExtractedText: "non spdx license",
},
},
},
{
name: "big licenses get hashed",
packages: []pkg.Package{pkg3},
expected: []*spdx.OtherLicense{
{
LicenseIdentifier: "LicenseRef-e9a1e42833d3e456f147052f4d312101bd171a0798893169fe596ca6b55c049e",
ExtractedText: bigText,
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
s := sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(test.packages...),
},
}
got := ToFormatModel(s)
require.Equal(t, test.expected, got.OtherLicenses)
})
}
}
func Test_joinLicenses(t *testing.T) {
tests := []struct {
name string
@@ -122,7 +198,18 @@ func Test_joinLicenses(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equalf(t, tt.want, joinLicenses(tt.args), "joinLicenses(%v)", tt.args)
assert.Equalf(t, tt.want, joinLicenses(toSpdxLicenses(tt.args)), "joinLicenses(%v)", tt.args)
})
}
}
func toSpdxLicenses(ids []string) (licenses []spdxLicense) {
for _, l := range ids {
license := spdxLicense{id: l}
if strings.HasPrefix(l, spdxlicense.LicenseRefPrefix) {
license.value = l
}
licenses = append(licenses, license)
}
return licenses
}
@@ -54,6 +54,8 @@ func SourceInfo(p pkg.Package) string {
answer = "acquired package info from nix store path"
case pkg.Rpkg:
answer = "acquired package info from R-package DESCRIPTION file"
case pkg.SwiftPkg:
answer = "acquired package info from resolved Swift package manifest"
default:
answer = "acquired package info from the following paths"
}
@@ -231,6 +231,14 @@ func Test_SourceInfo(t *testing.T) {
"acquired package info from R-package DESCRIPTION file",
},
},
{
input: pkg.Package{
Type: pkg.SwiftPkg,
},
expected: []string{
"from resolved Swift package manifest",
},
},
}
var pkgTypes []pkg.Type
for _, test := range tests {
@@ -9,10 +9,12 @@ import (
"strings"
"time"
"github.com/docker/distribution/reference"
"github.com/spdx/tools-golang/spdx"
"golang.org/x/exp/maps"
"golang.org/x/exp/slices"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/spdxlicense"
@@ -21,10 +23,20 @@ import (
"github.com/anchore/syft/syft/formats/common/util"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
const (
noAssertion = "NOASSERTION"
spdxPrimaryPurposeContainer = "CONTAINER"
spdxPrimaryPurposeFile = "FILE"
spdxPrimaryPurposeOther = "OTHER"
prefixImage = "Image"
prefixDirectory = "Directory"
prefixFile = "File"
prefixUnknown = "Unknown"
)
// ToFormatModel creates and populates a new SPDX document struct that follows the SPDX 2.3
@@ -33,23 +45,37 @@ const (
//nolint:funlen
func ToFormatModel(s sbom.SBOM) *spdx.Document {
name, namespace := DocumentNameAndNamespace(s.Source)
packages := toPackages(s.Artifacts.Packages, s)
relationships := toRelationships(s.RelationshipsSorted())
// for valid SPDX we need a document describes relationship
// TODO: remove this placeholder after deciding on correct behavior
// for the primary package purpose field:
// https://spdx.github.io/spdx-spec/v2.3/package-information/#724-primary-package-purpose-field
describesID := spdx.ElementID("DOCUMENT")
rootPackage := toRootPackage(s.Source)
if rootPackage != nil {
describesID = rootPackage.PackageSPDXIdentifier
// add all relationships from the document root to all other packages
relationships = append(relationships, toRootRelationships(rootPackage, packages)...)
// append the root package
packages = append(packages, rootPackage)
}
// add a relationship for the package the document describes
documentDescribesRelationship := &spdx.Relationship{
RefA: spdx.DocElementID{
ElementRefID: "DOCUMENT",
},
Relationship: string(DescribesRelationship),
RefB: spdx.DocElementID{
ElementRefID: "DOCUMENT",
ElementRefID: describesID,
},
RelationshipComment: "",
}
// add the root document relationship
relationships = append(relationships, documentDescribesRelationship)
return &spdx.Document{
@@ -123,19 +149,133 @@ func ToFormatModel(s sbom.SBOM) *spdx.Document {
// Cardinality: optional, one
CreatorComment: "",
},
Packages: toPackages(s.Artifacts.Packages, s),
Packages: packages,
Files: toFiles(s),
Relationships: relationships,
OtherLicenses: toOtherLicenses(s.Artifacts.Packages),
}
}
func toRootRelationships(rootPackage *spdx.Package, packages []*spdx.Package) (out []*spdx.Relationship) {
for _, p := range packages {
out = append(out, &spdx.Relationship{
RefA: spdx.DocElementID{
ElementRefID: rootPackage.PackageSPDXIdentifier,
},
Relationship: string(ContainsRelationship),
RefB: spdx.DocElementID{
ElementRefID: p.PackageSPDXIdentifier,
},
})
}
return
}
//nolint:funlen
func toRootPackage(s source.Description) *spdx.Package {
var prefix string
name := s.Name
version := s.Version
var purl *packageurl.PackageURL
purpose := ""
var checksums []spdx.Checksum
switch m := s.Metadata.(type) {
case source.StereoscopeImageSourceMetadata:
prefix = prefixImage
purpose = spdxPrimaryPurposeContainer
qualifiers := packageurl.Qualifiers{
{
Key: "arch",
Value: m.Architecture,
},
}
ref, _ := reference.Parse(m.UserInput)
if ref, ok := ref.(reference.NamedTagged); ok {
qualifiers = append(qualifiers, packageurl.Qualifier{
Key: "tag",
Value: ref.Tag(),
})
}
c := toChecksum(m.ManifestDigest)
if c != nil {
checksums = append(checksums, *c)
purl = &packageurl.PackageURL{
Type: "oci",
Name: s.Name,
Version: m.ManifestDigest,
Qualifiers: qualifiers,
}
}
case source.DirectorySourceMetadata:
prefix = prefixDirectory
purpose = spdxPrimaryPurposeFile
case source.FileSourceMetadata:
prefix = prefixFile
purpose = spdxPrimaryPurposeFile
for _, d := range m.Digests {
checksums = append(checksums, spdx.Checksum{
Algorithm: toChecksumAlgorithm(d.Algorithm),
Value: d.Value,
})
}
default:
prefix = prefixUnknown
purpose = spdxPrimaryPurposeOther
if name == "" {
name = s.ID
}
}
p := &spdx.Package{
PackageName: name,
PackageSPDXIdentifier: spdx.ElementID(SanitizeElementID(fmt.Sprintf("DocumentRoot-%s-%s", prefix, name))),
PackageVersion: version,
PackageChecksums: checksums,
PackageExternalReferences: nil,
PrimaryPackagePurpose: purpose,
PackageSupplier: &spdx.Supplier{
Supplier: NOASSERTION,
},
PackageDownloadLocation: NOASSERTION,
}
if purl != nil {
p.PackageExternalReferences = []*spdx.PackageExternalReference{
{
Category: string(PackageManagerReferenceCategory),
RefType: string(PurlExternalRefType),
Locator: purl.String(),
},
}
}
return p
}
func toSPDXID(identifiable artifact.Identifiable) spdx.ElementID {
maxLen := 40
id := ""
switch it := identifiable.(type) {
case pkg.Package:
id = SanitizeElementID(fmt.Sprintf("Package-%s-%s-%s", it.Type, it.Name, it.ID()))
switch {
case it.Type != "" && it.Name != "":
id = fmt.Sprintf("Package-%s-%s-%s", it.Type, it.Name, it.ID())
case it.Name != "":
id = fmt.Sprintf("Package-%s-%s", it.Name, it.ID())
case it.Type != "":
id = fmt.Sprintf("Package-%s-%s", it.Type, it.ID())
default:
id = fmt.Sprintf("Package-%s", it.ID())
}
case file.Coordinates:
p := ""
parts := strings.Split(it.RealPath, "/")
@@ -150,12 +290,12 @@ func toSPDXID(identifiable artifact.Identifiable) spdx.ElementID {
}
p = path.Join(part, p)
}
id = SanitizeElementID(fmt.Sprintf("File-%s-%s", p, it.ID()))
id = fmt.Sprintf("File-%s-%s", p, it.ID())
default:
id = string(identifiable.ID())
}
// NOTE: the spdx library prepend SPDXRef-, so we don't do it here
return spdx.ElementID(id)
return spdx.ElementID(SanitizeElementID(id))
}
// packages populates all Package Information from the package Collection (see https://spdx.github.io/spdx-spec/3-package-information/)
@@ -220,7 +360,7 @@ func toPackages(catalog *pkg.Collection, sbom sbom.SBOM) (results []*spdx.Packag
// 7.6: Package Originator: may have single result for either Person or Organization,
// or NOASSERTION
// Cardinality: optional, one
PackageSupplier: nil,
PackageSupplier: toPackageSupplier(p),
PackageOriginator: toPackageOriginator(p),
@@ -377,6 +517,21 @@ func toPackageOriginator(p pkg.Package) *spdx.Originator {
}
}
func toPackageSupplier(p pkg.Package) *spdx.Supplier {
// this uses the Originator function for now until
// a better distinction can be made for supplier
kind, supplier := Originator(p)
if kind == "" || supplier == "" {
return &spdx.Supplier{
Supplier: NOASSERTION,
}
}
return &spdx.Supplier{
Supplier: supplier,
SupplierType: kind,
}
}
func formatSPDXExternalRefs(p pkg.Package) (refs []*spdx.PackageExternalReference) {
for _, ref := range ExternalRefs(p) {
refs = append(refs, &spdx.PackageExternalReference{
@@ -494,6 +649,18 @@ func toFileChecksums(digests []file.Digest) (checksums []spdx.Checksum) {
return checksums
}
// toChecksum takes a checksum in the format <algorithm>:<hash> and returns an spdx.Checksum or nil if the string is invalid
func toChecksum(algorithmHash string) *spdx.Checksum {
parts := strings.Split(algorithmHash, ":")
if len(parts) < 2 {
return nil
}
return &spdx.Checksum{
Algorithm: toChecksumAlgorithm(parts[0]),
Value: parts[1],
}
}
func toChecksumAlgorithm(algorithm string) spdx.ChecksumAlgorithm {
// this needs to be an uppercase version of our algorithm
return spdx.ChecksumAlgorithm(strings.ToUpper(algorithm))
@@ -537,32 +704,31 @@ func toFileTypes(metadata *file.Metadata) (ty []string) {
// other licenses are for licenses from the pkg.Package that do not have an SPDXExpression
// field. The spdxexpression field is only filled given a validated Value field.
func toOtherLicenses(catalog *pkg.Collection) []*spdx.OtherLicense {
licenses := map[string]bool{}
for _, p := range catalog.Sorted() {
licenses := map[string]spdxLicense{}
for p := range catalog.Enumerate() {
declaredLicenses, concludedLicenses := parseLicenses(p.Licenses.ToSlice())
for _, license := range declaredLicenses {
if strings.HasPrefix(license, spdxlicense.LicenseRefPrefix) {
licenses[license] = true
for _, l := range declaredLicenses {
if l.value != "" {
licenses[l.id] = l
}
}
for _, license := range concludedLicenses {
if strings.HasPrefix(license, spdxlicense.LicenseRefPrefix) {
licenses[license] = true
for _, l := range concludedLicenses {
if l.value != "" {
licenses[l.id] = l
}
}
}
var result []*spdx.OtherLicense
sorted := maps.Keys(licenses)
slices.Sort(sorted)
for _, license := range sorted {
// separate the found value from the prefix
// this only contains licenses that are not found on the SPDX License List
name := strings.TrimPrefix(license, spdxlicense.LicenseRefPrefix)
ids := maps.Keys(licenses)
slices.Sort(ids)
for _, id := range ids {
license := licenses[id]
result = append(result, &spdx.OtherLicense{
LicenseIdentifier: SanitizeElementID(license),
ExtractedText: name,
LicenseIdentifier: license.id,
ExtractedText: license.value,
})
}
return result
@@ -5,17 +5,263 @@ import (
"regexp"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/spdx/v2/v2_3"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/internal/sourcemetadata"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
// TODO: Add ToFormatModel tests
func Test_toFormatModel(t *testing.T) {
tracker := sourcemetadata.NewCompletionTester(t)
tests := []struct {
name string
in sbom.SBOM
expected *spdx.Document
}{
{
name: "container",
in: sbom.SBOM{
Source: source.Description{
Name: "alpine",
Version: "sha256:d34db33f",
Metadata: source.StereoscopeImageSourceMetadata{
UserInput: "alpine:latest",
ManifestDigest: "sha256:d34db33f",
},
},
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(pkg.Package{
Name: "pkg-1",
Version: "version-1",
}),
},
},
expected: &spdx.Document{
SPDXIdentifier: "DOCUMENT",
SPDXVersion: spdx.Version,
DataLicense: spdx.DataLicense,
DocumentName: "alpine",
Packages: []*spdx.Package{
{
PackageSPDXIdentifier: "Package-pkg-1-pkg-1",
PackageName: "pkg-1",
PackageVersion: "version-1",
PackageSupplier: &spdx.Supplier{
Supplier: "NOASSERTION",
},
},
{
PackageSPDXIdentifier: "DocumentRoot-Image-alpine",
PackageName: "alpine",
PackageVersion: "sha256:d34db33f",
PrimaryPackagePurpose: "CONTAINER",
PackageChecksums: []spdx.Checksum{{Algorithm: "SHA256", Value: "d34db33f"}},
PackageExternalReferences: []*v2_3.PackageExternalReference{
{
Category: "PACKAGE-MANAGER",
RefType: "purl",
Locator: "pkg:oci/alpine@sha256:d34db33f?arch=&tag=latest",
},
},
PackageSupplier: &spdx.Supplier{
Supplier: "NOASSERTION",
},
},
},
Relationships: []*spdx.Relationship{
{
RefA: spdx.DocElementID{
ElementRefID: "DocumentRoot-Image-alpine",
},
RefB: spdx.DocElementID{
ElementRefID: "Package-pkg-1-pkg-1",
},
Relationship: spdx.RelationshipContains,
},
{
RefA: spdx.DocElementID{
ElementRefID: "DOCUMENT",
},
RefB: spdx.DocElementID{
ElementRefID: "DocumentRoot-Image-alpine",
},
Relationship: spdx.RelationshipDescribes,
},
},
},
},
{
name: "directory",
in: sbom.SBOM{
Source: source.Description{
Name: "some/directory",
Metadata: source.DirectorySourceMetadata{
Path: "some/directory",
},
},
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(pkg.Package{
Name: "pkg-1",
Version: "version-1",
}),
},
},
expected: &spdx.Document{
SPDXIdentifier: "DOCUMENT",
SPDXVersion: spdx.Version,
DataLicense: spdx.DataLicense,
DocumentName: "some/directory",
Packages: []*spdx.Package{
{
PackageSPDXIdentifier: "Package-pkg-1-pkg-1",
PackageName: "pkg-1",
PackageVersion: "version-1",
PackageSupplier: &spdx.Supplier{
Supplier: "NOASSERTION",
},
},
{
PackageSPDXIdentifier: "DocumentRoot-Directory-some-directory",
PackageName: "some/directory",
PackageVersion: "",
PrimaryPackagePurpose: "FILE",
PackageSupplier: &spdx.Supplier{
Supplier: "NOASSERTION",
},
},
},
Relationships: []*spdx.Relationship{
{
RefA: spdx.DocElementID{
ElementRefID: "DocumentRoot-Directory-some-directory",
},
RefB: spdx.DocElementID{
ElementRefID: "Package-pkg-1-pkg-1",
},
Relationship: spdx.RelationshipContains,
},
{
RefA: spdx.DocElementID{
ElementRefID: "DOCUMENT",
},
RefB: spdx.DocElementID{
ElementRefID: "DocumentRoot-Directory-some-directory",
},
Relationship: spdx.RelationshipDescribes,
},
},
},
},
{
name: "file",
in: sbom.SBOM{
Source: source.Description{
Name: "path/to/some.file",
Version: "sha256:d34db33f",
Metadata: source.FileSourceMetadata{
Path: "path/to/some.file",
Digests: []file.Digest{
{
Algorithm: "sha256",
Value: "d34db33f",
},
},
},
},
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(pkg.Package{
Name: "pkg-1",
Version: "version-1",
}),
},
},
expected: &spdx.Document{
SPDXIdentifier: "DOCUMENT",
SPDXVersion: spdx.Version,
DataLicense: spdx.DataLicense,
DocumentName: "path/to/some.file",
Packages: []*spdx.Package{
{
PackageSPDXIdentifier: "Package-pkg-1-pkg-1",
PackageName: "pkg-1",
PackageVersion: "version-1",
PackageSupplier: &spdx.Supplier{
Supplier: "NOASSERTION",
},
},
{
PackageSPDXIdentifier: "DocumentRoot-File-path-to-some.file",
PackageName: "path/to/some.file",
PackageVersion: "sha256:d34db33f",
PrimaryPackagePurpose: "FILE",
PackageChecksums: []spdx.Checksum{{Algorithm: "SHA256", Value: "d34db33f"}},
PackageSupplier: &spdx.Supplier{
Supplier: "NOASSERTION",
},
},
},
Relationships: []*spdx.Relationship{
{
RefA: spdx.DocElementID{
ElementRefID: "DocumentRoot-File-path-to-some.file",
},
RefB: spdx.DocElementID{
ElementRefID: "Package-pkg-1-pkg-1",
},
Relationship: spdx.RelationshipContains,
},
{
RefA: spdx.DocElementID{
ElementRefID: "DOCUMENT",
},
RefB: spdx.DocElementID{
ElementRefID: "DocumentRoot-File-path-to-some.file",
},
Relationship: spdx.RelationshipDescribes,
},
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
tracker.Tested(t, test.in.Source.Metadata)
// replace IDs with package names
var pkgs []pkg.Package
for p := range test.in.Artifacts.Packages.Enumerate() {
p.OverrideID(artifact.ID(p.Name))
pkgs = append(pkgs, p)
}
test.in.Artifacts.Packages = pkg.NewCollection(pkgs...)
// convert
got := ToFormatModel(test.in)
// check differences
if diff := cmp.Diff(test.expected, got,
cmpopts.IgnoreUnexported(spdx.Document{}, spdx.Package{}),
cmpopts.IgnoreFields(spdx.Document{}, "CreationInfo", "DocumentNamespace"),
cmpopts.IgnoreFields(spdx.Package{}, "PackageDownloadLocation", "IsFilesAnalyzedTagPresent", "PackageSourceInfo", "PackageLicenseConcluded", "PackageLicenseDeclared", "PackageCopyrightText"),
); diff != "" {
t.Error(diff)
}
})
}
}
func Test_toPackageChecksums(t *testing.T) {
tests := []struct {
name string
+242 -29
View File
@@ -2,14 +2,19 @@ package spdxhelpers
import (
"errors"
"fmt"
"net/url"
"path"
"regexp"
"strconv"
"strings"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/spdx/v2/common"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/file"
@@ -26,12 +31,10 @@ func ToSyftModel(doc *spdx.Document) (*sbom.SBOM, error) {
return nil, errors.New("cannot convert SPDX document to Syft model because document is nil")
}
spdxIDMap := make(map[string]interface{})
src := extractSourceFromNamespace(doc.DocumentNamespace)
spdxIDMap := make(map[string]any)
s := &sbom.SBOM{
Source: src,
Source: extractSource(spdxIDMap, doc),
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(),
FileMetadata: map[file.Coordinates]file.Metadata{},
@@ -40,7 +43,7 @@ func ToSyftModel(doc *spdx.Document) (*sbom.SBOM, error) {
},
}
collectSyftPackages(s, spdxIDMap, doc)
collectSyftPackages(s, spdxIDMap, doc.Packages)
collectSyftFiles(s, spdxIDMap, doc)
@@ -49,6 +52,166 @@ func ToSyftModel(doc *spdx.Document) (*sbom.SBOM, error) {
return s, nil
}
func isDirectory(name string) bool {
if name == "." || name == ".." || strings.HasSuffix(name, "/") || !strings.Contains(path.Base(name), ".") {
return true
}
return false
}
func removePackage(packages []*spdx.Package, remove *spdx.Package) (pkgs []*spdx.Package) {
for _, p := range packages {
if p == remove {
continue
}
pkgs = append(pkgs, p)
}
return
}
func removeRelationships(relationships []*spdx.Relationship, spdxID spdx.ElementID) (relations []*spdx.Relationship) {
for _, r := range relationships {
if r.RefA.ElementRefID == spdxID || r.RefB.ElementRefID == spdxID {
continue
}
relations = append(relations, r)
}
return
}
func findRootPackages(doc *spdx.Document) (out []*spdx.Package) {
for _, p := range doc.Packages {
for _, r := range doc.Relationships {
describes := r.RefA.ElementRefID == "DOCUMENT" &&
r.Relationship == spdx.RelationshipDescribes &&
r.RefB.ElementRefID == p.PackageSPDXIdentifier
describedBy := r.RefB.ElementRefID == "DOCUMENT" &&
r.Relationship == spdx.RelationshipDescribedBy &&
r.RefA.ElementRefID == p.PackageSPDXIdentifier
if !describes && !describedBy {
continue
}
out = append(out, p)
}
}
return
}
func extractSource(spdxIDMap map[string]any, doc *spdx.Document) source.Description {
src := extractSourceFromNamespace(doc.DocumentNamespace)
rootPackages := findRootPackages(doc)
if len(rootPackages) != 1 {
return src
}
p := rootPackages[0]
switch p.PrimaryPackagePurpose {
case spdxPrimaryPurposeContainer:
src = containerSource(p)
case spdxPrimaryPurposeFile:
src = fileSource(p)
default:
return src
}
spdxIDMap[string(p.PackageSPDXIdentifier)] = src
doc.Packages = removePackage(doc.Packages, p)
doc.Relationships = removeRelationships(doc.Relationships, p.PackageSPDXIdentifier)
return src
}
func containerSource(p *spdx.Package) source.Description {
id := string(p.PackageSPDXIdentifier)
container := p.PackageName
v := p.PackageVersion
if v != "" {
container += ":" + v
}
digest := ""
if len(p.PackageChecksums) > 0 {
c := p.PackageChecksums[0]
digest = fmt.Sprintf("%s:%s", fromChecksumAlgorithm(c.Algorithm), c.Value)
}
return source.Description{
ID: id,
Name: p.PackageName,
Version: p.PackageVersion,
Metadata: source.StereoscopeImageSourceMetadata{
UserInput: container,
ID: id,
Layers: nil, // TODO handle formats with nested layer packages like Tern and K8s BOM tool
ManifestDigest: digest,
},
}
}
func fileSource(p *spdx.Package) source.Description {
typeRegex := regexp.MustCompile("^DocumentRoot-([^-]+)-.*$")
typeName := typeRegex.ReplaceAllString(string(p.PackageSPDXIdentifier), "$1")
var version string
var metadata any
switch {
case typeName == prefixDirectory:
// is a Syft SBOM, explicitly a directory source
metadata, version = directorySourceMetadata(p)
case typeName == prefixFile:
// is a Syft SBOM, explicitly a file source
metadata, version = fileSourceMetadata(p)
case isDirectory(p.PackageName):
// is a non-Syft SBOM, which looks like a directory
metadata, version = directorySourceMetadata(p)
default:
// is a non-Syft SBOM, which is probably a file
metadata, version = fileSourceMetadata(p)
}
return source.Description{
ID: string(p.PackageSPDXIdentifier),
Name: p.PackageName,
Version: version,
Metadata: metadata,
}
}
func fileSourceMetadata(p *spdx.Package) (any, string) {
version := p.PackageVersion
m := source.FileSourceMetadata{
Path: p.PackageName,
}
// if this is a Syft SBOM, we might have output a digest as the version
checksum := toChecksum(p.PackageVersion)
for _, d := range p.PackageChecksums {
if checksum != nil && checksum.Value == d.Value {
version = ""
}
m.Digests = append(m.Digests, file.Digest{
Algorithm: fromChecksumAlgorithm(d.Algorithm),
Value: d.Value,
})
}
return m, version
}
func directorySourceMetadata(p *spdx.Package) (any, string) {
return source.DirectorySourceMetadata{
Path: p.PackageName,
Base: "",
}, p.PackageVersion
}
// NOTE(jonas): SPDX doesn't inform what an SBOM is about,
// image, directory, for example. This is our best effort to determine
// the scheme. Syft-generated SBOMs have in the namespace
@@ -114,15 +277,25 @@ func findLinuxReleaseByPURL(doc *spdx.Document) *linux.Release {
return nil
}
func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document) {
for _, p := range doc.Packages {
func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]any, packages []*spdx.Package) {
for _, p := range packages {
syftPkg := toSyftPackage(p)
spdxIDMap[string(p.PackageSPDXIdentifier)] = syftPkg
s.Artifacts.Packages.Add(*syftPkg)
s.Artifacts.Packages.Add(syftPkg)
}
}
func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document) {
func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]any, doc *spdx.Document) {
for _, p := range doc.Packages {
for _, f := range p.Files {
l := toSyftLocation(f)
spdxIDMap[string(f.FileSPDXIdentifier)] = l
s.Artifacts.FileMetadata[l.Coordinates] = toFileMetadata(f)
s.Artifacts.FileDigests[l.Coordinates] = toFileDigests(f)
}
}
for _, f := range doc.Files {
l := toSyftLocation(f)
spdxIDMap[string(f.FileSPDXIdentifier)] = l
@@ -135,13 +308,17 @@ func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.
func toFileDigests(f *spdx.File) (digests []file.Digest) {
for _, digest := range f.Checksums {
digests = append(digests, file.Digest{
Algorithm: string(digest.Algorithm),
Algorithm: fromChecksumAlgorithm(digest.Algorithm),
Value: digest.Value,
})
}
return digests
}
func fromChecksumAlgorithm(algorithm common.ChecksumAlgorithm) string {
return strings.ToLower(string(algorithm))
}
func toFileMetadata(f *spdx.File) (meta file.Metadata) {
// FIXME Syft is currently lossy due to the SPDX 2.2.1 spec not supporting arbitrary mimetypes
for _, typ := range f.FileTypes {
@@ -164,21 +341,28 @@ func toFileMetadata(f *spdx.File) (meta file.Metadata) {
return meta
}
func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document) []artifact.Relationship {
var out []artifact.Relationship
func toSyftRelationships(spdxIDMap map[string]any, doc *spdx.Document) []artifact.Relationship {
out := collectDocRelationships(spdxIDMap, doc)
out = append(out, collectPackageFileRelationships(spdxIDMap, doc)...)
return out
}
func collectDocRelationships(spdxIDMap map[string]any, doc *spdx.Document) (out []artifact.Relationship) {
for _, r := range doc.Relationships {
// FIXME what to do with r.RefA.DocumentRefID and r.RefA.SpecialID
// FIXME what to do with r.RefA.DocumentRefID and r.RefA.SpecialID
if r.RefA.DocumentRefID != "" && requireAndTrimPrefix(r.RefA.DocumentRefID, "DocumentRef-") != string(doc.SPDXIdentifier) {
log.Debugf("ignoring relationship to external document: %+v", r)
continue
}
a := spdxIDMap[string(r.RefA.ElementRefID)]
b := spdxIDMap[string(r.RefB.ElementRefID)]
from, fromOk := a.(*pkg.Package)
toPackage, toPackageOk := b.(*pkg.Package)
toLocation, toLocationOk := b.(*file.Location)
from, fromOk := a.(pkg.Package)
toPackage, toPackageOk := b.(pkg.Package)
toLocation, toLocationOk := b.(file.Location)
if !fromOk || !(toPackageOk || toLocationOk) {
log.Debugf("unable to find valid relationship mapping from SPDX 2.2 JSON, ignoring: (from: %+v) (to: %+v)", a, b)
log.Debugf("unable to find valid relationship mapping from SPDX, ignoring: (from: %+v) (to: %+v)", a, b)
continue
}
var to artifact.Identifiable
@@ -219,6 +403,30 @@ func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document) [
return out
}
// collectPackageFileRelationships add relationships for direct files
func collectPackageFileRelationships(spdxIDMap map[string]any, doc *spdx.Document) (out []artifact.Relationship) {
for _, p := range doc.Packages {
a := spdxIDMap[string(p.PackageSPDXIdentifier)]
from, fromOk := a.(pkg.Package)
if !fromOk {
continue
}
for _, f := range p.Files {
b := spdxIDMap[string(f.FileSPDXIdentifier)]
to, toLocationOk := b.(file.Location)
if !toLocationOk {
continue
}
out = append(out, artifact.Relationship{
From: from,
To: to,
Type: artifact.ContainsRelationship,
})
}
}
return out
}
func toSyftCoordinates(f *spdx.File) file.Coordinates {
const layerIDPrefix = "layerID: "
var fileSystemID string
@@ -234,9 +442,9 @@ func toSyftCoordinates(f *spdx.File) file.Coordinates {
}
}
func toSyftLocation(f *spdx.File) *file.Location {
func toSyftLocation(f *spdx.File) file.Location {
l := file.NewVirtualLocationFromCoordinates(toSyftCoordinates(f), f.FileName)
return &l
return l
}
func requireAndTrimPrefix(val interface{}, prefix string) string {
@@ -280,16 +488,16 @@ func extractPkgInfo(p *spdx.Package) pkgInfo {
}
}
func toSyftPackage(p *spdx.Package) *pkg.Package {
func toSyftPackage(p *spdx.Package) pkg.Package {
info := extractPkgInfo(p)
metadataType, metadata := extractMetadata(p, info)
sP := pkg.Package{
sP := &pkg.Package{
Type: info.typ,
Name: p.PackageName,
Version: p.PackageVersion,
Licenses: pkg.NewLicenseSet(parseSPDXLicenses(p)...),
CPEs: extractCPEs(p),
PURL: info.purl.String(),
PURL: purlValue(info.purl),
Language: info.lang,
MetadataType: metadataType,
Metadata: metadata,
@@ -297,7 +505,15 @@ func toSyftPackage(p *spdx.Package) *pkg.Package {
sP.SetID()
return &sP
return *sP
}
func purlValue(purl packageurl.PackageURL) string {
val := purl.String()
if _, err := packageurl.FromString(val); err != nil {
return ""
}
return val
}
func parseSPDXLicenses(p *spdx.Package) []pkg.License {
@@ -321,10 +537,7 @@ func parseSPDXLicenses(p *spdx.Package) []pkg.License {
}
func cleanSPDXID(id string) string {
if strings.HasPrefix(id, "LicenseRef-") {
return strings.TrimPrefix(id, "LicenseRef-")
}
return id
return strings.TrimPrefix(id, spdxlicense.LicenseRefPrefix)
}
//nolint:funlen
@@ -384,7 +597,7 @@ func extractMetadata(p *spdx.Package, info pkgInfo) (pkg.MetadataType, interface
case pkg.JavaPkg:
var digests []file.Digest
for _, value := range p.PackageChecksums {
digests = append(digests, file.Digest{Algorithm: string(value.Algorithm), Value: value.Value})
digests = append(digests, file.Digest{Algorithm: fromChecksumAlgorithm(value.Algorithm), Value: value.Value})
}
return pkg.JavaMetadataType, pkg.JavaMetadata{
ArchiveDigests: digests,
@@ -392,7 +605,7 @@ func extractMetadata(p *spdx.Package, info pkgInfo) (pkg.MetadataType, interface
case pkg.GoModulePkg:
var h1Digest string
for _, value := range p.PackageChecksums {
digest, err := util.HDigestFromSHA(string(value.Algorithm), value.Value)
digest, err := util.HDigestFromSHA(fromChecksumAlgorithm(value.Algorithm), value.Value)
if err != nil {
log.Debugf("invalid h1digest: %v %v", value, err)
continue
@@ -4,14 +4,18 @@ import (
"reflect"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/spdx/v2/common"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
@@ -324,7 +328,7 @@ func TestH1Digest(t *testing.T) {
func Test_toSyftRelationships(t *testing.T) {
type args struct {
spdxIDMap map[string]interface{}
spdxIDMap map[string]any
doc *spdx.Document
}
@@ -361,9 +365,9 @@ func Test_toSyftRelationships(t *testing.T) {
{
name: "evident-by relationship",
args: args{
spdxIDMap: map[string]interface{}{
string(toSPDXID(pkg1)): &pkg1,
string(toSPDXID(loc1)): &loc1,
spdxIDMap: map[string]any{
string(toSPDXID(pkg1)): pkg1,
string(toSPDXID(loc1)): loc1,
},
doc: &spdx.Document{
Relationships: []*spdx.Relationship{
@@ -391,9 +395,9 @@ func Test_toSyftRelationships(t *testing.T) {
{
name: "ownership-by-file-overlap relationship",
args: args{
spdxIDMap: map[string]interface{}{
string(toSPDXID(pkg2)): &pkg2,
string(toSPDXID(pkg3)): &pkg3,
spdxIDMap: map[string]any{
string(toSPDXID(pkg2)): pkg2,
string(toSPDXID(pkg3)): pkg3,
},
doc: &spdx.Document{
Relationships: []*spdx.Relationship{
@@ -431,3 +435,250 @@ func Test_toSyftRelationships(t *testing.T) {
})
}
}
func Test_convertToAndFromFormat(t *testing.T) {
packages := []pkg.Package{
{
Name: "pkg1",
MetadataType: pkg.UnknownMetadataType,
},
{
Name: "pkg2",
MetadataType: pkg.UnknownMetadataType,
},
}
for i := range packages {
(&packages[i]).SetID()
}
relationships := []artifact.Relationship{
{
From: packages[0],
To: packages[1],
Type: artifact.ContainsRelationship,
},
}
tests := []struct {
name string
source source.Description
packages []pkg.Package
relationships []artifact.Relationship
}{
{
name: "image source",
source: source.Description{
ID: "DocumentRoot-Image-some-image",
Metadata: source.StereoscopeImageSourceMetadata{
ID: "DocumentRoot-Image-some-image",
UserInput: "some-image:some-tag",
ManifestDigest: "sha256:ab8b83234bc28f28d8e",
},
Name: "some-image",
Version: "some-tag",
},
packages: packages,
relationships: relationships,
},
{
name: ". directory source",
source: source.Description{
ID: "DocumentRoot-Directory-.",
Name: ".",
Metadata: source.DirectorySourceMetadata{
Path: ".",
},
},
packages: packages,
relationships: relationships,
},
{
name: "directory source",
source: source.Description{
ID: "DocumentRoot-Directory-my-app",
Name: "my-app",
Metadata: source.DirectorySourceMetadata{
Path: "my-app",
},
},
packages: packages,
relationships: relationships,
},
{
name: "file source",
source: source.Description{
ID: "DocumentRoot-File-my-app.exe",
Metadata: source.FileSourceMetadata{
Path: "my-app.exe",
Digests: []file.Digest{
{
Algorithm: "sha256",
Value: "3723cae0b8b83234bc28f28d8e",
},
},
},
Name: "my-app.exe",
},
packages: packages,
relationships: relationships,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
src := &test.source
s := sbom.SBOM{
Source: *src,
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(test.packages...),
},
Relationships: test.relationships,
}
doc := ToFormatModel(s)
got, err := ToSyftModel(doc)
require.NoError(t, err)
if diff := cmp.Diff(&s, got,
cmpopts.IgnoreUnexported(artifact.Relationship{}),
cmpopts.IgnoreUnexported(file.LocationSet{}),
cmpopts.IgnoreUnexported(pkg.Collection{}),
cmpopts.IgnoreUnexported(pkg.Package{}),
cmpopts.IgnoreUnexported(pkg.LicenseSet{}),
cmpopts.IgnoreFields(pkg.Package{}, "MetadataType"),
cmpopts.IgnoreFields(sbom.Artifacts{}, "FileMetadata", "FileDigests"),
); diff != "" {
t.Fatalf("packages do not match:\n%s", diff)
}
})
}
}
func Test_purlValue(t *testing.T) {
tests := []struct {
purl packageurl.PackageURL
expected string
}{
{
purl: packageurl.PackageURL{},
expected: "",
},
{
purl: packageurl.PackageURL{
Name: "name",
Version: "version",
},
expected: "",
},
{
purl: packageurl.PackageURL{
Type: "typ",
Version: "version",
},
expected: "",
},
{
purl: packageurl.PackageURL{
Type: "typ",
Name: "name",
Version: "version",
},
expected: "pkg:typ/name@version",
},
{
purl: packageurl.PackageURL{
Type: "typ",
Name: "name",
Version: "version",
Qualifiers: packageurl.Qualifiers{
{
Key: "q",
Value: "v",
},
},
},
expected: "pkg:typ/name@version?q=v",
},
}
for _, test := range tests {
t.Run(test.purl.String(), func(t *testing.T) {
got := purlValue(test.purl)
require.Equal(t, test.expected, got)
})
}
}
func Test_directPackageFiles(t *testing.T) {
doc := &spdx.Document{
SPDXVersion: "SPDX-2.3",
Packages: []*spdx.Package{
{
PackageName: "some-package",
PackageSPDXIdentifier: "1",
PackageVersion: "1.0.5",
Files: []*spdx.File{
{
FileName: "some-file",
FileSPDXIdentifier: "2",
Checksums: []spdx.Checksum{
{
Algorithm: "SHA1",
Value: "a8d733c64f9123",
},
},
},
},
},
},
}
got, err := ToSyftModel(doc)
require.NoError(t, err)
p := pkg.Package{
Name: "some-package",
Version: "1.0.5",
MetadataType: pkg.UnknownMetadataType,
}
p.SetID()
f := file.Location{
LocationData: file.LocationData{
Coordinates: file.Coordinates{
RealPath: "some-file",
FileSystemID: "",
},
VirtualPath: "some-file",
},
LocationMetadata: file.LocationMetadata{
Annotations: map[string]string{},
},
}
s := &sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(p),
FileMetadata: map[file.Coordinates]file.Metadata{
f.Coordinates: {},
},
FileDigests: map[file.Coordinates][]file.Digest{
f.Coordinates: {
{
Algorithm: "sha1",
Value: "a8d733c64f9123",
},
},
},
},
Relationships: []artifact.Relationship{
{
From: p,
To: f,
Type: artifact.ContainsRelationship,
},
},
Source: source.Description{},
Descriptor: sbom.Descriptor{},
}
require.Equal(t, s, got)
}
+1 -1
View File
@@ -92,7 +92,7 @@ func TestSPDXJSONDecoder(t *testing.T) {
relationships:
for _, pkgName := range test.relationships {
for _, rel := range sbom.Relationships {
p, ok := rel.From.(*pkg.Package)
p, ok := rel.From.(pkg.Package)
if ok && p.Name == pkgName {
continue relationships
}
@@ -17,6 +17,7 @@
"name": "package-1",
"SPDXID": "SPDXRef-Package-python-package-1-9265397e5e15168a",
"versionInfo": "1.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"sourceInfo": "acquired package info from installed python package manifest file: /some/path/pkg1",
@@ -40,6 +41,7 @@
"name": "package-2",
"SPDXID": "SPDXRef-Package-deb-package-2-db4abfe497c180d3",
"versionInfo": "2.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"sourceInfo": "acquired package info from DPKG DB: /some/path/pkg1",
@@ -58,12 +60,30 @@
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
},
{
"name": "some/path",
"SPDXID": "SPDXRef-DocumentRoot-Directory-some-path",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"primaryPackagePurpose": "FILE"
}
],
"relationships": [
{
"spdxElementId": "SPDXRef-DocumentRoot-Directory-some-path",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-9265397e5e15168a",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Directory-some-path",
"relatedSpdxElement": "SPDXRef-Package-deb-package-2-db4abfe497c180d3",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DocumentRoot-Directory-some-path",
"relationshipType": "DESCRIBES"
}
]
@@ -17,6 +17,7 @@
"name": "package-1",
"SPDXID": "SPDXRef-Package-python-package-1-125840abc1c66dd7",
"versionInfo": "1.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"sourceInfo": "acquired package info from installed python package manifest file: /somefile-1.txt",
@@ -40,6 +41,7 @@
"name": "package-2",
"SPDXID": "SPDXRef-Package-deb-package-2-958443e2d9304af4",
"versionInfo": "2.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"sourceInfo": "acquired package info from DPKG DB: /somefile-2.txt",
@@ -58,12 +60,44 @@
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
},
{
"name": "user-image-input",
"SPDXID": "SPDXRef-DocumentRoot-Image-user-image-input",
"versionInfo": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"checksums": [
{
"algorithm": "SHA256",
"checksumValue": "2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
}
],
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch="
}
],
"primaryPackagePurpose": "CONTAINER"
}
],
"relationships": [
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-125840abc1c66dd7",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-deb-package-2-958443e2d9304af4",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DocumentRoot-Image-user-image-input",
"relationshipType": "DESCRIBES"
}
]
@@ -17,6 +17,7 @@
"name": "package-1",
"SPDXID": "SPDXRef-Package-python-package-1-125840abc1c66dd7",
"versionInfo": "1.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"sourceInfo": "acquired package info from installed python package manifest file: /somefile-1.txt",
@@ -40,6 +41,7 @@
"name": "package-2",
"SPDXID": "SPDXRef-Package-deb-package-2-958443e2d9304af4",
"versionInfo": "2.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"sourceInfo": "acquired package info from DPKG DB: /somefile-2.txt",
@@ -58,6 +60,28 @@
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
},
{
"name": "user-image-input",
"SPDXID": "SPDXRef-DocumentRoot-Image-user-image-input",
"versionInfo": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"checksums": [
{
"algorithm": "SHA256",
"checksumValue": "2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
}
],
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch="
}
],
"primaryPackagePurpose": "CONTAINER"
}
],
"files": [
@@ -183,9 +207,19 @@
"relatedSpdxElement": "SPDXRef-File-f2-f9e49132a4b96ccd",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-125840abc1c66dd7",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-deb-package-2-958443e2d9304af4",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DocumentRoot-Image-user-image-input",
"relationshipType": "DESCRIBES"
}
]
+60
View File
@@ -2,9 +2,13 @@ package spdxtagvalue
import (
"os"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/file"
)
// TODO: this is a temporary coverage see below
@@ -34,3 +38,59 @@ func TestSPDXTagValueDecoder(t *testing.T) {
})
}
}
func Test_packageDirectFiles(t *testing.T) {
contents := `
SPDXVersion: SPDX-2.2
DataLicense: CC0-1.0
SPDXID: SPDXRef-DOCUMENT
DocumentName: Some-SBOM
DocumentNamespace: https://example.org/some/namespace
Creator: Organization: Some-organization
Creator: Tool: Some-tool Version: 1.0
Created: 2021-12-29T17:02:21Z
PackageName: Some-package
PackageVersion: 5.1.2
SPDXID: SPDXRef-Package-43c51b08-cc7e-406d-8ad9-34aa292d1157
PackageSupplier: Organization: Some-organization
PackageDownloadLocation: https://example.org/download/location
FilesAnalyzed: true
PackageLicenseInfoFromFiles: NOASSERTION
PackageVerificationCode: 23460C5559C8D4DE3F6504E0E84E844CAC8B1D95
PackageLicenseConcluded: NOASSERTION
PackageLicenseDeclared: NOASSERTION
PackageCopyrightText: NOASSERTION
PackageChecksum: SHA1: 23460C5559C8D4DE3F6504E0E84E844CAC8B1D95
FileName: Some-file-name
SPDXID: SPDXRef-99545d55-933d-4e08-9eb5-9d826111cb79
FileContributor: Some-file-contributor
FileType: BINARY
FileChecksum: SHA1: 23460C5559C8D4DE3F6504E0E84E844CAC8B1D95
LicenseConcluded: NOASSERTION
LicenseInfoInFile: NOASSERTION
FileCopyrightText: NOASSERTION
`
s, err := decoder(strings.NewReader(contents))
require.NoError(t, err)
pkgs := s.Artifacts.Packages.Sorted()
assert.Len(t, pkgs, 1)
assert.Len(t, s.Artifacts.FileMetadata, 1)
assert.Len(t, s.Relationships, 1)
p := pkgs[0]
r := s.Relationships[0]
f := file.Location{}
for c := range s.Artifacts.FileMetadata {
f = file.Location{
LocationData: file.LocationData{
Coordinates: c,
VirtualPath: "",
},
LocationMetadata: file.LocationMetadata{},
}
break // there should only be 1
}
assert.Equal(t, p.ID(), r.From.ID())
assert.Equal(t, f.ID(), r.To.ID())
}
+2 -1
View File
@@ -61,7 +61,8 @@ func TestSPDXJSONSPDXIDs(t *testing.T) {
},
Relationships: nil,
Source: source.Description{
Metadata: source.DirectorySourceMetadata{Path: "foobar/baz"}, // in this case, foobar is used as the spdx docment name
Name: "foobar/baz", // in this case, foobar is used as the spdx document name
Metadata: source.DirectorySourceMetadata{},
},
Descriptor: sbom.Descriptor{
Name: "syft",
@@ -8,10 +8,20 @@ Creator: Organization: Anchore, Inc
Creator: Tool: syft-v0.42.0-bogus
Created: redacted
##### Package: foobar/baz
PackageName: foobar/baz
SPDXID: SPDXRef-DocumentRoot-Directory-foobar-baz
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
PrimaryPackagePurpose: FILE
FilesAnalyzed: false
##### Package: @at-sign
PackageName: @at-sign
SPDXID: SPDXRef-Package---at-sign-3732f7a5679bdec4
SPDXID: SPDXRef-Package--at-sign-3732f7a5679bdec4
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from the following paths:
@@ -22,7 +32,8 @@ PackageCopyrightText: NOASSERTION
##### Package: some/slashes
PackageName: some/slashes
SPDXID: SPDXRef-Package--some-slashes-1345166d4801153b
SPDXID: SPDXRef-Package-some-slashes-1345166d4801153b
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from the following paths:
@@ -33,7 +44,8 @@ PackageCopyrightText: NOASSERTION
##### Package: under_scores
PackageName: under_scores
SPDXID: SPDXRef-Package--under-scores-290d5c77210978c1
SPDXID: SPDXRef-Package-under-scores-290d5c77210978c1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from the following paths:
@@ -43,5 +55,8 @@ PackageCopyrightText: NOASSERTION
##### Relationships
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DOCUMENT
Relationship: SPDXRef-DocumentRoot-Directory-foobar-baz CONTAINS SPDXRef-Package--at-sign-3732f7a5679bdec4
Relationship: SPDXRef-DocumentRoot-Directory-foobar-baz CONTAINS SPDXRef-Package-some-slashes-1345166d4801153b
Relationship: SPDXRef-DocumentRoot-Directory-foobar-baz CONTAINS SPDXRef-Package-under-scores-290d5c77210978c1
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Directory-foobar-baz
@@ -46,11 +46,24 @@ FileType: OTHER
FileChecksum: SHA1: 0000000000000000000000000000000000000000
LicenseConcluded: NOASSERTION
##### Package: user-image-input
PackageName: user-image-input
SPDXID: SPDXRef-DocumentRoot-Image-user-image-input
PackageVersion: sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
PrimaryPackagePurpose: CONTAINER
FilesAnalyzed: false
PackageChecksum: SHA256: 2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368
ExternalRef: PACKAGE-MANAGER purl pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch=
##### Package: package-2
PackageName: package-2
SPDXID: SPDXRef-Package-deb-package-2-958443e2d9304af4
PackageVersion: 2.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from DPKG DB: /somefile-2.txt
@@ -65,6 +78,7 @@ ExternalRef: PACKAGE-MANAGER purl pkg:deb/debian/package-2@2.0.1
PackageName: package-1
SPDXID: SPDXRef-Package-python-package-1-125840abc1c66dd7
PackageVersion: 1.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from installed python package manifest file: /somefile-1.txt
@@ -82,5 +96,7 @@ Relationship: SPDXRef-Package-python-package-1-125840abc1c66dd7 CONTAINS SPDXRef
Relationship: SPDXRef-Package-python-package-1-125840abc1c66dd7 CONTAINS SPDXRef-File-d2-f4-c641caa71518099f
Relationship: SPDXRef-Package-python-package-1-125840abc1c66dd7 CONTAINS SPDXRef-File-d1-f3-c6f5b29dca12661f
Relationship: SPDXRef-Package-python-package-1-125840abc1c66dd7 CONTAINS SPDXRef-File-f2-f9e49132a4b96ccd
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DOCUMENT
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-python-package-1-125840abc1c66dd7
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-deb-package-2-958443e2d9304af4
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Image-user-image-input
@@ -8,11 +8,21 @@ Creator: Organization: Anchore, Inc
Creator: Tool: syft-v0.42.0-bogus
Created: redacted
##### Package: some/path
PackageName: some/path
SPDXID: SPDXRef-DocumentRoot-Directory-some-path
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
PrimaryPackagePurpose: FILE
FilesAnalyzed: false
##### Package: package-2
PackageName: package-2
SPDXID: SPDXRef-Package-deb-package-2-db4abfe497c180d3
PackageVersion: 2.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from DPKG DB: /some/path/pkg1
@@ -27,6 +37,7 @@ ExternalRef: PACKAGE-MANAGER purl pkg:deb/debian/package-2@2.0.1
PackageName: package-1
SPDXID: SPDXRef-Package-python-package-1-9265397e5e15168a
PackageVersion: 1.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from installed python package manifest file: /some/path/pkg1
@@ -38,5 +49,7 @@ ExternalRef: PACKAGE-MANAGER purl a-purl-2
##### Relationships
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DOCUMENT
Relationship: SPDXRef-DocumentRoot-Directory-some-path CONTAINS SPDXRef-Package-python-package-1-9265397e5e15168a
Relationship: SPDXRef-DocumentRoot-Directory-some-path CONTAINS SPDXRef-Package-deb-package-2-db4abfe497c180d3
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Directory-some-path
@@ -8,11 +8,24 @@ Creator: Organization: Anchore, Inc
Creator: Tool: syft-v0.42.0-bogus
Created: redacted
##### Package: user-image-input
PackageName: user-image-input
SPDXID: SPDXRef-DocumentRoot-Image-user-image-input
PackageVersion: sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
PrimaryPackagePurpose: CONTAINER
FilesAnalyzed: false
PackageChecksum: SHA256: 2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368
ExternalRef: PACKAGE-MANAGER purl pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch=
##### Package: package-2
PackageName: package-2
SPDXID: SPDXRef-Package-deb-package-2-958443e2d9304af4
PackageVersion: 2.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from DPKG DB: /somefile-2.txt
@@ -27,6 +40,7 @@ ExternalRef: PACKAGE-MANAGER purl pkg:deb/debian/package-2@2.0.1
PackageName: package-1
SPDXID: SPDXRef-Package-python-package-1-125840abc1c66dd7
PackageVersion: 1.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from installed python package manifest file: /somefile-1.txt
@@ -38,5 +52,7 @@ ExternalRef: PACKAGE-MANAGER purl a-purl-1
##### Relationships
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DOCUMENT
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-python-package-1-125840abc1c66dd7
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-deb-package-2-958443e2d9304af4
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Image-user-image-input
+1 -1
View File
@@ -6,5 +6,5 @@ import "github.com/anchore/syft/syft/pkg"
// AllTypes returns a list of all pkg metadata types that syft supports (that are represented in the pkg.Package.Metadata field).
func AllTypes() []any {
return []any{pkg.AlpmMetadata{}, pkg.ApkMetadata{}, pkg.BinaryMetadata{}, pkg.CargoPackageMetadata{}, pkg.CocoapodsMetadata{}, pkg.ConanLockMetadata{}, pkg.ConanMetadata{}, pkg.DartPubMetadata{}, pkg.DotnetDepsMetadata{}, pkg.DpkgMetadata{}, pkg.GemMetadata{}, pkg.GolangBinMetadata{}, pkg.GolangModMetadata{}, pkg.HackageMetadata{}, pkg.JavaMetadata{}, pkg.KbPackageMetadata{}, pkg.LinuxKernelMetadata{}, pkg.LinuxKernelModuleMetadata{}, pkg.MixLockMetadata{}, pkg.NixStoreMetadata{}, pkg.NpmPackageJSONMetadata{}, pkg.NpmPackageLockJSONMetadata{}, pkg.PhpComposerJSONMetadata{}, pkg.PortageMetadata{}, pkg.PythonPackageMetadata{}, pkg.PythonPipfileLockMetadata{}, pkg.PythonRequirementsMetadata{}, pkg.RDescriptionFileMetadata{}, pkg.RebarLockMetadata{}, pkg.RpmMetadata{}}
return []any{pkg.AlpmMetadata{}, pkg.ApkMetadata{}, pkg.BinaryMetadata{}, pkg.CargoPackageMetadata{}, pkg.CocoapodsMetadata{}, pkg.ConanLockMetadata{}, pkg.ConanMetadata{}, pkg.DartPubMetadata{}, pkg.DotnetDepsMetadata{}, pkg.DotnetPortableExecutableMetadata{}, pkg.DpkgMetadata{}, pkg.GemMetadata{}, pkg.GolangBinMetadata{}, pkg.GolangModMetadata{}, pkg.HackageMetadata{}, pkg.JavaMetadata{}, pkg.KbPackageMetadata{}, pkg.LinuxKernelMetadata{}, pkg.LinuxKernelModuleMetadata{}, pkg.MixLockMetadata{}, pkg.NixStoreMetadata{}, pkg.NpmPackageJSONMetadata{}, pkg.NpmPackageLockJSONMetadata{}, pkg.PhpComposerJSONMetadata{}, pkg.PortageMetadata{}, pkg.PythonPackageMetadata{}, pkg.PythonPipfileLockMetadata{}, pkg.PythonRequirementsMetadata{}, pkg.RDescriptionFileMetadata{}, pkg.RebarLockMetadata{}, pkg.RpmMetadata{}, pkg.SwiftPackageManagerMetadata{}}
}
+23 -1
View File
@@ -74,11 +74,33 @@ func CatalogPackages(src source.Source, cfg cataloger.Config) (*pkg.Collection,
catalog, relationships, err := cataloger.Catalog(resolver, release, cfg.Parallelism, catalogers...)
relationships = append(relationships, newSourceRelationshipsFromCatalog(src, catalog)...)
// apply exclusions to the package catalog
// default config value for this is true
// https://github.com/anchore/syft/issues/931
if cfg.ExcludeBinaryOverlapByOwnership {
for _, r := range relationships {
if cataloger.ExcludeBinaryByFileOwnershipOverlap(r, catalog) {
catalog.Delete(r.To.ID())
relationships = removeRelationshipsByID(relationships, r.To.ID())
}
}
}
// no need to consider source relationships for os -> binary exclusions
relationships = append(relationships, newSourceRelationshipsFromCatalog(src, catalog)...)
return catalog, relationships, release, err
}
func removeRelationshipsByID(relationships []artifact.Relationship, id artifact.ID) []artifact.Relationship {
var filtered []artifact.Relationship
for _, r := range relationships {
if r.To.ID() != id && r.From.ID() != id {
filtered = append(filtered, r)
}
}
return filtered
}
func newSourceRelationshipsFromCatalog(src source.Source, c *pkg.Collection) []artifact.Relationship {
relationships := make([]artifact.Relationship, 0) // Should we pre-allocate this by giving catalog a Len() method?
for p := range c.Enumerate() {
+42
View File
@@ -0,0 +1,42 @@
package syft
import (
"testing"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
)
func Test_removeRelationshipsByID(t *testing.T) {
p1 := pkg.Package{}
p1.OverrideID("1")
p2 := pkg.Package{}
p2.OverrideID("2")
p3 := pkg.Package{}
p3.OverrideID("3")
rel := func(pkgs ...pkg.Package) (out []artifact.Relationship) {
for _, p := range pkgs {
out = append(out, artifact.Relationship{
From: p,
To: p,
Type: artifact.OwnershipByFileOverlapRelationship,
})
}
return
}
relationships := rel(p1, p2, p3)
for _, r := range relationships {
if r.From.ID() == "1" || r.From.ID() == "2" {
relationships = removeRelationshipsByID(relationships, r.From.ID())
}
}
require.Equal(t, rel(p3), relationships)
}
@@ -643,6 +643,30 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Metadata: metadata("consul-binary"),
},
},
{
name: "positive-nginx-1.25.1",
fixtureDir: "test-fixtures/classifiers/positive/nginx-1.25.1",
expected: pkg.Package{
Name: "nginx",
Version: "1.25.1",
Type: "binary",
PURL: "pkg:generic/nginx@1.25.1",
Locations: locations("nginx"),
Metadata: metadata("nginx-binary"),
},
},
{
name: "positive-nginx-openresty-1.21.4.2",
fixtureDir: "test-fixtures/classifiers/positive/nginx-openresty-1.21.4.2",
expected: pkg.Package{
Name: "nginx",
Version: "1.21.4",
Type: "binary",
PURL: "pkg:generic/nginx@1.21.4",
Locations: locations("nginx"),
Metadata: metadata("nginx-binary"),
},
},
}
for _, test := range tests {
@@ -254,6 +254,21 @@ var defaultClassifiers = []classifier{
PURL: mustPURL("pkg:golang/github.com/hashicorp/consul@version"),
CPEs: singleCPE("cpe:2.3:a:hashicorp:consul:*:*:*:*:*:*:*:*"),
},
{
Class: "nginx-binary",
FileGlob: "**/nginx",
EvidenceMatcher: fileContentsVersionMatcher(
// [NUL]nginx version: nginx/1.25.1 - fetches '1.25.1'
// [NUL]nginx version: openresty/1.21.4.1 - fetches '1.21.4' as this is the nginx version part
`(?m)(\x00|\?)nginx version: [^\/]+\/(?P<version>[0-9]+\.[0-9]+\.[0-9]+(?:\+\d+)?(?:-\d+)?)`,
),
Package: "nginx",
PURL: mustPURL("pkg:generic/nginx@version"),
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:a:f5:nginx:*:*:*:*:*:*:*:*"),
cpe.Must("cpe:2.3:a:nginx:nginx:*:*:*:*:*:*:*:*"),
},
},
}
// in both binaries and shared libraries, the version pattern is [NUL]3.11.2[NUL]
+8 -1
View File
@@ -76,7 +76,14 @@ func runCataloger(cataloger pkg.Cataloger, resolver file.Resolver) (catalogerRes
for _, p := range packages {
// generate CPEs (note: this is excluded from package ID, so is safe to mutate)
// we might have binary classified CPE already with the package so we want to append here
p.CPEs = append(p.CPEs, cpe.Generate(p)...)
dictionaryCPE, ok := cpe.DictionaryFind(p)
if ok {
log.Debugf("used CPE dictionary to find CPE for %s package %q: %s", p.Type, p.Name, dictionaryCPE.BindToFmtString())
p.CPEs = append(p.CPEs, dictionaryCPE)
} else {
p.CPEs = append(p.CPEs, cpe.Generate(p)...)
}
// if we were not able to identify the language we have an opportunity
// to try and get this value from the PURL. Worst case we assert that
+13 -9
View File
@@ -45,8 +45,8 @@ func ImageCatalogers(cfg Config) []pkg.Cataloger {
apkdb.NewApkdbCataloger(),
binary.NewCataloger(),
deb.NewDpkgdbCataloger(),
dotnet.NewDotnetDepsCataloger(),
golang.NewGoModuleBinaryCataloger(cfg.Go()),
dotnet.NewDotnetPortableExecutableCataloger(),
golang.NewGoModuleBinaryCataloger(cfg.Golang),
java.NewJavaCataloger(cfg.Java()),
java.NewNativeImageCataloger(),
javascript.NewPackageCataloger(),
@@ -71,10 +71,11 @@ func DirectoryCatalogers(cfg Config) []pkg.Cataloger {
dart.NewPubspecLockCataloger(),
deb.NewDpkgdbCataloger(),
dotnet.NewDotnetDepsCataloger(),
dotnet.NewDotnetPortableExecutableCataloger(),
elixir.NewMixLockCataloger(),
erlang.NewRebarLockCataloger(),
golang.NewGoModFileCataloger(cfg.Go()),
golang.NewGoModuleBinaryCataloger(cfg.Go()),
golang.NewGoModFileCataloger(cfg.Golang),
golang.NewGoModuleBinaryCataloger(cfg.Golang),
haskell.NewHackageCataloger(),
java.NewJavaCataloger(cfg.Java()),
java.NewJavaGradleLockfileCataloger(),
@@ -84,7 +85,7 @@ func DirectoryCatalogers(cfg Config) []pkg.Cataloger {
nix.NewStoreCataloger(),
php.NewComposerLockCataloger(),
portage.NewPortageCataloger(),
python.NewPythonIndexCataloger(),
python.NewPythonIndexCataloger(cfg.Python),
python.NewPythonPackageCataloger(),
rpm.NewFileCataloger(),
rpm.NewRpmDBCataloger(),
@@ -92,6 +93,7 @@ func DirectoryCatalogers(cfg Config) []pkg.Cataloger {
rust.NewCargoLockCataloger(),
sbom.NewSBOMCataloger(),
swift.NewCocoapodsCataloger(),
swift.NewSwiftPackageManagerCataloger(),
}, cfg.Catalogers)
}
@@ -105,10 +107,11 @@ func AllCatalogers(cfg Config) []pkg.Cataloger {
dart.NewPubspecLockCataloger(),
deb.NewDpkgdbCataloger(),
dotnet.NewDotnetDepsCataloger(),
dotnet.NewDotnetPortableExecutableCataloger(),
elixir.NewMixLockCataloger(),
erlang.NewRebarLockCataloger(),
golang.NewGoModFileCataloger(cfg.Go()),
golang.NewGoModuleBinaryCataloger(cfg.Go()),
golang.NewGoModFileCataloger(cfg.Golang),
golang.NewGoModuleBinaryCataloger(cfg.Golang),
haskell.NewHackageCataloger(),
java.NewJavaCataloger(cfg.Java()),
java.NewJavaGradleLockfileCataloger(),
@@ -116,12 +119,12 @@ func AllCatalogers(cfg Config) []pkg.Cataloger {
java.NewNativeImageCataloger(),
javascript.NewLockCataloger(),
javascript.NewPackageCataloger(),
kernel.NewLinuxKernelCataloger(cfg.Kernel()),
kernel.NewLinuxKernelCataloger(cfg.LinuxKernel),
nix.NewStoreCataloger(),
php.NewComposerInstalledCataloger(),
php.NewComposerLockCataloger(),
portage.NewPortageCataloger(),
python.NewPythonIndexCataloger(),
python.NewPythonIndexCataloger(cfg.Python),
python.NewPythonPackageCataloger(),
r.NewPackageCataloger(),
rpm.NewFileCataloger(),
@@ -132,6 +135,7 @@ func AllCatalogers(cfg Config) []pkg.Cataloger {
rust.NewCargoLockCataloger(),
sbom.NewSBOMCataloger(),
swift.NewCocoapodsCataloger(),
swift.NewSwiftPackageManagerCataloger(),
}, cfg.Catalogers)
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,3 @@
package dictionary
//go:generate go run ./index-generator/ -o data/cpe-index.json
@@ -0,0 +1,230 @@
package main
import (
"compress/gzip"
"encoding/json"
"encoding/xml"
"fmt"
"io"
"log"
"strings"
"github.com/facebookincubator/nvdtools/wfn"
"golang.org/x/exp/slices"
"github.com/anchore/syft/syft/pkg/cataloger/common/cpe/dictionary"
)
func generateIndexedDictionaryJSON(rawGzipData io.Reader) ([]byte, error) {
gzipReader, err := gzip.NewReader(rawGzipData)
if err != nil {
return nil, fmt.Errorf("unable to decompress CPE dictionary: %w", err)
}
defer gzipReader.Close()
// Read XML data
data, err := io.ReadAll(gzipReader)
if err != nil {
return nil, fmt.Errorf("unable to read CPE dictionary: %w", err)
}
// Unmarshal XML
var cpeList CpeList
if err := xml.Unmarshal(data, &cpeList); err != nil {
return nil, fmt.Errorf("unable to unmarshal CPE dictionary XML: %w", err)
}
// Filter out data that's not applicable here
cpeList = filterCpeList(cpeList)
// Create indexed dictionary to help with looking up CPEs
indexedDictionary := indexCPEList(cpeList)
// Convert to JSON
jsonData, err := json.MarshalIndent(indexedDictionary, "", " ")
if err != nil {
return nil, fmt.Errorf("unable to marshal CPE dictionary to JSON: %w", err)
}
return jsonData, nil
}
// filterCpeList removes CPE items that are not applicable to software packages.
func filterCpeList(cpeList CpeList) CpeList {
var processedCpeList CpeList
seen := make(map[string]struct{})
for _, cpeItem := range cpeList.CpeItems {
// Skip CPE items that don't have any references.
if len(cpeItem.References) == 0 {
continue
}
// Skip CPE items where the CPE URI doesn't meet our criteria.
parsedName, err := wfn.Parse(cpeItem.Name)
if err != nil {
log.Printf("unable to parse CPE URI %q: %s", cpeItem.Name, err)
}
if slices.Contains([]string{"h", "o"}, parsedName.Part) {
continue
}
normalizedName := normalizeCPE(parsedName).BindToURI()
if _, ok := seen[normalizedName]; ok {
continue
}
seen[normalizedName] = struct{}{}
cpeItem.Name = normalizedName
parsedCPE, err := wfn.Parse(cpeItem.Cpe23Item.Name)
if err != nil {
log.Printf("unable to parse CPE value %q: %s", cpeItem.Cpe23Item.Name, err)
}
cpeItem.Cpe23Item.Name = normalizeCPE(parsedCPE).BindToFmtString()
processedCpeList.CpeItems = append(processedCpeList.CpeItems, cpeItem)
}
return processedCpeList
}
// normalizeCPE removes the version and update parts of a CPE.
func normalizeCPE(cpe *wfn.Attributes) *wfn.Attributes {
cpeCopy := *cpe
cpeCopy.Version = ""
cpeCopy.Update = ""
return &cpeCopy
}
const (
prefixForNPMPackages = "https://www.npmjs.com/package/"
prefixForRubyGems = "https://rubygems.org/gems/"
prefixForRubyGemsHTTP = "http://rubygems.org/gems/"
prefixForNativeRubyGems = "https://github.com/ruby/"
prefixForPyPIPackages = "https://pypi.org/project/"
prefixForJenkinsPlugins = "https://github.com/jenkinsci/"
prefixForRustCrates = "https://crates.io/crates/"
)
// indexCPEList creates an index of CPEs by ecosystem.
func indexCPEList(list CpeList) *dictionary.Indexed {
indexed := &dictionary.Indexed{
EcosystemPackages: make(map[string]dictionary.Packages),
}
for _, cpeItem := range list.CpeItems {
cpeItemName := cpeItem.Cpe23Item.Name
for _, reference := range cpeItem.References {
ref := reference.Reference.Href
switch {
case strings.HasPrefix(ref, prefixForNPMPackages):
addEntryForNPMPackage(indexed, ref, cpeItemName)
case strings.HasPrefix(ref, prefixForRubyGems), strings.HasPrefix(ref, prefixForRubyGemsHTTP):
addEntryForRubyGem(indexed, ref, cpeItemName)
case strings.HasPrefix(ref, prefixForNativeRubyGems):
addEntryForNativeRubyGem(indexed, ref, cpeItemName)
case strings.HasPrefix(ref, prefixForPyPIPackages):
addEntryForPyPIPackage(indexed, ref, cpeItemName)
case strings.HasPrefix(ref, prefixForJenkinsPlugins):
// It _might_ be a jenkins plugin!
addEntryForJenkinsPlugin(indexed, ref, cpeItemName)
case strings.HasPrefix(ref, prefixForRustCrates):
addEntryForRustCrate(indexed, ref, cpeItemName)
}
}
}
return indexed
}
func addEntryForRustCrate(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.TrimPrefix(ref, prefixForRustCrates)
ref = strings.Split(ref, "/")[0]
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemRustCrates]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemRustCrates] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemRustCrates][ref] = cpeItemName
}
func addEntryForJenkinsPlugin(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.TrimPrefix(ref, prefixForJenkinsPlugins)
ref = strings.Split(ref, "/")[0]
if !strings.HasSuffix(ref, "-plugin") {
// It's not a jenkins plugin!
return
}
ref = strings.TrimSuffix(ref, "-plugin")
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemJenkinsPlugins]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemJenkinsPlugins] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemJenkinsPlugins][ref] = cpeItemName
}
func addEntryForPyPIPackage(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.TrimPrefix(ref, prefixForPyPIPackages)
ref = strings.Split(ref, "/")[0]
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemPyPI]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemPyPI] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemPyPI][ref] = cpeItemName
}
func addEntryForNativeRubyGem(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.TrimPrefix(ref, prefixForNativeRubyGems)
ref = strings.Split(ref, "/")[0]
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemRubyGems]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemRubyGems] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemRubyGems][ref] = cpeItemName
}
func addEntryForRubyGem(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.TrimPrefix(ref, prefixForRubyGems)
ref = strings.TrimPrefix(ref, prefixForRubyGemsHTTP)
ref = strings.Split(ref, "/")[0]
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemRubyGems]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemRubyGems] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemRubyGems][ref] = cpeItemName
}
func addEntryForNPMPackage(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.Split(ref, "/v/")[0]
ref = strings.Split(ref, "?")[0]
ref = strings.TrimPrefix(ref, prefixForNPMPackages)
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemNPM]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemNPM] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemNPM][ref] = cpeItemName
}
@@ -0,0 +1,169 @@
package main
import (
"bytes"
"compress/gzip"
"io"
"os"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/pkg/cataloger/common/cpe/dictionary"
)
func Test_generateIndexedDictionaryJSON(t *testing.T) {
f, err := os.Open("testdata/official-cpe-dictionary_v2.3.xml")
require.NoError(t, err)
// Create a buffer to store the gzipped data in memory
buf := new(bytes.Buffer)
w := gzip.NewWriter(buf)
_, err = io.Copy(w, f)
require.NoError(t, err)
// (finalize the gzip stream)
err = w.Close()
require.NoError(t, err)
dictionaryJSON, err := generateIndexedDictionaryJSON(buf)
assert.NoError(t, err)
expected, err := os.ReadFile("./testdata/expected-cpe-index.json")
require.NoError(t, err)
expectedDictionaryJSONString := string(expected)
dictionaryJSONString := string(dictionaryJSON)
if diff := cmp.Diff(expectedDictionaryJSONString, dictionaryJSONString); diff != "" {
t.Errorf("generateIndexedDictionaryJSON() mismatch (-want +got):\n%s", diff)
}
}
func Test_addEntryFuncs(t *testing.T) {
tests := []struct {
name string
addEntryFunc func(indexed *dictionary.Indexed, ref string, cpeItemName string)
inputRef string
inputCpeItemName string
expectedIndexed dictionary.Indexed
}{
{
name: "addEntryForRustCrate",
addEntryFunc: addEntryForRustCrate,
inputRef: "https://crates.io/crates/unicycle/versions",
inputCpeItemName: "cpe:2.3:a:unicycle_project:unicycle:*:*:*:*:*:rust:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemRustCrates: {
"unicycle": "cpe:2.3:a:unicycle_project:unicycle:*:*:*:*:*:rust:*:*",
},
},
},
},
{
name: "addEntryForJenkinsPlugin",
addEntryFunc: addEntryForJenkinsPlugin,
inputRef: "https://github.com/jenkinsci/sonarqube-plugin",
inputCpeItemName: "cpe:2.3:a:sonarsource:sonarqube_scanner:2.7:*:*:*:*:jenkins:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemJenkinsPlugins: {
"sonarqube": "cpe:2.3:a:sonarsource:sonarqube_scanner:2.7:*:*:*:*:jenkins:*:*",
},
},
},
},
{
name: "addEntryForJenkinsPlugin: not actually a plugin",
addEntryFunc: addEntryForJenkinsPlugin,
inputRef: "https://github.com/jenkinsci/jenkins",
inputCpeItemName: "cpe:2.3:a:jenkins:jenkinsci:2.7:*:*:*:*:*:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{},
},
},
{
name: "addEntryForPyPIPackage",
addEntryFunc: addEntryForPyPIPackage,
inputRef: "https://pypi.org/project/vault-cli/#history",
inputCpeItemName: "cpe:2.3:a:vault-cli_project:vault-cli:*:*:*:*:*:python:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemPyPI: {
"vault-cli": "cpe:2.3:a:vault-cli_project:vault-cli:*:*:*:*:*:python:*:*",
},
},
},
},
{
name: "addEntryForNativeRubyGem",
addEntryFunc: addEntryForNativeRubyGem,
inputRef: "https://github.com/ruby/openssl/releases",
inputCpeItemName: "cpe:2.3:a:ruby-lang:openssl:-:*:*:*:*:ruby:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemRubyGems: {
"openssl": "cpe:2.3:a:ruby-lang:openssl:-:*:*:*:*:ruby:*:*",
},
},
},
},
{
name: "addEntryForRubyGem: https",
addEntryFunc: addEntryForRubyGem,
inputRef: "https://rubygems.org/gems/actionview/versions",
inputCpeItemName: "cpe:2.3:a:action_view_project:action_view:*:*:*:*:*:ruby:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemRubyGems: {
"actionview": "cpe:2.3:a:action_view_project:action_view:*:*:*:*:*:ruby:*:*",
},
},
},
},
{
name: "addEntryForRubyGem: http",
addEntryFunc: addEntryForRubyGem,
inputRef: "http://rubygems.org/gems/rbovirt",
inputCpeItemName: "cpe:2.3:a:amos_benari:rbovirt:*:*:*:*:*:ruby:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemRubyGems: {
"rbovirt": "cpe:2.3:a:amos_benari:rbovirt:*:*:*:*:*:ruby:*:*",
},
},
},
},
{
name: "addEntryForNPMPackage",
addEntryFunc: addEntryForNPMPackage,
inputRef: "https://www.npmjs.com/package/@nubosoftware/node-static",
inputCpeItemName: "cpe:2.3:a:\\@nubosoftware\\/node-static_project:\\@nubosoftware\\/node-static:-:*:*:*:*:node.js:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemNPM: {
"@nubosoftware/node-static": "cpe:2.3:a:\\@nubosoftware\\/node-static_project:\\@nubosoftware\\/node-static:-:*:*:*:*:node.js:*:*",
},
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
indexed := &dictionary.Indexed{
EcosystemPackages: make(map[string]dictionary.Packages),
}
tt.addEntryFunc(indexed, tt.inputRef, tt.inputCpeItemName)
if diff := cmp.Diff(tt.expectedIndexed, *indexed); diff != "" {
t.Errorf("addEntry* mismatch (-want +got):\n%s", diff)
}
})
}
}
@@ -0,0 +1,57 @@
// This program downloads the latest CPE dictionary from NIST and processes it into a JSON file that can be embedded into Syft for more accurate CPE results.
package main
import (
"errors"
"flag"
"fmt"
"log"
"net/http"
"os"
)
func mainE() error {
var outputFilename string
flag.StringVar(&outputFilename, "o", "", "file location to save CPE index")
flag.Parse()
if outputFilename == "" {
return errors.New("-o is required")
}
// Download and decompress file
fmt.Println("Fetching CPE dictionary...")
resp, err := http.Get(cpeDictionaryURL)
if err != nil {
return fmt.Errorf("unable to get CPE dictionary: %w", err)
}
defer resp.Body.Close()
fmt.Println("Generating index...")
dictionaryJSON, err := generateIndexedDictionaryJSON(resp.Body)
if err != nil {
return err
}
// Write CPE index (JSON data) to disk
err = os.WriteFile(outputFilename, dictionaryJSON, 0600)
if err != nil {
return fmt.Errorf("unable to write processed CPE dictionary to file: %w", err)
}
fmt.Println("Done!")
return nil
}
// errExit prints an error and exits with a non-zero exit code.
func errExit(err error) {
log.Printf("command failed: %s", err)
os.Exit(1)
}
func main() {
if err := mainE(); err != nil {
errExit(err)
}
}
@@ -0,0 +1,21 @@
package main
type CpeItem struct {
Name string `xml:"name,attr"`
Title string `xml:"title"`
References []struct {
Reference struct {
Href string `xml:"href,attr"`
Body string `xml:",chardata"`
} `xml:"reference"`
} `xml:"references"`
Cpe23Item struct {
Name string `xml:"name,attr"`
} `xml:"cpe23-item"`
}
type CpeList struct {
CpeItems []CpeItem `xml:"cpe-item"`
}
const cpeDictionaryURL = "https://nvd.nist.gov/feeds/xml/cpe/dictionary/official-cpe-dictionary_v2.3.xml.gz"
@@ -0,0 +1,23 @@
{
"ecosystems": {
"jenkins_plugins": {
"sonarqube": "cpe:2.3:a:sonarsource:sonarqube_scanner:*:*:*:*:*:jenkins:*:*"
},
"npm": {
"merge-recursive": "cpe:2.3:a:umbraengineering:merge-recursive:*:*:*:*:*:node.js:*:*",
"static-dev-server": "cpe:2.3:a:static-dev-server_project:static-dev-server:*:*:*:*:*:node.js:*:*",
"umount": "cpe:2.3:a:umount_project:umount:*:*:*:*:*:node.js:*:*",
"undefsafe": "cpe:2.3:a:undefsafe_project:undefsafe:*:*:*:*:*:node.js:*:*",
"underscore": "cpe:2.3:a:underscorejs:underscore:*:*:*:*:*:node.js:*:*",
"underscore-99xp": "cpe:2.3:a:underscore-99xp_project:underscore-99xp:*:*:*:*:*:node.js:*:*",
"unicode": "cpe:2.3:a:unicode_project:unicode:*:*:*:*:*:node.js:*:*",
"unicorn-list": "cpe:2.3:a:unicorn-list_project:unicorn-list:*:*:*:*:*:node.js:*:*"
},
"rubygems": {
"openssl": "cpe:2.3:a:ruby-lang:openssl:*:*:*:*:*:*:*:*"
},
"rust_crates": {
"unicycle": "cpe:2.3:a:unicycle_project:unicycle:*:*:*:*:*:rust:*:*"
}
}
}
@@ -0,0 +1,15 @@
package dictionary
const (
EcosystemNPM = "npm"
EcosystemRubyGems = "rubygems"
EcosystemPyPI = "pypi"
EcosystemJenkinsPlugins = "jenkins_plugins"
EcosystemRustCrates = "rust_crates"
)
type Indexed struct {
EcosystemPackages map[string]Packages `json:"ecosystems"`
}
type Packages map[string]string
+76
View File
@@ -3,16 +3,21 @@ package cpe
import (
"bufio"
"bytes"
_ "embed"
"encoding/json"
"fmt"
"sort"
"strings"
"sync"
"github.com/facebookincubator/nvdtools/wfn"
"github.com/scylladb/go-set/strset"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/common/cpe/dictionary"
)
// knownVendors contains vendor strings that are known to exist in
@@ -32,6 +37,77 @@ func newCPE(product, vendor, version, targetSW string) *wfn.Attributes {
return &c
}
//go:embed dictionary/data/cpe-index.json
var indexedCPEDictionaryData []byte
var indexedCPEDictionary *dictionary.Indexed
var indexedCPEDictionaryOnce sync.Once
func GetIndexedDictionary() (_ *dictionary.Indexed, err error) {
indexedCPEDictionaryOnce.Do(func() {
err = json.Unmarshal(indexedCPEDictionaryData, &indexedCPEDictionary)
})
if err != nil {
return
}
if indexedCPEDictionary == nil {
err = fmt.Errorf("failed to unmarshal indexed CPE dictionary")
return
}
return indexedCPEDictionary, err
}
func DictionaryFind(p pkg.Package) (cpe.CPE, bool) {
dict, err := GetIndexedDictionary()
if err != nil {
log.Debugf("dictionary CPE lookup not available: %+v", err)
return cpe.CPE{}, false
}
var (
cpeString string
ok bool
)
switch p.Type {
case pkg.NpmPkg:
cpeString, ok = dict.EcosystemPackages[dictionary.EcosystemNPM][p.Name]
case pkg.GemPkg:
cpeString, ok = dict.EcosystemPackages[dictionary.EcosystemRubyGems][p.Name]
case pkg.PythonPkg:
cpeString, ok = dict.EcosystemPackages[dictionary.EcosystemPyPI][p.Name]
case pkg.JenkinsPluginPkg:
cpeString, ok = dict.EcosystemPackages[dictionary.EcosystemJenkinsPlugins][p.Name]
case pkg.RustPkg:
cpeString, ok = dict.EcosystemPackages[dictionary.EcosystemRustCrates][p.Name]
default:
// The dictionary doesn't support this package type yet.
return cpe.CPE{}, false
}
if !ok {
// The dictionary doesn't have a CPE for this package.
return cpe.CPE{}, false
}
parsedCPE, err := cpe.New(cpeString)
if err != nil {
return cpe.CPE{}, false
}
parsedCPE.Version = p.Version
return parsedCPE, true
}
// Generate Create a list of CPEs for a given package, trying to guess the vendor, product tuple. We should be trying to
// generate the minimal set of representative CPEs, which implies that optional fields should not be included
// (such as target SW).
+70 -25
View File
@@ -354,11 +354,12 @@ func TestGeneratePackageCPEs(t *testing.T) {
{
name: "cloudbees jenkins package identified via groupId",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "com.cloudbees.jenkins.plugins",
@@ -368,16 +369,18 @@ func TestGeneratePackageCPEs(t *testing.T) {
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:cloudbees:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "jenkins.io package identified via groupId prefix",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "io.jenkins.plugins.name.something",
@@ -389,16 +392,19 @@ func TestGeneratePackageCPEs(t *testing.T) {
"cpe:2.3:a:name:something:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:something:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:something:something:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins:something:3.2:*:*:*:*:*:*:*",
},
},
{
name: "jenkins.io package identified via groupId",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "io.jenkins.plugins",
@@ -407,16 +413,18 @@ func TestGeneratePackageCPEs(t *testing.T) {
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "jenkins-ci.io package identified via groupId",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "io.jenkins-ci.plugins",
@@ -425,16 +433,20 @@ func TestGeneratePackageCPEs(t *testing.T) {
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins-ci:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins_ci:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "jenkins-ci.org package identified via groupId",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "org.jenkins-ci.plugins",
@@ -443,6 +455,9 @@ func TestGeneratePackageCPEs(t *testing.T) {
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins-ci:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins_ci:name:3.2:*:*:*:*:*:*:*",
},
},
{
@@ -969,3 +984,33 @@ func Test_addSeparatorVariations(t *testing.T) {
})
}
}
func TestDictionaryFindIsWired(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
want string
wantExists bool
}{
{
name: "sanity check that cpe data is wired up",
pkg: pkg.Package{
Name: "openssl",
Version: "1.0.2k",
Type: pkg.GemPkg,
},
want: "cpe:2.3:a:ruby-lang:openssl:1.0.2k:*:*:*:*:*:*:*",
// without the cpe data wired up, this would be empty (generation also creates cpe:2.3:a:openssl:openssl:1.0.2k:*:*:*:*:*:*:*)
wantExists: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, gotExists := DictionaryFind(tt.pkg)
assert.Equal(t, tt.want, got.BindToFmtString())
assert.Equal(t, tt.wantExists, gotExists)
})
}
}
+20 -46
View File
@@ -5,7 +5,6 @@ import (
"github.com/scylladb/go-set/strset"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/pkg"
)
@@ -13,14 +12,6 @@ var (
forbiddenProductGroupIDFields = strset.New("plugin", "plugins", "client")
forbiddenVendorGroupIDFields = strset.New("plugin", "plugins")
domains = []string{
"com",
"org",
"net",
"io",
"be",
}
primaryJavaManifestGroupIDFields = []string{
"Extension-Name",
"Specification-Vendor",
@@ -66,7 +57,7 @@ func vendorsFromJavaManifestNames(p pkg.Package) fieldCandidateSet {
for _, name := range javaManifestNameFields {
if metadata.Manifest.Main != nil {
if value, exists := metadata.Manifest.Main[name]; exists {
if !startsWithTopLevelDomain(value) {
if !looksLikeGroupID(value) {
vendors.add(fieldCandidate{
value: normalizePersonName(value),
disallowSubSelections: true,
@@ -80,7 +71,7 @@ func vendorsFromJavaManifestNames(p pkg.Package) fieldCandidateSet {
continue
}
if value, exists := section[name]; exists {
if !startsWithTopLevelDomain(value) {
if !looksLikeGroupID(value) {
vendors.add(fieldCandidate{
value: normalizePersonName(value),
disallowSubSelections: true,
@@ -144,7 +135,7 @@ func productsFromArtifactAndGroupIDs(artifactID string, groupIDs []string) []str
continue
}
// umbrella projects tend to have sub components that either start or end with the project name. We expect
// umbrella projects tend to have subcomponents that either start or end with the project name. We expect
// to identify fields that may represent the umbrella project, and not fields that indicate auxiliary
// information about the package.
couldBeProjectName := strings.HasPrefix(artifactID, field) || strings.HasSuffix(artifactID, field)
@@ -168,7 +159,7 @@ func artifactIDFromJavaPackage(p pkg.Package) string {
}
artifactID := strings.TrimSpace(metadata.PomProperties.ArtifactID)
if startsWithTopLevelDomain(artifactID) && len(strings.Split(artifactID, ".")) > 1 {
if looksLikeGroupID(artifactID) {
// there is a strong indication that the artifact ID is really a group ID, don't use it
return ""
}
@@ -181,13 +172,13 @@ func GroupIDsFromJavaPackage(p pkg.Package) (groupIDs []string) {
return nil
}
return GroupIDsFromJavaMetadata(metadata)
return GroupIDsFromJavaMetadata(p.Name, metadata)
}
func GroupIDsFromJavaMetadata(metadata pkg.JavaMetadata) (groupIDs []string) {
func GroupIDsFromJavaMetadata(pkgName string, metadata pkg.JavaMetadata) (groupIDs []string) {
groupIDs = append(groupIDs, groupIDsFromPomProperties(metadata.PomProperties)...)
groupIDs = append(groupIDs, groupIDsFromPomProject(metadata.PomProject)...)
groupIDs = append(groupIDs, groupIDsFromJavaManifest(metadata.Manifest)...)
groupIDs = append(groupIDs, groupIDsFromJavaManifest(pkgName, metadata.Manifest)...)
return groupIDs
}
@@ -197,16 +188,10 @@ func groupIDsFromPomProperties(properties *pkg.PomProperties) (groupIDs []string
return nil
}
if startsWithTopLevelDomain(properties.GroupID) {
if properties.GroupID != "" {
groupIDs = append(groupIDs, cleanGroupID(properties.GroupID))
}
// sometimes the publisher puts the group ID in the artifact ID field unintentionally
if startsWithTopLevelDomain(properties.ArtifactID) && len(strings.Split(properties.ArtifactID, ".")) > 1 {
// there is a strong indication that the artifact ID is really a group ID
groupIDs = append(groupIDs, cleanGroupID(properties.ArtifactID))
}
return groupIDs
}
@@ -215,33 +200,22 @@ func groupIDsFromPomProject(project *pkg.PomProject) (groupIDs []string) {
return nil
}
// extract the project info...
groupIDs = addGroupIDsFromGroupIDsAndArtifactID(project.GroupID, project.ArtifactID)
if project.Parent == nil {
return groupIDs
if project.GroupID != "" {
groupIDs = append(groupIDs, project.GroupID)
}
// extract the parent project info...
groupIDs = append(groupIDs, addGroupIDsFromGroupIDsAndArtifactID(project.Parent.GroupID, project.Parent.ArtifactID)...)
if project.Parent != nil && project.Parent.GroupID != "" {
groupIDs = append(groupIDs, project.Parent.GroupID)
}
return groupIDs
}
func addGroupIDsFromGroupIDsAndArtifactID(groupID, artifactID string) (groupIDs []string) {
if startsWithTopLevelDomain(groupID) {
groupIDs = append(groupIDs, cleanGroupID(groupID))
func groupIDsFromJavaManifest(pkgName string, manifest *pkg.JavaManifest) []string {
if groupID, ok := defaultArtifactIDToGroupID[pkgName]; ok {
return []string{groupID}
}
// sometimes the publisher puts the group ID in the artifact ID field unintentionally
if startsWithTopLevelDomain(artifactID) && len(strings.Split(artifactID, ".")) > 1 {
// there is a strong indication that the artifact ID is really a group ID
groupIDs = append(groupIDs, cleanGroupID(artifactID))
}
return groupIDs
}
func groupIDsFromJavaManifest(manifest *pkg.JavaManifest) []string {
if manifest == nil {
return nil
}
@@ -268,13 +242,13 @@ func getManifestFieldGroupIDs(manifest *pkg.JavaManifest, fields []string) (grou
for _, name := range fields {
if value, exists := manifest.Main[name]; exists {
if startsWithTopLevelDomain(value) {
if looksLikeGroupID(value) {
groupIDs = append(groupIDs, cleanGroupID(value))
}
}
for _, section := range manifest.NamedSections {
if value, exists := section[name]; exists {
if startsWithTopLevelDomain(value) {
if looksLikeGroupID(value) {
groupIDs = append(groupIDs, cleanGroupID(value))
}
}
@@ -295,6 +269,6 @@ func removeOSCIDirectives(groupID string) string {
return strings.Split(groupID, ";")[0]
}
func startsWithTopLevelDomain(value string) bool {
return internal.HasAnyOfPrefixes(value, domains...)
func looksLikeGroupID(value string) bool {
return strings.Contains(value, ".")
}
@@ -0,0 +1,69 @@
package cpe
var defaultArtifactIDToGroupID = map[string]string{
"ant": "org.apache.ant",
"ant-antlr": "org.apache.ant",
"ant-antunit": "org.apache.ant",
"ant-apache-bcel": "org.apache.ant",
"ant-apache-bsf": "org.apache.ant",
"ant-apache-log4j": "org.apache.ant",
"ant-apache-oro": "org.apache.ant",
"ant-apache-regexp": "org.apache.ant",
"ant-apache-resolver": "org.apache.ant",
"ant-apache-xalan2": "org.apache.ant",
"ant-commons-logging": "org.apache.ant",
"ant-commons-net": "org.apache.ant",
"ant-compress": "org.apache.ant",
"ant-dotnet": "org.apache.ant",
"ant-imageio": "org.apache.ant",
"ant-jai": "org.apache.ant",
"ant-jakartamail": "org.apache.ant",
"ant-javamail": "org.apache.ant",
"ant-jdepend": "org.apache.ant",
"ant-jmf": "org.apache.ant",
"ant-jsch": "org.apache.ant",
"ant-junit": "org.apache.ant",
"ant-junit4": "org.apache.ant",
"ant-junitlauncher": "org.apache.ant",
"ant-launcher": "org.apache.ant",
"ant-netrexx": "org.apache.ant",
"ant-nodeps": "org.apache.ant",
"ant-parent": "org.apache.ant",
"ant-starteam": "org.apache.ant",
"ant-stylebook": "org.apache.ant",
"ant-swing": "org.apache.ant",
"ant-testutil": "org.apache.ant",
"ant-trax": "org.apache.ant",
"ant-weblogic": "org.apache.ant",
"ant-xz": "org.apache.ant",
"spring": "org.springframework",
"spring-amqp": "org.springframework.amqp",
"spring-batch-core": "org.springframework.batch",
"spring-beans": "org.springframework",
"spring-boot": "org.springframework.boot",
"spring-boot-starter-web": "org.springframework.boot",
"spring-boot-starter-webflux": "org.springframework.boot",
"spring-cloud-function-context": "org.springframework.cloud",
"spring-cloud-function-parent": "org.springframework.cloud",
"spring-cloud-gateway": "org.springframework.cloud",
"spring-cloud-openfeign-core": "org.springframework.cloud",
"spring-cloud-task-dependencies": "org.springframework.cloud",
"spring-core": "org.springframework",
"spring-data-jpa": "org.springframework.data",
"spring-data-mongodb": "org.springframework.data",
"spring-data-rest-core": "org.springframework.data",
"spring-expression": "org.springframework",
"spring-integration-zip": "org.springframework.integration",
"spring-oxm": "org.springframework",
"spring-security-core": "org.springframework.security",
"spring-security-config": "org.springframework.security",
"spring-security-oauth": "org.springframework.security.oauth",
"spring-security-oauth-parent": "org.springframework.security.oauth",
"spring-security-oauth2-client": "org.springframework.security",
"spring-session-core": "org.springframework.session",
"spring-vault-core": "org.springframework.vault",
"spring-web": "org.springframework",
"spring-webflow": "org.springframework.webflow",
"spring-webflux": "org.springframework",
"spring-webmvc": "org.springframework",
}
@@ -5,6 +5,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/pkg"
)
@@ -427,3 +428,38 @@ func Test_vendorsFromJavaManifestNames(t *testing.T) {
})
}
}
func Test_groupIDsFromJavaManifest(t *testing.T) {
tests := []struct {
name string
manifest pkg.JavaManifest
expected []string
}{
{
name: "spring-security-core",
manifest: pkg.JavaManifest{},
expected: []string{"org.springframework.security"},
},
{
name: "spring-web",
manifest: pkg.JavaManifest{},
expected: []string{"org.springframework"},
},
{
name: "spring-foo",
manifest: pkg.JavaManifest{
Main: map[string]string{
"Implementation-Vendor": "org.foo",
},
},
expected: []string{"org.foo"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
got := groupIDsFromJavaManifest(test.name, &test.manifest)
require.Equal(t, test.expected, got)
})
}
}
+13 -17
View File
@@ -4,23 +4,27 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/golang"
"github.com/anchore/syft/syft/pkg/cataloger/java"
"github.com/anchore/syft/syft/pkg/cataloger/kernel"
"github.com/anchore/syft/syft/pkg/cataloger/python"
)
// TODO: these field naming vs helper function naming schemes are inconsistent.
type Config struct {
Search SearchConfig
Golang golang.GoCatalogerOpts
LinuxKernel kernel.LinuxCatalogerConfig
Catalogers []string
Parallelism int
Search SearchConfig
Golang golang.GoCatalogerOpts
LinuxKernel kernel.LinuxCatalogerConfig
Python python.CatalogerConfig
Catalogers []string
Parallelism int
ExcludeBinaryOverlapByOwnership bool
}
func DefaultConfig() Config {
return Config{
Search: DefaultSearchConfig(),
Parallelism: 1,
LinuxKernel: kernel.DefaultLinuxCatalogerConfig(),
Search: DefaultSearchConfig(),
Parallelism: 1,
LinuxKernel: kernel.DefaultLinuxCatalogerConfig(),
Python: python.DefaultCatalogerConfig(),
ExcludeBinaryOverlapByOwnership: true,
}
}
@@ -30,11 +34,3 @@ func (c Config) Java() java.Config {
SearchIndexedArchives: c.Search.IncludeIndexedArchives,
}
}
func (c Config) Go() golang.GoCatalogerOpts {
return c.Golang
}
func (c Config) Kernel() kernel.LinuxCatalogerConfig {
return c.LinuxKernel
}
+1 -1
View File
@@ -14,5 +14,5 @@ func NewDpkgdbCataloger() *generic.Cataloger {
return generic.NewCataloger(catalogerName).
// note: these globs have been intentionally split up in order to improve search performance,
// please do NOT combine into: "**/var/lib/dpkg/{status,status.d/*}"
WithParserByGlobs(parseDpkgDB, "**/var/lib/dpkg/status", "**/var/lib/dpkg/status.d/*")
WithParserByGlobs(parseDpkgDB, "**/var/lib/dpkg/status", "**/var/lib/dpkg/status.d/*", "**/lib/opkg/info/*.control", "**/lib/opkg/status")
}
+2
View File
@@ -168,6 +168,8 @@ func TestCataloger_Globs(t *testing.T) {
expected: []string{
"var/lib/dpkg/status",
"var/lib/dpkg/status.d/pkg-1.0",
"usr/lib/opkg/status",
"usr/lib/opkg/info/pkg-1.0.control",
},
},
}
@@ -0,0 +1 @@
bogus package
@@ -0,0 +1 @@
bogus status
+6 -3
View File
@@ -4,10 +4,13 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
const catalogerName = "dotnet-deps-cataloger"
// NewDotnetDepsCataloger returns a new Dotnet cataloger object base on deps json files.
func NewDotnetDepsCataloger() *generic.Cataloger {
return generic.NewCataloger(catalogerName).
return generic.NewCataloger("dotnet-deps-cataloger").
WithParserByGlobs(parseDotnetDeps, "**/*.deps.json")
}
func NewDotnetPortableExecutableCataloger() *generic.Cataloger {
return generic.NewCataloger("dotnet-portable-executable-cataloger").
WithParserByGlobs(parseDotnetPortableExecutable, "**/*.dll", "**/*.exe")
}
+18 -6
View File
@@ -3,22 +3,34 @@ package dotnet
import (
"testing"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
func TestCataloger_Globs(t *testing.T) {
tests := []struct {
name string
fixture string
expected []string
name string
fixture string
cataloger *generic.Cataloger
expected []string
}{
{
name: "obtain deps.json files",
fixture: "test-fixtures/glob-paths",
name: "obtain deps.json files",
fixture: "test-fixtures/glob-paths",
cataloger: NewDotnetDepsCataloger(),
expected: []string{
"src/something.deps.json",
},
},
{
name: "obtain portable executable files",
fixture: "test-fixtures/glob-paths",
cataloger: NewDotnetPortableExecutableCataloger(),
expected: []string{
"src/something.dll",
"src/something.exe",
},
},
}
for _, test := range tests {
@@ -26,7 +38,7 @@ func TestCataloger_Globs(t *testing.T) {
pkgtest.NewCatalogTester().
FromDirectory(t, test.fixture).
ExpectsResolverContentQueries(test.expected).
TestCataloger(t, NewDotnetDepsCataloger())
TestCataloger(t, test.cataloger)
})
}
}
@@ -0,0 +1,87 @@
package dotnet
import (
"fmt"
"io"
"github.com/saferwall/pe"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
var _ generic.Parser = parseDotnetPortableExecutable
func parseDotnetPortableExecutable(_ file.Resolver, _ *generic.Environment, f file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
by, err := io.ReadAll(f)
if err != nil {
return nil, nil, fmt.Errorf("unable to read file: %w", err)
}
peFile, err := pe.NewBytes(by, &pe.Options{})
if err != nil {
return nil, nil, fmt.Errorf("unable to create PE file instance: %w", err)
}
err = peFile.Parse()
if err != nil {
return nil, nil, fmt.Errorf("unable to parse PE file: %w", err)
}
versionResources, err := peFile.ParseVersionResources()
if err != nil {
// this is not a fatal error, just log and continue
// TODO: consider this case for "known unknowns" (same goes for cases below)
log.Tracef("unable to parse version resources in PE file: %s", f.RealPath)
return nil, nil, nil
}
name := versionResources["FileDescription"]
if name == "" {
log.Tracef("unable to find FileDescription in PE file: %s", f.RealPath)
return nil, nil, nil
}
version := versionResources["FileVersion"]
if version == "" {
log.Tracef("unable to find FileVersion in PE file: %s", f.RealPath)
return nil, nil, nil
}
purl := packageurl.NewPackageURL(
packageurl.TypeNuget, // See explanation in syft/pkg/cataloger/dotnet/package.go as to why this was chosen.
"",
name,
version,
nil,
"",
).ToString()
metadata := pkg.DotnetPortableExecutableMetadata{
AssemblyVersion: versionResources["Assembly Version"],
LegalCopyright: versionResources["LegalCopyright"],
Comments: versionResources["Comments"],
InternalName: versionResources["InternalName"],
CompanyName: versionResources["CompanyName"],
ProductName: versionResources["ProductName"],
ProductVersion: versionResources["ProductVersion"],
}
p := pkg.Package{
Name: name,
Version: version,
Locations: file.NewLocationSet(f.Location),
Type: pkg.DotnetPkg,
PURL: purl,
MetadataType: pkg.DotnetPortableExecutableMetadataType,
Metadata: metadata,
}
p.SetID()
return []pkg.Package{p}, nil, nil
}
@@ -0,0 +1,38 @@
package dotnet
import (
"testing"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
func TestParseDotnetPortableExecutable(t *testing.T) {
fixture := "test-fixtures/System.Buffers.dll"
fixtureLocationSet := file.NewLocationSet(file.NewLocation(fixture))
expected := []pkg.Package{
{
Name: "System.Buffers",
Version: "7.0.923.36201",
Locations: fixtureLocationSet,
Type: pkg.DotnetPkg,
PURL: "pkg:nuget/System.Buffers@7.0.923.36201",
MetadataType: pkg.DotnetPortableExecutableMetadataType,
Metadata: pkg.DotnetPortableExecutableMetadata{
AssemblyVersion: "7.0.0.0",
LegalCopyright: "© Microsoft Corporation. All rights reserved.",
Comments: "System.Buffers",
InternalName: "System.Buffers.dll",
CompanyName: "Microsoft Corporation",
ProductName: "Microsoft® .NET",
ProductVersion: "7.0.9+8e9a17b2216f51a5788f8b1c467a4cf3b769e7d7",
},
},
}
var expectedRelationships []artifact.Relationship
pkgtest.TestFileParser(t, fixture, parseDotnetPortableExecutable, expected, expectedRelationships)
}
@@ -0,0 +1,2 @@
!*.dll
!*.exe
@@ -0,0 +1 @@
bogus .dll (portable executable)
@@ -0,0 +1 @@
bogus .exe portable executable)
+1 -1
View File
@@ -9,7 +9,7 @@ import (
// PackageURL returns the PURL for the specific java package (see https://github.com/package-url/purl-spec)
func packageURL(name, version string, metadata pkg.JavaMetadata) string {
var groupID = name
groupIDs := cpe.GroupIDsFromJavaMetadata(metadata)
groupIDs := cpe.GroupIDsFromJavaMetadata(name, metadata)
if len(groupIDs) > 0 {
groupID = groupIDs[0]
}
@@ -57,7 +57,16 @@ func parseGradleLockfile(_ file.Resolver, _ *generic.Environment, reader file.Lo
Language: pkg.Java,
Type: pkg.JavaPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProject: &pkg.PomProject{
GroupID: dep.Group,
ArtifactID: dep.Name,
Version: dep.Version,
Name: dep.Name,
},
},
}
mappedPkg.SetID()
pkgs = append(pkgs, mappedPkg)
}
@@ -16,12 +16,25 @@ func Test_parserGradleLockfile(t *testing.T) {
{
input: "test-fixtures/gradle/gradle.lockfile",
expected: []pkg.Package{
{
Name: "commons-text",
Version: "1.8",
Language: pkg.Java,
Type: pkg.JavaPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProject: &pkg.PomProject{GroupID: "org.apache.commons", ArtifactID: "commons-text", Version: "1.8", Name: "commons-text"},
},
},
{
Name: "hamcrest-core",
Version: "1.3",
Language: pkg.Java,
Type: pkg.JavaPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProject: &pkg.PomProject{GroupID: "org.hamcrest", ArtifactID: "hamcrest-core", Version: "1.3", Name: "hamcrest-core"},
},
},
{
Name: "joda-time",
@@ -29,6 +42,9 @@ func Test_parserGradleLockfile(t *testing.T) {
Language: pkg.Java,
Type: pkg.JavaPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProject: &pkg.PomProject{GroupID: "joda-time", ArtifactID: "joda-time", Version: "2.2", Name: "joda-time"},
},
},
{
Name: "junit",
@@ -36,6 +52,9 @@ func Test_parserGradleLockfile(t *testing.T) {
Language: pkg.Java,
Type: pkg.JavaPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProject: &pkg.PomProject{GroupID: "junit", ArtifactID: "junit", Version: "4.12", Name: "junit"},
},
},
},
},
+67 -27
View File
@@ -28,17 +28,19 @@ func parserPomXML(_ file.Resolver, _ *generic.Environment, reader file.LocationR
}
var pkgs []pkg.Package
for _, dep := range pom.Dependencies {
p := newPackageFromPom(
pom,
dep,
reader.Location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
)
if p.Name == "" {
continue
}
if pom.Dependencies != nil {
for _, dep := range *pom.Dependencies {
p := newPackageFromPom(
pom,
dep,
reader.Location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
)
if p.Name == "" {
continue
}
pkgs = append(pkgs, p)
pkgs = append(pkgs, p)
}
}
return pkgs, nil, nil
@@ -53,15 +55,18 @@ func parsePomXMLProject(path string, reader io.Reader) (*pkg.PomProject, error)
}
func newPomProject(path string, p gopom.Project) *pkg.PomProject {
artifactID := safeString(p.ArtifactID)
name := safeString(p.Name)
projectURL := safeString(p.URL)
return &pkg.PomProject{
Path: path,
Parent: pomParent(p, p.Parent),
GroupID: resolveProperty(p, p.GroupID),
ArtifactID: p.ArtifactID,
ArtifactID: artifactID,
Version: resolveProperty(p, p.Version),
Name: p.Name,
Name: name,
Description: cleanDescription(p.Description),
URL: p.URL,
URL: projectURL,
}
}
@@ -74,7 +79,7 @@ func newPackageFromPom(pom gopom.Project, dep gopom.Dependency, locations ...fil
},
}
name := dep.ArtifactID
name := safeString(dep.ArtifactID)
version := resolveProperty(pom, dep.Version)
p := pkg.Package{
@@ -104,19 +109,29 @@ func decodePomXML(content io.Reader) (project gopom.Project, err error) {
return project, nil
}
func pomParent(pom gopom.Project, parent gopom.Parent) (result *pkg.PomParent) {
if parent.ArtifactID != "" || parent.GroupID != "" || parent.Version != "" {
result = &pkg.PomParent{
GroupID: resolveProperty(pom, parent.GroupID),
ArtifactID: parent.ArtifactID,
Version: resolveProperty(pom, parent.Version),
}
func pomParent(pom gopom.Project, parent *gopom.Parent) (result *pkg.PomParent) {
if parent == nil {
return nil
}
artifactID := safeString(parent.ArtifactID)
result = &pkg.PomParent{
GroupID: resolveProperty(pom, parent.GroupID),
ArtifactID: artifactID,
Version: resolveProperty(pom, parent.Version),
}
if result.GroupID == "" && result.ArtifactID == "" && result.Version == "" {
return nil
}
return result
}
func cleanDescription(original string) (cleaned string) {
descriptionLines := strings.Split(original, "\n")
func cleanDescription(original *string) (cleaned string) {
if original == nil {
return ""
}
descriptionLines := strings.Split(*original, "\n")
for _, line := range descriptionLines {
line = strings.TrimSpace(line)
if len(line) == 0 {
@@ -130,12 +145,17 @@ func cleanDescription(original string) (cleaned string) {
// resolveProperty emulates some maven property resolution logic by looking in the project's variables
// as well as supporting the project expressions like ${project.parent.groupId}.
// If no match is found, the entire expression including ${} is returned
func resolveProperty(pom gopom.Project, property string) string {
return propertyMatcher.ReplaceAllStringFunc(property, func(match string) string {
//
//nolint:gocognit
func resolveProperty(pom gopom.Project, property *string) string {
propertyCase := safeString(property)
return propertyMatcher.ReplaceAllStringFunc(propertyCase, func(match string) string {
propertyName := strings.TrimSpace(match[2 : len(match)-1])
if value, ok := pom.Properties.Entries[propertyName]; ok {
entries := pomProperties(pom)
if value, ok := entries[propertyName]; ok {
return value
}
// if we don't find anything directly in the pom properties,
// see if we have a project.x expression and process this based
// on the xml tags in gopom
@@ -151,9 +171,15 @@ func resolveProperty(pom gopom.Project, property string) string {
part := parts[partNum]
for fieldNum := 0; fieldNum < pomValueType.NumField(); fieldNum++ {
f := pomValueType.Field(fieldNum)
if part == f.Tag.Get("xml") {
tag := f.Tag.Get("xml")
tag = strings.TrimSuffix(tag, ",omitempty")
if part == tag {
pomValue = pomValue.Field(fieldNum)
pomValueType = pomValue.Type()
if pomValueType.Kind() == reflect.Ptr {
pomValue = pomValue.Elem()
pomValueType = pomValue.Type()
}
if partNum == numParts-1 {
return fmt.Sprintf("%v", pomValue.Interface())
}
@@ -165,3 +191,17 @@ func resolveProperty(pom gopom.Project, property string) string {
return match
})
}
func pomProperties(p gopom.Project) map[string]string {
if p.Properties != nil {
return p.Properties.Entries
}
return map[string]string{}
}
func safeString(s *string) string {
if s == nil {
return ""
}
return *s
}
+25 -16
View File
@@ -272,13 +272,13 @@ func Test_parsePomXMLProject(t *testing.T) {
func Test_pomParent(t *testing.T) {
tests := []struct {
name string
input gopom.Parent
input *gopom.Parent
expected *pkg.PomParent
}{
{
name: "only group ID",
input: gopom.Parent{
GroupID: "org.something",
input: &gopom.Parent{
GroupID: stringPointer("org.something"),
},
expected: &pkg.PomParent{
GroupID: "org.something",
@@ -286,8 +286,8 @@ func Test_pomParent(t *testing.T) {
},
{
name: "only artifact ID",
input: gopom.Parent{
ArtifactID: "something",
input: &gopom.Parent{
ArtifactID: stringPointer("something"),
},
expected: &pkg.PomParent{
ArtifactID: "something",
@@ -295,22 +295,27 @@ func Test_pomParent(t *testing.T) {
},
{
name: "only Version",
input: gopom.Parent{
Version: "something",
input: &gopom.Parent{
Version: stringPointer("something"),
},
expected: &pkg.PomParent{
Version: "something",
},
},
{
name: "nil",
input: nil,
expected: nil,
},
{
name: "empty",
input: gopom.Parent{},
input: &gopom.Parent{},
expected: nil,
},
{
name: "unused field",
input: gopom.Parent{
RelativePath: "something",
input: &gopom.Parent{
RelativePath: stringPointer("something"),
},
expected: nil,
},
@@ -341,7 +346,7 @@ func Test_cleanDescription(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, cleanDescription(test.input))
assert.Equal(t, test.expected, cleanDescription(stringPointer(test.input)))
})
}
}
@@ -357,7 +362,7 @@ func Test_resolveProperty(t *testing.T) {
name: "property",
property: "${version.number}",
pom: gopom.Project{
Properties: gopom.Properties{
Properties: &gopom.Properties{
Entries: map[string]string{
"version.number": "12.5.0",
},
@@ -369,7 +374,7 @@ func Test_resolveProperty(t *testing.T) {
name: "groupId",
property: "${project.groupId}",
pom: gopom.Project{
GroupID: "org.some.group",
GroupID: stringPointer("org.some.group"),
},
expected: "org.some.group",
},
@@ -377,8 +382,8 @@ func Test_resolveProperty(t *testing.T) {
name: "parent groupId",
property: "${project.parent.groupId}",
pom: gopom.Project{
Parent: gopom.Parent{
GroupID: "org.some.parent",
Parent: &gopom.Parent{
GroupID: stringPointer("org.some.parent"),
},
},
expected: "org.some.parent",
@@ -387,8 +392,12 @@ func Test_resolveProperty(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
resolved := resolveProperty(test.pom, test.property)
resolved := resolveProperty(test.pom, stringPointer(test.property))
assert.Equal(t, test.expected, resolved)
})
}
}
func stringPointer(s string) *string {
return &s
}
@@ -5,3 +5,4 @@ joda-time:joda-time:2.2=compileClasspath,runtimeClasspath,testCompileClasspath,t
junit:junit:4.12=testCompileClasspath,testRuntimeClasspath
org.hamcrest:hamcrest-core:1.3=testCompileClasspath,testRuntimeClasspath
empty=annotationProcessor,testAnnotationProcessor
org.apache.commons:commons-text:1.8=compileClasspath
+51
View File
@@ -0,0 +1,51 @@
package cataloger
import (
"golang.org/x/exp/slices"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
)
var (
osCatalogerTypes = []pkg.Type{
pkg.AlpmPkg,
pkg.ApkPkg,
pkg.DebPkg,
pkg.NixPkg,
pkg.PortagePkg,
pkg.RpmPkg,
}
binaryCatalogerTypes = []pkg.Type{
pkg.BinaryPkg,
}
)
// ExcludeBinaryByFileOwnershipOverlap will remove packages from a collection given the following properties are true
// 1) the relationship between packages is OwnershipByFileOverlap
// 2) the parent is an "os" package
// 3) the child is a synthetic package generated by the binary cataloger
// 4) the package names are identical
// This was implemented as a way to help resolve: https://github.com/anchore/syft/issues/931
func ExcludeBinaryByFileOwnershipOverlap(r artifact.Relationship, c *pkg.Collection) bool {
if artifact.OwnershipByFileOverlapRelationship != r.Type {
return false
}
parent := c.Package(r.From.ID())
if parent == nil {
return false
}
parentInExclusion := slices.Contains(osCatalogerTypes, parent.Type)
if !parentInExclusion {
return false
}
child := c.Package(r.To.ID())
if child == nil {
return false
}
return slices.Contains(binaryCatalogerTypes, child.Type)
}
@@ -0,0 +1,76 @@
package cataloger
import (
"testing"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
)
func TestExclude(t *testing.T) {
packageA := pkg.Package{Name: "package-a", Type: pkg.ApkPkg}
packageB := pkg.Package{Name: "package-a", Type: pkg.PythonPkg}
packageC := pkg.Package{Name: "package-a", Type: pkg.BinaryPkg}
packageD := pkg.Package{Name: "package-d", Type: pkg.BinaryPkg}
for _, p := range []*pkg.Package{&packageA, &packageB, &packageC, &packageD} {
p := p
p.SetID()
}
tests := []struct {
name string
relationship artifact.Relationship
packages *pkg.Collection
shouldExclude bool
}{
{
name: "no exclusions from os -> python",
relationship: artifact.Relationship{
Type: artifact.OwnershipByFileOverlapRelationship,
From: packageA,
To: packageB,
},
packages: pkg.NewCollection(packageA, packageB),
shouldExclude: false,
},
{
name: "exclusions from os -> binary",
relationship: artifact.Relationship{
Type: artifact.OwnershipByFileOverlapRelationship,
From: packageA,
To: packageC,
},
packages: pkg.NewCollection(packageA, packageC),
shouldExclude: true,
},
{
name: "no exclusions from python -> binary",
relationship: artifact.Relationship{
Type: artifact.OwnershipByFileOverlapRelationship,
From: packageB,
To: packageC,
},
packages: pkg.NewCollection(packageB, packageC),
shouldExclude: false,
},
{
name: "no exclusions for different package names",
relationship: artifact.Relationship{
Type: artifact.OwnershipByFileOverlapRelationship,
From: packageA,
To: packageD,
},
packages: pkg.NewCollection(packageA, packageD),
shouldExclude: false,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if !ExcludeBinaryByFileOwnershipOverlap(test.relationship, test.packages) && test.shouldExclude {
t.Errorf("expected to exclude relationship %+v", test.relationship)
}
})
}
}
+13 -2
View File
@@ -6,10 +6,21 @@ import (
const eggInfoGlob = "**/*.egg-info"
type CatalogerConfig struct {
GuessUnpinnedRequirements bool
}
func DefaultCatalogerConfig() CatalogerConfig {
return CatalogerConfig{
GuessUnpinnedRequirements: false,
}
}
// NewPythonIndexCataloger returns a new cataloger for python packages referenced from poetry lock files, requirements.txt files, and setup.py files.
func NewPythonIndexCataloger() *generic.Cataloger {
func NewPythonIndexCataloger(cfg CatalogerConfig) *generic.Cataloger {
rqp := newRequirementsParser(cfg)
return generic.NewCataloger("python-index-cataloger").
WithParserByGlobs(parseRequirementsTxt, "**/*requirements*.txt").
WithParserByGlobs(rqp.parseRequirementsTxt, "**/*requirements*.txt").
WithParserByGlobs(parsePoetryLock, "**/poetry.lock").
WithParserByGlobs(parsePipfileLock, "**/Pipfile.lock").
WithParserByGlobs(parseSetup, "**/setup.py")
+1 -1
View File
@@ -263,7 +263,7 @@ func Test_IndexCataloger_Globs(t *testing.T) {
pkgtest.NewCatalogTester().
FromDirectory(t, test.fixture).
ExpectsResolverContentQueries(test.expected).
TestCataloger(t, NewPythonIndexCataloger())
TestCataloger(t, NewPythonIndexCataloger(DefaultCatalogerConfig()))
})
}
}
+173 -99
View File
@@ -7,6 +7,10 @@ import (
"strings"
"unicode"
pep440 "github.com/aquasecurity/go-pep440-version"
"github.com/mitchellh/mapstructure"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
@@ -14,23 +18,99 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
var _ generic.Parser = parseRequirementsTxt
const (
// given the example requirement:
// requests[security] == 2.8.* ; python_version < "2.7" and sys_platform == "linux" \
// --hash=sha256:a9b3aaa1904eeb78e32394cd46c6f37ac0fb4af6dc488daa58971bdc7d7fcaf3 \
// --hash=sha256:e9535b8c84dc9571a48999094fda7f33e63c3f1b74f3e5f3ac0105a58405bb65 # some comment
var (
extrasRegex = regexp.MustCompile(`\[.*\]`)
urlRegex = regexp.MustCompile("@.*git.*")
// namePattern matches: requests[security]
namePattern = `(?P<name>\w[\w\[\],\s-_]+)`
// versionConstraintPattern matches: == 2.8.*
versionConstraintPattern = `(?P<versionConstraint>([^\S\r\n]*[~=>!<]+\s*[0-9a-zA-Z.*]+[^\S\r\n]*,?)+)?(@[^\S\r\n]*(?P<url>[^;]*))?`
// markersPattern matches: python_version < "2.7" and sys_platform == "linux"
markersPattern = `(;(?P<markers>.*))?`
// hashesPattern matches: --hash=sha256:a9b3aaa1904eeb78e32394cd46c6f37ac0fb4af6dc488daa58971bdc7d7fcaf3 --hash=sha256:e9535b8c84dc9571a48999094fda7f33e63c3f1b74f3e5f3ac0105a58405bb65
hashesPattern = `(?P<hashes>([^\S\r\n]*--hash=[a-zA-Z0-9:]+)+)?`
// whiteSpaceNoNewlinePattern matches: (any whitespace character except for \r and \n)
whiteSpaceNoNewlinePattern = `[^\S\r\n]*`
)
var requirementPattern = regexp.MustCompile(
`^` +
whiteSpaceNoNewlinePattern +
namePattern +
whiteSpaceNoNewlinePattern +
versionConstraintPattern +
markersPattern +
hashesPattern,
)
type unprocessedRequirement struct {
Name string `mapstructure:"name"`
VersionConstraint string `mapstructure:"versionConstraint"`
Markers string `mapstructure:"markers"`
URL string `mapstructure:"url"`
Hashes string `mapstructure:"hashes"`
}
func newRequirement(raw string) *unprocessedRequirement {
var r unprocessedRequirement
values := internal.MatchNamedCaptureGroups(requirementPattern, raw)
if err := mapstructure.Decode(values, &r); err != nil {
return nil
}
r.Name = strings.TrimSpace(r.Name)
r.VersionConstraint = strings.TrimSpace(r.VersionConstraint)
r.Markers = strings.TrimSpace(r.Markers)
r.URL = strings.TrimSpace(r.URL)
r.Hashes = strings.TrimSpace(r.Hashes)
if r.Name == "" {
return nil
}
return &r
}
type requirementsParser struct {
guessUnpinnedRequirements bool
}
func newRequirementsParser(cfg CatalogerConfig) requirementsParser {
return requirementsParser{
guessUnpinnedRequirements: cfg.GuessUnpinnedRequirements,
}
}
// parseRequirementsTxt takes a Python requirements.txt file, returning all Python packages that are locked to a
// specific version.
func parseRequirementsTxt(_ file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
func (rp requirementsParser) parseRequirementsTxt(_ file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
var packages []pkg.Package
scanner := bufio.NewScanner(reader)
var lastLine string
for scanner.Scan() {
line := scanner.Text()
rawLineNoComments := removeTrailingComment(line)
line = trimRequirementsTxtLine(line)
line := trimRequirementsTxtLine(scanner.Text())
if lastLine != "" {
line = lastLine + line
lastLine = ""
}
// remove line continuations... smashes the file into a single line
if strings.HasSuffix(line, "\\") {
// this line is a continuation of the previous line
lastLine += strings.TrimSuffix(line, "\\")
continue
}
if line == "" {
// nothing to parse on this line
@@ -42,35 +122,20 @@ func parseRequirementsTxt(_ file.Resolver, _ *generic.Environment, reader file.L
continue
}
if !strings.Contains(line, "==") {
// a package without a version, or a range (unpinned) which does not tell us
// exactly what will be installed.
continue
}
// parse a new requirement
parts := strings.Split(line, "==")
if len(parts) < 2 {
// this should never happen, but just in case
req := newRequirement(line)
if req == nil {
log.WithFields("path", reader.RealPath).Warnf("unable to parse requirements.txt line: %q", line)
continue
}
// check if the version contains hash declarations on the same line
version, _ := parseVersionAndHashes(parts[1])
name := removeExtras(req.Name)
version := parseVersion(req.VersionConstraint, rp.guessUnpinnedRequirements)
name := strings.TrimSpace(parts[0])
version = strings.TrimFunc(version, func(r rune) bool {
return !unicode.IsLetter(r) && !unicode.IsNumber(r)
})
// TODO: Update to support more than only ==
versionConstraint := fmt.Sprintf("== %s", version)
if name == "" || version == "" {
log.WithFields("path", reader.RealPath).Debugf("found empty package in requirements.txt line: %q", line)
if version == "" {
log.WithFields("path", reader.RealPath).Tracef("unable to determine package version in requirements.txt line: %q", line)
continue
}
packages = append(
packages,
newPackageForRequirementsWithMetadata(
@@ -78,10 +143,10 @@ func parseRequirementsTxt(_ file.Resolver, _ *generic.Environment, reader file.L
version,
pkg.PythonRequirementsMetadata{
Name: name,
Extras: parseExtras(rawLineNoComments),
VersionConstraint: versionConstraint,
URL: parseURL(rawLineNoComments),
Markers: parseMarkers(rawLineNoComments),
Extras: parseExtras(req.Name),
VersionConstraint: req.VersionConstraint,
URL: parseURL(req.URL),
Markers: req.Markers,
},
reader.Location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
),
@@ -95,13 +160,68 @@ func parseRequirementsTxt(_ file.Resolver, _ *generic.Environment, reader file.L
return packages, nil, nil
}
func parseVersionAndHashes(version string) (string, []string) {
parts := strings.Split(version, "--hash=")
if len(parts) < 2 {
return version, nil
func parseVersion(version string, guessFromConstraint bool) string {
if isPinnedConstraint(version) {
return strings.TrimSpace(strings.ReplaceAll(version, "==", ""))
}
return parts[0], parts[1:]
if guessFromConstraint {
return guessVersion(version)
}
return ""
}
func isPinnedConstraint(version string) bool {
return strings.Contains(version, "==") && !strings.ContainsAny(version, "*,<>!")
}
func guessVersion(constraint string) string {
// handle "2.8.*" -> "2.8.0"
constraint = strings.ReplaceAll(constraint, "*", "0")
if isPinnedConstraint(constraint) {
return strings.TrimSpace(strings.ReplaceAll(constraint, "==", ""))
}
constraints := strings.Split(constraint, ",")
filteredVersions := map[string]struct{}{}
for _, part := range constraints {
if strings.Contains(part, "!=") {
parts := strings.Split(part, "!=")
filteredVersions[strings.TrimSpace(parts[1])] = struct{}{}
}
}
var closestVersion *pep440.Version
for _, part := range constraints {
// ignore any parts that do not have '=' in them, >,<,~ are not valid semver
parts := strings.SplitAfter(part, "=")
if len(parts) < 2 {
continue
}
version, err := pep440.Parse(strings.TrimSpace(parts[1]))
if err != nil {
// ignore any parts that are not valid semver
continue
}
if _, ok := filteredVersions[version.String()]; ok {
continue
}
if strings.Contains(part, "==") {
parts := strings.Split(part, "==")
return strings.TrimSpace(parts[1])
}
if closestVersion == nil || version.GreaterThan(*closestVersion) {
closestVersion = &version
}
}
if closestVersion == nil {
return ""
}
return closestVersion.String()
}
// trimRequirementsTxtLine removes content from the given requirements.txt line
@@ -109,8 +229,6 @@ func parseVersionAndHashes(version string) (string, []string) {
func trimRequirementsTxtLine(line string) string {
line = strings.TrimSpace(line)
line = removeTrailingComment(line)
line = removeEnvironmentMarkers(line)
line = checkForRegex(line) // remove extras and url from line if found
return line
}
@@ -127,57 +245,29 @@ func removeTrailingComment(line string) string {
return parts[0]
}
// removeEnvironmentMarkers removes any instances of environment markers (delimited by ';') from the line.
// For more information, see https://www.python.org/dev/peps/pep-0508/#environment-markers.
func removeEnvironmentMarkers(line string) string {
parts := strings.SplitN(line, ";", 2)
if len(parts) < 2 {
// there aren't any environment markers
return line
func removeExtras(packageName string) string {
start := strings.Index(packageName, "[")
if start == -1 {
return packageName
}
return parts[0]
return strings.TrimSpace(packageName[:start])
}
func parseExtras(packageName string) []string {
if extrasRegex.MatchString(packageName) {
// Remove square brackets
extras := strings.TrimFunc(extrasRegex.FindString(packageName), func(r rune) bool {
return !unicode.IsLetter(r) && !unicode.IsNumber(r)
})
var extras []string
// Remove any additional whitespace
extras = strings.ReplaceAll(extras, " ", "")
return strings.Split(extras, ",")
start := strings.Index(packageName, "[")
stop := strings.Index(packageName, "]")
if start == -1 || stop == -1 {
return extras
}
return []string{}
}
func parseMarkers(line string) map[string]string {
markers := map[string]string{}
parts := strings.SplitN(line, ";", 2)
if len(parts) == 2 {
splittableMarkers := parts[1]
for _, combineString := range []string{" or ", " and "} {
splittableMarkers = strings.TrimSpace(
strings.ReplaceAll(splittableMarkers, combineString, ","),
)
}
splittableMarkers = strings.TrimSpace(splittableMarkers)
for _, mark := range strings.Split(splittableMarkers, ",") {
markparts := strings.Split(mark, " ")
markers[markparts[0]] = strings.Join(markparts[1:], " ")
}
extraString := packageName[start+1 : stop]
for _, extra := range strings.Split(extraString, ",") {
extras = append(extras, strings.TrimSpace(extra))
}
return markers
return extras
}
func parseURL(line string) string {
@@ -204,19 +294,3 @@ func parseURL(line string) string {
return ""
}
// function to check a string for all possilbe regex expressions, replacing it if found
func checkForRegex(stringToCheck string) string {
stringToReturn := stringToCheck
for _, r := range []*regexp.Regexp{
urlRegex,
extrasRegex,
} {
if r.MatchString(stringToCheck) {
stringToReturn = r.ReplaceAllString(stringToCheck, "")
}
}
return stringToReturn
}
@@ -3,6 +3,8 @@ package python
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
@@ -12,7 +14,8 @@ import (
func TestParseRequirementsTxt(t *testing.T) {
fixture := "test-fixtures/requires/requirements.txt"
locations := file.NewLocationSet(file.NewLocation(fixture))
expectedPkgs := []pkg.Package{
pinnedPkgs := []pkg.Package{
{
Name: "flask",
Version: "4.0.0",
@@ -23,10 +26,7 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "flask",
Extras: []string{},
VersionConstraint: "== 4.0.0",
URL: "",
Markers: map[string]string{},
},
},
{
@@ -39,10 +39,7 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "foo",
Extras: []string{},
VersionConstraint: "== 1.0.0",
URL: "",
Markers: map[string]string{},
},
},
{
@@ -55,10 +52,8 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "SomeProject",
Extras: []string{},
VersionConstraint: "== 5.4",
URL: "",
Markers: map[string]string{"python_version": "< '3.8'"},
VersionConstraint: "==5.4",
Markers: "python_version < '3.8'",
},
},
{
@@ -71,10 +66,7 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "argh",
Extras: []string{},
VersionConstraint: "== 0.26.2",
URL: "",
Markers: map[string]string{},
VersionConstraint: "==0.26.2",
},
},
{
@@ -87,10 +79,7 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "argh",
Extras: []string{},
VersionConstraint: "== 0.26.3",
URL: "",
Markers: map[string]string{},
VersionConstraint: "==0.26.3",
},
},
{
@@ -105,27 +94,6 @@ func TestParseRequirementsTxt(t *testing.T) {
Name: "celery",
Extras: []string{"redis", "pytest"},
VersionConstraint: "== 4.4.7",
URL: "",
Markers: map[string]string{},
},
},
{
Name: "requests",
Version: "2.8",
PURL: "pkg:pypi/requests@2.8",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "requests",
Extras: []string{"security"},
VersionConstraint: "== 2.8",
URL: "",
Markers: map[string]string{
"python_version": `< "2.7"`,
"sys_platform": `== "linux"`,
},
},
},
{
@@ -138,15 +106,238 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "GithubSampleProject",
Extras: []string{},
VersionConstraint: "== 3.7.1",
URL: "git+https://github.com/owner/repo@releases/tag/v3.7.1",
Markers: map[string]string{},
},
},
}
var expectedRelationships []artifact.Relationship
var testCases = []struct {
name string
fixture string
cfg CatalogerConfig
expectedPkgs []pkg.Package
expectedRelationships []artifact.Relationship
}{
{
name: "pinned dependencies only",
fixture: fixture,
cfg: CatalogerConfig{
GuessUnpinnedRequirements: false,
},
expectedPkgs: pinnedPkgs,
},
{
name: "guess unpinned requirements (lowest version)",
fixture: fixture,
cfg: CatalogerConfig{
GuessUnpinnedRequirements: true,
},
expectedPkgs: append([]pkg.Package{
{
Name: "Mopidy-Dirble",
Version: "1.1",
PURL: "pkg:pypi/Mopidy-Dirble@1.1",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "Mopidy-Dirble",
VersionConstraint: "~= 1.1",
},
},
{
Name: "sqlalchemy",
Version: "2.0.0",
PURL: "pkg:pypi/sqlalchemy@2.0.0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "sqlalchemy",
VersionConstraint: ">= 1.0.0, <= 2.0.0, != 3.0.0, <= 3.0.0",
},
},
{
Name: "bar",
Version: "2.0.0",
PURL: "pkg:pypi/bar@2.0.0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "bar",
VersionConstraint: ">= 1.0.0, <= 2.0.0, != 3.0.0, <= 3.0.0",
},
},
{
Name: "numpy",
Version: "3.4.1",
PURL: "pkg:pypi/numpy@3.4.1",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "numpy",
VersionConstraint: ">= 3.4.1",
Markers: `sys_platform == 'win32'`,
},
},
{
Name: "requests",
Version: "2.8.0",
PURL: "pkg:pypi/requests@2.8.0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "requests",
Extras: []string{"security"},
VersionConstraint: "== 2.8.*",
Markers: `python_version < "2.7" and sys_platform == "linux"`,
},
},
}, pinnedPkgs...),
},
}
pkgtest.TestFileParser(t, fixture, parseRequirementsTxt, expectedPkgs, expectedRelationships)
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
parser := newRequirementsParser(tc.cfg)
pkgtest.TestFileParser(t, tc.fixture, parser.parseRequirementsTxt, tc.expectedPkgs, tc.expectedRelationships)
})
}
}
func Test_newRequirement(t *testing.T) {
tests := []struct {
name string
raw string
want *unprocessedRequirement
}{
{
name: "simple",
raw: "requests==2.8",
want: &unprocessedRequirement{
Name: "requests",
VersionConstraint: "==2.8",
},
},
{
name: "comment + constraint",
raw: "Mopidy-Dirble ~= 1.1 # Compatible release. Same as >= 1.1, == 1.*",
want: &unprocessedRequirement{
Name: "Mopidy-Dirble",
VersionConstraint: "~= 1.1",
},
},
{
name: "hashes",
raw: "argh==0.26.3 --hash=sha256:a9b3aaa1904eeb78e32394cd46c6f37ac0fb4af6dc488daa58971bdc7d7fcaf3 --hash=sha256:e9535b8c84dc9571a48999094fda7f33e63c3f1b74f3e5f3ac0105a58405bb65",
want: &unprocessedRequirement{
Name: "argh",
VersionConstraint: "==0.26.3",
Hashes: "--hash=sha256:a9b3aaa1904eeb78e32394cd46c6f37ac0fb4af6dc488daa58971bdc7d7fcaf3 --hash=sha256:e9535b8c84dc9571a48999094fda7f33e63c3f1b74f3e5f3ac0105a58405bb65",
},
},
{
name: "extras",
raw: "celery[redis, pytest] == 4.4.7 # should remove [redis, pytest]",
want: &unprocessedRequirement{
Name: "celery[redis, pytest]",
VersionConstraint: "== 4.4.7",
},
},
{
name: "url",
raw: "GithubSampleProject == 3.7.1 @ git+https://github.com/owner/repo@releases/tag/v3.7.1",
want: &unprocessedRequirement{
Name: "GithubSampleProject",
VersionConstraint: "== 3.7.1",
URL: "git+https://github.com/owner/repo@releases/tag/v3.7.1",
},
},
{
name: "markers",
raw: "numpy >= 3.4.1 ; sys_platform == 'win32'",
want: &unprocessedRequirement{
Name: "numpy",
VersionConstraint: ">= 3.4.1",
Markers: "sys_platform == 'win32'",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, newRequirement(tt.raw))
})
}
}
// checkout https://www.darius.page/pipdev/ for help here! (github.com/nok/pipdev)
func Test_parseVersion(t *testing.T) {
tests := []struct {
name string
version string
guess bool
want string
}{
{
name: "exact",
version: "1.0.0",
want: "", // we can only parse constraints, not assume that a single version is a pin
},
{
name: "exact constraint",
version: " == 1.0.0 ",
want: "1.0.0",
},
{
name: "resolve lowest, simple constraint",
version: " >= 1.0.0 ",
guess: true,
want: "1.0.0",
},
{
name: "resolve lowest, compound constraint",
version: " < 2.0.0, >= 1.0.0, != 1.1.0 ",
guess: true,
want: "1.0.0",
},
{
name: "resolve lowest, handle asterisk",
version: "==2.8.*",
guess: true,
want: "2.8.0",
},
{
name: "resolve lowest, handle exceptions",
version: " !=4.0.2,!=4.1.0,!=4.2.0,>=4.0.1,!=4.3.0,!=5.0.0,!=5.1.0,<6.0.0",
guess: true,
want: "4.0.1",
},
{
name: "resolve lowest, compatible version constraint",
version: "~=0.6.10", // equates to >=0.6.10, ==0.6.*
guess: true,
want: "0.6.10",
},
{
name: "resolve lowest, with character in version",
version: "~=1.2b,<=1.3a,!=1.1,!=1.2",
guess: true,
want: "1.3a0", // note: 1.3a == 1.3a0
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, parseVersion(tt.version, tt.guess))
})
}
}
@@ -1,7 +1,9 @@
flask == 4.0.0
# a line that is ignored
sqlalchemy >= 1.0.0
sqlalchemy >= 1.0.0, <= 2.0.0, != 3.0.0, <= 3.0.0
foo == 1.0.0 # a comment that needs to be ignored
bar >= 1.0.0, <= 2.0.0, \
!= 3.0.0, <= 3.0.0
-e https://github.com/pecan/pecan.git
-r other-requirements.txt
--requirements super-secretrequirements.txt
+7 -2
View File
@@ -11,6 +11,11 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
const (
dbCatalogerName = "rpm-db-cataloger"
fileCatalogerName = "rpm-file-cataloger"
)
// NewRpmDBCataloger returns a new RPM DB cataloger object.
func NewRpmDBCataloger() *generic.Cataloger {
// check if a sqlite driver is available
@@ -18,14 +23,14 @@ func NewRpmDBCataloger() *generic.Cataloger {
log.Warnf("sqlite driver is not available, newer RPM databases might not be cataloged")
}
return generic.NewCataloger("rpm-db-cataloger").
return generic.NewCataloger(dbCatalogerName).
WithParserByGlobs(parseRpmDB, pkg.RpmDBGlob).
WithParserByGlobs(parseRpmManifest, pkg.RpmManifestGlob)
}
// NewFileCataloger returns a new RPM file cataloger object.
func NewFileCataloger() *generic.Cataloger {
return generic.NewCataloger("rpm-file-cataloger").
return generic.NewCataloger(fileCatalogerName).
WithParserByGlobs(parseRpm, "**/*.rpm")
}
+3
View File
@@ -23,6 +23,9 @@ func Test_DBCataloger_Globs(t *testing.T) {
"var/lib/rpm/Packages.db",
"var/lib/rpm/rpmdb.sqlite",
"var/lib/rpmmanifest/container-manifest-2",
"usr/lib/sysimage/rpm/Packages",
"usr/lib/sysimage/rpm/Packages.db",
"usr/lib/sysimage/rpm/rpmdb.sqlite",
},
},
}
+6 -1
View File
@@ -1,5 +1,5 @@
/*
Package swift provides a concrete Cataloger implementation for Podfile.lock files.
Package swift provides a concrete Cataloger implementation for Podfile.lock and Package.resolved files.
*/
package swift
@@ -7,6 +7,11 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
func NewSwiftPackageManagerCataloger() *generic.Cataloger {
return generic.NewCataloger("spm-cataloger").
WithParserByGlobs(parsePackageResolved, "**/Package.resolved", "**/.package.resolved")
}
// NewCocoapodsCataloger returns a new Swift Cocoapods lock file cataloger object.
func NewCocoapodsCataloger() *generic.Cataloger {
return generic.NewCataloger("cocoapods-cataloger").

Some files were not shown because too many files have changed in this diff Show More