Compare commits

...
59 Commits
Author SHA1 Message Date
Christopher Angelo Phillips 7a005fec06 update to point at new script for new release (#551)
* update to point at local script release

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-15 13:26:34 -04:00
Dan Luhring c0f55866ec Announce upcoming webinar (#553)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-10-15 12:13:28 -04:00
Christopher Angelo Phillips 9ea6330ebc update split in py package_cataloger to cap at 2 (#549)
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-14 12:32:06 -04:00
Christopher Angelo Phillips 7d13fbf0d4 540 add additional go package metadata (#546)
* update with distinct golang metadata types

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-13 11:00:28 -04:00
Dan Luhring 10368f6fe8 Add Syft logo (#545)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-10-11 21:06:27 +00:00
Josh Bressers f2840029d1 Allow Syft to build with podman (#537)
* Don't check the Built-By flag

Signed-off-by: Josh Bressers <josh@bress.net>

* Remove alpine pinning to resolve conflict with main

Signed-off-by: Josh Bressers <josh@bress.net>
2021-10-11 12:39:34 -04:00
Christopher Angelo Phillips b25f5b664b (464) - remove mod and cargo from image cataloger (#539)
* remove mod and cargo from image cataloger

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update test error messages for clear failures

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-07 15:18:20 -04:00
Christopher Angelo PhillipsandAlex Goodman 3462e18af3 478 identify go binaries and extract mod information (#534)
* add query by MIME type to source.FileResolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* import stereoscope lib changes to find mime type

- add bin cataloger
- add bin parser
- add mime type go utils
- import new resolver

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* add go std library code to unpack bin

- keep them in their own (original) files
- add note for "this code was copied from"
- comment the lines the required changing

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-07 12:16:38 -04:00
Alex Goodman 05ac3f1eff split application configuration parsing to multiple methods (#532)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-06 09:48:53 -04:00
Alex Goodman a000a2926b update linter + fix whitespace (#536)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-06 13:46:55 +00:00
Alex Goodman 9189ed68df Add query by MIME type to source.FileResolver (#529)
* add query by MIME type to source.FileResolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* pull in stereoscope MIME type feature

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 20:50:53 -04:00
Alex Goodman ba1cd8d753 specify optional file.Digests instances to abbreviate json output (#531)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 15:35:28 -04:00
Alex Goodman dd2c178085 add --file option to readme (#533)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 15:34:32 -04:00
Alex Goodman 1b23a94015 Add option to output SBOM report to a file (#530)
* add output to file option

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* log errors on close of the report destination

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove file option from persistent args

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update file option comments and logging

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow for multiple UI fallback options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update UI select signatures + tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 14:47:24 -04:00
Christopher Angelo Phillips f47a6a88b1 (#495) Update documentNamespace uniqueness for spdx-json output (#528)
* add unique namespace identifier

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-05 13:10:49 -04:00
Dan Luhring 4b7217f052 Improve documentation (#525)
* Make installation methods more obvious

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Create linkable section headers

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add badge for joining Slack

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Document requirement for signed commits

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-10-04 09:29:27 -04:00
Christopher Angelo Phillips 5e4b668e5a 497 stable sorted CPE array (JSON and SPDX) (#522)
* add small sorting change to our specificity

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-01 15:31:17 -04:00
Dan Luhring 6480f06173 Update description for Slack link (#523)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-09-30 21:43:43 +00:00
Christopher Angelo Phillips 75aed5f3ec (#460) Extend license mapping for common SPDX license names (#509)
Fixes #460 
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-30 16:54:36 -04:00
Jamie Magee 270fbb7665 Add mariner to distro types (#520)
Signed-off-by: Jamie Magee <jamie.magee@gmail.com>
2021-09-30 13:20:15 -04:00
Alex GoodmanandChristopher Angelo Phillips 0395c47445 Add vendor + product known good CPE field values (#517)
* add better infrastructure around CPE candidate additions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add test cases for CPE candidate additions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* small ergonomic updates

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* clean java OSGI directives from group ID

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* unquote CPE candidate field values (double quotes only)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add remaining CPE vendor & product additions (+ tests)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-27 18:31:59 -04:00
Keith Zantow e5caba043d Add SBOM to releases (#500)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-09-24 20:21:23 -04:00
Dan Luhring d1ac476986 Add announcement for KubeCon meetup (#515)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-09-24 11:42:06 -04:00
Dan Luhring dd23d49986 Prevent invalid CPE field values (#514)
* Fix CPE set comparison mismatch

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add failing test to assert CPE generation excludes URLs

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add removeByCondition method to fieldCandidateSet

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Prevent invalid CPE values for products and vendors

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Introduce removeWhere and rename filter to condition

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Refactor fieldCandidateSet and condition logic

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Move CPE parsing filter to end of CPE generation

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-09-24 09:23:58 -04:00
Alex Goodman 6d4d083acc Filter out CPE product candidates that are asterisks (#513)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-22 20:55:03 -04:00
Dan Luhring 316d4341c8 Use Anchore fork of packageurl lib without replace directive (#512)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-09-22 15:04:09 -04:00
Christopher Angelo Phillips 3e8afc5274 update log file permissions to 0644 (#511)
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-21 10:34:10 -04:00
Christopher Angelo Phillips 93d00dc340 Populate Files and Relationship fields for spdx-json output (#507)
* update spdx22 Document model to include relationships field

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update document and relationship to match current JSON spec
https://github.com/spdx/spdx-spec/blob/development/v2.2.1/schemas/spdx-schema.json
https://github.com/spdx/spdx-spec/pull/528
https://github.com/spdx/spdx-spec/pull/528#issuecomment-904180177

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update File struct based on SPDX schema

Required fields:
[ "SPDXID", "fileName", "copyrightText", "licenseConcluded" ]
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-17 09:06:12 -04:00
Alex Goodman 9fe1da8ee6 update go-rpmdb and restruct gomod references (#506)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-14 14:51:19 -04:00
Christopher Angelo Phillips 95df7dd035 Add command examples to release documentation (#503)
* update release document with commands run
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* small edits
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-14 09:08:37 -04:00
Alex Goodman ddfc8e20c0 Revert "disable docker releases (workaround) (#493)" (#501)
This reverts commit 06dcd3261d.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-13 13:06:23 -04:00
Christopher Angelo Phillips 3f305aa6cd (#342) - remove strong distro check (#496)
* add Type conversion to remove strong distro type limit
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update signatures to be correct variable from os-release
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-09 13:05:22 -04:00
houdini91 2f99a35f51 Power user command support for directory scans (#467)
* Power-user directory source support
Signed-off-by: Mikey Strauss <mikey@scribe-security.com>

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Remove newline

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Shared filetree (#1)

* Shared directory resolver filetree

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* PR - change error ErrObserve to ErrPath

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* PR - share directory resolver
* Use pointer to source struct

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Fix Lint

Signed-off-by: houdini91 <mdstrauss91@gmail.com>
2021-09-08 09:18:53 -04:00
Alex Goodman 06dcd3261d disable docker releases (workaround) (#493)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-03 21:29:34 +00:00
Alex Goodman abbba3fc19 Modify CPE vendor candidate generation approach (#484)
* consider additional vendor candidates for ruby, python, rpm, npm, and java

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add java pom.xml processing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow for downstream transform control in cpe generation processing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate CPE generation logic to dedicated package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split java manifest groupID extraction into two tiers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* extract groupID from pom parent project during CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update java groupID processing tests to cover multi-tier approach

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix constructor names for cpe.fieldCandidate

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename helper function to startsWithTopLevelDomain

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add nil changes for java manifest sections

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update comment to reflect parsing maven files

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split out java description parsing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split out pom parent processing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify vendorsFromGroupIDs and associated tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify test type for vendorsFromGroupIDs

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* copy candidate varidations to new instances

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename CPE generation string util functions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add an explanation around fieldCandidate

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify type for the cpe.fieldCandidateSet

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* make CPE filter function names more readable

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update groupIDsFromJavaManifest to use a guard clause

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* extract groupID extraction from artifactID fields into a separate function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump goreleaser version to combat failure

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-03 14:21:25 -04:00
Samuel Dacanay 0799fd9d46 Add support for older CentOS versions (6 & 5) by checking additional release files for information
Signed-off-by: Samuel Dacanay <sam.dacanay@anchore.com>
2021-09-03 16:06:10 +01:00
Alex Goodman 5de1a0a236 Revert "Add announcement for upcoming OSS meetup (#490)" (#491)
This reverts commit 035ef03b0c.

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-02 10:59:51 -04:00
Dan Luhring 035ef03b0c Add announcement for upcoming OSS meetup (#490)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-08-25 14:35:32 -04:00
Toure Dunnon 9acabb8307 Merge pull request #489 from anchore/SLES_support_berkeleyDB
Updated the distro package to include SLES
2021-08-23 16:14:52 -04:00
Toure Dunnon 0f0fe59bf2 Updated the distro package to include SLES type to expand support, this change will not include the new sqlite3 rpmDB format
Signed-off-by: Toure Dunnon <toure.dunnon@anchore.com>
2021-08-23 15:05:39 -04:00
Keith Zantow ff828fbac2 Skip package-lock.json and yarn.lock in node_modules #431 (#485)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-08-20 13:50:28 -04:00
cba5b5723b Added parser for Pipfile.lock to cataloger (#473)
* Added parser for Pipfile.lock to cataloger

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* make lint-fix

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* Update syft/pkg/cataloger/python/parse_pipfile_lock.go

Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* fix _version

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* swap method for trimming "==" prefix from pipfile pkg versions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-19 10:32:10 -04:00
Alex Goodman b0e9b92a25 Merge pull request #483 from anchore/fix-gomod-gosum-validation
Fix go module tidy checks for release pipeline
2021-08-18 11:00:49 -04:00
Alex Goodman d25d3b1fb3 fix go module tidy checks for release pipeline
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-18 10:13:56 -04:00
Keith Zantow 93b4bf9379 Add option to enable http registry connections (#482)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-08-17 12:52:51 -04:00
Keith Zantow 58f2be95fd chore: update packageurl-go and add encoding tests for #351 (#481)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-08-13 15:39:52 -04:00
Alex Goodman 6f038e9208 Merge pull request #447 from bureado/main
Minimal changes to use ID_LIKE
2021-08-12 14:20:13 -04:00
Alex Goodman 32a54b94be add unit tests for distro identification based on "like" field
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-12 14:09:22 -04:00
Alex Goodman 98d4749f86 Enhance CPE generation (#472)
* adjust CPE specificity sorting to include field length and bias certain fields

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove * vendor values from CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* re-enable generating CPEs for jenkins and jira plugins

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* improve CPE generation logic based on java artifactID and groupID

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add ruby-lang as target software candidate for gems in CPE generation logic

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename filterCpes to filterCPEs

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* refactor CPE filters and groupID processing (for linting)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use ruby-lang as vendor candidate not target software

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address PR comments for CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-09 19:52:19 +00:00
Alex Goodman 3a5168917e Update SPDX license list from 3.13 to 3.14 (#474)
* update SPDX license list from 3.13 to 3.14

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove license list version from spdx snapshot unit tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-09 19:47:33 +00:00
Alex Goodman 58b21c03c3 Merge pull request #461 from anchore/update-slack
Update slack channels
2021-07-12 08:28:01 -04:00
Alex Goodman 2093d2a1dc update slack channels
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-07-12 08:04:04 -04:00
Alex Goodman dd56450f92 Revert "Add callout for OSS meetup (#446)" (#459)
This reverts commit 66e71c39e2.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2021-07-02 14:29:27 +00:00
Alex Goodman 6f2b52971d directory resolver indexer should report one progressable (#457)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-07-02 10:18:41 -04:00
Alex Goodman fee74d57b8 Update documentation to show SPDX support (#452)
* add docs for spdx support

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* copy updates for format options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-07-01 14:52:57 +00:00
Alex Goodman fee35dd175 redirect cursor hide/show to stderr (#456)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-30 13:10:00 -04:00
Alex Goodman ecf4e5546a add help message when no arguments are provided (#455)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-30 12:53:12 -04:00
Jose Miguel Parrella c833472b6e If distro name is unknown, try with the ID_LIKE
Signed-off-by: Jose Miguel Parrella <j@bureado.com>
2021-06-25 22:18:40 -07:00
Jose Miguel Parrella c825ae2404 Completing the test for distro version
Signed-off-by: Jose Miguel Parrella <j@bureado.com>
2021-06-25 22:18:40 -07:00
291 changed files with 32330 additions and 2270 deletions
+2 -2
View File
@@ -6,7 +6,7 @@ permit:
- ISC
ignore-packages:
# packageurl-go is released under the MIT license located in the root of the repo at /mit.LICENSE
- github.com/package-url/packageurl-go
- github.com/anchore/packageurl-go
# from: https://github.com/spdx/tools-golang/blob/main/LICENSE.code
# The tools-golang source code is provided and may be used, at your option,
@@ -14,4 +14,4 @@ ignore-packages:
# * Apache License, version 2.0 (Apache-2.0), OR
# * GNU General Public License, version 2.0 or later (GPL-2.0-or-later).
# (we choose Apache-2.0)
- github.com/spdx/tools-golang
- github.com/spdx/tools-golang
+1 -1
View File
@@ -3,4 +3,4 @@ contact_links:
- name: Join the Slack community 💬
# link to our community Slack registration page
url: https://anchore.com/slack
about: 'Be sure to check out the channels #toolbox-help (ask us questions) and #toolbox-dev (collaborate with us on design and development)!'
about: 'Come chat with us! Ask for help, join our software development efforts, or just give us feedback!'
+15 -11
View File
@@ -1,27 +1,31 @@
#!/usr/bin/env bash
set -u
set -eu
TMP_GOMOD=$(mktemp)
TMP_GOSUM=$(mktemp)
ORIGINAL_STATE_DIR=$(mktemp -d "TEMP-original-state-XXXXXXXXX")
TIDY_STATE_DIR=$(mktemp -d "TEMP-tidy-state-XXXXXXXXX")
trap "rm -f ${TMP_GOSUM} ${TMP_GOMOD}" EXIT
trap "cp -v ${ORIGINAL_STATE_DIR}/* ./ && rm -fR ${ORIGINAL_STATE_DIR} ${TIDY_STATE_DIR}" EXIT
cp go.mod "${TMP_GOMOD}"
cp go.sum "${TMP_GOSUM}"
echo "Capturing original state of files..."
cp -v go.mod go.sum "${ORIGINAL_STATE_DIR}"
echo "Capturing state of go.mod and go.sum after running go mod tidy..."
go mod tidy
cp -v go.mod go.sum "${TIDY_STATE_DIR}"
echo ""
DIFF_MOD=$(diff -u "${TMP_GOMOD}" go.mod)
DIFF_SUM=$(diff -u "${TMP_GOSUM}" go.sum)
set +e
cp "${TMP_GOMOD}" go.mod
cp "${TMP_GOSUM}" go.sum
# Detect difference between the git HEAD state and the go mod tidy state
DIFF_MOD=$(diff -u "${ORIGINAL_STATE_DIR}/go.mod" "${TIDY_STATE_DIR}/go.mod")
DIFF_SUM=$(diff -u "${ORIGINAL_STATE_DIR}/go.sum" "${TIDY_STATE_DIR}/go.sum")
if [[ -n "${DIFF_MOD}" || -n "${DIFF_SUM}" ]]; then
echo "go.mod and/or go.sum are not tidy; please run go mod tidy"
echo "go.mod diff:"
echo "${DIFF_MOD}"
echo "go.sum diff:"
echo "${DIFF_SUM}"
echo ""
printf "FAILED! go.mod and/or go.sum are NOT tidy; please run 'go mod tidy'.\n\n"
exit 1
fi
+405
View File
@@ -0,0 +1,405 @@
#!/bin/sh
set -e
# Code generated by godownloader on 2019-12-25T12:47:14Z. DO NOT EDIT.
#
usage() {
this=$1
cat <<EOF
$this: download go binaries for goreleaser/goreleaser
Usage: $this [-b] bindir [-d] [tag]
-b sets bindir or installation directory, Defaults to ./bin
-d turns on debug logging
[tag] is a tag from
https://github.com/goreleaser/goreleaser/releases
If tag is missing, then the latest will be used.
Generated by godownloader
https://github.com/goreleaser/godownloader
EOF
exit 2
}
parse_args() {
#BINDIR is ./bin unless set be ENV
# over-ridden by flag below
BINDIR=${BINDIR:-./bin}
while getopts "b:dh?x" arg; do
case "$arg" in
b) BINDIR="$OPTARG" ;;
d) log_set_priority 10 ;;
h | \?) usage "$0" ;;
x) set -x ;;
esac
done
shift $((OPTIND - 1))
TAG=$1
}
# this function wraps all the destructive operations
# if a curl|bash cuts off the end of the script due to
# network, either nothing will happen or will syntax error
# out preventing half-done work
execute() {
tmpdir=$(mktemp -d)
log_debug "downloading files into ${tmpdir}"
http_download "${tmpdir}/${TARBALL}" "${TARBALL_URL}"
http_download "${tmpdir}/${CHECKSUM}" "${CHECKSUM_URL}"
hash_sha256_verify "${tmpdir}/${TARBALL}" "${tmpdir}/${CHECKSUM}"
srcdir="${tmpdir}"
(cd "${tmpdir}" && untar "${TARBALL}")
test ! -d "${BINDIR}" && install -d "${BINDIR}"
for binexe in $BINARIES; do
if [ "$OS" = "windows" ]; then
binexe="${binexe}.exe"
fi
install "${srcdir}/${binexe}" "${BINDIR}/"
log_info "installed ${BINDIR}/${binexe}"
done
rm -rf "${tmpdir}"
}
get_binaries() {
case "$PLATFORM" in
darwin/all) BINARIES="goreleaser" ;;
darwin/386) BINARIES="goreleaser" ;;
darwin/amd64) BINARIES="goreleaser" ;;
darwin/arm64) BINARIES="goreleaser" ;;
darwin/armv6) BINARIES="goreleaser" ;;
linux/386) BINARIES="goreleaser" ;;
linux/amd64) BINARIES="goreleaser" ;;
linux/arm64) BINARIES="goreleaser" ;;
linux/armv6) BINARIES="goreleaser" ;;
windows/386) BINARIES="goreleaser" ;;
windows/amd64) BINARIES="goreleaser" ;;
windows/arm64) BINARIES="goreleaser" ;;
windows/armv6) BINARIES="goreleaser" ;;
*)
log_crit "platform $PLATFORM is not supported. Make sure this script is up-to-date and file request at https://github.com/${PREFIX}/issues/new"
exit 1
;;
esac
}
tag_to_version() {
if [ -z "${TAG}" ]; then
log_info "checking GitHub for latest tag"
else
log_info "checking GitHub for tag '${TAG}'"
fi
REALTAG=$(github_release "$OWNER/$REPO" "${TAG}") && true
if test -z "$REALTAG"; then
log_crit "unable to find '${TAG}' - use 'latest' or see https://github.com/${PREFIX}/releases for details"
exit 1
fi
# if version starts with 'v', remove it
TAG="$REALTAG"
VERSION=${TAG#v}
}
adjust_format() {
# change format (tar.gz or zip) based on OS
case ${OS} in
windows) FORMAT=zip ;;
esac
true
}
adjust_os() {
# adjust archive name based on OS
case ${OS} in
386) OS=i386 ;;
amd64) OS=x86_64 ;;
darwin) OS=Darwin ;;
linux) OS=Linux ;;
windows) OS=Windows ;;
esac
true
}
adjust_arch() {
# adjust archive name based on ARCH
case ${ARCH} in
386) ARCH=i386 ;;
amd64) ARCH=x86_64 ;;
darwin) ARCH=Darwin ;;
linux) ARCH=Linux ;;
windows) ARCH=Windows ;;
esac
true
}
cat /dev/null <<EOF
------------------------------------------------------------------------
https://github.com/client9/shlib - portable posix shell functions
Public domain - http://unlicense.org
https://github.com/client9/shlib/blob/master/LICENSE.md
but credit (and pull requests) appreciated.
------------------------------------------------------------------------
EOF
is_command() {
command -v "$1" >/dev/null
}
echoerr() {
echo "$@" 1>&2
}
log_prefix() {
echo "$0"
}
_logp=6
log_set_priority() {
_logp="$1"
}
log_priority() {
if test -z "$1"; then
echo "$_logp"
return
fi
[ "$1" -le "$_logp" ]
}
log_tag() {
case $1 in
0) echo "emerg" ;;
1) echo "alert" ;;
2) echo "crit" ;;
3) echo "err" ;;
4) echo "warning" ;;
5) echo "notice" ;;
6) echo "info" ;;
7) echo "debug" ;;
*) echo "$1" ;;
esac
}
log_debug() {
log_priority 7 || return 0
echoerr "$(log_prefix)" "$(log_tag 7)" "$@"
}
log_info() {
log_priority 6 || return 0
echoerr "$(log_prefix)" "$(log_tag 6)" "$@"
}
log_err() {
log_priority 3 || return 0
echoerr "$(log_prefix)" "$(log_tag 3)" "$@"
}
log_crit() {
log_priority 2 || return 0
echoerr "$(log_prefix)" "$(log_tag 2)" "$@"
}
uname_os() {
os=$(uname -s | tr '[:upper:]' '[:lower:]')
case "$os" in
cygwin_nt*) os="windows" ;;
mingw*) os="windows" ;;
msys_nt*) os="windows" ;;
esac
echo "$os"
}
uname_arch() {
arch=$(uname -m)
case $arch in
x86_64) arch="amd64" ;;
x86) arch="386" ;;
i686) arch="386" ;;
i386) arch="386" ;;
aarch64) arch="arm64" ;;
armv5*) arch="armv5" ;;
armv6*) arch="armv6" ;;
armv7*) arch="armv7" ;;
esac
os=$(uname_os)
case "$os" in
darwin) arch="all" ;;
esac
echo ${arch}
}
uname_os_check() {
os=$(uname_os)
case "$os" in
darwin) return 0 ;;
dragonfly) return 0 ;;
freebsd) return 0 ;;
linux) return 0 ;;
android) return 0 ;;
nacl) return 0 ;;
netbsd) return 0 ;;
openbsd) return 0 ;;
plan9) return 0 ;;
solaris) return 0 ;;
windows) return 0 ;;
all) return 0 ;;
esac
log_crit "uname_os_check '$(uname -s)' got converted to '$os' which is not a GOOS value. Please file bug at https://github.com/client9/shlib"
return 1
}
uname_arch_check() {
arch=$(uname_arch)
case "$arch" in
386) return 0 ;;
all) return 0 ;;
amd64) return 0 ;;
arm64) return 0 ;;
armv5) return 0 ;;
armv6) return 0 ;;
armv7) return 0 ;;
ppc64) return 0 ;;
ppc64le) return 0 ;;
mips) return 0 ;;
mipsle) return 0 ;;
mips64) return 0 ;;
mips64le) return 0 ;;
s390x) return 0 ;;
amd64p32) return 0 ;;
esac
log_crit "uname_arch_check '$(uname -m)' got converted to '$arch' which is not a GOARCH value. Please file bug report at https://github.com/client9/shlib"
return 1
}
untar() {
tarball=$1
case "${tarball}" in
*.tar.gz | *.tgz) tar --no-same-owner -xzf "${tarball}" ;;
*.tar) tar --no-same-owner -xf "${tarball}" ;;
*.zip) unzip "${tarball}" ;;
*)
log_err "untar unknown archive format for ${tarball}"
return 1
;;
esac
}
http_download_curl() {
local_file=$1
source_url=$2
header=$3
if [ -z "$header" ]; then
code=$(curl -w '%{http_code}' -sL -o "$local_file" "$source_url")
else
code=$(curl -w '%{http_code}' -sL -H "$header" -o "$local_file" "$source_url")
fi
if [ "$code" != "200" ]; then
log_debug "http_download_curl received HTTP status $code"
return 1
fi
return 0
}
http_download_wget() {
local_file=$1
source_url=$2
header=$3
if [ -z "$header" ]; then
wget -q -O "$local_file" "$source_url"
else
wget -q --header "$header" -O "$local_file" "$source_url"
fi
}
http_download() {
log_debug "http_download $2"
if is_command curl; then
http_download_curl "$@"
return
elif is_command wget; then
http_download_wget "$@"
return
fi
log_crit "http_download unable to find wget or curl"
return 1
}
http_copy() {
tmp=$(mktemp)
http_download "${tmp}" "$1" "$2" || return 1
body=$(cat "$tmp")
rm -f "${tmp}"
echo "$body"
}
github_release() {
owner_repo=$1
version=$2
test -z "$version" && version="latest"
giturl="https://github.com/${owner_repo}/releases/${version}"
json=$(http_copy "$giturl" "Accept:application/json")
test -z "$json" && return 1
version=$(echo "$json" | tr -s '\n' ' ' | sed 's/.*"tag_name":"//' | sed 's/".*//')
test -z "$version" && return 1
echo "$version"
}
hash_sha256() {
TARGET=${1:-/dev/stdin}
if is_command gsha256sum; then
hash=$(gsha256sum "$TARGET") || return 1
echo "$hash" | cut -d ' ' -f 1
elif is_command sha256sum; then
hash=$(sha256sum "$TARGET") || return 1
echo "$hash" | cut -d ' ' -f 1
elif is_command shasum; then
hash=$(shasum -a 256 "$TARGET" 2>/dev/null) || return 1
echo "$hash" | cut -d ' ' -f 1
elif is_command openssl; then
hash=$(openssl -dst openssl dgst -sha256 "$TARGET") || return 1
echo "$hash" | cut -d ' ' -f a
else
log_crit "hash_sha256 unable to find command to compute sha-256 hash"
return 1
fi
}
hash_sha256_verify() {
TARGET=$1
checksums=$2
if [ -z "$checksums" ]; then
log_err "hash_sha256_verify checksum file not specified in arg2"
return 1
fi
BASENAME=${TARGET##*/}
want=$(grep "${BASENAME}" "${checksums}" 2>/dev/null | tr '\t' ' ' | cut -d ' ' -f 1)
if [ -z "$want" ]; then
log_err "hash_sha256_verify unable to find checksum for '${TARGET}' in '${checksums}'"
return 1
fi
got=$(hash_sha256 "$TARGET")
if [ "$want" != "$got" ]; then
log_err "hash_sha256_verify checksum for '$TARGET' did not verify ${want} vs $got"
return 1
fi
}
cat /dev/null <<EOF
------------------------------------------------------------------------
End of functions from https://github.com/client9/shlib
------------------------------------------------------------------------
EOF
PROJECT_NAME="goreleaser"
OWNER=goreleaser
REPO="goreleaser"
BINARY=goreleaser
FORMAT=tar.gz
OS=$(uname_os)
ARCH=$(uname_arch)
PREFIX="$OWNER/$REPO"
# use in logging routines
log_prefix() {
echo "$PREFIX"
}
PLATFORM="${OS}/${ARCH}"
GITHUB_DOWNLOAD=https://github.com/${OWNER}/${REPO}/releases/download
uname_os_check "$OS"
uname_arch_check "$ARCH"
parse_args "$@"
get_binaries
tag_to_version
adjust_format
adjust_os
adjust_arch
log_info "found version: ${VERSION} for ${TAG}/${OS}/${ARCH}"
NAME=${PROJECT_NAME}_${OS}_${ARCH}
TARBALL=${NAME}.${FORMAT}
TARBALL_URL=${GITHUB_DOWNLOAD}/${TAG}/${TARBALL}
CHECKSUM=checksums.txt
CHECKSUM_URL=${GITHUB_DOWNLOAD}/${TAG}/${CHECKSUM}
execute
log_err "this script is deprecated, please do not use it anymore. check https://github.com/goreleaser/godownloader/issues/207"
+4
View File
@@ -158,6 +158,10 @@ jobs:
AC_USERNAME: ${{ secrets.ENG_CI_APPLE_ID }} # Used during macOS notarization.
AC_PASSWORD: ${{ secrets.ENG_CI_APPLE_ID_PASS }} # Used during macOS notarization.
- uses: anchore/sbom-action@v0
with:
artifact-name: sbom.spdx.json
- uses: 8398a7/action-slack@v3
with:
status: ${{ job.status }}
+5 -3
View File
@@ -15,6 +15,7 @@ linters:
- dogsled
- dupl
- errcheck
- exportloopref
- funlen
- gocognit
- goconst
@@ -22,7 +23,6 @@ linters:
- gocyclo
- gofmt
- goimports
- golint
- goprintffuncname
- gosec
- gosimple
@@ -31,8 +31,8 @@ linters:
- misspell
- nakedret
- nolintlint
- revive
- rowserrcheck
- scopelint
- staticcheck
- structcheck
- stylecheck
@@ -49,11 +49,13 @@ linters:
# - godot
# - godox
# - goerr113
# - golint # deprecated
# - gomnd # this is too aggressive
# - interfacer # this is a good idea, but is no longer supported and is prone to false positives
# - lll # without a way to specify per-line exception cases, this is not usable
# - maligned # this is an excellent linter, but tricky to optimize and we are not sensitive to memory layout optimizations
# - nestif
# - prealloc # following this rule isn't consistently a good idea, as it sometimes forces unnecessary allocations that result in less idiomatic code
# - scopelint # deprecated
# - testpackage
# - wsl
# - wsl # this doens't have an auto-fixer yet and is pretty noisy (https://github.com/bombsimon/wsl/issues/90)
+11 -6
View File
@@ -1,12 +1,12 @@
# Contributing to `syft`
# Contributing to Syft
If you are looking to contribute to this project and want to open a Github pull request ("PR"), there are a few guidelines of what we are looking for in patches. Make sure you go through this document and ensure that your code proposal is aligned.
If you are looking to contribute to this project and want to open a GitHub pull request ("PR"), there are a few guidelines of what we are looking for in patches. Make sure you go through this document and ensure that your code proposal is aligned.
## Sign your work
## Sign off your work
The `sign-off` is an added line at the end of the explanation for the commit, certifying that you wrote it or otherwise have the right to submit it as an open-source patch. By submitting a contribution, you agree to be bound by the terms of the DCO Version 1.1 and Apache License Version 2.0.
Signing a commit certifies the below Developer's Certificate of Origin (DCO):
Signing off a commit certifies the below Developer's Certificate of Origin (DCO):
```text
Developer's Certificate of Origin 1.1
@@ -51,13 +51,13 @@ $ git config --global user.name "John Doe"
$ git config --global user.email "john.doe@example.com"
```
Creating a signed commit is then possible with `-s`:
Creating a signed-off commit is then possible with `-s` or `--signoff`:
```text
$ git commit -s -m "this is a commit message"
```
To double check if the commit was signed, look at the log output:
To double-check that the commit was signed-off, look at the log output:
```text
$ git log -1
@@ -82,6 +82,11 @@ Date: Mon Aug 1 11:27:13 2020 -0400
[//]: # (TODO: PR title and description)
## Sign your commits
To ensure the authenticity and integrity of code contributions, **we require that all commits are signed**. Signing commits proves that your commits were truly created by you, as the holder of a private key.
Configuring git to sign your commits is a straightforward process. To get set up, see [GitHub's documentation on signing commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits).
## Test your changes
+2 -3
View File
@@ -101,14 +101,13 @@ $(TEMPDIR):
.PHONY: bootstrap-tools
bootstrap-tools: $(TEMPDIR)
GO111MODULE=off GOBIN=$(shell realpath $(TEMPDIR)) go get -u golang.org/x/perf/cmd/benchstat
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ v1.26.0
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ v1.42.1
curl -sSfL https://raw.githubusercontent.com/wagoodman/go-bouncer/master/bouncer.sh | sh -s -- -b $(TEMPDIR)/ v0.2.0
curl -sfL https://install.goreleaser.com/github.com/goreleaser/goreleaser.sh | sh -s -- -b $(TEMPDIR)/ v0.160.0
.github/scripts/goreleaser-install.sh -b $(TEMPDIR)/ v0.182.1
.PHONY: bootstrap-go
bootstrap-go:
go mod download
go mod tidy # note: it is important that the go.sum is kept in a consistent state at all times (especially during release)
.PHONY: bootstrap
bootstrap: $(RESULTSDIR) bootstrap-go bootstrap-tools ## Download and install all go dependencies (+ prep tooling in the ./tmp dir)
+45 -36
View File
@@ -1,33 +1,23 @@
# syft
<p align="center">
<img src="https://user-images.githubusercontent.com/5199289/136844524-1527b09f-c5cb-4aa9-be54-5aa92a6086c1.png" width="271">
</p>
[![Validations](https://github.com/anchore/syft/actions/workflows/validations.yaml/badge.svg)](https://github.com/anchore/syft/actions/workflows/validations.yaml)
[![Go Report Card](https://goreportcard.com/badge/github.com/anchore/syft)](https://goreportcard.com/report/github.com/anchore/syft)
[![GitHub release](https://img.shields.io/github/release/anchore/syft.svg)](https://github.com/anchore/syft/releases/latest)
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/anchore/syft/blob/main/LICENSE)
[![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/anchore/syft.svg)](https://github.com/anchore/syft)
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/anchore/syft/blob/main/LICENSE)
[![Slack Invite](https://img.shields.io/badge/Slack-Join-blue?logo=slack)](https://anchore.com/slack)
A CLI tool and go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner tool like [Grype](https://github.com/anchore/grype).
## Happening soon: [OSS Virtual Meetup](https://get.anchore.com/anchore-oss-meetup-jun-30-2021/)
[**Register here!**](https://get.anchore.com/anchore-oss-meetup-jun-30-2021/)
**When:** June 30, 11am-noon PT
**What:** 3 fast-paced talks, with time for Q&A
- **Using Syft to Streamline Compliance** (Chet Burgess — Principal Engineer, Cisco)
- **Fast Container Scanning in CI: Three Ways to Use Grype in Your Builds** (Dan Luhring — Manager of Open Source Engineering, Anchore)
- **Building More Optimal Container Images with Dive** (Alex Goodman — Creator of [Dive](https://github.com/wagoodman/dive), Sr Software Engineer for Open Source, Anchore)
**Announcement:** Join us on October 20, 11am-noon PT for a webinar where we’ll discuss Anchore’s open source and enterprise solutions. [Save your seat!](https://get.anchore.com/anchore-open-source-vs-enterprise/)
---
![syft-demo](https://user-images.githubusercontent.com/590471/90277200-2a253000-de33-11ea-893f-32c219eea11a.gif)
**Features**
## Features
- Catalog container images and filesystems to discover packages and libraries.
- Supports packages and libraries from various ecosystems (APK, DEB, RPM, Ruby Bundles, Python Wheel/Egg/requirements.txt, JavaScript NPM/Yarn, Java JAR/EAR/WAR, Jenkins plugins JPI/HPI, Go modules)
- Linux distribution identification (supports Alpine, BusyBox, CentOS/RedHat, Debian/Ubuntu flavored distributions)
@@ -37,6 +27,27 @@ A CLI tool and go library for generating a Software Bill of Materials (SBOM) fro
If you encounter an issue, please [let us know using the issue tracker](https://github.com/anchore/syft/issues).
## Installation
### Recommended
```bash
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
```
...or, you can specify a release version and destination directory for the installation:
```
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b <DESTINATION_DIR> <RELEASE_VERSION>
```
### Homebrew
```bash
brew tap anchore/syft
brew install syft
```
**Note**: Currently, Syft is built only for macOS and Linux.
## Getting started
To generate an SBOM for a Docker or OCI image:
@@ -56,7 +67,10 @@ To include software from all image layers in the SBOM, regardless of its presenc
syft packages <image> --scope all-layers
```
### Supported sources
Syft can generate a SBOM from a variety of sources:
```
# catalog a container image archive (from the result of `docker image save ...`, `podman save ...`, or `skopeo copy` commands)
syft packages path/to/image.tar
@@ -66,6 +80,7 @@ syft packages path/to/dir
```
Sources can be explicitly provided with a scheme:
```
docker:yourrepo/yourimage:tag use images from the Docker daemon
docker-archive:path/to/yourimage.tar use a tarball from disk for archives created from "docker save"
@@ -75,7 +90,10 @@ dir:path/to/yourproject read directly from a path on disk (any di
registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
```
### Output formats
The output format for Syft is configurable as well:
```
syft packages <image> -o <format>
```
@@ -83,26 +101,11 @@ syft packages <image> -o <format>
Where the `format`s available are:
- `json`: Use this to get as much information out of Syft as possible!
- `text`: A row-oriented, human-and-machine-friendly output.
- `cyclonedx`: A XML report conforming to the [CycloneDX 1.2](https://cyclonedx.org/) specification.
- `cyclonedx`: A XML report conforming to the [CycloneDX 1.2 specification](https://cyclonedx.org/specification/overview/).
- `spdx`: A tag-value formatted report conforming to the [SPDX 2.2 specification](https://spdx.github.io/spdx-spec/).
- `spdx-json`: A JSON report conforming to the [SPDX 2.2 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.2/schemas/spdx-schema.json).
- `table`: A columnar summary (default).
## Installation
**Recommended (macOS and Linux)**
```bash
# install the latest version to /usr/local/bin
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
# install a specific version into a specific dir
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b <SOME_BIN_PATH> <RELEASE_VERSION>
```
**Homebrew (macOS)**
```bash
brew tap anchore/syft
brew install syft
```
## Configuration
Configuration search paths:
@@ -123,6 +126,9 @@ output: "table"
# same as -q ; SYFT_QUIET env var
quiet: false
# same as --file; write output report to a file (default is to write to stdout)
file: ""
# enable/disable checking for application updates on startup
# same as SYFT_CHECK_FOR_APP_UPDATE env var
check-for-app-update: true
@@ -219,6 +225,9 @@ registry:
# skip TLS verification when communicating with the registry
# SYFT_REGISTRY_INSECURE_SKIP_TLS_VERIFY env var
insecure-skip-tls-verify: false
# use http instead of https when connecting to the registry
# SYFT_REGISTRY_INSECURE_USE_HTTP env var
insecure-use-http: false
# credentials for specific registries
auth:
+5 -2
View File
@@ -87,9 +87,12 @@ This release process itself should be as automated as possible, and have only a
1. Tag the main branch with a full semantic-version, prefixed with a `v`. If there is a
milestone with a partial version, the full version should be used for the git tag (e.g.
with a Milestone of `v0.1` the tag should be `v0.1.0`).
with a Milestone of `v0.1` the tag should be `v0.1.0`). You can determine the changes going
into a release by running `make changelog-unreleased`. Use this change list to determine the
release increment. After determining the release increment (major, minor, patch), create the tag.
Given the above example the command to create the tag would be `git tag v0.1.0`.
1. Push the tag.
1. Push the tag. Given the above example the command to push the tag would be `git push origin v0.1.0`.
1. A release admin must approve the release on the GitHub Actions release pipeline run page.
Once approved, the release pipeline will generate all assets and draft a GitHub Release.
+16 -10
View File
@@ -2,6 +2,7 @@ package cmd
import (
"errors"
"fmt"
"os"
"github.com/anchore/syft/internal/log"
@@ -14,11 +15,13 @@ import (
// signal interrupts. Is responsible for handling each event relative to a given UI an to coordinate eventing until
// an eventual graceful exit.
// nolint:gocognit,funlen
func eventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *partybus.Subscription, ux ui.UI, cleanupFn func()) error {
func eventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *partybus.Subscription, cleanupFn func(), uxs ...ui.UI) error {
defer cleanupFn()
events := subscription.Events()
var err error
if ux, err = setupUI(subscription.Unsubscribe, ux); err != nil {
var ux ui.UI
if ux, err = setupUI(subscription.Unsubscribe, uxs...); err != nil {
return err
}
@@ -78,15 +81,18 @@ func eventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *
return retErr
}
func setupUI(unsubscribe func() error, ux ui.UI) (ui.UI, error) {
if err := ux.Setup(unsubscribe); err != nil {
// replace the existing UI with a (simpler) logger UI
ux = ui.NewLoggerUI()
// setupUI takes one or more UIs that responds to events and takes a event bus unsubscribe function for use
// during teardown. With the given UIs, the first UI which the ui.Setup() function does not return an error
// will be utilized in execution. Providing a set of UIs allows for the caller to provide graceful fallbacks
// when there are environmental problem (e.g. unable to setup a TUI with the current TTY).
func setupUI(unsubscribe func() error, uis ...ui.UI) (ui.UI, error) {
for _, ux := range uis {
if err := ux.Setup(unsubscribe); err != nil {
// something is very wrong, bail.
return ux, err
log.Warnf("unable to setup given UI, falling back to alternative UI: %+v", err)
continue
}
log.Errorf("unable to setup given UI, falling back to logger: %+v", err)
return ux, nil
}
return ux, nil
return nil, fmt.Errorf("unable to setup any UI")
}
+6 -6
View File
@@ -96,8 +96,8 @@ func Test_eventLoop_gracefulExit(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
)
@@ -159,8 +159,8 @@ func Test_eventLoop_workerError(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
workerErr,
"should have seen a worker error, but did not",
@@ -230,8 +230,8 @@ func Test_eventLoop_unsubscribeError(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
)
@@ -300,8 +300,8 @@ func Test_eventLoop_handlerError(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
finalEvent.Error,
"should have seen a event error, but did not",
@@ -355,8 +355,8 @@ func Test_eventLoop_signalsStopExecution(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
)
@@ -425,8 +425,8 @@ func Test_eventLoop_uiTeardownError(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
teardownError,
"should have seen a UI teardown error, but did not",
+33 -11
View File
@@ -28,12 +28,14 @@ import (
const (
packagesExample = ` {{.appName}} {{.command}} alpine:latest a summary of discovered packages
{{.appName}} {{.command}} alpine:latest -o json show all possible cataloging details
{{.appName}} {{.command}} alpine:latest -o cyclonedx show a CycloneDX SBOM
{{.appName}} {{.command}} alpine:latest -o cyclonedx show a CycloneDX formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx show a SPDX 2.2 tag-value formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx-json show a SPDX 2.2 JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -vv show verbose debug information
Supports the following image sources:
{{.appName}} {{.command}} yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} path/to/a/file/or/dir a Docker tar, OCI tar, OCI directory, or generic filesystem directory
{{.appName}} {{.command}} path/to/a/file/or/dir a Docker tar, OCI tar, OCI directory, or generic filesystem directory
You can also explicitly specify the scheme to use:
{{.appName}} {{.command}} docker:yourrepo/yourimage:tag explicitly use the Docker daemon
@@ -47,7 +49,7 @@ const (
var (
packagesPresenterOpt packages.PresenterOption
packagesArgs = cobra.MinimumNArgs(1)
packagesArgs = cobra.MaximumNArgs(1)
packagesCmd = &cobra.Command{
Use: "packages [SOURCE]",
Short: "Generate a package SBOM",
@@ -65,8 +67,7 @@ var (
if err != nil {
return err
}
// silently exit
return fmt.Errorf("")
return fmt.Errorf("an image/directory argument is required")
}
// set the presenter
@@ -101,7 +102,7 @@ func init() {
}
func setPackageFlags(flags *pflag.FlagSet) {
///////// Formatting & Input options //////////////////////////////////////////////
// Formatting & Input options //////////////////////////////////////////////
flags.StringP(
"scope", "s", source.SquashedScope.String(),
@@ -112,7 +113,12 @@ func setPackageFlags(flags *pflag.FlagSet) {
fmt.Sprintf("report output formatter, options=%v", packages.AllPresenters),
)
///////// Upload options //////////////////////////////////////////////////////////
flags.StringP(
"file", "", "",
"file to write the report output to (default is STDOUT)",
)
// Upload options //////////////////////////////////////////////////////////
flags.StringP(
"host", "H", "",
"the hostname or URL of the Anchore Enterprise instance to upload to",
@@ -145,7 +151,7 @@ func setPackageFlags(flags *pflag.FlagSet) {
}
func bindPackagesConfigOptions(flags *pflag.FlagSet) error {
///////// Formatting & Input options //////////////////////////////////////////////
// Formatting & Input options //////////////////////////////////////////////
if err := viper.BindPFlag("package.cataloger.scope", flags.Lookup("scope")); err != nil {
return err
@@ -155,7 +161,11 @@ func bindPackagesConfigOptions(flags *pflag.FlagSet) error {
return err
}
///////// Upload options //////////////////////////////////////////////////////////
if err := viper.BindPFlag("file", flags.Lookup("file")); err != nil {
return err
}
// Upload options //////////////////////////////////////////////////////////
if err := viper.BindPFlag("anchore.host", flags.Lookup("host")); err != nil {
return err
@@ -187,12 +197,24 @@ func bindPackagesConfigOptions(flags *pflag.FlagSet) error {
func packagesExec(_ *cobra.Command, args []string) error {
// could be an image or a directory, with or without a scheme
userInput := args[0]
reporter, closer, err := reportWriter()
defer func() {
if err := closer(); err != nil {
log.Warnf("unable to write to report destination: %+v", err)
}
}()
if err != nil {
return err
}
return eventLoop(
packagesExecWorker(userInput),
setupSignals(),
eventSubscription,
ui.Select(appConfig.CliOptions.Verbosity > 0, appConfig.Quiet),
stereoscope.Cleanup,
ui.Select(appConfig.CliOptions.Verbosity > 0, appConfig.Quiet, reporter)...,
)
}
@@ -236,7 +258,7 @@ func packagesExecWorker(userInput string) <-chan error {
return errs
}
func runPackageSbomUpload(src source.Source, s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, scope source.Scope) error {
func runPackageSbomUpload(src *source.Source, s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, scope source.Scope) error {
log.Infof("uploading results to %s", appConfig.Anchore.Host)
if src.Metadata.Scheme != source.ImageScheme {
+23 -7
View File
@@ -7,6 +7,7 @@ import (
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/presenter/poweruser"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft/event"
@@ -34,11 +35,19 @@ var powerUserCmd = &cobra.Command{
"appName": internal.ApplicationName,
"command": "power-user",
}),
Args: cobra.ExactArgs(1),
Args: cobra.MaximumNArgs(1),
Hidden: true,
SilenceUsage: true,
SilenceErrors: true,
PreRunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
err := cmd.Help()
if err != nil {
return err
}
return fmt.Errorf("an image/directory argument is required")
}
if appConfig.Dev.ProfileCPU && appConfig.Dev.ProfileMem {
return fmt.Errorf("cannot profile CPU and memory simultaneously")
}
@@ -65,12 +74,24 @@ func init() {
func powerUserExec(_ *cobra.Command, args []string) error {
// could be an image or a directory, with or without a scheme
userInput := args[0]
reporter, closer, err := reportWriter()
defer func() {
if err := closer(); err != nil {
log.Warnf("unable to write to report destination: %+v", err)
}
}()
if err != nil {
return err
}
return eventLoop(
powerUserExecWorker(userInput),
setupSignals(),
eventSubscription,
ui.Select(appConfig.CliOptions.Verbosity > 0, appConfig.Quiet),
stereoscope.Cleanup,
ui.Select(appConfig.CliOptions.Verbosity > 0, appConfig.Quiet, reporter)...,
)
}
@@ -94,11 +115,6 @@ func powerUserExecWorker(userInput string) <-chan error {
}
defer cleanup()
if src.Metadata.Scheme != source.ImageScheme {
errs <- fmt.Errorf("the power-user subcommand only allows for 'image' schemes, given %q", src.Metadata.Scheme)
return
}
analysisResults := poweruser.JSONDocumentConfig{
SourceMetadata: src.Metadata,
ApplicationConfig: *appConfig,
+7 -7
View File
@@ -10,7 +10,7 @@ import (
"github.com/anchore/syft/syft/source"
)
type powerUserTask func(*poweruser.JSONDocumentConfig, source.Source) error
type powerUserTask func(*poweruser.JSONDocumentConfig, *source.Source) error
func powerUserTasks() ([]powerUserTask, error) {
var tasks []powerUserTask
@@ -42,7 +42,7 @@ func catalogPackagesTask() (powerUserTask, error) {
return nil, nil
}
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
packageCatalog, theDistro, err := syft.CatalogPackages(src, appConfig.Package.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -64,7 +64,7 @@ func catalogFileMetadataTask() (powerUserTask, error) {
metadataCataloger := file.NewMetadataCataloger()
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -110,7 +110,7 @@ func catalogFileDigestsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -142,7 +142,7 @@ func catalogSecretsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.Secrets.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -170,7 +170,7 @@ func catalogFileClassificationsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileClassification.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -197,7 +197,7 @@ func catalogContentsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileContents.Cataloger.ScopeOpt)
if err != nil {
return err
+33
View File
@@ -0,0 +1,33 @@
package cmd
import (
"fmt"
"io"
"os"
"strings"
"github.com/anchore/syft/internal/log"
)
func reportWriter() (io.Writer, func() error, error) {
nop := func() error { return nil }
path := strings.TrimSpace(appConfig.File)
switch len(path) {
case 0:
return os.Stdout, nop, nil
default:
reportFile, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
return nil, nop, fmt.Errorf("unable to create report file: %w", err)
}
return reportFile, func() error {
log.Infof("report written to file=%q", path)
return reportFile.Close()
}, nil
}
}
+11 -9
View File
@@ -7,42 +7,44 @@ require (
github.com/adrg/xdg v0.2.1
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf
github.com/anchore/go-rpmdb v0.0.0-20210602151223-1f0f707a2894
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/stereoscope v0.0.0-20210524175238-3b7662f3a66f
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29
github.com/anchore/stereoscope v0.0.0-20211005213828-538011008578
github.com/antihax/optional v1.0.0
github.com/bmatcuk/doublestar/v2 v2.0.4
github.com/docker/docker v17.12.0-ce-rc1.0.20200309214505-aa6a9891b09c+incompatible
github.com/dustin/go-humanize v1.0.0
github.com/facebookincubator/nvdtools v0.1.4
github.com/go-test/deep v1.0.7
github.com/google/uuid v1.1.1
github.com/google/go-cmp v0.4.1
github.com/google/uuid v1.2.0
github.com/gookit/color v1.2.7
github.com/hashicorp/go-multierror v1.1.0
github.com/hashicorp/go-version v1.2.0
github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213
github.com/mitchellh/go-homedir v1.1.0
github.com/mitchellh/mapstructure v1.3.1
github.com/olekukonko/tablewriter v0.0.4
github.com/package-url/packageurl-go v0.1.0
github.com/pelletier/go-toml v1.8.0
github.com/pelletier/go-toml v1.8.1
github.com/pkg/profile v1.5.0
github.com/scylladb/go-set v1.0.2
github.com/sergi/go-diff v1.1.0
github.com/sirupsen/logrus v1.6.0
github.com/sirupsen/logrus v1.8.1
github.com/spdx/tools-golang v0.1.0
github.com/spf13/afero v1.2.2
github.com/spf13/cobra v1.0.1-0.20200909172742-8a63648dd905
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.7.0
github.com/stretchr/testify v1.7.0
github.com/vifraa/gopom v0.1.0
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240
github.com/wagoodman/jotframe v0.0.0-20200730190914-3517092dd163
github.com/x-cray/logrus-prefixed-formatter v0.5.2
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
golang.org/x/mod v0.3.0
gopkg.in/yaml.v2 v2.3.0
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1
gopkg.in/yaml.v2 v2.4.0
)
+32 -24
View File
@@ -73,7 +73,6 @@ github.com/Azure/go-autorest/autorest/validation v0.1.0/go.mod h1:Ha3z/SqBeaalWQ
github.com/Azure/go-autorest/autorest/validation v0.2.0/go.mod h1:3EEqHnBxQGHXRYq3HT1WyXAvT7LLY3tl70hw6tQIbjI=
github.com/Azure/go-autorest/logger v0.1.0/go.mod h1:oExouG+K6PryycPJfVSxi/koC6LSNgds39diKLz7Vrc=
github.com/Azure/go-autorest/tracing v0.5.0/go.mod h1:r/s2XiOKccPW3HrqB+W0TQzfbtp2fGCgRFtBroKn4Dk=
github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/Djarvur/go-err113 v0.0.0-20200410182137-af658d038157/go.mod h1:4UJr5HIiMZrwgkSPdsjy2uOQExX/WEILpIrO9UPGuXs=
@@ -109,19 +108,20 @@ github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRF
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf/go.mod h1:FaODhIA06mxO1E6R32JE0TL1JWZZkmjRIAd4ULvHUKk=
github.com/anchore/go-rpmdb v0.0.0-20210602151223-1f0f707a2894 h1:VvCq7fFNU8dwWkCTGUykm4p64nVaDCYkKrj87x350Sk=
github.com/anchore/go-rpmdb v0.0.0-20210602151223-1f0f707a2894/go.mod h1:8jNYOxCJC5kyD/Ct4MbzsDN2hOhRoCAzQcb/7KdYYGw=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63 h1:C9W/LAydEz/qdUhx1MdjO9l8NEcFKYknkxDVyo9LAoM=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63/go.mod h1:6qH8c6U/3CBVvDDDBZnPSTbTINq3cIdADUYTaVf75EM=
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04 h1:VzprUTpc0vW0nnNKJfJieyH/TZ9UYAnTZs5/gHTdAe8=
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04/go.mod h1:6dK64g27Qi1qGQZ67gFmBFvEHScy0/C8qhQhNe5B5pQ=
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZVsCYMrIZBpFxwV26CbsuoEh5muXD5I1Ods=
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/stereoscope v0.0.0-20210524175238-3b7662f3a66f h1:bFadyOLOkzME3BrZFZ5m8cf/b2hsn3aMSS9s+SKubRk=
github.com/anchore/stereoscope v0.0.0-20210524175238-3b7662f3a66f/go.mod h1:vhh1M99rfWx5ejMvz1lkQiFZUrC5wu32V12R4JXH+ZI=
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29 h1:K9LfnxwhqvihqU0+MF325FNy7fsKV9EGaUxdfR4gnWk=
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29/go.mod h1:Oc1UkGaJwY6ND6vtAqPSlYrptKRJngHwkwB6W7l1uP0=
github.com/anchore/stereoscope v0.0.0-20211005213828-538011008578 h1:gSpftl0RWfdTwlPmsOLgEawHIw16xwwG1mFD/wrVDRE=
github.com/anchore/stereoscope v0.0.0-20211005213828-538011008578/go.mod h1:kL7jfbXblrDcBhu5ja/s+VTYL3Mzof4eQNMJiSqcwXQ=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/antihax/optional v1.0.0 h1:xK2lYat7ZLaVVcIuj82J8kIro4V6kDe0AUDFboUCwcg=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/apex/log v1.1.4/go.mod h1:AlpoD9aScyQfJDVHmLMEcx4oU6LqzkWp4Mg9GdAcEvQ=
github.com/apex/log v1.3.0 h1:1fyfbPvUwD10nMoh3hY6MXzvZShJQn9/ck7ATgAt5pA=
github.com/apex/log v1.3.0/go.mod h1:jd8Vpsr46WAe3EZSQ/IUMs2qQD/GOycT5rPWCO1yGcs=
github.com/apex/logs v0.0.4/go.mod h1:XzxuLZ5myVHDy9SAmYpamKKRNApGj54PfYLcFrXqDwo=
github.com/aphistic/golf v0.0.0-20180712155816-02c07f170c5a/go.mod h1:3NqKYiepwy8kCu4PNA+aP7WUV72eXWJeP9/r3/K9aLE=
@@ -241,6 +241,8 @@ github.com/fortytw2/leaktest v1.3.0/go.mod h1:jDsjWgpAGjm2CA7WthBh/CdZYEPF31XHqu
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/gabriel-vasile/mimetype v1.3.0 h1:4YOHITFLyYwF+iqG0ybSLGArRItynpfwdlWRmJnd75E=
github.com/gabriel-vasile/mimetype v1.3.0/go.mod h1:fA8fi6KUiG7MgQQ+mEWotXoEOvmxRtOJlERCzSmRvr8=
github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-critic/go-critic v0.4.1/go.mod h1:7/14rZGnZbY6E38VEGk2kVhoq6itzc1E68facVDK23g=
@@ -266,8 +268,8 @@ github.com/go-openapi/spec v0.19.3/go.mod h1:FpwSN1ksY1eteniUU7X0N/BgJ7a4WvBFVA8
github.com/go-openapi/swag v0.0.0-20160704191624-1d0bd113de87/go.mod h1:DXUve3Dpr1UfpPtxFw+EFuQ41HhCWZfha5jSVRG7C7I=
github.com/go-openapi/swag v0.19.2/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
github.com/go-restruct/restruct v0.0.0-20191227155143-5734170a48a1 h1:LoN2wx/aN8JPGebG+2DaUyk4M+xRcqJXfuIbs8AWHdE=
github.com/go-restruct/restruct v0.0.0-20191227155143-5734170a48a1/go.mod h1:KqrpKpn4M8OLznErihXTGLlsXFGeLxHUrLRRI/1YjGk=
github.com/go-restruct/restruct v1.2.0-alpha h1:2Lp474S/9660+SJjpVxoKuWX09JsXHSrdV7Nv3/gkvc=
github.com/go-restruct/restruct v1.2.0-alpha/go.mod h1:KqrpKpn4M8OLznErihXTGLlsXFGeLxHUrLRRI/1YjGk=
github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
@@ -371,8 +373,9 @@ github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm4
github.com/google/rpmpack v0.0.0-20191226140753-aa36bfddb3a0/go.mod h1:RaTPr0KUf2K7fnZYLNDrr8rxAamWs3iNywJLtQ2AzBg=
github.com/google/subcommands v1.0.1/go.mod h1:ZjhPrFU+Olkh9WazFPsl27BQ4UPiG37m3yTrtFlrHVk=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.2.0 h1:qJYtXnJRWmpe7m/3XlyhrsLrEURqHRM2kxzoxXqyUDs=
github.com/google/uuid v1.2.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/wire v0.3.0/go.mod h1:i1DMg/Lu8Sz5yYl25iOdmc5CT5qusaa+zmRWs16741s=
github.com/google/wire v0.4.0/go.mod h1:ngWDr9Qvq3yZA10YrxfyGELY/AFWGVpy9c1LTRi1EoU=
github.com/googleapis/gax-go v2.0.2+incompatible/go.mod h1:SFVmujtThgffbyetf+mdk2eWhX2bMyUtNHzFKcPA9HY=
@@ -466,7 +469,6 @@ github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/X
github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213 h1:qGQQKEcAR99REcMpsXCp3lJ03zYT1PkRd3kQGPn9GVg=
github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213/go.mod h1:vNUNkEQ1e29fT/6vq2aBdFsgNPmy8qMdSay1npru+Sw=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
@@ -477,7 +479,6 @@ github.com/klauspost/cpuid v0.0.0-20180405133222-e7e905edc00e/go.mod h1:Pj4uuM52
github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.3 h1:CE8S1cTafDpPvMhIxNJKvHsGVBgn1xWYf1NbHQhywc8=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
@@ -590,13 +591,12 @@ github.com/opencontainers/runc v0.0.0-20190115041553-12f6a991201f/go.mod h1:qT5X
github.com/opencontainers/runc v0.1.1 h1:GlxAyO6x8rfZYN9Tt0Kti5a/cP41iuiO2yYT0IJGY8Y=
github.com/opencontainers/runc v0.1.1/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
github.com/opencontainers/runtime-spec v1.0.2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/package-url/packageurl-go v0.1.0 h1:efWBc98O/dBZRg1pw2xiDzovnlMjCa9NPnfaiBduh8I=
github.com/package-url/packageurl-go v0.1.0/go.mod h1:C/ApiuWpmbpni4DIOECf6WCjFUZV7O1Fx7VAzrZHgBw=
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k=
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/pelletier/go-toml v1.8.0 h1:Keo9qb7iRJs2voHvunFtuuYFsbWeOBh8/P9v/kVMFtw=
github.com/pelletier/go-toml v1.8.0/go.mod h1:D6yutnOGMveHEPV7VQOuvI/gXY61bv+9bAOTRnLElKs=
github.com/pelletier/go-toml v1.8.1 h1:1Nf83orprkJyknT6h7zbuEGUEjcyVlCxSUGTENmNCRM=
github.com/pelletier/go-toml v1.8.1/go.mod h1:T2/BmBdy8dvIRq1a/8aqjN41wvWlN4lrapLU/GW4pbc=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
github.com/phayes/checkstyle v0.0.0-20170904204023-bfd46e6a821d/go.mod h1:3OzsM7FXDQlpCiw2j81fOmAwQLnZnLGXVKUzeKQXIAw=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -660,8 +660,9 @@ github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPx
github.com/sirupsen/logrus v1.3.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.6.0 h1:UBcNElsrwanuuMsnGSlYmtmgbb23qDR5dG+6X6Oo89I=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
github.com/sirupsen/logrus v1.8.1 h1:dJKuHgqk1NNQlqoA6BTlM1Wf9DOH3NBjQyu0h9+AZZE=
github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
github.com/smartystreets/assertions v1.0.0 h1:UVQPSSmc3qtTi+zPPkCXvZX9VvW/xT/NsRvKfwY81a8=
github.com/smartystreets/assertions v1.0.0/go.mod h1:kHHU4qYBaI3q23Pp3VPrmWhuIUrLW/7eUrw0BU5VaoM=
@@ -713,6 +714,7 @@ github.com/stretchr/testify v1.3.1-0.20190311161405-34c6fa2dc709/go.mod h1:M5WIy
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/subosito/gotenv v1.2.0 h1:Slr1R9HxAlEKefgq5jn9U+DnETlIUa6HfgEzj0g5d7s=
@@ -723,7 +725,6 @@ github.com/tetafro/godot v0.3.7/go.mod h1:/7NLHhv08H1+8DNj0MElpAACw1ajsCuf3TKNQx
github.com/tetafro/godot v0.4.2/go.mod h1:/7NLHhv08H1+8DNj0MElpAACw1ajsCuf3TKNQxA5S+0=
github.com/timakin/bodyclose v0.0.0-20190930140734-f7f2e9bca95e/go.mod h1:Qimiffbc6q9tBWlVV6x0P9sat/ao1xEkREYPPj9hphk=
github.com/timakin/bodyclose v0.0.0-20200424151742-cb6215831a94/go.mod h1:Qimiffbc6q9tBWlVV6x0P9sat/ao1xEkREYPPj9hphk=
github.com/tj/assert v0.0.0-20171129193455-018094318fb0 h1:Rw8kxzWo1mr6FSaYXjQELRe88y2KdfynXdnK72rdjtA=
github.com/tj/assert v0.0.0-20171129193455-018094318fb0/go.mod h1:mZ9/Rh9oLWpLLDRpvE+3b7gP/C2YyLFYxNmcLnPTMe0=
github.com/tj/go-elastic v0.0.0-20171221160941-36157cbbebc2/go.mod h1:WjeM0Oo1eNAjXGDx2yma7uG2XoyRZTq1uv3M/o7imD0=
github.com/tj/go-kinesis v0.0.0-20171128231115-08b17f58cb1b/go.mod h1:/yhzCV0xPfx6jb1bBgRFjl5lytqVqZXEaeqWP8lTEao=
@@ -746,6 +747,8 @@ github.com/valyala/fasthttp v1.2.0/go.mod h1:4vX61m6KN+xDduDNwXrhIAVZaZaZiQ1luJk
github.com/valyala/quicktemplate v1.2.0/go.mod h1:EH+4AkTd43SvgIbQHYu59/cJyxDoOVRUAfrukLPuGJ4=
github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a/go.mod h1:v3UYOV9WzVtRmSR+PDvWpU/qWl4Wa5LApYYX4ZtKbio=
github.com/vdemeester/k8s-pkg-credentialprovider v1.17.4/go.mod h1:inCTmtUdr5KJbreVojo06krnTgaeAz/Z7lynpPk/Q2c=
github.com/vifraa/gopom v0.1.0 h1:v897eVxf6lflkEXzPmKbo4YhX2oS/LGjz7cqjWnSmCU=
github.com/vifraa/gopom v0.1.0/go.mod h1:oPa1dcrGrtlO37WPDBm5SqHAT+wTgF8An1Q71Z6Vv4o=
github.com/vmware/govmomi v0.20.3/go.mod h1:URlwyTFZX72RmxtxuaFL2Uj3fD1JTvZdx59bHWk6aFU=
github.com/wagoodman/go-partybus v0.0.0-20200526224238-eb215533f07d/go.mod h1:JPirS5jde/CF5qIjcK4WX+eQmKXdPc6vcZkJ/P0hfPw=
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5 h1:phTLPgMRDYTizrBSKsNSOa2zthoC2KsJsaY/8sg3rD8=
@@ -798,9 +801,8 @@ golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc/go.mod h1:yigFU9vqHzYiE8U
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191206172530-e9b2fee46413/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37 h1:cg5LA/zNPRzIXIWSCxQW10Rvpy94aQh3LT/ShoCpkHw=
golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 h1:psW17arqaxU48Z5kZ0CQnkZWQJsqcURM6tKiBApRjXI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190125153040-c74c464bbbf2/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
@@ -872,8 +874,9 @@ golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200602114024-627f9648deb9 h1:pNX+40auqi2JqRfOP1akLGtYcn15TUbkhwuCO3foqqM=
golang.org/x/net v0.0.0-20200602114024-627f9648deb9/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20210505024714-0287a6fb4125/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d h1:LO7XpTYMwTqxjLcGWPijK3vRXg1aWdlNOVOHRq45d7c=
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181106182150-f42d05182288/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -938,14 +941,18 @@ golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980 h1:OjiUf46hAmXblsZdnoSXsEUSKU8r1UEzcL5RVZ4gO9Y=
golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da h1:b3NXsE2LusjYGGjL5bxEVZZORm/YEFFrWFjR8eFrw/c=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.0.0-20160726164857-2910a502d2bf/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -1132,8 +1139,9 @@ gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo=
+37 -21
View File
@@ -29,6 +29,7 @@ type parser interface {
type Application struct {
ConfigPath string `yaml:",omitempty" json:"configPath"` // the location where the application config was read from (either from -c or discovered while loading)
Output string `yaml:"output" json:"output" mapstructure:"output"` // -o, the Presenter hint string to use for report formatting
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"` // -q, indicates to not show any status output to stderr (ETUI or logging UI)
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Anchore anchore `yaml:"anchore" json:"anchore" mapstructure:"anchore"` // options for interacting with Anchore Engine/Enterprise
@@ -88,10 +89,43 @@ func (cfg Application) loadDefaultValues(v *viper.Viper) {
}
}
// build inflates simple config values into syft native objects (or other complex objects) after the config is fully read in.
func (cfg *Application) parseConfigValues() error {
// parse application config options
for _, optionFn := range []func() error{
cfg.parseUploadOptions,
cfg.parseLogLevelOption,
} {
if err := optionFn(); err != nil {
return err
}
}
// parse nested config options
// for each field in the configuration struct, see if the field implements the parser interface
// note: the app config is a pointer, so we need to grab the elements explicitly (to traverse the address)
value := reflect.ValueOf(cfg).Elem()
for i := 0; i < value.NumField(); i++ {
// note: since the interface method of parser is a pointer receiver we need to get the value of the field as a pointer.
if parsable, ok := value.Field(i).Addr().Interface().(parser); ok {
// the field implements parser, call it
if err := parsable.parseConfigValues(); err != nil {
return err
}
}
}
return nil
}
func (cfg *Application) parseUploadOptions() error {
if cfg.Anchore.Host == "" && cfg.Anchore.Dockerfile != "" {
return fmt.Errorf("cannot provide dockerfile option without enabling upload")
}
return nil
}
func (cfg *Application) parseLogLevelOption() error {
if cfg.Quiet {
// TODO: this is bad: quiet option trumps all other logging options
// TODO: this is bad: quiet option trumps all other logging options (such as to a file on disk)
// we should be able to quiet the console logging and leave file logging alone...
// ... this will be an enhancement for later
cfg.Log.LevelOpt = logrus.PanicLevel
@@ -115,28 +149,10 @@ func (cfg *Application) parseConfigValues() error {
case v >= 2:
cfg.Log.LevelOpt = logrus.DebugLevel
default:
cfg.Log.LevelOpt = logrus.WarnLevel
cfg.Log.LevelOpt = logrus.ErrorLevel
}
}
}
if cfg.Anchore.Host == "" && cfg.Anchore.Dockerfile != "" {
return fmt.Errorf("cannot provide dockerfile option without enabling upload")
}
// for each field in the configuration struct, see if the field implements the parser interface
// note: the app config is a pointer, so we need to grab the elements explicitly (to traverse the address)
value := reflect.ValueOf(cfg).Elem()
for i := 0; i < value.NumField(); i++ {
// note: since the interface method of parser is a pointer receiver we need to get the value of the field as a pointer.
if parsable, ok := value.Field(i).Addr().Interface().(parser); ok {
// the field implements parser, call it
if err := parsable.parseConfigValues(); err != nil {
return err
}
}
}
return nil
}
+3
View File
@@ -20,11 +20,13 @@ type RegistryCredentials struct {
type registry struct {
InsecureSkipTLSVerify bool `yaml:"insecure-skip-tls-verify" json:"insecure-skip-tls-verify" mapstructure:"insecure-skip-tls-verify"`
InsecureUseHTTP bool `yaml:"insecure-use-http" json:"insecure-use-http" mapstructure:"insecure-use-http"`
Auth []RegistryCredentials `yaml:"auth" json:"auth" mapstructure:"auth"`
}
func (cfg registry) loadDefaultValues(v *viper.Viper) {
v.SetDefault("registry.insecure-skip-tls-verify", false)
v.SetDefault("registry.insecure-use-http", false)
v.SetDefault("registry.auth", []RegistryCredentials{})
}
@@ -67,6 +69,7 @@ func (cfg *registry) ToOptions() *image.RegistryOptions {
}
return &image.RegistryOptions{
InsecureSkipTLSVerify: cfg.InsecureSkipTLSVerify,
InsecureUseHTTP: cfg.InsecureUseHTTP,
Credentials: auth,
}
}
+55
View File
@@ -2,6 +2,7 @@ package config
import (
"fmt"
"github.com/anchore/stereoscope/pkg/image"
"testing"
"github.com/stretchr/testify/assert"
@@ -54,3 +55,57 @@ func TestHasNonEmptyCredentials(t *testing.T) {
})
}
}
func Test_registry_ToOptions(t *testing.T) {
tests := []struct {
name string
input registry
expected image.RegistryOptions
}{
{
name: "no registry options",
input: registry{},
expected: image.RegistryOptions{
Credentials: []image.RegistryCredentials{},
},
},
{
name: "set InsecureSkipTLSVerify",
input: registry{
InsecureSkipTLSVerify: true,
},
expected: image.RegistryOptions{
InsecureSkipTLSVerify: true,
Credentials: []image.RegistryCredentials{},
},
},
{
name: "set InsecureUseHTTP",
input: registry{
InsecureUseHTTP: true,
},
expected: image.RegistryOptions{
InsecureUseHTTP: true,
Credentials: []image.RegistryCredentials{},
},
},
{
name: "set all bool options",
input: registry{
InsecureSkipTLSVerify: true,
InsecureUseHTTP: true,
},
expected: image.RegistryOptions{
InsecureSkipTLSVerify: true,
InsecureUseHTTP: true,
Credentials: []image.RegistryCredentials{},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, &test.expected, test.input.ToOptions())
})
}
}
+24
View File
@@ -1,7 +1,9 @@
package internal
import (
"fmt"
"io"
"os"
"github.com/anchore/syft/internal/log"
)
@@ -12,3 +14,25 @@ func CloseAndLogError(closer io.Closer, location string) {
log.Warnf("unable to close file for location=%q: %+v", location, err)
}
}
type ErrPath struct {
Path string
Err error
}
func (e ErrPath) Error() string {
return fmt.Sprintf("unable to observe contents of %+v: %v", e.Path, e.Err)
}
func IsErrPath(err error) bool {
_, ok := err.(ErrPath)
return ok
}
func IsErrPathPermission(err error) bool {
pathErr, ok := err.(ErrPath)
if ok {
return os.IsPermission(pathErr.Err)
}
return ok
}
+5 -2
View File
@@ -3,6 +3,7 @@ package logger
import (
"fmt"
"io"
"io/fs"
"io/ioutil"
"os"
@@ -10,6 +11,8 @@ import (
prefixed "github.com/x-cray/logrus-prefixed-formatter"
)
const defaultLogFilePermissions fs.FileMode = 0644
// LogrusConfig contains all configurable values for the Logrus logger
type LogrusConfig struct {
EnableConsole bool
@@ -38,7 +41,7 @@ func NewLogrusLogger(cfg LogrusConfig) *LogrusLogger {
var output io.Writer
switch {
case cfg.EnableConsole && cfg.EnableFile:
logFile, err := os.OpenFile(cfg.FileLocation, os.O_WRONLY|os.O_CREATE, 0755)
logFile, err := os.OpenFile(cfg.FileLocation, os.O_WRONLY|os.O_CREATE, defaultLogFilePermissions)
if err != nil {
panic(fmt.Errorf("unable to setup log file: %w", err))
}
@@ -46,7 +49,7 @@ func NewLogrusLogger(cfg LogrusConfig) *LogrusLogger {
case cfg.EnableConsole:
output = os.Stderr
case cfg.EnableFile:
logFile, err := os.OpenFile(cfg.FileLocation, os.O_WRONLY|os.O_CREATE, 0755)
logFile, err := os.OpenFile(cfg.FileLocation, os.O_WRONLY|os.O_CREATE, defaultLogFilePermissions)
if err != nil {
panic(fmt.Errorf("unable to setup log file: %w", err))
}
@@ -10,7 +10,7 @@ type CreationInfo struct {
Created time.Time `json:"created"`
// Identify who (or what, in the case of a tool) created the SPDX file. If the SPDX file was created by an
// individual, indicate the person's name. If the SPDX file was created on behalf of a company or organization,
//indicate the entity name. If the SPDX file was created using a software tool, indicate the name and version
// indicate the entity name. If the SPDX file was created using a software tool, indicate the name and version
// for that tool. If multiple participants or tools were involved, use multiple instances of this field. Person
// name or organization name may be designated as “anonymous” if appropriate.
Creators []string `json:"creators"`
@@ -40,4 +40,6 @@ type Document struct {
Files []File `json:"files,omitempty"`
// Snippets referenced in the SPDX document
Snippets []Snippet `json:"snippets,omitempty"`
// Relationships referenced in the SPDX document
Relationships []Relationship `json:"relationships,omitempty"`
}
@@ -20,22 +20,22 @@ type File struct {
Item
// (At least one is required.) The checksum property provides a mechanism that can be used to verify that the
// contents of a File or Package have not changed.
Checksums []Checksum `json:"checksums"`
Checksums []Checksum `json:"checksums,omitempty"`
// This field provides a place for the SPDX file creator to record file contributors. Contributors could include
// names of copyright holders and/or authors who may not be copyright holders yet contributed to the file content.
FileContributors []string `json:"fileContributors"`
FileContributors []string `json:"fileContributors,omitempty"`
// Each element is a SPDX ID for a File.
FileDependencies []string `json:"fileDependencies"`
FileDependencies []string `json:"fileDependencies,omitempty"`
// The name of the file relative to the root of the package.
FileName string `json:"fileName"`
// The type of the file
FileTypes []string `json:"fileTypes"`
FileTypes []string `json:"fileTypes,omitempty"`
// This field provides a place for the SPDX file creator to record potential legal notices found in the file.
// This may or may not include copyright statements.
NoticeText string `json:"noticeText,omitempty"`
// Indicates the project in which the SpdxElement originated. Tools must preserve doap:homepage and doap:name
// properties and the URI (if one is known) of doap:Project resources that are values of this property. All other
// properties of doap:Projects are not directly supported by SPDX and may be dropped when translating to or
// from some SPDX formats.
ArtifactOf []string `json:"artifactOf"`
// from some SPDX formats(deprecated).
ArtifactOf []string `json:"artifactOf,omitempty"`
}
@@ -1,11 +1,13 @@
package spdx22
type Relationship struct {
// SPDX ID for SpdxElement. A related SpdxElement.
RelatedSpdxElement string `json:"relatedSpdxElement"`
// Id to which the SPDX element is related
SpdxElementID string `json:"spdxElementId"`
// Describes the type of relationship between two SPDX elements.
RelationshipType RelationshipType `json:"relationshipType"`
Comment string `json:"comment,omitempty"`
// SPDX ID for SpdxElement. A related SpdxElement.
RelatedSpdxElement string `json:"relatedSpdxElement"`
Comment string `json:"comment,omitempty"`
}
// source: https://spdx.github.io/spdx-spec/7-relationships-between-SPDX-elements/
@@ -1,7 +1,9 @@
package packages
import (
"crypto/sha256"
"fmt"
"path/filepath"
"strings"
"github.com/anchore/syft/internal/presenter/packages/model/spdx22"
@@ -29,6 +31,43 @@ func getSPDXExternalRefs(p *pkg.Package) (externalRefs []spdx22.ExternalRef) {
return externalRefs
}
func getSPDXFiles(packageSpdxID string, p *pkg.Package) (files []spdx22.File, fileIDs []string, relationships []spdx22.Relationship) {
files = make([]spdx22.File, 0)
fileIDs = make([]string, 0)
relationships = make([]spdx22.Relationship, 0)
pkgFileOwner, ok := p.Metadata.(pkg.FileOwner)
if !ok {
return files, fileIDs, relationships
}
for _, ownedFilePath := range pkgFileOwner.OwnedFiles() {
baseFileName := filepath.Base(ownedFilePath)
pathHash := sha256.Sum256([]byte(ownedFilePath))
fileSpdxID := spdx22.ElementID(fmt.Sprintf("File-%s-%x", p.Name, pathHash)).String()
fileIDs = append(fileIDs, fileSpdxID)
files = append(files, spdx22.File{
FileName: ownedFilePath,
Item: spdx22.Item{
Element: spdx22.Element{
SPDXID: fileSpdxID,
Name: baseFileName,
},
},
})
relationships = append(relationships, spdx22.Relationship{
SpdxElementID: packageSpdxID,
RelationshipType: spdx22.ContainsRelationship,
RelatedSpdxElement: fileSpdxID,
})
}
return files, fileIDs, relationships
}
func getSPDXLicense(p *pkg.Package) string {
// source: https://spdx.github.io/spdx-spec/3-package-information/#313-concluded-license
// The options to populate this field are limited to:
@@ -95,6 +95,15 @@ func Test_getSPDXLicense(t *testing.T) {
},
expected: "GPL-3.0",
},
{
name: "debian to spdx conversion",
input: pkg.Package{
Licenses: []string{
"GPL-2",
},
},
expected: "GPL-2.0",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
@@ -4,6 +4,8 @@ import (
"encoding/json"
"fmt"
"io"
"path"
"strings"
"time"
"github.com/anchore/syft/internal"
@@ -12,8 +14,11 @@ import (
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/google/uuid"
)
const anchoreNamespace = "https://anchore.com/syft"
// SPDXJsonPresenter is a SPDX presentation object for the syft results (see https://github.com/spdx/spdx-spec)
type SPDXJsonPresenter struct {
catalog *pkg.Catalog
@@ -39,15 +44,29 @@ func (pres *SPDXJsonPresenter) Present(output io.Writer) error {
return enc.Encode(&doc)
}
// newSPDXJsonDocument creates and populates a new JSON document struct that follows the SPDX 2.2 spec from the given cataloging results.
func newSPDXJsonDocument(catalog *pkg.Catalog, srcMetadata source.Metadata) spdx22.Document {
var name string
uniqueID := uuid.Must(uuid.NewRandom())
var name, input, identifier string
switch srcMetadata.Scheme {
case source.ImageScheme:
name = srcMetadata.ImageMetadata.UserInput
name = cleanSPDXName(srcMetadata.ImageMetadata.UserInput)
input = "image"
case source.DirectoryScheme:
name = srcMetadata.Path
name = cleanSPDXName(srcMetadata.Path)
input = "dir"
}
if name != "." {
identifier = path.Join(input, fmt.Sprintf("%s-%s", name, uniqueID.String()))
} else {
identifier = path.Join(input, uniqueID.String())
}
namespace := path.Join(anchoreNamespace, identifier)
packages, files, relationships := newSPDXJsonElements(catalog)
return spdx22.Document{
Element: spdx22.Element{
SPDXID: spdx22.ElementID("DOCUMENT").String(),
@@ -64,23 +83,35 @@ func newSPDXJsonDocument(catalog *pkg.Catalog, srcMetadata source.Metadata) spdx
LicenseListVersion: spdxlicense.Version,
},
DataLicense: "CC0-1.0",
DocumentNamespace: fmt.Sprintf("https://anchore.com/syft/image/%s", srcMetadata.ImageMetadata.UserInput),
Packages: newSPDXJsonPackages(catalog),
DocumentNamespace: namespace,
Packages: packages,
Files: files,
Relationships: relationships,
}
}
func newSPDXJsonPackages(catalog *pkg.Catalog) []spdx22.Package {
results := make([]spdx22.Package, 0)
func newSPDXJsonElements(catalog *pkg.Catalog) ([]spdx22.Package, []spdx22.File, []spdx22.Relationship) {
packages := make([]spdx22.Package, 0)
relationships := make([]spdx22.Relationship, 0)
files := make([]spdx22.File, 0)
for _, p := range catalog.Sorted() {
license := getSPDXLicense(p)
packageSpdxID := spdx22.ElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.Version)).String()
packageFiles, fileIDs, packageFileRelationships := getSPDXFiles(packageSpdxID, p)
files = append(files, packageFiles...)
relationships = append(relationships, packageFileRelationships...)
// note: the license concluded and declared should be the same since we are collecting license information
// from the project data itself (the installed package files).
results = append(results, spdx22.Package{
packages = append(packages, spdx22.Package{
Description: getSPDXDescription(p),
DownloadLocation: getSPDXDownloadLocation(p),
ExternalRefs: getSPDXExternalRefs(p),
FilesAnalyzed: false,
HasFiles: fileIDs,
Homepage: getSPDXHomepage(p),
LicenseDeclared: license, // The Declared License is what the authors of a project believe govern the package
Originator: getSPDXOriginator(p),
@@ -89,11 +120,23 @@ func newSPDXJsonPackages(catalog *pkg.Catalog) []spdx22.Package {
Item: spdx22.Item{
LicenseConcluded: license, // The Concluded License field is the license the SPDX file creator believes governs the package
Element: spdx22.Element{
SPDXID: spdx22.ElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.Version)).String(),
SPDXID: packageSpdxID,
Name: p.Name,
},
},
})
}
return results
return packages, files, relationships
}
func cleanSPDXName(name string) string {
// remove # according to specification
name = strings.ReplaceAll(name, "#", "-")
// remove : for url construction
name = strings.ReplaceAll(name, ":", "-")
// clean relative pathing
return path.Clean(name)
}
@@ -29,5 +29,12 @@ func TestSPDXJSONImagePresenter(t *testing.T) {
}
func spdxJsonRedactor(s []byte) []byte {
return regexp.MustCompile(`"created": .*`).ReplaceAll(s, []byte("redacted"))
// each SBOM reports the time it was generated, which is not useful during snapshot testing
s = regexp.MustCompile(`"created": .*`).ReplaceAll(s, []byte("redacted"))
// each SBOM reports a unique documentNamespace when generated, this is not useful for snapshot testing
s = regexp.MustCompile(`"documentNamespace": .*`).ReplaceAll(s, []byte("redacted"))
// the license list will be updated periodically, the value here should not be directly tested in snapshot tests
return regexp.MustCompile(`"licenseListVersion": .*`).ReplaceAll(s, []byte("redacted"))
}
@@ -29,5 +29,8 @@ func TestSPDXTagValueImagePresenter(t *testing.T) {
}
func spdxTagValueRedactor(s []byte) []byte {
return regexp.MustCompile(`Created: .*`).ReplaceAll(s, []byte("redacted"))
// each SBOM reports the time it was generated, which is not useful during snapshot testing
s = regexp.MustCompile(`Created: .*`).ReplaceAll(s, []byte("redacted"))
// the license list will be updated periodically, the value here should not be directly tested in snapshot tests
return regexp.MustCompile(`LicenseListVersion: .*`).ReplaceAll(s, []byte("redacted"))
}
@@ -27,6 +27,11 @@
"author": "",
"authorEmail": "",
"platform": "",
"files": [
{
"path": "/some/path/pkg1/depedencies/foo"
}
],
"sitePackagesRootPath": ""
}
},
@@ -9,7 +9,7 @@
"locations": [
{
"path": "/somefile-1.txt",
"layerID": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59"
"layerID": "sha256:ffb5e9eaa453a002110719d12c294960117ca2903953d1faa40f01dc3f77045c"
}
],
"licenses": [
@@ -40,7 +40,7 @@
"locations": [
{
"path": "/somefile-2.txt",
"layerID": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec"
"layerID": "sha256:8463854829fc53d47b9dcdf7ee79fe7eb4ca7933c910f67f8521412f7a2f5c21"
}
],
"licenses": [],
@@ -67,7 +67,7 @@
"type": "image",
"target": {
"userInput": "user-image-input",
"imageID": "sha256:5900c94a5bc1e083aa24ad1a223bf6eb9910dc8a6b01cb979ec306cb91709ea1",
"imageID": "sha256:112851310e48e604f7379e2a3acddab50e91ce926edacb598a532e60ff6b776a",
"manifestDigest": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"tags": [
@@ -77,17 +77,17 @@
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59",
"digest": "sha256:ffb5e9eaa453a002110719d12c294960117ca2903953d1faa40f01dc3f77045c",
"size": 22
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec",
"digest": "sha256:8463854829fc53d47b9dcdf7ee79fe7eb4ca7933c910f67f8521412f7a2f5c21",
"size": 16
}
],
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NjUsImRpZ2VzdCI6InNoYTI1Njo1OTAwYzk0YTViYzFlMDgzYWEyNGFkMWEyMjNiZjZlYjk5MTBkYzhhNmIwMWNiOTc5ZWMzMDZjYjkxNzA5ZWExIn0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1NjpmYjZiZWVjYjc1YjM5ZjRiYjgxM2RiZjE3N2U1MDFlZGQ1ZGRiM2U2OWJiNDVjZWRlYjc4YzY3NmVlMWI3YTU5In0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjMxOWI1ODhjZTY0MjUzYTg3YjUzM2M4ZWQwMWNmMDAyNWUwZWFjOThlN2I1MTZlMTI1MzI5NTdlMTI0NGZkZWMifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjEtMDQtMDZUMTk6MTM6NTIuNTI0Mzc4WiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIxLTA0LTA2VDE5OjEzOjUyLjQ1ODIwNjFaIiwiY3JlYXRlZF9ieSI6IkFERCBmaWxlLTEudHh0IC9zb21lZmlsZS0xLnR4dCAjIGJ1aWxka2l0IiwiY29tbWVudCI6ImJ1aWxka2l0LmRvY2tlcmZpbGUudjAifSx7ImNyZWF0ZWQiOiIyMDIxLTA0LTA2VDE5OjEzOjUyLjUyNDM3OFoiLCJjcmVhdGVkX2J5IjoiQUREIGZpbGUtMi50eHQgL3NvbWVmaWxlLTIudHh0ICMgYnVpbGRraXQiLCJjb21tZW50IjoiYnVpbGRraXQuZG9ja2VyZmlsZS52MCJ9XSwib3MiOiJsaW51eCIsInJvb3RmcyI6eyJ0eXBlIjoibGF5ZXJzIiwiZGlmZl9pZHMiOlsic2hhMjU2OmZiNmJlZWNiNzViMzlmNGJiODEzZGJmMTc3ZTUwMWVkZDVkZGIzZTY5YmI0NWNlZGViNzhjNjc2ZWUxYjdhNTkiLCJzaGEyNTY6MzE5YjU4OGNlNjQyNTNhODdiNTMzYzhlZDAxY2YwMDI1ZTBlYWM5OGU3YjUxNmUxMjUzMjk1N2UxMjQ0ZmRlYyJdfX0=",
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NzMsImRpZ2VzdCI6InNoYTI1NjoxMTI4NTEzMTBlNDhlNjA0ZjczNzllMmEzYWNkZGFiNTBlOTFjZTkyNmVkYWNiNTk4YTUzMmU2MGZmNmI3NzZhIn0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1NjpmZmI1ZTllYWE0NTNhMDAyMTEwNzE5ZDEyYzI5NDk2MDExN2NhMjkwMzk1M2QxZmFhNDBmMDFkYzNmNzcwNDVjIn0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2Ojg0NjM4NTQ4MjlmYzUzZDQ3YjlkY2RmN2VlNzlmZTdlYjRjYTc5MzNjOTEwZjY3Zjg1MjE0MTJmN2EyZjVjMjEifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjEtMDktMDhUMTc6MjE6NTguODk2NTI5MTkyWiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIxLTA5LTA4VDE3OjIxOjU4Ljg3OTY5MDgyNFoiLCJjcmVhdGVkX2J5IjoiQUREIGZpbGUtMS50eHQgL3NvbWVmaWxlLTEudHh0ICMgYnVpbGRraXQiLCJjb21tZW50IjoiYnVpbGRraXQuZG9ja2VyZmlsZS52MCJ9LHsiY3JlYXRlZCI6IjIwMjEtMDktMDhUMTc6MjE6NTguODk2NTI5MTkyWiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0yLnR4dCAvc29tZWZpbGUtMi50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn1dLCJvcyI6ImxpbnV4Iiwicm9vdGZzIjp7InR5cGUiOiJsYXllcnMiLCJkaWZmX2lkcyI6WyJzaGEyNTY6ZmZiNWU5ZWFhNDUzYTAwMjExMDcxOWQxMmMyOTQ5NjAxMTdjYTI5MDM5NTNkMWZhYTQwZjAxZGMzZjc3MDQ1YyIsInNoYTI1Njo4NDYzODU0ODI5ZmM1M2Q0N2I5ZGNkZjdlZTc5ZmU3ZWI0Y2E3OTMzYzkxMGY2N2Y4NTIxNDEyZjdhMmY1YzIxIl19fQ==",
"repoDigests": [],
"scope": "Squashed"
}
@@ -3,12 +3,12 @@
"name": "/some/path",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"created": "2021-06-23T17:48:32.734847Z",
"created": "2021-09-16T20:44:35.198887Z",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-[not provided]"
],
"licenseListVersion": "3.13"
"licenseListVersion": "3.14"
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https://anchore.com/syft/image/",
@@ -31,6 +31,9 @@
}
],
"filesAnalyzed": false,
"hasFiles": [
"SPDXRef-File-package-1-04cd22424378dcd6c77fce08beb52493b5494a60ea5e1f9bdf9b16dc0cacffe9"
],
"licenseDeclared": "MIT",
"sourceInfo": "acquired package info from installed python package manifest file: /some/path/pkg1",
"versionInfo": "1.0.1"
@@ -57,5 +60,20 @@
"sourceInfo": "acquired package info from DPKG DB: /some/path/pkg1",
"versionInfo": "2.0.1"
}
],
"files": [
{
"SPDXID": "SPDXRef-File-package-1-04cd22424378dcd6c77fce08beb52493b5494a60ea5e1f9bdf9b16dc0cacffe9",
"name": "foo",
"licenseConcluded": "",
"fileName": "/some/path/pkg1/depedencies/foo"
}
],
"relationships": [
{
"spdxElementId": "SPDXRef-Package-python-package-1-1.0.1",
"relationshipType": "CONTAINS",
"relatedSpdxElement": "SPDXRef-File-package-1-04cd22424378dcd6c77fce08beb52493b5494a60ea5e1f9bdf9b16dc0cacffe9"
}
]
}
@@ -3,12 +3,12 @@
"name": "user-image-input",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"created": "2021-06-23T17:48:32.7379Z",
"created": "2021-09-16T20:44:35.203911Z",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-[not provided]"
],
"licenseListVersion": "3.13"
"licenseListVersion": "3.14"
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https://anchore.com/syft/image/user-image-input",
@@ -1,11 +1,11 @@
[Image]
Layer: 0
Digest: sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59
Digest: sha256:ffb5e9eaa453a002110719d12c294960117ca2903953d1faa40f01dc3f77045c
Size: 22
MediaType: application/vnd.docker.image.rootfs.diff.tar.gzip
Layer: 1
Digest: sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec
Digest: sha256:8463854829fc53d47b9dcdf7ee79fe7eb4ca7933c910f67f8521412f7a2f5c21
Size: 16
MediaType: application/vnd.docker.image.rootfs.diff.tar.gzip
@@ -32,7 +32,7 @@ func (pres *TextPresenter) Present(output io.Writer) error {
switch pres.srcMetadata.Scheme {
case source.DirectoryScheme:
fmt.Fprintln(w, fmt.Sprintf("[Path: %s]", pres.srcMetadata.Path))
fmt.Fprintf(w, "[Path: %s]\n", pres.srcMetadata.Path)
case source.ImageScheme:
fmt.Fprintln(w, "[Image]")
@@ -51,7 +51,7 @@ func (pres *TextPresenter) Present(output io.Writer) error {
// populate artifacts...
rows := 0
for _, p := range pres.catalog.Sorted() {
fmt.Fprintln(w, fmt.Sprintf("[%s]", p.Name))
fmt.Fprintf(w, "[%s]\n", p.Name)
fmt.Fprintln(w, " Version:\t", p.Version)
fmt.Fprintln(w, " Type:\t", string(p.Type))
fmt.Fprintln(w, " Found by:\t", p.FoundBy)
@@ -158,6 +158,11 @@ func presenterDirectoryInput(t testing.TB) (*pkg.Catalog, source.Metadata, *dist
Metadata: pkg.PythonPackageMetadata{
Name: "package-1",
Version: "1.0.1",
Files: []pkg.PythonFileRecord{
{
Path: "/some/path/pkg1/depedencies/foo",
},
},
},
PURL: "a-purl-2",
CPEs: []pkg.CPE{
@@ -22,6 +22,7 @@ type JSONFileMetadataEntry struct {
UserID int `json:"userID"`
GroupID int `json:"groupID"`
Digests []file.Digest `json:"digests,omitempty"`
MIMEType string `json:"mimeType"`
}
func NewJSONFileMetadata(data map[source.Location]source.FileMetadata, digests map[source.Location][]file.Digest) ([]JSONFileMetadata, error) {
@@ -46,6 +47,7 @@ func NewJSONFileMetadata(data map[source.Location]source.FileMetadata, digests m
UserID: metadata.UserID,
GroupID: metadata.GroupID,
Digests: digestResults,
MIMEType: metadata.MIMEType,
},
})
}
@@ -16,7 +16,8 @@
"mode": 775,
"type": "directory",
"userID": 0,
"groupID": 0
"groupID": 0,
"mimeType": ""
}
},
{
@@ -33,7 +34,8 @@
"algorithm": "sha256",
"value": "366a3f5653e34673b875891b021647440d0127c2ef041e3b1a22da2a7d4f3703"
}
]
],
"mimeType": ""
}
},
{
@@ -45,7 +47,8 @@
"type": "symbolicLink",
"linkDestination": "/c",
"userID": 0,
"groupID": 0
"groupID": 0,
"mimeType": ""
}
},
{
@@ -62,7 +65,8 @@
"algorithm": "sha256",
"value": "1b3722da2a7d90d033b87581a2a3f12021647445653e34666ef041e3b4f3707c"
}
]
],
"mimeType": ""
}
}
],
@@ -168,6 +172,7 @@
"configuration": {
"configPath": "",
"output": "",
"file": "",
"quiet": false,
"check-for-app-update": false,
"anchore": {
@@ -227,6 +232,7 @@
},
"registry": {
"insecure-skip-tls-verify": false,
"insecure-use-http": false,
"auth": null
}
}
@@ -0,0 +1,190 @@
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
"os"
"regexp"
"sort"
"strings"
"text/template"
"time"
"github.com/scylladb/go-set/strset"
)
// This program generates license_list.go.
const (
source = "license_list.go"
url = "https://spdx.org/licenses/licenses.json"
)
var tmp = template.Must(template.New("").Parse(`// Code generated by go generate; DO NOT EDIT.
// This file was generated by robots at {{ .Timestamp }}
// using data from {{ .URL }}
package spdxlicense
const Version = {{ printf "%q" .Version }}
var licenseIDs = map[string]string{
{{- range $k, $v := .LicenseIDs }}
{{ printf "%q" $k }}: {{ printf "%q" $v }},
{{- end }}
}
`))
var versionMatch = regexp.MustCompile(`-([0-9]+)\.?([0-9]+)?\.?([0-9]+)?\.?`)
type LicenseList struct {
Version string `json:"licenseListVersion"`
Licenses []struct {
ID string `json:"licenseId"`
Name string `json:"name"`
Text string `json:"licenseText"`
Deprecated bool `json:"isDeprecatedLicenseId"`
OSIApproved bool `json:"isOsiApproved"`
SeeAlso []string `json:"seeAlso"`
} `json:"licenses"`
}
func main() {
if err := run(); err != nil {
fmt.Println(err.Error())
os.Exit(1)
}
}
func run() error {
resp, err := http.Get(url)
if err != nil {
return fmt.Errorf("unable to get licenses list: %+v", err)
}
var result LicenseList
if err = json.NewDecoder(resp.Body).Decode(&result); err != nil {
return fmt.Errorf("unable to decode license list: %+v", err)
}
defer func() {
if err := resp.Body.Close(); err != nil {
log.Fatalf("unable to close body: %+v", err)
}
}()
f, err := os.Create(source)
if err != nil {
return fmt.Errorf("unable to create %q: %+v", source, err)
}
defer func() {
if err := f.Close(); err != nil {
log.Fatalf("unable to close %q: %+v", source, err)
}
}()
licenseIDs := processSPDXLicense(result)
err = tmp.Execute(f, struct {
Timestamp time.Time
URL string
Version string
LicenseIDs map[string]string
}{
Timestamp: time.Now(),
URL: url,
Version: result.Version,
LicenseIDs: licenseIDs,
})
if err != nil {
return fmt.Errorf("unable to generate template: %+v", err)
}
return nil
}
// Parsing the provided SPDX license list necessitates a two pass approach.
// The first pass is only related to what SPDX considers the truth. These K:V pairs will never be overwritten.
// The second pass attempts to generate known short/long version listings for each key.
// For info on some short name conventions see this document:
// https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/#license-short-name.
// The short long listing generation attempts to build all license permutations for a given key.
// The new keys are then also associated with their relative SPDX value. If a key has already been entered
// we know to ignore it since it came from the first pass which is considered the SPDX source of truth.
// We also sort the licenses for the second pass so that cases like `GPL-1` associate to `GPL-1.0` and not `GPL-1.1`.
func processSPDXLicense(result LicenseList) map[string]string {
// first pass build map
var licenseIDs = make(map[string]string)
for _, l := range result.Licenses {
cleanID := strings.ToLower(l.ID)
if _, exists := licenseIDs[cleanID]; exists {
log.Fatalf("duplicate license ID found: %q", cleanID)
}
licenseIDs[cleanID] = l.ID
}
sort.Slice(result.Licenses, func(i, j int) bool {
return result.Licenses[i].ID < result.Licenses[j].ID
})
// second pass build exceptions
// do not overwrite if already exists
for _, l := range result.Licenses {
var multipleID []string
cleanID := strings.ToLower(l.ID)
multipleID = append(multipleID, buildLicensePermutations(cleanID)...)
for _, id := range multipleID {
if _, exists := licenseIDs[id]; !exists {
licenseIDs[id] = l.ID
}
}
}
return licenseIDs
}
func buildLicensePermutations(license string) (perms []string) {
lv := findLicenseVersion(license)
vp := versionPermutations(lv)
version := strings.Join(lv, ".")
for _, p := range vp {
perms = append(perms, strings.Replace(license, version, p, 1))
}
return perms
}
func findLicenseVersion(license string) (version []string) {
versionList := versionMatch.FindAllStringSubmatch(license, -1)
if len(versionList) == 0 {
return version
}
for i, v := range versionList[0] {
if v != "" && i != 0 {
version = append(version, v)
}
}
return version
}
func versionPermutations(version []string) []string {
ver := append([]string(nil), version...)
perms := strset.New()
for i := 1; i <= 3; i++ {
if len(ver) < i+1 {
ver = append(ver, "0")
}
perm := strings.Join(ver[:i], ".")
badCount := strings.Count(perm, "0") + strings.Count(perm, ".")
if badCount != len(perm) {
perms.Add(perm)
}
}
return perms.List()
}
@@ -0,0 +1,146 @@
package main
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
func TestLicensePermutations(t *testing.T) {
var tests = []struct {
shortName string
permutations []string
}{
{
"GPL-1-only",
[]string{
"GPL-1-only",
"GPL-1.0-only",
"GPL-1.0.0-only",
},
},
{
"GPL-2",
[]string{
"GPL-2",
"GPL-2.0",
"GPL-2.0.0",
},
},
{
"GPL-2.0+",
[]string{
"GPL-2+",
"GPL-2.0+",
"GPL-2.0.0+",
},
},
{
"GPL-3.0.0-or-later",
[]string{
"GPL-3-or-later",
"GPL-3.0-or-later",
"GPL-3.0.0-or-later",
},
},
{
"oldap-2.0",
[]string{
"oldap-2",
"oldap-2.0",
"oldap-2.0.0",
},
},
}
for _, test := range tests {
t.Run(test.shortName, func(t *testing.T) {
perms := buildLicensePermutations(test.shortName)
assert.ElementsMatch(t, test.permutations, perms)
})
}
}
func TestVersionPermutations(t *testing.T) {
var tests = []struct {
version []string
permutations []string
}{
{
[]string{"1", "0"},
[]string{"1", "1.0", "1.0.0"},
},
{
[]string{"2"},
[]string{"2", "2.0", "2.0.0"},
},
{
[]string{"2", "0"},
[]string{"2", "2.0", "2.0.0"},
},
{
[]string{"3", "0", "0"},
[]string{"3", "3.0", "3.0.0"},
},
{
[]string{"0", "3"},
[]string{"0.3", "0.3.0"},
},
{
[]string{"0", "0", "3"},
[]string{"0.0.3"},
},
}
for _, test := range tests {
t.Run(strings.Join(test.version, "."), func(t *testing.T) {
got := versionPermutations(test.version)
assert.ElementsMatch(t, test.permutations, got)
})
}
}
func TestFindLicenseVersion(t *testing.T) {
var tests = []struct {
license string
version []string
}{
{
"GPL-1.0-only",
[]string{"1", "0"},
},
{
"GPL-2.0",
[]string{"2", "0"},
},
{
"GPL-2.0.0",
[]string{"2", "0", "0"},
},
{
"GPL-2",
[]string{"2"},
},
{
"bzip2-1",
[]string{"1"},
},
{
"php-3.01",
[]string{"3", "01"},
},
{
"oldap-2.0",
[]string{"2", "0"},
},
}
for _, test := range tests {
t.Run(test.license, func(t *testing.T) {
got := findLicenseVersion(test.license)
assert.Equal(t, test.version, got)
})
}
}
@@ -1,92 +0,0 @@
// +build ignore
package main
import (
"encoding/json"
"log"
"net/http"
"os"
"strings"
"text/template"
"time"
)
// This program generates license_list.go.
const (
source = "license_list.go"
url = "https://spdx.org/licenses/licenses.json"
)
var tmp = template.Must(template.New("").Parse(`// Code generated by go generate; DO NOT EDIT.
// This file was generated by robots at {{ .Timestamp }}
// using data from {{ .URL }}
package spdxlicense
const Version = {{ printf "%q" .Version }}
var licenseIDs = map[string]string{
{{- range $k, $v := .LicenseIDs }}
{{ printf "%q" $k }}: {{ printf "%q" $v }},
{{- end }}
}
`))
type LicenseList struct {
Version string `json:"licenseListVersion"`
Licenses []struct {
ID string `json:"licenseId"`
Name string `json:"name"`
Text string `json:"licenseText"`
Deprecated bool `json:"isDeprecatedLicenseId"`
OSIApproved bool `json:"isOsiApproved"`
SeeAlso []string `json:"seeAlso"`
} `json:"licenses"`
}
func main() {
resp, err := http.Get(url)
if err != nil {
log.Fatalf("unable to get licenses list: %+v", err)
}
var result LicenseList
if err = json.NewDecoder(resp.Body).Decode(&result); err != nil {
log.Fatalf("unable to decode license list: %+v", err)
}
f, err := os.Create(source)
if err != nil {
log.Fatalf("unable to create %q: %+v", source, err)
}
defer func() {
if err := f.Close(); err != nil {
log.Fatalf("unable to close %q: %+v", source, err)
}
}()
var licenseIDs = make(map[string]string)
for _, l := range result.Licenses {
cleanID := strings.ToLower(l.ID)
if _, exists := licenseIDs[cleanID]; exists {
log.Fatalf("duplicate license ID found: %q", cleanID)
}
licenseIDs[cleanID] = l.ID
}
err = tmp.Execute(f, struct {
Timestamp time.Time
URL string
Version string
LicenseIDs map[string]string
}{
Timestamp: time.Now(),
URL: url,
Version: result.Version,
LicenseIDs: licenseIDs,
})
if err != nil {
log.Fatalf("unable to generate template: %+v", err)
}
}
+9 -1
View File
@@ -4,7 +4,15 @@ import (
"strings"
)
//go:generate go run generate_license_list.go
// https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/#license-short-name
// License generated in license_list.go uses a regular expression to help resolve cases where
// x.0.0 and x are supplied as version numbers. For SPDX compatibility, versions with trailing
// dot-zeroes are considered to be equivalent to versions without (e.g., “2.0.0” is considered equal to “2.0” and “2”).
// EX: gpl-2+ ---> GPL-2.0+
// EX: gpl-2.0.0-only ---> GPL-2.0-only
// See the debian link for more details on the spdx license differences
//go:generate go run generate/generate_license_list.go
func ID(id string) (string, bool) {
value, exists := licenseIDs[strings.ToLower(id)]
File diff suppressed because it is too large Load Diff
@@ -10,5 +10,6 @@ func TestLicenceListIDs(t *testing.T) {
// do a sanity check on the generated data
assert.Equal(t, "0BSD", licenseIDs["0bsd"])
assert.Equal(t, "ZPL-2.1", licenseIDs["zpl-2.1"])
assert.Equal(t, "GPL-2.0", licenseIDs["gpl-2"])
assert.NotEmpty(t, Version)
}
+60
View File
@@ -0,0 +1,60 @@
package spdxlicense
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestIDParse(t *testing.T) {
var tests = []struct {
shortName string
spdx string
}{
{
"GPL-1-only",
"GPL-1.0-only",
},
{
"GPL-2",
"GPL-2.0",
},
{
"GPL-2+",
"GPL-2.0+",
},
{
"GPL-3.0.0-or-later",
"GPL-3.0-or-later",
},
{
"GPL-3-with-autoconf-exception",
"GPL-3.0-with-autoconf-exception",
},
{
"CC-by-nc-3-de",
"CC-BY-NC-3.0-DE",
},
// the below few cases are NOT expected, however, seem unavoidable given the current approach
{
"w3c-20150513.0.0",
"W3C-20150513",
},
{
"spencer-86.0.0",
"Spencer-86",
},
{
"unicode-dfs-2015.0.0",
"Unicode-DFS-2015",
},
}
for _, test := range tests {
t.Run(test.shortName, func(t *testing.T) {
got, exists := ID(test.shortName)
assert.True(t, exists)
assert.Equal(t, test.spdx, got)
})
}
}
+28 -15
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"context"
"fmt"
"io"
"os"
"sync"
@@ -11,7 +12,6 @@ import (
"github.com/anchore/syft/internal/logger"
syftEvent "github.com/anchore/syft/syft/event"
"github.com/anchore/syft/ui"
"github.com/k0kubun/go-ansi"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/jotframe/pkg/frame"
)
@@ -33,23 +33,28 @@ import (
// or in the shared ui package as a function on the main handler object. All handler functions should be completed
// processing an event before the ETUI exits (coordinated with a sync.WaitGroup)
type ephemeralTerminalUI struct {
unsubscribe func() error
handler *ui.Handler
waitGroup *sync.WaitGroup
frame *frame.Frame
logBuffer *bytes.Buffer
unsubscribe func() error
handler *ui.Handler
waitGroup *sync.WaitGroup
frame *frame.Frame
logBuffer *bytes.Buffer
uiOutput *os.File
reportOutput io.Writer
}
func NewEphemeralTerminalUI() UI {
// NewEphemeralTerminalUI writes all events to a TUI and writes the final report to the given writer.
func NewEphemeralTerminalUI(reportWriter io.Writer) UI {
return &ephemeralTerminalUI{
handler: ui.NewHandler(),
waitGroup: &sync.WaitGroup{},
handler: ui.NewHandler(),
waitGroup: &sync.WaitGroup{},
uiOutput: os.Stderr,
reportOutput: reportWriter,
}
}
func (h *ephemeralTerminalUI) Setup(unsubscribe func() error) error {
h.unsubscribe = unsubscribe
ansi.CursorHide()
hideCursor(h.uiOutput)
// prep the logger to not clobber the screen from now on (logrus only)
h.logBuffer = bytes.NewBufferString("")
@@ -79,7 +84,7 @@ func (h *ephemeralTerminalUI) Handle(event partybus.Event) error {
// are about to write bytes to stdout, so we should reset the terminal state first
h.closeScreen(false)
if err := handleCatalogerPresenterReady(event); err != nil {
if err := handleCatalogerPresenterReady(event, h.reportOutput); err != nil {
log.Errorf("unable to show %s event: %+v", event.Type, err)
}
@@ -93,7 +98,7 @@ func (h *ephemeralTerminalUI) openScreen() error {
config := frame.Config{
PositionPolicy: frame.PolicyFloatForward,
// only report output to stderr, reserve report output for stdout
Output: os.Stderr,
Output: h.uiOutput,
}
fr, err := frame.New(config)
@@ -126,14 +131,22 @@ func (h *ephemeralTerminalUI) flushLog() {
logWrapper, ok := log.Log.(*logger.LogrusLogger)
if ok {
fmt.Fprint(logWrapper.Output, h.logBuffer.String())
logWrapper.Logger.SetOutput(os.Stderr)
logWrapper.Logger.SetOutput(h.uiOutput)
} else {
fmt.Fprint(os.Stderr, h.logBuffer.String())
fmt.Fprint(h.uiOutput, h.logBuffer.String())
}
}
func (h *ephemeralTerminalUI) Teardown(force bool) error {
h.closeScreen(force)
ansi.CursorShow()
showCursor(h.uiOutput)
return nil
}
func hideCursor(output io.Writer) {
fmt.Fprint(output, "\x1b[?25l")
}
func showCursor(output io.Writer) {
fmt.Fprint(output, "\x1b[?25h")
}
+2 -3
View File
@@ -4,7 +4,6 @@ import (
"context"
"fmt"
"io"
"os"
"sync"
"github.com/anchore/syft/internal"
@@ -17,14 +16,14 @@ import (
// handleCatalogerPresenterReady is a UI function for processing the CatalogerFinished bus event, displaying the catalog
// via the given presenter to stdout.
func handleCatalogerPresenterReady(event partybus.Event) error {
func handleCatalogerPresenterReady(event partybus.Event, reportOutput io.Writer) error {
// show the report to stdout
pres, err := syftEventParsers.ParsePresenterReady(event)
if err != nil {
return fmt.Errorf("bad CatalogerFinished event: %w", err)
}
if err := pres.Present(os.Stdout); err != nil {
if err := pres.Present(reportOutput); err != nil {
return fmt.Errorf("unable to show package catalog report: %w", err)
}
return nil
+10 -4
View File
@@ -1,17 +1,23 @@
package ui
import (
"io"
"github.com/anchore/syft/internal/log"
syftEvent "github.com/anchore/syft/syft/event"
"github.com/wagoodman/go-partybus"
)
type loggerUI struct {
unsubscribe func() error
unsubscribe func() error
reportOutput io.Writer
}
func NewLoggerUI() UI {
return &loggerUI{}
// NewLoggerUI writes all events to the common application logger and writes the final report to the given writer.
func NewLoggerUI(reportWriter io.Writer) UI {
return &loggerUI{
reportOutput: reportWriter,
}
}
func (l *loggerUI) Setup(unsubscribe func() error) error {
@@ -25,7 +31,7 @@ func (l loggerUI) Handle(event partybus.Event) error {
return nil
}
if err := handleCatalogerPresenterReady(event); err != nil {
if err := handleCatalogerPresenterReady(event, l.reportOutput); err != nil {
log.Warnf("unable to show catalog image finished event: %+v", err)
}
+12 -10
View File
@@ -1,29 +1,31 @@
package ui
import (
"io"
"os"
"runtime"
"golang.org/x/crypto/ssh/terminal"
"golang.org/x/term"
)
// TODO: build tags to exclude options from windows
// Select is responsible for determining the specific UI function given select user option, the current platform
// config values, and environment status (such as a TTY being present).
func Select(verbose, quiet bool) UI {
var ui UI
isStdoutATty := terminal.IsTerminal(int(os.Stdout.Fd()))
isStderrATty := terminal.IsTerminal(int(os.Stderr.Fd()))
// config values, and environment status (such as a TTY being present). The first UI in the returned slice of UIs
// is intended to be used and the UIs that follow are meant to be attempted only in a fallback posture when there
// are environmental problems (e.g. cannot write to the terminal). A writer is provided to capture the output of
// the final SBOM report.
func Select(verbose, quiet bool, reportWriter io.Writer) (uis []UI) {
isStdoutATty := term.IsTerminal(int(os.Stdout.Fd()))
isStderrATty := term.IsTerminal(int(os.Stderr.Fd()))
notATerminal := !isStderrATty && !isStdoutATty
switch {
case runtime.GOOS == "windows" || verbose || quiet || notATerminal || !isStderrATty:
ui = NewLoggerUI()
uis = append(uis, NewLoggerUI(reportWriter))
default:
ui = NewEphemeralTerminalUI()
uis = append(uis, NewEphemeralTerminalUI(reportWriter), NewLoggerUI(reportWriter))
}
return ui
return uis
}
+1 -5
View File
@@ -45,11 +45,7 @@ func build() *jsonschema.Schema {
reflector := &jsonschema.Reflector{
AllowAdditionalProperties: true,
TypeNamer: func(r reflect.Type) string {
name := r.Name()
if strings.HasPrefix(name, "JSON") {
name = strings.TrimPrefix(name, "JSON")
}
return name
return strings.TrimPrefix(r.Name(), "JSON")
},
}
documentSchema := reflector.ReflectFromType(reflect.TypeOf(&poweruser.JSONDocument{}))
+3 -1
View File
@@ -19,10 +19,12 @@ func NewDistro(t Type, ver, like string) (Distro, error) {
if ver == "" {
return Distro{Type: t}, nil
}
verObj, err := hashiVer.NewVersion(ver)
if err != nil {
return Distro{}, fmt.Errorf("could not create distro version: %w", err)
}
return Distro{
Type: t,
Version: verObj,
@@ -39,7 +41,7 @@ func (d Distro) Name() string {
// MajorVersion returns the major version value from the pseudo-semantically versioned distro version value.
func (d Distro) MajorVersion() string {
if d.Version == nil {
return fmt.Sprint("(version unknown)")
return "(version unknown)"
}
return fmt.Sprintf("%d", d.Version.Segments()[0])
}
+72 -10
View File
@@ -35,6 +35,16 @@ var identityFiles = []parseEntry{
path: "/bin/busybox",
fn: parseBusyBox,
},
{
// check for centos:6
path: "/etc/system-release-cpe",
fn: parseSystemReleaseCPE,
},
{
// last ditch effort for determining older centos version distro information
path: "/etc/redhat-release",
fn: parseRedhatRelease,
},
}
// Identify parses distro-specific files to determine distro metadata like version and release.
@@ -87,23 +97,30 @@ identifyLoop:
return distro
}
func assemble(name, version, like string) *Distro {
distroType, ok := IDMapping[name]
func assemble(id, version, like string) *Distro {
distroType, ok := IDMapping[id]
// Both distro and version must be present
if len(name) == 0 {
if len(id) == 0 && len(version) == 0 {
return nil
}
if ok {
distro, err := NewDistro(distroType, version, like)
if err != nil {
return nil
}
return &distro
// If it's an unknown distro, try mapping the ID_LIKE
if !ok && len(like) != 0 {
distroType, ok = IDMapping[like]
}
return nil
// If we still can't match allow name to be used in constructor
if !ok {
distroType = Type(id)
}
distro, err := NewDistro(distroType, version, like)
if err != nil {
return nil
}
return &distro
}
func parseOsRelease(contents string) *Distro {
@@ -140,3 +157,48 @@ func parseBusyBox(contents string) *Distro {
}
return nil
}
// TODO: we should update parseSystemReleaseCPE to use the CPE struct, pkg.CPE, which requires a refactor to avoid a circular import:
// TODO: pkg depends on distro to support pURLs. To avoid the circular import, either try to make pkg to not depend on distro (medium lift-ish)
// TODO: or migrate the cpe code out of the pkg package (small lift).
// example CPE: cpe:/o:centos:linux:6:GA
var systemReleaseCpeMatcher = regexp.MustCompile(`cpe:\/o:(.*?):.*?:(.*?):.*?$`)
// parseSystemReleaseCPE parses the older centos (6) file to determine distro metadata
func parseSystemReleaseCPE(contents string) *Distro {
matches := systemReleaseCpeMatcher.FindAllStringSubmatch(contents, -1)
for _, match := range matches {
if len(match) < 3 {
log.Warnf("system release cpe does not match expected format")
return nil
}
// note: in SubMatches (capture groups), the 0th index is the full match string
// see https://pkg.go.dev/regexp#pkg-overview for more info
distro := assemble(match[1], match[2], "")
if distro != nil {
return distro
}
}
return nil
}
// example: "CentOS release 6.10 (Final)"
var redhatReleaseMatcher = regexp.MustCompile(`(.*?)\srelease\s(\d\.\d+)`)
// parseRedhatRelease is a fallback parsing method for determining distro information in older redhat versions
func parseRedhatRelease(contents string) *Distro {
matches := redhatReleaseMatcher.FindAllStringSubmatch(contents, -1)
for _, match := range matches {
if len(match) < 3 {
log.Warnf("failed to parse redhat-release file, unexpected format")
return nil
}
// note: in SubMatches (capture groups), the 0th index is the full match string
// see https://pkg.go.dev/regexp#pkg-overview for more info
distro := assemble(strings.ToLower(match[1]), match[2], "")
if distro != nil {
return distro
}
}
return nil
}
+140 -28
View File
@@ -6,12 +6,15 @@ import (
"os"
"testing"
"github.com/anchore/syft/internal"
"github.com/stretchr/testify/assert"
hashiVer "github.com/hashicorp/go-version"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/source"
"github.com/stretchr/testify/assert"
)
const CustomDistro Type = "scientific"
func TestIdentifyDistro(t *testing.T) {
tests := []struct {
fixture string
@@ -68,14 +71,20 @@ func TestIdentifyDistro(t *testing.T) {
Type: UnknownDistroType,
},
{
fixture: "test-fixtures/os/unmatchable",
Type: UnknownDistroType,
fixture: "test-fixtures/os/custom",
Type: CustomDistro,
Version: "8.0.0",
},
{
fixture: "test-fixtures/os/opensuse-leap",
Type: OpenSuseLeap,
Version: "15.2.0",
},
{
fixture: "test-fixtures/os/sles",
Type: SLES,
Version: "15.2.0",
},
{
fixture: "test-fixtures/os/photon",
Type: Photon,
@@ -85,6 +94,35 @@ func TestIdentifyDistro(t *testing.T) {
fixture: "test-fixtures/os/arch",
Type: ArchLinux,
},
{
fixture: "test-fixtures/partial-fields/missing-id",
Type: Debian,
Version: "8.0.0",
},
{
fixture: "test-fixtures/partial-fields/unknown-id",
Type: Debian,
Version: "8.0.0",
},
{
fixture: "test-fixtures/partial-fields/missing-version",
Type: UnknownDistroType,
},
{
fixture: "test-fixtures/os/centos6",
Type: CentOS,
Version: "6.0.0",
},
{
fixture: "test-fixtures/os/centos5",
Type: CentOS,
Version: "5.7.0",
},
{
fixture: "test-fixtures/os/mariner",
Type: Mariner,
Version: "1.0.0",
},
}
observedDistros := internal.NewStringSet()
@@ -187,18 +225,7 @@ func TestParseOsRelease(t *testing.T) {
for _, test := range tests {
name := fmt.Sprintf("%s:%s", test.name, test.RawVersion)
fixture, err := os.Open(test.fixture)
if err != nil {
t.Fatalf("could not open test fixture=%s: %+v", test.fixture, err)
}
defer fixture.Close()
b, err := ioutil.ReadAll(fixture)
if err != nil {
t.Fatalf("unable to read fixture file: %+v", err)
}
contents := string(b)
contents := retrieveFixtureContentsAsString(test.fixture, t)
t.Run(name, func(t *testing.T) {
distro := parseOsRelease(contents)
@@ -227,18 +254,7 @@ func TestParseOsReleaseFailures(t *testing.T) {
for _, test := range tests {
name := fmt.Sprintf("%s:%s", test.name, test.fixture)
fixture, err := os.Open(test.fixture)
if err != nil {
t.Fatalf("could not open test fixture=%s: %+v", test.fixture, err)
}
defer fixture.Close()
b, err := ioutil.ReadAll(fixture)
if err != nil {
t.Fatalf("unable to read fixture file: %+v", err)
}
contents := string(b)
contents := retrieveFixtureContentsAsString(test.fixture, t)
t.Run(name, func(t *testing.T) {
distro := parseOsRelease(contents)
@@ -247,5 +263,101 @@ func TestParseOsReleaseFailures(t *testing.T) {
}
})
}
}
func TestParseSystemReleaseCPE(t *testing.T) {
centos6Version, _ := hashiVer.NewVersion("6")
tests := []struct {
fixture string
name string
expected *Distro
}{
{
fixture: "test-fixtures/os/centos6/etc/system-release-cpe",
name: "Centos 6",
expected: &Distro{
Type: CentOS,
Version: centos6Version,
RawVersion: "6",
},
},
{
fixture: "test-fixtures/bad-system-release-cpe",
name: "Centos 6 Bad CPE",
expected: nil,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
contents := retrieveFixtureContentsAsString(test.fixture, t)
actual := parseSystemReleaseCPE(contents)
if test.expected == nil {
assert.Nil(t, actual)
return
}
// not comparing the full distro object because the hashiVer is a pointer
assert.Equal(t, test.expected.Type, actual.Type)
assert.Equal(t, &test.expected.Version, &actual.Version)
assert.Equal(t, test.expected.RawVersion, actual.RawVersion)
})
}
}
func TestParseRedhatRelease(t *testing.T) {
centos5Version, _ := hashiVer.NewVersion("5.7")
tests := []struct {
fixture string
name string
expected *Distro
}{
{
fixture: "test-fixtures/os/centos5/etc/redhat-release",
name: "Centos 5",
expected: &Distro{
Type: CentOS,
Version: centos5Version,
RawVersion: "5.7",
},
},
{
fixture: "test-fixtures/bad-redhat-release",
name: "Centos 5 Bad Redhat Release",
expected: nil,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
contents := retrieveFixtureContentsAsString(test.fixture, t)
actual := parseRedhatRelease(contents)
if test.expected == nil {
assert.Nil(t, actual)
return
}
// not comparing the full distro object because the hashiVer is a pointer
assert.Equal(t, test.expected.Type, actual.Type)
assert.Equal(t, &test.expected.Version, &actual.Version)
assert.Equal(t, test.expected.RawVersion, actual.RawVersion)
})
}
}
func retrieveFixtureContentsAsString(fixturePath string, t *testing.T) string {
fixture, err := os.Open(fixturePath)
if err != nil {
t.Fatalf("could not open test fixture=%s: %+v", fixturePath, err)
}
defer fixture.Close()
b, err := ioutil.ReadAll(fixture)
if err != nil {
t.Fatalf("unable to read fixture file: %+v", err)
}
return string(b)
}
@@ -0,0 +1 @@
CentOS release 5 (Final)
@@ -0,0 +1 @@
cpe:/o:centos:6:GA
@@ -0,0 +1 @@
CentOS release 5.7 (Final)
@@ -0,0 +1 @@
cpe:/o:centos:linux:6:GA
@@ -0,0 +1,9 @@
NAME="Common Base Linux Mariner"
VERSION="1.0.20210901"
ID=mariner
VERSION_ID=1.0
PRETTY_NAME="CBL-Mariner/Linux"
ANSI_COLOR="1;34"
HOME_URL="https://aka.ms/cbl-mariner"
BUG_REPORT_URL="https://aka.ms/cbl-mariner"
SUPPORT_URL="https://aka.ms/cbl-mariner"
@@ -0,0 +1,9 @@
NAME="SLES"
VERSION="15-SP2"
VERSION_ID="15.2"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP2"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp2"
DOCUMENTATION_URL="https://documentation.suse.com/"
@@ -0,0 +1,3 @@
NAME="Debian GNU/Linux"
VERSION_ID="8"
ID_LIKE=debian
@@ -0,0 +1,2 @@
NAME="Debian GNU/Linux"
ID_LIKE=debian
@@ -0,0 +1,4 @@
NAME="Debian GNU/Linux"
VERSION_ID="8"
ID=my-awesome-distro
ID_LIKE=debian
+6
View File
@@ -17,8 +17,10 @@ const (
OracleLinux Type = "oraclelinux"
ArchLinux Type = "archlinux"
OpenSuseLeap Type = "opensuseleap"
SLES Type = "sles"
Photon Type = "photon"
Windows Type = "windows"
Mariner Type = "mariner"
)
// All contains all Linux distribution options
@@ -34,8 +36,10 @@ var All = []Type{
OracleLinux,
ArchLinux,
OpenSuseLeap,
SLES,
Photon,
Windows,
Mariner,
}
// IDMapping connects a distro ID like "ubuntu" to a Distro type
@@ -51,8 +55,10 @@ var IDMapping = map[string]Type{
"ol": OracleLinux,
"arch": ArchLinux,
"opensuse-leap": OpenSuseLeap,
"sles": SLES,
"photon": Photon,
"windows": Windows,
"mariner": Mariner,
}
// String returns the string representation of the given Linux distribution.
+1 -1
View File
@@ -31,7 +31,7 @@ func (i *ClassificationCataloger) Catalog(resolver source.FileResolver) (map[sou
}
}
}
log.Debugf("classification cataloger discovered %d results", numResults)
log.Debugf("file classifier discovered %d results", numResults)
return results, nil
}
+9 -2
View File
@@ -118,11 +118,18 @@ func TestClassifierCataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
loc := source.NewLocation(test.location)
if _, ok := actualResults[loc]; !ok {
ok := false
for actual_loc, actual_classification := range actualResults {
if loc.RealPath == actual_loc.RealPath {
ok = true
assert.Equal(t, test.expected, actual_classification)
}
}
if !ok {
t.Fatalf("could not find test location=%q", test.location)
}
assert.Equal(t, test.expected, actualResults[loc])
})
}
}
+5 -3
View File
@@ -3,7 +3,6 @@ package file
import (
"bytes"
"encoding/base64"
"fmt"
"io"
"github.com/anchore/syft/internal"
@@ -32,7 +31,6 @@ func (i *ContentsCataloger) Catalog(resolver source.FileResolver) (map[source.Lo
if err != nil {
return nil, err
}
for _, location := range locations {
metadata, err := resolver.FileMetadataByLocation(location)
if err != nil {
@@ -44,6 +42,10 @@ func (i *ContentsCataloger) Catalog(resolver source.FileResolver) (map[source.Lo
}
result, err := i.catalogLocation(resolver, location)
if internal.IsErrPathPermission(err) {
log.Debugf("file contents cataloger skipping - %+v", err)
continue
}
if err != nil {
return nil, err
}
@@ -63,7 +65,7 @@ func (i *ContentsCataloger) catalogLocation(resolver source.FileResolver, locati
buf := &bytes.Buffer{}
if _, err = io.Copy(base64.NewEncoder(base64.StdEncoding, buf), contentReader); err != nil {
return "", fmt.Errorf("unable to observe contents of %+v: %w", location.RealPath, err)
return "", internal.ErrPath{Path: location.RealPath, Err: err}
}
return buf.String(), nil
+7 -2
View File
@@ -39,6 +39,11 @@ func (i *DigestsCataloger) Catalog(resolver source.FileResolver) (map[source.Loc
for _, location := range locations {
stage.Current = location.RealPath
result, err := i.catalogLocation(resolver, location)
if internal.IsErrPathPermission(err) {
log.Debugf("file digests cataloger skipping - %+v", err)
continue
}
if err != nil {
return nil, err
}
@@ -67,7 +72,7 @@ func (i *DigestsCataloger) catalogLocation(resolver source.FileResolver, locatio
size, err := io.Copy(io.MultiWriter(writers...), contentReader)
if err != nil {
return nil, fmt.Errorf("unable to observe contents of %+v: %+v", location.RealPath, err)
return nil, internal.ErrPath{Path: location.RealPath, Err: err}
}
if size == 0 {
@@ -94,7 +99,7 @@ func DigestAlgorithmName(hash crypto.Hash) string {
func CleanDigestAlgorithmName(name string) string {
lower := strings.ToLower(name)
return strings.Replace(lower, "-", "", -1)
return strings.ReplaceAll(lower, "-", "")
}
func digestsCatalogingProgress(locations int64) (*progress.Stage, *progress.Manual) {
+28 -21
View File
@@ -49,11 +49,12 @@ func TestFileMetadataCataloger(t *testing.T) {
path: "/file-1.txt",
exists: true,
expected: source.FileMetadata{
Mode: 0644,
Type: "RegularFile",
UserID: 1,
GroupID: 2,
Size: 7,
Mode: 0644,
Type: "RegularFile",
UserID: 1,
GroupID: 2,
Size: 7,
MIMEType: "text/plain",
},
},
{
@@ -65,6 +66,7 @@ func TestFileMetadataCataloger(t *testing.T) {
LinkDestination: "file-1.txt",
UserID: 1,
GroupID: 2,
MIMEType: "",
},
},
{
@@ -76,46 +78,51 @@ func TestFileMetadataCataloger(t *testing.T) {
LinkDestination: "file-1.txt",
UserID: 0,
GroupID: 0,
MIMEType: "",
},
},
{
path: "/char-device-1",
exists: true,
expected: source.FileMetadata{
Mode: 0644 | os.ModeDevice | os.ModeCharDevice,
Type: "CharacterDevice",
UserID: 0,
GroupID: 0,
Mode: 0644 | os.ModeDevice | os.ModeCharDevice,
Type: "CharacterDevice",
UserID: 0,
GroupID: 0,
MIMEType: "",
},
},
{
path: "/block-device-1",
exists: true,
expected: source.FileMetadata{
Mode: 0644 | os.ModeDevice,
Type: "BlockDevice",
UserID: 0,
GroupID: 0,
Mode: 0644 | os.ModeDevice,
Type: "BlockDevice",
UserID: 0,
GroupID: 0,
MIMEType: "",
},
},
{
path: "/fifo-1",
exists: true,
expected: source.FileMetadata{
Mode: 0644 | os.ModeNamedPipe,
Type: "FIFONode",
UserID: 0,
GroupID: 0,
Mode: 0644 | os.ModeNamedPipe,
Type: "FIFONode",
UserID: 0,
GroupID: 0,
MIMEType: "",
},
},
{
path: "/bin",
exists: true,
expected: source.FileMetadata{
Mode: 0755 | os.ModeDir,
Type: "Directory",
UserID: 0,
GroupID: 0,
Mode: 0755 | os.ModeDir,
Type: "Directory",
UserID: 0,
GroupID: 0,
MIMEType: "",
},
},
}
+6 -1
View File
@@ -50,6 +50,11 @@ func (i *SecretsCataloger) Catalog(resolver source.FileResolver) (map[source.Loc
for _, location := range locations {
stage.Current = location.RealPath
result, err := i.catalogLocation(resolver, location)
if internal.IsErrPathPermission(err) {
log.Debugf("secrets cataloger skipping - %+v", err)
continue
}
if err != nil {
return nil, err
}
@@ -77,7 +82,7 @@ func (i *SecretsCataloger) catalogLocation(resolver source.FileResolver, locatio
// TODO: in the future we can swap out search strategies here
secrets, err := catalogLocationByLine(resolver, location, i.patterns)
if err != nil {
return nil, err
return nil, internal.ErrPath{Path: location.RealPath, Err: err}
}
if i.revealValues {
+1 -1
View File
@@ -32,7 +32,7 @@ import (
// CatalogPackages takes an inventory of packages from the given image from a particular perspective
// (e.g. squashed source, all-layers source). Returns the discovered set of packages, the identified Linux
// distribution, and the source object used to wrap the data source.
func CatalogPackages(src source.Source, scope source.Scope) (*pkg.Catalog, *distro.Distro, error) {
func CatalogPackages(src *source.Source, scope source.Scope) (*pkg.Catalog, *distro.Distro, error) {
resolver, err := src.FileResolver(scope)
if err != nil {
return nil, nil, fmt.Errorf("unable to determine resolver while cataloging packages: %w", err)
+9 -9
View File
@@ -5,13 +5,13 @@ import (
"github.com/anchore/syft/syft/file"
"github.com/package-url/packageurl-go"
"github.com/anchore/packageurl-go"
"github.com/scylladb/go-set/strset"
)
const ApkDbGlob = "**/lib/apk/db/installed"
const ApkDBGlob = "**/lib/apk/db/installed"
var _ fileOwner = (*ApkMetadata)(nil)
var _ FileOwner = (*ApkMetadata)(nil)
// ApkMetadata represents all captured data for a Alpine DB package entry.
// See the following sources for more information:
@@ -37,11 +37,11 @@ type ApkMetadata struct {
// ApkFileRecord represents a single file listing and metadata from a APK DB entry (which may have many of these file records).
type ApkFileRecord struct {
Path string `json:"path"`
OwnerUID string `json:"ownerUid,omitempty"`
OwnerGID string `json:"ownerGid,omitempty"`
Permissions string `json:"permissions,omitempty"`
Digest file.Digest `json:"digest,omitempty"`
Path string `json:"path"`
OwnerUID string `json:"ownerUid,omitempty"`
OwnerGID string `json:"ownerGid,omitempty"`
Permissions string `json:"permissions,omitempty"`
Digest *file.Digest `json:"digest,omitempty"`
}
// PackageURL returns the PURL for the specific Alpine package (see https://github.com/package-url/purl-spec)
@@ -63,7 +63,7 @@ func (m ApkMetadata) PackageURL() string {
return pURL.ToString()
}
func (m ApkMetadata) ownedFiles() (result []string) {
func (m ApkMetadata) OwnedFiles() (result []string) {
s := strset.New()
for _, f := range m.Files {
if f.Path != "" {
+40 -2
View File
@@ -4,6 +4,7 @@ import (
"strings"
"testing"
"github.com/anchore/packageurl-go"
"github.com/go-test/deep"
"github.com/sergi/go-diff/diffmatchpatch"
)
@@ -21,6 +22,23 @@ func TestApkMetadata_pURL(t *testing.T) {
},
expected: "pkg:alpine/p@v?arch=a",
},
// verify #351
{
metadata: ApkMetadata{
Package: "g++",
Version: "v84",
Architecture: "am86",
},
expected: "pkg:alpine/g++@v84?arch=am86",
},
{
metadata: ApkMetadata{
Package: "g plus plus",
Version: "v84",
Architecture: "am86",
},
expected: "pkg:alpine/g%20plus%20plus@v84?arch=am86",
},
}
for _, test := range tests {
@@ -31,11 +49,31 @@ func TestApkMetadata_pURL(t *testing.T) {
diffs := dmp.DiffMain(test.expected, actual, true)
t.Errorf("diff: %s", dmp.DiffPrettyText(diffs))
}
// verify packageurl can parse
purl, err := packageurl.FromString(actual)
if err != nil {
t.Errorf("cannot re-parse purl: %s", actual)
}
if purl.Name != test.metadata.Package {
dmp := diffmatchpatch.New()
diffs := dmp.DiffMain(test.metadata.Package, purl.Name, true)
t.Errorf("invalid purl name: %s", dmp.DiffPrettyText(diffs))
}
if purl.Version != test.metadata.Version {
dmp := diffmatchpatch.New()
diffs := dmp.DiffMain(test.metadata.Version, purl.Version, true)
t.Errorf("invalid purl version: %s", dmp.DiffPrettyText(diffs))
}
if purl.Qualifiers.Map()["arch"] != test.metadata.Architecture {
dmp := diffmatchpatch.New()
diffs := dmp.DiffMain(test.metadata.Architecture, purl.Qualifiers.Map()["arch"], true)
t.Errorf("invalid purl architecture: %s", dmp.DiffPrettyText(diffs))
}
})
}
}
func TestApkMetadata_fileOwner(t *testing.T) {
func TestApkMetadata_FileOwner(t *testing.T) {
tests := []struct {
metadata ApkMetadata
expected []string
@@ -69,7 +107,7 @@ func TestApkMetadata_fileOwner(t *testing.T) {
t.Run(strings.Join(test.expected, ","), func(t *testing.T) {
var i interface{}
i = test.metadata
actual := i.(fileOwner).ownedFiles()
actual := i.(FileOwner).OwnedFiles()
for _, d := range deep.Equal(test.expected, actual) {
t.Errorf("diff: %+v", d)
}
+1 -1
View File
@@ -11,7 +11,7 @@ import (
// NewApkdbCataloger returns a new Alpine DB cataloger object.
func NewApkdbCataloger() *common.GenericCataloger {
globParsers := map[string]common.ParserFn{
pkg.ApkDbGlob: parseApkDB,
pkg.ApkDBGlob: parseApkDB,
}
return common.NewGenericCataloger(nil, globParsers, "apkdb-cataloger")
+1 -1
View File
@@ -128,7 +128,7 @@ func parseApkDBEntry(reader io.Reader) (*pkg.ApkMetadata, error) {
log.Warnf("checksum field with no parent record: %q", value)
continue
}
fileRecord.Digest = file.Digest{
fileRecord.Digest = &file.Digest{
Algorithm: "sha1",
Value: value,
}
+38 -38
View File
@@ -41,7 +41,7 @@ func TestExtraFileAttributes(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1M0C9qfC/+kdRiOodeihG2GMRtkE=",
},
@@ -110,7 +110,7 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1Kja2+POZKxEkUOZqwSjC6kmaED4=",
},
@@ -126,7 +126,7 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1CVmFbdY+Hv6/jAHl1gec2Kbx1EY=",
},
@@ -136,7 +136,7 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1yFAhGggmL7ERgbIA7KQxyTzf3ks=",
},
@@ -146,7 +146,7 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1dAdYK8M/INibRQF5B3Rw7cmNDDA=",
},
@@ -156,7 +156,7 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1eR2Dz/WylabgbWMTkd2+hGmEya4=",
},
@@ -195,49 +195,49 @@ func TestSinglePackageDetails(t *testing.T) {
},
{
Path: "/etc/fstab",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q11Q7hNe8QpDS531guqCdrXBzoA/o=",
},
},
{
Path: "/etc/group",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1oJ16xWudgKOrXIEquEDzlF2Lsm4=",
},
},
{
Path: "/etc/hostname",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q16nVwYVXP/tChvUPdukVD2ifXOmc=",
},
},
{
Path: "/etc/hosts",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1BD6zJKZTRWyqGnPi4tSfd3krsMU=",
},
},
{
Path: "/etc/inittab",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1TsthbhW7QzWRe1E/NKwTOuD4pHc=",
},
},
{
Path: "/etc/modules",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1toogjUipHGcMgECgPJX64SwUT1M=",
},
},
{
Path: "/etc/motd",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1XmduVVNURHQ27TvYp1Lr5TMtFcA=",
},
@@ -247,35 +247,35 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "777",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1kiljhXXH1LlQroHsEJIkPZg2eiw=",
},
},
{
Path: "/etc/passwd",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1TchuuLUfur0izvfZQZxgN/LJhB8=",
},
},
{
Path: "/etc/profile",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1KpFb8kl5LvwXWlY3e58FNsjrI34=",
},
},
{
Path: "/etc/protocols",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q13FqXUnvuOpMDrH/6rehxuYAEE34=",
},
},
{
Path: "/etc/services",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1C6HJNgQvLWqt5VY+n7MZJ1rsDuY=",
},
@@ -285,21 +285,21 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "42",
Permissions: "640",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1ltrPIAW2zHeDiajsex2Bdmq3uqA=",
},
},
{
Path: "/etc/shells",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1ojm2YdpCJ6B/apGDaZ/Sdb2xJkA=",
},
},
{
Path: "/etc/sysctl.conf",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q14upz3tfnNxZkIEsUhWn7Xoiw96g=",
},
@@ -318,7 +318,7 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "600",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1vfk1apUWI4yLJGhhNRd0kJixfvY=",
},
@@ -331,28 +331,28 @@ func TestSinglePackageDetails(t *testing.T) {
},
{
Path: "/etc/modprobe.d/aliases.conf",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1WUbh6TBYNVK7e4Y+uUvLs/7viqk=",
},
},
{
Path: "/etc/modprobe.d/blacklist.conf",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1xxYGU6S6TLQvb7ervPrWWwAWqMg=",
},
},
{
Path: "/etc/modprobe.d/i386.conf",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1pnay/njn6ol9cCssL7KiZZ8etlc=",
},
},
{
Path: "/etc/modprobe.d/kms.conf",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1ynbLn3GYDpvajba/ldp1niayeog=",
},
@@ -401,14 +401,14 @@ func TestSinglePackageDetails(t *testing.T) {
},
{
Path: "/etc/profile.d/color_prompt",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q10wL23GuSCVfumMRgakabUI6EsSk=",
},
},
{
Path: "/etc/profile.d/locale",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1R4bIEpnKxxOSrlnZy9AoawqZ5DU=",
},
@@ -436,7 +436,7 @@ func TestSinglePackageDetails(t *testing.T) {
},
{
Path: "/lib/sysctl.d/00-alpine.conf",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1HpElzW1xEgmKfERtTy7oommnq6c=",
},
@@ -479,7 +479,7 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1YeuSmC7iDbEWrusPzA/zUQF6YSg=",
},
@@ -537,7 +537,7 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "777",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q11/SNZz/8cK2dSKK+cJpVrZIuF4Q=",
},
@@ -586,7 +586,7 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "777",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1dzbdazYZA2nTzSIG3YyNw7d4Juc=",
},
@@ -599,7 +599,7 @@ func TestSinglePackageDetails(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "777",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1OFZt+ZMp7j0Gny0rqSKuWJyqYmA=",
},
@@ -704,7 +704,7 @@ func TestMultiplePackages(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1Kja2+POZKxEkUOZqwSjC6kmaED4=",
},
@@ -720,7 +720,7 @@ func TestMultiplePackages(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1CVmFbdY+Hv6/jAHl1gec2Kbx1EY=",
},
@@ -730,7 +730,7 @@ func TestMultiplePackages(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1yFAhGggmL7ERgbIA7KQxyTzf3ks=",
},
@@ -740,7 +740,7 @@ func TestMultiplePackages(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1dAdYK8M/INibRQF5B3Rw7cmNDDA=",
},
@@ -750,7 +750,7 @@ func TestMultiplePackages(t *testing.T) {
OwnerUID: "0",
OwnerGID: "0",
Permissions: "755",
Digest: file.Digest{
Digest: &file.Digest{
Algorithm: "sha1",
Value: "Q1eR2Dz/WylabgbWMTkd2+hGmEya4=",
},
+2 -1
View File
@@ -6,6 +6,7 @@ import (
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/common/cpe"
"github.com/anchore/syft/syft/source"
"github.com/hashicorp/go-multierror"
"github.com/wagoodman/go-partybus"
@@ -59,7 +60,7 @@ func Catalog(resolver source.FileResolver, theDistro *distro.Distro, catalogers
for _, p := range packages {
// generate CPEs
p.CPEs = generatePackageCPEs(p)
p.CPEs = cpe.Generate(p)
// generate PURL
p.PURL = generatePackageURL(p, theDistro)
+3 -3
View File
@@ -39,8 +39,7 @@ func ImageCatalogers() []Cataloger {
rpmdb.NewRpmdbCataloger(),
java.NewJavaCataloger(),
apkdb.NewApkdbCataloger(),
golang.NewGoModCataloger(),
rust.NewCargoLockCataloger(),
golang.NewGoModuleBinaryCataloger(),
}
}
@@ -55,7 +54,8 @@ func DirectoryCatalogers() []Cataloger {
rpmdb.NewRpmdbCataloger(),
java.NewJavaCataloger(),
apkdb.NewApkdbCataloger(),
golang.NewGoModCataloger(),
golang.NewGoModuleBinaryCataloger(),
golang.NewGoModFileCataloger(),
rust.NewCargoLockCataloger(),
}
}
@@ -0,0 +1,188 @@
package cpe
import (
"github.com/anchore/syft/syft/pkg"
)
// candidateComposite is a convenience when creating the defaultCandidateAdditions set
type candidateComposite struct {
pkg.Type
candidateKey
candidateAddition
}
// defaultCandidateAdditions is all of the known cases for product and vendor field values that should be used when
// select package information is discovered
var defaultCandidateAdditions = buildCandidateLookup(
[]candidateComposite{
// Java packages
{
pkg.JavaPkg,
candidateKey{PkgName: "springframework"},
candidateAddition{AdditionalProducts: []string{"spring_framework", "springsource_spring_framework"}},
},
{
pkg.JavaPkg,
candidateKey{PkgName: "spring-core"},
candidateAddition{AdditionalProducts: []string{"spring_framework", "springsource_spring_framework"}},
},
{
// example image: docker.io/nuxeo:latest
pkg.JavaPkg,
candidateKey{PkgName: "elasticsearch"}, // , Vendor: "elasticsearch"},
candidateAddition{AdditionalVendors: []string{"elastic"}},
},
{
// example image: docker.io/kaazing-gateway:latest
pkg.JavaPkg,
candidateKey{PkgName: "log4j"}, // , Vendor: "apache-software-foundation"},
candidateAddition{AdditionalVendors: []string{"apache"}},
},
{
// example image: cassandra:latest
pkg.JavaPkg,
candidateKey{PkgName: "apache-cassandra"}, // , Vendor: "apache"},
candidateAddition{AdditionalProducts: []string{"cassandra"}},
},
// NPM packages
{
pkg.NpmPkg,
candidateKey{PkgName: "hapi"},
candidateAddition{AdditionalProducts: []string{"hapi_server_framework"}},
},
{
pkg.NpmPkg,
candidateKey{PkgName: "handlebars.js"},
candidateAddition{AdditionalProducts: []string{"handlebars"}},
},
{
pkg.NpmPkg,
candidateKey{PkgName: "is-my-json-valid"},
candidateAddition{AdditionalProducts: []string{"is_my_json_valid"}},
},
{
pkg.NpmPkg,
candidateKey{PkgName: "mustache"},
candidateAddition{AdditionalProducts: []string{"mustache.js"}},
},
// Gem packages
{
pkg.GemPkg,
candidateKey{PkgName: "Arabic-Prawn"},
candidateAddition{AdditionalProducts: []string{"arabic_prawn"}},
},
{
pkg.GemPkg,
candidateKey{PkgName: "bio-basespace-sdk"},
candidateAddition{AdditionalProducts: []string{"basespace_ruby_sdk"}},
},
{
pkg.GemPkg,
candidateKey{PkgName: "cremefraiche"},
candidateAddition{AdditionalProducts: []string{"creme_fraiche"}},
},
{
pkg.GemPkg,
candidateKey{PkgName: "html-sanitizer"},
candidateAddition{AdditionalProducts: []string{"html_sanitizer"}},
},
{
pkg.GemPkg,
candidateKey{PkgName: "sentry-raven"},
candidateAddition{AdditionalProducts: []string{"raven-ruby"}},
},
{
pkg.GemPkg,
candidateKey{PkgName: "RedCloth"},
candidateAddition{AdditionalProducts: []string{"redcloth_library"}},
},
{
pkg.GemPkg,
candidateKey{PkgName: "VladTheEnterprising"},
candidateAddition{AdditionalProducts: []string{"vladtheenterprising"}},
},
{
pkg.GemPkg,
candidateKey{PkgName: "yajl-ruby"},
candidateAddition{AdditionalProducts: []string{"yajl-ruby_gem"}},
},
// Python packages
{
pkg.PythonPkg,
candidateKey{PkgName: "python-rrdtool"},
candidateAddition{AdditionalProducts: []string{"rrdtool"}},
},
})
// buildCandidateLookup is a convenience function for creating the defaultCandidateAdditions set
func buildCandidateLookup(cc []candidateComposite) (ca map[pkg.Type]map[candidateKey]candidateAddition) {
ca = make(map[pkg.Type]map[candidateKey]candidateAddition)
for _, c := range cc {
if _, ok := ca[c.Type]; !ok {
ca[c.Type] = make(map[candidateKey]candidateAddition)
}
ca[c.Type][c.candidateKey] = c.candidateAddition
}
return ca
}
// candidateKey represents the set of inputs that should be matched on in order to signal more candidate additions to be used.
type candidateKey struct {
Vendor string
PkgName string
}
// candidateAddition are the specific additions that should be considered during CPE generation (given a specific candidateKey)
type candidateAddition struct {
AdditionalProducts []string
AdditionalVendors []string
}
// findAdditionalVendors searches all possible vendor additions that could be added during the CPE generation process (given package info + a vendor candidate)
func findAdditionalVendors(allAdditions map[pkg.Type]map[candidateKey]candidateAddition, ty pkg.Type, pkgName, vendor string) (vendors []string) {
additions, ok := allAdditions[ty]
if !ok {
return nil
}
if addition, ok := additions[candidateKey{
Vendor: vendor,
PkgName: pkgName,
}]; ok {
vendors = append(vendors, addition.AdditionalVendors...)
}
if addition, ok := additions[candidateKey{
PkgName: pkgName,
}]; ok {
vendors = append(vendors, addition.AdditionalVendors...)
}
if addition, ok := additions[candidateKey{
Vendor: vendor,
}]; ok {
vendors = append(vendors, addition.AdditionalVendors...)
}
return vendors
}
// findAdditionalProducts searches all possible product additions that could be added during the CPE generation process (given package info)
func findAdditionalProducts(allAdditions map[pkg.Type]map[candidateKey]candidateAddition, ty pkg.Type, pkgName string) (products []string) {
additions, ok := allAdditions[ty]
if !ok {
return nil
}
if addition, ok := additions[candidateKey{
PkgName: pkgName,
}]; ok {
products = append(products, addition.AdditionalProducts...)
}
return products
}
@@ -0,0 +1,155 @@
package cpe
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_additionalProducts(t *testing.T) {
tests := []struct {
name string
allAdditions map[pkg.Type]map[candidateKey]candidateAddition
ty pkg.Type
pkgName string
expected []string
}{
{
name: "product name addition",
allAdditions: map[pkg.Type]map[candidateKey]candidateAddition{
pkg.JavaPkg: {
candidateKey{
PkgName: "spring-core",
}: {
AdditionalProducts: []string{"spring_framework", "springsource_spring_framework"},
},
},
},
ty: pkg.JavaPkg,
pkgName: "spring-core",
expected: []string{"spring_framework", "springsource_spring_framework"},
},
{
name: "no addition found",
allAdditions: map[pkg.Type]map[candidateKey]candidateAddition{
pkg.JavaPkg: {
candidateKey{
PkgName: "spring-core",
}: {
AdditionalProducts: []string{"spring_framework", "springsource_spring_framework"},
},
},
},
ty: pkg.JavaPkg,
pkgName: "nothing",
expected: nil,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, findAdditionalProducts(test.allAdditions, test.ty, test.pkgName))
})
}
}
func Test_additionalVendors(t *testing.T) {
tests := []struct {
name string
allAdditions map[pkg.Type]map[candidateKey]candidateAddition
ty pkg.Type
pkgName string
vendor string
expected []string
}{
{
name: "vendor addition by input vendor",
allAdditions: map[pkg.Type]map[candidateKey]candidateAddition{
pkg.JavaPkg: {
candidateKey{
Vendor: "my-vendor",
}: {
AdditionalVendors: []string{"awesome-vendor-addition"},
},
// note: the below keys should not be matched
candidateKey{
PkgName: "my-package-name",
Vendor: "my-vendor",
}: {
AdditionalVendors: []string{"bad-addition"},
},
candidateKey{
PkgName: "my-package-name",
}: {
AdditionalVendors: []string{"bad-addition"},
},
},
},
ty: pkg.JavaPkg,
pkgName: "NOT-MY-PACKAGE",
vendor: "my-vendor",
expected: []string{"awesome-vendor-addition"},
},
{
name: "vendor addition by input package name",
allAdditions: map[pkg.Type]map[candidateKey]candidateAddition{
pkg.JavaPkg: {
candidateKey{
PkgName: "my-package-name",
}: {
AdditionalVendors: []string{"awesome-vendor-addition"},
},
// note: the below keys should not be matched
candidateKey{
PkgName: "my-package-name",
Vendor: "my-vendor",
}: {
AdditionalVendors: []string{"bad-addition"},
},
candidateKey{
Vendor: "my-vendor",
}: {
AdditionalVendors: []string{"bad-addition"},
},
},
},
ty: pkg.JavaPkg,
pkgName: "my-package-name",
vendor: "NOT-MY-VENDOR",
expected: []string{"awesome-vendor-addition"},
},
{
name: "vendor addition by input package name + vendor",
allAdditions: map[pkg.Type]map[candidateKey]candidateAddition{
pkg.JavaPkg: {
candidateKey{
PkgName: "my-package-name",
Vendor: "my-vendor",
}: {
AdditionalVendors: []string{"awesome-vendor-addition"},
},
// note: the below keys should not be matched
candidateKey{
PkgName: "my-package-name",
}: {
AdditionalVendors: []string{"one-good-addition"},
},
candidateKey{
Vendor: "my-vendor",
}: {
AdditionalVendors: []string{"another-good-addition"},
},
},
},
ty: pkg.JavaPkg,
pkgName: "my-package-name",
vendor: "my-vendor",
expected: []string{"awesome-vendor-addition", "one-good-addition", "another-good-addition"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, findAdditionalVendors(test.allAdditions, test.ty, test.pkgName, test.vendor))
})
}
}
@@ -0,0 +1,112 @@
package cpe
import (
"strconv"
"github.com/scylladb/go-set/strset"
)
// fieldCandidate represents a single "guess" for a specific field in a future CPE (vendor, product, target SW, etc).
// When generating these candidates depending on the field the value was sourced from there may be only a subset of
// transforms that should be applied (downstream of extraction). Expressing candidates in this struct allows for this
// flexibility such that downstream transforms can be elected into or skipped over.
type fieldCandidate struct {
value string
disallowSubSelections bool
disallowDelimiterVariations bool
}
type fieldCandidateSet map[fieldCandidate]struct{}
func newFieldCandidateSetFromSets(sets ...fieldCandidateSet) fieldCandidateSet {
s := newFieldCandidateSet()
for _, set := range sets {
s.add(set.list()...)
}
return s
}
func newFieldCandidateSet(values ...string) fieldCandidateSet {
s := make(fieldCandidateSet)
s.addValue(values...)
return s
}
func (s fieldCandidateSet) addValue(values ...string) {
for _, value := range values {
// default candidate as an allow-all
candidate := fieldCandidate{
value: cleanCandidateField(value),
}
s[candidate] = struct{}{}
}
}
func (s fieldCandidateSet) add(candidates ...fieldCandidate) {
for _, candidate := range candidates {
candidate.value = cleanCandidateField(candidate.value)
s[candidate] = struct{}{}
}
}
func (s fieldCandidateSet) removeByValue(values ...string) {
for _, value := range values {
s.removeWhere(valueEquals(value))
}
}
// removeWhere removes all entries from the fieldCandidateSet for which the condition function returns true.
func (s fieldCandidateSet) removeWhere(condition fieldCandidateCondition) {
for candidate := range s {
if condition(candidate) {
delete(s, candidate)
}
}
}
func (s fieldCandidateSet) clear() {
for k := range s {
delete(s, k)
}
}
func (s fieldCandidateSet) union(others ...fieldCandidateSet) {
for _, other := range others {
s.add(other.list()...)
}
}
func (s fieldCandidateSet) list() (results []fieldCandidate) {
for c := range s {
results = append(results, c)
}
return results
}
func (s fieldCandidateSet) values() (results []string) {
for _, c := range s.list() {
results = append(results, c.value)
}
return results
}
func (s fieldCandidateSet) uniqueValues() []string {
return strset.New(s.values()...).List()
}
func (s fieldCandidateSet) copy() fieldCandidateSet {
newSet := newFieldCandidateSet()
newSet.add(s.list()...)
return newSet
}
func cleanCandidateField(field string) string {
cleanedValue, err := strconv.Unquote(field)
if err != nil {
return field
}
return cleanedValue
}
@@ -0,0 +1,18 @@
package cpe
// A fieldCandidateCondition returns true if the condition is true for a given fieldCandidate.
type fieldCandidateCondition func(fieldCandidate) bool
func subSelectionsDisallowed(c fieldCandidate) bool {
return c.disallowSubSelections
}
func delimiterVariationsDisallowed(c fieldCandidate) bool {
return c.disallowDelimiterVariations
}
func valueEquals(v string) fieldCandidateCondition {
return func(candidate fieldCandidate) bool {
return candidate.value == v
}
}
@@ -0,0 +1,350 @@
package cpe
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
func Test_cpeCandidateValues_filter(t *testing.T) {
tests := []struct {
name string
input []fieldCandidate
exclusionConditions []fieldCandidateCondition
expect []string
}{
{
name: "gocase",
input: []fieldCandidate{
{
value: "allow anything",
},
{
value: "no-sub-selections",
disallowSubSelections: true,
},
{
value: "no-delimiter-variations",
disallowDelimiterVariations: true,
},
{
value: "allow nothing",
disallowSubSelections: true,
disallowDelimiterVariations: true,
},
},
expect: []string{
"allow anything",
"no-sub-selections",
"no-delimiter-variations",
"allow nothing",
},
},
{
name: "filter out sub-selections",
input: []fieldCandidate{
{
value: "allow anything",
},
{
value: "no-sub-selections",
disallowSubSelections: true,
},
{
value: "no-delimiter-variations",
disallowDelimiterVariations: true,
},
{
value: "allow nothing",
disallowSubSelections: true,
disallowDelimiterVariations: true,
},
},
exclusionConditions: []fieldCandidateCondition{
subSelectionsDisallowed,
},
expect: []string{
"allow anything",
"no-delimiter-variations",
},
},
{
name: "filter out delimiter-variations",
input: []fieldCandidate{
{
value: "allow anything",
},
{
value: "no-sub-selections",
disallowSubSelections: true,
},
{
value: "no-delimiter-variations",
disallowDelimiterVariations: true,
},
{
value: "allow nothing",
disallowSubSelections: true,
disallowDelimiterVariations: true,
},
},
exclusionConditions: []fieldCandidateCondition{
delimiterVariationsDisallowed,
},
expect: []string{
"allow anything",
"no-sub-selections",
},
},
{
name: "all exclusionConditions",
input: []fieldCandidate{
{
value: "allow anything",
},
{
value: "no-sub-selections",
disallowSubSelections: true,
},
{
value: "no-delimiter-variations",
disallowDelimiterVariations: true,
},
{
value: "allow nothing",
disallowSubSelections: true,
disallowDelimiterVariations: true,
},
},
exclusionConditions: []fieldCandidateCondition{
delimiterVariationsDisallowed,
subSelectionsDisallowed,
},
expect: []string{
"allow anything",
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
set := newFieldCandidateSet()
set.add(test.input...)
for _, condition := range test.exclusionConditions {
set.removeWhere(condition)
}
assert.ElementsMatch(t, test.expect, set.values())
})
}
}
func Test_cpeFieldCandidateSet_addValue(t *testing.T) {
s := newFieldCandidateSet()
// we should clean all values (unquote strings)
s.addValue(`"string!"`)
assert.ElementsMatch(t, []string{"string!"}, s.values())
}
func Test_cpeFieldCandidateSet_add(t *testing.T) {
s := newFieldCandidateSet()
// we should clean all values (unquote strings)
s.add(fieldCandidate{
value: `"string!"`,
})
assert.ElementsMatch(t, []string{"string!"}, s.values())
}
func Test_cpeFieldCandidateSet_clear(t *testing.T) {
s := newFieldCandidateSet("1", "2")
assert.NotEmpty(t, s.values())
s.clear()
assert.Empty(t, s.values())
}
func Test_cpeFieldCandidateSet_union(t *testing.T) {
s1 := newFieldCandidateSet("1", "2")
assert.Len(t, s1.list(), 2)
s2 := newFieldCandidateSet("2", "3", "4")
assert.Len(t, s2.list(), 3)
s3 := newFieldCandidateSet()
s3.add(
fieldCandidate{
value: "1",
disallowSubSelections: true,
disallowDelimiterVariations: false,
},
fieldCandidate{
value: "4",
disallowSubSelections: false,
disallowDelimiterVariations: true,
},
fieldCandidate{
value: "5",
disallowSubSelections: true,
disallowDelimiterVariations: true,
},
)
assert.Len(t, s3.list(), 3)
s1.union(s2, s3)
// 1 & 4 have duplicate entries since there are candidate conditions set
assert.ElementsMatch(t, s1.values(), []string{"1", "1", "2", "3", "4", "4", "5"})
assert.ElementsMatch(t, s1.list(), []fieldCandidate{
{
value: "1",
},
{
value: "1",
disallowSubSelections: true,
disallowDelimiterVariations: false,
},
{
value: "2",
},
{
value: "3",
},
{
value: "4",
},
{
value: "4",
disallowSubSelections: false,
disallowDelimiterVariations: true,
},
{
value: "5",
disallowSubSelections: true,
disallowDelimiterVariations: true,
},
})
}
func Test_cpeFieldCandidateSet_union_byValue(t *testing.T) {
s1 := newFieldCandidateSet("1", "2")
assert.Len(t, s1.list(), 2)
s2 := newFieldCandidateSet("2", "3", "4")
assert.Len(t, s2.list(), 3)
s3 := newFieldCandidateSet("1", "4", "5")
assert.Len(t, s3.list(), 3)
s1.union(s2, s3)
assert.ElementsMatch(t, s1.values(), []string{"1", "2", "3", "4", "5"})
assert.ElementsMatch(t, s1.list(), []fieldCandidate{
{
value: "1",
},
{
value: "2",
},
{
value: "3",
},
{
value: "4",
},
{
value: "5",
},
})
}
func Test_cpeFieldCandidateSet_uniqueValues(t *testing.T) {
set := newFieldCandidateSet()
set.add(
fieldCandidate{
value: "1",
},
fieldCandidate{
value: "1",
disallowSubSelections: true,
},
fieldCandidate{
value: "2",
disallowDelimiterVariations: true,
},
fieldCandidate{
value: "2",
},
fieldCandidate{
value: "3",
disallowSubSelections: true,
disallowDelimiterVariations: true,
},
)
assert.ElementsMatch(t, []string{"1", "2", "3"}, set.uniqueValues())
}
func Test_cpeFieldCandidateSet_removeByValue(t *testing.T) {
s := newFieldCandidateSet()
// should be removed
s.add(fieldCandidate{
value: "1",
disallowSubSelections: true,
disallowDelimiterVariations: true,
})
s.add(fieldCandidate{
value: "1",
disallowSubSelections: true,
})
s.add(fieldCandidate{
value: "1",
disallowDelimiterVariations: true,
})
s.add(fieldCandidate{
value: "1",
})
// should not be removed
s.add(fieldCandidate{
value: "2",
})
assert.Len(t, s.values(), 5)
s.removeByValue("1")
assert.Len(t, s.values(), 1)
}
func Test_cpeFieldCandidateSet_removeByCondition(t *testing.T) {
s := newFieldCandidateSet()
// should be removed
s.add(fieldCandidate{
value: "1",
disallowSubSelections: true,
})
s.add(fieldCandidate{
value: "hello-world",
})
// should not be removed
s.add(fieldCandidate{
value: "2",
})
assert.Len(t, s.values(), 3)
s.removeWhere(func(candidate fieldCandidate) bool {
return candidate.disallowSubSelections == true
})
assert.Len(t, s.values(), 2)
s.removeWhere(func(candidate fieldCandidate) bool {
return strings.Contains(candidate.value, "-")
})
assert.Len(t, s.values(), 1)
}
+73
View File
@@ -0,0 +1,73 @@
package cpe
import (
"strings"
"github.com/anchore/syft/syft/pkg"
"github.com/facebookincubator/nvdtools/wfn"
)
const jenkinsName = "jenkins"
// filterFn instances should return true if the given CPE should be removed from a collection for the given package
type filterFn func(cpe pkg.CPE, p pkg.Package) bool
var cpeFilters = []filterFn{
disallowJiraClientServerMismatch,
disallowJenkinsServerCPEForPluginPackage,
disallowJenkinsCPEsNotAssociatedWithJenkins,
disallowNonParseableCPEs,
}
func filter(cpes []pkg.CPE, p pkg.Package, filters ...filterFn) (result []pkg.CPE) {
cpeLoop:
for _, cpe := range cpes {
for _, fn := range filters {
if fn(cpe, p) {
continue cpeLoop
}
}
// all filter functions passed on filtering this CPE
result = append(result, cpe)
}
return result
}
func disallowNonParseableCPEs(cpe pkg.CPE, _ pkg.Package) bool {
v := cpe.BindToFmtString()
_, err := pkg.NewCPE(v)
cannotParse := err != nil
return cannotParse
}
// jenkins plugins should not match against jenkins
func disallowJenkinsServerCPEForPluginPackage(cpe pkg.CPE, p pkg.Package) bool {
if p.Type == pkg.JenkinsPluginPkg && cpe.Product == jenkinsName {
return true
}
return false
}
// filter to account that packages that are not for jenkins but have a CPE generated that will match against jenkins
func disallowJenkinsCPEsNotAssociatedWithJenkins(cpe pkg.CPE, p pkg.Package) bool {
// jenkins server should only match against a product with the name jenkins
if cpe.Product == jenkinsName && !strings.Contains(strings.ToLower(p.Name), jenkinsName) {
if cpe.Vendor == wfn.Any || cpe.Vendor == jenkinsName || cpe.Vendor == "cloudbees" {
return true
}
}
return false
}
// filter to account for packages which are jira client packages but have a CPE that will match against jira
func disallowJiraClientServerMismatch(cpe pkg.CPE, p pkg.Package) bool {
// jira / atlassian should not apply to clients
if cpe.Product == "jira" && strings.Contains(strings.ToLower(p.Name), "client") {
if cpe.Vendor == wfn.Any || cpe.Vendor == "jira" || cpe.Vendor == "atlassian" {
return true
}
}
return false
}
@@ -0,0 +1,167 @@
package cpe
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_disallowJenkinsServerCPEForPluginPackage(t *testing.T) {
tests := []struct {
name string
cpe pkg.CPE
pkg pkg.Package
expected bool
}{
{
name: "go case (filter out)",
cpe: mustCPE("cpe:2.3:a:name:jenkins:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Type: pkg.JenkinsPluginPkg,
},
expected: true,
},
{
name: "ignore jenkins plugins with unique name",
cpe: mustCPE("cpe:2.3:a:name:ci-jenkins:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Type: pkg.JenkinsPluginPkg,
},
expected: false,
},
{
name: "ignore java packages",
cpe: mustCPE("cpe:2.3:a:name:jenkins:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Type: pkg.JavaPkg,
},
expected: false,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, disallowJenkinsServerCPEForPluginPackage(test.cpe, test.pkg))
})
}
}
func Test_disallowJenkinsCPEsNotAssociatedWithJenkins(t *testing.T) {
tests := []struct {
name string
cpe pkg.CPE
pkg pkg.Package
expected bool
}{
{
name: "filter out mismatched name (cloudbees vendor)",
cpe: mustCPE("cpe:2.3:a:cloudbees:jenkins:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Name: "not-j*nkins",
Type: pkg.JavaPkg,
},
expected: true,
},
{
name: "filter out mismatched name (jenkins vendor)",
cpe: mustCPE("cpe:2.3:a:jenkins:jenkins:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Name: "not-j*nkins",
Type: pkg.JavaPkg,
},
expected: true,
},
{
name: "filter out mismatched name (any vendor)",
cpe: mustCPE("cpe:2.3:a:*:jenkins:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Name: "not-j*nkins",
Type: pkg.JavaPkg,
},
expected: true,
},
{
name: "ignore packages with the name jenkins",
cpe: mustCPE("cpe:2.3:a:*:jenkins:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Name: "jenkins-thing",
Type: pkg.JavaPkg,
},
expected: false,
},
{
name: "ignore product names that are not exactly 'jenkins'",
cpe: mustCPE("cpe:2.3:a:*:jenkins-something-else:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Name: "not-j*nkins",
Type: pkg.JavaPkg,
},
expected: false,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, disallowJenkinsCPEsNotAssociatedWithJenkins(test.cpe, test.pkg))
})
}
}
func Test_disallowJiraClientServerMismatch(t *testing.T) {
tests := []struct {
name string
cpe pkg.CPE
pkg pkg.Package
expected bool
}{
{
name: "filter out mismatched name (atlassian vendor)",
cpe: mustCPE("cpe:2.3:a:atlassian:jira:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Name: "something-client",
Type: pkg.JavaPkg,
},
expected: true,
},
{
name: "filter out mismatched name (jira vendor)",
cpe: mustCPE("cpe:2.3:a:jira:jira:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Name: "something-client",
Type: pkg.JavaPkg,
},
expected: true,
},
{
name: "filter out mismatched name (any vendor)",
cpe: mustCPE("cpe:2.3:a:*:jira:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Name: "something-client",
Type: pkg.JavaPkg,
},
expected: true,
},
{
name: "ignore package names that do not have 'client'",
cpe: mustCPE("cpe:2.3:a:*:jira:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Name: "jira-thing",
Type: pkg.JavaPkg,
},
expected: false,
},
{
name: "ignore product names that are not exactly 'jira'",
cpe: mustCPE("cpe:2.3:a:*:jira-something-else:3.2:*:*:*:*:*:*:*"),
pkg: pkg.Package{
Name: "not-j*ra",
Type: pkg.JavaPkg,
},
expected: false,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, disallowJiraClientServerMismatch(test.cpe, test.pkg))
})
}
}
+237
View File
@@ -0,0 +1,237 @@
package cpe
import (
"bufio"
"bytes"
"fmt"
"sort"
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/pkg"
"github.com/facebookincubator/nvdtools/wfn"
)
func newCPE(product, vendor, version, targetSW string) wfn.Attributes {
cpe := *(wfn.NewAttributesWithAny())
cpe.Part = "a"
cpe.Product = product
cpe.Vendor = vendor
cpe.Version = version
cpe.TargetSW = targetSW
return cpe
}
// Generate Create a list of CPEs for a given package, trying to guess the vendor, product tuple. We should be trying to
// generate the minimal set of representative CPEs, which implies that optional fields should not be included
// (such as target SW).
func Generate(p pkg.Package) []pkg.CPE {
vendors := candidateVendors(p)
products := candidateProducts(p)
if len(products) == 0 {
return nil
}
keys := internal.NewStringSet()
cpes := make([]pkg.CPE, 0)
for _, product := range products {
for _, vendor := range vendors {
// prevent duplicate entries...
key := fmt.Sprintf("%s|%s|%s", product, vendor, p.Version)
if keys.Contains(key) {
continue
}
keys.Add(key)
// add a new entry...
cpes = append(cpes, newCPE(product, vendor, p.Version, wfn.Any))
}
}
// filter out any known combinations that don't accurately represent this package
cpes = filter(cpes, p, cpeFilters...)
sort.Sort(BySpecificity(cpes))
return cpes
}
func candidateVendors(p pkg.Package) []string {
// in ecosystems where the packaging metadata does not have a clear field to indicate a vendor (or a field that
// could be interpreted indirectly as such) the project name tends to be a common stand in. Examples of this
// are the elasticsearch gem, xstream jar, and rack gem... all of these cases you can find vulnerabilities
// with CPEs where the vendor is the product name and doesn't appear to be derived from any available package
// metadata.
vendors := newFieldCandidateSet(candidateProducts(p)...)
switch p.Language {
case pkg.Ruby:
vendors.addValue("ruby-lang")
case pkg.Go:
// replace all candidates with only the golang-specific helper
vendors.clear()
vendor := candidateVendorForGo(p.Name)
if vendor != "" {
vendors.addValue(vendor)
}
}
// some ecosystems do not have enough metadata to determine the vendor accurately, in which case we selectively
// allow * as a candidate. Note: do NOT allow Java packages to have * vendors.
switch p.Language {
case pkg.Ruby, pkg.JavaScript:
vendors.addValue("*")
}
switch p.MetadataType {
case pkg.RpmdbMetadataType:
vendors.union(candidateVendorsForRPM(p))
case pkg.GemMetadataType:
vendors.union(candidateVendorsForRuby(p))
case pkg.PythonPackageMetadataType:
vendors.union(candidateVendorsForPython(p))
case pkg.JavaMetadataType:
vendors.union(candidateVendorsForJava(p))
}
// try swapping hyphens for underscores, vice versa, and removing separators altogether
addDelimiterVariations(vendors)
// generate sub-selections of each candidate based on separators (e.g. jenkins-ci -> [jenkins, jenkins-ci])
addAllSubSelections(vendors)
// add more candidates based on the package info for each vendor candidate
for _, vendor := range vendors.uniqueValues() {
vendors.addValue(findAdditionalVendors(defaultCandidateAdditions, p.Type, p.Name, vendor)...)
}
return vendors.uniqueValues()
}
func candidateProducts(p pkg.Package) []string {
products := newFieldCandidateSet(p.Name)
switch {
case p.Language == pkg.Python:
if !strings.HasPrefix(p.Name, "python") {
products.addValue("python-" + p.Name)
}
case p.Language == pkg.Java || p.MetadataType == pkg.JavaMetadataType:
products.addValue(candidateProductsForJava(p)...)
case p.Language == pkg.Go:
// replace all candidates with only the golang-specific helper
products.clear()
prod := candidateProductForGo(p.Name)
if prod != "" {
products.addValue(prod)
}
}
// it is never OK to have candidates with these values ["" and "*"] (since CPEs will match any other value)
products.removeByValue("")
products.removeByValue("*")
// try swapping hyphens for underscores, vice versa, and removing separators altogether
addDelimiterVariations(products)
// add known candidate additions
products.addValue(findAdditionalProducts(defaultCandidateAdditions, p.Type, p.Name)...)
return products.uniqueValues()
}
func addAllSubSelections(fields fieldCandidateSet) {
candidatesForVariations := fields.copy()
candidatesForVariations.removeWhere(subSelectionsDisallowed)
for _, candidate := range candidatesForVariations.values() {
fields.addValue(generateSubSelections(candidate)...)
}
}
// generateSubSelections attempts to split a field by hyphens and underscores and return a list of sensible sub-selections
// that can be used as product or vendor candidates. E.g. jenkins-ci-tools -> [jenkins-ci-tools, jenkins-ci, jenkins].
func generateSubSelections(field string) (results []string) {
scanner := bufio.NewScanner(strings.NewReader(field))
scanner.Split(scanByHyphenOrUnderscore)
var lastToken uint8
for scanner.Scan() {
rawCandidate := scanner.Text()
if len(rawCandidate) == 0 {
break
}
// trim any number of hyphen or underscore that is prefixed/suffixed on the given candidate. Since
// scanByHyphenOrUnderscore preserves delimiters (hyphens and underscores) they are guaranteed to be at least
// prefixed.
candidate := strings.TrimFunc(rawCandidate, trimHyphenOrUnderscore)
// capture the result (if there is content)
if len(candidate) > 0 {
if len(results) > 0 {
results = append(results, results[len(results)-1]+string(lastToken)+candidate)
} else {
results = append(results, candidate)
}
}
// keep track of the trailing separator for the next loop
lastToken = rawCandidate[len(rawCandidate)-1]
}
return results
}
// trimHyphenOrUnderscore is a character filter function for use with strings.TrimFunc in order to remove any hyphen or underscores.
func trimHyphenOrUnderscore(r rune) bool {
switch r {
case '-', '_':
return true
}
return false
}
// scanByHyphenOrUnderscore splits on hyphen or underscore and includes the separator in the split
func scanByHyphenOrUnderscore(data []byte, atEOF bool) (advance int, token []byte, err error) {
if atEOF && len(data) == 0 {
return 0, nil, nil
}
if i := bytes.IndexAny(data, "-_"); i >= 0 {
return i + 1, data[0 : i+1], nil
}
if atEOF {
return len(data), data, nil
}
return 0, nil, nil
}
func addDelimiterVariations(fields fieldCandidateSet) {
candidatesForVariations := fields.copy()
candidatesForVariations.removeWhere(delimiterVariationsDisallowed)
for _, candidate := range candidatesForVariations.list() {
field := candidate.value
hasHyphen := strings.Contains(field, "-")
hasUnderscore := strings.Contains(field, "_")
if hasHyphen {
// provide variations of hyphen candidates with an underscore
newValue := strings.ReplaceAll(field, "-", "_")
underscoreCandidate := candidate
underscoreCandidate.value = newValue
fields.add(underscoreCandidate)
}
if hasUnderscore {
// provide variations of underscore candidates with a hyphen
newValue := strings.ReplaceAll(field, "_", "-")
hyphenCandidate := candidate
hyphenCandidate.value = newValue
fields.add(hyphenCandidate)
}
}
}
@@ -0,0 +1,755 @@
package cpe
import (
"fmt"
"sort"
"strings"
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/scylladb/go-set"
"github.com/scylladb/go-set/strset"
"github.com/stretchr/testify/assert"
)
func TestGeneratePackageCPEs(t *testing.T) {
tests := []struct {
name string
p pkg.Package
expected []string
}{
{
name: "hyphen replacement",
p: pkg.Package{
Name: "name-part",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Python,
Type: pkg.DebPkg,
},
expected: []string{
"cpe:2.3:a:name-part:name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name-part:name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name-part:python-name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name-part:python_name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name:name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name:name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name:python-name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name:python_name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name_part:name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name_part:name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name_part:python-name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name_part:python_name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name-part:name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name-part:name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name-part:python-name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name-part:python_name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name:name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name:name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name:python-name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name:python_name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python:name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python:name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python:python-name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python:python_name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name:name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name:name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name:python-name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name:python_name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name_part:name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name_part:name_part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name_part:python-name-part:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name_part:python_name_part:3.2:*:*:*:*:*:*:*",
},
},
{
name: "python language",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Python,
Type: pkg.DebPkg,
MetadataType: pkg.PythonPackageMetadataType,
Metadata: pkg.PythonPackageMetadata{
Author: "alex goodman",
AuthorEmail: "william.goodman@anchore.com",
},
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name:python-name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name:python_name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name:python-name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python-name:python_name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python:python-name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python:python_name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name:python-name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:python_name:python_name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:alex_goodman:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:alex_goodman:python-name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:alex_goodman:python_name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:william-goodman:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:william-goodman:python-name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:william-goodman:python_name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:william_goodman:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:william_goodman:python-name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:william_goodman:python_name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "javascript language",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.JavaScript,
Type: pkg.DebPkg,
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:*:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "ruby language",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Ruby,
Type: pkg.DebPkg,
MetadataType: pkg.GemMetadataType,
Metadata: pkg.GemMetadata{
Authors: []string{
"someones name",
"someones.elses.name@gmail.com",
},
},
},
expected: []string{
"cpe:2.3:a:*:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby-lang:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby_lang:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:someones-elses-name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:someones-name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:someones_elses_name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:someones_name:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "java language",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JavaPkg,
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "java language with groupID",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JavaPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "org.sonatype.nexus",
},
},
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name:nexus:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:nexus:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:nexus:nexus:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:sonatype:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:sonatype:nexus:3.2:*:*:*:*:*:*:*",
},
},
{
name: "java with URL in metadata", // regression: https://github.com/anchore/grype/issues/417
p: pkg.Package{
Name: "wstx-asl",
Version: "3.2.7",
Type: pkg.JavaPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
Manifest: &pkg.JavaManifest{
Main: map[string]string{
"Ant-Version": "Apache Ant 1.6.5",
"Built-By": "tatu",
"Created-By": "1.4.2_03-b02 (Sun Microsystems Inc.)",
"Implementation-Title": "WoodSToX XML-processor",
"Implementation-Vendor": "woodstox.codehaus.org",
"Implementation-Version": "3.2.7",
"Manifest-Version": "1.0",
"Specification-Title": "StAX 1.0 API",
"Specification-Vendor": "http://jcp.org/en/jsr/detail?id=173",
"Specification-Version": "1.0",
},
},
},
},
expected: []string{
"cpe:2.3:a:woodstox_codehaus_org:wstx-asl:3.2.7:*:*:*:*:*:*:*",
"cpe:2.3:a:woodstox_codehaus_org:wstx_asl:3.2.7:*:*:*:*:*:*:*",
"cpe:2.3:a:woodstox-codehaus-org:wstx_asl:3.2.7:*:*:*:*:*:*:*",
"cpe:2.3:a:woodstox-codehaus-org:wstx-asl:3.2.7:*:*:*:*:*:*:*",
"cpe:2.3:a:wstx_asl:wstx-asl:3.2.7:*:*:*:*:*:*:*",
"cpe:2.3:a:wstx-asl:wstx-asl:3.2.7:*:*:*:*:*:*:*",
"cpe:2.3:a:wstx-asl:wstx_asl:3.2.7:*:*:*:*:*:*:*",
"cpe:2.3:a:wstx_asl:wstx_asl:3.2.7:*:*:*:*:*:*:*",
"cpe:2.3:a:wstx:wstx_asl:3.2.7:*:*:*:*:*:*:*",
"cpe:2.3:a:wstx:wstx-asl:3.2.7:*:*:*:*:*:*:*",
},
},
{
name: "jenkins package identified via pkg type",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "java language - multi tier manifest fields",
p: pkg.Package{
Name: "cxf-rt-bindings-xml",
Version: "3.3.10",
FoundBy: "java-cataloger",
Language: pkg.Java,
Type: pkg.JavaPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
VirtualPath: "/opt/jboss/keycloak/modules/system/layers/base/org/apache/cxf/impl/main/cxf-rt-bindings-xml-3.3.10.jar",
Manifest: &pkg.JavaManifest{
Main: map[string]string{
"Automatic-Module-Name": "org.apache.cxf.binding.xml",
"Bnd-LastModified": "1615836524860",
"Build-Jdk": "1.8.0_261",
"Built-By": "dkulp",
"Bundle-ActivationPolicy": "lazy",
"Bundle-Description": "Apache CXF Runtime XML Binding",
"Bundle-DocURL": "http://cxf.apache.org",
"Bundle-License": "https://www.apache.org/licenses/LICENSE-2.0.txt",
"Bundle-ManifestVersion": "2",
"Bundle-Name": "Apache CXF Runtime XML Binding",
"Bundle-SymbolicName": "org.apache.cxf.cxf-rt-bindings-xml",
"Bundle-Vendor": "The Apache Software Foundation",
"Bundle-Version": "3.3.10",
"Created-By": "Apache Maven Bundle Plugin",
"Export-Package": "org.apache.cxf.binding.xml;version=\"3.3.10\",org.apache.cxf.binding.xml.wsdl11;version=\"3.3.10\",org.apache.cxf.binding.xml.interceptor;version=\"3.3.10\",org.apache.cxf.bindings.xformat;version=\"3.3.10\"",
"Implementation-Vendor": "The Apache Software Foundation",
"Implementation-Vendor-Id": "org.apache",
"Implementation-Version": "3.3.10",
"Import-Package": "javax.xml.bind;version=\"[0,3)\",javax.xml.bind.annotation;version=\"[0,3)\",javax.wsdl;resolution:=optional,javax.wsdl.extensions;resolution:=optional,javax.wsdl.extensions.http;resolution:=optional,javax.xml.namespace,javax.xml.stream,org.apache.cxf;version=\"[3.3,4)\",org.apache.cxf.binding;version=\"[3.3,4)\",org.apache.cxf.binding.xml,org.apache.cxf.binding.xml.interceptor,org.apache.cxf.bindings.xformat,org.apache.cxf.common.i18n;version=\"[3.3,4)\",org.apache.cxf.common.injection;version=\"[3.3,4)\",org.apache.cxf.common.logging;version=\"[3.3,4)\",org.apache.cxf.common.util;version=\"[3.3,4)\",org.apache.cxf.endpoint;version=\"[3.3,4)\",org.apache.cxf.helpers;version=\"[3.3,4)\",org.apache.cxf.interceptor;version=\"[3.3,4)\",org.apache.cxf.message;version=\"[3.3,4)\",org.apache.cxf.service.model;version=\"[3.3,4)\",org.apache.cxf.staxutils;version=\"[3.3,4)\",org.apache.cxf.tools.common;version=\"[3.3,4)\";resolution:=optional,org.apache.cxf.tools.validator;version=\"[3.3,4)\";resolution:=optional,org.apache.cxf.transport;version=\"[3.3,4)\",org.apache.cxf.wsdl;version=\"[3.3,4)\";resolution:=optional,org.apache.cxf.wsdl.http;version=\"[3.3,4)\",org.apache.cxf.wsdl.interceptors;version=\"[3.3,4)\";resolution:=optional,org.w3c.dom",
"Manifest-Version": "1.0",
"Require-Capability": "osgi.ee;filter:=\"(&(osgi.ee=JavaSE)(version=1.8))\"",
"Specification-Vendor": "The Apache Software Foundation",
"Specification-Version": "3.3.10",
"Tool": "Bnd-4.2.0.201903051501",
},
},
PomProperties: &pkg.PomProperties{
Path: "META-INF/maven/org.apache.cxf/cxf-rt-bindings-xml/pom.properties",
GroupID: "org.apache.cxf",
ArtifactID: "cxf-rt-bindings-xml",
Version: "3.3.10",
},
},
},
expected: []string{
"cpe:2.3:a:apache-software-foundation:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache-software-foundation:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache-software-foundation:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache_software_foundation:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache_software_foundation:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache_software_foundation:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings-xml:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings-xml:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings-xml:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings_xml:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings_xml:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings_xml:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
},
},
{
name: "rpm vendor selection",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Type: pkg.RpmPkg,
MetadataType: pkg.RpmdbMetadataType,
Metadata: pkg.RpmdbMetadata{
Vendor: "some-vendor",
},
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:some-vendor:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:some_vendor:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "cloudbees jenkins package identified via groupId",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "com.cloudbees.jenkins.plugins",
},
},
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "jenkins.io package identified via groupId prefix",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "io.jenkins.plugins.name.something",
},
},
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name:something:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:something:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:something:something:3.2:*:*:*:*:*:*:*",
},
},
{
name: "jenkins.io package identified via groupId",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "io.jenkins.plugins",
},
},
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "jenkins-ci.io package identified via groupId",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "io.jenkins-ci.plugins",
},
},
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "jenkins-ci.org package identified via groupId",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JenkinsPluginPkg,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "org.jenkins-ci.plugins",
},
},
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
},
},
{
name: "jira-atlassian filtering",
p: pkg.Package{
Name: "jira_client_core",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JavaPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "org.atlassian.jira",
ArtifactID: "jira_client_core",
},
},
},
expected: []string{
"cpe:2.3:a:atlassian:jira-client-core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:atlassian:jira_client_core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira-client-core:jira-client-core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira-client-core:jira:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira-client-core:jira_client_core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira-client:jira-client-core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira-client:jira:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira-client:jira_client_core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira:jira-client-core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira:jira_client_core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira_client:jira-client-core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira_client:jira:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira_client:jira_client_core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira_client_core:jira-client-core:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira_client_core:jira:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jira_client_core:jira_client_core:3.2:*:*:*:*:*:*:*",
},
},
{
name: "jenkins filtering",
p: pkg.Package{
Name: "cloudbees-installation-manager",
Version: "2.89.0.33",
FoundBy: "some-analyzer",
Language: pkg.Java,
Type: pkg.JavaPkg,
MetadataType: pkg.JavaMetadataType,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "com.cloudbees.jenkins.modules",
ArtifactID: "cloudbees-installation-manager",
},
},
},
expected: []string{
"cpe:2.3:a:cloudbees-installation-manager:cloudbees-installation-manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:cloudbees-installation-manager:cloudbees_installation_manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:cloudbees-installation:cloudbees-installation-manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:cloudbees-installation:cloudbees_installation_manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:cloudbees:cloudbees-installation-manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:cloudbees:cloudbees_installation_manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:cloudbees_installation:cloudbees-installation-manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:cloudbees_installation:cloudbees_installation_manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:cloudbees_installation_manager:cloudbees-installation-manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:cloudbees_installation_manager:cloudbees_installation_manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins:cloudbees-installation-manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:jenkins:cloudbees_installation_manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:modules:cloudbees-installation-manager:2.89.0.33:*:*:*:*:*:*:*",
"cpe:2.3:a:modules:cloudbees_installation_manager:2.89.0.33:*:*:*:*:*:*:*",
},
},
{
name: "go product and vendor candidates are wired up",
p: pkg.Package{
Name: "github.com/someone/something",
Version: "3.2",
FoundBy: "go-cataloger",
Language: pkg.Go,
Type: pkg.GoModulePkg,
},
expected: []string{
"cpe:2.3:a:someone:something:3.2:*:*:*:*:*:*:*",
},
},
{
name: "generate no CPEs for indeterminate golang package name",
p: pkg.Package{
Name: "github.com/what",
Version: "3.2",
FoundBy: "go-cataloger",
Language: pkg.Go,
Type: pkg.GoModulePkg,
},
expected: []string{},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
actual := Generate(test.p)
expectedCpeSet := set.NewStringSet(test.expected...)
actualCpeSet := set.NewStringSet()
for _, a := range actual {
actualCpeSet.Add(a.BindToFmtString())
}
extra := strset.Difference(actualCpeSet, expectedCpeSet).List()
sort.Strings(extra)
if len(extra) > 0 {
t.Errorf("found extra CPEs:")
}
for _, d := range extra {
fmt.Printf(" %q,\n", d)
}
missing := strset.Difference(expectedCpeSet, actualCpeSet).List()
sort.Strings(missing)
if len(missing) > 0 {
t.Errorf("missing CPEs:")
}
for _, d := range missing {
fmt.Printf(" %q,\n", d)
}
})
}
}
func TestCandidateProducts(t *testing.T) {
tests := []struct {
name string
p pkg.Package
expected []string
}{
{
name: "apache-cassandra",
p: pkg.Package{
Name: "apache-cassandra",
Type: pkg.JavaPkg,
},
expected: []string{"cassandra" /* <-- known good names | default guess --> */, "apache-cassandra", "apache_cassandra"},
},
{
name: "springframework",
p: pkg.Package{
Name: "springframework",
Type: pkg.JavaPkg,
},
expected: []string{"spring_framework", "springsource_spring_framework" /* <-- known good names | default guess --> */, "springframework"},
},
{
name: "java",
p: pkg.Package{
Name: "some-java-package-with-group-id",
Type: pkg.JavaPkg,
Language: pkg.Java,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "com.apple.itunes",
},
},
},
expected: []string{"itunes", "some-java-package-with-group-id", "some_java_package_with_group_id"},
},
{
name: "java-with-asterisk",
p: pkg.Package{
Name: "some-java-package-with-group-id",
Type: pkg.JavaPkg,
Language: pkg.Java,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "com.apple.itunes.*",
},
},
},
expected: []string{"itunes", "some-java-package-with-group-id", "some_java_package_with_group_id"},
},
{
name: "jenkins-plugin",
p: pkg.Package{
Name: "some-jenkins-plugin",
Type: pkg.JenkinsPluginPkg,
Language: pkg.Java,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "com.cloudbees.jenkins.plugins",
},
},
},
expected: []string{"some-jenkins-plugin", "some_jenkins_plugin", "jenkins"},
},
{
name: "javascript",
p: pkg.Package{
Name: "handlebars.js",
Type: pkg.NpmPkg,
},
expected: []string{"handlebars" /* <-- known good names | default guess --> */, "handlebars.js"},
},
{
name: "gem",
p: pkg.Package{
Name: "RedCloth",
Type: pkg.GemPkg,
},
expected: []string{"redcloth_library" /* <-- known good names | default guess --> */, "RedCloth"},
},
{
name: "python",
p: pkg.Package{
Name: "python-rrdtool",
Type: pkg.PythonPkg,
},
expected: []string{"rrdtool" /* <-- known good names | default guess --> */, "python-rrdtool", "python_rrdtool"},
},
}
for _, test := range tests {
t.Run(fmt.Sprintf("%+v %+v", test.p, test.expected), func(t *testing.T) {
assert.ElementsMatch(t, test.expected, candidateProducts(test.p))
})
}
}
func TestCandidateVendor(t *testing.T) {
tests := []struct {
name string
p pkg.Package
expected []string
}{
{
name: "elasticsearch",
p: pkg.Package{
Name: "elasticsearch",
Type: pkg.JavaPkg,
},
expected: []string{"elastic" /* <-- known good names | default guess --> */, "elasticsearch"},
},
{
name: "log4j",
p: pkg.Package{
Name: "log4j",
Type: pkg.JavaPkg,
},
expected: []string{"apache" /* <-- known good names | default guess --> */, "log4j"},
},
}
for _, test := range tests {
t.Run(fmt.Sprintf("%+v %+v", test.p, test.expected), func(t *testing.T) {
assert.ElementsMatch(t, test.expected, candidateVendors(test.p))
})
}
}
func Test_generateSubSelections(t *testing.T) {
tests := []struct {
field string
expected []string
}{
{
field: "jenkins",
expected: []string{"jenkins"},
},
{
field: "jenkins-ci",
expected: []string{"jenkins", "jenkins-ci"},
},
{
field: "jenkins--ci",
expected: []string{"jenkins", "jenkins-ci"},
},
{
field: "jenkins_ci_tools",
expected: []string{"jenkins", "jenkins_ci", "jenkins_ci_tools"},
},
{
field: "-jenkins",
expected: []string{"jenkins"},
},
{
field: "jenkins_",
expected: []string{"jenkins"},
},
{
field: "",
expected: nil,
},
{
field: "-",
expected: nil,
},
{
field: "_",
expected: nil,
},
}
for _, test := range tests {
t.Run(test.field, func(t *testing.T) {
assert.ElementsMatch(t, test.expected, generateSubSelections(test.field))
})
}
}
func Test_addSeparatorVariations(t *testing.T) {
tests := []struct {
input []string
expected []string
}{
{
input: []string{"jenkins-ci"},
expected: []string{"jenkins-ci", "jenkins_ci"}, //, "jenkinsci"},
},
{
input: []string{"jenkins_ci"},
expected: []string{"jenkins_ci", "jenkins-ci"}, //, "jenkinsci"},
},
{
input: []string{"jenkins"},
expected: []string{"jenkins"},
},
{
input: []string{"jenkins-ci", "circle-ci"},
expected: []string{"jenkins-ci", "jenkins_ci", "circle-ci", "circle_ci"}, //, "jenkinsci", "circleci"},
},
}
for _, test := range tests {
t.Run(strings.Join(test.input, ","), func(t *testing.T) {
val := newFieldCandidateSet(test.input...)
addDelimiterVariations(val)
assert.ElementsMatch(t, test.expected, val.values())
})
}
}
+59
View File
@@ -0,0 +1,59 @@
package cpe
import (
"net/url"
"strings"
)
// candidateProductForGo attempts to find a single product name in a best-effort attempt. This implementation prefers
// to return no vendor over returning potentially nonsensical results.
func candidateProductForGo(name string) string {
// note: url.Parse requires a scheme for correct processing, which a golang module will not have, so one is provided.
u, err := url.Parse("http://" + name)
if err != nil {
return ""
}
cleanPath := strings.Trim(u.Path, "/")
pathElements := strings.Split(cleanPath, "/")
switch u.Host {
case "golang.org", "gopkg.in":
return cleanPath
case "google.golang.org":
return pathElements[0]
}
if len(pathElements) < 2 {
return ""
}
return pathElements[1]
}
// candidateVendorForGo attempts to find a single vendor name in a best-effort attempt. This implementation prefers
// to return no vendor over returning potentially nonsensical results.
func candidateVendorForGo(name string) string {
// note: url.Parse requires a scheme for correct processing, which a golang module will not have, so one is provided.
u, err := url.Parse("http://" + name)
if err != nil {
return ""
}
cleanPath := strings.Trim(u.Path, "/")
switch u.Host {
case "google.golang.org":
return "google"
case "golang.org":
return "golang"
case "gopkg.in":
return ""
}
pathElements := strings.Split(cleanPath, "/")
if len(pathElements) < 2 {
return ""
}
return pathElements[0]
}
+99
View File
@@ -0,0 +1,99 @@
package cpe
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestCandidateProductForGo(t *testing.T) {
tests := []struct {
pkg string
expected string
}{
{
pkg: "github.com/someone/something",
expected: "something",
},
{
pkg: "golang.org/x/xerrors",
expected: "x/xerrors",
},
{
pkg: "gopkg.in/yaml.v2",
expected: "yaml.v2",
},
{
pkg: "place",
expected: "",
},
{
pkg: "place.com/",
expected: "",
},
{
pkg: "place.com/someone-or-thing",
expected: "",
},
{
pkg: "google.golang.org/genproto/googleapis/rpc/status",
expected: "genproto",
},
{
pkg: "github.com/someone/something/long/package/name",
expected: "something",
},
}
for _, test := range tests {
t.Run(test.pkg, func(t *testing.T) {
assert.Equal(t, test.expected, candidateProductForGo(test.pkg))
})
}
}
func TestCandidateVendorForGo(t *testing.T) {
tests := []struct {
pkg string
expected string
}{
{
pkg: "github.com/someone/something",
expected: "someone",
},
{
pkg: "golang.org/x/xerrors",
expected: "golang",
},
{
pkg: "gopkg.in/yaml.v2",
expected: "",
},
{
pkg: "place",
expected: "",
},
{
pkg: "place.com/",
expected: "",
},
{
pkg: "place.com/someone-or-thing",
expected: "",
},
{
pkg: "google.golang.org/genproto/googleapis/rpc/status",
expected: "google",
},
{
pkg: "github.com/someone/something/long/package/name",
expected: "someone",
},
}
for _, test := range tests {
t.Run(test.pkg, func(t *testing.T) {
assert.Equal(t, test.expected, candidateVendorForGo(test.pkg))
})
}
}
+294
View File
@@ -0,0 +1,294 @@
package cpe
import (
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/pkg"
"github.com/scylladb/go-set/strset"
)
var (
forbiddenProductGroupIDFields = strset.New("plugin", "plugins", "client")
forbiddenVendorGroupIDFields = strset.New("plugin", "plugins")
domains = []string{
"com",
"org",
"net",
"io",
}
primaryJavaManifestGroupIDFields = []string{
"Extension-Name",
"Specification-Vendor",
"Implementation-Vendor",
"Bundle-SymbolicName",
"Implementation-Vendor-Id",
"Implementation-Title",
"Bundle-Activator",
}
secondaryJavaManifestGroupIDFields = []string{
"Automatic-Module-Name",
"Main-Class",
"Package",
}
javaManifestNameFields = []string{
"Specification-Vendor",
"Implementation-Vendor",
}
)
func candidateProductsForJava(p pkg.Package) []string {
return productsFromArtifactAndGroupIDs(artifactIDFromJavaPackage(p), groupIDsFromJavaPackage(p))
}
func candidateVendorsForJava(p pkg.Package) fieldCandidateSet {
gidVendors := vendorsFromGroupIDs(groupIDsFromJavaPackage(p))
nameVendors := vendorsFromJavaManifestNames(p)
return newFieldCandidateSetFromSets(gidVendors, nameVendors)
}
func vendorsFromJavaManifestNames(p pkg.Package) fieldCandidateSet {
vendors := newFieldCandidateSet()
metadata, ok := p.Metadata.(pkg.JavaMetadata)
if !ok {
return vendors
}
if metadata.Manifest == nil {
return vendors
}
for _, name := range javaManifestNameFields {
if metadata.Manifest.Main != nil {
if value, exists := metadata.Manifest.Main[name]; exists {
if !startsWithTopLevelDomain(value) {
vendors.add(fieldCandidate{
value: normalizePersonName(value),
disallowSubSelections: true,
})
}
}
}
if metadata.Manifest.NamedSections != nil {
for _, section := range metadata.Manifest.NamedSections {
if section == nil {
continue
}
if value, exists := section[name]; exists {
if !startsWithTopLevelDomain(value) {
vendors.add(fieldCandidate{
value: normalizePersonName(value),
disallowSubSelections: true,
})
}
}
}
}
}
return vendors
}
func vendorsFromGroupIDs(groupIDs []string) fieldCandidateSet {
vendors := newFieldCandidateSet()
for _, groupID := range groupIDs {
for i, field := range strings.Split(groupID, ".") {
field = strings.TrimSpace(field)
if len(field) == 0 {
continue
}
if forbiddenVendorGroupIDFields.Has(strings.ToLower(field)) {
continue
}
if i == 0 {
continue
}
vendors.addValue(field)
}
}
return vendors
}
func productsFromArtifactAndGroupIDs(artifactID string, groupIDs []string) []string {
products := strset.New()
if artifactID != "" {
products.Add(artifactID)
}
for _, groupID := range groupIDs {
isPlugin := strings.Contains(artifactID, "plugin") || strings.Contains(groupID, "plugin")
for i, field := range strings.Split(groupID, ".") {
field = strings.TrimSpace(field)
if len(field) == 0 {
continue
}
// don't add this field as a name if the name is implying the package is a plugin or client
if forbiddenProductGroupIDFields.Has(strings.ToLower(field)) {
continue
}
if i <= 1 {
continue
}
// umbrella projects tend to have sub components that either start or end with the project name. We expect
// to identify fields that may represent the umbrella project, and not fields that indicate auxiliary
// information about the package.
couldBeProjectName := strings.HasPrefix(artifactID, field) || strings.HasSuffix(artifactID, field)
if artifactID == "" || (couldBeProjectName && !isPlugin) {
products.Add(field)
}
}
}
return products.List()
}
func artifactIDFromJavaPackage(p pkg.Package) string {
metadata, ok := p.Metadata.(pkg.JavaMetadata)
if !ok {
return ""
}
if metadata.PomProperties == nil {
return ""
}
artifactID := strings.TrimSpace(metadata.PomProperties.ArtifactID)
if startsWithTopLevelDomain(artifactID) && len(strings.Split(artifactID, ".")) > 1 {
// there is a strong indication that the artifact ID is really a group ID, don't use it
return ""
}
return artifactID
}
func groupIDsFromJavaPackage(p pkg.Package) (groupIDs []string) {
metadata, ok := p.Metadata.(pkg.JavaMetadata)
if !ok {
return nil
}
groupIDs = append(groupIDs, groupIDsFromPomProperties(metadata.PomProperties)...)
groupIDs = append(groupIDs, groupIDsFromPomProject(metadata.PomProject)...)
groupIDs = append(groupIDs, groupIDsFromJavaManifest(metadata.Manifest)...)
return groupIDs
}
func groupIDsFromPomProperties(properties *pkg.PomProperties) (groupIDs []string) {
if properties == nil {
return nil
}
if startsWithTopLevelDomain(properties.GroupID) {
groupIDs = append(groupIDs, cleanGroupID(properties.GroupID))
}
// sometimes the publisher puts the group ID in the artifact ID field unintentionally
if startsWithTopLevelDomain(properties.ArtifactID) && len(strings.Split(properties.ArtifactID, ".")) > 1 {
// there is a strong indication that the artifact ID is really a group ID
groupIDs = append(groupIDs, cleanGroupID(properties.ArtifactID))
}
return groupIDs
}
func groupIDsFromPomProject(project *pkg.PomProject) (groupIDs []string) {
if project == nil {
return nil
}
// extract the project info...
groupIDs = addGroupIDsFromGroupIDsAndArtifactID(project.GroupID, project.ArtifactID)
if project.Parent == nil {
return groupIDs
}
// extract the parent project info...
groupIDs = append(groupIDs, addGroupIDsFromGroupIDsAndArtifactID(project.Parent.GroupID, project.Parent.ArtifactID)...)
return groupIDs
}
func addGroupIDsFromGroupIDsAndArtifactID(groupID, artifactID string) (groupIDs []string) {
if startsWithTopLevelDomain(groupID) {
groupIDs = append(groupIDs, cleanGroupID(groupID))
}
// sometimes the publisher puts the group ID in the artifact ID field unintentionally
if startsWithTopLevelDomain(artifactID) && len(strings.Split(artifactID, ".")) > 1 {
// there is a strong indication that the artifact ID is really a group ID
groupIDs = append(groupIDs, cleanGroupID(artifactID))
}
return groupIDs
}
func groupIDsFromJavaManifest(manifest *pkg.JavaManifest) []string {
if manifest == nil {
return nil
}
// try the common manifest fields first for a set of candidates
groupIDs := getManifestFieldGroupIDs(manifest, primaryJavaManifestGroupIDFields)
if len(groupIDs) != 0 {
return groupIDs
}
// if we haven't found anything yet, let's try a last ditch effort:
// attempt to get group-id-like info from the MANIFEST.MF "Automatic-Module-Name" and "Extension-Name" field.
// for more info see pkg:maven/commons-io/commons-io@2.8.0 within cloudbees/cloudbees-core-mm:2.263.4.2
// at /usr/share/jenkins/jenkins.war:WEB-INF/plugins/analysis-model-api.hpi:WEB-INF/lib/commons-io-2.8.0.jar
// as well as the ant package from cloudbees/cloudbees-core-mm:2.277.2.4-ra.
return getManifestFieldGroupIDs(manifest, secondaryJavaManifestGroupIDFields)
}
func getManifestFieldGroupIDs(manifest *pkg.JavaManifest, fields []string) (groupIDs []string) {
if manifest == nil {
return nil
}
for _, name := range fields {
if value, exists := manifest.Main[name]; exists {
if startsWithTopLevelDomain(value) {
groupIDs = append(groupIDs, cleanGroupID(value))
}
}
for _, section := range manifest.NamedSections {
if value, exists := section[name]; exists {
if startsWithTopLevelDomain(value) {
groupIDs = append(groupIDs, cleanGroupID(value))
}
}
}
}
return groupIDs
}
func cleanGroupID(groupID string) string {
return strings.TrimSpace(removeOSCIDirectives(groupID))
}
func removeOSCIDirectives(groupID string) string {
// for example:
// org.bar;uses:=“org.foo” -> org.bar
// more about OSGI directives see https://spring.io/blog/2008/10/20/understanding-the-osgi-uses-directive/
return strings.Split(groupID, ";")[0]
}
func startsWithTopLevelDomain(value string) bool {
return internal.HasAnyOfPrefixes(value, domains...)
}
+428
View File
@@ -0,0 +1,428 @@
package cpe
import (
"strings"
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_productsFromArtifactAndGroupIDs(t *testing.T) {
tests := []struct {
groupIDs []string
artifactID string
expected []string
}{
{
groupIDs: []string{"org.sonatype.nexus"},
artifactID: "nexus-extender",
expected: []string{"nexus", "nexus-extender"},
},
{
groupIDs: []string{"org.sonatype.nexus"},
expected: []string{"nexus"},
},
{
groupIDs: []string{"org.jenkins-ci.plugins"},
artifactID: "ant",
expected: []string{"ant"},
},
{
groupIDs: []string{"org.jenkins-ci.plugins"},
artifactID: "antisamy-markup-formatter",
expected: []string{"antisamy-markup-formatter"},
},
{
groupIDs: []string{"io.jenkins.plugins"},
artifactID: "aws-global-configuration",
expected: []string{"aws-global-configuration"},
},
{
groupIDs: []string{"com.cloudbees.jenkins.plugins"},
artifactID: "cloudbees-servicenow-jenkins-plugin",
expected: []string{"cloudbees-servicenow-jenkins-plugin"},
},
{
groupIDs: []string{"com.atlassian.confluence.plugins"},
artifactID: "confluence-mobile-plugin",
expected: []string{"confluence-mobile-plugin"},
},
{
groupIDs: []string{"com.atlassian.confluence.plugins"},
artifactID: "confluence-view-file-macro",
expected: []string{"confluence-view-file-macro"},
},
{
groupIDs: []string{"com.google.guava"},
artifactID: "failureaccess",
expected: []string{"failureaccess"},
},
}
for _, test := range tests {
t.Run(strings.Join(test.groupIDs, ",")+":"+test.artifactID, func(t *testing.T) {
actual := productsFromArtifactAndGroupIDs(test.artifactID, test.groupIDs)
assert.ElementsMatch(t, test.expected, actual, "different products")
})
}
}
func Test_candidateProductsForJava(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expected []string
}{
{
name: "duplicate groupID in artifactID field",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "org.sonatype.nexus",
ArtifactID: "org.sonatype.nexus",
},
},
},
expected: []string{"nexus"},
},
{
name: "detect groupID-like value in artifactID field",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
ArtifactID: "org.sonatype.nexus",
},
},
},
expected: []string{"nexus"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
actual := candidateProductsForJava(test.pkg)
assert.ElementsMatch(t, test.expected, actual, "different products")
})
}
}
func Test_vendorsFromGroupIDs(t *testing.T) {
tests := []struct {
groupID string
expected []string
}{
{
groupID: "org.sonatype.nexus",
expected: []string{"sonatype", "nexus"},
},
{
groupID: "org.jenkins-ci.plugins",
expected: []string{"jenkins-ci"},
},
{
groupID: "io.jenkins.plugins",
expected: []string{"jenkins"},
},
{
groupID: "com.cloudbees.jenkins.plugins",
expected: []string{"cloudbees", "jenkins"},
},
{
groupID: "com.atlassian.confluence.plugins",
expected: []string{"atlassian", "confluence"},
},
{
groupID: "com.google.guava",
expected: []string{"google", "guava"},
},
}
for _, test := range tests {
t.Run(test.groupID, func(t *testing.T) {
assert.ElementsMatch(t, test.expected, vendorsFromGroupIDs([]string{test.groupID}).values(), "different vendors")
})
}
}
func Test_groupIDsFromJavaPackage(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expects []string
}{
{
name: "go case",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "io.jenkins-ci.plugin.thing;version='[2,3)'",
},
},
},
expects: []string{"io.jenkins-ci.plugin.thing"},
},
{
name: "from artifactID",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
ArtifactID: "io.jenkins-ci.plugin.thing; version='[2,3)' ; org.something.else",
},
},
},
expects: []string{"io.jenkins-ci.plugin.thing"},
},
{
name: "from main Extension-Name field",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
Manifest: &pkg.JavaManifest{
Main: map[string]string{
"Extension-Name": "io.jenkins-ci.plugin.thing",
},
},
},
},
expects: []string{"io.jenkins-ci.plugin.thing"},
},
{
name: "from named section Extension-Name field",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
Manifest: &pkg.JavaManifest{
NamedSections: map[string]map[string]string{
"section": {
"Extension-Name": "io.jenkins-ci.plugin.thing",
},
},
},
},
},
expects: []string{"io.jenkins-ci.plugin.thing"},
},
{
name: "from main field - tier 1",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
Manifest: &pkg.JavaManifest{
Main: map[string]string{
// positive cases
// tier 1
"Extension-Name": "io.jenkins-ci.plugin.1",
"Specification-Vendor": "io.jenkins-ci.plugin.2",
"Implementation-Vendor": "io.jenkins-ci.plugin.3",
"Bundle-SymbolicName": "io.jenkins-ci.plugin.4",
"Implementation-Vendor-Id": "io.jenkins-ci.plugin.5",
"Implementation-Title": "io.jenkins-ci.plugin.6",
"Bundle-Activator": "io.jenkins-ci.plugin.7",
// tier 2
"Automatic-Module-Name": "io.jenkins-ci.plugin.8",
"Main-Class": "io.jenkins-ci.plugin.9",
"Package": "io.jenkins-ci.plugin.10",
},
},
},
},
expects: []string{
"io.jenkins-ci.plugin.1",
"io.jenkins-ci.plugin.2",
"io.jenkins-ci.plugin.3",
"io.jenkins-ci.plugin.4",
"io.jenkins-ci.plugin.5",
"io.jenkins-ci.plugin.6",
"io.jenkins-ci.plugin.7",
},
},
{
name: "from main field - tier 2",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
Manifest: &pkg.JavaManifest{
Main: map[string]string{
// positive cases
"Automatic-Module-Name": "io.jenkins-ci.plugin.8",
"Main-Class": "io.jenkins-ci.plugin.9",
"Package": "io.jenkins-ci.plugin.10",
},
},
},
},
expects: []string{
"io.jenkins-ci.plugin.8",
"io.jenkins-ci.plugin.9",
"io.jenkins-ci.plugin.10",
},
},
{
name: "from main field - negative cases",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
Manifest: &pkg.JavaManifest{
Main: map[string]string{
// negative cases
"Extension-Name": "not.a-group.id",
"bogus": "io.jenkins-ci.plugin.please-dont-find-me",
},
},
},
},
expects: nil,
},
{
name: "from named section field - tier 1",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
Manifest: &pkg.JavaManifest{
NamedSections: map[string]map[string]string{
"section": {
// positive cases
// tier 1
"Extension-Name": "io.jenkins-ci.plugin.1",
"Specification-Vendor": "io.jenkins-ci.plugin.2",
"Implementation-Vendor": "io.jenkins-ci.plugin.3",
"Bundle-SymbolicName": "io.jenkins-ci.plugin.4",
"Implementation-Vendor-Id": "io.jenkins-ci.plugin.5",
"Implementation-Title": "io.jenkins-ci.plugin.6",
"Bundle-Activator": "io.jenkins-ci.plugin.7",
// tier 2
"Automatic-Module-Name": "io.jenkins-ci.plugin.8",
"Main-Class": "io.jenkins-ci.plugin.9",
"Package": "io.jenkins-ci.plugin.10",
},
},
},
},
},
expects: []string{
"io.jenkins-ci.plugin.1",
"io.jenkins-ci.plugin.2",
"io.jenkins-ci.plugin.3",
"io.jenkins-ci.plugin.4",
"io.jenkins-ci.plugin.5",
"io.jenkins-ci.plugin.6",
"io.jenkins-ci.plugin.7",
},
},
{
name: "from named section field - negative cases",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
Manifest: &pkg.JavaManifest{
NamedSections: map[string]map[string]string{
"section": {
// negative cases
"Extension-Name": "not.a-group.id",
"bogus": "io.jenkins-ci.plugin.please-dont-find-me",
},
},
},
},
},
expects: nil,
},
{
name: "no manifest or pom info",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{},
},
expects: nil,
},
{
name: "no java info",
pkg: pkg.Package{},
expects: nil,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.ElementsMatch(t, test.expects, groupIDsFromJavaPackage(test.pkg))
})
}
}
func Test_artifactIDFromJavaPackage(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expects string
}{
{
name: "go case",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
ArtifactID: "cloudbees-installation-manager",
},
},
},
expects: "cloudbees-installation-manager",
},
{
name: "ignore groupID-like things",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
ArtifactID: "io.jenkins-ci.plugin.thing",
},
},
},
expects: "",
},
{
name: "no java info",
pkg: pkg.Package{},
expects: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expects, artifactIDFromJavaPackage(test.pkg))
})
}
}
func Test_vendorsFromJavaManifestNames(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expects []string
}{
{
name: "from manifest named section fields",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
Manifest: &pkg.JavaManifest{
NamedSections: map[string]map[string]string{
"section": {
// positive cases
"Specification-Vendor": "Alex Goodman",
"Implementation-Vendor": "William Goodman",
},
},
},
},
},
expects: []string{"alex_goodman", "william_goodman"},
},
{
name: "from manifest named section fields - negative cases",
pkg: pkg.Package{
Metadata: pkg.JavaMetadata{
Manifest: &pkg.JavaManifest{
NamedSections: map[string]map[string]string{
"section": {
// negative cases
"Specification-Vendor": "io.jenkins-ci.plugin.thing",
"Implementation-Vendor-ID": "William Goodman",
},
},
},
},
},
expects: nil,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.ElementsMatch(t, test.expects, vendorsFromJavaManifestNames(test.pkg).values())
})
}
}
+29
View File
@@ -0,0 +1,29 @@
package cpe
import "github.com/anchore/syft/syft/pkg"
func candidateVendorsForPython(p pkg.Package) fieldCandidateSet {
metadata, ok := p.Metadata.(pkg.PythonPackageMetadata)
if !ok {
return nil
}
vendors := newFieldCandidateSet()
if metadata.Author != "" {
vendors.add(fieldCandidate{
value: normalizePersonName(metadata.Author),
disallowSubSelections: true,
disallowDelimiterVariations: true,
})
}
if metadata.AuthorEmail != "" {
vendors.add(fieldCandidate{
value: normalizePersonName(stripEmailSuffix(metadata.AuthorEmail)),
disallowSubSelections: true,
})
}
return vendors
}
+21
View File
@@ -0,0 +1,21 @@
package cpe
import "github.com/anchore/syft/syft/pkg"
func candidateVendorsForRPM(p pkg.Package) fieldCandidateSet {
metadata, ok := p.Metadata.(pkg.RpmdbMetadata)
if !ok {
return nil
}
vendors := newFieldCandidateSet()
if metadata.Vendor != "" {
vendors.add(fieldCandidate{
value: normalizeName(metadata.Vendor),
disallowSubSelections: true,
})
}
return vendors
}
+21
View File
@@ -0,0 +1,21 @@
package cpe
import "github.com/anchore/syft/syft/pkg"
func candidateVendorsForRuby(p pkg.Package) fieldCandidateSet {
metadata, ok := p.Metadata.(pkg.GemMetadata)
if !ok {
return nil
}
vendors := newFieldCandidateSet()
for _, author := range metadata.Authors {
// author could be a name or an email
vendors.add(fieldCandidate{
value: normalizePersonName(stripEmailSuffix(author)),
disallowSubSelections: true,
})
}
return vendors
}
@@ -0,0 +1,57 @@
package cpe
import (
"sort"
"github.com/facebookincubator/nvdtools/wfn"
)
var _ sort.Interface = (*BySpecificity)(nil)
type BySpecificity []wfn.Attributes
func (c BySpecificity) Len() int { return len(c) }
func (c BySpecificity) Swap(i, j int) { c[i], c[j] = c[j], c[i] }
func (c BySpecificity) Less(i, j int) bool {
iScore := weightedCountForSpecifiedFields(c[i])
jScore := weightedCountForSpecifiedFields(c[j])
// check weighted sort first
if iScore != jScore {
return iScore > jScore
}
// sort longer fields to top
if countFieldLength(c[i]) != countFieldLength(c[j]) {
return countFieldLength(c[i]) > countFieldLength(c[j])
}
// if score and length are equal then text sort
return c[i].BindToFmtString() < c[j].BindToFmtString()
}
func countFieldLength(cpe wfn.Attributes) int {
return len(cpe.Part + cpe.Vendor + cpe.Product + cpe.Version + cpe.TargetSW)
}
func weightedCountForSpecifiedFields(cpe wfn.Attributes) int {
checksForSpecifiedField := []func(cpe wfn.Attributes) (bool, int){
func(cpe wfn.Attributes) (bool, int) { return cpe.Part != "", 2 },
func(cpe wfn.Attributes) (bool, int) { return cpe.Vendor != "", 3 },
func(cpe wfn.Attributes) (bool, int) { return cpe.Product != "", 4 },
func(cpe wfn.Attributes) (bool, int) { return cpe.Version != "", 1 },
func(cpe wfn.Attributes) (bool, int) { return cpe.TargetSW != "", 1 },
}
weightedCount := 0
for _, fieldIsSpecified := range checksForSpecifiedField {
isSpecified, weight := fieldIsSpecified(cpe)
if isSpecified {
weightedCount += weight
}
}
return weightedCount
}
@@ -0,0 +1,111 @@
package cpe
import (
"sort"
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func mustCPE(c string) pkg.CPE {
return must(pkg.NewCPE(c))
}
func must(c pkg.CPE, e error) pkg.CPE {
if e != nil {
panic(e)
}
return c
}
func TestCPESpecificity(t *testing.T) {
tests := []struct {
name string
input []pkg.CPE
expected []pkg.CPE
}{
{
name: "sort strictly by wfn *",
input: []pkg.CPE{
mustCPE("cpe:2.3:a:*:package:1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:some:package:1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:*:package:1:*:*:*:*:some:*:*"),
mustCPE("cpe:2.3:a:some:package:1:*:*:*:*:some:*:*"),
mustCPE("cpe:2.3:a:some:package:*:*:*:*:*:*:*:*"),
},
expected: []pkg.CPE{
mustCPE("cpe:2.3:a:some:package:1:*:*:*:*:some:*:*"),
mustCPE("cpe:2.3:a:some:package:1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:some:package:*:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:*:package:1:*:*:*:*:some:*:*"),
mustCPE("cpe:2.3:a:*:package:1:*:*:*:*:*:*:*"),
},
},
{
name: "sort strictly by field length",
input: []pkg.CPE{
mustCPE("cpe:2.3:a:1:22:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:55555:1:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:1:333:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:666666:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:1:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:1:1:*:*:*:*:4444:*:*"),
},
expected: []pkg.CPE{
mustCPE("cpe:2.3:a:1:666666:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:55555:1:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:1:1:*:*:*:*:4444:*:*"),
mustCPE("cpe:2.3:a:1:1:333:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:22:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:1:1:*:*:*:*:1:*:*"),
},
},
{
name: "sort by mix of field length and specificity",
input: []pkg.CPE{
mustCPE("cpe:2.3:a:1:666666:*:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:*:1:1:*:*:*:*:4444:*:*"),
mustCPE("cpe:2.3:a:1:*:333:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:1:1:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:22:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:55555:1:1:*:*:*:*:1:*:*"),
},
expected: []pkg.CPE{
mustCPE("cpe:2.3:a:55555:1:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:22:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:1:1:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:1:666666:*:*:*:*:*:1:*:*"),
mustCPE("cpe:2.3:a:*:1:1:*:*:*:*:4444:*:*"),
mustCPE("cpe:2.3:a:1:*:333:*:*:*:*:*:*:*"),
},
},
{
name: "sort by mix of field length, specificity, dash",
input: []pkg.CPE{
mustCPE("cpe:2.3:a:alpine:alpine_keys:2.3-r1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:alpine_keys:alpine_keys:2.3-r1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:alpine-keys:alpine_keys:2.3-r1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:alpine:alpine-keys:2.3-r1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:alpine-keys:alpine-keys:2.3-r1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:alpine_keys:alpine-keys:2.3-r1:*:*:*:*:*:*:*"),
},
expected: []pkg.CPE{
mustCPE("cpe:2.3:a:alpine-keys:alpine-keys:2.3-r1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:alpine-keys:alpine_keys:2.3-r1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:alpine_keys:alpine-keys:2.3-r1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:alpine_keys:alpine_keys:2.3-r1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:alpine:alpine-keys:2.3-r1:*:*:*:*:*:*:*"),
mustCPE("cpe:2.3:a:alpine:alpine_keys:2.3-r1:*:*:*:*:*:*:*"),
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
sort.Sort(BySpecificity(test.input))
assert.Equal(t, test.expected, test.input)
})
}
}

Some files were not shown because too many files have changed in this diff Show More