Compare commits

...
53 Commits
Author SHA1 Message Date
Alex Goodman 821210006d make updates due to linter update
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 22:51:36 -04:00
Alex Goodman 053768c6c6 update golangci + add additional lint rules
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 22:50:57 -04:00
Alex Goodman 9189ed68df Add query by MIME type to source.FileResolver (#529)
* add query by MIME type to source.FileResolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* pull in stereoscope MIME type feature

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 20:50:53 -04:00
Alex Goodman ba1cd8d753 specify optional file.Digests instances to abbreviate json output (#531)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 15:35:28 -04:00
Alex Goodman dd2c178085 add --file option to readme (#533)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 15:34:32 -04:00
Alex Goodman 1b23a94015 Add option to output SBOM report to a file (#530)
* add output to file option

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* log errors on close of the report destination

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove file option from persistent args

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update file option comments and logging

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow for multiple UI fallback options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update UI select signatures + tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 14:47:24 -04:00
Christopher Angelo Phillips f47a6a88b1 (#495) Update documentNamespace uniqueness for spdx-json output (#528)
* add unique namespace identifier

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-05 13:10:49 -04:00
Dan Luhring 4b7217f052 Improve documentation (#525)
* Make installation methods more obvious

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Create linkable section headers

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add badge for joining Slack

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Document requirement for signed commits

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-10-04 09:29:27 -04:00
Christopher Angelo Phillips 5e4b668e5a 497 stable sorted CPE array (JSON and SPDX) (#522)
* add small sorting change to our specificity

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-01 15:31:17 -04:00
Dan Luhring 6480f06173 Update description for Slack link (#523)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-09-30 21:43:43 +00:00
Christopher Angelo Phillips 75aed5f3ec (#460) Extend license mapping for common SPDX license names (#509)
Fixes #460 
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-30 16:54:36 -04:00
Jamie Magee 270fbb7665 Add mariner to distro types (#520)
Signed-off-by: Jamie Magee <jamie.magee@gmail.com>
2021-09-30 13:20:15 -04:00
Alex GoodmanandChristopher Angelo Phillips 0395c47445 Add vendor + product known good CPE field values (#517)
* add better infrastructure around CPE candidate additions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add test cases for CPE candidate additions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* small ergonomic updates

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* clean java OSGI directives from group ID

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* unquote CPE candidate field values (double quotes only)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add remaining CPE vendor & product additions (+ tests)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-27 18:31:59 -04:00
Keith Zantow e5caba043d Add SBOM to releases (#500)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-09-24 20:21:23 -04:00
Dan Luhring d1ac476986 Add announcement for KubeCon meetup (#515)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-09-24 11:42:06 -04:00
Dan Luhring dd23d49986 Prevent invalid CPE field values (#514)
* Fix CPE set comparison mismatch

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add failing test to assert CPE generation excludes URLs

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add removeByCondition method to fieldCandidateSet

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Prevent invalid CPE values for products and vendors

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Introduce removeWhere and rename filter to condition

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Refactor fieldCandidateSet and condition logic

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Move CPE parsing filter to end of CPE generation

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-09-24 09:23:58 -04:00
Alex Goodman 6d4d083acc Filter out CPE product candidates that are asterisks (#513)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-22 20:55:03 -04:00
Dan Luhring 316d4341c8 Use Anchore fork of packageurl lib without replace directive (#512)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-09-22 15:04:09 -04:00
Christopher Angelo Phillips 3e8afc5274 update log file permissions to 0644 (#511)
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-21 10:34:10 -04:00
Christopher Angelo Phillips 93d00dc340 Populate Files and Relationship fields for spdx-json output (#507)
* update spdx22 Document model to include relationships field

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update document and relationship to match current JSON spec
https://github.com/spdx/spdx-spec/blob/development/v2.2.1/schemas/spdx-schema.json
https://github.com/spdx/spdx-spec/pull/528
https://github.com/spdx/spdx-spec/pull/528#issuecomment-904180177

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update File struct based on SPDX schema

Required fields:
[ "SPDXID", "fileName", "copyrightText", "licenseConcluded" ]
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-17 09:06:12 -04:00
Alex Goodman 9fe1da8ee6 update go-rpmdb and restruct gomod references (#506)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-14 14:51:19 -04:00
Christopher Angelo Phillips 95df7dd035 Add command examples to release documentation (#503)
* update release document with commands run
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* small edits
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-14 09:08:37 -04:00
Alex Goodman ddfc8e20c0 Revert "disable docker releases (workaround) (#493)" (#501)
This reverts commit 06dcd3261d.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-13 13:06:23 -04:00
Christopher Angelo Phillips 3f305aa6cd (#342) - remove strong distro check (#496)
* add Type conversion to remove strong distro type limit
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update signatures to be correct variable from os-release
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-09 13:05:22 -04:00
houdini91 2f99a35f51 Power user command support for directory scans (#467)
* Power-user directory source support
Signed-off-by: Mikey Strauss <mikey@scribe-security.com>

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Remove newline

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Shared filetree (#1)

* Shared directory resolver filetree

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* PR - change error ErrObserve to ErrPath

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* PR - share directory resolver
* Use pointer to source struct

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Fix Lint

Signed-off-by: houdini91 <mdstrauss91@gmail.com>
2021-09-08 09:18:53 -04:00
Alex Goodman 06dcd3261d disable docker releases (workaround) (#493)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-03 21:29:34 +00:00
Alex Goodman abbba3fc19 Modify CPE vendor candidate generation approach (#484)
* consider additional vendor candidates for ruby, python, rpm, npm, and java

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add java pom.xml processing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow for downstream transform control in cpe generation processing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate CPE generation logic to dedicated package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split java manifest groupID extraction into two tiers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* extract groupID from pom parent project during CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update java groupID processing tests to cover multi-tier approach

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix constructor names for cpe.fieldCandidate

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename helper function to startsWithTopLevelDomain

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add nil changes for java manifest sections

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update comment to reflect parsing maven files

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split out java description parsing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split out pom parent processing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify vendorsFromGroupIDs and associated tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify test type for vendorsFromGroupIDs

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* copy candidate varidations to new instances

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename CPE generation string util functions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add an explanation around fieldCandidate

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify type for the cpe.fieldCandidateSet

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* make CPE filter function names more readable

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update groupIDsFromJavaManifest to use a guard clause

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* extract groupID extraction from artifactID fields into a separate function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump goreleaser version to combat failure

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-03 14:21:25 -04:00
Samuel Dacanay 0799fd9d46 Add support for older CentOS versions (6 & 5) by checking additional release files for information
Signed-off-by: Samuel Dacanay <sam.dacanay@anchore.com>
2021-09-03 16:06:10 +01:00
Alex Goodman 5de1a0a236 Revert "Add announcement for upcoming OSS meetup (#490)" (#491)
This reverts commit 035ef03b0c.

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-02 10:59:51 -04:00
Dan Luhring 035ef03b0c Add announcement for upcoming OSS meetup (#490)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-08-25 14:35:32 -04:00
Toure Dunnon 9acabb8307 Merge pull request #489 from anchore/SLES_support_berkeleyDB
Updated the distro package to include SLES
2021-08-23 16:14:52 -04:00
Toure Dunnon 0f0fe59bf2 Updated the distro package to include SLES type to expand support, this change will not include the new sqlite3 rpmDB format
Signed-off-by: Toure Dunnon <toure.dunnon@anchore.com>
2021-08-23 15:05:39 -04:00
Keith Zantow ff828fbac2 Skip package-lock.json and yarn.lock in node_modules #431 (#485)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-08-20 13:50:28 -04:00
cba5b5723b Added parser for Pipfile.lock to cataloger (#473)
* Added parser for Pipfile.lock to cataloger

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* make lint-fix

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* Update syft/pkg/cataloger/python/parse_pipfile_lock.go

Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* fix _version

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* swap method for trimming "==" prefix from pipfile pkg versions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-19 10:32:10 -04:00
Alex Goodman b0e9b92a25 Merge pull request #483 from anchore/fix-gomod-gosum-validation
Fix go module tidy checks for release pipeline
2021-08-18 11:00:49 -04:00
Alex Goodman d25d3b1fb3 fix go module tidy checks for release pipeline
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-18 10:13:56 -04:00
Keith Zantow 93b4bf9379 Add option to enable http registry connections (#482)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-08-17 12:52:51 -04:00
Keith Zantow 58f2be95fd chore: update packageurl-go and add encoding tests for #351 (#481)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-08-13 15:39:52 -04:00
Alex Goodman 6f038e9208 Merge pull request #447 from bureado/main
Minimal changes to use ID_LIKE
2021-08-12 14:20:13 -04:00
Alex Goodman 32a54b94be add unit tests for distro identification based on "like" field
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-12 14:09:22 -04:00
Alex Goodman 98d4749f86 Enhance CPE generation (#472)
* adjust CPE specificity sorting to include field length and bias certain fields

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove * vendor values from CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* re-enable generating CPEs for jenkins and jira plugins

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* improve CPE generation logic based on java artifactID and groupID

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add ruby-lang as target software candidate for gems in CPE generation logic

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename filterCpes to filterCPEs

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* refactor CPE filters and groupID processing (for linting)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use ruby-lang as vendor candidate not target software

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address PR comments for CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-09 19:52:19 +00:00
Alex Goodman 3a5168917e Update SPDX license list from 3.13 to 3.14 (#474)
* update SPDX license list from 3.13 to 3.14

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove license list version from spdx snapshot unit tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-09 19:47:33 +00:00
Alex Goodman 58b21c03c3 Merge pull request #461 from anchore/update-slack
Update slack channels
2021-07-12 08:28:01 -04:00
Alex Goodman 2093d2a1dc update slack channels
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-07-12 08:04:04 -04:00
Alex Goodman dd56450f92 Revert "Add callout for OSS meetup (#446)" (#459)
This reverts commit 66e71c39e2.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2021-07-02 14:29:27 +00:00
Alex Goodman 6f2b52971d directory resolver indexer should report one progressable (#457)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-07-02 10:18:41 -04:00
Alex Goodman fee74d57b8 Update documentation to show SPDX support (#452)
* add docs for spdx support

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* copy updates for format options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-07-01 14:52:57 +00:00
Alex Goodman fee35dd175 redirect cursor hide/show to stderr (#456)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-30 13:10:00 -04:00
Alex Goodman ecf4e5546a add help message when no arguments are provided (#455)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-30 12:53:12 -04:00
Alex Goodman 2de56c0749 force UI teardown when event is sourced from a signal interrupt (#453)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-29 18:16:32 -04:00
Alex Goodman fb0857ff93 Add support for indexing root filesystem (#442)
* change directory resolver to ignore system runtime paths + drive by index

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add event/etui support for filesystem indexing (for dir resolver)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add warnings for path indexing problems

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add directory resolver index tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* improve testing around directory resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* renamed p var to path when not conflicting with import

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* pull docker image in CLI dir scan timeout test

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* ensure file not exist errors do not stop directory resolver indexing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-29 22:06:47 +00:00
Jose Miguel Parrella c833472b6e If distro name is unknown, try with the ID_LIKE
Signed-off-by: Jose Miguel Parrella <j@bureado.com>
2021-06-25 22:18:40 -07:00
Jose Miguel Parrella c825ae2404 Completing the test for distro version
Signed-off-by: Jose Miguel Parrella <j@bureado.com>
2021-06-25 22:18:40 -07:00
318 changed files with 31848 additions and 2433 deletions
+2 -2
View File
@@ -6,7 +6,7 @@ permit:
- ISC
ignore-packages:
# packageurl-go is released under the MIT license located in the root of the repo at /mit.LICENSE
- github.com/package-url/packageurl-go
- github.com/anchore/packageurl-go
# from: https://github.com/spdx/tools-golang/blob/main/LICENSE.code
# The tools-golang source code is provided and may be used, at your option,
@@ -14,4 +14,4 @@ ignore-packages:
# * Apache License, version 2.0 (Apache-2.0), OR
# * GNU General Public License, version 2.0 or later (GPL-2.0-or-later).
# (we choose Apache-2.0)
- github.com/spdx/tools-golang
- github.com/spdx/tools-golang
+1 -1
View File
@@ -3,4 +3,4 @@ contact_links:
- name: Join the Slack community 💬
# link to our community Slack registration page
url: https://anchore.com/slack
about: 'Be sure to check out the channels #toolbox-help (ask us questions) and #toolbox-dev (collaborate with us on design and development)!'
about: 'Come chat with us! Ask for help, join our software development efforts, or just give us feedback!'
+15 -11
View File
@@ -1,27 +1,31 @@
#!/usr/bin/env bash
set -u
set -eu
TMP_GOMOD=$(mktemp)
TMP_GOSUM=$(mktemp)
ORIGINAL_STATE_DIR=$(mktemp -d "TEMP-original-state-XXXXXXXXX")
TIDY_STATE_DIR=$(mktemp -d "TEMP-tidy-state-XXXXXXXXX")
trap "rm -f ${TMP_GOSUM} ${TMP_GOMOD}" EXIT
trap "cp -v ${ORIGINAL_STATE_DIR}/* ./ && rm -fR ${ORIGINAL_STATE_DIR} ${TIDY_STATE_DIR}" EXIT
cp go.mod "${TMP_GOMOD}"
cp go.sum "${TMP_GOSUM}"
echo "Capturing original state of files..."
cp -v go.mod go.sum "${ORIGINAL_STATE_DIR}"
echo "Capturing state of go.mod and go.sum after running go mod tidy..."
go mod tidy
cp -v go.mod go.sum "${TIDY_STATE_DIR}"
echo ""
DIFF_MOD=$(diff -u "${TMP_GOMOD}" go.mod)
DIFF_SUM=$(diff -u "${TMP_GOSUM}" go.sum)
set +e
cp "${TMP_GOMOD}" go.mod
cp "${TMP_GOSUM}" go.sum
# Detect difference between the git HEAD state and the go mod tidy state
DIFF_MOD=$(diff -u "${ORIGINAL_STATE_DIR}/go.mod" "${TIDY_STATE_DIR}/go.mod")
DIFF_SUM=$(diff -u "${ORIGINAL_STATE_DIR}/go.sum" "${TIDY_STATE_DIR}/go.sum")
if [[ -n "${DIFF_MOD}" || -n "${DIFF_SUM}" ]]; then
echo "go.mod and/or go.sum are not tidy; please run go mod tidy"
echo "go.mod diff:"
echo "${DIFF_MOD}"
echo "go.sum diff:"
echo "${DIFF_SUM}"
echo ""
printf "FAILED! go.mod and/or go.sum are NOT tidy; please run 'go mod tidy'.\n\n"
exit 1
fi
+4
View File
@@ -158,6 +158,10 @@ jobs:
AC_USERNAME: ${{ secrets.ENG_CI_APPLE_ID }} # Used during macOS notarization.
AC_PASSWORD: ${{ secrets.ENG_CI_APPLE_ID_PASS }} # Used during macOS notarization.
- uses: anchore/sbom-action@v0
with:
artifact-name: sbom.spdx.json
- uses: 8398a7/action-slack@v3
with:
status: ${{ job.status }}
+39 -5
View File
@@ -4,35 +4,59 @@
# include:
# - EXC0002 # disable excluding of issues about comments from golint
linters-settings:
cyclop:
# the maximal code complexity to report
max-complexity: 15
linters:
# inverted configuration with `enable-all` and `disable` is not scalable during updates of golangci-lint
disable-all: true
enable:
- asciicheck
- bodyclose
- cyclop
- deadcode
- depguard
- dogsled
- dupl
- durationcheck
- errcheck
- errname
- errorlint
- exportloopref
- forbidigo
- forcetypeassert
- funlen
- gci
- gocognit
- goconst
- gocritic
- gocyclo
- gofmt
- gofumpt
- goimports
- golint
- gomoddirectives
- gomodguard
- goprintffuncname
- gosec
- gosimple
- govet
- ifshort
- importas
- ineffassign
# - ireturn # will be added in 1.43.0 (accept interfaces, return structs)
- makezero
- misspell
- nakedret
- nilerr
# - nilnil # will be added in 1.43.0
- nolintlint
- paralleltest
- predeclared
- revive
- rowserrcheck
- scopelint
- staticcheck
- structcheck
- stylecheck
@@ -41,19 +65,29 @@ linters:
- unparam
- unused
- varcheck
- wastedassign
- whitespace
# - wrapcheck
# do not enable...
# - exhaustive # makes to many assumptions about enums and if a switch case should contain all enum instances
# - exhaustivestruct # we like to depend on zero values
# - gochecknoglobals
# - gochecknoinits # this is too aggressive
# - godot
# - godox
# - goerr113
# - goerr113 # does not allow use of dynamic errors
# - golint # replaced by revive
# - gomnd # this is too aggressive
# - interfacer # this is a good idea, but is no longer supported and is prone to false positives
# - lll # without a way to specify per-line exception cases, this is not usable
# - maligned # this is an excellent linter, but tricky to optimize and we are not sensitive to memory layout optimizations
# - nestif
# - nestif # other complexity linters already cover this well
# - nlreturn # no robust auto-fixers supported
# - noctx # we don't use context a lot
# - prealloc # following this rule isn't consistently a good idea, as it sometimes forces unnecessary allocations that result in less idiomatic code
# - scopelint # replaced by exportloopref
# - tagliatelle # too presumptive at which struct tag conventions should be used
# - testpackage
# - wsl
# - wsl # very verbose and there is no auto-fixer for this project yet https://github.com/bombsimon/wsl/issues/90
+11 -6
View File
@@ -1,12 +1,12 @@
# Contributing to `syft`
# Contributing to Syft
If you are looking to contribute to this project and want to open a Github pull request ("PR"), there are a few guidelines of what we are looking for in patches. Make sure you go through this document and ensure that your code proposal is aligned.
If you are looking to contribute to this project and want to open a GitHub pull request ("PR"), there are a few guidelines of what we are looking for in patches. Make sure you go through this document and ensure that your code proposal is aligned.
## Sign your work
## Sign off your work
The `sign-off` is an added line at the end of the explanation for the commit, certifying that you wrote it or otherwise have the right to submit it as an open-source patch. By submitting a contribution, you agree to be bound by the terms of the DCO Version 1.1 and Apache License Version 2.0.
Signing a commit certifies the below Developer's Certificate of Origin (DCO):
Signing off a commit certifies the below Developer's Certificate of Origin (DCO):
```text
Developer's Certificate of Origin 1.1
@@ -51,13 +51,13 @@ $ git config --global user.name "John Doe"
$ git config --global user.email "john.doe@example.com"
```
Creating a signed commit is then possible with `-s`:
Creating a signed-off commit is then possible with `-s` or `--signoff`:
```text
$ git commit -s -m "this is a commit message"
```
To double check if the commit was signed, look at the log output:
To double-check that the commit was signed-off, look at the log output:
```text
$ git log -1
@@ -82,6 +82,11 @@ Date: Mon Aug 1 11:27:13 2020 -0400
[//]: # (TODO: PR title and description)
## Sign your commits
To ensure the authenticity and integrity of code contributions, **we require that all commits are signed**. Signing commits proves that your commits were truly created by you, as the holder of a private key.
Configuring git to sign your commits is a straightforward process. To get set up, see [GitHub's documentation on signing commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits).
## Test your changes
+2 -3
View File
@@ -101,14 +101,13 @@ $(TEMPDIR):
.PHONY: bootstrap-tools
bootstrap-tools: $(TEMPDIR)
GO111MODULE=off GOBIN=$(shell realpath $(TEMPDIR)) go get -u golang.org/x/perf/cmd/benchstat
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ v1.26.0
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ v1.42.1
curl -sSfL https://raw.githubusercontent.com/wagoodman/go-bouncer/master/bouncer.sh | sh -s -- -b $(TEMPDIR)/ v0.2.0
curl -sfL https://install.goreleaser.com/github.com/goreleaser/goreleaser.sh | sh -s -- -b $(TEMPDIR)/ v0.160.0
curl -sfL https://install.goreleaser.com/github.com/goreleaser/goreleaser.sh | sh -s -- -b $(TEMPDIR)/ v0.177.0
.PHONY: bootstrap-go
bootstrap-go:
go mod download
go mod tidy # note: it is important that the go.sum is kept in a consistent state at all times (especially during release)
.PHONY: bootstrap
bootstrap: $(RESULTSDIR) bootstrap-go bootstrap-tools ## Download and install all go dependencies (+ prep tooling in the ./tmp dir)
+43 -32
View File
@@ -3,27 +3,19 @@
[![Validations](https://github.com/anchore/syft/actions/workflows/validations.yaml/badge.svg)](https://github.com/anchore/syft/actions/workflows/validations.yaml)
[![Go Report Card](https://goreportcard.com/badge/github.com/anchore/syft)](https://goreportcard.com/report/github.com/anchore/syft)
[![GitHub release](https://img.shields.io/github/release/anchore/syft.svg)](https://github.com/anchore/syft/releases/latest)
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/anchore/syft/blob/main/LICENSE)
[![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/anchore/syft.svg)](https://github.com/anchore/syft)
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/anchore/syft/blob/main/LICENSE)
[![Slack Invite](https://img.shields.io/badge/Slack-Join-blue?logo=slack)](https://anchore.com/slack)
A CLI tool and go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner tool like [Grype](https://github.com/anchore/grype).
## Happening soon: [OSS Virtual Meetup](https://get.anchore.com/anchore-oss-meetup-jun-30-2021/)
## We'll be at KubeCon 2021!
[**Register here!**](https://get.anchore.com/anchore-oss-meetup-jun-30-2021/)
Attending KubeCon 2021 in person? Join us for a meetup on **Tuesday, October 12th**.
**When:** June 30, 11am-noon PT
**What:** 3 fast-paced talks, with time for Q&A
- **Using Syft to Streamline Compliance** (Chet Burgess — Principal Engineer, Cisco)
- **Fast Container Scanning in CI: Three Ways to Use Grype in Your Builds** (Dan Luhring — Manager of Open Source Engineering, Anchore)
- **Building More Optimal Container Images with Dive** (Alex Goodman — Creator of [Dive](https://github.com/wagoodman/dive), Sr Software Engineer for Open Source, Anchore)
We’ll have free swag, giveaways, snacks, and sips. Space will be limited, so make sure to [save your seat](https://get.anchore.com/2021-kubecon-na-opensource-happy-hour/)!
---
![syft-demo](https://user-images.githubusercontent.com/590471/90277200-2a253000-de33-11ea-893f-32c219eea11a.gif)
@@ -37,6 +29,27 @@ A CLI tool and go library for generating a Software Bill of Materials (SBOM) fro
If you encounter an issue, please [let us know using the issue tracker](https://github.com/anchore/syft/issues).
## Installation
### Recommended
```bash
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
```
...or, you can specify a release version and destination directory for the installation:
```
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b <DESTINATION_DIR> <RELEASE_VERSION>
```
### Homebrew
```bash
brew tap anchore/syft
brew install syft
```
**Note**: Currently, Syft is built only for macOS and Linux.
## Getting started
To generate an SBOM for a Docker or OCI image:
@@ -56,7 +69,10 @@ To include software from all image layers in the SBOM, regardless of its presenc
syft packages <image> --scope all-layers
```
### Supported sources
Syft can generate a SBOM from a variety of sources:
```
# catalog a container image archive (from the result of `docker image save ...`, `podman save ...`, or `skopeo copy` commands)
syft packages path/to/image.tar
@@ -66,6 +82,7 @@ syft packages path/to/dir
```
Sources can be explicitly provided with a scheme:
```
docker:yourrepo/yourimage:tag use images from the Docker daemon
docker-archive:path/to/yourimage.tar use a tarball from disk for archives created from "docker save"
@@ -75,7 +92,10 @@ dir:path/to/yourproject read directly from a path on disk (any di
registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
```
### Output formats
The output format for Syft is configurable as well:
```
syft packages <image> -o <format>
```
@@ -83,26 +103,11 @@ syft packages <image> -o <format>
Where the `format`s available are:
- `json`: Use this to get as much information out of Syft as possible!
- `text`: A row-oriented, human-and-machine-friendly output.
- `cyclonedx`: A XML report conforming to the [CycloneDX 1.2](https://cyclonedx.org/) specification.
- `cyclonedx`: A XML report conforming to the [CycloneDX 1.2 specification](https://cyclonedx.org/specification/overview/).
- `spdx`: A tag-value formatted report conforming to the [SPDX 2.2 specification](https://spdx.github.io/spdx-spec/).
- `spdx-json`: A JSON report conforming to the [SPDX 2.2 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.2/schemas/spdx-schema.json).
- `table`: A columnar summary (default).
## Installation
**Recommended (macOS and Linux)**
```bash
# install the latest version to /usr/local/bin
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
# install a specific version into a specific dir
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b <SOME_BIN_PATH> <RELEASE_VERSION>
```
**Homebrew (macOS)**
```bash
brew tap anchore/syft
brew install syft
```
## Configuration
Configuration search paths:
@@ -123,6 +128,9 @@ output: "table"
# same as -q ; SYFT_QUIET env var
quiet: false
# same as --file; write output report to a file (default is to write to stdout)
file: ""
# enable/disable checking for application updates on startup
# same as SYFT_CHECK_FOR_APP_UPDATE env var
check-for-app-update: true
@@ -219,6 +227,9 @@ registry:
# skip TLS verification when communicating with the registry
# SYFT_REGISTRY_INSECURE_SKIP_TLS_VERIFY env var
insecure-skip-tls-verify: false
# use http instead of https when connecting to the registry
# SYFT_REGISTRY_INSECURE_USE_HTTP env var
insecure-use-http: false
# credentials for specific registries
auth:
+5 -2
View File
@@ -87,9 +87,12 @@ This release process itself should be as automated as possible, and have only a
1. Tag the main branch with a full semantic-version, prefixed with a `v`. If there is a
milestone with a partial version, the full version should be used for the git tag (e.g.
with a Milestone of `v0.1` the tag should be `v0.1.0`).
with a Milestone of `v0.1` the tag should be `v0.1.0`). You can determine the changes going
into a release by running `make changelog-unreleased`. Use this change list to determine the
release increment. After determining the release increment (major, minor, patch), create the tag.
Given the above example the command to create the tag would be `git tag v0.1.0`.
1. Push the tag.
1. Push the tag. Given the above example the command to push the tag would be `git push origin v0.1.0`.
1. A release admin must approve the release on the GitHub Actions release pipeline run page.
Once approved, the release pipeline will generate all assets and draft a GitHub Release.
+2 -2
View File
@@ -4,14 +4,13 @@ import (
"fmt"
"os"
"github.com/spf13/cobra"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/logger"
"github.com/anchore/syft/syft"
"github.com/gookit/color"
"github.com/spf13/cobra"
"github.com/spf13/viper"
"github.com/wagoodman/go-partybus"
)
@@ -67,6 +66,7 @@ func initCmdAliasBindings() {
}
}
// nolint:forbidigo
func initAppConfig() {
cfg, err := config.LoadApplicationConfig(viper.GetViper(), persistentOpts)
if err != nil {
+1 -2
View File
@@ -8,7 +8,6 @@ import (
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/client"
"github.com/spf13/cobra"
)
@@ -87,7 +86,7 @@ func dockerImageValidArgsFunction(cmd *cobra.Command, args []string, toComplete
}
func listLocalDockerImages(prefix string) ([]string, error) {
var repoTags = make([]string, 0)
repoTags := make([]string, 0)
ctx := context.Background()
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
+20 -12
View File
@@ -2,6 +2,7 @@ package cmd
import (
"errors"
"fmt"
"os"
"github.com/anchore/syft/internal/log"
@@ -13,16 +14,19 @@ import (
// eventLoop listens to worker errors (from execution path), worker events (from a partybus subscription), and
// signal interrupts. Is responsible for handling each event relative to a given UI an to coordinate eventing until
// an eventual graceful exit.
// nolint:gocognit
func eventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *partybus.Subscription, ux ui.UI, cleanupFn func()) error {
// nolint:gocognit,funlen
func eventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *partybus.Subscription, cleanupFn func(), uxs ...ui.UI) error {
defer cleanupFn()
events := subscription.Events()
var err error
if ux, err = setupUI(subscription.Unsubscribe, ux); err != nil {
var ux ui.UI
if ux, err = setupUI(subscription.Unsubscribe, uxs...); err != nil {
return err
}
var retErr error
var forceTeardown bool
for {
if workerErrs == nil && events == nil {
@@ -66,25 +70,29 @@ func eventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *
// of processing.
events = nil
workerErrs = nil
forceTeardown = true
}
}
if err := ux.Teardown(); err != nil {
if err := ux.Teardown(forceTeardown); err != nil {
retErr = multierror.Append(retErr, err)
}
return retErr
}
func setupUI(unsubscribe func() error, ux ui.UI) (ui.UI, error) {
if err := ux.Setup(unsubscribe); err != nil {
// replace the existing UI with a (simpler) logger UI
ux = ui.NewLoggerUI()
// setupUI takes one or more UIs that responds to events and takes a event bus unsubscribe function for use
// during teardown. With the given UIs, the first UI which the ui.Setup() function does not return an error
// will be utilized in execution. Providing a set of UIs allows for the caller to provide graceful fallbacks
// when there are environmental problem (e.g. unable to setup a TUI with the current TTY).
func setupUI(unsubscribe func() error, uis ...ui.UI) (ui.UI, error) {
for _, ux := range uis {
if err := ux.Setup(unsubscribe); err != nil {
// something is very wrong, bail.
return ux, err
log.Warnf("unable to setup given UI, falling back to alternative UI: %+v", err)
continue
}
log.Errorf("unable to setup given UI, falling back to logger: %+v", err)
return ux, nil
}
return ux, nil
return nil, fmt.Errorf("unable to setup any UI")
}
+7 -7
View File
@@ -37,7 +37,7 @@ func (u *uiMock) Handle(event partybus.Event) error {
return u.Called(event).Error(0)
}
func (u *uiMock) Teardown() error {
func (u *uiMock) Teardown(_ bool) error {
u.t.Logf("UI Teardown called")
return u.Called().Error(0)
}
@@ -96,8 +96,8 @@ func Test_eventLoop_gracefulExit(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
)
@@ -159,8 +159,8 @@ func Test_eventLoop_workerError(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
workerErr,
"should have seen a worker error, but did not",
@@ -230,8 +230,8 @@ func Test_eventLoop_unsubscribeError(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
)
@@ -300,8 +300,8 @@ func Test_eventLoop_handlerError(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
finalEvent.Error,
"should have seen a event error, but did not",
@@ -355,8 +355,8 @@ func Test_eventLoop_signalsStopExecution(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
)
@@ -425,8 +425,8 @@ func Test_eventLoop_uiTeardownError(t *testing.T) {
worker(),
signaler(),
subscription,
ux,
cleanupFn,
ux,
),
teardownError,
"should have seen a UI teardown error, but did not",
+34 -12
View File
@@ -28,12 +28,14 @@ import (
const (
packagesExample = ` {{.appName}} {{.command}} alpine:latest a summary of discovered packages
{{.appName}} {{.command}} alpine:latest -o json show all possible cataloging details
{{.appName}} {{.command}} alpine:latest -o cyclonedx show a CycloneDX SBOM
{{.appName}} {{.command}} alpine:latest -o cyclonedx show a CycloneDX formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx show a SPDX 2.2 tag-value formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx-json show a SPDX 2.2 JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -vv show verbose debug information
Supports the following image sources:
{{.appName}} {{.command}} yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} path/to/a/file/or/dir a Docker tar, OCI tar, OCI directory, or generic filesystem directory
{{.appName}} {{.command}} path/to/a/file/or/dir a Docker tar, OCI tar, OCI directory, or generic filesystem directory
You can also explicitly specify the scheme to use:
{{.appName}} {{.command}} docker:yourrepo/yourimage:tag explicitly use the Docker daemon
@@ -47,7 +49,7 @@ const (
var (
packagesPresenterOpt packages.PresenterOption
packagesArgs = cobra.MinimumNArgs(1)
packagesArgs = cobra.MaximumNArgs(1)
packagesCmd = &cobra.Command{
Use: "packages [SOURCE]",
Short: "Generate a package SBOM",
@@ -65,8 +67,7 @@ var (
if err != nil {
return err
}
// silently exit
return fmt.Errorf("")
return fmt.Errorf("an image/directory argument is required")
}
// set the presenter
@@ -101,7 +102,7 @@ func init() {
}
func setPackageFlags(flags *pflag.FlagSet) {
///////// Formatting & Input options //////////////////////////////////////////////
// Formatting & Input options //////////////////////////////////////////////
flags.StringP(
"scope", "s", source.SquashedScope.String(),
@@ -112,7 +113,12 @@ func setPackageFlags(flags *pflag.FlagSet) {
fmt.Sprintf("report output formatter, options=%v", packages.AllPresenters),
)
///////// Upload options //////////////////////////////////////////////////////////
flags.StringP(
"file", "", "",
"file to write the report output to (default is STDOUT)",
)
// Upload options //////////////////////////////////////////////////////////
flags.StringP(
"host", "H", "",
"the hostname or URL of the Anchore Enterprise instance to upload to",
@@ -145,7 +151,7 @@ func setPackageFlags(flags *pflag.FlagSet) {
}
func bindPackagesConfigOptions(flags *pflag.FlagSet) error {
///////// Formatting & Input options //////////////////////////////////////////////
// Formatting & Input options //////////////////////////////////////////////
if err := viper.BindPFlag("package.cataloger.scope", flags.Lookup("scope")); err != nil {
return err
@@ -155,7 +161,11 @@ func bindPackagesConfigOptions(flags *pflag.FlagSet) error {
return err
}
///////// Upload options //////////////////////////////////////////////////////////
if err := viper.BindPFlag("file", flags.Lookup("file")); err != nil {
return err
}
// Upload options //////////////////////////////////////////////////////////
if err := viper.BindPFlag("anchore.host", flags.Lookup("host")); err != nil {
return err
@@ -187,12 +197,24 @@ func bindPackagesConfigOptions(flags *pflag.FlagSet) error {
func packagesExec(_ *cobra.Command, args []string) error {
// could be an image or a directory, with or without a scheme
userInput := args[0]
reporter, closer, err := reportWriter()
defer func() {
if err := closer(); err != nil {
log.Warnf("unable to write to report destination: %+v", err)
}
}()
if err != nil {
return err
}
return eventLoop(
packagesExecWorker(userInput),
setupSignals(),
eventSubscription,
ui.Select(appConfig.CliOptions.Verbosity > 0, appConfig.Quiet),
stereoscope.Cleanup,
ui.Select(appConfig.CliOptions.Verbosity > 0, appConfig.Quiet, reporter)...,
)
}
@@ -236,7 +258,7 @@ func packagesExecWorker(userInput string) <-chan error {
return errs
}
func runPackageSbomUpload(src source.Source, s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, scope source.Scope) error {
func runPackageSbomUpload(src *source.Source, s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, scope source.Scope) error {
log.Infof("uploading results to %s", appConfig.Anchore.Host)
if src.Metadata.Scheme != source.ImageScheme {
@@ -281,7 +303,7 @@ func runPackageSbomUpload(src source.Source, s source.Metadata, catalog *pkg.Cat
}
if err := c.Import(context.Background(), importCfg); err != nil {
return fmt.Errorf("failed to upload results to host=%s: %+v", appConfig.Anchore.Host, err)
return fmt.Errorf("failed to upload results to host=%s: %w", appConfig.Anchore.Host, err)
}
return nil
}
+23 -7
View File
@@ -7,6 +7,7 @@ import (
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/presenter/poweruser"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft/event"
@@ -34,11 +35,19 @@ var powerUserCmd = &cobra.Command{
"appName": internal.ApplicationName,
"command": "power-user",
}),
Args: cobra.ExactArgs(1),
Args: cobra.MaximumNArgs(1),
Hidden: true,
SilenceUsage: true,
SilenceErrors: true,
PreRunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
err := cmd.Help()
if err != nil {
return err
}
return fmt.Errorf("an image/directory argument is required")
}
if appConfig.Dev.ProfileCPU && appConfig.Dev.ProfileMem {
return fmt.Errorf("cannot profile CPU and memory simultaneously")
}
@@ -65,12 +74,24 @@ func init() {
func powerUserExec(_ *cobra.Command, args []string) error {
// could be an image or a directory, with or without a scheme
userInput := args[0]
reporter, closer, err := reportWriter()
defer func() {
if err := closer(); err != nil {
log.Warnf("unable to write to report destination: %+v", err)
}
}()
if err != nil {
return err
}
return eventLoop(
powerUserExecWorker(userInput),
setupSignals(),
eventSubscription,
ui.Select(appConfig.CliOptions.Verbosity > 0, appConfig.Quiet),
stereoscope.Cleanup,
ui.Select(appConfig.CliOptions.Verbosity > 0, appConfig.Quiet, reporter)...,
)
}
@@ -94,11 +115,6 @@ func powerUserExecWorker(userInput string) <-chan error {
}
defer cleanup()
if src.Metadata.Scheme != source.ImageScheme {
errs <- fmt.Errorf("the power-user subcommand only allows for 'image' schemes, given %q", src.Metadata.Scheme)
return
}
analysisResults := poweruser.JSONDocumentConfig{
SourceMetadata: src.Metadata,
ApplicationConfig: *appConfig,
+7 -7
View File
@@ -10,7 +10,7 @@ import (
"github.com/anchore/syft/syft/source"
)
type powerUserTask func(*poweruser.JSONDocumentConfig, source.Source) error
type powerUserTask func(*poweruser.JSONDocumentConfig, *source.Source) error
func powerUserTasks() ([]powerUserTask, error) {
var tasks []powerUserTask
@@ -42,7 +42,7 @@ func catalogPackagesTask() (powerUserTask, error) {
return nil, nil
}
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
packageCatalog, theDistro, err := syft.CatalogPackages(src, appConfig.Package.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -64,7 +64,7 @@ func catalogFileMetadataTask() (powerUserTask, error) {
metadataCataloger := file.NewMetadataCataloger()
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -110,7 +110,7 @@ func catalogFileDigestsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -142,7 +142,7 @@ func catalogSecretsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.Secrets.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -170,7 +170,7 @@ func catalogFileClassificationsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileClassification.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -197,7 +197,7 @@ func catalogContentsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src source.Source) error {
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileContents.Cataloger.ScopeOpt)
if err != nil {
return err
+29
View File
@@ -0,0 +1,29 @@
package cmd
import (
"fmt"
"io"
"os"
"strings"
"github.com/anchore/syft/internal/log"
)
func reportWriter() (io.Writer, func() error, error) {
nop := func() error { return nil }
path := strings.TrimSpace(appConfig.File)
switch len(path) {
case 0:
return os.Stdout, nop, nil
default:
reportFile, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0o644)
if err != nil {
return nil, nop, fmt.Errorf("unable to create report file: %w", err)
}
return reportFile, func() error {
log.Infof("report written to file=%q", path)
return reportFile.Close()
}, nil
}
}
+1
View File
@@ -24,6 +24,7 @@ var rootCmd = &cobra.Command{
ValidArgsFunction: packagesCmd.ValidArgsFunction,
}
// nolint:forbidigo
func init() {
// set universal flags
rootCmd.PersistentFlags().StringVarP(&persistentOpts.ConfigPath, "config", "c", "", "application config file")
+1 -1
View File
@@ -6,7 +6,6 @@ import (
"os"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/version"
"github.com/spf13/cobra"
)
@@ -24,6 +23,7 @@ func init() {
rootCmd.AddCommand(versionCmd)
}
// nolint:forbidigo
func printVersion(_ *cobra.Command, _ []string) {
versionInfo := version.FromBuild()
+5 -4
View File
@@ -7,10 +7,11 @@ require (
github.com/adrg/xdg v0.2.1
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf
github.com/anchore/go-rpmdb v0.0.0-20210602151223-1f0f707a2894
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/stereoscope v0.0.0-20210524175238-3b7662f3a66f
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29
github.com/anchore/stereoscope v0.0.0-20211005213828-538011008578
github.com/antihax/optional v1.0.0
github.com/bmatcuk/doublestar/v2 v2.0.4
github.com/docker/docker v17.12.0-ce-rc1.0.20200309214505-aa6a9891b09c+incompatible
@@ -21,11 +22,9 @@ require (
github.com/gookit/color v1.2.7
github.com/hashicorp/go-multierror v1.1.0
github.com/hashicorp/go-version v1.2.0
github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213
github.com/mitchellh/go-homedir v1.1.0
github.com/mitchellh/mapstructure v1.3.1
github.com/olekukonko/tablewriter v0.0.4
github.com/package-url/packageurl-go v0.1.0
github.com/pelletier/go-toml v1.8.0
github.com/pkg/profile v1.5.0
github.com/scylladb/go-set v1.0.2
@@ -37,6 +36,7 @@ require (
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.7.0
github.com/stretchr/testify v1.7.0
github.com/vifraa/gopom v0.1.0
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240
github.com/wagoodman/jotframe v0.0.0-20200730190914-3517092dd163
@@ -44,5 +44,6 @@ require (
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
golang.org/x/mod v0.3.0
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d
gopkg.in/yaml.v2 v2.3.0
)
+22 -16
View File
@@ -109,19 +109,20 @@ github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRF
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf/go.mod h1:FaODhIA06mxO1E6R32JE0TL1JWZZkmjRIAd4ULvHUKk=
github.com/anchore/go-rpmdb v0.0.0-20210602151223-1f0f707a2894 h1:VvCq7fFNU8dwWkCTGUykm4p64nVaDCYkKrj87x350Sk=
github.com/anchore/go-rpmdb v0.0.0-20210602151223-1f0f707a2894/go.mod h1:8jNYOxCJC5kyD/Ct4MbzsDN2hOhRoCAzQcb/7KdYYGw=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63 h1:C9W/LAydEz/qdUhx1MdjO9l8NEcFKYknkxDVyo9LAoM=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63/go.mod h1:6qH8c6U/3CBVvDDDBZnPSTbTINq3cIdADUYTaVf75EM=
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04 h1:VzprUTpc0vW0nnNKJfJieyH/TZ9UYAnTZs5/gHTdAe8=
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04/go.mod h1:6dK64g27Qi1qGQZ67gFmBFvEHScy0/C8qhQhNe5B5pQ=
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZVsCYMrIZBpFxwV26CbsuoEh5muXD5I1Ods=
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/stereoscope v0.0.0-20210524175238-3b7662f3a66f h1:bFadyOLOkzME3BrZFZ5m8cf/b2hsn3aMSS9s+SKubRk=
github.com/anchore/stereoscope v0.0.0-20210524175238-3b7662f3a66f/go.mod h1:vhh1M99rfWx5ejMvz1lkQiFZUrC5wu32V12R4JXH+ZI=
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29 h1:K9LfnxwhqvihqU0+MF325FNy7fsKV9EGaUxdfR4gnWk=
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29/go.mod h1:Oc1UkGaJwY6ND6vtAqPSlYrptKRJngHwkwB6W7l1uP0=
github.com/anchore/stereoscope v0.0.0-20211005213828-538011008578 h1:gSpftl0RWfdTwlPmsOLgEawHIw16xwwG1mFD/wrVDRE=
github.com/anchore/stereoscope v0.0.0-20211005213828-538011008578/go.mod h1:kL7jfbXblrDcBhu5ja/s+VTYL3Mzof4eQNMJiSqcwXQ=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/antihax/optional v1.0.0 h1:xK2lYat7ZLaVVcIuj82J8kIro4V6kDe0AUDFboUCwcg=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/apex/log v1.1.4/go.mod h1:AlpoD9aScyQfJDVHmLMEcx4oU6LqzkWp4Mg9GdAcEvQ=
github.com/apex/log v1.3.0 h1:1fyfbPvUwD10nMoh3hY6MXzvZShJQn9/ck7ATgAt5pA=
github.com/apex/log v1.3.0/go.mod h1:jd8Vpsr46WAe3EZSQ/IUMs2qQD/GOycT5rPWCO1yGcs=
github.com/apex/logs v0.0.4/go.mod h1:XzxuLZ5myVHDy9SAmYpamKKRNApGj54PfYLcFrXqDwo=
github.com/aphistic/golf v0.0.0-20180712155816-02c07f170c5a/go.mod h1:3NqKYiepwy8kCu4PNA+aP7WUV72eXWJeP9/r3/K9aLE=
@@ -241,6 +242,8 @@ github.com/fortytw2/leaktest v1.3.0/go.mod h1:jDsjWgpAGjm2CA7WthBh/CdZYEPF31XHqu
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/gabriel-vasile/mimetype v1.3.0 h1:4YOHITFLyYwF+iqG0ybSLGArRItynpfwdlWRmJnd75E=
github.com/gabriel-vasile/mimetype v1.3.0/go.mod h1:fA8fi6KUiG7MgQQ+mEWotXoEOvmxRtOJlERCzSmRvr8=
github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-critic/go-critic v0.4.1/go.mod h1:7/14rZGnZbY6E38VEGk2kVhoq6itzc1E68facVDK23g=
@@ -266,8 +269,8 @@ github.com/go-openapi/spec v0.19.3/go.mod h1:FpwSN1ksY1eteniUU7X0N/BgJ7a4WvBFVA8
github.com/go-openapi/swag v0.0.0-20160704191624-1d0bd113de87/go.mod h1:DXUve3Dpr1UfpPtxFw+EFuQ41HhCWZfha5jSVRG7C7I=
github.com/go-openapi/swag v0.19.2/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
github.com/go-restruct/restruct v0.0.0-20191227155143-5734170a48a1 h1:LoN2wx/aN8JPGebG+2DaUyk4M+xRcqJXfuIbs8AWHdE=
github.com/go-restruct/restruct v0.0.0-20191227155143-5734170a48a1/go.mod h1:KqrpKpn4M8OLznErihXTGLlsXFGeLxHUrLRRI/1YjGk=
github.com/go-restruct/restruct v1.2.0-alpha h1:2Lp474S/9660+SJjpVxoKuWX09JsXHSrdV7Nv3/gkvc=
github.com/go-restruct/restruct v1.2.0-alpha/go.mod h1:KqrpKpn4M8OLznErihXTGLlsXFGeLxHUrLRRI/1YjGk=
github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
@@ -466,7 +469,6 @@ github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/X
github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213 h1:qGQQKEcAR99REcMpsXCp3lJ03zYT1PkRd3kQGPn9GVg=
github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213/go.mod h1:vNUNkEQ1e29fT/6vq2aBdFsgNPmy8qMdSay1npru+Sw=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
@@ -590,8 +592,6 @@ github.com/opencontainers/runc v0.0.0-20190115041553-12f6a991201f/go.mod h1:qT5X
github.com/opencontainers/runc v0.1.1 h1:GlxAyO6x8rfZYN9Tt0Kti5a/cP41iuiO2yYT0IJGY8Y=
github.com/opencontainers/runc v0.1.1/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
github.com/opencontainers/runtime-spec v1.0.2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/package-url/packageurl-go v0.1.0 h1:efWBc98O/dBZRg1pw2xiDzovnlMjCa9NPnfaiBduh8I=
github.com/package-url/packageurl-go v0.1.0/go.mod h1:C/ApiuWpmbpni4DIOECf6WCjFUZV7O1Fx7VAzrZHgBw=
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k=
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
@@ -713,6 +713,7 @@ github.com/stretchr/testify v1.3.1-0.20190311161405-34c6fa2dc709/go.mod h1:M5WIy
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/subosito/gotenv v1.2.0 h1:Slr1R9HxAlEKefgq5jn9U+DnETlIUa6HfgEzj0g5d7s=
@@ -723,7 +724,6 @@ github.com/tetafro/godot v0.3.7/go.mod h1:/7NLHhv08H1+8DNj0MElpAACw1ajsCuf3TKNQx
github.com/tetafro/godot v0.4.2/go.mod h1:/7NLHhv08H1+8DNj0MElpAACw1ajsCuf3TKNQxA5S+0=
github.com/timakin/bodyclose v0.0.0-20190930140734-f7f2e9bca95e/go.mod h1:Qimiffbc6q9tBWlVV6x0P9sat/ao1xEkREYPPj9hphk=
github.com/timakin/bodyclose v0.0.0-20200424151742-cb6215831a94/go.mod h1:Qimiffbc6q9tBWlVV6x0P9sat/ao1xEkREYPPj9hphk=
github.com/tj/assert v0.0.0-20171129193455-018094318fb0 h1:Rw8kxzWo1mr6FSaYXjQELRe88y2KdfynXdnK72rdjtA=
github.com/tj/assert v0.0.0-20171129193455-018094318fb0/go.mod h1:mZ9/Rh9oLWpLLDRpvE+3b7gP/C2YyLFYxNmcLnPTMe0=
github.com/tj/go-elastic v0.0.0-20171221160941-36157cbbebc2/go.mod h1:WjeM0Oo1eNAjXGDx2yma7uG2XoyRZTq1uv3M/o7imD0=
github.com/tj/go-kinesis v0.0.0-20171128231115-08b17f58cb1b/go.mod h1:/yhzCV0xPfx6jb1bBgRFjl5lytqVqZXEaeqWP8lTEao=
@@ -746,6 +746,8 @@ github.com/valyala/fasthttp v1.2.0/go.mod h1:4vX61m6KN+xDduDNwXrhIAVZaZaZiQ1luJk
github.com/valyala/quicktemplate v1.2.0/go.mod h1:EH+4AkTd43SvgIbQHYu59/cJyxDoOVRUAfrukLPuGJ4=
github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a/go.mod h1:v3UYOV9WzVtRmSR+PDvWpU/qWl4Wa5LApYYX4ZtKbio=
github.com/vdemeester/k8s-pkg-credentialprovider v1.17.4/go.mod h1:inCTmtUdr5KJbreVojo06krnTgaeAz/Z7lynpPk/Q2c=
github.com/vifraa/gopom v0.1.0 h1:v897eVxf6lflkEXzPmKbo4YhX2oS/LGjz7cqjWnSmCU=
github.com/vifraa/gopom v0.1.0/go.mod h1:oPa1dcrGrtlO37WPDBm5SqHAT+wTgF8An1Q71Z6Vv4o=
github.com/vmware/govmomi v0.20.3/go.mod h1:URlwyTFZX72RmxtxuaFL2Uj3fD1JTvZdx59bHWk6aFU=
github.com/wagoodman/go-partybus v0.0.0-20200526224238-eb215533f07d/go.mod h1:JPirS5jde/CF5qIjcK4WX+eQmKXdPc6vcZkJ/P0hfPw=
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5 h1:phTLPgMRDYTizrBSKsNSOa2zthoC2KsJsaY/8sg3rD8=
@@ -872,8 +874,9 @@ golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200602114024-627f9648deb9 h1:pNX+40auqi2JqRfOP1akLGtYcn15TUbkhwuCO3foqqM=
golang.org/x/net v0.0.0-20200602114024-627f9648deb9/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20210505024714-0287a6fb4125/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d h1:LO7XpTYMwTqxjLcGWPijK3vRXg1aWdlNOVOHRq45d7c=
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181106182150-f42d05182288/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -938,14 +941,17 @@ golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980 h1:OjiUf46hAmXblsZdnoSXsEUSKU8r1UEzcL5RVZ4gO9Y=
golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da h1:b3NXsE2LusjYGGjL5bxEVZZORm/YEFFrWFjR8eFrw/c=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.0.0-20160726164857-2910a502d2bf/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+4 -5
View File
@@ -6,8 +6,6 @@ import (
"fmt"
"time"
"github.com/antihax/optional"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/internal/bus"
@@ -15,6 +13,7 @@ import (
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/antihax/optional"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
)
@@ -64,7 +63,7 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
stage.Current = "starting session"
startOperation, _, err := c.client.ImportsApi.CreateOperation(authedCtx)
if err != nil {
var detail = "no details given"
detail := "no details given"
var openAPIErr external.GenericOpenAPIError
if errors.As(err, &openAPIErr) {
detail = string(openAPIErr.Body())
@@ -106,7 +105,7 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
_, _, err = c.client.ImagesApi.AddImage(authedCtx, imageModel, &opts)
if err != nil {
var detail = "no details given"
detail := "no details given"
var openAPIErr external.GenericOpenAPIError
if errors.As(err, &openAPIErr) {
detail = string(openAPIErr.Body())
@@ -122,7 +121,7 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
}
func addImageModel(imageMetadata image.Metadata, packageDigest, manifestDigest, dockerfileDigest, configDigest, sessionID string) external.ImageAnalysisRequest {
var tags = make([]string, len(imageMetadata.Tags))
tags := make([]string, len(imageMetadata.Tags))
for i, t := range imageMetadata.Tags {
tags[i] = t.String()
}
+1 -2
View File
@@ -8,10 +8,9 @@ import (
"fmt"
"net/http"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/log"
"github.com/wagoodman/go-progress"
)
type configImportAPI interface {
+2 -4
View File
@@ -6,11 +6,9 @@ import (
"fmt"
"net/http"
"github.com/wagoodman/go-progress"
"github.com/anchore/syft/internal/log"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/log"
"github.com/wagoodman/go-progress"
)
type dockerfileImportAPI interface {
+1 -2
View File
@@ -8,10 +8,9 @@ import (
"fmt"
"net/http"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/log"
"github.com/wagoodman/go-progress"
)
type manifestImportAPI interface {
+4 -7
View File
@@ -8,16 +8,13 @@ import (
"fmt"
"net/http"
"github.com/anchore/syft/internal/presenter/packages"
"github.com/wagoodman/go-progress"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/source"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/presenter/packages"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/wagoodman/go-progress"
)
type packageSBOMImportAPI interface {
+4 -2
View File
@@ -15,8 +15,10 @@ package bus
import "github.com/wagoodman/go-partybus"
var publisher partybus.Publisher
var active bool
var (
publisher partybus.Publisher
active bool
)
// SetPublisher sets the singleton event bus publisher. This is optional; if no bus is provided, the library will
// behave no differently than if a bus had been provided.
+1 -1
View File
@@ -29,6 +29,7 @@ type parser interface {
type Application struct {
ConfigPath string `yaml:",omitempty" json:"configPath"` // the location where the application config was read from (either from -c or discovered while loading)
Output string `yaml:"output" json:"output" mapstructure:"output"` // -o, the Presenter hint string to use for report formatting
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"` // -q, indicates to not show any status output to stderr (ETUI or logging UI)
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Anchore anchore `yaml:"anchore" json:"anchore" mapstructure:"anchore"` // options for interacting with Anchore Engine/Enterprise
@@ -143,7 +144,6 @@ func (cfg *Application) parseConfigValues() error {
func (cfg Application) String() string {
// yaml is pretty human friendly (at least when compared to json)
appCfgStr, err := yaml.Marshal(&cfg)
if err != nil {
return err.Error()
}
+4 -2
View File
@@ -4,7 +4,6 @@ import (
"os"
"github.com/anchore/stereoscope/pkg/image"
"github.com/spf13/viper"
)
@@ -20,11 +19,13 @@ type RegistryCredentials struct {
type registry struct {
InsecureSkipTLSVerify bool `yaml:"insecure-skip-tls-verify" json:"insecure-skip-tls-verify" mapstructure:"insecure-skip-tls-verify"`
InsecureUseHTTP bool `yaml:"insecure-use-http" json:"insecure-use-http" mapstructure:"insecure-use-http"`
Auth []RegistryCredentials `yaml:"auth" json:"auth" mapstructure:"auth"`
}
func (cfg registry) loadDefaultValues(v *viper.Viper) {
v.SetDefault("registry.insecure-skip-tls-verify", false)
v.SetDefault("registry.insecure-use-http", false)
v.SetDefault("registry.auth", []RegistryCredentials{})
}
@@ -56,7 +57,7 @@ func hasNonEmptyCredentials(username, password, token string) bool {
}
func (cfg *registry) ToOptions() *image.RegistryOptions {
var auth = make([]image.RegistryCredentials, len(cfg.Auth))
auth := make([]image.RegistryCredentials, len(cfg.Auth))
for i, a := range cfg.Auth {
auth[i] = image.RegistryCredentials{
Authority: a.Authority,
@@ -67,6 +68,7 @@ func (cfg *registry) ToOptions() *image.RegistryOptions {
}
return &image.RegistryOptions{
InsecureSkipTLSVerify: cfg.InsecureSkipTLSVerify,
InsecureUseHTTP: cfg.InsecureUseHTTP,
Credentials: auth,
}
}
+55
View File
@@ -2,6 +2,7 @@ package config
import (
"fmt"
"github.com/anchore/stereoscope/pkg/image"
"testing"
"github.com/stretchr/testify/assert"
@@ -54,3 +55,57 @@ func TestHasNonEmptyCredentials(t *testing.T) {
})
}
}
func Test_registry_ToOptions(t *testing.T) {
tests := []struct {
name string
input registry
expected image.RegistryOptions
}{
{
name: "no registry options",
input: registry{},
expected: image.RegistryOptions{
Credentials: []image.RegistryCredentials{},
},
},
{
name: "set InsecureSkipTLSVerify",
input: registry{
InsecureSkipTLSVerify: true,
},
expected: image.RegistryOptions{
InsecureSkipTLSVerify: true,
Credentials: []image.RegistryCredentials{},
},
},
{
name: "set InsecureUseHTTP",
input: registry{
InsecureUseHTTP: true,
},
expected: image.RegistryOptions{
InsecureUseHTTP: true,
Credentials: []image.RegistryCredentials{},
},
},
{
name: "set all bool options",
input: registry{
InsecureSkipTLSVerify: true,
InsecureUseHTTP: true,
},
expected: image.RegistryOptions{
InsecureSkipTLSVerify: true,
InsecureUseHTTP: true,
Credentials: []image.RegistryCredentials{},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, &test.expected, test.input.ToOptions())
})
}
}
+24
View File
@@ -1,7 +1,10 @@
package internal
import (
"errors"
"fmt"
"io"
"os"
"github.com/anchore/syft/internal/log"
)
@@ -12,3 +15,24 @@ func CloseAndLogError(closer io.Closer, location string) {
log.Warnf("unable to close file for location=%q: %+v", location, err)
}
}
type PathError struct {
Path string
Err error
}
func (e PathError) Error() string {
return fmt.Sprintf("unable to observe contents of %+v: %v", e.Path, e.Err)
}
func IsPathError(err error) bool {
return errors.As(err, &PathError{})
}
func IsErrPathPermission(err error) bool {
var pathErr *PathError
if errors.As(err, pathErr) {
return os.IsPermission(pathErr.Err)
}
return false
}
+3 -3
View File
@@ -24,12 +24,12 @@ const (
const perFileReadLimit = 2 * GB
type errZipSlipDetected struct {
type zipSlipDetectedError struct {
Prefix string
JoinArgs []string
}
func (e *errZipSlipDetected) Error() string {
func (e *zipSlipDetectedError) Error() string {
return fmt.Sprintf("paths are not allowed to resolve outside of the root prefix (%q). Destination: %q", e.Prefix, e.JoinArgs)
}
@@ -197,7 +197,7 @@ func safeJoin(prefix string, dest ...string) (string, error) {
joinResult := filepath.Join(append([]string{prefix}, dest...)...)
cleanJoinResult := filepath.Clean(joinResult)
if !strings.HasPrefix(cleanJoinResult, filepath.Clean(prefix)) {
return "", &errZipSlipDetected{
return "", &zipSlipDetectedError{
Prefix: prefix,
JoinArgs: dest,
}
+3 -3
View File
@@ -299,7 +299,7 @@ func TestSafeJoin(t *testing.T) {
"../../../etc/passwd",
},
expected: "",
errAssertion: assertErrorAs(&errZipSlipDetected{}),
errAssertion: assertErrorAs(&zipSlipDetectedError{}),
},
{
prefix: "/a/place",
@@ -308,7 +308,7 @@ func TestSafeJoin(t *testing.T) {
"../",
},
expected: "",
errAssertion: assertErrorAs(&errZipSlipDetected{}),
errAssertion: assertErrorAs(&zipSlipDetectedError{}),
},
{
prefix: "/a/place",
@@ -316,7 +316,7 @@ func TestSafeJoin(t *testing.T) {
"../",
},
expected: "",
errAssertion: assertErrorAs(&errZipSlipDetected{}),
errAssertion: assertErrorAs(&zipSlipDetectedError{}),
},
}
+2 -1
View File
@@ -3,6 +3,7 @@ package file
import (
"archive/zip"
"encoding/binary"
"errors"
"fmt"
"io"
"os"
@@ -91,7 +92,7 @@ func findArchiveStartOffset(r io.ReaderAt, size int64) (startOfArchive uint64, e
bLen = size
}
buf = make([]byte, int(bLen))
if _, err := r.ReadAt(buf, size-bLen); err != nil && err != io.EOF {
if _, err := r.ReadAt(buf, size-bLen); err != nil && !errors.Is(err, io.EOF) {
return 0, err
}
if p := findSignatureInBlock(buf); p >= 0 {
+5 -2
View File
@@ -3,6 +3,7 @@ package logger
import (
"fmt"
"io"
"io/fs"
"io/ioutil"
"os"
@@ -10,6 +11,8 @@ import (
prefixed "github.com/x-cray/logrus-prefixed-formatter"
)
const defaultLogFilePermissions fs.FileMode = 0o644
// LogrusConfig contains all configurable values for the Logrus logger
type LogrusConfig struct {
EnableConsole bool
@@ -38,7 +41,7 @@ func NewLogrusLogger(cfg LogrusConfig) *LogrusLogger {
var output io.Writer
switch {
case cfg.EnableConsole && cfg.EnableFile:
logFile, err := os.OpenFile(cfg.FileLocation, os.O_WRONLY|os.O_CREATE, 0755)
logFile, err := os.OpenFile(cfg.FileLocation, os.O_WRONLY|os.O_CREATE, defaultLogFilePermissions)
if err != nil {
panic(fmt.Errorf("unable to setup log file: %w", err))
}
@@ -46,7 +49,7 @@ func NewLogrusLogger(cfg LogrusConfig) *LogrusLogger {
case cfg.EnableConsole:
output = os.Stderr
case cfg.EnableFile:
logFile, err := os.OpenFile(cfg.FileLocation, os.O_WRONLY|os.O_CREATE, 0755)
logFile, err := os.OpenFile(cfg.FileLocation, os.O_WRONLY|os.O_CREATE, defaultLogFilePermissions)
if err != nil {
panic(fmt.Errorf("unable to setup log file: %w", err))
}
+3 -3
View File
@@ -38,18 +38,18 @@ func NewJSONPackages(catalog *pkg.Catalog) ([]JSONPackage, error) {
// NewJSONPackage crates a new JSONPackage from the given pkg.Package.
func NewJSONPackage(p *pkg.Package) (JSONPackage, error) {
var cpes = make([]string, len(p.CPEs))
cpes := make([]string, len(p.CPEs))
for i, c := range p.CPEs {
cpes[i] = c.BindToFmtString()
}
// ensure collections are never nil for presentation reasons
var locations = make([]source.Location, 0)
locations := make([]source.Location, 0)
if p.Locations != nil {
locations = p.Locations
}
var licenses = make([]string, 0)
licenses := make([]string, 0)
if p.Licenses != nil {
licenses = p.Licenses
}
@@ -10,7 +10,7 @@ type CreationInfo struct {
Created time.Time `json:"created"`
// Identify who (or what, in the case of a tool) created the SPDX file. If the SPDX file was created by an
// individual, indicate the person's name. If the SPDX file was created on behalf of a company or organization,
//indicate the entity name. If the SPDX file was created using a software tool, indicate the name and version
// indicate the entity name. If the SPDX file was created using a software tool, indicate the name and version
// for that tool. If multiple participants or tools were involved, use multiple instances of this field. Person
// name or organization name may be designated as “anonymous” if appropriate.
Creators []string `json:"creators"`
@@ -40,4 +40,6 @@ type Document struct {
Files []File `json:"files,omitempty"`
// Snippets referenced in the SPDX document
Snippets []Snippet `json:"snippets,omitempty"`
// Relationships referenced in the SPDX document
Relationships []Relationship `json:"relationships,omitempty"`
}
@@ -20,22 +20,22 @@ type File struct {
Item
// (At least one is required.) The checksum property provides a mechanism that can be used to verify that the
// contents of a File or Package have not changed.
Checksums []Checksum `json:"checksums"`
Checksums []Checksum `json:"checksums,omitempty"`
// This field provides a place for the SPDX file creator to record file contributors. Contributors could include
// names of copyright holders and/or authors who may not be copyright holders yet contributed to the file content.
FileContributors []string `json:"fileContributors"`
FileContributors []string `json:"fileContributors,omitempty"`
// Each element is a SPDX ID for a File.
FileDependencies []string `json:"fileDependencies"`
FileDependencies []string `json:"fileDependencies,omitempty"`
// The name of the file relative to the root of the package.
FileName string `json:"fileName"`
// The type of the file
FileTypes []string `json:"fileTypes"`
FileTypes []string `json:"fileTypes,omitempty"`
// This field provides a place for the SPDX file creator to record potential legal notices found in the file.
// This may or may not include copyright statements.
NoticeText string `json:"noticeText,omitempty"`
// Indicates the project in which the SpdxElement originated. Tools must preserve doap:homepage and doap:name
// properties and the URI (if one is known) of doap:Project resources that are values of this property. All other
// properties of doap:Projects are not directly supported by SPDX and may be dropped when translating to or
// from some SPDX formats.
ArtifactOf []string `json:"artifactOf"`
// from some SPDX formats(deprecated).
ArtifactOf []string `json:"artifactOf,omitempty"`
}
@@ -1,11 +1,13 @@
package spdx22
type Relationship struct {
// SPDX ID for SpdxElement. A related SpdxElement.
RelatedSpdxElement string `json:"relatedSpdxElement"`
// Id to which the SPDX element is related
SpdxElementID string `json:"spdxElementId"`
// Describes the type of relationship between two SPDX elements.
RelationshipType RelationshipType `json:"relationshipType"`
Comment string `json:"comment,omitempty"`
// SPDX ID for SpdxElement. A related SpdxElement.
RelatedSpdxElement string `json:"relatedSpdxElement"`
Comment string `json:"comment,omitempty"`
}
// source: https://spdx.github.io/spdx-spec/7-relationships-between-SPDX-elements/
+40 -1
View File
@@ -1,7 +1,9 @@
package packages
import (
"crypto/sha256"
"fmt"
"path/filepath"
"strings"
"github.com/anchore/syft/internal/presenter/packages/model/spdx22"
@@ -29,6 +31,43 @@ func getSPDXExternalRefs(p *pkg.Package) (externalRefs []spdx22.ExternalRef) {
return externalRefs
}
func getSPDXFiles(packageSpdxID string, p *pkg.Package) (files []spdx22.File, fileIDs []string, relationships []spdx22.Relationship) {
files = make([]spdx22.File, 0)
fileIDs = make([]string, 0)
relationships = make([]spdx22.Relationship, 0)
pkgFileOwner, ok := p.Metadata.(pkg.FileOwner)
if !ok {
return files, fileIDs, relationships
}
for _, ownedFilePath := range pkgFileOwner.OwnedFiles() {
baseFileName := filepath.Base(ownedFilePath)
pathHash := sha256.Sum256([]byte(ownedFilePath))
fileSpdxID := spdx22.ElementID(fmt.Sprintf("File-%s-%x", p.Name, pathHash)).String()
fileIDs = append(fileIDs, fileSpdxID)
files = append(files, spdx22.File{
FileName: ownedFilePath,
Item: spdx22.Item{
Element: spdx22.Element{
SPDXID: fileSpdxID,
Name: baseFileName,
},
},
})
relationships = append(relationships, spdx22.Relationship{
SpdxElementID: packageSpdxID,
RelationshipType: spdx22.ContainsRelationship,
RelatedSpdxElement: fileSpdxID,
})
}
return files, fileIDs, relationships
}
func getSPDXLicense(p *pkg.Package) string {
// source: https://spdx.github.io/spdx-spec/3-package-information/#313-concluded-license
// The options to populate this field are limited to:
@@ -96,7 +135,7 @@ func getSPDXHomepage(p *pkg.Package) string {
}
func getSPDXSourceInfo(p *pkg.Package) string {
answer := ""
var answer string
switch p.Type {
case pkg.RpmPkg:
answer = "acquired package info from RPM DB"
@@ -95,6 +95,15 @@ func Test_getSPDXLicense(t *testing.T) {
},
expected: "GPL-3.0",
},
{
name: "debian to spdx conversion",
input: pkg.Package{
Licenses: []string{
"GPL-2",
},
},
expected: "GPL-2.0",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
@@ -4,6 +4,8 @@ import (
"encoding/json"
"fmt"
"io"
"path"
"strings"
"time"
"github.com/anchore/syft/internal"
@@ -12,8 +14,11 @@ import (
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/google/uuid"
)
const anchoreNamespace = "https://anchore.com/syft"
// SPDXJsonPresenter is a SPDX presentation object for the syft results (see https://github.com/spdx/spdx-spec)
type SPDXJsonPresenter struct {
catalog *pkg.Catalog
@@ -39,15 +44,29 @@ func (pres *SPDXJsonPresenter) Present(output io.Writer) error {
return enc.Encode(&doc)
}
// newSPDXJsonDocument creates and populates a new JSON document struct that follows the SPDX 2.2 spec from the given cataloging results.
func newSPDXJsonDocument(catalog *pkg.Catalog, srcMetadata source.Metadata) spdx22.Document {
var name string
uniqueID := uuid.Must(uuid.NewRandom())
var name, input, identifier string
switch srcMetadata.Scheme {
case source.ImageScheme:
name = srcMetadata.ImageMetadata.UserInput
name = cleanSPDXName(srcMetadata.ImageMetadata.UserInput)
input = "image"
case source.DirectoryScheme:
name = srcMetadata.Path
name = cleanSPDXName(srcMetadata.Path)
input = "dir"
}
if name != "." {
identifier = path.Join(input, fmt.Sprintf("%s-%s", name, uniqueID.String()))
} else {
identifier = path.Join(input, uniqueID.String())
}
namespace := path.Join(anchoreNamespace, identifier)
packages, files, relationships := newSPDXJsonElements(catalog)
return spdx22.Document{
Element: spdx22.Element{
SPDXID: spdx22.ElementID("DOCUMENT").String(),
@@ -64,23 +83,35 @@ func newSPDXJsonDocument(catalog *pkg.Catalog, srcMetadata source.Metadata) spdx
LicenseListVersion: spdxlicense.Version,
},
DataLicense: "CC0-1.0",
DocumentNamespace: fmt.Sprintf("https://anchore.com/syft/image/%s", srcMetadata.ImageMetadata.UserInput),
Packages: newSPDXJsonPackages(catalog),
DocumentNamespace: namespace,
Packages: packages,
Files: files,
Relationships: relationships,
}
}
func newSPDXJsonPackages(catalog *pkg.Catalog) []spdx22.Package {
results := make([]spdx22.Package, 0)
func newSPDXJsonElements(catalog *pkg.Catalog) ([]spdx22.Package, []spdx22.File, []spdx22.Relationship) {
packages := make([]spdx22.Package, 0)
relationships := make([]spdx22.Relationship, 0)
files := make([]spdx22.File, 0)
for _, p := range catalog.Sorted() {
license := getSPDXLicense(p)
packageSpdxID := spdx22.ElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.Version)).String()
packageFiles, fileIDs, packageFileRelationships := getSPDXFiles(packageSpdxID, p)
files = append(files, packageFiles...)
relationships = append(relationships, packageFileRelationships...)
// note: the license concluded and declared should be the same since we are collecting license information
// from the project data itself (the installed package files).
results = append(results, spdx22.Package{
packages = append(packages, spdx22.Package{
Description: getSPDXDescription(p),
DownloadLocation: getSPDXDownloadLocation(p),
ExternalRefs: getSPDXExternalRefs(p),
FilesAnalyzed: false,
HasFiles: fileIDs,
Homepage: getSPDXHomepage(p),
LicenseDeclared: license, // The Declared License is what the authors of a project believe govern the package
Originator: getSPDXOriginator(p),
@@ -89,11 +120,23 @@ func newSPDXJsonPackages(catalog *pkg.Catalog) []spdx22.Package {
Item: spdx22.Item{
LicenseConcluded: license, // The Concluded License field is the license the SPDX file creator believes governs the package
Element: spdx22.Element{
SPDXID: spdx22.ElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.Version)).String(),
SPDXID: packageSpdxID,
Name: p.Name,
},
},
})
}
return results
return packages, files, relationships
}
func cleanSPDXName(name string) string {
// remove # according to specification
name = strings.ReplaceAll(name, "#", "-")
// remove : for url construction
name = strings.ReplaceAll(name, ":", "-")
// clean relative pathing
return path.Clean(name)
}
@@ -29,5 +29,12 @@ func TestSPDXJSONImagePresenter(t *testing.T) {
}
func spdxJsonRedactor(s []byte) []byte {
return regexp.MustCompile(`"created": .*`).ReplaceAll(s, []byte("redacted"))
// each SBOM reports the time it was generated, which is not useful during snapshot testing
s = regexp.MustCompile(`"created": .*`).ReplaceAll(s, []byte("redacted"))
// each SBOM reports a unique documentNamespace when generated, this is not useful for snapshot testing
s = regexp.MustCompile(`"documentNamespace": .*`).ReplaceAll(s, []byte("redacted"))
// the license list will be updated periodically, the value here should not be directly tested in snapshot tests
return regexp.MustCompile(`"licenseListVersion": .*`).ReplaceAll(s, []byte("redacted"))
}
@@ -5,9 +5,8 @@ import (
"io"
"time"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
@@ -29,5 +29,8 @@ func TestSPDXTagValueImagePresenter(t *testing.T) {
}
func spdxTagValueRedactor(s []byte) []byte {
return regexp.MustCompile(`Created: .*`).ReplaceAll(s, []byte("redacted"))
// each SBOM reports the time it was generated, which is not useful during snapshot testing
s = regexp.MustCompile(`Created: .*`).ReplaceAll(s, []byte("redacted"))
// the license list will be updated periodically, the value here should not be directly tested in snapshot tests
return regexp.MustCompile(`LicenseListVersion: .*`).ReplaceAll(s, []byte("redacted"))
}
@@ -6,9 +6,8 @@ import (
"sort"
"strings"
"github.com/olekukonko/tablewriter"
"github.com/anchore/syft/syft/pkg"
"github.com/olekukonko/tablewriter"
)
type TablePresenter struct {
@@ -27,6 +27,11 @@
"author": "",
"authorEmail": "",
"platform": "",
"files": [
{
"path": "/some/path/pkg1/depedencies/foo"
}
],
"sitePackagesRootPath": ""
}
},
@@ -9,7 +9,7 @@
"locations": [
{
"path": "/somefile-1.txt",
"layerID": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59"
"layerID": "sha256:ffb5e9eaa453a002110719d12c294960117ca2903953d1faa40f01dc3f77045c"
}
],
"licenses": [
@@ -40,7 +40,7 @@
"locations": [
{
"path": "/somefile-2.txt",
"layerID": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec"
"layerID": "sha256:8463854829fc53d47b9dcdf7ee79fe7eb4ca7933c910f67f8521412f7a2f5c21"
}
],
"licenses": [],
@@ -67,7 +67,7 @@
"type": "image",
"target": {
"userInput": "user-image-input",
"imageID": "sha256:5900c94a5bc1e083aa24ad1a223bf6eb9910dc8a6b01cb979ec306cb91709ea1",
"imageID": "sha256:112851310e48e604f7379e2a3acddab50e91ce926edacb598a532e60ff6b776a",
"manifestDigest": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"tags": [
@@ -77,17 +77,17 @@
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59",
"digest": "sha256:ffb5e9eaa453a002110719d12c294960117ca2903953d1faa40f01dc3f77045c",
"size": 22
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec",
"digest": "sha256:8463854829fc53d47b9dcdf7ee79fe7eb4ca7933c910f67f8521412f7a2f5c21",
"size": 16
}
],
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NjUsImRpZ2VzdCI6InNoYTI1Njo1OTAwYzk0YTViYzFlMDgzYWEyNGFkMWEyMjNiZjZlYjk5MTBkYzhhNmIwMWNiOTc5ZWMzMDZjYjkxNzA5ZWExIn0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1NjpmYjZiZWVjYjc1YjM5ZjRiYjgxM2RiZjE3N2U1MDFlZGQ1ZGRiM2U2OWJiNDVjZWRlYjc4YzY3NmVlMWI3YTU5In0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjMxOWI1ODhjZTY0MjUzYTg3YjUzM2M4ZWQwMWNmMDAyNWUwZWFjOThlN2I1MTZlMTI1MzI5NTdlMTI0NGZkZWMifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjEtMDQtMDZUMTk6MTM6NTIuNTI0Mzc4WiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIxLTA0LTA2VDE5OjEzOjUyLjQ1ODIwNjFaIiwiY3JlYXRlZF9ieSI6IkFERCBmaWxlLTEudHh0IC9zb21lZmlsZS0xLnR4dCAjIGJ1aWxka2l0IiwiY29tbWVudCI6ImJ1aWxka2l0LmRvY2tlcmZpbGUudjAifSx7ImNyZWF0ZWQiOiIyMDIxLTA0LTA2VDE5OjEzOjUyLjUyNDM3OFoiLCJjcmVhdGVkX2J5IjoiQUREIGZpbGUtMi50eHQgL3NvbWVmaWxlLTIudHh0ICMgYnVpbGRraXQiLCJjb21tZW50IjoiYnVpbGRraXQuZG9ja2VyZmlsZS52MCJ9XSwib3MiOiJsaW51eCIsInJvb3RmcyI6eyJ0eXBlIjoibGF5ZXJzIiwiZGlmZl9pZHMiOlsic2hhMjU2OmZiNmJlZWNiNzViMzlmNGJiODEzZGJmMTc3ZTUwMWVkZDVkZGIzZTY5YmI0NWNlZGViNzhjNjc2ZWUxYjdhNTkiLCJzaGEyNTY6MzE5YjU4OGNlNjQyNTNhODdiNTMzYzhlZDAxY2YwMDI1ZTBlYWM5OGU3YjUxNmUxMjUzMjk1N2UxMjQ0ZmRlYyJdfX0=",
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NzMsImRpZ2VzdCI6InNoYTI1NjoxMTI4NTEzMTBlNDhlNjA0ZjczNzllMmEzYWNkZGFiNTBlOTFjZTkyNmVkYWNiNTk4YTUzMmU2MGZmNmI3NzZhIn0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1NjpmZmI1ZTllYWE0NTNhMDAyMTEwNzE5ZDEyYzI5NDk2MDExN2NhMjkwMzk1M2QxZmFhNDBmMDFkYzNmNzcwNDVjIn0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2Ojg0NjM4NTQ4MjlmYzUzZDQ3YjlkY2RmN2VlNzlmZTdlYjRjYTc5MzNjOTEwZjY3Zjg1MjE0MTJmN2EyZjVjMjEifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjEtMDktMDhUMTc6MjE6NTguODk2NTI5MTkyWiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIxLTA5LTA4VDE3OjIxOjU4Ljg3OTY5MDgyNFoiLCJjcmVhdGVkX2J5IjoiQUREIGZpbGUtMS50eHQgL3NvbWVmaWxlLTEudHh0ICMgYnVpbGRraXQiLCJjb21tZW50IjoiYnVpbGRraXQuZG9ja2VyZmlsZS52MCJ9LHsiY3JlYXRlZCI6IjIwMjEtMDktMDhUMTc6MjE6NTguODk2NTI5MTkyWiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0yLnR4dCAvc29tZWZpbGUtMi50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn1dLCJvcyI6ImxpbnV4Iiwicm9vdGZzIjp7InR5cGUiOiJsYXllcnMiLCJkaWZmX2lkcyI6WyJzaGEyNTY6ZmZiNWU5ZWFhNDUzYTAwMjExMDcxOWQxMmMyOTQ5NjAxMTdjYTI5MDM5NTNkMWZhYTQwZjAxZGMzZjc3MDQ1YyIsInNoYTI1Njo4NDYzODU0ODI5ZmM1M2Q0N2I5ZGNkZjdlZTc5ZmU3ZWI0Y2E3OTMzYzkxMGY2N2Y4NTIxNDEyZjdhMmY1YzIxIl19fQ==",
"repoDigests": [],
"scope": "Squashed"
}
@@ -3,12 +3,12 @@
"name": "/some/path",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"created": "2021-06-23T17:48:32.734847Z",
"created": "2021-09-16T20:44:35.198887Z",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-[not provided]"
],
"licenseListVersion": "3.13"
"licenseListVersion": "3.14"
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https://anchore.com/syft/image/",
@@ -31,6 +31,9 @@
}
],
"filesAnalyzed": false,
"hasFiles": [
"SPDXRef-File-package-1-04cd22424378dcd6c77fce08beb52493b5494a60ea5e1f9bdf9b16dc0cacffe9"
],
"licenseDeclared": "MIT",
"sourceInfo": "acquired package info from installed python package manifest file: /some/path/pkg1",
"versionInfo": "1.0.1"
@@ -57,5 +60,20 @@
"sourceInfo": "acquired package info from DPKG DB: /some/path/pkg1",
"versionInfo": "2.0.1"
}
],
"files": [
{
"SPDXID": "SPDXRef-File-package-1-04cd22424378dcd6c77fce08beb52493b5494a60ea5e1f9bdf9b16dc0cacffe9",
"name": "foo",
"licenseConcluded": "",
"fileName": "/some/path/pkg1/depedencies/foo"
}
],
"relationships": [
{
"spdxElementId": "SPDXRef-Package-python-package-1-1.0.1",
"relationshipType": "CONTAINS",
"relatedSpdxElement": "SPDXRef-File-package-1-04cd22424378dcd6c77fce08beb52493b5494a60ea5e1f9bdf9b16dc0cacffe9"
}
]
}
@@ -3,12 +3,12 @@
"name": "user-image-input",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"created": "2021-06-23T17:48:32.7379Z",
"created": "2021-09-16T20:44:35.203911Z",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-[not provided]"
],
"licenseListVersion": "3.13"
"licenseListVersion": "3.14"
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https://anchore.com/syft/image/user-image-input",
@@ -1,11 +1,11 @@
[Image]
Layer: 0
Digest: sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59
Digest: sha256:ffb5e9eaa453a002110719d12c294960117ca2903953d1faa40f01dc3f77045c
Size: 22
MediaType: application/vnd.docker.image.rootfs.diff.tar.gzip
Layer: 1
Digest: sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec
Digest: sha256:8463854829fc53d47b9dcdf7ee79fe7eb4ca7933c910f67f8521412f7a2f5c21
Size: 16
MediaType: application/vnd.docker.image.rootfs.diff.tar.gzip
@@ -2,7 +2,6 @@ package packages
import (
"fmt"
"io"
"text/tabwriter"
@@ -32,7 +31,7 @@ func (pres *TextPresenter) Present(output io.Writer) error {
switch pres.srcMetadata.Scheme {
case source.DirectoryScheme:
fmt.Fprintln(w, fmt.Sprintf("[Path: %s]", pres.srcMetadata.Path))
fmt.Fprintf(w, "[Path: %s]\n", pres.srcMetadata.Path)
case source.ImageScheme:
fmt.Fprintln(w, "[Image]")
@@ -51,7 +50,7 @@ func (pres *TextPresenter) Present(output io.Writer) error {
// populate artifacts...
rows := 0
for _, p := range pres.catalog.Sorted() {
fmt.Fprintln(w, fmt.Sprintf("[%s]", p.Name))
fmt.Fprintf(w, "[%s]\n", p.Name)
fmt.Fprintln(w, " Version:\t", p.Version)
fmt.Fprintln(w, " Type:\t", string(p.Type))
fmt.Fprintln(w, " Found by:\t", p.FoundBy)
@@ -158,6 +158,11 @@ func presenterDirectoryInput(t testing.TB) (*pkg.Catalog, source.Metadata, *dist
Metadata: pkg.PythonPackageMetadata{
Name: "package-1",
Version: "1.0.1",
Files: []pkg.PythonFileRecord{
{
Path: "/some/path/pkg1/depedencies/foo",
},
},
},
PURL: "a-purl-2",
CPEs: []pkg.CPE{
@@ -6,7 +6,6 @@ import (
"strconv"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/source"
)
@@ -22,6 +21,7 @@ type JSONFileMetadataEntry struct {
UserID int `json:"userID"`
GroupID int `json:"groupID"`
Digests []file.Digest `json:"digests,omitempty"`
MIMEType string `json:"mimeType"`
}
func NewJSONFileMetadata(data map[source.Location]source.FileMetadata, digests map[source.Location][]file.Digest) ([]JSONFileMetadata, error) {
@@ -46,6 +46,7 @@ func NewJSONFileMetadata(data map[source.Location]source.FileMetadata, digests m
UserID: metadata.UserID,
GroupID: metadata.GroupID,
Digests: digestResults,
MIMEType: metadata.MIMEType,
},
})
}
@@ -16,7 +16,8 @@
"mode": 775,
"type": "directory",
"userID": 0,
"groupID": 0
"groupID": 0,
"mimeType": ""
}
},
{
@@ -33,7 +34,8 @@
"algorithm": "sha256",
"value": "366a3f5653e34673b875891b021647440d0127c2ef041e3b1a22da2a7d4f3703"
}
]
],
"mimeType": ""
}
},
{
@@ -45,7 +47,8 @@
"type": "symbolicLink",
"linkDestination": "/c",
"userID": 0,
"groupID": 0
"groupID": 0,
"mimeType": ""
}
},
{
@@ -62,7 +65,8 @@
"algorithm": "sha256",
"value": "1b3722da2a7d90d033b87581a2a3f12021647445653e34666ef041e3b4f3707c"
}
]
],
"mimeType": ""
}
}
],
@@ -168,6 +172,7 @@
"configuration": {
"configPath": "",
"output": "",
"file": "",
"quiet": false,
"check-for-app-update": false,
"anchore": {
@@ -227,6 +232,7 @@
},
"registry": {
"insecure-skip-tls-verify": false,
"insecure-use-http": false,
"auth": null
}
}
@@ -0,0 +1,189 @@
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
"os"
"regexp"
"sort"
"strings"
"text/template"
"time"
"github.com/scylladb/go-set/strset"
)
// This program generates license_list.go.
const (
source = "license_list.go"
url = "https://spdx.org/licenses/licenses.json"
)
var tmp = template.Must(template.New("").Parse(`// Code generated by go generate; DO NOT EDIT.
// This file was generated by robots at {{ .Timestamp }}
// using data from {{ .URL }}
package spdxlicense
const Version = {{ printf "%q" .Version }}
var licenseIDs = map[string]string{
{{- range $k, $v := .LicenseIDs }}
{{ printf "%q" $k }}: {{ printf "%q" $v }},
{{- end }}
}
`))
var versionMatch = regexp.MustCompile(`-([0-9]+)\.?([0-9]+)?\.?([0-9]+)?\.?`)
type LicenseList struct {
Version string `json:"licenseListVersion"`
Licenses []struct {
ID string `json:"licenseId"`
Name string `json:"name"`
Text string `json:"licenseText"`
Deprecated bool `json:"isDeprecatedLicenseId"`
OSIApproved bool `json:"isOsiApproved"`
SeeAlso []string `json:"seeAlso"`
} `json:"licenses"`
}
func main() {
if err := run(); err != nil {
os.Exit(1)
}
}
func run() error {
resp, err := http.Get(url)
if err != nil {
return fmt.Errorf("unable to get licenses list: %w", err)
}
var result LicenseList
if err = json.NewDecoder(resp.Body).Decode(&result); err != nil {
return fmt.Errorf("unable to decode license list: %w", err)
}
defer func() {
if err := resp.Body.Close(); err != nil {
log.Fatalf("unable to close body: %+v", err)
}
}()
f, err := os.Create(source)
if err != nil {
return fmt.Errorf("unable to create %q: %w", source, err)
}
defer func() {
if err := f.Close(); err != nil {
log.Fatalf("unable to close %q: %+v", source, err)
}
}()
licenseIDs := processSPDXLicense(result)
err = tmp.Execute(f, struct {
Timestamp time.Time
URL string
Version string
LicenseIDs map[string]string
}{
Timestamp: time.Now(),
URL: url,
Version: result.Version,
LicenseIDs: licenseIDs,
})
if err != nil {
return fmt.Errorf("unable to generate template: %w", err)
}
return nil
}
// Parsing the provided SPDX license list necessitates a two pass approach.
// The first pass is only related to what SPDX considers the truth. These K:V pairs will never be overwritten.
// The second pass attempts to generate known short/long version listings for each key.
// For info on some short name conventions see this document:
// https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/#license-short-name.
// The short long listing generation attempts to build all license permutations for a given key.
// The new keys are then also associated with their relative SPDX value. If a key has already been entered
// we know to ignore it since it came from the first pass which is considered the SPDX source of truth.
// We also sort the licenses for the second pass so that cases like `GPL-1` associate to `GPL-1.0` and not `GPL-1.1`.
func processSPDXLicense(result LicenseList) map[string]string {
// first pass build map
licenseIDs := make(map[string]string)
for _, l := range result.Licenses {
cleanID := strings.ToLower(l.ID)
if _, exists := licenseIDs[cleanID]; exists {
log.Fatalf("duplicate license ID found: %q", cleanID)
}
licenseIDs[cleanID] = l.ID
}
sort.Slice(result.Licenses, func(i, j int) bool {
return result.Licenses[i].ID < result.Licenses[j].ID
})
// second pass build exceptions
// do not overwrite if already exists
for _, l := range result.Licenses {
var multipleID []string
cleanID := strings.ToLower(l.ID)
multipleID = append(multipleID, buildLicensePermutations(cleanID)...)
for _, id := range multipleID {
if _, exists := licenseIDs[id]; !exists {
licenseIDs[id] = l.ID
}
}
}
return licenseIDs
}
func buildLicensePermutations(license string) (perms []string) {
lv := findLicenseVersion(license)
vp := versionPermutations(lv)
version := strings.Join(lv, ".")
for _, p := range vp {
perms = append(perms, strings.Replace(license, version, p, 1))
}
return perms
}
func findLicenseVersion(license string) (version []string) {
versionList := versionMatch.FindAllStringSubmatch(license, -1)
if len(versionList) == 0 {
return version
}
for i, v := range versionList[0] {
if v != "" && i != 0 {
version = append(version, v)
}
}
return version
}
func versionPermutations(version []string) []string {
ver := append([]string(nil), version...)
perms := strset.New()
for i := 1; i <= 3; i++ {
if len(ver) < i+1 {
ver = append(ver, "0")
}
perm := strings.Join(ver[:i], ".")
badCount := strings.Count(perm, "0") + strings.Count(perm, ".")
if badCount != len(perm) {
perms.Add(perm)
}
}
return perms.List()
}
@@ -0,0 +1,146 @@
package main
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
func TestLicensePermutations(t *testing.T) {
var tests = []struct {
shortName string
permutations []string
}{
{
"GPL-1-only",
[]string{
"GPL-1-only",
"GPL-1.0-only",
"GPL-1.0.0-only",
},
},
{
"GPL-2",
[]string{
"GPL-2",
"GPL-2.0",
"GPL-2.0.0",
},
},
{
"GPL-2.0+",
[]string{
"GPL-2+",
"GPL-2.0+",
"GPL-2.0.0+",
},
},
{
"GPL-3.0.0-or-later",
[]string{
"GPL-3-or-later",
"GPL-3.0-or-later",
"GPL-3.0.0-or-later",
},
},
{
"oldap-2.0",
[]string{
"oldap-2",
"oldap-2.0",
"oldap-2.0.0",
},
},
}
for _, test := range tests {
t.Run(test.shortName, func(t *testing.T) {
perms := buildLicensePermutations(test.shortName)
assert.ElementsMatch(t, test.permutations, perms)
})
}
}
func TestVersionPermutations(t *testing.T) {
var tests = []struct {
version []string
permutations []string
}{
{
[]string{"1", "0"},
[]string{"1", "1.0", "1.0.0"},
},
{
[]string{"2"},
[]string{"2", "2.0", "2.0.0"},
},
{
[]string{"2", "0"},
[]string{"2", "2.0", "2.0.0"},
},
{
[]string{"3", "0", "0"},
[]string{"3", "3.0", "3.0.0"},
},
{
[]string{"0", "3"},
[]string{"0.3", "0.3.0"},
},
{
[]string{"0", "0", "3"},
[]string{"0.0.3"},
},
}
for _, test := range tests {
t.Run(strings.Join(test.version, "."), func(t *testing.T) {
got := versionPermutations(test.version)
assert.ElementsMatch(t, test.permutations, got)
})
}
}
func TestFindLicenseVersion(t *testing.T) {
var tests = []struct {
license string
version []string
}{
{
"GPL-1.0-only",
[]string{"1", "0"},
},
{
"GPL-2.0",
[]string{"2", "0"},
},
{
"GPL-2.0.0",
[]string{"2", "0", "0"},
},
{
"GPL-2",
[]string{"2"},
},
{
"bzip2-1",
[]string{"1"},
},
{
"php-3.01",
[]string{"3", "01"},
},
{
"oldap-2.0",
[]string{"2", "0"},
},
}
for _, test := range tests {
t.Run(test.license, func(t *testing.T) {
got := findLicenseVersion(test.license)
assert.Equal(t, test.version, got)
})
}
}
@@ -1,92 +0,0 @@
// +build ignore
package main
import (
"encoding/json"
"log"
"net/http"
"os"
"strings"
"text/template"
"time"
)
// This program generates license_list.go.
const (
source = "license_list.go"
url = "https://spdx.org/licenses/licenses.json"
)
var tmp = template.Must(template.New("").Parse(`// Code generated by go generate; DO NOT EDIT.
// This file was generated by robots at {{ .Timestamp }}
// using data from {{ .URL }}
package spdxlicense
const Version = {{ printf "%q" .Version }}
var licenseIDs = map[string]string{
{{- range $k, $v := .LicenseIDs }}
{{ printf "%q" $k }}: {{ printf "%q" $v }},
{{- end }}
}
`))
type LicenseList struct {
Version string `json:"licenseListVersion"`
Licenses []struct {
ID string `json:"licenseId"`
Name string `json:"name"`
Text string `json:"licenseText"`
Deprecated bool `json:"isDeprecatedLicenseId"`
OSIApproved bool `json:"isOsiApproved"`
SeeAlso []string `json:"seeAlso"`
} `json:"licenses"`
}
func main() {
resp, err := http.Get(url)
if err != nil {
log.Fatalf("unable to get licenses list: %+v", err)
}
var result LicenseList
if err = json.NewDecoder(resp.Body).Decode(&result); err != nil {
log.Fatalf("unable to decode license list: %+v", err)
}
f, err := os.Create(source)
if err != nil {
log.Fatalf("unable to create %q: %+v", source, err)
}
defer func() {
if err := f.Close(); err != nil {
log.Fatalf("unable to close %q: %+v", source, err)
}
}()
var licenseIDs = make(map[string]string)
for _, l := range result.Licenses {
cleanID := strings.ToLower(l.ID)
if _, exists := licenseIDs[cleanID]; exists {
log.Fatalf("duplicate license ID found: %q", cleanID)
}
licenseIDs[cleanID] = l.ID
}
err = tmp.Execute(f, struct {
Timestamp time.Time
URL string
Version string
LicenseIDs map[string]string
}{
Timestamp: time.Now(),
URL: url,
Version: result.Version,
LicenseIDs: licenseIDs,
})
if err != nil {
log.Fatalf("unable to generate template: %+v", err)
}
}
+9 -1
View File
@@ -4,7 +4,15 @@ import (
"strings"
)
//go:generate go run generate_license_list.go
// https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/#license-short-name
// License generated in license_list.go uses a regular expression to help resolve cases where
// x.0.0 and x are supplied as version numbers. For SPDX compatibility, versions with trailing
// dot-zeroes are considered to be equivalent to versions without (e.g., “2.0.0” is considered equal to “2.0” and “2”).
// EX: gpl-2+ ---> GPL-2.0+
// EX: gpl-2.0.0-only ---> GPL-2.0-only
// See the debian link for more details on the spdx license differences
//go:generate go run generate/generate_license_list.go
func ID(id string) (string, bool) {
value, exists := licenseIDs[strings.ToLower(id)]
File diff suppressed because it is too large Load Diff
@@ -10,5 +10,6 @@ func TestLicenceListIDs(t *testing.T) {
// do a sanity check on the generated data
assert.Equal(t, "0BSD", licenseIDs["0bsd"])
assert.Equal(t, "ZPL-2.1", licenseIDs["zpl-2.1"])
assert.Equal(t, "GPL-2.0", licenseIDs["gpl-2"])
assert.NotEmpty(t, Version)
}
+60
View File
@@ -0,0 +1,60 @@
package spdxlicense
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestIDParse(t *testing.T) {
var tests = []struct {
shortName string
spdx string
}{
{
"GPL-1-only",
"GPL-1.0-only",
},
{
"GPL-2",
"GPL-2.0",
},
{
"GPL-2+",
"GPL-2.0+",
},
{
"GPL-3.0.0-or-later",
"GPL-3.0-or-later",
},
{
"GPL-3-with-autoconf-exception",
"GPL-3.0-with-autoconf-exception",
},
{
"CC-by-nc-3-de",
"CC-BY-NC-3.0-DE",
},
// the below few cases are NOT expected, however, seem unavoidable given the current approach
{
"w3c-20150513.0.0",
"W3C-20150513",
},
{
"spencer-86.0.0",
"Spencer-86",
},
{
"unicode-dfs-2015.0.0",
"Unicode-DFS-2015",
},
}
for _, test := range tests {
t.Run(test.shortName, func(t *testing.T) {
got, exists := ID(test.shortName)
assert.True(t, exists)
assert.Equal(t, test.spdx, got)
})
}
}
+7
View File
@@ -12,3 +12,10 @@ func HasAnyOfPrefixes(input string, prefixes ...string) bool {
return false
}
func TruncateMiddleEllipsis(input string, maxLen int) string {
if len(input) <= maxLen {
return input
}
return input[:maxLen/2] + "..." + input[len(input)-(maxLen/2):]
}
+41
View File
@@ -1,6 +1,7 @@
package internal
import (
"strconv"
"testing"
"github.com/stretchr/testify/assert"
@@ -63,3 +64,43 @@ func TestHasAnyOfPrefixes(t *testing.T) {
})
}
}
func TestTruncateMiddleEllipsis(t *testing.T) {
tests := []struct {
input string
len int
expected string
}{
{
input: "nobody expects the spanish inquisition",
len: 39,
expected: "nobody expects the spanish inquisition",
},
{
input: "nobody expects the spanish inquisition",
len: 30,
expected: "nobody expects ...ish inquisition",
},
{
input: "nobody expects the spanish inquisition",
len: 38,
expected: "nobody expects the spanish inquisition",
},
{
input: "",
len: 30,
expected: "",
},
{
input: "",
len: 0,
expected: "",
},
}
for _, test := range tests {
t.Run(test.input+":"+strconv.Itoa(test.len), func(t *testing.T) {
assert.Equal(t, test.expected, TruncateMiddleEllipsis(test.input, test.len))
})
}
}
+35 -20
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"context"
"fmt"
"io"
"os"
"sync"
@@ -11,7 +12,6 @@ import (
"github.com/anchore/syft/internal/logger"
syftEvent "github.com/anchore/syft/syft/event"
"github.com/anchore/syft/ui"
"github.com/k0kubun/go-ansi"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/jotframe/pkg/frame"
)
@@ -33,23 +33,28 @@ import (
// or in the shared ui package as a function on the main handler object. All handler functions should be completed
// processing an event before the ETUI exits (coordinated with a sync.WaitGroup)
type ephemeralTerminalUI struct {
unsubscribe func() error
handler *ui.Handler
waitGroup *sync.WaitGroup
frame *frame.Frame
logBuffer *bytes.Buffer
unsubscribe func() error
handler *ui.Handler
waitGroup *sync.WaitGroup
frame *frame.Frame
logBuffer *bytes.Buffer
uiOutput *os.File
reportOutput io.Writer
}
func NewEphemeralTerminalUI() UI {
// NewEphemeralTerminalUI writes all events to a TUI and writes the final report to the given writer.
func NewEphemeralTerminalUI(reportWriter io.Writer) UI {
return &ephemeralTerminalUI{
handler: ui.NewHandler(),
waitGroup: &sync.WaitGroup{},
handler: ui.NewHandler(),
waitGroup: &sync.WaitGroup{},
uiOutput: os.Stderr,
reportOutput: reportWriter,
}
}
func (h *ephemeralTerminalUI) Setup(unsubscribe func() error) error {
h.unsubscribe = unsubscribe
ansi.CursorHide()
hideCursor(h.uiOutput)
// prep the logger to not clobber the screen from now on (logrus only)
h.logBuffer = bytes.NewBufferString("")
@@ -77,9 +82,9 @@ func (h *ephemeralTerminalUI) Handle(event partybus.Event) error {
case event.Type == syftEvent.PresenterReady:
// we need to close the screen now since signaling the the presenter is ready means that we
// are about to write bytes to stdout, so we should reset the terminal state first
h.closeScreen()
h.closeScreen(false)
if err := handleCatalogerPresenterReady(event); err != nil {
if err := handleCatalogerPresenterReady(event, h.reportOutput); err != nil {
log.Errorf("unable to show %s event: %+v", event.Type, err)
}
@@ -93,7 +98,7 @@ func (h *ephemeralTerminalUI) openScreen() error {
config := frame.Config{
PositionPolicy: frame.PolicyFloatForward,
// only report output to stderr, reserve report output for stdout
Output: os.Stderr,
Output: h.uiOutput,
}
fr, err := frame.New(config)
@@ -105,11 +110,13 @@ func (h *ephemeralTerminalUI) openScreen() error {
return nil
}
func (h *ephemeralTerminalUI) closeScreen() {
func (h *ephemeralTerminalUI) closeScreen(force bool) {
// we may have other background processes still displaying progress, wait for them to
// finish before discontinuing dynamic content and showing the final report
if !h.frame.IsClosed() {
h.waitGroup.Wait()
if !force {
h.waitGroup.Wait()
}
h.frame.Close()
// TODO: there is a race condition within frame.Close() that sometimes leads to an extra blank line being output
frame.Close()
@@ -124,14 +131,22 @@ func (h *ephemeralTerminalUI) flushLog() {
logWrapper, ok := log.Log.(*logger.LogrusLogger)
if ok {
fmt.Fprint(logWrapper.Output, h.logBuffer.String())
logWrapper.Logger.SetOutput(os.Stderr)
logWrapper.Logger.SetOutput(h.uiOutput)
} else {
fmt.Fprint(os.Stderr, h.logBuffer.String())
fmt.Fprint(h.uiOutput, h.logBuffer.String())
}
}
func (h *ephemeralTerminalUI) Teardown() error {
h.closeScreen()
ansi.CursorShow()
func (h *ephemeralTerminalUI) Teardown(force bool) error {
h.closeScreen(force)
showCursor(h.uiOutput)
return nil
}
func hideCursor(output io.Writer) {
fmt.Fprint(output, "\x1b[?25l")
}
func showCursor(output io.Writer) {
fmt.Fprint(output, "\x1b[?25h")
}
+4 -6
View File
@@ -4,27 +4,25 @@ import (
"context"
"fmt"
"io"
"os"
"sync"
"github.com/anchore/syft/internal"
"github.com/gookit/color"
"github.com/wagoodman/jotframe/pkg/frame"
syftEventParsers "github.com/anchore/syft/syft/event/parsers"
"github.com/gookit/color"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/jotframe/pkg/frame"
)
// handleCatalogerPresenterReady is a UI function for processing the CatalogerFinished bus event, displaying the catalog
// via the given presenter to stdout.
func handleCatalogerPresenterReady(event partybus.Event) error {
func handleCatalogerPresenterReady(event partybus.Event, reportOutput io.Writer) error {
// show the report to stdout
pres, err := syftEventParsers.ParsePresenterReady(event)
if err != nil {
return fmt.Errorf("bad CatalogerFinished event: %w", err)
}
if err := pres.Present(os.Stdout); err != nil {
if err := pres.Present(reportOutput); err != nil {
return fmt.Errorf("unable to show package catalog report: %w", err)
}
return nil
+11 -5
View File
@@ -1,17 +1,23 @@
package ui
import (
"io"
"github.com/anchore/syft/internal/log"
syftEvent "github.com/anchore/syft/syft/event"
"github.com/wagoodman/go-partybus"
)
type loggerUI struct {
unsubscribe func() error
unsubscribe func() error
reportOutput io.Writer
}
func NewLoggerUI() UI {
return &loggerUI{}
// NewLoggerUI writes all events to the common application logger and writes the final report to the given writer.
func NewLoggerUI(reportWriter io.Writer) UI {
return &loggerUI{
reportOutput: reportWriter,
}
}
func (l *loggerUI) Setup(unsubscribe func() error) error {
@@ -25,7 +31,7 @@ func (l loggerUI) Handle(event partybus.Event) error {
return nil
}
if err := handleCatalogerPresenterReady(event); err != nil {
if err := handleCatalogerPresenterReady(event, l.reportOutput); err != nil {
log.Warnf("unable to show catalog image finished event: %+v", err)
}
@@ -33,6 +39,6 @@ func (l loggerUI) Handle(event partybus.Event) error {
return l.unsubscribe()
}
func (l loggerUI) Teardown() error {
func (l loggerUI) Teardown(_ bool) error {
return nil
}
+9 -7
View File
@@ -1,6 +1,7 @@
package ui
import (
"io"
"os"
"runtime"
@@ -10,20 +11,21 @@ import (
// TODO: build tags to exclude options from windows
// Select is responsible for determining the specific UI function given select user option, the current platform
// config values, and environment status (such as a TTY being present).
func Select(verbose, quiet bool) UI {
var ui UI
// config values, and environment status (such as a TTY being present). The first UI in the returned slice of UIs
// is intended to be used and the UIs that follow are meant to be attempted only in a fallback posture when there
// are environmental problems (e.g. cannot write to the terminal). A writer is provided to capture the output of
// the final SBOM report.
func Select(verbose, quiet bool, reportWriter io.Writer) (uis []UI) {
isStdoutATty := terminal.IsTerminal(int(os.Stdout.Fd()))
isStderrATty := terminal.IsTerminal(int(os.Stderr.Fd()))
notATerminal := !isStderrATty && !isStdoutATty
switch {
case runtime.GOOS == "windows" || verbose || quiet || notATerminal || !isStderrATty:
ui = NewLoggerUI()
uis = append(uis, NewLoggerUI(reportWriter))
default:
ui = NewEphemeralTerminalUI()
uis = append(uis, NewEphemeralTerminalUI(reportWriter), NewLoggerUI(reportWriter))
}
return ui
return uis
}
+1 -1
View File
@@ -7,5 +7,5 @@ import (
type UI interface {
Setup(unsubscribe func() error) error
partybus.Handler
Teardown() error
Teardown(force bool) error
}
+7 -5
View File
@@ -12,11 +12,13 @@ import (
const valueNotProvided = "[not provided]"
// all variables here are provided as build-time arguments, with clear default values
var version = valueNotProvided
var gitCommit = valueNotProvided
var gitTreeState = valueNotProvided
var buildDate = valueNotProvided
var platform = fmt.Sprintf("%s/%s", runtime.GOOS, runtime.GOARCH)
var (
version = valueNotProvided
gitCommit = valueNotProvided
gitTreeState = valueNotProvided
buildDate = valueNotProvided
platform = fmt.Sprintf("%s/%s", runtime.GOOS, runtime.GOARCH)
)
// Version defines the application version details (generally from build information)
type Version struct {
+6 -9
View File
@@ -1,3 +1,4 @@
// nolint:forbidigo
package main
import (
@@ -45,11 +46,7 @@ func build() *jsonschema.Schema {
reflector := &jsonschema.Reflector{
AllowAdditionalProperties: true,
TypeNamer: func(r reflect.Type) string {
name := r.Name()
if strings.HasPrefix(name, "JSON") {
name = strings.TrimPrefix(name, "JSON")
}
return name
return strings.TrimPrefix(r.Name(), "JSON")
},
}
documentSchema := reflector.ReflectFromType(reflect.TypeOf(&poweruser.JSONDocument{}))
@@ -74,7 +71,7 @@ func build() *jsonschema.Schema {
// ensure the generated list of names is stable between runs
sort.Strings(metadataNames)
var metadataTypes = []map[string]string{
metadataTypes := []map[string]string{
// allow for no metadata to be provided
{"type": "null"},
}
@@ -93,13 +90,13 @@ func build() *jsonschema.Schema {
}
func encode(schema *jsonschema.Schema) []byte {
var newSchemaBuffer = new(bytes.Buffer)
newSchemaBuffer := new(bytes.Buffer)
enc := json.NewEncoder(newSchemaBuffer)
// prevent > and < from being escaped in the payload
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
err := enc.Encode(&schema)
if err != nil {
if err := enc.Encode(&schema); err != nil {
panic(err)
}
+3 -1
View File
@@ -19,10 +19,12 @@ func NewDistro(t Type, ver, like string) (Distro, error) {
if ver == "" {
return Distro{Type: t}, nil
}
verObj, err := hashiVer.NewVersion(ver)
if err != nil {
return Distro{}, fmt.Errorf("could not create distro version: %w", err)
}
return Distro{
Type: t,
Version: verObj,
@@ -39,7 +41,7 @@ func (d Distro) Name() string {
// MajorVersion returns the major version value from the pseudo-semantically versioned distro version value.
func (d Distro) MajorVersion() string {
if d.Version == nil {
return fmt.Sprint("(version unknown)")
return "(version unknown)"
}
return fmt.Sprintf("%d", d.Version.Segments()[0])
}
+72 -11
View File
@@ -6,7 +6,6 @@ import (
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/source"
)
@@ -35,6 +34,16 @@ var identityFiles = []parseEntry{
path: "/bin/busybox",
fn: parseBusyBox,
},
{
// check for centos:6
path: "/etc/system-release-cpe",
fn: parseSystemReleaseCPE,
},
{
// last ditch effort for determining older centos version distro information
path: "/etc/redhat-release",
fn: parseRedhatRelease,
},
}
// Identify parses distro-specific files to determine distro metadata like version and release.
@@ -87,23 +96,30 @@ identifyLoop:
return distro
}
func assemble(name, version, like string) *Distro {
distroType, ok := IDMapping[name]
func assemble(id, version, like string) *Distro {
distroType, ok := IDMapping[id]
// Both distro and version must be present
if len(name) == 0 {
if len(id) == 0 && len(version) == 0 {
return nil
}
if ok {
distro, err := NewDistro(distroType, version, like)
if err != nil {
return nil
}
return &distro
// If it's an unknown distro, try mapping the ID_LIKE
if !ok && len(like) != 0 {
distroType, ok = IDMapping[like]
}
return nil
// If we still can't match allow name to be used in constructor
if !ok {
distroType = Type(id)
}
distro, err := NewDistro(distroType, version, like)
if err != nil {
return nil
}
return &distro
}
func parseOsRelease(contents string) *Distro {
@@ -140,3 +156,48 @@ func parseBusyBox(contents string) *Distro {
}
return nil
}
// TODO: we should update parseSystemReleaseCPE to use the CPE struct, pkg.CPE, which requires a refactor to avoid a circular import:
// TODO: pkg depends on distro to support pURLs. To avoid the circular import, either try to make pkg to not depend on distro (medium lift-ish)
// TODO: or migrate the cpe code out of the pkg package (small lift).
// example CPE: cpe:/o:centos:linux:6:GA
var systemReleaseCpeMatcher = regexp.MustCompile(`cpe:\/o:(.*?):.*?:(.*?):.*?$`)
// parseSystemReleaseCPE parses the older centos (6) file to determine distro metadata
func parseSystemReleaseCPE(contents string) *Distro {
matches := systemReleaseCpeMatcher.FindAllStringSubmatch(contents, -1)
for _, match := range matches {
if len(match) < 3 {
log.Warnf("system release cpe does not match expected format")
return nil
}
// note: in SubMatches (capture groups), the 0th index is the full match string
// see https://pkg.go.dev/regexp#pkg-overview for more info
distro := assemble(match[1], match[2], "")
if distro != nil {
return distro
}
}
return nil
}
// example: "CentOS release 6.10 (Final)"
var redhatReleaseMatcher = regexp.MustCompile(`(.*?)\srelease\s(\d\.\d+)`)
// parseRedhatRelease is a fallback parsing method for determining distro information in older redhat versions
func parseRedhatRelease(contents string) *Distro {
matches := redhatReleaseMatcher.FindAllStringSubmatch(contents, -1)
for _, match := range matches {
if len(match) < 3 {
log.Warnf("failed to parse redhat-release file, unexpected format")
return nil
}
// note: in SubMatches (capture groups), the 0th index is the full match string
// see https://pkg.go.dev/regexp#pkg-overview for more info
distro := assemble(strings.ToLower(match[1]), match[2], "")
if distro != nil {
return distro
}
}
return nil
}
+140 -28
View File
@@ -6,12 +6,15 @@ import (
"os"
"testing"
"github.com/anchore/syft/internal"
"github.com/stretchr/testify/assert"
hashiVer "github.com/hashicorp/go-version"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/source"
"github.com/stretchr/testify/assert"
)
const CustomDistro Type = "scientific"
func TestIdentifyDistro(t *testing.T) {
tests := []struct {
fixture string
@@ -68,14 +71,20 @@ func TestIdentifyDistro(t *testing.T) {
Type: UnknownDistroType,
},
{
fixture: "test-fixtures/os/unmatchable",
Type: UnknownDistroType,
fixture: "test-fixtures/os/custom",
Type: CustomDistro,
Version: "8.0.0",
},
{
fixture: "test-fixtures/os/opensuse-leap",
Type: OpenSuseLeap,
Version: "15.2.0",
},
{
fixture: "test-fixtures/os/sles",
Type: SLES,
Version: "15.2.0",
},
{
fixture: "test-fixtures/os/photon",
Type: Photon,
@@ -85,6 +94,35 @@ func TestIdentifyDistro(t *testing.T) {
fixture: "test-fixtures/os/arch",
Type: ArchLinux,
},
{
fixture: "test-fixtures/partial-fields/missing-id",
Type: Debian,
Version: "8.0.0",
},
{
fixture: "test-fixtures/partial-fields/unknown-id",
Type: Debian,
Version: "8.0.0",
},
{
fixture: "test-fixtures/partial-fields/missing-version",
Type: UnknownDistroType,
},
{
fixture: "test-fixtures/os/centos6",
Type: CentOS,
Version: "6.0.0",
},
{
fixture: "test-fixtures/os/centos5",
Type: CentOS,
Version: "5.7.0",
},
{
fixture: "test-fixtures/os/mariner",
Type: Mariner,
Version: "1.0.0",
},
}
observedDistros := internal.NewStringSet()
@@ -187,18 +225,7 @@ func TestParseOsRelease(t *testing.T) {
for _, test := range tests {
name := fmt.Sprintf("%s:%s", test.name, test.RawVersion)
fixture, err := os.Open(test.fixture)
if err != nil {
t.Fatalf("could not open test fixture=%s: %+v", test.fixture, err)
}
defer fixture.Close()
b, err := ioutil.ReadAll(fixture)
if err != nil {
t.Fatalf("unable to read fixture file: %+v", err)
}
contents := string(b)
contents := retrieveFixtureContentsAsString(test.fixture, t)
t.Run(name, func(t *testing.T) {
distro := parseOsRelease(contents)
@@ -227,18 +254,7 @@ func TestParseOsReleaseFailures(t *testing.T) {
for _, test := range tests {
name := fmt.Sprintf("%s:%s", test.name, test.fixture)
fixture, err := os.Open(test.fixture)
if err != nil {
t.Fatalf("could not open test fixture=%s: %+v", test.fixture, err)
}
defer fixture.Close()
b, err := ioutil.ReadAll(fixture)
if err != nil {
t.Fatalf("unable to read fixture file: %+v", err)
}
contents := string(b)
contents := retrieveFixtureContentsAsString(test.fixture, t)
t.Run(name, func(t *testing.T) {
distro := parseOsRelease(contents)
@@ -247,5 +263,101 @@ func TestParseOsReleaseFailures(t *testing.T) {
}
})
}
}
func TestParseSystemReleaseCPE(t *testing.T) {
centos6Version, _ := hashiVer.NewVersion("6")
tests := []struct {
fixture string
name string
expected *Distro
}{
{
fixture: "test-fixtures/os/centos6/etc/system-release-cpe",
name: "Centos 6",
expected: &Distro{
Type: CentOS,
Version: centos6Version,
RawVersion: "6",
},
},
{
fixture: "test-fixtures/bad-system-release-cpe",
name: "Centos 6 Bad CPE",
expected: nil,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
contents := retrieveFixtureContentsAsString(test.fixture, t)
actual := parseSystemReleaseCPE(contents)
if test.expected == nil {
assert.Nil(t, actual)
return
}
// not comparing the full distro object because the hashiVer is a pointer
assert.Equal(t, test.expected.Type, actual.Type)
assert.Equal(t, &test.expected.Version, &actual.Version)
assert.Equal(t, test.expected.RawVersion, actual.RawVersion)
})
}
}
func TestParseRedhatRelease(t *testing.T) {
centos5Version, _ := hashiVer.NewVersion("5.7")
tests := []struct {
fixture string
name string
expected *Distro
}{
{
fixture: "test-fixtures/os/centos5/etc/redhat-release",
name: "Centos 5",
expected: &Distro{
Type: CentOS,
Version: centos5Version,
RawVersion: "5.7",
},
},
{
fixture: "test-fixtures/bad-redhat-release",
name: "Centos 5 Bad Redhat Release",
expected: nil,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
contents := retrieveFixtureContentsAsString(test.fixture, t)
actual := parseRedhatRelease(contents)
if test.expected == nil {
assert.Nil(t, actual)
return
}
// not comparing the full distro object because the hashiVer is a pointer
assert.Equal(t, test.expected.Type, actual.Type)
assert.Equal(t, &test.expected.Version, &actual.Version)
assert.Equal(t, test.expected.RawVersion, actual.RawVersion)
})
}
}
func retrieveFixtureContentsAsString(fixturePath string, t *testing.T) string {
fixture, err := os.Open(fixturePath)
if err != nil {
t.Fatalf("could not open test fixture=%s: %+v", fixturePath, err)
}
defer fixture.Close()
b, err := ioutil.ReadAll(fixture)
if err != nil {
t.Fatalf("unable to read fixture file: %+v", err)
}
return string(b)
}
@@ -0,0 +1 @@
CentOS release 5 (Final)
@@ -0,0 +1 @@
cpe:/o:centos:6:GA
@@ -0,0 +1 @@
CentOS release 5.7 (Final)
@@ -0,0 +1 @@
cpe:/o:centos:linux:6:GA
@@ -0,0 +1,9 @@
NAME="Common Base Linux Mariner"
VERSION="1.0.20210901"
ID=mariner
VERSION_ID=1.0
PRETTY_NAME="CBL-Mariner/Linux"
ANSI_COLOR="1;34"
HOME_URL="https://aka.ms/cbl-mariner"
BUG_REPORT_URL="https://aka.ms/cbl-mariner"
SUPPORT_URL="https://aka.ms/cbl-mariner"
@@ -0,0 +1,9 @@
NAME="SLES"
VERSION="15-SP2"
VERSION_ID="15.2"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP2"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp2"
DOCUMENTATION_URL="https://documentation.suse.com/"
@@ -0,0 +1,3 @@
NAME="Debian GNU/Linux"
VERSION_ID="8"
ID_LIKE=debian
@@ -0,0 +1,2 @@
NAME="Debian GNU/Linux"
ID_LIKE=debian
@@ -0,0 +1,4 @@
NAME="Debian GNU/Linux"
VERSION_ID="8"
ID=my-awesome-distro
ID_LIKE=debian
+6
View File
@@ -17,8 +17,10 @@ const (
OracleLinux Type = "oraclelinux"
ArchLinux Type = "archlinux"
OpenSuseLeap Type = "opensuseleap"
SLES Type = "sles"
Photon Type = "photon"
Windows Type = "windows"
Mariner Type = "mariner"
)
// All contains all Linux distribution options
@@ -34,8 +36,10 @@ var All = []Type{
OracleLinux,
ArchLinux,
OpenSuseLeap,
SLES,
Photon,
Windows,
Mariner,
}
// IDMapping connects a distro ID like "ubuntu" to a Distro type
@@ -51,8 +55,10 @@ var IDMapping = map[string]Type{
"ol": OracleLinux,
"arch": ArchLinux,
"opensuse-leap": OpenSuseLeap,
"sles": SLES,
"photon": Photon,
"windows": Windows,
"mariner": Mariner,
}
// String returns the string representation of the given Linux distribution.
+3
View File
@@ -23,6 +23,9 @@ const (
// FileDigestsCatalogerStarted is a partybus event that occurs when the file digests cataloging has begun
FileDigestsCatalogerStarted partybus.EventType = "syft-file-digests-cataloger-started-event"
// FileIndexingStarted is a partybus event that occurs when the directory resolver begins indexing a filesystem
FileIndexingStarted partybus.EventType = "syft-file-indexing-started-event"
// PresenterReady is a partybus event that occurs when an analysis result is ready for final presentation
PresenterReady partybus.EventType = "syft-presenter-ready-event"
+24 -9
View File
@@ -6,29 +6,26 @@ package parsers
import (
"fmt"
"github.com/anchore/syft/syft/presenter"
"github.com/anchore/syft/syft/file"
"github.com/wagoodman/go-progress"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg/cataloger"
"github.com/anchore/syft/syft/presenter"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
)
type ErrBadPayload struct {
type badPayloadError struct {
Type partybus.EventType
Field string
Value interface{}
}
func (e *ErrBadPayload) Error() string {
func (e *badPayloadError) Error() string {
return fmt.Sprintf("event='%s' has bad event payload field='%v': '%+v'", string(e.Type), e.Field, e.Value)
}
func newPayloadErr(t partybus.EventType, field string, value interface{}) error {
return &ErrBadPayload{
return &badPayloadError{
Type: t,
Field: field,
Value: value,
@@ -94,6 +91,24 @@ func ParseFileDigestsCatalogingStarted(e partybus.Event) (progress.StagedProgres
return prog, nil
}
func ParseFileIndexingStarted(e partybus.Event) (string, progress.StagedProgressable, error) {
if err := checkEventType(e.Type, event.FileIndexingStarted); err != nil {
return "", nil, err
}
path, ok := e.Source.(string)
if !ok {
return "", nil, newPayloadErr(e.Type, "Source", e.Source)
}
prog, ok := e.Value.(progress.StagedProgressable)
if !ok {
return "", nil, newPayloadErr(e.Type, "Value", e.Value)
}
return path, prog, nil
}
func ParsePresenterReady(e partybus.Event) (presenter.Presenter, error) {
if err := checkEventType(e.Type, event.PresenterReady); err != nil {
return nil, err
+1 -1
View File
@@ -31,7 +31,7 @@ func (i *ClassificationCataloger) Catalog(resolver source.FileResolver) (map[sou
}
}
}
log.Debugf("classification cataloger discovered %d results", numResults)
log.Debugf("file classifier discovered %d results", numResults)
return results, nil
}
+9 -2
View File
@@ -118,11 +118,18 @@ func TestClassifierCataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
loc := source.NewLocation(test.location)
if _, ok := actualResults[loc]; !ok {
ok := false
for actual_loc, actual_classification := range actualResults {
if loc.RealPath == actual_loc.RealPath {
ok = true
assert.Equal(t, test.expected, actual_classification)
}
}
if !ok {
t.Fatalf("could not find test location=%q", test.location)
}
assert.Equal(t, test.expected, actualResults[loc])
})
}
}
+5 -4
View File
@@ -3,11 +3,9 @@ package file
import (
"bytes"
"encoding/base64"
"fmt"
"io"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/source"
)
@@ -32,7 +30,6 @@ func (i *ContentsCataloger) Catalog(resolver source.FileResolver) (map[source.Lo
if err != nil {
return nil, err
}
for _, location := range locations {
metadata, err := resolver.FileMetadataByLocation(location)
if err != nil {
@@ -44,6 +41,10 @@ func (i *ContentsCataloger) Catalog(resolver source.FileResolver) (map[source.Lo
}
result, err := i.catalogLocation(resolver, location)
if internal.IsErrPathPermission(err) {
log.Debugf("file contents cataloger skipping - %+v", err)
continue
}
if err != nil {
return nil, err
}
@@ -63,7 +64,7 @@ func (i *ContentsCataloger) catalogLocation(resolver source.FileResolver, locati
buf := &bytes.Buffer{}
if _, err = io.Copy(base64.NewEncoder(base64.StdEncoding, buf), contentReader); err != nil {
return "", fmt.Errorf("unable to observe contents of %+v: %w", location.RealPath, err)
return "", internal.PathError{Path: location.RealPath, Err: err}
}
return buf.String(), nil
+9 -7
View File
@@ -8,15 +8,12 @@ import (
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/source"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
"github.com/anchore/syft/syft/source"
)
type DigestsCataloger struct {
@@ -39,6 +36,11 @@ func (i *DigestsCataloger) Catalog(resolver source.FileResolver) (map[source.Loc
for _, location := range locations {
stage.Current = location.RealPath
result, err := i.catalogLocation(resolver, location)
if internal.IsErrPathPermission(err) {
log.Debugf("file digests cataloger skipping - %+v", err)
continue
}
if err != nil {
return nil, err
}
@@ -67,7 +69,7 @@ func (i *DigestsCataloger) catalogLocation(resolver source.FileResolver, locatio
size, err := io.Copy(io.MultiWriter(writers...), contentReader)
if err != nil {
return nil, fmt.Errorf("unable to observe contents of %+v: %+v", location.RealPath, err)
return nil, internal.PathError{Path: location.RealPath, Err: err}
}
if size == 0 {
@@ -94,7 +96,7 @@ func DigestAlgorithmName(hash crypto.Hash) string {
func CleanDigestAlgorithmName(name string) string {
lower := strings.ToLower(name)
return strings.Replace(lower, "-", "", -1)
return strings.ReplaceAll(lower, "-", "")
}
func digestsCatalogingProgress(locations int64) (*progress.Stage, *progress.Manual) {
+1 -1
View File
@@ -11,7 +11,7 @@ import (
// GenerateSearchPatterns takes a set of named base patterns, a set of additional named patterns and an name exclusion list and generates a final
// set of regular expressions (indexed by name). The sets are aggregated roughly as such: (base - excluded) + additional.
func GenerateSearchPatterns(basePatterns map[string]string, additionalPatterns map[string]string, excludePatternNames []string) (map[string]*regexp.Regexp, error) {
var regexObjs = make(map[string]*regexp.Regexp)
regexObjs := make(map[string]*regexp.Regexp)
var errs error
addFn := func(name, pattern string) {
+1 -2
View File
@@ -9,8 +9,7 @@ import (
"github.com/wagoodman/go-progress"
)
type MetadataCataloger struct {
}
type MetadataCataloger struct{}
func NewMetadataCataloger() *MetadataCataloger {
return &MetadataCataloger{}
+28 -21
View File
@@ -49,11 +49,12 @@ func TestFileMetadataCataloger(t *testing.T) {
path: "/file-1.txt",
exists: true,
expected: source.FileMetadata{
Mode: 0644,
Type: "RegularFile",
UserID: 1,
GroupID: 2,
Size: 7,
Mode: 0644,
Type: "RegularFile",
UserID: 1,
GroupID: 2,
Size: 7,
MIMEType: "text/plain",
},
},
{
@@ -65,6 +66,7 @@ func TestFileMetadataCataloger(t *testing.T) {
LinkDestination: "file-1.txt",
UserID: 1,
GroupID: 2,
MIMEType: "",
},
},
{
@@ -76,46 +78,51 @@ func TestFileMetadataCataloger(t *testing.T) {
LinkDestination: "file-1.txt",
UserID: 0,
GroupID: 0,
MIMEType: "",
},
},
{
path: "/char-device-1",
exists: true,
expected: source.FileMetadata{
Mode: 0644 | os.ModeDevice | os.ModeCharDevice,
Type: "CharacterDevice",
UserID: 0,
GroupID: 0,
Mode: 0644 | os.ModeDevice | os.ModeCharDevice,
Type: "CharacterDevice",
UserID: 0,
GroupID: 0,
MIMEType: "",
},
},
{
path: "/block-device-1",
exists: true,
expected: source.FileMetadata{
Mode: 0644 | os.ModeDevice,
Type: "BlockDevice",
UserID: 0,
GroupID: 0,
Mode: 0644 | os.ModeDevice,
Type: "BlockDevice",
UserID: 0,
GroupID: 0,
MIMEType: "",
},
},
{
path: "/fifo-1",
exists: true,
expected: source.FileMetadata{
Mode: 0644 | os.ModeNamedPipe,
Type: "FIFONode",
UserID: 0,
GroupID: 0,
Mode: 0644 | os.ModeNamedPipe,
Type: "FIFONode",
UserID: 0,
GroupID: 0,
MIMEType: "",
},
},
{
path: "/bin",
exists: true,
expected: source.FileMetadata{
Mode: 0755 | os.ModeDir,
Type: "Directory",
UserID: 0,
GroupID: 0,
Mode: 0755 | os.ModeDir,
Type: "Directory",
UserID: 0,
GroupID: 0,
MIMEType: "",
},
},
}
+6 -2
View File
@@ -9,7 +9,6 @@ import (
"sort"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/event"
@@ -50,6 +49,11 @@ func (i *SecretsCataloger) Catalog(resolver source.FileResolver) (map[source.Loc
for _, location := range locations {
stage.Current = location.RealPath
result, err := i.catalogLocation(resolver, location)
if internal.IsErrPathPermission(err) {
log.Debugf("secrets cataloger skipping - %+v", err)
continue
}
if err != nil {
return nil, err
}
@@ -77,7 +81,7 @@ func (i *SecretsCataloger) catalogLocation(resolver source.FileResolver, locatio
// TODO: in the future we can swap out search strategies here
secrets, err := catalogLocationByLine(resolver, location, i.patterns)
if err != nil {
return nil, err
return nil, internal.PathError{Path: location.RealPath, Err: err}
}
if i.revealValues {
+4 -5
View File
@@ -9,7 +9,6 @@ import (
"regexp"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/source"
)
@@ -20,7 +19,7 @@ func catalogLocationByLine(resolver source.FileResolver, location source.Locatio
}
defer internal.CloseAndLogError(readCloser, location.VirtualPath)
var scanner = bufio.NewReader(readCloser)
scanner := bufio.NewReader(readCloser)
var position int64
var allSecrets []SearchResult
var lineNo int64
@@ -30,7 +29,7 @@ func catalogLocationByLine(resolver source.FileResolver, location source.Locatio
var line []byte
// TODO: we're at risk of large memory usage for very long lines
line, readErr = scanner.ReadBytes('\n')
if readErr != nil && readErr != io.EOF {
if readErr != nil && !errors.Is(readErr, io.EOF) {
return nil, readErr
}
@@ -119,9 +118,9 @@ func extractSecretFromPosition(readCloser io.ReadCloser, name string, pattern *r
}
// lineNoOfSecret are the number of lines which occur before the start of the secret value
var lineNoOfSecret = lineNo + int64(reader.newlinesBefore(start))
lineNoOfSecret := lineNo + int64(reader.newlinesBefore(start))
// lineOffsetOfSecret are the number of bytes that occur after the last newline but before the secret value.
var lineOffsetOfSecret = start - reader.newlinePositionBefore(start)
lineOffsetOfSecret := start - reader.newlinePositionBefore(start)
if lineNoOfSecret == lineNo {
// the secret value starts in the same line as the overall match, so we must consider that line offset
lineOffsetOfSecret += lineOffset
+1 -1
View File
@@ -32,7 +32,7 @@ import (
// CatalogPackages takes an inventory of packages from the given image from a particular perspective
// (e.g. squashed source, all-layers source). Returns the discovered set of packages, the identified Linux
// distribution, and the source object used to wrap the data source.
func CatalogPackages(src source.Source, scope source.Scope) (*pkg.Catalog, *distro.Distro, error) {
func CatalogPackages(src *source.Source, scope source.Scope) (*pkg.Catalog, *distro.Distro, error) {
resolver, err := src.FileResolver(scope)
if err != nil {
return nil, nil, fmt.Errorf("unable to determine resolver while cataloging packages: %w", err)
+9 -10
View File
@@ -3,15 +3,14 @@ package pkg
import (
"sort"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/file"
"github.com/package-url/packageurl-go"
"github.com/scylladb/go-set/strset"
)
const ApkDbGlob = "**/lib/apk/db/installed"
const ApkDBGlob = "**/lib/apk/db/installed"
var _ fileOwner = (*ApkMetadata)(nil)
var _ FileOwner = (*ApkMetadata)(nil)
// ApkMetadata represents all captured data for a Alpine DB package entry.
// See the following sources for more information:
@@ -37,11 +36,11 @@ type ApkMetadata struct {
// ApkFileRecord represents a single file listing and metadata from a APK DB entry (which may have many of these file records).
type ApkFileRecord struct {
Path string `json:"path"`
OwnerUID string `json:"ownerUid,omitempty"`
OwnerGID string `json:"ownerGid,omitempty"`
Permissions string `json:"permissions,omitempty"`
Digest file.Digest `json:"digest,omitempty"`
Path string `json:"path"`
OwnerUID string `json:"ownerUid,omitempty"`
OwnerGID string `json:"ownerGid,omitempty"`
Permissions string `json:"permissions,omitempty"`
Digest *file.Digest `json:"digest,omitempty"`
}
// PackageURL returns the PURL for the specific Alpine package (see https://github.com/package-url/purl-spec)
@@ -63,7 +62,7 @@ func (m ApkMetadata) PackageURL() string {
return pURL.ToString()
}
func (m ApkMetadata) ownedFiles() (result []string) {
func (m ApkMetadata) OwnedFiles() (result []string) {
s := strset.New()
for _, f := range m.Files {
if f.Path != "" {

Some files were not shown because too many files have changed in this diff Show More