Compare commits

..
56 Commits
Author SHA1 Message Date
Dan Luhring 37927b8b23 reduce logging severity for non-Go binaries (#983) 2022-05-03 09:38:14 -04:00
Christopher Angelo Phillips 03d51c36d0 golang.org/x/crypto upgrade (#979) 2022-05-02 21:33:40 +00:00
Dan Luhring 0bd3558fb2 reduce noise of log output (#976) 2022-05-02 14:54:30 +00:00
Christopher Angelo Phillips 4ce2edda9e add version info and remove double config call (#977) 2022-05-02 14:54:10 +00:00
Sambhav KothariandChristopher Phillips 36973021fa Rename syft-id to package-id (#970)
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-04-29 11:18:45 -04:00
Christian Köberl 7d8ea39ee5 update to cyclonedx-go 0.5.2 (#971) 2022-04-28 10:42:12 -04:00
Christopher Angelo Phillips 6029dd7c2e refactor command package to remove globals and add dependency injection 2022-04-26 18:23:03 +00:00
Jon McEwen 7304bbf8ee fix: #953 Derive language from pURL - https://github.com/anchore/syft… (#957)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-04-26 11:51:24 -04:00
Rob Dimsdale-Zucker c270ee2a02 Fix typo in CPE-parsing error (#966) 2022-04-22 12:08:44 -04:00
Alex Goodman 172ecc0d77 Preserve syft IDs on SBOM decode (#963) 2022-04-18 18:10:55 +00:00
Keith Zantow 248023baaf Update GitHub format package_url and correlator (#961) 2022-04-15 13:00:06 -04:00
Keith Zantow b7295b79de Ensure SPDXIDs are valid (#955) 2022-04-14 15:07:23 -04:00
Keith Zantow 321eddf874 Auto-PR needs to run go mod tidy (#958) 2022-04-13 16:30:35 -04:00
Keith Zantow 25bf679f8f Add workflow for automatic PR for new stereoscope updates (#954) 2022-04-13 13:20:40 -04:00
Keith Zantow 02a8fb6f8c Minor readme update to correct format information (#948) 2022-04-12 17:16:47 -04:00
Christopher Angelo Phillips b46d044d7e Update spdx22json to only take uppercase checksum algorithm (#946) 2022-04-11 14:56:04 -04:00
Weston Steimel 15e45a8ce1 add additional vendors for springframework (#945)
The Official CPE dictionary currently contains entries for springframework with three different vendors: springsource, vmware, and pivotal_software.  This appears to be because ownership has changed over time.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2022-04-11 14:38:52 +01:00
Christopher Angelo Phillips 782b2e3348 Add digest property to parent and nested java package metadata (#941) 2022-04-08 15:12:32 -04:00
Alex Goodman e415bb21e7 Update write permissions and log into ghcr.io for release (#942) 2022-04-06 21:15:55 +00:00
Alex Goodman 748cfbf006 Retry auth URL lookup without docker credentialhelper workaround (#939) 2022-04-06 16:27:13 +00:00
Sambhav KothariandAlex Goodman 8bc5d84481 Ensure that all cyclonedx components have bom-refs (#914)
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-04-01 12:19:30 -04:00
Alex Goodman 68b7ad9770 Additionally publish docker images to GHCR (#934) 2022-04-01 11:30:21 -04:00
Oscar Hallgren 1aeda6bb50 use filepath.Base() instead of path.Base() for temp files (#882) 2022-04-01 10:42:22 -04:00
Alex Goodman f24bbc1838 Deduplicate packages across multiple container image layers (#930) 2022-03-31 15:45:51 -04:00
Eric LarssenandAlex Goodman cb3e73e308 Add dart support (#919)
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-03-31 15:44:55 -04:00
Alex Goodman f157d7a862 Pull from DockerHub fails for public images when using SSO (#928) 2022-03-30 17:32:49 +00:00
Dan Luhring 028cd9e27e Fix nil pointer dereference in directory resolver's indexPath method (#924)
* Add failing test for dir resolver panic

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Fix panic

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-28 13:15:09 -04:00
Dan Luhring 5549939cc6 Fixups and clarifications in README (#920)
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-25 11:36:43 -04:00
briankoe741 47ea910868 Remove announcement for OSS Meetup (#915)
Proposing changes to remove our 3/23 meetup

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-25 00:17:14 +00:00
Dan Luhring a7db43f5ec Fix panic on empty sbom (#917)
* Implement fmt.Stringer with format.ID

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Add failing test for formats processing empty SBOMs

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Account for nil SPDX document during Syft model conversion

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-24 10:11:51 -04:00
Alex Goodman cc2c0e57a0 bump strset version to fix 386 builds (#911) 2022-03-23 14:34:54 -04:00
Alex Goodman 5253da4b36 Rollback referencing docker config items (#912) 2022-03-23 18:33:41 +00:00
Jonas XavierandAlex Goodman c0b547bdb2 Less verbose logging in Golang Cataloger (#904)
* Less verbose logging in Golang Cataloger

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* debug for known gray errors

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* only show warnings when a binary is not a go executable

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-03-22 10:19:18 -07:00
Alex Goodman cffcaf5984 Improve docker config support (#906) 2022-03-22 11:02:54 -04:00
Alex Goodman 7f9edf346a Bump golangci-lint to 1.45.0 (#909) 2022-03-22 11:02:36 -04:00
j-k a644a45ef4 Correct go.mod to enforce go 1.18 (#897)
Since syft now depends on debug/buildinfo go 1.18 is required to build
syft and as such go.mod needs updating

Signed-off-by: 06kellyjac <jack@control-plane.io>
2022-03-21 15:38:32 -04:00
Jonas Xavier 283db88dc4 Omit H1Digest when empty (#902)
* Omit HD1Field when empty

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update test-fixtures

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2022-03-21 11:59:10 -07:00
Alex Goodman 03e193e577 Add platform option to the README (#889) 2022-03-21 12:02:15 -04:00
Alex Goodman 069aa68b63 Fix image cleanup when there is an error (#905) 2022-03-21 14:48:11 +00:00
Keith Zantow 9240860f44 Correct ID handling during Syft JSON decoding (#900) 2022-03-18 17:03:26 -04:00
Christopher Angelo Phillips 4231f38fa2 add case to decode GolangBinMetadata for syftjson model (#901)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-03-18 15:15:10 -04:00
Dan Luhring 752b03b2d6 Remove commit signing requirement (#899)
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-18 12:48:00 -04:00
Keith Zantow 99c3339810 Fix CycloneDX license decoding panic (#898) 2022-03-18 09:44:51 -04:00
Keith Zantow f4734d28b3 Fix panic when CycloneDX BOM missing metadata.component (#895) 2022-03-17 10:22:35 -04:00
Jonas Xavier 6ef3e45ffc Use go 1.18 buildinfo to catalog binaries (#827)
* initial working version

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* added build settings to pkg metadata

wip - unit tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* handle mach-O FatFiles

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add support to mod replace

fixed golang catalger tests

trying GH Actions with go 1.18rc1

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* log error

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* use go-macholibre for extraction

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* cleaner tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add version to main module

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* check macho file with macholibre

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* run golangci in its own workflow

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci workflow

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix golangci wf yml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix golangci wf yml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci wf

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci wf

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* get arch from bin file headers

upgrade macholibre

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* test new stereoscope lazy reader interface

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove devel version from golang cataloger

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* switch github workflows to go1.18 stable

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add union reader interface in golang cataloger

update stereoscope

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* simpler golangci validation

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix makefile

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* get archs refactor

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* nolint for golang version

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix go bin tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* feedback changes

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* golangci nolint needs a \n before package

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* cleanup

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* move golangci-lint to its own jobs again

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix ci yaml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add support for xcoff files

add arch assets to test bin file types

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* clean up golangci-lint config

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* nolint for xcoff

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explain nolints

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove unused xcoff testdata assets

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* make go bin test-fixtures in docker

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix make clean with -f

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update json output schema

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update schema version in test fixture

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* feedback changes

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explain possible empty main module

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2022-03-16 17:07:02 -07:00
Keith Zantow ee0a1d172c panic parsing cyclonedx (#892) 2022-03-16 09:10:44 -04:00
Keith Zantow b9b3ccecf9 Update register link text (#891) 2022-03-15 17:02:33 +00:00
mikey strauss 95271fb10d NPM PURLs are invalid (#832)
Signed-off-by: houdini91 <mdstrauss91@gmail.com>
2022-03-15 11:54:33 -04:00
Christopher Angelo Phillips 93d2d57cd3 update README with OSS Meetup information (#890)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-03-15 11:37:56 -04:00
Kenny Moens 7cd3201fe9 Support the .be top-level domain (#873)
Signed-off-by: Kenny Moens <kenny.moens@cipalschaubroeck.be>
2022-03-15 10:59:13 -04:00
Keith Zantow c7cf8b0b26 Brew install gon failed (#885) 2022-03-11 13:17:57 -05:00
Frankie G-J 44a6e00f7a Include vendored modules in Go Module package list (#883)
* include vendored modules in package slice

Signed-off-by: Frankie Gallina-Jones <frankieg@vmware.com>

* add explanatory comments

Signed-off-by: Frankie Gallina-Jones <frankieg@vmware.com>
2022-03-11 12:57:33 -05:00
Keith Zantow 6c8102bf28 Correct CycloneDX distro decoding (#745) 2022-03-11 09:27:18 -05:00
Keith Zantow 7789506dc6 Experimental GitHub export (#836) 2022-03-10 22:38:12 -05:00
Christopher Angelo Phillips fa03723617 Upgrade vault api from v1.3.1 to v1.4.1 (#878)
* move v1.3.1 => v1.4.1
* go mod tidy

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-03-09 15:11:07 -05:00
Alex Goodman 2946813a74 RPM Epoch should be optional in the json schema (#880) 2022-03-09 14:51:43 -05:00
536 changed files with 14491 additions and 6843 deletions
+1
View File
@@ -5,6 +5,7 @@ permit:
- MPL.*
- ISC
ignore-packages:
- .
# packageurl-go is released under the MIT license located in the root of the repo at /mit.LICENSE
- github.com/anchore/packageurl-go
+1 -1
View File
@@ -29,7 +29,7 @@ notarize() {
fi
# install gon
which gon || (brew tap mitchellh/gon && brew install mitchellh/gon/gon)
which gon || (go install github.com/mitchellh/gon/cmd/gon@latest)
# create config (note: json via stdin with gon is broken, can only use HCL from file)
hcl_file=$(mktemp).hcl
@@ -21,6 +21,14 @@ fi
if [ -z "$DOCKER_PASSWORD" ]; then
exit_with_error "DOCKER_PASSWORD not set"
fi
if [ -z "$GHCR_USERNAME" ]; then
exit_with_error "GHCR_USERNAME not set"
fi
if [ -z "$GHCR_PASSWORD" ]; then
exit_with_error "GHCR_PASSWORD not set"
fi
set -u
# setup_signing
@@ -41,4 +49,5 @@ setup_signing() {
commentary "log into docker -- required for publishing (since the default keychain has now been replaced)"
echo "${DOCKER_PASSWORD}" | docker login docker.io -u "${DOCKER_USERNAME}" --password-stdin
echo "${GHCR_PASSWORD}" | docker login ghcr.io -u "${GHCR_USERNAME}" --password-stdin
}
+5 -1
View File
@@ -9,7 +9,7 @@ on:
- "v*"
env:
GO_VERSION: "1.17.x"
GO_VERSION: "1.18.x"
jobs:
quality-gate:
@@ -94,6 +94,8 @@ jobs:
needs: [quality-gate]
# due to our code signing process, it's vital that we run our release steps on macOS
runs-on: macos-latest
permissions:
packages: write
steps:
- uses: docker-practice/actions-setup-docker@v1
@@ -128,6 +130,8 @@ jobs:
- name: Build & publish release artifacts
run: make release
env:
GHCR_USERNAME: ${{ github.actor }}
GHCR_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
DOCKER_USERNAME: ${{ secrets.TOOLBOX_DOCKER_USER }}
DOCKER_PASSWORD: ${{ secrets.TOOLBOX_DOCKER_PASS }}
# we use a different token than GITHUB_SECRETS to additionally allow updating the homebrew repos
@@ -0,0 +1,51 @@
name: PR for latest Stereoscope release
on:
schedule:
- cron: "0 8 * * *" # 3 AM EST
workflow_dispatch:
env:
GO_VERSION: "1.18.x"
GO_STABLE_VERSION: true
jobs:
upgrade-stereoscope:
runs-on: ubuntu-latest
if: github.repository == 'anchore/syft' # only run for main repo
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- run: |
LATEST_VERSION=$(git ls-remote https://github.com/anchore/stereoscope main | head -n1 | awk '{print $1;}')
# update go.mod
go get github.com/anchore/stereoscope@$LATEST_VERSION
go mod tidy
# export the version for use with create-pull-request
echo "::set-output name=LATEST_VERSION::$LATEST_VERSION"
id: latest-version
- uses: tibdex/github-app-token@v1
id: generate-token
with:
app_id: ${{ secrets.TOKEN_APP_ID }}
private_key: ${{ secrets.TOKEN_APP_PRIVATE_KEY }}
- uses: peter-evans/create-pull-request@v4
with:
signoff: true
delete-branch: true
branch: auto/latest
labels: dependencies
commit-message: "Update Stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}"
title: "Update Stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}"
body: |
Update Stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}
token: ${{ steps.generate-token.outputs.token }}
+18 -1
View File
@@ -7,7 +7,8 @@ on:
pull_request:
env:
GO_VERSION: "1.17.x"
GO_VERSION: "1.18.x"
GO_STABLE_VERSION: true
jobs:
@@ -19,6 +20,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -56,6 +58,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -92,6 +95,16 @@ jobs:
path: syft/pkg/cataloger/java/test-fixtures/java-builds/packages
key: ${{ runner.os }}-unit-java-cache-${{ hashFiles( 'syft/pkg/cataloger/java/test-fixtures/java-builds/packages.fingerprint' ) }}
- name: Build cache key for go binary test-fixture blobs (for unit tests)
run: make go-binaries-fingerprint
- name: Restore Go binary test-fixture cache
id: unit-go-binary-cache
uses: actions/cache@v2.1.3
with:
path: syft/pkg/cataloger/golang/test-fixtures/archs/binaries
key: ${{ runner.os }}-unit-go-binaries-cache-${{ hashFiles( 'syft/pkg/cataloger/golang/test-fixtures/archs/binaries.fingerprint' ) }}
- name: Run unit tests
run: make unit
@@ -108,6 +121,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -159,6 +173,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -231,6 +246,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -338,6 +354,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
+19
View File
@@ -12,6 +12,7 @@ before:
builds:
- id: linux-build
dir: ./cmd/syft
binary: syft
goos:
- linux
@@ -32,6 +33,7 @@ builds:
-X github.com/anchore/syft/internal/version.gitDescription={{.Summary}}
- id: darwin-build
dir: ./cmd/syft
binary: syft
goos:
- darwin
@@ -49,6 +51,7 @@ builds:
- ./.github/scripts/apple-signing/sign.sh "{{ .Path }}" "{{ .IsSnapshot }}" "{{ .Target }}"
- id: windows-build
dir: ./cmd/syft
binary: syft
goos:
- windows
@@ -99,6 +102,11 @@ dockers:
- "anchore/syft:{{ .Tag }}-amd64"
- "anchore/syft:v{{ .Major }}-amd64"
- "anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64"
- "ghcr.io/anchore/syft:latest"
- "ghcr.io/anchore/syft:{{ .Tag }}-amd64"
- "ghcr.io/anchore/syft:v{{ .Major }}-amd64"
- "ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64"
goarch: amd64
dockerfile: Dockerfile
use: buildx
build_flag_templates:
@@ -112,6 +120,9 @@ dockers:
- "anchore/syft:{{ .Tag }}-arm64v8"
- "anchore/syft:v{{ .Major }}-arm64v8"
- "anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8"
- "ghcr.io/anchore/syft:{{ .Tag }}-arm64v8"
- "ghcr.io/anchore/syft:v{{ .Major }}-arm64v8"
- "ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8"
goarch: arm64
dockerfile: Dockerfile
use: buildx
@@ -131,3 +142,11 @@ docker_manifests:
image_templates:
- anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64
- anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8
- name_template: ghcr.io/anchore/syft:{{ .Tag }}
image_templates:
- ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64
- ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8
- name_template: ghcr.io/anchore/syft:latest
image_templates:
- ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64
- ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8
-6
View File
@@ -82,12 +82,6 @@ Date: Mon Aug 1 11:27:13 2020 -0400
[//]: # (TODO: PR title and description)
## Sign your commits
To ensure the authenticity and integrity of code contributions, **we require that all commits are signed**. Signing commits proves that your commits were truly created by you, as the holder of a private key.
Configuring git to sign your commits is a straightforward process. To get set up, see [GitHub's documentation on signing commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits).
## Test your changes
This project has a `Makefile` which includes many helpers running both unit and integration tests. Although PRs will have automatic checks for these, it is useful to run them locally, ensuring they pass before submitting changes. Ensure you've bootstrapped once before running tests:
+12 -6
View File
@@ -106,7 +106,7 @@ $(TEMPDIR):
.PHONY: bootstrap-tools
bootstrap-tools: $(TEMPDIR)
GO111MODULE=off GOBIN=$(shell realpath $(TEMPDIR)) go get -u golang.org/x/perf/cmd/benchstat
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ v1.42.1
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ v1.45.0
curl -sSfL https://raw.githubusercontent.com/wagoodman/go-bouncer/master/bouncer.sh | sh -s -- -b $(TEMPDIR)/ v0.3.0
curl -sSfL https://raw.githubusercontent.com/anchore/chronicle/main/install.sh | sh -s -- -b $(TEMPDIR)/ v0.3.0
.github/scripts/goreleaser-install.sh -d -b $(TEMPDIR)/ v1.4.1
@@ -122,7 +122,7 @@ bootstrap: $(RESULTSDIR) bootstrap-go bootstrap-tools ## Download and install al
$(call title,Bootstrapping dependencies)
.PHONY: static-analysis
static-analysis: lint check-go-mod-tidy check-licenses
static-analysis: check-go-mod-tidy check-licenses lint
.PHONY: lint
lint: ## Run gofmt + golangci lint checks
@@ -147,7 +147,7 @@ lint-fix: ## Auto-format all source code + run golangci lint fixers
.PHONY: check-licenses
check-licenses: ## Ensure transitive dependencies are compliant with the current license policy
$(TEMPDIR)/bouncer check
$(TEMPDIR)/bouncer check ./cmd/syft
check-go-mod-tidy:
@ .github/scripts/go-mod-tidy-check.sh && echo "go.mod and go.sum are tidy!"
@@ -211,13 +211,19 @@ integration-fingerprint:
.PHONY: java-packages-fingerprint
java-packages-fingerprint:
$(call title,Java test fixture fingerprint)
cd syft/cataloger/packages/java/test-fixtures/java-builds && \
cd syft/pkg/cataloger/java/test-fixtures/java-builds && \
make packages.fingerprint
.PHONY: go-binaries-fingerprint
go-binaries-fingerprint:
$(call title,Go binaries test fixture fingerprint)
cd syft/pkg/cataloger/golang/test-fixtures/archs && \
make binaries.fingerprint
.PHONY: fixtures
fixtures:
$(call title,Generating test fixtures)
cd syft/cataloger/packages/java/test-fixtures/java-builds && make
cd syft/pkg/cataloger/java/test-fixtures/java-builds && make
.PHONY: generate-json-schema
generate-json-schema: ## Generate a new json schema
@@ -299,7 +305,7 @@ cli: $(SNAPSHOTDIR) ## Run CLI tests
chmod 755 "$(SNAPSHOT_BIN)"
$(SNAPSHOT_BIN) version
SYFT_BINARY_LOCATION='$(SNAPSHOT_BIN)' \
go test -count=1 -v ./test/cli
go test -count=1 -timeout=15m -v ./test/cli
.PHONY: changelog
changelog: clean-changelog CHANGELOG.md
+65 -63
View File
@@ -9,7 +9,7 @@
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/anchore/syft/blob/main/LICENSE)
[![Slack Invite](https://img.shields.io/badge/Slack-Join-blue?logo=slack)](https://anchore.com/slack)
A CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner tool like [Grype](https://github.com/anchore/grype).
A CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner like [Grype](https://github.com/anchore/grype).
### Join our community meetings!
@@ -20,15 +20,26 @@ A CLI tool and Go library for generating a Software Bill of Materials (SBOM) fro
![syft-demo](https://user-images.githubusercontent.com/590471/90277200-2a253000-de33-11ea-893f-32c219eea11a.gif)
## Features
- Catalog container images and filesystems to discover packages and libraries.
- Generate in-toto attestations where an SBOM is included as the payload.
- Supports packages and libraries from various ecosystems (APK, DEB, RPM, Ruby Bundles, Python Wheel/Egg/requirements.txt, JavaScript NPM/Yarn, Java JAR/EAR/WAR/PAR/SAR, Jenkins plugins JPI/HPI, Go modules, PHP Composer)
- Linux distribution identification (supports Alpine, BusyBox, CentOS/RedHat, Debian/Ubuntu flavored distributions)
- Supports Docker and OCI image formats
- Direct support for [Grype](https://github.com/anchore/grype), a fast and powerful vulnerability matcher.
- Generates SBOMs for container images, filesystems, archives, and more to discover packages and libraries
- Supports OCI and Docker image formats
- Linux distribution identification
- Works seamlessly with [Grype](https://github.com/anchore/grype) (a fast, modern vulnerability scanner)
- Able to create signed SBOM attestations using the [in-toto specification](https://github.com/in-toto/attestation/blob/main/spec/README.md)
### Supported Ecosystems
If you encounter an issue, please [let us know using the issue tracker](https://github.com/anchore/syft/issues).
- Alpine (apk)
- Dart (pubs)
- Debian (dpkg)
- Go (go.mod, Go binaries)
- Java (jar, ear, war, par, sar)
- JavaScript (npm, yarn)
- Jenkins Plugins (jpi, hpi)
- PHP (composer)
- Python (wheel, egg, poetry, requirements.txt)
- Red Hat (rpm)
- Ruby (gem)
- Rust (cargo.lock)
## Installation
@@ -53,15 +64,13 @@ brew install syft
### Nix
**Note**: nix packaging of Syft is [community maintained](https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/syft/default.nix)
Also syft is currently only in the [unstable channel](https://nixos.wiki/wiki/Nix_channels#The_official_channels) awaiting the `22.05` release
**Note**: Nix packaging of Syft is [community maintained](https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/syft/default.nix). Syft is currently only in the [unstable channel](https://nixos.wiki/wiki/Nix_channels#The_official_channels) awaiting the `22.05` release
```bash
nix-env -i syft
```
... or, just try it out in an ephemeral nix shell
... or, just try it out in an ephemeral nix shell:
```bash
nix-shell -p syft
@@ -70,33 +79,28 @@ nix-shell -p syft
## Getting started
#### SBOM
To generate an SBOM for an OCI image:
To generate an SBOM for a container image:
```
syft <image>
```
**Note**: This is equivalent to specifying the `packages` subcommand:
```
syft packages <image>
```
The above output includes only software that is visible in the container (i.e., the squashed representation of the image).
To include software from all image layers in the SBOM, regardless of its presence in the final image, provide `--scope all-layers`:
The above output includes only software that is visible in the container (i.e., the squashed representation of the image). To include software from all image layers in the SBOM, regardless of its presence in the final image, provide `--scope all-layers`:
```
syft packages <image> --scope all-layers
syft <image> --scope all-layers
```
#### SBOM Attestation
To generate an attested SBOM for an OCI image as the predicate of an in-toto attestation
#### SBOM attestation
To generate an SBOM attestation for a container image:
```
syft attest --output [FORMAT] --key [KEY] [SOURCE] [flags]
```
The above output is in the form of the [DSSE envelope](https://github.com/secure-systems-lab/dsse/blob/master/envelope.md#dsse-envelope).
The payload is a base64 encoded `in-toto` statement with the SBOM as the predicate, the payload type is `application/vnd.in-toto+json`, and the signatures array is populated
with the contents needed for public key verification. For details on workflows using this command see [here](#adding-an-sbom-to-an-image-as-an-attestation-using-syft).
The payload is a base64 encoded `in-toto` statement with the generated SBOM as the predicate. For details on workflows using this command see [here](#adding-an-sbom-to-an-image-as-an-attestation-using-syft).
### Supported sources
@@ -104,10 +108,10 @@ Syft can generate a SBOM from a variety of sources:
```
# catalog a container image archive (from the result of `docker image save ...`, `podman save ...`, or `skopeo copy` commands)
syft packages path/to/image.tar
syft path/to/image.tar
# catalog a directory
syft packages path/to/dir
syft path/to/dir
```
Sources can be explicitly provided with a scheme:
@@ -146,15 +150,15 @@ The output format for Syft is configurable as well using the
`-o` (or `--output`) option:
```
syft packages <image> -o <format>
syft <image> -o <format>
```
Where the `formats` available are:
- `json`: Use this to get as much information out of Syft as possible!
- `text`: A row-oriented, human-and-machine-friendly output.
- `cyclonedx`: A XML report conforming to the [CycloneDX 1.3 specification](https://cyclonedx.org/specification/overview/).
- `cyclonedx-json`: A JSON report conforming to the [CycloneDX 1.3 specification](https://cyclonedx.org/specification/overview/).
- `spdx`: A tag-value formatted report conforming to the [SPDX 2.2 specification](https://spdx.github.io/spdx-spec/).
- `cyclonedx-xml`: A XML report conforming to the [CycloneDX 1.4 specification](https://cyclonedx.org/specification/overview/).
- `cyclonedx-json`: A JSON report conforming to the [CycloneDX 1.4 specification](https://cyclonedx.org/specification/overview/).
- `spdx-tag-value`: A tag-value formatted report conforming to the [SPDX 2.2 specification](https://spdx.github.io/spdx-spec/).
- `spdx-json`: A JSON report conforming to the [SPDX 2.2 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.2/schemas/spdx-schema.json).
- `table`: A columnar summary (default).
@@ -164,20 +168,16 @@ Syft can also output _multiple_ files in differing formats by appending
`=<file>` to the option, for example to output Syft JSON and SPDX JSON:
```shell
syft packages <image> -o json=sbom.syft.json -o spdx-json=sbom.spdx.json
syft <image> -o json=sbom.syft.json -o spdx-json=sbom.spdx.json
```
## Private Registry Authentication
### Local Docker Credentials
When a container runtime is not present, Syft can still utilize credentials configured in common credential sources (such as `~/.docker/config.json`).
It will pull images from private registries using these credentials. The config file is where your credentials are stored when authenticating with private registries via some command like `docker login`.
For more information see the `go-containerregistry` [documentation](https://github.com/google/go-containerregistry/tree/main/pkg/authn).
When a container runtime is not present, Syft can still utilize credentials configured in common credential sources (such as `~/.docker/config.json`). It will pull images from private registries using these credentials. The config file is where your credentials are stored when authenticating with private registries via some command like `docker login`. For more information see the `go-containerregistry` [documentation](https://github.com/google/go-containerregistry/tree/main/pkg/authn).
An example `config.json` looks something like this:
```
// config.json
```json
{
"auths": {
"registry.example.com": {
@@ -190,15 +190,17 @@ An example `config.json` looks something like this:
You can run the following command as an example. It details the mount/environment configuration a container needs to access a private registry:
`docker run -v ./config.json:/config/config.json -e "DOCKER_CONFIG=/config" anchore/syft:latest <private_image>`
```
docker run -v ./config.json:/config/config.json -e "DOCKER_CONFIG=/config" anchore/syft:latest <private_image>
```
### Docker Credentials in Kubernetes
The below section shows a simple workflow on how to mount this config file as a secret into a container on kubernetes.
1. Create a secret. The value of `config.json` is important. It refers to the specification detailed [here](https://github.com/google/go-containerregistry/tree/main/pkg/authn#the-config-file).
Below this section is the `secret.yaml` file that the pod configuration will consume as a volume.
The key `config.json` is important. It will end up being the name of the file when mounted into the pod.
```
Here's a simple workflow to mount this config file as a secret into a container on Kubernetes.
1. Create a secret. The value of `config.json` is important. It refers to the specification detailed [here](https://github.com/google/go-containerregistry/tree/main/pkg/authn#the-config-file). Below this section is the `secret.yaml` file that the pod configuration will consume as a volume. The key `config.json` is important. It will end up being the name of the file when mounted into the pod.
```yaml
# secret.yaml
apiVersion: v1
@@ -213,11 +215,9 @@ The key `config.json` is important. It will end up being the name of the file wh
`kubectl apply -f secret.yaml`
2. Create your pod running syft. The env `DOCKER_CONFIG` is important because it advertises where to look for the credential file.
In the below example, setting `DOCKER_CONFIG=/config` informs syft that credentials can be found at `/config/config.json`.
This is why we used `config.json` as the key for our secret. When mounted into containers the secrets' key is used as the filename.
The `volumeMounts` section mounts our secret to `/config`. The `volumes` section names our volume and leverages the secret we created in step one.
```
2. Create your pod running syft. The env `DOCKER_CONFIG` is important because it advertises where to look for the credential file. In the below example, setting `DOCKER_CONFIG=/config` informs syft that credentials can be found at `/config/config.json`. This is why we used `config.json` as the key for our secret. When mounted into containers the secrets' key is used as the filename. The `volumeMounts` section mounts our secret to `/config`. The `volumes` section names our volume and leverages the secret we created in step one.
```yaml
# pod.yaml
apiVersion: v1
@@ -246,10 +246,9 @@ The `volumeMounts` section mounts our secret to `/config`. The `volumes` section
`kubectl apply -f pod.yaml`
3. The user can now run `kubectl logs syft-private-registry-demo`. The logs should show the syft analysis for the `<private_image>` provided in the pod configuration.
3. The user can now run `kubectl logs syft-private-registry-demo`. The logs should show the Syft analysis for the `<private_image>` provided in the pod configuration.
Using the above information, users should be able to configure private registry access without having to do so in the `grype` or `syft` configuration files.
They will also not be dependent on a docker daemon, (or some other runtime software) for registry configuration and access.
Using the above information, users should be able to configure private registry access without having to do so in the `grype` or `syft` configuration files. They will also not be dependent on a Docker daemon, (or some other runtime software) for registry configuration and access.
## Configuration
@@ -286,7 +285,11 @@ check-for-app-update: true
# exclude:
# - "/etc/**"
# - "./out/**/*.json"
exclude:
exclude: []
# os and/or architecture to use when referencing container images (e.g. "windows/armv6" or "arm64")
# same as --platform; SYFT_PLATFORM env var
platform: ""
# cataloging packages is exposed through the packages and power-user subcommands
package:
@@ -460,26 +463,25 @@ anchore:
`syft attest --output [FORMAT] --key [KEY] [SOURCE] [flags]`
SBOMs themselves can serve as input to different analysis tools. The Anchore organization offers the vulnerability scanner
[grype](https://github.com/anchore/grype) as one such tool.
One of the foundational approaches to "trust" between tools is for producers to use the artifacts generated by syft as attestations to their images.
The DSSE output of `syft attest` can be used with the [cosign](https://github.com/sigstore/cosign) tool to attach an attestation to an image.
SBOMs themselves can serve as input to different analysis tools. [Grype](https://github.com/anchore/grype), a vulnerability scanner CLI tool from Anchore, is one such tool. Publishers of container images can use attestations to enable their consumers to trust Syft-generated SBOM descriptions of those container images. To create and provide these attestations, image publishers can run `syft attest` in conjunction with the [cosign](https://github.com/sigstore/cosign) tool to attach SBOM attestations to their images.
#### Example attest
#### Example attestation
Note for the following example replace `docker.io/image:latest` with an image you own. You should also have push access to
its remote reference. Replace $MY_PRIVATE_KEY with a private key you own or have generated with cosign.
its remote reference. Replace `$MY_PRIVATE_KEY` with a private key you own or have generated with cosign.
```bash
syft attest --key $MY_PRIVATE_KEY docker.io/image:latest > image_latest_sbom_attestation.json
cosign attach attestation --attestation image_latest_sbom_attestation.json docker.io/image:latest
```
Verify the new attestation exists on your image
Verify the new attestation exists on your image.
```bash
cosign verify-attestation -key $MY_PUBLIC_KEY docker.io/image:latest | jq '.payload | @base64d | .payload | fromjson | .predicate'
```
You should see this output along with the attached SBOM.
You should see this output along with the attached SBOM:
```
Verification for docker.io/image:latest --
The following checks were performed on each of these signatures:
-333
View File
@@ -1,333 +0,0 @@
package cmd
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"os"
"strings"
"github.com/anchore/syft/internal/formats/cyclonedxjson"
"github.com/anchore/syft/internal/formats/spdx22json"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/stereoscope"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/in-toto/in-toto-golang/in_toto"
"github.com/pkg/errors"
"github.com/pkg/profile"
"github.com/sigstore/cosign/cmd/cosign/cli/sign"
"github.com/sigstore/cosign/pkg/cosign"
"github.com/sigstore/cosign/pkg/cosign/attestation"
"github.com/sigstore/sigstore/pkg/signature/dsse"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
"github.com/wagoodman/go-partybus"
signatureoptions "github.com/sigstore/sigstore/pkg/signature/options"
)
const (
attestExample = ` {{.appName}} {{.command}} --output [FORMAT] --key [KEY] alpine:latest
Supports the following image sources:
{{.appName}} {{.command}} --key [KEY] yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} --key [KEY] path/to/a/file/or/dir only for OCI tar or OCI directory
`
attestSchemeHelp = "\n" + indent + schemeHelpHeader + "\n" + imageSchemeHelp
attestHelp = attestExample + attestSchemeHelp
intotoJSONDsseType = `application/vnd.in-toto+json`
)
var attestFormats = []sbom.FormatID{
syftjson.ID,
spdx22json.ID,
cyclonedxjson.ID,
}
var (
attestCmd = &cobra.Command{
Use: "attest --output [FORMAT] --key [KEY] [SOURCE]",
Short: "Generate a package SBOM as an attestation for the given [SOURCE] container image",
Long: "Generate a packaged-based Software Bill Of Materials (SBOM) from a container image as the predicate of an in-toto attestation",
Example: internal.Tprintf(attestHelp, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "attest",
}),
Args: validateInputArgs,
SilenceUsage: true,
SilenceErrors: true,
PreRunE: func(cmd *cobra.Command, args []string) (err error) {
if appConfig.Dev.ProfileCPU && appConfig.Dev.ProfileMem {
return fmt.Errorf("cannot profile CPU and memory simultaneously")
}
return nil
},
RunE: func(cmd *cobra.Command, args []string) error {
if appConfig.Dev.ProfileCPU {
defer profile.Start(profile.CPUProfile).Stop()
} else if appConfig.Dev.ProfileMem {
defer profile.Start(profile.MemProfile).Stop()
}
return attestExec(cmd.Context(), cmd, args)
},
}
)
func fetchPassword(_ bool) (b []byte, err error) {
potentiallyPipedInput, err := internal.IsPipedInput()
if err != nil {
log.Warnf("unable to determine if there is piped input: %+v", err)
}
switch {
case appConfig.Attest.Password != "":
return []byte(appConfig.Attest.Password), nil
case potentiallyPipedInput:
// handle piped in passwords
pwBytes, err := io.ReadAll(os.Stdin)
if err != nil {
return nil, fmt.Errorf("unable to get password from stdin: %w", err)
}
// be resilient to input that may have newline characters (in case someone is using echo without -n)
cleanPw := strings.TrimRight(string(pwBytes), "\n")
return []byte(cleanPw), nil
case internal.IsTerminal():
return cosign.GetPassFromTerm(false)
}
return nil, errors.New("no method available to fetch password")
}
func selectPassFunc(keypath string) (cosign.PassFunc, error) {
keyContents, err := os.ReadFile(keypath)
if err != nil {
return nil, err
}
var fn cosign.PassFunc = func(bool) (b []byte, err error) { return nil, nil }
_, err = cosign.LoadPrivateKey(keyContents, nil)
if err != nil {
fn = fetchPassword
}
return fn, nil
}
func attestExec(ctx context.Context, _ *cobra.Command, args []string) error {
// can only be an image for attestation or OCI DIR
userInput := args[0]
si, err := source.ParseInput(userInput, appConfig.Platform, false)
if err != nil {
return fmt.Errorf("could not generate source input for attest command: %w", err)
}
switch si.Type {
case source.ImageType, source.UnknownType:
// at this point we know that it cannot be dir: or file: schemes, so we will assume that the unknown scheme could represent an image
si.Type = source.ImageType
default:
return fmt.Errorf("attest command can only be used with image sources but discovered %q when given %q", si.Type, userInput)
}
// if the original detection was from a local daemon we want to short circuit
// that and attempt to generate the image source from a registry source instead
switch si.ImageSource {
case image.UnknownSource, image.OciRegistrySource:
si.ImageSource = image.OciRegistrySource
default:
return fmt.Errorf("attest command can only be used with image sources fetch directly from the registry, but discovered an image source of %q when given %q", si.ImageSource, userInput)
}
if len(appConfig.Outputs) > 1 {
return fmt.Errorf("unable to generate attestation for more than one output")
}
format := syft.FormatByName(appConfig.Outputs[0])
predicateType := formatPredicateType(format)
if predicateType == "" {
return fmt.Errorf("could not produce attestation predicate for given format: %q. Available formats: %+v", formatAliases(format.ID()), formatAliases(attestFormats...))
}
passFunc, err := selectPassFunc(appConfig.Attest.Key)
if err != nil {
return err
}
ko := sign.KeyOpts{
KeyRef: appConfig.Attest.Key,
PassFunc: passFunc,
}
sv, err := sign.SignerFromKeyOpts(ctx, "", ko)
if err != nil {
return err
}
defer sv.Close()
return eventLoop(
attestationExecWorker(*si, format, predicateType, sv),
setupSignals(),
eventSubscription,
stereoscope.Cleanup,
ui.Select(isVerbose(), appConfig.Quiet)...,
)
}
func attestationExecWorker(si source.Input, format sbom.Format, predicateType string, sv *sign.SignerVerifier) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
catalogingOpts, err := appConfig.ToCatalogingOptions()
if err != nil {
errs <- err
return
}
src, cleanup, err := source.NewFromRegistry(si, appConfig.Registry.ToOptions(), appConfig.Exclusions)
if cleanup != nil {
defer cleanup()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", si.UserInput, err)
return
}
s, err := syft.Catalog(src, catalogingOpts...)
if err != nil {
errs <- err
return
}
if s == nil {
errs <- fmt.Errorf("no SBOM produced for %q", si.UserInput)
return
}
sbomBytes, err := syft.Encode(*s, format)
if err != nil {
errs <- err
return
}
err = generateAttestation(sbomBytes, src, sv, predicateType)
if err != nil {
errs <- err
return
}
}()
return errs
}
func formatPredicateType(format sbom.Format) string {
switch format.ID() {
case spdx22json.ID:
return in_toto.PredicateSPDX
case cyclonedxjson.ID:
// Tentative see https://github.com/in-toto/attestation/issues/82
return "https://cyclonedx.org/bom"
case syftjson.ID:
return "https://syft.dev/bom"
default:
return ""
}
}
func findValidDigest(digests []string) string {
// since we are only using the OCI repo provider for this source we are safe that this is only 1 value
// see https://github.com/anchore/stereoscope/blob/25ebd49a842b5ac0a20c2e2b4b81335b64ad248c/pkg/image/oci/registry_provider.go#L57-L63
split := strings.Split(digests[0], "sha256:")
return split[1]
}
func generateAttestation(predicate []byte, src *source.Source, sv *sign.SignerVerifier, predicateType string) error {
switch len(src.Image.Metadata.RepoDigests) {
case 0:
return fmt.Errorf("cannot generate attestation since no repo digests were found; make sure you're passing an OCI registry source for the attest command")
case 1:
default:
return fmt.Errorf("cannot generate attestation since multiple repo digests were found for the image: %+v", src.Image.Metadata.RepoDigests)
}
wrapped := dsse.WrapSigner(sv, intotoJSONDsseType)
sh, err := attestation.GenerateStatement(attestation.GenerateOpts{
Predicate: bytes.NewBuffer(predicate),
Type: predicateType,
Digest: findValidDigest(src.Image.Metadata.RepoDigests),
})
if err != nil {
return err
}
payload, err := json.Marshal(sh)
if err != nil {
return err
}
signedPayload, err := wrapped.SignMessage(bytes.NewReader(payload), signatureoptions.WithContext(context.Background()))
if err != nil {
return errors.Wrap(err, "unable to sign SBOM")
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error {
_, err := os.Stdout.Write(signedPayload)
return err
},
})
return nil
}
func init() {
setAttestFlags(attestCmd.Flags())
if err := bindAttestConfigOptions(attestCmd.Flags()); err != nil {
panic(err)
}
rootCmd.AddCommand(attestCmd)
}
func setAttestFlags(flags *pflag.FlagSet) {
// key options
flags.StringP("key", "", "cosign.key",
"path to the private key file to use for attestation",
)
// in-toto attestations only support JSON predicates, so not all SBOM formats that syft can output are supported
flags.StringP(
"output", "o", formatAliases(syftjson.ID)[0],
fmt.Sprintf("the SBOM format encapsulated within the attestation, available options=%v", formatAliases(attestFormats...)),
)
flags.StringP(
"platform", "", "",
"an optional platform specifier for container image sources (e.g. 'linux/arm64', 'linux/arm64/v8', 'arm64', 'linux')",
)
}
func bindAttestConfigOptions(flags *pflag.FlagSet) error {
// note: output is not included since this configuration option is shared between multiple subcommands
if err := viper.BindPFlag("attest.key", flags.Lookup("key")); err != nil {
return err
}
return nil
}
-31
View File
@@ -1,31 +0,0 @@
package cmd
import (
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/event"
"github.com/wagoodman/go-partybus"
)
func checkForApplicationUpdate() {
if appConfig.CheckForAppUpdate {
log.Debugf("checking if new vesion of %s is available", internal.ApplicationName)
isAvailable, newVersion, err := version.IsUpdateAvailable()
if err != nil {
// this should never stop the application
log.Errorf(err.Error())
}
if isAvailable {
log.Infof("new version of %s is available: %s (current version is %s)", internal.ApplicationName, newVersion, version.FromBuild().Version)
bus.Publish(partybus.Event{
Type: event.AppUpdateAvailable,
Value: newVersion,
})
} else {
log.Debugf("no new %s update available", internal.ApplicationName)
}
}
}
-171
View File
@@ -1,171 +0,0 @@
package cmd
import (
"encoding/json"
"fmt"
"os"
"sort"
"github.com/spf13/pflag"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/logger"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/gookit/color"
"github.com/spf13/cobra"
"github.com/spf13/viper"
"github.com/wagoodman/go-partybus"
)
var (
appConfig *config.Application
eventBus *partybus.Bus
eventSubscription *partybus.Subscription
)
func init() {
cobra.OnInitialize(
initCmdAliasBindings,
initAppConfig,
initLogging,
logAppConfig,
checkForApplicationUpdate,
logAppVersion,
initEventBus,
)
}
func Execute() {
if err := rootCmd.Execute(); err != nil {
fmt.Fprintln(os.Stderr, color.Red.Sprint(err.Error()))
os.Exit(1)
}
}
// we must setup the config-cli bindings first before the application configuration is parsed. However, this cannot
// be done without determining what the primary command that the config options should be bound to since there are
// shared concerns (the root-packages alias).
func initCmdAliasBindings() {
activeCmd, _, err := rootCmd.Find(os.Args[1:])
if err != nil {
panic(err)
}
// enable all cataloger by default if power-user command is run
if activeCmd == powerUserCmd {
config.PowerUserCatalogerEnabledDefault()
}
// set bindings based on the packages alias
switch activeCmd {
case packagesCmd, rootCmd:
// note: we need to lazily bind config options since they are shared between both the root command
// and the packages command. Otherwise there will be global viper state that is in contention.
// See for more details: https://github.com/spf13/viper/issues/233 . Additionally, the bindings must occur BEFORE
// reading the application configuration, which implies that it must be an initializer (or rewrite the command
// initialization structure against typical patterns used with cobra, which is somewhat extreme for a
// temporary alias)
if err = bindPackagesConfigOptions(activeCmd.Flags()); err != nil {
panic(err)
}
case attestCmd:
// the --output and --platform options are independently defined flags, but a shared config option
if err = bindSharedConfigOption(attestCmd.Flags()); err != nil {
panic(err)
}
// even though the root command or packages command is NOT being run, we still need default bindings
// such that application config parsing passes.
if err = bindExclusivePackagesConfigOptions(packagesCmd.Flags()); err != nil {
panic(err)
}
default:
// even though the root command or packages command is NOT being run, we still need default bindings
// such that application config parsing passes.
if err = bindPackagesConfigOptions(packagesCmd.Flags()); err != nil {
panic(err)
}
}
}
func bindSharedConfigOption(flags *pflag.FlagSet) error {
if err := viper.BindPFlag("output", flags.Lookup("output")); err != nil {
return err
}
if err := viper.BindPFlag("platform", flags.Lookup("platform")); err != nil {
return err
}
return nil
}
func initAppConfig() {
cfg, err := config.LoadApplicationConfig(viper.GetViper(), persistentOpts)
if err != nil {
fmt.Printf("failed to load application config: \n\t%+v\n", err)
os.Exit(1)
}
appConfig = cfg
}
func initLogging() {
cfg := logger.LogrusConfig{
EnableConsole: (appConfig.Log.FileLocation == "" || appConfig.CliOptions.Verbosity > 0) && !appConfig.Quiet,
EnableFile: appConfig.Log.FileLocation != "",
Level: appConfig.Log.LevelOpt,
Structured: appConfig.Log.Structured,
FileLocation: appConfig.Log.FileLocation,
}
logWrapper := logger.NewLogrusLogger(cfg)
syft.SetLogger(logWrapper)
stereoscope.SetLogger(&logger.LogrusNestedLogger{
Logger: logWrapper.Logger.WithField("from-lib", "stereoscope"),
})
}
func logAppConfig() {
log.Debugf("application config:\n%+v", color.Magenta.Sprint(appConfig.String()))
}
func initEventBus() {
eventBus = partybus.NewBus()
eventSubscription = eventBus.Subscribe()
stereoscope.SetBus(eventBus)
syft.SetBus(eventBus)
}
func logAppVersion() {
versionInfo := version.FromBuild()
log.Infof("syft version: %s", versionInfo.Version)
var fields map[string]interface{}
bytes, err := json.Marshal(versionInfo)
if err != nil {
return
}
err = json.Unmarshal(bytes, &fields)
if err != nil {
return
}
keys := make([]string, 0, len(fields))
for k := range fields {
keys = append(keys, k)
}
sort.Strings(keys)
for idx, field := range keys {
value := fields[field]
branch := "├──"
if idx == len(fields)-1 {
branch = "└──"
}
log.Debugf(" %s %s: %s", branch, field, value)
}
}
-113
View File
@@ -1,113 +0,0 @@
package cmd
import (
"context"
"os"
"strings"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/client"
"github.com/spf13/cobra"
)
// completionCmd represents the completion command
var completionCmd = &cobra.Command{
Use: "completion [bash|zsh|fish]",
Short: "Generate a shell completion for Syft (listing local docker images)",
Long: `To load completions (docker image list):
Bash:
$ source <(syft completion bash)
# To load completions for each session, execute once:
Linux:
$ syft completion bash > /etc/bash_completion.d/syft
MacOS:
$ syft completion bash > /usr/local/etc/bash_completion.d/syft
Zsh:
# If shell completion is not already enabled in your environment you will need
# to enable it. You can execute the following once:
$ echo "autoload -U compinit; compinit" >> ~/.zshrc
# To load completions for each session, execute once:
$ syft completion zsh > "${fpath[1]}/_syft"
# You will need to start a new shell for this setup to take effect.
Fish:
$ syft completion fish | source
# To load completions for each session, execute once:
$ syft completion fish > ~/.config/fish/completions/syft.fish
`,
DisableFlagsInUseLine: true,
ValidArgs: []string{"bash", "zsh", "fish"},
Args: cobra.ExactValidArgs(1),
Run: func(cmd *cobra.Command, args []string) {
var err error
switch args[0] {
case "bash":
err = cmd.Root().GenBashCompletion(os.Stdout)
case "zsh":
err = cmd.Root().GenZshCompletion(os.Stdout)
case "fish":
err = cmd.Root().GenFishCompletion(os.Stdout, true)
}
if err != nil {
panic(err)
}
},
}
func init() {
rootCmd.AddCommand(completionCmd)
}
func dockerImageValidArgsFunction(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
// Since we use ValidArgsFunction, Cobra will call this AFTER having parsed all flags and arguments provided
dockerImageRepoTags, err := listLocalDockerImages(toComplete)
if err != nil {
// Indicates that an error occurred and completions should be ignored
return []string{"completion failed"}, cobra.ShellCompDirectiveError
}
if len(dockerImageRepoTags) == 0 {
return []string{"no docker images found"}, cobra.ShellCompDirectiveError
}
// ShellCompDirectiveDefault indicates that the shell will perform its default behavior after completions have
// been provided (without implying other possible directives)
return dockerImageRepoTags, cobra.ShellCompDirectiveDefault
}
func listLocalDockerImages(prefix string) ([]string, error) {
var repoTags = make([]string, 0)
ctx := context.Background()
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
return repoTags, err
}
// Only want to return tagged images
imageListArgs := filters.NewArgs()
imageListArgs.Add("dangling", "false")
images, err := cli.ImageList(ctx, types.ImageListOptions{All: false, Filters: imageListArgs})
if err != nil {
return repoTags, err
}
for _, image := range images {
// image may have multiple tags
for _, tag := range image.RepoTags {
if strings.HasPrefix(tag, prefix) {
repoTags = append(repoTags, tag)
}
}
}
return repoTags, nil
}
-64
View File
@@ -1,64 +0,0 @@
package cmd
import (
"fmt"
"strings"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/sbom"
"github.com/hashicorp/go-multierror"
)
// makeWriter creates a sbom.Writer for output or returns an error. this will either return a valid writer
// or an error but neither both and if there is no error, sbom.Writer.Close() should be called
func makeWriter(outputs []string, defaultFile string) (sbom.Writer, error) {
outputOptions, err := parseOptions(outputs, defaultFile)
if err != nil {
return nil, err
}
writer, err := sbom.NewWriter(outputOptions...)
if err != nil {
return nil, err
}
return writer, nil
}
// parseOptions utility to parse command-line option strings and retain the existing behavior of default format and file
func parseOptions(outputs []string, defaultFile string) (out []sbom.WriterOption, errs error) {
// always should have one option -- we generally get the default of "table", but just make sure
if len(outputs) == 0 {
outputs = append(outputs, string(table.ID))
}
for _, name := range outputs {
name = strings.TrimSpace(name)
// split to at most two parts for <format>=<file>
parts := strings.SplitN(name, "=", 2)
// the format name is the first part
name = parts[0]
// default to the --file or empty string if not specified
file := defaultFile
// If a file is specified as part of the output formatName, use that
if len(parts) > 1 {
file = parts[1]
}
format := syft.FormatByName(name)
if format == nil {
errs = multierror.Append(errs, fmt.Errorf("bad output format: '%s'", name))
continue
}
out = append(out, sbom.NewWriterOption(format, file))
}
return out, errs
}
-78
View File
@@ -1,78 +0,0 @@
package cmd
import (
"fmt"
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
func TestOutputWriterConfig(t *testing.T) {
tmp := t.TempDir() + "/"
tests := []struct {
outputs []string
file string
err bool
expected []string
}{
{
outputs: []string{},
expected: []string{""},
},
{
outputs: []string{"json"},
expected: []string{""},
},
{
file: "test-1.json",
expected: []string{"test-1.json"},
},
{
outputs: []string{"json=test-2.json"},
expected: []string{"test-2.json"},
},
{
outputs: []string{"json=test-3-1.json", "spdx-json=test-3-2.json"},
expected: []string{"test-3-1.json", "test-3-2.json"},
},
{
outputs: []string{"text", "json=test-4.json"},
expected: []string{"", "test-4.json"},
},
}
for _, test := range tests {
t.Run(fmt.Sprintf("%s/%s", test.outputs, test.file), func(t *testing.T) {
outputs := test.outputs
for i, val := range outputs {
outputs[i] = strings.Replace(val, "=", "="+tmp, 1)
}
file := test.file
if file != "" {
file = tmp + file
}
_, err := makeWriter(test.outputs, file)
if test.err {
assert.Error(t, err)
return
} else {
assert.NoError(t, err)
}
for _, expected := range test.expected {
if expected != "" {
assert.FileExists(t, tmp+expected)
} else if file != "" {
assert.FileExists(t, file)
} else {
assert.NoFileExists(t, expected)
}
}
})
}
}
-362
View File
@@ -1,362 +0,0 @@
package cmd
import (
"context"
"fmt"
"io/ioutil"
"os"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/anchore"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/pkg/profile"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
"github.com/wagoodman/go-partybus"
)
const (
packagesExample = ` {{.appName}} {{.command}} alpine:latest a summary of discovered packages
{{.appName}} {{.command}} alpine:latest -o json show all possible cataloging details
{{.appName}} {{.command}} alpine:latest -o cyclonedx show a CycloneDX formatted SBOM
{{.appName}} {{.command}} alpine:latest -o cyclonedx-json show a CycloneDX JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx show a SPDX 2.2 Tag-Value formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx-json show a SPDX 2.2 JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -vv show verbose debug information
Supports the following image sources:
{{.appName}} {{.command}} yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} path/to/a/file/or/dir a Docker tar, OCI tar, OCI directory, or generic filesystem directory
`
schemeHelpHeader = "You can also explicitly specify the scheme to use:"
imageSchemeHelp = ` {{.appName}} {{.command}} docker:yourrepo/yourimage:tag explicitly use the Docker daemon
{{.appName}} {{.command}} podman:yourrepo/yourimage:tag explicitly use the Podman daemon
{{.appName}} {{.command}} registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
{{.appName}} {{.command}} docker-archive:path/to/yourimage.tar use a tarball from disk for archives created from "docker save"
{{.appName}} {{.command}} oci-archive:path/to/yourimage.tar use a tarball from disk for OCI archives (from Skopeo or otherwise)
{{.appName}} {{.command}} oci-dir:path/to/yourimage read directly from a path on disk for OCI layout directories (from Skopeo or otherwise)
`
nonImageSchemeHelp = ` {{.appName}} {{.command}} dir:path/to/yourproject read directly from a path on disk (any directory)
{{.appName}} {{.command}} file:path/to/yourproject/file read directly from a path on disk (any single file)
`
packagesSchemeHelp = "\n" + indent + schemeHelpHeader + "\n" + imageSchemeHelp + nonImageSchemeHelp
packagesHelp = packagesExample + packagesSchemeHelp
)
var (
packagesCmd = &cobra.Command{
Use: "packages [SOURCE]",
Short: "Generate a package SBOM",
Long: "Generate a packaged-based Software Bill Of Materials (SBOM) from container images and filesystems",
Example: internal.Tprintf(packagesHelp, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "packages",
}),
Args: validateInputArgs,
SilenceUsage: true,
SilenceErrors: true,
PreRunE: func(cmd *cobra.Command, args []string) (err error) {
if appConfig.Dev.ProfileCPU && appConfig.Dev.ProfileMem {
return fmt.Errorf("cannot profile CPU and memory simultaneously")
}
return nil
},
RunE: func(cmd *cobra.Command, args []string) error {
if appConfig.Dev.ProfileCPU {
defer profile.Start(profile.CPUProfile).Stop()
} else if appConfig.Dev.ProfileMem {
defer profile.Start(profile.MemProfile).Stop()
}
return packagesExec(cmd, args)
},
ValidArgsFunction: dockerImageValidArgsFunction,
}
)
func init() {
setPackageFlags(packagesCmd.Flags())
rootCmd.AddCommand(packagesCmd)
}
func setPackageFlags(flags *pflag.FlagSet) {
// Formatting & Input options //////////////////////////////////////////////
flags.StringP(
"scope", "s", syft.DefaultCatalogingConfig().DefaultScope.String(),
fmt.Sprintf("selection of layers to catalog, options=%v", source.AllScopes))
flags.StringArrayP(
"output", "o", formatAliases(table.ID),
fmt.Sprintf("report output format, options=%v", formatAliases(syft.FormatIDs()...)),
)
flags.StringP(
"file", "", "",
"file to write the default report output to (default is STDOUT)",
)
flags.StringP(
"platform", "", "",
"an optional platform specifier for container image sources (e.g. 'linux/arm64', 'linux/arm64/v8', 'arm64', 'linux')",
)
flags.StringArrayP(
"cataloger", "", nil,
// TODO: have a nice way for a user to get the set of catalogers that exist
"set which catalogers to use when generating an SBOM",
)
// Upload options //////////////////////////////////////////////////////////
flags.StringP(
"host", "H", "",
"the hostname or URL of the Anchore Enterprise instance to upload to",
)
flags.StringP(
"username", "u", "",
"the username to authenticate against Anchore Enterprise",
)
flags.StringP(
"password", "p", "",
"the password to authenticate against Anchore Enterprise",
)
flags.StringP(
"dockerfile", "d", "",
"include dockerfile for upload to Anchore Enterprise",
)
flags.StringArrayP(
"exclude", "", nil,
"exclude paths from being scanned using a glob expression",
)
flags.Bool(
"overwrite-existing-image", false,
"overwrite an existing image during the upload to Anchore Enterprise",
)
flags.Uint(
"import-timeout", 30,
"set a timeout duration (in seconds) for the upload to Anchore Enterprise",
)
}
func bindPackagesConfigOptions(flags *pflag.FlagSet) error {
if err := bindExclusivePackagesConfigOptions(flags); err != nil {
return err
}
if err := bindSharedConfigOption(flags); err != nil {
return err
}
return nil
}
// NOTE(alex): Write a helper for the binding operation, which can be used to perform the binding but also double check that the intended effect was had or else return an error. Another thought is to somehow provide zero-valued defaults for all values in our config struct (maybe with reflection?). There may be a mechanism that already exists in viper that protects against this that I'm not aware of. ref: https://github.com/anchore/syft/pull/805#discussion_r801931192
func bindExclusivePackagesConfigOptions(flags *pflag.FlagSet) error {
// Formatting & Input options //////////////////////////////////////////////
// note: output is not included since this configuration option is shared between multiple subcommands
if err := viper.BindPFlag("scope", flags.Lookup("scope")); err != nil {
return err
}
if err := viper.BindPFlag("catalogers", flags.Lookup("cataloger")); err != nil {
return err
}
if err := viper.BindPFlag("file", flags.Lookup("file")); err != nil {
return err
}
if err := viper.BindPFlag("exclude", flags.Lookup("exclude")); err != nil {
return err
}
// Upload options //////////////////////////////////////////////////////////
if err := viper.BindPFlag("anchore.host", flags.Lookup("host")); err != nil {
return err
}
if err := viper.BindPFlag("anchore.username", flags.Lookup("username")); err != nil {
return err
}
if err := viper.BindPFlag("anchore.password", flags.Lookup("password")); err != nil {
return err
}
if err := viper.BindPFlag("anchore.dockerfile", flags.Lookup("dockerfile")); err != nil {
return err
}
if err := viper.BindPFlag("anchore.overwrite-existing-image", flags.Lookup("overwrite-existing-image")); err != nil {
return err
}
if err := viper.BindPFlag("anchore.import-timeout", flags.Lookup("import-timeout")); err != nil {
return err
}
return nil
}
func validateInputArgs(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
// in the case that no arguments are given we want to show the help text and return with a non-0 return code.
if err := cmd.Help(); err != nil {
return fmt.Errorf("unable to display help: %w", err)
}
return fmt.Errorf("an image/directory argument is required")
}
return cobra.MaximumNArgs(1)(cmd, args)
}
func packagesExec(_ *cobra.Command, args []string) error {
writer, err := makeWriter(appConfig.Outputs, appConfig.File)
if err != nil {
return err
}
defer func() {
if err := writer.Close(); err != nil {
log.Warnf("unable to write to report destination: %w", err)
}
}()
// could be an image or a directory, with or without a scheme
userInput := args[0]
si, err := source.ParseInput(userInput, appConfig.Platform, true)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
return eventLoop(
packagesExecWorker(*si, writer),
setupSignals(),
eventSubscription,
stereoscope.Cleanup,
ui.Select(isVerbose(), appConfig.Quiet)...,
)
}
func isVerbose() (result bool) {
isPipedInput, err := internal.IsPipedInput()
if err != nil {
// since we can't tell if there was piped input we assume that there could be to disable the ETUI
log.Warnf("unable to determine if there is piped input: %+v", err)
return true
}
// verbosity should consider if there is piped input (in which case we should not show the ETUI)
return appConfig.CliOptions.Verbosity > 0 || isPipedInput
}
func packagesExecWorker(si source.Input, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
catalogingOpts, err := appConfig.ToCatalogingOptions()
if err != nil {
errs <- err
return
}
src, cleanup, err := source.New(si, appConfig.Registry.ToOptions(), appConfig.Exclusions)
if cleanup != nil {
defer cleanup()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", si.UserInput, err)
return
}
s, err := syft.Catalog(src, catalogingOpts...)
if err != nil {
errs <- err
return
}
if s == nil {
errs <- fmt.Errorf("no SBOM produced for %q", si.UserInput)
return
}
if appConfig.Anchore.Host != "" {
if err := runPackageSbomUpload(src, *s); err != nil {
errs <- err
return
}
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error { return writer.Write(*s) },
})
}()
return errs
}
func runPackageSbomUpload(src *source.Source, s sbom.SBOM) error {
log.Infof("uploading results to %s", appConfig.Anchore.Host)
if src.Metadata.Scheme != source.ImageType {
return fmt.Errorf("unable to upload results: only images are supported")
}
var dockerfileContents []byte
if appConfig.Anchore.Dockerfile != "" {
if _, err := os.Stat(appConfig.Anchore.Dockerfile); os.IsNotExist(err) {
return fmt.Errorf("unable dockerfile=%q does not exist: %w", appConfig.Anchore.Dockerfile, err)
}
fh, err := os.Open(appConfig.Anchore.Dockerfile)
if err != nil {
return fmt.Errorf("unable to open dockerfile=%q: %w", appConfig.Anchore.Dockerfile, err)
}
dockerfileContents, err = ioutil.ReadAll(fh)
if err != nil {
return fmt.Errorf("unable to read dockerfile=%q: %w", appConfig.Anchore.Dockerfile, err)
}
}
c, err := anchore.NewClient(anchore.Configuration{
BaseURL: appConfig.Anchore.Host,
Username: appConfig.Anchore.Username,
Password: appConfig.Anchore.Password,
})
if err != nil {
return fmt.Errorf("failed to create anchore client: %w", err)
}
importCfg := anchore.ImportConfig{
ImageMetadata: src.Image.Metadata,
SBOM: s,
Dockerfile: dockerfileContents,
OverwriteExistingUpload: appConfig.Anchore.OverwriteExistingImage,
Timeout: appConfig.Anchore.ImportTimeout,
}
if err := c.Import(context.Background(), importCfg); err != nil {
return fmt.Errorf("failed to upload results to host=%s: %+v", appConfig.Anchore.Host, err)
}
return nil
}
-153
View File
@@ -1,153 +0,0 @@
package cmd
import (
"fmt"
"github.com/anchore/syft/syft"
"os"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/gookit/color"
"github.com/pkg/profile"
"github.com/spf13/cobra"
"github.com/wagoodman/go-partybus"
)
const powerUserExample = ` {{.appName}} {{.command}} <image>
DEPRECATED - THIS COMMAND WILL BE REMOVED in v1.0.0
Only image sources are supported (e.g. docker: , podman: , docker-archive: , oci: , etc.), the directory source (dir:) is not supported.
All behavior is controlled via application configuration and environment variables (see https://github.com/anchore/syft#configuration)
`
var powerUserOpts = struct {
configPath string
}{}
var powerUserCmd = &cobra.Command{
Use: "power-user [IMAGE]",
Short: "Run bulk operations on container images",
Example: internal.Tprintf(powerUserExample, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "power-user",
}),
Args: validateInputArgs,
Hidden: true,
SilenceUsage: true,
SilenceErrors: true,
PreRunE: func(cmd *cobra.Command, args []string) error {
if appConfig.Dev.ProfileCPU && appConfig.Dev.ProfileMem {
return fmt.Errorf("cannot profile CPU and memory simultaneously")
}
return nil
},
RunE: func(cmd *cobra.Command, args []string) error {
if appConfig.Dev.ProfileCPU {
defer profile.Start(profile.CPUProfile).Stop()
} else if appConfig.Dev.ProfileMem {
defer profile.Start(profile.MemProfile).Stop()
}
return powerUserExec(cmd, args)
},
ValidArgsFunction: dockerImageValidArgsFunction,
}
func init() {
powerUserCmd.Flags().StringVarP(&powerUserOpts.configPath, "config", "c", "", "config file path with all power-user options")
rootCmd.AddCommand(powerUserCmd)
}
func powerUserExec(_ *cobra.Command, args []string) error {
// could be an image or a directory, with or without a scheme
userInput := args[0]
writer, err := sbom.NewWriter(
sbom.NewWriterOption(
syftjson.Format(),
appConfig.File,
),
)
if err != nil {
return err
}
defer func() {
if err := writer.Close(); err != nil {
log.Warnf("unable to write to report destination: %+v", err)
}
// inform user at end of run that command will be removed
deprecated := color.Style{color.Red, color.OpBold}.Sprint("DEPRECATED: This command will be removed in v1.0.0")
fmt.Fprintln(os.Stderr, deprecated)
}()
return eventLoop(
powerUserExecWorker(userInput, writer),
setupSignals(),
eventSubscription,
stereoscope.Cleanup,
ui.Select(isVerbose(), appConfig.Quiet)...,
)
}
func powerUserExecWorker(userInput string, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
// TODO: replace
//appConfig.Secrets.Cataloger.Enabled = true
//appConfig.FileMetadata.Cataloger.Enabled = true
//appConfig.FileContents.Cataloger.Enabled = true
//appConfig.FileClassification.Cataloger.Enabled = true
catalogingOpts, err := appConfig.ToCatalogingOptions()
if err != nil {
errs <- err
return
}
si, err := source.ParseInput(userInput, appConfig.Platform, true)
if err != nil {
errs <- err
return
}
src, cleanup, err := source.New(*si, appConfig.Registry.ToOptions(), appConfig.Exclusions)
if err != nil {
errs <- err
return
}
if cleanup != nil {
defer cleanup()
}
s, err := syft.Catalog(src, catalogingOpts...)
if err != nil {
errs <- err
return
}
if s == nil {
errs <- fmt.Errorf("no SBOM produced for %q", si.UserInput)
return
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error { return writer.Write(*s) },
})
}()
return errs
}
-52
View File
@@ -1,52 +0,0 @@
package cmd
import (
"fmt"
"os"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/version"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
var persistentOpts = config.CliOnlyOptions{}
// rootCmd is currently an alias for the packages command
var rootCmd = &cobra.Command{
Short: packagesCmd.Short,
Long: packagesCmd.Long,
Args: packagesCmd.Args,
Example: packagesCmd.Example,
SilenceUsage: true,
SilenceErrors: true,
PreRunE: packagesCmd.PreRunE,
RunE: packagesCmd.RunE,
ValidArgsFunction: packagesCmd.ValidArgsFunction,
Version: version.FromBuild().Version,
}
const indent = " "
func init() {
// set universal flags
rootCmd.PersistentFlags().StringVarP(&persistentOpts.ConfigPath, "config", "c", "", "application config file")
// setting the version template to just print out the string since we already have a templatized version string
rootCmd.SetVersionTemplate(fmt.Sprintf("%s {{.Version}}\n", internal.ApplicationName))
flag := "quiet"
rootCmd.PersistentFlags().BoolP(
flag, "q", false,
"suppress all logging output",
)
if err := viper.BindPFlag(flag, rootCmd.PersistentFlags().Lookup(flag)); err != nil {
fmt.Printf("unable to bind flag '%s': %+v", flag, err)
os.Exit(1)
}
rootCmd.PersistentFlags().CountVarP(&persistentOpts.Verbosity, "verbose", "v", "increase verbosity (-v = info, -vv = debug)")
// set common options that are not universal (package subcommand-alias specific)
setPackageFlags(rootCmd.Flags())
}
+66
View File
@@ -0,0 +1,66 @@
package cli
import (
"fmt"
"log"
"github.com/anchore/syft/cmd/syft/cli/attest"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
const (
attestExample = ` {{.appName}} {{.command}} --output [FORMAT] --key [KEY] alpine:latest
Supports the following image sources:
{{.appName}} {{.command}} --key [KEY] yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} --key [KEY] path/to/a/file/or/dir only for OCI tar or OCI directory
`
attestSchemeHelp = "\n" + indent + schemeHelpHeader + "\n" + imageSchemeHelp
attestHelp = attestExample + attestSchemeHelp
)
func Attest(v *viper.Viper, app *config.Application, ro *options.RootOptions) *cobra.Command {
ao := options.AttestOptions{}
cmd := &cobra.Command{
Use: "attest --output [FORMAT] --key [KEY] [SOURCE]",
Short: "Generate a package SBOM as an attestation for the given [SOURCE] container image",
Long: "Generate a packaged-based Software Bill Of Materials (SBOM) from a container image as the predicate of an in-toto attestation",
Example: internal.Tprintf(attestHelp, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "attest",
}),
Args: func(cmd *cobra.Command, args []string) error {
// run to unmarshal viper object onto app config
// the viper object correctly
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
}
// configure logging for command
newLogWrapper(app)
logApplicationConfig(app)
return validateArgs(cmd, args)
},
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
// this MUST be called first to make sure app config decodes
// the viper object correctly
if app.CheckForAppUpdate {
checkForApplicationUpdate()
}
return attest.Run(cmd.Context(), app, args)
},
}
err := ao.AddFlags(cmd, v)
if err != nil {
log.Fatal(err)
}
return cmd
}
+216
View File
@@ -0,0 +1,216 @@
package attest
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os"
"strings"
"github.com/anchore/stereoscope"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/cmd/syft/cli/eventloop"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/cmd/syft/cli/packages"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/formats/cyclonedxjson"
"github.com/anchore/syft/internal/formats/spdx22json"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/in-toto/in-toto-golang/in_toto"
"github.com/pkg/errors"
"github.com/sigstore/cosign/cmd/cosign/cli/sign"
"github.com/sigstore/cosign/pkg/cosign/attestation"
"github.com/sigstore/sigstore/pkg/signature/dsse"
"github.com/wagoodman/go-partybus"
signatureoptions "github.com/sigstore/sigstore/pkg/signature/options"
)
var (
allowedAttestFormats = []sbom.FormatID{
syftjson.ID,
spdx22json.ID,
cyclonedxjson.ID,
}
intotoJSONDsseType = `application/vnd.in-toto+json`
)
func Run(ctx context.Context, app *config.Application, args []string) error {
// We cannot generate an attestation for more than one output
if len(app.Outputs) > 1 {
return fmt.Errorf("unable to generate attestation for more than one output")
}
// can only be an image for attestation or OCI DIR
userInput := args[0]
si, err := parseImageSource(userInput, app)
if err != nil {
return err
}
format := syft.FormatByName(app.Outputs[0])
predicateType := formatPredicateType(format)
if predicateType == "" {
return fmt.Errorf("could not produce attestation predicate for given format: %q. Available formats: %+v", options.FormatAliases(format.ID()), options.FormatAliases(allowedAttestFormats...))
}
passFunc, err := selectPassFunc(app.Attest.Key, app.Attest.Password)
if err != nil {
return err
}
ko := sign.KeyOpts{
KeyRef: app.Attest.Key,
PassFunc: passFunc,
}
sv, err := sign.SignerFromKeyOpts(ctx, "", "", ko)
if err != nil {
return err
}
defer sv.Close()
eventBus := partybus.NewBus()
stereoscope.SetBus(eventBus)
syft.SetBus(eventBus)
return eventloop.EventLoop(
execWorker(app, *si, format, predicateType, sv),
eventloop.SetupSignals(),
eventBus.Subscribe(),
stereoscope.Cleanup,
ui.Select(options.IsVerbose(app), app.Quiet)...,
)
}
func parseImageSource(userInput string, app *config.Application) (s *source.Input, err error) {
si, err := source.ParseInput(userInput, app.Platform, false)
if err != nil {
return nil, fmt.Errorf("could not generate source input for attest command: %w", err)
}
switch si.Scheme {
case source.ImageScheme, source.UnknownScheme:
// at this point we know that it cannot be dir: or file: schemes;
// we will assume that the unknown scheme could represent an image;
si.Scheme = source.ImageScheme
default:
return nil, fmt.Errorf("attest command can only be used with image sources but discovered %q when given %q", si.Scheme, userInput)
}
// if the original detection was from the local daemon we want to short circuit
// that and attempt to generate the image source from its current registry source instead
switch si.ImageSource {
case image.UnknownSource, image.OciRegistrySource:
si.ImageSource = image.OciRegistrySource
default:
return nil, fmt.Errorf("attest command can only be used with image sources fetch directly from the registry, but discovered an image source of %q when given %q", si.ImageSource, userInput)
}
return si, nil
}
func execWorker(app *config.Application, sourceInput source.Input, format sbom.Format, predicateType string, sv *sign.SignerVerifier) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
src, cleanup, err := source.NewFromRegistry(sourceInput, app.Registry.ToOptions(), app.Exclusions)
if cleanup != nil {
defer cleanup()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", sourceInput.UserInput, err)
return
}
s, err := packages.GenerateSBOM(src, errs, app)
if err != nil {
errs <- err
return
}
sbomBytes, err := syft.Encode(*s, format)
if err != nil {
errs <- err
return
}
err = generateAttestation(sbomBytes, src, sv, predicateType)
if err != nil {
errs <- err
return
}
}()
return errs
}
func generateAttestation(predicate []byte, src *source.Source, sv *sign.SignerVerifier, predicateType string) error {
switch len(src.Image.Metadata.RepoDigests) {
case 0:
return fmt.Errorf("cannot generate attestation since no repo digests were found; make sure you're passing an OCI registry source for the attest command")
case 1:
default:
return fmt.Errorf("cannot generate attestation since multiple repo digests were found for the image: %+v", src.Image.Metadata.RepoDigests)
}
wrapped := dsse.WrapSigner(sv, intotoJSONDsseType)
sh, err := attestation.GenerateStatement(attestation.GenerateOpts{
Predicate: bytes.NewBuffer(predicate),
Type: predicateType,
Digest: findValidDigest(src.Image.Metadata.RepoDigests),
})
if err != nil {
return err
}
payload, err := json.Marshal(sh)
if err != nil {
return err
}
signedPayload, err := wrapped.SignMessage(bytes.NewReader(payload), signatureoptions.WithContext(context.Background()))
if err != nil {
return errors.Wrap(err, "unable to sign SBOM")
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error {
_, err := os.Stdout.Write(signedPayload)
return err
},
})
return nil
}
func findValidDigest(digests []string) string {
// since we are only using the OCI repo provider for this source we are safe that this is only 1 value
// see https://github.com/anchore/stereoscope/blob/25ebd49a842b5ac0a20c2e2b4b81335b64ad248c/pkg/image/oci/registry_provider.go#L57-L63
split := strings.Split(digests[0], "sha256:")
return split[1]
}
func formatPredicateType(format sbom.Format) string {
switch format.ID() {
case spdx22json.ID:
return in_toto.PredicateSPDX
case cyclonedxjson.ID:
// Tentative see https://github.com/in-toto/attestation/issues/82
return "https://cyclonedx.org/bom"
case syftjson.ID:
return "https://syft.dev/bom"
default:
return ""
}
}
+56
View File
@@ -0,0 +1,56 @@
package attest
import (
"errors"
"fmt"
"io"
"os"
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/sigstore/cosign/pkg/cosign"
)
func selectPassFunc(keypath, password string) (cosign.PassFunc, error) {
keyContents, err := os.ReadFile(keypath)
if err != nil {
return nil, err
}
var fn cosign.PassFunc = func(bool) (b []byte, err error) { return nil, nil }
_, err = cosign.LoadPrivateKey(keyContents, nil)
if err != nil {
fn = func(bool) (b []byte, err error) {
return fetchPassword(password)
}
}
return fn, nil
}
func fetchPassword(password string) (b []byte, err error) {
potentiallyPipedInput, err := internal.IsPipedInput()
if err != nil {
log.Warnf("unable to determine if there is piped input: %+v", err)
}
switch {
case password != "":
return []byte(password), nil
case potentiallyPipedInput:
// handle piped in passwords
pwBytes, err := io.ReadAll(os.Stdin)
if err != nil {
return nil, fmt.Errorf("unable to get password from stdin: %w", err)
}
// be resilient to input that may have newline characters (in case someone is using echo without -n)
cleanPw := strings.TrimRight(string(pwBytes), "\n")
return []byte(cleanPw), nil
case internal.IsTerminal():
return cosign.GetPassFromTerm(false)
}
return nil, errors.New("no method available to fetch password")
}
+145
View File
@@ -0,0 +1,145 @@
package cli
import (
"fmt"
"strings"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal/logger"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/event"
"github.com/gookit/color"
"github.com/spf13/cobra"
"github.com/spf13/viper"
"github.com/wagoodman/go-partybus"
)
const indent = " "
// New constructs the `syft packages` command, aliases the root command to `syft packages`,
// and constructs the `syft power-user` and `syft attest` commands. It is also responsible for
// organizing flag usage and injecting the application config for each command.
// Because of how the `cobra` library behaves, the application's configuration is initialized
// at this level. Values from the config should only be used after `app.LoadAllValues` has been called.
// Cobra does not have knowledge of the user provided flags until the `RunE` block of each command.
// `RunE` is the earliest that the complete application configuration can be loaded.
func New() (*cobra.Command, error) {
app := &config.Application{}
// allow for nested options to be specified via environment variables
// e.g. pod.context = APPNAME_POD_CONTEXT
v := viper.NewWithOptions(viper.EnvKeyReplacer(strings.NewReplacer(".", "_", "-", "_")))
// since root is aliased as the packages cmd we need to construct this command first
// we also need the command to have information about the `root` options because of this alias
ro := &options.RootOptions{}
po := &options.PackagesOptions{}
packagesCmd := Packages(v, app, ro, po)
// root options are also passed to the attestCmd so that a user provided config location can be discovered
attestCmd := Attest(v, app, ro)
poweruserCmd := PowerUser(v, app, ro)
// rootCmd is currently an alias for the packages command
rootCmd := &cobra.Command{
Short: packagesCmd.Short,
Long: packagesCmd.Long,
Args: packagesCmd.Args,
Example: packagesCmd.Example,
SilenceUsage: true,
SilenceErrors: true,
RunE: packagesCmd.RunE,
Version: version.FromBuild().Version,
}
rootCmd.SetVersionTemplate(fmt.Sprintf("%s {{.Version}}\n", internal.ApplicationName))
// start adding flags to all the commands
err := ro.AddFlags(rootCmd, v)
if err != nil {
return nil, err
}
// package flags need to be decorated onto the rootCmd so that rootCmd can function as a packages alias
err = po.AddFlags(rootCmd, v)
if err != nil {
return nil, err
}
// attest also uses flags from the packagesCmd since it generates an sbom
err = po.AddFlags(attestCmd, v)
if err != nil {
return nil, err
}
// poweruser also uses the packagesCmd flags since it is a specialized version of the command
err = po.AddFlags(poweruserCmd, v)
if err != nil {
return nil, err
}
// Add sub-commands.
rootCmd.AddCommand(packagesCmd)
rootCmd.AddCommand(attestCmd)
rootCmd.AddCommand(poweruserCmd)
rootCmd.AddCommand(Completion())
rootCmd.AddCommand(Version(v, app))
return rootCmd, err
}
func validateArgs(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
// in the case that no arguments are given we want to show the help text and return with a non-0 return code.
if err := cmd.Help(); err != nil {
return fmt.Errorf("unable to display help: %w", err)
}
return fmt.Errorf("an image/directory argument is required")
}
return cobra.MaximumNArgs(1)(cmd, args)
}
func checkForApplicationUpdate() {
log.Debugf("checking if new vesion of %s is available", internal.ApplicationName)
isAvailable, newVersion, err := version.IsUpdateAvailable()
if err != nil {
// this should never stop the application
log.Errorf(err.Error())
}
if isAvailable {
log.Infof("new version of %s is available: %s (current version is %s)", internal.ApplicationName, newVersion, version.FromBuild().Version)
bus.Publish(partybus.Event{
Type: event.AppUpdateAvailable,
Value: newVersion,
})
} else {
log.Debugf("no new %s update available", internal.ApplicationName)
}
}
func logApplicationConfig(app *config.Application) {
versionInfo := version.FromBuild()
log.Infof("syft version: %+v", versionInfo.Version)
log.Debugf("application config:\n%+v", color.Magenta.Sprint(app.String()))
}
func newLogWrapper(app *config.Application) {
cfg := logger.LogrusConfig{
EnableConsole: (app.Log.FileLocation == "" || app.Verbosity > 0) && !app.Quiet,
EnableFile: app.Log.FileLocation != "",
Level: app.Log.LevelOpt,
Structured: app.Log.Structured,
FileLocation: app.Log.FileLocation,
}
logWrapper := logger.NewLogrusLogger(cfg)
syft.SetLogger(logWrapper)
stereoscope.SetLogger(&logger.LogrusNestedLogger{
Logger: logWrapper.Logger.WithField("from-lib", "stereoscope"),
})
}
+54
View File
@@ -0,0 +1,54 @@
package cli
import (
"os"
"github.com/spf13/cobra"
)
func Completion() *cobra.Command {
cmd := &cobra.Command{
Use: "completion [bash|zsh|fish]",
Short: "Generate a shell completion for Syft (listing local docker images)",
Long: `To load completions (docker image list):
Bash:
$ source <(syft completion bash)
# To load completions for each session, execute once:
Linux:
$ syft completion bash > /etc/bash_completion.d/syft
MacOS:
$ syft completion bash > /usr/local/etc/bash_completion.d/syft
Zsh:
# If shell completion is not already enabled in your environment you will need
# to enable it. You can execute the following once:
$ echo "autoload -U compinit; compinit" >> ~/.zshrc
# To load completions for each session, execute once:
$ syft completion zsh > "${fpath[1]}/_syft"
# You will need to start a new shell for this setup to take effect.
Fish:
$ syft completion fish | source
# To load completions for each session, execute once:
$ syft completion fish > ~/.config/fish/completions/syft.fish
`,
DisableFlagsInUseLine: true,
ValidArgs: []string{"bash", "zsh", "fish"},
Args: cobra.ExactValidArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
var err error
switch args[0] {
case "bash":
err = cmd.Root().GenBashCompletion(os.Stdout)
case "zsh":
err = cmd.Root().GenZshCompletion(os.Stdout)
case "fish":
err = cmd.Root().GenFishCompletion(os.Stdout, true)
}
if err != nil {
return err
}
return nil
},
}
return cmd
}
@@ -1,4 +1,4 @@
package cmd
package eventloop
import (
"errors"
@@ -15,7 +15,7 @@ import (
// signal interrupts. Is responsible for handling each event relative to a given UI an to coordinate eventing until
// an eventual graceful exit.
// nolint:funlen
func eventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *partybus.Subscription, cleanupFn func(), uxs ...ui.UI) error {
func EventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *partybus.Subscription, cleanupFn func(), uxs ...ui.UI) error {
defer cleanupFn()
events := subscription.Events()
var err error
@@ -1,4 +1,4 @@
package cmd
package eventloop
import (
"fmt"
@@ -42,7 +42,7 @@ func (u *uiMock) Teardown(_ bool) error {
return u.Called().Error(0)
}
func Test_eventLoop_gracefulExit(t *testing.T) {
func Test_EventLoop_gracefulExit(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -92,7 +92,7 @@ func Test_eventLoop_gracefulExit(t *testing.T) {
}
assert.NoError(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -109,7 +109,7 @@ func Test_eventLoop_gracefulExit(t *testing.T) {
testWithTimeout(t, 5*time.Second, test)
}
func Test_eventLoop_workerError(t *testing.T) {
func Test_EventLoop_workerError(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -155,7 +155,7 @@ func Test_eventLoop_workerError(t *testing.T) {
// ensure we see an error returned
assert.ErrorIs(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -174,7 +174,7 @@ func Test_eventLoop_workerError(t *testing.T) {
testWithTimeout(t, 5*time.Second, test)
}
func Test_eventLoop_unsubscribeError(t *testing.T) {
func Test_EventLoop_unsubscribeError(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -226,7 +226,7 @@ func Test_eventLoop_unsubscribeError(t *testing.T) {
// unsubscribe errors should be handled and ignored, not propagated. We are additionally asserting that
// this case is handled as a controlled shutdown (this test should not timeout)
assert.NoError(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -243,7 +243,7 @@ func Test_eventLoop_unsubscribeError(t *testing.T) {
testWithTimeout(t, 5*time.Second, test)
}
func Test_eventLoop_handlerError(t *testing.T) {
func Test_EventLoop_handlerError(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -296,7 +296,7 @@ func Test_eventLoop_handlerError(t *testing.T) {
// handle errors SHOULD propagate the event loop. We are additionally asserting that this case is
// handled as a controlled shutdown (this test should not timeout)
assert.ErrorIs(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -315,7 +315,7 @@ func Test_eventLoop_handlerError(t *testing.T) {
testWithTimeout(t, 5*time.Second, test)
}
func Test_eventLoop_signalsStopExecution(t *testing.T) {
func Test_EventLoop_signalsStopExecution(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -351,7 +351,7 @@ func Test_eventLoop_signalsStopExecution(t *testing.T) {
}
assert.NoError(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -368,7 +368,7 @@ func Test_eventLoop_signalsStopExecution(t *testing.T) {
testWithTimeout(t, 5*time.Second, test)
}
func Test_eventLoop_uiTeardownError(t *testing.T) {
func Test_EventLoop_uiTeardownError(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -421,7 +421,7 @@ func Test_eventLoop_uiTeardownError(t *testing.T) {
// ensure we see an error returned
assert.ErrorIs(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -1,4 +1,4 @@
package cmd
package eventloop
import (
"os"
@@ -6,7 +6,7 @@ import (
"syscall"
)
func setupSignals() <-chan os.Signal {
func SetupSignals() <-chan os.Signal {
c := make(chan os.Signal, 1) // Note: A buffered channel is recommended for this; see https://golang.org/pkg/os/signal/#Notify
interruptions := []os.Signal{
+232
View File
@@ -0,0 +1,232 @@
package eventloop
import (
"crypto"
"fmt"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
type Task func(*sbom.Artifacts, *source.Source) ([]artifact.Relationship, error)
func Tasks(app *config.Application) ([]Task, error) {
var tasks []Task
generators := []func(app *config.Application) (Task, error){
generateCatalogPackagesTask,
generateCatalogFileMetadataTask,
generateCatalogFileDigestsTask,
generateCatalogSecretsTask,
generateCatalogFileClassificationsTask,
generateCatalogContentsTask,
}
for _, generator := range generators {
task, err := generator(app)
if err != nil {
return nil, err
}
if task != nil {
tasks = append(tasks, task)
}
}
return tasks, nil
}
func generateCatalogPackagesTask(app *config.Application) (Task, error) {
if !app.Package.Cataloger.Enabled {
return nil, nil
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
packageCatalog, relationships, theDistro, err := syft.CatalogPackages(src, app.Package.ToConfig())
if err != nil {
return nil, err
}
results.PackageCatalog = packageCatalog
results.LinuxDistribution = theDistro
return relationships, nil
}
return task, nil
}
func generateCatalogFileMetadataTask(app *config.Application) (Task, error) {
if !app.FileMetadata.Cataloger.Enabled {
return nil, nil
}
metadataCataloger := file.NewMetadataCataloger()
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(app.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := metadataCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.FileMetadata = result
return nil, nil
}
return task, nil
}
func generateCatalogFileDigestsTask(app *config.Application) (Task, error) {
if !app.FileMetadata.Cataloger.Enabled {
return nil, nil
}
supportedHashAlgorithms := make(map[string]crypto.Hash)
for _, h := range []crypto.Hash{
crypto.MD5,
crypto.SHA1,
crypto.SHA256,
} {
supportedHashAlgorithms[file.DigestAlgorithmName(h)] = h
}
var hashes []crypto.Hash
for _, hashStr := range app.FileMetadata.Digests {
name := file.CleanDigestAlgorithmName(hashStr)
hashObj, ok := supportedHashAlgorithms[name]
if !ok {
return nil, fmt.Errorf("unsupported hash algorithm: %s", hashStr)
}
hashes = append(hashes, hashObj)
}
digestsCataloger, err := file.NewDigestsCataloger(hashes)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(app.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := digestsCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.FileDigests = result
return nil, nil
}
return task, nil
}
func generateCatalogSecretsTask(app *config.Application) (Task, error) {
if !app.Secrets.Cataloger.Enabled {
return nil, nil
}
patterns, err := file.GenerateSearchPatterns(file.DefaultSecretsPatterns, app.Secrets.AdditionalPatterns, app.Secrets.ExcludePatternNames)
if err != nil {
return nil, err
}
secretsCataloger, err := file.NewSecretsCataloger(patterns, app.Secrets.RevealValues, app.Secrets.SkipFilesAboveSize)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(app.Secrets.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := secretsCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.Secrets = result
return nil, nil
}
return task, nil
}
func generateCatalogFileClassificationsTask(app *config.Application) (Task, error) {
if !app.FileClassification.Cataloger.Enabled {
return nil, nil
}
// TODO: in the future we could expose out the classifiers via configuration
classifierCataloger, err := file.NewClassificationCataloger(file.DefaultClassifiers)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(app.FileClassification.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := classifierCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.FileClassifications = result
return nil, nil
}
return task, nil
}
func generateCatalogContentsTask(app *config.Application) (Task, error) {
if !app.FileContents.Cataloger.Enabled {
return nil, nil
}
contentsCataloger, err := file.NewContentsCataloger(app.FileContents.Globs, app.FileContents.SkipFilesAboveSize)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(app.FileContents.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := contentsCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.FileContents = result
return nil, nil
}
return task, nil
}
func RunTask(t Task, a *sbom.Artifacts, src *source.Source, c chan<- artifact.Relationship, errs chan<- error) {
defer close(c)
relationships, err := t(a, src)
if err != nil {
errs <- err
return
}
for _, relationship := range relationships {
c <- relationship
}
}
+28
View File
@@ -0,0 +1,28 @@
package options
import (
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
)
type AttestOptions struct {
Key string
}
var _ Interface = (*AttestOptions)(nil)
func (o *AttestOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.PersistentFlags().StringVarP(&o.Key, "key", "", "cosign.key",
"path to the private key file to use for attestation")
return bindAttestationConfigOptions(cmd.PersistentFlags(), v)
}
func bindAttestationConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
if err := v.BindPFlag("attest.key", flags.Lookup("key")); err != nil {
return err
}
return nil
}
@@ -1,11 +1,11 @@
package cmd
package options
import (
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/sbom"
)
func formatAliases(ids ...sbom.FormatID) (aliases []string) {
func FormatAliases(ids ...sbom.FormatID) (aliases []string) {
for _, id := range ids {
switch id {
case syft.JSONFormatID:
@@ -22,6 +22,8 @@ func formatAliases(ids ...sbom.FormatID) (aliases []string) {
aliases = append(aliases, "cyclonedx-xml")
case syft.CycloneDxJSONFormatID:
aliases = append(aliases, "cyclonedx-json")
case syft.GitHubID:
aliases = append(aliases, "github", "github-json")
default:
aliases = append(aliases, string(id))
}
+11
View File
@@ -0,0 +1,11 @@
package options
import (
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
type Interface interface {
// AddFlags adds this options' flags to the cobra command.
AddFlags(cmd *cobra.Command, v *viper.Viper) error
}
+118
View File
@@ -0,0 +1,118 @@
package options
import (
"fmt"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/pkg/cataloger"
"github.com/anchore/syft/syft/source"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
)
type PackagesOptions struct {
Scope string
Output []string
File string
Platform string
Host string
Username string
Password string
Dockerfile string
Exclude []string
OverwriteExistingImage bool
ImportTimeout uint
}
var _ Interface = (*PackagesOptions)(nil)
func (o *PackagesOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.PersistentFlags().StringVarP(&o.Scope, "scope", "s", cataloger.DefaultSearchConfig().Scope.String(),
fmt.Sprintf("selection of layers to catalog, options=%v", source.AllScopes))
cmd.PersistentFlags().StringArrayVarP(&o.Output, "output", "o", FormatAliases(table.ID),
fmt.Sprintf("report output format, options=%v", FormatAliases(syft.FormatIDs()...)))
cmd.PersistentFlags().StringVarP(&o.File, "file", "", "",
"file to write the default report output to (default is STDOUT)")
cmd.PersistentFlags().StringVarP(&o.Platform, "platform", "", "",
"an optional platform specifier for container image sources (e.g. 'linux/arm64', 'linux/arm64/v8', 'arm64', 'linux')")
cmd.PersistentFlags().StringVarP(&o.Host, "host", "H", "",
"the hostname or URL of the Anchore Enterprise instance to upload to")
cmd.PersistentFlags().StringVarP(&o.Username, "username", "u", "",
"the username to authenticate against Anchore Enterprise")
cmd.PersistentFlags().StringVarP(&o.Password, "password", "p", "",
"the password to authenticate against Anchore Enterprise")
cmd.PersistentFlags().StringVarP(&o.Dockerfile, "dockerfile", "d", "",
"include dockerfile for upload to Anchore Enterprise")
cmd.PersistentFlags().StringArrayVarP(&o.Exclude, "exclude", "", nil,
"exclude paths from being scanned using a glob expression")
cmd.PersistentFlags().BoolVarP(&o.OverwriteExistingImage, "overwrite-existing-image", "", false,
"overwrite an existing image during the upload to Anchore Enterprise")
cmd.PersistentFlags().UintVarP(&o.ImportTimeout, "import-timeout", "", 30,
"set a timeout duration (in seconds) for the upload to Anchore Enterprise")
return bindPackageConfigOptions(cmd.PersistentFlags(), v)
}
func bindPackageConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
// Formatting & Input options //////////////////////////////////////////////
if err := v.BindPFlag("package.cataloger.scope", flags.Lookup("scope")); err != nil {
return err
}
if err := v.BindPFlag("file", flags.Lookup("file")); err != nil {
return err
}
if err := v.BindPFlag("exclude", flags.Lookup("exclude")); err != nil {
return err
}
if err := v.BindPFlag("output", flags.Lookup("output")); err != nil {
return err
}
if err := v.BindPFlag("platform", flags.Lookup("platform")); err != nil {
return err
}
// Upload options //////////////////////////////////////////////////////////
if err := v.BindPFlag("anchore.host", flags.Lookup("host")); err != nil {
return err
}
if err := v.BindPFlag("anchore.username", flags.Lookup("username")); err != nil {
return err
}
if err := v.BindPFlag("anchore.password", flags.Lookup("password")); err != nil {
return err
}
if err := v.BindPFlag("anchore.dockerfile", flags.Lookup("dockerfile")); err != nil {
return err
}
if err := v.BindPFlag("anchore.overwrite-existing-image", flags.Lookup("overwrite-existing-image")); err != nil {
return err
}
if err := v.BindPFlag("anchore.import-timeout", flags.Lookup("import-timeout")); err != nil {
return err
}
return nil
}
+36
View File
@@ -0,0 +1,36 @@
package options
import (
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
)
type RootOptions struct {
Config string
Quiet bool
Verbose int
}
var _ Interface = (*RootOptions)(nil)
func (o *RootOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.PersistentFlags().StringVarP(&o.Config, "config", "c", "", "application config file")
cmd.PersistentFlags().CountVarP(&o.Verbose, "verbose", "v", "increase verbosity (-v = info, -vv = debug)")
cmd.PersistentFlags().BoolVarP(&o.Quiet, "quiet", "q", false, "suppress all logging output")
return bindRootConfigOptions(cmd.PersistentFlags(), v)
}
func bindRootConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
if err := v.BindPFlag("config", flags.Lookup("config")); err != nil {
return err
}
if err := v.BindPFlag("verbosity", flags.Lookup("verbose")); err != nil {
return err
}
if err := v.BindPFlag("quiet", flags.Lookup("quiet")); err != nil {
return err
}
return nil
}
+18
View File
@@ -0,0 +1,18 @@
package options
import (
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
)
func IsVerbose(app *config.Application) (result bool) {
isPipedInput, err := internal.IsPipedInput()
if err != nil {
// since we can't tell if there was piped input we assume that there could be to disable the ETUI
log.Warnf("unable to determine if there is piped input: %+v", err)
return true
}
// verbosity should consider if there is piped input (in which case we should not show the ETUI)
return app.Verbosity > 0 || isPipedInput
}
+17
View File
@@ -0,0 +1,17 @@
package options
import (
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
type VersionOptions struct {
Output string
}
var _ Interface = (*PackagesOptions)(nil)
func (o *VersionOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.Flags().StringVarP(&o.Output, "output", "o", "text", "format to show version information (available=[text, json])")
return nil
}
+79
View File
@@ -0,0 +1,79 @@
package cli
import (
"fmt"
"log"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/cmd/syft/cli/packages"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
const (
packagesExample = ` {{.appName}} {{.command}} alpine:latest a summary of discovered packages
{{.appName}} {{.command}} alpine:latest -o json show all possible cataloging details
{{.appName}} {{.command}} alpine:latest -o cyclonedx show a CycloneDX formatted SBOM
{{.appName}} {{.command}} alpine:latest -o cyclonedx-json show a CycloneDX JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx show a SPDX 2.2 Tag-Value formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx-json show a SPDX 2.2 JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -vv show verbose debug information
Supports the following image sources:
{{.appName}} {{.command}} yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} path/to/a/file/or/dir a Docker tar, OCI tar, OCI directory, or generic filesystem directory
`
schemeHelpHeader = "You can also explicitly specify the scheme to use:"
imageSchemeHelp = ` {{.appName}} {{.command}} docker:yourrepo/yourimage:tag explicitly use the Docker daemon
{{.appName}} {{.command}} podman:yourrepo/yourimage:tag explicitly use the Podman daemon
{{.appName}} {{.command}} registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
{{.appName}} {{.command}} docker-archive:path/to/yourimage.tar use a tarball from disk for archives created from "docker save"
{{.appName}} {{.command}} oci-archive:path/to/yourimage.tar use a tarball from disk for OCI archives (from Skopeo or otherwise)
{{.appName}} {{.command}} oci-dir:path/to/yourimage read directly from a path on disk for OCI layout directories (from Skopeo or otherwise)
`
nonImageSchemeHelp = ` {{.appName}} {{.command}} dir:path/to/yourproject read directly from a path on disk (any directory)
{{.appName}} {{.command}} file:path/to/yourproject/file read directly from a path on disk (any single file)
`
packagesSchemeHelp = "\n" + indent + schemeHelpHeader + "\n" + imageSchemeHelp + nonImageSchemeHelp
packagesHelp = packagesExample + packagesSchemeHelp
)
func Packages(v *viper.Viper, app *config.Application, ro *options.RootOptions, po *options.PackagesOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "packages [SOURCE]",
Short: "Generate a package SBOM",
Long: "Generate a packaged-based Software Bill Of Materials (SBOM) from container images and filesystems",
Example: internal.Tprintf(packagesHelp, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "packages",
}),
Args: func(cmd *cobra.Command, args []string) error {
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
}
// configure logging for command
newLogWrapper(app)
logApplicationConfig(app)
return validateArgs(cmd, args)
},
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
if app.CheckForAppUpdate {
checkForApplicationUpdate()
}
return packages.Run(cmd.Context(), app, args)
},
}
err := po.AddFlags(cmd, v)
if err != nil {
log.Fatal(err)
}
return cmd
}
+240
View File
@@ -0,0 +1,240 @@
package packages
import (
"context"
"fmt"
"io/ioutil"
"os"
"strings"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/cmd/syft/cli/eventloop"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/anchore"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/hashicorp/go-multierror"
"github.com/wagoodman/go-partybus"
)
func Run(ctx context.Context, app *config.Application, args []string) error {
writer, err := makeWriter(app.Outputs, app.File)
if err != nil {
return err
}
defer func() {
if err := writer.Close(); err != nil {
log.Warnf("unable to write to report destination: %w", err)
}
}()
// could be an image or a directory, with or without a scheme
userInput := args[0]
si, err := source.ParseInput(userInput, app.Platform, true)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
eventBus := partybus.NewBus()
stereoscope.SetBus(eventBus)
syft.SetBus(eventBus)
return eventloop.EventLoop(
execWorker(app, *si, writer),
eventloop.SetupSignals(),
eventBus.Subscribe(),
stereoscope.Cleanup,
ui.Select(options.IsVerbose(app), app.Quiet)...,
)
}
func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
src, cleanup, err := source.New(si, app.Registry.ToOptions(), app.Exclusions)
if cleanup != nil {
defer cleanup()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", si.UserInput, err)
return
}
s, err := GenerateSBOM(src, errs, app)
if err != nil {
errs <- err
return
}
if s == nil {
errs <- fmt.Errorf("no SBOM produced for %q", si.UserInput)
}
if app.Anchore.Host != "" {
if err := runPackageSbomUpload(src, *s, app); err != nil {
errs <- err
return
}
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error { return writer.Write(*s) },
})
}()
return errs
}
func GenerateSBOM(src *source.Source, errs chan error, app *config.Application) (*sbom.SBOM, error) {
tasks, err := eventloop.Tasks(app)
if err != nil {
return nil, err
}
s := sbom.SBOM{
Source: src.Metadata,
Descriptor: sbom.Descriptor{
Name: internal.ApplicationName,
Version: version.FromBuild().Version,
Configuration: app,
},
}
buildRelationships(&s, src, tasks, errs)
return &s, nil
}
func buildRelationships(s *sbom.SBOM, src *source.Source, tasks []eventloop.Task, errs chan error) {
var relationships []<-chan artifact.Relationship
for _, task := range tasks {
c := make(chan artifact.Relationship)
relationships = append(relationships, c)
go eventloop.RunTask(task, &s.Artifacts, src, c, errs)
}
s.Relationships = append(s.Relationships, MergeRelationships(relationships...)...)
}
func MergeRelationships(cs ...<-chan artifact.Relationship) (relationships []artifact.Relationship) {
for _, c := range cs {
for n := range c {
relationships = append(relationships, n)
}
}
return relationships
}
func runPackageSbomUpload(src *source.Source, s sbom.SBOM, app *config.Application) error {
log.Infof("uploading results to %s", app.Anchore.Host)
if src.Metadata.Scheme != source.ImageScheme {
return fmt.Errorf("unable to upload results: only images are supported")
}
var dockerfileContents []byte
if app.Anchore.Dockerfile != "" {
if _, err := os.Stat(app.Anchore.Dockerfile); os.IsNotExist(err) {
return fmt.Errorf("unable dockerfile=%q does not exist: %w", app.Anchore.Dockerfile, err)
}
fh, err := os.Open(app.Anchore.Dockerfile)
if err != nil {
return fmt.Errorf("unable to open dockerfile=%q: %w", app.Anchore.Dockerfile, err)
}
dockerfileContents, err = ioutil.ReadAll(fh)
if err != nil {
return fmt.Errorf("unable to read dockerfile=%q: %w", app.Anchore.Dockerfile, err)
}
}
c, err := anchore.NewClient(anchore.Configuration{
BaseURL: app.Anchore.Host,
Username: app.Anchore.Username,
Password: app.Anchore.Password,
})
if err != nil {
return fmt.Errorf("failed to create anchore client: %w", err)
}
importCfg := anchore.ImportConfig{
ImageMetadata: src.Image.Metadata,
SBOM: s,
Dockerfile: dockerfileContents,
OverwriteExistingUpload: app.Anchore.OverwriteExistingImage,
Timeout: app.Anchore.ImportTimeout,
}
if err := c.Import(context.Background(), importCfg); err != nil {
return fmt.Errorf("failed to upload results to host=%s: %+v", app.Anchore.Host, err)
}
return nil
}
// makeWriter creates a sbom.Writer for output or returns an error. this will either return a valid writer
// or an error but neither both and if there is no error, sbom.Writer.Close() should be called
func makeWriter(outputs []string, defaultFile string) (sbom.Writer, error) {
outputOptions, err := parseOptions(outputs, defaultFile)
if err != nil {
return nil, err
}
writer, err := sbom.NewWriter(outputOptions...)
if err != nil {
return nil, err
}
return writer, nil
}
// parseOptions utility to parse command-line option strings and retain the existing behavior of default format and file
func parseOptions(outputs []string, defaultFile string) (out []sbom.WriterOption, errs error) {
// always should have one option -- we generally get the default of "table", but just make sure
if len(outputs) == 0 {
outputs = append(outputs, string(table.ID))
}
for _, name := range outputs {
name = strings.TrimSpace(name)
// split to at most two parts for <format>=<file>
parts := strings.SplitN(name, "=", 2)
// the format name is the first part
name = parts[0]
// default to the --file or empty string if not specified
file := defaultFile
// If a file is specified as part of the output formatName, use that
if len(parts) > 1 {
file = parts[1]
}
format := syft.FormatByName(name)
if format == nil {
errs = multierror.Append(errs, fmt.Errorf("bad output format: '%s'", name))
continue
}
out = append(out, sbom.NewWriterOption(format, file))
}
return out, errs
}
+49
View File
@@ -0,0 +1,49 @@
package cli
import (
"fmt"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/cmd/syft/cli/poweruser"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
const powerUserExample = ` {{.appName}} {{.command}} <image>
DEPRECATED - THIS COMMAND WILL BE REMOVED in v1.0.0
Only image sources are supported (e.g. docker: , podman: , docker-archive: , oci: , etc.), the directory source (dir:) is not supported.
All behavior is controlled via application configuration and environment variables (see https://github.com/anchore/syft#configuration)
`
func PowerUser(v *viper.Viper, app *config.Application, ro *options.RootOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "power-user [IMAGE]",
Short: "Run bulk operations on container images",
Example: internal.Tprintf(powerUserExample, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "power-user",
}),
Args: func(cmd *cobra.Command, args []string) error {
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
}
// configure logging for command
newLogWrapper(app)
logApplicationConfig(app)
return validateArgs(cmd, args)
},
Hidden: true,
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
if app.CheckForAppUpdate {
checkForApplicationUpdate()
}
return poweruser.Run(cmd.Context(), app, args)
},
}
return cmd
}
+116
View File
@@ -0,0 +1,116 @@
package poweruser
import (
"context"
"fmt"
"os"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/cmd/syft/cli/eventloop"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/cmd/syft/cli/packages"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/gookit/color"
"github.com/wagoodman/go-partybus"
)
func Run(ctx context.Context, app *config.Application, args []string) error {
writer, err := sbom.NewWriter(sbom.WriterOption{
Format: syftjson.Format(),
Path: app.File,
})
if err != nil {
return err
}
defer func() {
if err := writer.Close(); err != nil {
log.Warnf("unable to write to report destination: %+v", err)
}
// inform user at end of run that command will be removed
deprecated := color.Style{color.Red, color.OpBold}.Sprint("DEPRECATED: This command will be removed in v1.0.0")
fmt.Fprintln(os.Stderr, deprecated)
}()
userInput := args[0]
si, err := source.ParseInput(userInput, app.Platform, true)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
eventBus := partybus.NewBus()
stereoscope.SetBus(eventBus)
syft.SetBus(eventBus)
return eventloop.EventLoop(
execWorker(app, *si, writer),
eventloop.SetupSignals(),
eventBus.Subscribe(),
stereoscope.Cleanup,
ui.Select(options.IsVerbose(app), app.Quiet)...,
)
}
func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
app.Secrets.Cataloger.Enabled = true
app.FileMetadata.Cataloger.Enabled = true
app.FileContents.Cataloger.Enabled = true
app.FileClassification.Cataloger.Enabled = true
tasks, err := eventloop.Tasks(app)
if err != nil {
errs <- err
return
}
src, cleanup, err := source.New(si, app.Registry.ToOptions(), app.Exclusions)
if err != nil {
errs <- err
return
}
if cleanup != nil {
defer cleanup()
}
s := sbom.SBOM{
Source: src.Metadata,
Descriptor: sbom.Descriptor{
Name: internal.ApplicationName,
Version: version.FromBuild().Version,
Configuration: app,
},
}
var relationships []<-chan artifact.Relationship
for _, task := range tasks {
c := make(chan artifact.Relationship)
relationships = append(relationships, c)
go eventloop.RunTask(task, &s.Artifacts, src, c, errs)
}
s.Relationships = append(s.Relationships, packages.MergeRelationships(relationships...)...)
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error { return writer.Write(s) },
})
}()
return errs
}
+25 -15
View File
@@ -1,33 +1,41 @@
package cmd
package cli
import (
"encoding/json"
"fmt"
"log"
"os"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/version"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
var versionCmdOutputFormat string
func Version(v *viper.Viper, app *config.Application) *cobra.Command {
o := &options.VersionOptions{}
cmd := &cobra.Command{
Use: "version",
Short: "show the version",
RunE: func(cmd *cobra.Command, args []string) error {
return printVersion(o.Output)
},
}
var versionCmd = &cobra.Command{
Use: "version",
Short: "show the version",
Run: printVersion,
err := o.AddFlags(cmd, v)
if err != nil {
log.Fatal(err)
}
return cmd
}
func init() {
versionCmd.Flags().StringVarP(&versionCmdOutputFormat, "output", "o", "text", "format to show version information (available=[text, json])")
rootCmd.AddCommand(versionCmd)
}
func printVersion(_ *cobra.Command, _ []string) {
func printVersion(output string) error {
versionInfo := version.FromBuild()
switch versionCmdOutputFormat {
switch output {
case "text":
fmt.Println("Application: ", internal.ApplicationName)
fmt.Println("Version: ", versionInfo.Version)
@@ -55,7 +63,9 @@ func printVersion(_ *cobra.Command, _ []string) {
os.Exit(1)
}
default:
fmt.Printf("unsupported output format: %s\n", versionCmdOutputFormat)
fmt.Printf("unsupported output format: %s\n", output)
os.Exit(1)
}
return nil
}
+18
View File
@@ -0,0 +1,18 @@
package main
import (
"log"
"github.com/anchore/syft/cmd/syft/cli"
)
func main() {
cli, err := cli.New()
if err != nil {
log.Fatalf("error during command construction: %v", err)
}
if err := cli.Execute(); err != nil {
log.Fatalf("error during command execution: %v", err)
}
}
+64 -60
View File
@@ -1,19 +1,20 @@
module github.com/anchore/syft
go 1.17
go 1.18
require (
github.com/CycloneDX/cyclonedx-go v0.5.0
github.com/CycloneDX/cyclonedx-go v0.5.2
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
github.com/acobaugh/osrelease v0.1.0
github.com/adrg/xdg v0.2.1
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29
github.com/anchore/stereoscope v0.0.0-20220307154759-8a5a70c227d3
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b
github.com/anchore/stereoscope v0.0.0-20220406160859-c03a18a6b270
github.com/antihax/optional v1.0.0
github.com/bmatcuk/doublestar/v4 v4.0.2
github.com/dustin/go-humanize v1.0.0
@@ -23,7 +24,6 @@ require (
github.com/google/uuid v1.3.0
github.com/gookit/color v1.4.2
github.com/hashicorp/go-multierror v1.1.1
github.com/in-toto/in-toto-golang v0.3.4-0.20211211042327-af1f9fb822bf
github.com/jinzhu/copier v0.3.2
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/mholt/archiver/v3 v3.5.1
@@ -32,44 +32,45 @@ require (
github.com/mitchellh/mapstructure v1.4.3
github.com/olekukonko/tablewriter v0.0.5
github.com/pelletier/go-toml v1.9.4
github.com/pkg/profile v1.5.0
github.com/scylladb/go-set v1.0.2
// pinned to pull in 386 arch fix: https://github.com/scylladb/go-set/commit/cc7b2070d91ebf40d233207b633e28f5bd8f03a5
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e
github.com/sergi/go-diff v1.2.0
github.com/sigstore/sigstore v1.1.1-0.20220217212907-e48ca03a5ba7
github.com/sirupsen/logrus v1.8.1
github.com/spdx/tools-golang v0.2.0
github.com/spf13/afero v1.8.0
github.com/spf13/cobra v1.3.0
github.com/spf13/cobra v1.4.0
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.10.1
github.com/stretchr/testify v1.7.0
github.com/stretchr/testify v1.7.1
github.com/vifraa/gopom v0.1.0
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb
github.com/x-cray/logrus-prefixed-formatter v0.5.2
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/mod v0.5.1
golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd
golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3
golang.org/x/net v0.0.0-20220325170049-de3da57026de
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
gopkg.in/yaml.v2 v2.4.0
)
require (
github.com/docker/docker v20.10.12+incompatible
github.com/sigstore/cosign v1.6.0
github.com/in-toto/in-toto-golang v0.3.4-0.20211211042327-af1f9fb822bf
github.com/sigstore/cosign v1.7.2
github.com/sigstore/sigstore v1.2.1-0.20220401110139-0e610e39782f
)
require (
bitbucket.org/creachadair/shell v0.0.6 // indirect
cloud.google.com/go v0.100.2 // indirect
cloud.google.com/go/compute v1.3.0 // indirect
cloud.google.com/go/iam v0.1.1 // indirect
cloud.google.com/go/kms v1.3.0 // indirect
cloud.google.com/go/storage v1.21.0 // indirect
github.com/Azure/azure-sdk-for-go v61.5.0+incompatible // indirect
cloud.google.com/go/compute v1.5.0 // indirect
cloud.google.com/go/iam v0.3.0 // indirect
cloud.google.com/go/kms v1.4.0 // indirect
cloud.google.com/go/storage v1.22.0 // indirect
github.com/Azure/azure-sdk-for-go v63.0.0+incompatible // indirect
github.com/Azure/go-autorest v14.2.0+incompatible // indirect
github.com/Azure/go-autorest/autorest v0.11.24 // indirect
github.com/Azure/go-autorest/autorest v0.11.25 // indirect
github.com/Azure/go-autorest/autorest/adal v0.9.18 // indirect
github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 // indirect
github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 // indirect
@@ -87,21 +88,21 @@ require (
github.com/armon/go-metrics v0.3.10 // indirect
github.com/armon/go-radix v1.0.0 // indirect
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
github.com/aws/aws-sdk-go v1.43.0 // indirect
github.com/aws/aws-sdk-go-v2 v1.13.0 // indirect
github.com/aws/aws-sdk-go-v2/config v1.13.1 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.8.0 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.10.0 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.4 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.2.0 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.5 // indirect
github.com/aws/aws-sdk-go-v2/service/ecr v1.14.0 // indirect
github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.11.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.7.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.9.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.14.0 // indirect
github.com/aws/smithy-go v1.10.0 // indirect
github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20220216180153-3d7835abdf40 // indirect
github.com/aws/aws-sdk-go v1.43.30 // indirect
github.com/aws/aws-sdk-go-v2 v1.14.0 // indirect
github.com/aws/aws-sdk-go-v2/config v1.14.0 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.9.0 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.11.0 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.5 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.3.0 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.6 // indirect
github.com/aws/aws-sdk-go-v2/service/ecr v1.15.0 // indirect
github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.12.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.8.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.10.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.15.0 // indirect
github.com/aws/smithy-go v1.11.0 // indirect
github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20220228164355-396b2034c795 // indirect
github.com/benbjohnson/clock v1.1.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bgentry/speakeasy v0.1.0 // indirect
@@ -132,7 +133,7 @@ require (
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/jsonreference v0.19.6 // indirect
github.com/go-openapi/loads v0.21.1 // indirect
github.com/go-openapi/runtime v0.23.1 // indirect
github.com/go-openapi/runtime v0.23.3 // indirect
github.com/go-openapi/spec v0.20.4 // indirect
github.com/go-openapi/strfmt v0.21.2 // indirect
github.com/go-openapi/swag v0.21.1 // indirect
@@ -142,7 +143,7 @@ require (
github.com/go-playground/universal-translator v0.18.0 // indirect
github.com/go-playground/validator/v10 v10.10.0 // indirect
github.com/go-stack/stack v1.8.1 // indirect
github.com/golang-jwt/jwt/v4 v4.2.0 // indirect
github.com/golang-jwt/jwt/v4 v4.3.0 // indirect
github.com/golang/glog v1.0.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/mock v1.6.0 // indirect
@@ -152,26 +153,27 @@ require (
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/trillian v1.4.0 // indirect
github.com/googleapis/gax-go/v2 v2.1.1 // indirect
github.com/googleapis/gax-go/v2 v2.2.0 // indirect
github.com/googleapis/gnostic v0.5.5 // indirect
github.com/googleapis/go-type-adapters v1.0.0 // indirect
github.com/gorilla/websocket v1.4.2 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-hclog v1.1.0 // indirect
github.com/hashicorp/go-hclog v1.2.0 // indirect
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
github.com/hashicorp/go-plugin v1.4.3 // indirect
github.com/hashicorp/go-retryablehttp v0.7.0 // indirect
github.com/hashicorp/go-rootcerts v1.0.2 // indirect
github.com/hashicorp/go-secure-stdlib/mlock v0.1.2 // indirect
github.com/hashicorp/go-secure-stdlib/parseutil v0.1.2 // indirect
github.com/hashicorp/go-secure-stdlib/parseutil v0.1.3 // indirect
github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 // indirect
github.com/hashicorp/go-sockaddr v1.0.2 // indirect
github.com/hashicorp/go-uuid v1.0.2 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/hashicorp/go-version v1.4.0 // indirect
github.com/hashicorp/golang-lru v0.5.4 // indirect
github.com/hashicorp/vault/api v1.3.1 // indirect
github.com/hashicorp/vault/api v1.5.0 // indirect
github.com/hashicorp/vault/sdk v0.4.1 // indirect
github.com/hashicorp/yamux v0.0.0-20211028200310-0bc27b27de87 // indirect
github.com/imdario/mergo v0.3.12 // indirect
@@ -182,6 +184,7 @@ require (
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/leodido/go-urn v1.2.1 // indirect
github.com/letsencrypt/boulder v0.0.0-20220331220046-b23ab962616e // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
github.com/miekg/pkcs11 v1.1.1 // indirect
@@ -194,7 +197,7 @@ require (
github.com/oklog/ulid v1.3.1 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pierrec/lz4 v2.6.1+incompatible // indirect
github.com/prometheus/client_golang v1.11.0 // indirect
github.com/prometheus/client_golang v1.12.1 // indirect
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.32.1 // indirect
github.com/prometheus/procfs v0.7.3 // indirect
@@ -209,14 +212,15 @@ require (
github.com/sigstore/rekor v0.4.1-0.20220114213500-23f583409af3 // indirect
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 // indirect
github.com/soheilhy/cmux v0.1.5 // indirect
github.com/spiffe/go-spiffe/v2 v2.0.0-beta.12 // indirect
github.com/spiffe/go-spiffe/v2 v2.0.0 // indirect
github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7 // indirect
github.com/tent/canonical-json-go v0.0.0-20130607151641-96e4ba3a7613 // indirect
github.com/thales-e-security/pool v0.0.2 // indirect
github.com/theupdateframework/go-tuf v0.0.0-20220211205608-f0c3294f63b9 // indirect
github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399 // indirect
github.com/tmc/grpc-websocket-proxy v0.0.0-20201229170055-e5319fda7802 // indirect
github.com/urfave/cli v1.22.5 // indirect
github.com/xanzy/go-gitlab v0.56.0 // indirect
github.com/xanzy/go-gitlab v0.62.0 // indirect
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
github.com/zeebo/errs v1.2.2 // indirect
@@ -247,23 +251,23 @@ require (
go.uber.org/atomic v1.9.0 // indirect
go.uber.org/multierr v1.7.0 // indirect
go.uber.org/zap v1.21.0 // indirect
golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 // indirect
golang.org/x/tools v0.1.8 // indirect
google.golang.org/api v0.70.0 // indirect
google.golang.org/protobuf v1.27.1 // indirect
golang.org/x/time v0.0.0-20220224211638-0e9765cccd65 // indirect
golang.org/x/tools v0.1.10 // indirect
google.golang.org/api v0.74.0 // indirect
google.golang.org/protobuf v1.28.0 // indirect
gopkg.in/cheggaaa/pb.v1 v1.0.28 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect
k8s.io/api v0.23.3 // indirect
k8s.io/apimachinery v0.23.3 // indirect
k8s.io/client-go v0.23.3 // indirect
k8s.io/klog/v2 v2.40.1 // indirect
k8s.io/api v0.23.5 // indirect
k8s.io/apimachinery v0.23.5 // indirect
k8s.io/client-go v0.23.5 // indirect
k8s.io/klog/v2 v2.60.1-0.20220317184644-43cc75f9ae89 // indirect
k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf // indirect
k8s.io/utils v0.0.0-20220127004650-9b3446523e65 // indirect
knative.dev/pkg v0.0.0-20220202132633-df430fa0dd96 // indirect
k8s.io/utils v0.0.0-20220210201930-3a6ce19ff2f9 // indirect
knative.dev/pkg v0.0.0-20220325200448-1f7514acd0c2 // indirect
sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 // indirect
sigs.k8s.io/release-utils v0.4.1-0.20220207182343-6dadf2228617 // indirect
sigs.k8s.io/release-utils v0.6.0 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.1 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)
@@ -315,15 +319,15 @@ require (
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
golang.org/x/crypto v0.0.0-20220213190939-1e6e3497d506 // indirect
golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 // indirect
golang.org/x/crypto v0.0.0-20220427172511-eb4f295cb31f // indirect
golang.org/x/oauth2 v0.0.0-20220309155454-6242fa91716a // indirect
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect
golang.org/x/sys v0.0.0-20220209214540-3681064d5158 // indirect
golang.org/x/sys v0.0.0-20220328115105-d36c6a25d886 // indirect
golang.org/x/text v0.3.7 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20220218161850-94dd64e39d7c // indirect
google.golang.org/grpc v1.44.0 // indirect
google.golang.org/genproto v0.0.0-20220405205423-9d709892a2bf // indirect
google.golang.org/grpc v1.45.0 // indirect
gopkg.in/ini.v1 v1.66.2 // indirect
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
)
+292 -263
View File
File diff suppressed because it is too large Load Diff
+10 -7
View File
@@ -6,15 +6,14 @@ import (
"fmt"
"time"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/syft/event"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/antihax/optional"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
)
type ImportConfig struct {
@@ -57,7 +56,7 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
authedCtx := c.newRequestContext(ctxWithTimeout)
stage.Current = "starting session"
startOperation, _, err := c.client.ImportsApi.CreateOperation(authedCtx)
startOperation, createResponse, err := c.client.ImportsApi.CreateOperation(authedCtx)
if err != nil {
var detail = "no details given"
var openAPIErr external.GenericOpenAPIError
@@ -66,6 +65,8 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
}
return fmt.Errorf("unable to start import session: %w: %s", err, detail)
}
defer createResponse.Body.Close()
prog.N++
sessionID := startOperation.Uuid
@@ -99,7 +100,7 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
Force: optional.NewBool(cfg.OverwriteExistingUpload),
}
_, _, err = c.client.ImagesApi.AddImage(authedCtx, imageModel, &opts)
_, addResponse, err := c.client.ImagesApi.AddImage(authedCtx, imageModel, &opts)
if err != nil {
var detail = "no details given"
var openAPIErr external.GenericOpenAPIError
@@ -108,6 +109,8 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
}
return fmt.Errorf("unable to complete import session=%q: %w: %s", sessionID, err, detail)
}
defer addResponse.Body.Close()
prog.N++
stage.Current = ""
+11 -13
View File
@@ -4,8 +4,6 @@ import (
"context"
"encoding/json"
"fmt"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/file"
"net/http"
"strings"
"testing"
@@ -24,7 +22,7 @@ import (
"github.com/wagoodman/go-progress"
)
func must(c cpe.CPE, e error) cpe.CPE {
func must(c pkg.CPE, e error) pkg.CPE {
if e != nil {
panic(e)
}
@@ -54,23 +52,23 @@ func (m *mockPackageSBOMImportAPI) ImportImagePackages(ctx context.Context, sess
func sbomFixture() sbom.SBOM {
return sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(pkg.Package{
PackageCatalog: pkg.NewCatalog(pkg.Package{
Name: "name",
Version: "version",
FoundBy: "foundBy",
Locations: []file.Location{
{
Coordinates: file.Coordinates{
Locations: source.NewLocationSet(
source.Location{
Coordinates: source.Coordinates{
RealPath: "path",
FileSystemID: "layerID",
},
},
},
),
Licenses: []string{"license"},
Language: pkg.Python,
Type: pkg.PythonPkg,
CPEs: []cpe.CPE{
must(cpe.New("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*")),
CPEs: []pkg.CPE{
must(pkg.NewCPE("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*")),
},
PURL: "purl",
MetadataType: pkg.PythonPackageMetadataType,
@@ -104,13 +102,13 @@ func sbomFixture() sbom.SBOM {
},
Relationships: []artifact.Relationship{
{
From: file.NewLocation("/place1"),
To: file.NewLocation("/place2"),
From: source.NewLocation("/place1"),
To: source.NewLocation("/place2"),
Type: artifact.ContainsRelationship,
},
},
Source: source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "user-in",
Layers: nil,
+103 -165
View File
@@ -3,26 +3,24 @@ package config
import (
"errors"
"fmt"
"github.com/anchore/syft/syft/source"
"path"
"reflect"
"strings"
"github.com/sirupsen/logrus"
"github.com/adrg/xdg"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/internal/log"
"github.com/mitchellh/go-homedir"
"github.com/sirupsen/logrus"
"github.com/spf13/viper"
"gopkg.in/yaml.v2"
)
var ErrApplicationConfigNotFound = fmt.Errorf("application config not found")
// TODO: set all catalogers when this is set
var catalogerEnabledDefault = false
var (
ErrApplicationConfigNotFound = fmt.Errorf("application config not found")
catalogerEnabledDefault = false
)
type defaultValueLoader interface {
loadDefaultValues(*viper.Viper)
@@ -34,128 +32,59 @@ type parser interface {
// Application is the main syft application configuration.
type Application struct {
ConfigPath string `yaml:",omitempty" json:"configPath"` // the location where the application config was read from (either from -c or discovered while loading)
Outputs []string `yaml:"output" json:"output" mapstructure:"output"` // -o, the format to use for output
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"` // -q, indicates to not show any status output to stderr (ETUI or logging UI)
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Anchore anchore `yaml:"anchore" json:"anchore" mapstructure:"anchore"` // options for interacting with Anchore Engine/Enterprise
CliOptions CliOnlyOptions `yaml:"-" json:"-"` // all options only available through the CLI (not via env vars or config)
Catalogers []string `yaml:"catalogers" json:"catalogers" mapstructure:"catalogers"`
Scope string `yaml:"scope" json:"scope" mapstructure:"scope"`
Dev development `yaml:"dev" json:"dev" mapstructure:"dev"`
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
FileMetadata fileMetadata `yaml:"file-metadata" json:"file-metadata" mapstructure:"file-metadata"`
FileContents fileContents `yaml:"file-contents" json:"file-contents" mapstructure:"file-contents"`
Secrets secretsCfg `yaml:"secrets" json:"secrets" mapstructure:"secrets"`
Registry registry `yaml:"registry" json:"registry" mapstructure:"registry"`
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
// the location where the application config was read from (either from -c or discovered while loading); default .syft.yaml
ConfigPath string `yaml:"configPath,omitempty" json:"configPath" mapstructure:"config"`
Verbosity uint `yaml:"verbosity,omitempty" json:"verbosity" mapstructure:"verbosity"`
// -q, indicates to not show any status output to stderr (ETUI or logging UI)
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"`
Outputs []string `yaml:"output" json:"output" mapstructure:"output"` // -o, the format to use for output
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Anchore anchore `yaml:"anchore" json:"anchore" mapstructure:"anchore"` // options for interacting with Anchore Engine/Enterprise
Dev development `yaml:"dev" json:"dev" mapstructure:"dev"`
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
FileMetadata FileMetadata `yaml:"file-metadata" json:"file-metadata" mapstructure:"file-metadata"`
FileClassification fileClassification `yaml:"file-classification" json:"file-classification" mapstructure:"file-classification"`
FileContents fileContents `yaml:"file-contents" json:"file-contents" mapstructure:"file-contents"`
Secrets secrets `yaml:"secrets" json:"secrets" mapstructure:"secrets"`
Registry registry `yaml:"registry" json:"registry" mapstructure:"registry"`
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
}
func (cfg Application) ToCatalogingOptions() ([]syft.CatalogingOption, error) {
func (cfg *Application) LoadAllValues(v *viper.Viper, configPath string) error {
// priority order: viper.Set, flag, env, config, kv, defaults
// flags have already been loaded into viper by command construction
digests, err := file.DigestHashesByName(cfg.FileMetadata.Digests...)
if err != nil {
return nil, fmt.Errorf("unable to parse config item 'file-metadata.digests': %w", err)
}
scopeOption := source.ParseScope(cfg.Scope)
if scopeOption == source.UnknownScope {
return nil, fmt.Errorf("bad scope value %q", cfg.Scope)
}
secretsConfig, err := cfg.Secrets.ToConfig()
if err != nil {
return nil, err
}
secretsScopeOption := source.ParseScope(cfg.Secrets.Scope)
if secretsScopeOption == source.UnknownScope {
return nil, fmt.Errorf("bad scope value %q", cfg.Secrets.Scope)
}
c := syft.DefaultCatalogingConfig()
c.ToolVersion = version.FromBuild().Version
c.ToolConfiguration = &c
c.DefaultScope = scopeOption
c.ProcessTasksInSerial = false
c.DigestHashes = digests
c.SecretsSearch = *secretsConfig
c.SecretsScope = secretsScopeOption
c.ContentsSearch = cfg.FileContents.ToConfig()
opts := []syft.CatalogingOption{
syft.WithConfig(c),
syft.WithCatalogers(cfg.Catalogers...),
}
return opts, nil
}
// PowerUserCatalogerEnabledDefault switches all catalogers to be enabled when running power-user command
func PowerUserCatalogerEnabledDefault() {
catalogerEnabledDefault = true
}
func newApplicationConfig(v *viper.Viper, cliOpts CliOnlyOptions) *Application {
config := &Application{
CliOptions: cliOpts,
}
config.loadDefaultValues(v)
return config
}
// LoadApplicationConfig populates the given viper object with application configuration discovered on disk
func LoadApplicationConfig(v *viper.Viper, cliOpts CliOnlyOptions) (*Application, error) {
// the user may not have a config, and this is OK, we can use the default config + default cobra cli values instead
config := newApplicationConfig(v, cliOpts)
if err := readConfig(v, cliOpts.ConfigPath); err != nil && !errors.Is(err, ErrApplicationConfigNotFound) {
return nil, err
}
if err := v.Unmarshal(config); err != nil {
return nil, fmt.Errorf("unable to parse config: %w", err)
}
config.ConfigPath = v.ConfigFileUsed()
if err := config.parseConfigValues(); err != nil {
return nil, fmt.Errorf("invalid application config: %w", err)
}
return config, nil
}
// init loads the default configuration values into the viper instance (before the config values are read and parsed).
func (cfg Application) loadDefaultValues(v *viper.Viper) {
// set the default values for primitive fields in this struct
v.SetDefault("check-for-app-update", true)
// for each field in the configuration struct, see if the field implements the defaultValueLoader interface and invoke it if it does
value := reflect.ValueOf(cfg)
for i := 0; i < value.NumField(); i++ {
// note: the defaultValueLoader method receiver is NOT a pointer receiver.
if loadable, ok := value.Field(i).Interface().(defaultValueLoader); ok {
// the field implements defaultValueLoader, call it
loadable.loadDefaultValues(v)
// check if user specified config; otherwise read all possible paths
if err := loadConfig(v, configPath); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); ok {
// Not Found; ignore this error
log.Debug("no config file found; proceeding with defaults")
}
}
// load default config values into viper
loadDefaultValues(v)
// load environment variables
v.SetEnvPrefix(internal.ApplicationName)
v.AllowEmptyEnv(true)
v.AutomaticEnv()
// unmarshal fully populated viper object onto config
err := v.Unmarshal(cfg)
if err != nil {
return err
}
// Convert all populated config options to their internal application values ex: scope string => scopeOpt source.Scope
return cfg.parseConfigValues()
}
func (cfg *Application) parseConfigValues() error {
// parse fields on application directyy
var catalogers []string
for _, c := range cfg.Catalogers {
for _, f := range strings.Split(c, ",") {
catalogers = append(catalogers, strings.TrimSpace(f))
}
}
cfg.Catalogers = catalogers
// parse application config options
for _, optionFn := range []func() error{
cfg.parseUploadOptions,
@@ -166,7 +95,6 @@ func (cfg *Application) parseConfigValues() error {
return err
}
}
// parse nested config options
// for each field in the configuration struct, see if the field implements the parser interface
// note: the app config is a pointer, so we need to grab the elements explicitly (to traverse the address)
@@ -183,17 +111,6 @@ func (cfg *Application) parseConfigValues() error {
return nil
}
func (cfg *Application) parseFile() error {
if cfg.File != "" {
expandedPath, err := homedir.Expand(cfg.File)
if err != nil {
return fmt.Errorf("unable to expand file path=%q: %w", cfg.File, err)
}
cfg.File = expandedPath
}
return nil
}
func (cfg *Application) parseUploadOptions() error {
if cfg.Anchore.Host == "" && cfg.Anchore.Dockerfile != "" {
return fmt.Errorf("cannot provide dockerfile option without enabling upload")
@@ -208,23 +125,9 @@ func (cfg *Application) parseLogLevelOption() error {
// we should be able to quiet the console logging and leave file logging alone...
// ... this will be an enhancement for later
cfg.Log.LevelOpt = logrus.PanicLevel
case cfg.Log.Level != "":
if cfg.CliOptions.Verbosity > 0 {
return fmt.Errorf("cannot explicitly set log level (cfg file or env var) and use -v flag together")
}
lvl, err := logrus.ParseLevel(strings.ToLower(cfg.Log.Level))
if err != nil {
return fmt.Errorf("bad log level configured (%q): %w", cfg.Log.Level, err)
}
cfg.Log.LevelOpt = lvl
if cfg.Log.LevelOpt >= logrus.InfoLevel {
cfg.CliOptions.Verbosity = 1
}
default:
switch v := cfg.CliOptions.Verbosity; {
case cfg.Verbosity > 0:
switch v := cfg.Verbosity; {
case v == 1:
cfg.Log.LevelOpt = logrus.InfoLevel
case v >= 2:
@@ -232,6 +135,18 @@ func (cfg *Application) parseLogLevelOption() error {
default:
cfg.Log.LevelOpt = logrus.ErrorLevel
}
case cfg.Log.Level != "":
lvl, err := logrus.ParseLevel(strings.ToLower(cfg.Log.Level))
if err != nil {
return fmt.Errorf("bad log level configured (%q): %w", cfg.Log.Level, err)
}
cfg.Log.LevelOpt = lvl
if cfg.Log.LevelOpt >= logrus.InfoLevel {
cfg.Verbosity = 1
}
default:
cfg.Log.LevelOpt = logrus.WarnLevel
}
if cfg.Log.Level == "" {
@@ -241,43 +156,64 @@ func (cfg *Application) parseLogLevelOption() error {
return nil
}
func (cfg *Application) parseFile() error {
if cfg.File != "" {
expandedPath, err := homedir.Expand(cfg.File)
if err != nil {
return fmt.Errorf("unable to expand file path=%q: %w", cfg.File, err)
}
cfg.File = expandedPath
}
return nil
}
// init loads the default configuration values into the viper instance (before the config values are read and parsed).
func loadDefaultValues(v *viper.Viper) {
// set the default values for primitive fields in this struct
v.SetDefault("quiet", false)
v.SetDefault("check-for-app-update", true)
// for each field in the configuration struct, see if the field implements the defaultValueLoader interface and invoke it if it does
value := reflect.ValueOf(Application{})
for i := 0; i < value.NumField(); i++ {
// note: the defaultValueLoader method receiver is NOT a pointer receiver.
if loadable, ok := value.Field(i).Interface().(defaultValueLoader); ok {
// the field implements defaultValueLoader, call it
loadable.loadDefaultValues(v)
}
}
}
func (cfg Application) String() string {
// yaml is pretty human friendly (at least when compared to json)
appCfgStr, err := yaml.Marshal(&cfg)
appaStr, err := yaml.Marshal(&cfg)
if err != nil {
return err.Error()
}
return string(appCfgStr)
return string(appaStr)
}
// readConfig attempts to read the given config path from disk or discover an alternate store location
// nolint:funlen
func readConfig(v *viper.Viper, configPath string) error {
func loadConfig(v *viper.Viper, configPath string) error {
var err error
v.AutomaticEnv()
v.SetEnvPrefix(internal.ApplicationName)
// allow for nested options to be specified via environment variables
// e.g. pod.context = APPNAME_POD_CONTEXT
v.SetEnvKeyReplacer(strings.NewReplacer(".", "_", "-", "_"))
// use explicitly the given user config
if configPath != "" {
v.SetConfigFile(configPath)
if err := v.ReadInConfig(); err != nil {
return fmt.Errorf("unable to read application config=%q : %w", configPath, err)
}
v.Set("config", v.ConfigFileUsed())
// don't fall through to other options if the config path was explicitly provided
return nil
}
// start searching for valid configs in order...
// 1. look for .<appname>.yaml (in the current directory)
v.AddConfigPath(".")
v.SetConfigName("." + internal.ApplicationName)
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
} else if !errors.As(err, &viper.ConfigFileNotFoundError{}) {
return fmt.Errorf("unable to parse config=%q: %w", v.ConfigFileUsed(), err)
@@ -287,6 +223,7 @@ func readConfig(v *viper.Viper, configPath string) error {
v.AddConfigPath("." + internal.ApplicationName)
v.SetConfigName("config")
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
} else if !errors.As(err, &viper.ConfigFileNotFoundError{}) {
return fmt.Errorf("unable to parse config=%q: %w", v.ConfigFileUsed(), err)
@@ -298,6 +235,7 @@ func readConfig(v *viper.Viper, configPath string) error {
v.AddConfigPath(home)
v.SetConfigName("." + internal.ApplicationName)
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
} else if !errors.As(err, &viper.ConfigFileNotFoundError{}) {
return fmt.Errorf("unable to parse config=%q: %w", v.ConfigFileUsed(), err)
@@ -311,10 +249,10 @@ func readConfig(v *viper.Viper, configPath string) error {
}
v.SetConfigName("config")
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
} else if !errors.As(err, &viper.ConfigFileNotFoundError{}) {
return fmt.Errorf("unable to parse config=%q: %w", v.ConfigFileUsed(), err)
}
return ErrApplicationConfigNotFound
return nil
}
-52
View File
@@ -1,52 +0,0 @@
package config
import (
"github.com/docker/docker/pkg/homedir"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"path/filepath"
"testing"
)
func TestApplication_parseFile(t *testing.T) {
tests := []struct {
name string
config Application
expected string
wantErr require.ErrorAssertionFunc
}{
{
name: "expand home dir",
config: Application{
File: "~/place.txt",
},
expected: filepath.Join(homedir.Get(), "place.txt"),
},
{
name: "passthrough other paths",
config: Application{
File: "/other/place.txt",
},
expected: "/other/place.txt",
},
{
name: "no path",
config: Application{
File: "",
},
expected: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := tt.config
if tt.wantErr == nil {
tt.wantErr = require.NoError
}
tt.wantErr(t, cfg.parseFile())
assert.Equal(t, tt.expected, cfg.File)
})
}
}
+29
View File
@@ -0,0 +1,29 @@
package config
import (
"fmt"
"github.com/spf13/viper"
"github.com/anchore/syft/syft/source"
)
type catalogerOptions struct {
Enabled bool `yaml:"enabled" json:"enabled" mapstructure:"enabled"`
Scope string `yaml:"scope" json:"scope" mapstructure:"scope"`
ScopeOpt source.Scope `yaml:"-" json:"-"`
}
func (cfg catalogerOptions) loadDefaultValues(v *viper.Viper) {
v.SetDefault("package.cataloger.enabled", true)
}
func (cfg *catalogerOptions) parseConfigValues() error {
scopeOption := source.ParseScope(cfg.Scope)
if scopeOption == source.UnknownScope {
return fmt.Errorf("bad scope value %q", cfg.Scope)
}
cfg.ScopeOpt = scopeOption
return nil
}
-7
View File
@@ -1,7 +0,0 @@
package config
// CliOnlyOptions are options that are in the application config in memory, but are only exposed via CLI switches (not from unmarshaling a config file)
type CliOnlyOptions struct {
ConfigPath string // -c. where the read config is on disk
Verbosity int // -v or -vv , controlling which UI (ETUI vs logging) and what the log level should be
}
+1
View File
@@ -0,0 +1 @@
package config
+19
View File
@@ -0,0 +1,19 @@
package config
import (
"github.com/anchore/syft/syft/source"
"github.com/spf13/viper"
)
type fileClassification struct {
Cataloger catalogerOptions `yaml:"cataloger" json:"cataloger" mapstructure:"cataloger"`
}
func (cfg fileClassification) loadDefaultValues(v *viper.Viper) {
v.SetDefault("file-classification.cataloger.enabled", catalogerEnabledDefault)
v.SetDefault("file-classification.cataloger.scope", source.SquashedScope)
}
func (cfg *fileClassification) parseConfigValues() error {
return cfg.Cataloger.parseConfigValues()
}
+9 -9
View File
@@ -1,24 +1,24 @@
package config
import (
"github.com/anchore/syft/syft/cataloger/files/filecontents"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/internal/file"
"github.com/anchore/syft/syft/source"
"github.com/spf13/viper"
)
type fileContents struct {
SkipFilesAboveSize int64 `yaml:"skip-files-above-size" json:"skip-files-above-size" mapstructure:"skip-files-above-size"`
Globs []string `yaml:"globs" json:"globs" mapstructure:"globs"`
Cataloger catalogerOptions `yaml:"cataloger" json:"cataloger" mapstructure:"cataloger"`
SkipFilesAboveSize int64 `yaml:"skip-files-above-size" json:"skip-files-above-size" mapstructure:"skip-files-above-size"`
Globs []string `yaml:"globs" json:"globs" mapstructure:"globs"`
}
func (cfg fileContents) loadDefaultValues(v *viper.Viper) {
v.SetDefault("file-contents.cataloger.enabled", catalogerEnabledDefault)
v.SetDefault("file-contents.cataloger.scope", source.SquashedScope)
v.SetDefault("file-contents.skip-files-above-size", 1*file.MB)
v.SetDefault("file-contents.globs", []string{})
}
func (cfg fileContents) ToConfig() filecontents.Config {
return filecontents.Config{
Globs: cfg.Globs,
SkipFilesAboveSizeInBytes: cfg.SkipFilesAboveSize,
}
func (cfg *fileContents) parseConfigValues() error {
return cfg.Cataloger.parseConfigValues()
}
+9 -5
View File
@@ -1,17 +1,21 @@
package config
import (
"github.com/anchore/syft/syft/source"
"github.com/spf13/viper"
)
type fileMetadata struct {
Digests []string `yaml:"digests" json:"digests" mapstructure:"digests"`
type FileMetadata struct {
Cataloger catalogerOptions `yaml:"cataloger" json:"cataloger" mapstructure:"cataloger"`
Digests []string `yaml:"digests" json:"digests" mapstructure:"digests"`
}
func (cfg fileMetadata) loadDefaultValues(v *viper.Viper) {
func (cfg FileMetadata) loadDefaultValues(v *viper.Viper) {
v.SetDefault("file-metadata.cataloger.enabled", catalogerEnabledDefault)
v.SetDefault("file-metadata.cataloger.scope", source.SquashedScope)
v.SetDefault("file-metadata.digests", []string{"sha256"})
}
func (cfg *fileMetadata) parseConfigValues() error {
return nil
func (cfg *FileMetadata) parseConfigValues() error {
return cfg.Cataloger.parseConfigValues()
}
+17 -8
View File
@@ -1,24 +1,33 @@
package config
import (
"github.com/anchore/syft/syft/cataloger/packages"
"github.com/anchore/syft/syft/pkg/cataloger"
"github.com/spf13/viper"
)
type pkg struct {
SearchUnindexedArchives bool `yaml:"search-unindexed-archives" json:"search-unindexed-archives" mapstructure:"search-unindexed-archives"`
SearchIndexedArchives bool `yaml:"search-indexed-archives" json:"search-indexed-archives" mapstructure:"search-indexed-archives"`
Cataloger catalogerOptions `yaml:"cataloger" json:"cataloger" mapstructure:"cataloger"`
SearchUnindexedArchives bool `yaml:"search-unindexed-archives" json:"search-unindexed-archives" mapstructure:"search-unindexed-archives"`
SearchIndexedArchives bool `yaml:"search-indexed-archives" json:"search-indexed-archives" mapstructure:"search-indexed-archives"`
}
func (cfg pkg) loadDefaultValues(v *viper.Viper) {
c := packages.DefaultSearchConfig()
cfg.Cataloger.loadDefaultValues(v)
c := cataloger.DefaultSearchConfig()
v.SetDefault("package.search-unindexed-archives", c.IncludeUnindexedArchives)
v.SetDefault("package.search-indexed-archives", c.IncludeIndexedArchives)
}
func (cfg pkg) ToConfig() packages.SearchConfig {
return packages.SearchConfig{
IncludeIndexedArchives: cfg.SearchIndexedArchives,
IncludeUnindexedArchives: cfg.SearchUnindexedArchives,
func (cfg *pkg) parseConfigValues() error {
return cfg.Cataloger.parseConfigValues()
}
func (cfg pkg) ToConfig() cataloger.Config {
return cataloger.Config{
Search: cataloger.SearchConfig{
IncludeIndexedArchives: cfg.SearchIndexedArchives,
IncludeUnindexedArchives: cfg.SearchUnindexedArchives,
Scope: cfg.Cataloger.ScopeOpt,
},
}
}
+8 -18
View File
@@ -1,38 +1,28 @@
package config
import (
"fmt"
"github.com/anchore/syft/syft/cataloger/files/secrets"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/internal/file"
"github.com/anchore/syft/syft/source"
"github.com/spf13/viper"
)
type secretsCfg struct {
type secrets struct {
Cataloger catalogerOptions `yaml:"cataloger" json:"cataloger" mapstructure:"cataloger"`
AdditionalPatterns map[string]string `yaml:"additional-patterns" json:"additional-patterns" mapstructure:"additional-patterns"`
ExcludePatternNames []string `yaml:"exclude-pattern-names" json:"exclude-pattern-names" mapstructure:"exclude-pattern-names"`
RevealValues bool `yaml:"reveal-values" json:"reveal-values" mapstructure:"reveal-values"`
SkipFilesAboveSize int64 `yaml:"skip-files-above-size" json:"skip-files-above-size" mapstructure:"skip-files-above-size"`
Scope string `yaml:"scope" json:"scope" mapstructure:"scope"`
}
func (cfg secretsCfg) loadDefaultValues(v *viper.Viper) {
v.SetDefault("secrets.scope", source.AllLayersScope)
func (cfg secrets) loadDefaultValues(v *viper.Viper) {
v.SetDefault("secrets.cataloger.enabled", catalogerEnabledDefault)
v.SetDefault("secrets.cataloger.scope", source.AllLayersScope)
v.SetDefault("secrets.reveal-values", false)
v.SetDefault("secrets.skip-files-above-size", 1*file.MB)
v.SetDefault("secrets.additional-patterns", map[string]string{})
v.SetDefault("secrets.exclude-pattern-names", []string{})
}
func (cfg secretsCfg) ToConfig() (*secrets.Config, error) {
patterns, err := file.GenerateSearchPatterns(secrets.DefaultSecretsPatterns, cfg.AdditionalPatterns, cfg.ExcludePatternNames)
if err != nil {
return nil, fmt.Errorf("unable to process secrets config patterns: %w", err)
}
return &secrets.Config{
Patterns: patterns,
RevealValues: cfg.RevealValues,
MaxFileSize: cfg.SkipFilesAboveSize,
}, nil
func (cfg *secrets) parseConfigValues() error {
return cfg.Cataloger.parseConfigValues()
}
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "3.0.1"
JSONSchemaVersion = "3.2.2"
)
@@ -1,14 +1,12 @@
package archive
package file
import (
"errors"
"fmt"
"io"
"github.com/anchore/syft/syft/file"
)
const perFileReadLimit = 2 * file.GB
const perFileReadLimit = 2 * GB
// safeCopy limits the copy from the reader. This is useful when extracting files from archives to
// protect against decompression bomb attacks.
@@ -1,8 +1,8 @@
package internal
package file
// FileNameGlobMatch evaluates the given glob pattern against the given "name" string, indicating if there is a match or not.
// GlobMatch evaluates the given glob pattern against the given "name" string, indicating if there is a match or not.
// Source: https://research.swtch.com/glob.go
func FileNameGlobMatch(pattern, name string) bool {
func GlobMatch(pattern, name string) bool {
px := 0
nx := 0
nextPx := 0
@@ -1,4 +1,4 @@
package internal
package file
import (
"strings"
@@ -32,7 +32,7 @@ func TestGlobMatch(t *testing.T) {
}
for _, test := range tests {
if FileNameGlobMatch(test.pattern, test.data) != test.ok {
if GlobMatch(test.pattern, test.data) != test.ok {
t.Errorf("failed glob='%s' data='%s'", test.pattern, test.data)
}
}
@@ -10,12 +10,6 @@ type Opener struct {
path string
}
func NewOpener(path string) Opener {
return Opener{
path: path,
}
}
// Open the stored path as a io.ReadCloser.
func (o Opener) Open() (io.ReadCloser, error) {
return os.Open(o.path)
@@ -1,39 +1,38 @@
package archive
package file
import (
"fmt"
"io/ioutil"
"path/filepath"
"github.com/anchore/syft/syft/file"
"github.com/bmatcuk/doublestar/v4"
"github.com/mholt/archiver/v3"
)
// ExtractGlobsFromTarToUniqueTempFile extracts paths matching the given globs within the given archive to a temporary directory, returning file openers for each file extracted.
func ExtractGlobsFromTarToUniqueTempFile(archivePath, dir string, globs ...string) (map[string]file.Opener, error) {
results := make(map[string]file.Opener)
func ExtractGlobsFromTarToUniqueTempFile(archivePath, dir string, globs ...string) (map[string]Opener, error) {
results := make(map[string]Opener)
// don't allow for full traversal, only select traversal from given paths
if len(globs) == 0 {
return results, nil
}
visitor := func(f archiver.File) error {
defer f.Close()
visitor := func(file archiver.File) error {
defer file.Close()
// ignore directories
if f.FileInfo.IsDir() {
if file.FileInfo.IsDir() {
return nil
}
// ignore any filename that doesn't match the given globs...
if !matchesAnyGlob(f.Name(), globs...) {
if !matchesAnyGlob(file.Name(), globs...) {
return nil
}
// we have a file we want to extract....
tempfilePrefix := filepath.Base(filepath.Clean(f.Name())) + "-"
tempfilePrefix := filepath.Base(filepath.Clean(file.Name())) + "-"
tempFile, err := ioutil.TempFile(dir, tempfilePrefix)
if err != nil {
return fmt.Errorf("unable to create temp file: %w", err)
@@ -43,11 +42,11 @@ func ExtractGlobsFromTarToUniqueTempFile(archivePath, dir string, globs ...strin
// provides a ReadCloser. It is up to the caller to handle closing the file explicitly.
defer tempFile.Close()
if err := safeCopy(tempFile, f.ReadCloser); err != nil {
return fmt.Errorf("unable to copy source=%q for tar=%q: %w", f.Name(), archivePath, err)
if err := safeCopy(tempFile, file.ReadCloser); err != nil {
return fmt.Errorf("unable to copy source=%q for tar=%q: %w", file.Name(), archivePath, err)
}
results[f.Name()] = file.NewOpener(tempFile.Name())
results[file.Name()] = Opener{path: tempFile.Name()}
return nil
}
@@ -1,4 +1,4 @@
package archive
package file
import (
"io/ioutil"
@@ -1,4 +1,4 @@
package archive
package file
import (
"fmt"
@@ -48,7 +48,7 @@ func (z ZipFileManifest) GlobMatch(patterns ...string) []string {
// so that glob logic is consistent inside and outside of ZIP archives
normalizedEntry := normalizeZipEntryName(entry)
if internal.FileNameGlobMatch(pattern, normalizedEntry) {
if GlobMatch(pattern, normalizedEntry) {
uniqueMatches.Add(entry)
}
}
@@ -1,7 +1,7 @@
//go:build !windows
// +build !windows
package archive
package file
import (
"encoding/json"
@@ -1,4 +1,4 @@
package archive
package file
import (
"archive/zip"
@@ -9,11 +9,17 @@ import (
"path/filepath"
"strings"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/internal/log"
)
const (
// represents the order of bytes
_ = iota
KB = 1 << (10 * iota)
MB
GB
)
type errZipSlipDetected struct {
Prefix string
JoinArgs []string
@@ -65,16 +71,16 @@ func TraverseFilesInZip(archivePath string, visitor func(*zip.File) error, paths
}
// ExtractFromZipToUniqueTempFile extracts select paths for the given archive to a temporary directory, returning file openers for each file extracted.
func ExtractFromZipToUniqueTempFile(archivePath, dir string, paths ...string) (map[string]file.Opener, error) {
results := make(map[string]file.Opener)
func ExtractFromZipToUniqueTempFile(archivePath, dir string, paths ...string) (map[string]Opener, error) {
results := make(map[string]Opener)
// don't allow for full traversal, only select traversal from given paths
if len(paths) == 0 {
return results, nil
}
visitor := func(f *zip.File) error {
tempfilePrefix := filepath.Base(filepath.Clean(f.Name)) + "-"
visitor := func(file *zip.File) error {
tempfilePrefix := filepath.Base(filepath.Clean(file.Name)) + "-"
tempFile, err := ioutil.TempFile(dir, tempfilePrefix)
if err != nil {
@@ -85,26 +91,26 @@ func ExtractFromZipToUniqueTempFile(archivePath, dir string, paths ...string) (m
// provides a ReadCloser. It is up to the caller to handle closing the file explicitly.
defer tempFile.Close()
zippedFile, err := f.Open()
zippedFile, err := file.Open()
if err != nil {
return fmt.Errorf("unable to read file=%q from zip=%q: %w", f.Name, archivePath, err)
return fmt.Errorf("unable to read file=%q from zip=%q: %w", file.Name, archivePath, err)
}
defer func() {
err := zippedFile.Close()
if err != nil {
log.Errorf("unable to close source file=%q from zip=%q: %+v", f.Name, archivePath, err)
log.Errorf("unable to close source file=%q from zip=%q: %+v", file.Name, archivePath, err)
}
}()
if f.FileInfo().IsDir() {
return fmt.Errorf("unable to extract directories, only files: %s", f.Name)
if file.FileInfo().IsDir() {
return fmt.Errorf("unable to extract directories, only files: %s", file.Name)
}
if err := safeCopy(tempFile, zippedFile); err != nil {
return fmt.Errorf("unable to copy source=%q for zip=%q: %w", f.Name, archivePath, err)
return fmt.Errorf("unable to copy source=%q for zip=%q: %w", file.Name, archivePath, err)
}
results[f.Name] = file.NewOpener(tempFile.Name())
results[file.Name] = Opener{path: tempFile.Name()}
return nil
}
@@ -1,7 +1,7 @@
//go:build !windows
// +build !windows
package archive
package file
import (
"crypto/sha256"
@@ -1,4 +1,4 @@
package archive
package file
import (
"archive/zip"
@@ -1,7 +1,7 @@
//go:build !windows
// +build !windows
package archive
package file
import (
"os"
@@ -3,19 +3,20 @@ package cyclonedxhelpers
import (
"reflect"
"github.com/anchore/syft/syft/file"
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal/formats/common"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
func encodeComponent(p pkg.Package) cyclonedx.Component {
props := encodeProperties(p, "syft:package")
props = append(props, encodeCPEs(p)...)
if len(p.Locations) > 0 {
props = append(props, encodeProperties(p.Locations, "syft:location")...)
locations := p.Locations.ToSlice()
if len(locations) > 0 {
props = append(props, encodeProperties(locations, "syft:location")...)
}
if hasMetadata(p) {
props = append(props, encodeProperties(p.Metadata, "syft:metadata")...)
@@ -39,17 +40,33 @@ func encodeComponent(p pkg.Package) cyclonedx.Component {
Description: encodeDescription(p),
ExternalReferences: encodeExternalReferences(p),
Properties: properties,
BOMRef: deriveBomRef(p),
}
}
func deriveBomRef(p pkg.Package) string {
// try and parse the PURL if possible and append syft id to it, to make
// the purl unique in the BOM.
// TODO: In the future we may want to dedupe by PURL and combine components with
// the same PURL while preserving their unique metadata.
if parsedPURL, err := packageurl.FromString(p.PURL); err == nil {
parsedPURL.Qualifiers = append(parsedPURL.Qualifiers, packageurl.Qualifier{Key: "package-id", Value: string(p.ID())})
return parsedPURL.ToString()
}
// fallback is to use strictly the ID if there is no valid pURL
return string(p.ID())
}
func hasMetadata(p pkg.Package) bool {
return p.Metadata != nil
}
func decodeComponent(c *cyclonedx.Component) *pkg.Package {
values := map[string]string{}
for _, p := range *c.Properties {
values[p.Name] = p.Value
if c.Properties != nil {
for _, p := range *c.Properties {
values[p.Name] = p.Value
}
}
p := &pkg.Package{
@@ -69,13 +86,20 @@ func decodeComponent(c *cyclonedx.Component) *pkg.Package {
p.Type = pkg.TypeFromPURL(p.PURL)
}
if p.Language == "" {
p.Language = pkg.LanguageFromPURL(p.PURL)
}
return p
}
func decodeLocations(vals map[string]string) []file.Location {
v := common.Decode(reflect.TypeOf([]file.Location{}), vals, "syft:location", CycloneDXFields)
out, _ := v.([]file.Location)
return out
func decodeLocations(vals map[string]string) source.LocationSet {
v := common.Decode(reflect.TypeOf([]source.Location{}), vals, "syft:location", CycloneDXFields)
out, ok := v.([]source.Location)
if !ok {
out = nil
}
return source.NewLocationSet(out...)
}
func decodePackageMetadata(vals map[string]string, c *cyclonedx.Component, typ pkg.MetadataType) interface{} {
@@ -1,13 +1,14 @@
package cyclonedxhelpers
import (
"github.com/anchore/syft/syft/file"
"fmt"
"testing"
"github.com/CycloneDX/cyclonedx-go"
"github.com/stretchr/testify/assert"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
func Test_encodeComponentProperties(t *testing.T) {
@@ -26,9 +27,9 @@ func Test_encodeComponentProperties(t *testing.T) {
name: "from apk",
input: pkg.Package{
FoundBy: "cataloger",
Locations: []file.Location{
{Coordinates: file.Coordinates{RealPath: "test"}},
},
Locations: source.NewLocationSet(
source.Location{Coordinates: source.Coordinates{RealPath: "test"}},
),
Metadata: pkg.ApkMetadata{
Package: "libc-utils",
OriginPackage: "libc-dev",
@@ -139,3 +140,82 @@ func Test_encodeComponentProperties(t *testing.T) {
})
}
}
func Test_deriveBomRef(t *testing.T) {
pkgWithPurl := pkg.Package{
Name: "django",
Version: "1.11.1",
PURL: "pkg:pypi/django@1.11.1",
}
pkgWithPurl.SetID()
pkgWithOutPurl := pkg.Package{
Name: "django",
Version: "1.11.1",
PURL: "",
}
pkgWithOutPurl.SetID()
pkgWithBadPurl := pkg.Package{
Name: "django",
Version: "1.11.1",
PURL: "pkg:pyjango@1.11.1",
}
pkgWithBadPurl.SetID()
tests := []struct {
name string
pkg pkg.Package
want string
}{
{
name: "use pURL-id hybrid",
pkg: pkgWithPurl,
want: fmt.Sprintf("pkg:pypi/django@1.11.1?package-id=%s", pkgWithPurl.ID()),
},
{
name: "fallback to ID when pURL is invalid",
pkg: pkgWithBadPurl,
want: string(pkgWithBadPurl.ID()),
},
{
name: "fallback to ID when pURL is missing",
pkg: pkgWithOutPurl,
want: string(pkgWithOutPurl.ID()),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tt.pkg.ID()
assert.Equal(t, tt.want, deriveBomRef(tt.pkg))
})
}
}
func Test_decodeComponent(t *testing.T) {
javaComponentWithNoSyftProperties := cyclonedx.Component{
Name: "ch.qos.logback/logback-classic",
Version: "1.2.3",
PackageURL: "pkg:maven/ch.qos.logback/logback-classic@1.2.3",
Type: "library",
BOMRef: "pkg:maven/ch.qos.logback/logback-classic@1.2.3",
}
tests := []struct {
name string
component cyclonedx.Component
want pkg.Language
}{
{
name: "derive language from pURL if missing",
component: javaComponentWithNoSyftProperties,
want: pkg.Java,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, decodeComponent(&tt.component).Language)
})
}
}
@@ -2,7 +2,6 @@ package cyclonedxhelpers
import (
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/pkg"
@@ -12,7 +11,7 @@ func encodeSingleCPE(p pkg.Package) string {
// Since the CPEs in a package are sorted by specificity
// we can extract the first CPE as the one to output in cyclonedx
if len(p.CPEs) > 0 {
return cpe.String(p.CPEs[0])
return pkg.CPEString(p.CPEs[0])
}
return ""
}
@@ -25,15 +24,15 @@ func encodeCPEs(p pkg.Package) (out []cyclonedx.Property) {
}
out = append(out, cyclonedx.Property{
Name: "syft:cpe23",
Value: cpe.String(c),
Value: pkg.CPEString(c),
})
}
return
}
func decodeCPEs(c *cyclonedx.Component) (out []cpe.CPE) {
func decodeCPEs(c *cyclonedx.Component) (out []pkg.CPE) {
if c.CPE != "" {
cp, err := cpe.New(c.CPE)
cp, err := pkg.NewCPE(c.CPE)
if err != nil {
log.Warnf("invalid CPE: %s", c.CPE)
} else {
@@ -44,7 +43,7 @@ func decodeCPEs(c *cyclonedx.Component) (out []cpe.CPE) {
if c.Properties != nil {
for _, p := range *c.Properties {
if p.Name == "syft:cpe23" {
cp, err := cpe.New(p.Value)
cp, err := pkg.NewCPE(p.Value)
if err != nil {
log.Warnf("invalid CPE: %s", p.Value)
} else {
@@ -1,7 +1,6 @@
package cyclonedxhelpers
import (
"github.com/anchore/syft/syft/cpe"
"testing"
"github.com/stretchr/testify/assert"
@@ -10,8 +9,8 @@ import (
)
func Test_encodeCPE(t *testing.T) {
testCPE := cpe.Must("cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*")
testCPE2 := cpe.Must("cpe:2.3:a:name:name2:3.2:*:*:*:*:*:*:*")
testCPE := pkg.MustCPE("cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*")
testCPE2 := pkg.MustCPE("cpe:2.3:a:name:name2:3.2:*:*:*:*:*:*:*")
tests := []struct {
name string
input pkg.Package
@@ -21,14 +20,14 @@ func Test_encodeCPE(t *testing.T) {
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "no metadata",
input: pkg.Package{
CPEs: []cpe.CPE{},
CPEs: []pkg.CPE{},
},
expected: "",
},
{
name: "single CPE",
input: pkg.Package{
CPEs: []cpe.CPE{
CPEs: []pkg.CPE{
testCPE,
},
},
@@ -37,7 +36,7 @@ func Test_encodeCPE(t *testing.T) {
{
name: "multiple CPEs",
input: pkg.Package{
CPEs: []cpe.CPE{
CPEs: []pkg.CPE{
testCPE2,
testCPE,
},
@@ -5,6 +5,8 @@ import (
"io"
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/syft/internal/formats/common"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
@@ -44,12 +46,12 @@ func GetDecoder(format cyclonedx.BOMFileFormat) sbom.Decoder {
func toSyftModel(bom *cyclonedx.BOM) (*sbom.SBOM, error) {
meta := source.Metadata{}
if bom.Metadata != nil {
if bom.Metadata != nil && bom.Metadata.Component != nil {
meta = decodeMetadata(bom.Metadata.Component)
}
s := &sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(),
PackageCatalog: pkg.NewCatalog(),
LinuxDistribution: linuxReleaseFromComponents(*bom.Components),
},
Source: meta,
@@ -86,7 +88,7 @@ func collectPackages(component *cyclonedx.Component, s *sbom.SBOM, idMap map[str
idMap[component.BOMRef] = p
// TODO there must be a better way than needing to call this manually:
p.SetID()
s.Artifacts.Packages.Add(*p)
s.Artifacts.PackageCatalog.Add(*p)
}
if component.Components != nil {
@@ -157,6 +159,14 @@ func linuxReleaseFromOSComponent(component *cyclonedx.Component) *linux.Release
}
}
if component.Properties != nil {
values := map[string]string{}
for _, p := range *component.Properties {
values[p.Name] = p.Value
}
common.DecodeInto(&rel, values, "syft:distro", CycloneDXFields)
}
return rel
}
@@ -199,7 +209,7 @@ func decodeMetadata(component *cyclonedx.Component) source.Metadata {
switch component.Type {
case cyclonedx.ComponentTypeContainer:
return source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: component.Name,
ID: component.BOMRef,
@@ -208,7 +218,7 @@ func decodeMetadata(component *cyclonedx.Component) source.Metadata {
}
case cyclonedx.ComponentTypeFile:
return source.Metadata{
Scheme: source.FileType, // or source.DirectoryType
Scheme: source.FileScheme, // or source.DirectoryScheme
Path: component.Name,
ImageMetadata: source.ImageMetadata{
UserInput: component.Name,
@@ -212,7 +212,7 @@ func Test_decode(t *testing.T) {
assert.Equal(t, e.ver, sbom.Artifacts.LinuxDistribution.VersionID)
}
if e.pkg != "" {
for p := range sbom.Artifacts.Packages.Enumerate() {
for p := range sbom.Artifacts.PackageCatalog.Enumerate() {
if e.pkg != p.Name {
continue
}
@@ -240,7 +240,7 @@ func Test_decode(t *testing.T) {
if e.relation != "" {
foundRelation := false
for _, r := range sbom.Relationships {
p := sbom.Artifacts.Packages.Package(r.To.ID())
p := sbom.Artifacts.PackageCatalog.Package(r.To.ID())
if e.relation == p.Name {
foundRelation = true
break
@@ -258,3 +258,28 @@ func Test_decode(t *testing.T) {
})
}
}
func Test_missingDataDecode(t *testing.T) {
bom := &cyclonedx.BOM{
Metadata: nil,
Components: &[]cyclonedx.Component{},
}
_, err := toSyftModel(bom)
assert.NoError(t, err)
bom.Metadata = &cyclonedx.Metadata{}
_, err = toSyftModel(bom)
assert.NoError(t, err)
pkg := decodeComponent(&cyclonedx.Component{
Licenses: &cyclonedx.Licenses{
{
License: nil,
},
},
})
assert.Len(t, pkg.Licenses, 0)
}
@@ -4,12 +4,15 @@ import (
"fmt"
"strings"
syftFile "github.com/anchore/syft/syft/file"
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/syft/syft/pkg"
)
//nolint:funlen, gocognit
func encodeExternalReferences(p pkg.Package) *[]cyclonedx.ExternalReference {
refs := []cyclonedx.ExternalReference{}
var refs []cyclonedx.ExternalReference
if hasMetadata(p) {
switch metadata := p.Metadata.(type) {
case pkg.ApkMetadata:
@@ -46,6 +49,19 @@ func encodeExternalReferences(p pkg.Package) *[]cyclonedx.ExternalReference {
Type: cyclonedx.ERTypeWebsite,
})
}
case pkg.JavaMetadata:
if len(metadata.ArchiveDigests) > 0 {
for _, digest := range metadata.ArchiveDigests {
refs = append(refs, cyclonedx.ExternalReference{
URL: "",
Type: cyclonedx.ERTypeBuildMeta,
Hashes: &[]cyclonedx.Hash{{
Algorithm: cyclonedx.HashAlgorithm(digest.Algorithm),
Value: digest.Value,
}},
})
}
}
case pkg.PythonPackageMetadata:
if metadata.DirectURLOrigin != nil && metadata.DirectURLOrigin.URL != "" {
ref := cyclonedx.ExternalReference{
@@ -79,6 +95,20 @@ func decodeExternalReferences(c *cyclonedx.Component, metadata interface{}) {
meta.Homepage = refURL(c, cyclonedx.ERTypeWebsite)
case *pkg.GemMetadata:
meta.Homepage = refURL(c, cyclonedx.ERTypeWebsite)
case *pkg.JavaMetadata:
var digests []syftFile.Digest
if ref := findExternalRef(c, cyclonedx.ERTypeBuildMeta); ref != nil {
if ref.Hashes != nil {
for _, hash := range *ref.Hashes {
digests = append(digests, syftFile.Digest{
Algorithm: string(hash.Algorithm),
Value: hash.Value,
})
}
}
}
meta.ArchiveDigests = digests
case *pkg.PythonPackageMetadata:
if meta.DirectURLOrigin == nil {
meta.DirectURLOrigin = &pkg.PythonDirectURLOriginInfo{}
@@ -25,7 +25,7 @@ func ToFormatModel(s sbom.SBOM) *cyclonedx.BOM {
cdxBOM.SerialNumber = uuid.New().URN()
cdxBOM.Metadata = toBomDescriptor(internal.ApplicationName, versionInfo.Version, s.Source)
packages := s.Artifacts.Packages.Sorted()
packages := s.Artifacts.PackageCatalog.Sorted()
components := make([]cyclonedx.Component, len(packages))
for i, p := range packages {
components[i] = encodeComponent(p)
@@ -155,7 +155,7 @@ func toDependencies(relationships []artifact.Relationship) []cyclonedx.Dependenc
func toBomDescriptorComponent(srcMetadata source.Metadata) *cyclonedx.Component {
switch srcMetadata.Scheme {
case source.ImageType:
case source.ImageScheme:
bomRef, err := artifact.IDByHash(srcMetadata.ImageMetadata.ID)
if err != nil {
log.Warnf("unable to get fingerprint of image metadata=%s: %+v", srcMetadata.ImageMetadata.ID, err)
@@ -166,7 +166,7 @@ func toBomDescriptorComponent(srcMetadata source.Metadata) *cyclonedx.Component
Name: srcMetadata.ImageMetadata.UserInput,
Version: srcMetadata.ImageMetadata.ManifestDigest,
}
case source.DirectoryType, source.FileType:
case source.DirectoryScheme, source.FileScheme:
bomRef, err := artifact.IDByHash(srcMetadata.Path)
if err != nil {
log.Warnf("unable to get fingerprint of source metadata path=%s: %+v", srcMetadata.Path, err)
@@ -26,7 +26,9 @@ func encodeLicenses(p pkg.Package) *cyclonedx.Licenses {
func decodeLicenses(c *cyclonedx.Component) (out []string) {
if c.Licenses != nil {
for _, l := range *c.Licenses {
out = append(out, l.License.ID)
if l.License != nil {
out = append(out, l.License.ID)
}
}
}
return
@@ -10,9 +10,9 @@ import (
func DocumentName(srcMetadata source.Metadata) (string, error) {
switch srcMetadata.Scheme {
case source.ImageType:
case source.ImageScheme:
return cleanName(srcMetadata.ImageMetadata.UserInput), nil
case source.DirectoryType, source.FileType:
case source.DirectoryScheme, source.FileScheme:
return cleanName(srcMetadata.Path), nil
}
@@ -14,7 +14,7 @@ import (
func Test_DocumentName(t *testing.T) {
allSchemes := strset.New()
for _, s := range source.AllTypes {
for _, s := range source.AllSchemes {
allSchemes.Add(string(s))
}
testedSchemes := strset.New()
@@ -29,7 +29,7 @@ func Test_DocumentName(t *testing.T) {
name: "image",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "image-repo/name:tag",
ID: "id",
@@ -42,7 +42,7 @@ func Test_DocumentName(t *testing.T) {
name: "directory",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.DirectoryType,
Scheme: source.DirectoryScheme,
Path: "some/path/to/place",
},
expected: "some/path/to/place",
@@ -51,7 +51,7 @@ func Test_DocumentName(t *testing.T) {
name: "file",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.FileType,
Scheme: source.FileScheme,
Path: "some/path/to/place",
},
expected: "some/path/to/place",
@@ -21,11 +21,11 @@ func DocumentNameAndNamespace(srcMetadata source.Metadata) (string, string, erro
func DocumentNamespace(name string, srcMetadata source.Metadata) string {
input := "unknown-source-type"
switch srcMetadata.Scheme {
case source.ImageType:
case source.ImageScheme:
input = "image"
case source.DirectoryType:
case source.DirectoryScheme:
input = "dir"
case source.FileType:
case source.FileScheme:
input = "file"
}
@@ -12,7 +12,7 @@ import (
func Test_documentNamespace(t *testing.T) {
allSchemes := strset.New()
for _, s := range source.AllTypes {
for _, s := range source.AllSchemes {
allSchemes.Add(string(s))
}
testedSchemes := strset.New()
@@ -27,7 +27,7 @@ func Test_documentNamespace(t *testing.T) {
name: "image",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "image-repo/name:tag",
ID: "id",
@@ -40,7 +40,7 @@ func Test_documentNamespace(t *testing.T) {
name: "directory",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.DirectoryType,
Scheme: source.DirectoryScheme,
Path: "some/path/to/place",
},
expected: "https://anchore.com/syft/dir/my-name-",
@@ -49,7 +49,7 @@ func Test_documentNamespace(t *testing.T) {
name: "file",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.FileType,
Scheme: source.FileScheme,
Path: "some/path/to/place",
},
expected: "https://anchore.com/syft/file/my-name-",
@@ -1,7 +1,6 @@
package spdxhelpers
import (
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/pkg"
)
@@ -11,7 +10,7 @@ func ExternalRefs(p pkg.Package) (externalRefs []ExternalRef) {
for _, c := range p.CPEs {
externalRefs = append(externalRefs, ExternalRef{
ReferenceCategory: SecurityReferenceCategory,
ReferenceLocator: cpe.String(c),
ReferenceLocator: pkg.CPEString(c),
ReferenceType: Cpe23ExternalRefType,
})
}
@@ -23,5 +22,6 @@ func ExternalRefs(p pkg.Package) (externalRefs []ExternalRef) {
ReferenceType: PurlExternalRefType,
})
}
return externalRefs
}
@@ -1,7 +1,6 @@
package spdxhelpers
import (
"github.com/anchore/syft/syft/cpe"
"testing"
"github.com/anchore/syft/syft/pkg"
@@ -9,7 +8,7 @@ import (
)
func Test_ExternalRefs(t *testing.T) {
testCPE := cpe.Must("cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*")
testCPE := pkg.MustCPE("cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*")
tests := []struct {
name string
input pkg.Package
@@ -18,7 +17,7 @@ func Test_ExternalRefs(t *testing.T) {
{
name: "cpe + purl",
input: pkg.Package{
CPEs: []cpe.CPE{
CPEs: []pkg.CPE{
testCPE,
},
PURL: "a-purl",
@@ -26,7 +25,7 @@ func Test_ExternalRefs(t *testing.T) {
expected: []ExternalRef{
{
ReferenceCategory: SecurityReferenceCategory,
ReferenceLocator: cpe.String(testCPE),
ReferenceLocator: pkg.CPEString(testCPE),
ReferenceType: Cpe23ExternalRefType,
},
{
@@ -13,6 +13,8 @@ func SourceInfo(p pkg.Package) string {
answer = "acquired package info from RPM DB"
case pkg.ApkPkg:
answer = "acquired package info from APK DB"
case pkg.DartPubPkg:
answer = "acquired package info from pubspec manifest"
case pkg.DebPkg:
answer = "acquired package info from DPKG DB"
case pkg.NpmPkg:
@@ -33,7 +35,7 @@ func SourceInfo(p pkg.Package) string {
answer = "acquired package info from the following paths"
}
var paths []string
for _, l := range p.Locations {
for _, l := range p.Locations.ToSlice() {
paths = append(paths, l.RealPath)
}
@@ -1,10 +1,10 @@
package spdxhelpers
import (
"github.com/anchore/syft/syft/file"
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/stretchr/testify/assert"
)
@@ -18,10 +18,10 @@ func Test_SourceInfo(t *testing.T) {
name: "locations are captured",
input: pkg.Package{
// note: no type given
Locations: []file.Location{
file.NewVirtualLocation("/a-place", "/b-place"),
file.NewVirtualLocation("/c-place", "/d-place"),
},
Locations: source.NewLocationSet(
source.NewVirtualLocation("/a-place", "/b-place"),
source.NewVirtualLocation("/c-place", "/d-place"),
),
},
expected: []string{
"from the following paths",
@@ -126,6 +126,14 @@ func Test_SourceInfo(t *testing.T) {
"from PHP composer manifest",
},
},
{
input: pkg.Package{
Type: pkg.DartPubPkg,
},
expected: []string{
"from pubspec manifest",
},
},
}
var pkgTypes []pkg.Type
for _, test := range tests {
@@ -0,0 +1,13 @@
package spdxhelpers
import (
"regexp"
)
var expr = regexp.MustCompile("[^a-zA-Z0-9.-]")
// SPDX spec says SPDXID must be:
// "SPDXRef-"[idstring] where [idstring] is a unique string containing letters, numbers, ., and/or -
func SanitizeElementID(id string) string {
return expr.ReplaceAllString(id, "-")
}
@@ -0,0 +1,39 @@
package spdxhelpers
import (
"testing"
"github.com/stretchr/testify/assert"
)
func Test_SanitizeElementID(t *testing.T) {
tests := []struct {
input string
expected string
}{
{
input: "letters",
expected: "letters",
},
{
input: "ssl-client",
expected: "ssl-client",
},
{
input: "ssl_client",
expected: "ssl-client",
},
{
input: "go-module-sigs.k8s.io/structured-merge-diff/v3",
expected: "go-module-sigs.k8s.io-structured-merge-diff-v3",
},
}
for _, test := range tests {
t.Run(test.input, func(t *testing.T) {
actual := SanitizeElementID(test.input)
assert.Equal(t, test.expected, actual)
})
}
}
@@ -1,7 +1,7 @@
package spdxhelpers
import (
"github.com/anchore/syft/syft/cpe"
"errors"
"strconv"
"strings"
@@ -14,16 +14,21 @@ import (
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
func ToSyftModel(doc *spdx.Document2_2) (*sbom.SBOM, error) {
if doc == nil {
return nil, errors.New("cannot convert SPDX document to Syft model because document is nil")
}
spdxIDMap := make(map[string]interface{})
s := &sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(),
FileMetadata: map[file.Coordinates]file.Metadata{},
FileDigests: map[file.Coordinates][]file.Digest{},
PackageCatalog: pkg.NewCatalog(),
FileMetadata: map[source.Coordinates]source.FileMetadata{},
FileDigests: map[source.Coordinates][]file.Digest{},
LinuxDistribution: findLinuxReleaseByPURL(doc),
},
}
@@ -74,7 +79,7 @@ func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *sp
for _, p := range doc.Packages {
syftPkg := toSyftPackage(p)
spdxIDMap[string(p.PackageSPDXIdentifier)] = syftPkg
s.Artifacts.Packages.Add(*syftPkg)
s.Artifacts.PackageCatalog.Add(*syftPkg)
}
}
@@ -98,7 +103,7 @@ func toFileDigests(f *spdx.File2_2) (digests []file.Digest) {
return digests
}
func toFileMetadata(f *spdx.File2_2) (meta file.Metadata) {
func toFileMetadata(f *spdx.File2_2) (meta source.FileMetadata) {
// FIXME Syft is currently lossy due to the SPDX 2.2.1 spec not supporting arbitrary mimetypes
for _, typ := range f.FileType {
switch FileType(typ) {
@@ -132,7 +137,7 @@ func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document2_2
b := spdxIDMap[string(r.RefB.ElementRefID)]
from, fromOk := a.(*pkg.Package)
toPackage, toPackageOk := b.(*pkg.Package)
toLocation, toLocationOk := b.(*file.Location)
toLocation, toLocationOk := b.(*source.Location)
if !fromOk || !(toPackageOk || toLocationOk) {
log.Debugf("unable to find valid relationship mapping from SPDX 2.2 JSON, ignoring: (from: %+v) (to: %+v)", a, b)
continue
@@ -174,7 +179,7 @@ func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document2_2
return out
}
func toSyftCoordinates(f *spdx.File2_2) file.Coordinates {
func toSyftCoordinates(f *spdx.File2_2) source.Coordinates {
const layerIDPrefix = "layerID: "
var fileSystemID string
if strings.Index(f.FileComment, layerIDPrefix) == 0 {
@@ -183,16 +188,16 @@ func toSyftCoordinates(f *spdx.File2_2) file.Coordinates {
if strings.Index(string(f.FileSPDXIdentifier), layerIDPrefix) == 0 {
fileSystemID = strings.TrimPrefix(string(f.FileSPDXIdentifier), layerIDPrefix)
}
return file.Coordinates{
return source.Coordinates{
RealPath: f.FileName,
FileSystemID: fileSystemID,
}
}
func toSyftLocation(f *spdx.File2_2) *file.Location {
return &file.Location{
func toSyftLocation(f *spdx.File2_2) *source.Location {
return &source.Location{
Coordinates: toSyftCoordinates(f),
AccessPath: f.FileName,
VirtualPath: f.FileName,
}
}
@@ -304,6 +309,14 @@ func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interf
Architecture: arch,
Maintainer: p.PackageOriginatorPerson,
}
case pkg.JavaPkg:
var digests []file.Digest
for algorithm, value := range p.PackageChecksums {
digests = append(digests, file.Digest{Algorithm: string(algorithm), Value: value.Value})
}
return pkg.JavaMetadataType, pkg.JavaMetadata{
ArchiveDigests: digests,
}
}
return pkg.UnknownMetadataType, nil
}
@@ -317,10 +330,10 @@ func findPURLValue(p *spdx.Package2_2) string {
return ""
}
func extractCPEs(p *spdx.Package2_2) (cpes []cpe.CPE) {
func extractCPEs(p *spdx.Package2_2) (cpes []pkg.CPE) {
for _, r := range p.PackageExternalReferences {
if r.RefType == string(Cpe23ExternalRefType) {
cpe, err := cpe.New(r.Locator)
cpe, err := pkg.NewCPE(r.Locator)
if err != nil {
log.Warnf("unable to extract SPDX CPE=%q: %+v", r.Locator, err)
continue
@@ -94,7 +94,7 @@ func TestToSyftModel(t *testing.T) {
assert.NotNil(t, sbom)
pkgs := sbom.Artifacts.Packages.Sorted()
pkgs := sbom.Artifacts.PackageCatalog.Sorted()
assert.Len(t, pkgs, 2)
+30 -33
View File
@@ -2,12 +2,9 @@ package testutils
import (
"bytes"
"github.com/anchore/syft/syft/cpe"
"strings"
"testing"
"github.com/anchore/syft/syft/file"
"github.com/anchore/go-testutils"
"github.com/anchore/stereoscope/pkg/filetree"
"github.com/anchore/stereoscope/pkg/image"
@@ -100,7 +97,7 @@ func AssertEncoderAgainstGoldenSnapshot(t *testing.T, format sbom.Format, sbom s
func ImageInput(t testing.TB, testImage string, options ...ImageOption) sbom.SBOM {
t.Helper()
catalog := pkg.NewCollection()
catalog := pkg.NewCatalog()
var cfg imageCfg
var img *image.Image
for _, opt := range options {
@@ -124,7 +121,7 @@ func ImageInput(t testing.TB, testImage string, options ...ImageOption) sbom.SBO
return sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: catalog,
PackageCatalog: catalog,
LinuxDistribution: &linux.Release{
PrettyName: "debian",
Name: "debian",
@@ -152,7 +149,7 @@ func carriageRedactor(s []byte) []byte {
return []byte(msg)
}
func populateImageCatalog(catalog pkg.Collection, img *image.Image) {
func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
_, ref1, _ := img.SquashedTree().File("/somefile-1.txt", filetree.FollowBasenameLinks)
_, ref2, _ := img.SquashedTree().File("/somefile-2.txt", filetree.FollowBasenameLinks)
@@ -160,9 +157,9 @@ func populateImageCatalog(catalog pkg.Collection, img *image.Image) {
catalog.Add(pkg.Package{
Name: "package-1",
Version: "1.0.1",
Locations: []file.Location{
file.NewLocationFromImage(string(ref1.RealPath), *ref1, img),
},
Locations: source.NewLocationSet(
source.NewLocationFromImage(string(ref1.RealPath), *ref1, img),
),
Type: pkg.PythonPkg,
FoundBy: "the-cataloger-1",
Language: pkg.Python,
@@ -172,17 +169,17 @@ func populateImageCatalog(catalog pkg.Collection, img *image.Image) {
Name: "package-1",
Version: "1.0.1",
},
PURL: "a-purl-1",
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"),
PURL: "a-purl-1", // intentionally a bad pURL for test fixtures
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"),
},
})
catalog.Add(pkg.Package{
Name: "package-2",
Version: "2.0.1",
Locations: []file.Location{
file.NewLocationFromImage(string(ref2.RealPath), *ref2, img),
},
Locations: source.NewLocationSet(
source.NewLocationFromImage(string(ref2.RealPath), *ref2, img),
),
Type: pkg.DebPkg,
FoundBy: "the-cataloger-2",
MetadataType: pkg.DpkgMetadataType,
@@ -190,9 +187,9 @@ func populateImageCatalog(catalog pkg.Collection, img *image.Image) {
Package: "package-2",
Version: "2.0.1",
},
PURL: "a-purl-2",
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
PURL: "pkg:deb/debian/package-2@2.0.1",
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
})
}
@@ -205,7 +202,7 @@ func DirectoryInput(t testing.TB) sbom.SBOM {
return sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: catalog,
PackageCatalog: catalog,
LinuxDistribution: &linux.Release{
PrettyName: "debian",
Name: "debian",
@@ -228,8 +225,8 @@ func DirectoryInput(t testing.TB) sbom.SBOM {
}
}
func newDirectoryCatalog() pkg.Collection {
catalog := pkg.NewCollection()
func newDirectoryCatalog() *pkg.Catalog {
catalog := pkg.NewCatalog()
// populate catalog with test data
catalog.Add(pkg.Package{
@@ -237,9 +234,9 @@ func newDirectoryCatalog() pkg.Collection {
Version: "1.0.1",
Type: pkg.PythonPkg,
FoundBy: "the-cataloger-1",
Locations: []file.Location{
file.NewLocation("/some/path/pkg1"),
},
Locations: source.NewLocationSet(
source.NewLocation("/some/path/pkg1"),
),
Language: pkg.Python,
MetadataType: pkg.PythonPackageMetadataType,
Licenses: []string{"MIT"},
@@ -252,9 +249,9 @@ func newDirectoryCatalog() pkg.Collection {
},
},
},
PURL: "a-purl-2",
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
PURL: "a-purl-2", // intentionally a bad pURL for test fixtures
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
})
catalog.Add(pkg.Package{
@@ -262,17 +259,17 @@ func newDirectoryCatalog() pkg.Collection {
Version: "2.0.1",
Type: pkg.DebPkg,
FoundBy: "the-cataloger-2",
Locations: []file.Location{
file.NewLocation("/some/path/pkg1"),
},
Locations: source.NewLocationSet(
source.NewLocation("/some/path/pkg1"),
),
MetadataType: pkg.DpkgMetadataType,
Metadata: pkg.DpkgMetadata{
Package: "package-2",
Version: "2.0.1",
},
PURL: "a-purl-2",
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
PURL: "pkg:deb/debian/package-2@2.0.1",
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
})
@@ -57,7 +57,7 @@ func Test_decodeJSON(t *testing.T) {
split = strings.SplitN(pkg, ":", 2)
name = split[0]
version = split[1]
for p := range bom.Artifacts.Packages.Enumerate() {
for p := range bom.Artifacts.PackageCatalog.Enumerate() {
if p.Name == name {
assert.Equal(t, version, p.Version)
continue pkgs
@@ -1,10 +1,10 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"serialNumber": "urn:uuid:498e659b-0758-4a7f-816e-91bee18df634",
"serialNumber": "urn:uuid:dec3f6b4-8458-48bb-b60d-dfd312f6ec4e",
"version": 1,
"metadata": {
"timestamp": "2022-03-08T12:30:39Z",
"timestamp": "2022-04-01T11:48:04-04:00",
"tools": [
{
"vendor": "anchore",
@@ -20,6 +20,7 @@
},
"components": [
{
"bom-ref": "b85dbb4e6ece5082",
"type": "library",
"name": "package-1",
"version": "1.0.1",
@@ -56,11 +57,12 @@
]
},
{
"bom-ref": "pkg:deb/debian/package-2@2.0.1?package-id=ceda99598967ae8d",
"type": "library",
"name": "package-2",
"version": "2.0.1",
"cpe": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"purl": "a-purl-2",
"purl": "pkg:deb/debian/package-2@2.0.1",
"properties": [
{
"name": "syft:package:foundBy",
@@ -1,10 +1,10 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"serialNumber": "urn:uuid:342c3d2c-d26e-47b6-94d6-92fbf41da945",
"serialNumber": "urn:uuid:054d973e-fe99-4762-92e4-eaf01997ae41",
"version": 1,
"metadata": {
"timestamp": "2022-03-08T12:30:39Z",
"timestamp": "2022-04-01T11:48:04-04:00",
"tools": [
{
"vendor": "anchore",
@@ -13,7 +13,7 @@
}
],
"component": {
"bom-ref": "711095b1cdf90cce",
"bom-ref": "e777314b02b362e4",
"type": "container",
"name": "user-image-input",
"version": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
@@ -21,6 +21,7 @@
},
"components": [
{
"bom-ref": "2a46171f91c8d4bc",
"type": "library",
"name": "package-1",
"version": "1.0.1",
@@ -52,7 +53,7 @@
},
{
"name": "syft:location:0:layerID",
"value": "sha256:16e64541f2ddf59a90391ce7bb8af90313f7d373f2105d88f3d3267b72e0ebab"
"value": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59"
},
{
"name": "syft:location:0:path",
@@ -61,11 +62,12 @@
]
},
{
"bom-ref": "pkg:deb/debian/package-2@2.0.1?package-id=ae77680e9b1d087e",
"type": "library",
"name": "package-2",
"version": "2.0.1",
"cpe": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"purl": "a-purl-2",
"purl": "pkg:deb/debian/package-2@2.0.1",
"properties": [
{
"name": "syft:package:foundBy",
@@ -81,7 +83,7 @@
},
{
"name": "syft:location:0:layerID",
"value": "sha256:de6c235f76ea24c8503ec08891445b5d6a8bdf8249117ed8d8b0b6fb3ebe4f67"
"value": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec"
},
{
"name": "syft:location:0:path",
@@ -57,7 +57,7 @@ func Test_decodeXML(t *testing.T) {
split = strings.SplitN(pkg, ":", 2)
name = split[0]
version = split[1]
for p := range bom.Artifacts.Packages.Enumerate() {
for p := range bom.Artifacts.PackageCatalog.Enumerate() {
if p.Name == name {
assert.Equal(t, version, p.Version)
continue pkgs
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:892f8304-0142-45b1-b411-cade3c53057f" version="1">
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:554fd820-210b-40c8-8c0b-75690274e21c" version="1">
<metadata>
<timestamp>2022-03-08T12:30:33Z</timestamp>
<timestamp>2022-04-01T11:57:46-04:00</timestamp>
<tools>
<tool>
<vendor>anchore</vendor>
@@ -14,7 +14,7 @@
</component>
</metadata>
<components>
<component type="library">
<component bom-ref="b85dbb4e6ece5082" type="library">
<name>package-1</name>
<version>1.0.1</version>
<licenses>
@@ -32,11 +32,11 @@
<property name="syft:location:0:path">/some/path/pkg1</property>
</properties>
</component>
<component type="library">
<component bom-ref="pkg:deb/debian/package-2@2.0.1?package-id=ceda99598967ae8d" type="library">
<name>package-2</name>
<version>2.0.1</version>
<cpe>cpe:2.3:*:some:package:2:*:*:*:*:*:*:*</cpe>
<purl>a-purl-2</purl>
<purl>pkg:deb/debian/package-2@2.0.1</purl>
<properties>
<property name="syft:package:foundBy">the-cataloger-2</property>
<property name="syft:package:metadataType">DpkgMetadata</property>
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:5fa94827-eb85-4f32-a62d-76fb6e89a2dd" version="1">
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:1535f940-172f-4d97-8280-d5a5764d1557" version="1">
<metadata>
<timestamp>2022-03-08T12:30:33Z</timestamp>
<timestamp>2022-04-01T11:57:46-04:00</timestamp>
<tools>
<tool>
<vendor>anchore</vendor>
@@ -9,13 +9,13 @@
<version>[not provided]</version>
</tool>
</tools>
<component bom-ref="711095b1cdf90cce" type="container">
<component bom-ref="e777314b02b362e4" type="container">
<name>user-image-input</name>
<version>sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368</version>
</component>
</metadata>
<components>
<component type="library">
<component bom-ref="2a46171f91c8d4bc" type="library">
<name>package-1</name>
<version>1.0.1</version>
<licenses>
@@ -30,20 +30,20 @@
<property name="syft:package:language">python</property>
<property name="syft:package:metadataType">PythonPackageMetadata</property>
<property name="syft:package:type">python</property>
<property name="syft:location:0:layerID">sha256:16e64541f2ddf59a90391ce7bb8af90313f7d373f2105d88f3d3267b72e0ebab</property>
<property name="syft:location:0:layerID">sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59</property>
<property name="syft:location:0:path">/somefile-1.txt</property>
</properties>
</component>
<component type="library">
<component bom-ref="pkg:deb/debian/package-2@2.0.1?package-id=ae77680e9b1d087e" type="library">
<name>package-2</name>
<version>2.0.1</version>
<cpe>cpe:2.3:*:some:package:2:*:*:*:*:*:*:*</cpe>
<purl>a-purl-2</purl>
<purl>pkg:deb/debian/package-2@2.0.1</purl>
<properties>
<property name="syft:package:foundBy">the-cataloger-2</property>
<property name="syft:package:metadataType">DpkgMetadata</property>
<property name="syft:package:type">deb</property>
<property name="syft:location:0:layerID">sha256:de6c235f76ea24c8503ec08891445b5d6a8bdf8249117ed8d8b0b6fb3ebe4f67</property>
<property name="syft:location:0:layerID">sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec</property>
<property name="syft:location:0:path">/somefile-2.txt</property>
<property name="syft:metadata:installedSize">0</property>
</properties>

Some files were not shown because too many files have changed in this diff Show More