Compare commits

...
52 Commits
Author SHA1 Message Date
Sambhav KothariandChristopher Phillips 36973021fa Rename syft-id to package-id (#970)
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-04-29 11:18:45 -04:00
Christian Köberl 7d8ea39ee5 update to cyclonedx-go 0.5.2 (#971) 2022-04-28 10:42:12 -04:00
Christopher Angelo Phillips 6029dd7c2e refactor command package to remove globals and add dependency injection 2022-04-26 18:23:03 +00:00
Jon McEwen 7304bbf8ee fix: #953 Derive language from pURL - https://github.com/anchore/syft… (#957)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-04-26 11:51:24 -04:00
Rob Dimsdale-Zucker c270ee2a02 Fix typo in CPE-parsing error (#966) 2022-04-22 12:08:44 -04:00
Alex Goodman 172ecc0d77 Preserve syft IDs on SBOM decode (#963) 2022-04-18 18:10:55 +00:00
Keith Zantow 248023baaf Update GitHub format package_url and correlator (#961) 2022-04-15 13:00:06 -04:00
Keith Zantow b7295b79de Ensure SPDXIDs are valid (#955) 2022-04-14 15:07:23 -04:00
Keith Zantow 321eddf874 Auto-PR needs to run go mod tidy (#958) 2022-04-13 16:30:35 -04:00
Keith Zantow 25bf679f8f Add workflow for automatic PR for new stereoscope updates (#954) 2022-04-13 13:20:40 -04:00
Keith Zantow 02a8fb6f8c Minor readme update to correct format information (#948) 2022-04-12 17:16:47 -04:00
Christopher Angelo Phillips b46d044d7e Update spdx22json to only take uppercase checksum algorithm (#946) 2022-04-11 14:56:04 -04:00
Weston Steimel 15e45a8ce1 add additional vendors for springframework (#945)
The Official CPE dictionary currently contains entries for springframework with three different vendors: springsource, vmware, and pivotal_software.  This appears to be because ownership has changed over time.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2022-04-11 14:38:52 +01:00
Christopher Angelo Phillips 782b2e3348 Add digest property to parent and nested java package metadata (#941) 2022-04-08 15:12:32 -04:00
Alex Goodman e415bb21e7 Update write permissions and log into ghcr.io for release (#942) 2022-04-06 21:15:55 +00:00
Alex Goodman 748cfbf006 Retry auth URL lookup without docker credentialhelper workaround (#939) 2022-04-06 16:27:13 +00:00
Sambhav KothariandAlex Goodman 8bc5d84481 Ensure that all cyclonedx components have bom-refs (#914)
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-04-01 12:19:30 -04:00
Alex Goodman 68b7ad9770 Additionally publish docker images to GHCR (#934) 2022-04-01 11:30:21 -04:00
Oscar Hallgren 1aeda6bb50 use filepath.Base() instead of path.Base() for temp files (#882) 2022-04-01 10:42:22 -04:00
Alex Goodman f24bbc1838 Deduplicate packages across multiple container image layers (#930) 2022-03-31 15:45:51 -04:00
Eric LarssenandAlex Goodman cb3e73e308 Add dart support (#919)
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-03-31 15:44:55 -04:00
Alex Goodman f157d7a862 Pull from DockerHub fails for public images when using SSO (#928) 2022-03-30 17:32:49 +00:00
Dan Luhring 028cd9e27e Fix nil pointer dereference in directory resolver's indexPath method (#924)
* Add failing test for dir resolver panic

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Fix panic

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-28 13:15:09 -04:00
Dan Luhring 5549939cc6 Fixups and clarifications in README (#920)
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-25 11:36:43 -04:00
briankoe741 47ea910868 Remove announcement for OSS Meetup (#915)
Proposing changes to remove our 3/23 meetup

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-25 00:17:14 +00:00
Dan Luhring a7db43f5ec Fix panic on empty sbom (#917)
* Implement fmt.Stringer with format.ID

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Add failing test for formats processing empty SBOMs

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Account for nil SPDX document during Syft model conversion

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-24 10:11:51 -04:00
Alex Goodman cc2c0e57a0 bump strset version to fix 386 builds (#911) 2022-03-23 14:34:54 -04:00
Alex Goodman 5253da4b36 Rollback referencing docker config items (#912) 2022-03-23 18:33:41 +00:00
Jonas XavierandAlex Goodman c0b547bdb2 Less verbose logging in Golang Cataloger (#904)
* Less verbose logging in Golang Cataloger

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* debug for known gray errors

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* only show warnings when a binary is not a go executable

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-03-22 10:19:18 -07:00
Alex Goodman cffcaf5984 Improve docker config support (#906) 2022-03-22 11:02:54 -04:00
Alex Goodman 7f9edf346a Bump golangci-lint to 1.45.0 (#909) 2022-03-22 11:02:36 -04:00
j-k a644a45ef4 Correct go.mod to enforce go 1.18 (#897)
Since syft now depends on debug/buildinfo go 1.18 is required to build
syft and as such go.mod needs updating

Signed-off-by: 06kellyjac <jack@control-plane.io>
2022-03-21 15:38:32 -04:00
Jonas Xavier 283db88dc4 Omit H1Digest when empty (#902)
* Omit HD1Field when empty

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update test-fixtures

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2022-03-21 11:59:10 -07:00
Alex Goodman 03e193e577 Add platform option to the README (#889) 2022-03-21 12:02:15 -04:00
Alex Goodman 069aa68b63 Fix image cleanup when there is an error (#905) 2022-03-21 14:48:11 +00:00
Keith Zantow 9240860f44 Correct ID handling during Syft JSON decoding (#900) 2022-03-18 17:03:26 -04:00
Christopher Angelo Phillips 4231f38fa2 add case to decode GolangBinMetadata for syftjson model (#901)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-03-18 15:15:10 -04:00
Dan Luhring 752b03b2d6 Remove commit signing requirement (#899)
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-18 12:48:00 -04:00
Keith Zantow 99c3339810 Fix CycloneDX license decoding panic (#898) 2022-03-18 09:44:51 -04:00
Keith Zantow f4734d28b3 Fix panic when CycloneDX BOM missing metadata.component (#895) 2022-03-17 10:22:35 -04:00
Jonas Xavier 6ef3e45ffc Use go 1.18 buildinfo to catalog binaries (#827)
* initial working version

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* added build settings to pkg metadata

wip - unit tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* handle mach-O FatFiles

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add support to mod replace

fixed golang catalger tests

trying GH Actions with go 1.18rc1

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* log error

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* use go-macholibre for extraction

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* cleaner tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add version to main module

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* check macho file with macholibre

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* run golangci in its own workflow

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci workflow

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix golangci wf yml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix golangci wf yml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci wf

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci wf

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* get arch from bin file headers

upgrade macholibre

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* test new stereoscope lazy reader interface

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove devel version from golang cataloger

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* switch github workflows to go1.18 stable

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add union reader interface in golang cataloger

update stereoscope

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* simpler golangci validation

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix makefile

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* get archs refactor

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* nolint for golang version

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix go bin tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* feedback changes

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* golangci nolint needs a \n before package

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* cleanup

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* move golangci-lint to its own jobs again

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix ci yaml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add support for xcoff files

add arch assets to test bin file types

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* clean up golangci-lint config

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* nolint for xcoff

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explain nolints

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove unused xcoff testdata assets

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* make go bin test-fixtures in docker

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix make clean with -f

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update json output schema

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update schema version in test fixture

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* feedback changes

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explain possible empty main module

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2022-03-16 17:07:02 -07:00
Keith Zantow ee0a1d172c panic parsing cyclonedx (#892) 2022-03-16 09:10:44 -04:00
Keith Zantow b9b3ccecf9 Update register link text (#891) 2022-03-15 17:02:33 +00:00
mikey strauss 95271fb10d NPM PURLs are invalid (#832)
Signed-off-by: houdini91 <mdstrauss91@gmail.com>
2022-03-15 11:54:33 -04:00
Christopher Angelo Phillips 93d2d57cd3 update README with OSS Meetup information (#890)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-03-15 11:37:56 -04:00
Kenny Moens 7cd3201fe9 Support the .be top-level domain (#873)
Signed-off-by: Kenny Moens <kenny.moens@cipalschaubroeck.be>
2022-03-15 10:59:13 -04:00
Keith Zantow c7cf8b0b26 Brew install gon failed (#885) 2022-03-11 13:17:57 -05:00
Frankie G-J 44a6e00f7a Include vendored modules in Go Module package list (#883)
* include vendored modules in package slice

Signed-off-by: Frankie Gallina-Jones <frankieg@vmware.com>

* add explanatory comments

Signed-off-by: Frankie Gallina-Jones <frankieg@vmware.com>
2022-03-11 12:57:33 -05:00
Keith Zantow 6c8102bf28 Correct CycloneDX distro decoding (#745) 2022-03-11 09:27:18 -05:00
Keith Zantow 7789506dc6 Experimental GitHub export (#836) 2022-03-10 22:38:12 -05:00
Christopher Angelo Phillips fa03723617 Upgrade vault api from v1.3.1 to v1.4.1 (#878)
* move v1.3.1 => v1.4.1
* go mod tidy

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-03-09 15:11:07 -05:00
Alex Goodman 2946813a74 RPM Epoch should be optional in the json schema (#880) 2022-03-09 14:51:43 -05:00
196 changed files with 11668 additions and 3260 deletions
+1
View File
@@ -5,6 +5,7 @@ permit:
- MPL.*
- ISC
ignore-packages:
- .
# packageurl-go is released under the MIT license located in the root of the repo at /mit.LICENSE
- github.com/anchore/packageurl-go
+1 -1
View File
@@ -29,7 +29,7 @@ notarize() {
fi
# install gon
which gon || (brew tap mitchellh/gon && brew install mitchellh/gon/gon)
which gon || (go install github.com/mitchellh/gon/cmd/gon@latest)
# create config (note: json via stdin with gon is broken, can only use HCL from file)
hcl_file=$(mktemp).hcl
@@ -21,6 +21,14 @@ fi
if [ -z "$DOCKER_PASSWORD" ]; then
exit_with_error "DOCKER_PASSWORD not set"
fi
if [ -z "$GHCR_USERNAME" ]; then
exit_with_error "GHCR_USERNAME not set"
fi
if [ -z "$GHCR_PASSWORD" ]; then
exit_with_error "GHCR_PASSWORD not set"
fi
set -u
# setup_signing
@@ -41,4 +49,5 @@ setup_signing() {
commentary "log into docker -- required for publishing (since the default keychain has now been replaced)"
echo "${DOCKER_PASSWORD}" | docker login docker.io -u "${DOCKER_USERNAME}" --password-stdin
echo "${GHCR_PASSWORD}" | docker login ghcr.io -u "${GHCR_USERNAME}" --password-stdin
}
+5 -1
View File
@@ -9,7 +9,7 @@ on:
- "v*"
env:
GO_VERSION: "1.17.x"
GO_VERSION: "1.18.x"
jobs:
quality-gate:
@@ -94,6 +94,8 @@ jobs:
needs: [quality-gate]
# due to our code signing process, it's vital that we run our release steps on macOS
runs-on: macos-latest
permissions:
packages: write
steps:
- uses: docker-practice/actions-setup-docker@v1
@@ -128,6 +130,8 @@ jobs:
- name: Build & publish release artifacts
run: make release
env:
GHCR_USERNAME: ${{ github.actor }}
GHCR_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
DOCKER_USERNAME: ${{ secrets.TOOLBOX_DOCKER_USER }}
DOCKER_PASSWORD: ${{ secrets.TOOLBOX_DOCKER_PASS }}
# we use a different token than GITHUB_SECRETS to additionally allow updating the homebrew repos
@@ -0,0 +1,51 @@
name: PR for latest Stereoscope release
on:
schedule:
- cron: "0 8 * * *" # 3 AM EST
workflow_dispatch:
env:
GO_VERSION: "1.18.x"
GO_STABLE_VERSION: true
jobs:
upgrade-stereoscope:
runs-on: ubuntu-latest
if: github.repository == 'anchore/syft' # only run for main repo
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- run: |
LATEST_VERSION=$(git ls-remote https://github.com/anchore/stereoscope main | head -n1 | awk '{print $1;}')
# update go.mod
go get github.com/anchore/stereoscope@$LATEST_VERSION
go mod tidy
# export the version for use with create-pull-request
echo "::set-output name=LATEST_VERSION::$LATEST_VERSION"
id: latest-version
- uses: tibdex/github-app-token@v1
id: generate-token
with:
app_id: ${{ secrets.TOKEN_APP_ID }}
private_key: ${{ secrets.TOKEN_APP_PRIVATE_KEY }}
- uses: peter-evans/create-pull-request@v4
with:
signoff: true
delete-branch: true
branch: auto/latest
labels: dependencies
commit-message: "Update Stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}"
title: "Update Stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}"
body: |
Update Stereoscope to ${{ steps.latest-version.outputs.LATEST_VERSION }}
token: ${{ steps.generate-token.outputs.token }}
+18 -1
View File
@@ -7,7 +7,8 @@ on:
pull_request:
env:
GO_VERSION: "1.17.x"
GO_VERSION: "1.18.x"
GO_STABLE_VERSION: true
jobs:
@@ -19,6 +20,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -56,6 +58,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -92,6 +95,16 @@ jobs:
path: syft/pkg/cataloger/java/test-fixtures/java-builds/packages
key: ${{ runner.os }}-unit-java-cache-${{ hashFiles( 'syft/pkg/cataloger/java/test-fixtures/java-builds/packages.fingerprint' ) }}
- name: Build cache key for go binary test-fixture blobs (for unit tests)
run: make go-binaries-fingerprint
- name: Restore Go binary test-fixture cache
id: unit-go-binary-cache
uses: actions/cache@v2.1.3
with:
path: syft/pkg/cataloger/golang/test-fixtures/archs/binaries
key: ${{ runner.os }}-unit-go-binaries-cache-${{ hashFiles( 'syft/pkg/cataloger/golang/test-fixtures/archs/binaries.fingerprint' ) }}
- name: Run unit tests
run: make unit
@@ -108,6 +121,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -159,6 +173,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -231,6 +246,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -338,6 +354,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
+19
View File
@@ -12,6 +12,7 @@ before:
builds:
- id: linux-build
dir: ./cmd/syft
binary: syft
goos:
- linux
@@ -32,6 +33,7 @@ builds:
-X github.com/anchore/syft/internal/version.gitDescription={{.Summary}}
- id: darwin-build
dir: ./cmd/syft
binary: syft
goos:
- darwin
@@ -49,6 +51,7 @@ builds:
- ./.github/scripts/apple-signing/sign.sh "{{ .Path }}" "{{ .IsSnapshot }}" "{{ .Target }}"
- id: windows-build
dir: ./cmd/syft
binary: syft
goos:
- windows
@@ -99,6 +102,11 @@ dockers:
- "anchore/syft:{{ .Tag }}-amd64"
- "anchore/syft:v{{ .Major }}-amd64"
- "anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64"
- "ghcr.io/anchore/syft:latest"
- "ghcr.io/anchore/syft:{{ .Tag }}-amd64"
- "ghcr.io/anchore/syft:v{{ .Major }}-amd64"
- "ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64"
goarch: amd64
dockerfile: Dockerfile
use: buildx
build_flag_templates:
@@ -112,6 +120,9 @@ dockers:
- "anchore/syft:{{ .Tag }}-arm64v8"
- "anchore/syft:v{{ .Major }}-arm64v8"
- "anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8"
- "ghcr.io/anchore/syft:{{ .Tag }}-arm64v8"
- "ghcr.io/anchore/syft:v{{ .Major }}-arm64v8"
- "ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8"
goarch: arm64
dockerfile: Dockerfile
use: buildx
@@ -131,3 +142,11 @@ docker_manifests:
image_templates:
- anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64
- anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8
- name_template: ghcr.io/anchore/syft:{{ .Tag }}
image_templates:
- ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64
- ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8
- name_template: ghcr.io/anchore/syft:latest
image_templates:
- ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64
- ghcr.io/anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8
-6
View File
@@ -82,12 +82,6 @@ Date: Mon Aug 1 11:27:13 2020 -0400
[//]: # (TODO: PR title and description)
## Sign your commits
To ensure the authenticity and integrity of code contributions, **we require that all commits are signed**. Signing commits proves that your commits were truly created by you, as the holder of a private key.
Configuring git to sign your commits is a straightforward process. To get set up, see [GitHub's documentation on signing commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits).
## Test your changes
This project has a `Makefile` which includes many helpers running both unit and integration tests. Although PRs will have automatic checks for these, it is useful to run them locally, ensuring they pass before submitting changes. Ensure you've bootstrapped once before running tests:
+10 -4
View File
@@ -106,7 +106,7 @@ $(TEMPDIR):
.PHONY: bootstrap-tools
bootstrap-tools: $(TEMPDIR)
GO111MODULE=off GOBIN=$(shell realpath $(TEMPDIR)) go get -u golang.org/x/perf/cmd/benchstat
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ v1.42.1
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ v1.45.0
curl -sSfL https://raw.githubusercontent.com/wagoodman/go-bouncer/master/bouncer.sh | sh -s -- -b $(TEMPDIR)/ v0.3.0
curl -sSfL https://raw.githubusercontent.com/anchore/chronicle/main/install.sh | sh -s -- -b $(TEMPDIR)/ v0.3.0
.github/scripts/goreleaser-install.sh -d -b $(TEMPDIR)/ v1.4.1
@@ -122,7 +122,7 @@ bootstrap: $(RESULTSDIR) bootstrap-go bootstrap-tools ## Download and install al
$(call title,Bootstrapping dependencies)
.PHONY: static-analysis
static-analysis: lint check-go-mod-tidy check-licenses
static-analysis: check-go-mod-tidy check-licenses lint
.PHONY: lint
lint: ## Run gofmt + golangci lint checks
@@ -147,7 +147,7 @@ lint-fix: ## Auto-format all source code + run golangci lint fixers
.PHONY: check-licenses
check-licenses: ## Ensure transitive dependencies are compliant with the current license policy
$(TEMPDIR)/bouncer check
$(TEMPDIR)/bouncer check ./cmd/syft
check-go-mod-tidy:
@ .github/scripts/go-mod-tidy-check.sh && echo "go.mod and go.sum are tidy!"
@@ -214,6 +214,12 @@ java-packages-fingerprint:
cd syft/pkg/cataloger/java/test-fixtures/java-builds && \
make packages.fingerprint
.PHONY: go-binaries-fingerprint
go-binaries-fingerprint:
$(call title,Go binaries test fixture fingerprint)
cd syft/pkg/cataloger/golang/test-fixtures/archs && \
make binaries.fingerprint
.PHONY: fixtures
fixtures:
$(call title,Generating test fixtures)
@@ -299,7 +305,7 @@ cli: $(SNAPSHOTDIR) ## Run CLI tests
chmod 755 "$(SNAPSHOT_BIN)"
$(SNAPSHOT_BIN) version
SYFT_BINARY_LOCATION='$(SNAPSHOT_BIN)' \
go test -count=1 -v ./test/cli
go test -count=1 -timeout=15m -v ./test/cli
.PHONY: changelog
changelog: clean-changelog CHANGELOG.md
+65 -63
View File
@@ -9,7 +9,7 @@
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/anchore/syft/blob/main/LICENSE)
[![Slack Invite](https://img.shields.io/badge/Slack-Join-blue?logo=slack)](https://anchore.com/slack)
A CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner tool like [Grype](https://github.com/anchore/grype).
A CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner like [Grype](https://github.com/anchore/grype).
### Join our community meetings!
@@ -20,15 +20,26 @@ A CLI tool and Go library for generating a Software Bill of Materials (SBOM) fro
![syft-demo](https://user-images.githubusercontent.com/590471/90277200-2a253000-de33-11ea-893f-32c219eea11a.gif)
## Features
- Catalog container images and filesystems to discover packages and libraries.
- Generate in-toto attestations where an SBOM is included as the payload.
- Supports packages and libraries from various ecosystems (APK, DEB, RPM, Ruby Bundles, Python Wheel/Egg/requirements.txt, JavaScript NPM/Yarn, Java JAR/EAR/WAR/PAR/SAR, Jenkins plugins JPI/HPI, Go modules, PHP Composer)
- Linux distribution identification (supports Alpine, BusyBox, CentOS/RedHat, Debian/Ubuntu flavored distributions)
- Supports Docker and OCI image formats
- Direct support for [Grype](https://github.com/anchore/grype), a fast and powerful vulnerability matcher.
- Generates SBOMs for container images, filesystems, archives, and more to discover packages and libraries
- Supports OCI and Docker image formats
- Linux distribution identification
- Works seamlessly with [Grype](https://github.com/anchore/grype) (a fast, modern vulnerability scanner)
- Able to create signed SBOM attestations using the [in-toto specification](https://github.com/in-toto/attestation/blob/main/spec/README.md)
### Supported Ecosystems
If you encounter an issue, please [let us know using the issue tracker](https://github.com/anchore/syft/issues).
- Alpine (apk)
- Dart (pubs)
- Debian (dpkg)
- Go (go.mod, Go binaries)
- Java (jar, ear, war, par, sar)
- JavaScript (npm, yarn)
- Jenkins Plugins (jpi, hpi)
- PHP (composer)
- Python (wheel, egg, poetry, requirements.txt)
- Red Hat (rpm)
- Ruby (gem)
- Rust (cargo.lock)
## Installation
@@ -53,15 +64,13 @@ brew install syft
### Nix
**Note**: nix packaging of Syft is [community maintained](https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/syft/default.nix)
Also syft is currently only in the [unstable channel](https://nixos.wiki/wiki/Nix_channels#The_official_channels) awaiting the `22.05` release
**Note**: Nix packaging of Syft is [community maintained](https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/syft/default.nix). Syft is currently only in the [unstable channel](https://nixos.wiki/wiki/Nix_channels#The_official_channels) awaiting the `22.05` release
```bash
nix-env -i syft
```
... or, just try it out in an ephemeral nix shell
... or, just try it out in an ephemeral nix shell:
```bash
nix-shell -p syft
@@ -70,33 +79,28 @@ nix-shell -p syft
## Getting started
#### SBOM
To generate an SBOM for an OCI image:
To generate an SBOM for a container image:
```
syft <image>
```
**Note**: This is equivalent to specifying the `packages` subcommand:
```
syft packages <image>
```
The above output includes only software that is visible in the container (i.e., the squashed representation of the image).
To include software from all image layers in the SBOM, regardless of its presence in the final image, provide `--scope all-layers`:
The above output includes only software that is visible in the container (i.e., the squashed representation of the image). To include software from all image layers in the SBOM, regardless of its presence in the final image, provide `--scope all-layers`:
```
syft packages <image> --scope all-layers
syft <image> --scope all-layers
```
#### SBOM Attestation
To generate an attested SBOM for an OCI image as the predicate of an in-toto attestation
#### SBOM attestation
To generate an SBOM attestation for a container image:
```
syft attest --output [FORMAT] --key [KEY] [SOURCE] [flags]
```
The above output is in the form of the [DSSE envelope](https://github.com/secure-systems-lab/dsse/blob/master/envelope.md#dsse-envelope).
The payload is a base64 encoded `in-toto` statement with the SBOM as the predicate, the payload type is `application/vnd.in-toto+json`, and the signatures array is populated
with the contents needed for public key verification. For details on workflows using this command see [here](#adding-an-sbom-to-an-image-as-an-attestation-using-syft).
The payload is a base64 encoded `in-toto` statement with the generated SBOM as the predicate. For details on workflows using this command see [here](#adding-an-sbom-to-an-image-as-an-attestation-using-syft).
### Supported sources
@@ -104,10 +108,10 @@ Syft can generate a SBOM from a variety of sources:
```
# catalog a container image archive (from the result of `docker image save ...`, `podman save ...`, or `skopeo copy` commands)
syft packages path/to/image.tar
syft path/to/image.tar
# catalog a directory
syft packages path/to/dir
syft path/to/dir
```
Sources can be explicitly provided with a scheme:
@@ -146,15 +150,15 @@ The output format for Syft is configurable as well using the
`-o` (or `--output`) option:
```
syft packages <image> -o <format>
syft <image> -o <format>
```
Where the `formats` available are:
- `json`: Use this to get as much information out of Syft as possible!
- `text`: A row-oriented, human-and-machine-friendly output.
- `cyclonedx`: A XML report conforming to the [CycloneDX 1.3 specification](https://cyclonedx.org/specification/overview/).
- `cyclonedx-json`: A JSON report conforming to the [CycloneDX 1.3 specification](https://cyclonedx.org/specification/overview/).
- `spdx`: A tag-value formatted report conforming to the [SPDX 2.2 specification](https://spdx.github.io/spdx-spec/).
- `cyclonedx-xml`: A XML report conforming to the [CycloneDX 1.4 specification](https://cyclonedx.org/specification/overview/).
- `cyclonedx-json`: A JSON report conforming to the [CycloneDX 1.4 specification](https://cyclonedx.org/specification/overview/).
- `spdx-tag-value`: A tag-value formatted report conforming to the [SPDX 2.2 specification](https://spdx.github.io/spdx-spec/).
- `spdx-json`: A JSON report conforming to the [SPDX 2.2 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.2/schemas/spdx-schema.json).
- `table`: A columnar summary (default).
@@ -164,20 +168,16 @@ Syft can also output _multiple_ files in differing formats by appending
`=<file>` to the option, for example to output Syft JSON and SPDX JSON:
```shell
syft packages <image> -o json=sbom.syft.json -o spdx-json=sbom.spdx.json
syft <image> -o json=sbom.syft.json -o spdx-json=sbom.spdx.json
```
## Private Registry Authentication
### Local Docker Credentials
When a container runtime is not present, Syft can still utilize credentials configured in common credential sources (such as `~/.docker/config.json`).
It will pull images from private registries using these credentials. The config file is where your credentials are stored when authenticating with private registries via some command like `docker login`.
For more information see the `go-containerregistry` [documentation](https://github.com/google/go-containerregistry/tree/main/pkg/authn).
When a container runtime is not present, Syft can still utilize credentials configured in common credential sources (such as `~/.docker/config.json`). It will pull images from private registries using these credentials. The config file is where your credentials are stored when authenticating with private registries via some command like `docker login`. For more information see the `go-containerregistry` [documentation](https://github.com/google/go-containerregistry/tree/main/pkg/authn).
An example `config.json` looks something like this:
```
// config.json
```json
{
"auths": {
"registry.example.com": {
@@ -190,15 +190,17 @@ An example `config.json` looks something like this:
You can run the following command as an example. It details the mount/environment configuration a container needs to access a private registry:
`docker run -v ./config.json:/config/config.json -e "DOCKER_CONFIG=/config" anchore/syft:latest <private_image>`
```
docker run -v ./config.json:/config/config.json -e "DOCKER_CONFIG=/config" anchore/syft:latest <private_image>
```
### Docker Credentials in Kubernetes
The below section shows a simple workflow on how to mount this config file as a secret into a container on kubernetes.
1. Create a secret. The value of `config.json` is important. It refers to the specification detailed [here](https://github.com/google/go-containerregistry/tree/main/pkg/authn#the-config-file).
Below this section is the `secret.yaml` file that the pod configuration will consume as a volume.
The key `config.json` is important. It will end up being the name of the file when mounted into the pod.
```
Here's a simple workflow to mount this config file as a secret into a container on Kubernetes.
1. Create a secret. The value of `config.json` is important. It refers to the specification detailed [here](https://github.com/google/go-containerregistry/tree/main/pkg/authn#the-config-file). Below this section is the `secret.yaml` file that the pod configuration will consume as a volume. The key `config.json` is important. It will end up being the name of the file when mounted into the pod.
```yaml
# secret.yaml
apiVersion: v1
@@ -213,11 +215,9 @@ The key `config.json` is important. It will end up being the name of the file wh
`kubectl apply -f secret.yaml`
2. Create your pod running syft. The env `DOCKER_CONFIG` is important because it advertises where to look for the credential file.
In the below example, setting `DOCKER_CONFIG=/config` informs syft that credentials can be found at `/config/config.json`.
This is why we used `config.json` as the key for our secret. When mounted into containers the secrets' key is used as the filename.
The `volumeMounts` section mounts our secret to `/config`. The `volumes` section names our volume and leverages the secret we created in step one.
```
2. Create your pod running syft. The env `DOCKER_CONFIG` is important because it advertises where to look for the credential file. In the below example, setting `DOCKER_CONFIG=/config` informs syft that credentials can be found at `/config/config.json`. This is why we used `config.json` as the key for our secret. When mounted into containers the secrets' key is used as the filename. The `volumeMounts` section mounts our secret to `/config`. The `volumes` section names our volume and leverages the secret we created in step one.
```yaml
# pod.yaml
apiVersion: v1
@@ -246,10 +246,9 @@ The `volumeMounts` section mounts our secret to `/config`. The `volumes` section
`kubectl apply -f pod.yaml`
3. The user can now run `kubectl logs syft-private-registry-demo`. The logs should show the syft analysis for the `<private_image>` provided in the pod configuration.
3. The user can now run `kubectl logs syft-private-registry-demo`. The logs should show the Syft analysis for the `<private_image>` provided in the pod configuration.
Using the above information, users should be able to configure private registry access without having to do so in the `grype` or `syft` configuration files.
They will also not be dependent on a docker daemon, (or some other runtime software) for registry configuration and access.
Using the above information, users should be able to configure private registry access without having to do so in the `grype` or `syft` configuration files. They will also not be dependent on a Docker daemon, (or some other runtime software) for registry configuration and access.
## Configuration
@@ -286,7 +285,11 @@ check-for-app-update: true
# exclude:
# - "/etc/**"
# - "./out/**/*.json"
exclude:
exclude: []
# os and/or architecture to use when referencing container images (e.g. "windows/armv6" or "arm64")
# same as --platform; SYFT_PLATFORM env var
platform: ""
# cataloging packages is exposed through the packages and power-user subcommands
package:
@@ -460,26 +463,25 @@ anchore:
`syft attest --output [FORMAT] --key [KEY] [SOURCE] [flags]`
SBOMs themselves can serve as input to different analysis tools. The Anchore organization offers the vulnerability scanner
[grype](https://github.com/anchore/grype) as one such tool.
One of the foundational approaches to "trust" between tools is for producers to use the artifacts generated by syft as attestations to their images.
The DSSE output of `syft attest` can be used with the [cosign](https://github.com/sigstore/cosign) tool to attach an attestation to an image.
SBOMs themselves can serve as input to different analysis tools. [Grype](https://github.com/anchore/grype), a vulnerability scanner CLI tool from Anchore, is one such tool. Publishers of container images can use attestations to enable their consumers to trust Syft-generated SBOM descriptions of those container images. To create and provide these attestations, image publishers can run `syft attest` in conjunction with the [cosign](https://github.com/sigstore/cosign) tool to attach SBOM attestations to their images.
#### Example attest
#### Example attestation
Note for the following example replace `docker.io/image:latest` with an image you own. You should also have push access to
its remote reference. Replace $MY_PRIVATE_KEY with a private key you own or have generated with cosign.
its remote reference. Replace `$MY_PRIVATE_KEY` with a private key you own or have generated with cosign.
```bash
syft attest --key $MY_PRIVATE_KEY docker.io/image:latest > image_latest_sbom_attestation.json
cosign attach attestation --attestation image_latest_sbom_attestation.json docker.io/image:latest
```
Verify the new attestation exists on your image
Verify the new attestation exists on your image.
```bash
cosign verify-attestation -key $MY_PUBLIC_KEY docker.io/image:latest | jq '.payload | @base64d | .payload | fromjson | .predicate'
```
You should see this output along with the attached SBOM.
You should see this output along with the attached SBOM:
```
Verification for docker.io/image:latest --
The following checks were performed on each of these signatures:
-322
View File
@@ -1,322 +0,0 @@
package cmd
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"os"
"strings"
"github.com/anchore/syft/internal/formats/cyclonedxjson"
"github.com/anchore/syft/internal/formats/spdx22json"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/stereoscope"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/in-toto/in-toto-golang/in_toto"
"github.com/pkg/errors"
"github.com/pkg/profile"
"github.com/sigstore/cosign/cmd/cosign/cli/sign"
"github.com/sigstore/cosign/pkg/cosign"
"github.com/sigstore/cosign/pkg/cosign/attestation"
"github.com/sigstore/sigstore/pkg/signature/dsse"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
"github.com/wagoodman/go-partybus"
signatureoptions "github.com/sigstore/sigstore/pkg/signature/options"
)
const (
attestExample = ` {{.appName}} {{.command}} --output [FORMAT] --key [KEY] alpine:latest
Supports the following image sources:
{{.appName}} {{.command}} --key [KEY] yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} --key [KEY] path/to/a/file/or/dir only for OCI tar or OCI directory
`
attestSchemeHelp = "\n" + indent + schemeHelpHeader + "\n" + imageSchemeHelp
attestHelp = attestExample + attestSchemeHelp
intotoJSONDsseType = `application/vnd.in-toto+json`
)
var attestFormats = []sbom.FormatID{
syftjson.ID,
spdx22json.ID,
cyclonedxjson.ID,
}
var (
attestCmd = &cobra.Command{
Use: "attest --output [FORMAT] --key [KEY] [SOURCE]",
Short: "Generate a package SBOM as an attestation for the given [SOURCE] container image",
Long: "Generate a packaged-based Software Bill Of Materials (SBOM) from a container image as the predicate of an in-toto attestation",
Example: internal.Tprintf(attestHelp, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "attest",
}),
Args: validateInputArgs,
SilenceUsage: true,
SilenceErrors: true,
PreRunE: func(cmd *cobra.Command, args []string) (err error) {
if appConfig.Dev.ProfileCPU && appConfig.Dev.ProfileMem {
return fmt.Errorf("cannot profile CPU and memory simultaneously")
}
return nil
},
RunE: func(cmd *cobra.Command, args []string) error {
if appConfig.Dev.ProfileCPU {
defer profile.Start(profile.CPUProfile).Stop()
} else if appConfig.Dev.ProfileMem {
defer profile.Start(profile.MemProfile).Stop()
}
return attestExec(cmd.Context(), cmd, args)
},
}
)
func fetchPassword(_ bool) (b []byte, err error) {
potentiallyPipedInput, err := internal.IsPipedInput()
if err != nil {
log.Warnf("unable to determine if there is piped input: %+v", err)
}
switch {
case appConfig.Attest.Password != "":
return []byte(appConfig.Attest.Password), nil
case potentiallyPipedInput:
// handle piped in passwords
pwBytes, err := io.ReadAll(os.Stdin)
if err != nil {
return nil, fmt.Errorf("unable to get password from stdin: %w", err)
}
// be resilient to input that may have newline characters (in case someone is using echo without -n)
cleanPw := strings.TrimRight(string(pwBytes), "\n")
return []byte(cleanPw), nil
case internal.IsTerminal():
return cosign.GetPassFromTerm(false)
}
return nil, errors.New("no method available to fetch password")
}
func selectPassFunc(keypath string) (cosign.PassFunc, error) {
keyContents, err := os.ReadFile(keypath)
if err != nil {
return nil, err
}
var fn cosign.PassFunc = func(bool) (b []byte, err error) { return nil, nil }
_, err = cosign.LoadPrivateKey(keyContents, nil)
if err != nil {
fn = fetchPassword
}
return fn, nil
}
func attestExec(ctx context.Context, _ *cobra.Command, args []string) error {
// can only be an image for attestation or OCI DIR
userInput := args[0]
si, err := source.ParseInput(userInput, appConfig.Platform, false)
if err != nil {
return fmt.Errorf("could not generate source input for attest command: %w", err)
}
switch si.Scheme {
case source.ImageScheme, source.UnknownScheme:
// at this point we know that it cannot be dir: or file: schemes, so we will assume that the unknown scheme could represent an image
si.Scheme = source.ImageScheme
default:
return fmt.Errorf("attest command can only be used with image sources but discovered %q when given %q", si.Scheme, userInput)
}
// if the original detection was from a local daemon we want to short circuit
// that and attempt to generate the image source from a registry source instead
switch si.ImageSource {
case image.UnknownSource, image.OciRegistrySource:
si.ImageSource = image.OciRegistrySource
default:
return fmt.Errorf("attest command can only be used with image sources fetch directly from the registry, but discovered an image source of %q when given %q", si.ImageSource, userInput)
}
if len(appConfig.Outputs) > 1 {
return fmt.Errorf("unable to generate attestation for more than one output")
}
format := syft.FormatByName(appConfig.Outputs[0])
predicateType := formatPredicateType(format)
if predicateType == "" {
return fmt.Errorf("could not produce attestation predicate for given format: %q. Available formats: %+v", formatAliases(format.ID()), formatAliases(attestFormats...))
}
passFunc, err := selectPassFunc(appConfig.Attest.Key)
if err != nil {
return err
}
ko := sign.KeyOpts{
KeyRef: appConfig.Attest.Key,
PassFunc: passFunc,
}
sv, err := sign.SignerFromKeyOpts(ctx, "", ko)
if err != nil {
return err
}
defer sv.Close()
return eventLoop(
attestationExecWorker(*si, format, predicateType, sv),
setupSignals(),
eventSubscription,
stereoscope.Cleanup,
ui.Select(isVerbose(), appConfig.Quiet)...,
)
}
func attestationExecWorker(sourceInput source.Input, format sbom.Format, predicateType string, sv *sign.SignerVerifier) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
src, cleanup, err := source.NewFromRegistry(sourceInput, appConfig.Registry.ToOptions(), appConfig.Exclusions)
if cleanup != nil {
defer cleanup()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", sourceInput.UserInput, err)
return
}
s, err := generateSBOM(src, errs)
if err != nil {
errs <- err
return
}
sbomBytes, err := syft.Encode(*s, format)
if err != nil {
errs <- err
return
}
err = generateAttestation(sbomBytes, src, sv, predicateType)
if err != nil {
errs <- err
return
}
}()
return errs
}
func formatPredicateType(format sbom.Format) string {
switch format.ID() {
case spdx22json.ID:
return in_toto.PredicateSPDX
case cyclonedxjson.ID:
// Tentative see https://github.com/in-toto/attestation/issues/82
return "https://cyclonedx.org/bom"
case syftjson.ID:
return "https://syft.dev/bom"
default:
return ""
}
}
func findValidDigest(digests []string) string {
// since we are only using the OCI repo provider for this source we are safe that this is only 1 value
// see https://github.com/anchore/stereoscope/blob/25ebd49a842b5ac0a20c2e2b4b81335b64ad248c/pkg/image/oci/registry_provider.go#L57-L63
split := strings.Split(digests[0], "sha256:")
return split[1]
}
func generateAttestation(predicate []byte, src *source.Source, sv *sign.SignerVerifier, predicateType string) error {
switch len(src.Image.Metadata.RepoDigests) {
case 0:
return fmt.Errorf("cannot generate attestation since no repo digests were found; make sure you're passing an OCI registry source for the attest command")
case 1:
default:
return fmt.Errorf("cannot generate attestation since multiple repo digests were found for the image: %+v", src.Image.Metadata.RepoDigests)
}
wrapped := dsse.WrapSigner(sv, intotoJSONDsseType)
sh, err := attestation.GenerateStatement(attestation.GenerateOpts{
Predicate: bytes.NewBuffer(predicate),
Type: predicateType,
Digest: findValidDigest(src.Image.Metadata.RepoDigests),
})
if err != nil {
return err
}
payload, err := json.Marshal(sh)
if err != nil {
return err
}
signedPayload, err := wrapped.SignMessage(bytes.NewReader(payload), signatureoptions.WithContext(context.Background()))
if err != nil {
return errors.Wrap(err, "unable to sign SBOM")
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error {
_, err := os.Stdout.Write(signedPayload)
return err
},
})
return nil
}
func init() {
setAttestFlags(attestCmd.Flags())
if err := bindAttestConfigOptions(attestCmd.Flags()); err != nil {
panic(err)
}
rootCmd.AddCommand(attestCmd)
}
func setAttestFlags(flags *pflag.FlagSet) {
// key options
flags.StringP("key", "", "cosign.key",
"path to the private key file to use for attestation",
)
// in-toto attestations only support JSON predicates, so not all SBOM formats that syft can output are supported
flags.StringP(
"output", "o", formatAliases(syftjson.ID)[0],
fmt.Sprintf("the SBOM format encapsulated within the attestation, available options=%v", formatAliases(attestFormats...)),
)
flags.StringP(
"platform", "", "",
"an optional platform specifier for container image sources (e.g. 'linux/arm64', 'linux/arm64/v8', 'arm64', 'linux')",
)
}
func bindAttestConfigOptions(flags *pflag.FlagSet) error {
// note: output is not included since this configuration option is shared between multiple subcommands
if err := viper.BindPFlag("attest.key", flags.Lookup("key")); err != nil {
return err
}
return nil
}
-31
View File
@@ -1,31 +0,0 @@
package cmd
import (
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/event"
"github.com/wagoodman/go-partybus"
)
func checkForApplicationUpdate() {
if appConfig.CheckForAppUpdate {
log.Debugf("checking if new vesion of %s is available", internal.ApplicationName)
isAvailable, newVersion, err := version.IsUpdateAvailable()
if err != nil {
// this should never stop the application
log.Errorf(err.Error())
}
if isAvailable {
log.Infof("new version of %s is available: %s (current version is %s)", internal.ApplicationName, newVersion, version.FromBuild().Version)
bus.Publish(partybus.Event{
Type: event.AppUpdateAvailable,
Value: newVersion,
})
} else {
log.Debugf("no new %s update available", internal.ApplicationName)
}
}
}
-171
View File
@@ -1,171 +0,0 @@
package cmd
import (
"encoding/json"
"fmt"
"os"
"sort"
"github.com/spf13/pflag"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/logger"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/gookit/color"
"github.com/spf13/cobra"
"github.com/spf13/viper"
"github.com/wagoodman/go-partybus"
)
var (
appConfig *config.Application
eventBus *partybus.Bus
eventSubscription *partybus.Subscription
)
func init() {
cobra.OnInitialize(
initCmdAliasBindings,
initAppConfig,
initLogging,
logAppConfig,
checkForApplicationUpdate,
logAppVersion,
initEventBus,
)
}
func Execute() {
if err := rootCmd.Execute(); err != nil {
fmt.Fprintln(os.Stderr, color.Red.Sprint(err.Error()))
os.Exit(1)
}
}
// we must setup the config-cli bindings first before the application configuration is parsed. However, this cannot
// be done without determining what the primary command that the config options should be bound to since there are
// shared concerns (the root-packages alias).
func initCmdAliasBindings() {
activeCmd, _, err := rootCmd.Find(os.Args[1:])
if err != nil {
panic(err)
}
// enable all cataloger by default if power-user command is run
if activeCmd == powerUserCmd {
config.PowerUserCatalogerEnabledDefault()
}
// set bindings based on the packages alias
switch activeCmd {
case packagesCmd, rootCmd:
// note: we need to lazily bind config options since they are shared between both the root command
// and the packages command. Otherwise there will be global viper state that is in contention.
// See for more details: https://github.com/spf13/viper/issues/233 . Additionally, the bindings must occur BEFORE
// reading the application configuration, which implies that it must be an initializer (or rewrite the command
// initialization structure against typical patterns used with cobra, which is somewhat extreme for a
// temporary alias)
if err = bindPackagesConfigOptions(activeCmd.Flags()); err != nil {
panic(err)
}
case attestCmd:
// the --output and --platform options are independently defined flags, but a shared config option
if err = bindSharedConfigOption(attestCmd.Flags()); err != nil {
panic(err)
}
// even though the root command or packages command is NOT being run, we still need default bindings
// such that application config parsing passes.
if err = bindExclusivePackagesConfigOptions(packagesCmd.Flags()); err != nil {
panic(err)
}
default:
// even though the root command or packages command is NOT being run, we still need default bindings
// such that application config parsing passes.
if err = bindPackagesConfigOptions(packagesCmd.Flags()); err != nil {
panic(err)
}
}
}
func bindSharedConfigOption(flags *pflag.FlagSet) error {
if err := viper.BindPFlag("output", flags.Lookup("output")); err != nil {
return err
}
if err := viper.BindPFlag("platform", flags.Lookup("platform")); err != nil {
return err
}
return nil
}
func initAppConfig() {
cfg, err := config.LoadApplicationConfig(viper.GetViper(), persistentOpts)
if err != nil {
fmt.Printf("failed to load application config: \n\t%+v\n", err)
os.Exit(1)
}
appConfig = cfg
}
func initLogging() {
cfg := logger.LogrusConfig{
EnableConsole: (appConfig.Log.FileLocation == "" || appConfig.CliOptions.Verbosity > 0) && !appConfig.Quiet,
EnableFile: appConfig.Log.FileLocation != "",
Level: appConfig.Log.LevelOpt,
Structured: appConfig.Log.Structured,
FileLocation: appConfig.Log.FileLocation,
}
logWrapper := logger.NewLogrusLogger(cfg)
syft.SetLogger(logWrapper)
stereoscope.SetLogger(&logger.LogrusNestedLogger{
Logger: logWrapper.Logger.WithField("from-lib", "stereoscope"),
})
}
func logAppConfig() {
log.Debugf("application config:\n%+v", color.Magenta.Sprint(appConfig.String()))
}
func initEventBus() {
eventBus = partybus.NewBus()
eventSubscription = eventBus.Subscribe()
stereoscope.SetBus(eventBus)
syft.SetBus(eventBus)
}
func logAppVersion() {
versionInfo := version.FromBuild()
log.Infof("syft version: %s", versionInfo.Version)
var fields map[string]interface{}
bytes, err := json.Marshal(versionInfo)
if err != nil {
return
}
err = json.Unmarshal(bytes, &fields)
if err != nil {
return
}
keys := make([]string, 0, len(fields))
for k := range fields {
keys = append(keys, k)
}
sort.Strings(keys)
for idx, field := range keys {
value := fields[field]
branch := "├──"
if idx == len(fields)-1 {
branch = "└──"
}
log.Debugf(" %s %s: %s", branch, field, value)
}
}
-113
View File
@@ -1,113 +0,0 @@
package cmd
import (
"context"
"os"
"strings"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/client"
"github.com/spf13/cobra"
)
// completionCmd represents the completion command
var completionCmd = &cobra.Command{
Use: "completion [bash|zsh|fish]",
Short: "Generate a shell completion for Syft (listing local docker images)",
Long: `To load completions (docker image list):
Bash:
$ source <(syft completion bash)
# To load completions for each session, execute once:
Linux:
$ syft completion bash > /etc/bash_completion.d/syft
MacOS:
$ syft completion bash > /usr/local/etc/bash_completion.d/syft
Zsh:
# If shell completion is not already enabled in your environment you will need
# to enable it. You can execute the following once:
$ echo "autoload -U compinit; compinit" >> ~/.zshrc
# To load completions for each session, execute once:
$ syft completion zsh > "${fpath[1]}/_syft"
# You will need to start a new shell for this setup to take effect.
Fish:
$ syft completion fish | source
# To load completions for each session, execute once:
$ syft completion fish > ~/.config/fish/completions/syft.fish
`,
DisableFlagsInUseLine: true,
ValidArgs: []string{"bash", "zsh", "fish"},
Args: cobra.ExactValidArgs(1),
Run: func(cmd *cobra.Command, args []string) {
var err error
switch args[0] {
case "bash":
err = cmd.Root().GenBashCompletion(os.Stdout)
case "zsh":
err = cmd.Root().GenZshCompletion(os.Stdout)
case "fish":
err = cmd.Root().GenFishCompletion(os.Stdout, true)
}
if err != nil {
panic(err)
}
},
}
func init() {
rootCmd.AddCommand(completionCmd)
}
func dockerImageValidArgsFunction(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
// Since we use ValidArgsFunction, Cobra will call this AFTER having parsed all flags and arguments provided
dockerImageRepoTags, err := listLocalDockerImages(toComplete)
if err != nil {
// Indicates that an error occurred and completions should be ignored
return []string{"completion failed"}, cobra.ShellCompDirectiveError
}
if len(dockerImageRepoTags) == 0 {
return []string{"no docker images found"}, cobra.ShellCompDirectiveError
}
// ShellCompDirectiveDefault indicates that the shell will perform its default behavior after completions have
// been provided (without implying other possible directives)
return dockerImageRepoTags, cobra.ShellCompDirectiveDefault
}
func listLocalDockerImages(prefix string) ([]string, error) {
var repoTags = make([]string, 0)
ctx := context.Background()
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
return repoTags, err
}
// Only want to return tagged images
imageListArgs := filters.NewArgs()
imageListArgs.Add("dangling", "false")
images, err := cli.ImageList(ctx, types.ImageListOptions{All: false, Filters: imageListArgs})
if err != nil {
return repoTags, err
}
for _, image := range images {
// image may have multiple tags
for _, tag := range image.RepoTags {
if strings.HasPrefix(tag, prefix) {
repoTags = append(repoTags, tag)
}
}
}
return repoTags, nil
}
-64
View File
@@ -1,64 +0,0 @@
package cmd
import (
"fmt"
"strings"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/sbom"
"github.com/hashicorp/go-multierror"
)
// makeWriter creates a sbom.Writer for output or returns an error. this will either return a valid writer
// or an error but neither both and if there is no error, sbom.Writer.Close() should be called
func makeWriter(outputs []string, defaultFile string) (sbom.Writer, error) {
outputOptions, err := parseOptions(outputs, defaultFile)
if err != nil {
return nil, err
}
writer, err := sbom.NewWriter(outputOptions...)
if err != nil {
return nil, err
}
return writer, nil
}
// parseOptions utility to parse command-line option strings and retain the existing behavior of default format and file
func parseOptions(outputs []string, defaultFile string) (out []sbom.WriterOption, errs error) {
// always should have one option -- we generally get the default of "table", but just make sure
if len(outputs) == 0 {
outputs = append(outputs, string(table.ID))
}
for _, name := range outputs {
name = strings.TrimSpace(name)
// split to at most two parts for <format>=<file>
parts := strings.SplitN(name, "=", 2)
// the format name is the first part
name = parts[0]
// default to the --file or empty string if not specified
file := defaultFile
// If a file is specified as part of the output formatName, use that
if len(parts) > 1 {
file = parts[1]
}
format := syft.FormatByName(name)
if format == nil {
errs = multierror.Append(errs, fmt.Errorf("bad output format: '%s'", name))
continue
}
out = append(out, sbom.NewWriterOption(format, file))
}
return out, errs
}
-78
View File
@@ -1,78 +0,0 @@
package cmd
import (
"fmt"
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
func TestOutputWriterConfig(t *testing.T) {
tmp := t.TempDir() + "/"
tests := []struct {
outputs []string
file string
err bool
expected []string
}{
{
outputs: []string{},
expected: []string{""},
},
{
outputs: []string{"json"},
expected: []string{""},
},
{
file: "test-1.json",
expected: []string{"test-1.json"},
},
{
outputs: []string{"json=test-2.json"},
expected: []string{"test-2.json"},
},
{
outputs: []string{"json=test-3-1.json", "spdx-json=test-3-2.json"},
expected: []string{"test-3-1.json", "test-3-2.json"},
},
{
outputs: []string{"text", "json=test-4.json"},
expected: []string{"", "test-4.json"},
},
}
for _, test := range tests {
t.Run(fmt.Sprintf("%s/%s", test.outputs, test.file), func(t *testing.T) {
outputs := test.outputs
for i, val := range outputs {
outputs[i] = strings.Replace(val, "=", "="+tmp, 1)
}
file := test.file
if file != "" {
file = tmp + file
}
_, err := makeWriter(test.outputs, file)
if test.err {
assert.Error(t, err)
return
} else {
assert.NoError(t, err)
}
for _, expected := range test.expected {
if expected != "" {
assert.FileExists(t, tmp+expected)
} else if file != "" {
assert.FileExists(t, file)
} else {
assert.NoFileExists(t, expected)
}
}
})
}
}
-389
View File
@@ -1,389 +0,0 @@
package cmd
import (
"context"
"fmt"
"io/ioutil"
"os"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/anchore"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/pkg/cataloger"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/pkg/profile"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
"github.com/wagoodman/go-partybus"
)
const (
packagesExample = ` {{.appName}} {{.command}} alpine:latest a summary of discovered packages
{{.appName}} {{.command}} alpine:latest -o json show all possible cataloging details
{{.appName}} {{.command}} alpine:latest -o cyclonedx show a CycloneDX formatted SBOM
{{.appName}} {{.command}} alpine:latest -o cyclonedx-json show a CycloneDX JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx show a SPDX 2.2 Tag-Value formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx-json show a SPDX 2.2 JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -vv show verbose debug information
Supports the following image sources:
{{.appName}} {{.command}} yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} path/to/a/file/or/dir a Docker tar, OCI tar, OCI directory, or generic filesystem directory
`
schemeHelpHeader = "You can also explicitly specify the scheme to use:"
imageSchemeHelp = ` {{.appName}} {{.command}} docker:yourrepo/yourimage:tag explicitly use the Docker daemon
{{.appName}} {{.command}} podman:yourrepo/yourimage:tag explicitly use the Podman daemon
{{.appName}} {{.command}} registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
{{.appName}} {{.command}} docker-archive:path/to/yourimage.tar use a tarball from disk for archives created from "docker save"
{{.appName}} {{.command}} oci-archive:path/to/yourimage.tar use a tarball from disk for OCI archives (from Skopeo or otherwise)
{{.appName}} {{.command}} oci-dir:path/to/yourimage read directly from a path on disk for OCI layout directories (from Skopeo or otherwise)
`
nonImageSchemeHelp = ` {{.appName}} {{.command}} dir:path/to/yourproject read directly from a path on disk (any directory)
{{.appName}} {{.command}} file:path/to/yourproject/file read directly from a path on disk (any single file)
`
packagesSchemeHelp = "\n" + indent + schemeHelpHeader + "\n" + imageSchemeHelp + nonImageSchemeHelp
packagesHelp = packagesExample + packagesSchemeHelp
)
var (
packagesCmd = &cobra.Command{
Use: "packages [SOURCE]",
Short: "Generate a package SBOM",
Long: "Generate a packaged-based Software Bill Of Materials (SBOM) from container images and filesystems",
Example: internal.Tprintf(packagesHelp, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "packages",
}),
Args: validateInputArgs,
SilenceUsage: true,
SilenceErrors: true,
PreRunE: func(cmd *cobra.Command, args []string) (err error) {
if appConfig.Dev.ProfileCPU && appConfig.Dev.ProfileMem {
return fmt.Errorf("cannot profile CPU and memory simultaneously")
}
return nil
},
RunE: func(cmd *cobra.Command, args []string) error {
if appConfig.Dev.ProfileCPU {
defer profile.Start(profile.CPUProfile).Stop()
} else if appConfig.Dev.ProfileMem {
defer profile.Start(profile.MemProfile).Stop()
}
return packagesExec(cmd, args)
},
ValidArgsFunction: dockerImageValidArgsFunction,
}
)
func init() {
setPackageFlags(packagesCmd.Flags())
rootCmd.AddCommand(packagesCmd)
}
func setPackageFlags(flags *pflag.FlagSet) {
// Formatting & Input options //////////////////////////////////////////////
flags.StringP(
"scope", "s", cataloger.DefaultSearchConfig().Scope.String(),
fmt.Sprintf("selection of layers to catalog, options=%v", source.AllScopes))
flags.StringArrayP(
"output", "o", formatAliases(table.ID),
fmt.Sprintf("report output format, options=%v", formatAliases(syft.FormatIDs()...)),
)
flags.StringP(
"file", "", "",
"file to write the default report output to (default is STDOUT)",
)
flags.StringP(
"platform", "", "",
"an optional platform specifier for container image sources (e.g. 'linux/arm64', 'linux/arm64/v8', 'arm64', 'linux')",
)
// Upload options //////////////////////////////////////////////////////////
flags.StringP(
"host", "H", "",
"the hostname or URL of the Anchore Enterprise instance to upload to",
)
flags.StringP(
"username", "u", "",
"the username to authenticate against Anchore Enterprise",
)
flags.StringP(
"password", "p", "",
"the password to authenticate against Anchore Enterprise",
)
flags.StringP(
"dockerfile", "d", "",
"include dockerfile for upload to Anchore Enterprise",
)
flags.StringArrayP(
"exclude", "", nil,
"exclude paths from being scanned using a glob expression",
)
flags.Bool(
"overwrite-existing-image", false,
"overwrite an existing image during the upload to Anchore Enterprise",
)
flags.Uint(
"import-timeout", 30,
"set a timeout duration (in seconds) for the upload to Anchore Enterprise",
)
}
func bindPackagesConfigOptions(flags *pflag.FlagSet) error {
if err := bindExclusivePackagesConfigOptions(flags); err != nil {
return err
}
if err := bindSharedConfigOption(flags); err != nil {
return err
}
return nil
}
// NOTE(alex): Write a helper for the binding operation, which can be used to perform the binding but also double check that the intended effect was had or else return an error. Another thought is to somehow provide zero-valued defaults for all values in our config struct (maybe with reflection?). There may be a mechanism that already exists in viper that protects against this that I'm not aware of. ref: https://github.com/anchore/syft/pull/805#discussion_r801931192
func bindExclusivePackagesConfigOptions(flags *pflag.FlagSet) error {
// Formatting & Input options //////////////////////////////////////////////
// note: output is not included since this configuration option is shared between multiple subcommands
if err := viper.BindPFlag("package.cataloger.scope", flags.Lookup("scope")); err != nil {
return err
}
if err := viper.BindPFlag("file", flags.Lookup("file")); err != nil {
return err
}
if err := viper.BindPFlag("exclude", flags.Lookup("exclude")); err != nil {
return err
}
// Upload options //////////////////////////////////////////////////////////
if err := viper.BindPFlag("anchore.host", flags.Lookup("host")); err != nil {
return err
}
if err := viper.BindPFlag("anchore.username", flags.Lookup("username")); err != nil {
return err
}
if err := viper.BindPFlag("anchore.password", flags.Lookup("password")); err != nil {
return err
}
if err := viper.BindPFlag("anchore.dockerfile", flags.Lookup("dockerfile")); err != nil {
return err
}
if err := viper.BindPFlag("anchore.overwrite-existing-image", flags.Lookup("overwrite-existing-image")); err != nil {
return err
}
if err := viper.BindPFlag("anchore.import-timeout", flags.Lookup("import-timeout")); err != nil {
return err
}
return nil
}
func validateInputArgs(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
// in the case that no arguments are given we want to show the help text and return with a non-0 return code.
if err := cmd.Help(); err != nil {
return fmt.Errorf("unable to display help: %w", err)
}
return fmt.Errorf("an image/directory argument is required")
}
return cobra.MaximumNArgs(1)(cmd, args)
}
func packagesExec(_ *cobra.Command, args []string) error {
writer, err := makeWriter(appConfig.Outputs, appConfig.File)
if err != nil {
return err
}
defer func() {
if err := writer.Close(); err != nil {
log.Warnf("unable to write to report destination: %w", err)
}
}()
// could be an image or a directory, with or without a scheme
userInput := args[0]
si, err := source.ParseInput(userInput, appConfig.Platform, true)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
return eventLoop(
packagesExecWorker(*si, writer),
setupSignals(),
eventSubscription,
stereoscope.Cleanup,
ui.Select(isVerbose(), appConfig.Quiet)...,
)
}
func isVerbose() (result bool) {
isPipedInput, err := internal.IsPipedInput()
if err != nil {
// since we can't tell if there was piped input we assume that there could be to disable the ETUI
log.Warnf("unable to determine if there is piped input: %+v", err)
return true
}
// verbosity should consider if there is piped input (in which case we should not show the ETUI)
return appConfig.CliOptions.Verbosity > 0 || isPipedInput
}
func generateSBOM(src *source.Source, errs chan error) (*sbom.SBOM, error) {
tasks, err := tasks()
if err != nil {
return nil, err
}
s := sbom.SBOM{
Source: src.Metadata,
Descriptor: sbom.Descriptor{
Name: internal.ApplicationName,
Version: version.FromBuild().Version,
Configuration: appConfig,
},
}
buildRelationships(&s, src, tasks, errs)
return &s, nil
}
func buildRelationships(s *sbom.SBOM, src *source.Source, tasks []task, errs chan error) {
var relationships []<-chan artifact.Relationship
for _, task := range tasks {
c := make(chan artifact.Relationship)
relationships = append(relationships, c)
go runTask(task, &s.Artifacts, src, c, errs)
}
s.Relationships = append(s.Relationships, mergeRelationships(relationships...)...)
}
func packagesExecWorker(si source.Input, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
src, cleanup, err := source.New(si, appConfig.Registry.ToOptions(), appConfig.Exclusions)
if cleanup != nil {
defer cleanup()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", si.UserInput, err)
return
}
s, err := generateSBOM(src, errs)
if err != nil {
errs <- err
return
}
if s == nil {
errs <- fmt.Errorf("no SBOM produced for %q", si.UserInput)
}
if appConfig.Anchore.Host != "" {
if err := runPackageSbomUpload(src, *s); err != nil {
errs <- err
return
}
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error { return writer.Write(*s) },
})
}()
return errs
}
func mergeRelationships(cs ...<-chan artifact.Relationship) (relationships []artifact.Relationship) {
for _, c := range cs {
for n := range c {
relationships = append(relationships, n)
}
}
return relationships
}
func runPackageSbomUpload(src *source.Source, s sbom.SBOM) error {
log.Infof("uploading results to %s", appConfig.Anchore.Host)
if src.Metadata.Scheme != source.ImageScheme {
return fmt.Errorf("unable to upload results: only images are supported")
}
var dockerfileContents []byte
if appConfig.Anchore.Dockerfile != "" {
if _, err := os.Stat(appConfig.Anchore.Dockerfile); os.IsNotExist(err) {
return fmt.Errorf("unable dockerfile=%q does not exist: %w", appConfig.Anchore.Dockerfile, err)
}
fh, err := os.Open(appConfig.Anchore.Dockerfile)
if err != nil {
return fmt.Errorf("unable to open dockerfile=%q: %w", appConfig.Anchore.Dockerfile, err)
}
dockerfileContents, err = ioutil.ReadAll(fh)
if err != nil {
return fmt.Errorf("unable to read dockerfile=%q: %w", appConfig.Anchore.Dockerfile, err)
}
}
c, err := anchore.NewClient(anchore.Configuration{
BaseURL: appConfig.Anchore.Host,
Username: appConfig.Anchore.Username,
Password: appConfig.Anchore.Password,
})
if err != nil {
return fmt.Errorf("failed to create anchore client: %w", err)
}
importCfg := anchore.ImportConfig{
ImageMetadata: src.Image.Metadata,
SBOM: s,
Dockerfile: dockerfileContents,
OverwriteExistingUpload: appConfig.Anchore.OverwriteExistingImage,
Timeout: appConfig.Anchore.ImportTimeout,
}
if err := c.Import(context.Background(), importCfg); err != nil {
return fmt.Errorf("failed to upload results to host=%s: %+v", appConfig.Anchore.Host, err)
}
return nil
}
-160
View File
@@ -1,160 +0,0 @@
package cmd
import (
"fmt"
"os"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/gookit/color"
"github.com/pkg/profile"
"github.com/spf13/cobra"
"github.com/wagoodman/go-partybus"
)
const powerUserExample = ` {{.appName}} {{.command}} <image>
DEPRECATED - THIS COMMAND WILL BE REMOVED in v1.0.0
Only image sources are supported (e.g. docker: , podman: , docker-archive: , oci: , etc.), the directory source (dir:) is not supported.
All behavior is controlled via application configuration and environment variables (see https://github.com/anchore/syft#configuration)
`
var powerUserOpts = struct {
configPath string
}{}
var powerUserCmd = &cobra.Command{
Use: "power-user [IMAGE]",
Short: "Run bulk operations on container images",
Example: internal.Tprintf(powerUserExample, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "power-user",
}),
Args: validateInputArgs,
Hidden: true,
SilenceUsage: true,
SilenceErrors: true,
PreRunE: func(cmd *cobra.Command, args []string) error {
if appConfig.Dev.ProfileCPU && appConfig.Dev.ProfileMem {
return fmt.Errorf("cannot profile CPU and memory simultaneously")
}
return nil
},
RunE: func(cmd *cobra.Command, args []string) error {
if appConfig.Dev.ProfileCPU {
defer profile.Start(profile.CPUProfile).Stop()
} else if appConfig.Dev.ProfileMem {
defer profile.Start(profile.MemProfile).Stop()
}
return powerUserExec(cmd, args)
},
ValidArgsFunction: dockerImageValidArgsFunction,
}
func init() {
powerUserCmd.Flags().StringVarP(&powerUserOpts.configPath, "config", "c", "", "config file path with all power-user options")
rootCmd.AddCommand(powerUserCmd)
}
func powerUserExec(_ *cobra.Command, args []string) error {
// could be an image or a directory, with or without a scheme
userInput := args[0]
writer, err := sbom.NewWriter(
sbom.NewWriterOption(
syftjson.Format(),
appConfig.File,
),
)
if err != nil {
return err
}
defer func() {
if err := writer.Close(); err != nil {
log.Warnf("unable to write to report destination: %+v", err)
}
// inform user at end of run that command will be removed
deprecated := color.Style{color.Red, color.OpBold}.Sprint("DEPRECATED: This command will be removed in v1.0.0")
fmt.Fprintln(os.Stderr, deprecated)
}()
return eventLoop(
powerUserExecWorker(userInput, writer),
setupSignals(),
eventSubscription,
stereoscope.Cleanup,
ui.Select(isVerbose(), appConfig.Quiet)...,
)
}
func powerUserExecWorker(userInput string, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
appConfig.Secrets.Cataloger.Enabled = true
appConfig.FileMetadata.Cataloger.Enabled = true
appConfig.FileContents.Cataloger.Enabled = true
appConfig.FileClassification.Cataloger.Enabled = true
tasks, err := tasks()
if err != nil {
errs <- err
return
}
si, err := source.ParseInput(userInput, appConfig.Platform, true)
if err != nil {
errs <- err
return
}
src, cleanup, err := source.New(*si, appConfig.Registry.ToOptions(), appConfig.Exclusions)
if err != nil {
errs <- err
return
}
if cleanup != nil {
defer cleanup()
}
s := sbom.SBOM{
Source: src.Metadata,
Descriptor: sbom.Descriptor{
Name: internal.ApplicationName,
Version: version.FromBuild().Version,
Configuration: appConfig,
},
}
var relationships []<-chan artifact.Relationship
for _, task := range tasks {
c := make(chan artifact.Relationship)
relationships = append(relationships, c)
go runTask(task, &s.Artifacts, src, c, errs)
}
s.Relationships = append(s.Relationships, mergeRelationships(relationships...)...)
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error { return writer.Write(s) },
})
}()
return errs
}
-52
View File
@@ -1,52 +0,0 @@
package cmd
import (
"fmt"
"os"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/version"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
var persistentOpts = config.CliOnlyOptions{}
// rootCmd is currently an alias for the packages command
var rootCmd = &cobra.Command{
Short: packagesCmd.Short,
Long: packagesCmd.Long,
Args: packagesCmd.Args,
Example: packagesCmd.Example,
SilenceUsage: true,
SilenceErrors: true,
PreRunE: packagesCmd.PreRunE,
RunE: packagesCmd.RunE,
ValidArgsFunction: packagesCmd.ValidArgsFunction,
Version: version.FromBuild().Version,
}
const indent = " "
func init() {
// set universal flags
rootCmd.PersistentFlags().StringVarP(&persistentOpts.ConfigPath, "config", "c", "", "application config file")
// setting the version template to just print out the string since we already have a templatized version string
rootCmd.SetVersionTemplate(fmt.Sprintf("%s {{.Version}}\n", internal.ApplicationName))
flag := "quiet"
rootCmd.PersistentFlags().BoolP(
flag, "q", false,
"suppress all logging output",
)
if err := viper.BindPFlag(flag, rootCmd.PersistentFlags().Lookup(flag)); err != nil {
fmt.Printf("unable to bind flag '%s': %+v", flag, err)
os.Exit(1)
}
rootCmd.PersistentFlags().CountVarP(&persistentOpts.Verbosity, "verbose", "v", "increase verbosity (-v = info, -vv = debug)")
// set common options that are not universal (package subcommand-alias specific)
setPackageFlags(rootCmd.Flags())
}
+69
View File
@@ -0,0 +1,69 @@
package cli
import (
"fmt"
"log"
"github.com/anchore/syft/cmd/syft/cli/attest"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
const (
attestExample = ` {{.appName}} {{.command}} --output [FORMAT] --key [KEY] alpine:latest
Supports the following image sources:
{{.appName}} {{.command}} --key [KEY] yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} --key [KEY] path/to/a/file/or/dir only for OCI tar or OCI directory
`
attestSchemeHelp = "\n" + indent + schemeHelpHeader + "\n" + imageSchemeHelp
attestHelp = attestExample + attestSchemeHelp
)
func Attest(v *viper.Viper, app *config.Application, ro *options.RootOptions) *cobra.Command {
ao := options.AttestOptions{}
cmd := &cobra.Command{
Use: "attest --output [FORMAT] --key [KEY] [SOURCE]",
Short: "Generate a package SBOM as an attestation for the given [SOURCE] container image",
Long: "Generate a packaged-based Software Bill Of Materials (SBOM) from a container image as the predicate of an in-toto attestation",
Example: internal.Tprintf(attestHelp, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "attest",
}),
Args: func(cmd *cobra.Command, args []string) error {
// run to unmarshal viper object onto app config
// the viper object correctly
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
}
// configure logging for command
newLogWrapper(app)
logApplicationConfig(app)
return validateArgs(cmd, args)
},
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
// this MUST be called first to make sure app config decodes
// the viper object correctly
newLogWrapper(app)
logApplicationConfig(app)
if app.CheckForAppUpdate {
checkForApplicationUpdate()
}
return attest.Run(cmd.Context(), app, args)
},
}
err := ao.AddFlags(cmd, v)
if err != nil {
log.Fatal(err)
}
return cmd
}
+216
View File
@@ -0,0 +1,216 @@
package attest
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os"
"strings"
"github.com/anchore/stereoscope"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/cmd/syft/cli/eventloop"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/cmd/syft/cli/packages"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/formats/cyclonedxjson"
"github.com/anchore/syft/internal/formats/spdx22json"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/in-toto/in-toto-golang/in_toto"
"github.com/pkg/errors"
"github.com/sigstore/cosign/cmd/cosign/cli/sign"
"github.com/sigstore/cosign/pkg/cosign/attestation"
"github.com/sigstore/sigstore/pkg/signature/dsse"
"github.com/wagoodman/go-partybus"
signatureoptions "github.com/sigstore/sigstore/pkg/signature/options"
)
var (
allowedAttestFormats = []sbom.FormatID{
syftjson.ID,
spdx22json.ID,
cyclonedxjson.ID,
}
intotoJSONDsseType = `application/vnd.in-toto+json`
)
func Run(ctx context.Context, app *config.Application, args []string) error {
// We cannot generate an attestation for more than one output
if len(app.Outputs) > 1 {
return fmt.Errorf("unable to generate attestation for more than one output")
}
// can only be an image for attestation or OCI DIR
userInput := args[0]
si, err := parseImageSource(userInput, app)
if err != nil {
return err
}
format := syft.FormatByName(app.Outputs[0])
predicateType := formatPredicateType(format)
if predicateType == "" {
return fmt.Errorf("could not produce attestation predicate for given format: %q. Available formats: %+v", options.FormatAliases(format.ID()), options.FormatAliases(allowedAttestFormats...))
}
passFunc, err := selectPassFunc(app.Attest.Key, app.Attest.Password)
if err != nil {
return err
}
ko := sign.KeyOpts{
KeyRef: app.Attest.Key,
PassFunc: passFunc,
}
sv, err := sign.SignerFromKeyOpts(ctx, "", "", ko)
if err != nil {
return err
}
defer sv.Close()
eventBus := partybus.NewBus()
stereoscope.SetBus(eventBus)
syft.SetBus(eventBus)
return eventloop.EventLoop(
execWorker(app, *si, format, predicateType, sv),
eventloop.SetupSignals(),
eventBus.Subscribe(),
stereoscope.Cleanup,
ui.Select(options.IsVerbose(app), app.Quiet)...,
)
}
func parseImageSource(userInput string, app *config.Application) (s *source.Input, err error) {
si, err := source.ParseInput(userInput, app.Platform, false)
if err != nil {
return nil, fmt.Errorf("could not generate source input for attest command: %w", err)
}
switch si.Scheme {
case source.ImageScheme, source.UnknownScheme:
// at this point we know that it cannot be dir: or file: schemes;
// we will assume that the unknown scheme could represent an image;
si.Scheme = source.ImageScheme
default:
return nil, fmt.Errorf("attest command can only be used with image sources but discovered %q when given %q", si.Scheme, userInput)
}
// if the original detection was from the local daemon we want to short circuit
// that and attempt to generate the image source from its current registry source instead
switch si.ImageSource {
case image.UnknownSource, image.OciRegistrySource:
si.ImageSource = image.OciRegistrySource
default:
return nil, fmt.Errorf("attest command can only be used with image sources fetch directly from the registry, but discovered an image source of %q when given %q", si.ImageSource, userInput)
}
return si, nil
}
func execWorker(app *config.Application, sourceInput source.Input, format sbom.Format, predicateType string, sv *sign.SignerVerifier) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
src, cleanup, err := source.NewFromRegistry(sourceInput, app.Registry.ToOptions(), app.Exclusions)
if cleanup != nil {
defer cleanup()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", sourceInput.UserInput, err)
return
}
s, err := packages.GenerateSBOM(src, errs, app)
if err != nil {
errs <- err
return
}
sbomBytes, err := syft.Encode(*s, format)
if err != nil {
errs <- err
return
}
err = generateAttestation(sbomBytes, src, sv, predicateType)
if err != nil {
errs <- err
return
}
}()
return errs
}
func generateAttestation(predicate []byte, src *source.Source, sv *sign.SignerVerifier, predicateType string) error {
switch len(src.Image.Metadata.RepoDigests) {
case 0:
return fmt.Errorf("cannot generate attestation since no repo digests were found; make sure you're passing an OCI registry source for the attest command")
case 1:
default:
return fmt.Errorf("cannot generate attestation since multiple repo digests were found for the image: %+v", src.Image.Metadata.RepoDigests)
}
wrapped := dsse.WrapSigner(sv, intotoJSONDsseType)
sh, err := attestation.GenerateStatement(attestation.GenerateOpts{
Predicate: bytes.NewBuffer(predicate),
Type: predicateType,
Digest: findValidDigest(src.Image.Metadata.RepoDigests),
})
if err != nil {
return err
}
payload, err := json.Marshal(sh)
if err != nil {
return err
}
signedPayload, err := wrapped.SignMessage(bytes.NewReader(payload), signatureoptions.WithContext(context.Background()))
if err != nil {
return errors.Wrap(err, "unable to sign SBOM")
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error {
_, err := os.Stdout.Write(signedPayload)
return err
},
})
return nil
}
func findValidDigest(digests []string) string {
// since we are only using the OCI repo provider for this source we are safe that this is only 1 value
// see https://github.com/anchore/stereoscope/blob/25ebd49a842b5ac0a20c2e2b4b81335b64ad248c/pkg/image/oci/registry_provider.go#L57-L63
split := strings.Split(digests[0], "sha256:")
return split[1]
}
func formatPredicateType(format sbom.Format) string {
switch format.ID() {
case spdx22json.ID:
return in_toto.PredicateSPDX
case cyclonedxjson.ID:
// Tentative see https://github.com/in-toto/attestation/issues/82
return "https://cyclonedx.org/bom"
case syftjson.ID:
return "https://syft.dev/bom"
default:
return ""
}
}
+56
View File
@@ -0,0 +1,56 @@
package attest
import (
"errors"
"fmt"
"io"
"os"
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/sigstore/cosign/pkg/cosign"
)
func selectPassFunc(keypath, password string) (cosign.PassFunc, error) {
keyContents, err := os.ReadFile(keypath)
if err != nil {
return nil, err
}
var fn cosign.PassFunc = func(bool) (b []byte, err error) { return nil, nil }
_, err = cosign.LoadPrivateKey(keyContents, nil)
if err != nil {
fn = func(bool) (b []byte, err error) {
return fetchPassword(password)
}
}
return fn, nil
}
func fetchPassword(password string) (b []byte, err error) {
potentiallyPipedInput, err := internal.IsPipedInput()
if err != nil {
log.Warnf("unable to determine if there is piped input: %+v", err)
}
switch {
case password != "":
return []byte(password), nil
case potentiallyPipedInput:
// handle piped in passwords
pwBytes, err := io.ReadAll(os.Stdin)
if err != nil {
return nil, fmt.Errorf("unable to get password from stdin: %w", err)
}
// be resilient to input that may have newline characters (in case someone is using echo without -n)
cleanPw := strings.TrimRight(string(pwBytes), "\n")
return []byte(cleanPw), nil
case internal.IsTerminal():
return cosign.GetPassFromTerm(false)
}
return nil, errors.New("no method available to fetch password")
}
+143
View File
@@ -0,0 +1,143 @@
package cli
import (
"fmt"
"strings"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal/logger"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/event"
"github.com/gookit/color"
"github.com/spf13/cobra"
"github.com/spf13/viper"
"github.com/wagoodman/go-partybus"
)
const indent = " "
// New constructs the `syft packages` command, aliases the root command to `syft packages`,
// and constructs the `syft power-user` and `syft attest` commands. It is also responsible for
// organizing flag usage and injecting the application config for each command.
// Because of how the `cobra` library behaves, the application's configuration is initialized
// at this level. Values from the config should only be used after `app.LoadAllValues` has been called.
// Cobra does not have knowledge of the user provided flags until the `RunE` block of each command.
// `RunE` is the earliest that the complete application configuration can be loaded.
func New() (*cobra.Command, error) {
app := &config.Application{}
// allow for nested options to be specified via environment variables
// e.g. pod.context = APPNAME_POD_CONTEXT
v := viper.NewWithOptions(viper.EnvKeyReplacer(strings.NewReplacer(".", "_", "-", "_")))
// since root is aliased as the packages cmd we need to construct this command first
// we also need the command to have information about the `root` options because of this alias
ro := &options.RootOptions{}
po := &options.PackagesOptions{}
packagesCmd := Packages(v, app, ro, po)
// root options are also passed to the attestCmd so that a user provided config location can be discovered
attestCmd := Attest(v, app, ro)
poweruserCmd := PowerUser(v, app, ro)
// rootCmd is currently an alias for the packages command
rootCmd := &cobra.Command{
Short: packagesCmd.Short,
Long: packagesCmd.Long,
Args: packagesCmd.Args,
Example: packagesCmd.Example,
SilenceUsage: true,
SilenceErrors: true,
RunE: packagesCmd.RunE,
Version: version.FromBuild().Version,
}
rootCmd.SetVersionTemplate(fmt.Sprintf("%s {{.Version}}\n", internal.ApplicationName))
// start adding flags to all the commands
err := ro.AddFlags(rootCmd, v)
if err != nil {
return nil, err
}
// package flags need to be decorated onto the rootCmd so that rootCmd can function as a packages alias
err = po.AddFlags(rootCmd, v)
if err != nil {
return nil, err
}
// attest also uses flags from the packagesCmd since it generates an sbom
err = po.AddFlags(attestCmd, v)
if err != nil {
return nil, err
}
// poweruser also uses the packagesCmd flags since it is a specialized version of the command
err = po.AddFlags(poweruserCmd, v)
if err != nil {
return nil, err
}
// Add sub-commands.
rootCmd.AddCommand(packagesCmd)
rootCmd.AddCommand(attestCmd)
rootCmd.AddCommand(poweruserCmd)
rootCmd.AddCommand(Completion())
rootCmd.AddCommand(Version(v, app))
return rootCmd, err
}
func validateArgs(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
// in the case that no arguments are given we want to show the help text and return with a non-0 return code.
if err := cmd.Help(); err != nil {
return fmt.Errorf("unable to display help: %w", err)
}
return fmt.Errorf("an image/directory argument is required")
}
return cobra.MaximumNArgs(1)(cmd, args)
}
func checkForApplicationUpdate() {
log.Debugf("checking if new vesion of %s is available", internal.ApplicationName)
isAvailable, newVersion, err := version.IsUpdateAvailable()
if err != nil {
// this should never stop the application
log.Errorf(err.Error())
}
if isAvailable {
log.Infof("new version of %s is available: %s (current version is %s)", internal.ApplicationName, newVersion, version.FromBuild().Version)
bus.Publish(partybus.Event{
Type: event.AppUpdateAvailable,
Value: newVersion,
})
} else {
log.Debugf("no new %s update available", internal.ApplicationName)
}
}
func logApplicationConfig(app *config.Application) {
log.Debugf("application config:\n%+v", color.Magenta.Sprint(app.String()))
}
func newLogWrapper(app *config.Application) {
cfg := logger.LogrusConfig{
EnableConsole: (app.Log.FileLocation == "" || app.Verbosity > 0) && !app.Quiet,
EnableFile: app.Log.FileLocation != "",
Level: app.Log.LevelOpt,
Structured: app.Log.Structured,
FileLocation: app.Log.FileLocation,
}
logWrapper := logger.NewLogrusLogger(cfg)
syft.SetLogger(logWrapper)
stereoscope.SetLogger(&logger.LogrusNestedLogger{
Logger: logWrapper.Logger.WithField("from-lib", "stereoscope"),
})
}
+54
View File
@@ -0,0 +1,54 @@
package cli
import (
"os"
"github.com/spf13/cobra"
)
func Completion() *cobra.Command {
cmd := &cobra.Command{
Use: "completion [bash|zsh|fish]",
Short: "Generate a shell completion for Syft (listing local docker images)",
Long: `To load completions (docker image list):
Bash:
$ source <(syft completion bash)
# To load completions for each session, execute once:
Linux:
$ syft completion bash > /etc/bash_completion.d/syft
MacOS:
$ syft completion bash > /usr/local/etc/bash_completion.d/syft
Zsh:
# If shell completion is not already enabled in your environment you will need
# to enable it. You can execute the following once:
$ echo "autoload -U compinit; compinit" >> ~/.zshrc
# To load completions for each session, execute once:
$ syft completion zsh > "${fpath[1]}/_syft"
# You will need to start a new shell for this setup to take effect.
Fish:
$ syft completion fish | source
# To load completions for each session, execute once:
$ syft completion fish > ~/.config/fish/completions/syft.fish
`,
DisableFlagsInUseLine: true,
ValidArgs: []string{"bash", "zsh", "fish"},
Args: cobra.ExactValidArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
var err error
switch args[0] {
case "bash":
err = cmd.Root().GenBashCompletion(os.Stdout)
case "zsh":
err = cmd.Root().GenZshCompletion(os.Stdout)
case "fish":
err = cmd.Root().GenFishCompletion(os.Stdout, true)
}
if err != nil {
return err
}
return nil
},
}
return cmd
}
@@ -1,4 +1,4 @@
package cmd
package eventloop
import (
"errors"
@@ -15,7 +15,7 @@ import (
// signal interrupts. Is responsible for handling each event relative to a given UI an to coordinate eventing until
// an eventual graceful exit.
// nolint:funlen
func eventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *partybus.Subscription, cleanupFn func(), uxs ...ui.UI) error {
func EventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *partybus.Subscription, cleanupFn func(), uxs ...ui.UI) error {
defer cleanupFn()
events := subscription.Events()
var err error
@@ -1,4 +1,4 @@
package cmd
package eventloop
import (
"fmt"
@@ -42,7 +42,7 @@ func (u *uiMock) Teardown(_ bool) error {
return u.Called().Error(0)
}
func Test_eventLoop_gracefulExit(t *testing.T) {
func Test_EventLoop_gracefulExit(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -92,7 +92,7 @@ func Test_eventLoop_gracefulExit(t *testing.T) {
}
assert.NoError(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -109,7 +109,7 @@ func Test_eventLoop_gracefulExit(t *testing.T) {
testWithTimeout(t, 5*time.Second, test)
}
func Test_eventLoop_workerError(t *testing.T) {
func Test_EventLoop_workerError(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -155,7 +155,7 @@ func Test_eventLoop_workerError(t *testing.T) {
// ensure we see an error returned
assert.ErrorIs(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -174,7 +174,7 @@ func Test_eventLoop_workerError(t *testing.T) {
testWithTimeout(t, 5*time.Second, test)
}
func Test_eventLoop_unsubscribeError(t *testing.T) {
func Test_EventLoop_unsubscribeError(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -226,7 +226,7 @@ func Test_eventLoop_unsubscribeError(t *testing.T) {
// unsubscribe errors should be handled and ignored, not propagated. We are additionally asserting that
// this case is handled as a controlled shutdown (this test should not timeout)
assert.NoError(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -243,7 +243,7 @@ func Test_eventLoop_unsubscribeError(t *testing.T) {
testWithTimeout(t, 5*time.Second, test)
}
func Test_eventLoop_handlerError(t *testing.T) {
func Test_EventLoop_handlerError(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -296,7 +296,7 @@ func Test_eventLoop_handlerError(t *testing.T) {
// handle errors SHOULD propagate the event loop. We are additionally asserting that this case is
// handled as a controlled shutdown (this test should not timeout)
assert.ErrorIs(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -315,7 +315,7 @@ func Test_eventLoop_handlerError(t *testing.T) {
testWithTimeout(t, 5*time.Second, test)
}
func Test_eventLoop_signalsStopExecution(t *testing.T) {
func Test_EventLoop_signalsStopExecution(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -351,7 +351,7 @@ func Test_eventLoop_signalsStopExecution(t *testing.T) {
}
assert.NoError(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -368,7 +368,7 @@ func Test_eventLoop_signalsStopExecution(t *testing.T) {
testWithTimeout(t, 5*time.Second, test)
}
func Test_eventLoop_uiTeardownError(t *testing.T) {
func Test_EventLoop_uiTeardownError(t *testing.T) {
test := func(t *testing.T) {
testBus := partybus.NewBus()
@@ -421,7 +421,7 @@ func Test_eventLoop_uiTeardownError(t *testing.T) {
// ensure we see an error returned
assert.ErrorIs(t,
eventLoop(
EventLoop(
worker(),
signaler(),
subscription,
@@ -1,4 +1,4 @@
package cmd
package eventloop
import (
"os"
@@ -6,7 +6,7 @@ import (
"syscall"
)
func setupSignals() <-chan os.Signal {
func SetupSignals() <-chan os.Signal {
c := make(chan os.Signal, 1) // Note: A buffered channel is recommended for this; see https://golang.org/pkg/os/signal/#Notify
interruptions := []os.Signal{
@@ -1,9 +1,10 @@
package cmd
package eventloop
import (
"crypto"
"fmt"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
@@ -11,12 +12,12 @@ import (
"github.com/anchore/syft/syft/source"
)
type task func(*sbom.Artifacts, *source.Source) ([]artifact.Relationship, error)
type Task func(*sbom.Artifacts, *source.Source) ([]artifact.Relationship, error)
func tasks() ([]task, error) {
var tasks []task
func Tasks(app *config.Application) ([]Task, error) {
var tasks []Task
generators := []func() (task, error){
generators := []func(app *config.Application) (Task, error){
generateCatalogPackagesTask,
generateCatalogFileMetadataTask,
generateCatalogFileDigestsTask,
@@ -26,7 +27,7 @@ func tasks() ([]task, error) {
}
for _, generator := range generators {
task, err := generator()
task, err := generator(app)
if err != nil {
return nil, err
}
@@ -39,13 +40,13 @@ func tasks() ([]task, error) {
return tasks, nil
}
func generateCatalogPackagesTask() (task, error) {
if !appConfig.Package.Cataloger.Enabled {
func generateCatalogPackagesTask(app *config.Application) (Task, error) {
if !app.Package.Cataloger.Enabled {
return nil, nil
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
packageCatalog, relationships, theDistro, err := syft.CatalogPackages(src, appConfig.Package.ToConfig())
packageCatalog, relationships, theDistro, err := syft.CatalogPackages(src, app.Package.ToConfig())
if err != nil {
return nil, err
}
@@ -59,15 +60,15 @@ func generateCatalogPackagesTask() (task, error) {
return task, nil
}
func generateCatalogFileMetadataTask() (task, error) {
if !appConfig.FileMetadata.Cataloger.Enabled {
func generateCatalogFileMetadataTask(app *config.Application) (Task, error) {
if !app.FileMetadata.Cataloger.Enabled {
return nil, nil
}
metadataCataloger := file.NewMetadataCataloger()
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
resolver, err := src.FileResolver(app.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
@@ -83,8 +84,8 @@ func generateCatalogFileMetadataTask() (task, error) {
return task, nil
}
func generateCatalogFileDigestsTask() (task, error) {
if !appConfig.FileMetadata.Cataloger.Enabled {
func generateCatalogFileDigestsTask(app *config.Application) (Task, error) {
if !app.FileMetadata.Cataloger.Enabled {
return nil, nil
}
@@ -98,7 +99,7 @@ func generateCatalogFileDigestsTask() (task, error) {
}
var hashes []crypto.Hash
for _, hashStr := range appConfig.FileMetadata.Digests {
for _, hashStr := range app.FileMetadata.Digests {
name := file.CleanDigestAlgorithmName(hashStr)
hashObj, ok := supportedHashAlgorithms[name]
if !ok {
@@ -113,7 +114,7 @@ func generateCatalogFileDigestsTask() (task, error) {
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
resolver, err := src.FileResolver(app.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
@@ -129,23 +130,23 @@ func generateCatalogFileDigestsTask() (task, error) {
return task, nil
}
func generateCatalogSecretsTask() (task, error) {
if !appConfig.Secrets.Cataloger.Enabled {
func generateCatalogSecretsTask(app *config.Application) (Task, error) {
if !app.Secrets.Cataloger.Enabled {
return nil, nil
}
patterns, err := file.GenerateSearchPatterns(file.DefaultSecretsPatterns, appConfig.Secrets.AdditionalPatterns, appConfig.Secrets.ExcludePatternNames)
patterns, err := file.GenerateSearchPatterns(file.DefaultSecretsPatterns, app.Secrets.AdditionalPatterns, app.Secrets.ExcludePatternNames)
if err != nil {
return nil, err
}
secretsCataloger, err := file.NewSecretsCataloger(patterns, appConfig.Secrets.RevealValues, appConfig.Secrets.SkipFilesAboveSize)
secretsCataloger, err := file.NewSecretsCataloger(patterns, app.Secrets.RevealValues, app.Secrets.SkipFilesAboveSize)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.Secrets.Cataloger.ScopeOpt)
resolver, err := src.FileResolver(app.Secrets.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
@@ -161,8 +162,8 @@ func generateCatalogSecretsTask() (task, error) {
return task, nil
}
func generateCatalogFileClassificationsTask() (task, error) {
if !appConfig.FileClassification.Cataloger.Enabled {
func generateCatalogFileClassificationsTask(app *config.Application) (Task, error) {
if !app.FileClassification.Cataloger.Enabled {
return nil, nil
}
@@ -173,7 +174,7 @@ func generateCatalogFileClassificationsTask() (task, error) {
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileClassification.Cataloger.ScopeOpt)
resolver, err := src.FileResolver(app.FileClassification.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
@@ -189,18 +190,18 @@ func generateCatalogFileClassificationsTask() (task, error) {
return task, nil
}
func generateCatalogContentsTask() (task, error) {
if !appConfig.FileContents.Cataloger.Enabled {
func generateCatalogContentsTask(app *config.Application) (Task, error) {
if !app.FileContents.Cataloger.Enabled {
return nil, nil
}
contentsCataloger, err := file.NewContentsCataloger(appConfig.FileContents.Globs, appConfig.FileContents.SkipFilesAboveSize)
contentsCataloger, err := file.NewContentsCataloger(app.FileContents.Globs, app.FileContents.SkipFilesAboveSize)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileContents.Cataloger.ScopeOpt)
resolver, err := src.FileResolver(app.FileContents.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
@@ -216,7 +217,7 @@ func generateCatalogContentsTask() (task, error) {
return task, nil
}
func runTask(t task, a *sbom.Artifacts, src *source.Source, c chan<- artifact.Relationship, errs chan<- error) {
func RunTask(t Task, a *sbom.Artifacts, src *source.Source, c chan<- artifact.Relationship, errs chan<- error) {
defer close(c)
relationships, err := t(a, src)
+28
View File
@@ -0,0 +1,28 @@
package options
import (
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
)
type AttestOptions struct {
Key string
}
var _ Interface = (*AttestOptions)(nil)
func (o *AttestOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.PersistentFlags().StringVarP(&o.Key, "key", "", "cosign.key",
"path to the private key file to use for attestation")
return bindAttestationConfigOptions(cmd.PersistentFlags(), v)
}
func bindAttestationConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
if err := v.BindPFlag("attest.key", flags.Lookup("key")); err != nil {
return err
}
return nil
}
@@ -1,11 +1,11 @@
package cmd
package options
import (
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/sbom"
)
func formatAliases(ids ...sbom.FormatID) (aliases []string) {
func FormatAliases(ids ...sbom.FormatID) (aliases []string) {
for _, id := range ids {
switch id {
case syft.JSONFormatID:
@@ -22,6 +22,8 @@ func formatAliases(ids ...sbom.FormatID) (aliases []string) {
aliases = append(aliases, "cyclonedx-xml")
case syft.CycloneDxJSONFormatID:
aliases = append(aliases, "cyclonedx-json")
case syft.GitHubID:
aliases = append(aliases, "github", "github-json")
default:
aliases = append(aliases, string(id))
}
+11
View File
@@ -0,0 +1,11 @@
package options
import (
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
type Interface interface {
// AddFlags adds this options' flags to the cobra command.
AddFlags(cmd *cobra.Command, v *viper.Viper) error
}
+118
View File
@@ -0,0 +1,118 @@
package options
import (
"fmt"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/pkg/cataloger"
"github.com/anchore/syft/syft/source"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
)
type PackagesOptions struct {
Scope string
Output []string
File string
Platform string
Host string
Username string
Password string
Dockerfile string
Exclude []string
OverwriteExistingImage bool
ImportTimeout uint
}
var _ Interface = (*PackagesOptions)(nil)
func (o *PackagesOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.PersistentFlags().StringVarP(&o.Scope, "scope", "s", cataloger.DefaultSearchConfig().Scope.String(),
fmt.Sprintf("selection of layers to catalog, options=%v", source.AllScopes))
cmd.PersistentFlags().StringArrayVarP(&o.Output, "output", "o", FormatAliases(table.ID),
fmt.Sprintf("report output format, options=%v", FormatAliases(syft.FormatIDs()...)))
cmd.PersistentFlags().StringVarP(&o.File, "file", "", "",
"file to write the default report output to (default is STDOUT)")
cmd.PersistentFlags().StringVarP(&o.Platform, "platform", "", "",
"an optional platform specifier for container image sources (e.g. 'linux/arm64', 'linux/arm64/v8', 'arm64', 'linux')")
cmd.PersistentFlags().StringVarP(&o.Host, "host", "H", "",
"the hostname or URL of the Anchore Enterprise instance to upload to")
cmd.PersistentFlags().StringVarP(&o.Username, "username", "u", "",
"the username to authenticate against Anchore Enterprise")
cmd.PersistentFlags().StringVarP(&o.Password, "password", "p", "",
"the password to authenticate against Anchore Enterprise")
cmd.PersistentFlags().StringVarP(&o.Dockerfile, "dockerfile", "d", "",
"include dockerfile for upload to Anchore Enterprise")
cmd.PersistentFlags().StringArrayVarP(&o.Exclude, "exclude", "", nil,
"exclude paths from being scanned using a glob expression")
cmd.PersistentFlags().BoolVarP(&o.OverwriteExistingImage, "overwrite-existing-image", "", false,
"overwrite an existing image during the upload to Anchore Enterprise")
cmd.PersistentFlags().UintVarP(&o.ImportTimeout, "import-timeout", "", 30,
"set a timeout duration (in seconds) for the upload to Anchore Enterprise")
return bindPackageConfigOptions(cmd.PersistentFlags(), v)
}
func bindPackageConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
// Formatting & Input options //////////////////////////////////////////////
if err := v.BindPFlag("package.cataloger.scope", flags.Lookup("scope")); err != nil {
return err
}
if err := v.BindPFlag("file", flags.Lookup("file")); err != nil {
return err
}
if err := v.BindPFlag("exclude", flags.Lookup("exclude")); err != nil {
return err
}
if err := v.BindPFlag("output", flags.Lookup("output")); err != nil {
return err
}
if err := v.BindPFlag("platform", flags.Lookup("platform")); err != nil {
return err
}
// Upload options //////////////////////////////////////////////////////////
if err := v.BindPFlag("anchore.host", flags.Lookup("host")); err != nil {
return err
}
if err := v.BindPFlag("anchore.username", flags.Lookup("username")); err != nil {
return err
}
if err := v.BindPFlag("anchore.password", flags.Lookup("password")); err != nil {
return err
}
if err := v.BindPFlag("anchore.dockerfile", flags.Lookup("dockerfile")); err != nil {
return err
}
if err := v.BindPFlag("anchore.overwrite-existing-image", flags.Lookup("overwrite-existing-image")); err != nil {
return err
}
if err := v.BindPFlag("anchore.import-timeout", flags.Lookup("import-timeout")); err != nil {
return err
}
return nil
}
+36
View File
@@ -0,0 +1,36 @@
package options
import (
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
)
type RootOptions struct {
Config string
Quiet bool
Verbose int
}
var _ Interface = (*RootOptions)(nil)
func (o *RootOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.PersistentFlags().StringVarP(&o.Config, "config", "c", "", "application config file")
cmd.PersistentFlags().CountVarP(&o.Verbose, "verbose", "v", "increase verbosity (-v = info, -vv = debug)")
cmd.PersistentFlags().BoolVarP(&o.Quiet, "quiet", "q", false, "suppress all logging output")
return bindRootConfigOptions(cmd.PersistentFlags(), v)
}
func bindRootConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
if err := v.BindPFlag("config", flags.Lookup("config")); err != nil {
return err
}
if err := v.BindPFlag("verbosity", flags.Lookup("verbose")); err != nil {
return err
}
if err := v.BindPFlag("quiet", flags.Lookup("quiet")); err != nil {
return err
}
return nil
}
+18
View File
@@ -0,0 +1,18 @@
package options
import (
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
)
func IsVerbose(app *config.Application) (result bool) {
isPipedInput, err := internal.IsPipedInput()
if err != nil {
// since we can't tell if there was piped input we assume that there could be to disable the ETUI
log.Warnf("unable to determine if there is piped input: %+v", err)
return true
}
// verbosity should consider if there is piped input (in which case we should not show the ETUI)
return app.Verbosity > 0 || isPipedInput
}
+17
View File
@@ -0,0 +1,17 @@
package options
import (
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
type VersionOptions struct {
Output string
}
var _ Interface = (*PackagesOptions)(nil)
func (o *VersionOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.Flags().StringVarP(&o.Output, "output", "o", "text", "format to show version information (available=[text, json])")
return nil
}
+79
View File
@@ -0,0 +1,79 @@
package cli
import (
"fmt"
"log"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/cmd/syft/cli/packages"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
const (
packagesExample = ` {{.appName}} {{.command}} alpine:latest a summary of discovered packages
{{.appName}} {{.command}} alpine:latest -o json show all possible cataloging details
{{.appName}} {{.command}} alpine:latest -o cyclonedx show a CycloneDX formatted SBOM
{{.appName}} {{.command}} alpine:latest -o cyclonedx-json show a CycloneDX JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx show a SPDX 2.2 Tag-Value formatted SBOM
{{.appName}} {{.command}} alpine:latest -o spdx-json show a SPDX 2.2 JSON formatted SBOM
{{.appName}} {{.command}} alpine:latest -vv show verbose debug information
Supports the following image sources:
{{.appName}} {{.command}} yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} path/to/a/file/or/dir a Docker tar, OCI tar, OCI directory, or generic filesystem directory
`
schemeHelpHeader = "You can also explicitly specify the scheme to use:"
imageSchemeHelp = ` {{.appName}} {{.command}} docker:yourrepo/yourimage:tag explicitly use the Docker daemon
{{.appName}} {{.command}} podman:yourrepo/yourimage:tag explicitly use the Podman daemon
{{.appName}} {{.command}} registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
{{.appName}} {{.command}} docker-archive:path/to/yourimage.tar use a tarball from disk for archives created from "docker save"
{{.appName}} {{.command}} oci-archive:path/to/yourimage.tar use a tarball from disk for OCI archives (from Skopeo or otherwise)
{{.appName}} {{.command}} oci-dir:path/to/yourimage read directly from a path on disk for OCI layout directories (from Skopeo or otherwise)
`
nonImageSchemeHelp = ` {{.appName}} {{.command}} dir:path/to/yourproject read directly from a path on disk (any directory)
{{.appName}} {{.command}} file:path/to/yourproject/file read directly from a path on disk (any single file)
`
packagesSchemeHelp = "\n" + indent + schemeHelpHeader + "\n" + imageSchemeHelp + nonImageSchemeHelp
packagesHelp = packagesExample + packagesSchemeHelp
)
func Packages(v *viper.Viper, app *config.Application, ro *options.RootOptions, po *options.PackagesOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "packages [SOURCE]",
Short: "Generate a package SBOM",
Long: "Generate a packaged-based Software Bill Of Materials (SBOM) from container images and filesystems",
Example: internal.Tprintf(packagesHelp, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "packages",
}),
Args: func(cmd *cobra.Command, args []string) error {
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
}
// configure logging for command
newLogWrapper(app)
logApplicationConfig(app)
return validateArgs(cmd, args)
},
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
if app.CheckForAppUpdate {
checkForApplicationUpdate()
}
return packages.Run(cmd.Context(), app, args)
},
}
err := po.AddFlags(cmd, v)
if err != nil {
log.Fatal(err)
}
return cmd
}
+240
View File
@@ -0,0 +1,240 @@
package packages
import (
"context"
"fmt"
"io/ioutil"
"os"
"strings"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/cmd/syft/cli/eventloop"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/anchore"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/hashicorp/go-multierror"
"github.com/wagoodman/go-partybus"
)
func Run(ctx context.Context, app *config.Application, args []string) error {
writer, err := makeWriter(app.Outputs, app.File)
if err != nil {
return err
}
defer func() {
if err := writer.Close(); err != nil {
log.Warnf("unable to write to report destination: %w", err)
}
}()
// could be an image or a directory, with or without a scheme
userInput := args[0]
si, err := source.ParseInput(userInput, app.Platform, true)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
eventBus := partybus.NewBus()
stereoscope.SetBus(eventBus)
syft.SetBus(eventBus)
return eventloop.EventLoop(
execWorker(app, *si, writer),
eventloop.SetupSignals(),
eventBus.Subscribe(),
stereoscope.Cleanup,
ui.Select(options.IsVerbose(app), app.Quiet)...,
)
}
func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
src, cleanup, err := source.New(si, app.Registry.ToOptions(), app.Exclusions)
if cleanup != nil {
defer cleanup()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", si.UserInput, err)
return
}
s, err := GenerateSBOM(src, errs, app)
if err != nil {
errs <- err
return
}
if s == nil {
errs <- fmt.Errorf("no SBOM produced for %q", si.UserInput)
}
if app.Anchore.Host != "" {
if err := runPackageSbomUpload(src, *s, app); err != nil {
errs <- err
return
}
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error { return writer.Write(*s) },
})
}()
return errs
}
func GenerateSBOM(src *source.Source, errs chan error, app *config.Application) (*sbom.SBOM, error) {
tasks, err := eventloop.Tasks(app)
if err != nil {
return nil, err
}
s := sbom.SBOM{
Source: src.Metadata,
Descriptor: sbom.Descriptor{
Name: internal.ApplicationName,
Version: version.FromBuild().Version,
Configuration: app,
},
}
buildRelationships(&s, src, tasks, errs)
return &s, nil
}
func buildRelationships(s *sbom.SBOM, src *source.Source, tasks []eventloop.Task, errs chan error) {
var relationships []<-chan artifact.Relationship
for _, task := range tasks {
c := make(chan artifact.Relationship)
relationships = append(relationships, c)
go eventloop.RunTask(task, &s.Artifacts, src, c, errs)
}
s.Relationships = append(s.Relationships, MergeRelationships(relationships...)...)
}
func MergeRelationships(cs ...<-chan artifact.Relationship) (relationships []artifact.Relationship) {
for _, c := range cs {
for n := range c {
relationships = append(relationships, n)
}
}
return relationships
}
func runPackageSbomUpload(src *source.Source, s sbom.SBOM, app *config.Application) error {
log.Infof("uploading results to %s", app.Anchore.Host)
if src.Metadata.Scheme != source.ImageScheme {
return fmt.Errorf("unable to upload results: only images are supported")
}
var dockerfileContents []byte
if app.Anchore.Dockerfile != "" {
if _, err := os.Stat(app.Anchore.Dockerfile); os.IsNotExist(err) {
return fmt.Errorf("unable dockerfile=%q does not exist: %w", app.Anchore.Dockerfile, err)
}
fh, err := os.Open(app.Anchore.Dockerfile)
if err != nil {
return fmt.Errorf("unable to open dockerfile=%q: %w", app.Anchore.Dockerfile, err)
}
dockerfileContents, err = ioutil.ReadAll(fh)
if err != nil {
return fmt.Errorf("unable to read dockerfile=%q: %w", app.Anchore.Dockerfile, err)
}
}
c, err := anchore.NewClient(anchore.Configuration{
BaseURL: app.Anchore.Host,
Username: app.Anchore.Username,
Password: app.Anchore.Password,
})
if err != nil {
return fmt.Errorf("failed to create anchore client: %w", err)
}
importCfg := anchore.ImportConfig{
ImageMetadata: src.Image.Metadata,
SBOM: s,
Dockerfile: dockerfileContents,
OverwriteExistingUpload: app.Anchore.OverwriteExistingImage,
Timeout: app.Anchore.ImportTimeout,
}
if err := c.Import(context.Background(), importCfg); err != nil {
return fmt.Errorf("failed to upload results to host=%s: %+v", app.Anchore.Host, err)
}
return nil
}
// makeWriter creates a sbom.Writer for output or returns an error. this will either return a valid writer
// or an error but neither both and if there is no error, sbom.Writer.Close() should be called
func makeWriter(outputs []string, defaultFile string) (sbom.Writer, error) {
outputOptions, err := parseOptions(outputs, defaultFile)
if err != nil {
return nil, err
}
writer, err := sbom.NewWriter(outputOptions...)
if err != nil {
return nil, err
}
return writer, nil
}
// parseOptions utility to parse command-line option strings and retain the existing behavior of default format and file
func parseOptions(outputs []string, defaultFile string) (out []sbom.WriterOption, errs error) {
// always should have one option -- we generally get the default of "table", but just make sure
if len(outputs) == 0 {
outputs = append(outputs, string(table.ID))
}
for _, name := range outputs {
name = strings.TrimSpace(name)
// split to at most two parts for <format>=<file>
parts := strings.SplitN(name, "=", 2)
// the format name is the first part
name = parts[0]
// default to the --file or empty string if not specified
file := defaultFile
// If a file is specified as part of the output formatName, use that
if len(parts) > 1 {
file = parts[1]
}
format := syft.FormatByName(name)
if format == nil {
errs = multierror.Append(errs, fmt.Errorf("bad output format: '%s'", name))
continue
}
out = append(out, sbom.NewWriterOption(format, file))
}
return out, errs
}
+49
View File
@@ -0,0 +1,49 @@
package cli
import (
"fmt"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/cmd/syft/cli/poweruser"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
const powerUserExample = ` {{.appName}} {{.command}} <image>
DEPRECATED - THIS COMMAND WILL BE REMOVED in v1.0.0
Only image sources are supported (e.g. docker: , podman: , docker-archive: , oci: , etc.), the directory source (dir:) is not supported.
All behavior is controlled via application configuration and environment variables (see https://github.com/anchore/syft#configuration)
`
func PowerUser(v *viper.Viper, app *config.Application, ro *options.RootOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "power-user [IMAGE]",
Short: "Run bulk operations on container images",
Example: internal.Tprintf(powerUserExample, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "power-user",
}),
Args: func(cmd *cobra.Command, args []string) error {
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
}
// configure logging for command
newLogWrapper(app)
logApplicationConfig(app)
return validateArgs(cmd, args)
},
Hidden: true,
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
if app.CheckForAppUpdate {
checkForApplicationUpdate()
}
return poweruser.Run(cmd.Context(), app, args)
},
}
return cmd
}
+116
View File
@@ -0,0 +1,116 @@
package poweruser
import (
"context"
"fmt"
"os"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/cmd/syft/cli/eventloop"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/cmd/syft/cli/packages"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/gookit/color"
"github.com/wagoodman/go-partybus"
)
func Run(ctx context.Context, app *config.Application, args []string) error {
writer, err := sbom.NewWriter(sbom.WriterOption{
Format: syftjson.Format(),
Path: app.File,
})
if err != nil {
return err
}
defer func() {
if err := writer.Close(); err != nil {
log.Warnf("unable to write to report destination: %+v", err)
}
// inform user at end of run that command will be removed
deprecated := color.Style{color.Red, color.OpBold}.Sprint("DEPRECATED: This command will be removed in v1.0.0")
fmt.Fprintln(os.Stderr, deprecated)
}()
userInput := args[0]
si, err := source.ParseInput(userInput, app.Platform, true)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
eventBus := partybus.NewBus()
stereoscope.SetBus(eventBus)
syft.SetBus(eventBus)
return eventloop.EventLoop(
execWorker(app, *si, writer),
eventloop.SetupSignals(),
eventBus.Subscribe(),
stereoscope.Cleanup,
ui.Select(options.IsVerbose(app), app.Quiet)...,
)
}
func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
app.Secrets.Cataloger.Enabled = true
app.FileMetadata.Cataloger.Enabled = true
app.FileContents.Cataloger.Enabled = true
app.FileClassification.Cataloger.Enabled = true
tasks, err := eventloop.Tasks(app)
if err != nil {
errs <- err
return
}
src, cleanup, err := source.New(si, app.Registry.ToOptions(), app.Exclusions)
if err != nil {
errs <- err
return
}
if cleanup != nil {
defer cleanup()
}
s := sbom.SBOM{
Source: src.Metadata,
Descriptor: sbom.Descriptor{
Name: internal.ApplicationName,
Version: version.FromBuild().Version,
Configuration: app,
},
}
var relationships []<-chan artifact.Relationship
for _, task := range tasks {
c := make(chan artifact.Relationship)
relationships = append(relationships, c)
go eventloop.RunTask(task, &s.Artifacts, src, c, errs)
}
s.Relationships = append(s.Relationships, packages.MergeRelationships(relationships...)...)
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error { return writer.Write(s) },
})
}()
return errs
}
+25 -15
View File
@@ -1,33 +1,41 @@
package cmd
package cli
import (
"encoding/json"
"fmt"
"log"
"os"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/version"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
var versionCmdOutputFormat string
func Version(v *viper.Viper, app *config.Application) *cobra.Command {
o := &options.VersionOptions{}
cmd := &cobra.Command{
Use: "version",
Short: "show the version",
RunE: func(cmd *cobra.Command, args []string) error {
return printVersion(o.Output)
},
}
var versionCmd = &cobra.Command{
Use: "version",
Short: "show the version",
Run: printVersion,
err := o.AddFlags(cmd, v)
if err != nil {
log.Fatal(err)
}
return cmd
}
func init() {
versionCmd.Flags().StringVarP(&versionCmdOutputFormat, "output", "o", "text", "format to show version information (available=[text, json])")
rootCmd.AddCommand(versionCmd)
}
func printVersion(_ *cobra.Command, _ []string) {
func printVersion(output string) error {
versionInfo := version.FromBuild()
switch versionCmdOutputFormat {
switch output {
case "text":
fmt.Println("Application: ", internal.ApplicationName)
fmt.Println("Version: ", versionInfo.Version)
@@ -55,7 +63,9 @@ func printVersion(_ *cobra.Command, _ []string) {
os.Exit(1)
}
default:
fmt.Printf("unsupported output format: %s\n", versionCmdOutputFormat)
fmt.Printf("unsupported output format: %s\n", output)
os.Exit(1)
}
return nil
}
+18
View File
@@ -0,0 +1,18 @@
package main
import (
"log"
"github.com/anchore/syft/cmd/syft/cli"
)
func main() {
cli, err := cli.New()
if err != nil {
log.Fatalf("error during command construction: %v", err)
}
if err := cli.Execute(); err != nil {
log.Fatalf("error during command execution: %v", err)
}
}
+64 -60
View File
@@ -1,19 +1,20 @@
module github.com/anchore/syft
go 1.17
go 1.18
require (
github.com/CycloneDX/cyclonedx-go v0.5.0
github.com/CycloneDX/cyclonedx-go v0.5.2
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
github.com/acobaugh/osrelease v0.1.0
github.com/adrg/xdg v0.2.1
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29
github.com/anchore/stereoscope v0.0.0-20220307154759-8a5a70c227d3
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b
github.com/anchore/stereoscope v0.0.0-20220406160859-c03a18a6b270
github.com/antihax/optional v1.0.0
github.com/bmatcuk/doublestar/v4 v4.0.2
github.com/dustin/go-humanize v1.0.0
@@ -23,7 +24,6 @@ require (
github.com/google/uuid v1.3.0
github.com/gookit/color v1.4.2
github.com/hashicorp/go-multierror v1.1.1
github.com/in-toto/in-toto-golang v0.3.4-0.20211211042327-af1f9fb822bf
github.com/jinzhu/copier v0.3.2
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/mholt/archiver/v3 v3.5.1
@@ -32,44 +32,45 @@ require (
github.com/mitchellh/mapstructure v1.4.3
github.com/olekukonko/tablewriter v0.0.5
github.com/pelletier/go-toml v1.9.4
github.com/pkg/profile v1.5.0
github.com/scylladb/go-set v1.0.2
// pinned to pull in 386 arch fix: https://github.com/scylladb/go-set/commit/cc7b2070d91ebf40d233207b633e28f5bd8f03a5
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e
github.com/sergi/go-diff v1.2.0
github.com/sigstore/sigstore v1.1.1-0.20220217212907-e48ca03a5ba7
github.com/sirupsen/logrus v1.8.1
github.com/spdx/tools-golang v0.2.0
github.com/spf13/afero v1.8.0
github.com/spf13/cobra v1.3.0
github.com/spf13/cobra v1.4.0
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.10.1
github.com/stretchr/testify v1.7.0
github.com/stretchr/testify v1.7.1
github.com/vifraa/gopom v0.1.0
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb
github.com/x-cray/logrus-prefixed-formatter v0.5.2
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/mod v0.5.1
golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd
golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3
golang.org/x/net v0.0.0-20220325170049-de3da57026de
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
gopkg.in/yaml.v2 v2.4.0
)
require (
github.com/docker/docker v20.10.12+incompatible
github.com/sigstore/cosign v1.6.0
github.com/in-toto/in-toto-golang v0.3.4-0.20211211042327-af1f9fb822bf
github.com/sigstore/cosign v1.7.2
github.com/sigstore/sigstore v1.2.1-0.20220401110139-0e610e39782f
)
require (
bitbucket.org/creachadair/shell v0.0.6 // indirect
cloud.google.com/go v0.100.2 // indirect
cloud.google.com/go/compute v1.3.0 // indirect
cloud.google.com/go/iam v0.1.1 // indirect
cloud.google.com/go/kms v1.3.0 // indirect
cloud.google.com/go/storage v1.21.0 // indirect
github.com/Azure/azure-sdk-for-go v61.5.0+incompatible // indirect
cloud.google.com/go/compute v1.5.0 // indirect
cloud.google.com/go/iam v0.3.0 // indirect
cloud.google.com/go/kms v1.4.0 // indirect
cloud.google.com/go/storage v1.22.0 // indirect
github.com/Azure/azure-sdk-for-go v63.0.0+incompatible // indirect
github.com/Azure/go-autorest v14.2.0+incompatible // indirect
github.com/Azure/go-autorest/autorest v0.11.24 // indirect
github.com/Azure/go-autorest/autorest v0.11.25 // indirect
github.com/Azure/go-autorest/autorest/adal v0.9.18 // indirect
github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 // indirect
github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 // indirect
@@ -87,21 +88,21 @@ require (
github.com/armon/go-metrics v0.3.10 // indirect
github.com/armon/go-radix v1.0.0 // indirect
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
github.com/aws/aws-sdk-go v1.43.0 // indirect
github.com/aws/aws-sdk-go-v2 v1.13.0 // indirect
github.com/aws/aws-sdk-go-v2/config v1.13.1 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.8.0 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.10.0 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.4 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.2.0 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.5 // indirect
github.com/aws/aws-sdk-go-v2/service/ecr v1.14.0 // indirect
github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.11.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.7.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.9.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.14.0 // indirect
github.com/aws/smithy-go v1.10.0 // indirect
github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20220216180153-3d7835abdf40 // indirect
github.com/aws/aws-sdk-go v1.43.30 // indirect
github.com/aws/aws-sdk-go-v2 v1.14.0 // indirect
github.com/aws/aws-sdk-go-v2/config v1.14.0 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.9.0 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.11.0 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.5 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.3.0 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.6 // indirect
github.com/aws/aws-sdk-go-v2/service/ecr v1.15.0 // indirect
github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.12.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.8.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.10.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.15.0 // indirect
github.com/aws/smithy-go v1.11.0 // indirect
github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20220228164355-396b2034c795 // indirect
github.com/benbjohnson/clock v1.1.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bgentry/speakeasy v0.1.0 // indirect
@@ -132,7 +133,7 @@ require (
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/jsonreference v0.19.6 // indirect
github.com/go-openapi/loads v0.21.1 // indirect
github.com/go-openapi/runtime v0.23.1 // indirect
github.com/go-openapi/runtime v0.23.3 // indirect
github.com/go-openapi/spec v0.20.4 // indirect
github.com/go-openapi/strfmt v0.21.2 // indirect
github.com/go-openapi/swag v0.21.1 // indirect
@@ -142,7 +143,7 @@ require (
github.com/go-playground/universal-translator v0.18.0 // indirect
github.com/go-playground/validator/v10 v10.10.0 // indirect
github.com/go-stack/stack v1.8.1 // indirect
github.com/golang-jwt/jwt/v4 v4.2.0 // indirect
github.com/golang-jwt/jwt/v4 v4.3.0 // indirect
github.com/golang/glog v1.0.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/mock v1.6.0 // indirect
@@ -152,26 +153,27 @@ require (
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/trillian v1.4.0 // indirect
github.com/googleapis/gax-go/v2 v2.1.1 // indirect
github.com/googleapis/gax-go/v2 v2.2.0 // indirect
github.com/googleapis/gnostic v0.5.5 // indirect
github.com/googleapis/go-type-adapters v1.0.0 // indirect
github.com/gorilla/websocket v1.4.2 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-hclog v1.1.0 // indirect
github.com/hashicorp/go-hclog v1.2.0 // indirect
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
github.com/hashicorp/go-plugin v1.4.3 // indirect
github.com/hashicorp/go-retryablehttp v0.7.0 // indirect
github.com/hashicorp/go-rootcerts v1.0.2 // indirect
github.com/hashicorp/go-secure-stdlib/mlock v0.1.2 // indirect
github.com/hashicorp/go-secure-stdlib/parseutil v0.1.2 // indirect
github.com/hashicorp/go-secure-stdlib/parseutil v0.1.3 // indirect
github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 // indirect
github.com/hashicorp/go-sockaddr v1.0.2 // indirect
github.com/hashicorp/go-uuid v1.0.2 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/hashicorp/go-version v1.4.0 // indirect
github.com/hashicorp/golang-lru v0.5.4 // indirect
github.com/hashicorp/vault/api v1.3.1 // indirect
github.com/hashicorp/vault/api v1.5.0 // indirect
github.com/hashicorp/vault/sdk v0.4.1 // indirect
github.com/hashicorp/yamux v0.0.0-20211028200310-0bc27b27de87 // indirect
github.com/imdario/mergo v0.3.12 // indirect
@@ -182,6 +184,7 @@ require (
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/leodido/go-urn v1.2.1 // indirect
github.com/letsencrypt/boulder v0.0.0-20220331220046-b23ab962616e // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
github.com/miekg/pkcs11 v1.1.1 // indirect
@@ -194,7 +197,7 @@ require (
github.com/oklog/ulid v1.3.1 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pierrec/lz4 v2.6.1+incompatible // indirect
github.com/prometheus/client_golang v1.11.0 // indirect
github.com/prometheus/client_golang v1.12.1 // indirect
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.32.1 // indirect
github.com/prometheus/procfs v0.7.3 // indirect
@@ -209,14 +212,15 @@ require (
github.com/sigstore/rekor v0.4.1-0.20220114213500-23f583409af3 // indirect
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 // indirect
github.com/soheilhy/cmux v0.1.5 // indirect
github.com/spiffe/go-spiffe/v2 v2.0.0-beta.12 // indirect
github.com/spiffe/go-spiffe/v2 v2.0.0 // indirect
github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7 // indirect
github.com/tent/canonical-json-go v0.0.0-20130607151641-96e4ba3a7613 // indirect
github.com/thales-e-security/pool v0.0.2 // indirect
github.com/theupdateframework/go-tuf v0.0.0-20220211205608-f0c3294f63b9 // indirect
github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399 // indirect
github.com/tmc/grpc-websocket-proxy v0.0.0-20201229170055-e5319fda7802 // indirect
github.com/urfave/cli v1.22.5 // indirect
github.com/xanzy/go-gitlab v0.56.0 // indirect
github.com/xanzy/go-gitlab v0.62.0 // indirect
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
github.com/zeebo/errs v1.2.2 // indirect
@@ -247,23 +251,23 @@ require (
go.uber.org/atomic v1.9.0 // indirect
go.uber.org/multierr v1.7.0 // indirect
go.uber.org/zap v1.21.0 // indirect
golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 // indirect
golang.org/x/tools v0.1.8 // indirect
google.golang.org/api v0.70.0 // indirect
google.golang.org/protobuf v1.27.1 // indirect
golang.org/x/time v0.0.0-20220224211638-0e9765cccd65 // indirect
golang.org/x/tools v0.1.10 // indirect
google.golang.org/api v0.74.0 // indirect
google.golang.org/protobuf v1.28.0 // indirect
gopkg.in/cheggaaa/pb.v1 v1.0.28 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect
k8s.io/api v0.23.3 // indirect
k8s.io/apimachinery v0.23.3 // indirect
k8s.io/client-go v0.23.3 // indirect
k8s.io/klog/v2 v2.40.1 // indirect
k8s.io/api v0.23.5 // indirect
k8s.io/apimachinery v0.23.5 // indirect
k8s.io/client-go v0.23.5 // indirect
k8s.io/klog/v2 v2.60.1-0.20220317184644-43cc75f9ae89 // indirect
k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf // indirect
k8s.io/utils v0.0.0-20220127004650-9b3446523e65 // indirect
knative.dev/pkg v0.0.0-20220202132633-df430fa0dd96 // indirect
k8s.io/utils v0.0.0-20220210201930-3a6ce19ff2f9 // indirect
knative.dev/pkg v0.0.0-20220325200448-1f7514acd0c2 // indirect
sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 // indirect
sigs.k8s.io/release-utils v0.4.1-0.20220207182343-6dadf2228617 // indirect
sigs.k8s.io/release-utils v0.6.0 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.1 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)
@@ -315,15 +319,15 @@ require (
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
golang.org/x/crypto v0.0.0-20220213190939-1e6e3497d506 // indirect
golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 // indirect
golang.org/x/crypto v0.0.0-20220214200702-86341886e292 // indirect
golang.org/x/oauth2 v0.0.0-20220309155454-6242fa91716a // indirect
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect
golang.org/x/sys v0.0.0-20220209214540-3681064d5158 // indirect
golang.org/x/sys v0.0.0-20220328115105-d36c6a25d886 // indirect
golang.org/x/text v0.3.7 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20220218161850-94dd64e39d7c // indirect
google.golang.org/grpc v1.44.0 // indirect
google.golang.org/genproto v0.0.0-20220405205423-9d709892a2bf // indirect
google.golang.org/grpc v1.45.0 // indirect
gopkg.in/ini.v1 v1.66.2 // indirect
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
)
+292 -263
View File
File diff suppressed because it is too large Load Diff
+6 -2
View File
@@ -56,7 +56,7 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
authedCtx := c.newRequestContext(ctxWithTimeout)
stage.Current = "starting session"
startOperation, _, err := c.client.ImportsApi.CreateOperation(authedCtx)
startOperation, createResponse, err := c.client.ImportsApi.CreateOperation(authedCtx)
if err != nil {
var detail = "no details given"
var openAPIErr external.GenericOpenAPIError
@@ -65,6 +65,8 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
}
return fmt.Errorf("unable to start import session: %w: %s", err, detail)
}
defer createResponse.Body.Close()
prog.N++
sessionID := startOperation.Uuid
@@ -98,7 +100,7 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
Force: optional.NewBool(cfg.OverwriteExistingUpload),
}
_, _, err = c.client.ImagesApi.AddImage(authedCtx, imageModel, &opts)
_, addResponse, err := c.client.ImagesApi.AddImage(authedCtx, imageModel, &opts)
if err != nil {
var detail = "no details given"
var openAPIErr external.GenericOpenAPIError
@@ -107,6 +109,8 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
}
return fmt.Errorf("unable to complete import session=%q: %w: %s", sessionID, err, detail)
}
defer addResponse.Body.Close()
prog.N++
stage.Current = ""
+3 -3
View File
@@ -56,14 +56,14 @@ func sbomFixture() sbom.SBOM {
Name: "name",
Version: "version",
FoundBy: "foundBy",
Locations: []source.Location{
{
Locations: source.NewLocationSet(
source.Location{
Coordinates: source.Coordinates{
RealPath: "path",
FileSystemID: "layerID",
},
},
},
),
Licenses: []string{"license"},
Language: pkg.Python,
Type: pkg.PythonPkg,
+88 -94
View File
@@ -7,17 +7,20 @@ import (
"reflect"
"strings"
"github.com/sirupsen/logrus"
"github.com/adrg/xdg"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/mitchellh/go-homedir"
"github.com/sirupsen/logrus"
"github.com/spf13/viper"
"gopkg.in/yaml.v2"
)
var ErrApplicationConfigNotFound = fmt.Errorf("application config not found")
var catalogerEnabledDefault = false
var (
ErrApplicationConfigNotFound = fmt.Errorf("application config not found")
catalogerEnabledDefault = false
)
type defaultValueLoader interface {
loadDefaultValues(*viper.Viper)
@@ -29,13 +32,15 @@ type parser interface {
// Application is the main syft application configuration.
type Application struct {
ConfigPath string `yaml:",omitempty" json:"configPath"` // the location where the application config was read from (either from -c or discovered while loading)
// the location where the application config was read from (either from -c or discovered while loading); default .syft.yaml
ConfigPath string `yaml:"configPath,omitempty" json:"configPath" mapstructure:"config"`
Verbosity uint `yaml:"verbosity,omitempty" json:"verbosity" mapstructure:"verbosity"`
// -q, indicates to not show any status output to stderr (ETUI or logging UI)
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"`
Outputs []string `yaml:"output" json:"output" mapstructure:"output"` // -o, the format to use for output
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"` // -q, indicates to not show any status output to stderr (ETUI or logging UI)
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Anchore anchore `yaml:"anchore" json:"anchore" mapstructure:"anchore"` // options for interacting with Anchore Engine/Enterprise
CliOptions CliOnlyOptions `yaml:"-" json:"-"` // all options only available through the CLI (not via env vars or config)
Dev development `yaml:"dev" json:"dev" mapstructure:"dev"`
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
@@ -49,54 +54,34 @@ type Application struct {
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
}
// PowerUserCatalogerEnabledDefault switches all catalogers to be enabled when running power-user command
func PowerUserCatalogerEnabledDefault() {
catalogerEnabledDefault = true
}
func (cfg *Application) LoadAllValues(v *viper.Viper, configPath string) error {
// priority order: viper.Set, flag, env, config, kv, defaults
// flags have already been loaded into viper by command construction
func newApplicationConfig(v *viper.Viper, cliOpts CliOnlyOptions) *Application {
config := &Application{
CliOptions: cliOpts,
}
config.loadDefaultValues(v)
return config
}
// LoadApplicationConfig populates the given viper object with application configuration discovered on disk
func LoadApplicationConfig(v *viper.Viper, cliOpts CliOnlyOptions) (*Application, error) {
// the user may not have a config, and this is OK, we can use the default config + default cobra cli values instead
config := newApplicationConfig(v, cliOpts)
if err := readConfig(v, cliOpts.ConfigPath); err != nil && !errors.Is(err, ErrApplicationConfigNotFound) {
return nil, err
}
if err := v.Unmarshal(config); err != nil {
return nil, fmt.Errorf("unable to parse config: %w", err)
}
config.ConfigPath = v.ConfigFileUsed()
if err := config.parseConfigValues(); err != nil {
return nil, fmt.Errorf("invalid application config: %w", err)
}
return config, nil
}
// init loads the default configuration values into the viper instance (before the config values are read and parsed).
func (cfg Application) loadDefaultValues(v *viper.Viper) {
// set the default values for primitive fields in this struct
v.SetDefault("check-for-app-update", true)
// for each field in the configuration struct, see if the field implements the defaultValueLoader interface and invoke it if it does
value := reflect.ValueOf(cfg)
for i := 0; i < value.NumField(); i++ {
// note: the defaultValueLoader method receiver is NOT a pointer receiver.
if loadable, ok := value.Field(i).Interface().(defaultValueLoader); ok {
// the field implements defaultValueLoader, call it
loadable.loadDefaultValues(v)
// check if user specified config; otherwise read all possible paths
if err := loadConfig(v, configPath); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); ok {
// Not Found; ignore this error
log.Debug("no config file found; proceeding with defaults")
}
}
// load default config values into viper
loadDefaultValues(v)
// load environment variables
v.SetEnvPrefix(internal.ApplicationName)
v.AllowEmptyEnv(true)
v.AutomaticEnv()
// unmarshal fully populated viper object onto config
err := v.Unmarshal(cfg)
if err != nil {
return err
}
// Convert all populated config options to their internal application values ex: scope string => scopeOpt source.Scope
return cfg.parseConfigValues()
}
func (cfg *Application) parseConfigValues() error {
@@ -110,7 +95,6 @@ func (cfg *Application) parseConfigValues() error {
return err
}
}
// parse nested config options
// for each field in the configuration struct, see if the field implements the parser interface
// note: the app config is a pointer, so we need to grab the elements explicitly (to traverse the address)
@@ -127,17 +111,6 @@ func (cfg *Application) parseConfigValues() error {
return nil
}
func (cfg *Application) parseFile() error {
if cfg.File != "" {
expandedPath, err := homedir.Expand(cfg.File)
if err != nil {
return fmt.Errorf("unable to expand file path=%q: %w", cfg.File, err)
}
cfg.File = expandedPath
}
return nil
}
func (cfg *Application) parseUploadOptions() error {
if cfg.Anchore.Host == "" && cfg.Anchore.Dockerfile != "" {
return fmt.Errorf("cannot provide dockerfile option without enabling upload")
@@ -152,23 +125,9 @@ func (cfg *Application) parseLogLevelOption() error {
// we should be able to quiet the console logging and leave file logging alone...
// ... this will be an enhancement for later
cfg.Log.LevelOpt = logrus.PanicLevel
case cfg.Log.Level != "":
if cfg.CliOptions.Verbosity > 0 {
return fmt.Errorf("cannot explicitly set log level (cfg file or env var) and use -v flag together")
}
lvl, err := logrus.ParseLevel(strings.ToLower(cfg.Log.Level))
if err != nil {
return fmt.Errorf("bad log level configured (%q): %w", cfg.Log.Level, err)
}
cfg.Log.LevelOpt = lvl
if cfg.Log.LevelOpt >= logrus.InfoLevel {
cfg.CliOptions.Verbosity = 1
}
default:
switch v := cfg.CliOptions.Verbosity; {
case cfg.Verbosity > 0:
switch v := cfg.Verbosity; {
case v == 1:
cfg.Log.LevelOpt = logrus.InfoLevel
case v >= 2:
@@ -176,6 +135,18 @@ func (cfg *Application) parseLogLevelOption() error {
default:
cfg.Log.LevelOpt = logrus.ErrorLevel
}
case cfg.Log.Level != "":
lvl, err := logrus.ParseLevel(strings.ToLower(cfg.Log.Level))
if err != nil {
return fmt.Errorf("bad log level configured (%q): %w", cfg.Log.Level, err)
}
cfg.Log.LevelOpt = lvl
if cfg.Log.LevelOpt >= logrus.InfoLevel {
cfg.Verbosity = 1
}
default:
cfg.Log.LevelOpt = logrus.InfoLevel
}
if cfg.Log.Level == "" {
@@ -185,43 +156,64 @@ func (cfg *Application) parseLogLevelOption() error {
return nil
}
func (cfg *Application) parseFile() error {
if cfg.File != "" {
expandedPath, err := homedir.Expand(cfg.File)
if err != nil {
return fmt.Errorf("unable to expand file path=%q: %w", cfg.File, err)
}
cfg.File = expandedPath
}
return nil
}
// init loads the default configuration values into the viper instance (before the config values are read and parsed).
func loadDefaultValues(v *viper.Viper) {
// set the default values for primitive fields in this struct
v.SetDefault("quiet", false)
v.SetDefault("check-for-app-update", true)
// for each field in the configuration struct, see if the field implements the defaultValueLoader interface and invoke it if it does
value := reflect.ValueOf(Application{})
for i := 0; i < value.NumField(); i++ {
// note: the defaultValueLoader method receiver is NOT a pointer receiver.
if loadable, ok := value.Field(i).Interface().(defaultValueLoader); ok {
// the field implements defaultValueLoader, call it
loadable.loadDefaultValues(v)
}
}
}
func (cfg Application) String() string {
// yaml is pretty human friendly (at least when compared to json)
appCfgStr, err := yaml.Marshal(&cfg)
appaStr, err := yaml.Marshal(&cfg)
if err != nil {
return err.Error()
}
return string(appCfgStr)
return string(appaStr)
}
// readConfig attempts to read the given config path from disk or discover an alternate store location
// nolint:funlen
func readConfig(v *viper.Viper, configPath string) error {
func loadConfig(v *viper.Viper, configPath string) error {
var err error
v.AutomaticEnv()
v.SetEnvPrefix(internal.ApplicationName)
// allow for nested options to be specified via environment variables
// e.g. pod.context = APPNAME_POD_CONTEXT
v.SetEnvKeyReplacer(strings.NewReplacer(".", "_", "-", "_"))
// use explicitly the given user config
if configPath != "" {
v.SetConfigFile(configPath)
if err := v.ReadInConfig(); err != nil {
return fmt.Errorf("unable to read application config=%q : %w", configPath, err)
}
v.Set("config", v.ConfigFileUsed())
// don't fall through to other options if the config path was explicitly provided
return nil
}
// start searching for valid configs in order...
// 1. look for .<appname>.yaml (in the current directory)
v.AddConfigPath(".")
v.SetConfigName("." + internal.ApplicationName)
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
} else if !errors.As(err, &viper.ConfigFileNotFoundError{}) {
return fmt.Errorf("unable to parse config=%q: %w", v.ConfigFileUsed(), err)
@@ -231,6 +223,7 @@ func readConfig(v *viper.Viper, configPath string) error {
v.AddConfigPath("." + internal.ApplicationName)
v.SetConfigName("config")
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
} else if !errors.As(err, &viper.ConfigFileNotFoundError{}) {
return fmt.Errorf("unable to parse config=%q: %w", v.ConfigFileUsed(), err)
@@ -242,6 +235,7 @@ func readConfig(v *viper.Viper, configPath string) error {
v.AddConfigPath(home)
v.SetConfigName("." + internal.ApplicationName)
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
} else if !errors.As(err, &viper.ConfigFileNotFoundError{}) {
return fmt.Errorf("unable to parse config=%q: %w", v.ConfigFileUsed(), err)
@@ -255,10 +249,10 @@ func readConfig(v *viper.Viper, configPath string) error {
}
v.SetConfigName("config")
if err = v.ReadInConfig(); err == nil {
v.Set("config", v.ConfigFileUsed())
return nil
} else if !errors.As(err, &viper.ConfigFileNotFoundError{}) {
return fmt.Errorf("unable to parse config=%q: %w", v.ConfigFileUsed(), err)
}
return ErrApplicationConfigNotFound
return nil
}
-52
View File
@@ -1,52 +0,0 @@
package config
import (
"github.com/docker/docker/pkg/homedir"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"path/filepath"
"testing"
)
func TestApplication_parseFile(t *testing.T) {
tests := []struct {
name string
config Application
expected string
wantErr require.ErrorAssertionFunc
}{
{
name: "expand home dir",
config: Application{
File: "~/place.txt",
},
expected: filepath.Join(homedir.Get(), "place.txt"),
},
{
name: "passthrough other paths",
config: Application{
File: "/other/place.txt",
},
expected: "/other/place.txt",
},
{
name: "no path",
config: Application{
File: "",
},
expected: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := tt.config
if tt.wantErr == nil {
tt.wantErr = require.NoError
}
tt.wantErr(t, cfg.parseFile())
assert.Equal(t, tt.expected, cfg.File)
})
}
}
-7
View File
@@ -1,7 +0,0 @@
package config
// CliOnlyOptions are options that are in the application config in memory, but are only exposed via CLI switches (not from unmarshaling a config file)
type CliOnlyOptions struct {
ConfigPath string // -c. where the read config is on disk
Verbosity int // -v or -vv , controlling which UI (ETUI vs logging) and what the log level should be
}
+1
View File
@@ -0,0 +1 @@
package config
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "3.0.1"
JSONSchemaVersion = "3.2.2"
)
@@ -5,6 +5,7 @@ import (
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal/formats/common"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
@@ -13,8 +14,9 @@ import (
func encodeComponent(p pkg.Package) cyclonedx.Component {
props := encodeProperties(p, "syft:package")
props = append(props, encodeCPEs(p)...)
if len(p.Locations) > 0 {
props = append(props, encodeProperties(p.Locations, "syft:location")...)
locations := p.Locations.ToSlice()
if len(locations) > 0 {
props = append(props, encodeProperties(locations, "syft:location")...)
}
if hasMetadata(p) {
props = append(props, encodeProperties(p.Metadata, "syft:metadata")...)
@@ -38,17 +40,33 @@ func encodeComponent(p pkg.Package) cyclonedx.Component {
Description: encodeDescription(p),
ExternalReferences: encodeExternalReferences(p),
Properties: properties,
BOMRef: deriveBomRef(p),
}
}
func deriveBomRef(p pkg.Package) string {
// try and parse the PURL if possible and append syft id to it, to make
// the purl unique in the BOM.
// TODO: In the future we may want to dedupe by PURL and combine components with
// the same PURL while preserving their unique metadata.
if parsedPURL, err := packageurl.FromString(p.PURL); err == nil {
parsedPURL.Qualifiers = append(parsedPURL.Qualifiers, packageurl.Qualifier{Key: "package-id", Value: string(p.ID())})
return parsedPURL.ToString()
}
// fallback is to use strictly the ID if there is no valid pURL
return string(p.ID())
}
func hasMetadata(p pkg.Package) bool {
return p.Metadata != nil
}
func decodeComponent(c *cyclonedx.Component) *pkg.Package {
values := map[string]string{}
for _, p := range *c.Properties {
values[p.Name] = p.Value
if c.Properties != nil {
for _, p := range *c.Properties {
values[p.Name] = p.Value
}
}
p := &pkg.Package{
@@ -68,13 +86,20 @@ func decodeComponent(c *cyclonedx.Component) *pkg.Package {
p.Type = pkg.TypeFromPURL(p.PURL)
}
if p.Language == "" {
p.Language = pkg.LanguageFromPURL(p.PURL)
}
return p
}
func decodeLocations(vals map[string]string) []source.Location {
func decodeLocations(vals map[string]string) source.LocationSet {
v := common.Decode(reflect.TypeOf([]source.Location{}), vals, "syft:location", CycloneDXFields)
out, _ := v.([]source.Location)
return out
out, ok := v.([]source.Location)
if !ok {
out = nil
}
return source.NewLocationSet(out...)
}
func decodePackageMetadata(vals map[string]string, c *cyclonedx.Component, typ pkg.MetadataType) interface{} {
@@ -1,6 +1,7 @@
package cyclonedxhelpers
import (
"fmt"
"testing"
"github.com/CycloneDX/cyclonedx-go"
@@ -26,9 +27,9 @@ func Test_encodeComponentProperties(t *testing.T) {
name: "from apk",
input: pkg.Package{
FoundBy: "cataloger",
Locations: []source.Location{
{Coordinates: source.Coordinates{RealPath: "test"}},
},
Locations: source.NewLocationSet(
source.Location{Coordinates: source.Coordinates{RealPath: "test"}},
),
Metadata: pkg.ApkMetadata{
Package: "libc-utils",
OriginPackage: "libc-dev",
@@ -139,3 +140,82 @@ func Test_encodeComponentProperties(t *testing.T) {
})
}
}
func Test_deriveBomRef(t *testing.T) {
pkgWithPurl := pkg.Package{
Name: "django",
Version: "1.11.1",
PURL: "pkg:pypi/django@1.11.1",
}
pkgWithPurl.SetID()
pkgWithOutPurl := pkg.Package{
Name: "django",
Version: "1.11.1",
PURL: "",
}
pkgWithOutPurl.SetID()
pkgWithBadPurl := pkg.Package{
Name: "django",
Version: "1.11.1",
PURL: "pkg:pyjango@1.11.1",
}
pkgWithBadPurl.SetID()
tests := []struct {
name string
pkg pkg.Package
want string
}{
{
name: "use pURL-id hybrid",
pkg: pkgWithPurl,
want: fmt.Sprintf("pkg:pypi/django@1.11.1?package-id=%s", pkgWithPurl.ID()),
},
{
name: "fallback to ID when pURL is invalid",
pkg: pkgWithBadPurl,
want: string(pkgWithBadPurl.ID()),
},
{
name: "fallback to ID when pURL is missing",
pkg: pkgWithOutPurl,
want: string(pkgWithOutPurl.ID()),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tt.pkg.ID()
assert.Equal(t, tt.want, deriveBomRef(tt.pkg))
})
}
}
func Test_decodeComponent(t *testing.T) {
javaComponentWithNoSyftProperties := cyclonedx.Component{
Name: "ch.qos.logback/logback-classic",
Version: "1.2.3",
PackageURL: "pkg:maven/ch.qos.logback/logback-classic@1.2.3",
Type: "library",
BOMRef: "pkg:maven/ch.qos.logback/logback-classic@1.2.3",
}
tests := []struct {
name string
component cyclonedx.Component
want pkg.Language
}{
{
name: "derive language from pURL if missing",
component: javaComponentWithNoSyftProperties,
want: pkg.Java,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, decodeComponent(&tt.component).Language)
})
}
}
@@ -5,6 +5,8 @@ import (
"io"
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/syft/internal/formats/common"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
@@ -44,7 +46,7 @@ func GetDecoder(format cyclonedx.BOMFileFormat) sbom.Decoder {
func toSyftModel(bom *cyclonedx.BOM) (*sbom.SBOM, error) {
meta := source.Metadata{}
if bom.Metadata != nil {
if bom.Metadata != nil && bom.Metadata.Component != nil {
meta = decodeMetadata(bom.Metadata.Component)
}
s := &sbom.SBOM{
@@ -157,6 +159,14 @@ func linuxReleaseFromOSComponent(component *cyclonedx.Component) *linux.Release
}
}
if component.Properties != nil {
values := map[string]string{}
for _, p := range *component.Properties {
values[p.Name] = p.Value
}
common.DecodeInto(&rel, values, "syft:distro", CycloneDXFields)
}
return rel
}
@@ -258,3 +258,28 @@ func Test_decode(t *testing.T) {
})
}
}
func Test_missingDataDecode(t *testing.T) {
bom := &cyclonedx.BOM{
Metadata: nil,
Components: &[]cyclonedx.Component{},
}
_, err := toSyftModel(bom)
assert.NoError(t, err)
bom.Metadata = &cyclonedx.Metadata{}
_, err = toSyftModel(bom)
assert.NoError(t, err)
pkg := decodeComponent(&cyclonedx.Component{
Licenses: &cyclonedx.Licenses{
{
License: nil,
},
},
})
assert.Len(t, pkg.Licenses, 0)
}
@@ -4,12 +4,15 @@ import (
"fmt"
"strings"
syftFile "github.com/anchore/syft/syft/file"
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/syft/syft/pkg"
)
//nolint:funlen, gocognit
func encodeExternalReferences(p pkg.Package) *[]cyclonedx.ExternalReference {
refs := []cyclonedx.ExternalReference{}
var refs []cyclonedx.ExternalReference
if hasMetadata(p) {
switch metadata := p.Metadata.(type) {
case pkg.ApkMetadata:
@@ -46,6 +49,19 @@ func encodeExternalReferences(p pkg.Package) *[]cyclonedx.ExternalReference {
Type: cyclonedx.ERTypeWebsite,
})
}
case pkg.JavaMetadata:
if len(metadata.ArchiveDigests) > 0 {
for _, digest := range metadata.ArchiveDigests {
refs = append(refs, cyclonedx.ExternalReference{
URL: "",
Type: cyclonedx.ERTypeBuildMeta,
Hashes: &[]cyclonedx.Hash{{
Algorithm: cyclonedx.HashAlgorithm(digest.Algorithm),
Value: digest.Value,
}},
})
}
}
case pkg.PythonPackageMetadata:
if metadata.DirectURLOrigin != nil && metadata.DirectURLOrigin.URL != "" {
ref := cyclonedx.ExternalReference{
@@ -79,6 +95,20 @@ func decodeExternalReferences(c *cyclonedx.Component, metadata interface{}) {
meta.Homepage = refURL(c, cyclonedx.ERTypeWebsite)
case *pkg.GemMetadata:
meta.Homepage = refURL(c, cyclonedx.ERTypeWebsite)
case *pkg.JavaMetadata:
var digests []syftFile.Digest
if ref := findExternalRef(c, cyclonedx.ERTypeBuildMeta); ref != nil {
if ref.Hashes != nil {
for _, hash := range *ref.Hashes {
digests = append(digests, syftFile.Digest{
Algorithm: string(hash.Algorithm),
Value: hash.Value,
})
}
}
}
meta.ArchiveDigests = digests
case *pkg.PythonPackageMetadata:
if meta.DirectURLOrigin == nil {
meta.DirectURLOrigin = &pkg.PythonDirectURLOriginInfo{}
@@ -26,7 +26,9 @@ func encodeLicenses(p pkg.Package) *cyclonedx.Licenses {
func decodeLicenses(c *cyclonedx.Component) (out []string) {
if c.Licenses != nil {
for _, l := range *c.Licenses {
out = append(out, l.License.ID)
if l.License != nil {
out = append(out, l.License.ID)
}
}
}
return
@@ -22,5 +22,6 @@ func ExternalRefs(p pkg.Package) (externalRefs []ExternalRef) {
ReferenceType: PurlExternalRefType,
})
}
return externalRefs
}
@@ -13,6 +13,8 @@ func SourceInfo(p pkg.Package) string {
answer = "acquired package info from RPM DB"
case pkg.ApkPkg:
answer = "acquired package info from APK DB"
case pkg.DartPubPkg:
answer = "acquired package info from pubspec manifest"
case pkg.DebPkg:
answer = "acquired package info from DPKG DB"
case pkg.NpmPkg:
@@ -33,7 +35,7 @@ func SourceInfo(p pkg.Package) string {
answer = "acquired package info from the following paths"
}
var paths []string
for _, l := range p.Locations {
for _, l := range p.Locations.ToSlice() {
paths = append(paths, l.RealPath)
}
@@ -18,10 +18,10 @@ func Test_SourceInfo(t *testing.T) {
name: "locations are captured",
input: pkg.Package{
// note: no type given
Locations: []source.Location{
Locations: source.NewLocationSet(
source.NewVirtualLocation("/a-place", "/b-place"),
source.NewVirtualLocation("/c-place", "/d-place"),
},
),
},
expected: []string{
"from the following paths",
@@ -126,6 +126,14 @@ func Test_SourceInfo(t *testing.T) {
"from PHP composer manifest",
},
},
{
input: pkg.Package{
Type: pkg.DartPubPkg,
},
expected: []string{
"from pubspec manifest",
},
},
}
var pkgTypes []pkg.Type
for _, test := range tests {
@@ -0,0 +1,13 @@
package spdxhelpers
import (
"regexp"
)
var expr = regexp.MustCompile("[^a-zA-Z0-9.-]")
// SPDX spec says SPDXID must be:
// "SPDXRef-"[idstring] where [idstring] is a unique string containing letters, numbers, ., and/or -
func SanitizeElementID(id string) string {
return expr.ReplaceAllString(id, "-")
}
@@ -0,0 +1,39 @@
package spdxhelpers
import (
"testing"
"github.com/stretchr/testify/assert"
)
func Test_SanitizeElementID(t *testing.T) {
tests := []struct {
input string
expected string
}{
{
input: "letters",
expected: "letters",
},
{
input: "ssl-client",
expected: "ssl-client",
},
{
input: "ssl_client",
expected: "ssl-client",
},
{
input: "go-module-sigs.k8s.io/structured-merge-diff/v3",
expected: "go-module-sigs.k8s.io-structured-merge-diff-v3",
},
}
for _, test := range tests {
t.Run(test.input, func(t *testing.T) {
actual := SanitizeElementID(test.input)
assert.Equal(t, test.expected, actual)
})
}
}
@@ -1,6 +1,7 @@
package spdxhelpers
import (
"errors"
"strconv"
"strings"
@@ -17,6 +18,10 @@ import (
)
func ToSyftModel(doc *spdx.Document2_2) (*sbom.SBOM, error) {
if doc == nil {
return nil, errors.New("cannot convert SPDX document to Syft model because document is nil")
}
spdxIDMap := make(map[string]interface{})
s := &sbom.SBOM{
@@ -304,6 +309,14 @@ func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interf
Architecture: arch,
Maintainer: p.PackageOriginatorPerson,
}
case pkg.JavaPkg:
var digests []file.Digest
for algorithm, value := range p.PackageChecksums {
digests = append(digests, file.Digest{Algorithm: string(algorithm), Value: value.Value})
}
return pkg.JavaMetadataType, pkg.JavaMetadata{
ArchiveDigests: digests,
}
}
return pkg.UnknownMetadataType, nil
}
+12 -12
View File
@@ -157,9 +157,9 @@ func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
catalog.Add(pkg.Package{
Name: "package-1",
Version: "1.0.1",
Locations: []source.Location{
Locations: source.NewLocationSet(
source.NewLocationFromImage(string(ref1.RealPath), *ref1, img),
},
),
Type: pkg.PythonPkg,
FoundBy: "the-cataloger-1",
Language: pkg.Python,
@@ -169,7 +169,7 @@ func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
Name: "package-1",
Version: "1.0.1",
},
PURL: "a-purl-1",
PURL: "a-purl-1", // intentionally a bad pURL for test fixtures
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"),
},
@@ -177,9 +177,9 @@ func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
catalog.Add(pkg.Package{
Name: "package-2",
Version: "2.0.1",
Locations: []source.Location{
Locations: source.NewLocationSet(
source.NewLocationFromImage(string(ref2.RealPath), *ref2, img),
},
),
Type: pkg.DebPkg,
FoundBy: "the-cataloger-2",
MetadataType: pkg.DpkgMetadataType,
@@ -187,7 +187,7 @@ func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
Package: "package-2",
Version: "2.0.1",
},
PURL: "a-purl-2",
PURL: "pkg:deb/debian/package-2@2.0.1",
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
@@ -234,9 +234,9 @@ func newDirectoryCatalog() *pkg.Catalog {
Version: "1.0.1",
Type: pkg.PythonPkg,
FoundBy: "the-cataloger-1",
Locations: []source.Location{
Locations: source.NewLocationSet(
source.NewLocation("/some/path/pkg1"),
},
),
Language: pkg.Python,
MetadataType: pkg.PythonPackageMetadataType,
Licenses: []string{"MIT"},
@@ -249,7 +249,7 @@ func newDirectoryCatalog() *pkg.Catalog {
},
},
},
PURL: "a-purl-2",
PURL: "a-purl-2", // intentionally a bad pURL for test fixtures
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
@@ -259,15 +259,15 @@ func newDirectoryCatalog() *pkg.Catalog {
Version: "2.0.1",
Type: pkg.DebPkg,
FoundBy: "the-cataloger-2",
Locations: []source.Location{
Locations: source.NewLocationSet(
source.NewLocation("/some/path/pkg1"),
},
),
MetadataType: pkg.DpkgMetadataType,
Metadata: pkg.DpkgMetadata{
Package: "package-2",
Version: "2.0.1",
},
PURL: "a-purl-2",
PURL: "pkg:deb/debian/package-2@2.0.1",
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
@@ -1,10 +1,10 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"serialNumber": "urn:uuid:498e659b-0758-4a7f-816e-91bee18df634",
"serialNumber": "urn:uuid:dec3f6b4-8458-48bb-b60d-dfd312f6ec4e",
"version": 1,
"metadata": {
"timestamp": "2022-03-08T12:30:39Z",
"timestamp": "2022-04-01T11:48:04-04:00",
"tools": [
{
"vendor": "anchore",
@@ -20,6 +20,7 @@
},
"components": [
{
"bom-ref": "b85dbb4e6ece5082",
"type": "library",
"name": "package-1",
"version": "1.0.1",
@@ -56,11 +57,12 @@
]
},
{
"bom-ref": "pkg:deb/debian/package-2@2.0.1?package-id=ceda99598967ae8d",
"type": "library",
"name": "package-2",
"version": "2.0.1",
"cpe": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"purl": "a-purl-2",
"purl": "pkg:deb/debian/package-2@2.0.1",
"properties": [
{
"name": "syft:package:foundBy",
@@ -1,10 +1,10 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"serialNumber": "urn:uuid:342c3d2c-d26e-47b6-94d6-92fbf41da945",
"serialNumber": "urn:uuid:054d973e-fe99-4762-92e4-eaf01997ae41",
"version": 1,
"metadata": {
"timestamp": "2022-03-08T12:30:39Z",
"timestamp": "2022-04-01T11:48:04-04:00",
"tools": [
{
"vendor": "anchore",
@@ -13,7 +13,7 @@
}
],
"component": {
"bom-ref": "711095b1cdf90cce",
"bom-ref": "e777314b02b362e4",
"type": "container",
"name": "user-image-input",
"version": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
@@ -21,6 +21,7 @@
},
"components": [
{
"bom-ref": "2a46171f91c8d4bc",
"type": "library",
"name": "package-1",
"version": "1.0.1",
@@ -52,7 +53,7 @@
},
{
"name": "syft:location:0:layerID",
"value": "sha256:16e64541f2ddf59a90391ce7bb8af90313f7d373f2105d88f3d3267b72e0ebab"
"value": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59"
},
{
"name": "syft:location:0:path",
@@ -61,11 +62,12 @@
]
},
{
"bom-ref": "pkg:deb/debian/package-2@2.0.1?package-id=ae77680e9b1d087e",
"type": "library",
"name": "package-2",
"version": "2.0.1",
"cpe": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"purl": "a-purl-2",
"purl": "pkg:deb/debian/package-2@2.0.1",
"properties": [
{
"name": "syft:package:foundBy",
@@ -81,7 +83,7 @@
},
{
"name": "syft:location:0:layerID",
"value": "sha256:de6c235f76ea24c8503ec08891445b5d6a8bdf8249117ed8d8b0b6fb3ebe4f67"
"value": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec"
},
{
"name": "syft:location:0:path",
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:892f8304-0142-45b1-b411-cade3c53057f" version="1">
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:554fd820-210b-40c8-8c0b-75690274e21c" version="1">
<metadata>
<timestamp>2022-03-08T12:30:33Z</timestamp>
<timestamp>2022-04-01T11:57:46-04:00</timestamp>
<tools>
<tool>
<vendor>anchore</vendor>
@@ -14,7 +14,7 @@
</component>
</metadata>
<components>
<component type="library">
<component bom-ref="b85dbb4e6ece5082" type="library">
<name>package-1</name>
<version>1.0.1</version>
<licenses>
@@ -32,11 +32,11 @@
<property name="syft:location:0:path">/some/path/pkg1</property>
</properties>
</component>
<component type="library">
<component bom-ref="pkg:deb/debian/package-2@2.0.1?package-id=ceda99598967ae8d" type="library">
<name>package-2</name>
<version>2.0.1</version>
<cpe>cpe:2.3:*:some:package:2:*:*:*:*:*:*:*</cpe>
<purl>a-purl-2</purl>
<purl>pkg:deb/debian/package-2@2.0.1</purl>
<properties>
<property name="syft:package:foundBy">the-cataloger-2</property>
<property name="syft:package:metadataType">DpkgMetadata</property>
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:5fa94827-eb85-4f32-a62d-76fb6e89a2dd" version="1">
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:1535f940-172f-4d97-8280-d5a5764d1557" version="1">
<metadata>
<timestamp>2022-03-08T12:30:33Z</timestamp>
<timestamp>2022-04-01T11:57:46-04:00</timestamp>
<tools>
<tool>
<vendor>anchore</vendor>
@@ -9,13 +9,13 @@
<version>[not provided]</version>
</tool>
</tools>
<component bom-ref="711095b1cdf90cce" type="container">
<component bom-ref="e777314b02b362e4" type="container">
<name>user-image-input</name>
<version>sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368</version>
</component>
</metadata>
<components>
<component type="library">
<component bom-ref="2a46171f91c8d4bc" type="library">
<name>package-1</name>
<version>1.0.1</version>
<licenses>
@@ -30,20 +30,20 @@
<property name="syft:package:language">python</property>
<property name="syft:package:metadataType">PythonPackageMetadata</property>
<property name="syft:package:type">python</property>
<property name="syft:location:0:layerID">sha256:16e64541f2ddf59a90391ce7bb8af90313f7d373f2105d88f3d3267b72e0ebab</property>
<property name="syft:location:0:layerID">sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59</property>
<property name="syft:location:0:path">/somefile-1.txt</property>
</properties>
</component>
<component type="library">
<component bom-ref="pkg:deb/debian/package-2@2.0.1?package-id=ae77680e9b1d087e" type="library">
<name>package-2</name>
<version>2.0.1</version>
<cpe>cpe:2.3:*:some:package:2:*:*:*:*:*:*:*</cpe>
<purl>a-purl-2</purl>
<purl>pkg:deb/debian/package-2@2.0.1</purl>
<properties>
<property name="syft:package:foundBy">the-cataloger-2</property>
<property name="syft:package:metadataType">DpkgMetadata</property>
<property name="syft:package:type">deb</property>
<property name="syft:location:0:layerID">sha256:de6c235f76ea24c8503ec08891445b5d6a8bdf8249117ed8d8b0b6fb3ebe4f67</property>
<property name="syft:location:0:layerID">sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec</property>
<property name="syft:location:0:path">/somefile-2.txt</property>
<property name="syft:metadata:installedSize">0</property>
</properties>
+185
View File
@@ -0,0 +1,185 @@
package github
import (
"fmt"
"strings"
"time"
"github.com/mholt/archiver/v3"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
// toGithubModel converts the provided SBOM to a GitHub dependency model
func toGithubModel(s *sbom.SBOM) DependencySnapshot {
scanTime := time.Now().Format(time.RFC3339) // TODO is there a record of this somewhere?
v := version.FromBuild().Version
if v == "[not provided]" {
v = "0.0.0-dev"
}
return DependencySnapshot{
Version: 0,
// TODO allow property input to specify the Job, Sha, and Ref
Detector: DetectorMetadata{
Name: internal.ApplicationName,
URL: "https://github.com/anchore/syft",
Version: v,
},
Metadata: toSnapshotMetadata(s),
Manifests: toGithubManifests(s),
Scanned: scanTime,
}
}
// toSnapshotMetadata captures the linux distribution information and other metadata
func toSnapshotMetadata(s *sbom.SBOM) Metadata {
out := Metadata{}
if s.Artifacts.LinuxDistribution != nil {
d := s.Artifacts.LinuxDistribution
qualifiers := packageurl.Qualifiers{}
if len(d.IDLike) > 0 {
qualifiers = append(qualifiers, packageurl.Qualifier{
Key: "like",
Value: strings.Join(d.IDLike, ","),
})
}
purl := packageurl.NewPackageURL("generic", "", d.ID, d.VersionID, qualifiers, "")
out["syft:distro"] = purl.ToString()
}
return out
}
func filesystem(p pkg.Package) string {
locations := p.Locations.ToSlice()
if len(locations) > 0 {
return locations[0].FileSystemID
}
return ""
}
// isArchive returns true if the path appears to be an archive
func isArchive(path string) bool {
_, err := archiver.ByExtension(path)
return err == nil
}
// toPath Generates a string representation of the package location, optionally including the layer hash
func toPath(s source.Metadata, p pkg.Package) string {
inputPath := strings.TrimPrefix(s.Path, "./")
if inputPath == "." {
inputPath = ""
}
locations := p.Locations.ToSlice()
if len(locations) > 0 {
location := locations[0]
packagePath := location.RealPath
if location.VirtualPath != "" {
packagePath = location.VirtualPath
}
packagePath = strings.TrimPrefix(packagePath, "/")
switch s.Scheme {
case source.ImageScheme:
image := strings.ReplaceAll(s.ImageMetadata.UserInput, ":/", "//")
return fmt.Sprintf("%s:/%s", image, packagePath)
case source.FileScheme:
if isArchive(inputPath) {
return fmt.Sprintf("%s:/%s", inputPath, packagePath)
}
return inputPath
case source.DirectoryScheme:
if inputPath != "" {
return fmt.Sprintf("%s/%s", inputPath, packagePath)
}
return packagePath
}
}
return fmt.Sprintf("%s%s", inputPath, s.ImageMetadata.UserInput)
}
// toGithubManifests manifests, each of which represents a specific location that has dependencies
func toGithubManifests(s *sbom.SBOM) Manifests {
manifests := map[string]*Manifest{}
for _, p := range s.Artifacts.PackageCatalog.Sorted() {
path := toPath(s.Source, p)
manifest, ok := manifests[path]
if !ok {
manifest = &Manifest{
Name: path,
File: FileInfo{
SourceLocation: path,
},
Resolved: DependencyGraph{},
}
fs := filesystem(p)
if fs != "" {
manifest.Metadata = Metadata{
"syft:filesystem": fs,
}
}
manifests[path] = manifest
}
name := dependencyName(p)
manifest.Resolved[name] = DependencyNode{
PackageURL: p.PURL,
Metadata: toDependencyMetadata(p),
Relationship: toDependencyRelationshipType(p),
Scope: toDependencyScope(p),
Dependencies: toDependencies(s, p),
}
}
out := Manifests{}
for k, v := range manifests {
out[k] = *v
}
return out
}
// dependencyName to make things a little nicer to read; this might end up being lossy
func dependencyName(p pkg.Package) string {
purl, err := packageurl.FromString(p.PURL)
if err != nil {
log.Warnf("Invalid PURL for package: '%s' PURL: '%s' (%w)", p.Name, p.PURL, err)
return ""
}
// don't use qualifiers for this
purl.Qualifiers = nil
return purl.ToString()
}
func toDependencyScope(_ pkg.Package) DependencyScope {
return DependencyScopeRuntime
}
func toDependencyRelationshipType(_ pkg.Package) DependencyRelationship {
return DependencyRelationshipDirect
}
func toDependencyMetadata(_ pkg.Package) Metadata {
// We have limited properties: up to 8 with reasonably small values
// For now, we are encoding the location as part of the key, we are encoding PURLs with most
// of the other information Grype might need; and the distro information at the top level
// so we don't need anything here yet
return Metadata{}
}
func toDependencies(s *sbom.SBOM, p pkg.Package) (out []string) {
for _, r := range s.Relationships {
if r.From.ID() == p.ID() {
if p, ok := r.To.(pkg.Package); ok {
out = append(out, dependencyName(p))
}
}
}
return
}
+167
View File
@@ -0,0 +1,167 @@
package github
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
func Test_toGithubModel(t *testing.T) {
s := sbom.SBOM{
Source: source.Metadata{
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "ubuntu:18.04",
Architecture: "amd64",
},
},
Artifacts: sbom.Artifacts{
LinuxDistribution: &linux.Release{
ID: "ubuntu",
VersionID: "18.04",
IDLike: []string{"debian"},
},
PackageCatalog: pkg.NewCatalog(),
},
}
for _, p := range []pkg.Package{
{
Name: "pkg-1",
Version: "1.0.1",
Locations: source.NewLocationSet(
source.Location{
Coordinates: source.Coordinates{
RealPath: "/usr/lib",
FileSystemID: "fsid-1",
},
},
),
},
{
Name: "pkg-2",
Version: "2.0.2",
Locations: source.NewLocationSet(
source.Location{
Coordinates: source.Coordinates{
RealPath: "/usr/lib",
FileSystemID: "fsid-1",
},
},
),
},
{
Name: "pkg-3",
Version: "3.0.3",
Locations: source.NewLocationSet(
source.Location{
Coordinates: source.Coordinates{
RealPath: "/etc",
FileSystemID: "fsid-1",
},
},
),
},
} {
p.PURL = packageurl.NewPackageURL(
"generic",
"",
p.Name,
p.Version,
nil,
"",
).ToString()
s.Artifacts.PackageCatalog.Add(p)
}
actual := toGithubModel(&s)
expected := DependencySnapshot{
Version: 0,
Detector: DetectorMetadata{
Name: "syft",
Version: "0.0.0-dev",
URL: "https://github.com/anchore/syft",
},
Metadata: Metadata{
"syft:distro": "pkg:generic/ubuntu@18.04?like=debian",
},
Scanned: actual.Scanned,
Manifests: Manifests{
"ubuntu:18.04:/usr/lib": Manifest{
Name: "ubuntu:18.04:/usr/lib",
File: FileInfo{
SourceLocation: "ubuntu:18.04:/usr/lib",
},
Metadata: Metadata{
"syft:filesystem": "fsid-1",
},
Resolved: DependencyGraph{
"pkg:generic/pkg-1@1.0.1": DependencyNode{
PackageURL: "pkg:generic/pkg-1@1.0.1",
Scope: DependencyScopeRuntime,
Relationship: DependencyRelationshipDirect,
},
"pkg:generic/pkg-2@2.0.2": DependencyNode{
PackageURL: "pkg:generic/pkg-2@2.0.2",
Scope: DependencyScopeRuntime,
Relationship: DependencyRelationshipDirect,
},
},
},
"ubuntu:18.04:/etc": Manifest{
Name: "ubuntu:18.04:/etc",
File: FileInfo{
SourceLocation: "ubuntu:18.04:/etc",
},
Metadata: Metadata{
"syft:filesystem": "fsid-1",
},
Resolved: DependencyGraph{
"pkg:generic/pkg-3@3.0.3": DependencyNode{
PackageURL: "pkg:generic/pkg-3@3.0.3",
Scope: DependencyScopeRuntime,
Relationship: DependencyRelationshipDirect,
},
},
},
},
}
// just using JSONEq because it gives a comprehensible diff
s1, _ := json.Marshal(expected)
s2, _ := json.Marshal(actual)
assert.JSONEq(t, string(s1), string(s2))
// Just test the other schemes:
s.Source.Path = "."
s.Source.Scheme = source.DirectoryScheme
actual = toGithubModel(&s)
assert.Equal(t, "etc", actual.Manifests["etc"].Name)
s.Source.Path = "./artifacts"
s.Source.Scheme = source.DirectoryScheme
actual = toGithubModel(&s)
assert.Equal(t, "artifacts/etc", actual.Manifests["artifacts/etc"].Name)
s.Source.Path = "/artifacts"
s.Source.Scheme = source.DirectoryScheme
actual = toGithubModel(&s)
assert.Equal(t, "/artifacts/etc", actual.Manifests["/artifacts/etc"].Name)
s.Source.Path = "./executable"
s.Source.Scheme = source.FileScheme
actual = toGithubModel(&s)
assert.Equal(t, "executable", actual.Manifests["executable"].Name)
s.Source.Path = "./archive.tar.gz"
s.Source.Scheme = source.FileScheme
actual = toGithubModel(&s)
assert.Equal(t, "archive.tar.gz:/etc", actual.Manifests["archive.tar.gz:/etc"].Name)
}
+29
View File
@@ -0,0 +1,29 @@
package github
import (
"encoding/json"
"io"
"github.com/anchore/syft/syft/sbom"
)
const ID sbom.FormatID = "github-0-json"
func Format() sbom.Format {
return sbom.NewFormat(
ID,
func(writer io.Writer, sbom sbom.SBOM) error {
bom := toGithubModel(&sbom)
bytes, err := json.MarshalIndent(bom, "", " ")
if err != nil {
return err
}
_, err = writer.Write(bytes)
return err
},
nil,
nil,
)
}
@@ -0,0 +1,78 @@
package github
// Derived from: https://gist.github.com/reiddraper/fdab2883db0f372c146d1a750fc1c43f
type DependencySnapshot struct {
Version int `json:"version"`
Job Job `json:"job,omitempty"` // !omitempty
Sha string `json:"sha,omitempty"` // !omitempty sha of the Git commit
Ref string `json:"ref,omitempty"` // !omitempty ref of the Git commit example "refs/heads/main"
Detector DetectorMetadata `json:"detector,omitempty"`
Metadata Metadata `json:"metadata,omitempty"`
Manifests Manifests `json:"manifests,omitempty"`
Scanned ISO8601Date `json:"scanned,omitempty"`
}
type Job struct {
Correlator string `json:"correlator,omitempty"` // !omitempty
ID string `json:"id,omitempty"` // !omitempty
HTMLURL string `json:"html_url,omitempty"`
}
type DetectorMetadata struct {
Name string `json:"name,omitempty"`
URL string `json:"url,omitempty"`
Version string `json:"version,omitempty"`
}
type Manifests map[string]Manifest
// Manifest A collection of related dependencies, either declared in a file,
// or representing a logical group of dependencies.
type Manifest struct {
Name string `json:"name"`
File FileInfo `json:"file"`
Metadata Metadata `json:"metadata,omitempty"`
Resolved DependencyGraph `json:"resolved,omitempty"`
}
type FileInfo struct {
SourceLocation string `json:"source_location,omitempty"`
}
// DependencyRelationship A notation of whether a dependency is requested directly
// by this manifest, or is a dependency of another dependency.
type DependencyRelationship string
const (
DependencyRelationshipDirect DependencyRelationship = "direct"
DependencyRelationshipIndirect DependencyRelationship = "indirect"
)
// DependencyScope A notation of whether the dependency is required for the primary
// build artifact (runtime), or is only used for development.
// Future versions of this specification may allow for more granular
// scopes, like `runtimeserver`, `runtimeshipped`,
// `developmenttest`, `developmentbenchmark`.
type DependencyScope string
const (
DependencyScopeRuntime DependencyScope = "runtime"
DependencyScopeDevelopment DependencyScope = "development"
)
type DependencyNode struct {
PackageURL string `json:"package_url,omitempty"`
Metadata Metadata `json:"metadata,omitempty"`
Relationship DependencyRelationship `json:"relationship,omitempty"`
Scope DependencyScope `json:"scope,omitempty"`
Dependencies []string `json:"dependencies,omitempty"`
}
type DependencyGraph map[string]DependencyNode
type ISO8601Date = string
type Scalar interface{} // should be: null | boolean | string | number
type Metadata map[string]Scalar
@@ -1,5 +1,7 @@
package model
import "github.com/anchore/syft/internal/formats/common/spdxhelpers"
// ElementID represents the identifier string portion of an SPDX element
// identifier. DocElementID should be used for any attributes which can
// contain identifiers defined in a different SPDX document.
@@ -7,31 +9,5 @@ package model
type ElementID string
func (e ElementID) String() string {
return "SPDXRef-" + string(e)
}
// DocElementID represents an SPDX element identifier that could be defined
// in a different SPDX document, and therefore could have a "DocumentRef-"
// portion, such as Relationship and Annotations.
// ElementID is used for attributes in which a "DocumentRef-" portion cannot
// appear, such as a Package or File definition (since it is necessarily
// being defined in the present document).
// DocumentRefID will be the empty string for elements defined in the
// present document.
// DocElementIDs should NOT contain the mandatory 'DocumentRef-' or
// 'SPDXRef-' portions.
type DocElementID struct {
DocumentRefID string
ElementRefID ElementID
}
// RenderDocElementID takes a DocElementID and returns the string equivalent,
// with the SPDXRef- prefix (and, if applicable, the DocumentRef- prefix)
// reinserted.
func (d DocElementID) String() string {
prefix := ""
if d.DocumentRefID != "" {
prefix = "DocumentRef-" + d.DocumentRefID + ":"
}
return prefix + d.ElementRefID.String()
return "SPDXRef-" + spdxhelpers.SanitizeElementID(string(e))
}
@@ -3,18 +3,18 @@
"name": "/some/path",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"created": "2021-12-20T19:12:47.869816Z",
"created": "2022-04-01T15:48:39.459232Z",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-[not provided]"
],
"licenseListVersion": "3.15"
"licenseListVersion": "3.16"
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https://anchore.com/syft/dir/some/path-4b896ded-7852-4e31-b764-136b53bdf346",
"documentNamespace": "https://anchore.com/syft/dir/some/path-8d335d81-29c9-4236-84f1-2292ea92aaf5",
"packages": [
{
"SPDXID": "SPDXRef-1d97af55efe9512f",
"SPDXID": "SPDXRef-b85dbb4e6ece5082",
"name": "package-1",
"licenseConcluded": "MIT",
"downloadLocation": "NOASSERTION",
@@ -36,7 +36,7 @@
"versionInfo": "1.0.1"
},
{
"SPDXID": "SPDXRef-ad3d1c4abd84bf75",
"SPDXID": "SPDXRef-ceda99598967ae8d",
"name": "package-2",
"licenseConcluded": "NONE",
"downloadLocation": "NOASSERTION",
@@ -48,7 +48,7 @@
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceLocator": "a-purl-2",
"referenceLocator": "pkg:deb/debian/package-2@2.0.1",
"referenceType": "purl"
}
],
@@ -3,18 +3,18 @@
"name": "user-image-input",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"created": "2021-12-20T19:13:07.647486Z",
"created": "2022-04-01T15:48:39.465643Z",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-[not provided]"
],
"licenseListVersion": "3.15"
"licenseListVersion": "3.16"
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https://anchore.com/syft/image/user-image-input-174da656-1824-4bd3-8604-28919f8a65bc",
"documentNamespace": "https://anchore.com/syft/image/user-image-input-e64e0be8-5031-4eec-842d-e59fb6deb518",
"packages": [
{
"SPDXID": "SPDXRef-d16127444133b5c1",
"SPDXID": "SPDXRef-2a46171f91c8d4bc",
"name": "package-1",
"licenseConcluded": "MIT",
"downloadLocation": "NOASSERTION",
@@ -36,7 +36,7 @@
"versionInfo": "1.0.1"
},
{
"SPDXID": "SPDXRef-24907357f3705420",
"SPDXID": "SPDXRef-ae77680e9b1d087e",
"name": "package-2",
"licenseConcluded": "NONE",
"downloadLocation": "NOASSERTION",
@@ -48,7 +48,7 @@
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceLocator": "a-purl-2",
"referenceLocator": "pkg:deb/debian/package-2@2.0.1",
"referenceType": "purl"
}
],
+19 -1
View File
@@ -55,14 +55,32 @@ func toPackages(catalog *pkg.Catalog, relationships []artifact.Relationship) []m
for _, p := range catalog.Sorted() {
license := spdxhelpers.License(p)
packageSpdxID := model.ElementID(p.ID()).String()
filesAnalyzed := false
// we generate digest for some Java packages
// see page 33 of the spdx specification for 2.2
// spdx.github.io/spdx-spec/package-information/#710-package-checksum-field
var checksums []model.Checksum
if p.MetadataType == pkg.JavaMetadataType {
javaMetadata := p.Metadata.(pkg.JavaMetadata)
if len(javaMetadata.ArchiveDigests) > 0 {
filesAnalyzed = true
for _, digest := range javaMetadata.ArchiveDigests {
checksums = append(checksums, model.Checksum{
Algorithm: strings.ToUpper(digest.Algorithm),
ChecksumValue: digest.Value,
})
}
}
}
// note: the license concluded and declared should be the same since we are collecting license information
// from the project data itself (the installed package files).
packages = append(packages, model.Package{
Checksums: checksums,
Description: spdxhelpers.Description(p),
DownloadLocation: spdxhelpers.DownloadLocation(p),
ExternalRefs: spdxhelpers.ExternalRefs(p),
FilesAnalyzed: false,
FilesAnalyzed: filesAnalyzed,
HasFiles: fileIDsForPackage(packageSpdxID, relationships),
Homepage: spdxhelpers.Homepage(p),
// The Declared License is what the authors of a project believe govern the package
@@ -6,6 +6,9 @@ import (
"testing"
"github.com/anchore/syft/internal/formats/common/testutils"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
var updateSpdxTagValue = flag.Bool("update-spdx-tv", false, "update the *.golden files for spdx-tv encoders")
@@ -31,6 +34,32 @@ func TestSPDXTagValueImageEncoder(t *testing.T) {
)
}
func TestSPDXJSONSPDXIDs(t *testing.T) {
var pkgs []pkg.Package
for _, name := range []string{"some/slashes", "@at-sign", "under_scores"} {
p := pkg.Package{
Name: name,
}
p.SetID()
pkgs = append(pkgs, p)
}
testutils.AssertEncoderAgainstGoldenSnapshot(t,
Format(),
sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: pkg.NewCatalog(pkgs...),
},
Relationships: nil,
Source: source.Metadata{
Scheme: source.DirectoryScheme,
},
Descriptor: sbom.Descriptor{},
},
true,
spdxTagValueRedactor,
)
}
func spdxTagValueRedactor(s []byte) []byte {
// each SBOM reports the time it was generated, which is not useful during snapshot testing
s = regexp.MustCompile(`Created: .*`).ReplaceAll(s, []byte("redacted"))
@@ -0,0 +1,40 @@
SPDXVersion: SPDX-2.2
DataLicense: CC0-1.0
SPDXID: SPDXRef-DOCUMENT
DocumentName: .
DocumentNamespace: https://anchore.com/syft/dir/644e30b7-5a31-4d0b-a903-b96c757921c2
LicenseListVersion: 3.16
Creator: Organization: Anchore, Inc
Creator: Tool: syft-[not provided]
Created: 2022-04-19T15:10:19Z
##### Package: @at-sign
PackageName: @at-sign
SPDXID: SPDXRef-Package---at-sign-739e4f0d93fb8298
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageLicenseConcluded: NONE
PackageLicenseDeclared: NONE
PackageCopyrightText: NOASSERTION
##### Package: some/slashes
PackageName: some/slashes
SPDXID: SPDXRef-Package--some-slashes-26db06648b24bff9
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageLicenseConcluded: NONE
PackageLicenseDeclared: NONE
PackageCopyrightText: NOASSERTION
##### Package: under_scores
PackageName: under_scores
SPDXID: SPDXRef-Package--under-scores-250cbfefcdea318b
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageLicenseConcluded: NONE
PackageLicenseDeclared: NONE
PackageCopyrightText: NOASSERTION
@@ -2,16 +2,16 @@ SPDXVersion: SPDX-2.2
DataLicense: CC0-1.0
SPDXID: SPDXRef-DOCUMENT
DocumentName: /some/path
DocumentNamespace: https://anchore.com/syft/dir/some/path-4b90f56d-d596-4ad8-b6a5-17f7d801350d
DocumentNamespace: https://anchore.com/syft/dir/some/path-d227b0f2-4ee8-4e10-ac43-019db86d16ff
LicenseListVersion: 3.16
Creator: Organization: Anchore, Inc
Creator: Tool: syft-[not provided]
Created: 2022-02-10T21:09:27Z
Created: 2022-04-01T15:48:44Z
##### Package: package-2
PackageName: package-2
SPDXID: SPDXRef-Package-deb-package-2-ad3d1c4abd84bf75
SPDXID: SPDXRef-Package-deb-package-2-ceda99598967ae8d
PackageVersion: 2.0.1
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
@@ -19,12 +19,12 @@ PackageLicenseConcluded: NONE
PackageLicenseDeclared: NONE
PackageCopyrightText: NOASSERTION
ExternalRef: SECURITY cpe23Type cpe:2.3:*:some:package:2:*:*:*:*:*:*:*
ExternalRef: PACKAGE_MANAGER purl a-purl-2
ExternalRef: PACKAGE_MANAGER purl pkg:deb/debian/package-2@2.0.1
##### Package: package-1
PackageName: package-1
SPDXID: SPDXRef-Package-python-package-1-1d97af55efe9512f
SPDXID: SPDXRef-Package-python-package-1-b85dbb4e6ece5082
PackageVersion: 1.0.1
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
@@ -2,16 +2,16 @@ SPDXVersion: SPDX-2.2
DataLicense: CC0-1.0
SPDXID: SPDXRef-DOCUMENT
DocumentName: user-image-input
DocumentNamespace: https://anchore.com/syft/image/user-image-input-26a2def6-53d0-4504-b99a-a046832508ac
DocumentNamespace: https://anchore.com/syft/image/user-image-input-49f98c61-3418-4427-9e00-8b1c735e9799
LicenseListVersion: 3.16
Creator: Organization: Anchore, Inc
Creator: Tool: syft-[not provided]
Created: 2022-02-10T21:09:27Z
Created: 2022-04-01T15:48:44Z
##### Package: package-2
PackageName: package-2
SPDXID: SPDXRef-Package-deb-package-2-73f796c846875b9e
SPDXID: SPDXRef-Package-deb-package-2-ae77680e9b1d087e
PackageVersion: 2.0.1
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
@@ -19,12 +19,12 @@ PackageLicenseConcluded: NONE
PackageLicenseDeclared: NONE
PackageCopyrightText: NOASSERTION
ExternalRef: SECURITY cpe23Type cpe:2.3:*:some:package:2:*:*:*:*:*:*:*
ExternalRef: PACKAGE_MANAGER purl a-purl-2
ExternalRef: PACKAGE_MANAGER purl pkg:deb/debian/package-2@2.0.1
##### Package: package-1
PackageName: package-1
SPDXID: SPDXRef-Package-python-package-1-d9527e708c11f8b9
SPDXID: SPDXRef-Package-python-package-1-2a46171f91c8d4bc
PackageVersion: 1.0.1
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
@@ -96,13 +96,31 @@ func toFormatPackages(catalog *pkg.Catalog) map[spdx.ElementID]*spdx.Package2_2
for _, p := range catalog.Sorted() {
// name should be guaranteed to be unique, but semantically useful and stable
id := fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.ID())
id := spdxhelpers.SanitizeElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.ID()))
// If the Concluded License is not the same as the Declared License, a written explanation should be provided
// in the Comments on License field (section 3.16). With respect to NOASSERTION, a written explanation in
// the Comments on License field (section 3.16) is preferred.
license := spdxhelpers.License(p)
filesAnalyzed := false
checksums := make(map[spdx.ChecksumAlgorithm]spdx.Checksum)
// If the pkg type is Java we have attempted to generated a digest
// FilesAnalyzed should be true in this case
if p.MetadataType == pkg.JavaMetadataType {
javaMetadata := p.Metadata.(pkg.JavaMetadata)
if len(javaMetadata.ArchiveDigests) > 0 {
filesAnalyzed = true
for _, digest := range javaMetadata.ArchiveDigests {
checksums[spdx.ChecksumAlgorithm(digest.Algorithm)] = spdx.Checksum{
Algorithm: spdx.ChecksumAlgorithm(digest.Algorithm),
Value: digest.Value,
}
}
}
}
results[spdx.ElementID(id)] = &spdx.Package2_2{
// NOT PART OF SPEC
@@ -159,7 +177,7 @@ func toFormatPackages(catalog *pkg.Catalog) map[spdx.ElementID]*spdx.Package2_2
// Intent: A package can refer to a project, product, artifact, distribution or a component that is
// external to the SPDX document.
FilesAnalyzed: false,
FilesAnalyzed: filesAnalyzed,
// NOT PART OF SPEC: did FilesAnalyzed tag appear?
IsFilesAnalyzedTagPresent: true,
@@ -180,6 +198,7 @@ func toFormatPackages(catalog *pkg.Catalog) map[spdx.ElementID]*spdx.Package2_2
// to determine if any file in the original package has been changed. If the SPDX file is to be included
// in a package, this value should not be calculated. The SHA-1 algorithm will be used to provide the
// checksum by default.
PackageChecksums: checksums,
// note: based on the purpose above no discovered checksums should be provided, but instead, only
// tool-derived checksums.
+6 -6
View File
@@ -42,13 +42,13 @@ func TestEncodeFullJSONDocument(t *testing.T) {
p1 := pkg.Package{
Name: "package-1",
Version: "1.0.1",
Locations: []source.Location{
{
Locations: source.NewLocationSet(
source.Location{
Coordinates: source.Coordinates{
RealPath: "/a/place/a",
},
},
},
),
Type: pkg.PythonPkg,
FoundBy: "the-cataloger-1",
Language: pkg.Python,
@@ -68,13 +68,13 @@ func TestEncodeFullJSONDocument(t *testing.T) {
p2 := pkg.Package{
Name: "package-2",
Version: "2.0.1",
Locations: []source.Location{
{
Locations: source.NewLocationSet(
source.Location{
Coordinates: source.Coordinates{
RealPath: "/b/place/b",
},
},
},
),
Type: pkg.DebPkg,
FoundBy: "the-cataloger-2",
MetadataType: pkg.DpkgMetadataType,
@@ -124,6 +124,18 @@ func (p *Package) UnmarshalJSON(b []byte) error {
return err
}
p.Metadata = payload
case pkg.GolangBinMetadataType:
var payload pkg.GolangBinMetadata
if err := json.Unmarshal(unpacker.Metadata, &payload); err != nil {
return err
}
p.Metadata = payload
case pkg.DartPubMetadataType:
var payload pkg.DartPubMetadata
if err := json.Unmarshal(unpacker.Metadata, &payload); err != nil {
return err
}
p.Metadata = payload
default:
log.Warnf("unknown package metadata type=%q for packageID=%q", p.MetadataType, p.ID)
}
@@ -0,0 +1,57 @@
package model
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func TestUnmarshalPackageGolang(t *testing.T) {
tests := []struct {
name string
p *Package
packageData []byte
}{
{
name: "Package.UnmarshalJSON unmarshals PackageBasicData",
p: &Package{},
packageData: []byte(`{
"id": "8b594519bc23da50",
"name": "gopkg.in/square/go-jose.v2",
"version": "v2.6.0",
"type": "go-module",
"foundBy": "go-module-binary-cataloger",
"locations": [
{
"path": "/Users/hal/go/bin/syft"
}
],
"licenses": [],
"language": "go",
"cpes": [],
"purl": "pkg:golang/gopkg.in/square/go-jose.v2@v2.6.0",
"metadataType": "GolangBinMetadata",
"metadata": {
"goCompiledVersion": "go1.18",
"architecture": "amd64",
"h1Digest": "h1:NGk74WTnPKBNUhNzQX7PYcTLUjoq7mzKk2OKbvwk2iI="
}
}`),
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
err := test.p.UnmarshalJSON(test.packageData)
if err != nil {
t.Fatalf("could not unmarshal packageData: %v", err)
}
assert.NotNil(t, test.p.Metadata)
golangMetadata := test.p.Metadata.(pkg.GolangBinMetadata)
assert.NotEmpty(t, golangMetadata)
assert.Equal(t, "go1.18", golangMetadata.GoCompiledVersion)
})
}
}
@@ -1,7 +1,7 @@
{
"artifacts": [
{
"id": "1d97af55efe9512f",
"id": "b85dbb4e6ece5082",
"name": "package-1",
"version": "1.0.1",
"type": "python",
@@ -36,7 +36,7 @@
}
},
{
"id": "ad3d1c4abd84bf75",
"id": "ceda99598967ae8d",
"name": "package-2",
"version": "2.0.1",
"type": "deb",
@@ -51,7 +51,7 @@
"cpes": [
"cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"
],
"purl": "a-purl-2",
"purl": "pkg:deb/debian/package-2@2.0.1",
"metadataType": "DpkgMetadata",
"metadata": {
"package": "package-2",
@@ -88,7 +88,7 @@
}
},
"schema": {
"version": "3.0.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.0.1.json"
"version": "3.2.2",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.2.2.json"
}
}
@@ -1,7 +1,7 @@
{
"artifacts": [
{
"id": "d9a7c58726ab4bef",
"id": "b3fa3ee64756b0c6",
"name": "package-1",
"version": "1.0.1",
"type": "python",
@@ -31,7 +31,7 @@
}
},
{
"id": "ac462e450060da2c",
"id": "b324f4d9ee5413fe",
"name": "package-2",
"version": "2.0.1",
"type": "deb",
@@ -184,7 +184,7 @@
}
},
"schema": {
"version": "3.0.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.0.1.json"
"version": "3.2.2",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.2.2.json"
}
}
@@ -1,7 +1,7 @@
{
"artifacts": [
{
"id": "d9527e708c11f8b9",
"id": "2a46171f91c8d4bc",
"name": "package-1",
"version": "1.0.1",
"type": "python",
@@ -32,7 +32,7 @@
}
},
{
"id": "73f796c846875b9e",
"id": "ae77680e9b1d087e",
"name": "package-2",
"version": "2.0.1",
"type": "deb",
@@ -48,7 +48,7 @@
"cpes": [
"cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"
],
"purl": "a-purl-2",
"purl": "pkg:deb/debian/package-2@2.0.1",
"metadataType": "DpkgMetadata",
"metadata": {
"package": "package-2",
@@ -111,7 +111,7 @@
}
},
"schema": {
"version": "3.0.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.0.1.json"
"version": "3.2.2",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.2.2.json"
}
}
+3 -2
View File
@@ -175,8 +175,9 @@ func toPackageModel(p pkg.Package) model.Package {
licenses = p.Licenses
}
var coordinates = make([]source.Coordinates, len(p.Locations))
for i, l := range p.Locations {
locations := p.Locations.ToSlice()
var coordinates = make([]source.Coordinates, len(locations))
for i, l := range locations {
coordinates[i] = l.Coordinates
}
+36 -13
View File
@@ -12,7 +12,9 @@ import (
)
func toSyftModel(doc model.Document) (*sbom.SBOM, error) {
catalog := toSyftCatalog(doc.Artifacts)
idAliases := make(map[string]string)
catalog := toSyftCatalog(doc.Artifacts, idAliases)
return &sbom.SBOM{
Artifacts: sbom.Artifacts{
@@ -21,7 +23,7 @@ func toSyftModel(doc model.Document) (*sbom.SBOM, error) {
},
Source: *toSyftSourceData(doc.Source),
Descriptor: toSyftDescriptor(doc.Descriptor),
Relationships: toSyftRelationships(&doc, catalog, doc.ArtifactRelationships),
Relationships: toSyftRelationships(&doc, catalog, doc.ArtifactRelationships, idAliases),
}, nil
}
@@ -46,12 +48,13 @@ func toSyftLinuxRelease(d model.LinuxRelease) *linux.Release {
}
}
func toSyftRelationships(doc *model.Document, catalog *pkg.Catalog, relationships []model.Relationship) []artifact.Relationship {
func toSyftRelationships(doc *model.Document, catalog *pkg.Catalog, relationships []model.Relationship, idAliases map[string]string) []artifact.Relationship {
idMap := make(map[string]interface{})
for _, p := range catalog.Sorted() {
idMap[string(p.ID())] = p
for _, l := range p.Locations {
locations := p.Locations.ToSlice()
for _, l := range locations {
idMap[string(l.Coordinates.ID())] = l.Coordinates
}
}
@@ -62,7 +65,7 @@ func toSyftRelationships(doc *model.Document, catalog *pkg.Catalog, relationship
var out []artifact.Relationship
for _, r := range relationships {
syftRelationship := toSyftRelationship(idMap, r)
syftRelationship := toSyftRelationship(idMap, r, idAliases)
if syftRelationship != nil {
out = append(out, *syftRelationship)
}
@@ -70,13 +73,20 @@ func toSyftRelationships(doc *model.Document, catalog *pkg.Catalog, relationship
return out
}
func toSyftRelationship(idMap map[string]interface{}, relationship model.Relationship) *artifact.Relationship {
from, ok := idMap[relationship.Parent].(artifact.Identifiable)
func toSyftRelationship(idMap map[string]interface{}, relationship model.Relationship, idAliases map[string]string) *artifact.Relationship {
id := func(id string) string {
aliased, ok := idAliases[id]
if ok {
return aliased
}
return id
}
from, ok := idMap[id(relationship.Parent)].(artifact.Identifiable)
if !ok {
log.Warnf("relationship mapping from key %s is not a valid artifact.Identifiable type: %+v", relationship.Parent, idMap[relationship.Parent])
return nil
}
to, ok := idMap[relationship.Child].(artifact.Identifiable)
to, ok := idMap[id(relationship.Child)].(artifact.Identifiable)
if !ok {
log.Warnf("relationship mapping to key %s is not a valid artifact.Identifiable type: %+v", relationship.Child, idMap[relationship.Child])
return nil
@@ -128,15 +138,15 @@ func toSyftSourceData(s model.Source) *source.Metadata {
return nil
}
func toSyftCatalog(pkgs []model.Package) *pkg.Catalog {
func toSyftCatalog(pkgs []model.Package, idAliases map[string]string) *pkg.Catalog {
catalog := pkg.NewCatalog()
for _, p := range pkgs {
catalog.Add(toSyftPackage(p))
catalog.Add(toSyftPackage(p, idAliases))
}
return catalog
}
func toSyftPackage(p model.Package) pkg.Package {
func toSyftPackage(p model.Package, idAliases map[string]string) pkg.Package {
var cpes []pkg.CPE
for _, c := range p.CPEs {
value, err := pkg.NewCPE(c)
@@ -153,11 +163,11 @@ func toSyftPackage(p model.Package) pkg.Package {
locations[i] = source.NewLocationFromCoordinates(c)
}
return pkg.Package{
out := pkg.Package{
Name: p.Name,
Version: p.Version,
FoundBy: p.FoundBy,
Locations: locations,
Locations: source.NewLocationSet(locations...),
Licenses: p.Licenses,
Language: p.Language,
Type: p.Type,
@@ -166,4 +176,17 @@ func toSyftPackage(p model.Package) pkg.Package {
MetadataType: p.MetadataType,
Metadata: p.Metadata,
}
// we don't know if this package ID is truly unique, however, we need to trust the user input in case there are
// external references to it. That is, we can't derive our own ID (using pkg.SetID()) since consumers won't
// be able to historically interact with data that references the IDs from the original SBOM document being decoded now.
out.OverrideID(artifact.ID(p.ID))
// this alias mapping is currently defunct, but could be useful in the future.
id := string(out.ID())
if id != p.ID {
idAliases[p.ID] = id
}
return out
}
@@ -4,6 +4,7 @@ import (
"testing"
"github.com/anchore/syft/internal/formats/syftjson/model"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/source"
"github.com/scylladb/go-set/strset"
"github.com/stretchr/testify/assert"
@@ -79,3 +80,46 @@ func Test_toSyftSourceData(t *testing.T) {
// assert all possible schemes were under test
assert.ElementsMatch(t, allSchemes.List(), testedSchemes.List(), "not all source.Schemes are under test")
}
func Test_idsHaveChanged(t *testing.T) {
s, err := toSyftModel(model.Document{
Source: model.Source{
Type: "file",
Target: "some/path",
},
Artifacts: []model.Package{
{
PackageBasicData: model.PackageBasicData{
ID: "1",
Name: "pkg-1",
},
},
{
PackageBasicData: model.PackageBasicData{
ID: "2",
Name: "pkg-2",
},
},
},
ArtifactRelationships: []model.Relationship{
{
Parent: "1",
Child: "2",
Type: string(artifact.ContainsRelationship),
},
},
})
assert.NoError(t, err)
assert.Len(t, s.Relationships, 1)
r := s.Relationships[0]
from := s.Artifacts.PackageCatalog.Package(r.From.ID())
assert.NotNil(t, from)
assert.Equal(t, "pkg-1", from.Name)
to := s.Artifacts.PackageCatalog.Package(r.To.ID())
assert.NotNil(t, to)
assert.Equal(t, "pkg-2", to.Name)
}
-14
View File
@@ -1,14 +0,0 @@
/*
Syft is a CLI tool and go library for generating a Software Bill of Materials (SBOM) from container images and filesystems.
Note that Syft is both a command line tool as well as a library. See the syft/ child package for library functionality.
*/
package main
import (
"github.com/anchore/syft/cmd"
)
func main() {
cmd.Execute()
}
+2 -2
View File
@@ -1,7 +1,7 @@
.DEFAULT_GOAL := validate-schema
.PHONY: validate-schema
validate-schema:
go run ../../main.go ubuntu:latest -vv -o cyclonedx > bom.xml
go run ../../cmd/syft/main.go ubuntu:latest -vv -o cyclonedx > bom.xml
xmllint --noout --schema ./cyclonedx.xsd bom.xml
go run ../../main.go ubuntu:latest -vv -o cyclonedx-json > bom.json
go run ../../cmd/syft/main.go ubuntu:latest -vv -o cyclonedx-json > bom.json
../../.tmp/yajsv -s cyclonedx.json bom.json
+1
View File
@@ -37,6 +37,7 @@ type artifactMetadataContainer struct {
Cargo pkg.CargoPackageMetadata
Go pkg.GolangBinMetadata
Php pkg.PhpComposerJSONMetadata
Dart pkg.DartPubMetadata
}
func main() {
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+16 -7
View File
@@ -77,30 +77,39 @@ func (i *DigestsCataloger) catalogLocation(resolver source.FileResolver, locatio
}
defer internal.CloseAndLogError(contentReader, location.VirtualPath)
digests, err := DigestsFromFile(contentReader, i.hashes)
if err != nil {
return nil, internal.ErrPath{Context: "digests-cataloger", Path: location.RealPath, Err: err}
}
return digests, nil
}
func DigestsFromFile(closer io.ReadCloser, hashes []crypto.Hash) ([]Digest, error) {
// create a set of hasher objects tied together with a single writer to feed content into
hashers := make([]hash.Hash, len(i.hashes))
writers := make([]io.Writer, len(i.hashes))
for idx, hashObj := range i.hashes {
hashers := make([]hash.Hash, len(hashes))
writers := make([]io.Writer, len(hashes))
for idx, hashObj := range hashes {
hashers[idx] = hashObj.New()
writers[idx] = hashers[idx]
}
size, err := io.Copy(io.MultiWriter(writers...), contentReader)
size, err := io.Copy(io.MultiWriter(writers...), closer)
if err != nil {
return nil, internal.ErrPath{Context: "digests-cataloger", Path: location.RealPath, Err: err}
return nil, err
}
if size == 0 {
return make([]Digest, 0), nil
}
result := make([]Digest, len(i.hashes))
result := make([]Digest, len(hashes))
// only capture digests when there is content. It is important to do this based on SIZE and not
// FILE TYPE. The reasoning is that it is possible for a tar to be crafted with a header-only
// file type but a body is still allowed.
for idx, hasher := range hashers {
result[idx] = Digest{
Algorithm: DigestAlgorithmName(i.hashes[idx]),
Algorithm: DigestAlgorithmName(hashes[idx]),
Value: fmt.Sprintf("%+x", hasher.Sum(nil)),
}
}

Some files were not shown because too many files have changed in this diff Show More