Compare commits

...
20 Commits
Author SHA1 Message Date
Christopher Angelo Phillips 45ea4177e8 update SPDX license list => 3.15 (#630)
* update SPDX license list 

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-11-15 14:41:46 -05:00
Alex Goodman e28b7b53b0 bump chronicle to v0.3.0 (#627)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-13 18:56:25 -08:00
Christopher Angelo Phillips fc39710d06 287 - Add retry for image fetch using user's input (#626)
* add fallback to user input if source hint fails

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* refactor for smaller functions

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-11-12 17:57:04 -05:00
Swathi Gangisetty e732f419f8 Add new distro and mapping to support Rocky Linux distro identification (#624)
Signed-off-by: Swathi Gangisetty <swathi@anchore.com>
2021-11-12 13:10:21 -08:00
NikitaandAlex Goodman 25835ef589 PHP parser composer.lock (#609)
* PHP parse composer.lock

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* rename PHP package type

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add PHP composer package support to source info SPDX helper

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update directory cataloger integration tests (make exception for images)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-10 14:08:41 -05:00
Christopher Angelo Phillips 319c6ee2eb document current draft for iana submission (#618)
* document current draft for IANA submission

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-11-10 11:28:47 -05:00
Alex Goodman 9c27fa7b0b use sbom.SBOM in ImportConfig (#621)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-08 18:39:06 -05:00
Christopher Angelo PhillipsandAlex Goodman 9bbc9ff633 Add layer information for go-module-binary-cataloger (#620)
* update functions to pass Location

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update unit tests to pass new locations

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* fix image source.FileResolvers to include layer info

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add non-empty location in golang binary cataloger testing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-08 16:05:53 -05:00
Alex GoodmanandChristopher Angelo Phillips bb0f35bac4 Introduce a single SBOM document (#606)
* [wip] single sbom doc

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix more tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update cli tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove scope in import path

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* swap SPDX tag-value formatter to single sbom document

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bust CLI cache

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update fixture to byte diff

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* byte for byte

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* bust the cache

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* who needs cache

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* add jar for testing

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* no more bit flips

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update apk with the delta for image and directory cases

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* restore cache workflow

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-11-05 10:05:49 -04:00
Jonas Galvão Xavier 6d0ee326d8 ignore irregular files while indexing directory (#616)
* stop hidding  command from help doc

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* do not index irregular files

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix fixture dir and err name

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* improve the description of irregular files

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explicitly check indexed file name

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2021-11-04 11:24:05 -07:00
Alex Goodman 23a67dd729 document file scheme (#613)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 19:47:25 -04:00
Christopher Angelo Phillips 3e20edee55 update readme with private registry section (#610) 2021-11-02 15:00:56 -04:00
Alex Goodman 8ec3f1d102 Use anchore fork of go-presenter (#611)
* use anchore fork of go-presenter

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* drop coverage threshold

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 18:09:26 +00:00
Alex Goodman ecab44bba1 Add SPDX tag-value format object (#605)
* add new spdx tag-value format

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove public presenter package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 13:45:34 -04:00
Alex GoodmanandChristopher Angelo Phillips 484284706c Use named pipe bit when checking for piped input (#603)
* use named pipe bit when checking for piped input

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-11-02 16:41:02 +00:00
Alex Goodman 3ac95ac4f6 Allow for cataloging a single file (#608)
* allow for cataloging a single file

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use all catalogers for file schemes

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 12:09:06 -04:00
Christopher Angelo Phillips a2882ee810 Update package identifier to be constant across multiple syft run (#595)
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-29 16:00:36 +00:00
Alex Goodman 358b3a2cf8 port text presenter to a format object (#604)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-29 11:31:15 -04:00
Alex Goodman 9aca23f766 Add SPDX JSON format object (#584)
* remove existing spdxjson presenter + helpers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add new spdx22json format

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add common sdpxhelpers (migrated)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use new common spdx helpers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* wire up new spdx22json format object

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove lossless syft-specific property bags

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove spdxjson decoder and validator

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add nil checks in spdx test helpers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove empty default case

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use explicit golden snapshot

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-29 14:55:20 +00:00
Alex Goodman 4a2d1d7225 Port cyclonedx presenter to format object (#589)
* add new cyclonedx format object

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove cyclonedx presenter

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove cyclonedx presenter call

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove dependence on golden images for format tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* wire up new formt + rename all-presenters ref

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add CLI test to ensure that all formats can be expressed as report output

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add cyclonedx version and encoding format to package name

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* optionally preserve format snapshot images

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting + text unit tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-29 10:17:03 -04:00
196 changed files with 3428 additions and 1913 deletions
+19 -22
View File
@@ -15,10 +15,10 @@ RESET := $(shell tput -T linux sgr0)
TITLE := $(BOLD)$(PURPLE)
SUCCESS := $(BOLD)$(GREEN)
# the quality gate lower threshold for unit test total % coverage (by function statements)
COVERAGE_THRESHOLD := 65
COVERAGE_THRESHOLD := 62
# CI cache busting values; change these if you want CI to not use previous stored cache
INTEGRATION_CACHE_BUSTER="88738d2f"
CLI_CACHE_BUSTER="789bacdf"
CLI_CACHE_BUSTER="9a2c03cf"
BOOTSTRAP_CACHE="c7afb99ad"
## Build variables
@@ -37,10 +37,6 @@ ifeq "$(strip $(VERSION))" ""
override VERSION = $(shell git describe --always --tags --dirty)
endif
# used to generate the changelog from the second to last tag to the current tag (used in the release pipeline when the release tag is in place)
LAST_TAG := $(shell git describe --abbrev=0 --tags $(shell git rev-list --tags --max-count=1))
SECOND_TO_LAST_TAG := $(shell git describe --abbrev=0 --tags $(shell git rev-list --tags --skip=1 --max-count=1))
## Variable assertions
ifndef TEMPDIR
@@ -103,7 +99,7 @@ bootstrap-tools: $(TEMPDIR)
GO111MODULE=off GOBIN=$(shell realpath $(TEMPDIR)) go get -u golang.org/x/perf/cmd/benchstat
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ v1.42.1
curl -sSfL https://raw.githubusercontent.com/wagoodman/go-bouncer/master/bouncer.sh | sh -s -- -b $(TEMPDIR)/ v0.2.0
curl -sSfL https://raw.githubusercontent.com/anchore/chronicle/main/install.sh | sh -s -- -b $(TEMPDIR)/ v0.2.0-beta
curl -sSfL https://raw.githubusercontent.com/anchore/chronicle/main/install.sh | sh -s -- -b $(TEMPDIR)/ v0.3.0
.github/scripts/goreleaser-install.sh -b $(TEMPDIR)/ v0.177.0
.PHONY: bootstrap-go
@@ -194,6 +190,11 @@ fixtures:
generate-json-schema: ## Generate a new json schema
cd schema/json && go run generate.go
.PHONY: generate-license-list
generate-license-list: ## Generate an updated spdx license list
go generate ./internal/spdxlicense/...
gofmt -s -w ./internal/spdxlicense
.PHONY: build
build: $(SNAPSHOTDIR) ## Build release snapshot binaries and packages
@@ -252,27 +253,19 @@ cli: $(SNAPSHOTDIR) ## Run CLI tests
SYFT_BINARY_LOCATION='$(SNAPSHOT_CMD)' \
go test -count=1 -v ./test/cli
.PHONY: changelog-release
changelog-release:
$(TEMPDIR)/chronicle --since-tag $(SECOND_TO_LAST_TAG) --until-tag $(LAST_TAG) -vv > CHANGELOG.md
@printf '\n$(BOLD)$(CYAN)Release $(VERSION) Changelog$(RESET)\n\n'
@cat CHANGELOG.md
.PHONY: changelog-unreleased
changelog-unreleased: ## show the current changelog that will be produced on the next release (note: requires GITHUB_TOKEN set)
$(TEMPDIR)/chronicle --since-tag $(LAST_TAG) -vv > CHANGELOG.md
@printf '\n$(BOLD)$(CYAN)Unreleased Changes (closed PRs and issues will not be in the final changelog)$(RESET)\n'
.PHONY: changelog
changelog: clean-changelog CHANGELOG.md
@docker run -it --rm \
-v $(shell pwd)/CHANGELOG.md:/CHANGELOG.md \
rawkode/mdv \
-t 748.5989 \
/CHANGELOG.md
CHANGELOG.md:
$(TEMPDIR)/chronicle -vv > CHANGELOG.md
.PHONY: release
release: clean-dist changelog-release ## Build and publish final binaries and packages. Intended to be run only on macOS.
release: clean-dist CHANGELOG.md ## Build and publish final binaries and packages. Intended to be run only on macOS.
$(call title,Publishing release artifacts)
# Prepare for macOS-specific signing process
@@ -313,9 +306,13 @@ clean-snapshot:
rm -rf $(SNAPSHOTDIR) $(TEMPDIR)/goreleaser.yaml
.PHONY: clean-dist
clean-dist:
clean-dist: clean-changelog
rm -rf $(DISTDIR) $(TEMPDIR)/goreleaser.yaml
.PHONY: clean-changelog
clean-changelog:
rm -f CHANGELOG.md
clean-test-image-cache: clean-test-image-tar-cache clean-test-image-docker-cache
.PHONY: clear-test-image-tar-cache
+85 -2
View File
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://user-images.githubusercontent.com/5199289/136844524-1527b09f-c5cb-4aa9-be54-5aa92a6086c1.png" width="271">
<img src="https://user-images.githubusercontent.com/5199289/136844524-1527b09f-c5cb-4aa9-be54-5aa92a6086c1.png" width="271" alt="Cute pink owl syft logo">
</p>
[![Validations](https://github.com/anchore/syft/actions/workflows/validations.yaml/badge.svg)](https://github.com/anchore/syft/actions/workflows/validations.yaml)
@@ -83,6 +83,7 @@ docker-archive:path/to/yourimage.tar use a tarball from disk for archives crea
oci-archive:path/to/yourimage.tar use a tarball from disk for OCI archives (from Skopeo or otherwise)
oci-dir:path/to/yourimage read directly from a path on disk for OCI layout directories (from Skopeo or otherwise)
dir:path/to/yourproject read directly from a path on disk (any directory)
file:path/to/yourproject/file read directly from a path on disk (any single file)
registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
```
@@ -94,7 +95,7 @@ The output format for Syft is configurable as well:
syft packages <image> -o <format>
```
Where the `format`s available are:
Where the `formats` available are:
- `json`: Use this to get as much information out of Syft as possible!
- `text`: A row-oriented, human-and-machine-friendly output.
- `cyclonedx`: A XML report conforming to the [CycloneDX 1.2 specification](https://cyclonedx.org/specification/overview/).
@@ -102,6 +103,88 @@ Where the `format`s available are:
- `spdx-json`: A JSON report conforming to the [SPDX 2.2 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.2/schemas/spdx-schema.json).
- `table`: A columnar summary (default).
## Private Registry Authentication
### Local Docker Credentials
When a container runtime is not present, Syft can still utilize credentials configured in common credential sources (such as `~/.docker/config.json`).
It will pull images from private registries using these credentials. The config file is where your credentials are stored when authenticating with private registries via some command like `docker login`.
For more information see the `go-containerregistry` [documentation](https://github.com/google/go-containerregistry/tree/main/pkg/authn).
An example `config.json` looks something like this:
```
// config.json
{
"auths": {
"registry.example.com": {
"username": "AzureDiamond",
"password": "hunter2"
}
}
}
```
You can run the following command as an example. It details the mount/environment configuration a container needs to access a private registry:
`docker run -v ./config.json:/config/config.json -e "DOCKER_CONFIG=/config" anchore/syft:latest <private_image>`
### Docker Credentials in Kubernetes
The below section shows a simple workflow on how to mount this config file as a secret into a container on kubernetes.
1. Create a secret. The value of `config.json` is important. It refers to the specification detailed [here](https://github.com/google/go-containerregistry/tree/main/pkg/authn#the-config-file).
Below this section is the `secret.yaml` file that the pod configuration will consume as a volume.
The key `config.json` is important. It will end up being the name of the file when mounted into the pod.
```
# secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: registry-config
namespace: syft
data:
config.json: <base64 encoded config.json>
```
`kubectl apply -f secret.yaml`
2. Create your pod running syft. The env `DOCKER_CONFIG` is important because it advertises where to look for the credential file.
In the below example, setting `DOCKER_CONFIG=/config` informs syft that credentials can be found at `/config/config.json`.
This is why we used `config.json` as the key for our secret. When mounted into containers the secrets' key is used as the filename.
The `volumeMounts` section mounts our secret to `/config`. The `volumes` section names our volume and leverages the secret we created in step one.
```
# pod.yaml
apiVersion: v1
kind: Pod
spec:
containers:
- image: anchore/syft:latest
name: syft-private-registry-demo
env:
- name: DOCKER_CONFIG
value: /config
volumeMounts:
- mountPath: /config
name: registry-config
readOnly: true
args:
- <private_image>
volumes:
- name: registry-config
secret:
secretName: registry-config
```
`kubectl apply -f pod.yaml`
3. The user can now run `kubectl logs syft-private-registry-demo`. The logs should show the syft analysis for the `<private_image>` provided in the pod configuration.
Using the above information, users should be able to configure private registry access without having to do so in the `grype` or `syft` configuration files.
They will also not be dependent on a docker daemon, (or some other runtime software) for registry configuration and access.
## Configuration
Configuration search paths:
+24 -19
View File
@@ -6,19 +6,17 @@ import (
"io/ioutil"
"os"
"github.com/anchore/syft/syft/format"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/anchore"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/formats"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/presenter/packages"
"github.com/anchore/syft/syft/format"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/pkg/profile"
"github.com/spf13/cobra"
@@ -45,6 +43,7 @@ const (
{{.appName}} {{.command}} oci-archive:path/to/yourimage.tar use a tarball from disk for OCI archives (from Skopeo or otherwise)
{{.appName}} {{.command}} oci-dir:path/to/yourimage read directly from a path on disk for OCI layout directories (from Skopeo or otherwise)
{{.appName}} {{.command}} dir:path/to/yourproject read directly from a path on disk (any directory)
{{.appName}} {{.command}} file:path/to/yourproject/file read directly from a path on disk (any single file)
{{.appName}} {{.command}} registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
`
)
@@ -103,7 +102,7 @@ func setPackageFlags(flags *pflag.FlagSet) {
flags.StringP(
"output", "o", string(format.TableOption),
fmt.Sprintf("report output formatter, options=%v", format.AllPresenters),
fmt.Sprintf("report output formatter, options=%v", format.AllOptions),
)
flags.StringP(
@@ -239,6 +238,12 @@ func packagesExecWorker(userInput string) <-chan error {
go func() {
defer close(errs)
f := formats.ByOption(packagesPresenterOpt)
if f == nil {
errs <- fmt.Errorf("unknown format: %s", packagesPresenterOpt)
return
}
checkForApplicationUpdate()
src, cleanup, err := source.New(userInput, appConfig.Registry.ToOptions())
@@ -254,27 +259,30 @@ func packagesExecWorker(userInput string) <-chan error {
return
}
sbomResult := sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: catalog,
Distro: d,
},
Source: src.Metadata,
}
if appConfig.Anchore.Host != "" {
if err := runPackageSbomUpload(src, src.Metadata, catalog, d, appConfig.Package.Cataloger.ScopeOpt); err != nil {
if err := runPackageSbomUpload(src, sbomResult); err != nil {
errs <- err
return
}
}
bus.Publish(partybus.Event{
Type: event.PresenterReady,
Value: packages.Presenter(packagesPresenterOpt, packages.PresenterConfig{
SourceMetadata: src.Metadata,
Catalog: catalog,
Distro: d,
Scope: appConfig.Package.Cataloger.ScopeOpt,
}),
Type: event.PresenterReady,
Value: f.Presenter(sbomResult),
})
}()
return errs
}
func runPackageSbomUpload(src *source.Source, s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, scope source.Scope) error {
func runPackageSbomUpload(src *source.Source, s sbom.SBOM) error {
log.Infof("uploading results to %s", appConfig.Anchore.Host)
if src.Metadata.Scheme != source.ImageScheme {
@@ -309,12 +317,9 @@ func runPackageSbomUpload(src *source.Source, s source.Metadata, catalog *pkg.Ca
importCfg := anchore.ImportConfig{
ImageMetadata: src.Image.Metadata,
SourceMetadata: s,
Catalog: catalog,
Distro: d,
SBOM: s,
Dockerfile: dockerfileContents,
OverwriteExistingUpload: appConfig.Anchore.OverwriteExistingImage,
Scope: scope,
Timeout: appConfig.Anchore.ImportTimeout,
}
+6 -5
View File
@@ -4,6 +4,8 @@ import (
"fmt"
"sync"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
@@ -107,9 +109,8 @@ func powerUserExecWorker(userInput string) <-chan error {
}
defer cleanup()
analysisResults := poweruser.JSONDocumentConfig{
SourceMetadata: src.Metadata,
ApplicationConfig: *appConfig,
analysisResults := sbom.SBOM{
Source: src.Metadata,
}
wg := &sync.WaitGroup{}
@@ -117,7 +118,7 @@ func powerUserExecWorker(userInput string) <-chan error {
wg.Add(1)
go func(task powerUserTask) {
defer wg.Done()
if err = task(&analysisResults, src); err != nil {
if err = task(&analysisResults.Artifacts, src); err != nil {
errs <- err
return
}
@@ -128,7 +129,7 @@ func powerUserExecWorker(userInput string) <-chan error {
bus.Publish(partybus.Event{
Type: event.PresenterReady,
Value: poweruser.NewJSONPresenter(analysisResults),
Value: poweruser.NewJSONPresenter(analysisResults, *appConfig),
})
}()
return errs
+9 -8
View File
@@ -4,13 +4,14 @@ import (
"crypto"
"fmt"
"github.com/anchore/syft/internal/presenter/poweruser"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/source"
)
type powerUserTask func(*poweruser.JSONDocumentConfig, *source.Source) error
type powerUserTask func(*sbom.Artifacts, *source.Source) error
func powerUserTasks() ([]powerUserTask, error) {
var tasks []powerUserTask
@@ -42,7 +43,7 @@ func catalogPackagesTask() (powerUserTask, error) {
return nil, nil
}
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) error {
packageCatalog, theDistro, err := syft.CatalogPackages(src, appConfig.Package.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -64,7 +65,7 @@ func catalogFileMetadataTask() (powerUserTask, error) {
metadataCataloger := file.NewMetadataCataloger()
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -110,7 +111,7 @@ func catalogFileDigestsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -142,7 +143,7 @@ func catalogSecretsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.Secrets.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -170,7 +171,7 @@ func catalogFileClassificationsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileClassification.Cataloger.ScopeOpt)
if err != nil {
return err
@@ -197,7 +198,7 @@ func catalogContentsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) error {
resolver, err := src.FileResolver(appConfig.FileContents.Cataloger.ScopeOpt)
if err != nil {
return err
+2
View File
@@ -7,6 +7,7 @@ require (
github.com/adrg/xdg v0.2.1
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf
github.com/anchore/go-presenter v0.0.0-20211102174526-0dbf20f6c7fa
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
@@ -24,6 +25,7 @@ require (
github.com/hashicorp/go-multierror v1.1.0
github.com/hashicorp/go-version v1.2.0
github.com/mitchellh/go-homedir v1.1.0
github.com/mitchellh/hashstructure v1.1.0
github.com/mitchellh/mapstructure v1.3.1
github.com/olekukonko/tablewriter v0.0.4
github.com/pelletier/go-toml v1.8.1
+4
View File
@@ -108,6 +108,8 @@ github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRF
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf/go.mod h1:FaODhIA06mxO1E6R32JE0TL1JWZZkmjRIAd4ULvHUKk=
github.com/anchore/go-presenter v0.0.0-20211102174526-0dbf20f6c7fa h1:mDLUAkgXsV5Z8D0EEj8eS6FBekolV/A+Xxbs9054bPw=
github.com/anchore/go-presenter v0.0.0-20211102174526-0dbf20f6c7fa/go.mod h1:29jwxTSAS6pBcrmuwf1U3r1Tqp1o1XpuiOJ0NT9NoGg=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63 h1:C9W/LAydEz/qdUhx1MdjO9l8NEcFKYknkxDVyo9LAoM=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63/go.mod h1:6qH8c6U/3CBVvDDDBZnPSTbTINq3cIdADUYTaVf75EM=
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04 h1:VzprUTpc0vW0nnNKJfJieyH/TZ9UYAnTZs5/gHTdAe8=
@@ -542,6 +544,8 @@ github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrk
github.com/mitchellh/go-ps v0.0.0-20190716172923-621e5597135b/go.mod h1:r1VsdOzOPt1ZSrGZWFoNhsAedKnEd6r9Np1+5blZCWk=
github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=
github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=
github.com/mitchellh/hashstructure v1.1.0 h1:P6P1hdjqAAknpY/M1CGipelZgp+4y9ja9kmUZPXP+H0=
github.com/mitchellh/hashstructure v1.1.0/go.mod h1:xUDAozZz0Wmdiufv0uyhnHkUTN6/6d8ulp4AwfLKrmA=
github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=
github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
+6 -12
View File
@@ -6,27 +6,21 @@ import (
"fmt"
"time"
"github.com/antihax/optional"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/syft/sbom"
"github.com/antihax/optional"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
)
type ImportConfig struct {
ImageMetadata image.Metadata
SourceMetadata source.Metadata
Catalog *pkg.Catalog
Distro *distro.Distro
SBOM sbom.SBOM
Dockerfile []byte
OverwriteExistingUpload bool
Scope source.Scope
Timeout uint
}
@@ -74,19 +68,19 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
prog.N++
sessionID := startOperation.Uuid
packageDigest, err := importPackageSBOM(authedCtx, c.client.ImportsApi, sessionID, cfg.SourceMetadata, cfg.Catalog, cfg.Distro, cfg.Scope, stage)
packageDigest, err := importPackageSBOM(authedCtx, c.client.ImportsApi, sessionID, cfg.SBOM, stage)
if err != nil {
return fmt.Errorf("failed to import Package SBOM: %w", err)
}
prog.N++
manifestDigest, err := importManifest(authedCtx, c.client.ImportsApi, sessionID, cfg.ImageMetadata.RawManifest, stage)
manifestDigest, err := importManifest(authedCtx, c.client.ImportsApi, sessionID, cfg.SBOM.Source.ImageMetadata.RawManifest, stage)
if err != nil {
return fmt.Errorf("failed to import Manifest: %w", err)
}
prog.N++
configDigest, err := importConfig(authedCtx, c.client.ImportsApi, sessionID, cfg.ImageMetadata.RawConfig, stage)
configDigest, err := importConfig(authedCtx, c.client.ImportsApi, sessionID, cfg.SBOM.Source.ImageMetadata.RawConfig, stage)
if err != nil {
return fmt.Errorf("failed to import Config: %w", err)
}
+6 -8
View File
@@ -8,26 +8,24 @@ import (
"fmt"
"net/http"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/wagoodman/go-progress"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/source"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/pkg"
)
type packageSBOMImportAPI interface {
ImportImagePackages(context.Context, string, external.ImagePackageManifest) (external.ImageImportContentResponse, *http.Response, error)
}
func packageSbomModel(s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, scope source.Scope) (*external.ImagePackageManifest, error) {
func packageSbomModel(s sbom.SBOM) (*external.ImagePackageManifest, error) {
var buf bytes.Buffer
err := syftjson.Format().Presenter(catalog, &s, d, scope).Present(&buf)
err := syftjson.Format().Presenter(s).Present(&buf)
if err != nil {
return nil, fmt.Errorf("unable to serialize results: %w", err)
}
@@ -41,11 +39,11 @@ func packageSbomModel(s source.Metadata, catalog *pkg.Catalog, d *distro.Distro,
return &model, nil
}
func importPackageSBOM(ctx context.Context, api packageSBOMImportAPI, sessionID string, s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, scope source.Scope, stage *progress.Stage) (string, error) {
func importPackageSBOM(ctx context.Context, api packageSBOMImportAPI, sessionID string, s sbom.SBOM, stage *progress.Stage) (string, error) {
log.Debug("importing package SBOM")
stage.Current = "package SBOM"
model, err := packageSbomModel(s, catalog, d, scope)
model, err := packageSbomModel(s)
if err != nil {
return "", fmt.Errorf("unable to create PackageSBOM model: %w", err)
}
+30 -4
View File
@@ -9,6 +9,8 @@ import (
"strings"
"testing"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/formats/syftjson"
syftjsonModel "github.com/anchore/syft/internal/formats/syftjson/model"
@@ -72,7 +74,15 @@ func TestPackageSbomToModel(t *testing.T) {
c := pkg.NewCatalog(p)
model, err := packageSbomModel(m, c, &d, source.AllLayersScope)
sbomResult := sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: c,
Distro: &d,
},
Source: m,
}
model, err := packageSbomModel(sbomResult)
if err != nil {
t.Fatalf("unable to generate model from source material: %+v", err)
}
@@ -84,8 +94,16 @@ func TestPackageSbomToModel(t *testing.T) {
t.Fatalf("unable to marshal model: %+v", err)
}
s := sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: c,
Distro: &d,
},
Source: m,
}
var buf bytes.Buffer
pres := syftjson.Format().Presenter(c, &m, &d, source.AllLayersScope)
pres := syftjson.Format().Presenter(s)
if err := pres.Present(&buf); err != nil {
t.Fatalf("unable to get expected json: %+v", err)
}
@@ -187,7 +205,15 @@ func TestPackageSbomImport(t *testing.T) {
d, _ := distro.NewDistro(distro.CentOS, "8.0", "")
theModel, err := packageSbomModel(m, catalog, &d, source.AllLayersScope)
sbomResult := sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: catalog,
Distro: &d,
},
Source: m,
}
theModel, err := packageSbomModel(sbomResult)
if err != nil {
t.Fatalf("could not get sbom model: %+v", err)
}
@@ -226,7 +252,7 @@ func TestPackageSbomImport(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
digest, err := importPackageSBOM(context.TODO(), test.api, sessionID, m, catalog, &d, source.AllLayersScope, &progress.Stage{})
digest, err := importPackageSBOM(context.TODO(), test.api, sessionID, sbomResult, &progress.Stage{})
// validate error handling
if err != nil && !test.expectsError {
@@ -0,0 +1,23 @@
package spdxhelpers
import "github.com/anchore/syft/syft/pkg"
func Description(p *pkg.Package) string {
if hasMetadata(p) {
switch metadata := p.Metadata.(type) {
case pkg.ApkMetadata:
return metadata.Description
case pkg.NpmPackageJSONMetadata:
return metadata.Description
}
}
return ""
}
func packageExists(p *pkg.Package) bool {
return p != nil
}
func hasMetadata(p *pkg.Package) bool {
return packageExists(p) && p.Metadata != nil
}
@@ -0,0 +1,56 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_Description(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected string
}{
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "no metadata",
input: pkg.Package{},
expected: "",
},
{
name: "from apk",
input: pkg.Package{
Metadata: pkg.ApkMetadata{
Description: "a description!",
},
},
expected: "a description!",
},
{
name: "from npm",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Description: "a description!",
},
},
expected: "a description!",
},
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "empty",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Homepage: "",
},
},
expected: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, Description(&test.input))
})
}
}
@@ -0,0 +1,23 @@
package spdxhelpers
import "github.com/anchore/syft/syft/pkg"
func DownloadLocation(p *pkg.Package) string {
// 3.7: Package Download Location
// Cardinality: mandatory, one
// NONE if there is no download location whatsoever.
// NOASSERTION if:
// (i) the SPDX file creator has attempted to but cannot reach a reasonable objective determination;
// (ii) the SPDX file creator has made no attempt to determine this field; or
// (iii) the SPDX file creator has intentionally provided no information (no meaning should be implied by doing so).
if hasMetadata(p) {
switch metadata := p.Metadata.(type) {
case pkg.ApkMetadata:
return NoneIfEmpty(metadata.URL)
case pkg.NpmPackageJSONMetadata:
return NoneIfEmpty(metadata.URL)
}
}
return "NOASSERTION"
}
@@ -0,0 +1,54 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_DownloadLocation(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected string
}{
{
name: "no metadata",
input: pkg.Package{},
expected: "NOASSERTION",
},
{
name: "from apk",
input: pkg.Package{
Metadata: pkg.ApkMetadata{
URL: "http://a-place.gov",
},
},
expected: "http://a-place.gov",
},
{
name: "from npm",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
URL: "http://a-place.gov",
},
},
expected: "http://a-place.gov",
},
{
name: "empty",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
URL: "",
},
},
expected: "NONE",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, DownloadLocation(&test.input))
})
}
}
@@ -0,0 +1,55 @@
package spdxhelpers
import (
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/pkg"
)
func ExternalRefs(p *pkg.Package) (externalRefs []model.ExternalRef) {
externalRefs = make([]model.ExternalRef, 0)
if !packageExists(p) {
return externalRefs
}
for _, c := range p.CPEs {
externalRefs = append(externalRefs, model.ExternalRef{
ReferenceCategory: model.SecurityReferenceCategory,
ReferenceLocator: c.BindToFmtString(),
ReferenceType: model.Cpe23ExternalRefType,
})
}
if p.PURL != "" {
externalRefs = append(externalRefs, model.ExternalRef{
ReferenceCategory: model.PackageManagerReferenceCategory,
ReferenceLocator: p.PURL,
ReferenceType: model.PurlExternalRefType,
})
}
return externalRefs
}
func ExtractPURL(refs []model.ExternalRef) string {
for _, r := range refs {
if r.ReferenceType == model.PurlExternalRefType {
return r.ReferenceLocator
}
}
return ""
}
func ExtractCPEs(refs []model.ExternalRef) (cpes []pkg.CPE) {
for _, r := range refs {
if r.ReferenceType == model.Cpe23ExternalRefType {
cpe, err := pkg.NewCPE(r.ReferenceLocator)
if err != nil {
log.Warnf("unable to extract SPDX CPE=%q: %+v", r.ReferenceLocator, err)
continue
}
cpes = append(cpes, cpe)
}
}
return cpes
}
@@ -0,0 +1,45 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_ExternalRefs(t *testing.T) {
testCPE := pkg.MustCPE("cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*")
tests := []struct {
name string
input pkg.Package
expected []model.ExternalRef
}{
{
name: "cpe + purl",
input: pkg.Package{
CPEs: []pkg.CPE{
testCPE,
},
PURL: "a-purl",
},
expected: []model.ExternalRef{
{
ReferenceCategory: model.SecurityReferenceCategory,
ReferenceLocator: testCPE.BindToFmtString(),
ReferenceType: model.Cpe23ExternalRefType,
},
{
ReferenceCategory: model.PackageManagerReferenceCategory,
ReferenceLocator: "a-purl",
ReferenceType: model.PurlExternalRefType,
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.ElementsMatch(t, test.expected, ExternalRefs(&test.input))
})
}
}
@@ -0,0 +1,51 @@
package spdxhelpers
import (
"crypto/sha256"
"fmt"
"path/filepath"
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/syft/pkg"
)
func Files(packageSpdxID string, p *pkg.Package) (files []model.File, fileIDs []string, relationships []model.Relationship) {
files = make([]model.File, 0)
fileIDs = make([]string, 0)
relationships = make([]model.Relationship, 0)
if !hasMetadata(p) {
return files, fileIDs, relationships
}
pkgFileOwner, ok := p.Metadata.(pkg.FileOwner)
if !ok {
return files, fileIDs, relationships
}
for _, ownedFilePath := range pkgFileOwner.OwnedFiles() {
baseFileName := filepath.Base(ownedFilePath)
pathHash := sha256.Sum256([]byte(ownedFilePath))
fileSpdxID := model.ElementID(fmt.Sprintf("File-%s-%x", p.Name, pathHash)).String()
fileIDs = append(fileIDs, fileSpdxID)
files = append(files, model.File{
FileName: ownedFilePath,
Item: model.Item{
Element: model.Element{
SPDXID: fileSpdxID,
Name: baseFileName,
},
},
})
relationships = append(relationships, model.Relationship{
SpdxElementID: packageSpdxID,
RelationshipType: model.ContainsRelationship,
RelatedSpdxElement: fileSpdxID,
})
}
return files, fileIDs, relationships
}
@@ -0,0 +1,15 @@
package spdxhelpers
import "github.com/anchore/syft/syft/pkg"
func Homepage(p *pkg.Package) string {
if hasMetadata(p) {
switch metadata := p.Metadata.(type) {
case pkg.GemMetadata:
return metadata.Homepage
case pkg.NpmPackageJSONMetadata:
return metadata.Homepage
}
}
return ""
}
@@ -0,0 +1,56 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_Homepage(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected string
}{
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "no metadata",
input: pkg.Package{},
expected: "",
},
{
name: "from gem",
input: pkg.Package{
Metadata: pkg.GemMetadata{
Homepage: "http://a-place.gov",
},
},
expected: "http://a-place.gov",
},
{
name: "from npm",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Homepage: "http://a-place.gov",
},
},
expected: "http://a-place.gov",
},
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "empty",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Homepage: "",
},
},
expected: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, Homepage(&test.input))
})
}
}
@@ -0,0 +1,37 @@
package spdxhelpers
import (
"strings"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/pkg"
)
func License(p *pkg.Package) string {
// source: https://spdx.github.io/spdx-spec/3-package-information/#313-concluded-license
// The options to populate this field are limited to:
// A valid SPDX License Expression as defined in Appendix IV;
// NONE, if the SPDX file creator concludes there is no license available for this package; or
// NOASSERTION if:
// (i) the SPDX file creator has attempted to but cannot reach a reasonable objective determination;
// (ii) the SPDX file creator has made no attempt to determine this field; or
// (iii) the SPDX file creator has intentionally provided no information (no meaning should be implied by doing so).
if !packageExists(p) || len(p.Licenses) == 0 {
return "NONE"
}
// take all licenses and assume an AND expression; for information about license expressions see https://spdx.github.io/spdx-spec/appendix-IV-SPDX-license-expressions/
var parsedLicenses []string
for _, l := range p.Licenses {
if value, exists := spdxlicense.ID(l); exists {
parsedLicenses = append(parsedLicenses, value)
}
}
if len(parsedLicenses) == 0 {
return "NOASSERTION"
}
return strings.Join(parsedLicenses, " AND ")
}
@@ -0,0 +1,73 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_License(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected string
}{
{
name: "no licenses",
input: pkg.Package{},
expected: "NONE",
},
{
name: "no SPDX licenses",
input: pkg.Package{
Licenses: []string{
"made-up",
},
},
expected: "NOASSERTION",
},
{
name: "with SPDX license",
input: pkg.Package{
Licenses: []string{
"MIT",
},
},
expected: "MIT",
},
{
name: "with SPDX license expression",
input: pkg.Package{
Licenses: []string{
"MIT",
"GPL-3.0",
},
},
expected: "MIT AND GPL-3.0",
},
{
name: "cap insensitive",
input: pkg.Package{
Licenses: []string{
"gpl-3.0",
},
},
expected: "GPL-3.0",
},
{
name: "debian to spdx conversion",
input: pkg.Package{
Licenses: []string{
"GPL-2",
},
},
expected: "GPL-2.0",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, License(&test.input))
})
}
}
@@ -0,0 +1,12 @@
package spdxhelpers
import (
"strings"
)
func NoneIfEmpty(value string) string {
if strings.TrimSpace(value) == "" {
return "NONE"
}
return value
}
@@ -0,0 +1,41 @@
package spdxhelpers
import (
"testing"
"github.com/stretchr/testify/assert"
)
func Test_noneIfEmpty(t *testing.T) {
tests := []struct {
name string
value string
expected string
}{
{
name: "non-zero value",
value: "something",
expected: "something",
},
{
name: "empty",
value: "",
expected: "NONE",
},
{
name: "space",
value: " ",
expected: "NONE",
},
{
name: "tab",
value: "\t",
expected: "NONE",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, NoneIfEmpty(test.value))
})
}
}
@@ -0,0 +1,114 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_Originator(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected string
}{
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "no metadata",
input: pkg.Package{},
expected: "",
},
{
name: "from gem",
input: pkg.Package{
Metadata: pkg.GemMetadata{
Authors: []string{
"auth1",
"auth2",
},
},
},
expected: "auth1",
},
{
name: "from npm",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Author: "auth",
},
},
expected: "auth",
},
{
name: "from apk",
input: pkg.Package{
Metadata: pkg.ApkMetadata{
Maintainer: "auth",
},
},
expected: "auth",
},
{
name: "from python - just name",
input: pkg.Package{
Metadata: pkg.PythonPackageMetadata{
Author: "auth",
},
},
expected: "auth",
},
{
name: "from python - just email",
input: pkg.Package{
Metadata: pkg.PythonPackageMetadata{
AuthorEmail: "auth@auth.gov",
},
},
expected: "auth@auth.gov",
},
{
name: "from python - both name and email",
input: pkg.Package{
Metadata: pkg.PythonPackageMetadata{
Author: "auth",
AuthorEmail: "auth@auth.gov",
},
},
expected: "auth <auth@auth.gov>",
},
{
name: "from rpm",
input: pkg.Package{
Metadata: pkg.RpmdbMetadata{
Vendor: "auth",
},
},
expected: "auth",
},
{
name: "from dpkg",
input: pkg.Package{
Metadata: pkg.DpkgMetadata{
Maintainer: "auth",
},
},
expected: "auth",
},
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "empty",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Author: "",
},
},
expected: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, Originator(&test.input))
})
}
}
@@ -0,0 +1,37 @@
package spdxhelpers
import (
"fmt"
"github.com/anchore/syft/syft/pkg"
)
func Originator(p *pkg.Package) string {
if hasMetadata(p) {
switch metadata := p.Metadata.(type) {
case pkg.ApkMetadata:
return metadata.Maintainer
case pkg.NpmPackageJSONMetadata:
return metadata.Author
case pkg.PythonPackageMetadata:
author := metadata.Author
if author == "" {
return metadata.AuthorEmail
}
if metadata.AuthorEmail != "" {
author += fmt.Sprintf(" <%s>", metadata.AuthorEmail)
}
return author
case pkg.GemMetadata:
if len(metadata.Authors) > 0 {
return metadata.Authors[0]
}
return ""
case pkg.RpmdbMetadata:
return metadata.Vendor
case pkg.DpkgMetadata:
return metadata.Maintainer
}
}
return ""
}
@@ -0,0 +1,45 @@
package spdxhelpers
import (
"strings"
"github.com/anchore/syft/syft/pkg"
)
func SourceInfo(p *pkg.Package) string {
if !packageExists(p) {
return ""
}
answer := ""
switch p.Type {
case pkg.RpmPkg:
answer = "acquired package info from RPM DB"
case pkg.ApkPkg:
answer = "acquired package info from APK DB"
case pkg.DebPkg:
answer = "acquired package info from DPKG DB"
case pkg.NpmPkg:
answer = "acquired package info from installed node module manifest file"
case pkg.PythonPkg:
answer = "acquired package info from installed python package manifest file"
case pkg.JavaPkg, pkg.JenkinsPluginPkg:
answer = "acquired package info from installed java archive"
case pkg.GemPkg:
answer = "acquired package info from installed gem metadata file"
case pkg.GoModulePkg:
answer = "acquired package info from go module information"
case pkg.RustPkg:
answer = "acquired package info from rust cargo manifest"
case pkg.PhpComposerPkg:
answer = "acquired package info from PHP composer manifest"
default:
answer = "acquired package info from the following paths"
}
var paths []string
for _, l := range p.Locations {
paths = append(paths, l.RealPath)
}
return answer + ": " + strings.Join(paths, ", ")
}
@@ -0,0 +1,149 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/stretchr/testify/assert"
)
func Test_SourceInfo(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected []string
}{
{
name: "locations are captured",
input: pkg.Package{
// note: no type given
Locations: []source.Location{
{
RealPath: "/a-place",
VirtualPath: "/b-place",
},
{
RealPath: "/c-place",
VirtualPath: "/d-place",
},
},
},
expected: []string{
"from the following paths",
"/a-place",
"/c-place",
},
},
{
// note: no specific support for this
input: pkg.Package{
Type: pkg.KbPkg,
},
expected: []string{
"from the following paths",
},
},
{
input: pkg.Package{
Type: pkg.RpmPkg,
},
expected: []string{
"from RPM DB",
},
},
{
input: pkg.Package{
Type: pkg.ApkPkg,
},
expected: []string{
"from APK DB",
},
},
{
input: pkg.Package{
Type: pkg.DebPkg,
},
expected: []string{
"from DPKG DB",
},
},
{
input: pkg.Package{
Type: pkg.NpmPkg,
},
expected: []string{
"from installed node module manifest file",
},
},
{
input: pkg.Package{
Type: pkg.PythonPkg,
},
expected: []string{
"from installed python package manifest file",
},
},
{
input: pkg.Package{
Type: pkg.JavaPkg,
},
expected: []string{
"from installed java archive",
},
},
{
input: pkg.Package{
Type: pkg.JenkinsPluginPkg,
},
expected: []string{
"from installed java archive",
},
},
{
input: pkg.Package{
Type: pkg.GemPkg,
},
expected: []string{
"from installed gem metadata file",
},
},
{
input: pkg.Package{
Type: pkg.GoModulePkg,
},
expected: []string{
"from go module information",
},
},
{
input: pkg.Package{
Type: pkg.RustPkg,
},
expected: []string{
"from rust cargo manifest",
},
},
{
input: pkg.Package{
Type: pkg.PhpComposerPkg,
},
expected: []string{
"from PHP composer manifest",
},
},
}
var pkgTypes []pkg.Type
for _, test := range tests {
t.Run(test.name+" "+string(test.input.Type), func(t *testing.T) {
if test.input.Type != "" {
pkgTypes = append(pkgTypes, test.input.Type)
}
actual := SourceInfo(&test.input)
for _, expected := range test.expected {
assert.Contains(t, actual, expected)
}
})
}
assert.ElementsMatch(t, pkg.AllPkgs, pkgTypes, "missing one or more package types to test against (maybe a package type was added?)")
}
+64 -18
View File
@@ -4,13 +4,14 @@ import (
"bytes"
"testing"
"github.com/anchore/syft/syft/presenter"
"github.com/anchore/go-presenter"
"github.com/anchore/go-testutils"
"github.com/anchore/stereoscope/pkg/filetree"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/stereoscope/pkg/imagetest"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/sergi/go-diff/diffmatchpatch"
"github.com/stretchr/testify/assert"
@@ -18,6 +19,18 @@ import (
type redactor func(s []byte) []byte
type imageCfg struct {
fromSnapshot bool
}
type ImageOption func(cfg *imageCfg)
func FromSnapshot() ImageOption {
return func(cfg *imageCfg) {
cfg.fromSnapshot = true
}
}
func AssertPresenterAgainstGoldenImageSnapshot(t *testing.T, pres presenter.Presenter, testImage string, updateSnapshot bool, redactors ...redactor) {
var buffer bytes.Buffer
@@ -78,11 +91,43 @@ func AssertPresenterAgainstGoldenSnapshot(t *testing.T, pres presenter.Presenter
}
}
func ImageInput(t testing.TB, testImage string) (*pkg.Catalog, source.Metadata, *distro.Distro) {
func ImageInput(t testing.TB, testImage string, options ...ImageOption) sbom.SBOM {
t.Helper()
catalog := pkg.NewCatalog()
img := imagetest.GetGoldenFixtureImage(t, testImage)
var cfg imageCfg
var img *image.Image
for _, opt := range options {
opt(&cfg)
}
switch cfg.fromSnapshot {
case true:
img = imagetest.GetGoldenFixtureImage(t, testImage)
default:
img = imagetest.GetFixtureImage(t, "docker-archive", testImage)
}
populateImageCatalog(catalog, img)
// this is a hard coded value that is not given by the fixture helper and must be provided manually
img.Metadata.ManifestDigest = "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
src, err := source.NewFromImage(img, "user-image-input")
assert.NoError(t, err)
dist, err := distro.NewDistro(distro.Debian, "1.2.3", "like!")
assert.NoError(t, err)
return sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: catalog,
Distro: &dist,
},
Source: src.Metadata,
}
}
func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
_, ref1, _ := img.SquashedTree().File("/somefile-1.txt", filetree.FollowBasenameLinks)
_, ref2, _ := img.SquashedTree().File("/somefile-2.txt", filetree.FollowBasenameLinks)
@@ -127,20 +172,27 @@ func ImageInput(t testing.TB, testImage string) (*pkg.Catalog, source.Metadata,
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
})
}
// this is a hard coded value that is not given by the fixture helper and must be provided manually
img.Metadata.ManifestDigest = "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
src, err := source.NewFromImage(img, "user-image-input")
assert.NoError(t, err)
func DirectoryInput(t testing.TB) sbom.SBOM {
catalog := newDirectoryCatalog()
dist, err := distro.NewDistro(distro.Debian, "1.2.3", "like!")
assert.NoError(t, err)
return catalog, src.Metadata, &dist
src, err := source.NewFromDirectory("/some/path")
assert.NoError(t, err)
return sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: catalog,
Distro: &dist,
},
Source: src.Metadata,
}
}
func DirectoryInput(t testing.TB) (*pkg.Catalog, source.Metadata, *distro.Distro) {
func newDirectoryCatalog() *pkg.Catalog {
catalog := pkg.NewCatalog()
// populate catalog with test data
@@ -190,11 +242,5 @@ func DirectoryInput(t testing.TB) (*pkg.Catalog, source.Metadata, *distro.Distro
},
})
dist, err := distro.NewDistro(distro.Debian, "1.2.3", "like!")
assert.NoError(t, err)
src, err := source.NewFromDirectory("/some/path")
assert.NoError(t, err)
return catalog, src.Metadata, &dist
return catalog
}
@@ -0,0 +1,26 @@
package cyclonedx12xml
import (
"encoding/xml"
"io"
"github.com/anchore/syft/syft/sbom"
)
func encoder(output io.Writer, s sbom.SBOM) error {
enc := xml.NewEncoder(output)
enc.Indent("", " ")
_, err := output.Write([]byte(xml.Header))
if err != nil {
return err
}
err = enc.Encode(toFormatModel(s))
if err != nil {
return err
}
_, err = output.Write([]byte("\n"))
return err
}
@@ -1,4 +1,4 @@
package packages
package cyclonedx12xml
import (
"flag"
@@ -11,9 +11,8 @@ import (
var updateCycloneDx = flag.Bool("update-cyclonedx", false, "update the *.golden files for cyclone-dx presenters")
func TestCycloneDxDirectoryPresenter(t *testing.T) {
catalog, metadata, _ := testutils.DirectoryInput(t)
testutils.AssertPresenterAgainstGoldenSnapshot(t,
NewCycloneDxPresenter(catalog, metadata),
Format().Presenter(testutils.DirectoryInput(t)),
*updateCycloneDx,
cycloneDxRedactor,
)
@@ -21,9 +20,8 @@ func TestCycloneDxDirectoryPresenter(t *testing.T) {
func TestCycloneDxImagePresenter(t *testing.T) {
testImage := "image-simple"
catalog, metadata, _ := testutils.ImageInput(t, testImage)
testutils.AssertPresenterAgainstGoldenImageSnapshot(t,
NewCycloneDxPresenter(catalog, metadata),
Format().Presenter(testutils.ImageInput(t, testImage)),
testImage,
*updateCycloneDx,
cycloneDxRedactor,
+12
View File
@@ -0,0 +1,12 @@
package cyclonedx12xml
import "github.com/anchore/syft/syft/format"
func Format() format.Format {
return format.NewFormat(
format.CycloneDxOption,
encoder,
nil,
nil,
)
}
@@ -0,0 +1,31 @@
package model
import (
"encoding/xml"
)
// Source: https://cyclonedx.org/ext/bom-descriptor/
// BomDescriptor represents all metadata surrounding the BOM report (such as when the BOM was made, with which tool, and the item being cataloged).
type BomDescriptor struct {
XMLName xml.Name `xml:"metadata"`
Timestamp string `xml:"timestamp,omitempty"` // The date and time (timestamp) when the document was created
Tools []BomDescriptorTool `xml:"tools>tool"` // The tool used to create the BOM.
Component *BomDescriptorComponent `xml:"component"` // The component that the BOM describes.
}
// BomDescriptorTool represents the tool that created the BOM report.
type BomDescriptorTool struct {
XMLName xml.Name `xml:"tool"`
Vendor string `xml:"vendor,omitempty"` // The vendor of the tool used to create the BOM.
Name string `xml:"name,omitempty"` // The name of the tool used to create the BOM.
Version string `xml:"version,omitempty"` // The version of the tool used to create the BOM.
// TODO: hashes, author, manufacture, supplier
// TODO: add user-defined fields for the remaining build/version parameters
}
// BomDescriptorComponent represents the software/package being cataloged.
type BomDescriptorComponent struct {
XMLName xml.Name `xml:"component"`
Component
}
@@ -0,0 +1,20 @@
package model
import "encoding/xml"
// Component represents a single element in the CycloneDX BOM
type Component struct {
XMLName xml.Name `xml:"component"`
Type string `xml:"type,attr"` // Required; Describes if the component is a library, framework, application, container, operating system, firmware, hardware device, or file
Supplier string `xml:"supplier,omitempty"` // The organization that supplied the component. The supplier may often be the manufacture, but may also be a distributor or repackager.
Author string `xml:"author,omitempty"` // The person(s) or organization(s) that authored the component
Publisher string `xml:"publisher,omitempty"` // The person(s) or organization(s) that published the component
Group string `xml:"group,omitempty"` // The high-level classification that a project self-describes as. This will often be a shortened, single name of the company or project that produced the component, or the source package or domain name.
Name string `xml:"name"` // Required; The name of the component as defined by the project
Version string `xml:"version"` // Required; The version of the component as defined by the project
Description string `xml:"description,omitempty"` // A description of the component
Licenses *[]License `xml:"licenses>license"` // A node describing zero or more license names, SPDX license IDs or expressions
PackageURL string `xml:"purl,omitempty"` // Specifies the package-url (PackageURL). The purl, if specified, must be valid and conform to the specification defined at: https://github.com/package-url/purl-spec
// TODO: source, hashes, copyright, cpe, purl, swid, modified, pedigree, externalReferences
// TODO: add user-defined parameters for syft-specific values (image layer index, cataloger, location path, etc.)
}
@@ -0,0 +1,17 @@
package model
import (
"encoding/xml"
)
// Source: https://github.com/CycloneDX/specification
// Document represents a CycloneDX BOM Document.
type Document struct {
XMLName xml.Name `xml:"bom"`
XMLNs string `xml:"xmlns,attr"`
Version int `xml:"version,attr"`
SerialNumber string `xml:"serialNumber,attr"`
BomDescriptor *BomDescriptor `xml:"metadata"` // The BOM descriptor extension
Components []Component `xml:"components>component"` // The BOM contents
}
@@ -0,0 +1,10 @@
package model
import "encoding/xml"
// License represents a single software license for a Component
type License struct {
XMLName xml.Name `xml:"license"`
ID string `xml:"id,omitempty"` // A valid SPDX license ID
Name string `xml:"name,omitempty"` // If SPDX does not define the license used, this field may be used to provide the license name
}
@@ -0,0 +1,95 @@
package cyclonedx12xml
import (
"encoding/xml"
"time"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/formats/cyclonedx12xml/model"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/google/uuid"
)
// toFormatModel creates and populates a new in-memory representation of a CycloneDX 1.2 document
func toFormatModel(s sbom.SBOM) model.Document {
versionInfo := version.FromBuild()
doc := model.Document{
XMLNs: "http://cyclonedx.org/schema/bom/1.2",
Version: 1,
SerialNumber: uuid.New().URN(),
BomDescriptor: toBomDescriptor(internal.ApplicationName, versionInfo.Version, s.Source),
}
// attach components
for _, p := range s.Artifacts.PackageCatalog.Sorted() {
doc.Components = append(doc.Components, toComponent(p))
}
return doc
}
func toComponent(p *pkg.Package) model.Component {
return model.Component{
Type: "library", // TODO: this is not accurate
Name: p.Name,
Version: p.Version,
PackageURL: p.PURL,
Licenses: toLicenses(p.Licenses),
}
}
// NewBomDescriptor returns a new BomDescriptor tailored for the current time and "syft" tool details.
func toBomDescriptor(name, version string, srcMetadata source.Metadata) *model.BomDescriptor {
return &model.BomDescriptor{
XMLName: xml.Name{},
Timestamp: time.Now().Format(time.RFC3339),
Tools: []model.BomDescriptorTool{
{
Vendor: "anchore",
Name: name,
Version: version,
},
},
Component: toBomDescriptorComponent(srcMetadata),
}
}
func toBomDescriptorComponent(srcMetadata source.Metadata) *model.BomDescriptorComponent {
switch srcMetadata.Scheme {
case source.ImageScheme:
return &model.BomDescriptorComponent{
Component: model.Component{
Type: "container",
Name: srcMetadata.ImageMetadata.UserInput,
Version: srcMetadata.ImageMetadata.ManifestDigest,
},
}
case source.DirectoryScheme:
return &model.BomDescriptorComponent{
Component: model.Component{
Type: "file",
Name: srcMetadata.Path,
},
}
}
return nil
}
func toLicenses(licenses []string) *[]model.License {
if len(licenses) == 0 {
return nil
}
var result []model.License
for _, licenseName := range licenses {
result = append(result, model.License{
Name: licenseName,
})
}
return &result
}
+9 -2
View File
@@ -3,9 +3,12 @@ package formats
import (
"bytes"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/internal/formats/cyclonedx12xml"
"github.com/anchore/syft/internal/formats/spdx22json"
"github.com/anchore/syft/internal/formats/spdx22tagvalue"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/internal/formats/text"
"github.com/anchore/syft/syft/format"
)
@@ -14,6 +17,10 @@ func All() []format.Format {
return []format.Format{
syftjson.Format(),
table.Format(),
cyclonedx12xml.Format(),
spdx22json.Format(),
spdx22tagvalue.Format(),
text.Format(),
}
}
+21
View File
@@ -0,0 +1,21 @@
package spdx22json
import (
"encoding/json"
"io"
"github.com/anchore/syft/syft/sbom"
)
const anchoreNamespace = "https://anchore.com/syft"
func encoder(output io.Writer, s sbom.SBOM) error {
doc := toFormatModel(s)
enc := json.NewEncoder(output)
// prevent > and < from being escaped in the payload
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
return enc.Encode(&doc)
}
@@ -1,4 +1,4 @@
package packages
package spdx22json
import (
"flag"
@@ -11,9 +11,8 @@ import (
var updateSpdxJson = flag.Bool("update-spdx-json", false, "update the *.golden files for spdx-json presenters")
func TestSPDXJSONDirectoryPresenter(t *testing.T) {
catalog, metadata, _ := testutils.DirectoryInput(t)
testutils.AssertPresenterAgainstGoldenSnapshot(t,
NewSPDXJSONPresenter(catalog, metadata),
Format().Presenter(testutils.DirectoryInput(t)),
*updateSpdxJson,
spdxJsonRedactor,
)
@@ -21,9 +20,8 @@ func TestSPDXJSONDirectoryPresenter(t *testing.T) {
func TestSPDXJSONImagePresenter(t *testing.T) {
testImage := "image-simple"
catalog, metadata, _ := testutils.ImageInput(t, testImage)
testutils.AssertPresenterAgainstGoldenImageSnapshot(t,
NewSPDXJSONPresenter(catalog, metadata),
Format().Presenter(testutils.ImageInput(t, testImage, testutils.FromSnapshot())),
testImage,
*updateSpdxJson,
spdxJsonRedactor,
+13
View File
@@ -0,0 +1,13 @@
package spdx22json
import "github.com/anchore/syft/syft/format"
// note: this format is LOSSY relative to the syftjson formation, which means that decoding and validation is not supported at this time
func Format() format.Format {
return format.NewFormat(
format.SPDXJSONOption,
encoder,
nil,
nil,
)
}
@@ -1,4 +1,4 @@
package spdx22
package model
import "time"
@@ -1,4 +1,4 @@
package spdx22
package model
type Checksum struct {
// Identifies the algorithm used to produce the subject Checksum. One of: "SHA256", "SHA1", "SHA384", "MD2", "MD4", "SHA512", "MD6", "MD5", "SHA224"
@@ -1,4 +1,4 @@
package spdx22
package model
import "time"
@@ -1,4 +1,4 @@
package spdx22
package model
// derived from:
// - https://spdx.github.io/spdx-spec/appendix-III-RDF-data-model-implementation-and-identifier-syntax/
@@ -1,4 +1,4 @@
package spdx22
package model
type Element struct {
SPDXID string `json:"SPDXID"`
@@ -1,4 +1,4 @@
package spdx22
package model
// ElementID represents the identifier string portion of an SPDX element
// identifier. DocElementID should be used for any attributes which can
@@ -1,4 +1,4 @@
package spdx22
package model
type ExternalDocumentRef struct {
// externalDocumentId is a string containing letters, numbers, ., - and/or + which uniquely identifies an external document within this document.
@@ -1,4 +1,4 @@
package spdx22
package model
type ReferenceCategory string
@@ -1,4 +1,4 @@
package spdx22
package model
type FileType string
@@ -1,4 +1,4 @@
package spdx22
package model
type HasExtractedLicensingInfo struct {
// Verbatim license or licensing notice text that was discovered.
@@ -1,4 +1,4 @@
package spdx22
package model
type Item struct {
Element
@@ -1,4 +1,4 @@
package spdx22
package model
type Package struct {
Item
@@ -1,4 +1,4 @@
package spdx22
package model
// Why are there two package identifier fields Package Checksum and Package Verification?
// Although the values of the two fields Package Checksum and Package Verification are similar, they each serve a
@@ -1,4 +1,4 @@
package spdx22
package model
type Relationship struct {
// Id to which the SPDX element is related
@@ -1,4 +1,4 @@
package spdx22
package model
type StartPointer struct {
Offset int `json:"offset,omitempty"`
@@ -1,3 +1,3 @@
package spdx22
package model
const Version = "SPDX-2.2"
@@ -0,0 +1,4 @@
# Note: changes to this file will result in updating several test values. Consider making a new image fixture instead of editing this one.
FROM scratch
ADD file-1.txt /somefile-1.txt
ADD file-2.txt /somefile-2.txt
@@ -0,0 +1 @@
this file has contents
@@ -0,0 +1 @@
file-2 contents!
@@ -3,7 +3,7 @@
"name": "/some/path",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"created": "2021-10-12T18:40:22.948394Z",
"created": "2021-10-29T16:26:08.995826Z",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-[not provided]"
@@ -11,7 +11,7 @@
"licenseListVersion": "3.14"
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https:/anchore.com/syft/dir/some/path-98ae71fb-f276-4c5c-acf7-25770bf7bca2",
"documentNamespace": "https:/anchore.com/syft/dir/some/path-5362d380-914a-458f-b059-d8d27899574c",
"packages": [
{
"SPDXID": "SPDXRef-Package-python-package-1-1.0.1",
@@ -3,7 +3,7 @@
"name": "user-image-input",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"created": "2021-10-12T18:40:22.953633Z",
"created": "2021-10-29T16:26:09.001799Z",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-[not provided]"
@@ -11,7 +11,7 @@
"licenseListVersion": "3.14"
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https:/anchore.com/syft/image/user-image-input-149edbad-3c01-4ee0-b3a0-75232312bf51",
"documentNamespace": "https:/anchore.com/syft/image/user-image-input-3ad8571c-513f-4fce-944e-5125353c3186",
"packages": [
{
"SPDXID": "SPDXRef-Package-python-package-1-1.0.1",
@@ -0,0 +1,128 @@
package spdx22json
import (
"fmt"
"path"
"strings"
"time"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/formats/common/spdxhelpers"
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/google/uuid"
)
// toFormatModel creates and populates a new JSON document struct that follows the SPDX 2.2 spec from the given cataloging results.
func toFormatModel(s sbom.SBOM) model.Document {
name := documentName(s.Source)
packages, files, relationships := extractFromCatalog(s.Artifacts.PackageCatalog)
return model.Document{
Element: model.Element{
SPDXID: model.ElementID("DOCUMENT").String(),
Name: name,
},
SPDXVersion: model.Version,
CreationInfo: model.CreationInfo{
Created: time.Now().UTC(),
Creators: []string{
// note: key-value format derived from the JSON example document examples: https://github.com/spdx/spdx-spec/blob/v2.2/examples/SPDXJSONExample-v2.2.spdx.json
"Organization: Anchore, Inc",
"Tool: " + internal.ApplicationName + "-" + version.FromBuild().Version,
},
LicenseListVersion: spdxlicense.Version,
},
DataLicense: "CC0-1.0",
DocumentNamespace: documentNamespace(name, s.Source),
Packages: packages,
Files: files,
Relationships: relationships,
}
}
func documentName(srcMetadata source.Metadata) string {
switch srcMetadata.Scheme {
case source.ImageScheme:
return cleanSPDXName(srcMetadata.ImageMetadata.UserInput)
case source.DirectoryScheme:
return cleanSPDXName(srcMetadata.Path)
}
// TODO: is this alright?
return uuid.Must(uuid.NewRandom()).String()
}
func documentNamespace(name string, srcMetadata source.Metadata) string {
input := "unknown-source-type"
switch srcMetadata.Scheme {
case source.ImageScheme:
input = "image"
case source.DirectoryScheme:
input = "dir"
}
uniqueID := uuid.Must(uuid.NewRandom())
identifier := path.Join(input, uniqueID.String())
if name != "." {
identifier = path.Join(input, fmt.Sprintf("%s-%s", name, uniqueID.String()))
}
return path.Join(anchoreNamespace, identifier)
}
func extractFromCatalog(catalog *pkg.Catalog) ([]model.Package, []model.File, []model.Relationship) {
packages := make([]model.Package, 0)
relationships := make([]model.Relationship, 0)
files := make([]model.File, 0)
for _, p := range catalog.Sorted() {
license := spdxhelpers.License(p)
packageSpdxID := model.ElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.Version)).String()
packageFiles, fileIDs, packageFileRelationships := spdxhelpers.Files(packageSpdxID, p)
files = append(files, packageFiles...)
relationships = append(relationships, packageFileRelationships...)
// note: the license concluded and declared should be the same since we are collecting license information
// from the project data itself (the installed package files).
packages = append(packages, model.Package{
Description: spdxhelpers.Description(p),
DownloadLocation: spdxhelpers.DownloadLocation(p),
ExternalRefs: spdxhelpers.ExternalRefs(p),
FilesAnalyzed: false,
HasFiles: fileIDs,
Homepage: spdxhelpers.Homepage(p),
LicenseDeclared: license, // The Declared License is what the authors of a project believe govern the package
Originator: spdxhelpers.Originator(p),
SourceInfo: spdxhelpers.SourceInfo(p),
VersionInfo: p.Version,
Item: model.Item{
LicenseConcluded: license, // The Concluded License field is the license the SPDX file creator believes governs the package
Element: model.Element{
SPDXID: packageSpdxID,
Name: p.Name,
},
},
})
}
return packages, files, relationships
}
func cleanSPDXName(name string) string {
// remove # according to specification
name = strings.ReplaceAll(name, "#", "-")
// remove : for url construction
name = strings.ReplaceAll(name, ":", "-")
// clean relative pathing
return path.Clean(name)
}
@@ -0,0 +1,13 @@
package spdx22tagvalue
import (
"io"
"github.com/anchore/syft/syft/sbom"
"github.com/spdx/tools-golang/tvsaver"
)
func encoder(output io.Writer, s sbom.SBOM) error {
model := toFormatModel(s)
return tvsaver.Save2_2(&model, output)
}
@@ -1,4 +1,4 @@
package packages
package spdx22tagvalue
import (
"flag"
@@ -11,9 +11,9 @@ import (
var updateSpdxTagValue = flag.Bool("update-spdx-tv", false, "update the *.golden files for spdx-tv presenters")
func TestSPDXTagValueDirectoryPresenter(t *testing.T) {
catalog, metadata, _ := testutils.DirectoryInput(t)
testutils.AssertPresenterAgainstGoldenSnapshot(t,
NewSPDXTagValuePresenter(catalog, metadata),
Format().Presenter(testutils.DirectoryInput(t)),
*updateSpdxTagValue,
spdxTagValueRedactor,
)
@@ -21,9 +21,8 @@ func TestSPDXTagValueDirectoryPresenter(t *testing.T) {
func TestSPDXTagValueImagePresenter(t *testing.T) {
testImage := "image-simple"
catalog, metadata, _ := testutils.ImageInput(t, testImage)
testutils.AssertPresenterAgainstGoldenImageSnapshot(t,
NewSPDXTagValuePresenter(catalog, metadata),
Format().Presenter(testutils.ImageInput(t, testImage, testutils.FromSnapshot())),
testImage,
*updateSpdxTagValue,
spdxTagValueRedactor,
+13
View File
@@ -0,0 +1,13 @@
package spdx22tagvalue
import "github.com/anchore/syft/syft/format"
// note: this format is LOSSY relative to the syftjson formation, which means that decoding and validation is not supported at this time
func Format() format.Format {
return format.NewFormat(
format.SPDXTagValueOption,
encoder,
nil,
nil,
)
}
@@ -0,0 +1,4 @@
# Note: changes to this file will result in updating several test values. Consider making a new image fixture instead of editing this one.
FROM scratch
ADD file-1.txt /somefile-1.txt
ADD file-2.txt /somefile-2.txt
@@ -0,0 +1 @@
this file has contents
@@ -0,0 +1 @@
file-2 contents!
@@ -1,38 +1,23 @@
package packages
package spdx22tagvalue
import (
"fmt"
"io"
"time"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/formats/common/spdxhelpers"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/tvsaver"
)
// SPDXTagValuePresenter is a SPDX presentation object for the syft results (see https://github.com/spdx/spdx-spec)
type SPDXTagValuePresenter struct {
catalog *pkg.Catalog
srcMetadata source.Metadata
}
// NewJSONPresenter creates a new JSON presenter object for the given cataloging results.
func NewSPDXTagValuePresenter(catalog *pkg.Catalog, srcMetadata source.Metadata) *SPDXTagValuePresenter {
return &SPDXTagValuePresenter{
catalog: catalog,
srcMetadata: srcMetadata,
}
}
// Present the catalog results to the given writer.
// nolint: funlen
func (pres *SPDXTagValuePresenter) Present(output io.Writer) error {
doc := spdx.Document2_2{
// toFormatModel creates and populates a new JSON document struct that follows the SPDX 2.2 spec from the given cataloging results.
// nolint:funlen
func toFormatModel(s sbom.SBOM) spdx.Document2_2 {
return spdx.Document2_2{
CreationInfo: &spdx.CreationInfo2_2{
// 2.1: SPDX Version; should be in the format "SPDX-2.2"
// Cardinality: mandatory, one
@@ -48,7 +33,7 @@ func (pres *SPDXTagValuePresenter) Present(output io.Writer) error {
// 2.4: Document Name
// Cardinality: mandatory, one
DocumentName: pres.srcMetadata.ImageMetadata.UserInput,
DocumentName: s.Source.ImageMetadata.UserInput,
// 2.5: Document Namespace
// Cardinality: mandatory, one
@@ -67,7 +52,7 @@ func (pres *SPDXTagValuePresenter) Present(output io.Writer) error {
// In many cases, the URI will point to a web accessible document, but this should not be assumed
// to be the case.
DocumentNamespace: fmt.Sprintf("https://anchore.com/syft/image/%s", pres.srcMetadata.ImageMetadata.UserInput),
DocumentNamespace: fmt.Sprintf("https://anchore.com/syft/image/%s", s.Source.ImageMetadata.UserInput),
// 2.6: External Document References
// Cardinality: optional, one or many
@@ -96,25 +81,23 @@ func (pres *SPDXTagValuePresenter) Present(output io.Writer) error {
// Cardinality: optional, one
DocumentComment: "",
},
Packages: pres.packages(),
Packages: toFormatPackages(s.Artifacts.PackageCatalog),
}
return tvsaver.Save2_2(&doc, output)
}
// packages populates all Package Information from the package Catalog (see https://spdx.github.io/spdx-spec/3-package-information/)
// nolint: funlen
func (pres *SPDXTagValuePresenter) packages() map[spdx.ElementID]*spdx.Package2_2 {
func toFormatPackages(catalog *pkg.Catalog) map[spdx.ElementID]*spdx.Package2_2 {
results := make(map[spdx.ElementID]*spdx.Package2_2)
for p := range pres.catalog.Enumerate() {
for p := range catalog.Enumerate() {
// name should be guaranteed to be unique, but semantically useful and stable
id := fmt.Sprintf("Package-%+v-%s", p.Type, p.Name)
// If the Concluded License is not the same as the Declared License, a written explanation should be provided
// in the Comments on License field (section 3.16). With respect to NOASSERTION, a written explanation in
// the Comments on License field (section 3.16) is preferred.
license := getSPDXLicense(p)
license := spdxhelpers.License(p)
results[spdx.ElementID(id)] = &spdx.Package2_2{
@@ -274,7 +257,7 @@ func (pres *SPDXTagValuePresenter) packages() map[spdx.ElementID]*spdx.Package2_
}
func formatSPDXExternalRefs(p *pkg.Package) (refs []*spdx.PackageExternalReference2_2) {
for _, ref := range getSPDXExternalRefs(p) {
for _, ref := range spdxhelpers.ExternalRefs(p) {
refs = append(refs, &spdx.PackageExternalReference2_2{
Category: string(ref.ReferenceCategory),
RefType: string(ref.ReferenceType),
+4 -5
View File
@@ -5,19 +5,18 @@ import (
"fmt"
"io"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal/formats/syftjson/model"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
func decoder(reader io.Reader) (*pkg.Catalog, *source.Metadata, *distro.Distro, source.Scope, error) {
func decoder(reader io.Reader) (*sbom.SBOM, error) {
dec := json.NewDecoder(reader)
var doc model.Document
err := dec.Decode(&doc)
if err != nil {
return nil, nil, nil, source.UnknownScope, fmt.Errorf("unable to decode syft-json: %w", err)
return nil, fmt.Errorf("unable to decode syft-json: %w", err)
}
return toSyftModel(doc)
+6 -8
View File
@@ -5,8 +5,6 @@ import (
"strings"
"testing"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/internal/formats/common/testutils"
"github.com/go-test/deep"
"github.com/stretchr/testify/assert"
@@ -14,20 +12,20 @@ import (
func TestEncodeDecodeCycle(t *testing.T) {
testImage := "image-simple"
originalCatalog, originalMetadata, _ := testutils.ImageInput(t, testImage)
originalSBOM := testutils.ImageInput(t, testImage)
var buf bytes.Buffer
assert.NoError(t, encoder(&buf, originalCatalog, &originalMetadata, nil, source.SquashedScope))
assert.NoError(t, encoder(&buf, originalSBOM))
actualCatalog, actualMetadata, _, _, err := decoder(bytes.NewReader(buf.Bytes()))
actualSBOM, err := decoder(bytes.NewReader(buf.Bytes()))
assert.NoError(t, err)
for _, d := range deep.Equal(originalMetadata, *actualMetadata) {
for _, d := range deep.Equal(originalSBOM.Source, actualSBOM.Source) {
t.Errorf("metadata difference: %+v", d)
}
actualPackages := actualCatalog.Sorted()
for idx, p := range originalCatalog.Sorted() {
actualPackages := actualSBOM.Artifacts.PackageCatalog.Sorted()
for idx, p := range originalSBOM.Artifacts.PackageCatalog.Sorted() {
if !assert.Equal(t, p.Name, actualPackages[idx].Name) {
t.Errorf("different package at idx=%d: %s vs %s", idx, p.Name, actualPackages[idx].Name)
continue
+3 -6
View File
@@ -4,15 +4,12 @@ import (
"encoding/json"
"io"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/syft/sbom"
)
func encoder(output io.Writer, catalog *pkg.Catalog, srcMetadata *source.Metadata, d *distro.Distro, scope source.Scope) error {
func encoder(output io.Writer, s sbom.SBOM) error {
// TODO: application config not available yet
doc := ToFormatModel(catalog, srcMetadata, d, scope, nil)
doc := ToFormatModel(s, nil)
enc := json.NewEncoder(output)
// prevent > and < from being escaped in the payload
+2 -7
View File
@@ -4,27 +4,22 @@ import (
"flag"
"testing"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/internal/formats/common/testutils"
"github.com/anchore/syft/syft/format"
)
var updateJson = flag.Bool("update-json", false, "update the *.golden files for json presenters")
func TestDirectoryPresenter(t *testing.T) {
catalog, metadata, distro := testutils.DirectoryInput(t)
testutils.AssertPresenterAgainstGoldenSnapshot(t,
format.NewPresenter(encoder, catalog, &metadata, distro, source.SquashedScope),
Format().Presenter(testutils.DirectoryInput(t)),
*updateJson,
)
}
func TestImagePresenter(t *testing.T) {
testImage := "image-simple"
catalog, metadata, distro := testutils.ImageInput(t, testImage)
testutils.AssertPresenterAgainstGoldenImageSnapshot(t,
format.NewPresenter(encoder, catalog, &metadata, distro, source.SquashedScope),
Format().Presenter(testutils.ImageInput(t, testImage, testutils.FromSnapshot())),
testImage,
*updateJson,
)
+1 -6
View File
@@ -20,11 +20,6 @@ type sourceUnpacker struct {
Target json.RawMessage `json:"target"`
}
type ImageSource struct {
source.ImageMetadata
Scope source.Scope `json:"scope"`
}
// UnmarshalJSON populates a source object from JSON bytes.
func (s *Source) UnmarshalJSON(b []byte) error {
var unpacker sourceUnpacker
@@ -43,7 +38,7 @@ func (s *Source) UnmarshalJSON(b []byte) error {
}
case "image":
var payload ImageSource
var payload source.ImageMetadata
if err := json.Unmarshal(unpacker.Target, &payload); err != nil {
return err
}
@@ -88,8 +88,7 @@
],
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NjcsImRpZ2VzdCI6InNoYTI1NjoyNDgwMTYwYjU1YmVjNDBjNDRkM2IxNDVjN2IyYzFjNDcxNjBkYjg1NzVjM2RjYWUwODZkNzZiOTM3MGFlN2NhIn0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1NjpmYjZiZWVjYjc1YjM5ZjRiYjgxM2RiZjE3N2U1MDFlZGQ1ZGRiM2U2OWJiNDVjZWRlYjc4YzY3NmVlMWI3YTU5In0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjMxOWI1ODhjZTY0MjUzYTg3YjUzM2M4ZWQwMWNmMDAyNWUwZWFjOThlN2I1MTZlMTI1MzI5NTdlMTI0NGZkZWMifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjEtMTAtMDRUMTE6NDA6MDAuNjM4Mzk0NVoiLCJoaXN0b3J5IjpbeyJjcmVhdGVkIjoiMjAyMS0xMC0wNFQxMTo0MDowMC41OTA3MzE2WiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0xLnR4dCAvc29tZWZpbGUtMS50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn0seyJjcmVhdGVkIjoiMjAyMS0xMC0wNFQxMTo0MDowMC42MzgzOTQ1WiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0yLnR4dCAvc29tZWZpbGUtMi50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn1dLCJvcyI6ImxpbnV4Iiwicm9vdGZzIjp7InR5cGUiOiJsYXllcnMiLCJkaWZmX2lkcyI6WyJzaGEyNTY6ZmI2YmVlY2I3NWIzOWY0YmI4MTNkYmYxNzdlNTAxZWRkNWRkYjNlNjliYjQ1Y2VkZWI3OGM2NzZlZTFiN2E1OSIsInNoYTI1NjozMTliNTg4Y2U2NDI1M2E4N2I1MzNjOGVkMDFjZjAwMjVlMGVhYzk4ZTdiNTE2ZTEyNTMyOTU3ZTEyNDRmZGVjIl19fQ==",
"repoDigests": [],
"scope": "Squashed"
"repoDigests": []
}
},
"distro": {
+10 -11
View File
@@ -3,6 +3,8 @@ package syftjson
import (
"fmt"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/formats/syftjson/model"
"github.com/anchore/syft/internal/log"
@@ -13,17 +15,17 @@ import (
)
// TODO: this is export4ed for the use of the power-user command (temp)
func ToFormatModel(catalog *pkg.Catalog, srcMetadata *source.Metadata, d *distro.Distro, scope source.Scope, applicationConfig interface{}) model.Document {
src, err := toSourceModel(srcMetadata, scope)
func ToFormatModel(s sbom.SBOM, applicationConfig interface{}) model.Document {
src, err := toSourceModel(s.Source)
if err != nil {
log.Warnf("unable to create syft-json source object: %+v", err)
}
return model.Document{
Artifacts: toPackageModels(catalog),
ArtifactRelationships: toRelationshipModel(pkg.NewRelationships(catalog)),
Artifacts: toPackageModels(s.Artifacts.PackageCatalog),
ArtifactRelationships: toRelationshipModel(pkg.NewRelationships(s.Artifacts.PackageCatalog)),
Source: src,
Distro: toDistroModel(d),
Distro: toDistroModel(s.Artifacts.Distro),
Descriptor: model.Descriptor{
Name: internal.ApplicationName,
Version: version.FromBuild().Version,
@@ -99,15 +101,12 @@ func toRelationshipModel(relationships []pkg.Relationship) []model.Relationship
}
// toSourceModel creates a new source object to be represented into JSON.
func toSourceModel(src *source.Metadata, scope source.Scope) (model.Source, error) {
func toSourceModel(src source.Metadata) (model.Source, error) {
switch src.Scheme {
case source.ImageScheme:
return model.Source{
Type: "image",
Target: model.ImageSource{
ImageMetadata: src.ImageMetadata,
Scope: scope,
},
Type: "image",
Target: src.ImageMetadata,
}, nil
case source.DirectoryScheme:
return model.Source{
+15 -11
View File
@@ -5,35 +5,39 @@ import (
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
func toSyftModel(doc model.Document) (*pkg.Catalog, *source.Metadata, *distro.Distro, source.Scope, error) {
func toSyftModel(doc model.Document) (*sbom.SBOM, error) {
dist, err := distro.NewDistro(distro.Type(doc.Distro.Name), doc.Distro.Version, doc.Distro.IDLike)
if err != nil {
return nil, nil, nil, source.UnknownScope, err
return nil, err
}
srcMetadata, scope := toSyftSourceData(doc.Source)
return toSyftCatalog(doc.Artifacts), srcMetadata, &dist, scope, nil
return &sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: toSyftCatalog(doc.Artifacts),
Distro: &dist,
},
Source: *toSyftSourceData(doc.Source),
}, nil
}
func toSyftSourceData(s model.Source) (*source.Metadata, source.Scope) {
func toSyftSourceData(s model.Source) *source.Metadata {
switch s.Type {
case "directory":
return &source.Metadata{
Scheme: source.DirectoryScheme,
Path: s.Target.(string),
}, source.UnknownScope
}
case "image":
parsedSource := s.Target.(model.ImageSource)
return &source.Metadata{
Scheme: source.ImageScheme,
ImageMetadata: parsedSource.ImageMetadata,
}, parsedSource.Scope
ImageMetadata: s.Target.(source.ImageMetadata),
}
}
return nil, source.UnknownScope
return nil
}
func toSyftCatalog(pkgs []model.Package) *pkg.Catalog {
+4 -6
View File
@@ -6,18 +6,16 @@ import (
"sort"
"strings"
"github.com/olekukonko/tablewriter"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/olekukonko/tablewriter"
)
func encoder(output io.Writer, catalog *pkg.Catalog, _ *source.Metadata, _ *distro.Distro, _ source.Scope) error {
func encoder(output io.Writer, s sbom.SBOM) error {
var rows [][]string
columns := []string{"Name", "Version", "Type"}
for _, p := range catalog.Sorted() {
for _, p := range s.Artifacts.PackageCatalog.Sorted() {
row := []string{
p.Name,
p.Version,
+1 -4
View File
@@ -5,17 +5,14 @@ import (
"testing"
"github.com/anchore/syft/internal/formats/common/testutils"
"github.com/anchore/syft/syft/format"
"github.com/anchore/syft/syft/source"
"github.com/go-test/deep"
)
var updateTableGoldenFiles = flag.Bool("update-table", false, "update the *.golden files for table format")
func TestTablePresenter(t *testing.T) {
catalog, metadata, distro := testutils.DirectoryInput(t)
testutils.AssertPresenterAgainstGoldenSnapshot(t,
format.NewPresenter(encoder, catalog, &metadata, distro, source.SquashedScope),
Format().Presenter(testutils.DirectoryInput(t)),
*updateTableGoldenFiles,
)
}
+54
View File
@@ -0,0 +1,54 @@
package text
import (
"fmt"
"io"
"text/tabwriter"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
func encoder(output io.Writer, s sbom.SBOM) error {
// init the tabular writer
w := new(tabwriter.Writer)
w.Init(output, 0, 8, 0, '\t', tabwriter.AlignRight)
switch s.Source.Scheme {
case source.DirectoryScheme:
fmt.Fprintf(w, "[Path: %s]\n", s.Source.Path)
case source.ImageScheme:
fmt.Fprintln(w, "[Image]")
for idx, l := range s.Source.ImageMetadata.Layers {
fmt.Fprintln(w, " Layer:\t", idx)
fmt.Fprintln(w, " Digest:\t", l.Digest)
fmt.Fprintln(w, " Size:\t", l.Size)
fmt.Fprintln(w, " MediaType:\t", l.MediaType)
fmt.Fprintln(w)
w.Flush()
}
default:
return fmt.Errorf("unsupported source: %T", s.Source.Scheme)
}
// populate artifacts...
rows := 0
for _, p := range s.Artifacts.PackageCatalog.Sorted() {
fmt.Fprintf(w, "[%s]\n", p.Name)
fmt.Fprintln(w, " Version:\t", p.Version)
fmt.Fprintln(w, " Type:\t", string(p.Type))
fmt.Fprintln(w, " Found by:\t", p.FoundBy)
fmt.Fprintln(w)
w.Flush()
rows++
}
if rows == 0 {
fmt.Fprintln(output, "No packages discovered")
return nil
}
return nil
}
@@ -1,4 +1,4 @@
package packages
package text
import (
"flag"
@@ -10,18 +10,16 @@ import (
var updateTextPresenterGoldenFiles = flag.Bool("update-text", false, "update the *.golden files for text presenters")
func TestTextDirectoryPresenter(t *testing.T) {
catalog, metadata, _ := testutils.DirectoryInput(t)
testutils.AssertPresenterAgainstGoldenSnapshot(t,
NewTextPresenter(catalog, metadata),
Format().Presenter(testutils.DirectoryInput(t)),
*updateTextPresenterGoldenFiles,
)
}
func TestTextImagePresenter(t *testing.T) {
testImage := "image-simple"
catalog, metadata, _ := testutils.ImageInput(t, testImage)
testutils.AssertPresenterAgainstGoldenImageSnapshot(t,
NewTextPresenter(catalog, metadata),
Format().Presenter(testutils.ImageInput(t, testImage, testutils.FromSnapshot())),
testImage,
*updateTextPresenterGoldenFiles,
)
+12
View File
@@ -0,0 +1,12 @@
package text
import "github.com/anchore/syft/syft/format"
func Format() format.Format {
return format.NewFormat(
format.TextOption,
encoder,
nil,
nil,
)
}
@@ -0,0 +1,4 @@
# Note: changes to this file will result in updating several test values. Consider making a new image fixture instead of editing this one.
FROM scratch
ADD file-1.txt /somefile-1.txt
ADD file-2.txt /somefile-2.txt
@@ -0,0 +1 @@
this file has contents
@@ -0,0 +1 @@
file-2 contents!
+5 -1
View File
@@ -12,5 +12,9 @@ func IsPipedInput() (bool, error) {
return false, fmt.Errorf("unable to determine if there is piped input: %w", err)
}
return fi.Mode()&os.ModeCharDevice == 0, nil
// note: we should NOT use the absence of a character device here as the hint that there may be input expected
// on stdin, as running syft as a subprocess you would expect no character device to be present but input can
// be from either stdin or indicated by the CLI. Checking if stdin is a pipe is the most direct way to determine
// if there *may* be bytes that will show up on stdin that should be used for the analysis source.
return fi.Mode()&os.ModeNamedPipe != 0, nil
}
@@ -1,69 +0,0 @@
package packages
import (
"encoding/xml"
"time"
"github.com/anchore/syft/syft/source"
)
// Source: https://cyclonedx.org/ext/bom-descriptor/
// CycloneDxBomDescriptor represents all metadata surrounding the BOM report (such as when the BOM was made, with which tool, and the item being cataloged).
type CycloneDxBomDescriptor struct {
XMLName xml.Name `xml:"metadata"`
Timestamp string `xml:"timestamp,omitempty"` // The date and time (timestamp) when the document was created
Tools []CycloneDxBdTool `xml:"tools>tool"` // The tool used to create the BOM.
Component *CycloneDxBdComponent `xml:"component"` // The component that the BOM describes.
}
// CycloneDxBdTool represents the tool that created the BOM report.
type CycloneDxBdTool struct {
XMLName xml.Name `xml:"tool"`
Vendor string `xml:"vendor,omitempty"` // The vendor of the tool used to create the BOM.
Name string `xml:"name,omitempty"` // The name of the tool used to create the BOM.
Version string `xml:"version,omitempty"` // The version of the tool used to create the BOM.
// TODO: hashes, author, manufacture, supplier
// TODO: add user-defined fields for the remaining build/version parameters
}
// CycloneDxBdComponent represents the software/package being cataloged.
type CycloneDxBdComponent struct {
XMLName xml.Name `xml:"component"`
CycloneDxComponent
}
// NewCycloneDxBomDescriptor returns a new CycloneDxBomDescriptor tailored for the current time and "syft" tool details.
func NewCycloneDxBomDescriptor(name, version string, srcMetadata source.Metadata) *CycloneDxBomDescriptor {
descriptor := CycloneDxBomDescriptor{
XMLName: xml.Name{},
Timestamp: time.Now().Format(time.RFC3339),
Tools: []CycloneDxBdTool{
{
Vendor: "anchore",
Name: name,
Version: version,
},
},
}
switch srcMetadata.Scheme {
case source.ImageScheme:
descriptor.Component = &CycloneDxBdComponent{
CycloneDxComponent: CycloneDxComponent{
Type: "container",
Name: srcMetadata.ImageMetadata.UserInput,
Version: srcMetadata.ImageMetadata.ManifestDigest,
},
}
case source.DirectoryScheme:
descriptor.Component = &CycloneDxBdComponent{
CycloneDxComponent: CycloneDxComponent{
Type: "file",
Name: srcMetadata.Path,
},
}
}
return &descriptor
}
@@ -1,27 +0,0 @@
package packages
import "encoding/xml"
// CycloneDxComponent represents a single element in the CycloneDX BOM
type CycloneDxComponent struct {
XMLName xml.Name `xml:"component"`
Type string `xml:"type,attr"` // Required; Describes if the component is a library, framework, application, container, operating system, firmware, hardware device, or file
Supplier string `xml:"supplier,omitempty"` // The organization that supplied the component. The supplier may often be the manufacture, but may also be a distributor or repackager.
Author string `xml:"author,omitempty"` // The person(s) or organization(s) that authored the component
Publisher string `xml:"publisher,omitempty"` // The person(s) or organization(s) that published the component
Group string `xml:"group,omitempty"` // The high-level classification that a project self-describes as. This will often be a shortened, single name of the company or project that produced the component, or the source package or domain name.
Name string `xml:"name"` // Required; The name of the component as defined by the project
Version string `xml:"version"` // Required; The version of the component as defined by the project
Description string `xml:"description,omitempty"` // A description of the component
Licenses *[]CycloneDxLicense `xml:"licenses>license"` // A node describing zero or more license names, SPDX license IDs or expressions
PackageURL string `xml:"purl,omitempty"` // Specifies the package-url (PackageURL). The purl, if specified, must be valid and conform to the specification defined at: https://github.com/package-url/purl-spec
// TODO: source, hashes, copyright, cpe, purl, swid, modified, pedigree, externalReferences
// TODO: add user-defined parameters for syft-specific values (image layer index, cataloger, location path, etc.)
}
// CycloneDxLicense represents a single software license for a CycloneDxComponent
type CycloneDxLicense struct {
XMLName xml.Name `xml:"license"`
ID string `xml:"id,omitempty"` // A valid SPDX license ID
Name string `xml:"name,omitempty"` // If SPDX does not define the license used, this field may be used to provide the license name
}

Some files were not shown because too many files have changed in this diff Show More