Compare commits

...
Author SHA1 Message Date
Alex Goodman ecf11f0e3d [WIP] migrate helper functions for spdx
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-10 14:32:44 -05:00
Alex Goodman 628c2e44a5 add test reminders to put new relationships under test
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-10 13:34:41 -05:00
Alex Goodman aba0de6e65 update ownership-by-file-overlap relationship description
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-10 11:15:18 -05:00
Alex Goodman cb0278f3bc hoist up package relationship discovery to analysis stage
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-10 11:05:01 -05:00
Alex Goodman b519340b86 enable source.Location to be identifiable
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-10 09:05:33 -05:00
Alex Goodman 2356539ebe use zero value for nils in ID generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-09 17:26:38 -05:00
Alex Goodman 1537e73083 rename Identity() to ID()
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-09 17:23:48 -05:00
Alex Goodman 52adfcbd44 stabilize package ID relative to encode-decode format cycles
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-09 16:53:16 -05:00
Alex Goodman a3f0d659da adjust spdx helpers to use copy of packages
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-09 10:53:55 -05:00
Alex Goodman 8a0fa5d3ad remove catalog ID assignment
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-09 10:49:37 -05:00
Alex Goodman 253faf5652 fix linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-08 16:50:35 -05:00
Alex Goodman 69d2b1ba3c add artifact.Identifiable by Identity() method
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-08 16:46:14 -05:00
Alex Goodman b08a11e46d fix more tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-08 16:46:12 -05:00
Alex Goodman fef951c29b return relationships from tasks
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-08 16:44:34 -05:00
Alex Goodman a906b9a03a migrate pkg.ID and pkg.Relationship to artifact package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-08 16:43:20 -05:00
Christopher Angelo PhillipsandAlex Goodman 9bbc9ff633 Add layer information for go-module-binary-cataloger (#620)
* update functions to pass Location

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update unit tests to pass new locations

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* fix image source.FileResolvers to include layer info

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add non-empty location in golang binary cataloger testing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-08 16:05:53 -05:00
Alex GoodmanandChristopher Angelo Phillips bb0f35bac4 Introduce a single SBOM document (#606)
* [wip] single sbom doc

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix more tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update cli tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove scope in import path

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* swap SPDX tag-value formatter to single sbom document

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bust CLI cache

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update fixture to byte diff

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* byte for byte

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* bust the cache

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* who needs cache

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* add jar for testing

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* no more bit flips

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update apk with the delta for image and directory cases

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* restore cache workflow

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-11-05 10:05:49 -04:00
Jonas Galvão Xavier 6d0ee326d8 ignore irregular files while indexing directory (#616)
* stop hidding  command from help doc

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* do not index irregular files

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix fixture dir and err name

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* improve the description of irregular files

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explicitly check indexed file name

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2021-11-04 11:24:05 -07:00
Alex Goodman 23a67dd729 document file scheme (#613)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 19:47:25 -04:00
Christopher Angelo Phillips 3e20edee55 update readme with private registry section (#610) 2021-11-02 15:00:56 -04:00
Alex Goodman 8ec3f1d102 Use anchore fork of go-presenter (#611)
* use anchore fork of go-presenter

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* drop coverage threshold

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 18:09:26 +00:00
Alex Goodman ecab44bba1 Add SPDX tag-value format object (#605)
* add new spdx tag-value format

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove public presenter package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 13:45:34 -04:00
Alex GoodmanandChristopher Angelo Phillips 484284706c Use named pipe bit when checking for piped input (#603)
* use named pipe bit when checking for piped input

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-11-02 16:41:02 +00:00
Alex Goodman 3ac95ac4f6 Allow for cataloging a single file (#608)
* allow for cataloging a single file

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use all catalogers for file schemes

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 12:09:06 -04:00
Christopher Angelo Phillips a2882ee810 Update package identifier to be constant across multiple syft run (#595)
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-29 16:00:36 +00:00
Alex Goodman 358b3a2cf8 port text presenter to a format object (#604)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-29 11:31:15 -04:00
Alex Goodman 9aca23f766 Add SPDX JSON format object (#584)
* remove existing spdxjson presenter + helpers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add new spdx22json format

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add common sdpxhelpers (migrated)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use new common spdx helpers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* wire up new spdx22json format object

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove lossless syft-specific property bags

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove spdxjson decoder and validator

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add nil checks in spdx test helpers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove empty default case

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use explicit golden snapshot

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-29 14:55:20 +00:00
Alex Goodman 4a2d1d7225 Port cyclonedx presenter to format object (#589)
* add new cyclonedx format object

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove cyclonedx presenter

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove cyclonedx presenter call

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove dependence on golden images for format tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* wire up new formt + rename all-presenters ref

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add CLI test to ensure that all formats can be expressed as report output

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add cyclonedx version and encoding format to package name

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* optionally preserve format snapshot images

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting + text unit tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-29 10:17:03 -04:00
Christopher Angelo Phillips 3286a4d4cc update docker manifest tagging (#600)
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-28 15:41:39 -04:00
Christopher Angelo Phillips 0ff7013382 update manifest to use Tag over Version
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-28 15:05:34 -04:00
Christopher Angelo Phillips 71708e5bf8 update goreleaser with correct tag information (#598)
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-28 13:37:22 -04:00
Jonas Galvão Xavier dd024298eb stop hidding command from help doc (#596)
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2021-10-27 13:49:04 -07:00
Christopher Angelo Phillips 1c63943055 Add arm64 image support and Darwin M1 support to .goreleaser.yaml (#591)
* update support arm64

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* small update syntax

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* restore release command

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* add docker manifests

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-26 09:42:35 -04:00
Alex Goodman ff5fa2dc1d pull in space suffix fix (#587)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-25 10:14:17 -04:00
Alex Goodman fb588ff500 replace table presenter with format object (#586)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-24 08:09:27 -04:00
Alex Goodman d5b425e1b5 force exiting the etui upon worker error (#583)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-22 13:44:50 -04:00
Dan Luhring 7b319e32bb Remove webinar announcement (#582)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-10-22 07:48:36 -04:00
250 changed files with 3268 additions and 2416 deletions
+3
View File
@@ -232,6 +232,9 @@ jobs:
- uses: actions/checkout@v2
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v2.1.3
+63 -8
View File
@@ -12,8 +12,27 @@ builds:
env:
- CGO_ENABLED=0
goos:
- windows
- linux
goarch:
- amd64
- arm64
# Set the modified timestamp on the output binary to the git timestamp (to ensure a reproducible build)
mod_timestamp: '{{ .CommitTimestamp }}'
ldflags: |
-w
-s
-extldflags '-static'
-X github.com/anchore/syft/internal/version.version={{.Version}}
-X github.com/anchore/syft/internal/version.gitCommit={{.Commit}}
-X github.com/anchore/syft/internal/version.buildDate={{.Date}}
-X github.com/anchore/syft/internal/version.gitTreeState={{.Env.BUILD_GIT_TREE_STATE}}
- binary: syft
id: syft-win
env:
- CGO_ENABLED=0
goos:
- windows
goarch:
- amd64
# Set the modified timestamp on the output binary to the git timestamp (to ensure a reproducible build)
@@ -26,6 +45,7 @@ builds:
-X github.com/anchore/syft/internal/version.gitCommit={{.Commit}}
-X github.com/anchore/syft/internal/version.buildDate={{.Date}}
-X github.com/anchore/syft/internal/version.gitTreeState={{.Env.BUILD_GIT_TREE_STATE}}
# For more info on this macOS build, see: https://github.com/mitchellh/gon#usage-with-goreleaser
- binary: syft
id: syft-macos
@@ -35,6 +55,7 @@ builds:
- darwin
goarch:
- amd64
- arm64
# Set the modified timestamp on the output binary to the git timestamp (to ensure a reproducible build)
mod_timestamp: '{{ .CommitTimestamp }}'
ldflags: |
@@ -50,10 +71,11 @@ archives:
- format: tar.gz
builds:
- syft # i.e. Linux only
- format: zip # This is a hack! We don't actually intend to use _this_ ZIP file, we just need goreleaser to consider the ZIP file produced by gon (which will have the same file name) to be an artifact so we can use it downstream in publishing (e.g. to a homebrew tap)
- format: zip # This is a hack for syft-macos! We don't actually intend to use _this_ ZIP file, we just need goreleaser to consider the ZIP file produced by gon (which will have the same file name) to be an artifact so we can use it downstream in publishing (e.g. to a homebrew tap)
id: syft-zip
builds:
- syft-macos
- syft-win
signs:
- artifacts: checksum
@@ -91,15 +113,48 @@ brews:
description: *description
dockers:
- dockerfile: Dockerfile
image_templates:
- image_templates:
- "anchore/syft:latest"
- "anchore/syft:{{ .Tag }}"
- "anchore/syft:v{{ .Major }}"
- "anchore/syft:v{{ .Major }}.{{ .Minor }}"
- "anchore/syft:{{ .Tag }}-amd64"
- "anchore/syft:v{{ .Major }}-amd64"
- "anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64"
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/amd64"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
use_buildx: true
- image_templates:
- "anchore/syft:{{ .Tag }}-arm64v8"
- "anchore/syft:v{{ .Major }}-arm64v8"
- "anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8"
goarch: arm64
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/arm64/v8"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
docker_manifests:
- name_template: anchore/syft:{{ .Tag }}
image_templates:
- anchore/syft:{{ .Tag }}-amd64
- anchore/syft:v{{ .Major }}-amd64
- anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64
- anchore/syft:{{ .Tag }}-arm64v8
- anchore/syft:v{{ .Major }}-arm64v8
- anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8
- name_template: anchore/syft:latest
image_templates:
- anchore/syft:{{ .Tag }}-amd64
- anchore/syft:v{{ .Major }}-amd64
- anchore/syft:v{{ .Major }}.{{ .Minor }}-amd64
- anchore/syft:{{ .Tag }}-arm64v8
- anchore/syft:v{{ .Major }}-arm64v8
- anchore/syft:v{{ .Major }}.{{ .Minor }}-arm64v8
+7 -3
View File
@@ -15,10 +15,10 @@ RESET := $(shell tput -T linux sgr0)
TITLE := $(BOLD)$(PURPLE)
SUCCESS := $(BOLD)$(GREEN)
# the quality gate lower threshold for unit test total % coverage (by function statements)
COVERAGE_THRESHOLD := 65
COVERAGE_THRESHOLD := 62
# CI cache busting values; change these if you want CI to not use previous stored cache
INTEGRATION_CACHE_BUSTER="88738d2f"
CLI_CACHE_BUSTER="789bacdf"
CLI_CACHE_BUSTER="9a2c03cf"
BOOTSTRAP_CACHE="c7afb99ad"
## Build variables
@@ -204,7 +204,9 @@ $(SNAPSHOTDIR): ## Build snapshot release binaries and packages
cat .goreleaser.yaml >> $(TEMPDIR)/goreleaser.yaml
# build release snapshots
# DOCKER_CLI_EXPERIMENTAL needed to support multi architecture builds for goreleaser
BUILD_GIT_TREE_STATE=$(GITTREESTATE) \
DOCKER_CLI_EXPERIMENTAL=enabled \
$(TEMPDIR)/goreleaser release --skip-publish --skip-sign --rm-dist --snapshot --config $(TEMPDIR)/goreleaser.yaml
# note: we cannot clean the snapshot directory since the pipeline builds the snapshot separately
@@ -250,7 +252,7 @@ cli: $(SNAPSHOTDIR) ## Run CLI tests
SYFT_BINARY_LOCATION='$(SNAPSHOT_CMD)' \
go test -count=1 -v ./test/cli
.PHONY: changlog-release
.PHONY: changelog-release
changelog-release:
$(TEMPDIR)/chronicle --since-tag $(SECOND_TO_LAST_TAG) --until-tag $(LAST_TAG) -vv > CHANGELOG.md
@@ -285,9 +287,11 @@ release: clean-dist changelog-release ## Build and publish final binaries and pa
cat .goreleaser.yaml >> $(TEMPDIR)/goreleaser.yaml
# release (note the version transformation from v0.7.0 --> 0.7.0)
# DOCKER_CLI_EXPERIMENTAL needed to support multi architecture builds for goreleaser
bash -c "\
BUILD_GIT_TREE_STATE=$(GITTREESTATE) \
VERSION=$(VERSION:v%=%) \
DOCKER_CLI_EXPERIMENTAL=enabled \
$(TEMPDIR)/goreleaser \
--rm-dist \
--config $(TEMPDIR)/goreleaser.yaml \
+86 -7
View File
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://user-images.githubusercontent.com/5199289/136844524-1527b09f-c5cb-4aa9-be54-5aa92a6086c1.png" width="271">
<img src="https://user-images.githubusercontent.com/5199289/136844524-1527b09f-c5cb-4aa9-be54-5aa92a6086c1.png" width="271" alt="Cute pink owl syft logo">
</p>
[![Validations](https://github.com/anchore/syft/actions/workflows/validations.yaml/badge.svg)](https://github.com/anchore/syft/actions/workflows/validations.yaml)
@@ -9,11 +9,7 @@
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/anchore/syft/blob/main/LICENSE)
[![Slack Invite](https://img.shields.io/badge/Slack-Join-blue?logo=slack)](https://anchore.com/slack)
A CLI tool and go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner tool like [Grype](https://github.com/anchore/grype).
**Announcement:** Join us on October 20, 11am-noon PT for a webinar where we’ll discuss Anchore’s open source and enterprise solutions. [Save your seat!](https://get.anchore.com/anchore-open-source-vs-enterprise/)
---
A CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner tool like [Grype](https://github.com/anchore/grype).
![syft-demo](https://user-images.githubusercontent.com/590471/90277200-2a253000-de33-11ea-893f-32c219eea11a.gif)
@@ -87,6 +83,7 @@ docker-archive:path/to/yourimage.tar use a tarball from disk for archives crea
oci-archive:path/to/yourimage.tar use a tarball from disk for OCI archives (from Skopeo or otherwise)
oci-dir:path/to/yourimage read directly from a path on disk for OCI layout directories (from Skopeo or otherwise)
dir:path/to/yourproject read directly from a path on disk (any directory)
file:path/to/yourproject/file read directly from a path on disk (any single file)
registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
```
@@ -98,7 +95,7 @@ The output format for Syft is configurable as well:
syft packages <image> -o <format>
```
Where the `format`s available are:
Where the `formats` available are:
- `json`: Use this to get as much information out of Syft as possible!
- `text`: A row-oriented, human-and-machine-friendly output.
- `cyclonedx`: A XML report conforming to the [CycloneDX 1.2 specification](https://cyclonedx.org/specification/overview/).
@@ -106,6 +103,88 @@ Where the `format`s available are:
- `spdx-json`: A JSON report conforming to the [SPDX 2.2 JSON Schema](https://github.com/spdx/spdx-spec/blob/v2.2/schemas/spdx-schema.json).
- `table`: A columnar summary (default).
## Private Registry Authentication
### Local Docker Credentials
When a container runtime is not present, Syft can still utilize credentials configured in common credential sources (such as `~/.docker/config.json`).
It will pull images from private registries using these credentials. The config file is where your credentials are stored when authenticating with private registries via some command like `docker login`.
For more information see the `go-containerregistry` [documentation](https://github.com/google/go-containerregistry/tree/main/pkg/authn).
An example `config.json` looks something like this:
```
// config.json
{
"auths": {
"registry.example.com": {
"username": "AzureDiamond",
"password": "hunter2"
}
}
}
```
You can run the following command as an example. It details the mount/environment configuration a container needs to access a private registry:
`docker run -v ./config.json:/config/config.json -e "DOCKER_CONFIG=/config" anchore/syft:latest <private_image>`
### Docker Credentials in Kubernetes
The below section shows a simple workflow on how to mount this config file as a secret into a container on kubernetes.
1. Create a secret. The value of `config.json` is important. It refers to the specification detailed [here](https://github.com/google/go-containerregistry/tree/main/pkg/authn#the-config-file).
Below this section is the `secret.yaml` file that the pod configuration will consume as a volume.
The key `config.json` is important. It will end up being the name of the file when mounted into the pod.
```
# secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: registry-config
namespace: syft
data:
config.json: <base64 encoded config.json>
```
`kubectl apply -f secret.yaml`
2. Create your pod running syft. The env `DOCKER_CONFIG` is important because it advertises where to look for the credential file.
In the below example, setting `DOCKER_CONFIG=/config` informs syft that credentials can be found at `/config/config.json`.
This is why we used `config.json` as the key for our secret. When mounted into containers the secrets' key is used as the filename.
The `volumeMounts` section mounts our secret to `/config`. The `volumes` section names our volume and leverages the secret we created in step one.
```
# pod.yaml
apiVersion: v1
kind: Pod
spec:
containers:
- image: anchore/syft:latest
name: syft-private-registry-demo
env:
- name: DOCKER_CONFIG
value: /config
volumeMounts:
- mountPath: /config
name: registry-config
readOnly: true
args:
- <private_image>
volumes:
- name: registry-config
secret:
secretName: registry-config
```
`kubectl apply -f pod.yaml`
3. The user can now run `kubectl logs syft-private-registry-demo`. The logs should show the syft analysis for the `<private_image>` provided in the pod configuration.
Using the above information, users should be able to configure private registry access without having to do so in the `grype` or `syft` configuration files.
They will also not be dependent on a docker daemon, (or some other runtime software) for registry configuration and access.
## Configuration
Configuration search paths:
+2 -3
View File
@@ -14,9 +14,8 @@ import (
// completionCmd represents the completion command
var completionCmd = &cobra.Command{
Hidden: true,
Use: "completion [bash|zsh|fish]",
Short: "Generate a shell completion for Syft (listing local docker images)",
Use: "completion [bash|zsh|fish]",
Short: "Generate a shell completion for Syft (listing local docker images)",
Long: `To load completions (docker image list):
Bash:
+3 -1
View File
@@ -44,6 +44,9 @@ func eventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *
if err := subscription.Unsubscribe(); err != nil {
retErr = multierror.Append(retErr, err)
}
// the worker has exited, we may have been mid-handling events for the UI which should now be
// ignored, in which case forcing a teardown of the UI irregardless of the state is required.
forceTeardown = true
}
case e, isOpen := <-events:
if !isOpen {
@@ -53,7 +56,6 @@ func eventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *
if err := ux.Handle(e); err != nil {
if errors.Is(err, partybus.ErrUnsubscribe) {
log.Warnf("unable to unsubscribe from the event bus")
events = nil
} else {
retErr = multierror.Append(retErr, err)
+25 -15
View File
@@ -6,19 +6,19 @@ import (
"io/ioutil"
"os"
"github.com/anchore/syft/syft/format"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/anchore"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/formats"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/format"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/presenter/packages"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/pkg/profile"
"github.com/spf13/cobra"
@@ -45,6 +45,7 @@ const (
{{.appName}} {{.command}} oci-archive:path/to/yourimage.tar use a tarball from disk for OCI archives (from Skopeo or otherwise)
{{.appName}} {{.command}} oci-dir:path/to/yourimage read directly from a path on disk for OCI layout directories (from Skopeo or otherwise)
{{.appName}} {{.command}} dir:path/to/yourproject read directly from a path on disk (any directory)
{{.appName}} {{.command}} file:path/to/yourproject/file read directly from a path on disk (any single file)
{{.appName}} {{.command}} registry:yourrepo/yourimage:tag pull image directly from a registry (no container runtime required)
`
)
@@ -103,7 +104,7 @@ func setPackageFlags(flags *pflag.FlagSet) {
flags.StringP(
"output", "o", string(format.TableOption),
fmt.Sprintf("report output formatter, options=%v", format.AllPresenters),
fmt.Sprintf("report output formatter, options=%v", format.AllOptions),
)
flags.StringP(
@@ -239,6 +240,12 @@ func packagesExecWorker(userInput string) <-chan error {
go func() {
defer close(errs)
f := formats.ByOption(packagesPresenterOpt)
if f == nil {
errs <- fmt.Errorf("unknown format: %s", packagesPresenterOpt)
return
}
checkForApplicationUpdate()
src, cleanup, err := source.New(userInput, appConfig.Registry.ToOptions())
@@ -248,33 +255,37 @@ func packagesExecWorker(userInput string) <-chan error {
}
defer cleanup()
catalog, d, err := syft.CatalogPackages(src, appConfig.Package.Cataloger.ScopeOpt)
catalog, relationships, d, err := syft.CatalogPackages(src, appConfig.Package.Cataloger.ScopeOpt)
if err != nil {
errs <- fmt.Errorf("failed to catalog input: %w", err)
return
}
if appConfig.Anchore.Host != "" {
if err := runPackageSbomUpload(src, src.Metadata, catalog, d, appConfig.Package.Cataloger.ScopeOpt); err != nil {
if err := runPackageSbomUpload(src, src.Metadata, catalog, d); err != nil {
errs <- err
return
}
}
bus.Publish(partybus.Event{
Type: event.PresenterReady,
Value: packages.Presenter(packagesPresenterOpt, packages.PresenterConfig{
SourceMetadata: src.Metadata,
Catalog: catalog,
sbomResult := sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: catalog,
Distro: d,
Scope: appConfig.Package.Cataloger.ScopeOpt,
}),
},
Relationships: relationships,
Source: src.Metadata,
}
bus.Publish(partybus.Event{
Type: event.PresenterReady,
Value: f.Presenter(sbomResult),
})
}()
return errs
}
func runPackageSbomUpload(src *source.Source, s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, scope source.Scope) error {
func runPackageSbomUpload(src *source.Source, s source.Metadata, catalog *pkg.Catalog, d *distro.Distro) error {
log.Infof("uploading results to %s", appConfig.Anchore.Host)
if src.Metadata.Scheme != source.ImageScheme {
@@ -314,7 +325,6 @@ func runPackageSbomUpload(src *source.Source, s source.Metadata, catalog *pkg.Ca
Distro: d,
Dockerfile: dockerfileContents,
OverwriteExistingUpload: appConfig.Anchore.OverwriteExistingImage,
Scope: scope,
Timeout: appConfig.Anchore.ImportTimeout,
}
+50 -15
View File
@@ -4,6 +4,10 @@ import (
"fmt"
"sync"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
@@ -86,7 +90,6 @@ func powerUserExec(_ *cobra.Command, args []string) error {
ui.Select(isVerbose(), appConfig.Quiet, reporter)...,
)
}
func powerUserExecWorker(userInput string) <-chan error {
errs := make(chan error)
go func() {
@@ -107,29 +110,61 @@ func powerUserExecWorker(userInput string) <-chan error {
}
defer cleanup()
analysisResults := poweruser.JSONDocumentConfig{
SourceMetadata: src.Metadata,
ApplicationConfig: *appConfig,
s := sbom.SBOM{
Source: src.Metadata,
}
wg := &sync.WaitGroup{}
var results []<-chan artifact.Relationship
for _, task := range tasks {
wg.Add(1)
go func(task powerUserTask) {
defer wg.Done()
if err = task(&analysisResults, src); err != nil {
errs <- err
return
}
}(task)
c := make(chan artifact.Relationship)
results = append(results, c)
go runTask(task, &s.Artifacts, src, c, errs)
}
wg.Wait()
for relationship := range mergeResults(results...) {
s.Relationships = append(s.Relationships, relationship)
}
bus.Publish(partybus.Event{
Type: event.PresenterReady,
Value: poweruser.NewJSONPresenter(analysisResults),
Value: poweruser.NewJSONPresenter(s, *appConfig),
})
}()
return errs
}
func runTask(t powerUserTask, a *sbom.Artifacts, src *source.Source, c chan<- artifact.Relationship, errs chan<- error) {
defer close(c)
relationships, err := t(a, src)
if err != nil {
errs <- err
return
}
for _, relationship := range relationships {
c <- relationship
}
}
func mergeResults(cs ...<-chan artifact.Relationship) <-chan artifact.Relationship {
var wg sync.WaitGroup
var results = make(chan artifact.Relationship)
wg.Add(len(cs))
for _, c := range cs {
go func(c <-chan artifact.Relationship) {
for n := range c {
results <- n
}
wg.Done()
}(c)
}
go func() {
wg.Wait()
close(results)
}()
return results
}
+29 -26
View File
@@ -4,13 +4,16 @@ import (
"crypto"
"fmt"
"github.com/anchore/syft/internal/presenter/poweruser"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/source"
)
type powerUserTask func(*poweruser.JSONDocumentConfig, *source.Source) error
type powerUserTask func(*sbom.Artifacts, *source.Source) ([]artifact.Relationship, error)
func powerUserTasks() ([]powerUserTask, error) {
var tasks []powerUserTask
@@ -42,16 +45,16 @@ func catalogPackagesTask() (powerUserTask, error) {
return nil, nil
}
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
packageCatalog, theDistro, err := syft.CatalogPackages(src, appConfig.Package.Cataloger.ScopeOpt)
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
packageCatalog, relationships, theDistro, err := syft.CatalogPackages(src, appConfig.Package.Cataloger.ScopeOpt)
if err != nil {
return err
return nil, err
}
results.PackageCatalog = packageCatalog
results.Distro = theDistro
return nil
return relationships, nil
}
return task, nil
@@ -64,18 +67,18 @@ func catalogFileMetadataTask() (powerUserTask, error) {
metadataCataloger := file.NewMetadataCataloger()
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return err
return nil, err
}
result, err := metadataCataloger.Catalog(resolver)
if err != nil {
return err
return nil, err
}
results.FileMetadata = result
return nil
return nil, nil
}
return task, nil
@@ -110,18 +113,18 @@ func catalogFileDigestsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return err
return nil, err
}
result, err := digestsCataloger.Catalog(resolver)
if err != nil {
return err
return nil, err
}
results.FileDigests = result
return nil
return nil, nil
}
return task, nil
@@ -142,18 +145,18 @@ func catalogSecretsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.Secrets.Cataloger.ScopeOpt)
if err != nil {
return err
return nil, err
}
result, err := secretsCataloger.Catalog(resolver)
if err != nil {
return err
return nil, err
}
results.Secrets = result
return nil
return nil, nil
}
return task, nil
@@ -170,18 +173,18 @@ func catalogFileClassificationsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileClassification.Cataloger.ScopeOpt)
if err != nil {
return err
return nil, err
}
result, err := classifierCataloger.Catalog(resolver)
if err != nil {
return err
return nil, err
}
results.FileClassifications = result
return nil
return nil, nil
}
return task, nil
@@ -197,18 +200,18 @@ func catalogContentsTask() (powerUserTask, error) {
return nil, err
}
task := func(results *poweruser.JSONDocumentConfig, src *source.Source) error {
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileContents.Cataloger.ScopeOpt)
if err != nil {
return err
return nil, err
}
result, err := contentsCataloger.Catalog(resolver)
if err != nil {
return err
return nil, err
}
results.FileContents = result
return nil
return nil, nil
}
return task, nil
+4 -1
View File
@@ -7,11 +7,12 @@ require (
github.com/adrg/xdg v0.2.1
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf
github.com/anchore/go-presenter v0.0.0-20211102174526-0dbf20f6c7fa
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29
github.com/anchore/stereoscope v0.0.0-20211021140357-9f8395cd95af
github.com/anchore/stereoscope v0.0.0-20211024152658-003132a67c10
github.com/antihax/optional v1.0.0
github.com/bmatcuk/doublestar/v2 v2.0.4
github.com/docker/docker v17.12.0-ce-rc1.0.20200309214505-aa6a9891b09c+incompatible
@@ -23,7 +24,9 @@ require (
github.com/gookit/color v1.2.7
github.com/hashicorp/go-multierror v1.1.0
github.com/hashicorp/go-version v1.2.0
github.com/jinzhu/copier v0.3.2
github.com/mitchellh/go-homedir v1.1.0
github.com/mitchellh/hashstructure/v2 v2.0.2
github.com/mitchellh/mapstructure v1.3.1
github.com/olekukonko/tablewriter v0.0.4
github.com/pelletier/go-toml v1.8.1
+8 -2
View File
@@ -108,6 +108,8 @@ github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRF
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf/go.mod h1:FaODhIA06mxO1E6R32JE0TL1JWZZkmjRIAd4ULvHUKk=
github.com/anchore/go-presenter v0.0.0-20211102174526-0dbf20f6c7fa h1:mDLUAkgXsV5Z8D0EEj8eS6FBekolV/A+Xxbs9054bPw=
github.com/anchore/go-presenter v0.0.0-20211102174526-0dbf20f6c7fa/go.mod h1:29jwxTSAS6pBcrmuwf1U3r1Tqp1o1XpuiOJ0NT9NoGg=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63 h1:C9W/LAydEz/qdUhx1MdjO9l8NEcFKYknkxDVyo9LAoM=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63/go.mod h1:6qH8c6U/3CBVvDDDBZnPSTbTINq3cIdADUYTaVf75EM=
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04 h1:VzprUTpc0vW0nnNKJfJieyH/TZ9UYAnTZs5/gHTdAe8=
@@ -116,8 +118,8 @@ github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZV
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29 h1:K9LfnxwhqvihqU0+MF325FNy7fsKV9EGaUxdfR4gnWk=
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29/go.mod h1:Oc1UkGaJwY6ND6vtAqPSlYrptKRJngHwkwB6W7l1uP0=
github.com/anchore/stereoscope v0.0.0-20211021140357-9f8395cd95af h1:0w24yOOJx6DiaLvptAfp/Mro2cQkEcR7P1WRbL2apHw=
github.com/anchore/stereoscope v0.0.0-20211021140357-9f8395cd95af/go.mod h1:Rqltg0KqVKoDy4kCuJMLPFDfUg7pEmSqUEA9pOO1L7c=
github.com/anchore/stereoscope v0.0.0-20211024152658-003132a67c10 h1:BmK/CgNlu+X9foWK2ZAIehxzYws760AZSGVNamQZpiw=
github.com/anchore/stereoscope v0.0.0-20211024152658-003132a67c10/go.mod h1:Rqltg0KqVKoDy4kCuJMLPFDfUg7pEmSqUEA9pOO1L7c=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/antihax/optional v1.0.0 h1:xK2lYat7ZLaVVcIuj82J8kIro4V6kDe0AUDFboUCwcg=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
@@ -452,6 +454,8 @@ github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NH
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/jarcoal/httpmock v1.0.5/go.mod h1:ATjnClrvW/3tijVmpL/va5Z3aAyGvqU3gCT8nX0Txik=
github.com/jingyugao/rowserrcheck v0.0.0-20191204022205-72ab7603b68a/go.mod h1:xRskid8CManxVta/ALEhJha/pweKBaVG6fWgc0yH25s=
github.com/jinzhu/copier v0.3.2 h1:QdBOCbaouLDYaIPFfi1bKv5F5tPpeTwXe4sD0jqtz5w=
github.com/jinzhu/copier v0.3.2/go.mod h1:24xnZezI2Yqac9J61UC6/dG/k76ttpq0DdJI3QmUvro=
github.com/jirfag/go-printf-func-name v0.0.0-20191110105641-45db9963cdd3/go.mod h1:HEWGJkRDzjJY2sqdDwxccsGicWEf9BQOZsq2tV+xzM0=
github.com/jirfag/go-printf-func-name v0.0.0-20200119135958-7558a9eaa5af/go.mod h1:HEWGJkRDzjJY2sqdDwxccsGicWEf9BQOZsq2tV+xzM0=
github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
@@ -542,6 +546,8 @@ github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrk
github.com/mitchellh/go-ps v0.0.0-20190716172923-621e5597135b/go.mod h1:r1VsdOzOPt1ZSrGZWFoNhsAedKnEd6r9Np1+5blZCWk=
github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=
github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=
github.com/mitchellh/hashstructure/v2 v2.0.2 h1:vGKWl0YJqUNxE8d+h8f6NJLcCJrgbhC4NcD46KavDd4=
github.com/mitchellh/hashstructure/v2 v2.0.2/go.mod h1:MG3aRVU/N29oo/V/IhBX8GR/zz4kQkprJgF2EVszyDE=
github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=
github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
+1 -2
View File
@@ -26,7 +26,6 @@ type ImportConfig struct {
Distro *distro.Distro
Dockerfile []byte
OverwriteExistingUpload bool
Scope source.Scope
Timeout uint
}
@@ -74,7 +73,7 @@ func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
prog.N++
sessionID := startOperation.Uuid
packageDigest, err := importPackageSBOM(authedCtx, c.client.ImportsApi, sessionID, cfg.SourceMetadata, cfg.Catalog, cfg.Distro, cfg.Scope, stage)
packageDigest, err := importPackageSBOM(authedCtx, c.client.ImportsApi, sessionID, cfg.SourceMetadata, cfg.Catalog, cfg.Distro, stage)
if err != nil {
return fmt.Errorf("failed to import Package SBOM: %w", err)
}
+15 -4
View File
@@ -8,6 +8,8 @@ import (
"fmt"
"net/http"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/wagoodman/go-progress"
@@ -24,10 +26,19 @@ type packageSBOMImportAPI interface {
ImportImagePackages(context.Context, string, external.ImagePackageManifest) (external.ImageImportContentResponse, *http.Response, error)
}
func packageSbomModel(s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, scope source.Scope) (*external.ImagePackageManifest, error) {
func packageSbomModel(srcMetadata source.Metadata, catalog *pkg.Catalog, d *distro.Distro) (*external.ImagePackageManifest, error) {
var buf bytes.Buffer
err := syftjson.Format().Presenter(catalog, &s, d, scope).Present(&buf)
// TODO: once the top-level API is refactored and SBOMs are the unit of work, then this function will be passed an SBOM and there would be no more need to create an SBOM object here.
s := sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: catalog,
Distro: d,
},
Source: srcMetadata,
}
err := syftjson.Format().Presenter(s).Present(&buf)
if err != nil {
return nil, fmt.Errorf("unable to serialize results: %w", err)
}
@@ -41,11 +52,11 @@ func packageSbomModel(s source.Metadata, catalog *pkg.Catalog, d *distro.Distro,
return &model, nil
}
func importPackageSBOM(ctx context.Context, api packageSBOMImportAPI, sessionID string, s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, scope source.Scope, stage *progress.Stage) (string, error) {
func importPackageSBOM(ctx context.Context, api packageSBOMImportAPI, sessionID string, s source.Metadata, catalog *pkg.Catalog, d *distro.Distro, stage *progress.Stage) (string, error) {
log.Debug("importing package SBOM")
stage.Current = "package SBOM"
model, err := packageSbomModel(s, catalog, d, scope)
model, err := packageSbomModel(s, catalog, d)
if err != nil {
return "", fmt.Errorf("unable to create PackageSBOM model: %w", err)
}
+14 -4
View File
@@ -9,6 +9,8 @@ import (
"strings"
"testing"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/formats/syftjson"
syftjsonModel "github.com/anchore/syft/internal/formats/syftjson/model"
@@ -72,7 +74,7 @@ func TestPackageSbomToModel(t *testing.T) {
c := pkg.NewCatalog(p)
model, err := packageSbomModel(m, c, &d, source.AllLayersScope)
model, err := packageSbomModel(m, c, &d)
if err != nil {
t.Fatalf("unable to generate model from source material: %+v", err)
}
@@ -84,8 +86,16 @@ func TestPackageSbomToModel(t *testing.T) {
t.Fatalf("unable to marshal model: %+v", err)
}
s := sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: c,
Distro: &d,
},
Source: m,
}
var buf bytes.Buffer
pres := syftjson.Format().Presenter(c, &m, &d, source.AllLayersScope)
pres := syftjson.Format().Presenter(s)
if err := pres.Present(&buf); err != nil {
t.Fatalf("unable to get expected json: %+v", err)
}
@@ -187,7 +197,7 @@ func TestPackageSbomImport(t *testing.T) {
d, _ := distro.NewDistro(distro.CentOS, "8.0", "")
theModel, err := packageSbomModel(m, catalog, &d, source.AllLayersScope)
theModel, err := packageSbomModel(m, catalog, &d)
if err != nil {
t.Fatalf("could not get sbom model: %+v", err)
}
@@ -226,7 +236,7 @@ func TestPackageSbomImport(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
digest, err := importPackageSBOM(context.TODO(), test.api, sessionID, m, catalog, &d, source.AllLayersScope, &progress.Stage{})
digest, err := importPackageSBOM(context.TODO(), test.api, sessionID, m, catalog, &d, &progress.Stage{})
// validate error handling
if err != nil && !test.expectsError {
@@ -0,0 +1,19 @@
package spdxhelpers
import "github.com/anchore/syft/syft/pkg"
func Description(p pkg.Package) string {
if hasMetadata(p) {
switch metadata := p.Metadata.(type) {
case pkg.ApkMetadata:
return metadata.Description
case pkg.NpmPackageJSONMetadata:
return metadata.Description
}
}
return ""
}
func hasMetadata(p pkg.Package) bool {
return p.Metadata != nil
}
@@ -0,0 +1,56 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_Description(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected string
}{
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "no metadata",
input: pkg.Package{},
expected: "",
},
{
name: "from apk",
input: pkg.Package{
Metadata: pkg.ApkMetadata{
Description: "a description!",
},
},
expected: "a description!",
},
{
name: "from npm",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Description: "a description!",
},
},
expected: "a description!",
},
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "empty",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Homepage: "",
},
},
expected: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, Description(test.input))
})
}
}
@@ -0,0 +1,32 @@
package spdxhelpers
import (
"path"
"strings"
"github.com/anchore/syft/syft/source"
"github.com/google/uuid"
)
func DocumentName(srcMetadata source.Metadata) string {
switch srcMetadata.Scheme {
case source.ImageScheme:
return cleanSPDXName(srcMetadata.ImageMetadata.UserInput)
case source.DirectoryScheme:
return cleanSPDXName(srcMetadata.Path)
}
// TODO: is this alright?
return uuid.Must(uuid.NewRandom()).String()
}
func cleanSPDXName(name string) string {
// remove # according to specification
name = strings.ReplaceAll(name, "#", "-")
// remove : for url construction
name = strings.ReplaceAll(name, ":", "-")
// clean relative pathing
return path.Clean(name)
}
@@ -0,0 +1,29 @@
package spdxhelpers
import (
"fmt"
"path"
"github.com/anchore/syft/syft/source"
"github.com/google/uuid"
)
const SyftDocumentNamespace = "https://anchore.com/syft"
func DocumentNamespace(name string, srcMetadata source.Metadata) string {
input := "unknown-source-type"
switch srcMetadata.Scheme {
case source.ImageScheme:
input = "image"
case source.DirectoryScheme:
input = "dir"
}
uniqueID := uuid.Must(uuid.NewRandom())
identifier := path.Join(input, uniqueID.String())
if name != "." {
identifier = path.Join(input, fmt.Sprintf("%s-%s", name, uniqueID.String()))
}
return path.Join(SyftDocumentNamespace, identifier)
}
@@ -0,0 +1,23 @@
package spdxhelpers
import "github.com/anchore/syft/syft/pkg"
func DownloadLocation(p pkg.Package) string {
// 3.7: Package Download Location
// Cardinality: mandatory, one
// NONE if there is no download location whatsoever.
// NOASSERTION if:
// (i) the SPDX file creator has attempted to but cannot reach a reasonable objective determination;
// (ii) the SPDX file creator has made no attempt to determine this field; or
// (iii) the SPDX file creator has intentionally provided no information (no meaning should be implied by doing so).
if hasMetadata(p) {
switch metadata := p.Metadata.(type) {
case pkg.ApkMetadata:
return NoneIfEmpty(metadata.URL)
case pkg.NpmPackageJSONMetadata:
return NoneIfEmpty(metadata.URL)
}
}
return "NOASSERTION"
}
@@ -0,0 +1,54 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_DownloadLocation(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected string
}{
{
name: "no metadata",
input: pkg.Package{},
expected: "NOASSERTION",
},
{
name: "from apk",
input: pkg.Package{
Metadata: pkg.ApkMetadata{
URL: "http://a-place.gov",
},
},
expected: "http://a-place.gov",
},
{
name: "from npm",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
URL: "http://a-place.gov",
},
},
expected: "http://a-place.gov",
},
{
name: "empty",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
URL: "",
},
},
expected: "NONE",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, DownloadLocation(test.input))
})
}
}
@@ -0,0 +1,51 @@
package spdxhelpers
import (
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/pkg"
)
func ExternalRefs(p pkg.Package) (externalRefs []model.ExternalRef) {
externalRefs = make([]model.ExternalRef, 0)
for _, c := range p.CPEs {
externalRefs = append(externalRefs, model.ExternalRef{
ReferenceCategory: model.SecurityReferenceCategory,
ReferenceLocator: c.BindToFmtString(),
ReferenceType: model.Cpe23ExternalRefType,
})
}
if p.PURL != "" {
externalRefs = append(externalRefs, model.ExternalRef{
ReferenceCategory: model.PackageManagerReferenceCategory,
ReferenceLocator: p.PURL,
ReferenceType: model.PurlExternalRefType,
})
}
return externalRefs
}
func ExtractPURL(refs []model.ExternalRef) string {
for _, r := range refs {
if r.ReferenceType == model.PurlExternalRefType {
return r.ReferenceLocator
}
}
return ""
}
func ExtractCPEs(refs []model.ExternalRef) (cpes []pkg.CPE) {
for _, r := range refs {
if r.ReferenceType == model.Cpe23ExternalRefType {
cpe, err := pkg.NewCPE(r.ReferenceLocator)
if err != nil {
log.Warnf("unable to extract SPDX CPE=%q: %+v", r.ReferenceLocator, err)
continue
}
cpes = append(cpes, cpe)
}
}
return cpes
}
@@ -0,0 +1,45 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_ExternalRefs(t *testing.T) {
testCPE := pkg.MustCPE("cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*")
tests := []struct {
name string
input pkg.Package
expected []model.ExternalRef
}{
{
name: "cpe + purl",
input: pkg.Package{
CPEs: []pkg.CPE{
testCPE,
},
PURL: "a-purl",
},
expected: []model.ExternalRef{
{
ReferenceCategory: model.SecurityReferenceCategory,
ReferenceLocator: testCPE.BindToFmtString(),
ReferenceType: model.Cpe23ExternalRefType,
},
{
ReferenceCategory: model.PackageManagerReferenceCategory,
ReferenceLocator: "a-purl",
ReferenceType: model.PurlExternalRefType,
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.ElementsMatch(t, test.expected, ExternalRefs(test.input))
})
}
}
@@ -0,0 +1,51 @@
package spdxhelpers
import (
"crypto/sha256"
"fmt"
"path/filepath"
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/syft/pkg"
)
func Files(packageSpdxID string, p pkg.Package) (files []model.File, fileIDs []string, relationships []model.Relationship) {
files = make([]model.File, 0)
fileIDs = make([]string, 0)
relationships = make([]model.Relationship, 0)
if !hasMetadata(p) {
return files, fileIDs, relationships
}
pkgFileOwner, ok := p.Metadata.(pkg.FileOwner)
if !ok {
return files, fileIDs, relationships
}
for _, ownedFilePath := range pkgFileOwner.OwnedFiles() {
baseFileName := filepath.Base(ownedFilePath)
pathHash := sha256.Sum256([]byte(ownedFilePath))
fileSpdxID := model.ElementID(fmt.Sprintf("File-%s-%x", p.Name, pathHash)).String()
fileIDs = append(fileIDs, fileSpdxID)
files = append(files, model.File{
FileName: ownedFilePath,
Item: model.Item{
Element: model.Element{
SPDXID: fileSpdxID,
Name: baseFileName,
},
},
})
relationships = append(relationships, model.Relationship{
SpdxElementID: packageSpdxID,
RelationshipType: model.ContainsRelationship,
RelatedSpdxElement: fileSpdxID,
})
}
return files, fileIDs, relationships
}
@@ -0,0 +1,15 @@
package spdxhelpers
import "github.com/anchore/syft/syft/pkg"
func Homepage(p pkg.Package) string {
if hasMetadata(p) {
switch metadata := p.Metadata.(type) {
case pkg.GemMetadata:
return metadata.Homepage
case pkg.NpmPackageJSONMetadata:
return metadata.Homepage
}
}
return ""
}
@@ -0,0 +1,56 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_Homepage(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected string
}{
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "no metadata",
input: pkg.Package{},
expected: "",
},
{
name: "from gem",
input: pkg.Package{
Metadata: pkg.GemMetadata{
Homepage: "http://a-place.gov",
},
},
expected: "http://a-place.gov",
},
{
name: "from npm",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Homepage: "http://a-place.gov",
},
},
expected: "http://a-place.gov",
},
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "empty",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Homepage: "",
},
},
expected: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, Homepage(test.input))
})
}
}
@@ -0,0 +1,37 @@
package spdxhelpers
import (
"strings"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/pkg"
)
func License(p pkg.Package) string {
// source: https://spdx.github.io/spdx-spec/3-package-information/#313-concluded-license
// The options to populate this field are limited to:
// A valid SPDX License Expression as defined in Appendix IV;
// NONE, if the SPDX file creator concludes there is no license available for this package; or
// NOASSERTION if:
// (i) the SPDX file creator has attempted to but cannot reach a reasonable objective determination;
// (ii) the SPDX file creator has made no attempt to determine this field; or
// (iii) the SPDX file creator has intentionally provided no information (no meaning should be implied by doing so).
if len(p.Licenses) == 0 {
return "NONE"
}
// take all licenses and assume an AND expression; for information about license expressions see https://spdx.github.io/spdx-spec/appendix-IV-SPDX-license-expressions/
var parsedLicenses []string
for _, l := range p.Licenses {
if value, exists := spdxlicense.ID(l); exists {
parsedLicenses = append(parsedLicenses, value)
}
}
if len(parsedLicenses) == 0 {
return "NOASSERTION"
}
return strings.Join(parsedLicenses, " AND ")
}
@@ -0,0 +1,73 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_License(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected string
}{
{
name: "no licenses",
input: pkg.Package{},
expected: "NONE",
},
{
name: "no SPDX licenses",
input: pkg.Package{
Licenses: []string{
"made-up",
},
},
expected: "NOASSERTION",
},
{
name: "with SPDX license",
input: pkg.Package{
Licenses: []string{
"MIT",
},
},
expected: "MIT",
},
{
name: "with SPDX license expression",
input: pkg.Package{
Licenses: []string{
"MIT",
"GPL-3.0",
},
},
expected: "MIT AND GPL-3.0",
},
{
name: "cap insensitive",
input: pkg.Package{
Licenses: []string{
"gpl-3.0",
},
},
expected: "GPL-3.0",
},
{
name: "debian to spdx conversion",
input: pkg.Package{
Licenses: []string{
"GPL-2",
},
},
expected: "GPL-2.0",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, License(test.input))
})
}
}
@@ -0,0 +1,12 @@
package spdxhelpers
import (
"strings"
)
func NoneIfEmpty(value string) string {
if strings.TrimSpace(value) == "" {
return "NONE"
}
return value
}
@@ -0,0 +1,41 @@
package spdxhelpers
import (
"testing"
"github.com/stretchr/testify/assert"
)
func Test_noneIfEmpty(t *testing.T) {
tests := []struct {
name string
value string
expected string
}{
{
name: "non-zero value",
value: "something",
expected: "something",
},
{
name: "empty",
value: "",
expected: "NONE",
},
{
name: "space",
value: " ",
expected: "NONE",
},
{
name: "tab",
value: "\t",
expected: "NONE",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, NoneIfEmpty(test.value))
})
}
}
@@ -0,0 +1,114 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func Test_Originator(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected string
}{
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "no metadata",
input: pkg.Package{},
expected: "",
},
{
name: "from gem",
input: pkg.Package{
Metadata: pkg.GemMetadata{
Authors: []string{
"auth1",
"auth2",
},
},
},
expected: "auth1",
},
{
name: "from npm",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Author: "auth",
},
},
expected: "auth",
},
{
name: "from apk",
input: pkg.Package{
Metadata: pkg.ApkMetadata{
Maintainer: "auth",
},
},
expected: "auth",
},
{
name: "from python - just name",
input: pkg.Package{
Metadata: pkg.PythonPackageMetadata{
Author: "auth",
},
},
expected: "auth",
},
{
name: "from python - just email",
input: pkg.Package{
Metadata: pkg.PythonPackageMetadata{
AuthorEmail: "auth@auth.gov",
},
},
expected: "auth@auth.gov",
},
{
name: "from python - both name and email",
input: pkg.Package{
Metadata: pkg.PythonPackageMetadata{
Author: "auth",
AuthorEmail: "auth@auth.gov",
},
},
expected: "auth <auth@auth.gov>",
},
{
name: "from rpm",
input: pkg.Package{
Metadata: pkg.RpmdbMetadata{
Vendor: "auth",
},
},
expected: "auth",
},
{
name: "from dpkg",
input: pkg.Package{
Metadata: pkg.DpkgMetadata{
Maintainer: "auth",
},
},
expected: "auth",
},
{
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "empty",
input: pkg.Package{
Metadata: pkg.NpmPackageJSONMetadata{
Author: "",
},
},
expected: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, Originator(test.input))
})
}
}
@@ -0,0 +1,37 @@
package spdxhelpers
import (
"fmt"
"github.com/anchore/syft/syft/pkg"
)
func Originator(p pkg.Package) string {
if hasMetadata(p) {
switch metadata := p.Metadata.(type) {
case pkg.ApkMetadata:
return metadata.Maintainer
case pkg.NpmPackageJSONMetadata:
return metadata.Author
case pkg.PythonPackageMetadata:
author := metadata.Author
if author == "" {
return metadata.AuthorEmail
}
if metadata.AuthorEmail != "" {
author += fmt.Sprintf(" <%s>", metadata.AuthorEmail)
}
return author
case pkg.GemMetadata:
if len(metadata.Authors) > 0 {
return metadata.Authors[0]
}
return ""
case pkg.RpmdbMetadata:
return metadata.Vendor
case pkg.DpkgMetadata:
return metadata.Maintainer
}
}
return ""
}
@@ -0,0 +1,39 @@
package spdxhelpers
import (
"strings"
"github.com/anchore/syft/syft/pkg"
)
func SourceInfo(p pkg.Package) string {
answer := ""
switch p.Type {
case pkg.RpmPkg:
answer = "acquired package info from RPM DB"
case pkg.ApkPkg:
answer = "acquired package info from APK DB"
case pkg.DebPkg:
answer = "acquired package info from DPKG DB"
case pkg.NpmPkg:
answer = "acquired package info from installed node module manifest file"
case pkg.PythonPkg:
answer = "acquired package info from installed python package manifest file"
case pkg.JavaPkg, pkg.JenkinsPluginPkg:
answer = "acquired package info from installed java archive"
case pkg.GemPkg:
answer = "acquired package info from installed gem metadata file"
case pkg.GoModulePkg:
answer = "acquired package info from go module information"
case pkg.RustPkg:
answer = "acquired package info from rust cargo manifest"
default:
answer = "acquired package info from the following paths"
}
var paths []string
for _, l := range p.Locations {
paths = append(paths, l.RealPath)
}
return answer + ": " + strings.Join(paths, ", ")
}
@@ -0,0 +1,141 @@
package spdxhelpers
import (
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/stretchr/testify/assert"
)
func Test_SourceInfo(t *testing.T) {
tests := []struct {
name string
input pkg.Package
expected []string
}{
{
name: "locations are captured",
input: pkg.Package{
// note: no type given
Locations: []source.Location{
{
RealPath: "/a-place",
VirtualPath: "/b-place",
},
{
RealPath: "/c-place",
VirtualPath: "/d-place",
},
},
},
expected: []string{
"from the following paths",
"/a-place",
"/c-place",
},
},
{
// note: no specific support for this
input: pkg.Package{
Type: pkg.KbPkg,
},
expected: []string{
"from the following paths",
},
},
{
input: pkg.Package{
Type: pkg.RpmPkg,
},
expected: []string{
"from RPM DB",
},
},
{
input: pkg.Package{
Type: pkg.ApkPkg,
},
expected: []string{
"from APK DB",
},
},
{
input: pkg.Package{
Type: pkg.DebPkg,
},
expected: []string{
"from DPKG DB",
},
},
{
input: pkg.Package{
Type: pkg.NpmPkg,
},
expected: []string{
"from installed node module manifest file",
},
},
{
input: pkg.Package{
Type: pkg.PythonPkg,
},
expected: []string{
"from installed python package manifest file",
},
},
{
input: pkg.Package{
Type: pkg.JavaPkg,
},
expected: []string{
"from installed java archive",
},
},
{
input: pkg.Package{
Type: pkg.JenkinsPluginPkg,
},
expected: []string{
"from installed java archive",
},
},
{
input: pkg.Package{
Type: pkg.GemPkg,
},
expected: []string{
"from installed gem metadata file",
},
},
{
input: pkg.Package{
Type: pkg.GoModulePkg,
},
expected: []string{
"from go module information",
},
},
{
input: pkg.Package{
Type: pkg.RustPkg,
},
expected: []string{
"from rust cargo manifest",
},
},
}
var pkgTypes []pkg.Type
for _, test := range tests {
t.Run(test.name+" "+string(test.input.Type), func(t *testing.T) {
if test.input.Type != "" {
pkgTypes = append(pkgTypes, test.input.Type)
}
actual := SourceInfo(test.input)
for _, expected := range test.expected {
assert.Contains(t, actual, expected)
}
})
}
assert.ElementsMatch(t, pkg.AllPkgs, pkgTypes, "missing one or more package types to test against (maybe a package type was added?)")
}
+64 -22
View File
@@ -4,13 +4,14 @@ import (
"bytes"
"testing"
"github.com/anchore/syft/syft/presenter"
"github.com/anchore/go-presenter"
"github.com/anchore/go-testutils"
"github.com/anchore/stereoscope/pkg/filetree"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/stereoscope/pkg/imagetest"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/sergi/go-diff/diffmatchpatch"
"github.com/stretchr/testify/assert"
@@ -18,6 +19,18 @@ import (
type redactor func(s []byte) []byte
type imageCfg struct {
fromSnapshot bool
}
type ImageOption func(cfg *imageCfg)
func FromSnapshot() ImageOption {
return func(cfg *imageCfg) {
cfg.fromSnapshot = true
}
}
func AssertPresenterAgainstGoldenImageSnapshot(t *testing.T, pres presenter.Presenter, testImage string, updateSnapshot bool, redactors ...redactor) {
var buffer bytes.Buffer
@@ -78,17 +91,48 @@ func AssertPresenterAgainstGoldenSnapshot(t *testing.T, pres presenter.Presenter
}
}
func ImageInput(t testing.TB, testImage string) (*pkg.Catalog, source.Metadata, *distro.Distro) {
func ImageInput(t testing.TB, testImage string, options ...ImageOption) sbom.SBOM {
t.Helper()
catalog := pkg.NewCatalog()
img := imagetest.GetGoldenFixtureImage(t, testImage)
var cfg imageCfg
var img *image.Image
for _, opt := range options {
opt(&cfg)
}
switch cfg.fromSnapshot {
case true:
img = imagetest.GetGoldenFixtureImage(t, testImage)
default:
img = imagetest.GetFixtureImage(t, "docker-archive", testImage)
}
populateImageCatalog(catalog, img)
// this is a hard coded value that is not given by the fixture helper and must be provided manually
img.Metadata.ManifestDigest = "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
src, err := source.NewFromImage(img, "user-image-input")
assert.NoError(t, err)
dist, err := distro.NewDistro(distro.Debian, "1.2.3", "like!")
assert.NoError(t, err)
return sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: catalog,
Distro: &dist,
},
Source: src.Metadata,
}
}
func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
_, ref1, _ := img.SquashedTree().File("/somefile-1.txt", filetree.FollowBasenameLinks)
_, ref2, _ := img.SquashedTree().File("/somefile-2.txt", filetree.FollowBasenameLinks)
// populate catalog with test data
catalog.Add(pkg.Package{
ID: "package-1-id",
Name: "package-1",
Version: "1.0.1",
Locations: []source.Location{
@@ -109,7 +153,6 @@ func ImageInput(t testing.TB, testImage string) (*pkg.Catalog, source.Metadata,
},
})
catalog.Add(pkg.Package{
ID: "package-2-id",
Name: "package-2",
Version: "2.0.1",
Locations: []source.Location{
@@ -127,25 +170,31 @@ func ImageInput(t testing.TB, testImage string) (*pkg.Catalog, source.Metadata,
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
})
}
// this is a hard coded value that is not given by the fixture helper and must be provided manually
img.Metadata.ManifestDigest = "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
src, err := source.NewFromImage(img, "user-image-input")
assert.NoError(t, err)
func DirectoryInput(t testing.TB) sbom.SBOM {
catalog := newDirectoryCatalog()
dist, err := distro.NewDistro(distro.Debian, "1.2.3", "like!")
assert.NoError(t, err)
return catalog, src.Metadata, &dist
src, err := source.NewFromDirectory("/some/path")
assert.NoError(t, err)
return sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: catalog,
Distro: &dist,
},
Source: src.Metadata,
}
}
func DirectoryInput(t testing.TB) (*pkg.Catalog, source.Metadata, *distro.Distro) {
func newDirectoryCatalog() *pkg.Catalog {
catalog := pkg.NewCatalog()
// populate catalog with test data
catalog.Add(pkg.Package{
ID: "package-1-id",
Name: "package-1",
Version: "1.0.1",
Type: pkg.PythonPkg,
@@ -171,7 +220,6 @@ func DirectoryInput(t testing.TB) (*pkg.Catalog, source.Metadata, *distro.Distro
},
})
catalog.Add(pkg.Package{
ID: "package-2-id",
Name: "package-2",
Version: "2.0.1",
Type: pkg.DebPkg,
@@ -190,11 +238,5 @@ func DirectoryInput(t testing.TB) (*pkg.Catalog, source.Metadata, *distro.Distro
},
})
dist, err := distro.NewDistro(distro.Debian, "1.2.3", "like!")
assert.NoError(t, err)
src, err := source.NewFromDirectory("/some/path")
assert.NoError(t, err)
return catalog, src.Metadata, &dist
return catalog
}
@@ -0,0 +1,26 @@
package cyclonedx12xml
import (
"encoding/xml"
"io"
"github.com/anchore/syft/syft/sbom"
)
func encoder(output io.Writer, s sbom.SBOM) error {
enc := xml.NewEncoder(output)
enc.Indent("", " ")
_, err := output.Write([]byte(xml.Header))
if err != nil {
return err
}
err = enc.Encode(toFormatModel(s))
if err != nil {
return err
}
_, err = output.Write([]byte("\n"))
return err
}
@@ -1,4 +1,4 @@
package packages
package cyclonedx12xml
import (
"flag"
@@ -11,9 +11,8 @@ import (
var updateCycloneDx = flag.Bool("update-cyclonedx", false, "update the *.golden files for cyclone-dx presenters")
func TestCycloneDxDirectoryPresenter(t *testing.T) {
catalog, metadata, _ := testutils.DirectoryInput(t)
testutils.AssertPresenterAgainstGoldenSnapshot(t,
NewCycloneDxPresenter(catalog, metadata),
Format().Presenter(testutils.DirectoryInput(t)),
*updateCycloneDx,
cycloneDxRedactor,
)
@@ -21,9 +20,8 @@ func TestCycloneDxDirectoryPresenter(t *testing.T) {
func TestCycloneDxImagePresenter(t *testing.T) {
testImage := "image-simple"
catalog, metadata, _ := testutils.ImageInput(t, testImage)
testutils.AssertPresenterAgainstGoldenImageSnapshot(t,
NewCycloneDxPresenter(catalog, metadata),
Format().Presenter(testutils.ImageInput(t, testImage)),
testImage,
*updateCycloneDx,
cycloneDxRedactor,
+12
View File
@@ -0,0 +1,12 @@
package cyclonedx12xml
import "github.com/anchore/syft/syft/format"
func Format() format.Format {
return format.NewFormat(
format.CycloneDxOption,
encoder,
nil,
nil,
)
}
@@ -0,0 +1,31 @@
package model
import (
"encoding/xml"
)
// Source: https://cyclonedx.org/ext/bom-descriptor/
// BomDescriptor represents all metadata surrounding the BOM report (such as when the BOM was made, with which tool, and the item being cataloged).
type BomDescriptor struct {
XMLName xml.Name `xml:"metadata"`
Timestamp string `xml:"timestamp,omitempty"` // The date and time (timestamp) when the document was created
Tools []BomDescriptorTool `xml:"tools>tool"` // The tool used to create the BOM.
Component *BomDescriptorComponent `xml:"component"` // The component that the BOM describes.
}
// BomDescriptorTool represents the tool that created the BOM report.
type BomDescriptorTool struct {
XMLName xml.Name `xml:"tool"`
Vendor string `xml:"vendor,omitempty"` // The vendor of the tool used to create the BOM.
Name string `xml:"name,omitempty"` // The name of the tool used to create the BOM.
Version string `xml:"version,omitempty"` // The version of the tool used to create the BOM.
// TODO: hashes, author, manufacture, supplier
// TODO: add user-defined fields for the remaining build/version parameters
}
// BomDescriptorComponent represents the software/package being cataloged.
type BomDescriptorComponent struct {
XMLName xml.Name `xml:"component"`
Component
}
@@ -0,0 +1,20 @@
package model
import "encoding/xml"
// Component represents a single element in the CycloneDX BOM
type Component struct {
XMLName xml.Name `xml:"component"`
Type string `xml:"type,attr"` // Required; Describes if the component is a library, framework, application, container, operating system, firmware, hardware device, or file
Supplier string `xml:"supplier,omitempty"` // The organization that supplied the component. The supplier may often be the manufacture, but may also be a distributor or repackager.
Author string `xml:"author,omitempty"` // The person(s) or organization(s) that authored the component
Publisher string `xml:"publisher,omitempty"` // The person(s) or organization(s) that published the component
Group string `xml:"group,omitempty"` // The high-level classification that a project self-describes as. This will often be a shortened, single name of the company or project that produced the component, or the source package or domain name.
Name string `xml:"name"` // Required; The name of the component as defined by the project
Version string `xml:"version"` // Required; The version of the component as defined by the project
Description string `xml:"description,omitempty"` // A description of the component
Licenses *[]License `xml:"licenses>license"` // A node describing zero or more license names, SPDX license IDs or expressions
PackageURL string `xml:"purl,omitempty"` // Specifies the package-url (PackageURL). The purl, if specified, must be valid and conform to the specification defined at: https://github.com/package-url/purl-spec
// TODO: source, hashes, copyright, cpe, purl, swid, modified, pedigree, externalReferences
// TODO: add user-defined parameters for syft-specific values (image layer index, cataloger, location path, etc.)
}
@@ -0,0 +1,17 @@
package model
import (
"encoding/xml"
)
// Source: https://github.com/CycloneDX/specification
// Document represents a CycloneDX BOM Document.
type Document struct {
XMLName xml.Name `xml:"bom"`
XMLNs string `xml:"xmlns,attr"`
Version int `xml:"version,attr"`
SerialNumber string `xml:"serialNumber,attr"`
BomDescriptor *BomDescriptor `xml:"metadata"` // The BOM descriptor extension
Components []Component `xml:"components>component"` // The BOM contents
}
@@ -0,0 +1,10 @@
package model
import "encoding/xml"
// License represents a single software license for a Component
type License struct {
XMLName xml.Name `xml:"license"`
ID string `xml:"id,omitempty"` // A valid SPDX license ID
Name string `xml:"name,omitempty"` // If SPDX does not define the license used, this field may be used to provide the license name
}
@@ -0,0 +1,95 @@
package cyclonedx12xml
import (
"encoding/xml"
"time"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/formats/cyclonedx12xml/model"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/google/uuid"
)
// toFormatModel creates and populates a new in-memory representation of a CycloneDX 1.2 document
func toFormatModel(s sbom.SBOM) model.Document {
versionInfo := version.FromBuild()
doc := model.Document{
XMLNs: "http://cyclonedx.org/schema/bom/1.2",
Version: 1,
SerialNumber: uuid.New().URN(),
BomDescriptor: toBomDescriptor(internal.ApplicationName, versionInfo.Version, s.Source),
}
// attach components
for _, p := range s.Artifacts.PackageCatalog.Sorted() {
doc.Components = append(doc.Components, toComponent(p))
}
return doc
}
func toComponent(p pkg.Package) model.Component {
return model.Component{
Type: "library", // TODO: this is not accurate
Name: p.Name,
Version: p.Version,
PackageURL: p.PURL,
Licenses: toLicenses(p.Licenses),
}
}
// NewBomDescriptor returns a new BomDescriptor tailored for the current time and "syft" tool details.
func toBomDescriptor(name, version string, srcMetadata source.Metadata) *model.BomDescriptor {
return &model.BomDescriptor{
XMLName: xml.Name{},
Timestamp: time.Now().Format(time.RFC3339),
Tools: []model.BomDescriptorTool{
{
Vendor: "anchore",
Name: name,
Version: version,
},
},
Component: toBomDescriptorComponent(srcMetadata),
}
}
func toBomDescriptorComponent(srcMetadata source.Metadata) *model.BomDescriptorComponent {
switch srcMetadata.Scheme {
case source.ImageScheme:
return &model.BomDescriptorComponent{
Component: model.Component{
Type: "container",
Name: srcMetadata.ImageMetadata.UserInput,
Version: srcMetadata.ImageMetadata.ManifestDigest,
},
}
case source.DirectoryScheme:
return &model.BomDescriptorComponent{
Component: model.Component{
Type: "file",
Name: srcMetadata.Path,
},
}
}
return nil
}
func toLicenses(licenses []string) *[]model.License {
if len(licenses) == 0 {
return nil
}
var result []model.License
for _, licenseName := range licenses {
result = append(result, model.License{
Name: licenseName,
})
}
return &result
}
+10
View File
@@ -3,7 +3,12 @@ package formats
import (
"bytes"
"github.com/anchore/syft/internal/formats/cyclonedx12xml"
"github.com/anchore/syft/internal/formats/spdx22json"
"github.com/anchore/syft/internal/formats/spdx22tagvalue"
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/internal/formats/text"
"github.com/anchore/syft/syft/format"
)
@@ -11,6 +16,11 @@ import (
func All() []format.Format {
return []format.Format{
syftjson.Format(),
table.Format(),
cyclonedx12xml.Format(),
spdx22json.Format(),
spdx22tagvalue.Format(),
text.Format(),
}
}
+22
View File
@@ -0,0 +1,22 @@
package spdx22json
import (
"encoding/json"
"fmt"
"io"
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/syft/sbom"
)
func decoder(reader io.Reader) (*sbom.SBOM, error) {
dec := json.NewDecoder(reader)
var doc model.Document
err := dec.Decode(&doc)
if err != nil {
return nil, fmt.Errorf("unable to decode syft-json: %w", err)
}
return toSyftModel(doc)
}
@@ -0,0 +1,46 @@
package spdx22json
import (
"bytes"
"strings"
"testing"
"github.com/anchore/syft/internal/formats/common/testutils"
"github.com/go-test/deep"
"github.com/stretchr/testify/assert"
)
func TestEncodeDecodeCycle(t *testing.T) {
testImage := "image-simple"
originalSBOM := testutils.ImageInput(t, testImage)
var buf bytes.Buffer
assert.NoError(t, encoder(&buf, originalSBOM))
actualSBOM, err := decoder(bytes.NewReader(buf.Bytes()))
assert.NoError(t, err)
for _, d := range deep.Equal(originalSBOM.Source, actualSBOM.Source) {
t.Errorf("metadata difference: %+v", d)
}
actualPackages := actualSBOM.Artifacts.PackageCatalog.Sorted()
for idx, p := range originalSBOM.Artifacts.PackageCatalog.Sorted() {
if !assert.Equal(t, p.Name, actualPackages[idx].Name) {
t.Errorf("different package at idx=%d: %s vs %s", idx, p.Name, actualPackages[idx].Name)
continue
}
for _, d := range deep.Equal(p, actualPackages[idx]) {
if strings.Contains(d, ".VirtualPath: ") {
// location.Virtual path is not exposed in the json output
continue
}
if strings.HasSuffix(d, "<nil slice> != []") {
// semantically the same
continue
}
t.Errorf("package difference (%s): %+v", p.Name, d)
}
}
}
+19
View File
@@ -0,0 +1,19 @@
package spdx22json
import (
"encoding/json"
"io"
"github.com/anchore/syft/syft/sbom"
)
func encoder(output io.Writer, s sbom.SBOM) error {
doc := toFormatModel(s)
enc := json.NewEncoder(output)
// prevent > and < from being escaped in the payload
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
return enc.Encode(&doc)
}
@@ -1,4 +1,4 @@
package packages
package spdx22json
import (
"flag"
@@ -11,9 +11,8 @@ import (
var updateSpdxJson = flag.Bool("update-spdx-json", false, "update the *.golden files for spdx-json presenters")
func TestSPDXJSONDirectoryPresenter(t *testing.T) {
catalog, metadata, _ := testutils.DirectoryInput(t)
testutils.AssertPresenterAgainstGoldenSnapshot(t,
NewSPDXJSONPresenter(catalog, metadata),
Format().Presenter(testutils.DirectoryInput(t)),
*updateSpdxJson,
spdxJsonRedactor,
)
@@ -21,9 +20,8 @@ func TestSPDXJSONDirectoryPresenter(t *testing.T) {
func TestSPDXJSONImagePresenter(t *testing.T) {
testImage := "image-simple"
catalog, metadata, _ := testutils.ImageInput(t, testImage)
testutils.AssertPresenterAgainstGoldenImageSnapshot(t,
NewSPDXJSONPresenter(catalog, metadata),
Format().Presenter(testutils.ImageInput(t, testImage, testutils.FromSnapshot())),
testImage,
*updateSpdxJson,
spdxJsonRedactor,
+13
View File
@@ -0,0 +1,13 @@
package spdx22json
import "github.com/anchore/syft/syft/format"
// note: this format is LOSSY relative to the syftjson formation, which means that decoding and validation is not supported at this time
func Format() format.Format {
return format.NewFormat(
format.SPDXJSONOption,
encoder,
nil,
nil,
)
}
@@ -1,4 +1,4 @@
package spdx22
package model
import "time"
@@ -1,4 +1,4 @@
package spdx22
package model
type Checksum struct {
// Identifies the algorithm used to produce the subject Checksum. One of: "SHA256", "SHA1", "SHA384", "MD2", "MD4", "SHA512", "MD6", "MD5", "SHA224"
@@ -1,4 +1,4 @@
package spdx22
package model
import "time"
@@ -1,4 +1,4 @@
package spdx22
package model
// derived from:
// - https://spdx.github.io/spdx-spec/appendix-III-RDF-data-model-implementation-and-identifier-syntax/
@@ -1,4 +1,4 @@
package spdx22
package model
type Element struct {
SPDXID string `json:"SPDXID"`
@@ -1,4 +1,4 @@
package spdx22
package model
// ElementID represents the identifier string portion of an SPDX element
// identifier. DocElementID should be used for any attributes which can
@@ -1,4 +1,4 @@
package spdx22
package model
type ExternalDocumentRef struct {
// externalDocumentId is a string containing letters, numbers, ., - and/or + which uniquely identifies an external document within this document.
@@ -1,4 +1,4 @@
package spdx22
package model
type ReferenceCategory string
@@ -1,4 +1,4 @@
package spdx22
package model
type FileType string
@@ -1,4 +1,4 @@
package spdx22
package model
type HasExtractedLicensingInfo struct {
// Verbatim license or licensing notice text that was discovered.
@@ -1,4 +1,4 @@
package spdx22
package model
type Item struct {
Element
@@ -1,4 +1,4 @@
package spdx22
package model
type Package struct {
Item
@@ -1,4 +1,4 @@
package spdx22
package model
// Why are there two package identifier fields Package Checksum and Package Verification?
// Although the values of the two fields Package Checksum and Package Verification are similar, they each serve a
@@ -1,4 +1,4 @@
package spdx22
package model
type Relationship struct {
// Id to which the SPDX element is related
@@ -1,4 +1,4 @@
package spdx22
package model
type StartPointer struct {
Offset int `json:"offset,omitempty"`
@@ -1,3 +1,3 @@
package spdx22
package model
const Version = "SPDX-2.2"
@@ -0,0 +1,4 @@
# Note: changes to this file will result in updating several test values. Consider making a new image fixture instead of editing this one.
FROM scratch
ADD file-1.txt /somefile-1.txt
ADD file-2.txt /somefile-2.txt
@@ -0,0 +1 @@
this file has contents
@@ -0,0 +1 @@
file-2 contents!
@@ -3,7 +3,7 @@
"name": "/some/path",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"created": "2021-10-12T18:40:22.948394Z",
"created": "2021-10-29T16:26:08.995826Z",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-[not provided]"
@@ -11,7 +11,7 @@
"licenseListVersion": "3.14"
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https:/anchore.com/syft/dir/some/path-98ae71fb-f276-4c5c-acf7-25770bf7bca2",
"documentNamespace": "https:/anchore.com/syft/dir/some/path-5362d380-914a-458f-b059-d8d27899574c",
"packages": [
{
"SPDXID": "SPDXRef-Package-python-package-1-1.0.1",
@@ -3,7 +3,7 @@
"name": "user-image-input",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"created": "2021-10-12T18:40:22.953633Z",
"created": "2021-10-29T16:26:09.001799Z",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-[not provided]"
@@ -11,7 +11,7 @@
"licenseListVersion": "3.14"
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https:/anchore.com/syft/image/user-image-input-149edbad-3c01-4ee0-b3a0-75232312bf51",
"documentNamespace": "https:/anchore.com/syft/image/user-image-input-3ad8571c-513f-4fce-944e-5125353c3186",
"packages": [
{
"SPDXID": "SPDXRef-Package-python-package-1-1.0.1",
@@ -0,0 +1,83 @@
package spdx22json
import (
"fmt"
"time"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/formats/common/spdxhelpers"
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
)
// toFormatModel creates and populates a new JSON document struct that follows the SPDX 2.2 spec from the given cataloging results.
func toFormatModel(s sbom.SBOM) model.Document {
name := spdxhelpers.DocumentName(s.Source)
packages, files, relationships := extractFromCatalog(s.Artifacts.PackageCatalog)
return model.Document{
Element: model.Element{
SPDXID: model.ElementID("DOCUMENT").String(),
Name: name,
},
SPDXVersion: model.Version,
CreationInfo: model.CreationInfo{
Created: time.Now().UTC(),
Creators: []string{
// note: key-value format derived from the JSON example document examples: https://github.com/spdx/spdx-spec/blob/v2.2/examples/SPDXJSONExample-v2.2.spdx.json
"Organization: Anchore, Inc",
"Tool: " + internal.ApplicationName + "-" + version.FromBuild().Version,
},
LicenseListVersion: spdxlicense.Version,
},
DataLicense: "CC0-1.0",
DocumentNamespace: spdxhelpers.DocumentNamespace(name, s.Source),
Packages: packages,
Files: files,
Relationships: relationships,
}
}
func extractFromCatalog(catalog *pkg.Catalog) ([]model.Package, []model.File, []model.Relationship) {
packages := make([]model.Package, 0)
relationships := make([]model.Relationship, 0)
files := make([]model.File, 0)
for _, p := range catalog.Sorted() {
license := spdxhelpers.License(p)
packageSpdxID := model.ElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.Version)).String()
packageFiles, fileIDs, packageFileRelationships := spdxhelpers.Files(packageSpdxID, p)
files = append(files, packageFiles...)
relationships = append(relationships, packageFileRelationships...)
// note: the license concluded and declared should be the same since we are collecting license information
// from the project data itself (the installed package files).
packages = append(packages, model.Package{
Description: spdxhelpers.Description(p),
DownloadLocation: spdxhelpers.DownloadLocation(p),
ExternalRefs: spdxhelpers.ExternalRefs(p),
FilesAnalyzed: false,
HasFiles: fileIDs,
Homepage: spdxhelpers.Homepage(p),
LicenseDeclared: license, // The Declared License is what the authors of a project believe govern the package
Originator: spdxhelpers.Originator(p),
SourceInfo: spdxhelpers.SourceInfo(p),
VersionInfo: p.Version,
Item: model.Item{
LicenseConcluded: license, // The Concluded License field is the license the SPDX file creator believes governs the package
Element: model.Element{
SPDXID: packageSpdxID,
Name: p.Name,
},
},
})
}
return packages, files, relationships
}
@@ -0,0 +1,10 @@
package spdx22json
import (
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/syft/sbom"
)
func toSyftModel(doc model.Document) (*sbom.SBOM, error) {
}
@@ -0,0 +1,13 @@
package spdx22tagvalue
import (
"io"
"github.com/anchore/syft/syft/sbom"
"github.com/spdx/tools-golang/tvsaver"
)
func encoder(output io.Writer, s sbom.SBOM) error {
model := toFormatModel(s)
return tvsaver.Save2_2(&model, output)
}
@@ -1,4 +1,4 @@
package packages
package spdx22tagvalue
import (
"flag"
@@ -11,9 +11,9 @@ import (
var updateSpdxTagValue = flag.Bool("update-spdx-tv", false, "update the *.golden files for spdx-tv presenters")
func TestSPDXTagValueDirectoryPresenter(t *testing.T) {
catalog, metadata, _ := testutils.DirectoryInput(t)
testutils.AssertPresenterAgainstGoldenSnapshot(t,
NewSPDXTagValuePresenter(catalog, metadata),
Format().Presenter(testutils.DirectoryInput(t)),
*updateSpdxTagValue,
spdxTagValueRedactor,
)
@@ -21,9 +21,8 @@ func TestSPDXTagValueDirectoryPresenter(t *testing.T) {
func TestSPDXTagValueImagePresenter(t *testing.T) {
testImage := "image-simple"
catalog, metadata, _ := testutils.ImageInput(t, testImage)
testutils.AssertPresenterAgainstGoldenImageSnapshot(t,
NewSPDXTagValuePresenter(catalog, metadata),
Format().Presenter(testutils.ImageInput(t, testImage, testutils.FromSnapshot())),
testImage,
*updateSpdxTagValue,
spdxTagValueRedactor,
+13
View File
@@ -0,0 +1,13 @@
package spdx22tagvalue
import "github.com/anchore/syft/syft/format"
// note: this format is LOSSY relative to the syftjson formation, which means that decoding and validation is not supported at this time
func Format() format.Format {
return format.NewFormat(
format.SPDXTagValueOption,
encoder,
nil,
nil,
)
}
@@ -0,0 +1,4 @@
# Note: changes to this file will result in updating several test values. Consider making a new image fixture instead of editing this one.
FROM scratch
ADD file-1.txt /somefile-1.txt
ADD file-2.txt /somefile-2.txt
@@ -0,0 +1 @@
this file has contents
@@ -0,0 +1 @@
file-2 contents!
@@ -1,38 +1,23 @@
package packages
package spdx22tagvalue
import (
"fmt"
"io"
"time"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/formats/common/spdxhelpers"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/tvsaver"
)
// SPDXTagValuePresenter is a SPDX presentation object for the syft results (see https://github.com/spdx/spdx-spec)
type SPDXTagValuePresenter struct {
catalog *pkg.Catalog
srcMetadata source.Metadata
}
// NewJSONPresenter creates a new JSON presenter object for the given cataloging results.
func NewSPDXTagValuePresenter(catalog *pkg.Catalog, srcMetadata source.Metadata) *SPDXTagValuePresenter {
return &SPDXTagValuePresenter{
catalog: catalog,
srcMetadata: srcMetadata,
}
}
// Present the catalog results to the given writer.
// nolint: funlen
func (pres *SPDXTagValuePresenter) Present(output io.Writer) error {
doc := spdx.Document2_2{
// toFormatModel creates and populates a new JSON document struct that follows the SPDX 2.2 spec from the given cataloging results.
// nolint:funlen
func toFormatModel(s sbom.SBOM) spdx.Document2_2 {
return spdx.Document2_2{
CreationInfo: &spdx.CreationInfo2_2{
// 2.1: SPDX Version; should be in the format "SPDX-2.2"
// Cardinality: mandatory, one
@@ -48,7 +33,7 @@ func (pres *SPDXTagValuePresenter) Present(output io.Writer) error {
// 2.4: Document Name
// Cardinality: mandatory, one
DocumentName: pres.srcMetadata.ImageMetadata.UserInput,
DocumentName: s.Source.ImageMetadata.UserInput,
// 2.5: Document Namespace
// Cardinality: mandatory, one
@@ -67,7 +52,7 @@ func (pres *SPDXTagValuePresenter) Present(output io.Writer) error {
// In many cases, the URI will point to a web accessible document, but this should not be assumed
// to be the case.
DocumentNamespace: fmt.Sprintf("https://anchore.com/syft/image/%s", pres.srcMetadata.ImageMetadata.UserInput),
DocumentNamespace: fmt.Sprintf("https://anchore.com/syft/image/%s", s.Source.ImageMetadata.UserInput),
// 2.6: External Document References
// Cardinality: optional, one or many
@@ -96,25 +81,23 @@ func (pres *SPDXTagValuePresenter) Present(output io.Writer) error {
// Cardinality: optional, one
DocumentComment: "",
},
Packages: pres.packages(),
Packages: toFormatPackages(s.Artifacts.PackageCatalog),
}
return tvsaver.Save2_2(&doc, output)
}
// packages populates all Package Information from the package Catalog (see https://spdx.github.io/spdx-spec/3-package-information/)
// nolint: funlen
func (pres *SPDXTagValuePresenter) packages() map[spdx.ElementID]*spdx.Package2_2 {
func toFormatPackages(catalog *pkg.Catalog) map[spdx.ElementID]*spdx.Package2_2 {
results := make(map[spdx.ElementID]*spdx.Package2_2)
for p := range pres.catalog.Enumerate() {
for p := range catalog.Enumerate() {
// name should be guaranteed to be unique, but semantically useful and stable
id := fmt.Sprintf("Package-%+v-%s", p.Type, p.Name)
// If the Concluded License is not the same as the Declared License, a written explanation should be provided
// in the Comments on License field (section 3.16). With respect to NOASSERTION, a written explanation in
// the Comments on License field (section 3.16) is preferred.
license := getSPDXLicense(p)
license := spdxhelpers.License(p)
results[spdx.ElementID(id)] = &spdx.Package2_2{
@@ -273,8 +256,8 @@ func (pres *SPDXTagValuePresenter) packages() map[spdx.ElementID]*spdx.Package2_
return results
}
func formatSPDXExternalRefs(p *pkg.Package) (refs []*spdx.PackageExternalReference2_2) {
for _, ref := range getSPDXExternalRefs(p) {
func formatSPDXExternalRefs(p pkg.Package) (refs []*spdx.PackageExternalReference2_2) {
for _, ref := range spdxhelpers.ExternalRefs(p) {
refs = append(refs, &spdx.PackageExternalReference2_2{
Category: string(ref.ReferenceCategory),
RefType: string(ref.ReferenceType),
+4 -5
View File
@@ -5,19 +5,18 @@ import (
"fmt"
"io"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal/formats/syftjson/model"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
func decoder(reader io.Reader) (*pkg.Catalog, *source.Metadata, *distro.Distro, source.Scope, error) {
func decoder(reader io.Reader) (*sbom.SBOM, error) {
dec := json.NewDecoder(reader)
var doc model.Document
err := dec.Decode(&doc)
if err != nil {
return nil, nil, nil, source.UnknownScope, fmt.Errorf("unable to decode syft-json: %w", err)
return nil, fmt.Errorf("unable to decode syft-json: %w", err)
}
return toSyftModel(doc)
+7 -13
View File
@@ -5,8 +5,6 @@ import (
"strings"
"testing"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/internal/formats/common/testutils"
"github.com/go-test/deep"
"github.com/stretchr/testify/assert"
@@ -14,30 +12,26 @@ import (
func TestEncodeDecodeCycle(t *testing.T) {
testImage := "image-simple"
originalCatalog, originalMetadata, _ := testutils.ImageInput(t, testImage)
originalSBOM := testutils.ImageInput(t, testImage)
var buf bytes.Buffer
assert.NoError(t, encoder(&buf, originalCatalog, &originalMetadata, nil, source.SquashedScope))
assert.NoError(t, encoder(&buf, originalSBOM))
actualCatalog, actualMetadata, _, _, err := decoder(bytes.NewReader(buf.Bytes()))
actualSBOM, err := decoder(bytes.NewReader(buf.Bytes()))
assert.NoError(t, err)
for _, d := range deep.Equal(originalMetadata, *actualMetadata) {
for _, d := range deep.Equal(originalSBOM.Source, actualSBOM.Source) {
t.Errorf("metadata difference: %+v", d)
}
actualPackages := actualCatalog.Sorted()
for idx, p := range originalCatalog.Sorted() {
actualPackages := actualSBOM.Artifacts.PackageCatalog.Sorted()
for idx, p := range originalSBOM.Artifacts.PackageCatalog.Sorted() {
if !assert.Equal(t, p.Name, actualPackages[idx].Name) {
t.Errorf("different package at idx=%d: %s vs %s", idx, p.Name, actualPackages[idx].Name)
continue
}
// ids will never be equal
p.ID = ""
actualPackages[idx].ID = ""
for _, d := range deep.Equal(*p, *actualPackages[idx]) {
for _, d := range deep.Equal(p, actualPackages[idx]) {
if strings.Contains(d, ".VirtualPath: ") {
// location.Virtual path is not exposed in the json output
continue
+3 -6
View File
@@ -4,15 +4,12 @@ import (
"encoding/json"
"io"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/syft/sbom"
)
func encoder(output io.Writer, catalog *pkg.Catalog, srcMetadata *source.Metadata, d *distro.Distro, scope source.Scope) error {
func encoder(output io.Writer, s sbom.SBOM) error {
// TODO: application config not available yet
doc := ToFormatModel(catalog, srcMetadata, d, scope, nil)
doc := ToFormatModel(s, nil)
enc := json.NewEncoder(output)
// prevent > and < from being escaped in the payload
+2 -7
View File
@@ -4,27 +4,22 @@ import (
"flag"
"testing"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/internal/formats/common/testutils"
"github.com/anchore/syft/syft/format"
)
var updateJson = flag.Bool("update-json", false, "update the *.golden files for json presenters")
func TestDirectoryPresenter(t *testing.T) {
catalog, metadata, distro := testutils.DirectoryInput(t)
testutils.AssertPresenterAgainstGoldenSnapshot(t,
format.NewPresenter(encoder, catalog, &metadata, distro, source.SquashedScope),
Format().Presenter(testutils.DirectoryInput(t)),
*updateJson,
)
}
func TestImagePresenter(t *testing.T) {
testImage := "image-simple"
catalog, metadata, distro := testutils.ImageInput(t, testImage)
testutils.AssertPresenterAgainstGoldenImageSnapshot(t,
format.NewPresenter(encoder, catalog, &metadata, distro, source.SquashedScope),
Format().Presenter(testutils.ImageInput(t, testImage, testutils.FromSnapshot())),
testImage,
*updateJson,
)
+1 -6
View File
@@ -20,11 +20,6 @@ type sourceUnpacker struct {
Target json.RawMessage `json:"target"`
}
type ImageSource struct {
source.ImageMetadata
Scope source.Scope `json:"scope"`
}
// UnmarshalJSON populates a source object from JSON bytes.
func (s *Source) UnmarshalJSON(b []byte) error {
var unpacker sourceUnpacker
@@ -43,7 +38,7 @@ func (s *Source) UnmarshalJSON(b []byte) error {
}
case "image":
var payload ImageSource
var payload source.ImageMetadata
if err := json.Unmarshal(unpacker.Target, &payload); err != nil {
return err
}
@@ -1,7 +1,7 @@
{
"artifacts": [
{
"id": "package-1-id",
"id": "cbf4f3077fc7deee",
"name": "package-1",
"version": "1.0.1",
"type": "python",
@@ -36,7 +36,7 @@
}
},
{
"id": "package-2-id",
"id": "1a39aadd9705c2b9",
"name": "package-2",
"version": "2.0.1",
"type": "deb",
@@ -1,7 +1,7 @@
{
"artifacts": [
{
"id": "package-1-id",
"id": "d1d433485a31ed07",
"name": "package-1",
"version": "1.0.1",
"type": "python",
@@ -32,7 +32,7 @@
}
},
{
"id": "package-2-id",
"id": "2db629ca48fa6786",
"name": "package-2",
"version": "2.0.1",
"type": "deb",
@@ -88,8 +88,7 @@
],
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NjcsImRpZ2VzdCI6InNoYTI1NjoyNDgwMTYwYjU1YmVjNDBjNDRkM2IxNDVjN2IyYzFjNDcxNjBkYjg1NzVjM2RjYWUwODZkNzZiOTM3MGFlN2NhIn0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1NjpmYjZiZWVjYjc1YjM5ZjRiYjgxM2RiZjE3N2U1MDFlZGQ1ZGRiM2U2OWJiNDVjZWRlYjc4YzY3NmVlMWI3YTU5In0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjMxOWI1ODhjZTY0MjUzYTg3YjUzM2M4ZWQwMWNmMDAyNWUwZWFjOThlN2I1MTZlMTI1MzI5NTdlMTI0NGZkZWMifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjEtMTAtMDRUMTE6NDA6MDAuNjM4Mzk0NVoiLCJoaXN0b3J5IjpbeyJjcmVhdGVkIjoiMjAyMS0xMC0wNFQxMTo0MDowMC41OTA3MzE2WiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0xLnR4dCAvc29tZWZpbGUtMS50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn0seyJjcmVhdGVkIjoiMjAyMS0xMC0wNFQxMTo0MDowMC42MzgzOTQ1WiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0yLnR4dCAvc29tZWZpbGUtMi50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn1dLCJvcyI6ImxpbnV4Iiwicm9vdGZzIjp7InR5cGUiOiJsYXllcnMiLCJkaWZmX2lkcyI6WyJzaGEyNTY6ZmI2YmVlY2I3NWIzOWY0YmI4MTNkYmYxNzdlNTAxZWRkNWRkYjNlNjliYjQ1Y2VkZWI3OGM2NzZlZTFiN2E1OSIsInNoYTI1NjozMTliNTg4Y2U2NDI1M2E4N2I1MzNjOGVkMDFjZjAwMjVlMGVhYzk4ZTdiNTE2ZTEyNTMyOTU3ZTEyNDRmZGVjIl19fQ==",
"repoDigests": [],
"scope": "Squashed"
"repoDigests": []
}
},
"distro": {
+18 -17
View File
@@ -3,6 +3,10 @@ package syftjson
import (
"fmt"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/formats/syftjson/model"
"github.com/anchore/syft/internal/log"
@@ -13,17 +17,17 @@ import (
)
// TODO: this is export4ed for the use of the power-user command (temp)
func ToFormatModel(catalog *pkg.Catalog, srcMetadata *source.Metadata, d *distro.Distro, scope source.Scope, applicationConfig interface{}) model.Document {
src, err := toSourceModel(srcMetadata, scope)
func ToFormatModel(s sbom.SBOM, applicationConfig interface{}) model.Document {
src, err := toSourceModel(s.Source)
if err != nil {
log.Warnf("unable to create syft-json source object: %+v", err)
}
return model.Document{
Artifacts: toPackageModels(catalog),
ArtifactRelationships: toRelationshipModel(pkg.NewRelationships(catalog)),
Artifacts: toPackageModels(s.Artifacts.PackageCatalog),
ArtifactRelationships: toRelationshipModel(s.Relationships),
Source: src,
Distro: toDistroModel(d),
Distro: toDistroModel(s.Artifacts.Distro),
Descriptor: model.Descriptor{
Name: internal.ApplicationName,
Version: version.FromBuild().Version,
@@ -48,7 +52,7 @@ func toPackageModels(catalog *pkg.Catalog) []model.Package {
}
// toPackageModel crates a new Package from the given pkg.Package.
func toPackageModel(p *pkg.Package) model.Package {
func toPackageModel(p pkg.Package) model.Package {
var cpes = make([]string, len(p.CPEs))
for i, c := range p.CPEs {
cpes[i] = c.BindToFmtString()
@@ -67,7 +71,7 @@ func toPackageModel(p *pkg.Package) model.Package {
return model.Package{
PackageBasicData: model.PackageBasicData{
ID: string(p.ID),
ID: string(p.ID()),
Name: p.Name,
Version: p.Version,
Type: p.Type,
@@ -85,29 +89,26 @@ func toPackageModel(p *pkg.Package) model.Package {
}
}
func toRelationshipModel(relationships []pkg.Relationship) []model.Relationship {
func toRelationshipModel(relationships []artifact.Relationship) []model.Relationship {
result := make([]model.Relationship, len(relationships))
for i, r := range relationships {
result[i] = model.Relationship{
Parent: string(r.Parent),
Child: string(r.Child),
Parent: string(r.From.ID()),
Child: string(r.To.ID()),
Type: string(r.Type),
Metadata: r.Metadata,
Metadata: r.Data,
}
}
return result
}
// toSourceModel creates a new source object to be represented into JSON.
func toSourceModel(src *source.Metadata, scope source.Scope) (model.Source, error) {
func toSourceModel(src source.Metadata) (model.Source, error) {
switch src.Scheme {
case source.ImageScheme:
return model.Source{
Type: "image",
Target: model.ImageSource{
ImageMetadata: src.ImageMetadata,
Scope: scope,
},
Type: "image",
Target: src.ImageMetadata,
}, nil
case source.DirectoryScheme:
return model.Source{
+15 -12
View File
@@ -5,35 +5,39 @@ import (
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/distro"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
func toSyftModel(doc model.Document) (*pkg.Catalog, *source.Metadata, *distro.Distro, source.Scope, error) {
func toSyftModel(doc model.Document) (*sbom.SBOM, error) {
dist, err := distro.NewDistro(distro.Type(doc.Distro.Name), doc.Distro.Version, doc.Distro.IDLike)
if err != nil {
return nil, nil, nil, source.UnknownScope, err
return nil, err
}
srcMetadata, scope := toSyftSourceData(doc.Source)
return toSyftCatalog(doc.Artifacts), srcMetadata, &dist, scope, nil
return &sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: toSyftCatalog(doc.Artifacts),
Distro: &dist,
},
Source: *toSyftSourceData(doc.Source),
}, nil
}
func toSyftSourceData(s model.Source) (*source.Metadata, source.Scope) {
func toSyftSourceData(s model.Source) *source.Metadata {
switch s.Type {
case "directory":
return &source.Metadata{
Scheme: source.DirectoryScheme,
Path: s.Target.(string),
}, source.UnknownScope
}
case "image":
parsedSource := s.Target.(model.ImageSource)
return &source.Metadata{
Scheme: source.ImageScheme,
ImageMetadata: parsedSource.ImageMetadata,
}, parsedSource.Scope
ImageMetadata: s.Target.(source.ImageMetadata),
}
}
return nil, source.UnknownScope
return nil
}
func toSyftCatalog(pkgs []model.Package) *pkg.Catalog {
@@ -57,7 +61,6 @@ func toSyftPackage(p model.Package) pkg.Package {
}
return pkg.Package{
ID: pkg.ID(p.ID),
Name: p.Name,
Version: p.Version,
FoundBy: p.FoundBy,
@@ -1,4 +1,4 @@
package packages
package table
import (
"fmt"
@@ -6,26 +6,16 @@ import (
"sort"
"strings"
"github.com/olekukonko/tablewriter"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/pkg"
"github.com/olekukonko/tablewriter"
)
type TablePresenter struct {
catalog *pkg.Catalog
}
func NewTablePresenter(catalog *pkg.Catalog) *TablePresenter {
return &TablePresenter{
catalog: catalog,
}
}
func (pres *TablePresenter) Present(output io.Writer) error {
rows := make([][]string, 0)
func encoder(output io.Writer, s sbom.SBOM) error {
var rows [][]string
columns := []string{"Name", "Version", "Type"}
for _, p := range pres.catalog.Sorted() {
for _, p := range s.Artifacts.PackageCatalog.Sorted() {
row := []string{
p.Name,
p.Version,
@@ -35,8 +25,8 @@ func (pres *TablePresenter) Present(output io.Writer) error {
}
if len(rows) == 0 {
fmt.Fprintln(output, "No packages discovered")
return nil
_, err := fmt.Fprintln(output, "No packages discovered")
return err
}
// sort by name, version, then type
@@ -1,23 +1,19 @@
package packages
package table
import (
"flag"
"testing"
"github.com/anchore/syft/internal/formats/common/testutils"
"github.com/go-test/deep"
)
var updateTablePresenterGoldenFiles = flag.Bool("update-table", false, "update the *.golden files for table presenters")
var updateTableGoldenFiles = flag.Bool("update-table", false, "update the *.golden files for table format")
func TestTablePresenter(t *testing.T) {
testImage := "image-simple"
catalog, _, _ := testutils.ImageInput(t, testImage)
testutils.AssertPresenterAgainstGoldenImageSnapshot(t,
NewTablePresenter(catalog),
testImage,
*updateTablePresenterGoldenFiles,
testutils.AssertPresenterAgainstGoldenSnapshot(t,
Format().Presenter(testutils.DirectoryInput(t)),
*updateTableGoldenFiles,
)
}
+12
View File
@@ -0,0 +1,12 @@
package table
import "github.com/anchore/syft/syft/format"
func Format() format.Format {
return format.NewFormat(
format.TableOption,
encoder,
nil,
nil,
)
}

Some files were not shown because too many files have changed in this diff Show More