Compare commits

...
41 Commits
Author SHA1 Message Date
anchore-actions-token-generator[bot]andwagoodman 247b054ab5 Update Stereoscope to 3b80d983223f6e6fc2d33b0ffa003d30268418e9 (#1376)
Signed-off-by: GitHub <noreply@github.com>

Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: wagoodman <wagoodman@users.noreply.github.com>
2022-11-30 16:11:57 +00:00
Keith Zantow 9e43725951 fix: Update node binary package name (#1375) 2022-11-30 10:30:57 -05:00
Keith Zantow 4f39287216 feat: Generic Binary Cataloger (#1336) 2022-11-29 18:28:10 -05:00
Alex Goodman 7a69e2129b recover from bad parsing of golang binary (#1371)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-29 15:56:46 +00:00
Dan Luhring f6996f7b9a Fix parsing of apk databases with large entries (#1365)
Closes https://github.com/anchore/syft/issues/1354
2022-11-29 10:16:36 -05:00
anchore-actions-token-generator[bot]andspiffcs bd523bdb5d Update syft bootstrap tools to latest versions. (#1369)
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2022-11-29 10:13:00 -05:00
Christopher Angelo Phillips 0cbd0cc703 fix: guard for locations < 1 in alpmdb parse (#1366) 2022-11-28 15:43:18 +00:00
Christopher Angelo Phillips b290a445ca fix: remove cabal.project.freeze panic on last pkg (#1363) 2022-11-23 22:33:18 +00:00
Christopher Angelo Phillips bcfe38c009 fix: requirements.txt - return unicode only letter/num for version (#1361) 2022-11-22 10:43:05 -05:00
anchore-actions-token-generator[bot]andspiffcs 74967a28ea Update syft bootstrap tools to latest versions. (#1356)
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2022-11-21 09:57:49 -05:00
Keith Zantow 098e61dcc8 fix: sort relationships in SPDX output (#1350) 2022-11-21 09:26:24 -05:00
Keith Zantow 0dddf51fd5 chore: add debug logging for decode errors (#1352) 2022-11-21 09:26:11 -05:00
mikcl 04880c06ce feat(npm): handle aliases in package-lock.json (#1349) 2022-11-20 11:32:10 -05:00
Christopher Angelo Phillips da4b2df576 fix: spdx java checksum correctness (#1348) 2022-11-18 13:42:55 -05:00
Rob Cresswell 9d8244bae6 feat: Add support for npm lockfile version 3 (#1206)
This PR adds support for npm lockfile version 3, which drops the
"dependencies" key and uses "packages" instead. I've refactored the
lockfile parser to make the distinction between the versions explicit
rather than the implicit behaviour before. It _might_ be worth splitting
into separate files at some point, but the logic is so minimal that I
haven't done it.

Fixes #1203
Signed-off-by: Rob Cresswell <robcresswell@users.noreply.github.com>
2022-11-18 12:41:31 -05:00
Christopher Angelo Phillips 67888ee855 1111 clean name bug (#1347) 2022-11-18 14:45:18 +00:00
Alex GoodmanandKeith Zantow 9afc9231c0 Add spdx relationship encoding for dependencies (#1342)
Co-authored-by: Keith Zantow <kzantow@gmail.com>
2022-11-18 09:40:10 -05:00
Keith Zantow 42cb0a47a4 feat: SPDX 2.3 support (#1311) 2022-11-18 08:54:39 -05:00
0c4b99c1c2 SBOM cataloger (#1029)
* SBOM cataloger

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* sbom-cataloger: turn off by default

and add integration test

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* SBOM cataloger

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* SBOM cataloger (optimize)

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* SBOM cataloger (fix)

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* SBOM cataloger (fix imports #1172)

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* SBOM cataloger (fix: support group attribute in CDX SBOMs)

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* port to generic cataloger and add relationship to original file

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* generalize parser for all format globs

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Tom Fay <tomfay@microsoft.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-16 14:11:45 -05:00
Christopher Angelo Phillips 0774ad15e2 chore: clean up linting configuration (#1343) 2022-11-16 16:28:09 +00:00
Keith Zantow f8be64d312 fix: Unmarshal Syft JSON with missing metadata (#1338) 2022-11-15 13:00:10 -05:00
Alex Goodman 1ae577a035 fix apk decode for older data shapes (#1341)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-15 16:38:31 +00:00
Weston Steimel 09bf5b062c chore: add unit test for wolfi os release identification (#1340)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2022-11-14 18:57:31 -05:00
Keith Zantow c1fdfce5f1 fix: Output only valid CPEs for CycloneDX OS components (#1339) 2022-11-14 15:24:19 -05:00
Justin Chadwell 10f43d75e0 feat: Add --name option to override name in output (#1269) 2022-11-10 14:03:23 -05:00
949cff158d Add support for dependency relationships for alpine (apk) (#1063)
* Fix type of pull deps and add support for provides

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* [wip] apk dependency lookup

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update whitespace for linter

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

* adjust test conditions

Signed-off-by: Timothy Gerla <tim@gerla.net>

* fix TODOs and improve Provides parser

* run simports after main merge

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

* add tests to cover apk relationship parsing cases

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* generate JSON schema for breaking changes to apk metadata

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update tests to account for additional dependencies

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] fix relationship encoding for cyclonedx

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify package relationships that can be expressed

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: Timothy Gerla <tim@gerla.net>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: Timothy Gerla <tim@gerla.net>
2022-11-09 15:43:37 +00:00
Alex Goodman e58d0aecb8 normalize alpm md5 refs (#1333)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-09 15:04:15 +00:00
Alex Goodman d7a51a69dd Update java generic cataloger (#1329)
* remove centralize pURL generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* port java cataloger to new generic cataloger pattern

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove common.GenericCataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update format test fixtures to reflect ID updates

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix package sort instability for encode-decode-encode cycles

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-09 14:55:54 +00:00
Keith Zantow f3528132a7 Support encoding map types to CycloneDX properties (#1332) 2022-11-08 18:59:37 -05:00
Alex Goodman 5ed002e1a9 Update swift cataloger to generic cataloger (#1324)
* port swift cataloger to new generic cataloger pattern

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add cocopods metadata to json schema defs

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update json test fixture with latest schema version

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 13:51:59 -04:00
Alex Goodman f319713821 port rust cataloger to new generic cataloger pattern (#1323)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 16:07:36 +00:00
Alex Goodman 41464bbd7f port ruby cataloger to new generic cataloger pattern (#1322)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 11:33:55 -04:00
Alex Goodman 3048382bbd port rpm cataloger to new generic cataloger pattern (#1321)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 10:41:04 -04:00
Alex Goodman 1076281566 port python cataloger to new generic cataloger pattern (#1319)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 08:59:03 -04:00
Alex Goodman 2deb96a801 Update portage cataloger to new generic cataloger (#1316)
* port portage (ha) cataloger to new generic cataloger pattern

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update JSON schema to account for removing portage fields

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-03 14:49:18 -04:00
Alex Goodman 891f2c576b port php cataloger to new generic cataloger pattern (#1315)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-03 13:00:44 -04:00
Kenny MoensandKeith Zantow bc9740d50a javascript cataloger: node binary: nil pointer dereference (#1313)
Co-authored-by: Keith Zantow <kzantow@gmail.com>
2022-11-03 14:38:49 +00:00
Keith Zantow 3e99c4d7d8 Fix: Include version information in binary cataloger CPEs (#1310) 2022-11-03 10:17:15 -04:00
Christopher Angelo Phillips 10464642e9 fix: only generate PURL on empty string (#1312) 2022-11-03 10:00:14 -04:00
Alex Goodman e0acfa98c7 add s3 credentials to release (#1309)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-02 15:48:37 +00:00
Alex Goodman 9634b42746 port javascript cataloger to new generic cataloger pattern (#1308)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-02 11:31:57 -04:00
307 changed files with 189013 additions and 6032 deletions
+6
View File
@@ -141,13 +141,19 @@ jobs:
- name: Build & publish release artifacts
run: make release
env:
# for mac signing and notarization...
QUILL_SIGN_P12: ${{ secrets.ANCHORE_APPLE_DEVELOPER_ID_CERT_CHAIN }}
QUILL_SIGN_PASSWORD: ${{ secrets.ANCHORE_APPLE_DEVELOPER_ID_CERT_PASS }}
QUILL_NOTARY_ISSUER: ${{ secrets.APPLE_NOTARY_ISSUER }}
QUILL_NOTARY_KEY_ID: ${{ secrets.APPLE_NOTARY_KEY_ID }}
QUILL_NOTARY_KEY: ${{ secrets.APPLE_NOTARY_KEY }}
# for creating the release (requires write access to packages and content)
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# for updating brew formula in anchore/homebrew-syft
GITHUB_BREW_TOKEN: ${{ secrets.ANCHORE_GIT_READ_TOKEN }}
# for updating the VERSION file in S3...
AWS_ACCESS_KEY_ID: ${{ secrets.TOOLBOX_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.TOOLBOX_AWS_SECRET_ACCESS_KEY }}
- uses: anchore/sbom-action@v0
continue-on-error: true
+20 -3
View File
@@ -11,9 +11,9 @@ linters:
- asciicheck
- bodyclose
- depguard
- dogsled
- dupl
- errcheck
- errorlint
- exportloopref
- funlen
- gocognit
@@ -21,13 +21,13 @@ linters:
- gocritic
- gocyclo
- gofmt
- goprintffuncname
- tparallel
- importas
- gosec
- gosimple
- govet
- ineffassign
- misspell
- nakedret
- nolintlint
- revive
- staticcheck
@@ -37,8 +37,25 @@ linters:
- unparam
- unused
- whitespace
linters-settings:
funlen:
# Checks the number of lines in a function.
# If lower than 0, disable the check.
# Default: 60
lines: 70
# Checks the number of statements in a function.
# If lower than 0, disable the check.
# Default: 40
statements: 50
output:
uniq-by-line: false
run:
timeout: 10m
# do not enable...
# - dogsled # found to be to niche and ineffective
# - goprintffuncname # does not catch all cases and there are exceptions
# - nakedret # does not catch all cases and should not fail a build
# - gochecknoglobals
# - gochecknoinits # this is too aggressive
# - rowserrcheck disabled per generics https://github.com/golangci/golangci-lint/issues/2649
+2 -2
View File
@@ -3,7 +3,7 @@ VERSION=$(shell git describe --dirty --always --tags)
TEMPDIR = ./.tmp
# commands and versions
LINTCMD = $(TEMPDIR)/golangci-lint run --tests=false --timeout=5m --config .golangci.yaml
LINTCMD = $(TEMPDIR)/golangci-lint run --tests=false
GOIMPORTS_CMD = $(TEMPDIR)/gosimports -local github.com/anchore
RELEASE_CMD=$(TEMPDIR)/goreleaser release --rm-dist
SNAPSHOT_CMD=$(RELEASE_CMD) --skip-publish --snapshot
@@ -13,7 +13,7 @@ GOLANGCILINT_VERSION = v1.50.1
GOSIMPORTS_VERSION = v0.3.4
BOUNCER_VERSION = v0.4.0
CHRONICLE_VERSION = v0.4.2
GORELEASER_VERSION = v1.12.3
GORELEASER_VERSION = v1.13.1
YAJSV_VERSION = v1.4.1
COSIGN_VERSION = v1.13.1
QUILL_VERSION = v0.2.0
+1 -1
View File
@@ -39,7 +39,7 @@ func Attest(v *viper.Viper, app *config.Application, ro *options.RootOptions) *c
// run to unmarshal viper object onto app config
// the viper object correctly
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
return fmt.Errorf("invalid application config: %w", err)
}
// configure logging for command
newLogWrapper(app)
+4 -4
View File
@@ -40,7 +40,7 @@ import (
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/spdx22json"
"github.com/anchore/syft/syft/formats/spdxjson"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
@@ -49,7 +49,7 @@ import (
var (
allowedAttestFormats = []sbom.FormatID{
syftjson.ID,
spdx22json.ID,
spdxjson.ID,
cyclonedxjson.ID,
}
@@ -124,7 +124,7 @@ func parseAttestationOutput(outputs []string) (format string) {
}
func parseImageSource(userInput string, app *config.Application) (s *source.Input, err error) {
si, err := source.ParseInput(userInput, app.Platform, false)
si, err := source.ParseInputWithName(userInput, app.Platform, false, app.Name)
if err != nil {
return nil, fmt.Errorf("could not generate source input for attest command: %w", err)
}
@@ -356,7 +356,7 @@ func uploadAttestation(app *config.Application, signedPayload []byte, digest nam
func formatPredicateType(format sbom.Format) string {
switch format.ID() {
case spdx22json.ID:
case spdxjson.ID:
return in_toto.PredicateSPDX
case cyclonedxjson.ID:
return in_toto.PredicateCycloneDX
-2
View File
@@ -32,8 +32,6 @@ const indent = " "
// at this level. Values from the config should only be used after `app.LoadAllValues` has been called.
// Cobra does not have knowledge of the user provided flags until the `RunE` block of each command.
// `RunE` is the earliest that the complete application configuration can be loaded.
//
//nolint:funlen
func New() (*cobra.Command, error) {
app := &config.Application{}
-2
View File
@@ -15,8 +15,6 @@ import (
// eventLoop listens to worker errors (from execution path), worker events (from a partybus subscription), and
// signal interrupts. Is responsible for handling each event relative to a given UI an to coordinate eventing until
// an eventual graceful exit.
//
//nolint:funlen
func EventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *partybus.Subscription, cleanupFn func(), uxs ...ui.UI) error {
defer cleanupFn()
events := subscription.Events()
-29
View File
@@ -22,7 +22,6 @@ func Tasks(app *config.Application) ([]Task, error) {
generateCatalogFileMetadataTask,
generateCatalogFileDigestsTask,
generateCatalogSecretsTask,
generateCatalogFileClassificationsTask,
generateCatalogContentsTask,
}
@@ -162,34 +161,6 @@ func generateCatalogSecretsTask(app *config.Application) (Task, error) {
return task, nil
}
func generateCatalogFileClassificationsTask(app *config.Application) (Task, error) {
if !app.FileClassification.Cataloger.Enabled {
return nil, nil
}
// TODO: in the future we could expose out the classifiers via configuration
classifierCataloger, err := file.NewClassificationCataloger(file.DefaultClassifiers)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(app.FileClassification.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := classifierCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.FileClassifications = result
return nil, nil
}
return task, nil
}
func generateCatalogContentsTask(app *config.Application) (Task, error) {
if !app.FileContents.Cataloger.Enabled {
return nil, nil
+16 -9
View File
@@ -1,28 +1,35 @@
package options
import (
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/cyclonedxxml"
"github.com/anchore/syft/syft/formats/github"
"github.com/anchore/syft/syft/formats/spdxjson"
"github.com/anchore/syft/syft/formats/spdxtagvalue"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/formats/text"
"github.com/anchore/syft/syft/sbom"
)
func FormatAliases(ids ...sbom.FormatID) (aliases []string) {
for _, id := range ids {
switch id {
case syft.JSONFormatID:
case syftjson.ID:
aliases = append(aliases, "syft-json")
case syft.TextFormatID:
case text.ID:
aliases = append(aliases, "text")
case syft.TableFormatID:
case table.ID:
aliases = append(aliases, "table")
case syft.SPDXJSONFormatID:
case spdxjson.ID:
aliases = append(aliases, "spdx-json")
case syft.SPDXTagValueFormatID:
case spdxtagvalue.ID:
aliases = append(aliases, "spdx-tag-value")
case syft.CycloneDxXMLFormatID:
case cyclonedxxml.ID:
aliases = append(aliases, "cyclonedx-xml")
case syft.CycloneDxJSONFormatID:
case cyclonedxjson.ID:
aliases = append(aliases, "cyclonedx-json")
case syft.GitHubID:
case github.ID:
aliases = append(aliases, "github", "github-json")
default:
aliases = append(aliases, string(id))
+8
View File
@@ -21,6 +21,7 @@ type PackagesOptions struct {
Platform string
Exclude []string
Catalogers []string
Name string
}
var _ Interface = (*PackagesOptions)(nil)
@@ -47,6 +48,9 @@ func (o *PackagesOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.Flags().StringArrayVarP(&o.Catalogers, "catalogers", "", nil,
"enable one or more package catalogers")
cmd.Flags().StringVarP(&o.Name, "name", "", "",
"set the name of the target being analyzed")
return bindPackageConfigOptions(cmd.Flags(), v)
}
@@ -69,6 +73,10 @@ func bindPackageConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
return err
}
if err := v.BindPFlag("name", flags.Lookup("name")); err != nil {
return err
}
if err := v.BindPFlag("output", flags.Lookup("output")); err != nil {
return err
}
+1 -1
View File
@@ -57,7 +57,7 @@ func Packages(v *viper.Viper, app *config.Application, ro *options.RootOptions,
}),
Args: func(cmd *cobra.Command, args []string) error {
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
return fmt.Errorf("invalid application config: %w", err)
}
// configure logging for command
newLogWrapper(app)
+1 -1
View File
@@ -42,7 +42,7 @@ func Run(ctx context.Context, app *config.Application, args []string) error {
// could be an image or a directory, with or without a scheme
userInput := args[0]
si, err := source.ParseInput(userInput, app.Platform, true)
si, err := source.ParseInputWithName(userInput, app.Platform, true, app.Name)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
+1 -1
View File
@@ -28,7 +28,7 @@ func PowerUser(v *viper.Viper, app *config.Application, ro *options.RootOptions)
}),
Args: func(cmd *cobra.Command, args []string) error {
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
return fmt.Errorf("invalid application config: %w", err)
}
// configure logging for command
newLogWrapper(app)
+1 -1
View File
@@ -46,7 +46,7 @@ func Run(ctx context.Context, app *config.Application, args []string) error {
}()
userInput := args[0]
si, err := source.ParseInput(userInput, app.Platform, true)
si, err := source.ParseInputWithName(userInput, app.Platform, true, app.Name)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
+3 -3
View File
@@ -7,7 +7,6 @@ require (
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
github.com/acobaugh/osrelease v0.1.0
github.com/adrg/xdg v0.3.3
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
@@ -33,7 +32,7 @@ require (
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e
github.com/sergi/go-diff v1.2.0
github.com/sirupsen/logrus v1.9.0
github.com/spdx/tools-golang v0.2.0
github.com/spdx/tools-golang v0.3.1-0.20221108182156-8a01147e6342
github.com/spf13/afero v1.8.2
github.com/spf13/cobra v1.6.0
github.com/spf13/pflag v1.0.5
@@ -53,10 +52,11 @@ require (
require (
github.com/Masterminds/sprig/v3 v3.2.2
github.com/anchore/go-logger v0.0.0-20220728155337-03b66a5207d8
github.com/anchore/stereoscope v0.0.0-20221006201143-d24c9d626b33
github.com/anchore/stereoscope v0.0.0-20221130153459-3b80d983223f
github.com/docker/docker v20.10.17+incompatible
github.com/google/go-containerregistry v0.11.0
github.com/in-toto/in-toto-golang v0.4.1-0.20221018183522-731d0640b65f
github.com/invopop/jsonschema v0.7.0
github.com/knqyf263/go-rpmdb v0.0.0-20221030135625-4082a22221ce
github.com/opencontainers/go-digest v1.0.0
github.com/sassoftware/go-rpmutils v0.2.0
+7 -6
View File
@@ -223,8 +223,6 @@ github.com/adrg/xdg v0.3.3 h1:s/tV7MdqQnzB1nKY8aqHvAMD+uCiuEDzVB5HLRY849U=
github.com/adrg/xdg v0.3.3/go.mod h1:61xAR2VZcggl2St4O9ohF5qCKe08+JDmE4VNzPFQvOQ=
github.com/afex/hystrix-go v0.0.0-20180502004556-fa1af6a1f4f5/go.mod h1:SkGFH1ia65gfNATL8TAiHDNxPzPdmEL5uirI2Uyuz6c=
github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7/go.mod h1:6zEj6s6u/ghQa61ZWa/C2Aw3RkjiTBOix7dkqa1VLIs=
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074 h1:Lw9q+WyJLFOR+AULchS5/2GKfM+6gOh4szzizdfH3MU=
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074/go.mod h1:/n6+1/DWPltRLWL/VKyUxg6tzsl5kHUCcraimt4vr60=
github.com/alecthomas/kingpin v2.2.6+incompatible/go.mod h1:59OFYbFVLKQKq+mqrL6Rw5bR0c3ACQaawgXx0QYndlE=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
@@ -282,8 +280,8 @@ github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZV
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/packageurl-go v0.1.1-0.20220428202044-a072fa3cb6d7 h1:kDrYkTSM9uIxaX/P9s0F4nKYNM+hnSgLJdLpqvsaQ/g=
github.com/anchore/packageurl-go v0.1.1-0.20220428202044-a072fa3cb6d7/go.mod h1:Blo6OgJNiYF41ufcgHKkbCKF2MDOMlrqhXv/ij6ocR4=
github.com/anchore/stereoscope v0.0.0-20221006201143-d24c9d626b33 h1:Y+9aMJMTEMH+kJJFEaxqrF6X1t9CkjpWjOzaCo1q5vM=
github.com/anchore/stereoscope v0.0.0-20221006201143-d24c9d626b33/go.mod h1:WOWtswyDxLkBnizq9LihYa9uw88r1FhBvJo7g//Ljcc=
github.com/anchore/stereoscope v0.0.0-20221130153459-3b80d983223f h1:JB4foD0R//XJ6oAtjK9h9ABjHhmzUAakjQRKMckPhnA=
github.com/anchore/stereoscope v0.0.0-20221130153459-3b80d983223f/go.mod h1:Oa0EpewvxiynI8zxLGj2SZ6gSoGtBPQrbZNBrYNBsvE=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
github.com/andybalholm/brotli v1.0.2/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
@@ -1220,6 +1218,8 @@ github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7P
github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/influxdata/influxdb1-client v0.0.0-20191209144304-8bf82d3c094d/go.mod h1:qj24IKcXYK6Iy9ceXlo3Tc+vtHo9lIhSX5JddghvEPo=
github.com/intel/goresctrl v0.2.0/go.mod h1:+CZdzouYFn5EsxgqAQTEzMfwKwuc0fVdMrT9FCCAVRQ=
github.com/invopop/jsonschema v0.7.0 h1:2vgQcBz1n256N+FpX3Jq7Y17AjYt46Ig3zIWyy770So=
github.com/invopop/jsonschema v0.7.0/go.mod h1:O9uiLokuu0+MGFlyiaqtWxwqJm41/+8Nj0lD7A36YH0=
github.com/j-keck/arping v0.0.0-20160618110441-2cf9dc699c56/go.mod h1:ymszkNOg6tORTn+6F6j+Jc8TOr5osrynvN6ivFWZ2GA=
github.com/j-keck/arping v1.0.2/go.mod h1:aJbELhR92bSk7tp79AWM/ftfc90EfEi2bQJrbBFOsPw=
github.com/jarcoal/httpmock v1.0.5/go.mod h1:ATjnClrvW/3tijVmpL/va5Z3aAyGvqU3gCT8nX0Txik=
@@ -1376,6 +1376,7 @@ github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope
github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.11/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-ieproxy v0.0.0-20190610004146-91bb50d98149/go.mod h1:31jz6HNzdxOmlERGGEc4v/dMssOfmp2p5bT/okiKFFc=
@@ -1817,8 +1818,8 @@ github.com/sony/gobreaker v0.4.1/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJ
github.com/sourcegraph/go-diff v0.6.1/go.mod h1:iBszgVvyxdc8SFZ7gm69go2KDdt3ag071iBaWPF6cjs=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spdx/gordf v0.0.0-20201111095634-7098f93598fb/go.mod h1:uKWaldnbMnjsSAXRurWqqrdyZen1R7kxl8TkmWk2OyM=
github.com/spdx/tools-golang v0.2.0 h1:KBNcw7xvVycRWeCWZK/5xQJA+plymW1+rTCs8ekJDro=
github.com/spdx/tools-golang v0.2.0/go.mod h1:RO4Y3IFROJnz+43JKm1YOrbtgQNljW4gAPpA/sY2eqo=
github.com/spdx/tools-golang v0.3.1-0.20221108182156-8a01147e6342 h1:6uvaOTv4GeRqQV6O1/znbpziqhctMRLTy3OGeZrNMic=
github.com/spdx/tools-golang v0.3.1-0.20221108182156-8a01147e6342/go.mod h1:VHzvNsKAfAGqs4ZvwRL+7a0dNsL20s7lGui4K9C0xQM=
github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=
github.com/spf13/afero v1.3.3/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4=
+6 -3
View File
@@ -55,6 +55,7 @@ type Application struct {
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
Name string `yaml:"name" json:"name" mapstructure:"name"`
}
func (cfg Application) ToCatalogerConfig() cataloger.Config {
@@ -74,9 +75,11 @@ func (cfg *Application) LoadAllValues(v *viper.Viper, configPath string) error {
// check if user specified config; otherwise read all possible paths
if err := loadConfig(v, configPath); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); ok {
// Not Found; ignore this error
log.Debug("no config file found; proceeding with defaults")
var notFound *viper.ConfigFileNotFoundError
if errors.As(err, &notFound) {
log.Debugf("no config file found, using defaults")
} else {
return fmt.Errorf("unable to load config: %w", err)
}
}
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "4.1.0"
JSONSchemaVersion = "6.0.0"
)
+6 -5
View File
@@ -1,6 +1,7 @@
package internal
import (
"errors"
"fmt"
"io"
"os"
@@ -26,14 +27,14 @@ func (e ErrPath) Error() string {
}
func IsErrPath(err error) bool {
_, ok := err.(ErrPath)
return ok
var pathErr ErrPath
return errors.As(err, &pathErr)
}
func IsErrPathPermission(err error) bool {
pathErr, ok := err.(ErrPath)
if ok {
var pathErr ErrPath
if errors.As(err, &pathErr) {
return os.IsPermission(pathErr.Err)
}
return ok
return false
}
+1 -1
View File
@@ -106,7 +106,7 @@ func findArchiveStartOffset(r io.ReaderAt, size int64) (startOfArchive uint64, e
bLen = size
}
buf = make([]byte, int(bLen))
if _, err := r.ReadAt(buf, size-bLen); err != nil && err != io.EOF {
if _, err := r.ReadAt(buf, size-bLen); err != nil && !errors.Is(err, io.EOF) {
return 0, err
}
if p := findSignatureInBlock(buf); p >= 0 {
@@ -47,12 +47,12 @@ func main() {
func run() error {
resp, err := http.Get(url)
if err != nil {
return fmt.Errorf("unable to get licenses list: %+v", err)
return fmt.Errorf("unable to get licenses list: %w", err)
}
var result LicenseList
if err = json.NewDecoder(resp.Body).Decode(&result); err != nil {
return fmt.Errorf("unable to decode license list: %+v", err)
return fmt.Errorf("unable to decode license list: %w", err)
}
defer func() {
if err := resp.Body.Close(); err != nil {
@@ -62,7 +62,7 @@ func run() error {
f, err := os.Create(source)
if err != nil {
return fmt.Errorf("unable to create %q: %+v", source, err)
return fmt.Errorf("unable to create %q: %w", source, err)
}
defer func() {
if err := f.Close(); err != nil {
@@ -85,7 +85,7 @@ func run() error {
})
if err != nil {
return fmt.Errorf("unable to generate template: %+v", err)
return fmt.Errorf("unable to generate template: %w", err)
}
return nil
}
+1 -1
View File
@@ -18,7 +18,7 @@ func handleExit(event partybus.Event) error {
}
if err := fn(); err != nil {
return fmt.Errorf("unable to show package catalog report: %v", err)
return fmt.Errorf("unable to show package catalog report: %w", err)
}
return nil
}
+22 -21
View File
@@ -10,7 +10,7 @@ import (
"sort"
"strings"
"github.com/alecthomas/jsonschema"
"github.com/invopop/jsonschema"
"github.com/anchore/syft/internal"
syftjsonModel "github.com/anchore/syft/syft/formats/syftjson/model"
@@ -28,24 +28,25 @@ can be extended to include specific package metadata struct shapes in the future
// When a new package metadata definition is created it will need to be manually added here. The variable name does
// not matter as long as it is exported.
type artifactMetadataContainer struct {
Apk pkg.ApkMetadata
Alpm pkg.AlpmMetadata
Dpkg pkg.DpkgMetadata
Gem pkg.GemMetadata
Java pkg.JavaMetadata
Npm pkg.NpmPackageJSONMetadata
Python pkg.PythonPackageMetadata
Rpm pkg.RpmMetadata
Cargo pkg.CargoPackageMetadata
Go pkg.GolangBinMetadata
Php pkg.PhpComposerJSONMetadata
Dart pkg.DartPubMetadata
Dotnet pkg.DotnetDepsMetadata
Portage pkg.PortageMetadata
Conan pkg.ConanMetadata
ConanLock pkg.ConanLockMetadata
KbPackage pkg.KbPackageMetadata
Hackage pkg.HackageMetadata
Apk pkg.ApkMetadata
Alpm pkg.AlpmMetadata
Dpkg pkg.DpkgMetadata
Gem pkg.GemMetadata
Java pkg.JavaMetadata
Npm pkg.NpmPackageJSONMetadata
Python pkg.PythonPackageMetadata
Rpm pkg.RpmMetadata
Cargo pkg.CargoPackageMetadata
Go pkg.GolangBinMetadata
Php pkg.PhpComposerJSONMetadata
Dart pkg.DartPubMetadata
Dotnet pkg.DotnetDepsMetadata
Portage pkg.PortageMetadata
Conan pkg.ConanMetadata
ConanLock pkg.ConanLockMetadata
KbPackage pkg.KbPackageMetadata
Hackage pkg.HackageMetadata
SwiftCocopods pkg.CocoapodsMetadata
}
func main() {
@@ -55,7 +56,7 @@ func main() {
func build() *jsonschema.Schema {
reflector := &jsonschema.Reflector{
AllowAdditionalProperties: true,
TypeNamer: func(r reflect.Type) string {
Namer: func(r reflect.Type) string {
return strings.TrimPrefix(r.Name(), "JSON")
},
}
@@ -87,7 +88,7 @@ func build() *jsonschema.Schema {
}
for _, name := range metadataNames {
metadataTypes = append(metadataTypes, map[string]string{
"$ref": fmt.Sprintf("#/definitions/%s", name),
"$ref": fmt.Sprintf("#/$defs/%s", name),
})
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -19,7 +19,7 @@ func IDByHash(obj interface{}) (ID, error) {
SlicesAsSets: true,
})
if err != nil {
return "", fmt.Errorf("could not build ID for object=%+v: %+v", obj, err)
return "", fmt.Errorf("could not build ID for object=%+v: %w", obj, err)
}
return ID(fmt.Sprintf("%x", f)), nil
+3 -9
View File
@@ -10,17 +10,11 @@ const (
// ContainsRelationship (supports any-to-any linkages) is a proxy for the SPDX 2.2 CONTAINS relationship.
ContainsRelationship RelationshipType = "contains"
// RuntimeDependencyOfRelationship is a proxy for the SPDX 2.2.1 RUNTIME_DEPENDENCY_OF relationship.
RuntimeDependencyOfRelationship RelationshipType = "runtime-dependency-of"
// DevDependencyOfRelationship is a proxy for the SPDX 2.2.1 DEV_DEPENDENCY_OF relationship.
DevDependencyOfRelationship RelationshipType = "dev-dependency-of"
// BuildDependencyOfRelationship is a proxy for the SPDX 2.2.1 BUILD_DEPENDENCY_OF relationship.
BuildDependencyOfRelationship RelationshipType = "build-dependency-of"
// DependencyOfRelationship is a proxy for the SPDX 2.2.1 DEPENDENCY_OF relationship.
DependencyOfRelationship RelationshipType = "dependency-of"
// DescribedByRelationship is a proxy for the SPDX 2.2.2 DESCRIBED_BY relationship.
DescribedByRelationship RelationshipType = "described-by"
)
type RelationshipType string
+5 -23
View File
@@ -1,36 +1,18 @@
package syft
import (
"bytes"
"fmt"
"io"
"github.com/anchore/syft/syft/formats"
"github.com/anchore/syft/syft/sbom"
)
// Encode takes all SBOM elements and a format option and encodes an SBOM document.
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func Encode(s sbom.SBOM, f sbom.Format) ([]byte, error) {
buff := bytes.Buffer{}
if err := f.Encode(&buff, s); err != nil {
return nil, fmt.Errorf("unable to encode sbom: %w", err)
}
return buff.Bytes(), nil
return formats.Encode(s, f)
}
// Decode takes a reader for an SBOM and generates all internal SBOM elements.
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func Decode(reader io.Reader) (*sbom.SBOM, sbom.Format, error) {
by, err := io.ReadAll(reader)
if err != nil {
return nil, nil, fmt.Errorf("unable to read sbom: %w", err)
}
f := IdentifyFormat(by)
if f == nil {
return nil, nil, fmt.Errorf("unable to identify format")
}
s, err := f.Decode(bytes.NewReader(by))
return s, f, err
return formats.Decode(reader)
}
-38
View File
@@ -1,38 +0,0 @@
package file
import (
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/source"
)
type ClassificationCataloger struct {
classifiers []Classifier
}
func NewClassificationCataloger(classifiers []Classifier) (*ClassificationCataloger, error) {
return &ClassificationCataloger{
classifiers: classifiers,
}, nil
}
func (i *ClassificationCataloger) Catalog(resolver source.FileResolver) (map[source.Coordinates][]Classification, error) {
results := make(map[source.Coordinates][]Classification)
numResults := 0
for _, location := range allRegularFiles(resolver) {
for _, classifier := range i.classifiers {
result, err := classifier.Classify(resolver, location)
if err != nil {
log.Warnf("file classification cataloger failed with class=%q at location=%+v: %+v", classifier.Class, location, err)
continue
}
if result != nil {
results[location.Coordinates] = append(results[location.Coordinates], *result)
numResults++
}
}
}
log.Debugf("file classifier discovered %d results", numResults)
return results, nil
}
-210
View File
@@ -1,210 +0,0 @@
package file
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/anchore/stereoscope/pkg/imagetest"
"github.com/anchore/syft/syft/source"
)
func TestClassifierCataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
tests := []struct {
name string
fixtureDir string
location string
expected []Classification
expectedErr func(assert.TestingT, error, ...interface{}) bool
}{
{
name: "positive-libpython3.7.so",
fixtureDir: "test-fixtures/classifiers/positive",
location: "libpython3.7.so",
expected: []Classification{
{
Class: "python-binary",
Metadata: map[string]string{
"version": "3.7.4a-vZ9",
},
},
},
expectedErr: assert.NoError,
},
{
name: "positive-python3.6",
fixtureDir: "test-fixtures/classifiers/positive",
location: "python3.6",
expected: []Classification{
{
Class: "python-binary",
Metadata: map[string]string{
"version": "3.6.3a-vZ9",
},
},
},
expectedErr: assert.NoError,
},
{
name: "positive-patchlevel.h",
fixtureDir: "test-fixtures/classifiers/positive",
location: "patchlevel.h",
expected: []Classification{
{
Class: "cpython-source",
Metadata: map[string]string{
"version": "3.9-aZ5",
},
},
},
expectedErr: assert.NoError,
},
{
name: "positive-go",
fixtureDir: "test-fixtures/classifiers/positive",
location: "go",
expected: []Classification{
{
Class: "go-binary",
Metadata: map[string]string{
"version": "1.14",
},
},
},
expectedErr: assert.NoError,
},
{
name: "positive-go-hint",
fixtureDir: "test-fixtures/classifiers/positive",
location: "VERSION",
expected: []Classification{
{
Class: "go-binary-hint",
Metadata: map[string]string{
"version": "1.15",
},
},
},
expectedErr: assert.NoError,
},
{
name: "positive-busybox",
fixtureDir: "test-fixtures/classifiers/positive",
location: "[", // note: busybox is a link to [
expected: []Classification{
{
Class: "busybox-binary",
VirtualPath: "busybox",
Metadata: map[string]string{
"version": "3.33.3",
},
},
},
expectedErr: assert.NoError,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
c, err := NewClassificationCataloger(DefaultClassifiers)
test.expectedErr(t, err)
src, err := source.NewFromDirectory(test.fixtureDir)
test.expectedErr(t, err)
resolver, err := src.FileResolver(source.SquashedScope)
test.expectedErr(t, err)
actualResults, err := c.Catalog(resolver)
test.expectedErr(t, err)
ok := false
for actualLoc, actualClassification := range actualResults {
if test.location == actualLoc.RealPath {
ok = true
assert.Equal(t, test.expected, actualClassification)
}
}
if !ok {
t.Fatalf("could not find test location=%q", test.location)
}
})
}
}
func TestClassifierCataloger_DefaultClassifiers_PositiveCases_Image(t *testing.T) {
tests := []struct {
name string
fixtureImage string
location string
expected []Classification
expectedErr func(assert.TestingT, error, ...interface{}) bool
}{
{
name: "busybox-regression",
fixtureImage: "image-busybox",
location: "/bin/[",
expected: []Classification{
{
Class: "busybox-binary",
VirtualPath: "/bin/busybox",
Metadata: map[string]string{
"version": "1.35.0",
},
},
},
expectedErr: assert.NoError,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
c, err := NewClassificationCataloger(DefaultClassifiers)
test.expectedErr(t, err)
img := imagetest.GetFixtureImage(t, "docker-archive", test.fixtureImage)
src, err := source.NewFromImage(img, "test-img")
test.expectedErr(t, err)
resolver, err := src.FileResolver(source.SquashedScope)
test.expectedErr(t, err)
actualResults, err := c.Catalog(resolver)
test.expectedErr(t, err)
ok := false
for actuaLoc, actualClassification := range actualResults {
if actuaLoc.RealPath == test.location {
ok = true
assert.Equal(t, test.expected, actualClassification)
}
}
if !ok {
t.Fatalf("could not find test location=%q", test.location)
}
})
}
}
func TestClassifierCataloger_DefaultClassifiers_NegativeCases(t *testing.T) {
c, err := NewClassificationCataloger(DefaultClassifiers)
assert.NoError(t, err)
src, err := source.NewFromDirectory("test-fixtures/classifiers/negative")
assert.NoError(t, err)
resolver, err := src.FileResolver(source.SquashedScope)
assert.NoError(t, err)
actualResults, err := c.Catalog(resolver)
assert.NoError(t, err)
assert.Equal(t, 0, len(actualResults))
}
-153
View File
@@ -1,153 +0,0 @@
package file
import (
"bytes"
"fmt"
"io"
"regexp"
"text/template"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/source"
)
var DefaultClassifiers = []Classifier{
{
Class: "python-binary",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)python(?P<version>[0-9]+\.[0-9]+)$`),
regexp.MustCompile(`(.*/|^)libpython(?P<version>[0-9]+\.[0-9]+).so.*$`),
},
EvidencePatternTemplates: []string{
`(?m)(?P<version>{{ .version }}\.[0-9]+[-_a-zA-Z0-9]*)`,
},
},
{
Class: "cpython-source",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)patchlevel.h$`),
},
EvidencePatternTemplates: []string{
`(?m)#define\s+PY_VERSION\s+"?(?P<version>[0-9\.\-_a-zA-Z]+)"?`,
},
},
{
Class: "go-binary",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)go$`),
},
EvidencePatternTemplates: []string{
`(?m)go(?P<version>[0-9]+\.[0-9]+(\.[0-9]+|beta[0-9]+|alpha[0-9]+|rc[0-9]+)?)`,
},
},
{
Class: "nodejs-binary",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)node$`),
},
EvidencePatternTemplates: []string{
// regex that matches node.js/vx.y.z
`(?m)node\.js\/v(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`,
},
},
{
Class: "go-binary-hint",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)VERSION$`),
},
EvidencePatternTemplates: []string{
`(?m)go(?P<version>[0-9]+\.[0-9]+(\.[0-9]+|beta[0-9]+|alpha[0-9]+|rc[0-9]+)?)`,
},
},
{
Class: "busybox-binary",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)busybox$`),
},
EvidencePatternTemplates: []string{
`(?m)BusyBox\s+v(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`,
},
},
}
type Classifier struct {
Class string
FilepathPatterns []*regexp.Regexp
EvidencePatternTemplates []string
}
type Classification struct {
Class string `json:"class"`
VirtualPath string `json:"virtual_path"`
Metadata map[string]string `json:"metadata"`
}
func (c Classifier) Classify(resolver source.FileResolver, location source.Location) (*Classification, error) {
doesFilepathMatch, filepathNamedGroupValues := FilepathMatches(c.FilepathPatterns, location)
if !doesFilepathMatch {
return nil, nil
}
contentReader, err := resolver.FileContentsByLocation(location)
if err != nil {
return nil, err
}
defer internal.CloseAndLogError(contentReader, location.VirtualPath)
// TODO: there is room for improvement here, as this may use an excessive amount of memory. Alternate approach is to leverage a RuneReader.
contents, err := io.ReadAll(contentReader)
if err != nil {
return nil, err
}
var result *Classification
for _, patternTemplate := range c.EvidencePatternTemplates {
tmpl, err := template.New("").Parse(patternTemplate)
if err != nil {
return nil, fmt.Errorf("unable to parse classifier template=%q : %w", patternTemplate, err)
}
patternBuf := &bytes.Buffer{}
err = tmpl.Execute(patternBuf, filepathNamedGroupValues)
if err != nil {
return nil, fmt.Errorf("unable to render template: %w", err)
}
pattern, err := regexp.Compile(patternBuf.String())
if err != nil {
return nil, fmt.Errorf("unable to compile rendered regex=%q: %w", patternBuf.String(), err)
}
if !pattern.Match(contents) {
continue
}
matchMetadata := internal.MatchNamedCaptureGroups(pattern, string(contents))
if result == nil {
result = &Classification{
Class: c.Class,
VirtualPath: location.VirtualPath,
Metadata: matchMetadata,
}
} else {
for key, value := range matchMetadata {
result.Metadata[key] = value
}
}
}
return result, nil
}
func FilepathMatches(patterns []*regexp.Regexp, location source.Location) (bool, map[string]string) {
for _, path := range []string{location.RealPath, location.VirtualPath} {
if path == "" {
continue
}
for _, pattern := range patterns {
if pattern.MatchString(path) {
return true, internal.MatchNamedCaptureGroups(pattern, path)
}
}
}
return false, nil
}
-97
View File
@@ -1,97 +0,0 @@
package file
import (
"regexp"
"testing"
"github.com/stretchr/testify/assert"
"github.com/anchore/syft/syft/source"
)
func TestFilepathMatches(t *testing.T) {
tests := []struct {
name string
location source.Location
patterns []string
expectedMatches bool
expectedNamedGroups map[string]string
}{
{
name: "simple-filename-match",
location: source.Location{
Coordinates: source.Coordinates{
RealPath: "python2.7",
},
},
patterns: []string{
`python([0-9]+\.[0-9]+)$`,
},
expectedMatches: true,
},
{
name: "filepath-match",
location: source.Location{
Coordinates: source.Coordinates{
RealPath: "/usr/bin/python2.7",
},
},
patterns: []string{
`python([0-9]+\.[0-9]+)$`,
},
expectedMatches: true,
},
{
name: "virtual-filepath-match",
location: source.Location{
VirtualPath: "/usr/bin/python2.7",
},
patterns: []string{
`python([0-9]+\.[0-9]+)$`,
},
expectedMatches: true,
},
{
name: "full-filepath-match",
location: source.Location{
VirtualPath: "/usr/bin/python2.7",
},
patterns: []string{
`.*/bin/python([0-9]+\.[0-9]+)$`,
},
expectedMatches: true,
},
{
name: "anchored-filename-match-FAILS",
location: source.Location{
Coordinates: source.Coordinates{
RealPath: "/usr/bin/python2.7",
},
},
patterns: []string{
`^python([0-9]+\.[0-9]+)$`,
},
expectedMatches: false,
},
{
name: "empty-filename-match-FAILS",
location: source.Location{},
patterns: []string{
`^python([0-9]+\.[0-9]+)$`,
},
expectedMatches: false,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
var patterns []*regexp.Regexp
for _, p := range test.patterns {
patterns = append(patterns, regexp.MustCompile(p))
}
actualMatches, actualNamedGroups := FilepathMatches(patterns, test.location)
assert.Equal(t, test.expectedMatches, actualMatches)
assert.Equal(t, test.expectedNamedGroups, actualNamedGroups)
})
}
}
+15 -76
View File
@@ -1,14 +1,12 @@
package syft
import (
"bytes"
"strings"
"github.com/anchore/syft/syft/formats"
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/cyclonedxxml"
"github.com/anchore/syft/syft/formats/github"
"github.com/anchore/syft/syft/formats/spdx22json"
"github.com/anchore/syft/syft/formats/spdx22tagvalue"
"github.com/anchore/syft/syft/formats/spdxjson"
"github.com/anchore/syft/syft/formats/spdxtagvalue"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/formats/template"
@@ -17,94 +15,35 @@ import (
)
// these have been exported for the benefit of API users
// TODO: deprecated: now that the formats package has been moved to syft/formats, will be removed in v1.0.0
const (
JSONFormatID = syftjson.ID
TextFormatID = text.ID
TableFormatID = table.ID
CycloneDxXMLFormatID = cyclonedxxml.ID
CycloneDxJSONFormatID = cyclonedxjson.ID
GitHubID = github.ID
SPDXTagValueFormatID = spdx22tagvalue.ID
SPDXJSONFormatID = spdx22json.ID
GitHubFormatID = github.ID
SPDXTagValueFormatID = spdxtagvalue.ID
SPDXJSONFormatID = spdxjson.ID
TemplateFormatID = template.ID
)
var formats []sbom.Format
func init() {
formats = []sbom.Format{
syftjson.Format(),
cyclonedxxml.Format(),
cyclonedxjson.Format(),
github.Format(),
spdx22tagvalue.Format(),
spdx22json.Format(),
table.Format(),
text.Format(),
template.Format(),
}
}
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func FormatIDs() (ids []sbom.FormatID) {
for _, f := range formats {
ids = append(ids, f.ID())
}
return ids
return formats.IDs()
}
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func FormatByID(id sbom.FormatID) sbom.Format {
for _, f := range formats {
if f.ID() == id {
return f
}
}
return nil
return formats.ByID(id)
}
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func FormatByName(name string) sbom.Format {
cleanName := cleanFormatName(name)
for _, f := range formats {
if cleanFormatName(string(f.ID())) == cleanName {
return f
}
}
// handle any aliases for any supported format
switch cleanName {
case "json", "syftjson":
return FormatByID(syftjson.ID)
case "cyclonedx", "cyclone", "cyclonedxxml":
return FormatByID(cyclonedxxml.ID)
case "cyclonedxjson":
return FormatByID(cyclonedxjson.ID)
case "github", "githubjson":
return FormatByID(github.ID)
case "spdx", "spdxtv", "spdxtagvalue":
return FormatByID(spdx22tagvalue.ID)
case "spdxjson":
return FormatByID(spdx22json.ID)
case "table":
return FormatByID(table.ID)
case "text":
return FormatByID(text.ID)
case "template":
FormatByID(template.ID)
}
return nil
}
func cleanFormatName(name string) string {
r := strings.NewReplacer("-", "", "_", "")
return strings.ToLower(r.Replace(name))
return formats.ByName(name)
}
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func IdentifyFormat(by []byte) sbom.Format {
for _, f := range formats {
if err := f.Validate(bytes.NewReader(by)); err != nil {
continue
}
return f
}
return nil
return formats.Identify(by)
}
@@ -2,6 +2,7 @@ package cyclonedxhelpers
import (
"fmt"
"reflect"
"testing"
"github.com/CycloneDX/cyclonedx-go"
@@ -31,20 +32,21 @@ func Test_encodeComponentProperties(t *testing.T) {
source.Location{Coordinates: source.Coordinates{RealPath: "test"}},
),
Metadata: pkg.ApkMetadata{
Package: "libc-utils",
OriginPackage: "libc-dev",
Maintainer: "Natanael Copa <ncopa@alpinelinux.org>",
Version: "0.7.2-r0",
License: "BSD",
Architecture: "x86_64",
URL: "http://alpinelinux.org",
Description: "Meta package to pull in correct libc",
Size: 0,
InstalledSize: 4096,
PullDependencies: "musl-utils",
PullChecksum: "Q1p78yvTLG094tHE1+dToJGbmYzQE=",
GitCommitOfAport: "97b1c2842faa3bfa30f5811ffbf16d5ff9f1a479",
Files: []pkg.ApkFileRecord{},
Package: "libc-utils",
OriginPackage: "libc-dev",
Maintainer: "Natanael Copa <ncopa@alpinelinux.org>",
Version: "0.7.2-r0",
License: "BSD",
Architecture: "x86_64",
URL: "http://alpinelinux.org",
Description: "Meta package to pull in correct libc",
Size: 0,
InstalledSize: 4096,
Dependencies: []string{"musl-utils"},
Provides: []string{"so:libc.so.1"},
Checksum: "Q1p78yvTLG094tHE1+dToJGbmYzQE=",
GitCommit: "97b1c2842faa3bfa30f5811ffbf16d5ff9f1a479",
Files: []pkg.ApkFileRecord{},
},
},
expected: &[]cyclonedx.Property{
@@ -53,8 +55,9 @@ func Test_encodeComponentProperties(t *testing.T) {
{Name: "syft:metadata:gitCommitOfApkPort", Value: "97b1c2842faa3bfa30f5811ffbf16d5ff9f1a479"},
{Name: "syft:metadata:installedSize", Value: "4096"},
{Name: "syft:metadata:originPackage", Value: "libc-dev"},
{Name: "syft:metadata:provides:0", Value: "so:libc.so.1"},
{Name: "syft:metadata:pullChecksum", Value: "Q1p78yvTLG094tHE1+dToJGbmYzQE="},
{Name: "syft:metadata:pullDependencies", Value: "musl-utils"},
{Name: "syft:metadata:pullDependencies:0", Value: "musl-utils"},
{Name: "syft:metadata:size", Value: "0"},
},
},
@@ -198,6 +201,7 @@ func Test_decodeComponent(t *testing.T) {
component cyclonedx.Component
wantLanguage pkg.Language
wantMetadataType pkg.MetadataType
wantMetadata interface{}
}{
{
name: "derive language from pURL if missing",
@@ -211,7 +215,7 @@ func Test_decodeComponent(t *testing.T) {
wantLanguage: pkg.Java,
},
{
name: "handle existing RpmdbMetadata type",
name: "handle RpmdbMetadata type without properties",
component: cyclonedx.Component{
Name: "acl",
Version: "2.2.53-1.el8",
@@ -226,6 +230,31 @@ func Test_decodeComponent(t *testing.T) {
},
},
wantMetadataType: pkg.RpmMetadataType,
wantMetadata: pkg.RpmMetadata{},
},
{
name: "handle RpmdbMetadata type with properties",
component: cyclonedx.Component{
Name: "acl",
Version: "2.2.53-1.el8",
PackageURL: "pkg:rpm/centos/acl@2.2.53-1.el8?arch=x86_64&upstream=acl-2.2.53-1.el8.src.rpm&distro=centos-8",
Type: "library",
BOMRef: "pkg:rpm/centos/acl@2.2.53-1.el8?arch=x86_64&upstream=acl-2.2.53-1.el8.src.rpm&distro=centos-8",
Properties: &[]cyclonedx.Property{
{
Name: "syft:package:metadataType",
Value: "RpmMetadata",
},
{
Name: "syft:metadata:release",
Value: "some-release",
},
},
},
wantMetadataType: pkg.RpmMetadataType,
wantMetadata: pkg.RpmMetadata{
Release: "some-release",
},
},
}
@@ -238,6 +267,9 @@ func Test_decodeComponent(t *testing.T) {
if tt.wantMetadataType != "" {
assert.Equal(t, tt.wantMetadataType, p.MetadataType)
}
if tt.wantMetadata != nil {
assert.Truef(t, reflect.DeepEqual(tt.wantMetadata, p.Metadata), "metadata should match: %+v != %+v", tt.wantMetadata, p.Metadata)
}
})
}
}
+35 -13
View File
@@ -6,6 +6,7 @@ import (
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/formats/common"
"github.com/anchore/syft/syft/linux"
@@ -88,6 +89,10 @@ func collectPackages(component *cyclonedx.Component, s *sbom.SBOM, idMap map[str
case cyclonedx.ComponentTypeApplication, cyclonedx.ComponentTypeFramework, cyclonedx.ComponentTypeLibrary:
p := decodeComponent(component)
idMap[component.BOMRef] = p
syftID := extractSyftPacakgeID(component.BOMRef)
if syftID != "" {
idMap[syftID] = p
}
// TODO there must be a better way than needing to call this manually:
p.SetID()
s.Artifacts.PackageCatalog.Add(*p)
@@ -100,6 +105,19 @@ func collectPackages(component *cyclonedx.Component, s *sbom.SBOM, idMap map[str
}
}
func extractSyftPacakgeID(i string) string {
instance, err := packageurl.FromString(i)
if err != nil {
return ""
}
for _, q := range instance.Qualifiers {
if q.Key == "package-id" {
return q.Value
}
}
return ""
}
func linuxReleaseFromComponents(components []cyclonedx.Component) *linux.Release {
for i := range components {
component := &components[i]
@@ -188,21 +206,25 @@ func collectRelationships(bom *cyclonedx.BOM, s *sbom.SBOM, idMap map[string]int
return
}
for _, d := range *bom.Dependencies {
from, fromOk := idMap[d.Ref].(artifact.Identifiable)
if fromOk {
if d.Dependencies == nil {
from, fromExists := idMap[d.Ref].(artifact.Identifiable)
if !fromExists {
continue
}
if d.Dependencies == nil {
continue
}
for _, t := range *d.Dependencies {
to, toExists := idMap[t.Ref].(artifact.Identifiable)
if !toExists {
continue
}
for _, t := range *d.Dependencies {
to, toOk := idMap[t.Ref].(artifact.Identifiable)
if toOk {
s.Relationships = append(s.Relationships, artifact.Relationship{
From: from,
To: to,
Type: artifact.DependencyOfRelationship, // FIXME this information is lost
})
}
}
s.Relationships = append(s.Relationships, artifact.Relationship{
From: from,
To: to,
Type: artifact.DependencyOfRelationship, // FIXME this information is lost
})
}
}
}
@@ -10,7 +10,7 @@ import (
"github.com/anchore/syft/syft/pkg"
)
//nolint:funlen, gocognit
//nolint:gocognit
func encodeExternalReferences(p pkg.Package) *[]cyclonedx.ExternalReference {
var refs []cyclonedx.ExternalReference
if hasMetadata(p) {
+35 -12
View File
@@ -10,6 +10,7 @@ import (
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
@@ -91,13 +92,22 @@ func toOSComponent(distro *linux.Release) []cyclonedx.Component {
Name: distro.ID,
Version: distro.VersionID,
// TODO should we add a PURL?
CPE: distro.CPEName,
CPE: formatCPE(distro.CPEName),
ExternalReferences: eRefs,
Properties: properties,
},
}
}
func formatCPE(cpeString string) string {
cpe, err := pkg.NewCPE(cpeString)
if err != nil {
log.Debugf("skipping invalid CPE: %s", cpeString)
return ""
}
return pkg.CPEString(cpe)
}
// NewBomDescriptor returns a new BomDescriptor tailored for the current time and "syft" tool details.
func toBomDescriptor(name, version string, srcMetadata source.Metadata) *cyclonedx.Metadata {
return &cyclonedx.Metadata{
@@ -120,16 +130,11 @@ func toBomDescriptor(name, version string, srcMetadata source.Metadata) *cyclone
// An example of a relationship to not include would be: OwnershipByFileOverlapRelationship.
func isExpressiblePackageRelationship(ty artifact.RelationshipType) bool {
switch ty {
case artifact.RuntimeDependencyOfRelationship:
return true
case artifact.DevDependencyOfRelationship:
return true
case artifact.BuildDependencyOfRelationship:
return true
case artifact.DependencyOfRelationship:
return true
default:
return false
}
return false
}
func toDependencies(relationships []artifact.Relationship) []cyclonedx.Dependency {
@@ -141,10 +146,21 @@ func toDependencies(relationships []artifact.Relationship) []cyclonedx.Dependenc
continue
}
// we only capture package-to-package relationships for now
fromPkg, ok := r.From.(*pkg.Package)
if !ok {
continue
}
toPkg, ok := r.To.(*pkg.Package)
if !ok {
continue
}
innerDeps := []cyclonedx.Dependency{}
innerDeps = append(innerDeps, cyclonedx.Dependency{Ref: string(r.From.ID())})
innerDeps = append(innerDeps, cyclonedx.Dependency{Ref: deriveBomRef(*fromPkg)})
result = append(result, cyclonedx.Dependency{
Ref: string(r.To.ID()),
Ref: deriveBomRef(*toPkg),
Dependencies: &innerDeps,
})
}
@@ -152,8 +168,12 @@ func toDependencies(relationships []artifact.Relationship) []cyclonedx.Dependenc
}
func toBomDescriptorComponent(srcMetadata source.Metadata) *cyclonedx.Component {
name := srcMetadata.Name
switch srcMetadata.Scheme {
case source.ImageScheme:
if name == "" {
name = srcMetadata.ImageMetadata.UserInput
}
bomRef, err := artifact.IDByHash(srcMetadata.ImageMetadata.ID)
if err != nil {
log.Warnf("unable to get fingerprint of image metadata=%s: %+v", srcMetadata.ImageMetadata.ID, err)
@@ -161,10 +181,13 @@ func toBomDescriptorComponent(srcMetadata source.Metadata) *cyclonedx.Component
return &cyclonedx.Component{
BOMRef: string(bomRef),
Type: cyclonedx.ComponentTypeContainer,
Name: srcMetadata.ImageMetadata.UserInput,
Name: name,
Version: srcMetadata.ImageMetadata.ManifestDigest,
}
case source.DirectoryScheme, source.FileScheme:
if name == "" {
name = srcMetadata.Path
}
bomRef, err := artifact.IDByHash(srcMetadata.Path)
if err != nil {
log.Warnf("unable to get fingerprint of source metadata path=%s: %+v", srcMetadata.Path, err)
@@ -172,7 +195,7 @@ func toBomDescriptorComponent(srcMetadata source.Metadata) *cyclonedx.Component
return &cyclonedx.Component{
BOMRef: string(bomRef),
Type: cyclonedx.ComponentTypeFile,
Name: srcMetadata.Path,
Name: name,
}
}
@@ -0,0 +1,34 @@
package cyclonedxhelpers
import (
"testing"
"github.com/stretchr/testify/assert"
)
func Test_formatCPE(t *testing.T) {
tests := []struct {
cpe string
expected string
}{
{
cpe: "cpe:2.3:o:amazon:amazon_linux:2",
expected: "cpe:2.3:o:amazon:amazon_linux:2:*:*:*:*:*:*:*",
},
{
cpe: "cpe:/o:opensuse:leap:15.2",
expected: "cpe:2.3:o:opensuse:leap:15.2:*:*:*:*:*:*:*",
},
{
cpe: "invalid-cpe",
expected: "",
},
}
for _, test := range tests {
t.Run(test.cpe, func(t *testing.T) {
out := formatCPE(test.cpe)
assert.Equal(t, test.expected, out)
})
}
}
+55 -1
View File
@@ -130,6 +130,11 @@ func encode(out map[string]string, value reflect.Value, prefix string, fn FieldN
}
encode(out, pv, name, fn)
}
case reflect.Map:
// currently only map[string]string is really supported
for _, key := range value.MapKeys() {
encode(out, value.MapIndex(key), fmt.Sprintf("%s:%v", prefix, key.Interface()), fn)
}
default:
log.Warnf("skipping encoding of unsupported property: %s", prefix)
}
@@ -284,6 +289,55 @@ func decode(vals map[string]string, value reflect.Value, prefix string, fn Field
} else {
return false
}
case reflect.Map:
values := false
keyType := typ.Key()
valueType := typ.Elem()
outMap := reflect.MakeMap(typ)
str := fmt.Sprintf("%s:", prefix)
keyVals := map[string]string{}
// iterate through all keys to find those prefixed with a reference to this map
// NOTE: this will not work for nested maps
for key := range vals {
// test for map prefix
if strings.HasPrefix(key, str) {
keyVals[key] = strings.TrimPrefix(key, str)
// create new placeholder and decode key
newKeyType := keyType
if keyType.Kind() == reflect.Ptr {
newKeyType = keyType.Elem()
}
k := reflect.New(newKeyType)
if !decode(keyVals, k.Elem(), key, fn) {
log.Debugf("unable to decode key for: %s", key)
continue
}
if keyType.Kind() != reflect.Ptr {
k = k.Elem()
}
// create new placeholder and decode value
newValueType := valueType
if valueType.Kind() == reflect.Ptr {
newValueType = valueType.Elem()
}
v := reflect.New(newValueType)
if decode(vals, v.Elem(), key, fn) {
if valueType.Kind() != reflect.Ptr {
v = v.Elem()
}
// set in map
outMap.SetMapIndex(k, v)
values = true
}
}
}
if values {
value.Set(outMap)
} else {
return false
}
case reflect.Struct:
values := false
for i := 0; i < typ.NumField(); i++ {
@@ -309,7 +363,7 @@ func decode(vals map[string]string, value reflect.Value, prefix string, fn Field
func PtrToStruct(ptr interface{}) interface{} {
v := reflect.ValueOf(ptr)
if v.IsZero() {
if v.IsZero() && v.Type().Kind() != reflect.Struct {
return nil
}
switch v.Type().Kind() {
@@ -37,6 +37,10 @@ type T4 struct {
IntPtr *int
}
type T5 struct {
Map map[string]string
}
func Test_EncodeDecodeCycle(t *testing.T) {
val := 99
@@ -118,6 +122,15 @@ func Test_EncodeDecodeCycle(t *testing.T) {
{"t2 elem 1"},
},
},
{
name: "map of strings",
value: &T5{
Map: map[string]string{
"key1": "value1",
"key2": "value2",
},
},
},
}
for _, test := range tests {
@@ -1,30 +1,20 @@
package spdxhelpers
import (
"path"
"strings"
"github.com/anchore/syft/syft/source"
)
func DocumentName(srcMetadata source.Metadata) string {
if srcMetadata.Name != "" {
return srcMetadata.Name
}
switch srcMetadata.Scheme {
case source.ImageScheme:
return cleanName(srcMetadata.ImageMetadata.UserInput)
return srcMetadata.ImageMetadata.UserInput
case source.DirectoryScheme, source.FileScheme:
return cleanName(srcMetadata.Path)
return srcMetadata.Path
default:
return "unknown"
}
}
func cleanName(name string) string {
// remove # according to specification
name = strings.ReplaceAll(name, "#", "-")
// remove : for url construction
name = strings.ReplaceAll(name, ":", "-")
// clean relative pathing
return path.Clean(name)
}
@@ -35,7 +35,7 @@ func Test_DocumentName(t *testing.T) {
ManifestDigest: "digest",
},
},
expected: "image-repo/name-tag",
expected: "image-repo/name:tag",
},
{
name: "directory",
@@ -4,6 +4,7 @@ import (
"fmt"
"net/url"
"path"
"strings"
"github.com/google/uuid"
@@ -11,20 +12,27 @@ import (
"github.com/anchore/syft/syft/source"
)
const (
inputImage = "image"
inputDirectory = "dir"
inputFile = "file"
)
func DocumentNameAndNamespace(srcMetadata source.Metadata) (string, string) {
name := DocumentName(srcMetadata)
return name, DocumentNamespace(name, srcMetadata)
}
func DocumentNamespace(name string, srcMetadata source.Metadata) string {
name = cleanName(name)
input := "unknown-source-type"
switch srcMetadata.Scheme {
case source.ImageScheme:
input = "image"
input = inputImage
case source.DirectoryScheme:
input = "dir"
input = inputDirectory
case source.FileScheme:
input = "file"
input = inputFile
}
uniqueID := uuid.Must(uuid.NewRandom())
@@ -41,3 +49,13 @@ func DocumentNamespace(name string, srcMetadata source.Metadata) string {
return u.String()
}
// see: https://spdx.github.io/spdx-spec/v2.3/document-creation-information/#65-spdx-document-namespace-field
func cleanName(name string) string {
// remove # according to specification
name = strings.ReplaceAll(name, "#", "-")
// remove : for url construction
name = strings.ReplaceAll(name, ":", "-")
// clean relative pathing
return path.Clean(name)
}
@@ -4,7 +4,7 @@ type ReferenceCategory string
const (
SecurityReferenceCategory ReferenceCategory = "SECURITY"
PackageManagerReferenceCategory ReferenceCategory = "PACKAGE_MANAGER"
PackageManagerReferenceCategory ReferenceCategory = "PACKAGE-MANAGER"
OtherReferenceCategory ReferenceCategory = "OTHER"
)
@@ -109,7 +109,11 @@ func Test_Originator(t *testing.T) {
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, Originator(test.input))
typ, value := Originator(test.input)
if typ != "" {
value = typ + ": " + value
}
assert.Equal(t, test.expected, value)
})
}
}
+9 -6
View File
@@ -9,9 +9,11 @@ import (
// Originator needs to conform to the SPDX spec here:
// https://spdx.github.io/spdx-spec/package-information/#76-package-originator-field
// Available options are: <omit>, NOASSERTION, Person: <person>, Organization: <org>
func Originator(p pkg.Package) string {
// return values are: <type>, <value>
func Originator(p pkg.Package) (string, string) {
typ := ""
author := ""
if hasMetadata(p) {
author := ""
switch metadata := p.Metadata.(type) {
case pkg.ApkMetadata:
author = metadata.Maintainer
@@ -29,13 +31,14 @@ func Originator(p pkg.Package) string {
author = metadata.Authors[0]
}
case pkg.RpmMetadata:
return "Organization: " + metadata.Vendor
typ = "Organization"
author = metadata.Vendor
case pkg.DpkgMetadata:
author = metadata.Maintainer
}
if author != "" {
return "Person: " + author
if typ == "" && author != "" {
typ = "Person"
}
}
return ""
return typ, author
}
@@ -6,7 +6,6 @@ import (
"github.com/anchore/syft/syft/pkg"
)
//nolint:funlen
func SourceInfo(p pkg.Package) string {
answer := ""
switch p.Type {
@@ -47,6 +46,9 @@ func SourceInfo(p pkg.Package) string {
default:
answer = "acquired package info from the following paths"
}
if p.FoundBy == "sbom-cataloger" {
answer = "acquired package info from SBOM"
}
var paths []string
for _, l := range p.Locations.ToSlice() {
paths = append(paths, l.RealPath)
@@ -0,0 +1,464 @@
package spdxhelpers
import (
"fmt"
"sort"
"strings"
"time"
"github.com/spdx/tools-golang/spdx/common"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/formats/common/util"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
const (
spdxVersion = "SPDX-2.3"
noAssertion = "NOASSERTION"
)
// ToFormatModel creates and populates a new SPDX document struct that follows the SPDX 2.3
// spec from the given SBOM model.
//
//nolint:funlen
func ToFormatModel(s sbom.SBOM) *spdx.Document {
name, namespace := DocumentNameAndNamespace(s.Source)
return &spdx.Document{
// 6.1: SPDX Version; should be in the format "SPDX-x.x"
// Cardinality: mandatory, one
SPDXVersion: spdxVersion,
// 6.2: Data License; should be "CC0-1.0"
// Cardinality: mandatory, one
DataLicense: "CC0-1.0",
// 6.3: SPDX Identifier; should be "DOCUMENT" to represent mandatory identifier of SPDXRef-DOCUMENT
// Cardinality: mandatory, one
SPDXIdentifier: "DOCUMENT",
// 6.4: Document Name
// Cardinality: mandatory, one
DocumentName: name,
// 6.5: Document Namespace
// Cardinality: mandatory, one
// Purpose: Provide an SPDX document specific namespace as a unique absolute Uniform Resource
// Identifier (URI) as specified in RFC-3986, with the exception of the ‘#’ delimiter. The SPDX
// Document URI cannot contain a URI "part" (e.g. the "#" character), since the ‘#’ is used in SPDX
// element URIs (packages, files, snippets, etc) to separate the document namespace from the
// element’s SPDX identifier. Additionally, a scheme (e.g. “https:”) is required.
// The URI must be unique for the SPDX document including the specific version of the SPDX document.
// If the SPDX document is updated, thereby creating a new version, a new URI for the updated
// document must be used. There can only be one URI for an SPDX document and only one SPDX document
// for a given URI.
// Note that the URI does not have to be accessible. It is only intended to provide a unique ID.
// In many cases, the URI will point to a web accessible document, but this should not be assumed
// to be the case.
DocumentNamespace: namespace,
// 6.6: External Document References
// Cardinality: optional, one or many
ExternalDocumentReferences: nil,
// 6.11: Document Comment
// Cardinality: optional, one
DocumentComment: "",
CreationInfo: &spdx.CreationInfo{
// 6.7: License List Version
// Cardinality: optional, one
LicenseListVersion: spdxlicense.Version,
// 6.8: Creators: may have multiple keys for Person, Organization
// and/or Tool
// Cardinality: mandatory, one or many
Creators: []common.Creator{
{
Creator: "Anchore, Inc",
CreatorType: "Organization",
},
{
Creator: internal.ApplicationName + "-" + s.Descriptor.Version,
CreatorType: "Tool",
},
},
// 6.9: Created: data format YYYY-MM-DDThh:mm:ssZ
// Cardinality: mandatory, one
Created: time.Now().UTC().Format(time.RFC3339),
// 6.10: Creator Comment
// Cardinality: optional, one
CreatorComment: "",
},
Packages: toPackages(s.Artifacts.PackageCatalog),
Files: toFiles(s),
Relationships: toRelationships(s.RelationshipsSorted()),
}
}
func toSPDXID(identifiable artifact.Identifiable) common.ElementID {
id := ""
if p, ok := identifiable.(pkg.Package); ok {
id = SanitizeElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.ID()))
} else {
id = string(identifiable.ID())
}
// NOTE: the spdx libraries prepend SPDXRef-, so we don't do it here
return common.ElementID(id)
}
// packages populates all Package Information from the package Catalog (see https://spdx.github.io/spdx-spec/3-package-information/)
//
//nolint:funlen
func toPackages(catalog *pkg.Catalog) (results []*spdx.Package) {
for _, p := range catalog.Sorted() {
// name should be guaranteed to be unique, but semantically useful and stable
id := toSPDXID(p)
// If the Concluded License is not the same as the Declared License, a written explanation should be provided
// in the Comments on License field (section 7.16). With respect to NOASSERTION, a written explanation in
// the Comments on License field (section 7.16) is preferred.
license := License(p)
checksums, filesAnalyzed := toPackageChecksums(p)
results = append(results, &spdx.Package{
// NOT PART OF SPEC
// flag: does this "package" contain files that were in fact "unpackaged",
// e.g. included directly in the Document without being in a Package?
IsUnpackaged: false,
// 7.1: Package Name
// Cardinality: mandatory, one
PackageName: p.Name,
// 7.2: Package SPDX Identifier: "SPDXRef-[idstring]"
// Cardinality: mandatory, one
PackageSPDXIdentifier: id,
// 7.3: Package Version
// Cardinality: optional, one
PackageVersion: p.Version,
// 7.4: Package File Name
// Cardinality: optional, one
PackageFileName: "",
// 7.5: Package Supplier: may have single result for either Person or Organization,
// or NOASSERTION
// Cardinality: optional, one
// 7.6: Package Originator: may have single result for either Person or Organization,
// or NOASSERTION
// Cardinality: optional, one
PackageSupplier: nil,
PackageOriginator: toPackageOriginator(p),
// 7.7: Package Download Location
// Cardinality: mandatory, one
// NONE if there is no download location whatsoever.
// NOASSERTION if:
// (i) the SPDX file creator has attempted to but cannot reach a reasonable objective determination;
// (ii) the SPDX file creator has made no attempt to determine this field; or
// (iii) the SPDX file creator has intentionally provided no information (no meaning should be implied by doing so).
PackageDownloadLocation: DownloadLocation(p),
// 7.8: FilesAnalyzed
// Cardinality: optional, one; default value is "true" if omitted
// Purpose: Indicates whether the file content of this package has been available for or subjected to
// analysis when creating the SPDX document. If false, indicates packages that represent metadata or
// URI references to a project, product, artifact, distribution or a component. If false, the package
// must not contain any files.
// Intent: A package can refer to a project, product, artifact, distribution or a component that is
// external to the SPDX document.
FilesAnalyzed: filesAnalyzed,
// NOT PART OF SPEC: did FilesAnalyzed tag appear?
IsFilesAnalyzedTagPresent: true,
// 7.9: Package Verification Code
// Cardinality: optional, one if filesAnalyzed is true / omitted;
// zero (must be omitted) if filesAnalyzed is false
PackageVerificationCode: nil,
// 7.10: Package Checksum: may have keys for SHA1, SHA256 and/or MD5
// Cardinality: optional, one or many
// 7.10.1 Purpose: Provide an independently reproducible mechanism that permits unique identification of
// a specific package that correlates to the data in this SPDX file. This identifier enables a recipient
// to determine if any file in the original package has been changed. If the SPDX file is to be included
// in a package, this value should not be calculated. The SHA-1 algorithm will be used to provide the
// checksum by default.
PackageChecksums: checksums,
// 7.11: Package Home Page
// Cardinality: optional, one
PackageHomePage: Homepage(p),
// 7.12: Source Information
// Cardinality: optional, one
PackageSourceInfo: SourceInfo(p),
// 7.13: Concluded License: SPDX License Expression, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one
// Purpose: Contain the license the SPDX file creator has concluded as governing the
// package or alternative values, if the governing license cannot be determined.
PackageLicenseConcluded: license,
// 7.14: All Licenses Info from Files: SPDX License Expression, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one or many if filesAnalyzed is true / omitted;
// zero (must be omitted) if filesAnalyzed is false
PackageLicenseInfoFromFiles: nil,
// 7.15: Declared License: SPDX License Expression, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one
// Purpose: List the licenses that have been declared by the authors of the package.
// Any license information that does not originate from the package authors, e.g. license
// information from a third party repository, should not be included in this field.
PackageLicenseDeclared: license,
// 7.16: Comments on License
// Cardinality: optional, one
PackageLicenseComments: "",
// 7.17: Copyright Text: copyright notice(s) text, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one
// Purpose: IdentifyFormat the copyright holders of the package, as well as any dates present. This will be a free form text field extracted from package information files. The options to populate this field are limited to:
//
// Any text related to a copyright notice, even if not complete;
// NONE if the package contains no copyright information whatsoever; or
// NOASSERTION, if
// (i) the SPDX document creator has made no attempt to determine this field; or
// (ii) the SPDX document creator has intentionally provided no information (no meaning should be implied by doing so).
//
PackageCopyrightText: noAssertion,
// 7.18: Package Summary Description
// Cardinality: optional, one
PackageSummary: "",
// 7.19: Package Detailed Description
// Cardinality: optional, one
PackageDescription: Description(p),
// 7.20: Package Comment
// Cardinality: optional, one
PackageComment: "",
// 7.21: Package External Reference
// Cardinality: optional, one or many
PackageExternalReferences: formatSPDXExternalRefs(p),
// 7.22: Package External Reference Comment
// Cardinality: conditional (optional, one) for each External Reference
// contained within PackageExternalReference2_1 struct, if present
// 7.23: Package Attribution Text
// Cardinality: optional, one or many
PackageAttributionTexts: nil,
})
}
return results
}
func toPackageOriginator(p pkg.Package) *common.Originator {
kind, originator := Originator(p)
if kind == "" || originator == "" {
return nil
}
return &common.Originator{
Originator: originator,
OriginatorType: kind,
}
}
func toPackageChecksums(p pkg.Package) ([]common.Checksum, bool) {
filesAnalyzed := false
var checksums []common.Checksum
switch meta := p.Metadata.(type) {
// we generate digest for some Java packages
// spdx.github.io/spdx-spec/package-information/#710-package-checksum-field
case pkg.JavaMetadata:
if len(meta.ArchiveDigests) > 0 {
filesAnalyzed = true
for _, digest := range meta.ArchiveDigests {
algo := strings.ToUpper(digest.Algorithm)
checksums = append(checksums, common.Checksum{
Algorithm: common.ChecksumAlgorithm(algo),
Value: digest.Value,
})
}
}
case pkg.GolangBinMetadata:
algo, hexStr, err := util.HDigestToSHA(meta.H1Digest)
if err != nil {
log.Debugf("invalid h1digest: %s: %v", meta.H1Digest, err)
break
}
algo = strings.ToUpper(algo)
checksums = append(checksums, common.Checksum{
Algorithm: common.ChecksumAlgorithm(algo),
Value: hexStr,
})
}
return checksums, filesAnalyzed
}
func formatSPDXExternalRefs(p pkg.Package) (refs []*spdx.PackageExternalReference) {
for _, ref := range ExternalRefs(p) {
refs = append(refs, &spdx.PackageExternalReference{
Category: string(ref.ReferenceCategory),
RefType: string(ref.ReferenceType),
Locator: ref.ReferenceLocator,
ExternalRefComment: ref.Comment,
})
}
return refs
}
func toRelationships(relationships []artifact.Relationship) (result []*spdx.Relationship) {
for _, r := range relationships {
exists, relationshipType, comment := lookupRelationship(r.Type)
if !exists {
log.Debugf("unable to convert relationship to SPDX, dropping: %+v", r)
continue
}
// FIXME: we are only currently including Package -> * relationships
if _, ok := r.From.(pkg.Package); !ok {
log.Debugf("skipping non-package relationship: %+v", r)
continue
}
result = append(result, &spdx.Relationship{
RefA: common.DocElementID{
ElementRefID: toSPDXID(r.From),
},
Relationship: string(relationshipType),
RefB: common.DocElementID{
ElementRefID: toSPDXID(r.To),
},
RelationshipComment: comment,
})
}
return result
}
func lookupRelationship(ty artifact.RelationshipType) (bool, RelationshipType, string) {
switch ty {
case artifact.ContainsRelationship:
return true, ContainsRelationship, ""
case artifact.DependencyOfRelationship:
return true, DependencyOfRelationship, ""
case artifact.OwnershipByFileOverlapRelationship:
return true, OtherRelationship, fmt.Sprintf("%s: indicates that the parent package claims ownership of a child package since the parent metadata indicates overlap with a location that a cataloger found the child package by", ty)
}
return false, "", ""
}
func toFiles(s sbom.SBOM) (results []*spdx.File) {
artifacts := s.Artifacts
for _, coordinates := range s.AllCoordinates() {
var metadata *source.FileMetadata
if metadataForLocation, exists := artifacts.FileMetadata[coordinates]; exists {
metadata = &metadataForLocation
}
var digests []file.Digest
if digestsForLocation, exists := artifacts.FileDigests[coordinates]; exists {
digests = digestsForLocation
}
// TODO: add file classifications (?) and content as a snippet
var comment string
if coordinates.FileSystemID != "" {
comment = fmt.Sprintf("layerID: %s", coordinates.FileSystemID)
}
results = append(results, &spdx.File{
FileSPDXIdentifier: toSPDXID(coordinates),
FileComment: comment,
// required, no attempt made to determine license information
LicenseConcluded: noAssertion,
Checksums: toFileChecksums(digests),
FileName: coordinates.RealPath,
FileTypes: toFileTypes(metadata),
})
}
// sort by real path then virtual path to ensure the result is stable across multiple runs
sort.SliceStable(results, func(i, j int) bool {
if results[i].FileName == results[j].FileName {
return results[i].FileSPDXIdentifier < results[j].FileSPDXIdentifier
}
return results[i].FileName < results[j].FileName
})
return results
}
func toFileChecksums(digests []file.Digest) (checksums []common.Checksum) {
for _, digest := range digests {
checksums = append(checksums, common.Checksum{
Algorithm: toChecksumAlgorithm(digest.Algorithm),
Value: digest.Value,
})
}
return checksums
}
func toChecksumAlgorithm(algorithm string) common.ChecksumAlgorithm {
// this needs to be an uppercase version of our algorithm
return common.ChecksumAlgorithm(strings.ToUpper(algorithm))
}
func toFileTypes(metadata *source.FileMetadata) (ty []string) {
if metadata == nil {
return nil
}
mimeTypePrefix := strings.Split(metadata.MIMEType, "/")[0]
switch mimeTypePrefix {
case "image":
ty = append(ty, string(ImageFileType))
case "video":
ty = append(ty, string(VideoFileType))
case "application":
ty = append(ty, string(ApplicationFileType))
case "text":
ty = append(ty, string(TextFileType))
case "audio":
ty = append(ty, string(AudioFileType))
}
if internal.IsExecutable(metadata.MIMEType) {
ty = append(ty, string(BinaryFileType))
}
if internal.IsArchive(metadata.MIMEType) {
ty = append(ty, string(ArchiveFileType))
}
// TODO: add support for source, spdx, and documentation file types
if len(ty) == 0 {
ty = append(ty, string(OtherFileType))
}
return ty
}
@@ -1,20 +1,115 @@
package spdx22json
package spdxhelpers
import (
"fmt"
"testing"
"github.com/spdx/tools-golang/spdx/common"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/formats/common/spdxhelpers"
"github.com/anchore/syft/syft/formats/spdx22json/model"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
// TODO: Add ToFormatModel tests
func Test_toPackageChecksums(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expected []common.Checksum
filesAnalyzed bool
}{
{
name: "Java Package",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Language: pkg.Java,
Metadata: pkg.JavaMetadata{
ArchiveDigests: []file.Digest{
{
Algorithm: "sha1", // SPDX expects these to be uppercase
Value: "1234",
},
},
},
},
expected: []common.Checksum{
{
Algorithm: "SHA1",
Value: "1234",
},
},
filesAnalyzed: true,
},
{
name: "Java Package with no archive digests",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Language: pkg.Java,
Metadata: pkg.JavaMetadata{
ArchiveDigests: []file.Digest{},
},
},
expected: []common.Checksum{},
filesAnalyzed: false,
},
{
name: "Java Package with no metadata",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Language: pkg.Java,
},
expected: []common.Checksum{},
filesAnalyzed: false,
},
{
name: "Go Binary Package",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Language: pkg.Go,
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
},
expected: []common.Checksum{
{
Algorithm: "SHA256",
Value: "f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
},
},
filesAnalyzed: false,
},
{
name: "Package with no metadata type",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Language: pkg.Java,
Metadata: struct{}{},
},
expected: []common.Checksum{},
filesAnalyzed: false,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
commonSum, filesAnalyzed := toPackageChecksums(test.pkg)
assert.ElementsMatch(t, test.expected, commonSum)
assert.Equal(t, test.filesAnalyzed, filesAnalyzed)
})
}
}
func Test_toFileTypes(t *testing.T) {
tests := []struct {
@@ -28,7 +123,7 @@ func Test_toFileTypes(t *testing.T) {
MIMEType: "application/vnd.unknown",
},
expected: []string{
string(spdxhelpers.ApplicationFileType),
string(ApplicationFileType),
},
},
{
@@ -37,8 +132,8 @@ func Test_toFileTypes(t *testing.T) {
MIMEType: "application/zip",
},
expected: []string{
string(spdxhelpers.ApplicationFileType),
string(spdxhelpers.ArchiveFileType),
string(ApplicationFileType),
string(ArchiveFileType),
},
},
{
@@ -47,7 +142,7 @@ func Test_toFileTypes(t *testing.T) {
MIMEType: "audio/ogg",
},
expected: []string{
string(spdxhelpers.AudioFileType),
string(AudioFileType),
},
},
{
@@ -56,7 +151,7 @@ func Test_toFileTypes(t *testing.T) {
MIMEType: "video/3gpp",
},
expected: []string{
string(spdxhelpers.VideoFileType),
string(VideoFileType),
},
},
{
@@ -65,7 +160,7 @@ func Test_toFileTypes(t *testing.T) {
MIMEType: "text/html",
},
expected: []string{
string(spdxhelpers.TextFileType),
string(TextFileType),
},
},
{
@@ -74,7 +169,7 @@ func Test_toFileTypes(t *testing.T) {
MIMEType: "image/png",
},
expected: []string{
string(spdxhelpers.ImageFileType),
string(ImageFileType),
},
},
{
@@ -83,8 +178,8 @@ func Test_toFileTypes(t *testing.T) {
MIMEType: "application/x-sharedlib",
},
expected: []string{
string(spdxhelpers.ApplicationFileType),
string(spdxhelpers.BinaryFileType),
string(ApplicationFileType),
string(BinaryFileType),
},
},
}
@@ -100,18 +195,18 @@ func Test_lookupRelationship(t *testing.T) {
tests := []struct {
input artifact.RelationshipType
exists bool
ty spdxhelpers.RelationshipType
ty RelationshipType
comment string
}{
{
input: artifact.ContainsRelationship,
exists: true,
ty: spdxhelpers.ContainsRelationship,
ty: ContainsRelationship,
},
{
input: artifact.OwnershipByFileOverlapRelationship,
exists: true,
ty: spdxhelpers.OtherRelationship,
ty: OtherRelationship,
comment: "ownership-by-file-overlap: indicates that the parent package claims ownership of a child package since the parent metadata indicates overlap with a location that a cataloger found the child package by",
},
{
@@ -133,7 +228,7 @@ func Test_toFileChecksums(t *testing.T) {
tests := []struct {
name string
digests []file.Digest
expected []model.Checksum
expected []common.Checksum
}{
{
name: "empty",
@@ -150,14 +245,14 @@ func Test_toFileChecksums(t *testing.T) {
Value: "meh",
},
},
expected: []model.Checksum{
expected: []common.Checksum{
{
Algorithm: "SHA256",
ChecksumValue: "deadbeefcafe",
Algorithm: "SHA256",
Value: "deadbeefcafe",
},
{
Algorithm: "MD5",
ChecksumValue: "meh",
Algorithm: "MD5",
Value: "meh",
},
},
},
@@ -170,7 +265,6 @@ func Test_toFileChecksums(t *testing.T) {
}
func Test_fileIDsForPackage(t *testing.T) {
p := pkg.Package{
Name: "bogus",
}
@@ -180,15 +274,19 @@ func Test_fileIDsForPackage(t *testing.T) {
FileSystemID: "nowhere",
}
docElementId := func(identifiable artifact.Identifiable) common.DocElementID {
return common.DocElementID{
ElementRefID: toSPDXID(identifiable),
}
}
tests := []struct {
name string
id string
relationships []artifact.Relationship
expected []string
expected []*spdx.Relationship
}{
{
name: "find file IDs for packages with package-file relationships",
id: model.ElementID(p.ID()).String(),
name: "package-to-file contains relationships",
relationships: []artifact.Relationship{
{
From: p,
@@ -196,13 +294,16 @@ func Test_fileIDsForPackage(t *testing.T) {
Type: artifact.ContainsRelationship,
},
},
expected: []string{
model.ElementID(c.ID()).String(),
expected: []*spdx.Relationship{
{
Relationship: "CONTAINS",
RefA: docElementId(p),
RefB: docElementId(c),
},
},
},
{
name: "ignore package-to-package",
id: model.ElementID(p.ID()).String(),
name: "package-to-package",
relationships: []artifact.Relationship{
{
From: p,
@@ -210,11 +311,16 @@ func Test_fileIDsForPackage(t *testing.T) {
Type: artifact.ContainsRelationship,
},
},
expected: []string{},
expected: []*spdx.Relationship{
{
Relationship: "CONTAINS",
RefA: docElementId(p),
RefB: docElementId(p),
},
},
},
{
name: "ignore file-to-file",
id: model.ElementID(p.ID()).String(),
relationships: []artifact.Relationship{
{
From: c,
@@ -222,11 +328,10 @@ func Test_fileIDsForPackage(t *testing.T) {
Type: artifact.ContainsRelationship,
},
},
expected: []string{},
expected: nil,
},
{
name: "ignore file-to-package",
id: model.ElementID(p.ID()).String(),
relationships: []artifact.Relationship{
{
From: c,
@@ -234,11 +339,10 @@ func Test_fileIDsForPackage(t *testing.T) {
Type: artifact.ContainsRelationship,
},
},
expected: []string{},
expected: nil,
},
{
name: "filter by relationship type",
id: model.ElementID(p.ID()).String(),
name: "include package-to-file overlap relationships",
relationships: []artifact.Relationship{
{
From: p,
@@ -246,12 +350,20 @@ func Test_fileIDsForPackage(t *testing.T) {
Type: artifact.OwnershipByFileOverlapRelationship,
},
},
expected: []string{},
expected: []*spdx.Relationship{
{
Relationship: "OTHER",
RefA: docElementId(p),
RefB: docElementId(c),
RelationshipComment: "ownership-by-file-overlap: indicates that the parent package claims ownership of a child package since the parent metadata indicates overlap with a location that a cataloger found the child package by",
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.ElementsMatch(t, test.expected, fileIDsForPackage(test.id, test.relationships))
relationships := toRelationships(test.relationships)
assert.Equal(t, test.expected, relationships)
})
}
}
@@ -303,15 +415,17 @@ func Test_H1Digest(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
catalog := pkg.NewCatalog(test.pkg)
pkgs := toPackages(catalog, nil)
pkgs := toPackages(catalog)
require.Len(t, pkgs, 1)
p := pkgs[0]
if test.expectedDigest == "" {
require.Len(t, p.Checksums, 0)
} else {
require.Len(t, p.Checksums, 1)
c := p.Checksums[0]
require.Equal(t, test.expectedDigest, fmt.Sprintf("%s:%s", c.Algorithm, c.ChecksumValue))
for _, p := range pkgs {
if test.expectedDigest == "" {
require.Len(t, p.PackageChecksums, 0)
} else {
require.Len(t, p.PackageChecksums, 1)
for _, c := range p.PackageChecksums {
require.Equal(t, test.expectedDigest, fmt.Sprintf("%s:%s", c.Algorithm, c.Value))
}
}
}
})
}
@@ -6,7 +6,7 @@ import (
"strconv"
"strings"
"github.com/spdx/tools-golang/spdx"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal/log"
@@ -19,7 +19,7 @@ import (
"github.com/anchore/syft/syft/source"
)
func ToSyftModel(doc *spdx.Document2_2) (*sbom.SBOM, error) {
func ToSyftModel(doc *spdx.Document) (*sbom.SBOM, error) {
if doc == nil {
return nil, errors.New("cannot convert SPDX document to Syft model because document is nil")
}
@@ -27,9 +27,7 @@ func ToSyftModel(doc *spdx.Document2_2) (*sbom.SBOM, error) {
spdxIDMap := make(map[string]interface{})
src := source.Metadata{Scheme: source.UnknownScheme}
if doc.CreationInfo != nil {
src.Scheme = extractSchemeFromNamespace(doc.CreationInfo.DocumentNamespace)
}
src.Scheme = extractSchemeFromNamespace(doc.DocumentNamespace)
s := &sbom.SBOM{
Source: src,
@@ -63,18 +61,18 @@ func extractSchemeFromNamespace(ns string) source.Scheme {
parts := strings.Split(u.Path, "/")
for _, p := range parts {
switch p {
case "file":
case inputFile:
return source.FileScheme
case "image":
case inputImage:
return source.ImageScheme
case "dir":
case inputDirectory:
return source.DirectoryScheme
}
}
return source.UnknownScheme
}
func findLinuxReleaseByPURL(doc *spdx.Document2_2) *linux.Release {
func findLinuxReleaseByPURL(doc *spdx.Document) *linux.Release {
for _, p := range doc.Packages {
purlValue := findPURLValue(p)
if purlValue == "" {
@@ -107,7 +105,7 @@ func findLinuxReleaseByPURL(doc *spdx.Document2_2) *linux.Release {
return nil
}
func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document2_2) {
func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document) {
for _, p := range doc.Packages {
syftPkg := toSyftPackage(p)
spdxIDMap[string(p.PackageSPDXIdentifier)] = syftPkg
@@ -115,8 +113,8 @@ func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *sp
}
}
func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document2_2) {
for _, f := range doc.UnpackagedFiles {
func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document) {
for _, f := range doc.Files {
l := toSyftLocation(f)
spdxIDMap[string(f.FileSPDXIdentifier)] = l
@@ -125,8 +123,8 @@ func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.
}
}
func toFileDigests(f *spdx.File2_2) (digests []file.Digest) {
for _, digest := range f.FileChecksums {
func toFileDigests(f *spdx.File) (digests []file.Digest) {
for _, digest := range f.Checksums {
digests = append(digests, file.Digest{
Algorithm: string(digest.Algorithm),
Value: digest.Value,
@@ -135,9 +133,9 @@ func toFileDigests(f *spdx.File2_2) (digests []file.Digest) {
return digests
}
func toFileMetadata(f *spdx.File2_2) (meta source.FileMetadata) {
func toFileMetadata(f *spdx.File) (meta source.FileMetadata) {
// FIXME Syft is currently lossy due to the SPDX 2.2.1 spec not supporting arbitrary mimetypes
for _, typ := range f.FileType {
for _, typ := range f.FileTypes {
switch FileType(typ) {
case ImageFileType:
meta.MIMEType = "image/"
@@ -157,11 +155,11 @@ func toFileMetadata(f *spdx.File2_2) (meta source.FileMetadata) {
return meta
}
func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document2_2) []artifact.Relationship {
func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document) []artifact.Relationship {
var out []artifact.Relationship
for _, r := range doc.Relationships {
// FIXME what to do with r.RefA.DocumentRefID and r.RefA.SpecialID
if r.RefA.DocumentRefID != "" && requireAndTrimPrefix(r.RefA.DocumentRefID, "DocumentRef-") != string(doc.CreationInfo.SPDXIdentifier) {
if r.RefA.DocumentRefID != "" && requireAndTrimPrefix(r.RefA.DocumentRefID, "DocumentRef-") != string(doc.SPDXIdentifier) {
log.Debugf("ignoring relationship to external document: %+v", r)
continue
}
@@ -186,16 +184,10 @@ func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document2_2
case ContainsRelationship:
typ = artifact.ContainsRelationship
to = toPackage
case BuildDependencyOfRelationship:
typ = artifact.BuildDependencyOfRelationship
to = toPackage
case RuntimeDependencyOfRelationship:
typ = artifact.RuntimeDependencyOfRelationship
to = toPackage
case OtherRelationship:
// Encoding uses a specifically formatted comment...
if strings.Index(r.RelationshipComment, string(artifact.OwnershipByFileOverlapRelationship)) == 0 {
typ = artifact.RuntimeDependencyOfRelationship
typ = artifact.DependencyOfRelationship
to = toPackage
}
}
@@ -211,7 +203,7 @@ func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document2_2
return out
}
func toSyftCoordinates(f *spdx.File2_2) source.Coordinates {
func toSyftCoordinates(f *spdx.File) source.Coordinates {
const layerIDPrefix = "layerID: "
var fileSystemID string
if strings.Index(f.FileComment, layerIDPrefix) == 0 {
@@ -226,7 +218,7 @@ func toSyftCoordinates(f *spdx.File2_2) source.Coordinates {
}
}
func toSyftLocation(f *spdx.File2_2) *source.Location {
func toSyftLocation(f *spdx.File) *source.Location {
return &source.Location{
Coordinates: toSyftCoordinates(f),
VirtualPath: f.FileName,
@@ -261,7 +253,7 @@ func findQualifierValue(purl packageurl.PackageURL, qualifier string) string {
return ""
}
func extractPkgInfo(p *spdx.Package2_2) pkgInfo {
func extractPkgInfo(p *spdx.Package) pkgInfo {
pu := findPURLValue(p)
purl, err := packageurl.FromString(pu)
if err != nil {
@@ -274,7 +266,7 @@ func extractPkgInfo(p *spdx.Package2_2) pkgInfo {
}
}
func toSyftPackage(p *spdx.Package2_2) *pkg.Package {
func toSyftPackage(p *spdx.Package) *pkg.Package {
info := extractPkgInfo(p)
metadataType, metadata := extractMetadata(p, info)
sP := pkg.Package{
@@ -295,7 +287,7 @@ func toSyftPackage(p *spdx.Package2_2) *pkg.Package {
}
//nolint:funlen
func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interface{}) {
func extractMetadata(p *spdx.Package, info pkgInfo) (pkg.MetadataType, interface{}) {
arch := info.qualifierValue(pkg.PURLQualifierArch)
upstreamValue := info.qualifierValue(pkg.PURLQualifierUpstream)
upstream := strings.SplitN(upstreamValue, "@", 2)
@@ -304,12 +296,20 @@ func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interf
if len(upstream) > 1 {
upstreamVersion = upstream[1]
}
supplier := ""
if p.PackageSupplier != nil {
supplier = p.PackageSupplier.Supplier
}
originator := ""
if p.PackageOriginator != nil {
originator = p.PackageOriginator.Originator
}
switch info.typ {
case pkg.ApkPkg:
return pkg.ApkMetadataType, pkg.ApkMetadata{
Package: p.PackageName,
OriginPackage: upstreamName,
Maintainer: p.PackageSupplierPerson,
Maintainer: supplier,
Version: p.PackageVersion,
License: p.PackageLicenseDeclared,
Architecture: arch,
@@ -335,7 +335,7 @@ func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interf
Arch: arch,
SourceRpm: upstreamValue,
License: license,
Vendor: p.PackageOriginatorOrganization,
Vendor: originator,
}
case pkg.DebPkg:
return pkg.DpkgMetadataType, pkg.DpkgMetadata{
@@ -344,12 +344,12 @@ func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interf
Version: p.PackageVersion,
SourceVersion: upstreamVersion,
Architecture: arch,
Maintainer: p.PackageOriginatorPerson,
Maintainer: originator,
}
case pkg.JavaPkg:
var digests []file.Digest
for algorithm, value := range p.PackageChecksums {
digests = append(digests, file.Digest{Algorithm: string(algorithm), Value: value.Value})
for _, value := range p.PackageChecksums {
digests = append(digests, file.Digest{Algorithm: string(value.Algorithm), Value: value.Value})
}
return pkg.JavaMetadataType, pkg.JavaMetadata{
ArchiveDigests: digests,
@@ -372,7 +372,7 @@ func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interf
return pkg.UnknownMetadataType, nil
}
func findPURLValue(p *spdx.Package2_2) string {
func findPURLValue(p *spdx.Package) string {
for _, r := range p.PackageExternalReferences {
if r.RefType == string(PurlExternalRefType) {
return r.Locator
@@ -381,7 +381,7 @@ func findPURLValue(p *spdx.Package2_2) string {
return ""
}
func extractCPEs(p *spdx.Package2_2) (cpes []pkg.CPE) {
func extractCPEs(p *spdx.Package) (cpes []pkg.CPE) {
for _, r := range p.PackageExternalReferences {
if r.RefType == string(Cpe23ExternalRefType) {
cpe, err := pkg.NewCPE(r.Locator)
@@ -3,7 +3,8 @@ package spdxhelpers
import (
"testing"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/spdx/common"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -12,32 +13,27 @@ import (
)
func TestToSyftModel(t *testing.T) {
sbom, err := ToSyftModel(&spdx.Document2_2{
CreationInfo: &spdx.CreationInfo2_2{
SPDXVersion: "1",
DataLicense: "GPL",
SPDXIdentifier: "id-doc-1",
DocumentName: "docName",
DocumentNamespace: "docNamespace",
ExternalDocumentReferences: nil,
LicenseListVersion: "",
CreatorPersons: nil,
CreatorOrganizations: nil,
CreatorTools: nil,
Created: "",
CreatorComment: "",
DocumentComment: "",
sbom, err := ToSyftModel(&spdx.Document{
SPDXVersion: "1",
DataLicense: "GPL",
SPDXIdentifier: "id-doc-1",
DocumentName: "docName",
DocumentNamespace: "docNamespace",
ExternalDocumentReferences: nil,
DocumentComment: "",
CreationInfo: &spdx.CreationInfo{
LicenseListVersion: "",
Created: "",
CreatorComment: "",
},
Packages: map[spdx.ElementID]*spdx.Package2_2{
"id-pkg-1": {
PackageName: "pkg-1",
PackageSPDXIdentifier: "id-pkg-1",
PackageVersion: "5.4.3",
PackageSupplierPerson: "",
PackageSupplierOrganization: "",
PackageLicenseDeclared: "",
PackageDescription: "",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
Packages: []*spdx.Package{
{
PackageName: "pkg-1",
PackageSPDXIdentifier: "id-pkg-1",
PackageVersion: "5.4.3",
PackageLicenseDeclared: "",
PackageDescription: "",
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "SECURITY",
Locator: "cpe:2.3:a:pkg-1:pkg-1:5.4.3:*:*:*:*:*:*:*",
@@ -49,22 +45,20 @@ func TestToSyftModel(t *testing.T) {
RefType: "cpe23Type",
},
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:alpine/pkg-1@5.4.3?arch=x86_64&upstream=p1-origin&distro=alpine-3.10.9",
RefType: "purl",
},
},
Files: nil,
},
"id-pkg-2": {
PackageName: "pkg-2",
PackageSPDXIdentifier: "id-pkg-2",
PackageVersion: "7.3.1",
PackageSupplierPerson: "",
PackageSupplierOrganization: "",
PackageLicenseDeclared: "",
PackageDescription: "",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
{
PackageName: "pkg-2",
PackageSPDXIdentifier: "id-pkg-2",
PackageVersion: "7.3.1",
PackageLicenseDeclared: "",
PackageDescription: "",
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "SECURITY",
Locator: "cpe:2.3:a:pkg-2:pkg-2:7.3.1:*:*:*:*:*:*:*",
@@ -81,7 +75,7 @@ func TestToSyftModel(t *testing.T) {
RefType: "cpe23Type",
},
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:deb/pkg-2@7.3.1?arch=x86_64&upstream=p2-origin@9.1.3&distro=debian-3.10.9",
RefType: "purl",
},
@@ -89,8 +83,7 @@ func TestToSyftModel(t *testing.T) {
Files: nil,
},
},
UnpackagedFiles: map[spdx.ElementID]*spdx.File2_2{},
Relationships: []*spdx.Relationship2_2{},
Relationships: []*spdx.Relationship{},
})
assert.NoError(t, err)
@@ -120,17 +113,17 @@ func TestToSyftModel(t *testing.T) {
func Test_extractMetadata(t *testing.T) {
oneTwoThreeFour := 1234
tests := []struct {
pkg spdx.Package2_2
pkg spdx.Package
metaType pkg.MetadataType
meta interface{}
}{
{
pkg: spdx.Package2_2{
pkg: spdx.Package{
PackageName: "SomeDebPkg",
PackageVersion: "43.1.235",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:deb/pkg-2@7.3.1?arch=x86_64&upstream=somedebpkg-origin@9.1.3&distro=debian-3.10.9",
RefType: "purl",
},
@@ -146,12 +139,12 @@ func Test_extractMetadata(t *testing.T) {
},
},
{
pkg: spdx.Package2_2{
pkg: spdx.Package{
PackageName: "SomeApkPkg",
PackageVersion: "3.2.9",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:alpine/pkg-2@7.3.1?arch=x86_64&upstream=apk-origin@9.1.3&distro=alpine-3.10.9",
RefType: "purl",
},
@@ -166,12 +159,12 @@ func Test_extractMetadata(t *testing.T) {
},
},
{
pkg: spdx.Package2_2{
pkg: spdx.Package{
PackageName: "SomeRpmPkg",
PackageVersion: "13.2.79",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:rpm/pkg-2@7.3.1?arch=x86_64&epoch=1234&upstream=some-rpm-origin-1.16.3&distro=alpine-3.10.9",
RefType: "purl",
},
@@ -238,24 +231,24 @@ func TestExtractSourceFromNamespaces(t *testing.T) {
func TestH1Digest(t *testing.T) {
tests := []struct {
name string
pkg spdx.Package2_2
pkg spdx.Package
expectedDigest string
}{
{
name: "valid h1digest",
pkg: spdx.Package2_2{
pkg: spdx.Package{
PackageName: "github.com/googleapis/gnostic",
PackageVersion: "v0.5.5",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:golang/github.com/googleapis/gnostic@v0.5.5",
RefType: "purl",
},
},
PackageChecksums: map[spdx.ChecksumAlgorithm]spdx.Checksum{
spdx.SHA256: {
Algorithm: spdx.SHA256,
PackageChecksums: []common.Checksum{
{
Algorithm: common.SHA256,
Value: "f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
},
},
@@ -264,19 +257,19 @@ func TestH1Digest(t *testing.T) {
},
{
name: "invalid h1digest algorithm",
pkg: spdx.Package2_2{
pkg: spdx.Package{
PackageName: "github.com/googleapis/gnostic",
PackageVersion: "v0.5.5",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:golang/github.com/googleapis/gnostic@v0.5.5",
RefType: "purl",
},
},
PackageChecksums: map[spdx.ChecksumAlgorithm]spdx.Checksum{
spdx.SHA256: {
Algorithm: spdx.SHA1,
PackageChecksums: []common.Checksum{
{
Algorithm: common.SHA1,
Value: "f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
},
},
@@ -285,19 +278,19 @@ func TestH1Digest(t *testing.T) {
},
{
name: "invalid h1digest digest",
pkg: spdx.Package2_2{
pkg: spdx.Package{
PackageName: "github.com/googleapis/gnostic",
PackageVersion: "v0.5.5",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:golang/github.com/googleapis/gnostic@v0.5.5",
RefType: "purl",
},
},
PackageChecksums: map[spdx.ChecksumAlgorithm]spdx.Checksum{
spdx.SHA256: {
Algorithm: spdx.SHA256,
PackageChecksums: []common.Checksum{
{
Algorithm: common.SHA256,
Value: "",
},
},
+25
View File
@@ -2,8 +2,10 @@ package testutils
import (
"bytes"
"math/rand"
"strings"
"testing"
"time"
"github.com/sergi/go-diff/diffmatchpatch"
"github.com/stretchr/testify/assert"
@@ -12,6 +14,7 @@ import (
"github.com/anchore/stereoscope/pkg/filetree"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/stereoscope/pkg/imagetest"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
@@ -276,3 +279,25 @@ func newDirectoryCatalog() *pkg.Catalog {
return catalog
}
//nolint:gosec
func AddSampleFileRelationships(s *sbom.SBOM) {
catalog := s.Artifacts.PackageCatalog.Sorted()
s.Artifacts.FileMetadata = map[source.Coordinates]source.FileMetadata{}
files := []string{"/f1", "/f2", "/d1/f3", "/d2/f4", "/z1/f5", "/a1/f6"}
rnd := rand.New(rand.NewSource(time.Now().UnixNano()))
rnd.Shuffle(len(files), func(i, j int) { files[i], files[j] = files[j], files[i] })
for _, f := range files {
meta := source.FileMetadata{}
coords := source.Coordinates{RealPath: f}
s.Artifacts.FileMetadata[coords] = meta
s.Relationships = append(s.Relationships, artifact.Relationship{
From: catalog[0],
To: coords,
Type: artifact.ContainsRelationship,
})
}
}
@@ -1,10 +1,10 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"serialNumber": "urn:uuid:f426926b-4867-4b52-9142-23997f685f2c",
"serialNumber": "urn:uuid:cfd602eb-022b-4a61-84d4-50d34612bd84",
"version": 1,
"metadata": {
"timestamp": "2022-10-24T09:54:37-04:00",
"timestamp": "2022-11-07T09:11:21-05:00",
"tools": [
{
"vendor": "anchore",
@@ -20,7 +20,7 @@
},
"components": [
{
"bom-ref": "e624319940d8d36a",
"bom-ref": "1b1d0be59ac59d2c",
"type": "library",
"name": "package-1",
"version": "1.0.1",
@@ -57,7 +57,7 @@
]
},
{
"bom-ref": "pkg:deb/debian/package-2@2.0.1?package-id=b8645f4ac2a0891e",
"bom-ref": "pkg:deb/debian/package-2@2.0.1?package-id=db4abfe497c180d3",
"type": "library",
"name": "package-2",
"version": "2.0.1",
@@ -1,10 +1,10 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"serialNumber": "urn:uuid:41bbbcc7-694d-4b07-a678-0afb67dabdf9",
"serialNumber": "urn:uuid:ced49687-6384-48fa-a930-ef83a258bf77",
"version": 1,
"metadata": {
"timestamp": "2022-10-24T09:54:37-04:00",
"timestamp": "2022-11-07T09:11:21-05:00",
"tools": [
{
"vendor": "anchore",
@@ -21,7 +21,7 @@
},
"components": [
{
"bom-ref": "5ffee24fb164cffc",
"bom-ref": "66ba429119b8bec6",
"type": "library",
"name": "package-1",
"version": "1.0.1",
@@ -62,7 +62,7 @@
]
},
{
"bom-ref": "pkg:deb/debian/package-2@2.0.1?package-id=8b16570b2b4155c3",
"bom-ref": "pkg:deb/debian/package-2@2.0.1?package-id=958443e2d9304af4",
"type": "library",
"name": "package-2",
"version": "2.0.1",
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:19df9583-d8b7-4683-81a6-e57cc8841321" version="1">
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:2939b822-b9cb-489d-8a8b-4431b755031d" version="1">
<metadata>
<timestamp>2022-10-24T09:54:54-04:00</timestamp>
<timestamp>2022-11-07T09:11:06-05:00</timestamp>
<tools>
<tool>
<vendor>anchore</vendor>
@@ -14,7 +14,7 @@
</component>
</metadata>
<components>
<component bom-ref="e624319940d8d36a" type="library">
<component bom-ref="1b1d0be59ac59d2c" type="library">
<name>package-1</name>
<version>1.0.1</version>
<licenses>
@@ -32,7 +32,7 @@
<property name="syft:location:0:path">/some/path/pkg1</property>
</properties>
</component>
<component bom-ref="pkg:deb/debian/package-2@2.0.1?package-id=b8645f4ac2a0891e" type="library">
<component bom-ref="pkg:deb/debian/package-2@2.0.1?package-id=db4abfe497c180d3" type="library">
<name>package-2</name>
<version>2.0.1</version>
<cpe>cpe:2.3:*:some:package:2:*:*:*:*:*:*:*</cpe>
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:5342511c-3580-4cae-b373-20bbf14ba7a3" version="1">
<bom xmlns="http://cyclonedx.org/schema/bom/1.4" serialNumber="urn:uuid:2896b5ce-2016-49e8-a422-239d662846c7" version="1">
<metadata>
<timestamp>2022-10-24T09:54:54-04:00</timestamp>
<timestamp>2022-11-07T09:11:06-05:00</timestamp>
<tools>
<tool>
<vendor>anchore</vendor>
@@ -15,7 +15,7 @@
</component>
</metadata>
<components>
<component bom-ref="5ffee24fb164cffc" type="library">
<component bom-ref="66ba429119b8bec6" type="library">
<name>package-1</name>
<version>1.0.1</version>
<licenses>
@@ -34,7 +34,7 @@
<property name="syft:location:0:path">/somefile-1.txt</property>
</properties>
</component>
<component bom-ref="pkg:deb/debian/package-2@2.0.1?package-id=8b16570b2b4155c3" type="library">
<component bom-ref="pkg:deb/debian/package-2@2.0.1?package-id=958443e2d9304af4" type="library">
<name>package-2</name>
<version>2.0.1</version>
<cpe>cpe:2.3:*:some:package:2:*:*:*:*:*:*:*</cpe>
+129
View File
@@ -0,0 +1,129 @@
package formats
import (
"bytes"
"errors"
"fmt"
"io"
"strings"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/cyclonedxxml"
"github.com/anchore/syft/syft/formats/github"
"github.com/anchore/syft/syft/formats/spdxjson"
"github.com/anchore/syft/syft/formats/spdxtagvalue"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/formats/template"
"github.com/anchore/syft/syft/formats/text"
"github.com/anchore/syft/syft/sbom"
)
func Formats() []sbom.Format {
return []sbom.Format{
syftjson.Format(),
cyclonedxxml.Format(),
cyclonedxjson.Format(),
github.Format(),
spdxtagvalue.Format(),
spdxjson.Format(),
table.Format(),
text.Format(),
template.Format(),
}
}
func Identify(by []byte) sbom.Format {
for _, f := range Formats() {
if err := f.Validate(bytes.NewReader(by)); err != nil {
if !errors.Is(err, sbom.ErrValidationNotSupported) {
log.Debugf("format %s returned err: %+v", f.ID(), err)
}
continue
}
return f
}
return nil
}
func ByName(name string) sbom.Format {
cleanName := cleanFormatName(name)
for _, f := range Formats() {
if cleanFormatName(string(f.ID())) == cleanName {
return f
}
}
// handle any aliases for any supported format
switch cleanName {
case "json", "syftjson":
return ByID(syftjson.ID)
case "cyclonedx", "cyclone", "cyclonedxxml":
return ByID(cyclonedxxml.ID)
case "cyclonedxjson":
return ByID(cyclonedxjson.ID)
case "github", "githubjson":
return ByID(github.ID)
case "spdx", "spdxtv", "spdxtagvalue":
return ByID(spdxtagvalue.ID)
case "spdxjson":
return ByID(spdxjson.ID)
case "table":
return ByID(table.ID)
case "text":
return ByID(text.ID)
case "template":
ByID(template.ID)
}
return nil
}
func IDs() (ids []sbom.FormatID) {
for _, f := range Formats() {
ids = append(ids, f.ID())
}
return ids
}
func ByID(id sbom.FormatID) sbom.Format {
for _, f := range Formats() {
if f.ID() == id {
return f
}
}
return nil
}
func cleanFormatName(name string) string {
r := strings.NewReplacer("-", "", "_", "")
return strings.ToLower(r.Replace(name))
}
// Encode takes all SBOM elements and a format option and encodes an SBOM document.
func Encode(s sbom.SBOM, f sbom.Format) ([]byte, error) {
buff := bytes.Buffer{}
if err := f.Encode(&buff, s); err != nil {
return nil, fmt.Errorf("unable to encode sbom: %w", err)
}
return buff.Bytes(), nil
}
// Decode takes a reader for an SBOM and generates all internal SBOM elements.
func Decode(reader io.Reader) (*sbom.SBOM, sbom.Format, error) {
by, err := io.ReadAll(reader)
if err != nil {
return nil, nil, fmt.Errorf("unable to read sbom: %w", err)
}
f := Identify(by)
if f == nil {
return nil, nil, fmt.Errorf("unable to identify format")
}
s, err := f.Decode(bytes.NewReader(by))
return s, f, err
}
@@ -1,4 +1,4 @@
package syft
package formats
import (
"bytes"
@@ -12,8 +12,8 @@ import (
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/cyclonedxxml"
"github.com/anchore/syft/syft/formats/github"
"github.com/anchore/syft/syft/formats/spdx22json"
"github.com/anchore/syft/syft/formats/spdx22tagvalue"
"github.com/anchore/syft/syft/formats/spdxjson"
"github.com/anchore/syft/syft/formats/spdxtagvalue"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/formats/template"
@@ -37,7 +37,7 @@ func TestIdentify(t *testing.T) {
assert.NoError(t, err)
by, err := io.ReadAll(f)
assert.NoError(t, err)
frmt := IdentifyFormat(by)
frmt := Identify(by)
assert.NotNil(t, frmt)
assert.Equal(t, test.expected, frmt.ID())
})
@@ -45,7 +45,7 @@ func TestIdentify(t *testing.T) {
}
func TestFormats_EmptyInput(t *testing.T) {
for _, format := range formats {
for _, format := range Formats() {
t.Run(format.ID().String(), func(t *testing.T) {
t.Run("format.Decode", func(t *testing.T) {
input := bytes.NewReader(nil)
@@ -69,7 +69,7 @@ func TestFormats_EmptyInput(t *testing.T) {
}
}
func TestFormatByName(t *testing.T) {
func TestByName(t *testing.T) {
tests := []struct {
name string
@@ -78,41 +78,41 @@ func TestFormatByName(t *testing.T) {
// SPDX Tag-Value
{
name: "spdx",
want: spdx22tagvalue.ID,
want: spdxtagvalue.ID,
},
{
name: "spdx-tag-value",
want: spdx22tagvalue.ID,
want: spdxtagvalue.ID,
},
{
name: "spdx-tv",
want: spdx22tagvalue.ID,
want: spdxtagvalue.ID,
},
{
name: "spdxtv", // clean variant
want: spdx22tagvalue.ID,
want: spdxtagvalue.ID,
},
{
name: "spdx-2-tag-value", // clean variant
want: spdx22tagvalue.ID,
want: spdxtagvalue.ID,
},
{
name: "spdx-2-tagvalue", // clean variant
want: spdx22tagvalue.ID,
want: spdxtagvalue.ID,
},
{
name: "spdx2-tagvalue", // clean variant
want: spdx22tagvalue.ID,
want: spdxtagvalue.ID,
},
// SPDX JSON
{
name: "spdx-json",
want: spdx22json.ID,
want: spdxjson.ID,
},
{
name: "spdx-2-json",
want: spdx22json.ID,
want: spdxjson.ID,
},
// Cyclonedx JSON
@@ -190,7 +190,7 @@ func TestFormatByName(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
f := FormatByName(tt.name)
f := ByName(tt.name)
if tt.want == "" {
require.Nil(t, f)
return
-28
View File
@@ -1,28 +0,0 @@
package spdx22json
import (
"fmt"
"io"
"github.com/spdx/tools-golang/jsonloader"
"github.com/anchore/syft/syft/formats/common/spdxhelpers"
"github.com/anchore/syft/syft/sbom"
)
func decoder(reader io.Reader) (s *sbom.SBOM, err error) {
defer func() {
// The spdx tools JSON parser panics in quite a lot of situations, just handle this as a parse failure
if v := recover(); v != nil {
s = nil
err = fmt.Errorf("an error occurred during SPDX JSON document parsing: %+v", v)
}
}()
doc, err := jsonloader.Load2_2(reader)
if err != nil {
return nil, fmt.Errorf("unable to decode spdx-json: %w", err)
}
return spdxhelpers.ToSyftModel(doc)
}
@@ -1,21 +0,0 @@
package model
import "time"
type AnnotationType string
const (
ReviewerAnnotationType AnnotationType = "REVIEWER"
OtherAnnotationType AnnotationType = "OTHER"
)
type Annotation struct {
// Identify when the comment was made. This is to be specified according to the combined date and time in the
// UTC format, as specified in the ISO 8601 standard.
AnnotationDate time.Time `json:"annotationDate"`
// Type of the annotation
AnnotationType AnnotationType `json:"annotationType"`
// This field identifies the person, organization or tool that has commented on a file, package, or the entire document.
Annotator string `json:"annotator"`
Comment string `json:"comment"`
}
@@ -1,7 +0,0 @@
package model
type Checksum struct {
// Identifies the algorithm used to produce the subject Checksum. One of: "SHA256", "SHA1", "SHA384", "MD2", "MD4", "SHA512", "MD6", "MD5", "SHA224"
Algorithm string `json:"algorithm"`
ChecksumValue string `json:"checksumValue"`
}
@@ -1,19 +0,0 @@
package model
import "time"
type CreationInfo struct {
Comment string `json:"comment,omitempty"`
// Identify when the SPDX file was originally created. The date is to be specified according to combined date and
// time in UTC format as specified in ISO 8601 standard. This field is distinct from the fields in section 8,
// which involves the addition of information during a subsequent review.
Created time.Time `json:"created"`
// Identify who (or what, in the case of a tool) created the SPDX file. If the SPDX file was created by an
// individual, indicate the person's name. If the SPDX file was created on behalf of a company or organization,
// indicate the entity name. If the SPDX file was created using a software tool, indicate the name and version
// for that tool. If multiple participants or tools were involved, use multiple instances of this field. Person
// name or organization name may be designated as “anonymous” if appropriate.
Creators []string `json:"creators"`
// An optional field for creators of the SPDX file to provide the version of the SPDX License List used when the SPDX file was created.
LicenseListVersion string `json:"licenseListVersion"`
}
-45
View File
@@ -1,45 +0,0 @@
package model
// derived from:
// - https://spdx.github.io/spdx-spec/appendix-III-RDF-data-model-implementation-and-identifier-syntax/
// - https://github.com/spdx/spdx-spec/blob/v2.2/schemas/spdx-schema.json
// - https://github.com/spdx/spdx-spec/tree/v2.2/ontology
type Document struct {
Element
SPDXVersion string `json:"spdxVersion"`
// One instance is required for each SPDX file produced. It provides the necessary information for forward
// and backward compatibility for processing tools.
CreationInfo CreationInfo `json:"creationInfo"`
// 2.2: Data License; should be "CC0-1.0"
// Cardinality: mandatory, one
// License expression for dataLicense. Compliance with the SPDX specification includes populating the SPDX
// fields therein with data related to such fields (\"SPDX-Metadata\"). The SPDX specification contains numerous
// fields where an SPDX document creator may provide relevant explanatory text in SPDX-Metadata. Without
// opining on the lawfulness of \"database rights\" (in jurisdictions where applicable), such explanatory text
// is copyrightable subject matter in most Berne Convention countries. By using the SPDX specification, or any
// portion hereof, you hereby agree that any copyright rights (as determined by your jurisdiction) in any
// SPDX-Metadata, including without limitation explanatory text, shall be subject to the terms of the Creative
// Commons CC0 1.0 Universal license. For SPDX-Metadata not containing any copyright rights, you hereby agree
// and acknowledge that the SPDX-Metadata is provided to you \"as-is\" and without any representations or
// warranties of any kind concerning the SPDX-Metadata, express, implied, statutory or otherwise, including
// without limitation warranties of title, merchantability, fitness for a particular purpose, non-infringement,
// or the absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not
// discoverable, all to the greatest extent permissible under applicable law.
DataLicense string `json:"dataLicense"`
// Information about an external SPDX document reference including the checksum. This allows for verification of the external references.
ExternalDocumentRefs []ExternalDocumentRef `json:"externalDocumentRefs,omitempty"`
// Indicates that a particular ExtractedLicensingInfo was defined in the subject SpdxDocument.
HasExtractedLicensingInfos []HasExtractedLicensingInfo `json:"hasExtractedLicensingInfos,omitempty"`
// note: found in example documents from SPDX, but not in the JSON schema. See https://spdx.github.io/spdx-spec/2-document-creation-information/#25-spdx-document-namespace
DocumentNamespace string `json:"documentNamespace"`
// note: found in example documents from SPDX, but not in the JSON schema
// DocumentDescribes []string `json:"documentDescribes"`
Packages []Package `json:"packages"`
// Files referenced in the SPDX document
Files []File `json:"files,omitempty"`
// Snippets referenced in the SPDX document
Snippets []Snippet `json:"snippets,omitempty"`
// Relationships referenced in the SPDX document
Relationships []Relationship `json:"relationships,omitempty"`
}
-12
View File
@@ -1,12 +0,0 @@
package model
type Element struct {
SPDXID string `json:"SPDXID"`
// Identify name of this SpdxElement.
Name string `json:"name,omitempty"`
// Relationships referenced in the SPDX document
Relationships []Relationship `json:"relationships,omitempty"`
// Provide additional information about an SpdxElement.
Annotations []Annotation `json:"annotations,omitempty"`
Comment string `json:"comment,omitempty"`
}
@@ -1,13 +0,0 @@
package model
import "github.com/anchore/syft/syft/formats/common/spdxhelpers"
// ElementID represents the identifier string portion of an SPDX element
// identifier. DocElementID should be used for any attributes which can
// contain identifiers defined in a different SPDX document.
// ElementIDs should NOT contain the mandatory 'SPDXRef-' portion.
type ElementID string
func (e ElementID) String() string {
return "SPDXRef-" + spdxhelpers.SanitizeElementID(string(e))
}
@@ -1,9 +0,0 @@
package model
type ExternalDocumentRef struct {
// externalDocumentId is a string containing letters, numbers, ., - and/or + which uniquely identifies an external document within this document.
ExternalDocumentID string `json:"externalDocumentId"`
Checksum Checksum `json:"checksum"`
// SPDX ID for SpdxDocument. A propoerty containing an SPDX document.
SpdxDocument string `json:"spdxDocument"`
}
-25
View File
@@ -1,25 +0,0 @@
package model
type File struct {
Item
// (At least one is required.) The checksum property provides a mechanism that can be used to verify that the
// contents of a File or Package have not changed.
Checksums []Checksum `json:"checksums,omitempty"`
// This field provides a place for the SPDX file creator to record file contributors. Contributors could include
// names of copyright holders and/or authors who may not be copyright holders yet contributed to the file content.
FileContributors []string `json:"fileContributors,omitempty"`
// Each element is a SPDX ID for a File.
FileDependencies []string `json:"fileDependencies,omitempty"`
// The name of the file relative to the root of the package.
FileName string `json:"fileName"`
// The type of the file
FileTypes []string `json:"fileTypes,omitempty"`
// This field provides a place for the SPDX file creator to record potential legal notices found in the file.
// This may or may not include copyright statements.
NoticeText string `json:"noticeText,omitempty"`
// Indicates the project in which the SpdxElement originated. Tools must preserve doap:homepage and doap:name
// properties and the URI (if one is known) of doap:Project resources that are values of this property. All other
// properties of doap:Projects are not directly supported by SPDX and may be dropped when translating to or
// from some SPDX formats (deprecated).
ArtifactOf []string `json:"artifactOf,omitempty"`
}
@@ -1,14 +0,0 @@
package model
type HasExtractedLicensingInfo struct {
// Verbatim license or licensing notice text that was discovered.
ExtractedText string `json:"extractedText"`
// A human readable short form license identifier for a license. The license ID is iether on the standard license
// oist or the form \"LicenseRef-\"[idString] where [idString] is a unique string containing letters,
// numbers, \".\", \"-\" or \"+\".
LicenseID string `json:"licenseId"`
Comment string `json:"comment,omitempty"`
// Identify name of this SpdxElement.
Name string `json:"name,omitempty"`
SeeAlsos []string `json:"seeAlsos,omitempty"`
}
-22
View File
@@ -1,22 +0,0 @@
package model
type Item struct {
Element
// The licenseComments property allows the preparer of the SPDX document to describe why the licensing in
// spdx:licenseConcluded was chosen.
LicenseComments string `json:"licenseComments,omitempty"`
LicenseConcluded string `json:"licenseConcluded"`
// The licensing information that was discovered directly within the package. There will be an instance of this
// property for each distinct value of alllicenseInfoInFile properties of all files contained in the package.
LicenseInfoFromFiles []string `json:"licenseInfoFromFiles,omitempty"`
// Licensing information that was discovered directly in the subject file. This is also considered a declared license for the file.
LicenseInfoInFiles []string `json:"licenseInfoInFiles,omitempty"`
// The text of copyright declarations recited in the Package or File.
CopyrightText string `json:"copyrightText,omitempty"`
// This field provides a place for the SPDX data creator to record acknowledgements that may be required to be
// communicated in some contexts. This is not meant to include the actual complete license text (see
// licenseConculded and licenseDeclared), and may or may not include copyright notices (see also copyrightText).
// The SPDX data creator may use this field to record other acknowledgements, such as particular clauses from
// license texts, which may be necessary or desirable to reproduce.
AttributionTexts []string `json:"attributionTexts,omitempty"`
}
-52
View File
@@ -1,52 +0,0 @@
package model
import "github.com/anchore/syft/syft/formats/common/spdxhelpers"
type Package struct {
Item
// The checksum property provides a mechanism that can be used to verify that the contents of a File or
// Package have not changed.
Checksums []Checksum `json:"checksums,omitempty"`
// Provides a detailed description of the package.
Description string `json:"description,omitempty"`
// The URI at which this package is available for download. Private (i.e., not publicly reachable) URIs are
// acceptable as values of this property. The values http://spdx.org/rdf/terms#none and http://spdx.org/rdf/terms#noassertion
// may be used to specify that the package is not downloadable or that no attempt was made to determine its
// download location, respectively.
DownloadLocation string `json:"downloadLocation,omitempty"`
// An External Reference allows a Package to reference an external source of additional information, metadata,
// enumerations, asset identifiers, or downloadable content believed to be relevant to the Package.
ExternalRefs []spdxhelpers.ExternalRef `json:"externalRefs,omitempty"`
// Indicates whether the file content of this package has been available for or subjected to analysis when
// creating the SPDX document. If false indicates packages that represent metadata or URI references to a
// project, product, artifact, distribution or a component. If set to false, the package must not contain any files
FilesAnalyzed bool `json:"filesAnalyzed"`
// Indicates that a particular file belongs to a package (elements are SPDX ID for a File).
HasFiles []string `json:"hasFiles,omitempty"`
// Provide a place for the SPDX file creator to record a web site that serves as the package's home page.
// This link can also be used to reference further information about the package referenced by the SPDX file creator.
Homepage string `json:"homepage,omitempty"`
// List the licenses that have been declared by the authors of the package. Any license information that does not
// originate from the package authors, e.g. license information from a third party repository, should not be included in this field.
LicenseDeclared string `json:"licenseDeclared"`
// The name and, optionally, contact information of the person or organization that originally created the package.
// Values of this property must conform to the agent and tool syntax.
Originator string `json:"originator,omitempty"`
// The base name of the package file name. For example, zlib-1.2.5.tar.gz.
PackageFileName string `json:"packageFileName,omitempty"`
// A manifest based verification code (the algorithm is defined in section 4.7 of the full specification) of the
// SPDX Item. This allows consumers of this data and/or database to determine if an SPDX item they have in hand
// is identical to the SPDX item from which the data was produced. This algorithm works even if the SPDX document
// is included in the SPDX item.
PackageVerificationCode *PackageVerificationCode `json:"packageVerificationCode,omitempty"`
// Allows the producer(s) of the SPDX document to describe how the package was acquired and/or changed from the original source.
SourceInfo string `json:"sourceInfo,omitempty"`
// Provides a short description of the package.
Summary string `json:"summary,omitempty"`
// The name and, optionally, contact information of the person or organization who was the immediate supplier
// of this package to the recipient. The supplier may be different than originator when the software has been
// repackaged. Values of this property must conform to the agent and tool syntax.
Supplier string `json:"supplier,omitempty"`
// Provides an indication of the version of the package that is described by this SpdxDocument.
VersionInfo string `json:"versionInfo,omitempty"`
}
@@ -1,23 +0,0 @@
package model
// Why are there two package identifier fields Package Checksum and Package Verification?
// Although the values of the two fields Package Checksum and Package Verification are similar, they each serve a
// different purpose. The Package Checksum provides a unique identifier of a software package which is computed by
// taking the SHA1 of the entire software package file. This enables one to quickly determine if two different copies
// of a package are the same. One disadvantage of this approach is that one cannot add an SPDX data file into the
// original package without changing the Package Checksum value. Alternatively, the Package Verification field enables
// the inclusion of an SPDX file. It enables one to quickly verify if one or more of the original package files has
// changed. The Package Verification field is a unique identifier that is based on SHAing only the original package
// files (e.g., excluding the SPDX file). This allows one to add an SPDX file to the original package without changing
// this unique identifier.
// source: https://wiki.spdx.org/view/SPDX_FAQ
type PackageVerificationCode struct {
// "A file that was excluded when calculating the package verification code. This is usually a file containing
// SPDX data regarding the package. If a package contains more than one SPDX file all SPDX files must be excluded
// from the package verification code. If this is not done it would be impossible to correctly calculate the
// verification codes in both files.
PackageVerificationCodeExcludedFiles []string `json:"packageVerificationCodeExcludedFiles"`
// The actual package verification code as a hex encoded value.
PackageVerificationCodeValue string `json:"packageVerificationCodeValue"`
}
@@ -1,13 +0,0 @@
package model
import "github.com/anchore/syft/syft/formats/common/spdxhelpers"
type Relationship struct {
// Id to which the SPDX element is related
SpdxElementID string `json:"spdxElementId"`
// Describes the type of relationship between two SPDX elements.
RelationshipType spdxhelpers.RelationshipType `json:"relationshipType"`
// SPDX ID for SpdxElement. A related SpdxElement.
RelatedSpdxElement string `json:"relatedSpdxElement"`
Comment string `json:"comment,omitempty"`
}
-32
View File
@@ -1,32 +0,0 @@
package model
type StartPointer struct {
Offset int `json:"offset,omitempty"`
LineNumber int `json:"lineNumber,omitempty"`
// SPDX ID for File
Reference string `json:"reference"`
}
type EndPointer struct {
Offset int `json:"offset,omitempty"`
LineNumber int `json:"lineNumber,omitempty"`
// SPDX ID for File
Reference string `json:"reference"`
}
type Range struct {
StartPointer StartPointer `json:"startPointer"`
EndPointer EndPointer `json:"endPointer"`
}
type Snippet struct {
Item
// Licensing information that was discovered directly in the subject snippet. This is also considered a declared
// license for the snippet. (elements are license expressions)
LicenseInfoInSnippets []string `json:"licenseInfoInSnippets"`
// SPDX ID for File. File containing the SPDX element (e.g. the file contaning a snippet).
SnippetFromFile string `json:"snippetFromFile"`
// (At least 1 range is required). This field defines the byte range in the original host file (in X.2) that the
// snippet information applies to.
Ranges []Range `json:"ranges"`
}
-3
View File
@@ -1,3 +0,0 @@
package model
const Version = "SPDX-2.2"
-268
View File
@@ -1,268 +0,0 @@
package spdx22json
import (
"fmt"
"sort"
"strings"
"time"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/formats/common/spdxhelpers"
"github.com/anchore/syft/syft/formats/common/util"
"github.com/anchore/syft/syft/formats/spdx22json/model"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
// toFormatModel creates and populates a new JSON document struct that follows the SPDX 2.2 spec from the given cataloging results.
func toFormatModel(s sbom.SBOM) *model.Document {
name, namespace := spdxhelpers.DocumentNameAndNamespace(s.Source)
relationships := s.RelationshipsSorted()
return &model.Document{
Element: model.Element{
SPDXID: model.ElementID("DOCUMENT").String(),
Name: name,
},
SPDXVersion: model.Version,
CreationInfo: model.CreationInfo{
Created: time.Now().UTC(),
Creators: []string{
// note: key-value format derived from the JSON example document examples: https://github.com/spdx/spdx-spec/blob/v2.2/examples/SPDXJSONExample-v2.2.spdx.json
"Organization: Anchore, Inc",
"Tool: " + internal.ApplicationName + "-" + s.Descriptor.Version,
},
LicenseListVersion: spdxlicense.Version,
},
DataLicense: "CC0-1.0",
DocumentNamespace: namespace,
Packages: toPackages(s.Artifacts.PackageCatalog, relationships),
Files: toFiles(s),
Relationships: toRelationships(relationships),
}
}
func toPackages(catalog *pkg.Catalog, relationships []artifact.Relationship) []model.Package {
packages := make([]model.Package, 0)
for _, p := range catalog.Sorted() {
license := spdxhelpers.License(p)
packageSpdxID := model.ElementID(p.ID()).String()
checksums, filesAnalyzed := toPackageChecksums(p)
// note: the license concluded and declared should be the same since we are collecting license information
// from the project data itself (the installed package files).
packages = append(packages, model.Package{
Checksums: checksums,
Description: spdxhelpers.Description(p),
DownloadLocation: spdxhelpers.DownloadLocation(p),
ExternalRefs: spdxhelpers.ExternalRefs(p),
FilesAnalyzed: filesAnalyzed,
HasFiles: fileIDsForPackage(packageSpdxID, relationships),
Homepage: spdxhelpers.Homepage(p),
// The Declared License is what the authors of a project believe govern the package
LicenseDeclared: license,
Originator: spdxhelpers.Originator(p),
SourceInfo: spdxhelpers.SourceInfo(p),
VersionInfo: p.Version,
Item: model.Item{
// The Concluded License field is the license the SPDX file creator believes governs the package
LicenseConcluded: license,
Element: model.Element{
SPDXID: packageSpdxID,
Name: p.Name,
},
},
})
}
return packages
}
func toPackageChecksums(p pkg.Package) ([]model.Checksum, bool) {
filesAnalyzed := false
var checksums []model.Checksum
switch meta := p.Metadata.(type) {
// we generate digest for some Java packages
// see page 33 of the spdx specification for 2.2
// spdx.github.io/spdx-spec/package-information/#710-package-checksum-field
case pkg.JavaMetadata:
if len(meta.ArchiveDigests) > 0 {
filesAnalyzed = true
for _, digest := range meta.ArchiveDigests {
checksums = append(checksums, model.Checksum{
Algorithm: strings.ToUpper(digest.Algorithm),
ChecksumValue: digest.Value,
})
}
}
case pkg.GolangBinMetadata:
algo, hexStr, err := util.HDigestToSHA(meta.H1Digest)
if err != nil {
log.Debugf("invalid h1digest: %s: %v", meta.H1Digest, err)
break
}
algo = strings.ToUpper(algo)
checksums = append(checksums, model.Checksum{
Algorithm: strings.ToUpper(algo),
ChecksumValue: hexStr,
})
}
return checksums, filesAnalyzed
}
func fileIDsForPackage(packageSpdxID string, relationships []artifact.Relationship) (fileIDs []string) {
for _, relationship := range relationships {
if relationship.Type != artifact.ContainsRelationship {
continue
}
if _, ok := relationship.From.(pkg.Package); !ok {
continue
}
if _, ok := relationship.To.(source.Coordinates); !ok {
continue
}
from := model.ElementID(relationship.From.ID()).String()
if from == packageSpdxID {
to := model.ElementID(relationship.To.ID()).String()
fileIDs = append(fileIDs, to)
}
}
return fileIDs
}
func toFiles(s sbom.SBOM) []model.File {
results := make([]model.File, 0)
artifacts := s.Artifacts
for _, coordinates := range s.AllCoordinates() {
var metadata *source.FileMetadata
if metadataForLocation, exists := artifacts.FileMetadata[coordinates]; exists {
metadata = &metadataForLocation
}
var digests []file.Digest
if digestsForLocation, exists := artifacts.FileDigests[coordinates]; exists {
digests = digestsForLocation
}
// TODO: add file classifications (?) and content as a snippet
var comment string
if coordinates.FileSystemID != "" {
comment = fmt.Sprintf("layerID: %s", coordinates.FileSystemID)
}
results = append(results, model.File{
Item: model.Item{
Element: model.Element{
SPDXID: model.ElementID(coordinates.ID()).String(),
Comment: comment,
},
// required, no attempt made to determine license information
LicenseConcluded: "NOASSERTION",
},
Checksums: toFileChecksums(digests),
FileName: coordinates.RealPath,
FileTypes: toFileTypes(metadata),
})
}
// sort by real path then virtual path to ensure the result is stable across multiple runs
sort.SliceStable(results, func(i, j int) bool {
if results[i].FileName == results[j].FileName {
return results[i].SPDXID < results[j].SPDXID
}
return results[i].FileName < results[j].FileName
})
return results
}
func toFileChecksums(digests []file.Digest) (checksums []model.Checksum) {
for _, digest := range digests {
checksums = append(checksums, model.Checksum{
Algorithm: toChecksumAlgorithm(digest.Algorithm),
ChecksumValue: digest.Value,
})
}
return checksums
}
func toChecksumAlgorithm(algorithm string) string {
// basically, we need an uppercase version of our algorithm:
// https://github.com/spdx/spdx-spec/blob/development/v2.2.2/schemas/spdx-schema.json#L165
return strings.ToUpper(algorithm)
}
func toFileTypes(metadata *source.FileMetadata) (ty []string) {
if metadata == nil {
return nil
}
mimeTypePrefix := strings.Split(metadata.MIMEType, "/")[0]
switch mimeTypePrefix {
case "image":
ty = append(ty, string(spdxhelpers.ImageFileType))
case "video":
ty = append(ty, string(spdxhelpers.VideoFileType))
case "application":
ty = append(ty, string(spdxhelpers.ApplicationFileType))
case "text":
ty = append(ty, string(spdxhelpers.TextFileType))
case "audio":
ty = append(ty, string(spdxhelpers.AudioFileType))
}
if internal.IsExecutable(metadata.MIMEType) {
ty = append(ty, string(spdxhelpers.BinaryFileType))
}
if internal.IsArchive(metadata.MIMEType) {
ty = append(ty, string(spdxhelpers.ArchiveFileType))
}
// TODO: add support for source, spdx, and documentation file types
if len(ty) == 0 {
ty = append(ty, string(spdxhelpers.OtherFileType))
}
return ty
}
func toRelationships(relationships []artifact.Relationship) (result []model.Relationship) {
for _, r := range relationships {
exists, relationshipType, comment := lookupRelationship(r.Type)
if !exists {
log.Warnf("unable to convert relationship from SPDX 2.2 JSON, dropping: %+v", r)
continue
}
result = append(result, model.Relationship{
SpdxElementID: model.ElementID(r.From.ID()).String(),
RelationshipType: relationshipType,
RelatedSpdxElement: model.ElementID(r.To.ID()).String(),
Comment: comment,
})
}
return result
}
func lookupRelationship(ty artifact.RelationshipType) (bool, spdxhelpers.RelationshipType, string) {
switch ty {
case artifact.ContainsRelationship:
return true, spdxhelpers.ContainsRelationship, ""
case artifact.OwnershipByFileOverlapRelationship:
return true, spdxhelpers.OtherRelationship, fmt.Sprintf("%s: indicates that the parent package claims ownership of a child package since the parent metadata indicates overlap with a location that a cataloger found the child package by", ty)
}
return false, "", ""
}
-14
View File
@@ -1,14 +0,0 @@
package spdx22tagvalue
import (
"io"
"github.com/spdx/tools-golang/tvsaver"
"github.com/anchore/syft/syft/sbom"
)
func encoder(output io.Writer, s sbom.SBOM) error {
model := toFormatModel(s)
return tvsaver.Save2_2(model, output)
}
@@ -1,311 +0,0 @@
package spdx22tagvalue
import (
"fmt"
"strings"
"time"
"github.com/spdx/tools-golang/spdx"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/formats/common/spdxhelpers"
"github.com/anchore/syft/syft/formats/common/util"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
)
// toFormatModel creates and populates a new JSON document struct that follows the SPDX 2.2 spec from the given cataloging results.
//
//nolint:funlen
func toFormatModel(s sbom.SBOM) *spdx.Document2_2 {
name, namespace := spdxhelpers.DocumentNameAndNamespace(s.Source)
return &spdx.Document2_2{
CreationInfo: &spdx.CreationInfo2_2{
// 2.1: SPDX Version; should be in the format "SPDX-2.2"
// Cardinality: mandatory, one
SPDXVersion: "SPDX-2.2",
// 2.2: Data License; should be "CC0-1.0"
// Cardinality: mandatory, one
DataLicense: "CC0-1.0",
// 2.3: SPDX Identifier; should be "DOCUMENT" to represent mandatory identifier of SPDXRef-DOCUMENT
// Cardinality: mandatory, one
SPDXIdentifier: spdx.ElementID("DOCUMENT"),
// 2.4: Document Name
// Cardinality: mandatory, one
DocumentName: name,
// 2.5: Document Namespace
// Cardinality: mandatory, one
// Purpose: Provide an SPDX document specific namespace as a unique absolute Uniform Resource
// Identifier (URI) as specified in RFC-3986, with the exception of the ‘#’ delimiter. The SPDX
// Document URI cannot contain a URI "part" (e.g. the "#" character), since the ‘#’ is used in SPDX
// element URIs (packages, files, snippets, etc) to separate the document namespace from the
// element’s SPDX identifier. Additionally, a scheme (e.g. “https:”) is required.
// The URI must be unique for the SPDX document including the specific version of the SPDX document.
// If the SPDX document is updated, thereby creating a new version, a new URI for the updated
// document must be used. There can only be one URI for an SPDX document and only one SPDX document
// for a given URI.
// Note that the URI does not have to be accessible. It is only intended to provide a unique ID.
// In many cases, the URI will point to a web accessible document, but this should not be assumed
// to be the case.
DocumentNamespace: namespace,
// 2.6: External Document References
// Cardinality: optional, one or many
ExternalDocumentReferences: nil,
// 2.7: License List Version
// Cardinality: optional, one
LicenseListVersion: spdxlicense.Version,
// 2.8: Creators: may have multiple keys for Person, Organization
// and/or Tool
// Cardinality: mandatory, one or many
CreatorPersons: nil,
CreatorOrganizations: []string{"Anchore, Inc"},
CreatorTools: []string{internal.ApplicationName + "-" + s.Descriptor.Version},
// 2.9: Created: data format YYYY-MM-DDThh:mm:ssZ
// Cardinality: mandatory, one
Created: time.Now().UTC().Format(time.RFC3339),
// 2.10: Creator Comment
// Cardinality: optional, one
CreatorComment: "",
// 2.11: Document Comment
// Cardinality: optional, one
DocumentComment: "",
},
Packages: toFormatPackages(s.Artifacts.PackageCatalog),
}
}
// packages populates all Package Information from the package Catalog (see https://spdx.github.io/spdx-spec/3-package-information/)
//
//nolint:funlen
func toFormatPackages(catalog *pkg.Catalog) map[spdx.ElementID]*spdx.Package2_2 {
results := make(map[spdx.ElementID]*spdx.Package2_2)
for _, p := range catalog.Sorted() {
// name should be guaranteed to be unique, but semantically useful and stable
id := spdxhelpers.SanitizeElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.ID()))
// If the Concluded License is not the same as the Declared License, a written explanation should be provided
// in the Comments on License field (section 3.16). With respect to NOASSERTION, a written explanation in
// the Comments on License field (section 3.16) is preferred.
license := spdxhelpers.License(p)
checksums, filesAnalyzed := toPackageChecksums(p)
results[spdx.ElementID(id)] = &spdx.Package2_2{
// NOT PART OF SPEC
// flag: does this "package" contain files that were in fact "unpackaged",
// e.g. included directly in the Document without being in a Package?
IsUnpackaged: false,
// 3.1: Package Name
// Cardinality: mandatory, one
PackageName: p.Name,
// 3.2: Package SPDX Identifier: "SPDXRef-[idstring]"
// Cardinality: mandatory, one
PackageSPDXIdentifier: spdx.ElementID(id),
// 3.3: Package Version
// Cardinality: optional, one
PackageVersion: p.Version,
// 3.4: Package File Name
// Cardinality: optional, one
PackageFileName: "",
// 3.5: Package Supplier: may have single result for either Person or Organization,
// or NOASSERTION
// Cardinality: optional, one
PackageSupplierPerson: "",
PackageSupplierOrganization: "",
PackageSupplierNOASSERTION: false,
// 3.6: Package Originator: may have single result for either Person or Organization,
// or NOASSERTION
// Cardinality: optional, one
PackageOriginatorPerson: "",
PackageOriginatorOrganization: "",
PackageOriginatorNOASSERTION: false,
// 3.7: Package Download Location
// Cardinality: mandatory, one
// NONE if there is no download location whatsoever.
// NOASSERTION if:
// (i) the SPDX file creator has attempted to but cannot reach a reasonable objective determination;
// (ii) the SPDX file creator has made no attempt to determine this field; or
// (iii) the SPDX file creator has intentionally provided no information (no meaning should be implied by doing so).
PackageDownloadLocation: "NOASSERTION",
// 3.8: FilesAnalyzed
// Cardinality: optional, one; default value is "true" if omitted
// Purpose: Indicates whether the file content of this package has been available for or subjected to
// analysis when creating the SPDX document. If false, indicates packages that represent metadata or
// URI references to a project, product, artifact, distribution or a component. If false, the package
// must not contain any files.
// Intent: A package can refer to a project, product, artifact, distribution or a component that is
// external to the SPDX document.
FilesAnalyzed: filesAnalyzed,
// NOT PART OF SPEC: did FilesAnalyzed tag appear?
IsFilesAnalyzedTagPresent: true,
// 3.9: Package Verification Code
// Cardinality: optional, one if filesAnalyzed is true / omitted;
// zero (must be omitted) if filesAnalyzed is false
PackageVerificationCode: "",
// Spec also allows specifying a single file to exclude from the
// verification code algorithm; intended to enable exclusion of
// the SPDX document file itself.
PackageVerificationCodeExcludedFile: "",
// 3.10: Package Checksum: may have keys for SHA1, SHA256 and/or MD5
// Cardinality: optional, one or many
// 3.10.1 Purpose: Provide an independently reproducible mechanism that permits unique identification of
// a specific package that correlates to the data in this SPDX file. This identifier enables a recipient
// to determine if any file in the original package has been changed. If the SPDX file is to be included
// in a package, this value should not be calculated. The SHA-1 algorithm will be used to provide the
// checksum by default.
PackageChecksums: checksums,
// note: based on the purpose above no discovered checksums should be provided, but instead, only
// tool-derived checksums.
//FIXME: this got removed between 0.1.0 and 0.2.0, is this right? it looks like
// it wasn't being used anyway
//PackageChecksumSHA1: "",
//PackageChecksumSHA256: "",
//PackageChecksumMD5: "",
// 3.11: Package Home Page
// Cardinality: optional, one
PackageHomePage: "",
// 3.12: Source Information
// Cardinality: optional, one
PackageSourceInfo: "",
// 3.13: Concluded License: SPDX License Expression, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one
// Purpose: Contain the license the SPDX file creator has concluded as governing the
// package or alternative values, if the governing license cannot be determined.
PackageLicenseConcluded: license,
// 3.14: All Licenses Info from Files: SPDX License Expression, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one or many if filesAnalyzed is true / omitted;
// zero (must be omitted) if filesAnalyzed is false
PackageLicenseInfoFromFiles: nil,
// 3.15: Declared License: SPDX License Expression, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one
// Purpose: List the licenses that have been declared by the authors of the package.
// Any license information that does not originate from the package authors, e.g. license
// information from a third party repository, should not be included in this field.
PackageLicenseDeclared: license,
// 3.16: Comments on License
// Cardinality: optional, one
PackageLicenseComments: "",
// 3.17: Copyright Text: copyright notice(s) text, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one
// Purpose: IdentifyFormat the copyright holders of the package, as well as any dates present. This will be a free form text field extracted from package information files. The options to populate this field are limited to:
//
// Any text related to a copyright notice, even if not complete;
// NONE if the package contains no copyright information whatsoever; or
// NOASSERTION, if
// (i) the SPDX document creator has made no attempt to determine this field; or
// (ii) the SPDX document creator has intentionally provided no information (no meaning should be implied by doing so).
//
PackageCopyrightText: "NOASSERTION",
// 3.18: Package Summary Description
// Cardinality: optional, one
PackageSummary: "",
// 3.19: Package Detailed Description
// Cardinality: optional, one
PackageDescription: "",
// 3.20: Package Comment
// Cardinality: optional, one
PackageComment: "",
// 3.21: Package External Reference
// Cardinality: optional, one or many
PackageExternalReferences: formatSPDXExternalRefs(p),
// 3.22: Package External Reference Comment
// Cardinality: conditional (optional, one) for each External Reference
// contained within PackageExternalReference2_1 struct, if present
// 3.23: Package Attribution Text
// Cardinality: optional, one or many
PackageAttributionTexts: nil,
// Files contained in this Package
Files: nil,
}
}
return results
}
func toPackageChecksums(p pkg.Package) (map[spdx.ChecksumAlgorithm]spdx.Checksum, bool) {
filesAnalyzed := false
checksums := map[spdx.ChecksumAlgorithm]spdx.Checksum{}
switch meta := p.Metadata.(type) {
// we generate digest for some Java packages
// see page 33 of the spdx specification for 2.2
// spdx.github.io/spdx-spec/package-information/#710-package-checksum-field
case pkg.JavaMetadata:
if len(meta.ArchiveDigests) > 0 {
filesAnalyzed = true
for _, digest := range meta.ArchiveDigests {
checksums[spdx.ChecksumAlgorithm(digest.Algorithm)] = spdx.Checksum{
Algorithm: spdx.ChecksumAlgorithm(digest.Algorithm),
Value: digest.Value,
}
}
}
case pkg.GolangBinMetadata:
algo, hexStr, err := util.HDigestToSHA(meta.H1Digest)
if err != nil {
log.Debugf("invalid h1digest: %s: %v", meta.H1Digest, err)
break
}
algo = strings.ToUpper(algo)
checksums[spdx.ChecksumAlgorithm(algo)] = spdx.Checksum{
Algorithm: spdx.ChecksumAlgorithm(algo),
Value: hexStr,
}
}
return checksums, filesAnalyzed
}
func formatSPDXExternalRefs(p pkg.Package) (refs []*spdx.PackageExternalReference2_2) {
for _, ref := range spdxhelpers.ExternalRefs(p) {
refs = append(refs, &spdx.PackageExternalReference2_2{
Category: string(ref.ReferenceCategory),
RefType: string(ref.ReferenceType),
Locator: ref.ReferenceLocator,
ExternalRefComment: ref.Comment,
})
}
return refs
}
@@ -1,73 +0,0 @@
package spdx22tagvalue
import (
"fmt"
"testing"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/pkg"
)
func Test_H1Digest(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expectedDigest string
}{
{
name: "valid h1digest",
pkg: pkg.Package{
Name: "github.com/googleapis/gnostic",
Version: "v0.5.5",
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
},
expectedDigest: "SHA256:f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
},
{
name: "invalid h1digest",
pkg: pkg.Package{
Name: "github.com/googleapis/gnostic",
Version: "v0.5.5",
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h1:9fHAtK0uzzz",
},
},
expectedDigest: "",
},
{
name: "unsupported h-digest",
pkg: pkg.Package{
Name: "github.com/googleapis/gnostic",
Version: "v0.5.5",
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h12:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
},
expectedDigest: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
catalog := pkg.NewCatalog(test.pkg)
pkgs := toFormatPackages(catalog)
require.Len(t, pkgs, 1)
for _, p := range pkgs {
if test.expectedDigest == "" {
require.Len(t, p.PackageChecksums, 0)
} else {
require.Len(t, p.PackageChecksums, 1)
for _, c := range p.PackageChecksums {
require.Equal(t, test.expectedDigest, fmt.Sprintf("%s:%s", c.Algorithm, c.Value))
}
}
}
})
}
}
+20
View File
@@ -0,0 +1,20 @@
package spdxjson
import (
"fmt"
"io"
spdx "github.com/spdx/tools-golang/json"
"github.com/anchore/syft/syft/formats/common/spdxhelpers"
"github.com/anchore/syft/syft/sbom"
)
func decoder(reader io.Reader) (s *sbom.SBOM, err error) {
doc, err := spdx.Load2_3(reader)
if err != nil {
return nil, fmt.Errorf("unable to decode spdx-json: %w", err)
}
return spdxhelpers.ToSyftModel(doc)
}
@@ -1,4 +1,4 @@
package spdx22json
package spdxjson
import (
"fmt"
@@ -6,6 +6,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/pkg"
)
@@ -60,15 +61,15 @@ func TestSPDXJSONDecoder(t *testing.T) {
for _, test := range tests {
t.Run(test.path, func(t *testing.T) {
f, err := os.Open("test-fixtures/spdx/" + test.path)
assert.NoError(t, err)
require.NoError(t, err)
sbom, err := decoder(f)
if test.fail {
assert.Error(t, err)
require.Error(t, err)
return
} else {
assert.NoError(t, err)
require.NoError(t, err)
}
if test.packages != nil {
@@ -1,14 +1,15 @@
package spdx22json
package spdxjson
import (
"encoding/json"
"io"
"github.com/anchore/syft/syft/formats/common/spdxhelpers"
"github.com/anchore/syft/syft/sbom"
)
func encoder(output io.Writer, s sbom.SBOM) error {
doc := toFormatModel(s)
doc := spdxhelpers.ToFormatModel(s)
enc := json.NewEncoder(output)
// prevent > and < from being escaped in the payload
@@ -1,14 +1,11 @@
package spdx22json
package spdxjson
import (
"flag"
"regexp"
"testing"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/formats/common/testutils"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
var updateSpdxJson = flag.Bool("update-spdx-json", false, "update the *.golden files for spdx-json encoders")
@@ -36,7 +33,7 @@ func TestSPDXJSONImageEncoder(t *testing.T) {
func TestSPDXRelationshipOrder(t *testing.T) {
testImage := "image-simple"
s := testutils.ImageInput(t, testImage, testutils.FromSnapshot())
addRelationships(&s)
testutils.AddSampleFileRelationships(&s)
testutils.AssertEncoderAgainstGoldenImageSnapshot(t,
Format(),
s,
@@ -46,23 +43,6 @@ func TestSPDXRelationshipOrder(t *testing.T) {
)
}
func addRelationships(s *sbom.SBOM) {
catalog := s.Artifacts.PackageCatalog.Sorted()
s.Artifacts.FileMetadata = map[source.Coordinates]source.FileMetadata{}
for _, f := range []string{"/f1", "/f2", "/d1/f3", "/d2/f4", "/z1/f5", "/a1/f6"} {
meta := source.FileMetadata{}
coords := source.Coordinates{RealPath: f}
s.Artifacts.FileMetadata[coords] = meta
s.Relationships = append(s.Relationships, artifact.Relationship{
From: catalog[0],
To: coords,
Type: artifact.ContainsRelationship,
})
}
}
func spdxJsonRedactor(s []byte) []byte {
// each SBOM reports the time it was generated, which is not useful during snapshot testing
s = regexp.MustCompile(`"created": .*`).ReplaceAll(s, []byte("redacted"))
@@ -1,4 +1,4 @@
package spdx22json
package spdxjson
import (
"github.com/anchore/syft/syft/sbom"
@@ -1,61 +1,66 @@
{
"spdxVersion": "SPDX-2.3",
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "/some/path",
"spdxVersion": "SPDX-2.2",
"documentNamespace": "https://anchore.com/syft/dir/some/path-4bf54cdd-0a0f-4560-bf4f-39cac2ef7a5b",
"creationInfo": {
"created": "2022-10-24T13:54:19.225779Z",
"licenseListVersion": "3.18",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-v0.42.0-bogus"
],
"licenseListVersion": "3.18"
"created": "2022-11-19T13:46:57Z",
"comment": ""
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https://anchore.com/syft/dir/some/path-cd89c782-240b-461e-81a1-63863e02642f",
"packages": [
{
"SPDXID": "SPDXRef-e624319940d8d36a",
"name": "package-1",
"licenseConcluded": "MIT",
"SPDXID": "SPDXRef-Package-python-package-1-1b1d0be59ac59d2c",
"versionInfo": "1.0.1",
"downloadLocation": "NOASSERTION",
"sourceInfo": "acquired package info from installed python package manifest file: /some/path/pkg1",
"licenseConcluded": "MIT",
"licenseDeclared": "MIT",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
"comment": ""
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "a-purl-2",
"referenceType": "purl"
"comment": ""
}
],
"filesAnalyzed": false,
"licenseDeclared": "MIT",
"sourceInfo": "acquired package info from installed python package manifest file: /some/path/pkg1",
"versionInfo": "1.0.1"
]
},
{
"SPDXID": "SPDXRef-b8645f4ac2a0891e",
"name": "package-2",
"licenseConcluded": "NONE",
"SPDXID": "SPDXRef-Package-deb-package-2-db4abfe497c180d3",
"versionInfo": "2.0.1",
"downloadLocation": "NOASSERTION",
"sourceInfo": "acquired package info from DPKG DB: /some/path/pkg1",
"licenseConcluded": "NONE",
"licenseDeclared": "NONE",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
"comment": ""
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:deb/debian/package-2@2.0.1",
"referenceType": "purl"
"comment": ""
}
],
"filesAnalyzed": false,
"licenseDeclared": "NONE",
"sourceInfo": "acquired package info from DPKG DB: /some/path/pkg1",
"versionInfo": "2.0.1"
]
}
]
}
@@ -1,61 +1,66 @@
{
"spdxVersion": "SPDX-2.3",
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "user-image-input",
"spdxVersion": "SPDX-2.2",
"documentNamespace": "https://anchore.com/syft/image/user-image-input-102ca7dc-3d1e-46d2-b130-28968831ebcc",
"creationInfo": {
"created": "2022-10-24T13:54:19.477217Z",
"licenseListVersion": "3.18",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-v0.42.0-bogus"
],
"licenseListVersion": "3.18"
"created": "2022-11-19T13:46:57Z",
"comment": ""
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https://anchore.com/syft/image/user-image-input-0b40ce75-7e54-4760-bd9d-4fa833b352dd",
"packages": [
{
"SPDXID": "SPDXRef-5ffee24fb164cffc",
"name": "package-1",
"licenseConcluded": "MIT",
"SPDXID": "SPDXRef-Package-python-package-1-66ba429119b8bec6",
"versionInfo": "1.0.1",
"downloadLocation": "NOASSERTION",
"sourceInfo": "acquired package info from installed python package manifest file: /somefile-1.txt",
"licenseConcluded": "MIT",
"licenseDeclared": "MIT",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:1:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
"comment": ""
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "a-purl-1",
"referenceType": "purl"
"comment": ""
}
],
"filesAnalyzed": false,
"licenseDeclared": "MIT",
"sourceInfo": "acquired package info from installed python package manifest file: /somefile-1.txt",
"versionInfo": "1.0.1"
]
},
{
"SPDXID": "SPDXRef-8b16570b2b4155c3",
"name": "package-2",
"licenseConcluded": "NONE",
"SPDXID": "SPDXRef-Package-deb-package-2-958443e2d9304af4",
"versionInfo": "2.0.1",
"downloadLocation": "NOASSERTION",
"sourceInfo": "acquired package info from DPKG DB: /somefile-2.txt",
"licenseConcluded": "NONE",
"licenseDeclared": "NONE",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
"comment": ""
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:deb/debian/package-2@2.0.1",
"referenceType": "purl"
"comment": ""
}
],
"filesAnalyzed": false,
"licenseDeclared": "NONE",
"sourceInfo": "acquired package info from DPKG DB: /somefile-2.txt",
"versionInfo": "2.0.1"
]
}
]
}
@@ -1,151 +1,160 @@
{
"spdxVersion": "SPDX-2.3",
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "user-image-input",
"spdxVersion": "SPDX-2.2",
"documentNamespace": "https://anchore.com/syft/image/user-image-input-55ad4afc-ecdc-46a4-8bc3-36b3e72da5d1",
"creationInfo": {
"created": "2022-10-24T13:54:19.48428Z",
"licenseListVersion": "3.18",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-v0.42.0-bogus"
],
"licenseListVersion": "3.18"
"created": "2022-11-19T13:46:57Z",
"comment": ""
},
"dataLicense": "CC0-1.0",
"documentNamespace": "https://anchore.com/syft/image/user-image-input-1a4dc179-1222-463c-b4e9-619131af7e97",
"packages": [
{
"SPDXID": "SPDXRef-5ffee24fb164cffc",
"name": "package-1",
"licenseConcluded": "MIT",
"SPDXID": "SPDXRef-Package-python-package-1-66ba429119b8bec6",
"versionInfo": "1.0.1",
"downloadLocation": "NOASSERTION",
"sourceInfo": "acquired package info from installed python package manifest file: /somefile-1.txt",
"licenseConcluded": "MIT",
"licenseDeclared": "MIT",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:1:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
"comment": ""
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "a-purl-1",
"referenceType": "purl"
"comment": ""
}
],
"filesAnalyzed": false,
"hasFiles": [
"SPDXRef-5265a4dde3edbf7c",
"SPDXRef-839d99ee67d9d174",
"SPDXRef-9c2f7510199b17f6",
"SPDXRef-c641caa71518099f",
"SPDXRef-c6f5b29dca12661f",
"SPDXRef-f9e49132a4b96ccd"
],
"licenseDeclared": "MIT",
"sourceInfo": "acquired package info from installed python package manifest file: /somefile-1.txt",
"versionInfo": "1.0.1"
]
},
{
"SPDXID": "SPDXRef-8b16570b2b4155c3",
"name": "package-2",
"licenseConcluded": "NONE",
"SPDXID": "SPDXRef-Package-deb-package-2-958443e2d9304af4",
"versionInfo": "2.0.1",
"downloadLocation": "NOASSERTION",
"sourceInfo": "acquired package info from DPKG DB: /somefile-2.txt",
"licenseConcluded": "NONE",
"licenseDeclared": "NONE",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
"comment": ""
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:deb/debian/package-2@2.0.1",
"referenceType": "purl"
"comment": ""
}
],
"filesAnalyzed": false,
"licenseDeclared": "NONE",
"sourceInfo": "acquired package info from DPKG DB: /somefile-2.txt",
"versionInfo": "2.0.1"
]
}
],
"files": [
{
"SPDXID": "SPDXRef-9c2f7510199b17f6",
"licenseConcluded": "NOASSERTION",
"fileName": "/a1/f6",
"SPDXID": "SPDXRef-9c2f7510199b17f6",
"fileTypes": [
"OTHER"
]
],
"checksums": null,
"licenseConcluded": "NOASSERTION",
"copyrightText": ""
},
{
"SPDXID": "SPDXRef-c6f5b29dca12661f",
"licenseConcluded": "NOASSERTION",
"fileName": "/d1/f3",
"SPDXID": "SPDXRef-c6f5b29dca12661f",
"fileTypes": [
"OTHER"
]
],
"checksums": null,
"licenseConcluded": "NOASSERTION",
"copyrightText": ""
},
{
"SPDXID": "SPDXRef-c641caa71518099f",
"licenseConcluded": "NOASSERTION",
"fileName": "/d2/f4",
"SPDXID": "SPDXRef-c641caa71518099f",
"fileTypes": [
"OTHER"
]
],
"checksums": null,
"licenseConcluded": "NOASSERTION",
"copyrightText": ""
},
{
"SPDXID": "SPDXRef-5265a4dde3edbf7c",
"licenseConcluded": "NOASSERTION",
"fileName": "/f1",
"SPDXID": "SPDXRef-5265a4dde3edbf7c",
"fileTypes": [
"OTHER"
]
],
"checksums": null,
"licenseConcluded": "NOASSERTION",
"copyrightText": ""
},
{
"SPDXID": "SPDXRef-f9e49132a4b96ccd",
"licenseConcluded": "NOASSERTION",
"fileName": "/f2",
"SPDXID": "SPDXRef-f9e49132a4b96ccd",
"fileTypes": [
"OTHER"
]
],
"checksums": null,
"licenseConcluded": "NOASSERTION",
"copyrightText": ""
},
{
"SPDXID": "SPDXRef-839d99ee67d9d174",
"licenseConcluded": "NOASSERTION",
"fileName": "/z1/f5",
"SPDXID": "SPDXRef-839d99ee67d9d174",
"fileTypes": [
"OTHER"
]
],
"checksums": null,
"licenseConcluded": "NOASSERTION",
"copyrightText": ""
}
],
"relationships": [
{
"spdxElementId": "SPDXRef-5ffee24fb164cffc",
"relationshipType": "CONTAINS",
"relatedSpdxElement": "SPDXRef-5265a4dde3edbf7c"
"spdxElementId": "SPDXRef-Package-python-package-1-66ba429119b8bec6",
"relatedSpdxElement": "SPDXRef-5265a4dde3edbf7c",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-5ffee24fb164cffc",
"relationshipType": "CONTAINS",
"relatedSpdxElement": "SPDXRef-839d99ee67d9d174"
"spdxElementId": "SPDXRef-Package-python-package-1-66ba429119b8bec6",
"relatedSpdxElement": "SPDXRef-839d99ee67d9d174",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-5ffee24fb164cffc",
"relationshipType": "CONTAINS",
"relatedSpdxElement": "SPDXRef-9c2f7510199b17f6"
"spdxElementId": "SPDXRef-Package-python-package-1-66ba429119b8bec6",
"relatedSpdxElement": "SPDXRef-9c2f7510199b17f6",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-5ffee24fb164cffc",
"relationshipType": "CONTAINS",
"relatedSpdxElement": "SPDXRef-c641caa71518099f"
"spdxElementId": "SPDXRef-Package-python-package-1-66ba429119b8bec6",
"relatedSpdxElement": "SPDXRef-c641caa71518099f",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-5ffee24fb164cffc",
"relationshipType": "CONTAINS",
"relatedSpdxElement": "SPDXRef-c6f5b29dca12661f"
"spdxElementId": "SPDXRef-Package-python-package-1-66ba429119b8bec6",
"relatedSpdxElement": "SPDXRef-c6f5b29dca12661f",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-5ffee24fb164cffc",
"relationshipType": "CONTAINS",
"relatedSpdxElement": "SPDXRef-f9e49132a4b96ccd"
"spdxElementId": "SPDXRef-Package-python-package-1-66ba429119b8bec6",
"relatedSpdxElement": "SPDXRef-f9e49132a4b96ccd",
"relationshipType": "CONTAINS"
}
]
}

Some files were not shown because too many files have changed in this diff Show More