Compare commits

...
48 Commits
Author SHA1 Message Date
Keith Zantow cc0a376aba fix: more python matching support (#1667) 2023-03-13 13:26:43 -04:00
anchore-actions-token-generator[bot]andspiffcs b379dd9f27 Update syft bootstrap tools to latest versions. (#1666)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2023-03-13 10:40:13 -04:00
witchcraze a81e0c8008 feat: add ruby classifier (#1665)
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-03-10 08:29:40 -05:00
anchore-actions-token-generator[bot] 41cbbe09b2 Update syft bootstrap tools to latest versions. (#1658) 2023-03-07 12:54:32 -05:00
Keith Zantow 7714bc0521 fix: improved Python binary detection (#1648) 2023-03-07 10:52:29 -05:00
Weston Steimel 096d2b7bff fix: suppress some known incorrect vendor candidates for npm CPEs (#1659)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-07 10:18:44 -05:00
Keith Zantow 7cfdffab5f fix: sanitize SPDX LicenseRefs (#1657)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-03-06 10:55:23 -05:00
dependabot[bot] f43953d225 chore(deps): bump golang.org/x/mod from 0.8.0 to 0.9.0 (#1655) 2023-03-06 15:49:34 +00:00
dependabot[bot] eea1b48cbb chore(deps): bump golang.org/x/net from 0.7.0 to 0.8.0 (#1653) 2023-03-06 15:38:34 +00:00
dependabot[bot] a063cf300b chore(deps): bump github.com/spf13/afero from 1.9.4 to 1.9.5 (#1654) 2023-03-06 15:21:35 +00:00
dependabot[bot] b73903519c chore(deps): bump golang.org/x/term from 0.5.0 to 0.6.0 (#1656) 2023-03-06 15:20:43 +00:00
Keith Zantow 304be4a5a1 fix: dotnet PURL types are invalid (#1649)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-03-03 16:45:20 -05:00
Weston Steimel c4cbe211a3 feat: disable cpe vendor wildcards to reduce false positives (#1647)
* improved parsing of vendor from github url

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* stop generating wildcard vendors

Add logic for parsing javascript and ruby package vendor candidates from
url and author fields and stop generating wildcard vendor candidates

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

---------

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-03 17:26:46 +00:00
Avi Deitcher 01230aa766 read relative etc/apk/repositories for alpine version when no OS provided (#1615)
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2023-03-02 13:04:56 -05:00
Keith Zantow 5f90d03718 fix: possible race condition (#1639) 2023-03-01 15:35:01 -05:00
Weston Steimel e2ebc9769f fix: remove APK OriginPackage cpe candidates (#1637)
Adding APK OriginPackage CPE candidates to the child package
results in false positives in grype because it can't associate
CPE-based findings to the corresponding OriginPackage APK fixes.

This reverts changing the `upstream` in the PURL for APK packages
as the logic in Grype that uses it expects it to be an APK package
name.  This also allows refactoring to unexport and move the APK
CPE candidate generation logic closer to where CPE generation occurs

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 17:24:43 +00:00
Keith Zantow 2e6e3b0c74 fix: rebar lock file decoding panic (#1628) 2023-03-01 10:08:29 -05:00
Keith Zantow 24584a4d27 fix: handle individual cataloger panics (#1636) 2023-03-01 10:03:34 -05:00
Weston Steimel 8e1205f7ab fix: apk product/vendor generation for old metadata (#1635)
This fixes some instances where the improved APK CPE generation
logic caused regressions for older alpine package APK metadata.
It now generates multiple "upstream" candidates with both name
and package type which reduces the amount of duplicated code in
the apk cpe gen logic.  This also improves the handling of stream
version packages, so now we can correctly identify packages such
as ruby3.2-rexml as the rexml ruby gem.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 14:58:35 +00:00
Weston Steimel e92b0fa629 feat: rust toolchain binary cataloger (#1601)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 14:53:37 +00:00
Weston Steimel bcc0751a40 feat: retain go package info when no module declared (#1632)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 14:26:44 +00:00
Weston Steimel f1169e56fc fix: improved CPE-generation for several more APK packages (#1631)
* fix: correct vendor for musl

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor for firefox and thunderbird

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor/product for chromium

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct product for apache http server

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct product for tiff

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor for ghostscript

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor for openjpeg

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor/product for xorg-server

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor for podofo

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: correct vendor for wpa_supplicant

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

---------

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 08:55:40 -05:00
Christopher Angelo Phillips 98e737fc27 chore: update deprecated release flag (#1629)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-02-27 15:57:56 -05:00
dependabot[bot] ff34594284 chore(deps): bump actions/upload-artifact from 2 to 3 (#1627)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-27 14:17:29 -05:00
Benji Visser 9e953b1da3 feat: add support for SUPPORT_END in /etc/os-release (#1612)
Signed-off-by: Benji Visser <benji@093b.org>
2023-02-27 13:43:19 -05:00
Weston Steimel fbda21f4f4 fix: further improvements to CPE generation for apk packages (#1623)
* fix: consider upstream logic during apk cpe gen
* fix: correct apk CPE for go
* fix: correct apk CPE for ruby
* fix: correct apk CPE for bazel
* fix: correct apk CPE for clang
* fix: correct apk CPE for openjdk
* fix: correct apk CPE for glibc
* fix: correct apk CPE for gli
* fix: correct apk CPE for bas
* fix: correct apk CPE for alsa-lib
* fix: correct apk CPE for alsa
* fix: determine apk cpe vendor from known URLs
* fix: add more url prefix->vendor mappings for apk
* refactor: allow reuse of vendor by url prefix logic
* feat: extract username as vendor candidate from github/gitlab

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-27 13:16:04 -05:00
dependabot[bot] d23b4d4cbd chore(deps): bump github.com/stretchr/testify from 1.8.1 to 1.8.2 (#1625)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-27 13:14:20 -05:00
dependabot[bot] f3acff81f3 chore(deps): bump actions/checkout from 2 to 3 (#1626)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-27 13:14:03 -05:00
Nils Hanke fa0a9fe8f9 feat: set cosign attest predicate type based on Syft output type (#1598)
Signed-off-by: Nils Hanke <nils.hanke@outlook.de>
2023-02-24 15:08:40 -05:00
dependabot[bot] 284bae9d5f chore(deps): bump github.com/spf13/afero from 1.9.3 to 1.9.4 (#1609)
Bumps [github.com/spf13/afero](https://github.com/spf13/afero) from 1.9.3 to 1.9.4.
- [Release notes](https://github.com/spf13/afero/releases)
- [Commits](https://github.com/spf13/afero/compare/v1.9.3...v1.9.4)

---
updated-dependencies:
- dependency-name: github.com/spf13/afero
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-24 15:07:52 -05:00
Weston Steimel 3ee1af0dc6 fix: correct apk purls for other distros (#1620)
The apk purl spec allows for vendor-specific namespace.  I noticed
in the embedded SBOMs from wolfi that the purls are of the form
`pkg:apk/wolfi/curl@7.83.0-r0?arch=x86`, but the current logic in
syft actually prevents purl generation entirely if the distro isn't
alpine, so this corrects that behaviour.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-24 15:07:07 -05:00
Weston Steimel 0c5f03235e refactor: move apk upstream logic to apk metadata (#1619)
* refactor: move apk upstream logic to apk metadata

Export the logic for parsing upstream APK package names
so it can be accessed from apk metadata objects directly.

This also tightens the upstream regex pattern as several
edge cases were being missed.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: ensure correct handling for apk packages beginning with digits

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: upstream generation for ruby

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

---------

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-24 15:59:19 +00:00
Keith Zantow 5e8aa4da5e fix: decoding null apk metadata pullDependencies (#1614)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-02-23 19:55:49 +00:00
Benji Visser abfec62219 feat: haproxy binary matcher (#1591)
Signed-off-by: Benji Visser <benji@093b.org>
2023-02-23 14:39:08 -05:00
Weston Steimel 0c05855131 fix: determine upstream for apk version streams (#1610)
Determines better upstream package name for version-stream apk packages:

Examples:

- postgresql-13 -> postgresql
- postgresql15 -> postgresql
- go-1.19 -> go
- perl100.55 -> perl

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-23 17:32:34 +00:00
Weston Steimel 1150772d06 fix: improve CPE generation for curl APK (#1608)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-23 17:32:12 +00:00
Alex Goodman 669fee84d5 Revert "add workaround for macos github actions cache issue (#1584)" (#1605)
This reverts commit 0076b19893.

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-22 15:03:12 -05:00
anchore-actions-token-generator[bot]andwagoodman aa151da5fe Update Stereoscope to fab1c9638abc2c21cd53dca1f205f37d71148ee0 (#1604)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: wagoodman <wagoodman@users.noreply.github.com>
2023-02-22 19:08:35 +00:00
Keith Zantow 4bf677d555 chore: fix cataloger_test (#1603)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-02-22 17:30:41 +00:00
Keith Zantow f5e20521e0 fix: merging of binary packages (#1583) 2023-02-22 12:03:15 -05:00
Keith Zantow 8f6a317fef fix: issue when matching format versions (#1585)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-02-22 10:32:05 -05:00
anchore-actions-token-generator[bot] d339ffdcb5 chore: update syft bootstrap tools to latest versions. (#1593) 2023-02-22 10:16:14 -05:00
Benji Visser e84ffc6003 feat: add perl binary classifier (#1592)
Signed-off-by: Benji Visser <benji@093b.org>
2023-02-22 09:58:05 -05:00
anchore-actions-token-generator[bot] bb52a25c8a Update Stereoscope to 529924d6d5aa6c708cceffc651883b6e1e27f5df (#1602)
Signed-off-by: GitHub <noreply@github.com>
2023-02-22 08:49:04 +00:00
Marc-Etienne Vargenau 4b7b709473 Update SPDX license list to 3.20 (#1600)
Signed-off-by: Marc-Etienne Vargenau <marc-etienne.vargenau@nokia.com>
2023-02-21 15:12:28 -05:00
Keith Zantow 2ef2eed382 chore: update SPDX license list (#1599) 2023-02-21 17:31:24 +00:00
Alex Goodman f6716092af fix cataloger selection to be more specific (#1582)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-17 15:35:54 +00:00
Alex Goodman 0076b19893 add workaround for macos github actions cache issue (#1584)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-17 10:29:33 -05:00
101 changed files with 243323 additions and 6172 deletions
+1 -1
View File
@@ -30,7 +30,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v3
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
+1 -1
View File
@@ -156,7 +156,7 @@ jobs:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_TOOLBOX_WEBHOOK_URL }}
if: ${{ success() }}
- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v3
with:
name: artifacts
path: dist/**/*
+7
View File
@@ -54,6 +54,13 @@ jobs:
path: syft/pkg/cataloger/golang/test-fixtures/archs/binaries
key: ${{ runner.os }}-unit-go-binaries-cache-${{ hashFiles( 'syft/pkg/cataloger/golang/test-fixtures/archs/binaries.fingerprint' ) }}
- name: Restore binary cataloger test-fixture cache
id: unit-binary-cataloger-cache
uses: actions/cache@v3
with:
path: syft/pkg/cataloger/binary/test-fixtures/classifiers/dynamic
key: ${{ runner.os }}-unit-binary-cataloger-cache-${{ hashFiles( 'syft/pkg/cataloger/binary/test-fixtures/cache.fingerprint' ) }}
- name: Run unit tests
run: make unit
+14 -4
View File
@@ -4,17 +4,17 @@ TEMP_DIR := ./.tmp
# Command templates #################################
LINT_CMD := $(TEMP_DIR)/golangci-lint run --tests=false
GOIMPORTS_CMD := $(TEMP_DIR)/gosimports -local github.com/anchore
RELEASE_CMD := $(TEMP_DIR)/goreleaser release --rm-dist
RELEASE_CMD := $(TEMP_DIR)/goreleaser release --clean
SNAPSHOT_CMD := $(RELEASE_CMD) --skip-publish --skip-sign --snapshot
CHRONICLE_CMD = $(TEMP_DIR)/chronicle
GLOW_CMD = $(TEMP_DIR)/glow
# Tool versions #################################
GOLANGCILINT_VERSION := v1.51.1
GOSIMPORTS_VERSION := v0.3.5
GOLANGCILINT_VERSION := v1.51.2
GOSIMPORTS_VERSION := v0.3.8
BOUNCER_VERSION := v0.4.0
CHRONICLE_VERSION := v0.6.0
GORELEASER_VERSION := v1.15.2
GORELEASER_VERSION := v1.16.1
YAJSV_VERSION := v1.4.1
COSIGN_VERSION := v1.13.1
QUILL_VERSION := v0.2.0
@@ -189,6 +189,10 @@ fingerprints:
cd test/integration/test-fixtures && \
make cache.fingerprint
# for BINARY test fixtures
cd syft/pkg/cataloger/binary/test-fixtures && \
make cache.fingerprint
# for JAVA BUILD test fixtures
cd syft/pkg/cataloger/java/test-fixtures/java-builds && \
make packages.fingerprint
@@ -214,6 +218,7 @@ fixtures:
$(call title,Generating test fixtures)
cd syft/pkg/cataloger/java/test-fixtures/java-builds && make
cd syft/pkg/cataloger/rpm/test-fixtures && make
cd syft/pkg/cataloger/binary/test-fixtures && make
.PHONY: show-test-image-cache
show-test-image-cache: ## Show all docker and image tar cache
@@ -247,6 +252,11 @@ install-test-ci-mac: $(SNAPSHOT_DIR)
cd test/install && \
make ci-test-mac
.PHONY: generate-compare-file
generate-compare-file:
$(call title,Generating compare test file)
go run ./cmd/syft $(COMPARE_TEST_IMAGE) -o json > $(COMPARE_DIR)/test-fixtures/acceptance-centos-8.2.2004.json
# note: we cannot clean the snapshot directory since the pipeline builds the snapshot separately
.PHONY: compare-mac
compare-mac: $(TEMP_DIR) $(SNAPSHOT_DIR) ## Run compare tests on build snapshot binaries and packages (Mac)
+18 -3
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"os"
"os/exec"
"strings"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
@@ -130,7 +131,21 @@ func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-
return
}
args := []string{"attest", si.UserInput, "--predicate", f.Name()}
// Select Cosign predicate type based on defined output type
// As orientation, check: https://github.com/sigstore/cosign/blob/main/pkg/cosign/attestation/attestation.go
var predicateType string
switch strings.ToLower(o) {
case "cyclonedx-json":
predicateType = "cyclonedx"
case "spdx-tag-value":
predicateType = "spdx"
case "spdx-json":
predicateType = "spdxjson"
default:
predicateType = "custom"
}
args := []string{"attest", si.UserInput, "--predicate", f.Name(), "--type", predicateType}
if app.Attest.Key != "" {
args = append(args, "--key", app.Attest.Key)
}
@@ -152,7 +167,7 @@ func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-
}
defer w.Close()
b := &busWriter{r: r, w: w, mon: &progress.Manual{N: -1}}
b := &busWriter{r: r, w: w, mon: progress.NewManual(-1)}
execCmd.Stdout = b
execCmd.Stderr = b
defer b.mon.SetCompleted()
@@ -160,7 +175,7 @@ func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-
// attest the SBOM
err = execCmd.Run()
if err != nil {
b.mon.Err = err
b.mon.SetError(err)
errs <- fmt.Errorf("unable to attest SBOM: %w", err)
return
}
+1 -4
View File
@@ -46,14 +46,11 @@ func generateCatalogPackagesTask(app *config.Application) (Task, error) {
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
packageCatalog, relationships, theDistro, err := syft.CatalogPackages(src, app.ToCatalogerConfig())
if err != nil {
return nil, err
}
results.PackageCatalog = packageCatalog
results.LinuxDistribution = theDistro
return relationships, nil
return relationships, err
}
return task, nil
+10 -10
View File
@@ -32,19 +32,19 @@ require (
github.com/sergi/go-diff v1.3.1
github.com/sirupsen/logrus v1.9.0
github.com/spdx/tools-golang v0.5.0-rc1
github.com/spf13/afero v1.9.3
github.com/spf13/afero v1.9.5
github.com/spf13/cobra v1.6.1
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.15.0
github.com/stretchr/testify v1.8.1
github.com/stretchr/testify v1.8.2
github.com/vifraa/gopom v0.2.1
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/mod v0.8.0
golang.org/x/net v0.7.0
golang.org/x/term v0.5.0
golang.org/x/mod v0.9.0
golang.org/x/net v0.8.0
golang.org/x/term v0.6.0
gopkg.in/yaml.v2 v2.4.0
)
@@ -52,7 +52,7 @@ require (
github.com/CycloneDX/cyclonedx-go v0.7.1-0.20221222100750-41a1ac565cce
github.com/Masterminds/sprig/v3 v3.2.3
github.com/anchore/go-logger v0.0.0-20220728155337-03b66a5207d8
github.com/anchore/stereoscope v0.0.0-20230216143338-4b5ebf8c7f4b
github.com/anchore/stereoscope v0.0.0-20230301191755-abfb374a1122
github.com/docker/docker v23.0.1+incompatible
github.com/google/go-containerregistry v0.13.0
github.com/invopop/jsonschema v0.7.0
@@ -126,9 +126,9 @@ require (
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.5.0 // indirect
golang.org/x/text v0.7.0 // indirect
golang.org/x/tools v0.2.0 // indirect
golang.org/x/sys v0.6.0 // indirect
golang.org/x/text v0.8.0 // indirect
golang.org/x/tools v0.6.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/genproto v0.0.0-20221227171554-f9683d7f8bef // indirect
google.golang.org/grpc v1.52.0 // indirect
+22 -20
View File
@@ -90,8 +90,8 @@ github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZV
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501 h1:AV7qjwMcM4r8wFhJq3jLRztew3ywIyPTRapl2T1s9o8=
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501/go.mod h1:Blo6OgJNiYF41ufcgHKkbCKF2MDOMlrqhXv/ij6ocR4=
github.com/anchore/stereoscope v0.0.0-20230216143338-4b5ebf8c7f4b h1:vMEAfz91QLjJq2W8JPxpIC4dG4OeynTY4MisHnZ19F0=
github.com/anchore/stereoscope v0.0.0-20230216143338-4b5ebf8c7f4b/go.mod h1:6oSG43mzahqiktzXZDctqi1o66fwU2wDk3xki0KlnbA=
github.com/anchore/stereoscope v0.0.0-20230301191755-abfb374a1122 h1:Oe2PE8zNbJH4nGZoCIC/VZBgpr62BInLnUqIMZICUOk=
github.com/anchore/stereoscope v0.0.0-20230301191755-abfb374a1122/go.mod h1:IihP/SUVHP94PBwIP2bepOB/c0MVadcII7lxo13Ijzs=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
github.com/andybalholm/brotli v1.0.4 h1:V7DdXeJtZscaqfNuAdSRuRFzuiKlHSC/Zh3zl9qY3JY=
@@ -519,8 +519,8 @@ github.com/spdx/tools-golang v0.5.0-rc1 h1:ooCSe48QatlidqEFd+nSI308tyeNTR6NJvauU
github.com/spdx/tools-golang v0.5.0-rc1/go.mod h1:LI6onw172PdO57Ob/hgnLDD4Y2PMnroeNT3wO/2WJJI=
github.com/spf13/afero v1.3.3/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4=
github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I=
github.com/spf13/afero v1.9.3 h1:41FoI0fD7OR7mGcKE/aOiLkGreyf8ifIOQmJANWogMk=
github.com/spf13/afero v1.9.3/go.mod h1:iUV7ddyEEZPO5gA3zD4fJt6iStLlL+Lg4m2cihcDf8Y=
github.com/spf13/afero v1.9.5 h1:stMpOSZFs//0Lv29HduCmli3GUfpFoF3Y1Q/aXj/wVM=
github.com/spf13/afero v1.9.5/go.mod h1:UBogFpq8E9Hx+xc5CNTTEpTnuHVmXDwZcZcE1eb/UhQ=
github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cast v1.4.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cast v1.5.0 h1:rj3WzYc11XZaIZMPKmwP96zkFEnnAmV8s6XbB2aY32w=
@@ -550,8 +550,9 @@ github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/subosito/gotenv v1.4.2 h1:X1TuBLAMDFbaTAChgCBLu3DU3UPyELpnF2jjJ2cz/S8=
github.com/subosito/gotenv v1.4.2/go.mod h1:ayKnFf/c6rvx/2iiLrJUk1e6plDbT3edrFNGqEflhK0=
@@ -575,8 +576,8 @@ github.com/vifraa/gopom v0.2.1 h1:MYVMAMyiGzXPPy10EwojzKIL670kl5Zbae+o3fFvQEM=
github.com/vifraa/gopom v0.2.1/go.mod h1:oPa1dcrGrtlO37WPDBm5SqHAT+wTgF8An1Q71Z6Vv4o=
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5 h1:phTLPgMRDYTizrBSKsNSOa2zthoC2KsJsaY/8sg3rD8=
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5/go.mod h1:JPirS5jde/CF5qIjcK4WX+eQmKXdPc6vcZkJ/P0hfPw=
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240 h1:r6BlIP7CVZtMlxUQhT40h1IE1TzEgKVqwmsVGuscvdk=
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240/go.mod h1:jLXFoL31zFaHKAAyZUh+sxiTDFe1L1ZHrcK2T1itVKA=
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5 h1:lwgTsTy18nYqASnH58qyfRW/ldj7Gt2zzBvgYPzdA4s=
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5/go.mod h1:jLXFoL31zFaHKAAyZUh+sxiTDFe1L1ZHrcK2T1itVKA=
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb h1:Yz6VVOcLuWLAHYlJzTw7JKnWxdV/WXpug2X0quEzRnY=
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb/go.mod h1:nDi3BAC5nEbVbg+WSJDHLbjHv0ZToq8nMPA97XMxF3E=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
@@ -626,7 +627,7 @@ golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWP
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.3.0 h1:a06MkbcxBrEFc0w0QIZWXrH/9cCX6KJyWbBOIwAn+7A=
golang.org/x/crypto v0.3.0/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
@@ -669,8 +670,8 @@ golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.5.0/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0 h1:LUYupSeNrTNCGzR/hVBk2NHZO4hXcVaW1k4Qx7rjPx8=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.9.0 h1:KENHtAZL2y3NLMYZeHY9DW8HW8V+kQyJsY/V9JlKvCs=
golang.org/x/mod v0.9.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -714,10 +715,11 @@ golang.org/x/net v0.0.0-20210410081132-afb366fc7cd1/go.mod h1:9tjilg8BloeKEkVJvy
golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210505024714-0287a6fb4125/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0 h1:Zrh2ngAOFYneWTAIAPethzeaQLuHwhuBkuV6ZiRnUaQ=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -823,15 +825,15 @@ golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0 h1:MVltZSvRTcU2ljQOhs94SXPftV6DCNnZViHeQps87pQ=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
golang.org/x/term v0.5.0 h1:n2a8QNdAb0sZNpU9R1ALUXBbY+w51fCQDN+7EdxNBsY=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.6.0 h1:clScbb1cHjoCkyRbWwBEUZ5H/tIFu5TAXIqaZD0Gcjw=
golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -842,8 +844,8 @@ golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.8.0 h1:57P1ETyNKtuIjB4SRd15iJxuhj8Gc416Y78H3qgMh68=
golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -907,8 +909,8 @@ golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.2.0 h1:G6AHpWxTMGY1KyEYoAQ5WTtIekUUvDNjan3ugu60JvE=
golang.org/x/tools v0.2.0/go.mod h1:y4OqIKeOV/fWJetJ8bXPU1sEVniLMIyDAZWeHdV+NTA=
golang.org/x/tools v0.6.0 h1:BOw41kyTf3PuCW1pVQf8+Cyg8pMlkYB1oo9iJ6D/lKM=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "6.2.0"
JSONSchemaVersion = "7.0.1"
)
@@ -60,6 +60,7 @@ func Test_processSPDXLicense(t *testing.T) {
"0bsd": "0BSD",
"aal": "AAL",
"abstyles": "Abstyles",
"adacore-doc": "AdaCore-doc",
"adobe-2006": "Adobe-2006",
"adobe-2006.0": "Adobe-2006",
"adobe-2006.0.0": "Adobe-2006",
@@ -145,6 +146,7 @@ func Test_processSPDXLicense(t *testing.T) {
"bahyph": "Bahyph",
"barr": "Barr",
"beerware": "Beerware",
"bitstream-charter": "Bitstream-Charter",
"bitstream-vera": "Bitstream-Vera",
"bittorrent-1": "BitTorrent-1.0",
"bittorrent-1.0": "BitTorrent-1.0",
@@ -156,6 +158,9 @@ func Test_processSPDXLicense(t *testing.T) {
"blueoak-1.0": "BlueOak-1.0.0",
"blueoak-1.0.0": "BlueOak-1.0.0",
"borceux": "Borceux",
"brian-gladman-3-clause": "Brian-Gladman-3-Clause",
"brian-gladman-3.0-clause": "Brian-Gladman-3-Clause",
"brian-gladman-3.0.0-clause": "Brian-Gladman-3-Clause",
"bsd-1-clause": "BSD-1-Clause",
"bsd-1.0-clause": "BSD-1-Clause",
"bsd-1.0.0-clause": "BSD-1-Clause",
@@ -213,6 +218,14 @@ func Test_processSPDXLicense(t *testing.T) {
"bsd-4.0.0-clause": "BSD-4-Clause",
"bsd-4.0.0-clause-shortened": "BSD-4-Clause-Shortened",
"bsd-4.0.0-clause-uc": "BSD-4-Clause-UC",
"bsd-4.3.0reno": "BSD-4.3RENO",
"bsd-4.3.0tahoe": "BSD-4.3TAHOE",
"bsd-4.3reno": "BSD-4.3RENO",
"bsd-4.3tahoe": "BSD-4.3TAHOE",
"bsd-4reno": "BSD-4.3RENO",
"bsd-4tahoe": "BSD-4.3TAHOE",
"bsd-advertising-acknowledgement": "BSD-Advertising-Acknowledgement",
"bsd-attribution-hpnd-disclaimer": "BSD-Attribution-HPND-disclaimer",
"bsd-protection": "BSD-Protection",
"bsd-source-code": "BSD-Source-Code",
"bsl-1": "BSL-1.0",
@@ -311,12 +324,15 @@ func Test_processSPDXLicense(t *testing.T) {
"cc-by-nc-sa-1.0": "CC-BY-NC-SA-1.0",
"cc-by-nc-sa-1.0.0": "CC-BY-NC-SA-1.0",
"cc-by-nc-sa-2": "CC-BY-NC-SA-2.0",
"cc-by-nc-sa-2-de": "CC-BY-NC-SA-2.0-DE",
"cc-by-nc-sa-2-fr": "CC-BY-NC-SA-2.0-FR",
"cc-by-nc-sa-2-uk": "CC-BY-NC-SA-2.0-UK",
"cc-by-nc-sa-2.0": "CC-BY-NC-SA-2.0",
"cc-by-nc-sa-2.0-de": "CC-BY-NC-SA-2.0-DE",
"cc-by-nc-sa-2.0-fr": "CC-BY-NC-SA-2.0-FR",
"cc-by-nc-sa-2.0-uk": "CC-BY-NC-SA-2.0-UK",
"cc-by-nc-sa-2.0.0": "CC-BY-NC-SA-2.0",
"cc-by-nc-sa-2.0.0-de": "CC-BY-NC-SA-2.0-DE",
"cc-by-nc-sa-2.0.0-fr": "CC-BY-NC-SA-2.0-FR",
"cc-by-nc-sa-2.0.0-uk": "CC-BY-NC-SA-2.0-UK",
"cc-by-nc-sa-2.5": "CC-BY-NC-SA-2.5",
@@ -423,8 +439,11 @@ func Test_processSPDXLicense(t *testing.T) {
"cern-ohl-w-2": "CERN-OHL-W-2.0",
"cern-ohl-w-2.0": "CERN-OHL-W-2.0",
"cern-ohl-w-2.0.0": "CERN-OHL-W-2.0",
"cfitsio": "CFITSIO",
"checkmk": "checkmk",
"clartistic": "ClArtistic",
"clips": "Clips",
"cmu-mach": "CMU-Mach",
"cnri-jython": "CNRI-Jython",
"cnri-python": "CNRI-Python",
"cnri-python-gpl-compatible": "CNRI-Python-GPL-Compatible",
@@ -440,6 +459,7 @@ func Test_processSPDXLicense(t *testing.T) {
"copyleft-next-0.3": "copyleft-next-0.3.0",
"copyleft-next-0.3.0": "copyleft-next-0.3.0",
"copyleft-next-0.3.1": "copyleft-next-0.3.1",
"cornell-lossless-jpeg": "Cornell-Lossless-JPEG",
"cpal-1": "CPAL-1.0",
"cpal-1.0": "CPAL-1.0",
"cpal-1.0.0": "CPAL-1.0",
@@ -637,6 +657,7 @@ func Test_processSPDXLicense(t *testing.T) {
"gpl-3.0.0-or-later": "GPL-3.0-or-later",
"gpl-3.0.0-with-autoconf-exception": "GPL-3.0-with-autoconf-exception",
"gpl-3.0.0-with-gcc-exception": "GPL-3.0-with-GCC-exception",
"graphics-gems": "Graphics-Gems",
"gsoap-1.3.0b": "gSOAP-1.3b",
"gsoap-1.3b": "gSOAP-1.3b",
"gsoap-1b": "gSOAP-1.3b",
@@ -644,12 +665,20 @@ func Test_processSPDXLicense(t *testing.T) {
"hippocratic-2": "Hippocratic-2.1",
"hippocratic-2.1": "Hippocratic-2.1",
"hippocratic-2.1.0": "Hippocratic-2.1",
"hp-1986": "HP-1986",
"hp-1986.0": "HP-1986",
"hp-1986.0.0": "HP-1986",
"hpnd": "HPND",
"hpnd-export-us": "HPND-export-US",
"hpnd-markus-kuhn": "HPND-Markus-Kuhn",
"hpnd-sell-variant": "HPND-sell-variant",
"hpnd-sell-variant-mit-disclaimer": "HPND-sell-variant-MIT-disclaimer",
"htmltidy": "HTMLTIDY",
"ibm-pibs": "IBM-pibs",
"icu": "ICU",
"iec-code-components-eula": "IEC-Code-Components-EULA",
"ijg": "IJG",
"ijg-short": "IJG-short",
"imagemagick": "ImageMagick",
"imatix": "iMatix",
"imlib2": "Imlib2",
@@ -668,8 +697,10 @@ func Test_processSPDXLicense(t *testing.T) {
"jasper-2": "JasPer-2.0",
"jasper-2.0": "JasPer-2.0",
"jasper-2.0.0": "JasPer-2.0",
"jpl-image": "JPL-image",
"jpnic": "JPNIC",
"json": "JSON",
"kazlib": "Kazlib",
"knuth-ctan": "Knuth-CTAN",
"lal-1": "LAL-1.2",
"lal-1.2": "LAL-1.2",
@@ -731,6 +762,7 @@ func Test_processSPDXLicense(t *testing.T) {
"liliq-rplus-1.1.0": "LiLiQ-Rplus-1.1",
"linux-man-pages-copyleft": "Linux-man-pages-copyleft",
"linux-openib": "Linux-OpenIB",
"loop": "LOOP",
"lpl-1": "LPL-1.0",
"lpl-1.0": "LPL-1.0",
"lpl-1.0.0": "LPL-1.0",
@@ -756,6 +788,7 @@ func Test_processSPDXLicense(t *testing.T) {
"lzma-sdk-9.22": "LZMA-SDK-9.22",
"lzma-sdk-9.22.0": "LZMA-SDK-9.22",
"makeindex": "MakeIndex",
"martin-birgmeier": "Martin-Birgmeier",
"minpack": "Minpack",
"miros": "MirOS",
"mit": "MIT",
@@ -766,6 +799,7 @@ func Test_processSPDXLicense(t *testing.T) {
"mit-feh": "MIT-feh",
"mit-modern-variant": "MIT-Modern-Variant",
"mit-open-group": "MIT-open-group",
"mit-wu": "MIT-Wu",
"mitnfa": "MITNFA",
"motosoto": "Motosoto",
"mpi-permissive": "mpi-permissive",
@@ -852,6 +886,7 @@ func Test_processSPDXLicense(t *testing.T) {
"odc-by-1": "ODC-By-1.0",
"odc-by-1.0": "ODC-By-1.0",
"odc-by-1.0.0": "ODC-By-1.0",
"offis": "OFFIS",
"ofl-1": "OFL-1.0",
"ofl-1-no-rfn": "OFL-1.0-no-RFN",
"ofl-1-rfn": "OFL-1.0-RFN",
@@ -918,6 +953,9 @@ func Test_processSPDXLicense(t *testing.T) {
"oldap-2.8": "OLDAP-2.8",
"oldap-2.8.0": "OLDAP-2.8",
"oml": "OML",
"openpbs-2": "OpenPBS-2.3",
"openpbs-2.3": "OpenPBS-2.3",
"openpbs-2.3.0": "OpenPBS-2.3",
"openssl": "OpenSSL",
"opl-1": "OPL-1.0",
"opl-1.0": "OPL-1.0",
@@ -974,8 +1012,11 @@ func Test_processSPDXLicense(t *testing.T) {
"python-2.0.1": "Python-2.0.1",
"qhull": "Qhull",
"qpl-1": "QPL-1.0",
"qpl-1-inria-2004": "QPL-1.0-INRIA-2004",
"qpl-1.0": "QPL-1.0",
"qpl-1.0-inria-2004": "QPL-1.0-INRIA-2004",
"qpl-1.0.0": "QPL-1.0",
"qpl-1.0.0-inria-2004": "QPL-1.0-INRIA-2004",
"rdisc": "Rdisc",
"rhecos-1": "RHeCos-1.1",
"rhecos-1.1": "RHeCos-1.1",
@@ -1022,6 +1063,7 @@ func Test_processSPDXLicense(t *testing.T) {
"smlnj": "SMLNJ",
"smppl": "SMPPL",
"snia": "SNIA",
"snprintf": "snprintf",
"spencer-86": "Spencer-86",
"spencer-86.0": "Spencer-86",
"spencer-86.0.0": "Spencer-86",
@@ -1043,7 +1085,9 @@ func Test_processSPDXLicense(t *testing.T) {
"sugarcrm-1": "SugarCRM-1.1.3",
"sugarcrm-1.1": "SugarCRM-1.1.3",
"sugarcrm-1.1.3": "SugarCRM-1.1.3",
"sunpro": "SunPro",
"swl": "SWL",
"symlinks": "Symlinks",
"tapr-ohl-1": "TAPR-OHL-1.0",
"tapr-ohl-1.0": "TAPR-OHL-1.0",
"tapr-ohl-1.0.0": "TAPR-OHL-1.0",
@@ -1054,12 +1098,18 @@ func Test_processSPDXLicense(t *testing.T) {
"torque-1.1": "TORQUE-1.1",
"torque-1.1.0": "TORQUE-1.1",
"tosl": "TOSL",
"tpdl": "TPDL",
"tpl-1": "TPL-1.0",
"tpl-1.0": "TPL-1.0",
"tpl-1.0.0": "TPL-1.0",
"ttwl": "TTWL",
"tu-berlin-1": "TU-Berlin-1.0",
"tu-berlin-1.0": "TU-Berlin-1.0",
"tu-berlin-1.0.0": "TU-Berlin-1.0",
"tu-berlin-2": "TU-Berlin-2.0",
"tu-berlin-2.0": "TU-Berlin-2.0",
"tu-berlin-2.0.0": "TU-Berlin-2.0",
"ucar": "UCAR",
"ucl-1": "UCL-1.0",
"ucl-1.0": "UCL-1.0",
"ucl-1.0.0": "UCL-1.0",
@@ -1086,6 +1136,7 @@ func Test_processSPDXLicense(t *testing.T) {
"w3c-20150513": "W3C-20150513",
"w3c-20150513.0": "W3C-20150513",
"w3c-20150513.0.0": "W3C-20150513",
"w3m": "w3m",
"watcom-1": "Watcom-1.0",
"watcom-1.0": "Watcom-1.0",
"watcom-1.0.0": "Watcom-1.0",
@@ -1099,6 +1150,7 @@ func Test_processSPDXLicense(t *testing.T) {
"xfree86-1.1": "XFree86-1.1",
"xfree86-1.1.0": "XFree86-1.1",
"xinetd": "xinetd",
"xlock": "xlock",
"xnet": "Xnet",
"xpp": "xpp",
"xskat": "XSkat",
File diff suppressed because it is too large Load Diff
+54 -2
View File
@@ -1,14 +1,15 @@
// Code generated by go generate; DO NOT EDIT.
// This file was generated by robots at 2022-12-02 12:51:33.814385 -0500 EST m=+1.688731869
// This file was generated by robots at 2023-02-21 18:58:18.115328967 +0100 CET m=+0.216357753
// using data from https://spdx.org/licenses/licenses.json
package spdxlicense
const Version = "3.19"
const Version = "3.20"
var licenseIDs = map[string]string{
"0bsd": "0BSD",
"aal": "AAL",
"abstyles": "Abstyles",
"adacore-doc": "AdaCore-doc",
"adobe-2006": "Adobe-2006",
"adobe-2006.0": "Adobe-2006",
"adobe-2006.0.0": "Adobe-2006",
@@ -94,6 +95,7 @@ var licenseIDs = map[string]string{
"bahyph": "Bahyph",
"barr": "Barr",
"beerware": "Beerware",
"bitstream-charter": "Bitstream-Charter",
"bitstream-vera": "Bitstream-Vera",
"bittorrent-1": "BitTorrent-1.0",
"bittorrent-1.0": "BitTorrent-1.0",
@@ -105,6 +107,9 @@ var licenseIDs = map[string]string{
"blueoak-1.0": "BlueOak-1.0.0",
"blueoak-1.0.0": "BlueOak-1.0.0",
"borceux": "Borceux",
"brian-gladman-3-clause": "Brian-Gladman-3-Clause",
"brian-gladman-3.0-clause": "Brian-Gladman-3-Clause",
"brian-gladman-3.0.0-clause": "Brian-Gladman-3-Clause",
"bsd-1-clause": "BSD-1-Clause",
"bsd-1.0-clause": "BSD-1-Clause",
"bsd-1.0.0-clause": "BSD-1-Clause",
@@ -162,6 +167,14 @@ var licenseIDs = map[string]string{
"bsd-4.0.0-clause": "BSD-4-Clause",
"bsd-4.0.0-clause-shortened": "BSD-4-Clause-Shortened",
"bsd-4.0.0-clause-uc": "BSD-4-Clause-UC",
"bsd-4.3.0reno": "BSD-4.3RENO",
"bsd-4.3.0tahoe": "BSD-4.3TAHOE",
"bsd-4.3reno": "BSD-4.3RENO",
"bsd-4.3tahoe": "BSD-4.3TAHOE",
"bsd-4reno": "BSD-4.3RENO",
"bsd-4tahoe": "BSD-4.3TAHOE",
"bsd-advertising-acknowledgement": "BSD-Advertising-Acknowledgement",
"bsd-attribution-hpnd-disclaimer": "BSD-Attribution-HPND-disclaimer",
"bsd-protection": "BSD-Protection",
"bsd-source-code": "BSD-Source-Code",
"bsl-1": "BSL-1.0",
@@ -260,12 +273,15 @@ var licenseIDs = map[string]string{
"cc-by-nc-sa-1.0": "CC-BY-NC-SA-1.0",
"cc-by-nc-sa-1.0.0": "CC-BY-NC-SA-1.0",
"cc-by-nc-sa-2": "CC-BY-NC-SA-2.0",
"cc-by-nc-sa-2-de": "CC-BY-NC-SA-2.0-DE",
"cc-by-nc-sa-2-fr": "CC-BY-NC-SA-2.0-FR",
"cc-by-nc-sa-2-uk": "CC-BY-NC-SA-2.0-UK",
"cc-by-nc-sa-2.0": "CC-BY-NC-SA-2.0",
"cc-by-nc-sa-2.0-de": "CC-BY-NC-SA-2.0-DE",
"cc-by-nc-sa-2.0-fr": "CC-BY-NC-SA-2.0-FR",
"cc-by-nc-sa-2.0-uk": "CC-BY-NC-SA-2.0-UK",
"cc-by-nc-sa-2.0.0": "CC-BY-NC-SA-2.0",
"cc-by-nc-sa-2.0.0-de": "CC-BY-NC-SA-2.0-DE",
"cc-by-nc-sa-2.0.0-fr": "CC-BY-NC-SA-2.0-FR",
"cc-by-nc-sa-2.0.0-uk": "CC-BY-NC-SA-2.0-UK",
"cc-by-nc-sa-2.5": "CC-BY-NC-SA-2.5",
@@ -372,8 +388,11 @@ var licenseIDs = map[string]string{
"cern-ohl-w-2": "CERN-OHL-W-2.0",
"cern-ohl-w-2.0": "CERN-OHL-W-2.0",
"cern-ohl-w-2.0.0": "CERN-OHL-W-2.0",
"cfitsio": "CFITSIO",
"checkmk": "checkmk",
"clartistic": "ClArtistic",
"clips": "Clips",
"cmu-mach": "CMU-Mach",
"cnri-jython": "CNRI-Jython",
"cnri-python": "CNRI-Python",
"cnri-python-gpl-compatible": "CNRI-Python-GPL-Compatible",
@@ -389,6 +408,7 @@ var licenseIDs = map[string]string{
"copyleft-next-0.3": "copyleft-next-0.3.0",
"copyleft-next-0.3.0": "copyleft-next-0.3.0",
"copyleft-next-0.3.1": "copyleft-next-0.3.1",
"cornell-lossless-jpeg": "Cornell-Lossless-JPEG",
"cpal-1": "CPAL-1.0",
"cpal-1.0": "CPAL-1.0",
"cpal-1.0.0": "CPAL-1.0",
@@ -586,6 +606,7 @@ var licenseIDs = map[string]string{
"gpl-3.0.0-or-later": "GPL-3.0-or-later",
"gpl-3.0.0-with-autoconf-exception": "GPL-3.0-with-autoconf-exception",
"gpl-3.0.0-with-gcc-exception": "GPL-3.0-with-GCC-exception",
"graphics-gems": "Graphics-Gems",
"gsoap-1.3.0b": "gSOAP-1.3b",
"gsoap-1.3b": "gSOAP-1.3b",
"gsoap-1b": "gSOAP-1.3b",
@@ -593,12 +614,20 @@ var licenseIDs = map[string]string{
"hippocratic-2": "Hippocratic-2.1",
"hippocratic-2.1": "Hippocratic-2.1",
"hippocratic-2.1.0": "Hippocratic-2.1",
"hp-1986": "HP-1986",
"hp-1986.0": "HP-1986",
"hp-1986.0.0": "HP-1986",
"hpnd": "HPND",
"hpnd-export-us": "HPND-export-US",
"hpnd-markus-kuhn": "HPND-Markus-Kuhn",
"hpnd-sell-variant": "HPND-sell-variant",
"hpnd-sell-variant-mit-disclaimer": "HPND-sell-variant-MIT-disclaimer",
"htmltidy": "HTMLTIDY",
"ibm-pibs": "IBM-pibs",
"icu": "ICU",
"iec-code-components-eula": "IEC-Code-Components-EULA",
"ijg": "IJG",
"ijg-short": "IJG-short",
"imagemagick": "ImageMagick",
"imatix": "iMatix",
"imlib2": "Imlib2",
@@ -617,8 +646,10 @@ var licenseIDs = map[string]string{
"jasper-2": "JasPer-2.0",
"jasper-2.0": "JasPer-2.0",
"jasper-2.0.0": "JasPer-2.0",
"jpl-image": "JPL-image",
"jpnic": "JPNIC",
"json": "JSON",
"kazlib": "Kazlib",
"knuth-ctan": "Knuth-CTAN",
"lal-1": "LAL-1.2",
"lal-1.2": "LAL-1.2",
@@ -680,6 +711,7 @@ var licenseIDs = map[string]string{
"liliq-rplus-1.1.0": "LiLiQ-Rplus-1.1",
"linux-man-pages-copyleft": "Linux-man-pages-copyleft",
"linux-openib": "Linux-OpenIB",
"loop": "LOOP",
"lpl-1": "LPL-1.0",
"lpl-1.0": "LPL-1.0",
"lpl-1.0.0": "LPL-1.0",
@@ -705,6 +737,7 @@ var licenseIDs = map[string]string{
"lzma-sdk-9.22": "LZMA-SDK-9.22",
"lzma-sdk-9.22.0": "LZMA-SDK-9.22",
"makeindex": "MakeIndex",
"martin-birgmeier": "Martin-Birgmeier",
"minpack": "Minpack",
"miros": "MirOS",
"mit": "MIT",
@@ -715,6 +748,7 @@ var licenseIDs = map[string]string{
"mit-feh": "MIT-feh",
"mit-modern-variant": "MIT-Modern-Variant",
"mit-open-group": "MIT-open-group",
"mit-wu": "MIT-Wu",
"mitnfa": "MITNFA",
"motosoto": "Motosoto",
"mpi-permissive": "mpi-permissive",
@@ -801,6 +835,7 @@ var licenseIDs = map[string]string{
"odc-by-1": "ODC-By-1.0",
"odc-by-1.0": "ODC-By-1.0",
"odc-by-1.0.0": "ODC-By-1.0",
"offis": "OFFIS",
"ofl-1": "OFL-1.0",
"ofl-1-no-rfn": "OFL-1.0-no-RFN",
"ofl-1-rfn": "OFL-1.0-RFN",
@@ -867,6 +902,9 @@ var licenseIDs = map[string]string{
"oldap-2.8": "OLDAP-2.8",
"oldap-2.8.0": "OLDAP-2.8",
"oml": "OML",
"openpbs-2": "OpenPBS-2.3",
"openpbs-2.3": "OpenPBS-2.3",
"openpbs-2.3.0": "OpenPBS-2.3",
"openssl": "OpenSSL",
"opl-1": "OPL-1.0",
"opl-1.0": "OPL-1.0",
@@ -923,8 +961,11 @@ var licenseIDs = map[string]string{
"python-2.0.1": "Python-2.0.1",
"qhull": "Qhull",
"qpl-1": "QPL-1.0",
"qpl-1-inria-2004": "QPL-1.0-INRIA-2004",
"qpl-1.0": "QPL-1.0",
"qpl-1.0-inria-2004": "QPL-1.0-INRIA-2004",
"qpl-1.0.0": "QPL-1.0",
"qpl-1.0.0-inria-2004": "QPL-1.0-INRIA-2004",
"rdisc": "Rdisc",
"rhecos-1": "RHeCos-1.1",
"rhecos-1.1": "RHeCos-1.1",
@@ -971,6 +1012,7 @@ var licenseIDs = map[string]string{
"smlnj": "SMLNJ",
"smppl": "SMPPL",
"snia": "SNIA",
"snprintf": "snprintf",
"spencer-86": "Spencer-86",
"spencer-86.0": "Spencer-86",
"spencer-86.0.0": "Spencer-86",
@@ -992,7 +1034,9 @@ var licenseIDs = map[string]string{
"sugarcrm-1": "SugarCRM-1.1.3",
"sugarcrm-1.1": "SugarCRM-1.1.3",
"sugarcrm-1.1.3": "SugarCRM-1.1.3",
"sunpro": "SunPro",
"swl": "SWL",
"symlinks": "Symlinks",
"tapr-ohl-1": "TAPR-OHL-1.0",
"tapr-ohl-1.0": "TAPR-OHL-1.0",
"tapr-ohl-1.0.0": "TAPR-OHL-1.0",
@@ -1003,12 +1047,18 @@ var licenseIDs = map[string]string{
"torque-1.1": "TORQUE-1.1",
"torque-1.1.0": "TORQUE-1.1",
"tosl": "TOSL",
"tpdl": "TPDL",
"tpl-1": "TPL-1.0",
"tpl-1.0": "TPL-1.0",
"tpl-1.0.0": "TPL-1.0",
"ttwl": "TTWL",
"tu-berlin-1": "TU-Berlin-1.0",
"tu-berlin-1.0": "TU-Berlin-1.0",
"tu-berlin-1.0.0": "TU-Berlin-1.0",
"tu-berlin-2": "TU-Berlin-2.0",
"tu-berlin-2.0": "TU-Berlin-2.0",
"tu-berlin-2.0.0": "TU-Berlin-2.0",
"ucar": "UCAR",
"ucl-1": "UCL-1.0",
"ucl-1.0": "UCL-1.0",
"ucl-1.0.0": "UCL-1.0",
@@ -1035,6 +1085,7 @@ var licenseIDs = map[string]string{
"w3c-20150513": "W3C-20150513",
"w3c-20150513.0": "W3C-20150513",
"w3c-20150513.0.0": "W3C-20150513",
"w3m": "w3m",
"watcom-1": "Watcom-1.0",
"watcom-1.0": "Watcom-1.0",
"watcom-1.0.0": "Watcom-1.0",
@@ -1048,6 +1099,7 @@ var licenseIDs = map[string]string{
"xfree86-1.1": "XFree86-1.1",
"xfree86-1.1.0": "XFree86-1.1",
"xinetd": "xinetd",
"xlock": "xlock",
"xnet": "Xnet",
"xpp": "xpp",
"xskat": "XSkat",
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+3 -5
View File
@@ -51,10 +51,10 @@ func (i *DigestsCataloger) Catalog(resolver source.FileResolver) (map[source.Coo
if err != nil {
return nil, err
}
prog.N++
prog.Increment()
results[location.Coordinates] = result
}
log.Debugf("file digests cataloger processed %d files", prog.N)
log.Debugf("file digests cataloger processed %d files", prog.Current())
prog.SetCompleted()
return results, nil
}
@@ -123,9 +123,7 @@ func CleanDigestAlgorithmName(name string) string {
func digestsCatalogingProgress(locations int64) (*progress.Stage, *progress.Manual) {
stage := &progress.Stage{}
prog := &progress.Manual{
Total: locations,
}
prog := progress.NewManual(locations)
bus.Publish(partybus.Event{
Type: event.FileDigestsCatalogerStarted,
+3 -5
View File
@@ -32,18 +32,16 @@ func (i *MetadataCataloger) Catalog(resolver source.FileResolver) (map[source.Co
}
results[location.Coordinates] = metadata
prog.N++
prog.Increment()
}
log.Debugf("file metadata cataloger processed %d files", prog.N)
log.Debugf("file metadata cataloger processed %d files", prog.Current())
prog.SetCompleted()
return results, nil
}
func metadataCatalogingProgress(locations int64) (*progress.Stage, *progress.Manual) {
stage := &progress.Stage{}
prog := &progress.Manual{
Total: locations,
}
prog := progress.NewManual(locations)
bus.Publish(partybus.Event{
Type: event.FileMetadataCatalogerStarted,
+4 -6
View File
@@ -55,12 +55,12 @@ func (i *SecretsCataloger) Catalog(resolver source.FileResolver) (map[source.Coo
return nil, err
}
if len(result) > 0 {
secretsDiscovered.N += int64(len(result))
secretsDiscovered.Add(int64(len(result)))
results[location.Coordinates] = result
}
prog.N++
prog.Increment()
}
log.Debugf("secrets cataloger discovered %d secrets", secretsDiscovered.N)
log.Debugf("secrets cataloger discovered %d secrets", secretsDiscovered.Current())
prog.SetCompleted()
return results, nil
}
@@ -139,9 +139,7 @@ type SecretsMonitor struct {
func secretsCatalogingProgress(locations int64) (*progress.Stage, *progress.Manual, *progress.Manual) {
stage := &progress.Stage{}
secretsDiscovered := &progress.Manual{}
prog := &progress.Manual{
Total: locations,
}
prog := progress.NewManual(locations)
bus.Publish(partybus.Event{
Type: event.SecretsCatalogerStarted,
@@ -24,6 +24,12 @@ func License(p pkg.Package) string {
// take all licenses and assume an AND expression; for information about license expressions see https://spdx.github.io/spdx-spec/appendix-IV-SPDX-license-expressions/
parsedLicenses := parseLicenses(p.Licenses)
for i, v := range parsedLicenses {
if strings.HasPrefix(v, spdxlicense.LicenseRefPrefix) {
parsedLicenses[i] = SanitizeElementID(v)
}
}
if len(parsedLicenses) == 0 {
return NOASSERTION
}
@@ -65,6 +65,17 @@ func Test_License(t *testing.T) {
},
expected: "GPL-2.0-only",
},
{
name: "includes valid LicenseRef-",
input: pkg.Package{
Licenses: []string{
"one thing first",
"two things/#$^second",
"MIT",
},
},
expected: "LicenseRef-one-thing-first AND LicenseRef-two-things----second AND MIT",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
@@ -514,8 +514,8 @@ func toFileTypes(metadata *source.FileMetadata) (ty []string) {
func toOtherLicenses(catalog *pkg.Catalog) []*spdx.OtherLicense {
licenses := map[string]bool{}
for _, pkg := range catalog.Sorted() {
for _, license := range parseLicenses(pkg.Licenses) {
for _, p := range catalog.Sorted() {
for _, license := range parseLicenses(p.Licenses) {
if strings.HasPrefix(license, spdxlicense.LicenseRefPrefix) {
licenses[license] = true
}
@@ -526,7 +526,7 @@ func toOtherLicenses(catalog *pkg.Catalog) []*spdx.OtherLicense {
// separate the actual ID from the prefix
name := strings.TrimPrefix(license, spdxlicense.LicenseRefPrefix)
result = append(result, &spdx.OtherLicense{
LicenseIdentifier: license,
LicenseIdentifier: SanitizeElementID(license),
LicenseName: name,
ExtractedText: NONE, // we probably should have some extracted text here, but this is good enough for now
})
@@ -369,7 +369,7 @@ func Test_fileIDsForPackage(t *testing.T) {
}
func Test_H1Digest(t *testing.T) {
sbom := sbom.SBOM{}
s := sbom.SBOM{}
tests := []struct {
name string
pkg pkg.Package
@@ -416,7 +416,7 @@ func Test_H1Digest(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
catalog := pkg.NewCatalog(test.pkg)
pkgs := toPackages(catalog, sbom)
pkgs := toPackages(catalog, s)
require.Len(t, pkgs, 1)
for _, p := range pkgs {
if test.expectedDigest == "" {
@@ -431,3 +431,67 @@ func Test_H1Digest(t *testing.T) {
})
}
}
func Test_OtherLicenses(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expected []*spdx.OtherLicense
}{
{
name: "no licenseRef",
pkg: pkg.Package{
Licenses: []string{
"MIT",
},
},
expected: nil,
},
{
name: "single licenseRef",
pkg: pkg.Package{
Licenses: []string{
"un known",
},
},
expected: []*spdx.OtherLicense{
{
LicenseIdentifier: "LicenseRef-un-known",
LicenseName: "un known",
ExtractedText: NONE,
},
},
},
{
name: "multiple licenseRef",
pkg: pkg.Package{
Licenses: []string{
"un known",
"not known %s",
"MIT",
},
},
expected: []*spdx.OtherLicense{
{
LicenseIdentifier: "LicenseRef-un-known",
LicenseName: "un known",
ExtractedText: NONE,
},
{
LicenseIdentifier: "LicenseRef-not-known--s",
LicenseName: "not known %s",
ExtractedText: NONE,
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
catalog := pkg.NewCatalog(test.pkg)
otherLicenses := toOtherLicenses(catalog)
require.Len(t, otherLicenses, len(test.expected))
require.Equal(t, test.expected, otherLicenses)
})
}
}
+1 -6
View File
@@ -85,14 +85,9 @@ func versionMatches(version string, match string) bool {
return true
}
dots := strings.Count(match, ".")
if dots == 0 {
match += ".*"
}
match = strings.ReplaceAll(match, ".", "\\.")
match = strings.ReplaceAll(match, "*", ".*")
match = strings.ReplaceAll(match, "\\..*", "(\\..*)*")
match = fmt.Sprintf("^%s$", match)
match = fmt.Sprintf("^%s(\\..*)*$", match)
matcher, err := regexp.Compile(match)
if err != nil {
return false
+49
View File
@@ -227,12 +227,61 @@ func Test_versionMatches(t *testing.T) {
match: "3.1",
matches: true,
},
{
name: "wildcard-version matches minor",
version: "7.1.3",
match: "7.*",
matches: true,
},
{
name: "wildcard-version matches patch",
version: "7.4.8",
match: "7.4.*",
matches: true,
},
{
name: "sub-version matches major",
version: "7.19.11",
match: "7",
matches: true,
},
{
name: "sub-version matches minor",
version: "7.55.2",
match: "7.55",
matches: true,
},
{
name: "sub-version matches patch",
version: "7.32.6",
match: "7.32.6",
matches: true,
},
// negative tests
{
name: "different number does not match",
version: "3",
match: "4",
matches: false,
},
{
name: "sub-version doesn't match major",
version: "7.2.5",
match: "8.2.5",
matches: false,
},
{
name: "sub-version doesn't match minor",
version: "7.2.9",
match: "7.1",
matches: false,
},
{
name: "sub-version doesn't match patch",
version: "7.32.6",
match: "7.32.5",
matches: false,
},
}
for _, test := range tests {
+1
View File
@@ -213,5 +213,6 @@ func TestEncodeFullJSONDocument(t *testing.T) {
s,
*updateJson,
true,
schemaVersionRedactor,
)
}
+2 -1
View File
@@ -1,6 +1,7 @@
package syftjson
import (
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/sbom"
)
@@ -8,7 +9,7 @@ const ID sbom.FormatID = "syft-json"
func Format() sbom.Format {
return sbom.NewFormat(
"6",
internal.JSONSchemaVersion,
encoder,
decoder,
validator,
@@ -24,6 +24,7 @@ type LinuxRelease struct {
BugReportURL string `json:"bugReportURL,omitempty"`
PrivacyPolicyURL string `json:"privacyPolicyURL,omitempty"`
CPEName string `json:"cpeName,omitempty"`
SupportEnd string `json:"supportEnd,omitempty"`
}
func (s *IDLikes) UnmarshalJSON(data []byte) error {
+1
View File
@@ -62,6 +62,7 @@ func toLinuxReleaser(d *linux.Release) model.LinuxRelease {
BugReportURL: d.BugReportURL,
PrivacyPolicyURL: d.PrivacyPolicyURL,
CPEName: d.CPEName,
SupportEnd: d.SupportEnd,
}
}
@@ -1,7 +1,6 @@
package syftjson
import (
"strings"
"testing"
"github.com/scylladb/go-set/strset"
@@ -9,16 +8,10 @@ import (
"github.com/stretchr/testify/require"
"github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/formats/syftjson/model"
"github.com/anchore/syft/syft/source"
)
func Test_SyftJsonID_Compatibility(t *testing.T) {
jsonMajorVersion := strings.Split(internal.JSONSchemaVersion, ".")[0]
assert.Equal(t, jsonMajorVersion, string(Format().Version()))
}
func Test_toSourceModel(t *testing.T) {
allSchemes := strset.New()
for _, s := range source.AllSchemes {
+1
View File
@@ -53,6 +53,7 @@ func toSyftLinuxRelease(d model.LinuxRelease) *linux.Release {
BugReportURL: d.BugReportURL,
PrivacyPolicyURL: d.PrivacyPolicyURL,
CPEName: d.CPEName,
SupportEnd: d.SupportEnd,
}
}
+1 -4
View File
@@ -70,13 +70,10 @@ func CatalogPackages(src *source.Source, cfg cataloger.Config) (*pkg.Catalog, []
}
catalog, relationships, err := cataloger.Catalog(resolver, release, cfg.Parallelism, catalogers...)
if err != nil {
return nil, nil, nil, err
}
relationships = append(relationships, newSourceRelationshipsFromCatalog(src, catalog)...)
return catalog, relationships, release, nil
return catalog, relationships, release, err
}
func newSourceRelationshipsFromCatalog(src *source.Source, c *pkg.Catalog) []artifact.Relationship {
+1
View File
@@ -125,6 +125,7 @@ func parseOsRelease(contents string) (*Release, error) {
BugReportURL: values["BUG_REPORT_URL"],
PrivacyPolicyURL: values["PRIVACY_POLICY_URL"],
CPEName: values["CPE_NAME"],
SupportEnd: values["SUPPORT_END"],
}
// don't allow for empty contents to result in a Release object being created
+7 -6
View File
@@ -89,19 +89,20 @@ func TestIdentifyRelease(t *testing.T) {
{
fixture: "test-fixtures/os/fedora",
release: &Release{
PrettyName: "Fedora 31 (Container Image)",
Name: "Fedora",
PrettyName: "Fedora Linux 36 (Container Image)",
Name: "Fedora Linux",
ID: "fedora",
IDLike: nil,
Version: "31 (Container Image)",
VersionID: "31",
Version: "36 (Container Image)",
VersionID: "36",
Variant: "Container Image",
VariantID: "container",
HomeURL: "https://fedoraproject.org/",
SupportURL: "https://fedoraproject.org/wiki/Communicating_and_getting_help",
SupportURL: "https://ask.fedoraproject.org/",
BugReportURL: "https://bugzilla.redhat.com/",
PrivacyPolicyURL: "https://fedoraproject.org/wiki/Legal:PrivacyPolicy",
CPEName: "cpe:/o:fedoraproject:fedora:31",
CPEName: "cpe:/o:fedoraproject:fedora:36",
SupportEnd: "2023-05-16",
},
},
{
+1
View File
@@ -19,6 +19,7 @@ type Release struct {
BugReportURL string
PrivacyPolicyURL string
CPEName string // A CPE name for the operating system, in URI binding syntax
SupportEnd string // The date at which support for this version of the OS ends.
}
func (r *Release) String() string {
@@ -1,21 +1,22 @@
NAME=Fedora
VERSION="31 (Container Image)"
NAME="Fedora Linux"
VERSION="36 (Container Image)"
ID=fedora
VERSION_ID=31
VERSION_ID=36
VERSION_CODENAME=""
PLATFORM_ID="platform:f31"
PRETTY_NAME="Fedora 31 (Container Image)"
ANSI_COLOR="0;34"
PLATFORM_ID="platform:f36"
PRETTY_NAME="Fedora Linux 36 (Container Image)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:31"
CPE_NAME="cpe:/o:fedoraproject:fedora:36"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f31/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f36/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=31
REDHAT_BUGZILLA_PRODUCT_VERSION=36
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=31
REDHAT_SUPPORT_PRODUCT_VERSION=36
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
SUPPORT_END=2023-05-16
VARIANT="Container Image"
VARIANT_ID=container
+2
View File
@@ -87,6 +87,8 @@ func (a *spaceDelimitedStringSlice) UnmarshalJSON(data []byte) error {
}
*a = s
return nil
case nil:
return nil
default:
return fmt.Errorf("invalid type for string array: %T", obj)
}
+9
View File
@@ -98,6 +98,15 @@ func TestApkMetadata_UnmarshalJSON(t *testing.T) {
Files: []ApkFileRecord{{Path: "/usr"}},
},
},
{
name: "null pullDependencies",
input: `{
"pullDependencies": null
}`,
want: ApkMetadata{
Dependencies: nil,
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
+8 -3
View File
@@ -1,7 +1,12 @@
package pkg
import "github.com/anchore/syft/syft/source"
type BinaryMetadata struct {
Classifier string `mapstructure:"Classifier" json:"classifier"`
RealPath string `mapstructure:"RealPath" json:"realPath"`
VirtualPath string `mapstructure:"VirtualPath" json:"virtualPath"`
Matches []ClassifierMatch `mapstructure:"Matches" json:"matches"`
}
type ClassifierMatch struct {
Classifier string `mapstructure:"Classifier" json:"classifier"`
Location source.Location `mapstructure:"Location" json:"location"`
}
@@ -24,6 +24,7 @@ func TestCataloger_Globs(t *testing.T) {
pkgtest.NewCatalogTester().
FromDirectory(t, test.fixture).
ExpectsResolverContentQueries(test.expected).
IgnoreUnfulfilledPathResponses("etc/apk/repositories").
TestCataloger(t, NewApkdbCataloger())
})
}
+4 -23
View File
@@ -9,10 +9,6 @@ import (
"github.com/anchore/syft/syft/source"
)
var (
prefixes = []string{"py-", "py2-", "py3-", "ruby-"}
)
func newPackage(d pkg.ApkMetadata, release *linux.Release, locations ...source.Location) pkg.Package {
p := pkg.Package{
Name: d.Package,
@@ -30,24 +26,9 @@ func newPackage(d pkg.ApkMetadata, release *linux.Release, locations ...source.L
return p
}
func generateUpstream(m pkg.ApkMetadata) string {
if m.OriginPackage != "" && m.OriginPackage != m.Package {
return m.OriginPackage
}
for _, p := range prefixes {
if strings.HasPrefix(m.Package, p) {
return strings.TrimPrefix(m.Package, p)
}
}
return m.Package
}
// packageURL returns the PURL for the specific Alpine package (see https://github.com/package-url/purl-spec)
func packageURL(m pkg.ApkMetadata, distro *linux.Release) string {
if distro == nil || distro.ID != "alpine" {
// note: there is no namespace variation (like with debian ID_LIKE for ubuntu ID, for example)
if distro == nil {
return ""
}
@@ -55,13 +36,13 @@ func packageURL(m pkg.ApkMetadata, distro *linux.Release) string {
pkg.PURLQualifierArch: m.Architecture,
}
if m.OriginPackage != "" {
qualifiers[pkg.PURLQualifierUpstream] = generateUpstream(m)
if m.OriginPackage != m.Package {
qualifiers[pkg.PURLQualifierUpstream] = m.OriginPackage
}
return packageurl.NewPackageURL(
packageurl.TypeAlpine,
"alpine",
strings.ToLower(distro.ID),
m.Package,
m.Version,
pkg.PURLQualifiers(
+9 -24
View File
@@ -20,7 +20,7 @@ func Test_PackageURL(t *testing.T) {
expected string
}{
{
name: "bad distro",
name: "non-alpine distro",
metadata: pkg.ApkMetadata{
Package: "p",
Version: "v",
@@ -30,7 +30,7 @@ func Test_PackageURL(t *testing.T) {
ID: "something else",
VersionID: "3.4.6",
},
expected: "",
expected: "pkg:apk/something%20else/p@v?arch=a&distro=something%20else-3.4.6",
},
{
name: "gocase",
@@ -97,32 +97,17 @@ func Test_PackageURL(t *testing.T) {
expected: "pkg:apk/alpine/p@v?arch=a&upstream=origin&distro=alpine-3.4.6",
},
{
name: "upstream python package information as qualifier",
name: "wolfi distro",
metadata: pkg.ApkMetadata{
Package: "py3-potatoes",
Version: "v",
Architecture: "a",
OriginPackage: "py3-potatoes",
Package: "p",
Version: "v",
Architecture: "a",
},
distro: linux.Release{
ID: "alpine",
VersionID: "3.4.6",
ID: "wolfi",
VersionID: "20221230",
},
expected: "pkg:apk/alpine/py3-potatoes@v?arch=a&upstream=potatoes&distro=alpine-3.4.6",
},
{
name: "python package with origin package as upstream",
metadata: pkg.ApkMetadata{
Package: "py3-non-existant",
Version: "v",
Architecture: "a",
OriginPackage: "abcdefg",
},
distro: linux.Release{
ID: "alpine",
VersionID: "3.4.6",
},
expected: "pkg:apk/alpine/py3-non-existant@v?arch=a&upstream=abcdefg&distro=alpine-3.4.6",
expected: "pkg:apk/wolfi/p@v?arch=a&distro=wolfi-20221230",
},
}
+73 -2
View File
@@ -3,7 +3,9 @@ package apkdb
import (
"bufio"
"fmt"
"io"
"path"
"regexp"
"strconv"
"strings"
@@ -20,11 +22,15 @@ import (
// integrity check
var _ generic.Parser = parseApkDB
var (
repoRegex = regexp.MustCompile(`(?m)^https://.*\.alpinelinux\.org/alpine/v([^/]+)/([a-zA-Z0-9_]+)$`)
)
// parseApkDB parses packages from a given APK installed DB file. For more
// information on specific fields, see https://wiki.alpinelinux.org/wiki/Apk_spec.
//
//nolint:funlen
func parseApkDB(_ source.FileResolver, env *generic.Environment, reader source.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
//nolint:funlen,gocognit
func parseApkDB(resolver source.FileResolver, env *generic.Environment, reader source.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
scanner := bufio.NewScanner(reader)
var apks []pkg.ApkMetadata
@@ -101,6 +107,19 @@ func parseApkDB(_ source.FileResolver, env *generic.Environment, reader source.L
if env != nil {
r = env.LinuxRelease
}
// this is somewhat ugly, but better than completely failing when we can't find the release,
// e.g. embedded deeper in the tree, like containers or chroots.
// but we now have no way of handling different repository sources. On the other hand,
// we never could before this. At least now, we can handle some.
// This should get fixed with https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10875
if r == nil {
// find the repositories file from the relative directory of the DB file
releases := findReleases(resolver, reader.Location.RealPath)
if len(releases) > 0 {
r = &releases[0]
}
}
pkgs := make([]pkg.Package, 0, len(apks))
for _, apk := range apks {
@@ -110,6 +129,58 @@ func parseApkDB(_ source.FileResolver, env *generic.Environment, reader source.L
return pkgs, discoverPackageDependencies(pkgs), nil
}
func findReleases(resolver source.FileResolver, dbPath string) []linux.Release {
if resolver == nil {
return nil
}
reposLocation := path.Clean(path.Join(path.Dir(dbPath), "../../../etc/apk/repositories"))
locations, err := resolver.FilesByPath(reposLocation)
if err != nil {
log.Tracef("unable to find APK repositories file %q: %+v", reposLocation, err)
return nil
}
if len(locations) == 0 {
return nil
}
location := locations[0]
reposReader, err := resolver.FileContentsByLocation(location)
if err != nil {
log.Tracef("unable to fetch contents for APK repositories file %q: %+v", reposLocation, err)
return nil
}
return parseReleasesFromAPKRepository(source.LocationReadCloser{
Location: location,
ReadCloser: reposReader,
})
}
func parseReleasesFromAPKRepository(reader source.LocationReadCloser) []linux.Release {
var releases []linux.Release
reposB, err := io.ReadAll(reader)
if err != nil {
log.Tracef("unable to read APK repositories file %q: %+v", reader.Location.RealPath, err)
return nil
}
parts := repoRegex.FindAllStringSubmatch(string(reposB), -1)
for _, part := range parts {
if len(part) >= 3 {
releases = append(releases, linux.Release{
Name: "Alpine Linux",
ID: "alpine",
VersionID: part[1],
})
}
}
return releases
}
func parseApkField(line string) *apkField {
parts := strings.SplitN(line, ":", 2)
if len(parts) != 2 {
@@ -1,8 +1,10 @@
package apkdb
import (
"io"
"os"
"path/filepath"
"strings"
"testing"
"github.com/google/go-cmp/cmp"
@@ -1186,3 +1188,49 @@ func Test_stripVersionSpecifier(t *testing.T) {
})
}
}
func TestParseReleasesFromAPKRepository(t *testing.T) {
tests := []struct {
repos string
want []linux.Release
desc string
}{
{
"https://foo.alpinelinux.org/alpine/v3.14/main",
[]linux.Release{
{Name: "Alpine Linux", ID: "alpine", VersionID: "3.14"},
},
"single repo",
},
{
`https://foo.alpinelinux.org/alpine/v3.14/main
https://foo.alpinelinux.org/alpine/v3.14/community`,
[]linux.Release{
{Name: "Alpine Linux", ID: "alpine", VersionID: "3.14"},
{Name: "Alpine Linux", ID: "alpine", VersionID: "3.14"},
},
"multiple repos",
},
{
``,
nil,
"empty",
},
{
`https://foo.bar.org/alpine/v3.14/main
https://foo.them.org/alpine/v3.14/community`,
nil,
"invalid repos",
},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
reposReader := io.NopCloser(strings.NewReader(tt.repos))
got := parseReleasesFromAPKRepository(source.LocationReadCloser{
Location: source.NewLocation("test"),
ReadCloser: reposReader,
})
assert.Equal(t, tt.want, got)
})
}
}
+30 -25
View File
@@ -33,49 +33,54 @@ func (c Cataloger) Catalog(resolver source.FileResolver) ([]pkg.Package, []artif
for _, cls := range defaultClassifiers {
log.WithFields("classifier", cls.Class).Trace("cataloging binaries")
pkgs, err := catalog(resolver, cls)
newPkgs, err := catalog(resolver, cls)
if err != nil {
log.WithFields("error", err, "classifier", cls.Class).Warn("unable to catalog binary package: %w", err)
continue
}
packages = append(packages, pkgs...)
newPackages:
for i := range newPkgs {
newPkg := &newPkgs[i]
for j := range packages {
p := &packages[j]
// consolidate identical packages found in different locations or by different classifiers
if packagesMatch(p, newPkg) {
mergePackages(p, newPkg)
continue newPackages
}
}
packages = append(packages, *newPkg)
}
}
return packages, relationships, nil
}
func catalog(resolver source.FileResolver, cls classifier) ([]pkg.Package, error) {
var pkgs []pkg.Package
// mergePackages merges information from the extra package into the target package
func mergePackages(target *pkg.Package, extra *pkg.Package) {
// add the locations
target.Locations.Add(extra.Locations.ToSlice()...)
// update the metadata to indicate which classifiers were used
meta, _ := target.Metadata.(pkg.BinaryMetadata)
if m, ok := extra.Metadata.(pkg.BinaryMetadata); ok {
meta.Matches = append(meta.Matches, m.Matches...)
}
target.Metadata = meta
}
func catalog(resolver source.FileResolver, cls classifier) (packages []pkg.Package, err error) {
locations, err := resolver.FilesByGlob(cls.FileGlob)
if err != nil {
return nil, err
}
for _, location := range locations {
reader, err := resolver.FileContentsByLocation(location)
pkgs, err := cls.EvidenceMatcher(resolver, cls, location)
if err != nil {
return nil, err
}
locationReader := source.NewLocationReadCloser(location, reader)
newPkgs, err := cls.EvidenceMatcher(cls, locationReader)
if err != nil {
return nil, err
}
newPackages:
for i := range newPkgs {
newPkg := &newPkgs[i]
for j := range pkgs {
p := &pkgs[j]
// consolidate identical packages found in different locations,
// but continue to track each location
if packagesMatch(p, newPkg) {
p.Locations.Add(newPkg.Locations.ToSlice()...)
continue newPackages
}
}
pkgs = append(pkgs, *newPkg)
}
packages = append(packages, pkgs...)
}
return pkgs, nil
return packages, nil
}
// packagesMatch returns true if the binary packages "match" based on basic criteria
+413 -158
View File
@@ -2,9 +2,13 @@ package binary
import (
"errors"
"fmt"
"io"
"strings"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -27,10 +31,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "15beta4",
Type: "binary",
PURL: "pkg:generic/postgresql@15beta4",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
Locations: locations("postgres"),
Metadata: metadata("postgresql-binary"),
},
},
{
@@ -41,10 +43,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "15.1",
Type: "binary",
PURL: "pkg:generic/postgresql@15.1",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
Locations: locations("postgres"),
Metadata: metadata("postgresql-binary"),
},
},
{
@@ -55,10 +55,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "9.6.24",
Type: "binary",
PURL: "pkg:generic/postgresql@9.6.24",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
Locations: locations("postgres"),
Metadata: metadata("postgresql-binary"),
},
},
{
@@ -69,10 +67,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "9.5alpha1",
Type: "binary",
PURL: "pkg:generic/postgresql@9.5alpha1",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
Locations: locations("postgres"),
Metadata: metadata("postgresql-binary"),
},
},
{
@@ -83,10 +79,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "2.9.6",
Type: "binary",
PURL: "pkg:generic/traefik@2.9.6",
Locations: singleLocation("traefik"),
Metadata: pkg.BinaryMetadata{
Classifier: "traefik-binary",
},
Locations: locations("traefik"),
Metadata: metadata("traefik-binary"),
},
},
{
@@ -97,10 +91,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "1.7.34",
Type: "binary",
PURL: "pkg:generic/traefik@1.7.34",
Locations: singleLocation("traefik"),
Metadata: pkg.BinaryMetadata{
Classifier: "traefik-binary",
},
Locations: locations("traefik"),
Metadata: metadata("traefik-binary"),
},
},
{
@@ -111,10 +103,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "1.6.18",
Type: "binary",
PURL: "pkg:generic/memcached@1.6.18",
Locations: singleLocation("memcached"),
Metadata: pkg.BinaryMetadata{
Classifier: "memcached-binary",
},
Locations: locations("memcached"),
Metadata: metadata("memcached-binary"),
},
},
{
@@ -125,10 +115,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "2.4.54",
Type: "binary",
PURL: "pkg:generic/httpd@2.4.54",
Locations: singleLocation("httpd"),
Metadata: pkg.BinaryMetadata{
Classifier: "httpd-binary",
},
Locations: locations("httpd"),
Metadata: metadata("httpd-binary"),
},
},
{
@@ -139,10 +127,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "8.2.1",
Type: "binary",
PURL: "pkg:generic/php-cli@8.2.1",
Locations: singleLocation("php"),
Metadata: pkg.BinaryMetadata{
Classifier: "php-cli-binary",
},
Locations: locations("php"),
Metadata: metadata("php-cli-binary"),
},
},
{
@@ -153,10 +139,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "8.2.1",
Type: "binary",
PURL: "pkg:generic/php-fpm@8.2.1",
Locations: singleLocation("php-fpm"),
Metadata: pkg.BinaryMetadata{
Classifier: "php-fpm-binary",
},
Locations: locations("php-fpm"),
Metadata: metadata("php-fpm-binary"),
},
},
{
@@ -167,10 +151,80 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "8.2.1",
Type: "binary",
PURL: "pkg:generic/php@8.2.1",
Locations: singleLocation("libphp.so"),
Metadata: pkg.BinaryMetadata{
Classifier: "php-apache-binary",
},
Locations: locations("libphp.so"),
Metadata: metadata("php-apache-binary"),
},
},
{
name: "positive-perl-5.12.5",
fixtureDir: "test-fixtures/classifiers/positive/perl-5.12.5",
expected: pkg.Package{
Name: "perl",
Version: "5.12.5",
Type: "binary",
PURL: "pkg:generic/perl@5.12.5",
Locations: locations("perl"),
Metadata: metadata("perl-binary"),
},
},
{
name: "positive-perl-5.20.0",
fixtureDir: "test-fixtures/classifiers/positive/perl-5.20.0",
expected: pkg.Package{
Name: "perl",
Version: "5.20.0",
Type: "binary",
PURL: "pkg:generic/perl@5.20.0",
Locations: locations("perl"),
Metadata: metadata("perl-binary"),
},
},
{
name: "positive-perl-5.37.8",
fixtureDir: "test-fixtures/classifiers/positive/perl-5.37.8",
expected: pkg.Package{
Name: "perl",
Version: "5.37.8",
Type: "binary",
PURL: "pkg:generic/perl@5.37.8",
Locations: locations("perl"),
Metadata: metadata("perl-binary"),
},
},
{
name: "positive-haproxy-1.5.14",
fixtureDir: "test-fixtures/classifiers/positive/haproxy-1.5.14",
expected: pkg.Package{
Name: "haproxy",
Version: "1.5.14",
Type: "binary",
PURL: "pkg:generic/haproxy@1.5.14",
Locations: locations("haproxy"),
Metadata: metadata("haproxy-binary"),
},
},
{
name: "positive-haproxy-1.8.22",
fixtureDir: "test-fixtures/classifiers/positive/haproxy-1.8.22",
expected: pkg.Package{
Name: "haproxy",
Version: "1.8.22",
Type: "binary",
PURL: "pkg:generic/haproxy@1.8.22",
Locations: locations("haproxy"),
Metadata: metadata("haproxy-binary"),
},
},
{
name: "positive-haproxy-2.7.3",
fixtureDir: "test-fixtures/classifiers/positive/haproxy-2.7.3",
expected: pkg.Package{
Name: "haproxy",
Version: "2.7.3",
Type: "binary",
PURL: "pkg:generic/haproxy@2.7.3",
Locations: locations("haproxy"),
Metadata: metadata("haproxy-binary"),
},
},
{
@@ -181,10 +235,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "2.8.23",
Type: "binary",
PURL: "pkg:generic/redis@2.8.23",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
Locations: locations("redis-server"),
Metadata: metadata("redis-binary"),
},
},
{
@@ -195,10 +247,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "4.0.11",
Type: "binary",
PURL: "pkg:generic/redis@4.0.11",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
Locations: locations("redis-server"),
Metadata: metadata("redis-binary"),
},
},
{
@@ -209,10 +259,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "5.0.0",
Type: "binary",
PURL: "pkg:generic/redis@5.0.0",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
Locations: locations("redis-server"),
Metadata: metadata("redis-binary"),
},
},
{
@@ -223,10 +271,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "6.0.16",
Type: "binary",
PURL: "pkg:generic/redis@6.0.16",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
Locations: locations("redis-server"),
Metadata: metadata("redis-binary"),
},
},
{
@@ -237,101 +283,168 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "7.0.0",
Type: "binary",
PURL: "pkg:generic/redis@7.0.0",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
Locations: locations("redis-server"),
Metadata: metadata("redis-binary"),
},
},
{
name: "positive-libpython3.7.so",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/python-binary-lib-3.7",
expected: pkg.Package{
Name: "python",
Version: "3.7.4a-vZ9",
PURL: "pkg:generic/python@3.7.4a-vZ9",
Locations: singleLocation("libpython3.7.so"),
Version: "3.7.4",
PURL: "pkg:generic/python@3.7.4",
Locations: locations("libpython3.7.so"),
Metadata: metadata("python-binary-lib"),
},
},
{
name: "positive-python-3.11.2-from-shared-lib",
fixtureDir: "test-fixtures/classifiers/dynamic/python-binary-shared-lib-3.11",
expected: pkg.Package{
Name: "python",
Version: "3.11.2",
PURL: "pkg:generic/python@3.11.2",
Locations: locations("python3", "libpython3.11.so.1.0"),
Metadata: pkg.BinaryMetadata{
Classifier: "python-binary-lib",
Matches: []pkg.ClassifierMatch{
match("python-binary", "python3"),
match("python-binary", "libpython3.11.so.1.0"),
match("python-binary-lib", "libpython3.11.so.1.0"),
},
},
},
},
{
name: "positive-python-3.9-from-shared-redhat-lib",
fixtureDir: "test-fixtures/classifiers/dynamic/python-binary-shared-lib-redhat-3.9",
expected: pkg.Package{
Name: "python",
Version: "3.9.13",
PURL: "pkg:generic/python@3.9.13",
Locations: locations("python3.9", "libpython3.9.so.1.0"),
Metadata: pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match("python-binary", "python3.9"),
match("python-binary", "libpython3.9.so.1.0"),
match("python-binary-lib", "libpython3.9.so.1.0"),
},
},
},
},
{
name: "positive-python-binary-with-version-3.9",
fixtureDir: "test-fixtures/classifiers/dynamic/python-binary-with-version-3.9",
expected: pkg.Package{
Name: "python",
Version: "3.9.2",
PURL: "pkg:generic/python@3.9.2",
Locations: locations("python3.9"),
Metadata: pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match("python-binary", "python3.9"),
},
},
},
},
{
name: "positive-python-binary-3.4-alpine",
fixtureDir: "test-fixtures/classifiers/dynamic/python-binary-3.4-alpine",
expected: pkg.Package{
Name: "python",
Version: "3.4.10",
PURL: "pkg:generic/python@3.4.10",
Locations: locations("python3.4", "libpython3.4m.so.1.0"),
Metadata: pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match("python-binary", "python3.4"),
match("python-binary", "libpython3.4m.so.1.0"),
match("python-binary-lib", "libpython3.4m.so.1.0"),
},
},
},
},
{
name: "positive-python-3.5-with-incorrect-match",
fixtureDir: "test-fixtures/classifiers/positive/python-3.5-with-incorrect-match",
expected: pkg.Package{
Name: "python",
Version: "3.5.3",
PURL: "pkg:generic/python@3.5.3",
Locations: locations("python3.5"),
Metadata: metadata("python-binary"),
},
},
{
name: "positive-python3.6",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/python-binary-3.6",
expected: pkg.Package{
Name: "python",
Version: "3.6.3a-vZ9",
PURL: "pkg:generic/python@3.6.3a-vZ9",
Locations: singleLocation("python3.6"),
Metadata: pkg.BinaryMetadata{
Classifier: "python-binary",
},
Version: "3.6.3",
PURL: "pkg:generic/python@3.6.3",
Locations: locations("python3.6"),
Metadata: metadata("python-binary"),
},
},
{
name: "positive-patchlevel.h",
fixtureDir: "test-fixtures/classifiers/positive",
name: "positive-python-duplicates",
fixtureDir: "test-fixtures/classifiers/positive/python-duplicates",
expected: pkg.Package{
Name: "python",
Version: "3.9-aZ5",
PURL: "pkg:generic/python@3.9-aZ5",
Locations: singleLocation("patchlevel.h"),
Version: "3.8.16",
Type: "binary",
PURL: "pkg:generic/python@3.8.16",
Locations: locations("dir/python3.8", "python3.8", "libpython3.8.so"),
Metadata: pkg.BinaryMetadata{
Classifier: "cpython-source",
Matches: []pkg.ClassifierMatch{
match("python-binary", "dir/python3.8"),
match("python-binary", "python3.8"),
match("python-binary-lib", "libpython3.8.so"),
},
},
},
},
{
name: "positive-go",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/go-1.14",
expected: pkg.Package{
Name: "go",
Version: "1.14",
PURL: "pkg:generic/go@1.14",
Locations: singleLocation("go"),
Metadata: pkg.BinaryMetadata{
Classifier: "go-binary",
},
Locations: locations("go"),
Metadata: metadata("go-binary"),
},
},
{
name: "positive-node",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/node-19.2.1",
expected: pkg.Package{
Name: "node",
Version: "19.2.1",
PURL: "pkg:generic/node@19.2.1",
Locations: singleLocation("node"),
Metadata: pkg.BinaryMetadata{
Classifier: "nodejs-binary",
},
Locations: locations("node"),
Metadata: metadata("nodejs-binary"),
},
},
{
name: "positive-go-hint",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/go-hint-1.15",
expected: pkg.Package{
Name: "go",
Version: "1.15",
PURL: "pkg:generic/go@1.15",
Locations: singleLocation("VERSION"),
Metadata: pkg.BinaryMetadata{
Classifier: "go-binary-hint",
},
Locations: locations("VERSION"),
Metadata: metadata("go-binary-hint"),
},
},
{
name: "positive-busybox",
fixtureDir: "test-fixtures/classifiers/positive",
fixtureDir: "test-fixtures/classifiers/positive/busybox-3.33.3",
expected: pkg.Package{
Name: "busybox",
Version: "3.33.3",
Locations: singleLocation("["), // note: busybox is a link to [
Metadata: pkg.BinaryMetadata{
Classifier: "busybox-binary",
VirtualPath: "busybox",
},
Locations: locations("["), // note: busybox is a link to [
Metadata: metadata("busybox-binary", "[", "busybox"),
},
},
{
@@ -342,11 +455,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "1.8.0_352-b08",
Type: "binary",
PURL: "pkg:generic/java@1.8.0_352-b08",
Locations: singleLocation("java"),
Metadata: pkg.BinaryMetadata{
Classifier: "java-binary-openjdk",
VirtualPath: "java",
},
Locations: locations("java"),
Metadata: metadata("java-binary-openjdk", "java"),
},
},
{
@@ -357,11 +467,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "11.0.17+8-LTS",
Type: "binary",
PURL: "pkg:generic/java@11.0.17+8-LTS",
Locations: singleLocation("java"),
Metadata: pkg.BinaryMetadata{
Classifier: "java-binary-openjdk",
VirtualPath: "java",
},
Locations: locations("java"),
Metadata: metadata("java-binary-openjdk", "java"),
},
},
{
@@ -372,11 +479,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "19.0.1+10-21",
Type: "binary",
PURL: "pkg:generic/java@19.0.1+10-21",
Locations: singleLocation("java"),
Metadata: pkg.BinaryMetadata{
Classifier: "java-binary-oracle",
VirtualPath: "java",
},
Locations: locations("java"),
Metadata: metadata("java-binary-oracle", "java"),
},
},
{
@@ -387,11 +491,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "19.0.1+10-21",
Type: "binary",
PURL: "pkg:generic/java@19.0.1+10-21",
Locations: singleLocation("java"),
Metadata: pkg.BinaryMetadata{
Classifier: "java-binary-oracle",
VirtualPath: "java",
},
Locations: locations("java"),
Metadata: metadata("java-binary-oracle", "java"),
},
},
{
@@ -402,13 +503,92 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Version: "1.8.0-foreman_2022_09_22_15_30-b00",
Type: "binary",
PURL: "pkg:generic/java@1.8.0-foreman_2022_09_22_15_30-b00",
Locations: singleLocation("java"),
Locations: locations("java"),
Metadata: metadata("java-binary-ibm", "java"),
},
},
{
name: "positive-rust-1.50.0-macos",
fixtureDir: "test-fixtures/classifiers/positive/rust-1.50.0",
expected: pkg.Package{
Name: "rust",
Version: "1.50.0",
Type: "binary",
PURL: "pkg:generic/rust@1.50.0",
Locations: locations("lib/rustlib/aarch64-apple-darwin/lib/libstd-f6f9eec1635e636a.dylib"),
Metadata: metadata("rust-standard-library-macos"),
},
},
{
name: "positive-rust-1.67.1-macos",
fixtureDir: "test-fixtures/classifiers/positive/rust-1.67.1/toolchains/stable-aarch64-apple-darwin",
expected: pkg.Package{
Name: "rust",
Version: "1.67.1",
Type: "binary",
PURL: "pkg:generic/rust@1.67.1",
Locations: locations("lib/libstd-16f2b65e77054c42.dylib"),
Metadata: metadata("rust-standard-library-macos"),
},
},
{
name: "positive-rust-1.67.1-linux",
fixtureDir: "test-fixtures/classifiers/positive/rust-1.67.1/toolchains/stable-x86_64-unknown-linux-musl",
expected: pkg.Package{
Name: "rust",
Version: "1.67.1",
Type: "binary",
PURL: "pkg:generic/rust@1.67.1",
Locations: locations("lib/libstd-86aefecbddda356d.so"),
Metadata: metadata("rust-standard-library-linux"),
},
},
{
name: "positive-ruby-3.2.1",
fixtureDir: "test-fixtures/classifiers/dynamic/ruby-library-3.2.1",
expected: pkg.Package{
Name: "ruby",
Version: "3.2.1",
Type: "binary",
PURL: "pkg:generic/ruby@3.2.1",
Locations: locations("ruby", "libruby.so.3.2.1"),
Metadata: pkg.BinaryMetadata{
Classifier: "java-binary-ibm",
VirtualPath: "java",
Matches: []pkg.ClassifierMatch{
match("ruby-binary", "ruby"),
match("ruby-binary", "libruby.so.3.2.1"),
},
},
},
},
{
name: "positive-ruby-2.7.7",
fixtureDir: "test-fixtures/classifiers/dynamic/ruby-library-2.7.7",
expected: pkg.Package{
Name: "ruby",
Version: "2.7.7p221",
Type: "binary",
PURL: "pkg:generic/ruby@2.7.7p221",
Locations: locations("ruby", "libruby.so.2.7.7"),
Metadata: pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match("ruby-binary", "ruby"),
match("ruby-binary", "libruby.so.2.7.7"),
},
},
},
},
{
name: "positive-ruby-1.9.3p551",
fixtureDir: "test-fixtures/classifiers/positive/ruby-1.9.3p551",
expected: pkg.Package{
Name: "ruby",
Version: "1.9.3p551",
Type: "binary",
PURL: "pkg:generic/ruby@1.9.3p551",
Locations: locations("ruby"),
Metadata: metadata("ruby-binary"),
},
},
}
for _, test := range tests {
@@ -424,18 +604,9 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
packages, _, err := c.Catalog(resolver)
require.NoError(t, err)
ok := false
for _, p := range packages {
if test.expected.Locations.ToSlice()[0].RealPath == p.Locations.ToSlice()[0].RealPath {
ok = true
assertPackagesAreEqual(t, test.expected, p)
}
}
if !ok {
t.Fatalf("could not find test location=%q", test.expected.Locations.ToSlice()[0].RealPath)
}
require.Len(t, packages, 1)
assertPackagesAreEqual(t, test.expected, packages[0])
})
}
}
@@ -452,11 +623,8 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases_Image(t *testing.T) {
expected: pkg.Package{
Name: "busybox",
Version: "1.35.0",
Locations: singleLocation("/bin/["),
Metadata: pkg.BinaryMetadata{
Classifier: "busybox-binary",
VirtualPath: "/bin/busybox",
},
Locations: locations("/bin/["),
Metadata: metadata("busybox-binary", "/bin/[", "/bin/busybox"),
},
},
}
@@ -475,18 +643,20 @@ func Test_Cataloger_DefaultClassifiers_PositiveCases_Image(t *testing.T) {
packages, _, err := c.Catalog(resolver)
require.NoError(t, err)
ok := false
for _, p := range packages {
if test.expected.Locations.ToSlice()[0].RealPath == p.Locations.ToSlice()[0].RealPath {
ok = true
assertPackagesAreEqual(t, test.expected, p)
expectedLocations := test.expected.Locations.ToSlice()
gotLocations := p.Locations.ToSlice()
require.Len(t, gotLocations, len(expectedLocations))
for i, expectedLocation := range expectedLocations {
gotLocation := gotLocations[i]
if expectedLocation.RealPath != gotLocation.RealPath {
t.Fatalf("locations do not match; expected: %v got: %v", expectedLocations, gotLocations)
}
}
}
if !ok {
t.Fatalf("could not find test location=%q", test.expected.Locations.ToSlice()[0].RealPath)
assertPackagesAreEqual(t, test.expected, p)
}
})
}
}
@@ -505,18 +675,101 @@ func TestClassifierCataloger_DefaultClassifiers_NegativeCases(t *testing.T) {
assert.Equal(t, 0, len(actualResults))
}
func singleLocation(s string) source.LocationSet {
return source.NewLocationSet(source.NewLocation(s))
func locations(locations ...string) source.LocationSet {
var locs []source.Location
for _, s := range locations {
locs = append(locs, source.NewLocation(s))
}
return source.NewLocationSet(locs...)
}
// metadata paths are: realPath, virtualPath
func metadata(classifier string, paths ...string) pkg.BinaryMetadata {
return pkg.BinaryMetadata{
Matches: []pkg.ClassifierMatch{
match(classifier, paths...),
},
}
}
// match paths are: realPath, virtualPath
func match(classifier string, paths ...string) pkg.ClassifierMatch {
realPath := ""
if len(paths) > 0 {
realPath = paths[0]
}
virtualPath := ""
if len(paths) > 1 {
virtualPath = paths[1]
}
return pkg.ClassifierMatch{
Classifier: classifier,
Location: source.Location{
Coordinates: source.Coordinates{
RealPath: realPath,
},
VirtualPath: virtualPath,
},
}
}
func assertPackagesAreEqual(t *testing.T, expected pkg.Package, p pkg.Package) {
meta1 := expected.Metadata.(pkg.BinaryMetadata)
meta2 := p.Metadata.(pkg.BinaryMetadata)
var failMessages []string
expectedLocations := expected.Locations.ToSlice()
gotLocations := p.Locations.ToSlice()
if len(expectedLocations) != len(gotLocations) {
failMessages = append(failMessages, "locations are not equal length")
} else {
for i, expectedLocation := range expectedLocations {
gotLocation := gotLocations[i]
if expectedLocation.RealPath != gotLocation.RealPath {
failMessages = append(failMessages, fmt.Sprintf("locations do not match; expected: %v got: %v", expectedLocation.RealPath, gotLocation.RealPath))
}
}
}
m1 := expected.Metadata.(pkg.BinaryMetadata).Matches
m2 := p.Metadata.(pkg.BinaryMetadata).Matches
matches := true
if len(m1) == len(m2) {
for i, m1 := range m1 {
m2 := m2[i]
if m1.Classifier != m2.Classifier {
matches = false
break
}
if m1.Location.RealPath != "" && m1.Location.RealPath != m2.Location.RealPath {
matches = false
break
}
if m1.Location.VirtualPath != "" && m1.Location.VirtualPath != m2.Location.VirtualPath {
matches = false
break
}
}
} else {
matches = false
}
if !matches {
failMessages = append(failMessages, "classifier matches not equal")
}
if expected.Name != p.Name ||
expected.Version != p.Version ||
expected.PURL != p.PURL ||
meta1.Classifier != meta2.Classifier {
assert.Failf(t, "packages not equal", "%v != %v", expected, p)
expected.PURL != p.PURL {
failMessages = append(failMessages, "packages do not match")
}
if len(failMessages) > 0 {
assert.Failf(t, strings.Join(failMessages, "; "), "diff: %s",
cmp.Diff(expected, p,
cmp.Transformer("Locations", func(l source.LocationSet) []source.Location {
return l.ToSlice()
}),
cmpopts.IgnoreUnexported(pkg.Package{}, source.Location{}),
cmpopts.IgnoreFields(pkg.Package{}, "CPEs", "FoundBy", "MetadataType", "Type"),
))
}
}
@@ -544,7 +797,7 @@ func (p *panicyResolver) FileContentsByLocation(_ source.Location) (io.ReadClose
return nil, errors.New("not implemented")
}
func (p *panicyResolver) HasPath(s string) bool {
func (p *panicyResolver) HasPath(_ string) bool {
return true
}
@@ -575,6 +828,8 @@ func (p *panicyResolver) FileMetadataByLocation(_ source.Location) (source.FileM
return source.FileMetadata{}, errors.New("not implemented")
}
var _ source.FileResolver = (*panicyResolver)(nil)
func Test_Cataloger_ResilientToErrors(t *testing.T) {
c := NewCataloger()
+133 -16
View File
@@ -2,14 +2,19 @@ package binary
import (
"bytes"
"debug/elf"
"debug/macho"
"debug/pe"
"fmt"
"io"
"reflect"
"regexp"
"strings"
"text/template"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/unionreader"
@@ -49,16 +54,36 @@ type classifier struct {
}
// evidenceMatcher is a function called to catalog Packages that match some sort of evidence
type evidenceMatcher func(classifier classifier, reader source.LocationReadCloser) ([]pkg.Package, error)
type evidenceMatcher func(resolver source.FileResolver, classifier classifier, location source.Location) ([]pkg.Package, error)
func evidenceMatchers(matchers ...evidenceMatcher) evidenceMatcher {
return func(resolver source.FileResolver, classifier classifier, location source.Location) ([]pkg.Package, error) {
for _, matcher := range matchers {
match, err := matcher(resolver, classifier, location)
if err != nil {
return nil, err
}
if match != nil {
return match, nil
}
}
return nil, nil
}
}
func fileNameTemplateVersionMatcher(fileNamePattern string, contentTemplate string) evidenceMatcher {
pat := regexp.MustCompile(fileNamePattern)
return func(classifier classifier, reader source.LocationReadCloser) ([]pkg.Package, error) {
if !pat.MatchString(reader.RealPath) {
return func(resolver source.FileResolver, classifier classifier, location source.Location) ([]pkg.Package, error) {
if !pat.MatchString(location.RealPath) {
return nil, nil
}
filepathNamedGroupValues := internal.MatchNamedCaptureGroups(pat, reader.RealPath)
filepathNamedGroupValues := internal.MatchNamedCaptureGroups(pat, location.RealPath)
// versions like 3.5 should not match any character, but explicit dot
for k, v := range filepathNamedGroupValues {
filepathNamedGroupValues[k] = strings.ReplaceAll(v, ".", "\\.")
}
tmpl, err := template.New("").Parse(contentTemplate)
if err != nil {
@@ -76,26 +101,70 @@ func fileNameTemplateVersionMatcher(fileNamePattern string, contentTemplate stri
return nil, fmt.Errorf("unable to compile rendered regex=%q: %w", patternBuf.String(), err)
}
contents, err := getContents(reader)
contents, err := getContents(resolver, location)
if err != nil {
return nil, fmt.Errorf("unable to get read contents for file: %w", err)
}
matchMetadata := internal.MatchNamedCaptureGroups(tmplPattern, string(contents))
return singlePackage(classifier, reader, matchMetadata), nil
return singlePackage(classifier, location, matchMetadata), nil
}
}
func fileContentsVersionMatcher(pattern string) evidenceMatcher {
pat := regexp.MustCompile(pattern)
return func(classifier classifier, reader source.LocationReadCloser) ([]pkg.Package, error) {
contents, err := getContents(reader)
return func(resolver source.FileResolver, classifier classifier, location source.Location) ([]pkg.Package, error) {
contents, err := getContents(resolver, location)
if err != nil {
return nil, fmt.Errorf("unable to get read contents for file: %w", err)
}
matchMetadata := internal.MatchNamedCaptureGroups(pat, string(contents))
return singlePackage(classifier, reader, matchMetadata), nil
return singlePackage(classifier, location, matchMetadata), nil
}
}
//nolint:gocognit
func sharedLibraryLookup(sharedLibraryPattern string, sharedLibraryMatcher evidenceMatcher) evidenceMatcher {
pat := regexp.MustCompile(sharedLibraryPattern)
return func(resolver source.FileResolver, classifier classifier, location source.Location) (packages []pkg.Package, _ error) {
libs, err := sharedLibraries(resolver, location)
if err != nil {
return nil, err
}
for _, lib := range libs {
if !pat.MatchString(lib) {
continue
}
locations, err := resolver.FilesByGlob("**/" + lib)
if err != nil {
return nil, err
}
for _, libraryLication := range locations {
pkgs, err := sharedLibraryMatcher(resolver, classifier, libraryLication)
if err != nil {
return nil, err
}
for _, p := range pkgs {
// set the source binary as the first location
locationSet := source.NewLocationSet(location)
locationSet.Add(p.Locations.ToSlice()...)
p.Locations = locationSet
meta, _ := p.Metadata.(pkg.BinaryMetadata)
p.Metadata = pkg.BinaryMetadata{
Matches: append([]pkg.ClassifierMatch{
{
Classifier: classifier.Class,
Location: location,
},
}, meta.Matches...),
}
packages = append(packages, p)
}
}
}
return packages, nil
}
}
@@ -107,7 +176,7 @@ func mustPURL(purl string) packageurl.PackageURL {
return p
}
func singlePackage(classifier classifier, reader source.LocationReadCloser, matchMetadata map[string]string) []pkg.Package {
func singlePackage(classifier classifier, location source.Location, matchMetadata map[string]string) []pkg.Package {
version, ok := matchMetadata["version"]
if !ok {
return nil
@@ -125,15 +194,18 @@ func singlePackage(classifier classifier, reader source.LocationReadCloser, matc
p := pkg.Package{
Name: classifier.Package,
Version: version,
Locations: source.NewLocationSet(reader.Location),
Locations: source.NewLocationSet(location),
Type: pkg.BinaryPkg,
CPEs: cpes,
FoundBy: catalogerName,
MetadataType: pkg.BinaryMetadataType,
Metadata: pkg.BinaryMetadata{
Classifier: classifier.Class,
RealPath: reader.RealPath,
VirtualPath: reader.VirtualPath,
Matches: []pkg.ClassifierMatch{
{
Classifier: classifier.Class,
Location: location,
},
},
},
}
@@ -156,8 +228,13 @@ func singlePackage(classifier classifier, reader source.LocationReadCloser, matc
return []pkg.Package{p}
}
func getContents(reader source.LocationReadCloser) ([]byte, error) {
unionReader, err := unionreader.GetUnionReader(reader.ReadCloser)
func getContents(resolver source.FileResolver, location source.Location) ([]byte, error) {
reader, err := resolver.FileContentsByLocation(location)
if err != nil {
return nil, err
}
unionReader, err := unionreader.GetUnionReader(reader)
if err != nil {
return nil, fmt.Errorf("unable to get union reader for file: %w", err)
}
@@ -177,3 +254,43 @@ func singleCPE(cpeString string) []cpe.CPE {
cpe.Must(cpeString),
}
}
// sharedLibraries returns a list of all shared libraries found within a binary, currently
// supporting: elf, macho, and windows pe
func sharedLibraries(resolver source.FileResolver, location source.Location) ([]string, error) {
contents, err := getContents(resolver, location)
if err != nil {
return nil, err
}
r := bytes.NewReader(contents)
e, _ := elf.NewFile(r)
if e != nil {
symbols, err := e.ImportedLibraries()
if err != nil {
log.Debugf("unable to read elf binary at: %s -- %s", location.RealPath, err)
}
return symbols, nil
}
m, _ := macho.NewFile(r)
if m != nil {
symbols, err := m.ImportedLibraries()
if err != nil {
log.Debugf("unable to read macho binary at: %s -- %s", location.RealPath, err)
}
return symbols, nil
}
p, _ := pe.NewFile(r)
if p != nil {
symbols, err := p.ImportedLibraries()
if err != nil {
log.Debugf("unable to read pe binary at: %s -- %s", location.RealPath, err)
}
return symbols, nil
}
return nil, nil
}
+2 -4
View File
@@ -67,10 +67,8 @@ func Test_ClassifierCPEs(t *testing.T) {
locations, err := resolver.FilesByPath(test.fixture)
require.NoError(t, err)
require.Len(t, locations, 1)
location := locations[0]
readCloser, err := resolver.FileContentsByLocation(location)
require.NoError(t, err)
pkgs, err := test.classifier.EvidenceMatcher(test.classifier, source.NewLocationReadCloser(location, readCloser))
pkgs, err := test.classifier.EvidenceMatcher(resolver, test.classifier, locations[0])
require.NoError(t, err)
require.Len(t, pkgs, 1)
@@ -9,9 +9,16 @@ var defaultClassifiers = []classifier{
{
Class: "python-binary",
FileGlob: "**/python*",
EvidenceMatcher: fileNameTemplateVersionMatcher(
`(.*/|^)python(?P<version>[0-9]+\.[0-9]+)$`,
`(?m)(?P<version>{{ .version }}\.[0-9]+[-_a-zA-Z0-9]*)`),
EvidenceMatcher: evidenceMatchers(
// try to find version information from libpython shared libraries
sharedLibraryLookup(
`^libpython[0-9]+(?:\.[0-9]+)+[a-z]?\.so.*$`,
libpythonMatcher),
// check for version information in the binary
fileNameTemplateVersionMatcher(
`(?:.*/|^)python(?P<version>[0-9]+(?:\.[0-9]+)+)$`,
pythonVersionTemplate),
),
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
CPEs: []cpe.CPE{
@@ -20,25 +27,11 @@ var defaultClassifiers = []classifier{
},
},
{
Class: "python-binary-lib",
FileGlob: "**/libpython*.so*",
EvidenceMatcher: fileNameTemplateVersionMatcher(
`(.*/|^)libpython(?P<version>[0-9]+\.[0-9]+).so.*$`,
`(?m)(?P<version>{{ .version }}\.[0-9]+[-_a-zA-Z0-9]*)`),
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:a:python_software_foundation:python:*:*:*:*:*:*:*:*"),
cpe.Must("cpe:2.3:a:python:python:*:*:*:*:*:*:*:*"),
},
},
{
Class: "cpython-source",
FileGlob: "**/patchlevel.h",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)#define\s+PY_VERSION\s+"?(?P<version>[0-9\.\-_a-zA-Z]+)"?`),
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
Class: "python-binary-lib",
FileGlob: "**/libpython*.so*",
EvidenceMatcher: libpythonMatcher,
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:a:python_software_foundation:python:*:*:*:*:*:*:*:*"),
cpe.Must("cpe:2.3:a:python:python:*:*:*:*:*:*:*:*"),
@@ -120,6 +113,26 @@ var defaultClassifiers = []classifier{
Package: "busybox",
CPEs: singleCPE("cpe:2.3:a:busybox:busybox:*:*:*:*:*:*:*:*"),
},
{
Class: "haproxy-binary",
FileGlob: "**/haproxy",
EvidenceMatcher: evidenceMatchers(
fileContentsVersionMatcher(`(?m)HA-Proxy version (?P<version>[0-9]+\.[0-9]+\.[0-9]+)`),
fileContentsVersionMatcher(`(?m)(?P<version>[0-9]+\.[0-9]+\.[0-9]+)-[0-9a-zA-Z]{7}.+HAProxy version`),
),
Package: "haproxy",
PURL: mustPURL("pkg:generic/haproxy@version"),
CPEs: singleCPE("cpe:2.3:a:haproxy:haproxy:*:*:*:*:*:*:*:*"),
},
{
Class: "perl-binary",
FileGlob: "**/perl",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)\/usr\/local\/lib\/perl\d\/(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`),
Package: "perl",
PURL: mustPURL("pkg:generic/perl@version"),
CPEs: singleCPE("cpe:2.3:a:perl:perl:*:*:*:*:*:*:*:*"),
},
{
Class: "php-cli-binary",
FileGlob: "**/php*",
@@ -187,4 +200,51 @@ var defaultClassifiers = []classifier{
Package: "postgresql",
PURL: mustPURL("pkg:generic/postgresql@version"),
},
{
Class: "rust-standard-library-linux",
FileGlob: "**/libstd-????????????????.so",
EvidenceMatcher: fileContentsVersionMatcher(
// clang LLVM (rustc version 1.48.0 (7eac88abb 2020-11-16))
`(?m)(\x00)clang LLVM \(rustc version (?P<version>[0-9]+(\.[0-9]+)?(\.[0-9]+)) \(\w+ \d{4}\-\d{2}\-\d{2}\)`),
Package: "rust",
PURL: mustPURL("pkg:generic/rust@version"),
CPEs: singleCPE("cpe:2.3:a:rust-lang:rust:*:*:*:*:*:*:*:*"),
},
{
Class: "rust-standard-library-macos",
FileGlob: "**/libstd-????????????????.dylib",
EvidenceMatcher: fileContentsVersionMatcher(
// c 1.48.0 (7eac88abb 2020-11-16)
`(?m)c (?P<version>[0-9]+(\.[0-9]+)?(\.[0-9]+)) \(\w+ \d{4}\-\d{2}\-\d{2}\)`),
Package: "rust",
PURL: mustPURL("pkg:generic/rust@version"),
CPEs: singleCPE("cpe:2.3:a:rust-lang:rust:*:*:*:*:*:*:*:*"),
},
{
Class: "ruby-binary",
FileGlob: "**/ruby",
EvidenceMatcher: evidenceMatchers(
rubyMatcher,
sharedLibraryLookup(
// try to find version information from libruby shared libraries
`^libruby\.so.*$`,
rubyMatcher),
),
Package: "ruby",
PURL: mustPURL("pkg:generic/ruby@version"),
CPEs: singleCPE("cpe:2.3:a:ruby-lang:ruby:*:*:*:*:*:*:*:*"),
},
}
// in both binaries and shared libraries, the version pattern is [NUL]3.11.2[NUL]
var pythonVersionTemplate = `(?m)\x00(?P<version>{{ .version }}[-._a-zA-Z0-9]*)\x00`
var libpythonMatcher = fileNameTemplateVersionMatcher(
`(?:.*/|^)libpython(?P<version>[0-9]+(?:\.[0-9]+)+)[a-z]?\.so.*$`,
pythonVersionTemplate,
)
var rubyMatcher = fileContentsVersionMatcher(
// ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux]
// ruby 2.7.7p221 (2022-11-24 revision 168ec2b1e5) [x86_64-linux]
`(?m)ruby (?P<version>[0-9]\.[0-9]\.[0-9](p[0-9]+)?) `)
@@ -0,0 +1 @@
classifiers/dynamic
@@ -0,0 +1,74 @@
.PHONY: all
all: \
classifiers/dynamic/python-binary-shared-lib-3.11 \
classifiers/dynamic/python-binary-shared-lib-redhat-3.9 \
classifiers/dynamic/python-binary-with-version-3.9 \
classifiers/dynamic/python-binary-3.4-alpine \
classifiers/dynamic/ruby-library-3.2.1 \
classifiers/dynamic/ruby-library-2.7.7
classifiers/dynamic/python-binary-shared-lib-3.11:
$(eval $@_image := "python:3.11-slim@sha256:0b106e1d2bf485c2a41474bc9cd5103e9eea4e179f40f10741b53b127059221e")
./get-image-file.sh $($@_image) \
/usr/local/bin/python3.11 \
$@/python3
./get-image-file.sh $($@_image) \
/usr/local/lib/libpython3.11.so.1.0 \
$@/libpython3.11.so.1.0
classifiers/dynamic/python-binary-shared-lib-redhat-3.9:
$(eval $@_image := "registry.access.redhat.com/ubi8/python-39@sha256:f3cf958b96ce016b63e3e163e488f52e42891304dafef5a0811563f22e3cbad0")
./get-image-file.sh $($@_image) \
/usr/bin/python3.9 \
$@/python3.9
./get-image-file.sh $($@_image) \
/usr/lib64/libpython3.9.so.1.0 \
$@/libpython3.9.so.1.0
classifiers/dynamic/python-binary-with-version-3.9:
$(eval $@_image := "python:3.9.16-bullseye@sha256:93fb93c461a2e47a2176706fad1f39eaacd5dd40e19c0b018699a28c03eb2e2a")
./get-image-file.sh $($@_image) \
/usr/bin/python3.9 \
$@/python3.9
classifiers/dynamic/python-binary-3.4-alpine:
$(eval $@_image := "python:3.4-alpine@sha256:c210b660e2ea553a7afa23b41a6ed112f85dbce25cbcb567c75dfe05342a4c4b")
./get-image-file.sh $($@_image) \
/usr/local/bin/python3.4 \
$@/python3.4
./get-image-file.sh $($@_image) \
/usr/local/lib/libpython3.4m.so.1.0 \
$@/libpython3.4m.so.1.0
classifiers/dynamic/ruby-library-3.2.1:
$(eval $@_image := "ruby:3.2.1-bullseye@sha256:b4a140656b0c5d26c0a80559b228b4d343f3fdbf56682fcbe88f6db1fa9afa6b")
./get-image-file.sh $($@_image) \
/usr/local/bin/ruby \
$@/ruby
./get-image-file.sh $($@_image) \
/usr/local/lib/libruby.so.3.2.1 \
$@/libruby.so.3.2.1
./get-image-file.sh $($@_image) \
/usr/local/lib/libruby.so.3.2 \
$@/libruby.so.3.2
classifiers/dynamic/ruby-library-2.7.7:
$(eval $@_image := "ruby:2.7.7-bullseye@sha256:055191740a063f33fef1f09423e5ed8f91143aae62a3772a90910118464c5120")
./get-image-file.sh $($@_image) \
/usr/local/bin/ruby \
$@/ruby
./get-image-file.sh $($@_image) \
/usr/local/lib/libruby.so.2.7.7 \
$@/libruby.so.2.7.7
./get-image-file.sh $($@_image) \
/usr/local/lib/libruby.so.2.7 \
$@/libruby.so.2.7
.PHONY: clean
clean:
rm -rf classifiers/dynamic
.PHONY: cache.fingerprint
cache.fingerprint: # for CI
$(title,Install test fixture fingerprint)
@find ./classifiers/dynamic/* -type f -exec md5sum {} + | awk '{print $1}' | sort | tee /dev/stderr | md5sum | tee cache.fingerprint >> cache.fingerprint
@@ -1,2 +1,2 @@
!libpython3.7.so
!libphp.so
!*.so
!VERSION
@@ -0,0 +1 @@
HA-Proxy version 1.5.14 2015/07/02Copyright 2000-2015 Willy Tarreau <willy@haproxy.org>
@@ -1,2 +0,0 @@
# note: this SHOULD match as python 3.7
noise3.7.4a-vZ9!morenoise
@@ -1,7 +0,0 @@
# note: this SHOULD match as python 3.9
some source code...
#define PY_VERSION 3.9-aZ5
more source!
@@ -0,0 +1 @@
#define PY_VERSION 3.8.16
@@ -1,3 +0,0 @@
# note: this SHOULD match as python 3.6
noise3.6.3a-vZ9!morenoise
@@ -0,0 +1,9 @@
super from singleton method that is defined to multiple classes is not supported; this will be fixed in 1.9.3 or later
/usr/local/lib/ruby/site_ruby/1.9.1
/usr/local/lib/ruby/site_ruby/1.9.1/x86_64-linux
/usr/local/lib/ruby/vendor_ruby/1.9.1
/usr/local/lib/ruby/vendor_ruby/1.9.1/x86_64-linux
/usr/local/lib/ruby/1.9.1
/usr/local/lib/ruby/1.9.1/x86_64-linux
ruby 1.9.3p551 (2014-11-13 revision 48407) [x86_64-linux]
1.9.3
+15
View File
@@ -0,0 +1,15 @@
#!/usr/bin/env bash
set -uxe
CTRID=$(docker create $1)
function cleanup() {
docker rm "${CTRID}"
}
trap cleanup EXIT
set +e
mkdir -p $(dirname $3)
docker cp ${CTRID}:$2 $3
+17 -13
View File
@@ -3,6 +3,7 @@ package cataloger
import (
"fmt"
"math"
"runtime/debug"
"sync"
"github.com/hashicorp/go-multierror"
@@ -49,8 +50,15 @@ func newMonitor() (*progress.Manual, *progress.Manual) {
return &filesProcessed, &packagesDiscovered
}
func runCataloger(cataloger pkg.Cataloger, resolver source.FileResolver) (*catalogResult, error) {
catalogerResult := new(catalogResult)
func runCataloger(cataloger pkg.Cataloger, resolver source.FileResolver) (catalogerResult *catalogResult, err error) {
// handle individual cataloger panics
defer func() {
if e := recover(); e != nil {
err = fmt.Errorf("%v at:\n%s", e, string(debug.Stack()))
}
}()
catalogerResult = new(catalogResult)
// find packages from the underlying raw data
log.WithFields("cataloger", cataloger.Name()).Trace("cataloging started")
@@ -88,7 +96,7 @@ func runCataloger(cataloger pkg.Cataloger, resolver source.FileResolver) (*catal
}
catalogerResult.Relationships = append(catalogerResult.Relationships, relationships...)
log.WithFields("cataloger", cataloger.Name()).Trace("cataloging complete")
return catalogerResult, nil
return catalogerResult, err
}
// Catalog a given source (container image or filesystem) with the given catalogers, returning all discovered packages.
@@ -100,7 +108,11 @@ func runCataloger(cataloger pkg.Cataloger, resolver source.FileResolver) (*catal
func Catalog(resolver source.FileResolver, release *linux.Release, parallelism int, catalogers ...pkg.Cataloger) (*pkg.Catalog, []artifact.Relationship, error) {
catalog := pkg.NewCatalog()
var allRelationships []artifact.Relationship
filesProcessed, packagesDiscovered := newMonitor()
defer filesProcessed.SetCompleted()
defer packagesDiscovered.SetCompleted()
// perform analysis, accumulating errors for each failed analysis
var errs error
@@ -139,7 +151,7 @@ func Catalog(resolver source.FileResolver, release *linux.Release, parallelism i
// dynamically show updated discovered package status
go func() {
for discovered := range discoveredPackages {
packagesDiscovered.N += discovered
packagesDiscovered.Add(discovered)
}
}()
@@ -158,7 +170,6 @@ func Catalog(resolver source.FileResolver, release *linux.Release, parallelism i
for result := range results {
if result.Error != nil {
errs = multierror.Append(errs, result.Error)
continue
}
for _, p := range result.Packages {
catalog.Add(p)
@@ -168,14 +179,7 @@ func Catalog(resolver source.FileResolver, release *linux.Release, parallelism i
allRelationships = append(allRelationships, pkg.NewRelationships(catalog)...)
if errs != nil {
return nil, nil, errs
}
filesProcessed.SetCompleted()
packagesDiscovered.SetCompleted()
return catalog, allRelationships, nil
return catalog, allRelationships, errs
}
func packageFileOwnershipRelationships(p pkg.Package, resolver source.FilePathResolver) ([]artifact.Relationship, error) {
+67
View File
@@ -0,0 +1,67 @@
package cataloger
import (
"testing"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
func Test_CatalogPanicHandling(t *testing.T) {
catalog, relationships, err := Catalog(
source.NewMockResolverForPaths(),
&linux.Release{},
1,
panickingCataloger{},
returningCataloger{},
)
require.Error(t, err)
require.Contains(t, err.Error(), "catalog_test.go")
require.Len(t, catalog.Sorted(), 2)
require.Len(t, relationships, 1)
}
type panickingCataloger struct{}
func (p panickingCataloger) Name() string {
return "panicking-cataloger"
}
func (p panickingCataloger) Catalog(_ source.FileResolver) ([]pkg.Package, []artifact.Relationship, error) {
panic("something bad happened")
}
var _ pkg.Cataloger = (*panickingCataloger)(nil)
type returningCataloger struct{}
func (p returningCataloger) Name() string {
return "returning-cataloger"
}
func (p returningCataloger) Catalog(_ source.FileResolver) ([]pkg.Package, []artifact.Relationship, error) {
pkg1 := pkg.Package{
Name: "package-1",
Version: "1.0",
}
pkg1.SetID()
pkg2 := pkg.Package{
Name: "package-2",
Version: "2.0",
}
pkg2.SetID()
return []pkg.Package{pkg1, pkg2}, []artifact.Relationship{
{
From: pkg1,
To: pkg2,
Type: artifact.DependencyOfRelationship,
},
}, nil
}
var _ pkg.Cataloger = (*returningCataloger)(nil)
+21 -1
View File
@@ -161,9 +161,29 @@ func contains(enabledPartial []string, catalogerName string) bool {
if partial == "" {
continue
}
if strings.Contains(catalogerName, partial) {
if hasFullWord(partial, catalogerName) {
return true
}
}
return false
}
func hasFullWord(targetPhrase, candidate string) bool {
if targetPhrase == "cataloger" || targetPhrase == "" {
return false
}
start := strings.Index(candidate, targetPhrase)
if start == -1 {
return false
}
if start > 0 && candidate[start-1] != '-' {
return false
}
end := start + len(targetPhrase)
if end < len(candidate) && candidate[end] != '-' {
return false
}
return true
}
+139 -2
View File
@@ -25,6 +25,38 @@ func (d dummy) Catalog(_ source.FileResolver) ([]pkg.Package, []artifact.Relatio
}
func Test_filterCatalogers(t *testing.T) {
largeCatalogerList := []string{
"alpmdb-cataloger",
"apkdb-cataloger",
"binary-cataloger",
"conan-cataloger",
"dartlang-lock-cataloger",
"dpkgdb-cataloger",
"dotnet-deps-cataloger",
"elixir-mix-lock-cataloger",
"erlang-rebar-lock-cataloger",
"go-mod-file-cataloger",
"go-module-binary-cataloger",
"haskell-cataloger",
"graalvm-native-image-cataloger",
"java-cataloger",
"java-pom-cataloger",
"javascript-package-cataloger",
"javascript-lock-cataloger",
"php-composer-installed-cataloger",
"php-composer-lock-cataloger",
"portage-cataloger",
"python-index-cataloger",
"python-package-cataloger",
"rpm-db-cataloger",
"rpm-file-cataloger",
"ruby-gemfile-cataloger",
"ruby-gemspec-cataloger",
"rust-cargo-lock-cataloger",
"cargo-auditable-binary-cataloger",
"sbom-cataloger",
"cocoapods-cataloger",
}
tests := []struct {
name string
patterns []string
@@ -144,6 +176,43 @@ func Test_filterCatalogers(t *testing.T) {
"go-module-binary-cataloger",
},
},
{
name: "don't cross match ecosystems with matching prefix",
patterns: []string{
"java-cataloger",
},
catalogers: []string{
"javascript-package-cataloger",
"java-cataloger",
},
want: []string{
"java-cataloger",
},
},
{
name: "don't cross match ecosystems with short, common name",
patterns: []string{
"go",
},
catalogers: largeCatalogerList,
want: []string{
"go-mod-file-cataloger",
"go-module-binary-cataloger",
//"rust-cargo-lock-cataloger", // with naive "contains" matching
//"cargo-auditable-binary-cataloger", // with naive "contains" matching
},
},
{
name: "ignore partial matches",
patterns: []string{
"mod",
},
catalogers: largeCatalogerList,
want: []string{
"go-mod-file-cataloger",
//"go-module-binary-cataloger", // unfortunately not a full word (this should probably be renamed)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
@@ -162,8 +231,6 @@ func Test_filterCatalogers(t *testing.T) {
}
func Test_contains(t *testing.T) {
type args struct {
}
tests := []struct {
name string
enabledCatalogers []string
@@ -201,3 +268,73 @@ func Test_contains(t *testing.T) {
})
}
}
func Test_hasFullWord(t *testing.T) {
tests := []struct {
name string
targetPhrase string
candidate string
want bool
}{
{
name: "exact match",
targetPhrase: "php-composer-installed-cataloger",
candidate: "php-composer-installed-cataloger",
want: true,
},
{
name: "partial, full word match",
targetPhrase: "composer",
candidate: "php-composer-installed-cataloger",
want: true,
},
{
name: "partial, full, multi-word match",
targetPhrase: "php-composer",
candidate: "php-composer-installed-cataloger",
want: true,
},
{
name: "prefix match",
targetPhrase: "php",
candidate: "php-composer-installed-cataloger",
want: true,
},
{
name: "postfix match with -cataloger suffix",
targetPhrase: "installed",
candidate: "php-composer-installed-cataloger",
want: true,
},
{
name: "postfix match",
targetPhrase: "installed",
candidate: "php-composer-installed",
want: true,
},
{
name: "ignore cataloger keyword",
targetPhrase: "cataloger",
candidate: "php-composer-installed-cataloger",
want: false,
},
{
name: "ignore partial match",
targetPhrase: "hp",
candidate: "php-composer-installed-cataloger",
want: false,
},
{
name: "ignore empty string",
targetPhrase: "",
candidate: "php-composer-installed-cataloger",
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equalf(t, tt.want, hasFullWord(tt.targetPhrase, tt.candidate), "hasFullWord(%v, %v)", tt.targetPhrase, tt.candidate)
})
}
}
+85 -127
View File
@@ -1,149 +1,62 @@
package cpe
import (
"fmt"
"regexp"
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/pkg"
)
var (
pythonPrefixes = []string{"py-", "py2-", "py3-"}
rubyPrefixes = []string{"ruby-"}
prefixesToPackageType = map[string]pkg.Type{
"py-": pkg.PythonPkg,
"ruby-": pkg.GemPkg,
}
streamVersionPkgNamePattern = regexp.MustCompile(`^(?P<stream>[a-zA-Z][\w-]*?)(?P<streamVersion>\-?\d[\d\.]*?)($|-(?P<subPackage>[a-zA-Z][\w-]*?)?)$`)
)
func pythonCandidateVendorsFromName(v string) fieldCandidateSet {
vendors := newFieldCandidateSet()
vendors.add(fieldCandidate{
value: v,
disallowSubSelections: true,
disallowDelimiterVariations: true,
})
vendors.addValue(findAdditionalVendors(defaultCandidateAdditions, pkg.PythonPkg, v, v)...)
vendors.removeByValue(findVendorsToRemove(defaultCandidateRemovals, pkg.PythonPkg, v)...)
for _, av := range additionalVendorsForPython(v) {
vendors.add(fieldCandidate{
value: av,
disallowSubSelections: true,
disallowDelimiterVariations: true,
})
vendors.addValue(findAdditionalVendors(defaultCandidateAdditions, pkg.PythonPkg, av, av)...)
vendors.removeByValue(findVendorsToRemove(defaultCandidateRemovals, pkg.PythonPkg, av)...)
}
return vendors
type upstreamCandidate struct {
Name string
Type pkg.Type
}
func pythonCandidateVendorsFromAPK(m pkg.ApkMetadata) fieldCandidateSet {
vendors := newFieldCandidateSet()
func upstreamCandidates(m pkg.ApkMetadata) (candidates []upstreamCandidate) {
// Do not consider OriginPackage variations when generating CPE candidates for the child package
// because doing so will result in false positives when matching to vulnerabilities in Grype since
// it won't know to lookup apk fix entries using the OriginPackage name.
for _, p := range pythonPrefixes {
if strings.HasPrefix(m.Package, p) {
t := strings.ToLower(strings.TrimPrefix(m.Package, p))
vendors.union(pythonCandidateVendorsFromName(t))
}
name := m.Package
groups := internal.MatchNamedCaptureGroups(streamVersionPkgNamePattern, m.Package)
stream, ok := groups["stream"]
if m.OriginPackage != m.Package && strings.HasPrefix(m.OriginPackage, p) {
t := strings.ToLower(strings.TrimPrefix(m.OriginPackage, p))
vendors.union(pythonCandidateVendorsFromName(t))
if ok && stream != "" {
sub, ok := groups["subPackage"]
if ok && sub != "" {
name = fmt.Sprintf("%s-%s", stream, sub)
} else {
name = stream
}
}
return vendors
}
func pythonCandidateProductsFromName(p string) fieldCandidateSet {
products := newFieldCandidateSet()
products.add(fieldCandidate{
value: p,
disallowSubSelections: true,
disallowDelimiterVariations: true,
})
products.addValue(findAdditionalProducts(defaultCandidateAdditions, pkg.PythonPkg, p)...)
products.removeByValue(findProductsToRemove(defaultCandidateRemovals, pkg.PythonPkg, p)...)
return products
}
func pythonCandidateProductsFromAPK(m pkg.ApkMetadata) fieldCandidateSet {
products := newFieldCandidateSet()
for _, p := range pythonPrefixes {
if strings.HasPrefix(m.Package, p) {
t := strings.ToLower(strings.TrimPrefix(m.Package, p))
products.union(pythonCandidateProductsFromName(t))
}
if m.OriginPackage != m.Package && strings.HasPrefix(m.OriginPackage, p) {
t := strings.ToLower(strings.TrimPrefix(m.OriginPackage, p))
products.union(pythonCandidateProductsFromName(t))
for prefix, typ := range prefixesToPackageType {
if strings.HasPrefix(name, prefix) {
t := strings.TrimPrefix(name, prefix)
if t != "" {
candidates = append(candidates, upstreamCandidate{Name: t, Type: typ})
return candidates
}
}
}
return products
}
func rubyCandidateVendorsFromName(v string) fieldCandidateSet {
vendors := newFieldCandidateSet()
vendors.add(fieldCandidate{
value: v,
disallowSubSelections: true,
disallowDelimiterVariations: true,
})
vendors.addValue(findAdditionalVendors(defaultCandidateAdditions, pkg.GemPkg, v, v)...)
vendors.removeByValue(findVendorsToRemove(defaultCandidateRemovals, pkg.GemPkg, v)...)
return vendors
}
func rubyCandidateVendorsFromAPK(m pkg.ApkMetadata) fieldCandidateSet {
vendors := newFieldCandidateSet()
for _, p := range rubyPrefixes {
if strings.HasPrefix(m.Package, p) {
t := strings.ToLower(strings.TrimPrefix(m.Package, p))
vendors.union(rubyCandidateVendorsFromName(t))
}
if m.OriginPackage != m.Package && strings.HasPrefix(m.OriginPackage, p) {
t := strings.ToLower(strings.TrimPrefix(m.OriginPackage, p))
vendors.union(rubyCandidateVendorsFromName(t))
}
if name != "" {
candidates = append(candidates, upstreamCandidate{Name: name, Type: pkg.UnknownPkg})
return candidates
}
return vendors
}
func rubyCandidateProductsFromName(p string) fieldCandidateSet {
products := newFieldCandidateSet()
products.add(fieldCandidate{
value: p,
disallowSubSelections: true,
disallowDelimiterVariations: true,
})
products.addValue(findAdditionalProducts(defaultCandidateAdditions, pkg.GemPkg, p)...)
products.removeByValue(findProductsToRemove(defaultCandidateRemovals, pkg.GemPkg, p)...)
return products
}
func rubyCandidateProductsFromAPK(m pkg.ApkMetadata) fieldCandidateSet {
products := newFieldCandidateSet()
for _, p := range rubyPrefixes {
if strings.HasPrefix(m.Package, p) {
t := strings.ToLower(strings.TrimPrefix(m.Package, p))
products.union(rubyCandidateProductsFromName(t))
}
if m.OriginPackage != m.Package && strings.HasPrefix(m.OriginPackage, p) {
t := strings.ToLower(strings.TrimPrefix(m.OriginPackage, p))
products.union(rubyCandidateProductsFromName(t))
}
}
return products
return candidates
}
func candidateVendorsForAPK(p pkg.Package) fieldCandidateSet {
@@ -153,8 +66,36 @@ func candidateVendorsForAPK(p pkg.Package) fieldCandidateSet {
}
vendors := newFieldCandidateSet()
vendors.union(pythonCandidateVendorsFromAPK(metadata))
vendors.union(rubyCandidateVendorsFromAPK(metadata))
candidates := upstreamCandidates(metadata)
for _, c := range candidates {
switch c.Type {
case pkg.UnknownPkg:
vendors.addValue(c.Name)
vendors.addValue(findAdditionalVendors(defaultCandidateAdditions, pkg.ApkPkg, c.Name, c.Name)...)
vendors.removeByValue(findVendorsToRemove(defaultCandidateRemovals, pkg.ApkPkg, c.Name)...)
case pkg.PythonPkg:
vendors.addValue(c.Name)
vendors.addValue(findAdditionalVendors(defaultCandidateAdditions, c.Type, c.Name, c.Name)...)
vendors.removeByValue(findVendorsToRemove(defaultCandidateRemovals, c.Type, c.Name)...)
for _, av := range additionalVendorsForPython(c.Name) {
vendors.addValue(av)
vendors.addValue(findAdditionalVendors(defaultCandidateAdditions, pkg.PythonPkg, av, av)...)
vendors.removeByValue(findVendorsToRemove(defaultCandidateRemovals, pkg.PythonPkg, av)...)
}
default:
vendors.addValue(c.Name)
vendors.addValue(findAdditionalVendors(defaultCandidateAdditions, c.Type, c.Name, c.Name)...)
vendors.removeByValue(findVendorsToRemove(defaultCandidateRemovals, c.Type, c.Name)...)
}
}
vendors.union(candidateVendorsFromURL(metadata.URL))
for v := range vendors {
v.disallowDelimiterVariations = true
v.disallowSubSelections = true
}
return vendors
}
@@ -166,8 +107,25 @@ func candidateProductsForAPK(p pkg.Package) fieldCandidateSet {
}
products := newFieldCandidateSet()
products.union(pythonCandidateProductsFromAPK(metadata))
products.union(rubyCandidateProductsFromAPK(metadata))
candidates := upstreamCandidates(metadata)
for _, c := range candidates {
switch c.Type {
case pkg.UnknownPkg:
products.addValue(c.Name)
products.addValue(findAdditionalProducts(defaultCandidateAdditions, pkg.ApkPkg, c.Name)...)
products.removeByValue(findProductsToRemove(defaultCandidateRemovals, pkg.ApkPkg, c.Name)...)
default:
products.addValue(c.Name)
products.addValue(findAdditionalProducts(defaultCandidateAdditions, c.Type, c.Name)...)
products.removeByValue(findProductsToRemove(defaultCandidateRemovals, c.Type, c.Name)...)
}
}
for p := range products {
p.disallowDelimiterVariations = true
p.disallowSubSelections = true
}
return products
}
+358 -6
View File
@@ -24,10 +24,11 @@ func Test_candidateVendorsForAPK(t *testing.T) {
expected: []string{"python-cryptography_project", "cryptography", "cryptographyproject", "cryptography_project"},
},
{
name: "py2-pypdf OriginPackage",
name: "py2-pypdf with explicit different origin",
pkg: pkg.Package{
Metadata: pkg.ApkMetadata{
OriginPackage: "py2-pypdf",
Package: "py2-pypdf",
OriginPackage: "abcdefg",
},
},
expected: []string{"pypdf", "pypdfproject", "pypdf_project"},
@@ -41,10 +42,67 @@ func Test_candidateVendorsForAPK(t *testing.T) {
},
expected: []string{"armadillo"},
},
{
name: "python-3.6",
pkg: pkg.Package{
Metadata: pkg.ApkMetadata{
Package: "python-3.6",
},
},
expected: []string{"python", "python_software_foundation"},
},
{
name: "ruby-3.6",
pkg: pkg.Package{
Metadata: pkg.ApkMetadata{
Package: "ruby-3.6",
URL: "https://www.ruby-lang.org/",
},
},
expected: []string{"ruby", "ruby-lang"},
},
{
name: "make",
pkg: pkg.Package{
Metadata: pkg.ApkMetadata{
Package: "make",
URL: "https://www.gnu.org/software/make",
},
},
expected: []string{"gnu", "make"},
},
{
name: "ruby-rake with matching origin",
pkg: pkg.Package{
Name: "ruby-rake",
Type: pkg.ApkPkg,
MetadataType: pkg.ApkMetadataType,
Metadata: pkg.ApkMetadata{
Package: "ruby-rake",
URL: "https://github.com/ruby/rake",
OriginPackage: "ruby-rake",
},
},
expected: []string{"rake", "ruby-lang", "ruby"},
},
{
name: "ruby-rake with non-matching origin",
pkg: pkg.Package{
Name: "ruby-rake",
Type: pkg.ApkPkg,
MetadataType: pkg.ApkMetadataType,
Metadata: pkg.ApkMetadata{
Package: "ruby-rake",
URL: "https://www.ruby-lang.org/",
OriginPackage: "ruby",
},
},
expected: []string{"rake", "ruby-lang"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.ElementsMatch(t, test.expected, candidateVendorsForAPK(test.pkg).values(), "different vendors")
assert.ElementsMatch(t, test.expected, candidateVendorsForAPK(test.pkg).uniqueValues(), "different vendors")
})
}
}
@@ -65,10 +123,11 @@ func Test_candidateProductsForAPK(t *testing.T) {
expected: []string{"cryptography", "python-cryptography"},
},
{
name: "py2-pypdf OriginPackage",
name: "py2-pypdf with explicit different origin",
pkg: pkg.Package{
Metadata: pkg.ApkMetadata{
OriginPackage: "py2-pypdf",
Package: "py2-pypdf",
OriginPackage: "abcdefg",
},
},
expected: []string{"pypdf"},
@@ -82,10 +141,303 @@ func Test_candidateProductsForAPK(t *testing.T) {
},
expected: []string{"armadillo"},
},
{
name: "python-3.6",
pkg: pkg.Package{
Metadata: pkg.ApkMetadata{
Package: "python-3.6",
},
},
expected: []string{"python"},
},
{
name: "ruby-3.6",
pkg: pkg.Package{
Metadata: pkg.ApkMetadata{
Package: "ruby-3.6",
URL: "https://www.ruby-lang.org/",
},
},
expected: []string{"ruby"},
},
{
name: "make",
pkg: pkg.Package{
Metadata: pkg.ApkMetadata{
Package: "make",
URL: "https://www.gnu.org/software/make",
},
},
expected: []string{"make"},
},
{
name: "ruby-rake with matching origin",
pkg: pkg.Package{
Metadata: pkg.ApkMetadata{
Package: "ruby-rake",
URL: "https://github.com/ruby/rake",
OriginPackage: "ruby-rake",
},
},
expected: []string{"rake"},
},
{
name: "ruby-rake with non-matching origin",
pkg: pkg.Package{
Name: "ruby-rake",
Type: pkg.ApkPkg,
MetadataType: pkg.ApkMetadataType,
Metadata: pkg.ApkMetadata{
Package: "ruby-rake",
URL: "https://www.ruby-lang.org/",
OriginPackage: "ruby",
},
},
expected: []string{"rake"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.ElementsMatch(t, test.expected, candidateProductsForAPK(test.pkg).values(), "different products")
assert.ElementsMatch(t, test.expected, candidateProductsForAPK(test.pkg).uniqueValues(), "different products")
})
}
}
func Test_upstreamCandidates(t *testing.T) {
tests := []struct {
name string
metadata pkg.ApkMetadata
expected []upstreamCandidate
}{
{
name: "gocase",
metadata: pkg.ApkMetadata{
Package: "p",
},
expected: []upstreamCandidate{
{Name: "p", Type: pkg.UnknownPkg},
},
},
{
name: "same package and origin simple case",
metadata: pkg.ApkMetadata{
Package: "p",
OriginPackage: "p",
},
expected: []upstreamCandidate{
{Name: "p", Type: pkg.UnknownPkg},
},
},
{
name: "different package and origin",
metadata: pkg.ApkMetadata{
Package: "p",
OriginPackage: "origin",
},
expected: []upstreamCandidate{
{Name: "p", Type: pkg.UnknownPkg},
},
},
{
name: "upstream python package information as qualifier py- prefix",
metadata: pkg.ApkMetadata{
Package: "py-potatoes",
OriginPackage: "py-potatoes",
},
expected: []upstreamCandidate{
{Name: "potatoes", Type: pkg.PythonPkg},
},
},
{
name: "upstream python package information as qualifier py3- prefix",
metadata: pkg.ApkMetadata{
Package: "py3-potatoes",
OriginPackage: "py3-potatoes",
},
expected: []upstreamCandidate{
{Name: "potatoes", Type: pkg.PythonPkg},
},
},
{
name: "python package with distinct origin package",
metadata: pkg.ApkMetadata{
Package: "py3-non-existant",
OriginPackage: "abcdefg",
},
expected: []upstreamCandidate{
{Name: "non-existant", Type: pkg.PythonPkg},
},
},
{
name: "upstream ruby package information as qualifier",
metadata: pkg.ApkMetadata{
Package: "ruby-something",
OriginPackage: "ruby-something",
},
expected: []upstreamCandidate{
{Name: "something", Type: pkg.GemPkg},
},
},
{
name: "ruby package with distinct origin package",
metadata: pkg.ApkMetadata{
Package: "ruby-something",
OriginPackage: "1234567",
},
expected: []upstreamCandidate{
{Name: "something", Type: pkg.GemPkg},
},
},
{
name: "postgesql-15 upstream postgresql",
metadata: pkg.ApkMetadata{
Package: "postgresql-15",
},
expected: []upstreamCandidate{
{Name: "postgresql", Type: pkg.UnknownPkg},
},
},
{
name: "postgesql15 upstream postgresql",
metadata: pkg.ApkMetadata{
Package: "postgresql15",
},
expected: []upstreamCandidate{
{Name: "postgresql", Type: pkg.UnknownPkg},
},
},
{
name: "go-1.19 upstream go",
metadata: pkg.ApkMetadata{
Package: "go-1.19",
},
expected: []upstreamCandidate{
{Name: "go", Type: pkg.UnknownPkg},
},
},
{
name: "go1.143 upstream go",
metadata: pkg.ApkMetadata{
Package: "go1.143",
},
expected: []upstreamCandidate{
{Name: "go", Type: pkg.UnknownPkg},
},
},
{
name: "abc-101.191.23456 upstream abc",
metadata: pkg.ApkMetadata{
Package: "abc-101.191.23456",
},
expected: []upstreamCandidate{
{Name: "abc", Type: pkg.UnknownPkg},
},
},
{
name: "abc101.191.23456 upstream abc",
metadata: pkg.ApkMetadata{
Package: "abc101.191.23456",
},
expected: []upstreamCandidate{
{Name: "abc", Type: pkg.UnknownPkg},
},
},
{
name: "abc101-12345-1045 upstream abc101-12345",
metadata: pkg.ApkMetadata{
Package: "abc101-12345-1045",
},
expected: []upstreamCandidate{
{Name: "abc101-12345", Type: pkg.UnknownPkg},
},
},
{
name: "abc101-a12345-1045 upstream abc101-a12345",
metadata: pkg.ApkMetadata{
Package: "abc101-a12345-1045",
},
expected: []upstreamCandidate{
{Name: "abc-a12345-1045", Type: pkg.UnknownPkg},
},
},
{
name: "package starting with single digit",
metadata: pkg.ApkMetadata{
Package: "3proxy",
},
expected: []upstreamCandidate{
{Name: "3proxy", Type: pkg.UnknownPkg},
},
},
{
name: "package starting with multiple digits",
metadata: pkg.ApkMetadata{
Package: "356proxy",
},
expected: []upstreamCandidate{
{Name: "356proxy", Type: pkg.UnknownPkg},
},
},
{
name: "package composed of only digits",
metadata: pkg.ApkMetadata{
Package: "123456",
},
expected: []upstreamCandidate{
{Name: "123456", Type: pkg.UnknownPkg},
},
},
{
name: "ruby-3.6 upstream ruby",
metadata: pkg.ApkMetadata{
Package: "ruby-3.6",
},
expected: []upstreamCandidate{
{Name: "ruby", Type: pkg.UnknownPkg},
},
},
{
name: "ruby3.6 upstream ruby",
metadata: pkg.ApkMetadata{
Package: "ruby3.6",
},
expected: []upstreamCandidate{
{Name: "ruby", Type: pkg.UnknownPkg},
},
},
{
name: "ruby3.6-tacos upstream tacos",
metadata: pkg.ApkMetadata{
Package: "ruby3.6-tacos",
},
expected: []upstreamCandidate{
{Name: "tacos", Type: pkg.GemPkg},
},
},
{
name: "ruby-3.6-tacos upstream tacos",
metadata: pkg.ApkMetadata{
Package: "ruby-3.6-tacos",
},
expected: []upstreamCandidate{
{Name: "tacos", Type: pkg.GemPkg},
},
},
{
name: "abc1234jksajflksa",
metadata: pkg.ApkMetadata{
Package: "abc1234jksajflksa",
},
expected: []upstreamCandidate{
{Name: "abc1234jksajflksa", Type: pkg.UnknownPkg},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
actual := upstreamCandidates(test.metadata)
assert.Equal(t, test.expected, actual)
})
}
}
@@ -178,6 +178,11 @@ var defaultCandidateAdditions = buildCandidateLookup(
candidateAddition{AdditionalVendors: []string{"pypa"}},
},
// Alpine packages
{
pkg.ApkPkg,
candidateKey{PkgName: "curl"},
candidateAddition{AdditionalVendors: []string{"haxx"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "python3"},
@@ -198,6 +203,137 @@ var defaultCandidateAdditions = buildCandidateLookup(
candidateKey{PkgName: "nodejs-current"},
candidateAddition{AdditionalProducts: []string{"node.js"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "go"},
candidateAddition{AdditionalVendors: []string{"golang"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "ruby"},
candidateAddition{AdditionalVendors: []string{"ruby-lang"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "bazel"},
candidateAddition{AdditionalVendors: []string{"google"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "clang"},
candidateAddition{AdditionalVendors: []string{"llvm"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "openjdk"},
candidateAddition{AdditionalVendors: []string{"oracle"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "glibc"},
candidateAddition{AdditionalVendors: []string{"gnu"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "glib"},
candidateAddition{AdditionalVendors: []string{"gnome"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "bash"},
candidateAddition{AdditionalVendors: []string{"gnu"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "alsa-lib"},
candidateAddition{AdditionalVendors: []string{"alsa-project"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "alsa"},
candidateAddition{AdditionalVendors: []string{"alsa-project"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "make"},
candidateAddition{AdditionalVendors: []string{"gnu"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "git"},
candidateAddition{AdditionalVendors: []string{"git-scm"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "bind"},
candidateAddition{AdditionalVendors: []string{"isc"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "libxpm"},
candidateAddition{AdditionalVendors: []string{"libxpm_project"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "musl"},
candidateAddition{AdditionalVendors: []string{"musl-libc"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "firefox"},
candidateAddition{AdditionalVendors: []string{"mozilla"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "firefox-esr"},
candidateAddition{AdditionalVendors: []string{"mozilla"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "thunderbird"},
candidateAddition{AdditionalVendors: []string{"mozilla"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "chromium"},
candidateAddition{AdditionalVendors: []string{"google"}, AdditionalProducts: []string{"chrome"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "apache"},
candidateAddition{AdditionalProducts: []string{"http_server"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "tiff"},
candidateAddition{AdditionalProducts: []string{"libtiff"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "ghostscript"},
candidateAddition{AdditionalVendors: []string{"artifex"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "openjpeg"},
candidateAddition{AdditionalVendors: []string{"uclouvain"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "xorg-server"},
candidateAddition{AdditionalVendors: []string{"x.org"}, AdditionalProducts: []string{"x_server"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "podofo"},
candidateAddition{AdditionalVendors: []string{"podofo_project"}},
},
{
pkg.ApkPkg,
candidateKey{PkgName: "wpa_supplicant"},
candidateAddition{AdditionalVendors: []string{"w1.fi"}},
},
//
// Binary packages
{
pkg.BinaryPkg,
@@ -214,6 +350,27 @@ var defaultCandidateRemovals = buildCandidateRemovalLookup(
candidateKey{PkgName: "redis"},
candidateRemovals{VendorsToRemove: []string{"redis"}},
},
// NPM packages
{
pkg.NpmPkg,
candidateKey{PkgName: "redis"},
candidateRemovals{VendorsToRemove: []string{"redis"}},
},
{
pkg.NpmPkg,
candidateKey{PkgName: "php"},
candidateRemovals{VendorsToRemove: []string{"php"}},
},
{
pkg.NpmPkg,
candidateKey{PkgName: "delegate"},
candidateRemovals{VendorsToRemove: []string{"delegate"}},
},
{
pkg.NpmPkg,
candidateKey{PkgName: "docker"},
candidateRemovals{VendorsToRemove: []string{"docker"}},
},
})
// buildCandidateLookup is a convenience function for creating the defaultCandidateAdditions set
+7 -7
View File
@@ -93,13 +93,6 @@ func candidateVendors(p pkg.Package) []string {
}
}
// some ecosystems do not have enough metadata to determine the vendor accurately, in which case we selectively
// allow * as a candidate. Note: do NOT allow Java packages to have * vendors.
switch p.Language {
case pkg.Ruby, pkg.JavaScript:
vendors.addValue(wfn.Any)
}
switch p.MetadataType {
case pkg.RpmMetadataType:
vendors.union(candidateVendorsForRPM(p))
@@ -111,8 +104,15 @@ func candidateVendors(p pkg.Package) []string {
vendors.union(candidateVendorsForJava(p))
case pkg.ApkMetadataType:
vendors.union(candidateVendorsForAPK(p))
case pkg.NpmPackageJSONMetadataType:
vendors.union(candidateVendorsForJavascript(p))
}
// We should no longer be generating vendor candidates with these values ["" and "*"]
// (since CPEs will match any other value)
vendors.removeByValue("")
vendors.removeByValue("*")
// try swapping hyphens for underscores, vice versa, and removing separators altogether
addDelimiterVariations(vendors)
+49 -8
View File
@@ -117,15 +117,20 @@ func TestGeneratePackageCPEs(t *testing.T) {
{
name: "javascript language",
p: pkg.Package{
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.JavaScript,
Type: pkg.DebPkg,
Name: "name",
Version: "3.2",
FoundBy: "some-analyzer",
Language: pkg.JavaScript,
MetadataType: pkg.NpmPackageJSONMetadataType,
Metadata: pkg.NpmPackageJSONMetadata{
Author: "jon",
URL: "https://github.com/bob/npm-name",
},
},
expected: []string{
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:*:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:jon:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:bob:name:3.2:*:*:*:*:*:*:*",
},
},
{
@@ -142,10 +147,10 @@ func TestGeneratePackageCPEs(t *testing.T) {
"someones name",
"someones.elses.name@gmail.com",
},
Homepage: "https://github.com/tom/ruby-name",
},
},
expected: []string{
"cpe:2.3:a:*:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby-lang:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby:name:3.2:*:*:*:*:*:*:*",
@@ -154,6 +159,7 @@ func TestGeneratePackageCPEs(t *testing.T) {
"cpe:2.3:a:someones-name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:someones_elses_name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:someones_name:name:3.2:*:*:*:*:*:*:*",
"cpe:2.3:a:tom:name:3.2:*:*:*:*:*:*:*",
},
},
{
@@ -641,7 +647,6 @@ func TestGeneratePackageCPEs(t *testing.T) {
},
},
expected: []string{
"cpe:2.3:a:*:bundler:2.1.4:*:*:*:*:*:*:*",
"cpe:2.3:a:bundler:bundler:2.1.4:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby-lang:bundler:2.1.4:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby:bundler:2.1.4:*:*:*:*:*:*:*",
@@ -673,6 +678,42 @@ func TestGeneratePackageCPEs(t *testing.T) {
"cpe:2.3:a:python_redis:redis:2.1.4:*:*:*:*:*:*:*",
},
},
{
name: "regression: ruby-rake apk missing expected ruby-lang:rake CPE",
p: pkg.Package{
Name: "ruby-rake",
Version: "2.7.6-r0",
Type: pkg.ApkPkg,
FoundBy: "apk-db-analyzer",
Language: pkg.UnknownLanguage,
MetadataType: pkg.ApkMetadataType,
Metadata: pkg.ApkMetadata{
Package: "ruby-rake",
URL: "https://www.ruby-lang.org/",
OriginPackage: "ruby",
},
},
expected: []string{
"cpe:2.3:a:ruby-lang:rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:rake:rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:rake:ruby-rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:rake:ruby_rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby-lang:ruby-rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby-lang:ruby_rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby-rake:rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby-rake:ruby-rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby-rake:ruby_rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby:rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby:ruby-rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby:ruby_rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby_lang:rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby_lang:ruby-rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby_lang:ruby_rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby_rake:rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby_rake:ruby-rake:2.7.6-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ruby_rake:ruby_rake:2.7.6-r0:*:*:*:*:*:*:*",
},
},
}
for _, test := range tests {
@@ -0,0 +1,32 @@
package cpe
import "github.com/anchore/syft/syft/pkg"
func candidateVendorsForJavascript(p pkg.Package) fieldCandidateSet {
if p.MetadataType != pkg.NpmPackageJSONMetadataType {
return nil
}
vendors := newFieldCandidateSet()
metadata, ok := p.Metadata.(pkg.NpmPackageJSONMetadata)
if !ok {
return nil
}
if metadata.Author != "" {
vendors.add(fieldCandidate{
value: normalizePersonName(stripEmailSuffix(metadata.Author)),
disallowSubSelections: true,
})
}
if metadata.URL != "" {
vendors.union(candidateVendorsFromURL(metadata.URL))
}
if metadata.Homepage != "" {
vendors.union(candidateVendorsFromURL(metadata.Homepage))
}
return vendors
}
+5
View File
@@ -17,5 +17,10 @@ func candidateVendorsForRuby(p pkg.Package) fieldCandidateSet {
disallowSubSelections: true,
})
}
if metadata.Homepage != "" {
vendors.union(candidateVendorsFromURL(metadata.Homepage))
}
return vendors
}
@@ -0,0 +1,60 @@
package cpe
import (
"regexp"
"strings"
"github.com/anchore/syft/internal"
)
var (
urlPrefixToVendors = map[string][]string{
"https://www.gnu.org/": {"gnu"},
"https://developer.gnome.org/": {"gnome"},
"https://www.ruby-lang.org/": {"ruby-lang"},
"https://llvm.org/": {"llvm"},
"https://www.isc.org/": {"isc"},
"https://musl.libc.org/": {"musl-libc"},
"https://www.mozilla.org/": {"mozilla"},
"https://www.x.org/": {"x.org"},
"https://w1.fi/": {"w1.fi"},
}
vendorExtractionPatterns = []*regexp.Regexp{
regexp.MustCompile(`^(?:https|http|git)://(?:github|gitlab)\.com/(?P<vendor>[\w\-]*?)/.*$`),
}
)
func candidateVendorsFromURL(url string) fieldCandidateSet {
vendors := newFieldCandidateSet()
for urlPrefix, additionalVendors := range urlPrefixToVendors {
if strings.HasPrefix(url, urlPrefix) {
for _, v := range additionalVendors {
vendors.add(fieldCandidate{
value: v,
disallowSubSelections: true,
disallowDelimiterVariations: true,
})
return vendors
}
}
}
for _, p := range vendorExtractionPatterns {
groups := internal.MatchNamedCaptureGroups(p, url)
if v, ok := groups["vendor"]; ok {
vendors.add(fieldCandidate{
value: v,
disallowSubSelections: true,
disallowDelimiterVariations: true,
})
return vendors
}
}
return vendors
}
@@ -0,0 +1,71 @@
package cpe
import (
"testing"
"github.com/stretchr/testify/assert"
)
func Test_candidateVendorsFromURL(t *testing.T) {
tests := []struct {
name string
url string
expected []string
}{
{
name: "empty",
url: "",
expected: []string{},
},
{
name: "no known vendors",
url: "https://something-unknown.com/126374623876/12345",
expected: []string{},
},
{
name: "gnu vendor from url",
url: "https://www.gnu.org/software/make",
expected: []string{"gnu"},
},
{
name: "github username as vendor",
url: "https://github.com/armadillo/abcxyz-12345",
expected: []string{"armadillo"},
},
{
name: "github username with - as vendor",
url: "https://github.com/1234-abc-xyz/hello",
expected: []string{"1234-abc-xyz"},
},
{
name: "gitlab username as vendor",
url: "https://gitlab.com/armadillo/abcxyz-12345",
expected: []string{"armadillo"},
},
{
name: "gitlab username with - as vendor",
url: "https://gitlab.com/1234-abc-xyz/hello",
expected: []string{"1234-abc-xyz"},
},
{
name: "github username as vendor from longer url",
url: "https://github.com/armadillo/abcxyz-12345/a/b/c/d/e/f/g",
expected: []string{"armadillo"},
},
{
name: "github username from git://",
url: "git://github.com/abc/xyz.git",
expected: []string{"abc"},
},
{
name: "github username from http://",
url: "http://github.com/abc/xyz.git",
expected: []string{"abc"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.ElementsMatch(t, test.expected, candidateVendorsFromURL(test.url).uniqueValues(), "different vendors")
})
}
}
+11 -1
View File
@@ -45,7 +45,17 @@ func packageURL(m pkg.DotnetDepsMetadata) string {
var qualifiers packageurl.Qualifiers
return packageurl.NewPackageURL(
packageurl.TypeDotnet,
// This originally was packageurl.TypeDotnet, but this isn't a valid PURL type, according to:
// https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst
// Some history:
// https://github.com/anchore/packageurl-go/pull/8 added the type to Anchore's fork
// due to this PR: https://github.com/anchore/syft/pull/951
// There were questions about "dotnet" being the right purlType at the time, but it was
// acknowledged that scanning a dotnet file does not necessarily mean the packages found
// are nuget packages and so the alternate type was added. Since this is still an invalid
// PURL type, however, we will use TypeNuget and revisit at such time there is a better
// official PURL type available.
packageurl.TypeNuget,
"",
m.Name,
m.Version,
@@ -16,7 +16,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "AWSSDK.Core",
Version: "3.7.10.6",
PURL: "pkg:dotnet/AWSSDK.Core@3.7.10.6",
PURL: "pkg:nuget/AWSSDK.Core@3.7.10.6",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -32,7 +32,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "Microsoft.Extensions.DependencyInjection.Abstractions",
Version: "6.0.0",
PURL: "pkg:dotnet/Microsoft.Extensions.DependencyInjection.Abstractions@6.0.0",
PURL: "pkg:nuget/Microsoft.Extensions.DependencyInjection.Abstractions@6.0.0",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -48,7 +48,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "Microsoft.Extensions.DependencyInjection",
Version: "6.0.0",
PURL: "pkg:dotnet/Microsoft.Extensions.DependencyInjection@6.0.0",
PURL: "pkg:nuget/Microsoft.Extensions.DependencyInjection@6.0.0",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -64,7 +64,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "Microsoft.Extensions.Logging.Abstractions",
Version: "6.0.0",
PURL: "pkg:dotnet/Microsoft.Extensions.Logging.Abstractions@6.0.0",
PURL: "pkg:nuget/Microsoft.Extensions.Logging.Abstractions@6.0.0",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -80,7 +80,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "Microsoft.Extensions.Logging",
Version: "6.0.0",
PURL: "pkg:dotnet/Microsoft.Extensions.Logging@6.0.0",
PURL: "pkg:nuget/Microsoft.Extensions.Logging@6.0.0",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -97,7 +97,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "Microsoft.Extensions.Options",
Version: "6.0.0",
PURL: "pkg:dotnet/Microsoft.Extensions.Options@6.0.0",
PURL: "pkg:nuget/Microsoft.Extensions.Options@6.0.0",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -113,7 +113,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "Microsoft.Extensions.Primitives",
Version: "6.0.0",
PURL: "pkg:dotnet/Microsoft.Extensions.Primitives@6.0.0",
PURL: "pkg:nuget/Microsoft.Extensions.Primitives@6.0.0",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -129,7 +129,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "Newtonsoft.Json",
Version: "13.0.1",
PURL: "pkg:dotnet/Newtonsoft.Json@13.0.1",
PURL: "pkg:nuget/Newtonsoft.Json@13.0.1",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -145,7 +145,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "Serilog.Sinks.Console",
Version: "4.0.1",
PURL: "pkg:dotnet/Serilog.Sinks.Console@4.0.1",
PURL: "pkg:nuget/Serilog.Sinks.Console@4.0.1",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -161,7 +161,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "Serilog",
Version: "2.10.0",
PURL: "pkg:dotnet/Serilog@2.10.0",
PURL: "pkg:nuget/Serilog@2.10.0",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -177,7 +177,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "System.Diagnostics.DiagnosticSource",
Version: "6.0.0",
PURL: "pkg:dotnet/System.Diagnostics.DiagnosticSource@6.0.0",
PURL: "pkg:nuget/System.Diagnostics.DiagnosticSource@6.0.0",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
@@ -193,7 +193,7 @@ func TestParseDotnetDeps(t *testing.T) {
{
Name: "System.Runtime.CompilerServices.Unsafe",
Version: "6.0.0",
PURL: "pkg:dotnet/System.Runtime.CompilerServices.Unsafe@6.0.0",
PURL: "pkg:nuget/System.Runtime.CompilerServices.Unsafe@6.0.0",
Locations: fixtureLocationSet,
Language: pkg.Dotnet,
Type: pkg.DotnetPkg,
+235
View File
@@ -0,0 +1,235 @@
package erlang
import (
"bytes"
"fmt"
"io"
"strings"
"unicode"
)
type erlangNode struct {
value interface{}
}
func (e erlangNode) Slice() []erlangNode {
out, ok := e.value.([]erlangNode)
if ok {
return out
}
return []erlangNode{}
}
func (e erlangNode) String() string {
out, ok := e.value.(string)
if ok {
return out
}
return ""
}
func (e erlangNode) Get(index int) erlangNode {
s := e.Slice()
if len(s) > index {
return s[index]
}
return erlangNode{}
}
func node(value interface{}) erlangNode {
return erlangNode{
value: value,
}
}
// parseErlang basic parser for erlang, used by rebar.lock
func parseErlang(reader io.Reader) (erlangNode, error) {
data, err := io.ReadAll(reader)
if err != nil {
return node(nil), err
}
out := erlangNode{
value: []erlangNode{},
}
i := 0
for i < len(data) {
item, err := parseErlangBlock(data, &i)
if err != nil {
return node(nil), fmt.Errorf("%w\n%s", err, printError(data, i))
}
skipWhitespace(data, &i)
if i, ok := item.value.(string); ok && i == "." {
continue
}
out.value = append(out.value.([]erlangNode), item)
}
return out, nil
}
func printError(data []byte, i int) string {
line := 1
char := 1
prev := []string{}
curr := bytes.Buffer{}
for idx, c := range data {
if c == '\n' {
prev = append(prev, curr.String())
curr.Reset()
if idx >= i {
break
} else {
line++
}
char = 1
continue
}
if idx < i {
char++
}
curr.WriteByte(c)
}
l1 := fmt.Sprintf("%d", line-1)
l2 := fmt.Sprintf("%d", line)
if len(l1) < len(l2) {
l1 = " " + l1
}
sep := ": "
lines := ""
if len(prev) > 1 {
lines += fmt.Sprintf("%s%s%s\n", l1, sep, prev[len(prev)-2])
}
if len(prev) > 0 {
lines += fmt.Sprintf("%s%s%s\n", l2, sep, prev[len(prev)-1])
}
pointer := strings.Repeat(" ", len(l2)+len(sep)+char-1) + "^"
return fmt.Sprintf("line: %v, char: %v\n%s%s", line, char, lines, pointer)
}
func skipWhitespace(data []byte, i *int) {
for *i < len(data) && isWhitespace(data[*i]) {
*i++
}
}
func parseErlangBlock(data []byte, i *int) (erlangNode, error) {
block, err := parseErlangNode(data, i)
if err != nil {
return node(nil), err
}
skipWhitespace(data, i)
*i++ // skip the trailing .
return block, nil
}
func parseErlangNode(data []byte, i *int) (erlangNode, error) {
skipWhitespace(data, i)
c := data[*i]
switch c {
case '[', '{':
return parseErlangList(data, i)
case '"':
return parseErlangString(data, i)
case '<':
return parseErlangAngleString(data, i)
}
if isLiteral(c) {
return parseErlangLiteral(data, i)
}
return erlangNode{}, fmt.Errorf("invalid literal character: %s", string(c))
}
func isWhitespace(c byte) bool {
return unicode.IsSpace(rune(c))
}
func isLiteral(c byte) bool {
r := rune(c)
return unicode.IsNumber(r) || unicode.IsLetter(r) || r == '.' || r == '_'
}
func parseErlangLiteral(data []byte, i *int) (erlangNode, error) {
var buf bytes.Buffer
for *i < len(data) {
c := data[*i]
if isLiteral(c) {
buf.WriteByte(c)
} else {
break
}
*i++
}
return node(buf.String()), nil
}
func parseErlangAngleString(data []byte, i *int) (erlangNode, error) {
*i += 2
out, err := parseErlangString(data, i)
*i += 2
return out, err
}
func parseErlangString(data []byte, i *int) (erlangNode, error) {
delim := data[*i]
*i++
var buf bytes.Buffer
for *i < len(data) {
c := data[*i]
if c == delim {
*i++
return node(buf.String()), nil
}
if c == '\\' {
*i++
if len(data) >= *i {
return node(nil), fmt.Errorf("invalid escape without closed string at %d", *i)
}
c = data[*i]
}
buf.WriteByte(c)
*i++
}
return node(buf.String()), nil
}
func parseErlangList(data []byte, i *int) (erlangNode, error) {
*i++
out := erlangNode{
value: []erlangNode{},
}
for *i < len(data) {
item, err := parseErlangNode(data, i)
if err != nil {
return node(nil), err
}
out.value = append(out.value.([]erlangNode), item)
skipWhitespace(data, i)
c := data[*i]
switch c {
case ',':
*i++
continue
case ']', '}':
*i++
return out, nil
default:
return node(nil), fmt.Errorf("unexpected character: %s", string(c))
}
}
return out, nil
}
@@ -0,0 +1,72 @@
package erlang
import (
"bytes"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_parseErlang(t *testing.T) {
tests := []struct {
name string
content string
wantErr require.ErrorAssertionFunc
}{
{
name: "basic valid content",
content: `
{"1.2.0",
[{<<"bcrypt">>,{pkg,<<"bcrypt">>,<<"1.1.5">>},0},
{<<"bson">>,
{git,"https://github.com/comtihon/bson-erlang",
{ref,"14308ab927cfa69324742c3de720578094e0bb19"}},
1},
{<<"syslog">>,{pkg,<<"syslog">>,<<"1.1.0">>},0},
{<<"unicode_util_compat">>,{pkg,<<"unicode_util_compat">>,<<"0.7.0">>},1},
{<<"vernemq_dev">>,
{git,"https://github.com/vernemq/vernemq_dev.git",
{ref,"6d622aa8c901ae7777433aef2bd049e380c474a6"}},
0}]
}.
[
{pkg_hash,[
{<<"bcrypt">>, <<"A6763BD4E1AF46D34776F85B7995E63A02978DE110C077E9570ED17006E03386">>},
{<<"unicode_util_compat">>, <<"BC84380C9AB48177092F43AC89E4DFA2C6D62B40B8BD132B1059ECC7232F9A78">>}]},
{pkg_hash_ext,[
{<<"bcrypt">>, <<"3418821BC17CE6E96A4A77D1A88D7485BF783E212069FACFC79510AFBFF95352">>},
{<<"unicode_util_compat">>, <<"25EEE6D67DF61960CF6A794239566599B09E17E668D3700247BC498638152521">>}]}
].`,
},
{
name: "invalid string content",
wantErr: require.Error,
content: `
{"1.2.0
">>},
].`,
},
{
name: "invalid content",
wantErr: require.Error,
content: `
{"1.2.0"}.
].`,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
value, err := parseErlang(bytes.NewReader([]byte(test.content)))
if test.wantErr == nil {
require.NoError(t, err)
} else {
test.wantErr(t, err)
}
assert.IsType(t, erlangNode{}, value)
})
}
}
+66 -55
View File
@@ -1,12 +1,6 @@
package erlang
import (
"bufio"
"errors"
"fmt"
"io"
"regexp"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
@@ -14,57 +8,44 @@ import (
"github.com/anchore/syft/syft/source"
)
// integrity check
var _ generic.Parser = parseRebarLock
var rebarLockDelimiter = regexp.MustCompile(`[\[{<">},: \]\n]+`)
// parseMixLock parses a mix.lock and returns the discovered Elixir packages.
// parseRebarLock parses a rebar.lock and returns the discovered Elixir packages.
//
//nolint:funlen
func parseRebarLock(_ source.FileResolver, _ *generic.Environment, reader source.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
r := bufio.NewReader(reader)
doc, err := parseErlang(reader)
if err != nil {
return nil, nil, err
}
pkgMap := make(map[string]*pkg.Package)
var names []string
loop:
for {
line, err := r.ReadString('\n')
switch {
case errors.Is(io.EOF, err):
break loop
case err != nil:
// TODO: return partial result and warn
return nil, nil, fmt.Errorf("failed to parse rebar.lock file: %w", err)
}
tokens := rebarLockDelimiter.Split(line, -1)
if len(tokens) < 4 {
continue
}
if len(tokens) < 5 {
name, hash := tokens[1], tokens[2]
sourcePkg := pkgMap[name]
metadata, ok := sourcePkg.Metadata.(pkg.RebarLockMetadata)
if !ok {
log.WithFields("package", name).Warn("unable to extract rebar.lock metadata to add hash metadata")
continue
}
// rebar.lock structure is:
// [
// ["version", [
// [<<"package-name">>, ["version-type", "version"]...
// ],
// [
// [pkg_hash, [
// [<<"package-name">>, <<"package-hash">>]
// ],
// [pkg_hash_ext, [
// [<<"package-name">>, <<"package-hash">>]
// ]
// ]
// ]
if metadata.PkgHash == "" {
metadata.PkgHash = hash
} else {
metadata.PkgHashExt = hash
}
sourcePkg.Metadata = metadata
continue
}
name, version := tokens[1], tokens[4]
versions := doc.Get(0)
deps := versions.Get(1)
sourcePkg := pkg.Package{
Name: name,
Version: version,
Language: pkg.Erlang,
Type: pkg.HexPkg,
MetadataType: pkg.RebarLockMetadataType,
for _, dep := range deps.Slice() {
name := dep.Get(0).String()
versionNode := dep.Get(1)
versionType := versionNode.Get(0).String()
version := versionNode.Get(2).String()
// capture git hashes if no version specified
if versionType == "git" {
version = versionNode.Get(2).Get(1).String()
}
p := newPackage(pkg.RebarLockMetadata{
@@ -72,15 +53,45 @@ loop:
Version: version,
})
names = append(names, name)
pkgMap[sourcePkg.Name] = &p
pkgMap[name] = &p
}
hashes := doc.Get(1)
for _, hashStruct := range hashes.Slice() {
hashType := hashStruct.Get(0).String()
for _, hashValue := range hashStruct.Get(1).Slice() {
name := hashValue.Get(0).String()
hash := hashValue.Get(1).String()
sourcePkg := pkgMap[name]
if sourcePkg == nil {
log.WithFields("package", name).Warn("unable find source package")
continue
}
metadata, ok := sourcePkg.Metadata.(pkg.RebarLockMetadata)
if !ok {
log.WithFields("package", name).Warn("unable to extract rebar.lock metadata to add hash metadata")
continue
}
switch hashType {
case "pkg_hash":
metadata.PkgHash = hash
case "pkg_hash_ext":
metadata.PkgHashExt = hash
}
sourcePkg.Metadata = metadata
}
}
var packages []pkg.Package
for _, name := range names {
p := pkgMap[name]
for _, p := range pkgMap {
p.SetID()
packages = append(packages, *p)
}
return packages, nil, nil
}
// integrity check
var _ generic.Parser = parseRebarLock
@@ -9,111 +9,259 @@ import (
)
func TestParseRebarLock(t *testing.T) {
expected := []pkg.Package{
tests := []struct {
fixture string
expected []pkg.Package
}{
{
Name: "certifi",
Version: "2.9.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/certifi@2.9.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "certifi",
Version: "2.9.0",
PkgHash: "6F2A475689DD47F19FB74334859D460A2DC4E3252A3324BD2111B8F0429E7E21",
PkgHashExt: "266DA46BDB06D6C6D35FDE799BCB28D36D985D424AD7C08B5BB48F5B5CDD4641",
fixture: "test-fixtures/rebar.lock",
expected: []pkg.Package{
{
Name: "certifi",
Version: "2.9.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/certifi@2.9.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "certifi",
Version: "2.9.0",
PkgHash: "6F2A475689DD47F19FB74334859D460A2DC4E3252A3324BD2111B8F0429E7E21",
PkgHashExt: "266DA46BDB06D6C6D35FDE799BCB28D36D985D424AD7C08B5BB48F5B5CDD4641",
},
},
{
Name: "idna",
Version: "6.1.1",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/idna@6.1.1",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "idna",
Version: "6.1.1",
PkgHash: "8A63070E9F7D0C62EB9D9FCB360A7DE382448200FBBD1B106CC96D3D8099DF8D",
PkgHashExt: "92376EB7894412ED19AC475E4A86F7B413C1B9FBB5BD16DCCD57934157944CEA",
},
},
{
Name: "metrics",
Version: "1.0.1",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/metrics@1.0.1",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "metrics",
Version: "1.0.1",
PkgHash: "25F094DEA2CDA98213CECC3AEFF09E940299D950904393B2A29D191C346A8486",
PkgHashExt: "69B09ADDDC4F74A40716AE54D140F93BEB0FB8978D8636EADED0C31B6F099F16",
},
},
{
Name: "mimerl",
Version: "1.2.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/mimerl@1.2.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "mimerl",
Version: "1.2.0",
PkgHash: "67E2D3F571088D5CFD3E550C383094B47159F3EEE8FFA08E64106CDF5E981BE3",
PkgHashExt: "F278585650AA581986264638EBF698F8BB19DF297F66AD91B18910DFC6E19323",
},
},
{
Name: "parse_trans",
Version: "3.3.1",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/parse_trans@3.3.1",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "parse_trans",
Version: "3.3.1",
PkgHash: "16328AB840CC09919BD10DAB29E431DA3AF9E9E7E7E6F0089DD5A2D2820011D8",
PkgHashExt: "07CD9577885F56362D414E8C4C4E6BDF10D43A8767ABB92D24CBE8B24C54888B",
},
},
{
Name: "ssl_verify_fun",
Version: "1.1.6",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/ssl_verify_fun@1.1.6",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "ssl_verify_fun",
Version: "1.1.6",
PkgHash: "CF344F5692C82D2CD7554F5EC8FD961548D4FD09E7D22F5B62482E5AEAEBD4B0",
PkgHashExt: "BDB0D2471F453C88FF3908E7686F86F9BE327D065CC1EC16FA4540197EA04680",
},
},
{
Name: "unicode_util_compat",
Version: "0.7.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/unicode_util_compat@0.7.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "unicode_util_compat",
Version: "0.7.0",
PkgHash: "BC84380C9AB48177092F43AC89E4DFA2C6D62B40B8BD132B1059ECC7232F9A78",
PkgHashExt: "25EEE6D67DF61960CF6A794239566599B09E17E668D3700247BC498638152521",
},
},
},
},
{
Name: "idna",
Version: "6.1.1",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/idna@6.1.1",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "idna",
Version: "6.1.1",
PkgHash: "8A63070E9F7D0C62EB9D9FCB360A7DE382448200FBBD1B106CC96D3D8099DF8D",
PkgHashExt: "92376EB7894412ED19AC475E4A86F7B413C1B9FBB5BD16DCCD57934157944CEA",
},
},
{
Name: "metrics",
Version: "1.0.1",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/metrics@1.0.1",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "metrics",
Version: "1.0.1",
PkgHash: "25F094DEA2CDA98213CECC3AEFF09E940299D950904393B2A29D191C346A8486",
PkgHashExt: "69B09ADDDC4F74A40716AE54D140F93BEB0FB8978D8636EADED0C31B6F099F16",
},
},
{
Name: "mimerl",
Version: "1.2.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/mimerl@1.2.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "mimerl",
Version: "1.2.0",
PkgHash: "67E2D3F571088D5CFD3E550C383094B47159F3EEE8FFA08E64106CDF5E981BE3",
PkgHashExt: "F278585650AA581986264638EBF698F8BB19DF297F66AD91B18910DFC6E19323",
},
},
{
Name: "parse_trans",
Version: "3.3.1",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/parse_trans@3.3.1",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "parse_trans",
Version: "3.3.1",
PkgHash: "16328AB840CC09919BD10DAB29E431DA3AF9E9E7E7E6F0089DD5A2D2820011D8",
PkgHashExt: "07CD9577885F56362D414E8C4C4E6BDF10D43A8767ABB92D24CBE8B24C54888B",
},
},
{
Name: "ssl_verify_fun",
Version: "1.1.6",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/ssl_verify_fun@1.1.6",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "ssl_verify_fun",
Version: "1.1.6",
PkgHash: "CF344F5692C82D2CD7554F5EC8FD961548D4FD09E7D22F5B62482E5AEAEBD4B0",
PkgHashExt: "BDB0D2471F453C88FF3908E7686F86F9BE327D065CC1EC16FA4540197EA04680",
},
},
{
Name: "unicode_util_compat",
Version: "0.7.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/unicode_util_compat@0.7.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "unicode_util_compat",
Version: "0.7.0",
PkgHash: "BC84380C9AB48177092F43AC89E4DFA2C6D62B40B8BD132B1059ECC7232F9A78",
PkgHashExt: "25EEE6D67DF61960CF6A794239566599B09E17E668D3700247BC498638152521",
fixture: "test-fixtures/rebar-2.lock",
expected: []pkg.Package{
//[{<<"bcrypt">>,{pkg,<<"bcrypt">>,<<"1.1.5">>},0},
// {<<"bcrypt">>, <<"A6763BD4E1AF46D34776F85B7995E63A02978DE110C077E9570ED17006E03386">>},
// {<<"bcrypt">>, <<"3418821BC17CE6E96A4A77D1A88D7485BF783E212069FACFC79510AFBFF95352">>},
{
Name: "bcrypt",
Version: "1.1.5",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/bcrypt@1.1.5",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "bcrypt",
Version: "1.1.5",
PkgHash: "A6763BD4E1AF46D34776F85B7995E63A02978DE110C077E9570ED17006E03386",
PkgHashExt: "3418821BC17CE6E96A4A77D1A88D7485BF783E212069FACFC79510AFBFF95352",
},
},
// {<<"bson">>,
// {git,"https://github.com/comtihon/bson-erlang",
// {ref,"14308ab927cfa69324742c3de720578094e0bb19"}},
// 1},
{
Name: "bson",
Version: "14308ab927cfa69324742c3de720578094e0bb19",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/bson@14308ab927cfa69324742c3de720578094e0bb19",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "bson",
Version: "14308ab927cfa69324742c3de720578094e0bb19",
},
},
// {<<"certifi">>,{pkg,<<"certifi">>,<<"2.9.0">>},1},
// {<<"certifi">>, <<"6F2A475689DD47F19FB74334859D460A2DC4E3252A3324BD2111B8F0429E7E21">>}, {<<"stdout_formatter">>, <<"EC24868D8619757A68F0798357C7190807A1CFC42CE90C18C23760E59249A21A">>},
// {<<"certifi">>, <<"266DA46BDB06D6C6D35FDE799BCB28D36D985D424AD7C08B5BB48F5B5CDD4641">>},
{
Name: "certifi",
Version: "2.9.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/certifi@2.9.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "certifi",
Version: "2.9.0",
PkgHash: "6F2A475689DD47F19FB74334859D460A2DC4E3252A3324BD2111B8F0429E7E21",
PkgHashExt: "266DA46BDB06D6C6D35FDE799BCB28D36D985D424AD7C08B5BB48F5B5CDD4641",
},
},
// {<<"stdout_formatter">>,{pkg,<<"stdout_formatter">>,<<"0.2.3">>},0},
// {<<"stdout_formatter">>, <<"EC24868D8619757A68F0798357C7190807A1CFC42CE90C18C23760E59249A21A">>},
// {<<"stdout_formatter">>, <<"6B9CAAD8930006F9BB35680C5D3311917AC67690C3AF1BA018623324C015ABE5">>},
{
Name: "stdout_formatter",
Version: "0.2.3",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/stdout_formatter@0.2.3",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "stdout_formatter",
Version: "0.2.3",
PkgHash: "EC24868D8619757A68F0798357C7190807A1CFC42CE90C18C23760E59249A21A",
PkgHashExt: "6B9CAAD8930006F9BB35680C5D3311917AC67690C3AF1BA018623324C015ABE5",
},
},
// {<<"swc">>,
// {git,"https://github.com/vernemq/ServerWideClocks.git",
// {ref,"4835239dca5a5f4ac7202dd94d7effcaa617d575"}},
// 0},
{
Name: "swc",
Version: "4835239dca5a5f4ac7202dd94d7effcaa617d575",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/swc@4835239dca5a5f4ac7202dd94d7effcaa617d575",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "swc",
Version: "4835239dca5a5f4ac7202dd94d7effcaa617d575",
},
},
// {<<"syslog">>,{pkg,<<"syslog">>,<<"1.1.0">>},0},
// {<<"syslog">>, <<"6419A232BEA84F07B56DC575225007FFE34D9FDC91ABE6F1B2F254FD71D8EFC2">>},
// {<<"syslog">>, <<"4C6A41373C7E20587BE33EF841D3DE6F3BEBA08519809329ECC4D27B15B659E1">>},
{
Name: "syslog",
Version: "1.1.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/syslog@1.1.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "syslog",
Version: "1.1.0",
PkgHash: "6419A232BEA84F07B56DC575225007FFE34D9FDC91ABE6F1B2F254FD71D8EFC2",
PkgHashExt: "4C6A41373C7E20587BE33EF841D3DE6F3BEBA08519809329ECC4D27B15B659E1",
},
},
// {<<"unicode_util_compat">>,{pkg,<<"unicode_util_compat">>,<<"0.7.0">>},1},
// {<<"unicode_util_compat">>, <<"BC84380C9AB48177092F43AC89E4DFA2C6D62B40B8BD132B1059ECC7232F9A78">>}]},
// {<<"unicode_util_compat">>, <<"25EEE6D67DF61960CF6A794239566599B09E17E668D3700247BC498638152521">>}]}
{
Name: "unicode_util_compat",
Version: "0.7.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/unicode_util_compat@0.7.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "unicode_util_compat",
Version: "0.7.0",
PkgHash: "BC84380C9AB48177092F43AC89E4DFA2C6D62B40B8BD132B1059ECC7232F9A78",
PkgHashExt: "25EEE6D67DF61960CF6A794239566599B09E17E668D3700247BC498638152521",
},
},
// {<<"vernemq_dev">>,
// {git,"https://github.com/vernemq/vernemq_dev.git",
// {ref,"6d622aa8c901ae7777433aef2bd049e380c474a6"}},
// 0}]}.
{
Name: "vernemq_dev",
Version: "6d622aa8c901ae7777433aef2bd049e380c474a6",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/vernemq_dev@6d622aa8c901ae7777433aef2bd049e380c474a6",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "vernemq_dev",
Version: "6d622aa8c901ae7777433aef2bd049e380c474a6",
},
},
},
},
}
fixture := "test-fixtures/rebar.lock"
for _, test := range tests {
t.Run(test.fixture, func(t *testing.T) {
// TODO: relationships are not under test
var expectedRelationships []artifact.Relationship
// TODO: relationships are not under test
var expectedRelationships []artifact.Relationship
pkgtest.TestFileParser(t, fixture, parseRebarLock, expected, expectedRelationships)
pkgtest.TestFileParser(t, test.fixture, parseRebarLock, test.expected, expectedRelationships)
})
}
}
@@ -0,0 +1,32 @@
{"1.2.0",
[{<<"bcrypt">>,{pkg,<<"bcrypt">>,<<"1.1.5">>},0},
{<<"bson">>,
{git,"https://github.com/comtihon/bson-erlang",
{ref,"14308ab927cfa69324742c3de720578094e0bb19"}},
1},
{<<"certifi">>,{pkg,<<"certifi">>,<<"2.9.0">>},1},
{<<"stdout_formatter">>,{pkg,<<"stdout_formatter">>,<<"0.2.3">>},0},
{<<"swc">>,
{git,"https://github.com/vernemq/ServerWideClocks.git",
{ref,"4835239dca5a5f4ac7202dd94d7effcaa617d575"}},
0},
{<<"syslog">>,{pkg,<<"syslog">>,<<"1.1.0">>},0},
{<<"unicode_util_compat">>,{pkg,<<"unicode_util_compat">>,<<"0.7.0">>},1},
{<<"vernemq_dev">>,
{git,"https://github.com/vernemq/vernemq_dev.git",
{ref,"6d622aa8c901ae7777433aef2bd049e380c474a6"}},
0}]}.
[
{pkg_hash,[
{<<"bcrypt">>, <<"A6763BD4E1AF46D34776F85B7995E63A02978DE110C077E9570ED17006E03386">>},
{<<"certifi">>, <<"6F2A475689DD47F19FB74334859D460A2DC4E3252A3324BD2111B8F0429E7E21">>},
{<<"stdout_formatter">>, <<"EC24868D8619757A68F0798357C7190807A1CFC42CE90C18C23760E59249A21A">>},
{<<"syslog">>, <<"6419A232BEA84F07B56DC575225007FFE34D9FDC91ABE6F1B2F254FD71D8EFC2">>},
{<<"unicode_util_compat">>, <<"BC84380C9AB48177092F43AC89E4DFA2C6D62B40B8BD132B1059ECC7232F9A78">>}]},
{pkg_hash_ext,[
{<<"bcrypt">>, <<"3418821BC17CE6E96A4A77D1A88D7485BF783E212069FACFC79510AFBFF95352">>},
{<<"certifi">>, <<"266DA46BDB06D6C6D35FDE799BCB28D36D985D424AD7C08B5BB48F5B5CDD4641">>},
{<<"stdout_formatter">>, <<"6B9CAAD8930006F9BB35680C5D3311917AC67690C3AF1BA018623324C015ABE5">>},
{<<"syslog">>, <<"4C6A41373C7E20587BE33EF841D3DE6F3BEBA08519809329ECC4D27B15B659E1">>},
{<<"unicode_util_compat">>, <<"25EEE6D67DF61960CF6A794239566599B09E17E668D3700247BC498638152521">>}]}
].
+11 -3
View File
@@ -179,12 +179,23 @@ func getBuildSettings(settings []debug.BuildSetting) map[string]string {
return m
}
func createMainModuleFromPath(path string) (mod debug.Module) {
mod.Path = path
mod.Version = devel
return
}
func buildGoPkgInfo(location source.Location, mod *debug.BuildInfo, arch string) []pkg.Package {
var pkgs []pkg.Package
if mod == nil {
return pkgs
}
var empty debug.Module
if mod.Main == empty && mod.Path != "" {
mod.Main = createMainModuleFromPath(mod.Path)
}
for _, dep := range mod.Deps {
if dep == nil {
continue
@@ -195,9 +206,6 @@ func buildGoPkgInfo(location source.Location, mod *debug.BuildInfo, arch string)
}
}
// NOTE(jonasagx): this use happened originally while creating unit tests. It might never
// happen in the wild, but I kept it as a safeguard against empty modules.
var empty debug.Module
if mod.Main == empty {
return pkgs
}
@@ -242,6 +242,48 @@ func TestBuildGoPkgInfo(t *testing.T) {
},
},
},
{
name: "parse a mod with path but no main module",
arch: archDetails,
mod: &debug.BuildInfo{
GoVersion: goCompiledVersion,
Settings: []debug.BuildSetting{
{Key: "GOARCH", Value: archDetails},
{Key: "GOOS", Value: "darwin"},
{Key: "GOAMD64", Value: "v1"},
},
Path: "github.com/a/b/c",
},
expected: []pkg.Package{
{
Name: "github.com/a/b/c",
Version: "(devel)",
PURL: "pkg:golang/github.com/a/b/c@(devel)",
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: source.NewLocationSet(
source.Location{
Coordinates: source.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
},
),
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "",
BuildSettings: map[string]string{
"GOAMD64": "v1",
"GOARCH": "amd64",
"GOOS": "darwin",
},
MainModule: "github.com/a/b/c",
},
},
},
},
{
name: "parse a mod without packages",
arch: archDetails,
+1 -3
View File
@@ -347,9 +347,7 @@ func requireFileInfo(_ string, info os.FileInfo, _ error) error {
func indexingProgress(path string) (*progress.Stage, *progress.Manual) {
stage := &progress.Stage{}
prog := &progress.Manual{
Total: -1,
}
prog := progress.NewManual(-1)
bus.Publish(partybus.Event{
Type: event.FileIndexingStarted,
+2 -2
View File
@@ -13,8 +13,8 @@ type Location struct {
Coordinates `cyclonedx:""` // Empty string here means there is no intermediate property name, e.g. syft:locations:0:path without "coordinates"
// note: it is IMPORTANT to ignore anything but the coordinates for a Location when considering the ID (hash value)
// since the coordinates are the minimally correct ID for a location (symlinks should not come into play)
VirtualPath string `hash:"ignore"` // The path to the file which may or may not have hardlinks / symlinks
ref file.Reference `hash:"ignore"` // The file reference relative to the stereoscope.FileCatalog that has more information about this location.
VirtualPath string `hash:"ignore" json:"virtualPath,omitempty"` // The path to the file which may or may not have hardlinks / symlinks
ref file.Reference `hash:"ignore"` // The file reference relative to the stereoscope.FileCatalog that has more information about this location.
}
// NewLocation creates a new Location representing a path without denoting a filesystem or FileCatalog reference.

Some files were not shown because too many files have changed in this diff Show More