Compare commits

...
35 Commits
Author SHA1 Message Date
Dan LuhringandAlex Goodman e58050bac0 Fix panic in apkdb parsing on empty "provides" values (#1494)
* Add failing test for strip version specifiers panic

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Fix test

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Prevent panic scenario in helper func

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Fix lint issue

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* add tests for apk stripVersionSpecifier() and remove caller empty value check

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-20 14:49:44 +00:00
Alex Goodman 36a0945c95 push detailed log statements to trace-level (#1500)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-20 14:33:23 +00:00
mikcl 396441e921 npm: package-lock license decoding to accept string or array (#1482)
Signed-off-by: mikcl <mikesmikes400@gmail.com>
2023-01-20 09:28:51 -05:00
Alex Goodman 972e4cdaeb always set the package ID for java packages (#1493)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-20 09:18:00 -05:00
Nils Hanke 99f55f6a81 fix: skip filling in empty fields in APK metadata (#1484)
Signed-off-by: Nils Hanke <nils.hanke@outlook.de>

Signed-off-by: Nils Hanke <nils.hanke@outlook.de>
2023-01-20 14:03:30 +00:00
dependabot[bot] 285112fe29 chore(deps): bump github.com/facebookincubator/nvdtools (#1499)
Bumps [github.com/facebookincubator/nvdtools](https://github.com/facebookincubator/nvdtools) from 0.1.4 to 0.1.5.
- [Release notes](https://github.com/facebookincubator/nvdtools/releases)
- [Commits](https://github.com/facebookincubator/nvdtools/compare/v0.1.4...v0.1.5)

---
updated-dependencies:
- dependency-name: github.com/facebookincubator/nvdtools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-20 14:02:47 +00:00
dependabot[bot] f29bea5921 chore(deps): bump github.com/jinzhu/copier from 0.3.2 to 0.3.5 (#1498)
Bumps [github.com/jinzhu/copier](https://github.com/jinzhu/copier) from 0.3.2 to 0.3.5.
- [Release notes](https://github.com/jinzhu/copier/releases)
- [Commits](https://github.com/jinzhu/copier/compare/v0.3.2...v0.3.5)

---
updated-dependencies:
- dependency-name: github.com/jinzhu/copier
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-20 08:51:20 -05:00
dependabot[bot] 39cdbc42aa chore(deps): bump github.com/vbatts/go-mtree from 0.5.0 to 0.5.2 (#1497)
Bumps [github.com/vbatts/go-mtree](https://github.com/vbatts/go-mtree) from 0.5.0 to 0.5.2.
- [Release notes](https://github.com/vbatts/go-mtree/releases)
- [Changelog](https://github.com/vbatts/go-mtree/blob/main/releases.md)
- [Commits](https://github.com/vbatts/go-mtree/compare/v0.5.0...v0.5.2)

---
updated-dependencies:
- dependency-name: github.com/vbatts/go-mtree
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-20 08:50:59 -05:00
dependabot[bot] 27b62ce833 chore(deps): bump github.com/gookit/color from 1.4.2 to 1.5.2 (#1496)
Bumps [github.com/gookit/color](https://github.com/gookit/color) from 1.4.2 to 1.5.2.
- [Release notes](https://github.com/gookit/color/releases)
- [Commits](https://github.com/gookit/color/compare/v1.4.2...v1.5.2)

---
updated-dependencies:
- dependency-name: github.com/gookit/color
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-20 08:50:37 -05:00
dependabot[bot] 499e7c4e16 chore(deps): bump github.com/spf13/viper from 1.14.0 to 1.15.0 (#1495)
Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/spf13/viper/releases)
- [Commits](https://github.com/spf13/viper/compare/v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/viper
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-20 08:50:19 -05:00
Alex GoodmanandChristopher Phillips 0f75f975c8 Relax error conditions for catalogers (#1492)
* binary cataloger should continue on errors

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* test: add redirect for cmd stderr stdout

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

* test: image update for test failure

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-01-19 19:28:42 -05:00
witchcraze 7427445fe9 feat: add memcached classifier (#1486)
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-01-19 11:22:11 -05:00
dependabot[bot] 09a5baf523 chore(deps): bump github.com/spf13/viper from 1.13.0 to 1.14.0 (#1488)
Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.13.0 to 1.14.0.
- [Release notes](https://github.com/spf13/viper/releases)
- [Commits](https://github.com/spf13/viper/compare/v1.13.0...v1.14.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/viper
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-19 10:39:04 -05:00
dependabot[bot] 33c08c8545 chore(deps): bump github.com/bmatcuk/doublestar/v4 from 4.0.2 to 4.6.0 (#1489)
Bumps [github.com/bmatcuk/doublestar/v4](https://github.com/bmatcuk/doublestar) from 4.0.2 to 4.6.0.
- [Release notes](https://github.com/bmatcuk/doublestar/releases)
- [Commits](https://github.com/bmatcuk/doublestar/compare/v4.0.2...v4.6.0)

---
updated-dependencies:
- dependency-name: github.com/bmatcuk/doublestar/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-19 10:38:50 -05:00
dependabot[bot] fd002db802 chore(deps): bump github.com/spf13/cobra from 1.6.0 to 1.6.1 (#1490)
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.6.0 to 1.6.1.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.6.0...v1.6.1)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-19 14:16:50 +00:00
dependabot[bot] cb3e4b8e49 chore(deps): bump github.com/go-test/deep from 1.0.8 to 1.1.0 (#1491)
Bumps [github.com/go-test/deep](https://github.com/go-test/deep) from 1.0.8 to 1.1.0.
- [Release notes](https://github.com/go-test/deep/releases)
- [Changelog](https://github.com/go-test/deep/blob/master/CHANGES.md)
- [Commits](https://github.com/go-test/deep/compare/v1.0.8...v1.1.0)

---
updated-dependencies:
- dependency-name: github.com/go-test/deep
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-19 14:01:33 +00:00
dependabot[bot] 5917f8d8f9 chore(deps): bump github.com/google/go-containerregistry (#1487)
Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.11.0 to 0.12.1.
- [Release notes](https://github.com/google/go-containerregistry/releases)
- [Changelog](https://github.com/google/go-containerregistry/blob/main/.goreleaser.yml)
- [Commits](https://github.com/google/go-containerregistry/compare/v0.11.0...v0.12.1)

---
updated-dependencies:
- dependency-name: github.com/google/go-containerregistry
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-19 13:47:36 +00:00
dependabot[bot] 70e6d0f2e3 chore(deps): bump golang.org/x/net from 0.4.0 to 0.5.0 (#1475)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 14:39:50 +00:00
dependabot[bot] 31a763c46d chore(deps): bump github.com/adrg/xdg from 0.3.3 to 0.4.0 (#1477)
Bumps [github.com/adrg/xdg](https://github.com/adrg/xdg) from 0.3.3 to 0.4.0.
- [Release notes](https://github.com/adrg/xdg/releases)
- [Commits](https://github.com/adrg/xdg/compare/v0.3.3...v0.4.0)

---
updated-dependencies:
- dependency-name: github.com/adrg/xdg
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 09:39:35 -05:00
dependabot[bot] ae6c9c2e97 chore(deps): bump github.com/sergi/go-diff from 1.2.0 to 1.3.1 (#1476)
Bumps [github.com/sergi/go-diff](https://github.com/sergi/go-diff) from 1.2.0 to 1.3.1.
- [Release notes](https://github.com/sergi/go-diff/releases)
- [Commits](https://github.com/sergi/go-diff/compare/v1.2.0...v1.3.1)

---
updated-dependencies:
- dependency-name: github.com/sergi/go-diff
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 09:39:15 -05:00
dependabot[bot] f6a0dd33d1 chore(deps): bump github.com/vifraa/gopom from 0.1.0 to 0.2.1 (#1474)
Bumps [github.com/vifraa/gopom](https://github.com/vifraa/gopom) from 0.1.0 to 0.2.1.
- [Release notes](https://github.com/vifraa/gopom/releases)
- [Commits](https://github.com/vifraa/gopom/compare/v0.1.0...v0.2.1)

---
updated-dependencies:
- dependency-name: github.com/vifraa/gopom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 09:38:30 -05:00
dependabot[bot] b77c104aa6 chore(deps): bump github/codeql-action from 1 to 2 (#1473)
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 1 to 2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 09:38:06 -05:00
dependabot[bot] 10ca7f56ab chore(deps): bump actions/setup-go from 2 to 3 (#1472)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 2 to 3.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 09:37:45 -05:00
Luca Comellini 6b2dc08ffb Add dependabot (#1451)
Signed-off-by: Luca Comellini <luca.com@gmail.com>

Signed-off-by: Luca Comellini <luca.com@gmail.com>
2023-01-18 09:29:24 -05:00
Christopher Angelo Phillips 03971ace43 chore: use checkout v3 with new depth (#1471) 2023-01-17 21:26:39 +00:00
Christopher Angelo Phillips 07aee798b0 chore: use checkout v2 for tag depth (#1470) 2023-01-17 21:03:29 +00:00
Keith ZantowandAlex Goodman 6cf668f749 fix: nil panic in graalvm cataloger (#1468)
* normalize error handling and recover from panics while parsing binaries
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-17 19:06:24 +00:00
Alex Goodman 2ec4371c95 add linter for type assertion checks (#1469)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-17 14:00:03 -05:00
Weston Steimel fc4d28f365 fix: bump golang.org/x/net to v0.4.0 (#1467)
resolves reporting of CVE-2022-41717

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-01-17 17:02:34 +00:00
Weston Steimel 5290dfb9c2 fix: bump golang.org/x/text to v0.3.8 (#1466)
This resolves reporting of GHSA-69ch-w2m2-3vjp

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-01-17 15:50:02 +00:00
Alex Goodman 05611c283d bootstrap within composite action (#1461)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-17 10:04:22 -05:00
Keith Zantow 934644232a chore: revert GolangBinMetadata name and make analogous GolangModMetadata (#1458) 2023-01-13 16:46:12 -05:00
Florian Klink 641bccc79b README: update Nix installation instructions (#1455)
22.05 has been released, update the instructions.

Signed-off-by: Florian Klink <flokli@flokli.de>
2023-01-13 15:43:25 +00:00
Keith Zantow ac94bf530c fix: update graalvm cataloger to fix panic (#1454)
Fixes https://github.com/anchore/syft/issues/1453
2023-01-12 17:42:13 -05:00
Weston Steimel e87cfe7319 chore: remove bumping cosign in go.mod when updating bootstrap tools (#1452) 2023-01-12 16:21:01 -05:00
66 changed files with 3098 additions and 1298 deletions
+81
View File
@@ -0,0 +1,81 @@
name: "Bootstrap"
description: "Bootstrap all tools and dependencies"
inputs:
go-version:
description: "Go version to install"
required: true
default: "1.18.x"
use-go-cache:
description: "Restore go cache"
required: true
default: "true"
cache-key-prefix:
description: "Prefix all cache keys with this value"
required: true
default: "831180ac25"
build-cache-key-prefix:
description: "Prefix build cache key with this value"
required: true
default: "f8b6d31dea"
bootstrap-apt-packages:
description: "Space delimited list of tools to install via apt"
default: "libxml2-utils"
runs:
using: "composite"
steps:
- uses: actions/setup-go@v3
with:
go-version: ${{ inputs.go-version }}
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v3
with:
path: ${{ github.workspace }}/.tmp
key: ${{ inputs.cache-key-prefix }}-${{ runner.os }}-tool-${{ hashFiles('Makefile') }}
# note: we need to keep restoring the go mod cache before bootstrapping tools since `go install` is used in
# some installations of project tools.
- name: Restore go module cache
id: go-mod-cache
if: inputs.use-go-cache == 'true'
uses: actions/cache@v3
with:
path: |
~/go/pkg/mod
key: ${{ inputs.cache-key-prefix }}-${{ runner.os }}-go-${{ inputs.go-version }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ inputs.cache-key-prefix }}-${{ runner.os }}-go-${{ inputs.go-version }}-
- name: (cache-miss) Bootstrap project tools
shell: bash
if: steps.tool-cache.outputs.cache-hit != 'true'
run: make bootstrap-tools
- name: Restore go build cache
id: go-cache
if: inputs.use-go-cache == 'true'
uses: actions/cache@v3
with:
path: |
~/.cache/go-build
key: ${{ inputs.cache-key-prefix }}-${{ inputs.build-cache-key-prefix }}-${{ runner.os }}-go-${{ inputs.go-version }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ inputs.cache-key-prefix }}-${{ inputs.build-cache-key-prefix }}-${{ runner.os }}-go-${{ inputs.go-version }}-
- name: (cache-miss) Bootstrap go dependencies
shell: bash
if: steps.go-mod-cache.outputs.cache-hit != 'true' && inputs.use-go-cache == 'true'
run: make bootstrap-go
- name: Install apt packages
if: inputs.bootstrap-apt-packages != ''
shell: bash
run: |
DEBIAN_FRONTEND=noninteractive sudo apt update && sudo -E apt install -y ${{ inputs.bootstrap-apt-packages }}
- name: Create all cache fingerprints
shell: bash
run: make fingerprints
+10
View File
@@ -0,0 +1,10 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: daily
- package-ecosystem: "gomod"
directory: "/"
schedule:
interval: daily
+11
View File
@@ -0,0 +1,11 @@
#!/usr/bin/env bash
red=$(tput setaf 1)
bold=$(tput bold)
normal=$(tput sgr0)
# assert we are running in CI (or die!)
if [[ -z "$CI" ]]; then
echo "${bold}${red}This script should ONLY be run in CI. Exiting...${normal}"
exit 1
fi
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env python3
import subprocess
import sys
import shlex
class bcolors:
HEADER = '\033[95m'
OKBLUE = '\033[94m'
OKCYAN = '\033[96m'
OKGREEN = '\033[92m'
WARNING = '\033[93m'
FAIL = '\033[91m'
ENDC = '\033[0m'
BOLD = '\033[1m'
UNDERLINE = '\033[4m'
if len(sys.argv) < 3:
print("Usage: coverage.py [threshold] [go-coverage-report]")
sys.exit(1)
threshold = float(sys.argv[1])
report = sys.argv[2]
args = shlex.split(f"go tool cover -func {report}")
p = subprocess.run(args, capture_output=True, text=True)
percent_coverage = float(p.stdout.splitlines()[-1].split()[-1].replace("%", ""))
print(f"{bcolors.BOLD}Coverage: {percent_coverage}%{bcolors.ENDC}")
if percent_coverage < threshold:
print(f"{bcolors.BOLD}{bcolors.FAIL}Coverage below threshold of {threshold}%{bcolors.ENDC}")
sys.exit(1)
+6 -7
View File
@@ -4,19 +4,18 @@ set -eu
ORIGINAL_STATE_DIR=$(mktemp -d "TEMP-original-state-XXXXXXXXX")
TIDY_STATE_DIR=$(mktemp -d "TEMP-tidy-state-XXXXXXXXX")
trap "cp -v ${ORIGINAL_STATE_DIR}/* ./ && rm -fR ${ORIGINAL_STATE_DIR} ${TIDY_STATE_DIR}" EXIT
trap "cp ${ORIGINAL_STATE_DIR}/* ./ && rm -fR ${ORIGINAL_STATE_DIR} ${TIDY_STATE_DIR}" EXIT
echo "Capturing original state of files..."
cp -v go.mod go.sum "${ORIGINAL_STATE_DIR}"
# capturing original state of files...
cp go.mod go.sum "${ORIGINAL_STATE_DIR}"
echo "Capturing state of go.mod and go.sum after running go mod tidy..."
# capturing state of go.mod and go.sum after running go mod tidy...
go mod tidy
cp -v go.mod go.sum "${TIDY_STATE_DIR}"
echo ""
cp go.mod go.sum "${TIDY_STATE_DIR}"
set +e
# Detect difference between the git HEAD state and the go mod tidy state
# detect difference between the git HEAD state and the go mod tidy state
DIFF_MOD=$(diff -u "${ORIGINAL_STATE_DIR}/go.mod" "${TIDY_STATE_DIR}/go.mod")
DIFF_SUM=$(diff -u "${ORIGINAL_STATE_DIR}/go.sum" "${TIDY_STATE_DIR}/go.sum")
+58
View File
@@ -0,0 +1,58 @@
name: "Benchmark testing"
on:
workflow_dispatch:
pull_request:
jobs:
Benchmark-Test:
name: "Benchmark tests"
runs-on: ubuntu-20.04
# note: we want benchmarks to run on pull_request events in order to publish results to a sticky comment, and
# we also want to run on push such that merges to main are recorded to the cache. For this reason we don't filter
# the job by event.
steps:
- uses: actions/checkout@v3
- name: Bootstrap environment
uses: ./.github/actions/bootstrap
- name: Restore base benchmark result
uses: actions/cache@v3
with:
path: test/results/benchmark-main.txt
# use base sha for PR or new commit hash for main push in benchmark result key
key: ${{ runner.os }}-bench-${{ (github.event.pull_request.base.sha != github.event.after) && github.event.pull_request.base.sha || github.event.after }}
- name: Run benchmark tests
id: benchmark
run: |
REF_NAME=${GITHUB_REF##*/} make benchmark
OUTPUT=$(make show-benchstat)
OUTPUT="${OUTPUT//'%'/'%25'}" # URL encode all '%' characters
OUTPUT="${OUTPUT//$'\n'/'%0A'}" # URL encode all '\n' characters
OUTPUT="${OUTPUT//$'\r'/'%0D'}" # URL encode all '\r' characters
echo "::set-output name=result::$OUTPUT"
- uses: actions/upload-artifact@v3
with:
name: benchmark-test-results
path: test/results/**/*
- name: Update PR benchmark results comment
uses: marocchino/sticky-pull-request-comment@v2
continue-on-error: true
with:
header: benchmark
message: |
### Benchmark Test Results
<details>
<summary>Benchmark results from the latest changes vs base branch</summary>
```
${{ steps.benchmark.outputs.result }}
```
</details>
+3 -3
View File
@@ -34,7 +34,7 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -45,7 +45,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1
uses: github/codeql-action/autobuild@v2
# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
@@ -59,4 +59,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
uses: github/codeql-action/analyze@v2
+12 -30
View File
@@ -16,7 +16,7 @@ jobs:
environment: release
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
# we don't want to release commits that have been pushed and tagged, but not necessarily merged onto main
- name: Ensure tagged commit is on main
@@ -26,7 +26,7 @@ jobs:
git merge-base --is-ancestor ${GITHUB_REF##*/} origin/main && echo "${GITHUB_REF##*/} is a commit on main!"
- name: Check static analysis results
uses: fountainhead/action-wait-for-check@v1.0.0
uses: fountainhead/action-wait-for-check@v1.1.0
id: static-analysis
with:
token: ${{ secrets.GITHUB_TOKEN }}
@@ -35,7 +35,7 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Check unit test results
uses: fountainhead/action-wait-for-check@v1.0.0
uses: fountainhead/action-wait-for-check@v1.1.0
id: unit
with:
token: ${{ secrets.GITHUB_TOKEN }}
@@ -44,7 +44,7 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Check integration test results
uses: fountainhead/action-wait-for-check@v1.0.0
uses: fountainhead/action-wait-for-check@v1.1.0
id: integration
with:
token: ${{ secrets.GITHUB_TOKEN }}
@@ -53,7 +53,7 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Check acceptance test results (linux)
uses: fountainhead/action-wait-for-check@v1.0.0
uses: fountainhead/action-wait-for-check@v1.1.0
id: acceptance-linux
with:
token: ${{ secrets.GITHUB_TOKEN }}
@@ -62,7 +62,7 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Check acceptance test results (mac)
uses: fountainhead/action-wait-for-check@v1.0.0
uses: fountainhead/action-wait-for-check@v1.1.0
id: acceptance-mac
with:
token: ${{ secrets.GITHUB_TOKEN }}
@@ -71,7 +71,7 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Check cli test results (linux)
uses: fountainhead/action-wait-for-check@v1.0.0
uses: fountainhead/action-wait-for-check@v1.1.0
id: cli-linux
with:
token: ${{ secrets.GITHUB_TOKEN }}
@@ -97,33 +97,15 @@ jobs:
contents: write
packages: write
steps:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v2.1.3
- name: Bootstrap environment
uses: ./.github/actions/bootstrap
with:
path: ${{ github.workspace }}/.tmp
key: ${{ runner.os }}-tool-${{ hashFiles('Makefile') }}
- name: Restore go cache
id: go-cache
uses: actions/cache@v2.1.3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-
- name: (cache-miss) Bootstrap all project dependencies
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
# use the same cache we used for building snapshots
build-cache-key-prefix: "snapshot"
- name: Login to Docker Hub
uses: docker/login-action@v2
+2 -6
View File
@@ -16,7 +16,7 @@ jobs:
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v2
- uses: actions/setup-go@v3
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
@@ -39,10 +39,6 @@ jobs:
sed -r -i -e 's/^(YAJSV_VERSION = ).*/\1'${YAJSV_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(COSIGN_VERSION = ).*/\1'${COSIGN_LATEST_VERSION}'/' Makefile
# update cosign in go.mod as well
go get github.com/sigstore/cosign@$COSIGN_LATEST_VERSION
go mod tidy
# export the versions for use with create-pull-request
echo "::set-output name=GOLANGCILINT::$GOLANGCILINT_LATEST_VERSION"
echo "::set-output name=BOUNCER::$BOUNCER_LATEST_VERSION"
@@ -76,4 +72,4 @@ jobs:
- [yajsv ${{ steps.latest-versions.outputs.YAJSV }}](https://github.com/neilpa/yajsv/releases/tag/${{ steps.latest-versions.outputs.YAJSV }})
- [cosign ${{ steps.latest-versions.outputs.COSIGN }}](https://github.com/sigstore/cosign/releases/tag/${{ steps.latest-versions.outputs.COSIGN }})
This is an auto-generated pull request to update all of the bootstrap tools to the latest versions.
token: ${{ steps.generate-token.outputs.token }}
token: ${{ steps.generate-token.outputs.token }}
@@ -16,7 +16,7 @@ jobs:
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v2
- uses: actions/setup-go@v3
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
+54 -259
View File
@@ -1,14 +1,11 @@
name: "Validations"
on:
workflow_dispatch:
pull_request:
push:
branches:
- main
pull_request:
env:
GO_VERSION: "1.18.x"
GO_STABLE_VERSION: true
jobs:
@@ -17,100 +14,42 @@ jobs:
name: "Static analysis"
runs-on: ubuntu-20.04
steps:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v3
- uses: actions/checkout@v2
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v2.1.3
with:
path: ${{ github.workspace }}/.tmp
key: ${{ runner.os }}-tool-${{ hashFiles('Makefile') }}
- name: Restore go cache
id: go-cache
uses: actions/cache@v2.1.3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-
- name: (cache-miss) Bootstrap all project dependencies
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
- name: Bootstrap CI environment dependencies
run: make ci-bootstrap
- name: Bootstrap environment
uses: ./.github/actions/bootstrap
- name: Run static analysis
run: make static-analysis
Unit-Test:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
name: "Unit tests"
runs-on: ubuntu-20.04
steps:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v3
- uses: actions/checkout@v2
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v2.1.3
with:
path: ${{ github.workspace }}/.tmp
key: ${{ runner.os }}-tool-${{ hashFiles('Makefile') }}
- name: Restore go cache
id: go-cache
uses: actions/cache@v2.1.3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-
- name: (cache-miss) Bootstrap all project dependencies
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
- name: Bootstrap CI environment dependencies
run: make ci-bootstrap
- name: Build cache key for java test-fixture blobs (for unit tests)
run: make java-packages-fingerprint
- name: Bootstrap environment
uses: ./.github/actions/bootstrap
- name: Restore Java test-fixture cache
id: unit-java-cache
uses: actions/cache@v2.1.3
uses: actions/cache@v3
with:
path: syft/pkg/cataloger/java/test-fixtures/java-builds/packages
key: ${{ runner.os }}-unit-java-cache-${{ hashFiles( 'syft/pkg/cataloger/java/test-fixtures/java-builds/packages.fingerprint' ) }}
- name: Build cache key for rpm test-fixture blobs (for unit tests)
run: make rpm-binaries-fingerprint
- name: Restore RPM test-fixture cache
id: unit-rpm-cache
uses: actions/cache@v2.1.3
uses: actions/cache@v3
with:
path: syft/pkg/cataloger/rpm/test-fixtures/rpms
key: ${{ runner.os }}-unit-rpm-cache-${{ hashFiles( 'syft/pkg/cataloger/rpm/test-fixtures/rpms.fingerprint' ) }}
- name: Build cache key for go binary test-fixture blobs (for unit tests)
run: make go-binaries-fingerprint
- name: Restore Go binary test-fixture cache
- name: Restore go binary test-fixture cache
id: unit-go-binary-cache
uses: actions/cache@v2.1.3
uses: actions/cache@v3
with:
path: syft/pkg/cataloger/golang/test-fixtures/archs/binaries
key: ${{ runner.os }}-unit-go-binaries-cache-${{ hashFiles( 'syft/pkg/cataloger/golang/test-fixtures/archs/binaries.fingerprint' ) }}
@@ -118,54 +57,22 @@ jobs:
- name: Run unit tests
run: make unit
- uses: actions/upload-artifact@v2
with:
name: unit-test-results
path: test/results/**/*
Integration-Test:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
name: "Integration tests"
runs-on: ubuntu-20.04
steps:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v3
- uses: actions/checkout@v2
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v2.1.3
with:
path: ${{ github.workspace }}/.tmp
key: ${{ runner.os }}-tool-${{ hashFiles('Makefile') }}
- name: Restore go cache
id: go-cache
uses: actions/cache@v2.1.3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-
- name: (cache-miss) Bootstrap all project dependencies
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
- name: Bootstrap CI environment dependencies
run: make ci-bootstrap
- name: Bootstrap environment
uses: ./.github/actions/bootstrap
- name: Validate syft output against the CycloneDX schema
run: make validate-cyclonedx-schema
- name: Build key for tar cache
run: make integration-fingerprint
- name: Restore integration test cache
uses: actions/cache@v2.1.3
uses: actions/cache@v3
with:
path: ${{ github.workspace }}/test/integration/test-fixtures/cache
key: ${{ runner.os }}-integration-test-cache-${{ hashFiles('test/integration/test-fixtures/cache.fingerprint') }}
@@ -173,123 +80,35 @@ jobs:
- name: Run integration tests
run: make integration
Benchmark-Test:
name: "Benchmark tests"
runs-on: ubuntu-20.04
# note: we want benchmarks to run on pull_request events in order to publish results to a sticky comment, and
# we also want to run on push such that merges to main are recorded to the cache. For this reason we don't filter
# the job by event.
steps:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v2.1.3
with:
path: ${{ github.workspace }}/.tmp
key: ${{ runner.os }}-tool-${{ hashFiles('Makefile') }}
- name: Restore go cache
id: go-cache
uses: actions/cache@v2.1.3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-
- name: (cache-miss) Bootstrap all project dependencies
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
- name: Bootstrap CI environment dependencies
run: make ci-bootstrap
- name: Restore base benchmark result
uses: actions/cache@v2
with:
path: test/results/benchmark-main.txt
# use base sha for PR or new commit hash for main push in benchmark result key
key: ${{ runner.os }}-bench-${{ (github.event.pull_request.base.sha != github.event.after) && github.event.pull_request.base.sha || github.event.after }}
- name: Run benchmark tests
id: benchmark
run: |
REF_NAME=${GITHUB_REF##*/} make benchmark
OUTPUT=$(make show-benchstat)
OUTPUT="${OUTPUT//'%'/'%25'}" # URL encode all '%' characters
OUTPUT="${OUTPUT//$'\n'/'%0A'}" # URL encode all '\n' characters
OUTPUT="${OUTPUT//$'\r'/'%0D'}" # URL encode all '\r' characters
echo "::set-output name=result::$OUTPUT"
- uses: actions/upload-artifact@v2
with:
name: benchmark-test-results
path: test/results/**/*
- name: Update PR benchmark results comment
uses: marocchino/sticky-pull-request-comment@v2
continue-on-error: true
with:
header: benchmark
message: |
### Benchmark Test Results
<details>
<summary>Benchmark results from the latest changes vs base branch</summary>
```
${{ steps.benchmark.outputs.result }}
```
</details>
Build-Snapshot-Artifacts:
name: "Build snapshot artifacts"
runs-on: ubuntu-20.04
steps:
- uses: actions/setup-go@v2
- uses: actions/checkout@v3
- name: Bootstrap environment
uses: ./.github/actions/bootstrap
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v2.1.3
with:
path: ${{ github.workspace }}/.tmp
key: ${{ runner.os }}-tool-${{ hashFiles('Makefile') }}
- name: Restore go cache
id: go-cache
uses: actions/cache@v2.1.3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-
- name: (cache-miss) Bootstrap all project dependencies
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
# why have another build cache key? We don't want unit/integration/etc test build caches to replace
# the snapshot build cache, which includes builds for all OSs and architectures. As long as this key is
# unique from the build-cache-key-prefix in other CI jobs, we should be fine.
#
# note: ideally this value should match what is used in release (just to help with build times).
build-cache-key-prefix: "snapshot"
bootstrap-apt-packages: ""
- name: Build snapshot artifacts
run: make snapshot
- uses: actions/upload-artifact@v2
# why not use actions/upload-artifact? It is very slow (3 minutes to upload ~600MB of data, vs 10 seconds with this approach).
# see https://github.com/actions/upload-artifact/issues/199 for more info
- name: Upload snapshot artifacts
uses: actions/cache/save@v3
with:
name: artifacts
path: snapshot/**/*
path: snapshot
key: snapshot-build-${{ github.run_id }}
Acceptance-Linux:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
@@ -297,22 +116,20 @@ jobs:
needs: [Build-Snapshot-Artifacts]
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- uses: actions/download-artifact@v2
- name: Download snapshot build
uses: actions/cache/restore@v3
with:
name: artifacts
path: snapshot
key: snapshot-build-${{ github.run_id }}
- name: Run comparison tests (Linux)
run: make compare-linux
- name: Build key for image cache
run: make install-fingerprint
- name: Restore install.sh test image cache
id: install-test-image-cache
uses: actions/cache@v2.1.3
uses: actions/cache@v3
with:
path: ${{ github.workspace }}/test/install/cache
key: ${{ runner.os }}-install-test-image-cache-${{ hashFiles('test/install/cache.fingerprint') }}
@@ -335,16 +152,17 @@ jobs:
needs: [Build-Snapshot-Artifacts]
runs-on: macos-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- uses: actions/download-artifact@v2
- name: Download snapshot build
uses: actions/cache/restore@v3
with:
name: artifacts
path: snapshot
key: snapshot-build-${{ github.run_id }}
- name: Restore docker image cache for compare testing
id: mac-compare-testing-cache
uses: actions/cache@v2.1.3
uses: actions/cache@v3
with:
path: image.tar
key: ${{ runner.os }}-${{ hashFiles('test/compare/mac.sh') }}
@@ -355,52 +173,29 @@ jobs:
- name: Run install.sh tests (Mac)
run: make install-test-ci-mac
Cli-Linux:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
name: "CLI tests (Linux)"
needs: [Build-Snapshot-Artifacts]
runs-on: ubuntu-20.04
steps:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v3
- uses: actions/checkout@v2
- name: Bootstrap environment
uses: ./.github/actions/bootstrap
- name: Restore go cache
id: go-cache
uses: actions/cache@v2.1.3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v2.1.3
with:
path: ${{ github.workspace }}/.tmp
key: ${{ runner.os }}-tool-${{ hashFiles('Makefile') }}
- name: (cache-miss) Bootstrap all project dependencies
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
- name: Build key for tar cache
run: make cli-fingerprint
- name: Restore CLI test cache
uses: actions/cache@v2.1.3
- name: Restore CLI test-fixture cache
uses: actions/cache@v3
with:
path: ${{ github.workspace }}/test/cli/test-fixtures/cache
key: ${{ runner.os }}-cli-test-cache-${{ hashFiles('test/cli/test-fixtures/cache.fingerprint') }}
- uses: actions/download-artifact@v2
- name: Download snapshot build
uses: actions/cache/restore@v3
with:
name: artifacts
path: snapshot
key: snapshot-build-${{ github.run_id }}
- name: Run CLI Tests (Linux)
run: make cli
+6 -2
View File
@@ -1,9 +1,12 @@
# TODO: enable this when we have coverage on docstring comments
#issues:
issues:
max-same-issues: 25
# TODO: enable this when we have coverage on docstring comments
# # The list of ids of default excludes to include or disable.
# include:
# - EXC0002 # disable excluding of issues about comments from golint
linters:
# inverted configuration with `enable-all` and `disable` is not scalable during updates of golangci-lint
disable-all: true
@@ -15,6 +18,7 @@ linters:
- errcheck
- errorlint
- exportloopref
- forcetypeassert
- funlen
- gocognit
- goconst
+1 -4
View File
@@ -5,6 +5,7 @@ release:
env:
# required to support multi architecture docker builds
- DOCKER_CLI_EXPERIMENTAL=enabled
- CGO_ENABLED=0
builds:
- id: linux-build
@@ -19,8 +20,6 @@ builds:
- s390x
# set the modified timestamp on the output binary to the git timestamp to ensure a reproducible build
mod_timestamp: &build-timestamp '{{ .CommitTimestamp }}'
env: &build-env
- CGO_ENABLED=0
ldflags: &build-ldflags |
-w
-s
@@ -39,7 +38,6 @@ builds:
- amd64
- arm64
mod_timestamp: *build-timestamp
env: *build-env
ldflags: *build-ldflags
hooks:
post:
@@ -55,7 +53,6 @@ builds:
goarch:
- amd64
mod_timestamp: *build-timestamp
env: *build-env
ldflags: *build-ldflags
archives:
+22 -40
View File
@@ -15,37 +15,16 @@ After cloning the following step can help you get setup:
- this command `go run cmd/syft/main.go alpine:latest` will compile and run syft against `alpine:latest`
5. view the README or syft help output for more output options
#### Make output
The main make tasks for common static analysis and testing are `lint`, `lint-fix`, `unit`, `integration`, and `cli`.
```
all Run all linux-based checks (linting, license check, unit, integration, and linux compare tests)
benchmark Run benchmark tests and compare against the baseline (if available)
bootstrap Download and install all tooling dependencies (+ prep tooling in the ./tmp dir)
build Build release snapshot binaries and packages
check-licenses Ensure transitive dependencies are compliant with the current license policy
clean-test-image-cache Clean test image cache
clean Remove previous builds, result reports, and test cache
cli Run CLI tests
compare-linux Run compare tests on build snapshot binaries and packages (Linux)
compare-mac Run compare tests on build snapshot binaries and packages (Mac)
generate-json-schema Generate a new json schema
generate-license-list Generate an updated spdx license list
help Display this help
integration Run integration tests
lint-fix Auto-format all source code + run golangci lint fixers
lint Run gofmt + golangci lint checks
show-test-image-cache Show all docker and image tar cache
show-test-snapshots Show all test snapshots
snapshot-with-signing Build snapshot release binaries and packages (with dummy signing)
test Run all tests (currently unit, integration, linux compare, and cli tests)
unit Run unit tests (with coverage)
```
See `make help` for all the current make tasks.
## Architecture
Syft is used to generate a Software Bill of Materials (SBOM) from different kinds of input.
### Code organization for the cmd package
Syft's entrypoint can be found in the `cmd` package at `cmd/syft/main.go`. `main.go` builds a new syft `cli` via `cli.New()`
and then executes the `cli` via `cli.Execute()`. The `cli` package is responsible for parsing command line arguments,
setting up the application context and configuration, and executing the application. Each of syft's commands
@@ -55,23 +34,24 @@ They are registered in `syft/cli/commands/go`.
.
└── syft/
├── cli/
│   ├── attest/
│   ├── attest.go
│   ├── commands.go
│   ├── completion.go
│   ├── convert/
│   ├── convert.go
│   ├── eventloop/
│   ├── options/
│   ├── packages/
│   ├── packages.go
│   ├── poweruser/
│   ├── poweruser.go
│   └── version.go
│ ├── attest/
│ ├── attest.go
│ ├── commands.go
│ ├── completion.go
│ ├── convert/
│ ├── convert.go
│ ├── eventloop/
│ ├── options/
│ ├── packages/
│ ├── packages.go
│ ├── poweruser/
│ ├── poweruser.go
│ └── version.go
└── main.go
```
#### Execution flow
```mermaid
sequenceDiagram
participant main as cmd/syft/main
@@ -100,19 +80,19 @@ sequenceDiagram
Syft's core library (see, exported) functionality is implemented in the `syft` package. The `syft` package is responsible for organizing the core
SBOM data model, it's translated output formats, and the core SBOM generation logic.
#### Organization and design notes for the syft library
- analysis creates a static SBOM which can be encoded and decoded
- format objects, should strive to not add or enrich data in encoding that could otherwise be done during analysis
- package catalogers and their organization can be viewed/added to the `syft/pkg/cataloger` package
- file catalogers and their organization can be viewed/added to the `syft/file` package
- The source package provides an abstraction to allow a user to loosely define a data source that can be cataloged
- Logging Abstraction ...
#### Code example of syft as a library
Here is a gist of using syft as a library to generate a SBOM from a docker image: [link](https://gist.github.com/wagoodman/57ed59a6d57600c23913071b8470175b).
Here is a gist of using syft as a library to generate a SBOM for a docker image: [link](https://gist.github.com/wagoodman/57ed59a6d57600c23913071b8470175b).
The execution flow for the example is detailed below.
#### Execution flow examples for the syft library
```mermaid
sequenceDiagram
participant source as source.New(ubuntu:latest)
@@ -135,12 +115,14 @@ sequenceDiagram
### Syft Catalogers
##### Summary
Catalogers are the way in which syft is able to identify and construct packages given some amount of source metadata.
For example, Syft can locate and process `package-lock.json` files when performing filesystem scans.
See: [how to specify file globs](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/javascript/cataloger.go#L16-L21)
and an implementation of the [package-lock.json parser](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/javascript/cataloger.go#L16-L21) fora quick review.
#### Building a new Cataloger
Catalogers must fulfill the interface [found here](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger.go).
This means that when building a new cataloger, the new struct must implement both method signatures of `Catalog` and `Name`.
+256 -292
View File
@@ -1,24 +1,23 @@
BIN = syft
VERSION=$(shell git describe --dirty --always --tags)
TEMPDIR = ./.tmp
BIN := syft
TEMP_DIR := ./.tmp
# commands and versions
LINTCMD = $(TEMPDIR)/golangci-lint run --tests=false
GOIMPORTS_CMD = $(TEMPDIR)/gosimports -local github.com/anchore
RELEASE_CMD=$(TEMPDIR)/goreleaser release --rm-dist
SNAPSHOT_CMD=$(RELEASE_CMD) --skip-publish --snapshot
# Command templates #################################
LINT_CMD := $(TEMP_DIR)/golangci-lint run --tests=false
GOIMPORTS_CMD := $(TEMP_DIR)/gosimports -local github.com/anchore
RELEASE_CMD := $(TEMP_DIR)/goreleaser release --rm-dist
SNAPSHOT_CMD := $(RELEASE_CMD) --skip-publish --skip-sign --snapshot
# tool versions
GOLANGCILINT_VERSION = v1.50.1
GOSIMPORTS_VERSION = v0.3.5
BOUNCER_VERSION = v0.4.0
CHRONICLE_VERSION = v0.4.2
GORELEASER_VERSION = v1.14.1
YAJSV_VERSION = v1.4.1
COSIGN_VERSION = v1.13.1
QUILL_VERSION = v0.2.0
# Tool versions #################################
GOLANGCILINT_VERSION := v1.50.1
GOSIMPORTS_VERSION := v0.3.5
BOUNCER_VERSION := v0.4.0
CHRONICLE_VERSION := v0.4.2
GORELEASER_VERSION := v1.14.1
YAJSV_VERSION := v1.4.1
COSIGN_VERSION := v1.13.1
QUILL_VERSION := v0.2.0
# formatting variables
# Formatting variables #################################
BOLD := $(shell tput -T linux bold)
PURPLE := $(shell tput -T linux setaf 5)
GREEN := $(shell tput -T linux setaf 2)
@@ -28,55 +27,23 @@ RESET := $(shell tput -T linux sgr0)
TITLE := $(BOLD)$(PURPLE)
SUCCESS := $(BOLD)$(GREEN)
# test variables
RESULTSDIR = test/results
COMPARE_DIR = ./test/compare
COMPARE_TEST_IMAGE = centos:8.2.2004
COVER_REPORT = $(RESULTSDIR)/unit-coverage-details.txt
COVER_TOTAL = $(RESULTSDIR)/unit-coverage-summary.txt
# the quality gate lower threshold for unit test total % coverage (by function statements)
COVERAGE_THRESHOLD := 62
# Test variables #################################
COMPARE_DIR := ./test/compare
COMPARE_TEST_IMAGE := centos:8.2.2004
COVERAGE_THRESHOLD := 62 # the quality gate lower threshold for unit test total % coverage (by function statements)
# CI cache busting values; change these if you want CI to not use previous stored cache
INTEGRATION_CACHE_BUSTER="894d8ca"
CLI_CACHE_BUSTER="e5cdfd8"
BOOTSTRAP_CACHE="c7afb99ad"
## Build variables
DISTDIR=./dist
SNAPSHOTDIR=./snapshot
OS=$(shell uname | tr '[:upper:]' '[:lower:]')
SNAPSHOT_BIN=$(realpath $(shell pwd)/$(SNAPSHOTDIR)/$(OS)-build_$(OS)_amd64_v1/$(BIN))
## Variable assertions
ifndef TEMPDIR
$(error TEMPDIR is not set)
endif
ifndef RESULTSDIR
$(error RESULTSDIR is not set)
endif
ifndef COMPARE_DIR
$(error COMPARE_DIR is not set)
endif
ifndef DISTDIR
$(error DISTDIR is not set)
endif
ifndef SNAPSHOTDIR
$(error SNAPSHOTDIR is not set)
endif
## Build variables #################################
VERSION := $(shell git describe --dirty --always --tags)
DIST_DIR := ./dist
SNAPSHOT_DIR := ./snapshot
CHANGELOG := CHANGELOG.md
OS := $(shell uname | tr '[:upper:]' '[:lower:]')
SNAPSHOT_BIN := $(realpath $(shell pwd)/$(SNAPSHOT_DIR)/$(OS)-build_$(OS)_amd64_v1/$(BIN))
ifndef VERSION
$(error VERSION is not set)
endif
ifndef REF_NAME
REF_NAME = $(VERSION)
endif
define title
@printf '$(TITLE)$(1)$(RESET)\n'
endef
@@ -89,65 +56,58 @@ define safe_rm_rf_children
bash -c 'test -z "$(1)" && false || rm -rf $(1)/*'
endef
## Default Task
.DEFAULT_GOAL:=help
## Tasks
.PHONY: all
all: clean static-analysis test ## Run all linux-based checks (linting, license check, unit, integration, and linux compare tests)
all: static-analysis test ## Run all linux-based checks (linting, license check, unit, integration, and linux compare tests)
@printf '$(SUCCESS)All checks pass!$(RESET)\n'
.PHONY: static-analysis
static-analysis: check-go-mod-tidy lint check-licenses check-json-schema-drift ## Run all static analysis checks
.PHONY: test
test: unit validate-cyclonedx-schema integration benchmark compare-linux cli ## Run all tests (currently unit, integration, linux compare, and cli tests)
test: unit integration validate-cyclonedx-schema benchmark cli ## Run all tests (currently unit, integration, linux compare, and cli tests)
.PHONY: ci-bootstrap
ci-bootstrap:
DEBIAN_FRONTEND=noninteractive sudo apt update && sudo -E apt install -y bc jq libxml2-utils
.PHONY:
ci-bootstrap-mac:
github_changelog_generator --version || sudo gem install github_changelog_generator
## Bootstrapping targets #################################
$(RESULTSDIR):
mkdir -p $(RESULTSDIR)
$(TEMPDIR):
mkdir -p $(TEMPDIR)
.PHONY: bootstrap
bootstrap: $(TEMP_DIR) bootstrap-go bootstrap-tools ## Download and install all tooling dependencies (+ prep tooling in the ./tmp dir)
$(call title,Bootstrapping dependencies)
.PHONY: bootstrap-tools
bootstrap-tools: $(TEMPDIR)
curl -sSfL https://raw.githubusercontent.com/anchore/quill/main/install.sh | sh -s -- -b $(TEMPDIR)/ $(QUILL_VERSION)
GO111MODULE=off GOBIN=$(realpath $(TEMPDIR)) go get -u golang.org/x/perf/cmd/benchstat
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ $(GOLANGCILINT_VERSION)
curl -sSfL https://raw.githubusercontent.com/wagoodman/go-bouncer/master/bouncer.sh | sh -s -- -b $(TEMPDIR)/ $(BOUNCER_VERSION)
curl -sSfL https://raw.githubusercontent.com/anchore/chronicle/main/install.sh | sh -s -- -b $(TEMPDIR)/ $(CHRONICLE_VERSION)
.github/scripts/goreleaser-install.sh -d -b $(TEMPDIR)/ $(GORELEASER_VERSION)
bootstrap-tools: $(TEMP_DIR)
curl -sSfL https://raw.githubusercontent.com/anchore/quill/main/install.sh | sh -s -- -b $(TEMP_DIR)/ $(QUILL_VERSION)
GO111MODULE=off GOBIN=$(realpath $(TEMP_DIR)) go get -u golang.org/x/perf/cmd/benchstat
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMP_DIR)/ $(GOLANGCILINT_VERSION)
curl -sSfL https://raw.githubusercontent.com/wagoodman/go-bouncer/master/bouncer.sh | sh -s -- -b $(TEMP_DIR)/ $(BOUNCER_VERSION)
curl -sSfL https://raw.githubusercontent.com/anchore/chronicle/main/install.sh | sh -s -- -b $(TEMP_DIR)/ $(CHRONICLE_VERSION)
.github/scripts/goreleaser-install.sh -d -b $(TEMP_DIR)/ $(GORELEASER_VERSION)
# the only difference between goimports and gosimports is that gosimports removes extra whitespace between import blocks (see https://github.com/golang/go/issues/20818)
GOBIN="$(realpath $(TEMPDIR))" go install github.com/rinchsan/gosimports/cmd/gosimports@$(GOSIMPORTS_VERSION)
GOBIN="$(realpath $(TEMPDIR))" go install github.com/neilpa/yajsv@$(YAJSV_VERSION)
GOBIN="$(realpath $(TEMPDIR))" go install github.com/sigstore/cosign/cmd/cosign@$(COSIGN_VERSION)
GOBIN="$(realpath $(TEMP_DIR))" go install github.com/rinchsan/gosimports/cmd/gosimports@$(GOSIMPORTS_VERSION)
GOBIN="$(realpath $(TEMP_DIR))" go install github.com/neilpa/yajsv@$(YAJSV_VERSION)
GOBIN="$(realpath $(TEMP_DIR))" go install github.com/sigstore/cosign/cmd/cosign@$(COSIGN_VERSION)
.PHONY: bootstrap-go
bootstrap-go:
go mod download
.PHONY: bootstrap
bootstrap: $(RESULTSDIR) bootstrap-go bootstrap-tools ## Download and install all tooling dependencies (+ prep tooling in the ./tmp dir)
$(call title,Bootstrapping dependencies)
$(TEMP_DIR):
mkdir -p $(TEMP_DIR)
.PHONY: static-analysis
static-analysis: check-go-mod-tidy check-licenses lint
## Static analysis targets #################################
.PHONY: lint
lint: ## Run gofmt + golangci lint checks
lint: ## Run gofmt + golangci lint checks
$(call title,Running linters)
# ensure there are no go fmt differences
@printf "files with gofmt issues: [$(shell gofmt -l -s .)]\n"
@test -z "$(shell gofmt -l -s .)"
# run all golangci-lint rules
$(LINTCMD)
$(LINT_CMD)
@[ -z "$(shell $(GOIMPORTS_CMD) -d .)" ] || (echo "goimports needs to be fixed" && false)
# go tooling does not play well with certain filename characters, ensure the common cases don't result in future "go get" failures
@@ -155,243 +115,107 @@ lint: ## Run gofmt + golangci lint checks
@bash -c "[[ '$(MALFORMED_FILENAMES)' == '' ]] || (printf '\nfound unsupported filename characters:\n$(MALFORMED_FILENAMES)\n\n' && false)"
.PHONY: lint-fix
lint-fix: ## Auto-format all source code + run golangci lint fixers
lint-fix: ## Auto-format all source code + run golangci lint fixers
$(call title,Running lint fixers)
gofmt -w -s .
$(GOIMPORTS_CMD) -w .
$(LINTCMD) --fix
$(LINT_CMD) --fix
go mod tidy
.PHONY: check-licenses
check-licenses: ## Ensure transitive dependencies are compliant with the current license policy
$(TEMPDIR)/bouncer check ./...
check-licenses: ## Ensure transitive dependencies are compliant with the current license policy
$(call title,Checking for license compliance)
$(TEMP_DIR)/bouncer check ./...
check-go-mod-tidy:
@ .github/scripts/go-mod-tidy-check.sh && echo "go.mod and go.sum are tidy!"
check-json-schema-drift:
$(call title,Ensure there is no drift between the JSON schema and the code)
@git diff-index --quiet HEAD -- || (echo "there are uncommitted changes, please commit them before running this check" && false)
@make generate-json-schema || (echo "$(RED)$(BOLD)JSON schema drift detected!$(RESET)" && false)
@git diff-index --quiet HEAD -- || (echo "$(RED)$(BOLD)JSON schema drift detected!$(RESET)" && false)
## Testing targets #################################
.PHONY: unit
unit: $(TEMP_DIR) fixtures ## Run unit tests (with coverage)
$(call title,Running unit tests)
go test -coverprofile $(TEMP_DIR)/unit-coverage-details.txt $(shell go list ./... | grep -v anchore/syft/test)
@.github/scripts/coverage.py $(COVERAGE_THRESHOLD) $(TEMP_DIR)/unit-coverage-details.txt
.PHONY: integration
integration: ## Run integration tests
$(call title,Running integration tests)
go test -v ./test/integration
.PHONY: validate-cyclonedx-schema
validate-cyclonedx-schema:
cd schema/cyclonedx && make
.PHONY: unit
unit: $(RESULTSDIR) fixtures ## Run unit tests (with coverage)
$(call title,Running unit tests)
go test -coverprofile $(COVER_REPORT) $(shell go list ./... | grep -v anchore/syft/test)
@go tool cover -func $(COVER_REPORT) | grep total | awk '{print substr($$3, 1, length($$3)-1)}' > $(COVER_TOTAL)
@echo "Coverage: $$(cat $(COVER_TOTAL))"
@if [ $$(echo "$$(cat $(COVER_TOTAL)) >= $(COVERAGE_THRESHOLD)" | bc -l) -ne 1 ]; then echo "$(RED)$(BOLD)Failed coverage quality gate (> $(COVERAGE_THRESHOLD)%)$(RESET)" && false; fi
.PHONY: cli
cli: $(SNAPSHOT_DIR) ## Run CLI tests
chmod 755 "$(SNAPSHOT_BIN)"
$(SNAPSHOT_BIN) version
SYFT_BINARY_LOCATION='$(SNAPSHOT_BIN)' \
go test -count=1 -timeout=15m -v ./test/cli
## Benchmark test targets #################################
.PHONY: benchmark
benchmark: $(RESULTSDIR) ## Run benchmark tests and compare against the baseline (if available)
benchmark: $(TEMP_DIR) ## Run benchmark tests and compare against the baseline (if available)
$(call title,Running benchmark tests)
go test -p 1 -run=^Benchmark -bench=. -count=5 -benchmem ./... | tee $(RESULTSDIR)/benchmark-$(REF_NAME).txt
(test -s $(RESULTSDIR)/benchmark-main.txt && \
$(TEMPDIR)/benchstat $(RESULTSDIR)/benchmark-main.txt $(RESULTSDIR)/benchmark-$(REF_NAME).txt || \
$(TEMPDIR)/benchstat $(RESULTSDIR)/benchmark-$(REF_NAME).txt) \
| tee $(RESULTSDIR)/benchstat.txt
go test -p 1 -run=^Benchmark -bench=. -count=7 -benchmem ./... | tee $(TEMP_DIR)/benchmark-$(VERSION).txt
(test -s $(TEMP_DIR)/benchmark-main.txt && \
$(TEMP_DIR)/benchstat $(TEMP_DIR)/benchmark-main.txt $(TEMP_DIR)/benchmark-$(VERSION).txt || \
$(TEMP_DIR)/benchstat $(TEMP_DIR)/benchmark-$(VERSION).txt) \
| tee $(TEMP_DIR)/benchstat.txt
.PHONY: show-benchstat
show-benchstat:
@cat $(RESULTSDIR)/benchstat.txt
@cat $(TEMP_DIR)/benchstat.txt
# note: this is used by CI to determine if the install test fixture cache (docker image tars) should be busted
install-fingerprint:
cd test/install && \
## Test-fixture-related targets #################################
# note: this is used by CI to determine if various test fixture cache should be restored or recreated
fingerprints:
$(call title,Creating all test cache input fingerprints)
# for IMAGE integration test fixtures
cd test/integration/test-fixtures && \
make cache.fingerprint
install-test: $(SNAPSHOTDIR)
cd test/install && \
make
install-test-cache-save: $(SNAPSHOTDIR)
cd test/install && \
make save
install-test-cache-load: $(SNAPSHOTDIR)
cd test/install && \
make load
install-test-ci-mac: $(SNAPSHOTDIR)
cd test/install && \
make ci-test-mac
.PHONY: integration
integration: ## Run integration tests
$(call title,Running integration tests)
go test -v ./test/integration
# note: this is used by CI to determine if the integration test fixture cache (docker image tars) should be busted
integration-fingerprint:
$(call title,Integration test fixture fingerprint)
find test/integration/test-fixtures/image-* -type f -exec md5sum {} + | awk '{print $1}' | sort | tee /dev/stderr | md5sum | tee test/integration/test-fixtures/cache.fingerprint && echo "$(INTEGRATION_CACHE_BUSTER)" >> test/integration/test-fixtures/cache.fingerprint
.PHONY: java-packages-fingerprint
java-packages-fingerprint:
$(call title,Java test fixture fingerprint)
# for JAVA BUILD test fixtures
cd syft/pkg/cataloger/java/test-fixtures/java-builds && \
make packages.fingerprint
.PHONY: go-binaries-fingerprint
go-binaries-fingerprint:
$(call title,Go binaries test fixture fingerprint)
# for GO BINARY test fixtures
cd syft/pkg/cataloger/golang/test-fixtures/archs && \
make binaries.fingerprint
.PHONY: rpm-binaries-fingerprint
rpm-binaries-fingerprint:
$(call title,RPM binary test fixture fingerprint)
# for RPM test fixtures
cd syft/pkg/cataloger/rpm/test-fixtures && \
make rpms.fingerprint
# for INSTALL integration test fixtures
cd test/install && \
make cache.fingerprint
# for CLI test fixtures
cd test/cli/test-fixtures && \
make cache.fingerprint
.PHONY: fixtures
fixtures:
$(call title,Generating test fixtures)
cd syft/pkg/cataloger/java/test-fixtures/java-builds && make
cd syft/pkg/cataloger/rpm/test-fixtures && make
.PHONY: generate-json-schema
generate-json-schema: ## Generate a new json schema
cd schema/json && go run generate.go
.PHONY: generate-license-list
generate-license-list: ## Generate an updated spdx license list
go generate ./internal/spdxlicense/...
gofmt -s -w ./internal/spdxlicense
.PHONY: build
build: $(SNAPSHOTDIR) ## Build release snapshot binaries and packages
$(SNAPSHOTDIR): ## Build snapshot release binaries and packages
$(call title,Building snapshot artifacts)
# create a config with the dist dir overridden
echo "dist: $(SNAPSHOTDIR)" > $(TEMPDIR)/goreleaser.yaml
cat .goreleaser.yaml >> $(TEMPDIR)/goreleaser.yaml
# build release snapshots
bash -c "SKIP_SIGNING=true $(SNAPSHOT_CMD) --skip-sign --config $(TEMPDIR)/goreleaser.yaml"
.PHONY: snapshot-with-signing
snapshot-with-signing: ## Build snapshot release binaries and packages (with dummy signing)
$(call title,Building snapshot artifacts (+ signing))
# create a config with the dist dir overridden
echo "dist: $(SNAPSHOTDIR)" > $(TEMPDIR)/goreleaser.yaml
cat .goreleaser.yaml >> $(TEMPDIR)/goreleaser.yaml
# build release snapshots
bash -c "\
$(SNAPSHOT_CMD) \
--config $(TEMPDIR)/goreleaser.yaml \
|| (cat /tmp/quill-*.log && false)"
# note: we cannot clean the snapshot directory since the pipeline builds the snapshot separately
.PHONY: compare-mac
compare-mac: $(RESULTSDIR) $(SNAPSHOTDIR) ## Run compare tests on build snapshot binaries and packages (Mac)
$(call title,Running compare test: Run on Mac)
$(COMPARE_DIR)/mac.sh \
$(SNAPSHOTDIR) \
$(COMPARE_DIR) \
$(COMPARE_TEST_IMAGE) \
$(RESULTSDIR)
# note: we cannot clean the snapshot directory since the pipeline builds the snapshot separately
.PHONY: compare-linux
compare-linux: compare-test-deb-package-install compare-test-rpm-package-install ## Run compare tests on build snapshot binaries and packages (Linux)
.PHONY: compare-test-deb-package-install
compare-test-deb-package-install: $(RESULTSDIR) $(SNAPSHOTDIR)
$(call title,Running compare test: DEB install)
$(COMPARE_DIR)/deb.sh \
$(SNAPSHOTDIR) \
$(COMPARE_DIR) \
$(COMPARE_TEST_IMAGE) \
$(RESULTSDIR)
.PHONY: compare-test-rpm-package-install
compare-test-rpm-package-install: $(RESULTSDIR) $(SNAPSHOTDIR)
$(call title,Running compare test: RPM install)
$(COMPARE_DIR)/rpm.sh \
$(SNAPSHOTDIR) \
$(COMPARE_DIR) \
$(COMPARE_TEST_IMAGE) \
$(RESULTSDIR)
# note: this is used by CI to determine if the integration test fixture cache (docker image tars) should be busted
cli-fingerprint:
$(call title,CLI test fixture fingerprint)
find test/cli/test-fixtures/image-* -type f -exec md5sum {} + | awk '{print $1}' | sort | md5sum | tee test/cli/test-fixtures/cache.fingerprint && echo "$(CLI_CACHE_BUSTER)" >> test/cli/test-fixtures/cache.fingerprint
.PHONY: cli
cli: $(SNAPSHOTDIR) ## Run CLI tests
chmod 755 "$(SNAPSHOT_BIN)"
$(SNAPSHOT_BIN) version
SYFT_BINARY_LOCATION='$(SNAPSHOT_BIN)' \
go test -count=1 -timeout=15m -v ./test/cli
.PHONY: changelog
changelog: clean-changelog CHANGELOG.md
@docker run -it --rm \
-v $(shell pwd)/CHANGELOG.md:/CHANGELOG.md \
rawkode/mdv \
-t 748.5989 \
/CHANGELOG.md
CHANGELOG.md:
$(TEMPDIR)/chronicle -vv > CHANGELOG.md
.PHONY: release
release: clean-dist CHANGELOG.md
$(call title,Publishing release artifacts)
# create a config with the dist dir overridden
echo "dist: $(DISTDIR)" > $(TEMPDIR)/goreleaser.yaml
cat .goreleaser.yaml >> $(TEMPDIR)/goreleaser.yaml
# note: notarization cannot be done in parallel, thus --parallelism 1
bash -c "\
$(RELEASE_CMD) \
--config $(TEMPDIR)/goreleaser.yaml \
--parallelism 1 \
--release-notes <(cat CHANGELOG.md) \
|| (cat /tmp/quill-*.log && false)"
# TODO: turn this into a post-release hook
# upload the version file that supports the application version update check (excluding pre-releases)
.github/scripts/update-version-file.sh "$(DISTDIR)" "$(VERSION)"
.PHONY: clean
clean: clean-dist clean-snapshot clean-test-image-cache ## Remove previous builds, result reports, and test cache
$(call safe_rm_rf_children,$(RESULTSDIR))
.PHONY: clean-snapshot
clean-snapshot:
$(call safe_rm_rf,$(SNAPSHOTDIR))
rm -f $(TEMPDIR)/goreleaser.yaml
.PHONY: clean-dist
clean-dist: clean-changelog
$(call safe_rm_rf,$(DISTDIR))
rm -f $(TEMPDIR)/goreleaser.yaml
.PHONY: clean-changelog
clean-changelog:
rm -f CHANGELOG.md
clean-test-image-cache: clean-test-image-tar-cache clean-test-image-docker-cache ## Clean test image cache
.PHONY: clear-test-image-tar-cache
clean-test-image-tar-cache:
## Delete all test cache (built docker image tars)
find . -type f -wholename "**/test-fixtures/cache/stereoscope-fixture-*.tar" -delete
.PHONY: clear-test-image-docker-cache
clean-test-image-docker-cache:
## Purge all test docker images
docker images --format '{{.ID}} {{.Repository}}' | grep stereoscope-fixture- | awk '{print $$1}' | uniq | xargs -r docker rmi --force
.PHONY: show-test-image-cache
show-test-image-cache: ## Show all docker and image tar cache
show-test-image-cache: ## Show all docker and image tar cache
$(call title,Docker daemon cache)
@docker images --format '{{.ID}} {{.Repository}}:{{.Tag}}' | grep stereoscope-fixture- | sort
@@ -399,10 +223,150 @@ show-test-image-cache: ## Show all docker and image tar cache
@find . -type f -wholename "**/test-fixtures/cache/stereoscope-fixture-*.tar" | sort
.PHONY: show-test-snapshots
show-test-snapshots: ## Show all test snapshots
show-test-snapshots: ## Show all test snapshots
$(call title,Test snapshots)
@find . -type f -wholename "**/test-fixtures/snapshot/*" | sort
## install.sh testing targets #################################
install-test: $(SNAPSHOT_DIR)
cd test/install && \
make
install-test-cache-save: $(SNAPSHOT_DIR)
cd test/install && \
make save
install-test-cache-load: $(SNAPSHOT_DIR)
cd test/install && \
make load
install-test-ci-mac: $(SNAPSHOT_DIR)
cd test/install && \
make ci-test-mac
# note: we cannot clean the snapshot directory since the pipeline builds the snapshot separately
.PHONY: compare-mac
compare-mac: $(TEMP_DIR) $(SNAPSHOT_DIR) ## Run compare tests on build snapshot binaries and packages (Mac)
$(call title,Running compare test: Run on Mac)
$(COMPARE_DIR)/mac.sh \
$(SNAPSHOT_DIR) \
$(COMPARE_DIR) \
$(COMPARE_TEST_IMAGE) \
$(TEMP_DIR)
# note: we cannot clean the snapshot directory since the pipeline builds the snapshot separately
.PHONY: compare-linux
compare-linux: compare-test-deb-package-install compare-test-rpm-package-install ## Run compare tests on build snapshot binaries and packages (Linux)
.PHONY: compare-test-deb-package-install
compare-test-deb-package-install: $(TEMP_DIR) $(SNAPSHOT_DIR)
$(call title,Running compare test: DEB install)
$(COMPARE_DIR)/deb.sh \
$(SNAPSHOT_DIR) \
$(COMPARE_DIR) \
$(COMPARE_TEST_IMAGE) \
$(TEMP_DIR)
.PHONY: compare-test-rpm-package-install
compare-test-rpm-package-install: $(TEMP_DIR) $(SNAPSHOT_DIR)
$(call title,Running compare test: RPM install)
$(COMPARE_DIR)/rpm.sh \
$(SNAPSHOT_DIR) \
$(COMPARE_DIR) \
$(COMPARE_TEST_IMAGE) \
$(TEMP_DIR)
## Code generation targets #################################
.PHONY: generate-json-schema
generate-json-schema: ## Generate a new json schema
cd schema/json && go run generate.go
.PHONY: generate-license-list
generate-license-list: ## Generate an updated spdx license list
go generate ./internal/spdxlicense/...
gofmt -s -w ./internal/spdxlicense
## Build-related targets #################################
.PHONY: build
build: $(SNAPSHOT_DIR) ## Build release snapshot binaries and packages
$(SNAPSHOT_DIR): ## Build snapshot release binaries and packages
$(call title,Building snapshot artifacts)
# create a config with the dist dir overridden
echo "dist: $(SNAPSHOT_DIR)" > $(TEMP_DIR)/goreleaser.yaml
cat .goreleaser.yaml >> $(TEMP_DIR)/goreleaser.yaml
# build release snapshots
$(SNAPSHOT_CMD) --config $(TEMP_DIR)/goreleaser.yaml
.PHONY: changelog
changelog: clean-changelog $(CHANGELOG) ## Generate and show the changelog for the current unreleased version
@docker run -it --rm \
-v $(shell pwd)/$(CHANGELOG):/$(CHANGELOG) \
rawkode/mdv \
-t 748.5989 \
/$(CHANGELOG)
$(CHANGELOG):
$(TEMP_DIR)/chronicle -vv > $(CHANGELOG)
.PHONY: release
release: clean-dist $(CHANGELOG)
$(call title,Publishing release artifacts)
@.github/scripts/ci-check.sh
# create a config with the dist dir overridden
echo "dist: $(DIST_DIR)" > $(TEMP_DIR)/goreleaser.yaml
cat .goreleaser.yaml >> $(TEMP_DIR)/goreleaser.yaml
bash -c "\
$(RELEASE_CMD) \
--config $(TEMP_DIR)/goreleaser.yaml \
--release-notes <(cat $(CHANGELOG)) \
|| (cat /tmp/quill-*.log && false)"
# upload the version file that supports the application version update check (excluding pre-releases)
.github/scripts/update-version-file.sh "$(DIST_DIR)" "$(VERSION)"
## Cleanup targets #################################
.PHONY: clean
clean: clean-dist clean-snapshot clean-test-image-cache ## Remove previous builds, result reports, and test cache
$(call safe_rm_rf_children,$(TEMP_DIR))
.PHONY: clean-snapshot
clean-snapshot:
$(call safe_rm_rf,$(SNAPSHOT_DIR))
rm -f $(TEMP_DIR)/goreleaser.yaml
.PHONY: clean-dist
clean-dist: clean-changelog
$(call safe_rm_rf,$(DIST_DIR))
rm -f $(TEMP_DIR)/goreleaser.yaml
.PHONY: clean-changelog
clean-changelog:
rm -f $(CHANGELOG)
clean-test-image-cache: clean-test-image-tar-cache clean-test-image-docker-cache ## Clean test image cache
.PHONY: clear-test-image-tar-cache
clean-test-image-tar-cache: ## Delete all test cache (built docker image tars)
find . -type f -wholename "**/test-fixtures/cache/stereoscope-fixture-*.tar" -delete
.PHONY: clear-test-image-docker-cache
clean-test-image-docker-cache: ## Purge all test docker images
docker images --format '{{.ID}} {{.Repository}}' | grep stereoscope-fixture- | awk '{print $$1}' | uniq | xargs -r docker rmi --force
## Halp! #################################
.PHONY: help
help: ## Display this help
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "$(BOLD)$(CYAN)%-25s$(RESET)%s\n", $$1, $$2}'
+1 -1
View File
@@ -82,7 +82,7 @@ brew install syft
### Nix
**Note**: Nix packaging of Syft is [community maintained](https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/syft/default.nix). Syft is currently only in the [unstable channel](https://nixos.wiki/wiki/Nix_channels#The_official_channels) awaiting the `22.05` release
**Note**: Nix packaging of Syft is [community maintained](https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/syft/default.nix). Syft is available in the [stable channel](https://nixos.wiki/wiki/Nix_channels#The_official_channels) since NixOS `22.05`.
```bash
nix-env -i syft
+34 -35
View File
@@ -5,20 +5,20 @@ go 1.18
require (
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
github.com/acobaugh/osrelease v0.1.0
github.com/adrg/xdg v0.3.3
github.com/adrg/xdg v0.4.0
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501
github.com/bmatcuk/doublestar/v4 v4.0.2
github.com/bmatcuk/doublestar/v4 v4.6.0
github.com/dustin/go-humanize v1.0.0
github.com/facebookincubator/nvdtools v0.1.4
github.com/go-test/deep v1.0.8
github.com/facebookincubator/nvdtools v0.1.5
github.com/go-test/deep v1.1.0
github.com/google/go-cmp v0.5.9
github.com/google/uuid v1.3.0
github.com/gookit/color v1.4.2
github.com/gookit/color v1.5.2
github.com/hashicorp/go-multierror v1.1.1
github.com/jinzhu/copier v0.3.2
github.com/jinzhu/copier v0.3.5
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/mholt/archiver/v3 v3.5.1
github.com/microsoft/go-rustaudit v0.0.0-20220730194248-4b17361d90a5
@@ -29,22 +29,22 @@ require (
github.com/pelletier/go-toml v1.9.5
// pinned to pull in 386 arch fix: https://github.com/scylladb/go-set/commit/cc7b2070d91ebf40d233207b633e28f5bd8f03a5
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e
github.com/sergi/go-diff v1.2.0
github.com/sergi/go-diff v1.3.1
github.com/sirupsen/logrus v1.9.0
github.com/spdx/tools-golang v0.3.1-0.20221108182156-8a01147e6342
github.com/spf13/afero v1.8.2
github.com/spf13/cobra v1.6.0
github.com/spf13/afero v1.9.3
github.com/spf13/cobra v1.6.1
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.13.0
github.com/spf13/viper v1.15.0
github.com/stretchr/testify v1.8.1
github.com/vifraa/gopom v0.1.0
github.com/vifraa/gopom v0.2.1
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4
golang.org/x/net v0.0.0-20221012135044-0b7e1fb9d458
golang.org/x/term v0.0.0-20220919170432-7a66f970e087
golang.org/x/mod v0.6.0
golang.org/x/net v0.5.0
golang.org/x/term v0.4.0
gopkg.in/yaml.v2 v2.4.0
)
@@ -53,13 +53,13 @@ require (
github.com/Masterminds/sprig/v3 v3.2.2
github.com/anchore/go-logger v0.0.0-20220728155337-03b66a5207d8
github.com/anchore/stereoscope v0.0.0-20221208011002-c5ff155d72f1
github.com/docker/docker v20.10.17+incompatible
github.com/google/go-containerregistry v0.11.0
github.com/docker/docker v20.10.20+incompatible
github.com/google/go-containerregistry v0.12.1
github.com/invopop/jsonschema v0.7.0
github.com/knqyf263/go-rpmdb v0.0.0-20221030135625-4082a22221ce
github.com/opencontainers/go-digest v1.0.0
github.com/sassoftware/go-rpmutils v0.2.0
github.com/vbatts/go-mtree v0.5.0
github.com/vbatts/go-mtree v0.5.2
golang.org/x/exp v0.0.0-20220823124025-807a23277127
gopkg.in/yaml.v3 v3.0.1
)
@@ -68,17 +68,17 @@ require (
github.com/DataDog/zstd v1.4.5 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.1.1 // indirect
github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/Microsoft/go-winio v0.6.0 // indirect
github.com/containerd/containerd v1.6.12 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.12.0 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.12.1 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/docker/cli v20.10.17+incompatible // indirect
github.com/docker/cli v20.10.20+incompatible // indirect
github.com/docker/distribution v2.8.1+incompatible // indirect
github.com/docker/docker-credential-helpers v0.6.4 // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.4.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
github.com/fsnotify/fsnotify v1.5.4 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.0 // indirect
github.com/go-restruct/restruct v1.2.0-alpha // indirect
github.com/gogo/protobuf v1.3.2 // indirect
@@ -91,10 +91,10 @@ require (
github.com/imdario/mergo v0.3.12 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/klauspost/compress v1.15.9 // indirect
github.com/klauspost/compress v1.15.11 // indirect
github.com/klauspost/pgzip v1.2.5 // indirect
github.com/logrusorgru/aurora v0.0.0-20200102142835-e9ef32dff381 // indirect
github.com/magiconair/properties v1.8.6 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
@@ -102,8 +102,8 @@ require (
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 // indirect
github.com/nwaples/rardecode v1.1.0 // indirect
github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 // indirect
github.com/pelletier/go-toml/v2 v2.0.5 // indirect
github.com/opencontainers/image-spec v1.1.0-rc2 // indirect
github.com/pelletier/go-toml/v2 v2.0.6 // indirect
github.com/pierrec/lz4/v4 v4.1.15 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 // indirect
@@ -113,7 +113,7 @@ require (
github.com/spf13/cast v1.5.0 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/stretchr/objx v0.5.0 // indirect
github.com/subosito/gotenv v1.4.1 // indirect
github.com/subosito/gotenv v1.4.2 // indirect
github.com/sylabs/sif/v2 v2.8.1 // indirect
github.com/sylabs/squashfs v0.6.1 // indirect
github.com/therootcompany/xz v1.0.1 // indirect
@@ -123,14 +123,13 @@ require (
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
golang.org/x/sync v0.0.0-20220929204114-8fcdb60fdcc0 // indirect
golang.org/x/sys v0.0.0-20220928140112-f11e5e49a4ec // indirect
golang.org/x/text v0.3.8-0.20211004125949-5bd84dd9b33b // indirect
golang.org/x/time v0.0.0-20220922220347-f3bd1da661af // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.4.0 // indirect
golang.org/x/text v0.6.0 // indirect
golang.org/x/tools v0.1.12 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/genproto v0.0.0-20221010155953-15ba04fc1c0e // indirect
google.golang.org/grpc v1.50.1 // indirect
google.golang.org/genproto v0.0.0-20221227171554-f9683d7f8bef // indirect
google.golang.org/grpc v1.52.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
lukechampine.com/uint128 v1.1.1 // indirect
@@ -150,7 +149,7 @@ require (
// go: warning: github.com/andybalholm/brotli@v1.0.1: retracted by module author: occasional panics and data corruption
github.com/andybalholm/brotli v1.0.4 // indirect
github.com/pkg/errors v0.9.1 // indirect
golang.org/x/crypto v0.0.0-20220926161630-eccd6366d1be // indirect
golang.org/x/crypto v0.1.0 // indirect
)
retract (
+129 -381
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "6.1.0"
JSONSchemaVersion = "6.2.0"
)
+2 -1
View File
@@ -40,7 +40,8 @@ type artifactMetadataContainer struct {
Dotnet pkg.DotnetDepsMetadata
Dpkg pkg.DpkgMetadata
Gem pkg.GemMetadata
Go pkg.GolangMetadata
GoBin pkg.GolangBinMetadata
GoMod pkg.GolangModMetadata
Hackage pkg.HackageMetadata
Java pkg.JavaMetadata
KbPackage pkg.KbPackageMetadata
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -366,10 +366,10 @@
"type" : "string"
},
"licenseInfoFromFiles" : {
"description" : "The licensing information that was discovered directly within the package. There will be an instance of this property for each distinct value of alllicenseInfoInFile properties of all files contained in the package.\n\nIf the licenseInfoFromFiles field is not present for a package and filesAnalyzed property for that same pacakge is true or omitted, it implies an equivalent meaning to NOASSERTION.",
"description" : "The licensing information that was discovered directly within the package. There will be an instance of this property for each distinct value of alllicenseInfoInFile properties of all files contained in the package.\n\nIf the licenseInfoFromFiles field is not present for a package and filesAnalyzed property for that same package is true or omitted, it implies an equivalent meaning to NOASSERTION.",
"type" : "array",
"items" : {
"description" : "License expression for licenseInfoFromFiles. See SPDX Annex D for the license expression syntax. The licensing information that was discovered directly within the package. There will be an instance of this property for each distinct value of alllicenseInfoInFile properties of all files contained in the package.\n\nIf the licenseInfoFromFiles field is not present for a package and filesAnalyzed property for that same pacakge is true or omitted, it implies an equivalent meaning to NOASSERTION.",
"description" : "License expression for licenseInfoFromFiles. See SPDX Annex D for the license expression syntax. The licensing information that was discovered directly within the package. There will be an instance of this property for each distinct value of alllicenseInfoInFile properties of all files contained in the package.\n\nIf the licenseInfoFromFiles field is not present for a package and filesAnalyzed property for that same package is true or omitted, it implies an equivalent meaning to NOASSERTION.",
"type" : "string"
}
},
@@ -90,8 +90,8 @@ func Test_encodeComponentProperties(t *testing.T) {
Version: "v0.0.0-20211006190231-62292e806868",
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: "1.17",
Architecture: "amd64",
H1Digest: "h1:KlOXYy8wQWTUJYFgkUI40Lzr06ofg5IRXUK5C7qZt1k=",
@@ -99,13 +99,32 @@ func Test_encodeComponentProperties(t *testing.T) {
},
expected: &[]cyclonedx.Property{
{Name: "syft:package:language", Value: pkg.Go.String()},
{Name: "syft:package:metadataType", Value: "GolangMetadata"},
{Name: "syft:package:metadataType", Value: "GolangBinMetadata"},
{Name: "syft:package:type", Value: "go-module"},
{Name: "syft:metadata:architecture", Value: "amd64"},
{Name: "syft:metadata:goCompiledVersion", Value: "1.17"},
{Name: "syft:metadata:h1Digest", Value: "h1:KlOXYy8wQWTUJYFgkUI40Lzr06ofg5IRXUK5C7qZt1k="},
},
},
{
name: "from go mod",
input: pkg.Package{
Name: "golang.org/x/net",
Version: "v0.0.0-20211006190231-62292e806868",
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{
H1Digest: "h1:KlOXYy8wQWTUJYFgkUI40Lzr06ofg5IRXUK5C7qZt1k=",
},
},
expected: &[]cyclonedx.Property{
{Name: "syft:package:language", Value: pkg.Go.String()},
{Name: "syft:package:metadataType", Value: "GolangModMetadata"},
{Name: "syft:package:type", Value: "go-module"},
{Name: "syft:metadata:h1Digest", Value: "h1:KlOXYy8wQWTUJYFgkUI40Lzr06ofg5IRXUK5C7qZt1k="},
},
},
{
name: "from rpm",
input: pkg.Package{
@@ -331,7 +331,7 @@ func toPackageChecksums(p pkg.Package) ([]common.Checksum, bool) {
})
}
}
case pkg.GolangMetadata:
case pkg.GolangBinMetadata:
// because the H1 digest is found in the Golang metadata we cannot claim that the files were analyzed
algo, hexStr, err := util.HDigestToSHA(meta.H1Digest)
if err != nil {
@@ -76,8 +76,8 @@ func Test_toPackageChecksums(t *testing.T) {
Name: "test",
Version: "1.0.0",
Language: pkg.Go,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
},
@@ -381,8 +381,8 @@ func Test_H1Digest(t *testing.T) {
pkg: pkg.Package{
Name: "github.com/googleapis/gnostic",
Version: "v0.5.5",
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
},
@@ -393,8 +393,8 @@ func Test_H1Digest(t *testing.T) {
pkg: pkg.Package{
Name: "github.com/googleapis/gnostic",
Version: "v0.5.5",
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h1:9fHAtK0uzzz",
},
},
@@ -405,8 +405,8 @@ func Test_H1Digest(t *testing.T) {
pkg: pkg.Package{
Name: "github.com/googleapis/gnostic",
Version: "v0.5.5",
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h12:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
},
@@ -366,7 +366,7 @@ func extractMetadata(p *spdx.Package, info pkgInfo) (pkg.MetadataType, interface
h1Digest = digest
break
}
return pkg.GolangMetadataType, pkg.GolangMetadata{
return pkg.GolangBinMetadataType, pkg.GolangBinMetadata{
H1Digest: h1Digest,
}
}
@@ -302,8 +302,8 @@ func TestH1Digest(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
p := toSyftPackage(&test.pkg)
require.Equal(t, pkg.GolangMetadataType, p.MetadataType)
meta := p.Metadata.(pkg.GolangMetadata)
require.Equal(t, pkg.GolangBinMetadataType, p.MetadataType)
meta := p.Metadata.(pkg.GolangBinMetadata)
require.Equal(t, test.expectedDigest, meta.H1Digest)
})
}
+9 -9
View File
@@ -34,7 +34,7 @@ func TestUnmarshalPackageGolang(t *testing.T) {
"language": "go",
"cpes": [],
"purl": "pkg:golang/gopkg.in/square/go-jose.v2@v2.6.0",
"metadataType": "GolangMetadata",
"metadataType": "GolangBinMetadata",
"metadata": {
"goCompiledVersion": "go1.18",
"architecture": "amd64",
@@ -43,7 +43,7 @@ func TestUnmarshalPackageGolang(t *testing.T) {
}`),
assert: func(p *Package) {
assert.NotNil(t, p.Metadata)
golangMetadata := p.Metadata.(pkg.GolangMetadata)
golangMetadata := p.Metadata.(pkg.GolangBinMetadata)
assert.NotEmpty(t, golangMetadata)
assert.Equal(t, "go1.18", golangMetadata.GoCompiledVersion)
},
@@ -93,7 +93,7 @@ func Test_unpackMetadata(t *testing.T) {
}{
{
name: "unmarshal package metadata",
metadataType: pkg.GolangMetadataType,
metadataType: pkg.GolangBinMetadataType,
packageData: []byte(`{
"id": "8b594519bc23da50",
"name": "gopkg.in/square/go-jose.v2",
@@ -109,7 +109,7 @@ func Test_unpackMetadata(t *testing.T) {
"language": "go",
"cpes": [],
"purl": "pkg:golang/gopkg.in/square/go-jose.v2@v2.6.0",
"metadataType": "GolangMetadata",
"metadataType": "GolangBinMetadata",
"metadata": {
"goCompiledVersion": "go1.18",
"architecture": "amd64",
@@ -215,18 +215,18 @@ func Test_unpackMetadata(t *testing.T) {
{
name: "can handle package with metadata type but missing metadata",
packageData: []byte(`{
"metadataType": "GolangMetadata"
"metadataType": "GolangBinMetadata"
}`),
metadataType: pkg.GolangMetadataType,
wantMetadata: pkg.GolangMetadata{},
metadataType: pkg.GolangBinMetadataType,
wantMetadata: pkg.GolangBinMetadata{},
},
{
name: "can handle package with golang bin metadata type",
packageData: []byte(`{
"metadataType": "GolangBinMetadata"
}`),
metadataType: pkg.GolangMetadataType,
wantMetadata: pkg.GolangMetadata{},
metadataType: pkg.GolangBinMetadataType,
wantMetadata: pkg.GolangBinMetadata{},
},
{
name: "can handle package with unknonwn metadata type and missing metadata",
@@ -89,7 +89,7 @@
}
},
"schema": {
"version": "6.0.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.0.0.json"
"version": "6.1.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.1.0.json"
}
}
@@ -185,7 +185,7 @@
}
},
"schema": {
"version": "6.1.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.1.0.json"
"version": "6.2.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.2.0.json"
}
}
@@ -9,7 +9,7 @@
"locations": [
{
"path": "/somefile-1.txt",
"layerID": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59"
"layerID": "sha256:6afd1cb55939d87ba4c298907d0a53059bb3742c2d65314643e2464071cf0a2d"
}
],
"licenses": [
@@ -40,7 +40,7 @@
"locations": [
{
"path": "/somefile-2.txt",
"layerID": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec"
"layerID": "sha256:657997cff9a836139186239bdfe77250239a700d0ed97d57e101c295e8244319"
}
],
"licenses": [],
@@ -64,11 +64,11 @@
],
"artifactRelationships": [],
"source": {
"id": "1a678f111c8ddc66fd82687bb024e0dd6af61314404937a80e810c0cf317b796",
"id": "c85f7ae1b0ac38342c1cf1a6f7ea2b4b1ddc49cd1b24219ebd05dc10b3303491",
"type": "image",
"target": {
"userInput": "user-image-input",
"imageID": "sha256:3c51b06feb0cda8ee62d0e3755ef2a8496a6b71f8a55b245f07f31c4bb813d31",
"imageID": "sha256:b5c0bfa8bcf70c75d92ebebbf76af667906d56e6fad50c37e7f93df824a64b79",
"manifestDigest": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"tags": [
@@ -78,17 +78,17 @@
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59",
"digest": "sha256:6afd1cb55939d87ba4c298907d0a53059bb3742c2d65314643e2464071cf0a2d",
"size": 22
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec",
"digest": "sha256:657997cff9a836139186239bdfe77250239a700d0ed97d57e101c295e8244319",
"size": 16
}
],
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NzMsImRpZ2VzdCI6InNoYTI1NjozYzUxYjA2ZmViMGNkYThlZTYyZDBlMzc1NWVmMmE4NDk2YTZiNzFmOGE1NWIyNDVmMDdmMzFjNGJiODEzZDMxIn0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1NjpmYjZiZWVjYjc1YjM5ZjRiYjgxM2RiZjE3N2U1MDFlZGQ1ZGRiM2U2OWJiNDVjZWRlYjc4YzY3NmVlMWI3YTU5In0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjMxOWI1ODhjZTY0MjUzYTg3YjUzM2M4ZWQwMWNmMDAyNWUwZWFjOThlN2I1MTZlMTI1MzI5NTdlMTI0NGZkZWMifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjItMDgtMDFUMjA6MDk6MjIuNTA5NDIxNzEyWiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIyLTA4LTAxVDIwOjA5OjIyLjQ4Nzg5NTUxOVoiLCJjcmVhdGVkX2J5IjoiQUREIGZpbGUtMS50eHQgL3NvbWVmaWxlLTEudHh0ICMgYnVpbGRraXQiLCJjb21tZW50IjoiYnVpbGRraXQuZG9ja2VyZmlsZS52MCJ9LHsiY3JlYXRlZCI6IjIwMjItMDgtMDFUMjA6MDk6MjIuNTA5NDIxNzEyWiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0yLnR4dCAvc29tZWZpbGUtMi50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn1dLCJvcyI6ImxpbnV4Iiwicm9vdGZzIjp7InR5cGUiOiJsYXllcnMiLCJkaWZmX2lkcyI6WyJzaGEyNTY6ZmI2YmVlY2I3NWIzOWY0YmI4MTNkYmYxNzdlNTAxZWRkNWRkYjNlNjliYjQ1Y2VkZWI3OGM2NzZlZTFiN2E1OSIsInNoYTI1NjozMTliNTg4Y2U2NDI1M2E4N2I1MzNjOGVkMDFjZjAwMjVlMGVhYzk4ZTdiNTE2ZTEyNTMyOTU3ZTEyNDRmZGVjIl19fQ==",
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NzMsImRpZ2VzdCI6InNoYTI1NjpiNWMwYmZhOGJjZjcwYzc1ZDkyZWJlYmJmNzZhZjY2NzkwNmQ1NmU2ZmFkNTBjMzdlN2Y5M2RmODI0YTY0Yjc5In0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1Njo2YWZkMWNiNTU5MzlkODdiYTRjMjk4OTA3ZDBhNTMwNTliYjM3NDJjMmQ2NTMxNDY0M2UyNDY0MDcxY2YwYTJkIn0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjY1Nzk5N2NmZjlhODM2MTM5MTg2MjM5YmRmZTc3MjUwMjM5YTcwMGQwZWQ5N2Q1N2UxMDFjMjk1ZTgyNDQzMTkifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjItMDgtMjVUMTY6MjI6MDguODkxMzY0Mjc4WiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIyLTA4LTI1VDE2OjIyOjA4Ljc2MzMzMDMyM1oiLCJjcmVhdGVkX2J5IjoiQUREIGZpbGUtMS50eHQgL3NvbWVmaWxlLTEudHh0ICMgYnVpbGRraXQiLCJjb21tZW50IjoiYnVpbGRraXQuZG9ja2VyZmlsZS52MCJ9LHsiY3JlYXRlZCI6IjIwMjItMDgtMjVUMTY6MjI6MDguODkxMzY0Mjc4WiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0yLnR4dCAvc29tZWZpbGUtMi50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn1dLCJvcyI6ImxpbnV4Iiwicm9vdGZzIjp7InR5cGUiOiJsYXllcnMiLCJkaWZmX2lkcyI6WyJzaGEyNTY6NmFmZDFjYjU1OTM5ZDg3YmE0YzI5ODkwN2QwYTUzMDU5YmIzNzQyYzJkNjUzMTQ2NDNlMjQ2NDA3MWNmMGEyZCIsInNoYTI1Njo2NTc5OTdjZmY5YTgzNjEzOTE4NjIzOWJkZmU3NzI1MDIzOWE3MDBkMGVkOTdkNTdlMTAxYzI5NWU4MjQ0MzE5Il19fQ==",
"repoDigests": [],
"architecture": "",
"os": ""
@@ -112,7 +112,7 @@
}
},
"schema": {
"version": "6.0.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.0.0.json"
"version": "6.1.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.1.0.json"
}
}
+18 -3
View File
@@ -145,22 +145,37 @@ func toSyftDescriptor(d model.Descriptor) sbom.Descriptor {
func toSyftSourceData(s model.Source) *source.Metadata {
switch s.Type {
case "directory":
path, ok := s.Target.(string)
if !ok {
log.Warnf("unable to parse source target as string: %+v", s.Target)
return nil
}
return &source.Metadata{
ID: s.ID,
Scheme: source.DirectoryScheme,
Path: s.Target.(string),
Path: path,
}
case "file":
path, ok := s.Target.(string)
if !ok {
log.Warnf("unable to parse source target as string: %+v", s.Target)
return nil
}
return &source.Metadata{
ID: s.ID,
Scheme: source.FileScheme,
Path: s.Target.(string),
Path: path,
}
case "image":
metadata, ok := s.Target.(source.ImageMetadata)
if !ok {
log.Warnf("unable to parse source target as image metadata: %+v", s.Target)
return nil
}
return &source.Metadata{
ID: s.ID,
Scheme: source.ImageScheme,
ImageMetadata: s.Target.(source.ImageMetadata),
ImageMetadata: metadata,
}
}
return nil
+15 -1
View File
@@ -71,6 +71,14 @@ func parseApkDB(_ source.FileResolver, env *generic.Environment, reader source.L
log.Warnf("unable to parse field data from line %q", line)
continue
}
if len(field.name) == 0 {
log.Warnf("failed to parse field name from line %q", line)
continue
}
if len(field.value) == 0 {
log.Debugf("line %q: parsed field %q appears to have an empty value, skipping", line, field.name)
continue
}
entryParsingInProgress = true
@@ -353,5 +361,11 @@ func stripVersionSpecifier(s string) string {
// examples:
// musl>=1 --> musl
// cmd:scanelf=1.3.4-r0 --> cmd:scanelf
return splitAny(s, "<>=")[0]
items := splitAny(s, "<>=")
if len(items) == 0 {
return s
}
return items[0]
}
@@ -160,6 +160,43 @@ func TestSinglePackageDetails(t *testing.T) {
},
},
},
{
fixture: "test-fixtures/empty-deps-and-provides",
expected: pkg.ApkMetadata{
Package: "alpine-baselayout-data",
OriginPackage: "alpine-baselayout",
Version: "3.4.0-r0",
Description: "Alpine base dir structure and init scripts",
Maintainer: "Natanael Copa <ncopa@alpinelinux.org>",
License: "GPL-2.0-only",
Architecture: "x86_64",
URL: "https://git.alpinelinux.org/cgit/aports/tree/main/alpine-baselayout",
Size: 11664,
InstalledSize: 77824,
Dependencies: []string{},
Provides: []string{},
Checksum: "Q15ffjKT28lB7iSXjzpI/eDdYRCwM=",
GitCommit: "bd965a7ebf7fd8f07d7a0cc0d7375bf3e4eb9b24",
Files: []pkg.ApkFileRecord{
{Path: "/etc"},
{Path: "/etc/fstab"},
{Path: "/etc/group"},
{Path: "/etc/hostname"},
{Path: "/etc/hosts"},
{Path: "/etc/inittab"},
{Path: "/etc/modules"},
{Path: "/etc/mtab", OwnerUID: "0", OwnerGID: "0", Permissions: "0777"},
{Path: "/etc/nsswitch.conf"},
{Path: "/etc/passwd"},
{Path: "/etc/profile"},
{Path: "/etc/protocols"},
{Path: "/etc/services"},
{Path: "/etc/shadow", OwnerUID: "0", OwnerGID: "148", Permissions: "0640"},
{Path: "/etc/shells"},
{Path: "/etc/sysctl.conf"},
},
},
},
{
fixture: "test-fixtures/base",
expected: pkg.ApkMetadata{
@@ -874,6 +911,27 @@ func Test_discoverPackageDependencies(t *testing.T) {
}
},
},
{
name: "strip version specifiers with empty provides value",
genFn: func() ([]pkg.Package, []artifact.Relationship) {
a := pkg.Package{
Name: "package-a",
Metadata: pkg.ApkMetadata{
Dependencies: []string{"so:libc.musl-x86_64.so.1"},
},
}
a.SetID()
b := pkg.Package{
Name: "package-b",
Metadata: pkg.ApkMetadata{
Provides: []string{""},
},
}
b.SetID()
return []pkg.Package{a, b}, nil
},
},
{
name: "depends on package name",
genFn: func() ([]pkg.Package, []artifact.Relationship) {
@@ -1089,3 +1147,42 @@ func newLocationReadCloser(t *testing.T, path string) source.LocationReadCloser
return source.NewLocationReadCloser(source.NewLocation(path), f)
}
func Test_stripVersionSpecifier(t *testing.T) {
tests := []struct {
name string
version string
want string
}{
{
name: "empty expression",
version: "",
want: "",
},
{
name: "no expression",
version: "cmd:foo",
want: "cmd:foo",
},
{
name: "=",
version: "cmd:scanelf=1.3.4-r0",
want: "cmd:scanelf",
},
{
name: ">=",
version: "cmd:scanelf>=1.3.4-r0",
want: "cmd:scanelf",
},
{
name: "<",
version: "cmd:scanelf<1.3.4-r0",
want: "cmd:scanelf",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, stripVersionSpecifier(tt.version))
})
}
}
@@ -0,0 +1,35 @@
P:alpine-baselayout-data
V:3.4.0-r0
A:x86_64
L:GPL-2.0-only
T:Alpine base dir structure and init scripts
o:alpine-baselayout
m:Natanael Copa <ncopa@alpinelinux.org>
U:https://git.alpinelinux.org/cgit/aports/tree/main/alpine-baselayout
D:
p:
c:bd965a7ebf7fd8f07d7a0cc0d7375bf3e4eb9b24
i:[]
t:1667573027
S:11664
I:77824
k:0
C:Q15ffjKT28lB7iSXjzpI/eDdYRCwM=
F:etc
R:fstab
R:group
R:hostname
R:hosts
R:inittab
R:modules
R:mtab
a:0:0:0777
R:nsswitch.conf
R:passwd
R:profile
R:protocols
R:services
R:shadow
a:0:148:0640
R:shells
R:sysctl.conf
+40 -28
View File
@@ -1,6 +1,7 @@
package binary
import (
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
@@ -30,41 +31,52 @@ func (c Cataloger) Catalog(resolver source.FileResolver) ([]pkg.Package, []artif
var packages []pkg.Package
var relationships []artifact.Relationship
for _, classifier := range defaultClassifiers {
locations, err := resolver.FilesByGlob(classifier.FileGlob)
for _, cls := range defaultClassifiers {
pkgs, err := catalog(resolver, cls)
if err != nil {
return nil, nil, err
}
for _, location := range locations {
reader, err := resolver.FileContentsByLocation(location)
if err != nil {
return nil, nil, err
}
locationReader := source.NewLocationReadCloser(location, reader)
newPkgs, err := classifier.EvidenceMatcher(classifier, locationReader)
if err != nil {
return nil, nil, err
}
newPackages:
for i := range newPkgs {
newPkg := &newPkgs[i]
for j := range packages {
p := &packages[j]
// consolidate identical packages found in different locations,
// but continue to track each location
if packagesMatch(p, newPkg) {
p.Locations.Add(newPkg.Locations.ToSlice()...)
continue newPackages
}
}
packages = append(packages, *newPkg)
}
log.WithFields("error", err, "classifier", cls.Class).Warn("unable to catalog binary package: %w", err)
continue
}
packages = append(packages, pkgs...)
}
return packages, relationships, nil
}
func catalog(resolver source.FileResolver, cls classifier) ([]pkg.Package, error) {
var pkgs []pkg.Package
locations, err := resolver.FilesByGlob(cls.FileGlob)
if err != nil {
return nil, err
}
for _, location := range locations {
reader, err := resolver.FileContentsByLocation(location)
if err != nil {
return nil, err
}
locationReader := source.NewLocationReadCloser(location, reader)
newPkgs, err := cls.EvidenceMatcher(cls, locationReader)
if err != nil {
return nil, err
}
newPackages:
for i := range newPkgs {
newPkg := &newPkgs[i]
for j := range pkgs {
p := &pkgs[j]
// consolidate identical packages found in different locations,
// but continue to track each location
if packagesMatch(p, newPkg) {
p.Locations.Add(newPkg.Locations.ToSlice()...)
continue newPackages
}
}
pkgs = append(pkgs, *newPkg)
}
}
return pkgs, nil
}
// packagesMatch returns true if the binary packages "match" based on basic criteria
func packagesMatch(p1 *pkg.Package, p2 *pkg.Package) bool {
if p1.Name != p2.Name ||
+64 -1
View File
@@ -1,6 +1,8 @@
package binary
import (
"errors"
"io"
"testing"
"github.com/stretchr/testify/assert"
@@ -17,6 +19,20 @@ func TestClassifierCataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
fixtureDir string
expected pkg.Package
}{
{
name: "positive-memcached-1.6.18",
fixtureDir: "test-fixtures/classifiers/positive/memcached-1.6.18",
expected: pkg.Package{
Name: "memcached",
Version: "1.6.18",
Type: "binary",
PURL: "pkg:generic/memcached@1.6.18",
Locations: singleLocation("memcached"),
Metadata: pkg.BinaryMetadata{
Classifier: "memcached-binary",
},
},
},
{
name: "positive-httpd-2.4.54",
fixtureDir: "test-fixtures/classifiers/positive/httpd-2.4.54",
@@ -30,7 +46,8 @@ func TestClassifierCataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
Classifier: "httpd-binary",
},
},
}, {
},
{
name: "positive-php-cli-8.2.1",
fixtureDir: "test-fixtures/classifiers/positive/php-cli-8.2.1",
expected: pkg.Package{
@@ -418,3 +435,49 @@ func assertPackagesAreEqual(t *testing.T, expected pkg.Package, p pkg.Package) {
assert.Failf(t, "packages not equal", "%v != %v", expected, p)
}
}
type panicyResolver struct {
globCalled bool
}
func (p panicyResolver) FileContentsByLocation(location source.Location) (io.ReadCloser, error) {
return nil, errors.New("not implemented")
}
func (p panicyResolver) HasPath(s string) bool {
return true
}
func (p panicyResolver) FilesByPath(paths ...string) ([]source.Location, error) {
return nil, errors.New("not implemented")
}
func (p *panicyResolver) FilesByGlob(patterns ...string) ([]source.Location, error) {
p.globCalled = true
return nil, errors.New("not implemented")
}
func (p panicyResolver) FilesByMIMEType(types ...string) ([]source.Location, error) {
return nil, errors.New("not implemented")
}
func (p panicyResolver) RelativeFileByPath(_ source.Location, path string) *source.Location {
return nil
}
func (p panicyResolver) AllLocations() <-chan source.Location {
return nil
}
func (p panicyResolver) FileMetadataByLocation(location source.Location) (source.FileMetadata, error) {
return source.FileMetadata{}, errors.New("not implemented")
}
func Test_Cataloger_ResilientToErrors(t *testing.T) {
c := NewCataloger()
resolver := &panicyResolver{}
_, _, err := c.Catalog(resolver)
assert.NoError(t, err)
assert.True(t, resolver.globCalled)
}
@@ -141,4 +141,12 @@ var defaultClassifiers = []classifier{
PURL: mustPURL("pkg:generic/httpd@version"),
CPEs: singleCPE("cpe:2.3:a:apache:http_server:*:*:*:*:*:*:*:*"),
},
{
Class: "memcached-binary",
FileGlob: "**/memcached",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)memcached\s(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`),
Package: "memcached",
PURL: mustPURL("pkg:generic/memcached@version"),
},
}
@@ -0,0 +1,13 @@
secs_since_last_cmd
listen
memcached 1.6.18
udp-port
memcached 1.6.18
Failed to allocate memory
%s,%s
--
bget
flush_all
VERSION 1.6.18
quit
BADCLASS invalid class id
+10 -2
View File
@@ -81,7 +81,11 @@ func packageURL(m pkg.DpkgMetadata, distro *linux.Release) string {
}
func addLicenses(resolver source.FileResolver, dbLocation source.Location, p *pkg.Package) {
metadata := p.Metadata.(pkg.DpkgMetadata)
metadata, ok := p.Metadata.(pkg.DpkgMetadata)
if !ok {
log.WithFields("package", p.String()).Warn("unable to extract DPKG metadata to add licenses")
return
}
// get license information from the copyright file
copyrightReader, copyrightLocation := fetchCopyrightContents(resolver, dbLocation, metadata)
@@ -97,7 +101,11 @@ func addLicenses(resolver source.FileResolver, dbLocation source.Location, p *pk
}
func mergeFileListing(resolver source.FileResolver, dbLocation source.Location, p *pkg.Package) {
metadata := p.Metadata.(pkg.DpkgMetadata)
metadata, ok := p.Metadata.(pkg.DpkgMetadata)
if !ok {
log.WithFields("package", p.String()).Warn("unable to extract DPKG metadata to file listing")
return
}
// get file listing (package files + additional config files)
files, infoLocations := getAdditionalFileListing(resolver, dbLocation, metadata)
@@ -7,6 +7,7 @@ import (
"io"
"regexp"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
@@ -42,7 +43,12 @@ loop:
if len(tokens) < 5 {
name, hash := tokens[1], tokens[2]
sourcePkg := pkgMap[name]
metadata := sourcePkg.Metadata.(pkg.RebarLockMetadata)
metadata, ok := sourcePkg.Metadata.(pkg.RebarLockMetadata)
if !ok {
log.WithFields("package", name).Warn("unable to extract rebar.lock metadata to add hash metadata")
continue
}
if metadata.PkgHash == "" {
metadata.PkgHash = hash
} else {
+9 -6
View File
@@ -28,7 +28,8 @@ func (c *Cataloger) WithParserByGlobs(parser Parser, globs ...string) *Cataloger
func(resolver source.FileResolver, env Environment) []request {
var requests []request
for _, g := range globs {
// TODO: add more trace logging here
log.WithFields("glob", g).Trace("searching for paths matching glob")
matches, err := resolver.FilesByGlob(g)
if err != nil {
log.Warnf("unable to process glob=%q: %+v", g, err)
@@ -47,7 +48,8 @@ func (c *Cataloger) WithParserByMimeTypes(parser Parser, types ...string) *Catal
func(resolver source.FileResolver, env Environment) []request {
var requests []request
for _, t := range types {
// TODO: add more trace logging here
log.WithFields("mimetype", t).Trace("searching for paths matching mimetype")
matches, err := resolver.FilesByMIMEType(t)
if err != nil {
log.Warnf("unable to process mimetype=%q: %+v", t, err)
@@ -65,11 +67,12 @@ func (c *Cataloger) WithParserByPath(parser Parser, paths ...string) *Cataloger
c.processor = append(c.processor,
func(resolver source.FileResolver, env Environment) []request {
var requests []request
for _, g := range paths {
// TODO: add more trace logging here
matches, err := resolver.FilesByPath(g)
for _, p := range paths {
log.WithFields("path", p).Trace("searching for path")
matches, err := resolver.FilesByPath(p)
if err != nil {
log.Warnf("unable to process path=%q: %+v", g, err)
log.Warnf("unable to process path=%q: %+v", p, err)
continue
}
requests = append(requests, makeRequests(parser, matches)...)
+2 -2
View File
@@ -22,8 +22,8 @@ func newGoBinaryPackage(dep *debug.Module, mainModule, goVersion, architecture s
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: source.NewLocationSet(locations...),
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goVersion,
H1Digest: dep.Sum,
Architecture: architecture,
@@ -143,8 +143,8 @@ func TestBuildGoPkgInfo(t *testing.T) {
},
},
),
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
BuildSettings: defaultBuildSettings,
@@ -190,8 +190,8 @@ func TestBuildGoPkgInfo(t *testing.T) {
},
},
),
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{},
},
},
},
@@ -233,8 +233,8 @@ func TestBuildGoPkgInfo(t *testing.T) {
},
},
),
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:VSVdnH7cQ7V+B33qSJHTCRlNgra1607Q8PzEmnvb2Ic=",
@@ -285,8 +285,8 @@ func TestBuildGoPkgInfo(t *testing.T) {
},
},
),
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
BuildSettings: map[string]string{
@@ -340,8 +340,8 @@ func TestBuildGoPkgInfo(t *testing.T) {
},
},
),
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:VSVdnH7cQ7V+B33qSJHTCRlNgra1607Q8PzEmnvb2Ic=",
@@ -362,8 +362,8 @@ func TestBuildGoPkgInfo(t *testing.T) {
},
},
),
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=",
@@ -417,8 +417,8 @@ func TestBuildGoPkgInfo(t *testing.T) {
},
},
),
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:PjhxBct4MZii8FFR8+oeS7QOvxKOTZXgk63EU2XpfJE=",
@@ -438,8 +438,8 @@ func TestBuildGoPkgInfo(t *testing.T) {
},
},
),
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:Ihq/mm/suC88gF8WFcVwk+OV6Tq+wyA1O0E5UEvDglI=",
+4 -4
View File
@@ -43,8 +43,8 @@ func parseGoModFile(resolver source.FileResolver, _ *generic.Environment, reader
PURL: packageURL(m.Mod.Path, m.Mod.Version),
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{
H1Digest: digests[fmt.Sprintf("%s %s", m.Mod.Path, m.Mod.Version)],
},
}
@@ -59,8 +59,8 @@ func parseGoModFile(resolver source.FileResolver, _ *generic.Environment, reader
PURL: packageURL(m.New.Path, m.New.Version),
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{
H1Digest: digests[fmt.Sprintf("%s %s", m.New.Path, m.New.Version)],
},
}
+18 -18
View File
@@ -23,8 +23,8 @@ func TestParseGoMod(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("test-fixtures/one-package")),
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{},
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{},
},
},
},
@@ -39,8 +39,8 @@ func TestParseGoMod(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("test-fixtures/many-packages")),
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{},
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{},
},
{
Name: "github.com/anchore/go-version",
@@ -49,8 +49,8 @@ func TestParseGoMod(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("test-fixtures/many-packages")),
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{},
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{},
},
{
Name: "github.com/anchore/stereoscope",
@@ -59,8 +59,8 @@ func TestParseGoMod(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("test-fixtures/many-packages")),
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{},
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{},
},
{
Name: "github.com/bmatcuk/doublestar",
@@ -69,8 +69,8 @@ func TestParseGoMod(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("test-fixtures/many-packages")),
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{},
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{},
},
{
Name: "github.com/go-test/deep",
@@ -79,8 +79,8 @@ func TestParseGoMod(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("test-fixtures/many-packages")),
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{},
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{},
},
},
},
@@ -112,8 +112,8 @@ func Test_GoSumHashes(t *testing.T) {
FoundBy: "go-mod-file-cataloger",
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{},
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{},
},
{
Name: "github.com/acarl005/stripansi",
@@ -123,8 +123,8 @@ func Test_GoSumHashes(t *testing.T) {
FoundBy: "go-mod-file-cataloger",
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{
H1Digest: "h1:licZJFw2RwpHMqeKTCYkitsPqHNxTmd4SNR5r94FGM8=",
},
},
@@ -136,8 +136,8 @@ func Test_GoSumHashes(t *testing.T) {
FoundBy: "go-mod-file-cataloger",
Language: pkg.Go,
Type: pkg.GoModulePkg,
MetadataType: pkg.GolangMetadataType,
Metadata: pkg.GolangMetadata{
MetadataType: pkg.GolangModMetadataType,
Metadata: pkg.GolangModMetadata{
H1Digest: "h1:5PJl274Y63IEHC+7izoQE9x6ikvDFZS2mDVS3drnohI=",
},
},
+2 -1
View File
@@ -53,7 +53,8 @@ func getBuildInfo(r io.ReaderAt) (bi *debug.BuildInfo, err error) {
if err != nil {
if err.Error() == "not a Go executable" {
// since the cataloger can only select executables and not distinguish if they are a go-compiled
// binary, we should not show warnings/logs in this case.
// binary, we should not show warnings/logs in this case. For this reason we nil-out err here.
err = nil
return
}
// in this case we could not read the or parse the file, but not explicitly because it is not a
+20 -4
View File
@@ -133,8 +133,14 @@ func (j *archiveParser) parse() ([]pkg.Package, []artifact.Relationship, error)
// add pURLs to all packages found
// note: since package information may change after initial creation when parsing multiple locations within the
// jar, we wait until the conclusion of the parsing process before synthesizing pURLs.
for i, p := range pkgs {
pkgs[i].PURL = packageURL(p.Name, p.Version, p.Metadata.(pkg.JavaMetadata))
for i := range pkgs {
p := &pkgs[i]
if m, ok := p.Metadata.(pkg.JavaMetadata); ok {
p.PURL = packageURL(p.Name, p.Version, m)
} else {
log.WithFields("package", p.String()).Warn("unable to extract java metadata to generate purl")
}
p.SetID()
}
return pkgs, relationships, nil
@@ -401,10 +407,20 @@ func packageIdentitiesMatch(p pkg.Package, parentPkg *pkg.Package) bool {
return true
}
metadata := p.Metadata.(pkg.JavaMetadata)
metadata, ok := p.Metadata.(pkg.JavaMetadata)
if !ok {
log.WithFields("package", p.String()).Warn("unable to extract java metadata to check for matching package identity")
return false
}
parentMetadata, ok := parentPkg.Metadata.(pkg.JavaMetadata)
if !ok {
log.WithFields("package", p.String()).Warn("unable to extract java metadata from parent for verifying virtual path")
return false
}
// the virtual path matches...
if parentPkg.Metadata.(pkg.JavaMetadata).VirtualPath == metadata.VirtualPath {
if parentMetadata.VirtualPath == metadata.VirtualPath {
return true
}
@@ -277,6 +277,10 @@ func TestParseJar(t *testing.T) {
}
for _, a := range actual {
if a.ID() == "" {
t.Fatalf("empty package ID: %+v", a)
}
e, ok := test.expected[a.Name]
if !ok {
t.Errorf("entry not found: %s", a.Name)
@@ -115,12 +115,12 @@ func (c *NativeImageCataloger) Name() string {
func getPackage(component nativeImageComponent) pkg.Package {
var cpes []cpe.CPE
for _, property := range component.Properties {
cpe, err := cpe.New(property.Value)
c, err := cpe.New(property.Value)
if err != nil {
log.Debugf("native-image cataloger: could not parse CPE: %v.", err)
log.Debugf("unable to parse CPE: %v", err)
continue
}
cpes = append(cpes, cpe)
cpes = append(cpes, c)
}
return pkg.Package{
Name: component.Name,
@@ -139,59 +139,59 @@ func getPackage(component nativeImageComponent) pkg.Package {
}
// decompressSbom returns the packages given within a native image executable's SBOM.
func decompressSbom(databuf []byte, sbomStart uint64, lengthStart uint64) ([]pkg.Package, error) {
func decompressSbom(dataBuf []byte, sbomStart uint64, lengthStart uint64) ([]pkg.Package, error) {
var pkgs []pkg.Package
lengthEnd := lengthStart + 8
buflen := len(databuf)
if lengthEnd > uint64(buflen) {
return nil, errors.New("the sbom_length symbol overflows the binary")
bufLen := len(dataBuf)
if lengthEnd > uint64(bufLen) {
return nil, errors.New("the 'sbom_length' symbol overflows the binary")
}
length := databuf[lengthStart:lengthEnd]
length := dataBuf[lengthStart:lengthEnd]
p := bytes.NewBuffer(length)
var storedLength uint64
err := binary.Read(p, binary.LittleEndian, &storedLength)
if err != nil {
log.Debugf("native-image-cataloger: could not read from binary file.")
return nil, err
return nil, fmt.Errorf("could not read from binary file: %w", err)
}
log.Tracef("native-image cataloger: found SBOM of length %d.", storedLength)
log.WithFields("len", storedLength).Trace("found java native-image SBOM")
sbomEnd := sbomStart + storedLength
if sbomEnd > uint64(buflen) {
if sbomEnd > uint64(bufLen) {
return nil, errors.New("the sbom symbol overflows the binary")
}
sbomCompressed := databuf[sbomStart:sbomEnd]
sbomCompressed := dataBuf[sbomStart:sbomEnd]
p = bytes.NewBuffer(sbomCompressed)
gzreader, err := gzip.NewReader(p)
if err != nil {
log.Debugf("native-image cataloger: could not decompress the SBOM.")
return nil, err
return nil, fmt.Errorf("could not decompress the java native-image SBOM: %w", err)
}
output, err := io.ReadAll(gzreader)
if err != nil {
log.Debugf("native-image cataloger: could not read the decompressed SBOM.")
return nil, err
return nil, fmt.Errorf("could not read the java native-image SBOM: %w", err)
}
var sbomContent nativeImageCycloneDX
err = json.Unmarshal(output, &sbomContent)
if err != nil {
log.Debugf("native-image cataloger: could not unmarshal JSON.")
return nil, err
return nil, fmt.Errorf("could not unmarshal the java native-image SBOM: %w", err)
}
for _, component := range sbomContent.Components {
p := getPackage(component)
pkgs = append(pkgs, p)
}
return pkgs, nil
}
// fileError logs an error message when an executable cannot be read.
func fileError(filename string, err error) (nativeImage, error) {
// We could not read the file as a binary for the desired platform, but it may still be a native-image executable.
log.Debugf("native-image cataloger: unable to read executable (file=%q): %v.", filename, err)
return nil, err
return nil, fmt.Errorf("unable to read executable (file=%q): %w", filename, err)
}
// newElf reads a Native Image from an ELF executable.
@@ -199,10 +199,18 @@ func newElf(filename string, r io.ReaderAt) (nativeImage, error) {
// First attempt to read an ELF file.
bi, err := elf.NewFile(r)
// The reader does not refer to an ELF file.
if err != nil {
var fmtErr *elf.FormatError
if errors.As(err, &fmtErr) {
// this is not an elf file
log.WithFields("filename", filename, "error", err).Trace("not an ELF binary")
return nil, nil
}
return fileError(filename, err)
}
if bi == nil {
return nil, nil
}
return nativeImageElf{
file: bi,
}, nil
@@ -213,9 +221,16 @@ func newMachO(filename string, r io.ReaderAt) (nativeImage, error) {
// First attempt to read an ELF file.
bi, err := macho.NewFile(r)
// The reader does not refer to an MachO file.
if err != nil {
return fileError(filename, err)
var fmtErr *macho.FormatError
if errors.As(err, &fmtErr) {
// this is not a MachO file
log.WithFields("filename", filename, "error", err).Trace("not a MachO binary")
return nil, nil
}
}
if bi == nil {
return nil, nil
}
return nativeImageMachO{
file: bi,
@@ -224,15 +239,28 @@ func newMachO(filename string, r io.ReaderAt) (nativeImage, error) {
// newPE reads a Native Image from a Portable Executable file.
func newPE(filename string, r io.ReaderAt) (nativeImage, error) {
// First attempt to read an ELF file.
// First attempt to read an PE file.
bi, err := pe.NewFile(r)
// The reader does not refer to an MachO file.
// The reader does not refer to a PE file.
if err != nil {
return fileError(filename, err)
// note: there isn't a good way to distinguish between a format error and other kinds of errors
log.WithFields("filename", filename, "error", err).Trace("not a PE binary")
return nil, nil
}
if bi == nil {
return nil, nil
}
var exportSymbolsDataDirectory pe.DataDirectory
switch h := bi.OptionalHeader.(type) {
case *pe.OptionalHeader32:
exportSymbolsDataDirectory = h.DataDirectory[0]
case *pe.OptionalHeader64:
exportSymbolsDataDirectory = h.DataDirectory[0]
default:
return nil, fmt.Errorf("unable to get 'exportSymbolsDataDirectory' from binary: %s", filename)
}
optionalHeader := bi.OptionalHeader.(*pe.OptionalHeader64)
exportSymbolsDataDirectory := optionalHeader.DataDirectory[0]
// If we have no exported symbols it is not a Native Image
if exportSymbolsDataDirectory.Size == 0 {
return fileError(filename, errors.New(nativeImageMissingExportedDataDirectoryError))
@@ -241,8 +269,7 @@ func newPE(filename string, r io.ReaderAt) (nativeImage, error) {
exports := make([]byte, exportSymbolsDataDirectory.Size)
_, err = r.ReadAt(exports, int64(exportSymbolsOffset))
if err != nil {
log.Debugf("native-image cataloger: could not read the exported symbols data directory: %v.", err)
return fileError(filename, err)
return fileError(filename, fmt.Errorf("could not read the exported symbols data directory: %w", err))
}
return nativeImagePE{
file: bi,
@@ -266,7 +293,15 @@ func newPE(filename string, r io.ReaderAt) (nativeImage, error) {
}
// fetchPkgs obtains the packages given in the binary.
func (ni nativeImageElf) fetchPkgs() ([]pkg.Package, error) {
func (ni nativeImageElf) fetchPkgs() (pkgs []pkg.Package, retErr error) {
defer func() {
if r := recover(); r != nil {
// this can happen in cases where a malformed binary is passed in can be initially parsed, but not
// used without error later down the line.
retErr = fmt.Errorf("recovered from panic: %v", r)
}
}()
bi := ni.file
var sbom elf.Symbol
var sbomLength elf.Symbol
@@ -274,8 +309,10 @@ func (ni nativeImageElf) fetchPkgs() ([]pkg.Package, error) {
si, err := bi.Symbols()
if err != nil {
log.Debugf("native-image cataloger: no symbols found.")
return nil, err
return nil, fmt.Errorf("no symbols found in binary: %w", err)
}
if si == nil {
return nil, errors.New(nativeImageMissingSymbolsError)
}
for _, s := range si {
switch s.Name {
@@ -288,19 +325,16 @@ func (ni nativeImageElf) fetchPkgs() ([]pkg.Package, error) {
}
}
if sbom.Value == 0 || sbomLength.Value == 0 || svmVersion.Value == 0 {
log.Debugf("native-image cataloger: %v", nativeImageMissingSymbolsError)
return nil, errors.New(nativeImageMissingSymbolsError)
}
dataSection := bi.Section(".data")
if dataSection == nil {
log.Debugf("native-image cataloger: .data section missing from ELF file.")
return nil, err
return nil, fmt.Errorf("no .data section found in binary: %w", err)
}
dataSectionBase := dataSection.SectionHeader.Addr
data, err := dataSection.Data()
if err != nil {
log.Debugf("native-image cataloger: cannot read the .data section.")
return nil, err
return nil, fmt.Errorf("cannot read the .data section: %w", err)
}
sbomLocation := sbom.Value - dataSectionBase
lengthLocation := sbomLength.Value - dataSectionBase
@@ -309,12 +343,23 @@ func (ni nativeImageElf) fetchPkgs() ([]pkg.Package, error) {
}
// fetchPkgs obtains the packages from a Native Image given as a Mach O file.
func (ni nativeImageMachO) fetchPkgs() ([]pkg.Package, error) {
func (ni nativeImageMachO) fetchPkgs() (pkgs []pkg.Package, retErr error) {
defer func() {
if r := recover(); r != nil {
// this can happen in cases where a malformed binary is passed in can be initially parsed, but not
// used without error later down the line.
retErr = fmt.Errorf("recovered from panic: %v", r)
}
}()
var sbom macho.Symbol
var sbomLength macho.Symbol
var svmVersion macho.Symbol
bi := ni.file
if bi.Symtab == nil {
return nil, errors.New(nativeImageMissingSymbolsError)
}
for _, s := range bi.Symtab.Syms {
switch s.Name {
case "_" + nativeImageSbomSymbol:
@@ -326,7 +371,6 @@ func (ni nativeImageMachO) fetchPkgs() ([]pkg.Package, error) {
}
}
if sbom.Value == 0 || sbomLength.Value == 0 || svmVersion.Value == 0 {
log.Debugf("native-image cataloger: %v.", nativeImageMissingSymbolsError)
return nil, errors.New(nativeImageMissingSymbolsError)
}
@@ -334,15 +378,15 @@ func (ni nativeImageMachO) fetchPkgs() ([]pkg.Package, error) {
if dataSegment == nil {
return nil, nil
}
databuf, err := dataSegment.Data()
dataBuf, err := dataSegment.Data()
if err != nil {
log.Debugf("native-image cataloger: cannot obtain buffer from data segment.")
log.Tracef("cannot obtain buffer from data segment")
return nil, nil
}
sbomLocation := sbom.Value - dataSegment.Addr
lengthLocation := sbomLength.Value - dataSegment.Addr
return decompressSbom(databuf, sbomLocation, lengthLocation)
return decompressSbom(dataBuf, sbomLocation, lengthLocation)
}
// fetchExportAttribute obtains an attribute from the exported symbols directory entry.
@@ -351,13 +395,13 @@ func (ni nativeImagePE) fetchExportAttribute(i int) (uint32, error) {
n := len(ni.exports)
j := int(unsafe.Sizeof(ni.header)) + i*int(unsafe.Sizeof(ni.t.headerAttribute))
if j+4 >= n {
log.Debugf("native-image cataloger: invalid index to export directory entry attribute: %v.", j)
log.Tracef("invalid index to export directory entry attribute: %v", j)
return uint32(0), errors.New(nativeImageInvalidIndexError)
}
p := bytes.NewBuffer(ni.exports[j : j+4])
err := binary.Read(p, binary.LittleEndian, &attribute)
if err != nil {
log.Debugf("native-image cataloger: error fetching export directory entry attribute: %v.", err)
log.Tracef("error fetching export directory entry attribute: %v", err)
return uint32(0), err
}
return attribute, nil
@@ -371,13 +415,13 @@ func (ni nativeImagePE) fetchExportFunctionPointer(functionsBase uint32, i uint3
sz := uint32(unsafe.Sizeof(ni.t.functionPointer))
j := functionsBase + i*sz
if j+sz >= n {
log.Debugf("native-image cataloger: invalid index to exported function: %v.", j)
log.Tracef("invalid index to exported function: %v", j)
return uint32(0), errors.New(nativeImageInvalidIndexError)
}
p := bytes.NewBuffer(ni.exports[j : j+sz])
err := binary.Read(p, binary.LittleEndian, &pointer)
if err != nil {
log.Debugf("native-image cataloger: error fetching exported function: %v.", err)
log.Tracef("error fetching exported function: %v", err)
return uint32(0), err
}
return pointer, nil
@@ -389,23 +433,19 @@ func (ni nativeImagePE) fetchExportContent() (*exportContentPE, error) {
var err error
content.numberOfFunctions, err = ni.fetchExportAttribute(0)
if err != nil {
log.Debugf("native-image cataloger: could not find the number of exported functions attribute: %v", err)
return nil, err
return nil, fmt.Errorf("could not find the number of exported 'number of functions' attribute: %w", err)
}
content.numberOfNames, err = ni.fetchExportAttribute(1)
if err != nil {
log.Debugf("native-image cataloger: could not find the number of exported names attribute: %v", err)
return nil, err
return nil, fmt.Errorf("could not find the number of exported 'number of names' attribute: %w", err)
}
content.addressOfFunctions, err = ni.fetchExportAttribute(2)
if err != nil {
log.Debugf("native-image cataloger: could not find the exported functions attribute: %v", err)
return nil, err
return nil, fmt.Errorf("could not find the exported 'address of functions' attribute: %w", err)
}
content.addressOfNames, err = ni.fetchExportAttribute(3)
if err != nil {
log.Debugf("native-image cataloger: could not find the exported names attribute: %v", err)
return nil, err
return nil, fmt.Errorf("could not find the exported 'address of names' attribute: %w", err)
}
return content, nil
}
@@ -425,7 +465,7 @@ func (ni nativeImagePE) fetchSbomSymbols(content *exportContentPE) {
k := addressBase + j
sz := uint32(unsafe.Sizeof(ni.t.namePointer))
if k+sz >= n {
log.Debugf("native-image cataloger: invalid index to exported function: %v.", k)
log.Tracef("invalid index to exported function: %v", k)
// If we are at the end of exports, stop looking
return
}
@@ -433,12 +473,12 @@ func (ni nativeImagePE) fetchSbomSymbols(content *exportContentPE) {
p := bytes.NewBuffer(ni.exports[k : k+sz])
err := binary.Read(p, binary.LittleEndian, &symbolAddress)
if err != nil {
log.Debugf("native-image cataloger: error fetching address of symbol %v.", err)
log.Tracef("error fetching address of symbol %v", err)
return
}
symbolBase := symbolAddress - ni.exportSymbols.VirtualAddress
if symbolBase >= n {
log.Debugf("native-image cataloger: invalid index to exported symbol: %v.", symbolBase)
log.Tracef("invalid index to exported symbol: %v", symbolBase)
return
}
switch {
@@ -453,70 +493,78 @@ func (ni nativeImagePE) fetchSbomSymbols(content *exportContentPE) {
}
// fetchPkgs obtains the packages from a Native Image given as a PE file.
func (ni nativeImagePE) fetchPkgs() ([]pkg.Package, error) {
func (ni nativeImagePE) fetchPkgs() (pkgs []pkg.Package, retErr error) {
defer func() {
if r := recover(); r != nil {
// this can happen in cases where a malformed binary is passed in can be initially parsed, but not
// used without error later down the line.
retErr = fmt.Errorf("recovered from panic: %v", r)
}
}()
content, err := ni.fetchExportContent()
if err != nil {
log.Debugf("native-image cataloger: could not fetch the content of the export directory entry: %v.", err)
log.Debugf("could not fetch the content of the export directory entry: %v", err)
return nil, err
}
ni.fetchSbomSymbols(content)
if content.addressOfSbom == uint32(0) || content.addressOfSbomLength == uint32(0) || content.addressOfSvmVersion == uint32(0) {
log.Debugf("native-image cataloger: %v.", nativeImageMissingSymbolsError)
return nil, errors.New(nativeImageMissingSymbolsError)
}
functionsBase := content.addressOfFunctions - ni.exportSymbols.VirtualAddress
sbomOffset := content.addressOfSbom
sbomAddress, err := ni.fetchExportFunctionPointer(functionsBase, sbomOffset)
if err != nil {
log.Debugf("native-image cataloger: cannot fetch SBOM pointer from exported functions: %v.", err)
return nil, err
return nil, fmt.Errorf("could not fetch SBOM pointer from exported functions: %w", err)
}
sbomLengthOffset := content.addressOfSbomLength
sbomLengthAddress, err := ni.fetchExportFunctionPointer(functionsBase, sbomLengthOffset)
if err != nil {
log.Debugf("native-image cataloger: cannot fetch SBOM length pointer from exported functions: %v.", err)
return nil, err
return nil, fmt.Errorf("could not fetch SBOM length pointer from exported functions: %w", err)
}
bi := ni.file
dataSection := bi.Section(".data")
if dataSection == nil {
return nil, nil
}
databuf, err := dataSection.Data()
dataBuf, err := dataSection.Data()
if err != nil {
log.Debugf("native-image cataloger: cannot obtain buffer from .data section.")
log.Tracef("cannot obtain buffer from the java native-image .data section")
return nil, nil
}
sbomLocation := sbomAddress - dataSection.VirtualAddress
lengthLocation := sbomLengthAddress - dataSection.VirtualAddress
return decompressSbom(databuf, uint64(sbomLocation), uint64(lengthLocation))
return decompressSbom(dataBuf, uint64(sbomLocation), uint64(lengthLocation))
}
// fetchPkgs provides the packages available in a UnionReader.
func fetchPkgs(reader unionreader.UnionReader, filename string) []pkg.Package {
var pkgs []pkg.Package
imageformats := []func(string, io.ReaderAt) (nativeImage, error){newElf, newMachO, newPE}
imageFormats := []func(string, io.ReaderAt) (nativeImage, error){newElf, newMachO, newPE}
// NOTE: multiple readers are returned to cover universal binaries, which are files
// with more than one binary
readers, err := unionreader.GetReaders(reader)
if err != nil {
log.Debugf("native-image cataloger: failed to open a binary: %v.", err)
log.Debugf("failed to open the java native-image binary: %v", err)
return nil
}
for _, r := range readers {
for _, makeNativeImage := range imageformats {
for _, makeNativeImage := range imageFormats {
ni, err := makeNativeImage(filename, r)
if err != nil {
continue
}
newpkgs, err := ni.fetchPkgs()
if err != nil {
log.Debugf("native-image cataloger: error extracting SBOM from %s: %v.", filename, err)
if ni == nil {
continue
}
pkgs = append(pkgs, newpkgs...)
newPkgs, err := ni.fetchPkgs()
if err != nil {
log.Tracef("unable to extract SBOM from possible java native-image %s: %v", filename, err)
continue
}
pkgs = append(pkgs, newPkgs...)
}
}
return pkgs
@@ -533,16 +581,16 @@ func (c *NativeImageCataloger) Catalog(resolver source.FileResolver) ([]pkg.Pack
for _, location := range fileMatches {
readerCloser, err := resolver.FileContentsByLocation(location)
if err != nil {
log.Debugf("native-image cataloger: error opening file: %v.", err)
log.Debugf("error opening file: %v", err)
continue
}
log.Tracef("native-image cataloger: found an executable file %v.", location)
reader, err := unionreader.GetUnionReader(readerCloser)
if err != nil {
return nil, nil, err
}
newpkgs := fetchPkgs(reader, location.RealPath)
pkgs = append(pkgs, newpkgs...)
newPkgs := fetchPkgs(reader, location.RealPath)
pkgs = append(pkgs, newPkgs...)
internal.CloseAndLogError(readerCloser, location.RealPath)
}
@@ -21,14 +21,20 @@ import (
func TestParseNativeImage(t *testing.T) {
tests := []struct {
fixture string
newFn func(filename string, r io.ReaderAt) (nativeImage, error)
}{
{
fixture: "test-fixtures/java-builds/packages/example-java-app",
fixture: "example-java-app",
newFn: newElf,
},
{
fixture: "gcc-amd64-darwin-exec-debug",
newFn: newMachO,
},
}
for _, test := range tests {
t.Run(path.Base(test.fixture), func(t *testing.T) {
f, err := os.Open(test.fixture)
t.Run(test.fixture, func(t *testing.T) {
f, err := os.Open("test-fixtures/java-builds/packages/" + test.fixture)
assert.NoError(t, err)
readerCloser := io.ReadCloser(ioutil.NopCloser(f))
reader, err := unionreader.GetUnionReader(readerCloser)
@@ -36,7 +42,7 @@ func TestParseNativeImage(t *testing.T) {
parsed := false
readers, err := unionreader.GetReaders(reader)
for _, r := range readers {
ni, err := newElf(test.fixture, r)
ni, err := test.newFn(test.fixture, r)
assert.NoError(t, err)
_, err = ni.fetchPkgs()
if err == nil {
@@ -108,12 +114,12 @@ func TestParseNativeImageSbom(t *testing.T) {
z := gzip.NewWriter(writebytes)
_, err = z.Write(sbom)
assert.NoError(t, err)
z.Close()
writebytes.Flush()
_ = z.Close()
_ = writebytes.Flush()
compressedsbom := b.Bytes()
sbomlength := uint64(len(compressedsbom))
binary.Write(writebytes, binary.LittleEndian, sbomlength)
writebytes.Flush()
_ = binary.Write(writebytes, binary.LittleEndian, sbomlength)
_ = writebytes.Flush()
compressedsbom = b.Bytes()
actual, err := decompressSbom(compressedsbom, 0, sbomlength)
assert.NoError(t, err)
@@ -17,7 +17,7 @@ jars: $(PKGSDIR)/example-java-app-maven-0.1.0.jar $(PKGSDIR)/example-java-app-gr
archives: $(PKGSDIR)/example-java-app-maven-0.1.0.zip $(PKGSDIR)/example-java-app-maven-0.1.0.tar $(PKGSDIR)/example-java-app-maven-0.1.0.tar.gz
native-image: $(PKGSDIR)/example-java-app
native-image: $(PKGSDIR)/example-java-app $(PKGSDIR)/gcc-amd64-darwin-exec-debug
# jars within archives...
@@ -68,8 +68,11 @@ clean-jenkins:
$(PKGSDIR)/example-java-app: $(PKGSDIR)/example-java-app-maven-0.1.0.jar
./build-example-java-app-native-image.sh $(PKGSDIR)
$(PKGSDIR)/gcc-amd64-darwin-exec-debug:
./build-example-macho-binary.sh $(PKGSDIR)
# we need a way to determine if CI should bust the test cache based on the source material
$(PKGSDIR).fingerprint: clean-examples
mkdir -p $(PKGSDIR)
find example-* -type f -exec sha256sum {} \; | sort | tee /dev/stderr | tee $(PKGSDIR).fingerprint
find example-* build-* Makefile -type f -exec sha256sum {} \; | sort | tee /dev/stderr | tee $(PKGSDIR).fingerprint
sha256sum $(PKGSDIR).fingerprint
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
set -uxe
PKGSDIR=$1
curl https://raw.githubusercontent.com/blacktop/go-macho/master/internal/testdata/gcc-amd64-darwin-exec-debug.base64 |
base64 -d > $PKGSDIR/gcc-amd64-darwin-exec-debug
+2 -2
View File
@@ -78,8 +78,8 @@ func newPackageLockV1Package(resolver source.FileResolver, location source.Locat
func newPackageLockV2Package(resolver source.FileResolver, location source.Location, name string, u lockPackage) pkg.Package {
var licenses []string
if u.License != "" {
licenses = append(licenses, u.License)
if u.License != nil {
licenses = u.License
}
return finalizeLockPkg(
@@ -7,6 +7,7 @@ import (
"io"
"strings"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
@@ -24,6 +25,39 @@ type packageLock struct {
Packages map[string]lockPackage
}
// packageLockLicense
type packageLockLicense []string
func (licenses *packageLockLicense) UnmarshalJSON(data []byte) (err error) {
// The license field could be either a string or an array.
// 1. An array
var arr []string
if err := json.Unmarshal(data, &arr); err == nil {
*licenses = arr
return nil
}
// 2. A string
var str string
if err = json.Unmarshal(data, &str); err == nil {
*licenses = make([]string, 1)
(*licenses)[0] = str
return nil
}
// debug the content we did not expect
if len(data) > 0 {
log.WithFields("license", string(data)).Debug("Unable to parse the following `license` value in package-lock.json")
}
// 3. Unexpected
// In case we are unable to parse the license field,
// i.e if we have not covered the full specification,
// we do not want to throw an error, instead assign nil.
return nil
}
// lockDependency represents a single package dependency listed in the package.lock json file
type lockDependency struct {
Version string `json:"version"`
@@ -32,11 +66,11 @@ type lockDependency struct {
}
type lockPackage struct {
Name string `json:"name"` // only present in the root package entry (named "")
Version string `json:"version"`
Resolved string `json:"resolved"`
Integrity string `json:"integrity"`
License string `json:"license"`
Name string `json:"name"` // only present in the root package entry (named "")
Version string `json:"version"`
Resolved string `json:"resolved"`
Integrity string `json:"integrity"`
License packageLockLicense `json:"license"`
}
// parsePackageLock parses a package-lock.json and returns the discovered JavaScript packages.
@@ -297,3 +297,44 @@ func TestParsePackageLockAlias(t *testing.T) {
pkgtest.TestFileParser(t, packageLock, parsePackageLock, expected, expectedRelationships)
}
}
func TestParsePackageLockLicenseWithArray(t *testing.T) {
fixture := "test-fixtures/pkg-lock/array-license-package-lock.json"
var expectedRelationships []artifact.Relationship
expectedPkgs := []pkg.Package{
{
Name: "tmp",
Version: "1.0.0",
Licenses: []string{"ISC"},
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/tmp@1.0.0",
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{},
},
{
Name: "pause-stream",
Version: "0.0.11",
Licenses: []string{"MIT", "Apache2"},
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/pause-stream@0.0.11",
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{},
},
{
Name: "through",
Version: "2.3.8",
Licenses: []string{"MIT"},
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/through@2.3.8",
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{},
},
}
for i := range expectedPkgs {
expectedPkgs[i].Locations.Add(source.NewLocation(fixture))
}
pkgtest.TestFileParser(t, fixture, parsePackageLock, expectedPkgs, expectedRelationships)
}
@@ -0,0 +1,41 @@
{
"name": "tmp",
"version": "1.0.0",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "tmp",
"version": "1.0.0",
"license": "ISC",
"dependencies": {
"pause-stream": "0.0.11"
}
},
"node_modules/pause-stream": {
"version": "0.0.11",
"license": [
"MIT",
"Apache2"
],
"dependencies": {
"through": "~2.3"
}
},
"node_modules/through": {
"version": "2.3.8",
"license": "MIT"
}
},
"dependencies": {
"pause-stream": {
"version": "0.0.11",
"requires": {
"through": "~2.3"
}
},
"through": {
"version": "2.3.8"
}
}
}
+7 -2
View File
@@ -1,10 +1,15 @@
package pkg
// GolangMetadata represents all captured data for a Golang Binary
type GolangMetadata struct {
// GolangBinMetadata represents all captured data for a Golang binary
type GolangBinMetadata struct {
BuildSettings map[string]string `json:"goBuildSettings,omitempty" cyclonedx:"goBuildSettings"`
GoCompiledVersion string `json:"goCompiledVersion" cyclonedx:"goCompiledVersion"`
Architecture string `json:"architecture" cyclonedx:"architecture"`
H1Digest string `json:"h1Digest,omitempty" cyclonedx:"h1Digest"`
MainModule string `json:"mainModule,omitempty" cyclonedx:"mainModule"`
}
// GolangModMetadata represents all captured data for a Golang source scan with go.mod/go.sum
type GolangModMetadata struct {
H1Digest string `json:"h1Digest,omitempty" cyclonedx:"h1Digest"`
}
+8 -5
View File
@@ -20,7 +20,8 @@ const (
DotnetDepsMetadataType MetadataType = "DotnetDepsMetadata"
DpkgMetadataType MetadataType = "DpkgMetadata"
GemMetadataType MetadataType = "GemMetadata"
GolangMetadataType MetadataType = "GolangMetadata"
GolangBinMetadataType MetadataType = "GolangBinMetadata"
GolangModMetadataType MetadataType = "GolangModMetadata"
HackageMetadataType MetadataType = "HackageMetadataType"
JavaMetadataType MetadataType = "JavaMetadata"
KbPackageMetadataType MetadataType = "KbPackageMetadata"
@@ -47,7 +48,8 @@ var AllMetadataTypes = []MetadataType{
DotnetDepsMetadataType,
DpkgMetadataType,
GemMetadataType,
GolangMetadataType,
GolangBinMetadataType,
GolangModMetadataType,
HackageMetadataType,
JavaMetadataType,
KbPackageMetadataType,
@@ -74,7 +76,8 @@ var MetadataTypeByName = map[MetadataType]reflect.Type{
DotnetDepsMetadataType: reflect.TypeOf(DotnetDepsMetadata{}),
DpkgMetadataType: reflect.TypeOf(DpkgMetadata{}),
GemMetadataType: reflect.TypeOf(GemMetadata{}),
GolangMetadataType: reflect.TypeOf(GolangMetadata{}),
GolangBinMetadataType: reflect.TypeOf(GolangBinMetadata{}),
GolangModMetadataType: reflect.TypeOf(GolangModMetadata{}),
HackageMetadataType: reflect.TypeOf(HackageMetadata{}),
JavaMetadataType: reflect.TypeOf(JavaMetadata{}),
KbPackageMetadataType: reflect.TypeOf(KbPackageMetadata{}),
@@ -94,8 +97,8 @@ func CleanMetadataType(typ MetadataType) MetadataType {
if typ == "RpmdbMetadata" {
return RpmMetadataType
}
if typ == "GolangBinMetadata" {
return GolangMetadataType
if typ == "GolangMetadata" {
return GolangBinMetadataType
}
return typ
}
+1 -1
View File
@@ -338,7 +338,7 @@ func (r directoryResolver) FilesByPath(userPaths ...string) ([]Location, error)
// we should be resolving symlinks and preserving this information as a VirtualPath to the real file
evaluatedPath, err := filepath.EvalSymlinks(userStrPath)
if err != nil {
log.Debugf("directory resolver unable to evaluate symlink for path=%q : %+v", userPath, err)
log.Tracef("unable to evaluate symlink for path=%q : %+v", userPath, err)
continue
}
+2
View File
@@ -36,6 +36,8 @@ func TestSpdxValidationTooling(t *testing.T) {
fixturesPath := filepath.Join(cwd, "test-fixtures", "image-java-spdx-tools")
buildCmd := exec.Command("make", "build")
buildCmd.Dir = fixturesPath
buildCmd.Stdout = os.Stdout
buildCmd.Stderr = os.Stderr
err = buildCmd.Run()
require.NoError(t, err)
},
+6
View File
@@ -0,0 +1,6 @@
# change these if you want CI to not use previous stored cache
CLI_CACHE_BUSTER := "e5cdfd8"
.PHONY: cache.fingerprint
cache.fingerprint:
find image-* -type f -exec md5sum {} + | awk '{print $1}' | sort | md5sum | tee cache.fingerprint && echo "$(CLI_CACHE_BUSTER)" >> cache.fingerprint
@@ -1,4 +1,4 @@
FROM cgr.dev/chainguard/jdk
FROM openjdk:11
RUN wget https://github.com/spdx/tools-java/releases/download/v1.1.3/tools-java-1.1.3.zip && \
unzip tools-java-1.1.3.zip && \
+6
View File
@@ -0,0 +1,6 @@
# change these if you want CI to not use previous stored cache
INTEGRATION_CACHE_BUSTER := "894d8ca"
.PHONY: cache.fingerprint
cache.fingerprint:
find image-* -type f -exec md5sum {} + | awk '{print $1}' | sort | tee /dev/stderr | md5sum | tee cache.fingerprint && echo "$(INTEGRATION_CACHE_BUSTER)" >> cache.fingerprint
+8 -2
View File
@@ -604,7 +604,10 @@ func AttestationStartedHandler(ctx context.Context, fr *frame.Frame, event party
line, _ := fr.Append()
if l.Len() > 5 {
elem := l.Front()
line := elem.Value.(*frame.Line)
line, ok := elem.Value.(*frame.Line)
if !ok {
continue
}
err = line.Remove()
if err != nil {
return
@@ -627,7 +630,10 @@ func AttestationStartedHandler(ctx context.Context, fr *frame.Frame, event party
if !failed {
// roll up logs into completed status (only if successful)
for e := l.Back(); e != nil; e = e.Prev() {
line := e.Value.(*frame.Line)
line, ok := e.Value.(*frame.Line)
if !ok {
continue
}
err = line.Remove()
if err != nil {
return