Compare commits

...
36 Commits
Author SHA1 Message Date
Alex Goodman 88c81d33ed switch from trigger-release target to release target (#1560)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-09 16:35:11 +00:00
Alex Goodman 988041ba6d Speed up cataloging by replacing globs searching with index lookups (#1510)
* replace raw globs with index equivelent operations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add cataloger test for alpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix import sorting for binary cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting for mock resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* separate portage cataloger parser impl from cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enhance cataloger pkgtest utils to account for resolver responses

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for alpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for apkdb cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dpkg cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for cpp cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dart cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dotnet cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for elixir cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for erlang cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for golang cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for haskell cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for java cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for javascript cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for php cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for portage cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for python cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for rpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for rust cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for sbom cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for swift cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow generic catloger to run all mimetype searches at once

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove stutter from php and javascript cataloger constructors

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add tests for generic.Search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add exceptions for java archive git ignore entries

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enhance basename and extension resolver methods to be variadic

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* dont allow * prefix on extension searches

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for ruby cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove unnecessary string casting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* incorporate surfacing of leaf link resolitions from stereoscope results

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] switch to stereoscope file metadata

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip + failing] revert to old globs but keep new resolvers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* index files, links, and dirs within the directory resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix several resolver bugs and inconsistencies

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move format testutils to internal package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update syft json to account for file type string normalization

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split up directory resolver from indexing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update docs to include details about searching

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] bump stereoscope to development version

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust symlinks fixture to be fixed to digest

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix all-locations resolver tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix test fixture reference

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename file.Type

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix PR comment to exclude extra *

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump to dev version of stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump to final version of stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move observing resolver to pkgtest

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-09 16:19:47 +00:00
anchore-actions-token-generator[bot] 550e2fc7c3 Update syft bootstrap tools to latest versions. (#1549) 2023-02-09 10:10:35 -05:00
witchcraze 43b53ac173 Fix installed versions (#1556)
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-02-09 09:03:40 -05:00
dependabot[bot] 08804842fa chore(deps): bump golang.org/x/net from 0.5.0 to 0.6.0 (#1558)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.6.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-09 09:01:56 -05:00
witchcrazeandAlex Goodman 284814153d feat: add postgresql classifier (#1536)
* add postgresql classifier
Signed-off-by: witchcraze <witchcraze@gmail.com>

* revert package dedup test upgrade

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-08 14:31:17 -05:00
Alex Goodman 8847ba5d0b Add release trigger (#1501)
* add release trigger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* deduplicate version and changelog calls + add gh checks

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add more chronicle verbosity, but not when triggering releases

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump chronicle version to get --version-file feature

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update bootstrap tool workflow to include glow

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add version prefix check on tags in release quality gate

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-08 11:38:27 -05:00
dependabot[bot] 48528efff3 chore(deps): bump golang.org/x/mod from 0.7.0 to 0.8.0 (#1552)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.7.0 to 0.8.0.
- [Release notes](https://github.com/golang/mod/releases)
- [Commits](https://github.com/golang/mod/compare/v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-08 10:07:37 -05:00
dependabot[bot] 8d856a7c7b chore(deps): bump golang.org/x/term from 0.4.0 to 0.5.0 (#1551)
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/golang/term/releases)
- [Commits](https://github.com/golang/term/compare/v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: golang.org/x/term
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-08 09:23:31 -05:00
Avi Deitcher 38a090c218 fix: add support for licenses not found on list (#1540)
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2023-02-07 11:47:04 -05:00
deb7052f41 Update syft bootstrap tools to latest versions. (#1541)
* Update syft bootstrap tools to latest versions.
Signed-off-by: GitHub <noreply@github.com>

* fix: remove ineff assign
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

---------

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-02-07 15:41:14 +00:00
Keith Zantow 9650473298 feat: Allow specific versions of formats to be specified (#1543) 2023-02-07 10:40:43 -05:00
anchore-actions-token-generator[bot]andkzantow 95201840d2 Update Stereoscope to c49244e4d66f1ee789027ea23acc746968799c3b (#1539)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: kzantow <kzantow@users.noreply.github.com>
2023-02-07 10:05:18 -05:00
Justin Chadwell 6ba595344a source: when base is set, responsePath should be absolute (#1542)
When base is set, it should appear identically to when we scan the root
filesystem - and as a result, the path should begin with the path
separator.

E.g. when scanning the root `./target/` with the same base,
`target/bin/busybox` should appear in the output as `/bin/busybox`, not
as previously as `bin/busybox`.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-06 12:06:04 -05:00
Christopher Angelo Phillips 9995950c70 fix: update config struct to not decode password/key (#1538)
* fix: update config struct to not decode password/key
* test: update tests to confirm no secrets in output

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-02-03 18:06:14 +00:00
anchore-actions-token-generator[bot]andspiffcs b6a496f18c Update syft bootstrap tools to latest versions. (#1537)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2023-02-03 12:35:33 -05:00
witchcraze 0853a50f4d feat: add traefik classifier (#1504)
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-02-02 13:59:26 -05:00
Nils Hanke a1b82c9664 fix: don't hardcode Cosign attest type (#1533)
Signed-off-by: Nils Hanke <nils.hanke@outlook.de>
2023-02-02 12:47:12 -05:00
dependabot[bot] ad8604c223 chore(deps): bump github.com/docker/docker (#1531)
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 20.10.23+incompatible to 23.0.0+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v20.10.23...v23.0.0)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-02 10:53:22 -05:00
anchore-actions-token-generator[bot] d80ee966dc Update syft bootstrap tools to latest versions. (#1530) 2023-02-02 08:47:29 -05:00
Keith Zantow 1530ef354f chore: update spdx/tools-golang to v0.5.0-rc1 (#1503) 2023-01-31 11:53:16 -05:00
Bradley Jones cdac2245b5 feat: update golang to 1.19 (#1526)
* feat: update golang to 1.19

Signed-off-by: Bradley Jones <bradley.jones@anchore.com>

* chore: break out json schema drift check into separate script

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* chore: update git index refresh

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

---------

Signed-off-by: Bradley Jones <bradley.jones@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-01-31 11:39:57 -05:00
anchore-actions-token-generator[bot] 8dba4c33dd Update syft bootstrap tools to latest versions. (#1525) 2023-01-31 09:27:38 -05:00
Justin Chadwell b81c9805dc Allow scanning unpacked container filesystems (#1485)
* source: avoid second-step of symlink resolution in directory resolver

We can use the already existing file tree to peform symlink resolution
for FilesByPath, instead of traversing the symlinks again.

This moves all of the symlink logic into the indexing code, and then we
can rely on syft's resolution algorithm over the index in this part of
the codebase.

Signed-off-by: Justin Chadwell <me@jedevc.com>

* source: add base parameter to directory resolver

The new base parameter is an optional parameter for the directory
resolver that resolves all symlinks relative to this root. There are two
intended use cases:

- base = "/". The previous behavior, symlinks are resolved relative to
the root filesystem.
- base = path. Symlinks are resolved relative to the target filesystem,
allowing correct behavior when scanning unpacked container filesystems
on disk.

Signed-off-by: Justin Chadwell <me@jedevc.com>

* source: add tests for new base parameter

Signed-off-by: Justin Chadwell <me@jedevc.com>

---------

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-30 13:47:24 -05:00
Keith Zantow ba55963104 fix: allow template for syft convert (#1521) 2023-01-26 12:07:36 -05:00
Christopher Angelo Phillips 8c91605541 1465 attestation with private key (#1502)
1465 attestation with private key (#1502)

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-01-26 11:19:13 -05:00
Weston Steimel 4c0aef09b8 fix: add relevant CPEs to python and busybox classifiers (#1517)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-01-25 17:18:24 +00:00
anchore-actions-token-generator[bot] 02fb757c21 Update syft bootstrap tools to latest versions. (#1515) 2023-01-25 10:31:53 -05:00
Keith Zantow 674a54512c chore: correct bootstrap tool script (#1514) 2023-01-25 10:22:28 -05:00
dependabot[bot] 21ba5d0806 chore(deps): bump github.com/google/go-containerregistry (#1513)
Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.12.1 to 0.13.0.
- [Release notes](https://github.com/google/go-containerregistry/releases)
- [Changelog](https://github.com/google/go-containerregistry/blob/main/.goreleaser.yml)
- [Commits](https://github.com/google/go-containerregistry/compare/v0.12.1...v0.13.0)

---
updated-dependencies:
- dependency-name: github.com/google/go-containerregistry
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-25 13:41:43 +00:00
Alex GoodmanandKeith Zantow 0ba57a5936 Fix AssertEncoderAgainstGoldenSnapshot calls to conditionally update (#1511)
* fix AssertEncoderAgainstGoldenSnapshot calls to conditionally update

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* chore: redact schema versions from syftjson comparison

Signed-off-by: Keith Zantow <kzantow@gmail.com>

* chore: does not need a multiline expression

Signed-off-by: Keith Zantow <kzantow@gmail.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
2023-01-24 21:41:57 +00:00
dependabot[bot] 3269bc98d4 chore(deps): bump golang.org/x/mod from 0.6.0 to 0.7.0 (#1505)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/mod/releases)
- [Commits](https://github.com/golang/mod/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-23 15:01:25 -05:00
dependabot[bot] 7f3382f7eb chore(deps): bump github.com/docker/docker (#1506)
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 20.10.20+incompatible to 20.10.23+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v20.10.20...v20.10.23)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-23 14:58:39 -05:00
dependabot[bot] 65e5ff63f0 chore(deps): bump github.com/Masterminds/sprig/v3 from 3.2.2 to 3.2.3 (#1507)
Bumps [github.com/Masterminds/sprig/v3](https://github.com/Masterminds/sprig) from 3.2.2 to 3.2.3.
- [Release notes](https://github.com/Masterminds/sprig/releases)
- [Changelog](https://github.com/Masterminds/sprig/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Masterminds/sprig/compare/v3.2.2...v3.2.3)

---
updated-dependencies:
- dependency-name: github.com/Masterminds/sprig/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-23 14:48:22 -05:00
dependabot[bot] d287c22b69 chore(deps): bump github.com/dustin/go-humanize from 1.0.0 to 1.0.1 (#1508)
Bumps [github.com/dustin/go-humanize](https://github.com/dustin/go-humanize) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/dustin/go-humanize/releases)
- [Commits](https://github.com/dustin/go-humanize/compare/v1.0.0...v1.0.1)

---
updated-dependencies:
- dependency-name: github.com/dustin/go-humanize
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-23 14:48:00 -05:00
Luca Comellini e8be93a8eb Bump github.com/spdx/tools-golang to v0.4.0 (#1450)
Signed-off-by: Luca Comellini <luca.com@gmail.com>
2023-01-20 17:00:21 -05:00
277 changed files with 5344 additions and 2675 deletions
+1 -1
View File
@@ -4,7 +4,7 @@ inputs:
go-version:
description: "Go version to install"
required: true
default: "1.18.x"
default: "1.19.x"
use-go-cache:
description: "Restore go cache"
required: true
+1 -1
View File
@@ -6,6 +6,6 @@ normal=$(tput sgr0)
# assert we are running in CI (or die!)
if [[ -z "$CI" ]]; then
echo "${bold}${red}This script should ONLY be run in CI. Exiting...${normal}"
echo "${bold}${red}This step should ONLY be run in CI. Exiting...${normal}"
exit 1
fi
+1 -1
View File
@@ -4,7 +4,7 @@ set -eu
ORIGINAL_STATE_DIR=$(mktemp -d "TEMP-original-state-XXXXXXXXX")
TIDY_STATE_DIR=$(mktemp -d "TEMP-tidy-state-XXXXXXXXX")
trap "cp ${ORIGINAL_STATE_DIR}/* ./ && rm -fR ${ORIGINAL_STATE_DIR} ${TIDY_STATE_DIR}" EXIT
trap "cp -p ${ORIGINAL_STATE_DIR}/* ./ && git update-index -q --refresh && rm -fR ${ORIGINAL_STATE_DIR} ${TIDY_STATE_DIR}" EXIT
# capturing original state of files...
cp go.mod go.sum "${ORIGINAL_STATE_DIR}"
+27
View File
@@ -0,0 +1,27 @@
#!/usr/bin/env bash
set -u
if ! git diff-index --quiet HEAD --; then
git diff-index HEAD --
git --no-pager diff
echo "there are uncommitted changes, please commit them before running this check"
exit 1
fi
success=true
if ! make generate-json-schema; then
echo "Generating json schema failed"
success=false
fi
if ! git diff-index --quiet HEAD --; then
git diff-index HEAD --
git --no-pager diff
echo "JSON schema drift detected!"
success=false
fi
if ! $success; then
exit 1
fi
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
set -eu
bold=$(tput bold)
normal=$(tput sgr0)
if ! [ -x "$(command -v gh)" ]; then
echo "The GitHub CLI could not be found. To continue follow the instructions at https://github.com/cli/cli#installation"
exit 1
fi
gh auth status
# we need all of the git state to determine the next version. Since tagging is done by
# the release pipeline it is possible to not have all of the tags from previous releases.
git fetch --tags
# populates the CHANGELOG.md and VERSION files
echo "${bold}Generating changelog...${normal}"
make changelog 2> /dev/null
NEXT_VERSION=$(cat VERSION)
if [[ "$NEXT_VERSION" == "" || "${NEXT_VERSION}" == "(Unreleased)" ]]; then
echo "Could not determine the next version to release. Exiting..."
exit 1
fi
while true; do
read -p "${bold}Do you want to trigger a release for version '${NEXT_VERSION}'?${normal} [y/n] " yn
case $yn in
[Yy]* ) echo; break;;
[Nn]* ) echo; echo "Cancelling release..."; exit;;
* ) echo "Please answer yes or no.";;
esac
done
echo "${bold}Kicking off release for ${NEXT_VERSION}${normal}..."
echo
gh workflow run release.yaml -f version=${NEXT_VERSION}
echo
echo "${bold}Waiting for release to start...${normal}"
sleep 10
set +e
echo "${bold}Head to the release workflow to monitor the release:${normal} $(gh run list --workflow=release.yaml --limit=1 --json url --jq '.[].url')"
id=$(gh run list --workflow=release.yaml --limit=1 --json databaseId --jq '.[].databaseId')
gh run watch $id --exit-status || (echo ; echo "${bold}Logs of failed step:${normal}" && GH_PAGER="" gh run view $id --log-failed)
+18 -14
View File
@@ -1,15 +1,13 @@
name: "Release"
on:
push:
# take no actions on push to any branch...
branches-ignore:
- "**"
# ... only act on release tags
tags:
- "v*"
workflow_dispatch:
inputs:
version:
description: tag the latest commit on main with the given version (prefixed with v)
required: true
env:
GO_VERSION: "1.18.x"
GO_VERSION: "1.19.x"
jobs:
quality-gate:
@@ -18,12 +16,11 @@ jobs:
steps:
- uses: actions/checkout@v3
# we don't want to release commits that have been pushed and tagged, but not necessarily merged onto main
- name: Ensure tagged commit is on main
- name: Check if tag already exists
# note: this will fail if the tag already exists
run: |
echo "Tag: ${GITHUB_REF##*/}"
git fetch origin main
git merge-base --is-ancestor ${GITHUB_REF##*/} origin/main && echo "${GITHUB_REF##*/} is a commit on main!"
[[ "${{ github.event.inputs.version }}" == v* ]] || (echo "version '${{ github.event.inputs.version }}' does not have a 'v' prefix" && exit 1)
git tag ${{ github.event.inputs.version }}
- name: Check static analysis results
uses: fountainhead/action-wait-for-check@v1.1.0
@@ -120,8 +117,15 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Tag release
run: |
git tag ${{ github.event.inputs.version }}
git push origin --tags
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Build & publish release artifacts
run: make release
run: make ci-release
env:
# for mac signing and notarization...
QUILL_SIGN_P12: ${{ secrets.ANCHORE_APPLE_DEVELOPER_ID_CERT_CHAIN }}
+19 -15
View File
@@ -6,7 +6,7 @@ on:
workflow_dispatch:
env:
GO_VERSION: "1.18.x"
GO_VERSION: "1.19.x"
GO_STABLE_VERSION: true
jobs:
@@ -29,24 +29,27 @@ jobs:
GOSIMPORTS_LATEST_VERSION=$(go list -m -json github.com/rinchsan/gosimports@latest 2>/dev/null | jq -r '.Version')
YAJSV_LATEST_VERSION=$(go list -m -json github.com/neilpa/yajsv@latest 2>/dev/null | jq -r '.Version')
COSIGN_LATEST_VERSION=$(go list -m -json github.com/sigstore/cosign@latest 2>/dev/null | jq -r '.Version')
GLOW_LATEST_VERSION=$(go list -m -json github.com/charmbracelet/glow@latest 2>/dev/null | jq -r '.Version')
# update version variables in the Makefile
sed -r -i -e 's/^(GOLANGCILINT_VERSION = ).*/\1'${GOLANGCILINT_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(BOUNCER_VERSION = ).*/\1'${BOUNCER_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(CHRONICLE_VERSION = ).*/\1'${CHRONICLE_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(GORELEASER_VERSION = ).*/\1'${GORELEASER_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(GOSIMPORTS_VERSION = ).*/\1'${GOSIMPORTS_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(YAJSV_VERSION = ).*/\1'${YAJSV_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(COSIGN_VERSION = ).*/\1'${COSIGN_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(GOLANGCILINT_VERSION := ).*/\1'${GOLANGCILINT_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(BOUNCER_VERSION := ).*/\1'${BOUNCER_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(CHRONICLE_VERSION := ).*/\1'${CHRONICLE_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(GORELEASER_VERSION := ).*/\1'${GORELEASER_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(GOSIMPORTS_VERSION := ).*/\1'${GOSIMPORTS_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(YAJSV_VERSION := ).*/\1'${YAJSV_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(COSIGN_VERSION := ).*/\1'${COSIGN_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(GLOW_VERSION := ).*/\1'${GLOW_LATEST_VERSION}'/' Makefile
# export the versions for use with create-pull-request
echo "::set-output name=GOLANGCILINT::$GOLANGCILINT_LATEST_VERSION"
echo "::set-output name=BOUNCER::$BOUNCER_LATEST_VERSION"
echo "::set-output name=CHRONICLE::$CHRONICLE_LATEST_VERSION"
echo "::set-output name=GORELEASER::$GORELEASER_LATEST_VERSION"
echo "::set-output name=GOSIMPORTS::$GOSIMPORTS_LATEST_VERSION"
echo "::set-output name=YAJSV::$YAJSV_LATEST_VERSION"
echo "::set-output name=COSIGN::$COSIGN_LATEST_VERSION"
echo "GOLANGCILINT=$GOLANGCILINT_LATEST_VERSION" >> $GITHUB_OUTPUT
echo "BOUNCER=$BOUNCER_LATEST_VERSION" >> $GITHUB_OUTPUT
echo "CHRONICLE=$CHRONICLE_LATEST_VERSION" >> $GITHUB_OUTPUT
echo "GORELEASER=$GORELEASER_LATEST_VERSION" >> $GITHUB_OUTPUT
echo "GOSIMPORTS=$GOSIMPORTS_LATEST_VERSION" >> $GITHUB_OUTPUT
echo "YAJSV=$YAJSV_LATEST_VERSION" >> $GITHUB_OUTPUT
echo "COSIGN=$COSIGN_LATEST_VERSION" >> $GITHUB_OUTPUT
echo "GLOW=GLOW_LATEST_VERSION" >> $GITHUB_OUTPUT
id: latest-versions
- uses: tibdex/github-app-token@v1
@@ -71,5 +74,6 @@ jobs:
- [gosimports ${{ steps.latest-versions.outputs.GOSIMPORTS }}](https://github.com/rinchsan/gosimports/releases/tag/${{ steps.latest-versions.outputs.GOSIMPORTS }})
- [yajsv ${{ steps.latest-versions.outputs.YAJSV }}](https://github.com/neilpa/yajsv/releases/tag/${{ steps.latest-versions.outputs.YAJSV }})
- [cosign ${{ steps.latest-versions.outputs.COSIGN }}](https://github.com/sigstore/cosign/releases/tag/${{ steps.latest-versions.outputs.COSIGN }})
- [glow ${{ steps.latest-versions.outputs.GLOW }}](https://github.com/charmbracelet/glow/releases/tag/${{ steps.latest-versions.outputs.GLOW }})
This is an auto-generated pull request to update all of the bootstrap tools to the latest versions.
token: ${{ steps.generate-token.outputs.token }}
@@ -6,7 +6,7 @@ on:
workflow_dispatch:
env:
GO_VERSION: "1.18.x"
GO_VERSION: "1.19.x"
GO_STABLE_VERSION: true
jobs:
+1
View File
@@ -1,4 +1,5 @@
CHANGELOG.md
VERSION
/test/results
/dist
/snapshot
+13 -10
View File
@@ -6,7 +6,6 @@ issues:
# include:
# - EXC0002 # disable excluding of issues about comments from golint
linters:
# inverted configuration with `enable-all` and `disable` is not scalable during updates of golangci-lint
disable-all: true
@@ -14,25 +13,24 @@ linters:
- asciicheck
- bodyclose
- depguard
- dogsled
- dupl
- errcheck
- errorlint
- exportloopref
- forcetypeassert
- funlen
- gocognit
- goconst
- gocritic
- gocyclo
- gofmt
- tparallel
- importas
- goimports
- goprintffuncname
- gosec
- gosimple
- govet
- ineffassign
- misspell
- nolintlint
- nakedret
- revive
- staticcheck
- stylecheck
@@ -41,6 +39,7 @@ linters:
- unparam
- unused
- whitespace
linters-settings:
funlen:
# Checks the number of lines in a function.
@@ -57,7 +56,7 @@ run:
timeout: 10m
# do not enable...
# - dogsled # found to be to niche and ineffective
# - deadcode # The owner seems to have abandoned the linter. Replaced by "unused".
# - goprintffuncname # does not catch all cases and there are exceptions
# - nakedret # does not catch all cases and should not fail a build
# - gochecknoglobals
@@ -73,7 +72,11 @@ run:
# - lll # without a way to specify per-line exception cases, this is not usable
# - maligned # this is an excellent linter, but tricky to optimize and we are not sensitive to memory layout optimizations
# - nestif
# - prealloc # following this rule isn't consistently a good idea, as it sometimes forces unnecessary allocations that result in less idiomatic code
# - scopelint # deprecated
# - nolintlint # as of go1.19 this conflicts with the behavior of gofmt, which is a deal-breaker (lint-fix will still fail when running lint)
# - prealloc # following this rule isn't consistently a good idea, as it sometimes forces unnecessary allocations that result in less idiomatic code
# - rowserrcheck # not in a repo with sql, so this is not useful
# - scopelint # deprecated
# - structcheck # The owner seems to have abandoned the linter. Replaced by "unused".
# - testpackage
# - wsl # this doens't have an auto-fixer yet and is pretty noisy (https://github.com/bombsimon/wsl/issues/90)
# - varcheck # The owner seems to have abandoned the linter. Replaced by "unused".
# - wsl # this doens't have an auto-fixer yet and is pretty noisy (https://github.com/bombsimon/wsl/issues/90)
+2 -2
View File
@@ -1,4 +1,4 @@
# Contributing to Syft
[#](#) Contributing to Syft
If you are looking to contribute to this project and want to open a GitHub pull request ("PR"), there are a few guidelines of what we are looking for in patches. Make sure you go through this document and ensure that your code proposal is aligned.
@@ -10,7 +10,7 @@ Before you can contribute to Syft, you need to configure your development enviro
You will need to install Go. The version on https://go.dev works best, using the system golang doesn't always work the way you might expect.
At the time of writing, Go 1.19 does not work correctly with Syft. Please use Go 1.18 for now.
Refer to the go.mod file in the root of this repo for the recommended version of Go to install.
You will also need Docker. There's no reason the system packages shouldn't work, but we used the official Docker package. You can find instructions for installing Docker in Debian [here](https://docs.docker.com/engine/install/debian/).
+22 -12
View File
@@ -118,45 +118,55 @@ sequenceDiagram
Catalogers are the way in which syft is able to identify and construct packages given some amount of source metadata.
For example, Syft can locate and process `package-lock.json` files when performing filesystem scans.
See: [how to specify file globs](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/javascript/cataloger.go#L16-L21)
and an implementation of the [package-lock.json parser](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/javascript/cataloger.go#L16-L21) fora quick review.
See: [how to specify file globs](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/javascript/cataloger.go#L16-L21)
and an implementation of the [package-lock.json parser](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/javascript/cataloger.go#L16-L21) fora quick review.
#### Building a new Cataloger
Catalogers must fulfill the interface [found here](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger.go).
Catalogers must fulfill the interface [found here](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger.go).
This means that when building a new cataloger, the new struct must implement both method signatures of `Catalog` and `Name`.
A top level view of the functions that construct all the catalogers can be found [here](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/cataloger.go).
A top level view of the functions that construct all the catalogers can be found [here](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/cataloger.go).
When an author has finished writing a new cataloger this is the spot to plug in the new catalog constructor.
For a top level view of how the catalogers are used see [this function](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/catalog.go#L41-L100) as a reference. It ranges over all catalogers passed as an argument and invokes the `Catalog` method:
For a top level view of how the catalogers are used see [this function](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/catalog.go#L41-L100) as a reference. It ranges over all catalogers passed as an argument and invokes the `Catalog` method:
Each cataloger has its own `Catalog` method, but this does not mean that they are all vastly different.
Take a look at the `apkdb` cataloger for alpine to see how it [constructs a generic.NewCataloger](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/apkdb/cataloger.go).
Take a look at the `apkdb` cataloger for alpine to see how it [constructs a generic.NewCataloger](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/apkdb/cataloger.go).
`generic.NewCataloger` is an abstraction syft uses to make writing common components easier. First, it takes the `catalogerName` to identify the cataloger.
On the other side of the call it uses two key pieces which inform the cataloger how to identify and return packages, the `globPatterns` and the `parseFunction`:
- The first piece is a `parseByGlob` matching pattern used to identify the files that contain the package metadata.
See [here for the APK example](https://github.com/anchore/syft/blob/main/syft/pkg/apk_metadata.go#L16-L41).
See [here for the APK example](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/apk_metadata.go#L16-L41).
- The other is a `parseFunction` which informs the cataloger what to do when it has found one of the above matches files.
See this [link for an example](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/apkdb/parse_apk_db.go#L22-L102).
See this [link for an example](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/apkdb/parse_apk_db.go#L22-L102).
If you're unsure about using the `Generic Cataloger` and think the use case being filled requires something more custom
just file an issue or ask in our slack, and we'd be more than happy to help on the design.
Identified packages share a common struct so be sure that when the new cataloger is constructing a new package it is using the [`Package` struct](https://github.com/anchore/syft/blob/main/syft/pkg/package.go#L16-L31).
Identified packages share a common struct so be sure that when the new cataloger is constructing a new package it is using the [`Package` struct](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/package.go#L16-L31).
Metadata Note: Identified packages are also assigned specific metadata that can be unique to their environment.
See [this folder](https://github.com/anchore/syft/tree/main/syft/pkg) for examples of the different metadata types.
See [this folder](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg) for examples of the different metadata types.
These are plugged into the `MetadataType` and `Metadata` fields in the above struct. `MetadataType` informs which type is being used. `Metadata` is an interface converted to that type.
Finally, here is an example of where the package construction is done in the apk cataloger. The first link is where `newPackage` is called in the `parseFunction`. The second link shows the package construction:
- [Call for new package](https://github.com/anchore/syft/blob/6a7d6e6071829c7ce2943266c0e187b27c0b325c/syft/pkg/cataloger/apkdb/parse_apk_db.go#L96-L99)
- [APK Package Constructor](https://github.com/anchore/syft/blob/6a7d6e6071829c7ce2943266c0e187b27c0b325c/syft/pkg/cataloger/apkdb/package.go#L12-L27)
- [Call for new package](https://github.com/anchore/syft/blob/v0.70.0/syft/pkg/cataloger/apkdb/parse_apk_db.go#L106)
- [APK Package Constructor](https://github.com/anchore/syft/tree/v0.70.0/syft/pkg/cataloger/apkdb/package.go#L12-L27)
If you have more questions about implementing a cataloger or questions about one you might be currently working
always feel free to file an issue or reach out to us [on slack](https://anchore.com/slack).
#### Searching for files
All catalogers are provided an instance of the [`source.FileResolver`](https://github.com/anchore/syft/blob/v0.70.0/syft/source/file_resolver.go#L8) to interface with the image and search for files. The implementations for these
abstractions leverage [`stereoscope`](https://github.com/anchore/stereoscope) in order to perform searching. Here is a
rough outline how that works:
1. a stereoscope `file.Index` is searched based on the input given (a path, glob, or MIME type). The index is relatively fast to search, but requires results to be filtered down to the files that exist in the specific layer(s) of interest. This is done automatically by the `filetree.Searcher` abstraction. This abstraction will fallback to searching directly against the raw `filetree.FileTree` if the index does not contain the file(s) of interest. Note: the `filetree.Searcher` is used by the `source.FileResolver` abstraction.
2. Once the set of files are returned from the `filetree.Searcher` the results are filtered down further to return the most unique file results. For example, you may have requested for files by a glob that returns multiple results. These results are filtered down to deduplicate by real files, so if a result contains two references to the same file, say one accessed via symlink and one accessed via the real path, then the real path reference is returned and the symlink reference is filtered out. If both were accessed by symlink then the first (by lexical order) is returned. This is done automatically by the `source.FileResolver` abstraction.
3. By the time results reach the `pkg.Cataloger` you are guaranteed to have a set of unique files that exist in the layer(s) of interest (relative to what the resolver supports).
## Testing
### Levels of testing
+22 -17
View File
@@ -6,16 +6,19 @@ LINT_CMD := $(TEMP_DIR)/golangci-lint run --tests=false
GOIMPORTS_CMD := $(TEMP_DIR)/gosimports -local github.com/anchore
RELEASE_CMD := $(TEMP_DIR)/goreleaser release --rm-dist
SNAPSHOT_CMD := $(RELEASE_CMD) --skip-publish --skip-sign --snapshot
CHRONICLE_CMD = $(TEMP_DIR)/chronicle
GLOW_CMD = $(TEMP_DIR)/glow
# Tool versions #################################
GOLANGCILINT_VERSION := v1.50.1
GOLANGCILINT_VERSION := v1.51.1
GOSIMPORTS_VERSION := v0.3.5
BOUNCER_VERSION := v0.4.0
CHRONICLE_VERSION := v0.4.2
GORELEASER_VERSION := v1.14.1
CHRONICLE_VERSION := v0.6.0
GORELEASER_VERSION := v1.15.2
YAJSV_VERSION := v1.4.1
COSIGN_VERSION := v1.13.1
QUILL_VERSION := v0.2.0
GLOW_VERSION := v1.5.0
# Formatting variables #################################
BOLD := $(shell tput -T linux bold)
@@ -88,6 +91,7 @@ bootstrap-tools: $(TEMP_DIR)
GOBIN="$(realpath $(TEMP_DIR))" go install github.com/rinchsan/gosimports/cmd/gosimports@$(GOSIMPORTS_VERSION)
GOBIN="$(realpath $(TEMP_DIR))" go install github.com/neilpa/yajsv@$(YAJSV_VERSION)
GOBIN="$(realpath $(TEMP_DIR))" go install github.com/sigstore/cosign/cmd/cosign@$(COSIGN_VERSION)
GOBIN="$(realpath $(TEMP_DIR))" go install github.com/charmbracelet/glow@$(GLOW_VERSION)
.PHONY: bootstrap-go
bootstrap-go:
@@ -132,10 +136,7 @@ check-go-mod-tidy:
check-json-schema-drift:
$(call title,Ensure there is no drift between the JSON schema and the code)
@git diff-index --quiet HEAD -- || (echo "there are uncommitted changes, please commit them before running this check" && false)
@make generate-json-schema || (echo "$(RED)$(BOLD)JSON schema drift detected!$(RESET)" && false)
@git diff-index --quiet HEAD -- || (echo "$(RED)$(BOLD)JSON schema drift detected!$(RESET)" && false)
@.github/scripts/json-schema-drift-check.sh
## Testing targets #################################
@@ -307,20 +308,20 @@ $(SNAPSHOT_DIR): ## Build snapshot release binaries and packages
$(SNAPSHOT_CMD) --config $(TEMP_DIR)/goreleaser.yaml
.PHONY: changelog
changelog: clean-changelog $(CHANGELOG) ## Generate and show the changelog for the current unreleased version
@docker run -it --rm \
-v $(shell pwd)/$(CHANGELOG):/$(CHANGELOG) \
rawkode/mdv \
-t 748.5989 \
/$(CHANGELOG)
changelog: clean-changelog ## Generate and show the changelog for the current unreleased version
$(CHRONICLE_CMD) -vvv -n --version-file VERSION > $(CHANGELOG)
@$(GLOW_CMD) $(CHANGELOG)
$(CHANGELOG):
$(TEMP_DIR)/chronicle -vv > $(CHANGELOG)
$(CHRONICLE_CMD) -vvv > $(CHANGELOG)
.PHONY: release
release: clean-dist $(CHANGELOG)
release:
@.github/scripts/trigger-release.sh
.PHONY: ci-release
ci-release: ci-check clean-dist $(CHANGELOG)
$(call title,Publishing release artifacts)
@.github/scripts/ci-check.sh
# create a config with the dist dir overridden
echo "dist: $(DIST_DIR)" > $(TEMP_DIR)/goreleaser.yaml
@@ -335,6 +336,10 @@ release: clean-dist $(CHANGELOG)
# upload the version file that supports the application version update check (excluding pre-releases)
.github/scripts/update-version-file.sh "$(DIST_DIR)" "$(VERSION)"
.PHONY: ci-check
ci-check:
@.github/scripts/ci-check.sh
## Cleanup targets #################################
.PHONY: clean
@@ -353,7 +358,7 @@ clean-dist: clean-changelog
.PHONY: clean-changelog
clean-changelog:
rm -f $(CHANGELOG)
rm -f $(CHANGELOG) VERSION
clean-test-image-cache: clean-test-image-tar-cache clean-test-image-docker-cache ## Clean test image cache
+51 -43
View File
@@ -1,14 +1,49 @@
# Release Process
# Release
## Creating a release
This release process itself should be as automated as possible, and has only a few steps:
1. **Trigger a new release with `make release`**. At this point you'll see a preview
changelog in the terminal. If you're happy with the changelog, press `y` to continue, otherwise
you can abort and adjust the labels on the PRs and issues to be included in the release and
re-run the release trigger command.
1. A release admin must approve the release on the GitHub Actions release pipeline run page.
Once approved, the release pipeline will generate all assets and publish a GitHub Release.
1. If there is a release Milestone, close it.
Ideally releasing should be done often with small increments when possible. Unless a
breaking change is blocking the release, or no fixes/features have been merged, a good
target release cadence is between every 1 or 2 weeks.
## Retracting a release
If a release is found to be problematic, it can be retracted with the following steps:
- Deleting the GitHub Release
- Untag the docker images in the `ghcr.io` and `docker.io` registries
- Revert the brew formula in [`anchore/homebrew-syft`](https://github.com/anchore/homebrew-syft) to point to the previous release
- Add a new `retract` entry in the go.mod for the versioned release
**Note**: do not delete release tags from the git repository since there may already be references to the release
in the go proxy, which will cause confusion when trying to reuse the tag later (the H1 hash will not match and there
will be a warning when users try to pull the new release).
## Background
A good release process has the following qualities:
1. There is a way to plan what should be in a release
1. There is a way to see what is actually in a release
1. Allow for different kinds of releases (major breaking vs backwards compatible
enhancements vs patch updates)
1. Allow for different kinds of releases (major breaking vs backwards compatible enhancements vs patch updates)
1. Specify a repeatable way to build and publish software artifacts
## Planning a release
### Planning a release
To indicate a set of features to be released together add each issue to an in-repository
Milestone named with major-minor version to be released (e.g. `v0.1`). It is OK for other
@@ -21,15 +56,13 @@ completed, would allow the release to be considered complete. A Milestone is onl
Not all releases need to be planned. For instance, patch releases for fixes should be
released when they are ready and when releasing would not interfere with another current
release (where some partial or breaking features have already been merged). Beta releases
and release candidates should not be independently planned from the non-beta release. That
is, the features for a `v0.1-beta.1` release should be planned under the `v0.1` Milestone,
not under a separate `v0.1-beta.1` Milestone.
release (where some partial or breaking features have already been merged).
Unless necessary, feature releases should be small and frequent, which may obviate the
need for regular release planning under a Milestone.
## What is in a release
### What is in a release
Milestones are specifically for planning a release, not necessarily tracking all changes
that a release may bring (and more importantly, not all releases are necessarily planned
@@ -42,9 +75,9 @@ Changelog line. Furthermore, there should be a place to see all released version
release date for each release, the semantic version of the release, and the set of changes
for each release.
This project auto-generates the Changelog contents for each current release and posts the
generated contents to the GitHub Release page. Leveraging the GitHub Releases feature
allows GitHub to manage the Changelog on each release outside of the git repository while
**This project auto-generates the Changelog contents for each current release and posts the
generated contents to the GitHub Release page**. Leveraging the GitHub Releases feature
allows GitHub to manage the Changelog on each release outside of the git source tree while
still being hosted with the released assets.
The Changelog is generated from the metadata from in-repository issues and PRs, using
@@ -60,45 +93,20 @@ The above suggestions imply that we should:
- The appropriate label is applied to PRs and/or issues to drive specific change type
sections (deprecated, breaking, security, bug, etc)
With this approach as we cultivate good organization of PRs and issues we automatically
get an equally good Changelog.
**With this approach as we cultivate good organization of PRs and issues we automatically
get an equally good Changelog.**
## Major, minor, and patch releases
### Major, minor, and patch releases
The latest version of the tool is the only supported version, which implies that multiple
parallel release branches will not be a regular process (if ever). Multiple releases can
be planned in parallel, however, only one can be actively developed at a time. That is, if
PRs attached to a release Milestone have been merged into the main branch, that release is
now the "next" release. This implies that the source of truth for release lies with the
git log and Changelog, not with the release Milestones (which are purely for planning and
now the "next" release. **This implies that the source of truth for release lies with the
git log and Changelog, not with the release Milestones** (which are purely for planning and
tracking).
Semantic versioning should be used to indicate breaking changes, new features, and fixes.
The exception to this is `< 1.0`, where the major version is not bumped for breaking changes,
instead the minor version indicates both new features and breaking changes.
## Cutting a release
Ideally releasing should be done often with small increments when possible. Unless a
breaking change is blocking the release, or no fixes/features have been merged, a good
target release cadence is between every 2 or 4 weeks.
This release process itself should be as automated as possible, and have only a few steps:
1. Tag the main branch with a full semantic-version, prefixed with a `v`. If there is a
milestone with a partial version, the full version should be used for the git tag (e.g.
with a Milestone of `v0.1` the tag should be `v0.1.0`). You can determine the changes going
into a release by running `make changelog-unreleased`. Use this change list to determine the
release increment. After determining the release increment (major, minor, patch), create the tag.
Given the above example the command to create the tag would be `git tag v0.1.0`.
1. Push the tag. Given the above example the command to push the tag would be `git push origin v0.1.0`.
1. A release admin must approve the release on the GitHub Actions release pipeline run page.
Once approved, the release pipeline will generate all assets and draft a GitHub Release.
1. Navigate to the GitHub Release draft page to review the final changelog and publish the
release. Once published, a release-follow-up pipeline will publish derivative artifacts
(docker image to DockerHub, brew formula to the external homebrew git repo, etc).
1. If there is a release Milestone, close it.
+8 -3
View File
@@ -20,8 +20,7 @@ const (
attestHelp = attestExample + attestSchemeHelp
)
//nolint:dupl
func Attest(v *viper.Viper, app *config.Application, ro *options.RootOptions, po *options.PackagesOptions) *cobra.Command {
func Attest(v *viper.Viper, app *config.Application, ro *options.RootOptions, po *options.PackagesOptions, ao *options.AttestOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "attest --output [FORMAT] <IMAGE>",
Short: "Generate an SBOM as an attestation for the given [SOURCE] container image",
@@ -50,11 +49,17 @@ func Attest(v *viper.Viper, app *config.Application, ro *options.RootOptions, po
},
}
// syft attest is an enhancment of the packages command, so it should have the same flags
// syft attest is an enhancement of the packages command, so it should have the same flags
err := po.AddFlags(cmd, v)
if err != nil {
log.Fatal(err)
}
// syft attest has its own options not included as part of the packages command
err = ao.AddFlags(cmd, v)
if err != nil {
log.Fatal(err)
}
return cmd
}
+15 -13
View File
@@ -22,7 +22,6 @@ import (
"github.com/anchore/syft/syft/event/monitor"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/formats/template"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
@@ -97,6 +96,7 @@ func buildSBOM(app *config.Application, si source.Input, writer sbom.Writer, err
return sBytes, nil
}
//nolint:funlen
func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
@@ -130,10 +130,19 @@ func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-
return
}
args := []string{"attest", si.UserInput, "--type", "custom", "--predicate", f.Name()}
args := []string{"attest", si.UserInput, "--predicate", f.Name()}
if app.Attest.Key != "" {
args = append(args, "--key", app.Attest.Key)
}
execCmd := exec.Command(cmd, args...)
execCmd.Env = os.Environ()
execCmd.Env = append(execCmd.Env, "COSIGN_EXPERIMENTAL=1")
if app.Attest.Key != "" {
execCmd.Env = append(execCmd.Env, fmt.Sprintf("COSIGN_PASSWORD=%s", app.Attest.Password))
} else {
// no key provided, use cosign's keyless mode
execCmd.Env = append(execCmd.Env, "COSIGN_EXPERIMENTAL=1")
}
// bus adapter for ui to hook into stdout via an os pipe
r, w, err := os.Pipe()
@@ -166,16 +175,9 @@ func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-
}
func ValidateOutputOptions(app *config.Application) error {
var usesTemplateOutput bool
for _, o := range app.Outputs {
if o == template.ID.String() {
usesTemplateOutput = true
break
}
}
if usesTemplateOutput && app.OutputTemplatePath == "" {
return fmt.Errorf(`must specify path to template file when using "template" output format`)
err := packages.ValidateOutputOptions(app)
if err != nil {
return err
}
if len(app.Outputs) > 1 {
+2 -1
View File
@@ -45,12 +45,13 @@ func New() (*cobra.Command, error) {
// we also need the command to have information about the `root` options because of this alias
ro := &options.RootOptions{}
po := &options.PackagesOptions{}
ao := &options.AttestOptions{}
packagesCmd := Packages(v, app, ro, po)
// root options are also passed to the attestCmd so that a user provided config location can be discovered
poweruserCmd := PowerUser(v, app, ro)
convertCmd := Convert(v, app, ro, po)
attestCmd := Attest(v, app, ro, po)
attestCmd := Attest(v, app, ro, po, ao)
// rootCmd is currently an alias for the packages command
rootCmd := &cobra.Command{
+7 -5
View File
@@ -8,12 +8,12 @@ import (
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/formats"
)
func Run(ctx context.Context, app *config.Application, args []string) error {
func Run(_ context.Context, app *config.Application, args []string) error {
log.Warn("convert is an experimental feature, run `syft convert -h` for help")
writer, err := options.MakeWriter(app.Outputs, app.File, "")
writer, err := options.MakeWriter(app.Outputs, app.File, app.OutputTemplatePath)
if err != nil {
return err
}
@@ -30,9 +30,11 @@ func Run(ctx context.Context, app *config.Application, args []string) error {
if err != nil {
return fmt.Errorf("failed to open SBOM file: %w", err)
}
defer f.Close()
defer func() {
_ = f.Close()
}()
sbom, _, err := syft.Decode(f)
sbom, _, err := formats.Decode(f)
if err != nil {
return fmt.Errorf("failed to decode SBOM: %w", err)
}
+25
View File
@@ -0,0 +1,25 @@
package options
import (
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
)
type AttestOptions struct {
Key string
}
var _ Interface = (*AttestOptions)(nil)
func (o AttestOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.Flags().StringVarP(&o.Key, "key", "k", "", "the key to use for the attestation")
return bindAttestConfigOptions(cmd.Flags(), v)
}
func bindAttestConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
if err := v.BindPFlag("attest.key", flags.Lookup("key")); err != nil {
return err
}
return nil
}
-39
View File
@@ -1,39 +0,0 @@
package options
import (
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/cyclonedxxml"
"github.com/anchore/syft/syft/formats/github"
"github.com/anchore/syft/syft/formats/spdxjson"
"github.com/anchore/syft/syft/formats/spdxtagvalue"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/formats/text"
"github.com/anchore/syft/syft/sbom"
)
func FormatAliases(ids ...sbom.FormatID) (aliases []string) {
for _, id := range ids {
switch id {
case syftjson.ID:
aliases = append(aliases, "syft-json")
case text.ID:
aliases = append(aliases, "text")
case table.ID:
aliases = append(aliases, "table")
case spdxjson.ID:
aliases = append(aliases, "spdx-json")
case spdxtagvalue.ID:
aliases = append(aliases, "spdx-tag-value")
case cyclonedxxml.ID:
aliases = append(aliases, "cyclonedx-xml")
case cyclonedxjson.ID:
aliases = append(aliases, "cyclonedx-json")
case github.ID:
aliases = append(aliases, "github", "github-json")
default:
aliases = append(aliases, string(id))
}
}
return aliases
}
+3 -3
View File
@@ -7,7 +7,7 @@ import (
"github.com/spf13/pflag"
"github.com/spf13/viper"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/formats"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/pkg/cataloger"
"github.com/anchore/syft/syft/source"
@@ -30,8 +30,8 @@ func (o *PackagesOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.Flags().StringVarP(&o.Scope, "scope", "s", cataloger.DefaultSearchConfig().Scope.String(),
fmt.Sprintf("selection of layers to catalog, options=%v", source.AllScopes))
cmd.Flags().StringArrayVarP(&o.Output, "output", "o", FormatAliases(table.ID),
fmt.Sprintf("report output format, options=%v", FormatAliases(syft.FormatIDs()...)))
cmd.Flags().StringArrayVarP(&o.Output, "output", "o", []string{string(table.ID)},
fmt.Sprintf("report output format, options=%v", formats.AllIDs()))
cmd.Flags().StringVarP(&o.File, "file", "", "",
"file to write the default report output to (default is STDOUT)")
+4 -4
View File
@@ -6,7 +6,7 @@ import (
"github.com/hashicorp/go-multierror"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/formats"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/formats/template"
"github.com/anchore/syft/syft/sbom"
@@ -32,7 +32,7 @@ func MakeWriter(outputs []string, defaultFile, templateFilePath string) (sbom.Wr
func parseOutputs(outputs []string, defaultFile, templateFilePath string) (out []sbom.WriterOption, errs error) {
// always should have one option -- we generally get the default of "table", but just make sure
if len(outputs) == 0 {
outputs = append(outputs, string(table.ID))
outputs = append(outputs, table.ID.String())
}
for _, name := range outputs {
@@ -52,9 +52,9 @@ func parseOutputs(outputs []string, defaultFile, templateFilePath string) (out [
file = parts[1]
}
format := syft.FormatByName(name)
format := formats.ByName(name)
if format == nil {
errs = multierror.Append(errs, fmt.Errorf(`unsupported output format "%s", supported formats are: %+v`, name, FormatAliases(syft.FormatIDs()...)))
errs = multierror.Append(errs, fmt.Errorf(`unsupported output format "%s", supported formats are: %+v`, name, formats.AllIDs()))
continue
}
+3 -2
View File
@@ -26,9 +26,10 @@ import (
"github.com/anchore/syft/syft/source"
)
func Run(ctx context.Context, app *config.Application, args []string) error {
func Run(_ context.Context, app *config.Application, args []string) error {
f := syftjson.Format()
writer, err := sbom.NewWriter(sbom.WriterOption{
Format: syftjson.Format(),
Format: f,
Path: app.File,
})
if err != nil {
+19 -17
View File
@@ -1,6 +1,6 @@
module github.com/anchore/syft
go 1.18
go 1.19
require (
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
@@ -11,7 +11,7 @@ require (
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501
github.com/bmatcuk/doublestar/v4 v4.6.0
github.com/dustin/go-humanize v1.0.0
github.com/dustin/go-humanize v1.0.1
github.com/facebookincubator/nvdtools v0.1.5
github.com/go-test/deep v1.1.0
github.com/google/go-cmp v0.5.9
@@ -31,7 +31,7 @@ require (
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e
github.com/sergi/go-diff v1.3.1
github.com/sirupsen/logrus v1.9.0
github.com/spdx/tools-golang v0.3.1-0.20221108182156-8a01147e6342
github.com/spdx/tools-golang v0.5.0-rc1
github.com/spf13/afero v1.9.3
github.com/spf13/cobra v1.6.1
github.com/spf13/pflag v1.0.5
@@ -42,33 +42,35 @@ require (
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/mod v0.6.0
golang.org/x/net v0.5.0
golang.org/x/term v0.4.0
golang.org/x/mod v0.8.0
golang.org/x/net v0.6.0
golang.org/x/term v0.5.0
gopkg.in/yaml.v2 v2.4.0
)
require (
github.com/CycloneDX/cyclonedx-go v0.7.1-0.20221222100750-41a1ac565cce
github.com/Masterminds/sprig/v3 v3.2.2
github.com/Masterminds/sprig/v3 v3.2.3
github.com/anchore/go-logger v0.0.0-20220728155337-03b66a5207d8
github.com/anchore/stereoscope v0.0.0-20221208011002-c5ff155d72f1
github.com/docker/docker v20.10.20+incompatible
github.com/google/go-containerregistry v0.12.1
github.com/anchore/stereoscope v0.0.0-20230208154630-5a306f07f2e7
github.com/docker/docker v23.0.0+incompatible
github.com/google/go-containerregistry v0.13.0
github.com/invopop/jsonschema v0.7.0
github.com/knqyf263/go-rpmdb v0.0.0-20221030135625-4082a22221ce
github.com/opencontainers/go-digest v1.0.0
github.com/sassoftware/go-rpmutils v0.2.0
github.com/vbatts/go-mtree v0.5.2
golang.org/x/exp v0.0.0-20220823124025-807a23277127
golang.org/x/exp v0.0.0-20230202163644-54bba9f4231b
gopkg.in/yaml.v3 v3.0.1
)
require (
github.com/DataDog/zstd v1.4.5 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.1.1 // indirect
github.com/Masterminds/semver/v3 v3.2.0 // indirect
github.com/Microsoft/go-winio v0.6.0 // indirect
github.com/anchore/go-struct-converter v0.0.0-20221118182256-c68fdcfa2092 // indirect
github.com/becheran/wildmatch-go v1.0.0 // indirect
github.com/containerd/containerd v1.6.12 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.12.1 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
@@ -86,7 +88,7 @@ require (
github.com/golang/snappy v0.0.4 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/huandu/xstrings v1.3.2 // indirect
github.com/huandu/xstrings v1.3.3 // indirect
github.com/iancoleman/orderedmap v0.0.0-20190318233801-ac98e3ecb4b0 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
@@ -124,9 +126,9 @@ require (
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.4.0 // indirect
golang.org/x/text v0.6.0 // indirect
golang.org/x/tools v0.1.12 // indirect
golang.org/x/sys v0.5.0 // indirect
golang.org/x/text v0.7.0 // indirect
golang.org/x/tools v0.2.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/genproto v0.0.0-20221227171554-f9683d7f8bef // indirect
google.golang.org/grpc v1.52.0 // indirect
@@ -149,7 +151,7 @@ require (
// go: warning: github.com/andybalholm/brotli@v1.0.1: retracted by module author: occasional panics and data corruption
github.com/andybalholm/brotli v1.0.4 // indirect
github.com/pkg/errors v0.9.1 // indirect
golang.org/x/crypto v0.1.0 // indirect
golang.org/x/crypto v0.3.0 // indirect
)
retract (
+45 -846
View File
File diff suppressed because it is too large Load Diff
+1
View File
@@ -47,6 +47,7 @@ type Application struct {
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Catalogers []string `yaml:"catalogers" json:"catalogers" mapstructure:"catalogers"`
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
FileMetadata FileMetadata `yaml:"file-metadata" json:"file-metadata" mapstructure:"file-metadata"`
FileClassification fileClassification `yaml:"file-classification" json:"file-classification" mapstructure:"file-classification"`
FileContents fileContents `yaml:"file-contents" json:"file-contents" mapstructure:"file-contents"`
+14
View File
@@ -0,0 +1,14 @@
package config
import "github.com/spf13/viper"
type attest struct {
// IMPORTANT: do not show the attestation key/password in any YAML/JSON output (sensitive information)
Key string `yaml:"-" json:"-" mapstructure:"key"`
Password string `yaml:"-" json:"-" mapstructure:"password"`
}
func (cfg attest) loadDefaultValues(v *viper.Viper) {
v.SetDefault("attest.key", "")
v.SetDefault("attest.password", "")
}
-2
View File
@@ -146,9 +146,7 @@ func findArchiveStartOffset(r io.ReaderAt, size int64) (startOfArchive uint64, e
if err != nil {
return 0, err
}
startOfArchive = 0 // Prefixed data not supported
}
startOfArchive = uint64(directoryEndOffset) - d.directorySize - d.directoryOffset
// Make sure directoryOffset points to somewhere in our file.
+16 -3
View File
@@ -12,9 +12,22 @@ import (
// EX: gpl-2.0.0-only ---> GPL-2.0-only
// See the debian link for more details on the spdx license differences
const (
LicenseRefPrefix = "LicenseRef-" // prefix for non-standard licenses
)
//go:generate go run ./generate
func ID(id string) (string, bool) {
value, exists := licenseIDs[strings.ToLower(id)]
return value, exists
func ID(id string) (value, other string, exists bool) {
id = strings.TrimSpace(id)
// ignore blank strings or the joiner
if id == "" || id == "AND" {
return "", "", false
}
// first look for a canonical license
if value, exists := licenseIDs[strings.ToLower(id)]; exists {
return value, "", exists
}
// we did not find, so treat it as a separate license
return "", id, true
}
+43 -4
View File
@@ -9,52 +9,91 @@ import (
func TestIDParse(t *testing.T) {
var tests = []struct {
shortName string
spdx string
id string
other string
found bool
}{
{
"GPL-1-only",
"GPL-1.0-only",
"",
true,
},
{
"GPL-2",
"GPL-2.0-only",
"",
true,
},
{
"GPL-2+",
"GPL-2.0-or-later",
"",
true,
},
{
"GPL-3.0.0-or-later",
"GPL-3.0-or-later",
"",
true,
},
{
"GPL-3-with-autoconf-exception",
"GPL-3.0-with-autoconf-exception",
"",
true,
},
{
"CC-by-nc-3-de",
"CC-BY-NC-3.0-DE",
"",
true,
},
// the below few cases are NOT expected, however, seem unavoidable given the current approach
{
"w3c-20150513.0.0",
"W3C-20150513",
"",
true,
},
{
"spencer-86.0.0",
"Spencer-86",
"",
true,
},
{
"unicode-dfs-2015.0.0",
"Unicode-DFS-2015",
"",
true,
},
{
"Unknown",
"",
"Unknown",
true,
},
{
" ",
"",
"",
false,
},
{
"AND",
"",
"",
false,
},
}
for _, test := range tests {
t.Run(test.shortName, func(t *testing.T) {
got, exists := ID(test.shortName)
assert.True(t, exists)
assert.Equal(t, test.spdx, got)
value, other, exists := ID(test.shortName)
assert.Equal(t, test.found, exists)
assert.Equal(t, test.id, value)
assert.Equal(t, test.other, other)
})
}
}
+7
View File
@@ -28,3 +28,10 @@ func StringInSlice(a string, list []string) bool {
}
return false
}
func SplitAny(s string, seps string) []string {
splitter := func(r rune) bool {
return strings.ContainsRune(seps, r)
}
return strings.FieldsFunc(s, splitter)
}
+34
View File
@@ -104,3 +104,37 @@ func TestTruncateMiddleEllipsis(t *testing.T) {
})
}
}
func TestSplitAny(t *testing.T) {
tests := []struct {
name string
input string
fields string
want []string
}{
{
name: "simple",
input: "a,b,c",
fields: ",",
want: []string{"a", "b", "c"},
},
{
name: "empty",
input: "",
fields: ",",
want: []string{},
},
{
name: "multiple separators",
input: "a,b\nc:d",
fields: ",:\n",
want: []string{"a", "b", "c", "d"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, SplitAny(tt.input, tt.fields))
})
}
}
+2 -1
View File
@@ -1,6 +1,7 @@
package file
import (
"github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/source"
)
@@ -20,7 +21,7 @@ func allRegularFiles(resolver source.FileResolver) (locations []source.Location)
continue
}
if metadata.Type != source.RegularFile {
if metadata.Type != file.TypeRegular {
continue
}
locations = append(locations, resolvedLocation)
+3 -2
View File
@@ -3,6 +3,7 @@ package file
import (
"testing"
"github.com/google/go-cmp/cmp"
"github.com/scylladb/go-set/strset"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -69,8 +70,8 @@ func Test_allRegularFiles(t *testing.T) {
virtualLocations.Add(l.VirtualPath)
}
}
assert.ElementsMatch(t, tt.wantRealPaths.List(), realLocations.List(), "mismatched real paths")
assert.ElementsMatch(t, tt.wantVirtualPaths.List(), virtualLocations.List(), "mismatched virtual paths")
assert.ElementsMatch(t, tt.wantRealPaths.List(), realLocations.List(), "real paths differ: "+cmp.Diff(tt.wantRealPaths.List(), realLocations.List()))
assert.ElementsMatch(t, tt.wantVirtualPaths.List(), virtualLocations.List(), "virtual paths differ: "+cmp.Diff(tt.wantVirtualPaths.List(), virtualLocations.List()))
})
}
}
+2 -1
View File
@@ -11,6 +11,7 @@ import (
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
"github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/log"
@@ -65,7 +66,7 @@ func (i *DigestsCataloger) catalogLocation(resolver source.FileResolver, locatio
}
// we should only attempt to report digests for files that are regular files (don't attempt to resolve links)
if meta.Type != source.RegularFile {
if meta.Type != file.TypeRegular {
return nil, errUndigestableFile
}
+1 -1
View File
@@ -146,7 +146,7 @@ func TestDigestsCataloger_MixFileTypes(t *testing.T) {
if err != nil {
t.Fatalf("unable to get file=%q : %+v", test.path, err)
}
l := source.NewLocationFromImage(test.path, *ref, img)
l := source.NewLocationFromImage(test.path, *ref.Reference, img)
if len(actual[l.Coordinates]) == 0 {
if test.expected != "" {
+18 -11
View File
@@ -6,6 +6,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/stereoscope/pkg/imagetest"
@@ -50,8 +51,9 @@ func TestFileMetadataCataloger(t *testing.T) {
path: "/file-1.txt",
exists: true,
expected: source.FileMetadata{
Path: "/file-1.txt",
Mode: 0644,
Type: "RegularFile",
Type: file.TypeRegular,
UserID: 1,
GroupID: 2,
Size: 7,
@@ -62,8 +64,9 @@ func TestFileMetadataCataloger(t *testing.T) {
path: "/hardlink-1",
exists: true,
expected: source.FileMetadata{
Path: "/hardlink-1",
Mode: 0644,
Type: "HardLink",
Type: file.TypeHardLink,
LinkDestination: "file-1.txt",
UserID: 1,
GroupID: 2,
@@ -74,8 +77,9 @@ func TestFileMetadataCataloger(t *testing.T) {
path: "/symlink-1",
exists: true,
expected: source.FileMetadata{
Path: "/symlink-1",
Mode: 0777 | os.ModeSymlink,
Type: "SymbolicLink",
Type: file.TypeSymLink,
LinkDestination: "file-1.txt",
UserID: 0,
GroupID: 0,
@@ -86,8 +90,9 @@ func TestFileMetadataCataloger(t *testing.T) {
path: "/char-device-1",
exists: true,
expected: source.FileMetadata{
Path: "/char-device-1",
Mode: 0644 | os.ModeDevice | os.ModeCharDevice,
Type: "CharacterDevice",
Type: file.TypeCharacterDevice,
UserID: 0,
GroupID: 0,
MIMEType: "",
@@ -97,8 +102,9 @@ func TestFileMetadataCataloger(t *testing.T) {
path: "/block-device-1",
exists: true,
expected: source.FileMetadata{
Path: "/block-device-1",
Mode: 0644 | os.ModeDevice,
Type: "BlockDevice",
Type: file.TypeBlockDevice,
UserID: 0,
GroupID: 0,
MIMEType: "",
@@ -108,8 +114,9 @@ func TestFileMetadataCataloger(t *testing.T) {
path: "/fifo-1",
exists: true,
expected: source.FileMetadata{
Path: "/fifo-1",
Mode: 0644 | os.ModeNamedPipe,
Type: "FIFONode",
Type: file.TypeFIFO,
UserID: 0,
GroupID: 0,
MIMEType: "",
@@ -119,11 +126,13 @@ func TestFileMetadataCataloger(t *testing.T) {
path: "/bin",
exists: true,
expected: source.FileMetadata{
Path: "/bin",
Mode: 0755 | os.ModeDir,
Type: "Directory",
Type: file.TypeDirectory,
UserID: 0,
GroupID: 0,
MIMEType: "",
IsDir: true,
},
},
}
@@ -131,11 +140,9 @@ func TestFileMetadataCataloger(t *testing.T) {
for _, test := range tests {
t.Run(test.path, func(t *testing.T) {
_, ref, err := img.SquashedTree().File(file.Path(test.path))
if err != nil {
t.Fatalf("unable to get file: %+v", err)
}
require.NoError(t, err)
l := source.NewLocationFromImage(test.path, *ref, img)
l := source.NewLocationFromImage(test.path, *ref.Reference, img)
assert.Equal(t, test.expected, actual[l.Coordinates], "mismatched metadata")
+2 -2
View File
@@ -30,12 +30,12 @@ const (
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func FormatIDs() (ids []sbom.FormatID) {
return formats.IDs()
return formats.AllIDs()
}
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func FormatByID(id sbom.FormatID) sbom.Format {
return formats.ByID(id)
return formats.ByNameAndVersion(string(id), "")
}
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
@@ -10,10 +10,11 @@ import (
func encodeLicenses(p pkg.Package) *cyclonedx.Licenses {
lc := cyclonedx.Licenses{}
for _, licenseName := range p.Licenses {
if value, exists := spdxlicense.ID(licenseName); exists {
if value, other, exists := spdxlicense.ID(licenseName); exists {
lc = append(lc, cyclonedx.LicenseChoice{
License: &cyclonedx.License{
ID: value,
ID: value,
Name: other,
},
})
}
@@ -28,7 +29,16 @@ func decodeLicenses(c *cyclonedx.Component) (out []string) {
if c.Licenses != nil {
for _, l := range *c.Licenses {
if l.License != nil {
out = append(out, l.License.ID)
var lic string
switch {
case l.License.ID != "":
lic = l.License.ID
case l.License.Name != "":
lic = l.License.Name
default:
continue
}
out = append(out, lic)
}
}
}
@@ -27,7 +27,9 @@ func Test_encodeLicense(t *testing.T) {
"made-up",
},
},
expected: nil,
expected: &cyclonedx.Licenses{
{License: &cyclonedx.License{Name: "made-up"}},
},
},
{
name: "with SPDX license",
+14 -6
View File
@@ -22,12 +22,7 @@ func License(p pkg.Package) string {
}
// take all licenses and assume an AND expression; for information about license expressions see https://spdx.github.io/spdx-spec/appendix-IV-SPDX-license-expressions/
var parsedLicenses []string
for _, l := range p.Licenses {
if value, exists := spdxlicense.ID(l); exists {
parsedLicenses = append(parsedLicenses, value)
}
}
parsedLicenses := parseLicenses(p.Licenses)
if len(parsedLicenses) == 0 {
return NOASSERTION
@@ -35,3 +30,16 @@ func License(p pkg.Package) string {
return strings.Join(parsedLicenses, " AND ")
}
func parseLicenses(raw []string) (parsedLicenses []string) {
for _, l := range raw {
if value, other, exists := spdxlicense.ID(l); exists {
parsed := value
if other != "" {
parsed = spdxlicense.LicenseRefPrefix + other
}
parsedLicenses = append(parsedLicenses, parsed)
}
}
return
}
@@ -26,7 +26,7 @@ func Test_License(t *testing.T) {
"made-up",
},
},
expected: NOASSERTION,
expected: "LicenseRef-made-up",
},
{
name: "with SPDX license",
@@ -8,8 +8,7 @@ import (
"strings"
"time"
"github.com/spdx/tools-golang/spdx/common"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/spdx/tools-golang/spdx"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
@@ -23,7 +22,6 @@ import (
)
const (
spdxVersion = "SPDX-2.3"
noAssertion = "NOASSERTION"
)
@@ -40,11 +38,11 @@ func ToFormatModel(s sbom.SBOM) *spdx.Document {
// for the primary package purpose field:
// https://spdx.github.io/spdx-spec/v2.3/package-information/#724-primary-package-purpose-field
documentDescribesRelationship := &spdx.Relationship{
RefA: common.DocElementID{
RefA: spdx.DocElementID{
ElementRefID: "DOCUMENT",
},
Relationship: string(DescribesRelationship),
RefB: common.DocElementID{
RefB: spdx.DocElementID{
ElementRefID: "DOCUMENT",
},
RelationshipComment: "",
@@ -55,11 +53,11 @@ func ToFormatModel(s sbom.SBOM) *spdx.Document {
return &spdx.Document{
// 6.1: SPDX Version; should be in the format "SPDX-x.x"
// Cardinality: mandatory, one
SPDXVersion: spdxVersion,
SPDXVersion: spdx.Version,
// 6.2: Data License; should be "CC0-1.0"
// Cardinality: mandatory, one
DataLicense: "CC0-1.0",
DataLicense: spdx.DataLicense,
// 6.3: SPDX Identifier; should be "DOCUMENT" to represent mandatory identifier of SPDXRef-DOCUMENT
// Cardinality: mandatory, one
@@ -104,7 +102,7 @@ func ToFormatModel(s sbom.SBOM) *spdx.Document {
// 6.8: Creators: may have multiple keys for Person, Organization
// and/or Tool
// Cardinality: mandatory, one or many
Creators: []common.Creator{
Creators: []spdx.Creator{
{
Creator: "Anchore, Inc",
CreatorType: "Organization",
@@ -126,10 +124,11 @@ func ToFormatModel(s sbom.SBOM) *spdx.Document {
Packages: toPackages(s.Artifacts.PackageCatalog, s),
Files: toFiles(s),
Relationships: relationships,
OtherLicenses: toOtherLicenses(s.Artifacts.PackageCatalog),
}
}
func toSPDXID(identifiable artifact.Identifiable) common.ElementID {
func toSPDXID(identifiable artifact.Identifiable) spdx.ElementID {
id := ""
if p, ok := identifiable.(pkg.Package); ok {
id = SanitizeElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.ID()))
@@ -137,7 +136,7 @@ func toSPDXID(identifiable artifact.Identifiable) common.ElementID {
id = string(identifiable.ID())
}
// NOTE: the spdx libraries prepend SPDXRef-, so we don't do it here
return common.ElementID(id)
return spdx.ElementID(id)
}
// packages populates all Package Information from the package Catalog (see https://spdx.github.io/spdx-spec/3-package-information/)
@@ -313,9 +312,9 @@ func toPackages(catalog *pkg.Catalog, sbom sbom.SBOM) (results []*spdx.Package)
return results
}
func toPackageChecksums(p pkg.Package) ([]common.Checksum, bool) {
func toPackageChecksums(p pkg.Package) ([]spdx.Checksum, bool) {
filesAnalyzed := false
var checksums []common.Checksum
var checksums []spdx.Checksum
switch meta := p.Metadata.(type) {
// we generate digest for some Java packages
// spdx.github.io/spdx-spec/package-information/#710-package-checksum-field
@@ -325,8 +324,8 @@ func toPackageChecksums(p pkg.Package) ([]common.Checksum, bool) {
filesAnalyzed = true
for _, digest := range meta.ArchiveDigests {
algo := strings.ToUpper(digest.Algorithm)
checksums = append(checksums, common.Checksum{
Algorithm: common.ChecksumAlgorithm(algo),
checksums = append(checksums, spdx.Checksum{
Algorithm: spdx.ChecksumAlgorithm(algo),
Value: digest.Value,
})
}
@@ -339,20 +338,20 @@ func toPackageChecksums(p pkg.Package) ([]common.Checksum, bool) {
break
}
algo = strings.ToUpper(algo)
checksums = append(checksums, common.Checksum{
Algorithm: common.ChecksumAlgorithm(algo),
checksums = append(checksums, spdx.Checksum{
Algorithm: spdx.ChecksumAlgorithm(algo),
Value: hexStr,
})
}
return checksums, filesAnalyzed
}
func toPackageOriginator(p pkg.Package) *common.Originator {
func toPackageOriginator(p pkg.Package) *spdx.Originator {
kind, originator := Originator(p)
if kind == "" || originator == "" {
return nil
}
return &common.Originator{
return &spdx.Originator{
Originator: originator,
OriginatorType: kind,
}
@@ -386,11 +385,11 @@ func toRelationships(relationships []artifact.Relationship) (result []*spdx.Rela
}
result = append(result, &spdx.Relationship{
RefA: common.DocElementID{
RefA: spdx.DocElementID{
ElementRefID: toSPDXID(r.From),
},
Relationship: string(relationshipType),
RefB: common.DocElementID{
RefB: spdx.DocElementID{
ElementRefID: toSPDXID(r.To),
},
RelationshipComment: comment,
@@ -462,10 +461,10 @@ func toFiles(s sbom.SBOM) (results []*spdx.File) {
return results
}
func toFileChecksums(digests []file.Digest) (checksums []common.Checksum) {
checksums = make([]common.Checksum, 0, len(digests))
func toFileChecksums(digests []file.Digest) (checksums []spdx.Checksum) {
checksums = make([]spdx.Checksum, 0, len(digests))
for _, digest := range digests {
checksums = append(checksums, common.Checksum{
checksums = append(checksums, spdx.Checksum{
Algorithm: toChecksumAlgorithm(digest.Algorithm),
Value: digest.Value,
})
@@ -473,9 +472,9 @@ func toFileChecksums(digests []file.Digest) (checksums []common.Checksum) {
return checksums
}
func toChecksumAlgorithm(algorithm string) common.ChecksumAlgorithm {
func toChecksumAlgorithm(algorithm string) spdx.ChecksumAlgorithm {
// this needs to be an uppercase version of our algorithm
return common.ChecksumAlgorithm(strings.ToUpper(algorithm))
return spdx.ChecksumAlgorithm(strings.ToUpper(algorithm))
}
func toFileTypes(metadata *source.FileMetadata) (ty []string) {
@@ -513,11 +512,33 @@ func toFileTypes(metadata *source.FileMetadata) (ty []string) {
return ty
}
func toOtherLicenses(catalog *pkg.Catalog) []*spdx.OtherLicense {
licenses := map[string]bool{}
for _, pkg := range catalog.Sorted() {
for _, license := range parseLicenses(pkg.Licenses) {
if strings.HasPrefix(license, spdxlicense.LicenseRefPrefix) {
licenses[license] = true
}
}
}
var result []*spdx.OtherLicense
for license := range licenses {
// separate the actual ID from the prefix
name := strings.TrimPrefix(license, spdxlicense.LicenseRefPrefix)
result = append(result, &spdx.OtherLicense{
LicenseIdentifier: license,
LicenseName: name,
ExtractedText: NONE, // we probably should have some extracted text here, but this is good enough for now
})
}
return result
}
// TODO: handle SPDX excludes file case
// f file is an "excludes" file, skip it /* exclude SPDX analysis file(s) */
// see: https://spdx.github.io/spdx-spec/v2.3/package-information/#79-package-verification-code-field
// the above link contains the SPDX algorithm for a package verification code
func newPackageVerificationCode(p pkg.Package, sbom sbom.SBOM) *common.PackageVerificationCode {
func newPackageVerificationCode(p pkg.Package, sbom sbom.SBOM) *spdx.PackageVerificationCode {
// key off of the contains relationship;
// spdx validator will fail if a package claims to contain a file but no sha1 provided
// if a sha1 for a file is provided then the validator will fail if the package does not have
@@ -558,7 +579,7 @@ func newPackageVerificationCode(p pkg.Package, sbom sbom.SBOM) *common.PackageVe
//nolint:gosec
hasher := sha1.New()
_, _ = hasher.Write([]byte(b.String()))
return &common.PackageVerificationCode{
return &spdx.PackageVerificationCode{
// 7.9.1: Package Verification Code Value
// Cardinality: mandatory, one
Value: fmt.Sprintf("%+x", hasher.Sum(nil)),
@@ -4,8 +4,7 @@ import (
"fmt"
"testing"
"github.com/spdx/tools-golang/spdx/common"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/spdx/tools-golang/spdx"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -21,7 +20,7 @@ func Test_toPackageChecksums(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expected []common.Checksum
expected []spdx.Checksum
filesAnalyzed bool
}{
{
@@ -39,7 +38,7 @@ func Test_toPackageChecksums(t *testing.T) {
},
},
},
expected: []common.Checksum{
expected: []spdx.Checksum{
{
Algorithm: "SHA1",
Value: "1234",
@@ -57,7 +56,7 @@ func Test_toPackageChecksums(t *testing.T) {
ArchiveDigests: []file.Digest{},
},
},
expected: []common.Checksum{},
expected: []spdx.Checksum{},
filesAnalyzed: false,
},
{
@@ -67,7 +66,7 @@ func Test_toPackageChecksums(t *testing.T) {
Version: "1.0.0",
Language: pkg.Java,
},
expected: []common.Checksum{},
expected: []spdx.Checksum{},
filesAnalyzed: false,
},
{
@@ -81,7 +80,7 @@ func Test_toPackageChecksums(t *testing.T) {
H1Digest: "h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
},
expected: []common.Checksum{
expected: []spdx.Checksum{
{
Algorithm: "SHA256",
Value: "f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
@@ -97,7 +96,7 @@ func Test_toPackageChecksums(t *testing.T) {
Language: pkg.Java,
Metadata: struct{}{},
},
expected: []common.Checksum{},
expected: []spdx.Checksum{},
filesAnalyzed: false,
},
}
@@ -229,7 +228,7 @@ func Test_toFileChecksums(t *testing.T) {
tests := []struct {
name string
digests []file.Digest
expected []common.Checksum
expected []spdx.Checksum
}{
{
name: "empty",
@@ -246,7 +245,7 @@ func Test_toFileChecksums(t *testing.T) {
Value: "meh",
},
},
expected: []common.Checksum{
expected: []spdx.Checksum{
{
Algorithm: "SHA256",
Value: "deadbeefcafe",
@@ -275,8 +274,8 @@ func Test_fileIDsForPackage(t *testing.T) {
FileSystemID: "nowhere",
}
docElementId := func(identifiable artifact.Identifiable) common.DocElementID {
return common.DocElementID{
docElementId := func(identifiable artifact.Identifiable) spdx.DocElementID {
return spdx.DocElementID{
ElementRefID: toSPDXID(identifiable),
}
}
@@ -6,7 +6,7 @@ import (
"strconv"
"strings"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/spdx/tools-golang/spdx"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal/log"
@@ -3,8 +3,7 @@ package spdxhelpers
import (
"testing"
"github.com/spdx/tools-golang/spdx/common"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/spdx/tools-golang/spdx"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -246,9 +245,9 @@ func TestH1Digest(t *testing.T) {
RefType: "purl",
},
},
PackageChecksums: []common.Checksum{
PackageChecksums: []spdx.Checksum{
{
Algorithm: common.SHA256,
Algorithm: spdx.SHA256,
Value: "f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
},
},
@@ -267,9 +266,9 @@ func TestH1Digest(t *testing.T) {
RefType: "purl",
},
},
PackageChecksums: []common.Checksum{
PackageChecksums: []spdx.Checksum{
{
Algorithm: common.SHA1,
Algorithm: spdx.SHA1,
Value: "f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
},
},
@@ -288,9 +287,9 @@ func TestH1Digest(t *testing.T) {
RefType: "purl",
},
},
PackageChecksums: []common.Checksum{
PackageChecksums: []spdx.Checksum{
{
Algorithm: common.SHA256,
Algorithm: spdx.SHA256,
Value: "",
},
},
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"regexp"
"testing"
"github.com/anchore/syft/syft/formats/common/testutils"
"github.com/anchore/syft/syft/formats/internal/testutils"
)
var updateCycloneDx = flag.Bool("update-cyclonedx", false, "update the *.golden files for cyclone-dx encoders")
+3 -2
View File
@@ -7,13 +7,14 @@ import (
"github.com/anchore/syft/syft/sbom"
)
const ID sbom.FormatID = "cyclonedx-1-json"
const ID sbom.FormatID = "cyclonedx-json"
func Format() sbom.Format {
return sbom.NewFormat(
ID,
sbom.AnyVersion,
encoder,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatJSON),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatJSON),
ID,
)
}
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"regexp"
"testing"
"github.com/anchore/syft/syft/formats/common/testutils"
"github.com/anchore/syft/syft/formats/internal/testutils"
)
var updateCycloneDx = flag.Bool("update-cyclonedx", false, "update the *.golden files for cyclone-dx encoders")
+3 -2
View File
@@ -7,13 +7,14 @@ import (
"github.com/anchore/syft/syft/sbom"
)
const ID sbom.FormatID = "cyclonedx-1-xml"
const ID sbom.FormatID = "cyclonedx-xml"
func Format() sbom.Format {
return sbom.NewFormat(
ID,
sbom.AnyVersion,
encoder,
cyclonedxhelpers.GetDecoder(cyclonedx.BOMFileFormatXML),
cyclonedxhelpers.GetValidator(cyclonedx.BOMFileFormatXML),
ID, "cyclonedx", "cyclone",
)
}
+57 -43
View File
@@ -5,8 +5,11 @@ import (
"errors"
"fmt"
"io"
"regexp"
"strings"
"golang.org/x/exp/slices"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/cyclonedxxml"
@@ -26,8 +29,11 @@ func Formats() []sbom.Format {
cyclonedxxml.Format(),
cyclonedxjson.Format(),
github.Format(),
spdxtagvalue.Format(),
spdxjson.Format(),
spdxtagvalue.Format2_1(),
spdxtagvalue.Format2_2(),
spdxtagvalue.Format2_3(),
spdxjson.Format2_2(),
spdxjson.Format2_3(),
table.Format(),
text.Format(),
template.Format(),
@@ -38,7 +44,7 @@ func Identify(by []byte) sbom.Format {
for _, f := range Formats() {
if err := f.Validate(bytes.NewReader(by)); err != nil {
if !errors.Is(err, sbom.ErrValidationNotSupported) {
log.Debugf("format %s returned err: %+v", f.ID(), err)
log.WithFields("error", err).Tracef("format validation for %s failed", f.ID())
}
continue
}
@@ -47,53 +53,51 @@ func Identify(by []byte) sbom.Format {
return nil
}
// ByName accepts a name@version string, such as:
//
// spdx-json@2.1 or cyclonedx@2
func ByName(name string) sbom.Format {
cleanName := cleanFormatName(name)
for _, f := range Formats() {
if cleanFormatName(string(f.ID())) == cleanName {
return f
}
parts := strings.SplitN(name, "@", 2)
version := sbom.AnyVersion
if len(parts) > 1 {
version = parts[1]
}
// handle any aliases for any supported format
switch cleanName {
case "json", "syftjson":
return ByID(syftjson.ID)
case "cyclonedx", "cyclone", "cyclonedxxml":
return ByID(cyclonedxxml.ID)
case "cyclonedxjson":
return ByID(cyclonedxjson.ID)
case "github", "githubjson":
return ByID(github.ID)
case "spdx", "spdxtv", "spdxtagvalue":
return ByID(spdxtagvalue.ID)
case "spdxjson":
return ByID(spdxjson.ID)
case "table":
return ByID(table.ID)
case "text":
return ByID(text.ID)
case "template":
ByID(template.ID)
}
return nil
return ByNameAndVersion(parts[0], version)
}
func IDs() (ids []sbom.FormatID) {
func ByNameAndVersion(name string, version string) sbom.Format {
name = cleanFormatName(name)
var mostRecentFormat sbom.Format
for _, f := range Formats() {
ids = append(ids, f.ID())
}
return ids
}
func ByID(id sbom.FormatID) sbom.Format {
for _, f := range Formats() {
if f.ID() == id {
return f
for _, n := range f.IDs() {
if cleanFormatName(string(n)) == name && versionMatches(f.Version(), version) {
if mostRecentFormat == nil || f.Version() > mostRecentFormat.Version() {
mostRecentFormat = f
}
}
}
}
return nil
return mostRecentFormat
}
func versionMatches(version string, match string) bool {
if version == sbom.AnyVersion || match == sbom.AnyVersion {
return true
}
dots := strings.Count(match, ".")
if dots == 0 {
match += ".*"
}
match = strings.ReplaceAll(match, ".", "\\.")
match = strings.ReplaceAll(match, "*", ".*")
match = strings.ReplaceAll(match, "\\..*", "(\\..*)*")
match = fmt.Sprintf("^%s$", match)
matcher, err := regexp.Compile(match)
if err != nil {
return false
}
return matcher.MatchString(version)
}
func cleanFormatName(name string) string {
@@ -127,3 +131,13 @@ func Decode(reader io.Reader) (*sbom.SBOM, sbom.Format, error) {
s, err := f.Decode(bytes.NewReader(by))
return s, f, err
}
func AllIDs() (ids []sbom.FormatID) {
for _, f := range Formats() {
if slices.Contains(ids, f.ID()) {
continue
}
ids = append(ids, f.ID())
}
return ids
}
+61 -19
View File
@@ -92,18 +92,6 @@ func TestByName(t *testing.T) {
name: "spdxtv", // clean variant
want: spdxtagvalue.ID,
},
{
name: "spdx-2-tag-value", // clean variant
want: spdxtagvalue.ID,
},
{
name: "spdx-2-tagvalue", // clean variant
want: spdxtagvalue.ID,
},
{
name: "spdx2-tagvalue", // clean variant
want: spdxtagvalue.ID,
},
// SPDX JSON
{
@@ -111,7 +99,7 @@ func TestByName(t *testing.T) {
want: spdxjson.ID,
},
{
name: "spdx-2-json",
name: "spdxjson", // clean variant
want: spdxjson.ID,
},
@@ -121,7 +109,7 @@ func TestByName(t *testing.T) {
want: cyclonedxjson.ID,
},
{
name: "cyclonedx-1-json",
name: "cyclonedxjson", // clean variant
want: cyclonedxjson.ID,
},
@@ -135,7 +123,7 @@ func TestByName(t *testing.T) {
want: cyclonedxxml.ID,
},
{
name: "cyclonedx-1-xml",
name: "cyclonedxxml", // clean variant
want: cyclonedxxml.ID,
},
@@ -144,7 +132,6 @@ func TestByName(t *testing.T) {
name: "table",
want: table.ID,
},
{
name: "syft-table",
want: table.ID,
@@ -155,7 +142,6 @@ func TestByName(t *testing.T) {
name: "text",
want: text.ID,
},
{
name: "syft-text",
want: text.ID,
@@ -166,23 +152,26 @@ func TestByName(t *testing.T) {
name: "json",
want: syftjson.ID,
},
{
name: "syft-json",
want: syftjson.ID,
},
{
name: "syftjson", // clean variant
want: syftjson.ID,
},
// GitHub JSON
{
name: "github",
want: github.ID,
},
{
name: "github-json",
want: github.ID,
},
// Syft template
{
name: "template",
want: template.ID,
@@ -200,3 +189,56 @@ func TestByName(t *testing.T) {
})
}
}
func Test_versionMatches(t *testing.T) {
tests := []struct {
name string
version string
match string
matches bool
}{
{
name: "any version matches number",
version: string(sbom.AnyVersion),
match: "6",
matches: true,
},
{
name: "number matches any version",
version: "6",
match: string(sbom.AnyVersion),
matches: true,
},
{
name: "same number matches",
version: "3",
match: "3",
matches: true,
},
{
name: "same major number matches",
version: "3.1",
match: "3",
matches: true,
},
{
name: "same minor number matches",
version: "3.1",
match: "3.1",
matches: true,
},
{
name: "different number does not match",
version: "3",
match: "4",
matches: false,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
matches := versionMatches(test.version, test.match)
assert.Equal(t, test.matches, matches)
})
}
}
+3 -2
View File
@@ -7,11 +7,11 @@ import (
"github.com/anchore/syft/syft/sbom"
)
const ID sbom.FormatID = "github-0-json"
const ID sbom.FormatID = "github-json"
func Format() sbom.Format {
return sbom.NewFormat(
ID,
sbom.AnyVersion,
func(writer io.Writer, sbom sbom.SBOM) error {
bom := toGithubModel(&sbom)
@@ -25,5 +25,6 @@ func Format() sbom.Format {
},
nil,
nil,
ID, "github",
)
}
@@ -168,7 +168,7 @@ func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
Name: "package-1",
Version: "1.0.1",
Locations: source.NewLocationSet(
source.NewLocationFromImage(string(ref1.RealPath), *ref1, img),
source.NewLocationFromImage(string(ref1.RealPath), *ref1.Reference, img),
),
Type: pkg.PythonPkg,
FoundBy: "the-cataloger-1",
@@ -188,7 +188,7 @@ func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
Name: "package-2",
Version: "2.0.1",
Locations: source.NewLocationSet(
source.NewLocationFromImage(string(ref2.RealPath), *ref2, img),
source.NewLocationFromImage(string(ref2.RealPath), *ref2.Reference, img),
),
Type: pkg.DebPkg,
FoundBy: "the-cataloger-2",
+2 -2
View File
@@ -4,14 +4,14 @@ import (
"fmt"
"io"
spdx "github.com/spdx/tools-golang/json"
"github.com/spdx/tools-golang/json"
"github.com/anchore/syft/syft/formats/common/spdxhelpers"
"github.com/anchore/syft/syft/sbom"
)
func decoder(reader io.Reader) (s *sbom.SBOM, err error) {
doc, err := spdx.Load2_3(reader)
doc, err := json.Read(reader)
if err != nil {
return nil, fmt.Errorf("unable to decode spdx-json: %w", err)
}
+19 -1
View File
@@ -4,13 +4,31 @@ import (
"encoding/json"
"io"
"github.com/spdx/tools-golang/convert"
"github.com/spdx/tools-golang/spdx/v2/v2_2"
"github.com/anchore/syft/syft/formats/common/spdxhelpers"
"github.com/anchore/syft/syft/sbom"
)
func encoder(output io.Writer, s sbom.SBOM) error {
func encoder2_3(output io.Writer, s sbom.SBOM) error {
doc := spdxhelpers.ToFormatModel(s)
return encodeJSON(output, doc)
}
func encoder2_2(output io.Writer, s sbom.SBOM) error {
doc := spdxhelpers.ToFormatModel(s)
var out v2_2.Document
err := convert.Document(doc, &out)
if err != nil {
return err
}
return encodeJSON(output, out)
}
func encodeJSON(output io.Writer, doc interface{}) error {
enc := json.NewEncoder(output)
// prevent > and < from being escaped in the payload
enc.SetEscapeHTML(false)
+4 -4
View File
@@ -5,7 +5,7 @@ import (
"regexp"
"testing"
"github.com/anchore/syft/syft/formats/common/testutils"
"github.com/anchore/syft/syft/formats/internal/testutils"
)
var updateSpdxJson = flag.Bool("update-spdx-json", false, "update the *.golden files for spdx-json encoders")
@@ -48,11 +48,11 @@ func TestSPDXRelationshipOrder(t *testing.T) {
func spdxJsonRedactor(s []byte) []byte {
// each SBOM reports the time it was generated, which is not useful during snapshot testing
s = regexp.MustCompile(`"created":\s+"[^"]*",?`).ReplaceAll(s, []byte(""))
s = regexp.MustCompile(`"created":\s+"[^"]*"`).ReplaceAll(s, []byte(`"created":""`))
// each SBOM reports a unique documentNamespace when generated, this is not useful for snapshot testing
s = regexp.MustCompile(`"documentNamespace":\s+"[^"]*",?`).ReplaceAll(s, []byte(""))
s = regexp.MustCompile(`"documentNamespace":\s+"[^"]*"`).ReplaceAll(s, []byte(`"documentNamespace":""`))
// the license list will be updated periodically, the value here should not be directly tested in snapshot tests
return regexp.MustCompile(`"licenseListVersion":\s+"[^"]*",?`).ReplaceAll(s, []byte(""))
return regexp.MustCompile(`"licenseListVersion":\s+"[^"]*"`).ReplaceAll(s, []byte(`"licenseListVersion":""`))
}
+20 -4
View File
@@ -4,14 +4,30 @@ import (
"github.com/anchore/syft/syft/sbom"
)
const ID sbom.FormatID = "spdx-2-json"
const ID sbom.FormatID = "spdx-json"
var IDs = []sbom.FormatID{ID}
// note: this format is LOSSY relative to the syftjson format
func Format() sbom.Format {
func Format2_2() sbom.Format {
return sbom.NewFormat(
ID,
encoder,
"2.2",
encoder2_2,
decoder,
validator,
IDs...,
)
}
func Format2_3() sbom.Format {
return sbom.NewFormat(
"2.3",
encoder2_3,
decoder,
validator,
IDs...,
)
}
var Format = Format2_3
@@ -3,15 +3,14 @@
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "/some/path",
"documentNamespace": "https://anchore.com/syft/dir/some/path-e13c8924-4bbc-42f8-bd30-4e1554472d62",
"documentNamespace": "https://anchore.com/syft/dir/some/path-1fe34646-a616-48c7-974b-3d1e27d406e3",
"creationInfo": {
"licenseListVersion": "3.19",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-v0.42.0-bogus"
],
"created": "2022-12-22T23:33:52Z",
"comment": ""
"created": "2023-01-20T21:41:03Z"
},
"packages": [
{
@@ -27,14 +26,12 @@
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"comment": ""
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"
},
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "a-purl-2",
"comment": ""
"referenceLocator": "a-purl-2"
}
]
},
@@ -51,14 +48,12 @@
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"comment": ""
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"
},
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:deb/debian/package-2@2.0.1",
"comment": ""
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
}
@@ -3,15 +3,14 @@
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "user-image-input",
"documentNamespace": "https://anchore.com/syft/image/user-image-input-a1cc9d58-830a-4a4b-9dcd-f41ea3001216",
"documentNamespace": "https://anchore.com/syft/image/user-image-input-33759ac3-6006-4f2c-bdc4-f40b9287a7f0",
"creationInfo": {
"licenseListVersion": "3.19",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-v0.42.0-bogus"
],
"created": "2022-12-22T23:33:53Z",
"comment": ""
"created": "2023-01-20T21:41:03Z"
},
"packages": [
{
@@ -27,14 +26,12 @@
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:1:*:*:*:*:*:*:*",
"comment": ""
"referenceLocator": "cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"
},
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "a-purl-1",
"comment": ""
"referenceLocator": "a-purl-1"
}
]
},
@@ -51,14 +48,12 @@
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"comment": ""
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"
},
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:deb/debian/package-2@2.0.1",
"comment": ""
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
}
@@ -3,15 +3,14 @@
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "user-image-input",
"documentNamespace": "https://anchore.com/syft/image/user-image-input-fc663ee3-0f9b-402e-827f-3f29aeff164e",
"documentNamespace": "https://anchore.com/syft/image/user-image-input-ce98f51f-b483-4e93-9a15-5a8a16d35de6",
"creationInfo": {
"licenseListVersion": "3.19",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-v0.42.0-bogus"
],
"created": "2022-12-22T23:33:53Z",
"comment": ""
"created": "2023-01-20T21:41:03Z"
},
"packages": [
{
@@ -27,14 +26,12 @@
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:1:*:*:*:*:*:*:*",
"comment": ""
"referenceLocator": "cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"
},
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "a-purl-1",
"comment": ""
"referenceLocator": "a-purl-1"
}
]
},
@@ -51,14 +48,12 @@
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"comment": ""
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"
},
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:deb/debian/package-2@2.0.1",
"comment": ""
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
}
+2 -2
View File
@@ -4,14 +4,14 @@ import (
"fmt"
"io"
"github.com/spdx/tools-golang/tvloader"
"github.com/spdx/tools-golang/tagvalue"
"github.com/anchore/syft/syft/formats/common/spdxhelpers"
"github.com/anchore/syft/syft/sbom"
)
func decoder(reader io.Reader) (*sbom.SBOM, error) {
doc, err := tvloader.Load2_3(reader)
doc, err := tagvalue.Read(reader)
if err != nil {
return nil, fmt.Errorf("unable to decode spdx-tag-value: %w", err)
}
+26 -3
View File
@@ -3,13 +3,36 @@ package spdxtagvalue
import (
"io"
"github.com/spdx/tools-golang/tvsaver"
"github.com/spdx/tools-golang/convert"
"github.com/spdx/tools-golang/spdx/v2/v2_1"
"github.com/spdx/tools-golang/spdx/v2/v2_2"
"github.com/spdx/tools-golang/tagvalue"
"github.com/anchore/syft/syft/formats/common/spdxhelpers"
"github.com/anchore/syft/syft/sbom"
)
func encoder(output io.Writer, s sbom.SBOM) error {
func encoder2_3(output io.Writer, s sbom.SBOM) error {
model := spdxhelpers.ToFormatModel(s)
return tvsaver.Save2_3(model, output)
return tagvalue.Write(model, output)
}
func encoder2_2(output io.Writer, s sbom.SBOM) error {
model := spdxhelpers.ToFormatModel(s)
var out v2_2.Document
err := convert.Document(model, &out)
if err != nil {
return err
}
return tagvalue.Write(out, output)
}
func encoder2_1(output io.Writer, s sbom.SBOM) error {
model := spdxhelpers.ToFormatModel(s)
var out v2_1.Document
err := convert.Document(model, &out)
if err != nil {
return err
}
return tagvalue.Write(out, output)
}
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"regexp"
"testing"
"github.com/anchore/syft/syft/formats/common/testutils"
"github.com/anchore/syft/syft/formats/internal/testutils"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
+30 -5
View File
@@ -4,14 +4,39 @@ import (
"github.com/anchore/syft/syft/sbom"
)
const ID sbom.FormatID = "spdx-2-tag-value"
const ID sbom.FormatID = "spdx-tag-value"
// note: this format is LOSSY relative to the syftjson formation, which means that decoding and validation is not supported at this time
func Format() sbom.Format {
var IDs = []sbom.FormatID{ID, "spdx", "spdx-tv"}
// note: this format is LOSSY relative to the syftjson format
func Format2_1() sbom.Format {
return sbom.NewFormat(
ID,
encoder,
"2.1",
encoder2_1,
decoder,
validator,
IDs...,
)
}
func Format2_2() sbom.Format {
return sbom.NewFormat(
"2.2",
encoder2_2,
decoder,
validator,
IDs...,
)
}
func Format2_3() sbom.Format {
return sbom.NewFormat(
"2.3",
encoder2_3,
decoder,
validator,
IDs...,
)
}
var Format = Format2_3
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"github.com/go-test/deep"
"github.com/stretchr/testify/assert"
"github.com/anchore/syft/syft/formats/common/testutils"
"github.com/anchore/syft/syft/formats/internal/testutils"
)
func TestEncodeDecodeCycle(t *testing.T) {
+17 -7
View File
@@ -2,12 +2,14 @@ package syftjson
import (
"flag"
"regexp"
"testing"
stereoFile "github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/formats/common/testutils"
"github.com/anchore/syft/syft/formats/internal/testutils"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
@@ -20,8 +22,9 @@ func TestDirectoryEncoder(t *testing.T) {
testutils.AssertEncoderAgainstGoldenSnapshot(t,
Format(),
testutils.DirectoryInput(t),
true,
*updateJson,
true,
schemaVersionRedactor,
)
}
@@ -31,11 +34,18 @@ func TestImageEncoder(t *testing.T) {
Format(),
testutils.ImageInput(t, testImage, testutils.FromSnapshot()),
testImage,
true,
*updateJson,
true,
schemaVersionRedactor,
)
}
func schemaVersionRedactor(s []byte) []byte {
pattern := regexp.MustCompile(`,?\s*"schema":\s*\{[^}]*}`)
out := pattern.ReplaceAll(s, []byte(""))
return out
}
func TestEncodeFullJSONDocument(t *testing.T) {
catalog := pkg.NewCatalog()
@@ -98,26 +108,26 @@ func TestEncodeFullJSONDocument(t *testing.T) {
FileMetadata: map[source.Coordinates]source.FileMetadata{
source.NewLocation("/a/place").Coordinates: {
Mode: 0775,
Type: "directory",
Type: stereoFile.TypeDirectory,
UserID: 0,
GroupID: 0,
},
source.NewLocation("/a/place/a").Coordinates: {
Mode: 0775,
Type: "regularFile",
Type: stereoFile.TypeRegular,
UserID: 0,
GroupID: 0,
},
source.NewLocation("/b").Coordinates: {
Mode: 0775,
Type: "symbolicLink",
Type: stereoFile.TypeSymLink,
LinkDestination: "/c",
UserID: 0,
GroupID: 0,
},
source.NewLocation("/b/place/b").Coordinates: {
Mode: 0644,
Type: "regularFile",
Type: stereoFile.TypeRegular,
UserID: 1,
GroupID: 2,
},
+3 -2
View File
@@ -4,13 +4,14 @@ import (
"github.com/anchore/syft/syft/sbom"
)
const ID sbom.FormatID = "syft-6-json"
const ID sbom.FormatID = "syft-json"
func Format() sbom.Format {
return sbom.NewFormat(
ID,
"6",
encoder,
decoder,
validator,
ID, "json", "syft",
)
}
+6 -6
View File
@@ -14,10 +14,10 @@ type File struct {
}
type FileMetadataEntry struct {
Mode int `json:"mode"`
Type source.FileType `json:"type"`
LinkDestination string `json:"linkDestination,omitempty"`
UserID int `json:"userID"`
GroupID int `json:"groupID"`
MIMEType string `json:"mimeType"`
Mode int `json:"mode"`
Type string `json:"type"`
LinkDestination string `json:"linkDestination,omitempty"`
UserID int `json:"userID"`
GroupID int `json:"groupID"`
MIMEType string `json:"mimeType"`
}
@@ -89,7 +89,7 @@
}
},
"schema": {
"version": "6.1.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.1.0.json"
"version": "6.2.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.2.0.json"
}
}
@@ -78,7 +78,7 @@
},
"metadata": {
"mode": 775,
"type": "directory",
"type": "Directory",
"userID": 0,
"groupID": 0,
"mimeType": ""
@@ -91,7 +91,7 @@
},
"metadata": {
"mode": 775,
"type": "regularFile",
"type": "RegularFile",
"userID": 0,
"groupID": 0,
"mimeType": ""
@@ -111,7 +111,7 @@
},
"metadata": {
"mode": 775,
"type": "symbolicLink",
"type": "SymbolicLink",
"linkDestination": "/c",
"userID": 0,
"groupID": 0,
@@ -125,7 +125,7 @@
},
"metadata": {
"mode": 644,
"type": "regularFile",
"type": "RegularFile",
"userID": 1,
"groupID": 2,
"mimeType": ""
@@ -9,7 +9,7 @@
"locations": [
{
"path": "/somefile-1.txt",
"layerID": "sha256:6afd1cb55939d87ba4c298907d0a53059bb3742c2d65314643e2464071cf0a2d"
"layerID": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59"
}
],
"licenses": [
@@ -40,7 +40,7 @@
"locations": [
{
"path": "/somefile-2.txt",
"layerID": "sha256:657997cff9a836139186239bdfe77250239a700d0ed97d57e101c295e8244319"
"layerID": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec"
}
],
"licenses": [],
@@ -64,11 +64,11 @@
],
"artifactRelationships": [],
"source": {
"id": "c85f7ae1b0ac38342c1cf1a6f7ea2b4b1ddc49cd1b24219ebd05dc10b3303491",
"id": "1a678f111c8ddc66fd82687bb024e0dd6af61314404937a80e810c0cf317b796",
"type": "image",
"target": {
"userInput": "user-image-input",
"imageID": "sha256:b5c0bfa8bcf70c75d92ebebbf76af667906d56e6fad50c37e7f93df824a64b79",
"imageID": "sha256:3c51b06feb0cda8ee62d0e3755ef2a8496a6b71f8a55b245f07f31c4bb813d31",
"manifestDigest": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"tags": [
@@ -78,17 +78,17 @@
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:6afd1cb55939d87ba4c298907d0a53059bb3742c2d65314643e2464071cf0a2d",
"digest": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59",
"size": 22
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:657997cff9a836139186239bdfe77250239a700d0ed97d57e101c295e8244319",
"digest": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec",
"size": 16
}
],
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NzMsImRpZ2VzdCI6InNoYTI1NjpiNWMwYmZhOGJjZjcwYzc1ZDkyZWJlYmJmNzZhZjY2NzkwNmQ1NmU2ZmFkNTBjMzdlN2Y5M2RmODI0YTY0Yjc5In0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1Njo2YWZkMWNiNTU5MzlkODdiYTRjMjk4OTA3ZDBhNTMwNTliYjM3NDJjMmQ2NTMxNDY0M2UyNDY0MDcxY2YwYTJkIn0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjY1Nzk5N2NmZjlhODM2MTM5MTg2MjM5YmRmZTc3MjUwMjM5YTcwMGQwZWQ5N2Q1N2UxMDFjMjk1ZTgyNDQzMTkifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjItMDgtMjVUMTY6MjI6MDguODkxMzY0Mjc4WiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIyLTA4LTI1VDE2OjIyOjA4Ljc2MzMzMDMyM1oiLCJjcmVhdGVkX2J5IjoiQUREIGZpbGUtMS50eHQgL3NvbWVmaWxlLTEudHh0ICMgYnVpbGRraXQiLCJjb21tZW50IjoiYnVpbGRraXQuZG9ja2VyZmlsZS52MCJ9LHsiY3JlYXRlZCI6IjIwMjItMDgtMjVUMTY6MjI6MDguODkxMzY0Mjc4WiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0yLnR4dCAvc29tZWZpbGUtMi50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn1dLCJvcyI6ImxpbnV4Iiwicm9vdGZzIjp7InR5cGUiOiJsYXllcnMiLCJkaWZmX2lkcyI6WyJzaGEyNTY6NmFmZDFjYjU1OTM5ZDg3YmE0YzI5ODkwN2QwYTUzMDU5YmIzNzQyYzJkNjUzMTQ2NDNlMjQ2NDA3MWNmMGEyZCIsInNoYTI1Njo2NTc5OTdjZmY5YTgzNjEzOTE4NjIzOWJkZmU3NzI1MDIzOWE3MDBkMGVkOTdkNTdlMTAxYzI5NWU4MjQ0MzE5Il19fQ==",
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NzMsImRpZ2VzdCI6InNoYTI1NjozYzUxYjA2ZmViMGNkYThlZTYyZDBlMzc1NWVmMmE4NDk2YTZiNzFmOGE1NWIyNDVmMDdmMzFjNGJiODEzZDMxIn0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1NjpmYjZiZWVjYjc1YjM5ZjRiYjgxM2RiZjE3N2U1MDFlZGQ1ZGRiM2U2OWJiNDVjZWRlYjc4YzY3NmVlMWI3YTU5In0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjMxOWI1ODhjZTY0MjUzYTg3YjUzM2M4ZWQwMWNmMDAyNWUwZWFjOThlN2I1MTZlMTI1MzI5NTdlMTI0NGZkZWMifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjItMDgtMDFUMjA6MDk6MjIuNTA5NDIxNzEyWiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIyLTA4LTAxVDIwOjA5OjIyLjQ4Nzg5NTUxOVoiLCJjcmVhdGVkX2J5IjoiQUREIGZpbGUtMS50eHQgL3NvbWVmaWxlLTEudHh0ICMgYnVpbGRraXQiLCJjb21tZW50IjoiYnVpbGRraXQuZG9ja2VyZmlsZS52MCJ9LHsiY3JlYXRlZCI6IjIwMjItMDgtMDFUMjA6MDk6MjIuNTA5NDIxNzEyWiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0yLnR4dCAvc29tZWZpbGUtMi50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn1dLCJvcyI6ImxpbnV4Iiwicm9vdGZzIjp7InR5cGUiOiJsYXllcnMiLCJkaWZmX2lkcyI6WyJzaGEyNTY6ZmI2YmVlY2I3NWIzOWY0YmI4MTNkYmYxNzdlNTAxZWRkNWRkYjNlNjliYjQ1Y2VkZWI3OGM2NzZlZTFiN2E1OSIsInNoYTI1NjozMTliNTg4Y2U2NDI1M2E4N2I1MzNjOGVkMDFjZjAwMjVlMGVhYzk4ZTdiNTE2ZTEyNTMyOTU3ZTEyNDRmZGVjIl19fQ==",
"repoDigests": [],
"architecture": "",
"os": ""
@@ -112,7 +112,7 @@
}
},
"schema": {
"version": "6.1.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.1.0.json"
"version": "6.2.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.2.0.json"
}
}
+27 -1
View File
@@ -5,6 +5,7 @@ import (
"sort"
"strconv"
stereoscopeFile "github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
@@ -137,7 +138,7 @@ func toFileMetadataEntry(coordinates source.Coordinates, metadata *source.FileMe
return &model.FileMetadataEntry{
Mode: mode,
Type: metadata.Type,
Type: toFileType(metadata.Type),
LinkDestination: metadata.LinkDestination,
UserID: metadata.UserID,
GroupID: metadata.GroupID,
@@ -145,6 +146,31 @@ func toFileMetadataEntry(coordinates source.Coordinates, metadata *source.FileMe
}
}
func toFileType(ty stereoscopeFile.Type) string {
switch ty {
case stereoscopeFile.TypeSymLink:
return "SymbolicLink"
case stereoscopeFile.TypeHardLink:
return "HardLink"
case stereoscopeFile.TypeDirectory:
return "Directory"
case stereoscopeFile.TypeSocket:
return "Socket"
case stereoscopeFile.TypeBlockDevice:
return "BlockDevice"
case stereoscopeFile.TypeCharacterDevice:
return "CharacterDevice"
case stereoscopeFile.TypeFIFO:
return "FIFONode"
case stereoscopeFile.TypeRegular:
return "RegularFile"
case stereoscopeFile.TypeIrregular:
return "IrregularFile"
default:
return "Unknown"
}
}
func toPackageModels(catalog *pkg.Catalog) []model.Package {
artifacts := make([]model.Package, 0)
if catalog == nil {
+63 -2
View File
@@ -8,6 +8,7 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/stereoscope/pkg/file"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/formats/syftjson/model"
"github.com/anchore/syft/syft/source"
@@ -15,8 +16,7 @@ import (
func Test_SyftJsonID_Compatibility(t *testing.T) {
jsonMajorVersion := strings.Split(internal.JSONSchemaVersion, ".")[0]
syftJsonIDVersion := strings.Split(string(ID), "-")[1]
assert.Equal(t, jsonMajorVersion, syftJsonIDVersion)
assert.Equal(t, jsonMajorVersion, string(Format().Version()))
}
func Test_toSourceModel(t *testing.T) {
@@ -98,3 +98,64 @@ func Test_toSourceModel(t *testing.T) {
// assert all possible schemes were under test
assert.ElementsMatch(t, allSchemes.List(), testedSchemes.List(), "not all source.Schemes are under test")
}
func Test_toFileType(t *testing.T) {
badType := file.Type(0x1337)
var allTypesTested []file.Type
tests := []struct {
ty file.Type
name string
}{
{
ty: file.TypeRegular,
name: "RegularFile",
},
{
ty: file.TypeDirectory,
name: "Directory",
},
{
ty: file.TypeSymLink,
name: "SymbolicLink",
},
{
ty: file.TypeHardLink,
name: "HardLink",
},
{
ty: file.TypeSocket,
name: "Socket",
},
{
ty: file.TypeCharacterDevice,
name: "CharacterDevice",
},
{
ty: file.TypeBlockDevice,
name: "BlockDevice",
},
{
ty: file.TypeFIFO,
name: "FIFONode",
},
{
ty: file.TypeIrregular,
name: "IrregularFile",
},
{
ty: badType,
name: "Unknown",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equalf(t, tt.name, toFileType(tt.ty), "toFileType(%v)", tt.ty)
if tt.ty != badType {
allTypesTested = append(allTypesTested, tt.ty)
}
})
}
assert.ElementsMatch(t, allTypesTested, file.AllTypes(), "not all file.Types are under test")
}
+2 -2
View File
@@ -6,7 +6,7 @@ import (
"github.com/go-test/deep"
"github.com/anchore/syft/syft/formats/common/testutils"
"github.com/anchore/syft/syft/formats/internal/testutils"
)
var updateTableGoldenFiles = flag.Bool("update-table", false, "update the *.golden files for table format")
@@ -15,8 +15,8 @@ func TestTableEncoder(t *testing.T) {
testutils.AssertEncoderAgainstGoldenSnapshot(t,
Format(),
testutils.DirectoryInput(t),
false,
*updateTableGoldenFiles,
false,
)
}
+2 -1
View File
@@ -8,9 +8,10 @@ const ID sbom.FormatID = "syft-table"
func Format() sbom.Format {
return sbom.NewFormat(
ID,
sbom.AnyVersion,
encoder,
nil,
nil,
ID, "table",
)
}
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"github.com/stretchr/testify/assert"
"github.com/anchore/syft/syft/formats/common/testutils"
"github.com/anchore/syft/syft/formats/internal/testutils"
)
var updateTmpl = flag.Bool("update-tmpl", false, "update the *.golden files for json encoders")
+17 -2
View File
@@ -1,6 +1,7 @@
package template
import (
"fmt"
"io"
"github.com/anchore/syft/syft/formats/syftjson"
@@ -23,7 +24,19 @@ func (f OutputFormat) ID() sbom.FormatID {
return ID
}
func (f OutputFormat) Decode(reader io.Reader) (*sbom.SBOM, error) {
func (f OutputFormat) IDs() []sbom.FormatID {
return []sbom.FormatID{ID}
}
func (f OutputFormat) Version() string {
return sbom.AnyVersion
}
func (f OutputFormat) String() string {
return fmt.Sprintf("template: " + f.templateFilePath)
}
func (f OutputFormat) Decode(_ io.Reader) (*sbom.SBOM, error) {
return nil, sbom.ErrDecodingNotSupported
}
@@ -37,7 +50,7 @@ func (f OutputFormat) Encode(output io.Writer, s sbom.SBOM) error {
return tmpl.Execute(output, doc)
}
func (f OutputFormat) Validate(reader io.Reader) error {
func (f OutputFormat) Validate(_ io.Reader) error {
return sbom.ErrValidationNotSupported
}
@@ -45,3 +58,5 @@ func (f OutputFormat) Validate(reader io.Reader) error {
func (f *OutputFormat) SetTemplatePath(filePath string) {
f.templateFilePath = filePath
}
var _ sbom.Format = (*OutputFormat)(nil)
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"flag"
"testing"
"github.com/anchore/syft/syft/formats/common/testutils"
"github.com/anchore/syft/syft/formats/internal/testutils"
)
var updateTextEncoderGoldenFiles = flag.Bool("update-text", false, "update the *.golden files for text encoder")
+2 -1
View File
@@ -8,9 +8,10 @@ const ID sbom.FormatID = "syft-text"
func Format() sbom.Format {
return sbom.NewFormat(
ID,
sbom.AnyVersion,
encoder,
nil,
nil,
ID, "text",
)
}
+209
View File
@@ -0,0 +1,209 @@
package alpm
import (
"testing"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
"github.com/anchore/syft/syft/source"
)
func TestAlpmCataloger(t *testing.T) {
expectedPkgs := []pkg.Package{
{
Name: "gmp",
Version: "6.2.1-2",
Type: pkg.AlpmPkg,
FoundBy: "alpmdb-cataloger",
Licenses: []string{"LGPL3", "GPL"},
Locations: source.NewLocationSet(source.NewLocation("var/lib/pacman/local/gmp-6.2.1-2/desc")),
CPEs: nil,
PURL: "",
MetadataType: "AlpmMetadata",
Metadata: pkg.AlpmMetadata{
BasePackage: "gmp",
Package: "gmp",
Version: "6.2.1-2",
Description: "A free library for arbitrary precision arithmetic",
Architecture: "x86_64",
Size: 1044438,
Packager: "Antonio Rojas <arojas@archlinux.org>",
License: "LGPL3\nGPL",
URL: "https://gmplib.org/",
Validation: "pgp",
Reason: 1,
Files: []pkg.AlpmFileRecord{
{
Path: "/usr",
Type: "dir",
Digests: []file.Digest{},
},
{
Path: "/usr/include",
Type: "dir",
Digests: []file.Digest{},
},
{
Path: "/usr/include/gmp.h",
Size: "84140",
Digests: []file.Digest{
{Algorithm: "md5", Value: "76595f70565c72550eb520809bf86856"},
{Algorithm: "sha256", Value: "91a614b9202453153fe3b7512d15e89659108b93ce8841c8e13789eb85da9e3a"},
},
},
{
Path: "/usr/include/gmpxx.h",
Size: "129113",
Digests: []file.Digest{
{Algorithm: "md5", Value: "ea3d21de4bcf7c696799c5c55dd3655b"},
{Algorithm: "sha256", Value: "0011ae411a0bc1030e07d968b32fdc1343f5ac2a17b7d28f493e7976dde2ac82"},
},
},
{
Path: "/usr/lib",
Type: "dir",
Digests: []file.Digest{},
},
{
Path: "/usr/lib/libgmp.so",
Type: "link",
Link: "libgmp.so.10.4.1",
Digests: []file.Digest{},
},
{
Path: "/usr/lib/libgmp.so.10",
Type: "link",
Link: "libgmp.so.10.4.1",
Digests: []file.Digest{},
},
{
Path: "/usr/lib/libgmp.so.10.4.1",
Size: "663224",
Digests: []file.Digest{
{Algorithm: "md5", Value: "d6d03eadacdd9048d5b2adf577e9d722"},
{Algorithm: "sha256", Value: "39898bd3d8d6785222432fa8b8aef7ce3b7e5bbfc66a52b7c0da09bed4adbe6a"},
},
},
{
Path: "/usr/lib/libgmpxx.so",
Type: "link",
Link: "libgmpxx.so.4.6.1",
Digests: []file.Digest{},
},
{
Path: "/usr/lib/libgmpxx.so.4",
Type: "link",
Link: "libgmpxx.so.4.6.1",
Digests: []file.Digest{},
},
{
Path: "/usr/lib/libgmpxx.so.4.6.1",
Size: "30680",
Digests: []file.Digest{
{Algorithm: "md5", Value: "dd5f0c4d635fa599fa7f4339c0e8814d"},
{Algorithm: "sha256", Value: "0ef67cbde4841f58d2e4b41f59425eb87c9eeaf4e649c060b326342c53bedbec"},
},
},
{
Path: "/usr/lib/pkgconfig",
Type: "dir",
Digests: []file.Digest{},
},
{
Path: "/usr/lib/pkgconfig/gmp.pc",
Size: "245",
Digests: []file.Digest{
{Algorithm: "md5", Value: "a91a9f1b66218cb77b9cd2cdf341756d"},
{Algorithm: "sha256", Value: "4e9de547a48c4e443781e9fa702a1ec5a23ee28b4bc520306cff2541a855be37"},
},
},
{
Path: "/usr/lib/pkgconfig/gmpxx.pc",
Size: "280",
Digests: []file.Digest{
{Algorithm: "md5", Value: "8c0f54e987934352177a6a30a811b001"},
{Algorithm: "sha256", Value: "fc5dbfbe75977057ba50953d94b9daecf696c9fdfe5b94692b832b44ecca871b"},
},
},
{
Path: "/usr/share",
Type: "dir",
Digests: []file.Digest{},
},
{
Path: "/usr/share/info",
Type: "dir",
Digests: []file.Digest{},
},
{
Path: "/usr/share/info/gmp.info-1.gz",
Size: "85892",
Digests: []file.Digest{
{Algorithm: "md5", Value: "63304d4d2f0247fb8a999fae66a81c19"},
{Algorithm: "sha256", Value: "86288c1531a2789db5da8b9838b5cde4db07bda230ae11eba23a1f33698bd14e"},
},
},
{
Path: "/usr/share/info/gmp.info-2.gz",
Size: "48484",
Digests: []file.Digest{
{Algorithm: "md5", Value: "4bb0dadec416d305232cac6eae712ff7"},
{Algorithm: "sha256", Value: "b7443c1b529588d98a074266087f79b595657ac7274191c34b10a9ceedfa950e"},
},
},
{
Path: "/usr/share/info/gmp.info.gz",
Size: "2380",
Digests: []file.Digest{
{Algorithm: "md5", Value: "cf6880fb0d862ee1da0d13c3831b5720"},
{Algorithm: "sha256", Value: "a13c8eecda3f3e5ad1e09773e47a9686f07d9d494eaddf326f3696bbef1548fd"},
},
},
},
Backup: []pkg.AlpmFileRecord{},
},
},
}
// TODO: relationships are not under test yet
var expectedRelationships []artifact.Relationship
pkgtest.NewCatalogTester().
FromDirectory(t, "test-fixtures/gmp-fixture").
WithCompareOptions(cmpopts.IgnoreFields(pkg.AlpmFileRecord{}, "Time")).
Expects(expectedPkgs, expectedRelationships).
TestCataloger(t, NewAlpmdbCataloger())
}
func TestCataloger_Globs(t *testing.T) {
tests := []struct {
name string
fixture string
expected []string
}{
{
name: "obtain description files",
fixture: "test-fixtures/glob-paths",
expected: []string{
"var/lib/pacman/local/base-1.0/desc",
"var/lib/pacman/local/dive-0.10.0/desc",
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
pkgtest.NewCatalogTester().
FromDirectory(t, test.fixture).
ExpectsResolverContentQueries(test.expected).
IgnoreUnfulfilledPathResponses("var/lib/pacman/local/base-1.0/mtree", "var/lib/pacman/local/dive-0.10.0/mtree").
TestCataloger(t, NewAlpmdbCataloger())
})
}
}
+2 -3
View File
@@ -1,9 +1,8 @@
package alpm
import (
"strings"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
@@ -15,7 +14,7 @@ func newPackage(m pkg.AlpmMetadata, release *linux.Release, locations ...source.
Version: m.Version,
Locations: source.NewLocationSet(locations...),
Type: pkg.AlpmPkg,
Licenses: strings.Split(m.License, " "),
Licenses: internal.SplitAny(m.License, " \n"),
PURL: packageURL(m, release),
MetadataType: pkg.AlpmMetadataType,
Metadata: m,
+4
View File
@@ -65,6 +65,10 @@ func parseAlpmDB(resolver source.FileResolver, env *generic.Environment, reader
metadata.Backup = filesMetadata.Backup
}
if metadata.Package == "" {
return nil, nil, nil
}
return []pkg.Package{
newPackage(*metadata, env.LinuxRelease, reader.Location),
}, nil, nil
@@ -16,10 +16,12 @@ import (
func TestDatabaseParser(t *testing.T) {
tests := []struct {
name string
fixture string
expected pkg.AlpmMetadata
}{
{
name: "test alpm database parsing",
name: "test alpm database parsing",
fixture: "test-fixtures/files",
expected: pkg.AlpmMetadata{
Backup: []pkg.AlpmFileRecord{
{
@@ -90,7 +92,7 @@ func TestDatabaseParser(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
f, err := os.Open("test-fixtures/files")
f, err := os.Open(test.fixture)
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, f.Close()) })
@@ -0,0 +1 @@
bogus desc file
@@ -0,0 +1,44 @@
%NAME%
gmp
%VERSION%
6.2.1-2
%BASE%
gmp
%DESC%
A free library for arbitrary precision arithmetic
%URL%
https://gmplib.org/
%ARCH%
x86_64
%BUILDDATE%
1653121258
%INSTALLDATE%
1665878640
%PACKAGER%
Antonio Rojas <arojas@archlinux.org>
%SIZE%
1044438
%REASON%
1
%LICENSE%
LGPL3
GPL
%VALIDATION%
pgp
%DEPENDS%
gcc-libs
sh
@@ -0,0 +1,21 @@
%FILES%
usr/
usr/include/
usr/include/gmp.h
usr/include/gmpxx.h
usr/lib/
usr/lib/libgmp.so
usr/lib/libgmp.so.10
usr/lib/libgmp.so.10.4.1
usr/lib/libgmpxx.so
usr/lib/libgmpxx.so.4
usr/lib/libgmpxx.so.4.6.1
usr/lib/pkgconfig/
usr/lib/pkgconfig/gmp.pc
usr/lib/pkgconfig/gmpxx.pc
usr/share/
usr/share/info/
usr/share/info/gmp.info-1.gz
usr/share/info/gmp.info-2.gz
usr/share/info/gmp.info.gz
@@ -0,0 +1,30 @@
package apkdb
import (
"testing"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
func TestCataloger_Globs(t *testing.T) {
tests := []struct {
name string
fixture string
expected []string
}{
{
name: "obtain DB files",
fixture: "test-fixtures/glob-paths",
expected: []string{"lib/apk/db/installed"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
pkgtest.NewCatalogTester().
FromDirectory(t, test.fixture).
ExpectsResolverContentQueries(test.expected).
TestCataloger(t, NewApkdbCataloger())
})
}
}
+2 -8
View File
@@ -7,6 +7,7 @@ import (
"strconv"
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
@@ -350,19 +351,12 @@ func discoverPackageDependencies(pkgs []pkg.Package) (relationships []artifact.R
return relationships
}
func splitAny(s string, seps string) []string {
splitter := func(r rune) bool {
return strings.ContainsRune(seps, r)
}
return strings.FieldsFunc(s, splitter)
}
func stripVersionSpecifier(s string) string {
// examples:
// musl>=1 --> musl
// cmd:scanelf=1.3.4-r0 --> cmd:scanelf
items := splitAny(s, "<>=")
items := internal.SplitAny(s, "<>=")
if len(items) == 0 {
return s
}
@@ -0,0 +1 @@
bogus db contents
+1
View File
@@ -32,6 +32,7 @@ func (c Cataloger) Catalog(resolver source.FileResolver) ([]pkg.Package, []artif
var relationships []artifact.Relationship
for _, cls := range defaultClassifiers {
log.WithFields("classifier", cls.Class).Trace("cataloging binaries")
pkgs, err := catalog(resolver, cls)
if err != nil {
log.WithFields("error", err, "classifier", cls.Class).Warn("unable to catalog binary package: %w", err)
+115 -13
View File
@@ -13,12 +13,96 @@ import (
"github.com/anchore/syft/syft/source"
)
func TestClassifierCataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
func Test_Cataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
tests := []struct {
name string
fixtureDir string
expected pkg.Package
}{
{
name: "positive-postgresql-15beta4",
fixtureDir: "test-fixtures/classifiers/positive/postgresql-15beta4",
expected: pkg.Package{
Name: "postgresql",
Version: "15beta4",
Type: "binary",
PURL: "pkg:generic/postgresql@15beta4",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
},
},
{
name: "positive-postgresql-15.1",
fixtureDir: "test-fixtures/classifiers/positive/postgresql-15.1",
expected: pkg.Package{
Name: "postgresql",
Version: "15.1",
Type: "binary",
PURL: "pkg:generic/postgresql@15.1",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
},
},
{
name: "positive-postgresql-9.6.24",
fixtureDir: "test-fixtures/classifiers/positive/postgresql-9.6.24",
expected: pkg.Package{
Name: "postgresql",
Version: "9.6.24",
Type: "binary",
PURL: "pkg:generic/postgresql@9.6.24",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
},
},
{
name: "positive-postgresql-9.5alpha1",
fixtureDir: "test-fixtures/classifiers/positive/postgresql-9.5alpha1",
expected: pkg.Package{
Name: "postgresql",
Version: "9.5alpha1",
Type: "binary",
PURL: "pkg:generic/postgresql@9.5alpha1",
Locations: singleLocation("postgres"),
Metadata: pkg.BinaryMetadata{
Classifier: "postgresql-binary",
},
},
},
{
name: "positive-traefik-2.9.6",
fixtureDir: "test-fixtures/classifiers/positive/traefik-2.9.6",
expected: pkg.Package{
Name: "traefik",
Version: "2.9.6",
Type: "binary",
PURL: "pkg:generic/traefik@2.9.6",
Locations: singleLocation("traefik"),
Metadata: pkg.BinaryMetadata{
Classifier: "traefik-binary",
},
},
},
{
name: "positive-traefik-1.7.34",
fixtureDir: "test-fixtures/classifiers/positive/traefik-1.7.34",
expected: pkg.Package{
Name: "traefik",
Version: "1.7.34",
Type: "binary",
PURL: "pkg:generic/traefik@1.7.34",
Locations: singleLocation("traefik"),
Metadata: pkg.BinaryMetadata{
Classifier: "traefik-binary",
},
},
},
{
name: "positive-memcached-1.6.18",
fixtureDir: "test-fixtures/classifiers/positive/memcached-1.6.18",
@@ -356,7 +440,7 @@ func TestClassifierCataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
}
}
func TestClassifierCataloger_DefaultClassifiers_PositiveCases_Image(t *testing.T) {
func Test_Cataloger_DefaultClassifiers_PositiveCases_Image(t *testing.T) {
tests := []struct {
name string
fixtureImage string
@@ -437,39 +521,57 @@ func assertPackagesAreEqual(t *testing.T, expected pkg.Package, p pkg.Package) {
}
type panicyResolver struct {
globCalled bool
searchCalled bool
}
func (p panicyResolver) FileContentsByLocation(location source.Location) (io.ReadCloser, error) {
func (p *panicyResolver) FilesByExtension(_ ...string) ([]source.Location, error) {
p.searchCalled = true
return nil, errors.New("not implemented")
}
func (p panicyResolver) HasPath(s string) bool {
func (p *panicyResolver) FilesByBasename(_ ...string) ([]source.Location, error) {
p.searchCalled = true
return nil, errors.New("not implemented")
}
func (p *panicyResolver) FilesByBasenameGlob(_ ...string) ([]source.Location, error) {
p.searchCalled = true
return nil, errors.New("not implemented")
}
func (p *panicyResolver) FileContentsByLocation(_ source.Location) (io.ReadCloser, error) {
p.searchCalled = true
return nil, errors.New("not implemented")
}
func (p *panicyResolver) HasPath(s string) bool {
return true
}
func (p panicyResolver) FilesByPath(paths ...string) ([]source.Location, error) {
func (p *panicyResolver) FilesByPath(_ ...string) ([]source.Location, error) {
p.searchCalled = true
return nil, errors.New("not implemented")
}
func (p *panicyResolver) FilesByGlob(patterns ...string) ([]source.Location, error) {
p.globCalled = true
func (p *panicyResolver) FilesByGlob(_ ...string) ([]source.Location, error) {
p.searchCalled = true
return nil, errors.New("not implemented")
}
func (p panicyResolver) FilesByMIMEType(types ...string) ([]source.Location, error) {
func (p *panicyResolver) FilesByMIMEType(_ ...string) ([]source.Location, error) {
p.searchCalled = true
return nil, errors.New("not implemented")
}
func (p panicyResolver) RelativeFileByPath(_ source.Location, path string) *source.Location {
func (p *panicyResolver) RelativeFileByPath(_ source.Location, _ string) *source.Location {
return nil
}
func (p panicyResolver) AllLocations() <-chan source.Location {
func (p *panicyResolver) AllLocations() <-chan source.Location {
return nil
}
func (p panicyResolver) FileMetadataByLocation(location source.Location) (source.FileMetadata, error) {
func (p *panicyResolver) FileMetadataByLocation(_ source.Location) (source.FileMetadata, error) {
return source.FileMetadata{}, errors.New("not implemented")
}
@@ -479,5 +581,5 @@ func Test_Cataloger_ResilientToErrors(t *testing.T) {
resolver := &panicyResolver{}
_, _, err := c.Catalog(resolver)
assert.NoError(t, err)
assert.True(t, resolver.globCalled)
assert.True(t, resolver.searchCalled)
}
+3 -3
View File
@@ -21,7 +21,7 @@ func Test_ClassifierCPEs(t *testing.T) {
fixture: "test-fixtures/version.txt",
classifier: classifier{
Package: "some-app",
FileGlob: ".*/version.txt",
FileGlob: "**/version.txt",
EvidenceMatcher: fileContentsVersionMatcher(`(?m)my-verison:(?P<version>[0-9.]+)`),
CPEs: []cpe.CPE{},
},
@@ -32,7 +32,7 @@ func Test_ClassifierCPEs(t *testing.T) {
fixture: "test-fixtures/version.txt",
classifier: classifier{
Package: "some-app",
FileGlob: ".*/version.txt",
FileGlob: "**/version.txt",
EvidenceMatcher: fileContentsVersionMatcher(`(?m)my-verison:(?P<version>[0-9.]+)`),
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:a:some:app:*:*:*:*:*:*:*:*"),
@@ -47,7 +47,7 @@ func Test_ClassifierCPEs(t *testing.T) {
fixture: "test-fixtures/version.txt",
classifier: classifier{
Package: "some-app",
FileGlob: ".*/version.txt",
FileGlob: "**/version.txt",
EvidenceMatcher: fileContentsVersionMatcher(`(?m)my-verison:(?P<version>[0-9.]+)`),
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:a:some:app:*:*:*:*:*:*:*:*"),
@@ -1,6 +1,9 @@
package binary
import "github.com/anchore/syft/syft/pkg"
import (
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/pkg"
)
var defaultClassifiers = []classifier{
{
@@ -11,6 +14,10 @@ var defaultClassifiers = []classifier{
`(?m)(?P<version>{{ .version }}\.[0-9]+[-_a-zA-Z0-9]*)`),
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:a:python_software_foundation:python:*:*:*:*:*:*:*:*"),
cpe.Must("cpe:2.3:a:python:python:*:*:*:*:*:*:*:*"),
},
},
{
Class: "python-binary-lib",
@@ -20,6 +27,10 @@ var defaultClassifiers = []classifier{
`(?m)(?P<version>{{ .version }}\.[0-9]+[-_a-zA-Z0-9]*)`),
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:a:python_software_foundation:python:*:*:*:*:*:*:*:*"),
cpe.Must("cpe:2.3:a:python:python:*:*:*:*:*:*:*:*"),
},
},
{
Class: "cpython-source",
@@ -28,6 +39,10 @@ var defaultClassifiers = []classifier{
`(?m)#define\s+PY_VERSION\s+"?(?P<version>[0-9\.\-_a-zA-Z]+)"?`),
Package: "python",
PURL: mustPURL("pkg:generic/python@version"),
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:a:python_software_foundation:python:*:*:*:*:*:*:*:*"),
cpe.Must("cpe:2.3:a:python:python:*:*:*:*:*:*:*:*"),
},
},
{
Class: "go-binary",
@@ -103,6 +118,7 @@ var defaultClassifiers = []classifier{
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)BusyBox\s+v(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`),
Package: "busybox",
CPEs: singleCPE("cpe:2.3:a:busybox:busybox:*:*:*:*:*:*:*:*"),
},
{
Class: "php-cli-binary",
@@ -149,4 +165,26 @@ var defaultClassifiers = []classifier{
Package: "memcached",
PURL: mustPURL("pkg:generic/memcached@version"),
},
{
Class: "traefik-binary",
FileGlob: "**/traefik",
EvidenceMatcher: fileContentsVersionMatcher(
// [NUL]v1.7.34[NUL]
// [NUL]2.9.6[NUL]
`(?m)\x00v?(?P<version>[0-9]+\.[0-9]+\.[0-9]+(-alpha[0-9]|-beta[0-9]|-rc[0-9])?)\x00`),
Package: "traefik",
PURL: mustPURL("pkg:generic/traefik@version"),
},
{
Class: "postgresql-binary",
FileGlob: "**/postgres",
EvidenceMatcher: fileContentsVersionMatcher(
// [NUL]PostgreSQL 15beta4
// [NUL]PostgreSQL 15.1
// [NUL]PostgreSQL 9.6.24
// ?PostgreSQL 9.5alpha1
`(?m)(\x00|\?)PostgreSQL (?P<version>[0-9]+(\.[0-9]+)?(\.[0-9]+)?(alpha[0-9]|beta[0-9]|rc[0-9])?)`),
Package: "postgresql",
PURL: mustPURL("pkg:generic/postgresql@version"),
},
}

Some files were not shown because too many files have changed in this diff Show More