Compare commits

...
96 Commits
Author SHA1 Message Date
Christopher Angelo Phillips 23a3173c9f feat: add catalog delete (#1377) 2022-12-12 12:55:12 -05:00
Keith Zantow 17aa8287e6 docs: remove file classifier (#1397) 2022-12-08 16:50:29 +00:00
Christopher Angelo Phillips 730d3e3187 chore: update latest cyclonedx library (#1390) 2022-12-08 11:36:08 -05:00
Keith Zantow 997fbdfcf3 feat: Add Java binary catalogers (#1392) 2022-12-08 10:50:28 -05:00
Marc-Etienne Vargenau 13ceed9336 chore: Update SPDX license list to 3.19 (#1389) 2022-12-08 10:29:27 -05:00
Chapman Pendery 668f102340 fix: add manual vendor/product removal to fix false flags (#1070)
Closes https://github.com/anchore/syft/issues/1066
Closes https://github.com/anchore/grype/issues/800
Closes https://github.com/anchore/grype/issues/491
2022-12-08 09:57:42 -05:00
anchore-actions-token-generator[bot] f1a124209a Update Stereoscope to c5ff155d72f166e2332e160a75c3ff2b8e9c7e2e (#1395)
Signed-off-by: GitHub <noreply@github.com>
2022-12-08 08:32:49 +00:00
Keith Zantow 5dbb3fc41d chore: fix test busybox image sha (#1393) 2022-12-07 20:15:39 -05:00
Keith Zantow 614ea00905 fix: go version not properly identified in binary (#1384) 2022-12-02 13:24:36 -05:00
anchore-actions-token-generator[bot]andwagoodman 247b054ab5 Update Stereoscope to 3b80d983223f6e6fc2d33b0ffa003d30268418e9 (#1376)
Signed-off-by: GitHub <noreply@github.com>

Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: wagoodman <wagoodman@users.noreply.github.com>
2022-11-30 16:11:57 +00:00
Keith Zantow 9e43725951 fix: Update node binary package name (#1375) 2022-11-30 10:30:57 -05:00
Keith Zantow 4f39287216 feat: Generic Binary Cataloger (#1336) 2022-11-29 18:28:10 -05:00
Alex Goodman 7a69e2129b recover from bad parsing of golang binary (#1371)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-29 15:56:46 +00:00
Dan Luhring f6996f7b9a Fix parsing of apk databases with large entries (#1365)
Closes https://github.com/anchore/syft/issues/1354
2022-11-29 10:16:36 -05:00
anchore-actions-token-generator[bot]andspiffcs bd523bdb5d Update syft bootstrap tools to latest versions. (#1369)
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2022-11-29 10:13:00 -05:00
Christopher Angelo Phillips 0cbd0cc703 fix: guard for locations < 1 in alpmdb parse (#1366) 2022-11-28 15:43:18 +00:00
Christopher Angelo Phillips b290a445ca fix: remove cabal.project.freeze panic on last pkg (#1363) 2022-11-23 22:33:18 +00:00
Christopher Angelo Phillips bcfe38c009 fix: requirements.txt - return unicode only letter/num for version (#1361) 2022-11-22 10:43:05 -05:00
anchore-actions-token-generator[bot]andspiffcs 74967a28ea Update syft bootstrap tools to latest versions. (#1356)
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2022-11-21 09:57:49 -05:00
Keith Zantow 098e61dcc8 fix: sort relationships in SPDX output (#1350) 2022-11-21 09:26:24 -05:00
Keith Zantow 0dddf51fd5 chore: add debug logging for decode errors (#1352) 2022-11-21 09:26:11 -05:00
mikcl 04880c06ce feat(npm): handle aliases in package-lock.json (#1349) 2022-11-20 11:32:10 -05:00
Christopher Angelo Phillips da4b2df576 fix: spdx java checksum correctness (#1348) 2022-11-18 13:42:55 -05:00
Rob Cresswell 9d8244bae6 feat: Add support for npm lockfile version 3 (#1206)
This PR adds support for npm lockfile version 3, which drops the
"dependencies" key and uses "packages" instead. I've refactored the
lockfile parser to make the distinction between the versions explicit
rather than the implicit behaviour before. It _might_ be worth splitting
into separate files at some point, but the logic is so minimal that I
haven't done it.

Fixes #1203
Signed-off-by: Rob Cresswell <robcresswell@users.noreply.github.com>
2022-11-18 12:41:31 -05:00
Christopher Angelo Phillips 67888ee855 1111 clean name bug (#1347) 2022-11-18 14:45:18 +00:00
Alex GoodmanandKeith Zantow 9afc9231c0 Add spdx relationship encoding for dependencies (#1342)
Co-authored-by: Keith Zantow <kzantow@gmail.com>
2022-11-18 09:40:10 -05:00
Keith Zantow 42cb0a47a4 feat: SPDX 2.3 support (#1311) 2022-11-18 08:54:39 -05:00
0c4b99c1c2 SBOM cataloger (#1029)
* SBOM cataloger

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* sbom-cataloger: turn off by default

and add integration test

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* SBOM cataloger

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* SBOM cataloger (optimize)

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* SBOM cataloger (fix)

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* SBOM cataloger (fix imports #1172)

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* SBOM cataloger (fix: support group attribute in CDX SBOMs)

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>

* port to generic cataloger and add relationship to original file

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* generalize parser for all format globs

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Patrik Beno <patrik.beno@greenhorn.sk>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Tom Fay <tomfay@microsoft.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-16 14:11:45 -05:00
Christopher Angelo Phillips 0774ad15e2 chore: clean up linting configuration (#1343) 2022-11-16 16:28:09 +00:00
Keith Zantow f8be64d312 fix: Unmarshal Syft JSON with missing metadata (#1338) 2022-11-15 13:00:10 -05:00
Alex Goodman 1ae577a035 fix apk decode for older data shapes (#1341)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-15 16:38:31 +00:00
Weston Steimel 09bf5b062c chore: add unit test for wolfi os release identification (#1340)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2022-11-14 18:57:31 -05:00
Keith Zantow c1fdfce5f1 fix: Output only valid CPEs for CycloneDX OS components (#1339) 2022-11-14 15:24:19 -05:00
Justin Chadwell 10f43d75e0 feat: Add --name option to override name in output (#1269) 2022-11-10 14:03:23 -05:00
949cff158d Add support for dependency relationships for alpine (apk) (#1063)
* Fix type of pull deps and add support for provides

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* [wip] apk dependency lookup

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update whitespace for linter

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

* adjust test conditions

Signed-off-by: Timothy Gerla <tim@gerla.net>

* fix TODOs and improve Provides parser

* run simports after main merge

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

* add tests to cover apk relationship parsing cases

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* generate JSON schema for breaking changes to apk metadata

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update tests to account for additional dependencies

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] fix relationship encoding for cyclonedx

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify package relationships that can be expressed

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: Timothy Gerla <tim@gerla.net>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: Timothy Gerla <tim@gerla.net>
2022-11-09 15:43:37 +00:00
Alex Goodman e58d0aecb8 normalize alpm md5 refs (#1333)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-09 15:04:15 +00:00
Alex Goodman d7a51a69dd Update java generic cataloger (#1329)
* remove centralize pURL generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* port java cataloger to new generic cataloger pattern

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove common.GenericCataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update format test fixtures to reflect ID updates

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix package sort instability for encode-decode-encode cycles

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-09 14:55:54 +00:00
Keith Zantow f3528132a7 Support encoding map types to CycloneDX properties (#1332) 2022-11-08 18:59:37 -05:00
Alex Goodman 5ed002e1a9 Update swift cataloger to generic cataloger (#1324)
* port swift cataloger to new generic cataloger pattern

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add cocopods metadata to json schema defs

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update json test fixture with latest schema version

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 13:51:59 -04:00
Alex Goodman f319713821 port rust cataloger to new generic cataloger pattern (#1323)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 16:07:36 +00:00
Alex Goodman 41464bbd7f port ruby cataloger to new generic cataloger pattern (#1322)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 11:33:55 -04:00
Alex Goodman 3048382bbd port rpm cataloger to new generic cataloger pattern (#1321)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 10:41:04 -04:00
Alex Goodman 1076281566 port python cataloger to new generic cataloger pattern (#1319)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 08:59:03 -04:00
Alex Goodman 2deb96a801 Update portage cataloger to new generic cataloger (#1316)
* port portage (ha) cataloger to new generic cataloger pattern

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update JSON schema to account for removing portage fields

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-03 14:49:18 -04:00
Alex Goodman 891f2c576b port php cataloger to new generic cataloger pattern (#1315)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-03 13:00:44 -04:00
Kenny MoensandKeith Zantow bc9740d50a javascript cataloger: node binary: nil pointer dereference (#1313)
Co-authored-by: Keith Zantow <kzantow@gmail.com>
2022-11-03 14:38:49 +00:00
Keith Zantow 3e99c4d7d8 Fix: Include version information in binary cataloger CPEs (#1310) 2022-11-03 10:17:15 -04:00
Christopher Angelo Phillips 10464642e9 fix: only generate PURL on empty string (#1312) 2022-11-03 10:00:14 -04:00
Alex Goodman e0acfa98c7 add s3 credentials to release (#1309)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-02 15:48:37 +00:00
Alex Goodman 9634b42746 port javascript cataloger to new generic cataloger pattern (#1308)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-02 11:31:57 -04:00
Keith Zantow 35f0f2931e chore: update goreleaser brew token (#1306) 2022-11-02 10:05:20 -04:00
Keith Zantow 95c7378109 fix: Decode binary and unknown metadata (#1307) 2022-11-01 21:26:00 +00:00
Keith Zantow ba57f3db51 chore: update github token permissions for goreleaser (#1305) 2022-11-01 16:28:37 +00:00
Christopher Angelo Phillips 4c5c6f6319 fix: update ci secret to use new password (#1304) 2022-11-01 14:30:29 +00:00
Christopher Angelo Phillips 1b69fbd566 fix: update secret value to use new cert cahin (#1303)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-11-01 00:05:40 +00:00
Christopher Angelo Phillips 7ed91c0e31 fix: verbose quill release failures (#1302)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-10-31 22:21:30 +00:00
Christopher Angelo Phillips 8e2fc29bc7 fix: unterminated quoted string (#1300)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-10-31 20:07:43 +00:00
Christopher Angelo Phillips e9fbc38064 fix: update Makefile to remove old signing arch (#1299) 2022-10-31 18:56:30 +00:00
Christopher Angelo Phillips edeba9c01c feat: add nodejs-binary package classifier (#1296) 2022-10-31 16:45:11 +00:00
Weston Steimel 919c929798 update go-rpmdb to improve parsing of installed files (#1297) 2022-10-30 23:55:17 -04:00
Nick Piper 91f4467a1f docs: update attestation directions with new cosign changes 2022-10-29 20:18:53 +00:00
Keith Zantow c489c37e35 fix: Continue parsing Python RECORD files when bad lines encountered (#1295) 2022-10-29 15:45:59 -04:00
Marc-Etienne Vargenau dd89461ba3 Fix #1245 Update SPDX license list to 3.18 (#1259)
* Fix #1245 Update SPDX license list to 3.18

Signed-off-by: Marc-Etienne Vargenau <marc-etienne.vargenau@nokia.com>
2022-10-27 23:46:54 -04:00
Rob Tompkins fb388c0f25 fix: Resolve Maven POM expressions (#1251) (#1278) 2022-10-27 13:01:07 -04:00
Alex Goodman e52aa3bc03 port haskell cataloger to new generic cataloger pattern (#1290)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-27 11:30:22 -04:00
Alex Goodman 6826d7603b port golang cataloger to new generic cataloger pattern (#1289)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-25 16:42:50 +00:00
Alex Goodman 52cb7269bf port deb/dpkg cataloger to new generic cataloger pattern (#1288)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-25 15:47:32 +00:00
Alex Goodman bd5adbc9b3 update cataloger tests to use pkgtest utils (#1287)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-25 11:20:02 -04:00
Alex Goodman c7a653060d port dotnet cataloger to new generic cataloger pattern (#1286)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-24 21:17:27 +00:00
Alex Goodman fbdde6d4f4 port dart cataloger to new generic cataloger pattern (#1285)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-24 20:37:42 +00:00
Alex Goodman eb8ebd9ffc port conan cataloger to new generic cataloger pattern (#1284)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-24 20:11:20 +00:00
Alex Goodman f36c0ca971 port apk cataloger to new generic cataloger pattern (#1283)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-24 13:51:09 -04:00
Alex Goodman 28cadfdb5d replace signing tooling with quill (#1280)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-24 13:03:10 -04:00
Alex Goodman b44f441c82 Upgrade generic cataloger (#1281)
* add second generation of generic cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* upgrade aplm cataloger to use generic.Cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove pacakge found-by attribute from the definition of a package ID

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-24 11:12:12 -04:00
anchore-actions-token-generator[bot]andspiffcs 7a8b96abc2 Update syft bootstrap tools to latest versions. (#1282)
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2022-10-24 10:17:06 -04:00
Alex Goodman d8c659b65b replace logger interface with anchore/go-logger (#1279)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-21 15:12:14 +00:00
anchore-actions-token-generator[bot] 5568cc0dd5 Update syft bootstrap tools to latest versions. (#1267) 2022-10-21 09:42:13 -04:00
Keith Zantow 78a0af2e2d Add go binary h1 digest to SPDX (#1265) 2022-10-19 16:33:10 -04:00
Christopher Angelo Phillips 6e764815d0 fix: move reproduction to top of issue (#1264) 2022-10-19 12:51:38 -04:00
Christopher Angelo Phillips ec229b993c fix: update syftjson ID to match major schema version (#1274) 2022-10-19 16:08:38 +00:00
Arnaud J Le HorsandChristopher Phillips d3ee24017e Use in-toto CycloneDX predicate to be compatible with cosign (#1270)
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-10-19 09:37:52 -04:00
Marc-Etienne Vargenau e2d06cecb7 chore: handle deprecated SPDX license: StandardML-NJ (#1266) 2022-10-17 13:45:36 -04:00
Marc-Etienne Vargenau 41bc6bb410 Fixes #1179 Deprecated SPDX license (#1263) 2022-10-14 15:54:57 -04:00
Christopher Angelo Phillips 89575199b8 feat: add RelationshipsBySourceOwnership to syft json output (#1248) 2022-10-11 15:11:03 -04:00
Christopher Angelo Phillips fa0b3c0438 fix: reset merged package into map; (#1258) 2022-10-11 18:35:46 +00:00
Keith Zantow 780e1c310c refactor: Remove experimental Anchore Enterprise upload functionality (#1257) 2022-10-10 16:16:47 -04:00
anchore-actions-token-generator[bot] d89e320dcd Update syft bootstrap tools to latest versions. (#1254) 2022-10-07 13:54:42 -04:00
anchore-actions-token-generator[bot]andwestonsteimel 71187c6416 Update Stereoscope to d24c9d626b33fa720210b007a20767801827b532 (#1253)
Co-authored-by: westonsteimel <westonsteimel@users.noreply.github.com>
2022-10-06 22:04:49 -04:00
anchore-actions-token-generator[bot]andspiffcs 58504d5bf3 Update syft bootstrap tools to latest versions. (#1244)
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2022-10-06 08:38:36 +00:00
Alex Goodman 48f0a4680f fix apkdb checksum representation (#1247)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-05 16:29:05 -04:00
Christopher Angelo Phillips 7d2fe9d95e feat: add identifiable field to source object (#1243) 2022-10-05 14:01:40 -04:00
Adam Hughes 40d294a89e feat: attest support for Singularity images (#1201) 2022-10-04 12:34:59 -04:00
anchore-actions-token-generator[bot]andspiffcs 91eece47ff Update syft bootstrap tools to latest versions. (#1239)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2022-10-04 10:18:21 +01:00
anchore-actions-token-generator[bot]andkzantow 1fa4bab7a7 Update Stereoscope to 1b1b744a919964f38d14e1416fb3f25221b761ce (#1240)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: kzantow <kzantow@users.noreply.github.com>
2022-10-04 10:17:29 +01:00
Keith Zantow 4d2f18218c fix: Follow symlinks when searching for globs in all-layers scope (#1221) 2022-09-30 13:01:08 -04:00
Christopher Angelo Phillips dd7c654ed0 update requires to use list; remove field (#1234) 2022-09-30 11:57:13 -04:00
441 changed files with 202102 additions and 17314 deletions
+2 -2
View File
@@ -7,12 +7,12 @@ assignees: ''
---
**Please provide a set of steps on how to reproduce the issue**
**What happened**:
**What you expected to happen**:
**How to reproduce it (as minimally and precisely as possible)**:
**Anything else we need to know?**:
**Environment**:
-3
View File
@@ -1,3 +0,0 @@
dev-pki
log
signing-identity.txt
-11
View File
@@ -1,11 +0,0 @@
#!/usr/bin/env bash
set -eu
# grab utilities
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
. "$SCRIPT_DIR"/utils.sh
# cleanup any dev certs left behind
. "$SCRIPT_DIR"/setup-import-cert.sh # defines KEYCHAIN_NAME and KEYCHAIN_PATH
. "$SCRIPT_DIR"/setup-dev.sh
cleanup_dev_signing
-53
View File
@@ -1,53 +0,0 @@
#!/usr/bin/env bash
set +xu
if [ -z "$AC_USERNAME" ]; then
exit_with_error "AC_USERNAME not set"
fi
if [ -z "$AC_PASSWORD" ]; then
exit_with_error "AC_PASSWORD not set"
fi
set -u
# notarize [archive-path]
#
notarize() {
binary_path=$1
archive_path=${binary_path}-archive-for-notarization.zip
title "archiving release binary into ${archive_path}"
parent=$(dirname "$binary_path")
(
cd "${parent}" && zip "${archive_path}" "$(basename ${binary_path})"
)
if [ ! -f "$archive_path" ]; then
exit_with_error "cannot find payload for notarization: $archive_path"
fi
# install gon
which gon || (go install github.com/mitchellh/gon/cmd/gon@latest)
# create config (note: json via stdin with gon is broken, can only use HCL from file)
hcl_file=$(mktemp).hcl
cat <<EOF > "$hcl_file"
notarize {
path = "$archive_path"
bundle_id = "com.anchore.toolbox.syft"
}
apple_id {
username = "$AC_USERNAME"
password = "@env:AC_PASSWORD"
}
EOF
gon -log-level info "$hcl_file"
rm "${hcl_file}" "${archive_path}"
}
-128
View File
@@ -1,128 +0,0 @@
#!/usr/bin/env bash
set -eu
NAME=syft-dev
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
DIR=$SCRIPT_DIR/dev-pki
FILE_PREFIX=$DIR/$NAME
IDENTITY=${NAME}-id-415d8c69793
## OpenSSL material
KEY_PASSWORD="letthedevin"
P12_PASSWORD="popeofnope"
KEY_FILE=$FILE_PREFIX-key.pem
CSR_FILE=$FILE_PREFIX-csr.pem
CERT_FILE=$FILE_PREFIX-cert.pem
EXT_FILE=$FILE_PREFIX-ext.cnf
P12_FILE=$FILE_PREFIX.p12
EXT_SECTION=codesign_reqext
# setup_signing
#
# preps the MAC_SIGNING_IDENTITY env var for use in the signing process, using ephemeral developer certificate material
#
function setup_signing() {
# check to see if this has already been done... if so, bail!
set +ue
if security find-identity -p codesigning "$KEYCHAIN_PATH" | grep $IDENTITY ; then
export MAC_SIGNING_IDENTITY=$IDENTITY
commentary "skipping creating dev certificate material (already exists)"
commentary "setting MAC_SIGNING_IDENTITY=${IDENTITY}"
return 0
fi
set -ue
title "setting up developer certificate material"
mkdir -p "${DIR}"
# configure the openssl extensions
cat << EOF > "$EXT_FILE"
[ req ]
default_bits = 2048 # RSA key size
encrypt_key = yes # Protect private key
default_md = sha256 # MD to use
utf8 = yes # Input is UTF-8
string_mask = utf8only # Emit UTF-8 strings
prompt = yes # Prompt for DN
distinguished_name = codesign_dn # DN template
req_extensions = $EXT_SECTION # Desired extensions
[ codesign_dn ]
commonName = $IDENTITY
commonName_max = 64
[ $EXT_SECTION ]
keyUsage = critical,digitalSignature
extendedKeyUsage = critical,codeSigning
subjectKeyIdentifier = hash
EOF
title "create the private key"
openssl genrsa \
-des3 \
-out "$KEY_FILE" \
-passout "pass:$KEY_PASSWORD" \
2048
title "create the csr"
openssl req \
-new \
-key "$KEY_FILE" \
-out "$CSR_FILE" \
-passin "pass:$KEY_PASSWORD" \
-config "$EXT_FILE" \
-subj "/CN=$IDENTITY"
commentary "verify the csr: we should see X509 v3 extensions for codesigning in the CSR"
openssl req -in "$CSR_FILE" -noout -text | grep -A1 "X509v3" || exit_with_error "could not find x509 extensions in CSR"
title "create the certificate"
# note: Extensions in certificates are not transferred to certificate requests and vice versa. This means that
# just because the CSR has x509 v3 extensions doesn't mean that you'll see these extensions in the cert output.
# To prove this do:
# openssl x509 -text -noout -in server.crt | grep -A10 "X509v3 extensions:"
# ... and you will see no output (if -extensions is not used). (see https://www.openssl.org/docs/man1.1.0/man1/x509.html#BUGS)
# To get the extensions, use "-extensions codesign_reqext" when creating the cert. The codesign_reqext value matches
# the section name in the ext file used in CSR / cert creation (-extfile and -config).
openssl x509 \
-req \
-days 10000 \
-in "$CSR_FILE" \
-signkey "$KEY_FILE" \
-out "$CERT_FILE" \
-extfile "$EXT_FILE" \
-passin "pass:$KEY_PASSWORD" \
-extensions $EXT_SECTION
commentary "verify the certificate: we should see our extensions"
openssl x509 -text -noout -in "$CERT_FILE" | grep -A1 'X509v3' || exit_with_error "could not find x509 extensions in certificate"
title "export cert and private key to .p12 file"
# note: this step may be entirely optional, however, I found it useful to follow the prod path which goes the route of using a p12
openssl pkcs12 \
-export \
-out "$P12_FILE" \
-inkey "$KEY_FILE" \
-in "$CERT_FILE" \
-passin "pass:$KEY_PASSWORD" \
-passout "pass:$P12_PASSWORD"
# delete the keychain if it already exists
if [ -f "${KEYCHAIN_PATH}" ]; then
cleanup_dev_signing
fi
import_signing_certificate "$P12_FILE" "$P12_PASSWORD" "$IDENTITY"
}
function cleanup_dev_signing() {
title "delete the dev keychain and all certificate material"
set -xue
security delete-keychain "$KEYCHAIN_NAME" || true
rm -f "$KEYCHAIN_PATH" || true
rm -rf "${DIR}" || true
}
@@ -1,55 +0,0 @@
#!/usr/bin/env bash
set -eu
KEYCHAIN_NAME="syft-ephemeral-keychain"
KEYCHAIN_PATH="$HOME/Library/Keychains/${KEYCHAIN_NAME}-db"
# import_signing_certificate
#
# imports a cert from a p12 file into a keychain used for codesigning
#
function import_signing_certificate() {
p12_file=$1
p12_password=$2
identity=$3
keychain_password="$(openssl rand -base64 100)"
title "create the a new keychain"
security create-keychain -p "$keychain_password" "$KEYCHAIN_NAME"
security set-keychain-settings -lut 21600 "$KEYCHAIN_PATH"
security unlock-keychain -p "$keychain_password" "$KEYCHAIN_PATH"
if [ ! -f "$KEYCHAIN_PATH" ]; then
exit_with_error "cannot find keychain '$KEYCHAIN_PATH'"
fi
set +e
if ! security verify-cert -k "$KEYCHAIN_PATH" -c "$p12_file" &> /dev/null; then
set -e
title "import the cert into the new keychain if it is not already trusted by the system"
# '-t cert' is vital since it side-steps the need for user interaction with "security add-trusted-cert" (which has wider security implications)
security import "$p12_file" -P "$p12_password" -t cert -f pkcs12 -k "$KEYCHAIN_PATH" -T /usr/bin/codesign
# note: set the partition list for this certificate's private key to include "apple-tool:" and "apple:" allows the codesign command to access this keychain item without an interactive user prompt.
security set-key-partition-list -S "apple-tool:,apple:,codesign:" -s -k "$keychain_password" "$KEYCHAIN_PATH"
else
set -e
commentary "...cert has already been imported onto the new keychain"
fi
commentary "make certain there are identities that can be used for code signing"
security find-identity -p codesigning "$KEYCHAIN_PATH" | grep -C 30 "$identity" || exit_with_error "could not find identity that can be used with codesign"
title "add the new keychain to the search path for codesign"
add_keychain "$KEYCHAIN_NAME"
commentary "verify the new keychain can be found by the security sub-system"
security list-keychains | grep "$KEYCHAIN_NAME" || exit_with_error "could not find new keychain"
export MAC_SIGNING_IDENTITY=$identity
commentary "setting MAC_SIGNING_IDENTITY=${identity}"
}
@@ -1,32 +0,0 @@
#!/usr/bin/env bash
set -eu
assert_in_ci
IDENTITY="Developer ID Application: ANCHORE, INC. (9MJHKYX5AT)"
set +xu
if [ -z "$APPLE_DEVELOPER_ID_CERT" ]; then
exit_with_error "APPLE_DEVELOPER_ID_CERT not set"
fi
if [ -z "$APPLE_DEVELOPER_ID_CERT_PASS" ]; then
exit_with_error "APPLE_DEVELOPER_ID_CERT_PASS not set"
fi
# setup_signing
#
# preps the MAC_SIGNING_IDENTITY env var for use in the signing process, using production certificate material
#
setup_signing() {
title "setting up production certificate material"
# Write signing certificate to disk from environment variable.
p12_file="$HOME/developer_id_certificate.p12"
echo -n "$APPLE_DEVELOPER_ID_CERT" | base64 --decode > "$p12_file"
import_signing_certificate "$p12_file" "$APPLE_DEVELOPER_ID_CERT_PASS" "$IDENTITY"
# Make this new keychain the user's default keychain, so that codesign will be able to find this certificate when we specify it during signing.
security default-keychain -d "user" -s "${KEYCHAIN_PATH}"
}
-48
View File
@@ -1,48 +0,0 @@
#!/usr/bin/env bash
set -eu -o pipefail
IS_SNAPSHOT="$1"
## grab utilities
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
. "$SCRIPT_DIR"/utils.sh
mkdir -p "$SCRIPT_DIR/log"
main() {
# defines KEYCHAIN_NAME and KEYCHAIN_PATH
. "$SCRIPT_DIR"/setup-import-cert.sh
case "$IS_SNAPSHOT" in
"1" | "true" | "yes")
commentary "assuming development setup..."
. "$SCRIPT_DIR"/setup-dev.sh
;;
"0" | "false" | "no")
commentary "assuming production setup..."
. "$SCRIPT_DIR"/setup-prod.sh
;;
*)
exit_with_error "could not determine if this was a production build (isSnapshot='$IS_SNAPSHOT')"
;;
esac
# load up all signing material into a keychain (note: this should set the MAC_SIGNING_IDENTITY env var)
setup_signing
# write out identity to a file
echo -n "$MAC_SIGNING_IDENTITY" > "$SCRIPT_DIR/$SIGNING_IDENTITY_FILENAME"
}
# capture all output from a subshell to log output additionally to a file (as well as the terminal)
( (
set +u
if [ -n "$SKIP_SIGNING" ]; then
commentary "skipping signing setup..."
else
set -u
main
fi
) 2>&1) | tee "$SCRIPT_DIR/log/setup.txt"
-98
View File
@@ -1,98 +0,0 @@
#!/usr/bin/env bash
set -eu -o pipefail
BINARY_PATH="$1"
IS_SNAPSHOT="$2"
TARGET_NAME="$3"
## grab utilities
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
. "$SCRIPT_DIR"/utils.sh
mkdir -p "$SCRIPT_DIR/log"
# sign_binary [binary-path] [signing-identity]
#
# signs a single binary with cosign
#
sign_binary() {
exe_path=$1
identity=$2
if [ -x "$exe_path" ] && file -b "$exe_path" | grep -q "Mach-O"
then
echo "signing $exe_path ..."
else
echo "skip signing $exe_path ..."
return 0
fi
set -x
codesign \
-s "$identity" \
-f \
--verbose=4 \
--timestamp \
--options runtime \
"$exe_path"
if [ $? -ne 0 ]; then
exit_with_error "signing failed"
fi
codesign --verify "$exe_path" --verbose=4
set +x
}
main() {
binary_abs_path=$(realpath "$BINARY_PATH")
if [ ! -f "$binary_abs_path" ]; then
echo "archive does not exist: $binary_abs_path"
fi
case "$IS_SNAPSHOT" in
"1" | "true" | "yes")
commentary "disabling notarization..."
perform_notarization=false
;;
"0" | "false" | "no")
commentary "enabling notarization..."
. "$SCRIPT_DIR"/notarize.sh
perform_notarization=true
;;
*)
exit_with_error "could not determine if this was a production build (isSnapshot='$IS_SNAPSHOT')"
;;
esac
# grab the signing identity from the local temp file (setup by setup.sh)
MAC_SIGNING_IDENTITY=$(cat "$SCRIPT_DIR/$SIGNING_IDENTITY_FILENAME")
# sign all of the binaries in the archive and recreate the input archive with the signed binaries
sign_binary "$binary_abs_path" "$MAC_SIGNING_IDENTITY"
# send all of the binaries off to apple to bless
if $perform_notarization ; then
notarize "$binary_abs_path"
else
commentary "skipping notarization..."
fi
}
# capture all output from a subshell to log output additionally to a file (as well as the terminal)
( (
set +u
if [ -n "$SKIP_SIGNING" ]; then
commentary "skipping signing..."
else
set -u
main
fi
) 2>&1) | tee "$SCRIPT_DIR/log/signing-$(basename $BINARY_PATH)-$TARGET_NAME.txt"
-78
View File
@@ -1,78 +0,0 @@
SIGNING_IDENTITY_FILENAME=signing-identity.txt
## terminal goodies
PURPLE='\033[0;35m'
GREEN='\033[0;32m'
RED='\033[0;31m'
BOLD=$(tput -T linux bold)
RESET='\033[0m'
function success() {
echo -e "\n${GREEN}${BOLD}$@${RESET}"
}
function title() {
success "Task: $@"
}
function commentary() {
echo -e "\n${PURPLE}# $@${RESET}"
}
function error() {
echo -e "${RED}${BOLD}error: $@${RESET}"
}
function exit_with_error() {
error $@
exit 1
}
function exit_with_message() {
success $@
exit 0
}
function realpath {
echo "$(cd $(dirname $1); pwd)/$(basename $1)";
}
# this function adds all of the existing keychains plus the new one which is the same as going to Keychain Access
# and selecting "Add Keychain" to make the keychain visible under "Custom Keychains". This is done with
# "security list-keychains -s" for some reason. The downside is that this sets the search path, not appends
# to it, so you will loose existing keychains in the search path... which is truly terrible.
function add_keychain() {
keychains=$(security list-keychains -d user)
keychainNames=();
for keychain in $keychains
do
basename=$(basename "$keychain")
keychainName=${basename::${#basename}-4}
keychainNames+=("$keychainName")
done
echo "existing user keychains: ${keychainNames[@]}"
security -v list-keychains -s "${keychainNames[@]}" "$1"
}
function exit_not_ci() {
printf "WARNING! It looks like this isn't the CI environment. This script modifies the macOS Keychain setup in ways you probably wouldn't want for your own machine. It also requires an Apple Developer ID Certificate that you shouldn't have outside of the CI environment.\n\nExiting early to make sure nothing bad happens.\n"
exit 1
}
CI_HOME="/Users/runner"
function assert_in_ci() {
if [[ "${HOME}" != "${CI_HOME}" ]]; then
exit_not_ci
fi
set +u
if [ -z "${GITHUB_ACTIONS}" ]; then
exit_not_ci
fi
set -u
}
+27 -50
View File
@@ -14,7 +14,7 @@ env:
jobs:
quality-gate:
environment: release
runs-on: ubuntu-latest
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
@@ -92,9 +92,9 @@ jobs:
release:
needs: [quality-gate]
# due to our code signing process, it's vital that we run our release steps on macOS
runs-on: macos-latest
runs-on: ubuntu-20.04
permissions:
contents: write
packages: write
steps:
- uses: actions/setup-go@v2
@@ -125,18 +125,35 @@ jobs:
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.TOOLBOX_DOCKER_USER }}
password: ${{ secrets.TOOLBOX_DOCKER_PASS }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build & publish release artifacts
run: make release
env:
GITHUB_TOKEN: ${{ secrets.ANCHORE_GIT_READ_TOKEN }}
# for mac signing and notarization...
QUILL_SIGN_P12: ${{ secrets.ANCHORE_APPLE_DEVELOPER_ID_CERT_CHAIN }}
QUILL_SIGN_PASSWORD: ${{ secrets.ANCHORE_APPLE_DEVELOPER_ID_CERT_PASS }}
QUILL_NOTARY_ISSUER: ${{ secrets.APPLE_NOTARY_ISSUER }}
QUILL_NOTARY_KEY_ID: ${{ secrets.APPLE_NOTARY_KEY_ID }}
QUILL_NOTARY_KEY: ${{ secrets.APPLE_NOTARY_KEY }}
# for creating the release (requires write access to packages and content)
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# for updating brew formula in anchore/homebrew-syft
GITHUB_BREW_TOKEN: ${{ secrets.ANCHORE_GIT_READ_TOKEN }}
# for updating the VERSION file in S3...
AWS_ACCESS_KEY_ID: ${{ secrets.TOOLBOX_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.TOOLBOX_AWS_SECRET_ACCESS_KEY }}
# used during macOS code signing
APPLE_DEVELOPER_ID_CERT: ${{ secrets.APPLE_DEVELOPER_ID_CERT }}
APPLE_DEVELOPER_ID_CERT_PASS: ${{ secrets.APPLE_DEVELOPER_ID_CERT_PASS }}
# used during macOS notarization
AC_USERNAME: ${{ secrets.ENG_CI_APPLE_ID }}
AC_PASSWORD: ${{ secrets.ENG_CI_APPLE_ID_PASS }}
- uses: anchore/sbom-action@v0
continue-on-error: true
@@ -158,43 +175,3 @@ jobs:
name: artifacts
path: dist/**/*
release-docker-assets:
needs: [release]
# code signing requires we run on mac-os runners. docker does not come installed on the mac-os runner
# a previous release process installed and configured docker on the mac-os runner which lead to blocked releases
# the anchore tools team opted to break this step out to a separate process to remove this work constraint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.TOOLBOX_DOCKER_USER }}
password: ${{ secrets.TOOLBOX_DOCKER_PASS }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Restore go cache
id: go-cache
uses: actions/cache@v2.1.3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-
- name: (cache-miss) Bootstrap all project dependencies
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
- name: Build & Publish docker images
run: make release-docker-assets
+20 -3
View File
@@ -11,9 +11,9 @@ linters:
- asciicheck
- bodyclose
- depguard
- dogsled
- dupl
- errcheck
- errorlint
- exportloopref
- funlen
- gocognit
@@ -21,13 +21,13 @@ linters:
- gocritic
- gocyclo
- gofmt
- goprintffuncname
- tparallel
- importas
- gosec
- gosimple
- govet
- ineffassign
- misspell
- nakedret
- nolintlint
- revive
- staticcheck
@@ -37,8 +37,25 @@ linters:
- unparam
- unused
- whitespace
linters-settings:
funlen:
# Checks the number of lines in a function.
# If lower than 0, disable the check.
# Default: 60
lines: 70
# Checks the number of statements in a function.
# If lower than 0, disable the check.
# Default: 40
statements: 50
output:
uniq-by-line: false
run:
timeout: 10m
# do not enable...
# - dogsled # found to be to niche and ineffective
# - goprintffuncname # does not catch all cases and there are exceptions
# - nakedret # does not catch all cases and should not fail a build
# - gochecknoglobals
# - gochecknoinits # this is too aggressive
# - rowserrcheck disabled per generics https://github.com/golangci/golangci-lint/issues/2649
+164 -7
View File
@@ -2,9 +2,9 @@ release:
prerelease: auto
draft: false
before:
hooks:
- ./.github/scripts/apple-signing/setup.sh {{ .IsSnapshot }}
env:
# required to support multi architecture docker builds
- DOCKER_CLI_EXPERIMENTAL=enabled
builds:
- id: linux-build
@@ -43,10 +43,9 @@ builds:
ldflags: *build-ldflags
hooks:
post:
# we must have signing as a build hook instead of the signs section. The signs section must register a new
# asset, where we want to replace an existing asset. A post-build hook has the advantage of not needing to
# unpackage and repackage a tar.gz with a signed binary
- ./.github/scripts/apple-signing/sign.sh "{{ .Path }}" "{{ .IsSnapshot }}" "{{ .Target }}"
- cmd: .tmp/quill sign-and-notarize "{{ .Path }}" --dry-run={{ .IsSnapshot }} --ad-hoc={{ .IsSnapshot }} -vv
env:
- QUILL_LOG_FILE=/tmp/quill-{{ .Target }}.log
- id: windows-build
dir: ./cmd/syft
@@ -87,9 +86,167 @@ brews:
- tap:
owner: anchore
name: homebrew-syft
token: "{{.Env.GITHUB_BREW_TOKEN}}"
ids:
- darwin-archives
- linux-archives
homepage: *website
description: *description
license: "Apache License 2.0"
dockers:
- image_templates:
- anchore/syft:debug
- anchore/syft:{{.Tag}}-debug
- ghcr.io/anchore/syft:debug
- ghcr.io/anchore/syft:{{.Tag}}-debug
goarch: amd64
dockerfile: Dockerfile.debug
use: buildx
build_flag_templates:
- "--platform=linux/amd64"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:debug-arm64v8
- anchore/syft:{{.Tag}}-debug-arm64v8
- ghcr.io/anchore/syft:debug-arm64v8
- ghcr.io/anchore/syft:{{.Tag}}-debug-arm64v8
goarch: arm64
dockerfile: Dockerfile.debug
use: buildx
build_flag_templates:
- "--platform=linux/arm64/v8"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:debug-ppc64le
- anchore/syft:{{.Tag}}-debug-ppc64le
- ghcr.io/anchore/syft:debug-ppc64le
- ghcr.io/anchore/syft:{{.Tag}}-debug-ppc64le
goarch: ppc64le
dockerfile: Dockerfile.debug
use: buildx
build_flag_templates:
- "--platform=linux/ppc64le"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:debug-s390x
- anchore/syft:{{.Tag}}-debug-s390x
- ghcr.io/anchore/syft:debug-s390x
- ghcr.io/anchore/syft:{{.Tag}}-debug-s390x
goarch: s390x
dockerfile: Dockerfile.debug
use: buildx
build_flag_templates:
- "--platform=linux/s390x"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:latest
- anchore/syft:{{.Tag}}
- ghcr.io/anchore/syft:latest
- ghcr.io/anchore/syft:{{.Tag}}
goarch: amd64
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/amd64"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:{{.Tag}}-arm64v8
- ghcr.io/anchore/syft:{{.Tag}}-arm64v8
goarch: arm64
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/arm64/v8"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:{{.Tag}}-ppc64le
- ghcr.io/anchore/syft:{{.Tag}}-ppc64le
goarch: ppc64le
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/ppc64le"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:{{.Tag}}-s390x
- ghcr.io/anchore/syft:{{.Tag}}-s390x
goarch: s390x
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/s390x"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
docker_manifests:
- name_template: anchore/syft:latest
image_templates:
- anchore/syft:{{.Tag}}
- anchore/syft:{{.Tag}}-arm64v8
- anchore/syft:{{.Tag}}-ppc64le
- anchore/syft:{{.Tag}}-s390x
- name_template: anchore/syft:debug
- anchore/syft:{{.Tag}}-debug
- anchore/syft:{{.Tag}}-debug-arm64v8
- anchore/syft:{{.Tag}}-debug-ppc64le
- anchore/syft:{{.Tag}}-debug-s390x
- name_template: anchore/syft:{{.Tag}}
image_templates:
- anchore/syft:{{.Tag}}
- anchore/syft:{{.Tag}}-arm64v8
- anchore/syft:{{.Tag}}-ppc64le
- anchore/syft:{{.Tag}}-s390x
- name_template: ghcr.io/anchore/syft:latest
image_templates:
- ghcr.io/anchore/syft:{{.Tag}}
- ghcr.io/anchore/syft:{{.Tag}}-arm64v8
- ghcr.io/anchore/syft:{{.Tag}}-ppc64le
- ghcr.io/anchore/syft:{{.Tag}}-s390x
- name_template: ghcr.io/anchore/syft:debug
image_templates:
- ghcr.io/anchore/syft:{{.Tag}}-debug
- ghcr.io/anchore/syft:{{.Tag}}-debug-arm64v8
- ghcr.io/anchore/syft:{{.Tag}}-debug-ppc64le
- ghcr.io/anchore/syft:{{.Tag}}-debug-s390x
- name_template: ghcr.io/anchore/syft:{{.Tag}}
image_templates:
- ghcr.io/anchore/syft:{{.Tag}}
- ghcr.io/anchore/syft:{{.Tag}}-arm64v8
- ghcr.io/anchore/syft:{{.Tag}}-ppc64le
- ghcr.io/anchore/syft:{{.Tag}}-s390x
-193
View File
@@ -1,193 +0,0 @@
# Separate docker configuration to isolate docker dependency away from
# mac-os runner on github actions.
# See:
# https://github.com/anchore/syft/issues/577
# https://github.com/anchore/syft/issues/519
# https://github.com/anchore/syft/issues/576
release:
disable: true
env:
# required to support multi architecture docker builds
- DOCKER_CLI_EXPERIMENTAL=enabled
builds:
- id: linux-build
dir: ./cmd/syft
binary: syft
goos:
- linux
goarch:
- amd64
- arm64
- ppc64le
- s390x
# set the modified timestamp on the output binary to the git timestamp to ensure a reproducible build
mod_timestamp: &build-timestamp '{{ .CommitTimestamp }}'
env: &build-env
- CGO_ENABLED=0
ldflags: &build-ldflags |
-w
-s
-extldflags '-static'
-X github.com/anchore/syft/internal/version.version={{.Version}}
-X github.com/anchore/syft/internal/version.gitCommit={{.Commit}}
-X github.com/anchore/syft/internal/version.buildDate={{.Date}}
-X github.com/anchore/syft/internal/version.gitDescription={{.Summary}}
dockers:
- image_templates:
- anchore/syft:debug
- anchore/syft:{{.Tag}}-debug
- ghcr.io/anchore/syft:debug
- ghcr.io/anchore/syft:{{.Tag}}-debug
goarch: amd64
dockerfile: Dockerfile.debug
use: buildx
build_flag_templates:
- "--platform=linux/amd64"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:debug-arm64v8
- anchore/syft:{{.Tag}}-debug-arm64v8
- ghcr.io/anchore/syft:debug-arm64v8
- ghcr.io/anchore/syft:{{.Tag}}-debug-arm64v8
goarch: arm64
dockerfile: Dockerfile.debug
use: buildx
build_flag_templates:
- "--platform=linux/arm64/v8"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:debug-ppc64le
- anchore/syft:{{.Tag}}-debug-ppc64le
- ghcr.io/anchore/syft:debug-ppc64le
- ghcr.io/anchore/syft:{{.Tag}}-debug-ppc64le
goarch: ppc64le
dockerfile: Dockerfile.debug
use: buildx
build_flag_templates:
- "--platform=linux/ppc64le"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:debug-s390x
- anchore/syft:{{.Tag}}-debug-s390x
- ghcr.io/anchore/syft:debug-s390x
- ghcr.io/anchore/syft:{{.Tag}}-debug-s390x
goarch: s390x
dockerfile: Dockerfile.debug
use: buildx
build_flag_templates:
- "--platform=linux/s390x"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:latest
- anchore/syft:{{.Tag}}
- ghcr.io/anchore/syft:latest
- ghcr.io/anchore/syft:{{.Tag}}
goarch: amd64
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/amd64"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:{{.Tag}}-arm64v8
- ghcr.io/anchore/syft:{{.Tag}}-arm64v8
goarch: arm64
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/arm64/v8"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:{{.Tag}}-ppc64le
- ghcr.io/anchore/syft:{{.Tag}}-ppc64le
goarch: ppc64le
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/ppc64le"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
- image_templates:
- anchore/syft:{{.Tag}}-s390x
- ghcr.io/anchore/syft:{{.Tag}}-s390x
goarch: s390x
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/s390x"
- "--build-arg=BUILD_DATE={{.Date}}"
- "--build-arg=BUILD_VERSION={{.Version}}"
- "--build-arg=VCS_REF={{.FullCommit}}"
- "--build-arg=VCS_URL={{.GitURL}}"
docker_manifests:
- name_template: anchore/syft:latest
image_templates:
- anchore/syft:{{.Tag}}
- anchore/syft:{{.Tag}}-arm64v8
- anchore/syft:{{.Tag}}-ppc64le
- anchore/syft:{{.Tag}}-s390x
- name_template: anchore/syft:debug
- anchore/syft:{{.Tag}}-debug
- anchore/syft:{{.Tag}}-debug-arm64v8
- anchore/syft:{{.Tag}}-debug-ppc64le
- anchore/syft:{{.Tag}}-debug-s390x
- name_template: anchore/syft:{{.Tag}}
image_templates:
- anchore/syft:{{.Tag}}
- anchore/syft:{{.Tag}}-arm64v8
- anchore/syft:{{.Tag}}-ppc64le
- anchore/syft:{{.Tag}}-s390x
- name_template: ghcr.io/anchore/syft:latest
image_templates:
- ghcr.io/anchore/syft:{{.Tag}}
- ghcr.io/anchore/syft:{{.Tag}}-arm64v8
- ghcr.io/anchore/syft:{{.Tag}}-ppc64le
- ghcr.io/anchore/syft:{{.Tag}}-s390x
- name_template: ghcr.io/anchore/syft:debug
image_templates:
- ghcr.io/anchore/syft:{{.Tag}}-debug
- ghcr.io/anchore/syft:{{.Tag}}-debug-arm64v8
- ghcr.io/anchore/syft:{{.Tag}}-debug-ppc64le
- ghcr.io/anchore/syft:{{.Tag}}-debug-s390x
- name_template: ghcr.io/anchore/syft:{{.Tag}}
image_templates:
- ghcr.io/anchore/syft:{{.Tag}}
- ghcr.io/anchore/syft:{{.Tag}}-arm64v8
- ghcr.io/anchore/syft:{{.Tag}}-ppc64le
- ghcr.io/anchore/syft:{{.Tag}}-s390x
+13 -40
View File
@@ -3,17 +3,20 @@ VERSION=$(shell git describe --dirty --always --tags)
TEMPDIR = ./.tmp
# commands and versions
LINTCMD = $(TEMPDIR)/golangci-lint run --tests=false --timeout=5m --config .golangci.yaml
LINTCMD = $(TEMPDIR)/golangci-lint run --tests=false
GOIMPORTS_CMD = $(TEMPDIR)/gosimports -local github.com/anchore
RELEASE_CMD=$(TEMPDIR)/goreleaser release --rm-dist
SNAPSHOT_CMD=$(RELEASE_CMD) --skip-publish --snapshot
GOLANGCILINT_VERSION = v1.49.0
GOSIMPORTS_VERSION = v0.3.2
# tool versions
GOLANGCILINT_VERSION = v1.50.1
GOSIMPORTS_VERSION = v0.3.4
BOUNCER_VERSION = v0.4.0
CHRONICLE_VERSION = v0.4.1
GORELEASER_VERSION = v1.11.4
CHRONICLE_VERSION = v0.4.2
GORELEASER_VERSION = v1.13.1
YAJSV_VERSION = v1.4.1
COSIGN_VERSION = v1.12.1
COSIGN_VERSION = v1.13.1
QUILL_VERSION = v0.2.0
# formatting variables
BOLD := $(shell tput -T linux bold)
@@ -114,6 +117,7 @@ $(TEMPDIR):
.PHONY: bootstrap-tools
bootstrap-tools: $(TEMPDIR)
curl -sSfL https://raw.githubusercontent.com/anchore/quill/main/install.sh | sh -s -- -b $(TEMPDIR)/ $(QUILL_VERSION)
GO111MODULE=off GOBIN=$(realpath $(TEMPDIR)) go get -u golang.org/x/perf/cmd/benchstat
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(TEMPDIR)/ $(GOLANGCILINT_VERSION)
curl -sSfL https://raw.githubusercontent.com/wagoodman/go-bouncer/master/bouncer.sh | sh -s -- -b $(TEMPDIR)/ $(BOUNCER_VERSION)
@@ -275,25 +279,11 @@ snapshot-with-signing: ## Build snapshot release binaries and packages (with dum
echo "dist: $(SNAPSHOTDIR)" > $(TEMPDIR)/goreleaser.yaml
cat .goreleaser.yaml >> $(TEMPDIR)/goreleaser.yaml
rm -f .github/scripts/apple-signing/log/*.txt
# build release snapshots
bash -c "$(SNAPSHOT_CMD) --config $(TEMPDIR)/goreleaser.yaml || (cat .github/scripts/apple-signing/log/*.txt && false)"
# remove the keychain with the trusted self-signed cert automatically
.github/scripts/apple-signing/cleanup.sh
snapshot-docker-assets: # Build snapshot images of docker images that will be published on release
$(call title,Building snapshot docker release assets)
# create a config with the dist dir overridden
echo "dist: $(DISTDIR)" > $(TEMPDIR)/goreleaser.yaml
cat .goreleaser_docker.yaml >> $(TEMPDIR)/goreleaser.yaml
bash -c "\
$(SNAPSHOT_CMD) \
--config $(TEMPDIR)/goreleaser.yaml \
--parallelism 1"
|| (cat /tmp/quill-*.log && false)"
# note: we cannot clean the snapshot directory since the pipeline builds the snapshot separately
.PHONY: compare-mac
@@ -358,35 +348,18 @@ release: clean-dist CHANGELOG.md
echo "dist: $(DISTDIR)" > $(TEMPDIR)/goreleaser.yaml
cat .goreleaser.yaml >> $(TEMPDIR)/goreleaser.yaml
rm -f .github/scripts/apple-signing/log/*.txt
# note: notarization cannot be done in parallel, thus --parallelism 1
bash -c "\
$(RELEASE_CMD) \
--config $(TEMPDIR)/goreleaser.yaml \
--parallelism 1 \
--release-notes <(cat CHANGELOG.md)\
|| (cat .github/scripts/apple-signing/log/*.txt && false)"
cat .github/scripts/apple-signing/log/*.txt
--release-notes <(cat CHANGELOG.md) \
|| (cat /tmp/quill-*.log && false)"
# TODO: turn this into a post-release hook
# upload the version file that supports the application version update check (excluding pre-releases)
.github/scripts/update-version-file.sh "$(DISTDIR)" "$(VERSION)"
.PHONY: release-docker-assets
release-docker-assets:
$(call title,Publishing docker release assets)
# create a config with the dist dir overridden
echo "dist: $(DISTDIR)" > $(TEMPDIR)/goreleaser.yaml
cat .goreleaser_docker.yaml >> $(TEMPDIR)/goreleaser.yaml
bash -c "\
$(RELEASE_CMD) \
--config $(TEMPDIR)/goreleaser.yaml \
--parallelism 1"
.PHONY: clean
clean: clean-dist clean-snapshot clean-test-image-cache ## Remove previous builds, result reports, and test cache
$(call safe_rm_rf_children,$(RESULTSDIR))
+9 -35
View File
@@ -489,17 +489,6 @@ package:
# same as -s ; SYFT_PACKAGE_CATALOGER_SCOPE env var
scope: "squashed"
# cataloging file classifications is exposed through the power-user subcommand
file-classification:
cataloger:
# enable/disable cataloging of file classifications
# SYFT_FILE_CLASSIFICATION_CATALOGER_ENABLED env var
enabled: true
# the search space to look for file classifications (options: all-layers, squashed)
# SYFT_FILE_CLASSIFICATION_CATALOGER_SCOPE env var
scope: "squashed"
# cataloging file contents is exposed through the power-user subcommand
file-contents:
cataloger:
@@ -610,28 +599,6 @@ log:
# location to write the log file (default is not to have a log file)
# same as SYFT_LOG_FILE env var
file: ""
# uploading package SBOM is exposed through the packages subcommand
anchore:
# (feature-preview) the Anchore Enterprise Host or URL to upload results to (supported on Enterprise 3.0+)
# same as -H ; SYFT_ANCHORE_HOST env var
host: ""
# (feature-preview) the path after the host to the Anchore External API (supported on Enterprise 3.0+)
# same as SYFT_ANCHORE_PATH env var
path: ""
# (feature-preview) the username to authenticate against Anchore Enterprise (supported on Enterprise 3.0+)
# same as -u ; SYFT_ANCHORE_USERNAME env var
username: ""
# (feature-preview) the password to authenticate against Anchore Enterprise (supported on Enterprise 3.0+)
# same as -p ; SYFT_ANCHORE_PASSWORD env var
password: ""
# (feature-preview) path to dockerfile to be uploaded with the syft results to Anchore Enterprise (supported on Enterprise 3.0+)
# same as -d ; SYFT_ANCHORE_DOCKERFILE env var
dockerfile: ""
```
### Adding an SBOM to an image as an attestation using Syft
@@ -645,14 +612,14 @@ Note for the following example replace `docker.io/image:latest` with an image yo
its remote reference. Replace `$MY_PRIVATE_KEY` with a private key you own or have generated with cosign.
```bash
syft attest --key $MY_PRIVATE_KEY docker.io/image:latest > image_latest_sbom_attestation.json
syft attest --key $MY_PRIVATE_KEY -o spdx-json docker.io/image:latest > image_latest_sbom_attestation.json
cosign attach attestation --attestation image_latest_sbom_attestation.json docker.io/image:latest
```
Verify the new attestation exists on your image.
```bash
cosign verify-attestation -key $MY_PUBLIC_KEY docker.io/image:latest | jq '.payload | @base64d | .payload | fromjson | .predicate'
cosign verify-attestation --key $MY_PUBLIC_KEY --type spdxjson docker.io/image:latest | jq '.payload | @base64d | .payload | fromjson | .predicate'
```
You should see this output along with the attached SBOM:
@@ -666,3 +633,10 @@ The following checks were performed on each of these signatures:
```
Consumers of your image can now trust that the SBOM associated with your image is correct and from a trusted source.
The SBOM can be piped to Grype:
```bash
cosign verify-attestation --key $MY_PUBLIC_KEY --type spdxjson docker.io/image:latest | jq '.payload | @base64d | .payload | fromjson | .predicate' | grype
```
+1 -1
View File
@@ -39,7 +39,7 @@ func Attest(v *viper.Viper, app *config.Application, ro *options.RootOptions) *c
// run to unmarshal viper object onto app config
// the viper object correctly
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
return fmt.Errorf("invalid application config: %w", err)
}
// configure logging for command
newLogWrapper(app)
+76 -56
View File
@@ -10,7 +10,6 @@ import (
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/in-toto/in-toto-golang/in_toto"
"github.com/pkg/errors"
sigopts "github.com/sigstore/cosign/cmd/cosign/cli/options"
"github.com/sigstore/cosign/cmd/cosign/cli/rekor"
"github.com/sigstore/cosign/cmd/cosign/cli/sign"
@@ -41,7 +40,7 @@ import (
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/spdx22json"
"github.com/anchore/syft/syft/formats/spdxjson"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
@@ -50,7 +49,7 @@ import (
var (
allowedAttestFormats = []sbom.FormatID{
syftjson.ID,
spdx22json.ID,
spdxjson.ID,
cyclonedxjson.ID,
}
@@ -108,7 +107,7 @@ func Run(ctx context.Context, app *config.Application, ko sigopts.KeyOpts, args
subscription := eventBus.Subscribe()
return eventloop.EventLoop(
execWorker(app, *si, format, predicateType, sv, app.File),
execWorker(app, *si, format, predicateType, sv),
eventloop.SetupSignals(),
subscription,
stereoscope.Cleanup,
@@ -125,7 +124,7 @@ func parseAttestationOutput(outputs []string) (format string) {
}
func parseImageSource(userInput string, app *config.Application) (s *source.Input, err error) {
si, err := source.ParseInput(userInput, app.Platform, false)
si, err := source.ParseInputWithName(userInput, app.Platform, false, app.Name)
if err != nil {
return nil, fmt.Errorf("could not generate source input for attest command: %w", err)
}
@@ -144,6 +143,7 @@ func parseImageSource(userInput string, app *config.Application) (s *source.Inpu
switch si.ImageSource {
case image.UnknownSource, image.OciRegistrySource:
si.ImageSource = image.OciRegistrySource
case image.SingularitySource:
default:
return nil, fmt.Errorf("attest command can only be used with image sources fetch directly from the registry, but discovered an image source of %q when given %q", si.ImageSource, userInput)
}
@@ -151,7 +151,7 @@ func parseImageSource(userInput string, app *config.Application) (s *source.Inpu
return si, nil
}
func execWorker(app *config.Application, sourceInput source.Input, format sbom.Format, predicateType string, sv *sign.SignerVerifier, file string) <-chan error {
func execWorker(app *config.Application, sourceInput source.Input, format sbom.Format, predicateType string, sv *sign.SignerVerifier) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
@@ -177,36 +177,57 @@ func execWorker(app *config.Application, sourceInput source.Input, format sbom.F
return
}
err = generateAttestation(app, sbomBytes, src, sv, predicateType, file)
signedPayload, err := generateAttestation(sourceInput, sbomBytes, src, sv, predicateType)
if err != nil {
errs <- err
return
}
err = publishAttestation(app, signedPayload, src, sv)
if err != nil {
errs <- err
return
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error {
return nil
},
})
}()
return errs
}
func generateAttestation(app *config.Application, predicate []byte, src *source.Source, sv *sign.SignerVerifier, predicateType string, file string) error {
switch len(src.Image.Metadata.RepoDigests) {
case 0:
return fmt.Errorf("cannot generate attestation since no repo digests were found; make sure you're passing an OCI registry source for the attest command")
case 1:
default:
return fmt.Errorf("cannot generate attestation since multiple repo digests were found for the image: %+v", src.Image.Metadata.RepoDigests)
}
func generateAttestation(si source.Input, predicate []byte, src *source.Source, sv *sign.SignerVerifier, predicateType string) ([]byte, error) {
var h v1.Hash
wrapped := dsse.WrapSigner(sv, intotoJSONDsseType)
ref, err := name.ParseReference(src.Metadata.ImageMetadata.UserInput)
if err != nil {
return err
}
switch si.ImageSource {
case image.OciRegistrySource:
switch len(src.Image.Metadata.RepoDigests) {
case 0:
return nil, fmt.Errorf("cannot generate attestation since no repo digests were found; make sure you're passing an OCI registry source for the attest command")
case 1:
d, err := name.NewDigest(src.Image.Metadata.RepoDigests[0])
if err != nil {
return nil, err
}
digest, err := ociremote.ResolveDigest(ref)
if err != nil {
return err
}
h, err = v1.NewHash(d.Identifier())
if err != nil {
return nil, err
}
default:
return nil, fmt.Errorf("cannot generate attestation since multiple repo digests were found for the image: %+v", src.Image.Metadata.RepoDigests)
}
h, _ := v1.NewHash(digest.Identifier())
case image.SingularitySource:
var err error
h, err = v1.NewHash(src.Image.Metadata.ID)
if err != nil {
return nil, err
}
}
sh, err := attestation.GenerateStatement(attestation.GenerateOpts{
Predicate: bytes.NewBuffer(predicate),
@@ -214,38 +235,44 @@ func generateAttestation(app *config.Application, predicate []byte, src *source.
Digest: h.Hex,
})
if err != nil {
return err
return nil, err
}
payload, err := json.Marshal(sh)
if err != nil {
return err
return nil, err
}
signedPayload, err := wrapped.SignMessage(bytes.NewReader(payload), signatureoptions.WithContext(context.Background()))
if err != nil {
return errors.Wrap(err, "unable to sign SBOM")
}
wrapped := dsse.WrapSigner(sv, intotoJSONDsseType)
return wrapped.SignMessage(bytes.NewReader(payload), signatureoptions.WithContext(context.Background()))
}
// publishAttestation publishes signedPayload to the location specified by the user.
func publishAttestation(app *config.Application, signedPayload []byte, src *source.Source, sv *sign.SignerVerifier) error {
switch {
// We want to give the option to not upload the generated attestation
// if passed or if the user is using local PKI
if app.Attest.NoUpload || app.Attest.KeyRef != "" {
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error {
var err error
if file != "" {
err = os.WriteFile(file, signedPayload, 0600)
} else {
_, err = os.Stdout.Write(signedPayload)
}
return err
},
})
return nil
}
case app.Attest.NoUpload || app.Attest.KeyRef != "":
if app.File != "" {
return os.WriteFile(app.File, signedPayload, 0600)
}
return uploadAttestation(app, signedPayload, digest, sv)
_, err := os.Stdout.Write(signedPayload)
return err
default:
ref, err := name.ParseReference(src.Metadata.ImageMetadata.UserInput)
if err != nil {
return err
}
digest, err := ociremote.ResolveDigest(ref)
if err != nil {
return err
}
return uploadAttestation(app, signedPayload, digest, sv)
}
}
func trackUploadAttestation() (*progress.Stage, *progress.Manual) {
@@ -324,22 +351,15 @@ func uploadAttestation(app *config.Application, signedPayload []byte, digest nam
prog.SetCompleted()
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error {
return nil
},
})
return nil
}
func formatPredicateType(format sbom.Format) string {
switch format.ID() {
case spdx22json.ID:
case spdxjson.ID:
return in_toto.PredicateSPDX
case cyclonedxjson.ID:
// Tentative see https://github.com/in-toto/attestation/issues/82
return "https://cyclonedx.org/bom"
return in_toto.PredicateCycloneDX
case syftjson.ID:
return "https://syft.dev/bom"
default:
+23 -13
View File
@@ -6,17 +6,18 @@ import (
cranecmd "github.com/google/go-containerregistry/cmd/crane/cmd"
"github.com/gookit/color"
logrusUpstream "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/spf13/viper"
"github.com/wagoodman/go-partybus"
"github.com/anchore/go-logger/adapter/logrus"
"github.com/anchore/stereoscope"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/logger"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/event"
@@ -31,8 +32,6 @@ const indent = " "
// at this level. Values from the config should only be used after `app.LoadAllValues` has been called.
// Cobra does not have knowledge of the user provided flags until the `RunE` block of each command.
// `RunE` is the earliest that the complete application configuration can be loaded.
//
//nolint:funlen
func New() (*cobra.Command, error) {
app := &config.Application{}
@@ -118,7 +117,7 @@ func validateArgs(cmd *cobra.Command, args []string) error {
}
func checkForApplicationUpdate() {
log.Debugf("checking if new vesion of %s is available", internal.ApplicationName)
log.Debugf("checking if a new version of %s is available", internal.ApplicationName)
isAvailable, newVersion, err := version.IsUpdateAvailable()
if err != nil {
// this should never stop the application
@@ -138,22 +137,33 @@ func checkForApplicationUpdate() {
func logApplicationConfig(app *config.Application) {
versionInfo := version.FromBuild()
log.Infof("syft version: %+v", versionInfo.Version)
log.Infof("%s version: %+v", internal.ApplicationName, versionInfo.Version)
log.Debugf("application config:\n%+v", color.Magenta.Sprint(app.String()))
}
func newLogWrapper(app *config.Application) {
cfg := logger.LogrusConfig{
cfg := logrus.Config{
EnableConsole: (app.Log.FileLocation == "" || app.Verbosity > 0) && !app.Quiet,
EnableFile: app.Log.FileLocation != "",
Level: app.Log.LevelOpt,
Structured: app.Log.Structured,
FileLocation: app.Log.FileLocation,
Level: app.Log.Level,
}
logWrapper := logger.NewLogrusLogger(cfg)
if app.Log.Structured {
cfg.Formatter = &logrusUpstream.JSONFormatter{
TimestampFormat: "2006-01-02 15:04:05",
DisableTimestamp: false,
DisableHTMLEscape: false,
PrettyPrint: false,
}
}
logWrapper, err := logrus.New(cfg)
if err != nil {
// this is kinda circular, but we can't return an error... ¯\_(ツ)_/¯
// I'm going to leave this here in case we one day have a different default logger other than the "discard" logger
log.Error("unable to initialize logger: %+v", err)
return
}
syft.SetLogger(logWrapper)
stereoscope.SetLogger(&logger.LogrusNestedLogger{
Logger: logWrapper.Logger.WithField("from-lib", "stereoscope"),
})
stereoscope.SetLogger(logWrapper.Nested("from-lib", "stereoscope"))
}
+1 -1
View File
@@ -32,7 +32,7 @@ func Completion() *cobra.Command {
`,
DisableFlagsInUseLine: true,
ValidArgs: []string{"bash", "zsh", "fish"},
Args: cobra.ExactValidArgs(1),
Args: cobra.MatchAll(cobra.ExactArgs(1), cobra.OnlyValidArgs),
RunE: func(cmd *cobra.Command, args []string) error {
var err error
switch args[0] {
-2
View File
@@ -15,8 +15,6 @@ import (
// eventLoop listens to worker errors (from execution path), worker events (from a partybus subscription), and
// signal interrupts. Is responsible for handling each event relative to a given UI an to coordinate eventing until
// an eventual graceful exit.
//
//nolint:funlen
func EventLoop(workerErrs <-chan error, signals <-chan os.Signal, subscription *partybus.Subscription, cleanupFn func(), uxs ...ui.UI) error {
defer cleanupFn()
events := subscription.Events()
-29
View File
@@ -22,7 +22,6 @@ func Tasks(app *config.Application) ([]Task, error) {
generateCatalogFileMetadataTask,
generateCatalogFileDigestsTask,
generateCatalogSecretsTask,
generateCatalogFileClassificationsTask,
generateCatalogContentsTask,
}
@@ -162,34 +161,6 @@ func generateCatalogSecretsTask(app *config.Application) (Task, error) {
return task, nil
}
func generateCatalogFileClassificationsTask(app *config.Application) (Task, error) {
if !app.FileClassification.Cataloger.Enabled {
return nil, nil
}
// TODO: in the future we could expose out the classifiers via configuration
classifierCataloger, err := file.NewClassificationCataloger(file.DefaultClassifiers)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(app.FileClassification.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := classifierCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.FileClassifications = result
return nil, nil
}
return task, nil
}
func generateCatalogContentsTask(app *config.Application) (Task, error) {
if !app.FileContents.Cataloger.Enabled {
return nil, nil
+16 -9
View File
@@ -1,28 +1,35 @@
package options
import (
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/cyclonedxxml"
"github.com/anchore/syft/syft/formats/github"
"github.com/anchore/syft/syft/formats/spdxjson"
"github.com/anchore/syft/syft/formats/spdxtagvalue"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/formats/text"
"github.com/anchore/syft/syft/sbom"
)
func FormatAliases(ids ...sbom.FormatID) (aliases []string) {
for _, id := range ids {
switch id {
case syft.JSONFormatID:
case syftjson.ID:
aliases = append(aliases, "syft-json")
case syft.TextFormatID:
case text.ID:
aliases = append(aliases, "text")
case syft.TableFormatID:
case table.ID:
aliases = append(aliases, "table")
case syft.SPDXJSONFormatID:
case spdxjson.ID:
aliases = append(aliases, "spdx-json")
case syft.SPDXTagValueFormatID:
case spdxtagvalue.ID:
aliases = append(aliases, "spdx-tag-value")
case syft.CycloneDxXMLFormatID:
case cyclonedxxml.ID:
aliases = append(aliases, "cyclonedx-xml")
case syft.CycloneDxJSONFormatID:
case cyclonedxjson.ID:
aliases = append(aliases, "cyclonedx-json")
case syft.GitHubID:
case github.ID:
aliases = append(aliases, "github", "github-json")
default:
aliases = append(aliases, string(id))
+14 -56
View File
@@ -14,19 +14,14 @@ import (
)
type PackagesOptions struct {
Scope string
Output []string
OutputTemplatePath string
File string
Platform string
Host string
Username string
Password string
Dockerfile string
Exclude []string
OverwriteExistingImage bool
ImportTimeout uint
Catalogers []string
Scope string
Output []string
OutputTemplatePath string
File string
Platform string
Exclude []string
Catalogers []string
Name string
}
var _ Interface = (*PackagesOptions)(nil)
@@ -47,29 +42,14 @@ func (o *PackagesOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.Flags().StringVarP(&o.Platform, "platform", "", "",
"an optional platform specifier for container image sources (e.g. 'linux/arm64', 'linux/arm64/v8', 'arm64', 'linux')")
cmd.Flags().StringVarP(&o.Host, "host", "H", "",
"the hostname or URL of the Anchore Enterprise instance to upload to")
cmd.Flags().StringVarP(&o.Username, "username", "u", "",
"the username to authenticate against Anchore Enterprise")
cmd.Flags().StringVarP(&o.Password, "password", "p", "",
"the password to authenticate against Anchore Enterprise")
cmd.Flags().StringVarP(&o.Dockerfile, "dockerfile", "d", "",
"include dockerfile for upload to Anchore Enterprise")
cmd.Flags().StringArrayVarP(&o.Exclude, "exclude", "", nil,
"exclude paths from being scanned using a glob expression")
cmd.Flags().StringArrayVarP(&o.Catalogers, "catalogers", "", nil,
"enable one or more package catalogers")
cmd.Flags().BoolVarP(&o.OverwriteExistingImage, "overwrite-existing-image", "", false,
"overwrite an existing image during the upload to Anchore Enterprise")
cmd.Flags().UintVarP(&o.ImportTimeout, "import-timeout", "", 30,
"set a timeout duration (in seconds) for the upload to Anchore Enterprise")
cmd.Flags().StringVarP(&o.Name, "name", "", "",
"set the name of the target being analyzed")
return bindPackageConfigOptions(cmd.Flags(), v)
}
@@ -93,6 +73,10 @@ func bindPackageConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
return err
}
if err := v.BindPFlag("name", flags.Lookup("name")); err != nil {
return err
}
if err := v.BindPFlag("output", flags.Lookup("output")); err != nil {
return err
}
@@ -105,31 +89,5 @@ func bindPackageConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
return err
}
// Upload options //////////////////////////////////////////////////////////
if err := v.BindPFlag("anchore.host", flags.Lookup("host")); err != nil {
return err
}
if err := v.BindPFlag("anchore.username", flags.Lookup("username")); err != nil {
return err
}
if err := v.BindPFlag("anchore.password", flags.Lookup("password")); err != nil {
return err
}
if err := v.BindPFlag("anchore.dockerfile", flags.Lookup("dockerfile")); err != nil {
return err
}
if err := v.BindPFlag("anchore.overwrite-existing-image", flags.Lookup("overwrite-existing-image")); err != nil {
return err
}
if err := v.BindPFlag("anchore.import-timeout", flags.Lookup("import-timeout")); err != nil {
return err
}
return nil
}
+1 -1
View File
@@ -57,7 +57,7 @@ func Packages(v *viper.Viper, app *config.Application, ro *options.RootOptions,
}),
Args: func(cmd *cobra.Command, args []string) error {
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
return fmt.Errorf("invalid application config: %w", err)
}
// configure logging for command
newLogWrapper(app)
+1 -60
View File
@@ -3,8 +3,6 @@ package packages
import (
"context"
"fmt"
"io"
"os"
"github.com/wagoodman/go-partybus"
@@ -12,7 +10,6 @@ import (
"github.com/anchore/syft/cmd/syft/cli/eventloop"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/anchore"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
@@ -45,7 +42,7 @@ func Run(ctx context.Context, app *config.Application, args []string) error {
// could be an image or a directory, with or without a scheme
userInput := args[0]
si, err := source.ParseInput(userInput, app.Platform, true)
si, err := source.ParseInputWithName(userInput, app.Platform, true, app.Name)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
@@ -88,13 +85,6 @@ func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-
errs <- fmt.Errorf("no SBOM produced for %q", si.UserInput)
}
if app.Anchore.Host != "" {
if err := runPackageSbomUpload(src, *s, app); err != nil {
errs <- err
return
}
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error { return writer.Write(*s) },
@@ -144,55 +134,6 @@ func MergeRelationships(cs ...<-chan artifact.Relationship) (relationships []art
return relationships
}
func runPackageSbomUpload(src *source.Source, s sbom.SBOM, app *config.Application) error {
log.Infof("uploading results to %s", app.Anchore.Host)
if src.Metadata.Scheme != source.ImageScheme {
return fmt.Errorf("unable to upload results: only images are supported")
}
var dockerfileContents []byte
if app.Anchore.Dockerfile != "" {
if _, err := os.Stat(app.Anchore.Dockerfile); os.IsNotExist(err) {
return fmt.Errorf("unable dockerfile=%q does not exist: %w", app.Anchore.Dockerfile, err)
}
fh, err := os.Open(app.Anchore.Dockerfile)
if err != nil {
return fmt.Errorf("unable to open dockerfile=%q: %w", app.Anchore.Dockerfile, err)
}
dockerfileContents, err = io.ReadAll(fh)
if err != nil {
return fmt.Errorf("unable to read dockerfile=%q: %w", app.Anchore.Dockerfile, err)
}
}
c, err := anchore.NewClient(anchore.Configuration{
BaseURL: app.Anchore.Host,
Username: app.Anchore.Username,
Password: app.Anchore.Password,
})
if err != nil {
return fmt.Errorf("failed to create anchore client: %w", err)
}
importCfg := anchore.ImportConfig{
ImageMetadata: src.Image.Metadata,
SBOM: s,
Dockerfile: dockerfileContents,
OverwriteExistingUpload: app.Anchore.OverwriteExistingImage,
Timeout: app.Anchore.ImportTimeout,
}
if err := c.Import(context.Background(), importCfg); err != nil {
return fmt.Errorf("failed to upload results to host=%s: %+v", app.Anchore.Host, err)
}
return nil
}
func validateOutputOptions(app *config.Application) error {
var usesTemplateOutput bool
for _, o := range app.Outputs {
+1 -1
View File
@@ -28,7 +28,7 @@ func PowerUser(v *viper.Viper, app *config.Application, ro *options.RootOptions)
}),
Args: func(cmd *cobra.Command, args []string) error {
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %v", err)
return fmt.Errorf("invalid application config: %w", err)
}
// configure logging for command
newLogWrapper(app)
+1 -1
View File
@@ -46,7 +46,7 @@ func Run(ctx context.Context, app *config.Application, args []string) error {
}()
userInput := args[0]
si, err := source.ParseInput(userInput, app.Platform, true)
si, err := source.ParseInputWithName(userInput, app.Platform, true, app.Name)
if err != nil {
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
+51 -53
View File
@@ -3,17 +3,14 @@ module github.com/anchore/syft
go 1.18
require (
github.com/CycloneDX/cyclonedx-go v0.5.2
github.com/CycloneDX/cyclonedx-go v0.7.0
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
github.com/acobaugh/osrelease v0.1.0
github.com/adrg/xdg v0.3.3
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.1.1-0.20220428202044-a072fa3cb6d7
github.com/antihax/optional v1.0.0
github.com/bmatcuk/doublestar/v4 v4.0.2
github.com/dustin/go-humanize v1.0.0
github.com/facebookincubator/nvdtools v0.1.4
@@ -35,9 +32,9 @@ require (
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e
github.com/sergi/go-diff v1.2.0
github.com/sirupsen/logrus v1.9.0
github.com/spdx/tools-golang v0.2.0
github.com/spdx/tools-golang v0.3.1-0.20221108182156-8a01147e6342
github.com/spf13/afero v1.8.2
github.com/spf13/cobra v1.5.0
github.com/spf13/cobra v1.6.0
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.13.0
github.com/stretchr/testify v1.8.0
@@ -45,39 +42,41 @@ require (
github.com/wagoodman/go-partybus v0.0.0-20210627031916-db1f5573bbc5
github.com/wagoodman/go-progress v0.0.0-20200731105512-1020f39e6240
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb
github.com/x-cray/logrus-prefixed-formatter v0.5.2
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4
golang.org/x/net v0.0.0-20220909164309-bea034e7d591
golang.org/x/term v0.0.0-20220526004731-065cf7ba2467
golang.org/x/net v0.0.0-20221012135044-0b7e1fb9d458
golang.org/x/term v0.0.0-20220919170432-7a66f970e087
gopkg.in/yaml.v2 v2.4.0
)
require (
github.com/Masterminds/sprig/v3 v3.2.2
github.com/anchore/stereoscope v0.0.0-20220921141924-56552770e555
github.com/anchore/go-logger v0.0.0-20220728155337-03b66a5207d8
github.com/anchore/stereoscope v0.0.0-20221208011002-c5ff155d72f1
github.com/docker/docker v20.10.17+incompatible
github.com/google/go-containerregistry v0.11.0
github.com/in-toto/in-toto-golang v0.3.4-0.20220709202702-fa494aaa0add
github.com/knqyf263/go-rpmdb v0.0.0-20220629110411-9a3bd2ebb923
github.com/in-toto/in-toto-golang v0.4.1-0.20221018183522-731d0640b65f
github.com/invopop/jsonschema v0.7.0
github.com/knqyf263/go-rpmdb v0.0.0-20221030135625-4082a22221ce
github.com/opencontainers/go-digest v1.0.0
github.com/sassoftware/go-rpmutils v0.2.0
github.com/sigstore/cosign v1.12.1
github.com/sigstore/cosign v1.13.1
github.com/sigstore/rekor v0.12.1-0.20220915152154-4bb6f441c1b2
github.com/sigstore/sigstore v1.4.1
github.com/sigstore/sigstore v1.4.4
github.com/vbatts/go-mtree v0.5.0
gopkg.in/yaml.v3 v3.0.1
)
require (
bitbucket.org/creachadair/shell v0.0.7 // indirect
cloud.google.com/go/compute v1.7.0 // indirect
cloud.google.com/go/compute v1.10.0 // indirect
github.com/AliyunContainerService/ack-ram-tool/pkg/credentials/alibabacloudsdkgo/helper v0.2.0 // indirect
github.com/Azure/azure-sdk-for-go v66.0.0+incompatible // indirect
github.com/Azure/go-autorest v14.2.0+incompatible // indirect
github.com/Azure/go-autorest/autorest v0.11.28 // indirect
github.com/Azure/go-autorest/autorest/adal v0.9.20 // indirect
github.com/Azure/go-autorest/autorest/adal v0.9.21 // indirect
github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 // indirect
github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 // indirect
github.com/Azure/go-autorest/autorest/azure/cli v0.4.6 // indirect
github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect
github.com/Azure/go-autorest/logger v0.2.1 // indirect
github.com/Azure/go-autorest/tracing v0.6.0 // indirect
@@ -98,20 +97,20 @@ require (
github.com/alibabacloud-go/tea-xml v1.1.2 // indirect
github.com/aliyun/credentials-go v1.2.3 // indirect
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
github.com/aws/aws-sdk-go-v2 v1.16.14 // indirect
github.com/aws/aws-sdk-go-v2/config v1.17.5 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.12.18 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.15 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.21 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.15 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.22 // indirect
github.com/aws/aws-sdk-go-v2 v1.16.16 // indirect
github.com/aws/aws-sdk-go-v2/config v1.17.8 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.12.21 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.17 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.23 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.17 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.24 // indirect
github.com/aws/aws-sdk-go-v2/service/ecr v1.15.0 // indirect
github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.12.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.15 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.11.21 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.13.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.16.17 // indirect
github.com/aws/smithy-go v1.13.2 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.17 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.11.23 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.13.6 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.16.19 // indirect
github.com/aws/smithy-go v1.13.3 // indirect
github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20220517224237-e6f29200ae04 // indirect
github.com/benbjohnson/clock v1.1.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
@@ -125,7 +124,7 @@ require (
github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4 // indirect
github.com/cncf/xds/go v0.0.0-20211130200136-a8f946100490 // indirect
github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be // indirect
github.com/containerd/containerd v1.5.13 // indirect
github.com/containerd/containerd v1.6.12 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.12.0 // indirect
github.com/coreos/go-oidc/v3 v3.4.0 // indirect
github.com/coreos/go-semver v0.3.0 // indirect
@@ -153,7 +152,7 @@ require (
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/jsonreference v0.20.0 // indirect
github.com/go-openapi/loads v0.21.2 // indirect
github.com/go-openapi/runtime v0.24.1 // indirect
github.com/go-openapi/runtime v0.24.2 // indirect
github.com/go-openapi/spec v0.20.7 // indirect
github.com/go-openapi/strfmt v0.21.3 // indirect
github.com/go-openapi/swag v0.22.3 // indirect
@@ -177,13 +176,13 @@ require (
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/trillian v1.5.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.1.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.0 // indirect
github.com/googleapis/gnostic v0.5.5 // indirect
github.com/gorilla/websocket v1.4.2 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.2 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-retryablehttp v0.7.1 // indirect
@@ -191,9 +190,9 @@ require (
github.com/huandu/xstrings v1.3.2 // indirect
github.com/iancoleman/orderedmap v0.0.0-20190318233801-ac98e3ecb4b0 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
github.com/jedisct1/go-minisign v0.0.0-20211028175153-1c139d1cc84b // indirect
github.com/jhump/protoreflect v1.12.0 // indirect
github.com/jhump/protoreflect v1.13.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/jonboulle/clockwork v0.3.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
@@ -202,14 +201,14 @@ require (
github.com/klauspost/compress v1.15.9 // indirect
github.com/klauspost/pgzip v1.2.5 // indirect
github.com/leodido/go-urn v1.2.1 // indirect
github.com/letsencrypt/boulder v0.0.0-20220723181115-27de4befb95e // indirect
github.com/letsencrypt/boulder v0.0.0-20220929215747-76583552c2be // indirect
github.com/logrusorgru/aurora v0.0.0-20200102142835-e9ef32dff381 // indirect
github.com/magiconair/properties v1.8.6 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.12 // indirect
github.com/mattn/go-isatty v0.0.14 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/miekg/pkcs11 v1.1.1 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
@@ -218,7 +217,6 @@ require (
github.com/mozillazg/docker-credential-acr-helper v0.3.0 // indirect
github.com/nwaples/rardecode v1.1.0 // indirect
github.com/oklog/ulid v1.3.1 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pelletier/go-toml/v2 v2.0.5 // indirect
@@ -236,7 +234,7 @@ require (
github.com/segmentio/ksuid v1.0.4 // indirect
github.com/shibumi/go-pathspec v1.3.0 // indirect
github.com/shopspring/decimal v1.2.0 // indirect
github.com/sigstore/fulcio v0.5.3 // indirect
github.com/sigstore/fulcio v0.6.0 // indirect
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 // indirect
github.com/soheilhy/cmux v0.1.5 // indirect
github.com/spf13/cast v1.5.0 // indirect
@@ -244,13 +242,13 @@ require (
github.com/spiffe/go-spiffe/v2 v2.1.1 // indirect
github.com/stretchr/objx v0.4.0 // indirect
github.com/subosito/gotenv v1.4.1 // indirect
github.com/sylabs/sif/v2 v2.7.2 // indirect
github.com/sylabs/sif/v2 v2.8.1 // indirect
github.com/sylabs/squashfs v0.6.1 // indirect
github.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d // indirect
github.com/tent/canonical-json-go v0.0.0-20130607151641-96e4ba3a7613 // indirect
github.com/thales-e-security/pool v0.0.2 // indirect
github.com/therootcompany/xz v1.0.1 // indirect
github.com/theupdateframework/go-tuf v0.5.0 // indirect
github.com/theupdateframework/go-tuf v0.5.2-0.20220930112810-3890c1e7ace4 // indirect
github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399 // indirect
github.com/tjfoc/gmsm v1.3.2 // indirect
github.com/tmc/grpc-websocket-proxy v0.0.0-20201229170055-e5319fda7802 // indirect
@@ -291,17 +289,17 @@ require (
go.uber.org/multierr v1.8.0 // indirect
go.uber.org/zap v1.23.0 // indirect
golang.org/x/exp v0.0.0-20220823124025-807a23277127 // indirect
golang.org/x/oauth2 v0.0.0-20220822191816-0ebed06d0094 // indirect
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4 // indirect
golang.org/x/sys v0.0.0-20220907062415-87db552b00fd // indirect
golang.org/x/oauth2 v0.0.0-20221006150949-b44042a4b9c1 // indirect
golang.org/x/sync v0.0.0-20220929204114-8fcdb60fdcc0 // indirect
golang.org/x/sys v0.0.0-20220928140112-f11e5e49a4ec // indirect
golang.org/x/text v0.3.8-0.20211004125949-5bd84dd9b33b // indirect
golang.org/x/time v0.0.0-20220722155302-e5dcc9cfc0b9 // indirect
golang.org/x/time v0.0.0-20220922220347-f3bd1da661af // indirect
golang.org/x/tools v0.1.12 // indirect
golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f // indirect
google.golang.org/api v0.96.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/api v0.99.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20220805133916-01dd62135a58 // indirect
google.golang.org/grpc v1.49.0 // indirect
google.golang.org/genproto v0.0.0-20221010155953-15ba04fc1c0e // indirect
google.golang.org/grpc v1.50.1 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/cheggaaa/pb.v1 v1.0.28 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
@@ -334,8 +332,8 @@ require (
// we are hinting brotli to latest due to warning when installing archiver v3:
// go: warning: github.com/andybalholm/brotli@v1.0.1: retracted by module author: occasional panics and data corruption
github.com/andybalholm/brotli v1.0.4 // indirect
github.com/pkg/errors v0.9.1
golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90 // indirect
github.com/pkg/errors v0.9.1 // indirect
golang.org/x/crypto v0.0.0-20220926161630-eccd6366d1be // indirect
)
retract (
+246 -110
View File
File diff suppressed because it is too large Load Diff
-115
View File
@@ -1,115 +0,0 @@
package anchore
import (
"context"
"errors"
"fmt"
"path"
"strings"
"unicode"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/version"
)
type Configuration struct {
BaseURL string
Username string
Password string
UserAgent string
}
type Client struct {
config Configuration
client *external.APIClient
}
func NewClient(cfg Configuration) (*Client, error) {
if cfg.UserAgent == "" {
versionInfo := version.FromBuild()
// format: product / product-version comment
cfg.UserAgent = fmt.Sprintf("%s / %s %s", internal.ApplicationName, versionInfo.Version, versionInfo.Platform)
}
baseURL, err := prepareBaseURLForClient(cfg.BaseURL)
if err != nil {
return nil, fmt.Errorf("unable to create client: %w", err)
}
return &Client{
config: cfg,
client: external.NewAPIClient(&external.Configuration{
BasePath: baseURL,
UserAgent: cfg.UserAgent,
}),
}, nil
}
func (c *Client) newRequestContext(parentContext context.Context) context.Context {
if parentContext == nil {
parentContext = context.Background()
}
return context.WithValue(
parentContext,
external.ContextBasicAuth,
external.BasicAuth{
UserName: c.config.Username,
Password: c.config.Password,
},
)
}
var ErrInvalidBaseURLInput = errors.New("invalid base URL input")
func prepareBaseURLForClient(baseURL string) (string, error) {
if err := checkBaseURLInput(baseURL); err != nil {
return "", err
}
scheme, urlWithoutScheme := splitSchemeFromURL(baseURL)
if scheme == "" {
scheme = "http"
}
urlWithoutScheme = path.Clean(urlWithoutScheme)
const requiredSuffix = "v1"
if path.Base(urlWithoutScheme) != requiredSuffix {
urlWithoutScheme = path.Join(urlWithoutScheme, requiredSuffix)
}
preparedBaseURL := scheme + "://" + urlWithoutScheme
return preparedBaseURL, nil
}
func checkBaseURLInput(url string) error {
if url == "" {
return ErrInvalidBaseURLInput
}
firstCharacter := rune(url[0])
if !(unicode.IsLetter(firstCharacter)) {
return ErrInvalidBaseURLInput
}
return nil
}
func splitSchemeFromURL(url string) (scheme, urlWithoutScheme string) {
if hasScheme(url) {
urlParts := strings.SplitN(url, "://", 2)
scheme = urlParts[0]
urlWithoutScheme = urlParts[1]
return
}
return "", url
}
func hasScheme(url string) bool {
parts := strings.Split(url, "://")
return len(parts) > 1
}
-210
View File
@@ -1,210 +0,0 @@
package anchore
import "testing"
func TestHasScheme(t *testing.T) {
cases := []struct {
url string
expected bool
}{
{
url: "http://localhost",
expected: true,
},
{
url: "https://anchore.com:8443",
expected: true,
},
{
url: "google.com",
expected: false,
},
{
url: "",
expected: false,
},
}
for _, testCase := range cases {
t.Run(testCase.url, func(t *testing.T) {
result := hasScheme(testCase.url)
if testCase.expected != result {
t.Errorf("expected %t but got %t", testCase.expected, result)
}
})
}
}
func TestPrepareBaseURLForClient(t *testing.T) {
cases := []struct {
inputURL string
expectedURL string
expectedErr error
}{
{
inputURL: "",
expectedURL: "",
expectedErr: ErrInvalidBaseURLInput,
},
{
inputURL: "localhost",
expectedURL: "http://localhost/v1",
expectedErr: nil,
},
{
inputURL: "https://localhost",
expectedURL: "https://localhost/v1",
expectedErr: nil,
},
{
inputURL: "https://localhost/",
expectedURL: "https://localhost/v1",
expectedErr: nil,
},
{
inputURL: "https://localhost/v1/",
expectedURL: "https://localhost/v1",
expectedErr: nil,
},
{
inputURL: "https://localhost/v1//",
expectedURL: "https://localhost/v1",
expectedErr: nil,
},
{
inputURL: "http://something.com/platform/v1/services/anchore",
expectedURL: "http://something.com/platform/v1/services/anchore/v1",
expectedErr: nil,
},
{
inputURL: "my-host:8228",
expectedURL: "http://my-host:8228/v1",
expectedErr: nil,
},
{
inputURL: "v1/v1",
expectedURL: "http://v1/v1",
expectedErr: nil,
},
{
inputURL: "/v1",
expectedURL: "",
expectedErr: ErrInvalidBaseURLInput,
},
{
inputURL: "/imports/images",
expectedURL: "",
expectedErr: ErrInvalidBaseURLInput,
},
}
for _, testCase := range cases {
t.Run(testCase.inputURL, func(t *testing.T) {
resultURL, err := prepareBaseURLForClient(testCase.inputURL)
if err != testCase.expectedErr {
t.Errorf("expected err to be '%v' but got '%v'", testCase.expectedErr, err)
}
if resultURL != testCase.expectedURL {
t.Errorf("expected URL to be '%v' but got '%v'", testCase.expectedURL, resultURL)
}
})
}
}
func TestCheckBaseURLInput(t *testing.T) {
cases := []struct {
input string
expected error
}{
{
input: "",
expected: ErrInvalidBaseURLInput,
},
{
input: "x",
expected: nil,
},
{
input: "localhost:8000",
expected: nil,
},
{
input: ":80",
expected: ErrInvalidBaseURLInput,
},
{
input: "/v1",
expected: ErrInvalidBaseURLInput,
},
}
for _, testCase := range cases {
t.Run(testCase.input, func(t *testing.T) {
resultErr := checkBaseURLInput(testCase.input)
if testCase.expected != resultErr {
t.Errorf("expected err to be '%v' but got '%v'", testCase.expected, resultErr)
}
})
}
}
func TestSplitSchemeFromURL(t *testing.T) {
cases := []struct {
input string
expectedScheme string
expectedURLWithoutScheme string
}{
{
input: "",
expectedScheme: "",
expectedURLWithoutScheme: "",
},
{
input: "localhost",
expectedScheme: "",
expectedURLWithoutScheme: "localhost",
},
{
input: "https://anchore.com/path",
expectedScheme: "https",
expectedURLWithoutScheme: "anchore.com/path",
},
{
input: "tcp://host:1234",
expectedScheme: "tcp",
expectedURLWithoutScheme: "host:1234",
},
{
input: "/hello",
expectedScheme: "",
expectedURLWithoutScheme: "/hello",
},
{
input: "://host",
expectedScheme: "",
expectedURLWithoutScheme: "host",
},
{
input: "http//localhost",
expectedScheme: "",
expectedURLWithoutScheme: "http//localhost",
},
}
for _, testCase := range cases {
t.Run(testCase.input, func(t *testing.T) {
resultScheme, resultURLWithoutScheme := splitSchemeFromURL(testCase.input)
if testCase.expectedScheme != resultScheme {
t.Errorf("expected scheme to be '%s' but got '%s'", testCase.expectedScheme, resultScheme)
}
if testCase.expectedURLWithoutScheme != resultURLWithoutScheme {
t.Errorf("expected urlWithoutScheme to be '%s' but got '%s'", testCase.expectedURLWithoutScheme, resultURLWithoutScheme)
}
})
}
}
-145
View File
@@ -1,145 +0,0 @@
package anchore
import (
"context"
"errors"
"fmt"
"time"
"github.com/antihax/optional"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
)
type ImportConfig struct {
ImageMetadata image.Metadata
SBOM sbom.SBOM
Dockerfile []byte
OverwriteExistingUpload bool
Timeout uint
}
func importProgress(source string) (*progress.Stage, *progress.Manual) {
stage := &progress.Stage{}
prog := &progress.Manual{
// this is the number of stages to expect; start + individual endpoints + stop
Total: 6,
}
bus.Publish(partybus.Event{
Type: event.ImportStarted,
Source: source,
Value: progress.StagedProgressable(&struct {
progress.Stager
progress.Progressable
}{
Stager: progress.Stager(stage),
Progressable: prog,
}),
})
return stage, prog
}
//nolint:funlen
func (c *Client) Import(ctx context.Context, cfg ImportConfig) error {
stage, prog := importProgress(c.config.BaseURL)
timeout := time.Duration(cfg.Timeout) * time.Second
ctxWithTimeout, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
authedCtx := c.newRequestContext(ctxWithTimeout)
stage.Current = "starting session"
startOperation, createResponse, err := c.client.ImportsApi.CreateOperation(authedCtx)
if err != nil {
var detail = "no details given"
var openAPIErr external.GenericOpenAPIError
if errors.As(err, &openAPIErr) {
detail = string(openAPIErr.Body())
}
return fmt.Errorf("unable to start import session: %w: %s", err, detail)
}
defer createResponse.Body.Close()
prog.N++
sessionID := startOperation.Uuid
packageDigest, err := importPackageSBOM(authedCtx, c.client.ImportsApi, sessionID, cfg.SBOM, stage)
if err != nil {
return fmt.Errorf("failed to import Package SBOM: %w", err)
}
prog.N++
manifestDigest, err := importManifest(authedCtx, c.client.ImportsApi, sessionID, cfg.SBOM.Source.ImageMetadata.RawManifest, stage)
if err != nil {
return fmt.Errorf("failed to import Manifest: %w", err)
}
prog.N++
configDigest, err := importConfig(authedCtx, c.client.ImportsApi, sessionID, cfg.SBOM.Source.ImageMetadata.RawConfig, stage)
if err != nil {
return fmt.Errorf("failed to import Config: %w", err)
}
prog.N++
dockerfileDigest, err := importDockerfile(authedCtx, c.client.ImportsApi, sessionID, cfg.Dockerfile, stage)
if err != nil {
return fmt.Errorf("failed to import Dockerfile: %w", err)
}
prog.N++
stage.Current = "finalizing"
imageModel := addImageModel(cfg.ImageMetadata, packageDigest, manifestDigest, dockerfileDigest, configDigest, sessionID)
opts := external.AddImageOpts{
Force: optional.NewBool(cfg.OverwriteExistingUpload),
}
_, addResponse, err := c.client.ImagesApi.AddImage(authedCtx, imageModel, &opts)
if err != nil {
var detail = "no details given"
var openAPIErr external.GenericOpenAPIError
if errors.As(err, &openAPIErr) {
detail = string(openAPIErr.Body())
}
return fmt.Errorf("unable to complete import session=%q: %w: %s", sessionID, err, detail)
}
defer addResponse.Body.Close()
prog.N++
stage.Current = ""
prog.SetCompleted()
return nil
}
func addImageModel(imageMetadata image.Metadata, packageDigest, manifestDigest, dockerfileDigest, configDigest, sessionID string) external.ImageAnalysisRequest {
var tags = make([]string, len(imageMetadata.Tags))
for i, t := range imageMetadata.Tags {
tags[i] = t.String()
}
return external.ImageAnalysisRequest{
Source: external.ImageSource{
Import: &external.ImageImportManifest{
Contents: external.ImportContentDigests{
Packages: packageDigest,
Manifest: manifestDigest,
Dockerfile: dockerfileDigest,
ImageConfig: configDigest,
},
Tags: tags,
Digest: imageMetadata.ManifestDigest,
LocalImageId: imageMetadata.ID,
OperationUuid: sessionID,
},
},
}
}
-50
View File
@@ -1,50 +0,0 @@
//nolint:dupl
package anchore
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/log"
)
type configImportAPI interface {
ImportImageConfig(ctx context.Context, sessionID string, contents interface{}) (external.ImageImportContentResponse, *http.Response, error)
}
func importConfig(ctx context.Context, api configImportAPI, sessionID string, config []byte, stage *progress.Stage) (string, error) {
if len(config) > 0 {
log.Debug("importing image config")
stage.Current = "image config"
// API requires an object, but we do not verify the shape of this object locally
var sender map[string]interface{}
if err := json.Unmarshal(config, &sender); err != nil {
return "", err
}
response, httpResponse, err := api.ImportImageConfig(ctx, sessionID, sender)
if err != nil {
var openAPIErr external.GenericOpenAPIError
if errors.As(err, &openAPIErr) {
log.Errorf("api response: %+v", string(openAPIErr.Body()))
}
return "", fmt.Errorf("unable to import Config: %w", err)
}
defer httpResponse.Body.Close()
if httpResponse.StatusCode != 200 {
return "", fmt.Errorf("unable to import Config: %s", httpResponse.Status)
}
return response.Digest, nil
}
return "", nil
}
-129
View File
@@ -1,129 +0,0 @@
package anchore
import (
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"testing"
"github.com/docker/docker/pkg/ioutils"
"github.com/go-test/deep"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
)
type mockConfigImportAPI struct {
sessionID string
model interface{}
httpResponse *http.Response
err error
ctx context.Context
responseDigest string
wasCalled bool
}
func (m *mockConfigImportAPI) ImportImageConfig(ctx context.Context, sessionID string, contents interface{}) (external.ImageImportContentResponse, *http.Response, error) {
m.wasCalled = true
m.model = contents
m.sessionID = sessionID
m.ctx = ctx
if m.httpResponse == nil {
m.httpResponse = &http.Response{}
}
m.httpResponse.Body = ioutils.NewReadCloserWrapper(strings.NewReader(""), func() error { return nil })
return external.ImageImportContentResponse{Digest: m.responseDigest}, m.httpResponse, m.err
}
func TestConfigImport(t *testing.T) {
sessionID := "my-session"
tests := []struct {
name string
manifestJSONStr string
api *mockConfigImportAPI
expectsError bool
expectsCall bool
}{
{
name: "Go case: import works",
manifestJSONStr: `{ "key": "the-manifest-contents!" }`,
api: &mockConfigImportAPI{
httpResponse: &http.Response{StatusCode: 200},
responseDigest: "digest!",
},
expectsCall: true,
},
{
name: "No manifest provided",
manifestJSONStr: "",
api: &mockConfigImportAPI{},
expectsCall: false,
},
{
name: "API returns an error",
manifestJSONStr: `{ "key": "the-manifest-contents!" }`,
api: &mockConfigImportAPI{
err: fmt.Errorf("api error, something went wrong"),
},
expectsError: true,
expectsCall: true,
},
{
name: "API HTTP-level error",
manifestJSONStr: `{ "key": "the-manifest-contents!" }`,
api: &mockConfigImportAPI{
httpResponse: &http.Response{StatusCode: 404},
},
expectsError: true,
expectsCall: true,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
digest, err := importConfig(context.TODO(), test.api, sessionID, []byte(test.manifestJSONStr), &progress.Stage{})
// validate error handling
if err != nil && !test.expectsError {
t.Fatalf("did not expect an error, but got: %+v", err)
} else if err == nil && test.expectsError {
t.Fatalf("did expect an error, but got none")
}
if !test.api.wasCalled && test.expectsCall {
t.Fatalf("was not called!")
} else if test.api.wasCalled && !test.expectsCall {
t.Fatalf("should not have been called")
}
if !test.expectsCall {
return
}
if digest != test.api.responseDigest {
t.Errorf("unexpected content digest: %q != %q", digest, test.api.responseDigest)
}
// validating that the mock got the right parameters
if test.api.sessionID != sessionID {
t.Errorf("different session ID: %s != %s", test.api.sessionID, sessionID)
}
var expected map[string]interface{}
if err := json.Unmarshal([]byte(test.manifestJSONStr), &expected); err != nil {
t.Fatalf("could not unmarshal expected results")
}
for _, d := range deep.Equal(test.api.model, expected) {
t.Errorf("model difference: %s", d)
}
})
}
}
-42
View File
@@ -1,42 +0,0 @@
package anchore
import (
"context"
"errors"
"fmt"
"net/http"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/log"
)
type dockerfileImportAPI interface {
ImportImageDockerfile(ctx context.Context, sessionID string, contents string) (external.ImageImportContentResponse, *http.Response, error)
}
func importDockerfile(ctx context.Context, api dockerfileImportAPI, sessionID string, dockerfile []byte, stage *progress.Stage) (string, error) {
if len(dockerfile) > 0 {
log.Debug("importing dockerfile")
stage.Current = "dockerfile"
response, httpResponse, err := api.ImportImageDockerfile(ctx, sessionID, string(dockerfile))
if err != nil {
var openAPIErr external.GenericOpenAPIError
if errors.As(err, &openAPIErr) {
log.Errorf("api response: %+v", string(openAPIErr.Body()))
}
return "", fmt.Errorf("unable to import Dockerfile: %w", err)
}
defer httpResponse.Body.Close()
if httpResponse.StatusCode != 200 {
return "", fmt.Errorf("unable to import Dockerfile: %s", httpResponse.Status)
}
return response.Digest, nil
}
return "", nil
}
-123
View File
@@ -1,123 +0,0 @@
package anchore
import (
"context"
"fmt"
"net/http"
"strings"
"testing"
"github.com/docker/docker/pkg/ioutils"
"github.com/go-test/deep"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
)
type mockDockerfileImportAPI struct {
sessionID string
model string
httpResponse *http.Response
err error
ctx context.Context
responseDigest string
wasCalled bool
}
func (m *mockDockerfileImportAPI) ImportImageDockerfile(ctx context.Context, sessionID string, contents string) (external.ImageImportContentResponse, *http.Response, error) {
m.wasCalled = true
m.model = contents
m.sessionID = sessionID
m.ctx = ctx
if m.httpResponse == nil {
m.httpResponse = &http.Response{}
}
m.httpResponse.Body = ioutils.NewReadCloserWrapper(strings.NewReader(""), func() error { return nil })
return external.ImageImportContentResponse{Digest: m.responseDigest}, m.httpResponse, m.err
}
func TestDockerfileImport(t *testing.T) {
sessionID := "my-session"
tests := []struct {
name string
dockerfile string
api *mockDockerfileImportAPI
expectsError bool
expectsCall bool
}{
{
name: "Go case: import works",
dockerfile: "the-manifest-contents!",
api: &mockDockerfileImportAPI{
httpResponse: &http.Response{StatusCode: 200},
responseDigest: "digest!",
},
expectsCall: true,
},
{
name: "No manifest provided",
dockerfile: "",
api: &mockDockerfileImportAPI{},
expectsCall: false,
},
{
name: "API returns an error",
dockerfile: "the-manifest-contents!",
api: &mockDockerfileImportAPI{
err: fmt.Errorf("api error, something went wrong"),
},
expectsError: true,
expectsCall: true,
},
{
name: "API HTTP-level error",
dockerfile: "the-manifest-contents!",
api: &mockDockerfileImportAPI{
httpResponse: &http.Response{StatusCode: 404},
},
expectsError: true,
expectsCall: true,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
digest, err := importDockerfile(context.TODO(), test.api, sessionID, []byte(test.dockerfile), &progress.Stage{})
// validate error handling
if err != nil && !test.expectsError {
t.Fatalf("did not expect an error, but got: %+v", err)
} else if err == nil && test.expectsError {
t.Fatalf("did expect an error, but got none")
}
if !test.api.wasCalled && test.expectsCall {
t.Fatalf("was not called!")
} else if test.api.wasCalled && !test.expectsCall {
t.Fatalf("should not have been called")
}
if !test.expectsCall {
return
}
if digest != test.api.responseDigest {
t.Errorf("unexpected content digest: %q != %q", digest, test.api.responseDigest)
}
// validating that the mock got the right parameters
if test.api.sessionID != sessionID {
t.Errorf("different session ID: %s != %s", test.api.sessionID, sessionID)
}
for _, d := range deep.Equal(test.api.model, test.dockerfile) {
t.Errorf("model difference: %s", d)
}
})
}
}
-50
View File
@@ -1,50 +0,0 @@
//nolint:dupl
package anchore
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/log"
)
type manifestImportAPI interface {
ImportImageManifest(ctx context.Context, sessionID string, contents interface{}) (external.ImageImportContentResponse, *http.Response, error)
}
func importManifest(ctx context.Context, api manifestImportAPI, sessionID string, manifest []byte, stage *progress.Stage) (string, error) {
if len(manifest) > 0 {
log.Debug("importing image manifest")
stage.Current = "image manifest"
// API requires an object, but we do not verify the shape of this object locally
var sender map[string]interface{}
if err := json.Unmarshal(manifest, &sender); err != nil {
return "", err
}
response, httpResponse, err := api.ImportImageManifest(ctx, sessionID, sender)
if err != nil {
var openAPIErr external.GenericOpenAPIError
if errors.As(err, &openAPIErr) {
log.Errorf("api response: %+v", string(openAPIErr.Body()))
}
return "", fmt.Errorf("unable to import Manifest: %w", err)
}
defer httpResponse.Body.Close()
if httpResponse.StatusCode != 200 {
return "", fmt.Errorf("unable to import Manifest: %s", httpResponse.Status)
}
return response.Digest, nil
}
return "", nil
}
-129
View File
@@ -1,129 +0,0 @@
package anchore
import (
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"testing"
"github.com/docker/docker/pkg/ioutils"
"github.com/go-test/deep"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
)
type mockManifestImportAPI struct {
sessionID string
model interface{}
httpResponse *http.Response
err error
ctx context.Context
responseDigest string
wasCalled bool
}
func (m *mockManifestImportAPI) ImportImageManifest(ctx context.Context, sessionID string, contents interface{}) (external.ImageImportContentResponse, *http.Response, error) {
m.wasCalled = true
m.model = contents
m.sessionID = sessionID
m.ctx = ctx
if m.httpResponse == nil {
m.httpResponse = &http.Response{}
}
m.httpResponse.Body = ioutils.NewReadCloserWrapper(strings.NewReader(""), func() error { return nil })
return external.ImageImportContentResponse{Digest: m.responseDigest}, m.httpResponse, m.err
}
func TestManifestImport(t *testing.T) {
sessionID := "my-session"
tests := []struct {
name string
manifest string
api *mockManifestImportAPI
expectsError bool
expectsCall bool
}{
{
name: "Go case: import works",
manifest: `{ "key": "the-config-contents!" }`,
api: &mockManifestImportAPI{
httpResponse: &http.Response{StatusCode: 200},
responseDigest: "digest!",
},
expectsCall: true,
},
{
name: "No manifest provided",
manifest: "",
api: &mockManifestImportAPI{},
expectsCall: false,
},
{
name: "API returns an error",
manifest: `{ "key": "the-config-contents!" }`,
api: &mockManifestImportAPI{
err: fmt.Errorf("api error, something went wrong"),
},
expectsError: true,
expectsCall: true,
},
{
name: "API HTTP-level error",
manifest: `{ "key": "the-config-contents!" }`,
api: &mockManifestImportAPI{
httpResponse: &http.Response{StatusCode: 404},
},
expectsError: true,
expectsCall: true,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
digest, err := importManifest(context.TODO(), test.api, sessionID, []byte(test.manifest), &progress.Stage{})
// validate error handling
if err != nil && !test.expectsError {
t.Fatalf("did not expect an error, but got: %+v", err)
} else if err == nil && test.expectsError {
t.Fatalf("did expect an error, but got none")
}
if !test.api.wasCalled && test.expectsCall {
t.Fatalf("was not called!")
} else if test.api.wasCalled && !test.expectsCall {
t.Fatalf("should not have been called")
}
if !test.expectsCall {
return
}
if digest != test.api.responseDigest {
t.Errorf("unexpected content digest: %q != %q", digest, test.api.responseDigest)
}
// validating that the mock got the right parameters
if test.api.sessionID != sessionID {
t.Errorf("different session ID: %s != %s", test.api.sessionID, sessionID)
}
var expected map[string]interface{}
if err := json.Unmarshal([]byte(test.manifest), &expected); err != nil {
t.Fatalf("could not unmarshal expected results")
}
for _, d := range deep.Equal(test.api.model, expected) {
t.Errorf("model difference: %s", d)
}
})
}
}
-119
View File
@@ -1,119 +0,0 @@
package anchore
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/formats/syftjson"
syftjsonModel "github.com/anchore/syft/syft/formats/syftjson/model"
"github.com/anchore/syft/syft/sbom"
)
type packageSBOMImportAPI interface {
ImportImagePackages(context.Context, string, external.ImagePackageManifest) (external.ImageImportContentResponse, *http.Response, error)
}
// importSBOM mirrors all elements found on the syftjson model format object relative to the anchore engine import schema.
type importSBOM struct {
Artifacts []syftjsonModel.Package `json:"artifacts"` // Artifacts is the list of packages discovered and placed into the catalog
ArtifactRelationships []syftjsonModel.Relationship `json:"artifactRelationships"`
Files []syftjsonModel.File `json:"files,omitempty"` // note: must have omitempty
Secrets []syftjsonModel.Secrets `json:"secrets,omitempty"` // note: must have omitempty
Source syftjsonModel.Source `json:"source"` // Source represents the original object that was cataloged
Distro external.ImportDistribution `json:"distro"` // Distro represents the Linux distribution that was detected from the source
Descriptor syftjsonModel.Descriptor `json:"descriptor"` // Descriptor is a block containing self-describing information about syft
Schema syftjsonModel.Schema `json:"schema"` // Schema is a block reserved for defining the version for the shape of this JSON document and where to find the schema document to validate the shape
}
// toImportSBOMModel transforms the current sbom shape into what is needed for the current anchore import api shape.
func toImportSBOMModel(s sbom.SBOM) importSBOM {
m := syftjson.ToFormatModel(s)
var idLike string
if len(m.Distro.IDLike) > 0 {
idLike = m.Distro.IDLike[0]
}
var version = m.Distro.VersionID // note: version is intentionally not used as the default
if version == "" {
version = m.Distro.Version
}
var name = m.Distro.ID // note: name is intentionally not used as the default
if name == "" {
name = m.Distro.Name
}
return importSBOM{
Artifacts: m.Artifacts,
ArtifactRelationships: m.ArtifactRelationships,
Files: m.Files,
Secrets: m.Secrets,
Source: m.Source,
Distro: external.ImportDistribution{
Name: name,
Version: version,
IdLike: idLike,
},
Descriptor: m.Descriptor,
Schema: m.Schema,
}
}
func packageSbomModel(s sbom.SBOM) (*external.ImagePackageManifest, error) {
var buf bytes.Buffer
doc := toImportSBOMModel(s)
enc := json.NewEncoder(&buf)
// prevent > and < from being escaped in the payload
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
if err := enc.Encode(&doc); err != nil {
return nil, fmt.Errorf("unable to encode import JSON model: %w", err)
}
// the model is 1:1 the JSON output of today. As the schema changes, this will need to be converted into individual mappings.
var model external.ImagePackageManifest
if err := json.Unmarshal(buf.Bytes(), &model); err != nil {
return nil, fmt.Errorf("unable to convert JSON output to import model: %w", err)
}
return &model, nil
}
func importPackageSBOM(ctx context.Context, api packageSBOMImportAPI, sessionID string, s sbom.SBOM, stage *progress.Stage) (string, error) {
log.Debug("importing package SBOM")
stage.Current = "package SBOM"
model, err := packageSbomModel(s)
if err != nil {
return "", fmt.Errorf("unable to create PackageSBOM model: %w", err)
}
response, httpResponse, err := api.ImportImagePackages(ctx, sessionID, *model)
if err != nil {
var openAPIErr external.GenericOpenAPIError
if errors.As(err, &openAPIErr) {
log.Errorf("api response: %+v", string(openAPIErr.Body()))
}
return "", fmt.Errorf("unable to import PackageSBOM: %w", err)
}
defer httpResponse.Body.Close()
if httpResponse.StatusCode != 200 {
return "", fmt.Errorf("unable to import PackageSBOM: %s", httpResponse.Status)
}
return response.Digest, nil
}
@@ -1,398 +0,0 @@
package anchore
import (
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"testing"
"github.com/docker/docker/pkg/ioutils"
"github.com/go-test/deep"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
func must(c pkg.CPE, e error) pkg.CPE {
if e != nil {
panic(e)
}
return c
}
type mockPackageSBOMImportAPI struct {
sessionID string
model external.ImagePackageManifest
httpResponse *http.Response
err error
ctx context.Context
responseDigest string
}
func (m *mockPackageSBOMImportAPI) ImportImagePackages(ctx context.Context, sessionID string, model external.ImagePackageManifest) (external.ImageImportContentResponse, *http.Response, error) {
m.model = model
m.sessionID = sessionID
m.ctx = ctx
if m.httpResponse == nil {
m.httpResponse = &http.Response{}
}
m.httpResponse.Body = ioutils.NewReadCloserWrapper(strings.NewReader(""), func() error { return nil })
return external.ImageImportContentResponse{Digest: m.responseDigest}, m.httpResponse, m.err
}
func sbomFixture() sbom.SBOM {
return sbom.SBOM{
Artifacts: sbom.Artifacts{
PackageCatalog: pkg.NewCatalog(pkg.Package{
Name: "name",
Version: "version",
FoundBy: "foundBy",
Locations: source.NewLocationSet(
source.Location{
Coordinates: source.Coordinates{
RealPath: "path",
FileSystemID: "layerID",
},
},
),
Licenses: []string{"license"},
Language: pkg.Python,
Type: pkg.PythonPkg,
CPEs: []pkg.CPE{
must(pkg.NewCPE("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*")),
},
PURL: "purl",
MetadataType: pkg.PythonPackageMetadataType,
Metadata: pkg.PythonPackageMetadata{
Name: "p-name",
Version: "p-version",
License: "p-license",
Author: "p-author",
AuthorEmail: "p-email",
Platform: "p-platform",
Files: []pkg.PythonFileRecord{
{
Path: "p-path",
Digest: &pkg.PythonFileDigest{
Algorithm: "p-alg",
Value: "p-digest",
},
Size: "p-size",
},
},
SitePackagesRootPath: "p-site-packages-root",
TopLevelPackages: []string{"top-level"},
},
}),
LinuxDistribution: &linux.Release{
ID: "centos",
Version: "8.0",
VersionID: "8.0",
IDLike: []string{"rhel"},
},
},
Relationships: []artifact.Relationship{
{
From: source.NewLocation("/place1"),
To: source.NewLocation("/place2"),
Type: artifact.ContainsRelationship,
},
},
Source: source.Metadata{
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "user-in",
Layers: nil,
Size: 10,
ManifestDigest: "sha256:digest!",
MediaType: "mediatype!",
Tags: nil,
},
},
}
}
func TestPackageSbomImport(t *testing.T) {
sbomResult := sbomFixture()
theModel, err := packageSbomModel(sbomResult)
if err != nil {
t.Fatalf("could not get sbom model: %+v", err)
}
sessionID := "my-session"
tests := []struct {
name string
api *mockPackageSBOMImportAPI
expectsError bool
}{
{
name: "Go case: import works",
api: &mockPackageSBOMImportAPI{
httpResponse: &http.Response{StatusCode: 200},
responseDigest: "digest!",
},
},
{
name: "API returns an error",
api: &mockPackageSBOMImportAPI{
err: fmt.Errorf("API error, something went wrong."),
},
expectsError: true,
},
{
name: "API HTTP-level error",
api: &mockPackageSBOMImportAPI{
httpResponse: &http.Response{StatusCode: 404},
},
expectsError: true,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
digest, err := importPackageSBOM(context.TODO(), test.api, sessionID, sbomResult, &progress.Stage{})
// validate error handling
if err != nil && !test.expectsError {
t.Fatalf("did not expect an error, but got: %+v", err)
} else if err == nil && test.expectsError {
t.Fatalf("did expect an error, but got none")
}
if digest != test.api.responseDigest {
t.Errorf("unexpected content digest: %q != %q", digest, test.api.responseDigest)
}
// validating that the mock got the right parameters (api.ImportImagePackages)
if test.api.sessionID != sessionID {
t.Errorf("different session ID: %s != %s", test.api.sessionID, sessionID)
}
for _, d := range deep.Equal(&test.api.model, theModel) {
t.Errorf("model difference: %s", d)
}
})
}
}
type modelAssertion func(t *testing.T, model *external.ImagePackageManifest)
func Test_packageSbomModel(t *testing.T) {
fix := sbomFixture()
tests := []struct {
name string
sbom sbom.SBOM
traits []modelAssertion
}{
{
name: "distro: has single distro id-like",
sbom: sbom.SBOM{
Artifacts: sbom.Artifacts{
LinuxDistribution: &linux.Release{
Name: "centos-name",
ID: "centos-id",
IDLike: []string{
"centos-id-like-1",
},
Version: "version",
VersionID: "version-id",
},
},
},
traits: []modelAssertion{
hasDistroInfo("centos-id", "version-id", "centos-id-like-1"),
},
},
{
name: "distro: has multiple distro id-like",
sbom: sbom.SBOM{
Artifacts: sbom.Artifacts{
LinuxDistribution: &linux.Release{
Name: "centos-name",
ID: "centos-id",
IDLike: []string{
"centos-id-like-1",
"centos-id-like-2",
},
Version: "version",
VersionID: "version-id",
},
},
},
traits: []modelAssertion{
hasDistroInfo("centos-id", "version-id", "centos-id-like-1"),
},
},
{
name: "distro: has no distro id-like",
sbom: sbom.SBOM{
Artifacts: sbom.Artifacts{
LinuxDistribution: &linux.Release{
Name: "centos-name",
ID: "centos-id",
IDLike: []string{},
Version: "version",
VersionID: "version-id",
},
},
},
traits: []modelAssertion{
hasDistroInfo("centos-id", "version-id", ""),
},
},
{
name: "distro: has no version-id",
sbom: sbom.SBOM{
Artifacts: sbom.Artifacts{
LinuxDistribution: &linux.Release{
Name: "centos-name",
ID: "centos-id",
IDLike: []string{},
Version: "version",
VersionID: "",
},
},
},
traits: []modelAssertion{
hasDistroInfo("centos-id", "version", ""),
},
},
{
name: "distro: has no id",
sbom: sbom.SBOM{
Artifacts: sbom.Artifacts{
LinuxDistribution: &linux.Release{
Name: "centos-name",
ID: "",
IDLike: []string{},
Version: "version",
VersionID: "version-id",
},
},
},
traits: []modelAssertion{
hasDistroInfo("centos-name", "version-id", ""),
},
},
{
name: "should have expected packages",
sbom: fix,
traits: []modelAssertion{
func(t *testing.T, model *external.ImagePackageManifest) {
require.Len(t, model.Artifacts, 1)
modelPkg := model.Artifacts
modelBytes, err := json.Marshal(&modelPkg)
require.NoError(t, err)
fixPkg := syftjson.ToFormatModel(fix).Artifacts
fixBytes, err := json.Marshal(&fixPkg)
require.NoError(t, err)
assert.JSONEq(t, string(fixBytes), string(modelBytes))
},
},
},
{
name: "should have expected relationships",
sbom: fix,
traits: []modelAssertion{
func(t *testing.T, model *external.ImagePackageManifest) {
modelPkg := model.ArtifactRelationships
modelBytes, err := json.Marshal(&modelPkg)
require.NoError(t, err)
fixPkg := syftjson.ToFormatModel(fix).ArtifactRelationships
fixBytes, err := json.Marshal(&fixPkg)
require.NoError(t, err)
assert.JSONEq(t, string(fixBytes), string(modelBytes))
},
},
},
{
name: "should have expected schema",
sbom: fix,
traits: []modelAssertion{
func(t *testing.T, model *external.ImagePackageManifest) {
modelPkg := model.Schema
modelBytes, err := json.Marshal(&modelPkg)
require.NoError(t, err)
fixPkg := syftjson.ToFormatModel(fix).Schema
fixBytes, err := json.Marshal(&fixPkg)
require.NoError(t, err)
assert.JSONEq(t, string(fixBytes), string(modelBytes))
},
},
},
{
name: "should have expected descriptor",
sbom: fix,
traits: []modelAssertion{
func(t *testing.T, model *external.ImagePackageManifest) {
modelPkg := model.Descriptor
modelBytes, err := json.Marshal(&modelPkg)
require.NoError(t, err)
fixPkg := syftjson.ToFormatModel(fix).Descriptor
fixBytes, err := json.Marshal(&fixPkg)
require.NoError(t, err)
assert.JSONEq(t, string(fixBytes), string(modelBytes))
},
},
},
{
name: "should have expected source",
sbom: fix,
traits: []modelAssertion{
func(t *testing.T, model *external.ImagePackageManifest) {
modelPkg := model.Source
modelBytes, err := json.Marshal(&modelPkg)
require.NoError(t, err)
fixPkg := syftjson.ToFormatModel(fix).Source
fixBytes, err := json.Marshal(&fixPkg)
require.NoError(t, err)
assert.JSONEq(t, string(fixBytes), string(modelBytes))
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := packageSbomModel(tt.sbom)
require.NoError(t, err)
for _, fn := range tt.traits {
fn(t, got)
}
})
}
}
func hasDistroInfo(name, version, idLike string) modelAssertion {
return func(t *testing.T, model *external.ImagePackageManifest) {
assert.Equal(t, name, model.Distro.Name)
assert.Equal(t, version, model.Distro.Version)
assert.Equal(t, idLike, model.Distro.IdLike)
}
}
-21
View File
@@ -1,21 +0,0 @@
package config
import "github.com/spf13/viper"
type anchore struct {
// upload options
Host string `yaml:"host" json:"host" mapstructure:"host"` // -H , hostname of the engine/enterprise instance to upload to (setting this value enables upload)
Path string `yaml:"path" json:"path" mapstructure:"path"` // override the engine/enterprise API upload path
// IMPORTANT: do not show the username in any YAML/JSON output (sensitive information)
Username string `yaml:"-" json:"-" mapstructure:"username"` // -u , username to authenticate upload
// IMPORTANT: do not show the password in any YAML/JSON output (sensitive information)
Password string `yaml:"-" json:"-" mapstructure:"password"` // -p , password to authenticate upload
Dockerfile string `yaml:"dockerfile" json:"dockerfile" mapstructure:"dockerfile"` // -d , dockerfile to attach for upload
OverwriteExistingImage bool `yaml:"overwrite-existing-image" json:"overwrite-existing-image" mapstructure:"overwrite-existing-image"` // --overwrite-existing-image , if any of the SBOM components have already been uploaded this flag will ensure they are overwritten with the current upload
ImportTimeout uint `yaml:"import-timeout" json:"import-timeout" mapstructure:"import-timeout"` // --import-timeout
// , customize the number of seconds within which the SBOM import must be completed or canceled
}
func (cfg anchore) loadDefaultValues(v *viper.Viper) {
v.SetDefault("anchore.path", "")
}
+15 -31
View File
@@ -10,10 +10,10 @@ import (
"github.com/adrg/xdg"
"github.com/mitchellh/go-homedir"
"github.com/sirupsen/logrus"
"github.com/spf13/viper"
"gopkg.in/yaml.v2"
"github.com/anchore/go-logger"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/pkg/cataloger"
@@ -43,7 +43,6 @@ type Application struct {
OutputTemplatePath string `yaml:"output-template-path" json:"output-template-path" mapstructure:"output-template-path"` // -t template file to use for output
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Anchore anchore `yaml:"anchore" json:"anchore" mapstructure:"anchore"` // options for interacting with Anchore Engine/Enterprise
Dev development `yaml:"dev" json:"dev" mapstructure:"dev"`
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Catalogers []string `yaml:"catalogers" json:"catalogers" mapstructure:"catalogers"`
@@ -56,6 +55,7 @@ type Application struct {
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
Name string `yaml:"name" json:"name" mapstructure:"name"`
}
func (cfg Application) ToCatalogerConfig() cataloger.Config {
@@ -75,9 +75,11 @@ func (cfg *Application) LoadAllValues(v *viper.Viper, configPath string) error {
// check if user specified config; otherwise read all possible paths
if err := loadConfig(v, configPath); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); ok {
// Not Found; ignore this error
log.Debug("no config file found; proceeding with defaults")
var notFound *viper.ConfigFileNotFoundError
if errors.As(err, &notFound) {
log.Debugf("no config file found, using defaults")
} else {
return fmt.Errorf("unable to load config: %w", err)
}
}
@@ -112,7 +114,6 @@ func (cfg *Application) parseConfigValues() error {
// parse application config options
for _, optionFn := range []func() error{
cfg.parseUploadOptions,
cfg.parseLogLevelOption,
cfg.parseFile,
} {
@@ -136,46 +137,29 @@ func (cfg *Application) parseConfigValues() error {
return nil
}
func (cfg *Application) parseUploadOptions() error {
if cfg.Anchore.Host == "" && cfg.Anchore.Dockerfile != "" {
return fmt.Errorf("cannot provide dockerfile option without enabling upload")
}
return nil
}
func (cfg *Application) parseLogLevelOption() error {
switch {
case cfg.Quiet:
// TODO: this is bad: quiet option trumps all other logging options (such as to a file on disk)
// we should be able to quiet the console logging and leave file logging alone...
// ... this will be an enhancement for later
cfg.Log.LevelOpt = logrus.PanicLevel
cfg.Log.Level = logger.DisabledLevel
case cfg.Verbosity > 0:
switch v := cfg.Verbosity; {
case v == 1:
cfg.Log.LevelOpt = logrus.InfoLevel
case v >= 2:
cfg.Log.LevelOpt = logrus.DebugLevel
default:
cfg.Log.LevelOpt = logrus.ErrorLevel
}
cfg.Log.Level = logger.LevelFromVerbosity(int(cfg.Verbosity), logger.WarnLevel, logger.InfoLevel, logger.DebugLevel, logger.TraceLevel)
case cfg.Log.Level != "":
lvl, err := logrus.ParseLevel(strings.ToLower(cfg.Log.Level))
var err error
cfg.Log.Level, err = logger.LevelFromString(string(cfg.Log.Level))
if err != nil {
return fmt.Errorf("bad log level configured (%q): %w", cfg.Log.Level, err)
return err
}
cfg.Log.LevelOpt = lvl
if cfg.Log.LevelOpt >= logrus.InfoLevel {
if logger.IsVerbose(cfg.Log.Level) {
cfg.Verbosity = 1
}
default:
cfg.Log.LevelOpt = logrus.WarnLevel
}
if cfg.Log.Level == "" {
cfg.Log.Level = cfg.Log.LevelOpt.String()
cfg.Log.Level = logger.WarnLevel
}
return nil
-1
View File
@@ -1 +0,0 @@
package config
+4 -3
View File
@@ -4,15 +4,15 @@ import (
"fmt"
"github.com/mitchellh/go-homedir"
"github.com/sirupsen/logrus"
"github.com/spf13/viper"
"github.com/anchore/go-logger"
)
// logging contains all logging-related configuration options available to the user via the application config.
type logging struct {
Structured bool `yaml:"structured" json:"structured" mapstructure:"structured"` // show all log entries as JSON formatted strings
LevelOpt logrus.Level `yaml:"-" json:"-"` // the native log level object used by the logger
Level string `yaml:"level" json:"level" mapstructure:"level"` // the log level string hint
Level logger.Level `yaml:"level" json:"level" mapstructure:"level"` // the log level string hint
FileLocation string `yaml:"file" json:"file-location" mapstructure:"file"` // the file path to write logs to
}
@@ -30,4 +30,5 @@ func (cfg *logging) parseConfigValues() error {
func (cfg logging) loadDefaultValues(v *viper.Viper) {
v.SetDefault("log.structured", false)
v.SetDefault("log.file", "")
v.SetDefault("log.level", string(logger.WarnLevel))
}
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "4.0.0"
JSONSchemaVersion = "6.0.0"
)
+6 -5
View File
@@ -1,6 +1,7 @@
package internal
import (
"errors"
"fmt"
"io"
"os"
@@ -26,14 +27,14 @@ func (e ErrPath) Error() string {
}
func IsErrPath(err error) bool {
_, ok := err.(ErrPath)
return ok
var pathErr ErrPath
return errors.As(err, &pathErr)
}
func IsErrPathPermission(err error) bool {
pathErr, ok := err.(ErrPath)
if ok {
var pathErr ErrPath
if errors.As(err, &pathErr) {
return os.IsPermission(pathErr.Err)
}
return ok
return false
}
+1 -1
View File
@@ -106,7 +106,7 @@ func findArchiveStartOffset(r io.ReaderAt, size int64) (startOfArchive uint64, e
bLen = size
}
buf = make([]byte, int(bLen))
if _, err := r.ReadAt(buf, size-bLen); err != nil && err != io.EOF {
if _, err := r.ReadAt(buf, size-bLen); err != nil && !errors.Is(err, io.EOF) {
return 0, err
}
if p := findSignatureInBlock(buf); p >= 0 {
+25 -2
View File
@@ -3,10 +3,13 @@ Package log contains the singleton object and helper functions for facilitating
*/
package log
import "github.com/anchore/syft/syft/logger"
import (
"github.com/anchore/go-logger"
"github.com/anchore/go-logger/adapter/discard"
)
// Log is the singleton used to facilitate logging internally within syft
var Log logger.Logger = &nopLogger{}
var Log logger.Logger = discard.New()
// Errorf takes a formatted template string and template arguments for the error logging level.
func Errorf(format string, args ...interface{}) {
@@ -47,3 +50,23 @@ func Debugf(format string, args ...interface{}) {
func Debug(args ...interface{}) {
Log.Debug(args...)
}
// Tracef takes a formatted template string and template arguments for the trace logging level.
func Tracef(format string, args ...interface{}) {
Log.Tracef(format, args...)
}
// Trace logs the given arguments at the trace logging level.
func Trace(args ...interface{}) {
Log.Trace(args...)
}
// WithFields returns a message logger with multiple key-value fields.
func WithFields(fields ...interface{}) logger.MessageLogger {
return Log.WithFields(fields...)
}
// Nested returns a new logger with hard coded key-value pairs
func Nested(fields ...interface{}) logger.Logger {
return Log.Nested(fields...)
}
-12
View File
@@ -1,12 +0,0 @@
package log
type nopLogger struct{}
func (l *nopLogger) Errorf(format string, args ...interface{}) {}
func (l *nopLogger) Error(args ...interface{}) {}
func (l *nopLogger) Warnf(format string, args ...interface{}) {}
func (l *nopLogger) Warn(args ...interface{}) {}
func (l *nopLogger) Infof(format string, args ...interface{}) {}
func (l *nopLogger) Info(args ...interface{}) {}
func (l *nopLogger) Debugf(format string, args ...interface{}) {}
func (l *nopLogger) Debug(args ...interface{}) {}
-4
View File
@@ -1,4 +0,0 @@
/*
Package logger contains implementations for the syft.logger.Logger interface.
*/
package logger
-163
View File
@@ -1,163 +0,0 @@
package logger
import (
"fmt"
"io"
"io/fs"
"os"
"github.com/sirupsen/logrus"
prefixed "github.com/x-cray/logrus-prefixed-formatter"
)
const defaultLogFilePermissions fs.FileMode = 0644
// LogrusConfig contains all configurable values for the Logrus logger
type LogrusConfig struct {
EnableConsole bool
EnableFile bool
Structured bool
Level logrus.Level
FileLocation string
}
// LogrusLogger contains all runtime values for using Logrus with the configured output target and input configuration values.
type LogrusLogger struct {
Config LogrusConfig
Logger *logrus.Logger
Output io.Writer
}
// LogrusNestedLogger is a wrapper for Logrus to enable nested logging configuration (loggers that always attach key-value pairs to all log entries)
type LogrusNestedLogger struct {
Logger *logrus.Entry
}
// NewLogrusLogger creates a new LogrusLogger with the given configuration
func NewLogrusLogger(cfg LogrusConfig) *LogrusLogger {
appLogger := logrus.New()
var output io.Writer
switch {
case cfg.EnableConsole && cfg.EnableFile:
logFile, err := os.OpenFile(cfg.FileLocation, os.O_WRONLY|os.O_CREATE, defaultLogFilePermissions)
if err != nil {
panic(fmt.Errorf("unable to setup log file: %w", err))
}
output = io.MultiWriter(os.Stderr, logFile)
case cfg.EnableConsole:
output = os.Stderr
case cfg.EnableFile:
logFile, err := os.OpenFile(cfg.FileLocation, os.O_WRONLY|os.O_CREATE, defaultLogFilePermissions)
if err != nil {
panic(fmt.Errorf("unable to setup log file: %w", err))
}
output = logFile
default:
output = io.Discard
}
appLogger.SetOutput(output)
appLogger.SetLevel(cfg.Level)
if cfg.Structured {
appLogger.SetFormatter(&logrus.JSONFormatter{
TimestampFormat: "2006-01-02 15:04:05",
DisableTimestamp: false,
DisableHTMLEscape: false,
PrettyPrint: false,
})
} else {
appLogger.SetFormatter(&prefixed.TextFormatter{
TimestampFormat: "2006-01-02 15:04:05",
ForceColors: true,
ForceFormatting: true,
})
}
return &LogrusLogger{
Config: cfg,
Logger: appLogger,
Output: output,
}
}
// Debugf takes a formatted template string and template arguments for the debug logging level.
func (l *LogrusLogger) Debugf(format string, args ...interface{}) {
l.Logger.Debugf(format, args...)
}
// Infof takes a formatted template string and template arguments for the info logging level.
func (l *LogrusLogger) Infof(format string, args ...interface{}) {
l.Logger.Infof(format, args...)
}
// Warnf takes a formatted template string and template arguments for the warning logging level.
func (l *LogrusLogger) Warnf(format string, args ...interface{}) {
l.Logger.Warnf(format, args...)
}
// Errorf takes a formatted template string and template arguments for the error logging level.
func (l *LogrusLogger) Errorf(format string, args ...interface{}) {
l.Logger.Errorf(format, args...)
}
// Debug logs the given arguments at the debug logging level.
func (l *LogrusLogger) Debug(args ...interface{}) {
l.Logger.Debug(args...)
}
// Info logs the given arguments at the info logging level.
func (l *LogrusLogger) Info(args ...interface{}) {
l.Logger.Info(args...)
}
// Warn logs the given arguments at the warning logging level.
func (l *LogrusLogger) Warn(args ...interface{}) {
l.Logger.Warn(args...)
}
// Error logs the given arguments at the error logging level.
func (l *LogrusLogger) Error(args ...interface{}) {
l.Logger.Error(args...)
}
// Debugf takes a formatted template string and template arguments for the debug logging level.
func (l *LogrusNestedLogger) Debugf(format string, args ...interface{}) {
l.Logger.Debugf(format, args...)
}
// Infof takes a formatted template string and template arguments for the info logging level.
func (l *LogrusNestedLogger) Infof(format string, args ...interface{}) {
l.Logger.Infof(format, args...)
}
// Warnf takes a formatted template string and template arguments for the warning logging level.
func (l *LogrusNestedLogger) Warnf(format string, args ...interface{}) {
l.Logger.Warnf(format, args...)
}
// Errorf takes a formatted template string and template arguments for the error logging level.
func (l *LogrusNestedLogger) Errorf(format string, args ...interface{}) {
l.Logger.Errorf(format, args...)
}
// Debug logs the given arguments at the debug logging level.
func (l *LogrusNestedLogger) Debug(args ...interface{}) {
l.Logger.Debug(args...)
}
// Info logs the given arguments at the info logging level.
func (l *LogrusNestedLogger) Info(args ...interface{}) {
l.Logger.Info(args...)
}
// Warn logs the given arguments at the warning logging level.
func (l *LogrusNestedLogger) Warn(args ...interface{}) {
l.Logger.Warn(args...)
}
// Error logs the given arguments at the error logging level.
func (l *LogrusNestedLogger) Error(args ...interface{}) {
l.Logger.Error(args...)
}
+1
View File
@@ -59,6 +59,7 @@ var (
"application/x-elf",
"application/x-sharedlib",
"application/vnd.microsoft.portable-executable",
"application/x-executable",
}...,
)
)
@@ -47,12 +47,12 @@ func main() {
func run() error {
resp, err := http.Get(url)
if err != nil {
return fmt.Errorf("unable to get licenses list: %+v", err)
return fmt.Errorf("unable to get licenses list: %w", err)
}
var result LicenseList
if err = json.NewDecoder(resp.Body).Decode(&result); err != nil {
return fmt.Errorf("unable to decode license list: %+v", err)
return fmt.Errorf("unable to decode license list: %w", err)
}
defer func() {
if err := resp.Body.Close(); err != nil {
@@ -62,7 +62,7 @@ func run() error {
f, err := os.Create(source)
if err != nil {
return fmt.Errorf("unable to create %q: %+v", source, err)
return fmt.Errorf("unable to create %q: %w", source, err)
}
defer func() {
if err := f.Close(); err != nil {
@@ -85,7 +85,7 @@ func run() error {
})
if err != nil {
return fmt.Errorf("unable to generate template: %+v", err)
return fmt.Errorf("unable to generate template: %w", err)
}
return nil
}
@@ -22,18 +22,18 @@ func TestReplaceDeprecatedLicenses(t *testing.T) {
}
expected := map[string]string{
"abc-1": "ABC-1.0-Only",
"abc-1-only": "ABC-1.0-Only",
"abc-1.0": "ABC-1.0-Only",
"abc-1.0.0": "ABC-1.0-Only",
"abc-1.0-only": "ABC-1.0-Only",
"abc-1.0.0-only": "ABC-1.0-Only",
"abc-1+": "ABC-1.0-Or-later",
"abc-1.0+": "ABC-1.0-Or-later",
"abc-1.0.0+": "ABC-1.0-Or-later",
"abc-1-or-later": "ABC-1.0-Or-later",
"abc-1.0-or-later": "ABC-1.0-Or-later",
"abc-1.0.0-or-later": "ABC-1.0-Or-later",
"abc-1": "ABC-1.0-only",
"abc-1-only": "ABC-1.0-only",
"abc-1.0": "ABC-1.0-only",
"abc-1.0.0": "ABC-1.0-only",
"abc-1.0-only": "ABC-1.0-only",
"abc-1.0.0-only": "ABC-1.0-only",
"abc-1+": "ABC-1.0-or-later",
"abc-1.0+": "ABC-1.0-or-later",
"abc-1.0.0+": "ABC-1.0-or-later",
"abc-1-or-later": "ABC-1.0-or-later",
"abc-1.0-or-later": "ABC-1.0-or-later",
"abc-1.0.0-or-later": "ABC-1.0-or-later",
"duh-1": "Duh-1.0",
"duh-1.0": "Duh-1.0",
"duh-1.0.0": "Duh-1.0",
@@ -79,22 +79,22 @@ func Test_processSPDXLicense(t *testing.T) {
"afl-3.0": "AFL-3.0",
"afl-3.0.0": "AFL-3.0",
"afmparse": "Afmparse",
"agpl-1": "AGPL-1.0",
"agpl-1": "AGPL-1.0-only",
"agpl-1-only": "AGPL-1.0-only",
"agpl-1-or-later": "AGPL-1.0-or-later",
"agpl-1.0": "AGPL-1.0",
"agpl-1.0": "AGPL-1.0-only",
"agpl-1.0-only": "AGPL-1.0-only",
"agpl-1.0-or-later": "AGPL-1.0-or-later",
"agpl-1.0.0": "AGPL-1.0",
"agpl-1.0.0": "AGPL-1.0-only",
"agpl-1.0.0-only": "AGPL-1.0-only",
"agpl-1.0.0-or-later": "AGPL-1.0-or-later",
"agpl-3": "AGPL-3.0",
"agpl-3": "AGPL-3.0-only",
"agpl-3-only": "AGPL-3.0-only",
"agpl-3-or-later": "AGPL-3.0-or-later",
"agpl-3.0": "AGPL-3.0",
"agpl-3.0": "AGPL-3.0-only",
"agpl-3.0-only": "AGPL-3.0-only",
"agpl-3.0-or-later": "AGPL-3.0-or-later",
"agpl-3.0.0": "AGPL-3.0",
"agpl-3.0.0": "AGPL-3.0-only",
"agpl-3.0.0-only": "AGPL-3.0-only",
"agpl-3.0.0-or-later": "AGPL-3.0-or-later",
"aladdin": "Aladdin",
@@ -160,18 +160,18 @@ func Test_processSPDXLicense(t *testing.T) {
"bsd-1.0-clause": "BSD-1-Clause",
"bsd-1.0.0-clause": "BSD-1-Clause",
"bsd-2-clause": "BSD-2-Clause",
"bsd-2-clause-freebsd": "BSD-2-Clause-FreeBSD",
"bsd-2-clause-netbsd": "BSD-2-Clause-NetBSD",
"bsd-2-clause-freebsd": "BSD-2-Clause-Views",
"bsd-2-clause-netbsd": "BSD-2-Clause",
"bsd-2-clause-patent": "BSD-2-Clause-Patent",
"bsd-2-clause-views": "BSD-2-Clause-Views",
"bsd-2.0-clause": "BSD-2-Clause",
"bsd-2.0-clause-freebsd": "BSD-2-Clause-FreeBSD",
"bsd-2.0-clause-netbsd": "BSD-2-Clause-NetBSD",
"bsd-2.0-clause-freebsd": "BSD-2-Clause-Views",
"bsd-2.0-clause-netbsd": "BSD-2-Clause",
"bsd-2.0-clause-patent": "BSD-2-Clause-Patent",
"bsd-2.0-clause-views": "BSD-2-Clause-Views",
"bsd-2.0.0-clause": "BSD-2-Clause",
"bsd-2.0.0-clause-freebsd": "BSD-2-Clause-FreeBSD",
"bsd-2.0.0-clause-netbsd": "BSD-2-Clause-NetBSD",
"bsd-2.0.0-clause-freebsd": "BSD-2-Clause-Views",
"bsd-2.0.0-clause-netbsd": "BSD-2-Clause",
"bsd-2.0.0-clause-patent": "BSD-2-Clause-Patent",
"bsd-2.0.0-clause-views": "BSD-2-Clause-Views",
"bsd-3-clause": "BSD-3-Clause",
@@ -221,9 +221,9 @@ func Test_processSPDXLicense(t *testing.T) {
"busl-1": "BUSL-1.1",
"busl-1.1": "BUSL-1.1",
"busl-1.1.0": "BUSL-1.1",
"bzip2-1": "bzip2-1.0.5",
"bzip2-1.0": "bzip2-1.0.5",
"bzip2-1.0.5": "bzip2-1.0.5",
"bzip2-1": "bzip2-1.0.6",
"bzip2-1.0": "bzip2-1.0.6",
"bzip2-1.0.5": "bzip2-1.0.6",
"bzip2-1.0.6": "bzip2-1.0.6",
"c-uda-1": "C-UDA-1.0",
"c-uda-1.0": "C-UDA-1.0",
@@ -252,16 +252,19 @@ func Test_processSPDXLicense(t *testing.T) {
"cc-by-3": "CC-BY-3.0",
"cc-by-3-at": "CC-BY-3.0-AT",
"cc-by-3-de": "CC-BY-3.0-DE",
"cc-by-3-igo": "CC-BY-3.0-IGO",
"cc-by-3-nl": "CC-BY-3.0-NL",
"cc-by-3-us": "CC-BY-3.0-US",
"cc-by-3.0": "CC-BY-3.0",
"cc-by-3.0-at": "CC-BY-3.0-AT",
"cc-by-3.0-de": "CC-BY-3.0-DE",
"cc-by-3.0-igo": "CC-BY-3.0-IGO",
"cc-by-3.0-nl": "CC-BY-3.0-NL",
"cc-by-3.0-us": "CC-BY-3.0-US",
"cc-by-3.0.0": "CC-BY-3.0",
"cc-by-3.0.0-at": "CC-BY-3.0-AT",
"cc-by-3.0.0-de": "CC-BY-3.0-DE",
"cc-by-3.0.0-igo": "CC-BY-3.0-IGO",
"cc-by-3.0.0-nl": "CC-BY-3.0-NL",
"cc-by-3.0.0-us": "CC-BY-3.0-US",
"cc-by-4": "CC-BY-4.0",
@@ -420,6 +423,7 @@ func Test_processSPDXLicense(t *testing.T) {
"cern-ohl-w-2": "CERN-OHL-W-2.0",
"cern-ohl-w-2.0": "CERN-OHL-W-2.0",
"cern-ohl-w-2.0.0": "CERN-OHL-W-2.0",
"checkmk": "checkmk",
"clartistic": "ClArtistic",
"cnri-jython": "CNRI-Jython",
"cnri-python": "CNRI-Python",
@@ -518,51 +522,52 @@ func Test_processSPDXLicense(t *testing.T) {
"fsfap": "FSFAP",
"fsful": "FSFUL",
"fsfullr": "FSFULLR",
"fsfullrwd": "FSFULLRWD",
"ftl": "FTL",
"gd": "GD",
"gfdl-1": "GFDL-1.1",
"gfdl-1": "GFDL-1.1-only",
"gfdl-1-invariants-only": "GFDL-1.1-invariants-only",
"gfdl-1-invariants-or-later": "GFDL-1.1-invariants-or-later",
"gfdl-1-no-invariants-only": "GFDL-1.1-no-invariants-only",
"gfdl-1-no-invariants-or-later": "GFDL-1.1-no-invariants-or-later",
"gfdl-1-only": "GFDL-1.1-only",
"gfdl-1-or-later": "GFDL-1.1-or-later",
"gfdl-1.1": "GFDL-1.1",
"gfdl-1.1": "GFDL-1.1-only",
"gfdl-1.1-invariants-only": "GFDL-1.1-invariants-only",
"gfdl-1.1-invariants-or-later": "GFDL-1.1-invariants-or-later",
"gfdl-1.1-no-invariants-only": "GFDL-1.1-no-invariants-only",
"gfdl-1.1-no-invariants-or-later": "GFDL-1.1-no-invariants-or-later",
"gfdl-1.1-only": "GFDL-1.1-only",
"gfdl-1.1-or-later": "GFDL-1.1-or-later",
"gfdl-1.1.0": "GFDL-1.1",
"gfdl-1.1.0": "GFDL-1.1-only",
"gfdl-1.1.0-invariants-only": "GFDL-1.1-invariants-only",
"gfdl-1.1.0-invariants-or-later": "GFDL-1.1-invariants-or-later",
"gfdl-1.1.0-no-invariants-only": "GFDL-1.1-no-invariants-only",
"gfdl-1.1.0-no-invariants-or-later": "GFDL-1.1-no-invariants-or-later",
"gfdl-1.1.0-only": "GFDL-1.1-only",
"gfdl-1.1.0-or-later": "GFDL-1.1-or-later",
"gfdl-1.2": "GFDL-1.2",
"gfdl-1.2": "GFDL-1.2-only",
"gfdl-1.2-invariants-only": "GFDL-1.2-invariants-only",
"gfdl-1.2-invariants-or-later": "GFDL-1.2-invariants-or-later",
"gfdl-1.2-no-invariants-only": "GFDL-1.2-no-invariants-only",
"gfdl-1.2-no-invariants-or-later": "GFDL-1.2-no-invariants-or-later",
"gfdl-1.2-only": "GFDL-1.2-only",
"gfdl-1.2-or-later": "GFDL-1.2-or-later",
"gfdl-1.2.0": "GFDL-1.2",
"gfdl-1.2.0": "GFDL-1.2-only",
"gfdl-1.2.0-invariants-only": "GFDL-1.2-invariants-only",
"gfdl-1.2.0-invariants-or-later": "GFDL-1.2-invariants-or-later",
"gfdl-1.2.0-no-invariants-only": "GFDL-1.2-no-invariants-only",
"gfdl-1.2.0-no-invariants-or-later": "GFDL-1.2-no-invariants-or-later",
"gfdl-1.2.0-only": "GFDL-1.2-only",
"gfdl-1.2.0-or-later": "GFDL-1.2-or-later",
"gfdl-1.3": "GFDL-1.3",
"gfdl-1.3": "GFDL-1.3-only",
"gfdl-1.3-invariants-only": "GFDL-1.3-invariants-only",
"gfdl-1.3-invariants-or-later": "GFDL-1.3-invariants-or-later",
"gfdl-1.3-no-invariants-only": "GFDL-1.3-no-invariants-only",
"gfdl-1.3-no-invariants-or-later": "GFDL-1.3-no-invariants-or-later",
"gfdl-1.3-only": "GFDL-1.3-only",
"gfdl-1.3-or-later": "GFDL-1.3-or-later",
"gfdl-1.3.0": "GFDL-1.3",
"gfdl-1.3.0": "GFDL-1.3-only",
"gfdl-1.3.0-invariants-only": "GFDL-1.3-invariants-only",
"gfdl-1.3.0-invariants-or-later": "GFDL-1.3-invariants-or-later",
"gfdl-1.3.0-no-invariants-only": "GFDL-1.3-no-invariants-only",
@@ -665,7 +670,7 @@ func Test_processSPDXLicense(t *testing.T) {
"jasper-2.0.0": "JasPer-2.0",
"jpnic": "JPNIC",
"json": "JSON",
"kicad-libraries-exception": "KiCad-libraries-exception",
"knuth-ctan": "Knuth-CTAN",
"lal-1": "LAL-1.2",
"lal-1.2": "LAL-1.2",
"lal-1.2.0": "LAL-1.2",
@@ -686,11 +691,11 @@ func Test_processSPDXLicense(t *testing.T) {
"lgpl-2.0.0-only": "LGPL-2.0-only",
"lgpl-2.0.0-or-later": "LGPL-2.0-or-later",
"lgpl-2.1": "LGPL-2.1-only",
"lgpl-2.1+": "LGPL-2.1+",
"lgpl-2.1+": "LGPL-2.1-or-later",
"lgpl-2.1-only": "LGPL-2.1-only",
"lgpl-2.1-or-later": "LGPL-2.1-or-later",
"lgpl-2.1.0": "LGPL-2.1-only",
"lgpl-2.1.0+": "LGPL-2.1+",
"lgpl-2.1.0+": "LGPL-2.1-or-later",
"lgpl-2.1.0-only": "LGPL-2.1-only",
"lgpl-2.1.0-or-later": "LGPL-2.1-or-later",
"lgpl-3": "LGPL-3.0-only",
@@ -714,6 +719,7 @@ func Test_processSPDXLicense(t *testing.T) {
"libselinux-1.0": "libselinux-1.0",
"libselinux-1.0.0": "libselinux-1.0",
"libtiff": "libtiff",
"libutil-david-nugent": "libutil-David-Nugent",
"liliq-p-1": "LiLiQ-P-1.1",
"liliq-p-1.1": "LiLiQ-P-1.1",
"liliq-p-1.1.0": "LiLiQ-P-1.1",
@@ -743,7 +749,14 @@ func Test_processSPDXLicense(t *testing.T) {
"lppl-1.3c": "LPPL-1.3c",
"lppl-1a": "LPPL-1.3a",
"lppl-1c": "LPPL-1.3c",
"lzma-sdk-9": "LZMA-SDK-9.22",
"lzma-sdk-9-to-9.20": "LZMA-SDK-9.11-to-9.20",
"lzma-sdk-9.11-to-9.20": "LZMA-SDK-9.11-to-9.20",
"lzma-sdk-9.11.0-to-9.20": "LZMA-SDK-9.11-to-9.20",
"lzma-sdk-9.22": "LZMA-SDK-9.22",
"lzma-sdk-9.22.0": "LZMA-SDK-9.22",
"makeindex": "MakeIndex",
"minpack": "Minpack",
"miros": "MirOS",
"mit": "MIT",
"mit-0": "MIT-0",
@@ -755,6 +768,7 @@ func Test_processSPDXLicense(t *testing.T) {
"mit-open-group": "MIT-open-group",
"mitnfa": "MITNFA",
"motosoto": "Motosoto",
"mpi-permissive": "mpi-permissive",
"mpich2": "mpich2",
"mpl-1": "MPL-1.0",
"mpl-1.0": "MPL-1.0",
@@ -768,6 +782,7 @@ func Test_processSPDXLicense(t *testing.T) {
"mpl-2.0.0": "MPL-2.0",
"mpl-2.0.0-no-copyleft-exception": "MPL-2.0-no-copyleft-exception",
"mplus": "mplus",
"ms-lpl": "MS-LPL",
"ms-pl": "MS-PL",
"ms-rl": "MS-RL",
"mtll": "MTLL",
@@ -797,6 +812,9 @@ func Test_processSPDXLicense(t *testing.T) {
"netcdf": "NetCDF",
"newsletr": "Newsletr",
"ngpl": "NGPL",
"nicta-1": "NICTA-1.0",
"nicta-1.0": "NICTA-1.0",
"nicta-1.0.0": "NICTA-1.0",
"nist-pd": "NIST-PD",
"nist-pd-fallback": "NIST-PD-fallback",
"nlod-1": "NLOD-1.0",
@@ -953,6 +971,7 @@ func Test_processSPDXLicense(t *testing.T) {
"python-2": "Python-2.0",
"python-2.0": "Python-2.0",
"python-2.0.0": "Python-2.0",
"python-2.0.1": "Python-2.0.1",
"qhull": "Qhull",
"qpl-1": "QPL-1.0",
"qpl-1.0": "QPL-1.0",
@@ -1020,7 +1039,7 @@ func Test_processSPDXLicense(t *testing.T) {
"sspl-1": "SSPL-1.0",
"sspl-1.0": "SSPL-1.0",
"sspl-1.0.0": "SSPL-1.0",
"standardml-nj": "StandardML-NJ",
"standardml-nj": "SMLNJ",
"sugarcrm-1": "SugarCRM-1.1.3",
"sugarcrm-1.1": "SugarCRM-1.1.3",
"sugarcrm-1.1.3": "SugarCRM-1.1.3",
+20 -1
View File
@@ -25,6 +25,25 @@ func (l License) canReplace(other License) bool {
return false
}
// We want to replace deprecated licenses with non-deprecated counterparts
// For more information, see: https://github.com/spdx/license-list-XML/issues/1676
if other.Deprecated {
switch {
case strings.ReplaceAll(l.ID, "-only", "") == other.ID:
return true
case strings.ReplaceAll(l.ID, "-or-later", "+") == other.ID:
return true
case l.ID == "BSD-2-Clause" && other.ID == "BSD-2-Clause-NetBSD":
return true
case l.ID == "BSD-2-Clause-Views" && other.ID == "BSD-2-Clause-FreeBSD":
return true
case l.ID == "bzip2-1.0.6" && other.ID == "bzip2-1.0.5":
return true
case l.ID == "SMLNJ" && other.ID == "StandardML-NJ":
return true
}
}
if l.Name != other.Name {
return false
}
@@ -43,7 +62,7 @@ func (l License) canReplace(other License) bool {
}
}
return l.ID != other.ID
return l.ID == other.ID
}
func (ll LicenseList) findReplacementLicense(deprecated License) *License {
@@ -15,7 +15,7 @@ var (
}
license2 = License{
ID: "ABC-1.0-Or-later",
ID: "ABC-1.0-or-later",
Name: "The ABC License 1.0",
}
@@ -26,7 +26,7 @@ var (
}
license4 = License{
ID: "ABC-1.0-Only",
ID: "ABC-1.0-only",
Name: "The ABC License 1.0 Only",
}
license5 = License{
@@ -53,7 +53,9 @@ func TestLicense_canReplace(t *testing.T) {
}
for _, tt := range tests {
assert.Equal(t, tt.expected, tt.l1.canReplace(tt.l2))
t.Run(tt.l1.ID+" - "+tt.l2.ID, func(t *testing.T) {
assert.Equal(t, tt.expected, tt.l1.canReplace(tt.l2))
})
}
}
File diff suppressed because it is too large Load Diff
+31 -27
View File
@@ -1,9 +1,9 @@
// Code generated by go generate; DO NOT EDIT.
// This file was generated by robots at 2022-08-16 13:37:43.053262 -0400 EDT m=+0.183888850
// This file was generated by robots at 2022-12-02 12:51:33.814385 -0500 EST m=+1.688731869
// using data from https://spdx.org/licenses/licenses.json
package spdxlicense
const Version = "3.18"
const Version = "3.19"
var licenseIDs = map[string]string{
"0bsd": "0BSD",
@@ -28,22 +28,22 @@ var licenseIDs = map[string]string{
"afl-3.0": "AFL-3.0",
"afl-3.0.0": "AFL-3.0",
"afmparse": "Afmparse",
"agpl-1": "AGPL-1.0",
"agpl-1": "AGPL-1.0-only",
"agpl-1-only": "AGPL-1.0-only",
"agpl-1-or-later": "AGPL-1.0-or-later",
"agpl-1.0": "AGPL-1.0",
"agpl-1.0": "AGPL-1.0-only",
"agpl-1.0-only": "AGPL-1.0-only",
"agpl-1.0-or-later": "AGPL-1.0-or-later",
"agpl-1.0.0": "AGPL-1.0",
"agpl-1.0.0": "AGPL-1.0-only",
"agpl-1.0.0-only": "AGPL-1.0-only",
"agpl-1.0.0-or-later": "AGPL-1.0-or-later",
"agpl-3": "AGPL-3.0",
"agpl-3": "AGPL-3.0-only",
"agpl-3-only": "AGPL-3.0-only",
"agpl-3-or-later": "AGPL-3.0-or-later",
"agpl-3.0": "AGPL-3.0",
"agpl-3.0": "AGPL-3.0-only",
"agpl-3.0-only": "AGPL-3.0-only",
"agpl-3.0-or-later": "AGPL-3.0-or-later",
"agpl-3.0.0": "AGPL-3.0",
"agpl-3.0.0": "AGPL-3.0-only",
"agpl-3.0.0-only": "AGPL-3.0-only",
"agpl-3.0.0-or-later": "AGPL-3.0-or-later",
"aladdin": "Aladdin",
@@ -109,18 +109,18 @@ var licenseIDs = map[string]string{
"bsd-1.0-clause": "BSD-1-Clause",
"bsd-1.0.0-clause": "BSD-1-Clause",
"bsd-2-clause": "BSD-2-Clause",
"bsd-2-clause-freebsd": "BSD-2-Clause-FreeBSD",
"bsd-2-clause-netbsd": "BSD-2-Clause-NetBSD",
"bsd-2-clause-freebsd": "BSD-2-Clause-Views",
"bsd-2-clause-netbsd": "BSD-2-Clause",
"bsd-2-clause-patent": "BSD-2-Clause-Patent",
"bsd-2-clause-views": "BSD-2-Clause-Views",
"bsd-2.0-clause": "BSD-2-Clause",
"bsd-2.0-clause-freebsd": "BSD-2-Clause-FreeBSD",
"bsd-2.0-clause-netbsd": "BSD-2-Clause-NetBSD",
"bsd-2.0-clause-freebsd": "BSD-2-Clause-Views",
"bsd-2.0-clause-netbsd": "BSD-2-Clause",
"bsd-2.0-clause-patent": "BSD-2-Clause-Patent",
"bsd-2.0-clause-views": "BSD-2-Clause-Views",
"bsd-2.0.0-clause": "BSD-2-Clause",
"bsd-2.0.0-clause-freebsd": "BSD-2-Clause-FreeBSD",
"bsd-2.0.0-clause-netbsd": "BSD-2-Clause-NetBSD",
"bsd-2.0.0-clause-freebsd": "BSD-2-Clause-Views",
"bsd-2.0.0-clause-netbsd": "BSD-2-Clause",
"bsd-2.0.0-clause-patent": "BSD-2-Clause-Patent",
"bsd-2.0.0-clause-views": "BSD-2-Clause-Views",
"bsd-3-clause": "BSD-3-Clause",
@@ -170,9 +170,9 @@ var licenseIDs = map[string]string{
"busl-1": "BUSL-1.1",
"busl-1.1": "BUSL-1.1",
"busl-1.1.0": "BUSL-1.1",
"bzip2-1": "bzip2-1.0.5",
"bzip2-1.0": "bzip2-1.0.5",
"bzip2-1.0.5": "bzip2-1.0.5",
"bzip2-1": "bzip2-1.0.6",
"bzip2-1.0": "bzip2-1.0.6",
"bzip2-1.0.5": "bzip2-1.0.6",
"bzip2-1.0.6": "bzip2-1.0.6",
"c-uda-1": "C-UDA-1.0",
"c-uda-1.0": "C-UDA-1.0",
@@ -372,6 +372,7 @@ var licenseIDs = map[string]string{
"cern-ohl-w-2": "CERN-OHL-W-2.0",
"cern-ohl-w-2.0": "CERN-OHL-W-2.0",
"cern-ohl-w-2.0.0": "CERN-OHL-W-2.0",
"checkmk": "checkmk",
"clartistic": "ClArtistic",
"cnri-jython": "CNRI-Jython",
"cnri-python": "CNRI-Python",
@@ -470,51 +471,52 @@ var licenseIDs = map[string]string{
"fsfap": "FSFAP",
"fsful": "FSFUL",
"fsfullr": "FSFULLR",
"fsfullrwd": "FSFULLRWD",
"ftl": "FTL",
"gd": "GD",
"gfdl-1": "GFDL-1.1",
"gfdl-1": "GFDL-1.1-only",
"gfdl-1-invariants-only": "GFDL-1.1-invariants-only",
"gfdl-1-invariants-or-later": "GFDL-1.1-invariants-or-later",
"gfdl-1-no-invariants-only": "GFDL-1.1-no-invariants-only",
"gfdl-1-no-invariants-or-later": "GFDL-1.1-no-invariants-or-later",
"gfdl-1-only": "GFDL-1.1-only",
"gfdl-1-or-later": "GFDL-1.1-or-later",
"gfdl-1.1": "GFDL-1.1",
"gfdl-1.1": "GFDL-1.1-only",
"gfdl-1.1-invariants-only": "GFDL-1.1-invariants-only",
"gfdl-1.1-invariants-or-later": "GFDL-1.1-invariants-or-later",
"gfdl-1.1-no-invariants-only": "GFDL-1.1-no-invariants-only",
"gfdl-1.1-no-invariants-or-later": "GFDL-1.1-no-invariants-or-later",
"gfdl-1.1-only": "GFDL-1.1-only",
"gfdl-1.1-or-later": "GFDL-1.1-or-later",
"gfdl-1.1.0": "GFDL-1.1",
"gfdl-1.1.0": "GFDL-1.1-only",
"gfdl-1.1.0-invariants-only": "GFDL-1.1-invariants-only",
"gfdl-1.1.0-invariants-or-later": "GFDL-1.1-invariants-or-later",
"gfdl-1.1.0-no-invariants-only": "GFDL-1.1-no-invariants-only",
"gfdl-1.1.0-no-invariants-or-later": "GFDL-1.1-no-invariants-or-later",
"gfdl-1.1.0-only": "GFDL-1.1-only",
"gfdl-1.1.0-or-later": "GFDL-1.1-or-later",
"gfdl-1.2": "GFDL-1.2",
"gfdl-1.2": "GFDL-1.2-only",
"gfdl-1.2-invariants-only": "GFDL-1.2-invariants-only",
"gfdl-1.2-invariants-or-later": "GFDL-1.2-invariants-or-later",
"gfdl-1.2-no-invariants-only": "GFDL-1.2-no-invariants-only",
"gfdl-1.2-no-invariants-or-later": "GFDL-1.2-no-invariants-or-later",
"gfdl-1.2-only": "GFDL-1.2-only",
"gfdl-1.2-or-later": "GFDL-1.2-or-later",
"gfdl-1.2.0": "GFDL-1.2",
"gfdl-1.2.0": "GFDL-1.2-only",
"gfdl-1.2.0-invariants-only": "GFDL-1.2-invariants-only",
"gfdl-1.2.0-invariants-or-later": "GFDL-1.2-invariants-or-later",
"gfdl-1.2.0-no-invariants-only": "GFDL-1.2-no-invariants-only",
"gfdl-1.2.0-no-invariants-or-later": "GFDL-1.2-no-invariants-or-later",
"gfdl-1.2.0-only": "GFDL-1.2-only",
"gfdl-1.2.0-or-later": "GFDL-1.2-or-later",
"gfdl-1.3": "GFDL-1.3",
"gfdl-1.3": "GFDL-1.3-only",
"gfdl-1.3-invariants-only": "GFDL-1.3-invariants-only",
"gfdl-1.3-invariants-or-later": "GFDL-1.3-invariants-or-later",
"gfdl-1.3-no-invariants-only": "GFDL-1.3-no-invariants-only",
"gfdl-1.3-no-invariants-or-later": "GFDL-1.3-no-invariants-or-later",
"gfdl-1.3-only": "GFDL-1.3-only",
"gfdl-1.3-or-later": "GFDL-1.3-or-later",
"gfdl-1.3.0": "GFDL-1.3",
"gfdl-1.3.0": "GFDL-1.3-only",
"gfdl-1.3.0-invariants-only": "GFDL-1.3-invariants-only",
"gfdl-1.3.0-invariants-or-later": "GFDL-1.3-invariants-or-later",
"gfdl-1.3.0-no-invariants-only": "GFDL-1.3-no-invariants-only",
@@ -617,6 +619,7 @@ var licenseIDs = map[string]string{
"jasper-2.0.0": "JasPer-2.0",
"jpnic": "JPNIC",
"json": "JSON",
"knuth-ctan": "Knuth-CTAN",
"lal-1": "LAL-1.2",
"lal-1.2": "LAL-1.2",
"lal-1.2.0": "LAL-1.2",
@@ -637,11 +640,11 @@ var licenseIDs = map[string]string{
"lgpl-2.0.0-only": "LGPL-2.0-only",
"lgpl-2.0.0-or-later": "LGPL-2.0-or-later",
"lgpl-2.1": "LGPL-2.1-only",
"lgpl-2.1+": "LGPL-2.1+",
"lgpl-2.1+": "LGPL-2.1-or-later",
"lgpl-2.1-only": "LGPL-2.1-only",
"lgpl-2.1-or-later": "LGPL-2.1-or-later",
"lgpl-2.1.0": "LGPL-2.1-only",
"lgpl-2.1.0+": "LGPL-2.1+",
"lgpl-2.1.0+": "LGPL-2.1-or-later",
"lgpl-2.1.0-only": "LGPL-2.1-only",
"lgpl-2.1.0-or-later": "LGPL-2.1-or-later",
"lgpl-3": "LGPL-3.0-only",
@@ -665,6 +668,7 @@ var licenseIDs = map[string]string{
"libselinux-1.0": "libselinux-1.0",
"libselinux-1.0.0": "libselinux-1.0",
"libtiff": "libtiff",
"libutil-david-nugent": "libutil-David-Nugent",
"liliq-p-1": "LiLiQ-P-1.1",
"liliq-p-1.1": "LiLiQ-P-1.1",
"liliq-p-1.1.0": "LiLiQ-P-1.1",
@@ -984,7 +988,7 @@ var licenseIDs = map[string]string{
"sspl-1": "SSPL-1.0",
"sspl-1.0": "SSPL-1.0",
"sspl-1.0.0": "SSPL-1.0",
"standardml-nj": "StandardML-NJ",
"standardml-nj": "SMLNJ",
"sugarcrm-1": "SugarCRM-1.1.3",
"sugarcrm-1.1": "SugarCRM-1.1.3",
"sugarcrm-1.1.3": "SugarCRM-1.1.3",
+9
View File
@@ -19,3 +19,12 @@ func TruncateMiddleEllipsis(input string, maxLen int) string {
}
return input[:maxLen/2] + "..." + input[len(input)-(maxLen/2):]
}
func StringInSlice(a string, list []string) bool {
for _, b := range list {
if b == a {
return true
}
}
return false
}
+1 -1
View File
@@ -18,7 +18,7 @@ func handleExit(event partybus.Event) error {
}
if err := fn(); err != nil {
return fmt.Errorf("unable to show package catalog report: %v", err)
return fmt.Errorf("unable to show package catalog report: %w", err)
}
return nil
}
+6 -6
View File
@@ -14,8 +14,8 @@ import (
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/jotframe/pkg/frame"
"github.com/anchore/go-logger"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/logger"
syftEvent "github.com/anchore/syft/syft/event"
"github.com/anchore/syft/ui"
)
@@ -60,9 +60,9 @@ func (h *ephemeralTerminalUI) Setup(unsubscribe func() error) error {
// prep the logger to not clobber the screen from now on (logrus only)
h.logBuffer = bytes.NewBufferString("")
logWrapper, ok := log.Log.(*logger.LogrusLogger)
logController, ok := log.Log.(logger.Controller)
if ok {
logWrapper.Logger.SetOutput(h.logBuffer)
logController.SetOutput(h.logBuffer)
}
return h.openScreen()
@@ -130,10 +130,10 @@ func (h *ephemeralTerminalUI) closeScreen(force bool) {
func (h *ephemeralTerminalUI) flushLog() {
// flush any errors to the screen before the report
logWrapper, ok := log.Log.(*logger.LogrusLogger)
logController, ok := log.Log.(logger.Controller)
if ok {
fmt.Fprint(logWrapper.Output, h.logBuffer.String())
logWrapper.Logger.SetOutput(h.uiOutput)
fmt.Fprint(logController.GetOutput(), h.logBuffer.String())
logController.SetOutput(h.uiOutput)
} else {
fmt.Fprint(h.uiOutput, h.logBuffer.String())
}
+22 -21
View File
@@ -10,7 +10,7 @@ import (
"sort"
"strings"
"github.com/alecthomas/jsonschema"
"github.com/invopop/jsonschema"
"github.com/anchore/syft/internal"
syftjsonModel "github.com/anchore/syft/syft/formats/syftjson/model"
@@ -28,24 +28,25 @@ can be extended to include specific package metadata struct shapes in the future
// When a new package metadata definition is created it will need to be manually added here. The variable name does
// not matter as long as it is exported.
type artifactMetadataContainer struct {
Apk pkg.ApkMetadata
Alpm pkg.AlpmMetadata
Dpkg pkg.DpkgMetadata
Gem pkg.GemMetadata
Java pkg.JavaMetadata
Npm pkg.NpmPackageJSONMetadata
Python pkg.PythonPackageMetadata
Rpm pkg.RpmMetadata
Cargo pkg.CargoPackageMetadata
Go pkg.GolangBinMetadata
Php pkg.PhpComposerJSONMetadata
Dart pkg.DartPubMetadata
Dotnet pkg.DotnetDepsMetadata
Portage pkg.PortageMetadata
Conan pkg.ConanMetadata
ConanLock pkg.ConanLockMetadata
KbPackage pkg.KbPackageMetadata
Hackage pkg.HackageMetadata
Apk pkg.ApkMetadata
Alpm pkg.AlpmMetadata
Dpkg pkg.DpkgMetadata
Gem pkg.GemMetadata
Java pkg.JavaMetadata
Npm pkg.NpmPackageJSONMetadata
Python pkg.PythonPackageMetadata
Rpm pkg.RpmMetadata
Cargo pkg.CargoPackageMetadata
Go pkg.GolangBinMetadata
Php pkg.PhpComposerJSONMetadata
Dart pkg.DartPubMetadata
Dotnet pkg.DotnetDepsMetadata
Portage pkg.PortageMetadata
Conan pkg.ConanMetadata
ConanLock pkg.ConanLockMetadata
KbPackage pkg.KbPackageMetadata
Hackage pkg.HackageMetadata
SwiftCocopods pkg.CocoapodsMetadata
}
func main() {
@@ -55,7 +56,7 @@ func main() {
func build() *jsonschema.Schema {
reflector := &jsonschema.Reflector{
AllowAdditionalProperties: true,
TypeNamer: func(r reflect.Type) string {
Namer: func(r reflect.Type) string {
return strings.TrimPrefix(r.Name(), "JSON")
},
}
@@ -87,7 +88,7 @@ func build() *jsonschema.Schema {
}
for _, name := range metadataNames {
metadataTypes = append(metadataTypes, map[string]string{
"$ref": fmt.Sprintf("#/definitions/%s", name),
"$ref": fmt.Sprintf("#/$defs/%s", name),
})
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -19,7 +19,7 @@ func IDByHash(obj interface{}) (ID, error) {
SlicesAsSets: true,
})
if err != nil {
return "", fmt.Errorf("could not build ID for object=%+v: %+v", obj, err)
return "", fmt.Errorf("could not build ID for object=%+v: %w", obj, err)
}
return ID(fmt.Sprintf("%x", f)), nil
+3 -9
View File
@@ -10,17 +10,11 @@ const (
// ContainsRelationship (supports any-to-any linkages) is a proxy for the SPDX 2.2 CONTAINS relationship.
ContainsRelationship RelationshipType = "contains"
// RuntimeDependencyOfRelationship is a proxy for the SPDX 2.2.1 RUNTIME_DEPENDENCY_OF relationship.
RuntimeDependencyOfRelationship RelationshipType = "runtime-dependency-of"
// DevDependencyOfRelationship is a proxy for the SPDX 2.2.1 DEV_DEPENDENCY_OF relationship.
DevDependencyOfRelationship RelationshipType = "dev-dependency-of"
// BuildDependencyOfRelationship is a proxy for the SPDX 2.2.1 BUILD_DEPENDENCY_OF relationship.
BuildDependencyOfRelationship RelationshipType = "build-dependency-of"
// DependencyOfRelationship is a proxy for the SPDX 2.2.1 DEPENDENCY_OF relationship.
DependencyOfRelationship RelationshipType = "dependency-of"
// DescribedByRelationship is a proxy for the SPDX 2.2.2 DESCRIBED_BY relationship.
DescribedByRelationship RelationshipType = "described-by"
)
type RelationshipType string
+5 -23
View File
@@ -1,36 +1,18 @@
package syft
import (
"bytes"
"fmt"
"io"
"github.com/anchore/syft/syft/formats"
"github.com/anchore/syft/syft/sbom"
)
// Encode takes all SBOM elements and a format option and encodes an SBOM document.
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func Encode(s sbom.SBOM, f sbom.Format) ([]byte, error) {
buff := bytes.Buffer{}
if err := f.Encode(&buff, s); err != nil {
return nil, fmt.Errorf("unable to encode sbom: %w", err)
}
return buff.Bytes(), nil
return formats.Encode(s, f)
}
// Decode takes a reader for an SBOM and generates all internal SBOM elements.
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func Decode(reader io.Reader) (*sbom.SBOM, sbom.Format, error) {
by, err := io.ReadAll(reader)
if err != nil {
return nil, nil, fmt.Errorf("unable to read sbom: %w", err)
}
f := IdentifyFormat(by)
if f == nil {
return nil, nil, fmt.Errorf("unable to identify format")
}
s, err := f.Decode(bytes.NewReader(by))
return s, f, err
return formats.Decode(reader)
}
-38
View File
@@ -1,38 +0,0 @@
package file
import (
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/source"
)
type ClassificationCataloger struct {
classifiers []Classifier
}
func NewClassificationCataloger(classifiers []Classifier) (*ClassificationCataloger, error) {
return &ClassificationCataloger{
classifiers: classifiers,
}, nil
}
func (i *ClassificationCataloger) Catalog(resolver source.FileResolver) (map[source.Coordinates][]Classification, error) {
results := make(map[source.Coordinates][]Classification)
numResults := 0
for _, location := range allRegularFiles(resolver) {
for _, classifier := range i.classifiers {
result, err := classifier.Classify(resolver, location)
if err != nil {
log.Warnf("file classification cataloger failed with class=%q at location=%+v: %+v", classifier.Class, location, err)
continue
}
if result != nil {
results[location.Coordinates] = append(results[location.Coordinates], *result)
numResults++
}
}
}
log.Debugf("file classifier discovered %d results", numResults)
return results, nil
}
-208
View File
@@ -1,208 +0,0 @@
package file
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/anchore/stereoscope/pkg/imagetest"
"github.com/anchore/syft/syft/source"
)
func TestClassifierCataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
tests := []struct {
name string
fixtureDir string
location string
expected []Classification
expectedErr func(assert.TestingT, error, ...interface{}) bool
}{
{
name: "positive-libpython3.7.so",
fixtureDir: "test-fixtures/classifiers/positive",
location: "libpython3.7.so",
expected: []Classification{
{
Class: "python-binary",
Metadata: map[string]string{
"version": "3.7.4a-vZ9",
},
},
},
expectedErr: assert.NoError,
},
{
name: "positive-python3.6",
fixtureDir: "test-fixtures/classifiers/positive",
location: "python3.6",
expected: []Classification{
{
Class: "python-binary",
Metadata: map[string]string{
"version": "3.6.3a-vZ9",
},
},
},
expectedErr: assert.NoError,
},
{
name: "positive-patchlevel.h",
fixtureDir: "test-fixtures/classifiers/positive",
location: "patchlevel.h",
expected: []Classification{
{
Class: "cpython-source",
Metadata: map[string]string{
"version": "3.9-aZ5",
},
},
},
expectedErr: assert.NoError,
},
{
name: "positive-go",
fixtureDir: "test-fixtures/classifiers/positive",
location: "go",
expected: []Classification{
{
Class: "go-binary",
Metadata: map[string]string{
"version": "1.14",
},
},
},
expectedErr: assert.NoError,
},
{
name: "positive-go-hint",
fixtureDir: "test-fixtures/classifiers/positive",
location: "VERSION",
expected: []Classification{
{
Class: "go-binary-hint",
Metadata: map[string]string{
"version": "1.15",
},
},
},
expectedErr: assert.NoError,
},
{
name: "positive-busybox",
fixtureDir: "test-fixtures/classifiers/positive",
location: "[", // note: busybox is a link to [
expected: []Classification{
{
Class: "busybox-binary",
Metadata: map[string]string{
"version": "3.33.3",
},
},
},
expectedErr: assert.NoError,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
c, err := NewClassificationCataloger(DefaultClassifiers)
test.expectedErr(t, err)
src, err := source.NewFromDirectory(test.fixtureDir)
test.expectedErr(t, err)
resolver, err := src.FileResolver(source.SquashedScope)
test.expectedErr(t, err)
actualResults, err := c.Catalog(resolver)
test.expectedErr(t, err)
ok := false
for actualLoc, actualClassification := range actualResults {
if test.location == actualLoc.RealPath {
ok = true
assert.Equal(t, test.expected, actualClassification)
}
}
if !ok {
t.Fatalf("could not find test location=%q", test.location)
}
})
}
}
func TestClassifierCataloger_DefaultClassifiers_PositiveCases_Image(t *testing.T) {
tests := []struct {
name string
fixtureImage string
location string
expected []Classification
expectedErr func(assert.TestingT, error, ...interface{}) bool
}{
{
name: "busybox-regression",
fixtureImage: "image-busybox",
location: "/bin/[",
expected: []Classification{
{
Class: "busybox-binary",
Metadata: map[string]string{
"version": "1.35.0",
},
},
},
expectedErr: assert.NoError,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
c, err := NewClassificationCataloger(DefaultClassifiers)
test.expectedErr(t, err)
img := imagetest.GetFixtureImage(t, "docker-archive", test.fixtureImage)
src, err := source.NewFromImage(img, "test-img")
test.expectedErr(t, err)
resolver, err := src.FileResolver(source.SquashedScope)
test.expectedErr(t, err)
actualResults, err := c.Catalog(resolver)
test.expectedErr(t, err)
ok := false
for actuaLoc, actualClassification := range actualResults {
if actuaLoc.RealPath == test.location {
ok = true
assert.Equal(t, test.expected, actualClassification)
}
}
if !ok {
t.Fatalf("could not find test location=%q", test.location)
}
})
}
}
func TestClassifierCataloger_DefaultClassifiers_NegativeCases(t *testing.T) {
c, err := NewClassificationCataloger(DefaultClassifiers)
assert.NoError(t, err)
src, err := source.NewFromDirectory("test-fixtures/classifiers/negative")
assert.NoError(t, err)
resolver, err := src.FileResolver(source.SquashedScope)
assert.NoError(t, err)
actualResults, err := c.Catalog(resolver)
assert.NoError(t, err)
assert.Equal(t, 0, len(actualResults))
}
-141
View File
@@ -1,141 +0,0 @@
package file
import (
"bytes"
"fmt"
"io"
"regexp"
"text/template"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/source"
)
var DefaultClassifiers = []Classifier{
{
Class: "python-binary",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)python(?P<version>[0-9]+\.[0-9]+)$`),
regexp.MustCompile(`(.*/|^)libpython(?P<version>[0-9]+\.[0-9]+).so.*$`),
},
EvidencePatternTemplates: []string{
`(?m)(?P<version>{{ .version }}\.[0-9]+[-_a-zA-Z0-9]*)`,
},
},
{
Class: "cpython-source",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)patchlevel.h$`),
},
EvidencePatternTemplates: []string{
`(?m)#define\s+PY_VERSION\s+"?(?P<version>[0-9\.\-_a-zA-Z]+)"?`,
},
},
{
Class: "go-binary",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)go$`),
},
EvidencePatternTemplates: []string{
`(?m)go(?P<version>[0-9]+\.[0-9]+(\.[0-9]+|beta[0-9]+|alpha[0-9]+|rc[0-9]+)?)`,
},
},
{
Class: "go-binary-hint",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)VERSION$`),
},
EvidencePatternTemplates: []string{
`(?m)go(?P<version>[0-9]+\.[0-9]+(\.[0-9]+|beta[0-9]+|alpha[0-9]+|rc[0-9]+)?)`,
},
},
{
Class: "busybox-binary",
FilepathPatterns: []*regexp.Regexp{
regexp.MustCompile(`(.*/|^)busybox$`),
},
EvidencePatternTemplates: []string{
`(?m)BusyBox\s+v(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`,
},
},
}
type Classifier struct {
Class string
FilepathPatterns []*regexp.Regexp
EvidencePatternTemplates []string
}
type Classification struct {
Class string `json:"class"`
Metadata map[string]string `json:"metadata"`
}
func (c Classifier) Classify(resolver source.FileResolver, location source.Location) (*Classification, error) {
doesFilepathMatch, filepathNamedGroupValues := filepathMatches(c.FilepathPatterns, location)
if !doesFilepathMatch {
return nil, nil
}
contentReader, err := resolver.FileContentsByLocation(location)
if err != nil {
return nil, err
}
defer internal.CloseAndLogError(contentReader, location.VirtualPath)
// TODO: there is room for improvement here, as this may use an excessive amount of memory. Alternate approach is to leverage a RuneReader.
contents, err := io.ReadAll(contentReader)
if err != nil {
return nil, err
}
var result *Classification
for _, patternTemplate := range c.EvidencePatternTemplates {
tmpl, err := template.New("").Parse(patternTemplate)
if err != nil {
return nil, fmt.Errorf("unable to parse classifier template=%q : %w", patternTemplate, err)
}
patternBuf := &bytes.Buffer{}
err = tmpl.Execute(patternBuf, filepathNamedGroupValues)
if err != nil {
return nil, fmt.Errorf("unable to render template: %w", err)
}
pattern, err := regexp.Compile(patternBuf.String())
if err != nil {
return nil, fmt.Errorf("unable to compile rendered regex=%q: %w", patternBuf.String(), err)
}
if !pattern.Match(contents) {
continue
}
matchMetadata := internal.MatchNamedCaptureGroups(pattern, string(contents))
if result == nil {
result = &Classification{
Class: c.Class,
Metadata: matchMetadata,
}
} else {
for key, value := range matchMetadata {
result.Metadata[key] = value
}
}
}
return result, nil
}
func filepathMatches(patterns []*regexp.Regexp, location source.Location) (bool, map[string]string) {
for _, path := range []string{location.RealPath, location.VirtualPath} {
if path == "" {
continue
}
for _, pattern := range patterns {
if pattern.MatchString(path) {
return true, internal.MatchNamedCaptureGroups(pattern, path)
}
}
}
return false, nil
}
-97
View File
@@ -1,97 +0,0 @@
package file
import (
"regexp"
"testing"
"github.com/stretchr/testify/assert"
"github.com/anchore/syft/syft/source"
)
func TestFilepathMatches(t *testing.T) {
tests := []struct {
name string
location source.Location
patterns []string
expectedMatches bool
expectedNamedGroups map[string]string
}{
{
name: "simple-filename-match",
location: source.Location{
Coordinates: source.Coordinates{
RealPath: "python2.7",
},
},
patterns: []string{
`python([0-9]+\.[0-9]+)$`,
},
expectedMatches: true,
},
{
name: "filepath-match",
location: source.Location{
Coordinates: source.Coordinates{
RealPath: "/usr/bin/python2.7",
},
},
patterns: []string{
`python([0-9]+\.[0-9]+)$`,
},
expectedMatches: true,
},
{
name: "virtual-filepath-match",
location: source.Location{
VirtualPath: "/usr/bin/python2.7",
},
patterns: []string{
`python([0-9]+\.[0-9]+)$`,
},
expectedMatches: true,
},
{
name: "full-filepath-match",
location: source.Location{
VirtualPath: "/usr/bin/python2.7",
},
patterns: []string{
`.*/bin/python([0-9]+\.[0-9]+)$`,
},
expectedMatches: true,
},
{
name: "anchored-filename-match-FAILS",
location: source.Location{
Coordinates: source.Coordinates{
RealPath: "/usr/bin/python2.7",
},
},
patterns: []string{
`^python([0-9]+\.[0-9]+)$`,
},
expectedMatches: false,
},
{
name: "empty-filename-match-FAILS",
location: source.Location{},
patterns: []string{
`^python([0-9]+\.[0-9]+)$`,
},
expectedMatches: false,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
var patterns []*regexp.Regexp
for _, p := range test.patterns {
patterns = append(patterns, regexp.MustCompile(p))
}
actualMatches, actualNamedGroups := filepathMatches(patterns, test.location)
assert.Equal(t, test.expectedMatches, actualMatches)
assert.Equal(t, test.expectedNamedGroups, actualNamedGroups)
})
}
}
@@ -1 +0,0 @@
go1.14
+15 -76
View File
@@ -1,14 +1,12 @@
package syft
import (
"bytes"
"strings"
"github.com/anchore/syft/syft/formats"
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/cyclonedxxml"
"github.com/anchore/syft/syft/formats/github"
"github.com/anchore/syft/syft/formats/spdx22json"
"github.com/anchore/syft/syft/formats/spdx22tagvalue"
"github.com/anchore/syft/syft/formats/spdxjson"
"github.com/anchore/syft/syft/formats/spdxtagvalue"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/formats/template"
@@ -17,94 +15,35 @@ import (
)
// these have been exported for the benefit of API users
// TODO: deprecated: now that the formats package has been moved to syft/formats, will be removed in v1.0.0
const (
JSONFormatID = syftjson.ID
TextFormatID = text.ID
TableFormatID = table.ID
CycloneDxXMLFormatID = cyclonedxxml.ID
CycloneDxJSONFormatID = cyclonedxjson.ID
GitHubID = github.ID
SPDXTagValueFormatID = spdx22tagvalue.ID
SPDXJSONFormatID = spdx22json.ID
GitHubFormatID = github.ID
SPDXTagValueFormatID = spdxtagvalue.ID
SPDXJSONFormatID = spdxjson.ID
TemplateFormatID = template.ID
)
var formats []sbom.Format
func init() {
formats = []sbom.Format{
syftjson.Format(),
cyclonedxxml.Format(),
cyclonedxjson.Format(),
github.Format(),
spdx22tagvalue.Format(),
spdx22json.Format(),
table.Format(),
text.Format(),
template.Format(),
}
}
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func FormatIDs() (ids []sbom.FormatID) {
for _, f := range formats {
ids = append(ids, f.ID())
}
return ids
return formats.IDs()
}
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func FormatByID(id sbom.FormatID) sbom.Format {
for _, f := range formats {
if f.ID() == id {
return f
}
}
return nil
return formats.ByID(id)
}
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func FormatByName(name string) sbom.Format {
cleanName := cleanFormatName(name)
for _, f := range formats {
if cleanFormatName(string(f.ID())) == cleanName {
return f
}
}
// handle any aliases for any supported format
switch cleanName {
case "json", "syftjson":
return FormatByID(syftjson.ID)
case "cyclonedx", "cyclone", "cyclonedxxml":
return FormatByID(cyclonedxxml.ID)
case "cyclonedxjson":
return FormatByID(cyclonedxjson.ID)
case "github", "githubjson":
return FormatByID(github.ID)
case "spdx", "spdxtv", "spdxtagvalue":
return FormatByID(spdx22tagvalue.ID)
case "spdxjson":
return FormatByID(spdx22json.ID)
case "table":
return FormatByID(table.ID)
case "text":
return FormatByID(text.ID)
case "template":
FormatByID(template.ID)
}
return nil
}
func cleanFormatName(name string) string {
r := strings.NewReplacer("-", "", "_", "")
return strings.ToLower(r.Replace(name))
return formats.ByName(name)
}
// TODO: deprecated, moved to syft/formats/formats.go. will be removed in v1.0.0
func IdentifyFormat(by []byte) sbom.Format {
for _, f := range formats {
if err := f.Validate(bytes.NewReader(by)); err != nil {
continue
}
return f
}
return nil
return formats.Identify(by)
}
@@ -2,6 +2,7 @@ package cyclonedxhelpers
import (
"fmt"
"reflect"
"testing"
"github.com/CycloneDX/cyclonedx-go"
@@ -31,20 +32,21 @@ func Test_encodeComponentProperties(t *testing.T) {
source.Location{Coordinates: source.Coordinates{RealPath: "test"}},
),
Metadata: pkg.ApkMetadata{
Package: "libc-utils",
OriginPackage: "libc-dev",
Maintainer: "Natanael Copa <ncopa@alpinelinux.org>",
Version: "0.7.2-r0",
License: "BSD",
Architecture: "x86_64",
URL: "http://alpinelinux.org",
Description: "Meta package to pull in correct libc",
Size: 0,
InstalledSize: 4096,
PullDependencies: "musl-utils",
PullChecksum: "Q1p78yvTLG094tHE1+dToJGbmYzQE=",
GitCommitOfAport: "97b1c2842faa3bfa30f5811ffbf16d5ff9f1a479",
Files: []pkg.ApkFileRecord{},
Package: "libc-utils",
OriginPackage: "libc-dev",
Maintainer: "Natanael Copa <ncopa@alpinelinux.org>",
Version: "0.7.2-r0",
License: "BSD",
Architecture: "x86_64",
URL: "http://alpinelinux.org",
Description: "Meta package to pull in correct libc",
Size: 0,
InstalledSize: 4096,
Dependencies: []string{"musl-utils"},
Provides: []string{"so:libc.so.1"},
Checksum: "Q1p78yvTLG094tHE1+dToJGbmYzQE=",
GitCommit: "97b1c2842faa3bfa30f5811ffbf16d5ff9f1a479",
Files: []pkg.ApkFileRecord{},
},
},
expected: &[]cyclonedx.Property{
@@ -53,8 +55,9 @@ func Test_encodeComponentProperties(t *testing.T) {
{Name: "syft:metadata:gitCommitOfApkPort", Value: "97b1c2842faa3bfa30f5811ffbf16d5ff9f1a479"},
{Name: "syft:metadata:installedSize", Value: "4096"},
{Name: "syft:metadata:originPackage", Value: "libc-dev"},
{Name: "syft:metadata:provides:0", Value: "so:libc.so.1"},
{Name: "syft:metadata:pullChecksum", Value: "Q1p78yvTLG094tHE1+dToJGbmYzQE="},
{Name: "syft:metadata:pullDependencies", Value: "musl-utils"},
{Name: "syft:metadata:pullDependencies:0", Value: "musl-utils"},
{Name: "syft:metadata:size", Value: "0"},
},
},
@@ -198,6 +201,7 @@ func Test_decodeComponent(t *testing.T) {
component cyclonedx.Component
wantLanguage pkg.Language
wantMetadataType pkg.MetadataType
wantMetadata interface{}
}{
{
name: "derive language from pURL if missing",
@@ -211,7 +215,7 @@ func Test_decodeComponent(t *testing.T) {
wantLanguage: pkg.Java,
},
{
name: "handle existing RpmdbMetadata type",
name: "handle RpmdbMetadata type without properties",
component: cyclonedx.Component{
Name: "acl",
Version: "2.2.53-1.el8",
@@ -226,6 +230,31 @@ func Test_decodeComponent(t *testing.T) {
},
},
wantMetadataType: pkg.RpmMetadataType,
wantMetadata: pkg.RpmMetadata{},
},
{
name: "handle RpmdbMetadata type with properties",
component: cyclonedx.Component{
Name: "acl",
Version: "2.2.53-1.el8",
PackageURL: "pkg:rpm/centos/acl@2.2.53-1.el8?arch=x86_64&upstream=acl-2.2.53-1.el8.src.rpm&distro=centos-8",
Type: "library",
BOMRef: "pkg:rpm/centos/acl@2.2.53-1.el8?arch=x86_64&upstream=acl-2.2.53-1.el8.src.rpm&distro=centos-8",
Properties: &[]cyclonedx.Property{
{
Name: "syft:package:metadataType",
Value: "RpmMetadata",
},
{
Name: "syft:metadata:release",
Value: "some-release",
},
},
},
wantMetadataType: pkg.RpmMetadataType,
wantMetadata: pkg.RpmMetadata{
Release: "some-release",
},
},
}
@@ -238,6 +267,9 @@ func Test_decodeComponent(t *testing.T) {
if tt.wantMetadataType != "" {
assert.Equal(t, tt.wantMetadataType, p.MetadataType)
}
if tt.wantMetadata != nil {
assert.Truef(t, reflect.DeepEqual(tt.wantMetadata, p.Metadata), "metadata should match: %+v != %+v", tt.wantMetadata, p.Metadata)
}
})
}
}
+35 -13
View File
@@ -6,6 +6,7 @@ import (
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/formats/common"
"github.com/anchore/syft/syft/linux"
@@ -88,6 +89,10 @@ func collectPackages(component *cyclonedx.Component, s *sbom.SBOM, idMap map[str
case cyclonedx.ComponentTypeApplication, cyclonedx.ComponentTypeFramework, cyclonedx.ComponentTypeLibrary:
p := decodeComponent(component)
idMap[component.BOMRef] = p
syftID := extractSyftPacakgeID(component.BOMRef)
if syftID != "" {
idMap[syftID] = p
}
// TODO there must be a better way than needing to call this manually:
p.SetID()
s.Artifacts.PackageCatalog.Add(*p)
@@ -100,6 +105,19 @@ func collectPackages(component *cyclonedx.Component, s *sbom.SBOM, idMap map[str
}
}
func extractSyftPacakgeID(i string) string {
instance, err := packageurl.FromString(i)
if err != nil {
return ""
}
for _, q := range instance.Qualifiers {
if q.Key == "package-id" {
return q.Value
}
}
return ""
}
func linuxReleaseFromComponents(components []cyclonedx.Component) *linux.Release {
for i := range components {
component := &components[i]
@@ -188,21 +206,25 @@ func collectRelationships(bom *cyclonedx.BOM, s *sbom.SBOM, idMap map[string]int
return
}
for _, d := range *bom.Dependencies {
from, fromOk := idMap[d.Ref].(artifact.Identifiable)
if fromOk {
if d.Dependencies == nil {
from, fromExists := idMap[d.Ref].(artifact.Identifiable)
if !fromExists {
continue
}
if d.Dependencies == nil {
continue
}
for _, t := range *d.Dependencies {
to, toExists := idMap[t].(artifact.Identifiable)
if !toExists {
continue
}
for _, t := range *d.Dependencies {
to, toOk := idMap[t.Ref].(artifact.Identifiable)
if toOk {
s.Relationships = append(s.Relationships, artifact.Relationship{
From: from,
To: to,
Type: artifact.DependencyOfRelationship, // FIXME this information is lost
})
}
}
s.Relationships = append(s.Relationships, artifact.Relationship{
From: from,
To: to,
Type: artifact.DependencyOfRelationship, // FIXME this information is lost
})
}
}
}
@@ -173,12 +173,8 @@ func Test_decode(t *testing.T) {
},
Dependencies: &[]cyclonedx.Dependency{
{
Ref: "p1",
Dependencies: &[]cyclonedx.Dependency{
{
Ref: "p2",
},
},
Ref: "p1",
Dependencies: &[]string{"p2"},
},
},
},
@@ -263,8 +259,9 @@ func Test_decode(t *testing.T) {
func Test_missingDataDecode(t *testing.T) {
bom := &cyclonedx.BOM{
Metadata: nil,
Components: &[]cyclonedx.Component{},
Metadata: nil,
Components: &[]cyclonedx.Component{},
SpecVersion: cyclonedx.SpecVersion1_4,
}
_, err := ToSyftModel(bom)
@@ -287,7 +284,9 @@ func Test_missingDataDecode(t *testing.T) {
}
func Test_missingComponentsDecode(t *testing.T) {
bom := &cyclonedx.BOM{}
bom := &cyclonedx.BOM{
SpecVersion: cyclonedx.SpecVersion1_4,
}
bomBytes, _ := json.Marshal(&bom)
decode := GetDecoder(cyclonedx.BOMFileFormatJSON)
@@ -10,7 +10,7 @@ import (
"github.com/anchore/syft/syft/pkg"
)
//nolint:funlen, gocognit
//nolint:gocognit
func encodeExternalReferences(p pkg.Package) *[]cyclonedx.ExternalReference {
var refs []cyclonedx.ExternalReference
if hasMetadata(p) {
+38 -13
View File
@@ -10,6 +10,7 @@ import (
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
@@ -91,13 +92,22 @@ func toOSComponent(distro *linux.Release) []cyclonedx.Component {
Name: distro.ID,
Version: distro.VersionID,
// TODO should we add a PURL?
CPE: distro.CPEName,
CPE: formatCPE(distro.CPEName),
ExternalReferences: eRefs,
Properties: properties,
},
}
}
func formatCPE(cpeString string) string {
cpe, err := pkg.NewCPE(cpeString)
if err != nil {
log.Debugf("skipping invalid CPE: %s", cpeString)
return ""
}
return pkg.CPEString(cpe)
}
// NewBomDescriptor returns a new BomDescriptor tailored for the current time and "syft" tool details.
func toBomDescriptor(name, version string, srcMetadata source.Metadata) *cyclonedx.Metadata {
return &cyclonedx.Metadata{
@@ -120,16 +130,11 @@ func toBomDescriptor(name, version string, srcMetadata source.Metadata) *cyclone
// An example of a relationship to not include would be: OwnershipByFileOverlapRelationship.
func isExpressiblePackageRelationship(ty artifact.RelationshipType) bool {
switch ty {
case artifact.RuntimeDependencyOfRelationship:
return true
case artifact.DevDependencyOfRelationship:
return true
case artifact.BuildDependencyOfRelationship:
return true
case artifact.DependencyOfRelationship:
return true
default:
return false
}
return false
}
func toDependencies(relationships []artifact.Relationship) []cyclonedx.Dependency {
@@ -141,10 +146,23 @@ func toDependencies(relationships []artifact.Relationship) []cyclonedx.Dependenc
continue
}
innerDeps := []cyclonedx.Dependency{}
innerDeps = append(innerDeps, cyclonedx.Dependency{Ref: string(r.From.ID())})
// we only capture package-to-package relationships for now
fromPkg, ok := r.From.(*pkg.Package)
if !ok {
continue
}
toPkg, ok := r.To.(*pkg.Package)
if !ok {
continue
}
// ind dep
innerDeps := []string{}
innerDeps = append(innerDeps, deriveBomRef(*fromPkg))
result = append(result, cyclonedx.Dependency{
Ref: string(r.To.ID()),
Ref: deriveBomRef(*toPkg),
Dependencies: &innerDeps,
})
}
@@ -152,8 +170,12 @@ func toDependencies(relationships []artifact.Relationship) []cyclonedx.Dependenc
}
func toBomDescriptorComponent(srcMetadata source.Metadata) *cyclonedx.Component {
name := srcMetadata.Name
switch srcMetadata.Scheme {
case source.ImageScheme:
if name == "" {
name = srcMetadata.ImageMetadata.UserInput
}
bomRef, err := artifact.IDByHash(srcMetadata.ImageMetadata.ID)
if err != nil {
log.Warnf("unable to get fingerprint of image metadata=%s: %+v", srcMetadata.ImageMetadata.ID, err)
@@ -161,10 +183,13 @@ func toBomDescriptorComponent(srcMetadata source.Metadata) *cyclonedx.Component
return &cyclonedx.Component{
BOMRef: string(bomRef),
Type: cyclonedx.ComponentTypeContainer,
Name: srcMetadata.ImageMetadata.UserInput,
Name: name,
Version: srcMetadata.ImageMetadata.ManifestDigest,
}
case source.DirectoryScheme, source.FileScheme:
if name == "" {
name = srcMetadata.Path
}
bomRef, err := artifact.IDByHash(srcMetadata.Path)
if err != nil {
log.Warnf("unable to get fingerprint of source metadata path=%s: %+v", srcMetadata.Path, err)
@@ -172,7 +197,7 @@ func toBomDescriptorComponent(srcMetadata source.Metadata) *cyclonedx.Component
return &cyclonedx.Component{
BOMRef: string(bomRef),
Type: cyclonedx.ComponentTypeFile,
Name: srcMetadata.Path,
Name: name,
}
}
@@ -0,0 +1,34 @@
package cyclonedxhelpers
import (
"testing"
"github.com/stretchr/testify/assert"
)
func Test_formatCPE(t *testing.T) {
tests := []struct {
cpe string
expected string
}{
{
cpe: "cpe:2.3:o:amazon:amazon_linux:2",
expected: "cpe:2.3:o:amazon:amazon_linux:2:*:*:*:*:*:*:*",
},
{
cpe: "cpe:/o:opensuse:leap:15.2",
expected: "cpe:2.3:o:opensuse:leap:15.2:*:*:*:*:*:*:*",
},
{
cpe: "invalid-cpe",
expected: "",
},
}
for _, test := range tests {
t.Run(test.cpe, func(t *testing.T) {
out := formatCPE(test.cpe)
assert.Equal(t, test.expected, out)
})
}
}
+55 -1
View File
@@ -130,6 +130,11 @@ func encode(out map[string]string, value reflect.Value, prefix string, fn FieldN
}
encode(out, pv, name, fn)
}
case reflect.Map:
// currently only map[string]string is really supported
for _, key := range value.MapKeys() {
encode(out, value.MapIndex(key), fmt.Sprintf("%s:%v", prefix, key.Interface()), fn)
}
default:
log.Warnf("skipping encoding of unsupported property: %s", prefix)
}
@@ -284,6 +289,55 @@ func decode(vals map[string]string, value reflect.Value, prefix string, fn Field
} else {
return false
}
case reflect.Map:
values := false
keyType := typ.Key()
valueType := typ.Elem()
outMap := reflect.MakeMap(typ)
str := fmt.Sprintf("%s:", prefix)
keyVals := map[string]string{}
// iterate through all keys to find those prefixed with a reference to this map
// NOTE: this will not work for nested maps
for key := range vals {
// test for map prefix
if strings.HasPrefix(key, str) {
keyVals[key] = strings.TrimPrefix(key, str)
// create new placeholder and decode key
newKeyType := keyType
if keyType.Kind() == reflect.Ptr {
newKeyType = keyType.Elem()
}
k := reflect.New(newKeyType)
if !decode(keyVals, k.Elem(), key, fn) {
log.Debugf("unable to decode key for: %s", key)
continue
}
if keyType.Kind() != reflect.Ptr {
k = k.Elem()
}
// create new placeholder and decode value
newValueType := valueType
if valueType.Kind() == reflect.Ptr {
newValueType = valueType.Elem()
}
v := reflect.New(newValueType)
if decode(vals, v.Elem(), key, fn) {
if valueType.Kind() != reflect.Ptr {
v = v.Elem()
}
// set in map
outMap.SetMapIndex(k, v)
values = true
}
}
}
if values {
value.Set(outMap)
} else {
return false
}
case reflect.Struct:
values := false
for i := 0; i < typ.NumField(); i++ {
@@ -309,7 +363,7 @@ func decode(vals map[string]string, value reflect.Value, prefix string, fn Field
func PtrToStruct(ptr interface{}) interface{} {
v := reflect.ValueOf(ptr)
if v.IsZero() {
if v.IsZero() && v.Type().Kind() != reflect.Struct {
return nil
}
switch v.Type().Kind() {
@@ -37,6 +37,10 @@ type T4 struct {
IntPtr *int
}
type T5 struct {
Map map[string]string
}
func Test_EncodeDecodeCycle(t *testing.T) {
val := 99
@@ -118,6 +122,15 @@ func Test_EncodeDecodeCycle(t *testing.T) {
{"t2 elem 1"},
},
},
{
name: "map of strings",
value: &T5{
Map: map[string]string{
"key1": "value1",
"key2": "value2",
},
},
},
}
for _, test := range tests {
@@ -1,30 +1,20 @@
package spdxhelpers
import (
"path"
"strings"
"github.com/anchore/syft/syft/source"
)
func DocumentName(srcMetadata source.Metadata) string {
if srcMetadata.Name != "" {
return srcMetadata.Name
}
switch srcMetadata.Scheme {
case source.ImageScheme:
return cleanName(srcMetadata.ImageMetadata.UserInput)
return srcMetadata.ImageMetadata.UserInput
case source.DirectoryScheme, source.FileScheme:
return cleanName(srcMetadata.Path)
return srcMetadata.Path
default:
return "unknown"
}
}
func cleanName(name string) string {
// remove # according to specification
name = strings.ReplaceAll(name, "#", "-")
// remove : for url construction
name = strings.ReplaceAll(name, ":", "-")
// clean relative pathing
return path.Clean(name)
}
@@ -35,7 +35,7 @@ func Test_DocumentName(t *testing.T) {
ManifestDigest: "digest",
},
},
expected: "image-repo/name-tag",
expected: "image-repo/name:tag",
},
{
name: "directory",
@@ -4,6 +4,7 @@ import (
"fmt"
"net/url"
"path"
"strings"
"github.com/google/uuid"
@@ -11,20 +12,27 @@ import (
"github.com/anchore/syft/syft/source"
)
const (
inputImage = "image"
inputDirectory = "dir"
inputFile = "file"
)
func DocumentNameAndNamespace(srcMetadata source.Metadata) (string, string) {
name := DocumentName(srcMetadata)
return name, DocumentNamespace(name, srcMetadata)
}
func DocumentNamespace(name string, srcMetadata source.Metadata) string {
name = cleanName(name)
input := "unknown-source-type"
switch srcMetadata.Scheme {
case source.ImageScheme:
input = "image"
input = inputImage
case source.DirectoryScheme:
input = "dir"
input = inputDirectory
case source.FileScheme:
input = "file"
input = inputFile
}
uniqueID := uuid.Must(uuid.NewRandom())
@@ -41,3 +49,13 @@ func DocumentNamespace(name string, srcMetadata source.Metadata) string {
return u.String()
}
// see: https://spdx.github.io/spdx-spec/v2.3/document-creation-information/#65-spdx-document-namespace-field
func cleanName(name string) string {
// remove # according to specification
name = strings.ReplaceAll(name, "#", "-")
// remove : for url construction
name = strings.ReplaceAll(name, ":", "-")
// clean relative pathing
return path.Clean(name)
}
@@ -4,7 +4,7 @@ type ReferenceCategory string
const (
SecurityReferenceCategory ReferenceCategory = "SECURITY"
PackageManagerReferenceCategory ReferenceCategory = "PACKAGE_MANAGER"
PackageManagerReferenceCategory ReferenceCategory = "PACKAGE-MANAGER"
OtherReferenceCategory ReferenceCategory = "OTHER"
)
@@ -109,7 +109,11 @@ func Test_Originator(t *testing.T) {
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.expected, Originator(test.input))
typ, value := Originator(test.input)
if typ != "" {
value = typ + ": " + value
}
assert.Equal(t, test.expected, value)
})
}
}
+9 -6
View File
@@ -9,9 +9,11 @@ import (
// Originator needs to conform to the SPDX spec here:
// https://spdx.github.io/spdx-spec/package-information/#76-package-originator-field
// Available options are: <omit>, NOASSERTION, Person: <person>, Organization: <org>
func Originator(p pkg.Package) string {
// return values are: <type>, <value>
func Originator(p pkg.Package) (string, string) {
typ := ""
author := ""
if hasMetadata(p) {
author := ""
switch metadata := p.Metadata.(type) {
case pkg.ApkMetadata:
author = metadata.Maintainer
@@ -29,13 +31,14 @@ func Originator(p pkg.Package) string {
author = metadata.Authors[0]
}
case pkg.RpmMetadata:
return "Organization: " + metadata.Vendor
typ = "Organization"
author = metadata.Vendor
case pkg.DpkgMetadata:
author = metadata.Maintainer
}
if author != "" {
return "Person: " + author
if typ == "" && author != "" {
typ = "Person"
}
}
return ""
return typ, author
}
@@ -6,7 +6,6 @@ import (
"github.com/anchore/syft/syft/pkg"
)
//nolint:funlen
func SourceInfo(p pkg.Package) string {
answer := ""
switch p.Type {
@@ -47,6 +46,9 @@ func SourceInfo(p pkg.Package) string {
default:
answer = "acquired package info from the following paths"
}
if p.FoundBy == "sbom-cataloger" {
answer = "acquired package info from SBOM"
}
var paths []string
for _, l := range p.Locations.ToSlice() {
paths = append(paths, l.RealPath)
@@ -183,6 +183,14 @@ func Test_SourceInfo(t *testing.T) {
"from cabal or stack manifest files",
},
},
{
input: pkg.Package{
Type: pkg.BinaryPkg,
},
expected: []string{
"acquired package info from the following paths",
},
},
}
var pkgTypes []pkg.Type
for _, test := range tests {
@@ -0,0 +1,464 @@
package spdxhelpers
import (
"fmt"
"sort"
"strings"
"time"
"github.com/spdx/tools-golang/spdx/common"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/formats/common/util"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
const (
spdxVersion = "SPDX-2.3"
noAssertion = "NOASSERTION"
)
// ToFormatModel creates and populates a new SPDX document struct that follows the SPDX 2.3
// spec from the given SBOM model.
//
//nolint:funlen
func ToFormatModel(s sbom.SBOM) *spdx.Document {
name, namespace := DocumentNameAndNamespace(s.Source)
return &spdx.Document{
// 6.1: SPDX Version; should be in the format "SPDX-x.x"
// Cardinality: mandatory, one
SPDXVersion: spdxVersion,
// 6.2: Data License; should be "CC0-1.0"
// Cardinality: mandatory, one
DataLicense: "CC0-1.0",
// 6.3: SPDX Identifier; should be "DOCUMENT" to represent mandatory identifier of SPDXRef-DOCUMENT
// Cardinality: mandatory, one
SPDXIdentifier: "DOCUMENT",
// 6.4: Document Name
// Cardinality: mandatory, one
DocumentName: name,
// 6.5: Document Namespace
// Cardinality: mandatory, one
// Purpose: Provide an SPDX document specific namespace as a unique absolute Uniform Resource
// Identifier (URI) as specified in RFC-3986, with the exception of the ‘#’ delimiter. The SPDX
// Document URI cannot contain a URI "part" (e.g. the "#" character), since the ‘#’ is used in SPDX
// element URIs (packages, files, snippets, etc) to separate the document namespace from the
// element’s SPDX identifier. Additionally, a scheme (e.g. “https:”) is required.
// The URI must be unique for the SPDX document including the specific version of the SPDX document.
// If the SPDX document is updated, thereby creating a new version, a new URI for the updated
// document must be used. There can only be one URI for an SPDX document and only one SPDX document
// for a given URI.
// Note that the URI does not have to be accessible. It is only intended to provide a unique ID.
// In many cases, the URI will point to a web accessible document, but this should not be assumed
// to be the case.
DocumentNamespace: namespace,
// 6.6: External Document References
// Cardinality: optional, one or many
ExternalDocumentReferences: nil,
// 6.11: Document Comment
// Cardinality: optional, one
DocumentComment: "",
CreationInfo: &spdx.CreationInfo{
// 6.7: License List Version
// Cardinality: optional, one
LicenseListVersion: spdxlicense.Version,
// 6.8: Creators: may have multiple keys for Person, Organization
// and/or Tool
// Cardinality: mandatory, one or many
Creators: []common.Creator{
{
Creator: "Anchore, Inc",
CreatorType: "Organization",
},
{
Creator: internal.ApplicationName + "-" + s.Descriptor.Version,
CreatorType: "Tool",
},
},
// 6.9: Created: data format YYYY-MM-DDThh:mm:ssZ
// Cardinality: mandatory, one
Created: time.Now().UTC().Format(time.RFC3339),
// 6.10: Creator Comment
// Cardinality: optional, one
CreatorComment: "",
},
Packages: toPackages(s.Artifacts.PackageCatalog),
Files: toFiles(s),
Relationships: toRelationships(s.RelationshipsSorted()),
}
}
func toSPDXID(identifiable artifact.Identifiable) common.ElementID {
id := ""
if p, ok := identifiable.(pkg.Package); ok {
id = SanitizeElementID(fmt.Sprintf("Package-%+v-%s-%s", p.Type, p.Name, p.ID()))
} else {
id = string(identifiable.ID())
}
// NOTE: the spdx libraries prepend SPDXRef-, so we don't do it here
return common.ElementID(id)
}
// packages populates all Package Information from the package Catalog (see https://spdx.github.io/spdx-spec/3-package-information/)
//
//nolint:funlen
func toPackages(catalog *pkg.Catalog) (results []*spdx.Package) {
for _, p := range catalog.Sorted() {
// name should be guaranteed to be unique, but semantically useful and stable
id := toSPDXID(p)
// If the Concluded License is not the same as the Declared License, a written explanation should be provided
// in the Comments on License field (section 7.16). With respect to NOASSERTION, a written explanation in
// the Comments on License field (section 7.16) is preferred.
license := License(p)
checksums, filesAnalyzed := toPackageChecksums(p)
results = append(results, &spdx.Package{
// NOT PART OF SPEC
// flag: does this "package" contain files that were in fact "unpackaged",
// e.g. included directly in the Document without being in a Package?
IsUnpackaged: false,
// 7.1: Package Name
// Cardinality: mandatory, one
PackageName: p.Name,
// 7.2: Package SPDX Identifier: "SPDXRef-[idstring]"
// Cardinality: mandatory, one
PackageSPDXIdentifier: id,
// 7.3: Package Version
// Cardinality: optional, one
PackageVersion: p.Version,
// 7.4: Package File Name
// Cardinality: optional, one
PackageFileName: "",
// 7.5: Package Supplier: may have single result for either Person or Organization,
// or NOASSERTION
// Cardinality: optional, one
// 7.6: Package Originator: may have single result for either Person or Organization,
// or NOASSERTION
// Cardinality: optional, one
PackageSupplier: nil,
PackageOriginator: toPackageOriginator(p),
// 7.7: Package Download Location
// Cardinality: mandatory, one
// NONE if there is no download location whatsoever.
// NOASSERTION if:
// (i) the SPDX file creator has attempted to but cannot reach a reasonable objective determination;
// (ii) the SPDX file creator has made no attempt to determine this field; or
// (iii) the SPDX file creator has intentionally provided no information (no meaning should be implied by doing so).
PackageDownloadLocation: DownloadLocation(p),
// 7.8: FilesAnalyzed
// Cardinality: optional, one; default value is "true" if omitted
// Purpose: Indicates whether the file content of this package has been available for or subjected to
// analysis when creating the SPDX document. If false, indicates packages that represent metadata or
// URI references to a project, product, artifact, distribution or a component. If false, the package
// must not contain any files.
// Intent: A package can refer to a project, product, artifact, distribution or a component that is
// external to the SPDX document.
FilesAnalyzed: filesAnalyzed,
// NOT PART OF SPEC: did FilesAnalyzed tag appear?
IsFilesAnalyzedTagPresent: true,
// 7.9: Package Verification Code
// Cardinality: optional, one if filesAnalyzed is true / omitted;
// zero (must be omitted) if filesAnalyzed is false
PackageVerificationCode: nil,
// 7.10: Package Checksum: may have keys for SHA1, SHA256 and/or MD5
// Cardinality: optional, one or many
// 7.10.1 Purpose: Provide an independently reproducible mechanism that permits unique identification of
// a specific package that correlates to the data in this SPDX file. This identifier enables a recipient
// to determine if any file in the original package has been changed. If the SPDX file is to be included
// in a package, this value should not be calculated. The SHA-1 algorithm will be used to provide the
// checksum by default.
PackageChecksums: checksums,
// 7.11: Package Home Page
// Cardinality: optional, one
PackageHomePage: Homepage(p),
// 7.12: Source Information
// Cardinality: optional, one
PackageSourceInfo: SourceInfo(p),
// 7.13: Concluded License: SPDX License Expression, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one
// Purpose: Contain the license the SPDX file creator has concluded as governing the
// package or alternative values, if the governing license cannot be determined.
PackageLicenseConcluded: license,
// 7.14: All Licenses Info from Files: SPDX License Expression, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one or many if filesAnalyzed is true / omitted;
// zero (must be omitted) if filesAnalyzed is false
PackageLicenseInfoFromFiles: nil,
// 7.15: Declared License: SPDX License Expression, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one
// Purpose: List the licenses that have been declared by the authors of the package.
// Any license information that does not originate from the package authors, e.g. license
// information from a third party repository, should not be included in this field.
PackageLicenseDeclared: license,
// 7.16: Comments on License
// Cardinality: optional, one
PackageLicenseComments: "",
// 7.17: Copyright Text: copyright notice(s) text, "NONE" or "NOASSERTION"
// Cardinality: mandatory, one
// Purpose: IdentifyFormat the copyright holders of the package, as well as any dates present. This will be a free form text field extracted from package information files. The options to populate this field are limited to:
//
// Any text related to a copyright notice, even if not complete;
// NONE if the package contains no copyright information whatsoever; or
// NOASSERTION, if
// (i) the SPDX document creator has made no attempt to determine this field; or
// (ii) the SPDX document creator has intentionally provided no information (no meaning should be implied by doing so).
//
PackageCopyrightText: noAssertion,
// 7.18: Package Summary Description
// Cardinality: optional, one
PackageSummary: "",
// 7.19: Package Detailed Description
// Cardinality: optional, one
PackageDescription: Description(p),
// 7.20: Package Comment
// Cardinality: optional, one
PackageComment: "",
// 7.21: Package External Reference
// Cardinality: optional, one or many
PackageExternalReferences: formatSPDXExternalRefs(p),
// 7.22: Package External Reference Comment
// Cardinality: conditional (optional, one) for each External Reference
// contained within PackageExternalReference2_1 struct, if present
// 7.23: Package Attribution Text
// Cardinality: optional, one or many
PackageAttributionTexts: nil,
})
}
return results
}
func toPackageOriginator(p pkg.Package) *common.Originator {
kind, originator := Originator(p)
if kind == "" || originator == "" {
return nil
}
return &common.Originator{
Originator: originator,
OriginatorType: kind,
}
}
func toPackageChecksums(p pkg.Package) ([]common.Checksum, bool) {
filesAnalyzed := false
var checksums []common.Checksum
switch meta := p.Metadata.(type) {
// we generate digest for some Java packages
// spdx.github.io/spdx-spec/package-information/#710-package-checksum-field
case pkg.JavaMetadata:
if len(meta.ArchiveDigests) > 0 {
filesAnalyzed = true
for _, digest := range meta.ArchiveDigests {
algo := strings.ToUpper(digest.Algorithm)
checksums = append(checksums, common.Checksum{
Algorithm: common.ChecksumAlgorithm(algo),
Value: digest.Value,
})
}
}
case pkg.GolangBinMetadata:
algo, hexStr, err := util.HDigestToSHA(meta.H1Digest)
if err != nil {
log.Debugf("invalid h1digest: %s: %v", meta.H1Digest, err)
break
}
algo = strings.ToUpper(algo)
checksums = append(checksums, common.Checksum{
Algorithm: common.ChecksumAlgorithm(algo),
Value: hexStr,
})
}
return checksums, filesAnalyzed
}
func formatSPDXExternalRefs(p pkg.Package) (refs []*spdx.PackageExternalReference) {
for _, ref := range ExternalRefs(p) {
refs = append(refs, &spdx.PackageExternalReference{
Category: string(ref.ReferenceCategory),
RefType: string(ref.ReferenceType),
Locator: ref.ReferenceLocator,
ExternalRefComment: ref.Comment,
})
}
return refs
}
func toRelationships(relationships []artifact.Relationship) (result []*spdx.Relationship) {
for _, r := range relationships {
exists, relationshipType, comment := lookupRelationship(r.Type)
if !exists {
log.Debugf("unable to convert relationship to SPDX, dropping: %+v", r)
continue
}
// FIXME: we are only currently including Package -> * relationships
if _, ok := r.From.(pkg.Package); !ok {
log.Debugf("skipping non-package relationship: %+v", r)
continue
}
result = append(result, &spdx.Relationship{
RefA: common.DocElementID{
ElementRefID: toSPDXID(r.From),
},
Relationship: string(relationshipType),
RefB: common.DocElementID{
ElementRefID: toSPDXID(r.To),
},
RelationshipComment: comment,
})
}
return result
}
func lookupRelationship(ty artifact.RelationshipType) (bool, RelationshipType, string) {
switch ty {
case artifact.ContainsRelationship:
return true, ContainsRelationship, ""
case artifact.DependencyOfRelationship:
return true, DependencyOfRelationship, ""
case artifact.OwnershipByFileOverlapRelationship:
return true, OtherRelationship, fmt.Sprintf("%s: indicates that the parent package claims ownership of a child package since the parent metadata indicates overlap with a location that a cataloger found the child package by", ty)
}
return false, "", ""
}
func toFiles(s sbom.SBOM) (results []*spdx.File) {
artifacts := s.Artifacts
for _, coordinates := range s.AllCoordinates() {
var metadata *source.FileMetadata
if metadataForLocation, exists := artifacts.FileMetadata[coordinates]; exists {
metadata = &metadataForLocation
}
var digests []file.Digest
if digestsForLocation, exists := artifacts.FileDigests[coordinates]; exists {
digests = digestsForLocation
}
// TODO: add file classifications (?) and content as a snippet
var comment string
if coordinates.FileSystemID != "" {
comment = fmt.Sprintf("layerID: %s", coordinates.FileSystemID)
}
results = append(results, &spdx.File{
FileSPDXIdentifier: toSPDXID(coordinates),
FileComment: comment,
// required, no attempt made to determine license information
LicenseConcluded: noAssertion,
Checksums: toFileChecksums(digests),
FileName: coordinates.RealPath,
FileTypes: toFileTypes(metadata),
})
}
// sort by real path then virtual path to ensure the result is stable across multiple runs
sort.SliceStable(results, func(i, j int) bool {
if results[i].FileName == results[j].FileName {
return results[i].FileSPDXIdentifier < results[j].FileSPDXIdentifier
}
return results[i].FileName < results[j].FileName
})
return results
}
func toFileChecksums(digests []file.Digest) (checksums []common.Checksum) {
for _, digest := range digests {
checksums = append(checksums, common.Checksum{
Algorithm: toChecksumAlgorithm(digest.Algorithm),
Value: digest.Value,
})
}
return checksums
}
func toChecksumAlgorithm(algorithm string) common.ChecksumAlgorithm {
// this needs to be an uppercase version of our algorithm
return common.ChecksumAlgorithm(strings.ToUpper(algorithm))
}
func toFileTypes(metadata *source.FileMetadata) (ty []string) {
if metadata == nil {
return nil
}
mimeTypePrefix := strings.Split(metadata.MIMEType, "/")[0]
switch mimeTypePrefix {
case "image":
ty = append(ty, string(ImageFileType))
case "video":
ty = append(ty, string(VideoFileType))
case "application":
ty = append(ty, string(ApplicationFileType))
case "text":
ty = append(ty, string(TextFileType))
case "audio":
ty = append(ty, string(AudioFileType))
}
if internal.IsExecutable(metadata.MIMEType) {
ty = append(ty, string(BinaryFileType))
}
if internal.IsArchive(metadata.MIMEType) {
ty = append(ty, string(ArchiveFileType))
}
// TODO: add support for source, spdx, and documentation file types
if len(ty) == 0 {
ty = append(ty, string(OtherFileType))
}
return ty
}
@@ -0,0 +1,432 @@
package spdxhelpers
import (
"fmt"
"testing"
"github.com/spdx/tools-golang/spdx/common"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
// TODO: Add ToFormatModel tests
func Test_toPackageChecksums(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expected []common.Checksum
filesAnalyzed bool
}{
{
name: "Java Package",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Language: pkg.Java,
Metadata: pkg.JavaMetadata{
ArchiveDigests: []file.Digest{
{
Algorithm: "sha1", // SPDX expects these to be uppercase
Value: "1234",
},
},
},
},
expected: []common.Checksum{
{
Algorithm: "SHA1",
Value: "1234",
},
},
filesAnalyzed: true,
},
{
name: "Java Package with no archive digests",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Language: pkg.Java,
Metadata: pkg.JavaMetadata{
ArchiveDigests: []file.Digest{},
},
},
expected: []common.Checksum{},
filesAnalyzed: false,
},
{
name: "Java Package with no metadata",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Language: pkg.Java,
},
expected: []common.Checksum{},
filesAnalyzed: false,
},
{
name: "Go Binary Package",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Language: pkg.Go,
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
},
expected: []common.Checksum{
{
Algorithm: "SHA256",
Value: "f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
},
},
filesAnalyzed: false,
},
{
name: "Package with no metadata type",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Language: pkg.Java,
Metadata: struct{}{},
},
expected: []common.Checksum{},
filesAnalyzed: false,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
commonSum, filesAnalyzed := toPackageChecksums(test.pkg)
assert.ElementsMatch(t, test.expected, commonSum)
assert.Equal(t, test.filesAnalyzed, filesAnalyzed)
})
}
}
func Test_toFileTypes(t *testing.T) {
tests := []struct {
name string
metadata source.FileMetadata
expected []string
}{
{
name: "application",
metadata: source.FileMetadata{
MIMEType: "application/vnd.unknown",
},
expected: []string{
string(ApplicationFileType),
},
},
{
name: "archive",
metadata: source.FileMetadata{
MIMEType: "application/zip",
},
expected: []string{
string(ApplicationFileType),
string(ArchiveFileType),
},
},
{
name: "audio",
metadata: source.FileMetadata{
MIMEType: "audio/ogg",
},
expected: []string{
string(AudioFileType),
},
},
{
name: "video",
metadata: source.FileMetadata{
MIMEType: "video/3gpp",
},
expected: []string{
string(VideoFileType),
},
},
{
name: "text",
metadata: source.FileMetadata{
MIMEType: "text/html",
},
expected: []string{
string(TextFileType),
},
},
{
name: "image",
metadata: source.FileMetadata{
MIMEType: "image/png",
},
expected: []string{
string(ImageFileType),
},
},
{
name: "binary",
metadata: source.FileMetadata{
MIMEType: "application/x-sharedlib",
},
expected: []string{
string(ApplicationFileType),
string(BinaryFileType),
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.ElementsMatch(t, test.expected, toFileTypes(&test.metadata))
})
}
}
func Test_lookupRelationship(t *testing.T) {
tests := []struct {
input artifact.RelationshipType
exists bool
ty RelationshipType
comment string
}{
{
input: artifact.ContainsRelationship,
exists: true,
ty: ContainsRelationship,
},
{
input: artifact.OwnershipByFileOverlapRelationship,
exists: true,
ty: OtherRelationship,
comment: "ownership-by-file-overlap: indicates that the parent package claims ownership of a child package since the parent metadata indicates overlap with a location that a cataloger found the child package by",
},
{
input: "made-up",
exists: false,
},
}
for _, test := range tests {
t.Run(string(test.input), func(t *testing.T) {
exists, ty, comment := lookupRelationship(test.input)
assert.Equal(t, exists, test.exists)
assert.Equal(t, ty, test.ty)
assert.Equal(t, comment, test.comment)
})
}
}
func Test_toFileChecksums(t *testing.T) {
tests := []struct {
name string
digests []file.Digest
expected []common.Checksum
}{
{
name: "empty",
},
{
name: "has digests",
digests: []file.Digest{
{
Algorithm: "SHA256",
Value: "deadbeefcafe",
},
{
Algorithm: "md5",
Value: "meh",
},
},
expected: []common.Checksum{
{
Algorithm: "SHA256",
Value: "deadbeefcafe",
},
{
Algorithm: "MD5",
Value: "meh",
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
assert.ElementsMatch(t, test.expected, toFileChecksums(test.digests))
})
}
}
func Test_fileIDsForPackage(t *testing.T) {
p := pkg.Package{
Name: "bogus",
}
c := source.Coordinates{
RealPath: "/path",
FileSystemID: "nowhere",
}
docElementId := func(identifiable artifact.Identifiable) common.DocElementID {
return common.DocElementID{
ElementRefID: toSPDXID(identifiable),
}
}
tests := []struct {
name string
relationships []artifact.Relationship
expected []*spdx.Relationship
}{
{
name: "package-to-file contains relationships",
relationships: []artifact.Relationship{
{
From: p,
To: c,
Type: artifact.ContainsRelationship,
},
},
expected: []*spdx.Relationship{
{
Relationship: "CONTAINS",
RefA: docElementId(p),
RefB: docElementId(c),
},
},
},
{
name: "package-to-package",
relationships: []artifact.Relationship{
{
From: p,
To: p,
Type: artifact.ContainsRelationship,
},
},
expected: []*spdx.Relationship{
{
Relationship: "CONTAINS",
RefA: docElementId(p),
RefB: docElementId(p),
},
},
},
{
name: "ignore file-to-file",
relationships: []artifact.Relationship{
{
From: c,
To: c,
Type: artifact.ContainsRelationship,
},
},
expected: nil,
},
{
name: "ignore file-to-package",
relationships: []artifact.Relationship{
{
From: c,
To: p,
Type: artifact.ContainsRelationship,
},
},
expected: nil,
},
{
name: "include package-to-file overlap relationships",
relationships: []artifact.Relationship{
{
From: p,
To: c,
Type: artifact.OwnershipByFileOverlapRelationship,
},
},
expected: []*spdx.Relationship{
{
Relationship: "OTHER",
RefA: docElementId(p),
RefB: docElementId(c),
RelationshipComment: "ownership-by-file-overlap: indicates that the parent package claims ownership of a child package since the parent metadata indicates overlap with a location that a cataloger found the child package by",
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
relationships := toRelationships(test.relationships)
assert.Equal(t, test.expected, relationships)
})
}
}
func Test_H1Digest(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
expectedDigest string
}{
{
name: "valid h1digest",
pkg: pkg.Package{
Name: "github.com/googleapis/gnostic",
Version: "v0.5.5",
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
},
expectedDigest: "SHA256:f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
},
{
name: "invalid h1digest",
pkg: pkg.Package{
Name: "github.com/googleapis/gnostic",
Version: "v0.5.5",
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h1:9fHAtK0uzzz",
},
},
expectedDigest: "",
},
{
name: "unsupported h-digest",
pkg: pkg.Package{
Name: "github.com/googleapis/gnostic",
Version: "v0.5.5",
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
H1Digest: "h12:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
},
expectedDigest: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
catalog := pkg.NewCatalog(test.pkg)
pkgs := toPackages(catalog)
require.Len(t, pkgs, 1)
for _, p := range pkgs {
if test.expectedDigest == "" {
require.Len(t, p.PackageChecksums, 0)
} else {
require.Len(t, p.PackageChecksums, 1)
for _, c := range p.PackageChecksums {
require.Equal(t, test.expectedDigest, fmt.Sprintf("%s:%s", c.Algorithm, c.Value))
}
}
}
})
}
}
@@ -6,19 +6,20 @@ import (
"strconv"
"strings"
"github.com/spdx/tools-golang/spdx"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/formats/common/util"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
func ToSyftModel(doc *spdx.Document2_2) (*sbom.SBOM, error) {
func ToSyftModel(doc *spdx.Document) (*sbom.SBOM, error) {
if doc == nil {
return nil, errors.New("cannot convert SPDX document to Syft model because document is nil")
}
@@ -26,9 +27,7 @@ func ToSyftModel(doc *spdx.Document2_2) (*sbom.SBOM, error) {
spdxIDMap := make(map[string]interface{})
src := source.Metadata{Scheme: source.UnknownScheme}
if doc.CreationInfo != nil {
src.Scheme = extractSchemeFromNamespace(doc.CreationInfo.DocumentNamespace)
}
src.Scheme = extractSchemeFromNamespace(doc.DocumentNamespace)
s := &sbom.SBOM{
Source: src,
@@ -62,18 +61,18 @@ func extractSchemeFromNamespace(ns string) source.Scheme {
parts := strings.Split(u.Path, "/")
for _, p := range parts {
switch p {
case "file":
case inputFile:
return source.FileScheme
case "image":
case inputImage:
return source.ImageScheme
case "dir":
case inputDirectory:
return source.DirectoryScheme
}
}
return source.UnknownScheme
}
func findLinuxReleaseByPURL(doc *spdx.Document2_2) *linux.Release {
func findLinuxReleaseByPURL(doc *spdx.Document) *linux.Release {
for _, p := range doc.Packages {
purlValue := findPURLValue(p)
if purlValue == "" {
@@ -106,7 +105,7 @@ func findLinuxReleaseByPURL(doc *spdx.Document2_2) *linux.Release {
return nil
}
func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document2_2) {
func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document) {
for _, p := range doc.Packages {
syftPkg := toSyftPackage(p)
spdxIDMap[string(p.PackageSPDXIdentifier)] = syftPkg
@@ -114,8 +113,8 @@ func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *sp
}
}
func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document2_2) {
for _, f := range doc.UnpackagedFiles {
func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document) {
for _, f := range doc.Files {
l := toSyftLocation(f)
spdxIDMap[string(f.FileSPDXIdentifier)] = l
@@ -124,8 +123,8 @@ func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.
}
}
func toFileDigests(f *spdx.File2_2) (digests []file.Digest) {
for _, digest := range f.FileChecksums {
func toFileDigests(f *spdx.File) (digests []file.Digest) {
for _, digest := range f.Checksums {
digests = append(digests, file.Digest{
Algorithm: string(digest.Algorithm),
Value: digest.Value,
@@ -134,9 +133,9 @@ func toFileDigests(f *spdx.File2_2) (digests []file.Digest) {
return digests
}
func toFileMetadata(f *spdx.File2_2) (meta source.FileMetadata) {
func toFileMetadata(f *spdx.File) (meta source.FileMetadata) {
// FIXME Syft is currently lossy due to the SPDX 2.2.1 spec not supporting arbitrary mimetypes
for _, typ := range f.FileType {
for _, typ := range f.FileTypes {
switch FileType(typ) {
case ImageFileType:
meta.MIMEType = "image/"
@@ -156,11 +155,11 @@ func toFileMetadata(f *spdx.File2_2) (meta source.FileMetadata) {
return meta
}
func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document2_2) []artifact.Relationship {
func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document) []artifact.Relationship {
var out []artifact.Relationship
for _, r := range doc.Relationships {
// FIXME what to do with r.RefA.DocumentRefID and r.RefA.SpecialID
if r.RefA.DocumentRefID != "" && requireAndTrimPrefix(r.RefA.DocumentRefID, "DocumentRef-") != string(doc.CreationInfo.SPDXIdentifier) {
if r.RefA.DocumentRefID != "" && requireAndTrimPrefix(r.RefA.DocumentRefID, "DocumentRef-") != string(doc.SPDXIdentifier) {
log.Debugf("ignoring relationship to external document: %+v", r)
continue
}
@@ -185,16 +184,10 @@ func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document2_2
case ContainsRelationship:
typ = artifact.ContainsRelationship
to = toPackage
case BuildDependencyOfRelationship:
typ = artifact.BuildDependencyOfRelationship
to = toPackage
case RuntimeDependencyOfRelationship:
typ = artifact.RuntimeDependencyOfRelationship
to = toPackage
case OtherRelationship:
// Encoding uses a specifically formatted comment...
if strings.Index(r.RelationshipComment, string(artifact.OwnershipByFileOverlapRelationship)) == 0 {
typ = artifact.RuntimeDependencyOfRelationship
typ = artifact.DependencyOfRelationship
to = toPackage
}
}
@@ -210,7 +203,7 @@ func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document2_2
return out
}
func toSyftCoordinates(f *spdx.File2_2) source.Coordinates {
func toSyftCoordinates(f *spdx.File) source.Coordinates {
const layerIDPrefix = "layerID: "
var fileSystemID string
if strings.Index(f.FileComment, layerIDPrefix) == 0 {
@@ -225,7 +218,7 @@ func toSyftCoordinates(f *spdx.File2_2) source.Coordinates {
}
}
func toSyftLocation(f *spdx.File2_2) *source.Location {
func toSyftLocation(f *spdx.File) *source.Location {
return &source.Location{
Coordinates: toSyftCoordinates(f),
VirtualPath: f.FileName,
@@ -260,7 +253,7 @@ func findQualifierValue(purl packageurl.PackageURL, qualifier string) string {
return ""
}
func extractPkgInfo(p *spdx.Package2_2) pkgInfo {
func extractPkgInfo(p *spdx.Package) pkgInfo {
pu := findPURLValue(p)
purl, err := packageurl.FromString(pu)
if err != nil {
@@ -273,7 +266,7 @@ func extractPkgInfo(p *spdx.Package2_2) pkgInfo {
}
}
func toSyftPackage(p *spdx.Package2_2) *pkg.Package {
func toSyftPackage(p *spdx.Package) *pkg.Package {
info := extractPkgInfo(p)
metadataType, metadata := extractMetadata(p, info)
sP := pkg.Package{
@@ -293,7 +286,8 @@ func toSyftPackage(p *spdx.Package2_2) *pkg.Package {
return &sP
}
func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interface{}) {
//nolint:funlen
func extractMetadata(p *spdx.Package, info pkgInfo) (pkg.MetadataType, interface{}) {
arch := info.qualifierValue(pkg.PURLQualifierArch)
upstreamValue := info.qualifierValue(pkg.PURLQualifierUpstream)
upstream := strings.SplitN(upstreamValue, "@", 2)
@@ -302,12 +296,20 @@ func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interf
if len(upstream) > 1 {
upstreamVersion = upstream[1]
}
supplier := ""
if p.PackageSupplier != nil {
supplier = p.PackageSupplier.Supplier
}
originator := ""
if p.PackageOriginator != nil {
originator = p.PackageOriginator.Originator
}
switch info.typ {
case pkg.ApkPkg:
return pkg.ApkMetadataType, pkg.ApkMetadata{
Package: p.PackageName,
OriginPackage: upstreamName,
Maintainer: p.PackageSupplierPerson,
Maintainer: supplier,
Version: p.PackageVersion,
License: p.PackageLicenseDeclared,
Architecture: arch,
@@ -333,7 +335,7 @@ func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interf
Arch: arch,
SourceRpm: upstreamValue,
License: license,
Vendor: p.PackageOriginatorOrganization,
Vendor: originator,
}
case pkg.DebPkg:
return pkg.DpkgMetadataType, pkg.DpkgMetadata{
@@ -342,21 +344,35 @@ func extractMetadata(p *spdx.Package2_2, info pkgInfo) (pkg.MetadataType, interf
Version: p.PackageVersion,
SourceVersion: upstreamVersion,
Architecture: arch,
Maintainer: p.PackageOriginatorPerson,
Maintainer: originator,
}
case pkg.JavaPkg:
var digests []file.Digest
for algorithm, value := range p.PackageChecksums {
digests = append(digests, file.Digest{Algorithm: string(algorithm), Value: value.Value})
for _, value := range p.PackageChecksums {
digests = append(digests, file.Digest{Algorithm: string(value.Algorithm), Value: value.Value})
}
return pkg.JavaMetadataType, pkg.JavaMetadata{
ArchiveDigests: digests,
}
case pkg.GoModulePkg:
var h1Digest string
for _, value := range p.PackageChecksums {
digest, err := util.HDigestFromSHA(string(value.Algorithm), value.Value)
if err != nil {
log.Debugf("invalid h1digest: %v %v", value, err)
continue
}
h1Digest = digest
break
}
return pkg.GolangBinMetadataType, pkg.GolangBinMetadata{
H1Digest: h1Digest,
}
}
return pkg.UnknownMetadataType, nil
}
func findPURLValue(p *spdx.Package2_2) string {
func findPURLValue(p *spdx.Package) string {
for _, r := range p.PackageExternalReferences {
if r.RefType == string(PurlExternalRefType) {
return r.Locator
@@ -365,7 +381,7 @@ func findPURLValue(p *spdx.Package2_2) string {
return ""
}
func extractCPEs(p *spdx.Package2_2) (cpes []pkg.CPE) {
func extractCPEs(p *spdx.Package) (cpes []pkg.CPE) {
for _, r := range p.PackageExternalReferences {
if r.RefType == string(Cpe23ExternalRefType) {
cpe, err := pkg.NewCPE(r.Locator)
@@ -3,7 +3,8 @@ package spdxhelpers
import (
"testing"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/spdx/common"
spdx "github.com/spdx/tools-golang/spdx/v2_3"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -12,32 +13,27 @@ import (
)
func TestToSyftModel(t *testing.T) {
sbom, err := ToSyftModel(&spdx.Document2_2{
CreationInfo: &spdx.CreationInfo2_2{
SPDXVersion: "1",
DataLicense: "GPL",
SPDXIdentifier: "id-doc-1",
DocumentName: "docName",
DocumentNamespace: "docNamespace",
ExternalDocumentReferences: nil,
LicenseListVersion: "",
CreatorPersons: nil,
CreatorOrganizations: nil,
CreatorTools: nil,
Created: "",
CreatorComment: "",
DocumentComment: "",
sbom, err := ToSyftModel(&spdx.Document{
SPDXVersion: "1",
DataLicense: "GPL",
SPDXIdentifier: "id-doc-1",
DocumentName: "docName",
DocumentNamespace: "docNamespace",
ExternalDocumentReferences: nil,
DocumentComment: "",
CreationInfo: &spdx.CreationInfo{
LicenseListVersion: "",
Created: "",
CreatorComment: "",
},
Packages: map[spdx.ElementID]*spdx.Package2_2{
"id-pkg-1": {
PackageName: "pkg-1",
PackageSPDXIdentifier: "id-pkg-1",
PackageVersion: "5.4.3",
PackageSupplierPerson: "",
PackageSupplierOrganization: "",
PackageLicenseDeclared: "",
PackageDescription: "",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
Packages: []*spdx.Package{
{
PackageName: "pkg-1",
PackageSPDXIdentifier: "id-pkg-1",
PackageVersion: "5.4.3",
PackageLicenseDeclared: "",
PackageDescription: "",
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "SECURITY",
Locator: "cpe:2.3:a:pkg-1:pkg-1:5.4.3:*:*:*:*:*:*:*",
@@ -49,22 +45,20 @@ func TestToSyftModel(t *testing.T) {
RefType: "cpe23Type",
},
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:alpine/pkg-1@5.4.3?arch=x86_64&upstream=p1-origin&distro=alpine-3.10.9",
RefType: "purl",
},
},
Files: nil,
},
"id-pkg-2": {
PackageName: "pkg-2",
PackageSPDXIdentifier: "id-pkg-2",
PackageVersion: "7.3.1",
PackageSupplierPerson: "",
PackageSupplierOrganization: "",
PackageLicenseDeclared: "",
PackageDescription: "",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
{
PackageName: "pkg-2",
PackageSPDXIdentifier: "id-pkg-2",
PackageVersion: "7.3.1",
PackageLicenseDeclared: "",
PackageDescription: "",
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "SECURITY",
Locator: "cpe:2.3:a:pkg-2:pkg-2:7.3.1:*:*:*:*:*:*:*",
@@ -81,7 +75,7 @@ func TestToSyftModel(t *testing.T) {
RefType: "cpe23Type",
},
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:deb/pkg-2@7.3.1?arch=x86_64&upstream=p2-origin@9.1.3&distro=debian-3.10.9",
RefType: "purl",
},
@@ -89,8 +83,7 @@ func TestToSyftModel(t *testing.T) {
Files: nil,
},
},
UnpackagedFiles: map[spdx.ElementID]*spdx.File2_2{},
Relationships: []*spdx.Relationship2_2{},
Relationships: []*spdx.Relationship{},
})
assert.NoError(t, err)
@@ -120,17 +113,17 @@ func TestToSyftModel(t *testing.T) {
func Test_extractMetadata(t *testing.T) {
oneTwoThreeFour := 1234
tests := []struct {
pkg spdx.Package2_2
pkg spdx.Package
metaType pkg.MetadataType
meta interface{}
}{
{
pkg: spdx.Package2_2{
pkg: spdx.Package{
PackageName: "SomeDebPkg",
PackageVersion: "43.1.235",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:deb/pkg-2@7.3.1?arch=x86_64&upstream=somedebpkg-origin@9.1.3&distro=debian-3.10.9",
RefType: "purl",
},
@@ -146,12 +139,12 @@ func Test_extractMetadata(t *testing.T) {
},
},
{
pkg: spdx.Package2_2{
pkg: spdx.Package{
PackageName: "SomeApkPkg",
PackageVersion: "3.2.9",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:alpine/pkg-2@7.3.1?arch=x86_64&upstream=apk-origin@9.1.3&distro=alpine-3.10.9",
RefType: "purl",
},
@@ -166,12 +159,12 @@ func Test_extractMetadata(t *testing.T) {
},
},
{
pkg: spdx.Package2_2{
pkg: spdx.Package{
PackageName: "SomeRpmPkg",
PackageVersion: "13.2.79",
PackageExternalReferences: []*spdx.PackageExternalReference2_2{
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE_MANAGER",
Category: "PACKAGE-MANAGER",
Locator: "pkg:rpm/pkg-2@7.3.1?arch=x86_64&epoch=1234&upstream=some-rpm-origin-1.16.3&distro=alpine-3.10.9",
RefType: "purl",
},
@@ -234,3 +227,84 @@ func TestExtractSourceFromNamespaces(t *testing.T) {
require.Equal(t, tt.expected, extractSchemeFromNamespace(tt.namespace))
}
}
func TestH1Digest(t *testing.T) {
tests := []struct {
name string
pkg spdx.Package
expectedDigest string
}{
{
name: "valid h1digest",
pkg: spdx.Package{
PackageName: "github.com/googleapis/gnostic",
PackageVersion: "v0.5.5",
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE-MANAGER",
Locator: "pkg:golang/github.com/googleapis/gnostic@v0.5.5",
RefType: "purl",
},
},
PackageChecksums: []common.Checksum{
{
Algorithm: common.SHA256,
Value: "f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
},
},
},
expectedDigest: "h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=",
},
{
name: "invalid h1digest algorithm",
pkg: spdx.Package{
PackageName: "github.com/googleapis/gnostic",
PackageVersion: "v0.5.5",
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE-MANAGER",
Locator: "pkg:golang/github.com/googleapis/gnostic@v0.5.5",
RefType: "purl",
},
},
PackageChecksums: []common.Checksum{
{
Algorithm: common.SHA1,
Value: "f5f1c0b4ad2e0dfa6f79eaaaa3586411925c16f61702208ddd4bad2fc17dc47c",
},
},
},
expectedDigest: "",
},
{
name: "invalid h1digest digest",
pkg: spdx.Package{
PackageName: "github.com/googleapis/gnostic",
PackageVersion: "v0.5.5",
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE-MANAGER",
Locator: "pkg:golang/github.com/googleapis/gnostic@v0.5.5",
RefType: "purl",
},
},
PackageChecksums: []common.Checksum{
{
Algorithm: common.SHA256,
Value: "",
},
},
},
expectedDigest: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
p := toSyftPackage(&test.pkg)
require.Equal(t, pkg.GolangBinMetadataType, p.MetadataType)
meta := p.Metadata.(pkg.GolangBinMetadata)
require.Equal(t, test.expectedDigest, meta.H1Digest)
})
}
}
+25
View File
@@ -2,8 +2,10 @@ package testutils
import (
"bytes"
"math/rand"
"strings"
"testing"
"time"
"github.com/sergi/go-diff/diffmatchpatch"
"github.com/stretchr/testify/assert"
@@ -12,6 +14,7 @@ import (
"github.com/anchore/stereoscope/pkg/filetree"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/stereoscope/pkg/imagetest"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
@@ -276,3 +279,25 @@ func newDirectoryCatalog() *pkg.Catalog {
return catalog
}
//nolint:gosec
func AddSampleFileRelationships(s *sbom.SBOM) {
catalog := s.Artifacts.PackageCatalog.Sorted()
s.Artifacts.FileMetadata = map[source.Coordinates]source.FileMetadata{}
files := []string{"/f1", "/f2", "/d1/f3", "/d2/f4", "/z1/f5", "/a1/f6"}
rnd := rand.New(rand.NewSource(time.Now().UnixNano()))
rnd.Shuffle(len(files), func(i, j int) { files[i], files[j] = files[j], files[i] })
for _, f := range files {
meta := source.FileMetadata{}
coords := source.Coordinates{RealPath: f}
s.Artifacts.FileMetadata[coords] = meta
s.Relationships = append(s.Relationships, artifact.Relationship{
From: catalog[0],
To: coords,
Type: artifact.ContainsRelationship,
})
}
}
+57
View File
@@ -0,0 +1,57 @@
package util
import (
"encoding/base64"
"encoding/hex"
"fmt"
"strings"
)
// HDigestToSHA converts a h# digest, such as h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= to an
// algorithm such as sha256 and a hex encoded digest
func HDigestToSHA(digest string) (string, string, error) {
// hash is base64, but we need hex encode
parts := strings.Split(digest, ":")
if len(parts) == 2 {
algo := parts[0]
hash := parts[1]
checksum, err := base64.StdEncoding.DecodeString(hash)
if err != nil {
return "", "", err
}
hexStr := hex.EncodeToString(checksum)
switch algo {
// golang h1 hash == sha256
case "h1":
algo = "sha256"
default:
return "", "", fmt.Errorf("unsupported h#digest algorithm: %s", algo)
}
return algo, hexStr, nil
}
return "", "", fmt.Errorf("invalid h#digest: %s", digest)
}
// HDigestFromSHA converts an algorithm, such sha256 with a hex encoded digest to a
// h# value such as h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
func HDigestFromSHA(algorithm string, digest string) (string, error) {
if digest == "" {
return "", fmt.Errorf("no digest value provided")
}
// digest is hex, but we need to base64 encode
algorithm = strings.ToLower(algorithm)
if algorithm == "sha256" {
checksum, err := hex.DecodeString(digest)
if err != nil {
return "", err
}
// hash is hex, but we need base64
b64digest := base64.StdEncoding.EncodeToString(checksum)
return fmt.Sprintf("h1:%s", b64digest), nil
}
return "", fmt.Errorf("not a recognized h#digest algorithm: %s", algorithm)
}
+107
View File
@@ -0,0 +1,107 @@
package util
import (
"fmt"
"strings"
"testing"
"github.com/stretchr/testify/require"
)
func Test_HDigestToSHA(t *testing.T) {
tests := []struct {
name string
hDigest string
expected string
error bool
}{
{
name: "valid h1digest",
hDigest: "h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=",
expected: "sha256:f10a9c0e0ceb52a9546ff1b63d04d68ae786a33b91d7cf3881d74ccb093a88b5",
error: false,
},
{
name: "other valid h1digest",
hDigest: "h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=",
expected: "sha256:4933fc0ef0f273f748e5bf13e61b21b648d2f84e364e7cac34250f7637221a16",
error: false,
},
{
name: "invalid h1digest",
hDigest: "h12:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=",
expected: "",
error: true,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
algo, digest, err := HDigestToSHA(test.hDigest)
if test.error {
require.Error(t, err)
return
} else {
require.NoError(t, err)
}
got := fmt.Sprintf("%s:%s", algo, digest)
require.Equal(t, test.expected, got)
})
}
}
func Test_HDigestFromSHA(t *testing.T) {
tests := []struct {
name string
sha string
expected string
error bool
}{
{
name: "valid sha",
sha: "sha256:f10a9c0e0ceb52a9546ff1b63d04d68ae786a33b91d7cf3881d74ccb093a88b5",
expected: "h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=",
error: false,
},
{
name: "other valid sha",
sha: "sha256:4933fc0ef0f273f748e5bf13e61b21b648d2f84e364e7cac34250f7637221a16",
expected: "h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=",
error: false,
},
{
name: "invalid sha",
expected: "h12:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=",
sha: "sha256:f10a9c0e0zzzzceb52a99968ae786a33b91d7cf3881d74ccb093a88b5",
error: true,
},
{
name: "invalid algorithm",
expected: "h12:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=",
sha: "sha1:f10a9c0e0ceb52a9546ff1b63d04d68ae786a33b91d7cf3881d74ccb093a88b5",
error: true,
},
{
name: "empty sha",
expected: "",
sha: "sha256:",
error: true,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
parts := strings.Split(test.sha, ":")
algo := parts[0]
digest := parts[1]
got, err := HDigestFromSHA(algo, digest)
if test.error {
require.Error(t, err)
return
} else {
require.NoError(t, err)
}
require.Equal(t, test.expected, got)
})
}
}

Some files were not shown because too many files have changed in this diff Show More