Compare commits

...
Author SHA1 Message Date
Will Murphy 4fb9970481 Prevent hang if stderr is tty but stdout is not
Previously, the rich TUI would be enabled if either stderr or stdout is
a TTY, but this could cause syft to hang if stderr was a TTY and stdout
was not, as in some cases of redirection.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
2023-07-29 13:36:27 -04:00
Christopher Angelo Phillips 3aae316456 chore: update to latest commit in tools-golang (#1969)
* chore: update to latest commit in tools-golang

---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-07-27 15:29:22 -04:00
Alex Goodmanandmanifestori 063e9da65d Guess unpinned versions in python requirements.txt (#1966)
* feat: python requirements.txt parsing inclusive

Signed-off-by: manifestori <ori@manifestcyber.com>

* refactor: parseVersion

Signed-off-by: manifestori <ori@manifestcyber.com>

* add python config for optional requirements version constraint resolution

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix tests

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* allow for python requirements metadata to be optional

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* restore cyclonedx dependency

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: manifestori <ori@manifestcyber.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: manifestori <ori@manifestcyber.com>
2023-07-27 14:26:59 -04:00
dependabot[bot] bf1102c3f1 chore(deps): bump github.com/vifraa/gopom from 0.2.1 to 0.2.2 (#1965)
Bumps [github.com/vifraa/gopom](https://github.com/vifraa/gopom) from 0.2.1 to 0.2.2.
- [Release notes](https://github.com/vifraa/gopom/releases)
- [Commits](https://github.com/vifraa/gopom/compare/v0.2.1...v0.2.2)

---
updated-dependencies:
- dependency-name: github.com/vifraa/gopom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-27 13:28:42 -04:00
Alex Goodman bbd2d42dbb Fix panic condition on docker pull failure (#1968)
* [wip] add image pull error handlers

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix panic and ui hang on docker pull failure

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* linter fix

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-27 11:32:02 -04:00
Alex Goodman d84120f499 bump JSON schema to account for simplified python env markers (#1967)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-27 14:13:17 +00:00
Keith Zantow 9480f10ccd feat: support top-level SPDX package and graph (#1934)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-07-26 13:54:32 -04:00
dependabot[bot] 1e4d26f526 chore(deps): bump github.com/go-git/go-git/v5 from 5.8.0 to 5.8.1 (#1959)
Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) from 5.8.0 to 5.8.1.
- [Release notes](https://github.com/go-git/go-git/releases)
- [Commits](https://github.com/go-git/go-git/compare/v5.8.0...v5.8.1)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-git/v5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-26 13:34:03 +00:00
Tristan Farkas e1c1832f84 Add cataloger for Swift Package Manager. (#1919)
Signed-off-by: Tristan Farkas <Tristan.Farkas@axis.com>
2023-07-25 14:35:21 -04:00
anchore-actions-token-generator[bot]andkzantow 9a73380f29 chore(deps): update stereoscope to d515761c6ca2743a67d7d08053db69235ae76d1d (#1953)
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: kzantow <kzantow@users.noreply.github.com>
2023-07-25 10:49:21 -04:00
dependabot[bot] 2e718cf865 chore(deps): bump github.com/docker/docker (#1955)
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.2+incompatible to 24.0.5+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v24.0.2...v24.0.5)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-25 10:37:16 -04:00
dependabot[bot] 4000a84624 chore(deps): bump github.com/go-git/go-git/v5 from 5.7.0 to 5.8.0 (#1951)
Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) from 5.7.0 to 5.8.0.
- [Release notes](https://github.com/go-git/go-git/releases)
- [Commits](https://github.com/go-git/go-git/compare/v5.7.0...v5.8.0)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-git/v5
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-24 11:28:54 -04:00
Dan LuhringandAlex Goodman 99d172f0d1 Introduce indexed embedded CPE dictionary (#1897)
* Introduce indexed embedded CPE dictionary

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Don't generate cpe-index on make snapshot

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Add unit tests for individual addEntry funcs

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* migrate CPE index build to go generate and add periodic workflow

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add test to ensure generated cpe index is wired up to function that uses it

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-21 13:54:19 +00:00
dependabot[bot] 3f5c601620 chore(deps): bump github.com/gookit/color from 1.5.3 to 1.5.4 (#1949)
Bumps [github.com/gookit/color](https://github.com/gookit/color) from 1.5.3 to 1.5.4.
- [Release notes](https://github.com/gookit/color/releases)
- [Commits](https://github.com/gookit/color/compare/v1.5.3...v1.5.4)

---
updated-dependencies:
- dependency-name: github.com/gookit/color
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-21 08:50:47 -04:00
Dan LuhringandAlex Goodman 8478e0bef7 Add support for parsing .NET assemblies (#1943)
* Add support for parsing .NET assemblies

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: 69c33fe4d77357d843c11590f3b07825bc6249ac

* Add dll and exe files

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: b9d204efa6d2ef385b5fbb7a59a3474ecabea641

* Add PE cataloger to directory catalogers

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: 9711c00d9da92e2887e0c1f92edd740ea5345849

* Don't set language to dotnet for PEs

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: 368313fddac9160d8a06a01ebe8c5ac7990232f5

* Fix spelling of cataloger in constructor

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: e42fd77b2f8b6d42e076a84f6cce386861260941

* Adjust which cases in PE parsing return errors

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

Former-commit-id: 95b25f8fc3a7d4e18fe30e489b09851f316795ff

* remove build binary from branch

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

Former-commit-id: fa54c0d0aef0998d5520e9f44cae51f5f9cd38a2

* Fix failing CLI tests

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

---------

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-19 15:34:07 -04:00
Christopher Angelo Phillips 0327fdc88a docs: capture artifactory dev settings from 1895 (#1947)
* docs: capture artifactory dev settings from 1895

---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-07-19 16:54:18 +00:00
Alex Goodman 88b3d1e9bb remove build binary and add explicit git ignore
Former-commit-id: 6455f2d8a5f1910b4a8b681ddef79919886d638d
2023-07-18 14:06:34 -04:00
Christopher Angelo Phillips 204b790012 docs: update docs with new docker specific instructions (#1941)
* docs: update docs with new docker specific instructions
---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Former-commit-id: c67c76e84df84e3e24aa307637d884ca8b7e3eea
2023-07-17 18:19:21 +00:00
Alex Goodman 35699f6fdc remove jotframe UI (#1932)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-13 13:21:52 -04:00
Christopher Angelo Phillips 2e7fd031d4 fix: remove indirect dependency of circl v1.1.0 (#1940)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-07-13 12:30:37 -04:00
Christopher Angelo Phillips 32296f5943 chore: move wait before iteration to guarantee read before tea (#1931)
* chore: move wait before iteration to guarantee read before tea

---------

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-07-12 13:59:31 -04:00
88 changed files with 34785 additions and 1171 deletions
+1 -1
View File
@@ -6,7 +6,7 @@ on:
workflow_dispatch:
env:
GO_VERSION: "1.19.x"
GO_VERSION: "1.20.x"
GO_STABLE_VERSION: true
jobs:
@@ -0,0 +1,43 @@
name: PR to update CPE dictionary index
on:
schedule:
- cron: "0 1 * * 1" # every monday at 1 AM
workflow_dispatch:
env:
GO_VERSION: "1.20.x"
GO_STABLE_VERSION: true
jobs:
upgrade-cpe-dictionary-index:
runs-on: ubuntu-latest
if: github.repository == 'anchore/syft' # only run for main repo
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- run: |
make generate-cpe-dictionary-index
- uses: tibdex/github-app-token@v1
id: generate-token
with:
app_id: ${{ secrets.TOKEN_APP_ID }}
private_key: ${{ secrets.TOKEN_APP_PRIVATE_KEY }}
- uses: peter-evans/create-pull-request@v5
with:
signoff: true
delete-branch: true
branch: auto/latest-cpe-dictionary-index
labels: dependencies
commit-message: "chore(deps): update CPE dictionary index"
title: "chore(deps): update CPE dictionary index"
body: |
Update CPE dictionary index based on the latest available CPE dictionary
token: ${{ steps.generate-token.outputs.token }}
@@ -6,7 +6,7 @@ on:
workflow_dispatch:
env:
GO_VERSION: "1.19.x"
GO_VERSION: "1.20.x"
GO_STABLE_VERSION: true
jobs:
+2
View File
@@ -2,6 +2,7 @@ go.work
go.work.sum
/bin
/.bin
/build
CHANGELOG.md
VERSION
/test/results
@@ -19,6 +20,7 @@ VERSION
*.hpi
*.zip
.idea/
*.iml
*.log
.images
.tmp/
+4
View File
@@ -5,3 +5,7 @@ The following Syft components were contributed by external authors/organizations
## GraalVM Native Image
A cataloger contributed by Oracle Corporation that extracts packages given within GraalVM Native Image SBOMs.
## Swift Package Manager
A cataloger contributed by Axis Communications that catalogs packages resolved by Swift Package Manager.
+39
View File
@@ -7,6 +7,25 @@ In order to test and develop in this repo you will need the following dependenci
- docker
- make
### Docker settings for getting started
Make sure you've updated your docker settings so the default docker socket path is available.
Go to:
docker -> settings -> advanced
Make sure:
```
Allow the default Docker socket to be used
```
is checked.
Also double check that the docker context being used is the default context. If it is not, run:
`docker context use default`
After cloning the following step can help you get setup:
1. run `make bootstrap` to download go mod dependencies, create the `/.tmp` dir, and download helper utilities.
2. run `make` to view the selection of developer commands in the Makefile
@@ -19,6 +38,26 @@ The main make tasks for common static analysis and testing are `lint`, `format`,
See `make help` for all the current make tasks.
### Internal Artifactory Settings
**Not always applicable**
Some companies have Artifactory setup internally as a solution for sourcing secure dependencies.
If you're seeing an issue where the unit tests won't run because of the below error then this section might be relevant for your use case.
```
[ERROR] [ERROR] Some problems were encountered while processing the POMs
```
If you're dealing with an issue where the unit tests will not pull/build certain java fixtures check some of these settings:
- a `settings.xml` file should be available to help you communicate with your internal artifactory deployment
- this can be moved to `syft/pkg/cataloger/java/test-fixtures/java-builds/example-jenkins-plugin/` to help build the unit test-fixtures
- you'll also want to modify the `build-example-jenkins-plugin.sh` to use `settings.xml`
For more information on this setup and troubleshooting see [issue 1895](https://github.com/anchore/syft/issues/1895#issuecomment-1610085319)
## Architecture
Syft is used to generate a Software Bill of Materials (SBOM) from different kinds of input.
+8 -2
View File
@@ -298,7 +298,7 @@ compare-test-rpm-package-install: $(TEMP_DIR) $(SNAPSHOT_DIR)
$(TEMP_DIR)
## Code generation targets #################################
## Code and data generation targets #################################
.PHONY: generate-json-schema
generate-json-schema: ## Generate a new json schema
@@ -309,11 +309,17 @@ generate-license-list: ## Generate an updated spdx license list
go generate ./internal/spdxlicense/...
gofmt -s -w ./internal/spdxlicense
.PHONY: generate-cpe-dictionary-index
generate-cpe-dictionary-index: ## Build the CPE index based off of the latest available CPE dictionary
$(call title,Building CPE index)
go generate ./syft/pkg/cataloger/common/cpe/dictionary
## Build-related targets #################################
.PHONY: build
build: $(SNAPSHOT_DIR) ## Build release snapshot binaries and packages
build:
CGO_ENABLED=0 go build -trimpath -ldflags "$(LDFLAGS)" -o $@ ./cmd/syft
$(SNAPSHOT_DIR): ## Build snapshot release binaries and packages
$(call title,Building snapshot artifacts)
+10 -1
View File
@@ -53,7 +53,7 @@ For commercial support options with Syft or Grype, please [contact Anchore](http
- Red Hat (rpm)
- Ruby (gem)
- Rust (cargo.lock)
- Swift (cocoapods)
- Swift (cocoapods, swift-package-manager)
## Installation
@@ -211,6 +211,7 @@ You can override the list of enabled/disabled catalogers by using the "cataloger
- ruby-gemfile
- rust-cargo-lock
- sbom
- swift-package-manager
##### Non Default:
- cargo-auditable-binary
@@ -521,6 +522,7 @@ platform: ""
# - ruby-gemspec-cataloger
# - rust-cargo-lock-cataloger
# - sbom-cataloger
# - spm-cataloger
catalogers:
# cataloging packages is exposed through the packages and power-user subcommands
@@ -575,6 +577,13 @@ linux-kernel:
# SYFT_LINUX_KERNEL_CATALOG_MODULES env var
catalog-modules: true
python:
# when running across entries in requirements.txt that do not specify a specific version
# (e.g. "sqlalchemy >= 1.0.0, <= 2.0.0, != 3.0.0, <= 3.0.0"), attempt to guess what the version could
# be based on the version requirements specified (e.g. "1.0.0"). When enabled the lowest expressible version
# when given an arbitrary constraint will be used (even if that version may not be available/published).
guess-unpinned-requirements: false
# cataloging file contents is exposed through the power-user subcommand
file-contents:
cataloger:
+9 -3
View File
@@ -15,6 +15,7 @@ import (
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/file"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
@@ -101,14 +102,19 @@ func execWorker(app *config.Application, userInput string, writer sbom.Writer) <
},
)
if src != nil {
defer src.Close()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", userInput, err)
return
}
defer func() {
if src != nil {
if err := src.Close(); err != nil {
log.Tracef("unable to close source: %+v", err)
}
}
}()
s, err := GenerateSBOM(src, errs, app)
if err != nil {
errs <- err
+6 -5
View File
@@ -19,6 +19,12 @@ func runModel(t testing.TB, m tea.Model, iterations int, message tea.Msg, h ...*
return message
}
for _, each := range h {
if each != nil {
each.Wait()
}
}
for i := 0; cmd != nil && i < iterations; i++ {
msgs := flatten(cmd())
var nextCmds []tea.Cmd
@@ -31,11 +37,6 @@ func runModel(t testing.TB, m tea.Model, iterations int, message tea.Msg, h ...*
cmd = tea.Batch(nextCmds...)
}
for _, each := range h {
if each != nil {
each.Wait()
}
}
return m.View()
}
+1 -1
View File
@@ -21,7 +21,7 @@ import (
func Select(verbose, quiet bool) (uis []clio.UI) {
isStdoutATty := term.IsTerminal(int(os.Stdout.Fd()))
isStderrATty := term.IsTerminal(int(os.Stderr.Fd()))
notATerminal := !isStderrATty && !isStdoutATty
notATerminal := !isStderrATty || !isStdoutATty
switch {
case runtime.GOOS == "windows" || verbose || quiet || notATerminal || !isStderrATty:
+5 -2
View File
@@ -82,12 +82,15 @@ func (m *UI) Teardown(force bool) error {
if !force {
m.handler.Running.Wait()
m.program.Quit()
// typically in all cases we would want to wait for the UI to finish. However there are still error cases
// that are not accounted for, resulting in hangs. For now, we'll just wait for the UI to finish in the
// happy path only. There will always be an indication of the problem to the user via reporting the error
// string from the worker (outside of the UI after teardown).
m.running.Wait()
} else {
m.program.Kill()
}
m.running.Wait()
// TODO: allow for writing out the full log output to the screen (only a partial log is shown currently)
// this needs coordination to know what the last frame event is to change the state accordingly (which isn't possible now)
+16 -11
View File
@@ -16,7 +16,7 @@ require (
github.com/go-test/deep v1.1.0
github.com/google/go-cmp v0.5.9
github.com/google/uuid v1.3.0
github.com/gookit/color v1.5.3
github.com/gookit/color v1.5.4
github.com/hashicorp/go-multierror v1.1.1
github.com/jinzhu/copier v0.3.5
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
@@ -31,16 +31,15 @@ require (
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e
github.com/sergi/go-diff v1.3.1
github.com/sirupsen/logrus v1.9.3
github.com/spdx/tools-golang v0.5.2
github.com/spdx/tools-golang v0.5.3
github.com/spf13/afero v1.9.5
github.com/spf13/cobra v1.7.0
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.16.0
github.com/stretchr/testify v1.8.4
github.com/vifraa/gopom v0.2.1
github.com/vifraa/gopom v0.2.2
github.com/wagoodman/go-partybus v0.0.0-20230516145632-8ccac152c651
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/mod v0.12.0
golang.org/x/net v0.12.0
@@ -55,21 +54,24 @@ require (
github.com/anchore/bubbly v0.0.0-20230712165553-812110ab0a10
github.com/anchore/clio v0.0.0-20230602170917-e747e60c4aa0
github.com/anchore/go-logger v0.0.0-20230531193951-db5ae83e7dbe
github.com/anchore/stereoscope v0.0.0-20230627195312-cd49355d934e
github.com/anchore/stereoscope v0.0.0-20230724160817-d515761c6ca2
github.com/aquasecurity/go-pep440-version v0.0.0-20210121094942-22b2f8951d46
github.com/charmbracelet/bubbletea v0.24.2
github.com/charmbracelet/lipgloss v0.7.1
github.com/dave/jennifer v1.6.1
github.com/deitch/magic v0.0.0-20230404182410-1ff89d7342da
github.com/docker/docker v24.0.2+incompatible
github.com/docker/distribution v2.8.2+incompatible
github.com/docker/docker v24.0.5+incompatible
github.com/github/go-spdx/v2 v2.1.2
github.com/gkampitakis/go-snaps v0.4.7
github.com/go-git/go-billy/v5 v5.4.1
github.com/go-git/go-git/v5 v5.7.0
github.com/go-git/go-git/v5 v5.8.1
github.com/google/go-containerregistry v0.15.2
github.com/google/licensecheck v0.3.1
github.com/invopop/jsonschema v0.7.0
github.com/knqyf263/go-rpmdb v0.0.0-20230301153543-ba94b245509b
github.com/opencontainers/go-digest v1.0.0
github.com/saferwall/pe v1.4.4
github.com/sassoftware/go-rpmutils v0.2.0
github.com/vbatts/go-mtree v0.5.3
github.com/zyedidia/generic v1.2.2-0.20230320175451-4410d2372cb1
@@ -79,15 +81,17 @@ require (
)
require (
dario.cat/mergo v1.0.0 // indirect
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
github.com/DataDog/zstd v1.4.5 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.2.0 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20230518184743-7afd39499903 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95 // indirect
github.com/acomagu/bufpipe v1.0.4 // indirect
github.com/anchore/fangs v0.0.0-20230531202914-48a718c6b4ba // indirect
github.com/anchore/go-struct-converter v0.0.0-20221118182256-c68fdcfa2092 // indirect
github.com/aquasecurity/go-version v0.0.0-20210121072130-637058cfe492 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/becheran/wildmatch-go v1.0.0 // indirect
github.com/charmbracelet/bubbles v0.16.1 // indirect
@@ -98,11 +102,11 @@ require (
github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/docker/cli v23.0.5+incompatible // indirect
github.com/docker/distribution v2.8.2+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
github.com/edsrzf/mmap-go v1.1.0 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/felixge/fgprof v0.9.3 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
@@ -154,12 +158,12 @@ require (
github.com/rivo/uniseg v0.2.0 // indirect
github.com/rogpeppe/go-internal v1.9.0 // indirect
github.com/shopspring/decimal v1.2.0 // indirect
github.com/skeema/knownhosts v1.1.1 // indirect
github.com/skeema/knownhosts v1.2.0 // indirect
github.com/spf13/cast v1.5.1 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/stretchr/objx v0.5.0 // indirect
github.com/subosito/gotenv v1.4.2 // indirect
github.com/sylabs/sif/v2 v2.8.1 // indirect
github.com/sylabs/sif/v2 v2.11.5 // indirect
github.com/sylabs/squashfs v0.6.1 // indirect
github.com/therootcompany/xz v1.0.1 // indirect
github.com/tidwall/gjson v1.14.4 // indirect
@@ -173,6 +177,7 @@ require (
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.10.0 // indirect
golang.org/x/text v0.11.0 // indirect
+37 -26
View File
@@ -48,6 +48,8 @@ cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohl
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
cloud.google.com/go/storage v1.14.0/go.mod h1:GrKmX003DSIwi9o29oFT7YDnHYwZoctc3fOKtUw0Xmo=
dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 h1:UQHMgLO+TxOElx5B5HZ4hJQsoJ/PvUvKRhJHDQXO8P8=
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
@@ -72,8 +74,8 @@ github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v
github.com/Microsoft/go-winio v0.6.1 h1:9/kr64B9VUZrLm5YYwbGtUJnMgqWVOdUAXu6Migciow=
github.com/Microsoft/go-winio v0.6.1/go.mod h1:LRdKpFKfdobln8UmuiYcKPot9D2v6svN5+sAH+4kjUM=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/ProtonMail/go-crypto v0.0.0-20230518184743-7afd39499903 h1:ZK3C5DtzV2nVAQTx5S5jQvMeDqWtD1By5mOoyY/xJek=
github.com/ProtonMail/go-crypto v0.0.0-20230518184743-7afd39499903/go.mod h1:8TI4H3IbrackdNgv+92dI+rhpCaLqM0IfpgCgenFvRE=
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95 h1:KLq8BE0KwCL+mmXnjLWEAOYO+2l2AE4YMmqG1ZpZHBs=
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95/go.mod h1:EjAoLdwvbIOoOQr3ihjnSoLZRtE8azugULFRteWMNc0=
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d h1:licZJFw2RwpHMqeKTCYkitsPqHNxTmd4SNR5r94FGM8=
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d/go.mod h1:asat636LX7Bqt5lYEZ27JNDcqxfjdBQuJ/MM4CN/Lzo=
github.com/acobaugh/osrelease v0.1.0 h1:Yb59HQDGGNhCj4suHaFQQfBps5wyoKLSSX/J/+UifRE=
@@ -104,14 +106,18 @@ github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZV
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501 h1:AV7qjwMcM4r8wFhJq3jLRztew3ywIyPTRapl2T1s9o8=
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501/go.mod h1:Blo6OgJNiYF41ufcgHKkbCKF2MDOMlrqhXv/ij6ocR4=
github.com/anchore/stereoscope v0.0.0-20230627195312-cd49355d934e h1:zhk3ZLtomMJ750nNCE+c24PonMzoO/SeL/4uTr1L9kM=
github.com/anchore/stereoscope v0.0.0-20230627195312-cd49355d934e/go.mod h1:0LsgHgXO4QFnk2hsYwtqd3fR18PIZXlFLIl2qb9tu3g=
github.com/anchore/stereoscope v0.0.0-20230724160817-d515761c6ca2 h1:qIxlZFFZIl5YNHwxsVQ6UJG8QtkR3W0W9ozaI9JQiPk=
github.com/anchore/stereoscope v0.0.0-20230724160817-d515761c6ca2/go.mod h1:0LsgHgXO4QFnk2hsYwtqd3fR18PIZXlFLIl2qb9tu3g=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
github.com/andybalholm/brotli v1.0.4 h1:V7DdXeJtZscaqfNuAdSRuRFzuiKlHSC/Zh3zl9qY3JY=
github.com/andybalholm/brotli v1.0.4/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/aquasecurity/go-pep440-version v0.0.0-20210121094942-22b2f8951d46 h1:vmXNl+HDfqqXgr0uY1UgK1GAhps8nbAAtqHNBcgyf+4=
github.com/aquasecurity/go-pep440-version v0.0.0-20210121094942-22b2f8951d46/go.mod h1:olhPNdiiAAMiSujemd1O/sc6GcyePr23f/6uGKtthNg=
github.com/aquasecurity/go-version v0.0.0-20210121072130-637058cfe492 h1:rcEG5HI490FF0a7zuvxOxen52ddygCfNVjP0XOCMl+M=
github.com/aquasecurity/go-version v0.0.0-20210121072130-637058cfe492/go.mod h1:9Beu8XsUNNfzml7WBf3QmyPToP1wm1Gj/Vc5UJKqTzU=
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=
github.com/armon/go-metrics v0.3.10/go.mod h1:4O98XIr/9W0sxpJ8UaYkvjk10Iff7SnFrb4QAOwNTFc=
@@ -129,7 +135,7 @@ github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kB
github.com/bmatcuk/doublestar/v4 v4.6.0 h1:HTuxyug8GyFbRkrffIpzNCSK4luc0TY3wzXvzIZhEXc=
github.com/bmatcuk/doublestar/v4 v4.6.0/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/bradleyjkemp/cupaloy/v2 v2.8.0 h1:any4BmKE+jGIaMpnU8YgH/I2LPiLBufr6oMMlVBbn9M=
github.com/bwesterb/go-ristretto v1.2.0/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
github.com/bwesterb/go-ristretto v1.2.3/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/census-instrumentation/opencensus-proto v0.3.0/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
@@ -149,7 +155,6 @@ github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMn
github.com/circonus-labs/circonus-gometrics v2.3.1+incompatible/go.mod h1:nmEj6Dob7S7YxXgwXpfOuvO54S+tGdZdw9fuRZt25Ag=
github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp5jckzBHf4XRpQvBOLI+I=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cloudflare/circl v1.1.0/go.mod h1:prBCrKB9DV4poKZY1l9zBXg2QJY7mvgRvtMxxK7fi4I=
github.com/cloudflare/circl v1.3.3 h1:fE/Qz0QdIGqeWfnwq0RE0R7MI51s0M2E4Ga9kq5AEMs=
github.com/cloudflare/circl v1.3.3/go.mod h1:5XYMA4rFBvNIrhs50XuiBJ15vF2pZn4nnUKZrLbUZFA=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
@@ -170,6 +175,7 @@ github.com/containerd/stargz-snapshotter/estargz v0.14.3 h1:OqlDCK3ZVUO6C3B/5FSk
github.com/containerd/stargz-snapshotter/estargz v0.14.3/go.mod h1:KY//uOCIkSuNAHhJogcZtrNHdKrA99/FCCRjE3HD36o=
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/cpuguy83/go-md2man/v2 v2.0.1/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
@@ -185,8 +191,8 @@ github.com/docker/cli v23.0.5+incompatible h1:ufWmAOuD3Vmr7JP2G5K3cyuNC4YZWiAsuD
github.com/docker/cli v23.0.5+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.2+incompatible h1:T3de5rq0dB1j30rp0sA2rER+m322EBzniBPB6ZIzuh8=
github.com/docker/distribution v2.8.2+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v24.0.2+incompatible h1:eATx+oLz9WdNVkQrr0qjQ8HvRJ4bOOxfzEo8R+dA3cg=
github.com/docker/docker v24.0.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v24.0.5+incompatible h1:WmgcE4fxyI6EEXxBRxsHnZXrO1pQ3smi0k/jho4HLeY=
github.com/docker/docker v24.0.5+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.7.0 h1:xtCHsjxogADNZcdv1pKUHXryefjlVRqWqIhk/uXJp0A=
github.com/docker/docker-credential-helpers v0.7.0/go.mod h1:rETQfLdHNT3foU5kuNkFR1R1V12OJRRO5lzt2D1b5X0=
github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
@@ -198,6 +204,8 @@ github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5/go.mod h1:qssHWj6
github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdfkVLjJ8T6VcRQv3SXugXy999NBtR9aFY=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/edsrzf/mmap-go v1.1.0 h1:6EUwBLQ/Mcr1EYLE4Tn1VdW1A4ckqCQWZBw8Hr0kjpQ=
github.com/edsrzf/mmap-go v1.1.0/go.mod h1:19H/e8pUPLicwkyNgOykDXkJ9F0MHE+Z52B8EIth78Q=
github.com/elazarl/goproxy v0.0.0-20221015165544-a0805db90819 h1:RIB4cRk+lBqKK3Oy0r2gRX4ui7tuhiZq2SuTtTCi0/0=
github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ=
@@ -244,8 +252,8 @@ github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmS
github.com/go-git/go-billy/v5 v5.4.1 h1:Uwp5tDRkPr+l/TnbHOQzp+tmJfLceOlbVucgpTz8ix4=
github.com/go-git/go-billy/v5 v5.4.1/go.mod h1:vjbugF6Fz7JIflbVpl1hJsGjSHNltrSw45YK/ukIvQg=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20230305113008-0c11038e723f h1:Pz0DHeFij3XFhoBRGUDPzSJ+w2UcK5/0JvF8DRI58r8=
github.com/go-git/go-git/v5 v5.7.0 h1:t9AudWVLmqzlo+4bqdf7GY+46SUuRsx59SboFxkq2aE=
github.com/go-git/go-git/v5 v5.7.0/go.mod h1:coJHKEOk5kUClpsNlXrUvPrDxY3w3gjHvhcZd8Fodw8=
github.com/go-git/go-git/v5 v5.8.1 h1:Zo79E4p7TRk0xoRgMq0RShiTHGKcKI4+DI6BfJc/Q+A=
github.com/go-git/go-git/v5 v5.8.1/go.mod h1:FHFuoD6yGz5OSKEBK+aWN9Oah0q54Jxl0abmj6GnqAo=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
@@ -355,8 +363,8 @@ github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pf
github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM=
github.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g=
github.com/gookit/color v1.2.5/go.mod h1:AhIE+pS6D4Ql0SQWbBeXPHw7gY0/sjHoA4s/n1KB7xg=
github.com/gookit/color v1.5.3 h1:twfIhZs4QLCtimkP7MOxlF3A0U/5cDPseRT9M/+2SCE=
github.com/gookit/color v1.5.3/go.mod h1:NUzwzeehUfl7GIb36pqId+UGmRfQcU/WiiyTTeNjHtE=
github.com/gookit/color v1.5.4 h1:FZmqs7XOyGgCAxmWyPslpiok1k05wmY3SJTytgvYFs0=
github.com/gookit/color v1.5.4/go.mod h1:pZJOeOS8DM43rXbp4AZo1n9zCU2qjpcRko0b6/QJi9w=
github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
github.com/hashicorp/consul/api v1.11.0/go.mod h1:XjsvQN+RJGWI2TWy1/kqaE16HrR2J/FWgkYjdZQsX9M=
github.com/hashicorp/consul/sdk v0.8.0/go.mod h1:GBvyrGALthsZObzUGsfgHZQDXjg4lOjagTIwIR1vPms=
@@ -420,7 +428,6 @@ github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHm
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213/go.mod h1:vNUNkEQ1e29fT/6vq2aBdFsgNPmy8qMdSay1npru+Sw=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/kevinburke/ssh_config v1.2.0 h1:x584FjTGwHzMwvHx18PXxbBVzfnxogHaAReU4gf13a4=
@@ -467,7 +474,6 @@ github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.6/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84=
github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
@@ -584,8 +590,11 @@ github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6L
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
github.com/saferwall/pe v1.4.4 h1:Ml++7/2/Z1iKwV4zCsd1nIqTEAdUQKAetwbbcCarhOg=
github.com/saferwall/pe v1.4.4/go.mod h1:SNzv3cdgk8SBI0UwHfyTcdjawfdnN+nbydnEL7GZ25s=
github.com/sagikazarmark/crypt v0.3.0/go.mod h1:uD/D+6UF4SrIR1uGEv7bBNkNqLGqUr43MRiaGWX1Nig=
github.com/sassoftware/go-rpmutils v0.2.0 h1:pKW0HDYMFWQ5b4JQPiI3WI12hGsVoW0V8+GMoZiI/JE=
github.com/sassoftware/go-rpmutils v0.2.0/go.mod h1:TJJQYtLe/BeEmEjelI3b7xNZjzAukEkeWKmoakvaOoI=
@@ -599,18 +608,19 @@ github.com/sergi/go-diff v1.3.1 h1:xkr+Oxo4BOQKmkn/B9eMK0g5Kg/983T9DqqPHwYqD+8=
github.com/sergi/go-diff v1.3.1/go.mod h1:aMJSSKb2lpPvRNec0+w3fl7LP9IOFzdc9Pa4NFbPK1I=
github.com/shopspring/decimal v1.2.0 h1:abSATXmQEYyShuxI4/vyW3tV1MrKAJzCZ/0zLUXYbsQ=
github.com/shopspring/decimal v1.2.0/go.mod h1:DKyhrW/HYNuLGql+MJL6WCR6knT2jwCFRcu2hWCYk4o=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/skeema/knownhosts v1.1.1 h1:MTk78x9FPgDFVFkDLTrsnnfCJl7g1C/nnKvePgrIngE=
github.com/skeema/knownhosts v1.1.1/go.mod h1:g4fPeYpque7P0xefxtGzV81ihjC8sX2IqpAoNkjxbMo=
github.com/skeema/knownhosts v1.2.0 h1:h9r9cf0+u7wSE+M183ZtMGgOJKiL96brpaz5ekfJCpM=
github.com/skeema/knownhosts v1.2.0/go.mod h1:g4fPeYpque7P0xefxtGzV81ihjC8sX2IqpAoNkjxbMo=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spdx/gordf v0.0.0-20201111095634-7098f93598fb/go.mod h1:uKWaldnbMnjsSAXRurWqqrdyZen1R7kxl8TkmWk2OyM=
github.com/spdx/tools-golang v0.5.2 h1:dtMNjJreWPe37584ajk7m/rQtfJaLpRMk7pUGgvekOg=
github.com/spdx/tools-golang v0.5.2/go.mod h1:/ETOahiAo96Ob0/RAIBmFZw6XN0yTnyr/uFZm2NTMhI=
github.com/spdx/tools-golang v0.5.3 h1:ialnHeEYUC4+hkm5vJm4qz2x+oEJbS0mAMFrNXdQraY=
github.com/spdx/tools-golang v0.5.3/go.mod h1:/ETOahiAo96Ob0/RAIBmFZw6XN0yTnyr/uFZm2NTMhI=
github.com/spf13/afero v1.3.3/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4=
github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I=
github.com/spf13/afero v1.9.5 h1:stMpOSZFs//0Lv29HduCmli3GUfpFoF3Y1Q/aXj/wVM=
@@ -650,8 +660,8 @@ github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXl
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/subosito/gotenv v1.4.2 h1:X1TuBLAMDFbaTAChgCBLu3DU3UPyELpnF2jjJ2cz/S8=
github.com/subosito/gotenv v1.4.2/go.mod h1:ayKnFf/c6rvx/2iiLrJUk1e6plDbT3edrFNGqEflhK0=
github.com/sylabs/sif/v2 v2.8.1 h1:whr4Vz12RXfLnYyVGHoD/rD/hbF2g9OW7BJHa+WIqW8=
github.com/sylabs/sif/v2 v2.8.1/go.mod h1:LQOdYXC9a8i7BleTKRw9lohi0rTbXkJOeS9u0ebvgyM=
github.com/sylabs/sif/v2 v2.11.5 h1:7ssPH3epSonsTrzbS1YxeJ9KuqAN7ISlSM61a7j/mQM=
github.com/sylabs/sif/v2 v2.11.5/go.mod h1:GBoZs9LU3e4yJH1dcZ3Akf/jsqYgy5SeguJQC+zd75Y=
github.com/sylabs/squashfs v0.6.1 h1:4hgvHnD9JGlYWwT0bPYNt9zaz23mAV3Js+VEgQoRGYQ=
github.com/sylabs/squashfs v0.6.1/go.mod h1:ZwpbPCj0ocIvMy2br6KZmix6Gzh6fsGQcCnydMF+Kx8=
github.com/therootcompany/xz v1.0.1 h1:CmOtsn1CbtmyYiusbfmhmkpAAETj0wBIH6kCYaX+xzw=
@@ -672,18 +682,17 @@ github.com/ulikunitz/xz v0.5.9/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oW
github.com/ulikunitz/xz v0.5.10 h1:t92gobL9l3HE202wg3rlk19F6X+JOxl9BBrCCMYEYd8=
github.com/ulikunitz/xz v0.5.10/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/urfave/cli v1.22.12/go.mod h1:sSBEIC79qR6OvcmsD4U3KABeOTxDqQtdDnaFuUN30b8=
github.com/urfave/cli/v2 v2.3.0/go.mod h1:LJmUH05zAU44vOAcrfzZQKsZbVcdbOG8rtL3/XcUArI=
github.com/vbatts/go-mtree v0.5.3 h1:S/jYlfG8rZ+a0bhZd+RANXejy7M4Js8fq9U+XoWTd5w=
github.com/vbatts/go-mtree v0.5.3/go.mod h1:eXsdoPMdL2jcJx6HweWi9lYQxBsTp4lNhqqAjgkZUg8=
github.com/vbatts/tar-split v0.11.3 h1:hLFqsOLQ1SsppQNTMpkpPXClLDfC2A3Zgy9OUU+RVck=
github.com/vbatts/tar-split v0.11.3/go.mod h1:9QlHN18E+fEH7RdG+QAJJcuya3rqT7eXSTY7wGrAokY=
github.com/vifraa/gopom v0.2.1 h1:MYVMAMyiGzXPPy10EwojzKIL670kl5Zbae+o3fFvQEM=
github.com/vifraa/gopom v0.2.1/go.mod h1:oPa1dcrGrtlO37WPDBm5SqHAT+wTgF8An1Q71Z6Vv4o=
github.com/vifraa/gopom v0.2.2 h1:zrqoCUVIplcsETouv3xxHPvfI/WV1GUPrdX2+Diahzo=
github.com/vifraa/gopom v0.2.2/go.mod h1:oPa1dcrGrtlO37WPDBm5SqHAT+wTgF8An1Q71Z6Vv4o=
github.com/wagoodman/go-partybus v0.0.0-20230516145632-8ccac152c651 h1:jIVmlAFIqV3d+DOxazTR9v+zgj8+VYuQBzPgBZvWBHA=
github.com/wagoodman/go-partybus v0.0.0-20230516145632-8ccac152c651/go.mod h1:b26F2tHLqaoRQf8DywqzVaV1MQ9yvjb0OMcNl7Nxu20=
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5 h1:lwgTsTy18nYqASnH58qyfRW/ldj7Gt2zzBvgYPzdA4s=
github.com/wagoodman/go-progress v0.0.0-20230301185719-21920a456ad5/go.mod h1:jLXFoL31zFaHKAAyZUh+sxiTDFe1L1ZHrcK2T1itVKA=
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb h1:Yz6VVOcLuWLAHYlJzTw7JKnWxdV/WXpug2X0quEzRnY=
github.com/wagoodman/jotframe v0.0.0-20211129225309-56b0d0a4aebb/go.mod h1:nDi3BAC5nEbVbg+WSJDHLbjHv0ZToq8nMPA97XMxF3E=
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
@@ -708,6 +717,8 @@ github.com/zyedidia/generic v1.2.2-0.20230320175451-4410d2372cb1/go.mod h1:ly2RB
go.etcd.io/etcd/api/v3 v3.5.1/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs=
go.etcd.io/etcd/client/pkg/v3 v3.5.1/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g=
go.etcd.io/etcd/client/v2 v2.305.1/go.mod h1:pMEacxZW7o8pg4CrFE7pquyCJJzZvkvdD2RibOCCCGs=
go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352 h1:CCriYyAfq1Br1aIYettdHZTy8mBTIPo7We18TuO/bak=
go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352/go.mod h1:SNgMg+EgDFwmvSmLRTNKC5fegJjB7v23qTQ0XLGUNHk=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
@@ -723,7 +734,6 @@ go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9i
go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190228161510-8dd112bcdc25/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
@@ -738,6 +748,7 @@ golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5y
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.3.0/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/crypto v0.3.1-0.20221117191849-2c476679df9a/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
golang.org/x/crypto v0.11.0 h1:6Ewdq3tDic1mg5xRO4milcWCfMVQhI4NkqWWvqejpuA=
golang.org/x/crypto v0.11.0/go.mod h1:xgJhtzW8F9jGdVFWZESrid1U1bjeNy4zgy5cRr/CIio=
@@ -934,6 +945,7 @@ golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211205182925-97ca703d548d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -949,7 +961,6 @@ golang.org/x/sys v0.10.0 h1:SqMFp9UcQJZa+pmYuAKjd9xq1f0j5rLcDIk0mj4qAsA=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
+5
View File
@@ -20,6 +20,7 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger"
golangCataloger "github.com/anchore/syft/syft/pkg/cataloger/golang"
"github.com/anchore/syft/syft/pkg/cataloger/kernel"
pythonCataloger "github.com/anchore/syft/syft/pkg/cataloger/python"
)
var (
@@ -52,6 +53,7 @@ type Application struct {
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
Golang golang `yaml:"golang" json:"golang" mapstructure:"golang"`
LinuxKernel linuxKernel `yaml:"linux-kernel" json:"linux-kernel" mapstructure:"linux-kernel"`
Python python `yaml:"python" json:"python" mapstructure:"python"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
FileMetadata FileMetadata `yaml:"file-metadata" json:"file-metadata" mapstructure:"file-metadata"`
FileClassification fileClassification `yaml:"file-classification" json:"file-classification" mapstructure:"file-classification"`
@@ -85,6 +87,9 @@ func (cfg Application) ToCatalogerConfig() cataloger.Config {
LinuxKernel: kernel.LinuxCatalogerConfig{
CatalogModules: cfg.LinuxKernel.CatalogModules,
},
Python: pythonCataloger.CatalogerConfig{
GuessUnpinnedRequirements: cfg.Python.GuessUnpinnedRequirements,
},
}
}
+13
View File
@@ -0,0 +1,13 @@
package config
import (
"github.com/spf13/viper"
)
type python struct {
GuessUnpinnedRequirements bool `json:"guess-unpinned-requirements" yaml:"guess-unpinned-requirements" mapstructure:"guess-unpinned-requirements"`
}
func (cfg python) loadDefaultValues(v *viper.Viper) {
v.SetDefault("python.guess-unpinned-requirements", false)
}
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "9.0.0"
JSONSchemaVersion = "10.0.0"
)
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -4,12 +4,12 @@ import (
"github.com/anchore/syft/syft/source"
)
func DocumentName(srcMetadata source.Description) string {
if srcMetadata.Name != "" {
return srcMetadata.Name
func DocumentName(src source.Description) string {
if src.Name != "" {
return src.Name
}
switch metadata := srcMetadata.Metadata.(type) {
switch metadata := src.Metadata.(type) {
case source.StereoscopeImageSourceMetadata:
return metadata.UserInput
case source.DirectorySourceMetadata:
@@ -17,13 +17,11 @@ func Test_DocumentName(t *testing.T) {
tests := []struct {
name string
inputName string
srcMetadata source.Description
expected string
}{
{
name: "image",
inputName: "my-name",
name: "image",
srcMetadata: source.Description{
Metadata: source.StereoscopeImageSourceMetadata{
UserInput: "image-repo/name:tag",
@@ -34,21 +32,27 @@ func Test_DocumentName(t *testing.T) {
expected: "image-repo/name:tag",
},
{
name: "directory",
inputName: "my-name",
name: "directory",
srcMetadata: source.Description{
Metadata: source.DirectorySourceMetadata{Path: "some/path/to/place"},
},
expected: "some/path/to/place",
},
{
name: "file",
inputName: "my-name",
name: "file",
srcMetadata: source.Description{
Metadata: source.FileSourceMetadata{Path: "some/path/to/place"},
},
expected: "some/path/to/place",
},
{
name: "named",
srcMetadata: source.Description{
Name: "some/name",
Metadata: source.FileSourceMetadata{Path: "some/path/to/place"},
},
expected: "some/name",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
@@ -54,6 +54,8 @@ func SourceInfo(p pkg.Package) string {
answer = "acquired package info from nix store path"
case pkg.Rpkg:
answer = "acquired package info from R-package DESCRIPTION file"
case pkg.SwiftPkg:
answer = "acquired package info from resolved Swift package manifest"
default:
answer = "acquired package info from the following paths"
}
@@ -231,6 +231,14 @@ func Test_SourceInfo(t *testing.T) {
"acquired package info from R-package DESCRIPTION file",
},
},
{
input: pkg.Package{
Type: pkg.SwiftPkg,
},
expected: []string{
"from resolved Swift package manifest",
},
},
}
var pkgTypes []pkg.Type
for _, test := range tests {
@@ -9,10 +9,12 @@ import (
"strings"
"time"
"github.com/docker/distribution/reference"
"github.com/spdx/tools-golang/spdx"
"golang.org/x/exp/maps"
"golang.org/x/exp/slices"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/spdxlicense"
@@ -21,10 +23,20 @@ import (
"github.com/anchore/syft/syft/formats/common/util"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
const (
noAssertion = "NOASSERTION"
spdxPrimaryPurposeContainer = "CONTAINER"
spdxPrimaryPurposeFile = "FILE"
spdxPrimaryPurposeOther = "OTHER"
prefixImage = "Image"
prefixDirectory = "Directory"
prefixFile = "File"
prefixUnknown = "Unknown"
)
// ToFormatModel creates and populates a new SPDX document struct that follows the SPDX 2.3
@@ -33,23 +45,37 @@ const (
//nolint:funlen
func ToFormatModel(s sbom.SBOM) *spdx.Document {
name, namespace := DocumentNameAndNamespace(s.Source)
packages := toPackages(s.Artifacts.Packages, s)
relationships := toRelationships(s.RelationshipsSorted())
// for valid SPDX we need a document describes relationship
// TODO: remove this placeholder after deciding on correct behavior
// for the primary package purpose field:
// https://spdx.github.io/spdx-spec/v2.3/package-information/#724-primary-package-purpose-field
describesID := spdx.ElementID("DOCUMENT")
rootPackage := toRootPackage(s.Source)
if rootPackage != nil {
describesID = rootPackage.PackageSPDXIdentifier
// add all relationships from the document root to all other packages
relationships = append(relationships, toRootRelationships(rootPackage, packages)...)
// append the root package
packages = append(packages, rootPackage)
}
// add a relationship for the package the document describes
documentDescribesRelationship := &spdx.Relationship{
RefA: spdx.DocElementID{
ElementRefID: "DOCUMENT",
},
Relationship: string(DescribesRelationship),
RefB: spdx.DocElementID{
ElementRefID: "DOCUMENT",
ElementRefID: describesID,
},
RelationshipComment: "",
}
// add the root document relationship
relationships = append(relationships, documentDescribesRelationship)
return &spdx.Document{
@@ -123,19 +149,130 @@ func ToFormatModel(s sbom.SBOM) *spdx.Document {
// Cardinality: optional, one
CreatorComment: "",
},
Packages: toPackages(s.Artifacts.Packages, s),
Packages: packages,
Files: toFiles(s),
Relationships: relationships,
OtherLicenses: toOtherLicenses(s.Artifacts.Packages),
}
}
func toRootRelationships(rootPackage *spdx.Package, packages []*spdx.Package) (out []*spdx.Relationship) {
for _, p := range packages {
out = append(out, &spdx.Relationship{
RefA: spdx.DocElementID{
ElementRefID: rootPackage.PackageSPDXIdentifier,
},
Relationship: string(ContainsRelationship),
RefB: spdx.DocElementID{
ElementRefID: p.PackageSPDXIdentifier,
},
})
}
return
}
//nolint:funlen
func toRootPackage(s source.Description) *spdx.Package {
var prefix string
name := s.Name
version := s.Version
var purl *packageurl.PackageURL
purpose := ""
var checksums []spdx.Checksum
switch m := s.Metadata.(type) {
case source.StereoscopeImageSourceMetadata:
prefix = prefixImage
purpose = spdxPrimaryPurposeContainer
qualifiers := packageurl.Qualifiers{
{
Key: "arch",
Value: m.Architecture,
},
}
ref, _ := reference.Parse(m.UserInput)
if ref, ok := ref.(reference.NamedTagged); ok {
qualifiers = append(qualifiers, packageurl.Qualifier{
Key: "tag",
Value: ref.Tag(),
})
}
c := toChecksum(m.ManifestDigest)
if c != nil {
checksums = append(checksums, *c)
purl = &packageurl.PackageURL{
Type: "oci",
Name: s.Name,
Version: m.ManifestDigest,
Qualifiers: qualifiers,
}
}
case source.DirectorySourceMetadata:
prefix = prefixDirectory
purpose = spdxPrimaryPurposeFile
case source.FileSourceMetadata:
prefix = prefixFile
purpose = spdxPrimaryPurposeFile
for _, d := range m.Digests {
checksums = append(checksums, spdx.Checksum{
Algorithm: toChecksumAlgorithm(d.Algorithm),
Value: d.Value,
})
}
default:
prefix = prefixUnknown
purpose = spdxPrimaryPurposeOther
if name == "" {
name = s.ID
}
}
p := &spdx.Package{
PackageName: name,
PackageSPDXIdentifier: spdx.ElementID(SanitizeElementID(fmt.Sprintf("DocumentRoot-%s-%s", prefix, name))),
PackageVersion: version,
PackageChecksums: checksums,
PackageSupplier: nil,
PackageExternalReferences: nil,
PrimaryPackagePurpose: purpose,
}
if purl != nil {
p.PackageExternalReferences = []*spdx.PackageExternalReference{
{
Category: string(PackageManagerReferenceCategory),
RefType: string(PurlExternalRefType),
Locator: purl.String(),
},
}
}
return p
}
func toSPDXID(identifiable artifact.Identifiable) spdx.ElementID {
maxLen := 40
id := ""
switch it := identifiable.(type) {
case pkg.Package:
id = SanitizeElementID(fmt.Sprintf("Package-%s-%s-%s", it.Type, it.Name, it.ID()))
switch {
case it.Type != "" && it.Name != "":
id = fmt.Sprintf("Package-%s-%s-%s", it.Type, it.Name, it.ID())
case it.Name != "":
id = fmt.Sprintf("Package-%s-%s", it.Name, it.ID())
case it.Type != "":
id = fmt.Sprintf("Package-%s-%s", it.Type, it.ID())
default:
id = fmt.Sprintf("Package-%s", it.ID())
}
case file.Coordinates:
p := ""
parts := strings.Split(it.RealPath, "/")
@@ -150,12 +287,12 @@ func toSPDXID(identifiable artifact.Identifiable) spdx.ElementID {
}
p = path.Join(part, p)
}
id = SanitizeElementID(fmt.Sprintf("File-%s-%s", p, it.ID()))
id = fmt.Sprintf("File-%s-%s", p, it.ID())
default:
id = string(identifiable.ID())
}
// NOTE: the spdx library prepend SPDXRef-, so we don't do it here
return spdx.ElementID(id)
return spdx.ElementID(SanitizeElementID(id))
}
// packages populates all Package Information from the package Collection (see https://spdx.github.io/spdx-spec/3-package-information/)
@@ -494,6 +631,18 @@ func toFileChecksums(digests []file.Digest) (checksums []spdx.Checksum) {
return checksums
}
// toChecksum takes a checksum in the format <algorithm>:<hash> and returns an spdx.Checksum or nil if the string is invalid
func toChecksum(algorithmHash string) *spdx.Checksum {
parts := strings.Split(algorithmHash, ":")
if len(parts) < 2 {
return nil
}
return &spdx.Checksum{
Algorithm: toChecksumAlgorithm(parts[0]),
Value: parts[1],
}
}
func toChecksumAlgorithm(algorithm string) spdx.ChecksumAlgorithm {
// this needs to be an uppercase version of our algorithm
return spdx.ChecksumAlgorithm(strings.ToUpper(algorithm))
@@ -5,17 +5,247 @@ import (
"regexp"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/spdx/v2/v2_3"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/internal/sourcemetadata"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
// TODO: Add ToFormatModel tests
func Test_toFormatModel(t *testing.T) {
tracker := sourcemetadata.NewCompletionTester(t)
tests := []struct {
name string
in sbom.SBOM
expected *spdx.Document
}{
{
name: "container",
in: sbom.SBOM{
Source: source.Description{
Name: "alpine",
Version: "sha256:d34db33f",
Metadata: source.StereoscopeImageSourceMetadata{
UserInput: "alpine:latest",
ManifestDigest: "sha256:d34db33f",
},
},
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(pkg.Package{
Name: "pkg-1",
Version: "version-1",
}),
},
},
expected: &spdx.Document{
SPDXIdentifier: "DOCUMENT",
SPDXVersion: spdx.Version,
DataLicense: spdx.DataLicense,
DocumentName: "alpine",
Packages: []*spdx.Package{
{
PackageSPDXIdentifier: "Package-pkg-1-pkg-1",
PackageName: "pkg-1",
PackageVersion: "version-1",
},
{
PackageSPDXIdentifier: "DocumentRoot-Image-alpine",
PackageName: "alpine",
PackageVersion: "sha256:d34db33f",
PrimaryPackagePurpose: "CONTAINER",
PackageChecksums: []spdx.Checksum{{Algorithm: "SHA256", Value: "d34db33f"}},
PackageExternalReferences: []*v2_3.PackageExternalReference{
{
Category: "PACKAGE-MANAGER",
RefType: "purl",
Locator: "pkg:oci/alpine@sha256:d34db33f?arch=&tag=latest",
},
},
},
},
Relationships: []*spdx.Relationship{
{
RefA: spdx.DocElementID{
ElementRefID: "DocumentRoot-Image-alpine",
},
RefB: spdx.DocElementID{
ElementRefID: "Package-pkg-1-pkg-1",
},
Relationship: spdx.RelationshipContains,
},
{
RefA: spdx.DocElementID{
ElementRefID: "DOCUMENT",
},
RefB: spdx.DocElementID{
ElementRefID: "DocumentRoot-Image-alpine",
},
Relationship: spdx.RelationshipDescribes,
},
},
},
},
{
name: "directory",
in: sbom.SBOM{
Source: source.Description{
Name: "some/directory",
Metadata: source.DirectorySourceMetadata{
Path: "some/directory",
},
},
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(pkg.Package{
Name: "pkg-1",
Version: "version-1",
}),
},
},
expected: &spdx.Document{
SPDXIdentifier: "DOCUMENT",
SPDXVersion: spdx.Version,
DataLicense: spdx.DataLicense,
DocumentName: "some/directory",
Packages: []*spdx.Package{
{
PackageSPDXIdentifier: "Package-pkg-1-pkg-1",
PackageName: "pkg-1",
PackageVersion: "version-1",
},
{
PackageSPDXIdentifier: "DocumentRoot-Directory-some-directory",
PackageName: "some/directory",
PackageVersion: "",
PrimaryPackagePurpose: "FILE",
},
},
Relationships: []*spdx.Relationship{
{
RefA: spdx.DocElementID{
ElementRefID: "DocumentRoot-Directory-some-directory",
},
RefB: spdx.DocElementID{
ElementRefID: "Package-pkg-1-pkg-1",
},
Relationship: spdx.RelationshipContains,
},
{
RefA: spdx.DocElementID{
ElementRefID: "DOCUMENT",
},
RefB: spdx.DocElementID{
ElementRefID: "DocumentRoot-Directory-some-directory",
},
Relationship: spdx.RelationshipDescribes,
},
},
},
},
{
name: "file",
in: sbom.SBOM{
Source: source.Description{
Name: "path/to/some.file",
Version: "sha256:d34db33f",
Metadata: source.FileSourceMetadata{
Path: "path/to/some.file",
Digests: []file.Digest{
{
Algorithm: "sha256",
Value: "d34db33f",
},
},
},
},
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(pkg.Package{
Name: "pkg-1",
Version: "version-1",
}),
},
},
expected: &spdx.Document{
SPDXIdentifier: "DOCUMENT",
SPDXVersion: spdx.Version,
DataLicense: spdx.DataLicense,
DocumentName: "path/to/some.file",
Packages: []*spdx.Package{
{
PackageSPDXIdentifier: "Package-pkg-1-pkg-1",
PackageName: "pkg-1",
PackageVersion: "version-1",
},
{
PackageSPDXIdentifier: "DocumentRoot-File-path-to-some.file",
PackageName: "path/to/some.file",
PackageVersion: "sha256:d34db33f",
PrimaryPackagePurpose: "FILE",
PackageChecksums: []spdx.Checksum{{Algorithm: "SHA256", Value: "d34db33f"}},
},
},
Relationships: []*spdx.Relationship{
{
RefA: spdx.DocElementID{
ElementRefID: "DocumentRoot-File-path-to-some.file",
},
RefB: spdx.DocElementID{
ElementRefID: "Package-pkg-1-pkg-1",
},
Relationship: spdx.RelationshipContains,
},
{
RefA: spdx.DocElementID{
ElementRefID: "DOCUMENT",
},
RefB: spdx.DocElementID{
ElementRefID: "DocumentRoot-File-path-to-some.file",
},
Relationship: spdx.RelationshipDescribes,
},
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
tracker.Tested(t, test.in.Source.Metadata)
// replace IDs with package names
var pkgs []pkg.Package
for p := range test.in.Artifacts.Packages.Enumerate() {
p.OverrideID(artifact.ID(p.Name))
pkgs = append(pkgs, p)
}
test.in.Artifacts.Packages = pkg.NewCollection(pkgs...)
// convert
got := ToFormatModel(test.in)
// check differences
if diff := cmp.Diff(test.expected, got,
cmpopts.IgnoreUnexported(spdx.Document{}, spdx.Package{}),
cmpopts.IgnoreFields(spdx.Document{}, "CreationInfo", "DocumentNamespace"),
cmpopts.IgnoreFields(spdx.Package{}, "PackageDownloadLocation", "IsFilesAnalyzedTagPresent", "PackageSourceInfo", "PackageLicenseConcluded", "PackageLicenseDeclared", "PackageCopyrightText"),
); diff != "" {
t.Error(diff)
}
})
}
}
func Test_toPackageChecksums(t *testing.T) {
tests := []struct {
name string
+198 -24
View File
@@ -2,11 +2,15 @@ package spdxhelpers
import (
"errors"
"fmt"
"net/url"
"path"
"regexp"
"strconv"
"strings"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/spdx/v2/common"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal/log"
@@ -26,12 +30,10 @@ func ToSyftModel(doc *spdx.Document) (*sbom.SBOM, error) {
return nil, errors.New("cannot convert SPDX document to Syft model because document is nil")
}
spdxIDMap := make(map[string]interface{})
src := extractSourceFromNamespace(doc.DocumentNamespace)
spdxIDMap := make(map[string]any)
s := &sbom.SBOM{
Source: src,
Source: extractSource(spdxIDMap, doc),
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(),
FileMetadata: map[file.Coordinates]file.Metadata{},
@@ -40,7 +42,7 @@ func ToSyftModel(doc *spdx.Document) (*sbom.SBOM, error) {
},
}
collectSyftPackages(s, spdxIDMap, doc)
collectSyftPackages(s, spdxIDMap, doc.Packages)
collectSyftFiles(s, spdxIDMap, doc)
@@ -49,6 +51,166 @@ func ToSyftModel(doc *spdx.Document) (*sbom.SBOM, error) {
return s, nil
}
func isDirectory(name string) bool {
if name == "." || name == ".." || strings.HasSuffix(name, "/") || !strings.Contains(path.Base(name), ".") {
return true
}
return false
}
func removePackage(packages []*spdx.Package, remove *spdx.Package) (pkgs []*spdx.Package) {
for _, p := range packages {
if p == remove {
continue
}
pkgs = append(pkgs, p)
}
return
}
func removeRelationships(relationships []*spdx.Relationship, spdxID spdx.ElementID) (relations []*spdx.Relationship) {
for _, r := range relationships {
if r.RefA.ElementRefID == spdxID || r.RefB.ElementRefID == spdxID {
continue
}
relations = append(relations, r)
}
return
}
func findRootPackages(doc *spdx.Document) (out []*spdx.Package) {
for _, p := range doc.Packages {
for _, r := range doc.Relationships {
describes := r.RefA.ElementRefID == "DOCUMENT" &&
r.Relationship == spdx.RelationshipDescribes &&
r.RefB.ElementRefID == p.PackageSPDXIdentifier
describedBy := r.RefB.ElementRefID == "DOCUMENT" &&
r.Relationship == spdx.RelationshipDescribedBy &&
r.RefA.ElementRefID == p.PackageSPDXIdentifier
if !describes && !describedBy {
continue
}
out = append(out, p)
}
}
return
}
func extractSource(spdxIDMap map[string]any, doc *spdx.Document) source.Description {
src := extractSourceFromNamespace(doc.DocumentNamespace)
rootPackages := findRootPackages(doc)
if len(rootPackages) != 1 {
return src
}
p := rootPackages[0]
switch p.PrimaryPackagePurpose {
case spdxPrimaryPurposeContainer:
src = containerSource(p)
case spdxPrimaryPurposeFile:
src = fileSource(p)
default:
return src
}
spdxIDMap[string(p.PackageSPDXIdentifier)] = src
doc.Packages = removePackage(doc.Packages, p)
doc.Relationships = removeRelationships(doc.Relationships, p.PackageSPDXIdentifier)
return src
}
func containerSource(p *spdx.Package) source.Description {
id := string(p.PackageSPDXIdentifier)
container := p.PackageName
v := p.PackageVersion
if v != "" {
container += ":" + v
}
digest := ""
if len(p.PackageChecksums) > 0 {
c := p.PackageChecksums[0]
digest = fmt.Sprintf("%s:%s", fromChecksumAlgorithm(c.Algorithm), c.Value)
}
return source.Description{
ID: id,
Name: p.PackageName,
Version: p.PackageVersion,
Metadata: source.StereoscopeImageSourceMetadata{
UserInput: container,
ID: id,
Layers: nil, // TODO handle formats with nested layer packages like Tern and K8s BOM tool
ManifestDigest: digest,
},
}
}
func fileSource(p *spdx.Package) source.Description {
typeRegex := regexp.MustCompile("^DocumentRoot-([^-]+)-.*$")
typeName := typeRegex.ReplaceAllString(string(p.PackageSPDXIdentifier), "$1")
var version string
var metadata any
switch {
case typeName == prefixDirectory:
// is a Syft SBOM, explicitly a directory source
metadata, version = directorySourceMetadata(p)
case typeName == prefixFile:
// is a Syft SBOM, explicitly a file source
metadata, version = fileSourceMetadata(p)
case isDirectory(p.PackageName):
// is a non-Syft SBOM, which looks like a directory
metadata, version = directorySourceMetadata(p)
default:
// is a non-Syft SBOM, which is probably a file
metadata, version = fileSourceMetadata(p)
}
return source.Description{
ID: string(p.PackageSPDXIdentifier),
Name: p.PackageName,
Version: version,
Metadata: metadata,
}
}
func fileSourceMetadata(p *spdx.Package) (any, string) {
version := p.PackageVersion
m := source.FileSourceMetadata{
Path: p.PackageName,
}
// if this is a Syft SBOM, we might have output a digest as the version
checksum := toChecksum(p.PackageVersion)
for _, d := range p.PackageChecksums {
if checksum != nil && checksum.Value == d.Value {
version = ""
}
m.Digests = append(m.Digests, file.Digest{
Algorithm: fromChecksumAlgorithm(d.Algorithm),
Value: d.Value,
})
}
return m, version
}
func directorySourceMetadata(p *spdx.Package) (any, string) {
return source.DirectorySourceMetadata{
Path: p.PackageName,
Base: "",
}, p.PackageVersion
}
// NOTE(jonas): SPDX doesn't inform what an SBOM is about,
// image, directory, for example. This is our best effort to determine
// the scheme. Syft-generated SBOMs have in the namespace
@@ -114,15 +276,15 @@ func findLinuxReleaseByPURL(doc *spdx.Document) *linux.Release {
return nil
}
func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document) {
for _, p := range doc.Packages {
func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]any, packages []*spdx.Package) {
for _, p := range packages {
syftPkg := toSyftPackage(p)
spdxIDMap[string(p.PackageSPDXIdentifier)] = syftPkg
s.Artifacts.Packages.Add(*syftPkg)
s.Artifacts.Packages.Add(syftPkg)
}
}
func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.Document) {
func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]any, doc *spdx.Document) {
for _, f := range doc.Files {
l := toSyftLocation(f)
spdxIDMap[string(f.FileSPDXIdentifier)] = l
@@ -135,13 +297,17 @@ func collectSyftFiles(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *spdx.
func toFileDigests(f *spdx.File) (digests []file.Digest) {
for _, digest := range f.Checksums {
digests = append(digests, file.Digest{
Algorithm: string(digest.Algorithm),
Algorithm: fromChecksumAlgorithm(digest.Algorithm),
Value: digest.Value,
})
}
return digests
}
func fromChecksumAlgorithm(algorithm common.ChecksumAlgorithm) string {
return strings.ToLower(string(algorithm))
}
func toFileMetadata(f *spdx.File) (meta file.Metadata) {
// FIXME Syft is currently lossy due to the SPDX 2.2.1 spec not supporting arbitrary mimetypes
for _, typ := range f.FileTypes {
@@ -164,21 +330,21 @@ func toFileMetadata(f *spdx.File) (meta file.Metadata) {
return meta
}
func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document) []artifact.Relationship {
func toSyftRelationships(spdxIDMap map[string]any, doc *spdx.Document) []artifact.Relationship {
var out []artifact.Relationship
for _, r := range doc.Relationships {
// FIXME what to do with r.RefA.DocumentRefID and r.RefA.SpecialID
// FIXME what to do with r.RefA.DocumentRefID and r.RefA.SpecialID
if r.RefA.DocumentRefID != "" && requireAndTrimPrefix(r.RefA.DocumentRefID, "DocumentRef-") != string(doc.SPDXIdentifier) {
log.Debugf("ignoring relationship to external document: %+v", r)
continue
}
a := spdxIDMap[string(r.RefA.ElementRefID)]
b := spdxIDMap[string(r.RefB.ElementRefID)]
from, fromOk := a.(*pkg.Package)
toPackage, toPackageOk := b.(*pkg.Package)
toLocation, toLocationOk := b.(*file.Location)
from, fromOk := a.(pkg.Package)
toPackage, toPackageOk := b.(pkg.Package)
toLocation, toLocationOk := b.(file.Location)
if !fromOk || !(toPackageOk || toLocationOk) {
log.Debugf("unable to find valid relationship mapping from SPDX 2.2 JSON, ignoring: (from: %+v) (to: %+v)", a, b)
log.Debugf("unable to find valid relationship mapping from SPDX, ignoring: (from: %+v) (to: %+v)", a, b)
continue
}
var to artifact.Identifiable
@@ -234,9 +400,9 @@ func toSyftCoordinates(f *spdx.File) file.Coordinates {
}
}
func toSyftLocation(f *spdx.File) *file.Location {
func toSyftLocation(f *spdx.File) file.Location {
l := file.NewVirtualLocationFromCoordinates(toSyftCoordinates(f), f.FileName)
return &l
return l
}
func requireAndTrimPrefix(val interface{}, prefix string) string {
@@ -280,16 +446,16 @@ func extractPkgInfo(p *spdx.Package) pkgInfo {
}
}
func toSyftPackage(p *spdx.Package) *pkg.Package {
func toSyftPackage(p *spdx.Package) pkg.Package {
info := extractPkgInfo(p)
metadataType, metadata := extractMetadata(p, info)
sP := pkg.Package{
sP := &pkg.Package{
Type: info.typ,
Name: p.PackageName,
Version: p.PackageVersion,
Licenses: pkg.NewLicenseSet(parseSPDXLicenses(p)...),
CPEs: extractCPEs(p),
PURL: info.purl.String(),
PURL: purlValue(info.purl),
Language: info.lang,
MetadataType: metadataType,
Metadata: metadata,
@@ -297,7 +463,15 @@ func toSyftPackage(p *spdx.Package) *pkg.Package {
sP.SetID()
return &sP
return *sP
}
func purlValue(purl packageurl.PackageURL) string {
p := purl.String()
if p == "pkg:/" {
return ""
}
return p
}
func parseSPDXLicenses(p *spdx.Package) []pkg.License {
@@ -384,7 +558,7 @@ func extractMetadata(p *spdx.Package, info pkgInfo) (pkg.MetadataType, interface
case pkg.JavaPkg:
var digests []file.Digest
for _, value := range p.PackageChecksums {
digests = append(digests, file.Digest{Algorithm: string(value.Algorithm), Value: value.Value})
digests = append(digests, file.Digest{Algorithm: fromChecksumAlgorithm(value.Algorithm), Value: value.Value})
}
return pkg.JavaMetadataType, pkg.JavaMetadata{
ArchiveDigests: digests,
@@ -392,7 +566,7 @@ func extractMetadata(p *spdx.Package, info pkgInfo) (pkg.MetadataType, interface
case pkg.GoModulePkg:
var h1Digest string
for _, value := range p.PackageChecksums {
digest, err := util.HDigestFromSHA(string(value.Algorithm), value.Value)
digest, err := util.HDigestFromSHA(fromChecksumAlgorithm(value.Algorithm), value.Value)
if err != nil {
log.Debugf("invalid h1digest: %v %v", value, err)
continue
@@ -4,6 +4,8 @@ import (
"reflect"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/spdx/v2/common"
"github.com/stretchr/testify/assert"
@@ -12,6 +14,7 @@ import (
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
@@ -324,7 +327,7 @@ func TestH1Digest(t *testing.T) {
func Test_toSyftRelationships(t *testing.T) {
type args struct {
spdxIDMap map[string]interface{}
spdxIDMap map[string]any
doc *spdx.Document
}
@@ -361,9 +364,9 @@ func Test_toSyftRelationships(t *testing.T) {
{
name: "evident-by relationship",
args: args{
spdxIDMap: map[string]interface{}{
string(toSPDXID(pkg1)): &pkg1,
string(toSPDXID(loc1)): &loc1,
spdxIDMap: map[string]any{
string(toSPDXID(pkg1)): pkg1,
string(toSPDXID(loc1)): loc1,
},
doc: &spdx.Document{
Relationships: []*spdx.Relationship{
@@ -391,9 +394,9 @@ func Test_toSyftRelationships(t *testing.T) {
{
name: "ownership-by-file-overlap relationship",
args: args{
spdxIDMap: map[string]interface{}{
string(toSPDXID(pkg2)): &pkg2,
string(toSPDXID(pkg3)): &pkg3,
spdxIDMap: map[string]any{
string(toSPDXID(pkg2)): pkg2,
string(toSPDXID(pkg3)): pkg3,
},
doc: &spdx.Document{
Relationships: []*spdx.Relationship{
@@ -431,3 +434,121 @@ func Test_toSyftRelationships(t *testing.T) {
})
}
}
func Test_convertToAndFromFormat(t *testing.T) {
packages := []pkg.Package{
{
Name: "pkg1",
MetadataType: pkg.UnknownMetadataType,
},
{
Name: "pkg2",
MetadataType: pkg.UnknownMetadataType,
},
}
for i := range packages {
(&packages[i]).SetID()
}
relationships := []artifact.Relationship{
{
From: packages[0],
To: packages[1],
Type: artifact.ContainsRelationship,
},
}
tests := []struct {
name string
source source.Description
packages []pkg.Package
relationships []artifact.Relationship
}{
{
name: "image source",
source: source.Description{
ID: "DocumentRoot-Image-some-image",
Metadata: source.StereoscopeImageSourceMetadata{
ID: "DocumentRoot-Image-some-image",
UserInput: "some-image:some-tag",
ManifestDigest: "sha256:ab8b83234bc28f28d8e",
},
Name: "some-image",
Version: "some-tag",
},
packages: packages,
relationships: relationships,
},
{
name: ". directory source",
source: source.Description{
ID: "DocumentRoot-Directory-.",
Name: ".",
Metadata: source.DirectorySourceMetadata{
Path: ".",
},
},
packages: packages,
relationships: relationships,
},
{
name: "directory source",
source: source.Description{
ID: "DocumentRoot-Directory-my-app",
Name: "my-app",
Metadata: source.DirectorySourceMetadata{
Path: "my-app",
},
},
packages: packages,
relationships: relationships,
},
{
name: "file source",
source: source.Description{
ID: "DocumentRoot-File-my-app.exe",
Metadata: source.FileSourceMetadata{
Path: "my-app.exe",
Digests: []file.Digest{
{
Algorithm: "sha256",
Value: "3723cae0b8b83234bc28f28d8e",
},
},
},
Name: "my-app.exe",
},
packages: packages,
relationships: relationships,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
src := &test.source
s := sbom.SBOM{
Source: *src,
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(test.packages...),
},
Relationships: test.relationships,
}
doc := ToFormatModel(s)
got, err := ToSyftModel(doc)
require.NoError(t, err)
if diff := cmp.Diff(&s, got,
cmpopts.IgnoreUnexported(artifact.Relationship{}),
cmpopts.IgnoreUnexported(file.LocationSet{}),
cmpopts.IgnoreUnexported(pkg.Collection{}),
cmpopts.IgnoreUnexported(pkg.Package{}),
cmpopts.IgnoreUnexported(pkg.LicenseSet{}),
cmpopts.IgnoreFields(pkg.Package{}, "MetadataType"),
cmpopts.IgnoreFields(sbom.Artifacts{}, "FileMetadata", "FileDigests"),
); diff != "" {
t.Fatalf("packages do not match:\n%s", diff)
}
})
}
}
+1 -1
View File
@@ -92,7 +92,7 @@ func TestSPDXJSONDecoder(t *testing.T) {
relationships:
for _, pkgName := range test.relationships {
for _, rel := range sbom.Relationships {
p, ok := rel.From.(*pkg.Package)
p, ok := rel.From.(pkg.Package)
if ok && p.Name == pkgName {
continue relationships
}
@@ -58,12 +58,29 @@
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
},
{
"name": "some/path",
"SPDXID": "SPDXRef-DocumentRoot-Directory-some-path",
"downloadLocation": "",
"filesAnalyzed": false,
"primaryPackagePurpose": "FILE"
}
],
"relationships": [
{
"spdxElementId": "SPDXRef-DocumentRoot-Directory-some-path",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-9265397e5e15168a",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Directory-some-path",
"relatedSpdxElement": "SPDXRef-Package-deb-package-2-db4abfe497c180d3",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DocumentRoot-Directory-some-path",
"relationshipType": "DESCRIBES"
}
]
@@ -58,12 +58,43 @@
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
},
{
"name": "user-image-input",
"SPDXID": "SPDXRef-DocumentRoot-Image-user-image-input",
"versionInfo": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"downloadLocation": "",
"filesAnalyzed": false,
"checksums": [
{
"algorithm": "SHA256",
"checksumValue": "2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
}
],
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch="
}
],
"primaryPackagePurpose": "CONTAINER"
}
],
"relationships": [
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-125840abc1c66dd7",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-deb-package-2-958443e2d9304af4",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DocumentRoot-Image-user-image-input",
"relationshipType": "DESCRIBES"
}
]
@@ -58,6 +58,27 @@
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
},
{
"name": "user-image-input",
"SPDXID": "SPDXRef-DocumentRoot-Image-user-image-input",
"versionInfo": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"downloadLocation": "",
"filesAnalyzed": false,
"checksums": [
{
"algorithm": "SHA256",
"checksumValue": "2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
}
],
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch="
}
],
"primaryPackagePurpose": "CONTAINER"
}
],
"files": [
@@ -183,9 +204,19 @@
"relatedSpdxElement": "SPDXRef-File-f2-f9e49132a4b96ccd",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-125840abc1c66dd7",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-deb-package-2-958443e2d9304af4",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DocumentRoot-Image-user-image-input",
"relationshipType": "DESCRIBES"
}
]
+2 -1
View File
@@ -61,7 +61,8 @@ func TestSPDXJSONSPDXIDs(t *testing.T) {
},
Relationships: nil,
Source: source.Description{
Metadata: source.DirectorySourceMetadata{Path: "foobar/baz"}, // in this case, foobar is used as the spdx docment name
Name: "foobar/baz", // in this case, foobar is used as the spdx document name
Metadata: source.DirectorySourceMetadata{},
},
Descriptor: sbom.Descriptor{
Name: "syft",
@@ -8,10 +8,17 @@ Creator: Organization: Anchore, Inc
Creator: Tool: syft-v0.42.0-bogus
Created: redacted
##### Package: foobar/baz
PackageName: foobar/baz
SPDXID: SPDXRef-DocumentRoot-Directory-foobar-baz
PrimaryPackagePurpose: FILE
FilesAnalyzed: false
##### Package: @at-sign
PackageName: @at-sign
SPDXID: SPDXRef-Package---at-sign-3732f7a5679bdec4
SPDXID: SPDXRef-Package--at-sign-3732f7a5679bdec4
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from the following paths:
@@ -22,7 +29,7 @@ PackageCopyrightText: NOASSERTION
##### Package: some/slashes
PackageName: some/slashes
SPDXID: SPDXRef-Package--some-slashes-1345166d4801153b
SPDXID: SPDXRef-Package-some-slashes-1345166d4801153b
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from the following paths:
@@ -33,7 +40,7 @@ PackageCopyrightText: NOASSERTION
##### Package: under_scores
PackageName: under_scores
SPDXID: SPDXRef-Package--under-scores-290d5c77210978c1
SPDXID: SPDXRef-Package-under-scores-290d5c77210978c1
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from the following paths:
@@ -43,5 +50,8 @@ PackageCopyrightText: NOASSERTION
##### Relationships
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DOCUMENT
Relationship: SPDXRef-DocumentRoot-Directory-foobar-baz CONTAINS SPDXRef-Package--at-sign-3732f7a5679bdec4
Relationship: SPDXRef-DocumentRoot-Directory-foobar-baz CONTAINS SPDXRef-Package-some-slashes-1345166d4801153b
Relationship: SPDXRef-DocumentRoot-Directory-foobar-baz CONTAINS SPDXRef-Package-under-scores-290d5c77210978c1
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Directory-foobar-baz
@@ -46,6 +46,16 @@ FileType: OTHER
FileChecksum: SHA1: 0000000000000000000000000000000000000000
LicenseConcluded: NOASSERTION
##### Package: user-image-input
PackageName: user-image-input
SPDXID: SPDXRef-DocumentRoot-Image-user-image-input
PackageVersion: sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368
PrimaryPackagePurpose: CONTAINER
FilesAnalyzed: false
PackageChecksum: SHA256: 2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368
ExternalRef: PACKAGE-MANAGER purl pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch=
##### Package: package-2
PackageName: package-2
@@ -82,5 +92,7 @@ Relationship: SPDXRef-Package-python-package-1-125840abc1c66dd7 CONTAINS SPDXRef
Relationship: SPDXRef-Package-python-package-1-125840abc1c66dd7 CONTAINS SPDXRef-File-d2-f4-c641caa71518099f
Relationship: SPDXRef-Package-python-package-1-125840abc1c66dd7 CONTAINS SPDXRef-File-d1-f3-c6f5b29dca12661f
Relationship: SPDXRef-Package-python-package-1-125840abc1c66dd7 CONTAINS SPDXRef-File-f2-f9e49132a4b96ccd
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DOCUMENT
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-python-package-1-125840abc1c66dd7
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-deb-package-2-958443e2d9304af4
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Image-user-image-input
@@ -8,6 +8,13 @@ Creator: Organization: Anchore, Inc
Creator: Tool: syft-v0.42.0-bogus
Created: redacted
##### Package: some/path
PackageName: some/path
SPDXID: SPDXRef-DocumentRoot-Directory-some-path
PrimaryPackagePurpose: FILE
FilesAnalyzed: false
##### Package: package-2
PackageName: package-2
@@ -38,5 +45,7 @@ ExternalRef: PACKAGE-MANAGER purl a-purl-2
##### Relationships
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DOCUMENT
Relationship: SPDXRef-DocumentRoot-Directory-some-path CONTAINS SPDXRef-Package-python-package-1-9265397e5e15168a
Relationship: SPDXRef-DocumentRoot-Directory-some-path CONTAINS SPDXRef-Package-deb-package-2-db4abfe497c180d3
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Directory-some-path
@@ -8,6 +8,16 @@ Creator: Organization: Anchore, Inc
Creator: Tool: syft-v0.42.0-bogus
Created: redacted
##### Package: user-image-input
PackageName: user-image-input
SPDXID: SPDXRef-DocumentRoot-Image-user-image-input
PackageVersion: sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368
PrimaryPackagePurpose: CONTAINER
FilesAnalyzed: false
PackageChecksum: SHA256: 2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368
ExternalRef: PACKAGE-MANAGER purl pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch=
##### Package: package-2
PackageName: package-2
@@ -38,5 +48,7 @@ ExternalRef: PACKAGE-MANAGER purl a-purl-1
##### Relationships
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DOCUMENT
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-python-package-1-125840abc1c66dd7
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-deb-package-2-958443e2d9304af4
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Image-user-image-input
+1 -1
View File
@@ -6,5 +6,5 @@ import "github.com/anchore/syft/syft/pkg"
// AllTypes returns a list of all pkg metadata types that syft supports (that are represented in the pkg.Package.Metadata field).
func AllTypes() []any {
return []any{pkg.AlpmMetadata{}, pkg.ApkMetadata{}, pkg.BinaryMetadata{}, pkg.CargoPackageMetadata{}, pkg.CocoapodsMetadata{}, pkg.ConanLockMetadata{}, pkg.ConanMetadata{}, pkg.DartPubMetadata{}, pkg.DotnetDepsMetadata{}, pkg.DpkgMetadata{}, pkg.GemMetadata{}, pkg.GolangBinMetadata{}, pkg.GolangModMetadata{}, pkg.HackageMetadata{}, pkg.JavaMetadata{}, pkg.KbPackageMetadata{}, pkg.LinuxKernelMetadata{}, pkg.LinuxKernelModuleMetadata{}, pkg.MixLockMetadata{}, pkg.NixStoreMetadata{}, pkg.NpmPackageJSONMetadata{}, pkg.NpmPackageLockJSONMetadata{}, pkg.PhpComposerJSONMetadata{}, pkg.PortageMetadata{}, pkg.PythonPackageMetadata{}, pkg.PythonPipfileLockMetadata{}, pkg.PythonRequirementsMetadata{}, pkg.RDescriptionFileMetadata{}, pkg.RebarLockMetadata{}, pkg.RpmMetadata{}}
return []any{pkg.AlpmMetadata{}, pkg.ApkMetadata{}, pkg.BinaryMetadata{}, pkg.CargoPackageMetadata{}, pkg.CocoapodsMetadata{}, pkg.ConanLockMetadata{}, pkg.ConanMetadata{}, pkg.DartPubMetadata{}, pkg.DotnetDepsMetadata{}, pkg.DotnetPortableExecutableMetadata{}, pkg.DpkgMetadata{}, pkg.GemMetadata{}, pkg.GolangBinMetadata{}, pkg.GolangModMetadata{}, pkg.HackageMetadata{}, pkg.JavaMetadata{}, pkg.KbPackageMetadata{}, pkg.LinuxKernelMetadata{}, pkg.LinuxKernelModuleMetadata{}, pkg.MixLockMetadata{}, pkg.NixStoreMetadata{}, pkg.NpmPackageJSONMetadata{}, pkg.NpmPackageLockJSONMetadata{}, pkg.PhpComposerJSONMetadata{}, pkg.PortageMetadata{}, pkg.PythonPackageMetadata{}, pkg.PythonPipfileLockMetadata{}, pkg.PythonRequirementsMetadata{}, pkg.RDescriptionFileMetadata{}, pkg.RebarLockMetadata{}, pkg.RpmMetadata{}, pkg.SwiftPackageManagerMetadata{}}
}
+8 -1
View File
@@ -76,7 +76,14 @@ func runCataloger(cataloger pkg.Cataloger, resolver file.Resolver) (catalogerRes
for _, p := range packages {
// generate CPEs (note: this is excluded from package ID, so is safe to mutate)
// we might have binary classified CPE already with the package so we want to append here
p.CPEs = append(p.CPEs, cpe.Generate(p)...)
dictionaryCPE, ok := cpe.DictionaryFind(p)
if ok {
log.Debugf("used CPE dictionary to find CPE for %s package %q: %s", p.Type, p.Name, dictionaryCPE.BindToFmtString())
p.CPEs = append(p.CPEs, dictionaryCPE)
} else {
p.CPEs = append(p.CPEs, cpe.Generate(p)...)
}
// if we were not able to identify the language we have an opportunity
// to try and get this value from the PURL. Worst case we assert that
+13 -9
View File
@@ -45,8 +45,8 @@ func ImageCatalogers(cfg Config) []pkg.Cataloger {
apkdb.NewApkdbCataloger(),
binary.NewCataloger(),
deb.NewDpkgdbCataloger(),
dotnet.NewDotnetDepsCataloger(),
golang.NewGoModuleBinaryCataloger(cfg.Go()),
dotnet.NewDotnetPortableExecutableCataloger(),
golang.NewGoModuleBinaryCataloger(cfg.Golang),
java.NewJavaCataloger(cfg.Java()),
java.NewNativeImageCataloger(),
javascript.NewPackageCataloger(),
@@ -71,10 +71,11 @@ func DirectoryCatalogers(cfg Config) []pkg.Cataloger {
dart.NewPubspecLockCataloger(),
deb.NewDpkgdbCataloger(),
dotnet.NewDotnetDepsCataloger(),
dotnet.NewDotnetPortableExecutableCataloger(),
elixir.NewMixLockCataloger(),
erlang.NewRebarLockCataloger(),
golang.NewGoModFileCataloger(cfg.Go()),
golang.NewGoModuleBinaryCataloger(cfg.Go()),
golang.NewGoModFileCataloger(cfg.Golang),
golang.NewGoModuleBinaryCataloger(cfg.Golang),
haskell.NewHackageCataloger(),
java.NewJavaCataloger(cfg.Java()),
java.NewJavaGradleLockfileCataloger(),
@@ -84,7 +85,7 @@ func DirectoryCatalogers(cfg Config) []pkg.Cataloger {
nix.NewStoreCataloger(),
php.NewComposerLockCataloger(),
portage.NewPortageCataloger(),
python.NewPythonIndexCataloger(),
python.NewPythonIndexCataloger(cfg.Python),
python.NewPythonPackageCataloger(),
rpm.NewFileCataloger(),
rpm.NewRpmDBCataloger(),
@@ -92,6 +93,7 @@ func DirectoryCatalogers(cfg Config) []pkg.Cataloger {
rust.NewCargoLockCataloger(),
sbom.NewSBOMCataloger(),
swift.NewCocoapodsCataloger(),
swift.NewSwiftPackageManagerCataloger(),
}, cfg.Catalogers)
}
@@ -105,10 +107,11 @@ func AllCatalogers(cfg Config) []pkg.Cataloger {
dart.NewPubspecLockCataloger(),
deb.NewDpkgdbCataloger(),
dotnet.NewDotnetDepsCataloger(),
dotnet.NewDotnetPortableExecutableCataloger(),
elixir.NewMixLockCataloger(),
erlang.NewRebarLockCataloger(),
golang.NewGoModFileCataloger(cfg.Go()),
golang.NewGoModuleBinaryCataloger(cfg.Go()),
golang.NewGoModFileCataloger(cfg.Golang),
golang.NewGoModuleBinaryCataloger(cfg.Golang),
haskell.NewHackageCataloger(),
java.NewJavaCataloger(cfg.Java()),
java.NewJavaGradleLockfileCataloger(),
@@ -116,12 +119,12 @@ func AllCatalogers(cfg Config) []pkg.Cataloger {
java.NewNativeImageCataloger(),
javascript.NewLockCataloger(),
javascript.NewPackageCataloger(),
kernel.NewLinuxKernelCataloger(cfg.Kernel()),
kernel.NewLinuxKernelCataloger(cfg.LinuxKernel),
nix.NewStoreCataloger(),
php.NewComposerInstalledCataloger(),
php.NewComposerLockCataloger(),
portage.NewPortageCataloger(),
python.NewPythonIndexCataloger(),
python.NewPythonIndexCataloger(cfg.Python),
python.NewPythonPackageCataloger(),
r.NewPackageCataloger(),
rpm.NewFileCataloger(),
@@ -132,6 +135,7 @@ func AllCatalogers(cfg Config) []pkg.Cataloger {
rust.NewCargoLockCataloger(),
sbom.NewSBOMCataloger(),
swift.NewCocoapodsCataloger(),
swift.NewSwiftPackageManagerCataloger(),
}, cfg.Catalogers)
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,3 @@
package dictionary
//go:generate go run ./index-generator/ -o data/cpe-index.json
@@ -0,0 +1,230 @@
package main
import (
"compress/gzip"
"encoding/json"
"encoding/xml"
"fmt"
"io"
"log"
"strings"
"github.com/facebookincubator/nvdtools/wfn"
"golang.org/x/exp/slices"
"github.com/anchore/syft/syft/pkg/cataloger/common/cpe/dictionary"
)
func generateIndexedDictionaryJSON(rawGzipData io.Reader) ([]byte, error) {
gzipReader, err := gzip.NewReader(rawGzipData)
if err != nil {
return nil, fmt.Errorf("unable to decompress CPE dictionary: %w", err)
}
defer gzipReader.Close()
// Read XML data
data, err := io.ReadAll(gzipReader)
if err != nil {
return nil, fmt.Errorf("unable to read CPE dictionary: %w", err)
}
// Unmarshal XML
var cpeList CpeList
if err := xml.Unmarshal(data, &cpeList); err != nil {
return nil, fmt.Errorf("unable to unmarshal CPE dictionary XML: %w", err)
}
// Filter out data that's not applicable here
cpeList = filterCpeList(cpeList)
// Create indexed dictionary to help with looking up CPEs
indexedDictionary := indexCPEList(cpeList)
// Convert to JSON
jsonData, err := json.MarshalIndent(indexedDictionary, "", " ")
if err != nil {
return nil, fmt.Errorf("unable to marshal CPE dictionary to JSON: %w", err)
}
return jsonData, nil
}
// filterCpeList removes CPE items that are not applicable to software packages.
func filterCpeList(cpeList CpeList) CpeList {
var processedCpeList CpeList
seen := make(map[string]struct{})
for _, cpeItem := range cpeList.CpeItems {
// Skip CPE items that don't have any references.
if len(cpeItem.References) == 0 {
continue
}
// Skip CPE items where the CPE URI doesn't meet our criteria.
parsedName, err := wfn.Parse(cpeItem.Name)
if err != nil {
log.Printf("unable to parse CPE URI %q: %s", cpeItem.Name, err)
}
if slices.Contains([]string{"h", "o"}, parsedName.Part) {
continue
}
normalizedName := normalizeCPE(parsedName).BindToURI()
if _, ok := seen[normalizedName]; ok {
continue
}
seen[normalizedName] = struct{}{}
cpeItem.Name = normalizedName
parsedCPE, err := wfn.Parse(cpeItem.Cpe23Item.Name)
if err != nil {
log.Printf("unable to parse CPE value %q: %s", cpeItem.Cpe23Item.Name, err)
}
cpeItem.Cpe23Item.Name = normalizeCPE(parsedCPE).BindToFmtString()
processedCpeList.CpeItems = append(processedCpeList.CpeItems, cpeItem)
}
return processedCpeList
}
// normalizeCPE removes the version and update parts of a CPE.
func normalizeCPE(cpe *wfn.Attributes) *wfn.Attributes {
cpeCopy := *cpe
cpeCopy.Version = ""
cpeCopy.Update = ""
return &cpeCopy
}
const (
prefixForNPMPackages = "https://www.npmjs.com/package/"
prefixForRubyGems = "https://rubygems.org/gems/"
prefixForRubyGemsHTTP = "http://rubygems.org/gems/"
prefixForNativeRubyGems = "https://github.com/ruby/"
prefixForPyPIPackages = "https://pypi.org/project/"
prefixForJenkinsPlugins = "https://github.com/jenkinsci/"
prefixForRustCrates = "https://crates.io/crates/"
)
// indexCPEList creates an index of CPEs by ecosystem.
func indexCPEList(list CpeList) *dictionary.Indexed {
indexed := &dictionary.Indexed{
EcosystemPackages: make(map[string]dictionary.Packages),
}
for _, cpeItem := range list.CpeItems {
cpeItemName := cpeItem.Cpe23Item.Name
for _, reference := range cpeItem.References {
ref := reference.Reference.Href
switch {
case strings.HasPrefix(ref, prefixForNPMPackages):
addEntryForNPMPackage(indexed, ref, cpeItemName)
case strings.HasPrefix(ref, prefixForRubyGems), strings.HasPrefix(ref, prefixForRubyGemsHTTP):
addEntryForRubyGem(indexed, ref, cpeItemName)
case strings.HasPrefix(ref, prefixForNativeRubyGems):
addEntryForNativeRubyGem(indexed, ref, cpeItemName)
case strings.HasPrefix(ref, prefixForPyPIPackages):
addEntryForPyPIPackage(indexed, ref, cpeItemName)
case strings.HasPrefix(ref, prefixForJenkinsPlugins):
// It _might_ be a jenkins plugin!
addEntryForJenkinsPlugin(indexed, ref, cpeItemName)
case strings.HasPrefix(ref, prefixForRustCrates):
addEntryForRustCrate(indexed, ref, cpeItemName)
}
}
}
return indexed
}
func addEntryForRustCrate(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.TrimPrefix(ref, prefixForRustCrates)
ref = strings.Split(ref, "/")[0]
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemRustCrates]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemRustCrates] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemRustCrates][ref] = cpeItemName
}
func addEntryForJenkinsPlugin(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.TrimPrefix(ref, prefixForJenkinsPlugins)
ref = strings.Split(ref, "/")[0]
if !strings.HasSuffix(ref, "-plugin") {
// It's not a jenkins plugin!
return
}
ref = strings.TrimSuffix(ref, "-plugin")
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemJenkinsPlugins]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemJenkinsPlugins] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemJenkinsPlugins][ref] = cpeItemName
}
func addEntryForPyPIPackage(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.TrimPrefix(ref, prefixForPyPIPackages)
ref = strings.Split(ref, "/")[0]
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemPyPI]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemPyPI] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemPyPI][ref] = cpeItemName
}
func addEntryForNativeRubyGem(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.TrimPrefix(ref, prefixForNativeRubyGems)
ref = strings.Split(ref, "/")[0]
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemRubyGems]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemRubyGems] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemRubyGems][ref] = cpeItemName
}
func addEntryForRubyGem(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.TrimPrefix(ref, prefixForRubyGems)
ref = strings.TrimPrefix(ref, prefixForRubyGemsHTTP)
ref = strings.Split(ref, "/")[0]
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemRubyGems]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemRubyGems] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemRubyGems][ref] = cpeItemName
}
func addEntryForNPMPackage(indexed *dictionary.Indexed, ref string, cpeItemName string) {
// Prune off the non-package-name parts of the URL
ref = strings.Split(ref, "/v/")[0]
ref = strings.Split(ref, "?")[0]
ref = strings.TrimPrefix(ref, prefixForNPMPackages)
if _, ok := indexed.EcosystemPackages[dictionary.EcosystemNPM]; !ok {
indexed.EcosystemPackages[dictionary.EcosystemNPM] = make(dictionary.Packages)
}
indexed.EcosystemPackages[dictionary.EcosystemNPM][ref] = cpeItemName
}
@@ -0,0 +1,169 @@
package main
import (
"bytes"
"compress/gzip"
"io"
"os"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/pkg/cataloger/common/cpe/dictionary"
)
func Test_generateIndexedDictionaryJSON(t *testing.T) {
f, err := os.Open("testdata/official-cpe-dictionary_v2.3.xml")
require.NoError(t, err)
// Create a buffer to store the gzipped data in memory
buf := new(bytes.Buffer)
w := gzip.NewWriter(buf)
_, err = io.Copy(w, f)
require.NoError(t, err)
// (finalize the gzip stream)
err = w.Close()
require.NoError(t, err)
dictionaryJSON, err := generateIndexedDictionaryJSON(buf)
assert.NoError(t, err)
expected, err := os.ReadFile("./testdata/expected-cpe-index.json")
require.NoError(t, err)
expectedDictionaryJSONString := string(expected)
dictionaryJSONString := string(dictionaryJSON)
if diff := cmp.Diff(expectedDictionaryJSONString, dictionaryJSONString); diff != "" {
t.Errorf("generateIndexedDictionaryJSON() mismatch (-want +got):\n%s", diff)
}
}
func Test_addEntryFuncs(t *testing.T) {
tests := []struct {
name string
addEntryFunc func(indexed *dictionary.Indexed, ref string, cpeItemName string)
inputRef string
inputCpeItemName string
expectedIndexed dictionary.Indexed
}{
{
name: "addEntryForRustCrate",
addEntryFunc: addEntryForRustCrate,
inputRef: "https://crates.io/crates/unicycle/versions",
inputCpeItemName: "cpe:2.3:a:unicycle_project:unicycle:*:*:*:*:*:rust:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemRustCrates: {
"unicycle": "cpe:2.3:a:unicycle_project:unicycle:*:*:*:*:*:rust:*:*",
},
},
},
},
{
name: "addEntryForJenkinsPlugin",
addEntryFunc: addEntryForJenkinsPlugin,
inputRef: "https://github.com/jenkinsci/sonarqube-plugin",
inputCpeItemName: "cpe:2.3:a:sonarsource:sonarqube_scanner:2.7:*:*:*:*:jenkins:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemJenkinsPlugins: {
"sonarqube": "cpe:2.3:a:sonarsource:sonarqube_scanner:2.7:*:*:*:*:jenkins:*:*",
},
},
},
},
{
name: "addEntryForJenkinsPlugin: not actually a plugin",
addEntryFunc: addEntryForJenkinsPlugin,
inputRef: "https://github.com/jenkinsci/jenkins",
inputCpeItemName: "cpe:2.3:a:jenkins:jenkinsci:2.7:*:*:*:*:*:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{},
},
},
{
name: "addEntryForPyPIPackage",
addEntryFunc: addEntryForPyPIPackage,
inputRef: "https://pypi.org/project/vault-cli/#history",
inputCpeItemName: "cpe:2.3:a:vault-cli_project:vault-cli:*:*:*:*:*:python:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemPyPI: {
"vault-cli": "cpe:2.3:a:vault-cli_project:vault-cli:*:*:*:*:*:python:*:*",
},
},
},
},
{
name: "addEntryForNativeRubyGem",
addEntryFunc: addEntryForNativeRubyGem,
inputRef: "https://github.com/ruby/openssl/releases",
inputCpeItemName: "cpe:2.3:a:ruby-lang:openssl:-:*:*:*:*:ruby:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemRubyGems: {
"openssl": "cpe:2.3:a:ruby-lang:openssl:-:*:*:*:*:ruby:*:*",
},
},
},
},
{
name: "addEntryForRubyGem: https",
addEntryFunc: addEntryForRubyGem,
inputRef: "https://rubygems.org/gems/actionview/versions",
inputCpeItemName: "cpe:2.3:a:action_view_project:action_view:*:*:*:*:*:ruby:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemRubyGems: {
"actionview": "cpe:2.3:a:action_view_project:action_view:*:*:*:*:*:ruby:*:*",
},
},
},
},
{
name: "addEntryForRubyGem: http",
addEntryFunc: addEntryForRubyGem,
inputRef: "http://rubygems.org/gems/rbovirt",
inputCpeItemName: "cpe:2.3:a:amos_benari:rbovirt:*:*:*:*:*:ruby:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemRubyGems: {
"rbovirt": "cpe:2.3:a:amos_benari:rbovirt:*:*:*:*:*:ruby:*:*",
},
},
},
},
{
name: "addEntryForNPMPackage",
addEntryFunc: addEntryForNPMPackage,
inputRef: "https://www.npmjs.com/package/@nubosoftware/node-static",
inputCpeItemName: "cpe:2.3:a:\\@nubosoftware\\/node-static_project:\\@nubosoftware\\/node-static:-:*:*:*:*:node.js:*:*",
expectedIndexed: dictionary.Indexed{
EcosystemPackages: map[string]dictionary.Packages{
dictionary.EcosystemNPM: {
"@nubosoftware/node-static": "cpe:2.3:a:\\@nubosoftware\\/node-static_project:\\@nubosoftware\\/node-static:-:*:*:*:*:node.js:*:*",
},
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
indexed := &dictionary.Indexed{
EcosystemPackages: make(map[string]dictionary.Packages),
}
tt.addEntryFunc(indexed, tt.inputRef, tt.inputCpeItemName)
if diff := cmp.Diff(tt.expectedIndexed, *indexed); diff != "" {
t.Errorf("addEntry* mismatch (-want +got):\n%s", diff)
}
})
}
}
@@ -0,0 +1,57 @@
// This program downloads the latest CPE dictionary from NIST and processes it into a JSON file that can be embedded into Syft for more accurate CPE results.
package main
import (
"errors"
"flag"
"fmt"
"log"
"net/http"
"os"
)
func mainE() error {
var outputFilename string
flag.StringVar(&outputFilename, "o", "", "file location to save CPE index")
flag.Parse()
if outputFilename == "" {
return errors.New("-o is required")
}
// Download and decompress file
fmt.Println("Fetching CPE dictionary...")
resp, err := http.Get(cpeDictionaryURL)
if err != nil {
return fmt.Errorf("unable to get CPE dictionary: %w", err)
}
defer resp.Body.Close()
fmt.Println("Generating index...")
dictionaryJSON, err := generateIndexedDictionaryJSON(resp.Body)
if err != nil {
return err
}
// Write CPE index (JSON data) to disk
err = os.WriteFile(outputFilename, dictionaryJSON, 0600)
if err != nil {
return fmt.Errorf("unable to write processed CPE dictionary to file: %w", err)
}
fmt.Println("Done!")
return nil
}
// errExit prints an error and exits with a non-zero exit code.
func errExit(err error) {
log.Printf("command failed: %s", err)
os.Exit(1)
}
func main() {
if err := mainE(); err != nil {
errExit(err)
}
}
@@ -0,0 +1,21 @@
package main
type CpeItem struct {
Name string `xml:"name,attr"`
Title string `xml:"title"`
References []struct {
Reference struct {
Href string `xml:"href,attr"`
Body string `xml:",chardata"`
} `xml:"reference"`
} `xml:"references"`
Cpe23Item struct {
Name string `xml:"name,attr"`
} `xml:"cpe23-item"`
}
type CpeList struct {
CpeItems []CpeItem `xml:"cpe-item"`
}
const cpeDictionaryURL = "https://nvd.nist.gov/feeds/xml/cpe/dictionary/official-cpe-dictionary_v2.3.xml.gz"
@@ -0,0 +1,23 @@
{
"ecosystems": {
"jenkins_plugins": {
"sonarqube": "cpe:2.3:a:sonarsource:sonarqube_scanner:*:*:*:*:*:jenkins:*:*"
},
"npm": {
"merge-recursive": "cpe:2.3:a:umbraengineering:merge-recursive:*:*:*:*:*:node.js:*:*",
"static-dev-server": "cpe:2.3:a:static-dev-server_project:static-dev-server:*:*:*:*:*:node.js:*:*",
"umount": "cpe:2.3:a:umount_project:umount:*:*:*:*:*:node.js:*:*",
"undefsafe": "cpe:2.3:a:undefsafe_project:undefsafe:*:*:*:*:*:node.js:*:*",
"underscore": "cpe:2.3:a:underscorejs:underscore:*:*:*:*:*:node.js:*:*",
"underscore-99xp": "cpe:2.3:a:underscore-99xp_project:underscore-99xp:*:*:*:*:*:node.js:*:*",
"unicode": "cpe:2.3:a:unicode_project:unicode:*:*:*:*:*:node.js:*:*",
"unicorn-list": "cpe:2.3:a:unicorn-list_project:unicorn-list:*:*:*:*:*:node.js:*:*"
},
"rubygems": {
"openssl": "cpe:2.3:a:ruby-lang:openssl:*:*:*:*:*:*:*:*"
},
"rust_crates": {
"unicycle": "cpe:2.3:a:unicycle_project:unicycle:*:*:*:*:*:rust:*:*"
}
}
}
@@ -0,0 +1,15 @@
package dictionary
const (
EcosystemNPM = "npm"
EcosystemRubyGems = "rubygems"
EcosystemPyPI = "pypi"
EcosystemJenkinsPlugins = "jenkins_plugins"
EcosystemRustCrates = "rust_crates"
)
type Indexed struct {
EcosystemPackages map[string]Packages `json:"ecosystems"`
}
type Packages map[string]string
+76
View File
@@ -3,16 +3,21 @@ package cpe
import (
"bufio"
"bytes"
_ "embed"
"encoding/json"
"fmt"
"sort"
"strings"
"sync"
"github.com/facebookincubator/nvdtools/wfn"
"github.com/scylladb/go-set/strset"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/common/cpe/dictionary"
)
// knownVendors contains vendor strings that are known to exist in
@@ -32,6 +37,77 @@ func newCPE(product, vendor, version, targetSW string) *wfn.Attributes {
return &c
}
//go:embed dictionary/data/cpe-index.json
var indexedCPEDictionaryData []byte
var indexedCPEDictionary *dictionary.Indexed
var indexedCPEDictionaryOnce sync.Once
func GetIndexedDictionary() (_ *dictionary.Indexed, err error) {
indexedCPEDictionaryOnce.Do(func() {
err = json.Unmarshal(indexedCPEDictionaryData, &indexedCPEDictionary)
})
if err != nil {
return
}
if indexedCPEDictionary == nil {
err = fmt.Errorf("failed to unmarshal indexed CPE dictionary")
return
}
return indexedCPEDictionary, err
}
func DictionaryFind(p pkg.Package) (cpe.CPE, bool) {
dict, err := GetIndexedDictionary()
if err != nil {
log.Debugf("dictionary CPE lookup not available: %+v", err)
return cpe.CPE{}, false
}
var (
cpeString string
ok bool
)
switch p.Type {
case pkg.NpmPkg:
cpeString, ok = dict.EcosystemPackages[dictionary.EcosystemNPM][p.Name]
case pkg.GemPkg:
cpeString, ok = dict.EcosystemPackages[dictionary.EcosystemRubyGems][p.Name]
case pkg.PythonPkg:
cpeString, ok = dict.EcosystemPackages[dictionary.EcosystemPyPI][p.Name]
case pkg.JenkinsPluginPkg:
cpeString, ok = dict.EcosystemPackages[dictionary.EcosystemJenkinsPlugins][p.Name]
case pkg.RustPkg:
cpeString, ok = dict.EcosystemPackages[dictionary.EcosystemRustCrates][p.Name]
default:
// The dictionary doesn't support this package type yet.
return cpe.CPE{}, false
}
if !ok {
// The dictionary doesn't have a CPE for this package.
return cpe.CPE{}, false
}
parsedCPE, err := cpe.New(cpeString)
if err != nil {
return cpe.CPE{}, false
}
parsedCPE.Version = p.Version
return parsedCPE, true
}
// Generate Create a list of CPEs for a given package, trying to guess the vendor, product tuple. We should be trying to
// generate the minimal set of representative CPEs, which implies that optional fields should not be included
// (such as target SW).
@@ -969,3 +969,33 @@ func Test_addSeparatorVariations(t *testing.T) {
})
}
}
func TestDictionaryFindIsWired(t *testing.T) {
tests := []struct {
name string
pkg pkg.Package
want string
wantExists bool
}{
{
name: "sanity check that cpe data is wired up",
pkg: pkg.Package{
Name: "openssl",
Version: "1.0.2k",
Type: pkg.GemPkg,
},
want: "cpe:2.3:a:ruby-lang:openssl:1.0.2k:*:*:*:*:*:*:*",
// without the cpe data wired up, this would be empty (generation also creates cpe:2.3:a:openssl:openssl:1.0.2k:*:*:*:*:*:*:*)
wantExists: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, gotExists := DictionaryFind(tt.pkg)
assert.Equal(t, tt.want, got.BindToFmtString())
assert.Equal(t, tt.wantExists, gotExists)
})
}
}
+3 -8
View File
@@ -4,6 +4,7 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/golang"
"github.com/anchore/syft/syft/pkg/cataloger/java"
"github.com/anchore/syft/syft/pkg/cataloger/kernel"
"github.com/anchore/syft/syft/pkg/cataloger/python"
)
// TODO: these field naming vs helper function naming schemes are inconsistent.
@@ -12,6 +13,7 @@ type Config struct {
Search SearchConfig
Golang golang.GoCatalogerOpts
LinuxKernel kernel.LinuxCatalogerConfig
Python python.CatalogerConfig
Catalogers []string
Parallelism int
}
@@ -21,6 +23,7 @@ func DefaultConfig() Config {
Search: DefaultSearchConfig(),
Parallelism: 1,
LinuxKernel: kernel.DefaultLinuxCatalogerConfig(),
Python: python.DefaultCatalogerConfig(),
}
}
@@ -30,11 +33,3 @@ func (c Config) Java() java.Config {
SearchIndexedArchives: c.Search.IncludeIndexedArchives,
}
}
func (c Config) Go() golang.GoCatalogerOpts {
return c.Golang
}
func (c Config) Kernel() kernel.LinuxCatalogerConfig {
return c.LinuxKernel
}
+6 -3
View File
@@ -4,10 +4,13 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
const catalogerName = "dotnet-deps-cataloger"
// NewDotnetDepsCataloger returns a new Dotnet cataloger object base on deps json files.
func NewDotnetDepsCataloger() *generic.Cataloger {
return generic.NewCataloger(catalogerName).
return generic.NewCataloger("dotnet-deps-cataloger").
WithParserByGlobs(parseDotnetDeps, "**/*.deps.json")
}
func NewDotnetPortableExecutableCataloger() *generic.Cataloger {
return generic.NewCataloger("dotnet-portable-executable-cataloger").
WithParserByGlobs(parseDotnetPortableExecutable, "**/*.dll", "**/*.exe")
}
+18 -6
View File
@@ -3,22 +3,34 @@ package dotnet
import (
"testing"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
func TestCataloger_Globs(t *testing.T) {
tests := []struct {
name string
fixture string
expected []string
name string
fixture string
cataloger *generic.Cataloger
expected []string
}{
{
name: "obtain deps.json files",
fixture: "test-fixtures/glob-paths",
name: "obtain deps.json files",
fixture: "test-fixtures/glob-paths",
cataloger: NewDotnetDepsCataloger(),
expected: []string{
"src/something.deps.json",
},
},
{
name: "obtain portable executable files",
fixture: "test-fixtures/glob-paths",
cataloger: NewDotnetPortableExecutableCataloger(),
expected: []string{
"src/something.dll",
"src/something.exe",
},
},
}
for _, test := range tests {
@@ -26,7 +38,7 @@ func TestCataloger_Globs(t *testing.T) {
pkgtest.NewCatalogTester().
FromDirectory(t, test.fixture).
ExpectsResolverContentQueries(test.expected).
TestCataloger(t, NewDotnetDepsCataloger())
TestCataloger(t, test.cataloger)
})
}
}
@@ -0,0 +1,87 @@
package dotnet
import (
"fmt"
"io"
"github.com/saferwall/pe"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
var _ generic.Parser = parseDotnetPortableExecutable
func parseDotnetPortableExecutable(_ file.Resolver, _ *generic.Environment, f file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
by, err := io.ReadAll(f)
if err != nil {
return nil, nil, fmt.Errorf("unable to read file: %w", err)
}
peFile, err := pe.NewBytes(by, &pe.Options{})
if err != nil {
return nil, nil, fmt.Errorf("unable to create PE file instance: %w", err)
}
err = peFile.Parse()
if err != nil {
return nil, nil, fmt.Errorf("unable to parse PE file: %w", err)
}
versionResources, err := peFile.ParseVersionResources()
if err != nil {
// this is not a fatal error, just log and continue
// TODO: consider this case for "known unknowns" (same goes for cases below)
log.Tracef("unable to parse version resources in PE file: %s", f.RealPath)
return nil, nil, nil
}
name := versionResources["FileDescription"]
if name == "" {
log.Tracef("unable to find FileDescription in PE file: %s", f.RealPath)
return nil, nil, nil
}
version := versionResources["FileVersion"]
if version == "" {
log.Tracef("unable to find FileVersion in PE file: %s", f.RealPath)
return nil, nil, nil
}
purl := packageurl.NewPackageURL(
packageurl.TypeNuget, // See explanation in syft/pkg/cataloger/dotnet/package.go as to why this was chosen.
"",
name,
version,
nil,
"",
).ToString()
metadata := pkg.DotnetPortableExecutableMetadata{
AssemblyVersion: versionResources["Assembly Version"],
LegalCopyright: versionResources["LegalCopyright"],
Comments: versionResources["Comments"],
InternalName: versionResources["InternalName"],
CompanyName: versionResources["CompanyName"],
ProductName: versionResources["ProductName"],
ProductVersion: versionResources["ProductVersion"],
}
p := pkg.Package{
Name: name,
Version: version,
Locations: file.NewLocationSet(f.Location),
Type: pkg.DotnetPkg,
PURL: purl,
MetadataType: pkg.DotnetPortableExecutableMetadataType,
Metadata: metadata,
}
p.SetID()
return []pkg.Package{p}, nil, nil
}
@@ -0,0 +1,38 @@
package dotnet
import (
"testing"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
func TestParseDotnetPortableExecutable(t *testing.T) {
fixture := "test-fixtures/System.Buffers.dll"
fixtureLocationSet := file.NewLocationSet(file.NewLocation(fixture))
expected := []pkg.Package{
{
Name: "System.Buffers",
Version: "7.0.923.36201",
Locations: fixtureLocationSet,
Type: pkg.DotnetPkg,
PURL: "pkg:nuget/System.Buffers@7.0.923.36201",
MetadataType: pkg.DotnetPortableExecutableMetadataType,
Metadata: pkg.DotnetPortableExecutableMetadata{
AssemblyVersion: "7.0.0.0",
LegalCopyright: "© Microsoft Corporation. All rights reserved.",
Comments: "System.Buffers",
InternalName: "System.Buffers.dll",
CompanyName: "Microsoft Corporation",
ProductName: "Microsoft® .NET",
ProductVersion: "7.0.9+8e9a17b2216f51a5788f8b1c467a4cf3b769e7d7",
},
},
}
var expectedRelationships []artifact.Relationship
pkgtest.TestFileParser(t, fixture, parseDotnetPortableExecutable, expected, expectedRelationships)
}
@@ -0,0 +1,2 @@
!*.dll
!*.exe
@@ -0,0 +1 @@
bogus .dll (portable executable)
@@ -0,0 +1 @@
bogus .exe portable executable)
+13 -2
View File
@@ -6,10 +6,21 @@ import (
const eggInfoGlob = "**/*.egg-info"
type CatalogerConfig struct {
GuessUnpinnedRequirements bool
}
func DefaultCatalogerConfig() CatalogerConfig {
return CatalogerConfig{
GuessUnpinnedRequirements: false,
}
}
// NewPythonIndexCataloger returns a new cataloger for python packages referenced from poetry lock files, requirements.txt files, and setup.py files.
func NewPythonIndexCataloger() *generic.Cataloger {
func NewPythonIndexCataloger(cfg CatalogerConfig) *generic.Cataloger {
rqp := newRequirementsParser(cfg)
return generic.NewCataloger("python-index-cataloger").
WithParserByGlobs(parseRequirementsTxt, "**/*requirements*.txt").
WithParserByGlobs(rqp.parseRequirementsTxt, "**/*requirements*.txt").
WithParserByGlobs(parsePoetryLock, "**/poetry.lock").
WithParserByGlobs(parsePipfileLock, "**/Pipfile.lock").
WithParserByGlobs(parseSetup, "**/setup.py")
+1 -1
View File
@@ -263,7 +263,7 @@ func Test_IndexCataloger_Globs(t *testing.T) {
pkgtest.NewCatalogTester().
FromDirectory(t, test.fixture).
ExpectsResolverContentQueries(test.expected).
TestCataloger(t, NewPythonIndexCataloger())
TestCataloger(t, NewPythonIndexCataloger(DefaultCatalogerConfig()))
})
}
}
+173 -99
View File
@@ -7,6 +7,10 @@ import (
"strings"
"unicode"
pep440 "github.com/aquasecurity/go-pep440-version"
"github.com/mitchellh/mapstructure"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
@@ -14,23 +18,99 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
var _ generic.Parser = parseRequirementsTxt
const (
// given the example requirement:
// requests[security] == 2.8.* ; python_version < "2.7" and sys_platform == "linux" \
// --hash=sha256:a9b3aaa1904eeb78e32394cd46c6f37ac0fb4af6dc488daa58971bdc7d7fcaf3 \
// --hash=sha256:e9535b8c84dc9571a48999094fda7f33e63c3f1b74f3e5f3ac0105a58405bb65 # some comment
var (
extrasRegex = regexp.MustCompile(`\[.*\]`)
urlRegex = regexp.MustCompile("@.*git.*")
// namePattern matches: requests[security]
namePattern = `(?P<name>\w[\w\[\],\s-_]+)`
// versionConstraintPattern matches: == 2.8.*
versionConstraintPattern = `(?P<versionConstraint>([^\S\r\n]*[~=>!<]+\s*[0-9a-zA-Z.*]+[^\S\r\n]*,?)+)?(@[^\S\r\n]*(?P<url>[^;]*))?`
// markersPattern matches: python_version < "2.7" and sys_platform == "linux"
markersPattern = `(;(?P<markers>.*))?`
// hashesPattern matches: --hash=sha256:a9b3aaa1904eeb78e32394cd46c6f37ac0fb4af6dc488daa58971bdc7d7fcaf3 --hash=sha256:e9535b8c84dc9571a48999094fda7f33e63c3f1b74f3e5f3ac0105a58405bb65
hashesPattern = `(?P<hashes>([^\S\r\n]*--hash=[a-zA-Z0-9:]+)+)?`
// whiteSpaceNoNewlinePattern matches: (any whitespace character except for \r and \n)
whiteSpaceNoNewlinePattern = `[^\S\r\n]*`
)
var requirementPattern = regexp.MustCompile(
`^` +
whiteSpaceNoNewlinePattern +
namePattern +
whiteSpaceNoNewlinePattern +
versionConstraintPattern +
markersPattern +
hashesPattern,
)
type unprocessedRequirement struct {
Name string `mapstructure:"name"`
VersionConstraint string `mapstructure:"versionConstraint"`
Markers string `mapstructure:"markers"`
URL string `mapstructure:"url"`
Hashes string `mapstructure:"hashes"`
}
func newRequirement(raw string) *unprocessedRequirement {
var r unprocessedRequirement
values := internal.MatchNamedCaptureGroups(requirementPattern, raw)
if err := mapstructure.Decode(values, &r); err != nil {
return nil
}
r.Name = strings.TrimSpace(r.Name)
r.VersionConstraint = strings.TrimSpace(r.VersionConstraint)
r.Markers = strings.TrimSpace(r.Markers)
r.URL = strings.TrimSpace(r.URL)
r.Hashes = strings.TrimSpace(r.Hashes)
if r.Name == "" {
return nil
}
return &r
}
type requirementsParser struct {
guessUnpinnedRequirements bool
}
func newRequirementsParser(cfg CatalogerConfig) requirementsParser {
return requirementsParser{
guessUnpinnedRequirements: cfg.GuessUnpinnedRequirements,
}
}
// parseRequirementsTxt takes a Python requirements.txt file, returning all Python packages that are locked to a
// specific version.
func parseRequirementsTxt(_ file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
func (rp requirementsParser) parseRequirementsTxt(_ file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
var packages []pkg.Package
scanner := bufio.NewScanner(reader)
var lastLine string
for scanner.Scan() {
line := scanner.Text()
rawLineNoComments := removeTrailingComment(line)
line = trimRequirementsTxtLine(line)
line := trimRequirementsTxtLine(scanner.Text())
if lastLine != "" {
line = lastLine + line
lastLine = ""
}
// remove line continuations... smashes the file into a single line
if strings.HasSuffix(line, "\\") {
// this line is a continuation of the previous line
lastLine += strings.TrimSuffix(line, "\\")
continue
}
if line == "" {
// nothing to parse on this line
@@ -42,35 +122,20 @@ func parseRequirementsTxt(_ file.Resolver, _ *generic.Environment, reader file.L
continue
}
if !strings.Contains(line, "==") {
// a package without a version, or a range (unpinned) which does not tell us
// exactly what will be installed.
continue
}
// parse a new requirement
parts := strings.Split(line, "==")
if len(parts) < 2 {
// this should never happen, but just in case
req := newRequirement(line)
if req == nil {
log.WithFields("path", reader.RealPath).Warnf("unable to parse requirements.txt line: %q", line)
continue
}
// check if the version contains hash declarations on the same line
version, _ := parseVersionAndHashes(parts[1])
name := removeExtras(req.Name)
version := parseVersion(req.VersionConstraint, rp.guessUnpinnedRequirements)
name := strings.TrimSpace(parts[0])
version = strings.TrimFunc(version, func(r rune) bool {
return !unicode.IsLetter(r) && !unicode.IsNumber(r)
})
// TODO: Update to support more than only ==
versionConstraint := fmt.Sprintf("== %s", version)
if name == "" || version == "" {
log.WithFields("path", reader.RealPath).Debugf("found empty package in requirements.txt line: %q", line)
if version == "" {
log.WithFields("path", reader.RealPath).Tracef("unable to determine package version in requirements.txt line: %q", line)
continue
}
packages = append(
packages,
newPackageForRequirementsWithMetadata(
@@ -78,10 +143,10 @@ func parseRequirementsTxt(_ file.Resolver, _ *generic.Environment, reader file.L
version,
pkg.PythonRequirementsMetadata{
Name: name,
Extras: parseExtras(rawLineNoComments),
VersionConstraint: versionConstraint,
URL: parseURL(rawLineNoComments),
Markers: parseMarkers(rawLineNoComments),
Extras: parseExtras(req.Name),
VersionConstraint: req.VersionConstraint,
URL: parseURL(req.URL),
Markers: req.Markers,
},
reader.Location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
),
@@ -95,13 +160,68 @@ func parseRequirementsTxt(_ file.Resolver, _ *generic.Environment, reader file.L
return packages, nil, nil
}
func parseVersionAndHashes(version string) (string, []string) {
parts := strings.Split(version, "--hash=")
if len(parts) < 2 {
return version, nil
func parseVersion(version string, guessFromConstraint bool) string {
if isPinnedConstraint(version) {
return strings.TrimSpace(strings.ReplaceAll(version, "==", ""))
}
return parts[0], parts[1:]
if guessFromConstraint {
return guessVersion(version)
}
return ""
}
func isPinnedConstraint(version string) bool {
return strings.Contains(version, "==") && !strings.ContainsAny(version, "*,<>!")
}
func guessVersion(constraint string) string {
// handle "2.8.*" -> "2.8.0"
constraint = strings.ReplaceAll(constraint, "*", "0")
if isPinnedConstraint(constraint) {
return strings.TrimSpace(strings.ReplaceAll(constraint, "==", ""))
}
constraints := strings.Split(constraint, ",")
filteredVersions := map[string]struct{}{}
for _, part := range constraints {
if strings.Contains(part, "!=") {
parts := strings.Split(part, "!=")
filteredVersions[strings.TrimSpace(parts[1])] = struct{}{}
}
}
var closestVersion *pep440.Version
for _, part := range constraints {
// ignore any parts that do not have '=' in them, >,<,~ are not valid semver
parts := strings.SplitAfter(part, "=")
if len(parts) < 2 {
continue
}
version, err := pep440.Parse(strings.TrimSpace(parts[1]))
if err != nil {
// ignore any parts that are not valid semver
continue
}
if _, ok := filteredVersions[version.String()]; ok {
continue
}
if strings.Contains(part, "==") {
parts := strings.Split(part, "==")
return strings.TrimSpace(parts[1])
}
if closestVersion == nil || version.GreaterThan(*closestVersion) {
closestVersion = &version
}
}
if closestVersion == nil {
return ""
}
return closestVersion.String()
}
// trimRequirementsTxtLine removes content from the given requirements.txt line
@@ -109,8 +229,6 @@ func parseVersionAndHashes(version string) (string, []string) {
func trimRequirementsTxtLine(line string) string {
line = strings.TrimSpace(line)
line = removeTrailingComment(line)
line = removeEnvironmentMarkers(line)
line = checkForRegex(line) // remove extras and url from line if found
return line
}
@@ -127,57 +245,29 @@ func removeTrailingComment(line string) string {
return parts[0]
}
// removeEnvironmentMarkers removes any instances of environment markers (delimited by ';') from the line.
// For more information, see https://www.python.org/dev/peps/pep-0508/#environment-markers.
func removeEnvironmentMarkers(line string) string {
parts := strings.SplitN(line, ";", 2)
if len(parts) < 2 {
// there aren't any environment markers
return line
func removeExtras(packageName string) string {
start := strings.Index(packageName, "[")
if start == -1 {
return packageName
}
return parts[0]
return strings.TrimSpace(packageName[:start])
}
func parseExtras(packageName string) []string {
if extrasRegex.MatchString(packageName) {
// Remove square brackets
extras := strings.TrimFunc(extrasRegex.FindString(packageName), func(r rune) bool {
return !unicode.IsLetter(r) && !unicode.IsNumber(r)
})
var extras []string
// Remove any additional whitespace
extras = strings.ReplaceAll(extras, " ", "")
return strings.Split(extras, ",")
start := strings.Index(packageName, "[")
stop := strings.Index(packageName, "]")
if start == -1 || stop == -1 {
return extras
}
return []string{}
}
func parseMarkers(line string) map[string]string {
markers := map[string]string{}
parts := strings.SplitN(line, ";", 2)
if len(parts) == 2 {
splittableMarkers := parts[1]
for _, combineString := range []string{" or ", " and "} {
splittableMarkers = strings.TrimSpace(
strings.ReplaceAll(splittableMarkers, combineString, ","),
)
}
splittableMarkers = strings.TrimSpace(splittableMarkers)
for _, mark := range strings.Split(splittableMarkers, ",") {
markparts := strings.Split(mark, " ")
markers[markparts[0]] = strings.Join(markparts[1:], " ")
}
extraString := packageName[start+1 : stop]
for _, extra := range strings.Split(extraString, ",") {
extras = append(extras, strings.TrimSpace(extra))
}
return markers
return extras
}
func parseURL(line string) string {
@@ -204,19 +294,3 @@ func parseURL(line string) string {
return ""
}
// function to check a string for all possilbe regex expressions, replacing it if found
func checkForRegex(stringToCheck string) string {
stringToReturn := stringToCheck
for _, r := range []*regexp.Regexp{
urlRegex,
extrasRegex,
} {
if r.MatchString(stringToCheck) {
stringToReturn = r.ReplaceAllString(stringToCheck, "")
}
}
return stringToReturn
}
@@ -3,6 +3,8 @@ package python
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
@@ -12,7 +14,8 @@ import (
func TestParseRequirementsTxt(t *testing.T) {
fixture := "test-fixtures/requires/requirements.txt"
locations := file.NewLocationSet(file.NewLocation(fixture))
expectedPkgs := []pkg.Package{
pinnedPkgs := []pkg.Package{
{
Name: "flask",
Version: "4.0.0",
@@ -23,10 +26,7 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "flask",
Extras: []string{},
VersionConstraint: "== 4.0.0",
URL: "",
Markers: map[string]string{},
},
},
{
@@ -39,10 +39,7 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "foo",
Extras: []string{},
VersionConstraint: "== 1.0.0",
URL: "",
Markers: map[string]string{},
},
},
{
@@ -55,10 +52,8 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "SomeProject",
Extras: []string{},
VersionConstraint: "== 5.4",
URL: "",
Markers: map[string]string{"python_version": "< '3.8'"},
VersionConstraint: "==5.4",
Markers: "python_version < '3.8'",
},
},
{
@@ -71,10 +66,7 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "argh",
Extras: []string{},
VersionConstraint: "== 0.26.2",
URL: "",
Markers: map[string]string{},
VersionConstraint: "==0.26.2",
},
},
{
@@ -87,10 +79,7 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "argh",
Extras: []string{},
VersionConstraint: "== 0.26.3",
URL: "",
Markers: map[string]string{},
VersionConstraint: "==0.26.3",
},
},
{
@@ -105,27 +94,6 @@ func TestParseRequirementsTxt(t *testing.T) {
Name: "celery",
Extras: []string{"redis", "pytest"},
VersionConstraint: "== 4.4.7",
URL: "",
Markers: map[string]string{},
},
},
{
Name: "requests",
Version: "2.8",
PURL: "pkg:pypi/requests@2.8",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "requests",
Extras: []string{"security"},
VersionConstraint: "== 2.8",
URL: "",
Markers: map[string]string{
"python_version": `< "2.7"`,
"sys_platform": `== "linux"`,
},
},
},
{
@@ -138,15 +106,238 @@ func TestParseRequirementsTxt(t *testing.T) {
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "GithubSampleProject",
Extras: []string{},
VersionConstraint: "== 3.7.1",
URL: "git+https://github.com/owner/repo@releases/tag/v3.7.1",
Markers: map[string]string{},
},
},
}
var expectedRelationships []artifact.Relationship
var testCases = []struct {
name string
fixture string
cfg CatalogerConfig
expectedPkgs []pkg.Package
expectedRelationships []artifact.Relationship
}{
{
name: "pinned dependencies only",
fixture: fixture,
cfg: CatalogerConfig{
GuessUnpinnedRequirements: false,
},
expectedPkgs: pinnedPkgs,
},
{
name: "guess unpinned requirements (lowest version)",
fixture: fixture,
cfg: CatalogerConfig{
GuessUnpinnedRequirements: true,
},
expectedPkgs: append([]pkg.Package{
{
Name: "Mopidy-Dirble",
Version: "1.1",
PURL: "pkg:pypi/Mopidy-Dirble@1.1",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "Mopidy-Dirble",
VersionConstraint: "~= 1.1",
},
},
{
Name: "sqlalchemy",
Version: "2.0.0",
PURL: "pkg:pypi/sqlalchemy@2.0.0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "sqlalchemy",
VersionConstraint: ">= 1.0.0, <= 2.0.0, != 3.0.0, <= 3.0.0",
},
},
{
Name: "bar",
Version: "2.0.0",
PURL: "pkg:pypi/bar@2.0.0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "bar",
VersionConstraint: ">= 1.0.0, <= 2.0.0, != 3.0.0, <= 3.0.0",
},
},
{
Name: "numpy",
Version: "3.4.1",
PURL: "pkg:pypi/numpy@3.4.1",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "numpy",
VersionConstraint: ">= 3.4.1",
Markers: `sys_platform == 'win32'`,
},
},
{
Name: "requests",
Version: "2.8.0",
PURL: "pkg:pypi/requests@2.8.0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonRequirementsMetadataType,
Metadata: pkg.PythonRequirementsMetadata{
Name: "requests",
Extras: []string{"security"},
VersionConstraint: "== 2.8.*",
Markers: `python_version < "2.7" and sys_platform == "linux"`,
},
},
}, pinnedPkgs...),
},
}
pkgtest.TestFileParser(t, fixture, parseRequirementsTxt, expectedPkgs, expectedRelationships)
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
parser := newRequirementsParser(tc.cfg)
pkgtest.TestFileParser(t, tc.fixture, parser.parseRequirementsTxt, tc.expectedPkgs, tc.expectedRelationships)
})
}
}
func Test_newRequirement(t *testing.T) {
tests := []struct {
name string
raw string
want *unprocessedRequirement
}{
{
name: "simple",
raw: "requests==2.8",
want: &unprocessedRequirement{
Name: "requests",
VersionConstraint: "==2.8",
},
},
{
name: "comment + constraint",
raw: "Mopidy-Dirble ~= 1.1 # Compatible release. Same as >= 1.1, == 1.*",
want: &unprocessedRequirement{
Name: "Mopidy-Dirble",
VersionConstraint: "~= 1.1",
},
},
{
name: "hashes",
raw: "argh==0.26.3 --hash=sha256:a9b3aaa1904eeb78e32394cd46c6f37ac0fb4af6dc488daa58971bdc7d7fcaf3 --hash=sha256:e9535b8c84dc9571a48999094fda7f33e63c3f1b74f3e5f3ac0105a58405bb65",
want: &unprocessedRequirement{
Name: "argh",
VersionConstraint: "==0.26.3",
Hashes: "--hash=sha256:a9b3aaa1904eeb78e32394cd46c6f37ac0fb4af6dc488daa58971bdc7d7fcaf3 --hash=sha256:e9535b8c84dc9571a48999094fda7f33e63c3f1b74f3e5f3ac0105a58405bb65",
},
},
{
name: "extras",
raw: "celery[redis, pytest] == 4.4.7 # should remove [redis, pytest]",
want: &unprocessedRequirement{
Name: "celery[redis, pytest]",
VersionConstraint: "== 4.4.7",
},
},
{
name: "url",
raw: "GithubSampleProject == 3.7.1 @ git+https://github.com/owner/repo@releases/tag/v3.7.1",
want: &unprocessedRequirement{
Name: "GithubSampleProject",
VersionConstraint: "== 3.7.1",
URL: "git+https://github.com/owner/repo@releases/tag/v3.7.1",
},
},
{
name: "markers",
raw: "numpy >= 3.4.1 ; sys_platform == 'win32'",
want: &unprocessedRequirement{
Name: "numpy",
VersionConstraint: ">= 3.4.1",
Markers: "sys_platform == 'win32'",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, newRequirement(tt.raw))
})
}
}
// checkout https://www.darius.page/pipdev/ for help here! (github.com/nok/pipdev)
func Test_parseVersion(t *testing.T) {
tests := []struct {
name string
version string
guess bool
want string
}{
{
name: "exact",
version: "1.0.0",
want: "", // we can only parse constraints, not assume that a single version is a pin
},
{
name: "exact constraint",
version: " == 1.0.0 ",
want: "1.0.0",
},
{
name: "resolve lowest, simple constraint",
version: " >= 1.0.0 ",
guess: true,
want: "1.0.0",
},
{
name: "resolve lowest, compound constraint",
version: " < 2.0.0, >= 1.0.0, != 1.1.0 ",
guess: true,
want: "1.0.0",
},
{
name: "resolve lowest, handle asterisk",
version: "==2.8.*",
guess: true,
want: "2.8.0",
},
{
name: "resolve lowest, handle exceptions",
version: " !=4.0.2,!=4.1.0,!=4.2.0,>=4.0.1,!=4.3.0,!=5.0.0,!=5.1.0,<6.0.0",
guess: true,
want: "4.0.1",
},
{
name: "resolve lowest, compatible version constraint",
version: "~=0.6.10", // equates to >=0.6.10, ==0.6.*
guess: true,
want: "0.6.10",
},
{
name: "resolve lowest, with character in version",
version: "~=1.2b,<=1.3a,!=1.1,!=1.2",
guess: true,
want: "1.3a0", // note: 1.3a == 1.3a0
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, parseVersion(tt.version, tt.guess))
})
}
}
@@ -1,7 +1,9 @@
flask == 4.0.0
# a line that is ignored
sqlalchemy >= 1.0.0
sqlalchemy >= 1.0.0, <= 2.0.0, != 3.0.0, <= 3.0.0
foo == 1.0.0 # a comment that needs to be ignored
bar >= 1.0.0, <= 2.0.0, \
!= 3.0.0, <= 3.0.0
-e https://github.com/pecan/pecan.git
-r other-requirements.txt
--requirements super-secretrequirements.txt
+6 -1
View File
@@ -1,5 +1,5 @@
/*
Package swift provides a concrete Cataloger implementation for Podfile.lock files.
Package swift provides a concrete Cataloger implementation for Podfile.lock and Package.resolved files.
*/
package swift
@@ -7,6 +7,11 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
func NewSwiftPackageManagerCataloger() *generic.Cataloger {
return generic.NewCataloger("spm-cataloger").
WithParserByGlobs(parsePackageResolved, "**/Package.resolved", "**/.package.resolved")
}
// NewCocoapodsCataloger returns a new Swift Cocoapods lock file cataloger object.
func NewCocoapodsCataloger() *generic.Cataloger {
return generic.NewCataloger("cocoapods-cataloger").
+37 -3
View File
@@ -1,16 +1,37 @@
package swift
import (
"strings"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
func newPackage(name, version, hash string, locations ...file.Location) pkg.Package {
func newSwiftPackageManagerPackage(name, version, sourceURL, revision string, locations ...file.Location) pkg.Package {
p := pkg.Package{
Name: name,
Version: version,
PURL: packageURL(name, version),
PURL: swiftPackageManagerPackageURL(name, version, sourceURL),
Locations: file.NewLocationSet(locations...),
Type: pkg.SwiftPkg,
Language: pkg.Swift,
MetadataType: pkg.SwiftPackageManagerMetadataType,
Metadata: pkg.SwiftPackageManagerMetadata{
Revision: revision,
},
}
p.SetID()
return p
}
func newCocoaPodsPackage(name, version, hash string, locations ...file.Location) pkg.Package {
p := pkg.Package{
Name: name,
Version: version,
PURL: cocoaPodsPackageURL(name, version),
Locations: file.NewLocationSet(locations...),
Type: pkg.CocoapodsPkg,
Language: pkg.Swift,
@@ -25,7 +46,7 @@ func newPackage(name, version, hash string, locations ...file.Location) pkg.Pack
return p
}
func packageURL(name, version string) string {
func cocoaPodsPackageURL(name, version string) string {
var qualifiers packageurl.Qualifiers
return packageurl.NewPackageURL(
@@ -37,3 +58,16 @@ func packageURL(name, version string) string {
"",
).ToString()
}
func swiftPackageManagerPackageURL(name, version, sourceURL string) string {
var qualifiers packageurl.Qualifiers
return packageurl.NewPackageURL(
packageurl.TypeSwift,
strings.Replace(sourceURL, "https://", "", 1),
name,
version,
qualifiers,
"",
).ToString()
}
+2 -2
View File
@@ -6,7 +6,7 @@ import (
"github.com/stretchr/testify/assert"
)
func Test_packageURL(t *testing.T) {
func Test_cocoaPodsPackageURL(t *testing.T) {
type args struct {
name string
version string
@@ -27,7 +27,7 @@ func Test_packageURL(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, packageURL(tt.args.name, tt.args.version))
assert.Equal(t, tt.want, cocoaPodsPackageURL(tt.args.name, tt.args.version))
})
}
}
@@ -0,0 +1,134 @@
package swift
import (
"encoding/json"
"errors"
"fmt"
"io"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
var _ generic.Parser = parsePackageResolved
// swift package manager has two versions (1 and 2) of the resolved files, the types below describes the serialization strategies for each version
// with its suffix indicating which version its specific to.
type packageResolvedV1 struct {
PackageObject packageObjectV1 `json:"object"`
Version int `json:"version"`
}
type packageObjectV1 struct {
Pins []packagePinsV1
}
type packagePinsV1 struct {
Name string `json:"package"`
RepositoryURL string `json:"repositoryURL"`
State packageState `json:"state"`
}
type packageResolvedV2 struct {
Pins []packagePinsV2
}
type packagePinsV2 struct {
Identity string `json:"identity"`
Kind string `json:"kind"`
Location string `json:"location"`
State packageState `json:"state"`
}
type packagePin struct {
Identity string
Location string
Revision string
Version string
}
type packageState struct {
Revision string `json:"revision"`
Version string `json:"version"`
}
// parsePackageResolved is a parser for the contents of a Package.resolved file, which is generated by Xcode after it's resolved Swift Package Manger packages.
func parsePackageResolved(_ file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
dec := json.NewDecoder(reader)
var packageResolvedData map[string]interface{}
for {
if err := dec.Decode(&packageResolvedData); errors.Is(err, io.EOF) {
break
} else if err != nil {
return nil, nil, fmt.Errorf("failed to parse Package.resolved file: %w", err)
}
}
var pins, err = pinsForVersion(packageResolvedData, packageResolvedData["version"].(float64))
if err != nil {
return nil, nil, err
}
var pkgs []pkg.Package
for _, packagePin := range pins {
pkgs = append(
pkgs,
newSwiftPackageManagerPackage(
packagePin.Identity,
packagePin.Version,
packagePin.Location,
packagePin.Revision,
reader.Location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
),
)
}
return pkgs, nil, nil
}
func pinsForVersion(data map[string]interface{}, version float64) ([]packagePin, error) {
var genericPins []packagePin
switch version {
case 1:
t := packageResolvedV1{}
jsonString, err := json.Marshal(data)
if err != nil {
return nil, err
}
parseErr := json.Unmarshal(jsonString, &t)
if parseErr != nil {
return nil, parseErr
}
for _, pin := range t.PackageObject.Pins {
genericPins = append(genericPins, packagePin{
pin.Name,
pin.RepositoryURL,
pin.State.Revision,
pin.State.Version,
})
}
case 2:
t := packageResolvedV2{}
jsonString, err := json.Marshal(data)
if err != nil {
return nil, err
}
parseErr := json.Unmarshal(jsonString, &t)
if parseErr != nil {
return nil, parseErr
}
for _, pin := range t.Pins {
genericPins = append(genericPins, packagePin{
pin.Identity,
pin.Location,
pin.State.Revision,
pin.State.Version,
})
}
default:
return nil, fmt.Errorf("unknown swift package manager version, %f", version)
}
return genericPins, nil
}
@@ -0,0 +1,82 @@
package swift
import (
"testing"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
func TestParsePackageResolved(t *testing.T) {
fixture := "test-fixtures/Package.resolved"
locations := file.NewLocationSet(file.NewLocation(fixture))
expectedPkgs := []pkg.Package{
{
Name: "swift-algorithms",
Version: "1.0.0",
PURL: "pkg:swift/github.com/apple/swift-algorithms.git/swift-algorithms@1.0.0",
Locations: locations,
Language: pkg.Swift,
Type: pkg.SwiftPkg,
MetadataType: pkg.SwiftPackageManagerMetadataType,
Metadata: pkg.SwiftPackageManagerMetadata{
Revision: "b14b7f4c528c942f121c8b860b9410b2bf57825e",
},
},
{
Name: "swift-async-algorithms",
Version: "0.1.0",
PURL: "pkg:swift/github.com/apple/swift-async-algorithms.git/swift-async-algorithms@0.1.0",
Locations: locations,
Language: pkg.Swift,
Type: pkg.SwiftPkg,
MetadataType: pkg.SwiftPackageManagerMetadataType,
Metadata: pkg.SwiftPackageManagerMetadata{
Revision: "9cfed92b026c524674ed869a4ff2dcfdeedf8a2a",
},
},
{
Name: "swift-atomics",
Version: "1.1.0",
PURL: "pkg:swift/github.com/apple/swift-atomics.git/swift-atomics@1.1.0",
Locations: locations,
Language: pkg.Swift,
Type: pkg.SwiftPkg,
MetadataType: pkg.SwiftPackageManagerMetadataType,
Metadata: pkg.SwiftPackageManagerMetadata{
Revision: "6c89474e62719ddcc1e9614989fff2f68208fe10",
},
},
{
Name: "swift-collections",
Version: "1.0.4",
PURL: "pkg:swift/github.com/apple/swift-collections.git/swift-collections@1.0.4",
Locations: locations,
Language: pkg.Swift,
Type: pkg.SwiftPkg,
MetadataType: pkg.SwiftPackageManagerMetadataType,
Metadata: pkg.SwiftPackageManagerMetadata{
Revision: "937e904258d22af6e447a0b72c0bc67583ef64a2",
},
},
{
Name: "swift-numerics",
Version: "1.0.2",
PURL: "pkg:swift/github.com/apple/swift-numerics/swift-numerics@1.0.2",
Locations: locations,
Language: pkg.Swift,
Type: pkg.SwiftPkg,
MetadataType: pkg.SwiftPackageManagerMetadataType,
Metadata: pkg.SwiftPackageManagerMetadata{
Revision: "0a5bc04095a675662cf24757cc0640aa2204253b",
},
},
}
// TODO: no relationships are under test yet
var expectedRelationships []artifact.Relationship
pkgtest.TestFileParser(t, fixture, parsePackageResolved, expectedPkgs, expectedRelationships)
}
@@ -61,7 +61,7 @@ func parsePodfileLock(_ file.Resolver, _ *generic.Environment, reader file.Locat
pkgs = append(
pkgs,
newPackage(
newCocoaPodsPackage(
podName,
podVersion,
pkgHash,
@@ -0,0 +1,50 @@
{
"pins" : [
{
"identity" : "swift-algorithms",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-algorithms.git",
"state" : {
"revision" : "b14b7f4c528c942f121c8b860b9410b2bf57825e",
"version" : "1.0.0"
}
},
{
"identity" : "swift-async-algorithms",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-async-algorithms.git",
"state" : {
"revision" : "9cfed92b026c524674ed869a4ff2dcfdeedf8a2a",
"version" : "0.1.0"
}
},
{
"identity" : "swift-atomics",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-atomics.git",
"state" : {
"revision" : "6c89474e62719ddcc1e9614989fff2f68208fe10",
"version" : "1.1.0"
}
},
{
"identity" : "swift-collections",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-collections.git",
"state" : {
"revision" : "937e904258d22af6e447a0b72c0bc67583ef64a2",
"version" : "1.0.4"
}
},
{
"identity" : "swift-numerics",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-numerics",
"state" : {
"revision" : "0a5bc04095a675662cf24757cc0640aa2204253b",
"version" : "1.0.2"
}
}
],
"version" : 2
}
@@ -0,0 +1,11 @@
package pkg
type DotnetPortableExecutableMetadata struct {
AssemblyVersion string `json:"assemblyVersion"`
LegalCopyright string `json:"legalCopyright"`
Comments string `json:"comments,omitempty"`
InternalName string `json:"internalName,omitempty"`
CompanyName string `json:"companyName"`
ProductName string `json:"productName"`
ProductVersion string `json:"productVersion"`
}
+1 -1
View File
@@ -82,7 +82,7 @@ func LanguageByName(name string) Language {
return Dart
case packageurl.TypeDotnet:
return Dotnet
case packageurl.TypeCocoapods, packageurl.TypeSwift, string(CocoapodsPkg):
case packageurl.TypeCocoapods, packageurl.TypeSwift, string(CocoapodsPkg), string(SwiftPkg):
return Swift
case packageurl.TypeConan, string(CPP):
return CPP
+7 -3
View File
@@ -70,9 +70,13 @@ func TestLanguageFromPURL(t *testing.T) {
purl: "pkg:cran/base@4.3.0",
want: R,
},
{
purl: "pkg:swift/github.com/apple/swift-numerics/swift-numerics@1.0.2",
want: Swift,
},
}
var languages []string
var languages = strset.New()
var expectedLanguages = strset.New()
for _, ty := range AllLanguages {
expectedLanguages.Add(string(ty))
@@ -87,14 +91,14 @@ func TestLanguageFromPURL(t *testing.T) {
actual := LanguageFromPURL(tt.purl)
if actual != "" {
languages = append(languages, string(actual))
languages.Add(string(actual))
}
assert.Equalf(t, tt.want, actual, "LanguageFromPURL(%v)", tt.purl)
})
}
assert.ElementsMatch(t, expectedLanguages.List(), languages, "missing one or more languages to test against (maybe a package type was added?)")
assert.ElementsMatch(t, expectedLanguages.List(), languages.List(), "missing one or more languages to test against (maybe a package type was added?)")
}
+67 -61
View File
@@ -10,37 +10,39 @@ type MetadataType string
const (
// this is the full set of data shapes that can be represented within the pkg.Package.Metadata field
UnknownMetadataType MetadataType = "UnknownMetadata"
AlpmMetadataType MetadataType = "AlpmMetadata"
ApkMetadataType MetadataType = "ApkMetadata"
BinaryMetadataType MetadataType = "BinaryMetadata"
CocoapodsMetadataType MetadataType = "CocoapodsMetadataType"
ConanLockMetadataType MetadataType = "ConanLockMetadataType"
ConanMetadataType MetadataType = "ConanMetadataType"
DartPubMetadataType MetadataType = "DartPubMetadata"
DotnetDepsMetadataType MetadataType = "DotnetDepsMetadata"
DpkgMetadataType MetadataType = "DpkgMetadata"
GemMetadataType MetadataType = "GemMetadata"
GolangBinMetadataType MetadataType = "GolangBinMetadata"
GolangModMetadataType MetadataType = "GolangModMetadata"
HackageMetadataType MetadataType = "HackageMetadataType"
JavaMetadataType MetadataType = "JavaMetadata"
KbPackageMetadataType MetadataType = "KbPackageMetadata"
LinuxKernelMetadataType MetadataType = "LinuxKernelMetadata"
LinuxKernelModuleMetadataType MetadataType = "LinuxKernelModuleMetadata"
MixLockMetadataType MetadataType = "MixLockMetadataType"
NixStoreMetadataType MetadataType = "NixStoreMetadata"
NpmPackageJSONMetadataType MetadataType = "NpmPackageJsonMetadata"
NpmPackageLockJSONMetadataType MetadataType = "NpmPackageLockJsonMetadata"
PhpComposerJSONMetadataType MetadataType = "PhpComposerJsonMetadata"
PortageMetadataType MetadataType = "PortageMetadata"
PythonPackageMetadataType MetadataType = "PythonPackageMetadata"
PythonPipfileLockMetadataType MetadataType = "PythonPipfileLockMetadata"
PythonRequirementsMetadataType MetadataType = "PythonRequirementsMetadata"
RebarLockMetadataType MetadataType = "RebarLockMetadataType"
RDescriptionFileMetadataType MetadataType = "RDescriptionFileMetadataType"
RpmMetadataType MetadataType = "RpmMetadata"
RustCargoPackageMetadataType MetadataType = "RustCargoPackageMetadata"
UnknownMetadataType MetadataType = "UnknownMetadata"
AlpmMetadataType MetadataType = "AlpmMetadata"
ApkMetadataType MetadataType = "ApkMetadata"
BinaryMetadataType MetadataType = "BinaryMetadata"
CocoapodsMetadataType MetadataType = "CocoapodsMetadataType"
ConanLockMetadataType MetadataType = "ConanLockMetadataType"
ConanMetadataType MetadataType = "ConanMetadataType"
DartPubMetadataType MetadataType = "DartPubMetadata"
DotnetDepsMetadataType MetadataType = "DotnetDepsMetadata"
DotnetPortableExecutableMetadataType MetadataType = "DotnetPortableExecutableMetadata"
DpkgMetadataType MetadataType = "DpkgMetadata"
GemMetadataType MetadataType = "GemMetadata"
GolangBinMetadataType MetadataType = "GolangBinMetadata"
GolangModMetadataType MetadataType = "GolangModMetadata"
HackageMetadataType MetadataType = "HackageMetadataType"
JavaMetadataType MetadataType = "JavaMetadata"
KbPackageMetadataType MetadataType = "KbPackageMetadata"
LinuxKernelMetadataType MetadataType = "LinuxKernelMetadata"
LinuxKernelModuleMetadataType MetadataType = "LinuxKernelModuleMetadata"
MixLockMetadataType MetadataType = "MixLockMetadataType"
NixStoreMetadataType MetadataType = "NixStoreMetadata"
NpmPackageJSONMetadataType MetadataType = "NpmPackageJsonMetadata"
NpmPackageLockJSONMetadataType MetadataType = "NpmPackageLockJsonMetadata"
PhpComposerJSONMetadataType MetadataType = "PhpComposerJsonMetadata"
PortageMetadataType MetadataType = "PortageMetadata"
PythonPackageMetadataType MetadataType = "PythonPackageMetadata"
PythonPipfileLockMetadataType MetadataType = "PythonPipfileLockMetadata"
PythonRequirementsMetadataType MetadataType = "PythonRequirementsMetadata"
RebarLockMetadataType MetadataType = "RebarLockMetadataType"
RDescriptionFileMetadataType MetadataType = "RDescriptionFileMetadataType"
RpmMetadataType MetadataType = "RpmMetadata"
RustCargoPackageMetadataType MetadataType = "RustCargoPackageMetadata"
SwiftPackageManagerMetadataType MetadataType = "SwiftPackageManagerMetadata"
)
var AllMetadataTypes = []MetadataType{
@@ -52,6 +54,7 @@ var AllMetadataTypes = []MetadataType{
ConanMetadataType,
DartPubMetadataType,
DotnetDepsMetadataType,
DotnetPortableExecutableMetadataType,
DpkgMetadataType,
GemMetadataType,
GolangBinMetadataType,
@@ -74,39 +77,42 @@ var AllMetadataTypes = []MetadataType{
RebarLockMetadataType,
RpmMetadataType,
RustCargoPackageMetadataType,
SwiftPackageManagerMetadataType,
}
var MetadataTypeByName = map[MetadataType]reflect.Type{
AlpmMetadataType: reflect.TypeOf(AlpmMetadata{}),
ApkMetadataType: reflect.TypeOf(ApkMetadata{}),
BinaryMetadataType: reflect.TypeOf(BinaryMetadata{}),
CocoapodsMetadataType: reflect.TypeOf(CocoapodsMetadata{}),
ConanLockMetadataType: reflect.TypeOf(ConanLockMetadata{}),
ConanMetadataType: reflect.TypeOf(ConanMetadata{}),
DartPubMetadataType: reflect.TypeOf(DartPubMetadata{}),
DotnetDepsMetadataType: reflect.TypeOf(DotnetDepsMetadata{}),
DpkgMetadataType: reflect.TypeOf(DpkgMetadata{}),
GemMetadataType: reflect.TypeOf(GemMetadata{}),
GolangBinMetadataType: reflect.TypeOf(GolangBinMetadata{}),
GolangModMetadataType: reflect.TypeOf(GolangModMetadata{}),
HackageMetadataType: reflect.TypeOf(HackageMetadata{}),
JavaMetadataType: reflect.TypeOf(JavaMetadata{}),
KbPackageMetadataType: reflect.TypeOf(KbPackageMetadata{}),
LinuxKernelMetadataType: reflect.TypeOf(LinuxKernelMetadata{}),
LinuxKernelModuleMetadataType: reflect.TypeOf(LinuxKernelModuleMetadata{}),
MixLockMetadataType: reflect.TypeOf(MixLockMetadata{}),
NixStoreMetadataType: reflect.TypeOf(NixStoreMetadata{}),
NpmPackageJSONMetadataType: reflect.TypeOf(NpmPackageJSONMetadata{}),
NpmPackageLockJSONMetadataType: reflect.TypeOf(NpmPackageLockJSONMetadata{}),
PhpComposerJSONMetadataType: reflect.TypeOf(PhpComposerJSONMetadata{}),
PortageMetadataType: reflect.TypeOf(PortageMetadata{}),
PythonPackageMetadataType: reflect.TypeOf(PythonPackageMetadata{}),
PythonPipfileLockMetadataType: reflect.TypeOf(PythonPipfileLockMetadata{}),
PythonRequirementsMetadataType: reflect.TypeOf(PythonRequirementsMetadata{}),
RDescriptionFileMetadataType: reflect.TypeOf(RDescriptionFileMetadata{}),
RebarLockMetadataType: reflect.TypeOf(RebarLockMetadata{}),
RpmMetadataType: reflect.TypeOf(RpmMetadata{}),
RustCargoPackageMetadataType: reflect.TypeOf(CargoPackageMetadata{}),
AlpmMetadataType: reflect.TypeOf(AlpmMetadata{}),
ApkMetadataType: reflect.TypeOf(ApkMetadata{}),
BinaryMetadataType: reflect.TypeOf(BinaryMetadata{}),
CocoapodsMetadataType: reflect.TypeOf(CocoapodsMetadata{}),
ConanLockMetadataType: reflect.TypeOf(ConanLockMetadata{}),
ConanMetadataType: reflect.TypeOf(ConanMetadata{}),
DartPubMetadataType: reflect.TypeOf(DartPubMetadata{}),
DotnetDepsMetadataType: reflect.TypeOf(DotnetDepsMetadata{}),
DotnetPortableExecutableMetadataType: reflect.TypeOf(DotnetPortableExecutableMetadata{}),
DpkgMetadataType: reflect.TypeOf(DpkgMetadata{}),
GemMetadataType: reflect.TypeOf(GemMetadata{}),
GolangBinMetadataType: reflect.TypeOf(GolangBinMetadata{}),
GolangModMetadataType: reflect.TypeOf(GolangModMetadata{}),
HackageMetadataType: reflect.TypeOf(HackageMetadata{}),
JavaMetadataType: reflect.TypeOf(JavaMetadata{}),
KbPackageMetadataType: reflect.TypeOf(KbPackageMetadata{}),
LinuxKernelMetadataType: reflect.TypeOf(LinuxKernelMetadata{}),
LinuxKernelModuleMetadataType: reflect.TypeOf(LinuxKernelModuleMetadata{}),
MixLockMetadataType: reflect.TypeOf(MixLockMetadata{}),
NixStoreMetadataType: reflect.TypeOf(NixStoreMetadata{}),
NpmPackageJSONMetadataType: reflect.TypeOf(NpmPackageJSONMetadata{}),
NpmPackageLockJSONMetadataType: reflect.TypeOf(NpmPackageLockJSONMetadata{}),
PhpComposerJSONMetadataType: reflect.TypeOf(PhpComposerJSONMetadata{}),
PortageMetadataType: reflect.TypeOf(PortageMetadata{}),
PythonPackageMetadataType: reflect.TypeOf(PythonPackageMetadata{}),
PythonPipfileLockMetadataType: reflect.TypeOf(PythonPipfileLockMetadata{}),
PythonRequirementsMetadataType: reflect.TypeOf(PythonRequirementsMetadata{}),
RDescriptionFileMetadataType: reflect.TypeOf(RDescriptionFileMetadata{}),
RebarLockMetadataType: reflect.TypeOf(RebarLockMetadata{}),
RpmMetadataType: reflect.TypeOf(RpmMetadata{}),
RustCargoPackageMetadataType: reflect.TypeOf(CargoPackageMetadata{}),
SwiftPackageManagerMetadataType: reflect.TypeOf(SwiftPackageManagerMetadata{}),
}
func CleanMetadataType(typ MetadataType) MetadataType {
+5 -5
View File
@@ -1,9 +1,9 @@
package pkg
type PythonRequirementsMetadata struct {
Name string `json:"name" mapstruct:"Name"`
Extras []string `json:"extras" mapstruct:"Extras"`
VersionConstraint string `json:"versionConstraint" mapstruct:"VersionConstraint"`
URL string `json:"url" mapstruct:"URL"`
Markers map[string]string `json:"markers" mapstruct:"Markers"`
Name string `json:"name" mapstruct:"Name"`
Extras []string `json:"extras,omitempty" mapstruct:"Extras"`
VersionConstraint string `json:"versionConstraint" mapstruct:"VersionConstraint"`
URL string `json:"url,omitempty" mapstruct:"URL"`
Markers string `json:"markers,omitempty" mapstruct:"Markers"`
}
+5
View File
@@ -0,0 +1,5 @@
package pkg
type SwiftPackageManagerMetadata struct {
Revision string `mapstructure:"revision" json:"revision"`
}
+6
View File
@@ -36,6 +36,7 @@ const (
Rpkg Type = "R-package"
RpmPkg Type = "rpm"
RustPkg Type = "rust-crate"
SwiftPkg Type = "swift"
)
// AllPkgs represents all supported package types
@@ -65,6 +66,7 @@ var AllPkgs = []Type{
Rpkg,
RpmPkg,
RustPkg,
SwiftPkg,
}
// PackageURLType returns the PURL package type for the current package.
@@ -114,6 +116,8 @@ func (t Type) PackageURLType() string {
return packageurl.TypeRPM
case RustPkg:
return "cargo"
case SwiftPkg:
return packageurl.TypeSwift
default:
// TODO: should this be a "generic" purl type instead?
return ""
@@ -179,6 +183,8 @@ func TypeByName(name string) Type {
return NixPkg
case packageurl.TypeCran:
return Rpkg
case packageurl.TypeSwift:
return SwiftPkg
default:
return UnknownPkg
}
+4
View File
@@ -95,6 +95,10 @@ func TestTypeFromPURL(t *testing.T) {
purl: "pkg:cran/base@4.3.0",
expected: Rpkg,
},
{
purl: "pkg:swift/github.com/apple/swift-numerics/swift-numerics@1.0.2",
expected: SwiftPkg,
},
}
var pkgTypes []string
+12 -4
View File
@@ -5,6 +5,7 @@ import (
"golang.org/x/exp/slices"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/linux"
@@ -77,11 +78,18 @@ func (s SBOM) RelationshipsForPackage(p pkg.Package, rt ...artifact.Relationship
var relationships []artifact.Relationship
for _, relationship := range s.Relationships {
// check if the relationship is one we're searching for; rt is inclusive
idx := slices.IndexFunc(rt, func(r artifact.RelationshipType) bool { return relationship.Type == r })
if relationship.From.ID() == p.ID() && idx != -1 {
relationships = append(relationships, relationship)
if relationship.From == nil || relationship.To == nil {
log.Debugf("relationship has nil edge, skipping: %#v", relationship)
continue
}
if relationship.From.ID() != p.ID() {
continue
}
// check if the relationship is one we're searching for; rt is inclusive
if !slices.ContainsFunc(rt, func(r artifact.RelationshipType) bool { return relationship.Type == r }) {
continue
}
relationships = append(relationships, relationship)
}
return relationships
+3
View File
@@ -120,6 +120,9 @@ func (s DirectorySource) Describe() Description {
if a.Name != "" {
name = a.Name
}
if a.Version != "" {
version = a.Version
}
}
return Description{
ID: string(s.id),
+36 -7
View File
@@ -5,10 +5,12 @@ import (
"fmt"
"github.com/bmatcuk/doublestar/v4"
"github.com/docker/distribution/reference"
"github.com/opencontainers/go-digest"
"github.com/anchore/stereoscope"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/internal/fileresolver"
@@ -83,18 +85,45 @@ func (s StereoscopeImageSource) ID() artifact.ID {
}
func (s StereoscopeImageSource) Describe() Description {
name := s.metadata.UserInput
version := s.metadata.ManifestDigest
a := s.config.Alias
if a.Name != "" {
name = a.Name
name := a.Name
nameIfUnset := func(n string) {
if name != "" {
return
}
name = n
}
if a.Version != "" {
version = a.Version
version := a.Version
versionIfUnset := func(v string) {
if version != "" && version != "latest" {
return
}
version = v
}
ref, err := reference.Parse(s.metadata.UserInput)
if err != nil {
log.Debugf("unable to parse image ref: %s", s.config.Reference)
} else {
if ref, ok := ref.(reference.Named); ok {
nameIfUnset(ref.Name())
} else {
nameIfUnset(s.metadata.UserInput)
}
if ref, ok := ref.(reference.NamedTagged); ok {
versionIfUnset(ref.Tag())
}
if ref, ok := ref.(reference.Digested); ok {
versionIfUnset(ref.Digest().String())
}
}
versionIfUnset(s.metadata.ManifestDigest)
return Description{
ID: string(s.id),
Name: name,
+7 -7
View File
@@ -10,7 +10,7 @@ func TestPackagesCmdFlags(t *testing.T) {
hiddenPackagesImage := "docker-archive:" + getFixtureImage(t, "image-hidden-packages")
coverageImage := "docker-archive:" + getFixtureImage(t, "image-pkg-coverage")
nodeBinaryImage := "docker-archive:" + getFixtureImage(t, "image-node-binary")
//badBinariesImage := "docker-archive:" + getFixtureImage(t, "image-bad-binaries")
// badBinariesImage := "docker-archive:" + getFixtureImage(t, "image-bad-binaries")
tmp := t.TempDir() + "/"
tests := []struct {
@@ -51,7 +51,7 @@ func TestPackagesCmdFlags(t *testing.T) {
// fail: https://github.com/anchore/syft/runs/4611343586?check_suite_focus=true
// For the meantime this test will be commented out, but should be added back in as soon as possible.
//
//{
// {
// name: "regression-survive-bad-binaries",
// // this image has all sorts of rich binaries from the clang-13 test suite that should do pretty bad things
// // to the go cataloger binary path. We should NEVER let a panic stop the cataloging process for these
@@ -64,7 +64,7 @@ func TestPackagesCmdFlags(t *testing.T) {
// assertInOutput("could not parse possible go binary"),
// assertSuccessfulReturnCode,
// },
//},
// },
{
name: "output-env-binding",
env: map[string]string{
@@ -96,7 +96,7 @@ func TestPackagesCmdFlags(t *testing.T) {
name: "squashed-scope-flag",
args: []string{"packages", "-o", "json", "-s", "squashed", coverageImage},
assertions: []traitAssertion{
assertPackageCount(36),
assertPackageCount(24),
assertSuccessfulReturnCode,
},
},
@@ -213,7 +213,7 @@ func TestPackagesCmdFlags(t *testing.T) {
// the application config in the log matches that of what we expect to have been configured.
assertInOutput("parallelism: 2"),
assertInOutput("parallelism=2"),
assertPackageCount(36),
assertPackageCount(24),
assertSuccessfulReturnCode,
},
},
@@ -224,7 +224,7 @@ func TestPackagesCmdFlags(t *testing.T) {
// the application config in the log matches that of what we expect to have been configured.
assertInOutput("parallelism: 1"),
assertInOutput("parallelism=1"),
assertPackageCount(36),
assertPackageCount(24),
assertSuccessfulReturnCode,
},
},
@@ -238,7 +238,7 @@ func TestPackagesCmdFlags(t *testing.T) {
assertions: []traitAssertion{
assertNotInOutput("secret_password"),
assertNotInOutput("secret_key_path"),
assertPackageCount(36),
assertPackageCount(24),
assertSuccessfulReturnCode,
},
},
@@ -356,6 +356,18 @@ var dirOnlyTestCases = []testCase{
"unicode_util_compat": "0.7.0",
},
},
{
name: "find swift package manager packages",
pkgType: pkg.SwiftPkg,
pkgLanguage: pkg.Swift,
pkgInfo: map[string]string{
"swift-algorithms": "1.0.0",
"swift-async-algorithms": "0.1.0",
"swift-atomics": "1.1.0",
"swift-collections": "1.0.4",
"swift-numerics": "1.0.2",
},
},
}
var commonTestCases = []testCase{
@@ -95,6 +95,7 @@ func TestPkgCoverageImage(t *testing.T) {
definedPkgs.Remove(string(pkg.HexPkg))
definedPkgs.Remove(string(pkg.LinuxKernelPkg))
definedPkgs.Remove(string(pkg.LinuxKernelModulePkg))
definedPkgs.Remove(string(pkg.SwiftPkg))
var cases []testCase
cases = append(cases, commonTestCases...)
@@ -0,0 +1,50 @@
{
"pins" : [
{
"identity" : "swift-algorithms",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-algorithms.git",
"state" : {
"revision" : "b14b7f4c528c942f121c8b860b9410b2bf57825e",
"version" : "1.0.0"
}
},
{
"identity" : "swift-async-algorithms",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-async-algorithms.git",
"state" : {
"revision" : "9cfed92b026c524674ed869a4ff2dcfdeedf8a2a",
"version" : "0.1.0"
}
},
{
"identity" : "swift-atomics",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-atomics.git",
"state" : {
"revision" : "6c89474e62719ddcc1e9614989fff2f68208fe10",
"version" : "1.1.0"
}
},
{
"identity" : "swift-collections",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-collections.git",
"state" : {
"revision" : "937e904258d22af6e447a0b72c0bc67583ef64a2",
"version" : "1.0.4"
}
},
{
"identity" : "swift-numerics",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-numerics",
"state" : {
"revision" : "0a5bc04095a675662cf24757cc0640aa2204253b",
"version" : "1.0.2"
}
}
],
"version" : 2
}
-783
View File
@@ -1,783 +0,0 @@
/*
Package ui provides all public UI elements intended to be repurposed in other applications. Specifically, a single
Handler object is provided to allow consuming applications (such as grype) to check if there are UI elements the handler
can respond to (given a specific event type) and handle the event in context of the given screen frame object.
*/
package ui
import (
"bufio"
"container/list"
"context"
"errors"
"fmt"
"io"
"strings"
"sync"
"time"
"github.com/dustin/go-humanize"
"github.com/gookit/color"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
"github.com/wagoodman/go-progress/format"
"github.com/wagoodman/jotframe/pkg/frame"
stereoscopeEvent "github.com/anchore/stereoscope/pkg/event"
stereoEventParsers "github.com/anchore/stereoscope/pkg/event/parsers"
"github.com/anchore/stereoscope/pkg/image/docker"
"github.com/anchore/syft/internal"
syftEvent "github.com/anchore/syft/syft/event"
syftEventParsers "github.com/anchore/syft/syft/event/parsers"
)
const maxBarWidth = 50
const statusSet = SpinnerDotSet
const completedStatus = "✔"
const failedStatus = "✘"
const titleFormat = color.Bold
const subTitleFormat = color.Normal
const interval = 150 * time.Millisecond
// StatusTitleColumn is the column index in a given row where status text will be displayed.
const StatusTitleColumn = 31
var (
auxInfoFormat = color.HEX("#777777")
dockerPullCompletedColor = color.HEX("#fcba03")
dockerPullDownloadColor = color.HEX("#777777")
dockerPullExtractColor = color.White
dockerPullStageChars = strings.Split("▁▃▄▅▆▇█", "")
statusTitleTemplate = fmt.Sprintf(" %%s %%-%ds ", StatusTitleColumn)
subStatusTitleTemplate = fmt.Sprintf(" └── %%-%ds ", StatusTitleColumn-3)
)
// Handler is an aggregated event handler for the set of supported events (PullDockerImage, ReadImage, FetchImage, PackageCatalogerStarted)
// Deprecated: use the bubbletea event handler in cmd/syft/ui/handler.go instead.
type Handler struct {
}
// NewHandler returns an empty Handler
// Deprecated: use the bubbletea event handler in cmd/syft/ui/handler.go instead.
func NewHandler() *Handler {
return &Handler{}
}
// RespondsTo indicates if the handler is capable of handling the given event.
// Deprecated: use the bubbletea event handler in cmd/syft/ui/handler.go instead.
func (r *Handler) RespondsTo(event partybus.Event) bool {
switch event.Type {
case stereoscopeEvent.PullDockerImage,
stereoscopeEvent.ReadImage,
stereoscopeEvent.FetchImage,
syftEvent.PackageCatalogerStarted,
syftEvent.SecretsCatalogerStarted,
syftEvent.FileDigestsCatalogerStarted,
syftEvent.FileMetadataCatalogerStarted,
syftEvent.FileIndexingStarted,
syftEvent.AttestationStarted,
syftEvent.CatalogerTaskStarted:
return true
default:
return false
}
}
// Handle calls the specific event handler for the given event within the context of the screen frame.
// Deprecated: use the bubbletea event handler in cmd/syft/ui/handler.go instead.
func (r *Handler) Handle(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
switch event.Type {
case stereoscopeEvent.PullDockerImage:
return PullDockerImageHandler(ctx, fr, event, wg)
case stereoscopeEvent.ReadImage:
return ReadImageHandler(ctx, fr, event, wg)
case stereoscopeEvent.FetchImage:
return FetchImageHandler(ctx, fr, event, wg)
case syftEvent.PackageCatalogerStarted:
return PackageCatalogerStartedHandler(ctx, fr, event, wg)
case syftEvent.SecretsCatalogerStarted:
return SecretsCatalogerStartedHandler(ctx, fr, event, wg)
case syftEvent.FileDigestsCatalogerStarted:
return FileDigestsCatalogerStartedHandler(ctx, fr, event, wg)
case syftEvent.FileMetadataCatalogerStarted:
return FileMetadataCatalogerStartedHandler(ctx, fr, event, wg)
case syftEvent.FileIndexingStarted:
return FileIndexingStartedHandler(ctx, fr, event, wg)
case syftEvent.AttestationStarted:
return AttestationStartedHandler(ctx, fr, event, wg)
case syftEvent.CatalogerTaskStarted:
return CatalogerTaskStartedHandler(ctx, fr, event, wg)
}
return nil
}
const (
SpinnerDotSet = "⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏"
)
type spinner struct {
index int
charset []string
lock sync.Mutex
}
func newSpinner(charset string) spinner {
return spinner{
charset: strings.Split(charset, ""),
}
}
func (s *spinner) Current() string {
s.lock.Lock()
defer s.lock.Unlock()
return s.charset[s.index]
}
func (s *spinner) Next() string {
s.lock.Lock()
defer s.lock.Unlock()
c := s.charset[s.index]
s.index++
if s.index >= len(s.charset) {
s.index = 0
}
return c
}
// startProcess is a helper function for providing common elements for long-running UI elements (such as a
// progress bar formatter and status spinner)
func startProcess() (format.Simple, *spinner) {
width, _ := frame.GetTerminalSize()
barWidth := int(0.25 * float64(width))
if barWidth > maxBarWidth {
barWidth = maxBarWidth
}
formatter := format.NewSimpleWithTheme(barWidth, format.HeavyNoBarTheme, format.ColorCompleted, format.ColorTodo)
spinner := newSpinner(statusSet)
return formatter, &spinner
}
// formatDockerPullPhase returns a single character that represents the status of a layer pull.
func formatDockerPullPhase(phase docker.PullPhase, inputStr string) string {
switch phase {
case docker.WaitingPhase:
// ignore any progress related to waiting
return " "
case docker.PullingFsPhase, docker.DownloadingPhase:
return dockerPullDownloadColor.Sprint(inputStr)
case docker.DownloadCompletePhase:
return dockerPullDownloadColor.Sprint(dockerPullStageChars[len(dockerPullStageChars)-1])
case docker.ExtractingPhase:
return dockerPullExtractColor.Sprint(inputStr)
case docker.VerifyingChecksumPhase, docker.PullCompletePhase:
return dockerPullCompletedColor.Sprint(inputStr)
case docker.AlreadyExistsPhase:
return dockerPullCompletedColor.Sprint(dockerPullStageChars[len(dockerPullStageChars)-1])
default:
return inputStr
}
}
// formatDockerImagePullStatus writes the docker image pull status summarized into a single line for the given state.
func formatDockerImagePullStatus(pullStatus *docker.PullStatus, spinner *spinner, line *frame.Line) {
var size, current uint64
title := titleFormat.Sprint("Pulling image")
layers := pullStatus.Layers()
status := make(map[docker.LayerID]docker.LayerState)
completed := make([]string, len(layers))
// fetch the current state
for idx, layer := range layers {
completed[idx] = " "
status[layer] = pullStatus.Current(layer)
}
numCompleted := 0
for idx, layer := range layers {
prog := status[layer].PhaseProgress
current := prog.Current()
size := prog.Size()
if progress.IsCompleted(prog) {
input := dockerPullStageChars[len(dockerPullStageChars)-1]
completed[idx] = formatDockerPullPhase(status[layer].Phase, input)
} else if current != 0 {
var ratio float64
switch {
case current == 0 || size < 0:
ratio = 0
case current >= size:
ratio = 1
default:
ratio = float64(current) / float64(size)
}
i := int(ratio * float64(len(dockerPullStageChars)-1))
input := dockerPullStageChars[i]
completed[idx] = formatDockerPullPhase(status[layer].Phase, input)
}
if progress.IsErrCompleted(status[layer].DownloadProgress.Error()) {
numCompleted++
}
}
for _, layer := range layers {
prog := status[layer].DownloadProgress
size += uint64(prog.Size())
current += uint64(prog.Current())
}
var progStr, auxInfo string
if len(layers) > 0 {
render := strings.Join(completed, "")
prefix := dockerPullCompletedColor.Sprintf("%d Layers", len(layers))
auxInfo = auxInfoFormat.Sprintf("[%s / %s]", humanize.Bytes(current), humanize.Bytes(size))
if len(layers) == numCompleted {
auxInfo = auxInfoFormat.Sprintf("[%s] Extracting...", humanize.Bytes(size))
}
progStr = fmt.Sprintf("%s▕%s▏", prefix, render)
}
spin := color.Magenta.Sprint(spinner.Next())
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate+"%s%s", spin, title, progStr, auxInfo))
}
// PullDockerImageHandler periodically writes a formatted line widget representing a docker image pull event.
func PullDockerImageHandler(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
_, pullStatus, err := stereoEventParsers.ParsePullDockerImage(event)
if err != nil {
return fmt.Errorf("bad %s event: %w", event.Type, err)
}
line, err := fr.Append()
if err != nil {
return err
}
wg.Add(1)
_, spinner := startProcess()
go func() {
defer wg.Done()
loop:
for {
select {
case <-ctx.Done():
break loop
case <-time.After(interval):
formatDockerImagePullStatus(pullStatus, spinner, line)
if pullStatus.Complete() {
break loop
}
}
}
if pullStatus.Complete() {
spin := color.Green.Sprint(completedStatus)
title := titleFormat.Sprint("Pulled image")
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate, spin, title))
}
}()
return err
}
// FetchImageHandler periodically writes a the image save and write-to-disk process in the form of a progress bar.
func FetchImageHandler(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
_, prog, err := stereoEventParsers.ParseFetchImage(event)
if err != nil {
return fmt.Errorf("bad %s event: %w", event.Type, err)
}
line, err := fr.Append()
if err != nil {
return err
}
wg.Add(1)
formatter, spinner := startProcess()
stream := progress.Stream(ctx, prog, interval)
title := titleFormat.Sprint("Loading image")
formatFn := func(p progress.Progress) {
progStr, err := formatter.Format(p)
spin := color.Magenta.Sprint(spinner.Next())
if err != nil {
_, _ = io.WriteString(line, fmt.Sprintf("Error: %+v", err))
} else {
auxInfo := auxInfoFormat.Sprintf("[%s]", prog.Stage())
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate+"%s %s", spin, title, progStr, auxInfo))
}
}
go func() {
defer wg.Done()
formatFn(progress.Progress{})
for p := range stream {
formatFn(p)
}
spin := color.Green.Sprint(completedStatus)
title = titleFormat.Sprint("Loaded image")
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate, spin, title))
}()
return err
}
// ReadImageHandler periodically writes a the image read/parse/build-tree status in the form of a progress bar.
func ReadImageHandler(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
_, prog, err := stereoEventParsers.ParseReadImage(event)
if err != nil {
return fmt.Errorf("bad %s event: %w", event.Type, err)
}
line, err := fr.Append()
if err != nil {
return err
}
wg.Add(1)
formatter, spinner := startProcess()
stream := progress.Stream(ctx, prog, interval)
title := titleFormat.Sprint("Parsing image")
formatFn := func(p progress.Progress) {
progStr, err := formatter.Format(p)
spin := color.Magenta.Sprint(spinner.Next())
if err != nil {
_, _ = io.WriteString(line, fmt.Sprintf("Error: %+v", err))
} else {
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate+"%s", spin, title, progStr))
}
}
go func() {
defer wg.Done()
formatFn(progress.Progress{})
for p := range stream {
formatFn(p)
}
spin := color.Green.Sprint(completedStatus)
title = titleFormat.Sprint("Parsed image")
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate, spin, title))
}()
return nil
}
// PackageCatalogerStartedHandler periodically writes catalog statistics to a single line.
func PackageCatalogerStartedHandler(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
monitor, err := syftEventParsers.ParsePackageCatalogerStarted(event)
if err != nil {
return fmt.Errorf("bad %s event: %w", event.Type, err)
}
line, err := fr.Append()
if err != nil {
return err
}
wg.Add(1)
_, spinner := startProcess()
stream := progress.StreamMonitors(ctx, []progress.Monitorable{monitor.FilesProcessed, monitor.PackagesDiscovered}, interval)
title := titleFormat.Sprint("Cataloging packages")
formatFn := func(p int64) {
spin := color.Magenta.Sprint(spinner.Next())
auxInfo := auxInfoFormat.Sprintf("[packages %d]", p)
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate+"%s", spin, title, auxInfo))
}
go func() {
defer wg.Done()
formatFn(0)
for p := range stream {
formatFn(p[1])
}
spin := color.Green.Sprint(completedStatus)
title = titleFormat.Sprint("Cataloged packages")
auxInfo := auxInfoFormat.Sprintf("[%d packages]", monitor.PackagesDiscovered.Current())
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate+"%s", spin, title, auxInfo))
}()
return nil
}
// SecretsCatalogerStartedHandler shows the intermittent secrets searching progress.
func SecretsCatalogerStartedHandler(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
prog, err := syftEventParsers.ParseSecretsCatalogingStarted(event)
if err != nil {
return fmt.Errorf("bad %s event: %w", event.Type, err)
}
line, err := fr.Append()
if err != nil {
return err
}
wg.Add(1)
formatter, spinner := startProcess()
stream := progress.Stream(ctx, prog, interval)
title := titleFormat.Sprint("Cataloging secrets")
formatFn := func(p progress.Progress) {
progStr, err := formatter.Format(p)
spin := color.Magenta.Sprint(spinner.Next())
if err != nil {
_, _ = io.WriteString(line, fmt.Sprintf("Error: %+v", err))
} else {
auxInfo := auxInfoFormat.Sprintf("[%d secrets]", prog.SecretsDiscovered.Current())
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate+"%s %s", spin, title, progStr, auxInfo))
}
}
go func() {
defer wg.Done()
formatFn(progress.Progress{})
for p := range stream {
formatFn(p)
}
spin := color.Green.Sprint(completedStatus)
title = titleFormat.Sprint("Cataloged secrets")
auxInfo := auxInfoFormat.Sprintf("[%d secrets]", prog.SecretsDiscovered.Current())
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate+"%s", spin, title, auxInfo))
}()
return err
}
// FileMetadataCatalogerStartedHandler shows the intermittent secrets searching progress.
//
//nolint:dupl
func FileMetadataCatalogerStartedHandler(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
prog, err := syftEventParsers.ParseFileMetadataCatalogingStarted(event)
if err != nil {
return fmt.Errorf("bad %s event: %w", event.Type, err)
}
line, err := fr.Append()
if err != nil {
return err
}
wg.Add(1)
formatter, spinner := startProcess()
stream := progress.Stream(ctx, prog, interval)
title := titleFormat.Sprint("Cataloging file metadata")
formatFn := func(p progress.Progress) {
progStr, err := formatter.Format(p)
spin := color.Magenta.Sprint(spinner.Next())
if err != nil {
_, _ = io.WriteString(line, fmt.Sprintf("Error: %+v", err))
} else {
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate+"%s", spin, title, progStr))
}
}
go func() {
defer wg.Done()
formatFn(progress.Progress{})
for p := range stream {
formatFn(p)
}
spin := color.Green.Sprint(completedStatus)
title = titleFormat.Sprint("Cataloged file metadata")
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate, spin, title))
}()
return err
}
// FileIndexingStartedHandler shows the intermittent indexing progress from a directory resolver.
func FileIndexingStartedHandler(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
path, prog, err := syftEventParsers.ParseFileIndexingStarted(event)
if err != nil {
return fmt.Errorf("bad %s event: %w", event.Type, err)
}
line, err := fr.Append()
if err != nil {
return err
}
wg.Add(1)
_, spinner := startProcess()
stream := progress.Stream(ctx, prog, interval)
title := titleFormat.Sprintf("Indexing %s", path)
formatFn := func(_ progress.Progress) {
spin := color.Magenta.Sprint(spinner.Next())
if err != nil {
_, _ = io.WriteString(line, fmt.Sprintf("Error: %+v", err))
} else {
auxInfo := auxInfoFormat.Sprintf("[file: %s]", internal.TruncateMiddleEllipsis(prog.Stage(), 100))
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate+"%s", spin, title, auxInfo))
}
}
go func() {
defer wg.Done()
formatFn(progress.Progress{})
for p := range stream {
formatFn(p)
}
spin := color.Green.Sprint(completedStatus)
title = titleFormat.Sprintf("Indexed %s", path)
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate, spin, title))
}()
return err
}
// FileMetadataCatalogerStartedHandler shows the intermittent secrets searching progress.
//
//nolint:dupl
func FileDigestsCatalogerStartedHandler(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
prog, err := syftEventParsers.ParseFileDigestsCatalogingStarted(event)
if err != nil {
return fmt.Errorf("bad %s event: %w", event.Type, err)
}
line, err := fr.Append()
if err != nil {
return err
}
wg.Add(1)
formatter, spinner := startProcess()
stream := progress.Stream(ctx, prog, interval)
title := titleFormat.Sprint("Cataloging file digests")
formatFn := func(p progress.Progress) {
progStr, err := formatter.Format(p)
spin := color.Magenta.Sprint(spinner.Next())
if err != nil {
_, _ = io.WriteString(line, fmt.Sprintf("Error: %+v", err))
} else {
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate+"%s", spin, title, progStr))
}
}
go func() {
defer wg.Done()
formatFn(progress.Progress{})
for p := range stream {
formatFn(p)
}
spin := color.Green.Sprint(completedStatus)
title = titleFormat.Sprint("Cataloged file digests")
_, _ = io.WriteString(line, fmt.Sprintf(statusTitleTemplate, spin, title))
}()
return err
}
// AttestationStartedHandler takes bytes from a event.ShellOutput and publishes them to the frame.
//
//nolint:funlen,gocognit
func AttestationStartedHandler(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
reader, prog, taskInfo, err := syftEventParsers.ParseAttestationStartedEvent(event)
if err != nil {
return fmt.Errorf("bad %s event: %w", event.Type, err)
}
titleLine, err := fr.Append()
if err != nil {
return err
}
wg.Add(2)
_, spinner := startProcess()
title := titleFormat.Sprintf(taskInfo.Title.WhileRunning)
s := bufio.NewScanner(reader)
l := list.New()
formatFn := func() {
auxInfo := auxInfoFormat.Sprintf("[running %s]", taskInfo.Context)
spin := color.Magenta.Sprint(spinner.Next())
_, _ = io.WriteString(titleLine, fmt.Sprintf(statusTitleTemplate+"%s", spin, title, auxInfo))
}
formatFn()
var failed bool
formatComplete := func(aux string) {
spin := color.Green.Sprint(completedStatus)
if failed {
spin = color.Red.Sprint(failedStatus)
aux = prog.Error().Error()
} else {
title = titleFormat.Sprintf(taskInfo.Title.OnSuccess)
}
auxInfo := auxInfoFormat.Sprintf("[%s]", aux)
_, _ = io.WriteString(titleLine, fmt.Sprintf(statusTitleTemplate+"%s", spin, title, auxInfo))
}
endWg := &sync.WaitGroup{}
endWg.Add(1)
go func() {
defer wg.Done()
defer endWg.Done()
stream := progress.Stream(ctx, prog, interval)
for range stream {
formatFn()
}
err := prog.Error()
if err != nil && !errors.Is(err, io.EOF) {
failed = true
}
}()
go func() {
defer wg.Done()
var tlogEntry string
// only show the last 5 lines of the shell output
for s.Scan() {
line, _ := fr.Append()
if l.Len() > 5 {
elem := l.Front()
line, ok := elem.Value.(*frame.Line)
if !ok {
continue
}
err = line.Remove()
if err != nil {
return
}
l.Remove(elem)
}
l.PushBack(line)
text := s.Text()
if strings.Contains(text, "tlog entry created with index") {
tlogEntry = text
} else {
// no tlog entry create so user used personal PKI
tlogEntry = "signed attestation using provided key"
}
_, err = line.Write([]byte(fmt.Sprintf(" %s %s", auxInfoFormat.Sprintf("░░"), text)))
if err != nil {
return
}
}
endWg.Wait()
if !failed {
// roll up logs into completed status (only if successful)
for e := l.Back(); e != nil; e = e.Prev() {
line, ok := e.Value.(*frame.Line)
if !ok {
continue
}
err = line.Remove()
if err != nil {
return
}
}
}
formatComplete(tlogEntry)
}()
return nil
}
// CatalogerTaskStartedHandler shows the intermittent progress for a cataloger subprocess messages
func CatalogerTaskStartedHandler(ctx context.Context, fr *frame.Frame, event partybus.Event, wg *sync.WaitGroup) error {
prog, err := syftEventParsers.ParseCatalogerTaskStarted(event)
if err != nil {
return fmt.Errorf("bad %s event: %w", event.Type, err)
}
line, err := fr.Append()
if err != nil {
return err
}
wg.Add(1)
stream := progress.Stream(ctx, prog.GetMonitor(), interval)
_, spinner := startProcess()
formatLine := func(complete bool, auxInfo string) string {
title := prog.Title
if complete && prog.TitleOnCompletion != "" {
title = prog.TitleOnCompletion
}
if prog.SubStatus {
title = subTitleFormat.Sprintf("%s", title)
if auxInfo == "" {
return fmt.Sprintf(subStatusTitleTemplate, title)
}
return fmt.Sprintf(subStatusTitleTemplate+"%s", title, auxInfo)
}
spin := color.Magenta.Sprint(spinner.Next())
if complete {
spin = color.Green.Sprint(completedStatus)
}
title = titleFormat.Sprintf("%s", title)
if auxInfo == "" {
return fmt.Sprintf(statusTitleTemplate, spin, title)
}
return fmt.Sprintf(statusTitleTemplate+"%s", spin, title, auxInfo)
}
formatFn := func() {
if err != nil {
_, _ = io.WriteString(line, fmt.Sprintf("Error: %+v", err))
} else {
auxInfo := auxInfoFormat.Sprintf("[%s]", internal.TruncateMiddleEllipsis(prog.GetValue(), 100))
_, _ = io.WriteString(line, formatLine(false, auxInfo))
}
}
go func() {
defer wg.Done()
formatFn()
for range stream {
formatFn()
}
if prog.RemoveOnCompletion {
_ = fr.Remove(line)
} else {
_, _ = io.WriteString(line, formatLine(true, ""))
}
}()
return err
}