Compare commits

..
Author SHA1 Message Date
Alex Goodman 47cc8b58a7 persist artifact ID as supplemental package data
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2025-05-07 21:51:58 -04:00
dependabot[bot] 8aaf36b1ad chore(deps): bump github.com/github/go-spdx/v2 from 2.3.2 to 2.3.3 (#3863)
Bumps [github.com/github/go-spdx/v2](https://github.com/github/go-spdx) from 2.3.2 to 2.3.3.
- [Release notes](https://github.com/github/go-spdx/releases)
- [Commits](https://github.com/github/go-spdx/compare/v2.3.2...v2.3.3)

---
updated-dependencies:
- dependency-name: github.com/github/go-spdx/v2
  dependency-version: 2.3.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-07 10:00:05 -04:00
dependabot[bot] af273002b8 chore(deps): bump golang.org/x/net from 0.39.0 to 0.40.0 (#3859)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.39.0 to 0.40.0.
- [Commits](https://github.com/golang/net/compare/v0.39.0...v0.40.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.40.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-06 12:12:58 -04:00
Christopher Angelo Phillips 6eff158ad3 chore: update license sort to be stable with contents field (#3860)
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
2025-05-06 11:45:47 -04:00
Alex Goodman 7b25ea5eda annotate hidden paths in all-layers scope (#3855)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2025-05-06 09:50:04 -04:00
Christopher Angelo Phillips 1ba1186410 fix: use "contents" field and remove "fullText" license field (#3857)
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
2025-05-05 21:40:09 +00:00
GGMUandAlex Goodman 6db60c5975 Add deep-squashed scope to annotate all layers where a package exists (#3138)
* add squash all layers resolver

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* add squash with all layers logic

Signed-off-by: tomersein <tomersein@gmail.com>

* add squash with all layers logic

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squashed all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squash with all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* squash with all layers

Signed-off-by: tomersein <tomersein@gmail.com>

* adjust resolver to strictly return squash paths only

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* show all packages have locations + primary evidence

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix race condition in test

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* consider access paths

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: tomersein <tomersein@gmail.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2025-05-05 14:35:57 -04:00
Kudryavcev Nikolay e13c9e7813 fix: propagate unarchive error of file source (#3845)
Signed-off-by: Kudryavcev Nikolay <kydry.nikolau@gmail.com>
2025-05-05 15:37:47 +00:00
Alex Goodman d47a6c3a6d Improve support for cataloging nix package relationships (#3837)
* add nix DB cataloger

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add derivation path to nix store pkg metadata

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* go mod tidy

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* allow for derivation path to be optional

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* repin build image and disable syscall filtering

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* bump storage capacity

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* track nix derivation details on packages

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* image fixture should have derivation examples

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* address comments

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2025-05-05 15:35:13 +00:00
anchore-actions-token-generator[bot]andwagoodman 7505a04aad chore(deps): update tools to latest versions (#3848)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: wagoodman <590471+wagoodman@users.noreply.github.com>
2025-05-05 15:26:14 +00:00
anchore-actions-token-generator[bot]andwagoodman f1620b120a chore(deps): update CPE dictionary index (#3851)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: wagoodman <590471+wagoodman@users.noreply.github.com>
2025-05-05 15:14:28 +00:00
Christopher Angelo Phillips 00f53b1777 chore: upgrade fixtures to use version 4 lockfile (#3852)
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
2025-05-05 14:38:23 +00:00
dependabot[bot] 3faf43d592 chore(deps): bump github/codeql-action from 3.28.16 to 3.28.17 (#3846)
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.16 to 3.28.17.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/28deaeda66b76a05916b6923827895f2b14ab387...60168efe1c415ce0f5521ea06d5c2062adbeed1b)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 3.28.17
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 10:23:08 -04:00
Christopher Angelo Phillips 6ba087c72c fix: Do not use hashes for SPDX license names/expressions (#3844)
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
2025-05-02 09:34:08 -04:00
Christopher Angelo Phillips 94e63eb367 feat: detect when full license text has been provided and preserve as separate field (#3450)
* feat: add full text field to syft license struct
---------
Signed-off-by: Christopher Angelo Phillips <32073428+spiffcs@users.noreply.github.com>
2025-05-01 15:00:46 -04:00
dependabot[bot] 4999de4114 chore(deps): bump github.com/Masterminds/semver/v3 from 3.3.0 to 3.3.1 (#3843)
Bumps [github.com/Masterminds/semver/v3](https://github.com/Masterminds/semver) from 3.3.0 to 3.3.1.
- [Release notes](https://github.com/Masterminds/semver/releases)
- [Changelog](https://github.com/Masterminds/semver/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Masterminds/semver/compare/v3.3.0...v3.3.1)

---
updated-dependencies:
- dependency-name: github.com/Masterminds/semver/v3
  dependency-version: 3.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-01 10:06:11 -04:00
anchore-actions-token-generator[bot]andspiffcs 9ecfe9a53c chore(deps): update tools to latest versions (#3841)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: spiffcs <32073428+spiffcs@users.noreply.github.com>
2025-05-01 09:29:19 -04:00
Alan PopeandAlex Goodman baa1080ef6 Update github.com/Masterminds/semver to v3 (#3836)
* Update semver to v3. Fixes #3829

Signed-off-by: Alan Pope <alan.pope@anchore.com>

* use single instance of regex obj

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alan Pope <alan.pope@anchore.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2025-04-30 20:38:12 +00:00
Laurent GoderreandAlex Goodman 529840bfc0 Add support for PHP Pear (#2775)
* Add support for PHP Pear and unify PECL with it

Signed-off-by: Laurent Goderre <laurent.goderre@docker.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix tests

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* remove log statements

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix struct comment

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Laurent Goderre <laurent.goderre@docker.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2025-04-30 20:16:58 +00:00
Oleksandr Vodotiiets 78ef2cf53b fix: Improve detection of erlang binary in alpine Linux (#3839)
Signed-off-by: Oleksandr Vodotiiets <alex.vodotiiets@gmail.com>
Signed-off-by: Oleksandr Vodotiiets <61548316+avodotiiets@users.noreply.github.com>
2025-04-30 18:50:12 +00:00
VictorHuu 09c3b7cbea fix:Resolve ancestral symlinks correctly (#3783)
* Resolve upstream symlinks correctly

Signed-off-by: Yuntao Hu <victorhu493@gmail.com>

* in case of the root directory

Signed-off-by: Yuntao Hu <victorhu493@gmail.com>

* for static analysis check pass

Signed-off-by: Yuntao Hu <victorhu493@gmail.com>

* add unit test cases for the symlink scenarios

Signed-off-by: Yuntao Hu <victorhu493@gmail.com>

---------

Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
2025-04-30 14:47:32 -04:00
anchore-actions-token-generator[bot]andwagoodman 6dca10fe1f chore(deps): update CPE dictionary index (#3834)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: wagoodman <590471+wagoodman@users.noreply.github.com>
2025-04-30 14:40:52 -04:00
anchore-actions-token-generator[bot]andspiffcs 1ecf1ce7bf chore(deps): update tools to latest versions (#3835)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: spiffcs <32073428+spiffcs@users.noreply.github.com>
2025-04-30 14:40:37 -04:00
dependabot[bot] 20ca60de8b chore(deps): bump github.com/charmbracelet/bubbletea from 1.3.4 to 1.3.5 (#3838)
Bumps [github.com/charmbracelet/bubbletea](https://github.com/charmbracelet/bubbletea) from 1.3.4 to 1.3.5.
- [Release notes](https://github.com/charmbracelet/bubbletea/releases)
- [Changelog](https://github.com/charmbracelet/bubbletea/blob/main/.goreleaser.yml)
- [Commits](https://github.com/charmbracelet/bubbletea/compare/v1.3.4...v1.3.5)

---
updated-dependencies:
- dependency-name: github.com/charmbracelet/bubbletea
  dependency-version: 1.3.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-30 14:40:00 -04:00
VictorHuu fa599547a3 fix the fluent-bit regex detection pattern (#3817)
Signed-off-by: Yuntao Hu <victorhu493@gmail.com>
2025-04-25 11:50:45 -04:00
dependabot[bot] 22d8b30813 chore(deps): bump anchore/sbom-action from 0.18.0 to 0.19.0 (#3832)
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.18.0 to 0.19.0.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/f325610c9f50a54015d37c8d16cb3b0e2c8f4de0...9f7302141466aa6482940f15371237e9d9f4c34a)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-25 11:29:24 -04:00
anchore-actions-token-generator[bot]andspiffcs a714fb8391 chore(deps): update tools to latest versions (#3830)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: spiffcs <32073428+spiffcs@users.noreply.github.com>
2025-04-25 09:59:29 -04:00
Alex Goodman 03fa142de9 Resolve owned file paths when searching for overlaps (#3828)
* resolve owned file paths when searching for overlaps

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* dont remove empty paths

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2025-04-24 21:59:45 +00:00
124 changed files with 14065 additions and 1070 deletions
+6 -6
View File
@@ -26,7 +26,7 @@ tools:
# used for linting
- name: golangci-lint
version:
want: v2.1.2
want: v2.1.6
method: github-release
with:
repo: golangci/golangci-lint
@@ -58,7 +58,7 @@ tools:
# used to release all artifacts
- name: goreleaser
version:
want: v2.8.2
want: v2.9.0
method: github-release
with:
repo: goreleaser/goreleaser
@@ -103,7 +103,7 @@ tools:
# used for running all local and CI tasks
- name: task
version:
want: v3.43.2
want: v3.43.3
method: github-release
with:
repo: go-task/task
@@ -111,7 +111,7 @@ tools:
# used for triggering a release
- name: gh
version:
want: v2.71.0
want: v2.72.0
method: github-release
with:
repo: cli/cli
@@ -119,7 +119,7 @@ tools:
# used to upload test fixture cache
- name: oras
version:
want: v1.2.2
want: v1.2.3
method: github-release
with:
repo: oras-project/oras
@@ -127,7 +127,7 @@ tools:
# used to upload test fixture cache
- name: yq
version:
want: v4.45.1
want: v4.45.2
method: github-release
with:
repo: mikefarah/yq
+3 -3
View File
@@ -45,7 +45,7 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@28deaeda66b76a05916b6923827895f2b14ab387 #v3.28.16
uses: github/codeql-action/init@60168efe1c415ce0f5521ea06d5c2062adbeed1b #v3.28.17
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -56,7 +56,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@28deaeda66b76a05916b6923827895f2b14ab387 #v3.28.16
uses: github/codeql-action/autobuild@60168efe1c415ce0f5521ea06d5c2062adbeed1b #v3.28.17
# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
@@ -70,4 +70,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@28deaeda66b76a05916b6923827895f2b14ab387 #v3.28.16
uses: github/codeql-action/analyze@60168efe1c415ce0f5521ea06d5c2062adbeed1b #v3.28.17
+1 -1
View File
@@ -148,7 +148,7 @@ jobs:
# for updating brew formula in anchore/homebrew-syft
GITHUB_BREW_TOKEN: ${{ secrets.ANCHOREOPS_GITHUB_OSS_WRITE_TOKEN }}
- uses: anchore/sbom-action@f325610c9f50a54015d37c8d16cb3b0e2c8f4de0 #v0.18.0
- uses: anchore/sbom-action@9f7302141466aa6482940f15371237e9d9f4c34a #v0.19.0
continue-on-error: true
with:
file: go.mod
+2 -1
View File
@@ -29,7 +29,8 @@ jobs:
Unit-Test:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
name: "Unit tests"
runs-on: ubuntu-24.04
# we need more storage than what's on the default runner
runs-on: ubuntu-22.04-4core-16gb
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 #v4.2.2
+2 -2
View File
@@ -140,11 +140,11 @@ Note that flags using the @<version> can be used for earlier versions of each sp
- Linux kernel archives (vmlinz)
- Linux kernel modules (ko)
- Nix (outputs in /nix/store)
- PHP (composer)
- PHP (composer, PECL, Pear)
- Python (wheel, egg, poetry, requirements.txt)
- Red Hat (rpm)
- Ruby (gem)
- Rust (cargo.lock)
- Rust (cargo.lock, auditable binary)
- Swift (cocoapods, swift-package-manager)
- Wordpress plugins
- Terraform providers (.terraform.lock.hcl)
+5
View File
@@ -24,6 +24,7 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/java"
"github.com/anchore/syft/syft/pkg/cataloger/javascript"
"github.com/anchore/syft/syft/pkg/cataloger/kernel"
"github.com/anchore/syft/syft/pkg/cataloger/nix"
"github.com/anchore/syft/syft/pkg/cataloger/python"
"github.com/anchore/syft/syft/source"
)
@@ -48,6 +49,7 @@ type Catalog struct {
Java javaConfig `yaml:"java" json:"java" mapstructure:"java"`
JavaScript javaScriptConfig `yaml:"javascript" json:"javascript" mapstructure:"javascript"`
LinuxKernel linuxKernelConfig `yaml:"linux-kernel" json:"linux-kernel" mapstructure:"linux-kernel"`
Nix nixConfig `yaml:"nix" json:"nix" mapstructure:"nix"`
Python pythonConfig `yaml:"python" json:"python" mapstructure:"python"`
// configuration for the source (the subject being analyzed)
@@ -75,6 +77,7 @@ func DefaultCatalog() Catalog {
Package: defaultPackageConfig(),
License: defaultLicenseConfig(),
LinuxKernel: defaultLinuxKernelConfig(),
Nix: defaultNixConfig(),
Dotnet: defaultDotnetConfig(),
Golang: defaultGolangConfig(),
Java: defaultJavaConfig(),
@@ -193,6 +196,8 @@ func (cfg Catalog) ToPackagesConfig() pkgcataloging.Config {
LinuxKernel: kernel.LinuxKernelCatalogerConfig{
CatalogModules: cfg.LinuxKernel.CatalogModules,
},
Nix: nix.DefaultConfig().
WithCaptureOwnedFiles(cfg.Nix.CaptureOwnedFiles),
Python: python.CatalogerConfig{
GuessUnpinnedRequirements: cfg.Python.GuessUnpinnedRequirements,
},
+25
View File
@@ -0,0 +1,25 @@
package options
import (
"github.com/anchore/clio"
"github.com/anchore/syft/syft/pkg/cataloger/nix"
)
type nixConfig struct {
CaptureOwnedFiles bool `json:"capture-owned-files" yaml:"capture-owned-files" mapstructure:"capture-owned-files"`
}
func defaultNixConfig() nixConfig {
def := nix.DefaultConfig()
return nixConfig{
def.CaptureOwnedFiles,
}
}
var _ interface {
clio.FieldDescriber
} = (*nixConfig)(nil)
func (o *nixConfig) DescribeFields(descriptions clio.FieldDescriptionSet) {
descriptions.Add(&o.CaptureOwnedFiles, `enumerate all files owned by packages found within Nix store paths`)
}
@@ -502,8 +502,8 @@ var commonTestCases = []testCase{
},
},
{
name: "find php pecl package",
pkgType: pkg.PhpPeclPkg,
name: "find php pear/pecl package",
pkgType: pkg.PhpPearPkg,
pkgLanguage: pkg.PHP,
pkgInfo: map[string]string{
"memcached": "3.2.0",
@@ -12,6 +12,7 @@ import (
"github.com/anchore/stereoscope/pkg/imagetest"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
@@ -85,6 +86,7 @@ func TestPkgCoverageImage(t *testing.T) {
definedPkgs.Remove(string(pkg.GithubActionPkg))
definedPkgs.Remove(string(pkg.GithubActionWorkflowPkg))
definedPkgs.Remove(string(pkg.TerraformPkg))
definedPkgs.Remove(string(pkg.PhpPeclPkg)) // we have coverage for pear instead
var cases []testCase
cases = append(cases, commonTestCases...)
@@ -92,40 +94,7 @@ func TestPkgCoverageImage(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
pkgCount := 0
for a := range sbom.Artifacts.Packages.Enumerate(c.pkgType) {
if a.Language.String() != "" {
observedLanguages.Add(a.Language.String())
}
observedPkgs.Add(string(a.Type))
expectedVersion, ok := c.pkgInfo[a.Name]
if !ok {
t.Errorf("unexpected package found: %s", a.Name)
}
if expectedVersion != a.Version {
t.Errorf("unexpected package version (pkg=%s): %s, expected: %s", a.Name, a.Version, expectedVersion)
}
if a.Language != c.pkgLanguage {
t.Errorf("bad language (pkg=%+v): %+v", a.Name, a.Language)
}
if a.Type != c.pkgType {
t.Errorf("bad package type (pkg=%+v): %+v", a.Name, a.Type)
}
pkgCount++
}
if pkgCount != len(c.pkgInfo)+c.duplicates {
t.Logf("Discovered packages of type %+v", c.pkgType)
for a := range sbom.Artifacts.Packages.Enumerate(c.pkgType) {
t.Log(" ", a)
}
t.Fatalf("unexpected package count: %d!=%d", pkgCount, len(c.pkgInfo))
}
assertPackages(t, sbom, c, observedLanguages, observedPkgs)
})
}
@@ -175,44 +144,7 @@ func TestPkgCoverageDirectory(t *testing.T) {
for _, test := range cases {
t.Run(test.name, func(t *testing.T) {
actualPkgCount := 0
for actualPkg := range sbom.Artifacts.Packages.Enumerate(test.pkgType) {
observedLanguages.Add(actualPkg.Language.String())
observedPkgs.Add(string(actualPkg.Type))
expectedVersion, ok := test.pkgInfo[actualPkg.Name]
if !ok {
t.Errorf("unexpected package found: %s", actualPkg.Name)
}
if expectedVersion != actualPkg.Version {
t.Errorf("unexpected package version (pkg=%s): %s", actualPkg.Name, actualPkg.Version)
}
var foundLang bool
for _, lang := range strings.Split(test.pkgLanguage.String(), ",") {
if actualPkg.Language.String() == lang {
foundLang = true
break
}
}
if !foundLang {
t.Errorf("bad language (pkg=%+v): %+v", actualPkg.Name, actualPkg.Language)
}
if actualPkg.Type != test.pkgType {
t.Errorf("bad package type (pkg=%+v): %+v", actualPkg.Name, actualPkg.Type)
}
actualPkgCount++
}
if actualPkgCount != len(test.pkgInfo)+test.duplicates {
for actualPkg := range sbom.Artifacts.Packages.Enumerate(test.pkgType) {
t.Log(" ", actualPkg)
}
t.Fatalf("unexpected package count: %d!=%d", actualPkgCount, len(test.pkgInfo))
}
assertPackages(t, sbom, test, observedLanguages, observedPkgs)
})
}
@@ -227,6 +159,7 @@ func TestPkgCoverageDirectory(t *testing.T) {
definedPkgs.Remove(string(pkg.LinuxKernelModulePkg))
definedPkgs.Remove(string(pkg.Rpkg))
definedPkgs.Remove(string(pkg.UnknownPkg))
definedPkgs.Remove(string(pkg.PhpPeclPkg)) // this is covered as pear packages
// for directory scans we should not expect to see any of the following package types
definedPkgs.Remove(string(pkg.KbPkg))
@@ -241,6 +174,61 @@ func TestPkgCoverageDirectory(t *testing.T) {
}
}
func assertPackages(t *testing.T, sbom sbom.SBOM, test testCase, observedLanguages *strset.Set, observedPkgs *strset.Set) {
actualPkgCount := 0
for actualPkg := range sbom.Artifacts.Packages.Enumerate(test.pkgType) {
observedLanguages.Add(actualPkg.Language.String())
observedPkgs.Add(string(actualPkg.Type))
expectedVersion, ok := test.pkgInfo[actualPkg.Name]
if !ok {
t.Errorf("unexpected package found: %s", actualPkg.Name)
}
if expectedVersion != actualPkg.Version {
t.Errorf("unexpected package version (pkg=%s): %s", actualPkg.Name, actualPkg.Version)
}
var foundLang bool
for _, lang := range strings.Split(test.pkgLanguage.String(), ",") {
if actualPkg.Language.String() == lang {
foundLang = true
break
}
}
if !foundLang {
t.Errorf("bad language (pkg=%+v): %+v", actualPkg.Name, actualPkg.Language)
}
if actualPkg.Type != test.pkgType {
t.Errorf("bad package type (pkg=%+v): %+v", actualPkg.Name, actualPkg.Type)
}
actualPkgCount++
// all packages should have at least one location associated with it, and of those locations at least one should be primary evidence
locs := actualPkg.Locations.ToSlice()
assert.NotEmpty(t, locs, "package %q has no locations (type=%q)", actualPkg.Name, actualPkg.Type)
var primaryEvidenceFound bool
for _, l := range locs {
if _, exists := l.Annotations[pkg.EvidenceAnnotationKey]; !exists {
t.Errorf("missing evidence annotation (pkg=%s type=%s)", actualPkg.Name, actualPkg.Type)
}
if l.Annotations[pkg.EvidenceAnnotationKey] == pkg.PrimaryEvidenceAnnotation {
primaryEvidenceFound = true
}
}
assert.True(t, primaryEvidenceFound, "no primary evidence found for package %q", actualPkg.Name)
}
if actualPkgCount != len(test.pkgInfo)+test.duplicates {
for actualPkg := range sbom.Artifacts.Packages.Enumerate(test.pkgType) {
t.Log(" ", actualPkg)
}
t.Fatalf("unexpected package count: %d!=%d", actualPkgCount, len(test.pkgInfo))
}
}
func TestPkgCoverageImage_HasEvidence(t *testing.T) {
sbom, _ := catalogFixtureImage(t, "image-pkg-coverage", source.SquashedScope)
@@ -56,6 +56,8 @@ func TestAllPackageCatalogersReachableInTasks(t *testing.T) {
// not reachable since they are deprecated
"dotnet-portable-executable-cataloger",
"dotnet-deps-cataloger",
"nix-store-cataloger",
"php-pecl-serialized-cataloger",
// not reachable by design
"sbom-cataloger",
)
@@ -1 +1 @@
a:5:{s:4:"name";s:9:"memcached";s:4:"date";s:10:"2022-01-11";s:4:"time";s:8:"15:23:47";s:7:"version";a:2:{s:7:"release";s:5:"3.2.0";s:3:"api";s:5:"3.2.0";}s:7:"license";a:2:{s:7:"attribs";a:1:{s:3:"uri";s:26:"http://www.php.net/license";}s:8:"_content";s:11:"PHP License";}}
a:6:{s:4:"name";s:9:"memcached";s:7:"channel";s:12:"pecl.php.net";s:4:"date";s:10:"2022-01-11";s:4:"time";s:8:"15:23:47";s:7:"version";a:2:{s:7:"release";s:5:"3.2.0";s:3:"api";s:5:"3.2.0";}s:7:"license";a:2:{s:7:"attribs";a:1:{s:3:"uri";s:26:"http://www.php.net/license";}s:8:"_content";s:11:"PHP License";}}
+12 -14
View File
@@ -5,7 +5,7 @@ go 1.24.1
require (
github.com/BurntSushi/toml v1.5.0
github.com/CycloneDX/cyclonedx-go v0.9.2
github.com/Masterminds/semver v1.5.0
github.com/Masterminds/semver/v3 v3.3.1
github.com/Masterminds/sprig/v3 v3.3.0
github.com/OneOfOne/xxhash v1.2.8
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
@@ -19,6 +19,7 @@ require (
github.com/anchore/go-homedir v0.0.0-20250319154043-c29668562e4d
github.com/anchore/go-logger v0.0.0-20250318195838-07ae343dd722
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb
github.com/anchore/go-sync v0.0.0-20250326131806-4eda43a485b6
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.1.1-0.20250220190351-d62adb6e1115
@@ -29,7 +30,7 @@ require (
github.com/blakesmith/ar v0.0.0-20190502131153-809d4375e1fb
github.com/bmatcuk/doublestar/v4 v4.8.1
github.com/charmbracelet/bubbles v0.21.0
github.com/charmbracelet/bubbletea v1.3.4
github.com/charmbracelet/bubbletea v1.3.5
github.com/charmbracelet/lipgloss v1.1.0
github.com/dave/jennifer v1.7.1
github.com/deitch/magic v0.0.0-20230404182410-1ff89d7342da
@@ -37,7 +38,7 @@ require (
github.com/dustin/go-humanize v1.0.1
github.com/elliotchance/phpserialize v1.4.0
github.com/facebookincubator/nvdtools v0.1.5
github.com/github/go-spdx/v2 v2.3.2
github.com/github/go-spdx/v2 v2.3.3
github.com/gkampitakis/go-snaps v0.5.11
github.com/go-git/go-billy/v5 v5.6.2
github.com/go-git/go-git/v5 v5.16.0
@@ -60,6 +61,7 @@ require (
github.com/magiconair/properties v1.8.10
github.com/mholt/archives v0.1.1
github.com/moby/sys/mountinfo v0.7.2
github.com/nix-community/go-nix v0.0.0-20250101154619-4bdde671e0a1
github.com/olekukonko/tablewriter v0.0.5
github.com/opencontainers/go-digest v1.0.0
github.com/pelletier/go-toml v1.9.5
@@ -85,7 +87,7 @@ require (
go.uber.org/goleak v1.3.0
golang.org/x/exp v0.0.0-20250305212735-054e65f0b394
golang.org/x/mod v0.24.0
golang.org/x/net v0.39.0
golang.org/x/net v0.40.0
gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v1.37.0
)
@@ -96,7 +98,6 @@ require (
github.com/AdamKorcz/go-118-fuzz-build v0.0.0-20230306123547-8075edf89bb0 // indirect
github.com/DataDog/zstd v1.5.5 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.3.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/Microsoft/hcsshim v0.11.7 // indirect
github.com/ProtonMail/go-crypto v1.1.6 // indirect
@@ -176,6 +177,7 @@ require (
github.com/mattn/go-localereader v0.0.2-0.20220822084749-2491eb6c1c75 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/minio/minlz v1.0.0 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
@@ -240,11 +242,11 @@ require (
go.uber.org/atomic v1.9.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
go4.org v0.0.0-20230225012048-214862532bf5 // indirect
golang.org/x/crypto v0.37.0 // indirect
golang.org/x/sync v0.13.0 // indirect
golang.org/x/sys v0.32.0 // indirect
golang.org/x/term v0.31.0 // indirect
golang.org/x/text v0.24.0 // indirect
golang.org/x/crypto v0.38.0 // indirect
golang.org/x/sync v0.14.0 // indirect
golang.org/x/sys v0.33.0 // indirect
golang.org/x/term v0.32.0 // indirect
golang.org/x/text v0.25.0 // indirect
golang.org/x/tools v0.31.0 // indirect
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
google.golang.org/genproto v0.0.0-20241118233622-e639e219e697 // indirect
@@ -257,10 +259,6 @@ require (
modernc.org/memory v1.9.1 // indirect
)
require github.com/anchore/go-sync v0.0.0-20250326131806-4eda43a485b6
require github.com/minio/minlz v1.0.0 // indirect
retract (
v0.53.2
v0.53.1 // Published accidentally with incorrect license in depdencies
+22 -20
View File
@@ -66,10 +66,8 @@ github.com/DataDog/zstd v1.5.5 h1:oWf5W7GtOLgp6bciQYDmhHHjdhYkALu6S/5Ni9ZgSvQ=
github.com/DataDog/zstd v1.5.5/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw=
github.com/Masterminds/goutils v1.1.1 h1:5nUrii3FMTL5diU80unEVvNevw1nH4+ZV4DSLVJLSYI=
github.com/Masterminds/goutils v1.1.1/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU=
github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
github.com/Masterminds/semver v1.5.0/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF078ddwwvV3Y=
github.com/Masterminds/semver/v3 v3.3.0 h1:B8LGeaivUe71a5qox1ICM/JLl0NqZSW5CHyL+hmvYS0=
github.com/Masterminds/semver/v3 v3.3.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/Masterminds/semver/v3 v3.3.1 h1:QtNSWtVZ3nBfk8mAOu/B6v7FMJ+NHTIgUPi7rj+4nv4=
github.com/Masterminds/semver/v3 v3.3.1/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/Masterminds/sprig/v3 v3.3.0 h1:mQh0Yrg1XPo6vjYXgtf5OtijNAKJRNcTdOOGZe3tPhs=
github.com/Masterminds/sprig/v3 v3.3.0/go.mod h1:Zy1iXRYNqNLUolqCpL4uhk6SHUMAOSCzdgBfDb35Lz0=
github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
@@ -183,8 +181,8 @@ github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UF
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/charmbracelet/bubbles v0.21.0 h1:9TdC97SdRVg/1aaXNVWfFH3nnLAwOXr8Fn6u6mfQdFs=
github.com/charmbracelet/bubbles v0.21.0/go.mod h1:HF+v6QUR4HkEpz62dx7ym2xc71/KBHg+zKwJtMw+qtg=
github.com/charmbracelet/bubbletea v1.3.4 h1:kCg7B+jSCFPLYRA52SDZjr51kG/fMUEoPoZrkaDHyoI=
github.com/charmbracelet/bubbletea v1.3.4/go.mod h1:dtcUCyCGEX3g9tosuYiut3MXgY/Jsv9nKVdibKKRRXo=
github.com/charmbracelet/bubbletea v1.3.5 h1:JAMNLTbqMOhSwoELIr0qyP4VidFq72/6E9j7HHmRKQc=
github.com/charmbracelet/bubbletea v1.3.5/go.mod h1:TkCnmH+aBd4LrXhXcqrKiYwRs7qyQx5rBgH5fVY3v54=
github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc h1:4pZI35227imm7yK2bGPcfpFEmuY1gc2YSTShr4iJBfs=
github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc/go.mod h1:X4/0JoqgTIPSFcRA/P6INZzIuyqdFY5rm8tb41s9okk=
github.com/charmbracelet/harmonica v0.2.0 h1:8NxJWRWg/bzKqqEaaeFNipOu77YR5t8aSwG4pgaUBiQ=
@@ -323,8 +321,8 @@ github.com/fsnotify/fsnotify v1.8.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8
github.com/gabriel-vasile/mimetype v1.4.9 h1:5k+WDwEsD9eTLL8Tz3L0VnmVh9QxGjRmjBvAG7U/oYY=
github.com/gabriel-vasile/mimetype v1.4.9/go.mod h1:WnSQhFKJuBlRyLiKohA/2DtIlPFAbguNaG7QCHcyGok=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/github/go-spdx/v2 v2.3.2 h1:IfdyNHTqzs4zAJjXdVQfRnxt1XMfycXoHBE2Vsm1bjs=
github.com/github/go-spdx/v2 v2.3.2/go.mod h1:2ZxKsOhvBp+OYBDlsGnUMcchLeo2mrpEBn2L1C+U3IQ=
github.com/github/go-spdx/v2 v2.3.3 h1:QI7evnHWEfWkT54eJwkoV/f3a0xD3gLlnVmT5wQG6LE=
github.com/github/go-spdx/v2 v2.3.3/go.mod h1:2ZxKsOhvBp+OYBDlsGnUMcchLeo2mrpEBn2L1C+U3IQ=
github.com/gkampitakis/ciinfo v0.3.1 h1:lzjbemlGI4Q+XimPg64ss89x8Mf3xihJqy/0Mgagapo=
github.com/gkampitakis/ciinfo v0.3.1/go.mod h1:1NIwaOcFChN4fa/B0hEBdAb6npDlFL8Bwx4dfRLRqAo=
github.com/gkampitakis/go-diff v1.3.2 h1:Qyn0J9XJSDTgnsgHRdz9Zp24RaJeKMUHg2+PDZZdC4M=
@@ -656,6 +654,10 @@ github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/nix-community/go-nix v0.0.0-20250101154619-4bdde671e0a1 h1:kpt9ZfKcm+EDG4s40hMwE//d5SBgDjUOrITReV2u4aA=
github.com/nix-community/go-nix v0.0.0-20250101154619-4bdde671e0a1/go.mod h1:qgCw4bBKZX8qMgGeEZzGFVT3notl42dBjNqO2jut0M0=
github.com/nsf/jsondiff v0.0.0-20210926074059-1e845ec5d249 h1:NHrXEjTNQY7P0Zfx1aMrNhpgxHmow66XQtm0aQLY0AE=
github.com/nsf/jsondiff v0.0.0-20210926074059-1e845ec5d249/go.mod h1:mpRZBD8SJ55OIICQ3iWH0Yz3cjzA61JdqMLoWXeB2+8=
github.com/nwaples/rardecode v1.1.3 h1:cWCaZwfM5H7nAD6PyEdcVnczzV8i/JtotnyW/dD9lEc=
github.com/nwaples/rardecode v1.1.3/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0=
github.com/nwaples/rardecode/v2 v2.1.0 h1:JQl9ZoBPDy+nIZGb1mx8+anfHp/LV3NE2MjMiv0ct/U=
@@ -923,8 +925,8 @@ golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.37.0 h1:kJNSjF/Xp7kU0iB2Z+9viTPMW4EqqsrywMXLJOOsXSE=
golang.org/x/crypto v0.37.0/go.mod h1:vg+k43peMZ0pUMhYmVAWysMK35e6ioLh3wB8ZCAfbVc=
golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8=
golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -1010,8 +1012,8 @@ golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qx
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY=
golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E=
golang.org/x/net v0.40.0 h1:79Xs7wF06Gbdcg4kdCCIQArK11Z1hr5POQ6+fIYHNuY=
golang.org/x/net v0.40.0/go.mod h1:y0hY0exeL2Pku80/zKK7tpntoX23cqL3Oa6njdgRtds=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -1041,8 +1043,8 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.13.0 h1:AauUjRAJ9OSnvULf/ARrrVywoJDy0YS2AwQ98I37610=
golang.org/x/sync v0.13.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ=
golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -1118,13 +1120,13 @@ golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.32.0 h1:s77OFDvIQeibCmezSnk/q6iAfkdiQaJi4VzroCFrN20=
golang.org/x/sys v0.32.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.31.0 h1:erwDkOK1Msy6offm1mOgvspSkslFnIGsFnxOKoufg3o=
golang.org/x/term v0.31.0/go.mod h1:R4BeIy7D95HzImkxGkTW1UQTtP54tio2RyHz7PwK0aw=
golang.org/x/term v0.32.0 h1:DR4lr0TjUs3epypdhTOkMmuF5CDFJ/8pOnbzMZPQ7bg=
golang.org/x/term v0.32.0/go.mod h1:uZG1FhGx848Sqfsq4/DlJr3xGGsYMu/L5GW4abiaEPQ=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -1135,8 +1137,8 @@ golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0=
golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU=
golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4=
golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+1 -1
View File
@@ -3,5 +3,5 @@ package internal
const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "16.0.25"
JSONSchemaVersion = "16.0.28"
)
@@ -0,0 +1,5 @@
Knuth CTAN License
Full name
Knuth CTAN License
This software is copyrighted. Unlimited copying and redistribution of this package and/or its individual files are permitted as long as there are no modifications. Modifications, and redistribution of modifications, are also permitted, but only if the resulting package and/or files are renamed.
+45 -21
View File
@@ -9,6 +9,7 @@ import (
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/sbomsync"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
)
@@ -32,11 +33,11 @@ type ownershipByFilesMetadata struct {
Files []string `json:"files"`
}
func ByFileOwnershipOverlapWorker(accessor sbomsync.Accessor) {
func ByFileOwnershipOverlapWorker(resolver file.Resolver, accessor sbomsync.Accessor) {
var relationships []artifact.Relationship
accessor.ReadFromSBOM(func(s *sbom.SBOM) {
relationships = byFileOwnershipOverlap(s.Artifacts.Packages)
relationships = byFileOwnershipOverlap(resolver, s.Artifacts.Packages)
})
accessor.WriteToSBOM(func(s *sbom.SBOM) {
@@ -46,8 +47,8 @@ func ByFileOwnershipOverlapWorker(accessor sbomsync.Accessor) {
// byFileOwnershipOverlap creates a package-to-package relationship based on discovering which packages have
// evidence locations that overlap with ownership claim from another package's package manager metadata.
func byFileOwnershipOverlap(catalog *pkg.Collection) []artifact.Relationship {
var relationships = findOwnershipByFilesRelationships(catalog)
func byFileOwnershipOverlap(resolver file.Resolver, catalog *pkg.Collection) []artifact.Relationship {
var relationships = findOwnershipByFilesRelationships(resolver, catalog)
var edges []artifact.Relationship
for parentID, children := range relationships {
@@ -84,7 +85,7 @@ func byFileOwnershipOverlap(catalog *pkg.Collection) []artifact.Relationship {
// findOwnershipByFilesRelationships find overlaps in file ownership with a file that defines another package. Specifically, a .Location.Path of
// a package is found to be owned by another (from the owner's .Metadata.Files[]).
func findOwnershipByFilesRelationships(catalog *pkg.Collection) map[artifact.ID]map[artifact.ID]*strset.Set {
func findOwnershipByFilesRelationships(resolver file.Resolver, catalog *pkg.Collection) map[artifact.ID]map[artifact.ID]*strset.Set { //nolint:gocognit
var relationships = make(map[artifact.ID]map[artifact.ID]*strset.Set)
if catalog == nil {
@@ -102,27 +103,31 @@ func findOwnershipByFilesRelationships(catalog *pkg.Collection) map[artifact.ID]
if !ok {
continue
}
for _, ownedFilePath := range pkgFileOwner.OwnedFiles() {
if matchesAny(ownedFilePath, globsForbiddenFromBeingOwned) {
// we skip over known exceptions to file ownership, such as the RPM package owning
// the RPM DB path, otherwise the RPM package would "own" all RPMs, which is not intended
continue
}
// look for package(s) in the catalog that may be owned by this package and mark the relationship
for _, subPackage := range catalog.PackagesByPath(ownedFilePath) {
subID := subPackage.ID()
if subID == id {
for _, ownedFilePath := range pkgFileOwner.OwnedFiles() {
// find the first path that results in a hit
for _, ownedPath := range allPaths(ownedFilePath, resolver) {
if matchesAny(ownedPath, globsForbiddenFromBeingOwned) {
// we skip over known exceptions to file ownership, such as the RPM package owning
// the RPM DB path, otherwise the RPM package would "own" all RPMs, which is not intended
continue
}
if _, exists := relationships[id]; !exists {
relationships[id] = make(map[artifact.ID]*strset.Set)
}
if _, exists := relationships[id][subID]; !exists {
relationships[id][subID] = strset.New()
// look for package(s) in the catalog that may be owned by this package and mark the relationship
for _, subPackage := range catalog.PackagesByPath(ownedPath) {
subID := subPackage.ID()
if subID == id {
continue
}
if _, exists := relationships[id]; !exists {
relationships[id] = make(map[artifact.ID]*strset.Set)
}
if _, exists := relationships[id][subID]; !exists {
relationships[id][subID] = strset.New()
}
relationships[id][subID].Add(ownedPath)
}
relationships[id][subID].Add(ownedFilePath)
}
}
}
@@ -130,6 +135,25 @@ func findOwnershipByFilesRelationships(catalog *pkg.Collection) map[artifact.ID]
return relationships
}
func allPaths(ownedFilePath string, resolver file.Resolver) []string {
// though we have a string path, we need to resolve symlinks and other filesystem oddities since we cannot assume this is a real path
var locs []file.Location
var err error
if resolver != nil {
locs, err = resolver.FilesByPath(ownedFilePath)
if err != nil {
log.WithFields("error", err, "path", ownedFilePath).Trace("unable to find path for owned file")
locs = nil
}
}
ownedFilePaths := strset.New(ownedFilePath)
for _, loc := range locs {
ownedFilePaths.Add(loc.RealPath)
}
return ownedFilePaths.List()
}
func matchesAny(s string, globs []string) bool {
for _, g := range globs {
matches, err := doublestar.Match(g, s)
@@ -4,7 +4,7 @@ import (
"testing"
"github.com/google/go-cmp/cmp"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal/cmptest"
"github.com/anchore/syft/syft/artifact"
@@ -12,11 +12,29 @@ import (
"github.com/anchore/syft/syft/pkg"
)
type mockFR struct {
file.Resolver
translate map[string]string
}
func (m mockFR) FilesByPath(paths ...string) ([]file.Location, error) {
var results []file.Location
for _, p := range paths {
tPath, ok := m.translate[p]
if !ok {
tPath = p
}
results = append(results, file.NewLocation(tPath))
}
return results, nil
}
func TestOwnershipByFilesRelationship(t *testing.T) {
tests := []struct {
name string
setup func(t testing.TB) ([]pkg.Package, []artifact.Relationship)
name string
resolver file.Resolver
setup func(t testing.TB) ([]pkg.Package, []artifact.Relationship)
}{
{
name: "owns-by-real-path",
@@ -60,6 +78,75 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
return []pkg.Package{parent, child}, []artifact.Relationship{relationship}
},
},
{
name: "misses-by-dead-symlink",
resolver: mockFR{
translate: map[string]string{
"/bin/gzip": "", // treat this as a dead symlink
},
},
setup: func(t testing.TB) ([]pkg.Package, []artifact.Relationship) {
parent := pkg.Package{
Type: pkg.DebPkg,
Metadata: pkg.DpkgDBEntry{
Files: []pkg.DpkgFileRecord{
{Path: "/bin/gzip"}, // this symlinks to gzip via /bin -> /usr/bin
},
},
}
parent.SetID()
child := pkg.Package{
Locations: file.NewLocationSet(
file.NewVirtualLocation("/usr/bin/gzip", "/usr/bin/gzip"),
),
Type: pkg.BinaryPkg,
}
child.SetID()
return []pkg.Package{parent, child}, nil // importantly, no relationship is expected
},
},
{
name: "owns-by-symlink",
resolver: mockFR{
translate: map[string]string{
"/bin/gzip": "/usr/bin/gzip", // if there is a string path of /bin/gzip then return the real path of /usr/bin/gzip
},
},
setup: func(t testing.TB) ([]pkg.Package, []artifact.Relationship) {
parent := pkg.Package{
Type: pkg.DebPkg,
Metadata: pkg.DpkgDBEntry{
Files: []pkg.DpkgFileRecord{
{Path: "/bin/gzip"}, // this symlinks to gzip via /bin -> /usr/bin
},
},
}
parent.SetID()
child := pkg.Package{
Locations: file.NewLocationSet(
file.NewVirtualLocation("/usr/bin/gzip", "/usr/bin/gzip"),
),
Type: pkg.BinaryPkg,
}
child.SetID()
relationship := artifact.Relationship{
From: parent,
To: child,
Type: artifact.OwnershipByFileOverlapRelationship,
Data: ownershipByFilesMetadata{
Files: []string{
"/usr/bin/gzip",
},
},
}
return []pkg.Package{parent, child}, []artifact.Relationship{relationship}
},
},
{
name: "owns-by-virtual-path",
setup: func(t testing.TB) ([]pkg.Package, []artifact.Relationship) {
@@ -140,9 +227,9 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
t.Run(test.name, func(t *testing.T) {
pkgs, expectedRelations := test.setup(t)
c := pkg.NewCollection(pkgs...)
relationships := byFileOwnershipOverlap(c)
relationships := byFileOwnershipOverlap(test.resolver, c)
assert.Len(t, relationships, len(expectedRelations))
require.Len(t, relationships, len(expectedRelations))
for idx, expectedRelationship := range expectedRelations {
actualRelationship := relationships[idx]
if d := cmp.Diff(expectedRelationship, actualRelationship, cmptest.DefaultOptions()...); d != "" {
+10 -2
View File
@@ -100,7 +100,7 @@ func DefaultPackageTaskFactories() Factories {
pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, JavaScript, Node, NPM,
),
newSimplePackageTaskFactory(php.NewComposerLockCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "php", "composer"),
newSimplePackageTaskFactory(php.NewPeclCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, pkgcataloging.ImageTag, "php", "pecl"),
newSimplePackageTaskFactory(php.NewPearCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, pkgcataloging.ImageTag, "php", "pear"),
newPackageTaskFactory(
func(cfg CatalogingFactoryConfig) pkg.Cataloger {
return python.NewPackageCataloger(cfg.PackagesConfig.Python)
@@ -137,7 +137,12 @@ func DefaultPackageTaskFactories() Factories {
pkgcataloging.DirectoryTag, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.LanguageTag, Java, Maven,
),
newSimplePackageTaskFactory(java.NewNativeImageCataloger, pkgcataloging.DirectoryTag, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.LanguageTag, Java),
newSimplePackageTaskFactory(nix.NewStoreCataloger, pkgcataloging.DirectoryTag, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.LanguageTag, "nix"),
newPackageTaskFactory(
func(cfg CatalogingFactoryConfig) pkg.Cataloger {
return nix.NewCataloger(cfg.PackagesConfig.Nix)
},
pkgcataloging.DirectoryTag, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.LanguageTag, "nix",
),
newSimplePackageTaskFactory(lua.NewPackageCataloger, pkgcataloging.DirectoryTag, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.LanguageTag, "lua"),
// other package catalogers ///////////////////////////////////////////////////////////////////////////
@@ -166,5 +171,8 @@ func DefaultPackageTaskFactories() Factories {
// these are catalogers that should not be selectable other than specific inclusion via name or "deprecated" tag (to remain backwards compatible)
newSimplePackageTaskFactory(dotnet.NewDotnetDepsCataloger, pkgcataloging.DeprecatedTag), // TODO: remove in syft v2.0
newSimplePackageTaskFactory(dotnet.NewDotnetPortableExecutableCataloger, pkgcataloging.DeprecatedTag), // TODO: remove in syft v2.0
newSimplePackageTaskFactory(php.NewPeclCataloger, pkgcataloging.DeprecatedTag), // TODO: remove in syft v2.0
newSimplePackageTaskFactory(nix.NewStoreCataloger, pkgcataloging.DeprecatedTag), // TODO: remove in syft v2.0
}
}
+1 -1
View File
@@ -46,7 +46,7 @@ func finalizeRelationships(resolver file.Resolver, builder sbomsync.Builder, cfg
// add relationships showing packages that are evident by a file which is owned by another package (package-to-package)
if cfg.PackageFileOwnershipOverlap {
relationship.ByFileOwnershipOverlapWorker(accessor)
relationship.ByFileOwnershipOverlapWorker(resolver, accessor)
}
// conditionally remove binary packages based on file ownership overlap relationships found
+63
View File
@@ -0,0 +1,63 @@
package task
import (
"context"
"fmt"
"github.com/anchore/syft/internal/sbomsync"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
)
func NewDeepSquashedScopeCleanupTask() Task {
fn := func(_ context.Context, _ file.Resolver, builder sbomsync.Builder) error {
accessor := builder.(sbomsync.Accessor)
// remove all packages that doesn't exist in the final state of the image
builder.DeletePackages(packagesToRemove(accessor)...)
return nil
}
return NewTask("deep-squashed-cleaner", fn)
}
func packagesToRemove(accessor sbomsync.Accessor) []artifact.ID {
pkgsToDelete := make([]artifact.ID, 0)
accessor.ReadFromSBOM(func(s *sbom.SBOM) {
filterDuplicates := make(map[string]bool)
for p := range s.Artifacts.Packages.Enumerate() {
noSquashed := true
noPrimary := true
for _, l := range p.Locations.ToSlice() {
isPrimaryEvidence := l.Annotations[pkg.EvidenceAnnotationKey] == pkg.PrimaryEvidenceAnnotation
switch l.Annotations[file.VisibleAnnotationKey] {
case file.VisibleAnnotation:
if isPrimaryEvidence || p.Type == pkg.BinaryPkg {
noSquashed = false
break
}
case "":
if isPrimaryEvidence {
if exists := filterDuplicates[getKey(p, l)]; exists {
break
}
filterDuplicates[getKey(p, l)] = true
noPrimary = false
break
}
}
}
if noSquashed && noPrimary {
pkgsToDelete = append(pkgsToDelete, p.ID())
}
}
})
return pkgsToDelete
}
func getKey(pkg pkg.Package, loc file.Location) string {
return fmt.Sprintf("%s-%s-%s-%s", pkg.Name, pkg.Version, loc.RealPath, loc.AccessPath)
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+80 -4
View File
@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "anchore.io/schema/syft/json/16.0.25/document",
"$id": "anchore.io/schema/syft/json/16.0.28/document",
"$ref": "#/$defs/Document",
"$defs": {
"AlpmDbEntry": {
@@ -1391,6 +1391,9 @@
"value": {
"type": "string"
},
"fullText": {
"type": "string"
},
"spdxExpression": {
"type": "string"
},
@@ -1416,6 +1419,7 @@
"type": "object",
"required": [
"value",
"fullText",
"spdxExpression",
"type",
"urls",
@@ -1656,14 +1660,57 @@
"kb"
]
},
"NixDerivation": {
"properties": {
"path": {
"type": "string"
},
"system": {
"type": "string"
},
"inputDerivations": {
"items": {
"$ref": "#/$defs/NixDerivationReference"
},
"type": "array"
},
"inputSources": {
"items": {
"type": "string"
},
"type": "array"
}
},
"type": "object"
},
"NixDerivationReference": {
"properties": {
"path": {
"type": "string"
},
"outputs": {
"items": {
"type": "string"
},
"type": "array"
}
},
"type": "object"
},
"NixStoreEntry": {
"properties": {
"outputHash": {
"path": {
"type": "string"
},
"output": {
"type": "string"
},
"outputHash": {
"type": "string"
},
"derivation": {
"$ref": "#/$defs/NixDerivation"
},
"files": {
"items": {
"type": "string"
@@ -1673,8 +1720,7 @@
},
"type": "object",
"required": [
"outputHash",
"files"
"outputHash"
]
},
"OpamPackage": {
@@ -1872,6 +1918,9 @@
{
"$ref": "#/$defs/PhpComposerLockEntry"
},
{
"$ref": "#/$defs/PhpPearEntry"
},
{
"$ref": "#/$defs/PhpPeclEntry"
},
@@ -2164,11 +2213,38 @@
"dist"
]
},
"PhpPearEntry": {
"properties": {
"name": {
"type": "string"
},
"channel": {
"type": "string"
},
"version": {
"type": "string"
},
"license": {
"items": {
"type": "string"
},
"type": "array"
}
},
"type": "object",
"required": [
"name",
"version"
]
},
"PhpPeclEntry": {
"properties": {
"name": {
"type": "string"
},
"channel": {
"type": "string"
},
"version": {
"type": "string"
},
+10 -2
View File
@@ -7,6 +7,7 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/java"
"github.com/anchore/syft/syft/pkg/cataloger/javascript"
"github.com/anchore/syft/syft/pkg/cataloger/kernel"
"github.com/anchore/syft/syft/pkg/cataloger/nix"
"github.com/anchore/syft/syft/pkg/cataloger/python"
)
@@ -17,6 +18,7 @@ type Config struct {
JavaArchive java.ArchiveCatalogerConfig `yaml:"java-archive" json:"java-archive" mapstructure:"java-archive"`
JavaScript javascript.CatalogerConfig `yaml:"javascript" json:"javascript" mapstructure:"javascript"`
LinuxKernel kernel.LinuxKernelCatalogerConfig `yaml:"linux-kernel" json:"linux-kernel" mapstructure:"linux-kernel"`
Nix nix.Config `yaml:"nix" json:"nix" mapstructure:"nix"`
Python python.CatalogerConfig `yaml:"python" json:"python" mapstructure:"python"`
}
@@ -25,12 +27,18 @@ func DefaultConfig() Config {
Binary: binary.DefaultClassifierCatalogerConfig(),
Dotnet: dotnet.DefaultCatalogerConfig(),
Golang: golang.DefaultCatalogerConfig(),
LinuxKernel: kernel.DefaultLinuxKernelCatalogerConfig(),
Python: python.DefaultCatalogerConfig(),
JavaArchive: java.DefaultArchiveCatalogerConfig(),
LinuxKernel: kernel.DefaultLinuxKernelCatalogerConfig(),
Nix: nix.DefaultConfig(),
Python: python.DefaultCatalogerConfig(),
}
}
func (c Config) WithNixConfig(cfg nix.Config) Config {
c.Nix = cfg
return c
}
func (c Config) WithBinaryConfig(cfg binary.ClassifierCatalogerConfig) Config {
c.Binary = cfg
return c
+17
View File
@@ -188,6 +188,7 @@ func (c *CreateSBOMConfig) makeTaskGroups(src source.Description) ([][]task.Task
// generate package and file tasks based on the configuration
environmentTasks := c.environmentTasks()
scopeTasks := c.scopeTasks()
relationshipsTasks := c.relationshipTasks(src)
unknownTasks := c.unknownsTasks()
@@ -204,6 +205,11 @@ func (c *CreateSBOMConfig) makeTaskGroups(src source.Description) ([][]task.Task
taskGroups = append(taskGroups, append(pkgTasks, fileTasks...))
}
// all scope work must be done after all nodes (files and packages) have been cataloged and before the relationship
if len(scopeTasks) > 0 {
taskGroups = append(taskGroups, scopeTasks)
}
// all relationship work must be done after all nodes (files and packages) have been cataloged
if len(relationshipsTasks) > 0 {
taskGroups = append(taskGroups, relationshipsTasks)
@@ -391,6 +397,17 @@ func (c *CreateSBOMConfig) userPackageTasks(cfg task.CatalogingFactoryConfig) ([
return persistentPackageTasks, selectablePackageTasks, nil
}
// scopeTasks returns the set of tasks that should be run to generate additional scope information
func (c *CreateSBOMConfig) scopeTasks() []task.Task {
var tsks []task.Task
if c.Search.Scope == source.DeepSquashedScope {
if t := task.NewDeepSquashedScopeCleanupTask(); t != nil {
tsks = append(tsks, t)
}
}
return tsks
}
// relationshipTasks returns the set of tasks that should be run to generate additional relationships as well as
// prune existing relationships.
func (c *CreateSBOMConfig) relationshipTasks(src source.Description) []task.Task {
+14
View File
@@ -9,6 +9,17 @@ import (
"github.com/anchore/stereoscope/pkg/image"
)
const (
// VisibleAnnotationKey is the key used to indicate if the location is visible or not at runtime
VisibleAnnotationKey = "visible"
// HiddenAnnotation is the value used to indicate that the location is not visible at runtime because it was deleted
HiddenAnnotation = "false"
// VisibleAnnotation is the value used to indicate that the location is visible at runtime
VisibleAnnotation = "true"
)
// Location represents a path relative to a particular filesystem resolved to a specific file.Reference. This struct is used as a key
// in content fetching to uniquely identify a file relative to a request (the AccessPath).
type Location struct {
@@ -48,6 +59,9 @@ func (m *LocationMetadata) merge(other LocationMetadata) error {
}
func (l Location) WithAnnotation(key, value string) Location {
if key == "" || value == "" {
return l
}
if l.Annotations == nil {
l.Annotations = map[string]string{}
}
@@ -759,13 +759,18 @@ func toOtherLicenses(catalog *pkg.Collection) []*spdx.OtherLicense {
for _, id := range ids {
license := licenses[id]
value := license.Value
fullText := license.FullText
// handle cases where LicenseRef needs to be included in hasExtractedLicensingInfos
if license.Value == "" {
value, _ = strings.CutPrefix(license.ID, "LicenseRef-")
}
other := &spdx.OtherLicense{
LicenseIdentifier: license.ID,
ExtractedText: value,
}
if fullText != "" {
other.ExtractedText = fullText
} else {
other.ExtractedText = value
}
customPrefix := spdxlicense.LicenseRefPrefix + helpers.SanitizeElementID(internallicenses.UnknownLicensePrefix)
if strings.HasPrefix(license.ID, customPrefix) {
@@ -499,14 +499,15 @@ func extractPkgInfo(p *spdx.Package) pkgInfo {
func toSyftPackage(p *spdx.Package) pkg.Package {
info := extractPkgInfo(p)
sP := &pkg.Package{
Type: info.typ,
Name: p.PackageName,
Version: p.PackageVersion,
Licenses: pkg.NewLicenseSet(parseSPDXLicenses(p)...),
CPEs: extractCPEs(p),
PURL: purlValue(info.purl),
Language: info.lang,
Metadata: extractMetadata(p, info),
Type: info.typ,
Name: p.PackageName,
Version: p.PackageVersion,
Licenses: pkg.NewLicenseSet(parseSPDXLicenses(p)...),
CPEs: extractCPEs(p),
PURL: purlValue(info.purl),
Language: info.lang,
Metadata: extractMetadata(p, info),
SupplementalData: []any{artifact.ID(p.PackageSPDXIdentifier)},
}
sP.SetID()
@@ -589,12 +589,18 @@ func Test_convertToAndFromFormat(t *testing.T) {
got, err := ToSyftModel(doc)
require.NoError(t, err)
for _, p := range got.Artifacts.Packages.Sorted() {
// all decoders should be setting an ID of sorts here (another test will verify the correctness of the value)
assert.NotEmpty(t, p.SupplementalData)
}
if diff := cmp.Diff(&s, got,
cmpopts.IgnoreUnexported(artifact.Relationship{}),
cmpopts.IgnoreUnexported(file.LocationSet{}),
cmpopts.IgnoreUnexported(pkg.Collection{}),
cmpopts.IgnoreUnexported(pkg.Package{}),
cmpopts.IgnoreUnexported(pkg.LicenseSet{}),
cmpopts.IgnoreFields(pkg.Package{}, "SupplementalData"), // this is used by decoders to store additional data from the original format
cmpopts.IgnoreFields(sbom.Artifacts{}, "FileMetadata", "FileDigests"),
); diff != "" {
t.Fatalf("packages do not match:\n%s", diff)
@@ -664,7 +670,7 @@ func Test_directPackageFiles(t *testing.T) {
Packages: []*spdx.Package{
{
PackageName: "some-package",
PackageSPDXIdentifier: "1",
PackageSPDXIdentifier: "1", // important!
PackageVersion: "1.0.5",
Files: []*spdx.File{
{
@@ -686,10 +692,11 @@ func Test_directPackageFiles(t *testing.T) {
require.NoError(t, err)
p := pkg.Package{
Name: "some-package",
Version: "1.0.5",
Name: "some-package",
Version: "1.0.5",
SupplementalData: []any{artifact.ID("1")}, // set by the decoders from the original element ID
}
p.SetID()
p.OverrideID("1") // the same as the spdxID on the package element
f := file.Location{
LocationData: file.LocationData{
Coordinates: file.Coordinates{
+9 -3
View File
@@ -3,6 +3,7 @@ package cyclonedxjson
import (
"bytes"
"flag"
"regexp"
"strings"
"testing"
@@ -116,6 +117,14 @@ func TestCycloneDxImageEncoder(t *testing.T) {
func redactor(values ...string) testutil.Redactor {
return testutil.NewRedactions().
WithValuesRedacted(values...).
WithPatternRedactorSpec(
testutil.PatternReplacement{
// only the source component bom-ref (not package or other component bom-refs)
Search: regexp.MustCompile(`"component": \{[^}]*"bom-ref":\s*"(?P<redact>.+)"[^}]*}`),
Groups: []string{"redact"}, // use the regex to anchore the search, but only replace bytes within the capture group
Replace: "redacted",
},
).
WithPatternRedactors(
map[string]string{
// UUIDs
@@ -126,9 +135,6 @@ func redactor(values ...string) testutil.Redactor {
// image hashes
`sha256:[A-Fa-f0-9]{64}`: `sha256:redacted`,
// BOM refs
`"bom-ref":\s*"[^"]+"`: `"bom-ref":"redacted"`,
},
)
}
@@ -17,14 +17,14 @@
]
},
"component": {
"bom-ref":"redacted",
"bom-ref": "redacted",
"type": "file",
"name": "some/path"
}
},
"components": [
{
"bom-ref":"redacted",
"bom-ref": "4dd25c6ee16b729a",
"type": "library",
"name": "package-1",
"version": "1.0.1",
@@ -61,7 +61,7 @@
]
},
{
"bom-ref":"redacted",
"bom-ref": "pkg:deb/debian/package-2@2.0.1?package-id=39392bb5e270f669",
"type": "library",
"name": "package-2",
"version": "2.0.1",
@@ -91,7 +91,7 @@
]
},
{
"bom-ref":"redacted",
"bom-ref": "os:debian@1.2.3",
"type": "operating-system",
"name": "debian",
"version": "1.2.3",
@@ -17,7 +17,7 @@
]
},
"component": {
"bom-ref":"redacted",
"bom-ref": "redacted",
"type": "container",
"name": "user-image-input",
"version": "sha256:redacted"
@@ -25,7 +25,7 @@
},
"components": [
{
"bom-ref":"redacted",
"bom-ref": "72567175418f73f8",
"type": "library",
"name": "package-1",
"version": "1.0.1",
@@ -66,7 +66,7 @@
]
},
{
"bom-ref":"redacted",
"bom-ref": "pkg:deb/debian/package-2@2.0.1?package-id=4b756c6f6fb127a3",
"type": "library",
"name": "package-2",
"version": "2.0.1",
@@ -100,7 +100,7 @@
]
},
{
"bom-ref":"redacted",
"bom-ref": "os:debian@1.2.3",
"type": "operating-system",
"name": "debian",
"version": "1.2.3",
+11 -3
View File
@@ -90,16 +90,24 @@ func TestCycloneDxImageEncoder(t *testing.T) {
func redactor(values ...string) testutil.Redactor {
return testutil.NewRedactions().
WithValuesRedacted(values...).
WithPatternRedactorSpec(
testutil.PatternReplacement{
// only the source component bom-ref (not package or other component bom-refs)
Search: regexp.MustCompile(`<component bom-ref="(?P<redact>[^"]*)" type="file">`),
Groups: []string{"redact"}, // use the regex to anchore the search, but only replace bytes within the capture group
Replace: "redacted",
},
).
WithPatternRedactors(
map[string]string{
// dates
`([0-9]+)-(0[1-9]|1[012])-(0[1-9]|[12][0-9]|3[01])[Tt]([01][0-9]|2[0-3]):([0-5][0-9]):([0-5][0-9]|60)(\.[0-9]+)?(([Zz])|([+|\-]([01][0-9]|2[0-3]):[0-5][0-9]))`: `redacted`,
// image hashes and BOM refs
// image hashes
`sha256:[A-Za-z0-9]{64}`: `sha256:redacted`,
// serial numbers and BOM refs
`(serialNumber|bom-ref)="[^"]+"`: `$1="redacted"`,
// serial numbers
`(serialNumber)="[^"]+"`: `$1="redacted"`,
},
)
}
@@ -16,7 +16,7 @@
</component>
</metadata>
<components>
<component bom-ref="redacted" type="library">
<component bom-ref="4dd25c6ee16b729a" type="library">
<name>package-1</name>
<version>1.0.1</version>
<licenses>
@@ -34,7 +34,7 @@
<property name="syft:location:0:path">/some/path/pkg1</property>
</properties>
</component>
<component bom-ref="redacted" type="library">
<component bom-ref="pkg:deb/debian/package-2@2.0.1?package-id=39392bb5e270f669" type="library">
<name>package-2</name>
<version>2.0.1</version>
<cpe>cpe:2.3:*:some:package:2:*:*:*:*:*:*:*</cpe>
@@ -47,7 +47,7 @@
<property name="syft:metadata:installedSize">0</property>
</properties>
</component>
<component bom-ref="redacted" type="operating-system">
<component bom-ref="os:debian@1.2.3" type="operating-system">
<name>debian</name>
<version>1.2.3</version>
<description>debian</description>
@@ -11,13 +11,13 @@
</component>
</components>
</tools>
<component bom-ref="redacted" type="container">
<component bom-ref="f28a4ba3ddfdddad" type="container">
<name>user-image-input</name>
<version>sha256:redacted</version>
</component>
</metadata>
<components>
<component bom-ref="redacted" type="library">
<component bom-ref="72567175418f73f8" type="library">
<name>package-1</name>
<version>1.0.1</version>
<licenses>
@@ -36,7 +36,7 @@
<property name="syft:location:0:path">/somefile-1.txt</property>
</properties>
</component>
<component bom-ref="redacted" type="library">
<component bom-ref="pkg:deb/debian/package-2@2.0.1?package-id=4b756c6f6fb127a3" type="library">
<name>package-2</name>
<version>2.0.1</version>
<cpe>cpe:2.3:*:some:package:2:*:*:*:*:*:*:*</cpe>
@@ -50,7 +50,7 @@
<property name="syft:metadata:installedSize">0</property>
</properties>
</component>
<component bom-ref="redacted" type="operating-system">
<component bom-ref="os:debian@1.2.3" type="operating-system">
<name>debian</name>
<version>1.2.3</version>
<description>debian</description>
@@ -6,6 +6,7 @@ import (
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/internal/packagemetadata"
"github.com/anchore/syft/syft/pkg"
@@ -84,12 +85,13 @@ func decodeComponent(c *cyclonedx.Component) *pkg.Package {
}
p := &pkg.Package{
Name: c.Name,
Version: c.Version,
Locations: decodeLocations(values),
Licenses: pkg.NewLicenseSet(decodeLicenses(c)...),
CPEs: decodeCPEs(c),
PURL: c.PackageURL,
Name: c.Name,
Version: c.Version,
Locations: decodeLocations(values),
Licenses: pkg.NewLicenseSet(decodeLicenses(c)...),
CPEs: decodeCPEs(c),
PURL: c.PackageURL,
SupplementalData: []any{artifact.ID(c.BOMRef)},
}
DecodeInto(p, values, "syft:package", CycloneDXFields)
@@ -70,7 +70,6 @@ func collectPackages(component *cyclonedx.Component, s *sbom.SBOM, idMap map[str
if syftID != "" {
idMap[syftID] = p
}
// TODO there must be a better way than needing to call this manually:
p.SetID()
s.Artifacts.Packages.Add(*p)
}
@@ -5,7 +5,6 @@ import (
"fmt"
"strings"
"github.com/anchore/syft/internal/licenses"
"github.com/anchore/syft/internal/spdxlicense"
"github.com/anchore/syft/syft/license"
"github.com/anchore/syft/syft/pkg"
@@ -59,39 +58,14 @@ func joinLicenses(licenses []SPDXLicense) string {
}
type SPDXLicense struct {
ID string
Value string
ID string
Value string
FullText string
}
func ParseLicenses(raw []pkg.License) (concluded, declared []SPDXLicense) {
for _, l := range raw {
if l.Value == "" {
continue
}
candidate := SPDXLicense{}
if l.SPDXExpression != "" && !strings.HasPrefix(l.SPDXExpression, licenses.UnknownLicensePrefix) {
candidate.ID = l.SPDXExpression
} else {
candidate.Value = l.Value
// we did not find a valid SPDX license ID so treat as separate license
if strings.HasPrefix(l.SPDXExpression, licenses.UnknownLicensePrefix) {
candidate.ID = spdxlicense.LicenseRefPrefix + SanitizeElementID(l.SPDXExpression)
if len(l.Contents) > 0 {
candidate.Value = l.Contents
}
} else {
if len(l.Value) <= 64 {
// if the license text is less than the size of the hash,
// just use it directly so the id is more readable
candidate.ID = spdxlicense.LicenseRefPrefix + SanitizeElementID(l.Value)
} else {
hash := sha256.Sum256([]byte(l.Value))
candidate.ID = fmt.Sprintf("%s%x", spdxlicense.LicenseRefPrefix, hash)
}
}
}
candidate := createSPDXLicense(l)
switch l.Type {
case license.Concluded:
concluded = append(concluded, candidate)
@@ -102,3 +76,33 @@ func ParseLicenses(raw []pkg.License) (concluded, declared []SPDXLicense) {
return concluded, declared
}
func createSPDXLicense(l pkg.License) SPDXLicense {
candidate := SPDXLicense{
ID: generateLicenseID(l),
FullText: l.Contents,
}
if l.SPDXExpression == "" {
candidate.Value = l.Value
}
return candidate
}
func generateLicenseID(l pkg.License) string {
if l.SPDXExpression != "" {
return l.SPDXExpression
}
if l.Value != "" {
return spdxlicense.LicenseRefPrefix + SanitizeElementID(l.Value)
}
return licenseSum(l.Contents)
}
func licenseSum(s string) string {
if len(s) <= 64 {
return spdxlicense.LicenseRefPrefix + SanitizeElementID(s)
}
hash := sha256.Sum256([]byte(s))
return fmt.Sprintf("%s%x", spdxlicense.LicenseRefPrefix, hash)
}
@@ -105,6 +105,58 @@ func Test_License(t *testing.T) {
}
}
func TestGenerateLicenseID(t *testing.T) {
tests := []struct {
name string
license pkg.License
expected string
}{
{
name: "SPDX expression is preferred",
license: pkg.License{
SPDXExpression: "Apache-2.0",
Value: "SomeValue",
Contents: "Some text",
},
expected: "Apache-2.0",
},
{
name: "Uses value if no SPDX expression",
license: pkg.License{
Value: "MIT",
},
expected: spdxlicense.LicenseRefPrefix + "MIT",
},
{
name: "Long value is sanitized correctly",
license: pkg.License{
Value: "LGPLv2+ and LGPLv2+ with exceptions and GPLv2+ and GPLv2+ with exceptions and BSD and Inner-Net and ISC and Public Domain and GFDL",
},
expected: spdxlicense.LicenseRefPrefix +
"LGPLv2--and-LGPLv2--with-exceptions-and-GPLv2--and-GPLv2--with-exceptions-and-BSD-and-Inner-Net-and-ISC-and-Public-Domain-and-GFDL",
},
{
name: "Uses hash of contents when nothing else is provided",
license: pkg.License{
Contents: "This is a very long custom license text that should be hashed because it's more than 64 characters long.",
},
expected: "", // We'll verify it starts with the correct prefix
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
id := generateLicenseID(tt.license)
if tt.expected == "" {
assert.True(t, len(id) > len(spdxlicense.LicenseRefPrefix))
assert.Contains(t, id, spdxlicense.LicenseRefPrefix)
} else {
assert.Equal(t, tt.expected, id)
}
})
}
}
func Test_joinLicenses(t *testing.T) {
tests := []struct {
name string
@@ -34,6 +34,7 @@ func Test_OriginatorSupplier(t *testing.T) {
pkg.NixStoreEntry{},
pkg.NpmPackageLockEntry{},
pkg.PhpComposerInstalledEntry{},
pkg.PhpPearEntry{},
pkg.PhpPeclEntry{},
pkg.PortageEntry{},
pkg.PythonPipfileLockEntry{},
@@ -40,6 +40,8 @@ func SourceInfo(p pkg.Package) string {
answer = "acquired package info from rust cargo manifest"
case pkg.PhpComposerPkg:
answer = "acquired package info from PHP composer manifest"
case pkg.PhpPearPkg:
answer = "acquired package info from PHP Pear manifest"
case pkg.PhpPeclPkg:
answer = "acquired package info from PHP Pecl manifest"
case pkg.CocoapodsPkg:
@@ -143,6 +143,14 @@ func Test_SourceInfo(t *testing.T) {
"from PHP Pecl manifest",
},
},
{
input: pkg.Package{
Type: pkg.PhpPearPkg,
},
expected: []string{
"from PHP Pear manifest",
},
},
{
input: pkg.Package{
Type: pkg.DartPubPkg,
@@ -8,6 +8,7 @@ var expr = regexp.MustCompile("[^a-zA-Z0-9.-]")
// SPDX spec says SPDXID must be:
// "SPDXRef-"[idstring] where [idstring] is a unique string containing letters, numbers, ., and/or -
// https://spdx.github.io/spdx-spec/v2.3/snippet-information/
func SanitizeElementID(id string) string {
return expr.ReplaceAllString(id, "-")
}
+69 -1
View File
@@ -28,6 +28,7 @@ func (r RedactorFn) Redact(b []byte) []byte {
type PatternReplacement struct {
Search *regexp.Regexp
Groups []string
Replace string
}
@@ -39,7 +40,67 @@ func NewPatternReplacement(r *regexp.Regexp) PatternReplacement {
}
func (p PatternReplacement) Redact(b []byte) []byte {
return p.Search.ReplaceAll(b, []byte(p.Replace))
if len(p.Groups) == 0 {
return p.Search.ReplaceAll(b, []byte(p.Replace))
}
return p.redactNamedGroups(b)
}
func (p PatternReplacement) redactNamedGroups(b []byte) []byte {
groupsToReplace := make(map[string]bool)
for _, g := range p.Groups {
groupsToReplace[g] = true
}
subexpNames := p.Search.SubexpNames()
return p.Search.ReplaceAllFunc(b, func(match []byte) []byte {
indexes := p.Search.FindSubmatchIndex(match)
if indexes == nil {
return match
}
result := make([]byte, len(match))
copy(result, match)
// keep track of the offset as we replace groups
offset := 0
// process each named group
for i, name := range subexpNames {
// skip the full match (i==0) and groups we don't want to replace
if i == 0 || !groupsToReplace[name] {
continue
}
// get the start and end positions of this group
startPos := indexes[2*i]
endPos := indexes[2*i+1]
// skip if the group didn't match
if startPos < 0 || endPos < 0 {
continue
}
// adjust positions based on previous replacements
startPos += offset
endPos += offset
// replace the group with our replacement text
beforeGroup := result[:startPos]
afterGroup := result[endPos:]
// calculate the new offset
oldLen := endPos - startPos
newLen := len(p.Replace)
offset += (newLen - oldLen)
result = append(beforeGroup, append([]byte(p.Replace), afterGroup...)...) //nolint:gocritic
}
return result
})
}
// Replace by value //////////////////////////////
@@ -86,6 +147,13 @@ func (r *Redactions) WithPatternRedactors(values map[string]string) *Redactions
return r
}
func (r *Redactions) WithPatternRedactorSpec(values ...PatternReplacement) *Redactions {
for _, v := range values {
r.redactors = append(r.redactors, v)
}
return r
}
func (r *Redactions) WithValueRedactors(values map[string]string) *Redactions {
for k, v := range values {
r.redactors = append(r.redactors,
@@ -0,0 +1,130 @@
{
"artifacts": [
{
"id": "25f6913140cb5286",
"name": "package-1",
"version": "1.0.1",
"type": "python",
"foundBy": "the-cataloger-1",
"locations": [
{
"path": "/somefile-1.txt",
"layerID": "sha256:100d5a55f9032faead28b7427fa3e650e4f0158f86ea89d06e1489df00cb8c6f",
"accessPath": "/somefile-1.txt"
}
],
"licenses": [
{
"value": "MIT",
"fullText": "",
"spdxExpression": "MIT",
"type": "declared",
"urls": [],
"locations": []
}
],
"language": "python",
"cpes": [
{
"cpe": "cpe:2.3:*:some:package:1:*:*:*:*:*:*:*",
"source": "syft-generated"
}
],
"purl": "a-purl-1",
"metadataType": "python-package",
"metadata": {
"name": "package-1",
"version": "1.0.1",
"author": "",
"authorEmail": "",
"platform": "",
"sitePackagesRootPath": ""
}
},
{
"id": "4b756c6f6fb127a3",
"name": "package-2",
"version": "2.0.1",
"type": "deb",
"foundBy": "the-cataloger-2",
"locations": [
{
"path": "/somefile-2.txt",
"layerID": "sha256:000fb9200890d3a19138478b20023023c0dce1c54352007c2863716780f049eb",
"accessPath": "/somefile-2.txt"
}
],
"licenses": [],
"language": "",
"cpes": [
{
"cpe": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*",
"source": "nvd-cpe-dictionary"
}
],
"purl": "pkg:deb/debian/package-2@2.0.1",
"metadataType": "dpkg-db-entry",
"metadata": {
"package": "package-2",
"source": "",
"version": "2.0.1",
"sourceVersion": "",
"architecture": "",
"maintainer": "",
"installedSize": 0,
"files": null
}
}
],
"artifactRelationships": [],
"source": {
"id": "34d40fdc6ca13e9a3fa18415db216b50bff047716fae7d95a225c09732fe83fb",
"name": "user-image-input",
"version": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"type": "image",
"metadata": {
"userInput": "user-image-input",
"imageID": "sha256:bf783ea304a3f02b5c7d2ece521800f5e2182e65ed5bb5116f578e17d6e82be4",
"manifestDigest": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"tags": [
"stereoscope-fixture-image-simple:85066c51088bdd274f7a89e99e00490f666c49e72ffc955707cd6e18f0e22c5b"
],
"imageSize": 38,
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:100d5a55f9032faead28b7427fa3e650e4f0158f86ea89d06e1489df00cb8c6f",
"size": 22
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:000fb9200890d3a19138478b20023023c0dce1c54352007c2863716780f049eb",
"size": 16
}
],
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NzIsImRpZ2VzdCI6InNoYTI1NjpiZjc4M2VhMzA0YTNmMDJiNWM3ZDJlY2U1MjE4MDBmNWUyMTgyZTY1ZWQ1YmI1MTE2ZjU3OGUxN2Q2ZTgyYmU0In0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1NjoxMDBkNWE1NWY5MDMyZmFlYWQyOGI3NDI3ZmEzZTY1MGU0ZjAxNThmODZlYTg5ZDA2ZTE0ODlkZjAwY2I4YzZmIn0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjAwMGZiOTIwMDg5MGQzYTE5MTM4NDc4YjIwMDIzMDIzYzBkY2UxYzU0MzUyMDA3YzI4NjM3MTY3ODBmMDQ5ZWIifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhcm02NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjMtMDktMjhUMTI6MjM6MzUuNDAwNjcyODg1WiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIzLTA5LTI4VDEyOjIzOjM1LjM5Mzk4NjUxWiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0xLnR4dCAvc29tZWZpbGUtMS50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn0seyJjcmVhdGVkIjoiMjAyMy0wOS0yOFQxMjoyMzozNS40MDA2NzI4ODVaIiwiY3JlYXRlZF9ieSI6IkFERCBmaWxlLTIudHh0IC9zb21lZmlsZS0yLnR4dCAjIGJ1aWxka2l0IiwiY29tbWVudCI6ImJ1aWxka2l0LmRvY2tlcmZpbGUudjAifV0sIm9zIjoibGludXgiLCJyb290ZnMiOnsidHlwZSI6ImxheWVycyIsImRpZmZfaWRzIjpbInNoYTI1NjoxMDBkNWE1NWY5MDMyZmFlYWQyOGI3NDI3ZmEzZTY1MGU0ZjAxNThmODZlYTg5ZDA2ZTE0ODlkZjAwY2I4YzZmIiwic2hhMjU2OjAwMGZiOTIwMDg5MGQzYTE5MTM4NDc4YjIwMDIzMDIzYzBkY2UxYzU0MzUyMDA3YzI4NjM3MTY3ODBmMDQ5ZWIiXX19",
"repoDigests": [],
"architecture": "",
"os": ""
}
},
"distro": {
"prettyName": "debian",
"name": "debian",
"id": "debian",
"idLike": [
"like!"
],
"version": "1.2.3",
"versionID": "1.2.3"
},
"descriptor": {
"name": "syft",
"version": "v0.42.0-bogus",
"configuration": {
"config-key": "config-value"
}
}
}
@@ -1,106 +0,0 @@
{
"spdxVersion": "SPDX-2.3",
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "user-image-input",
"documentNamespace":"redacted",
"creationInfo": {
"licenseListVersion":"redacted",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-v0.42.0-bogus"
],
"created":"redacted"
},
"packages": [
{
"name": "package-1",
"SPDXID": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"versionInfo": "1.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"sourceInfo": "acquired package info from installed python package manifest file: /somefile-1.txt",
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "MIT",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"
},
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "a-purl-1"
}
]
},
{
"name": "package-2",
"SPDXID": "SPDXRef-Package-deb-package-2-4b756c6f6fb127a3",
"versionInfo": "2.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"sourceInfo": "acquired package info from DPKG DB: /somefile-2.txt",
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "NOASSERTION",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"
},
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
},
{
"name": "user-image-input",
"SPDXID": "SPDXRef-DocumentRoot-Image-user-image-input",
"versionInfo": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"checksums": [
{
"algorithm": "SHA256",
"checksumValue": "2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
}
],
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch="
}
],
"primaryPackagePurpose": "CONTAINER"
}
],
"relationships": [
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-deb-package-2-4b756c6f6fb127a3",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DocumentRoot-Image-user-image-input",
"relationshipType": "DESCRIBES"
}
]
}
@@ -1,246 +0,0 @@
{
"spdxVersion": "SPDX-2.3",
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "user-image-input",
"documentNamespace":"redacted",
"creationInfo": {
"licenseListVersion":"redacted",
"creators": [
"Organization: Anchore, Inc",
"Tool: syft-v0.42.0-bogus"
],
"created":"redacted"
},
"packages": [
{
"name": "package-1",
"SPDXID": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"versionInfo": "1.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"sourceInfo": "acquired package info from installed python package manifest file: /somefile-1.txt",
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "MIT",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"
},
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "a-purl-1"
}
]
},
{
"name": "package-2",
"SPDXID": "SPDXRef-Package-deb-package-2-4b756c6f6fb127a3",
"versionInfo": "2.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"sourceInfo": "acquired package info from DPKG DB: /somefile-2.txt",
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "NOASSERTION",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceType": "cpe23Type",
"referenceLocator": "cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"
},
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:deb/debian/package-2@2.0.1"
}
]
},
{
"name": "user-image-input",
"SPDXID": "SPDXRef-DocumentRoot-Image-user-image-input",
"versionInfo": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"checksums": [
{
"algorithm": "SHA256",
"checksumValue": "2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368"
}
],
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch="
}
],
"primaryPackagePurpose": "CONTAINER"
}
],
"files": [
{
"fileName": "/a1/f6",
"SPDXID": "SPDXRef-File-a1-f6-9c2f7510199b17f6",
"fileTypes": [
"OTHER"
],
"checksums": [
{
"algorithm": "SHA1",
"checksumValue": "0000000000000000000000000000000000000000"
}
],
"licenseConcluded": "NOASSERTION",
"licenseInfoInFiles": [
"NOASSERTION"
],
"copyrightText": "NOASSERTION"
},
{
"fileName": "/d1/f3",
"SPDXID": "SPDXRef-File-d1-f3-c6f5b29dca12661f",
"fileTypes": [
"OTHER"
],
"checksums": [
{
"algorithm": "SHA1",
"checksumValue": "0000000000000000000000000000000000000000"
}
],
"licenseConcluded": "NOASSERTION",
"licenseInfoInFiles": [
"NOASSERTION"
],
"copyrightText": "NOASSERTION"
},
{
"fileName": "/d2/f4",
"SPDXID": "SPDXRef-File-d2-f4-c641caa71518099f",
"fileTypes": [
"OTHER"
],
"checksums": [
{
"algorithm": "SHA1",
"checksumValue": "0000000000000000000000000000000000000000"
}
],
"licenseConcluded": "NOASSERTION",
"licenseInfoInFiles": [
"NOASSERTION"
],
"copyrightText": "NOASSERTION"
},
{
"fileName": "/f1",
"SPDXID": "SPDXRef-File-f1-5265a4dde3edbf7c",
"fileTypes": [
"OTHER"
],
"checksums": [
{
"algorithm": "SHA1",
"checksumValue": "0000000000000000000000000000000000000000"
}
],
"licenseConcluded": "NOASSERTION",
"licenseInfoInFiles": [
"NOASSERTION"
],
"copyrightText": "NOASSERTION"
},
{
"fileName": "/f2",
"SPDXID": "SPDXRef-File-f2-f9e49132a4b96ccd",
"fileTypes": [
"OTHER"
],
"checksums": [
{
"algorithm": "SHA1",
"checksumValue": "0000000000000000000000000000000000000000"
}
],
"licenseConcluded": "NOASSERTION",
"licenseInfoInFiles": [
"NOASSERTION"
],
"copyrightText": "NOASSERTION"
},
{
"fileName": "/z1/f5",
"SPDXID": "SPDXRef-File-z1-f5-839d99ee67d9d174",
"fileTypes": [
"OTHER"
],
"checksums": [
{
"algorithm": "SHA1",
"checksumValue": "0000000000000000000000000000000000000000"
}
],
"licenseConcluded": "NOASSERTION",
"licenseInfoInFiles": [
"NOASSERTION"
],
"copyrightText": "NOASSERTION"
}
],
"relationships": [
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"relatedSpdxElement": "SPDXRef-File-f1-5265a4dde3edbf7c",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"relatedSpdxElement": "SPDXRef-File-z1-f5-839d99ee67d9d174",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"relatedSpdxElement": "SPDXRef-File-a1-f6-9c2f7510199b17f6",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"relatedSpdxElement": "SPDXRef-File-d2-f4-c641caa71518099f",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"relatedSpdxElement": "SPDXRef-File-d1-f3-c6f5b29dca12661f",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"relatedSpdxElement": "SPDXRef-File-f2-f9e49132a4b96ccd",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-deb-package-2-4b756c6f6fb127a3",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DOCUMENT",
"relatedSpdxElement": "SPDXRef-DocumentRoot-Image-user-image-input",
"relationshipType": "DESCRIBES"
}
]
}
@@ -0,0 +1,60 @@
SPDXVersion: SPDX-2.3
DataLicense: CC0-1.0
SPDXID: SPDXRef-DOCUMENT
DocumentName: user-image-input
DocumentNamespace: redacted
LicenseListVersion: redacted
Creator: Organization: Anchore, Inc
Creator: Tool: syft-v0.42.0-bogus
Created: redacted
##### Package: user-image-input
PackageName: user-image-input
SPDXID: SPDXRef-DocumentRoot-Image-user-image-input
PackageVersion: sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
PrimaryPackagePurpose: CONTAINER
FilesAnalyzed: false
PackageChecksum: SHA256: 2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368
PackageLicenseConcluded: NOASSERTION
PackageLicenseDeclared: NOASSERTION
ExternalRef: PACKAGE-MANAGER purl pkg:oci/user-image-input@sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368?arch=
##### Package: package-2
PackageName: package-2
SPDXID: SPDXRef-Package-deb-package-2-4b756c6f6fb127a3
PackageVersion: 2.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from DPKG DB: /somefile-2.txt
PackageLicenseConcluded: NOASSERTION
PackageLicenseDeclared: NOASSERTION
PackageCopyrightText: NOASSERTION
ExternalRef: SECURITY cpe23Type cpe:2.3:*:some:package:2:*:*:*:*:*:*:*
ExternalRef: PACKAGE-MANAGER purl pkg:deb/debian/package-2@2.0.1
##### Package: package-1
PackageName: package-1
SPDXID: SPDXRef-Package-python-package-1-25f6913140cb5286
PackageVersion: 1.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageSourceInfo: acquired package info from installed python package manifest file: /somefile-1.txt
PackageLicenseConcluded: NOASSERTION
PackageLicenseDeclared: MIT
PackageCopyrightText: NOASSERTION
ExternalRef: SECURITY cpe23Type cpe:2.3:*:some:package:1:*:*:*:*:*:*:*
ExternalRef: PACKAGE-MANAGER purl a-purl-1
##### Relationships
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-python-package-1-25f6913140cb5286
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-deb-package-2-4b756c6f6fb127a3
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Image-user-image-input
@@ -15,7 +15,7 @@
"packages": [
{
"name": "package-1",
"SPDXID": "SPDXRef-Package-python-package-1-5a2b1ae000fcb51e",
"SPDXID": "SPDXRef-Package-python-package-1-4dd25c6ee16b729a",
"versionInfo": "1.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
@@ -76,7 +76,7 @@
"relationships": [
{
"spdxElementId": "SPDXRef-DocumentRoot-Directory-some-path",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-5a2b1ae000fcb51e",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-4dd25c6ee16b729a",
"relationshipType": "CONTAINS"
},
{
@@ -15,7 +15,7 @@
"packages": [
{
"name": "package-1",
"SPDXID": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"SPDXID": "SPDXRef-Package-python-package-1-72567175418f73f8",
"versionInfo": "1.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
@@ -90,7 +90,7 @@
"relationships": [
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-72567175418f73f8",
"relationshipType": "CONTAINS"
},
{
@@ -15,7 +15,7 @@
"packages": [
{
"name": "package-1",
"SPDXID": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"SPDXID": "SPDXRef-Package-python-package-1-72567175418f73f8",
"versionInfo": "1.0.1",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
@@ -199,38 +199,38 @@
],
"relationships": [
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"spdxElementId": "SPDXRef-Package-python-package-1-72567175418f73f8",
"relatedSpdxElement": "SPDXRef-File-f1-5265a4dde3edbf7c",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"spdxElementId": "SPDXRef-Package-python-package-1-72567175418f73f8",
"relatedSpdxElement": "SPDXRef-File-z1-f5-839d99ee67d9d174",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"spdxElementId": "SPDXRef-Package-python-package-1-72567175418f73f8",
"relatedSpdxElement": "SPDXRef-File-a1-f6-9c2f7510199b17f6",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"spdxElementId": "SPDXRef-Package-python-package-1-72567175418f73f8",
"relatedSpdxElement": "SPDXRef-File-d2-f4-c641caa71518099f",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"spdxElementId": "SPDXRef-Package-python-package-1-72567175418f73f8",
"relatedSpdxElement": "SPDXRef-File-d1-f3-c6f5b29dca12661f",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"spdxElementId": "SPDXRef-Package-python-package-1-72567175418f73f8",
"relatedSpdxElement": "SPDXRef-File-f2-f9e49132a4b96ccd",
"relationshipType": "CONTAINS"
},
{
"spdxElementId": "SPDXRef-DocumentRoot-Image-user-image-input",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-c5cf7ac34cbca450",
"relatedSpdxElement": "SPDXRef-Package-python-package-1-72567175418f73f8",
"relationshipType": "CONTAINS"
},
{
@@ -91,7 +91,7 @@ ExternalRef: PACKAGE-MANAGER purl pkg:deb/debian/package-2@2.0.1
##### Package: package-1
PackageName: package-1
SPDXID: SPDXRef-Package-python-package-1-c5cf7ac34cbca450
SPDXID: SPDXRef-Package-python-package-1-72567175418f73f8
PackageVersion: 1.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
@@ -105,13 +105,13 @@ ExternalRef: PACKAGE-MANAGER purl a-purl-1
##### Relationships
Relationship: SPDXRef-Package-python-package-1-c5cf7ac34cbca450 CONTAINS SPDXRef-File-f1-5265a4dde3edbf7c
Relationship: SPDXRef-Package-python-package-1-c5cf7ac34cbca450 CONTAINS SPDXRef-File-z1-f5-839d99ee67d9d174
Relationship: SPDXRef-Package-python-package-1-c5cf7ac34cbca450 CONTAINS SPDXRef-File-a1-f6-9c2f7510199b17f6
Relationship: SPDXRef-Package-python-package-1-c5cf7ac34cbca450 CONTAINS SPDXRef-File-d2-f4-c641caa71518099f
Relationship: SPDXRef-Package-python-package-1-c5cf7ac34cbca450 CONTAINS SPDXRef-File-d1-f3-c6f5b29dca12661f
Relationship: SPDXRef-Package-python-package-1-c5cf7ac34cbca450 CONTAINS SPDXRef-File-f2-f9e49132a4b96ccd
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-python-package-1-c5cf7ac34cbca450
Relationship: SPDXRef-Package-python-package-1-72567175418f73f8 CONTAINS SPDXRef-File-f1-5265a4dde3edbf7c
Relationship: SPDXRef-Package-python-package-1-72567175418f73f8 CONTAINS SPDXRef-File-z1-f5-839d99ee67d9d174
Relationship: SPDXRef-Package-python-package-1-72567175418f73f8 CONTAINS SPDXRef-File-a1-f6-9c2f7510199b17f6
Relationship: SPDXRef-Package-python-package-1-72567175418f73f8 CONTAINS SPDXRef-File-d2-f4-c641caa71518099f
Relationship: SPDXRef-Package-python-package-1-72567175418f73f8 CONTAINS SPDXRef-File-d1-f3-c6f5b29dca12661f
Relationship: SPDXRef-Package-python-package-1-72567175418f73f8 CONTAINS SPDXRef-File-f2-f9e49132a4b96ccd
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-python-package-1-72567175418f73f8
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-deb-package-2-4b756c6f6fb127a3
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Image-user-image-input
@@ -38,7 +38,7 @@ ExternalRef: PACKAGE-MANAGER purl pkg:deb/debian/package-2@2.0.1
##### Package: package-1
PackageName: package-1
SPDXID: SPDXRef-Package-python-package-1-5a2b1ae000fcb51e
SPDXID: SPDXRef-Package-python-package-1-4dd25c6ee16b729a
PackageVersion: 1.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
@@ -52,7 +52,7 @@ ExternalRef: PACKAGE-MANAGER purl a-purl-2
##### Relationships
Relationship: SPDXRef-DocumentRoot-Directory-some-path CONTAINS SPDXRef-Package-python-package-1-5a2b1ae000fcb51e
Relationship: SPDXRef-DocumentRoot-Directory-some-path CONTAINS SPDXRef-Package-python-package-1-4dd25c6ee16b729a
Relationship: SPDXRef-DocumentRoot-Directory-some-path CONTAINS SPDXRef-Package-deb-package-2-39392bb5e270f669
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Directory-some-path
@@ -41,7 +41,7 @@ ExternalRef: PACKAGE-MANAGER purl pkg:deb/debian/package-2@2.0.1
##### Package: package-1
PackageName: package-1
SPDXID: SPDXRef-Package-python-package-1-c5cf7ac34cbca450
SPDXID: SPDXRef-Package-python-package-1-72567175418f73f8
PackageVersion: 1.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: NOASSERTION
@@ -55,7 +55,7 @@ ExternalRef: PACKAGE-MANAGER purl a-purl-1
##### Relationships
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-python-package-1-c5cf7ac34cbca450
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-python-package-1-72567175418f73f8
Relationship: SPDXRef-DocumentRoot-Image-user-image-input CONTAINS SPDXRef-Package-deb-package-2-4b756c6f6fb127a3
Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Image-user-image-input
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"io"
"strings"
"github.com/Masterminds/semver"
"github.com/Masterminds/semver/v3"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
+1
View File
@@ -47,6 +47,7 @@ type licenses []License
type License struct {
Value string `json:"value"`
FullText string `json:"fullText"`
SPDXExpression string `json:"spdxExpression"`
Type license.Type `json:"type"`
URLs []string `json:"urls"`
@@ -1,7 +1,7 @@
{
"artifacts": [
{
"id": "5a2b1ae000fcb51e",
"id": "4dd25c6ee16b729a",
"name": "package-1",
"version": "1.0.1",
"type": "python",
@@ -15,6 +15,7 @@
"licenses": [
{
"value": "MIT",
"fullText": "",
"spdxExpression": "MIT",
"type": "declared",
"urls": [],
@@ -1,7 +1,7 @@
{
"artifacts": [
{
"id": "ad3ecac55fe1c30f",
"id": "fba4ca04d4906f25",
"name": "package-1",
"version": "1.0.1",
"type": "python",
@@ -15,6 +15,7 @@
"licenses": [
{
"value": "MIT",
"fullText": "",
"spdxExpression": "MIT",
"type": "declared",
"urls": [],
@@ -1,7 +1,7 @@
{
"artifacts": [
{
"id": "c5cf7ac34cbca450",
"id": "72567175418f73f8",
"name": "package-1",
"version": "1.0.1",
"type": "python",
@@ -16,6 +16,7 @@
"licenses": [
{
"value": "MIT",
"fullText": "",
"spdxExpression": "MIT",
"type": "declared",
"urls": [],
+1 -1
View File
@@ -231,10 +231,10 @@ func toLicenseModel(pkgLicenses []pkg.License) (modelLicenses []model.License) {
modelLicenses = append(modelLicenses, model.License{
Value: l.Value,
SPDXExpression: l.SPDXExpression,
Contents: l.Contents,
Type: l.Type,
URLs: urls,
Locations: locations,
Contents: l.Contents,
})
}
return
@@ -16,8 +16,9 @@ var _ file.Resolver = (*ContainerImageAllLayers)(nil)
// ContainerImageAllLayers implements path and content access for the AllLayers source option for container image data sources.
type ContainerImageAllLayers struct {
img *image.Image
layers []int
img *image.Image
layers []int
markVisibility bool
}
// NewFromContainerImageAllLayers returns a new resolver from the perspective of all image layers for the given image.
@@ -33,6 +34,10 @@ func NewFromContainerImageAllLayers(img *image.Image) (*ContainerImageAllLayers,
return &ContainerImageAllLayers{
img: img,
layers: layers,
// This is the entrypoint for the user-facing implementation, which should always annotate locations.
// We have other resolvers that use this implementation that are already responsible
// for marking visibility, so we don't need to do it all of the time (a small performance optimization).
markVisibility: true,
}, nil
}
@@ -112,7 +117,9 @@ func (r *ContainerImageAllLayers) FilesByPath(paths ...string) ([]file.Location,
return nil, err
}
for _, result := range results {
uniqueLocations = append(uniqueLocations, file.NewLocationFromImage(path, result, r.img))
l := file.NewLocationFromImage(path, result, r.img)
r.annotateLocation(&l)
uniqueLocations = append(uniqueLocations, l)
}
}
}
@@ -156,7 +163,9 @@ func (r *ContainerImageAllLayers) FilesByGlob(patterns ...string) ([]file.Locati
return nil, err
}
for _, refResult := range refResults {
uniqueLocations = append(uniqueLocations, file.NewLocationFromImage(string(result.RequestPath), refResult, r.img))
l := file.NewLocationFromImage(string(result.RequestPath), refResult, r.img)
r.annotateLocation(&l)
uniqueLocations = append(uniqueLocations, l)
}
}
}
@@ -172,7 +181,7 @@ func (r *ContainerImageAllLayers) RelativeFileByPath(location file.Location, pat
exists, relativeRef, err := layer.SquashedTree.File(stereoscopeFile.Path(path), filetree.FollowBasenameLinks)
if err != nil {
log.Errorf("failed to find path=%q in squash: %+w", path, err)
log.Errorf("failed to find path=%q in squash: %+v", path, err)
return nil
}
if !exists && !relativeRef.HasReference() {
@@ -180,6 +189,7 @@ func (r *ContainerImageAllLayers) RelativeFileByPath(location file.Location, pat
}
relativeLocation := file.NewLocationFromImage(path, *relativeRef.Reference, r.img)
r.annotateLocation(&relativeLocation)
return &relativeLocation
}
@@ -228,7 +238,9 @@ func (r *ContainerImageAllLayers) FilesByMIMEType(types ...string) ([]file.Locat
return nil, err
}
for _, refResult := range refResults {
uniqueLocations = append(uniqueLocations, file.NewLocationFromImage(string(ref.RequestPath), refResult, r.img))
l := file.NewLocationFromImage(string(ref.RequestPath), refResult, r.img)
r.annotateLocation(&l)
uniqueLocations = append(uniqueLocations, l)
}
}
}
@@ -243,10 +255,12 @@ func (r *ContainerImageAllLayers) AllLocations(ctx context.Context) <-chan file.
for _, layerIdx := range r.layers {
tree := r.img.Layers[layerIdx].Tree
for _, ref := range tree.AllFiles(stereoscopeFile.AllTypes()...) {
l := file.NewLocationFromImage(string(ref.RealPath), ref, r.img)
r.annotateLocation(&l)
select {
case <-ctx.Done():
return
case results <- file.NewLocationFromImage(string(ref.RealPath), ref, r.img):
case results <- l:
continue
}
}
@@ -258,3 +272,36 @@ func (r *ContainerImageAllLayers) AllLocations(ctx context.Context) <-chan file.
func (r *ContainerImageAllLayers) FileMetadataByLocation(location file.Location) (file.Metadata, error) {
return fileMetadataByLocation(r.img, location)
}
func (r *ContainerImageAllLayers) annotateLocation(l *file.Location) {
if !r.markVisibility || l == nil {
return
}
givenRef := l.Reference()
annotation := file.VisibleAnnotation
// if we find a location for a path that matches the query (e.g. **/node_modules) but is not present in the squashed tree, skip it
ref, err := r.img.SquashedSearchContext.SearchByPath(l.RealPath, filetree.DoNotFollowDeadBasenameLinks)
if err != nil || !ref.HasReference() {
annotation = file.HiddenAnnotation
} else if ref.ID() != givenRef.ID() {
// we may have the path in the squashed tree, but this must not be in the same layer
annotation = file.HiddenAnnotation
}
// not only should the real path to the file exist, but the way we took to get there should also exist
// (e.g. if we are looking for /etc/passwd, but the real path is /etc/passwd -> /etc/passwd-1, then we should
// make certain that /etc/passwd-1 exists)
if annotation == file.VisibleAnnotation && l.AccessPath != "" {
ref, err := r.img.SquashedSearchContext.SearchByPath(l.AccessPath, filetree.DoNotFollowDeadBasenameLinks)
if err != nil || !ref.HasReference() {
annotation = file.HiddenAnnotation
} else if ref.ID() != givenRef.ID() {
// we may have the path in the squashed tree, but this must not be in the same layer
annotation = file.HiddenAnnotation
}
}
l.Annotations[file.VisibleAnnotationKey] = annotation
}
@@ -522,16 +522,26 @@ func TestAllLayersResolver_AllLocations(t *testing.T) {
paths := strset.New()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
visibleSet := strset.New()
hiddenSet := strset.New()
for loc := range resolver.AllLocations(ctx) {
paths.Add(loc.RealPath)
switch loc.Annotations[file.VisibleAnnotationKey] {
case file.VisibleAnnotation:
visibleSet.Add(loc.RealPath)
case file.HiddenAnnotation:
hiddenSet.Add(loc.RealPath)
case "":
t.Errorf("expected visibility annotation for location: %+v", loc)
}
}
expected := []string{
visible := []string{
"/Dockerfile",
"/file-1.txt",
"/file-3.txt",
"/file-3.txt", // this is a deadlink pointing to /file-1.txt (which has been deleted)
"/target",
"/target/file-2.txt",
}
hidden := []string{
"/.wh.bin",
"/.wh.file-1.txt",
"/.wh.lib",
@@ -619,6 +629,7 @@ func TestAllLayersResolver_AllLocations(t *testing.T) {
"/bin/usleep",
"/bin/watch",
"/bin/zcat",
"/file-1.txt",
"/lib",
"/lib/apk",
"/lib/apk/db",
@@ -641,15 +652,41 @@ func TestAllLayersResolver_AllLocations(t *testing.T) {
"/lib/sysctl.d/00-alpine.conf",
}
// depending on how the image is built (either from linux or mac), sys and proc might accidentally be added to the image.
// this isn't important for the test, so we remove them.
paths.Remove("/proc", "/sys", "/dev", "/etc")
var expected []string
expected = append(expected, visible...)
expected = append(expected, hidden...)
sort.Strings(expected)
// Remove cache created by Mac Rosetta when emulating different arches
paths.Remove("/.cache/rosetta", "/.cache")
cleanPaths := func(s *strset.Set) {
// depending on how the image is built (either from linux or mac), sys and proc might accidentally be added to the image.
// this isn't important for the test, so we remove them.
s.Remove("/proc", "/sys", "/dev", "/etc")
// Remove cache created by Mac Rosetta when emulating different arches
s.Remove("/.cache/rosetta", "/.cache")
}
cleanPaths(paths)
cleanPaths(visibleSet)
cleanPaths(hiddenSet)
pathsList := paths.List()
sort.Strings(pathsList)
visibleSetList := visibleSet.List()
sort.Strings(visibleSetList)
hiddenSetList := hiddenSet.List()
sort.Strings(hiddenSetList)
if d := cmp.Diff(expected, pathsList); d != "" {
t.Errorf("unexpected paths (-want +got):\n%s", d)
}
if d := cmp.Diff(visible, visibleSetList); d != "" {
t.Errorf("unexpected visible paths (-want +got):\n%s", d)
}
if d := cmp.Diff(hidden, hiddenSetList); d != "" {
t.Errorf("unexpected hidden paths (-want +got):\n%s", d)
}
assert.ElementsMatchf(t, expected, pathsList, "expected all paths to be indexed, but found different paths: \n%s", cmp.Diff(expected, paths.List()))
}
@@ -0,0 +1,255 @@
package fileresolver
import (
"context"
"io"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/syft/file"
)
var _ file.Resolver = (*ContainerImageDeepSquash)(nil)
// ContainerImageDeepSquash implements path and content access for the paths in the squashed tree, but with additional
// depth from all layers. The goal of this is to allow for producing results where the first layer which the material
// was added can be annotated in the SBOM (as opposed to the last [visible] layer for the path like with the squashed
// file resolver).
type ContainerImageDeepSquash struct {
squashed file.Resolver
allLayers file.Resolver
}
// NewFromContainerImageDeepSquash returns a new resolver from the perspective of all image layers for the given image.
func NewFromContainerImageDeepSquash(img *image.Image) (*ContainerImageDeepSquash, error) {
squashed, err := NewFromContainerImageSquash(img)
if err != nil {
return nil, err
}
allLayers, err := NewFromContainerImageAllLayers(img)
if err != nil {
return nil, err
}
// we will do the work here to mark visibility with results from two resolvers (don't do the work twice!)
allLayers.markVisibility = false
return &ContainerImageDeepSquash{
squashed: squashed,
allLayers: allLayers,
}, nil
}
// HasPath indicates if the given path exists in the underlying source.
func (i *ContainerImageDeepSquash) HasPath(path string) bool {
// there is no need to merge results from all layers since path-based results should always be adjusted relative to the squashed tree (which is different when considering layers)
return i.squashed.HasPath(path)
}
// FilesByPath returns all file.References that match the given paths from any layer in the image.
func (i *ContainerImageDeepSquash) FilesByPath(paths ...string) ([]file.Location, error) {
squashedLocations, err := i.squashed.FilesByPath(paths...)
if err != nil {
return nil, err
}
if len(squashedLocations) == 0 {
// this is meant to return all files in all layers only for paths that are present in the squashed tree. If
// there are no results from the squashed tree then there are no paths to raise up.
return nil, nil
}
allLayersLocations, err := i.allLayers.FilesByPath(paths...)
if err != nil {
return nil, err
}
return i.mergeLocations(squashedLocations, allLayersLocations), nil
}
// FilesByGlob returns all file.References that match the given path glob pattern from any layer in the image.
func (i *ContainerImageDeepSquash) FilesByGlob(patterns ...string) ([]file.Location, error) {
squashedLocations, err := i.squashed.FilesByGlob(patterns...)
if err != nil {
return nil, err
}
if len(squashedLocations) == 0 {
// this is meant to return all files in all layers only for paths that are present in the squashed tree. If
// there are no results from the squashed tree then there are no paths to raise up.
return nil, nil
}
allLayersLocations, err := i.allLayers.FilesByGlob(patterns...)
if err != nil {
return nil, err
}
return i.mergeLocations(squashedLocations, allLayersLocations), nil
}
// RelativeFileByPath fetches a single file at the given path relative to the layer squash of the given reference.
// This is helpful when attempting to find a file that is in the same layer or lower as another file.
func (i *ContainerImageDeepSquash) RelativeFileByPath(location file.Location, path string) *file.Location {
if !i.squashed.HasPath(path) {
return nil
}
l := i.squashed.RelativeFileByPath(location, path)
if l != nil {
loc := l.WithAnnotation(file.VisibleAnnotationKey, file.VisibleAnnotation)
return &loc
}
l = i.allLayers.RelativeFileByPath(location, path)
if l != nil {
loc := l.WithAnnotation(file.VisibleAnnotationKey, file.HiddenAnnotation)
return &loc
}
return nil
}
// FileContentsByLocation fetches file contents for a single file reference.
// If the path does not exist an error is returned.
func (i *ContainerImageDeepSquash) FileContentsByLocation(location file.Location) (io.ReadCloser, error) {
// regardless of the layer or scope, if the user gives us a specific path+layer location, then we should always
// return the contents for that specific location (thus all-layers scope must always be used)
return i.allLayers.FileContentsByLocation(location)
}
func (i *ContainerImageDeepSquash) FilesByMIMEType(types ...string) ([]file.Location, error) {
squashedLocations, err := i.squashed.FilesByMIMEType(types...)
if err != nil {
return nil, err
}
if len(squashedLocations) == 0 {
// this is meant to return all files in all layers only for paths that are present in the squashed tree. If
// there are no results from the squashed tree then there are no paths to raise up.
return nil, nil
}
allLayersLocations, err := i.allLayers.FilesByMIMEType(types...)
if err != nil {
return nil, err
}
return i.mergeLocations(squashedLocations, allLayersLocations), nil
}
func (i *ContainerImageDeepSquash) AllLocations(ctx context.Context) <-chan file.Location {
return i.mergeLocationStreams(ctx, i.squashed.AllLocations(ctx), i.allLayers.AllLocations(ctx))
}
func (i *ContainerImageDeepSquash) FileMetadataByLocation(location file.Location) (file.Metadata, error) {
// regardless of the layer or scope, if the user gives us a specific path+layer location, then we should always
// return the metadata for that specific location (thus all-layers scope must always be used)
return i.allLayers.FileMetadataByLocation(location)
}
func (i *ContainerImageDeepSquash) mergeLocations(squashedLocations, allLayersLocations []file.Location) []file.Location {
var result []file.Location
if len(squashedLocations) == 0 {
// this is meant to return all files in all layers only for paths that are present in the squashed tree. If
// there are no results from the squashed tree then there are no paths to raise up.
return nil
}
// we are using a location set to deduplicate locations, but we don't use it for the returned
// results in order to preserve the order of the locations from the underlying filetree query
squashedCoords := file.NewLocationSet()
for _, l := range squashedLocations {
result = append(result, l.WithAnnotation(file.VisibleAnnotationKey, file.VisibleAnnotation))
squashedCoords.Add(l)
}
for _, l := range allLayersLocations {
if squashedCoords.Contains(l) {
// this path + layer is already in the squashed tree results, skip it (deduplicate location results)
continue
}
if !i.squashed.HasPath(l.RealPath) {
// if we find a location for a path that matches the query (e.g. **/node_modules) but is not present in the squashed tree, skip it
continue
}
// not only should the real path to the file exist, but the way we took to get there should also exist
// (e.g. if we are looking for /etc/passwd, but the real path is /etc/passwd -> /etc/passwd-1, then we should
// make certain that /etc/passwd-1 exists)
if l.AccessPath != "" && !i.squashed.HasPath(l.AccessPath) {
continue
}
result = append(result, l.WithAnnotation(file.VisibleAnnotationKey, file.HiddenAnnotation))
}
return result
}
func (i *ContainerImageDeepSquash) mergeLocationStreams(ctx context.Context, squashedLocations, allLayersLocations <-chan file.Location) <-chan file.Location {
result := make(chan file.Location)
go func() {
defer close(result)
// we are using a location set to deduplicate locations, but we don't use it for the returned
// results in order to preserve the order of the locations from the underlying filetree query
squashedCoords := file.NewLocationSet()
var isDone bool
for l := range squashedLocations {
if isDone {
// bleed off the rest of the results from the squashed stream and not leak a goroutine
continue
}
select {
case <-ctx.Done():
isDone = true
default:
result <- l.WithAnnotation(file.VisibleAnnotationKey, file.VisibleAnnotation)
squashedCoords.Add(l)
}
}
for l := range allLayersLocations {
if isDone {
// bleed off the rest of the results from the squashed stream and not leak a goroutine
continue
}
if squashedCoords.Empty() {
// this is meant to return all files in all layers only for paths that are present in the squashed tree.
// If there are no results from the squashed tree, then there are no paths to raise up.
// That being said, we need to bleed off the rest of the results from the allLayersLocations stream
// and not leak a goroutine.
continue
}
if squashedCoords.Contains(l) {
// we've already seen this location from the squashed stream, skip it
continue
}
if !i.squashed.HasPath(l.RealPath) {
// if we find a location for a path that matches the query (e.g. **/node_modules) but is not present in the squashed tree, skip it
continue
}
// not only should the real path to the file exist, but the way we took to get there should also exist
// (e.g. if we are looking for /etc/passwd, but the real path is /etc/passwd -> /etc/passwd-1, then we should
// make certain that /etc/passwd-1 exists)
if l.AccessPath != "" && !i.squashed.HasPath(l.AccessPath) {
continue
}
select {
case <-ctx.Done():
isDone = true
default:
result <- l.WithAnnotation(file.VisibleAnnotationKey, file.HiddenAnnotation)
}
}
}()
return result
}
File diff suppressed because it is too large Load Diff
@@ -379,6 +379,28 @@ func (r directoryIndexer) addSymlinkToIndex(p string, info os.FileInfo) (string,
// if the base is set, then we first need to resolve the link,
// before finding it's location in the base
dir, err := filepath.Rel(r.base, filepath.Dir(p))
// if the relative path to the base contains "..",i.e. p is the parent or ancestor of the base
// For example:
// dir: "/root/asymlink" -> "/root/realdir" (linkTarget:"realdir")
// base: "/root/asymlink"
// so the relative path of /root to the "/root/asymlink" is ".."
// we cannot directly concatenate ".." to "/root/symlink",however,
// the parent directory of linkTarget should be "/root"
for strings.HasPrefix(dir, "..") {
if strings.HasPrefix(dir, "../") {
dir = strings.TrimPrefix(dir, "../")
} else {
dir = strings.TrimPrefix(dir, "..")
}
lastSlash := strings.LastIndex(r.base, "/")
if lastSlash != -1 {
r.base = r.base[:lastSlash]
}
// In case of the root directory
if r.base == "" {
r.base = "/"
}
}
if err != nil {
return "", fmt.Errorf("unable to resolve relative path for path=%q: %w", p, err)
}
@@ -1,10 +1,12 @@
package fileresolver
import (
"fmt"
"io/fs"
"os"
"path"
"path/filepath"
"runtime"
"sort"
"strings"
"testing"
@@ -223,6 +225,55 @@ func TestDirectoryIndexer_index(t *testing.T) {
}
}
func TestDirectoryIndexer_index_for_AncestorSymlinks(t *testing.T) {
// note: this test is testing the effects from NewFromDirectory, indexTree, and addPathToIndex
_, filename, _, ok := runtime.Caller(0)
require.True(t, ok)
dir := filepath.Dir(filename)
tests := []struct {
name string
path string
relative_base string
}{
{
name: "the parent directory has symlink target",
path: "test-fixtures/system_paths/target/symlinks-to-dev",
relative_base: "test-fixtures/system_paths/target/symlinks-to-dev",
},
{
name: "the ancestor directory has symlink target",
path: "test-fixtures/system_paths/target/symlinks-to-hierarchical-dev",
relative_base: "test-fixtures/system_paths/target/symlinks-to-hierarchical-dev/module_1/module_1_1",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
indexer := newDirectoryIndexer("test-fixtures/system_paths/target",
fmt.Sprintf("%v/%v", dir, test.relative_base))
tree, index, err := indexer.build()
require.NoError(t, err)
info, err := os.Stat(test.path)
assert.NoError(t, err)
// note: the index uses absolute paths, so assertions MUST keep this in mind
cwd, err := os.Getwd()
require.NoError(t, err)
p := file.Path(path.Join(cwd, test.path))
assert.Equal(t, true, tree.HasPath(p))
exists, ref, err := tree.File(p)
assert.Equal(t, true, exists)
if assert.NoError(t, err) {
return
}
entry, err := index.Get(*ref.Reference)
require.NoError(t, err)
assert.Equal(t, info.Mode(), entry.Mode)
})
}
}
func TestDirectoryIndexer_index_survive_badSymlink(t *testing.T) {
// test-fixtures/bad-symlinks
// ├── root
@@ -811,7 +811,7 @@ func TestDirectoryResolverDoesNotIgnoreRelativeSystemPaths(t *testing.T) {
// 4: within target/
// 1: target/link --> relative path to "place" // NOTE: this is filtered out since it not unique relative to outside_root/link_target/place
// 1: outside_root/link_target/place
assert.Len(t, locations, 5)
assert.Len(t, locations, 6)
// ensure that symlink indexing outside of root worked
testLocation := "test-fixtures/system_paths/outside_root/link_target/place"
@@ -1542,8 +1542,6 @@ func Test_fileResolver_FileContentsByLocation(t *testing.T) {
}
func TestFileResolver_AllLocations_errorOnDirRequest(t *testing.T) {
defer goleak.VerifyNone(t)
filePath := "./test-fixtures/system_paths/target/home/place"
parentPath, err := absoluteSymlinkFreePathToParent(filePath)
require.NoError(t, err)
@@ -1557,9 +1555,8 @@ func TestFileResolver_AllLocations_errorOnDirRequest(t *testing.T) {
for loc := range resolver.AllLocations(ctx) {
entry, err := resolver.index.Get(loc.Reference())
require.NoError(t, err)
if entry.Metadata.IsDir() {
if dirLoc == nil && entry.Metadata.IsDir() {
dirLoc = &loc
break
}
}
@@ -1568,6 +1565,8 @@ func TestFileResolver_AllLocations_errorOnDirRequest(t *testing.T) {
reader, err := resolver.FileContentsByLocation(*dirLoc)
require.Error(t, err)
require.Nil(t, reader)
goleak.VerifyNone(t)
}
func TestFileResolver_AllLocations(t *testing.T) {
@@ -1592,10 +1591,11 @@ func TestFileResolver_AllLocations(t *testing.T) {
sort.Strings(pathsList)
assert.ElementsMatchf(t, expected, pathsList, "expected all paths to be indexed, but found different paths: \n%s", cmp.Diff(expected, paths.List()))
goleak.VerifyNone(t)
}
func Test_FileResolver_AllLocationsDoesNotLeakGoRoutine(t *testing.T) {
defer goleak.VerifyNone(t)
filePath := "./test-fixtures/system_paths/target/home/place"
parentPath, err := absoluteSymlinkFreePathToParent(filePath)
require.NoError(t, err)
@@ -1609,4 +1609,6 @@ func Test_FileResolver_AllLocationsDoesNotLeakGoRoutine(t *testing.T) {
break
}
cancel()
goleak.VerifyNone(t)
}
@@ -0,0 +1 @@
dev
@@ -0,0 +1 @@
hierarchical-dev
@@ -833,7 +833,7 @@ func Test_UnindexedDirectoryResolverDoesNotIgnoreRelativeSystemPaths(t *testing.
// 4: within target/
// 1: target/link --> relative path to "place" // NOTE: this is filtered out since it not unique relative to outside_root/link_target/place
// 1: outside_root/link_target/place
assert.Len(t, locations, 5)
assert.Len(t, locations, 6)
// ensure that symlink indexing outside of root worked
testLocation := "../outside_root/link_target/place"
@@ -42,6 +42,7 @@ func AllTypes() []any {
pkg.OpamPackage{},
pkg.PhpComposerInstalledEntry{},
pkg.PhpComposerLockEntry{},
pkg.PhpPearEntry{},
pkg.PhpPeclEntry{},
pkg.PortageEntry{},
pkg.PythonPackage{},
+1
View File
@@ -96,6 +96,7 @@ var jsonTypes = makeJSONTypes(
jsonNames(pkg.PhpComposerLockEntry{}, "php-composer-lock-entry", "PhpComposerJsonMetadata"),
jsonNamesWithoutLookup(pkg.PhpComposerInstalledEntry{}, "php-composer-installed-entry", "PhpComposerJsonMetadata"), // the legacy value is split into two types, where the other is preferred
jsonNames(pkg.PhpPeclEntry{}, "php-pecl-entry", "PhpPeclMetadata"),
jsonNames(pkg.PhpPearEntry{}, "php-pear-entry"),
jsonNames(pkg.PortageEntry{}, "portage-db-entry", "PortageMetadata"),
jsonNames(pkg.PythonPackage{}, "python-package", "PythonPackageMetadata"),
jsonNames(pkg.PythonPipfileLockEntry{}, "python-pipfile-lock-entry", "PythonPipfileLockMetadata"),
@@ -1067,6 +1067,17 @@ func Test_Cataloger_PositiveCases(t *testing.T) {
Metadata: metadata("erlang-alpine-binary"),
},
},
{
logicalFixture: "erlang/26.1.2/linux-arm64",
expected: pkg.Package{
Name: "erlang",
Version: "26.1.2",
Type: "binary",
PURL: "pkg:generic/erlang@26.1.2",
Locations: locations("beam.smp"),
Metadata: metadata("erlang-alpine-binary"),
},
},
{
logicalFixture: "swipl/9.3.8/linux-amd64",
expected: pkg.Package{
@@ -1243,6 +1254,17 @@ func Test_Cataloger_PositiveCases(t *testing.T) {
Metadata: metadata("fluent-bit-binary"),
},
},
{
logicalFixture: "fluent-bit/1.3.10/linux-arm",
expected: pkg.Package{
Name: "fluent-bit",
Version: "1.3.10",
Type: "binary",
PURL: "pkg:github/fluent/fluent-bit@1.3.10",
Locations: locations("fluent-bit"),
Metadata: metadata("fluent-bit-binary"),
},
},
{
logicalFixture: "wp/2.9.0/linux-amd64",
expected: pkg.Package{
+6 -1
View File
@@ -426,6 +426,10 @@ func DefaultClassifiers() []Classifier {
// <artificial>[NUL]/usr/local/src/otp-25.3.2.7/erts/
`(?m)/usr/local/src/otp-(?P<version>[0-9]+\.[0-9]+(\.[0-9]+){0,2}(-rc[0-9])?)/erts/`,
),
FileContentsVersionMatcher(
// [NUL][NUL]26.1.2[NUL][NUL][NUL][NUL][NUL][NUL][NUL]NUL[NUL][NUL]Erlang/OTP
`\x00+(?P<version>[0-9]+\.[0-9]+(\.[0-9]+){0,2}(-rc[0-9])?)\x00+Erlang/OTP`,
),
),
Package: "erlang",
PURL: mustPURL("pkg:generic/erlang@version"),
@@ -573,7 +577,8 @@ func DefaultClassifiers() []Classifier {
// [NUL]2.2.3[NUL]Fluent Bit
// [NUL]2.2.1[NUL][NUL][NUL]Fluent Bit
// [NUL]1.7.0[NUL]\x1b[1m[NUL]%sFluent Bit (versions 1.7.0-dev-3 through 1.7.0-dev-9 and 1.7.0-rc4 through 1.7.0-rc8)
`\x00(?P<version>[0-9]+\.[0-9]+\.[0-9]+)\x00(\x1b\[1m\x00|\x00\x00)?(%s)?Fluent`,
// [NUL][NUL]1.3.10[NUL][NUL]Fluent Bit v%s
`\x00(\x00)?(?P<version>[0-9]+\.[0-9]+\.[0-9]+)\x00(\x1b\[1m\x00|\x00|\x00\x00)?(%s)?Fluent`,
),
Package: "fluent-bit",
PURL: mustPURL("pkg:github/fluent/fluent-bit@version"),
@@ -71,6 +71,14 @@ from-images:
paths:
- /usr/local/lib/erlang/erts-15.0/bin/beam.smp
- name: erlang
version: 26.1.2
images:
- ref: hexpm/elixir:1.15.6-erlang-26.1.2-alpine-3.18.4@sha256:e84f6667f4e3d2fe00fcdb9d1c2cdd614733296c39b20b28b092743f1161bc11
platform: linux/arm64
paths:
- /usr/local/lib/erlang/erts-14.1.1/bin/beam.smp
- version: 9.3.8
images:
- ref: swipl:9.3.8@sha256:6a15e6a03afe943228924a5502ba763e653ff28d9b3391e2b3e1fc3e991f37d4
@@ -701,6 +709,13 @@ from-images:
paths:
- /fluent-bit/bin/fluent-bit
- version: 1.3.10
images:
- ref: fluent/fluent-bit:1.3.10@sha256:98b32ef90c3ee3f6981b2cf31b70dc7d6104c9800b7287d4758318e0c19f96a9
platform: linux/arm
paths:
- /fluent-bit/bin/fluent-bit
- name: openssl
version: 1.1.1w
images:
@@ -3,6 +3,8 @@ package bitnami
import (
"testing"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
@@ -493,6 +495,28 @@ func TestBitnamiCataloger(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
pkgtest.NewCatalogTester().
FromDirectory(t, tt.fixture).
WithCompareOptions(cmpopts.IgnoreFields(pkg.Package{}, "SupplementalData")).
ExpectsAssertion(
func(t *testing.T, pkgs []pkg.Package, relationships []artifact.Relationship) {
for _, p := range pkgs {
// assert there are supplemental data as artifact.ID and ID() matches
assert.NotEmpty(t, p.SupplementalData)
var id artifact.ID
for _, data := range p.SupplementalData {
switch d := data.(type) {
case artifact.ID:
id = d
break
case artifact.Identifiable:
id = d.ID()
break
}
}
assert.NotEmpty(t, id)
assert.Equal(t, p.ID(), id)
}
},
).
Expects(tt.wantPkgs, tt.wantRelationships).
WithErrorAssertion(tt.wantErr).
TestCataloger(t, NewCataloger())
+11 -12
View File
@@ -7,7 +7,6 @@ import (
"regexp"
"sort"
"github.com/Masterminds/semver"
"gopkg.in/yaml.v3"
"github.com/anchore/syft/internal/log"
@@ -136,6 +135,13 @@ func (psl *pubspecLock) getSdkVersion(sdk string) (string, error) {
return parseMinimumSdkVersion(constraint)
}
// semverRegex is a regex pattern that allows for both two-part (major.minor) and three-part (major.minor.patch) versions.
// additionally allows for:
// 1. start with either "^" or ">=" (Dart SDK constraints only use those two)
// 2. followed by a valid semantic version (which may be two or three components)
// 3. followed by a space (if there's a range) or end of string
var semverRegex = regexp.MustCompile(`^(\^|>=)(?P<version>(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)(?:\.(?:0|[1-9]\d*))?(?:-[0-9A-Za-z\-\.]+)?(?:\+[0-9A-Za-z\-\.]+)?)( |$)`)
// Parse a given version range constraint and return its lowest supported version.
//
// This is intended for packages that are part of an SDK (e.g. Flutter) and don't
@@ -153,21 +159,14 @@ func (psl *pubspecLock) getSdkVersion(sdk string) (string, error) {
// see https://dart.dev/tools/pub/dependencies#version-constraints for the
// constraint format used in Dart SDK defintions.
func parseMinimumSdkVersion(constraint string) (string, error) {
// Match strings that
// 1. start with either "^" or ">=" (Dart SDK constraints only use those two)
// 2. followed by a valid semantic version, matched as "version" named subexpression
// 3. followed by a space (if there's a range) or end of string (if there's only a lower boundary)
// |---1--||------------------2------------------||-3-|
re := regexp.MustCompile(`^(\^|>=)(?P<version>` + semver.SemVerRegex + `)( |$)`)
if !re.MatchString(constraint) {
if !semverRegex.MatchString(constraint) {
return "", fmt.Errorf("unsupported or invalid constraint '%s'", constraint)
}
// Read "version" subexpression (see 2. above) into version variable
// Read "version" subexpression into version variable
var version []byte
matchIndex := re.FindStringSubmatchIndex(constraint)
version = re.ExpandString(version, "$version", constraint, matchIndex)
matchIndex := semverRegex.FindStringSubmatchIndex(constraint)
version = semverRegex.ExpandString(version, "$version", constraint, matchIndex)
return string(version), nil
}
@@ -235,9 +235,11 @@ func Test_sdkVersionParser_valid(t *testing.T) {
}
for constraint, expected := range patterns {
version, err = parseMinimumSdkVersion(constraint)
assert.NoError(t, err)
assert.Equalf(t, expected, version, "constraint '%s", constraint)
t.Run(constraint, func(t *testing.T) {
version, err = parseMinimumSdkVersion(constraint)
assert.NoError(t, err)
assert.Equal(t, expected, version)
})
}
}
@@ -76,6 +76,9 @@
"github.com/libp2p/go-libp2p": [
"cpe:2.3:a:protocol:libp2p:*:*:*:*:*:go:*:*"
],
"github.com/mholt/archiver": [
"cpe:2.3:a:mholt:archiver:*:*:*:*:*:go:*:*"
],
"github.com/mindersec/minder": [
"cpe:2.3:a:lfprojects:minder:*:*:*:*:*:go:*:*"
],
@@ -238,6 +241,9 @@
"artifact-repository-parameter": [
"cpe:2.3:a:jenkins:artifact_repository_parameter:*:*:*:*:*:jenkins:*:*"
],
"asakusa-satellite-plugin": [
"cpe:2.3:a:jenkins:asakusasatellite:*:*:*:*:*:jenkins:*:*"
],
"assembla": [
"cpe:2.3:a:jenkins:assembla:*:*:*:*:*:jenkins:*:*"
],
@@ -335,6 +341,9 @@
"batch-task": [
"cpe:2.3:a:jenkins:batch_task:*:*:*:*:*:jenkins:*:*"
],
"bazaar": [
"cpe:2.3:a:jenkins:bazaar:*:*:*:*:*:jenkins:*:*"
],
"beaker-builder": [
"cpe:2.3:a:jenkins:beaker_builder:*:*:*:*:*:jenkins:*:*"
],
@@ -435,6 +444,9 @@
"checkstyle": [
"cpe:2.3:a:jenkins:checkstyle:*:*:*:*:*:jenkins:*:*"
],
"chef-identity": [
"cpe:2.3:a:jenkins:chef_identity:*:*:*:*:*:jenkins:*:*"
],
"chosen-views-tabbar": [
"cpe:2.3:a:jenkins:chosen-views-tabbar:*:*:*:*:*:jenkins:*:*"
],
@@ -3429,6 +3441,12 @@
"html-janitor": [
"cpe:2.3:a:html-janitor_project:html-janitor:*:*:*:*:*:node.js:*:*"
],
"html-minifier": [
"cpe:2.3:a:kangax:html-minifier:*:*:*:*:*:node.js:*:*"
],
"html-minifier-terser": [
"cpe:2.3:a:terser:html-minifier-terser:*:*:*:*:*:node.js:*:*"
],
"html-pages": [
"cpe:2.3:a:html-pages_project:html-pages:*:*:*:*:*:node.js:*:*"
],
@@ -5579,6 +5597,9 @@
"MechanicalSoup": [
"cpe:2.3:a:mechanicalsoup_project:mechanicalsoup:*:*:*:*:*:python:*:*"
],
"Mezzanine": [
"cpe:2.3:a:jupo:mezzanine:*:*:*:*:*:*:*:*"
],
"Pillow": [
"cpe:2.3:a:python:pillow:*:*:*:*:*:*:*:*"
],
@@ -7458,6 +7479,9 @@
"adminpad": [
"cpe:2.3:a:adminpad_project:adminpad:*:*:*:*:*:wordpress:*:*"
],
"admire-extra": [
"cpe:2.3:a:sktthemes:admire_extra:*:*:*:*:*:wordpress:*:*"
],
"adrotate": [
"cpe:2.3:a:adrotate_banner_manager_project:adrotate_banner_manager:*:*:*:*:*:wordpress:*:*",
"cpe:2.3:a:adrotate_project:adrotate:*:*:*:*:*:wordpress:*:*",
@@ -7665,6 +7689,9 @@
"akismet-privacy-policies": [
"cpe:2.3:a:inpsyde:akismet_privacy_policies:*:*:*:*:*:wordpress:*:*"
],
"aklamator-infeed": [
"cpe:2.3:a:aklamator:infeed:*:*:*:*:*:wordpress:*:*"
],
"albo-pretorio-on-line": [
"cpe:2.3:a:albo_pretorio_on_line_project:albo_pretorio_on_line:*:*:*:*:*:wordpress:*:*"
],
@@ -7743,6 +7770,9 @@
"altos-connect": [
"cpe:2.3:a:altosresearch:altos-connect:*:*:*:*:*:wordpress:*:*"
],
"alttext-ai": [
"cpe:2.3:a:alttext:alt_text_ai:*:*:*:*:*:wordpress:*:*"
],
"am-hili-affiliate-manager-for-publishers": [
"cpe:2.3:a:am-hili_project:am-hili:*:*:*:*:*:wordpress:*:*"
],
@@ -9025,6 +9055,9 @@
"cf7-invisible-recaptcha": [
"cpe:2.3:a:vsourz:cf7_invisible_recaptcha:*:*:*:*:*:wordpress:*:*"
],
"cf7-message-filter": [
"cpe:2.3:a:kofimokome:message_filter_for_contact_form_7:*:*:*:*:*:wordpress:*:*"
],
"cf7-redirect-thank-you-page": [
"cpe:2.3:a:wpplugin:contact_form_7_redirect_\\\u0026_thank_you_page:*:*:*:*:*:wordpress:*:*"
],
@@ -9076,6 +9109,9 @@
"chat-bee": [
"cpe:2.3:a:chat_bee_project:chat_bee:*:*:*:*:*:wordpress:*:*"
],
"chat-bubble": [
"cpe:2.3:a:bluecoral:chat_bubble:*:*:*:*:*:wordpress:*:*"
],
"chatbot": [
"cpe:2.3:a:quantumcloud:ai_chatbot:*:*:*:*:*:wordpress:*:*"
],
@@ -9208,6 +9244,12 @@
"cluevo-lms": [
"cpe:2.3:a:cluevo:learning_management_system:*:*:*:*:*:wordpress:*:*"
],
"cm-ad-changer": [
"cpe:2.3:a:cminds:cm_ad_changer:*:*:*:*:*:wordpress:*:*"
],
"cm-answers": [
"cpe:2.3:a:cminds:cm_answers:*:*:*:*:*:wordpress:*:*"
],
"cm-download-manager": [
"cpe:2.3:a:cminds:cm_download_manager:*:*:*:*:*:wordpress:*:*"
],
@@ -9501,7 +9543,7 @@
"cpe:2.3:a:wpdevart:contact_form_builder:*:*:*:*:*:wordpress:*:*"
],
"contact-list": [
"cpe:2.3:a:contactlistpro:contact_list:*:*:*:*:*:wordpress:*:*"
"cpe:2.3:a:tammersoft:contact_list:*:*:*:*:*:wordpress:*:*"
],
"contact-us-page-contact-people": [
"cpe:2.3:a:a3rev:contact_us_page_-_contact_people:*:*:*:*:*:wordpress:*:*"
@@ -9600,6 +9642,9 @@
"cookie-notice-and-consent-banner": [
"cpe:2.3:a:gdprinfo:cookie_notice_\\\u0026_consent_banner_for_gdpr_\\\u0026_ccpa_compliance:*:*:*:*:*:wordpress:*:*"
],
"cookie-notice-bar": [
"cpe:2.3:a:dcurasi:cookie_notice_bar:*:*:*:*:*:wordpress:*:*"
],
"cookie-notice-consent": [
"cpe:2.3:a:christophrado:cookie_notice_\\\u0026_consent:*:*:*:*:*:wordpress:*:*"
],
@@ -10069,6 +10114,9 @@
"dearpdf-lite": [
"cpe:2.3:a:dearhive:dearpdf:*:*:*:*:*:wordpress:*:*"
],
"debounce-io-email-validator": [
"cpe:2.3:a:debounce:email_validator:*:*:*:*:*:wordpress:*:*"
],
"debrandify": [
"cpe:2.3:a:tahoe:debrandify:*:*:*:*:*:wordpress:*:*"
],
@@ -10235,6 +10283,7 @@
"cpe:2.3:a:apasionados:dofollow_case_by_case:*:*:*:*:*:wordpress:*:*"
],
"dokan-lite": [
"cpe:2.3:a:dokan:dokan:*:*:*:*:lite:wordpress:*:*",
"cpe:2.3:a:wedevs:dokan:*:*:*:*:*:wordpress:*:*"
],
"dokan-pro": [
@@ -10979,6 +11028,9 @@
"everest-faq-manager-lite": [
"cpe:2.3:a:accesspressthemes:everest_faq_manager_lite:*:*:*:*:*:wordpress:*:*"
],
"everest-forms": [
"cpe:2.3:a:wpeverest:everest_forms:*:*:*:*:*:wordpress:*:*"
],
"everest-gallery-lite": [
"cpe:2.3:a:accesspressthemes:everest_gallery_lite:*:*:*:*:*:wordpress:*:*"
],
@@ -11640,7 +11692,7 @@
"cpe:2.3:a:gd-mylist_project:gd-mylist:*:*:*:*:*:wordpress:*:*"
],
"gd-rating-system": [
"cpe:2.3:a:gdragon:gd_rating_system:*:*:*:*:*:wordpress:*:*"
"cpe:2.3:a:dev4press:gd_rating_system:*:*:*:*:free:wordpress:*:*"
],
"gd-security-headers": [
"cpe:2.3:a:dev4press:gd_security_headers:*:*:*:*:*:wordpress:*:*"
@@ -12023,6 +12075,9 @@
"gutenberg": [
"cpe:2.3:a:wordpress:gutenberg:*:*:*:*:*:wordpress:*:*"
],
"gutenkit-blocks-addon": [
"cpe:2.3:a:wpmet:gutenkit:*:*:*:*:*:wordpress:*:*"
],
"gutenslider": [
"cpe:2.3:a:gutenslider:gutenslider:*:*:*:*:*:wordpress:*:*"
],
@@ -12030,7 +12085,7 @@
"cpe:2.3:a:gutentor:gutentor:*:*:*:*:*:wordpress:*:*"
],
"gutenverse": [
"cpe:2.3:a:jegstudio:gutenverse:*:*:*:*:*:wordpress:*:*"
"cpe:2.3:a:jegstudio:gutenverse:*:*:*:*:free:wordpress:*:*"
],
"gwa-autoresponder": [
"cpe:2.3:a:\\[gwa\\]_autoresponder_project:\\[gwa\\]_autoresponder:*:*:*:*:*:wordpress:*:*"
@@ -12625,6 +12680,9 @@
"ip-address-blocker": [
"cpe:2.3:a:lionscripts:ip_blocker_lite:*:*:*:*:*:wordpress:*:*"
],
"ip-based-login": [
"cpe:2.3:a:brijeshk89:ip_based_login:*:*:*:*:*:wordpress:*:*"
],
"ip-blacklist-cloud": [
"cpe:2.3:a:ip_blacklist_cloud_project:ip_blacklist_cloud:*:*:*:*:*:wordpress:*:*"
],
@@ -12720,6 +12778,9 @@
"jobboardwp": [
"cpe:2.3:a:ultimatemember:jobboardwp:*:*:*:*:*:wordpress:*:*"
],
"jobcareer": [
"cpe:2.3:a:chimpgroup:jobcareer:*:*:*:*:*:wordpress:*:*"
],
"jobwp": [
"cpe:2.3:a:hmplugin:jobwp:*:*:*:*:*:wordpress:*:*",
"cpe:2.3:a:hmplugin:wordpress_job_board_and_recruitment_plugin_-_jobwp:*:*:*:*:*:wordpress:*:*"
@@ -12839,6 +12900,9 @@
"kimili-flash-embed": [
"cpe:2.3:a:kimili:kimili_flash_embed:*:*:*:*:*:wordpress:*:*"
],
"king-ie": [
"cpe:2.3:a:kingblack:king_ie:*:*:*:*:*:wordpress:*:*"
],
"kitestudio-core": [
"cpe:2.3:a:kitestudio:core_plugin_for_kitestudio_themes:*:*:*:*:*:wordpress:*:*"
],
@@ -12854,6 +12918,9 @@
"kk-star-ratings": [
"cpe:2.3:a:kamalkhan:kk_star_ratings:*:*:*:*:*:wordpress:*:*"
],
"klarna-checkout-for-woocommerce": [
"cpe:2.3:a:klarna:klarna_checkout_for_woocommerce:*:*:*:*:*:wordpress:*:*"
],
"klarna-payments-for-woocommerce": [
"cpe:2.3:a:klarna:klarna_for_woocommerce:*:*:*:*:*:wordpress:*:*"
],
@@ -13112,12 +13179,18 @@
"liquid-speech-balloon": [
"cpe:2.3:a:lqd:liquid_speech_balloon:*:*:*:*:*:wordpress:*:*"
],
"lis-video-gallery": [
"cpe:2.3:a:lis:video_gallery:*:*:*:*:*:wordpress:*:*"
],
"list-all-posts-by-authors-nested-categories-and-titles": [
"cpe:2.3:a:marzocca:list_all_posts_by_authors_nested_categories_and_titles:*:*:*:*:*:wordpress:*:*"
],
"list-category-posts": [
"cpe:2.3:a:fernandobriano:list_category_posts:*:*:*:*:*:wordpress:*:*"
],
"list-last-changes": [
"cpe:2.3:a:rolandbaer:list_last_changes:*:*:*:*:*:wordpress:*:*"
],
"listamester": [
"cpe:2.3:a:listamester:listamester:*:*:*:*:*:wordpress:*:*"
],
@@ -13786,7 +13859,7 @@
"cpe:2.3:a:sonaar:mp3_audio_player_for_music\\,_radio_\\\u0026_podcast:*:*:*:*:*:wordpress:*:*"
],
"mpl-publisher": [
"cpe:2.3:a:mpl-publisher_project:mpl-publisher:*:*:*:*:*:wordpress:*:*"
"cpe:2.3:a:mpl-publisher:mpl-publisher:*:*:*:*:*:wordpress:*:*"
],
"mpoperationlogs": [
"cpe:2.3:a:mrpeng:mpoperationlogs:*:*:*:*:*:wordpress:*:*"
@@ -13944,6 +14017,9 @@
"n-media-woocommerce-checkout-fields": [
"cpe:2.3:a:najeebmedia:woocommerce_checkout_field_manager:*:*:*:*:*:wordpress:*:*"
],
"n-media-wp-simple-quiz": [
"cpe:2.3:a:najeebmedia:easy_quiz_maker:*:*:*:*:*:wordpress:*:*"
],
"namaste-lms": [
"cpe:2.3:a:kibokolabs:namaste\\!_lms:*:*:*:*:*:wordpress:*:*"
],
@@ -14116,6 +14192,7 @@
"cpe:2.3:a:nitinmaurya:wordpress_visitors:*:*:*:*:*:wordpress:*:*"
],
"nmedia-user-file-uploader": [
"cpe:2.3:a:najeebmedia:frontend_file_manager:*:*:*:*:*:wordpress:*:*",
"cpe:2.3:a:najeebmedia:frontend_file_manager_plugin:*:*:*:*:*:wordpress:*:*"
],
"nmr-strava-activities": [
@@ -14124,6 +14201,9 @@
"no-future-posts": [
"cpe:2.3:a:no_future_posts_project:no_future_posts:*:*:*:*:*:wordpress:*:*"
],
"no-update-nag": [
"cpe:2.3:a:coffee2code:no_update_nag:*:*:*:*:*:wordpress:*:*"
],
"nofollow": [
"cpe:2.3:a:ultimate_nofollow_project:ultimate_nofollow:*:*:*:*:*:wordpress:*:*"
],
@@ -14311,6 +14391,9 @@
"optinmonster": [
"cpe:2.3:a:optinmonster:optinmonster:*:*:*:*:*:wordpress:*:*"
],
"option-editor": [
"cpe:2.3:a:backie:option_editor:*:*:*:*:*:*:*:*"
],
"option-tree": [
"cpe:2.3:a:optiontree_project:optiontree:*:*:*:*:*:wordpress:*:*"
],
@@ -14754,7 +14837,7 @@
"cpe:2.3:a:poeditor:poeditor:*:*:*:*:*:wordpress:*:*"
],
"poll-maker": [
"cpe:2.3:a:ays-pro:poll_maker:*:*:*:*:*:wordpress:*:*"
"cpe:2.3:a:ays-pro:poll_maker:*:*:*:*:free:wordpress:*:*"
],
"poll-wp": [
"cpe:2.3:a:total-soft:responsive_poll:*:*:*:*:*:wordpress:*:*",
@@ -14763,6 +14846,9 @@
"polldaddy": [
"cpe:2.3:a:automattic:crowdsignal_dashboard:*:*:*:*:*:wordpress:*:*"
],
"pollin": [
"cpe:2.3:a:bin-co:pollin:*:*:*:*:*:wordpress:*:*"
],
"polls-widget": [
"cpe:2.3:a:wpdevart:poll\\,_survey\\,_questionnaire_and_voting_system:*:*:*:*:*:wordpress:*:*"
],
@@ -15311,6 +15397,9 @@
"radio-forge": [
"cpe:2.3:a:radioforge:radio_forge_muses_player_with_skins:*:*:*:*:*:wordpress:*:*"
],
"radio-player": [
"cpe:2.3:a:softlabbd:radio_player:*:*:*:*:*:wordpress:*:*"
],
"rafflepress": [
"cpe:2.3:a:seedprod:rafflepress:*:*:*:*:*:wordpress:*:*"
],
@@ -15338,6 +15427,9 @@
"rating-widget": [
"cpe:2.3:a:rating-widget:ratingwidget:*:*:*:*:*:wordpress:*:*"
],
"ratings-shorttags": [
"cpe:2.3:a:moc:review_ratings:*:*:*:*:*:wordpress:*:*"
],
"ravpage": [
"cpe:2.3:a:matiskiba:ravpage:*:*:*:*:*:wordpress:*:*",
"cpe:2.3:a:ravpage_project:ravpage:*:*:*:*:*:wordpress:*:*"
@@ -15420,6 +15512,9 @@
"reciply": [
"cpe:2.3:a:reciply_project:reciply:*:*:*:*:*:wordpress:*:*"
],
"recover-wc-abandoned-cart": [
"cpe:2.3:a:sktthemes:recover_abandoned_cart_for_woocommerce:*:*:*:*:*:wordpress:*:*"
],
"recurring-donation": [
"cpe:2.3:a:wp-ecommerce:recurring_paypal_donations:*:*:*:*:*:wordpress:*:*"
],
@@ -15674,6 +15769,9 @@
"role-scoper": [
"cpe:2.3:a:role_scoper_project:role_scoper:*:*:*:*:*:wordpress:*:*"
],
"rometheme-for-elementor": [
"cpe:2.3:a:rometheme:romethemekit_for_elementor:*:*:*:*:*:wordpress:*:*"
],
"roomcloud": [
"cpe:2.3:a:roomcloud:roomcloud:*:*:*:*:*:wordpress:*:*"
],
@@ -15730,6 +15828,9 @@
"ruven-toolkit": [
"cpe:2.3:a:ruven-toolkit_project:ruven-toolkit:*:*:*:*:*:wordpress:*:*"
],
"s2member": [
"cpe:2.3:a:wpsharks:s2member:*:*:*:*:pro:wordpress:*:*"
],
"s3bubble-amazon-s3-audio-streaming": [
"cpe:2.3:a:s3bubble:s3bubble-amazon-s3-audio-streaming:*:*:*:*:*:wordpress:*:*"
],
@@ -16004,6 +16105,9 @@
"shared-files": [
"cpe:2.3:a:tammersoft:shared_files:*:*:*:*:*:wordpress:*:*"
],
"shariff": [
"cpe:2.3:a:datenverwurstungszentrale:shariff_wrapper:*:*:*:*:*:wordpress:*:*"
],
"sheets-to-wp-table-live-sync": [
"cpe:2.3:a:wppool:sheets_to_wp_table_live_sync:*:*:*:*:*:wordpress:*:*"
],
@@ -16052,6 +16156,9 @@
"shortcode-variables": [
"cpe:2.3:a:yeken:snippet_shortcodes:*:*:*:*:*:wordpress:*:*"
],
"shortcodes-anywhere": [
"cpe:2.3:a:happyplugins:shortcodes_anywhere:*:*:*:*:*:wordpress:*:*"
],
"shortcodes-finder": [
"cpe:2.3:a:scribit:shortcodes_finder:*:*:*:*:*:wordpress:*:*"
],
@@ -16149,6 +16256,9 @@
"simple-bitcoin-faucets": [
"cpe:2.3:a:simple_bitcoin_faucets_project:simple_bitcoin_faucets:*:*:*:*:*:wordpress:*:*"
],
"simple-calendar-for-elementor": [
"cpe:2.3:a:migaweb:simple_calendar_for_elementor:*:*:*:*:*:wordpress:*:*"
],
"simple-cloudflare-turnstile": [
"cpe:2.3:a:replywp:simple_cloudfare_turnstile:*:*:*:*:*:wordpress:*:*"
],
@@ -16465,6 +16575,9 @@
"skt-addons-for-elementor": [
"cpe:2.3:a:sktthemes:skt_addons_for_elementor:*:*:*:*:*:wordpress:*:*"
],
"skt-blocks": [
"cpe:2.3:a:sktthemes:skt_blocks:*:*:*:*:*:wordpress:*:*"
],
"skt-templates": [
"cpe:2.3:a:sktthemes:skt_templates:*:*:*:*:*:wordpress:*:*"
],
@@ -16668,6 +16781,9 @@
"social-login-bws": [
"cpe:2.3:a:bestwebsoft:social_login:*:*:*:*:*:wordpress:*:*"
],
"social-media-builder": [
"cpe:2.3:a:sygnoos:social_media_share_buttons:*:*:*:*:*:wordpress:*:*"
],
"social-media-feather": [
"cpe:2.3:a:sharethis:social_media_feather:*:*:*:*:*:wordpress:*:*"
],
@@ -16702,6 +16818,9 @@
"social-warfare": [
"cpe:2.3:a:warfareplugins:social_warfare:*:*:*:*:*:wordpress:*:*"
],
"social-web-suite": [
"cpe:2.3:a:hypestudio:social_web_suite:*:*:*:*:*:wordpress:*:*"
],
"sodahead-polls": [
"cpe:2.3:a:sodahead:sodahead_polls:*:*:*:*:*:wordpress:*:*"
],
@@ -16753,6 +16872,9 @@
"sparkle-demo-importer": [
"cpe:2.3:a:wpneuron:sparkle_demo_importer:*:*:*:*:*:wordpress:*:*"
],
"special-feed-items": [
"cpe:2.3:a:moc:special_feed_items:*:*:*:*:*:wordpress:*:*"
],
"speed-booster-pack": [
"cpe:2.3:a:optimocha:speed_booster_pack:*:*:*:*:*:wordpress:*:*"
],
@@ -17191,6 +17313,9 @@
"testimonials": [
"cpe:2.3:a:testimonials_project:testimonials:*:*:*:*:*:wordpress:*:*"
],
"testimonials-carousel-elementor": [
"cpe:2.3:a:uapp:testimonial_carousel_for_elementor:*:*:*:*:*:wordpress:*:*"
],
"testimonials-widget": [
"cpe:2.3:a:axelerant:testimonials_widget:*:*:*:*:*:wordpress:*:*"
],
@@ -17806,7 +17931,8 @@
"cpe:2.3:a:userprivatefiles:wordpress_file_sharing_plugin:*:*:*:*:*:wordpress:*:*"
],
"user-registration": [
"cpe:2.3:a:wpeverest:user_registration:*:*:*:*:*:wordpress:*:*"
"cpe:2.3:a:wpeverest:user_registration:*:*:*:*:*:wordpress:*:*",
"cpe:2.3:a:wpeverest:user_registration_\\\u0026_membership:*:*:*:*:free:wordpress:*:*"
],
"user-role": [
"cpe:2.3:a:bestwebsoft:user_role:*:*:*:*:*:wordpress:*:*"
@@ -17850,6 +17976,9 @@
"utubevideo-gallery": [
"cpe:2.3:a:utubevideo_gallery_project:utubevideo_gallery:*:*:*:*:*:wordpress:*:*"
],
"v-form": [
"cpe:2.3:a:vikasratudi:lifetime_free_drag_\\\u0026_drop_contact_form_builder:*:*:*:*:*:wordpress:*:*"
],
"vaultpress": [
"cpe:2.3:a:automattic:vaultpress:*:*:*:*:*:wordpress:*:*"
],
@@ -18178,6 +18307,9 @@
"website-monetization-by-magenet": [
"cpe:2.3:a:magenet:website_monetization:*:*:*:*:*:wordpress:*:*"
],
"webtexttool": [
"cpe:2.3:a:textmetrics:textmetrics:*:*:*:*:*:wordpress:*:*"
],
"webwinkelkeur": [
"cpe:2.3:a:webwinkelkeur_project:webwinkelkeur:*:*:*:*:*:wordpress:*:*"
],
@@ -18636,6 +18768,9 @@
"word-balloon": [
"cpe:2.3:a:back2nature:word_balloon:*:*:*:*:*:wordpress:*:*"
],
"word-replacer-ultra": [
"cpe:2.3:a:charlestsmith:word_replacer_pro:*:*:*:*:*:wordpress:*:*"
],
"wordapp": [
"cpe:2.3:a:wordapp:wordapp:*:*:*:*:*:wordpress:*:*"
],
@@ -19595,6 +19730,9 @@
"wp-post-columns": [
"cpe:2.3:a:wp_post_columns_project:wp_post_columns:*:*:*:*:*:wordpress:*:*"
],
"wp-post-frontend": [
"cpe:2.3:a:najeebmedia:post_front-end_form:*:*:*:*:*:wordpress:*:*"
],
"wp-post-page-clone": [
"cpe:2.3:a:wp_post_page_clone_project:wp_post_page_clone:*:*:*:*:*:wordpress:*:*"
],
@@ -19655,6 +19793,9 @@
"wp-recipe-maker": [
"cpe:2.3:a:bootstrapped:wp_recipe_maker:*:*:*:*:free:wordpress:*:*"
],
"wp-registration": [
"cpe:2.3:a:najeebmedia:simple_user_registration:*:*:*:*:*:wordpress:*:*"
],
"wp-remote-site-search": [
"cpe:2.3:a:brainstormforce:wp_remote_site_search:*:*:*:*:*:wordpress:*:*"
],
@@ -20539,6 +20680,9 @@
"zerobounce": [
"cpe:2.3:a:zerobounce:zerobounce_email_verification_\\\u0026_validation:*:*:*:*:*:wordpress:*:*"
],
"ziggeo": [
"cpe:2.3:a:oliverfriedmann:ziggeo:*:*:*:*:*:wordpress:*:*"
],
"zip-attachments": [
"cpe:2.3:a:zip_attachments_project:zip_attachments:*:*:*:*:*:wordpress:*:*"
],
+33 -80
View File
@@ -1,104 +1,57 @@
/*
Package nix provides a concrete Cataloger implementation for packages within the Nix packaging ecosystem.
*/
package nix
import (
"context"
"fmt"
"github.com/bmatcuk/doublestar/v4"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
const catalogerName = "nix-store-cataloger"
// storeCataloger finds package outputs installed in the Nix store location (/nix/store/*).
type storeCataloger struct{}
func NewStoreCataloger() pkg.Cataloger {
return &storeCataloger{}
type Config struct {
CaptureOwnedFiles bool `json:"capture-owned-files" yaml:"capture-owned-files" mapstructure:"capture-owned-files"`
}
func (c *storeCataloger) Name() string {
return catalogerName
func (c Config) WithCaptureOwnedFiles(set bool) Config {
c.CaptureOwnedFiles = set
return c
}
func (c *storeCataloger) Catalog(ctx context.Context, resolver file.Resolver) ([]pkg.Package, []artifact.Relationship, error) {
// we want to search for only directories, which isn't possible via the stereoscope API, so we need to apply the glob manually on all returned paths
var pkgs []pkg.Package
var filesByPath = make(map[string]*file.LocationSet)
ctx, cancel := context.WithCancel(ctx)
defer cancel()
for location := range resolver.AllLocations(ctx) {
matchesStorePath, err := doublestar.Match("**/nix/store/*", location.RealPath)
if err != nil {
return nil, nil, fmt.Errorf("failed to match nix store path: %w", err)
}
parentStorePath := findParentNixStorePath(location.RealPath)
if parentStorePath != "" {
if _, ok := filesByPath[parentStorePath]; !ok {
s := file.NewLocationSet()
filesByPath[parentStorePath] = &s
}
filesByPath[parentStorePath].Add(location)
}
if !matchesStorePath {
continue
}
storePath := parseNixStorePath(location.RealPath)
if storePath == nil || !storePath.isValidPackage() {
continue
}
p := newNixStorePackage(*storePath, location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation))
pkgs = append(pkgs, p)
func DefaultConfig() Config {
return Config{
CaptureOwnedFiles: false,
}
// add file sets to packages
for i := range pkgs {
p := &pkgs[i]
locations := p.Locations.ToSlice()
if len(locations) == 0 {
log.WithFields("package", p.Name).Debug("nix package has no evidence locations associated")
continue
}
parentStorePath := locations[0].RealPath
files, ok := filesByPath[parentStorePath]
if !ok {
log.WithFields("path", parentStorePath, "nix-store-path", parentStorePath).Debug("found a nix store file for a non-existent package")
continue
}
appendFiles(p, files.ToSlice()...)
}
return pkgs, nil, nil
}
func appendFiles(p *pkg.Package, location ...file.Location) {
metadata, ok := p.Metadata.(pkg.NixStoreEntry)
if !ok {
log.WithFields("package", p.Name).Debug("nix package metadata missing")
return
// cataloger finds package outputs installed in the Nix store location (/nix/store/*) or in the internal nix database (/nix/var/nix/db/db.sqlite).
type cataloger struct {
dbParser dbCataloger
storeCataloger storeCataloger
}
func NewCataloger(cfg Config) pkg.Cataloger {
name := "nix-cataloger"
return cataloger{
dbParser: newDBCataloger(cfg, name),
storeCataloger: newStoreCataloger(cfg, name),
}
}
func (c cataloger) Name() string {
return c.dbParser.catalogerName
}
for _, l := range location {
metadata.Files = append(metadata.Files, l.RealPath)
func (c cataloger) Catalog(ctx context.Context, resolver file.Resolver) ([]pkg.Package, []artifact.Relationship, error) {
// always try the DB cataloger first (based off of information recorded by actions taken by nix tooling)
pkgs, rels, err := c.dbParser.catalog(resolver)
if err != nil {
return nil, nil, fmt.Errorf("failed to catalog nix packages from database: %w", err)
}
if metadata.Files == nil {
// note: we always have an allocated collection for output
metadata.Files = []string{}
if len(pkgs) > 0 {
return pkgs, rels, nil
}
p.Metadata = metadata
p.SetID()
// there are no results from the DB cataloger, then use the store path cataloger (not as accurate / detailed in information)
return c.storeCataloger.Catalog(ctx, resolver)
}
+220 -17
View File
@@ -3,13 +3,145 @@ package nix
import (
"testing"
"github.com/google/go-cmp/cmp"
"github.com/scylladb/go-set/strset"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
_ "modernc.org/sqlite"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
func TestCataloger_Catalog(t *testing.T) {
func TestCataloger_Image(t *testing.T) {
tests := []struct {
fixture string
wantPkgs []string
wantRel []string
}{
{
// $ nix-store -q --tree $(which jq)
//
// /nix/store/nzwfgsp28vgxv7n2gl5fxqkca9awh4dz-jq-1.6-bin3.4
// ├───/nix/store/02mqs1by2vab9yzw0qc4j7463w78p3ps-glibc-2.37-8
// │ ├───/nix/store/cw8fpl8r1x9rmaqj55fwbfnnrgw7b40k-libidn2-2.3.4
// │ │ ├───/nix/store/h1ysk4vvw48winwmh38rvnsj0dlsz7c1-libunistring-1.1
// │ │ │ └───/nix/store/h1ysk4vvw48winwmh38rvnsj0dlsz7c1-libunistring-1.1 [...]
// │ │ └───/nix/store/cw8fpl8r1x9rmaqj55fwbfnnrgw7b40k-libidn2-2.3.4 [...]
// │ ├───/nix/store/fmz62d844wf4blb11k21f4m0q6n6hdfp-xgcc-12.3.0-libgcc
// │ └───/nix/store/02mqs1by2vab9yzw0qc4j7463w78p3ps-glibc-2.37-8 [...]
// ├───/nix/store/mzj90j6m3c3a1vv8j9pl920f98i2yz9q-oniguruma-6.9.8-lib
// │ ├───/nix/store/02mqs1by2vab9yzw0qc4j7463w78p3ps-glibc-2.37-8 [...]
// │ └───/nix/store/mzj90j6m3c3a1vv8j9pl920f98i2yz9q-oniguruma-6.9.8-lib [...]
// └───/nix/store/1x3s2v9wc9m302cspfqcn2iwar0b5w99-jq-1.6-lib
// ├───/nix/store/02mqs1by2vab9yzw0qc4j7463w78p3ps-glibc-2.37-8 [...]
// ├───/nix/store/mzj90j6m3c3a1vv8j9pl920f98i2yz9q-oniguruma-6.9.8-lib [...]
// └───/nix/store/1x3s2v9wc9m302cspfqcn2iwar0b5w99-jq-1.6-lib [...]
fixture: "image-nixos-jq-pkg-db",
wantPkgs: []string{
"glibc @ 2.37-8 (/nix/var/nix/db/db.sqlite)",
"jq @ 1.6 (/nix/var/nix/db/db.sqlite)", // lib output
"jq @ 1.6 (/nix/var/nix/db/db.sqlite)", // bin output
"libidn2 @ 2.3.4 (/nix/var/nix/db/db.sqlite)",
"libunistring @ 1.1 (/nix/var/nix/db/db.sqlite)",
"oniguruma @ 6.9.8 (/nix/var/nix/db/db.sqlite)",
"xgcc @ 12.3.0 (/nix/var/nix/db/db.sqlite)",
},
wantRel: []string{
// used the DB cataloger, thus has a complete dependency graph
"glibc @ 2.37-8 (/nix/var/nix/db/db.sqlite) [dependency-of] jq @ 1.6 (/nix/var/nix/db/db.sqlite)", // jq bin output
"glibc @ 2.37-8 (/nix/var/nix/db/db.sqlite) [dependency-of] jq @ 1.6 (/nix/var/nix/db/db.sqlite)", // jq lib output
"glibc @ 2.37-8 (/nix/var/nix/db/db.sqlite) [dependency-of] oniguruma @ 6.9.8 (/nix/var/nix/db/db.sqlite)",
"jq @ 1.6 (/nix/var/nix/db/db.sqlite) [dependency-of] jq @ 1.6 (/nix/var/nix/db/db.sqlite)", // jq bin to lib output dependency
"libidn2 @ 2.3.4 (/nix/var/nix/db/db.sqlite) [dependency-of] glibc @ 2.37-8 (/nix/var/nix/db/db.sqlite)",
"libunistring @ 1.1 (/nix/var/nix/db/db.sqlite) [dependency-of] libidn2 @ 2.3.4 (/nix/var/nix/db/db.sqlite)",
"oniguruma @ 6.9.8 (/nix/var/nix/db/db.sqlite) [dependency-of] jq @ 1.6 (/nix/var/nix/db/db.sqlite)", // jq bin output
"oniguruma @ 6.9.8 (/nix/var/nix/db/db.sqlite) [dependency-of] jq @ 1.6 (/nix/var/nix/db/db.sqlite)", // jq lib output
"xgcc @ 12.3.0 (/nix/var/nix/db/db.sqlite) [dependency-of] glibc @ 2.37-8 (/nix/var/nix/db/db.sqlite)",
},
},
}
for _, tt := range tests {
t.Run(tt.fixture, func(t *testing.T) {
pkgtest.NewCatalogTester().
WithImageResolver(t, tt.fixture).
ExpectsPackageStrings(tt.wantPkgs).
ExpectsRelationshipStrings(tt.wantRel).
TestCataloger(t, NewCataloger(DefaultConfig()))
})
}
}
func TestCataloger_Image_FilesListing(t *testing.T) {
tests := []struct {
fixture string
wantPkgFiles map[string][]string
}{
{
fixture: "image-nixos-jq-pkg-db",
wantPkgFiles: map[string][]string{
"libidn2": {
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/lib/libidn2.la",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/lib/libidn2.so.0.3.8",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/cs/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/da/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/de/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/eo/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/es/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/fi/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/fr/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/fur/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/hr/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/hu/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/id/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/it/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/ja/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/ka/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/ko/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/nl/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/pl/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/pt_BR/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/ro/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/ru/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/sr/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/sv/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/uk/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/vi/LC_MESSAGES/libidn2.mo",
"/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4/share/locale/zh_CN/LC_MESSAGES/libidn2.mo",
},
},
},
}
for _, tt := range tests {
t.Run(tt.fixture, func(t *testing.T) {
pkgtest.NewCatalogTester().
WithImageResolver(t, tt.fixture).
ExpectsAssertion(func(t *testing.T, pkgs []pkg.Package, relationships []artifact.Relationship) {
found := strset.New()
for _, p := range pkgs {
if files, ok := tt.wantPkgFiles[p.Name]; ok {
m, ok := p.Metadata.(pkg.NixStoreEntry)
require.True(t, ok)
if d := cmp.Diff(files, m.Files); d != "" {
t.Errorf("unexpected files for package %q: %s", p.Name, d)
}
found.Add(p.Name)
}
}
expected := strset.New()
for n := range tt.wantPkgFiles {
expected.Add(n)
}
assert.ElementsMatch(t, expected.List(), found.List())
}).
TestCataloger(t, NewCataloger(Config{CaptureOwnedFiles: true}))
})
}
}
func TestCataloger_Directory(t *testing.T) {
tests := []struct {
fixture string
@@ -20,22 +152,95 @@ func TestCataloger_Catalog(t *testing.T) {
fixture: "test-fixtures/fixture-1",
wantPkgs: []pkg.Package{
{
Name: "glibc",
Version: "2.34-210",
PURL: "pkg:nix/glibc@2.34-210?output=bin&outputhash=h0cnbmfcn93xm5dg2x27ixhag1cwndga",
Locations: file.NewLocationSet(file.NewLocation("nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin")),
FoundBy: catalogerName,
Type: pkg.NixPkg,
Name: "glibc",
Version: "2.34-210",
PURL: "pkg:nix/glibc@2.34-210?drvpath=5av396z8xa13jg89g9jws145c0k26k2x-glibc-2.34-210.drv&output=bin&outputhash=h0cnbmfcn93xm5dg2x27ixhag1cwndga",
Locations: file.NewLocationSet(
file.NewLocation("nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin").WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
file.NewLocation("nix/store/5av396z8xa13jg89g9jws145c0k26k2x-glibc-2.34-210.drv").WithAnnotation(pkg.EvidenceAnnotationKey, pkg.SupportingEvidenceAnnotation),
),
FoundBy: "nix-cataloger",
Type: pkg.NixPkg,
Metadata: pkg.NixStoreEntry{
Path: "/nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin",
Derivation: pkg.NixDerivation{
Path: "nix/store/5av396z8xa13jg89g9jws145c0k26k2x-glibc-2.34-210.drv",
System: "aarch64-linux",
InputDerivations: []pkg.NixDerivationReference{
{
Path: "/nix/store/1zi0k7y01rhqr2gfqb42if0icswg65sj-locale-C.diff.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/45j86ggi8mlpfslcrgvjf7m6phia21fp-raw.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/4fnfsd9sc7bam6886hwyaprdsww66dg3-bison-3.8.2.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/51azdrrvcqrk2hbky7ryphlwd99yz25d-linux-headers-5.18.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/67s0qc21gyarmdwc181bqmjc3qzv8zkz-libidn2-2.3.2.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/9rhliwskh3mrrs5nfzgz0x6wrccyfg7k-bootstrap-stage0-glibc-bootstrap.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/cl1wcw2v1ifzjlkzi50h32a6lms9m25s-binutils-2.38.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/ghjc8bkfk8lh53z14mk2nk7h059zh7vx-python3-minimal-3.10.5.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/k3786wfzw637r7sylccdmm92saqp73d8-glibc-2.34.tar.xz.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/l5zr5m1agvvnic49fg6qc44g5fgj3la1-glibc-reinstate-prlimit64-fallback.patch?id=eab07e78b691ae7866267fc04d31c7c3ad6b0eeb.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/mf5kz6d01ab8h0rswzyr04mbcd6g5x9n-bootstrap-stage2-stdenv-linux.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/nd1zy67vp028707pbh466qhrfqh4cpq6-bootstrap-stage2-gcc-wrapper-.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/ra77ww7p2xx8jh8n4m9vmj6wc8wxijdb-bootstrap-tools.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/wlldapf5bg58kivw520ll5bw0fmlaid7-raw.drv",
Outputs: []string{"out"},
},
},
InputSources: []string{
"/nix/store/001gp43bjqzx60cg345n2slzg7131za8-nix-nss-open-files.patch",
"/nix/store/7kw224hdyxd7115lrqh9a4dv2x8msq2s-fix-x64-abi.patch",
"/nix/store/8haph3ng4mgsqr6p4024vj8k6kg3mqc4-nix-locale-archive.patch",
"/nix/store/95hp6hs9g73h93safadb8x6vajyqkv6q-0001-Revert-Remove-all-usage-of-BASH-or-BASH-in-installed.patch",
"/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh",
"/nix/store/b1w7zbvm39ff1i52iyjggyvw2rdxz104-dont-use-system-ld-so-cache.patch",
"/nix/store/ikmqczy0y20n04a2b8qfflzwihv8139g-separate-debug-info.sh",
"/nix/store/mgx19wbmgrh3rblbxhs6vi47sha15n11-2.34-master.patch.gz",
"/nix/store/mnglr8rr7nl444h7p50ysyq8qd0fm1lm-dont-use-system-ld-so-preload.patch",
"/nix/store/xkd50xxii6k7l1kmw4l5x6xzbhamcs87-allow-kernel-2.6.32.patch",
"/nix/store/za0pg7fmysrcwrqcal26fnmzw6vycgdn-fix_path_attribute_in_getconf.patch",
},
},
OutputHash: "h0cnbmfcn93xm5dg2x27ixhag1cwndga",
Output: "bin",
Files: []string{
"nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin/lib",
"nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin/lib/glibc.so",
"nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin/share",
"nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin/share/man",
"nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin/share/man/glibc.1",
},
Files: nil, // default cataloger configure does not capture owned files
},
},
},
@@ -43,12 +248,10 @@ func TestCataloger_Catalog(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.fixture, func(t *testing.T) {
c := NewStoreCataloger()
pkgtest.NewCatalogTester().
FromDirectory(t, tt.fixture).
Expects(tt.wantPkgs, tt.wantRel).
TestCataloger(t, c)
TestCataloger(t, NewCataloger(DefaultConfig()))
})
}
}
+138
View File
@@ -0,0 +1,138 @@
package nix
import (
"fmt"
"io"
"path"
"strconv"
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/unknown"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
const defaultSchema = 10
type dbProcessor func(config Config, dbLocation file.Location, resolver file.Resolver, catalogerName string) ([]pkg.Package, []artifact.Relationship, error)
type dbCataloger struct {
config Config
schemaProcessor map[int]dbProcessor
catalogerName string
}
func newDBCataloger(cfg Config, catalogerName string) dbCataloger {
return dbCataloger{
config: cfg,
catalogerName: catalogerName,
schemaProcessor: map[int]dbProcessor{
10: processV10DB,
},
}
}
type dbPackageEntry struct {
ID int
DrvID int
nixStorePath
DeriverPath string
*derivationFile
Location *file.Location
Files []string
}
func (c dbCataloger) catalog(resolver file.Resolver) ([]pkg.Package, []artifact.Relationship, error) {
dbLocs, err := resolver.FilesByGlob("**/nix/var/nix/db/db.sqlite")
if err != nil {
return nil, nil, fmt.Errorf("failed to find Nix database: %w", err)
}
if len(dbLocs) == 0 {
return nil, nil, nil
}
var pkgs []pkg.Package
var relationships []artifact.Relationship
var errs error
for _, dbLoc := range dbLocs {
parser, schema := c.selectDBParser(dbLoc, resolver)
if parser == nil {
errs = unknown.Appendf(errs, dbLoc.Coordinates, "unsupported Nix database schema for schema=%d at %q", schema, dbLoc.RealPath)
continue
}
newPkgs, newRelationships, err := parser(c.config, dbLoc, resolver, c.catalogerName)
if err != nil {
errs = unknown.Append(errs, dbLoc.Coordinates, err)
continue
}
pkgs = append(pkgs, newPkgs...)
relationships = append(relationships, newRelationships...)
}
return pkgs, relationships, errs
}
func (c dbCataloger) selectDBParser(dbLocation file.Location, resolver file.Resolver) (dbProcessor, int) {
loc := resolver.RelativeFileByPath(dbLocation, path.Join(path.Dir(dbLocation.RealPath), "schema"))
if loc == nil {
log.WithFields("path", dbLocation.RealPath).Tracef("failed to detect Nix database schema, assuming %d", defaultSchema)
return c.schemaProcessor[defaultSchema], 0
}
schemaContents, err := resolver.FileContentsByLocation(*loc)
defer internal.CloseAndLogError(schemaContents, loc.RealPath)
if err != nil {
log.WithFields("path", loc.RealPath).Tracef("failed to open Nix database schema file, assuming %d", defaultSchema)
return c.schemaProcessor[defaultSchema], 0
}
contents, err := io.ReadAll(schemaContents)
if err != nil {
log.WithFields("path", loc.RealPath).Tracef("failed to read Nix database schema file, assuming %d", defaultSchema)
return c.schemaProcessor[defaultSchema], 0
}
schema, err := strconv.Atoi(strings.TrimSpace(string(contents)))
if err != nil {
log.WithFields("path", loc.RealPath).Tracef("failed to parse Nix database schema file, assuming %d", defaultSchema)
return c.schemaProcessor[defaultSchema], 0
}
processor := c.schemaProcessor[schema]
if processor == nil {
closestSchema := c.findClosestSchema(schema)
if closestSchema == 0 {
schema = defaultSchema
}
processor = c.schemaProcessor[closestSchema]
log.WithFields("path", loc.RealPath).Tracef("unsupported Nix database schema (%d), treating as closest available schema (%d)", schema, closestSchema)
}
return processor, schema
}
func (c dbCataloger) findClosestSchema(got int) int {
var closest int
var closestDiff int
for schema := range c.schemaProcessor {
if schema == got {
return schema
}
diff := schema - got
if diff < 0 {
diff = -diff
}
if diff < closestDiff {
closestDiff = diff
closest = schema
}
}
return closest
}
+249
View File
@@ -0,0 +1,249 @@
package nix
import (
"database/sql"
"fmt"
"io"
"os"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
var _ dbProcessor = processV10DB
func processV10DB(config Config, dbLocation file.Location, resolver file.Resolver, catalogerName string) ([]pkg.Package, []artifact.Relationship, error) {
dbContents, err := resolver.FileContentsByLocation(dbLocation)
defer internal.CloseAndLogError(dbContents, dbLocation.RealPath)
if err != nil {
return nil, nil, fmt.Errorf("unable to read Nix database: %w", err)
}
tempDB, err := createTempDB(dbContents)
if err != nil {
return nil, nil, fmt.Errorf("failed to create temporary database: %w", err)
}
defer os.RemoveAll(tempDB.Name())
db, err := sql.Open("sqlite", tempDB.Name())
if err != nil {
return nil, nil, fmt.Errorf("failed to open database: %w", err)
}
db.SetConnMaxLifetime(0)
defer db.Close()
packageEntries, err := extractV10DBPackages(config, db, dbLocation, resolver)
if err != nil {
return nil, nil, err
}
pkgs, relationships, err := finalizeV10DBResults(db, packageEntries, catalogerName)
if err != nil {
return nil, nil, err
}
return pkgs, relationships, nil
}
func extractV10DBPackages(config Config, db *sql.DB, dbLocation file.Location, resolver file.Resolver) (map[int]*dbPackageEntry, error) {
pkgs, err := extractV10DBValidPaths(config, db, dbLocation, resolver)
if err != nil {
return nil, err
}
err = extractV10DBDerivationOutputs(db, pkgs)
if err != nil {
return nil, err
}
return pkgs, nil
}
func extractV10DBValidPaths(config Config, db *sql.DB, dbLocation file.Location, resolver file.Resolver) (map[int]*dbPackageEntry, error) {
packages := make(map[int]*dbPackageEntry)
rows, err := db.Query("SELECT id, path, hash, deriver FROM ValidPaths")
if err != nil {
return nil, fmt.Errorf("failed to query ValidPaths: %w", err)
}
defer rows.Close()
for rows.Next() {
var id int
var path, hash, deriver sql.NullString
if err := rows.Scan(&id, &path, &hash, &deriver); err != nil {
return nil, fmt.Errorf("failed to scan ValidPaths row: %w", err)
}
if !path.Valid {
continue
}
nsp := parseNixStorePath(path.String)
if nsp == nil {
nsp = &nixStorePath{}
}
// always trust the DB values over string parsing
nsp.OutputHash = hash.String
nsp.StorePath = path.String
var files []string
if config.CaptureOwnedFiles {
files = listOutputPaths(path.String, resolver)
}
df, err := newDerivationFromPath(deriver.String, resolver)
if err != nil {
log.WithFields("path", deriver.String, "error", err).Trace("unable to find derivation")
df = nil
}
packages[id] = &dbPackageEntry{
ID: id,
nixStorePath: *nsp,
derivationFile: df,
DeriverPath: deriver.String,
Location: &dbLocation,
Files: files,
}
}
return packages, nil
}
func listOutputPaths(storePath string, resolver file.Resolver) []string {
if storePath == "" {
return nil
}
searchGlob := storePath + "/**/*"
locations, err := resolver.FilesByGlob(searchGlob)
if err != nil {
log.WithFields("path", storePath, "error", err).Trace("unable to find output paths")
return nil
}
return filePaths(locations)
}
func extractV10DBDerivationOutputs(db *sql.DB, packages map[int]*dbPackageEntry) error {
outputRows, err := db.Query("SELECT drv, id, path FROM DerivationOutputs")
if err != nil {
return fmt.Errorf("failed to query DerivationOutputs: %w", err)
}
defer outputRows.Close()
pkgsByPath := make(map[string]*dbPackageEntry)
for _, p := range packages {
pkgsByPath[p.StorePath] = p
}
for outputRows.Next() {
var drvID int
var outputID, outputPath string
if err := outputRows.Scan(&drvID, &outputID, &outputPath); err != nil {
return fmt.Errorf("failed to scan DerivationOutputs row: %w", err)
}
if _, ok := pkgsByPath[outputPath]; !ok {
continue
}
pkgsByPath[outputPath].Output = outputID
pkgsByPath[outputPath].DrvID = drvID
}
return nil
}
func finalizeV10DBResults(db *sql.DB, packageEntries map[int]*dbPackageEntry, catalogerName string) ([]pkg.Package, []artifact.Relationship, error) {
// make Syft packages for each package entry
syftPackages := make(map[int]pkg.Package)
for id, entry := range packageEntries {
syftPackages[id] = newDBPackage(entry, catalogerName)
}
var relationships []artifact.Relationship
query := `
SELECT r.referrer, r.reference
FROM Refs r
JOIN ValidPaths v1 ON r.referrer = v1.id
JOIN ValidPaths v2 ON r.reference = v2.id
`
refRows, err := db.Query(query)
if err != nil {
return nil, nil, fmt.Errorf("failed to query Refs with ValidPaths JOIN: %w", err)
}
defer refRows.Close()
relExists := make(map[int]map[int]bool)
for refRows.Next() {
var referrerID, referenceID int
if err := refRows.Scan(&referrerID, &referenceID); err != nil {
return nil, nil, fmt.Errorf("failed to scan Refs row: %w", err)
}
if referrerID == referenceID {
// skip self-references
continue
}
referrer, refExists := syftPackages[referrerID]
reference, refeeExists := syftPackages[referenceID]
if !refExists || !refeeExists {
// only include relationships for packages we have discovered
continue
}
if _, ok := relExists[referrerID]; !ok {
relExists[referrerID] = make(map[int]bool)
}
if relExists[referrerID][referenceID] {
// deduplicate existing relationships
continue
}
relExists[referrerID][referenceID] = true
rel := artifact.Relationship{
From: reference,
To: referrer,
Type: artifact.DependencyOfRelationship,
}
relationships = append(relationships, rel)
}
var pkgs []pkg.Package
for _, p := range syftPackages {
pkgs = append(pkgs, p)
}
return pkgs, relationships, nil
}
func createTempDB(content io.ReadCloser) (*os.File, error) {
tempFile, err := os.CreateTemp("", "nix-db-*.sqlite")
if err != nil {
return nil, err
}
_, err = io.Copy(tempFile, content)
if err != nil {
tempFile.Close()
os.Remove(tempFile.Name())
return nil, err
}
return tempFile, nil
}
+122
View File
@@ -0,0 +1,122 @@
package nix
import (
"fmt"
"strings"
"github.com/nix-community/go-nix/pkg/derivation"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/file"
)
type derivationFile struct {
Location file.Location
derivation.Derivation
}
func newDerivationFromPath(p string, resolver file.Resolver) (*derivationFile, error) {
locs, err := resolver.FilesByPath(p)
if err != nil {
return nil, fmt.Errorf("failed to find derivation: %w", err)
}
if len(locs) == 0 {
return nil, nil
}
// only use one reference
return newDerivationFromLocation(locs[0], resolver)
}
func newDerivationFromLocation(loc file.Location, resolver file.Resolver) (*derivationFile, error) {
reader, err := resolver.FileContentsByLocation(loc)
if err != nil {
return nil, fmt.Errorf("failed to read derivation: %w", err)
}
defer internal.CloseAndLogError(reader, loc.RealPath)
d, err := derivation.ReadDerivation(reader)
if err != nil {
return nil, fmt.Errorf("failed to parse derivation: %w", err)
}
return &derivationFile{
Location: loc,
Derivation: *d,
}, nil
}
type derivations struct {
derivationsByDrvPath map[string]derivationFile
drvPathByOutputPath map[string]string
}
func newDerivations() *derivations {
return &derivations{
derivationsByDrvPath: make(map[string]derivationFile),
drvPathByOutputPath: make(map[string]string),
}
}
func (c *derivations) add(df derivationFile) {
c.derivationsByDrvPath[df.Location.RealPath] = df
for _, output := range df.Outputs {
if output == nil || output.Path == "" {
continue
}
c.drvPathByOutputPath[output.Path] = df.Location.RealPath
}
}
func (c *derivations) findDerivationForOutputPath(outputPath string) *derivationFile {
if !strings.HasPrefix(outputPath, "/") {
outputPath = "/" + outputPath
}
if drvPath, ok := c.drvPathByOutputPath[outputPath]; ok {
d, ok := c.derivationsByDrvPath[drvPath]
if ok {
return &d
}
}
return nil
}
// given a path as input, assuming it's an output path for a derivation, find all input store paths needed for this particular output path.
func (c *derivations) findDependencies(p string) []string {
if d, ok := c.derivationsByDrvPath[p]; ok {
var deps []string
for drvPath, names := range d.InputDerivations {
if len(names) == 0 {
continue
}
for _, n := range names {
outputPath := c.namedOutputStorePath(drvPath, n)
if outputPath == "" {
continue
}
deps = append(deps, outputPath)
}
}
for _, inputSrc := range d.InputSources {
if inputSrc == "" {
continue
}
deps = append(deps, inputSrc)
}
return deps
}
if drvPath, ok := c.drvPathByOutputPath[p]; ok {
return c.findDependencies(drvPath)
}
return nil
}
func (c *derivations) namedOutputStorePath(drvPath, name string) string {
if d, ok := c.derivationsByDrvPath[drvPath]; ok {
if output, ok := d.Outputs[name]; ok {
return output.Path
}
}
return ""
}
+407
View File
@@ -0,0 +1,407 @@
package nix
import (
"testing"
"github.com/nix-community/go-nix/pkg/derivation"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/file"
)
func TestDerivationCollection_Add(t *testing.T) {
c := newDerivations()
d := derivationFile{
Location: file.NewLocation("/nix/store/xyz789-foo.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/abc123-foo",
},
"dev": {
Path: "/nix/store/def456-foo-dev",
},
},
},
}
c.add(d)
assert.Len(t, c.derivationsByDrvPath, 1)
assert.Len(t, c.drvPathByOutputPath, 2)
assert.Equal(t, "/nix/store/xyz789-foo.drv", c.drvPathByOutputPath["/nix/store/abc123-foo"])
assert.Equal(t, "/nix/store/xyz789-foo.drv", c.drvPathByOutputPath["/nix/store/def456-foo-dev"])
}
func TestDerivationCollection_AddNilOutputs(t *testing.T) {
c := newDerivations()
d := derivationFile{
Location: file.NewLocation("/nix/store/xyz789-foo.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": nil,
"dev": {
Path: "",
},
},
},
}
c.add(d)
assert.Len(t, c.derivationsByDrvPath, 1)
assert.Empty(t, c.drvPathByOutputPath)
}
func TestDerivationCollection_FindDerivationForOutputPath(t *testing.T) {
c := newDerivations()
// standard derivation
standardDrv := derivationFile{
Location: file.NewLocation("/nix/store/xyz789-foo.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/abc123-foo",
},
},
},
}
c.add(standardDrv)
// derivation with multiple outputs
multiOutputDrv := derivationFile{
Location: file.NewLocation("/nix/store/multi-output.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/multi-out-path",
},
"dev": {
Path: "/nix/store/multi-dev-path",
},
"doc": {
Path: "/nix/store/multi-doc-path",
},
},
},
}
c.add(multiOutputDrv)
// derivation with special characters in path
specialCharsDrv := derivationFile{
Location: file.NewLocation("/nix/store/special-chars+_.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/special-chars+_-output",
},
},
},
}
c.add(specialCharsDrv)
// derivation with same output path as another (should override)
duplicateOutputDrv := derivationFile{
Location: file.NewLocation("/nix/store/duplicate.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/abc123-foo", // same as standardDrv output
},
},
},
}
c.add(duplicateOutputDrv)
tests := []struct {
name string
outputPath string
expected *derivationFile
}{
{
name: "output path exists",
outputPath: "/nix/store/abc123-foo",
expected: &duplicateOutputDrv,
},
{
name: "output path exists without leading slash",
outputPath: "nix/store/abc123-foo",
expected: &duplicateOutputDrv,
},
{
name: "output path does not exist",
outputPath: "/nix/store/nonexistent",
},
{
name: "multiple output derivation - out path",
outputPath: "/nix/store/multi-out-path",
expected: &multiOutputDrv,
},
{
name: "multiple output derivation - dev path",
outputPath: "/nix/store/multi-dev-path",
expected: &multiOutputDrv,
},
{
name: "special characters in path",
outputPath: "/nix/store/special-chars+_-output",
expected: &specialCharsDrv,
},
{
name: "empty string path",
outputPath: "",
},
{
name: "path with just a slash",
outputPath: "/",
},
{
name: "drv path exists in mapping but not in derivations",
outputPath: "/nix/store/missing",
},
}
// add a path mapping to a derivation that doesn't exist
c.drvPathByOutputPath["/nix/store/missing"] = "/nix/store/nonexistent.drv"
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := c.findDerivationForOutputPath(tt.outputPath)
if tt.expected == nil {
assert.Nil(t, result)
} else {
require.NotNil(t, result)
assert.Equal(t, tt.expected.Location.RealPath, result.Location.RealPath)
}
})
}
}
func TestDerivationCollection_FindDependencies(t *testing.T) {
c := newDerivations()
// set up a dependency tree:
// - foo depends on bar and baz
// - bar depends on qux
// create "qux" derivation
quxDrv := derivationFile{
Location: file.NewLocation("/nix/store/qux.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/qux-path",
},
},
},
}
c.add(quxDrv)
// create "bar" derivation which depends on qux
barDrv := derivationFile{
Location: file.NewLocation("/nix/store/bar.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/bar-path",
},
},
InputDerivations: map[string][]string{
"/nix/store/qux.drv": {"out"},
},
},
}
c.add(barDrv)
// create "baz" derivation
bazDrv := derivationFile{
Location: file.NewLocation("/nix/store/baz.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/baz-path",
},
},
},
}
c.add(bazDrv)
// create "foo" derivation which depends on bar and baz
fooDrv := derivationFile{
Location: file.NewLocation("/nix/store/foo.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/foo-path",
},
},
InputDerivations: map[string][]string{
"/nix/store/bar.drv": {"out"},
"/nix/store/baz.drv": {"out"},
},
InputSources: []string{
"/nix/store/src1",
"/nix/store/src2",
},
},
}
c.add(fooDrv)
// add a test case for empty input names
emptyNamesDrv := derivationFile{
Location: file.NewLocation("/nix/store/empty-names.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/empty-names-path",
},
},
InputDerivations: map[string][]string{
"/nix/store/bar.drv": {},
},
},
}
c.add(emptyNamesDrv)
// add a test case for empty input sources
emptySourcesDrv := derivationFile{
Location: file.NewLocation("/nix/store/empty-sources.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/empty-sources-path",
},
},
InputDerivations: map[string][]string{
"/nix/store/bar.drv": {"out"},
},
InputSources: []string{
"",
},
},
}
c.add(emptySourcesDrv)
tests := []struct {
name string
path string
expected []string
}{
{
name: "lookup by derivation path",
path: "/nix/store/foo.drv",
expected: []string{
"/nix/store/bar-path",
"/nix/store/baz-path",
"/nix/store/src1",
"/nix/store/src2",
},
},
{
name: "lookup by output path",
path: "/nix/store/foo-path",
expected: []string{
"/nix/store/bar-path",
"/nix/store/baz-path",
"/nix/store/src1",
"/nix/store/src2",
},
},
{
name: "lookup by derivation with no inputs",
path: "/nix/store/qux.drv",
expected: nil,
},
{
name: "lookup nonexistent path",
path: "/nix/store/nonexistent",
expected: nil,
},
{
name: "lookup derivation with empty input names",
path: "/nix/store/empty-names.drv",
expected: nil,
},
{
name: "lookup derivation with empty input sources",
path: "/nix/store/empty-sources.drv",
expected: []string{
"/nix/store/bar-path",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := c.findDependencies(tt.path)
if tt.expected == nil {
assert.Nil(t, result)
} else {
require.NotNil(t, result)
assert.ElementsMatch(t, tt.expected, result)
}
})
}
}
func TestDerivationCollection_NamedOutputStorePath(t *testing.T) {
c := newDerivations()
d := derivationFile{
Location: file.NewLocation("/nix/store/xyz789-foo.drv"),
Derivation: derivation.Derivation{
Outputs: map[string]*derivation.Output{
"out": {
Path: "/nix/store/abc123-foo",
},
"dev": {
Path: "/nix/store/def456-foo-dev",
},
},
},
}
c.add(d)
tests := []struct {
name string
drvPath string
outName string
expected string
}{
{
name: "existing drv and output",
drvPath: "/nix/store/xyz789-foo.drv",
outName: "out",
expected: "/nix/store/abc123-foo",
},
{
name: "existing drv and dev output",
drvPath: "/nix/store/xyz789-foo.drv",
outName: "dev",
expected: "/nix/store/def456-foo-dev",
},
{
name: "existing drv but nonexistent output",
drvPath: "/nix/store/xyz789-foo.drv",
outName: "nonexistent",
expected: "",
},
{
name: "nonexistent drv",
drvPath: "/nix/store/nonexistent.drv",
outName: "out",
expected: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := c.namedOutputStorePath(tt.drvPath, tt.outName)
assert.Equal(t, tt.expected, result)
})
}
}
+102 -15
View File
@@ -1,22 +1,43 @@
package nix
import (
"path"
"sort"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
func newNixStorePackage(storePath nixStorePath, locations ...file.Location) pkg.Package {
type nixStorePackage struct {
Location *file.Location
Files []string
*derivationFile
nixStorePath
}
func newNixStorePackage(pp nixStorePackage, catalogerName string) pkg.Package {
locations := file.NewLocationSet(pp.Location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation))
var derivationPath string
if pp.derivationFile != nil {
derivationPath = pp.derivationFile.Location.RealPath
locations.Add(pp.derivationFile.Location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.SupportingEvidenceAnnotation))
}
p := pkg.Package{
Name: storePath.name,
Version: storePath.version,
Name: pp.Name,
Version: pp.Version,
FoundBy: catalogerName,
Locations: file.NewLocationSet(locations...),
Locations: locations,
Type: pkg.NixPkg,
PURL: packageURL(storePath),
PURL: packageURL(pp.nixStorePath, derivationPath),
Metadata: pkg.NixStoreEntry{
OutputHash: storePath.outputHash,
Output: storePath.output,
Path: pp.StorePath,
Derivation: newDerivation(pp.derivationFile),
OutputHash: pp.OutputHash,
Output: pp.Output,
Files: pp.Files,
},
}
@@ -25,24 +46,90 @@ func newNixStorePackage(storePath nixStorePath, locations ...file.Location) pkg.
return p
}
func packageURL(storePath nixStorePath) string {
func newDBPackage(entry *dbPackageEntry, catalogerName string) pkg.Package {
locations := file.NewLocationSet(
entry.Location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
file.NewLocation(entry.StorePath).WithAnnotation(pkg.EvidenceAnnotationKey, pkg.SupportingEvidenceAnnotation),
)
if entry.derivationFile != nil {
locations.Add(entry.derivationFile.Location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.SupportingEvidenceAnnotation))
}
p := pkg.Package{
Name: entry.Name,
Version: entry.Version,
FoundBy: catalogerName,
Locations: locations,
Type: pkg.NixPkg,
PURL: packageURL(entry.nixStorePath, entry.DeriverPath),
Metadata: pkg.NixStoreEntry{
Path: entry.StorePath,
Derivation: newDerivation(entry.derivationFile),
OutputHash: entry.OutputHash,
Output: entry.Output,
Files: entry.Files,
},
}
p.SetID()
return p
}
func newDerivation(df *derivationFile) pkg.NixDerivation {
if df == nil {
return pkg.NixDerivation{}
}
var inputDerivations []pkg.NixDerivationReference
for drvPath, names := range df.InputDerivations {
sort.Strings(names)
inputDerivations = append(inputDerivations, pkg.NixDerivationReference{
Path: drvPath,
Outputs: names,
})
}
sort.Slice(inputDerivations, func(i, j int) bool {
return inputDerivations[i].Path < inputDerivations[j].Path
})
sources := df.InputSources
sort.Strings(sources)
return pkg.NixDerivation{
Path: df.Location.RealPath,
System: df.Platform,
InputDerivations: inputDerivations,
InputSources: sources,
}
}
func packageURL(storePath nixStorePath, drvPath string) string {
var qualifiers packageurl.Qualifiers
if storePath.output != "" {
if storePath.Output != "" {
// since there is no nix pURL type yet, this is a guess, however, it is reasonable to assume that
// if only a single output is installed the pURL should be able to express this.
qualifiers = append(qualifiers,
packageurl.Qualifier{
Key: "output",
Value: storePath.output,
Value: storePath.Output,
},
)
}
if storePath.outputHash != "" {
// it's not immediately clear if the hash found in the store path should be encoded in the pURL
if storePath.OutputHash != "" {
qualifiers = append(qualifiers,
packageurl.Qualifier{
Key: "outputhash",
Value: storePath.outputHash,
Value: storePath.OutputHash,
},
)
}
if drvPath != "" {
qualifiers = append(qualifiers,
packageurl.Qualifier{
Key: "drvpath",
Value: path.Base(drvPath),
},
)
}
@@ -50,8 +137,8 @@ func packageURL(storePath nixStorePath) string {
// TODO: nix pURL type has not been accepted yet (only proposed at this time)
"nix",
"",
storePath.name,
storePath.version,
storePath.Name,
storePath.Version,
qualifiers,
"")
return pURL.ToString()
+20 -10
View File
@@ -11,39 +11,49 @@ func Test_packageURL(t *testing.T) {
tests := []struct {
name string
storePath nixStorePath
drvPath string
want string
}{
{
name: "name + version",
storePath: nixStorePath{
name: "glibc",
version: "2.34",
Name: "glibc",
Version: "2.34",
},
want: "pkg:nix/glibc@2.34",
},
{
name: "hash qualifier",
storePath: nixStorePath{
name: "glibc",
version: "2.34",
outputHash: "h0cnbmfcn93xm5dg2x27ixhag1cwndga",
Name: "glibc",
Version: "2.34",
OutputHash: "h0cnbmfcn93xm5dg2x27ixhag1cwndga",
},
want: "pkg:nix/glibc@2.34?outputhash=h0cnbmfcn93xm5dg2x27ixhag1cwndga",
},
{
name: "output qualifier",
storePath: nixStorePath{
name: "glibc",
version: "2.34",
outputHash: "h0cnbmfcn93xm5dg2x27ixhag1cwndga",
output: "bin",
Name: "glibc",
Version: "2.34",
OutputHash: "h0cnbmfcn93xm5dg2x27ixhag1cwndga",
Output: "bin",
},
want: "pkg:nix/glibc@2.34?output=bin&outputhash=h0cnbmfcn93xm5dg2x27ixhag1cwndga",
},
{
name: "derivation qualifier",
storePath: nixStorePath{
Name: "glibc",
Version: "2.34",
},
drvPath: "/nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34.drv",
want: "pkg:nix/glibc@2.34?drvpath=h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34.drv",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, packageURL(tt.storePath))
assert.Equal(t, tt.want, packageURL(tt.storePath, tt.drvPath))
})
}
}
@@ -18,7 +18,7 @@ var (
// minor: "34"
// patch: "210"
// (there are other capture groups, but they can be ignored)
rightMostVersionIshPattern = regexp.MustCompile(`-(?P<version>(?P<major>[0-9][a-zA-Z0-9]*)(\.(?P<minor>[0-9][a-zA-Z0-9]*))?(\.(?P<patch>0|[1-9][a-zA-Z0-9]*)){0,3}(?:-(?P<prerelease>\d*[.a-zA-Z-][.0-9a-zA-Z-]*)*)?(?:\+(?P<metadata>[.0-9a-zA-Z-]+(?:\.[.0-9a-zA-Z-]+)*))?)`)
rightMostVersionIshPattern = regexp.MustCompile(`-(?P<version>(?P<major>[0-9][a-zA-Z0-9]*)(\.(?P<minor>[0-9][a-zA-Z0-9]*))?(\.(?P<patch>0|[1-9][a-zA-Z0-9]*)){0,3}(?:-(?P<prerelease>\d*[.0-9a-zA-Z-]*)*)?(?:\+(?P<metadata>[.0-9a-zA-Z-]+(?:\.[.0-9a-zA-Z-]+)*))?)`)
unstableVersion = regexp.MustCompile(`-(?P<version>unstable-\d{4}-\d{2}-\d{2})$`)
)
@@ -26,14 +26,15 @@ var (
// checkout the package naming conventions here: https://nixos.org/manual/nixpkgs/stable/#sec-package-naming
type nixStorePath struct {
outputHash string
name string
version string
output string
StorePath string
OutputHash string
Name string
Version string
Output string
}
func (p nixStorePath) isValidPackage() bool {
return p.name != "" && p.version != ""
return p.Name != "" && p.Version != ""
}
func findParentNixStorePath(source string) string {
@@ -54,13 +55,13 @@ func findParentNixStorePath(source string) string {
return source[0:startOfSubPath]
}
func parseNixStorePath(source string) *nixStorePath {
if strings.HasSuffix(source, ".drv") {
func parseNixStorePath(og string) *nixStorePath {
if strings.HasSuffix(og, ".drv") {
// ignore derivations
return nil
}
source = path.Base(source)
source := path.Base(og)
versionStartIdx, versionIsh, prerelease := findVersionIsh(source)
if versionStartIdx == -1 {
@@ -85,11 +86,16 @@ func parseNixStorePath(source string) *nixStorePath {
output = lastPrereleaseField
}
if og != "" && !strings.HasPrefix(og, "/") {
og = fmt.Sprintf("/%s", og)
}
return &nixStorePath{
outputHash: hash,
name: name,
version: version,
output: output,
StorePath: og,
OutputHash: hash,
Name: name,
Version: version,
Output: output,
}
}
@@ -102,33 +108,25 @@ func findVersionIsh(input string) (int, string, string) {
// note that the match indices are in the form of [start, end, start, end, ...]. Also note that the
// capture group for version in both regexes are the same index, but if the regexes are changed
// this code will start to fail.
versionGroup := 1
match := unstableVersion.FindAllStringSubmatchIndex(input, -1)
if len(match) > 0 && len(match[0]) > 0 {
return match[0][versionGroup*2], input[match[0][versionGroup*2]:match[0][(versionGroup*2)+1]], ""
// check for unstable version pattern first
if match := unstableVersion.FindStringSubmatch(input); match != nil {
indices := unstableVersion.FindStringSubmatchIndex(input)
versionStart := indices[2] // index of first capture group's start
version := match[1] // first capture group is the version
return versionStart, version, ""
}
match = rightMostVersionIshPattern.FindAllStringSubmatchIndex(input, -1)
if len(match) == 0 || len(match[0]) == 0 {
// try the regular version pattern
match := rightMostVersionIshPattern.FindStringSubmatch(input)
if match == nil {
return -1, "", ""
}
var version string
versionStart, versionStop := match[0][versionGroup*2], match[0][(versionGroup*2)+1]
if versionStart != -1 || versionStop != -1 {
version = input[versionStart:versionStop]
}
version := match[1] // capture group 1 is the version
indices := rightMostVersionIshPattern.FindStringSubmatchIndex(input)
versionStart := indices[2] // index of first capture group's start
prerelease := match[7] // capture group 7 is the prerelease version
prereleaseGroup := 7
var prerelease string
prereleaseStart, prereleaseStop := match[0][prereleaseGroup*2], match[0][(prereleaseGroup*2)+1]
if prereleaseStart != -1 && prereleaseStop != -1 {
prerelease = input[prereleaseStart:prereleaseStop]
}
return versionStart,
version,
prerelease
return versionStart, version, prerelease
}
@@ -1,7 +1,6 @@
package nix
import (
"path"
"testing"
"github.com/stretchr/testify/assert"
@@ -125,13 +124,21 @@ func Test_findVersionIsh(t *testing.T) {
wantVersion: "unstable-2021-08-16",
wantPreRelease: "",
},
{
name: "version with release suffix and no output name",
input: "/nix/store/02mqs1by2vab9yzw0qc4j7463w78p3ps-glibc-2.37-8",
wantIdx: 50,
wantVersion: "2.37-8",
wantPreRelease: "8",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
gotIdx, gotVersion, gotPreRelease := findVersionIsh(tt.input)
assert.Equal(t, tt.wantIdx, gotIdx)
assert.Equal(t, tt.wantVersion, gotVersion)
assert.Equal(t, tt.wantPreRelease, gotPreRelease)
assert.Equal(t, tt.wantIdx, gotIdx, "bad index")
assert.Equal(t, tt.wantVersion, gotVersion, "bad version")
assert.Equal(t, tt.wantPreRelease, gotPreRelease, "bad pre-release")
})
}
}
@@ -139,121 +146,132 @@ func Test_findVersionIsh(t *testing.T) {
func Test_parseNixStorePath(t *testing.T) {
tests := []struct {
source string
want *nixStorePath
name string
want *nixStorePath
}{
{
source: "/nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin",
name: "/nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin",
want: &nixStorePath{
outputHash: "h0cnbmfcn93xm5dg2x27ixhag1cwndga",
name: "glibc",
version: "2.34-210",
output: "bin",
OutputHash: "h0cnbmfcn93xm5dg2x27ixhag1cwndga",
Name: "glibc",
Version: "2.34-210",
Output: "bin",
},
},
{
source: "/nix/store/0296qxvn30z9b2ah1g5p97k5wr9k8y78-busybox-static-x86_64-unknown-linux-musl-1.35.0",
name: "/nix/store/02mqs1by2vab9yzw0qc4j7463w78p3ps-glibc-2.37-8",
want: &nixStorePath{
outputHash: "0296qxvn30z9b2ah1g5p97k5wr9k8y78",
name: "busybox-static-x86_64-unknown-linux-musl",
version: "1.35.0",
OutputHash: "02mqs1by2vab9yzw0qc4j7463w78p3ps",
Name: "glibc",
Version: "2.37-8",
},
},
{
source: "/nix/store/5zzrvdmlkc5rh3k5862krd3wfb3pqhyf-perl5.34.1-TimeDate-2.33",
name: "/nix/store/0296qxvn30z9b2ah1g5p97k5wr9k8y78-busybox-static-x86_64-unknown-linux-musl-1.35.0",
want: &nixStorePath{
outputHash: "5zzrvdmlkc5rh3k5862krd3wfb3pqhyf",
name: "perl5.34.1-TimeDate",
version: "2.33",
OutputHash: "0296qxvn30z9b2ah1g5p97k5wr9k8y78",
Name: "busybox-static-x86_64-unknown-linux-musl",
Version: "1.35.0",
},
},
{
source: "/nix/store/q38q8ng57zwjg1h15ry5zx0lb0xyax4b-libcap-2.63-lib",
name: "/nix/store/5zzrvdmlkc5rh3k5862krd3wfb3pqhyf-perl5.34.1-TimeDate-2.33",
want: &nixStorePath{
outputHash: "q38q8ng57zwjg1h15ry5zx0lb0xyax4b",
name: "libcap",
version: "2.63",
output: "lib",
OutputHash: "5zzrvdmlkc5rh3k5862krd3wfb3pqhyf",
Name: "perl5.34.1-TimeDate",
Version: "2.33",
},
},
{
source: "/nix/store/p0y8fbpbqr2jm5zfrdll0rgyg2lvp5g2-util-linux-minimal-2.37.4-bin",
name: "/nix/store/q38q8ng57zwjg1h15ry5zx0lb0xyax4b-libcap-2.63-lib",
want: &nixStorePath{
outputHash: "p0y8fbpbqr2jm5zfrdll0rgyg2lvp5g2",
name: "util-linux-minimal",
version: "2.37.4",
output: "bin",
OutputHash: "q38q8ng57zwjg1h15ry5zx0lb0xyax4b",
Name: "libcap",
Version: "2.63",
Output: "lib",
},
},
{
source: "/nix/store/z24qs6f5d1mmwdp73n1jfc3swj4v2c5s-krb5-1.19.3.9.10",
name: "/nix/store/p0y8fbpbqr2jm5zfrdll0rgyg2lvp5g2-util-linux-minimal-2.37.4-bin",
want: &nixStorePath{
outputHash: "z24qs6f5d1mmwdp73n1jfc3swj4v2c5s",
name: "krb5",
version: "1.19.3.9.10",
OutputHash: "p0y8fbpbqr2jm5zfrdll0rgyg2lvp5g2",
Name: "util-linux-minimal",
Version: "2.37.4",
Output: "bin",
},
},
{
source: "/nix/store/zkgyp2vra0bgqm0dv1qi514l5fd0aksx-bash-interactive-5.1-p16-man",
name: "/nix/store/z24qs6f5d1mmwdp73n1jfc3swj4v2c5s-krb5-1.19.3.9.10",
want: &nixStorePath{
outputHash: "zkgyp2vra0bgqm0dv1qi514l5fd0aksx",
name: "bash-interactive",
version: "5.1-p16",
output: "man",
OutputHash: "z24qs6f5d1mmwdp73n1jfc3swj4v2c5s",
Name: "krb5",
Version: "1.19.3.9.10",
},
},
{
source: "/nix/store/nwf2y0nc48ybim56308cr5ccvwkabcqc-openssl-1.1.1q",
name: "/nix/store/zkgyp2vra0bgqm0dv1qi514l5fd0aksx-bash-interactive-5.1-p16-man",
want: &nixStorePath{
outputHash: "nwf2y0nc48ybim56308cr5ccvwkabcqc",
name: "openssl",
version: "1.1.1q",
OutputHash: "zkgyp2vra0bgqm0dv1qi514l5fd0aksx",
Name: "bash-interactive",
Version: "5.1-p16",
Output: "man",
},
},
{
source: "/nix/store/nwv742f1bxv6g78hy9yc6slxdbxlmqhb-kmod-29",
name: "/nix/store/nwf2y0nc48ybim56308cr5ccvwkabcqc-openssl-1.1.1q",
want: &nixStorePath{
outputHash: "nwv742f1bxv6g78hy9yc6slxdbxlmqhb",
name: "kmod",
version: "29",
OutputHash: "nwf2y0nc48ybim56308cr5ccvwkabcqc",
Name: "openssl",
Version: "1.1.1q",
},
},
{
source: "/nix/store/n83qx7m848kg51lcjchwbkmlgdaxfckf-tzdata-2022a",
name: "/nix/store/nwv742f1bxv6g78hy9yc6slxdbxlmqhb-kmod-29",
want: &nixStorePath{
outputHash: "n83qx7m848kg51lcjchwbkmlgdaxfckf",
name: "tzdata",
version: "2022a",
OutputHash: "nwv742f1bxv6g78hy9yc6slxdbxlmqhb",
Name: "kmod",
Version: "29",
},
},
{
source: "'/nix/store/q5dhwzcn82by5ndc7g0q83wsnn13qkqw-webdav-server-rs-unstable-2021-08-16",
name: "/nix/store/n83qx7m848kg51lcjchwbkmlgdaxfckf-tzdata-2022a",
want: &nixStorePath{
outputHash: "q5dhwzcn82by5ndc7g0q83wsnn13qkqw",
name: "webdav-server-rs",
version: "unstable-2021-08-16",
OutputHash: "n83qx7m848kg51lcjchwbkmlgdaxfckf",
Name: "tzdata",
Version: "2022a",
},
},
{
name: "/nix/store/q5dhwzcn82by5ndc7g0q83wsnn13qkqw-webdav-server-rs-unstable-2021-08-16",
want: &nixStorePath{
OutputHash: "q5dhwzcn82by5ndc7g0q83wsnn13qkqw",
Name: "webdav-server-rs",
Version: "unstable-2021-08-16",
},
},
// negative cases...
{
source: "'z33yk02rsr6b4rb56lgb80bnvxx6yw39-?id=21ee35dde73aec5eba35290587d479218c6dd824.drv'",
name: "'z33yk02rsr6b4rb56lgb80bnvxx6yw39-?id=21ee35dde73aec5eba35290587d479218c6dd824.drv'",
},
{
source: "/nix/store/yzahni8aig6mdrvcsccgwm2515lcpi5q-git-minimal-2.36.0.drv",
name: "/nix/store/yzahni8aig6mdrvcsccgwm2515lcpi5q-git-minimal-2.36.0.drv",
},
{
source: "/nix/store/z9yvxs0s3xdkp5jgmzis4g50bfq3dgvm-0018-pkg-config-derive-prefix-from-prefix.patch",
name: "/nix/store/z9yvxs0s3xdkp5jgmzis4g50bfq3dgvm-0018-pkg-config-derive-prefix-from-prefix.patch",
},
{
source: "/nix/store/w3hl7zrmc9qvzadc0k7cp9ysxiyz88j6-base-system",
name: "/nix/store/w3hl7zrmc9qvzadc0k7cp9ysxiyz88j6-base-system",
},
{
source: "/nix/store/zz1lc28x25fcx6al6xwk3dk8kp7wx47y-Test-RequiresInternet-0.05.tar.gz.drv",
name: "/nix/store/zz1lc28x25fcx6al6xwk3dk8kp7wx47y-Test-RequiresInternet-0.05.tar.gz.drv",
},
}
for _, tt := range tests {
t.Run(path.Base(tt.source), func(t *testing.T) {
assert.Equal(t, tt.want, parseNixStorePath(tt.source))
t.Run(tt.name, func(t *testing.T) {
if tt.want != nil {
tt.want.StorePath = tt.name
}
assert.Equal(t, tt.want, parseNixStorePath(tt.name))
})
}
}
+196
View File
@@ -0,0 +1,196 @@
/*
Package nix provides a concrete Cataloger implementation for packages within the Nix packaging ecosystem.
*/
package nix
import (
"context"
"errors"
"fmt"
"github.com/bmatcuk/doublestar/v4"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/unknown"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
// storeCataloger finds package outputs installed in the Nix store location (/nix/store/*).
type storeCataloger struct {
config Config
name string
}
// NewStoreCataloger returns a new cataloger object initialized for Nix store files.
// Deprecated: please use NewCataloger instead
func NewStoreCataloger() pkg.Cataloger {
return newStoreCataloger(Config{CaptureOwnedFiles: true}, "nix-store-cataloger")
}
func newStoreCataloger(cfg Config, name string) storeCataloger {
return storeCataloger{
config: cfg,
name: name,
}
}
func (c storeCataloger) Name() string {
return c.name
}
func (c storeCataloger) Catalog(ctx context.Context, resolver file.Resolver) ([]pkg.Package, []artifact.Relationship, error) {
prototypes, err := c.findPackagesFromStore(ctx, resolver)
if err != nil {
return nil, nil, fmt.Errorf("failed to find nix packages: %w", err)
}
drvs, err := c.findDerivationsFromStore(resolver, prototypes)
if err != nil {
// preserve unknown errors, but suppress would-be fatal errors
var cErr *unknown.CoordinateError
if !errors.As(err, &cErr) {
// let's ignore fatal errors from this path, since it only enriches packages
drvs = newDerivations()
err = nil
log.WithFields("error", err).Trace("failed to find nix derivations")
}
}
pkgs, rels := c.finalizeStorePackages(prototypes, drvs)
return pkgs, rels, err
}
func (c storeCataloger) finalizeStorePackages(pkgPrototypes []nixStorePackage, drvs *derivations) ([]pkg.Package, []artifact.Relationship) {
var pkgs []pkg.Package
var pkgByStorePath = make(map[string]pkg.Package)
for _, pp := range pkgPrototypes {
if pp.Location == nil {
continue
}
p := newNixStorePackage(pp, c.name)
pkgs = append(pkgs, p)
pkgByStorePath[pp.Location.RealPath] = p
}
var relationships []artifact.Relationship
for storePath, p := range pkgByStorePath {
deps := drvs.findDependencies(storePath)
for _, dep := range deps {
if depPkg, ok := pkgByStorePath[dep]; ok {
relationships = append(relationships, artifact.Relationship{
From: depPkg,
To: p,
Type: artifact.DependencyOfRelationship,
})
}
}
}
return pkgs, relationships
}
func (c storeCataloger) findDerivationsFromStore(resolver file.Resolver, pkgPrototypes []nixStorePackage) (*derivations, error) {
locs, err := resolver.FilesByGlob("**/nix/store/*.drv")
if err != nil {
return nil, fmt.Errorf("failed to find derivations: %w", err)
}
var errs error
dvs := newDerivations()
for _, loc := range locs {
d, err := newDerivationFromLocation(loc, resolver)
if err != nil {
errs = unknown.Append(errs, loc.Coordinates, err)
continue
}
if d == nil {
continue
}
dvs.add(*d)
}
// attach derivations to the packages they belong to
for i := range pkgPrototypes {
p := &pkgPrototypes[i]
p.derivationFile = dvs.findDerivationForOutputPath(p.Location.RealPath)
}
return dvs, errs
}
func (c storeCataloger) findPackagesFromStore(ctx context.Context, resolver file.Resolver) ([]nixStorePackage, error) {
// we want to search for only directories, which isn't possible via the stereoscope API, so we need to apply the glob manually on all returned paths
var prototypes []nixStorePackage
var filesByStorePath = make(map[string]*file.LocationSet)
ctx, cancel := context.WithCancel(ctx)
defer cancel()
for location := range resolver.AllLocations(ctx) {
matchesStorePath, err := doublestar.Match("**/nix/store/*", location.RealPath)
if err != nil {
return nil, fmt.Errorf("failed to match nix store path: %w", err)
}
parentStorePath := findParentNixStorePath(location.RealPath)
if c.config.CaptureOwnedFiles && parentStorePath != "" {
fileInfo, err := resolver.FileMetadataByLocation(location)
if err != nil {
log.WithFields("path", location.RealPath).Trace("failed to get file metadata")
continue
}
if fileInfo.IsDir() {
// we should only add non-directories to the file set
continue
}
if _, ok := filesByStorePath[parentStorePath]; !ok {
s := file.NewLocationSet()
filesByStorePath[parentStorePath] = &s
}
filesByStorePath[parentStorePath].Add(location)
}
if !matchesStorePath {
continue
}
storePath := parseNixStorePath(location.RealPath)
if storePath == nil || !storePath.isValidPackage() {
continue
}
prototypes = append(prototypes, nixStorePackage{
Location: &location,
nixStorePath: *storePath,
})
}
// add file sets to packages
for i := range prototypes {
p := &prototypes[i]
if p.Location == nil {
log.WithFields("package", p.nixStorePath.Name).Debug("nix package has no evidence locations associated")
continue
}
parentStorePath := p.Location.RealPath
files, ok := filesByStorePath[parentStorePath]
if !ok {
log.WithFields("path", parentStorePath, "nix-store-path", parentStorePath).Debug("found a nix store file for a non-existent package")
continue
}
p.Files = filePaths(files.ToSlice())
}
return prototypes, nil
}
func filePaths(files []file.Location) []string {
var relativePaths []string
for _, f := range files {
relativePaths = append(relativePaths, f.RealPath)
}
return relativePaths
}
@@ -0,0 +1,183 @@
package nix
import (
"testing"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
func TestStoreCataloger_Image(t *testing.T) {
tests := []struct {
fixture string
wantPkgs []string
wantRel []string
}{
{
// $ nix-store -q --tree $(which jq)
//
// /nix/store/nzwfgsp28vgxv7n2gl5fxqkca9awh4dz-jq-1.6-bin3.4
// ├───/nix/store/02mqs1by2vab9yzw0qc4j7463w78p3ps-glibc-2.37-8
// │ ├───/nix/store/cw8fpl8r1x9rmaqj55fwbfnnrgw7b40k-libidn2-2.3.4
// │ │ ├───/nix/store/h1ysk4vvw48winwmh38rvnsj0dlsz7c1-libunistring-1.1
// │ │ │ └───/nix/store/h1ysk4vvw48winwmh38rvnsj0dlsz7c1-libunistring-1.1 [...]
// │ │ └───/nix/store/cw8fpl8r1x9rmaqj55fwbfnnrgw7b40k-libidn2-2.3.4 [...]
// │ ├───/nix/store/fmz62d844wf4blb11k21f4m0q6n6hdfp-xgcc-12.3.0-libgcc
// │ └───/nix/store/02mqs1by2vab9yzw0qc4j7463w78p3ps-glibc-2.37-8 [...]
// ├───/nix/store/mzj90j6m3c3a1vv8j9pl920f98i2yz9q-oniguruma-6.9.8-lib
// │ ├───/nix/store/02mqs1by2vab9yzw0qc4j7463w78p3ps-glibc-2.37-8 [...]
// │ └───/nix/store/mzj90j6m3c3a1vv8j9pl920f98i2yz9q-oniguruma-6.9.8-lib [...]
// └───/nix/store/1x3s2v9wc9m302cspfqcn2iwar0b5w99-jq-1.6-lib
// ├───/nix/store/02mqs1by2vab9yzw0qc4j7463w78p3ps-glibc-2.37-8 [...]
// ├───/nix/store/mzj90j6m3c3a1vv8j9pl920f98i2yz9q-oniguruma-6.9.8-lib [...]
// └───/nix/store/1x3s2v9wc9m302cspfqcn2iwar0b5w99-jq-1.6-lib [...]
fixture: "image-nixos-jq-pkg-store",
wantPkgs: []string{
"glibc @ 2.37-8 (/nix/store/aw2fw9ag10wr9pf0qk4nk5sxi0q0bn56-glibc-2.37-8)",
"jq @ 1.6 (/nix/store/3xpzpmcqmzsdblkzqa9d9s6l302pnk4g-jq-1.6-lib)", // jq lib output
"jq @ 1.6 (/nix/store/aj8lqifsyynq8iknivvxkrsqnblj7qzs-jq-1.6-bin)", // jq bin output
"libidn2 @ 2.3.4 (/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4)",
"libunistring @ 1.1 (/nix/store/s2gi8pfjszy6rq3ydx0z1vwbbskw994i-libunistring-1.1)",
"oniguruma @ 6.9.8 (/nix/store/dpcyirvyblnflf7cp14dnr1420va93zx-oniguruma-6.9.8-lib)",
"xgcc @ 12.3.0 (/nix/store/jbwb8d8l28lg9z0xzl784wyb9vlbwss6-xgcc-12.3.0-libgcc)",
},
wantRel: []string{
// note: parsing all relationships from only derivations results in partial results! (this is why the DB cataloger exists)
"libidn2 @ 2.3.4 (/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4) [dependency-of] glibc @ 2.37-8 (/nix/store/aw2fw9ag10wr9pf0qk4nk5sxi0q0bn56-glibc-2.37-8)",
"libunistring @ 1.1 (/nix/store/s2gi8pfjszy6rq3ydx0z1vwbbskw994i-libunistring-1.1) [dependency-of] libidn2 @ 2.3.4 (/nix/store/k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4)",
"xgcc @ 12.3.0 (/nix/store/jbwb8d8l28lg9z0xzl784wyb9vlbwss6-xgcc-12.3.0-libgcc) [dependency-of] glibc @ 2.37-8 (/nix/store/aw2fw9ag10wr9pf0qk4nk5sxi0q0bn56-glibc-2.37-8)",
},
},
}
for _, tt := range tests {
t.Run(tt.fixture, func(t *testing.T) {
c := NewStoreCataloger()
pkgtest.NewCatalogTester().
WithImageResolver(t, tt.fixture).
ExpectsPackageStrings(tt.wantPkgs).
ExpectsRelationshipStrings(tt.wantRel).
TestCataloger(t, c)
})
}
}
func TestStoreCataloger_Directory(t *testing.T) {
tests := []struct {
fixture string
wantPkgs []pkg.Package
wantRel []artifact.Relationship
}{
{
fixture: "test-fixtures/fixture-1",
wantPkgs: []pkg.Package{
{
Name: "glibc",
Version: "2.34-210",
PURL: "pkg:nix/glibc@2.34-210?drvpath=5av396z8xa13jg89g9jws145c0k26k2x-glibc-2.34-210.drv&output=bin&outputhash=h0cnbmfcn93xm5dg2x27ixhag1cwndga",
Locations: file.NewLocationSet(
file.NewLocation("nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin").WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
file.NewLocation("nix/store/5av396z8xa13jg89g9jws145c0k26k2x-glibc-2.34-210.drv").WithAnnotation(pkg.EvidenceAnnotationKey, pkg.SupportingEvidenceAnnotation),
),
FoundBy: "nix-store-cataloger",
Type: pkg.NixPkg,
Metadata: pkg.NixStoreEntry{
Path: "/nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin",
Derivation: pkg.NixDerivation{
Path: "nix/store/5av396z8xa13jg89g9jws145c0k26k2x-glibc-2.34-210.drv",
System: "aarch64-linux",
InputDerivations: []pkg.NixDerivationReference{
{
Path: "/nix/store/1zi0k7y01rhqr2gfqb42if0icswg65sj-locale-C.diff.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/45j86ggi8mlpfslcrgvjf7m6phia21fp-raw.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/4fnfsd9sc7bam6886hwyaprdsww66dg3-bison-3.8.2.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/51azdrrvcqrk2hbky7ryphlwd99yz25d-linux-headers-5.18.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/67s0qc21gyarmdwc181bqmjc3qzv8zkz-libidn2-2.3.2.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/9rhliwskh3mrrs5nfzgz0x6wrccyfg7k-bootstrap-stage0-glibc-bootstrap.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/cl1wcw2v1ifzjlkzi50h32a6lms9m25s-binutils-2.38.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/ghjc8bkfk8lh53z14mk2nk7h059zh7vx-python3-minimal-3.10.5.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/k3786wfzw637r7sylccdmm92saqp73d8-glibc-2.34.tar.xz.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/l5zr5m1agvvnic49fg6qc44g5fgj3la1-glibc-reinstate-prlimit64-fallback.patch?id=eab07e78b691ae7866267fc04d31c7c3ad6b0eeb.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/mf5kz6d01ab8h0rswzyr04mbcd6g5x9n-bootstrap-stage2-stdenv-linux.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/nd1zy67vp028707pbh466qhrfqh4cpq6-bootstrap-stage2-gcc-wrapper-.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/ra77ww7p2xx8jh8n4m9vmj6wc8wxijdb-bootstrap-tools.drv",
Outputs: []string{"out"},
},
{
Path: "/nix/store/wlldapf5bg58kivw520ll5bw0fmlaid7-raw.drv",
Outputs: []string{"out"},
},
},
InputSources: []string{
"/nix/store/001gp43bjqzx60cg345n2slzg7131za8-nix-nss-open-files.patch",
"/nix/store/7kw224hdyxd7115lrqh9a4dv2x8msq2s-fix-x64-abi.patch",
"/nix/store/8haph3ng4mgsqr6p4024vj8k6kg3mqc4-nix-locale-archive.patch",
"/nix/store/95hp6hs9g73h93safadb8x6vajyqkv6q-0001-Revert-Remove-all-usage-of-BASH-or-BASH-in-installed.patch",
"/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh",
"/nix/store/b1w7zbvm39ff1i52iyjggyvw2rdxz104-dont-use-system-ld-so-cache.patch",
"/nix/store/ikmqczy0y20n04a2b8qfflzwihv8139g-separate-debug-info.sh",
"/nix/store/mgx19wbmgrh3rblbxhs6vi47sha15n11-2.34-master.patch.gz",
"/nix/store/mnglr8rr7nl444h7p50ysyq8qd0fm1lm-dont-use-system-ld-so-preload.patch",
"/nix/store/xkd50xxii6k7l1kmw4l5x6xzbhamcs87-allow-kernel-2.6.32.patch",
"/nix/store/za0pg7fmysrcwrqcal26fnmzw6vycgdn-fix_path_attribute_in_getconf.patch",
},
},
OutputHash: "h0cnbmfcn93xm5dg2x27ixhag1cwndga",
Output: "bin",
Files: []string{
// the legacy cataloger captures files by default
"nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin/lib/glibc.so",
"nix/store/h0cnbmfcn93xm5dg2x27ixhag1cwndga-glibc-2.34-210-bin/share/man/glibc.1",
},
},
},
},
},
}
for _, tt := range tests {
t.Run(tt.fixture, func(t *testing.T) {
c := NewStoreCataloger()
pkgtest.NewCatalogTester().
FromDirectory(t, tt.fixture).
Expects(tt.wantPkgs, tt.wantRel).
TestCataloger(t, c)
})
}
}
@@ -0,0 +1,43 @@
FROM --platform=linux/amd64 nixos/nix:2.28.2@sha256:4215204b5f65c7b756b26a6dd47a6af77f1d906e5edf62b184c95420a7dfa08f AS builder
# cross-platform builds cannot use bpf features
RUN mkdir -p /etc/nix && \
echo 'filter-syscalls = false' > /etc/nix/nix.conf && \
echo 'experimental-features = nix-command flakes' >> /etc/nix/nix.conf
# pin Nixpkgs to a specific commit (2023.11.17)
RUN mkdir -p /root/nix && \
echo 'import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/46688f8eb5.tar.gz") {}' > /root/nix/pinned-nixpkgs.nix
# install jq + sqlite using the pinned Nixpkgs
RUN nix-env -f /root/nix/pinned-nixpkgs.nix -iA jq sqlite
COPY clean_db.sql /tmp/clean_db.sql
RUN echo "path" > /tmp/required_paths.txt
RUN . /root/.nix-profile/etc/profile.d/nix.sh && \
PAGER='' nix-store -q --requisites $(which jq) >> /tmp/required_paths.txt
RUN sqlite3 /nix/var/nix/db/db.sqlite "CREATE TEMP TABLE IF NOT EXISTS RequiredPaths (path TEXT PRIMARY KEY);"
RUN sqlite3 /nix/var/nix/db/db.sqlite ".mode list" ".import /tmp/required_paths.txt RequiredPaths"
RUN sqlite3 /nix/var/nix/db/db.sqlite < /tmp/clean_db.sql
# create a directory with only the required dependencies + any derivations
RUN mkdir -p /nix-minimal && \
for dep in $(nix-store -q --requisites $(which jq)); do \
mkdir -p /nix-minimal$(dirname $dep) && \
cp -a $dep /nix-minimal$dep; \
done
# now add all the drv files from the store
RUN for drv in $(find /nix/store -name "*.drv"); do \
mkdir -p /nix-minimal$(dirname $drv) && \
cp -a $drv /nix-minimal$drv; \
done
FROM scratch
# get packages + relationships from here
COPY --from=builder /nix/var/nix/db/db.sqlite /nix/var/nix/db/db.sqlite
# get files owned by each package here
COPY --from=builder /nix-minimal/nix/store /nix/store
@@ -0,0 +1,15 @@
-- Delete DerivationOutputs where path is not in RequiredPaths
DELETE FROM DerivationOutputs
WHERE path NOT IN (SELECT path FROM RequiredPaths);
-- Delete ValidPaths where path is not in RequiredPaths
DELETE FROM ValidPaths
WHERE path NOT IN (SELECT path FROM RequiredPaths);
DELETE FROM Refs
WHERE referrer NOT IN (SELECT id FROM ValidPaths WHERE path IN (SELECT path FROM RequiredPaths))
OR reference NOT IN (SELECT id FROM ValidPaths WHERE path IN (SELECT path FROM RequiredPaths));
-- Run VACUUM to clean up the database file
VACUUM;
@@ -0,0 +1,30 @@
FROM --platform=linux/amd64 nixos/nix:2.28.2@sha256:4215204b5f65c7b756b26a6dd47a6af77f1d906e5edf62b184c95420a7dfa08f AS builder
# cross-platform builds cannot use bpf features
RUN mkdir -p /etc/nix && \
echo 'filter-syscalls = false' > /etc/nix/nix.conf && \
echo 'experimental-features = nix-command flakes' >> /etc/nix/nix.conf
# pin Nixpkgs to a specific commit (2023.11.17)
RUN mkdir -p /root/nix && \
echo 'import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/46688f8eb5.tar.gz") {}' > /root/nix/pinned-nixpkgs.nix
# install jq using the pinned Nixpkgs
RUN nix-env -f /root/nix/pinned-nixpkgs.nix -iA jq
# create a directory with only the required dependencies + any derivations
RUN mkdir -p /nix-minimal && \
for dep in $(nix-store -q --requisites $(which jq)); do \
mkdir -p /nix-minimal$(dirname $dep) && \
cp -a $dep /nix-minimal$dep; \
done
# now add all the drv files from the store
RUN for drv in $(find /nix/store -name "*.drv"); do \
mkdir -p /nix-minimal$(dirname $drv) && \
cp -a $drv /nix-minimal$drv; \
done
FROM scratch
COPY --from=builder /nix-minimal/nix/store /nix/store
+10 -2
View File
@@ -23,8 +23,16 @@ func NewComposerLockCataloger() pkg.Cataloger {
WithParserByGlobs(parseComposerLock, "**/composer.lock")
}
// NewPeclCataloger returns a new cataloger for PHP PECL metadata“.
// NewPearCataloger returns a new cataloger for PHP Pear metadata (including Pecl metadata).
func NewPearCataloger() pkg.Cataloger {
return generic.NewCataloger("php-pear-serialized-cataloger").
WithParserByGlobs(parsePear, "**/php/.registry/**/*.reg")
}
// NewPeclCataloger returns a new cataloger for PHP Pecl metadata. Note: this will also catalog Pear metadata so should
// not be used in conjunction with the Pear Cataloger.
// Deprecated: please use NewPearCataloger instead.
func NewPeclCataloger() pkg.Cataloger {
return generic.NewCataloger("php-pecl-serialized-cataloger").
WithParserByGlobs(parsePeclSerialized, "**/php/.registry/.channel.*/*.reg")
WithParserByGlobs(parsePecl, "**/php/.registry/.channel.*/*.reg")
}

Some files were not shown because too many files have changed in this diff Show More