Compare commits

..
13 Commits
Author SHA1 Message Date
Keith Zantow f4734d28b3 Fix panic when CycloneDX BOM missing metadata.component (#895) 2022-03-17 10:22:35 -04:00
Jonas Xavier 6ef3e45ffc Use go 1.18 buildinfo to catalog binaries (#827)
* initial working version

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* added build settings to pkg metadata

wip - unit tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* handle mach-O FatFiles

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add support to mod replace

fixed golang catalger tests

trying GH Actions with go 1.18rc1

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* log error

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* use go-macholibre for extraction

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* cleaner tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add version to main module

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* check macho file with macholibre

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* run golangci in its own workflow

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci workflow

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix golangci wf yml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix golangci wf yml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci wf

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci wf

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* get arch from bin file headers

upgrade macholibre

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* test new stereoscope lazy reader interface

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove devel version from golang cataloger

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* switch github workflows to go1.18 stable

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add union reader interface in golang cataloger

update stereoscope

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* simpler golangci validation

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix makefile

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* get archs refactor

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* nolint for golang version

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix go bin tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* feedback changes

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* golangci nolint needs a \n before package

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* cleanup

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* move golangci-lint to its own jobs again

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix ci yaml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add support for xcoff files

add arch assets to test bin file types

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* clean up golangci-lint config

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* nolint for xcoff

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explain nolints

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove unused xcoff testdata assets

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* make go bin test-fixtures in docker

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix make clean with -f

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update json output schema

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update schema version in test fixture

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* feedback changes

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explain possible empty main module

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2022-03-16 17:07:02 -07:00
Keith Zantow ee0a1d172c panic parsing cyclonedx (#892) 2022-03-16 09:10:44 -04:00
Keith Zantow b9b3ccecf9 Update register link text (#891) 2022-03-15 17:02:33 +00:00
mikey strauss 95271fb10d NPM PURLs are invalid (#832)
Signed-off-by: houdini91 <mdstrauss91@gmail.com>
2022-03-15 11:54:33 -04:00
Christopher Angelo Phillips 93d2d57cd3 update README with OSS Meetup information (#890)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-03-15 11:37:56 -04:00
Kenny Moens 7cd3201fe9 Support the .be top-level domain (#873)
Signed-off-by: Kenny Moens <kenny.moens@cipalschaubroeck.be>
2022-03-15 10:59:13 -04:00
Keith Zantow c7cf8b0b26 Brew install gon failed (#885) 2022-03-11 13:17:57 -05:00
Frankie G-J 44a6e00f7a Include vendored modules in Go Module package list (#883)
* include vendored modules in package slice

Signed-off-by: Frankie Gallina-Jones <frankieg@vmware.com>

* add explanatory comments

Signed-off-by: Frankie Gallina-Jones <frankieg@vmware.com>
2022-03-11 12:57:33 -05:00
Keith Zantow 6c8102bf28 Correct CycloneDX distro decoding (#745) 2022-03-11 09:27:18 -05:00
Keith Zantow 7789506dc6 Experimental GitHub export (#836) 2022-03-10 22:38:12 -05:00
Christopher Angelo Phillips fa03723617 Upgrade vault api from v1.3.1 to v1.4.1 (#878)
* move v1.3.1 => v1.4.1
* go mod tidy

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-03-09 15:11:07 -05:00
Alex Goodman 2946813a74 RPM Epoch should be optional in the json schema (#880) 2022-03-09 14:51:43 -05:00
444 changed files with 7660 additions and 4320 deletions
+1 -1
View File
@@ -29,7 +29,7 @@ notarize() {
fi
# install gon
which gon || (brew tap mitchellh/gon && brew install mitchellh/gon/gon)
which gon || (go install github.com/mitchellh/gon/cmd/gon@latest)
# create config (note: json via stdin with gon is broken, can only use HCL from file)
hcl_file=$(mktemp).hcl
+13 -2
View File
@@ -9,7 +9,7 @@ on:
- "v*"
env:
GO_VERSION: "1.17.x"
GO_VERSION: "1.18.x"
jobs:
quality-gate:
@@ -34,6 +34,16 @@ jobs:
checkName: "Static analysis"
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Check static analysis results - golangci-lint
uses: fountainhead/action-wait-for-check@v1.0.0
id: static-analysis-golangci-lint
with:
token: ${{ secrets.GITHUB_TOKEN }}
# This check name is defined as the github action job name (in .github/workflows/testing.yaml)
checkName: "Static analysis - golangci-lint"
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Check unit test results
uses: fountainhead/action-wait-for-check@v1.0.0
id: unit
@@ -80,9 +90,10 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Quality gate
if: steps.static-analysis.outputs.conclusion != 'success' || steps.unit.outputs.conclusion != 'success' || steps.integration.outputs.conclusion != 'success' || steps.cli-linux.outputs.conclusion != 'success' || steps.acceptance-linux.outputs.conclusion != 'success' || steps.acceptance-mac.outputs.conclusion != 'success'
if: steps.static-analysis.outputs.conclusion != 'success' || steps.static-analysis-golangci-lint.outputs.conclusion != 'success' || steps.unit.outputs.conclusion != 'success' || steps.integration.outputs.conclusion != 'success' || steps.cli-linux.outputs.conclusion != 'success' || steps.acceptance-linux.outputs.conclusion != 'success' || steps.acceptance-mac.outputs.conclusion != 'success'
run: |
echo "Static Analysis Status: ${{ steps.static-analysis.conclusion }}"
echo "Static Analysis Status - Golangci-Lint: ${{ steps.static-analysis-golangci-lint.conclusion }}"
echo "Unit Test Status: ${{ steps.unit.outputs.conclusion }}"
echo "Integration Test Status: ${{ steps.integration.outputs.conclusion }}"
echo "Acceptance Test (Linux) Status: ${{ steps.acceptance-linux.outputs.conclusion }}"
+57 -1
View File
@@ -7,9 +7,49 @@ on:
pull_request:
env:
GO_VERSION: "1.17.x"
GO_VERSION: "1.18.x"
GO_STABLE_VERSION: true
jobs:
# TODO: remove this job once golanci-lint is compatible with g01.18+
Static-Analysis-Golanci-lint:
name: "Static analysis - golangci-lint"
runs-on: ubuntu-20.04
steps:
- uses: actions/setup-go@v2
with:
go-version: "1.17" # NOTE: please use GO_VERSION once golangci supports go1.18+
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v2.1.3
with:
path: ${{ github.workspace }}/.tmp
key: ${{ runner.os }}-tool-${{ hashFiles('Makefile') }}
- name: Restore go cache
id: go-cache
uses: actions/cache@v2.1.3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-
- name: (cache-miss) Bootstrap all project dependencies
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
- name: Bootstrap CI environment dependencies
run: make ci-bootstrap
- name: Run static analysis
run: make static-analysis-golanci-lint
Static-Analysis:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
@@ -19,6 +59,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -56,6 +97,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -92,6 +134,16 @@ jobs:
path: syft/pkg/cataloger/java/test-fixtures/java-builds/packages
key: ${{ runner.os }}-unit-java-cache-${{ hashFiles( 'syft/pkg/cataloger/java/test-fixtures/java-builds/packages.fingerprint' ) }}
- name: Build cache key for go binary test-fixture blobs (for unit tests)
run: make go-binaries-fingerprint
- name: Restore Go binary test-fixture cache
id: unit-go-binary-cache
uses: actions/cache@v2.1.3
with:
path: syft/pkg/cataloger/golang/test-fixtures/archs/binaries
key: ${{ runner.os }}-unit-go-binaries-cache-${{ hashFiles( 'syft/pkg/cataloger/golang/test-fixtures/archs/binaries.fingerprint' ) }}
- name: Run unit tests
run: make unit
@@ -108,6 +160,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -159,6 +212,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -231,6 +285,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
@@ -338,6 +393,7 @@ jobs:
- uses: actions/setup-go@v2
with:
go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
+14 -3
View File
@@ -122,7 +122,12 @@ bootstrap: $(RESULTSDIR) bootstrap-go bootstrap-tools ## Download and install al
$(call title,Bootstrapping dependencies)
.PHONY: static-analysis
static-analysis: lint check-go-mod-tidy check-licenses
static-analysis: check-go-mod-tidy check-licenses
# NOTE: isolating golanci-lint so it runs over go1.17 in CI, since it is not compatible with
# go1.18+
.PHONY: static-analysis-golanci-lint
static-analysis-golanci-lint: lint
.PHONY: lint
lint: ## Run gofmt + golangci lint checks
@@ -211,13 +216,19 @@ integration-fingerprint:
.PHONY: java-packages-fingerprint
java-packages-fingerprint:
$(call title,Java test fixture fingerprint)
cd syft/cataloger/packages/java/test-fixtures/java-builds && \
cd syft/pkg/cataloger/java/test-fixtures/java-builds && \
make packages.fingerprint
.PHONY: go-binaries-fingerprint
go-binaries-fingerprint:
$(call title,Go binaries test fixture fingerprint)
cd syft/pkg/cataloger/golang/test-fixtures/archs && \
make binaries.fingerprint
.PHONY: fixtures
fixtures:
$(call title,Generating test fixtures)
cd syft/cataloger/packages/java/test-fixtures/java-builds && make
cd syft/pkg/cataloger/java/test-fixtures/java-builds && make
.PHONY: generate-json-schema
generate-json-schema: ## Generate a new json schema
+8
View File
@@ -11,6 +11,14 @@
A CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner tool like [Grype](https://github.com/anchore/grype).
### Join our Virtual OSS Meetup!
You are invited to join us on March 23rd, 11AM-Noon PT for our virtual open source meetup.
Hosts Priyanka “Pinky” Ravi of Weaveworks and Dan Luhring from Anchore OSS will explore how open source project Flux is utilizing Syft-generated SBOMs to secure the software supply chain.
[Register here!](https://get.anchore.com/anchore-oss-meetup-mar-23-2022/)
### Join our community meetings!
- Calendar: https://calendar.google.com/calendar/u/0/r?cid=Y182OTM4dGt0MjRtajI0NnNzOThiaGtnM29qNEBncm91cC5jYWxlbmRhci5nb29nbGUuY29t
+8 -19
View File
@@ -136,12 +136,12 @@ func attestExec(ctx context.Context, _ *cobra.Command, args []string) error {
return fmt.Errorf("could not generate source input for attest command: %w", err)
}
switch si.Type {
case source.ImageType, source.UnknownType:
switch si.Scheme {
case source.ImageScheme, source.UnknownScheme:
// at this point we know that it cannot be dir: or file: schemes, so we will assume that the unknown scheme could represent an image
si.Type = source.ImageType
si.Scheme = source.ImageScheme
default:
return fmt.Errorf("attest command can only be used with image sources but discovered %q when given %q", si.Type, userInput)
return fmt.Errorf("attest command can only be used with image sources but discovered %q when given %q", si.Scheme, userInput)
}
// if the original detection was from a local daemon we want to short circuit
@@ -188,37 +188,26 @@ func attestExec(ctx context.Context, _ *cobra.Command, args []string) error {
)
}
func attestationExecWorker(si source.Input, format sbom.Format, predicateType string, sv *sign.SignerVerifier) <-chan error {
func attestationExecWorker(sourceInput source.Input, format sbom.Format, predicateType string, sv *sign.SignerVerifier) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
catalogingOpts, err := appConfig.ToCatalogingOptions()
if err != nil {
errs <- err
return
}
src, cleanup, err := source.NewFromRegistry(si, appConfig.Registry.ToOptions(), appConfig.Exclusions)
src, cleanup, err := source.NewFromRegistry(sourceInput, appConfig.Registry.ToOptions(), appConfig.Exclusions)
if cleanup != nil {
defer cleanup()
}
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", si.UserInput, err)
errs <- fmt.Errorf("failed to construct source from user input %q: %w", sourceInput.UserInput, err)
return
}
s, err := syft.Catalog(src, catalogingOpts...)
s, err := generateSBOM(src, errs)
if err != nil {
errs <- err
return
}
if s == nil {
errs <- fmt.Errorf("no SBOM produced for %q", si.UserInput)
return
}
sbomBytes, err := syft.Encode(*s, format)
if err != nil {
errs <- err
+2
View File
@@ -22,6 +22,8 @@ func formatAliases(ids ...sbom.FormatID) (aliases []string) {
aliases = append(aliases, "cyclonedx-xml")
case syft.CycloneDxJSONFormatID:
aliases = append(aliases, "cyclonedx-json")
case syft.GitHubID:
aliases = append(aliases, "github", "github-json")
default:
aliases = append(aliases, string(id))
}
+48 -21
View File
@@ -13,8 +13,11 @@ import (
"github.com/anchore/syft/internal/formats/table"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/pkg/cataloger"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/pkg/profile"
@@ -94,7 +97,7 @@ func init() {
func setPackageFlags(flags *pflag.FlagSet) {
// Formatting & Input options //////////////////////////////////////////////
flags.StringP(
"scope", "s", syft.DefaultCatalogingConfig().DefaultScope.String(),
"scope", "s", cataloger.DefaultSearchConfig().Scope.String(),
fmt.Sprintf("selection of layers to catalog, options=%v", source.AllScopes))
flags.StringArrayP(
@@ -112,12 +115,6 @@ func setPackageFlags(flags *pflag.FlagSet) {
"an optional platform specifier for container image sources (e.g. 'linux/arm64', 'linux/arm64/v8', 'arm64', 'linux')",
)
flags.StringArrayP(
"cataloger", "", nil,
// TODO: have a nice way for a user to get the set of catalogers that exist
"set which catalogers to use when generating an SBOM",
)
// Upload options //////////////////////////////////////////////////////////
flags.StringP(
"host", "H", "",
@@ -171,11 +168,7 @@ func bindExclusivePackagesConfigOptions(flags *pflag.FlagSet) error {
// note: output is not included since this configuration option is shared between multiple subcommands
if err := viper.BindPFlag("scope", flags.Lookup("scope")); err != nil {
return err
}
if err := viper.BindPFlag("catalogers", flags.Lookup("cataloger")); err != nil {
if err := viper.BindPFlag("package.cataloger.scope", flags.Lookup("scope")); err != nil {
return err
}
@@ -267,17 +260,42 @@ func isVerbose() (result bool) {
return appConfig.CliOptions.Verbosity > 0 || isPipedInput
}
func generateSBOM(src *source.Source, errs chan error) (*sbom.SBOM, error) {
tasks, err := tasks()
if err != nil {
return nil, err
}
s := sbom.SBOM{
Source: src.Metadata,
Descriptor: sbom.Descriptor{
Name: internal.ApplicationName,
Version: version.FromBuild().Version,
Configuration: appConfig,
},
}
buildRelationships(&s, src, tasks, errs)
return &s, nil
}
func buildRelationships(s *sbom.SBOM, src *source.Source, tasks []task, errs chan error) {
var relationships []<-chan artifact.Relationship
for _, task := range tasks {
c := make(chan artifact.Relationship)
relationships = append(relationships, c)
go runTask(task, &s.Artifacts, src, c, errs)
}
s.Relationships = append(s.Relationships, mergeRelationships(relationships...)...)
}
func packagesExecWorker(si source.Input, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
catalogingOpts, err := appConfig.ToCatalogingOptions()
if err != nil {
errs <- err
return
}
src, cleanup, err := source.New(si, appConfig.Registry.ToOptions(), appConfig.Exclusions)
if cleanup != nil {
defer cleanup()
@@ -287,7 +305,7 @@ func packagesExecWorker(si source.Input, writer sbom.Writer) <-chan error {
return
}
s, err := syft.Catalog(src, catalogingOpts...)
s, err := generateSBOM(src, errs)
if err != nil {
errs <- err
return
@@ -295,7 +313,6 @@ func packagesExecWorker(si source.Input, writer sbom.Writer) <-chan error {
if s == nil {
errs <- fmt.Errorf("no SBOM produced for %q", si.UserInput)
return
}
if appConfig.Anchore.Host != "" {
@@ -313,10 +330,20 @@ func packagesExecWorker(si source.Input, writer sbom.Writer) <-chan error {
return errs
}
func mergeRelationships(cs ...<-chan artifact.Relationship) (relationships []artifact.Relationship) {
for _, c := range cs {
for n := range c {
relationships = append(relationships, n)
}
}
return relationships
}
func runPackageSbomUpload(src *source.Source, s sbom.SBOM) error {
log.Infof("uploading results to %s", appConfig.Anchore.Host)
if src.Metadata.Scheme != source.ImageType {
if src.Metadata.Scheme != source.ImageScheme {
return fmt.Errorf("unable to upload results: only images are supported")
}
+23 -16
View File
@@ -2,7 +2,6 @@ package cmd
import (
"fmt"
"github.com/anchore/syft/syft"
"os"
"github.com/anchore/stereoscope"
@@ -11,6 +10,8 @@ import (
"github.com/anchore/syft/internal/formats/syftjson"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
@@ -105,13 +106,11 @@ func powerUserExecWorker(userInput string, writer sbom.Writer) <-chan error {
go func() {
defer close(errs)
// TODO: replace
//appConfig.Secrets.Cataloger.Enabled = true
//appConfig.FileMetadata.Cataloger.Enabled = true
//appConfig.FileContents.Cataloger.Enabled = true
//appConfig.FileClassification.Cataloger.Enabled = true
catalogingOpts, err := appConfig.ToCatalogingOptions()
appConfig.Secrets.Cataloger.Enabled = true
appConfig.FileMetadata.Cataloger.Enabled = true
appConfig.FileContents.Cataloger.Enabled = true
appConfig.FileClassification.Cataloger.Enabled = true
tasks, err := tasks()
if err != nil {
errs <- err
return
@@ -132,20 +131,28 @@ func powerUserExecWorker(userInput string, writer sbom.Writer) <-chan error {
defer cleanup()
}
s, err := syft.Catalog(src, catalogingOpts...)
if err != nil {
errs <- err
return
s := sbom.SBOM{
Source: src.Metadata,
Descriptor: sbom.Descriptor{
Name: internal.ApplicationName,
Version: version.FromBuild().Version,
Configuration: appConfig,
},
}
if s == nil {
errs <- fmt.Errorf("no SBOM produced for %q", si.UserInput)
return
var relationships []<-chan artifact.Relationship
for _, task := range tasks {
c := make(chan artifact.Relationship)
relationships = append(relationships, c)
go runTask(task, &s.Artifacts, src, c, errs)
}
s.Relationships = append(s.Relationships, mergeRelationships(relationships...)...)
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error { return writer.Write(*s) },
Value: func() error { return writer.Write(s) },
})
}()
+231
View File
@@ -0,0 +1,231 @@
package cmd
import (
"crypto"
"fmt"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
type task func(*sbom.Artifacts, *source.Source) ([]artifact.Relationship, error)
func tasks() ([]task, error) {
var tasks []task
generators := []func() (task, error){
generateCatalogPackagesTask,
generateCatalogFileMetadataTask,
generateCatalogFileDigestsTask,
generateCatalogSecretsTask,
generateCatalogFileClassificationsTask,
generateCatalogContentsTask,
}
for _, generator := range generators {
task, err := generator()
if err != nil {
return nil, err
}
if task != nil {
tasks = append(tasks, task)
}
}
return tasks, nil
}
func generateCatalogPackagesTask() (task, error) {
if !appConfig.Package.Cataloger.Enabled {
return nil, nil
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
packageCatalog, relationships, theDistro, err := syft.CatalogPackages(src, appConfig.Package.ToConfig())
if err != nil {
return nil, err
}
results.PackageCatalog = packageCatalog
results.LinuxDistribution = theDistro
return relationships, nil
}
return task, nil
}
func generateCatalogFileMetadataTask() (task, error) {
if !appConfig.FileMetadata.Cataloger.Enabled {
return nil, nil
}
metadataCataloger := file.NewMetadataCataloger()
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := metadataCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.FileMetadata = result
return nil, nil
}
return task, nil
}
func generateCatalogFileDigestsTask() (task, error) {
if !appConfig.FileMetadata.Cataloger.Enabled {
return nil, nil
}
supportedHashAlgorithms := make(map[string]crypto.Hash)
for _, h := range []crypto.Hash{
crypto.MD5,
crypto.SHA1,
crypto.SHA256,
} {
supportedHashAlgorithms[file.DigestAlgorithmName(h)] = h
}
var hashes []crypto.Hash
for _, hashStr := range appConfig.FileMetadata.Digests {
name := file.CleanDigestAlgorithmName(hashStr)
hashObj, ok := supportedHashAlgorithms[name]
if !ok {
return nil, fmt.Errorf("unsupported hash algorithm: %s", hashStr)
}
hashes = append(hashes, hashObj)
}
digestsCataloger, err := file.NewDigestsCataloger(hashes)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileMetadata.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := digestsCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.FileDigests = result
return nil, nil
}
return task, nil
}
func generateCatalogSecretsTask() (task, error) {
if !appConfig.Secrets.Cataloger.Enabled {
return nil, nil
}
patterns, err := file.GenerateSearchPatterns(file.DefaultSecretsPatterns, appConfig.Secrets.AdditionalPatterns, appConfig.Secrets.ExcludePatternNames)
if err != nil {
return nil, err
}
secretsCataloger, err := file.NewSecretsCataloger(patterns, appConfig.Secrets.RevealValues, appConfig.Secrets.SkipFilesAboveSize)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.Secrets.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := secretsCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.Secrets = result
return nil, nil
}
return task, nil
}
func generateCatalogFileClassificationsTask() (task, error) {
if !appConfig.FileClassification.Cataloger.Enabled {
return nil, nil
}
// TODO: in the future we could expose out the classifiers via configuration
classifierCataloger, err := file.NewClassificationCataloger(file.DefaultClassifiers)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileClassification.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := classifierCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.FileClassifications = result
return nil, nil
}
return task, nil
}
func generateCatalogContentsTask() (task, error) {
if !appConfig.FileContents.Cataloger.Enabled {
return nil, nil
}
contentsCataloger, err := file.NewContentsCataloger(appConfig.FileContents.Globs, appConfig.FileContents.SkipFilesAboveSize)
if err != nil {
return nil, err
}
task := func(results *sbom.Artifacts, src *source.Source) ([]artifact.Relationship, error) {
resolver, err := src.FileResolver(appConfig.FileContents.Cataloger.ScopeOpt)
if err != nil {
return nil, err
}
result, err := contentsCataloger.Catalog(resolver)
if err != nil {
return nil, err
}
results.FileContents = result
return nil, nil
}
return task, nil
}
func runTask(t task, a *sbom.Artifacts, src *source.Source, c chan<- artifact.Relationship, errs chan<- error) {
defer close(c)
relationships, err := t(a, src)
if err != nil {
errs <- err
return
}
for _, relationship := range relationships {
c <- relationship
}
}
+4 -3
View File
@@ -9,11 +9,12 @@ require (
github.com/adrg/xdg v0.2.1
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29
github.com/anchore/stereoscope v0.0.0-20220307154759-8a5a70c227d3
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b
github.com/anchore/stereoscope v0.0.0-20220315185520-25183ec78f40
github.com/antihax/optional v1.0.0
github.com/bmatcuk/doublestar/v4 v4.0.2
github.com/dustin/go-humanize v1.0.0
@@ -171,7 +172,7 @@ require (
github.com/hashicorp/go-uuid v1.0.2 // indirect
github.com/hashicorp/go-version v1.4.0 // indirect
github.com/hashicorp/golang-lru v0.5.4 // indirect
github.com/hashicorp/vault/api v1.3.1 // indirect
github.com/hashicorp/vault/api v1.4.1 // indirect
github.com/hashicorp/vault/sdk v0.4.1 // indirect
github.com/hashicorp/yamux v0.0.0-20211028200310-0bc27b27de87 // indirect
github.com/imdario/mergo v0.3.12 // indirect
+8 -5
View File
@@ -276,16 +276,18 @@ github.com/alexflint/go-filemutex v0.0.0-20171022225611-72bdc8eae2ae/go.mod h1:C
github.com/alexkohler/prealloc v1.0.0/go.mod h1:VetnK3dIgFBBKmg0YnD9F9x6Icjd+9cvfHR56wJVlKE=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf/go.mod h1:FaODhIA06mxO1E6R32JE0TL1JWZZkmjRIAd4ULvHUKk=
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb h1:iDMnx6LIjtjZ46C0akqveX83WFzhpTD3eqOthawb5vU=
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb/go.mod h1:DmTY2Mfcv38hsHbG78xMiTDdxFtkHpgYNVDPsF2TgHk=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63 h1:C9W/LAydEz/qdUhx1MdjO9l8NEcFKYknkxDVyo9LAoM=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63/go.mod h1:6qH8c6U/3CBVvDDDBZnPSTbTINq3cIdADUYTaVf75EM=
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04 h1:VzprUTpc0vW0nnNKJfJieyH/TZ9UYAnTZs5/gHTdAe8=
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04/go.mod h1:6dK64g27Qi1qGQZ67gFmBFvEHScy0/C8qhQhNe5B5pQ=
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZVsCYMrIZBpFxwV26CbsuoEh5muXD5I1Ods=
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29 h1:K9LfnxwhqvihqU0+MF325FNy7fsKV9EGaUxdfR4gnWk=
github.com/anchore/packageurl-go v0.0.0-20210922164639-b3fa992ebd29/go.mod h1:Oc1UkGaJwY6ND6vtAqPSlYrptKRJngHwkwB6W7l1uP0=
github.com/anchore/stereoscope v0.0.0-20220307154759-8a5a70c227d3 h1:Kx2jlMdENAf4cVjYGYLI+fiavVhzhtmU89GUYDITJ1w=
github.com/anchore/stereoscope v0.0.0-20220307154759-8a5a70c227d3/go.mod h1:XESZQTgFETDBatmyoet6XZ0zVknoIMDSAhj2INj2a5w=
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b h1:YJWYt/6KQXR9JR46lLHrTTYi8rcye42tKcyjREA/hvA=
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b/go.mod h1:Blo6OgJNiYF41ufcgHKkbCKF2MDOMlrqhXv/ij6ocR4=
github.com/anchore/stereoscope v0.0.0-20220315185520-25183ec78f40 h1:OSyFMZgEwQW0wFyv10kEi9kmB52FQFRUQmc2H6d8LTY=
github.com/anchore/stereoscope v0.0.0-20220315185520-25183ec78f40/go.mod h1:OVVJ4y/L26pspeeHNvNa7GeQfoLPWxJe13iODl9x5l4=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.0.0/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
@@ -1349,8 +1351,9 @@ github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/J
github.com/hashicorp/serf v0.9.5/go.mod h1:UWDWwZeL5cuWDJdl0C6wrvrUwEqtQ4ZKBKKENpqIUyk=
github.com/hashicorp/serf v0.9.6/go.mod h1:TXZNMjZQijwlDvp+r0b63xZ45H7JmCmgg4gpTwn9UV4=
github.com/hashicorp/vault/api v1.3.0/go.mod h1:EabNQLI0VWbWoGlA+oBLC8PXmR9D60aUVgQGvangFWQ=
github.com/hashicorp/vault/api v1.3.1 h1:pkDkcgTh47PRjY1NEFeofqR4W/HkNUi9qIakESO2aRM=
github.com/hashicorp/vault/api v1.3.1/go.mod h1:QeJoWxMFt+MsuWcYhmwRLwKEXrjwAFFywzhptMsTIUw=
github.com/hashicorp/vault/api v1.4.1 h1:mWLfPT0RhxBitjKr6swieCEP2v5pp/M//t70S3kMLRo=
github.com/hashicorp/vault/api v1.4.1/go.mod h1:LkMdrZnWNrFaQyYYazWVn7KshilfDidgVBq6YiTq/bM=
github.com/hashicorp/vault/sdk v0.3.0/go.mod h1:aZ3fNuL5VNydQk8GcLJ2TV8YCRVvyaakYkhZRoVuhj0=
github.com/hashicorp/vault/sdk v0.4.1 h1:3SaHOJY687jY1fnB61PtL0cOkKItphrbLmux7T92HBo=
github.com/hashicorp/vault/sdk v0.4.1/go.mod h1:aZ3fNuL5VNydQk8GcLJ2TV8YCRVvyaakYkhZRoVuhj0=
+4 -5
View File
@@ -6,15 +6,14 @@ import (
"fmt"
"time"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/syft/event"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
"github.com/anchore/client-go/pkg/external"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/sbom"
"github.com/antihax/optional"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
)
type ImportConfig struct {
+9 -11
View File
@@ -4,8 +4,6 @@ import (
"context"
"encoding/json"
"fmt"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/file"
"net/http"
"strings"
"testing"
@@ -24,7 +22,7 @@ import (
"github.com/wagoodman/go-progress"
)
func must(c cpe.CPE, e error) cpe.CPE {
func must(c pkg.CPE, e error) pkg.CPE {
if e != nil {
panic(e)
}
@@ -54,13 +52,13 @@ func (m *mockPackageSBOMImportAPI) ImportImagePackages(ctx context.Context, sess
func sbomFixture() sbom.SBOM {
return sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(pkg.Package{
PackageCatalog: pkg.NewCatalog(pkg.Package{
Name: "name",
Version: "version",
FoundBy: "foundBy",
Locations: []file.Location{
Locations: []source.Location{
{
Coordinates: file.Coordinates{
Coordinates: source.Coordinates{
RealPath: "path",
FileSystemID: "layerID",
},
@@ -69,8 +67,8 @@ func sbomFixture() sbom.SBOM {
Licenses: []string{"license"},
Language: pkg.Python,
Type: pkg.PythonPkg,
CPEs: []cpe.CPE{
must(cpe.New("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*")),
CPEs: []pkg.CPE{
must(pkg.NewCPE("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*")),
},
PURL: "purl",
MetadataType: pkg.PythonPackageMetadataType,
@@ -104,13 +102,13 @@ func sbomFixture() sbom.SBOM {
},
Relationships: []artifact.Relationship{
{
From: file.NewLocation("/place1"),
To: file.NewLocation("/place2"),
From: source.NewLocation("/place1"),
To: source.NewLocation("/place2"),
Type: artifact.ContainsRelationship,
},
},
Source: source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "user-in",
Layers: nil,
+18 -74
View File
@@ -3,16 +3,12 @@ package config
import (
"errors"
"fmt"
"github.com/anchore/syft/syft/source"
"path"
"reflect"
"strings"
"github.com/adrg/xdg"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/file"
"github.com/mitchellh/go-homedir"
"github.com/sirupsen/logrus"
"github.com/spf13/viper"
@@ -21,7 +17,6 @@ import (
var ErrApplicationConfigNotFound = fmt.Errorf("application config not found")
// TODO: set all catalogers when this is set
var catalogerEnabledDefault = false
type defaultValueLoader interface {
@@ -34,66 +29,24 @@ type parser interface {
// Application is the main syft application configuration.
type Application struct {
ConfigPath string `yaml:",omitempty" json:"configPath"` // the location where the application config was read from (either from -c or discovered while loading)
Outputs []string `yaml:"output" json:"output" mapstructure:"output"` // -o, the format to use for output
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"` // -q, indicates to not show any status output to stderr (ETUI or logging UI)
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Anchore anchore `yaml:"anchore" json:"anchore" mapstructure:"anchore"` // options for interacting with Anchore Engine/Enterprise
CliOptions CliOnlyOptions `yaml:"-" json:"-"` // all options only available through the CLI (not via env vars or config)
Catalogers []string `yaml:"catalogers" json:"catalogers" mapstructure:"catalogers"`
Scope string `yaml:"scope" json:"scope" mapstructure:"scope"`
Dev development `yaml:"dev" json:"dev" mapstructure:"dev"`
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
FileMetadata fileMetadata `yaml:"file-metadata" json:"file-metadata" mapstructure:"file-metadata"`
FileContents fileContents `yaml:"file-contents" json:"file-contents" mapstructure:"file-contents"`
Secrets secretsCfg `yaml:"secrets" json:"secrets" mapstructure:"secrets"`
Registry registry `yaml:"registry" json:"registry" mapstructure:"registry"`
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
}
func (cfg Application) ToCatalogingOptions() ([]syft.CatalogingOption, error) {
digests, err := file.DigestHashesByName(cfg.FileMetadata.Digests...)
if err != nil {
return nil, fmt.Errorf("unable to parse config item 'file-metadata.digests': %w", err)
}
scopeOption := source.ParseScope(cfg.Scope)
if scopeOption == source.UnknownScope {
return nil, fmt.Errorf("bad scope value %q", cfg.Scope)
}
secretsConfig, err := cfg.Secrets.ToConfig()
if err != nil {
return nil, err
}
secretsScopeOption := source.ParseScope(cfg.Secrets.Scope)
if secretsScopeOption == source.UnknownScope {
return nil, fmt.Errorf("bad scope value %q", cfg.Secrets.Scope)
}
c := syft.DefaultCatalogingConfig()
c.ToolVersion = version.FromBuild().Version
c.ToolConfiguration = &c
c.DefaultScope = scopeOption
c.ProcessTasksInSerial = false
c.DigestHashes = digests
c.SecretsSearch = *secretsConfig
c.SecretsScope = secretsScopeOption
c.ContentsSearch = cfg.FileContents.ToConfig()
opts := []syft.CatalogingOption{
syft.WithConfig(c),
syft.WithCatalogers(cfg.Catalogers...),
}
return opts, nil
ConfigPath string `yaml:",omitempty" json:"configPath"` // the location where the application config was read from (either from -c or discovered while loading)
Outputs []string `yaml:"output" json:"output" mapstructure:"output"` // -o, the format to use for output
File string `yaml:"file" json:"file" mapstructure:"file"` // --file, the file to write report output to
Quiet bool `yaml:"quiet" json:"quiet" mapstructure:"quiet"` // -q, indicates to not show any status output to stderr (ETUI or logging UI)
CheckForAppUpdate bool `yaml:"check-for-app-update" json:"check-for-app-update" mapstructure:"check-for-app-update"` // whether to check for an application update on start up or not
Anchore anchore `yaml:"anchore" json:"anchore" mapstructure:"anchore"` // options for interacting with Anchore Engine/Enterprise
CliOptions CliOnlyOptions `yaml:"-" json:"-"` // all options only available through the CLI (not via env vars or config)
Dev development `yaml:"dev" json:"dev" mapstructure:"dev"`
Log logging `yaml:"log" json:"log" mapstructure:"log"` // all logging-related options
Package pkg `yaml:"package" json:"package" mapstructure:"package"`
FileMetadata FileMetadata `yaml:"file-metadata" json:"file-metadata" mapstructure:"file-metadata"`
FileClassification fileClassification `yaml:"file-classification" json:"file-classification" mapstructure:"file-classification"`
FileContents fileContents `yaml:"file-contents" json:"file-contents" mapstructure:"file-contents"`
Secrets secrets `yaml:"secrets" json:"secrets" mapstructure:"secrets"`
Registry registry `yaml:"registry" json:"registry" mapstructure:"registry"`
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
}
// PowerUserCatalogerEnabledDefault switches all catalogers to be enabled when running power-user command
@@ -147,15 +100,6 @@ func (cfg Application) loadDefaultValues(v *viper.Viper) {
}
func (cfg *Application) parseConfigValues() error {
// parse fields on application directyy
var catalogers []string
for _, c := range cfg.Catalogers {
for _, f := range strings.Split(c, ",") {
catalogers = append(catalogers, strings.TrimSpace(f))
}
}
cfg.Catalogers = catalogers
// parse application config options
for _, optionFn := range []func() error{
cfg.parseUploadOptions,
+29
View File
@@ -0,0 +1,29 @@
package config
import (
"fmt"
"github.com/spf13/viper"
"github.com/anchore/syft/syft/source"
)
type catalogerOptions struct {
Enabled bool `yaml:"enabled" json:"enabled" mapstructure:"enabled"`
Scope string `yaml:"scope" json:"scope" mapstructure:"scope"`
ScopeOpt source.Scope `yaml:"-" json:"-"`
}
func (cfg catalogerOptions) loadDefaultValues(v *viper.Viper) {
v.SetDefault("package.cataloger.enabled", true)
}
func (cfg *catalogerOptions) parseConfigValues() error {
scopeOption := source.ParseScope(cfg.Scope)
if scopeOption == source.UnknownScope {
return fmt.Errorf("bad scope value %q", cfg.Scope)
}
cfg.ScopeOpt = scopeOption
return nil
}
+19
View File
@@ -0,0 +1,19 @@
package config
import (
"github.com/anchore/syft/syft/source"
"github.com/spf13/viper"
)
type fileClassification struct {
Cataloger catalogerOptions `yaml:"cataloger" json:"cataloger" mapstructure:"cataloger"`
}
func (cfg fileClassification) loadDefaultValues(v *viper.Viper) {
v.SetDefault("file-classification.cataloger.enabled", catalogerEnabledDefault)
v.SetDefault("file-classification.cataloger.scope", source.SquashedScope)
}
func (cfg *fileClassification) parseConfigValues() error {
return cfg.Cataloger.parseConfigValues()
}
+9 -9
View File
@@ -1,24 +1,24 @@
package config
import (
"github.com/anchore/syft/syft/cataloger/files/filecontents"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/internal/file"
"github.com/anchore/syft/syft/source"
"github.com/spf13/viper"
)
type fileContents struct {
SkipFilesAboveSize int64 `yaml:"skip-files-above-size" json:"skip-files-above-size" mapstructure:"skip-files-above-size"`
Globs []string `yaml:"globs" json:"globs" mapstructure:"globs"`
Cataloger catalogerOptions `yaml:"cataloger" json:"cataloger" mapstructure:"cataloger"`
SkipFilesAboveSize int64 `yaml:"skip-files-above-size" json:"skip-files-above-size" mapstructure:"skip-files-above-size"`
Globs []string `yaml:"globs" json:"globs" mapstructure:"globs"`
}
func (cfg fileContents) loadDefaultValues(v *viper.Viper) {
v.SetDefault("file-contents.cataloger.enabled", catalogerEnabledDefault)
v.SetDefault("file-contents.cataloger.scope", source.SquashedScope)
v.SetDefault("file-contents.skip-files-above-size", 1*file.MB)
v.SetDefault("file-contents.globs", []string{})
}
func (cfg fileContents) ToConfig() filecontents.Config {
return filecontents.Config{
Globs: cfg.Globs,
SkipFilesAboveSizeInBytes: cfg.SkipFilesAboveSize,
}
func (cfg *fileContents) parseConfigValues() error {
return cfg.Cataloger.parseConfigValues()
}
+9 -5
View File
@@ -1,17 +1,21 @@
package config
import (
"github.com/anchore/syft/syft/source"
"github.com/spf13/viper"
)
type fileMetadata struct {
Digests []string `yaml:"digests" json:"digests" mapstructure:"digests"`
type FileMetadata struct {
Cataloger catalogerOptions `yaml:"cataloger" json:"cataloger" mapstructure:"cataloger"`
Digests []string `yaml:"digests" json:"digests" mapstructure:"digests"`
}
func (cfg fileMetadata) loadDefaultValues(v *viper.Viper) {
func (cfg FileMetadata) loadDefaultValues(v *viper.Viper) {
v.SetDefault("file-metadata.cataloger.enabled", catalogerEnabledDefault)
v.SetDefault("file-metadata.cataloger.scope", source.SquashedScope)
v.SetDefault("file-metadata.digests", []string{"sha256"})
}
func (cfg *fileMetadata) parseConfigValues() error {
return nil
func (cfg *FileMetadata) parseConfigValues() error {
return cfg.Cataloger.parseConfigValues()
}
+17 -8
View File
@@ -1,24 +1,33 @@
package config
import (
"github.com/anchore/syft/syft/cataloger/packages"
"github.com/anchore/syft/syft/pkg/cataloger"
"github.com/spf13/viper"
)
type pkg struct {
SearchUnindexedArchives bool `yaml:"search-unindexed-archives" json:"search-unindexed-archives" mapstructure:"search-unindexed-archives"`
SearchIndexedArchives bool `yaml:"search-indexed-archives" json:"search-indexed-archives" mapstructure:"search-indexed-archives"`
Cataloger catalogerOptions `yaml:"cataloger" json:"cataloger" mapstructure:"cataloger"`
SearchUnindexedArchives bool `yaml:"search-unindexed-archives" json:"search-unindexed-archives" mapstructure:"search-unindexed-archives"`
SearchIndexedArchives bool `yaml:"search-indexed-archives" json:"search-indexed-archives" mapstructure:"search-indexed-archives"`
}
func (cfg pkg) loadDefaultValues(v *viper.Viper) {
c := packages.DefaultSearchConfig()
cfg.Cataloger.loadDefaultValues(v)
c := cataloger.DefaultSearchConfig()
v.SetDefault("package.search-unindexed-archives", c.IncludeUnindexedArchives)
v.SetDefault("package.search-indexed-archives", c.IncludeIndexedArchives)
}
func (cfg pkg) ToConfig() packages.SearchConfig {
return packages.SearchConfig{
IncludeIndexedArchives: cfg.SearchIndexedArchives,
IncludeUnindexedArchives: cfg.SearchUnindexedArchives,
func (cfg *pkg) parseConfigValues() error {
return cfg.Cataloger.parseConfigValues()
}
func (cfg pkg) ToConfig() cataloger.Config {
return cataloger.Config{
Search: cataloger.SearchConfig{
IncludeIndexedArchives: cfg.SearchIndexedArchives,
IncludeUnindexedArchives: cfg.SearchUnindexedArchives,
Scope: cfg.Cataloger.ScopeOpt,
},
}
}
+8 -18
View File
@@ -1,38 +1,28 @@
package config
import (
"fmt"
"github.com/anchore/syft/syft/cataloger/files/secrets"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/internal/file"
"github.com/anchore/syft/syft/source"
"github.com/spf13/viper"
)
type secretsCfg struct {
type secrets struct {
Cataloger catalogerOptions `yaml:"cataloger" json:"cataloger" mapstructure:"cataloger"`
AdditionalPatterns map[string]string `yaml:"additional-patterns" json:"additional-patterns" mapstructure:"additional-patterns"`
ExcludePatternNames []string `yaml:"exclude-pattern-names" json:"exclude-pattern-names" mapstructure:"exclude-pattern-names"`
RevealValues bool `yaml:"reveal-values" json:"reveal-values" mapstructure:"reveal-values"`
SkipFilesAboveSize int64 `yaml:"skip-files-above-size" json:"skip-files-above-size" mapstructure:"skip-files-above-size"`
Scope string `yaml:"scope" json:"scope" mapstructure:"scope"`
}
func (cfg secretsCfg) loadDefaultValues(v *viper.Viper) {
v.SetDefault("secrets.scope", source.AllLayersScope)
func (cfg secrets) loadDefaultValues(v *viper.Viper) {
v.SetDefault("secrets.cataloger.enabled", catalogerEnabledDefault)
v.SetDefault("secrets.cataloger.scope", source.AllLayersScope)
v.SetDefault("secrets.reveal-values", false)
v.SetDefault("secrets.skip-files-above-size", 1*file.MB)
v.SetDefault("secrets.additional-patterns", map[string]string{})
v.SetDefault("secrets.exclude-pattern-names", []string{})
}
func (cfg secretsCfg) ToConfig() (*secrets.Config, error) {
patterns, err := file.GenerateSearchPatterns(secrets.DefaultSecretsPatterns, cfg.AdditionalPatterns, cfg.ExcludePatternNames)
if err != nil {
return nil, fmt.Errorf("unable to process secrets config patterns: %w", err)
}
return &secrets.Config{
Patterns: patterns,
RevealValues: cfg.RevealValues,
MaxFileSize: cfg.SkipFilesAboveSize,
}, nil
func (cfg *secrets) parseConfigValues() error {
return cfg.Cataloger.parseConfigValues()
}
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "3.0.1"
JSONSchemaVersion = "3.1.1"
)
@@ -1,14 +1,12 @@
package archive
package file
import (
"errors"
"fmt"
"io"
"github.com/anchore/syft/syft/file"
)
const perFileReadLimit = 2 * file.GB
const perFileReadLimit = 2 * GB
// safeCopy limits the copy from the reader. This is useful when extracting files from archives to
// protect against decompression bomb attacks.
@@ -1,8 +1,8 @@
package internal
package file
// FileNameGlobMatch evaluates the given glob pattern against the given "name" string, indicating if there is a match or not.
// GlobMatch evaluates the given glob pattern against the given "name" string, indicating if there is a match or not.
// Source: https://research.swtch.com/glob.go
func FileNameGlobMatch(pattern, name string) bool {
func GlobMatch(pattern, name string) bool {
px := 0
nx := 0
nextPx := 0
@@ -1,4 +1,4 @@
package internal
package file
import (
"strings"
@@ -32,7 +32,7 @@ func TestGlobMatch(t *testing.T) {
}
for _, test := range tests {
if FileNameGlobMatch(test.pattern, test.data) != test.ok {
if GlobMatch(test.pattern, test.data) != test.ok {
t.Errorf("failed glob='%s' data='%s'", test.pattern, test.data)
}
}
@@ -10,12 +10,6 @@ type Opener struct {
path string
}
func NewOpener(path string) Opener {
return Opener{
path: path,
}
}
// Open the stored path as a io.ReadCloser.
func (o Opener) Open() (io.ReadCloser, error) {
return os.Open(o.path)
@@ -1,39 +1,38 @@
package archive
package file
import (
"fmt"
"io/ioutil"
"path/filepath"
"github.com/anchore/syft/syft/file"
"github.com/bmatcuk/doublestar/v4"
"github.com/mholt/archiver/v3"
)
// ExtractGlobsFromTarToUniqueTempFile extracts paths matching the given globs within the given archive to a temporary directory, returning file openers for each file extracted.
func ExtractGlobsFromTarToUniqueTempFile(archivePath, dir string, globs ...string) (map[string]file.Opener, error) {
results := make(map[string]file.Opener)
func ExtractGlobsFromTarToUniqueTempFile(archivePath, dir string, globs ...string) (map[string]Opener, error) {
results := make(map[string]Opener)
// don't allow for full traversal, only select traversal from given paths
if len(globs) == 0 {
return results, nil
}
visitor := func(f archiver.File) error {
defer f.Close()
visitor := func(file archiver.File) error {
defer file.Close()
// ignore directories
if f.FileInfo.IsDir() {
if file.FileInfo.IsDir() {
return nil
}
// ignore any filename that doesn't match the given globs...
if !matchesAnyGlob(f.Name(), globs...) {
if !matchesAnyGlob(file.Name(), globs...) {
return nil
}
// we have a file we want to extract....
tempfilePrefix := filepath.Base(filepath.Clean(f.Name())) + "-"
tempfilePrefix := filepath.Base(filepath.Clean(file.Name())) + "-"
tempFile, err := ioutil.TempFile(dir, tempfilePrefix)
if err != nil {
return fmt.Errorf("unable to create temp file: %w", err)
@@ -43,11 +42,11 @@ func ExtractGlobsFromTarToUniqueTempFile(archivePath, dir string, globs ...strin
// provides a ReadCloser. It is up to the caller to handle closing the file explicitly.
defer tempFile.Close()
if err := safeCopy(tempFile, f.ReadCloser); err != nil {
return fmt.Errorf("unable to copy source=%q for tar=%q: %w", f.Name(), archivePath, err)
if err := safeCopy(tempFile, file.ReadCloser); err != nil {
return fmt.Errorf("unable to copy source=%q for tar=%q: %w", file.Name(), archivePath, err)
}
results[f.Name()] = file.NewOpener(tempFile.Name())
results[file.Name()] = Opener{path: tempFile.Name()}
return nil
}
@@ -1,4 +1,4 @@
package archive
package file
import (
"io/ioutil"
@@ -1,4 +1,4 @@
package archive
package file
import (
"fmt"
@@ -48,7 +48,7 @@ func (z ZipFileManifest) GlobMatch(patterns ...string) []string {
// so that glob logic is consistent inside and outside of ZIP archives
normalizedEntry := normalizeZipEntryName(entry)
if internal.FileNameGlobMatch(pattern, normalizedEntry) {
if GlobMatch(pattern, normalizedEntry) {
uniqueMatches.Add(entry)
}
}
@@ -1,7 +1,7 @@
//go:build !windows
// +build !windows
package archive
package file
import (
"encoding/json"
@@ -1,4 +1,4 @@
package archive
package file
import (
"archive/zip"
@@ -9,11 +9,17 @@ import (
"path/filepath"
"strings"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/internal/log"
)
const (
// represents the order of bytes
_ = iota
KB = 1 << (10 * iota)
MB
GB
)
type errZipSlipDetected struct {
Prefix string
JoinArgs []string
@@ -65,16 +71,16 @@ func TraverseFilesInZip(archivePath string, visitor func(*zip.File) error, paths
}
// ExtractFromZipToUniqueTempFile extracts select paths for the given archive to a temporary directory, returning file openers for each file extracted.
func ExtractFromZipToUniqueTempFile(archivePath, dir string, paths ...string) (map[string]file.Opener, error) {
results := make(map[string]file.Opener)
func ExtractFromZipToUniqueTempFile(archivePath, dir string, paths ...string) (map[string]Opener, error) {
results := make(map[string]Opener)
// don't allow for full traversal, only select traversal from given paths
if len(paths) == 0 {
return results, nil
}
visitor := func(f *zip.File) error {
tempfilePrefix := filepath.Base(filepath.Clean(f.Name)) + "-"
visitor := func(file *zip.File) error {
tempfilePrefix := filepath.Base(filepath.Clean(file.Name)) + "-"
tempFile, err := ioutil.TempFile(dir, tempfilePrefix)
if err != nil {
@@ -85,26 +91,26 @@ func ExtractFromZipToUniqueTempFile(archivePath, dir string, paths ...string) (m
// provides a ReadCloser. It is up to the caller to handle closing the file explicitly.
defer tempFile.Close()
zippedFile, err := f.Open()
zippedFile, err := file.Open()
if err != nil {
return fmt.Errorf("unable to read file=%q from zip=%q: %w", f.Name, archivePath, err)
return fmt.Errorf("unable to read file=%q from zip=%q: %w", file.Name, archivePath, err)
}
defer func() {
err := zippedFile.Close()
if err != nil {
log.Errorf("unable to close source file=%q from zip=%q: %+v", f.Name, archivePath, err)
log.Errorf("unable to close source file=%q from zip=%q: %+v", file.Name, archivePath, err)
}
}()
if f.FileInfo().IsDir() {
return fmt.Errorf("unable to extract directories, only files: %s", f.Name)
if file.FileInfo().IsDir() {
return fmt.Errorf("unable to extract directories, only files: %s", file.Name)
}
if err := safeCopy(tempFile, zippedFile); err != nil {
return fmt.Errorf("unable to copy source=%q for zip=%q: %w", f.Name, archivePath, err)
return fmt.Errorf("unable to copy source=%q for zip=%q: %w", file.Name, archivePath, err)
}
results[f.Name] = file.NewOpener(tempFile.Name())
results[file.Name] = Opener{path: tempFile.Name()}
return nil
}
@@ -1,7 +1,7 @@
//go:build !windows
// +build !windows
package archive
package file
import (
"crypto/sha256"
@@ -1,4 +1,4 @@
package archive
package file
import (
"archive/zip"
@@ -1,7 +1,7 @@
//go:build !windows
// +build !windows
package archive
package file
import (
"os"
@@ -3,12 +3,11 @@ package cyclonedxhelpers
import (
"reflect"
"github.com/anchore/syft/syft/file"
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/syft/internal/formats/common"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
func encodeComponent(p pkg.Package) cyclonedx.Component {
@@ -48,8 +47,10 @@ func hasMetadata(p pkg.Package) bool {
func decodeComponent(c *cyclonedx.Component) *pkg.Package {
values := map[string]string{}
for _, p := range *c.Properties {
values[p.Name] = p.Value
if c.Properties != nil {
for _, p := range *c.Properties {
values[p.Name] = p.Value
}
}
p := &pkg.Package{
@@ -72,9 +73,9 @@ func decodeComponent(c *cyclonedx.Component) *pkg.Package {
return p
}
func decodeLocations(vals map[string]string) []file.Location {
v := common.Decode(reflect.TypeOf([]file.Location{}), vals, "syft:location", CycloneDXFields)
out, _ := v.([]file.Location)
func decodeLocations(vals map[string]string) []source.Location {
v := common.Decode(reflect.TypeOf([]source.Location{}), vals, "syft:location", CycloneDXFields)
out, _ := v.([]source.Location)
return out
}
@@ -1,13 +1,13 @@
package cyclonedxhelpers
import (
"github.com/anchore/syft/syft/file"
"testing"
"github.com/CycloneDX/cyclonedx-go"
"github.com/stretchr/testify/assert"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
func Test_encodeComponentProperties(t *testing.T) {
@@ -26,8 +26,8 @@ func Test_encodeComponentProperties(t *testing.T) {
name: "from apk",
input: pkg.Package{
FoundBy: "cataloger",
Locations: []file.Location{
{Coordinates: file.Coordinates{RealPath: "test"}},
Locations: []source.Location{
{Coordinates: source.Coordinates{RealPath: "test"}},
},
Metadata: pkg.ApkMetadata{
Package: "libc-utils",
@@ -2,7 +2,6 @@ package cyclonedxhelpers
import (
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/pkg"
@@ -12,7 +11,7 @@ func encodeSingleCPE(p pkg.Package) string {
// Since the CPEs in a package are sorted by specificity
// we can extract the first CPE as the one to output in cyclonedx
if len(p.CPEs) > 0 {
return cpe.String(p.CPEs[0])
return pkg.CPEString(p.CPEs[0])
}
return ""
}
@@ -25,15 +24,15 @@ func encodeCPEs(p pkg.Package) (out []cyclonedx.Property) {
}
out = append(out, cyclonedx.Property{
Name: "syft:cpe23",
Value: cpe.String(c),
Value: pkg.CPEString(c),
})
}
return
}
func decodeCPEs(c *cyclonedx.Component) (out []cpe.CPE) {
func decodeCPEs(c *cyclonedx.Component) (out []pkg.CPE) {
if c.CPE != "" {
cp, err := cpe.New(c.CPE)
cp, err := pkg.NewCPE(c.CPE)
if err != nil {
log.Warnf("invalid CPE: %s", c.CPE)
} else {
@@ -44,7 +43,7 @@ func decodeCPEs(c *cyclonedx.Component) (out []cpe.CPE) {
if c.Properties != nil {
for _, p := range *c.Properties {
if p.Name == "syft:cpe23" {
cp, err := cpe.New(p.Value)
cp, err := pkg.NewCPE(p.Value)
if err != nil {
log.Warnf("invalid CPE: %s", p.Value)
} else {
@@ -1,7 +1,6 @@
package cyclonedxhelpers
import (
"github.com/anchore/syft/syft/cpe"
"testing"
"github.com/stretchr/testify/assert"
@@ -10,8 +9,8 @@ import (
)
func Test_encodeCPE(t *testing.T) {
testCPE := cpe.Must("cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*")
testCPE2 := cpe.Must("cpe:2.3:a:name:name2:3.2:*:*:*:*:*:*:*")
testCPE := pkg.MustCPE("cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*")
testCPE2 := pkg.MustCPE("cpe:2.3:a:name:name2:3.2:*:*:*:*:*:*:*")
tests := []struct {
name string
input pkg.Package
@@ -21,14 +20,14 @@ func Test_encodeCPE(t *testing.T) {
// note: since this is an optional field, no value is preferred over NONE or NOASSERTION
name: "no metadata",
input: pkg.Package{
CPEs: []cpe.CPE{},
CPEs: []pkg.CPE{},
},
expected: "",
},
{
name: "single CPE",
input: pkg.Package{
CPEs: []cpe.CPE{
CPEs: []pkg.CPE{
testCPE,
},
},
@@ -37,7 +36,7 @@ func Test_encodeCPE(t *testing.T) {
{
name: "multiple CPEs",
input: pkg.Package{
CPEs: []cpe.CPE{
CPEs: []pkg.CPE{
testCPE2,
testCPE,
},
@@ -5,6 +5,8 @@ import (
"io"
"github.com/CycloneDX/cyclonedx-go"
"github.com/anchore/syft/internal/formats/common"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
@@ -44,12 +46,12 @@ func GetDecoder(format cyclonedx.BOMFileFormat) sbom.Decoder {
func toSyftModel(bom *cyclonedx.BOM) (*sbom.SBOM, error) {
meta := source.Metadata{}
if bom.Metadata != nil {
if bom.Metadata != nil && bom.Metadata.Component != nil {
meta = decodeMetadata(bom.Metadata.Component)
}
s := &sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(),
PackageCatalog: pkg.NewCatalog(),
LinuxDistribution: linuxReleaseFromComponents(*bom.Components),
},
Source: meta,
@@ -86,7 +88,7 @@ func collectPackages(component *cyclonedx.Component, s *sbom.SBOM, idMap map[str
idMap[component.BOMRef] = p
// TODO there must be a better way than needing to call this manually:
p.SetID()
s.Artifacts.Packages.Add(*p)
s.Artifacts.PackageCatalog.Add(*p)
}
if component.Components != nil {
@@ -157,6 +159,14 @@ func linuxReleaseFromOSComponent(component *cyclonedx.Component) *linux.Release
}
}
if component.Properties != nil {
values := map[string]string{}
for _, p := range *component.Properties {
values[p.Name] = p.Value
}
common.DecodeInto(&rel, values, "syft:distro", CycloneDXFields)
}
return rel
}
@@ -199,7 +209,7 @@ func decodeMetadata(component *cyclonedx.Component) source.Metadata {
switch component.Type {
case cyclonedx.ComponentTypeContainer:
return source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: component.Name,
ID: component.BOMRef,
@@ -208,7 +218,7 @@ func decodeMetadata(component *cyclonedx.Component) source.Metadata {
}
case cyclonedx.ComponentTypeFile:
return source.Metadata{
Scheme: source.FileType, // or source.DirectoryType
Scheme: source.FileScheme, // or source.DirectoryScheme
Path: component.Name,
ImageMetadata: source.ImageMetadata{
UserInput: component.Name,
@@ -212,7 +212,7 @@ func Test_decode(t *testing.T) {
assert.Equal(t, e.ver, sbom.Artifacts.LinuxDistribution.VersionID)
}
if e.pkg != "" {
for p := range sbom.Artifacts.Packages.Enumerate() {
for p := range sbom.Artifacts.PackageCatalog.Enumerate() {
if e.pkg != p.Name {
continue
}
@@ -240,7 +240,7 @@ func Test_decode(t *testing.T) {
if e.relation != "" {
foundRelation := false
for _, r := range sbom.Relationships {
p := sbom.Artifacts.Packages.Package(r.To.ID())
p := sbom.Artifacts.PackageCatalog.Package(r.To.ID())
if e.relation == p.Name {
foundRelation = true
break
@@ -258,3 +258,18 @@ func Test_decode(t *testing.T) {
})
}
}
func Test_missingDataDecode(t *testing.T) {
bom := &cyclonedx.BOM{
Metadata: nil,
Components: &[]cyclonedx.Component{},
}
_, err := toSyftModel(bom)
assert.NoError(t, err)
bom.Metadata = &cyclonedx.Metadata{}
_, err = toSyftModel(bom)
assert.NoError(t, err)
}
@@ -25,7 +25,7 @@ func ToFormatModel(s sbom.SBOM) *cyclonedx.BOM {
cdxBOM.SerialNumber = uuid.New().URN()
cdxBOM.Metadata = toBomDescriptor(internal.ApplicationName, versionInfo.Version, s.Source)
packages := s.Artifacts.Packages.Sorted()
packages := s.Artifacts.PackageCatalog.Sorted()
components := make([]cyclonedx.Component, len(packages))
for i, p := range packages {
components[i] = encodeComponent(p)
@@ -155,7 +155,7 @@ func toDependencies(relationships []artifact.Relationship) []cyclonedx.Dependenc
func toBomDescriptorComponent(srcMetadata source.Metadata) *cyclonedx.Component {
switch srcMetadata.Scheme {
case source.ImageType:
case source.ImageScheme:
bomRef, err := artifact.IDByHash(srcMetadata.ImageMetadata.ID)
if err != nil {
log.Warnf("unable to get fingerprint of image metadata=%s: %+v", srcMetadata.ImageMetadata.ID, err)
@@ -166,7 +166,7 @@ func toBomDescriptorComponent(srcMetadata source.Metadata) *cyclonedx.Component
Name: srcMetadata.ImageMetadata.UserInput,
Version: srcMetadata.ImageMetadata.ManifestDigest,
}
case source.DirectoryType, source.FileType:
case source.DirectoryScheme, source.FileScheme:
bomRef, err := artifact.IDByHash(srcMetadata.Path)
if err != nil {
log.Warnf("unable to get fingerprint of source metadata path=%s: %+v", srcMetadata.Path, err)
@@ -10,9 +10,9 @@ import (
func DocumentName(srcMetadata source.Metadata) (string, error) {
switch srcMetadata.Scheme {
case source.ImageType:
case source.ImageScheme:
return cleanName(srcMetadata.ImageMetadata.UserInput), nil
case source.DirectoryType, source.FileType:
case source.DirectoryScheme, source.FileScheme:
return cleanName(srcMetadata.Path), nil
}
@@ -14,7 +14,7 @@ import (
func Test_DocumentName(t *testing.T) {
allSchemes := strset.New()
for _, s := range source.AllTypes {
for _, s := range source.AllSchemes {
allSchemes.Add(string(s))
}
testedSchemes := strset.New()
@@ -29,7 +29,7 @@ func Test_DocumentName(t *testing.T) {
name: "image",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "image-repo/name:tag",
ID: "id",
@@ -42,7 +42,7 @@ func Test_DocumentName(t *testing.T) {
name: "directory",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.DirectoryType,
Scheme: source.DirectoryScheme,
Path: "some/path/to/place",
},
expected: "some/path/to/place",
@@ -51,7 +51,7 @@ func Test_DocumentName(t *testing.T) {
name: "file",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.FileType,
Scheme: source.FileScheme,
Path: "some/path/to/place",
},
expected: "some/path/to/place",
@@ -21,11 +21,11 @@ func DocumentNameAndNamespace(srcMetadata source.Metadata) (string, string, erro
func DocumentNamespace(name string, srcMetadata source.Metadata) string {
input := "unknown-source-type"
switch srcMetadata.Scheme {
case source.ImageType:
case source.ImageScheme:
input = "image"
case source.DirectoryType:
case source.DirectoryScheme:
input = "dir"
case source.FileType:
case source.FileScheme:
input = "file"
}
@@ -12,7 +12,7 @@ import (
func Test_documentNamespace(t *testing.T) {
allSchemes := strset.New()
for _, s := range source.AllTypes {
for _, s := range source.AllSchemes {
allSchemes.Add(string(s))
}
testedSchemes := strset.New()
@@ -27,7 +27,7 @@ func Test_documentNamespace(t *testing.T) {
name: "image",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "image-repo/name:tag",
ID: "id",
@@ -40,7 +40,7 @@ func Test_documentNamespace(t *testing.T) {
name: "directory",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.DirectoryType,
Scheme: source.DirectoryScheme,
Path: "some/path/to/place",
},
expected: "https://anchore.com/syft/dir/my-name-",
@@ -49,7 +49,7 @@ func Test_documentNamespace(t *testing.T) {
name: "file",
inputName: "my-name",
srcMetadata: source.Metadata{
Scheme: source.FileType,
Scheme: source.FileScheme,
Path: "some/path/to/place",
},
expected: "https://anchore.com/syft/file/my-name-",
@@ -1,7 +1,6 @@
package spdxhelpers
import (
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/pkg"
)
@@ -11,7 +10,7 @@ func ExternalRefs(p pkg.Package) (externalRefs []ExternalRef) {
for _, c := range p.CPEs {
externalRefs = append(externalRefs, ExternalRef{
ReferenceCategory: SecurityReferenceCategory,
ReferenceLocator: cpe.String(c),
ReferenceLocator: pkg.CPEString(c),
ReferenceType: Cpe23ExternalRefType,
})
}
@@ -1,7 +1,6 @@
package spdxhelpers
import (
"github.com/anchore/syft/syft/cpe"
"testing"
"github.com/anchore/syft/syft/pkg"
@@ -9,7 +8,7 @@ import (
)
func Test_ExternalRefs(t *testing.T) {
testCPE := cpe.Must("cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*")
testCPE := pkg.MustCPE("cpe:2.3:a:name:name:3.2:*:*:*:*:*:*:*")
tests := []struct {
name string
input pkg.Package
@@ -18,7 +17,7 @@ func Test_ExternalRefs(t *testing.T) {
{
name: "cpe + purl",
input: pkg.Package{
CPEs: []cpe.CPE{
CPEs: []pkg.CPE{
testCPE,
},
PURL: "a-purl",
@@ -26,7 +25,7 @@ func Test_ExternalRefs(t *testing.T) {
expected: []ExternalRef{
{
ReferenceCategory: SecurityReferenceCategory,
ReferenceLocator: cpe.String(testCPE),
ReferenceLocator: pkg.CPEString(testCPE),
ReferenceType: Cpe23ExternalRefType,
},
{
@@ -1,10 +1,10 @@
package spdxhelpers
import (
"github.com/anchore/syft/syft/file"
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
"github.com/stretchr/testify/assert"
)
@@ -18,9 +18,9 @@ func Test_SourceInfo(t *testing.T) {
name: "locations are captured",
input: pkg.Package{
// note: no type given
Locations: []file.Location{
file.NewVirtualLocation("/a-place", "/b-place"),
file.NewVirtualLocation("/c-place", "/d-place"),
Locations: []source.Location{
source.NewVirtualLocation("/a-place", "/b-place"),
source.NewVirtualLocation("/c-place", "/d-place"),
},
},
expected: []string{
@@ -1,7 +1,6 @@
package spdxhelpers
import (
"github.com/anchore/syft/syft/cpe"
"strconv"
"strings"
@@ -14,6 +13,7 @@ import (
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
func ToSyftModel(doc *spdx.Document2_2) (*sbom.SBOM, error) {
@@ -21,9 +21,9 @@ func ToSyftModel(doc *spdx.Document2_2) (*sbom.SBOM, error) {
s := &sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: pkg.NewCollection(),
FileMetadata: map[file.Coordinates]file.Metadata{},
FileDigests: map[file.Coordinates][]file.Digest{},
PackageCatalog: pkg.NewCatalog(),
FileMetadata: map[source.Coordinates]source.FileMetadata{},
FileDigests: map[source.Coordinates][]file.Digest{},
LinuxDistribution: findLinuxReleaseByPURL(doc),
},
}
@@ -74,7 +74,7 @@ func collectSyftPackages(s *sbom.SBOM, spdxIDMap map[string]interface{}, doc *sp
for _, p := range doc.Packages {
syftPkg := toSyftPackage(p)
spdxIDMap[string(p.PackageSPDXIdentifier)] = syftPkg
s.Artifacts.Packages.Add(*syftPkg)
s.Artifacts.PackageCatalog.Add(*syftPkg)
}
}
@@ -98,7 +98,7 @@ func toFileDigests(f *spdx.File2_2) (digests []file.Digest) {
return digests
}
func toFileMetadata(f *spdx.File2_2) (meta file.Metadata) {
func toFileMetadata(f *spdx.File2_2) (meta source.FileMetadata) {
// FIXME Syft is currently lossy due to the SPDX 2.2.1 spec not supporting arbitrary mimetypes
for _, typ := range f.FileType {
switch FileType(typ) {
@@ -132,7 +132,7 @@ func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document2_2
b := spdxIDMap[string(r.RefB.ElementRefID)]
from, fromOk := a.(*pkg.Package)
toPackage, toPackageOk := b.(*pkg.Package)
toLocation, toLocationOk := b.(*file.Location)
toLocation, toLocationOk := b.(*source.Location)
if !fromOk || !(toPackageOk || toLocationOk) {
log.Debugf("unable to find valid relationship mapping from SPDX 2.2 JSON, ignoring: (from: %+v) (to: %+v)", a, b)
continue
@@ -174,7 +174,7 @@ func toSyftRelationships(spdxIDMap map[string]interface{}, doc *spdx.Document2_2
return out
}
func toSyftCoordinates(f *spdx.File2_2) file.Coordinates {
func toSyftCoordinates(f *spdx.File2_2) source.Coordinates {
const layerIDPrefix = "layerID: "
var fileSystemID string
if strings.Index(f.FileComment, layerIDPrefix) == 0 {
@@ -183,16 +183,16 @@ func toSyftCoordinates(f *spdx.File2_2) file.Coordinates {
if strings.Index(string(f.FileSPDXIdentifier), layerIDPrefix) == 0 {
fileSystemID = strings.TrimPrefix(string(f.FileSPDXIdentifier), layerIDPrefix)
}
return file.Coordinates{
return source.Coordinates{
RealPath: f.FileName,
FileSystemID: fileSystemID,
}
}
func toSyftLocation(f *spdx.File2_2) *file.Location {
return &file.Location{
func toSyftLocation(f *spdx.File2_2) *source.Location {
return &source.Location{
Coordinates: toSyftCoordinates(f),
AccessPath: f.FileName,
VirtualPath: f.FileName,
}
}
@@ -317,10 +317,10 @@ func findPURLValue(p *spdx.Package2_2) string {
return ""
}
func extractCPEs(p *spdx.Package2_2) (cpes []cpe.CPE) {
func extractCPEs(p *spdx.Package2_2) (cpes []pkg.CPE) {
for _, r := range p.PackageExternalReferences {
if r.RefType == string(Cpe23ExternalRefType) {
cpe, err := cpe.New(r.Locator)
cpe, err := pkg.NewCPE(r.Locator)
if err != nil {
log.Warnf("unable to extract SPDX CPE=%q: %+v", r.Locator, err)
continue
@@ -94,7 +94,7 @@ func TestToSyftModel(t *testing.T) {
assert.NotNil(t, sbom)
pkgs := sbom.Artifacts.Packages.Sorted()
pkgs := sbom.Artifacts.PackageCatalog.Sorted()
assert.Len(t, pkgs, 2)
+22 -25
View File
@@ -2,12 +2,9 @@ package testutils
import (
"bytes"
"github.com/anchore/syft/syft/cpe"
"strings"
"testing"
"github.com/anchore/syft/syft/file"
"github.com/anchore/go-testutils"
"github.com/anchore/stereoscope/pkg/filetree"
"github.com/anchore/stereoscope/pkg/image"
@@ -100,7 +97,7 @@ func AssertEncoderAgainstGoldenSnapshot(t *testing.T, format sbom.Format, sbom s
func ImageInput(t testing.TB, testImage string, options ...ImageOption) sbom.SBOM {
t.Helper()
catalog := pkg.NewCollection()
catalog := pkg.NewCatalog()
var cfg imageCfg
var img *image.Image
for _, opt := range options {
@@ -124,7 +121,7 @@ func ImageInput(t testing.TB, testImage string, options ...ImageOption) sbom.SBO
return sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: catalog,
PackageCatalog: catalog,
LinuxDistribution: &linux.Release{
PrettyName: "debian",
Name: "debian",
@@ -152,7 +149,7 @@ func carriageRedactor(s []byte) []byte {
return []byte(msg)
}
func populateImageCatalog(catalog pkg.Collection, img *image.Image) {
func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
_, ref1, _ := img.SquashedTree().File("/somefile-1.txt", filetree.FollowBasenameLinks)
_, ref2, _ := img.SquashedTree().File("/somefile-2.txt", filetree.FollowBasenameLinks)
@@ -160,8 +157,8 @@ func populateImageCatalog(catalog pkg.Collection, img *image.Image) {
catalog.Add(pkg.Package{
Name: "package-1",
Version: "1.0.1",
Locations: []file.Location{
file.NewLocationFromImage(string(ref1.RealPath), *ref1, img),
Locations: []source.Location{
source.NewLocationFromImage(string(ref1.RealPath), *ref1, img),
},
Type: pkg.PythonPkg,
FoundBy: "the-cataloger-1",
@@ -173,15 +170,15 @@ func populateImageCatalog(catalog pkg.Collection, img *image.Image) {
Version: "1.0.1",
},
PURL: "a-purl-1",
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"),
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"),
},
})
catalog.Add(pkg.Package{
Name: "package-2",
Version: "2.0.1",
Locations: []file.Location{
file.NewLocationFromImage(string(ref2.RealPath), *ref2, img),
Locations: []source.Location{
source.NewLocationFromImage(string(ref2.RealPath), *ref2, img),
},
Type: pkg.DebPkg,
FoundBy: "the-cataloger-2",
@@ -191,8 +188,8 @@ func populateImageCatalog(catalog pkg.Collection, img *image.Image) {
Version: "2.0.1",
},
PURL: "a-purl-2",
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
})
}
@@ -205,7 +202,7 @@ func DirectoryInput(t testing.TB) sbom.SBOM {
return sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: catalog,
PackageCatalog: catalog,
LinuxDistribution: &linux.Release{
PrettyName: "debian",
Name: "debian",
@@ -228,8 +225,8 @@ func DirectoryInput(t testing.TB) sbom.SBOM {
}
}
func newDirectoryCatalog() pkg.Collection {
catalog := pkg.NewCollection()
func newDirectoryCatalog() *pkg.Catalog {
catalog := pkg.NewCatalog()
// populate catalog with test data
catalog.Add(pkg.Package{
@@ -237,8 +234,8 @@ func newDirectoryCatalog() pkg.Collection {
Version: "1.0.1",
Type: pkg.PythonPkg,
FoundBy: "the-cataloger-1",
Locations: []file.Location{
file.NewLocation("/some/path/pkg1"),
Locations: []source.Location{
source.NewLocation("/some/path/pkg1"),
},
Language: pkg.Python,
MetadataType: pkg.PythonPackageMetadataType,
@@ -253,8 +250,8 @@ func newDirectoryCatalog() pkg.Collection {
},
},
PURL: "a-purl-2",
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
})
catalog.Add(pkg.Package{
@@ -262,8 +259,8 @@ func newDirectoryCatalog() pkg.Collection {
Version: "2.0.1",
Type: pkg.DebPkg,
FoundBy: "the-cataloger-2",
Locations: []file.Location{
file.NewLocation("/some/path/pkg1"),
Locations: []source.Location{
source.NewLocation("/some/path/pkg1"),
},
MetadataType: pkg.DpkgMetadataType,
Metadata: pkg.DpkgMetadata{
@@ -271,8 +268,8 @@ func newDirectoryCatalog() pkg.Collection {
Version: "2.0.1",
},
PURL: "a-purl-2",
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
})
@@ -57,7 +57,7 @@ func Test_decodeJSON(t *testing.T) {
split = strings.SplitN(pkg, ":", 2)
name = split[0]
version = split[1]
for p := range bom.Artifacts.Packages.Enumerate() {
for p := range bom.Artifacts.PackageCatalog.Enumerate() {
if p.Name == name {
assert.Equal(t, version, p.Version)
continue pkgs
@@ -57,7 +57,7 @@ func Test_decodeXML(t *testing.T) {
split = strings.SplitN(pkg, ":", 2)
name = split[0]
version = split[1]
for p := range bom.Artifacts.Packages.Enumerate() {
for p := range bom.Artifacts.PackageCatalog.Enumerate() {
if p.Name == name {
assert.Equal(t, version, p.Version)
continue pkgs
+183
View File
@@ -0,0 +1,183 @@
package github
import (
"fmt"
"strings"
"time"
"github.com/mholt/archiver/v3"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/version"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
// toGithubModel converts the provided SBOM to a GitHub dependency model
func toGithubModel(s *sbom.SBOM) DependencySnapshot {
scanTime := time.Now().Format(time.RFC3339) // TODO is there a record of this somewhere?
v := version.FromBuild().Version
if v == "[not provided]" {
v = "0.0.0-dev"
}
return DependencySnapshot{
Version: 0,
// TODO allow property input to specify the Job, Sha, and Ref
Detector: DetectorMetadata{
Name: internal.ApplicationName,
URL: "https://github.com/anchore/syft",
Version: v,
},
Metadata: toSnapshotMetadata(s),
Manifests: toGithubManifests(s),
Scanned: scanTime,
}
}
// toSnapshotMetadata captures the linux distribution information and other metadata
func toSnapshotMetadata(s *sbom.SBOM) Metadata {
out := Metadata{}
if s.Artifacts.LinuxDistribution != nil {
d := s.Artifacts.LinuxDistribution
qualifiers := packageurl.Qualifiers{}
if len(d.IDLike) > 0 {
qualifiers = append(qualifiers, packageurl.Qualifier{
Key: "like",
Value: strings.Join(d.IDLike, ","),
})
}
purl := packageurl.NewPackageURL("generic", "", d.ID, d.VersionID, qualifiers, "")
out["syft:distro"] = purl.ToString()
}
return out
}
func filesystem(p pkg.Package) string {
if len(p.Locations) > 0 {
return p.Locations[0].FileSystemID
}
return ""
}
// isArchive returns true if the path appears to be an archive
func isArchive(path string) bool {
_, err := archiver.ByExtension(path)
return err == nil
}
// toPath Generates a string representation of the package location, optionally including the layer hash
func toPath(s source.Metadata, p pkg.Package) string {
inputPath := strings.TrimPrefix(s.Path, "./")
if inputPath == "." {
inputPath = ""
}
if len(p.Locations) > 0 {
location := p.Locations[0]
packagePath := location.RealPath
if location.VirtualPath != "" {
packagePath = location.VirtualPath
}
packagePath = strings.TrimPrefix(packagePath, "/")
switch s.Scheme {
case source.ImageScheme:
image := strings.ReplaceAll(s.ImageMetadata.UserInput, ":/", "//")
return fmt.Sprintf("%s:/%s", image, packagePath)
case source.FileScheme:
if isArchive(inputPath) {
return fmt.Sprintf("%s:/%s", inputPath, packagePath)
}
return inputPath
case source.DirectoryScheme:
if inputPath != "" {
return fmt.Sprintf("%s/%s", inputPath, packagePath)
}
return packagePath
}
}
return fmt.Sprintf("%s%s", inputPath, s.ImageMetadata.UserInput)
}
// toGithubManifests manifests, each of which represents a specific location that has dependencies
func toGithubManifests(s *sbom.SBOM) Manifests {
manifests := map[string]*Manifest{}
for _, p := range s.Artifacts.PackageCatalog.Sorted() {
path := toPath(s.Source, p)
manifest, ok := manifests[path]
if !ok {
manifest = &Manifest{
Name: path,
File: FileInfo{
SourceLocation: path,
},
Resolved: DependencyGraph{},
}
fs := filesystem(p)
if fs != "" {
manifest.Metadata = Metadata{
"syft:filesystem": fs,
}
}
manifests[path] = manifest
}
name := dependencyName(p)
manifest.Resolved[name] = DependencyNode{
Purl: p.PURL,
Metadata: toDependencyMetadata(p),
Relationship: toDependencyRelationshipType(p),
Scope: toDependencyScope(p),
Dependencies: toDependencies(s, p),
}
}
out := Manifests{}
for k, v := range manifests {
out[k] = *v
}
return out
}
// dependencyName to make things a little nicer to read; this might end up being lossy
func dependencyName(p pkg.Package) string {
purl, err := packageurl.FromString(p.PURL)
if err != nil {
log.Warnf("Invalid PURL for package: '%s' PURL: '%s' (%w)", p.Name, p.PURL, err)
return ""
}
// don't use qualifiers for this
purl.Qualifiers = nil
return purl.ToString()
}
func toDependencyScope(_ pkg.Package) DependencyScope {
return DependencyScopeRuntime
}
func toDependencyRelationshipType(_ pkg.Package) DependencyRelationship {
return DependencyRelationshipDirect
}
func toDependencyMetadata(_ pkg.Package) Metadata {
// We have limited properties: up to 8 with reasonably small values
// For now, we are encoding the location as part of the key, we are encoding PURLs with most
// of the other information Grype might need; and the distro information at the top level
// so we don't need anything here yet
return Metadata{}
}
func toDependencies(s *sbom.SBOM, p pkg.Package) (out []string) {
for _, r := range s.Relationships {
if r.From.ID() == p.ID() {
if p, ok := r.To.(pkg.Package); ok {
out = append(out, dependencyName(p))
}
}
}
return
}
+161
View File
@@ -0,0 +1,161 @@
package github
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
func Test_toGithubModel(t *testing.T) {
s := sbom.SBOM{
Source: source.Metadata{
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "ubuntu:18.04",
Architecture: "amd64",
},
},
Artifacts: sbom.Artifacts{
LinuxDistribution: &linux.Release{
ID: "ubuntu",
VersionID: "18.04",
IDLike: []string{"debian"},
},
PackageCatalog: pkg.NewCatalog(),
},
}
for _, p := range []pkg.Package{
{
Name: "pkg-1",
Version: "1.0.1",
Locations: []source.Location{{
Coordinates: source.Coordinates{
RealPath: "/usr/lib",
FileSystemID: "fsid-1",
},
}},
},
{
Name: "pkg-2",
Version: "2.0.2",
Locations: []source.Location{{
Coordinates: source.Coordinates{
RealPath: "/usr/lib",
FileSystemID: "fsid-1",
},
}},
},
{
Name: "pkg-3",
Version: "3.0.3",
Locations: []source.Location{{
Coordinates: source.Coordinates{
RealPath: "/etc",
FileSystemID: "fsid-1",
},
}},
},
} {
p.PURL = packageurl.NewPackageURL(
"generic",
"",
p.Name,
p.Version,
nil,
"",
).ToString()
s.Artifacts.PackageCatalog.Add(p)
}
actual := toGithubModel(&s)
expected := DependencySnapshot{
Version: 0,
Detector: DetectorMetadata{
Name: "syft",
Version: "0.0.0-dev",
URL: "https://github.com/anchore/syft",
},
Metadata: Metadata{
"syft:distro": "pkg:generic/ubuntu@18.04?like=debian",
},
Scanned: actual.Scanned,
Manifests: Manifests{
"ubuntu:18.04:/usr/lib": Manifest{
Name: "ubuntu:18.04:/usr/lib",
File: FileInfo{
SourceLocation: "ubuntu:18.04:/usr/lib",
},
Metadata: Metadata{
"syft:filesystem": "fsid-1",
},
Resolved: DependencyGraph{
"pkg:generic/pkg-1@1.0.1": DependencyNode{
Purl: "pkg:generic/pkg-1@1.0.1",
Scope: DependencyScopeRuntime,
Relationship: DependencyRelationshipDirect,
},
"pkg:generic/pkg-2@2.0.2": DependencyNode{
Purl: "pkg:generic/pkg-2@2.0.2",
Scope: DependencyScopeRuntime,
Relationship: DependencyRelationshipDirect,
},
},
},
"ubuntu:18.04:/etc": Manifest{
Name: "ubuntu:18.04:/etc",
File: FileInfo{
SourceLocation: "ubuntu:18.04:/etc",
},
Metadata: Metadata{
"syft:filesystem": "fsid-1",
},
Resolved: DependencyGraph{
"pkg:generic/pkg-3@3.0.3": DependencyNode{
Purl: "pkg:generic/pkg-3@3.0.3",
Scope: DependencyScopeRuntime,
Relationship: DependencyRelationshipDirect,
},
},
},
},
}
// just using JSONEq because it gives a comprehensible diff
s1, _ := json.Marshal(expected)
s2, _ := json.Marshal(actual)
assert.JSONEq(t, string(s1), string(s2))
// Just test the other schemes:
s.Source.Path = "."
s.Source.Scheme = source.DirectoryScheme
actual = toGithubModel(&s)
assert.Equal(t, "etc", actual.Manifests["etc"].Name)
s.Source.Path = "./artifacts"
s.Source.Scheme = source.DirectoryScheme
actual = toGithubModel(&s)
assert.Equal(t, "artifacts/etc", actual.Manifests["artifacts/etc"].Name)
s.Source.Path = "/artifacts"
s.Source.Scheme = source.DirectoryScheme
actual = toGithubModel(&s)
assert.Equal(t, "/artifacts/etc", actual.Manifests["/artifacts/etc"].Name)
s.Source.Path = "./executable"
s.Source.Scheme = source.FileScheme
actual = toGithubModel(&s)
assert.Equal(t, "executable", actual.Manifests["executable"].Name)
s.Source.Path = "./archive.tar.gz"
s.Source.Scheme = source.FileScheme
actual = toGithubModel(&s)
assert.Equal(t, "archive.tar.gz:/etc", actual.Manifests["archive.tar.gz:/etc"].Name)
}
+29
View File
@@ -0,0 +1,29 @@
package github
import (
"encoding/json"
"io"
"github.com/anchore/syft/syft/sbom"
)
const ID sbom.FormatID = "github-0-json"
func Format() sbom.Format {
return sbom.NewFormat(
ID,
func(writer io.Writer, sbom sbom.SBOM) error {
bom := toGithubModel(&sbom)
bytes, err := json.MarshalIndent(bom, "", " ")
if err != nil {
return err
}
_, err = writer.Write(bytes)
return err
},
nil,
nil,
)
}
@@ -0,0 +1,78 @@
package github
// Derived from: https://gist.github.com/reiddraper/fdab2883db0f372c146d1a750fc1c43f
type DependencySnapshot struct {
Version int `json:"version"`
Job Job `json:"job,omitempty"` // !omitempty
Sha string `json:"sha,omitempty"` // !omitempty sha of the Git commit
Ref string `json:"ref,omitempty"` // !omitempty ref of the Git commit example "refs/heads/main"
Detector DetectorMetadata `json:"detector,omitempty"`
Metadata Metadata `json:"metadata,omitempty"`
Manifests Manifests `json:"manifests,omitempty"`
Scanned ISO8601Date `json:"scanned,omitempty"`
}
type Job struct {
Name string `json:"name,omitempty"` // !omitempty
ID string `json:"id,omitempty"` // !omitempty
HTMLURL string `json:"html_url,omitempty"`
}
type DetectorMetadata struct {
Name string `json:"name,omitempty"`
URL string `json:"url,omitempty"`
Version string `json:"version,omitempty"`
}
type Manifests map[string]Manifest
// Manifest A collection of related dependencies, either declared in a file,
// or representing a logical group of dependencies.
type Manifest struct {
Name string `json:"name"`
File FileInfo `json:"file"`
Metadata Metadata `json:"metadata,omitempty"`
Resolved DependencyGraph `json:"resolved,omitempty"`
}
type FileInfo struct {
SourceLocation string `json:"source_location,omitempty"`
}
// DependencyRelationship A notation of whether a dependency is requested directly
// by this manifest, or is a dependency of another dependency.
type DependencyRelationship string
const (
DependencyRelationshipDirect DependencyRelationship = "direct"
DependencyRelationshipIndirect DependencyRelationship = "indirect"
)
// DependencyScope A notation of whether the dependency is required for the primary
// build artifact (runtime), or is only used for development.
// Future versions of this specification may allow for more granular
// scopes, like `runtimeserver`, `runtimeshipped`,
// `developmenttest`, `developmentbenchmark`.
type DependencyScope string
const (
DependencyScopeRuntime DependencyScope = "runtime"
DependencyScopeDevelopment DependencyScope = "development"
)
type DependencyNode struct {
Purl string `json:"purl,omitempty"`
Metadata Metadata `json:"metadata,omitempty"`
Relationship DependencyRelationship `json:"relationship,omitempty"`
Scope DependencyScope `json:"scope,omitempty"`
Dependencies []string `json:"dependencies,omitempty"`
}
type DependencyGraph map[string]DependencyNode
type ISO8601Date = string
type Scalar interface{} // should be: null | boolean | string | number
type Metadata map[string]Scalar
+2 -2
View File
@@ -71,11 +71,11 @@ func TestSPDXJSONDecoder(t *testing.T) {
}
if test.packages != nil {
assert.Equal(t, sbom.Artifacts.Packages.Size(), len(test.packages))
assert.Equal(t, sbom.Artifacts.PackageCatalog.PackageCount(), len(test.packages))
packages:
for _, pkgName := range test.packages {
for _, p := range sbom.Artifacts.Packages.Sorted() {
for _, p := range sbom.Artifacts.PackageCatalog.Sorted() {
if p.Name == pkgName {
continue packages
}
@@ -16,6 +16,7 @@ import (
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
// toFormatModel creates and populates a new JSON document struct that follows the SPDX 2.2 spec from the given cataloging results.
@@ -42,13 +43,13 @@ func toFormatModel(s sbom.SBOM) (*model.Document, error) {
},
DataLicense: "CC0-1.0",
DocumentNamespace: namespace,
Packages: toPackages(s.Artifacts.Packages, s.Relationships),
Packages: toPackages(s.Artifacts.PackageCatalog, s.Relationships),
Files: toFiles(s),
Relationships: toRelationships(s.Relationships),
}, nil
}
func toPackages(catalog pkg.Collection, relationships []artifact.Relationship) []model.Package {
func toPackages(catalog *pkg.Catalog, relationships []artifact.Relationship) []model.Package {
packages := make([]model.Package, 0)
for _, p := range catalog.Sorted() {
@@ -93,7 +94,7 @@ func fileIDsForPackage(packageSpdxID string, relationships []artifact.Relationsh
continue
}
if _, ok := relationship.To.(file.Coordinates); !ok {
if _, ok := relationship.To.(source.Coordinates); !ok {
continue
}
@@ -109,7 +110,7 @@ func toFiles(s sbom.SBOM) []model.File {
artifacts := s.Artifacts
for _, coordinates := range sbom.AllCoordinates(s) {
var metadata *file.Metadata
var metadata *source.FileMetadata
if metadataForLocation, exists := artifacts.FileMetadata[coordinates]; exists {
metadata = &metadataForLocation
}
@@ -164,7 +165,7 @@ func toChecksumAlgorithm(algorithm string) string {
return strings.ToUpper(algorithm)
}
func toFileTypes(metadata *file.Metadata) (ty []string) {
func toFileTypes(metadata *source.FileMetadata) (ty []string) {
if metadata == nil {
return nil
}
@@ -11,6 +11,7 @@ import (
"github.com/anchore/syft/internal/formats/common/spdxhelpers"
"github.com/anchore/syft/internal/formats/spdx22json/model"
"github.com/anchore/syft/syft/source"
"github.com/stretchr/testify/assert"
)
@@ -18,12 +19,12 @@ func Test_toFileTypes(t *testing.T) {
tests := []struct {
name string
metadata file.Metadata
metadata source.FileMetadata
expected []string
}{
{
name: "application",
metadata: file.Metadata{
metadata: source.FileMetadata{
MIMEType: "application/vnd.unknown",
},
expected: []string{
@@ -32,7 +33,7 @@ func Test_toFileTypes(t *testing.T) {
},
{
name: "archive",
metadata: file.Metadata{
metadata: source.FileMetadata{
MIMEType: "application/zip",
},
expected: []string{
@@ -42,7 +43,7 @@ func Test_toFileTypes(t *testing.T) {
},
{
name: "audio",
metadata: file.Metadata{
metadata: source.FileMetadata{
MIMEType: "audio/ogg",
},
expected: []string{
@@ -51,7 +52,7 @@ func Test_toFileTypes(t *testing.T) {
},
{
name: "video",
metadata: file.Metadata{
metadata: source.FileMetadata{
MIMEType: "video/3gpp",
},
expected: []string{
@@ -60,7 +61,7 @@ func Test_toFileTypes(t *testing.T) {
},
{
name: "text",
metadata: file.Metadata{
metadata: source.FileMetadata{
MIMEType: "text/html",
},
expected: []string{
@@ -69,7 +70,7 @@ func Test_toFileTypes(t *testing.T) {
},
{
name: "image",
metadata: file.Metadata{
metadata: source.FileMetadata{
MIMEType: "image/png",
},
expected: []string{
@@ -78,7 +79,7 @@ func Test_toFileTypes(t *testing.T) {
},
{
name: "binary",
metadata: file.Metadata{
metadata: source.FileMetadata{
MIMEType: "application/x-sharedlib",
},
expected: []string{
@@ -174,7 +175,7 @@ func Test_fileIDsForPackage(t *testing.T) {
Name: "bogus",
}
c := file.Coordinates{
c := source.Coordinates{
RealPath: "/path",
FileSystemID: "nowhere",
}
@@ -85,13 +85,13 @@ func toFormatModel(s sbom.SBOM) (*spdx.Document2_2, error) {
// Cardinality: optional, one
DocumentComment: "",
},
Packages: toFormatPackages(s.Artifacts.Packages),
Packages: toFormatPackages(s.Artifacts.PackageCatalog),
}, nil
}
// packages populates all Package Information from the package Collection (see https://spdx.github.io/spdx-spec/3-package-information/)
// packages populates all Package Information from the package Catalog (see https://spdx.github.io/spdx-spec/3-package-information/)
// nolint: funlen
func toFormatPackages(catalog pkg.Collection) map[spdx.ElementID]*spdx.Package2_2 {
func toFormatPackages(catalog *pkg.Catalog) map[spdx.ElementID]*spdx.Package2_2 {
results := make(map[spdx.ElementID]*spdx.Package2_2)
for _, p := range catalog.Sorted() {
+2 -2
View File
@@ -28,8 +28,8 @@ func TestEncodeDecodeCycle(t *testing.T) {
t.Errorf("metadata difference: %+v", d)
}
actualPackages := actualSBOM.Artifacts.Packages.Sorted()
for idx, p := range originalSBOM.Artifacts.Packages.Sorted() {
actualPackages := actualSBOM.Artifacts.PackageCatalog.Sorted()
for idx, p := range originalSBOM.Artifacts.PackageCatalog.Sorted() {
if !assert.Equal(t, p.Name, actualPackages[idx].Name) {
t.Errorf("different package at idx=%d: %s vs %s", idx, p.Name, actualPackages[idx].Name)
continue
+21 -22
View File
@@ -2,7 +2,6 @@ package syftjson
import (
"flag"
"github.com/anchore/syft/syft/cpe"
"testing"
"github.com/anchore/syft/syft/file"
@@ -38,14 +37,14 @@ func TestImageEncoder(t *testing.T) {
}
func TestEncodeFullJSONDocument(t *testing.T) {
catalog := pkg.NewCollection()
catalog := pkg.NewCatalog()
p1 := pkg.Package{
Name: "package-1",
Version: "1.0.1",
Locations: []file.Location{
Locations: []source.Location{
{
Coordinates: file.Coordinates{
Coordinates: source.Coordinates{
RealPath: "/a/place/a",
},
},
@@ -61,17 +60,17 @@ func TestEncodeFullJSONDocument(t *testing.T) {
Files: []pkg.PythonFileRecord{},
},
PURL: "a-purl-1",
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"),
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:1:*:*:*:*:*:*:*"),
},
}
p2 := pkg.Package{
Name: "package-2",
Version: "2.0.1",
Locations: []file.Location{
Locations: []source.Location{
{
Coordinates: file.Coordinates{
Coordinates: source.Coordinates{
RealPath: "/b/place/b",
},
},
@@ -85,8 +84,8 @@ func TestEncodeFullJSONDocument(t *testing.T) {
Files: []pkg.DpkgFileRecord{},
},
PURL: "a-purl-2",
CPEs: []cpe.CPE{
cpe.Must("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
CPEs: []pkg.CPE{
pkg.MustCPE("cpe:2.3:*:some:package:2:*:*:*:*:*:*:*"),
},
}
@@ -95,50 +94,50 @@ func TestEncodeFullJSONDocument(t *testing.T) {
s := sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: catalog,
FileMetadata: map[file.Coordinates]file.Metadata{
file.NewLocation("/a/place").Coordinates: {
PackageCatalog: catalog,
FileMetadata: map[source.Coordinates]source.FileMetadata{
source.NewLocation("/a/place").Coordinates: {
Mode: 0775,
Type: "directory",
UserID: 0,
GroupID: 0,
},
file.NewLocation("/a/place/a").Coordinates: {
source.NewLocation("/a/place/a").Coordinates: {
Mode: 0775,
Type: "regularFile",
UserID: 0,
GroupID: 0,
},
file.NewLocation("/b").Coordinates: {
source.NewLocation("/b").Coordinates: {
Mode: 0775,
Type: "symbolicLink",
LinkDestination: "/c",
UserID: 0,
GroupID: 0,
},
file.NewLocation("/b/place/b").Coordinates: {
source.NewLocation("/b/place/b").Coordinates: {
Mode: 0644,
Type: "regularFile",
UserID: 1,
GroupID: 2,
},
},
FileDigests: map[file.Coordinates][]file.Digest{
file.NewLocation("/a/place/a").Coordinates: {
FileDigests: map[source.Coordinates][]file.Digest{
source.NewLocation("/a/place/a").Coordinates: {
{
Algorithm: "sha256",
Value: "366a3f5653e34673b875891b021647440d0127c2ef041e3b1a22da2a7d4f3703",
},
},
file.NewLocation("/b/place/b").Coordinates: {
source.NewLocation("/b/place/b").Coordinates: {
{
Algorithm: "sha256",
Value: "1b3722da2a7d90d033b87581a2a3f12021647445653e34666ef041e3b4f3707c",
},
},
},
FileContents: map[file.Coordinates]string{
file.NewLocation("/a/place/a").Coordinates: "the-contents",
FileContents: map[source.Coordinates]string{
source.NewLocation("/a/place/a").Coordinates: "the-contents",
},
LinuxDistribution: &linux.Release{
ID: "redhat",
@@ -160,7 +159,7 @@ func TestEncodeFullJSONDocument(t *testing.T) {
},
},
Source: source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "user-image-input",
ID: "sha256:c2b46b4eb06296933b7cf0722683964e9ecbd93265b9ef6ae9642e3952afbba0",
+9 -7
View File
@@ -2,11 +2,13 @@ package model
import (
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/source"
)
type File struct {
ID string `json:"id"`
Location file.Coordinates `json:"location"`
Location source.Coordinates `json:"location"`
Metadata *FileMetadataEntry `json:"metadata,omitempty"`
Contents string `json:"contents,omitempty"`
Digests []file.Digest `json:"digests,omitempty"`
@@ -14,10 +16,10 @@ type File struct {
}
type FileMetadataEntry struct {
Mode int `json:"mode"`
Type file.Type `json:"type"`
LinkDestination string `json:"linkDestination,omitempty"`
UserID int `json:"userID"`
GroupID int `json:"groupID"`
MIMEType string `json:"mimeType"`
Mode int `json:"mode"`
Type source.FileType `json:"type"`
LinkDestination string `json:"linkDestination,omitempty"`
UserID int `json:"userID"`
GroupID int `json:"groupID"`
MIMEType string `json:"mimeType"`
}
+11 -11
View File
@@ -4,7 +4,7 @@ import (
"encoding/json"
"fmt"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/pkg"
@@ -18,16 +18,16 @@ type Package struct {
// PackageBasicData contains non-ambiguous values (type-wise) from pkg.Package.
type PackageBasicData struct {
ID string `json:"id"`
Name string `json:"name"`
Version string `json:"version"`
Type pkg.Type `json:"type"`
FoundBy string `json:"foundBy"`
Locations []file.Coordinates `json:"locations"`
Licenses []string `json:"licenses"`
Language pkg.Language `json:"language"`
CPEs []string `json:"cpes"`
PURL string `json:"purl"`
ID string `json:"id"`
Name string `json:"name"`
Version string `json:"version"`
Type pkg.Type `json:"type"`
FoundBy string `json:"foundBy"`
Locations []source.Coordinates `json:"locations"`
Licenses []string `json:"licenses"`
Language pkg.Language `json:"language"`
CPEs []string `json:"cpes"`
PURL string `json:"purl"`
}
// PackageCustomData contains ambiguous values (type-wise) from pkg.Package.
+2 -1
View File
@@ -2,9 +2,10 @@ package model
import (
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/source"
)
type Secrets struct {
Location file.Coordinates `json:"location"`
Location source.Coordinates `json:"location"`
Secrets []file.SearchResult `json:"secrets"`
}
@@ -88,7 +88,7 @@
}
},
"schema": {
"version": "3.0.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.0.1.json"
"version": "3.1.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.1.1.json"
}
}
@@ -184,7 +184,7 @@
}
},
"schema": {
"version": "3.0.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.0.1.json"
"version": "3.1.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.1.1.json"
}
}
@@ -111,7 +111,7 @@
}
},
"schema": {
"version": "3.0.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.0.1.json"
"version": "3.1.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.1.1.json"
}
}
+10 -11
View File
@@ -2,7 +2,6 @@ package syftjson
import (
"fmt"
"github.com/anchore/syft/syft/cpe"
"sort"
"strconv"
@@ -31,7 +30,7 @@ func ToFormatModel(s sbom.SBOM) model.Document {
}
return model.Document{
Artifacts: toPackageModels(s.Artifacts.Packages),
Artifacts: toPackageModels(s.Artifacts.PackageCatalog),
ArtifactRelationships: toRelationshipModel(s.Relationships),
Files: toFile(s),
Secrets: toSecrets(s.Artifacts.Secrets),
@@ -74,7 +73,7 @@ func toDescriptor(d sbom.Descriptor) model.Descriptor {
}
}
func toSecrets(data map[file.Coordinates][]file.SearchResult) []model.Secrets {
func toSecrets(data map[source.Coordinates][]file.SearchResult) []model.Secrets {
results := make([]model.Secrets, 0)
for coordinates, secrets := range data {
results = append(results, model.Secrets{
@@ -95,7 +94,7 @@ func toFile(s sbom.SBOM) []model.File {
artifacts := s.Artifacts
for _, coordinates := range sbom.AllCoordinates(s) {
var metadata *file.Metadata
var metadata *source.FileMetadata
if metadataForLocation, exists := artifacts.FileMetadata[coordinates]; exists {
metadata = &metadataForLocation
}
@@ -132,7 +131,7 @@ func toFile(s sbom.SBOM) []model.File {
return results
}
func toFileMetadataEntry(coordinates file.Coordinates, metadata *file.Metadata) *model.FileMetadataEntry {
func toFileMetadataEntry(coordinates source.Coordinates, metadata *source.FileMetadata) *model.FileMetadataEntry {
if metadata == nil {
return nil
}
@@ -153,7 +152,7 @@ func toFileMetadataEntry(coordinates file.Coordinates, metadata *file.Metadata)
}
}
func toPackageModels(catalog pkg.Collection) []model.Package {
func toPackageModels(catalog *pkg.Catalog) []model.Package {
artifacts := make([]model.Package, 0)
if catalog == nil {
return artifacts
@@ -168,7 +167,7 @@ func toPackageModels(catalog pkg.Collection) []model.Package {
func toPackageModel(p pkg.Package) model.Package {
var cpes = make([]string, len(p.CPEs))
for i, c := range p.CPEs {
cpes[i] = cpe.String(c)
cpes[i] = pkg.CPEString(c)
}
var licenses = make([]string, 0)
@@ -176,7 +175,7 @@ func toPackageModel(p pkg.Package) model.Package {
licenses = p.Licenses
}
var coordinates = make([]file.Coordinates, len(p.Locations))
var coordinates = make([]source.Coordinates, len(p.Locations))
for i, l := range p.Locations {
coordinates[i] = l.Coordinates
}
@@ -217,7 +216,7 @@ func toRelationshipModel(relationships []artifact.Relationship) []model.Relation
// toSourceModel creates a new source object to be represented into JSON.
func toSourceModel(src source.Metadata) (model.Source, error) {
switch src.Scheme {
case source.ImageType:
case source.ImageScheme:
metadata := src.ImageMetadata
// ensure that empty collections are not shown as null
if metadata.RepoDigests == nil {
@@ -230,12 +229,12 @@ func toSourceModel(src source.Metadata) (model.Source, error) {
Type: "image",
Target: metadata,
}, nil
case source.DirectoryType:
case source.DirectoryScheme:
return model.Source{
Type: "directory",
Target: src.Path,
}, nil
case source.FileType:
case source.FileScheme:
return model.Source{
Type: "file",
Target: src.Path,
@@ -13,7 +13,7 @@ import (
func Test_toSourceModel(t *testing.T) {
allSchemes := strset.New()
for _, s := range source.AllTypes {
for _, s := range source.AllSchemes {
allSchemes.Add(string(s))
}
testedSchemes := strset.New()
@@ -26,7 +26,7 @@ func Test_toSourceModel(t *testing.T) {
{
name: "directory",
src: source.Metadata{
Scheme: source.DirectoryType,
Scheme: source.DirectoryScheme,
Path: "some/path",
},
expected: model.Source{
@@ -37,7 +37,7 @@ func Test_toSourceModel(t *testing.T) {
{
name: "file",
src: source.Metadata{
Scheme: source.FileType,
Scheme: source.FileScheme,
Path: "some/path",
},
expected: model.Source{
@@ -48,7 +48,7 @@ func Test_toSourceModel(t *testing.T) {
{
name: "image",
src: source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "user-input",
ID: "id...",
+11 -13
View File
@@ -4,8 +4,6 @@ import (
"github.com/anchore/syft/internal/formats/syftjson/model"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/linux"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
@@ -18,7 +16,7 @@ func toSyftModel(doc model.Document) (*sbom.SBOM, error) {
return &sbom.SBOM{
Artifacts: sbom.Artifacts{
Packages: catalog,
PackageCatalog: catalog,
LinuxDistribution: toSyftLinuxRelease(doc.Distro),
},
Source: *toSyftSourceData(doc.Source),
@@ -48,7 +46,7 @@ func toSyftLinuxRelease(d model.LinuxRelease) *linux.Release {
}
}
func toSyftRelationships(doc *model.Document, catalog pkg.Collection, relationships []model.Relationship) []artifact.Relationship {
func toSyftRelationships(doc *model.Document, catalog *pkg.Catalog, relationships []model.Relationship) []artifact.Relationship {
idMap := make(map[string]interface{})
for _, p := range catalog.Sorted() {
@@ -113,25 +111,25 @@ func toSyftSourceData(s model.Source) *source.Metadata {
switch s.Type {
case "directory":
return &source.Metadata{
Scheme: source.DirectoryType,
Scheme: source.DirectoryScheme,
Path: s.Target.(string),
}
case "file":
return &source.Metadata{
Scheme: source.FileType,
Scheme: source.FileScheme,
Path: s.Target.(string),
}
case "image":
return &source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: s.Target.(source.ImageMetadata),
}
}
return nil
}
func toSyftCatalog(pkgs []model.Package) pkg.Collection {
catalog := pkg.NewCollection()
func toSyftCatalog(pkgs []model.Package) *pkg.Catalog {
catalog := pkg.NewCatalog()
for _, p := range pkgs {
catalog.Add(toSyftPackage(p))
}
@@ -139,9 +137,9 @@ func toSyftCatalog(pkgs []model.Package) pkg.Collection {
}
func toSyftPackage(p model.Package) pkg.Package {
var cpes []cpe.CPE
var cpes []pkg.CPE
for _, c := range p.CPEs {
value, err := cpe.New(c)
value, err := pkg.NewCPE(c)
if err != nil {
log.Warnf("excluding invalid CPE %q: %v", c, err)
continue
@@ -150,9 +148,9 @@ func toSyftPackage(p model.Package) pkg.Package {
cpes = append(cpes, value)
}
var locations = make([]file.Location, len(p.Locations))
var locations = make([]source.Location, len(p.Locations))
for i, c := range p.Locations {
locations[i] = file.NewLocationFromCoordinates(c)
locations[i] = source.NewLocationFromCoordinates(c)
}
return pkg.Package{
@@ -11,7 +11,7 @@ import (
func Test_toSyftSourceData(t *testing.T) {
allSchemes := strset.New()
for _, s := range source.AllTypes {
for _, s := range source.AllSchemes {
allSchemes.Add(string(s))
}
testedSchemes := strset.New()
@@ -24,7 +24,7 @@ func Test_toSyftSourceData(t *testing.T) {
{
name: "directory",
expected: source.Metadata{
Scheme: source.DirectoryType,
Scheme: source.DirectoryScheme,
Path: "some/path",
},
src: model.Source{
@@ -35,7 +35,7 @@ func Test_toSyftSourceData(t *testing.T) {
{
name: "file",
expected: source.Metadata{
Scheme: source.FileType,
Scheme: source.FileScheme,
Path: "some/path",
},
src: model.Source{
@@ -46,7 +46,7 @@ func Test_toSyftSourceData(t *testing.T) {
{
name: "image",
expected: source.Metadata{
Scheme: source.ImageType,
Scheme: source.ImageScheme,
ImageMetadata: source.ImageMetadata{
UserInput: "user-input",
ID: "id...",
+1 -1
View File
@@ -15,7 +15,7 @@ func encoder(output io.Writer, s sbom.SBOM) error {
var rows [][]string
columns := []string{"Name", "Version", "Type"}
for _, p := range s.Artifacts.Packages.Sorted() {
for _, p := range s.Artifacts.PackageCatalog.Sorted() {
row := []string{
p.Name,
p.Version,
+3 -3
View File
@@ -16,9 +16,9 @@ func encoder(output io.Writer, s sbom.SBOM) error {
w.Init(output, 0, 8, 0, '\t', tabwriter.AlignRight)
switch s.Source.Scheme {
case source.DirectoryType, source.FileType:
case source.DirectoryScheme, source.FileScheme:
fmt.Fprintf(w, "[Path: %s]\n", s.Source.Path)
case source.ImageType:
case source.ImageScheme:
fmt.Fprintln(w, "[Image]")
for idx, l := range s.Source.ImageMetadata.Layers {
@@ -35,7 +35,7 @@ func encoder(output io.Writer, s sbom.SBOM) error {
// populate artifacts...
rows := 0
for _, p := range s.Artifacts.Packages.Sorted() {
for _, p := range s.Artifacts.PackageCatalog.Sorted() {
fmt.Fprintf(w, "[%s]\n", p.Name)
fmt.Fprintln(w, " Version:\t", p.Version)
fmt.Fprintln(w, " Type:\t", string(p.Type))
-47
View File
@@ -1,47 +0,0 @@
package version
import (
"runtime/debug"
"strings"
"github.com/anchore/syft/internal/log"
)
func Guess() string {
v := FromBuild().Version
if strings.HasPrefix(v, "v") {
return v
}
buildInfo, ok := debug.ReadBuildInfo()
if !ok {
log.Warn("syft version could not be determined: unable to find the buildinfo section of the binary")
return v
}
var found bool
for _, d := range buildInfo.Deps {
if d.Path == "github.com/anchore/syft" {
v = d.Version
found = true
break
}
}
if !found {
// look for probable forks
for _, d := range buildInfo.Deps {
if strings.HasSuffix(d.Path, "/syft") {
v = d.Version
found = true
break
}
}
}
if !found {
log.Warn("syft version could not be determined: unable to find syft within the buildinfo section of the binary")
}
return v
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
-172
View File
@@ -1,172 +0,0 @@
package syft
import (
"fmt"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/event/monitor"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/hashicorp/go-multierror"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
)
type monitorableCollection struct {
pkg.Collection
monitor *progress.Manual
}
func (m *monitorableCollection) Add(p pkg.Package) {
m.monitor.N++
m.Collection.Add(p)
}
func Catalog(src *source.Source, options ...CatalogingOption) (*sbom.SBOM, error) {
var config = DefaultCatalogingConfig()
for _, optFn := range options {
if err := optFn(src, &config); err != nil {
return nil, fmt.Errorf("unable to apply cataloging option: %w", err)
}
}
if config.availableTasks == nil {
var err error
config.availableTasks, err = newTaskCollection()
if err != nil {
return nil, err
}
}
if len(config.EnabledCatalogers) == 0 {
return nil, fmt.Errorf("no cataloging tasks configured to run")
}
catalogingTasks, err := config.availableTasks.tasks(config, config.EnabledCatalogers...)
if err != nil {
return nil, err
}
if len(catalogingTasks) == 0 {
return nil, fmt.Errorf("no cataloging tasks found to run")
}
// special case: we need to identify the linux distro for downstream processing
identifyLinuxDistroTask, err := newIdentifyDistroTask(config)
if err != nil {
return nil, fmt.Errorf("unable to create linux distro identification task: %+v", err)
}
synthesizePackageRelationshipsTask, err := newSynthesizePackageRelationshipsTasks(config)
if err != nil {
return nil, fmt.Errorf("unable to create task to synthesize package relationships: %+v", err)
}
taskGroups := [][]task{
{
identifyLinuxDistroTask,
},
catalogingTasks,
{
synthesizePackageRelationshipsTask,
},
}
files, pkgs := newCatalogerMonitor()
defer func() {
files.SetCompleted() // TODO: files monitor is unused... should we remove?
pkgs.SetCompleted()
}()
s := sbom.SBOM{
Source: src.Metadata,
Descriptor: sbom.Descriptor{
Name: config.ToolName,
Version: config.ToolVersion,
Configuration: config.ToolConfiguration,
},
Artifacts: sbom.Artifacts{
Packages: &monitorableCollection{
Collection: pkg.NewCollection(),
monitor: pkgs,
},
},
}
for _, tasks := range taskGroups {
if err := runTasks(&s, src, config.ProcessTasksInSerial, tasks...); err != nil {
return &s, err
}
}
return &s, nil
}
// newCatalogerMonitor creates a new CatalogingMonitor object and publishes the object on the bus as a CatalogingStarted event.
func newCatalogerMonitor() (*progress.Manual, *progress.Manual) {
filesProcessed := progress.Manual{}
packagesDiscovered := progress.Manual{}
bus.Publish(partybus.Event{
Type: event.CatalogingStarted,
Value: monitor.CatalogingMonitor{
FilesProcessed: progress.Monitorable(&filesProcessed),
PackagesDiscovered: progress.Monitorable(&packagesDiscovered),
},
})
return &filesProcessed, &packagesDiscovered
}
func runTasks(s *sbom.SBOM, src *source.Source, serial bool, tasks ...task) error {
var relationships []<-chan artifact.Relationship
var errs = make(chan error)
for _, t := range tasks {
r := make(chan artifact.Relationship)
relationships = append(relationships, r)
if serial {
runTask(t, &s.Artifacts, src, r, errs)
} else {
go runTask(t, &s.Artifacts, src, r, errs)
}
}
s.Relationships = append(s.Relationships, mergeRelationships(relationships...)...)
close(errs)
return mergeErrors(errs)
}
func mergeRelationships(cs ...<-chan artifact.Relationship) (relationships []artifact.Relationship) {
for _, c := range cs {
for n := range c {
relationships = append(relationships, n)
}
}
return relationships
}
func mergeErrors(errs <-chan error) (allErrs error) {
for err := range errs {
if err != nil {
allErrs = multierror.Append(allErrs, err)
}
}
return allErrs
}
func runTask(t task, a *sbom.Artifacts, src *source.Source, r chan<- artifact.Relationship, errs chan<- error) {
defer close(r)
relationships, err := t.Run(a, src)
if err != nil {
errs <- err
return
}
for _, relationship := range relationships {
r <- relationship
}
}
@@ -1,82 +0,0 @@
package filecontents
import (
"github.com/anchore/syft/syft/file"
"testing"
"github.com/stretchr/testify/assert"
)
func TestContentsCataloger(t *testing.T) {
allFiles := []string{"test-fixtures/last/path.txt", "test-fixtures/another-path.txt", "test-fixtures/a-path.txt"}
tests := []struct {
name string
globs []string
maxSize int64
files []string
expected map[file.Coordinates]string
}{
{
name: "multi-pattern",
globs: []string{"test-fixtures/last/*.txt", "test-fixtures/*.txt"},
files: allFiles,
expected: map[file.Coordinates]string{
file.NewLocation("test-fixtures/last/path.txt").Coordinates: "dGVzdC1maXh0dXJlcy9sYXN0L3BhdGgudHh0IGZpbGUgY29udGVudHMh",
file.NewLocation("test-fixtures/another-path.txt").Coordinates: "dGVzdC1maXh0dXJlcy9hbm90aGVyLXBhdGgudHh0IGZpbGUgY29udGVudHMh",
file.NewLocation("test-fixtures/a-path.txt").Coordinates: "dGVzdC1maXh0dXJlcy9hLXBhdGgudHh0IGZpbGUgY29udGVudHMh",
},
},
{
name: "no-patterns",
globs: []string{},
files: []string{"test-fixtures/last/path.txt", "test-fixtures/another-path.txt", "test-fixtures/a-path.txt"},
expected: map[file.Coordinates]string{},
},
{
name: "all-txt",
globs: []string{"**/*.txt"},
files: allFiles,
expected: map[file.Coordinates]string{
file.NewLocation("test-fixtures/last/path.txt").Coordinates: "dGVzdC1maXh0dXJlcy9sYXN0L3BhdGgudHh0IGZpbGUgY29udGVudHMh",
file.NewLocation("test-fixtures/another-path.txt").Coordinates: "dGVzdC1maXh0dXJlcy9hbm90aGVyLXBhdGgudHh0IGZpbGUgY29udGVudHMh",
file.NewLocation("test-fixtures/a-path.txt").Coordinates: "dGVzdC1maXh0dXJlcy9hLXBhdGgudHh0IGZpbGUgY29udGVudHMh",
},
},
{
name: "subpath",
globs: []string{"test-fixtures/*.txt"},
files: allFiles,
expected: map[file.Coordinates]string{
file.NewLocation("test-fixtures/another-path.txt").Coordinates: "dGVzdC1maXh0dXJlcy9hbm90aGVyLXBhdGgudHh0IGZpbGUgY29udGVudHMh",
file.NewLocation("test-fixtures/a-path.txt").Coordinates: "dGVzdC1maXh0dXJlcy9hLXBhdGgudHh0IGZpbGUgY29udGVudHMh",
},
},
{
name: "size-filter",
maxSize: 42,
globs: []string{"**/*.txt"},
files: allFiles,
expected: map[file.Coordinates]string{
file.NewLocation("test-fixtures/last/path.txt").Coordinates: "dGVzdC1maXh0dXJlcy9sYXN0L3BhdGgudHh0IGZpbGUgY29udGVudHMh",
file.NewLocation("test-fixtures/a-path.txt").Coordinates: "dGVzdC1maXh0dXJlcy9hLXBhdGgudHh0IGZpbGUgY29udGVudHMh",
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
c, err := NewCataloger(Config{
Globs: test.globs,
SkipFilesAboveSizeInBytes: test.maxSize,
})
assert.NoError(t, err)
resolver := file.NewMockResolverForPaths(test.files...)
actual, err := c.Catalog(resolver)
assert.NoError(t, err)
assert.Equal(t, test.expected, actual, "mismatched contents")
})
}
}
@@ -1,12 +0,0 @@
FROM busybox:latest
ADD file-1.txt .
RUN chmod 644 file-1.txt
RUN chown 1:2 file-1.txt
RUN ln -s file-1.txt symlink-1
RUN ln file-1.txt hardlink-1
RUN mknod char-device-1 c 89 1
RUN mknod block-device-1 b 0 1
RUN mknod fifo-1 p
RUN mkdir /dir
RUN rm -rf home etc/group etc/localtime etc/mtab etc/network etc/passwd etc/shadow var usr bin/*
@@ -1,12 +0,0 @@
FROM busybox:latest
ADD file-1.txt .
RUN chmod 644 file-1.txt
RUN chown 1:2 file-1.txt
RUN ln -s file-1.txt symlink-1
RUN ln file-1.txt hardlink-1
RUN mknod char-device-1 c 89 1
RUN mknod block-device-1 b 0 1
RUN mknod fifo-1 p
RUN mkdir /dir
RUN rm -rf home etc/group etc/localtime etc/mtab etc/network etc/passwd etc/shadow var usr bin/*
-40
View File
@@ -1,40 +0,0 @@
package cataloger
const (
ApkDBID ID = "os-apkdb"
DpkgID ID = "os-dpkg"
RpmDBID ID = "os-rpmdb"
RubyGemspecID ID = "ruby-gem-spec"
RubyGemfileLockID ID = "ruby-gem-file-lock"
PythonPackageID ID = "python-package"
PythonRequirementsID ID = "python-requirements"
PythonPoetryID ID = "python-poetry"
PythonSetupID ID = "python-setup"
PythonPipFileID ID = "python-pipfile"
JavascriptPackageJSONID ID = "javascript-package-json"
JavascriptPackageLockID ID = "javascript-package-lock"
JavaScriptYarnLockID ID = "javascript-yarn-lock"
JavaArchiveID ID = "java-archive"
GoModID ID = "go-mod"
GoBinaryID ID = "go-binary"
RustCargoLockID ID = "rust-cargo-lock"
PHPInstalledJSONID ID = "php-installed-json"
PHPComposerLockID ID = "php-composer-lock"
FileMetadataID ID = "file-metadata"
FileDigestsID ID = "file-digest"
SecretsID ID = "secrets"
FileClassifierID ID = "file-classifier"
FileContentsID ID = "file-content"
)
type ID string
type IDs []ID
func (c IDs) Len() int { return len(c) }
func (c IDs) Swap(i, j int) { c[i], c[j] = c[j], c[i] }
func (c IDs) Less(i, j int) bool {
return c[i] < c[j]
}
@@ -1,24 +0,0 @@
package packages
import (
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
func FindRelationships(catalog pkg.Collection, resolver file.Resolver) []artifact.Relationship {
var allRelationships []artifact.Relationship
for p := range catalog.Enumerate() {
relationships, err := createFileOwnershipRelationships(p, resolver)
if err != nil {
log.Warnf("unable to create any package-file relationships for package name=%q: %w", p.Name, err)
continue
}
allRelationships = append(allRelationships, relationships...)
}
allRelationships = append(allRelationships, findOwnershipByFileOverlapRelationship(catalog)...)
return allRelationships
}
-12
View File
@@ -1,12 +0,0 @@
package generic
import (
"io"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/artifact"
)
// Parser standardizes a function signature for parser functions that accept the virtual file path (not usable for file reads) and contents and return any discovered packages from that file
type Parser func(string, io.Reader) ([]*pkg.Package, []artifact.Relationship, error)
@@ -1 +0,0 @@
test-fixtures/last/path.txt file contents!
@@ -1,49 +0,0 @@
/*
Package golang provides a concrete Cataloger implementation for go.mod files.
*/
package golang
import (
"fmt"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
)
type Cataloger struct{}
// NewGoModuleBinaryCataloger returns a new Golang cataloger object.
func NewGoModuleBinaryCataloger() *Cataloger {
return &Cataloger{}
}
// Catalog is given an object to resolve file references and content, this function returns any discovered Packages after analyzing rpm db installation.
func (c *Cataloger) Catalog(resolver file.Resolver) ([]pkg.Package, []artifact.Relationship, error) {
var pkgs []pkg.Package
fileMatches, err := resolver.FilesByMIMEType(internal.ExecutableMIMETypeSet.List()...)
if err != nil {
return pkgs, nil, fmt.Errorf("failed to find bin by mime types: %w", err)
}
for _, location := range fileMatches {
r, err := resolver.FileContentsByLocation(location)
if err != nil {
return pkgs, nil, fmt.Errorf("failed to resolve file contents by location=%q: %w", location.RealPath, err)
}
goPkgs, err := parseGoBin(location, r, openExe)
if err != nil {
log.Warnf("could not parse possible go binary at %q: %+v", location.RealPath, err)
}
internal.CloseAndLogError(r, location.RealPath)
pkgs = append(pkgs, goPkgs...)
}
return pkgs, nil, nil
}
-354
View File
@@ -1,354 +0,0 @@
// This code was copied from the Go std library.
// https://github.com/golang/go/blob/master/src/cmd/go/internal/version/exe.go
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//nolint
package golang
import (
"bytes"
"debug/elf"
"debug/macho"
"debug/pe"
"fmt"
"io"
"strings"
)
// An exe is a generic interface to an OS executable (ELF, Mach-O, PE, XCOFF).
type exe interface {
// Close closes the underlying file.
Close() error
// ReadData reads and returns up to size byte starting at virtual address addr.
ReadData(addr, size uint64) ([]byte, error)
// ArchName returns a string that represents the CPU architecture of the executable.
ArchName() string
// DataStart returns the writable data segment start address.
DataStart() uint64
}
// openExe opens file and returns it as an exe.
// we changed this signature from accpeting a string
// to a ReadCloser so we could adapt the code to the
// stereoscope api. We removed the file open methods.
func openExe(file io.ReadCloser) ([]exe, error) {
/*
f, err := os.Open(file)
if err != nil {
return nil, err
}
data := make([]byte, 16)
if _, err := io.ReadFull(f, data); err != nil {
return nil, err
}
f.Seek(0, 0)
*/
data, err := io.ReadAll(file)
if err != nil {
return nil, err
}
r := bytes.NewReader(data)
f := io.NewSectionReader(r, 0, int64(len(data)))
if bytes.HasPrefix(data, []byte("\x7FELF")) {
e, err := elf.NewFile(f)
if err != nil {
return nil, err
}
return []exe{&elfExe{file, e}}, nil
}
if bytes.HasPrefix(data, []byte("MZ")) {
e, err := pe.NewFile(f)
if err != nil {
return nil, err
}
return []exe{&peExe{file, e}}, nil
}
if bytes.HasPrefix(data, []byte("\xFE\xED\xFA")) || bytes.HasPrefix(data[1:], []byte("\xFA\xED\xFE")) {
e, err := macho.NewFile(f)
if err != nil {
return nil, err
}
return []exe{&machoExe{file, e}}, nil
}
// adding macho multi-architecture support (both for 64bit and 32 bit)... this case is not in the stdlib yet
if bytes.HasPrefix(data, []byte("\xCA\xFE\xBA\xBE")) || bytes.HasPrefix(data, []byte("\xCA\xFE\xBA\xBF")) {
fatExe, err := macho.NewFatFile(f)
if err != nil {
return nil, err
}
var exes []exe
for _, arch := range fatExe.Arches {
exes = append(exes, &machoExe{file, arch.File})
}
return exes, nil
}
return nil, fmt.Errorf("unrecognized executable format")
}
// elfExe is the ELF implementation of the exe interface.
// updated os to be io.ReadCloser to interopt with stereoscope
type elfExe struct {
os io.ReadCloser
f *elf.File
}
func (x *elfExe) Close() error {
return x.os.Close()
}
func (x *elfExe) ArchName() string {
return cleanElfArch(x.f.Machine)
}
func cleanElfArch(machine elf.Machine) string {
return strings.TrimPrefix(strings.ToLower(machine.String()), "em_")
}
func (x *elfExe) ReadData(addr, size uint64) ([]byte, error) {
for _, prog := range x.f.Progs {
if prog.Vaddr <= addr && addr <= prog.Vaddr+prog.Filesz-1 {
n := prog.Vaddr + prog.Filesz - addr
if n > size {
n = size
}
data := make([]byte, n)
_, err := prog.ReadAt(data, int64(addr-prog.Vaddr))
if err != nil {
return nil, err
}
return data, nil
}
}
return nil, fmt.Errorf("address not mapped")
}
func (x *elfExe) DataStart() uint64 {
for _, s := range x.f.Sections {
if s.Name == ".go.buildinfo" {
return s.Addr
}
}
for _, p := range x.f.Progs {
if p.Type == elf.PT_LOAD && p.Flags&(elf.PF_X|elf.PF_W) == elf.PF_W {
return p.Vaddr
}
}
return 0
}
// peExe is the PE (Windows Portable Executable) implementation of the exe interface.
type peExe struct {
os io.ReadCloser
f *pe.File
}
func (x *peExe) Close() error {
return x.os.Close()
}
func (x *peExe) ArchName() string {
// from: debug/pe/pe.go
switch x.f.Machine {
case pe.IMAGE_FILE_MACHINE_AM33:
return "amd33"
case pe.IMAGE_FILE_MACHINE_AMD64:
return "amd64"
case pe.IMAGE_FILE_MACHINE_ARM:
return "arm"
case pe.IMAGE_FILE_MACHINE_ARMNT:
return "armnt"
case pe.IMAGE_FILE_MACHINE_ARM64:
return "arm64"
case pe.IMAGE_FILE_MACHINE_EBC:
return "ebc"
case pe.IMAGE_FILE_MACHINE_I386:
return "i386"
case pe.IMAGE_FILE_MACHINE_IA64:
return "ia64"
case pe.IMAGE_FILE_MACHINE_M32R:
return "m32r"
case pe.IMAGE_FILE_MACHINE_MIPS16:
return "mips16"
case pe.IMAGE_FILE_MACHINE_MIPSFPU:
return "mipsfpu"
case pe.IMAGE_FILE_MACHINE_MIPSFPU16:
return "mipsfpu16"
case pe.IMAGE_FILE_MACHINE_POWERPC:
return "ppc"
case pe.IMAGE_FILE_MACHINE_POWERPCFP:
return "ppcfp"
case pe.IMAGE_FILE_MACHINE_R4000:
return "r4000"
case pe.IMAGE_FILE_MACHINE_SH3:
return "sh3"
case pe.IMAGE_FILE_MACHINE_SH3DSP:
return "sh3dsp"
case pe.IMAGE_FILE_MACHINE_SH4:
return "sh4"
case pe.IMAGE_FILE_MACHINE_SH5:
return "sh5"
case pe.IMAGE_FILE_MACHINE_THUMB:
return "thumb"
case pe.IMAGE_FILE_MACHINE_WCEMIPSV2:
return "wcemipsv2"
default:
return fmt.Sprintf("unknown-pe-machine-%d", x.f.Machine)
}
}
func (x *peExe) imageBase() uint64 {
switch oh := x.f.OptionalHeader.(type) {
case *pe.OptionalHeader32:
return uint64(oh.ImageBase)
case *pe.OptionalHeader64:
return oh.ImageBase
}
return 0
}
func (x *peExe) ReadData(addr, size uint64) ([]byte, error) {
addr -= x.imageBase()
for _, sect := range x.f.Sections {
if uint64(sect.VirtualAddress) <= addr && addr <= uint64(sect.VirtualAddress+sect.Size-1) {
n := uint64(sect.VirtualAddress+sect.Size) - addr
if n > size {
n = size
}
data := make([]byte, n)
_, err := sect.ReadAt(data, int64(addr-uint64(sect.VirtualAddress)))
if err != nil {
return nil, err
}
return data, nil
}
}
return nil, fmt.Errorf("address not mapped")
}
func (x *peExe) DataStart() uint64 {
// Assume data is first writable section.
const (
IMAGE_SCN_CNT_CODE = 0x00000020
IMAGE_SCN_CNT_INITIALIZED_DATA = 0x00000040
IMAGE_SCN_CNT_UNINITIALIZED_DATA = 0x00000080
IMAGE_SCN_MEM_EXECUTE = 0x20000000
IMAGE_SCN_MEM_READ = 0x40000000
IMAGE_SCN_MEM_WRITE = 0x80000000
IMAGE_SCN_MEM_DISCARDABLE = 0x2000000
IMAGE_SCN_LNK_NRELOC_OVFL = 0x1000000
IMAGE_SCN_ALIGN_32BYTES = 0x600000
)
for _, sect := range x.f.Sections {
if sect.VirtualAddress != 0 && sect.Size != 0 &&
sect.Characteristics&^IMAGE_SCN_ALIGN_32BYTES == IMAGE_SCN_CNT_INITIALIZED_DATA|IMAGE_SCN_MEM_READ|IMAGE_SCN_MEM_WRITE {
return uint64(sect.VirtualAddress) + x.imageBase()
}
}
return 0
}
// machoExe is the Mach-O (Apple macOS/iOS) implementation of the exe interface.
type machoExe struct {
os io.ReadCloser
f *macho.File
}
func (x *machoExe) Close() error {
return x.os.Close()
}
func (x *machoExe) ArchName() string {
return cleanMachoArch(x.f.Cpu)
}
func cleanMachoArch(cpu macho.Cpu) string {
return strings.TrimPrefix(strings.ToLower(cpu.String()), "cpu")
}
func (x *machoExe) ReadData(addr, size uint64) ([]byte, error) {
for _, load := range x.f.Loads {
seg, ok := load.(*macho.Segment)
if !ok {
continue
}
if seg.Addr <= addr && addr <= seg.Addr+seg.Filesz-1 {
if seg.Name == "__PAGEZERO" {
continue
}
n := seg.Addr + seg.Filesz - addr
if n > size {
n = size
}
data := make([]byte, n)
_, err := seg.ReadAt(data, int64(addr-seg.Addr))
if err != nil {
return nil, err
}
return data, nil
}
}
return nil, fmt.Errorf("address not mapped")
}
func (x *machoExe) DataStart() uint64 {
// Look for section named "__go_buildinfo".
for _, sec := range x.f.Sections {
if sec.Name == "__go_buildinfo" {
return sec.Addr
}
}
// Try the first non-empty writable segment.
const RW = 3
for _, load := range x.f.Loads {
seg, ok := load.(*macho.Segment)
if ok && seg.Addr != 0 && seg.Filesz != 0 && seg.Prot == RW && seg.Maxprot == RW {
return seg.Addr
}
}
return 0
}
/*
// xcoffExe is the XCOFF (AIX eXtended COFF) implementation of the exe interface.
type xcoffExe struct {
os *os.File
f *xcoff.File
}
func (x *xcoffExe) Close() error {
return x.os.Close()
}
func (x *xcoffExe) ReadData(addr, size uint64) ([]byte, error) {
for _, sect := range x.f.Sections {
if uint64(sect.VirtualAddress) <= addr && addr <= uint64(sect.VirtualAddress+sect.Size-1) {
n := uint64(sect.VirtualAddress+sect.Size) - addr
if n > size {
n = size
}
data := make([]byte, n)
_, err := sect.ReadAt(data, int64(addr-uint64(sect.VirtualAddress)))
if err != nil {
return nil, err
}
return data, nil
}
}
return nil, fmt.Errorf("address not mapped")
}
func (x *xcoffExe) DataStart() uint64 {
return x.f.SectionByType(xcoff.STYP_DATA).VirtualAddress
}
*/
@@ -1,43 +0,0 @@
package golang
import (
"debug/elf"
"debug/macho"
"testing"
"github.com/stretchr/testify/assert"
)
func Test_cleanElfArch(t *testing.T) {
tests := []struct {
machine elf.Machine
want string
}{
{
machine: elf.EM_X86_64,
want: "x86_64",
},
}
for _, test := range tests {
t.Run(test.machine.String(), func(t *testing.T) {
assert.Equalf(t, test.want, cleanElfArch(test.machine), "cleanElfArch(%v)", test.machine)
})
}
}
func Test_cleanMachoArch(t *testing.T) {
tests := []struct {
cpu macho.Cpu
want string
}{
{
cpu: macho.CpuAmd64,
want: "amd64",
},
}
for _, test := range tests {
t.Run(test.cpu.String(), func(t *testing.T) {
assert.Equalf(t, test.want, cleanMachoArch(test.cpu), "cleanMachoArch(%v)", test.cpu)
})
}
}
@@ -1,17 +0,0 @@
/*
Package golang provides a concrete Cataloger implementation for go.mod files.
*/
package golang
import (
"github.com/anchore/syft/syft/cataloger/packages/generic"
)
// NewGoModFileCataloger returns a new Go module cataloger object.
func NewGoModFileCataloger() *generic.Cataloger {
globParsers := map[string]generic.Parser{
"**/go.mod": parseGoMod,
}
return generic.NewCataloger(nil, globParsers)
}
@@ -1,88 +0,0 @@
package golang
import (
"bufio"
"fmt"
"io"
"strings"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
const (
packageIdentifier = "dep"
replaceIdentifier = "=>"
)
type exeOpener func(file io.ReadCloser) ([]exe, error)
func newGoBinaryPackage(name, version, h1Digest, goVersion, architecture string, location file.Location) pkg.Package {
p := pkg.Package{
Name: name,
Version: version,
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: []file.Location{
location,
},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goVersion,
H1Digest: h1Digest,
Architecture: architecture,
},
}
p.SetID()
return p
}
func parseGoBin(location file.Location, reader io.ReadCloser, opener exeOpener) (pkgs []pkg.Package, err error) {
var exes []exe
// it has been found that there are stdlib paths within openExe that can panic. We want to prevent this behavior
// bubbling up and halting execution. For this reason we try to recover from any panic and return an error.
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("recovered from panic while parse go binary at %q: %+v", location.RealPath, r)
}
}()
// Identify if bin was compiled by go
exes, err = opener(reader)
if err != nil {
return pkgs, err
}
for _, x := range exes {
goVersion, mod := findVers(x)
pkgs = append(pkgs, buildGoPkgInfo(location, mod, goVersion, x.ArchName())...)
}
return pkgs, err
}
func buildGoPkgInfo(location file.Location, mod, goVersion, arch string) []pkg.Package {
pkgsSlice := make([]pkg.Package, 0)
scanner := bufio.NewScanner(strings.NewReader(mod))
// filter mod dependencies: [dep, name, version, sha]
for scanner.Scan() {
fields := strings.Fields(scanner.Text())
// must have dep, name, version, sha
if len(fields) < 4 {
continue
}
if fields[0] == packageIdentifier || fields[0] == replaceIdentifier {
name := fields[1]
version := fields[2]
h1Digest := fields[3]
pkgsSlice = append(pkgsSlice, newGoBinaryPackage(name, version, h1Digest, goVersion, arch, location))
}
}
return pkgsSlice
}
@@ -1,186 +0,0 @@
package golang
import (
"github.com/anchore/syft/syft/file"
"io"
"testing"
"github.com/anchore/syft/syft/pkg"
"github.com/stretchr/testify/assert"
)
func TestBuildGoPkgInfo(t *testing.T) {
const (
goCompiledVersion = "1.17"
archDetails = "amd64"
)
tests := []struct {
name string
mod string
expected []pkg.Package
}{
{
name: "buildGoPkgInfo parses a blank mod string and returns no packages",
mod: "",
expected: make([]pkg.Package, 0),
},
{
name: "buildGoPkgInfo parses a populated mod string and returns packages but no source info",
mod: `path github.com/anchore/syft mod github.com/anchore/syft (devel)
dep github.com/adrg/xdg v0.2.1 h1:VSVdnH7cQ7V+B33qSJHTCRlNgra1607Q8PzEmnvb2Ic=
dep github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=`,
expected: []pkg.Package{
{
Name: "github.com/adrg/xdg",
Version: "v0.2.1",
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: []file.Location{
{
Coordinates: file.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
},
},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:VSVdnH7cQ7V+B33qSJHTCRlNgra1607Q8PzEmnvb2Ic=",
},
},
{
Name: "github.com/anchore/client-go",
Version: "v0.0.0-20210222170800-9c70f9b80bcf",
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: []file.Location{
{
Coordinates: file.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
},
},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=",
},
},
},
},
{
name: "buildGoPkgInfo parses a populated mod string and returns packages when a replace directive exists",
mod: `path github.com/anchore/test
mod github.com/anchore/test (devel)
dep golang.org/x/net v0.0.0-20211006190231-62292e806868 h1:KlOXYy8wQWTUJYFgkUI40Lzr06ofg5IRXUK5C7qZt1k=
dep golang.org/x/sys v0.0.0-20211006194710-c8a6f5223071 h1:PjhxBct4MZii8FFR8+oeS7QOvxKOTZXgk63EU2XpfJE=
dep golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
=> golang.org/x/term v0.0.0-20210916214954-140adaaadfaf h1:Ihq/mm/suC88gF8WFcVwk+OV6Tq+wyA1O0E5UEvDglI=`,
expected: []pkg.Package{
{
Name: "golang.org/x/net",
Version: "v0.0.0-20211006190231-62292e806868",
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: []file.Location{
{
Coordinates: file.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
},
},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:KlOXYy8wQWTUJYFgkUI40Lzr06ofg5IRXUK5C7qZt1k=",
},
},
{
Name: "golang.org/x/sys",
Version: "v0.0.0-20211006194710-c8a6f5223071",
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: []file.Location{
{
Coordinates: file.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
},
},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:PjhxBct4MZii8FFR8+oeS7QOvxKOTZXgk63EU2XpfJE=",
},
},
{
Name: "golang.org/x/term",
Version: "v0.0.0-20210916214954-140adaaadfaf",
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: []file.Location{
{
Coordinates: file.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
},
},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:Ihq/mm/suC88gF8WFcVwk+OV6Tq+wyA1O0E5UEvDglI=",
},
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
for i := range test.expected {
p := &test.expected[i]
p.SetID()
}
location := file.Location{
Coordinates: file.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
}
pkgs := buildGoPkgInfo(location, test.mod, goCompiledVersion, archDetails)
assert.Equal(t, test.expected, pkgs)
})
}
}
func Test_parseGoBin_recoversFromPanic(t *testing.T) {
freakOut := func(file io.ReadCloser) ([]exe, error) {
panic("baaahhh!")
}
tests := []struct {
name string
wantPkgs []pkg.Package
wantErr assert.ErrorAssertionFunc
}{
{
name: "recovers from panic",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
pkgs, err := parseGoBin(file.NewLocation("some/path"), nil, freakOut)
assert.Error(t, err)
assert.Nil(t, pkgs)
})
}
}

Some files were not shown because too many files have changed in this diff Show More