Compare commits

...
18 Commits
Author SHA1 Message Date
Keith Zantow ac94bf530c fix: update graalvm cataloger to fix panic (#1454)
Fixes https://github.com/anchore/syft/issues/1453
2023-01-12 17:42:13 -05:00
Weston Steimel e87cfe7319 chore: remove bumping cosign in go.mod when updating bootstrap tools (#1452) 2023-01-12 16:21:01 -05:00
Asi GreenholtsandAlex Goodman 260cb4c72d feat: Add the origin field to the output format of syftjson (#1327)
* moved the relevant fields to the Metadata field

Signed-off-by: Asaf Greenholts <asaf@cidersecurity.io>

* added metadata types

Signed-off-by: Asaf Greenholts <asaf@cidersecurity.io>

* Added hashes to metadata of packge-lock.json and Pipfile.lock

Signed-off-by: Asaf Greenholts <asaf@cidersecurity.io>

* move package metadata types to "pkg" package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* re-generate json schema to include new npm, python, and binary metadatas

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Asaf Greenholts <asaf@cidersecurity.io>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-12 15:03:05 -05:00
Keith Zantow 85bddaa43d chore: update schema (#1449) 2023-01-12 14:25:47 -05:00
Arnout EngelenandAlex Goodman a864dc9505 feat: prefer known CPE vendors over other candidates (#1294)
* feat: prefer known CPE vendors over other candidates

All ASF projects will be under the `apache` vendor in CPE, and
indeed this is already one of the candidates, but the logic
for selecting the 'most specific' CPE string would select for
example `apache_software_foundation` or `commons-text`.

This is not necessarily 'wrong' in the CPE candidate selection
logic: there is no way to reliably determine the right candidate.
I think it makes sense to use specific data around the vendor
candidate generation, somewhat similar to
'defaultCandidateAdditions'.

Unfortunately there are still a few CVE's for old (pre-5.x,
long unsupported) tomcat versions that are actually tagged with
`apache_software_foundation`, but I'm not sure those are worth
spending time on.

Signed-off-by: Arnout Engelen <arnout@bzzt.net>

* chore: swap out array of vendors for set data structure

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Arnout Engelen <arnout@bzzt.net>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-12 19:16:53 +00:00
Christopher Angelo PhillipsandAlex Goodman 44e8ae2577 fix: update attestation code to remove library dependencies and shellout for keyless flow (#1442)
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-12 17:22:05 +00:00
Chapman PenderyandAlex Goodman ac8f72fdd1 feat: add BeamVM Hex support (#1073)
* feat: initial commit providing mix support

Signed-off-by: cpendery <cpendery@vt.edu>

* feat: add rebar parser

Signed-off-by: cpendery <cpendery@vt.edu>

* fix: add beam/hex everywhere else required for Syft runtime

Signed-off-by: cpendery <cpendery@vt.edu>

* style: fix lints

Signed-off-by: cpendery <cpendery@vt.edu>

* ci: fix failing tests

Signed-off-by: cpendery <cpendery@vt.edu>

* docs: update with new supported languages

Signed-off-by: cpendery <cpendery@vt.edu>

* chore: update elixir/erlang catalogers to generic cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: cpendery <cpendery@vt.edu>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-12 12:10:46 -05:00
witchcraze e063471c66 feat: add apache httpd binary classifier (#1448)
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-01-12 10:50:01 -05:00
Batuhan Apaydın 645debe7a4 chore: claim artifacthub package ownership from developer-guy (#881)
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>

Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2023-01-11 15:25:42 -05:00
mikclandAlex Goodman 4bfb849310 Parallel package catalog processing (#1355)
* catalog: run cataloggers concurrently

Signed-off-by: mikcl <mikesmikes400@gmail.com>

* frontend: expose workers as a configurable option

Signed-off-by: mikcl <mikesmikes400@gmail.com>

* fixup! frontend: expose workers as a configurable option

Signed-off-by: mikcl <mikesmikes400@gmail.com>

* update logging statements

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* test: assert for debug logging

Signed-off-by: mikcl <mikesmikes400@gmail.com>

Signed-off-by: mikcl <mikesmikes400@gmail.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-11 15:18:02 -05:00
witchcraze d524bd5fc3 feat: Add php binary catalogers (#1444)
* add php classifier
Signed-off-by: witchcraze <witchcraze@gmail.com>

* make lint-fix
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-01-11 13:46:20 -05:00
anchore-actions-token-generator[bot]andspiffcs a8416d674b Update syft bootstrap tools to latest versions. (#1443)
Co-authored-by: spiffcs <spiffcs@users.noreply.github.com>
2023-01-11 12:50:40 -05:00
Keith Zantow 725529f43f fix: duplicate file in tar archive causes read to fail (#1445) 2023-01-10 14:55:02 -05:00
William Blair e480443c8c Add support for GraalVM Native Image executables. (#1276)
Signed-off-by: William Blair <william.blair@oracle.com>
2023-01-06 18:31:22 -05:00
Benji Visser db386baf81 Add redis binary classifier (#1438)
Signed-off-by: Benji Visser <benji@093b.org>
2023-01-06 12:50:48 -05:00
Christopher Angelo Phillips 795a63f1c9 docs: add cataloger construction summary (#1434) 2023-01-05 17:03:00 +00:00
anchore-actions-token-generator[bot] d4f9993b8d chore: update bootstrap tools to latest versions. (#1428) 2023-01-05 10:20:58 -05:00
Benji Visser bb6fc6525c Add alpine type to purl (#1431)
Signed-off-by: Benji Visser <benji@093b.org>
2023-01-04 17:35:46 -05:00
107 changed files with 4598 additions and 2602 deletions
+1 -5
View File
@@ -39,10 +39,6 @@ jobs:
sed -r -i -e 's/^(YAJSV_VERSION = ).*/\1'${YAJSV_LATEST_VERSION}'/' Makefile
sed -r -i -e 's/^(COSIGN_VERSION = ).*/\1'${COSIGN_LATEST_VERSION}'/' Makefile
# update cosign in go.mod as well
go get github.com/sigstore/cosign@$COSIGN_LATEST_VERSION
go mod tidy
# export the versions for use with create-pull-request
echo "::set-output name=GOLANGCILINT::$GOLANGCILINT_LATEST_VERSION"
echo "::set-output name=BOUNCER::$BOUNCER_LATEST_VERSION"
@@ -76,4 +72,4 @@ jobs:
- [yajsv ${{ steps.latest-versions.outputs.YAJSV }}](https://github.com/neilpa/yajsv/releases/tag/${{ steps.latest-versions.outputs.YAJSV }})
- [cosign ${{ steps.latest-versions.outputs.COSIGN }}](https://github.com/sigstore/cosign/releases/tag/${{ steps.latest-versions.outputs.COSIGN }})
This is an auto-generated pull request to update all of the bootstrap tools to the latest versions.
token: ${{ steps.generate-token.outputs.token }}
token: ${{ steps.generate-token.outputs.token }}
+7
View File
@@ -0,0 +1,7 @@
# Syft Contributors
The following Syft components were contributed by external authors/organizations.
## GraalVM Native Image
A cataloger contributed by Oracle Corporation that extracts packages given within GraalVM Native Image SBOMs.
+42
View File
@@ -132,6 +132,48 @@ sequenceDiagram
```
### Syft Catalogers
##### Summary
Catalogers are the way in which syft is able to identify and construct packages given some amount of source metadata.
For example, Syft can locate and process `package-lock.json` files when performing filesystem scans.
See: [how to specify file globs](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/javascript/cataloger.go#L16-L21)
and an implementation of the [package-lock.json parser](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/javascript/cataloger.go#L16-L21) fora quick review.
#### Building a new Cataloger
Catalogers must fulfill the interface [found here](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger.go).
This means that when building a new cataloger, the new struct must implement both method signatures of `Catalog` and `Name`.
A top level view of the functions that construct all the catalogers can be found [here](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/cataloger.go).
When an author has finished writing a new cataloger this is the spot to plug in the new catalog constructor.
For a top level view of how the catalogers are used see [this function](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/catalog.go#L41-L100) as a reference. It ranges over all catalogers passed as an argument and invokes the `Catalog` method:
Each cataloger has its own `Catalog` method, but this does not mean that they are all vastly different.
Take a look at the `apkdb` cataloger for alpine to see how it [constructs a generic.NewCataloger](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/apkdb/cataloger.go).
`generic.NewCataloger` is an abstraction syft uses to make writing common components easier. First, it takes the `catalogerName` to identify the cataloger.
On the other side of the call it uses two key pieces which inform the cataloger how to identify and return packages, the `globPatterns` and the `parseFunction`:
- The first piece is a `parseByGlob` matching pattern used to identify the files that contain the package metadata.
See [here for the APK example](https://github.com/anchore/syft/blob/main/syft/pkg/apk_metadata.go#L16-L41).
- The other is a `parseFunction` which informs the cataloger what to do when it has found one of the above matches files.
See this [link for an example](https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/apkdb/parse_apk_db.go#L22-L102).
If you're unsure about using the `Generic Cataloger` and think the use case being filled requires something more custom
just file an issue or ask in our slack, and we'd be more than happy to help on the design.
Identified packages share a common struct so be sure that when the new cataloger is constructing a new package it is using the [`Package` struct](https://github.com/anchore/syft/blob/main/syft/pkg/package.go#L16-L31).
Metadata Note: Identified packages are also assigned specific metadata that can be unique to their environment.
See [this folder](https://github.com/anchore/syft/tree/main/syft/pkg) for examples of the different metadata types.
These are plugged into the `MetadataType` and `Metadata` fields in the above struct. `MetadataType` informs which type is being used. `Metadata` is an interface converted to that type.
Finally, here is an example of where the package construction is done in the apk cataloger. The first link is where `newPackage` is called in the `parseFunction`. The second link shows the package construction:
- [Call for new package](https://github.com/anchore/syft/blob/6a7d6e6071829c7ce2943266c0e187b27c0b325c/syft/pkg/cataloger/apkdb/parse_apk_db.go#L96-L99)
- [APK Package Constructor](https://github.com/anchore/syft/blob/6a7d6e6071829c7ce2943266c0e187b27c0b325c/syft/pkg/cataloger/apkdb/package.go#L12-L27)
If you have more questions about implementing a cataloger or questions about one you might be currently working
always feel free to file an issue or reach out to us [on slack](https://anchore.com/slack).
## Testing
+2 -2
View File
@@ -10,10 +10,10 @@ SNAPSHOT_CMD=$(RELEASE_CMD) --skip-publish --snapshot
# tool versions
GOLANGCILINT_VERSION = v1.50.1
GOSIMPORTS_VERSION = v0.3.4
GOSIMPORTS_VERSION = v0.3.5
BOUNCER_VERSION = v0.4.0
CHRONICLE_VERSION = v0.4.2
GORELEASER_VERSION = v1.13.1
GORELEASER_VERSION = v1.14.1
YAJSV_VERSION = v1.4.1
COSIGN_VERSION = v1.13.1
QUILL_VERSION = v0.2.0
+8 -1
View File
@@ -38,9 +38,11 @@ For commercial support options with Syft or Grype, please [contact Anchore](http
- Debian (dpkg)
- Dotnet (deps.json)
- Objective-C (cocoapods)
- Elixir (mix)
- Erlang (rebar3)
- Go (go.mod, Go binaries)
- Haskell (cabal, stack)
- Java (jar, ear, war, par, sar)
- Java (jar, ear, war, par, sar, native-image)
- JavaScript (npm, yarn)
- Jenkins Plugins (jpi, hpi)
- PHP (composer)
@@ -365,6 +367,8 @@ syft convert sbom.syft.json -o cyclonedx-json=sbom.cdx.json # convert it to Cyc
### Keyless support
Syft supports generating attestations using cosign's [keyless](https://github.com/sigstore/cosign/blob/main/KEYLESS.md) signatures.
Note: users need to have >= v1.12.0 of cosign installed for this command to function
To use this feature with a format like CycloneDX json simply run:
```
syft attest --output cyclonedx-json <IMAGE WITH OCI WRITE ACCESS>
@@ -522,6 +526,9 @@ file-metadata:
# SYFT_FILE_METADATA_DIGESTS env var
digests: ["sha256"]
# maximum number of workers used to process the list of package catalogers in parallel
parallelism: 1
# cataloging secrets is exposed through the power-user subcommand
secrets:
cataloger:
+1 -1
View File
@@ -1,5 +1,5 @@
# See documentation here: https://github.com/artifacthub/hub/blob/v1.6.0/docs/metadata/artifacthub-repo.yml
repositoryID: eced152f-b15d-4879-8b3b-1175397192ba
owners:
- name: wagoodman
email: wagoodman@gmail.com
+12 -32
View File
@@ -4,7 +4,6 @@ import (
"fmt"
"log"
sigopts "github.com/sigstore/cosign/cmd/cosign/cli/options"
"github.com/spf13/cobra"
"github.com/spf13/viper"
@@ -15,33 +14,27 @@ import (
)
const (
attestExample = ` {{.appName}} {{.command}} --output [FORMAT] --key [KEY] alpine:latest
Supports the following image sources:
{{.appName}} {{.command}} --key [KEY] yourrepo/yourimage:tag defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry.
{{.appName}} {{.command}} --key [KEY] path/to/a/file/or/dir only for OCI tar or OCI directory
attestExample = ` {{.appName}} {{.command}} --output [FORMAT] alpine:latest defaults to using images from a Docker daemon. If Docker is not present, the image is pulled directly from the registry
`
attestSchemeHelp = "\n" + indent + schemeHelpHeader + "\n" + imageSchemeHelp
attestHelp = attestExample + attestSchemeHelp
attestHelp = attestExample + attestSchemeHelp
)
func Attest(v *viper.Viper, app *config.Application, ro *options.RootOptions) *cobra.Command {
ao := options.AttestOptions{}
//nolint:dupl
func Attest(v *viper.Viper, app *config.Application, ro *options.RootOptions, po *options.PackagesOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "attest --output [FORMAT] --key [KEY] [SOURCE]",
Short: "Generate a package SBOM as an attestation for the given [SOURCE] container image",
Long: "Generate a packaged-based Software Bill Of Materials (SBOM) from a container image as the predicate of an in-toto attestation",
Use: "attest --output [FORMAT] <IMAGE>",
Short: "Generate an SBOM as an attestation for the given [SOURCE] container image",
Long: "Generate a packaged-based Software Bill Of Materials (SBOM) from a container image as the predicate of an in-toto attestation that will be uploaded to the image registry",
Example: internal.Tprintf(attestHelp, map[string]interface{}{
"appName": internal.ApplicationName,
"command": "attest",
}),
Args: func(cmd *cobra.Command, args []string) error {
// run to unmarshal viper object onto app config
// the viper object correctly
if err := app.LoadAllValues(v, ro.Config); err != nil {
return fmt.Errorf("invalid application config: %w", err)
return fmt.Errorf("unable to load configuration: %w", err)
}
// configure logging for command
newLogWrapper(app)
logApplicationConfig(app)
return validateArgs(cmd, args)
@@ -49,29 +42,16 @@ func Attest(v *viper.Viper, app *config.Application, ro *options.RootOptions) *c
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
// this MUST be called first to make sure app config decodes
// the viper object correctly
if app.CheckForAppUpdate {
checkForApplicationUpdate()
}
// build cosign key options for attestation
ko := sigopts.KeyOpts{
KeyRef: app.Attest.KeyRef,
FulcioURL: app.Attest.FulcioURL,
IDToken: app.Attest.FulcioIdentityToken,
InsecureSkipFulcioVerify: app.Attest.InsecureSkipFulcioVerify,
RekorURL: app.Attest.RekorURL,
OIDCIssuer: app.Attest.OIDCIssuer,
OIDCClientID: app.Attest.OIDCClientID,
OIDCRedirectURL: app.Attest.OIDCRedirectURL,
}
return attest.Run(cmd.Context(), app, ko, args)
return attest.Run(cmd.Context(), app, args)
},
}
err := ao.AddFlags(cmd, v)
// syft attest is an enhancment of the packages command, so it should have the same flags
err := po.AddFlags(cmd, v)
if err != nil {
log.Fatal(err)
}
+134 -305
View File
@@ -1,105 +1,60 @@
package attest
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os"
"os/exec"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/in-toto/in-toto-golang/in_toto"
sigopts "github.com/sigstore/cosign/cmd/cosign/cli/options"
"github.com/sigstore/cosign/cmd/cosign/cli/rekor"
"github.com/sigstore/cosign/cmd/cosign/cli/sign"
"github.com/sigstore/cosign/pkg/cosign"
"github.com/sigstore/cosign/pkg/cosign/attestation"
cbundle "github.com/sigstore/cosign/pkg/cosign/bundle"
"github.com/sigstore/cosign/pkg/oci/mutate"
ociremote "github.com/sigstore/cosign/pkg/oci/remote"
"github.com/sigstore/cosign/pkg/oci/static"
sigs "github.com/sigstore/cosign/pkg/signature"
"github.com/sigstore/cosign/pkg/types"
"github.com/sigstore/rekor/pkg/generated/client"
"github.com/sigstore/rekor/pkg/generated/models"
"github.com/sigstore/sigstore/pkg/signature/dsse"
signatureoptions "github.com/sigstore/sigstore/pkg/signature/options"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
"golang.org/x/exp/slices"
"github.com/anchore/stereoscope"
"github.com/anchore/stereoscope/pkg/image"
"github.com/anchore/syft/cmd/syft/cli/eventloop"
"github.com/anchore/syft/cmd/syft/cli/options"
"github.com/anchore/syft/cmd/syft/cli/packages"
"github.com/anchore/syft/internal/bus"
"github.com/anchore/syft/internal/config"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/internal/ui"
"github.com/anchore/syft/syft"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/formats/cyclonedxjson"
"github.com/anchore/syft/syft/formats/spdxjson"
"github.com/anchore/syft/syft/event/monitor"
"github.com/anchore/syft/syft/formats/syftjson"
"github.com/anchore/syft/syft/formats/table"
"github.com/anchore/syft/syft/formats/template"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
var (
allowedAttestFormats = []sbom.FormatID{
syftjson.ID,
spdxjson.ID,
cyclonedxjson.ID,
}
intotoJSONDsseType = `application/vnd.in-toto+json`
)
func Run(ctx context.Context, app *config.Application, ko sigopts.KeyOpts, args []string) error {
// We cannot generate an attestation for more than one output
if len(app.Outputs) > 1 {
return fmt.Errorf("unable to generate attestation for more than one output")
}
// can only be an image for attestation or OCI DIR
userInput := args[0]
si, err := parseImageSource(userInput, app)
func Run(ctx context.Context, app *config.Application, args []string) error {
err := ValidateOutputOptions(app)
if err != nil {
return err
}
output := parseAttestationOutput(app.Outputs)
format := syft.FormatByName(output)
// user typo or unknown outputs provided
if format == nil {
format = syft.FormatByID(syftjson.ID) // default attestation format
}
predicateType := formatPredicateType(format)
if predicateType == "" {
return fmt.Errorf(
"could not produce attestation predicate for given format: %q. Available formats: %+v",
options.FormatAliases(format.ID()),
options.FormatAliases(allowedAttestFormats...),
)
writer, err := options.MakeWriter(app.Outputs, app.File, app.OutputTemplatePath)
if err != nil {
return fmt.Errorf("unable to write to report destination: %w", err)
}
if app.Attest.KeyRef != "" {
passFunc, err := selectPassFunc(app.Attest.KeyRef, app.Attest.Password)
if err != nil {
return err
defer func() {
if err := writer.Close(); err != nil {
fmt.Printf("unable to close report destination: %+v", err)
}
}()
ko.PassFunc = passFunc
}
sv, err := sign.SignerFromKeyOpts(ctx, "", "", ko)
// could be an image or a directory, with or without a scheme
// TODO: validate that source is image
userInput := args[0]
si, err := source.ParseInputWithName(userInput, app.Platform, true, app.Name)
if err != nil {
return err
return fmt.Errorf("could not generate source input for packages command: %w", err)
}
if si.Scheme != source.ImageScheme {
return fmt.Errorf("attestations are only supported for oci images at this time")
}
defer sv.Close()
eventBus := partybus.NewBus()
stereoscope.SetBus(eventBus)
@@ -107,7 +62,7 @@ func Run(ctx context.Context, app *config.Application, ko sigopts.KeyOpts, args
subscription := eventBus.Subscribe()
return eventloop.EventLoop(
execWorker(app, *si, format, predicateType, sv),
execWorker(app, *si, writer),
eventloop.SetupSignals(),
subscription,
stereoscope.Cleanup,
@@ -115,284 +70,158 @@ func Run(ctx context.Context, app *config.Application, ko sigopts.KeyOpts, args
)
}
func parseAttestationOutput(outputs []string) (format string) {
if len(outputs) == 0 {
outputs = append(outputs, string(syftjson.ID))
func buildSBOM(app *config.Application, si source.Input, writer sbom.Writer, errs chan error) ([]byte, error) {
src, cleanup, err := source.New(si, app.Registry.ToOptions(), app.Exclusions)
if cleanup != nil {
defer cleanup()
}
return outputs[0]
}
func parseImageSource(userInput string, app *config.Application) (s *source.Input, err error) {
si, err := source.ParseInputWithName(userInput, app.Platform, false, app.Name)
if err != nil {
return nil, fmt.Errorf("could not generate source input for attest command: %w", err)
return nil, fmt.Errorf("failed to construct source from user input %q: %w", si.UserInput, err)
}
switch si.Scheme {
case source.ImageScheme, source.UnknownScheme:
// at this point we know that it cannot be dir: or file: schemes;
// we will assume that the unknown scheme could represent an image;
si.Scheme = source.ImageScheme
default:
return nil, fmt.Errorf("attest command can only be used with image sources but discovered %q when given %q", si.Scheme, userInput)
s, err := packages.GenerateSBOM(src, errs, app)
if err != nil {
return nil, err
}
// if the original detection was from the local daemon we want to short circuit
// that and attempt to generate the image source from its current registry source instead
switch si.ImageSource {
case image.UnknownSource, image.OciRegistrySource:
si.ImageSource = image.OciRegistrySource
case image.SingularitySource:
default:
return nil, fmt.Errorf("attest command can only be used with image sources fetch directly from the registry, but discovered an image source of %q when given %q", si.ImageSource, userInput)
if s == nil {
return nil, fmt.Errorf("no SBOM produced for %q", si.UserInput)
}
return si, nil
// note: only works for single format no multi writer support
sBytes, err := writer.Bytes(*s)
if err != nil {
return nil, fmt.Errorf("unable to build SBOM bytes: %w", err)
}
return sBytes, nil
}
func execWorker(app *config.Application, sourceInput source.Input, format sbom.Format, predicateType string, sv *sign.SignerVerifier) <-chan error {
func execWorker(app *config.Application, si source.Input, writer sbom.Writer) <-chan error {
errs := make(chan error)
go func() {
defer close(errs)
src, cleanup, err := source.NewFromRegistry(sourceInput, app.Registry.ToOptions(), app.Exclusions)
if cleanup != nil {
defer cleanup()
}
sBytes, err := buildSBOM(app, si, writer, errs)
if err != nil {
errs <- fmt.Errorf("failed to construct source from user input %q: %w", sourceInput.UserInput, err)
errs <- fmt.Errorf("unable to build SBOM: %w", err)
return
}
s, err := packages.GenerateSBOM(src, errs, app)
if err != nil {
errs <- err
return
}
// TODO: add multi writer support
for _, o := range app.Outputs {
f, err := os.CreateTemp("", o)
if err != nil {
errs <- fmt.Errorf("unable to create temp file: %w", err)
return
}
sbomBytes, err := syft.Encode(*s, format)
if err != nil {
errs <- err
return
}
defer f.Close()
defer os.Remove(f.Name())
signedPayload, err := generateAttestation(sourceInput, sbomBytes, src, sv, predicateType)
if err != nil {
errs <- err
return
}
if _, err := f.Write(sBytes); err != nil {
errs <- fmt.Errorf("unable to write SBOM to temp file: %w", err)
return
}
err = publishAttestation(app, signedPayload, src, sv)
if err != nil {
errs <- err
return
// TODO: what other validation here besides binary name?
cmd := "cosign"
if !commandExists(cmd) {
errs <- fmt.Errorf("unable to find cosign in PATH; make sure you have it installed")
return
}
args := []string{"attest", si.UserInput, "--type", "custom", "--predicate", f.Name()}
execCmd := exec.Command(cmd, args...)
execCmd.Env = os.Environ()
execCmd.Env = append(execCmd.Env, "COSIGN_EXPERIMENTAL=1")
// bus adapter for ui to hook into stdout via an os pipe
r, w, err := os.Pipe()
if err != nil {
errs <- fmt.Errorf("unable to create os pipe: %w", err)
return
}
defer w.Close()
b := &busWriter{r: r, w: w, mon: &progress.Manual{N: -1}}
execCmd.Stdout = b
execCmd.Stderr = b
defer b.mon.SetCompleted()
// attest the SBOM
err = execCmd.Run()
if err != nil {
b.mon.Err = err
errs <- fmt.Errorf("unable to attest SBOM: %w", err)
return
}
}
bus.Publish(partybus.Event{
Type: event.Exit,
Value: func() error {
return nil
},
Type: event.Exit,
Value: func() error { return nil },
})
}()
return errs
}
func generateAttestation(si source.Input, predicate []byte, src *source.Source, sv *sign.SignerVerifier, predicateType string) ([]byte, error) {
var h v1.Hash
switch si.ImageSource {
case image.OciRegistrySource:
switch len(src.Image.Metadata.RepoDigests) {
case 0:
return nil, fmt.Errorf("cannot generate attestation since no repo digests were found; make sure you're passing an OCI registry source for the attest command")
case 1:
d, err := name.NewDigest(src.Image.Metadata.RepoDigests[0])
if err != nil {
return nil, err
}
h, err = v1.NewHash(d.Identifier())
if err != nil {
return nil, err
}
default:
return nil, fmt.Errorf("cannot generate attestation since multiple repo digests were found for the image: %+v", src.Image.Metadata.RepoDigests)
}
case image.SingularitySource:
var err error
h, err = v1.NewHash(src.Image.Metadata.ID)
if err != nil {
return nil, err
func ValidateOutputOptions(app *config.Application) error {
var usesTemplateOutput bool
for _, o := range app.Outputs {
if o == template.ID.String() {
usesTemplateOutput = true
break
}
}
sh, err := attestation.GenerateStatement(attestation.GenerateOpts{
Predicate: bytes.NewBuffer(predicate),
Type: predicateType,
Digest: h.Hex,
})
if err != nil {
return nil, err
if usesTemplateOutput && app.OutputTemplatePath == "" {
return fmt.Errorf(`must specify path to template file when using "template" output format`)
}
payload, err := json.Marshal(sh)
if err != nil {
return nil, err
if len(app.Outputs) > 1 {
return fmt.Errorf("multiple SBOM format is not supported for attest at this time")
}
wrapped := dsse.WrapSigner(sv, intotoJSONDsseType)
return wrapped.SignMessage(bytes.NewReader(payload), signatureoptions.WithContext(context.Background()))
}
// publishAttestation publishes signedPayload to the location specified by the user.
func publishAttestation(app *config.Application, signedPayload []byte, src *source.Source, sv *sign.SignerVerifier) error {
switch {
// We want to give the option to not upload the generated attestation
// if passed or if the user is using local PKI
case app.Attest.NoUpload || app.Attest.KeyRef != "":
if app.File != "" {
return os.WriteFile(app.File, signedPayload, 0600)
}
_, err := os.Stdout.Write(signedPayload)
return err
default:
ref, err := name.ParseReference(src.Metadata.ImageMetadata.UserInput)
if err != nil {
return err
}
digest, err := ociremote.ResolveDigest(ref)
if err != nil {
return err
}
return uploadAttestation(app, signedPayload, digest, sv)
// cannot use table as default output format when using template output
if slices.Contains(app.Outputs, table.ID.String()) {
app.Outputs = []string{syftjson.ID.String()}
}
}
func trackUploadAttestation() (*progress.Stage, *progress.Manual) {
stage := &progress.Stage{}
prog := &progress.Manual{}
bus.Publish(partybus.Event{
Type: event.UploadAttestation,
Value: progress.StagedProgressable(&struct {
progress.Stager
progress.Progressable
}{
Stager: stage,
Progressable: prog,
}),
})
return stage, prog
}
// uploads signed SBOM payload to Rekor transparency log along with key information;
// returns a bundle for attestation annotations
// rekor bundle includes a signed payload and rekor timestamp;
// the bundle is then wrapped onto an OCI signed entity and uploaded to
// the user's image's OCI registry repository as *.att
func uploadAttestation(app *config.Application, signedPayload []byte, digest name.Digest, sv *sign.SignerVerifier) error {
// add application/vnd.dsse.envelope.v1+json as media type for other applications to decode attestation
opts := []static.Option{static.WithLayerMediaType(types.DssePayloadType)}
if sv.Cert != nil {
opts = append(opts, static.WithCertChain(sv.Cert, sv.Chain))
}
stage, prog := trackUploadAttestation()
defer prog.SetCompleted() // just in case we return early
prog.Total = 2
stage.Current = "uploading signing information to transparency log"
// uploads payload to Rekor transparency log along with key information;
// returns bundle for attesation annotations
// rekor bundle includes a signed payload and rekor timestamp;
// the bundle is then wrapped onto an OCI signed entity and uploaded to
// the user's image's OCI registry repository as *.att
bundle, err := uploadToTlog(context.TODO(), sv, app.Attest.RekorURL, func(r *client.Rekor, b []byte) (*models.LogEntryAnon, error) {
return cosign.TLogUploadInTotoAttestation(context.TODO(), r, signedPayload, b)
})
if err != nil {
return err
}
prog.N = 1
stage.Current = "uploading attestation to OCI registry"
// add bundle OCI attestation that is uploaded to
opts = append(opts, static.WithBundle(bundle))
sig, err := static.NewAttestation(signedPayload, opts...)
if err != nil {
return err
}
se, err := ociremote.SignedEntity(digest)
if err != nil {
return err
}
newSE, err := mutate.AttachAttestationToEntity(se, sig)
if err != nil {
return err
}
// Publish the attestations associated with this entity
err = ociremote.WriteAttestations(digest.Repository, newSE)
if err != nil {
return err
}
prog.SetCompleted()
return nil
}
func formatPredicateType(format sbom.Format) string {
switch format.ID() {
case spdxjson.ID:
return in_toto.PredicateSPDX
case cyclonedxjson.ID:
return in_toto.PredicateCycloneDX
case syftjson.ID:
return "https://syft.dev/bom"
default:
return ""
}
type busWriter struct {
w *os.File
r *os.File
hasWritten bool
mon *progress.Manual
}
type tlogUploadFn func(*client.Rekor, []byte) (*models.LogEntryAnon, error)
func uploadToTlog(ctx context.Context, sv *sign.SignerVerifier, rekorURL string, upload tlogUploadFn) (*cbundle.RekorBundle, error) {
var rekorBytes []byte
// Upload the cert or the public key, depending on what we have
if sv.Cert != nil {
rekorBytes = sv.Cert
} else {
pemBytes, err := sigs.PublicKeyPem(sv, signatureoptions.WithContext(ctx))
if err != nil {
return nil, err
}
rekorBytes = pemBytes
func (b *busWriter) Write(p []byte) (n int, err error) {
if !b.hasWritten {
b.hasWritten = true
bus.Publish(
partybus.Event{
Type: event.AttestationStarted,
Source: monitor.GenericTask{
Title: monitor.Title{
Default: "Create attestation",
WhileRunning: "Creating attestation",
OnSuccess: "Created attestation",
},
Context: "cosign",
},
Value: &monitor.ShellProgress{
Reader: b.r,
Manual: b.mon,
},
},
)
}
return b.w.Write(p)
}
rekorClient, err := rekor.NewClient(rekorURL)
if err != nil {
return nil, err
}
entry, err := upload(rekorClient, rekorBytes)
if err != nil {
return nil, err
}
if entry.LogIndex != nil {
log.Debugf("transparency log entry created with index: %v", *entry.LogIndex)
}
return cbundle.EntryToBundle(entry), nil
func commandExists(cmd string) bool {
_, err := exec.LookPath(cmd)
return err == nil
}
-57
View File
@@ -1,57 +0,0 @@
package attest
import (
"errors"
"fmt"
"io"
"os"
"strings"
"github.com/sigstore/cosign/pkg/cosign"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
)
func selectPassFunc(keypath, password string) (cosign.PassFunc, error) {
keyContents, err := os.ReadFile(keypath)
if err != nil {
return nil, err
}
var fn cosign.PassFunc = func(bool) (b []byte, err error) { return nil, nil }
_, err = cosign.LoadPrivateKey(keyContents, nil)
if err != nil {
fn = func(bool) (b []byte, err error) {
return fetchPassword(password)
}
}
return fn, nil
}
func fetchPassword(password string) (b []byte, err error) {
potentiallyPipedInput, err := internal.IsPipedInput()
if err != nil {
log.Warnf("unable to determine if there is piped input: %+v", err)
}
switch {
case password != "":
return []byte(password), nil
case potentiallyPipedInput:
// handle piped in passwords
pwBytes, err := io.ReadAll(os.Stdin)
if err != nil {
return nil, fmt.Errorf("unable to get password from stdin: %w", err)
}
// be resilient to input that may have newline characters (in case someone is using echo without -n)
cleanPw := strings.TrimRight(string(pwBytes), "\n")
return []byte(cleanPw), nil
case internal.IsTerminal():
return cosign.GetPassFromTerm(false)
}
return nil, errors.New("no method available to fetch password")
}
+8 -12
View File
@@ -26,8 +26,10 @@ import (
const indent = " "
// New constructs the `syft packages` command, aliases the root command to `syft packages`,
// and constructs the `syft power-user` and `syft attest` commands. It is also responsible for
// and constructs the `syft power-user` command. It is also responsible for
// organizing flag usage and injecting the application config for each command.
// It also constructs the syft attest command and the syft version command.
// Because of how the `cobra` library behaves, the application's configuration is initialized
// at this level. Values from the config should only be used after `app.LoadAllValues` has been called.
// Cobra does not have knowledge of the user provided flags until the `RunE` block of each command.
@@ -46,9 +48,9 @@ func New() (*cobra.Command, error) {
packagesCmd := Packages(v, app, ro, po)
// root options are also passed to the attestCmd so that a user provided config location can be discovered
attestCmd := Attest(v, app, ro)
poweruserCmd := PowerUser(v, app, ro)
convertCmd := Convert(v, app, ro, po)
attestCmd := Attest(v, app, ro, po)
// rootCmd is currently an alias for the packages command
rootCmd := &cobra.Command{
@@ -73,11 +75,7 @@ func New() (*cobra.Command, error) {
if err != nil {
return nil, err
}
// attest also uses flags from the packagesCmd since it generates an sbom
err = po.AddFlags(attestCmd, v)
if err != nil {
return nil, err
}
// poweruser also uses the packagesCmd flags since it is a specialized version of the command
err = po.AddFlags(poweruserCmd, v)
if err != nil {
@@ -87,13 +85,11 @@ func New() (*cobra.Command, error) {
// commands to add to root
cmds := []*cobra.Command{
packagesCmd,
attestCmd,
poweruserCmd,
convertCmd,
poweruserCmd,
poweruserCmd,
Completion(),
attestCmd,
Version(v, app),
cranecmd.NewCmdAuthLogin("syft"),
cranecmd.NewCmdAuthLogin("syft"), // syft login uses the same command as crane
}
// Add sub-commands.
-54
View File
@@ -1,54 +0,0 @@
package cli
import (
"os"
"github.com/spf13/cobra"
)
func Completion() *cobra.Command {
cmd := &cobra.Command{
Use: "completion [bash|zsh|fish]",
Short: "Generate a shell completion for Syft (listing local docker images)",
Long: `To load completions (docker image list):
Bash:
$ source <(syft completion bash)
# To load completions for each session, execute once:
Linux:
$ syft completion bash > /etc/bash_completion.d/syft
MacOS:
$ syft completion bash > /usr/local/etc/bash_completion.d/syft
Zsh:
# If shell completion is not already enabled in your environment you will need
# to enable it. You can execute the following once:
$ echo "autoload -U compinit; compinit" >> ~/.zshrc
# To load completions for each session, execute once:
$ syft completion zsh > "${fpath[1]}/_syft"
# You will need to start a new shell for this setup to take effect.
Fish:
$ syft completion fish | source
# To load completions for each session, execute once:
$ syft completion fish > ~/.config/fish/completions/syft.fish
`,
DisableFlagsInUseLine: true,
ValidArgs: []string{"bash", "zsh", "fish"},
Args: cobra.MatchAll(cobra.ExactArgs(1), cobra.OnlyValidArgs),
RunE: func(cmd *cobra.Command, args []string) error {
var err error
switch args[0] {
case "bash":
err = cmd.Root().GenBashCompletion(os.Stdout)
case "zsh":
err = cmd.Root().GenZshCompletion(os.Stdout)
case "fish":
err = cmd.Root().GenFishCompletion(os.Stdout, true)
}
if err != nil {
return err
}
return nil
},
}
return cmd
}
-77
View File
@@ -1,77 +0,0 @@
package options
import (
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
)
const defaultKeyFileName = "cosign.key"
type AttestOptions struct {
Key string
Cert string
CertChain string
NoUpload bool
Force bool
Recursive bool
Rekor RekorOptions
Fulcio FulcioOptions
OIDC OIDCOptions
}
var _ Interface = (*AttestOptions)(nil)
func (o *AttestOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
if err := o.Rekor.AddFlags(cmd, v); err != nil {
return err
}
if err := o.Fulcio.AddFlags(cmd, v); err != nil {
return err
}
if err := o.OIDC.AddFlags(cmd, v); err != nil {
return err
}
cmd.Flags().StringVarP(&o.Key, "key", "", defaultKeyFileName,
"path to the private key file to use for attestation")
cmd.Flags().StringVarP(&o.Cert, "cert", "", "",
"path to the x.509 certificate in PEM format to include in the OCI Signature")
cmd.Flags().BoolVarP(&o.NoUpload, "no-upload", "", false,
"do not upload the generated attestation")
cmd.Flags().BoolVarP(&o.Force, "force", "", false,
"skip warnings and confirmations")
cmd.Flags().BoolVarP(&o.Recursive, "recursive", "", false,
"if a multi-arch image is specified, additionally sign each discrete image")
return bindAttestationConfigOptions(cmd.Flags(), v)
}
func bindAttestationConfigOptions(flags *pflag.FlagSet, v *viper.Viper) error {
if err := v.BindPFlag("attest.key", flags.Lookup("key")); err != nil {
return err
}
if err := v.BindPFlag("attest.cert", flags.Lookup("cert")); err != nil {
return err
}
if err := v.BindPFlag("attest.no-upload", flags.Lookup("no-upload")); err != nil {
return err
}
if err := v.BindPFlag("attest.force", flags.Lookup("force")); err != nil {
return err
}
if err := v.BindPFlag("attest.recursive", flags.Lookup("recursive")); err != nil {
return err
}
return nil
}
+1 -1
View File
@@ -9,7 +9,7 @@ type VersionOptions struct {
Output string
}
var _ Interface = (*PackagesOptions)(nil)
var _ Interface = (*VersionOptions)(nil)
func (o *VersionOptions) AddFlags(cmd *cobra.Command, v *viper.Viper) error {
cmd.Flags().StringVarP(&o.Output, "output", "o", "text", "format to show version information (available=[text, json])")
+2 -2
View File
@@ -24,7 +24,7 @@ import (
)
func Run(ctx context.Context, app *config.Application, args []string) error {
err := validateOutputOptions(app)
err := ValidateOutputOptions(app)
if err != nil {
return err
}
@@ -134,7 +134,7 @@ func MergeRelationships(cs ...<-chan artifact.Relationship) (relationships []art
return relationships
}
func validateOutputOptions(app *config.Application) error {
func ValidateOutputOptions(app *config.Application) error {
var usesTemplateOutput bool
for _, o := range app.Outputs {
if o == template.ID.String() {
+4 -187
View File
@@ -3,14 +3,13 @@ module github.com/anchore/syft
go 1.18
require (
github.com/CycloneDX/cyclonedx-go v0.7.1-0.20221222100750-41a1ac565cce
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
github.com/acobaugh/osrelease v0.1.0
github.com/adrg/xdg v0.3.3
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.1.1-0.20220428202044-a072fa3cb6d7
github.com/anchore/packageurl-go v0.1.1-0.20230104203445-02e0a6721501
github.com/bmatcuk/doublestar/v4 v4.0.2
github.com/dustin/go-humanize v1.0.0
github.com/facebookincubator/nvdtools v0.1.4
@@ -50,268 +49,90 @@ require (
)
require (
github.com/CycloneDX/cyclonedx-go v0.7.1-0.20221222100750-41a1ac565cce
github.com/Masterminds/sprig/v3 v3.2.2
github.com/anchore/go-logger v0.0.0-20220728155337-03b66a5207d8
github.com/anchore/stereoscope v0.0.0-20221208011002-c5ff155d72f1
github.com/docker/docker v20.10.17+incompatible
github.com/google/go-containerregistry v0.11.0
github.com/in-toto/in-toto-golang v0.4.1-0.20221018183522-731d0640b65f
github.com/invopop/jsonschema v0.7.0
github.com/knqyf263/go-rpmdb v0.0.0-20221030135625-4082a22221ce
github.com/opencontainers/go-digest v1.0.0
github.com/sassoftware/go-rpmutils v0.2.0
github.com/sigstore/cosign v1.13.1
github.com/sigstore/rekor v0.12.1-0.20220915152154-4bb6f441c1b2
github.com/sigstore/sigstore v1.4.4
github.com/vbatts/go-mtree v0.5.0
golang.org/x/exp v0.0.0-20220823124025-807a23277127
gopkg.in/yaml.v3 v3.0.1
)
require (
bitbucket.org/creachadair/shell v0.0.7 // indirect
cloud.google.com/go/compute v1.10.0 // indirect
github.com/AliyunContainerService/ack-ram-tool/pkg/credentials/alibabacloudsdkgo/helper v0.2.0 // indirect
github.com/Azure/azure-sdk-for-go v66.0.0+incompatible // indirect
github.com/Azure/go-autorest v14.2.0+incompatible // indirect
github.com/Azure/go-autorest/autorest v0.11.28 // indirect
github.com/Azure/go-autorest/autorest/adal v0.9.21 // indirect
github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 // indirect
github.com/Azure/go-autorest/autorest/azure/cli v0.4.6 // indirect
github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect
github.com/Azure/go-autorest/logger v0.2.1 // indirect
github.com/Azure/go-autorest/tracing v0.6.0 // indirect
github.com/DataDog/zstd v1.4.5 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.1.1 // indirect
github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/ThalesIgnite/crypto11 v1.2.5 // indirect
github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.4 // indirect
github.com/alibabacloud-go/cr-20160607 v1.0.1 // indirect
github.com/alibabacloud-go/cr-20181201 v1.0.10 // indirect
github.com/alibabacloud-go/darabonba-openapi v0.1.18 // indirect
github.com/alibabacloud-go/debug v0.0.0-20190504072949-9472017b5c68 // indirect
github.com/alibabacloud-go/endpoint-util v1.1.1 // indirect
github.com/alibabacloud-go/openapi-util v0.0.11 // indirect
github.com/alibabacloud-go/tea v1.1.18 // indirect
github.com/alibabacloud-go/tea-utils v1.4.4 // indirect
github.com/alibabacloud-go/tea-xml v1.1.2 // indirect
github.com/aliyun/credentials-go v1.2.3 // indirect
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
github.com/aws/aws-sdk-go-v2 v1.16.16 // indirect
github.com/aws/aws-sdk-go-v2/config v1.17.8 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.12.21 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.17 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.23 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.17 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.24 // indirect
github.com/aws/aws-sdk-go-v2/service/ecr v1.15.0 // indirect
github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.12.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.17 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.11.23 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.13.6 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.16.19 // indirect
github.com/aws/smithy-go v1.13.3 // indirect
github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20220517224237-e6f29200ae04 // indirect
github.com/benbjohnson/clock v1.1.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bgentry/speakeasy v0.1.0 // indirect
github.com/blang/semver v3.5.1+incompatible // indirect
github.com/cenkalti/backoff/v4 v4.1.3 // indirect
github.com/census-instrumentation/opencensus-proto v0.3.0 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/chrismellard/docker-credential-acr-env v0.0.0-20220119192733-fe33c00cee21 // indirect
github.com/clbanning/mxj/v2 v2.5.6 // indirect
github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4 // indirect
github.com/cncf/xds/go v0.0.0-20211130200136-a8f946100490 // indirect
github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be // indirect
github.com/containerd/containerd v1.6.12 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.12.0 // indirect
github.com/coreos/go-oidc/v3 v3.4.0 // indirect
github.com/coreos/go-semver v0.3.0 // indirect
github.com/coreos/go-systemd/v22 v22.3.2 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
github.com/cyberphone/json-canonicalization v0.0.0-20210823021906-dc406ceaf94b // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dimchansky/utfbom v1.1.1 // indirect
github.com/docker/cli v20.10.17+incompatible // indirect
github.com/docker/distribution v2.8.1+incompatible // indirect
github.com/docker/docker-credential-helpers v0.6.4 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.4.0 // indirect
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
github.com/envoyproxy/go-control-plane v0.10.2-0.20220325020618-49ff273808a1 // indirect
github.com/envoyproxy/protoc-gen-validate v0.6.2 // indirect
github.com/fsnotify/fsnotify v1.5.4 // indirect
github.com/fullstorydev/grpcurl v1.8.7 // indirect
github.com/gabriel-vasile/mimetype v1.4.0 // indirect
github.com/go-chi/chi v4.1.2+incompatible // indirect
github.com/go-logr/logr v1.2.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/analysis v0.21.4 // indirect
github.com/go-openapi/errors v0.20.3 // indirect
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/jsonreference v0.20.0 // indirect
github.com/go-openapi/loads v0.21.2 // indirect
github.com/go-openapi/runtime v0.24.2 // indirect
github.com/go-openapi/spec v0.20.7 // indirect
github.com/go-openapi/strfmt v0.21.3 // indirect
github.com/go-openapi/swag v0.22.3 // indirect
github.com/go-openapi/validate v0.22.0 // indirect
github.com/go-piv/piv-go v1.10.0 // indirect
github.com/go-playground/locales v0.14.0 // indirect
github.com/go-playground/universal-translator v0.18.0 // indirect
github.com/go-playground/validator/v10 v10.11.0 // indirect
github.com/go-restruct/restruct v1.2.0-alpha // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt v3.2.2+incompatible // indirect
github.com/golang-jwt/jwt/v4 v4.4.2 // indirect
github.com/golang/glog v1.0.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/mock v1.6.0 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/btree v1.1.2 // indirect
github.com/google/certificate-transparency-go v1.1.3 // indirect
github.com/google/go-github/v45 v45.2.0 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/trillian v1.5.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.0 // indirect
github.com/googleapis/gnostic v0.5.5 // indirect
github.com/gorilla/websocket v1.4.2 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-retryablehttp v0.7.1 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/huandu/xstrings v1.3.2 // indirect
github.com/iancoleman/orderedmap v0.0.0-20190318233801-ac98e3ecb4b0 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
github.com/jedisct1/go-minisign v0.0.0-20211028175153-1c139d1cc84b // indirect
github.com/jhump/protoreflect v1.13.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/jonboulle/clockwork v0.3.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/klauspost/compress v1.15.9 // indirect
github.com/klauspost/pgzip v1.2.5 // indirect
github.com/leodido/go-urn v1.2.1 // indirect
github.com/letsencrypt/boulder v0.0.0-20220929215747-76583552c2be // indirect
github.com/logrusorgru/aurora v0.0.0-20200102142835-e9ef32dff381 // indirect
github.com/magiconair/properties v1.8.6 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/miekg/pkcs11 v1.1.1 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/mozillazg/docker-credential-acr-helper v0.3.0 // indirect
github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 // indirect
github.com/nwaples/rardecode v1.1.0 // indirect
github.com/oklog/ulid v1.3.1 // indirect
github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pelletier/go-toml/v2 v2.0.5 // indirect
github.com/pierrec/lz4/v4 v4.1.15 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.13.0 // indirect
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.37.0 // indirect
github.com/prometheus/procfs v0.8.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 // indirect
github.com/rivo/uniseg v0.2.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/sassoftware/relic v0.0.0-20210427151427-dfb082b79b74 // indirect
github.com/secure-systems-lab/go-securesystemslib v0.4.0 // indirect
github.com/segmentio/ksuid v1.0.4 // indirect
github.com/shibumi/go-pathspec v1.3.0 // indirect
github.com/rogpeppe/go-internal v1.8.0 // indirect
github.com/shopspring/decimal v1.2.0 // indirect
github.com/sigstore/fulcio v0.6.0 // indirect
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 // indirect
github.com/soheilhy/cmux v0.1.5 // indirect
github.com/spf13/cast v1.5.0 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/spiffe/go-spiffe/v2 v2.1.1 // indirect
github.com/stretchr/objx v0.5.0 // indirect
github.com/subosito/gotenv v1.4.1 // indirect
github.com/sylabs/sif/v2 v2.8.1 // indirect
github.com/sylabs/squashfs v0.6.1 // indirect
github.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d // indirect
github.com/tent/canonical-json-go v0.0.0-20130607151641-96e4ba3a7613 // indirect
github.com/thales-e-security/pool v0.0.2 // indirect
github.com/therootcompany/xz v1.0.1 // indirect
github.com/theupdateframework/go-tuf v0.5.2-0.20220930112810-3890c1e7ace4 // indirect
github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399 // indirect
github.com/tjfoc/gmsm v1.3.2 // indirect
github.com/tmc/grpc-websocket-proxy v0.0.0-20201229170055-e5319fda7802 // indirect
github.com/transparency-dev/merkle v0.0.1 // indirect
github.com/ulikunitz/xz v0.5.10 // indirect
github.com/urfave/cli v1.22.7 // indirect
github.com/vbatts/tar-split v0.11.2 // indirect
github.com/xanzy/go-gitlab v0.73.1 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
github.com/zeebo/errs v1.2.2 // indirect
go.etcd.io/bbolt v1.3.6 // indirect
go.etcd.io/etcd/api/v3 v3.6.0-alpha.0 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.0-alpha.0 // indirect
go.etcd.io/etcd/client/v2 v2.306.0-alpha.0 // indirect
go.etcd.io/etcd/client/v3 v3.6.0-alpha.0 // indirect
go.etcd.io/etcd/etcdctl/v3 v3.6.0-alpha.0 // indirect
go.etcd.io/etcd/etcdutl/v3 v3.6.0-alpha.0 // indirect
go.etcd.io/etcd/pkg/v3 v3.6.0-alpha.0 // indirect
go.etcd.io/etcd/raft/v3 v3.6.0-alpha.0 // indirect
go.etcd.io/etcd/server/v3 v3.6.0-alpha.0 // indirect
go.etcd.io/etcd/tests/v3 v3.6.0-alpha.0 // indirect
go.etcd.io/etcd/v3 v3.6.0-alpha.0 // indirect
go.mongodb.org/mongo-driver v1.10.0 // indirect
go.opencensus.io v0.23.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.28.0 // indirect
go.opentelemetry.io/otel v1.7.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.7.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.7.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.7.0 // indirect
go.opentelemetry.io/otel/sdk v1.7.0 // indirect
go.opentelemetry.io/otel/trace v1.7.0 // indirect
go.opentelemetry.io/proto/otlp v0.16.0 // indirect
go.uber.org/atomic v1.10.0 // indirect
go.uber.org/multierr v1.8.0 // indirect
go.uber.org/zap v1.23.0 // indirect
golang.org/x/oauth2 v0.0.0-20221006150949-b44042a4b9c1 // indirect
golang.org/x/sync v0.0.0-20220929204114-8fcdb60fdcc0 // indirect
golang.org/x/sys v0.0.0-20220928140112-f11e5e49a4ec // indirect
golang.org/x/text v0.3.8-0.20211004125949-5bd84dd9b33b // indirect
golang.org/x/time v0.0.0-20220922220347-f3bd1da661af // indirect
golang.org/x/tools v0.1.12 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/api v0.99.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20221010155953-15ba04fc1c0e // indirect
google.golang.org/grpc v1.50.1 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/cheggaaa/pb.v1 v1.0.28 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect
k8s.io/api v0.23.5 // indirect
k8s.io/apimachinery v0.23.5 // indirect
k8s.io/client-go v0.23.5 // indirect
k8s.io/klog/v2 v2.60.1-0.20220317184644-43cc75f9ae89 // indirect
k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf // indirect
k8s.io/utils v0.0.0-20220210201930-3a6ce19ff2f9 // indirect
lukechampine.com/uint128 v1.1.1 // indirect
modernc.org/cc/v3 v3.36.0 // indirect
modernc.org/ccgo/v3 v3.16.6 // indirect
@@ -322,10 +143,6 @@ require (
modernc.org/sqlite v1.17.3 // indirect
modernc.org/strutil v1.1.1 // indirect
modernc.org/token v1.0.0 // indirect
sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 // indirect
sigs.k8s.io/release-utils v0.7.3 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.1 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)
require (
+5 -971
View File
File diff suppressed because it is too large Load Diff
+4 -2
View File
@@ -53,9 +53,9 @@ type Application struct {
Secrets secrets `yaml:"secrets" json:"secrets" mapstructure:"secrets"`
Registry registry `yaml:"registry" json:"registry" mapstructure:"registry"`
Exclusions []string `yaml:"exclude" json:"exclude" mapstructure:"exclude"`
Attest attest `yaml:"attest" json:"attest" mapstructure:"attest"`
Platform string `yaml:"platform" json:"platform" mapstructure:"platform"`
Name string `yaml:"name" json:"name" mapstructure:"name"`
Parallelism int `yaml:"parallelism" json:"parallelism" mapstructure:"parallelism"` // the number of catalog workers to run in parallel
}
func (cfg Application) ToCatalogerConfig() cataloger.Config {
@@ -65,7 +65,8 @@ func (cfg Application) ToCatalogerConfig() cataloger.Config {
IncludeUnindexedArchives: cfg.Package.SearchUnindexedArchives,
Scope: cfg.Package.Cataloger.ScopeOpt,
},
Catalogers: cfg.Catalogers,
Catalogers: cfg.Catalogers,
Parallelism: cfg.Parallelism,
}
}
@@ -182,6 +183,7 @@ func loadDefaultValues(v *viper.Viper) {
v.SetDefault("quiet", false)
v.SetDefault("check-for-app-update", true)
v.SetDefault("catalogers", nil)
v.SetDefault("parallelism", 1)
// for each field in the configuration struct, see if the field implements the defaultValueLoader interface and invoke it if it does
value := reflect.ValueOf(Application{})
-56
View File
@@ -1,56 +0,0 @@
package config
import (
"fmt"
"os"
"github.com/mitchellh/go-homedir"
"github.com/sigstore/cosign/cmd/cosign/cli/options"
"github.com/spf13/viper"
)
// IMPORTANT: do not show the password in any YAML/JSON output (sensitive information)
type attest struct {
KeyRef string `yaml:"key" json:"key" mapstructure:"key"` // same as --key, file path to the private key
Cert string `yaml:"cert" json:"cert" mapstructure:"cert"`
NoUpload bool `yaml:"no_upload" json:"noUpload" mapstructure:"no_upload"`
Force bool `yaml:"force" json:"force" mapstructure:"force"`
Recursive bool `yaml:"recursive" json:"recursive" mapstructure:"recursive"`
Replace bool `yaml:"replace" json:"replace" mapstructure:"replace"`
Password string `yaml:"-" json:"-" mapstructure:"password"` // password for the private key
FulcioURL string `yaml:"fulcio_url" json:"fulcioUrl" mapstructure:"fulcio_url"`
FulcioIdentityToken string `yaml:"fulcio_identity_token" json:"fulcio_identity_token" mapstructure:"fulcio_identity_token"`
InsecureSkipFulcioVerify bool `yaml:"insecure_skip_verify" json:"insecure_skip_verify" mapstructure:"insecure_skip_verify"`
RekorURL string `yaml:"rekor_url" json:"rekorUrl" mapstructure:"rekor_url"`
OIDCIssuer string `yaml:"oidc_issuer" json:"oidcIssuer" mapstructure:"oidc_issuer"`
OIDCClientID string `yaml:"oidc_client_id" json:"oidcClientId" mapstructure:"oidc_client_id"`
OIDCRedirectURL string `yaml:"oidc_redirect_url" json:"OIDCRedirectURL" mapstructure:"oidc_redirect_url"`
}
func (cfg *attest) parseConfigValues() error {
if cfg.KeyRef != "" {
expandedPath, err := homedir.Expand(cfg.KeyRef)
if err != nil {
return fmt.Errorf("unable to expand key path=%q: %w", cfg.KeyRef, err)
}
cfg.KeyRef = expandedPath
}
if cfg.Password == "" {
// we allow for configuration via syft config/env vars and additionally interop with known cosign config env vars
if pw, ok := os.LookupEnv("COSIGN_PASSWORD"); ok {
cfg.Password = pw
}
}
return nil
}
func (cfg attest) loadDefaultValues(v *viper.Viper) {
v.SetDefault("attest.key", "")
v.SetDefault("attest.password", "")
v.SetDefault("attest.fulcio_url", options.DefaultFulcioURL)
v.SetDefault("attest.rekor_url", options.DefaultRekorURL)
v.SetDefault("attest.oidc_issuer", options.DefaultOIDCIssuerURL)
v.SetDefault("attest.oidc_client_id", "sigstore")
}
+1 -1
View File
@@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "6.0.0"
JSONSchemaVersion = "6.1.0"
)
+27 -20
View File
@@ -27,26 +27,33 @@ can be extended to include specific package metadata struct shapes in the future
// This should represent all possible metadatas represented in the pkg.Package.Metadata field (an interface{}).
// When a new package metadata definition is created it will need to be manually added here. The variable name does
// not matter as long as it is exported.
// TODO: this should be generated from reflection of whats in the pkg package
type artifactMetadataContainer struct {
Apk pkg.ApkMetadata
Alpm pkg.AlpmMetadata
Dpkg pkg.DpkgMetadata
Gem pkg.GemMetadata
Java pkg.JavaMetadata
Npm pkg.NpmPackageJSONMetadata
Python pkg.PythonPackageMetadata
Rpm pkg.RpmMetadata
Cargo pkg.CargoPackageMetadata
Go pkg.GolangMetadata
Php pkg.PhpComposerJSONMetadata
Dart pkg.DartPubMetadata
Dotnet pkg.DotnetDepsMetadata
Portage pkg.PortageMetadata
Conan pkg.ConanMetadata
ConanLock pkg.ConanLockMetadata
KbPackage pkg.KbPackageMetadata
Hackage pkg.HackageMetadata
SwiftCocopods pkg.CocoapodsMetadata
Alpm pkg.AlpmMetadata
Apk pkg.ApkMetadata
Binary pkg.BinaryMetadata
Cocopods pkg.CocoapodsMetadata
Conan pkg.ConanMetadata
ConanLock pkg.ConanLockMetadata
Dart pkg.DartPubMetadata
Dotnet pkg.DotnetDepsMetadata
Dpkg pkg.DpkgMetadata
Gem pkg.GemMetadata
Go pkg.GolangMetadata
Hackage pkg.HackageMetadata
Java pkg.JavaMetadata
KbPackage pkg.KbPackageMetadata
NpmPackage pkg.NpmPackageJSONMetadata
NpmPackageLock pkg.NpmPackageLockJSONMetadata
MixLock pkg.MixLockMetadata
Php pkg.PhpComposerJSONMetadata
Portage pkg.PortageMetadata
PythonPackage pkg.PythonPackageMetadata
PythonPipfilelock pkg.PythonPipfileLockMetadata
Rebar pkg.RebarLockMetadata
Rpm pkg.RpmMetadata
RustCargo pkg.CargoPackageMetadata
}
func main() {
@@ -136,7 +143,7 @@ func write(schema []byte) {
}
// the generated schema is different, bail with error :(
fmt.Printf("Cowardly refusing to overwrite existing schema (%s)!\nSee the scheam/json/README.md for how to increment\n", filename)
fmt.Printf("Cowardly refusing to overwrite existing schema (%s)!\nSee the schema/json/README.md for how to increment\n", filename)
os.Exit(1)
}
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -32,6 +32,6 @@ const (
// ImportStarted is a partybus event that occurs when an SBOM upload process has begun
ImportStarted partybus.EventType = "syft-import-started-event"
// UploadAttestation is a partybus event that occurs when syft uploads an attestation to an OCI registry (+ any transparency log)
UploadAttestation partybus.EventType = "syft-upload-attestation"
// AttestationStarted is a partybus event that occurs when starting an SBOM attestation process
AttestationStarted partybus.EventType = "syft-attestation-started-event"
)
+23
View File
@@ -0,0 +1,23 @@
package monitor
import (
"io"
"github.com/wagoodman/go-progress"
)
type ShellProgress struct {
io.Reader
*progress.Manual
}
type Title struct {
Default string
WhileRunning string
OnSuccess string
}
type GenericTask struct {
Title Title
Context string
}
+13 -6
View File
@@ -5,11 +5,13 @@ package parsers
import (
"fmt"
"io"
"github.com/wagoodman/go-partybus"
"github.com/wagoodman/go-progress"
"github.com/anchore/syft/syft/event"
"github.com/anchore/syft/syft/event/monitor"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg/cataloger"
)
@@ -153,15 +155,20 @@ func ParseImportStarted(e partybus.Event) (string, progress.StagedProgressable,
return host, prog, nil
}
func ParseUploadAttestation(e partybus.Event) (progress.StagedProgressable, error) {
if err := checkEventType(e.Type, event.UploadAttestation); err != nil {
return nil, err
func ParseAttestationStartedEvent(e partybus.Event) (io.Reader, progress.Progressable, *monitor.GenericTask, error) {
if err := checkEventType(e.Type, event.AttestationStarted); err != nil {
return nil, nil, nil, err
}
prog, ok := e.Value.(progress.StagedProgressable)
source, ok := e.Source.(monitor.GenericTask)
if !ok {
return nil, newPayloadErr(e.Type, "Value", e.Value)
return nil, nil, nil, newPayloadErr(e.Type, "Source", e.Source)
}
return prog, nil
sp, ok := e.Value.(*monitor.ShellProgress)
if !ok {
return nil, nil, nil, newPayloadErr(e.Type, "Value", e.Value)
}
return sp.Reader, sp.Manual, &source, nil
}
@@ -92,7 +92,7 @@ func Test_decode(t *testing.T) {
},
},
CPE: "cpe:2.3:*:another:package:2:*:*:*:*:*:*:*",
PackageURL: "pkg:alpine/alpine-baselayout@3.2.0-r16?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.14.2",
PackageURL: "pkg:apk/alpine/alpine-baselayout@3.2.0-r16?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.14.2",
Properties: &[]cyclonedx.Property{
{
@@ -193,7 +193,7 @@ func Test_decode(t *testing.T) {
{
pkg: "package-2",
ver: "2.0.2",
purl: "pkg:alpine/alpine-baselayout@3.2.0-r16?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.14.2",
purl: "pkg:apk/alpine/alpine-baselayout@3.2.0-r16?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.14.2",
},
},
},
@@ -43,6 +43,8 @@ func SourceInfo(p pkg.Package) string {
answer = "acquired package info from portage DB"
case pkg.HackagePkg:
answer = "acquired package info from cabal or stack manifest files"
case pkg.HexPkg:
answer = "acquired package info from rebar3 or mix manifest file"
default:
answer = "acquired package info from the following paths"
}
@@ -191,6 +191,14 @@ func Test_SourceInfo(t *testing.T) {
"acquired package info from the following paths",
},
},
{
input: pkg.Package{
Type: pkg.HexPkg,
},
expected: []string{
"from rebar3 or mix manifest file",
},
},
}
var pkgTypes []pkg.Type
for _, test := range tests {
@@ -46,7 +46,7 @@ func TestToSyftModel(t *testing.T) {
},
{
Category: "PACKAGE-MANAGER",
Locator: "pkg:alpine/pkg-1@5.4.3?arch=x86_64&upstream=p1-origin&distro=alpine-3.10.9",
Locator: "pkg:apk/alpine/pkg-1@5.4.3?arch=x86_64&upstream=p1-origin&distro=alpine-3.10.9",
RefType: "purl",
},
},
@@ -145,7 +145,7 @@ func Test_extractMetadata(t *testing.T) {
PackageExternalReferences: []*spdx.PackageExternalReference{
{
Category: "PACKAGE-MANAGER",
Locator: "pkg:alpine/pkg-2@7.3.1?arch=x86_64&upstream=apk-origin@9.1.3&distro=alpine-3.10.9",
Locator: "pkg:apk/alpine/pkg-2@7.3.1?arch=x86_64&upstream=apk-origin@9.1.3&distro=alpine-3.10.9",
RefType: "purl",
},
},
@@ -27,7 +27,7 @@
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceLocator": "pkg:alpine/busybox@1.30.1-r5?arch=x86_64&distro=alpine-3.10.9",
"referenceLocator": "pkg:apk/alpine/busybox@1.30.1-r5?arch=x86_64&distro=alpine-3.10.9",
"referenceType": "purl"
}
],
@@ -51,7 +51,7 @@
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceLocator": "pkg:alpine/libssl1.1@1.1.1k-r0?arch=x86_64&distro=alpine-3.10.9",
"referenceLocator": "pkg:apk/alpine/libssl1.1@1.1.1k-r0?arch=x86_64&distro=alpine-3.10.9",
"referenceType": "purl"
}
],
@@ -100,7 +100,7 @@
},
{
"referenceCategory": "PACKAGE_MANAGER",
"referenceLocator": "pkg:alpine/ssl_client@1.30.1-r5?arch=x86_64&upstream=busybox&distro=alpine-3.10.9",
"referenceLocator": "pkg:apk/alpine/ssl_client@1.30.1-r5?arch=x86_64&upstream=busybox&distro=alpine-3.10.9",
"referenceType": "purl"
}
],
@@ -21,7 +21,7 @@ PackageLicenseDeclared: GPL-2.0-only
PackageCopyrightText: NOASSERTION
PackageDescription: Size optimized toolbox of many common UNIX utilities
ExternalRef: SECURITY cpe23Type cpe:2.3:a:busybox:busybox:1.31.1-r19:*:*:*:*:*:*:*
ExternalRef: PACKAGE-MANAGER purl pkg:alpine/busybox@1.31.1-r19?arch=x86_64&upstream=busybox&distro=alpine-3.12.5
ExternalRef: PACKAGE-MANAGER purl pkg:apk/alpine/busybox@1.31.1-r19?arch=x86_64&upstream=busybox&distro=alpine-3.12.5
##### Package: my-app
@@ -185,7 +185,7 @@
}
},
"schema": {
"version": "6.0.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.0.0.json"
"version": "6.1.0",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-6.1.0.json"
}
}
@@ -9,7 +9,7 @@
"locations": [
{
"path": "/somefile-1.txt",
"layerID": "sha256:62058900d4ce269c900160b8dd255fe310c3a459dda236d041102fa070f84406"
"layerID": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59"
}
],
"licenses": [
@@ -40,7 +40,7 @@
"locations": [
{
"path": "/somefile-2.txt",
"layerID": "sha256:623ad97366f39ae279f1925673cdacb4851ddf2e3266f04e63010ec080a098c1"
"layerID": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec"
}
],
"licenses": [],
@@ -64,11 +64,11 @@
],
"artifactRelationships": [],
"source": {
"id": "6de18113d4a87f3503d13d10d1e69ebe2f5b41880aec96fb506b2113fd8df2f8",
"id": "1a678f111c8ddc66fd82687bb024e0dd6af61314404937a80e810c0cf317b796",
"type": "image",
"target": {
"userInput": "user-image-input",
"imageID": "sha256:fdd8b25b9402be0c8cf99e1edde6da614310d6bacd2e45a03781a3ef4a59025f",
"imageID": "sha256:3c51b06feb0cda8ee62d0e3755ef2a8496a6b71f8a55b245f07f31c4bb813d31",
"manifestDigest": "sha256:2731251dc34951c0e50fcc643b4c5f74922dad1a5d98f302b504cf46cd5d9368",
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"tags": [
@@ -78,17 +78,17 @@
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:62058900d4ce269c900160b8dd255fe310c3a459dda236d041102fa070f84406",
"digest": "sha256:fb6beecb75b39f4bb813dbf177e501edd5ddb3e69bb45cedeb78c676ee1b7a59",
"size": 22
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:623ad97366f39ae279f1925673cdacb4851ddf2e3266f04e63010ec080a098c1",
"digest": "sha256:319b588ce64253a87b533c8ed01cf0025e0eac98e7b516e12532957e1244fdec",
"size": 16
}
],
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NzMsImRpZ2VzdCI6InNoYTI1NjpmZGQ4YjI1Yjk0MDJiZTBjOGNmOTllMWVkZGU2ZGE2MTQzMTBkNmJhY2QyZTQ1YTAzNzgxYTNlZjRhNTkwMjVmIn0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1Njo2MjA1ODkwMGQ0Y2UyNjljOTAwMTYwYjhkZDI1NWZlMzEwYzNhNDU5ZGRhMjM2ZDA0MTEwMmZhMDcwZjg0NDA2In0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjYyM2FkOTczNjZmMzlhZTI3OWYxOTI1NjczY2RhY2I0ODUxZGRmMmUzMjY2ZjA0ZTYzMDEwZWMwODBhMDk4YzEifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjItMTItMjFUMTk6MDc6NDQuMDcyMTgyMTk0WiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIyLTEyLTIxVDE5OjA3OjQ0LjA0NjkyNTc5M1oiLCJjcmVhdGVkX2J5IjoiQUREIGZpbGUtMS50eHQgL3NvbWVmaWxlLTEudHh0ICMgYnVpbGRraXQiLCJjb21tZW50IjoiYnVpbGRraXQuZG9ja2VyZmlsZS52MCJ9LHsiY3JlYXRlZCI6IjIwMjItMTItMjFUMTk6MDc6NDQuMDcyMTgyMTk0WiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0yLnR4dCAvc29tZWZpbGUtMi50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn1dLCJvcyI6ImxpbnV4Iiwicm9vdGZzIjp7InR5cGUiOiJsYXllcnMiLCJkaWZmX2lkcyI6WyJzaGEyNTY6NjIwNTg5MDBkNGNlMjY5YzkwMDE2MGI4ZGQyNTVmZTMxMGMzYTQ1OWRkYTIzNmQwNDExMDJmYTA3MGY4NDQwNiIsInNoYTI1Njo2MjNhZDk3MzY2ZjM5YWUyNzlmMTkyNTY3M2NkYWNiNDg1MWRkZjJlMzI2NmYwNGU2MzAxMGVjMDgwYTA5OGMxIl19fQ==",
"manifest": "eyJzY2hlbWFWZXJzaW9uIjoyLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmRpc3RyaWJ1dGlvbi5tYW5pZmVzdC52Mitqc29uIiwiY29uZmlnIjp7Im1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5kb2NrZXIuY29udGFpbmVyLmltYWdlLnYxK2pzb24iLCJzaXplIjo2NzMsImRpZ2VzdCI6InNoYTI1NjozYzUxYjA2ZmViMGNkYThlZTYyZDBlMzc1NWVmMmE4NDk2YTZiNzFmOGE1NWIyNDVmMDdmMzFjNGJiODEzZDMxIn0sImxheWVycyI6W3sibWVkaWFUeXBlIjoiYXBwbGljYXRpb24vdm5kLmRvY2tlci5pbWFnZS5yb290ZnMuZGlmZi50YXIuZ3ppcCIsInNpemUiOjIwNDgsImRpZ2VzdCI6InNoYTI1NjpmYjZiZWVjYjc1YjM5ZjRiYjgxM2RiZjE3N2U1MDFlZGQ1ZGRiM2U2OWJiNDVjZWRlYjc4YzY3NmVlMWI3YTU5In0seyJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuZG9ja2VyLmltYWdlLnJvb3Rmcy5kaWZmLnRhci5nemlwIiwic2l6ZSI6MjA0OCwiZGlnZXN0Ijoic2hhMjU2OjMxOWI1ODhjZTY0MjUzYTg3YjUzM2M4ZWQwMWNmMDAyNWUwZWFjOThlN2I1MTZlMTI1MzI5NTdlMTI0NGZkZWMifV19",
"config": "eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsImNvbmZpZyI6eyJFbnYiOlsiUEFUSD0vdXNyL2xvY2FsL3NiaW46L3Vzci9sb2NhbC9iaW46L3Vzci9zYmluOi91c3IvYmluOi9zYmluOi9iaW4iXSwiV29ya2luZ0RpciI6Ii8iLCJPbkJ1aWxkIjpudWxsfSwiY3JlYXRlZCI6IjIwMjItMDgtMDFUMjA6MDk6MjIuNTA5NDIxNzEyWiIsImhpc3RvcnkiOlt7ImNyZWF0ZWQiOiIyMDIyLTA4LTAxVDIwOjA5OjIyLjQ4Nzg5NTUxOVoiLCJjcmVhdGVkX2J5IjoiQUREIGZpbGUtMS50eHQgL3NvbWVmaWxlLTEudHh0ICMgYnVpbGRraXQiLCJjb21tZW50IjoiYnVpbGRraXQuZG9ja2VyZmlsZS52MCJ9LHsiY3JlYXRlZCI6IjIwMjItMDgtMDFUMjA6MDk6MjIuNTA5NDIxNzEyWiIsImNyZWF0ZWRfYnkiOiJBREQgZmlsZS0yLnR4dCAvc29tZWZpbGUtMi50eHQgIyBidWlsZGtpdCIsImNvbW1lbnQiOiJidWlsZGtpdC5kb2NrZXJmaWxlLnYwIn1dLCJvcyI6ImxpbnV4Iiwicm9vdGZzIjp7InR5cGUiOiJsYXllcnMiLCJkaWZmX2lkcyI6WyJzaGEyNTY6ZmI2YmVlY2I3NWIzOWY0YmI4MTNkYmYxNzdlNTAxZWRkNWRkYjNlNjliYjQ1Y2VkZWI3OGM2NzZlZTFiN2E1OSIsInNoYTI1NjozMTliNTg4Y2U2NDI1M2E4N2I1MzNjOGVkMDFjZjAwMjVlMGVhYzk4ZTdiNTE2ZTEyNTMyOTU3ZTEyNDRmZGVjIl19fQ==",
"repoDigests": [],
"architecture": "",
"os": ""
+14 -14
View File
@@ -24,7 +24,7 @@
"cpe:2.3:a:alpine:alpine-baselayout:3.2.0-r16:*:*:*:*:*:*:*",
"cpe:2.3:a:alpine:alpine_baselayout:3.2.0-r16:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/alpine-baselayout@3.2.0-r16?arch=x86_64",
"purl": "pkg:apk/alpine/alpine-baselayout@3.2.0-r16?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "alpine-baselayout",
@@ -504,7 +504,7 @@
"cpe:2.3:a:alpine:alpine-keys:2.3-r1:*:*:*:*:*:*:*",
"cpe:2.3:a:alpine:alpine_keys:2.3-r1:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/alpine-keys@2.3-r1?arch=x86_64",
"purl": "pkg:apk/alpine/alpine-keys@2.3-r1?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "alpine-keys",
@@ -790,7 +790,7 @@
"cpe:2.3:a:apk:apk-tools:2.12.7-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:apk:apk_tools:2.12.7-r0:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/apk-tools@2.12.7-r0?arch=x86_64",
"purl": "pkg:apk/alpine/apk-tools@2.12.7-r0?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "apk-tools",
@@ -882,7 +882,7 @@
"cpes": [
"cpe:2.3:a:busybox:busybox:1.33.1-r3:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/busybox@1.33.1-r3?arch=x86_64",
"purl": "pkg:apk/alpine/busybox@1.33.1-r3?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "busybox",
@@ -1059,7 +1059,7 @@
"cpe:2.3:a:ca:ca-certificates-bundle:20191127-r5:*:*:*:*:*:*:*",
"cpe:2.3:a:ca:ca_certificates_bundle:20191127-r5:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/ca-certificates-bundle@20191127-r5?arch=x86_64",
"purl": "pkg:apk/alpine/ca-certificates-bundle@20191127-r5?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "ca-certificates-bundle",
@@ -1131,7 +1131,7 @@
"cpe:2.3:a:libc:libc-utils:0.7.2-r3:*:*:*:*:*:*:*",
"cpe:2.3:a:libc:libc_utils:0.7.2-r3:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/libc-utils@0.7.2-r3?arch=x86_64",
"purl": "pkg:apk/alpine/libc-utils@0.7.2-r3?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "libc-utils",
@@ -1169,7 +1169,7 @@
"cpes": [
"cpe:2.3:a:libcrypto1.1:libcrypto1.1:1.1.1l-r0:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/libcrypto1.1@1.1.1l-r0?arch=x86_64",
"purl": "pkg:apk/alpine/libcrypto1.1@1.1.1l-r0?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "libcrypto1.1",
@@ -1347,7 +1347,7 @@
"cpes": [
"cpe:2.3:a:libretls:libretls:3.3.3p1-r2:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/libretls@3.3.3p1-r2?arch=x86_64",
"purl": "pkg:apk/alpine/libretls@3.3.3p1-r2?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "libretls",
@@ -1412,7 +1412,7 @@
"cpes": [
"cpe:2.3:a:libssl1.1:libssl1.1:1.1.1l-r0:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/libssl1.1@1.1.1l-r0?arch=x86_64",
"purl": "pkg:apk/alpine/libssl1.1@1.1.1l-r0?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "libssl1.1",
@@ -1480,7 +1480,7 @@
"cpes": [
"cpe:2.3:a:musl:musl:1.2.2-r3:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/musl@1.2.2-r3?arch=x86_64",
"purl": "pkg:apk/alpine/musl@1.2.2-r3?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "musl",
@@ -1549,7 +1549,7 @@
"cpe:2.3:a:musl:musl-utils:1.2.2-r3:*:*:*:*:*:*:*",
"cpe:2.3:a:musl:musl_utils:1.2.2-r3:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/musl-utils@1.2.2-r3?arch=x86_64",
"purl": "pkg:apk/alpine/musl-utils@1.2.2-r3?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "musl-utils",
@@ -1647,7 +1647,7 @@
"cpes": [
"cpe:2.3:a:scanelf:scanelf:1.3.2-r0:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/scanelf@1.3.2-r0?arch=x86_64",
"purl": "pkg:apk/alpine/scanelf@1.3.2-r0?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "scanelf",
@@ -1707,7 +1707,7 @@
"cpe:2.3:a:ssl:ssl-client:1.33.1-r3:*:*:*:*:*:*:*",
"cpe:2.3:a:ssl:ssl_client:1.33.1-r3:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/ssl_client@1.33.1-r3?arch=x86_64",
"purl": "pkg:apk/alpine/ssl_client@1.33.1-r3?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "ssl_client",
@@ -1762,7 +1762,7 @@
"cpes": [
"cpe:2.3:a:zlib:zlib:1.2.11-r3:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/zlib@1.2.11-r3?arch=x86_64",
"purl": "pkg:apk/alpine/zlib@1.2.11-r3?arch=x86_64",
"metadataType": "ApkMetadata",
"metadata": {
"package": "zlib",
+1 -1
View File
@@ -69,7 +69,7 @@ func CatalogPackages(src *source.Source, cfg cataloger.Config) (*pkg.Catalog, []
}
}
catalog, relationships, err := cataloger.Catalog(resolver, release, catalogers...)
catalog, relationships, err := cataloger.Catalog(resolver, release, cfg.Parallelism, catalogers...)
if err != nil {
return nil, nil, nil, err
}
+1 -3
View File
@@ -42,10 +42,8 @@ func packageURL(m pkg.ApkMetadata, distro *linux.Release) string {
}
return packageurl.NewPackageURL(
// note: this is currently a candidate and not technically within spec
// see https://github.com/package-url/purl-spec#other-candidate-types-to-define
packageurl.TypeAlpine,
"alpine",
"",
m.Package,
m.Version,
pkg.PURLQualifiers(
+5 -5
View File
@@ -43,7 +43,7 @@ func Test_PackageURL(t *testing.T) {
ID: "alpine",
VersionID: "3.4.6",
},
expected: "pkg:alpine/p@v?arch=a&distro=alpine-3.4.6",
expected: "pkg:apk/alpine/p@v?arch=a&distro=alpine-3.4.6",
},
{
name: "missing architecture",
@@ -55,7 +55,7 @@ func Test_PackageURL(t *testing.T) {
ID: "alpine",
VersionID: "3.4.6",
},
expected: "pkg:alpine/p@v?distro=alpine-3.4.6",
expected: "pkg:apk/alpine/p@v?distro=alpine-3.4.6",
},
// verify #351
{
@@ -68,7 +68,7 @@ func Test_PackageURL(t *testing.T) {
ID: "alpine",
VersionID: "3.4.6",
},
expected: "pkg:alpine/g++@v84?arch=am86&distro=alpine-3.4.6",
expected: "pkg:apk/alpine/g++@v84?arch=am86&distro=alpine-3.4.6",
},
{
metadata: pkg.ApkMetadata{
@@ -80,7 +80,7 @@ func Test_PackageURL(t *testing.T) {
ID: "alpine",
VersionID: "3.15.0",
},
expected: "pkg:alpine/g%20plus%20plus@v84?arch=am86&distro=alpine-3.15.0",
expected: "pkg:apk/alpine/g%20plus%20plus@v84?arch=am86&distro=alpine-3.15.0",
},
{
name: "add source information as qualifier",
@@ -94,7 +94,7 @@ func Test_PackageURL(t *testing.T) {
ID: "alpine",
VersionID: "3.4.6",
},
expected: "pkg:alpine/p@v?arch=a&upstream=origin&distro=alpine-3.4.6",
expected: "pkg:apk/alpine/p@v?arch=a&upstream=origin&distro=alpine-3.4.6",
},
}
@@ -637,7 +637,7 @@ func TestMultiplePackages(t *testing.T) {
Version: "0.7.2-r0",
Licenses: []string{"BSD"},
Type: pkg.ApkPkg,
PURL: "pkg:alpine/libc-utils@0.7.2-r0?arch=x86_64&upstream=libc-dev&distro=alpine-3.12",
PURL: "pkg:apk/alpine/libc-utils@0.7.2-r0?arch=x86_64&upstream=libc-dev&distro=alpine-3.12",
Locations: fixtureLocationSet,
MetadataType: pkg.ApkMetadataType,
Metadata: pkg.ApkMetadata{
@@ -663,7 +663,7 @@ func TestMultiplePackages(t *testing.T) {
Version: "1.1.24-r2",
Licenses: []string{"MIT", "BSD", "GPL2+"},
Type: pkg.ApkPkg,
PURL: "pkg:alpine/musl-utils@1.1.24-r2?arch=x86_64&upstream=musl&distro=alpine-3.12",
PURL: "pkg:apk/alpine/musl-utils@1.1.24-r2?arch=x86_64&upstream=musl&distro=alpine-3.12",
Locations: fixtureLocationSet,
MetadataType: pkg.ApkMetadataType,
Metadata: pkg.ApkMetadata{
+125
View File
@@ -17,6 +17,131 @@ func TestClassifierCataloger_DefaultClassifiers_PositiveCases(t *testing.T) {
fixtureDir string
expected pkg.Package
}{
{
name: "positive-httpd-2.4.54",
fixtureDir: "test-fixtures/classifiers/positive/httpd-2.4.54",
expected: pkg.Package{
Name: "httpd",
Version: "2.4.54",
Type: "binary",
PURL: "pkg:generic/httpd@2.4.54",
Locations: singleLocation("httpd"),
Metadata: pkg.BinaryMetadata{
Classifier: "httpd-binary",
},
},
}, {
name: "positive-php-cli-8.2.1",
fixtureDir: "test-fixtures/classifiers/positive/php-cli-8.2.1",
expected: pkg.Package{
Name: "php-cli",
Version: "8.2.1",
Type: "binary",
PURL: "pkg:generic/php-cli@8.2.1",
Locations: singleLocation("php"),
Metadata: pkg.BinaryMetadata{
Classifier: "php-cli-binary",
},
},
},
{
name: "positive-php-fpm-8.2.1",
fixtureDir: "test-fixtures/classifiers/positive/php-fpm-8.2.1",
expected: pkg.Package{
Name: "php-fpm",
Version: "8.2.1",
Type: "binary",
PURL: "pkg:generic/php-fpm@8.2.1",
Locations: singleLocation("php-fpm"),
Metadata: pkg.BinaryMetadata{
Classifier: "php-fpm-binary",
},
},
},
{
name: "positive-php-apache-8.2.1",
fixtureDir: "test-fixtures/classifiers/positive/php-apache-8.2.1",
expected: pkg.Package{
Name: "libphp",
Version: "8.2.1",
Type: "binary",
PURL: "pkg:generic/php@8.2.1",
Locations: singleLocation("libphp.so"),
Metadata: pkg.BinaryMetadata{
Classifier: "php-apache-binary",
},
},
},
{
name: "positive-redis-2.8.23",
fixtureDir: "test-fixtures/classifiers/positive/redis-server-2.8.23",
expected: pkg.Package{
Name: "redis",
Version: "2.8.23",
Type: "binary",
PURL: "pkg:generic/redis@2.8.23",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
},
},
{
name: "positive-redis-4.0.11",
fixtureDir: "test-fixtures/classifiers/positive/redis-server-4.0.11",
expected: pkg.Package{
Name: "redis",
Version: "4.0.11",
Type: "binary",
PURL: "pkg:generic/redis@4.0.11",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
},
},
{
name: "positive-redis-5.0.0",
fixtureDir: "test-fixtures/classifiers/positive/redis-server-5.0.0",
expected: pkg.Package{
Name: "redis",
Version: "5.0.0",
Type: "binary",
PURL: "pkg:generic/redis@5.0.0",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
},
},
{
name: "positive-redis-6.0.16",
fixtureDir: "test-fixtures/classifiers/positive/redis-server-6.0.16",
expected: pkg.Package{
Name: "redis",
Version: "6.0.16",
Type: "binary",
PURL: "pkg:generic/redis@6.0.16",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
},
},
{
name: "positive-redis-7.0.0",
fixtureDir: "test-fixtures/classifiers/positive/redis-server-7.0.0",
expected: pkg.Package{
Name: "redis",
Version: "7.0.0",
Type: "binary",
PURL: "pkg:generic/redis@7.0.0",
Locations: singleLocation("redis-server"),
Metadata: pkg.BinaryMetadata{
Classifier: "redis-binary",
},
},
},
{
name: "positive-libpython3.7.so",
fixtureDir: "test-fixtures/classifiers/positive",
-1
View File
@@ -125,7 +125,6 @@ func singlePackage(classifier classifier, reader source.LocationReadCloser, matc
p := pkg.Package{
Name: classifier.Package,
Version: version,
Language: pkg.Binary,
Locations: source.NewLocationSet(reader.Location),
Type: pkg.BinaryPkg,
CPEs: cpes,
@@ -38,6 +38,15 @@ var defaultClassifiers = []classifier{
PURL: mustPURL("pkg:generic/go@version"),
CPEs: singleCPE("cpe:2.3:a:golang:go:*:*:*:*:*:*:*:*"),
},
{
Class: "redis-binary",
FileGlob: "**/redis-server",
EvidenceMatcher: fileContentsVersionMatcher(
`(?s)payload %5.*(?P<version>\d.\d\.\d\d*?)[a-z0-9]{12}-[0-9]{19}`),
Package: "redis",
PURL: mustPURL("pkg:generic/redis@version"),
CPEs: singleCPE("cpe:2.3:a:redislabs:redis:*:*:*:*:*:*:*:*"),
},
{
Class: "java-binary-openjdk",
FileGlob: "**/java",
@@ -95,4 +104,41 @@ var defaultClassifiers = []classifier{
`(?m)BusyBox\s+v(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`),
Package: "busybox",
},
{
Class: "php-cli-binary",
FileGlob: "**/php*",
EvidenceMatcher: fileNameTemplateVersionMatcher(
`(.*/|^)php[0-9]*$`,
`(?m)X-Powered-By: PHP\/(?P<version>[0-9]+\.[0-9]+\.[0-9]+(beta[0-9]+|alpha[0-9]+|RC[0-9]+)?)`),
Package: "php-cli",
PURL: mustPURL("pkg:generic/php-cli@version"),
CPEs: singleCPE("cpe:2.3:a:php:php:*:*:*:*:*:*:*:*"),
},
{
Class: "php-fpm-binary",
FileGlob: "**/php-fpm*",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)X-Powered-By: PHP\/(?P<version>[0-9]+\.[0-9]+\.[0-9]+(beta[0-9]+|alpha[0-9]+|RC[0-9]+)?)`),
Package: "php-fpm",
PURL: mustPURL("pkg:generic/php-fpm@version"),
CPEs: singleCPE("cpe:2.3:a:php:php:*:*:*:*:*:*:*:*"),
},
{
Class: "php-apache-binary",
FileGlob: "**/libphp*.so",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)X-Powered-By: PHP\/(?P<version>[0-9]+\.[0-9]+\.[0-9]+(beta[0-9]+|alpha[0-9]+|RC[0-9]+)?)`),
Package: "libphp",
PURL: mustPURL("pkg:generic/php@version"),
CPEs: singleCPE("cpe:2.3:a:php:php:*:*:*:*:*:*:*:*"),
},
{
Class: "httpd-binary",
FileGlob: "**/httpd",
EvidenceMatcher: fileContentsVersionMatcher(
`(?m)Apache\/(?P<version>[0-9]+\.[0-9]+\.[0-9]+)`),
Package: "httpd",
PURL: mustPURL("pkg:generic/httpd@version"),
CPEs: singleCPE("cpe:2.3:a:apache:http_server:*:*:*:*:*:*:*:*"),
},
}
@@ -1 +1,2 @@
!libpython3.7.so
!libpython3.7.so
!libphp.so
@@ -0,0 +1,18 @@
<Directory "
mailto:
Apache/2.4.54
Server at <a href="
<address>
--
</a> Port
Server at
Apache/2.4.54 (Unix)
ap_logio_add_bytes_out
ap_ident_lookup
--
access_compat_ap_satisfies
Apache
Apache/2.4
Apache/2
<VirtualHost
<Limit
@@ -0,0 +1,5 @@
[null] Script: '%s'
%s,%s
X-Powered-By: PHP/8.2.1
index pointer
PHP_VERSION
@@ -0,0 +1,5 @@
[null] Script: '%s'
%s,%s
X-Powered-By: PHP/8.2.1
index pointer
PHP_VERSION
@@ -0,0 +1,5 @@
[null] Script: '%s'
%s,%s
X-Powered-By: PHP/8.2.1
index pointer
PHP_VERSION
+113 -34
View File
@@ -2,6 +2,8 @@ package cataloger
import (
"fmt"
"math"
"sync"
"github.com/hashicorp/go-multierror"
"github.com/wagoodman/go-partybus"
@@ -23,6 +25,15 @@ type Monitor struct {
PackagesDiscovered progress.Monitorable // the number of packages discovered from all registered catalogers
}
// catalogResult provides the result of running a single cataloger against source
type catalogResult struct {
Packages []pkg.Package
Relationships []artifact.Relationship
// Discovered may sometimes be more than len(packages)
Discovered int64
Error error
}
// newMonitor creates a new Monitor object and publishes the object on the bus as a PackageCatalogerStarted event.
func newMonitor() (*progress.Manual, *progress.Manual) {
filesProcessed := progress.Manual{}
@@ -38,53 +49,121 @@ func newMonitor() (*progress.Manual, *progress.Manual) {
return &filesProcessed, &packagesDiscovered
}
func runCataloger(cataloger pkg.Cataloger, resolver source.FileResolver) (*catalogResult, error) {
catalogerResult := new(catalogResult)
// find packages from the underlying raw data
log.WithFields("cataloger", cataloger.Name()).Trace("cataloging started")
packages, relationships, err := cataloger.Catalog(resolver)
if err != nil {
log.WithFields("cataloger", cataloger.Name()).Warn("error while cataloging")
return catalogerResult, err
}
catalogedPackages := len(packages)
log.WithFields("cataloger", cataloger.Name()).Debugf("discovered %d packages", catalogedPackages)
catalogerResult.Discovered = int64(catalogedPackages)
for _, p := range packages {
// generate CPEs (note: this is excluded from package ID, so is safe to mutate)
// we might have binary classified CPE already with the package so we want to append here
p.CPEs = append(p.CPEs, cpe.Generate(p)...)
// if we were not able to identify the language we have an opportunity
// to try and get this value from the PURL. Worst case we assert that
// we could not identify the language at either stage and set UnknownLanguage
if p.Language == "" {
p.Language = pkg.LanguageFromPURL(p.PURL)
}
// create file-to-package relationships for files owned by the package
owningRelationships, err := packageFileOwnershipRelationships(p, resolver)
if err != nil {
log.WithFields("cataloger", cataloger.Name(), "package", p.Name, "error", err).Warnf("unable to create any package-file relationships")
} else {
catalogerResult.Relationships = append(catalogerResult.Relationships, owningRelationships...)
}
catalogerResult.Packages = append(catalogerResult.Packages, p)
}
catalogerResult.Relationships = append(catalogerResult.Relationships, relationships...)
log.WithFields("cataloger", cataloger.Name()).Trace("cataloging complete")
return catalogerResult, nil
}
// Catalog a given source (container image or filesystem) with the given catalogers, returning all discovered packages.
// In order to efficiently retrieve contents from a underlying container image the content fetch requests are
// done in bulk. Specifically, all files of interest are collected from each catalogers and accumulated into a single
// request.
func Catalog(resolver source.FileResolver, release *linux.Release, catalogers ...pkg.Cataloger) (*pkg.Catalog, []artifact.Relationship, error) {
//
//nolint:funlen
func Catalog(resolver source.FileResolver, release *linux.Release, parallelism int, catalogers ...pkg.Cataloger) (*pkg.Catalog, []artifact.Relationship, error) {
catalog := pkg.NewCatalog()
var allRelationships []artifact.Relationship
filesProcessed, packagesDiscovered := newMonitor()
// perform analysis, accumulating errors for each failed analysis
var errs error
for _, c := range catalogers {
// find packages from the underlying raw data
log.Debugf("cataloging with %q", c.Name())
packages, relationships, err := c.Catalog(resolver)
if err != nil {
errs = multierror.Append(errs, err)
nCatalogers := len(catalogers)
// we do not need more parallelism than there are `catalogers`.
parallelism = int(math.Min(float64(nCatalogers), math.Max(1.0, float64(parallelism))))
log.WithFields("parallelism", parallelism, "catalogers", nCatalogers).Debug("cataloging packages")
jobs := make(chan pkg.Cataloger, nCatalogers)
results := make(chan *catalogResult, nCatalogers)
discoveredPackages := make(chan int64, nCatalogers)
waitGroup := sync.WaitGroup{}
for i := 0; i < parallelism; i++ {
waitGroup.Add(1)
go func() {
defer waitGroup.Done()
// wait for / get the next cataloger job available.
for cataloger := range jobs {
result, err := runCataloger(cataloger, resolver)
// ensure we set the error to be aggregated
result.Error = err
discoveredPackages <- result.Discovered
results <- result
}
}()
}
// dynamically show updated discovered package status
go func() {
for discovered := range discoveredPackages {
packagesDiscovered.N += discovered
}
}()
// Enqueue the jobs
for _, cataloger := range catalogers {
jobs <- cataloger
}
close(jobs)
// Wait for the jobs to finish
waitGroup.Wait()
close(results)
close(discoveredPackages)
// collect the results
for result := range results {
if result.Error != nil {
errs = multierror.Append(errs, result.Error)
continue
}
catalogedPackages := len(packages)
log.Debugf("discovered %d packages", catalogedPackages)
packagesDiscovered.N += int64(catalogedPackages)
for _, p := range packages {
// generate CPEs (note: this is excluded from package ID, so is safe to mutate)
// we might have binary classified CPE already with the package so we want to append here
p.CPEs = append(p.CPEs, cpe.Generate(p)...)
// if we were not able to identify the language we have an opportunity
// to try and get this value from the PURL. Worst case we assert that
// we could not identify the language at either stage and set UnknownLanguage
if p.Language == "" {
p.Language = pkg.LanguageFromPURL(p.PURL)
}
// create file-to-package relationships for files owned by the package
owningRelationships, err := packageFileOwnershipRelationships(p, resolver)
if err != nil {
log.Warnf("unable to create any package-file relationships for package name=%q: %w", p.Name, err)
} else {
allRelationships = append(allRelationships, owningRelationships...)
}
for _, p := range result.Packages {
catalog.Add(p)
}
allRelationships = append(allRelationships, relationships...)
allRelationships = append(allRelationships, result.Relationships...)
}
allRelationships = append(allRelationships, pkg.NewRelationships(catalog)...)
+9
View File
@@ -17,6 +17,8 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/dart"
"github.com/anchore/syft/syft/pkg/cataloger/deb"
"github.com/anchore/syft/syft/pkg/cataloger/dotnet"
"github.com/anchore/syft/syft/pkg/cataloger/elixir"
"github.com/anchore/syft/syft/pkg/cataloger/erlang"
"github.com/anchore/syft/syft/pkg/cataloger/golang"
"github.com/anchore/syft/syft/pkg/cataloger/haskell"
"github.com/anchore/syft/syft/pkg/cataloger/java"
@@ -44,6 +46,7 @@ func ImageCatalogers(cfg Config) []pkg.Cataloger {
deb.NewDpkgdbCataloger(),
rpm.NewRpmDBCataloger(),
java.NewJavaCataloger(cfg.Java()),
java.NewNativeImageCataloger(),
apkdb.NewApkdbCataloger(),
golang.NewGoModuleBinaryCataloger(),
dotnet.NewDotnetDepsCataloger(),
@@ -67,6 +70,7 @@ func DirectoryCatalogers(cfg Config) []pkg.Cataloger {
rpm.NewFileCataloger(),
java.NewJavaCataloger(cfg.Java()),
java.NewJavaPomCataloger(),
java.NewNativeImageCataloger(),
apkdb.NewApkdbCataloger(),
golang.NewGoModuleBinaryCataloger(),
golang.NewGoModFileCataloger(),
@@ -79,6 +83,8 @@ func DirectoryCatalogers(cfg Config) []pkg.Cataloger {
haskell.NewHackageCataloger(),
sbom.NewSBOMCataloger(),
binary.NewCataloger(),
elixir.NewMixLockCataloger(),
erlang.NewRebarLockCataloger(),
}, cfg.Catalogers)
}
@@ -97,6 +103,7 @@ func AllCatalogers(cfg Config) []pkg.Cataloger {
rpm.NewFileCataloger(),
java.NewJavaCataloger(cfg.Java()),
java.NewJavaPomCataloger(),
java.NewNativeImageCataloger(),
apkdb.NewApkdbCataloger(),
golang.NewGoModuleBinaryCataloger(),
golang.NewGoModFileCataloger(),
@@ -112,6 +119,8 @@ func AllCatalogers(cfg Config) []pkg.Cataloger {
haskell.NewHackageCataloger(),
sbom.NewSBOMCataloger(),
binary.NewCataloger(),
elixir.NewMixLockCataloger(),
erlang.NewRebarLockCataloger(),
}, cfg.Catalogers)
}
+15 -1
View File
@@ -8,12 +8,17 @@ import (
"strings"
"github.com/facebookincubator/nvdtools/wfn"
"github.com/scylladb/go-set/strset"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/pkg"
)
// knownVendors contains vendor strings that are known to exist in
// the CPE database, so they will be preferred over other candidates:
var knownVendors = strset.New("apache")
func newCPE(product, vendor, version, targetSW string) *wfn.Attributes {
c := *(wfn.NewAttributesWithAny())
c.Part = "a"
@@ -120,7 +125,16 @@ func candidateVendors(p pkg.Package) []string {
// remove known mis
vendors.removeByValue(findVendorsToRemove(defaultCandidateRemovals, p.Type, p.Name)...)
return vendors.uniqueValues()
uniqueVendors := vendors.uniqueValues()
// if any known vendor was detected, pick that one.
for _, vendor := range uniqueVendors {
if knownVendors.Has(vendor) {
return []string{vendor}
}
}
return uniqueVendors
}
func candidateProducts(p pkg.Package) []string {
+1 -28
View File
@@ -279,36 +279,9 @@ func TestGeneratePackageCPEs(t *testing.T) {
},
},
expected: []string{
"cpe:2.3:a:apache-software-foundation:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache-software-foundation:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache-software-foundation:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache_software_foundation:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache_software_foundation:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:apache_software_foundation:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings-xml:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings-xml:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings-xml:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt-bindings:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf-rt:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings_xml:cxf-rt-bindings-xml:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings_xml:cxf:3.3.10:*:*:*:*:*:*:*",
"cpe:2.3:a:cxf_rt_bindings_xml:cxf_rt_bindings_xml:3.3.10:*:*:*:*:*:*:*",
},
},
{
@@ -838,7 +811,7 @@ func TestCandidateVendor(t *testing.T) {
Name: "log4j",
Type: pkg.JavaPkg,
},
expected: []string{"apache" /* <-- known good names | default guess --> */, "log4j"},
expected: []string{"apache"},
},
}
+5 -3
View File
@@ -5,13 +5,15 @@ import (
)
type Config struct {
Search SearchConfig
Catalogers []string
Search SearchConfig
Catalogers []string
Parallelism int
}
func DefaultConfig() Config {
return Config{
Search: DefaultSearchConfig(),
Search: DefaultSearchConfig(),
Parallelism: 1,
}
}
+16
View File
@@ -0,0 +1,16 @@
/*
Package elixir provides a concrete Cataloger implementation for elixir specific package manger files.
*/
package elixir
import (
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
const catalogerName = "elixir-mix-lock-cataloger"
// NewMixLockCataloger returns parses mix.lock files and returns packages
func NewMixLockCataloger() *generic.Cataloger {
return generic.NewCataloger(catalogerName).
WithParserByGlobs(parseMixLock, "**/mix.lock")
}
+37
View File
@@ -0,0 +1,37 @@
package elixir
import (
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
func newPackage(d pkg.MixLockMetadata, locations ...source.Location) pkg.Package {
p := pkg.Package{
Name: d.Name,
Version: d.Version,
Language: pkg.Elixir,
Locations: source.NewLocationSet(locations...),
PURL: packageURL(d),
Type: pkg.HexPkg,
MetadataType: pkg.MixLockMetadataType,
Metadata: d,
}
p.SetID()
return p
}
func packageURL(m pkg.MixLockMetadata) string {
var qualifiers packageurl.Qualifiers
return packageurl.NewPackageURL(
packageurl.TypeHex,
"",
m.Name,
m.Version,
qualifiers,
"",
).ToString()
}
@@ -0,0 +1,53 @@
package elixir
import (
"bufio"
"errors"
"fmt"
"io"
"regexp"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
"github.com/anchore/syft/syft/source"
)
// integrity check
var _ generic.Parser = parseMixLock
var mixLockDelimiter = regexp.MustCompile(`[%{}\n" ,:]+`)
// parseMixLock parses a mix.lock and returns the discovered Elixir packages.
func parseMixLock(_ source.FileResolver, _ *generic.Environment, reader source.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
r := bufio.NewReader(reader)
var packages []pkg.Package
for {
line, err := r.ReadString('\n')
switch {
case errors.Is(io.EOF, err):
return packages, nil, nil
case err != nil:
return nil, nil, fmt.Errorf("failed to parse mix.lock file: %w", err)
}
tokens := mixLockDelimiter.Split(line, -1)
if len(tokens) < 6 {
continue
}
name, version, hash, hashExt := tokens[1], tokens[4], tokens[5], tokens[len(tokens)-2]
if name == "" {
log.WithFields("path", reader.RealPath).Debug("skipping empty package name from mix.lock file")
continue
}
packages = append(packages, newPackage(pkg.MixLockMetadata{
Name: name,
Version: version,
PkgHash: hash,
PkgHashExt: hashExt,
}))
}
}
@@ -0,0 +1,231 @@
package elixir
import (
"testing"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
func TestParseMixLock(t *testing.T) {
expected := []pkg.Package{
{
Name: "castore",
Version: "0.1.17",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/castore@0.1.17",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "castore",
Version: "0.1.17",
PkgHash: "ba672681de4e51ed8ec1f74ed624d104c0db72742ea1a5e74edbc770c815182f",
PkgHashExt: "d9844227ed52d26e7519224525cb6868650c272d4a3d327ce3ca5570c12163f9",
},
},
{
Name: "connection",
Version: "1.1.0",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/connection@1.1.0",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "connection",
Version: "1.1.0",
PkgHash: "ff2a49c4b75b6fb3e674bfc5536451607270aac754ffd1bdfe175abe4a6d7a68",
PkgHashExt: "722c1eb0a418fbe91ba7bd59a47e28008a189d47e37e0e7bb85585a016b2869c",
},
},
{
Name: "cowboy",
Version: "2.9.0",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/cowboy@2.9.0",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "cowboy",
Version: "2.9.0",
PkgHash: "865dd8b6607e14cf03282e10e934023a1bd8be6f6bacf921a7e2a96d800cd452",
PkgHashExt: "2c729f934b4e1aa149aff882f57c6372c15399a20d54f65c8d67bef583021bde",
},
},
{
Name: "cowboy_telemetry",
Version: "0.4.0",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/cowboy_telemetry@0.4.0",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "cowboy_telemetry",
Version: "0.4.0",
PkgHash: "f239f68b588efa7707abce16a84d0d2acf3a0f50571f8bb7f56a15865aae820c",
PkgHashExt: "7d98bac1ee4565d31b62d59f8823dfd8356a169e7fcbb83831b8a5397404c9de",
},
},
{
Name: "cowlib",
Version: "2.11.0",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/cowlib@2.11.0",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "cowlib",
Version: "2.11.0",
PkgHash: "0b9ff9c346629256c42ebe1eeb769a83c6cb771a6ee5960bd110ab0b9b872063",
PkgHashExt: "2b3e9da0b21c4565751a6d4901c20d1b4cc25cbb7fd50d91d2ab6dd287bc86a9",
},
},
{
Name: "db_connection",
Version: "2.4.2",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/db_connection@2.4.2",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "db_connection",
Version: "2.4.2",
PkgHash: "f92e79aff2375299a16bcb069a14ee8615c3414863a6fef93156aee8e86c2ff3",
PkgHashExt: "4fe53ca91b99f55ea249693a0229356a08f4d1a7931d8ffa79289b145fe83668",
},
},
{
Name: "decimal",
Version: "2.0.0",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/decimal@2.0.0",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "decimal",
Version: "2.0.0",
PkgHash: "a78296e617b0f5dd4c6caf57c714431347912ffb1d0842e998e9792b5642d697",
PkgHashExt: "34666e9c55dea81013e77d9d87370fe6cb6291d1ef32f46a1600230b1d44f577",
},
},
{
Name: "earmark_parser",
Version: "1.4.25",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/earmark_parser@1.4.25",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "earmark_parser",
Version: "1.4.25",
PkgHash: "2024618731c55ebfcc5439d756852ec4e85978a39d0d58593763924d9a15916f",
PkgHashExt: "56749c5e1c59447f7b7a23ddb235e4b3defe276afc220a6227237f3efe83f51e",
},
},
{
Name: "ecto",
Version: "3.8.1",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/ecto@3.8.1",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "ecto",
Version: "3.8.1",
PkgHash: "35e0bd8c8eb772e14a5191a538cd079706ecb45164ea08a7523b4fc69ab70f56",
PkgHashExt: "f1b68f8d5fe3ab89e24f57c03db5b5d0aed3602077972098b3a6006a1be4b69b",
},
},
{
Name: "ecto_sql",
Version: "3.8.1",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/ecto_sql@3.8.1",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "ecto_sql",
Version: "3.8.1",
PkgHash: "1acaaba32ca0551fd19e492fc7c80414e72fc1a7140fc9395aaa53c2e8629798",
PkgHashExt: "ba7fc75882edce6f2ceca047315d5db27ead773cafea47f1724e35f1e7964525",
},
},
{
Name: "esbuild",
Version: "0.5.0",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/esbuild@0.5.0",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "esbuild",
Version: "0.5.0",
PkgHash: "d5bb08ff049d7880ee3609ed5c4b864bd2f46445ea40b16b4acead724fb4c4a3",
PkgHashExt: "f183a0b332d963c4cfaf585477695ea59eef9a6f2204fdd0efa00e099694ffe5",
},
},
{
Name: "ex_doc",
Version: "0.28.4",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/ex_doc@0.28.4",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "ex_doc",
Version: "0.28.4",
PkgHash: "001a0ea6beac2f810f1abc3dbf4b123e9593eaa5f00dd13ded024eae7c523298",
PkgHashExt: "bf85d003dd34911d89c8ddb8bda1a958af3471a274a4c2150a9c01c78ac3f8ed",
},
},
{
Name: "gettext",
Version: "0.19.1",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/gettext@0.19.1",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "gettext",
Version: "0.19.1",
PkgHash: "564953fd21f29358e68b91634799d9d26989f8d039d7512622efb3c3b1c97892",
PkgHashExt: "10c656c0912b8299adba9b061c06947511e3f109ab0d18b44a866a4498e77222",
},
},
{
Name: "hpax",
Version: "0.1.1",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/hpax@0.1.1",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "hpax",
Version: "0.1.1",
PkgHash: "2396c313683ada39e98c20a75a82911592b47e5c24391363343bde74f82396ca",
PkgHashExt: "0ae7d5a0b04a8a60caf7a39fcf3ec476f35cc2cc16c05abea730d3ce6ac6c826",
},
},
{
Name: "jason",
Version: "1.3.0",
Language: pkg.Elixir,
Type: pkg.HexPkg,
PURL: "pkg:hex/jason@1.3.0",
MetadataType: pkg.MixLockMetadataType,
Metadata: pkg.MixLockMetadata{
Name: "jason",
Version: "1.3.0",
PkgHash: "fa6b82a934feb176263ad2df0dbd91bf633d4a46ebfdffea0c8ae82953714946",
PkgHashExt: "53fc1f51255390e0ec7e50f9cb41e751c260d065dcba2bf0d08dc51a4002c2ac",
},
},
}
fixture := "test-fixtures/mix.lock"
// TODO: relationships are not under test
var expectedRelationships []artifact.Relationship
pkgtest.TestFileParser(t, fixture, parseMixLock, expected, expectedRelationships)
}
@@ -0,0 +1,17 @@
%{
"castore": {:hex, :castore, "0.1.17", "ba672681de4e51ed8ec1f74ed624d104c0db72742ea1a5e74edbc770c815182f", [:mix], [], "hexpm", "d9844227ed52d26e7519224525cb6868650c272d4a3d327ce3ca5570c12163f9"},
"connection": {:hex, :connection, "1.1.0", "ff2a49c4b75b6fb3e674bfc5536451607270aac754ffd1bdfe175abe4a6d7a68", [:mix], [], "hexpm", "722c1eb0a418fbe91ba7bd59a47e28008a189d47e37e0e7bb85585a016b2869c"},
"cowboy": {:hex, :cowboy, "2.9.0", "865dd8b6607e14cf03282e10e934023a1bd8be6f6bacf921a7e2a96d800cd452", [:make, :rebar3], [{:cowlib, "2.11.0", [hex: :cowlib, repo: "hexpm", optional: false]}, {:ranch, "1.8.0", [hex: :ranch, repo: "hexpm", optional: false]}], "hexpm", "2c729f934b4e1aa149aff882f57c6372c15399a20d54f65c8d67bef583021bde"},
"cowboy_telemetry": {:hex, :cowboy_telemetry, "0.4.0", "f239f68b588efa7707abce16a84d0d2acf3a0f50571f8bb7f56a15865aae820c", [:rebar3], [{:cowboy, "~> 2.7", [hex: :cowboy, repo: "hexpm", optional: false]}, {:telemetry, "~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "7d98bac1ee4565d31b62d59f8823dfd8356a169e7fcbb83831b8a5397404c9de"},
"cowlib": {:hex, :cowlib, "2.11.0", "0b9ff9c346629256c42ebe1eeb769a83c6cb771a6ee5960bd110ab0b9b872063", [:make, :rebar3], [], "hexpm", "2b3e9da0b21c4565751a6d4901c20d1b4cc25cbb7fd50d91d2ab6dd287bc86a9"},
"db_connection": {:hex, :db_connection, "2.4.2", "f92e79aff2375299a16bcb069a14ee8615c3414863a6fef93156aee8e86c2ff3", [:mix], [{:connection, "~> 1.0", [hex: :connection, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "4fe53ca91b99f55ea249693a0229356a08f4d1a7931d8ffa79289b145fe83668"},
"decimal": {:hex, :decimal, "2.0.0", "a78296e617b0f5dd4c6caf57c714431347912ffb1d0842e998e9792b5642d697", [:mix], [], "hexpm", "34666e9c55dea81013e77d9d87370fe6cb6291d1ef32f46a1600230b1d44f577"},
"earmark_parser": {:hex, :earmark_parser, "1.4.25", "2024618731c55ebfcc5439d756852ec4e85978a39d0d58593763924d9a15916f", [:mix], [], "hexpm", "56749c5e1c59447f7b7a23ddb235e4b3defe276afc220a6227237f3efe83f51e"},
"ecto": {:hex, :ecto, "3.8.1", "35e0bd8c8eb772e14a5191a538cd079706ecb45164ea08a7523b4fc69ab70f56", [:mix], [{:decimal, "~> 1.6 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "f1b68f8d5fe3ab89e24f57c03db5b5d0aed3602077972098b3a6006a1be4b69b"},
"ecto_sql": {:hex, :ecto_sql, "3.8.1", "1acaaba32ca0551fd19e492fc7c80414e72fc1a7140fc9395aaa53c2e8629798", [:mix], [{:db_connection, "~> 2.5 or ~> 2.4.1", [hex: :db_connection, repo: "hexpm", optional: false]}, {:ecto, "~> 3.8.1", [hex: :ecto, repo: "hexpm", optional: false]}, {:myxql, "~> 0.6.0", [hex: :myxql, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.15.0 or ~> 0.16.0 or ~> 1.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:tds, "~> 2.1.1 or ~> 2.2", [hex: :tds, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4.0 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "ba7fc75882edce6f2ceca047315d5db27ead773cafea47f1724e35f1e7964525"},
"esbuild": {:hex, :esbuild, "0.5.0", "d5bb08ff049d7880ee3609ed5c4b864bd2f46445ea40b16b4acead724fb4c4a3", [:mix], [{:castore, ">= 0.0.0", [hex: :castore, repo: "hexpm", optional: false]}], "hexpm", "f183a0b332d963c4cfaf585477695ea59eef9a6f2204fdd0efa00e099694ffe5"},
"ex_doc": {:hex, :ex_doc, "0.28.4", "001a0ea6beac2f810f1abc3dbf4b123e9593eaa5f00dd13ded024eae7c523298", [:mix], [{:earmark_parser, "~> 1.4.19", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.14", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1", [hex: :makeup_erlang, repo: "hexpm", optional: false]}], "hexpm", "bf85d003dd34911d89c8ddb8bda1a958af3471a274a4c2150a9c01c78ac3f8ed"},
"gettext": {:hex, :gettext, "0.19.1", "564953fd21f29358e68b91634799d9d26989f8d039d7512622efb3c3b1c97892", [:mix], [], "hexpm", "10c656c0912b8299adba9b061c06947511e3f109ab0d18b44a866a4498e77222"},
"hpax": {:hex, :hpax, "0.1.1", "2396c313683ada39e98c20a75a82911592b47e5c24391363343bde74f82396ca", [:mix], [], "hexpm", "0ae7d5a0b04a8a60caf7a39fcf3ec476f35cc2cc16c05abea730d3ce6ac6c826"},
"jason": {:hex, :jason, "1.3.0", "fa6b82a934feb176263ad2df0dbd91bf633d4a46ebfdffea0c8ae82953714946", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "53fc1f51255390e0ec7e50f9cb41e751c260d065dcba2bf0d08dc51a4002c2ac"},
}
+16
View File
@@ -0,0 +1,16 @@
/*
Package erlang provides a concrete Cataloger implementation for erlang specific package manger files.
*/
package erlang
import (
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
const catalogerName = "erlang-rebar-lock-cataloger"
// NewRebarLockCataloger returns parses rebar.lock files and returns packages.
func NewRebarLockCataloger() *generic.Cataloger {
return generic.NewCataloger(catalogerName).
WithParserByGlobs(parseRebarLock, "**/rebar.lock")
}
+37
View File
@@ -0,0 +1,37 @@
package erlang
import (
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source"
)
func newPackage(d pkg.RebarLockMetadata, locations ...source.Location) pkg.Package {
p := pkg.Package{
Name: d.Name,
Version: d.Version,
Language: pkg.Erlang,
Locations: source.NewLocationSet(locations...),
PURL: packageURL(d),
Type: pkg.HexPkg,
MetadataType: pkg.RebarLockMetadataType,
Metadata: d,
}
p.SetID()
return p
}
func packageURL(m pkg.RebarLockMetadata) string {
var qualifiers packageurl.Qualifiers
return packageurl.NewPackageURL(
packageurl.TypeHex,
"",
m.Name,
m.Version,
qualifiers,
"",
).ToString()
}
@@ -0,0 +1,80 @@
package erlang
import (
"bufio"
"errors"
"fmt"
"io"
"regexp"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
"github.com/anchore/syft/syft/source"
)
// integrity check
var _ generic.Parser = parseRebarLock
var rebarLockDelimiter = regexp.MustCompile(`[\[{<">},: \]\n]+`)
// parseMixLock parses a mix.lock and returns the discovered Elixir packages.
func parseRebarLock(_ source.FileResolver, _ *generic.Environment, reader source.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
r := bufio.NewReader(reader)
pkgMap := make(map[string]*pkg.Package)
var names []string
loop:
for {
line, err := r.ReadString('\n')
switch {
case errors.Is(io.EOF, err):
break loop
case err != nil:
// TODO: return partial result and warn
return nil, nil, fmt.Errorf("failed to parse rebar.lock file: %w", err)
}
tokens := rebarLockDelimiter.Split(line, -1)
if len(tokens) < 4 {
continue
}
if len(tokens) < 5 {
name, hash := tokens[1], tokens[2]
sourcePkg := pkgMap[name]
metadata := sourcePkg.Metadata.(pkg.RebarLockMetadata)
if metadata.PkgHash == "" {
metadata.PkgHash = hash
} else {
metadata.PkgHashExt = hash
}
sourcePkg.Metadata = metadata
continue
}
name, version := tokens[1], tokens[4]
sourcePkg := pkg.Package{
Name: name,
Version: version,
Language: pkg.Erlang,
Type: pkg.HexPkg,
MetadataType: pkg.RebarLockMetadataType,
}
p := newPackage(pkg.RebarLockMetadata{
Name: name,
Version: version,
})
names = append(names, name)
pkgMap[sourcePkg.Name] = &p
}
var packages []pkg.Package
for _, name := range names {
p := pkgMap[name]
p.SetID()
packages = append(packages, *p)
}
return packages, nil, nil
}
@@ -0,0 +1,119 @@
package erlang
import (
"testing"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
func TestParseRebarLock(t *testing.T) {
expected := []pkg.Package{
{
Name: "certifi",
Version: "2.9.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/certifi@2.9.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "certifi",
Version: "2.9.0",
PkgHash: "6F2A475689DD47F19FB74334859D460A2DC4E3252A3324BD2111B8F0429E7E21",
PkgHashExt: "266DA46BDB06D6C6D35FDE799BCB28D36D985D424AD7C08B5BB48F5B5CDD4641",
},
},
{
Name: "idna",
Version: "6.1.1",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/idna@6.1.1",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "idna",
Version: "6.1.1",
PkgHash: "8A63070E9F7D0C62EB9D9FCB360A7DE382448200FBBD1B106CC96D3D8099DF8D",
PkgHashExt: "92376EB7894412ED19AC475E4A86F7B413C1B9FBB5BD16DCCD57934157944CEA",
},
},
{
Name: "metrics",
Version: "1.0.1",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/metrics@1.0.1",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "metrics",
Version: "1.0.1",
PkgHash: "25F094DEA2CDA98213CECC3AEFF09E940299D950904393B2A29D191C346A8486",
PkgHashExt: "69B09ADDDC4F74A40716AE54D140F93BEB0FB8978D8636EADED0C31B6F099F16",
},
},
{
Name: "mimerl",
Version: "1.2.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/mimerl@1.2.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "mimerl",
Version: "1.2.0",
PkgHash: "67E2D3F571088D5CFD3E550C383094B47159F3EEE8FFA08E64106CDF5E981BE3",
PkgHashExt: "F278585650AA581986264638EBF698F8BB19DF297F66AD91B18910DFC6E19323",
},
},
{
Name: "parse_trans",
Version: "3.3.1",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/parse_trans@3.3.1",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "parse_trans",
Version: "3.3.1",
PkgHash: "16328AB840CC09919BD10DAB29E431DA3AF9E9E7E7E6F0089DD5A2D2820011D8",
PkgHashExt: "07CD9577885F56362D414E8C4C4E6BDF10D43A8767ABB92D24CBE8B24C54888B",
},
},
{
Name: "ssl_verify_fun",
Version: "1.1.6",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/ssl_verify_fun@1.1.6",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "ssl_verify_fun",
Version: "1.1.6",
PkgHash: "CF344F5692C82D2CD7554F5EC8FD961548D4FD09E7D22F5B62482E5AEAEBD4B0",
PkgHashExt: "BDB0D2471F453C88FF3908E7686F86F9BE327D065CC1EC16FA4540197EA04680",
},
},
{
Name: "unicode_util_compat",
Version: "0.7.0",
Language: pkg.Erlang,
Type: pkg.HexPkg,
PURL: "pkg:hex/unicode_util_compat@0.7.0",
MetadataType: pkg.RebarLockMetadataType,
Metadata: pkg.RebarLockMetadata{
Name: "unicode_util_compat",
Version: "0.7.0",
PkgHash: "BC84380C9AB48177092F43AC89E4DFA2C6D62B40B8BD132B1059ECC7232F9A78",
PkgHashExt: "25EEE6D67DF61960CF6A794239566599B09E17E668D3700247BC498638152521",
},
},
}
fixture := "test-fixtures/rebar.lock"
// TODO: relationships are not under test
var expectedRelationships []artifact.Relationship
pkgtest.TestFileParser(t, fixture, parseRebarLock, expected, expectedRelationships)
}
@@ -0,0 +1,26 @@
{"1.2.0",
[{<<"certifi">>,{pkg,<<"certifi">>,<<"2.9.0">>},0},
{<<"idna">>,{pkg,<<"idna">>,<<"6.1.1">>},0},
{<<"metrics">>,{pkg,<<"metrics">>,<<"1.0.1">>},0},
{<<"mimerl">>,{pkg,<<"mimerl">>,<<"1.2.0">>},0},
{<<"parse_trans">>,{pkg,<<"parse_trans">>,<<"3.3.1">>},0},
{<<"ssl_verify_fun">>,{pkg,<<"ssl_verify_fun">>,<<"1.1.6">>},0},
{<<"unicode_util_compat">>,{pkg,<<"unicode_util_compat">>,<<"0.7.0">>},0}]}.
[
{pkg_hash,[
{<<"certifi">>, <<"6F2A475689DD47F19FB74334859D460A2DC4E3252A3324BD2111B8F0429E7E21">>},
{<<"idna">>, <<"8A63070E9F7D0C62EB9D9FCB360A7DE382448200FBBD1B106CC96D3D8099DF8D">>},
{<<"metrics">>, <<"25F094DEA2CDA98213CECC3AEFF09E940299D950904393B2A29D191C346A8486">>},
{<<"mimerl">>, <<"67E2D3F571088D5CFD3E550C383094B47159F3EEE8FFA08E64106CDF5E981BE3">>},
{<<"parse_trans">>, <<"16328AB840CC09919BD10DAB29E431DA3AF9E9E7E7E6F0089DD5A2D2820011D8">>},
{<<"ssl_verify_fun">>, <<"CF344F5692C82D2CD7554F5EC8FD961548D4FD09E7D22F5B62482E5AEAEBD4B0">>},
{<<"unicode_util_compat">>, <<"BC84380C9AB48177092F43AC89E4DFA2C6D62B40B8BD132B1059ECC7232F9A78">>}]},
{pkg_hash_ext,[
{<<"certifi">>, <<"266DA46BDB06D6C6D35FDE799BCB28D36D985D424AD7C08B5BB48F5B5CDD4641">>},
{<<"idna">>, <<"92376EB7894412ED19AC475E4A86F7B413C1B9FBB5BD16DCCD57934157944CEA">>},
{<<"metrics">>, <<"69B09ADDDC4F74A40716AE54D140F93BEB0FB8978D8636EADED0C31B6F099F16">>},
{<<"mimerl">>, <<"F278585650AA581986264638EBF698F8BB19DF297F66AD91B18910DFC6E19323">>},
{<<"parse_trans">>, <<"07CD9577885F56362D414E8C4C4E6BDF10D43A8767ABB92D24CBE8B24C54888B">>},
{<<"ssl_verify_fun">>, <<"BDB0D2471F453C88FF3908E7686F86F9BE327D065CC1EC16FA4540197EA04680">>},
{<<"unicode_util_compat">>, <<"25EEE6D67DF61960CF6A794239566599B09E17E668D3700247BC498638152521">>}]}
].
+1 -1
View File
@@ -1,5 +1,5 @@
/*
Package java provides a concrete Cataloger implementation for Java archives (jar, war, ear, par, sar, jpi, hpi formats).
Package java provides a concrete Cataloger implementation for Java archives (jar, war, ear, par, sar, jpi, hpi, and native-image formats).
*/
package java
@@ -0,0 +1,557 @@
package java
import (
"bytes"
"compress/gzip"
"debug/elf"
"debug/macho"
"debug/pe"
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"io"
"unsafe"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/unionreader"
"github.com/anchore/syft/syft/source"
)
type nativeImageCycloneDX struct {
BomFormat string `json:"bomFormat"`
SpecVersion string `json:"specVersion"`
Version int `json:"version"`
Components []nativeImageComponent `json:"components"`
}
type nativeImageComponent struct {
Type string `json:"type"`
Group string `json:"group"`
Name string `json:"name"`
Version string `json:"version"`
Properties []nativeImageCPE `json:"properties"`
}
type nativeImageCPE struct {
Name string `json:"name"`
Value string `json:"value"`
}
type nativeImage interface {
fetchPkgs() ([]pkg.Package, error)
}
type nativeImageElf struct {
file *elf.File
}
type nativeImageMachO struct {
file *macho.File
}
type exportTypesPE struct {
functionPointer uint32
namePointer uint32
headerAttribute uint32
}
type exportPrefixPE struct {
characteristics uint32
timeDateStamp uint32
majorVersion uint16
minorVersion uint16
name uint32
base uint32
}
type exportContentPE struct {
// Directory Entry Contents for finding SBOM symbols
numberOfFunctions uint32
numberOfNames uint32
addressOfFunctions uint32
addressOfNames uint32
// Locations of SBOM symbols in the .data section
addressOfSbom uint32
addressOfSbomLength uint32
addressOfSvmVersion uint32
}
// A nativeImagePE must maintain the underlying reader to fetch information unavailable in the Golang API.
type nativeImagePE struct {
file *pe.File
reader io.ReaderAt
exportSymbols pe.DataDirectory
exports []byte
t exportTypesPE
header exportPrefixPE
}
type NativeImageCataloger struct{}
const nativeImageCatalogerName = "graalvm-native-image-cataloger"
const nativeImageSbomSymbol = "sbom"
const nativeImageSbomLengthSymbol = "sbom_length"
const nativeImageSbomVersionSymbol = "__svm_version_info"
const nativeImageMissingSymbolsError = "one or more symbols are missing from the native image executable"
const nativeImageInvalidIndexError = "parsing the executable file generated an invalid index"
const nativeImageMissingExportedDataDirectoryError = "exported data directory is missing"
// newNativeImageCataloger returns a new Native Image cataloger object.
func NewNativeImageCataloger() *NativeImageCataloger {
return &NativeImageCataloger{}
}
// Name returns a string that uniquely describes a native image cataloger
func (c *NativeImageCataloger) Name() string {
return nativeImageCatalogerName
}
// getPackage returns the package given within a NativeImageComponent.
func getPackage(component nativeImageComponent) pkg.Package {
var cpes []cpe.CPE
for _, property := range component.Properties {
cpe, err := cpe.New(property.Value)
if err != nil {
log.Debugf("native-image cataloger: could not parse CPE: %v.", err)
continue
}
cpes = append(cpes, cpe)
}
return pkg.Package{
Name: component.Name,
Version: component.Version,
Language: pkg.Java,
Type: pkg.GraalVMNativeImagePkg,
MetadataType: pkg.JavaMetadataType,
FoundBy: nativeImageCatalogerName,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: component.Group,
},
},
CPEs: cpes,
}
}
// decompressSbom returns the packages given within a native image executable's SBOM.
func decompressSbom(databuf []byte, sbomStart uint64, lengthStart uint64) ([]pkg.Package, error) {
var pkgs []pkg.Package
lengthEnd := lengthStart + 8
buflen := len(databuf)
if lengthEnd > uint64(buflen) {
return nil, errors.New("the sbom_length symbol overflows the binary")
}
length := databuf[lengthStart:lengthEnd]
p := bytes.NewBuffer(length)
var storedLength uint64
err := binary.Read(p, binary.LittleEndian, &storedLength)
if err != nil {
log.Debugf("native-image-cataloger: could not read from binary file.")
return nil, err
}
log.Tracef("native-image cataloger: found SBOM of length %d.", storedLength)
sbomEnd := sbomStart + storedLength
if sbomEnd > uint64(buflen) {
return nil, errors.New("the sbom symbol overflows the binary")
}
sbomCompressed := databuf[sbomStart:sbomEnd]
p = bytes.NewBuffer(sbomCompressed)
gzreader, err := gzip.NewReader(p)
if err != nil {
log.Debugf("native-image cataloger: could not decompress the SBOM.")
return nil, err
}
output, err := io.ReadAll(gzreader)
if err != nil {
log.Debugf("native-image cataloger: could not read the decompressed SBOM.")
return nil, err
}
var sbomContent nativeImageCycloneDX
err = json.Unmarshal(output, &sbomContent)
if err != nil {
log.Debugf("native-image cataloger: could not unmarshal JSON.")
return nil, err
}
for _, component := range sbomContent.Components {
p := getPackage(component)
pkgs = append(pkgs, p)
}
return pkgs, nil
}
// fileError logs an error message when an executable cannot be read.
func fileError(filename string, err error) (nativeImage, error) {
// We could not read the file as a binary for the desired platform, but it may still be a native-image executable.
log.Debugf("native-image cataloger: unable to read executable (file=%q): %v.", filename, err)
return nil, err
}
// newElf reads a Native Image from an ELF executable.
func newElf(filename string, r io.ReaderAt) (nativeImage, error) {
// First attempt to read an ELF file.
bi, err := elf.NewFile(r)
// The reader does not refer to an ELF file.
if err != nil {
return fileError(filename, err)
}
return nativeImageElf{
file: bi,
}, nil
}
// newMachO reads a Native Image from a Mach O executable.
func newMachO(filename string, r io.ReaderAt) (nativeImage, error) {
// First attempt to read an ELF file.
bi, err := macho.NewFile(r)
// The reader does not refer to an MachO file.
if err != nil {
return fileError(filename, err)
}
return nativeImageMachO{
file: bi,
}, nil
}
// newPE reads a Native Image from a Portable Executable file.
func newPE(filename string, r io.ReaderAt) (nativeImage, error) {
// First attempt to read an ELF file.
bi, err := pe.NewFile(r)
// The reader does not refer to an MachO file.
if err != nil {
return fileError(filename, err)
}
var exportSymbolsDataDirectory pe.DataDirectory
switch h := bi.OptionalHeader.(type) {
case *pe.OptionalHeader32:
exportSymbolsDataDirectory = h.DataDirectory[0]
case *pe.OptionalHeader64:
exportSymbolsDataDirectory = h.DataDirectory[0]
default:
return nil, fmt.Errorf("unable to get exportSymbolsDataDirectory from binary: %s", filename)
}
// If we have no exported symbols it is not a Native Image
if exportSymbolsDataDirectory.Size == 0 {
return fileError(filename, errors.New(nativeImageMissingExportedDataDirectoryError))
}
exportSymbolsOffset := uint64(exportSymbolsDataDirectory.VirtualAddress)
exports := make([]byte, exportSymbolsDataDirectory.Size)
_, err = r.ReadAt(exports, int64(exportSymbolsOffset))
if err != nil {
log.Debugf("native-image cataloger: could not read the exported symbols data directory: %v.", err)
return fileError(filename, err)
}
return nativeImagePE{
file: bi,
reader: r,
exportSymbols: exportSymbolsDataDirectory,
exports: exports,
t: exportTypesPE{
functionPointer: 0,
namePointer: 0,
headerAttribute: 0,
},
header: exportPrefixPE{
characteristics: 0,
timeDateStamp: 0,
majorVersion: 0,
minorVersion: 0,
name: 0,
base: 0,
},
}, nil
}
// fetchPkgs obtains the packages given in the binary.
func (ni nativeImageElf) fetchPkgs() ([]pkg.Package, error) {
bi := ni.file
var sbom elf.Symbol
var sbomLength elf.Symbol
var svmVersion elf.Symbol
si, err := bi.Symbols()
if err != nil {
log.Debugf("native-image cataloger: no symbols found.")
return nil, err
}
for _, s := range si {
switch s.Name {
case nativeImageSbomSymbol:
sbom = s
case nativeImageSbomLengthSymbol:
sbomLength = s
case nativeImageSbomVersionSymbol:
svmVersion = s
}
}
if sbom.Value == 0 || sbomLength.Value == 0 || svmVersion.Value == 0 {
log.Debugf("native-image cataloger: %v", nativeImageMissingSymbolsError)
return nil, errors.New(nativeImageMissingSymbolsError)
}
dataSection := bi.Section(".data")
if dataSection == nil {
log.Debugf("native-image cataloger: .data section missing from ELF file.")
return nil, err
}
dataSectionBase := dataSection.SectionHeader.Addr
data, err := dataSection.Data()
if err != nil {
log.Debugf("native-image cataloger: cannot read the .data section.")
return nil, err
}
sbomLocation := sbom.Value - dataSectionBase
lengthLocation := sbomLength.Value - dataSectionBase
return decompressSbom(data, sbomLocation, lengthLocation)
}
// fetchPkgs obtains the packages from a Native Image given as a Mach O file.
func (ni nativeImageMachO) fetchPkgs() ([]pkg.Package, error) {
var sbom macho.Symbol
var sbomLength macho.Symbol
var svmVersion macho.Symbol
bi := ni.file
for _, s := range bi.Symtab.Syms {
switch s.Name {
case "_" + nativeImageSbomSymbol:
sbom = s
case "_" + nativeImageSbomLengthSymbol:
sbomLength = s
case "_" + nativeImageSbomVersionSymbol:
svmVersion = s
}
}
if sbom.Value == 0 || sbomLength.Value == 0 || svmVersion.Value == 0 {
log.Debugf("native-image cataloger: %v.", nativeImageMissingSymbolsError)
return nil, errors.New(nativeImageMissingSymbolsError)
}
dataSegment := bi.Segment("__DATA")
if dataSegment == nil {
return nil, nil
}
databuf, err := dataSegment.Data()
if err != nil {
log.Debugf("native-image cataloger: cannot obtain buffer from data segment.")
return nil, nil
}
sbomLocation := sbom.Value - dataSegment.Addr
lengthLocation := sbomLength.Value - dataSegment.Addr
return decompressSbom(databuf, sbomLocation, lengthLocation)
}
// fetchExportAttribute obtains an attribute from the exported symbols directory entry.
func (ni nativeImagePE) fetchExportAttribute(i int) (uint32, error) {
var attribute uint32
n := len(ni.exports)
j := int(unsafe.Sizeof(ni.header)) + i*int(unsafe.Sizeof(ni.t.headerAttribute))
if j+4 >= n {
log.Debugf("native-image cataloger: invalid index to export directory entry attribute: %v.", j)
return uint32(0), errors.New(nativeImageInvalidIndexError)
}
p := bytes.NewBuffer(ni.exports[j : j+4])
err := binary.Read(p, binary.LittleEndian, &attribute)
if err != nil {
log.Debugf("native-image cataloger: error fetching export directory entry attribute: %v.", err)
return uint32(0), err
}
return attribute, nil
}
// fetchExportFunctionPointer obtains a function pointer from the exported symbols directory entry.
func (ni nativeImagePE) fetchExportFunctionPointer(functionsBase uint32, i uint32) (uint32, error) {
var pointer uint32
n := uint32(len(ni.exports))
sz := uint32(unsafe.Sizeof(ni.t.functionPointer))
j := functionsBase + i*sz
if j+sz >= n {
log.Debugf("native-image cataloger: invalid index to exported function: %v.", j)
return uint32(0), errors.New(nativeImageInvalidIndexError)
}
p := bytes.NewBuffer(ni.exports[j : j+sz])
err := binary.Read(p, binary.LittleEndian, &pointer)
if err != nil {
log.Debugf("native-image cataloger: error fetching exported function: %v.", err)
return uint32(0), err
}
return pointer, nil
}
// fetchExportContent obtains the content of the export directory entry relevant to the SBOM.
func (ni nativeImagePE) fetchExportContent() (*exportContentPE, error) {
content := new(exportContentPE)
var err error
content.numberOfFunctions, err = ni.fetchExportAttribute(0)
if err != nil {
log.Debugf("native-image cataloger: could not find the number of exported functions attribute: %v", err)
return nil, err
}
content.numberOfNames, err = ni.fetchExportAttribute(1)
if err != nil {
log.Debugf("native-image cataloger: could not find the number of exported names attribute: %v", err)
return nil, err
}
content.addressOfFunctions, err = ni.fetchExportAttribute(2)
if err != nil {
log.Debugf("native-image cataloger: could not find the exported functions attribute: %v", err)
return nil, err
}
content.addressOfNames, err = ni.fetchExportAttribute(3)
if err != nil {
log.Debugf("native-image cataloger: could not find the exported names attribute: %v", err)
return nil, err
}
return content, nil
}
// fetchSbomSymbols enumerates the symbols exported by a binary to detect Native Image's SBOM symbols.
func (ni nativeImagePE) fetchSbomSymbols(content *exportContentPE) {
// Appending NULL bytes to symbol names simplifies finding them in the export data directory
sbomBytes := []byte(nativeImageSbomSymbol + "\x00")
sbomLengthBytes := []byte(nativeImageSbomLengthSymbol + "\x00")
svmVersionInfoBytes := []byte(nativeImageSbomVersionSymbol + "\x00")
n := uint32(len(ni.exports))
// Find SBOM, SBOM Length, and SVM Version Symbol
for i := uint32(0); i < content.numberOfNames; i++ {
j := i * uint32(unsafe.Sizeof(ni.t.namePointer))
addressBase := content.addressOfNames - ni.exportSymbols.VirtualAddress
k := addressBase + j
sz := uint32(unsafe.Sizeof(ni.t.namePointer))
if k+sz >= n {
log.Debugf("native-image cataloger: invalid index to exported function: %v.", k)
// If we are at the end of exports, stop looking
return
}
var symbolAddress uint32
p := bytes.NewBuffer(ni.exports[k : k+sz])
err := binary.Read(p, binary.LittleEndian, &symbolAddress)
if err != nil {
log.Debugf("native-image cataloger: error fetching address of symbol %v.", err)
return
}
symbolBase := symbolAddress - ni.exportSymbols.VirtualAddress
if symbolBase >= n {
log.Debugf("native-image cataloger: invalid index to exported symbol: %v.", symbolBase)
return
}
switch {
case bytes.HasPrefix(ni.exports[symbolBase:], sbomBytes):
content.addressOfSbom = i
case bytes.HasPrefix(ni.exports[symbolBase:], sbomLengthBytes):
content.addressOfSbomLength = i
case bytes.HasPrefix(ni.exports[symbolBase:], svmVersionInfoBytes):
content.addressOfSvmVersion = i
}
}
}
// fetchPkgs obtains the packages from a Native Image given as a PE file.
func (ni nativeImagePE) fetchPkgs() ([]pkg.Package, error) {
content, err := ni.fetchExportContent()
if err != nil {
log.Debugf("native-image cataloger: could not fetch the content of the export directory entry: %v.", err)
return nil, err
}
ni.fetchSbomSymbols(content)
if content.addressOfSbom == uint32(0) || content.addressOfSbomLength == uint32(0) || content.addressOfSvmVersion == uint32(0) {
log.Debugf("native-image cataloger: %v.", nativeImageMissingSymbolsError)
return nil, errors.New(nativeImageMissingSymbolsError)
}
functionsBase := content.addressOfFunctions - ni.exportSymbols.VirtualAddress
sbomOffset := content.addressOfSbom
sbomAddress, err := ni.fetchExportFunctionPointer(functionsBase, sbomOffset)
if err != nil {
log.Debugf("native-image cataloger: cannot fetch SBOM pointer from exported functions: %v.", err)
return nil, err
}
sbomLengthOffset := content.addressOfSbomLength
sbomLengthAddress, err := ni.fetchExportFunctionPointer(functionsBase, sbomLengthOffset)
if err != nil {
log.Debugf("native-image cataloger: cannot fetch SBOM length pointer from exported functions: %v.", err)
return nil, err
}
bi := ni.file
dataSection := bi.Section(".data")
if dataSection == nil {
return nil, nil
}
databuf, err := dataSection.Data()
if err != nil {
log.Debugf("native-image cataloger: cannot obtain buffer from .data section.")
return nil, nil
}
sbomLocation := sbomAddress - dataSection.VirtualAddress
lengthLocation := sbomLengthAddress - dataSection.VirtualAddress
return decompressSbom(databuf, uint64(sbomLocation), uint64(lengthLocation))
}
// fetchPkgs provides the packages available in a UnionReader.
func fetchPkgs(reader unionreader.UnionReader, filename string) []pkg.Package {
var pkgs []pkg.Package
imageformats := []func(string, io.ReaderAt) (nativeImage, error){newElf, newMachO, newPE}
// NOTE: multiple readers are returned to cover universal binaries, which are files
// with more than one binary
readers, err := unionreader.GetReaders(reader)
if err != nil {
log.Debugf("native-image cataloger: failed to open a binary: %v.", err)
return nil
}
for _, r := range readers {
for _, makeNativeImage := range imageformats {
ni, err := makeNativeImage(filename, r)
if err != nil {
continue
}
newpkgs, err := ni.fetchPkgs()
if err != nil {
log.Debugf("native-image cataloger: error extracting SBOM from %s: %v.", filename, err)
continue
}
pkgs = append(pkgs, newpkgs...)
}
}
return pkgs
}
// Catalog attempts to find any native image executables reachable from a resolver.
func (c *NativeImageCataloger) Catalog(resolver source.FileResolver) ([]pkg.Package, []artifact.Relationship, error) {
var pkgs []pkg.Package
fileMatches, err := resolver.FilesByMIMEType(internal.ExecutableMIMETypeSet.List()...)
if err != nil {
return pkgs, nil, fmt.Errorf("failed to find binaries by mime types: %w", err)
}
for _, location := range fileMatches {
readerCloser, err := resolver.FileContentsByLocation(location)
if err != nil {
log.Debugf("native-image cataloger: error opening file: %v.", err)
continue
}
log.Tracef("native-image cataloger: found an executable file %v.", location)
reader, err := unionreader.GetUnionReader(readerCloser)
if err != nil {
return nil, nil, err
}
newpkgs := fetchPkgs(reader, location.RealPath)
pkgs = append(pkgs, newpkgs...)
internal.CloseAndLogError(readerCloser, location.RealPath)
}
return pkgs, nil, nil
}
@@ -0,0 +1,123 @@
package java
import (
"bufio"
"bytes"
"compress/gzip"
"encoding/binary"
"io"
"io/ioutil"
"os"
"path"
"testing"
"github.com/stretchr/testify/assert"
"github.com/anchore/syft/syft/cpe"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/unionreader"
)
func TestParseNativeImage(t *testing.T) {
tests := []struct {
fixture string
}{
{
fixture: "test-fixtures/java-builds/packages/example-java-app",
},
}
for _, test := range tests {
t.Run(path.Base(test.fixture), func(t *testing.T) {
f, err := os.Open(test.fixture)
assert.NoError(t, err)
readerCloser := io.ReadCloser(ioutil.NopCloser(f))
reader, err := unionreader.GetUnionReader(readerCloser)
assert.NoError(t, err)
parsed := false
readers, err := unionreader.GetReaders(reader)
for _, r := range readers {
ni, err := newElf(test.fixture, r)
assert.NoError(t, err)
_, err = ni.fetchPkgs()
if err == nil {
t.Fatalf("should have failed to extract SBOM.")
}
// If we can enumerate the symbols in the binary, we can parse the binary.
if err.Error() == nativeImageMissingSymbolsError {
parsed = true
}
}
if !parsed {
t.Fatalf("Could not parse the Native Image executable: %v", test.fixture)
}
})
}
}
func TestParseNativeImageSbom(t *testing.T) {
tests := []struct {
fixture string
expected []pkg.Package
}{
{
fixture: "test-fixtures/graalvm-sbom/micronaut.json",
expected: []pkg.Package{
{
Name: "netty-codec-http2",
Version: "4.1.73.Final",
Language: pkg.Java,
Type: pkg.GraalVMNativeImagePkg,
MetadataType: pkg.JavaMetadataType,
FoundBy: nativeImageCatalogerName,
Metadata: pkg.JavaMetadata{
PomProperties: &pkg.PomProperties{
GroupID: "io.netty",
},
},
CPEs: []cpe.CPE{
{
Part: "a",
Vendor: "codec",
Product: "codec",
Version: "4.1.73.Final",
},
{
Part: "a",
Vendor: "codec",
Product: "netty-codec-http2",
Version: "4.1.73.Final",
},
{
Part: "a",
Vendor: "codec",
Product: "netty_codec_http2",
Version: "4.1.73.Final",
},
},
},
},
},
}
for _, test := range tests {
t.Run(path.Base(test.fixture), func(t *testing.T) {
// Create a buffer to resemble a compressed SBOM in a native image.
sbom, err := ioutil.ReadFile(test.fixture)
assert.NoError(t, err)
var b bytes.Buffer
writebytes := bufio.NewWriter(&b)
z := gzip.NewWriter(writebytes)
_, err = z.Write(sbom)
assert.NoError(t, err)
z.Close()
writebytes.Flush()
compressedsbom := b.Bytes()
sbomlength := uint64(len(compressedsbom))
binary.Write(writebytes, binary.LittleEndian, sbomlength)
writebytes.Flush()
compressedsbom = b.Bytes()
actual, err := decompressSbom(compressedsbom, 0, sbomlength)
assert.NoError(t, err)
assert.Equal(t, test.expected, actual)
})
}
}
@@ -0,0 +1,28 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"version": 1,
"components": [
{
"type": "library",
"group": "io.netty",
"name": "netty-codec-http2",
"version": "4.1.73.Final",
"properties": [
{
"name": "syft:cpe23",
"value": "cpe:2.3:a:codec:codec:4.1.73.Final:*:*:*:*:*:*:*"
},
{
"name": "syft:cpe23",
"value": "cpe:2.3:a:codec:netty-codec-http2:4.1.73.Final:*:*:*:*:*:*:*"
},
{
"name": "syft:cpe23",
"value": "cpe:2.3:a:codec:netty_codec_http2:4.1.73.Final:*:*:*:*:*:*:*"
}
]
}
],
"serialNumber": "urn:uuid:43538af4-f715-3d85-9629-336fdd3790ad"
}
@@ -4,7 +4,7 @@ ifndef PKGSDIR
$(error PKGSDIR is not set)
endif
all: jars archives
all: jars archives native-image
clean: clean-examples
rm -f $(PKGSDIR)/*
@@ -17,6 +17,8 @@ jars: $(PKGSDIR)/example-java-app-maven-0.1.0.jar $(PKGSDIR)/example-java-app-gr
archives: $(PKGSDIR)/example-java-app-maven-0.1.0.zip $(PKGSDIR)/example-java-app-maven-0.1.0.tar $(PKGSDIR)/example-java-app-maven-0.1.0.tar.gz
native-image: $(PKGSDIR)/example-java-app
# jars within archives...
$(PKGSDIR)/example-java-app-maven-0.1.0.zip: $(PKGSDIR)/example-java-app-maven-0.1.0.jar
@@ -62,6 +64,10 @@ clean-jenkins:
example-jenkins-plugin/dependency-reduced-pom.xml \
example-jenkins-plugin/*.exploding
# Native Image...
$(PKGSDIR)/example-java-app: $(PKGSDIR)/example-java-app-maven-0.1.0.jar
./build-example-java-app-native-image.sh $(PKGSDIR)
# we need a way to determine if CI should bust the test cache based on the source material
$(PKGSDIR).fingerprint: clean-examples
mkdir -p $(PKGSDIR)
@@ -0,0 +1,18 @@
#!/usr/bin/env bash
set -uxe
PKGSDIR=$1
CTRID=$(docker create -v /example-java-app ghcr.io/graalvm/native-image:22.2.0 -cp /example-java-app/example-java-app-maven-0.1.0.jar --no-fallback -H:Class=hello.HelloWorld -H:Name=example-java-app)
function cleanup() {
docker rm "${CTRID}"
}
trap cleanup EXIT
set +e
docker cp "${PKGSDIR}/example-java-app-maven-0.1.0.jar" "${CTRID}:/example-java-app/"
docker start -a "${CTRID}"
docker cp "${CTRID}:/app/example-java-app" $PKGSDIR
+101 -79
View File
@@ -12,105 +12,127 @@ func Test_JavascriptCataloger(t *testing.T) {
locationSet := source.NewLocationSet(source.NewLocation("package-lock.json"))
expectedPkgs := []pkg.Package{
{
Name: "@actions/core",
Version: "1.6.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/%40actions/core@1.6.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
Name: "@actions/core",
Version: "1.6.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/%40actions/core@1.6.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/@actions/core/-/core-1.6.0.tgz", Integrity: "sha512-NB1UAZomZlCV/LmJqkLhNTqtKfFXJZAUPcfl/zqG7EfsQdeUJtaWO98SGbuQ3pydJ3fHl2CvI/51OKYlCYYcaw=="},
},
{
Name: "ansi-regex",
Version: "3.0.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/ansi-regex@3.0.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "ansi-regex",
Version: "3.0.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/ansi-regex@3.0.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.0.0.tgz", Integrity: "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg="},
},
{
Name: "cowsay",
Version: "1.4.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/cowsay@1.4.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
Name: "cowsay",
Version: "1.4.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/cowsay@1.4.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/cowsay/-/cowsay-1.4.0.tgz", Integrity: "sha512-rdg5k5PsHFVJheO/pmE3aDg2rUDDTfPJau6yYkZYlHFktUz+UxbE+IgnUAEyyCyv4noL5ltxXD0gZzmHPCy/9g=="},
},
{
Name: "get-stdin",
Version: "5.0.1",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/get-stdin@5.0.1",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "get-stdin",
Version: "5.0.1",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/get-stdin@5.0.1",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/get-stdin/-/get-stdin-5.0.1.tgz", Integrity: "sha1-Ei4WFZHiH/TFJTAwVpPyDmOTo5g="},
},
{
Name: "is-fullwidth-code-point",
Version: "2.0.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/is-fullwidth-code-point@2.0.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "is-fullwidth-code-point",
Version: "2.0.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/is-fullwidth-code-point@2.0.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-2.0.0.tgz", Integrity: "sha1-o7MKXE8ZkYMWeqq5O+764937ZU8="},
},
{
Name: "minimist",
Version: "0.0.10",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/minimist@0.0.10",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "minimist",
Version: "0.0.10",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/minimist@0.0.10",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/minimist/-/minimist-0.0.10.tgz", Integrity: "sha1-3j+YVD2/lggr5IrRoMfNqDYwHc8="},
},
{
Name: "optimist",
Version: "0.6.1",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/optimist@0.6.1",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "optimist",
Version: "0.6.1",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/optimist@0.6.1",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/optimist/-/optimist-0.6.1.tgz", Integrity: "sha1-2j6nRob6IaGaERwybpDrFaAZZoY="},
},
{
Name: "string-width",
Version: "2.1.1",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/string-width@2.1.1",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "string-width",
Version: "2.1.1",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/string-width@2.1.1",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/string-width/-/string-width-2.1.1.tgz", Integrity: "sha512-nOqH59deCq9SRHlxq1Aw85Jnt4w6KvLKqWVik6oA9ZklXLNIOlqg4F2yrT1MVaTjAqvVwdfeZ7w7aCvJD7ugkw=="},
},
{
Name: "strip-ansi",
Version: "4.0.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/strip-ansi@4.0.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "strip-ansi",
Version: "4.0.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/strip-ansi@4.0.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/strip-ansi/-/strip-ansi-4.0.0.tgz", Integrity: "sha1-qEeQIusaw2iocTibY1JixQXuNo8="},
},
{
Name: "strip-eof",
Version: "1.0.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/strip-eof@1.0.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "strip-eof",
Version: "1.0.0",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/strip-eof@1.0.0",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/strip-eof/-/strip-eof-1.0.0.tgz", Integrity: "sha1-u0P/VZim6wXYm1n80SnJgzE2Br8="},
},
{
Name: "wordwrap",
Version: "0.0.3",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/wordwrap@0.0.3",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "wordwrap",
Version: "0.0.3",
FoundBy: "javascript-lock-cataloger",
PURL: "pkg:npm/wordwrap@0.0.3",
Locations: locationSet,
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz", Integrity: "sha1-o9XabNXAvAAI03I0u68b7WMFkQc="},
},
}
+17 -13
View File
@@ -63,12 +63,14 @@ func newPackageLockV1Package(resolver source.FileResolver, location source.Locat
resolver,
location,
pkg.Package{
Name: name,
Version: version,
Locations: source.NewLocationSet(location),
PURL: packageURL(name, version),
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: name,
Version: version,
Locations: source.NewLocationSet(location),
PURL: packageURL(name, version),
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: u.Resolved, Integrity: u.Integrity},
},
)
}
@@ -84,13 +86,15 @@ func newPackageLockV2Package(resolver source.FileResolver, location source.Locat
resolver,
location,
pkg.Package{
Name: name,
Version: u.Version,
Locations: source.NewLocationSet(location),
PURL: packageURL(name, u.Version),
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: licenses,
Name: name,
Version: u.Version,
Locations: source.NewLocationSet(location),
PURL: packageURL(name, u.Version),
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: licenses,
MetadataType: pkg.NpmPackageLockJSONMetadataType,
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: u.Resolved, Integrity: u.Integrity},
},
)
}
@@ -13,81 +13,103 @@ func TestParsePackageLock(t *testing.T) {
var expectedRelationships []artifact.Relationship
expectedPkgs := []pkg.Package{
{
Name: "@actions/core",
Version: "1.6.0",
PURL: "pkg:npm/%40actions/core@1.6.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "@actions/core",
Version: "1.6.0",
PURL: "pkg:npm/%40actions/core@1.6.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/@actions/core/-/core-1.6.0.tgz", Integrity: "sha512-NB1UAZomZlCV/LmJqkLhNTqtKfFXJZAUPcfl/zqG7EfsQdeUJtaWO98SGbuQ3pydJ3fHl2CvI/51OKYlCYYcaw=="},
},
{
Name: "ansi-regex",
Version: "3.0.0",
PURL: "pkg:npm/ansi-regex@3.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "ansi-regex",
Version: "3.0.0",
PURL: "pkg:npm/ansi-regex@3.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.0.0.tgz", Integrity: "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg="},
},
{
Name: "cowsay",
Version: "1.4.0",
PURL: "pkg:npm/cowsay@1.4.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "cowsay",
Version: "1.4.0",
PURL: "pkg:npm/cowsay@1.4.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/cowsay/-/cowsay-1.4.0.tgz", Integrity: "sha512-rdg5k5PsHFVJheO/pmE3aDg2rUDDTfPJau6yYkZYlHFktUz+UxbE+IgnUAEyyCyv4noL5ltxXD0gZzmHPCy/9g=="},
},
{
Name: "get-stdin",
Version: "5.0.1",
PURL: "pkg:npm/get-stdin@5.0.1",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "get-stdin",
Version: "5.0.1",
PURL: "pkg:npm/get-stdin@5.0.1",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/get-stdin/-/get-stdin-5.0.1.tgz", Integrity: "sha1-Ei4WFZHiH/TFJTAwVpPyDmOTo5g="},
},
{
Name: "is-fullwidth-code-point",
Version: "2.0.0",
PURL: "pkg:npm/is-fullwidth-code-point@2.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "is-fullwidth-code-point",
Version: "2.0.0",
PURL: "pkg:npm/is-fullwidth-code-point@2.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-2.0.0.tgz", Integrity: "sha1-o7MKXE8ZkYMWeqq5O+764937ZU8="},
},
{
Name: "minimist",
Version: "0.0.10",
PURL: "pkg:npm/minimist@0.0.10",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "minimist",
Version: "0.0.10",
PURL: "pkg:npm/minimist@0.0.10",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/minimist/-/minimist-0.0.10.tgz", Integrity: "sha1-3j+YVD2/lggr5IrRoMfNqDYwHc8="},
},
{
Name: "optimist",
Version: "0.6.1",
PURL: "pkg:npm/optimist@0.6.1",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "optimist",
Version: "0.6.1",
PURL: "pkg:npm/optimist@0.6.1",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/optimist/-/optimist-0.6.1.tgz", Integrity: "sha1-2j6nRob6IaGaERwybpDrFaAZZoY="},
},
{
Name: "string-width",
Version: "2.1.1",
PURL: "pkg:npm/string-width@2.1.1",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "string-width",
Version: "2.1.1",
PURL: "pkg:npm/string-width@2.1.1",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/string-width/-/string-width-2.1.1.tgz", Integrity: "sha512-nOqH59deCq9SRHlxq1Aw85Jnt4w6KvLKqWVik6oA9ZklXLNIOlqg4F2yrT1MVaTjAqvVwdfeZ7w7aCvJD7ugkw=="},
},
{
Name: "strip-ansi",
Version: "4.0.0",
PURL: "pkg:npm/strip-ansi@4.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "strip-ansi",
Version: "4.0.0",
PURL: "pkg:npm/strip-ansi@4.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/strip-ansi/-/strip-ansi-4.0.0.tgz", Integrity: "sha1-qEeQIusaw2iocTibY1JixQXuNo8="},
},
{
Name: "strip-eof",
Version: "1.0.0",
PURL: "pkg:npm/strip-eof@1.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "strip-eof",
Version: "1.0.0",
PURL: "pkg:npm/strip-eof@1.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/strip-eof/-/strip-eof-1.0.0.tgz", Integrity: "sha1-u0P/VZim6wXYm1n80SnJgzE2Br8="},
},
{
Name: "wordwrap",
Version: "0.0.3",
PURL: "pkg:npm/wordwrap@0.0.3",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "wordwrap",
Version: "0.0.3",
PURL: "pkg:npm/wordwrap@0.0.3",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz", Integrity: "sha1-o9XabNXAvAAI03I0u68b7WMFkQc="},
},
}
fixture := "test-fixtures/pkg-lock/package-lock.json"
@@ -103,43 +125,53 @@ func TestParsePackageLockV2(t *testing.T) {
var expectedRelationships []artifact.Relationship
expectedPkgs := []pkg.Package{
{
Name: "npm",
Version: "6.14.6",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/npm@6.14.6",
Name: "npm",
Version: "6.14.6",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/npm@6.14.6",
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{},
},
{
Name: "@types/prop-types",
Version: "15.7.5",
PURL: "pkg:npm/%40types/prop-types@15.7.5",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
Name: "@types/prop-types",
Version: "15.7.5",
PURL: "pkg:npm/%40types/prop-types@15.7.5",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/@types/prop-types/-/prop-types-15.7.5.tgz", Integrity: "sha1-XxnSuFqY6VWANvajysyIGUIPBc8="},
},
{
Name: "@types/react",
Version: "18.0.17",
PURL: "pkg:npm/%40types/react@18.0.17",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
Name: "@types/react",
Version: "18.0.17",
PURL: "pkg:npm/%40types/react@18.0.17",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/@types/react/-/react-18.0.17.tgz", Integrity: "sha1-RYPZwyLWfv5LOak10iPtzHBQzPQ="},
},
{
Name: "@types/scheduler",
Version: "0.16.2",
PURL: "pkg:npm/%40types/scheduler@0.16.2",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
Name: "@types/scheduler",
Version: "0.16.2",
PURL: "pkg:npm/%40types/scheduler@0.16.2",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/@types/scheduler/-/scheduler-0.16.2.tgz", Integrity: "sha1-GmL4lSVyPd4kuhsBsJK/XfitTTk="},
},
{
Name: "csstype",
Version: "3.1.0",
PURL: "pkg:npm/csstype@3.1.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
Name: "csstype",
Version: "3.1.0",
PURL: "pkg:npm/csstype@3.1.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"MIT"},
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/csstype/-/csstype-3.1.0.tgz", Integrity: "sha1-TdysNxjXh8+d8NG30VAzklyPKfI="},
},
}
for i := range expectedPkgs {
@@ -153,39 +185,49 @@ func TestParsePackageLockV3(t *testing.T) {
var expectedRelationships []artifact.Relationship
expectedPkgs := []pkg.Package{
{
Name: "lock-v3-fixture",
Version: "1.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/lock-v3-fixture@1.0.0",
Name: "lock-v3-fixture",
Version: "1.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/lock-v3-fixture@1.0.0",
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{},
},
{
Name: "@types/prop-types",
Version: "15.7.5",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/%40types/prop-types@15.7.5",
Name: "@types/prop-types",
Version: "15.7.5",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/%40types/prop-types@15.7.5",
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/@types/prop-types/-/prop-types-15.7.5.tgz", Integrity: "sha512-JCB8C6SnDoQf0cNycqd/35A7MjcnK+ZTqE7judS6o7utxUCg6imJg3QK2qzHKszlTjcj2cn+NwMB2i96ubpj7w=="},
},
{
Name: "@types/react",
Version: "18.0.20",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/%40types/react@18.0.20",
Name: "@types/react",
Version: "18.0.20",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/%40types/react@18.0.20",
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/@types/react/-/react-18.0.20.tgz", Integrity: "sha512-MWul1teSPxujEHVwZl4a5HxQ9vVNsjTchVA+xRqv/VYGCuKGAU6UhfrTdF5aBefwD1BHUD8i/zq+O/vyCm/FrA=="},
},
{
Name: "@types/scheduler",
Version: "0.16.2",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/%40types/scheduler@0.16.2",
Name: "@types/scheduler",
Version: "0.16.2",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/%40types/scheduler@0.16.2",
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/@types/scheduler/-/scheduler-0.16.2.tgz", Integrity: "sha512-hppQEBDmlwhFAXKJX2KnWLYu5yMfi91yazPb2l+lbJiwW+wdo1gNeRA+3RgNSO39WYX2euey41KEwnqesU2Jew=="},
},
{
Name: "csstype",
Version: "3.1.1",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/csstype@3.1.1",
Name: "csstype",
Version: "3.1.1",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
PURL: "pkg:npm/csstype@3.1.1",
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/csstype/-/csstype-3.1.1.tgz", Integrity: "sha512-DJR/VvkAvSZW9bTouZue2sSxDwdTN92uHjqeKVm+0dAqdfNykRzQ95tay8aXMBAAPpUiq4Qcug2L7neoRh2Egw=="},
},
}
for i := range expectedPkgs {
@@ -198,35 +240,43 @@ func TestParsePackageLockAlias(t *testing.T) {
var expectedRelationships []artifact.Relationship
commonPkgs := []pkg.Package{
{
Name: "case",
Version: "1.6.2",
PURL: "pkg:npm/case@1.6.2",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "case",
Version: "1.6.2",
PURL: "pkg:npm/case@1.6.2",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/case/-/case-1.6.2.tgz", Integrity: "sha512-ll380ZRoraT7mUK2G92UbH+FJVD5AwdVIAYk9xhV1tauh0carDgYByUD1HhjCWsWgxrfQvCeHvtfj7IYR6TKeg=="},
},
{
Name: "case",
Version: "1.6.3",
PURL: "pkg:npm/case@1.6.3",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "case",
Version: "1.6.3",
PURL: "pkg:npm/case@1.6.3",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/case/-/case-1.6.3.tgz", Integrity: "sha512-mzDSXIPaFwVDvZAHqZ9VlbyF4yyXRuX6IvB06WvPYkqJVO24kX1PPhv9bfpKNFZyxYFmmgo03HUiD8iklmJYRQ=="},
},
{
Name: "@bundled-es-modules/chai",
Version: "4.2.2",
PURL: "pkg:npm/%40bundled-es-modules/chai@4.2.2",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Name: "@bundled-es-modules/chai",
Version: "4.2.2",
PURL: "pkg:npm/%40bundled-es-modules/chai@4.2.2",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{Resolved: "https://registry.npmjs.org/@bundled-es-modules/chai/-/chai-4.2.2.tgz", Integrity: "sha512-iGmVYw2/zJCoqyKTtWEYCtFmMyi8WmACQKtky0lpNyEKWX0YIOpKWGD7saMXL+tPpllss0otilxV0SLwyi3Ytg=="},
},
}
v2Pkg := pkg.Package{
Name: "alias-check",
Version: "1.0.0",
PURL: "pkg:npm/alias-check@1.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"ISC"},
Name: "alias-check",
Version: "1.0.0",
PURL: "pkg:npm/alias-check@1.0.0",
Language: pkg.JavaScript,
Type: pkg.NpmPkg,
Licenses: []string{"ISC"},
MetadataType: "NpmPackageLockJsonMetadata",
Metadata: pkg.NpmPackageLockJSONMetadata{},
}
packageLockV1 := "test-fixtures/pkg-lock/alias-package-lock-1.json"
+17
View File
@@ -23,6 +23,23 @@ func newPackageForIndex(name, version string, locations ...source.Location) pkg.
return p
}
func newPackageForIndexWithMetadata(name, version string, metadata pkg.PythonPipfileLockMetadata, locations ...source.Location) pkg.Package {
p := pkg.Package{
Name: name,
Version: version,
Locations: source.NewLocationSet(locations...),
PURL: packageURL(name, version, nil),
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonPipfileLockMetadataType,
Metadata: metadata,
}
p.SetID()
return p
}
func newPackageForPackage(m pkg.PythonPackageMetadata, sources ...source.Location) pkg.Package {
var licenses []string
if m.License != "" {
@@ -33,7 +33,9 @@ type pipfileLock struct {
}
type Dependency struct {
Version string `json:"version"`
Hashes []string `json:"hashes"`
Version string `json:"version"`
Index string `json:"index"`
}
var _ generic.Parser = parsePipfileLock
@@ -50,9 +52,20 @@ func parsePipfileLock(_ source.FileResolver, _ *generic.Environment, reader sour
} else if err != nil {
return nil, nil, fmt.Errorf("failed to parse Pipfile.lock file: %w", err)
}
sourcesMap := map[string]string{}
for _, source := range lock.Meta.Sources {
sourcesMap[source.Name] = source.URL
}
for name, pkgMeta := range lock.Default {
var index string
if pkgMeta.Index != "" {
index = sourcesMap[pkgMeta.Index]
} else {
// https://pipenv.pypa.io/en/latest/advanced/#specifying-package-indexes
index = "https://pypi.org/simple"
}
version := strings.TrimPrefix(pkgMeta.Version, "==")
pkgs = append(pkgs, newPackageForIndex(name, version, reader.Location))
pkgs = append(pkgs, newPackageForIndexWithMetadata(name, version, pkg.PythonPipfileLockMetadata{Index: index, Hashes: pkgMeta.Hashes}, reader.Location))
}
}
@@ -15,36 +15,67 @@ func TestParsePipFileLock(t *testing.T) {
locations := source.NewLocationSet(source.NewLocation(fixture))
expectedPkgs := []pkg.Package{
{
Name: "aio-pika",
Version: "6.8.0",
PURL: "pkg:pypi/aio-pika@6.8.0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
Name: "aio-pika",
Version: "6.8.0",
PURL: "pkg:pypi/aio-pika@6.8.0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonPipfileLockMetadataType,
Metadata: pkg.PythonPipfileLockMetadata{
Index: "https://pypi.org/simple",
Hashes: []string{
"sha256:1d4305a5f78af3857310b4fe48348cdcf6c097e0e275ea88c2cd08570531a369",
"sha256:e69afef8695f47c5d107bbdba21bdb845d5c249acb3be53ef5c2d497b02657c0",
}},
},
{
Name: "aiodns",
Version: "2.0.0",
PURL: "pkg:pypi/aiodns@2.0.0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
Name: "aiodns",
Version: "2.0.0",
PURL: "pkg:pypi/aiodns@2.0.0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonPipfileLockMetadataType,
Metadata: pkg.PythonPipfileLockMetadata{
Index: "https://test.pypi.org/simple",
Hashes: []string{
"sha256:815fdef4607474295d68da46978a54481dd1e7be153c7d60f9e72773cd38d77d",
"sha256:aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de",
},
},
},
{
Name: "aiohttp",
Version: "3.7.4.post0",
PURL: "pkg:pypi/aiohttp@3.7.4.post0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
Name: "aiohttp",
Version: "3.7.4.post0",
PURL: "pkg:pypi/aiohttp@3.7.4.post0",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonPipfileLockMetadataType,
Metadata: pkg.PythonPipfileLockMetadata{
Index: "https://pypi.org/simple",
Hashes: []string{
"sha256:02f46fc0e3c5ac58b80d4d56eb0a7c7d97fcef69ace9326289fb9f1955e65cfe",
"sha256:0563c1b3826945eecd62186f3f5c7d31abb7391fedc893b7e2b26303b5a9f3fe",
},
},
},
{
Name: "aiohttp-jinja2",
Version: "1.4.2",
PURL: "pkg:pypi/aiohttp-jinja2@1.4.2",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
Name: "aiohttp-jinja2",
Version: "1.4.2",
PURL: "pkg:pypi/aiohttp-jinja2@1.4.2",
Locations: locations,
Language: pkg.Python,
Type: pkg.PythonPkg,
MetadataType: pkg.PythonPipfileLockMetadataType,
Metadata: pkg.PythonPipfileLockMetadata{
Index: "https://pypi.org/simple",
Hashes: []string{
"sha256:860da7582efa866744bad5883947557d0f82e457d69903ea65d666b66f8a69ca",
"sha256:9c22a0e48e3b277fc145c67dd8c3b8f609dab36bce9eb337f70dfe716663c9a0",
},
},
},
}
@@ -12,6 +12,11 @@
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
},
{
"name": "test",
"url": "https://test.pypi.org/simple",
"verify_ssl": true
}
]
},
@@ -29,7 +34,7 @@
"sha256:815fdef4607474295d68da46978a54481dd1e7be153c7d60f9e72773cd38d77d",
"sha256:aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de"
],
"index": "pypi",
"index": "test",
"version": "==2.0.0"
},
"aiohttp": {
+14 -14
View File
@@ -41,7 +41,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"GPL-2.0-only"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/alpine-baselayout@3.2.0-r23?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/alpine-baselayout@3.2.0-r23?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:alpine-baselayout:alpine-baselayout:3.2.0-r23:*:*:*:*:*:*:*",
"cpe:2.3:a:alpine-baselayout:alpine_baselayout:3.2.0-r23:*:*:*:*:*:*:*",
@@ -58,7 +58,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"GPL-2.0-only"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/alpine-baselayout-data@3.2.0-r23?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/alpine-baselayout-data@3.2.0-r23?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:alpine-baselayout-data:alpine-baselayout-data:3.2.0-r23:*:*:*:*:*:*:*",
"cpe:2.3:a:alpine-baselayout-data:alpine_baselayout_data:3.2.0-r23:*:*:*:*:*:*:*",
@@ -79,7 +79,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"MIT"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/alpine-keys@2.4-r1?arch=x86_64&upstream=alpine-keys&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/alpine-keys@2.4-r1?arch=x86_64&upstream=alpine-keys&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:alpine-keys:alpine-keys:2.4-r1:*:*:*:*:*:*:*",
"cpe:2.3:a:alpine-keys:alpine_keys:2.4-r1:*:*:*:*:*:*:*",
@@ -96,7 +96,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"GPL-2.0-only"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/apk-tools@2.12.9-r3?arch=x86_64&upstream=apk-tools&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/apk-tools@2.12.9-r3?arch=x86_64&upstream=apk-tools&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:apk-tools:apk-tools:2.12.9-r3:*:*:*:*:*:*:*",
"cpe:2.3:a:apk-tools:apk_tools:2.12.9-r3:*:*:*:*:*:*:*",
@@ -113,7 +113,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"GPL-2.0-only"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/busybox@1.35.0-r17?arch=x86_64&upstream=busybox&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/busybox@1.35.0-r17?arch=x86_64&upstream=busybox&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:busybox:busybox:1.35.0-r17:*:*:*:*:*:*:*",
),
@@ -125,7 +125,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"MPL-2.0", "AND", "MIT"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/ca-certificates-bundle@20220614-r0?arch=x86_64&upstream=ca-certificates&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/ca-certificates-bundle@20220614-r0?arch=x86_64&upstream=ca-certificates&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:ca-certificates-bundle:ca-certificates-bundle:20220614-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ca-certificates-bundle:ca_certificates_bundle:20220614-r0:*:*:*:*:*:*:*",
@@ -146,7 +146,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"BSD-2-Clause", "AND", "BSD-3-Clause"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/libc-utils@0.7.2-r3?arch=x86_64&upstream=libc-dev&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/libc-utils@0.7.2-r3?arch=x86_64&upstream=libc-dev&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:libc-utils:libc-utils:0.7.2-r3:*:*:*:*:*:*:*",
"cpe:2.3:a:libc-utils:libc_utils:0.7.2-r3:*:*:*:*:*:*:*",
@@ -163,7 +163,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"OpenSSL"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/libcrypto1.1@1.1.1s-r0?arch=x86_64&upstream=openssl&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/libcrypto1.1@1.1.1s-r0?arch=x86_64&upstream=openssl&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:libcrypto1.1:libcrypto1.1:1.1.1s-r0:*:*:*:*:*:*:*",
),
@@ -175,7 +175,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"OpenSSL"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/libssl1.1@1.1.1s-r0?arch=x86_64&upstream=openssl&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/libssl1.1@1.1.1s-r0?arch=x86_64&upstream=openssl&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:libssl1.1:libssl1.1:1.1.1s-r0:*:*:*:*:*:*:*",
),
@@ -187,7 +187,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"MIT"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/musl@1.2.3-r1?arch=x86_64&upstream=musl&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/musl@1.2.3-r1?arch=x86_64&upstream=musl&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:musl:musl:1.2.3-r1:*:*:*:*:*:*:*",
),
@@ -199,7 +199,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"MIT", "BSD", "GPL2+"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/musl-utils@1.2.3-r1?arch=x86_64&upstream=musl&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/musl-utils@1.2.3-r1?arch=x86_64&upstream=musl&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:musl-utils:musl-utils:1.2.3-r1:*:*:*:*:*:*:*",
"cpe:2.3:a:musl-utils:musl_utils:1.2.3-r1:*:*:*:*:*:*:*",
@@ -216,7 +216,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"GPL-2.0-only"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/scanelf@1.3.4-r0?arch=x86_64&upstream=pax-utils&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/scanelf@1.3.4-r0?arch=x86_64&upstream=pax-utils&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:scanelf:scanelf:1.3.4-r0:*:*:*:*:*:*:*",
),
@@ -228,7 +228,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"GPL-2.0-only"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/ssl_client@1.35.0-r17?arch=x86_64&upstream=busybox&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/ssl_client@1.35.0-r17?arch=x86_64&upstream=busybox&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:ssl-client:ssl-client:1.35.0-r17:*:*:*:*:*:*:*",
"cpe:2.3:a:ssl-client:ssl_client:1.35.0-r17:*:*:*:*:*:*:*",
@@ -245,7 +245,7 @@ func Test_parseSBOM(t *testing.T) {
Locations: source.NewLocationSet(source.NewLocation("sbom.syft.json")),
Licenses: []string{"Zlib"},
FoundBy: "sbom-cataloger",
PURL: "pkg:alpine/zlib@1.2.12-r3?arch=x86_64&upstream=zlib&distro=alpine-3.16.3",
PURL: "pkg:apk/alpine/zlib@1.2.12-r3?arch=x86_64&upstream=zlib&distro=alpine-3.16.3",
CPEs: mustCPEs(
"cpe:2.3:a:zlib:zlib:1.2.12-r3:*:*:*:*:*:*:*",
),
@@ -24,7 +24,7 @@
"cpe:2.3:a:alpine:alpine-baselayout:3.2.0-r23:*:*:*:*:*:*:*",
"cpe:2.3:a:alpine:alpine_baselayout:3.2.0-r23:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/alpine-baselayout@3.2.0-r23?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/alpine-baselayout@3.2.0-r23?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "alpine-baselayout",
@@ -411,7 +411,7 @@
"cpe:2.3:a:alpine:alpine-baselayout-data:3.2.0-r23:*:*:*:*:*:*:*",
"cpe:2.3:a:alpine:alpine_baselayout_data:3.2.0-r23:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/alpine-baselayout-data@3.2.0-r23?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/alpine-baselayout-data@3.2.0-r23?arch=x86_64&upstream=alpine-baselayout&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "alpine-baselayout-data",
@@ -570,7 +570,7 @@
"cpe:2.3:a:alpine:alpine-keys:2.4-r1:*:*:*:*:*:*:*",
"cpe:2.3:a:alpine:alpine_keys:2.4-r1:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/alpine-keys@2.4-r1?arch=x86_64&upstream=alpine-keys&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/alpine-keys@2.4-r1?arch=x86_64&upstream=alpine-keys&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "alpine-keys",
@@ -1007,7 +1007,7 @@
"cpe:2.3:a:apk:apk-tools:2.12.9-r3:*:*:*:*:*:*:*",
"cpe:2.3:a:apk:apk_tools:2.12.9-r3:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/apk-tools@2.12.9-r3?arch=x86_64&upstream=apk-tools&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/apk-tools@2.12.9-r3?arch=x86_64&upstream=apk-tools&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "apk-tools",
@@ -1110,7 +1110,7 @@
"cpes": [
"cpe:2.3:a:busybox:busybox:1.35.0-r17:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/busybox@1.35.0-r17?arch=x86_64&upstream=busybox&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/busybox@1.35.0-r17?arch=x86_64&upstream=busybox&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "busybox",
@@ -1294,7 +1294,7 @@
"cpe:2.3:a:ca:ca-certificates-bundle:20220614-r0:*:*:*:*:*:*:*",
"cpe:2.3:a:ca:ca_certificates_bundle:20220614-r0:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/ca-certificates-bundle@20220614-r0?arch=x86_64&upstream=ca-certificates&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/ca-certificates-bundle@20220614-r0?arch=x86_64&upstream=ca-certificates&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "ca-certificates-bundle",
@@ -1369,7 +1369,7 @@
"cpe:2.3:a:libc:libc-utils:0.7.2-r3:*:*:*:*:*:*:*",
"cpe:2.3:a:libc:libc_utils:0.7.2-r3:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/libc-utils@0.7.2-r3?arch=x86_64&upstream=libc-dev&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/libc-utils@0.7.2-r3?arch=x86_64&upstream=libc-dev&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "libc-utils",
@@ -1410,7 +1410,7 @@
"cpes": [
"cpe:2.3:a:libcrypto1.1:libcrypto1.1:1.1.1s-r0:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/libcrypto1.1@1.1.1s-r0?arch=x86_64&upstream=openssl&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/libcrypto1.1@1.1.1s-r0?arch=x86_64&upstream=openssl&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "libcrypto1.1",
@@ -1589,7 +1589,7 @@
"cpes": [
"cpe:2.3:a:libssl1.1:libssl1.1:1.1.1s-r0:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/libssl1.1@1.1.1s-r0?arch=x86_64&upstream=openssl&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/libssl1.1@1.1.1s-r0?arch=x86_64&upstream=openssl&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "libssl1.1",
@@ -1663,7 +1663,7 @@
"cpes": [
"cpe:2.3:a:musl:musl:1.2.3-r1:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/musl@1.2.3-r1?arch=x86_64&upstream=musl&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/musl@1.2.3-r1?arch=x86_64&upstream=musl&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "musl",
@@ -1735,7 +1735,7 @@
"cpe:2.3:a:musl:musl-utils:1.2.3-r1:*:*:*:*:*:*:*",
"cpe:2.3:a:musl:musl_utils:1.2.3-r1:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/musl-utils@1.2.3-r1?arch=x86_64&upstream=musl&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/musl-utils@1.2.3-r1?arch=x86_64&upstream=musl&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "musl-utils",
@@ -1843,7 +1843,7 @@
"cpes": [
"cpe:2.3:a:scanelf:scanelf:1.3.4-r0:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/scanelf@1.3.4-r0?arch=x86_64&upstream=pax-utils&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/scanelf@1.3.4-r0?arch=x86_64&upstream=pax-utils&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "scanelf",
@@ -1908,7 +1908,7 @@
"cpe:2.3:a:ssl:ssl-client:1.35.0-r17:*:*:*:*:*:*:*",
"cpe:2.3:a:ssl:ssl_client:1.35.0-r17:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/ssl_client@1.35.0-r17?arch=x86_64&upstream=busybox&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/ssl_client@1.35.0-r17?arch=x86_64&upstream=busybox&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "ssl_client",
@@ -1970,7 +1970,7 @@
"cpes": [
"cpe:2.3:a:zlib:zlib:1.2.12-r3:*:*:*:*:*:*:*"
],
"purl": "pkg:alpine/zlib@1.2.12-r3?arch=x86_64&upstream=zlib&distro=alpine-3.16.3",
"purl": "pkg:apk/alpine/zlib@1.2.12-r3?arch=x86_64&upstream=zlib&distro=alpine-3.16.3",
"metadataType": "ApkMetadata",
"metadata": {
"package": "zlib",
+27 -20
View File
@@ -12,36 +12,38 @@ type Language string
const (
// the full set of supported programming languages
UnknownLanguage Language = ""
Java Language = "java"
JavaScript Language = "javascript"
Python Language = "python"
PHP Language = "php"
Ruby Language = "ruby"
Go Language = "go"
Rust Language = "rust"
CPP Language = "c++"
Dart Language = "dart"
Dotnet Language = "dotnet"
Swift Language = "swift"
CPP Language = "c++"
Elixir Language = "elixir"
Erlang Language = "erlang"
Go Language = "go"
Haskell Language = "haskell"
Binary Language = "binary"
File Language = "file"
Java Language = "java"
JavaScript Language = "javascript"
PHP Language = "php"
Python Language = "python"
Ruby Language = "ruby"
Rust Language = "rust"
Swift Language = "swift"
)
// AllLanguages is a set of all programming languages detected by syft.
var AllLanguages = []Language{
Java,
JavaScript,
Python,
PHP,
Ruby,
Go,
Rust,
CPP,
Dart,
Dotnet,
Swift,
CPP,
Elixir,
Erlang,
Go,
Haskell,
Java,
JavaScript,
PHP,
Python,
Ruby,
Rust,
Swift,
}
// String returns the string representation of the language.
@@ -84,6 +86,11 @@ func LanguageByName(name string) Language {
return CPP
case packageurl.TypeHackage, string(Haskell):
return Haskell
case packageurl.TypeHex, "beam", "elixir", "erlang":
// should we support returning multiple languages to support this case?
// answer: no. We want this to definitively answer "which language does this package represent?"
// which might not be possible in all cases. See for more context: https://github.com/package-url/purl-spec/pull/178
return UnknownLanguage
default:
return UnknownLanguage
}
+8
View File
@@ -62,6 +62,10 @@ func TestLanguageFromPURL(t *testing.T) {
purl: "pkg:hackage/HTTP@4000.3.16",
want: Haskell,
},
{
purl: "pkg:hex/hpax/hpax@0.1.1",
want: UnknownLanguage,
},
}
var languages []string
@@ -70,6 +74,10 @@ func TestLanguageFromPURL(t *testing.T) {
expectedLanguages.Add(string(ty))
}
// we cannot determine the language from these purl ecosystems (yet?)
expectedLanguages.Remove(Elixir.String())
expectedLanguages.Remove(Erlang.String())
for _, tt := range tests {
t.Run(tt.purl, func(t *testing.T) {
actual := LanguageFromPURL(tt.purl)
+67 -56
View File
@@ -9,74 +9,85 @@ type MetadataType string
const (
// this is the full set of data shapes that can be represented within the pkg.Package.Metadata field
UnknownMetadataType MetadataType = "UnknownMetadata"
ApkMetadataType MetadataType = "ApkMetadata"
AlpmMetadataType MetadataType = "AlpmMetadata"
BinaryMetadataType MetadataType = "BinaryMetadata"
DpkgMetadataType MetadataType = "DpkgMetadata"
GemMetadataType MetadataType = "GemMetadata"
JavaMetadataType MetadataType = "JavaMetadata"
NpmPackageJSONMetadataType MetadataType = "NpmPackageJsonMetadata"
RpmMetadataType MetadataType = "RpmMetadata"
DartPubMetadataType MetadataType = "DartPubMetadata"
DotnetDepsMetadataType MetadataType = "DotnetDepsMetadata"
PythonPackageMetadataType MetadataType = "PythonPackageMetadata"
RustCargoPackageMetadataType MetadataType = "RustCargoPackageMetadata"
KbPackageMetadataType MetadataType = "KbPackageMetadata"
GolangMetadataType MetadataType = "GolangMetadata"
PhpComposerJSONMetadataType MetadataType = "PhpComposerJsonMetadata"
CocoapodsMetadataType MetadataType = "CocoapodsMetadataType"
ConanMetadataType MetadataType = "ConanMetadataType"
ConanLockMetadataType MetadataType = "ConanLockMetadataType"
PortageMetadataType MetadataType = "PortageMetadata"
HackageMetadataType MetadataType = "HackageMetadataType"
UnknownMetadataType MetadataType = "UnknownMetadata"
AlpmMetadataType MetadataType = "AlpmMetadata"
ApkMetadataType MetadataType = "ApkMetadata"
BinaryMetadataType MetadataType = "BinaryMetadata"
CocoapodsMetadataType MetadataType = "CocoapodsMetadataType"
ConanLockMetadataType MetadataType = "ConanLockMetadataType"
ConanMetadataType MetadataType = "ConanMetadataType"
DartPubMetadataType MetadataType = "DartPubMetadata"
DotnetDepsMetadataType MetadataType = "DotnetDepsMetadata"
DpkgMetadataType MetadataType = "DpkgMetadata"
GemMetadataType MetadataType = "GemMetadata"
GolangMetadataType MetadataType = "GolangMetadata"
HackageMetadataType MetadataType = "HackageMetadataType"
JavaMetadataType MetadataType = "JavaMetadata"
KbPackageMetadataType MetadataType = "KbPackageMetadata"
MixLockMetadataType MetadataType = "MixLockMetadataType"
NpmPackageJSONMetadataType MetadataType = "NpmPackageJsonMetadata"
NpmPackageLockJSONMetadataType MetadataType = "NpmPackageLockJsonMetadata"
PhpComposerJSONMetadataType MetadataType = "PhpComposerJsonMetadata"
PortageMetadataType MetadataType = "PortageMetadata"
PythonPackageMetadataType MetadataType = "PythonPackageMetadata"
PythonPipfileLockMetadataType MetadataType = "PythonPipfileLockMetadata"
RebarLockMetadataType MetadataType = "RebarLockMetadataType"
RpmMetadataType MetadataType = "RpmMetadata"
RustCargoPackageMetadataType MetadataType = "RustCargoPackageMetadata"
)
var AllMetadataTypes = []MetadataType{
ApkMetadataType,
AlpmMetadataType,
ApkMetadataType,
BinaryMetadataType,
DpkgMetadataType,
GemMetadataType,
JavaMetadataType,
NpmPackageJSONMetadataType,
RpmMetadataType,
CocoapodsMetadataType,
ConanLockMetadataType,
ConanMetadataType,
DartPubMetadataType,
DotnetDepsMetadataType,
PythonPackageMetadataType,
RustCargoPackageMetadataType,
KbPackageMetadataType,
DpkgMetadataType,
GemMetadataType,
GolangMetadataType,
PhpComposerJSONMetadataType,
CocoapodsMetadataType,
ConanMetadataType,
ConanLockMetadataType,
PortageMetadataType,
HackageMetadataType,
JavaMetadataType,
KbPackageMetadataType,
MixLockMetadataType,
NpmPackageJSONMetadataType,
NpmPackageLockJSONMetadataType,
PhpComposerJSONMetadataType,
PortageMetadataType,
PythonPackageMetadataType,
PythonPipfileLockMetadataType,
RebarLockMetadataType,
RpmMetadataType,
RustCargoPackageMetadataType,
}
var MetadataTypeByName = map[MetadataType]reflect.Type{
ApkMetadataType: reflect.TypeOf(ApkMetadata{}),
AlpmMetadataType: reflect.TypeOf(AlpmMetadata{}),
BinaryMetadataType: reflect.TypeOf(BinaryMetadata{}),
DpkgMetadataType: reflect.TypeOf(DpkgMetadata{}),
GemMetadataType: reflect.TypeOf(GemMetadata{}),
JavaMetadataType: reflect.TypeOf(JavaMetadata{}),
NpmPackageJSONMetadataType: reflect.TypeOf(NpmPackageJSONMetadata{}),
RpmMetadataType: reflect.TypeOf(RpmMetadata{}),
DartPubMetadataType: reflect.TypeOf(DartPubMetadata{}),
DotnetDepsMetadataType: reflect.TypeOf(DotnetDepsMetadata{}),
PythonPackageMetadataType: reflect.TypeOf(PythonPackageMetadata{}),
RustCargoPackageMetadataType: reflect.TypeOf(CargoPackageMetadata{}),
KbPackageMetadataType: reflect.TypeOf(KbPackageMetadata{}),
GolangMetadataType: reflect.TypeOf(GolangMetadata{}),
PhpComposerJSONMetadataType: reflect.TypeOf(PhpComposerJSONMetadata{}),
CocoapodsMetadataType: reflect.TypeOf(CocoapodsMetadata{}),
ConanMetadataType: reflect.TypeOf(ConanMetadata{}),
ConanLockMetadataType: reflect.TypeOf(ConanLockMetadata{}),
PortageMetadataType: reflect.TypeOf(PortageMetadata{}),
HackageMetadataType: reflect.TypeOf(HackageMetadata{}),
AlpmMetadataType: reflect.TypeOf(AlpmMetadata{}),
ApkMetadataType: reflect.TypeOf(ApkMetadata{}),
BinaryMetadataType: reflect.TypeOf(BinaryMetadata{}),
CocoapodsMetadataType: reflect.TypeOf(CocoapodsMetadata{}),
ConanLockMetadataType: reflect.TypeOf(ConanLockMetadata{}),
ConanMetadataType: reflect.TypeOf(ConanMetadata{}),
DartPubMetadataType: reflect.TypeOf(DartPubMetadata{}),
DotnetDepsMetadataType: reflect.TypeOf(DotnetDepsMetadata{}),
DpkgMetadataType: reflect.TypeOf(DpkgMetadata{}),
GemMetadataType: reflect.TypeOf(GemMetadata{}),
GolangMetadataType: reflect.TypeOf(GolangMetadata{}),
HackageMetadataType: reflect.TypeOf(HackageMetadata{}),
JavaMetadataType: reflect.TypeOf(JavaMetadata{}),
KbPackageMetadataType: reflect.TypeOf(KbPackageMetadata{}),
MixLockMetadataType: reflect.TypeOf(MixLockMetadata{}),
NpmPackageJSONMetadataType: reflect.TypeOf(NpmPackageJSONMetadata{}),
NpmPackageLockJSONMetadataType: reflect.TypeOf(NpmPackageLockJSONMetadata{}),
PhpComposerJSONMetadataType: reflect.TypeOf(PhpComposerJSONMetadata{}),
PortageMetadataType: reflect.TypeOf(PortageMetadata{}),
PythonPackageMetadataType: reflect.TypeOf(PythonPackageMetadata{}),
PythonPipfileLockMetadataType: reflect.TypeOf(PythonPipfileLockMetadata{}),
RebarLockMetadataType: reflect.TypeOf(RebarLockMetadata{}),
RpmMetadataType: reflect.TypeOf(RpmMetadata{}),
RustCargoPackageMetadataType: reflect.TypeOf(CargoPackageMetadata{}),
}
func CleanMetadataType(typ MetadataType) MetadataType {
+8
View File
@@ -0,0 +1,8 @@
package pkg
type MixLockMetadata struct {
Name string `mapstructure:"name" json:"name"`
Version string `mapstructure:"version" json:"version"`
PkgHash string `mapstructure:"pkgHash" json:"pkgHash"`
PkgHashExt string `mapstructure:"pkgHashExt" json:"pkgHashExt"`
}
+1 -1
View File
@@ -1,6 +1,6 @@
package pkg
// NpmPackageJSONMetadata holds extra information that is used in pkg.Package
// NpmPackageJSONMetadata holds parsing information for a javascript package.json file
type NpmPackageJSONMetadata struct {
Name string `mapstructure:"name" json:"name"`
Version string `mapstructure:"version" json:"version"`
@@ -0,0 +1,7 @@
package pkg
// NpmPackageLockJSONMetadata holds parsing information for a javascript package-lock.json file
type NpmPackageLockJSONMetadata struct {
Resolved string `mapstructure:"resolved" json:"resolved"`
Integrity string `mapstructure:"integrity" json:"integrity"`
}
@@ -0,0 +1,6 @@
package pkg
type PythonPipfileLockMetadata struct {
Hashes []string `mapstructure:"hashes" json:"hashes"`
Index string `mapstructure:"index" json:"index"`
}
+8
View File
@@ -0,0 +1,8 @@
package pkg
type RebarLockMetadata struct {
Name string `mapstructure:"name" json:"name"`
Version string `mapstructure:"version" json:"version"`
PkgHash string `mapstructure:"pkgHash" json:"pkgHash"`
PkgHashExt string `mapstructure:"pkgHashExt" json:"pkgHashExt"`
}
+74 -65
View File
@@ -1,96 +1,103 @@
package pkg
import "github.com/anchore/packageurl-go"
import (
"github.com/anchore/packageurl-go"
)
// Type represents a Package Type for or within a language ecosystem (there may be multiple package types within a language ecosystem)
type Type string
const (
// the full set of supported packages
UnknownPkg Type = "UnknownPackage"
BinaryPkg Type = "binary"
ApkPkg Type = "apk"
AlpmPkg Type = "alpm"
GemPkg Type = "gem"
DebPkg Type = "deb"
RpmPkg Type = "rpm"
NpmPkg Type = "npm"
PythonPkg Type = "python"
PhpComposerPkg Type = "php-composer"
JavaPkg Type = "java-archive"
JenkinsPluginPkg Type = "jenkins-plugin"
GoModulePkg Type = "go-module"
RustPkg Type = "rust-crate"
KbPkg Type = "msrc-kb"
DartPubPkg Type = "dart-pub"
DotnetPkg Type = "dotnet"
CocoapodsPkg Type = "pod"
ConanPkg Type = "conan"
PortagePkg Type = "portage"
HackagePkg Type = "hackage"
UnknownPkg Type = "UnknownPackage"
AlpmPkg Type = "alpm"
ApkPkg Type = "apk"
BinaryPkg Type = "binary"
CocoapodsPkg Type = "pod"
ConanPkg Type = "conan"
DartPubPkg Type = "dart-pub"
DebPkg Type = "deb"
DotnetPkg Type = "dotnet"
GemPkg Type = "gem"
GoModulePkg Type = "go-module"
GraalVMNativeImagePkg Type = "graalvm-native-image"
HackagePkg Type = "hackage"
HexPkg Type = "hex"
JavaPkg Type = "java-archive"
JenkinsPluginPkg Type = "jenkins-plugin"
KbPkg Type = "msrc-kb"
NpmPkg Type = "npm"
PhpComposerPkg Type = "php-composer"
PortagePkg Type = "portage"
PythonPkg Type = "python"
RpmPkg Type = "rpm"
RustPkg Type = "rust-crate"
)
// AllPkgs represents all supported package types
var AllPkgs = []Type{
ApkPkg,
AlpmPkg,
ApkPkg,
BinaryPkg,
GemPkg,
DebPkg,
RpmPkg,
NpmPkg,
PythonPkg,
PhpComposerPkg,
JavaPkg,
JenkinsPluginPkg,
GoModulePkg,
RustPkg,
KbPkg,
DartPubPkg,
DotnetPkg,
CocoapodsPkg,
ConanPkg,
PortagePkg,
DartPubPkg,
DebPkg,
DotnetPkg,
GemPkg,
GoModulePkg,
HackagePkg,
HexPkg,
JavaPkg,
JenkinsPluginPkg,
KbPkg,
NpmPkg,
PhpComposerPkg,
PortagePkg,
PythonPkg,
RpmPkg,
RustPkg,
}
// PackageURLType returns the PURL package type for the current package.
func (t Type) PackageURLType() string {
switch t {
case ApkPkg:
return "alpine"
case AlpmPkg:
return "alpm"
case GemPkg:
return packageurl.TypeGem
case DebPkg:
return "deb"
case PythonPkg:
return packageurl.TypePyPi
case PhpComposerPkg:
return packageurl.TypeComposer
case NpmPkg:
return packageurl.TypeNPM
case JavaPkg, JenkinsPluginPkg:
return packageurl.TypeMaven
case RpmPkg:
return packageurl.TypeRPM
case GoModulePkg:
return packageurl.TypeGolang
case RustPkg:
return "cargo"
case DartPubPkg:
return packageurl.TypePub
case DotnetPkg:
return packageurl.TypeDotnet
case ApkPkg:
return packageurl.TypeAlpine
case CocoapodsPkg:
return packageurl.TypeCocoapods
case ConanPkg:
return packageurl.TypeConan
case PortagePkg:
return "portage"
case DartPubPkg:
return packageurl.TypePub
case DebPkg:
return "deb"
case DotnetPkg:
return packageurl.TypeDotnet
case GemPkg:
return packageurl.TypeGem
case HexPkg:
return packageurl.TypeHex
case GoModulePkg:
return packageurl.TypeGolang
case HackagePkg:
return packageurl.TypeHackage
case JavaPkg, JenkinsPluginPkg:
return packageurl.TypeMaven
case PhpComposerPkg:
return packageurl.TypeComposer
case PythonPkg:
return packageurl.TypePyPi
case PortagePkg:
return "portage"
case NpmPkg:
return packageurl.TypeNPM
case RpmPkg:
return packageurl.TypeRPM
case RustPkg:
return "cargo"
default:
// TODO: should this be a "generic" purl type instead?
return ""
@@ -108,13 +115,13 @@ func TypeFromPURL(p string) Type {
func TypeByName(name string) Type {
switch name {
case packageurl.TypeDebian, "deb":
case packageurl.TypeDebian:
return DebPkg
case packageurl.TypeRPM:
return RpmPkg
case "alpm":
return AlpmPkg
case "alpine":
case packageurl.TypeAlpine, "alpine":
return ApkPkg
case packageurl.TypeMaven:
return JavaPkg
@@ -142,6 +149,8 @@ func TypeByName(name string) Type {
return HackagePkg
case "portage":
return PortagePkg
case packageurl.TypeHex:
return HexPkg
default:
return UnknownPkg
}
+5 -1
View File
@@ -19,7 +19,7 @@ func TestTypeFromPURL(t *testing.T) {
expected: RpmPkg,
},
{
purl: "pkg:alpine/util-linux@2.32.1",
purl: "pkg:apk/alpine/util-linux@2.32.1",
expected: ApkPkg,
},
{
@@ -79,6 +79,10 @@ func TestTypeFromPURL(t *testing.T) {
purl: "pkg:hackage/HTTP@4000.3.16",
expected: HackagePkg,
},
{
purl: "pkg:hex/hpax/hpax@0.1.1",
expected: HexPkg,
},
}
var pkgTypes []string
+12
View File
@@ -99,6 +99,18 @@ func (m *multiWriter) Write(s SBOM) (errs error) {
return errs
}
// Bytes returns the bytes of the SBOM that would be written
func (m *multiWriter) Bytes(s SBOM) (bytes []byte, err error) {
for _, w := range m.writers {
b, err := w.Bytes(s)
if err != nil {
return nil, err
}
bytes = append(bytes, b...)
}
return bytes, nil
}
// Close closes all writers
func (m *multiWriter) Close() (errs error) {
for _, w := range m.writers {
+11
View File
@@ -1,6 +1,7 @@
package sbom
import (
"bytes"
"io"
)
@@ -16,6 +17,16 @@ func (w *streamWriter) Write(s SBOM) error {
return w.format.Encode(w.out, s)
}
// Bytes returns the bytes of the SBOM that would be written
func (w *streamWriter) Bytes(s SBOM) ([]byte, error) {
var buffer bytes.Buffer
err := w.format.Encode(&buffer, s)
if err != nil {
return nil, err
}
return buffer.Bytes(), nil
}
// Close any resources, such as open files
func (w *streamWriter) Close() error {
if w.close != nil {
+3
View File
@@ -7,6 +7,9 @@ type Writer interface {
// Write writes the provided SBOM
Write(SBOM) error
// Bytes returns the bytes of the SBOM that would be written
Bytes(SBOM) ([]byte, error)
// Closer a resource cleanup hook which will be called after SBOM
// is written or if an error occurs before Write is called
io.Closer
+6
View File
@@ -301,6 +301,12 @@ func fileAnalysisPath(path string) (string, func()) {
// unarchived.
envelopedUnarchiver, err := archiver.ByExtension(path)
if unarchiver, ok := envelopedUnarchiver.(archiver.Unarchiver); err == nil && ok {
if tar, ok := unarchiver.(*archiver.Tar); ok {
// when tar files are extracted, if there are multiple entries at the same
// location, the last entry wins
// NOTE: this currently does not display any messages if an overwrite happens
tar.OverwriteExisting = true
}
unarchivedPath, tmpCleanup, err := unarchiveToTmp(path, unarchiver)
if err != nil {
log.Warnf("file could not be unarchived: %+v", err)
+33 -4
View File
@@ -4,6 +4,7 @@
package source
import (
"io"
"io/ioutil"
"os"
"os/exec"
@@ -252,6 +253,8 @@ func TestNewFromFile_WithArchive(t *testing.T) {
expString string
inputPaths []string
expRefs int
layer2 bool
contents string
}{
{
desc: "path detected",
@@ -259,10 +262,18 @@ func TestNewFromFile_WithArchive(t *testing.T) {
inputPaths: []string{"/.vimrc"},
expRefs: 1,
},
{
desc: "lest entry for duplicate paths",
input: "test-fixtures/path-detected",
inputPaths: []string{"/.vimrc"},
expRefs: 1,
layer2: true,
contents: "Another .vimrc file",
},
}
for _, test := range testCases {
t.Run(test.desc, func(t *testing.T) {
archivePath := setupArchiveTest(t, test.input)
archivePath := setupArchiveTest(t, test.input, test.layer2)
src, cleanup := NewFromFile(archivePath)
if cleanup != nil {
@@ -279,6 +290,16 @@ func TestNewFromFile_WithArchive(t *testing.T) {
require.NoError(t, err)
assert.Len(t, refs, test.expRefs)
if test.contents != "" {
reader, err := resolver.FileContentsByLocation(refs[0])
require.NoError(t, err)
data, err := io.ReadAll(reader)
require.NoError(t, err)
assert.Equal(t, test.contents, string(data))
}
})
}
}
@@ -716,7 +737,7 @@ func Test_crossPlatformExclusions(t *testing.T) {
}
// createArchive creates a new archive file at destinationArchivePath based on the directory found at sourceDirPath.
func createArchive(t testing.TB, sourceDirPath, destinationArchivePath string) {
func createArchive(t testing.TB, sourceDirPath, destinationArchivePath string, layer2 bool) {
t.Helper()
cwd, err := os.Getwd()
@@ -749,13 +770,21 @@ func createArchive(t testing.TB, sourceDirPath, destinationArchivePath string) {
}
}
if layer2 {
cmd = exec.Command("tar", "-rvf", destinationArchivePath, ".")
cmd.Dir = filepath.Join(cwd, "test-fixtures", path.Base(sourceDirPath+"-2"))
if err := cmd.Start(); err != nil {
t.Fatalf("unable to start tar appending fixture script: %+v", err)
}
_ = cmd.Wait()
}
}
// setupArchiveTest encapsulates common test setup work for tar file tests. It returns a cleanup function,
// which should be called (typically deferred) by the caller, the path of the created tar archive, and an error,
// which should trigger a fatal test failure in the consuming test. The returned cleanup function will never be nil
// (even if there's an error), and it should always be called.
func setupArchiveTest(t testing.TB, sourceDirPath string) string {
func setupArchiveTest(t testing.TB, sourceDirPath string, layer2 bool) string {
t.Helper()
archivePrefix, err := ioutil.TempFile("", "syft-archive-TEST-")
@@ -771,7 +800,7 @@ func setupArchiveTest(t testing.TB, sourceDirPath string) string {
destinationArchiveFilePath := archivePrefix.Name() + ".tar"
t.Logf("archive path: %s", destinationArchiveFilePath)
createArchive(t, sourceDirPath, destinationArchiveFilePath)
createArchive(t, sourceDirPath, destinationArchiveFilePath, layer2)
t.Cleanup(
assertNoError(t,
@@ -0,0 +1 @@
Another .vimrc file
-68
View File
@@ -1,68 +0,0 @@
package cli
import (
"strings"
"testing"
)
func TestAttestCmd(t *testing.T) {
img := "registry:busybox:latest"
tests := []struct {
name string
args []string
env map[string]string
assertions []traitAssertion
pw string
}{
{
name: "no-args-shows-help",
args: []string{"attest"},
assertions: []traitAssertion{
assertInOutput("an image/directory argument is required"), // specific error that should be shown
assertInOutput("from a container image as the predicate of an in-toto attestation"), // excerpt from help description
assertFailingReturnCode,
},
pw: "",
},
{
name: "can encode syft.json as the predicate given a password",
args: []string{"attest", "-o", "json", "--key", "cosign.key", img},
assertions: []traitAssertion{
assertSuccessfulReturnCode,
},
pw: "test",
},
{
name: "can encode syft.json as the predicate given a blank password",
args: []string{"attest", "-o", "json", "--key", "cosign.key", img},
assertions: []traitAssertion{
assertSuccessfulReturnCode,
},
pw: "",
},
{
name: "can encode syft.json as the predicate given a user format typo",
args: []string{"attest", "-o", "spdx-jsonx", "--key", "cosign.key", img},
assertions: []traitAssertion{
assertSuccessfulReturnCode,
},
pw: "",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
cleanup := setupPKI(t, test.pw)
defer cleanup()
cmd, stdout, stderr := runSyft(t, test.env, test.args...)
for _, traitFn := range test.assertions {
traitFn(t, stdout, stderr, cmd.ProcessState.ExitCode())
}
if t.Failed() {
t.Log("STDOUT:\n", stdout)
t.Log("STDERR:\n", stderr)
t.Log("COMMAND:", strings.Join(cmd.Args, " "))
}
})
}
}
-150
View File
@@ -1,150 +0,0 @@
package cli
import (
"net/http"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestCosignWorkflow(t *testing.T) {
// found under test-fixtures/registry/Makefile
img := "localhost:5000/attest:latest"
attestationFile := "attestation.json"
tests := []struct {
name string
syftArgs []string
cosignAttachArgs []string
cosignVerifyArgs []string
env map[string]string
assertions []traitAssertion
setup func(*testing.T)
cleanup func()
}{
{
name: "cosign verify syft attest",
syftArgs: []string{
"attest",
"-o",
"json",
"--key",
"cosign.key",
img,
},
// cosign attach attestation
cosignAttachArgs: []string{
"attach",
"attestation",
"--attestation",
attestationFile,
img,
},
// cosign verify-attestation
cosignVerifyArgs: []string{
"verify-attestation",
"-key",
"cosign.pub",
img,
},
assertions: []traitAssertion{
assertSuccessfulReturnCode,
},
setup: func(t *testing.T) {
cwd, err := os.Getwd()
require.NoErrorf(t, err, "unable to get cwd: %+v", err)
// get working directory for local registry
fixturesPath := filepath.Join(cwd, "test-fixtures", "registry")
makeTask := filepath.Join(fixturesPath, "Makefile")
t.Logf("Generating Fixture from 'make %s'", makeTask)
cmd := exec.Command("make")
cmd.Dir = fixturesPath
runAndShow(t, cmd)
var done = make(chan struct{})
defer close(done)
for interval := range testRetryIntervals(done) {
resp, err := http.Get("http://127.0.0.1:5000/v2/")
if err != nil {
t.Logf("waiting for registry err=%+v", err)
} else {
if resp.StatusCode == http.StatusOK {
break
}
t.Logf("waiting for registry code=%+v", resp.StatusCode)
}
time.Sleep(interval)
}
cmd = exec.Command("make", "push")
cmd.Dir = fixturesPath
runAndShow(t, cmd)
},
cleanup: func() {
cwd, err := os.Getwd()
assert.NoErrorf(t, err, "unable to get cwd: %+v", err)
fixturesPath := filepath.Join(cwd, "test-fixtures", "registry")
makeTask := filepath.Join(fixturesPath, "Makefile")
t.Logf("Generating Fixture from 'make %s'", makeTask)
// delete attestation file
os.Remove(attestationFile)
cmd := exec.Command("make", "stop")
cmd.Dir = fixturesPath
runAndShow(t, cmd)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Cleanup(tt.cleanup)
tt.setup(t)
pkiCleanup := setupPKI(t, "") // blank password
defer pkiCleanup()
// attest
cmd, stdout, stderr := runSyft(t, tt.env, tt.syftArgs...)
for _, traitFn := range tt.assertions {
traitFn(t, stdout, stderr, cmd.ProcessState.ExitCode())
}
checkCmdFailure(t, stdout, stderr, cmd)
require.NoError(t, os.WriteFile(attestationFile, []byte(stdout), 0666))
// attach
cmd, stdout, stderr = runCosign(t, tt.env, tt.cosignAttachArgs...)
for _, traitFn := range tt.assertions {
traitFn(t, stdout, stderr, cmd.ProcessState.ExitCode())
}
checkCmdFailure(t, stdout, stderr, cmd)
// attest
cmd, stdout, stderr = runCosign(t, tt.env, tt.cosignAttachArgs...)
for _, traitFn := range tt.assertions {
traitFn(t, stdout, stderr, cmd.ProcessState.ExitCode())
}
checkCmdFailure(t, stdout, stderr, cmd)
})
}
}
func checkCmdFailure(t testing.TB, stdout, stderr string, cmd *exec.Cmd) {
require.Falsef(t, t.Failed(), "%s %s trait assertion failed", cmd.Path, strings.Join(cmd.Args, " "))
if t.Failed() {
t.Log("STDOUT:\n", stdout)
t.Log("STDERR:\n", stderr)
t.Log("COMMAND:", strings.Join(cmd.Args, " "))
}
}
+25
View File
@@ -204,6 +204,31 @@ func TestPackagesCmdFlags(t *testing.T) {
assertSuccessfulReturnCode,
},
},
{
name: "override-default-parallelism",
args: []string{"packages", "-vvv", "-o", "json", coverageImage},
env: map[string]string{
"SYFT_PARALLELISM": "2",
},
assertions: []traitAssertion{
// the application config in the log matches that of what we expect to have been configured.
assertInOutput("parallelism: 2"),
assertInOutput("parallelism=2"),
assertPackageCount(34),
assertSuccessfulReturnCode,
},
},
{
name: "default-parallelism",
args: []string{"packages", "-vvv", "-o", "json", coverageImage},
assertions: []traitAssertion{
// the application config in the log matches that of what we expect to have been configured.
assertInOutput("parallelism: 1"),
assertInOutput("parallelism=1"),
assertPackageCount(34),
assertSuccessfulReturnCode,
},
},
}
for _, test := range tests {

Some files were not shown because too many files have changed in this diff Show More