mirror of
https://github.com/anchore/syft.git
synced 2026-10-25 11:32:22 +01:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f24bbc1838 | ||
|
|
cb3e73e308 | ||
|
|
f157d7a862 | ||
|
|
028cd9e27e | ||
|
|
5549939cc6 | ||
|
|
47ea910868 |
@@ -9,15 +9,7 @@
|
||||
[](https://github.com/anchore/syft/blob/main/LICENSE)
|
||||
[](https://anchore.com/slack)
|
||||
|
||||
A CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner tool like [Grype](https://github.com/anchore/grype).
|
||||
|
||||
### Join our Virtual OSS Meetup!
|
||||
|
||||
You are invited to join us on March 23rd, 11AM-Noon PT for our virtual open source meetup.
|
||||
|
||||
Hosts Priyanka “Pinky” Ravi of Weaveworks and Dan Luhring from Anchore OSS will explore how open source project Flux is utilizing Syft-generated SBOMs to secure the software supply chain.
|
||||
|
||||
[Register here!](https://get.anchore.com/anchore-oss-meetup-mar-23-2022/)
|
||||
A CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner like [Grype](https://github.com/anchore/grype).
|
||||
|
||||
### Join our community meetings!
|
||||
|
||||
@@ -28,15 +20,26 @@ Hosts Priyanka “Pinky” Ravi of Weaveworks and Dan Luhring from Anchore OSS w
|
||||

|
||||
|
||||
## Features
|
||||
- Catalog container images and filesystems to discover packages and libraries.
|
||||
- Generate in-toto attestations where an SBOM is included as the payload.
|
||||
- Supports packages and libraries from various ecosystems (APK, DEB, RPM, Ruby Bundles, Python Wheel/Egg/requirements.txt, JavaScript NPM/Yarn, Java JAR/EAR/WAR/PAR/SAR, Jenkins plugins JPI/HPI, Go modules, PHP Composer)
|
||||
- Linux distribution identification (supports Alpine, BusyBox, CentOS/RedHat, Debian/Ubuntu flavored distributions)
|
||||
- Supports Docker and OCI image formats
|
||||
- Direct support for [Grype](https://github.com/anchore/grype), a fast and powerful vulnerability matcher.
|
||||
- Generates SBOMs for container images, filesystems, archives, and more to discover packages and libraries
|
||||
- Supports OCI and Docker image formats
|
||||
- Linux distribution identification
|
||||
- Works seamlessly with [Grype](https://github.com/anchore/grype) (a fast, modern vulnerability scanner)
|
||||
- Able to create signed SBOM attestations using the [in-toto specification](https://github.com/in-toto/attestation/blob/main/spec/README.md)
|
||||
|
||||
### Supported Ecosystems
|
||||
|
||||
If you encounter an issue, please [let us know using the issue tracker](https://github.com/anchore/syft/issues).
|
||||
- Alpine (apk)
|
||||
- Dart (pubs)
|
||||
- Debian (dpkg)
|
||||
- Go (go.mod, Go binaries)
|
||||
- Java (jar, ear, war, par, sar)
|
||||
- JavaScript (npm, yarn)
|
||||
- Jenkins Plugins (jpi, hpi)
|
||||
- PHP (composer)
|
||||
- Python (wheel, egg, poetry, requirements.txt)
|
||||
- Red Hat (rpm)
|
||||
- Ruby (gem)
|
||||
- Rust (cargo.lock)
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -61,15 +64,13 @@ brew install syft
|
||||
|
||||
### Nix
|
||||
|
||||
**Note**: nix packaging of Syft is [community maintained](https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/syft/default.nix)
|
||||
|
||||
Also syft is currently only in the [unstable channel](https://nixos.wiki/wiki/Nix_channels#The_official_channels) awaiting the `22.05` release
|
||||
**Note**: Nix packaging of Syft is [community maintained](https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/syft/default.nix). Syft is currently only in the [unstable channel](https://nixos.wiki/wiki/Nix_channels#The_official_channels) awaiting the `22.05` release
|
||||
|
||||
```bash
|
||||
nix-env -i syft
|
||||
```
|
||||
|
||||
... or, just try it out in an ephemeral nix shell
|
||||
... or, just try it out in an ephemeral nix shell:
|
||||
|
||||
```bash
|
||||
nix-shell -p syft
|
||||
@@ -78,33 +79,28 @@ nix-shell -p syft
|
||||
## Getting started
|
||||
|
||||
#### SBOM
|
||||
To generate an SBOM for an OCI image:
|
||||
|
||||
To generate an SBOM for a container image:
|
||||
|
||||
```
|
||||
syft <image>
|
||||
```
|
||||
|
||||
**Note**: This is equivalent to specifying the `packages` subcommand:
|
||||
```
|
||||
syft packages <image>
|
||||
```
|
||||
|
||||
The above output includes only software that is visible in the container (i.e., the squashed representation of the image).
|
||||
To include software from all image layers in the SBOM, regardless of its presence in the final image, provide `--scope all-layers`:
|
||||
The above output includes only software that is visible in the container (i.e., the squashed representation of the image). To include software from all image layers in the SBOM, regardless of its presence in the final image, provide `--scope all-layers`:
|
||||
|
||||
```
|
||||
syft packages <image> --scope all-layers
|
||||
syft <image> --scope all-layers
|
||||
```
|
||||
|
||||
#### SBOM Attestation
|
||||
To generate an attested SBOM for an OCI image as the predicate of an in-toto attestation
|
||||
#### SBOM attestation
|
||||
|
||||
To generate an SBOM attestation for a container image:
|
||||
```
|
||||
syft attest --output [FORMAT] --key [KEY] [SOURCE] [flags]
|
||||
```
|
||||
|
||||
The above output is in the form of the [DSSE envelope](https://github.com/secure-systems-lab/dsse/blob/master/envelope.md#dsse-envelope).
|
||||
The payload is a base64 encoded `in-toto` statement with the SBOM as the predicate, the payload type is `application/vnd.in-toto+json`, and the signatures array is populated
|
||||
with the contents needed for public key verification. For details on workflows using this command see [here](#adding-an-sbom-to-an-image-as-an-attestation-using-syft).
|
||||
|
||||
The payload is a base64 encoded `in-toto` statement with the generated SBOM as the predicate. For details on workflows using this command see [here](#adding-an-sbom-to-an-image-as-an-attestation-using-syft).
|
||||
|
||||
### Supported sources
|
||||
|
||||
@@ -112,10 +108,10 @@ Syft can generate a SBOM from a variety of sources:
|
||||
|
||||
```
|
||||
# catalog a container image archive (from the result of `docker image save ...`, `podman save ...`, or `skopeo copy` commands)
|
||||
syft packages path/to/image.tar
|
||||
syft path/to/image.tar
|
||||
|
||||
# catalog a directory
|
||||
syft packages path/to/dir
|
||||
syft path/to/dir
|
||||
```
|
||||
|
||||
Sources can be explicitly provided with a scheme:
|
||||
@@ -154,7 +150,7 @@ The output format for Syft is configurable as well using the
|
||||
`-o` (or `--output`) option:
|
||||
|
||||
```
|
||||
syft packages <image> -o <format>
|
||||
syft <image> -o <format>
|
||||
```
|
||||
|
||||
Where the `formats` available are:
|
||||
@@ -172,20 +168,16 @@ Syft can also output _multiple_ files in differing formats by appending
|
||||
`=<file>` to the option, for example to output Syft JSON and SPDX JSON:
|
||||
|
||||
```shell
|
||||
syft packages <image> -o json=sbom.syft.json -o spdx-json=sbom.spdx.json
|
||||
syft <image> -o json=sbom.syft.json -o spdx-json=sbom.spdx.json
|
||||
```
|
||||
|
||||
## Private Registry Authentication
|
||||
|
||||
### Local Docker Credentials
|
||||
When a container runtime is not present, Syft can still utilize credentials configured in common credential sources (such as `~/.docker/config.json`).
|
||||
It will pull images from private registries using these credentials. The config file is where your credentials are stored when authenticating with private registries via some command like `docker login`.
|
||||
For more information see the `go-containerregistry` [documentation](https://github.com/google/go-containerregistry/tree/main/pkg/authn).
|
||||
|
||||
When a container runtime is not present, Syft can still utilize credentials configured in common credential sources (such as `~/.docker/config.json`). It will pull images from private registries using these credentials. The config file is where your credentials are stored when authenticating with private registries via some command like `docker login`. For more information see the `go-containerregistry` [documentation](https://github.com/google/go-containerregistry/tree/main/pkg/authn).
|
||||
|
||||
An example `config.json` looks something like this:
|
||||
```
|
||||
// config.json
|
||||
```json
|
||||
{
|
||||
"auths": {
|
||||
"registry.example.com": {
|
||||
@@ -198,15 +190,17 @@ An example `config.json` looks something like this:
|
||||
|
||||
You can run the following command as an example. It details the mount/environment configuration a container needs to access a private registry:
|
||||
|
||||
`docker run -v ./config.json:/config/config.json -e "DOCKER_CONFIG=/config" anchore/syft:latest <private_image>`
|
||||
|
||||
```
|
||||
docker run -v ./config.json:/config/config.json -e "DOCKER_CONFIG=/config" anchore/syft:latest <private_image>
|
||||
```
|
||||
|
||||
### Docker Credentials in Kubernetes
|
||||
The below section shows a simple workflow on how to mount this config file as a secret into a container on kubernetes.
|
||||
1. Create a secret. The value of `config.json` is important. It refers to the specification detailed [here](https://github.com/google/go-containerregistry/tree/main/pkg/authn#the-config-file).
|
||||
Below this section is the `secret.yaml` file that the pod configuration will consume as a volume.
|
||||
The key `config.json` is important. It will end up being the name of the file when mounted into the pod.
|
||||
```
|
||||
|
||||
Here's a simple workflow to mount this config file as a secret into a container on Kubernetes.
|
||||
|
||||
1. Create a secret. The value of `config.json` is important. It refers to the specification detailed [here](https://github.com/google/go-containerregistry/tree/main/pkg/authn#the-config-file). Below this section is the `secret.yaml` file that the pod configuration will consume as a volume. The key `config.json` is important. It will end up being the name of the file when mounted into the pod.
|
||||
|
||||
```yaml
|
||||
# secret.yaml
|
||||
|
||||
apiVersion: v1
|
||||
@@ -221,11 +215,9 @@ The key `config.json` is important. It will end up being the name of the file wh
|
||||
`kubectl apply -f secret.yaml`
|
||||
|
||||
|
||||
2. Create your pod running syft. The env `DOCKER_CONFIG` is important because it advertises where to look for the credential file.
|
||||
In the below example, setting `DOCKER_CONFIG=/config` informs syft that credentials can be found at `/config/config.json`.
|
||||
This is why we used `config.json` as the key for our secret. When mounted into containers the secrets' key is used as the filename.
|
||||
The `volumeMounts` section mounts our secret to `/config`. The `volumes` section names our volume and leverages the secret we created in step one.
|
||||
```
|
||||
2. Create your pod running syft. The env `DOCKER_CONFIG` is important because it advertises where to look for the credential file. In the below example, setting `DOCKER_CONFIG=/config` informs syft that credentials can be found at `/config/config.json`. This is why we used `config.json` as the key for our secret. When mounted into containers the secrets' key is used as the filename. The `volumeMounts` section mounts our secret to `/config`. The `volumes` section names our volume and leverages the secret we created in step one.
|
||||
|
||||
```yaml
|
||||
# pod.yaml
|
||||
|
||||
apiVersion: v1
|
||||
@@ -254,10 +246,9 @@ The `volumeMounts` section mounts our secret to `/config`. The `volumes` section
|
||||
`kubectl apply -f pod.yaml`
|
||||
|
||||
|
||||
3. The user can now run `kubectl logs syft-private-registry-demo`. The logs should show the syft analysis for the `<private_image>` provided in the pod configuration.
|
||||
3. The user can now run `kubectl logs syft-private-registry-demo`. The logs should show the Syft analysis for the `<private_image>` provided in the pod configuration.
|
||||
|
||||
Using the above information, users should be able to configure private registry access without having to do so in the `grype` or `syft` configuration files.
|
||||
They will also not be dependent on a docker daemon, (or some other runtime software) for registry configuration and access.
|
||||
Using the above information, users should be able to configure private registry access without having to do so in the `grype` or `syft` configuration files. They will also not be dependent on a Docker daemon, (or some other runtime software) for registry configuration and access.
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -472,26 +463,25 @@ anchore:
|
||||
|
||||
`syft attest --output [FORMAT] --key [KEY] [SOURCE] [flags]`
|
||||
|
||||
SBOMs themselves can serve as input to different analysis tools. The Anchore organization offers the vulnerability scanner
|
||||
[grype](https://github.com/anchore/grype) as one such tool.
|
||||
One of the foundational approaches to "trust" between tools is for producers to use the artifacts generated by syft as attestations to their images.
|
||||
The DSSE output of `syft attest` can be used with the [cosign](https://github.com/sigstore/cosign) tool to attach an attestation to an image.
|
||||
SBOMs themselves can serve as input to different analysis tools. [Grype](https://github.com/anchore/grype), a vulnerability scanner CLI tool from Anchore, is one such tool. Publishers of container images can use attestations to enable their consumers to trust Syft-generated SBOM descriptions of those container images. To create and provide these attestations, image publishers can run `syft attest` in conjunction with the [cosign](https://github.com/sigstore/cosign) tool to attach SBOM attestations to their images.
|
||||
|
||||
#### Example attest
|
||||
#### Example attestation
|
||||
Note for the following example replace `docker.io/image:latest` with an image you own. You should also have push access to
|
||||
its remote reference. Replace $MY_PRIVATE_KEY with a private key you own or have generated with cosign.
|
||||
its remote reference. Replace `$MY_PRIVATE_KEY` with a private key you own or have generated with cosign.
|
||||
|
||||
```bash
|
||||
syft attest --key $MY_PRIVATE_KEY docker.io/image:latest > image_latest_sbom_attestation.json
|
||||
cosign attach attestation --attestation image_latest_sbom_attestation.json docker.io/image:latest
|
||||
```
|
||||
|
||||
Verify the new attestation exists on your image
|
||||
Verify the new attestation exists on your image.
|
||||
|
||||
```bash
|
||||
cosign verify-attestation -key $MY_PUBLIC_KEY docker.io/image:latest | jq '.payload | @base64d | .payload | fromjson | .predicate'
|
||||
```
|
||||
|
||||
You should see this output along with the attached SBOM.
|
||||
You should see this output along with the attached SBOM:
|
||||
|
||||
```
|
||||
Verification for docker.io/image:latest --
|
||||
The following checks were performed on each of these signatures:
|
||||
|
||||
@@ -14,7 +14,7 @@ require (
|
||||
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
|
||||
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
|
||||
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b
|
||||
github.com/anchore/stereoscope v0.0.0-20220323180633-4bdc70599cde
|
||||
github.com/anchore/stereoscope v0.0.0-20220330165332-7fc73ee7b0f0
|
||||
github.com/antihax/optional v1.0.0
|
||||
github.com/bmatcuk/doublestar/v4 v4.0.2
|
||||
github.com/dustin/go-humanize v1.0.0
|
||||
|
||||
@@ -280,8 +280,8 @@ github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZV
|
||||
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
|
||||
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b h1:YJWYt/6KQXR9JR46lLHrTTYi8rcye42tKcyjREA/hvA=
|
||||
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b/go.mod h1:Blo6OgJNiYF41ufcgHKkbCKF2MDOMlrqhXv/ij6ocR4=
|
||||
github.com/anchore/stereoscope v0.0.0-20220323180633-4bdc70599cde h1:yOi6g1D2q3WwzID0QQyF+x9VkzUyGSJy3cEYbp7ZOyk=
|
||||
github.com/anchore/stereoscope v0.0.0-20220323180633-4bdc70599cde/go.mod h1:OVVJ4y/L26pspeeHNvNa7GeQfoLPWxJe13iODl9x5l4=
|
||||
github.com/anchore/stereoscope v0.0.0-20220330165332-7fc73ee7b0f0 h1:mObz7bepZ6DtbIrsB2mxuOE9XEYmVtA/P/EDlKwfbjs=
|
||||
github.com/anchore/stereoscope v0.0.0-20220330165332-7fc73ee7b0f0/go.mod h1:yoCLUZY0k/pYLNIy0L80p2Ko0PKVNXm8rHtgxp4OiSc=
|
||||
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
|
||||
github.com/andybalholm/brotli v1.0.0/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
|
||||
github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
|
||||
@@ -1877,7 +1877,6 @@ github.com/sassoftware/relic v0.0.0-20210427151427-dfb082b79b74 h1:sUNzanSKA9z/h
|
||||
github.com/sassoftware/relic v0.0.0-20210427151427-dfb082b79b74/go.mod h1:YlB8wFIZmFLZ1JllNBfSURzz52fBxbliNgYALk1UDmk=
|
||||
github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
|
||||
github.com/sclevine/spec v1.4.0/go.mod h1:LvpgJaFyvQzRvc1kaDs0bulYwzC70PbiYjC4QnFHkOM=
|
||||
github.com/scylladb/go-set v1.0.2/go.mod h1:DkpGd78rljTxKAnTDPFqXSGxvETQnJyuSOQwsHycqfs=
|
||||
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e h1:7q6NSFZDeGfvvtIRwBrU/aegEYJYmvev0cHAwo17zZQ=
|
||||
github.com/scylladb/go-set v1.0.3-0.20200225121959-cc7b2070d91e/go.mod h1:DkpGd78rljTxKAnTDPFqXSGxvETQnJyuSOQwsHycqfs=
|
||||
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
|
||||
|
||||
@@ -56,14 +56,14 @@ func sbomFixture() sbom.SBOM {
|
||||
Name: "name",
|
||||
Version: "version",
|
||||
FoundBy: "foundBy",
|
||||
Locations: []source.Location{
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "path",
|
||||
FileSystemID: "layerID",
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
Licenses: []string{"license"},
|
||||
Language: pkg.Python,
|
||||
Type: pkg.PythonPkg,
|
||||
|
||||
@@ -6,5 +6,5 @@ const (
|
||||
|
||||
// JSONSchemaVersion is the current schema version output by the JSON encoder
|
||||
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
|
||||
JSONSchemaVersion = "3.2.1"
|
||||
JSONSchemaVersion = "3.2.2"
|
||||
)
|
||||
|
||||
@@ -13,8 +13,9 @@ import (
|
||||
func encodeComponent(p pkg.Package) cyclonedx.Component {
|
||||
props := encodeProperties(p, "syft:package")
|
||||
props = append(props, encodeCPEs(p)...)
|
||||
if len(p.Locations) > 0 {
|
||||
props = append(props, encodeProperties(p.Locations, "syft:location")...)
|
||||
locations := p.Locations.ToSlice()
|
||||
if len(locations) > 0 {
|
||||
props = append(props, encodeProperties(locations, "syft:location")...)
|
||||
}
|
||||
if hasMetadata(p) {
|
||||
props = append(props, encodeProperties(p.Metadata, "syft:metadata")...)
|
||||
@@ -73,10 +74,13 @@ func decodeComponent(c *cyclonedx.Component) *pkg.Package {
|
||||
return p
|
||||
}
|
||||
|
||||
func decodeLocations(vals map[string]string) []source.Location {
|
||||
func decodeLocations(vals map[string]string) source.LocationSet {
|
||||
v := common.Decode(reflect.TypeOf([]source.Location{}), vals, "syft:location", CycloneDXFields)
|
||||
out, _ := v.([]source.Location)
|
||||
return out
|
||||
out, ok := v.([]source.Location)
|
||||
if !ok {
|
||||
out = nil
|
||||
}
|
||||
return source.NewLocationSet(out...)
|
||||
}
|
||||
|
||||
func decodePackageMetadata(vals map[string]string, c *cyclonedx.Component, typ pkg.MetadataType) interface{} {
|
||||
|
||||
@@ -26,9 +26,9 @@ func Test_encodeComponentProperties(t *testing.T) {
|
||||
name: "from apk",
|
||||
input: pkg.Package{
|
||||
FoundBy: "cataloger",
|
||||
Locations: []source.Location{
|
||||
{Coordinates: source.Coordinates{RealPath: "test"}},
|
||||
},
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{Coordinates: source.Coordinates{RealPath: "test"}},
|
||||
),
|
||||
Metadata: pkg.ApkMetadata{
|
||||
Package: "libc-utils",
|
||||
OriginPackage: "libc-dev",
|
||||
|
||||
@@ -13,6 +13,8 @@ func SourceInfo(p pkg.Package) string {
|
||||
answer = "acquired package info from RPM DB"
|
||||
case pkg.ApkPkg:
|
||||
answer = "acquired package info from APK DB"
|
||||
case pkg.DartPubPkg:
|
||||
answer = "acquired package info from pubspec manifest"
|
||||
case pkg.DebPkg:
|
||||
answer = "acquired package info from DPKG DB"
|
||||
case pkg.NpmPkg:
|
||||
@@ -33,7 +35,7 @@ func SourceInfo(p pkg.Package) string {
|
||||
answer = "acquired package info from the following paths"
|
||||
}
|
||||
var paths []string
|
||||
for _, l := range p.Locations {
|
||||
for _, l := range p.Locations.ToSlice() {
|
||||
paths = append(paths, l.RealPath)
|
||||
}
|
||||
|
||||
|
||||
@@ -18,10 +18,10 @@ func Test_SourceInfo(t *testing.T) {
|
||||
name: "locations are captured",
|
||||
input: pkg.Package{
|
||||
// note: no type given
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/a-place", "/b-place"),
|
||||
source.NewVirtualLocation("/c-place", "/d-place"),
|
||||
},
|
||||
),
|
||||
},
|
||||
expected: []string{
|
||||
"from the following paths",
|
||||
@@ -126,6 +126,14 @@ func Test_SourceInfo(t *testing.T) {
|
||||
"from PHP composer manifest",
|
||||
},
|
||||
},
|
||||
{
|
||||
input: pkg.Package{
|
||||
Type: pkg.DartPubPkg,
|
||||
},
|
||||
expected: []string{
|
||||
"from pubspec manifest",
|
||||
},
|
||||
},
|
||||
}
|
||||
var pkgTypes []pkg.Type
|
||||
for _, test := range tests {
|
||||
|
||||
@@ -157,9 +157,9 @@ func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
|
||||
catalog.Add(pkg.Package{
|
||||
Name: "package-1",
|
||||
Version: "1.0.1",
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewLocationFromImage(string(ref1.RealPath), *ref1, img),
|
||||
},
|
||||
),
|
||||
Type: pkg.PythonPkg,
|
||||
FoundBy: "the-cataloger-1",
|
||||
Language: pkg.Python,
|
||||
@@ -177,9 +177,9 @@ func populateImageCatalog(catalog *pkg.Catalog, img *image.Image) {
|
||||
catalog.Add(pkg.Package{
|
||||
Name: "package-2",
|
||||
Version: "2.0.1",
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewLocationFromImage(string(ref2.RealPath), *ref2, img),
|
||||
},
|
||||
),
|
||||
Type: pkg.DebPkg,
|
||||
FoundBy: "the-cataloger-2",
|
||||
MetadataType: pkg.DpkgMetadataType,
|
||||
@@ -234,9 +234,9 @@ func newDirectoryCatalog() *pkg.Catalog {
|
||||
Version: "1.0.1",
|
||||
Type: pkg.PythonPkg,
|
||||
FoundBy: "the-cataloger-1",
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewLocation("/some/path/pkg1"),
|
||||
},
|
||||
),
|
||||
Language: pkg.Python,
|
||||
MetadataType: pkg.PythonPackageMetadataType,
|
||||
Licenses: []string{"MIT"},
|
||||
@@ -259,9 +259,9 @@ func newDirectoryCatalog() *pkg.Catalog {
|
||||
Version: "2.0.1",
|
||||
Type: pkg.DebPkg,
|
||||
FoundBy: "the-cataloger-2",
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewLocation("/some/path/pkg1"),
|
||||
},
|
||||
),
|
||||
MetadataType: pkg.DpkgMetadataType,
|
||||
Metadata: pkg.DpkgMetadata{
|
||||
Package: "package-2",
|
||||
|
||||
@@ -58,8 +58,9 @@ func toSnapshotMetadata(s *sbom.SBOM) Metadata {
|
||||
}
|
||||
|
||||
func filesystem(p pkg.Package) string {
|
||||
if len(p.Locations) > 0 {
|
||||
return p.Locations[0].FileSystemID
|
||||
locations := p.Locations.ToSlice()
|
||||
if len(locations) > 0 {
|
||||
return locations[0].FileSystemID
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -76,8 +77,9 @@ func toPath(s source.Metadata, p pkg.Package) string {
|
||||
if inputPath == "." {
|
||||
inputPath = ""
|
||||
}
|
||||
if len(p.Locations) > 0 {
|
||||
location := p.Locations[0]
|
||||
locations := p.Locations.ToSlice()
|
||||
if len(locations) > 0 {
|
||||
location := locations[0]
|
||||
packagePath := location.RealPath
|
||||
if location.VirtualPath != "" {
|
||||
packagePath = location.VirtualPath
|
||||
|
||||
@@ -35,32 +35,38 @@ func Test_toGithubModel(t *testing.T) {
|
||||
{
|
||||
Name: "pkg-1",
|
||||
Version: "1.0.1",
|
||||
Locations: []source.Location{{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/usr/lib",
|
||||
FileSystemID: "fsid-1",
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/usr/lib",
|
||||
FileSystemID: "fsid-1",
|
||||
},
|
||||
},
|
||||
}},
|
||||
),
|
||||
},
|
||||
{
|
||||
Name: "pkg-2",
|
||||
Version: "2.0.2",
|
||||
Locations: []source.Location{{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/usr/lib",
|
||||
FileSystemID: "fsid-1",
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/usr/lib",
|
||||
FileSystemID: "fsid-1",
|
||||
},
|
||||
},
|
||||
}},
|
||||
),
|
||||
},
|
||||
{
|
||||
Name: "pkg-3",
|
||||
Version: "3.0.3",
|
||||
Locations: []source.Location{{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/etc",
|
||||
FileSystemID: "fsid-1",
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/etc",
|
||||
FileSystemID: "fsid-1",
|
||||
},
|
||||
},
|
||||
}},
|
||||
),
|
||||
},
|
||||
} {
|
||||
p.PURL = packageurl.NewPackageURL(
|
||||
|
||||
+5
-5
@@ -3,18 +3,18 @@
|
||||
"name": "/some/path",
|
||||
"spdxVersion": "SPDX-2.2",
|
||||
"creationInfo": {
|
||||
"created": "2021-12-20T19:12:47.869816Z",
|
||||
"created": "2022-03-30T21:48:28.297464Z",
|
||||
"creators": [
|
||||
"Organization: Anchore, Inc",
|
||||
"Tool: syft-[not provided]"
|
||||
],
|
||||
"licenseListVersion": "3.15"
|
||||
"licenseListVersion": "3.16"
|
||||
},
|
||||
"dataLicense": "CC0-1.0",
|
||||
"documentNamespace": "https://anchore.com/syft/dir/some/path-4b896ded-7852-4e31-b764-136b53bdf346",
|
||||
"documentNamespace": "https://anchore.com/syft/dir/some/path-e188d59b-76f6-4c7f-a9f2-1ae7d0577781",
|
||||
"packages": [
|
||||
{
|
||||
"SPDXID": "SPDXRef-1d97af55efe9512f",
|
||||
"SPDXID": "SPDXRef-b85dbb4e6ece5082",
|
||||
"name": "package-1",
|
||||
"licenseConcluded": "MIT",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
@@ -36,7 +36,7 @@
|
||||
"versionInfo": "1.0.1"
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-ad3d1c4abd84bf75",
|
||||
"SPDXID": "SPDXRef-ceda99598967ae8d",
|
||||
"name": "package-2",
|
||||
"licenseConcluded": "NONE",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
|
||||
@@ -3,18 +3,18 @@
|
||||
"name": "user-image-input",
|
||||
"spdxVersion": "SPDX-2.2",
|
||||
"creationInfo": {
|
||||
"created": "2021-12-20T19:13:07.647486Z",
|
||||
"created": "2022-03-30T21:48:28.303986Z",
|
||||
"creators": [
|
||||
"Organization: Anchore, Inc",
|
||||
"Tool: syft-[not provided]"
|
||||
],
|
||||
"licenseListVersion": "3.15"
|
||||
"licenseListVersion": "3.16"
|
||||
},
|
||||
"dataLicense": "CC0-1.0",
|
||||
"documentNamespace": "https://anchore.com/syft/image/user-image-input-174da656-1824-4bd3-8604-28919f8a65bc",
|
||||
"documentNamespace": "https://anchore.com/syft/image/user-image-input-9e4f4190-c5ae-4e31-a852-d1ab71357516",
|
||||
"packages": [
|
||||
{
|
||||
"SPDXID": "SPDXRef-d16127444133b5c1",
|
||||
"SPDXID": "SPDXRef-2a46171f91c8d4bc",
|
||||
"name": "package-1",
|
||||
"licenseConcluded": "MIT",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
@@ -36,7 +36,7 @@
|
||||
"versionInfo": "1.0.1"
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-24907357f3705420",
|
||||
"SPDXID": "SPDXRef-ae77680e9b1d087e",
|
||||
"name": "package-2",
|
||||
"licenseConcluded": "NONE",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
|
||||
BIN
Binary file not shown.
+4
-4
@@ -2,16 +2,16 @@ SPDXVersion: SPDX-2.2
|
||||
DataLicense: CC0-1.0
|
||||
SPDXID: SPDXRef-DOCUMENT
|
||||
DocumentName: /some/path
|
||||
DocumentNamespace: https://anchore.com/syft/dir/some/path-4b90f56d-d596-4ad8-b6a5-17f7d801350d
|
||||
DocumentNamespace: https://anchore.com/syft/dir/some/path-71aa3553-1a73-405f-9f1f-6347d6d4593b
|
||||
LicenseListVersion: 3.16
|
||||
Creator: Organization: Anchore, Inc
|
||||
Creator: Tool: syft-[not provided]
|
||||
Created: 2022-02-10T21:09:27Z
|
||||
Created: 2022-03-30T21:48:22Z
|
||||
|
||||
##### Package: package-2
|
||||
|
||||
PackageName: package-2
|
||||
SPDXID: SPDXRef-Package-deb-package-2-ad3d1c4abd84bf75
|
||||
SPDXID: SPDXRef-Package-deb-package-2-ceda99598967ae8d
|
||||
PackageVersion: 2.0.1
|
||||
PackageDownloadLocation: NOASSERTION
|
||||
FilesAnalyzed: false
|
||||
@@ -24,7 +24,7 @@ ExternalRef: PACKAGE_MANAGER purl a-purl-2
|
||||
##### Package: package-1
|
||||
|
||||
PackageName: package-1
|
||||
SPDXID: SPDXRef-Package-python-package-1-1d97af55efe9512f
|
||||
SPDXID: SPDXRef-Package-python-package-1-b85dbb4e6ece5082
|
||||
PackageVersion: 1.0.1
|
||||
PackageDownloadLocation: NOASSERTION
|
||||
FilesAnalyzed: false
|
||||
|
||||
+4
-4
@@ -2,16 +2,16 @@ SPDXVersion: SPDX-2.2
|
||||
DataLicense: CC0-1.0
|
||||
SPDXID: SPDXRef-DOCUMENT
|
||||
DocumentName: user-image-input
|
||||
DocumentNamespace: https://anchore.com/syft/image/user-image-input-26a2def6-53d0-4504-b99a-a046832508ac
|
||||
DocumentNamespace: https://anchore.com/syft/image/user-image-input-e46e20f4-43a4-40e7-9f82-fd55b8a89e5f
|
||||
LicenseListVersion: 3.16
|
||||
Creator: Organization: Anchore, Inc
|
||||
Creator: Tool: syft-[not provided]
|
||||
Created: 2022-02-10T21:09:27Z
|
||||
Created: 2022-03-30T21:48:22Z
|
||||
|
||||
##### Package: package-2
|
||||
|
||||
PackageName: package-2
|
||||
SPDXID: SPDXRef-Package-deb-package-2-73f796c846875b9e
|
||||
SPDXID: SPDXRef-Package-deb-package-2-ae77680e9b1d087e
|
||||
PackageVersion: 2.0.1
|
||||
PackageDownloadLocation: NOASSERTION
|
||||
FilesAnalyzed: false
|
||||
@@ -24,7 +24,7 @@ ExternalRef: PACKAGE_MANAGER purl a-purl-2
|
||||
##### Package: package-1
|
||||
|
||||
PackageName: package-1
|
||||
SPDXID: SPDXRef-Package-python-package-1-d9527e708c11f8b9
|
||||
SPDXID: SPDXRef-Package-python-package-1-2a46171f91c8d4bc
|
||||
PackageVersion: 1.0.1
|
||||
PackageDownloadLocation: NOASSERTION
|
||||
FilesAnalyzed: false
|
||||
|
||||
@@ -42,13 +42,13 @@ func TestEncodeFullJSONDocument(t *testing.T) {
|
||||
p1 := pkg.Package{
|
||||
Name: "package-1",
|
||||
Version: "1.0.1",
|
||||
Locations: []source.Location{
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/a/place/a",
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
Type: pkg.PythonPkg,
|
||||
FoundBy: "the-cataloger-1",
|
||||
Language: pkg.Python,
|
||||
@@ -68,13 +68,13 @@ func TestEncodeFullJSONDocument(t *testing.T) {
|
||||
p2 := pkg.Package{
|
||||
Name: "package-2",
|
||||
Version: "2.0.1",
|
||||
Locations: []source.Location{
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/b/place/b",
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
Type: pkg.DebPkg,
|
||||
FoundBy: "the-cataloger-2",
|
||||
MetadataType: pkg.DpkgMetadataType,
|
||||
|
||||
@@ -130,6 +130,12 @@ func (p *Package) UnmarshalJSON(b []byte) error {
|
||||
return err
|
||||
}
|
||||
p.Metadata = payload
|
||||
case pkg.DartPubMetadataType:
|
||||
var payload pkg.DartPubMetadata
|
||||
if err := json.Unmarshal(unpacker.Metadata, &payload); err != nil {
|
||||
return err
|
||||
}
|
||||
p.Metadata = payload
|
||||
default:
|
||||
log.Warnf("unknown package metadata type=%q for packageID=%q", p.MetadataType, p.ID)
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"artifacts": [
|
||||
{
|
||||
"id": "1d97af55efe9512f",
|
||||
"id": "b85dbb4e6ece5082",
|
||||
"name": "package-1",
|
||||
"version": "1.0.1",
|
||||
"type": "python",
|
||||
@@ -36,7 +36,7 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "ad3d1c4abd84bf75",
|
||||
"id": "ceda99598967ae8d",
|
||||
"name": "package-2",
|
||||
"version": "2.0.1",
|
||||
"type": "deb",
|
||||
@@ -88,7 +88,7 @@
|
||||
}
|
||||
},
|
||||
"schema": {
|
||||
"version": "3.2.1",
|
||||
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.2.1.json"
|
||||
"version": "3.2.2",
|
||||
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.2.2.json"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"artifacts": [
|
||||
{
|
||||
"id": "d9a7c58726ab4bef",
|
||||
"id": "b3fa3ee64756b0c6",
|
||||
"name": "package-1",
|
||||
"version": "1.0.1",
|
||||
"type": "python",
|
||||
@@ -31,7 +31,7 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "ac462e450060da2c",
|
||||
"id": "b324f4d9ee5413fe",
|
||||
"name": "package-2",
|
||||
"version": "2.0.1",
|
||||
"type": "deb",
|
||||
@@ -184,7 +184,7 @@
|
||||
}
|
||||
},
|
||||
"schema": {
|
||||
"version": "3.2.1",
|
||||
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.2.1.json"
|
||||
"version": "3.2.2",
|
||||
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.2.2.json"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"artifacts": [
|
||||
{
|
||||
"id": "d9527e708c11f8b9",
|
||||
"id": "2a46171f91c8d4bc",
|
||||
"name": "package-1",
|
||||
"version": "1.0.1",
|
||||
"type": "python",
|
||||
@@ -32,7 +32,7 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "73f796c846875b9e",
|
||||
"id": "ae77680e9b1d087e",
|
||||
"name": "package-2",
|
||||
"version": "2.0.1",
|
||||
"type": "deb",
|
||||
@@ -111,7 +111,7 @@
|
||||
}
|
||||
},
|
||||
"schema": {
|
||||
"version": "3.2.1",
|
||||
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.2.1.json"
|
||||
"version": "3.2.2",
|
||||
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.2.2.json"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -175,8 +175,9 @@ func toPackageModel(p pkg.Package) model.Package {
|
||||
licenses = p.Licenses
|
||||
}
|
||||
|
||||
var coordinates = make([]source.Coordinates, len(p.Locations))
|
||||
for i, l := range p.Locations {
|
||||
locations := p.Locations.ToSlice()
|
||||
var coordinates = make([]source.Coordinates, len(locations))
|
||||
for i, l := range locations {
|
||||
coordinates[i] = l.Coordinates
|
||||
}
|
||||
|
||||
|
||||
@@ -53,7 +53,8 @@ func toSyftRelationships(doc *model.Document, catalog *pkg.Catalog, relationship
|
||||
|
||||
for _, p := range catalog.Sorted() {
|
||||
idMap[string(p.ID())] = p
|
||||
for _, l := range p.Locations {
|
||||
locations := p.Locations.ToSlice()
|
||||
for _, l := range locations {
|
||||
idMap[string(l.Coordinates.ID())] = l.Coordinates
|
||||
}
|
||||
}
|
||||
@@ -166,7 +167,7 @@ func toSyftPackage(p model.Package, idAliases map[string]string) pkg.Package {
|
||||
Name: p.Name,
|
||||
Version: p.Version,
|
||||
FoundBy: p.FoundBy,
|
||||
Locations: locations,
|
||||
Locations: source.NewLocationSet(locations...),
|
||||
Licenses: p.Licenses,
|
||||
Language: p.Language,
|
||||
Type: p.Type,
|
||||
|
||||
@@ -37,6 +37,7 @@ type artifactMetadataContainer struct {
|
||||
Cargo pkg.CargoPackageMetadata
|
||||
Go pkg.GolangBinMetadata
|
||||
Php pkg.PhpComposerJSONMetadata
|
||||
Dart pkg.DartPubMetadata
|
||||
}
|
||||
|
||||
func main() {
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+73
-16
@@ -10,11 +10,28 @@ import (
|
||||
"github.com/jinzhu/copier"
|
||||
)
|
||||
|
||||
type orderedIDSet struct {
|
||||
slice []artifact.ID
|
||||
}
|
||||
|
||||
func (s *orderedIDSet) add(ids ...artifact.ID) {
|
||||
loopNewIDs:
|
||||
for _, newID := range ids {
|
||||
for _, existingID := range s.slice {
|
||||
if existingID == newID {
|
||||
continue loopNewIDs
|
||||
}
|
||||
}
|
||||
s.slice = append(s.slice, newID)
|
||||
}
|
||||
}
|
||||
|
||||
// Catalog represents a collection of Packages.
|
||||
type Catalog struct {
|
||||
byID map[artifact.ID]Package
|
||||
idsByType map[Type][]artifact.ID
|
||||
idsByPath map[string][]artifact.ID // note: this is real path or virtual path
|
||||
idsByName map[string]orderedIDSet
|
||||
idsByType map[Type]orderedIDSet
|
||||
idsByPath map[string]orderedIDSet // note: this is real path or virtual path
|
||||
lock sync.RWMutex
|
||||
}
|
||||
|
||||
@@ -22,8 +39,9 @@ type Catalog struct {
|
||||
func NewCatalog(pkgs ...Package) *Catalog {
|
||||
catalog := Catalog{
|
||||
byID: make(map[artifact.ID]Package),
|
||||
idsByType: make(map[Type][]artifact.ID),
|
||||
idsByPath: make(map[string][]artifact.ID),
|
||||
idsByName: make(map[string]orderedIDSet),
|
||||
idsByType: make(map[Type]orderedIDSet),
|
||||
idsByPath: make(map[string]orderedIDSet),
|
||||
}
|
||||
|
||||
for _, p := range pkgs {
|
||||
@@ -55,7 +73,12 @@ func (c *Catalog) Package(id artifact.ID) *Package {
|
||||
|
||||
// PackagesByPath returns all packages that were discovered from the given path.
|
||||
func (c *Catalog) PackagesByPath(path string) []Package {
|
||||
return c.Packages(c.idsByPath[path])
|
||||
return c.Packages(c.idsByPath[path].slice)
|
||||
}
|
||||
|
||||
// PackagesByName returns all packages that were discovered with a matching name.
|
||||
func (c *Catalog) PackagesByName(name string) []Package {
|
||||
return c.Packages(c.idsByName[name].slice)
|
||||
}
|
||||
|
||||
// Packages returns all packages for the given ID.
|
||||
@@ -81,26 +104,58 @@ func (c *Catalog) Add(p Package) {
|
||||
id = p.ID()
|
||||
}
|
||||
|
||||
// store by package ID
|
||||
c.byID[id] = p
|
||||
if existing, exists := c.byID[id]; exists {
|
||||
// there is already a package with this fingerprint merge the existing record with the new one
|
||||
if err := existing.merge(p); err != nil {
|
||||
log.Warnf("failed to merge packages: %+v", err)
|
||||
} else {
|
||||
c.addPathsToIndex(p)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// store by package type
|
||||
c.idsByType[p.Type] = append(c.idsByType[p.Type], id)
|
||||
c.addToIndex(p)
|
||||
}
|
||||
|
||||
// store by file location paths
|
||||
func (c *Catalog) addToIndex(p Package) {
|
||||
c.byID[p.id] = p
|
||||
c.addNameToIndex(p)
|
||||
c.addTypeToIndex(p)
|
||||
c.addPathsToIndex(p)
|
||||
}
|
||||
|
||||
func (c *Catalog) addNameToIndex(p Package) {
|
||||
nameIndex := c.idsByName[p.Name]
|
||||
nameIndex.add(p.id)
|
||||
c.idsByName[p.Name] = nameIndex
|
||||
}
|
||||
|
||||
func (c *Catalog) addTypeToIndex(p Package) {
|
||||
typeIndex := c.idsByType[p.Type]
|
||||
typeIndex.add(p.id)
|
||||
c.idsByType[p.Type] = typeIndex
|
||||
}
|
||||
|
||||
func (c *Catalog) addPathsToIndex(p Package) {
|
||||
observedPaths := internal.NewStringSet()
|
||||
for _, l := range p.Locations {
|
||||
for _, l := range p.Locations.ToSlice() {
|
||||
if l.RealPath != "" && !observedPaths.Contains(l.RealPath) {
|
||||
c.idsByPath[l.RealPath] = append(c.idsByPath[l.RealPath], id)
|
||||
c.addPathToIndex(p.id, l.RealPath)
|
||||
observedPaths.Add(l.RealPath)
|
||||
}
|
||||
if l.VirtualPath != "" && l.RealPath != l.VirtualPath && !observedPaths.Contains(l.VirtualPath) {
|
||||
c.idsByPath[l.VirtualPath] = append(c.idsByPath[l.VirtualPath], id)
|
||||
c.addPathToIndex(p.id, l.VirtualPath)
|
||||
observedPaths.Add(l.VirtualPath)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Catalog) addPathToIndex(id artifact.ID, path string) {
|
||||
pathIndex := c.idsByPath[path]
|
||||
pathIndex.add(id)
|
||||
c.idsByPath[path] = pathIndex
|
||||
}
|
||||
|
||||
// Enumerate all packages for the given type(s), enumerating all packages if no type is specified.
|
||||
func (c *Catalog) Enumerate(types ...Type) <-chan Package {
|
||||
channel := make(chan Package)
|
||||
@@ -124,7 +179,7 @@ func (c *Catalog) Enumerate(types ...Type) <-chan Package {
|
||||
continue
|
||||
}
|
||||
}
|
||||
for _, id := range ids {
|
||||
for _, id := range ids.slice {
|
||||
p := c.Package(id)
|
||||
if p != nil {
|
||||
channel <- *p
|
||||
@@ -145,8 +200,10 @@ func (c *Catalog) Sorted(types ...Type) (pkgs []Package) {
|
||||
sort.SliceStable(pkgs, func(i, j int) bool {
|
||||
if pkgs[i].Name == pkgs[j].Name {
|
||||
if pkgs[i].Version == pkgs[j].Version {
|
||||
if pkgs[i].Type == pkgs[j].Type && len(pkgs[i].Locations) > 0 && len(pkgs[j].Locations) > 0 {
|
||||
return pkgs[i].Locations[0].String() < pkgs[j].Locations[0].String()
|
||||
iLocations := pkgs[i].Locations.ToSlice()
|
||||
jLocations := pkgs[j].Locations.ToSlice()
|
||||
if pkgs[i].Type == pkgs[j].Type && len(iLocations) > 0 && len(jLocations) > 0 {
|
||||
return iLocations[0].String() < jLocations[0].String()
|
||||
}
|
||||
return pkgs[i].Type < pkgs[j].Type
|
||||
}
|
||||
|
||||
+141
-19
@@ -3,11 +3,11 @@ package pkg
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
|
||||
"github.com/scylladb/go-set/strset"
|
||||
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"github.com/scylladb/go-set/strset"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
type expectedIndexes struct {
|
||||
@@ -19,17 +19,17 @@ func TestCatalogAddPopulatesIndex(t *testing.T) {
|
||||
|
||||
var pkgs = []Package{
|
||||
{
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/a/path", "/another/path"),
|
||||
source.NewVirtualLocation("/b/path", "/bee/path"),
|
||||
},
|
||||
),
|
||||
Type: RpmPkg,
|
||||
},
|
||||
{
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/c/path", "/another/path"),
|
||||
source.NewVirtualLocation("/d/path", "/another/path"),
|
||||
},
|
||||
),
|
||||
Type: NpmPkg,
|
||||
},
|
||||
}
|
||||
@@ -106,47 +106,169 @@ func assertIndexes(t *testing.T, c *Catalog, expectedIndexes expectedIndexes) {
|
||||
|
||||
func TestCatalog_PathIndexDeduplicatesRealVsVirtualPaths(t *testing.T) {
|
||||
p1 := Package{
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/b/path", "/another/path"),
|
||||
source.NewVirtualLocation("/b/path", "/b/path"),
|
||||
},
|
||||
),
|
||||
Type: RpmPkg,
|
||||
Name: "Package-1",
|
||||
}
|
||||
|
||||
p2 := Package{
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/b/path", "/b/path"),
|
||||
},
|
||||
),
|
||||
Type: RpmPkg,
|
||||
Name: "Package-2",
|
||||
}
|
||||
p2Dup := Package{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/b/path", "/another/path"),
|
||||
source.NewVirtualLocation("/b/path", "/c/path/b/dup"),
|
||||
),
|
||||
Type: RpmPkg,
|
||||
Name: "Package-2",
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
pkg Package
|
||||
name string
|
||||
pkgs []Package
|
||||
paths []string
|
||||
}{
|
||||
{
|
||||
name: "multiple locations with shared path",
|
||||
pkg: p1,
|
||||
pkgs: []Package{p1},
|
||||
paths: []string{
|
||||
"/b/path",
|
||||
"/another/path",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "one location with shared path",
|
||||
pkg: p2,
|
||||
pkgs: []Package{p2},
|
||||
paths: []string{
|
||||
"/b/path",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "two instances with similar locations",
|
||||
pkgs: []Package{p2, p2Dup},
|
||||
paths: []string{
|
||||
"/b/path",
|
||||
"/another/path",
|
||||
"/c/path/b/dup", // this updated the path index on merge
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
actual := NewCatalog(test.pkg).PackagesByPath("/b/path")
|
||||
if len(actual) != 1 {
|
||||
t.Errorf("expected exactly one package path, got %d", len(actual))
|
||||
for _, path := range test.paths {
|
||||
actualPackages := NewCatalog(test.pkgs...).PackagesByPath(path)
|
||||
require.Len(t, actualPackages, 1)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
func TestCatalog_MergeRecords(t *testing.T) {
|
||||
var tests = []struct {
|
||||
name string
|
||||
pkgs []Package
|
||||
expectedLocations []source.Location
|
||||
}{
|
||||
{
|
||||
name: "multiple Locations with shared path",
|
||||
pkgs: []Package{
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/b/path",
|
||||
FileSystemID: "a",
|
||||
},
|
||||
VirtualPath: "/another/path",
|
||||
},
|
||||
),
|
||||
Type: RpmPkg,
|
||||
},
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/b/path",
|
||||
FileSystemID: "b",
|
||||
},
|
||||
VirtualPath: "/another/path",
|
||||
},
|
||||
),
|
||||
Type: RpmPkg,
|
||||
},
|
||||
},
|
||||
expectedLocations: []source.Location{
|
||||
{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/b/path",
|
||||
FileSystemID: "a",
|
||||
},
|
||||
VirtualPath: "/another/path",
|
||||
},
|
||||
{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/b/path",
|
||||
FileSystemID: "b",
|
||||
},
|
||||
VirtualPath: "/another/path",
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
actual := NewCatalog(tt.pkgs...).PackagesByPath("/b/path")
|
||||
require.Len(t, actual, 1)
|
||||
assert.Equal(t, tt.expectedLocations, actual[0].Locations.ToSlice())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCatalog_EnumerateNilCatalog(t *testing.T) {
|
||||
var c *Catalog
|
||||
assert.Empty(t, c.Enumerate())
|
||||
}
|
||||
|
||||
func Test_idOrderedSet_add(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input []artifact.ID
|
||||
expected []artifact.ID
|
||||
}{
|
||||
{
|
||||
name: "elements deduplicated when added",
|
||||
input: []artifact.ID{
|
||||
"1", "2", "3", "4", "1", "2", "3", "4", "1", "2", "3", "4",
|
||||
},
|
||||
expected: []artifact.ID{
|
||||
"1", "2", "3", "4",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "elements retain ordering when added",
|
||||
input: []artifact.ID{
|
||||
"4", "3", "2", "1",
|
||||
},
|
||||
expected: []artifact.ID{
|
||||
"4", "3", "2", "1",
|
||||
},
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var s orderedIDSet
|
||||
s.add(tt.input...)
|
||||
assert.Equal(t, tt.expected, s.slice)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/apkdb"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/dart"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/deb"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/golang"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/java"
|
||||
@@ -61,6 +62,7 @@ func DirectoryCatalogers(cfg Config) []Cataloger {
|
||||
golang.NewGoModuleBinaryCataloger(),
|
||||
golang.NewGoModFileCataloger(),
|
||||
rust.NewCargoLockCataloger(),
|
||||
dart.NewPubspecLockCataloger(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -80,5 +82,6 @@ func AllCatalogers(cfg Config) []Cataloger {
|
||||
golang.NewGoModuleBinaryCataloger(),
|
||||
golang.NewGoModFileCataloger(),
|
||||
rust.NewCargoLockCataloger(),
|
||||
dart.NewPubspecLockCataloger(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -16,7 +16,7 @@ func Test_disallowJenkinsServerCPEForPluginPackage(t *testing.T) {
|
||||
}{
|
||||
{
|
||||
name: "go case (filter out)",
|
||||
cpe: mustCPE("cpe:2.3:a:name:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:name:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Type: pkg.JenkinsPluginPkg,
|
||||
},
|
||||
@@ -24,7 +24,7 @@ func Test_disallowJenkinsServerCPEForPluginPackage(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "ignore jenkins plugins with unique name",
|
||||
cpe: mustCPE("cpe:2.3:a:name:ci-jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:name:ci-jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Type: pkg.JenkinsPluginPkg,
|
||||
},
|
||||
@@ -32,7 +32,7 @@ func Test_disallowJenkinsServerCPEForPluginPackage(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "ignore java packages",
|
||||
cpe: mustCPE("cpe:2.3:a:name:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:name:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Type: pkg.JavaPkg,
|
||||
},
|
||||
@@ -55,7 +55,7 @@ func Test_disallowJenkinsCPEsNotAssociatedWithJenkins(t *testing.T) {
|
||||
}{
|
||||
{
|
||||
name: "filter out mismatched name (cloudbees vendor)",
|
||||
cpe: mustCPE("cpe:2.3:a:cloudbees:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:cloudbees:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Name: "not-j*nkins",
|
||||
Type: pkg.JavaPkg,
|
||||
@@ -64,7 +64,7 @@ func Test_disallowJenkinsCPEsNotAssociatedWithJenkins(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "filter out mismatched name (jenkins vendor)",
|
||||
cpe: mustCPE("cpe:2.3:a:jenkins:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:jenkins:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Name: "not-j*nkins",
|
||||
Type: pkg.JavaPkg,
|
||||
@@ -73,7 +73,7 @@ func Test_disallowJenkinsCPEsNotAssociatedWithJenkins(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "filter out mismatched name (any vendor)",
|
||||
cpe: mustCPE("cpe:2.3:a:*:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:*:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Name: "not-j*nkins",
|
||||
Type: pkg.JavaPkg,
|
||||
@@ -82,7 +82,7 @@ func Test_disallowJenkinsCPEsNotAssociatedWithJenkins(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "ignore packages with the name jenkins",
|
||||
cpe: mustCPE("cpe:2.3:a:*:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:*:jenkins:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Name: "jenkins-thing",
|
||||
Type: pkg.JavaPkg,
|
||||
@@ -91,7 +91,7 @@ func Test_disallowJenkinsCPEsNotAssociatedWithJenkins(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "ignore product names that are not exactly 'jenkins'",
|
||||
cpe: mustCPE("cpe:2.3:a:*:jenkins-something-else:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:*:jenkins-something-else:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Name: "not-j*nkins",
|
||||
Type: pkg.JavaPkg,
|
||||
@@ -115,7 +115,7 @@ func Test_disallowJiraClientServerMismatch(t *testing.T) {
|
||||
}{
|
||||
{
|
||||
name: "filter out mismatched name (atlassian vendor)",
|
||||
cpe: mustCPE("cpe:2.3:a:atlassian:jira:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:atlassian:jira:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Name: "something-client",
|
||||
Type: pkg.JavaPkg,
|
||||
@@ -124,7 +124,7 @@ func Test_disallowJiraClientServerMismatch(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "filter out mismatched name (jira vendor)",
|
||||
cpe: mustCPE("cpe:2.3:a:jira:jira:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:jira:jira:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Name: "something-client",
|
||||
Type: pkg.JavaPkg,
|
||||
@@ -133,7 +133,7 @@ func Test_disallowJiraClientServerMismatch(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "filter out mismatched name (any vendor)",
|
||||
cpe: mustCPE("cpe:2.3:a:*:jira:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:*:jira:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Name: "something-client",
|
||||
Type: pkg.JavaPkg,
|
||||
@@ -142,7 +142,7 @@ func Test_disallowJiraClientServerMismatch(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "ignore package names that do not have 'client'",
|
||||
cpe: mustCPE("cpe:2.3:a:*:jira:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:*:jira:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Name: "jira-thing",
|
||||
Type: pkg.JavaPkg,
|
||||
@@ -151,7 +151,7 @@ func Test_disallowJiraClientServerMismatch(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "ignore product names that are not exactly 'jira'",
|
||||
cpe: mustCPE("cpe:2.3:a:*:jira-something-else:3.2:*:*:*:*:*:*:*"),
|
||||
cpe: pkg.MustCPE("cpe:2.3:a:*:jira-something-else:3.2:*:*:*:*:*:*:*"),
|
||||
pkg: pkg.Package{
|
||||
Name: "not-j*ra",
|
||||
Type: pkg.JavaPkg,
|
||||
|
||||
@@ -55,7 +55,7 @@ func Generate(p pkg.Package) []pkg.CPE {
|
||||
// filter out any known combinations that don't accurately represent this package
|
||||
cpes = filter(cpes, p, cpeFilters...)
|
||||
|
||||
sort.Sort(BySpecificity(cpes))
|
||||
sort.Sort(pkg.CPEBySpecificity(cpes))
|
||||
|
||||
return cpes
|
||||
}
|
||||
|
||||
@@ -58,9 +58,8 @@ func (c *GenericCataloger) Catalog(resolver source.FileResolver) ([]pkg.Package,
|
||||
|
||||
for _, p := range discoveredPackages {
|
||||
p.FoundBy = c.upstreamCataloger
|
||||
p.Locations = append(p.Locations, location)
|
||||
p.Locations.Add(location)
|
||||
p.SetID()
|
||||
|
||||
packages = append(packages, *p)
|
||||
}
|
||||
|
||||
|
||||
@@ -53,7 +53,7 @@ func TestGenericCataloger(t *testing.T) {
|
||||
assert.Len(t, actualPkgs, len(expectedPkgs))
|
||||
|
||||
for _, p := range actualPkgs {
|
||||
ref := p.Locations[0]
|
||||
ref := p.Locations.ToSlice()[0]
|
||||
exP, ok := expectedPkgs[ref.RealPath]
|
||||
if !ok {
|
||||
t.Errorf("missing expected pkg: ref=%+v", ref)
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
package dart
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/common"
|
||||
)
|
||||
|
||||
// NewPubspecLockCataloger returns a new Dartlang cataloger object base on pubspec lock files.
|
||||
func NewPubspecLockCataloger() *common.GenericCataloger {
|
||||
globParsers := map[string]common.ParserFn{
|
||||
"**/pubspec.lock": parsePubspecLock,
|
||||
}
|
||||
|
||||
return common.NewGenericCataloger(nil, globParsers, "dartlang-lock-cataloger")
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
package dart
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/url"
|
||||
|
||||
"github.com/anchore/syft/internal/log"
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/common"
|
||||
"gopkg.in/yaml.v2"
|
||||
)
|
||||
|
||||
// integrity check
|
||||
var _ common.ParserFn = parsePubspecLock
|
||||
|
||||
const defaultPubRegistry string = "https://pub.dartlang.org"
|
||||
|
||||
type pubspecLock struct {
|
||||
Packages map[string]pubspecLockPackage `yaml:"packages"`
|
||||
Sdks map[string]string `yaml:"sdks"`
|
||||
}
|
||||
|
||||
type pubspecLockPackage struct {
|
||||
Dependency string `yaml:"dependency" mapstructure:"dependency"`
|
||||
Description pubspecLockDescription `yaml:"description" mapstructure:"description"`
|
||||
Source string `yaml:"source" mapstructure:"source"`
|
||||
Version string `yaml:"version" mapstructure:"version"`
|
||||
}
|
||||
|
||||
type pubspecLockDescription struct {
|
||||
Name string `yaml:"name" mapstructure:"name"`
|
||||
URL string `yaml:"url" mapstructure:"url"`
|
||||
Path string `yaml:"path" mapstructure:"path"`
|
||||
Ref string `yaml:"ref" mapstructure:"ref"`
|
||||
ResolvedRef string `yaml:"resolved-ref" mapstructure:"resolved-ref"`
|
||||
}
|
||||
|
||||
func parsePubspecLock(path string, reader io.Reader) ([]*pkg.Package, []artifact.Relationship, error) {
|
||||
var packages []*pkg.Package
|
||||
|
||||
dec := yaml.NewDecoder(reader)
|
||||
|
||||
var p pubspecLock
|
||||
if err := dec.Decode(&p); err != nil {
|
||||
return nil, nil, fmt.Errorf("failed to parse pubspec.lock file: %w", err)
|
||||
}
|
||||
|
||||
for name, pubPkg := range p.Packages {
|
||||
packages = append(packages, newPubspecLockPackage(name, pubPkg))
|
||||
}
|
||||
|
||||
return packages, nil, nil
|
||||
}
|
||||
|
||||
func newPubspecLockPackage(name string, p pubspecLockPackage) *pkg.Package {
|
||||
return &pkg.Package{
|
||||
Name: name,
|
||||
Version: p.Version,
|
||||
Language: pkg.Dart,
|
||||
Type: pkg.DartPubPkg,
|
||||
MetadataType: pkg.DartPubMetadataType,
|
||||
Metadata: &pkg.DartPubMetadata{
|
||||
Name: name,
|
||||
Version: p.Version,
|
||||
HostedURL: p.getHostedURL(),
|
||||
VcsURL: p.getVcsURL(),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (p *pubspecLockPackage) getVcsURL() string {
|
||||
if p.Source == "git" {
|
||||
if p.Description.Path == "." {
|
||||
return fmt.Sprintf("%s@%s", p.Description.URL, p.Description.ResolvedRef)
|
||||
}
|
||||
|
||||
return fmt.Sprintf("%s@%s#%s", p.Description.URL, p.Description.ResolvedRef, p.Description.Path)
|
||||
}
|
||||
|
||||
return ""
|
||||
}
|
||||
|
||||
func (p *pubspecLockPackage) getHostedURL() string {
|
||||
if p.Source == "hosted" && p.Description.URL != defaultPubRegistry {
|
||||
u, err := url.Parse(p.Description.URL)
|
||||
if err != nil {
|
||||
log.Debugf("Unable to parse registry url %w", err)
|
||||
return p.Description.URL
|
||||
}
|
||||
return u.Host
|
||||
}
|
||||
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
package dart
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
)
|
||||
|
||||
func assertPackagesEqual(t *testing.T, actual []*pkg.Package, expected map[string]*pkg.Package) {
|
||||
assert.Len(t, actual, len(expected))
|
||||
}
|
||||
|
||||
func TestParsePubspecLock(t *testing.T) {
|
||||
expected := map[string]*pkg.Package{
|
||||
"ale": {
|
||||
Name: "ale",
|
||||
Version: "3.3.0",
|
||||
Language: pkg.Dart,
|
||||
Type: pkg.DartPubPkg,
|
||||
MetadataType: pkg.DartPubMetadataType,
|
||||
Metadata: pkg.DartPubMetadata{
|
||||
Name: "ale",
|
||||
Version: "3.3.0",
|
||||
HostedURL: "pub.hosted.org",
|
||||
},
|
||||
},
|
||||
"analyzer": {
|
||||
Name: "analyzer",
|
||||
Version: "0.40.7",
|
||||
Language: pkg.Dart,
|
||||
Type: pkg.DartPubPkg,
|
||||
MetadataType: pkg.DartPubMetadataType,
|
||||
Metadata: pkg.DartPubMetadata{
|
||||
Name: "analyzer",
|
||||
Version: "0.40.7",
|
||||
},
|
||||
},
|
||||
"ansicolor": {
|
||||
Name: "ansicolor",
|
||||
Version: "1.1.1",
|
||||
Language: pkg.Dart,
|
||||
Type: pkg.DartPubPkg,
|
||||
MetadataType: pkg.DartPubMetadataType,
|
||||
Metadata: pkg.DartPubMetadata{
|
||||
Name: "ansicolor",
|
||||
Version: "1.1.1",
|
||||
},
|
||||
},
|
||||
"archive": {
|
||||
Name: "archive",
|
||||
Version: "2.0.13",
|
||||
Language: pkg.Dart,
|
||||
Type: pkg.DartPubPkg,
|
||||
MetadataType: pkg.DartPubMetadataType,
|
||||
Metadata: pkg.DartPubMetadata{
|
||||
Name: "archive",
|
||||
Version: "2.0.13",
|
||||
},
|
||||
},
|
||||
"args": {
|
||||
Name: "args",
|
||||
Version: "1.6.0",
|
||||
Language: pkg.Dart,
|
||||
Type: pkg.DartPubPkg,
|
||||
MetadataType: pkg.DartPubMetadataType,
|
||||
Metadata: pkg.DartPubMetadata{
|
||||
Name: "args",
|
||||
Version: "1.6.0",
|
||||
},
|
||||
},
|
||||
"key_binder": {
|
||||
Name: "key_binder",
|
||||
Version: "1.11.20",
|
||||
Language: pkg.Dart,
|
||||
Type: pkg.DartPubPkg,
|
||||
MetadataType: pkg.DartPubMetadataType,
|
||||
Metadata: pkg.DartPubMetadata{
|
||||
Name: "key_binder",
|
||||
Version: "1.11.20",
|
||||
VcsURL: "git@github.com:Workiva/key_binder.git#3f7b3a6350e73c7dcac45301c0e18fbd42af02f7",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
fixture, err := os.Open("test-fixtures/pubspec.lock")
|
||||
if err != nil {
|
||||
t.Fatalf("failed to open fixture: %+v", err)
|
||||
}
|
||||
|
||||
actual, _, err := parsePubspecLock(fixture.Name(), fixture)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to parse pubspec.lock: %+v", err)
|
||||
}
|
||||
|
||||
assertPackagesEqual(t, actual, expected)
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
# Generated by pub
|
||||
# See https://dart.dev/tools/pub/glossary#lockfile
|
||||
packages:
|
||||
ale:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: ale
|
||||
url: "https://pub.hosted.org"
|
||||
source: hosted
|
||||
version: "3.3.0"
|
||||
analyzer:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: analyzer
|
||||
url: "https://pub.dartlang.org"
|
||||
source: hosted
|
||||
version: "0.40.7"
|
||||
ansicolor:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: ansicolor
|
||||
url: "https://pub.dartlang.org"
|
||||
source: hosted
|
||||
version: "1.1.1"
|
||||
archive:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: archive
|
||||
url: "https://pub.dartlang.org"
|
||||
source: hosted
|
||||
version: "2.0.13"
|
||||
args:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: args
|
||||
url: "https://pub.dartlang.org"
|
||||
source: hosted
|
||||
version: "1.6.0"
|
||||
key_binder:
|
||||
dependency: "direct main"
|
||||
description:
|
||||
path: "."
|
||||
ref: HEAD
|
||||
resolved-ref: "3f7b3a6350e73c7dcac45301c0e18fbd42af02f7"
|
||||
url: "git@github.com:Workiva/key_binder.git"
|
||||
source: git
|
||||
version: "1.11.20"
|
||||
sdks:
|
||||
dart: ">=2.12.0 <3.0.0"
|
||||
@@ -60,7 +60,7 @@ func (c *Cataloger) Catalog(resolver source.FileResolver) ([]pkg.Package, []arti
|
||||
for i := range pkgs {
|
||||
p := &pkgs[i]
|
||||
p.FoundBy = c.Name()
|
||||
p.Locations = []source.Location{dbLocation}
|
||||
p.Locations.Add(dbLocation)
|
||||
|
||||
// the current entry only has what may have been listed in the status file, however, there are additional
|
||||
// files that are listed in multiple other locations. We should retrieve them all and merge the file lists
|
||||
@@ -88,7 +88,7 @@ func addLicenses(resolver source.FileResolver, dbLocation source.Location, p *pk
|
||||
p.Licenses = parseLicensesFromCopyright(copyrightReader)
|
||||
|
||||
// keep a record of the file where this was discovered
|
||||
p.Locations = append(p.Locations, *copyrightLocation)
|
||||
p.Locations.Add(*copyrightLocation)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -117,7 +117,7 @@ loopNewFiles:
|
||||
p.Metadata = metadata
|
||||
|
||||
// persist location information from each new source of information
|
||||
p.Locations = append(p.Locations, infoLocations...)
|
||||
p.Locations.Add(infoLocations...)
|
||||
}
|
||||
|
||||
func getAdditionalFileListing(resolver source.FileResolver, dbLocation source.Location, p *pkg.Package) ([]pkg.DpkgFileRecord, []source.Location) {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package deb
|
||||
|
||||
import (
|
||||
"github.com/stretchr/testify/assert"
|
||||
"testing"
|
||||
|
||||
"github.com/anchore/syft/syft/file"
|
||||
@@ -115,15 +116,13 @@ func TestDpkgCataloger(t *testing.T) {
|
||||
for idx := range actual {
|
||||
a := &actual[idx]
|
||||
// we will test the sources separately
|
||||
var sourcesList = make([]string, len(a.Locations))
|
||||
for i, s := range a.Locations {
|
||||
var sourcesList = make([]string, len(a.Locations.ToSlice()))
|
||||
for i, s := range a.Locations.ToSlice() {
|
||||
sourcesList[i] = s.RealPath
|
||||
}
|
||||
a.Locations = nil
|
||||
a.Locations = source.NewLocationSet()
|
||||
|
||||
for _, d := range deep.Equal(sourcesList, test.sources[a.Name]) {
|
||||
t.Errorf("diff: %+v", d)
|
||||
}
|
||||
assert.ElementsMatch(t, sourcesList, test.sources[a.Name])
|
||||
}
|
||||
|
||||
// test remaining fields...
|
||||
|
||||
@@ -44,14 +44,12 @@ func newGoBinaryPackage(dep *debug.Module, goVersion, architecture string, locat
|
||||
}
|
||||
|
||||
p := pkg.Package{
|
||||
FoundBy: catalogerName,
|
||||
Name: dep.Path,
|
||||
Version: dep.Version,
|
||||
Language: pkg.Go,
|
||||
Type: pkg.GoModulePkg,
|
||||
Locations: []source.Location{
|
||||
location,
|
||||
},
|
||||
FoundBy: catalogerName,
|
||||
Name: dep.Path,
|
||||
Version: dep.Version,
|
||||
Language: pkg.Go,
|
||||
Type: pkg.GoModulePkg,
|
||||
Locations: source.NewLocationSet(location),
|
||||
MetadataType: pkg.GolangBinMetadataType,
|
||||
Metadata: pkg.GolangBinMetadata{
|
||||
GoCompiledVersion: goVersion,
|
||||
|
||||
@@ -133,14 +133,14 @@ func TestBuildGoPkgInfo(t *testing.T) {
|
||||
FoundBy: catalogerName,
|
||||
Language: pkg.Go,
|
||||
Type: pkg.GoModulePkg,
|
||||
Locations: []source.Location{
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/a-path",
|
||||
FileSystemID: "layer-id",
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
MetadataType: pkg.GolangBinMetadataType,
|
||||
Metadata: pkg.GolangBinMetadata{
|
||||
GoCompiledVersion: goCompiledVersion,
|
||||
@@ -190,14 +190,14 @@ func TestBuildGoPkgInfo(t *testing.T) {
|
||||
Version: "v0.2.1",
|
||||
Language: pkg.Go,
|
||||
Type: pkg.GoModulePkg,
|
||||
Locations: []source.Location{
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/a-path",
|
||||
FileSystemID: "layer-id",
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
MetadataType: pkg.GolangBinMetadataType,
|
||||
Metadata: pkg.GolangBinMetadata{
|
||||
GoCompiledVersion: goCompiledVersion,
|
||||
@@ -252,14 +252,14 @@ func TestBuildGoPkgInfo(t *testing.T) {
|
||||
Version: "v0.2.1",
|
||||
Language: pkg.Go,
|
||||
Type: pkg.GoModulePkg,
|
||||
Locations: []source.Location{
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/a-path",
|
||||
FileSystemID: "layer-id",
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
MetadataType: pkg.GolangBinMetadataType,
|
||||
Metadata: pkg.GolangBinMetadata{
|
||||
GoCompiledVersion: goCompiledVersion,
|
||||
@@ -273,14 +273,14 @@ func TestBuildGoPkgInfo(t *testing.T) {
|
||||
Version: "v0.0.0-20210222170800-9c70f9b80bcf",
|
||||
Language: pkg.Go,
|
||||
Type: pkg.GoModulePkg,
|
||||
Locations: []source.Location{
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/a-path",
|
||||
FileSystemID: "layer-id",
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
MetadataType: pkg.GolangBinMetadataType,
|
||||
Metadata: pkg.GolangBinMetadata{
|
||||
GoCompiledVersion: goCompiledVersion,
|
||||
@@ -327,12 +327,14 @@ func TestBuildGoPkgInfo(t *testing.T) {
|
||||
Version: "v0.0.0-20211006194710-c8a6f5223071",
|
||||
Language: pkg.Go,
|
||||
Type: pkg.GoModulePkg,
|
||||
Locations: []source.Location{
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/a-path",
|
||||
FileSystemID: "layer-id",
|
||||
}}},
|
||||
},
|
||||
},
|
||||
),
|
||||
MetadataType: pkg.GolangBinMetadataType,
|
||||
Metadata: pkg.GolangBinMetadata{
|
||||
GoCompiledVersion: goCompiledVersion,
|
||||
@@ -344,14 +346,14 @@ func TestBuildGoPkgInfo(t *testing.T) {
|
||||
Version: "v0.0.0-20210916214954-140adaaadfaf",
|
||||
Language: pkg.Go,
|
||||
Type: pkg.GoModulePkg,
|
||||
Locations: []source.Location{
|
||||
{
|
||||
Locations: source.NewLocationSet(
|
||||
source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "/a-path",
|
||||
FileSystemID: "layer-id",
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
MetadataType: pkg.GolangBinMetadataType,
|
||||
Metadata: pkg.GolangBinMetadata{
|
||||
GoCompiledVersion: goCompiledVersion,
|
||||
|
||||
@@ -80,7 +80,7 @@ func (c *PackageCataloger) catalogEggOrWheel(resolver source.FileResolver, metad
|
||||
Name: metadata.Name,
|
||||
Version: metadata.Version,
|
||||
FoundBy: c.Name(),
|
||||
Locations: sources,
|
||||
Locations: source.NewLocationSet(sources...),
|
||||
Licenses: licenses,
|
||||
Language: pkg.Python,
|
||||
Type: pkg.PythonPkg,
|
||||
|
||||
@@ -144,7 +144,7 @@ func TestPythonPackageWheelCataloger(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
test.expectedPackage.Locations = locations
|
||||
test.expectedPackage.Locations = source.NewLocationSet(locations...)
|
||||
|
||||
actual, _, err := NewPythonPackageCataloger().Catalog(resolver)
|
||||
if err != nil {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package python
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
@@ -48,7 +49,14 @@ func TestParsePipFileLock(t *testing.T) {
|
||||
t.Fatalf("failed to parse requirements: %+v", err)
|
||||
}
|
||||
|
||||
if diff := cmp.Diff(expected, actual, cmp.AllowUnexported(pkg.Package{})); diff != "" {
|
||||
if diff := cmp.Diff(expected, actual,
|
||||
cmp.AllowUnexported(pkg.Package{}),
|
||||
cmp.Comparer(
|
||||
func(x, y source.LocationSet) bool {
|
||||
return cmp.Equal(x.ToSlice(), y.ToSlice())
|
||||
},
|
||||
),
|
||||
); diff != "" {
|
||||
t.Errorf("unexpected result from parsing (-expected +actual)\n%s", diff)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package python
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
@@ -42,7 +43,14 @@ func TestParseRequirementsTxt(t *testing.T) {
|
||||
t.Fatalf("failed to parse requirements: %+v", err)
|
||||
}
|
||||
|
||||
if diff := cmp.Diff(expected, actual, cmp.AllowUnexported(pkg.Package{})); diff != "" {
|
||||
if diff := cmp.Diff(expected, actual,
|
||||
cmp.AllowUnexported(pkg.Package{}),
|
||||
cmp.Comparer(
|
||||
func(x, y source.LocationSet) bool {
|
||||
return cmp.Equal(x.ToSlice(), y.ToSlice())
|
||||
},
|
||||
),
|
||||
); diff != "" {
|
||||
t.Errorf("unexpected result from parsing (-expected +actual)\n%s", diff)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package python
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
@@ -54,7 +55,14 @@ func TestParseSetup(t *testing.T) {
|
||||
t.Fatalf("failed to parse requirements: %+v", err)
|
||||
}
|
||||
|
||||
if diff := cmp.Diff(expected, actual, cmp.AllowUnexported(pkg.Package{})); diff != "" {
|
||||
if diff := cmp.Diff(expected, actual,
|
||||
cmp.AllowUnexported(pkg.Package{}),
|
||||
cmp.Comparer(
|
||||
func(x, y source.LocationSet) bool {
|
||||
return cmp.Equal(x.ToSlice(), y.ToSlice())
|
||||
},
|
||||
),
|
||||
); diff != "" {
|
||||
t.Errorf("unexpected result from parsing (-expected +actual)\n%s", diff)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -63,7 +63,7 @@ func parseRpmDB(resolver source.FilePathResolver, dbLocation source.Location, re
|
||||
p := pkg.Package{
|
||||
Name: entry.Name,
|
||||
Version: toELVersion(metadata),
|
||||
Locations: []source.Location{dbLocation},
|
||||
Locations: source.NewLocationSet(dbLocation),
|
||||
FoundBy: catalogerName,
|
||||
Type: pkg.RpmPkg,
|
||||
MetadataType: pkg.RpmdbMetadataType,
|
||||
|
||||
@@ -71,7 +71,7 @@ func TestParseRpmDB(t *testing.T) {
|
||||
"dive": {
|
||||
Name: "dive",
|
||||
Version: "0.9.2-1",
|
||||
Locations: []source.Location{dbLocation},
|
||||
Locations: source.NewLocationSet(dbLocation),
|
||||
FoundBy: catalogerName,
|
||||
Type: pkg.RpmPkg,
|
||||
MetadataType: pkg.RpmdbMetadataType,
|
||||
@@ -98,7 +98,7 @@ func TestParseRpmDB(t *testing.T) {
|
||||
"dive": {
|
||||
Name: "dive",
|
||||
Version: "0.9.2-1",
|
||||
Locations: []source.Location{dbLocation},
|
||||
Locations: source.NewLocationSet(dbLocation),
|
||||
FoundBy: catalogerName,
|
||||
Type: pkg.RpmPkg,
|
||||
MetadataType: pkg.RpmdbMetadataType,
|
||||
|
||||
+6
-6
@@ -1,4 +1,4 @@
|
||||
package cpe
|
||||
package pkg
|
||||
|
||||
import (
|
||||
"sort"
|
||||
@@ -6,15 +6,15 @@ import (
|
||||
"github.com/facebookincubator/nvdtools/wfn"
|
||||
)
|
||||
|
||||
var _ sort.Interface = (*BySpecificity)(nil)
|
||||
var _ sort.Interface = (*CPEBySpecificity)(nil)
|
||||
|
||||
type BySpecificity []wfn.Attributes
|
||||
type CPEBySpecificity []wfn.Attributes
|
||||
|
||||
func (c BySpecificity) Len() int { return len(c) }
|
||||
func (c CPEBySpecificity) Len() int { return len(c) }
|
||||
|
||||
func (c BySpecificity) Swap(i, j int) { c[i], c[j] = c[j], c[i] }
|
||||
func (c CPEBySpecificity) Swap(i, j int) { c[i], c[j] = c[j], c[i] }
|
||||
|
||||
func (c BySpecificity) Less(i, j int) bool {
|
||||
func (c CPEBySpecificity) Less(i, j int) bool {
|
||||
iScore := weightedCountForSpecifiedFields(c[i])
|
||||
jScore := weightedCountForSpecifiedFields(c[j])
|
||||
|
||||
+14
-22
@@ -1,40 +1,32 @@
|
||||
package cpe
|
||||
package pkg
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"testing"
|
||||
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func mustCPE(c string) pkg.CPE {
|
||||
return must(pkg.NewCPE(c))
|
||||
}
|
||||
|
||||
func must(c pkg.CPE, e error) pkg.CPE {
|
||||
if e != nil {
|
||||
panic(e)
|
||||
}
|
||||
return c
|
||||
func mustCPE(c string) CPE {
|
||||
return must(NewCPE(c))
|
||||
}
|
||||
|
||||
func TestCPESpecificity(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input []pkg.CPE
|
||||
expected []pkg.CPE
|
||||
input []CPE
|
||||
expected []CPE
|
||||
}{
|
||||
{
|
||||
name: "sort strictly by wfn *",
|
||||
input: []pkg.CPE{
|
||||
input: []CPE{
|
||||
mustCPE("cpe:2.3:a:*:package:1:*:*:*:*:*:*:*"),
|
||||
mustCPE("cpe:2.3:a:some:package:1:*:*:*:*:*:*:*"),
|
||||
mustCPE("cpe:2.3:a:*:package:1:*:*:*:*:some:*:*"),
|
||||
mustCPE("cpe:2.3:a:some:package:1:*:*:*:*:some:*:*"),
|
||||
mustCPE("cpe:2.3:a:some:package:*:*:*:*:*:*:*:*"),
|
||||
},
|
||||
expected: []pkg.CPE{
|
||||
expected: []CPE{
|
||||
mustCPE("cpe:2.3:a:some:package:1:*:*:*:*:some:*:*"),
|
||||
mustCPE("cpe:2.3:a:some:package:1:*:*:*:*:*:*:*"),
|
||||
mustCPE("cpe:2.3:a:some:package:*:*:*:*:*:*:*:*"),
|
||||
@@ -44,7 +36,7 @@ func TestCPESpecificity(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "sort strictly by field length",
|
||||
input: []pkg.CPE{
|
||||
input: []CPE{
|
||||
mustCPE("cpe:2.3:a:1:22:1:*:*:*:*:1:*:*"),
|
||||
mustCPE("cpe:2.3:a:55555:1:1:*:*:*:*:1:*:*"),
|
||||
mustCPE("cpe:2.3:a:1:1:333:*:*:*:*:1:*:*"),
|
||||
@@ -52,7 +44,7 @@ func TestCPESpecificity(t *testing.T) {
|
||||
mustCPE("cpe:2.3:a:1:1:1:*:*:*:*:1:*:*"),
|
||||
mustCPE("cpe:2.3:a:1:1:1:*:*:*:*:4444:*:*"),
|
||||
},
|
||||
expected: []pkg.CPE{
|
||||
expected: []CPE{
|
||||
mustCPE("cpe:2.3:a:1:666666:1:*:*:*:*:1:*:*"),
|
||||
mustCPE("cpe:2.3:a:55555:1:1:*:*:*:*:1:*:*"),
|
||||
mustCPE("cpe:2.3:a:1:1:1:*:*:*:*:4444:*:*"),
|
||||
@@ -63,7 +55,7 @@ func TestCPESpecificity(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "sort by mix of field length and specificity",
|
||||
input: []pkg.CPE{
|
||||
input: []CPE{
|
||||
mustCPE("cpe:2.3:a:1:666666:*:*:*:*:*:1:*:*"),
|
||||
mustCPE("cpe:2.3:a:*:1:1:*:*:*:*:4444:*:*"),
|
||||
mustCPE("cpe:2.3:a:1:*:333:*:*:*:*:*:*:*"),
|
||||
@@ -71,7 +63,7 @@ func TestCPESpecificity(t *testing.T) {
|
||||
mustCPE("cpe:2.3:a:1:22:1:*:*:*:*:1:*:*"),
|
||||
mustCPE("cpe:2.3:a:55555:1:1:*:*:*:*:1:*:*"),
|
||||
},
|
||||
expected: []pkg.CPE{
|
||||
expected: []CPE{
|
||||
mustCPE("cpe:2.3:a:55555:1:1:*:*:*:*:1:*:*"),
|
||||
mustCPE("cpe:2.3:a:1:22:1:*:*:*:*:1:*:*"),
|
||||
mustCPE("cpe:2.3:a:1:1:1:*:*:*:*:1:*:*"),
|
||||
@@ -82,7 +74,7 @@ func TestCPESpecificity(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "sort by mix of field length, specificity, dash",
|
||||
input: []pkg.CPE{
|
||||
input: []CPE{
|
||||
mustCPE("cpe:2.3:a:alpine:alpine_keys:2.3-r1:*:*:*:*:*:*:*"),
|
||||
mustCPE("cpe:2.3:a:alpine_keys:alpine_keys:2.3-r1:*:*:*:*:*:*:*"),
|
||||
mustCPE("cpe:2.3:a:alpine-keys:alpine_keys:2.3-r1:*:*:*:*:*:*:*"),
|
||||
@@ -90,7 +82,7 @@ func TestCPESpecificity(t *testing.T) {
|
||||
mustCPE("cpe:2.3:a:alpine-keys:alpine-keys:2.3-r1:*:*:*:*:*:*:*"),
|
||||
mustCPE("cpe:2.3:a:alpine_keys:alpine-keys:2.3-r1:*:*:*:*:*:*:*"),
|
||||
},
|
||||
expected: []pkg.CPE{
|
||||
expected: []CPE{
|
||||
mustCPE("cpe:2.3:a:alpine-keys:alpine-keys:2.3-r1:*:*:*:*:*:*:*"),
|
||||
mustCPE("cpe:2.3:a:alpine-keys:alpine_keys:2.3-r1:*:*:*:*:*:*:*"),
|
||||
mustCPE("cpe:2.3:a:alpine_keys:alpine-keys:2.3-r1:*:*:*:*:*:*:*"),
|
||||
@@ -103,7 +95,7 @@ func TestCPESpecificity(t *testing.T) {
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
sort.Sort(BySpecificity(test.input))
|
||||
sort.Sort(CPEBySpecificity(test.input))
|
||||
assert.Equal(t, test.expected, test.input)
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package pkg
|
||||
|
||||
import (
|
||||
"github.com/anchore/packageurl-go"
|
||||
"github.com/anchore/syft/syft/linux"
|
||||
)
|
||||
|
||||
type DartPubMetadata struct {
|
||||
Name string `mapstructure:"name" json:"name"`
|
||||
Version string `mapstructure:"version" json:"version"`
|
||||
HostedURL string `mapstructure:"hosted_url" json:"hosted_url,omitempty"`
|
||||
VcsURL string `mapstructure:"vcs_url" json:"vcs_url,omitempty"`
|
||||
}
|
||||
|
||||
func (m DartPubMetadata) PackageURL(_ *linux.Release) string {
|
||||
var qualifiers packageurl.Qualifiers
|
||||
|
||||
if m.HostedURL != "" {
|
||||
qualifiers = append(qualifiers, packageurl.Qualifier{
|
||||
Key: "hosted_url",
|
||||
Value: m.HostedURL,
|
||||
})
|
||||
} else if m.VcsURL != "" { // Default to using Hosted if somehow both are provided
|
||||
qualifiers = append(qualifiers, packageurl.Qualifier{
|
||||
Key: "vcs_url",
|
||||
Value: m.VcsURL,
|
||||
})
|
||||
}
|
||||
|
||||
return packageurl.NewPackageURL(
|
||||
packageurl.TypePub,
|
||||
"",
|
||||
m.Name,
|
||||
m.Version,
|
||||
qualifiers,
|
||||
"",
|
||||
).ToString()
|
||||
}
|
||||
@@ -20,7 +20,7 @@ var jenkinsPluginPomPropertiesGroupIDs = []string{
|
||||
|
||||
// JavaMetadata encapsulates all Java ecosystem metadata for a package as well as an (optional) parent relationship.
|
||||
type JavaMetadata struct {
|
||||
VirtualPath string `json:"virtualPath"`
|
||||
VirtualPath string `json:"virtualPath" cyclonedx:"virtualPath"` // we need to include the virtual path in cyclonedx documents to prevent deduplication of jars within jars
|
||||
Manifest *JavaManifest `mapstructure:"Manifest" json:"manifest,omitempty"`
|
||||
PomProperties *PomProperties `mapstructure:"PomProperties" json:"pomProperties,omitempty" cyclonedx:"-"`
|
||||
PomProject *PomProject `mapstructure:"PomProject" json:"pomProject,omitempty"`
|
||||
|
||||
@@ -19,6 +19,7 @@ const (
|
||||
Ruby Language = "ruby"
|
||||
Go Language = "go"
|
||||
Rust Language = "rust"
|
||||
Dart Language = "dart"
|
||||
)
|
||||
|
||||
// AllLanguages is a set of all programming languages detected by syft.
|
||||
@@ -30,6 +31,7 @@ var AllLanguages = []Language{
|
||||
Ruby,
|
||||
Go,
|
||||
Rust,
|
||||
Dart,
|
||||
}
|
||||
|
||||
// String returns the string representation of the language.
|
||||
@@ -62,6 +64,8 @@ func LanguageByName(name string) Language {
|
||||
return Ruby
|
||||
case purlCargoPkgType:
|
||||
return Rust
|
||||
case packageurl.TypePub, string(Dart):
|
||||
return Dart
|
||||
default:
|
||||
return UnknownLanguage
|
||||
}
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
package pkg
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/scylladb/go-set/strset"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestLanguageFromPURL(t *testing.T) {
|
||||
@@ -29,6 +30,10 @@ func TestLanguageFromPURL(t *testing.T) {
|
||||
purl: "pkg:golang/github.com/gorilla/context@234fd47e07d1004f0aed9c",
|
||||
want: Go,
|
||||
},
|
||||
{
|
||||
purl: "pkg:pub/util@1.2.34",
|
||||
want: Dart,
|
||||
},
|
||||
{
|
||||
purl: "pkg:cargo/clap@2.33.0",
|
||||
want: Rust,
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
package pkg
|
||||
|
||||
import (
|
||||
"sort"
|
||||
)
|
||||
|
||||
func mergeCPEs(a, b []CPE) (result []CPE) {
|
||||
aCPEs := make(map[string]CPE)
|
||||
|
||||
// keep all CPEs from a and create a quick string-based lookup
|
||||
for _, aCPE := range a {
|
||||
aCPEs[aCPE.BindToFmtString()] = aCPE
|
||||
result = append(result, aCPE)
|
||||
}
|
||||
|
||||
// keep all unique CPEs from b
|
||||
for _, bCPE := range b {
|
||||
if _, exists := aCPEs[bCPE.BindToFmtString()]; !exists {
|
||||
result = append(result, bCPE)
|
||||
}
|
||||
}
|
||||
|
||||
sort.Sort(CPEBySpecificity(result))
|
||||
return result
|
||||
}
|
||||
@@ -17,6 +17,7 @@ const (
|
||||
JavaMetadataType MetadataType = "JavaMetadata"
|
||||
NpmPackageJSONMetadataType MetadataType = "NpmPackageJsonMetadata"
|
||||
RpmdbMetadataType MetadataType = "RpmdbMetadata"
|
||||
DartPubMetadataType MetadataType = "DartPubMetadata"
|
||||
PythonPackageMetadataType MetadataType = "PythonPackageMetadata"
|
||||
RustCargoPackageMetadataType MetadataType = "RustCargoPackageMetadata"
|
||||
KbPackageMetadataType MetadataType = "KbPackageMetadata"
|
||||
@@ -31,6 +32,7 @@ var AllMetadataTypes = []MetadataType{
|
||||
JavaMetadataType,
|
||||
NpmPackageJSONMetadataType,
|
||||
RpmdbMetadataType,
|
||||
DartPubMetadataType,
|
||||
PythonPackageMetadataType,
|
||||
RustCargoPackageMetadataType,
|
||||
KbPackageMetadataType,
|
||||
@@ -45,6 +47,7 @@ var MetadataTypeByName = map[MetadataType]reflect.Type{
|
||||
JavaMetadataType: reflect.TypeOf(JavaMetadata{}),
|
||||
NpmPackageJSONMetadataType: reflect.TypeOf(NpmPackageJSONMetadata{}),
|
||||
RpmdbMetadataType: reflect.TypeOf(RpmdbMetadata{}),
|
||||
DartPubMetadataType: reflect.TypeOf(DartPubMetadata{}),
|
||||
PythonPackageMetadataType: reflect.TypeOf(PythonPackageMetadata{}),
|
||||
RustCargoPackageMetadataType: reflect.TypeOf(CargoMetadata{}),
|
||||
KbPackageMetadataType: reflect.TypeOf(KbPackageMetadata{}),
|
||||
|
||||
+30
-12
@@ -14,18 +14,18 @@ import (
|
||||
// Package represents an application or library that has been bundled into a distributable format.
|
||||
// TODO: if we ignore FoundBy for ID generation should we merge the field to show it was found in two places?
|
||||
type Package struct {
|
||||
id artifact.ID `hash:"ignore"`
|
||||
Name string // the package name
|
||||
Version string // the version of the package
|
||||
FoundBy string `cyclonedx:"foundBy"` // the specific cataloger that discovered this package
|
||||
Locations []source.Location // the locations that lead to the discovery of this package (note: this is not necessarily the locations that make up this package)
|
||||
Licenses []string // licenses discovered with the package metadata
|
||||
Language Language `cyclonedx:"language"` // the language ecosystem this package belongs to (e.g. JavaScript, Python, etc)
|
||||
Type Type `cyclonedx:"type"` // the package type (e.g. Npm, Yarn, Python, Rpm, Deb, etc)
|
||||
CPEs []CPE `hash:"ignore"` // all possible Common Platform Enumerators (note: this is NOT included in the definition of the ID since all fields on a CPE are derived from other fields)
|
||||
PURL string `hash:"ignore"` // the Package URL (see https://github.com/package-url/purl-spec) (note: this is NOT included in the definition of the ID since all fields on a pURL are derived from other fields)
|
||||
MetadataType MetadataType `cyclonedx:"metadataType"` // the shape of the additional data in the "metadata" field
|
||||
Metadata interface{} // additional data found while parsing the package source
|
||||
id artifact.ID `hash:"ignore"`
|
||||
Name string // the package name
|
||||
Version string // the version of the package
|
||||
FoundBy string `cyclonedx:"foundBy"` // the specific cataloger that discovered this package
|
||||
Locations source.LocationSet // the locations that lead to the discovery of this package (note: this is not necessarily the locations that make up this package)
|
||||
Licenses []string // licenses discovered with the package metadata
|
||||
Language Language `cyclonedx:"language"` // the language ecosystem this package belongs to (e.g. JavaScript, Python, etc)
|
||||
Type Type `cyclonedx:"type"` // the package type (e.g. Npm, Yarn, Python, Rpm, Deb, etc)
|
||||
CPEs []CPE `hash:"ignore"` // all possible Common Platform Enumerators (note: this is NOT included in the definition of the ID since all fields on a CPE are derived from other fields)
|
||||
PURL string `hash:"ignore"` // the Package URL (see https://github.com/package-url/purl-spec)
|
||||
MetadataType MetadataType `cyclonedx:"metadataType"` // the shape of the additional data in the "metadata" field
|
||||
Metadata interface{} // additional data found while parsing the package source
|
||||
}
|
||||
|
||||
func (p *Package) SetID() {
|
||||
@@ -46,3 +46,21 @@ func (p Package) ID() artifact.ID {
|
||||
func (p Package) String() string {
|
||||
return fmt.Sprintf("Pkg(name=%q version=%q type=%q id=%q)", p.Name, p.Version, p.Type, p.id)
|
||||
}
|
||||
|
||||
func (p *Package) merge(other Package) error {
|
||||
if p.id != other.id {
|
||||
return fmt.Errorf("cannot merge packages with different IDs: %q vs %q", p.id, other.id)
|
||||
}
|
||||
if p.PURL != other.PURL {
|
||||
log.Warnf("merging packages have with different pURLs: %q=%q vs %q=%q", p.id, p.PURL, other.id, other.PURL)
|
||||
}
|
||||
|
||||
p.Locations.Add(other.Locations.ToSlice()...)
|
||||
|
||||
p.CPEs = mergeCPEs(p.CPEs, other.CPEs)
|
||||
|
||||
if p.PURL == "" {
|
||||
p.PURL = other.PURL
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
+266
-32
@@ -3,24 +3,28 @@ package pkg
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/anchore/stereoscope/pkg/file"
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"github.com/google/go-cmp/cmp"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestFingerprint(t *testing.T) {
|
||||
func TestIDUniqueness(t *testing.T) {
|
||||
originalLocation := source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "39.0742° N, 21.8243° E",
|
||||
FileSystemID: "Earth",
|
||||
},
|
||||
VirtualPath: "/Ancient-Greece",
|
||||
}
|
||||
originalPkg := Package{
|
||||
Name: "pi",
|
||||
Version: "3.14",
|
||||
FoundBy: "Archimedes",
|
||||
Locations: []source.Location{
|
||||
{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "39.0742° N, 21.8243° E",
|
||||
FileSystemID: "Earth",
|
||||
},
|
||||
VirtualPath: "/Ancient-Greece",
|
||||
},
|
||||
},
|
||||
Locations: source.NewLocationSet(
|
||||
originalLocation,
|
||||
),
|
||||
Licenses: []string{
|
||||
"cc0-1.0",
|
||||
"MIT",
|
||||
@@ -45,9 +49,9 @@ func TestFingerprint(t *testing.T) {
|
||||
|
||||
// this is a set of differential tests, ensuring that select mutations are reflected in the fingerprint (or not)
|
||||
tests := []struct {
|
||||
name string
|
||||
transform func(pkg Package) Package
|
||||
expectIdentical bool
|
||||
name string
|
||||
transform func(pkg Package) Package
|
||||
expectedIDComparison assert.ComparisonAssertionFunc
|
||||
}{
|
||||
{
|
||||
name: "go case (no transform)",
|
||||
@@ -55,7 +59,7 @@ func TestFingerprint(t *testing.T) {
|
||||
// do nothing!
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: true,
|
||||
expectedIDComparison: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "same metadata is ignored",
|
||||
@@ -72,7 +76,7 @@ func TestFingerprint(t *testing.T) {
|
||||
}
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: true,
|
||||
expectedIDComparison: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "licenses order is ignored",
|
||||
@@ -84,7 +88,7 @@ func TestFingerprint(t *testing.T) {
|
||||
}
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: true,
|
||||
expectedIDComparison: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "name is reflected",
|
||||
@@ -92,7 +96,42 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.Name = "new!"
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: false,
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "location is reflected",
|
||||
transform: func(pkg Package) Package {
|
||||
locations := source.NewLocationSet(pkg.Locations.ToSlice()...)
|
||||
locations.Add(source.NewLocation("/somewhere/new"))
|
||||
pkg.Locations = locations
|
||||
return pkg
|
||||
},
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "same path for different filesystem is NOT reflected",
|
||||
transform: func(pkg Package) Package {
|
||||
newLocation := originalLocation
|
||||
newLocation.FileSystemID = "Mars"
|
||||
|
||||
pkg.Locations = source.NewLocationSet(newLocation)
|
||||
return pkg
|
||||
},
|
||||
expectedIDComparison: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "multiple equivalent paths for different filesystem is NOT reflected",
|
||||
transform: func(pkg Package) Package {
|
||||
newLocation := originalLocation
|
||||
newLocation.FileSystemID = "Mars"
|
||||
|
||||
locations := source.NewLocationSet(pkg.Locations.ToSlice()...)
|
||||
locations.Add(newLocation, originalLocation)
|
||||
|
||||
pkg.Locations = locations
|
||||
return pkg
|
||||
},
|
||||
expectedIDComparison: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "version is reflected",
|
||||
@@ -100,7 +139,7 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.Version = "new!"
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: false,
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "licenses is reflected",
|
||||
@@ -108,7 +147,7 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.Licenses = []string{"new!"}
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: false,
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "type is reflected",
|
||||
@@ -116,7 +155,7 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.Type = RustPkg
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: false,
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "metadata type is reflected",
|
||||
@@ -124,7 +163,7 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.MetadataType = RustCargoPackageMetadataType
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: false,
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "CPEs is ignored",
|
||||
@@ -132,7 +171,7 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.CPEs = []CPE{}
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: true,
|
||||
expectedIDComparison: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "pURL is ignored",
|
||||
@@ -140,7 +179,7 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.PURL = "new!"
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: true,
|
||||
expectedIDComparison: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "language is reflected",
|
||||
@@ -148,7 +187,7 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.Language = Rust
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: false,
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "foundBy is reflected",
|
||||
@@ -156,7 +195,7 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.FoundBy = "new!"
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: false,
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "metadata mutation is reflected",
|
||||
@@ -166,7 +205,7 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.Metadata = metadata
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: false,
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "new metadata is reflected",
|
||||
@@ -176,7 +215,7 @@ func TestFingerprint(t *testing.T) {
|
||||
}
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: false,
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "nil metadata is reflected",
|
||||
@@ -184,7 +223,7 @@ func TestFingerprint(t *testing.T) {
|
||||
pkg.Metadata = nil
|
||||
return pkg
|
||||
},
|
||||
expectIdentical: false,
|
||||
expectedIDComparison: assert.NotEqual,
|
||||
},
|
||||
}
|
||||
|
||||
@@ -199,12 +238,207 @@ func TestFingerprint(t *testing.T) {
|
||||
transformedFingerprint := transformedPkg.ID()
|
||||
assert.NotEmpty(t, transformedFingerprint)
|
||||
|
||||
if test.expectIdentical {
|
||||
assert.Equal(t, originalFingerprint, transformedFingerprint)
|
||||
} else {
|
||||
assert.NotEqual(t, originalFingerprint, transformedFingerprint)
|
||||
test.expectedIDComparison(t, originalFingerprint, transformedFingerprint)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestPackage_Merge(t *testing.T) {
|
||||
originalLocation := source.Location{
|
||||
Coordinates: source.Coordinates{
|
||||
RealPath: "39.0742° N, 21.8243° E",
|
||||
FileSystemID: "Earth",
|
||||
},
|
||||
VirtualPath: "/Ancient-Greece",
|
||||
}
|
||||
|
||||
similarLocation := originalLocation
|
||||
similarLocation.FileSystemID = "Mars"
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
subject Package
|
||||
other Package
|
||||
expected *Package
|
||||
}{
|
||||
{
|
||||
name: "merge two packages (different cpes + locations)",
|
||||
subject: Package{
|
||||
Name: "pi",
|
||||
Version: "3.14",
|
||||
FoundBy: "Archimedes",
|
||||
Locations: source.NewLocationSet(
|
||||
originalLocation,
|
||||
),
|
||||
Licenses: []string{
|
||||
"cc0-1.0",
|
||||
"MIT",
|
||||
},
|
||||
Language: "math",
|
||||
Type: PythonPkg,
|
||||
CPEs: []CPE{
|
||||
must(NewCPE(`cpe:2.3:a:Archimedes:pi:3.14:*:*:*:*:math:*:*`)),
|
||||
},
|
||||
PURL: "pkg:pypi/pi@3.14",
|
||||
MetadataType: PythonPackageMetadataType,
|
||||
Metadata: PythonPackageMetadata{
|
||||
Name: "pi",
|
||||
Version: "3.14",
|
||||
License: "cc0-1.0",
|
||||
Author: "Archimedes",
|
||||
AuthorEmail: "Archimedes@circles.io",
|
||||
Platform: "universe",
|
||||
SitePackagesRootPath: "Pi",
|
||||
},
|
||||
},
|
||||
other: Package{
|
||||
Name: "pi",
|
||||
Version: "3.14",
|
||||
FoundBy: "Archimedes",
|
||||
Locations: source.NewLocationSet(
|
||||
similarLocation, // NOTE: difference; we have a different layer but the same path
|
||||
),
|
||||
Licenses: []string{
|
||||
"cc0-1.0",
|
||||
"MIT",
|
||||
},
|
||||
Language: "math",
|
||||
Type: PythonPkg,
|
||||
CPEs: []CPE{
|
||||
must(NewCPE(`cpe:2.3:a:DIFFERENT:pi:3.14:*:*:*:*:math:*:*`)), // NOTE: difference
|
||||
},
|
||||
PURL: "pkg:pypi/pi@3.14",
|
||||
MetadataType: PythonPackageMetadataType,
|
||||
Metadata: PythonPackageMetadata{
|
||||
Name: "pi",
|
||||
Version: "3.14",
|
||||
License: "cc0-1.0",
|
||||
Author: "Archimedes",
|
||||
AuthorEmail: "Archimedes@circles.io",
|
||||
Platform: "universe",
|
||||
SitePackagesRootPath: "Pi",
|
||||
},
|
||||
},
|
||||
expected: &Package{
|
||||
Name: "pi",
|
||||
Version: "3.14",
|
||||
FoundBy: "Archimedes",
|
||||
Locations: source.NewLocationSet(
|
||||
originalLocation,
|
||||
similarLocation, // NOTE: merge!
|
||||
),
|
||||
Licenses: []string{
|
||||
"cc0-1.0",
|
||||
"MIT",
|
||||
},
|
||||
Language: "math",
|
||||
Type: PythonPkg,
|
||||
CPEs: []CPE{
|
||||
must(NewCPE(`cpe:2.3:a:Archimedes:pi:3.14:*:*:*:*:math:*:*`)),
|
||||
must(NewCPE(`cpe:2.3:a:DIFFERENT:pi:3.14:*:*:*:*:math:*:*`)), // NOTE: merge!
|
||||
},
|
||||
PURL: "pkg:pypi/pi@3.14",
|
||||
MetadataType: PythonPackageMetadataType,
|
||||
Metadata: PythonPackageMetadata{
|
||||
Name: "pi",
|
||||
Version: "3.14",
|
||||
License: "cc0-1.0",
|
||||
Author: "Archimedes",
|
||||
AuthorEmail: "Archimedes@circles.io",
|
||||
Platform: "universe",
|
||||
SitePackagesRootPath: "Pi",
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "error when there are different IDs",
|
||||
subject: Package{
|
||||
Name: "pi",
|
||||
Version: "3.14",
|
||||
FoundBy: "Archimedes",
|
||||
Locations: source.NewLocationSet(
|
||||
originalLocation,
|
||||
),
|
||||
Licenses: []string{
|
||||
"cc0-1.0",
|
||||
"MIT",
|
||||
},
|
||||
Language: "math",
|
||||
Type: PythonPkg,
|
||||
CPEs: []CPE{
|
||||
must(NewCPE(`cpe:2.3:a:Archimedes:pi:3.14:*:*:*:*:math:*:*`)),
|
||||
},
|
||||
PURL: "pkg:pypi/pi@3.14",
|
||||
MetadataType: PythonPackageMetadataType,
|
||||
Metadata: PythonPackageMetadata{
|
||||
Name: "pi",
|
||||
Version: "3.14",
|
||||
License: "cc0-1.0",
|
||||
Author: "Archimedes",
|
||||
AuthorEmail: "Archimedes@circles.io",
|
||||
Platform: "universe",
|
||||
SitePackagesRootPath: "Pi",
|
||||
},
|
||||
},
|
||||
other: Package{
|
||||
Name: "pi-DIFFERENT", // difference
|
||||
Version: "3.14",
|
||||
FoundBy: "Archimedes",
|
||||
Locations: source.NewLocationSet(
|
||||
originalLocation,
|
||||
),
|
||||
Licenses: []string{
|
||||
"cc0-1.0",
|
||||
"MIT",
|
||||
},
|
||||
Language: "math",
|
||||
Type: PythonPkg,
|
||||
CPEs: []CPE{
|
||||
must(NewCPE(`cpe:2.3:a:Archimedes:pi:3.14:*:*:*:*:math:*:*`)),
|
||||
},
|
||||
PURL: "pkg:pypi/pi@3.14",
|
||||
MetadataType: PythonPackageMetadataType,
|
||||
Metadata: PythonPackageMetadata{
|
||||
Name: "pi",
|
||||
Version: "3.14",
|
||||
License: "cc0-1.0",
|
||||
Author: "Archimedes",
|
||||
AuthorEmail: "Archimedes@circles.io",
|
||||
Platform: "universe",
|
||||
SitePackagesRootPath: "Pi",
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
tt.subject.SetID()
|
||||
tt.other.SetID()
|
||||
|
||||
err := tt.subject.merge(tt.other)
|
||||
if tt.expected == nil {
|
||||
require.Error(t, err)
|
||||
return
|
||||
}
|
||||
require.NoError(t, err)
|
||||
|
||||
tt.expected.SetID()
|
||||
require.Equal(t, tt.expected.id, tt.subject.id)
|
||||
|
||||
if diff := cmp.Diff(*tt.expected, tt.subject,
|
||||
cmp.AllowUnexported(Package{}),
|
||||
cmp.Comparer(
|
||||
func(x, y source.LocationSet) bool {
|
||||
return cmp.Equal(
|
||||
x.ToSlice(), y.ToSlice(),
|
||||
cmp.AllowUnexported(source.Location{}),
|
||||
cmp.AllowUnexported(file.Reference{}),
|
||||
)
|
||||
},
|
||||
),
|
||||
); diff != "" {
|
||||
t.Errorf("unexpected result from parsing (-expected +actual)\n%s", diff)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -18,10 +18,10 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
|
||||
name: "owns-by-real-path",
|
||||
setup: func(t testing.TB) ([]Package, []artifact.Relationship) {
|
||||
parent := Package{
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/a/path", "/another/path"),
|
||||
source.NewVirtualLocation("/b/path", "/bee/path"),
|
||||
},
|
||||
),
|
||||
Type: RpmPkg,
|
||||
MetadataType: RpmdbMetadataType,
|
||||
Metadata: RpmdbMetadata{
|
||||
@@ -35,10 +35,10 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
|
||||
parent.SetID()
|
||||
|
||||
child := Package{
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/c/path", "/another/path"),
|
||||
source.NewVirtualLocation("/d/path", "/another/path"),
|
||||
},
|
||||
),
|
||||
Type: NpmPkg,
|
||||
}
|
||||
child.SetID()
|
||||
@@ -61,10 +61,10 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
|
||||
name: "owns-by-virtual-path",
|
||||
setup: func(t testing.TB) ([]Package, []artifact.Relationship) {
|
||||
parent := Package{
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/a/path", "/some/other/path"),
|
||||
source.NewVirtualLocation("/b/path", "/bee/path"),
|
||||
},
|
||||
),
|
||||
Type: RpmPkg,
|
||||
MetadataType: RpmdbMetadataType,
|
||||
Metadata: RpmdbMetadata{
|
||||
@@ -78,10 +78,10 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
|
||||
parent.SetID()
|
||||
|
||||
child := Package{
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/c/path", "/another/path"),
|
||||
source.NewLocation("/d/path"),
|
||||
},
|
||||
),
|
||||
Type: NpmPkg,
|
||||
}
|
||||
child.SetID()
|
||||
@@ -103,10 +103,10 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
|
||||
name: "ignore-empty-path",
|
||||
setup: func(t testing.TB) ([]Package, []artifact.Relationship) {
|
||||
parent := Package{
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/a/path", "/some/other/path"),
|
||||
source.NewVirtualLocation("/b/path", "/bee/path"),
|
||||
},
|
||||
),
|
||||
Type: RpmPkg,
|
||||
MetadataType: RpmdbMetadataType,
|
||||
Metadata: RpmdbMetadata{
|
||||
@@ -121,10 +121,10 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
|
||||
parent.SetID()
|
||||
|
||||
child := Package{
|
||||
Locations: []source.Location{
|
||||
Locations: source.NewLocationSet(
|
||||
source.NewVirtualLocation("/c/path", "/another/path"),
|
||||
source.NewLocation("/d/path"),
|
||||
},
|
||||
),
|
||||
Type: NpmPkg,
|
||||
}
|
||||
|
||||
|
||||
@@ -20,6 +20,7 @@ const (
|
||||
GoModulePkg Type = "go-module"
|
||||
RustPkg Type = "rust-crate"
|
||||
KbPkg Type = "msrc-kb"
|
||||
DartPubPkg Type = "dart-pub"
|
||||
)
|
||||
|
||||
// AllPkgs represents all supported package types
|
||||
@@ -36,6 +37,7 @@ var AllPkgs = []Type{
|
||||
GoModulePkg,
|
||||
RustPkg,
|
||||
KbPkg,
|
||||
DartPubPkg,
|
||||
}
|
||||
|
||||
// PackageURLType returns the PURL package type for the current package.
|
||||
@@ -61,6 +63,8 @@ func (t Type) PackageURLType() string {
|
||||
return packageurl.TypeGolang
|
||||
case RustPkg:
|
||||
return "cargo"
|
||||
case DartPubPkg:
|
||||
return packageurl.TypePub
|
||||
default:
|
||||
// TODO: should this be a "generic" purl type instead?
|
||||
return ""
|
||||
@@ -98,6 +102,8 @@ func TypeByName(name string) Type {
|
||||
return GemPkg
|
||||
case "cargo", "crate":
|
||||
return RustPkg
|
||||
case packageurl.TypePub:
|
||||
return DartPubPkg
|
||||
default:
|
||||
return UnknownPkg
|
||||
}
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
package pkg
|
||||
|
||||
import (
|
||||
"github.com/scylladb/go-set/strset"
|
||||
"testing"
|
||||
|
||||
"github.com/scylladb/go-set/strset"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
@@ -46,6 +47,10 @@ func TestTypeFromPURL(t *testing.T) {
|
||||
purl: "pkg:cargo/clap@2.33.0",
|
||||
expected: RustPkg,
|
||||
},
|
||||
{
|
||||
purl: "pkg:pub/util@1.2.34?hosted_url=pub.hosted.org",
|
||||
expected: DartPubPkg,
|
||||
},
|
||||
{
|
||||
purl: "pkg:composer/laravel/laravel@5.5.0",
|
||||
expected: PhpComposerPkg,
|
||||
|
||||
@@ -25,6 +25,20 @@ func TestPackageURL(t *testing.T) {
|
||||
},
|
||||
expected: "pkg:golang/github.com/anchore/syft@v0.1.0",
|
||||
},
|
||||
{
|
||||
name: "pub",
|
||||
pkg: Package{
|
||||
Name: "bad-name",
|
||||
Version: "0.1.0",
|
||||
Type: DartPubPkg,
|
||||
Metadata: DartPubMetadata{
|
||||
Name: "name",
|
||||
Version: "0.2.0",
|
||||
HostedURL: "pub.hosted.org",
|
||||
},
|
||||
},
|
||||
expected: "pkg:pub/name@0.2.0?hosted_url=pub.hosted.org",
|
||||
},
|
||||
{
|
||||
name: "python",
|
||||
pkg: Package{
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
package source
|
||||
|
||||
import (
|
||||
"sort"
|
||||
|
||||
"github.com/mitchellh/hashstructure/v2"
|
||||
"github.com/scylladb/go-set/strset"
|
||||
)
|
||||
|
||||
type CoordinateSet struct {
|
||||
set map[Coordinates]struct{}
|
||||
}
|
||||
|
||||
func NewCoordinateSet(coordinates ...Coordinates) (s CoordinateSet) {
|
||||
for _, l := range coordinates {
|
||||
s.Add(l)
|
||||
}
|
||||
|
||||
return s
|
||||
}
|
||||
|
||||
func (s *CoordinateSet) Add(coordinates ...Coordinates) {
|
||||
if s.set == nil {
|
||||
s.set = make(map[Coordinates]struct{})
|
||||
}
|
||||
for _, l := range coordinates {
|
||||
s.set[l] = struct{}{}
|
||||
}
|
||||
}
|
||||
|
||||
func (s CoordinateSet) Remove(coordinates ...Coordinates) {
|
||||
if s.set == nil {
|
||||
return
|
||||
}
|
||||
for _, l := range coordinates {
|
||||
delete(s.set, l)
|
||||
}
|
||||
}
|
||||
|
||||
func (s CoordinateSet) Contains(l Coordinates) bool {
|
||||
if s.set == nil {
|
||||
return false
|
||||
}
|
||||
_, ok := s.set[l]
|
||||
return ok
|
||||
}
|
||||
|
||||
func (s CoordinateSet) Paths() []string {
|
||||
if s.set == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
paths := strset.New()
|
||||
for _, c := range s.ToSlice() {
|
||||
paths.Add(c.RealPath)
|
||||
}
|
||||
pathSlice := paths.List()
|
||||
sort.Strings(pathSlice)
|
||||
return pathSlice
|
||||
}
|
||||
|
||||
func (s CoordinateSet) ToSlice() []Coordinates {
|
||||
if s.set == nil {
|
||||
return nil
|
||||
}
|
||||
coordinates := make([]Coordinates, len(s.set))
|
||||
idx := 0
|
||||
for v := range s.set {
|
||||
coordinates[idx] = v
|
||||
idx++
|
||||
}
|
||||
sort.SliceStable(coordinates, func(i, j int) bool {
|
||||
if coordinates[i].RealPath == coordinates[j].RealPath {
|
||||
return coordinates[i].FileSystemID < coordinates[j].FileSystemID
|
||||
}
|
||||
return coordinates[i].RealPath < coordinates[j].RealPath
|
||||
})
|
||||
return coordinates
|
||||
}
|
||||
|
||||
func (s CoordinateSet) Hash() (uint64, error) {
|
||||
return hashstructure.Hash(s.ToSlice(), hashstructure.FormatV2, &hashstructure.HashOptions{
|
||||
ZeroNil: true,
|
||||
SlicesAsSets: true,
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
package source
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/stretchr/testify/require"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestCoordinatesSet(t *testing.T) {
|
||||
|
||||
binA := Coordinates{
|
||||
RealPath: "/bin",
|
||||
FileSystemID: "a",
|
||||
}
|
||||
|
||||
binB := Coordinates{
|
||||
RealPath: "/bin",
|
||||
FileSystemID: "b",
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
input []Coordinates
|
||||
expected []Coordinates
|
||||
}{
|
||||
{
|
||||
name: "de-dup same location",
|
||||
input: []Coordinates{
|
||||
binA, binA, binA,
|
||||
},
|
||||
expected: []Coordinates{
|
||||
binA,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "dont de-dup different filesystem",
|
||||
input: []Coordinates{
|
||||
binB, binA,
|
||||
},
|
||||
expected: []Coordinates{
|
||||
binA, binB,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
set := NewCoordinateSet(test.input...)
|
||||
assert.Equal(t, test.expected, set.ToSlice())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCoordinateSet_Hash(t *testing.T) {
|
||||
etcA := Coordinates{
|
||||
RealPath: "/etc",
|
||||
FileSystemID: "a",
|
||||
}
|
||||
|
||||
etcB := Coordinates{
|
||||
RealPath: "/etc",
|
||||
FileSystemID: "b",
|
||||
}
|
||||
|
||||
binA := Coordinates{
|
||||
RealPath: "/bin",
|
||||
FileSystemID: "a",
|
||||
}
|
||||
|
||||
binB := Coordinates{
|
||||
RealPath: "/bin",
|
||||
FileSystemID: "b",
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
setA CoordinateSet
|
||||
setB CoordinateSet
|
||||
want assert.ComparisonAssertionFunc
|
||||
}{
|
||||
{
|
||||
name: "empty sets have the same hash",
|
||||
setA: NewCoordinateSet(),
|
||||
setB: NewCoordinateSet(),
|
||||
want: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "sets with same elements have the same hash",
|
||||
setA: NewCoordinateSet(binA, etcA),
|
||||
setB: NewCoordinateSet(etcA, binA),
|
||||
want: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "sets with different elements have different hashes",
|
||||
setA: NewCoordinateSet(binA, etcA),
|
||||
setB: NewCoordinateSet(binA),
|
||||
want: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "sets with same paths but different FS IDs have different hashes",
|
||||
setA: NewCoordinateSet(etcA, binA),
|
||||
setB: NewCoordinateSet(etcB, binB),
|
||||
want: assert.NotEqual,
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
gotA, err := artifact.IDByHash(tt.setA)
|
||||
require.NoError(t, err)
|
||||
gotB, err := artifact.IDByHash(tt.setB)
|
||||
require.NoError(t, err)
|
||||
tt.want(t, gotA, gotB)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -2,7 +2,6 @@ package source
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
|
||||
"github.com/anchore/syft/internal/log"
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
@@ -14,18 +13,6 @@ type Coordinates struct {
|
||||
FileSystemID string `json:"layerID,omitempty" cyclonedx:"layerID"` // An ID representing the filesystem. For container images, this is a layer digest. For directories or a root filesystem, this is blank.
|
||||
}
|
||||
|
||||
// CoordinateSet represents a set of string types.
|
||||
type CoordinateSet map[Coordinates]struct{}
|
||||
|
||||
// NewCoordinateSet creates a CoordinateSet populated with values from the given slice.
|
||||
func NewCoordinateSet(start ...Coordinates) CoordinateSet {
|
||||
ret := make(CoordinateSet)
|
||||
for _, s := range start {
|
||||
ret.Add(s)
|
||||
}
|
||||
return ret
|
||||
}
|
||||
|
||||
func (c Coordinates) ID() artifact.ID {
|
||||
f, err := artifact.IDByHash(c)
|
||||
if err != nil {
|
||||
@@ -45,37 +32,3 @@ func (c Coordinates) String() string {
|
||||
}
|
||||
return fmt.Sprintf("Location<%s>", str)
|
||||
}
|
||||
|
||||
// Add a string to the set.
|
||||
func (s CoordinateSet) Add(i Coordinates) {
|
||||
s[i] = struct{}{}
|
||||
}
|
||||
|
||||
// Remove a string from the set.
|
||||
func (s CoordinateSet) Remove(i Coordinates) {
|
||||
delete(s, i)
|
||||
}
|
||||
|
||||
// Contains indicates if the given string is contained within the set.
|
||||
func (s CoordinateSet) Contains(i Coordinates) bool {
|
||||
_, ok := s[i]
|
||||
return ok
|
||||
}
|
||||
|
||||
// ToSlice returns a sorted slice of Locations that are contained within the set.
|
||||
func (s CoordinateSet) ToSlice() []Coordinates {
|
||||
ret := make([]Coordinates, len(s))
|
||||
idx := 0
|
||||
for v := range s {
|
||||
ret[idx] = v
|
||||
idx++
|
||||
}
|
||||
|
||||
sort.SliceStable(ret, func(i, j int) bool {
|
||||
if ret[i].RealPath == ret[j].RealPath {
|
||||
return ret[i].FileSystemID < ret[j].FileSystemID
|
||||
}
|
||||
return ret[i].RealPath < ret[j].RealPath
|
||||
})
|
||||
return ret
|
||||
}
|
||||
|
||||
@@ -1,51 +0,0 @@
|
||||
package source
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestCoordinateSet(t *testing.T) {
|
||||
|
||||
binA := Coordinates{
|
||||
RealPath: "/bin",
|
||||
FileSystemID: "a",
|
||||
}
|
||||
|
||||
binB := Coordinates{
|
||||
RealPath: "/bin",
|
||||
FileSystemID: "b",
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
input []Coordinates
|
||||
expected []Coordinates
|
||||
}{
|
||||
{
|
||||
name: "de-dup same location",
|
||||
input: []Coordinates{
|
||||
binA, binA, binA,
|
||||
},
|
||||
expected: []Coordinates{
|
||||
binA,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "dont de-dup different filesystem",
|
||||
input: []Coordinates{
|
||||
binB, binA,
|
||||
},
|
||||
expected: []Coordinates{
|
||||
binA, binB,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
assert.Equal(t, test.expected, NewCoordinateSet(test.input...).ToSlice())
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -132,10 +132,15 @@ func (r *directoryResolver) indexTree(root string, stager *progress.Stage) ([]st
|
||||
}
|
||||
|
||||
func (r *directoryResolver) indexPath(path string, info os.FileInfo, err error) (string, error) {
|
||||
// link cycles could cause a revisit --we should not allow this
|
||||
if r.hasBeenIndexed(path) {
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// ignore any path which a filter function returns true
|
||||
for _, filterFn := range r.pathFilterFns {
|
||||
if filterFn != nil && filterFn(path, info) {
|
||||
if info.IsDir() {
|
||||
if info != nil && info.IsDir() {
|
||||
return "", fs.SkipDir
|
||||
}
|
||||
return "", nil
|
||||
@@ -146,11 +151,6 @@ func (r *directoryResolver) indexPath(path string, info os.FileInfo, err error)
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// link cycles could cause a revisit --we should not allow this
|
||||
if r.hasBeenIndexed(path) {
|
||||
return "", nil
|
||||
}
|
||||
|
||||
if info == nil {
|
||||
// walk may not be able to provide a FileInfo object, don't allow for this to stop indexing; keep track of the paths and continue.
|
||||
r.errPaths[path] = fmt.Errorf("no file info observable at path=%q", path)
|
||||
|
||||
@@ -865,3 +865,24 @@ func Test_IncludeRootPathInIndex(t *testing.T) {
|
||||
_, exists = resolver.metadata[ref.ID()]
|
||||
require.True(t, exists)
|
||||
}
|
||||
|
||||
func TestDirectoryResolver_indexPath(t *testing.T) {
|
||||
// TODO: Ideally we can use an OS abstraction, which would obviate the need for real FS setup.
|
||||
tempFile, err := os.CreateTemp("", "")
|
||||
require.NoError(t, err)
|
||||
|
||||
resolver, err := newDirectoryResolver(tempFile.Name())
|
||||
require.NoError(t, err)
|
||||
|
||||
t.Run("filtering path with nil os.FileInfo", func(t *testing.T) {
|
||||
// We use one of these prefixes in order to trigger a pathFilterFn
|
||||
filteredPath := unixSystemRuntimePrefixes[0]
|
||||
|
||||
var fileInfo os.FileInfo = nil
|
||||
|
||||
assert.NotPanics(t, func() {
|
||||
_, err := resolver.indexPath(filteredPath, fileInfo, nil)
|
||||
assert.NoError(t, err)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
@@ -109,3 +109,9 @@ func (l Location) String() string {
|
||||
}
|
||||
return fmt.Sprintf("Location<%s>", str)
|
||||
}
|
||||
|
||||
func (l Location) Equals(other Location) bool {
|
||||
return l.RealPath == other.RealPath &&
|
||||
l.VirtualPath == other.VirtualPath &&
|
||||
l.FileSystemID == other.FileSystemID
|
||||
}
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
package source
|
||||
|
||||
import (
|
||||
"sort"
|
||||
|
||||
"github.com/mitchellh/hashstructure/v2"
|
||||
)
|
||||
|
||||
type LocationSet struct {
|
||||
set map[Location]struct{}
|
||||
}
|
||||
|
||||
func NewLocationSet(locations ...Location) (s LocationSet) {
|
||||
for _, l := range locations {
|
||||
s.Add(l)
|
||||
}
|
||||
|
||||
return s
|
||||
}
|
||||
|
||||
func (s *LocationSet) Add(locations ...Location) {
|
||||
if s.set == nil {
|
||||
s.set = make(map[Location]struct{})
|
||||
}
|
||||
for _, l := range locations {
|
||||
s.set[l] = struct{}{}
|
||||
}
|
||||
}
|
||||
|
||||
func (s LocationSet) Remove(locations ...Location) {
|
||||
if s.set == nil {
|
||||
return
|
||||
}
|
||||
for _, l := range locations {
|
||||
delete(s.set, l)
|
||||
}
|
||||
}
|
||||
|
||||
func (s LocationSet) Contains(l Location) bool {
|
||||
if s.set == nil {
|
||||
return false
|
||||
}
|
||||
_, ok := s.set[l]
|
||||
return ok
|
||||
}
|
||||
|
||||
func (s LocationSet) ToSlice() []Location {
|
||||
if s.set == nil {
|
||||
return nil
|
||||
}
|
||||
locations := make([]Location, len(s.set))
|
||||
idx := 0
|
||||
for v := range s.set {
|
||||
locations[idx] = v
|
||||
idx++
|
||||
}
|
||||
sort.Sort(Locations(locations))
|
||||
return locations
|
||||
}
|
||||
|
||||
func (s *LocationSet) CoordinateSet() CoordinateSet {
|
||||
if s.set == nil {
|
||||
return NewCoordinateSet()
|
||||
}
|
||||
set := NewCoordinateSet()
|
||||
for l := range s.set {
|
||||
set.Add(l.Coordinates)
|
||||
}
|
||||
return set
|
||||
}
|
||||
|
||||
func (s LocationSet) Hash() (uint64, error) {
|
||||
// access paths and filesystem IDs are not considered when hashing a location set, only the real paths
|
||||
return hashstructure.Hash(s.CoordinateSet().Paths(), hashstructure.FormatV2, &hashstructure.HashOptions{
|
||||
ZeroNil: true,
|
||||
SlicesAsSets: true,
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,178 @@
|
||||
package source
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/stretchr/testify/require"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestLocationSet(t *testing.T) {
|
||||
|
||||
etcHostsLinkVar := Location{
|
||||
Coordinates: Coordinates{
|
||||
RealPath: "/etc/hosts",
|
||||
FileSystemID: "a",
|
||||
},
|
||||
VirtualPath: "/var/etc/hosts",
|
||||
}
|
||||
|
||||
etcHostsLinkHome := Location{
|
||||
Coordinates: Coordinates{
|
||||
RealPath: "/etc/hosts",
|
||||
FileSystemID: "a",
|
||||
},
|
||||
VirtualPath: "/home/wagoodman/hosts",
|
||||
}
|
||||
|
||||
binA := Location{
|
||||
Coordinates: Coordinates{
|
||||
RealPath: "/bin",
|
||||
FileSystemID: "a",
|
||||
},
|
||||
VirtualPath: "/usr/bin",
|
||||
}
|
||||
|
||||
binB := Location{
|
||||
Coordinates: Coordinates{
|
||||
RealPath: "/bin",
|
||||
FileSystemID: "b",
|
||||
},
|
||||
VirtualPath: "/usr/bin",
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
input []Location
|
||||
expected []Location
|
||||
}{
|
||||
{
|
||||
name: "de-dup same location",
|
||||
input: []Location{
|
||||
binA, binA, binA,
|
||||
},
|
||||
expected: []Location{
|
||||
binA,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "dont de-dup different filesystem",
|
||||
input: []Location{
|
||||
binB, binA,
|
||||
},
|
||||
expected: []Location{
|
||||
binA, binB,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "dont de-dup different virtual paths",
|
||||
input: []Location{
|
||||
etcHostsLinkVar, etcHostsLinkHome,
|
||||
},
|
||||
expected: []Location{
|
||||
etcHostsLinkHome, etcHostsLinkVar,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
set := NewLocationSet(test.input...)
|
||||
assert.Equal(t, test.expected, set.ToSlice())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLocationSet_Hash(t *testing.T) {
|
||||
etcAlink := Location{
|
||||
Coordinates: Coordinates{
|
||||
RealPath: "/etc/hosts",
|
||||
FileSystemID: "a",
|
||||
},
|
||||
VirtualPath: "/var/etc/hosts",
|
||||
}
|
||||
|
||||
etcA := Location{
|
||||
Coordinates: Coordinates{
|
||||
RealPath: "/etc/hosts",
|
||||
FileSystemID: "a",
|
||||
},
|
||||
}
|
||||
|
||||
etcB := Location{
|
||||
Coordinates: Coordinates{
|
||||
RealPath: "/etc/hosts",
|
||||
FileSystemID: "b",
|
||||
},
|
||||
}
|
||||
|
||||
binA := Location{
|
||||
Coordinates: Coordinates{
|
||||
RealPath: "/bin",
|
||||
FileSystemID: "a",
|
||||
},
|
||||
VirtualPath: "/usr/bin",
|
||||
}
|
||||
|
||||
binB := Location{
|
||||
Coordinates: Coordinates{
|
||||
RealPath: "/bin",
|
||||
FileSystemID: "b",
|
||||
},
|
||||
VirtualPath: "/usr/bin",
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
setA LocationSet
|
||||
setB LocationSet
|
||||
want assert.ComparisonAssertionFunc
|
||||
}{
|
||||
{
|
||||
name: "empty sets have the same hash",
|
||||
setA: NewLocationSet(),
|
||||
setB: NewLocationSet(),
|
||||
want: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "sets with same elements accessed through different paths have the same hash",
|
||||
setA: NewLocationSet(binA, etcA),
|
||||
setB: NewLocationSet(etcAlink, binA),
|
||||
want: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "sets with same elements have the same hash",
|
||||
setA: NewLocationSet(binA, etcA),
|
||||
setB: NewLocationSet(etcA, binA),
|
||||
want: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "sets with different element counts have different hashes",
|
||||
setA: NewLocationSet(binA, etcA),
|
||||
setB: NewLocationSet(binA),
|
||||
want: assert.NotEqual,
|
||||
},
|
||||
{
|
||||
name: "sets with same path but different FS IDs have the same hash",
|
||||
setA: NewLocationSet(binA),
|
||||
setB: NewLocationSet(binB),
|
||||
want: assert.Equal,
|
||||
},
|
||||
{
|
||||
name: "sets with same paths but different FS IDs have the same hash",
|
||||
setA: NewLocationSet(etcA, binA),
|
||||
setB: NewLocationSet(binB, etcB),
|
||||
want: assert.Equal,
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
gotA, err := artifact.IDByHash(tt.setA)
|
||||
require.NoError(t, err)
|
||||
gotB, err := artifact.IDByHash(tt.setB)
|
||||
require.NoError(t, err)
|
||||
tt.want(t, gotA, gotB)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
package source
|
||||
|
||||
type Locations []Location
|
||||
|
||||
func (l Locations) Len() int {
|
||||
return len(l)
|
||||
}
|
||||
|
||||
func (l Locations) Less(i, j int) bool {
|
||||
if l[i].RealPath == l[j].RealPath {
|
||||
if l[i].VirtualPath == l[j].VirtualPath {
|
||||
return l[i].FileSystemID < l[j].FileSystemID
|
||||
}
|
||||
return l[i].VirtualPath < l[j].VirtualPath
|
||||
}
|
||||
return l[i].RealPath < l[j].RealPath
|
||||
}
|
||||
|
||||
func (l Locations) Swap(i, j int) {
|
||||
l[i], l[j] = l[j], l[i]
|
||||
}
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
)
|
||||
|
||||
func TestPackagesCmdFlags(t *testing.T) {
|
||||
hiddenPackagesImage := "docker-archive:" + getFixtureImage(t, "image-hidden-packages")
|
||||
coverageImage := "docker-archive:" + getFixtureImage(t, "image-pkg-coverage")
|
||||
//badBinariesImage := "docker-archive:" + getFixtureImage(t, "image-bad-binaries")
|
||||
tmp := t.TempDir() + "/"
|
||||
@@ -100,21 +101,34 @@ func TestPackagesCmdFlags(t *testing.T) {
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "all-layers-scope-flag",
|
||||
args: []string{"packages", "-o", "json", "-s", "all-layers", coverageImage},
|
||||
name: "squashed-scope-flag-hidden-packages",
|
||||
args: []string{"packages", "-o", "json", "-s", "squashed", hiddenPackagesImage},
|
||||
assertions: []traitAssertion{
|
||||
assertPackageCount(22),
|
||||
assertPackageCount(162),
|
||||
assertNotInOutput("vsftpd"), // hidden package
|
||||
assertSuccessfulReturnCode,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "all-layers-scope-flag",
|
||||
args: []string{"packages", "-o", "json", "-s", "all-layers", hiddenPackagesImage},
|
||||
assertions: []traitAssertion{
|
||||
assertPackageCount(163), // packages are now deduplicated for this case
|
||||
assertInOutput("all-layers"),
|
||||
assertInOutput("vsftpd"), // hidden package
|
||||
assertSuccessfulReturnCode,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "all-layers-scope-flag-by-env",
|
||||
args: []string{"packages", "-o", "json", coverageImage},
|
||||
args: []string{"packages", "-o", "json", hiddenPackagesImage},
|
||||
env: map[string]string{
|
||||
"SYFT_PACKAGE_CATALOGER_SCOPE": "all-layers",
|
||||
},
|
||||
assertions: []traitAssertion{
|
||||
assertPackageCount(22),
|
||||
assertPackageCount(163), // packages are now deduplicated for this case
|
||||
assertInOutput("all-layers"),
|
||||
assertInOutput("vsftpd"), // hidden package
|
||||
assertSuccessfulReturnCode,
|
||||
},
|
||||
},
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
FROM centos:7.9.2009
|
||||
# all-layers scope should pickup on vsftpd
|
||||
RUN yum install -y vsftpd
|
||||
RUN yum remove -y vsftpd
|
||||
@@ -186,6 +186,19 @@ var dirOnlyTestCases = []testCase{
|
||||
"alcaeus/mongo-php-adapter": "1.1.11",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "find pubspec lock packages",
|
||||
pkgType: pkg.DartPubPkg,
|
||||
pkgLanguage: pkg.Dart,
|
||||
pkgInfo: map[string]string{
|
||||
"ansicolor": "1.1.1",
|
||||
"archive": "2.0.13",
|
||||
"args": "1.6.0",
|
||||
"key_binder": "1.11.20",
|
||||
"ale": "3.3.0",
|
||||
"analyzer": "0.40.7",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
var commonTestCases = []testCase{
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
package integration
|
||||
|
||||
import (
|
||||
"github.com/stretchr/testify/require"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"github.com/anchore/syft/syft/linux"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger"
|
||||
"github.com/google/go-cmp/cmp"
|
||||
@@ -53,7 +54,7 @@ func BenchmarkImagePackageCatalogers(b *testing.B) {
|
||||
}
|
||||
|
||||
func TestPkgCoverageImage(t *testing.T) {
|
||||
sbom, _ := catalogFixtureImage(t, "image-pkg-coverage")
|
||||
sbom, _ := catalogFixtureImage(t, "image-pkg-coverage", source.SquashedScope)
|
||||
|
||||
observedLanguages := internal.NewStringSet()
|
||||
definedLanguages := internal.NewStringSet()
|
||||
@@ -64,6 +65,7 @@ func TestPkgCoverageImage(t *testing.T) {
|
||||
// for image scans we should not expect to see any of the following package types
|
||||
definedLanguages.Remove(pkg.Go.String())
|
||||
definedLanguages.Remove(pkg.Rust.String())
|
||||
definedLanguages.Remove(pkg.Dart.String())
|
||||
|
||||
observedPkgs := internal.NewStringSet()
|
||||
definedPkgs := internal.NewStringSet()
|
||||
@@ -75,6 +77,7 @@ func TestPkgCoverageImage(t *testing.T) {
|
||||
definedPkgs.Remove(string(pkg.KbPkg))
|
||||
definedPkgs.Remove(string(pkg.GoModulePkg))
|
||||
definedPkgs.Remove(string(pkg.RustPkg))
|
||||
definedPkgs.Remove(string(pkg.DartPubPkg))
|
||||
|
||||
var cases []testCase
|
||||
cases = append(cases, commonTestCases...)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package integration
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
@@ -9,7 +10,7 @@ import (
|
||||
)
|
||||
|
||||
func TestDistroImage(t *testing.T) {
|
||||
sbom, _ := catalogFixtureImage(t, "image-distro-id")
|
||||
sbom, _ := catalogFixtureImage(t, "image-distro-id", source.SquashedScope)
|
||||
|
||||
expected := &linux.Release{
|
||||
PrettyName: "BusyBox v1.31.1",
|
||||
|
||||
@@ -2,12 +2,14 @@ package integration
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"regexp"
|
||||
"testing"
|
||||
|
||||
"fmt"
|
||||
"github.com/anchore/syft/internal/formats/cyclonedxjson"
|
||||
"github.com/anchore/syft/internal/formats/cyclonedxxml"
|
||||
"github.com/anchore/syft/internal/formats/syftjson"
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"regexp"
|
||||
"testing"
|
||||
|
||||
"github.com/anchore/syft/syft/sbom"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
@@ -51,12 +53,12 @@ func TestEncodeDecodeEncodeCycleComparison(t *testing.T) {
|
||||
},
|
||||
},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(string(test.formatOption), func(t *testing.T) {
|
||||
|
||||
// use second image for relationships
|
||||
for _, image := range []string{"image-pkg-coverage", "image-owning-package"} {
|
||||
originalSBOM, _ := catalogFixtureImage(t, image)
|
||||
for _, test := range tests {
|
||||
// use second image for relationships
|
||||
for _, image := range []string{"image-pkg-coverage", "image-owning-package"} {
|
||||
t.Run(fmt.Sprintf("%s/%s", test.formatOption, image), func(t *testing.T) {
|
||||
originalSBOM, _ := catalogFixtureImage(t, image, source.SquashedScope)
|
||||
|
||||
format := syft.FormatByID(test.formatOption)
|
||||
require.NotNil(t, format)
|
||||
@@ -87,7 +89,7 @@ func TestEncodeDecodeEncodeCycleComparison(t *testing.T) {
|
||||
t.Errorf("diff: %s", dmp.DiffPrettyText(diffs))
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -14,7 +14,7 @@ func TestNpmPackageLockDirectory(t *testing.T) {
|
||||
foundPackages := internal.NewStringSet()
|
||||
|
||||
for actualPkg := range sbom.Artifacts.PackageCatalog.Enumerate(pkg.NpmPkg) {
|
||||
for _, actualLocation := range actualPkg.Locations {
|
||||
for _, actualLocation := range actualPkg.Locations.ToSlice() {
|
||||
if strings.Contains(actualLocation.RealPath, "node_modules") {
|
||||
t.Errorf("found packages from package-lock.json in node_modules: %s", actualLocation)
|
||||
}
|
||||
@@ -35,7 +35,7 @@ func TestYarnPackageLockDirectory(t *testing.T) {
|
||||
foundPackages := internal.NewStringSet()
|
||||
|
||||
for actualPkg := range sbom.Artifacts.PackageCatalog.Enumerate(pkg.NpmPkg) {
|
||||
for _, actualLocation := range actualPkg.Locations {
|
||||
for _, actualLocation := range actualPkg.Locations.ToSlice() {
|
||||
if strings.Contains(actualLocation.RealPath, "node_modules") {
|
||||
t.Errorf("found packages from yarn.lock in node_modules: %s", actualLocation)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
package integration
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestPackageDeduplication(t *testing.T) {
|
||||
tests := []struct {
|
||||
scope source.Scope
|
||||
packageCount int
|
||||
instanceCount map[string]int
|
||||
locationCount map[string]int
|
||||
}{
|
||||
{
|
||||
scope: source.AllLayersScope,
|
||||
packageCount: 172, // without deduplication this would be 618
|
||||
instanceCount: map[string]int{
|
||||
"basesystem": 1,
|
||||
"wget": 1,
|
||||
"curl": 2, // upgraded in the image
|
||||
"vsftpd": 1,
|
||||
"httpd": 1,
|
||||
},
|
||||
locationCount: map[string]int{
|
||||
"basesystem-10.0-7.el7.centos": 4,
|
||||
"curl-7.29.0-59.el7": 1, // from base image
|
||||
"curl-7.29.0-59.el7_9.1": 3, // upgrade
|
||||
"wget-1.14-18.el7_6.1": 3,
|
||||
"vsftpd-3.0.2-29.el7_9": 2,
|
||||
"httpd-2.4.6-97.el7.centos.5": 1,
|
||||
},
|
||||
},
|
||||
{
|
||||
scope: source.SquashedScope,
|
||||
packageCount: 170,
|
||||
instanceCount: map[string]int{
|
||||
"basesystem": 1,
|
||||
"wget": 1,
|
||||
"curl": 1, // upgraded, but the most recent
|
||||
"vsftpd": 1,
|
||||
"httpd": 1,
|
||||
},
|
||||
locationCount: map[string]int{
|
||||
"basesystem-10.0-7.el7.centos": 1,
|
||||
"curl-7.29.0-59.el7_9.1": 1, // upgrade
|
||||
"wget-1.14-18.el7_6.1": 1,
|
||||
"vsftpd-3.0.2-29.el7_9": 1,
|
||||
"httpd-2.4.6-97.el7.centos.5": 1,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(string(tt.scope), func(t *testing.T) {
|
||||
sbom, _ := catalogFixtureImage(t, "image-vertical-package-dups", tt.scope)
|
||||
|
||||
assert.Equal(t, tt.packageCount, sbom.Artifacts.PackageCatalog.PackageCount())
|
||||
for name, expectedInstanceCount := range tt.instanceCount {
|
||||
pkgs := sbom.Artifacts.PackageCatalog.PackagesByName(name)
|
||||
|
||||
// with multiple packages with the same name, something is wrong (or this is the wrong fixture)
|
||||
require.Len(t, pkgs, expectedInstanceCount)
|
||||
for _, p := range pkgs {
|
||||
nameVersion := fmt.Sprintf("%s-%s", name, p.Version)
|
||||
expectedLocationCount, ok := tt.locationCount[nameVersion]
|
||||
if !ok {
|
||||
t.Fatalf("missing name-version: %s", nameVersion)
|
||||
}
|
||||
|
||||
// we should see merged locations (assumption, there was 1 location for each package)
|
||||
assert.Len(t, p.Locations.ToSlice(), expectedLocationCount)
|
||||
|
||||
// all paths should match
|
||||
assert.Len(t, p.Locations.CoordinateSet().Paths(), 1)
|
||||
}
|
||||
}
|
||||
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,7 @@ package integration
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"testing"
|
||||
|
||||
"github.com/anchore/syft/internal/formats/syftjson"
|
||||
@@ -22,7 +23,7 @@ func TestPackageOwnershipRelationships(t *testing.T) {
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.fixture, func(t *testing.T) {
|
||||
sbom, _ := catalogFixtureImage(t, test.fixture)
|
||||
sbom, _ := catalogFixtureImage(t, test.fixture, source.SquashedScope)
|
||||
|
||||
output := bytes.NewBufferString("")
|
||||
err := syftjson.Format().Encode(output, sbom)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package integration
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"testing"
|
||||
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
@@ -9,7 +10,7 @@ import (
|
||||
func TestRegression212ApkBufferSize(t *testing.T) {
|
||||
// This is a regression test for issue #212 (https://github.com/anchore/syft/issues/212) in which the apk db could
|
||||
// not be processed due to a scanner buffer that was too small
|
||||
sbom, _ := catalogFixtureImage(t, "image-large-apk-data")
|
||||
sbom, _ := catalogFixtureImage(t, "image-large-apk-data", source.SquashedScope)
|
||||
|
||||
expectedPkgs := 58
|
||||
actualPkgs := 0
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package integration
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -15,12 +16,12 @@ func TestRegressionGoArchDiscovery(t *testing.T) {
|
||||
)
|
||||
// This is a regression test to make sure the way we detect go binary packages
|
||||
// stays consistent and reproducible as the tool chain evolves
|
||||
sbom, _ := catalogFixtureImage(t, "image-go-bin-arch-coverage")
|
||||
sbom, _ := catalogFixtureImage(t, "image-go-bin-arch-coverage", source.SquashedScope)
|
||||
|
||||
var actualELF, actualWIN, actualMACOS int
|
||||
|
||||
for p := range sbom.Artifacts.PackageCatalog.Enumerate(pkg.GoModulePkg) {
|
||||
for _, l := range p.Locations {
|
||||
for _, l := range p.Locations.ToSlice() {
|
||||
switch {
|
||||
case strings.Contains(l.RealPath, "elf"):
|
||||
actualELF++
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
package integration
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRegressionJavaNoMainPackage(t *testing.T) { // Regression: https://github.com/anchore/syft/issues/252
|
||||
catalogFixtureImage(t, "image-java-no-main-package")
|
||||
catalogFixtureImage(t, "image-java-no-main-package", source.SquashedScope)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
# Generated by pub
|
||||
# See https://dart.dev/tools/pub/glossary#lockfile
|
||||
packages:
|
||||
ale:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: ale
|
||||
url: "https://pub.hosted.org"
|
||||
source: hosted
|
||||
version: "3.3.0"
|
||||
analyzer:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: analyzer
|
||||
url: "https://pub.dartlang.org"
|
||||
source: hosted
|
||||
version: "0.40.7"
|
||||
ansicolor:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: ansicolor
|
||||
url: "https://pub.dartlang.org"
|
||||
source: hosted
|
||||
version: "1.1.1"
|
||||
archive:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: archive
|
||||
url: "https://pub.dartlang.org"
|
||||
source: hosted
|
||||
version: "2.0.13"
|
||||
args:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: args
|
||||
url: "https://pub.dartlang.org"
|
||||
source: hosted
|
||||
version: "1.6.0"
|
||||
key_binder:
|
||||
dependency: "direct main"
|
||||
description:
|
||||
path: "."
|
||||
ref: HEAD
|
||||
resolved-ref: "3f7b3a6350e73c7dcac45301c0e18fbd42af02f7"
|
||||
url: "git@github.com:Workiva/key_binder.git"
|
||||
source: git
|
||||
version: "1.11.20"
|
||||
sdks:
|
||||
dart: ">=2.12.0 <3.0.0"
|
||||
@@ -0,0 +1,6 @@
|
||||
FROM centos:7.9.2009
|
||||
# modifying the RPM DB multiple times will result in duplicate packages when using all-layers (if there was no de-dup logic)
|
||||
# curl is tricky, it already exists in the image and is being upgraded
|
||||
RUN yum install -y wget curl
|
||||
RUN yum install -y vsftpd
|
||||
RUN yum install -y httpd
|
||||
@@ -13,7 +13,7 @@ import (
|
||||
"github.com/anchore/syft/syft/source"
|
||||
)
|
||||
|
||||
func catalogFixtureImage(t *testing.T, fixtureImageName string) (sbom.SBOM, *source.Source) {
|
||||
func catalogFixtureImage(t *testing.T, fixtureImageName string, scope source.Scope) (sbom.SBOM, *source.Source) {
|
||||
imagetest.GetFixtureImage(t, "docker-archive", fixtureImageName)
|
||||
tarPath := imagetest.GetFixtureImageTarPath(t, fixtureImageName)
|
||||
userInput := "docker-archive:" + tarPath
|
||||
@@ -25,7 +25,7 @@ func catalogFixtureImage(t *testing.T, fixtureImageName string) (sbom.SBOM, *sou
|
||||
|
||||
// TODO: this would be better with functional options (after/during API refactor)
|
||||
c := cataloger.DefaultConfig()
|
||||
c.Search.Scope = source.SquashedScope
|
||||
c.Search.Scope = scope
|
||||
pkgCatalog, relationships, actualDistro, err := syft.CatalogPackages(theSource, c)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to catalog image: %+v", err)
|
||||
|
||||
Reference in New Issue
Block a user