Commit Graph
371 Commits
Author SHA1 Message Date
Alex Goodman 172ecc0d77 Preserve syft IDs on SBOM decode (#963) 2022-04-18 18:10:55 +00:00
Keith Zantow 248023baaf Update GitHub format package_url and correlator (#961) 2022-04-15 13:00:06 -04:00
Keith Zantow b7295b79de Ensure SPDXIDs are valid (#955) 2022-04-14 15:07:23 -04:00
Christopher Angelo Phillips b46d044d7e Update spdx22json to only take uppercase checksum algorithm (#946) 2022-04-11 14:56:04 -04:00
Christopher Angelo Phillips 782b2e3348 Add digest property to parent and nested java package metadata (#941) 2022-04-08 15:12:32 -04:00
Sambhav KothariandAlex Goodman 8bc5d84481 Ensure that all cyclonedx components have bom-refs (#914)
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-04-01 12:19:30 -04:00
Alex Goodman f24bbc1838 Deduplicate packages across multiple container image layers (#930) 2022-03-31 15:45:51 -04:00
Eric LarssenandAlex Goodman cb3e73e308 Add dart support (#919)
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-03-31 15:44:55 -04:00
Dan Luhring a7db43f5ec Fix panic on empty sbom (#917)
* Implement fmt.Stringer with format.ID

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Add failing test for formats processing empty SBOMs

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Account for nil SPDX document during Syft model conversion

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-03-24 10:11:51 -04:00
Alex Goodman 7f9edf346a Bump golangci-lint to 1.45.0 (#909) 2022-03-22 11:02:36 -04:00
Jonas Xavier 283db88dc4 Omit H1Digest when empty (#902)
* Omit HD1Field when empty

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update test-fixtures

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2022-03-21 11:59:10 -07:00
Keith Zantow 9240860f44 Correct ID handling during Syft JSON decoding (#900) 2022-03-18 17:03:26 -04:00
Christopher Angelo Phillips 4231f38fa2 add case to decode GolangBinMetadata for syftjson model (#901)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-03-18 15:15:10 -04:00
Keith Zantow 99c3339810 Fix CycloneDX license decoding panic (#898) 2022-03-18 09:44:51 -04:00
Keith Zantow f4734d28b3 Fix panic when CycloneDX BOM missing metadata.component (#895) 2022-03-17 10:22:35 -04:00
Jonas Xavier 6ef3e45ffc Use go 1.18 buildinfo to catalog binaries (#827)
* initial working version

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* added build settings to pkg metadata

wip - unit tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* handle mach-O FatFiles

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add support to mod replace

fixed golang catalger tests

trying GH Actions with go 1.18rc1

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* log error

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* use go-macholibre for extraction

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* cleaner tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add version to main module

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* check macho file with macholibre

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* run golangci in its own workflow

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci workflow

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix golangci wf yml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix golangci wf yml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci wf

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci wf

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* get arch from bin file headers

upgrade macholibre

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* test new stereoscope lazy reader interface

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove devel version from golang cataloger

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* switch github workflows to go1.18 stable

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add union reader interface in golang cataloger

update stereoscope

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* simpler golangci validation

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix makefile

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* get archs refactor

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* nolint for golang version

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix go bin tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* feedback changes

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* golangci nolint needs a \n before package

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* cleanup

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* move golangci-lint to its own jobs again

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix ci yaml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add support for xcoff files

add arch assets to test bin file types

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* clean up golangci-lint config

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* nolint for xcoff

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explain nolints

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove unused xcoff testdata assets

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* make go bin test-fixtures in docker

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix make clean with -f

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update json output schema

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update schema version in test fixture

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* feedback changes

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explain possible empty main module

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2022-03-16 17:07:02 -07:00
Keith Zantow ee0a1d172c panic parsing cyclonedx (#892) 2022-03-16 09:10:44 -04:00
Keith Zantow 6c8102bf28 Correct CycloneDX distro decoding (#745) 2022-03-11 09:27:18 -05:00
Keith Zantow 7789506dc6 Experimental GitHub export (#836) 2022-03-10 22:38:12 -05:00
Alex Goodman 2946813a74 RPM Epoch should be optional in the json schema (#880) 2022-03-09 14:51:43 -05:00
Christopher Angelo Phillips 003d28ad48 Add SchemaVersion to version command output (#877)
* make JsonSchemaVersion available programmatically via syft version command

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-03-09 13:12:52 -05:00
Sambhav Kothari 39737a2825 Update cyclonedx to v1.4 (#820) 2022-03-08 12:09:55 -05:00
Alex Goodman 07d3c9af52 Fix file creation for output options (#875) 2022-03-08 15:37:28 +00:00
Alex Goodman a86dd3704e Add platform selection (#866) 2022-03-04 22:41:38 +00:00
Alex Goodman 4af32c5bee Migrate format definitions to sbom package (#864) 2022-03-04 17:22:40 -05:00
Keith Zantow b2ab4671b9 Correct SPDX-JSON checksum algorithm (#863) 2022-03-03 17:13:13 -05:00
Keith Zantow edac8c7bf7 Update CycloneDX to use syft namespace and output multiple CPEs (#849) 2022-03-01 17:37:52 -05:00
Alex Goodman 99bb93d0fe Resolve symlinks when fetching file contents (#782) 2022-02-24 10:01:59 -05:00
Christopher Angelo Phillips 256e85bc12 510 - SBOM attestation stdout (#785)
add syft attest command to produce an attestation as application/vnd.in-toto+json to standard out using on disk PKI

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-02-22 21:45:12 -05:00
Keith Zantow 20c1d14f6e Add CycloneDX decoder (#811) 2022-02-18 11:19:02 -05:00
Alex Goodman 51c6eb30f5 bump stereoscope to include functional options (#823)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-02-14 20:40:51 -05:00
Alex Goodman 220f3a24fd deduplicate SPDX tag-value package IDs (#813)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-02-10 21:18:00 +00:00
Keith Zantow 76f8205936 Suport SPDX SBOM decoding (#738) 2022-02-09 14:11:20 -05:00
Jonas Xavier ca081ae5e0 use SYFT_LOG_FILE env var (#805)
* use SYFT_LOG_FILE

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* enable debug logs when SYFT_LOG_FILE is set

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* set log.file and add tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* test log file in temp directory

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add note on binding refactor

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove unused function

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2022-02-09 10:04:08 -08:00
Keith Zantow 1e338502ff Update SPDX license list (#801) 2022-02-07 15:24:08 +00:00
Alex Goodman 341288ba29 Normalize snapshot and release artifacts (#789)
* refactor signing steps in release/snapshot workflows

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* show signing logs on snapshot or release failure

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update install.sh + tests to account for new goreleaser changes

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update cli tests to account for new goreleaser build names

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix acceptance test to use new snapshot bin path

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add notarization

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address review comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-02-04 12:41:37 -05:00
Christopher Angelo Phillipsandhectorj2f 024a5a9f3f Add dependencies to cyclonedx (#768)
Add dependencies to cyclonedx

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: hectorj2f <hectorf@vmware.com>
2022-01-25 15:34:16 -05:00
Peter BaloghandAlex Goodman 161fa7be4a [CycloneDX] Add artifactID and groupID to the cycloneDX properties (support lower level struct as properties) (#758)
* [CycloneDX] Add artifactID and groupID to the cycloneDX properties

Signed-off-by: Peter Balogh <p.balogh.sa@gmail.com>

* update comment

Signed-off-by: Peter Balogh <p.balogh.sa@gmail.com>

* additional checks for value

Signed-off-by: Peter Balogh <p.balogh.sa@gmail.com>

* fill group filed with groupID in the case of Java

Signed-off-by: Peter Balogh <p.balogh.sa@gmail.com>

* fix linter warning

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-01-25 10:36:15 -05:00
Sambhav Kothari aebe843c6f Improve CycloneDX format output (#710)
* Improve CycloneDX format output

## Additions to CycloneDX output

* CPEs
* Authors
* Publishers
* External References (Website, Distribution, VCS)
* Description

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
2022-01-19 11:43:16 -05:00
Alex Goodman 829e500aa9 Add additional PHP metadata (#753)
* add php related metadata

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enable decoding of php metadata for syftjson format

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add php metadata to json schema

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-01-19 11:42:16 -05:00
Toure Dunnon 814f2bf8b9 Update Syft formats for SyftJson (#752)
* Update Syft formats for SyftJson

This change will introduce omitempty struct tag to PackageCustomData.
This struct tag will cause null and empty values to be dropped on serialization
for consumers downstream.

Signed-off-by: Toure Dunnon <toure.dunnon@anchore.com>

* Updated the golden files for syftjson to allow for proper
test coverage.

Signed-off-by: Toure Dunnon <toure.dunnon@anchore.com>
2022-01-18 17:18:34 -05:00
Dan Luhring c61204f56d Add support for "file" source type in syftjson unmarshaling (#750)
* Add tests for image and directory syftjson source

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Add failing test case for file source unmarshaling

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Fix file source unmarshaling

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Add test case for unknown source type

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-01-18 13:40:39 -05:00
Keith Zantow f59af255e3 Align SPDX export more with SPDX 2.2 specification (#743) 2022-01-13 15:27:06 -05:00
Alex Goodman 706f291679 Replace distro type (#742)
* remove strong distro type

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump json schema to v3 (breaking distro shape)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow for v2 decoding of distro idLikes field in v3 json decoder

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix casing in simple linux release name

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use discovered name as pretty name in simple linux release

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-01-12 12:13:42 -05:00
Keith Zantow 5e5312c72d Add support for multiple output files in different formats (#732) 2022-01-06 17:52:20 -05:00
Alex Goodman 38c4b17847 Add support for searching for jars within archives (#734)
* add support for searching jars within archives

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add package cataloger config options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address review comments + factor out safeCopy helper

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update config docs regarding package archive search options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* show that unindexed archive cataloging defaults to false

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove lies about -s

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address review comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update search archive note about java

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-01-06 21:40:51 +00:00
Christopher Angelo Phillips 01dc78ccc3 683 windows filepath (#735)
Support Windows Directory Resolver
Add function that converts windows to posix functionality
Add function that converts posix to windows
Add build tags to remove windows developer environment errors
redact carriage return specific windows issues

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-01-06 11:39:04 -05:00
Sambhav Kothari 2a7325a965 Fix CPE encode/decode when it contains special chars (#714)
* Fix CPE generation when the generated CPE contains invalid characters

Currently syft seems to generate invalid CPEs which do not
conform with the official CPE spec. This is because the underlying
nvdtools library is not a completely spec compliant implementation
and has some interesting bugs/issues.

The following are the list of issues I have encountered with nvdtools:

1. It parses strings which are not CPEs incorrectly as valid CPEs. This
messes up our filter function which is supposed to filter out any
incorrect CPEs we generate. In order to fix this, I have introduced
a new regex in the NewCPE function which follows the upstream spec and
filters out any incorrect CPEs.

2. Introduce wfn.WFNize for any cpe attributes we infer from packages.
This ensures that we are escaping and quoting any special characters
before putting them into CPEs. Note that nvdtools has yet another bug
in the WFNize function, specifically the "addSlashesAt" part of the
function which stops the loop as soon as it encounters ":" a valid
character for a WFN attribute after quoting, but the way nvdtools
handles it causes it to truncate strings that container ":". As a result
strings like "prefix:1.2" which would have been quoted as "prefix\:1.2"
end up becoming "prefix" instead causing loss of information and
incorrect CPEs being generated. As a result in such cases, we remove out
strings containing ":" in any part entirely for now. This is similar
to the way we were handling CPE filtering in the past with http urls as
vendor strings

3. Add special handling for version which contain ":" due to epochs in
debian and rpm. In this case, we strip out the parts before ":" i.e.
the epoch and only output the actual function. This ensures we are not
discarding valid version strings due to pt #.2.

In the future we should look at moving to a more spec compliant cpe
parsing library to avoid such shenanigans.

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>

* Remove WFNize for input strings

WFNize seems to not be part of the standard as per
https://pkg.go.dev/github.com/facebookincubator/nvdtools@v0.1.4/wfn#WFNize
and seems to have bugs/issues with encode/decode cycles, so I am
just removing it at this point and relying on the CPE regex to filter
out invalid CPEs for now.

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>

* Quote the string on decode to ensure consistent CPE string generation

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>

* Add test cases for round-tripping the CPE and fix strip slashes

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>

* Add comprehensive tests for cpe parsing

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>

* Use strings.Builder instead of byte buffer

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
2022-01-06 09:56:53 -05:00
Sambhav Kothari 4903b4d73f Bump syft JSON schema and add v2.0.2 (#716)
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
2021-12-22 13:55:34 -05:00
Sambhav KothariandChristopher Angelo Phillips cc20a8f341 Add tests for direct-url information and add it to the output purl (#708)
* add direct_url.json fields to python metadata

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* rename DirectURLOrigin struct; add stub for file

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* add detection for direct_url.json

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* Add tests for direct-url information and add it to the output purl

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>

* Update golden snapshot ids after adding new python package metadata field

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>

* Add test names for packageurl tests

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>

Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-12-20 15:54:25 -05:00