* [wip] get assets based on gh api
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* put install.sh download_asset fn under test
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* put install.sh install_asset fn under test
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use zip for darwin installs
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix install.sh negative test cases
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* allow errors to propagate in install.sh
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove exit on error from install.sh tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add more docs around install.sh helpers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add integration tests for install.sh
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add install.sh testing to pipeline
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add install test cache to CI
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* make colors globally available
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* test download against github release
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* always test release-based install against latest release
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use better install.sh test names
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add language detection from pURLs
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add package type detection from pURLs
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add cargo and npm pURL support
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix npm tests and linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add php related metadata
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* enable decoding of php metadata for syftjson format
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add php metadata to json schema
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove strong distro type
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bump json schema to v3 (breaking distro shape)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* allow for v2 decoding of distro idLikes field in v3 json decoder
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix casing in simple linux release name
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use discovered name as pretty name in simple linux release
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Fix CPE generation when the generated CPE contains invalid characters
Currently syft seems to generate invalid CPEs which do not
conform with the official CPE spec. This is because the underlying
nvdtools library is not a completely spec compliant implementation
and has some interesting bugs/issues.
The following are the list of issues I have encountered with nvdtools:
1. It parses strings which are not CPEs incorrectly as valid CPEs. This
messes up our filter function which is supposed to filter out any
incorrect CPEs we generate. In order to fix this, I have introduced
a new regex in the NewCPE function which follows the upstream spec and
filters out any incorrect CPEs.
2. Introduce wfn.WFNize for any cpe attributes we infer from packages.
This ensures that we are escaping and quoting any special characters
before putting them into CPEs. Note that nvdtools has yet another bug
in the WFNize function, specifically the "addSlashesAt" part of the
function which stops the loop as soon as it encounters ":" a valid
character for a WFN attribute after quoting, but the way nvdtools
handles it causes it to truncate strings that container ":". As a result
strings like "prefix:1.2" which would have been quoted as "prefix\:1.2"
end up becoming "prefix" instead causing loss of information and
incorrect CPEs being generated. As a result in such cases, we remove out
strings containing ":" in any part entirely for now. This is similar
to the way we were handling CPE filtering in the past with http urls as
vendor strings
3. Add special handling for version which contain ":" due to epochs in
debian and rpm. In this case, we strip out the parts before ":" i.e.
the epoch and only output the actual function. This ensures we are not
discarding valid version strings due to pt #.2.
In the future we should look at moving to a more spec compliant cpe
parsing library to avoid such shenanigans.
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Remove WFNize for input strings
WFNize seems to not be part of the standard as per
https://pkg.go.dev/github.com/facebookincubator/nvdtools@v0.1.4/wfn#WFNize
and seems to have bugs/issues with encode/decode cycles, so I am
just removing it at this point and relying on the CPE regex to filter
out invalid CPEs for now.
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Quote the string on decode to ensure consistent CPE string generation
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add test cases for round-tripping the CPE and fix strip slashes
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add comprehensive tests for cpe parsing
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Use strings.Builder instead of byte buffer
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* add lpkg support to java cataloger
linter clean up
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix comment formatting
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add filename test for lpkg
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* commment on lpkg file extension tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix comment typo
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix import format
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* simpler test validation
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add direct_url.json fields to python metadata
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* rename DirectURLOrigin struct; add stub for file
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* add detection for direct_url.json
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* Add tests for direct-url information and add it to the output purl
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Update golden snapshot ids after adding new python package metadata field
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add test names for packageurl tests
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* set package ID in catalogers and improve hashing performance
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update setting ID + tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Add failing test for extra empty lines in manifest
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Handle extra empty lines in Java manifests
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* ignore target link files based on path
log when files are actually indexed
add test for sym link resolution
golang test nits
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* nil catalog should act like an empty catalog
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove dir path filtering in favor of file type filtering
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split out addPathToIndex into specialized functions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add test for nul catalog enumeration
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* conditionally discover MIME types for file based on file resolver index
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* change logging around cataloging
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add tests to cover possible infinite symlink loop for resolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
* Add failing test for missing versions
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Look through all named sections for version
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Consistent installation of yajsv
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* Adjust output text for test assertion
Signed-off-by: Dan Luhring <dan+github@luhrings.com>
* recover from panics in stdlib binary parsing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add CLI test to cover regression case
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add cataloging within universal binaries
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update json test fixtures
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add comments + correct 32 bit multi arch magic check
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove power-user document shape
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add power-user specific fields to syft-json format
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* port remaining spdx-json relationships to sbom model
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add coordinate set
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add SBOM file path helper
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use internal mimetype helper in go binary cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add new package-of relationship
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update json schema to v2
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* replace power-user presenter with syft-json format
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix tests and linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove "package-of" relationship (in favor of "contains")
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add tests for spdx22json format encoding enhancements
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update TODO and log entries
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* introduce sbom.Descriptor
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split source.Location and create source.Coordinates for minimal path addressing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* move coordinates into separate file
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Update syft/source/coordinates.go
Co-authored-by: Dan Luhring <luhring@users.noreply.github.com>
* migrate pkg.ID and pkg.Relationship to artifact package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* return relationships from tasks
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix more tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add artifact.Identifiable by Identity() method
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove catalog ID assignment
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust spdx helpers to use copy of packages
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* stabilize package ID relative to encode-decode format cycles
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename Identity() to ID()
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use zero value for nils in ID generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* enable source.Location to be identifiable
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* hoist up package relationship discovery to analysis stage
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update ownership-by-file-overlap relationship description
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add test reminders to put new relationships under test
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust PHP composer.lock parser function to return relationships
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update functions to pass Location
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* update unit tests to pass new locations
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* fix image source.FileResolvers to include layer info
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add non-empty location in golang binary cataloger testing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
* allow for cataloging a single file
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use all catalogers for file schemes
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Don't check the Built-By flag
Signed-off-by: Josh Bressers <josh@bress.net>
* Remove alpine pinning to resolve conflict with main
Signed-off-by: Josh Bressers <josh@bress.net>
* remove mod and cargo from image cataloger
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* update test error messages for clear failures
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* add query by MIME type to source.FileResolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* import stereoscope lib changes to find mime type
- add bin cataloger
- add bin parser
- add mime type go utils
- import new resolver
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* add go std library code to unpack bin
- keep them in their own (original) files
- add note for "this code was copied from"
- comment the lines the required changing
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
* add query by MIME type to source.FileResolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* pull in stereoscope MIME type feature
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Fix CPE set comparison mismatch
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Add failing test to assert CPE generation excludes URLs
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Add removeByCondition method to fieldCandidateSet
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Prevent invalid CPE values for products and vendors
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Introduce removeWhere and rename filter to condition
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Refactor fieldCandidateSet and condition logic
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Move CPE parsing filter to end of CPE generation
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* consider additional vendor candidates for ruby, python, rpm, npm, and java
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add java pom.xml processing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* allow for downstream transform control in cpe generation processing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* migrate CPE generation logic to dedicated package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split java manifest groupID extraction into two tiers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* extract groupID from pom parent project during CPE generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update java groupID processing tests to cover multi-tier approach
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix constructor names for cpe.fieldCandidate
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename helper function to startsWithTopLevelDomain
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add nil changes for java manifest sections
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update comment to reflect parsing maven files
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split out java description parsing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split out pom parent processing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* simplify vendorsFromGroupIDs and associated tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* simplify test type for vendorsFromGroupIDs
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* copy candidate varidations to new instances
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename CPE generation string util functions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add an explanation around fieldCandidate
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* simplify type for the cpe.fieldCandidateSet
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* make CPE filter function names more readable
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update groupIDsFromJavaManifest to use a guard clause
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* extract groupID extraction from artifactID fields into a separate function
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bump goreleaser version to combat failure
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust CPE specificity sorting to include field length and bias certain fields
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove * vendor values from CPE generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* re-enable generating CPEs for jenkins and jira plugins
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* improve CPE generation logic based on java artifactID and groupID
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add ruby-lang as target software candidate for gems in CPE generation logic
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename filterCpes to filterCPEs
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* refactor CPE filters and groupID processing (for linting)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use ruby-lang as vendor candidate not target software
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* address PR comments for CPE generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add initial spdx support
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* expose FileOwner and use in SPDX presenter
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add initial json support for SPDX
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add remaining package fields
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add spdx license list generation + tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* keep fileOwner unexported from pkg
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* restore cli test util
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add external refs to spdx tag-value format
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add golang support to CPE generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use tag-value format as default "spdx" format flavor
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add tests around spdx presenters + refactor presenter tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add bouncer exception for spdx tools-golang repo
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove spdx model questions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
