Separate the logic for CPE and PURL generation.
PURL generation needs a single answer for groupID based on a priority of discovering the field.
CPE generation still uses multiple potential groupID to populate the candidate cpe.
Improve GroupID detection.
Currently syft does not use any hierarchy for GroupID detection and treats all sources as equal.
It treats fields from the manifest file with priority. This change adds a hierarchy to the fields and returns a single answer based on that hierarchy.
---------
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
Adding APK OriginPackage CPE candidates to the child package
results in false positives in grype because it can't associate
CPE-based findings to the corresponding OriginPackage APK fixes.
This reverts changing the `upstream` in the PURL for APK packages
as the logic in Grype that uses it expects it to be an APK package
name. This also allows refactoring to unexport and move the APK
CPE candidate generation logic closer to where CPE generation occurs
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
This fixes some instances where the improved APK CPE generation
logic caused regressions for older alpine package APK metadata.
It now generates multiple "upstream" candidates with both name
and package type which reduces the amount of duplicated code in
the apk cpe gen logic. This also improves the handling of stream
version packages, so now we can correctly identify packages such
as ruby3.2-rexml as the rexml ruby gem.
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
* feat: prefer known CPE vendors over other candidates
All ASF projects will be under the `apache` vendor in CPE, and
indeed this is already one of the candidates, but the logic
for selecting the 'most specific' CPE string would select for
example `apache_software_foundation` or `commons-text`.
This is not necessarily 'wrong' in the CPE candidate selection
logic: there is no way to reliably determine the right candidate.
I think it makes sense to use specific data around the vendor
candidate generation, somewhat similar to
'defaultCandidateAdditions'.
Unfortunately there are still a few CVE's for old (pre-5.x,
long unsupported) tomcat versions that are actually tagged with
`apache_software_foundation`, but I'm not sure those are worth
spending time on.
Signed-off-by: Arnout Engelen <arnout@bzzt.net>
* chore: swap out array of vendors for set data structure
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Arnout Engelen <arnout@bzzt.net>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
* golang module CPE with full path
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
* add note on longer Golang CPEs
Signed-off-by: Jonas Xavier <jonasx@anchore.com>
The Official CPE dictionary currently contains entries for springframework with three different vendors: springsource, vmware, and pivotal_software. This appears to be because ownership has changed over time.
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
* Fix CPE generation when the generated CPE contains invalid characters
Currently syft seems to generate invalid CPEs which do not
conform with the official CPE spec. This is because the underlying
nvdtools library is not a completely spec compliant implementation
and has some interesting bugs/issues.
The following are the list of issues I have encountered with nvdtools:
1. It parses strings which are not CPEs incorrectly as valid CPEs. This
messes up our filter function which is supposed to filter out any
incorrect CPEs we generate. In order to fix this, I have introduced
a new regex in the NewCPE function which follows the upstream spec and
filters out any incorrect CPEs.
2. Introduce wfn.WFNize for any cpe attributes we infer from packages.
This ensures that we are escaping and quoting any special characters
before putting them into CPEs. Note that nvdtools has yet another bug
in the WFNize function, specifically the "addSlashesAt" part of the
function which stops the loop as soon as it encounters ":" a valid
character for a WFN attribute after quoting, but the way nvdtools
handles it causes it to truncate strings that container ":". As a result
strings like "prefix:1.2" which would have been quoted as "prefix\:1.2"
end up becoming "prefix" instead causing loss of information and
incorrect CPEs being generated. As a result in such cases, we remove out
strings containing ":" in any part entirely for now. This is similar
to the way we were handling CPE filtering in the past with http urls as
vendor strings
3. Add special handling for version which contain ":" due to epochs in
debian and rpm. In this case, we strip out the parts before ":" i.e.
the epoch and only output the actual function. This ensures we are not
discarding valid version strings due to pt #.2.
In the future we should look at moving to a more spec compliant cpe
parsing library to avoid such shenanigans.
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Remove WFNize for input strings
WFNize seems to not be part of the standard as per
https://pkg.go.dev/github.com/facebookincubator/nvdtools@v0.1.4/wfn#WFNize
and seems to have bugs/issues with encode/decode cycles, so I am
just removing it at this point and relying on the CPE regex to filter
out invalid CPEs for now.
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Quote the string on decode to ensure consistent CPE string generation
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add test cases for round-tripping the CPE and fix strip slashes
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add comprehensive tests for cpe parsing
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Use strings.Builder instead of byte buffer
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* migrate pkg.ID and pkg.Relationship to artifact package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* return relationships from tasks
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix more tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add artifact.Identifiable by Identity() method
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove catalog ID assignment
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust spdx helpers to use copy of packages
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* stabilize package ID relative to encode-decode format cycles
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename Identity() to ID()
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use zero value for nils in ID generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* enable source.Location to be identifiable
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* hoist up package relationship discovery to analysis stage
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update ownership-by-file-overlap relationship description
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add test reminders to put new relationships under test
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust PHP composer.lock parser function to return relationships
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Fix CPE set comparison mismatch
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Add failing test to assert CPE generation excludes URLs
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Add removeByCondition method to fieldCandidateSet
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Prevent invalid CPE values for products and vendors
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Introduce removeWhere and rename filter to condition
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Refactor fieldCandidateSet and condition logic
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* Move CPE parsing filter to end of CPE generation
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
* consider additional vendor candidates for ruby, python, rpm, npm, and java
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add java pom.xml processing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* allow for downstream transform control in cpe generation processing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* migrate CPE generation logic to dedicated package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split java manifest groupID extraction into two tiers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* extract groupID from pom parent project during CPE generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update java groupID processing tests to cover multi-tier approach
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix constructor names for cpe.fieldCandidate
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename helper function to startsWithTopLevelDomain
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add nil changes for java manifest sections
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update comment to reflect parsing maven files
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split out java description parsing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split out pom parent processing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* simplify vendorsFromGroupIDs and associated tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* simplify test type for vendorsFromGroupIDs
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* copy candidate varidations to new instances
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename CPE generation string util functions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add an explanation around fieldCandidate
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* simplify type for the cpe.fieldCandidateSet
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* make CPE filter function names more readable
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update groupIDsFromJavaManifest to use a guard clause
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* extract groupID extraction from artifactID fields into a separate function
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bump goreleaser version to combat failure
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
witchcraze
Christopher Angelo PhillipsandKeith Zantow
anchore-actions-token-generator[bot]andwagoodman
Dan LuhringandAlex Goodman
Josh Bressers
Weston Steimel
Arnout EngelenandAlex Goodman
Chapman Pendery
Jonas Xavier
Kenny Moens
Sambhav Kothari