Commit Graph
26 Commits
Author SHA1 Message Date
Alex Goodman 5d156b8241 Add annotations for evidence on package locations (#1723)
* add location annotations + deb evidence annotations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename LocationData struct and Annotation helper function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add failing integration test for evidence coverage

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add evidence to aplm cataloger locations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* change location annotation helper to return a location copy

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add evidence to binary cataloger locations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* updated remaining catalogers with location annotations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix unit tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump json schema

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* partial addressing of review comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename location.WithAnnotation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-04-13 17:02:29 -04:00
Avi Deitcher 01230aa766 read relative etc/apk/repositories for alpine version when no OS provided (#1615)
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2023-03-02 13:04:56 -05:00
Weston Steimel e2ebc9769f fix: remove APK OriginPackage cpe candidates (#1637)
Adding APK OriginPackage CPE candidates to the child package
results in false positives in grype because it can't associate
CPE-based findings to the corresponding OriginPackage APK fixes.

This reverts changing the `upstream` in the PURL for APK packages
as the logic in Grype that uses it expects it to be an APK package
name.  This also allows refactoring to unexport and move the APK
CPE candidate generation logic closer to where CPE generation occurs

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 17:24:43 +00:00
Weston Steimel 8e1205f7ab fix: apk product/vendor generation for old metadata (#1635)
This fixes some instances where the improved APK CPE generation
logic caused regressions for older alpine package APK metadata.
It now generates multiple "upstream" candidates with both name
and package type which reduces the amount of duplicated code in
the apk cpe gen logic.  This also improves the handling of stream
version packages, so now we can correctly identify packages such
as ruby3.2-rexml as the rexml ruby gem.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-03-01 14:58:35 +00:00
Weston Steimel 3ee1af0dc6 fix: correct apk purls for other distros (#1620)
The apk purl spec allows for vendor-specific namespace.  I noticed
in the embedded SBOMs from wolfi that the purls are of the form
`pkg:apk/wolfi/curl@7.83.0-r0?arch=x86`, but the current logic in
syft actually prevents purl generation entirely if the distro isn't
alpine, so this corrects that behaviour.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-24 15:07:07 -05:00
Weston Steimel 0c5f03235e refactor: move apk upstream logic to apk metadata (#1619)
* refactor: move apk upstream logic to apk metadata

Export the logic for parsing upstream APK package names
so it can be accessed from apk metadata objects directly.

This also tightens the upstream regex pattern as several
edge cases were being missed.

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: ensure correct handling for apk packages beginning with digits

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: upstream generation for ruby

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

---------

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-24 15:59:19 +00:00
Weston Steimel 0c05855131 fix: determine upstream for apk version streams (#1610)
Determines better upstream package name for version-stream apk packages:

Examples:

- postgresql-13 -> postgresql
- postgresql15 -> postgresql
- go-1.19 -> go
- perl100.55 -> perl

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-23 17:32:34 +00:00
Weston Steimel 57a13ae355 fix: improve CPE and upstream generation logic for Alpine packages (#1567)
* fix: improved CPE-generation logic for alpine packages

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: improved alpine upstream name generation

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: improve CPE vendor for alpine

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: python vendor CPE gen

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: alpine cpe gen logic

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: apk CPE update for nodejs-current

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: CPE update for python pip

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: CPE update for some ruby packages

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix linting

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

---------

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-13 17:23:13 +00:00
Alex Goodman 988041ba6d Speed up cataloging by replacing globs searching with index lookups (#1510)
* replace raw globs with index equivelent operations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add cataloger test for alpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix import sorting for binary cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting for mock resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* separate portage cataloger parser impl from cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enhance cataloger pkgtest utils to account for resolver responses

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for alpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for apkdb cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dpkg cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for cpp cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dart cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dotnet cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for elixir cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for erlang cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for golang cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for haskell cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for java cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for javascript cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for php cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for portage cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for python cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for rpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for rust cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for sbom cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for swift cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow generic catloger to run all mimetype searches at once

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove stutter from php and javascript cataloger constructors

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add tests for generic.Search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add exceptions for java archive git ignore entries

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enhance basename and extension resolver methods to be variadic

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* dont allow * prefix on extension searches

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for ruby cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove unnecessary string casting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* incorporate surfacing of leaf link resolitions from stereoscope results

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] switch to stereoscope file metadata

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip + failing] revert to old globs but keep new resolvers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* index files, links, and dirs within the directory resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix several resolver bugs and inconsistencies

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move format testutils to internal package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update syft json to account for file type string normalization

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split up directory resolver from indexing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update docs to include details about searching

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] bump stereoscope to development version

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust symlinks fixture to be fixed to digest

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix all-locations resolver tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix test fixture reference

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename file.Type

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix PR comment to exclude extra *

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump to dev version of stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump to final version of stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move observing resolver to pkgtest

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-09 16:19:47 +00:00
Dan LuhringandAlex Goodman e58050bac0 Fix panic in apkdb parsing on empty "provides" values (#1494)
* Add failing test for strip version specifiers panic

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Fix test

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Prevent panic scenario in helper func

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Fix lint issue

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* add tests for apk stripVersionSpecifier() and remove caller empty value check

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-20 14:49:44 +00:00
Nils Hanke 99f55f6a81 fix: skip filling in empty fields in APK metadata (#1484)
Signed-off-by: Nils Hanke <nils.hanke@outlook.de>

Signed-off-by: Nils Hanke <nils.hanke@outlook.de>
2023-01-20 14:03:30 +00:00
Benji Visser bb6fc6525c Add alpine type to purl (#1431)
Signed-off-by: Benji Visser <benji@093b.org>
2023-01-04 17:35:46 -05:00
Dan Luhring f6996f7b9a Fix parsing of apk databases with large entries (#1365)
Closes https://github.com/anchore/syft/issues/1354
2022-11-29 10:16:36 -05:00
949cff158d Add support for dependency relationships for alpine (apk) (#1063)
* Fix type of pull deps and add support for provides

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* [wip] apk dependency lookup

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update whitespace for linter

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

* adjust test conditions

Signed-off-by: Timothy Gerla <tim@gerla.net>

* fix TODOs and improve Provides parser

* run simports after main merge

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

* add tests to cover apk relationship parsing cases

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* generate JSON schema for breaking changes to apk metadata

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update tests to account for additional dependencies

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] fix relationship encoding for cyclonedx

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify package relationships that can be expressed

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: Timothy Gerla <tim@gerla.net>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: Timothy Gerla <tim@gerla.net>
2022-11-09 15:43:37 +00:00
Alex Goodman 52cb7269bf port deb/dpkg cataloger to new generic cataloger pattern (#1288)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-25 15:47:32 +00:00
Alex Goodman bd5adbc9b3 update cataloger tests to use pkgtest utils (#1287)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-25 11:20:02 -04:00
Alex Goodman f36c0ca971 port apk cataloger to new generic cataloger pattern (#1283)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-24 13:51:09 -04:00
Alex Goodman 48f0a4680f fix apkdb checksum representation (#1247)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-05 16:29:05 -04:00
Keith Zantow b20310eaf8 Add gosimports (#1205) 2022-09-14 13:38:18 -04:00
anchore-actions-token-generator[bot]andWeston Steimel b0fc955e0c Update syft bootstrap tools to latest versions. (#1171)
* Update syft bootstrap tools to latest versions.

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
Co-authored-by: Weston Steimel <weston.steimel@anchore.com>
2022-08-23 20:36:59 +01:00
Alex Goodman a27907659d Performance improvements around package ID (#698)
* set package ID in catalogers and improve hashing performance

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update setting ID + tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-12-16 08:55:53 -05:00
Alex Goodman ef627d82ef Introduce relationships as first-class objects (#607)
* migrate pkg.ID and pkg.Relationship to artifact package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* return relationships from tasks

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix more tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add artifact.Identifiable by Identity() method

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove catalog ID assignment

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust spdx helpers to use copy of packages

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* stabilize package ID relative to encode-decode format cycles

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename Identity() to ID()

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use zero value for nils in ID generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enable source.Location to be identifiable

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* hoist up package relationship discovery to analysis stage

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update ownership-by-file-overlap relationship description

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add test reminders to put new relationships under test

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust PHP composer.lock parser function to return relationships

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-16 14:14:13 -05:00
Alex Goodman a000a2926b update linter + fix whitespace (#536)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-06 13:46:55 +00:00
Alex Goodman ba1cd8d753 specify optional file.Digests instances to abbreviate json output (#531)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 15:35:28 -04:00
Alex Goodman 136e439dc2 use the standard file.Digest for apk checksums
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-19 16:42:17 -04:00
Alex Goodman 4666ca8469 migrate syft/cataloger to syft/pkg/cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:46:51 -04:00