Commit Graph
534 Commits
Author SHA1 Message Date
Alex Goodman f6716092af fix cataloger selection to be more specific (#1582)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-17 15:35:54 +00:00
Weston Steimel 57a13ae355 fix: improve CPE and upstream generation logic for Alpine packages (#1567)
* fix: improved CPE-generation logic for alpine packages

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: improved alpine upstream name generation

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: improve CPE vendor for alpine

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: python vendor CPE gen

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: alpine cpe gen logic

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: apk CPE update for nodejs-current

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: CPE update for python pip

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix: CPE update for some ruby packages

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* fix linting

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

---------

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-13 17:23:13 +00:00
Keith Zantow 890fb3f0e8 fix: missing APK node vulnerabilities (#1565)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2023-02-10 15:52:13 -05:00
Weston Steimel e236054668 fix: python CPE generation for alpine (#1564)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-02-10 18:04:16 +00:00
Alex Goodman 988041ba6d Speed up cataloging by replacing globs searching with index lookups (#1510)
* replace raw globs with index equivelent operations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add cataloger test for alpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix import sorting for binary cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting for mock resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* separate portage cataloger parser impl from cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enhance cataloger pkgtest utils to account for resolver responses

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for alpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for apkdb cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dpkg cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for cpp cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dart cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dotnet cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for elixir cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for erlang cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for golang cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for haskell cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for java cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for javascript cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for php cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for portage cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for python cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for rpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for rust cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for sbom cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for swift cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow generic catloger to run all mimetype searches at once

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove stutter from php and javascript cataloger constructors

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add tests for generic.Search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add exceptions for java archive git ignore entries

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enhance basename and extension resolver methods to be variadic

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* dont allow * prefix on extension searches

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for ruby cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove unnecessary string casting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* incorporate surfacing of leaf link resolitions from stereoscope results

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] switch to stereoscope file metadata

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip + failing] revert to old globs but keep new resolvers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* index files, links, and dirs within the directory resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix several resolver bugs and inconsistencies

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move format testutils to internal package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update syft json to account for file type string normalization

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split up directory resolver from indexing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update docs to include details about searching

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] bump stereoscope to development version

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust symlinks fixture to be fixed to digest

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix all-locations resolver tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix test fixture reference

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename file.Type

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix PR comment to exclude extra *

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump to dev version of stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump to final version of stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move observing resolver to pkgtest

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-09 16:19:47 +00:00
witchcrazeandAlex Goodman 284814153d feat: add postgresql classifier (#1536)
* add postgresql classifier
Signed-off-by: witchcraze <witchcraze@gmail.com>

* revert package dedup test upgrade

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-08 14:31:17 -05:00
Avi Deitcher 38a090c218 fix: add support for licenses not found on list (#1540)
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2023-02-07 11:47:04 -05:00
Keith Zantow 9650473298 feat: Allow specific versions of formats to be specified (#1543) 2023-02-07 10:40:43 -05:00
Justin Chadwell 6ba595344a source: when base is set, responsePath should be absolute (#1542)
When base is set, it should appear identically to when we scan the root
filesystem - and as a result, the path should begin with the path
separator.

E.g. when scanning the root `./target/` with the same base,
`target/bin/busybox` should appear in the output as `/bin/busybox`, not
as previously as `bin/busybox`.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-06 12:06:04 -05:00
witchcraze 0853a50f4d feat: add traefik classifier (#1504)
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-02-02 13:59:26 -05:00
Keith Zantow 1530ef354f chore: update spdx/tools-golang to v0.5.0-rc1 (#1503) 2023-01-31 11:53:16 -05:00
Justin Chadwell b81c9805dc Allow scanning unpacked container filesystems (#1485)
* source: avoid second-step of symlink resolution in directory resolver

We can use the already existing file tree to peform symlink resolution
for FilesByPath, instead of traversing the symlinks again.

This moves all of the symlink logic into the indexing code, and then we
can rely on syft's resolution algorithm over the index in this part of
the codebase.

Signed-off-by: Justin Chadwell <me@jedevc.com>

* source: add base parameter to directory resolver

The new base parameter is an optional parameter for the directory
resolver that resolves all symlinks relative to this root. There are two
intended use cases:

- base = "/". The previous behavior, symlinks are resolved relative to
the root filesystem.
- base = path. Symlinks are resolved relative to the target filesystem,
allowing correct behavior when scanning unpacked container filesystems
on disk.

Signed-off-by: Justin Chadwell <me@jedevc.com>

* source: add tests for new base parameter

Signed-off-by: Justin Chadwell <me@jedevc.com>

---------

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-30 13:47:24 -05:00
Weston Steimel 4c0aef09b8 fix: add relevant CPEs to python and busybox classifiers (#1517)
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
2023-01-25 17:18:24 +00:00
Alex GoodmanandKeith Zantow 0ba57a5936 Fix AssertEncoderAgainstGoldenSnapshot calls to conditionally update (#1511)
* fix AssertEncoderAgainstGoldenSnapshot calls to conditionally update

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* chore: redact schema versions from syftjson comparison

Signed-off-by: Keith Zantow <kzantow@gmail.com>

* chore: does not need a multiline expression

Signed-off-by: Keith Zantow <kzantow@gmail.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
2023-01-24 21:41:57 +00:00
Luca Comellini e8be93a8eb Bump github.com/spdx/tools-golang to v0.4.0 (#1450)
Signed-off-by: Luca Comellini <luca.com@gmail.com>
2023-01-20 17:00:21 -05:00
Dan LuhringandAlex Goodman e58050bac0 Fix panic in apkdb parsing on empty "provides" values (#1494)
* Add failing test for strip version specifiers panic

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Fix test

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Prevent panic scenario in helper func

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* Fix lint issue

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>

* add tests for apk stripVersionSpecifier() and remove caller empty value check

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Dan Luhring <dluhring@chainguard.dev>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-20 14:49:44 +00:00
Alex Goodman 36a0945c95 push detailed log statements to trace-level (#1500)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-20 14:33:23 +00:00
mikcl 396441e921 npm: package-lock license decoding to accept string or array (#1482)
Signed-off-by: mikcl <mikesmikes400@gmail.com>
2023-01-20 09:28:51 -05:00
Alex Goodman 972e4cdaeb always set the package ID for java packages (#1493)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-20 09:18:00 -05:00
Nils Hanke 99f55f6a81 fix: skip filling in empty fields in APK metadata (#1484)
Signed-off-by: Nils Hanke <nils.hanke@outlook.de>

Signed-off-by: Nils Hanke <nils.hanke@outlook.de>
2023-01-20 14:03:30 +00:00
Alex GoodmanandChristopher Phillips 0f75f975c8 Relax error conditions for catalogers (#1492)
* binary cataloger should continue on errors

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* test: add redirect for cmd stderr stdout

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

* test: image update for test failure

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-01-19 19:28:42 -05:00
witchcraze 7427445fe9 feat: add memcached classifier (#1486)
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-01-19 11:22:11 -05:00
Keith ZantowandAlex Goodman 6cf668f749 fix: nil panic in graalvm cataloger (#1468)
* normalize error handling and recover from panics while parsing binaries
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-17 19:06:24 +00:00
Alex Goodman 2ec4371c95 add linter for type assertion checks (#1469)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-17 14:00:03 -05:00
Alex Goodman 05611c283d bootstrap within composite action (#1461)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-17 10:04:22 -05:00
Keith Zantow 934644232a chore: revert GolangBinMetadata name and make analogous GolangModMetadata (#1458) 2023-01-13 16:46:12 -05:00
Keith Zantow ac94bf530c fix: update graalvm cataloger to fix panic (#1454)
Fixes https://github.com/anchore/syft/issues/1453
2023-01-12 17:42:13 -05:00
Asi GreenholtsandAlex Goodman 260cb4c72d feat: Add the origin field to the output format of syftjson (#1327)
* moved the relevant fields to the Metadata field

Signed-off-by: Asaf Greenholts <asaf@cidersecurity.io>

* added metadata types

Signed-off-by: Asaf Greenholts <asaf@cidersecurity.io>

* Added hashes to metadata of packge-lock.json and Pipfile.lock

Signed-off-by: Asaf Greenholts <asaf@cidersecurity.io>

* move package metadata types to "pkg" package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* re-generate json schema to include new npm, python, and binary metadatas

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Asaf Greenholts <asaf@cidersecurity.io>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-12 15:03:05 -05:00
Keith Zantow 85bddaa43d chore: update schema (#1449) 2023-01-12 14:25:47 -05:00
Arnout EngelenandAlex Goodman a864dc9505 feat: prefer known CPE vendors over other candidates (#1294)
* feat: prefer known CPE vendors over other candidates

All ASF projects will be under the `apache` vendor in CPE, and
indeed this is already one of the candidates, but the logic
for selecting the 'most specific' CPE string would select for
example `apache_software_foundation` or `commons-text`.

This is not necessarily 'wrong' in the CPE candidate selection
logic: there is no way to reliably determine the right candidate.
I think it makes sense to use specific data around the vendor
candidate generation, somewhat similar to
'defaultCandidateAdditions'.

Unfortunately there are still a few CVE's for old (pre-5.x,
long unsupported) tomcat versions that are actually tagged with
`apache_software_foundation`, but I'm not sure those are worth
spending time on.

Signed-off-by: Arnout Engelen <arnout@bzzt.net>

* chore: swap out array of vendors for set data structure

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Arnout Engelen <arnout@bzzt.net>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-12 19:16:53 +00:00
Christopher Angelo PhillipsandAlex Goodman 44e8ae2577 fix: update attestation code to remove library dependencies and shellout for keyless flow (#1442)
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-12 17:22:05 +00:00
Chapman PenderyandAlex Goodman ac8f72fdd1 feat: add BeamVM Hex support (#1073)
* feat: initial commit providing mix support

Signed-off-by: cpendery <cpendery@vt.edu>

* feat: add rebar parser

Signed-off-by: cpendery <cpendery@vt.edu>

* fix: add beam/hex everywhere else required for Syft runtime

Signed-off-by: cpendery <cpendery@vt.edu>

* style: fix lints

Signed-off-by: cpendery <cpendery@vt.edu>

* ci: fix failing tests

Signed-off-by: cpendery <cpendery@vt.edu>

* docs: update with new supported languages

Signed-off-by: cpendery <cpendery@vt.edu>

* chore: update elixir/erlang catalogers to generic cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: cpendery <cpendery@vt.edu>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-12 12:10:46 -05:00
witchcraze e063471c66 feat: add apache httpd binary classifier (#1448)
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-01-12 10:50:01 -05:00
mikclandAlex Goodman 4bfb849310 Parallel package catalog processing (#1355)
* catalog: run cataloggers concurrently

Signed-off-by: mikcl <mikesmikes400@gmail.com>

* frontend: expose workers as a configurable option

Signed-off-by: mikcl <mikesmikes400@gmail.com>

* fixup! frontend: expose workers as a configurable option

Signed-off-by: mikcl <mikesmikes400@gmail.com>

* update logging statements

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* test: assert for debug logging

Signed-off-by: mikcl <mikesmikes400@gmail.com>

Signed-off-by: mikcl <mikesmikes400@gmail.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-11 15:18:02 -05:00
witchcraze d524bd5fc3 feat: Add php binary catalogers (#1444)
* add php classifier
Signed-off-by: witchcraze <witchcraze@gmail.com>

* make lint-fix
Signed-off-by: witchcraze <witchcraze@gmail.com>
2023-01-11 13:46:20 -05:00
Keith Zantow 725529f43f fix: duplicate file in tar archive causes read to fail (#1445) 2023-01-10 14:55:02 -05:00
William Blair e480443c8c Add support for GraalVM Native Image executables. (#1276)
Signed-off-by: William Blair <william.blair@oracle.com>
2023-01-06 18:31:22 -05:00
Benji Visser db386baf81 Add redis binary classifier (#1438)
Signed-off-by: Benji Visser <benji@093b.org>
2023-01-06 12:50:48 -05:00
Benji Visser bb6fc6525c Add alpine type to purl (#1431)
Signed-off-by: Benji Visser <benji@093b.org>
2023-01-04 17:35:46 -05:00
Benji Visser bc1edb9c8a adding purl types for binary classifiers (#1435)
Signed-off-by: Benji Visser <benji@093b.org>
2023-01-04 11:34:37 -05:00
Keith Zantow 64be0a1072 chore: refactor basic CPE functionality to its own package (#1436) 2023-01-04 11:26:28 -05:00
Justin Chadwell e3d6ffd30e fix: typo in os.Getwd error message (#1433)
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-03 14:56:20 +00:00
Justin Chadwell 8d36b21237 fix: additional excessive go binary warnings (#1432)
The original fix b125ea83ba didn't catch
all the excessive warnings, it seems like getArches can also be called
on binaries that aren't neccessarily go binaries, so the messages from
this should also be Trace instead of Warn.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-03 09:54:08 -05:00
Keith Zantow e1e489a284 fix: unicode output in cyclonedx-json format (#1420) 2022-12-23 08:37:47 -05:00
Keith Zantow b125ea83ba fix: excessive go binary warnings (#1424) 2022-12-23 08:36:49 -05:00
Christopher Angelo Phillips 3690f979b3 feat: update spdx format model to produce valid spdx json documents (#1418) 2022-12-21 15:56:03 -05:00
Alex Goodman 5dd726fc86 clean package names in python parsers (#1417)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-12-21 18:31:49 +00:00
Keith Zantow 7b08608adb feat: add h1digest when scanning go.mod (#1405)
Fixes https://github.com/anchore/syft/issues/1277
2022-12-20 02:18:35 +00:00
dja-fr 82f32c7301 feat: Add license parsing for java (#1385) 2022-12-19 20:10:15 -05:00
Keith Zantow 4ffbeeeea5 fix: cyclonedx component type for binaries (#1406) 2022-12-19 19:49:27 -05:00