Commit Graph
749 Commits
Author SHA1 Message Date
Christopher Angelo Phillips 7d13fbf0d4 540 add additional go package metadata (#546)
* update with distinct golang metadata types

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-13 11:00:28 -04:00
Josh Bressers f2840029d1 Allow Syft to build with podman (#537)
* Don't check the Built-By flag

Signed-off-by: Josh Bressers <josh@bress.net>

* Remove alpine pinning to resolve conflict with main

Signed-off-by: Josh Bressers <josh@bress.net>
2021-10-11 12:39:34 -04:00
Christopher Angelo Phillips b25f5b664b (464) - remove mod and cargo from image cataloger (#539)
* remove mod and cargo from image cataloger

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update test error messages for clear failures

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-07 15:18:20 -04:00
Christopher Angelo PhillipsandAlex Goodman 3462e18af3 478 identify go binaries and extract mod information (#534)
* add query by MIME type to source.FileResolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* import stereoscope lib changes to find mime type

- add bin cataloger
- add bin parser
- add mime type go utils
- import new resolver

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* add go std library code to unpack bin

- keep them in their own (original) files
- add note for "this code was copied from"
- comment the lines the required changing

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-07 12:16:38 -04:00
Alex Goodman a000a2926b update linter + fix whitespace (#536)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-06 13:46:55 +00:00
Alex Goodman 9189ed68df Add query by MIME type to source.FileResolver (#529)
* add query by MIME type to source.FileResolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* pull in stereoscope MIME type feature

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 20:50:53 -04:00
Alex Goodman ba1cd8d753 specify optional file.Digests instances to abbreviate json output (#531)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-05 15:35:28 -04:00
Christopher Angelo Phillips 5e4b668e5a 497 stable sorted CPE array (JSON and SPDX) (#522)
* add small sorting change to our specificity

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-01 15:31:17 -04:00
Alex GoodmanandChristopher Angelo Phillips 0395c47445 Add vendor + product known good CPE field values (#517)
* add better infrastructure around CPE candidate additions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add test cases for CPE candidate additions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* small ergonomic updates

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* clean java OSGI directives from group ID

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* unquote CPE candidate field values (double quotes only)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add remaining CPE vendor & product additions (+ tests)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-09-27 18:31:59 -04:00
Dan Luhring dd23d49986 Prevent invalid CPE field values (#514)
* Fix CPE set comparison mismatch

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add failing test to assert CPE generation excludes URLs

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add removeByCondition method to fieldCandidateSet

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Prevent invalid CPE values for products and vendors

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Introduce removeWhere and rename filter to condition

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Refactor fieldCandidateSet and condition logic

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Move CPE parsing filter to end of CPE generation

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-09-24 09:23:58 -04:00
Alex Goodman 6d4d083acc Filter out CPE product candidates that are asterisks (#513)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-22 20:55:03 -04:00
Dan Luhring 316d4341c8 Use Anchore fork of packageurl lib without replace directive (#512)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-09-22 15:04:09 -04:00
Alex Goodman abbba3fc19 Modify CPE vendor candidate generation approach (#484)
* consider additional vendor candidates for ruby, python, rpm, npm, and java

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add java pom.xml processing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow for downstream transform control in cpe generation processing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate CPE generation logic to dedicated package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split java manifest groupID extraction into two tiers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* extract groupID from pom parent project during CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update java groupID processing tests to cover multi-tier approach

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix constructor names for cpe.fieldCandidate

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename helper function to startsWithTopLevelDomain

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add nil changes for java manifest sections

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update comment to reflect parsing maven files

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split out java description parsing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split out pom parent processing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify vendorsFromGroupIDs and associated tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify test type for vendorsFromGroupIDs

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* copy candidate varidations to new instances

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename CPE generation string util functions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add an explanation around fieldCandidate

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* simplify type for the cpe.fieldCandidateSet

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* make CPE filter function names more readable

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update groupIDsFromJavaManifest to use a guard clause

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* extract groupID extraction from artifactID fields into a separate function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump goreleaser version to combat failure

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-09-03 14:21:25 -04:00
Keith Zantow ff828fbac2 Skip package-lock.json and yarn.lock in node_modules #431 (#485)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-08-20 13:50:28 -04:00
cba5b5723b Added parser for Pipfile.lock to cataloger (#473)
* Added parser for Pipfile.lock to cataloger

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* make lint-fix

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* Update syft/pkg/cataloger/python/parse_pipfile_lock.go

Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* fix _version

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* swap method for trimming "==" prefix from pipfile pkg versions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-19 10:32:10 -04:00
Alex Goodman 98d4749f86 Enhance CPE generation (#472)
* adjust CPE specificity sorting to include field length and bias certain fields

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove * vendor values from CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* re-enable generating CPEs for jenkins and jira plugins

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* improve CPE generation logic based on java artifactID and groupID

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add ruby-lang as target software candidate for gems in CPE generation logic

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename filterCpes to filterCPEs

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* refactor CPE filters and groupID processing (for linting)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use ruby-lang as vendor candidate not target software

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address PR comments for CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-08-09 19:52:19 +00:00
Alex Goodman 706322f826 Add SPDX support (#445)
* add initial spdx support

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* expose FileOwner and use in SPDX presenter

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add initial json support for SPDX

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add remaining package fields

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add spdx license list generation + tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* keep fileOwner unexported from pkg

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* restore cli test util

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add external refs to spdx tag-value format

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add golang support to CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use tag-value format as default "spdx" format flavor

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add tests around spdx presenters + refactor presenter tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add bouncer exception for spdx tools-golang repo

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove spdx model questions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-25 16:30:41 -04:00
Dan Luhring 50928ebd05 Add comments with examples for yarn.lock regexps (#439)
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-06-17 09:24:05 -04:00
Dan Luhring 67b7d63875 Fix yarn.lock parsing (#437) 2021-06-15 09:57:54 -04:00
Alex Goodman 8f85c8affc update springboot fixture to create jar with a prepended shell script
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-04 12:11:49 -04:00
Alex Goodman 5ea1d78464 rename helper fn to toELVersion
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-04 09:38:15 -04:00
Alex Goodman afbd8f8ea0 remove variadic functionality from intRef() test helper
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-02 17:13:31 -04:00
Alex Goodman 2f81a2548c allow for RPM package epoch to be optionally provided in the version string
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-02 15:28:12 -04:00
Alex Goodman a6c798f438 close all ReadClosers explicitly retrieved from resolvers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-05-12 17:21:30 -04:00
Alex Goodman 18af21d2a5 add jenkins filter for known bad CPE field combinations
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-22 15:51:59 -04:00
Alex Goodman 170681943c remove type assertion check in packageIdentitiesMatch fn
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-22 14:00:31 -04:00
Alex Goodman 46043510ae update parent pom persistence with regard to shaded jars
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-22 14:00:31 -04:00
Alex Goodman d9de63c837 Enhance CPE generation for java GroupId and filtering (#402)
* enhance cpe generation for group id and filtering

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename group id const + add doc comment for HasAnyOfPrefixes

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-22 12:22:56 +00:00
Alex Goodman 0c29090b42 Add hyphen replacement logic for CPE generation (#397)
* add hyphen replacement logic for CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate "python-" vendor prefix to product candidate processing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump linter timeout for CI

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update cpe candidate product tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-21 12:59:48 +00:00
Dan Luhring 060e60b6dd Add more tests to CPE generation
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-04-20 20:42:40 -04:00
Dan Luhring 33e6be0b74 Identify Jenkins plugin upstream of CPE generation
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-04-20 19:36:34 -04:00
Dan Luhring fa7fd718cb Refactor Java archive parsing logic
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-04-20 19:34:59 -04:00
Dan Luhring 65e4e17590 Pin gradle builder container image
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-04-20 19:34:59 -04:00
Dan Luhring 091fd1f0b0 Improve CPE generation for Jenkins/Jira plugins
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-04-20 19:34:53 -04:00
Alex Goodman b301b56db1 add nomatch_inclusion engine mechanism in cpe generation logic (#394)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-20 19:57:04 +00:00
Alex Goodman 676bdf9816 refactor pom properties to modify parent pkg less often (#392)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-20 16:19:28 +00:00
Alex Goodman 136e439dc2 use the standard file.Digest for apk checksums
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-19 16:42:17 -04:00
Alex Goodman 0c7706f254 add extra RPM file record fields (user, group, flags)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-16 10:25:46 -04:00
Alex Goodman b5d4b2f7b2 simplify test assertions + rename file contents cataloger size limiter var
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:34:56 -04:00
Alex Goodman d451a5ad30 update the json schema and tests with file contents section
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:33:20 -04:00
Alex Goodman 5743e32e02 add tests around MatchNamedCaptureGroups + rename
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:08:50 -04:00
Alex Goodman 0511972dfa clarify default collection value + fix appending conffiles location
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-09 11:41:44 -04:00
Alex Goodman c56690fc52 fix DpkgMetadata.Files test to ensure it is never nil
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-09 08:46:59 -04:00
Alex Goodman ba3407a767 add dpkg conffile update to json schema + json test snapshots
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-09 08:37:28 -04:00
Alex Goodman 269832ce8d add conffile listing to dpkg metadata + normalize digests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-08 14:21:23 -04:00
Alex Goodman 9ec09add67 Add secrets search capability (#367)
* add initial secrets cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update ETUI elements with new catalogers (file metadata, digests, and secrets)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update secrets cataloger to read full contents into memory for searching

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype of parallelization secret regex search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype with single aggregated regex

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype for secret search line-by-line

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype hybrid secrets search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add secrets cataloger with line strategy

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust verbiage towards SearchResults instead of Secrets + add tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update json schema with secrets cataloger results

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address PR comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update readme with secrets config options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* ensure file catalogers call AllLocations once

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-01 21:34:15 +00:00
Alex Goodman 929b78efbf remove prealloc nolint rule in catalogers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-23 12:56:00 -04:00
Alex Goodman 36e4af1953 adjust jsom schema version + adopt java pom properies test fixtures
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 11:27:01 -04:00
Alex Goodman 4666ca8469 migrate syft/cataloger to syft/pkg/cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:46:51 -04:00