219 Commits

Author SHA1 Message Date
Alex Goodman
b301b56db1
add nomatch_inclusion engine mechanism in cpe generation logic (#394)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-20 19:57:04 +00:00
Alex Goodman
676bdf9816
refactor pom properties to modify parent pkg less often (#392)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-20 16:19:28 +00:00
Alex Goodman
136e439dc2
use the standard file.Digest for apk checksums
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-19 16:42:17 -04:00
Alex Goodman
0c7706f254
add extra RPM file record fields (user, group, flags)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-16 10:25:46 -04:00
Alex Goodman
b5d4b2f7b2
simplify test assertions + rename file contents cataloger size limiter var
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:34:56 -04:00
Alex Goodman
d451a5ad30
update the json schema and tests with file contents section
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:33:20 -04:00
Alex Goodman
5743e32e02
add tests around MatchNamedCaptureGroups + rename
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:08:50 -04:00
Alex Goodman
0511972dfa
clarify default collection value + fix appending conffiles location
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-09 11:41:44 -04:00
Alex Goodman
c56690fc52
fix DpkgMetadata.Files test to ensure it is never nil
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-09 08:46:59 -04:00
Alex Goodman
ba3407a767
add dpkg conffile update to json schema + json test snapshots
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-09 08:37:28 -04:00
Alex Goodman
269832ce8d
add conffile listing to dpkg metadata + normalize digests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-08 14:21:23 -04:00
Alex Goodman
9ec09add67
Add secrets search capability (#367)
* add initial secrets cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update ETUI elements with new catalogers (file metadata, digests, and secrets)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update secrets cataloger to read full contents into memory for searching

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype of parallelization secret regex search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype with single aggregated regex

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype for secret search line-by-line

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype hybrid secrets search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add secrets cataloger with line strategy

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust verbiage towards SearchResults instead of Secrets + add tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update json schema with secrets cataloger results

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address PR comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update readme with secrets config options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* ensure file catalogers call AllLocations once

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-01 21:34:15 +00:00
Zach Hill
c332ba0867 Use cataloger Sorted() output instead of Enumerate() for stable result sorting in presenters. Fixes #331
Also adds artifact location to sort key for Sorted() to ensure
consistent sorts when artifacts of same name, version, and type are
found in different locations in the image. Location should be sufficient
since we assume only one package of a given name and version can exist
in one location, even if that location is an package-db like rpmdb.

Signed-off-by: Zach Hill <zach@anchore.com>
2021-04-01 01:40:41 -07:00
Alex Goodman
9abdb174d5
ensure pkg.Catalog path index deduplicates real vs virtual paths
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-26 13:27:24 -04:00
Alex Goodman
929b78efbf
remove prealloc nolint rule in catalogers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-23 12:56:00 -04:00
Alex Goodman
36e4af1953
adjust jsom schema version + adopt java pom properies test fixtures
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 11:27:01 -04:00
Alex Goodman
f180d1c537
improve config parsing + fix command deprecation warning
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:58:09 -04:00
Alex Goodman
4666ca8469
migrate syft/cataloger to syft/pkg/cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:46:51 -04:00
Alex Goodman
48a1d4711a
update presenter tests with json schema 1.0.3 examples + rename rust metadata type
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 09:44:55 -04:00
Weston Steimel
35cc00b74a
capture extra metadata about rust crates from the Cargo.lock file
Signed-off-by: Weston Steimel <weston.steimel@gmail.com>
2021-03-22 09:13:17 -04:00
Weston Steimel
ba81bfe529
add cataloger for rust crates from Cargo.lock files
Signed-off-by: Weston Steimel <weston.steimel@gmail.com>
2021-03-18 03:12:51 +00:00
Weston Steimel
b963be219b
allow ability to catalog packages from /var/lib/dpkg/status.d/
Some debian-based variants (such as Google's Distroless images)
don't write a single file to `/var/lib/dpkg/status`, but rather write
a file per package to `/var/lib/dpkg/status.d/`

related to #44

Signed-off-by: Weston Steimel <weston.steimel@gmail.com>
2021-02-27 01:20:44 +00:00
Alex Goodman
1aa8a10f6b
ignore ownership for shared copyright resources
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-02-26 12:30:24 -05:00
Alex Goodman
6d5ff0fd8e
Mark package relations by file ownership (#329)
* add marking package relations by file ownership

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* correct json schema version; ensure fileOwners dont return dups; pin test pkg versions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* extract package relationships into separate section

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* pull in client-go features for import of PackageRelationships

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move unit test for ownership by files relationship further down

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename relationship to "ownership-by-file-overlap"

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-02-25 13:47:13 -05:00
Zach Hill
7962002f81
Split dpk source into name and version (#297)
* Split dpk source into name and version

Signed-off-by: Zach Hill <zach@anchore.com>

* update dpkg status source name parsing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Dan Luhring <luhring@users.noreply.github.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2020-12-18 14:08:19 -05:00
Alex Goodman
3aaa0e5566
move package purl and cpes (identities) to pkg.Package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-19 11:25:51 -05:00
Alex Goodman
c892c3609e
improve doc comments
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:37:13 -05:00
Alex Goodman
6f7a4fd3e4
move source metadata upstream and fix tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:37:12 -05:00
Alex Goodman
aa0d444fd4
fix tests to use location instead of file.Reference
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:37:12 -05:00
Alex Goodman
9668341a14
rename scope to source
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:37:11 -05:00
Alex Goodman
495fb0a45f
add sbom document import lib helper function
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:36:51 -05:00
Alex Goodman
2a329002b8
enhance dpkg support by parsing md5sum and copyright file sources
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-11 11:06:34 -05:00
Alex Goodman
8095cd9980
add rpmdb file info to cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-06 07:21:23 -05:00
Alex Goodman
a52750bdd3
enhance alpine file discovery (#248)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-04 09:56:52 -05:00
Alex Goodman
03dbfb8dfb
improve java name and version extraction as well as parent pkg pairing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-30 08:12:25 -04:00
Alex Goodman
a5cba13ddf
enable more flexible java manifest structure (closer to the spec)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-29 16:26:04 -04:00
Alex Goodman
1230650771
allow for java manifest data to be optional
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-29 11:28:27 -04:00
Alex Goodman
2675891110
remove parent java package from json && add java manifest section parsing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-28 17:19:06 -04:00
Alex Goodman
62f6146c37
do not include optional fields
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-28 15:58:54 -04:00
Alex Goodman
8bcba7d830
update json schema and test for java virtualPath
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-28 15:25:30 -04:00
Alex Goodman
76c8f24090
add java virtual path
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-28 13:53:44 -04:00
Alex Goodman
6eb36bc3c8
allow for optional fields in npm metadata
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-23 15:00:55 -04:00
Toure Dunnon
94ffc2caa8 Merge branch 'main' into javascript_parser_fix_author
Signed-off-by: Toure Dunnon <toure.dunnon@anchore.com>
2020-10-23 12:05:30 -04:00
Alex Goodman
7d55bca0a0
allow for python metadata fields to be optional
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-23 10:29:52 -04:00
Toure Dunnon
15379d1075 Fix for errors+failures parsing package.json
closes: #230

Signed-off-by: Toure Dunnon <toure.dunnon@anchore.com>
2020-10-22 13:58:18 -04:00
Alex Goodman
2e5ff4a995
add top_level.txt processing to python package cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-22 12:39:03 -04:00
Alex Goodman
1414d1fbc3
add test coverage for python pacakge cataloger and update catalog interface
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-22 09:51:29 -04:00
Alex Goodman
7fc926d40d
add python author and emil fields + add metadata type
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-22 09:51:29 -04:00
Alex Goodman
0ce8701e73
split python package catalogers by image vs directory
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-21 11:48:26 -04:00
Alex Goodman
931c796158
add dynamic package.json parsing of author field
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-10-20 15:23:04 -04:00