1199 Commits

Author SHA1 Message Date
Alex Goodman
c363b2b532
Add ability to pull images directly from a registry (#378)
* add registry image source

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use explicit source for fetching image + add scheme and registry tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust test variable name and add credential helper function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-13 13:30:57 +00:00
Alex Goodman
b5d4b2f7b2
simplify test assertions + rename file contents cataloger size limiter var
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:34:56 -04:00
Alex Goodman
d451a5ad30
update the json schema and tests with file contents section
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:33:20 -04:00
Alex Goodman
c1551a03c5
add file contents cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:30:57 -04:00
Alex Goodman
28c407572c
simplify test assertions for classification cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:11:43 -04:00
Alex Goodman
309cbf5264
add missing file classification test fixtures + gitignore adjustment
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:11:43 -04:00
Alex Goodman
46fffe5c6f
update json schema + presenter snapshots with file classification section
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:11:40 -04:00
Alex Goodman
46bfb68113
add file classifier + tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:08:50 -04:00
Alex Goodman
5743e32e02
add tests around MatchNamedCaptureGroups + rename
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:08:50 -04:00
Alex Goodman
0511972dfa
clarify default collection value + fix appending conffiles location
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-09 11:41:44 -04:00
Alex Goodman
c56690fc52
fix DpkgMetadata.Files test to ensure it is never nil
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-09 08:46:59 -04:00
Alex Goodman
ba3407a767
add dpkg conffile update to json schema + json test snapshots
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-09 08:37:28 -04:00
Alex Goodman
269832ce8d
add conffile listing to dpkg metadata + normalize digests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-08 14:21:23 -04:00
Alex Goodman
9ec09add67
Add secrets search capability (#367)
* add initial secrets cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update ETUI elements with new catalogers (file metadata, digests, and secrets)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update secrets cataloger to read full contents into memory for searching

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype of parallelization secret regex search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype with single aggregated regex

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype for secret search line-by-line

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype hybrid secrets search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add secrets cataloger with line strategy

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust verbiage towards SearchResults instead of Secrets + add tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update json schema with secrets cataloger results

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address PR comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update readme with secrets config options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* ensure file catalogers call AllLocations once

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-01 21:34:15 +00:00
Zach Hill
c332ba0867 Use cataloger Sorted() output instead of Enumerate() for stable result sorting in presenters. Fixes #331
Also adds artifact location to sort key for Sorted() to ensure
consistent sorts when artifacts of same name, version, and type are
found in different locations in the image. Location should be sufficient
since we assume only one package of a given name and version can exist
in one location, even if that location is an package-db like rpmdb.

Signed-off-by: Zach Hill <zach@anchore.com>
2021-04-01 01:40:41 -07:00
Alex Goodman
cd925e5664
account for repoDigests data in power-user json testing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-30 08:46:42 -04:00
Alex Goodman
db35186c7d
allow file metadata digests to be optional + add link destination
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-29 12:07:42 -04:00
Alex Goodman
8551168702
capitalize file type strings
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-29 12:07:41 -04:00
Robert Prince
ab83b7f840
Merge pull request #350 from octarinesec/repo-digests
Add repo digests if available from Stereoscope to the image metadata
2021-03-29 08:47:50 -07:00
Alex Goodman
9abdb174d5
ensure pkg.Catalog path index deduplicates real vs virtual paths
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-26 13:27:24 -04:00
Haim Helman
13a066ea18 Add repo digests if available from Stereoscope to the image metadata
Signed-off-by: Haim Helman <hhelman@vmware.com>
2021-03-25 05:57:23 -07:00
Alex Goodman
1b7c755536
add file-type mix as golden image
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-23 16:50:57 -04:00
Alex Goodman
3ef30f99be
pull in stereoscope cache fix + add test cache makefile target helpers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-23 14:31:59 -04:00
Alex Goodman
929b78efbf
remove prealloc nolint rule in catalogers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-23 12:56:00 -04:00
Alex Goodman
d420368ba9
add tests around new file metadata cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-23 11:00:59 -04:00
Alex Goodman
36e4af1953
adjust jsom schema version + adopt java pom properies test fixtures
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 11:27:01 -04:00
Alex Goodman
f180d1c537
improve config parsing + fix command deprecation warning
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:58:09 -04:00
Alex Goodman
6a960ec1f3
update json schema with optional poweruser data shape
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:56:16 -04:00
Alex Goodman
a6cba5d9db
remove multi* content fetching from resolvers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:53:51 -04:00
Alex Goodman
ff4ed40d50
migrate syft/presenter to internal/presenter
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:52:33 -04:00
Alex Goodman
4666ca8469
migrate syft/cataloger to syft/pkg/cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:46:51 -04:00
Alex Goodman
cb5e7d0e08
Merge pull request #345 from westonsteimel/rust-cargo-cataloger
add cataloger for rust crates from Cargo.lock files
2021-03-22 09:51:57 -04:00
Alex Goodman
48a1d4711a
update presenter tests with json schema 1.0.3 examples + rename rust metadata type
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 09:44:55 -04:00
Weston Steimel
35cc00b74a
capture extra metadata about rust crates from the Cargo.lock file
Signed-off-by: Weston Steimel <weston.steimel@gmail.com>
2021-03-22 09:13:17 -04:00
Alex Goodman
e9e9fe6d9c
prefer warnings over errors when parsing java manifests and pom properties
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-20 08:08:46 -04:00
Alex Goodman
2f8a568d4f
check for multiple delimiters when parsing pom properties
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-20 08:00:08 -04:00
Weston Steimel
ba81bfe529
add cataloger for rust crates from Cargo.lock files
Signed-off-by: Weston Steimel <weston.steimel@gmail.com>
2021-03-18 03:12:51 +00:00
Haim Helman
87e1a0f501 Added Photon OS to the list of distros
Signed-off-by: Haim Helman <hhelman@vmware.com>
2021-03-11 19:06:17 -08:00
Weston Steimel
b963be219b
allow ability to catalog packages from /var/lib/dpkg/status.d/
Some debian-based variants (such as Google's Distroless images)
don't write a single file to `/var/lib/dpkg/status`, but rather write
a file per package to `/var/lib/dpkg/status.d/`

related to #44

Signed-off-by: Weston Steimel <weston.steimel@gmail.com>
2021-02-27 01:20:44 +00:00
Alex Goodman
1aa8a10f6b
ignore ownership for shared copyright resources
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-02-26 12:30:24 -05:00
Alex Goodman
6d5ff0fd8e
Mark package relations by file ownership (#329)
* add marking package relations by file ownership

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* correct json schema version; ensure fileOwners dont return dups; pin test pkg versions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* extract package relationships into separate section

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* pull in client-go features for import of PackageRelationships

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move unit test for ownership by files relationship further down

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename relationship to "ownership-by-file-overlap"

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-02-25 13:47:13 -05:00
Alfredo Deza
05b9d35ee6 python: log a warning when invalid fields in PKG-INFO are found
Signed-off-by: Alfredo Deza <adeza@anchore.com>
2021-02-16 15:27:08 -05:00
Alfredo Deza
555376e250 python: test leniency with invalid fields in PKG-INFO
Signed-off-by: Alfredo Deza <adeza@anchore.com>
2021-02-16 14:20:07 -05:00
Dan Luhring
1416e3cb7a
Invert if statement to reduce nesting in archive parser
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-01-27 10:34:52 -05:00
Alex Goodman
bb1facbf81
ensure java parent pkg ref isnt nil when looking for parent matches
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-01-27 10:12:41 -05:00
Dan Luhring
4576c081b9
Improve output for test case
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-01-22 15:20:42 -05:00
Dan Luhring
d5779a9822
Clarify debug message for package.json omissions
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-01-22 15:20:24 -05:00
Dan Luhring
9ec3ad58c8
Update regression test expected value and pin deps
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-01-22 14:03:50 -05:00
Dan Luhring
176dfdd9c1
Don't create packages unless package.json has name and version
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-01-22 13:56:54 -05:00
Alex Goodman
6a49717314
add VirtualPath to source.Location
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-01-05 21:16:42 -05:00