Commit Graph
34 Commits
Author SHA1 Message Date
Idan Frimark 087a6356b9 chore: return both failures when failed to retrieve an image with a scheme (#1801)
Signed-off-by: Idan Frimark <idanf@cisco.com>
2023-05-23 10:32:12 -04:00
Christopher Angelo Phillips dfcc07e512 feat: Add config option to allow user to select the default image source location
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-03-31 10:04:10 -04:00
Christopher Angelo Phillips 61362c04fa fix: move defer after error to protect panic case (#1670)
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2023-03-15 15:29:10 -04:00
razzle 1d9ef34ec7 defer closing file (#1668)
Signed-off-by: razzle <harry@razzle.cloud>
2023-03-15 14:50:42 -04:00
Alex Goodman 988041ba6d Speed up cataloging by replacing globs searching with index lookups (#1510)
* replace raw globs with index equivelent operations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add cataloger test for alpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix import sorting for binary cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting for mock resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* separate portage cataloger parser impl from cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enhance cataloger pkgtest utils to account for resolver responses

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for alpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for apkdb cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dpkg cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for cpp cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dart cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dotnet cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for elixir cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for erlang cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for golang cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for haskell cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for java cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for javascript cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for php cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for portage cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for python cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for rpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for rust cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for sbom cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for swift cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow generic catloger to run all mimetype searches at once

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove stutter from php and javascript cataloger constructors

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add tests for generic.Search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add exceptions for java archive git ignore entries

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enhance basename and extension resolver methods to be variadic

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* dont allow * prefix on extension searches

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for ruby cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove unnecessary string casting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* incorporate surfacing of leaf link resolitions from stereoscope results

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] switch to stereoscope file metadata

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip + failing] revert to old globs but keep new resolvers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* index files, links, and dirs within the directory resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix several resolver bugs and inconsistencies

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move format testutils to internal package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update syft json to account for file type string normalization

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split up directory resolver from indexing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update docs to include details about searching

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] bump stereoscope to development version

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust symlinks fixture to be fixed to digest

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix all-locations resolver tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix test fixture reference

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename file.Type

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix PR comment to exclude extra *

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump to dev version of stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump to final version of stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move observing resolver to pkgtest

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-09 16:19:47 +00:00
Justin Chadwell b81c9805dc Allow scanning unpacked container filesystems (#1485)
* source: avoid second-step of symlink resolution in directory resolver

We can use the already existing file tree to peform symlink resolution
for FilesByPath, instead of traversing the symlinks again.

This moves all of the symlink logic into the indexing code, and then we
can rely on syft's resolution algorithm over the index in this part of
the codebase.

Signed-off-by: Justin Chadwell <me@jedevc.com>

* source: add base parameter to directory resolver

The new base parameter is an optional parameter for the directory
resolver that resolves all symlinks relative to this root. There are two
intended use cases:

- base = "/". The previous behavior, symlinks are resolved relative to
the root filesystem.
- base = path. Symlinks are resolved relative to the target filesystem,
allowing correct behavior when scanning unpacked container filesystems
on disk.

Signed-off-by: Justin Chadwell <me@jedevc.com>

* source: add tests for new base parameter

Signed-off-by: Justin Chadwell <me@jedevc.com>

---------

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-30 13:47:24 -05:00
Keith Zantow 725529f43f fix: duplicate file in tar archive causes read to fail (#1445) 2023-01-10 14:55:02 -05:00
Justin Chadwell 10f43d75e0 feat: Add --name option to override name in output (#1269) 2022-11-10 14:03:23 -05:00
Christopher Angelo Phillips 89575199b8 feat: add RelationshipsBySourceOwnership to syft json output (#1248) 2022-10-11 15:11:03 -04:00
Christopher Angelo Phillips 7d2fe9d95e feat: add identifiable field to source object (#1243) 2022-10-05 14:01:40 -04:00
Keith Zantow 16c62a1378 fix: support exclude patterns on Windows (#1228) 2022-09-26 10:59:19 -04:00
Keith Zantow b20310eaf8 Add gosimports (#1205) 2022-09-14 13:38:18 -04:00
Christopher Angelo PhillipsandAlex Goodman 615f933d98 Bug fix for 1095 - syft conversion option error (#1177)
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
2022-08-25 17:36:15 -04:00
Alex Goodman a86dd3704e Add platform selection (#866) 2022-03-04 22:41:38 +00:00
Christopher Angelo PhillipsandAlex Goodman afc0c1acd9 855 attest registry source only (#856)
Add source.NewFromRegistry function so that the syft attest command can always explicitly ask for an OCIRegistry provider rather than rely on local daemon detection for image sources.

Attestation can not be used where local images loaded in a daemon are the source. Digest values for the layer identification step in attestation can sometimes vary across workstations.

This fix makes it so that attest is generating an SBOM for, and attesting to, a source that exists in an OCI registry. It should never load a source from a local user docker/podman daemon.

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2022-03-01 23:16:42 -05:00
Alex Goodman 99bb93d0fe Resolve symlinks when fetching file contents (#782) 2022-02-24 10:01:59 -05:00
Christopher Angelo Phillips 256e85bc12 510 - SBOM attestation stdout (#785)
add syft attest command to produce an attestation as application/vnd.in-toto+json to standard out using on disk PKI

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
2022-02-22 21:45:12 -05:00
Alex Goodman 51c6eb30f5 bump stereoscope to include functional options (#823)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-02-14 20:40:51 -05:00
Jonas Xavier a04fa68539 Ensure completion of UI progress bar (#810)
* update stereoscope

fetches latest fixes for UI

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* use context when getting image

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
2022-02-09 11:23:58 -08:00
Alex Goodman 38c4b17847 Add support for searching for jars within archives (#734)
* add support for searching jars within archives

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add package cataloger config options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address review comments + factor out safeCopy helper

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update config docs regarding package archive search options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* show that unindexed archive cataloging defaults to false

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove lies about -s

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address review comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update search archive note about java

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-01-06 21:40:51 +00:00
Dan Luhring 7de5e1288f Fix unhelpful error message for oci-archive scheme (#705)
* Improve error message

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Return error from stereoscope immediately

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Bump version of stereoscope

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Conditionally retry image retrieval

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Update error message for source construction failure

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Update stereoscope

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Retry image pull without predetermined image source

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Add comment to image pull source determination

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2021-12-22 18:53:23 +00:00
Keith Zantow b7979dbc7d exclusions does not need to be variadic (#711) 2021-12-21 10:13:15 -05:00
Keith Zantow 006ba9b557 Add --exclude flag (#695) 2021-12-20 10:35:25 -05:00
Alex Goodman d76c868481 Catalog archive contents for single-file input (#637)
* add first-level archive processing when input is a file

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add license exception for github.com/xi2/xz

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* always return cleanup function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* change source.NewFromFile log entry to warn

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* ensure file source always has cleanup function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* ensure we are always preferring the unarchive cleanup function for source

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-19 14:16:25 +00:00
Christopher Angelo Phillips fc39710d06 287 - Add retry for image fetch using user's input (#626)
* add fallback to user input if source hint fails

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* refactor for smaller functions

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-11-12 17:57:04 -05:00
Alex Goodman 3ac95ac4f6 Allow for cataloging a single file (#608)
* allow for cataloging a single file

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use all catalogers for file schemes

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 12:09:06 -04:00
houdini91 2f99a35f51 Power user command support for directory scans (#467)
* Power-user directory source support
Signed-off-by: Mikey Strauss <mikey@scribe-security.com>

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Remove newline

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Shared filetree (#1)

* Shared directory resolver filetree

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* PR - change error ErrObserve to ErrPath

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* PR - share directory resolver
* Use pointer to source struct

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Fix Lint

Signed-off-by: houdini91 <mdstrauss91@gmail.com>
2021-09-08 09:18:53 -04:00
Alex Goodman fb0857ff93 Add support for indexing root filesystem (#442)
* change directory resolver to ignore system runtime paths + drive by index

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add event/etui support for filesystem indexing (for dir resolver)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add warnings for path indexing problems

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add directory resolver index tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* improve testing around directory resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* renamed p var to path when not conflicting with import

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* pull docker image in CLI dir scan timeout test

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* ensure file not exist errors do not stop directory resolver indexing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-29 22:06:47 +00:00
Alex Goodman 962e82297c Split UI from event handling (#448)
* split UI from event handling

Signed-off-by: Alex Goodman <wagoodman@gmail.com>

* add event loop tests

Signed-off-by: Alex Goodman <wagoodman@gmail.com>

* use stereoscope cleanup function during signal handling

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* correct error wrapping in packages cmd

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate ui event handlers to ui package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* clarify command worker input var + remove dead comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-29 18:28:09 +00:00
Alex Goodman c363b2b532 Add ability to pull images directly from a registry (#378)
* add registry image source

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use explicit source for fetching image + add scheme and registry tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust test variable name and add credential helper function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-13 13:30:57 +00:00
Alex Goodman a6cba5d9db remove multi* content fetching from resolvers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:53:51 -04:00
Alex Goodman f46de19c6b migrate scope option to image metadata (from source)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:37:12 -05:00
Alex Goodman 6f7a4fd3e4 move source metadata upstream and fix tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:37:12 -05:00
Alex Goodman 9668341a14 rename scope to source
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:37:11 -05:00