12 Commits

Author SHA1 Message Date
Alex Goodman
edd910f88f [wip] more concurrent catalogers
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2024-10-01 10:18:44 -04:00
Alex Goodman
05e8ba948d
Add python wheel egg relationships (#2903)
* add python package relationships

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* nil for empty relationships collections

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* new json schema for optional python requiremenets

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update format snapshots for python packages

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* decompose python parsers more + add tests around plural fields

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update JSON schema with python dep refs

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2024-05-25 20:11:38 +00:00
Alex Goodman
4a18895545
Add abstraction for adding relationships from package cataloger results (#2853)
* add internal dependency resolver

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* refactor dependency relationship resolution to common object

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* replace cataloger decorator with generic processor

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* refactor resolver to be a single function

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* use common dependency specifier for debian

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* use common dependency specifier for arch

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* use common dependency specifier for alpine

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* allow for generic pkg and rel assertions in testpkg helper

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* do not allow for empty results

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* move stable deduplicate comment

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* remove relationship resolver type

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2024-05-14 13:27:36 +00:00
William Murphy
93a7d2ee27
fix: better clean up of file handles (#2823)
* always close ELF cataloger file handles

The elf-binary-package-cataloger does its own file IO to account for the
possibility of a logical ELF package being broken across multiple
physical files. However, this casued it to skip the normal invocation
pattern in the generic cataloger code that prevented file leaks. Ensure
this cataloger always closes its file handles.

Signed-off-by: Will Murphy <will.murphy@anchore.com>

* defer closing of generic cataloger file handles

Otherwise, a panicking cataloger could leak file handles.

Signed-off-by: Will Murphy <will.murphy@anchore.com>

* add unit test for file closed on panic parser

Signed-off-by: Will Murphy <will.murphy@anchore.com>

* make invoke parser a static function

Signed-off-by: Will Murphy <will.murphy@anchore.com>

* push error logging down into invoke parser

Signed-off-by: Will Murphy <will.murphy@anchore.com>

---------

Signed-off-by: Will Murphy <will.murphy@anchore.com>
2024-05-01 12:58:17 -04:00
anchore-actions-token-generator[bot]
17ef243956
chore(deps): update tools to latest versions (#2616)
* chore(deps): update tools to latest versions

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: update to new linter rules

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: spiffcs <32073428+spiffcs@users.noreply.github.com>
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
2024-02-14 14:59:49 +00:00
Alex Goodman
cdad5e767a
plumb context through catalogers (#2528)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2024-01-22 15:54:51 -05:00
Alex Goodman
3f13d209a5
rename file.Location.VirtualPath to AccessPath (#2288)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-11-09 11:30:08 -06:00
Alex Goodman
07e76907f6
Migrate location-related structs to the file package (#1751)
* migrate location structs to file package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* replace source.Location refs with file package call

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove hardlink test for file based catalogers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove hardlink test for all-regular-files testing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate file resolver implementations to separate package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] migrate resolvers to internal

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate resolvers to syft/internal

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: <>
2023-05-24 17:06:38 -04:00
Alex Goodman
988041ba6d
Speed up cataloging by replacing globs searching with index lookups (#1510)
* replace raw globs with index equivelent operations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add cataloger test for alpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix import sorting for binary cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting for mock resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* separate portage cataloger parser impl from cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enhance cataloger pkgtest utils to account for resolver responses

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for alpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for apkdb cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dpkg cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for cpp cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dart cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for dotnet cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for elixir cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for erlang cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for golang cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for haskell cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for java cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for javascript cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for php cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for portage cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for python cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for rpm cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for rust cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for sbom cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for swift cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* allow generic catloger to run all mimetype searches at once

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove stutter from php and javascript cataloger constructors

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add tests for generic.Search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add exceptions for java archive git ignore entries

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enhance basename and extension resolver methods to be variadic

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* dont allow * prefix on extension searches

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add glob-based cataloger tests for ruby cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove unnecessary string casting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* incorporate surfacing of leaf link resolitions from stereoscope results

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] switch to stereoscope file metadata

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip + failing] revert to old globs but keep new resolvers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* index files, links, and dirs within the directory resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix several resolver bugs and inconsistencies

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move format testutils to internal package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update syft json to account for file type string normalization

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* split up directory resolver from indexing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update docs to include details about searching

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] bump stereoscope to development version

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust symlinks fixture to be fixed to digest

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix all-locations resolver tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix test fixture reference

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename file.Type

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix PR comment to exclude extra *

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump to dev version of stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump to final version of stereoscope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move observing resolver to pkgtest

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-02-09 16:19:47 +00:00
Alex Goodman
36a0945c95
push detailed log statements to trace-level (#1500)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-01-20 14:33:23 +00:00
Alex Goodman
7a69e2129b
recover from bad parsing of golang binary (#1371)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-29 15:56:46 +00:00
Alex Goodman
b44f441c82
Upgrade generic cataloger (#1281)
* add second generation of generic cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* upgrade aplm cataloger to use generic.Cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove pacakge found-by attribute from the definition of a package ID

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-10-24 11:12:12 -04:00