241 Commits

Author SHA1 Message Date
Nikita
25835ef589
PHP parser composer.lock (#609)
* PHP parse composer.lock

Signed-off-by: Nikita <33390074+Zilborg@users.noreply.github.com>

* rename PHP package type

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add PHP composer package support to source info SPDX helper

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update directory cataloger integration tests (make exception for images)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-10 14:08:41 -05:00
Alex Goodman
bb0f35bac4
Introduce a single SBOM document (#606)
* [wip] single sbom doc

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix more tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update cli tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove scope in import path

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* swap SPDX tag-value formatter to single sbom document

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bust CLI cache

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update fixture to byte diff

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* byte for byte

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* bust the cache

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* who needs cache

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* add jar for testing

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* no more bit flips

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update apk with the delta for image and directory cases

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* restore cache workflow

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-11-05 10:05:49 -04:00
Alex Goodman
ecab44bba1
Add SPDX tag-value format object (#605)
* add new spdx tag-value format

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove public presenter package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 13:45:34 -04:00
Alex Goodman
3ac95ac4f6
Allow for cataloging a single file (#608)
* allow for cataloging a single file

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use all catalogers for file schemes

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-02 12:09:06 -04:00
Alex Goodman
4a2d1d7225
Port cyclonedx presenter to format object (#589)
* add new cyclonedx format object

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove cyclonedx presenter

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove cyclonedx presenter call

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove dependence on golden images for format tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* wire up new formt + rename all-presenters ref

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add CLI test to ensure that all formats can be expressed as report output

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add cyclonedx version and encoding format to package name

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* optionally preserve format snapshot images

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting + text unit tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-29 10:17:03 -04:00
Alex Goodman
560b05c2c9
Introduce new format pattern + port json processing (#550)
* add new format pattern

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add syftjson format

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add internal formats helper

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add SBOM encode/decode to lib API

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove json presenter + update presenter tests to use common utils

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove presenter format enum type + add formats shim in presenter helper

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add MustCPE helper for tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update usage of format enum

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add test fixtures for encode/decode tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix integration test

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate format detection to use reader

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address review comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-20 21:36:34 +00:00
Alex Goodman
6f3fa494d7
Show help text when no args are given (#573)
* show help text when no args are given

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* repurpose the input args validation function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* ensure app does not check for update in cli tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-20 09:51:05 -04:00
Christopher Angelo Phillips
b25f5b664b
(464) - remove mod and cargo from image cataloger (#539)
* remove mod and cargo from image cataloger

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* update test error messages for clear failures

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
2021-10-07 15:18:20 -04:00
Christopher Angelo Phillips
3462e18af3
478 identify go binaries and extract mod information (#534)
* add query by MIME type to source.FileResolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* import stereoscope lib changes to find mime type

- add bin cataloger
- add bin parser
- add mime type go utils
- import new resolver

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>

* add go std library code to unpack bin

- keep them in their own (original) files
- add note for "this code was copied from"
- comment the lines the required changing

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-10-07 12:16:38 -04:00
houdini91
2f99a35f51
Power user command support for directory scans (#467)
* Power-user directory source support
Signed-off-by: Mikey Strauss <mikey@scribe-security.com>

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Remove newline

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Shared filetree (#1)

* Shared directory resolver filetree

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* PR - change error ErrObserve to ErrPath

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* PR - share directory resolver
* Use pointer to source struct

Signed-off-by: houdini91 <mdstrauss91@gmail.com>

* Fix Lint

Signed-off-by: houdini91 <mdstrauss91@gmail.com>
2021-09-08 09:18:53 -04:00
Keith Zantow
ff828fbac2
Skip package-lock.json and yarn.lock in node_modules #431 (#485)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-08-20 13:50:28 -04:00
Keith Zantow
93b4bf9379
Add option to enable http registry connections (#482)
Signed-off-by: Keith Zantow <kzantow@gmail.com>
2021-08-17 12:52:51 -04:00
Alex Goodman
ecf4e5546a
add help message when no arguments are provided (#455)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-30 12:53:12 -04:00
Alex Goodman
fb0857ff93
Add support for indexing root filesystem (#442)
* change directory resolver to ignore system runtime paths + drive by index

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add event/etui support for filesystem indexing (for dir resolver)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add warnings for path indexing problems

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add directory resolver index tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* improve testing around directory resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* renamed p var to path when not conflicting with import

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* pull docker image in CLI dir scan timeout test

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* ensure file not exist errors do not stop directory resolver indexing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-29 22:06:47 +00:00
Alex Goodman
706322f826
Add SPDX support (#445)
* add initial spdx support

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* expose FileOwner and use in SPDX presenter

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add initial json support for SPDX

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add remaining package fields

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add spdx license list generation + tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* keep fileOwner unexported from pkg

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* restore cli test util

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add external refs to spdx tag-value format

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add golang support to CPE generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use tag-value format as default "spdx" format flavor

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add tests around spdx presenters + refactor presenter tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add bouncer exception for spdx tools-golang repo

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove spdx model questions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-06-25 16:30:41 -04:00
Samuel Dacanay
5a2e2eb679 Export the presenter interface so it may be used by external projects
Signed-off-by: Samuel Dacanay <sam.dacanay@anchore.com>
2021-06-09 10:21:55 -07:00
Dan Luhring
17bbf840cf
Allow registry auth config without authority value (#420)
* Allow registry auth config without authority value

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Update CLI tests for new stereoscope log output

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-05-24 16:05:56 -04:00
Alfredo Deza
4cce32f5c9 tests: remove KbPkg from defined pkgs since it isn't parsed
Signed-off-by: Alfredo Deza <adeza@anchore.com>
2021-04-23 10:17:16 -04:00
Alex Goodman
d9de63c837
Enhance CPE generation for java GroupId and filtering (#402)
* enhance cpe generation for group id and filtering

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename group id const + add doc comment for HasAnyOfPrefixes

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-22 12:22:56 +00:00
Alex Goodman
0c100c4037
change mac acceptance test image.tar path
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-16 09:57:13 -04:00
Alex Goodman
c363b2b532
Add ability to pull images directly from a registry (#378)
* add registry image source

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use explicit source for fetching image + add scheme and registry tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust test variable name and add credential helper function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-13 13:30:57 +00:00
Alex Goodman
51b13f8221
rename Contents to FileContents in app config and documentation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:34:54 -04:00
Alex Goodman
c1551a03c5
add file contents cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-12 17:30:57 -04:00
Alex Goodman
9ec09add67
Add secrets search capability (#367)
* add initial secrets cataloger

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update ETUI elements with new catalogers (file metadata, digests, and secrets)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update secrets cataloger to read full contents into memory for searching

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype of parallelization secret regex search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype with single aggregated regex

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype for secret search line-by-line

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* quick prototype hybrid secrets search

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add secrets cataloger with line strategy

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust verbiage towards SearchResults instead of Secrets + add tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update json schema with secrets cataloger results

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* address PR comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update readme with secrets config options

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* ensure file catalogers call AllLocations once

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-04-01 21:34:15 +00:00
Alex Goodman
8551168702
capitalize file type strings
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-29 12:07:41 -04:00
Alex Goodman
8854d83934
cache mac acceptance image
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-23 15:03:25 -04:00
Alex Goodman
46636ccb3c
simplify command alias logic + remove deprecation warning for root command
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 15:25:04 -04:00
Alex Goodman
f180d1c537
improve config parsing + fix command deprecation warning
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:58:09 -04:00
Alex Goodman
cf516add95
add cli testing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:56:18 -04:00
Alex Goodman
68d698e9f2
remove inline-compare testing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:56:18 -04:00
Alex Goodman
efcd8a8b9a
update integration and acceptance tests for new resolvers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:56:18 -04:00
Weston Steimel
ba81bfe529
add cataloger for rust crates from Cargo.lock files
Signed-off-by: Weston Steimel <weston.steimel@gmail.com>
2021-03-18 03:12:51 +00:00
Alex Goodman
5e62bca72f
Revert "Add docker image and refactor release pipeline (#310)"
This reverts commit 6195002ae54056131521f2226ee21a0822ab1019.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2021-03-11 12:42:18 -05:00
Dan Luhring
6195002ae5
Add docker image and refactor release pipeline (#310)
* Create independent build targets for Mac and Linux

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Create targets for macOS signing and notarization

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Create target for Linux packaging

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Update release workflow and leverage new make targets

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add release assets to release draft

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add homebrew formula release follow-up and improve Makefile

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add follow-up workflow for updating version check file

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Get rid of fetch depth 0 for checkout action

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Add follow-up workflow for Docker images

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Restore wait-for-checks job

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Replace make functions with shell functions

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* Account for envsubst command in bootstrap-ci-linux

Signed-off-by: Dan Luhring <dan.luhring@anchore.com>

* move homebrew generation into script

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add release approval step; remove goreleaser; add docker image smoke testing in acceptance step

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* replace homebrew formula template file with heredoc template

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update release documentation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-10 13:25:31 -05:00
Weston Steimel
6c80f9910f
add to dpkg integration test cases to detect packages from /var/lib/dpkg/status.d/
Signed-off-by: Weston Steimel <weston.steimel@gmail.com>
2021-02-27 01:58:59 +00:00
Alex Goodman
6d5ff0fd8e
Mark package relations by file ownership (#329)
* add marking package relations by file ownership

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* correct json schema version; ensure fileOwners dont return dups; pin test pkg versions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* extract package relationships into separate section

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* pull in client-go features for import of PackageRelationships

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* move unit test for ownership by files relationship further down

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename relationship to "ownership-by-file-overlap"

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-02-25 13:47:13 -05:00
Dan Luhring
0ccfee03f1
Fix dependency resolution issue in test image
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-01-27 10:30:49 -05:00
Dan Luhring
ae7cd6bbb7
Add test for Java cataloging with no main package
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-01-27 10:12:41 -05:00
Dan Luhring
9ec3ad58c8
Update regression test expected value and pin deps
Signed-off-by: Dan Luhring <dan.luhring@anchore.com>
2021-01-22 14:03:50 -05:00
Alex Goodman
6a49717314
add VirtualPath to source.Location
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-01-05 21:16:42 -05:00
Alex Goodman
133d180eec
update gemspec glob to include named spec dirs
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-01-04 19:33:52 -05:00
Alex Goodman
7f4e8ab97d
Fix symlink resolutions for constituent paths (#304)
* bump stereoscope to pull in content API refactors

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* incorporate symlink fixes

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* with filetree.File() adjustments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* regress all-layers scope to not include dead-links + default tests to squashed scope

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* restore all layers resolver glob behavior (custom + lazy link resolution)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* incorporate filetree link resolution options and restore no-follow dead link option for resolvers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* removed path from lower-level FileTree.File() calls

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump stereoscope to pull in latest link resolution fixes

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump doublestar to v2 for directory resolver

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-01-04 16:41:45 -05:00
Alex Goodman
75d89293ce
fix acceptance tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-12-18 14:32:29 -05:00
Alex Goodman
52bac6e2fd
Add enterprise upload capability (#285)
* add support to upload results to enterprise

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add package sbom upload

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add dockerfile support

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add manifest, index, and dockerfile import functions

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* schema version to json output + enhance json schema generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* modify package SBOM shape to be entire syft document + add etui updates

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add import image config and manifest support

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add config options for import to enterprise

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* incorporate final stereoscope and client-go deps

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-12-09 22:20:53 -05:00
Alex Goodman
2989d3d975
include CPEs with elementds from POM GroupId fields
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-12-01 08:00:49 -05:00
Alex Goodman
8a17bfb69f
generate json schema from struct definitions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-20 15:47:10 -05:00
Alex Goodman
3aaa0e5566
move package purl and cpes (identities) to pkg.Package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-19 11:25:51 -05:00
Alex Goodman
569a598df7
minimize pointer usage & order return types consistently
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:37:13 -05:00
Alex Goodman
f46de19c6b
migrate scope option to image metadata (from source)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:37:12 -05:00
Alex Goodman
91baabe5a1
add image metadata as catalogFromJSON return
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2020-11-17 12:37:12 -05:00