Replace core SBOM-creation API with builder pattern (#1383)

* remove existing cataloging API

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add file cataloging config

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add package cataloging config

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add configs for cross-cutting concerns

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* rename CLI option configs to not require import aliases later

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update all nested structs for the Catalog struct

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update Catalog cli options

- add new cataloger selection options (selection and default)
- remove the excludeBinaryOverlapByOwnership
- deprecate "catalogers" flag
- add new javascript configuration

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* migrate relationship capabilities to separate internal package

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* refactor golang cataloger to use configuration options when creating packages

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* create internal object to facilitate reading from and writing to an SBOM

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* create a command-like object (task) to facilitate partial SBOM creation

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add cataloger selection capability

- be able to parse string expressions into a set of resolved actions against sets
- be able to use expressions to select/add/remove tasks to/from the final set of tasks to run

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add package, file, and environment related tasks

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update existing file catalogers to use nested UI elements

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add CreateSBOMConfig that drives the SBOM creation process

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* capture SBOM creation info as a struct

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add CreateSBOM() function

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix tests

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update docs with SBOM selection help + breaking changes

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix multiple override default inputs

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix deprecation flag printing to stdout

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* refactor cataloger selection description to separate object

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* address review comments

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* keep expression errors and show specific suggestions only

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* address additional review feedback

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* address more review comments

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* addressed additional PR review feedback

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix file selection references

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* remove guess language data generation option

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add tests for coordinatesForSelection

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* rename relationship attributes

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add descriptions to relationships config fields

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* improve documentation around configuration options

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add explicit errors around legacy config entries

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
This commit is contained in:
Alex Goodman
2024-01-12 17:39:13 -05:00
committed by GitHub
parent 1bffc6ef0c
commit b0ab75fd89
126 changed files with 5740 additions and 1841 deletions
+42 -8
View File
@@ -162,7 +162,7 @@ func TestPackagesCmdFlags(t *testing.T) {
name: "all-layers-scope-flag-by-env",
args: []string{"scan", "-o", "json", hiddenPackagesImage},
env: map[string]string{
"SYFT_PACKAGE_CATALOGER_SCOPE": "all-layers",
"SYFT_SCOPE": "all-layers",
},
assertions: []traitAssertion{
assertPackageCount(163), // packages are now deduplicated for this case
@@ -234,14 +234,50 @@ func TestPackagesCmdFlags(t *testing.T) {
},
},
{
name: "catalogers-option",
// This will detect enable python-package-cataloger, python-installed-package-cataloger and ruby-gemspec cataloger
args: []string{"scan", "-o", "json", "--catalogers", "python,ruby-gemspec", coverageImage},
name: "legacy-catalogers-option",
// This will detect enable:
// - python-installed-package-cataloger
// - python-package-cataloger
// - ruby-gemspec-cataloger
// - ruby-installed-gemspec-cataloger
args: []string{"packages", "-o", "json", "--catalogers", "python,gemspec", coverageImage},
assertions: []traitAssertion{
assertInOutput("Flag --catalogers has been deprecated, use: override-default-catalogers and select-catalogers"),
assertPackageCount(13),
assertSuccessfulReturnCode,
},
},
{
name: "select-catalogers-option",
// This will detect enable:
// - python-installed-package-cataloger
// - ruby-installed-gemspec-cataloger
args: []string{"scan", "-o", "json", "--select-catalogers", "python,gemspec", coverageImage},
assertions: []traitAssertion{
assertPackageCount(6),
assertSuccessfulReturnCode,
},
},
{
name: "override-default-catalogers-option",
// This will detect enable:
// - python-installed-package-cataloger
// - python-package-cataloger
// - ruby-gemspec-cataloger
// - ruby-installed-gemspec-cataloger
args: []string{"packages", "-o", "json", "--override-default-catalogers", "python,gemspec", coverageImage},
assertions: []traitAssertion{
assertPackageCount(13),
assertSuccessfulReturnCode,
},
},
{
name: "new and old cataloger options are mutually exclusive",
args: []string{"packages", "-o", "json", "--override-default-catalogers", "python", "--catalogers", "gemspec", coverageImage},
assertions: []traitAssertion{
assertFailingReturnCode,
},
},
{
name: "override-default-parallelism",
args: []string{"scan", "-vvv", "-o", "json", coverageImage},
@@ -250,8 +286,7 @@ func TestPackagesCmdFlags(t *testing.T) {
},
assertions: []traitAssertion{
// the application config in the log matches that of what we expect to have been configured.
assertInOutput("parallelism: 2"),
assertInOutput("parallelism=2"),
assertInOutput(`parallelism: 2`),
assertPackageCount(coverageImageSquashedPackageCount),
assertSuccessfulReturnCode,
},
@@ -261,8 +296,7 @@ func TestPackagesCmdFlags(t *testing.T) {
args: []string{"scan", "-vvv", "-o", "json", coverageImage},
assertions: []traitAssertion{
// the application config in the log matches that of what we expect to have been configured.
assertInOutput("parallelism: 1"),
assertInOutput("parallelism=1"),
assertInOutput(`parallelism: 1`),
assertPackageCount(coverageImageSquashedPackageCount),
assertSuccessfulReturnCode,
},