Commit Graph
13 Commits
Author SHA1 Message Date
Alex Goodman 1aaa644007 Remove MetadataType from core package object and normalize JSON metadataType values (#1983)
* [wip]

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* distinct the package metadata functions

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* remove metadata type from package core model

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* incorporate review feedback for names

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add RPM archive metadata and split parser helpers

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* clarify the python package metadata type

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* rename the KB metadata type

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* break hackage and composer types by use case

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* linting fix

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix encoding and decoding for syft-json and cyclonedx

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* bump json schema to 11

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update cyclonedx-json snapshots

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update cyclonedx-xml snapshots

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update spdx-json snapshots

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update spdx-tv snapshots

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update syft-json snapshots

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* correct metadata type in stack yaml parser test

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix bom-ref redactor for cyclonedx-xml

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add tests for legacy package metadata names

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* regenerate json schema v11

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix legacy HackageMetadataType reflect type value check

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix linting

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* packagemetadata discovery should account for type shadowing

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix linting

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix cli tests

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* bump json schema version to v12

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update json schema to incorporate changes from main

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add syft-json legacy config option

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add tests around v11-v12 json decoding

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add docs for SYFT_JSON_LEGACY

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* rename structs to be compliant with new naming scheme

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-10-30 12:12:04 -04:00
Alex Goodmanandmanifestori 063e9da65d Guess unpinned versions in python requirements.txt (#1966)
* feat: python requirements.txt parsing inclusive

Signed-off-by: manifestori <ori@manifestcyber.com>

* refactor: parseVersion

Signed-off-by: manifestori <ori@manifestcyber.com>

* add python config for optional requirements version constraint resolution

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix tests

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* allow for python requirements metadata to be optional

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* restore cyclonedx dependency

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: manifestori <ori@manifestcyber.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: manifestori <ori@manifestcyber.com>
2023-07-27 14:26:59 -04:00
Alex Goodman d84120f499 bump JSON schema to account for simplified python env markers (#1967)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2023-07-27 14:13:17 +00:00
Alex Goodman 07e76907f6 Migrate location-related structs to the file package (#1751)
* migrate location structs to file package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* replace source.Location refs with file package call

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove hardlink test for file based catalogers

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove hardlink test for all-regular-files testing

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate file resolver implementations to separate package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* [wip] migrate resolvers to internal

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate resolvers to syft/internal

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: <>
2023-05-24 17:06:38 -04:00
Shane Dell a07bfe7dfa Create python requirements metadata (#1759)
- Create new metadata struct and type for python requirements.
- Update parsing of python requirements to use python requirements metadata.
- Remove extras and url from line. Add them to metadata instead.
- Add unit test to test that extras are removed from package name.
- Update test to look at requirements metadata.
- Will need updated in future to support more than just == for the version constraint.
- Update JSON schema data

Closes anchore/grype#1246
Closes anchore/grype#1251

Signed-off-by: Shane Dell <shanedell100@gmail.com>
2023-04-27 09:04:30 -04:00
Alex Goodman 5d156b8241 Add annotations for evidence on package locations (#1723)
* add location annotations + deb evidence annotations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename LocationData struct and Annotation helper function

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add failing integration test for evidence coverage

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add evidence to aplm cataloger locations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* change location annotation helper to return a location copy

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add evidence to binary cataloger locations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* updated remaining catalogers with location annotations

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix unit tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* bump json schema

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* partial addressing of review comments

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename location.WithAnnotation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2023-04-13 17:02:29 -04:00
Alex Goodman 5dd726fc86 clean package names in python parsers (#1417)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-12-21 18:31:49 +00:00
Christopher Angelo Phillips bcfe38c009 fix: requirements.txt - return unicode only letter/num for version (#1361) 2022-11-22 10:43:05 -05:00
Alex Goodman 1076281566 port python cataloger to new generic cataloger pattern (#1319)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2022-11-04 08:59:03 -04:00
Dan Luhring 641c44f449 Fix panic in requirements.txt parsing (#834)
* Stable sort for pipfile.lock parsing

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Adjust python parsing tests to use go-cmp

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Add failing cases for requirements.txt parsing

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Fix failing cases for requirements.txt parsing

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Refactor parseRequirementsTxt

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Fix static-analysis failure

Signed-off-by: Dan Luhring <dan+github@luhrings.com>

* Fix comment

Signed-off-by: Dan Luhring <dan+github@luhrings.com>
2022-02-17 10:00:16 -05:00
Alex Goodman a27907659d Performance improvements around package ID (#698)
* set package ID in catalogers and improve hashing performance

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update setting ID + tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-12-16 08:55:53 -05:00
Alex Goodman ef627d82ef Introduce relationships as first-class objects (#607)
* migrate pkg.ID and pkg.Relationship to artifact package

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* return relationships from tasks

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix more tests

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add artifact.Identifiable by Identity() method

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* fix linting

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove catalog ID assignment

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust spdx helpers to use copy of packages

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* stabilize package ID relative to encode-decode format cycles

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* rename Identity() to ID()

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* use zero value for nils in ID generation

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* enable source.Location to be identifiable

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* hoist up package relationship discovery to analysis stage

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* update ownership-by-file-overlap relationship description

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add test reminders to put new relationships under test

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* adjust PHP composer.lock parser function to return relationships

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-11-16 14:14:13 -05:00
Alex Goodman 4666ca8469 migrate syft/cataloger to syft/pkg/cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
2021-03-22 10:46:51 -04:00