Remove MetadataType from core package object and normalize JSON metadataType values (#1983)

* [wip]

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* distinct the package metadata functions

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* remove metadata type from package core model

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* incorporate review feedback for names

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add RPM archive metadata and split parser helpers

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* clarify the python package metadata type

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* rename the KB metadata type

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* break hackage and composer types by use case

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* linting fix

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix encoding and decoding for syft-json and cyclonedx

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* bump json schema to 11

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update cyclonedx-json snapshots

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update cyclonedx-xml snapshots

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update spdx-json snapshots

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update spdx-tv snapshots

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update syft-json snapshots

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* correct metadata type in stack yaml parser test

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix bom-ref redactor for cyclonedx-xml

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add tests for legacy package metadata names

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* regenerate json schema v11

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix legacy HackageMetadataType reflect type value check

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix linting

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* packagemetadata discovery should account for type shadowing

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix linting

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix cli tests

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* bump json schema version to v12

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* update json schema to incorporate changes from main

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add syft-json legacy config option

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add tests around v11-v12 json decoding

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* add docs for SYFT_JSON_LEGACY

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* rename structs to be compliant with new naming scheme

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
This commit is contained in:
Alex Goodman
2023-10-30 12:12:04 -04:00
committed by GitHub
parent f442586ec9
commit 1aaa644007
261 changed files with 7010 additions and 4040 deletions
+78
View File
@@ -0,0 +1,78 @@
package pkg
import (
"sort"
"github.com/scylladb/go-set/strset"
"github.com/anchore/syft/syft/file"
)
const DpkgDBGlob = "**/var/lib/dpkg/{status,status.d/**}"
var _ FileOwner = (*DpkgDBEntry)(nil)
// DpkgDBEntry represents all captured data for a Debian package DB entry; available fields are described
// at http://manpages.ubuntu.com/manpages/xenial/man1/dpkg-query.1.html in the --showformat section.
// Additional information about how these fields are used can be found at
// - https://www.debian.org/doc/debian-policy/ch-controlfields.html
// - https://www.debian.org/doc/debian-policy/ch-relationships.html
// - https://www.debian.org/doc/debian-policy/ch-binary.html#s-virtual-pkg
// - https://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual
type DpkgDBEntry struct {
Package string `json:"package"`
Source string `json:"source" cyclonedx:"source"`
Version string `json:"version"`
SourceVersion string `json:"sourceVersion" cyclonedx:"sourceVersion"`
// Architecture can include the following sets of values depending on context and the control file used:
// - a unique single word identifying a Debian machine architecture as described in Architecture specification string (https://www.debian.org/doc/debian-policy/ch-customized-programs.html#s-arch-spec) .
// - an architecture wildcard identifying a set of Debian machine architectures, see Architecture wildcards (https://www.debian.org/doc/debian-policy/ch-customized-programs.html#s-arch-wildcard-spec). any matches all Debian machine architectures and is the most frequently used.
// - "all", which indicates an architecture-independent package.
// - "source", which indicates a source package.
Architecture string `json:"architecture"`
// Maintainer is the package maintainer’s name and email address. The name must come first, then the email
// address inside angle brackets <> (in RFC822 format).
Maintainer string `json:"maintainer"`
InstalledSize int `json:"installedSize" cyclonedx:"installedSize"`
// Description contains a description of the binary package, consisting of two parts, the synopsis or the short
// description, and the long description (in a multiline format).
Description string `hash:"ignore" json:"-"`
// Provides is a virtual package that is provided by one or more packages. A virtual package is one which appears
// in the Provides control field of another package. The effect is as if the package(s) which provide a particular
// virtual package name had been listed by name everywhere the virtual package name appears. (See also Virtual packages)
Provides []string `json:"provides,omitempty"`
// Depends This declares an absolute dependency. A package will not be configured unless all of the packages listed in
// its Depends field have been correctly configured (unless there is a circular dependency).
Depends []string `json:"depends,omitempty"`
// PreDepends is like Depends, except that it also forces dpkg to complete installation of the packages named
// before even starting the installation of the package which declares the pre-dependency.
PreDepends []string `json:"preDepends,omitempty"`
Files []DpkgFileRecord `json:"files"`
}
// DpkgFileRecord represents a single file attributed to a debian package.
type DpkgFileRecord struct {
Path string `json:"path"`
Digest *file.Digest `json:"digest,omitempty"`
IsConfigFile bool `json:"isConfigFile"`
}
func (m DpkgDBEntry) OwnedFiles() (result []string) {
s := strset.New()
for _, f := range m.Files {
if f.Path != "" {
s.Add(f.Path)
}
}
result = s.List()
sort.Strings(result)
return
}