mirror of
https://github.com/anchore/syft.git
synced 2026-08-19 08:38:25 +02:00
feat: catalog CPAN distributions installed by perl clients
Adds `perl` as a language and `cpan` as a package type, with two catalogers behind them.
- `perl-cpan-installed-cataloger` reads `.meta/*/install.json` (cpanm, cpm, carton) and `auto/**/.packlist` (anything installed through `ExtUtils::MakeMaker` or `Module::Build`, including CPAN.pm). Both globs are unanchored, so a local-lib or carton application tree is found the same as a system install.
- `perl-cpan-meta-cataloger` reads an unpacked release's own `META.json` or `META.yml`, gated on a sibling `MANIFEST` so source checkouts are ignored.
Packages are keyed on the **distribution**, not the module, because that is what CPAN, MetaCPAN and the advisory data all use. `LWP.pm` belongs to `libwww-perl` and reporting it as `LWP` would make every advisory for it unreachable. The distribution name comes from `install.json`'s `dist` field, and from the `auto/` path for packlists.
purls are `pkg:cpan/<distribution>@<version>`, with the PAUSE author added as an `author` qualifier when the evidence carries it. Metadata comes in two types: `cpan-distribution` for installed evidence and `cpan-unpacked-release` for a release sitting on disk. The tiers differ in more than a name, so a consumer should not have to string-match a cataloger name to tell them apart: an unpacked release has no PAUSE path and therefore no author and no file list, and "installed and loadable by the interpreter" is a materially stronger claim than "a source tree exists here".
A packlist's version comes from `perllocal.pod`. EUMM writes it and the packlist from the same variables in the same install target, so the `auto/` path segments and the perllocal `Module` name are the same key, and the recorded `VERSION` is what was evaluated at build time rather than what can be read back statically. Scraping `$VERSION` out of the main `.pm` is the fallback, since `NO_PERLLOCAL` suppresses the file and Module::Build never writes one. Three rules pick the stanza: dashed module name, longest `installed into` libdir that prefixes the packlist path, and last match wins because the file is append-only. Without the libdir rule a second perl on the image silently supplies the version.
Where scraping is the fallback, it reads more than a plain `our $VERSION = '...'`. A version declared in the package statement (`package Foo::Bar v1.0.0;`) counts, which matters because a distribution can declare it that way and carry no `$VERSION` at all: `CPAN::02Packages::Search` is one, and without this it reports no version and matches nothing. Fully qualified `$Foo::Bar::VERSION` counts too, which is what older Dist::Zilla emitted and what real `JSON::PP` 2.27300 still carries. `qv('1.2.3')` is handled. A version computed at runtime is not, and those are reported without one rather than guessed at.
Packlist entries are parsed the way `ExtUtils::Packlist` writes them: a line can carry space-separated metadata after the path (`/path/to/File.pm type=file`), which is what `installperl` produces, so the suffix is stripped rather than the line being cut at its first space.
`META.yml` is read alongside `META.json`, because about 43% of current CPAN releases ship no `META.json` and they skew old, which is where the advisories are. Where both sit in one directory the `META.json` wins. A `META.yml` a strict parser rejects skips that directory rather than failing the scan, which is routine rather than defensive for pre-spec releases.
A packlist is literally a file list, so its paths are surfaced through `pkg.FileOwner`. They are reported as recorded, unfiltered: a path the packlist claims and the filesystem lacks means the file was removed or overwritten out from under the installer, which is worth seeing rather than hiding.
Build leftovers under `~/.cpanm/work` and `~/.cpan/build` are skipped. They genuinely are unpacked release tarballs, `MANIFEST` included, so the `MANIFEST` gate admits them and a path exclusion is the only signal available. Without it every distribution on an image that did not clean up is reported twice. The cost is that a tarball deliberately kept under `~/.cpan/build` stops being reported.
Two behaviors that look wrong but are not:
- a distribution can be reported twice at different versions. `libwww-perl` 5.836 bundles `HTTP-Date`, `HTTP-Message` and `LWP-MediaTypes`, and installing it over the modern standalone releases overwrites their `.pm` files. Both versions are genuinely present, so the merge refuses to pair disagreeing versions rather than hiding one.
- a distribution with no readable version is reported without one rather than dropped, so it stays visible.
The coverage boundary is stated in full in the `package perl` doc comment. In short: modules installed from distro packages are out of scope, since packagers strip CPAN metadata with `NO_PACKLIST` and deb, rpm and apk already report them. Core and dual-life distributions bundled with the interpreter have no coverage, because nothing on disk carries their versions; the interpreter itself is reported separately. Vendored trees, `App::FatPacker` output and PAR archives are invisible, because what they carry is module identity with no offline map to a distribution. A packlist-derived name is the installer's `NAME` and may not be the distribution name, which is left to the vulnerability data to resolve.
One correction worth calling out, since it is easy to arrive at twice: a sibling `MANIFEST` is the only thing separating an unpacked release from a source checkout. An earlier version of the guard also rejected any tree containing a `dist.ini`, on the theory that it marked a Dist::Zilla source tree. dzil ships `dist.ini` *inside* the tarballs it builds and lists it in the generated `MANIFEST`, so that exclusion was silently skipping real releases, `URI` among them.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
This commit is contained in:
parent
08e913a2ce
commit
1827ce2f4f
@ -524,4 +524,12 @@ var commonTestCases = []testCase{
|
||||
"kong": "3.7.0-0",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "find cpan distribution",
|
||||
pkgType: pkg.CpanPkg,
|
||||
pkgLanguage: pkg.Perl,
|
||||
pkgInfo: map[string]string{
|
||||
"URI": "5.35",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
@ -0,0 +1,9 @@
|
||||
{
|
||||
"abstract": "Uniform Resource Identifiers (absolute and relative)",
|
||||
"license": ["perl_5"],
|
||||
"meta-spec": {"version": 2},
|
||||
"name": "URI",
|
||||
"prereqs": {"runtime": {"requires": {"Carp": "0", "perl": "5.008001"}}},
|
||||
"release_status": "stable",
|
||||
"version": "5.35"
|
||||
}
|
||||
@ -0,0 +1,11 @@
|
||||
{
|
||||
"dist": "URI-5.35",
|
||||
"name": "URI",
|
||||
"pathname": "O/OA/OALDERS/URI-5.35.tar.gz",
|
||||
"provides": {
|
||||
"URI": {"file": "lib/URI.pm", "version": "5.35"},
|
||||
"URI::Escape": {"file": "lib/URI/Escape.pm", "version": "5.35"}
|
||||
},
|
||||
"target": "URI",
|
||||
"version": "5.35"
|
||||
}
|
||||
@ -3,7 +3,7 @@ package internal
|
||||
const (
|
||||
// JSONSchemaVersion is the current schema version output by the JSON encoder
|
||||
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
|
||||
JSONSchemaVersion = "16.1.10"
|
||||
JSONSchemaVersion = "16.1.11"
|
||||
|
||||
// Changelog
|
||||
// 16.1.0 - reformulated the python pdm fields (added "URL" and removed the unused "path" field).
|
||||
@ -17,4 +17,5 @@ const (
|
||||
// 16.1.8 - add VcpkgManifest metadata type for vcpkg manifest support
|
||||
// 16.1.9 - add Symbols (grouped by owning package import path) to GolangBinaryBuildinfoEntry metadata
|
||||
// 16.1.10 - add SafeTensorsModelInfo metadata type for the safetensors AI model cataloger
|
||||
// 16.1.11 - add CpanDistribution and CpanUnpackedRelease metadata types for the perl catalogers
|
||||
)
|
||||
|
||||
@ -19,6 +19,8 @@ func AllTypes() []any {
|
||||
pkg.ConanfileEntry{},
|
||||
pkg.ConaninfoEntry{},
|
||||
pkg.CondaMetaPackage{},
|
||||
pkg.CpanDistribution{},
|
||||
pkg.CpanUnpackedRelease{},
|
||||
pkg.DartPubspec{},
|
||||
pkg.DartPubspecLockEntry{},
|
||||
pkg.DenoLockEntry{},
|
||||
|
||||
@ -71,6 +71,8 @@ var jsonTypes = makeJSONTypes(
|
||||
jsonNames(pkg.ConanV2LockEntry{}, "c-conan-lock-v2-entry"),
|
||||
jsonNames(pkg.ConanfileEntry{}, "c-conan-file-entry", "ConanMetadataType"),
|
||||
jsonNames(pkg.ConaninfoEntry{}, "c-conan-info-entry"),
|
||||
jsonNames(pkg.CpanDistribution{}, "cpan-distribution"),
|
||||
jsonNames(pkg.CpanUnpackedRelease{}, "cpan-unpacked-release"),
|
||||
jsonNames(pkg.DartPubspecLockEntry{}, "dart-pubspec-lock-entry", "DartPubMetadata"),
|
||||
jsonNames(pkg.DartPubspec{}, "dart-pubspec"),
|
||||
jsonNames(pkg.DenoLockEntry{}, "deno-lock-entry"),
|
||||
|
||||
@ -279,6 +279,46 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
|
||||
return []pkg.Package{parent, child}, nil
|
||||
},
|
||||
},
|
||||
{
|
||||
// a CPAN distribution owns whatever its .packlist recorded, which is how a distribution that
|
||||
// installed a binary or a shared object is related to the package cataloged from it
|
||||
name: "cpan-distribution-owns-packlist-paths",
|
||||
setup: func(t testing.TB) ([]pkg.Package, []artifact.Relationship) {
|
||||
parent := pkg.Package{
|
||||
Locations: file.NewLocationSet(
|
||||
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/Net/SSLeay/.packlist"),
|
||||
),
|
||||
Type: pkg.CpanPkg,
|
||||
Metadata: pkg.CpanDistribution{
|
||||
MainModule: "Net::SSLeay",
|
||||
Files: []string{
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/Net/SSLeay.pm",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/Net/SSLeay/SSLeay.so",
|
||||
},
|
||||
},
|
||||
}
|
||||
parent.SetID()
|
||||
|
||||
child := pkg.Package{
|
||||
Locations: file.NewLocationSet(
|
||||
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/Net/SSLeay/SSLeay.so"),
|
||||
),
|
||||
Type: pkg.BinaryPkg,
|
||||
}
|
||||
child.SetID()
|
||||
|
||||
return []pkg.Package{parent, child}, []artifact.Relationship{
|
||||
{
|
||||
From: parent,
|
||||
To: child,
|
||||
Type: artifact.OwnershipByFileOverlapRelationship,
|
||||
Data: ownershipByFilesMetadata{
|
||||
Files: []string{"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/Net/SSLeay/SSLeay.so"},
|
||||
},
|
||||
},
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
|
||||
@ -27,6 +27,7 @@ import (
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/lua"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/nix"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/ocaml"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/perl"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/php"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/python"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/r"
|
||||
@ -132,6 +133,10 @@ func DefaultPackageTaskFactories() Factories {
|
||||
newSimplePackageTaskFactory(swift.NewSwiftPackageManagerCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "swift", "spm"),
|
||||
newSimplePackageTaskFactory(swipl.NewSwiplPackCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "swipl", "pack"),
|
||||
newSimplePackageTaskFactory(ocaml.NewOpamPackageManagerCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "ocaml", "opam"),
|
||||
// note: no image tag -- an unpacked release's own META.json describes a distribution present on
|
||||
// disk but not installed into the interpreter's @INC, and declared information is not cataloged
|
||||
// inside container images
|
||||
newSimplePackageTaskFactory(perl.NewCpanMetaCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "perl", "cpan"),
|
||||
|
||||
// language-specific package for both image and directory scans (but not necessarily declared) ////////////////////////////////////////
|
||||
newPackageTaskFactory(
|
||||
@ -162,6 +167,7 @@ func DefaultPackageTaskFactories() Factories {
|
||||
pkgcataloging.DirectoryTag, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.LanguageTag, "nix",
|
||||
),
|
||||
newSimplePackageTaskFactory(lua.NewPackageCataloger, pkgcataloging.DirectoryTag, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.LanguageTag, "lua"),
|
||||
newSimplePackageTaskFactory(perl.NewCpanInstalledCataloger, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "perl", "cpan"),
|
||||
|
||||
// other package catalogers ///////////////////////////////////////////////////////////////////////////
|
||||
newPackageTaskFactory(
|
||||
|
||||
4647
schema/json/schema-16.1.11.json
Normal file
4647
schema/json/schema-16.1.11.json
Normal file
File diff suppressed because it is too large
Load Diff
@ -1,6 +1,6 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "anchore.io/schema/syft/json/16.1.10/document",
|
||||
"$id": "anchore.io/schema/syft/json/16.1.11/document",
|
||||
"$ref": "#/$defs/Document",
|
||||
"$defs": {
|
||||
"AlpmDbEntry": {
|
||||
@ -729,6 +729,72 @@
|
||||
],
|
||||
"description": "Coordinates contains the minimal information needed to describe how to find a file within any possible source object (e.g."
|
||||
},
|
||||
"CpanDistribution": {
|
||||
"properties": {
|
||||
"dist": {
|
||||
"type": "string",
|
||||
"description": "Dist is the distvname install.json recorded (e.g. URI-5.35), the raw value the package name and\nversion were recovered from"
|
||||
},
|
||||
"mainModule": {
|
||||
"type": "string",
|
||||
"description": "MainModule is the module name recovered from the .packlist path (e.g. LWP for libwww-perl).\nEmpty when the name came from install.json, which is authoritative. Set, it means the name is the\ninstaller's NAME and may not be the distribution name."
|
||||
},
|
||||
"author": {
|
||||
"type": "string",
|
||||
"description": "Author is the PAUSE ID of the distribution author (e.g. OALDERS), parsed out of Path"
|
||||
},
|
||||
"path": {
|
||||
"type": "string",
|
||||
"description": "Path is the raw PAUSE path of the release archive (e.g. O/OA/OALDERS/URI-5.35.tar.gz)"
|
||||
},
|
||||
"modules": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/CpanModule"
|
||||
},
|
||||
"type": "array",
|
||||
"description": "Modules are the modules this distribution provides, sorted by name"
|
||||
},
|
||||
"files": {
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"type": "array",
|
||||
"description": "Files are the paths the .packlist recorded as installed by this distribution, kept in the\npacklist's own order and unfiltered: a path the packlist claims and the filesystem lacks means the\nfile was removed or overwritten out from under the installer, which is signal rather than noise"
|
||||
}
|
||||
},
|
||||
"type": "object",
|
||||
"description": "CpanDistribution describes a CPAN distribution installed on disk, as recorded by a CPAN client's install.json and by the .packlist an installer wrote."
|
||||
},
|
||||
"CpanModule": {
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"description": "Name is the module name as it would be used in a perl `use` statement (e.g. URI::Escape)"
|
||||
},
|
||||
"version": {
|
||||
"type": "string",
|
||||
"description": "Version is the module version, which may differ from the distribution version"
|
||||
}
|
||||
},
|
||||
"type": "object",
|
||||
"required": [
|
||||
"name"
|
||||
],
|
||||
"description": "CpanModule is a single perl module provided by a CPAN distribution."
|
||||
},
|
||||
"CpanUnpackedRelease": {
|
||||
"properties": {
|
||||
"modules": {
|
||||
"items": {
|
||||
"$ref": "#/$defs/CpanModule"
|
||||
},
|
||||
"type": "array",
|
||||
"description": "Modules are the modules the release declares it provides, sorted by name"
|
||||
}
|
||||
},
|
||||
"type": "object",
|
||||
"description": "CpanUnpackedRelease describes an unpacked CPAN release found on disk, read from the release's own META.json or META.yml."
|
||||
},
|
||||
"DartPubspec": {
|
||||
"properties": {
|
||||
"homepage": {
|
||||
@ -2772,6 +2838,12 @@
|
||||
{
|
||||
"$ref": "#/$defs/CondaMetadataEntry"
|
||||
},
|
||||
{
|
||||
"$ref": "#/$defs/CpanDistribution"
|
||||
},
|
||||
{
|
||||
"$ref": "#/$defs/CpanUnpackedRelease"
|
||||
},
|
||||
{
|
||||
"$ref": "#/$defs/DartPubspec"
|
||||
},
|
||||
|
||||
@ -20,6 +20,8 @@ func Test_OriginatorSupplier(t *testing.T) {
|
||||
pkg.ConanfileEntry{},
|
||||
pkg.ConaninfoEntry{},
|
||||
pkg.CondaMetaPackage{},
|
||||
pkg.CpanDistribution{}, // the author is a PAUSE ID, not a person or organization name
|
||||
pkg.CpanUnpackedRelease{},
|
||||
pkg.DartPubspecLockEntry{},
|
||||
pkg.DartPubspec{},
|
||||
pkg.DotnetDepsEntry{},
|
||||
|
||||
@ -66,6 +66,8 @@ func SourceInfo(p pkg.Package) string {
|
||||
answer = "acquired package info from nix store path"
|
||||
case pkg.Rpkg:
|
||||
answer = "acquired package info from R-package DESCRIPTION file"
|
||||
case pkg.CpanPkg:
|
||||
answer = "acquired package info from CPAN distribution metadata"
|
||||
case pkg.LuaRocksPkg:
|
||||
answer = "acquired package info from Rockspec package file"
|
||||
case pkg.SwiftPkg:
|
||||
|
||||
@ -287,6 +287,14 @@ func Test_SourceInfo(t *testing.T) {
|
||||
"acquired package info from R-package DESCRIPTION file",
|
||||
},
|
||||
},
|
||||
{
|
||||
input: pkg.Package{
|
||||
Type: pkg.CpanPkg,
|
||||
},
|
||||
expected: []string{
|
||||
"acquired package info from CPAN distribution metadata",
|
||||
},
|
||||
},
|
||||
{
|
||||
input: pkg.Package{
|
||||
Type: pkg.LuaRocksPkg,
|
||||
|
||||
@ -184,8 +184,13 @@ func FromPackageAttributes(p pkg.Package) []cpe.CPE {
|
||||
}
|
||||
|
||||
func candidateTargetSw(p pkg.Package) []string {
|
||||
if p.Type == pkg.WordpressPluginPkg {
|
||||
switch {
|
||||
case p.Type == pkg.WordpressPluginPkg:
|
||||
return []string{"wordpress"}
|
||||
case p.Language == pkg.Perl:
|
||||
// NVD records CPAN distributions with perl in target_sw, e.g.
|
||||
// cpe:2.3:a:mojolicious:mojolicious:*:*:*:*:*:perl:*:*
|
||||
return []string{"perl"}
|
||||
}
|
||||
return []string{cpe.Any}
|
||||
}
|
||||
|
||||
@ -777,6 +777,25 @@ func TestGeneratePackageCPEs(t *testing.T) {
|
||||
"cpe:2.3:a:wow_estore:wp_coder:2.5.1:*:*:*:*:wordpress:*:*",
|
||||
},
|
||||
},
|
||||
{
|
||||
// NVD records CPAN distributions with perl in target_sw
|
||||
name: "cpan distribution",
|
||||
p: pkg.Package{
|
||||
Name: "Mojolicious",
|
||||
Version: "9.10",
|
||||
Type: pkg.CpanPkg,
|
||||
Language: pkg.Perl,
|
||||
Metadata: pkg.CpanDistribution{
|
||||
Author: "SRI",
|
||||
Path: "S/SR/SRI/Mojolicious-9.10.tar.gz",
|
||||
},
|
||||
},
|
||||
expected: []string{
|
||||
// note: the distribution name keeps its casing here, as it does for every other ecosystem.
|
||||
// NVD spells the same CPE lowercase (cpe:2.3:a:mojolicious:mojolicious:*:*:*:*:*:perl:*:*)
|
||||
"cpe:2.3:a:Mojolicious:Mojolicious:9.10:*:*:*:*:perl:*:*",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "dotnet deps.json",
|
||||
p: pkg.Package{
|
||||
|
||||
128
syft/pkg/cataloger/perl/capabilities.yaml
Normal file
128
syft/pkg/cataloger/perl/capabilities.yaml
Normal file
@ -0,0 +1,128 @@
|
||||
# Cataloger capabilities. See ../README.md for documentation.
|
||||
|
||||
catalogers:
|
||||
- ecosystem: perl # MANUAL
|
||||
name: perl-cpan-meta-cataloger # AUTO-GENERATED
|
||||
type: generic # AUTO-GENERATED
|
||||
source: # AUTO-GENERATED
|
||||
file: syft/pkg/cataloger/perl/cataloger.go
|
||||
function: NewCpanMetaCataloger
|
||||
selectors: # AUTO-GENERATED
|
||||
- cpan
|
||||
- declared
|
||||
- directory
|
||||
- language
|
||||
- package
|
||||
- perl
|
||||
parsers: # AUTO-GENERATED structure
|
||||
- function: parseReleaseMeta
|
||||
detector: # AUTO-GENERATED
|
||||
method: glob # AUTO-GENERATED
|
||||
criteria: # AUTO-GENERATED
|
||||
- '**/META.json'
|
||||
- '**/META.yml'
|
||||
metadata_types: # AUTO-GENERATED
|
||||
- pkg.CpanUnpackedRelease
|
||||
package_types: # AUTO-GENERATED
|
||||
- cpan
|
||||
purl_types: # AUTO-GENERATED
|
||||
- cpan
|
||||
json_schema_types: # AUTO-GENERATED
|
||||
- CpanUnpackedRelease
|
||||
capabilities: # MANUAL - preserved across regeneration
|
||||
- name: license
|
||||
default: true
|
||||
comment: the x_spdx_expression or license field of an unpacked release's own META.json or META.yml
|
||||
- name: dependency.depth
|
||||
default: []
|
||||
comment: release meta prereqs are not resolved; only installed evidence produces edges
|
||||
- name: dependency.edges
|
||||
default: ""
|
||||
- name: dependency.kinds
|
||||
default: []
|
||||
- name: package_manager.files.listing
|
||||
default: false
|
||||
- name: package_manager.files.digests
|
||||
default: false
|
||||
- name: package_manager.package_integrity_hash
|
||||
default: false
|
||||
- ecosystem: perl # MANUAL
|
||||
name: perl-cpan-installed-cataloger # AUTO-GENERATED
|
||||
type: generic # AUTO-GENERATED
|
||||
source: # AUTO-GENERATED
|
||||
file: syft/pkg/cataloger/perl/cataloger.go
|
||||
function: NewCpanInstalledCataloger
|
||||
selectors: # AUTO-GENERATED
|
||||
- cpan
|
||||
- directory
|
||||
- image
|
||||
- installed
|
||||
- language
|
||||
- package
|
||||
- perl
|
||||
parsers: # AUTO-GENERATED structure
|
||||
- function: parsePacklist
|
||||
detector: # AUTO-GENERATED
|
||||
method: glob # AUTO-GENERATED
|
||||
criteria: # AUTO-GENERATED
|
||||
- '**/auto/**/.packlist'
|
||||
metadata_types: # AUTO-GENERATED
|
||||
- pkg.CpanDistribution
|
||||
package_types: # AUTO-GENERATED
|
||||
- cpan
|
||||
purl_types: # AUTO-GENERATED
|
||||
- cpan
|
||||
json_schema_types: # AUTO-GENERATED
|
||||
- CpanDistribution
|
||||
capabilities: # MANUAL - preserved across regeneration
|
||||
- name: license
|
||||
default: false
|
||||
comment: a packlist carries no metadata beyond the file list
|
||||
- name: dependency.depth
|
||||
default: []
|
||||
- name: dependency.edges
|
||||
default: ""
|
||||
- name: dependency.kinds
|
||||
default: []
|
||||
- name: package_manager.files.listing
|
||||
default: true
|
||||
comment: the packlist is a file list, and it is surfaced through OwnedFiles so ownership relationships are emitted
|
||||
- name: package_manager.files.digests
|
||||
default: false
|
||||
- name: package_manager.package_integrity_hash
|
||||
default: false
|
||||
- function: parseInstallJSON
|
||||
detector: # AUTO-GENERATED
|
||||
method: glob # AUTO-GENERATED
|
||||
criteria: # AUTO-GENERATED
|
||||
- '**/.meta/*/install.json'
|
||||
metadata_types: # AUTO-GENERATED
|
||||
- pkg.CpanDistribution
|
||||
package_types: # AUTO-GENERATED
|
||||
- cpan
|
||||
purl_types: # AUTO-GENERATED
|
||||
- cpan
|
||||
json_schema_types: # AUTO-GENERATED
|
||||
- CpanDistribution
|
||||
capabilities: # MANUAL - preserved across regeneration
|
||||
- name: license
|
||||
default: true
|
||||
comment: the x_spdx_expression or license field of the MYMETA.json written beside install.json
|
||||
- name: dependency.depth
|
||||
default:
|
||||
- direct
|
||||
- indirect
|
||||
comment: prereqs are resolved across every cataloged distribution, so the graph is the full installed closure
|
||||
- name: dependency.edges
|
||||
default: complete
|
||||
- name: dependency.kinds
|
||||
default:
|
||||
- runtime
|
||||
- build
|
||||
comment: develop and test prereqs describe how the distribution was authored, not what a deployed copy needs
|
||||
- name: package_manager.files.listing
|
||||
default: false
|
||||
- name: package_manager.files.digests
|
||||
default: false
|
||||
- name: package_manager.package_integrity_hash
|
||||
default: false
|
||||
62
syft/pkg/cataloger/perl/cataloger.go
Normal file
62
syft/pkg/cataloger/perl/cataloger.go
Normal file
@ -0,0 +1,62 @@
|
||||
// Package perl provides Cataloger implementations relating to CPAN distributions within the Perl
|
||||
// language ecosystem.
|
||||
//
|
||||
// Only distribution-level identity is emitted. Three evidence sources are read: install.json under a
|
||||
// .meta directory, .packlist paired with perllocal.pod, and an unpacked release's own META.json or
|
||||
// META.yml alongside a MANIFEST.
|
||||
//
|
||||
// The coverage boundary that follows from that:
|
||||
//
|
||||
// - CPAN-client installs are cataloged wherever they live, including local-lib trees such as an
|
||||
// application's local/lib/perl5. cpanm, cpm, and carton write install.json only when the
|
||||
// distribution was resolved from a mirror; a local directory or tarball install (cpanm .,
|
||||
// cpanm ./Foo-1.0.tar.gz) writes none, and those fall back to the packlist pass. CPAN.pm never
|
||||
// writes install.json at all.
|
||||
// - A packlist is written by the ExtUtils::MakeMaker and Module::Build install targets unless
|
||||
// suppressed with NO_PACKLIST, which is what distro packagers set.
|
||||
// - A packlist-derived name is the installer's NAME, which is usually but not always the
|
||||
// distribution name: libwww-perl installs to auto/LWP/.packlist. Resolving those to
|
||||
// distributions is left to the vulnerability data, which carries the mapping.
|
||||
// - Perl modules installed from distro packages are not cataloged here. They carry no CPAN-native
|
||||
// metadata, since packagers suppress it, and they are already reported by the deb, rpm, and apk
|
||||
// catalogers.
|
||||
// - Core and dual-life distributions bundled with the interpreter are not cataloged. No on-disk
|
||||
// artifact carries their versions. The interpreter itself is reported separately.
|
||||
// - Build leftovers under ~/.cpanm/work and ~/.cpan/build are skipped. They are unpacked release
|
||||
// tarballs, MANIFEST included, but they describe a copy that is already installed elsewhere in
|
||||
// the same tree.
|
||||
// - Vendored .pm trees, App::FatPacker output, and PAR archives are invisible. What they carry is
|
||||
// module identity, and there is no offline map from a module to the distribution that shipped
|
||||
// it. Fatpacked output does embed the .pm sources verbatim, $VERSION lines included, so the
|
||||
// missing fact is the distribution rather than the version.
|
||||
// - A cpanfile or cpanfile.snapshot is not parsed yet. The snapshot is a resolved lockfile and
|
||||
// the only evidence in a source checkout or CI workspace, where local/ does not exist, so it
|
||||
// is deferred rather than out of scope.
|
||||
package perl
|
||||
|
||||
import (
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/generic"
|
||||
)
|
||||
|
||||
// NewCpanInstalledCataloger returns a cataloger for CPAN distributions installed on disk.
|
||||
// Globs are deliberately unanchored so local-lib trees (carton, cpanm -L) are found wherever they live.
|
||||
func NewCpanInstalledCataloger() pkg.Cataloger {
|
||||
return generic.NewCataloger("perl-cpan-installed-cataloger").
|
||||
WithParserByGlobs(parseInstallJSON, "**/.meta/*/install.json").
|
||||
WithParserByGlobs(parsePacklist, "**/auto/**/.packlist").
|
||||
// registration order is execution order, and resolvePacklistVersions has to precede
|
||||
// mergeDistributions: merging pairs a packlist package to its install.json package only when the
|
||||
// versions agree or the packlist version is absent, so filling versions in afterwards splits every
|
||||
// distribution whose two evidence kinds disagree into two packages
|
||||
WithResolvingProcessors(resolvePacklistVersions).
|
||||
WithProcessors(mergeDistributions).
|
||||
WithResolvingProcessors(resolveDependencies)
|
||||
}
|
||||
|
||||
// NewCpanMetaCataloger returns a cataloger for unpacked CPAN releases: release tarballs and vendored
|
||||
// distribution trees. CPAN client build leftovers are excluded, see isBuildLeftover.
|
||||
func NewCpanMetaCataloger() pkg.Cataloger {
|
||||
return generic.NewCataloger("perl-cpan-meta-cataloger").
|
||||
WithParserByGlobs(parseReleaseMeta, "**/META.json", "**/META.yml")
|
||||
}
|
||||
567
syft/pkg/cataloger/perl/cataloger_test.go
Normal file
567
syft/pkg/cataloger/perl/cataloger_test.go
Normal file
@ -0,0 +1,567 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"github.com/anchore/stereoscope/pkg/imagetest"
|
||||
"github.com/anchore/syft/internal/packagemetadata"
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/anchore/syft/syft/file"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
|
||||
"github.com/anchore/syft/syft/source"
|
||||
"github.com/anchore/syft/syft/source/stereoscopesource"
|
||||
)
|
||||
|
||||
// every fixture here is an image built by a Dockerfile under testdata/, one per evidence tier. The
|
||||
// installs are performed by the real clients inside the image, so the on-disk layout, the file
|
||||
// contents and the versions are whatever the toolchain actually produced.
|
||||
//
|
||||
// an image is only how the fixture is delivered. perl-cpan-meta-cataloger is registered declared and
|
||||
// directory, never image, so through the CLI it never runs against a container image at all; the tests
|
||||
// below that drive it through an image resolver are parser tests, and the file tree they read is what
|
||||
// they are actually asserting on.
|
||||
|
||||
// TestCpanInstalledCataloger_mirrorInstalls covers the only case that produces an install.json:
|
||||
// cpanm, cpm and carton resolving distributions from a mirror. The image installs libwww-perl and its
|
||||
// whole dependency closure with cpanm, one distribution with NO_PACKLIST, one with carton into an
|
||||
// application's local/, and one with cpm into an unrelated prefix.
|
||||
func TestCpanInstalledCataloger_mirrorInstalls(t *testing.T) {
|
||||
pkgtest.NewCatalogTester().
|
||||
WithImageResolver(t, "image-cpan-mirror-installs").
|
||||
ExpectsPackageStrings([]string{
|
||||
// preinstalled in the base image by its own EUMM installs: a packlist and a perllocal.pod
|
||||
// stanza, no .meta, which is why these three have no author qualifier on their purl
|
||||
"App-cpanminus @ 1.7049 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/App/cpanminus/.packlist)",
|
||||
"IO-Socket-SSL @ 2.099 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/IO/Socket/SSL/.packlist)",
|
||||
"Net-SSLeay @ 1.96 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Net/SSLeay/.packlist)",
|
||||
|
||||
// cpm, into a prefix that is not under /usr, which is the whole reason the globs are unanchored
|
||||
"Capture-Tiny @ 0.48 (/srv/api/local/lib/perl5/x86_64-linux-gnu/.meta/Capture-Tiny-0.48/install.json)",
|
||||
|
||||
// carton, into local/lib/perl5 beside the cpanfile
|
||||
"Text-CSV @ 2.06 (/app/local/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json)",
|
||||
|
||||
// installed with NO_PACKLIST and NO_PERLLOCAL, the way distro packagers do it, so
|
||||
// install.json is the only evidence there is
|
||||
"JSON-PP @ 4.16 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/JSON-PP-4.16/install.json)",
|
||||
|
||||
// libwww-perl is reported under its distribution name even though install.json's `name` field
|
||||
// says LWP and its packlist is auto/LWP/.packlist. There is deliberately no LWP package here:
|
||||
// reporting one is the bug this image exists to catch.
|
||||
"libwww-perl @ 6.83 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/libwww-perl-6.83/install.json)",
|
||||
|
||||
// the rest of the closure cpanm pulled in for libwww-perl
|
||||
"Clone @ 0.50 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Clone-0.50/install.json)",
|
||||
"Encode-Locale @ 1.05 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Encode-Locale-1.05/install.json)",
|
||||
"File-Listing @ 6.16 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/File-Listing-6.16/install.json)",
|
||||
"HTML-Parser @ 3.85 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTML-Parser-3.85/install.json)",
|
||||
"HTML-Tagset @ 3.24 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTML-Tagset-3.24/install.json)",
|
||||
"HTTP-Cookies @ 6.12 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTTP-Cookies-6.12/install.json)",
|
||||
"HTTP-Date @ 6.08 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTTP-Date-6.08/install.json)",
|
||||
"HTTP-Message @ 7.04 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTTP-Message-7.04/install.json)",
|
||||
"HTTP-Negotiate @ 6.01 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTTP-Negotiate-6.01/install.json)",
|
||||
"IO-HTML @ 1.004 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/IO-HTML-1.004/install.json)",
|
||||
"LWP-MediaTypes @ 6.04 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/LWP-MediaTypes-6.04/install.json)",
|
||||
"MIME-Base32 @ 1.303 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/MIME-Base32-1.303/install.json)",
|
||||
"Net-HTTP @ 6.24 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Net-HTTP-6.24/install.json)",
|
||||
"TimeDate @ 2.35 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/TimeDate-2.35/install.json)",
|
||||
"Try-Tiny @ 0.32 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Try-Tiny-0.32/install.json)",
|
||||
"URI @ 5.35 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/URI-5.35/install.json)",
|
||||
"WWW-RobotRules @ 6.03 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/WWW-RobotRules-6.03/install.json)",
|
||||
|
||||
// note: no package for the core perl packlist, which has no auto/ segment, and none for the
|
||||
// distributions perllocal.pod names but that have no packlist of their own
|
||||
}).
|
||||
TestCataloger(t, NewCpanInstalledCataloger())
|
||||
}
|
||||
|
||||
// TestCpanInstalledCataloger_prereqRelationships pins the dependency graph read out of the
|
||||
// MYMETA.json files cpanm left beside each install.json. Prereqs name modules, so each one has to be
|
||||
// resolved through the provides map of the cataloged set; a prereq that resolves to nothing (a core
|
||||
// module such as Carp or strict, or a distribution that is simply not installed) is dropped.
|
||||
//
|
||||
// Packages are stringed without their locations here, because the full form makes each relationship
|
||||
// unreadable.
|
||||
func TestCpanInstalledCataloger_prereqRelationships(t *testing.T) {
|
||||
pkgtest.NewCatalogTester().
|
||||
WithImageResolver(t, "image-cpan-mirror-installs").
|
||||
WithPackageStringer(func(p pkg.Package) string { return p.Name + " @ " + p.Version }).
|
||||
ExpectsRelationshipStrings([]string{
|
||||
"Clone @ 0.50 [dependency-of] HTTP-Message @ 7.04",
|
||||
"Encode-Locale @ 1.05 [dependency-of] HTTP-Message @ 7.04",
|
||||
"Encode-Locale @ 1.05 [dependency-of] libwww-perl @ 6.83",
|
||||
"File-Listing @ 6.16 [dependency-of] libwww-perl @ 6.83",
|
||||
"HTML-Parser @ 3.85 [dependency-of] libwww-perl @ 6.83",
|
||||
"HTML-Tagset @ 3.24 [dependency-of] HTML-Parser @ 3.85",
|
||||
"HTTP-Cookies @ 6.12 [dependency-of] libwww-perl @ 6.83",
|
||||
"HTTP-Date @ 6.08 [dependency-of] File-Listing @ 6.16",
|
||||
"HTTP-Date @ 6.08 [dependency-of] HTTP-Cookies @ 6.12",
|
||||
"HTTP-Date @ 6.08 [dependency-of] HTTP-Message @ 7.04",
|
||||
"HTTP-Date @ 6.08 [dependency-of] libwww-perl @ 6.83",
|
||||
"HTTP-Message @ 7.04 [dependency-of] HTML-Parser @ 3.85",
|
||||
"HTTP-Message @ 7.04 [dependency-of] HTTP-Cookies @ 6.12",
|
||||
"HTTP-Message @ 7.04 [dependency-of] HTTP-Negotiate @ 6.01",
|
||||
"HTTP-Message @ 7.04 [dependency-of] libwww-perl @ 6.83",
|
||||
"HTTP-Negotiate @ 6.01 [dependency-of] libwww-perl @ 6.83",
|
||||
"IO-HTML @ 1.004 [dependency-of] HTTP-Message @ 7.04",
|
||||
"LWP-MediaTypes @ 6.04 [dependency-of] HTTP-Message @ 7.04",
|
||||
"LWP-MediaTypes @ 6.04 [dependency-of] libwww-perl @ 6.83",
|
||||
// URI 5.35 needs MIME::Base32 to build, which is why cpanm installed it at all
|
||||
"MIME-Base32 @ 1.303 [dependency-of] URI @ 5.35",
|
||||
"Net-HTTP @ 6.24 [dependency-of] libwww-perl @ 6.83",
|
||||
// the module TimeDate provides is Date::Format, so this only resolves through the provides map
|
||||
"TimeDate @ 2.35 [dependency-of] HTTP-Date @ 6.08",
|
||||
"Try-Tiny @ 0.32 [dependency-of] libwww-perl @ 6.83",
|
||||
"URI @ 5.35 [dependency-of] HTML-Parser @ 3.85",
|
||||
"URI @ 5.35 [dependency-of] HTTP-Message @ 7.04",
|
||||
"URI @ 5.35 [dependency-of] Net-HTTP @ 6.24",
|
||||
"URI @ 5.35 [dependency-of] WWW-RobotRules @ 6.03",
|
||||
"URI @ 5.35 [dependency-of] libwww-perl @ 6.83",
|
||||
"WWW-RobotRules @ 6.03 [dependency-of] libwww-perl @ 6.83",
|
||||
}).
|
||||
TestCataloger(t, NewCpanInstalledCataloger())
|
||||
}
|
||||
|
||||
func TestCpanInstalledCataloger_mirrorInstallEvidence(t *testing.T) {
|
||||
pkgtest.NewCatalogTester().
|
||||
WithImageResolver(t, "image-cpan-mirror-installs").
|
||||
ExpectsAssertion(func(t *testing.T, pkgs []pkg.Package, _ []artifact.Relationship) {
|
||||
byName := make(map[string]pkg.Package)
|
||||
for _, p := range pkgs {
|
||||
byName[p.Name] = p
|
||||
}
|
||||
|
||||
uri := byName["URI"]
|
||||
require.NotEmpty(t, uri.Name)
|
||||
assert.Equal(t, "pkg:cpan/URI@5.35?author=OALDERS", uri.PURL)
|
||||
assert.Equal(t, pkg.Perl, uri.Language)
|
||||
assert.Equal(t, pkg.CpanPkg, uri.Type)
|
||||
|
||||
// licenses come from the MYMETA.json beside install.json, and are not duplicated into
|
||||
// metadata. URI's own x_spdx_expression is what reaches the package, not the perl_5 token
|
||||
// beside it: the expression is already SPDX, where perl_5 is a CPAN::Meta shortname that
|
||||
// would have to be translated.
|
||||
require.Len(t, uri.Licenses.ToSlice(), 1)
|
||||
assert.Equal(t, "Artistic-1.0-Perl OR GPL-1.0-or-later", uri.Licenses.ToSlice()[0].Value)
|
||||
|
||||
// install.json, its sibling MYMETA.json, the packlist and the perllocal.pod that resolved
|
||||
// the packlist's version are all evidence. The main .pm is not: perllocal.pod answered
|
||||
// first, so it was never scraped.
|
||||
assert.ElementsMatch(t, []string{
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/URI-5.35/install.json",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/URI-5.35/MYMETA.json",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/URI/.packlist",
|
||||
"/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod",
|
||||
}, realPaths(uri))
|
||||
|
||||
// a distribution with no x_spdx_expression falls back to the CPAN::Meta license token
|
||||
require.Len(t, byName["Capture-Tiny"].Licenses.ToSlice(), 1)
|
||||
assert.Equal(t, "apache_2_0", byName["Capture-Tiny"].Licenses.ToSlice()[0].Value)
|
||||
|
||||
// a packlist-derived package carries no author, so the purl gets no qualifier
|
||||
assert.Equal(t, "pkg:cpan/IO-Socket-SSL@2.099", byName["IO-Socket-SSL"].PURL)
|
||||
|
||||
// libwww-perl's packlist is auto/LWP/.packlist, so it shares no (name, version) key with
|
||||
// the distribution and is paired through the provides module map instead
|
||||
assert.NotContains(t, byName, "LWP", "the module-named packlist must not become its own package")
|
||||
|
||||
lwp := byName["libwww-perl"]
|
||||
require.NotEmpty(t, lwp.Name)
|
||||
assert.Equal(t, "pkg:cpan/libwww-perl@6.83?author=OALDERS", lwp.PURL)
|
||||
assert.ElementsMatch(t, []string{
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/libwww-perl-6.83/install.json",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/libwww-perl-6.83/MYMETA.json",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/LWP/.packlist",
|
||||
"/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod",
|
||||
}, realPaths(lwp))
|
||||
|
||||
// carton's packlist points at the application-local copy, not the site_perl one
|
||||
assert.ElementsMatch(t, []string{
|
||||
"/app/local/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json",
|
||||
"/app/local/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/MYMETA.json",
|
||||
"/app/local/lib/perl5/x86_64-linux-gnu/auto/Text/CSV/.packlist",
|
||||
"/app/local/lib/perl5/x86_64-linux-gnu/perllocal.pod",
|
||||
}, realPaths(byName["Text-CSV"]))
|
||||
|
||||
// name provenance: Dist records the distvname install.json was read from, MainModule
|
||||
// records that the name was inferred from an auto/ path instead. Never both, and a merged
|
||||
// package keeps install.json's.
|
||||
metadata := func(name string) pkg.CpanDistribution {
|
||||
md, ok := byName[name].Metadata.(pkg.CpanDistribution)
|
||||
require.True(t, ok, name)
|
||||
return md
|
||||
}
|
||||
assert.Equal(t, "libwww-perl-6.83", metadata("libwww-perl").Dist)
|
||||
assert.Empty(t, metadata("libwww-perl").MainModule)
|
||||
assert.Equal(t, "IO::Socket::SSL", metadata("IO-Socket-SSL").MainModule)
|
||||
assert.Empty(t, metadata("IO-Socket-SSL").Dist)
|
||||
|
||||
// the paths are reported as the packlist recorded them, not filtered to the ones still on
|
||||
// disk. IO/Socket/SSL/Utils.pm is deleted in the image on purpose: a path the packlist
|
||||
// claims and the filesystem lacks means the file was removed or overwritten, which is
|
||||
// exactly what an SBOM consumer wants to see.
|
||||
assert.Equal(t, []string{
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL.pm",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL.pod",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL/Intercept.pm",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL/PublicSuffix.pm",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL/Utils.pm",
|
||||
}, metadata("IO-Socket-SSL").OwnedFiles())
|
||||
|
||||
// the merge has to carry the packlist's file list onto the install.json package that won,
|
||||
// or a merged distribution would own nothing
|
||||
assert.Contains(t, metadata("libwww-perl").OwnedFiles(), "/usr/local/lib/perl5/site_perl/5.40.4/LWP.pm")
|
||||
assert.Contains(t, metadata("libwww-perl").OwnedFiles(), "/usr/local/bin/lwp-request")
|
||||
|
||||
// NO_PACKLIST leaves an install.json and nothing else, so there is no file list at all
|
||||
assert.Empty(t, metadata("JSON-PP").OwnedFiles())
|
||||
}).
|
||||
TestCataloger(t, NewCpanInstalledCataloger())
|
||||
}
|
||||
|
||||
// TestCpanInstalledCataloger_packlistOnly covers the installs that write no install.json, which is
|
||||
// the majority of real images and where the version has to come from perllocal.pod or the installed
|
||||
// .pm. The image runs ExtUtils::MakeMaker directly, cpanm against local tarballs, and CPAN.pm; the
|
||||
// Dockerfile asserts that none of them left a .meta directory behind.
|
||||
func TestCpanInstalledCataloger_packlistOnly(t *testing.T) {
|
||||
pkgtest.NewCatalogTester().
|
||||
WithImageResolver(t, "image-cpan-packlist-only").
|
||||
ExpectsPackageStrings([]string{
|
||||
"App-cpanminus @ 1.7049 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/App/cpanminus/.packlist)",
|
||||
// Encode's $VERSION is declared bare and computed with sprintf from an RCS Revision keyword
|
||||
// inside a BEGIN block, so nothing static can read it. This version exists only because
|
||||
// perllocal.pod records what EUMM evaluated at build time.
|
||||
"Encode @ 3.24 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Encode/.packlist)",
|
||||
"IO-Socket-SSL @ 2.099 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/IO/Socket/SSL/.packlist)",
|
||||
// installed with NO_PERLLOCAL, so this one had to be scraped out of JSON/PP.pm
|
||||
"JSON-PP @ 4.16 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/JSON/PP/.packlist)",
|
||||
// libwww-perl's EUMM NAME is LWP, so an installed tree only ever yields "LWP". Resolving
|
||||
// that back to the distribution is left to the vulnerability data, which carries the map.
|
||||
"LWP @ 5.836 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/LWP/.packlist)",
|
||||
"Net-SSLeay @ 1.96 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Net/SSLeay/.packlist)",
|
||||
// installed at 2.02 and then upgraded to 2.06. perllocal.pod is append-only, so both
|
||||
// stanzas are still there and the later one has to win.
|
||||
"Text-CSV @ 2.06 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Text/CSV/.packlist)",
|
||||
}).
|
||||
ExpectsAssertion(func(t *testing.T, pkgs []pkg.Package, _ []artifact.Relationship) {
|
||||
byName := make(map[string]pkg.Package)
|
||||
for _, p := range pkgs {
|
||||
byName[p.Name] = p
|
||||
}
|
||||
|
||||
// the perllocal.pod that supplied the version is recorded as supporting evidence, and no
|
||||
// .pm was read at all
|
||||
assert.ElementsMatch(t, []string{
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Encode/.packlist",
|
||||
"/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod",
|
||||
}, realPaths(byName["Encode"]))
|
||||
|
||||
// NO_PERLLOCAL wrote no stanza, so the .pm named by the packlist is the evidence instead
|
||||
assert.ElementsMatch(t, []string{
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/JSON/PP/.packlist",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/JSON/PP.pm",
|
||||
}, realPaths(byName["JSON-PP"]))
|
||||
|
||||
// no install.json anywhere, so no distribution has an author and no purl has a qualifier
|
||||
for _, p := range pkgs {
|
||||
assert.NotContains(t, p.PURL, "author=", p.Name)
|
||||
assert.Empty(t, p.Licenses.ToSlice(), p.Name)
|
||||
}
|
||||
}).
|
||||
TestCataloger(t, NewCpanInstalledCataloger())
|
||||
}
|
||||
|
||||
// TestCpanInstalledCataloger_twoInstallTrees covers an image carrying two perl lib trees, each with
|
||||
// its own perllocal.pod naming the same module at a different version. Picking the stanza from the
|
||||
// wrong tree would be silent, so both are pinned.
|
||||
//
|
||||
// The same image installs Text-CSV 2.06 from a mirror into both trees, so both hold an install.json
|
||||
// agreeing on name and version. Those are two installed things and stay two packages: the tree is
|
||||
// part of what pairs evidence, not the name and version alone.
|
||||
func TestCpanInstalledCataloger_twoInstallTrees(t *testing.T) {
|
||||
pkgtest.NewCatalogTester().
|
||||
WithImageResolver(t, "image-cpan-two-install-trees").
|
||||
ExpectsPackageStrings([]string{
|
||||
"App-cpanminus @ 1.7049 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/App/cpanminus/.packlist)",
|
||||
"IO-Socket-SSL @ 2.099 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/IO/Socket/SSL/.packlist)",
|
||||
"JSON-PP @ 2.27300 (/opt/app/lib/perl5/x86_64-linux-gnu/auto/JSON/PP/.packlist)",
|
||||
"JSON-PP @ 4.16 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/JSON/PP/.packlist)",
|
||||
"Net-SSLeay @ 1.96 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Net/SSLeay/.packlist)",
|
||||
"Text-CSV @ 2.06 (/opt/app/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json)",
|
||||
"Text-CSV @ 2.06 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json)",
|
||||
}).
|
||||
ExpectsAssertion(func(t *testing.T, pkgs []pkg.Package, _ []artifact.Relationship) {
|
||||
byVersion := make(map[string]pkg.Package)
|
||||
var textCSV []pkg.Package
|
||||
for _, p := range pkgs {
|
||||
switch p.Name {
|
||||
case "JSON-PP":
|
||||
byVersion[p.Version] = p
|
||||
case "Text-CSV":
|
||||
textCSV = append(textCSV, p)
|
||||
}
|
||||
}
|
||||
require.Len(t, byVersion, 2)
|
||||
|
||||
// each tree's Text-CSV carries only its own tree's evidence, and within a tree the
|
||||
// install.json and the packlist still merged into the one package
|
||||
require.Len(t, textCSV, 2)
|
||||
byTree := make(map[string][]string)
|
||||
for _, p := range textCSV {
|
||||
byTree[installTree(p)] = realPaths(p)
|
||||
}
|
||||
require.Len(t, byTree, 2)
|
||||
assert.ElementsMatch(t, []string{
|
||||
"/opt/app/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json",
|
||||
"/opt/app/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/MYMETA.json",
|
||||
"/opt/app/lib/perl5/x86_64-linux-gnu/auto/Text/CSV/.packlist",
|
||||
"/opt/app/lib/perl5/x86_64-linux-gnu/perllocal.pod",
|
||||
}, byTree["/opt/app/lib/perl5/x86_64-linux-gnu"])
|
||||
assert.ElementsMatch(t, []string{
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Text-CSV-2.06/MYMETA.json",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Text/CSV/.packlist",
|
||||
"/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod",
|
||||
}, byTree["/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu"])
|
||||
|
||||
// each tree's version came from that tree's own perllocal.pod. Both installs are of the
|
||||
// same module, so without the libdir rule one of these would carry the other's version.
|
||||
assert.Contains(t, realPaths(byVersion["2.27300"]), "/opt/app/lib/perl5/x86_64-linux-gnu/perllocal.pod")
|
||||
assert.Contains(t, realPaths(byVersion["4.16"]), "/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod")
|
||||
|
||||
// ExtUtils::Packlist merges an existing packlist for the same module when writing a new
|
||||
// one, so the local-lib install's packlist genuinely claims the site_perl copy too. The
|
||||
// file list is reported as written rather than filtered to the prefix.
|
||||
owned := byVersion["2.27300"].Metadata.(pkg.CpanDistribution).OwnedFiles()
|
||||
assert.Contains(t, owned, "/opt/app/lib/perl5/JSON/PP.pm")
|
||||
assert.Contains(t, owned, "/usr/local/lib/perl5/site_perl/5.40.4/JSON/PP.pm")
|
||||
}).
|
||||
TestCataloger(t, NewCpanInstalledCataloger())
|
||||
}
|
||||
|
||||
// TestCpanMetaCataloger_unpackedReleases covers release tarballs unpacked but never installed, plus a
|
||||
// source repository working tree.
|
||||
func TestCpanMetaCataloger_unpackedReleases(t *testing.T) {
|
||||
pkgtest.NewCatalogTester().
|
||||
WithImageResolver(t, "image-cpan-unpacked-releases").
|
||||
ExpectsPackageStrings([]string{
|
||||
// CPAN Meta Spec 1.4, which is all that roughly 43% of current releases ship. Genuine EUMM
|
||||
// 6.56 output: license is a bare string rather than an array and version is an unquoted
|
||||
// number, both of which the lenient field types absorb.
|
||||
"Text-CSV @ 1.21 (/opt/releases/Text-CSV-1.21/META.yml)",
|
||||
// a Dist::Zilla release, which ships dist.ini inside the tarball and lists it in its own
|
||||
// MANIFEST, so a dist.ini beside a META.json says nothing about whether this is a release.
|
||||
// It ships both meta files and META.json is reported, once.
|
||||
"Try-Tiny @ 0.32 (/opt/releases/Try-Tiny-0.32/META.json)",
|
||||
|
||||
// nothing from /opt/releases/Parallel-Depend-4.10: its META.yml is real EUMM output that no
|
||||
// strict YAML parser accepts, because the distribution's multi-line ABSTRACT was written
|
||||
// straight into `abstract:` with no quoting and no block scalar. That has to skip the
|
||||
// directory, not fail the scan.
|
||||
|
||||
// nothing from /src/libwww-perl either: a committed META.json records the last release and
|
||||
// drifts from the working tree (6.83 there against 6.84 in lib/LWP.pm on the same commit).
|
||||
// The absence of a MANIFEST is what rejects it, since dzil generates one at build time.
|
||||
}).
|
||||
ExpectsAssertion(func(t *testing.T, pkgs []pkg.Package, _ []artifact.Relationship) {
|
||||
byName := make(map[string]pkg.Package)
|
||||
for _, p := range pkgs {
|
||||
byName[p.Name] = p
|
||||
}
|
||||
|
||||
// Try-Tiny's META.json carries an x_spdx_expression, which is preferred over the
|
||||
// license token beside it
|
||||
require.Len(t, byName["Try-Tiny"].Licenses.ToSlice(), 1)
|
||||
assert.Equal(t, "MIT", byName["Try-Tiny"].Licenses.ToSlice()[0].Value)
|
||||
|
||||
// spec 1.4 has no x_spdx_expression, so the bare license string is what is left
|
||||
require.Len(t, byName["Text-CSV"].Licenses.ToSlice(), 1)
|
||||
assert.Equal(t, "perl", byName["Text-CSV"].Licenses.ToSlice()[0].Value)
|
||||
}).
|
||||
TestCataloger(t, NewCpanMetaCataloger())
|
||||
}
|
||||
|
||||
// TestCpanCatalogers_buildLeftoversAndVendoredRelease covers the ordinary state of any tree that
|
||||
// installed distributions and did not clean up after itself: cpanm leaves ~/.cpanm/work and CPAN.pm
|
||||
// leaves ~/.cpan/build, neither by choice of the user.
|
||||
//
|
||||
// Both leftovers genuinely are unpacked release tarballs, MANIFEST included, so the sibling-MANIFEST
|
||||
// gate is working correctly and still admits them. They have to be excluded on the path instead, or
|
||||
// every distribution in such a tree is reported twice: once from its installed evidence and once from
|
||||
// the copy the client unpacked to build it. Both catalogers are registered for directory scans, so
|
||||
// that is where the duplication would be user-visible.
|
||||
//
|
||||
// The same fixture carries a release deliberately unpacked under /opt/vendor, which is the other half
|
||||
// of the path rule and is reported. It is the same distribution and version as the installed copy,
|
||||
// and it produces a second package rather than merging, for the reason
|
||||
// TestCpanCatalogers_vendoredReleaseCannotMerge documents.
|
||||
func TestCpanCatalogers_buildLeftoversAndVendoredRelease(t *testing.T) {
|
||||
found := catalogBoth(t, "image-cpan-build-leftovers")
|
||||
|
||||
var names []string
|
||||
for _, p := range found {
|
||||
names = append(names, p.Name+" @ "+p.Version+" ("+p.FoundBy+")")
|
||||
}
|
||||
|
||||
// ElementsMatch rather than Equal: the two HTML-Tagset packages agree on name and version, so
|
||||
// pkg.Collection.Sorted breaks the tie on package ID, which is a structural hash and not something
|
||||
// worth pinning
|
||||
assert.ElementsMatch(t, []string{
|
||||
"App-cpanminus @ 1.7049 (perl-cpan-installed-cataloger)",
|
||||
"HTML-Tagset @ 3.24 (perl-cpan-installed-cataloger)",
|
||||
"HTML-Tagset @ 3.24 (perl-cpan-meta-cataloger)",
|
||||
"IO-Socket-SSL @ 2.099 (perl-cpan-installed-cataloger)",
|
||||
"MIME-Base32 @ 1.303 (perl-cpan-installed-cataloger)",
|
||||
"Net-SSLeay @ 1.96 (perl-cpan-installed-cataloger)",
|
||||
"URI @ 5.35 (perl-cpan-installed-cataloger)",
|
||||
}, names, "each installed distribution is reported once, nothing comes from the leftover trees, and the vendored copy is reported separately")
|
||||
}
|
||||
|
||||
// TestCpanCatalogers_vendoredReleaseCannotMerge asserts what happens when a release deliberately
|
||||
// unpacked outside any build directory sits beside the installed copy of the same distribution and
|
||||
// version.
|
||||
//
|
||||
// The intuitive expectation is one package carrying both locations as evidence. That cannot hold, and
|
||||
// this test pins the real behavior at two packages rather than asserting the intuition and failing.
|
||||
//
|
||||
// Why: the two packages come from two different catalogers, so nothing inside either cataloger can
|
||||
// pair them. mergeDistributions only ever sees one cataloger's output. The only cross-cataloger merge
|
||||
// syft has is pkg.Collection.add, at syft/pkg/collection.go:119, which merges on artifact.ID equality
|
||||
// alone. artifact.ID is a structural hash of the whole Package (syft/pkg/package.go:58, via
|
||||
// artifact.IDByHash), and Locations is a hashed field: the `hash:"ignore"` tags at
|
||||
// syft/pkg/package.go:30-51 exempt FoundBy, Language, CPEs, and PURL, but not Locations, Licenses, or
|
||||
// Metadata. Two packages whose locations differ therefore cannot have the same ID, and two packages
|
||||
// found at the same location are the same evidence and not this case at all. So "same identity,
|
||||
// merged locations" is unreachable by construction, for any pair of catalogers, not just these two.
|
||||
func TestCpanCatalogers_vendoredReleaseCannotMerge(t *testing.T) {
|
||||
var installed, unpacked pkg.Package
|
||||
for _, p := range catalogBoth(t, "image-cpan-build-leftovers") {
|
||||
if p.Name != "HTML-Tagset" {
|
||||
continue
|
||||
}
|
||||
// the two agree on name and version, so the duplication is visible and consistent
|
||||
assert.Equal(t, "3.24", p.Version)
|
||||
assert.Equal(t, "pkg:cpan/HTML-Tagset@3.24", p.PURL)
|
||||
|
||||
switch p.FoundBy {
|
||||
case "perl-cpan-installed-cataloger":
|
||||
installed = p
|
||||
case "perl-cpan-meta-cataloger":
|
||||
unpacked = p
|
||||
}
|
||||
}
|
||||
require.NotEmpty(t, installed.Name)
|
||||
require.NotEmpty(t, unpacked.Name)
|
||||
|
||||
assert.ElementsMatch(t, []string{
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/HTML/Tagset/.packlist",
|
||||
"/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod",
|
||||
}, realPaths(installed))
|
||||
assert.ElementsMatch(t, []string{
|
||||
"/opt/vendor/HTML-Tagset-3.24/META.json",
|
||||
}, realPaths(unpacked))
|
||||
|
||||
// Locations, Licenses and Metadata are the whole of the difference here, and every other differing
|
||||
// field (FoundBy, PURL, CPEs) is `hash:"ignore"`. Overwriting just those three hashed fields makes
|
||||
// the IDs equal, which is the direct demonstration of what keeps the two packages apart.
|
||||
require.NotEqual(t, installed.ID(), unpacked.ID())
|
||||
assert.Empty(t, installed.Licenses.ToSlice(), "a packlist carries no license")
|
||||
assert.Equal(t, "artistic_2", unpacked.Licenses.ToSlice()[0].Value, "META.json does, so Licenses differs too")
|
||||
|
||||
relocated := unpacked
|
||||
relocated.Locations = installed.Locations
|
||||
relocated.Licenses = installed.Licenses
|
||||
relocated.Metadata = installed.Metadata
|
||||
relocated.SetID()
|
||||
assert.Equal(t, installed.ID(), relocated.ID(), "with the hashed fields equalized the IDs match, and only then")
|
||||
}
|
||||
|
||||
// TestCpanCatalogers_coreDistributionsNotCataloged pins the zero coverage of distributions bundled
|
||||
// inside the interpreter, which the package doc comment states as a gap.
|
||||
//
|
||||
// The image is a perl with no CPAN installs at all: Encode and Storable sit in the core lib tree, the
|
||||
// only packlist is perl's own and has no auto/ segment, and there is no .meta and no distro package.
|
||||
// Nothing on disk carries a core distribution's version.
|
||||
//
|
||||
// Storable.pm does declare a readable $VERSION, so the interesting half is that no package is
|
||||
// invented from it, and neither is one invented from the perllocal.pod stanzas that name the
|
||||
// distributions the base image preinstalled into a site_perl this image does not have. The
|
||||
// interpreter's version is never used as a stopgap either: a distribution name carrying a wrong or
|
||||
// absent version matches either everything or nothing in an advisory range, which is worse than the
|
||||
// gap. Closing this needs a generated Module::CoreList table keyed by perl version.
|
||||
func TestCpanCatalogers_coreDistributionsNotCataloged(t *testing.T) {
|
||||
assert.Empty(t, catalogBoth(t, "image-perl-core-only"))
|
||||
}
|
||||
|
||||
// TestCpanCatalogers_metadataTypes pins the metadata type each cataloger emits along with the JSON
|
||||
// name it serializes as. Both are user-visible API: a consumer decides whether a finding is installed
|
||||
// and loadable or merely unpacked on disk from the type name, and must not have to string match
|
||||
// FoundBy.
|
||||
func TestCpanCatalogers_metadataTypes(t *testing.T) {
|
||||
installed := catalogImage(t, "image-cpan-mirror-installs", NewCpanInstalledCataloger())
|
||||
unpacked := catalogImage(t, "image-cpan-unpacked-releases", NewCpanMetaCataloger())
|
||||
|
||||
require.NotEmpty(t, installed)
|
||||
for _, p := range installed {
|
||||
assert.IsType(t, pkg.CpanDistribution{}, p.Metadata, p.Name)
|
||||
assert.Equal(t, "cpan-distribution", packagemetadata.JSONName(p.Metadata), p.Name)
|
||||
}
|
||||
|
||||
require.NotEmpty(t, unpacked)
|
||||
for _, p := range unpacked {
|
||||
assert.IsType(t, pkg.CpanUnpackedRelease{}, p.Metadata, p.Name)
|
||||
assert.Equal(t, "cpan-unpacked-release", packagemetadata.JSONName(p.Metadata), p.Name)
|
||||
}
|
||||
}
|
||||
|
||||
// imageResolver resolves an image fixture the way pkgtest's WithImageResolver does, for the tests
|
||||
// that need the resolver itself rather than the CatalogTester around it.
|
||||
func imageResolver(t *testing.T, fixture string) file.Resolver {
|
||||
t.Helper()
|
||||
|
||||
img := imagetest.GetFixtureImage(t, "docker-archive", fixture)
|
||||
resolver, err := stereoscopesource.New(img, stereoscopesource.ImageConfig{Reference: fixture}).
|
||||
FileResolver(source.SquashedScope)
|
||||
require.NoError(t, err)
|
||||
|
||||
return resolver
|
||||
}
|
||||
|
||||
func catalogImage(t *testing.T, fixture string, cataloger pkg.Cataloger) []pkg.Package {
|
||||
t.Helper()
|
||||
|
||||
pkgs, _, err := cataloger.Catalog(pkgtest.Context(t), imageResolver(t, fixture))
|
||||
require.NoError(t, err)
|
||||
|
||||
return pkgs
|
||||
}
|
||||
|
||||
// catalogBoth runs both catalogers over one image and aggregates into a pkg.Collection, which is what
|
||||
// the real cataloging pipeline does, so any cross-cataloger merge that production performs happens
|
||||
// here too.
|
||||
func catalogBoth(t *testing.T, fixture string) []pkg.Package {
|
||||
t.Helper()
|
||||
|
||||
resolver := imageResolver(t, fixture)
|
||||
|
||||
collection := pkg.NewCollection()
|
||||
for _, cataloger := range []pkg.Cataloger{NewCpanInstalledCataloger(), NewCpanMetaCataloger()} {
|
||||
pkgs, _, err := cataloger.Catalog(pkgtest.Context(t), resolver)
|
||||
require.NoError(t, err)
|
||||
collection.Add(pkgs...)
|
||||
}
|
||||
|
||||
return collection.Sorted()
|
||||
}
|
||||
|
||||
func realPaths(p pkg.Package) []string {
|
||||
var paths []string
|
||||
for _, l := range p.Locations.ToSlice() {
|
||||
paths = append(paths, l.RealPath)
|
||||
}
|
||||
return paths
|
||||
}
|
||||
185
syft/pkg/cataloger/perl/meta.go
Normal file
185
syft/pkg/cataloger/perl/meta.go
Normal file
@ -0,0 +1,185 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"path"
|
||||
|
||||
"go.yaml.in/yaml/v3"
|
||||
|
||||
"github.com/anchore/syft/internal"
|
||||
"github.com/anchore/syft/internal/log"
|
||||
"github.com/anchore/syft/syft/file"
|
||||
)
|
||||
|
||||
// cpanMeta models the parts of a CPAN Meta Spec document syft uses. One shape covers all three:
|
||||
// META.json (spec 2), MYMETA.json (the configure-time rendering of META.json), and META.yml (spec 1.4).
|
||||
type cpanMeta struct {
|
||||
Name string `json:"name" yaml:"name"`
|
||||
Version scalar `json:"version" yaml:"version"`
|
||||
Abstract string `json:"abstract" yaml:"abstract"`
|
||||
License stringList `json:"license" yaml:"license"`
|
||||
Provides map[string]providesEntry `json:"provides" yaml:"provides"`
|
||||
|
||||
// SPDX is preferred over License when present: a single SPDX expression the author wrote, rather than
|
||||
// a CPAN::Meta license token that has to be translated
|
||||
SPDX string `json:"x_spdx_expression" yaml:"x_spdx_expression"`
|
||||
|
||||
// Prereqs is the spec 2 shape: phase -> relation -> module -> version
|
||||
Prereqs map[string]map[string]map[string]scalar `json:"prereqs" yaml:"prereqs"`
|
||||
|
||||
// Requires and BuildRequires are the spec 1.4 shape, which has no phase nesting. develop and test
|
||||
// have no 1.4 equivalent, which costs nothing because they are excluded anyway.
|
||||
Requires map[string]scalar `json:"requires" yaml:"requires"`
|
||||
BuildRequires map[string]scalar `json:"build_requires" yaml:"build_requires"`
|
||||
}
|
||||
|
||||
type providesEntry struct {
|
||||
Version scalar `json:"version" yaml:"version"`
|
||||
}
|
||||
|
||||
// scalar is a value that CPAN metadata writes as either a quoted string or a bare number. Old
|
||||
// distributions ship `"version": 1.23` where the spec calls for a string, and a strict string field
|
||||
// there would fail the whole document rather than one field.
|
||||
type scalar string
|
||||
|
||||
func (s *scalar) UnmarshalJSON(b []byte) error {
|
||||
raw := string(b)
|
||||
if raw == "null" {
|
||||
return nil
|
||||
}
|
||||
if len(raw) >= 2 && raw[0] == '"' {
|
||||
var str string
|
||||
if err := json.Unmarshal(b, &str); err != nil {
|
||||
return err
|
||||
}
|
||||
*s = scalar(str)
|
||||
return nil
|
||||
}
|
||||
*s = scalar(raw)
|
||||
return nil
|
||||
}
|
||||
|
||||
// UnmarshalYAML exists for the same reason UnmarshalJSON does, and separately from it: the YAML decoder
|
||||
// does not honor json.Unmarshaler, so without this a `version: 1.23` in a META.yml would fail the whole
|
||||
// document. A YAML scalar node's value is already the text as written, so there is nothing to convert.
|
||||
func (s *scalar) UnmarshalYAML(value *yaml.Node) error {
|
||||
if value.Tag == "!!null" {
|
||||
return nil
|
||||
}
|
||||
*s = scalar(value.Value)
|
||||
return nil
|
||||
}
|
||||
|
||||
// stringList is a value that is a list of strings in CPAN Meta Spec 2 but a bare string in spec 1.4.
|
||||
type stringList []string
|
||||
|
||||
func (s *stringList) UnmarshalJSON(b []byte) error {
|
||||
var many []string
|
||||
if err := json.Unmarshal(b, &many); err == nil {
|
||||
*s = many
|
||||
return nil
|
||||
}
|
||||
var one string
|
||||
if err := json.Unmarshal(b, &one); err == nil {
|
||||
*s = stringList{one}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *stringList) UnmarshalYAML(value *yaml.Node) error {
|
||||
var many []string
|
||||
if err := value.Decode(&many); err == nil {
|
||||
*s = many
|
||||
return nil
|
||||
}
|
||||
var one string
|
||||
if err := value.Decode(&one); err == nil {
|
||||
*s = stringList{one}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// readMyMeta reads the MYMETA.json that cpanm, cpm, and carton write beside install.json. A missing or
|
||||
// unreadable MYMETA is not an error: the distribution is still real, it just has fewer facts attached.
|
||||
func readMyMeta(resolver file.Resolver, installJSONLocation file.Location) (*file.Location, *cpanMeta) {
|
||||
if resolver == nil {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
p := path.Join(path.Dir(installJSONLocation.Path()), "MYMETA.json")
|
||||
|
||||
locations, err := resolver.FilesByPath(p)
|
||||
if err != nil || len(locations) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
m, err := decodeMeta(resolver, locations[0])
|
||||
if err != nil {
|
||||
log.WithFields("path", p, "error", err).Debug("unable to parse CPAN MYMETA.json")
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
return &locations[0], m
|
||||
}
|
||||
|
||||
func decodeMeta(resolver file.Resolver, location file.Location) (*cpanMeta, error) {
|
||||
return decode(resolver, location, func(r io.Reader, m *cpanMeta) error {
|
||||
return json.NewDecoder(r).Decode(m)
|
||||
})
|
||||
}
|
||||
|
||||
// decodeMetaYAML reads a spec 1.4 META.yml. CPAN META.yml files are rejected by strict YAML parsers often
|
||||
// enough to matter: pre-6.46 ExtUtils::MakeMaker writes values with no quoting or escaping at all, so a
|
||||
// multi-line ABSTRACT, a value that opens a quote it never closes, and a value starting with >= (which
|
||||
// YAML reads as a folded block scalar) all get emitted verbatim. Measured over 454 releases from 2004 to
|
||||
// 2009, five fail. That is a routine failure, not a defensive one, and the caller degrades on it.
|
||||
func decodeMetaYAML(resolver file.Resolver, location file.Location) (*cpanMeta, error) {
|
||||
return decode(resolver, location, func(r io.Reader, m *cpanMeta) error {
|
||||
return yaml.NewDecoder(r).Decode(m)
|
||||
})
|
||||
}
|
||||
|
||||
func decode(resolver file.Resolver, location file.Location, into func(io.Reader, *cpanMeta) error) (*cpanMeta, error) {
|
||||
reader, err := resolver.FileContentsByLocation(location)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer internal.CloseAndLogError(reader, location.Path())
|
||||
|
||||
var m cpanMeta
|
||||
if err := into(reader, &m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &m, nil
|
||||
}
|
||||
|
||||
// prereqModules returns the module names this distribution requires at runtime and build time.
|
||||
// develop and test phases are excluded: they describe how the distribution was authored, not what a
|
||||
// deployed copy of it needs. A spec 1.4 document has no phases at all, so its flat requires and
|
||||
// build_requires maps feed the same two buckets.
|
||||
func (m *cpanMeta) prereqModules() []string {
|
||||
var modules []string
|
||||
for _, phase := range []string{"runtime", "build"} {
|
||||
for module := range m.Prereqs[phase]["requires"] {
|
||||
modules = append(modules, module)
|
||||
}
|
||||
}
|
||||
for _, flat := range []map[string]scalar{m.Requires, m.BuildRequires} {
|
||||
for module := range flat {
|
||||
modules = append(modules, module)
|
||||
}
|
||||
}
|
||||
return modules
|
||||
}
|
||||
|
||||
// licenseExpressions returns the license values to record on the package. x_spdx_expression wins when
|
||||
// present: it is a single SPDX expression, which syft's license parsing understands directly, rather
|
||||
// than a CPAN license shortname such as perl_5 or artistic_2.
|
||||
func (m *cpanMeta) licenseExpressions() []string {
|
||||
if m.SPDX != "" {
|
||||
return []string{m.SPDX}
|
||||
}
|
||||
return m.License
|
||||
}
|
||||
128
syft/pkg/cataloger/perl/meta_test.go
Normal file
128
syft/pkg/cataloger/perl/meta_test.go
Normal file
@ -0,0 +1,128 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"go.yaml.in/yaml/v3"
|
||||
)
|
||||
|
||||
// yaml.v3 does not honor json.Unmarshaler, so the lenient types that already tolerate both license and
|
||||
// version shapes in JSON need their own UnmarshalYAML or a spec 1.4 META.yml fails on one field.
|
||||
func Test_cpanMeta_lenientFieldsInYAML(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
contents string
|
||||
expectedVersion scalar
|
||||
expectedLicense stringList
|
||||
}{
|
||||
{
|
||||
// what a real ExtUtils::MakeMaker 6.x META.yml writes: version unquoted, license a bare string
|
||||
name: "spec 1.4 shapes",
|
||||
contents: "name: Text-CSV\nversion: 1.21\nlicense: perl\n",
|
||||
expectedVersion: "1.21",
|
||||
expectedLicense: stringList{"perl"},
|
||||
},
|
||||
{
|
||||
name: "quoted version and a license list",
|
||||
contents: "name: URI\nversion: '5.35'\nlicense:\n - perl_5\n - artistic_2\n",
|
||||
expectedVersion: "5.35",
|
||||
expectedLicense: stringList{"perl_5", "artistic_2"},
|
||||
},
|
||||
{
|
||||
// a version that would read as a v-string rather than a number
|
||||
name: "v-string version",
|
||||
contents: "name: App-cpm\nversion: v1.1.4\n",
|
||||
expectedVersion: "v1.1.4",
|
||||
},
|
||||
{
|
||||
name: "explicit nulls",
|
||||
contents: "name: Foo\nversion: ~\nlicense: ~\n",
|
||||
expectedVersion: "",
|
||||
expectedLicense: nil,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var m cpanMeta
|
||||
require.NoError(t, yaml.Unmarshal([]byte(tt.contents), &m))
|
||||
assert.Equal(t, tt.expectedVersion, m.Version)
|
||||
assert.Equal(t, tt.expectedLicense, m.License)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func Test_cpanMeta_licenseExpressions(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
contents string
|
||||
expected []string
|
||||
}{
|
||||
{
|
||||
// an SPDX expression is what syft's license parsing understands directly, where perl_5 is a
|
||||
// CPAN::Meta token that has to be translated
|
||||
name: "x_spdx_expression wins",
|
||||
contents: `{"license":["perl_5"],"x_spdx_expression":"Artistic-1.0-Perl OR GPL-1.0-or-later"}`,
|
||||
expected: []string{"Artistic-1.0-Perl OR GPL-1.0-or-later"},
|
||||
},
|
||||
{
|
||||
name: "license is used when there is no expression",
|
||||
contents: `{"license":["perl_5"]}`,
|
||||
expected: []string{"perl_5"},
|
||||
},
|
||||
{
|
||||
name: "single-string license, as spec 1.4 writes it",
|
||||
contents: `{"license":"perl"}`,
|
||||
expected: []string{"perl"},
|
||||
},
|
||||
{
|
||||
name: "neither present",
|
||||
contents: `{"name":"Foo"}`,
|
||||
expected: nil,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var m cpanMeta
|
||||
require.NoError(t, json.Unmarshal([]byte(tt.contents), &m))
|
||||
assert.Equal(t, tt.expected, m.licenseExpressions())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func Test_cpanMeta_prereqModules(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
contents string
|
||||
expected []string
|
||||
}{
|
||||
{
|
||||
name: "spec 2 prereqs, keyed by phase",
|
||||
contents: `{"prereqs":{
|
||||
"runtime": {"requires": {"MIME::Base64": "2.1"}},
|
||||
"build": {"requires": {"ExtUtils::MakeMaker": "0"}},
|
||||
"test": {"requires": {"Test::More": "0.88"}},
|
||||
"develop": {"requires": {"Pod::Coverage": "0"}}
|
||||
}}`,
|
||||
expected: []string{"MIME::Base64", "ExtUtils::MakeMaker"},
|
||||
},
|
||||
{
|
||||
// spec 1.4 has no phase nesting at all, so the two flat maps feed the same two buckets
|
||||
name: "spec 1.4 flat requirements",
|
||||
contents: `{"requires":{"IO::Handle":"0"},"build_requires":{"Test::More":"0.47"}}`,
|
||||
expected: []string{"IO::Handle", "Test::More"},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var m cpanMeta
|
||||
require.NoError(t, json.Unmarshal([]byte(tt.contents), &m))
|
||||
assert.ElementsMatch(t, tt.expected, m.prereqModules())
|
||||
})
|
||||
}
|
||||
}
|
||||
74
syft/pkg/cataloger/perl/package.go
Normal file
74
syft/pkg/cataloger/perl/package.go
Normal file
@ -0,0 +1,74 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/anchore/packageurl-go"
|
||||
"github.com/anchore/syft/syft/file"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
)
|
||||
|
||||
// newCpanPackage builds a package for a CPAN distribution. Note that the name and version are passed
|
||||
// separately rather than living on the metadata struct: they would be verbatim copies of the package
|
||||
// fields, so they are not duplicated into metadata.
|
||||
//
|
||||
// The metadata is an `any` because the two evidence tiers carry different types, and the author is an
|
||||
// explicit parameter for the same reason: pkg.CpanUnpackedRelease has no PAUSE path, so there is no
|
||||
// field on it the purl could be built from.
|
||||
func newCpanPackage(name, version, author string, md any, licenses []pkg.License, locations ...file.Location) pkg.Package {
|
||||
p := pkg.Package{
|
||||
Name: name,
|
||||
Version: version,
|
||||
PURL: packageURL(name, version, author),
|
||||
Locations: file.NewLocationSet(locations...),
|
||||
Licenses: pkg.NewLicenseSet(licenses...),
|
||||
Language: pkg.Perl,
|
||||
Type: pkg.CpanPkg,
|
||||
Metadata: md,
|
||||
}
|
||||
|
||||
p.SetID()
|
||||
|
||||
return p
|
||||
}
|
||||
|
||||
// packageURL returns the PURL for a CPAN distribution. The distribution name is used verbatim: CPAN
|
||||
// names are case sensitive. The author is emitted as a qualifier rather than as the namespace, which
|
||||
// is what the current purl spec prefers, and is omitted entirely when unknown rather than guessed.
|
||||
func packageURL(name, version, author string) string {
|
||||
var qualifiers packageurl.Qualifiers
|
||||
if author != "" {
|
||||
qualifiers = append(qualifiers, packageurl.Qualifier{Key: "author", Value: author})
|
||||
}
|
||||
|
||||
return packageurl.NewPackageURL(packageurl.TypeCpan, "", name, version, qualifiers, "").ToString()
|
||||
}
|
||||
|
||||
// authorFromPathname extracts the CPAN author (PAUSE) ID from a PAUSE path such as
|
||||
// O/OA/OALDERS/URI-5.35.tar.gz. A path with fewer than three segments yields no author rather than a
|
||||
// partial one.
|
||||
func authorFromPathname(p string) string {
|
||||
parts := strings.Split(p, "/")
|
||||
if len(parts) < 3 {
|
||||
return ""
|
||||
}
|
||||
return parts[2]
|
||||
}
|
||||
|
||||
// modulesFromProvides converts a CPAN metadata `provides` map into a stable, sorted module list.
|
||||
// The `file` field is dropped: the installed path is mechanical from the module name.
|
||||
func modulesFromProvides(provides map[string]providesEntry) []pkg.CpanModule {
|
||||
if len(provides) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
modules := make([]pkg.CpanModule, 0, len(provides))
|
||||
for name, entry := range provides {
|
||||
modules = append(modules, pkg.CpanModule{Name: name, Version: string(entry.Version)})
|
||||
}
|
||||
|
||||
sort.Slice(modules, func(i, j int) bool { return modules[i].Name < modules[j].Name })
|
||||
|
||||
return modules
|
||||
}
|
||||
76
syft/pkg/cataloger/perl/package_test.go
Normal file
76
syft/pkg/cataloger/perl/package_test.go
Normal file
@ -0,0 +1,76 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func Test_packageURL(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
dist string
|
||||
version string
|
||||
author string
|
||||
want string
|
||||
}{
|
||||
{
|
||||
name: "author available",
|
||||
dist: "URI",
|
||||
version: "5.35",
|
||||
author: "OALDERS",
|
||||
want: "pkg:cpan/URI@5.35?author=OALDERS",
|
||||
},
|
||||
{
|
||||
name: "no author known, e.g. a packlist-derived package",
|
||||
dist: "Text-CSV",
|
||||
version: "2.06",
|
||||
want: "pkg:cpan/Text-CSV@2.06",
|
||||
},
|
||||
{
|
||||
name: "case is preserved",
|
||||
dist: "JSON-MaybeXS",
|
||||
version: "1.004008",
|
||||
author: "ETHER",
|
||||
want: "pkg:cpan/JSON-MaybeXS@1.004008?author=ETHER",
|
||||
},
|
||||
{
|
||||
name: "v-prefixed version is preserved",
|
||||
dist: "CPAN-02Packages-Search",
|
||||
version: "v1.0.0",
|
||||
author: "SKAJI",
|
||||
want: "pkg:cpan/CPAN-02Packages-Search@v1.0.0?author=SKAJI",
|
||||
},
|
||||
{
|
||||
name: "no version",
|
||||
dist: "IO-Socket-SSL",
|
||||
want: "pkg:cpan/IO-Socket-SSL",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
assert.Equal(t, tt.want, packageURL(tt.dist, tt.version, tt.author))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func Test_authorFromPathname(t *testing.T) {
|
||||
tests := []struct {
|
||||
pathname string
|
||||
want string
|
||||
}{
|
||||
{pathname: "O/OA/OALDERS/URI-5.35.tar.gz", want: "OALDERS"},
|
||||
{pathname: "S/SK/SKAJI/CPAN-02Packages-Search-v1.0.0.tar.gz", want: "SKAJI"},
|
||||
// not enough segments to name an author: better to omit than to guess
|
||||
{pathname: "OALDERS/URI-5.35.tar.gz", want: ""},
|
||||
{pathname: "URI-5.35.tar.gz", want: ""},
|
||||
{pathname: "", want: ""},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.pathname, func(t *testing.T) {
|
||||
assert.Equal(t, tt.want, authorFromPathname(tt.pathname))
|
||||
})
|
||||
}
|
||||
}
|
||||
95
syft/pkg/cataloger/perl/parse_install_json.go
Normal file
95
syft/pkg/cataloger/perl/parse_install_json.go
Normal file
@ -0,0 +1,95 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"github.com/anchore/syft/internal/log"
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/anchore/syft/syft/file"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/generic"
|
||||
)
|
||||
|
||||
// installJSON is the record cpanm (>= 1.5000), cpm, and carton write for every distribution they install.
|
||||
type installJSON struct {
|
||||
// Name is the MODULE name, not the distribution: libwww-perl records "LWP" here, and
|
||||
// CPAN-02Packages-Search records "CPAN::02Packages::Search". It is never the package name.
|
||||
Name string `json:"name"`
|
||||
Dist string `json:"dist"` // distribution with version, e.g. libwww-perl-5.836
|
||||
Version scalar `json:"version"`
|
||||
Pathname string `json:"pathname"` // PAUSE path, e.g. O/OA/OALDERS/URI-5.35.tar.gz
|
||||
Provides map[string]providesEntry `json:"provides"`
|
||||
}
|
||||
|
||||
func parseInstallJSON(ctx context.Context, resolver file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
|
||||
var doc installJSON
|
||||
if err := json.NewDecoder(reader).Decode(&doc); err != nil {
|
||||
log.WithFields("path", reader.Path(), "error", err).Debug("unable to parse CPAN install.json")
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
name := distributionName(doc)
|
||||
if name == "" || doc.Version == "" {
|
||||
log.WithFields("path", reader.Path()).Debug("CPAN install.json is missing a distribution name or version")
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
locations := []file.Location{reader.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation)}
|
||||
|
||||
var licenses []pkg.License
|
||||
if myMetaLocation, myMeta := readMyMeta(resolver, reader.Location); myMeta != nil {
|
||||
licenses = pkg.NewLicensesFromLocationWithContext(ctx, *myMetaLocation, myMeta.licenseExpressions()...)
|
||||
locations = append(locations, myMetaLocation.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.SupportingEvidenceAnnotation))
|
||||
}
|
||||
|
||||
md := pkg.CpanDistribution{
|
||||
Dist: doc.Dist,
|
||||
Author: authorFromPathname(doc.Pathname),
|
||||
Path: doc.Pathname,
|
||||
Modules: modulesFromProvides(doc.Provides),
|
||||
}
|
||||
|
||||
return []pkg.Package{newCpanPackage(name, string(doc.Version), md.Author, md, licenses, locations...)}, nil, nil
|
||||
}
|
||||
|
||||
// distributionName resolves the distribution name, which every consumer of these packages is keyed by.
|
||||
// The `name` field cannot be used: it holds the module name, so libwww-perl would be reported as LWP
|
||||
// and CGI-Session as CGI::Session, which no advisory or index is filed against.
|
||||
//
|
||||
// `dist` carries the distribution with its version appended, and the version is known, so trimming the
|
||||
// suffix is exact. That also resolves the case that looks ambiguous when splitting on the last dash:
|
||||
// CPAN-02Packages-Search-v1.0.0 minus v1.0.0 leaves CPAN-02Packages-Search, where a dash rule would
|
||||
// have to guess whether v1.0.0 is one segment or three.
|
||||
func distributionName(doc installJSON) string {
|
||||
if doc.Dist != "" {
|
||||
if name := strings.TrimSuffix(doc.Dist, "-"+string(doc.Version)); name != "" && name != doc.Dist {
|
||||
return name
|
||||
}
|
||||
// no version suffix to trim, so dist is already the bare name. cpm was checked and does append
|
||||
// the version just as cpanm does (see TestInstallJSON_cpmFormatting), so this is a guard against
|
||||
// records neither installer is known to write rather than a shape any of them produce.
|
||||
return doc.Dist
|
||||
}
|
||||
|
||||
// older cpanm and hand-rolled records omit dist; the PAUSE path basename carries the same
|
||||
// <Dist>-<Version>.tar.gz shape
|
||||
return distributionFromPathname(doc.Pathname, string(doc.Version))
|
||||
}
|
||||
|
||||
// distributionFromPathname recovers the distribution from a PAUSE path such as
|
||||
// G/GA/GAAS/libwww-perl-5.836.tar.gz, which is the only evidence left when `dist` is absent.
|
||||
func distributionFromPathname(pathname, version string) string {
|
||||
base := path.Base(pathname)
|
||||
if base == "." || base == "/" {
|
||||
return ""
|
||||
}
|
||||
|
||||
for _, ext := range []string{".tar.gz", ".tar.bz2", ".tgz", ".zip"} {
|
||||
base = strings.TrimSuffix(base, ext)
|
||||
}
|
||||
|
||||
return strings.TrimSuffix(base, "-"+version)
|
||||
}
|
||||
157
syft/pkg/cataloger/perl/parse_install_json_test.go
Normal file
157
syft/pkg/cataloger/perl/parse_install_json_test.go
Normal file
@ -0,0 +1,157 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"github.com/anchore/syft/internal"
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/anchore/syft/syft/file"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
|
||||
)
|
||||
|
||||
// the record shape here is cpanm's, reduced to the fields the parser reads. What matters is that
|
||||
// `name` is the module (CPAN::02Packages::Search) and must not become the package name, and that the
|
||||
// version is a v-string, so trimming it off `dist` is what makes the CPAN-02Packages-Search-v1.0.0
|
||||
// split unambiguous where a rule based on the last dash would have to guess.
|
||||
//
|
||||
// The byte-level shape of what the clients really write is covered against real files by
|
||||
// TestInstallJSON_clientFormatting.
|
||||
const installJSONFixture = `{"target":"CPAN::02Packages::Search","module_name":"CPAN::02Packages::Search",` +
|
||||
`"version":"v1.0.0","dist":"CPAN-02Packages-Search-v1.0.0","pathname":"S/SK/SKAJI/CPAN-02Packages-Search-v1.0.0.tar.gz",` +
|
||||
`"provides":{"CPAN::02Packages::Search":{"file":"lib/CPAN/02Packages/Search.pm","version":"v1.0.0"}},` +
|
||||
`"name":"CPAN::02Packages::Search"}`
|
||||
|
||||
func TestParseInstallJSON(t *testing.T) {
|
||||
const location = "usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/CPAN-02Packages-Search-v1.0.0/install.json"
|
||||
|
||||
expected := []pkg.Package{
|
||||
{
|
||||
Name: "CPAN-02Packages-Search",
|
||||
Version: "v1.0.0",
|
||||
PURL: "pkg:cpan/CPAN-02Packages-Search@v1.0.0?author=SKAJI",
|
||||
Locations: file.NewLocationSet(file.NewLocation(location)),
|
||||
Language: pkg.Perl,
|
||||
Type: pkg.CpanPkg,
|
||||
Metadata: pkg.CpanDistribution{
|
||||
Dist: "CPAN-02Packages-Search-v1.0.0",
|
||||
Author: "SKAJI",
|
||||
Path: "S/SK/SKAJI/CPAN-02Packages-Search-v1.0.0.tar.gz",
|
||||
// the provides map, sorted by module name, with per-module versions: a distribution
|
||||
// built with inherit_version = 0 ships modules at versions other than its own
|
||||
Modules: []pkg.CpanModule{
|
||||
{Name: "CPAN::02Packages::Search", Version: "v1.0.0"},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
// no MYMETA.json is reachable from a single-string resolver, so the package is reported with no
|
||||
// licenses and no error
|
||||
pkgtest.NewCatalogTester().
|
||||
FromString(location, installJSONFixture).
|
||||
Expects(expected, []artifact.Relationship(nil)).
|
||||
TestParser(t, parseInstallJSON)
|
||||
}
|
||||
|
||||
// libwww-perl is the canonical name mismatch: `name` is LWP while the distribution every advisory and
|
||||
// index is filed against is libwww-perl. Reporting LWP here is the bug this guards. The end-to-end
|
||||
// form of it is TestCpanInstalledCataloger_mirrorInstallEvidence.
|
||||
func TestParseInstallJSON_distributionNameNotModuleName(t *testing.T) {
|
||||
const record = `{"name":"LWP","version":"6.83","dist":"libwww-perl-6.83",` +
|
||||
`"pathname":"O/OA/OALDERS/libwww-perl-6.83.tar.gz",` +
|
||||
`"provides":{"LWP":{"file":"lib/LWP.pm","version":"6.83"},"LWP::UserAgent":{"file":"lib/LWP/UserAgent.pm","version":"6.83"}}}`
|
||||
|
||||
pkgs := parseString(t, "libwww-perl-6.83/install.json", record)
|
||||
require.Len(t, pkgs, 1)
|
||||
|
||||
assert.Equal(t, "libwww-perl", pkgs[0].Name, "the package name must be the distribution, not the LWP module")
|
||||
assert.Equal(t, "pkg:cpan/libwww-perl@6.83?author=OALDERS", pkgs[0].PURL)
|
||||
|
||||
// the module name is not lost, it belongs in the provides map
|
||||
assert.Equal(t, []string{"LWP", "LWP::UserAgent"}, providedModules(pkgs[0]))
|
||||
}
|
||||
|
||||
// TestInstallJSON_clientFormatting records what was measured about cpm versus cpanm, since the
|
||||
// question of whether cpm needs its own coverage kept coming up. Both files are read out of the
|
||||
// image, where cpanm (through carton) and cpm each installed one distribution.
|
||||
//
|
||||
// Finding: they are NOT byte-identical. cpanm writes compact JSON with no trailing newline; cpm writes
|
||||
// canonical JSON, indented three spaces with spaces around the colons and a trailing newline. The
|
||||
// record itself has the same shape, including `dist` carrying the version suffix, which is worth
|
||||
// stating because it is the field the distribution name is derived from.
|
||||
//
|
||||
// So anything that reads these files by decoding JSON is unaffected by the difference; anything that
|
||||
// pattern matches the bytes is not.
|
||||
func TestInstallJSON_clientFormatting(t *testing.T) {
|
||||
resolver := imageResolver(t, "image-cpan-mirror-installs")
|
||||
|
||||
read := func(path string) string {
|
||||
locations, err := resolver.FilesByPath(path)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, locations, 1, path)
|
||||
|
||||
reader, err := resolver.FileContentsByLocation(locations[0])
|
||||
require.NoError(t, err)
|
||||
defer internal.CloseAndLogError(reader, path)
|
||||
|
||||
contents, err := io.ReadAll(reader)
|
||||
require.NoError(t, err)
|
||||
return string(contents)
|
||||
}
|
||||
|
||||
// written by cpanm, which carton shells out to
|
||||
cpanmWritten := read("/app/local/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json")
|
||||
cpmWritten := read("/srv/api/local/lib/perl5/x86_64-linux-gnu/.meta/Capture-Tiny-0.48/install.json")
|
||||
|
||||
assert.NotContains(t, cpanmWritten, "\n", "cpanm writes a single compact line")
|
||||
assert.Contains(t, cpmWritten, "\"dist\" : ", "cpm writes canonical JSON, with spaces around the colons")
|
||||
assert.Contains(t, cpmWritten, "\n", "and a trailing newline")
|
||||
|
||||
var fromCpanm, fromCpm installJSON
|
||||
require.NoError(t, json.Unmarshal([]byte(cpanmWritten), &fromCpanm))
|
||||
require.NoError(t, json.Unmarshal([]byte(cpmWritten), &fromCpm))
|
||||
|
||||
assert.Equal(t, "Text-CSV-2.06", fromCpanm.Dist)
|
||||
assert.Equal(t, "Capture-Tiny-0.48", fromCpm.Dist, "cpm does carry the version suffix on dist, just as cpanm does")
|
||||
}
|
||||
|
||||
func parseString(t *testing.T, location, content string) []pkg.Package {
|
||||
t.Helper()
|
||||
|
||||
pkgs, _, err := parseInstallJSON(context.Background(), nil, nil,
|
||||
file.NewLocationReadCloser(file.NewLocation(location), io.NopCloser(strings.NewReader(content))))
|
||||
require.NoError(t, err)
|
||||
|
||||
return pkgs
|
||||
}
|
||||
|
||||
func TestParseInstallJSON_degrades(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
content string
|
||||
}{
|
||||
{name: "unparseable json", content: "not json at all\n"},
|
||||
{
|
||||
// never fall back to splitting the containing directory name
|
||||
name: "missing name and version",
|
||||
content: `{"provides":{"URI":{"file":"lib/URI.pm"}}}`,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
pkgtest.NewCatalogTester().
|
||||
FromString("URI-5.35/install.json", tt.content).
|
||||
Expects(nil, nil).
|
||||
TestParser(t, parseInstallJSON)
|
||||
})
|
||||
}
|
||||
}
|
||||
143
syft/pkg/cataloger/perl/parse_packlist.go
Normal file
143
syft/pkg/cataloger/perl/parse_packlist.go
Normal file
@ -0,0 +1,143 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"io"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"github.com/anchore/syft/internal"
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/anchore/syft/syft/file"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/generic"
|
||||
)
|
||||
|
||||
// versionPattern matches the $VERSION assignment perl modules declare, e.g. `our $VERSION = '2.06';`.
|
||||
// The value is captured as written so that both decimal (1.303) and v-string (v1.1.4) forms survive intact.
|
||||
//
|
||||
// Requiring a literal after the `=` is what makes the pattern decline the computed forms, which cannot be
|
||||
// read statically at all: real Encode.pm is a bare `our $VERSION;` followed by
|
||||
// `$VERSION = sprintf "%d.%02d", q$Revision: 3.24 $ =~ /(\d+)/g;` inside a BEGIN block. Those come from
|
||||
// perllocal.pod instead, which records what the installer evaluated.
|
||||
//
|
||||
// The optional package prefix covers the fully qualified form older Dist::Zilla wrote,
|
||||
// `$JSON::PP::VERSION = '2.27300';`, and qv() covers `our $VERSION = qv('1.2.3');`.
|
||||
var versionPattern = regexp.MustCompile(`^[ \t]*(?:our[ \t]+|my[ \t]+)?\$(?:[A-Za-z_][A-Za-z0-9_]*::)*VERSION[ \t]*=[ \t]*(?:qv\()?['"]?(v?[0-9][0-9._]*)`)
|
||||
|
||||
// packageVersionPattern matches a version declared in the package statement itself,
|
||||
// `package CPAN::02Packages::Search v1.0.0;`, which perl 5.12 and later allow. A distribution using it
|
||||
// need carry no $VERSION assignment anywhere: real CPAN::02Packages::Search v1.0.0 does not.
|
||||
var packageVersionPattern = regexp.MustCompile(`^[ \t]*package[ \t]+[A-Za-z_][A-Za-z0-9_:]*[ \t]+(v?[0-9][0-9._]*)[ \t]*[;{]`)
|
||||
|
||||
// packlistMetadataSuffix matches the trailing `key=value` metadata that ExtUtils::Packlist::read strips
|
||||
// from a packlist line, following the `/^(.*?)( \w+=.*)$/` it uses. Cutting at the first space instead
|
||||
// truncates any real path that contains one, which corrupts both the owned-file list and the main-.pm
|
||||
// version fallback.
|
||||
var packlistMetadataSuffix = regexp.MustCompile(` \w+=.*$`)
|
||||
|
||||
// parsePacklist catalogs a distribution installed by any ExtUtils::MakeMaker or Module::Build run,
|
||||
// which covers CPAN.pm (which writes no .meta) and the perl image's own preinstalled distributions.
|
||||
func parsePacklist(_ context.Context, _ file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
|
||||
name := distNameFromPacklistPath(reader.Path())
|
||||
if name == "" {
|
||||
// the perl core packlist has no auto/<Dist> segment; it lists thousands of files and describes
|
||||
// no single distribution
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
var files []string
|
||||
scanner := bufio.NewScanner(reader)
|
||||
for scanner.Scan() {
|
||||
// packlist lines are "<path>" or "<path> type=link from=<path>"
|
||||
if f := strings.TrimSpace(packlistMetadataSuffix.ReplaceAllString(scanner.Text(), "")); f != "" {
|
||||
files = append(files, f)
|
||||
}
|
||||
}
|
||||
|
||||
location := reader.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation)
|
||||
md := pkg.CpanDistribution{MainModule: moduleFromDashedName(name), Files: files}
|
||||
|
||||
// the version is left to resolvePacklistVersions, which prefers perllocal.pod and needs the whole
|
||||
// cataloged set to find it. A distribution with no readable version is still reported, so that it
|
||||
// stays visible.
|
||||
return []pkg.Package{newCpanPackage(name, "", "", md, nil, location)}, nil, nil
|
||||
}
|
||||
|
||||
// distNameFromPacklistPath derives the distribution name from the packlist path, e.g.
|
||||
// .../auto/IO/Socket/SSL/.packlist yields IO-Socket-SSL. A path with no auto/ segment yields "".
|
||||
func distNameFromPacklistPath(p string) string {
|
||||
parts := strings.Split(strings.Trim(p, "/"), "/")
|
||||
|
||||
last := -1
|
||||
for i, part := range parts {
|
||||
if part == "auto" {
|
||||
last = i
|
||||
}
|
||||
}
|
||||
|
||||
if last < 0 || last+1 >= len(parts)-1 {
|
||||
return ""
|
||||
}
|
||||
|
||||
return strings.Join(parts[last+1:len(parts)-1], "-")
|
||||
}
|
||||
|
||||
// mainModuleFile picks the .pm file in the packlist that corresponds to the distribution's main module,
|
||||
// e.g. Text-CSV is expected to install Text/CSV.pm.
|
||||
//
|
||||
// The suffix can match more than one entry, and taking the first is wrong: libwww-perl's packlist lists
|
||||
// both Bundle/LWP.pm (5.835) and LWP.pm (5.836), with the Bundle copy first. The main module sits at the
|
||||
// lib root, so the shallowest match is the right one; a nested match is some other module that merely
|
||||
// ends in the same basename.
|
||||
func mainModuleFile(dist string, files []string) string {
|
||||
suffix := "/" + strings.ReplaceAll(dist, "-", "/") + ".pm"
|
||||
|
||||
best := ""
|
||||
bestDepth := 0
|
||||
for _, f := range files {
|
||||
if !strings.HasSuffix(f, suffix) {
|
||||
continue
|
||||
}
|
||||
if depth := strings.Count(f, "/"); best == "" || depth < bestDepth {
|
||||
best, bestDepth = f, depth
|
||||
}
|
||||
}
|
||||
|
||||
return best
|
||||
}
|
||||
|
||||
func findFile(resolver file.Resolver, p string) (file.Location, bool) {
|
||||
if resolver == nil || p == "" {
|
||||
return file.Location{}, false
|
||||
}
|
||||
locations, err := resolver.FilesByPath(p)
|
||||
if err != nil || len(locations) == 0 {
|
||||
return file.Location{}, false
|
||||
}
|
||||
return locations[0], true
|
||||
}
|
||||
|
||||
func readVersionFromPM(resolver file.Resolver, location file.Location) string {
|
||||
reader, err := resolver.FileContentsByLocation(location)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
defer internal.CloseAndLogError(reader, location.Path())
|
||||
|
||||
return versionFromPM(reader)
|
||||
}
|
||||
|
||||
func versionFromPM(r io.Reader) string {
|
||||
scanner := bufio.NewScanner(r)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
for _, pattern := range []*regexp.Regexp{versionPattern, packageVersionPattern} {
|
||||
if match := pattern.FindStringSubmatch(line); match != nil {
|
||||
return strings.TrimRight(match[1], ".")
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
105
syft/pkg/cataloger/perl/parse_packlist_test.go
Normal file
105
syft/pkg/cataloger/perl/parse_packlist_test.go
Normal file
@ -0,0 +1,105 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"github.com/anchore/syft/syft/file"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
)
|
||||
|
||||
func Test_distNameFromPacklistPath(t *testing.T) {
|
||||
tests := []struct {
|
||||
path string
|
||||
want string
|
||||
}{
|
||||
{path: "/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/Text/CSV/.packlist", want: "Text-CSV"},
|
||||
{path: "/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/IO/Socket/SSL/.packlist", want: "IO-Socket-SSL"},
|
||||
{path: "auto/URI/.packlist", want: "URI"},
|
||||
// the perl core install has a packlist with no auto/ segment; it must produce nothing
|
||||
{path: "/usr/local/lib/perl5/5.40.4/x86_64-linux/.packlist", want: ""},
|
||||
{path: "/some/auto/.packlist", want: ""},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.path, func(t *testing.T) {
|
||||
assert.Equal(t, tt.want, distNameFromPacklistPath(tt.path))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func Test_mainModuleFile(t *testing.T) {
|
||||
files := []string{
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/Text/CSV_PP.pm",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/Text/CSV.pm",
|
||||
}
|
||||
|
||||
assert.Equal(t, "/usr/local/lib/perl5/site_perl/5.40.4/Text/CSV.pm", mainModuleFile("Text-CSV", files))
|
||||
assert.Equal(t, "", mainModuleFile("Text-CSV-XS", files))
|
||||
}
|
||||
|
||||
func Test_versionFromPM(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
contents string
|
||||
want string
|
||||
}{
|
||||
{name: "our, single quoted", contents: "package Text::CSV;\n\nour $VERSION = '2.06';\n", want: "2.06"},
|
||||
// what Text::CSV and JSON::PP really do: use vars, then assign inside a BEGIN block
|
||||
{name: "assigned inside BEGIN", contents: "use vars qw( $VERSION );\n\nBEGIN {\n $VERSION = '2.06';\n}\n", want: "2.06"},
|
||||
{name: "bare assignment", contents: "$VERSION = \"1.303\";\n", want: "1.303"},
|
||||
{name: "unquoted decimal", contents: "our $VERSION = 5.35;\n", want: "5.35"},
|
||||
{name: "v-string is preserved", contents: "our $VERSION = 'v1.1.4';\n", want: "v1.1.4"},
|
||||
// the fully qualified form older Dist::Zilla wrote, as real JSON::PP 2.27300 carries
|
||||
{name: "package qualified", contents: "package JSON::PP;\n\n$JSON::PP::VERSION = '2.27300';\n", want: "2.27300"},
|
||||
{name: "qv wrapper", contents: "our $VERSION = qv('1.2.3');\n", want: "1.2.3"},
|
||||
// perl 5.12 and later allow the version in the package statement, and a distribution using it
|
||||
// need carry no $VERSION at all
|
||||
{name: "package statement v-string", contents: "package CPAN::02Packages::Search v1.0.0;\nuse v5.24;\n", want: "v1.0.0"},
|
||||
{name: "package statement decimal", contents: "package Foo::Bar 1.23;\n", want: "1.23"},
|
||||
{name: "package statement block form", contents: "package Foo::Bar v2.0.1 {\n}\n", want: "v2.0.1"},
|
||||
// a $VERSION assignment later in the file still wins over a version-less package statement
|
||||
{name: "package statement without a version", contents: "package Foo::Bar;\nour $VERSION = '1.00';\n", want: "1.00"},
|
||||
// $XS_VERSION and friends must not be mistaken for $VERSION
|
||||
{name: "other version variables ignored", contents: "our $XS_VERSION = '9.99';\nour $VERSION = '1.00';\n", want: "1.00"},
|
||||
// real Encode.pm: declared bare, then computed from an RCS keyword inside BEGIN. Nothing static can
|
||||
// read that, and guessing at it would be worse than the empty version perllocal.pod then fills in.
|
||||
{name: "computed from an RCS Revision keyword", contents: "our $VERSION;\nBEGIN {\n $VERSION = sprintf \"%d.%02d\", q$Revision: 3.24 $ =~ /(\\d+)/g;\n}\n", want: ""},
|
||||
{name: "no version at all", contents: "package Foo;\n1;\n", want: ""},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
assert.Equal(t, tt.want, versionFromPM(strings.NewReader(tt.contents)))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Test_parsePacklist_metadataSuffix pins the cut against what ExtUtils::Packlist::read does, which is to
|
||||
// strip a suffix matching /^(.*?)( \w+=.*)$/ rather than to cut at the first space. A perl tree under a
|
||||
// directory whose name contains a space is ordinary on macOS and Windows.
|
||||
func Test_parsePacklist_metadataSuffix(t *testing.T) {
|
||||
const packlist = `/usr/local/lib/perl5/site_perl/5.40.4/Text/CSV.pm
|
||||
/Users/me/My Perl/lib/Text/CSV_PP.pm
|
||||
/usr/local/lib/perl5/site_perl/5.40.4/auto/Text/CSV/CSV.so type=file
|
||||
/usr/local/bin/csvdiff type=link from=/usr/local/bin/csvdiff-2.06
|
||||
`
|
||||
|
||||
pkgs, _, err := parsePacklist(context.Background(), nil, nil,
|
||||
file.NewLocationReadCloser(file.NewLocation("usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/Text/CSV/.packlist"),
|
||||
io.NopCloser(strings.NewReader(packlist))))
|
||||
require.NoError(t, err)
|
||||
require.Len(t, pkgs, 1)
|
||||
|
||||
assert.Equal(t, []string{
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/Text/CSV.pm",
|
||||
"/Users/me/My Perl/lib/Text/CSV_PP.pm",
|
||||
"/usr/local/lib/perl5/site_perl/5.40.4/auto/Text/CSV/CSV.so",
|
||||
"/usr/local/bin/csvdiff",
|
||||
}, pkgs[0].Metadata.(pkg.CpanDistribution).Files)
|
||||
}
|
||||
100
syft/pkg/cataloger/perl/parse_release_meta.go
Normal file
100
syft/pkg/cataloger/perl/parse_release_meta.go
Normal file
@ -0,0 +1,100 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"github.com/anchore/syft/internal/log"
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/anchore/syft/syft/file"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/generic"
|
||||
)
|
||||
|
||||
// parseReleaseMeta catalogs the single distribution described by the META.json or META.yml at the root of
|
||||
// an unpacked CPAN distribution: a release tarball or a vendored distribution tree.
|
||||
//
|
||||
// Both formats are read because META.json is not the common case. Measured on MetaCPAN's file index over
|
||||
// 41,722 latest releases, root level only: 23,714 ship a META.json (57%), 37,114 a META.yml (89%), and
|
||||
// 41,008 a MANIFEST (98%). Reading only META.json skips roughly 43% of current releases, skewed toward the
|
||||
// old ones where the advisories concentrate.
|
||||
func parseReleaseMeta(ctx context.Context, resolver file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
|
||||
if isBuildLeftover(reader.Path()) || !isUnpackedRelease(resolver, reader.Location) {
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
isYAML := strings.EqualFold(path.Base(reader.Path()), "META.yml")
|
||||
|
||||
// where both sit in one directory the META.yml invocation yields nothing. META.json is the spec 2
|
||||
// document and is what release tooling generates last, so where they disagree it is the newer
|
||||
// rendering. Skipping at this end keeps the cataloger free of dedup processors.
|
||||
if isYAML && hasSiblingMetaJSON(resolver, reader.Location) {
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
decoder := decodeMeta
|
||||
if isYAML {
|
||||
decoder = decodeMetaYAML
|
||||
}
|
||||
|
||||
m, err := decoder(resolver, reader.Location)
|
||||
if err != nil {
|
||||
// a META.yml a strict parser rejects is routine rather than exceptional, so this skips the
|
||||
// directory and never fails the scan
|
||||
log.WithFields("path", reader.Path(), "error", err).Debug("unable to parse CPAN meta file")
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
if m.Name == "" || m.Version == "" {
|
||||
log.WithFields("path", reader.Path()).Debug("CPAN meta file is missing a name or version")
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
location := reader.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation)
|
||||
licenses := pkg.NewLicensesFromLocationWithContext(ctx, reader.Location, m.licenseExpressions()...)
|
||||
md := pkg.CpanUnpackedRelease{Modules: modulesFromProvides(m.Provides)}
|
||||
|
||||
return []pkg.Package{newCpanPackage(m.Name, string(m.Version), "", md, licenses, location)}, nil, nil
|
||||
}
|
||||
|
||||
// isUnpackedRelease decides whether a meta file describes an unpacked distribution or a source
|
||||
// repository. A committed META.json records the last release and drifts from the working tree
|
||||
// (libwww-perl carries 6.83 in META.json and 6.84 in lib/LWP.pm on the same commit), so it must not be
|
||||
// trusted for version. A sibling MANIFEST is what tells the two apart: release tarballs ship one, while a
|
||||
// source checkout does not, since the MANIFEST is generated at build time.
|
||||
func isUnpackedRelease(resolver file.Resolver, metaLocation file.Location) bool {
|
||||
_, ok := findFile(resolver, path.Join(path.Dir(metaLocation.Path()), "MANIFEST"))
|
||||
|
||||
return ok
|
||||
}
|
||||
|
||||
// buildLeftoverDirs are the trees CPAN clients unpack into. A cpanm work directory genuinely is an
|
||||
// unpacked release tarball, MANIFEST included, so the isUnpackedRelease gate is working correctly and
|
||||
// still admits it; a path exclusion is the only signal available. What makes it a leftover is that a copy
|
||||
// of the same distribution was installed elsewhere in the same tree, and the two would be reported as
|
||||
// separate packages, since two catalogers reporting at different locations can never share a package ID.
|
||||
// Both catalogers run on directory scans, so that duplication is reachable there.
|
||||
//
|
||||
// cpanm leaves ~/.cpanm/work/<epoch>/<Dist>-<Version>/ and CPAN.pm leaves
|
||||
// ~/.cpan/build/<Dist>-<Version>-XXXX/ with a random suffix, neither of which is cleaned up by default,
|
||||
// so any tree that did not rm -rf them reported every distribution it installed twice.
|
||||
//
|
||||
// The honest cost: an unpacked tarball a user deliberately keeps under ~/.cpan/build stops being reported.
|
||||
var buildLeftoverDirs = []string{".cpanm/work", ".cpan/build"}
|
||||
|
||||
func isBuildLeftover(p string) bool {
|
||||
normalized := normalizePath(p)
|
||||
for _, dir := range buildLeftoverDirs {
|
||||
if strings.Contains(normalized, "/"+dir+"/") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func hasSiblingMetaJSON(resolver file.Resolver, metaLocation file.Location) bool {
|
||||
_, ok := findFile(resolver, path.Join(path.Dir(metaLocation.Path()), "META.json"))
|
||||
|
||||
return ok
|
||||
}
|
||||
32
syft/pkg/cataloger/perl/parse_release_meta_test.go
Normal file
32
syft/pkg/cataloger/perl/parse_release_meta_test.go
Normal file
@ -0,0 +1,32 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func Test_isBuildLeftover(t *testing.T) {
|
||||
tests := []struct {
|
||||
path string
|
||||
want bool
|
||||
}{
|
||||
{path: "/root/.cpanm/work/1785183030.2/Clone-0.50/META.json", want: true},
|
||||
{path: "/root/.cpan/build/Encode-3.24-0/META.json", want: true},
|
||||
// a directory scan reports paths relative to its root
|
||||
{path: "root/.cpanm/work/1785183030.2/Clone-0.50/META.yml", want: true},
|
||||
{path: "/home/app/.cpan/build/Text-CSV-2.06-mQ3Ktb/META.json", want: true},
|
||||
// a release deliberately unpacked somewhere else is still reported
|
||||
{path: "/opt/vendor/Text-CSV-2.06/META.json", want: false},
|
||||
{path: "/src/Encode-3.24/META.json", want: false},
|
||||
// the client's own directories are not build trees; only work/ and build/ under them are
|
||||
{path: "/root/.cpanm/latest-build/META.json", want: false},
|
||||
{path: "/root/.cpan/sources/META.json", want: false},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.path, func(t *testing.T) {
|
||||
assert.Equal(t, tt.want, isBuildLeftover(tt.path))
|
||||
})
|
||||
}
|
||||
}
|
||||
224
syft/pkg/cataloger/perl/perllocal.go
Normal file
224
syft/pkg/cataloger/perl/perllocal.go
Normal file
@ -0,0 +1,224 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"io"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"github.com/anchore/syft/internal"
|
||||
"github.com/anchore/syft/internal/log"
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/anchore/syft/syft/file"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
)
|
||||
|
||||
// perllocal.pod stanzas look like this, one per install:
|
||||
//
|
||||
// =head2 Mon Jul 27 20:10:38 2026: C<Module> L<Net::HTTP|Net::HTTP>
|
||||
// =item * C<installed into: /usr/local/lib/perl5/site_perl/5.40.4>
|
||||
// =item * C<VERSION: 6.24>
|
||||
//
|
||||
// The date prefix has changed across ExtUtils::MakeMaker versions (localtime became
|
||||
// gmtime($ENV{SOURCE_DATE_EPOCH} || time) for reproducible builds), so the stanza is recognized by the
|
||||
// C<Module> marker rather than by anything in the heading. Every EUMM version checked emits =head2, so
|
||||
// the heading level is not known to vary; keying on the marker just means nothing here depends on it.
|
||||
var (
|
||||
perllocalModulePattern = regexp.MustCompile(`C<Module>\s+L<([^|>]+)`)
|
||||
perllocalInstalledInto = regexp.MustCompile(`C<installed into:\s*([^>]*)>`)
|
||||
perllocalVersionPattern = regexp.MustCompile(`C<VERSION:\s*([^>]*)>`)
|
||||
)
|
||||
|
||||
// perllocalEntry is one Module stanza of a perllocal.pod, in the order it was written.
|
||||
type perllocalEntry struct {
|
||||
Module string // as written, e.g. Net::HTTP
|
||||
InstalledInto string // the INSTALLSITELIB the installer recorded
|
||||
Version string
|
||||
}
|
||||
|
||||
// perllocalPod is a parsed perllocal.pod together with where it was found, since the file that supplied
|
||||
// a version is recorded as supporting evidence on the package.
|
||||
type perllocalPod struct {
|
||||
location file.Location
|
||||
entries []perllocalEntry
|
||||
}
|
||||
|
||||
// resolvePacklistVersions fills in the version of every package that came from a .packlist.
|
||||
//
|
||||
// perllocal.pod is the primary source because ExtUtils::MakeMaker writes it and the packlist from the
|
||||
// same variables in the same install target: the packlist goes to $(SITEARCHEXP)/auto/$(FULLEXT)/.packlist
|
||||
// while the stanza records "Module" "$(NAME)" with VERSION "$(VERSION)", and FULLEXT is NAME with :: turned
|
||||
// into /. So the auto/ path segments and the stanza's module name are the same key. $(VERSION) is what EUMM
|
||||
// evaluated at build time, so a version the module computes rather than declares is recorded correctly,
|
||||
// which static .pm parsing cannot do at all.
|
||||
//
|
||||
// This is a processor rather than a per-packlist read because perllocal.pod is not a sibling of anything:
|
||||
// EUMM writes it to DESTINSTALLARCHLIB, the core arch directory, even for site installs, so it has to be
|
||||
// found with a resolver glob. There is one per install tree against many packlists, so it is parsed once
|
||||
// here rather than once per packlist.
|
||||
//
|
||||
// It must run before mergeDistributions. Merging pairs a packlist package to its install.json package only
|
||||
// when the versions agree or the packlist version is absent, so filling the version in afterwards would
|
||||
// leave every distribution whose two evidence kinds disagree split into two packages.
|
||||
func resolvePacklistVersions(_ context.Context, resolver file.Resolver, pkgs []pkg.Package, rels []artifact.Relationship, err error) ([]pkg.Package, []artifact.Relationship, error) {
|
||||
if resolver == nil || len(pkgs) == 0 {
|
||||
return pkgs, rels, err
|
||||
}
|
||||
|
||||
var pods []perllocalPod
|
||||
for _, location := range perllocalLocations(resolver) {
|
||||
if entries := readPerllocal(resolver, location); len(entries) > 0 {
|
||||
pods = append(pods, perllocalPod{location: location, entries: entries})
|
||||
}
|
||||
}
|
||||
|
||||
for i := range pkgs {
|
||||
md, ok := pkgs[i].Metadata.(pkg.CpanDistribution)
|
||||
if !ok || md.MainModule == "" || pkgs[i].Version != "" {
|
||||
// not a packlist-derived package, so there is nothing to resolve
|
||||
continue
|
||||
}
|
||||
|
||||
version, evidence := packlistVersion(resolver, pods, pkgs[i], md)
|
||||
if version == "" && evidence == nil {
|
||||
// still emitted with an empty version rather than dropped, so the distribution stays visible
|
||||
continue
|
||||
}
|
||||
|
||||
pkgs[i].Version = version
|
||||
pkgs[i].PURL = packageURL(pkgs[i].Name, version, md.Author)
|
||||
if evidence != nil {
|
||||
locations := pkgs[i].Locations
|
||||
locations.Add(evidence.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.SupportingEvidenceAnnotation))
|
||||
pkgs[i].Locations = locations
|
||||
}
|
||||
|
||||
// Version and Locations both participate in the package ID
|
||||
pkgs[i].SetID()
|
||||
}
|
||||
|
||||
return pkgs, rels, err
|
||||
}
|
||||
|
||||
// packlistVersion resolves one packlist package's version, preferring perllocal.pod and falling back to
|
||||
// scraping $VERSION out of the distribution's main .pm. The fallback is not vestigial: NO_PERLLOCAL
|
||||
// suppresses the file the same way NO_PACKLIST suppresses the packlist, and Module::Build and
|
||||
// Module::Build::Tiny write packlists but no perllocal.pod at all.
|
||||
func packlistVersion(resolver file.Resolver, pods []perllocalPod, p pkg.Package, md pkg.CpanDistribution) (string, *file.Location) {
|
||||
packlistPath := primaryPath(p)
|
||||
|
||||
if version, location := perllocalVersion(pods, p.Name, packlistPath); version != "" {
|
||||
return version, location
|
||||
}
|
||||
|
||||
pmLocation, ok := findFile(resolver, mainModuleFile(p.Name, md.Files))
|
||||
if !ok {
|
||||
return "", nil
|
||||
}
|
||||
|
||||
return readVersionFromPM(resolver, pmLocation), &pmLocation
|
||||
}
|
||||
|
||||
// perllocalVersion picks the stanza that describes the packlist at packlistPath, by three rules:
|
||||
//
|
||||
// 1. Key on the dashed module name: auto/Net/HTTP/.packlist and "Module Net::HTTP" both give Net-HTTP.
|
||||
// 2. The stanza's "installed into" libdir must be a path prefix of the packlist, since in every standard
|
||||
// perl layout the arch directory nests under the lib directory. Where an image carries more than one
|
||||
// perl, the core libdir of one tree can also prefix-match another tree's packlist, so the longest
|
||||
// matching libdir wins. Without that a version from the wrong perl silently wins, which is worse than
|
||||
// no version at all.
|
||||
// 3. The file is append-only, so an upgrade leaves a stale stanza beside the current one forever. Stanzas
|
||||
// are chronological, so the last match wins. The dates are not parsed; file order is the ordering.
|
||||
func perllocalVersion(pods []perllocalPod, dashedName, packlistPath string) (string, *file.Location) {
|
||||
var (
|
||||
version string
|
||||
evidence *file.Location
|
||||
bestLibdir int
|
||||
)
|
||||
|
||||
for i := range pods {
|
||||
for _, entry := range pods[i].entries {
|
||||
if entry.Version == "" || strings.ReplaceAll(entry.Module, "::", "-") != dashedName {
|
||||
continue
|
||||
}
|
||||
|
||||
libdir := strings.TrimSuffix(entry.InstalledInto, "/")
|
||||
if libdir == "" || !strings.HasPrefix(normalizePath(packlistPath), normalizePath(libdir)+"/") {
|
||||
continue
|
||||
}
|
||||
|
||||
// >= rather than > so that the last stanza of an equally specific libdir wins
|
||||
if len(libdir) >= bestLibdir {
|
||||
version, evidence, bestLibdir = entry.Version, &pods[i].location, len(libdir)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return version, evidence
|
||||
}
|
||||
|
||||
// normalizePath makes the resolver's relative paths and perllocal's absolute libdirs comparable. A
|
||||
// directory scan reports paths relative to its root, while perllocal records what the installer saw.
|
||||
func normalizePath(p string) string {
|
||||
return "/" + strings.TrimPrefix(p, "/")
|
||||
}
|
||||
|
||||
func perllocalLocations(resolver file.Resolver) []file.Location {
|
||||
locations, err := resolver.FilesByGlob("**/perllocal.pod")
|
||||
if err != nil {
|
||||
log.WithFields("error", err).Debug("unable to search for perllocal.pod")
|
||||
return nil
|
||||
}
|
||||
return locations
|
||||
}
|
||||
|
||||
func readPerllocal(resolver file.Resolver, location file.Location) []perllocalEntry {
|
||||
reader, err := resolver.FileContentsByLocation(location)
|
||||
if err != nil {
|
||||
log.WithFields("path", location.Path(), "error", err).Debug("unable to read perllocal.pod")
|
||||
return nil
|
||||
}
|
||||
defer internal.CloseAndLogError(reader, location.Path())
|
||||
|
||||
return parsePerllocal(reader)
|
||||
}
|
||||
|
||||
func parsePerllocal(r io.Reader) []perllocalEntry {
|
||||
var entries []perllocalEntry
|
||||
|
||||
scanner := bufio.NewScanner(r)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
|
||||
if match := perllocalModulePattern.FindStringSubmatch(line); match != nil {
|
||||
entries = append(entries, perllocalEntry{Module: strings.TrimSpace(match[1])})
|
||||
continue
|
||||
}
|
||||
|
||||
if len(entries) == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
current := &entries[len(entries)-1]
|
||||
if match := perllocalInstalledInto.FindStringSubmatch(line); match != nil {
|
||||
current.InstalledInto = strings.TrimSpace(match[1])
|
||||
}
|
||||
if match := perllocalVersionPattern.FindStringSubmatch(line); match != nil {
|
||||
current.Version = strings.TrimSpace(match[1])
|
||||
}
|
||||
}
|
||||
|
||||
return entries
|
||||
}
|
||||
|
||||
// primaryPath returns the path of the evidence the package was built from, which for a packlist package
|
||||
// is the .packlist itself.
|
||||
func primaryPath(p pkg.Package) string {
|
||||
for _, location := range p.Locations.ToSlice() {
|
||||
if location.Annotations[pkg.EvidenceAnnotationKey] == pkg.PrimaryEvidenceAnnotation {
|
||||
return location.Path()
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
170
syft/pkg/cataloger/perl/perllocal_test.go
Normal file
170
syft/pkg/cataloger/perl/perllocal_test.go
Normal file
@ -0,0 +1,170 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"github.com/anchore/syft/syft/file"
|
||||
)
|
||||
|
||||
func Test_parsePerllocal(t *testing.T) {
|
||||
// two stanzas as ExtUtils::MakeMaker writes them, with the second module upgraded in place so both a
|
||||
// stale entry and a current one are present. Real EUMM emits =head2; the =head1 here is a deliberate
|
||||
// variation, to pin that the parser keys on the C<Module> marker and not on the heading level.
|
||||
const pod = `=head0 L<Perl documentation|perl>
|
||||
|
||||
=head1 Sun Jul 26 18:04:12 2026: C<Module> L<Encode|Encode>
|
||||
|
||||
=over 4
|
||||
|
||||
=item *
|
||||
|
||||
C<installed into: /usr/local/lib/perl5/site_perl/5.40.4>
|
||||
|
||||
=item *
|
||||
|
||||
C<LINKTYPE: dynamic>
|
||||
|
||||
=item *
|
||||
|
||||
C<VERSION: 3.24>
|
||||
|
||||
=item *
|
||||
|
||||
C<EXE_FILES: bin/enc2xs>
|
||||
|
||||
=back
|
||||
|
||||
=head2 Mon Jul 27 20:10:38 2026: C<Module> L<Net::HTTP|Net::HTTP>
|
||||
|
||||
=over 4
|
||||
|
||||
=item *
|
||||
|
||||
C<installed into: /usr/local/lib/perl5/site_perl/5.40.4>
|
||||
|
||||
=item *
|
||||
|
||||
C<VERSION: 6.24>
|
||||
|
||||
=back
|
||||
`
|
||||
|
||||
assert.Equal(t, []perllocalEntry{
|
||||
{Module: "Encode", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4", Version: "3.24"},
|
||||
{Module: "Net::HTTP", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4", Version: "6.24"},
|
||||
}, parsePerllocal(strings.NewReader(pod)))
|
||||
}
|
||||
|
||||
func Test_parsePerllocal_degrades(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
contents string
|
||||
}{
|
||||
{name: "empty", contents: ""},
|
||||
{name: "no module stanzas", contents: "=head0 L<Perl documentation|perl>\n"},
|
||||
// a stanza body with no C<Module> heading before it has nothing to attach to
|
||||
{name: "orphaned body", contents: "=item *\n\nC<VERSION: 1.00>\n"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
assert.Empty(t, parsePerllocal(strings.NewReader(tt.contents)))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func Test_perllocalVersion(t *testing.T) {
|
||||
pods := []perllocalPod{
|
||||
{
|
||||
location: file.NewLocation("/usr/local/lib/perl5/5.40.4/aarch64-linux-gnu/perllocal.pod"),
|
||||
entries: []perllocalEntry{
|
||||
{Module: "Net::HTTP", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4", Version: "6.24"},
|
||||
// the file is append-only, so an upgrade leaves the earlier stanza in place forever
|
||||
{Module: "Text::CSV", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4", Version: "2.02"},
|
||||
{Module: "Text::CSV", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4", Version: "2.06"},
|
||||
// a stanza whose libdir belongs to another install tree entirely
|
||||
{Module: "JSON::PP", InstalledInto: "/opt/perl-5.36.0/lib/perl5/site_perl/5.36.0", Version: "2.27300"},
|
||||
// an INSTALL_BASE install and a local::lib nested inside it both prefix-match one packlist
|
||||
{Module: "Try::Tiny", InstalledInto: "/opt/app/lib/perl5", Version: "0.30"},
|
||||
{Module: "Try::Tiny", InstalledInto: "/opt/app/lib/perl5/local", Version: "0.32"},
|
||||
{Module: "Carp", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4"},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
dashedName string
|
||||
packlistPath string
|
||||
want string
|
||||
}{
|
||||
{
|
||||
name: "module name dashes to the packlist name",
|
||||
dashedName: "Net-HTTP",
|
||||
packlistPath: "/usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/Net/HTTP/.packlist",
|
||||
want: "6.24",
|
||||
},
|
||||
{
|
||||
name: "the last stanza wins over a stale one",
|
||||
dashedName: "Text-CSV",
|
||||
packlistPath: "/usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/Text/CSV/.packlist",
|
||||
want: "2.06",
|
||||
},
|
||||
{
|
||||
// without the libdir rule the other perl's version would silently win, which is worse than
|
||||
// reporting no version at all
|
||||
name: "a stanza from another install tree is not used",
|
||||
dashedName: "JSON-PP",
|
||||
packlistPath: "/usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/JSON/PP/.packlist",
|
||||
want: "",
|
||||
},
|
||||
{
|
||||
name: "the longest matching libdir wins",
|
||||
dashedName: "Try-Tiny",
|
||||
packlistPath: "/opt/app/lib/perl5/local/aarch64-linux-gnu/auto/Try/Tiny/.packlist",
|
||||
want: "0.32",
|
||||
},
|
||||
{
|
||||
name: "the outer libdir still matches its own packlist",
|
||||
dashedName: "Try-Tiny",
|
||||
packlistPath: "/opt/app/lib/perl5/aarch64-linux-gnu/auto/Try/Tiny/.packlist",
|
||||
want: "0.30",
|
||||
},
|
||||
{
|
||||
// a directory scan reports paths relative to its root while perllocal records absolute libdirs
|
||||
name: "relative packlist path from a directory scan",
|
||||
dashedName: "Net-HTTP",
|
||||
packlistPath: "usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/Net/HTTP/.packlist",
|
||||
want: "6.24",
|
||||
},
|
||||
{
|
||||
name: "no stanza for this distribution",
|
||||
dashedName: "IO-Socket-SSL",
|
||||
packlistPath: "/usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/IO/Socket/SSL/.packlist",
|
||||
want: "",
|
||||
},
|
||||
{
|
||||
name: "a stanza with no VERSION is not a match",
|
||||
dashedName: "Carp",
|
||||
packlistPath: "/usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/Carp/.packlist",
|
||||
want: "",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
version, evidence := perllocalVersion(pods, tt.dashedName, tt.packlistPath)
|
||||
assert.Equal(t, tt.want, version)
|
||||
if tt.want == "" {
|
||||
assert.Nil(t, evidence)
|
||||
return
|
||||
}
|
||||
require.NotNil(t, evidence)
|
||||
assert.Equal(t, "/usr/local/lib/perl5/5.40.4/aarch64-linux-gnu/perllocal.pod", evidence.Path())
|
||||
})
|
||||
}
|
||||
}
|
||||
225
syft/pkg/cataloger/perl/processor.go
Normal file
225
syft/pkg/cataloger/perl/processor.go
Normal file
@ -0,0 +1,225 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"github.com/anchore/syft/syft/artifact"
|
||||
"github.com/anchore/syft/syft/file"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
"github.com/anchore/syft/syft/pkg/cataloger/internal/dependency"
|
||||
)
|
||||
|
||||
// mergeDistributions collapses the two kinds of installed evidence into one package per installed
|
||||
// distribution. A distribution installed by cpanm has both an install.json and a .packlist.
|
||||
//
|
||||
// Pairing is scoped to one install tree, so two installs of the same distribution and version into
|
||||
// separate trees stay two packages. A scan target can hold more than one project tree, and preserving
|
||||
// that there are two trees is the point: collapsing them asserts one installed thing where there are
|
||||
// two and loses which tree each belongs to. Multiple projects in one scan target are treated the same
|
||||
// way elsewhere.
|
||||
//
|
||||
// (name, version) alone is not enough to pair them either. A packlist lives at auto/<Module>/.packlist, so its
|
||||
// name is derived from a MODULE path, while install.json yields the distribution: libwww-perl's packlist
|
||||
// is auto/LWP/.packlist and produces "LWP" against the distribution "libwww-perl". Those never collide,
|
||||
// so pairing also consults the `provides` module map, which lists LWP under libwww-perl. Without that
|
||||
// step every distribution whose main module is named differently from the distribution produces two
|
||||
// packages, one with the right name and no version and one with the wrong name and the right version,
|
||||
// and neither one matches an advisory.
|
||||
func mergeDistributions(pkgs []pkg.Package, rels []artifact.Relationship, err error) ([]pkg.Package, []artifact.Relationship, error) {
|
||||
if len(pkgs) < 2 {
|
||||
return pkgs, rels, err
|
||||
}
|
||||
|
||||
var merged []pkg.Package
|
||||
indexByKey := make(map[string]int)
|
||||
|
||||
// install.json evidence is placed first so that the module index is complete before any packlist
|
||||
// package needs to resolve against it, independent of the order the parsers ran in
|
||||
ordered := make([]pkg.Package, 0, len(pkgs))
|
||||
for _, p := range pkgs {
|
||||
if fromInstallJSON(p) {
|
||||
ordered = append(ordered, p)
|
||||
}
|
||||
}
|
||||
for _, p := range pkgs {
|
||||
if !fromInstallJSON(p) {
|
||||
ordered = append(ordered, p)
|
||||
}
|
||||
}
|
||||
|
||||
indexByModule := make(map[string]int)
|
||||
|
||||
for _, p := range ordered {
|
||||
tree := installTree(p)
|
||||
key := tree + "|" + p.Name + "@" + p.Version
|
||||
if idx, exists := indexByKey[key]; exists {
|
||||
merged[idx] = mergeInto(merged[idx], p)
|
||||
continue
|
||||
}
|
||||
|
||||
// a packlist package names a module, so fold it into whichever distribution in the same tree
|
||||
// provides it. The version must still agree, or be unreadable: a packlist and an install.json
|
||||
// in one tree that disagree describe one install that overwrote another's files, and merging
|
||||
// them would report a single version and hide the other.
|
||||
if !fromInstallJSON(p) {
|
||||
if idx, exists := indexByModule[tree+"|"+moduleFromDashedName(p.Name)]; exists &&
|
||||
(p.Version == "" || p.Version == merged[idx].Version) {
|
||||
merged[idx] = mergeInto(merged[idx], p)
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
indexByKey[key] = len(merged)
|
||||
for _, m := range providedModules(p) {
|
||||
// first distribution to claim a module wins; duplicate claims are not expected and
|
||||
// silently reassigning would make the result order dependent
|
||||
if _, taken := indexByModule[tree+"|"+m]; !taken {
|
||||
indexByModule[tree+"|"+m] = len(merged)
|
||||
}
|
||||
}
|
||||
merged = append(merged, p)
|
||||
}
|
||||
|
||||
// locations and metadata drive the package ID, and both may have changed
|
||||
for i := range merged {
|
||||
merged[i].SetID()
|
||||
}
|
||||
|
||||
return merged, rels, err
|
||||
}
|
||||
|
||||
// installTree returns the library tree the package's own evidence was found in: the directory holding
|
||||
// the .meta or auto directory the file sits under. Both kinds of evidence for one install land there,
|
||||
// in sibling directories rather than the same one, so
|
||||
// /opt/app/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json and
|
||||
// /opt/app/lib/perl5/x86_64-linux-gnu/auto/Text/CSV/.packlist both yield
|
||||
// /opt/app/lib/perl5/x86_64-linux-gnu and still pair.
|
||||
//
|
||||
// The owned-file list cannot stand in for this: ExtUtils::Packlist merges an existing packlist for the
|
||||
// same module when it writes a new one, so a local-lib install's packlist legitimately claims files
|
||||
// living outside its own prefix.
|
||||
func installTree(p pkg.Package) string {
|
||||
for _, location := range p.Locations.ToSlice() {
|
||||
if tree := libRoot(location.Path()); tree != "" {
|
||||
return tree
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// libRoot cuts a path at the last auto/ or .meta/ segment, which is where a lib tree ends and the
|
||||
// installer's own bookkeeping begins. A path with neither, such as a perllocal.pod or a scraped .pm,
|
||||
// yields "": those are supporting evidence and never the only location a package has.
|
||||
func libRoot(p string) string {
|
||||
parts := strings.Split(p, "/")
|
||||
for i := len(parts) - 1; i > 0; i-- {
|
||||
if parts[i] == "auto" || parts[i] == ".meta" {
|
||||
return strings.Join(parts[:i], "/")
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// moduleFromDashedName converts a packlist-derived name back into the module it came from, since
|
||||
// auto/IO/Socket/SSL/.packlist flattens IO::Socket::SSL into IO-Socket-SSL.
|
||||
func moduleFromDashedName(name string) string {
|
||||
return strings.ReplaceAll(name, "-", "::")
|
||||
}
|
||||
|
||||
// mergeInto keeps the install.json-derived package, which is the higher fidelity of the two, and folds
|
||||
// the other's evidence into it.
|
||||
func mergeInto(a, b pkg.Package) pkg.Package {
|
||||
winner, loser := a, b
|
||||
if !fromInstallJSON(a) && fromInstallJSON(b) {
|
||||
winner, loser = b, a
|
||||
}
|
||||
|
||||
locations := file.NewLocationSet(winner.Locations.ToSlice()...)
|
||||
locations.Add(loser.Locations.ToSlice()...)
|
||||
winner.Locations = locations
|
||||
|
||||
licenses := pkg.NewLicenseSet(winner.Licenses.ToSlice()...)
|
||||
licenses.Add(loser.Licenses.ToSlice()...)
|
||||
winner.Licenses = licenses
|
||||
|
||||
// the file list only ever comes from a packlist, so the install.json winner has to take the loser's or
|
||||
// the merged package would own nothing. OwnedFiles dedupes, which covers a distribution that merged
|
||||
// more than one packlist.
|
||||
from, fromOK := loser.Metadata.(pkg.CpanDistribution)
|
||||
into, intoOK := winner.Metadata.(pkg.CpanDistribution)
|
||||
if fromOK && intoOK && len(from.Files) > 0 {
|
||||
into.Files = append(into.Files, from.Files...)
|
||||
winner.Metadata = into
|
||||
}
|
||||
|
||||
return winner
|
||||
}
|
||||
|
||||
// fromInstallJSON reports whether a package was built from an install.json rather than from a
|
||||
// .packlist. MainModule is only ever set by the packlist parser, so its absence is the direct signal.
|
||||
func fromInstallJSON(p pkg.Package) bool {
|
||||
md, ok := p.Metadata.(pkg.CpanDistribution)
|
||||
return ok && md.MainModule == ""
|
||||
}
|
||||
|
||||
// resolveDependencies turns MYMETA.json prereqs into relationships. Prereqs name modules while packages
|
||||
// are keyed by distribution, so each prereq is looked up in the provides map of the cataloged set. A
|
||||
// prereq that resolves to nothing (a core module such as Carp, strict, or perl, or a distribution that
|
||||
// simply is not installed) is dropped rather than turned into a placeholder package.
|
||||
func resolveDependencies(_ context.Context, resolver file.Resolver, pkgs []pkg.Package, rels []artifact.Relationship, err error) ([]pkg.Package, []artifact.Relationship, error) {
|
||||
if resolver == nil || len(pkgs) == 0 {
|
||||
return pkgs, rels, err
|
||||
}
|
||||
|
||||
requires := make(map[artifact.ID][]string)
|
||||
for i := range pkgs {
|
||||
if pkgs[i].ID() == "" {
|
||||
pkgs[i].SetID()
|
||||
}
|
||||
requires[pkgs[i].ID()] = prereqModules(resolver, pkgs[i])
|
||||
}
|
||||
|
||||
specifier := func(p pkg.Package) dependency.Specification {
|
||||
return dependency.Specification{
|
||||
ProvidesRequires: dependency.ProvidesRequires{
|
||||
Provides: providedModules(p),
|
||||
Requires: requires[p.ID()],
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
return pkgs, append(rels, dependency.Resolve(specifier, pkgs)...), err
|
||||
}
|
||||
|
||||
func providedModules(p pkg.Package) []string {
|
||||
md, ok := p.Metadata.(pkg.CpanDistribution)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
|
||||
modules := make([]string, 0, len(md.Modules))
|
||||
for _, m := range md.Modules {
|
||||
modules = append(modules, m.Name)
|
||||
}
|
||||
|
||||
return modules
|
||||
}
|
||||
|
||||
// prereqModules re-reads the MYMETA.json already recorded as evidence on the package. It is read a
|
||||
// second time (the parser reads it for licenses) rather than carried on the package, since prereqs
|
||||
// belong in relationships and not in the metadata struct.
|
||||
func prereqModules(resolver file.Resolver, p pkg.Package) []string {
|
||||
for _, location := range p.Locations.ToSlice() {
|
||||
if path.Base(location.Path()) != "MYMETA.json" {
|
||||
continue
|
||||
}
|
||||
m, err := decodeMeta(resolver, location)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
return m.prereqModules()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
100
syft/pkg/cataloger/perl/processor_test.go
Normal file
100
syft/pkg/cataloger/perl/processor_test.go
Normal file
@ -0,0 +1,100 @@
|
||||
package perl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"github.com/anchore/syft/syft/file"
|
||||
"github.com/anchore/syft/syft/pkg"
|
||||
)
|
||||
|
||||
func Test_mergeDistributions(t *testing.T) {
|
||||
fromInstallJSON := newCpanPackage("URI", "5.35", "OALDERS",
|
||||
pkg.CpanDistribution{
|
||||
Dist: "URI-5.35",
|
||||
Author: "OALDERS",
|
||||
Path: "O/OA/OALDERS/URI-5.35.tar.gz",
|
||||
Modules: []pkg.CpanModule{{Name: "URI", Version: "5.35"}},
|
||||
},
|
||||
[]pkg.License{pkg.NewLicenseWithContext(context.Background(), "perl_5")},
|
||||
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/.meta/URI-5.35/install.json"),
|
||||
)
|
||||
|
||||
fromPacklist := newCpanPackage("URI", "5.35", "", pkg.CpanDistribution{MainModule: "URI"}, nil,
|
||||
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/URI/.packlist"),
|
||||
)
|
||||
|
||||
otherVersion := newCpanPackage("URI", "5.34", "", pkg.CpanDistribution{MainModule: "URI"}, nil,
|
||||
file.NewLocation("/opt/lib/perl5/x86_64-linux/auto/URI/.packlist"),
|
||||
)
|
||||
|
||||
// the packlist arriving first must not change which package wins
|
||||
for _, order := range [][]pkg.Package{
|
||||
{fromInstallJSON, fromPacklist, otherVersion},
|
||||
{fromPacklist, fromInstallJSON, otherVersion},
|
||||
} {
|
||||
merged, _, err := mergeDistributions(order, nil, nil)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, merged, 2)
|
||||
|
||||
uri := merged[0]
|
||||
assert.Equal(t, "5.35", uri.Version)
|
||||
assert.Equal(t, "pkg:cpan/URI@5.35?author=OALDERS", uri.PURL)
|
||||
assert.Equal(t, "OALDERS", uri.Metadata.(pkg.CpanDistribution).Author)
|
||||
assert.Len(t, uri.Licenses.ToSlice(), 1)
|
||||
assert.Len(t, uri.Locations.ToSlice(), 2, "the packlist should be kept as additional evidence")
|
||||
|
||||
// a different version of the same distribution is a different package
|
||||
assert.Equal(t, "5.34", merged[1].Version)
|
||||
}
|
||||
}
|
||||
|
||||
// a packlist lives at auto/<Module>/.packlist, so for any distribution whose main module is named
|
||||
// differently from the distribution the two kinds of evidence never share a (name, version) key.
|
||||
// libwww-perl is the canonical case: auto/LWP/.packlist against the distribution libwww-perl.
|
||||
func Test_mergeDistributions_packlistNamedAfterModule(t *testing.T) {
|
||||
fromInstallJSON := newCpanPackage("libwww-perl", "5.836", "GAAS",
|
||||
pkg.CpanDistribution{
|
||||
Dist: "libwww-perl-5.836",
|
||||
Author: "GAAS",
|
||||
Path: "G/GA/GAAS/libwww-perl-5.836.tar.gz",
|
||||
Modules: []pkg.CpanModule{{Name: "LWP", Version: "5.836"}, {Name: "LWP::UserAgent", Version: "5.834"}},
|
||||
},
|
||||
nil,
|
||||
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/.meta/libwww-perl-5.836/install.json"),
|
||||
)
|
||||
|
||||
fromPacklist := newCpanPackage("LWP", "5.836", "", pkg.CpanDistribution{MainModule: "LWP"}, nil,
|
||||
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/LWP/.packlist"),
|
||||
)
|
||||
|
||||
// a packlist whose version could not be read still belongs to the distribution providing it
|
||||
versionless := newCpanPackage("LWP-UserAgent", "", "", pkg.CpanDistribution{MainModule: "LWP::UserAgent"}, nil,
|
||||
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/LWP/UserAgent/.packlist"),
|
||||
)
|
||||
|
||||
// the same module installed into a second lib tree is a second install, and the module index is
|
||||
// scoped to a tree so that it cannot be claimed by the distribution in the first one
|
||||
otherTree := newCpanPackage("LWP", "5.836", "", pkg.CpanDistribution{MainModule: "LWP"}, nil,
|
||||
file.NewLocation("/opt/lib/perl5/x86_64-linux/auto/LWP/.packlist"),
|
||||
)
|
||||
|
||||
for _, order := range [][]pkg.Package{
|
||||
{fromInstallJSON, fromPacklist, versionless, otherTree},
|
||||
{otherTree, versionless, fromPacklist, fromInstallJSON},
|
||||
} {
|
||||
merged, _, err := mergeDistributions(order, nil, nil)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, merged, 2, "the same tree's module-named packlists belong to libwww-perl, the other tree's does not")
|
||||
|
||||
assert.Equal(t, "libwww-perl", merged[0].Name)
|
||||
assert.Equal(t, "pkg:cpan/libwww-perl@5.836?author=GAAS", merged[0].PURL)
|
||||
assert.Len(t, merged[0].Locations.ToSlice(), 3)
|
||||
|
||||
assert.Equal(t, "LWP", merged[1].Name)
|
||||
assert.Equal(t, []string{"/opt/lib/perl5/x86_64-linux/auto/LWP/.packlist"}, realPaths(merged[1]))
|
||||
}
|
||||
}
|
||||
44
syft/pkg/cataloger/perl/testdata/image-cpan-build-leftovers/Dockerfile
vendored
Normal file
44
syft/pkg/cataloger/perl/testdata/image-cpan-build-leftovers/Dockerfile
vendored
Normal file
@ -0,0 +1,44 @@
|
||||
# an image that installed distributions and did not clean up after itself, which is the ordinary
|
||||
# state of anything built with a CPAN client: cpanm leaves ~/.cpanm/work and CPAN.pm leaves
|
||||
# ~/.cpan/build, neither by choice of the user. Both leftovers genuinely are unpacked release
|
||||
# tarballs, MANIFEST included, so nothing about their contents disqualifies them and they have to be
|
||||
# excluded on the path.
|
||||
#
|
||||
# A release deliberately unpacked outside any build directory sits beside the installed copy of the
|
||||
# same distribution and version, so the same image shows both halves of the path rule: the leftover
|
||||
# is skipped and the vendored tree is not.
|
||||
#
|
||||
# The digest is perl:5.40-slim's linux/amd64 manifest; see image-cpan-mirror-installs/Dockerfile.
|
||||
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
|
||||
|
||||
# cpanm, mirror-resolved so these carry install.json, and pinned by PAUSE path so the versions do
|
||||
# not drift. The unpacked build trees are left in /root/.cpanm/work.
|
||||
RUN cpanm --notest \
|
||||
R/RE/REHSACK/MIME-Base32-1.303.tar.gz \
|
||||
O/OA/OALDERS/URI-5.35.tar.gz
|
||||
|
||||
# CPAN.pm, which writes no .meta and leaves its build tree in /root/.cpan/build under a
|
||||
# randomly suffixed directory
|
||||
RUN PERL_MM_USE_DEFAULT=1 cpan -T -i P/PE/PETDANCE/HTML-Tagset-3.24.tar.gz
|
||||
|
||||
# the same release, unpacked somewhere a user would keep it on purpose. Nothing pairs this with the
|
||||
# installed copy: the two come from different catalogers, and two packages found at different
|
||||
# locations can never share a package ID.
|
||||
RUN mkdir -p /opt/vendor \
|
||||
&& curl -sSfL https://backpan.perl.org/authors/id/P/PE/PETDANCE/HTML-Tagset-3.24.tar.gz \
|
||||
| tar xz -C /opt/vendor
|
||||
|
||||
# the clients' download caches under ~/.cpanm/sources and ~/.cpan/sources are not evidence of
|
||||
# anything and are the bulk of the bytes, so only the unpacked build trees come along
|
||||
RUN mkdir -p /out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu /out/root/.cpanm /out/root/.cpan /out/opt \
|
||||
&& cp -a /usr/local/lib/perl5/site_perl /out/usr/local/lib/perl5/site_perl \
|
||||
&& cp -a /usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod \
|
||||
/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/.packlist \
|
||||
/out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/ \
|
||||
&& cp -a /root/.cpanm/work /out/root/.cpanm/work \
|
||||
&& cp -a /root/.cpan/build /out/root/.cpan/build \
|
||||
&& cp -a /opt/vendor /out/opt/vendor
|
||||
|
||||
FROM scratch
|
||||
|
||||
COPY --from=builder /out/ /
|
||||
84
syft/pkg/cataloger/perl/testdata/image-cpan-mirror-installs/Dockerfile
vendored
Normal file
84
syft/pkg/cataloger/perl/testdata/image-cpan-mirror-installs/Dockerfile
vendored
Normal file
@ -0,0 +1,84 @@
|
||||
# mirror-resolved CPAN client installs: cpanm, cpm and carton. This is the only case where an
|
||||
# install.json is written, so it is also the only image carrying MYMETA.json and a `provides` map.
|
||||
#
|
||||
# The digest is perl:5.40-slim's linux/amd64 manifest rather than its index, which pins the
|
||||
# architecture as well as the contents: perl's lib layout embeds the arch name
|
||||
# (.../5.40.4/x86_64-linux-gnu/), so a build on an arm host would move every path under test.
|
||||
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
|
||||
|
||||
# Clone and HTML::Parser are XS and the slim image ships no compiler
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends gcc libc6-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# cpm and carton go into a throwaway prefix: their own dependency trees are large and are not under
|
||||
# test, and nothing from /tools reaches the final image
|
||||
RUN cpanm --notest --quiet -L /tools App::cpm Carton
|
||||
|
||||
# cpanm, site install of libwww-perl and its whole dependency closure.
|
||||
#
|
||||
# LWP is the module name; the distribution is libwww-perl, so this is the name mismatch that makes
|
||||
# install.json's `name` field unusable as a package name, and it installs to auto/LWP/.packlist
|
||||
# rather than auto/libwww-perl/. The closure comes along on purpose: it is what gives the image real
|
||||
# MYMETA.json prereqs to resolve into relationships.
|
||||
#
|
||||
# Every distribution is named by its PAUSE path, which is what pins the version. A bare module name
|
||||
# resolves to whatever the index says today, so the closure would drift the moment any of these
|
||||
# authors releases again. cpanm still treats a PAUSE path as a mirror resolution, so install.json is
|
||||
# written for all of them, which a URL or a local tarball would not do. Dependencies come first so
|
||||
# that nothing in the list is ever satisfied by an unpinned fetch.
|
||||
RUN cpanm --notest \
|
||||
E/ET/ETHER/Try-Tiny-0.32.tar.gz \
|
||||
A/AT/ATOOMIC/Clone-0.50.tar.gz \
|
||||
G/GA/GAAS/Encode-Locale-1.05.tar.gz \
|
||||
P/PE/PETDANCE/HTML-Tagset-3.24.tar.gz \
|
||||
C/CJ/CJM/IO-HTML-1.004.tar.gz \
|
||||
O/OA/OALDERS/LWP-MediaTypes-6.04.tar.gz \
|
||||
R/RE/REHSACK/MIME-Base32-1.303.tar.gz \
|
||||
A/AT/ATOOMIC/TimeDate-2.35.tar.gz \
|
||||
O/OA/OALDERS/URI-5.35.tar.gz \
|
||||
O/OA/OALDERS/HTTP-Date-6.08.tar.gz \
|
||||
O/OA/OALDERS/HTTP-Message-7.04.tar.gz \
|
||||
O/OA/OALDERS/HTML-Parser-3.85.tar.gz \
|
||||
P/PL/PLICEASE/File-Listing-6.16.tar.gz \
|
||||
O/OA/OALDERS/HTTP-Cookies-6.12.tar.gz \
|
||||
G/GA/GAAS/HTTP-Negotiate-6.01.tar.gz \
|
||||
O/OA/OALDERS/Net-HTTP-6.24.tar.gz \
|
||||
O/OA/OALDERS/WWW-RobotRules-6.03.tar.gz \
|
||||
O/OA/OALDERS/libwww-perl-6.83.tar.gz
|
||||
|
||||
# a distribution installed the way distro packagers do it: install.json and no packlist at all.
|
||||
# NO_PACKLIST and NO_PERLLOCAL are ExtUtils::MakeMaker knobs, so this has to be an EUMM
|
||||
# distribution; a Module::Build::Tiny one would ignore both and write its packlist anyway.
|
||||
RUN PERL_MM_OPT="NO_PACKLIST=1 NO_PERLLOCAL=1" cpanm --notest I/IS/ISHIGAKI/JSON-PP-4.16.tar.gz
|
||||
|
||||
# carton, writing local/lib/perl5 beside the cpanfile it read. The cpanfile and cpanfile.snapshot
|
||||
# are deliberately left in place and deliberately not parsed by the cataloger.
|
||||
RUN mkdir -p /app \
|
||||
&& printf "requires 'Text::CSV', '== 2.06';\n" > /app/cpanfile \
|
||||
&& cd /app \
|
||||
&& PERL5LIB=/tools/lib/perl5 PATH=/tools/bin:$PATH carton install
|
||||
|
||||
# cpm, into an arbitrary prefix. cpm serializes install.json differently from cpanm (canonical JSON,
|
||||
# three-space indent, spaces around the colons, trailing newline) and that difference is under test.
|
||||
RUN PERL5LIB=/tools/lib/perl5 /tools/bin/cpm install -L /srv/api/local --no-test 'Capture::Tiny@0.48'
|
||||
|
||||
# a path a packlist claims and the filesystem lacks is what an overwritten or removed file looks
|
||||
# like, and the owned-file list is expected to report it anyway
|
||||
RUN rm /usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL/Utils.pm
|
||||
|
||||
# assemble only what the catalogers read: the site_perl tree, the core arch directory's
|
||||
# perllocal.pod and packlist, and the two local-lib trees. The rest of the core lib tree is tens of
|
||||
# megabytes of interpreter that carries no CPAN evidence.
|
||||
RUN mkdir -p /out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu \
|
||||
&& cp -a /usr/local/lib/perl5/site_perl /out/usr/local/lib/perl5/site_perl \
|
||||
&& cp -a /usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod \
|
||||
/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/.packlist \
|
||||
/out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/ \
|
||||
&& mkdir -p /out/app /out/srv/api \
|
||||
&& cp -a /app/cpanfile /app/cpanfile.snapshot /app/local /out/app/ \
|
||||
&& cp -a /srv/api/local /out/srv/api/local
|
||||
|
||||
FROM scratch
|
||||
|
||||
COPY --from=builder /out/ /
|
||||
69
syft/pkg/cataloger/perl/testdata/image-cpan-packlist-only/Dockerfile
vendored
Normal file
69
syft/pkg/cataloger/perl/testdata/image-cpan-packlist-only/Dockerfile
vendored
Normal file
@ -0,0 +1,69 @@
|
||||
# installs that leave a .packlist and no install.json, which is the majority of real Perl images and
|
||||
# where the interesting failures live: the distribution name has to come out of the auto/ path and
|
||||
# the version out of perllocal.pod or the installed .pm.
|
||||
#
|
||||
# Four real installers are exercised, none of which writes a .meta directory:
|
||||
#
|
||||
# - ExtUtils::MakeMaker run directly (perl Makefile.PL && make install), which is what distro
|
||||
# packagers and vendored builds do. It resolves no dependencies at all.
|
||||
# - cpanm against a local tarball. cpanm's save_meta returns early unless the distribution was
|
||||
# resolved from a mirror, so a local source writes no install.json.
|
||||
# - CPAN.pm, which never writes a .meta directory in any circumstance.
|
||||
# - an EUMM install with NO_PERLLOCAL, which is the case that forces the version to be scraped out
|
||||
# of the installed .pm.
|
||||
#
|
||||
# Old versions are pinned and pinned versions come from backpan: the mirrors drop them and answer
|
||||
# with an HTML 404 body, which only fails later as "gzip: stdin: not in gzip format".
|
||||
#
|
||||
# The digest is perl:5.40-slim's linux/amd64 manifest; see image-cpan-mirror-installs/Dockerfile.
|
||||
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
|
||||
|
||||
# Encode is XS and the slim image ships no compiler
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends gcc libc6-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /tarballs
|
||||
RUN curl -sSfLO https://backpan.perl.org/authors/id/G/GA/GAAS/libwww-perl-5.836.tar.gz \
|
||||
&& curl -sSfLO https://backpan.perl.org/authors/id/I/IS/ISHIGAKI/Text-CSV-2.02.tar.gz \
|
||||
&& curl -sSfLO https://backpan.perl.org/authors/id/I/IS/ISHIGAKI/Text-CSV-2.06.tar.gz \
|
||||
&& curl -sSfLO https://backpan.perl.org/authors/id/I/IS/ISHIGAKI/JSON-PP-4.16.tar.gz
|
||||
|
||||
# ExtUtils::MakeMaker, run by hand. libwww-perl's EUMM NAME is LWP, so the packlist lands at
|
||||
# auto/LWP/.packlist and the distribution is only ever recoverable as "LWP" from an installed tree.
|
||||
# EUMM installs no prerequisites and only warns about the missing ones, which is what keeps this
|
||||
# image free of the mirror-resolved records those would have written.
|
||||
RUN tar xzf libwww-perl-5.836.tar.gz \
|
||||
&& cd libwww-perl-5.836 \
|
||||
&& perl Makefile.PL \
|
||||
&& make \
|
||||
&& make install
|
||||
|
||||
# cpanm from a local tarball, twice, upgrading in place. perllocal.pod is append-only, so 2.02's
|
||||
# stanza stays beside 2.06's forever and the later one has to win.
|
||||
RUN cpanm --notest ./Text-CSV-2.02.tar.gz \
|
||||
&& cpanm --notest ./Text-CSV-2.06.tar.gz
|
||||
|
||||
# a packlist with no perllocal.pod stanza at all, which is what NO_PERLLOCAL leaves and what
|
||||
# Module::Build::Tiny produces by default. The version can then only come from the installed .pm.
|
||||
RUN PERL_MM_OPT="NO_PERLLOCAL=1" cpanm --notest ./JSON-PP-4.16.tar.gz
|
||||
|
||||
# CPAN.pm. Encode's $VERSION is declared bare and computed with sprintf from an RCS Revision keyword
|
||||
# inside a BEGIN block, so no static read of Encode.pm can recover it; only perllocal.pod, which
|
||||
# records what EUMM evaluated at build time, has the value.
|
||||
RUN PERL_MM_USE_DEFAULT=1 cpan -T -i D/DA/DANKOGAI/Encode-3.24.tar.gz
|
||||
|
||||
# guard the premise of the whole image rather than trusting it
|
||||
RUN if find /usr/local/lib/perl5 -name install.json | grep -q .; then \
|
||||
echo "an install.json was written; this image is supposed to have none"; exit 1; \
|
||||
fi
|
||||
|
||||
RUN mkdir -p /out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu \
|
||||
&& cp -a /usr/local/lib/perl5/site_perl /out/usr/local/lib/perl5/site_perl \
|
||||
&& cp -a /usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod \
|
||||
/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/.packlist \
|
||||
/out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/
|
||||
|
||||
FROM scratch
|
||||
|
||||
COPY --from=builder /out/ /
|
||||
46
syft/pkg/cataloger/perl/testdata/image-cpan-two-install-trees/Dockerfile
vendored
Normal file
46
syft/pkg/cataloger/perl/testdata/image-cpan-two-install-trees/Dockerfile
vendored
Normal file
@ -0,0 +1,46 @@
|
||||
# two perl lib trees on one image, each with its own perllocal.pod naming the same module at a
|
||||
# different version. Picking the stanza from the wrong tree would be silent, and both installed .pm
|
||||
# files are the real ones, so scraping cannot cover for it: each tree would be reported at the
|
||||
# other's version.
|
||||
#
|
||||
# Both JSON-PP installs come from local tarballs, so neither writes an install.json and the version
|
||||
# can only come from perllocal.pod. A second full perl is not needed; a local-lib prefix reproduces
|
||||
# the same ambiguity, because cpanm -L writes a perllocal.pod into the prefix's own arch directory.
|
||||
#
|
||||
# The image also carries one distribution installed into both trees at the same version, from a
|
||||
# mirror, so that both trees hold an install.json agreeing on name and version. Those two records
|
||||
# describe two installed things and are only distinguishable by the tree they landed in.
|
||||
#
|
||||
# The digest is perl:5.40-slim's linux/amd64 manifest; see image-cpan-mirror-installs/Dockerfile.
|
||||
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
|
||||
|
||||
WORKDIR /tarballs
|
||||
RUN curl -sSfLO https://backpan.perl.org/authors/id/M/MA/MAKAMAKA/JSON-PP-2.27300.tar.gz \
|
||||
&& curl -sSfLO https://backpan.perl.org/authors/id/I/IS/ISHIGAKI/JSON-PP-4.16.tar.gz
|
||||
|
||||
# the site install. 4.16 declares `our $VERSION = '4.16';`, which is scrapable, so this tree is the
|
||||
# one where perllocal.pod and the .pm agree.
|
||||
RUN cpanm --notest ./JSON-PP-4.16.tar.gz
|
||||
|
||||
# the second tree. 2.27300 declares the fully qualified `$JSON::PP::VERSION = '2.27300';` form that
|
||||
# older Dist::Zilla wrote.
|
||||
RUN cpanm --notest -L /opt/app ./JSON-PP-2.27300.tar.gz
|
||||
|
||||
# the same distribution and version into both trees. Resolving from a mirror is what makes cpanm
|
||||
# write an install.json, and the PAUSE path pins the version the way a bare module name would not.
|
||||
# Text-CSV is pure perl and pulls in nothing that is not core, so this needs no compiler and adds no
|
||||
# unpinned fetch.
|
||||
RUN cpanm --notest I/IS/ISHIGAKI/Text-CSV-2.06.tar.gz \
|
||||
&& cpanm --notest -L /opt/app I/IS/ISHIGAKI/Text-CSV-2.06.tar.gz
|
||||
|
||||
RUN mkdir -p /out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu \
|
||||
&& cp -a /usr/local/lib/perl5/site_perl /out/usr/local/lib/perl5/site_perl \
|
||||
&& cp -a /usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod \
|
||||
/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/.packlist \
|
||||
/out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/ \
|
||||
&& mkdir -p /out/opt \
|
||||
&& cp -a /opt/app /out/opt/app
|
||||
|
||||
FROM scratch
|
||||
|
||||
COPY --from=builder /out/ /
|
||||
43
syft/pkg/cataloger/perl/testdata/image-cpan-unpacked-releases/Dockerfile
vendored
Normal file
43
syft/pkg/cataloger/perl/testdata/image-cpan-unpacked-releases/Dockerfile
vendored
Normal file
@ -0,0 +1,43 @@
|
||||
# release tarballs unpacked but never installed, plus a source repository working tree. Nothing here
|
||||
# has a packlist or an install.json, so the whole image is meta-file evidence.
|
||||
#
|
||||
# Versions are pinned, and pinned versions have to come from backpan: the mirrors drop old releases
|
||||
# and answer with an HTML 404 that only fails later, at the point of untarring.
|
||||
#
|
||||
# The digest is perl:5.40-slim's linux/amd64 manifest rather than its index, so every image in this
|
||||
# directory is built from the same architecture; see image-cpan-mirror-installs/Dockerfile.
|
||||
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
|
||||
|
||||
WORKDIR /opt/releases
|
||||
|
||||
# a Dist::Zilla release: it ships dist.ini inside the tarball, and MANIFEST lists it, so a dist.ini
|
||||
# beside a META.json says nothing about whether this is a release or a checkout. It also ships both
|
||||
# META.json and META.yml, which is the ordinary case for dzil and where META.json has to win.
|
||||
RUN curl -sSfL https://backpan.perl.org/authors/id/E/ET/ETHER/Try-Tiny-0.32.tar.gz | tar xz
|
||||
|
||||
# CPAN Meta Spec 1.4, which is all that roughly 43% of current releases ship. Genuine
|
||||
# ExtUtils::MakeMaker 6.56 output: license is a bare string rather than an array, version is an
|
||||
# unquoted number, and there is no provides block because EUMM does not generate one.
|
||||
RUN curl -sSfL https://backpan.perl.org/authors/id/M/MA/MAKAMAKA/Text-CSV-1.21.tar.gz | tar xz
|
||||
|
||||
# a META.yml no strict YAML parser accepts: EUMM wrote this distribution's multi-line ABSTRACT
|
||||
# straight into `abstract:` with no quoting and no block scalar, so the continuation lines land at
|
||||
# column zero. This has to skip the directory rather than fail the scan.
|
||||
RUN curl -sSfL https://backpan.perl.org/authors/id/L/LE/LEMBARK/Parallel-Depend-4.10.tar.gz | tar xz
|
||||
|
||||
# a source repository working tree, pinned to one commit. Its committed META.json records the last
|
||||
# release at 6.83 while lib/LWP.pm on the same commit says 6.84, which is why a committed META.json
|
||||
# cannot be trusted for version. There is no MANIFEST, because Dist::Zilla generates it at build
|
||||
# time, and that absence is the only thing separating this from a release.
|
||||
#
|
||||
# This is GitHub's tarball of the commit rather than a clone: it is the same working tree, it needs
|
||||
# no git in the image, and it leaves no .git directory to prune.
|
||||
RUN mkdir -p /src \
|
||||
&& curl -sSfL https://codeload.github.com/libwww-perl/libwww-perl/tar.gz/65c6384c3d9e8a4f88a7fbe84307895dc78ccceb \
|
||||
| tar xz -C /src \
|
||||
&& mv /src/libwww-perl-* /src/libwww-perl
|
||||
|
||||
FROM scratch
|
||||
|
||||
COPY --from=builder /opt/releases /opt/releases
|
||||
COPY --from=builder /src/libwww-perl /src/libwww-perl
|
||||
24
syft/pkg/cataloger/perl/testdata/image-perl-core-only/Dockerfile
vendored
Normal file
24
syft/pkg/cataloger/perl/testdata/image-perl-core-only/Dockerfile
vendored
Normal file
@ -0,0 +1,24 @@
|
||||
# a perl with no CPAN installs at all, which pins the zero coverage of distributions bundled inside
|
||||
# the interpreter. site_perl is dropped entirely, so what is left is the core lib tree: Encode and
|
||||
# Storable sit in it, the only packlist is perl's own and has no auto/ segment, and there is no
|
||||
# .meta and no distro package. Nothing here carries a core distribution's version.
|
||||
#
|
||||
# perllocal.pod comes along even though it names the distributions the base image preinstalled,
|
||||
# because a perllocal.pod stanza on its own must not invent a package. Storable.pm does declare a
|
||||
# readable $VERSION, so the interesting half is that no package is made out of it either.
|
||||
#
|
||||
# The digest is perl:5.40-slim's linux/amd64 manifest; see image-cpan-mirror-installs/Dockerfile.
|
||||
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
|
||||
|
||||
# the whole core lib tree is tens of megabytes of interpreter, so this takes the two dual-life
|
||||
# distributions under discussion, their shared libraries, and the two files the catalogers look for
|
||||
RUN O=/out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu \
|
||||
&& mkdir -p $O/auto/Encode $O/auto/Storable \
|
||||
&& cd /usr/local/lib/perl5/5.40.4/x86_64-linux-gnu \
|
||||
&& cp -a Encode.pm Storable.pm perllocal.pod .packlist $O/ \
|
||||
&& cp -a auto/Encode/Encode.so $O/auto/Encode/ \
|
||||
&& cp -a auto/Storable/Storable.so $O/auto/Storable/
|
||||
|
||||
FROM scratch
|
||||
|
||||
COPY --from=builder /out/ /
|
||||
@ -23,6 +23,7 @@ const (
|
||||
JavaScript Language = "javascript"
|
||||
Lua Language = "lua"
|
||||
OCaml Language = "ocaml"
|
||||
Perl Language = "perl"
|
||||
PHP Language = "php"
|
||||
Python Language = "python"
|
||||
R Language = "R"
|
||||
@ -45,6 +46,7 @@ var AllLanguages = []Language{
|
||||
JavaScript,
|
||||
Lua,
|
||||
OCaml,
|
||||
Perl,
|
||||
PHP,
|
||||
Python,
|
||||
R,
|
||||
@ -80,6 +82,9 @@ func LanguageByName(name string) Language {
|
||||
return JavaScript
|
||||
case packageurl.TypeLuaRocks, string(Lua):
|
||||
return Lua
|
||||
// note: packageurl.TypeCpan and CpanPkg are the same string, so only one can be listed here
|
||||
case packageurl.TypeCpan, string(Perl):
|
||||
return Perl
|
||||
case packageurl.TypePyPi, string(Python):
|
||||
return Python
|
||||
case packageurl.TypeGem, string(Ruby):
|
||||
|
||||
@ -90,6 +90,10 @@ func TestLanguageFromPURL(t *testing.T) {
|
||||
purl: "pkg:opam/ocaml-base-compiler@5.2.0",
|
||||
want: OCaml,
|
||||
},
|
||||
{
|
||||
purl: "pkg:cpan/URI@5.35?author=OALDERS",
|
||||
want: Perl,
|
||||
},
|
||||
}
|
||||
|
||||
var languages = strset.New()
|
||||
|
||||
61
syft/pkg/perl.go
Normal file
61
syft/pkg/perl.go
Normal file
@ -0,0 +1,61 @@
|
||||
package pkg
|
||||
|
||||
import (
|
||||
"sort"
|
||||
|
||||
"github.com/scylladb/go-set/strset"
|
||||
)
|
||||
|
||||
// CpanDistribution describes a CPAN distribution installed on disk, as recorded by a CPAN client's
|
||||
// install.json and by the .packlist an installer wrote. There is deliberately no name or version
|
||||
// field: both would be verbatim copies of the package name and version. Licenses live on the package
|
||||
// and dependencies live in relationships.
|
||||
type CpanDistribution struct {
|
||||
// Dist is the distvname install.json recorded (e.g. URI-5.35), the raw value the package name and
|
||||
// version were recovered from
|
||||
Dist string `json:"dist,omitempty"`
|
||||
|
||||
// MainModule is the module name recovered from the .packlist path (e.g. LWP for libwww-perl).
|
||||
// Empty when the name came from install.json, which is authoritative. Set, it means the name is the
|
||||
// installer's NAME and may not be the distribution name.
|
||||
MainModule string `json:"mainModule,omitempty"`
|
||||
|
||||
// Author is the PAUSE ID of the distribution author (e.g. OALDERS), parsed out of Path
|
||||
Author string `json:"author,omitempty"`
|
||||
|
||||
// Path is the raw PAUSE path of the release archive (e.g. O/OA/OALDERS/URI-5.35.tar.gz)
|
||||
Path string `json:"path,omitempty"`
|
||||
|
||||
// Modules are the modules this distribution provides, sorted by name
|
||||
Modules []CpanModule `json:"modules,omitempty"`
|
||||
|
||||
// Files are the paths the .packlist recorded as installed by this distribution, kept in the
|
||||
// packlist's own order and unfiltered: a path the packlist claims and the filesystem lacks means the
|
||||
// file was removed or overwritten out from under the installer, which is signal rather than noise
|
||||
Files []string `json:"files,omitempty"`
|
||||
}
|
||||
|
||||
// OwnedFiles satisfies pkg.FileOwner, so a distribution that installed files a distro package also
|
||||
// owns is related to it by file ownership rather than reported as an unrelated duplicate.
|
||||
func (m CpanDistribution) OwnedFiles() (result []string) {
|
||||
result = strset.New(m.Files...).List()
|
||||
sort.Strings(result)
|
||||
return
|
||||
}
|
||||
|
||||
// CpanUnpackedRelease describes an unpacked CPAN release found on disk, read from the release's own
|
||||
// META.json or META.yml. Weaker evidence than an installed distribution: the code is present but
|
||||
// nothing says the interpreter can load it. It has no PAUSE path, so it has no author.
|
||||
type CpanUnpackedRelease struct {
|
||||
// Modules are the modules the release declares it provides, sorted by name
|
||||
Modules []CpanModule `json:"modules,omitempty"`
|
||||
}
|
||||
|
||||
// CpanModule is a single perl module provided by a CPAN distribution. Shared by both types above.
|
||||
type CpanModule struct {
|
||||
// Name is the module name as it would be used in a perl `use` statement (e.g. URI::Escape)
|
||||
Name string `json:"name"`
|
||||
|
||||
// Version is the module version, which may differ from the distribution version
|
||||
Version string `json:"version,omitempty"`
|
||||
}
|
||||
48
syft/pkg/perl_test.go
Normal file
48
syft/pkg/perl_test.go
Normal file
@ -0,0 +1,48 @@
|
||||
package pkg
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestCpanDistribution_OwnedFiles(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
metadata CpanDistribution
|
||||
expected []string
|
||||
}{
|
||||
{
|
||||
name: "sorted and deduplicated",
|
||||
metadata: CpanDistribution{
|
||||
// a distribution that merged more than one packlist can claim the same path twice
|
||||
Files: []string{"/lib/URI.pm", "/lib/URI/Escape.pm", "/lib/URI.pm"},
|
||||
},
|
||||
expected: []string{"/lib/URI.pm", "/lib/URI/Escape.pm"},
|
||||
},
|
||||
{
|
||||
// this is what a NO_PACKLIST install leaves behind: an install.json and nothing else
|
||||
name: "install.json only owns nothing",
|
||||
metadata: CpanDistribution{Dist: "URI-5.35", Modules: []CpanModule{{Name: "URI"}}},
|
||||
expected: []string{},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
assert.Equal(t, tt.expected, tt.metadata.OwnedFiles())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCpanDistribution_isFileOwner(t *testing.T) {
|
||||
var owner any = CpanDistribution{}
|
||||
_, ok := owner.(FileOwner)
|
||||
require.True(t, ok, "the file-ownership relationship workers only see metadata that satisfies FileOwner")
|
||||
|
||||
// the unpacked-release tier has no packlist, so it deliberately does not
|
||||
var release any = CpanUnpackedRelease{}
|
||||
_, ok = release.(FileOwner)
|
||||
assert.False(t, ok)
|
||||
}
|
||||
@ -21,6 +21,7 @@ const (
|
||||
CocoapodsPkg Type = "pod"
|
||||
ConanPkg Type = "conan"
|
||||
CondaPkg Type = "conda"
|
||||
CpanPkg Type = "cpan"
|
||||
DartPubPkg Type = "dart-pub"
|
||||
DebPkg Type = "deb"
|
||||
DotnetPkg Type = "dotnet"
|
||||
@ -68,6 +69,7 @@ var AllPkgs = []Type{
|
||||
CocoapodsPkg,
|
||||
ConanPkg,
|
||||
CondaPkg,
|
||||
CpanPkg,
|
||||
DartPubPkg,
|
||||
DebPkg,
|
||||
DotnetPkg,
|
||||
@ -123,6 +125,8 @@ func (t Type) PackageURLType() string {
|
||||
return packageurl.TypeConan
|
||||
case CondaPkg:
|
||||
return packageurl.TypeGeneric
|
||||
case CpanPkg:
|
||||
return packageurl.TypeCpan
|
||||
case DartPubPkg:
|
||||
return packageurl.TypePub
|
||||
case DebPkg:
|
||||
@ -224,6 +228,8 @@ func TypeByName(name string) Type {
|
||||
return RustPkg
|
||||
case "conda":
|
||||
return CondaPkg
|
||||
case packageurl.TypeCpan:
|
||||
return CpanPkg
|
||||
case packageurl.TypePub:
|
||||
return DartPubPkg
|
||||
case "dotnet", packageurl.TypeNuget: // "dotnet" is here to support legacy use cases; "nuget" is the canonical purl type
|
||||
|
||||
@ -125,6 +125,10 @@ func TestTypeFromPURL(t *testing.T) {
|
||||
purl: "pkg:luarocks/kong@3.7.0",
|
||||
expected: LuaRocksPkg,
|
||||
},
|
||||
{
|
||||
purl: "pkg:cpan/URI@5.35?author=OALDERS",
|
||||
expected: CpanPkg,
|
||||
},
|
||||
{
|
||||
purl: "pkg:swift/github.com/apple/swift-numerics/swift-numerics@1.0.2",
|
||||
expected: SwiftPkg,
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user