feat: catalog CPAN distributions installed by perl clients

Adds `perl` as a language and `cpan` as a package type, with two catalogers behind them.

- `perl-cpan-installed-cataloger` reads `.meta/*/install.json` (cpanm, cpm, carton) and `auto/**/.packlist` (anything installed through `ExtUtils::MakeMaker` or `Module::Build`, including CPAN.pm). Both globs are unanchored, so a local-lib or carton application tree is found the same as a system install.
- `perl-cpan-meta-cataloger` reads an unpacked release's own `META.json` or `META.yml`, gated on a sibling `MANIFEST` so source checkouts are ignored.

Packages are keyed on the **distribution**, not the module, because that is what CPAN, MetaCPAN and the advisory data all use. `LWP.pm` belongs to `libwww-perl` and reporting it as `LWP` would make every advisory for it unreachable. The distribution name comes from `install.json`'s `dist` field, and from the `auto/` path for packlists.

purls are `pkg:cpan/<distribution>@<version>`, with the PAUSE author added as an `author` qualifier when the evidence carries it. Metadata comes in two types: `cpan-distribution` for installed evidence and `cpan-unpacked-release` for a release sitting on disk. The tiers differ in more than a name, so a consumer should not have to string-match a cataloger name to tell them apart: an unpacked release has no PAUSE path and therefore no author and no file list, and "installed and loadable by the interpreter" is a materially stronger claim than "a source tree exists here".

A packlist's version comes from `perllocal.pod`. EUMM writes it and the packlist from the same variables in the same install target, so the `auto/` path segments and the perllocal `Module` name are the same key, and the recorded `VERSION` is what was evaluated at build time rather than what can be read back statically. Scraping `$VERSION` out of the main `.pm` is the fallback, since `NO_PERLLOCAL` suppresses the file and Module::Build never writes one. Three rules pick the stanza: dashed module name, longest `installed into` libdir that prefixes the packlist path, and last match wins because the file is append-only. Without the libdir rule a second perl on the image silently supplies the version.

Where scraping is the fallback, it reads more than a plain `our $VERSION = '...'`. A version declared in the package statement (`package Foo::Bar v1.0.0;`) counts, which matters because a distribution can declare it that way and carry no `$VERSION` at all: `CPAN::02Packages::Search` is one, and without this it reports no version and matches nothing. Fully qualified `$Foo::Bar::VERSION` counts too, which is what older Dist::Zilla emitted and what real `JSON::PP` 2.27300 still carries. `qv('1.2.3')` is handled. A version computed at runtime is not, and those are reported without one rather than guessed at.

Packlist entries are parsed the way `ExtUtils::Packlist` writes them: a line can carry space-separated metadata after the path (`/path/to/File.pm type=file`), which is what `installperl` produces, so the suffix is stripped rather than the line being cut at its first space.

`META.yml` is read alongside `META.json`, because about 43% of current CPAN releases ship no `META.json` and they skew old, which is where the advisories are. Where both sit in one directory the `META.json` wins. A `META.yml` a strict parser rejects skips that directory rather than failing the scan, which is routine rather than defensive for pre-spec releases.

A packlist is literally a file list, so its paths are surfaced through `pkg.FileOwner`. They are reported as recorded, unfiltered: a path the packlist claims and the filesystem lacks means the file was removed or overwritten out from under the installer, which is worth seeing rather than hiding.

Build leftovers under `~/.cpanm/work` and `~/.cpan/build` are skipped. They genuinely are unpacked release tarballs, `MANIFEST` included, so the `MANIFEST` gate admits them and a path exclusion is the only signal available. Without it every distribution on an image that did not clean up is reported twice. The cost is that a tarball deliberately kept under `~/.cpan/build` stops being reported.

Two behaviors that look wrong but are not:

- a distribution can be reported twice at different versions. `libwww-perl` 5.836 bundles `HTTP-Date`, `HTTP-Message` and `LWP-MediaTypes`, and installing it over the modern standalone releases overwrites their `.pm` files. Both versions are genuinely present, so the merge refuses to pair disagreeing versions rather than hiding one.
- a distribution with no readable version is reported without one rather than dropped, so it stays visible.

The coverage boundary is stated in full in the `package perl` doc comment. In short: modules installed from distro packages are out of scope, since packagers strip CPAN metadata with `NO_PACKLIST` and deb, rpm and apk already report them. Core and dual-life distributions bundled with the interpreter have no coverage, because nothing on disk carries their versions; the interpreter itself is reported separately. Vendored trees, `App::FatPacker` output and PAR archives are invisible, because what they carry is module identity with no offline map to a distribution. A packlist-derived name is the installer's `NAME` and may not be the distribution name, which is left to the vulnerability data to resolve.

One correction worth calling out, since it is easy to arrive at twice: a sibling `MANIFEST` is the only thing separating an unpacked release from a source checkout. An earlier version of the guard also rejected any tree containing a `dist.ini`, on the theory that it marked a Dist::Zilla source tree. dzil ships `dist.ini` *inside* the tarballs it builds and lists it in the generated `MANIFEST`, so that exclusion was silently skipping real releases, `URI` among them.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
This commit is contained in:
Alex Goodman 2026-08-04 11:30:30 -04:00
parent 08e913a2ce
commit 1827ce2f4f
No known key found for this signature in database
44 changed files with 7846 additions and 3 deletions

View File

@ -524,4 +524,12 @@ var commonTestCases = []testCase{
"kong": "3.7.0-0",
},
},
{
name: "find cpan distribution",
pkgType: pkg.CpanPkg,
pkgLanguage: pkg.Perl,
pkgInfo: map[string]string{
"URI": "5.35",
},
},
}

View File

@ -0,0 +1,9 @@
{
"abstract": "Uniform Resource Identifiers (absolute and relative)",
"license": ["perl_5"],
"meta-spec": {"version": 2},
"name": "URI",
"prereqs": {"runtime": {"requires": {"Carp": "0", "perl": "5.008001"}}},
"release_status": "stable",
"version": "5.35"
}

View File

@ -0,0 +1,11 @@
{
"dist": "URI-5.35",
"name": "URI",
"pathname": "O/OA/OALDERS/URI-5.35.tar.gz",
"provides": {
"URI": {"file": "lib/URI.pm", "version": "5.35"},
"URI::Escape": {"file": "lib/URI/Escape.pm", "version": "5.35"}
},
"target": "URI",
"version": "5.35"
}

View File

@ -3,7 +3,7 @@ package internal
const (
// JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "16.1.10"
JSONSchemaVersion = "16.1.11"
// Changelog
// 16.1.0 - reformulated the python pdm fields (added "URL" and removed the unused "path" field).
@ -17,4 +17,5 @@ const (
// 16.1.8 - add VcpkgManifest metadata type for vcpkg manifest support
// 16.1.9 - add Symbols (grouped by owning package import path) to GolangBinaryBuildinfoEntry metadata
// 16.1.10 - add SafeTensorsModelInfo metadata type for the safetensors AI model cataloger
// 16.1.11 - add CpanDistribution and CpanUnpackedRelease metadata types for the perl catalogers
)

View File

@ -19,6 +19,8 @@ func AllTypes() []any {
pkg.ConanfileEntry{},
pkg.ConaninfoEntry{},
pkg.CondaMetaPackage{},
pkg.CpanDistribution{},
pkg.CpanUnpackedRelease{},
pkg.DartPubspec{},
pkg.DartPubspecLockEntry{},
pkg.DenoLockEntry{},

View File

@ -71,6 +71,8 @@ var jsonTypes = makeJSONTypes(
jsonNames(pkg.ConanV2LockEntry{}, "c-conan-lock-v2-entry"),
jsonNames(pkg.ConanfileEntry{}, "c-conan-file-entry", "ConanMetadataType"),
jsonNames(pkg.ConaninfoEntry{}, "c-conan-info-entry"),
jsonNames(pkg.CpanDistribution{}, "cpan-distribution"),
jsonNames(pkg.CpanUnpackedRelease{}, "cpan-unpacked-release"),
jsonNames(pkg.DartPubspecLockEntry{}, "dart-pubspec-lock-entry", "DartPubMetadata"),
jsonNames(pkg.DartPubspec{}, "dart-pubspec"),
jsonNames(pkg.DenoLockEntry{}, "deno-lock-entry"),

View File

@ -279,6 +279,46 @@ func TestOwnershipByFilesRelationship(t *testing.T) {
return []pkg.Package{parent, child}, nil
},
},
{
// a CPAN distribution owns whatever its .packlist recorded, which is how a distribution that
// installed a binary or a shared object is related to the package cataloged from it
name: "cpan-distribution-owns-packlist-paths",
setup: func(t testing.TB) ([]pkg.Package, []artifact.Relationship) {
parent := pkg.Package{
Locations: file.NewLocationSet(
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/Net/SSLeay/.packlist"),
),
Type: pkg.CpanPkg,
Metadata: pkg.CpanDistribution{
MainModule: "Net::SSLeay",
Files: []string{
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/Net/SSLeay.pm",
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/Net/SSLeay/SSLeay.so",
},
},
}
parent.SetID()
child := pkg.Package{
Locations: file.NewLocationSet(
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/Net/SSLeay/SSLeay.so"),
),
Type: pkg.BinaryPkg,
}
child.SetID()
return []pkg.Package{parent, child}, []artifact.Relationship{
{
From: parent,
To: child,
Type: artifact.OwnershipByFileOverlapRelationship,
Data: ownershipByFilesMetadata{
Files: []string{"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/Net/SSLeay/SSLeay.so"},
},
},
}
},
},
}
for _, test := range tests {

View File

@ -27,6 +27,7 @@ import (
"github.com/anchore/syft/syft/pkg/cataloger/lua"
"github.com/anchore/syft/syft/pkg/cataloger/nix"
"github.com/anchore/syft/syft/pkg/cataloger/ocaml"
"github.com/anchore/syft/syft/pkg/cataloger/perl"
"github.com/anchore/syft/syft/pkg/cataloger/php"
"github.com/anchore/syft/syft/pkg/cataloger/python"
"github.com/anchore/syft/syft/pkg/cataloger/r"
@ -132,6 +133,10 @@ func DefaultPackageTaskFactories() Factories {
newSimplePackageTaskFactory(swift.NewSwiftPackageManagerCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "swift", "spm"),
newSimplePackageTaskFactory(swipl.NewSwiplPackCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "swipl", "pack"),
newSimplePackageTaskFactory(ocaml.NewOpamPackageManagerCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "ocaml", "opam"),
// note: no image tag -- an unpacked release's own META.json describes a distribution present on
// disk but not installed into the interpreter's @INC, and declared information is not cataloged
// inside container images
newSimplePackageTaskFactory(perl.NewCpanMetaCataloger, pkgcataloging.DeclaredTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "perl", "cpan"),
// language-specific package for both image and directory scans (but not necessarily declared) ////////////////////////////////////////
newPackageTaskFactory(
@ -162,6 +167,7 @@ func DefaultPackageTaskFactories() Factories {
pkgcataloging.DirectoryTag, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.LanguageTag, "nix",
),
newSimplePackageTaskFactory(lua.NewPackageCataloger, pkgcataloging.DirectoryTag, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.LanguageTag, "lua"),
newSimplePackageTaskFactory(perl.NewCpanInstalledCataloger, pkgcataloging.InstalledTag, pkgcataloging.ImageTag, pkgcataloging.DirectoryTag, pkgcataloging.LanguageTag, "perl", "cpan"),
// other package catalogers ///////////////////////////////////////////////////////////////////////////
newPackageTaskFactory(

File diff suppressed because it is too large Load Diff

View File

@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "anchore.io/schema/syft/json/16.1.10/document",
"$id": "anchore.io/schema/syft/json/16.1.11/document",
"$ref": "#/$defs/Document",
"$defs": {
"AlpmDbEntry": {
@ -729,6 +729,72 @@
],
"description": "Coordinates contains the minimal information needed to describe how to find a file within any possible source object (e.g."
},
"CpanDistribution": {
"properties": {
"dist": {
"type": "string",
"description": "Dist is the distvname install.json recorded (e.g. URI-5.35), the raw value the package name and\nversion were recovered from"
},
"mainModule": {
"type": "string",
"description": "MainModule is the module name recovered from the .packlist path (e.g. LWP for libwww-perl).\nEmpty when the name came from install.json, which is authoritative. Set, it means the name is the\ninstaller's NAME and may not be the distribution name."
},
"author": {
"type": "string",
"description": "Author is the PAUSE ID of the distribution author (e.g. OALDERS), parsed out of Path"
},
"path": {
"type": "string",
"description": "Path is the raw PAUSE path of the release archive (e.g. O/OA/OALDERS/URI-5.35.tar.gz)"
},
"modules": {
"items": {
"$ref": "#/$defs/CpanModule"
},
"type": "array",
"description": "Modules are the modules this distribution provides, sorted by name"
},
"files": {
"items": {
"type": "string"
},
"type": "array",
"description": "Files are the paths the .packlist recorded as installed by this distribution, kept in the\npacklist's own order and unfiltered: a path the packlist claims and the filesystem lacks means the\nfile was removed or overwritten out from under the installer, which is signal rather than noise"
}
},
"type": "object",
"description": "CpanDistribution describes a CPAN distribution installed on disk, as recorded by a CPAN client's install.json and by the .packlist an installer wrote."
},
"CpanModule": {
"properties": {
"name": {
"type": "string",
"description": "Name is the module name as it would be used in a perl `use` statement (e.g. URI::Escape)"
},
"version": {
"type": "string",
"description": "Version is the module version, which may differ from the distribution version"
}
},
"type": "object",
"required": [
"name"
],
"description": "CpanModule is a single perl module provided by a CPAN distribution."
},
"CpanUnpackedRelease": {
"properties": {
"modules": {
"items": {
"$ref": "#/$defs/CpanModule"
},
"type": "array",
"description": "Modules are the modules the release declares it provides, sorted by name"
}
},
"type": "object",
"description": "CpanUnpackedRelease describes an unpacked CPAN release found on disk, read from the release's own META.json or META.yml."
},
"DartPubspec": {
"properties": {
"homepage": {
@ -2772,6 +2838,12 @@
{
"$ref": "#/$defs/CondaMetadataEntry"
},
{
"$ref": "#/$defs/CpanDistribution"
},
{
"$ref": "#/$defs/CpanUnpackedRelease"
},
{
"$ref": "#/$defs/DartPubspec"
},

View File

@ -20,6 +20,8 @@ func Test_OriginatorSupplier(t *testing.T) {
pkg.ConanfileEntry{},
pkg.ConaninfoEntry{},
pkg.CondaMetaPackage{},
pkg.CpanDistribution{}, // the author is a PAUSE ID, not a person or organization name
pkg.CpanUnpackedRelease{},
pkg.DartPubspecLockEntry{},
pkg.DartPubspec{},
pkg.DotnetDepsEntry{},

View File

@ -66,6 +66,8 @@ func SourceInfo(p pkg.Package) string {
answer = "acquired package info from nix store path"
case pkg.Rpkg:
answer = "acquired package info from R-package DESCRIPTION file"
case pkg.CpanPkg:
answer = "acquired package info from CPAN distribution metadata"
case pkg.LuaRocksPkg:
answer = "acquired package info from Rockspec package file"
case pkg.SwiftPkg:

View File

@ -287,6 +287,14 @@ func Test_SourceInfo(t *testing.T) {
"acquired package info from R-package DESCRIPTION file",
},
},
{
input: pkg.Package{
Type: pkg.CpanPkg,
},
expected: []string{
"acquired package info from CPAN distribution metadata",
},
},
{
input: pkg.Package{
Type: pkg.LuaRocksPkg,

View File

@ -184,8 +184,13 @@ func FromPackageAttributes(p pkg.Package) []cpe.CPE {
}
func candidateTargetSw(p pkg.Package) []string {
if p.Type == pkg.WordpressPluginPkg {
switch {
case p.Type == pkg.WordpressPluginPkg:
return []string{"wordpress"}
case p.Language == pkg.Perl:
// NVD records CPAN distributions with perl in target_sw, e.g.
// cpe:2.3:a:mojolicious:mojolicious:*:*:*:*:*:perl:*:*
return []string{"perl"}
}
return []string{cpe.Any}
}

View File

@ -777,6 +777,25 @@ func TestGeneratePackageCPEs(t *testing.T) {
"cpe:2.3:a:wow_estore:wp_coder:2.5.1:*:*:*:*:wordpress:*:*",
},
},
{
// NVD records CPAN distributions with perl in target_sw
name: "cpan distribution",
p: pkg.Package{
Name: "Mojolicious",
Version: "9.10",
Type: pkg.CpanPkg,
Language: pkg.Perl,
Metadata: pkg.CpanDistribution{
Author: "SRI",
Path: "S/SR/SRI/Mojolicious-9.10.tar.gz",
},
},
expected: []string{
// note: the distribution name keeps its casing here, as it does for every other ecosystem.
// NVD spells the same CPE lowercase (cpe:2.3:a:mojolicious:mojolicious:*:*:*:*:*:perl:*:*)
"cpe:2.3:a:Mojolicious:Mojolicious:9.10:*:*:*:*:perl:*:*",
},
},
{
name: "dotnet deps.json",
p: pkg.Package{

View File

@ -0,0 +1,128 @@
# Cataloger capabilities. See ../README.md for documentation.
catalogers:
- ecosystem: perl # MANUAL
name: perl-cpan-meta-cataloger # AUTO-GENERATED
type: generic # AUTO-GENERATED
source: # AUTO-GENERATED
file: syft/pkg/cataloger/perl/cataloger.go
function: NewCpanMetaCataloger
selectors: # AUTO-GENERATED
- cpan
- declared
- directory
- language
- package
- perl
parsers: # AUTO-GENERATED structure
- function: parseReleaseMeta
detector: # AUTO-GENERATED
method: glob # AUTO-GENERATED
criteria: # AUTO-GENERATED
- '**/META.json'
- '**/META.yml'
metadata_types: # AUTO-GENERATED
- pkg.CpanUnpackedRelease
package_types: # AUTO-GENERATED
- cpan
purl_types: # AUTO-GENERATED
- cpan
json_schema_types: # AUTO-GENERATED
- CpanUnpackedRelease
capabilities: # MANUAL - preserved across regeneration
- name: license
default: true
comment: the x_spdx_expression or license field of an unpacked release's own META.json or META.yml
- name: dependency.depth
default: []
comment: release meta prereqs are not resolved; only installed evidence produces edges
- name: dependency.edges
default: ""
- name: dependency.kinds
default: []
- name: package_manager.files.listing
default: false
- name: package_manager.files.digests
default: false
- name: package_manager.package_integrity_hash
default: false
- ecosystem: perl # MANUAL
name: perl-cpan-installed-cataloger # AUTO-GENERATED
type: generic # AUTO-GENERATED
source: # AUTO-GENERATED
file: syft/pkg/cataloger/perl/cataloger.go
function: NewCpanInstalledCataloger
selectors: # AUTO-GENERATED
- cpan
- directory
- image
- installed
- language
- package
- perl
parsers: # AUTO-GENERATED structure
- function: parsePacklist
detector: # AUTO-GENERATED
method: glob # AUTO-GENERATED
criteria: # AUTO-GENERATED
- '**/auto/**/.packlist'
metadata_types: # AUTO-GENERATED
- pkg.CpanDistribution
package_types: # AUTO-GENERATED
- cpan
purl_types: # AUTO-GENERATED
- cpan
json_schema_types: # AUTO-GENERATED
- CpanDistribution
capabilities: # MANUAL - preserved across regeneration
- name: license
default: false
comment: a packlist carries no metadata beyond the file list
- name: dependency.depth
default: []
- name: dependency.edges
default: ""
- name: dependency.kinds
default: []
- name: package_manager.files.listing
default: true
comment: the packlist is a file list, and it is surfaced through OwnedFiles so ownership relationships are emitted
- name: package_manager.files.digests
default: false
- name: package_manager.package_integrity_hash
default: false
- function: parseInstallJSON
detector: # AUTO-GENERATED
method: glob # AUTO-GENERATED
criteria: # AUTO-GENERATED
- '**/.meta/*/install.json'
metadata_types: # AUTO-GENERATED
- pkg.CpanDistribution
package_types: # AUTO-GENERATED
- cpan
purl_types: # AUTO-GENERATED
- cpan
json_schema_types: # AUTO-GENERATED
- CpanDistribution
capabilities: # MANUAL - preserved across regeneration
- name: license
default: true
comment: the x_spdx_expression or license field of the MYMETA.json written beside install.json
- name: dependency.depth
default:
- direct
- indirect
comment: prereqs are resolved across every cataloged distribution, so the graph is the full installed closure
- name: dependency.edges
default: complete
- name: dependency.kinds
default:
- runtime
- build
comment: develop and test prereqs describe how the distribution was authored, not what a deployed copy needs
- name: package_manager.files.listing
default: false
- name: package_manager.files.digests
default: false
- name: package_manager.package_integrity_hash
default: false

View File

@ -0,0 +1,62 @@
// Package perl provides Cataloger implementations relating to CPAN distributions within the Perl
// language ecosystem.
//
// Only distribution-level identity is emitted. Three evidence sources are read: install.json under a
// .meta directory, .packlist paired with perllocal.pod, and an unpacked release's own META.json or
// META.yml alongside a MANIFEST.
//
// The coverage boundary that follows from that:
//
// - CPAN-client installs are cataloged wherever they live, including local-lib trees such as an
// application's local/lib/perl5. cpanm, cpm, and carton write install.json only when the
// distribution was resolved from a mirror; a local directory or tarball install (cpanm .,
// cpanm ./Foo-1.0.tar.gz) writes none, and those fall back to the packlist pass. CPAN.pm never
// writes install.json at all.
// - A packlist is written by the ExtUtils::MakeMaker and Module::Build install targets unless
// suppressed with NO_PACKLIST, which is what distro packagers set.
// - A packlist-derived name is the installer's NAME, which is usually but not always the
// distribution name: libwww-perl installs to auto/LWP/.packlist. Resolving those to
// distributions is left to the vulnerability data, which carries the mapping.
// - Perl modules installed from distro packages are not cataloged here. They carry no CPAN-native
// metadata, since packagers suppress it, and they are already reported by the deb, rpm, and apk
// catalogers.
// - Core and dual-life distributions bundled with the interpreter are not cataloged. No on-disk
// artifact carries their versions. The interpreter itself is reported separately.
// - Build leftovers under ~/.cpanm/work and ~/.cpan/build are skipped. They are unpacked release
// tarballs, MANIFEST included, but they describe a copy that is already installed elsewhere in
// the same tree.
// - Vendored .pm trees, App::FatPacker output, and PAR archives are invisible. What they carry is
// module identity, and there is no offline map from a module to the distribution that shipped
// it. Fatpacked output does embed the .pm sources verbatim, $VERSION lines included, so the
// missing fact is the distribution rather than the version.
// - A cpanfile or cpanfile.snapshot is not parsed yet. The snapshot is a resolved lockfile and
// the only evidence in a source checkout or CI workspace, where local/ does not exist, so it
// is deferred rather than out of scope.
package perl
import (
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
// NewCpanInstalledCataloger returns a cataloger for CPAN distributions installed on disk.
// Globs are deliberately unanchored so local-lib trees (carton, cpanm -L) are found wherever they live.
func NewCpanInstalledCataloger() pkg.Cataloger {
return generic.NewCataloger("perl-cpan-installed-cataloger").
WithParserByGlobs(parseInstallJSON, "**/.meta/*/install.json").
WithParserByGlobs(parsePacklist, "**/auto/**/.packlist").
// registration order is execution order, and resolvePacklistVersions has to precede
// mergeDistributions: merging pairs a packlist package to its install.json package only when the
// versions agree or the packlist version is absent, so filling versions in afterwards splits every
// distribution whose two evidence kinds disagree into two packages
WithResolvingProcessors(resolvePacklistVersions).
WithProcessors(mergeDistributions).
WithResolvingProcessors(resolveDependencies)
}
// NewCpanMetaCataloger returns a cataloger for unpacked CPAN releases: release tarballs and vendored
// distribution trees. CPAN client build leftovers are excluded, see isBuildLeftover.
func NewCpanMetaCataloger() pkg.Cataloger {
return generic.NewCataloger("perl-cpan-meta-cataloger").
WithParserByGlobs(parseReleaseMeta, "**/META.json", "**/META.yml")
}

View File

@ -0,0 +1,567 @@
package perl
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/stereoscope/pkg/imagetest"
"github.com/anchore/syft/internal/packagemetadata"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/syft/source/stereoscopesource"
)
// every fixture here is an image built by a Dockerfile under testdata/, one per evidence tier. The
// installs are performed by the real clients inside the image, so the on-disk layout, the file
// contents and the versions are whatever the toolchain actually produced.
//
// an image is only how the fixture is delivered. perl-cpan-meta-cataloger is registered declared and
// directory, never image, so through the CLI it never runs against a container image at all; the tests
// below that drive it through an image resolver are parser tests, and the file tree they read is what
// they are actually asserting on.
// TestCpanInstalledCataloger_mirrorInstalls covers the only case that produces an install.json:
// cpanm, cpm and carton resolving distributions from a mirror. The image installs libwww-perl and its
// whole dependency closure with cpanm, one distribution with NO_PACKLIST, one with carton into an
// application's local/, and one with cpm into an unrelated prefix.
func TestCpanInstalledCataloger_mirrorInstalls(t *testing.T) {
pkgtest.NewCatalogTester().
WithImageResolver(t, "image-cpan-mirror-installs").
ExpectsPackageStrings([]string{
// preinstalled in the base image by its own EUMM installs: a packlist and a perllocal.pod
// stanza, no .meta, which is why these three have no author qualifier on their purl
"App-cpanminus @ 1.7049 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/App/cpanminus/.packlist)",
"IO-Socket-SSL @ 2.099 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/IO/Socket/SSL/.packlist)",
"Net-SSLeay @ 1.96 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Net/SSLeay/.packlist)",
// cpm, into a prefix that is not under /usr, which is the whole reason the globs are unanchored
"Capture-Tiny @ 0.48 (/srv/api/local/lib/perl5/x86_64-linux-gnu/.meta/Capture-Tiny-0.48/install.json)",
// carton, into local/lib/perl5 beside the cpanfile
"Text-CSV @ 2.06 (/app/local/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json)",
// installed with NO_PACKLIST and NO_PERLLOCAL, the way distro packagers do it, so
// install.json is the only evidence there is
"JSON-PP @ 4.16 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/JSON-PP-4.16/install.json)",
// libwww-perl is reported under its distribution name even though install.json's `name` field
// says LWP and its packlist is auto/LWP/.packlist. There is deliberately no LWP package here:
// reporting one is the bug this image exists to catch.
"libwww-perl @ 6.83 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/libwww-perl-6.83/install.json)",
// the rest of the closure cpanm pulled in for libwww-perl
"Clone @ 0.50 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Clone-0.50/install.json)",
"Encode-Locale @ 1.05 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Encode-Locale-1.05/install.json)",
"File-Listing @ 6.16 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/File-Listing-6.16/install.json)",
"HTML-Parser @ 3.85 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTML-Parser-3.85/install.json)",
"HTML-Tagset @ 3.24 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTML-Tagset-3.24/install.json)",
"HTTP-Cookies @ 6.12 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTTP-Cookies-6.12/install.json)",
"HTTP-Date @ 6.08 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTTP-Date-6.08/install.json)",
"HTTP-Message @ 7.04 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTTP-Message-7.04/install.json)",
"HTTP-Negotiate @ 6.01 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/HTTP-Negotiate-6.01/install.json)",
"IO-HTML @ 1.004 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/IO-HTML-1.004/install.json)",
"LWP-MediaTypes @ 6.04 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/LWP-MediaTypes-6.04/install.json)",
"MIME-Base32 @ 1.303 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/MIME-Base32-1.303/install.json)",
"Net-HTTP @ 6.24 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Net-HTTP-6.24/install.json)",
"TimeDate @ 2.35 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/TimeDate-2.35/install.json)",
"Try-Tiny @ 0.32 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Try-Tiny-0.32/install.json)",
"URI @ 5.35 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/URI-5.35/install.json)",
"WWW-RobotRules @ 6.03 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/WWW-RobotRules-6.03/install.json)",
// note: no package for the core perl packlist, which has no auto/ segment, and none for the
// distributions perllocal.pod names but that have no packlist of their own
}).
TestCataloger(t, NewCpanInstalledCataloger())
}
// TestCpanInstalledCataloger_prereqRelationships pins the dependency graph read out of the
// MYMETA.json files cpanm left beside each install.json. Prereqs name modules, so each one has to be
// resolved through the provides map of the cataloged set; a prereq that resolves to nothing (a core
// module such as Carp or strict, or a distribution that is simply not installed) is dropped.
//
// Packages are stringed without their locations here, because the full form makes each relationship
// unreadable.
func TestCpanInstalledCataloger_prereqRelationships(t *testing.T) {
pkgtest.NewCatalogTester().
WithImageResolver(t, "image-cpan-mirror-installs").
WithPackageStringer(func(p pkg.Package) string { return p.Name + " @ " + p.Version }).
ExpectsRelationshipStrings([]string{
"Clone @ 0.50 [dependency-of] HTTP-Message @ 7.04",
"Encode-Locale @ 1.05 [dependency-of] HTTP-Message @ 7.04",
"Encode-Locale @ 1.05 [dependency-of] libwww-perl @ 6.83",
"File-Listing @ 6.16 [dependency-of] libwww-perl @ 6.83",
"HTML-Parser @ 3.85 [dependency-of] libwww-perl @ 6.83",
"HTML-Tagset @ 3.24 [dependency-of] HTML-Parser @ 3.85",
"HTTP-Cookies @ 6.12 [dependency-of] libwww-perl @ 6.83",
"HTTP-Date @ 6.08 [dependency-of] File-Listing @ 6.16",
"HTTP-Date @ 6.08 [dependency-of] HTTP-Cookies @ 6.12",
"HTTP-Date @ 6.08 [dependency-of] HTTP-Message @ 7.04",
"HTTP-Date @ 6.08 [dependency-of] libwww-perl @ 6.83",
"HTTP-Message @ 7.04 [dependency-of] HTML-Parser @ 3.85",
"HTTP-Message @ 7.04 [dependency-of] HTTP-Cookies @ 6.12",
"HTTP-Message @ 7.04 [dependency-of] HTTP-Negotiate @ 6.01",
"HTTP-Message @ 7.04 [dependency-of] libwww-perl @ 6.83",
"HTTP-Negotiate @ 6.01 [dependency-of] libwww-perl @ 6.83",
"IO-HTML @ 1.004 [dependency-of] HTTP-Message @ 7.04",
"LWP-MediaTypes @ 6.04 [dependency-of] HTTP-Message @ 7.04",
"LWP-MediaTypes @ 6.04 [dependency-of] libwww-perl @ 6.83",
// URI 5.35 needs MIME::Base32 to build, which is why cpanm installed it at all
"MIME-Base32 @ 1.303 [dependency-of] URI @ 5.35",
"Net-HTTP @ 6.24 [dependency-of] libwww-perl @ 6.83",
// the module TimeDate provides is Date::Format, so this only resolves through the provides map
"TimeDate @ 2.35 [dependency-of] HTTP-Date @ 6.08",
"Try-Tiny @ 0.32 [dependency-of] libwww-perl @ 6.83",
"URI @ 5.35 [dependency-of] HTML-Parser @ 3.85",
"URI @ 5.35 [dependency-of] HTTP-Message @ 7.04",
"URI @ 5.35 [dependency-of] Net-HTTP @ 6.24",
"URI @ 5.35 [dependency-of] WWW-RobotRules @ 6.03",
"URI @ 5.35 [dependency-of] libwww-perl @ 6.83",
"WWW-RobotRules @ 6.03 [dependency-of] libwww-perl @ 6.83",
}).
TestCataloger(t, NewCpanInstalledCataloger())
}
func TestCpanInstalledCataloger_mirrorInstallEvidence(t *testing.T) {
pkgtest.NewCatalogTester().
WithImageResolver(t, "image-cpan-mirror-installs").
ExpectsAssertion(func(t *testing.T, pkgs []pkg.Package, _ []artifact.Relationship) {
byName := make(map[string]pkg.Package)
for _, p := range pkgs {
byName[p.Name] = p
}
uri := byName["URI"]
require.NotEmpty(t, uri.Name)
assert.Equal(t, "pkg:cpan/URI@5.35?author=OALDERS", uri.PURL)
assert.Equal(t, pkg.Perl, uri.Language)
assert.Equal(t, pkg.CpanPkg, uri.Type)
// licenses come from the MYMETA.json beside install.json, and are not duplicated into
// metadata. URI's own x_spdx_expression is what reaches the package, not the perl_5 token
// beside it: the expression is already SPDX, where perl_5 is a CPAN::Meta shortname that
// would have to be translated.
require.Len(t, uri.Licenses.ToSlice(), 1)
assert.Equal(t, "Artistic-1.0-Perl OR GPL-1.0-or-later", uri.Licenses.ToSlice()[0].Value)
// install.json, its sibling MYMETA.json, the packlist and the perllocal.pod that resolved
// the packlist's version are all evidence. The main .pm is not: perllocal.pod answered
// first, so it was never scraped.
assert.ElementsMatch(t, []string{
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/URI-5.35/install.json",
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/URI-5.35/MYMETA.json",
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/URI/.packlist",
"/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod",
}, realPaths(uri))
// a distribution with no x_spdx_expression falls back to the CPAN::Meta license token
require.Len(t, byName["Capture-Tiny"].Licenses.ToSlice(), 1)
assert.Equal(t, "apache_2_0", byName["Capture-Tiny"].Licenses.ToSlice()[0].Value)
// a packlist-derived package carries no author, so the purl gets no qualifier
assert.Equal(t, "pkg:cpan/IO-Socket-SSL@2.099", byName["IO-Socket-SSL"].PURL)
// libwww-perl's packlist is auto/LWP/.packlist, so it shares no (name, version) key with
// the distribution and is paired through the provides module map instead
assert.NotContains(t, byName, "LWP", "the module-named packlist must not become its own package")
lwp := byName["libwww-perl"]
require.NotEmpty(t, lwp.Name)
assert.Equal(t, "pkg:cpan/libwww-perl@6.83?author=OALDERS", lwp.PURL)
assert.ElementsMatch(t, []string{
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/libwww-perl-6.83/install.json",
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/libwww-perl-6.83/MYMETA.json",
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/LWP/.packlist",
"/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod",
}, realPaths(lwp))
// carton's packlist points at the application-local copy, not the site_perl one
assert.ElementsMatch(t, []string{
"/app/local/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json",
"/app/local/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/MYMETA.json",
"/app/local/lib/perl5/x86_64-linux-gnu/auto/Text/CSV/.packlist",
"/app/local/lib/perl5/x86_64-linux-gnu/perllocal.pod",
}, realPaths(byName["Text-CSV"]))
// name provenance: Dist records the distvname install.json was read from, MainModule
// records that the name was inferred from an auto/ path instead. Never both, and a merged
// package keeps install.json's.
metadata := func(name string) pkg.CpanDistribution {
md, ok := byName[name].Metadata.(pkg.CpanDistribution)
require.True(t, ok, name)
return md
}
assert.Equal(t, "libwww-perl-6.83", metadata("libwww-perl").Dist)
assert.Empty(t, metadata("libwww-perl").MainModule)
assert.Equal(t, "IO::Socket::SSL", metadata("IO-Socket-SSL").MainModule)
assert.Empty(t, metadata("IO-Socket-SSL").Dist)
// the paths are reported as the packlist recorded them, not filtered to the ones still on
// disk. IO/Socket/SSL/Utils.pm is deleted in the image on purpose: a path the packlist
// claims and the filesystem lacks means the file was removed or overwritten, which is
// exactly what an SBOM consumer wants to see.
assert.Equal(t, []string{
"/usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL.pm",
"/usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL.pod",
"/usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL/Intercept.pm",
"/usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL/PublicSuffix.pm",
"/usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL/Utils.pm",
}, metadata("IO-Socket-SSL").OwnedFiles())
// the merge has to carry the packlist's file list onto the install.json package that won,
// or a merged distribution would own nothing
assert.Contains(t, metadata("libwww-perl").OwnedFiles(), "/usr/local/lib/perl5/site_perl/5.40.4/LWP.pm")
assert.Contains(t, metadata("libwww-perl").OwnedFiles(), "/usr/local/bin/lwp-request")
// NO_PACKLIST leaves an install.json and nothing else, so there is no file list at all
assert.Empty(t, metadata("JSON-PP").OwnedFiles())
}).
TestCataloger(t, NewCpanInstalledCataloger())
}
// TestCpanInstalledCataloger_packlistOnly covers the installs that write no install.json, which is
// the majority of real images and where the version has to come from perllocal.pod or the installed
// .pm. The image runs ExtUtils::MakeMaker directly, cpanm against local tarballs, and CPAN.pm; the
// Dockerfile asserts that none of them left a .meta directory behind.
func TestCpanInstalledCataloger_packlistOnly(t *testing.T) {
pkgtest.NewCatalogTester().
WithImageResolver(t, "image-cpan-packlist-only").
ExpectsPackageStrings([]string{
"App-cpanminus @ 1.7049 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/App/cpanminus/.packlist)",
// Encode's $VERSION is declared bare and computed with sprintf from an RCS Revision keyword
// inside a BEGIN block, so nothing static can read it. This version exists only because
// perllocal.pod records what EUMM evaluated at build time.
"Encode @ 3.24 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Encode/.packlist)",
"IO-Socket-SSL @ 2.099 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/IO/Socket/SSL/.packlist)",
// installed with NO_PERLLOCAL, so this one had to be scraped out of JSON/PP.pm
"JSON-PP @ 4.16 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/JSON/PP/.packlist)",
// libwww-perl's EUMM NAME is LWP, so an installed tree only ever yields "LWP". Resolving
// that back to the distribution is left to the vulnerability data, which carries the map.
"LWP @ 5.836 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/LWP/.packlist)",
"Net-SSLeay @ 1.96 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Net/SSLeay/.packlist)",
// installed at 2.02 and then upgraded to 2.06. perllocal.pod is append-only, so both
// stanzas are still there and the later one has to win.
"Text-CSV @ 2.06 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Text/CSV/.packlist)",
}).
ExpectsAssertion(func(t *testing.T, pkgs []pkg.Package, _ []artifact.Relationship) {
byName := make(map[string]pkg.Package)
for _, p := range pkgs {
byName[p.Name] = p
}
// the perllocal.pod that supplied the version is recorded as supporting evidence, and no
// .pm was read at all
assert.ElementsMatch(t, []string{
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Encode/.packlist",
"/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod",
}, realPaths(byName["Encode"]))
// NO_PERLLOCAL wrote no stanza, so the .pm named by the packlist is the evidence instead
assert.ElementsMatch(t, []string{
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/JSON/PP/.packlist",
"/usr/local/lib/perl5/site_perl/5.40.4/JSON/PP.pm",
}, realPaths(byName["JSON-PP"]))
// no install.json anywhere, so no distribution has an author and no purl has a qualifier
for _, p := range pkgs {
assert.NotContains(t, p.PURL, "author=", p.Name)
assert.Empty(t, p.Licenses.ToSlice(), p.Name)
}
}).
TestCataloger(t, NewCpanInstalledCataloger())
}
// TestCpanInstalledCataloger_twoInstallTrees covers an image carrying two perl lib trees, each with
// its own perllocal.pod naming the same module at a different version. Picking the stanza from the
// wrong tree would be silent, so both are pinned.
//
// The same image installs Text-CSV 2.06 from a mirror into both trees, so both hold an install.json
// agreeing on name and version. Those are two installed things and stay two packages: the tree is
// part of what pairs evidence, not the name and version alone.
func TestCpanInstalledCataloger_twoInstallTrees(t *testing.T) {
pkgtest.NewCatalogTester().
WithImageResolver(t, "image-cpan-two-install-trees").
ExpectsPackageStrings([]string{
"App-cpanminus @ 1.7049 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/App/cpanminus/.packlist)",
"IO-Socket-SSL @ 2.099 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/IO/Socket/SSL/.packlist)",
"JSON-PP @ 2.27300 (/opt/app/lib/perl5/x86_64-linux-gnu/auto/JSON/PP/.packlist)",
"JSON-PP @ 4.16 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/JSON/PP/.packlist)",
"Net-SSLeay @ 1.96 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Net/SSLeay/.packlist)",
"Text-CSV @ 2.06 (/opt/app/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json)",
"Text-CSV @ 2.06 (/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json)",
}).
ExpectsAssertion(func(t *testing.T, pkgs []pkg.Package, _ []artifact.Relationship) {
byVersion := make(map[string]pkg.Package)
var textCSV []pkg.Package
for _, p := range pkgs {
switch p.Name {
case "JSON-PP":
byVersion[p.Version] = p
case "Text-CSV":
textCSV = append(textCSV, p)
}
}
require.Len(t, byVersion, 2)
// each tree's Text-CSV carries only its own tree's evidence, and within a tree the
// install.json and the packlist still merged into the one package
require.Len(t, textCSV, 2)
byTree := make(map[string][]string)
for _, p := range textCSV {
byTree[installTree(p)] = realPaths(p)
}
require.Len(t, byTree, 2)
assert.ElementsMatch(t, []string{
"/opt/app/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json",
"/opt/app/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/MYMETA.json",
"/opt/app/lib/perl5/x86_64-linux-gnu/auto/Text/CSV/.packlist",
"/opt/app/lib/perl5/x86_64-linux-gnu/perllocal.pod",
}, byTree["/opt/app/lib/perl5/x86_64-linux-gnu"])
assert.ElementsMatch(t, []string{
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json",
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/Text-CSV-2.06/MYMETA.json",
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/Text/CSV/.packlist",
"/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod",
}, byTree["/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu"])
// each tree's version came from that tree's own perllocal.pod. Both installs are of the
// same module, so without the libdir rule one of these would carry the other's version.
assert.Contains(t, realPaths(byVersion["2.27300"]), "/opt/app/lib/perl5/x86_64-linux-gnu/perllocal.pod")
assert.Contains(t, realPaths(byVersion["4.16"]), "/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod")
// ExtUtils::Packlist merges an existing packlist for the same module when writing a new
// one, so the local-lib install's packlist genuinely claims the site_perl copy too. The
// file list is reported as written rather than filtered to the prefix.
owned := byVersion["2.27300"].Metadata.(pkg.CpanDistribution).OwnedFiles()
assert.Contains(t, owned, "/opt/app/lib/perl5/JSON/PP.pm")
assert.Contains(t, owned, "/usr/local/lib/perl5/site_perl/5.40.4/JSON/PP.pm")
}).
TestCataloger(t, NewCpanInstalledCataloger())
}
// TestCpanMetaCataloger_unpackedReleases covers release tarballs unpacked but never installed, plus a
// source repository working tree.
func TestCpanMetaCataloger_unpackedReleases(t *testing.T) {
pkgtest.NewCatalogTester().
WithImageResolver(t, "image-cpan-unpacked-releases").
ExpectsPackageStrings([]string{
// CPAN Meta Spec 1.4, which is all that roughly 43% of current releases ship. Genuine EUMM
// 6.56 output: license is a bare string rather than an array and version is an unquoted
// number, both of which the lenient field types absorb.
"Text-CSV @ 1.21 (/opt/releases/Text-CSV-1.21/META.yml)",
// a Dist::Zilla release, which ships dist.ini inside the tarball and lists it in its own
// MANIFEST, so a dist.ini beside a META.json says nothing about whether this is a release.
// It ships both meta files and META.json is reported, once.
"Try-Tiny @ 0.32 (/opt/releases/Try-Tiny-0.32/META.json)",
// nothing from /opt/releases/Parallel-Depend-4.10: its META.yml is real EUMM output that no
// strict YAML parser accepts, because the distribution's multi-line ABSTRACT was written
// straight into `abstract:` with no quoting and no block scalar. That has to skip the
// directory, not fail the scan.
// nothing from /src/libwww-perl either: a committed META.json records the last release and
// drifts from the working tree (6.83 there against 6.84 in lib/LWP.pm on the same commit).
// The absence of a MANIFEST is what rejects it, since dzil generates one at build time.
}).
ExpectsAssertion(func(t *testing.T, pkgs []pkg.Package, _ []artifact.Relationship) {
byName := make(map[string]pkg.Package)
for _, p := range pkgs {
byName[p.Name] = p
}
// Try-Tiny's META.json carries an x_spdx_expression, which is preferred over the
// license token beside it
require.Len(t, byName["Try-Tiny"].Licenses.ToSlice(), 1)
assert.Equal(t, "MIT", byName["Try-Tiny"].Licenses.ToSlice()[0].Value)
// spec 1.4 has no x_spdx_expression, so the bare license string is what is left
require.Len(t, byName["Text-CSV"].Licenses.ToSlice(), 1)
assert.Equal(t, "perl", byName["Text-CSV"].Licenses.ToSlice()[0].Value)
}).
TestCataloger(t, NewCpanMetaCataloger())
}
// TestCpanCatalogers_buildLeftoversAndVendoredRelease covers the ordinary state of any tree that
// installed distributions and did not clean up after itself: cpanm leaves ~/.cpanm/work and CPAN.pm
// leaves ~/.cpan/build, neither by choice of the user.
//
// Both leftovers genuinely are unpacked release tarballs, MANIFEST included, so the sibling-MANIFEST
// gate is working correctly and still admits them. They have to be excluded on the path instead, or
// every distribution in such a tree is reported twice: once from its installed evidence and once from
// the copy the client unpacked to build it. Both catalogers are registered for directory scans, so
// that is where the duplication would be user-visible.
//
// The same fixture carries a release deliberately unpacked under /opt/vendor, which is the other half
// of the path rule and is reported. It is the same distribution and version as the installed copy,
// and it produces a second package rather than merging, for the reason
// TestCpanCatalogers_vendoredReleaseCannotMerge documents.
func TestCpanCatalogers_buildLeftoversAndVendoredRelease(t *testing.T) {
found := catalogBoth(t, "image-cpan-build-leftovers")
var names []string
for _, p := range found {
names = append(names, p.Name+" @ "+p.Version+" ("+p.FoundBy+")")
}
// ElementsMatch rather than Equal: the two HTML-Tagset packages agree on name and version, so
// pkg.Collection.Sorted breaks the tie on package ID, which is a structural hash and not something
// worth pinning
assert.ElementsMatch(t, []string{
"App-cpanminus @ 1.7049 (perl-cpan-installed-cataloger)",
"HTML-Tagset @ 3.24 (perl-cpan-installed-cataloger)",
"HTML-Tagset @ 3.24 (perl-cpan-meta-cataloger)",
"IO-Socket-SSL @ 2.099 (perl-cpan-installed-cataloger)",
"MIME-Base32 @ 1.303 (perl-cpan-installed-cataloger)",
"Net-SSLeay @ 1.96 (perl-cpan-installed-cataloger)",
"URI @ 5.35 (perl-cpan-installed-cataloger)",
}, names, "each installed distribution is reported once, nothing comes from the leftover trees, and the vendored copy is reported separately")
}
// TestCpanCatalogers_vendoredReleaseCannotMerge asserts what happens when a release deliberately
// unpacked outside any build directory sits beside the installed copy of the same distribution and
// version.
//
// The intuitive expectation is one package carrying both locations as evidence. That cannot hold, and
// this test pins the real behavior at two packages rather than asserting the intuition and failing.
//
// Why: the two packages come from two different catalogers, so nothing inside either cataloger can
// pair them. mergeDistributions only ever sees one cataloger's output. The only cross-cataloger merge
// syft has is pkg.Collection.add, at syft/pkg/collection.go:119, which merges on artifact.ID equality
// alone. artifact.ID is a structural hash of the whole Package (syft/pkg/package.go:58, via
// artifact.IDByHash), and Locations is a hashed field: the `hash:"ignore"` tags at
// syft/pkg/package.go:30-51 exempt FoundBy, Language, CPEs, and PURL, but not Locations, Licenses, or
// Metadata. Two packages whose locations differ therefore cannot have the same ID, and two packages
// found at the same location are the same evidence and not this case at all. So "same identity,
// merged locations" is unreachable by construction, for any pair of catalogers, not just these two.
func TestCpanCatalogers_vendoredReleaseCannotMerge(t *testing.T) {
var installed, unpacked pkg.Package
for _, p := range catalogBoth(t, "image-cpan-build-leftovers") {
if p.Name != "HTML-Tagset" {
continue
}
// the two agree on name and version, so the duplication is visible and consistent
assert.Equal(t, "3.24", p.Version)
assert.Equal(t, "pkg:cpan/HTML-Tagset@3.24", p.PURL)
switch p.FoundBy {
case "perl-cpan-installed-cataloger":
installed = p
case "perl-cpan-meta-cataloger":
unpacked = p
}
}
require.NotEmpty(t, installed.Name)
require.NotEmpty(t, unpacked.Name)
assert.ElementsMatch(t, []string{
"/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/auto/HTML/Tagset/.packlist",
"/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod",
}, realPaths(installed))
assert.ElementsMatch(t, []string{
"/opt/vendor/HTML-Tagset-3.24/META.json",
}, realPaths(unpacked))
// Locations, Licenses and Metadata are the whole of the difference here, and every other differing
// field (FoundBy, PURL, CPEs) is `hash:"ignore"`. Overwriting just those three hashed fields makes
// the IDs equal, which is the direct demonstration of what keeps the two packages apart.
require.NotEqual(t, installed.ID(), unpacked.ID())
assert.Empty(t, installed.Licenses.ToSlice(), "a packlist carries no license")
assert.Equal(t, "artistic_2", unpacked.Licenses.ToSlice()[0].Value, "META.json does, so Licenses differs too")
relocated := unpacked
relocated.Locations = installed.Locations
relocated.Licenses = installed.Licenses
relocated.Metadata = installed.Metadata
relocated.SetID()
assert.Equal(t, installed.ID(), relocated.ID(), "with the hashed fields equalized the IDs match, and only then")
}
// TestCpanCatalogers_coreDistributionsNotCataloged pins the zero coverage of distributions bundled
// inside the interpreter, which the package doc comment states as a gap.
//
// The image is a perl with no CPAN installs at all: Encode and Storable sit in the core lib tree, the
// only packlist is perl's own and has no auto/ segment, and there is no .meta and no distro package.
// Nothing on disk carries a core distribution's version.
//
// Storable.pm does declare a readable $VERSION, so the interesting half is that no package is
// invented from it, and neither is one invented from the perllocal.pod stanzas that name the
// distributions the base image preinstalled into a site_perl this image does not have. The
// interpreter's version is never used as a stopgap either: a distribution name carrying a wrong or
// absent version matches either everything or nothing in an advisory range, which is worse than the
// gap. Closing this needs a generated Module::CoreList table keyed by perl version.
func TestCpanCatalogers_coreDistributionsNotCataloged(t *testing.T) {
assert.Empty(t, catalogBoth(t, "image-perl-core-only"))
}
// TestCpanCatalogers_metadataTypes pins the metadata type each cataloger emits along with the JSON
// name it serializes as. Both are user-visible API: a consumer decides whether a finding is installed
// and loadable or merely unpacked on disk from the type name, and must not have to string match
// FoundBy.
func TestCpanCatalogers_metadataTypes(t *testing.T) {
installed := catalogImage(t, "image-cpan-mirror-installs", NewCpanInstalledCataloger())
unpacked := catalogImage(t, "image-cpan-unpacked-releases", NewCpanMetaCataloger())
require.NotEmpty(t, installed)
for _, p := range installed {
assert.IsType(t, pkg.CpanDistribution{}, p.Metadata, p.Name)
assert.Equal(t, "cpan-distribution", packagemetadata.JSONName(p.Metadata), p.Name)
}
require.NotEmpty(t, unpacked)
for _, p := range unpacked {
assert.IsType(t, pkg.CpanUnpackedRelease{}, p.Metadata, p.Name)
assert.Equal(t, "cpan-unpacked-release", packagemetadata.JSONName(p.Metadata), p.Name)
}
}
// imageResolver resolves an image fixture the way pkgtest's WithImageResolver does, for the tests
// that need the resolver itself rather than the CatalogTester around it.
func imageResolver(t *testing.T, fixture string) file.Resolver {
t.Helper()
img := imagetest.GetFixtureImage(t, "docker-archive", fixture)
resolver, err := stereoscopesource.New(img, stereoscopesource.ImageConfig{Reference: fixture}).
FileResolver(source.SquashedScope)
require.NoError(t, err)
return resolver
}
func catalogImage(t *testing.T, fixture string, cataloger pkg.Cataloger) []pkg.Package {
t.Helper()
pkgs, _, err := cataloger.Catalog(pkgtest.Context(t), imageResolver(t, fixture))
require.NoError(t, err)
return pkgs
}
// catalogBoth runs both catalogers over one image and aggregates into a pkg.Collection, which is what
// the real cataloging pipeline does, so any cross-cataloger merge that production performs happens
// here too.
func catalogBoth(t *testing.T, fixture string) []pkg.Package {
t.Helper()
resolver := imageResolver(t, fixture)
collection := pkg.NewCollection()
for _, cataloger := range []pkg.Cataloger{NewCpanInstalledCataloger(), NewCpanMetaCataloger()} {
pkgs, _, err := cataloger.Catalog(pkgtest.Context(t), resolver)
require.NoError(t, err)
collection.Add(pkgs...)
}
return collection.Sorted()
}
func realPaths(p pkg.Package) []string {
var paths []string
for _, l := range p.Locations.ToSlice() {
paths = append(paths, l.RealPath)
}
return paths
}

View File

@ -0,0 +1,185 @@
package perl
import (
"encoding/json"
"io"
"path"
"go.yaml.in/yaml/v3"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/file"
)
// cpanMeta models the parts of a CPAN Meta Spec document syft uses. One shape covers all three:
// META.json (spec 2), MYMETA.json (the configure-time rendering of META.json), and META.yml (spec 1.4).
type cpanMeta struct {
Name string `json:"name" yaml:"name"`
Version scalar `json:"version" yaml:"version"`
Abstract string `json:"abstract" yaml:"abstract"`
License stringList `json:"license" yaml:"license"`
Provides map[string]providesEntry `json:"provides" yaml:"provides"`
// SPDX is preferred over License when present: a single SPDX expression the author wrote, rather than
// a CPAN::Meta license token that has to be translated
SPDX string `json:"x_spdx_expression" yaml:"x_spdx_expression"`
// Prereqs is the spec 2 shape: phase -> relation -> module -> version
Prereqs map[string]map[string]map[string]scalar `json:"prereqs" yaml:"prereqs"`
// Requires and BuildRequires are the spec 1.4 shape, which has no phase nesting. develop and test
// have no 1.4 equivalent, which costs nothing because they are excluded anyway.
Requires map[string]scalar `json:"requires" yaml:"requires"`
BuildRequires map[string]scalar `json:"build_requires" yaml:"build_requires"`
}
type providesEntry struct {
Version scalar `json:"version" yaml:"version"`
}
// scalar is a value that CPAN metadata writes as either a quoted string or a bare number. Old
// distributions ship `"version": 1.23` where the spec calls for a string, and a strict string field
// there would fail the whole document rather than one field.
type scalar string
func (s *scalar) UnmarshalJSON(b []byte) error {
raw := string(b)
if raw == "null" {
return nil
}
if len(raw) >= 2 && raw[0] == '"' {
var str string
if err := json.Unmarshal(b, &str); err != nil {
return err
}
*s = scalar(str)
return nil
}
*s = scalar(raw)
return nil
}
// UnmarshalYAML exists for the same reason UnmarshalJSON does, and separately from it: the YAML decoder
// does not honor json.Unmarshaler, so without this a `version: 1.23` in a META.yml would fail the whole
// document. A YAML scalar node's value is already the text as written, so there is nothing to convert.
func (s *scalar) UnmarshalYAML(value *yaml.Node) error {
if value.Tag == "!!null" {
return nil
}
*s = scalar(value.Value)
return nil
}
// stringList is a value that is a list of strings in CPAN Meta Spec 2 but a bare string in spec 1.4.
type stringList []string
func (s *stringList) UnmarshalJSON(b []byte) error {
var many []string
if err := json.Unmarshal(b, &many); err == nil {
*s = many
return nil
}
var one string
if err := json.Unmarshal(b, &one); err == nil {
*s = stringList{one}
}
return nil
}
func (s *stringList) UnmarshalYAML(value *yaml.Node) error {
var many []string
if err := value.Decode(&many); err == nil {
*s = many
return nil
}
var one string
if err := value.Decode(&one); err == nil {
*s = stringList{one}
}
return nil
}
// readMyMeta reads the MYMETA.json that cpanm, cpm, and carton write beside install.json. A missing or
// unreadable MYMETA is not an error: the distribution is still real, it just has fewer facts attached.
func readMyMeta(resolver file.Resolver, installJSONLocation file.Location) (*file.Location, *cpanMeta) {
if resolver == nil {
return nil, nil
}
p := path.Join(path.Dir(installJSONLocation.Path()), "MYMETA.json")
locations, err := resolver.FilesByPath(p)
if err != nil || len(locations) == 0 {
return nil, nil
}
m, err := decodeMeta(resolver, locations[0])
if err != nil {
log.WithFields("path", p, "error", err).Debug("unable to parse CPAN MYMETA.json")
return nil, nil
}
return &locations[0], m
}
func decodeMeta(resolver file.Resolver, location file.Location) (*cpanMeta, error) {
return decode(resolver, location, func(r io.Reader, m *cpanMeta) error {
return json.NewDecoder(r).Decode(m)
})
}
// decodeMetaYAML reads a spec 1.4 META.yml. CPAN META.yml files are rejected by strict YAML parsers often
// enough to matter: pre-6.46 ExtUtils::MakeMaker writes values with no quoting or escaping at all, so a
// multi-line ABSTRACT, a value that opens a quote it never closes, and a value starting with >= (which
// YAML reads as a folded block scalar) all get emitted verbatim. Measured over 454 releases from 2004 to
// 2009, five fail. That is a routine failure, not a defensive one, and the caller degrades on it.
func decodeMetaYAML(resolver file.Resolver, location file.Location) (*cpanMeta, error) {
return decode(resolver, location, func(r io.Reader, m *cpanMeta) error {
return yaml.NewDecoder(r).Decode(m)
})
}
func decode(resolver file.Resolver, location file.Location, into func(io.Reader, *cpanMeta) error) (*cpanMeta, error) {
reader, err := resolver.FileContentsByLocation(location)
if err != nil {
return nil, err
}
defer internal.CloseAndLogError(reader, location.Path())
var m cpanMeta
if err := into(reader, &m); err != nil {
return nil, err
}
return &m, nil
}
// prereqModules returns the module names this distribution requires at runtime and build time.
// develop and test phases are excluded: they describe how the distribution was authored, not what a
// deployed copy of it needs. A spec 1.4 document has no phases at all, so its flat requires and
// build_requires maps feed the same two buckets.
func (m *cpanMeta) prereqModules() []string {
var modules []string
for _, phase := range []string{"runtime", "build"} {
for module := range m.Prereqs[phase]["requires"] {
modules = append(modules, module)
}
}
for _, flat := range []map[string]scalar{m.Requires, m.BuildRequires} {
for module := range flat {
modules = append(modules, module)
}
}
return modules
}
// licenseExpressions returns the license values to record on the package. x_spdx_expression wins when
// present: it is a single SPDX expression, which syft's license parsing understands directly, rather
// than a CPAN license shortname such as perl_5 or artistic_2.
func (m *cpanMeta) licenseExpressions() []string {
if m.SPDX != "" {
return []string{m.SPDX}
}
return m.License
}

View File

@ -0,0 +1,128 @@
package perl
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.yaml.in/yaml/v3"
)
// yaml.v3 does not honor json.Unmarshaler, so the lenient types that already tolerate both license and
// version shapes in JSON need their own UnmarshalYAML or a spec 1.4 META.yml fails on one field.
func Test_cpanMeta_lenientFieldsInYAML(t *testing.T) {
tests := []struct {
name string
contents string
expectedVersion scalar
expectedLicense stringList
}{
{
// what a real ExtUtils::MakeMaker 6.x META.yml writes: version unquoted, license a bare string
name: "spec 1.4 shapes",
contents: "name: Text-CSV\nversion: 1.21\nlicense: perl\n",
expectedVersion: "1.21",
expectedLicense: stringList{"perl"},
},
{
name: "quoted version and a license list",
contents: "name: URI\nversion: '5.35'\nlicense:\n - perl_5\n - artistic_2\n",
expectedVersion: "5.35",
expectedLicense: stringList{"perl_5", "artistic_2"},
},
{
// a version that would read as a v-string rather than a number
name: "v-string version",
contents: "name: App-cpm\nversion: v1.1.4\n",
expectedVersion: "v1.1.4",
},
{
name: "explicit nulls",
contents: "name: Foo\nversion: ~\nlicense: ~\n",
expectedVersion: "",
expectedLicense: nil,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var m cpanMeta
require.NoError(t, yaml.Unmarshal([]byte(tt.contents), &m))
assert.Equal(t, tt.expectedVersion, m.Version)
assert.Equal(t, tt.expectedLicense, m.License)
})
}
}
func Test_cpanMeta_licenseExpressions(t *testing.T) {
tests := []struct {
name string
contents string
expected []string
}{
{
// an SPDX expression is what syft's license parsing understands directly, where perl_5 is a
// CPAN::Meta token that has to be translated
name: "x_spdx_expression wins",
contents: `{"license":["perl_5"],"x_spdx_expression":"Artistic-1.0-Perl OR GPL-1.0-or-later"}`,
expected: []string{"Artistic-1.0-Perl OR GPL-1.0-or-later"},
},
{
name: "license is used when there is no expression",
contents: `{"license":["perl_5"]}`,
expected: []string{"perl_5"},
},
{
name: "single-string license, as spec 1.4 writes it",
contents: `{"license":"perl"}`,
expected: []string{"perl"},
},
{
name: "neither present",
contents: `{"name":"Foo"}`,
expected: nil,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var m cpanMeta
require.NoError(t, json.Unmarshal([]byte(tt.contents), &m))
assert.Equal(t, tt.expected, m.licenseExpressions())
})
}
}
func Test_cpanMeta_prereqModules(t *testing.T) {
tests := []struct {
name string
contents string
expected []string
}{
{
name: "spec 2 prereqs, keyed by phase",
contents: `{"prereqs":{
"runtime": {"requires": {"MIME::Base64": "2.1"}},
"build": {"requires": {"ExtUtils::MakeMaker": "0"}},
"test": {"requires": {"Test::More": "0.88"}},
"develop": {"requires": {"Pod::Coverage": "0"}}
}}`,
expected: []string{"MIME::Base64", "ExtUtils::MakeMaker"},
},
{
// spec 1.4 has no phase nesting at all, so the two flat maps feed the same two buckets
name: "spec 1.4 flat requirements",
contents: `{"requires":{"IO::Handle":"0"},"build_requires":{"Test::More":"0.47"}}`,
expected: []string{"IO::Handle", "Test::More"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var m cpanMeta
require.NoError(t, json.Unmarshal([]byte(tt.contents), &m))
assert.ElementsMatch(t, tt.expected, m.prereqModules())
})
}
}

View File

@ -0,0 +1,74 @@
package perl
import (
"sort"
"strings"
"github.com/anchore/packageurl-go"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
// newCpanPackage builds a package for a CPAN distribution. Note that the name and version are passed
// separately rather than living on the metadata struct: they would be verbatim copies of the package
// fields, so they are not duplicated into metadata.
//
// The metadata is an `any` because the two evidence tiers carry different types, and the author is an
// explicit parameter for the same reason: pkg.CpanUnpackedRelease has no PAUSE path, so there is no
// field on it the purl could be built from.
func newCpanPackage(name, version, author string, md any, licenses []pkg.License, locations ...file.Location) pkg.Package {
p := pkg.Package{
Name: name,
Version: version,
PURL: packageURL(name, version, author),
Locations: file.NewLocationSet(locations...),
Licenses: pkg.NewLicenseSet(licenses...),
Language: pkg.Perl,
Type: pkg.CpanPkg,
Metadata: md,
}
p.SetID()
return p
}
// packageURL returns the PURL for a CPAN distribution. The distribution name is used verbatim: CPAN
// names are case sensitive. The author is emitted as a qualifier rather than as the namespace, which
// is what the current purl spec prefers, and is omitted entirely when unknown rather than guessed.
func packageURL(name, version, author string) string {
var qualifiers packageurl.Qualifiers
if author != "" {
qualifiers = append(qualifiers, packageurl.Qualifier{Key: "author", Value: author})
}
return packageurl.NewPackageURL(packageurl.TypeCpan, "", name, version, qualifiers, "").ToString()
}
// authorFromPathname extracts the CPAN author (PAUSE) ID from a PAUSE path such as
// O/OA/OALDERS/URI-5.35.tar.gz. A path with fewer than three segments yields no author rather than a
// partial one.
func authorFromPathname(p string) string {
parts := strings.Split(p, "/")
if len(parts) < 3 {
return ""
}
return parts[2]
}
// modulesFromProvides converts a CPAN metadata `provides` map into a stable, sorted module list.
// The `file` field is dropped: the installed path is mechanical from the module name.
func modulesFromProvides(provides map[string]providesEntry) []pkg.CpanModule {
if len(provides) == 0 {
return nil
}
modules := make([]pkg.CpanModule, 0, len(provides))
for name, entry := range provides {
modules = append(modules, pkg.CpanModule{Name: name, Version: string(entry.Version)})
}
sort.Slice(modules, func(i, j int) bool { return modules[i].Name < modules[j].Name })
return modules
}

View File

@ -0,0 +1,76 @@
package perl
import (
"testing"
"github.com/stretchr/testify/assert"
)
func Test_packageURL(t *testing.T) {
tests := []struct {
name string
dist string
version string
author string
want string
}{
{
name: "author available",
dist: "URI",
version: "5.35",
author: "OALDERS",
want: "pkg:cpan/URI@5.35?author=OALDERS",
},
{
name: "no author known, e.g. a packlist-derived package",
dist: "Text-CSV",
version: "2.06",
want: "pkg:cpan/Text-CSV@2.06",
},
{
name: "case is preserved",
dist: "JSON-MaybeXS",
version: "1.004008",
author: "ETHER",
want: "pkg:cpan/JSON-MaybeXS@1.004008?author=ETHER",
},
{
name: "v-prefixed version is preserved",
dist: "CPAN-02Packages-Search",
version: "v1.0.0",
author: "SKAJI",
want: "pkg:cpan/CPAN-02Packages-Search@v1.0.0?author=SKAJI",
},
{
name: "no version",
dist: "IO-Socket-SSL",
want: "pkg:cpan/IO-Socket-SSL",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, packageURL(tt.dist, tt.version, tt.author))
})
}
}
func Test_authorFromPathname(t *testing.T) {
tests := []struct {
pathname string
want string
}{
{pathname: "O/OA/OALDERS/URI-5.35.tar.gz", want: "OALDERS"},
{pathname: "S/SK/SKAJI/CPAN-02Packages-Search-v1.0.0.tar.gz", want: "SKAJI"},
// not enough segments to name an author: better to omit than to guess
{pathname: "OALDERS/URI-5.35.tar.gz", want: ""},
{pathname: "URI-5.35.tar.gz", want: ""},
{pathname: "", want: ""},
}
for _, tt := range tests {
t.Run(tt.pathname, func(t *testing.T) {
assert.Equal(t, tt.want, authorFromPathname(tt.pathname))
})
}
}

View File

@ -0,0 +1,95 @@
package perl
import (
"context"
"encoding/json"
"path"
"strings"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
// installJSON is the record cpanm (>= 1.5000), cpm, and carton write for every distribution they install.
type installJSON struct {
// Name is the MODULE name, not the distribution: libwww-perl records "LWP" here, and
// CPAN-02Packages-Search records "CPAN::02Packages::Search". It is never the package name.
Name string `json:"name"`
Dist string `json:"dist"` // distribution with version, e.g. libwww-perl-5.836
Version scalar `json:"version"`
Pathname string `json:"pathname"` // PAUSE path, e.g. O/OA/OALDERS/URI-5.35.tar.gz
Provides map[string]providesEntry `json:"provides"`
}
func parseInstallJSON(ctx context.Context, resolver file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
var doc installJSON
if err := json.NewDecoder(reader).Decode(&doc); err != nil {
log.WithFields("path", reader.Path(), "error", err).Debug("unable to parse CPAN install.json")
return nil, nil, nil
}
name := distributionName(doc)
if name == "" || doc.Version == "" {
log.WithFields("path", reader.Path()).Debug("CPAN install.json is missing a distribution name or version")
return nil, nil, nil
}
locations := []file.Location{reader.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation)}
var licenses []pkg.License
if myMetaLocation, myMeta := readMyMeta(resolver, reader.Location); myMeta != nil {
licenses = pkg.NewLicensesFromLocationWithContext(ctx, *myMetaLocation, myMeta.licenseExpressions()...)
locations = append(locations, myMetaLocation.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.SupportingEvidenceAnnotation))
}
md := pkg.CpanDistribution{
Dist: doc.Dist,
Author: authorFromPathname(doc.Pathname),
Path: doc.Pathname,
Modules: modulesFromProvides(doc.Provides),
}
return []pkg.Package{newCpanPackage(name, string(doc.Version), md.Author, md, licenses, locations...)}, nil, nil
}
// distributionName resolves the distribution name, which every consumer of these packages is keyed by.
// The `name` field cannot be used: it holds the module name, so libwww-perl would be reported as LWP
// and CGI-Session as CGI::Session, which no advisory or index is filed against.
//
// `dist` carries the distribution with its version appended, and the version is known, so trimming the
// suffix is exact. That also resolves the case that looks ambiguous when splitting on the last dash:
// CPAN-02Packages-Search-v1.0.0 minus v1.0.0 leaves CPAN-02Packages-Search, where a dash rule would
// have to guess whether v1.0.0 is one segment or three.
func distributionName(doc installJSON) string {
if doc.Dist != "" {
if name := strings.TrimSuffix(doc.Dist, "-"+string(doc.Version)); name != "" && name != doc.Dist {
return name
}
// no version suffix to trim, so dist is already the bare name. cpm was checked and does append
// the version just as cpanm does (see TestInstallJSON_cpmFormatting), so this is a guard against
// records neither installer is known to write rather than a shape any of them produce.
return doc.Dist
}
// older cpanm and hand-rolled records omit dist; the PAUSE path basename carries the same
// <Dist>-<Version>.tar.gz shape
return distributionFromPathname(doc.Pathname, string(doc.Version))
}
// distributionFromPathname recovers the distribution from a PAUSE path such as
// G/GA/GAAS/libwww-perl-5.836.tar.gz, which is the only evidence left when `dist` is absent.
func distributionFromPathname(pathname, version string) string {
base := path.Base(pathname)
if base == "." || base == "/" {
return ""
}
for _, ext := range []string{".tar.gz", ".tar.bz2", ".tgz", ".zip"} {
base = strings.TrimSuffix(base, ext)
}
return strings.TrimSuffix(base, "-"+version)
}

View File

@ -0,0 +1,157 @@
package perl
import (
"context"
"encoding/json"
"io"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
)
// the record shape here is cpanm's, reduced to the fields the parser reads. What matters is that
// `name` is the module (CPAN::02Packages::Search) and must not become the package name, and that the
// version is a v-string, so trimming it off `dist` is what makes the CPAN-02Packages-Search-v1.0.0
// split unambiguous where a rule based on the last dash would have to guess.
//
// The byte-level shape of what the clients really write is covered against real files by
// TestInstallJSON_clientFormatting.
const installJSONFixture = `{"target":"CPAN::02Packages::Search","module_name":"CPAN::02Packages::Search",` +
`"version":"v1.0.0","dist":"CPAN-02Packages-Search-v1.0.0","pathname":"S/SK/SKAJI/CPAN-02Packages-Search-v1.0.0.tar.gz",` +
`"provides":{"CPAN::02Packages::Search":{"file":"lib/CPAN/02Packages/Search.pm","version":"v1.0.0"}},` +
`"name":"CPAN::02Packages::Search"}`
func TestParseInstallJSON(t *testing.T) {
const location = "usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux-gnu/.meta/CPAN-02Packages-Search-v1.0.0/install.json"
expected := []pkg.Package{
{
Name: "CPAN-02Packages-Search",
Version: "v1.0.0",
PURL: "pkg:cpan/CPAN-02Packages-Search@v1.0.0?author=SKAJI",
Locations: file.NewLocationSet(file.NewLocation(location)),
Language: pkg.Perl,
Type: pkg.CpanPkg,
Metadata: pkg.CpanDistribution{
Dist: "CPAN-02Packages-Search-v1.0.0",
Author: "SKAJI",
Path: "S/SK/SKAJI/CPAN-02Packages-Search-v1.0.0.tar.gz",
// the provides map, sorted by module name, with per-module versions: a distribution
// built with inherit_version = 0 ships modules at versions other than its own
Modules: []pkg.CpanModule{
{Name: "CPAN::02Packages::Search", Version: "v1.0.0"},
},
},
},
}
// no MYMETA.json is reachable from a single-string resolver, so the package is reported with no
// licenses and no error
pkgtest.NewCatalogTester().
FromString(location, installJSONFixture).
Expects(expected, []artifact.Relationship(nil)).
TestParser(t, parseInstallJSON)
}
// libwww-perl is the canonical name mismatch: `name` is LWP while the distribution every advisory and
// index is filed against is libwww-perl. Reporting LWP here is the bug this guards. The end-to-end
// form of it is TestCpanInstalledCataloger_mirrorInstallEvidence.
func TestParseInstallJSON_distributionNameNotModuleName(t *testing.T) {
const record = `{"name":"LWP","version":"6.83","dist":"libwww-perl-6.83",` +
`"pathname":"O/OA/OALDERS/libwww-perl-6.83.tar.gz",` +
`"provides":{"LWP":{"file":"lib/LWP.pm","version":"6.83"},"LWP::UserAgent":{"file":"lib/LWP/UserAgent.pm","version":"6.83"}}}`
pkgs := parseString(t, "libwww-perl-6.83/install.json", record)
require.Len(t, pkgs, 1)
assert.Equal(t, "libwww-perl", pkgs[0].Name, "the package name must be the distribution, not the LWP module")
assert.Equal(t, "pkg:cpan/libwww-perl@6.83?author=OALDERS", pkgs[0].PURL)
// the module name is not lost, it belongs in the provides map
assert.Equal(t, []string{"LWP", "LWP::UserAgent"}, providedModules(pkgs[0]))
}
// TestInstallJSON_clientFormatting records what was measured about cpm versus cpanm, since the
// question of whether cpm needs its own coverage kept coming up. Both files are read out of the
// image, where cpanm (through carton) and cpm each installed one distribution.
//
// Finding: they are NOT byte-identical. cpanm writes compact JSON with no trailing newline; cpm writes
// canonical JSON, indented three spaces with spaces around the colons and a trailing newline. The
// record itself has the same shape, including `dist` carrying the version suffix, which is worth
// stating because it is the field the distribution name is derived from.
//
// So anything that reads these files by decoding JSON is unaffected by the difference; anything that
// pattern matches the bytes is not.
func TestInstallJSON_clientFormatting(t *testing.T) {
resolver := imageResolver(t, "image-cpan-mirror-installs")
read := func(path string) string {
locations, err := resolver.FilesByPath(path)
require.NoError(t, err)
require.Len(t, locations, 1, path)
reader, err := resolver.FileContentsByLocation(locations[0])
require.NoError(t, err)
defer internal.CloseAndLogError(reader, path)
contents, err := io.ReadAll(reader)
require.NoError(t, err)
return string(contents)
}
// written by cpanm, which carton shells out to
cpanmWritten := read("/app/local/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json")
cpmWritten := read("/srv/api/local/lib/perl5/x86_64-linux-gnu/.meta/Capture-Tiny-0.48/install.json")
assert.NotContains(t, cpanmWritten, "\n", "cpanm writes a single compact line")
assert.Contains(t, cpmWritten, "\"dist\" : ", "cpm writes canonical JSON, with spaces around the colons")
assert.Contains(t, cpmWritten, "\n", "and a trailing newline")
var fromCpanm, fromCpm installJSON
require.NoError(t, json.Unmarshal([]byte(cpanmWritten), &fromCpanm))
require.NoError(t, json.Unmarshal([]byte(cpmWritten), &fromCpm))
assert.Equal(t, "Text-CSV-2.06", fromCpanm.Dist)
assert.Equal(t, "Capture-Tiny-0.48", fromCpm.Dist, "cpm does carry the version suffix on dist, just as cpanm does")
}
func parseString(t *testing.T, location, content string) []pkg.Package {
t.Helper()
pkgs, _, err := parseInstallJSON(context.Background(), nil, nil,
file.NewLocationReadCloser(file.NewLocation(location), io.NopCloser(strings.NewReader(content))))
require.NoError(t, err)
return pkgs
}
func TestParseInstallJSON_degrades(t *testing.T) {
tests := []struct {
name string
content string
}{
{name: "unparseable json", content: "not json at all\n"},
{
// never fall back to splitting the containing directory name
name: "missing name and version",
content: `{"provides":{"URI":{"file":"lib/URI.pm"}}}`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
pkgtest.NewCatalogTester().
FromString("URI-5.35/install.json", tt.content).
Expects(nil, nil).
TestParser(t, parseInstallJSON)
})
}
}

View File

@ -0,0 +1,143 @@
package perl
import (
"bufio"
"context"
"io"
"regexp"
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
// versionPattern matches the $VERSION assignment perl modules declare, e.g. `our $VERSION = '2.06';`.
// The value is captured as written so that both decimal (1.303) and v-string (v1.1.4) forms survive intact.
//
// Requiring a literal after the `=` is what makes the pattern decline the computed forms, which cannot be
// read statically at all: real Encode.pm is a bare `our $VERSION;` followed by
// `$VERSION = sprintf "%d.%02d", q$Revision: 3.24 $ =~ /(\d+)/g;` inside a BEGIN block. Those come from
// perllocal.pod instead, which records what the installer evaluated.
//
// The optional package prefix covers the fully qualified form older Dist::Zilla wrote,
// `$JSON::PP::VERSION = '2.27300';`, and qv() covers `our $VERSION = qv('1.2.3');`.
var versionPattern = regexp.MustCompile(`^[ \t]*(?:our[ \t]+|my[ \t]+)?\$(?:[A-Za-z_][A-Za-z0-9_]*::)*VERSION[ \t]*=[ \t]*(?:qv\()?['"]?(v?[0-9][0-9._]*)`)
// packageVersionPattern matches a version declared in the package statement itself,
// `package CPAN::02Packages::Search v1.0.0;`, which perl 5.12 and later allow. A distribution using it
// need carry no $VERSION assignment anywhere: real CPAN::02Packages::Search v1.0.0 does not.
var packageVersionPattern = regexp.MustCompile(`^[ \t]*package[ \t]+[A-Za-z_][A-Za-z0-9_:]*[ \t]+(v?[0-9][0-9._]*)[ \t]*[;{]`)
// packlistMetadataSuffix matches the trailing `key=value` metadata that ExtUtils::Packlist::read strips
// from a packlist line, following the `/^(.*?)( \w+=.*)$/` it uses. Cutting at the first space instead
// truncates any real path that contains one, which corrupts both the owned-file list and the main-.pm
// version fallback.
var packlistMetadataSuffix = regexp.MustCompile(` \w+=.*$`)
// parsePacklist catalogs a distribution installed by any ExtUtils::MakeMaker or Module::Build run,
// which covers CPAN.pm (which writes no .meta) and the perl image's own preinstalled distributions.
func parsePacklist(_ context.Context, _ file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
name := distNameFromPacklistPath(reader.Path())
if name == "" {
// the perl core packlist has no auto/<Dist> segment; it lists thousands of files and describes
// no single distribution
return nil, nil, nil
}
var files []string
scanner := bufio.NewScanner(reader)
for scanner.Scan() {
// packlist lines are "<path>" or "<path> type=link from=<path>"
if f := strings.TrimSpace(packlistMetadataSuffix.ReplaceAllString(scanner.Text(), "")); f != "" {
files = append(files, f)
}
}
location := reader.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation)
md := pkg.CpanDistribution{MainModule: moduleFromDashedName(name), Files: files}
// the version is left to resolvePacklistVersions, which prefers perllocal.pod and needs the whole
// cataloged set to find it. A distribution with no readable version is still reported, so that it
// stays visible.
return []pkg.Package{newCpanPackage(name, "", "", md, nil, location)}, nil, nil
}
// distNameFromPacklistPath derives the distribution name from the packlist path, e.g.
// .../auto/IO/Socket/SSL/.packlist yields IO-Socket-SSL. A path with no auto/ segment yields "".
func distNameFromPacklistPath(p string) string {
parts := strings.Split(strings.Trim(p, "/"), "/")
last := -1
for i, part := range parts {
if part == "auto" {
last = i
}
}
if last < 0 || last+1 >= len(parts)-1 {
return ""
}
return strings.Join(parts[last+1:len(parts)-1], "-")
}
// mainModuleFile picks the .pm file in the packlist that corresponds to the distribution's main module,
// e.g. Text-CSV is expected to install Text/CSV.pm.
//
// The suffix can match more than one entry, and taking the first is wrong: libwww-perl's packlist lists
// both Bundle/LWP.pm (5.835) and LWP.pm (5.836), with the Bundle copy first. The main module sits at the
// lib root, so the shallowest match is the right one; a nested match is some other module that merely
// ends in the same basename.
func mainModuleFile(dist string, files []string) string {
suffix := "/" + strings.ReplaceAll(dist, "-", "/") + ".pm"
best := ""
bestDepth := 0
for _, f := range files {
if !strings.HasSuffix(f, suffix) {
continue
}
if depth := strings.Count(f, "/"); best == "" || depth < bestDepth {
best, bestDepth = f, depth
}
}
return best
}
func findFile(resolver file.Resolver, p string) (file.Location, bool) {
if resolver == nil || p == "" {
return file.Location{}, false
}
locations, err := resolver.FilesByPath(p)
if err != nil || len(locations) == 0 {
return file.Location{}, false
}
return locations[0], true
}
func readVersionFromPM(resolver file.Resolver, location file.Location) string {
reader, err := resolver.FileContentsByLocation(location)
if err != nil {
return ""
}
defer internal.CloseAndLogError(reader, location.Path())
return versionFromPM(reader)
}
func versionFromPM(r io.Reader) string {
scanner := bufio.NewScanner(r)
for scanner.Scan() {
line := scanner.Text()
for _, pattern := range []*regexp.Regexp{versionPattern, packageVersionPattern} {
if match := pattern.FindStringSubmatch(line); match != nil {
return strings.TrimRight(match[1], ".")
}
}
}
return ""
}

View File

@ -0,0 +1,105 @@
package perl
import (
"context"
"io"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
func Test_distNameFromPacklistPath(t *testing.T) {
tests := []struct {
path string
want string
}{
{path: "/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/Text/CSV/.packlist", want: "Text-CSV"},
{path: "/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/IO/Socket/SSL/.packlist", want: "IO-Socket-SSL"},
{path: "auto/URI/.packlist", want: "URI"},
// the perl core install has a packlist with no auto/ segment; it must produce nothing
{path: "/usr/local/lib/perl5/5.40.4/x86_64-linux/.packlist", want: ""},
{path: "/some/auto/.packlist", want: ""},
}
for _, tt := range tests {
t.Run(tt.path, func(t *testing.T) {
assert.Equal(t, tt.want, distNameFromPacklistPath(tt.path))
})
}
}
func Test_mainModuleFile(t *testing.T) {
files := []string{
"/usr/local/lib/perl5/site_perl/5.40.4/Text/CSV_PP.pm",
"/usr/local/lib/perl5/site_perl/5.40.4/Text/CSV.pm",
}
assert.Equal(t, "/usr/local/lib/perl5/site_perl/5.40.4/Text/CSV.pm", mainModuleFile("Text-CSV", files))
assert.Equal(t, "", mainModuleFile("Text-CSV-XS", files))
}
func Test_versionFromPM(t *testing.T) {
tests := []struct {
name string
contents string
want string
}{
{name: "our, single quoted", contents: "package Text::CSV;\n\nour $VERSION = '2.06';\n", want: "2.06"},
// what Text::CSV and JSON::PP really do: use vars, then assign inside a BEGIN block
{name: "assigned inside BEGIN", contents: "use vars qw( $VERSION );\n\nBEGIN {\n $VERSION = '2.06';\n}\n", want: "2.06"},
{name: "bare assignment", contents: "$VERSION = \"1.303\";\n", want: "1.303"},
{name: "unquoted decimal", contents: "our $VERSION = 5.35;\n", want: "5.35"},
{name: "v-string is preserved", contents: "our $VERSION = 'v1.1.4';\n", want: "v1.1.4"},
// the fully qualified form older Dist::Zilla wrote, as real JSON::PP 2.27300 carries
{name: "package qualified", contents: "package JSON::PP;\n\n$JSON::PP::VERSION = '2.27300';\n", want: "2.27300"},
{name: "qv wrapper", contents: "our $VERSION = qv('1.2.3');\n", want: "1.2.3"},
// perl 5.12 and later allow the version in the package statement, and a distribution using it
// need carry no $VERSION at all
{name: "package statement v-string", contents: "package CPAN::02Packages::Search v1.0.0;\nuse v5.24;\n", want: "v1.0.0"},
{name: "package statement decimal", contents: "package Foo::Bar 1.23;\n", want: "1.23"},
{name: "package statement block form", contents: "package Foo::Bar v2.0.1 {\n}\n", want: "v2.0.1"},
// a $VERSION assignment later in the file still wins over a version-less package statement
{name: "package statement without a version", contents: "package Foo::Bar;\nour $VERSION = '1.00';\n", want: "1.00"},
// $XS_VERSION and friends must not be mistaken for $VERSION
{name: "other version variables ignored", contents: "our $XS_VERSION = '9.99';\nour $VERSION = '1.00';\n", want: "1.00"},
// real Encode.pm: declared bare, then computed from an RCS keyword inside BEGIN. Nothing static can
// read that, and guessing at it would be worse than the empty version perllocal.pod then fills in.
{name: "computed from an RCS Revision keyword", contents: "our $VERSION;\nBEGIN {\n $VERSION = sprintf \"%d.%02d\", q$Revision: 3.24 $ =~ /(\\d+)/g;\n}\n", want: ""},
{name: "no version at all", contents: "package Foo;\n1;\n", want: ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, versionFromPM(strings.NewReader(tt.contents)))
})
}
}
// Test_parsePacklist_metadataSuffix pins the cut against what ExtUtils::Packlist::read does, which is to
// strip a suffix matching /^(.*?)( \w+=.*)$/ rather than to cut at the first space. A perl tree under a
// directory whose name contains a space is ordinary on macOS and Windows.
func Test_parsePacklist_metadataSuffix(t *testing.T) {
const packlist = `/usr/local/lib/perl5/site_perl/5.40.4/Text/CSV.pm
/Users/me/My Perl/lib/Text/CSV_PP.pm
/usr/local/lib/perl5/site_perl/5.40.4/auto/Text/CSV/CSV.so type=file
/usr/local/bin/csvdiff type=link from=/usr/local/bin/csvdiff-2.06
`
pkgs, _, err := parsePacklist(context.Background(), nil, nil,
file.NewLocationReadCloser(file.NewLocation("usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/Text/CSV/.packlist"),
io.NopCloser(strings.NewReader(packlist))))
require.NoError(t, err)
require.Len(t, pkgs, 1)
assert.Equal(t, []string{
"/usr/local/lib/perl5/site_perl/5.40.4/Text/CSV.pm",
"/Users/me/My Perl/lib/Text/CSV_PP.pm",
"/usr/local/lib/perl5/site_perl/5.40.4/auto/Text/CSV/CSV.so",
"/usr/local/bin/csvdiff",
}, pkgs[0].Metadata.(pkg.CpanDistribution).Files)
}

View File

@ -0,0 +1,100 @@
package perl
import (
"context"
"path"
"strings"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/generic"
)
// parseReleaseMeta catalogs the single distribution described by the META.json or META.yml at the root of
// an unpacked CPAN distribution: a release tarball or a vendored distribution tree.
//
// Both formats are read because META.json is not the common case. Measured on MetaCPAN's file index over
// 41,722 latest releases, root level only: 23,714 ship a META.json (57%), 37,114 a META.yml (89%), and
// 41,008 a MANIFEST (98%). Reading only META.json skips roughly 43% of current releases, skewed toward the
// old ones where the advisories concentrate.
func parseReleaseMeta(ctx context.Context, resolver file.Resolver, _ *generic.Environment, reader file.LocationReadCloser) ([]pkg.Package, []artifact.Relationship, error) {
if isBuildLeftover(reader.Path()) || !isUnpackedRelease(resolver, reader.Location) {
return nil, nil, nil
}
isYAML := strings.EqualFold(path.Base(reader.Path()), "META.yml")
// where both sit in one directory the META.yml invocation yields nothing. META.json is the spec 2
// document and is what release tooling generates last, so where they disagree it is the newer
// rendering. Skipping at this end keeps the cataloger free of dedup processors.
if isYAML && hasSiblingMetaJSON(resolver, reader.Location) {
return nil, nil, nil
}
decoder := decodeMeta
if isYAML {
decoder = decodeMetaYAML
}
m, err := decoder(resolver, reader.Location)
if err != nil {
// a META.yml a strict parser rejects is routine rather than exceptional, so this skips the
// directory and never fails the scan
log.WithFields("path", reader.Path(), "error", err).Debug("unable to parse CPAN meta file")
return nil, nil, nil
}
if m.Name == "" || m.Version == "" {
log.WithFields("path", reader.Path()).Debug("CPAN meta file is missing a name or version")
return nil, nil, nil
}
location := reader.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation)
licenses := pkg.NewLicensesFromLocationWithContext(ctx, reader.Location, m.licenseExpressions()...)
md := pkg.CpanUnpackedRelease{Modules: modulesFromProvides(m.Provides)}
return []pkg.Package{newCpanPackage(m.Name, string(m.Version), "", md, licenses, location)}, nil, nil
}
// isUnpackedRelease decides whether a meta file describes an unpacked distribution or a source
// repository. A committed META.json records the last release and drifts from the working tree
// (libwww-perl carries 6.83 in META.json and 6.84 in lib/LWP.pm on the same commit), so it must not be
// trusted for version. A sibling MANIFEST is what tells the two apart: release tarballs ship one, while a
// source checkout does not, since the MANIFEST is generated at build time.
func isUnpackedRelease(resolver file.Resolver, metaLocation file.Location) bool {
_, ok := findFile(resolver, path.Join(path.Dir(metaLocation.Path()), "MANIFEST"))
return ok
}
// buildLeftoverDirs are the trees CPAN clients unpack into. A cpanm work directory genuinely is an
// unpacked release tarball, MANIFEST included, so the isUnpackedRelease gate is working correctly and
// still admits it; a path exclusion is the only signal available. What makes it a leftover is that a copy
// of the same distribution was installed elsewhere in the same tree, and the two would be reported as
// separate packages, since two catalogers reporting at different locations can never share a package ID.
// Both catalogers run on directory scans, so that duplication is reachable there.
//
// cpanm leaves ~/.cpanm/work/<epoch>/<Dist>-<Version>/ and CPAN.pm leaves
// ~/.cpan/build/<Dist>-<Version>-XXXX/ with a random suffix, neither of which is cleaned up by default,
// so any tree that did not rm -rf them reported every distribution it installed twice.
//
// The honest cost: an unpacked tarball a user deliberately keeps under ~/.cpan/build stops being reported.
var buildLeftoverDirs = []string{".cpanm/work", ".cpan/build"}
func isBuildLeftover(p string) bool {
normalized := normalizePath(p)
for _, dir := range buildLeftoverDirs {
if strings.Contains(normalized, "/"+dir+"/") {
return true
}
}
return false
}
func hasSiblingMetaJSON(resolver file.Resolver, metaLocation file.Location) bool {
_, ok := findFile(resolver, path.Join(path.Dir(metaLocation.Path()), "META.json"))
return ok
}

View File

@ -0,0 +1,32 @@
package perl
import (
"testing"
"github.com/stretchr/testify/assert"
)
func Test_isBuildLeftover(t *testing.T) {
tests := []struct {
path string
want bool
}{
{path: "/root/.cpanm/work/1785183030.2/Clone-0.50/META.json", want: true},
{path: "/root/.cpan/build/Encode-3.24-0/META.json", want: true},
// a directory scan reports paths relative to its root
{path: "root/.cpanm/work/1785183030.2/Clone-0.50/META.yml", want: true},
{path: "/home/app/.cpan/build/Text-CSV-2.06-mQ3Ktb/META.json", want: true},
// a release deliberately unpacked somewhere else is still reported
{path: "/opt/vendor/Text-CSV-2.06/META.json", want: false},
{path: "/src/Encode-3.24/META.json", want: false},
// the client's own directories are not build trees; only work/ and build/ under them are
{path: "/root/.cpanm/latest-build/META.json", want: false},
{path: "/root/.cpan/sources/META.json", want: false},
}
for _, tt := range tests {
t.Run(tt.path, func(t *testing.T) {
assert.Equal(t, tt.want, isBuildLeftover(tt.path))
})
}
}

View File

@ -0,0 +1,224 @@
package perl
import (
"bufio"
"context"
"io"
"regexp"
"strings"
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
// perllocal.pod stanzas look like this, one per install:
//
// =head2 Mon Jul 27 20:10:38 2026: C<Module> L<Net::HTTP|Net::HTTP>
// =item * C<installed into: /usr/local/lib/perl5/site_perl/5.40.4>
// =item * C<VERSION: 6.24>
//
// The date prefix has changed across ExtUtils::MakeMaker versions (localtime became
// gmtime($ENV{SOURCE_DATE_EPOCH} || time) for reproducible builds), so the stanza is recognized by the
// C<Module> marker rather than by anything in the heading. Every EUMM version checked emits =head2, so
// the heading level is not known to vary; keying on the marker just means nothing here depends on it.
var (
perllocalModulePattern = regexp.MustCompile(`C<Module>\s+L<([^|>]+)`)
perllocalInstalledInto = regexp.MustCompile(`C<installed into:\s*([^>]*)>`)
perllocalVersionPattern = regexp.MustCompile(`C<VERSION:\s*([^>]*)>`)
)
// perllocalEntry is one Module stanza of a perllocal.pod, in the order it was written.
type perllocalEntry struct {
Module string // as written, e.g. Net::HTTP
InstalledInto string // the INSTALLSITELIB the installer recorded
Version string
}
// perllocalPod is a parsed perllocal.pod together with where it was found, since the file that supplied
// a version is recorded as supporting evidence on the package.
type perllocalPod struct {
location file.Location
entries []perllocalEntry
}
// resolvePacklistVersions fills in the version of every package that came from a .packlist.
//
// perllocal.pod is the primary source because ExtUtils::MakeMaker writes it and the packlist from the
// same variables in the same install target: the packlist goes to $(SITEARCHEXP)/auto/$(FULLEXT)/.packlist
// while the stanza records "Module" "$(NAME)" with VERSION "$(VERSION)", and FULLEXT is NAME with :: turned
// into /. So the auto/ path segments and the stanza's module name are the same key. $(VERSION) is what EUMM
// evaluated at build time, so a version the module computes rather than declares is recorded correctly,
// which static .pm parsing cannot do at all.
//
// This is a processor rather than a per-packlist read because perllocal.pod is not a sibling of anything:
// EUMM writes it to DESTINSTALLARCHLIB, the core arch directory, even for site installs, so it has to be
// found with a resolver glob. There is one per install tree against many packlists, so it is parsed once
// here rather than once per packlist.
//
// It must run before mergeDistributions. Merging pairs a packlist package to its install.json package only
// when the versions agree or the packlist version is absent, so filling the version in afterwards would
// leave every distribution whose two evidence kinds disagree split into two packages.
func resolvePacklistVersions(_ context.Context, resolver file.Resolver, pkgs []pkg.Package, rels []artifact.Relationship, err error) ([]pkg.Package, []artifact.Relationship, error) {
if resolver == nil || len(pkgs) == 0 {
return pkgs, rels, err
}
var pods []perllocalPod
for _, location := range perllocalLocations(resolver) {
if entries := readPerllocal(resolver, location); len(entries) > 0 {
pods = append(pods, perllocalPod{location: location, entries: entries})
}
}
for i := range pkgs {
md, ok := pkgs[i].Metadata.(pkg.CpanDistribution)
if !ok || md.MainModule == "" || pkgs[i].Version != "" {
// not a packlist-derived package, so there is nothing to resolve
continue
}
version, evidence := packlistVersion(resolver, pods, pkgs[i], md)
if version == "" && evidence == nil {
// still emitted with an empty version rather than dropped, so the distribution stays visible
continue
}
pkgs[i].Version = version
pkgs[i].PURL = packageURL(pkgs[i].Name, version, md.Author)
if evidence != nil {
locations := pkgs[i].Locations
locations.Add(evidence.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.SupportingEvidenceAnnotation))
pkgs[i].Locations = locations
}
// Version and Locations both participate in the package ID
pkgs[i].SetID()
}
return pkgs, rels, err
}
// packlistVersion resolves one packlist package's version, preferring perllocal.pod and falling back to
// scraping $VERSION out of the distribution's main .pm. The fallback is not vestigial: NO_PERLLOCAL
// suppresses the file the same way NO_PACKLIST suppresses the packlist, and Module::Build and
// Module::Build::Tiny write packlists but no perllocal.pod at all.
func packlistVersion(resolver file.Resolver, pods []perllocalPod, p pkg.Package, md pkg.CpanDistribution) (string, *file.Location) {
packlistPath := primaryPath(p)
if version, location := perllocalVersion(pods, p.Name, packlistPath); version != "" {
return version, location
}
pmLocation, ok := findFile(resolver, mainModuleFile(p.Name, md.Files))
if !ok {
return "", nil
}
return readVersionFromPM(resolver, pmLocation), &pmLocation
}
// perllocalVersion picks the stanza that describes the packlist at packlistPath, by three rules:
//
// 1. Key on the dashed module name: auto/Net/HTTP/.packlist and "Module Net::HTTP" both give Net-HTTP.
// 2. The stanza's "installed into" libdir must be a path prefix of the packlist, since in every standard
// perl layout the arch directory nests under the lib directory. Where an image carries more than one
// perl, the core libdir of one tree can also prefix-match another tree's packlist, so the longest
// matching libdir wins. Without that a version from the wrong perl silently wins, which is worse than
// no version at all.
// 3. The file is append-only, so an upgrade leaves a stale stanza beside the current one forever. Stanzas
// are chronological, so the last match wins. The dates are not parsed; file order is the ordering.
func perllocalVersion(pods []perllocalPod, dashedName, packlistPath string) (string, *file.Location) {
var (
version string
evidence *file.Location
bestLibdir int
)
for i := range pods {
for _, entry := range pods[i].entries {
if entry.Version == "" || strings.ReplaceAll(entry.Module, "::", "-") != dashedName {
continue
}
libdir := strings.TrimSuffix(entry.InstalledInto, "/")
if libdir == "" || !strings.HasPrefix(normalizePath(packlistPath), normalizePath(libdir)+"/") {
continue
}
// >= rather than > so that the last stanza of an equally specific libdir wins
if len(libdir) >= bestLibdir {
version, evidence, bestLibdir = entry.Version, &pods[i].location, len(libdir)
}
}
}
return version, evidence
}
// normalizePath makes the resolver's relative paths and perllocal's absolute libdirs comparable. A
// directory scan reports paths relative to its root, while perllocal records what the installer saw.
func normalizePath(p string) string {
return "/" + strings.TrimPrefix(p, "/")
}
func perllocalLocations(resolver file.Resolver) []file.Location {
locations, err := resolver.FilesByGlob("**/perllocal.pod")
if err != nil {
log.WithFields("error", err).Debug("unable to search for perllocal.pod")
return nil
}
return locations
}
func readPerllocal(resolver file.Resolver, location file.Location) []perllocalEntry {
reader, err := resolver.FileContentsByLocation(location)
if err != nil {
log.WithFields("path", location.Path(), "error", err).Debug("unable to read perllocal.pod")
return nil
}
defer internal.CloseAndLogError(reader, location.Path())
return parsePerllocal(reader)
}
func parsePerllocal(r io.Reader) []perllocalEntry {
var entries []perllocalEntry
scanner := bufio.NewScanner(r)
for scanner.Scan() {
line := scanner.Text()
if match := perllocalModulePattern.FindStringSubmatch(line); match != nil {
entries = append(entries, perllocalEntry{Module: strings.TrimSpace(match[1])})
continue
}
if len(entries) == 0 {
continue
}
current := &entries[len(entries)-1]
if match := perllocalInstalledInto.FindStringSubmatch(line); match != nil {
current.InstalledInto = strings.TrimSpace(match[1])
}
if match := perllocalVersionPattern.FindStringSubmatch(line); match != nil {
current.Version = strings.TrimSpace(match[1])
}
}
return entries
}
// primaryPath returns the path of the evidence the package was built from, which for a packlist package
// is the .packlist itself.
func primaryPath(p pkg.Package) string {
for _, location := range p.Locations.ToSlice() {
if location.Annotations[pkg.EvidenceAnnotationKey] == pkg.PrimaryEvidenceAnnotation {
return location.Path()
}
}
return ""
}

View File

@ -0,0 +1,170 @@
package perl
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/file"
)
func Test_parsePerllocal(t *testing.T) {
// two stanzas as ExtUtils::MakeMaker writes them, with the second module upgraded in place so both a
// stale entry and a current one are present. Real EUMM emits =head2; the =head1 here is a deliberate
// variation, to pin that the parser keys on the C<Module> marker and not on the heading level.
const pod = `=head0 L<Perl documentation|perl>
=head1 Sun Jul 26 18:04:12 2026: C<Module> L<Encode|Encode>
=over 4
=item *
C<installed into: /usr/local/lib/perl5/site_perl/5.40.4>
=item *
C<LINKTYPE: dynamic>
=item *
C<VERSION: 3.24>
=item *
C<EXE_FILES: bin/enc2xs>
=back
=head2 Mon Jul 27 20:10:38 2026: C<Module> L<Net::HTTP|Net::HTTP>
=over 4
=item *
C<installed into: /usr/local/lib/perl5/site_perl/5.40.4>
=item *
C<VERSION: 6.24>
=back
`
assert.Equal(t, []perllocalEntry{
{Module: "Encode", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4", Version: "3.24"},
{Module: "Net::HTTP", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4", Version: "6.24"},
}, parsePerllocal(strings.NewReader(pod)))
}
func Test_parsePerllocal_degrades(t *testing.T) {
tests := []struct {
name string
contents string
}{
{name: "empty", contents: ""},
{name: "no module stanzas", contents: "=head0 L<Perl documentation|perl>\n"},
// a stanza body with no C<Module> heading before it has nothing to attach to
{name: "orphaned body", contents: "=item *\n\nC<VERSION: 1.00>\n"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Empty(t, parsePerllocal(strings.NewReader(tt.contents)))
})
}
}
func Test_perllocalVersion(t *testing.T) {
pods := []perllocalPod{
{
location: file.NewLocation("/usr/local/lib/perl5/5.40.4/aarch64-linux-gnu/perllocal.pod"),
entries: []perllocalEntry{
{Module: "Net::HTTP", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4", Version: "6.24"},
// the file is append-only, so an upgrade leaves the earlier stanza in place forever
{Module: "Text::CSV", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4", Version: "2.02"},
{Module: "Text::CSV", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4", Version: "2.06"},
// a stanza whose libdir belongs to another install tree entirely
{Module: "JSON::PP", InstalledInto: "/opt/perl-5.36.0/lib/perl5/site_perl/5.36.0", Version: "2.27300"},
// an INSTALL_BASE install and a local::lib nested inside it both prefix-match one packlist
{Module: "Try::Tiny", InstalledInto: "/opt/app/lib/perl5", Version: "0.30"},
{Module: "Try::Tiny", InstalledInto: "/opt/app/lib/perl5/local", Version: "0.32"},
{Module: "Carp", InstalledInto: "/usr/local/lib/perl5/site_perl/5.40.4"},
},
},
}
tests := []struct {
name string
dashedName string
packlistPath string
want string
}{
{
name: "module name dashes to the packlist name",
dashedName: "Net-HTTP",
packlistPath: "/usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/Net/HTTP/.packlist",
want: "6.24",
},
{
name: "the last stanza wins over a stale one",
dashedName: "Text-CSV",
packlistPath: "/usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/Text/CSV/.packlist",
want: "2.06",
},
{
// without the libdir rule the other perl's version would silently win, which is worse than
// reporting no version at all
name: "a stanza from another install tree is not used",
dashedName: "JSON-PP",
packlistPath: "/usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/JSON/PP/.packlist",
want: "",
},
{
name: "the longest matching libdir wins",
dashedName: "Try-Tiny",
packlistPath: "/opt/app/lib/perl5/local/aarch64-linux-gnu/auto/Try/Tiny/.packlist",
want: "0.32",
},
{
name: "the outer libdir still matches its own packlist",
dashedName: "Try-Tiny",
packlistPath: "/opt/app/lib/perl5/aarch64-linux-gnu/auto/Try/Tiny/.packlist",
want: "0.30",
},
{
// a directory scan reports paths relative to its root while perllocal records absolute libdirs
name: "relative packlist path from a directory scan",
dashedName: "Net-HTTP",
packlistPath: "usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/Net/HTTP/.packlist",
want: "6.24",
},
{
name: "no stanza for this distribution",
dashedName: "IO-Socket-SSL",
packlistPath: "/usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/IO/Socket/SSL/.packlist",
want: "",
},
{
name: "a stanza with no VERSION is not a match",
dashedName: "Carp",
packlistPath: "/usr/local/lib/perl5/site_perl/5.40.4/aarch64-linux-gnu/auto/Carp/.packlist",
want: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
version, evidence := perllocalVersion(pods, tt.dashedName, tt.packlistPath)
assert.Equal(t, tt.want, version)
if tt.want == "" {
assert.Nil(t, evidence)
return
}
require.NotNil(t, evidence)
assert.Equal(t, "/usr/local/lib/perl5/5.40.4/aarch64-linux-gnu/perllocal.pod", evidence.Path())
})
}
}

View File

@ -0,0 +1,225 @@
package perl
import (
"context"
"path"
"strings"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/dependency"
)
// mergeDistributions collapses the two kinds of installed evidence into one package per installed
// distribution. A distribution installed by cpanm has both an install.json and a .packlist.
//
// Pairing is scoped to one install tree, so two installs of the same distribution and version into
// separate trees stay two packages. A scan target can hold more than one project tree, and preserving
// that there are two trees is the point: collapsing them asserts one installed thing where there are
// two and loses which tree each belongs to. Multiple projects in one scan target are treated the same
// way elsewhere.
//
// (name, version) alone is not enough to pair them either. A packlist lives at auto/<Module>/.packlist, so its
// name is derived from a MODULE path, while install.json yields the distribution: libwww-perl's packlist
// is auto/LWP/.packlist and produces "LWP" against the distribution "libwww-perl". Those never collide,
// so pairing also consults the `provides` module map, which lists LWP under libwww-perl. Without that
// step every distribution whose main module is named differently from the distribution produces two
// packages, one with the right name and no version and one with the wrong name and the right version,
// and neither one matches an advisory.
func mergeDistributions(pkgs []pkg.Package, rels []artifact.Relationship, err error) ([]pkg.Package, []artifact.Relationship, error) {
if len(pkgs) < 2 {
return pkgs, rels, err
}
var merged []pkg.Package
indexByKey := make(map[string]int)
// install.json evidence is placed first so that the module index is complete before any packlist
// package needs to resolve against it, independent of the order the parsers ran in
ordered := make([]pkg.Package, 0, len(pkgs))
for _, p := range pkgs {
if fromInstallJSON(p) {
ordered = append(ordered, p)
}
}
for _, p := range pkgs {
if !fromInstallJSON(p) {
ordered = append(ordered, p)
}
}
indexByModule := make(map[string]int)
for _, p := range ordered {
tree := installTree(p)
key := tree + "|" + p.Name + "@" + p.Version
if idx, exists := indexByKey[key]; exists {
merged[idx] = mergeInto(merged[idx], p)
continue
}
// a packlist package names a module, so fold it into whichever distribution in the same tree
// provides it. The version must still agree, or be unreadable: a packlist and an install.json
// in one tree that disagree describe one install that overwrote another's files, and merging
// them would report a single version and hide the other.
if !fromInstallJSON(p) {
if idx, exists := indexByModule[tree+"|"+moduleFromDashedName(p.Name)]; exists &&
(p.Version == "" || p.Version == merged[idx].Version) {
merged[idx] = mergeInto(merged[idx], p)
continue
}
}
indexByKey[key] = len(merged)
for _, m := range providedModules(p) {
// first distribution to claim a module wins; duplicate claims are not expected and
// silently reassigning would make the result order dependent
if _, taken := indexByModule[tree+"|"+m]; !taken {
indexByModule[tree+"|"+m] = len(merged)
}
}
merged = append(merged, p)
}
// locations and metadata drive the package ID, and both may have changed
for i := range merged {
merged[i].SetID()
}
return merged, rels, err
}
// installTree returns the library tree the package's own evidence was found in: the directory holding
// the .meta or auto directory the file sits under. Both kinds of evidence for one install land there,
// in sibling directories rather than the same one, so
// /opt/app/lib/perl5/x86_64-linux-gnu/.meta/Text-CSV-2.06/install.json and
// /opt/app/lib/perl5/x86_64-linux-gnu/auto/Text/CSV/.packlist both yield
// /opt/app/lib/perl5/x86_64-linux-gnu and still pair.
//
// The owned-file list cannot stand in for this: ExtUtils::Packlist merges an existing packlist for the
// same module when it writes a new one, so a local-lib install's packlist legitimately claims files
// living outside its own prefix.
func installTree(p pkg.Package) string {
for _, location := range p.Locations.ToSlice() {
if tree := libRoot(location.Path()); tree != "" {
return tree
}
}
return ""
}
// libRoot cuts a path at the last auto/ or .meta/ segment, which is where a lib tree ends and the
// installer's own bookkeeping begins. A path with neither, such as a perllocal.pod or a scraped .pm,
// yields "": those are supporting evidence and never the only location a package has.
func libRoot(p string) string {
parts := strings.Split(p, "/")
for i := len(parts) - 1; i > 0; i-- {
if parts[i] == "auto" || parts[i] == ".meta" {
return strings.Join(parts[:i], "/")
}
}
return ""
}
// moduleFromDashedName converts a packlist-derived name back into the module it came from, since
// auto/IO/Socket/SSL/.packlist flattens IO::Socket::SSL into IO-Socket-SSL.
func moduleFromDashedName(name string) string {
return strings.ReplaceAll(name, "-", "::")
}
// mergeInto keeps the install.json-derived package, which is the higher fidelity of the two, and folds
// the other's evidence into it.
func mergeInto(a, b pkg.Package) pkg.Package {
winner, loser := a, b
if !fromInstallJSON(a) && fromInstallJSON(b) {
winner, loser = b, a
}
locations := file.NewLocationSet(winner.Locations.ToSlice()...)
locations.Add(loser.Locations.ToSlice()...)
winner.Locations = locations
licenses := pkg.NewLicenseSet(winner.Licenses.ToSlice()...)
licenses.Add(loser.Licenses.ToSlice()...)
winner.Licenses = licenses
// the file list only ever comes from a packlist, so the install.json winner has to take the loser's or
// the merged package would own nothing. OwnedFiles dedupes, which covers a distribution that merged
// more than one packlist.
from, fromOK := loser.Metadata.(pkg.CpanDistribution)
into, intoOK := winner.Metadata.(pkg.CpanDistribution)
if fromOK && intoOK && len(from.Files) > 0 {
into.Files = append(into.Files, from.Files...)
winner.Metadata = into
}
return winner
}
// fromInstallJSON reports whether a package was built from an install.json rather than from a
// .packlist. MainModule is only ever set by the packlist parser, so its absence is the direct signal.
func fromInstallJSON(p pkg.Package) bool {
md, ok := p.Metadata.(pkg.CpanDistribution)
return ok && md.MainModule == ""
}
// resolveDependencies turns MYMETA.json prereqs into relationships. Prereqs name modules while packages
// are keyed by distribution, so each prereq is looked up in the provides map of the cataloged set. A
// prereq that resolves to nothing (a core module such as Carp, strict, or perl, or a distribution that
// simply is not installed) is dropped rather than turned into a placeholder package.
func resolveDependencies(_ context.Context, resolver file.Resolver, pkgs []pkg.Package, rels []artifact.Relationship, err error) ([]pkg.Package, []artifact.Relationship, error) {
if resolver == nil || len(pkgs) == 0 {
return pkgs, rels, err
}
requires := make(map[artifact.ID][]string)
for i := range pkgs {
if pkgs[i].ID() == "" {
pkgs[i].SetID()
}
requires[pkgs[i].ID()] = prereqModules(resolver, pkgs[i])
}
specifier := func(p pkg.Package) dependency.Specification {
return dependency.Specification{
ProvidesRequires: dependency.ProvidesRequires{
Provides: providedModules(p),
Requires: requires[p.ID()],
},
}
}
return pkgs, append(rels, dependency.Resolve(specifier, pkgs)...), err
}
func providedModules(p pkg.Package) []string {
md, ok := p.Metadata.(pkg.CpanDistribution)
if !ok {
return nil
}
modules := make([]string, 0, len(md.Modules))
for _, m := range md.Modules {
modules = append(modules, m.Name)
}
return modules
}
// prereqModules re-reads the MYMETA.json already recorded as evidence on the package. It is read a
// second time (the parser reads it for licenses) rather than carried on the package, since prereqs
// belong in relationships and not in the metadata struct.
func prereqModules(resolver file.Resolver, p pkg.Package) []string {
for _, location := range p.Locations.ToSlice() {
if path.Base(location.Path()) != "MYMETA.json" {
continue
}
m, err := decodeMeta(resolver, location)
if err != nil {
continue
}
return m.prereqModules()
}
return nil
}

View File

@ -0,0 +1,100 @@
package perl
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
)
func Test_mergeDistributions(t *testing.T) {
fromInstallJSON := newCpanPackage("URI", "5.35", "OALDERS",
pkg.CpanDistribution{
Dist: "URI-5.35",
Author: "OALDERS",
Path: "O/OA/OALDERS/URI-5.35.tar.gz",
Modules: []pkg.CpanModule{{Name: "URI", Version: "5.35"}},
},
[]pkg.License{pkg.NewLicenseWithContext(context.Background(), "perl_5")},
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/.meta/URI-5.35/install.json"),
)
fromPacklist := newCpanPackage("URI", "5.35", "", pkg.CpanDistribution{MainModule: "URI"}, nil,
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/URI/.packlist"),
)
otherVersion := newCpanPackage("URI", "5.34", "", pkg.CpanDistribution{MainModule: "URI"}, nil,
file.NewLocation("/opt/lib/perl5/x86_64-linux/auto/URI/.packlist"),
)
// the packlist arriving first must not change which package wins
for _, order := range [][]pkg.Package{
{fromInstallJSON, fromPacklist, otherVersion},
{fromPacklist, fromInstallJSON, otherVersion},
} {
merged, _, err := mergeDistributions(order, nil, nil)
require.NoError(t, err)
require.Len(t, merged, 2)
uri := merged[0]
assert.Equal(t, "5.35", uri.Version)
assert.Equal(t, "pkg:cpan/URI@5.35?author=OALDERS", uri.PURL)
assert.Equal(t, "OALDERS", uri.Metadata.(pkg.CpanDistribution).Author)
assert.Len(t, uri.Licenses.ToSlice(), 1)
assert.Len(t, uri.Locations.ToSlice(), 2, "the packlist should be kept as additional evidence")
// a different version of the same distribution is a different package
assert.Equal(t, "5.34", merged[1].Version)
}
}
// a packlist lives at auto/<Module>/.packlist, so for any distribution whose main module is named
// differently from the distribution the two kinds of evidence never share a (name, version) key.
// libwww-perl is the canonical case: auto/LWP/.packlist against the distribution libwww-perl.
func Test_mergeDistributions_packlistNamedAfterModule(t *testing.T) {
fromInstallJSON := newCpanPackage("libwww-perl", "5.836", "GAAS",
pkg.CpanDistribution{
Dist: "libwww-perl-5.836",
Author: "GAAS",
Path: "G/GA/GAAS/libwww-perl-5.836.tar.gz",
Modules: []pkg.CpanModule{{Name: "LWP", Version: "5.836"}, {Name: "LWP::UserAgent", Version: "5.834"}},
},
nil,
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/.meta/libwww-perl-5.836/install.json"),
)
fromPacklist := newCpanPackage("LWP", "5.836", "", pkg.CpanDistribution{MainModule: "LWP"}, nil,
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/LWP/.packlist"),
)
// a packlist whose version could not be read still belongs to the distribution providing it
versionless := newCpanPackage("LWP-UserAgent", "", "", pkg.CpanDistribution{MainModule: "LWP::UserAgent"}, nil,
file.NewLocation("/usr/local/lib/perl5/site_perl/5.40.4/x86_64-linux/auto/LWP/UserAgent/.packlist"),
)
// the same module installed into a second lib tree is a second install, and the module index is
// scoped to a tree so that it cannot be claimed by the distribution in the first one
otherTree := newCpanPackage("LWP", "5.836", "", pkg.CpanDistribution{MainModule: "LWP"}, nil,
file.NewLocation("/opt/lib/perl5/x86_64-linux/auto/LWP/.packlist"),
)
for _, order := range [][]pkg.Package{
{fromInstallJSON, fromPacklist, versionless, otherTree},
{otherTree, versionless, fromPacklist, fromInstallJSON},
} {
merged, _, err := mergeDistributions(order, nil, nil)
require.NoError(t, err)
require.Len(t, merged, 2, "the same tree's module-named packlists belong to libwww-perl, the other tree's does not")
assert.Equal(t, "libwww-perl", merged[0].Name)
assert.Equal(t, "pkg:cpan/libwww-perl@5.836?author=GAAS", merged[0].PURL)
assert.Len(t, merged[0].Locations.ToSlice(), 3)
assert.Equal(t, "LWP", merged[1].Name)
assert.Equal(t, []string{"/opt/lib/perl5/x86_64-linux/auto/LWP/.packlist"}, realPaths(merged[1]))
}
}

View File

@ -0,0 +1,44 @@
# an image that installed distributions and did not clean up after itself, which is the ordinary
# state of anything built with a CPAN client: cpanm leaves ~/.cpanm/work and CPAN.pm leaves
# ~/.cpan/build, neither by choice of the user. Both leftovers genuinely are unpacked release
# tarballs, MANIFEST included, so nothing about their contents disqualifies them and they have to be
# excluded on the path.
#
# A release deliberately unpacked outside any build directory sits beside the installed copy of the
# same distribution and version, so the same image shows both halves of the path rule: the leftover
# is skipped and the vendored tree is not.
#
# The digest is perl:5.40-slim's linux/amd64 manifest; see image-cpan-mirror-installs/Dockerfile.
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
# cpanm, mirror-resolved so these carry install.json, and pinned by PAUSE path so the versions do
# not drift. The unpacked build trees are left in /root/.cpanm/work.
RUN cpanm --notest \
R/RE/REHSACK/MIME-Base32-1.303.tar.gz \
O/OA/OALDERS/URI-5.35.tar.gz
# CPAN.pm, which writes no .meta and leaves its build tree in /root/.cpan/build under a
# randomly suffixed directory
RUN PERL_MM_USE_DEFAULT=1 cpan -T -i P/PE/PETDANCE/HTML-Tagset-3.24.tar.gz
# the same release, unpacked somewhere a user would keep it on purpose. Nothing pairs this with the
# installed copy: the two come from different catalogers, and two packages found at different
# locations can never share a package ID.
RUN mkdir -p /opt/vendor \
&& curl -sSfL https://backpan.perl.org/authors/id/P/PE/PETDANCE/HTML-Tagset-3.24.tar.gz \
| tar xz -C /opt/vendor
# the clients' download caches under ~/.cpanm/sources and ~/.cpan/sources are not evidence of
# anything and are the bulk of the bytes, so only the unpacked build trees come along
RUN mkdir -p /out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu /out/root/.cpanm /out/root/.cpan /out/opt \
&& cp -a /usr/local/lib/perl5/site_perl /out/usr/local/lib/perl5/site_perl \
&& cp -a /usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod \
/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/.packlist \
/out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/ \
&& cp -a /root/.cpanm/work /out/root/.cpanm/work \
&& cp -a /root/.cpan/build /out/root/.cpan/build \
&& cp -a /opt/vendor /out/opt/vendor
FROM scratch
COPY --from=builder /out/ /

View File

@ -0,0 +1,84 @@
# mirror-resolved CPAN client installs: cpanm, cpm and carton. This is the only case where an
# install.json is written, so it is also the only image carrying MYMETA.json and a `provides` map.
#
# The digest is perl:5.40-slim's linux/amd64 manifest rather than its index, which pins the
# architecture as well as the contents: perl's lib layout embeds the arch name
# (.../5.40.4/x86_64-linux-gnu/), so a build on an arm host would move every path under test.
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
# Clone and HTML::Parser are XS and the slim image ships no compiler
RUN apt-get update \
&& apt-get install -y --no-install-recommends gcc libc6-dev \
&& rm -rf /var/lib/apt/lists/*
# cpm and carton go into a throwaway prefix: their own dependency trees are large and are not under
# test, and nothing from /tools reaches the final image
RUN cpanm --notest --quiet -L /tools App::cpm Carton
# cpanm, site install of libwww-perl and its whole dependency closure.
#
# LWP is the module name; the distribution is libwww-perl, so this is the name mismatch that makes
# install.json's `name` field unusable as a package name, and it installs to auto/LWP/.packlist
# rather than auto/libwww-perl/. The closure comes along on purpose: it is what gives the image real
# MYMETA.json prereqs to resolve into relationships.
#
# Every distribution is named by its PAUSE path, which is what pins the version. A bare module name
# resolves to whatever the index says today, so the closure would drift the moment any of these
# authors releases again. cpanm still treats a PAUSE path as a mirror resolution, so install.json is
# written for all of them, which a URL or a local tarball would not do. Dependencies come first so
# that nothing in the list is ever satisfied by an unpinned fetch.
RUN cpanm --notest \
E/ET/ETHER/Try-Tiny-0.32.tar.gz \
A/AT/ATOOMIC/Clone-0.50.tar.gz \
G/GA/GAAS/Encode-Locale-1.05.tar.gz \
P/PE/PETDANCE/HTML-Tagset-3.24.tar.gz \
C/CJ/CJM/IO-HTML-1.004.tar.gz \
O/OA/OALDERS/LWP-MediaTypes-6.04.tar.gz \
R/RE/REHSACK/MIME-Base32-1.303.tar.gz \
A/AT/ATOOMIC/TimeDate-2.35.tar.gz \
O/OA/OALDERS/URI-5.35.tar.gz \
O/OA/OALDERS/HTTP-Date-6.08.tar.gz \
O/OA/OALDERS/HTTP-Message-7.04.tar.gz \
O/OA/OALDERS/HTML-Parser-3.85.tar.gz \
P/PL/PLICEASE/File-Listing-6.16.tar.gz \
O/OA/OALDERS/HTTP-Cookies-6.12.tar.gz \
G/GA/GAAS/HTTP-Negotiate-6.01.tar.gz \
O/OA/OALDERS/Net-HTTP-6.24.tar.gz \
O/OA/OALDERS/WWW-RobotRules-6.03.tar.gz \
O/OA/OALDERS/libwww-perl-6.83.tar.gz
# a distribution installed the way distro packagers do it: install.json and no packlist at all.
# NO_PACKLIST and NO_PERLLOCAL are ExtUtils::MakeMaker knobs, so this has to be an EUMM
# distribution; a Module::Build::Tiny one would ignore both and write its packlist anyway.
RUN PERL_MM_OPT="NO_PACKLIST=1 NO_PERLLOCAL=1" cpanm --notest I/IS/ISHIGAKI/JSON-PP-4.16.tar.gz
# carton, writing local/lib/perl5 beside the cpanfile it read. The cpanfile and cpanfile.snapshot
# are deliberately left in place and deliberately not parsed by the cataloger.
RUN mkdir -p /app \
&& printf "requires 'Text::CSV', '== 2.06';\n" > /app/cpanfile \
&& cd /app \
&& PERL5LIB=/tools/lib/perl5 PATH=/tools/bin:$PATH carton install
# cpm, into an arbitrary prefix. cpm serializes install.json differently from cpanm (canonical JSON,
# three-space indent, spaces around the colons, trailing newline) and that difference is under test.
RUN PERL5LIB=/tools/lib/perl5 /tools/bin/cpm install -L /srv/api/local --no-test 'Capture::Tiny@0.48'
# a path a packlist claims and the filesystem lacks is what an overwritten or removed file looks
# like, and the owned-file list is expected to report it anyway
RUN rm /usr/local/lib/perl5/site_perl/5.40.4/IO/Socket/SSL/Utils.pm
# assemble only what the catalogers read: the site_perl tree, the core arch directory's
# perllocal.pod and packlist, and the two local-lib trees. The rest of the core lib tree is tens of
# megabytes of interpreter that carries no CPAN evidence.
RUN mkdir -p /out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu \
&& cp -a /usr/local/lib/perl5/site_perl /out/usr/local/lib/perl5/site_perl \
&& cp -a /usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod \
/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/.packlist \
/out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/ \
&& mkdir -p /out/app /out/srv/api \
&& cp -a /app/cpanfile /app/cpanfile.snapshot /app/local /out/app/ \
&& cp -a /srv/api/local /out/srv/api/local
FROM scratch
COPY --from=builder /out/ /

View File

@ -0,0 +1,69 @@
# installs that leave a .packlist and no install.json, which is the majority of real Perl images and
# where the interesting failures live: the distribution name has to come out of the auto/ path and
# the version out of perllocal.pod or the installed .pm.
#
# Four real installers are exercised, none of which writes a .meta directory:
#
# - ExtUtils::MakeMaker run directly (perl Makefile.PL && make install), which is what distro
# packagers and vendored builds do. It resolves no dependencies at all.
# - cpanm against a local tarball. cpanm's save_meta returns early unless the distribution was
# resolved from a mirror, so a local source writes no install.json.
# - CPAN.pm, which never writes a .meta directory in any circumstance.
# - an EUMM install with NO_PERLLOCAL, which is the case that forces the version to be scraped out
# of the installed .pm.
#
# Old versions are pinned and pinned versions come from backpan: the mirrors drop them and answer
# with an HTML 404 body, which only fails later as "gzip: stdin: not in gzip format".
#
# The digest is perl:5.40-slim's linux/amd64 manifest; see image-cpan-mirror-installs/Dockerfile.
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
# Encode is XS and the slim image ships no compiler
RUN apt-get update \
&& apt-get install -y --no-install-recommends gcc libc6-dev \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /tarballs
RUN curl -sSfLO https://backpan.perl.org/authors/id/G/GA/GAAS/libwww-perl-5.836.tar.gz \
&& curl -sSfLO https://backpan.perl.org/authors/id/I/IS/ISHIGAKI/Text-CSV-2.02.tar.gz \
&& curl -sSfLO https://backpan.perl.org/authors/id/I/IS/ISHIGAKI/Text-CSV-2.06.tar.gz \
&& curl -sSfLO https://backpan.perl.org/authors/id/I/IS/ISHIGAKI/JSON-PP-4.16.tar.gz
# ExtUtils::MakeMaker, run by hand. libwww-perl's EUMM NAME is LWP, so the packlist lands at
# auto/LWP/.packlist and the distribution is only ever recoverable as "LWP" from an installed tree.
# EUMM installs no prerequisites and only warns about the missing ones, which is what keeps this
# image free of the mirror-resolved records those would have written.
RUN tar xzf libwww-perl-5.836.tar.gz \
&& cd libwww-perl-5.836 \
&& perl Makefile.PL \
&& make \
&& make install
# cpanm from a local tarball, twice, upgrading in place. perllocal.pod is append-only, so 2.02's
# stanza stays beside 2.06's forever and the later one has to win.
RUN cpanm --notest ./Text-CSV-2.02.tar.gz \
&& cpanm --notest ./Text-CSV-2.06.tar.gz
# a packlist with no perllocal.pod stanza at all, which is what NO_PERLLOCAL leaves and what
# Module::Build::Tiny produces by default. The version can then only come from the installed .pm.
RUN PERL_MM_OPT="NO_PERLLOCAL=1" cpanm --notest ./JSON-PP-4.16.tar.gz
# CPAN.pm. Encode's $VERSION is declared bare and computed with sprintf from an RCS Revision keyword
# inside a BEGIN block, so no static read of Encode.pm can recover it; only perllocal.pod, which
# records what EUMM evaluated at build time, has the value.
RUN PERL_MM_USE_DEFAULT=1 cpan -T -i D/DA/DANKOGAI/Encode-3.24.tar.gz
# guard the premise of the whole image rather than trusting it
RUN if find /usr/local/lib/perl5 -name install.json | grep -q .; then \
echo "an install.json was written; this image is supposed to have none"; exit 1; \
fi
RUN mkdir -p /out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu \
&& cp -a /usr/local/lib/perl5/site_perl /out/usr/local/lib/perl5/site_perl \
&& cp -a /usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod \
/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/.packlist \
/out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/
FROM scratch
COPY --from=builder /out/ /

View File

@ -0,0 +1,46 @@
# two perl lib trees on one image, each with its own perllocal.pod naming the same module at a
# different version. Picking the stanza from the wrong tree would be silent, and both installed .pm
# files are the real ones, so scraping cannot cover for it: each tree would be reported at the
# other's version.
#
# Both JSON-PP installs come from local tarballs, so neither writes an install.json and the version
# can only come from perllocal.pod. A second full perl is not needed; a local-lib prefix reproduces
# the same ambiguity, because cpanm -L writes a perllocal.pod into the prefix's own arch directory.
#
# The image also carries one distribution installed into both trees at the same version, from a
# mirror, so that both trees hold an install.json agreeing on name and version. Those two records
# describe two installed things and are only distinguishable by the tree they landed in.
#
# The digest is perl:5.40-slim's linux/amd64 manifest; see image-cpan-mirror-installs/Dockerfile.
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
WORKDIR /tarballs
RUN curl -sSfLO https://backpan.perl.org/authors/id/M/MA/MAKAMAKA/JSON-PP-2.27300.tar.gz \
&& curl -sSfLO https://backpan.perl.org/authors/id/I/IS/ISHIGAKI/JSON-PP-4.16.tar.gz
# the site install. 4.16 declares `our $VERSION = '4.16';`, which is scrapable, so this tree is the
# one where perllocal.pod and the .pm agree.
RUN cpanm --notest ./JSON-PP-4.16.tar.gz
# the second tree. 2.27300 declares the fully qualified `$JSON::PP::VERSION = '2.27300';` form that
# older Dist::Zilla wrote.
RUN cpanm --notest -L /opt/app ./JSON-PP-2.27300.tar.gz
# the same distribution and version into both trees. Resolving from a mirror is what makes cpanm
# write an install.json, and the PAUSE path pins the version the way a bare module name would not.
# Text-CSV is pure perl and pulls in nothing that is not core, so this needs no compiler and adds no
# unpinned fetch.
RUN cpanm --notest I/IS/ISHIGAKI/Text-CSV-2.06.tar.gz \
&& cpanm --notest -L /opt/app I/IS/ISHIGAKI/Text-CSV-2.06.tar.gz
RUN mkdir -p /out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu \
&& cp -a /usr/local/lib/perl5/site_perl /out/usr/local/lib/perl5/site_perl \
&& cp -a /usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/perllocal.pod \
/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/.packlist \
/out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu/ \
&& mkdir -p /out/opt \
&& cp -a /opt/app /out/opt/app
FROM scratch
COPY --from=builder /out/ /

View File

@ -0,0 +1,43 @@
# release tarballs unpacked but never installed, plus a source repository working tree. Nothing here
# has a packlist or an install.json, so the whole image is meta-file evidence.
#
# Versions are pinned, and pinned versions have to come from backpan: the mirrors drop old releases
# and answer with an HTML 404 that only fails later, at the point of untarring.
#
# The digest is perl:5.40-slim's linux/amd64 manifest rather than its index, so every image in this
# directory is built from the same architecture; see image-cpan-mirror-installs/Dockerfile.
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
WORKDIR /opt/releases
# a Dist::Zilla release: it ships dist.ini inside the tarball, and MANIFEST lists it, so a dist.ini
# beside a META.json says nothing about whether this is a release or a checkout. It also ships both
# META.json and META.yml, which is the ordinary case for dzil and where META.json has to win.
RUN curl -sSfL https://backpan.perl.org/authors/id/E/ET/ETHER/Try-Tiny-0.32.tar.gz | tar xz
# CPAN Meta Spec 1.4, which is all that roughly 43% of current releases ship. Genuine
# ExtUtils::MakeMaker 6.56 output: license is a bare string rather than an array, version is an
# unquoted number, and there is no provides block because EUMM does not generate one.
RUN curl -sSfL https://backpan.perl.org/authors/id/M/MA/MAKAMAKA/Text-CSV-1.21.tar.gz | tar xz
# a META.yml no strict YAML parser accepts: EUMM wrote this distribution's multi-line ABSTRACT
# straight into `abstract:` with no quoting and no block scalar, so the continuation lines land at
# column zero. This has to skip the directory rather than fail the scan.
RUN curl -sSfL https://backpan.perl.org/authors/id/L/LE/LEMBARK/Parallel-Depend-4.10.tar.gz | tar xz
# a source repository working tree, pinned to one commit. Its committed META.json records the last
# release at 6.83 while lib/LWP.pm on the same commit says 6.84, which is why a committed META.json
# cannot be trusted for version. There is no MANIFEST, because Dist::Zilla generates it at build
# time, and that absence is the only thing separating this from a release.
#
# This is GitHub's tarball of the commit rather than a clone: it is the same working tree, it needs
# no git in the image, and it leaves no .git directory to prune.
RUN mkdir -p /src \
&& curl -sSfL https://codeload.github.com/libwww-perl/libwww-perl/tar.gz/65c6384c3d9e8a4f88a7fbe84307895dc78ccceb \
| tar xz -C /src \
&& mv /src/libwww-perl-* /src/libwww-perl
FROM scratch
COPY --from=builder /opt/releases /opt/releases
COPY --from=builder /src/libwww-perl /src/libwww-perl

View File

@ -0,0 +1,24 @@
# a perl with no CPAN installs at all, which pins the zero coverage of distributions bundled inside
# the interpreter. site_perl is dropped entirely, so what is left is the core lib tree: Encode and
# Storable sit in it, the only packlist is perl's own and has no auto/ segment, and there is no
# .meta and no distro package. Nothing here carries a core distribution's version.
#
# perllocal.pod comes along even though it names the distributions the base image preinstalled,
# because a perllocal.pod stanza on its own must not invent a package. Storable.pm does declare a
# readable $VERSION, so the interesting half is that no package is made out of it either.
#
# The digest is perl:5.40-slim's linux/amd64 manifest; see image-cpan-mirror-installs/Dockerfile.
FROM perl:5.40-slim@sha256:af886c2fe0170cdf0bd83fa8254c7a55da227d5f20988a2e96c836646f96aa7a AS builder
# the whole core lib tree is tens of megabytes of interpreter, so this takes the two dual-life
# distributions under discussion, their shared libraries, and the two files the catalogers look for
RUN O=/out/usr/local/lib/perl5/5.40.4/x86_64-linux-gnu \
&& mkdir -p $O/auto/Encode $O/auto/Storable \
&& cd /usr/local/lib/perl5/5.40.4/x86_64-linux-gnu \
&& cp -a Encode.pm Storable.pm perllocal.pod .packlist $O/ \
&& cp -a auto/Encode/Encode.so $O/auto/Encode/ \
&& cp -a auto/Storable/Storable.so $O/auto/Storable/
FROM scratch
COPY --from=builder /out/ /

View File

@ -23,6 +23,7 @@ const (
JavaScript Language = "javascript"
Lua Language = "lua"
OCaml Language = "ocaml"
Perl Language = "perl"
PHP Language = "php"
Python Language = "python"
R Language = "R"
@ -45,6 +46,7 @@ var AllLanguages = []Language{
JavaScript,
Lua,
OCaml,
Perl,
PHP,
Python,
R,
@ -80,6 +82,9 @@ func LanguageByName(name string) Language {
return JavaScript
case packageurl.TypeLuaRocks, string(Lua):
return Lua
// note: packageurl.TypeCpan and CpanPkg are the same string, so only one can be listed here
case packageurl.TypeCpan, string(Perl):
return Perl
case packageurl.TypePyPi, string(Python):
return Python
case packageurl.TypeGem, string(Ruby):

View File

@ -90,6 +90,10 @@ func TestLanguageFromPURL(t *testing.T) {
purl: "pkg:opam/ocaml-base-compiler@5.2.0",
want: OCaml,
},
{
purl: "pkg:cpan/URI@5.35?author=OALDERS",
want: Perl,
},
}
var languages = strset.New()

61
syft/pkg/perl.go Normal file
View File

@ -0,0 +1,61 @@
package pkg
import (
"sort"
"github.com/scylladb/go-set/strset"
)
// CpanDistribution describes a CPAN distribution installed on disk, as recorded by a CPAN client's
// install.json and by the .packlist an installer wrote. There is deliberately no name or version
// field: both would be verbatim copies of the package name and version. Licenses live on the package
// and dependencies live in relationships.
type CpanDistribution struct {
// Dist is the distvname install.json recorded (e.g. URI-5.35), the raw value the package name and
// version were recovered from
Dist string `json:"dist,omitempty"`
// MainModule is the module name recovered from the .packlist path (e.g. LWP for libwww-perl).
// Empty when the name came from install.json, which is authoritative. Set, it means the name is the
// installer's NAME and may not be the distribution name.
MainModule string `json:"mainModule,omitempty"`
// Author is the PAUSE ID of the distribution author (e.g. OALDERS), parsed out of Path
Author string `json:"author,omitempty"`
// Path is the raw PAUSE path of the release archive (e.g. O/OA/OALDERS/URI-5.35.tar.gz)
Path string `json:"path,omitempty"`
// Modules are the modules this distribution provides, sorted by name
Modules []CpanModule `json:"modules,omitempty"`
// Files are the paths the .packlist recorded as installed by this distribution, kept in the
// packlist's own order and unfiltered: a path the packlist claims and the filesystem lacks means the
// file was removed or overwritten out from under the installer, which is signal rather than noise
Files []string `json:"files,omitempty"`
}
// OwnedFiles satisfies pkg.FileOwner, so a distribution that installed files a distro package also
// owns is related to it by file ownership rather than reported as an unrelated duplicate.
func (m CpanDistribution) OwnedFiles() (result []string) {
result = strset.New(m.Files...).List()
sort.Strings(result)
return
}
// CpanUnpackedRelease describes an unpacked CPAN release found on disk, read from the release's own
// META.json or META.yml. Weaker evidence than an installed distribution: the code is present but
// nothing says the interpreter can load it. It has no PAUSE path, so it has no author.
type CpanUnpackedRelease struct {
// Modules are the modules the release declares it provides, sorted by name
Modules []CpanModule `json:"modules,omitempty"`
}
// CpanModule is a single perl module provided by a CPAN distribution. Shared by both types above.
type CpanModule struct {
// Name is the module name as it would be used in a perl `use` statement (e.g. URI::Escape)
Name string `json:"name"`
// Version is the module version, which may differ from the distribution version
Version string `json:"version,omitempty"`
}

48
syft/pkg/perl_test.go Normal file
View File

@ -0,0 +1,48 @@
package pkg
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestCpanDistribution_OwnedFiles(t *testing.T) {
tests := []struct {
name string
metadata CpanDistribution
expected []string
}{
{
name: "sorted and deduplicated",
metadata: CpanDistribution{
// a distribution that merged more than one packlist can claim the same path twice
Files: []string{"/lib/URI.pm", "/lib/URI/Escape.pm", "/lib/URI.pm"},
},
expected: []string{"/lib/URI.pm", "/lib/URI/Escape.pm"},
},
{
// this is what a NO_PACKLIST install leaves behind: an install.json and nothing else
name: "install.json only owns nothing",
metadata: CpanDistribution{Dist: "URI-5.35", Modules: []CpanModule{{Name: "URI"}}},
expected: []string{},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.expected, tt.metadata.OwnedFiles())
})
}
}
func TestCpanDistribution_isFileOwner(t *testing.T) {
var owner any = CpanDistribution{}
_, ok := owner.(FileOwner)
require.True(t, ok, "the file-ownership relationship workers only see metadata that satisfies FileOwner")
// the unpacked-release tier has no packlist, so it deliberately does not
var release any = CpanUnpackedRelease{}
_, ok = release.(FileOwner)
assert.False(t, ok)
}

View File

@ -21,6 +21,7 @@ const (
CocoapodsPkg Type = "pod"
ConanPkg Type = "conan"
CondaPkg Type = "conda"
CpanPkg Type = "cpan"
DartPubPkg Type = "dart-pub"
DebPkg Type = "deb"
DotnetPkg Type = "dotnet"
@ -68,6 +69,7 @@ var AllPkgs = []Type{
CocoapodsPkg,
ConanPkg,
CondaPkg,
CpanPkg,
DartPubPkg,
DebPkg,
DotnetPkg,
@ -123,6 +125,8 @@ func (t Type) PackageURLType() string {
return packageurl.TypeConan
case CondaPkg:
return packageurl.TypeGeneric
case CpanPkg:
return packageurl.TypeCpan
case DartPubPkg:
return packageurl.TypePub
case DebPkg:
@ -224,6 +228,8 @@ func TypeByName(name string) Type {
return RustPkg
case "conda":
return CondaPkg
case packageurl.TypeCpan:
return CpanPkg
case packageurl.TypePub:
return DartPubPkg
case "dotnet", packageurl.TypeNuget: // "dotnet" is here to support legacy use cases; "nuget" is the canonical purl type

View File

@ -125,6 +125,10 @@ func TestTypeFromPURL(t *testing.T) {
purl: "pkg:luarocks/kong@3.7.0",
expected: LuaRocksPkg,
},
{
purl: "pkg:cpan/URI@5.35?author=OALDERS",
expected: CpanPkg,
},
{
purl: "pkg:swift/github.com/apple/swift-numerics/swift-numerics@1.0.2",
expected: SwiftPkg,