mirror of
https://github.com/anchore/syft.git
synced 2026-08-19 16:48:27 +02:00
Adds `perl` as a language and `cpan` as a package type, with two catalogers behind them.
- `perl-cpan-installed-cataloger` reads `.meta/*/install.json` (cpanm, cpm, carton) and `auto/**/.packlist` (anything installed through `ExtUtils::MakeMaker` or `Module::Build`, including CPAN.pm). Both globs are unanchored, so a local-lib or carton application tree is found the same as a system install.
- `perl-cpan-meta-cataloger` reads an unpacked release's own `META.json` or `META.yml`, gated on a sibling `MANIFEST` so source checkouts are ignored.
Packages are keyed on the **distribution**, not the module, because that is what CPAN, MetaCPAN and the advisory data all use. `LWP.pm` belongs to `libwww-perl` and reporting it as `LWP` would make every advisory for it unreachable. The distribution name comes from `install.json`'s `dist` field, and from the `auto/` path for packlists.
purls are `pkg:cpan/<distribution>@<version>`, with the PAUSE author added as an `author` qualifier when the evidence carries it. Metadata comes in two types: `cpan-distribution` for installed evidence and `cpan-unpacked-release` for a release sitting on disk. The tiers differ in more than a name, so a consumer should not have to string-match a cataloger name to tell them apart: an unpacked release has no PAUSE path and therefore no author and no file list, and "installed and loadable by the interpreter" is a materially stronger claim than "a source tree exists here".
A packlist's version comes from `perllocal.pod`. EUMM writes it and the packlist from the same variables in the same install target, so the `auto/` path segments and the perllocal `Module` name are the same key, and the recorded `VERSION` is what was evaluated at build time rather than what can be read back statically. Scraping `$VERSION` out of the main `.pm` is the fallback, since `NO_PERLLOCAL` suppresses the file and Module::Build never writes one. Three rules pick the stanza: dashed module name, longest `installed into` libdir that prefixes the packlist path, and last match wins because the file is append-only. Without the libdir rule a second perl on the image silently supplies the version.
Where scraping is the fallback, it reads more than a plain `our $VERSION = '...'`. A version declared in the package statement (`package Foo::Bar v1.0.0;`) counts, which matters because a distribution can declare it that way and carry no `$VERSION` at all: `CPAN::02Packages::Search` is one, and without this it reports no version and matches nothing. Fully qualified `$Foo::Bar::VERSION` counts too, which is what older Dist::Zilla emitted and what real `JSON::PP` 2.27300 still carries. `qv('1.2.3')` is handled. A version computed at runtime is not, and those are reported without one rather than guessed at.
Packlist entries are parsed the way `ExtUtils::Packlist` writes them: a line can carry space-separated metadata after the path (`/path/to/File.pm type=file`), which is what `installperl` produces, so the suffix is stripped rather than the line being cut at its first space.
`META.yml` is read alongside `META.json`, because about 43% of current CPAN releases ship no `META.json` and they skew old, which is where the advisories are. Where both sit in one directory the `META.json` wins. A `META.yml` a strict parser rejects skips that directory rather than failing the scan, which is routine rather than defensive for pre-spec releases.
A packlist is literally a file list, so its paths are surfaced through `pkg.FileOwner`. They are reported as recorded, unfiltered: a path the packlist claims and the filesystem lacks means the file was removed or overwritten out from under the installer, which is worth seeing rather than hiding.
Build leftovers under `~/.cpanm/work` and `~/.cpan/build` are skipped. They genuinely are unpacked release tarballs, `MANIFEST` included, so the `MANIFEST` gate admits them and a path exclusion is the only signal available. Without it every distribution on an image that did not clean up is reported twice. The cost is that a tarball deliberately kept under `~/.cpan/build` stops being reported.
Two behaviors that look wrong but are not:
- a distribution can be reported twice at different versions. `libwww-perl` 5.836 bundles `HTTP-Date`, `HTTP-Message` and `LWP-MediaTypes`, and installing it over the modern standalone releases overwrites their `.pm` files. Both versions are genuinely present, so the merge refuses to pair disagreeing versions rather than hiding one.
- a distribution with no readable version is reported without one rather than dropped, so it stays visible.
The coverage boundary is stated in full in the `package perl` doc comment. In short: modules installed from distro packages are out of scope, since packagers strip CPAN metadata with `NO_PACKLIST` and deb, rpm and apk already report them. Core and dual-life distributions bundled with the interpreter have no coverage, because nothing on disk carries their versions; the interpreter itself is reported separately. Vendored trees, `App::FatPacker` output and PAR archives are invisible, because what they carry is module identity with no offline map to a distribution. A packlist-derived name is the installer's `NAME` and may not be the distribution name, which is left to the vulnerability data to resolve.
One correction worth calling out, since it is easy to arrive at twice: a sibling `MANIFEST` is the only thing separating an unpacked release from a source checkout. An earlier version of the guard also rejected any tree containing a `dist.ini`, on the theory that it marked a Dist::Zilla source tree. dzil ships `dist.ini` *inside* the tarballs it builds and lists it in the generated `MANIFEST`, so that exclusion was silently skipping real releases, `URI` among them.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
280 lines
4.3 KiB
Go
280 lines
4.3 KiB
Go
package pkg
|
|
|
|
import (
|
|
"testing"
|
|
|
|
"github.com/scylladb/go-set/strset"
|
|
"github.com/stretchr/testify/assert"
|
|
)
|
|
|
|
func TestLanguageFromPURL(t *testing.T) {
|
|
|
|
tests := []struct {
|
|
purl string
|
|
want Language
|
|
}{
|
|
|
|
{
|
|
purl: "pkg:npm/util@2.32",
|
|
want: JavaScript,
|
|
},
|
|
{
|
|
purl: "pkg:pypi/util-linux@2.32.1-27.el8",
|
|
want: Python,
|
|
},
|
|
{
|
|
purl: "pkg:gem/ruby-advisory-db-check@0.12.4",
|
|
want: Ruby,
|
|
},
|
|
{
|
|
purl: "pkg:golang/github.com/gorilla/context@234fd47e07d1004f0aed9c",
|
|
want: Go,
|
|
},
|
|
{
|
|
purl: "pkg:pub/util@1.2.34",
|
|
want: Dart,
|
|
},
|
|
{
|
|
purl: "pkg:dotnet/Microsoft.CodeAnalysis.Razor@2.2.0",
|
|
want: Dotnet,
|
|
},
|
|
{
|
|
purl: "pkg:nuget/Newtonsoft.Json@13.0.0",
|
|
want: Dotnet,
|
|
},
|
|
{
|
|
purl: "pkg:cargo/clap@2.33.0",
|
|
want: Rust,
|
|
},
|
|
{
|
|
purl: "pkg:composer/laravel/laravel@5.5.0",
|
|
want: PHP,
|
|
},
|
|
{
|
|
purl: "pkg:maven/org.apache.xmlgraphics/batik-anim@1.9.1?type=zip&classifier=dist",
|
|
want: Java,
|
|
},
|
|
{
|
|
purl: "pkg:cocoapods/GlossButtonNode@3.1.2",
|
|
want: Swift,
|
|
},
|
|
{
|
|
purl: "pkg:conan/catch2@2.13.8",
|
|
want: CPP,
|
|
},
|
|
{
|
|
purl: "pkg:hackage/HTTP@4000.3.16",
|
|
want: Haskell,
|
|
},
|
|
{
|
|
purl: "pkg:hex/hpax/hpax@0.1.1",
|
|
want: UnknownLanguage,
|
|
},
|
|
{
|
|
purl: "pkg:cran/base@4.3.0",
|
|
want: R,
|
|
},
|
|
{
|
|
purl: "pkg:swift/github.com/apple/swift-numerics/swift-numerics@1.0.2",
|
|
want: Swift,
|
|
},
|
|
{
|
|
purl: "pkg:swiplpack/conditon@0.1.1",
|
|
want: Swipl,
|
|
},
|
|
{
|
|
purl: "pkg:luarocks/kong@3.7.0",
|
|
want: Lua,
|
|
},
|
|
{
|
|
purl: "pkg:opam/ocaml-base-compiler@5.2.0",
|
|
want: OCaml,
|
|
},
|
|
{
|
|
purl: "pkg:cpan/URI@5.35?author=OALDERS",
|
|
want: Perl,
|
|
},
|
|
}
|
|
|
|
var languages = strset.New()
|
|
var expectedLanguages = strset.New()
|
|
for _, ty := range AllLanguages {
|
|
expectedLanguages.Add(string(ty))
|
|
}
|
|
|
|
// we cannot determine the language from these purl ecosystems (yet?)
|
|
expectedLanguages.Remove(Elixir.String())
|
|
expectedLanguages.Remove(Erlang.String())
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.purl, func(t *testing.T) {
|
|
actual := LanguageFromPURL(tt.purl)
|
|
|
|
if actual != "" {
|
|
languages.Add(string(actual))
|
|
}
|
|
|
|
assert.Equalf(t, tt.want, actual, "LanguageFromPURL(%v)", tt.purl)
|
|
})
|
|
}
|
|
|
|
assert.ElementsMatch(t, expectedLanguages.List(), languages.List(), "missing one or more languages to test against (maybe a package type was added?)")
|
|
|
|
}
|
|
|
|
func TestLanguageByName(t *testing.T) {
|
|
tests := []struct {
|
|
name string
|
|
language Language
|
|
}{
|
|
{
|
|
name: "maven",
|
|
language: Java,
|
|
},
|
|
{
|
|
name: "java",
|
|
language: Java,
|
|
},
|
|
{
|
|
name: "java-archive",
|
|
language: Java,
|
|
},
|
|
{
|
|
name: "java",
|
|
language: Java,
|
|
},
|
|
{
|
|
name: "composer",
|
|
language: PHP,
|
|
},
|
|
{
|
|
name: "php-composer",
|
|
language: PHP,
|
|
},
|
|
{
|
|
name: "php",
|
|
language: PHP,
|
|
},
|
|
{
|
|
name: "go",
|
|
language: Go,
|
|
},
|
|
{
|
|
name: "golang",
|
|
language: Go,
|
|
},
|
|
{
|
|
name: "go-module",
|
|
language: Go,
|
|
},
|
|
{
|
|
name: "npm",
|
|
language: JavaScript,
|
|
},
|
|
{
|
|
name: "javascript",
|
|
language: JavaScript,
|
|
},
|
|
{
|
|
name: "node.js",
|
|
language: JavaScript,
|
|
},
|
|
{
|
|
name: "nodejs",
|
|
language: JavaScript,
|
|
},
|
|
{
|
|
name: "pypi",
|
|
language: Python,
|
|
},
|
|
{
|
|
name: "python",
|
|
language: Python,
|
|
},
|
|
{
|
|
name: "gem",
|
|
language: Ruby,
|
|
},
|
|
{
|
|
name: "ruby",
|
|
language: Ruby,
|
|
},
|
|
{
|
|
name: "rust",
|
|
language: Rust,
|
|
},
|
|
{
|
|
name: "rust-crate",
|
|
language: Rust,
|
|
},
|
|
{
|
|
name: "cargo",
|
|
language: Rust,
|
|
},
|
|
{
|
|
name: "dart",
|
|
language: Dart,
|
|
},
|
|
{
|
|
name: "dart-pub",
|
|
language: Dart,
|
|
},
|
|
{
|
|
name: "pub",
|
|
language: Dart,
|
|
},
|
|
{
|
|
name: "dotnet",
|
|
language: Dotnet,
|
|
},
|
|
{
|
|
name: "swift",
|
|
language: Swift,
|
|
},
|
|
{
|
|
name: "swiplpack",
|
|
language: Swipl,
|
|
},
|
|
{
|
|
name: "opam",
|
|
language: OCaml,
|
|
},
|
|
{
|
|
name: "pod",
|
|
language: Swift,
|
|
},
|
|
{
|
|
name: "cocoapods",
|
|
language: Swift,
|
|
},
|
|
{
|
|
name: "unknown",
|
|
language: UnknownLanguage,
|
|
},
|
|
{
|
|
name: "conan",
|
|
language: CPP,
|
|
},
|
|
{
|
|
name: "c++",
|
|
language: CPP,
|
|
},
|
|
{
|
|
name: "hackage",
|
|
language: Haskell,
|
|
},
|
|
{
|
|
name: "haskell",
|
|
language: Haskell,
|
|
},
|
|
{
|
|
name: "R",
|
|
language: R,
|
|
},
|
|
}
|
|
|
|
for _, test := range tests {
|
|
assert.Equal(t, LanguageByName(test.name), test.language)
|
|
}
|
|
}
|