feat(golang): add extended-stdlib scope and module patterns for symbol capture (#5154)

* feat(golang): add extended-stdlib scope and include patterns for symbol capture

`golang.capture-symbols` decides how much symbol data lands in the SBOM for grype's reachability analysis. It's `none`, `stdlib`, or `all` today, and the useful middle is missing: `stdlib` stops at the standard library, `all` multiplies SBOM size.

A new `extended-stdlib` configurable covers stdlib plus everything under `golang.org/x/`:

```yaml
golang:
  capture-symbols: extended-stdlib
```

Also, a new `capture-symbols-include` configurable for modules that are noisy in your binaries but not everyone's. It's unioned with whatever the scope selects, so it only ever widens:

```yaml
golang:
  capture-symbols: extended-stdlib
  capture-symbols-include:
    - github.com/klauspost/**
```

Patterns are standard doublestar globs, which matters because module paths carry `/v2`-style suffixes:

```yaml
golang:
  capture-symbols-include:
    - github.com/klauspost/*     # compress, but not compress/v2
    - github.com/klauspost/**    # both
    - k8s.io/client-go           # exact match only
```

Ordering is `none` < `stdlib` < `extended-stdlib` < `all`. The existing three values
and the `none` default are unchanged, and the include list is inert under `none`.
Presets compile into glob lists internally, so a single matcher answers "does this
module get symbols" instead of a preset branch sitting next to a separate glob branch.

An unrecognized `capture-symbols` value still falls back to `none`, but warns now
instead of doing it silently. A malformed include pattern warns and gets skipped.

One thing worth a look beyond the feature: the `Symbols` field description in the JSON
schema was wrong after this (it claimed only `all` and `stdlib` populate anything), and
that description lives in the already-published `16.1.10`. Rather than bump a version for
a sentence, `16.1.10` is amended in place and `schema/json/README.md` grows an explicit
exception for description-only changes: descriptions only, no shape change of any kind,
`$id` unchanged. Anything else still needs a bump. Happy to split that into its own PR if
you'd rather review the policy separately.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* refactor(golang): rename capture-symbols-include to capture-symbols-modules

The key's entries are go module paths, and `-include` sitting next to `capture-symbols` reads as plausibly taking symbol or package names instead. Those spellings parse and match nothing, which is quieter than the confusion `-include` was picked to avoid, so the name now says what the list holds.

`golang.CatalogerConfig.CaptureSymbolsModules` and `WithCaptureSymbolsModules` rename with it. Nothing behavioral changes; the key is new in this PR so there is no compatibility surface.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* feat(golang): match capture-symbols-modules across major version suffixes

`github.com/anchore/*` covered `github.com/anchore/syft` and silently stopped covering it the day it became `github.com/anchore/syft/v2`. The config keeps parsing, nothing warns, and symbols quietly go missing from the SBOM. Exact paths had the same hole: `github.com/klauspost/compress` did not cover `compress/v2` either, so no spelling short of `**` survived a major bump.

A major version suffix is part of a module's path but not part of its identity, so patterns are now matched against the module path both with and without it, using `module.SplitPathVersion` from `golang.org/x/mod` (already a direct dep, already used in this package for `PseudoVersion`).

```yaml
golang:
  capture-symbols-modules:
    - github.com/klauspost/*          # compress and compress/v2
    - github.com/klauspost/compress   # same module at every major version
    - github.com/klauspost/compress/v2  # v2 alone
```

Only a trailing suffix is a version, which is Go's own rule. In `github.com/anchore/syft/v2/thing` the `v2` is an ordinary path element naming a major subdirectory a nested module lives in, so it stays literal and `github.com/anchore/**/thing` is how you reach it. `/v0` and `/v1` are not valid suffixes and are left alone.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
This commit is contained in:
Alex Goodman
2026-08-07 12:44:42 +00:00
committed by GitHub
parent 68da404bd7
commit da745b13e8
16 changed files with 648 additions and 28 deletions
+6
View File
@@ -12,6 +12,10 @@ const (
// SymbolScopeStdlib captures symbols only for the synthetic "stdlib" package, leaving module packages without symbols.
SymbolScopeStdlib SymbolScope = "stdlib"
// SymbolScopeExtendedStdlib captures symbols for the synthetic "stdlib" package as well as every module
// under golang.org/x/ (the extended standard library).
SymbolScopeExtendedStdlib SymbolScope = "extended-stdlib"
// SymbolScopeAll captures symbols for all module packages as well as the synthetic "stdlib" package.
SymbolScopeAll SymbolScope = "all"
)
@@ -21,6 +25,8 @@ func (s SymbolScope) Parse() SymbolScope {
switch strings.ToLower(strings.TrimSpace(string(s))) {
case string(SymbolScopeAll):
return SymbolScopeAll
case string(SymbolScopeExtendedStdlib):
return SymbolScopeExtendedStdlib
case string(SymbolScopeStdlib):
return SymbolScopeStdlib
}
+5
View File
@@ -16,6 +16,11 @@ func Test_SymbolScope_Parse(t *testing.T) {
{" all ", SymbolScopeAll},
{"stdlib", SymbolScopeStdlib},
{"Stdlib", SymbolScopeStdlib},
{"extended-stdlib", SymbolScopeExtendedStdlib},
{"Extended-Stdlib", SymbolScopeExtendedStdlib},
{"EXTENDED-STDLIB", SymbolScopeExtendedStdlib},
{" extended-stdlib ", SymbolScopeExtendedStdlib},
{"extended_stdlib", SymbolScopeNone},
{"none", SymbolScopeNone},
{"", SymbolScopeNone},
{"true", SymbolScopeNone},
+4 -1
View File
@@ -25,8 +25,11 @@ configs: # AUTO-GENERATED - config structs and their fields
description: NoProxy is a list of glob patterns that match go module names that should not be fetched from the go proxy. When not set, syft will use the GOPRIVATE and GONOPROXY env vars.
app_key: golang.no-proxy
- key: CaptureSymbols
description: CaptureSymbols controls extracting function symbols from the binary symbol table (pclntab). Valid values are "none" (disabled), "stdlib" (only the synthetic stdlib package), and "all" (all module packages plus stdlib).
description: CaptureSymbols controls extracting function symbols from the binary symbol table (pclntab). Valid values are "none" (disabled), "stdlib" (only the synthetic stdlib package), "extended-stdlib" (stdlib plus every module under golang.org/x/), and "all" (all module packages plus stdlib).
app_key: golang.capture-symbols
- key: CaptureSymbolsModules
description: 'CaptureSymbolsModules is a list of glob patterns (doublestar syntax, where ** crosses path separators and * does not) matched against go module paths. Matching modules get symbols in addition to whatever CaptureSymbols selects, so this can only widen the selection and never narrow it. It has no effect under the "none" scope. A trailing major version suffix is not part of a module''s identity, so a pattern matches with or without it: both "github.com/foo/bar" and "github.com/foo/*" select "github.com/foo/bar/v2". A pattern that spells out a suffix selects only that major version.'
app_key: golang.capture-symbols-modules
catalogers:
- ecosystem: go # MANUAL
name: go-module-binary-cataloger # AUTO-GENERATED
+16 -1
View File
@@ -51,10 +51,20 @@ type CatalogerConfig struct {
MainModuleVersion MainModuleVersionConfig `yaml:"main-module-version" json:"main-module-version" mapstructure:"main-module-version"`
// CaptureSymbols controls extracting function symbols from the binary symbol table (pclntab). Valid values are
// "none" (disabled), "stdlib" (only the synthetic stdlib package), and "all" (all module packages plus stdlib).
// "none" (disabled), "stdlib" (only the synthetic stdlib package), "extended-stdlib" (stdlib plus every module
// under golang.org/x/), and "all" (all module packages plus stdlib).
// app-config: golang.capture-symbols
CaptureSymbols cataloging.SymbolScope `yaml:"capture-symbols" json:"capture-symbols" mapstructure:"capture-symbols"`
// CaptureSymbolsModules is a list of glob patterns (doublestar syntax, where ** crosses path separators
// and * does not) matched against go module paths. Matching modules get symbols in addition to whatever
// CaptureSymbols selects, so this can only widen the selection and never narrow it. It has no effect
// under the "none" scope. A trailing major version suffix is not part of a module's identity, so a
// pattern matches with or without it: both "github.com/foo/bar" and "github.com/foo/*" select
// "github.com/foo/bar/v2". A pattern that spells out a suffix selects only that major version.
// app-config: golang.capture-symbols-modules
CaptureSymbolsModules []string `yaml:"capture-symbols-modules,omitempty" json:"capture-symbols-modules,omitempty" mapstructure:"capture-symbols-modules"`
// Whether to use the golang.org/x/tools/go/packages, which executes golang tooling found on the path in addition to potential network access
UsePackagesLib bool `json:"use-packages-lib" yaml:"use-packages-lib" mapstructure:"use-packages-lib"`
}
@@ -196,6 +206,11 @@ func (g CatalogerConfig) WithCaptureSymbols(input cataloging.SymbolScope) Catalo
return g
}
func (g CatalogerConfig) WithCaptureSymbolsModules(input []string) CatalogerConfig {
g.CaptureSymbolsModules = input
return g
}
func (g CatalogerConfig) WithUsePackagesLib(useLib bool) CatalogerConfig {
g.UsePackagesLib = useLib
return g
+7 -9
View File
@@ -21,7 +21,6 @@ import (
"github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/cataloging"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/internal/unionreader"
"github.com/anchore/syft/syft/pkg"
@@ -51,7 +50,7 @@ const devel = "(devel)"
type goBinaryCataloger struct {
licenseResolver goLicenseResolver
mainModuleVersion MainModuleVersionConfig
symbolScope cataloging.SymbolScope
symbolSelector symbolSelector
// stdlibSymbols holds the standard-library function symbols discovered per binary (keyed by the
// binary's location), grouped by import path, populated during parsing and consumed by stdlibProcessor
@@ -65,7 +64,7 @@ func newGoBinaryCataloger(opts CatalogerConfig) *goBinaryCataloger {
return &goBinaryCataloger{
licenseResolver: newGoLicenseResolver(binaryCatalogerName, opts),
mainModuleVersion: opts.MainModuleVersion,
symbolScope: opts.CaptureSymbols,
symbolSelector: newSymbolSelector(opts.CaptureSymbols, opts.CaptureSymbolsModules),
stdlibSymbols: make(map[file.Coordinates]map[string][]string),
}
}
@@ -121,7 +120,7 @@ func (c *goBinaryCataloger) parseGoBinary(ctx context.Context, resolver file.Res
}
defer internal.CloseAndLogError(reader.ReadCloser, reader.RealPath)
mods, errs := scanFile(reader.Location, unionReader, c.symbolScope != cataloging.SymbolScopeNone)
mods, errs := scanFile(reader.Location, unionReader, c.symbolSelector.enabled())
var rels []artifact.Relationship
for _, mod := range mods {
@@ -184,11 +183,10 @@ func (c *goBinaryCataloger) buildGoPkgInfo(ctx context.Context, resolver file.Re
symbolsByModule, stdlibSymbols := moduleSymbols(mod.symbols, &mod.Main, mod.Deps)
c.recordStdlibSymbols(location.Coordinates, stdlibSymbols)
if c.symbolScope != cataloging.SymbolScopeAll {
// only the "all" scope attaches per-module symbols; for the "stdlib" scope we keep just the
// recorded stdlib symbols. nil map lookups below then yield nil symbol lists for each module.
symbolsByModule = nil
}
// keep only the modules the selector covers; the main module goes through the same map, so this
// incidentally decides the main module too (which is intended: it is treated like any dependency).
// unselected modules fall out entirely, so the lookups below yield nil rather than an empty map.
symbolsByModule = c.symbolSelector.filter(symbolsByModule)
var pkgs []pkg.Package
for _, dep := range mod.Deps {
@@ -4,12 +4,14 @@ import (
"bufio"
"bytes"
"context"
"encoding/json"
"errors"
"io"
"os"
"os/exec"
"path/filepath"
"runtime/debug"
"slices"
"strconv"
"strings"
"syscall"
@@ -1441,9 +1443,12 @@ func Test_buildGoPkgInfo_symbolScope(t *testing.T) {
tests := []struct {
name string
scope cataloging.SymbolScope
modules []string
extraDeps []*debug.Module
symbols []binarySymbol
wantMainSyms map[string][]string
wantDepSyms map[string][]string
wantExtraSyms []map[string][]string
wantStdlibSyms map[string][]string
}{
{
@@ -1468,6 +1473,84 @@ func Test_buildGoPkgInfo_symbolScope(t *testing.T) {
},
wantStdlibSyms: map[string][]string{"net/http": {"(*Client).Do"}},
},
{
// golang.org/x/net is reached through a vendored import path here, which must still be
// attributed to (and selected by) the module that owns it
name: "extended-stdlib captures golang.org/x modules and stdlib only",
scope: cataloging.SymbolScopeExtendedStdlib,
extraDeps: extendedDeps,
symbols: slices.Concat(populatedSymbols, extendedSymbols),
wantExtraSyms: []map[string][]string{
{"golang.org/x/net/http2": {"NewClientConn"}},
nil,
},
wantStdlibSyms: map[string][]string{"net/http": {"(*Client).Do"}},
},
{
name: "module patterns widen extended-stdlib",
scope: cataloging.SymbolScopeExtendedStdlib,
modules: []string{"github.com/klauspost/**"},
extraDeps: extendedDeps,
symbols: slices.Concat(populatedSymbols, extendedSymbols),
wantExtraSyms: []map[string][]string{
{"golang.org/x/net/http2": {"NewClientConn"}},
{"github.com/klauspost/compress/zstd": {"NewReader"}},
},
wantStdlibSyms: map[string][]string{"net/http": {"(*Client).Do"}},
},
{
name: "module patterns can select the main module",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/anchore/**"},
extraDeps: extendedDeps,
symbols: slices.Concat(populatedSymbols, extendedSymbols),
// the main package is keyed by the "main" import path the linker assigns, not its real path
wantMainSyms: map[string][]string{"main": {"main"}},
wantExtraSyms: []map[string][]string{nil, nil},
wantStdlibSyms: map[string][]string{"net/http": {"(*Client).Do"}},
},
{
name: "module patterns are inert under the none scope",
scope: cataloging.SymbolScopeNone,
modules: []string{"github.com/**", "golang.org/x/**"},
extraDeps: extendedDeps,
// scanFile never runs under "none", so the build info carries no symbols to begin with
symbols: nil,
wantExtraSyms: []map[string][]string{nil, nil},
},
{
// all short-circuits ahead of the pattern walk, so a narrow module pattern list cannot subtract
// from it. this is also the guard on that short-circuit still existing.
name: "module patterns cannot narrow the all scope",
scope: cataloging.SymbolScopeAll,
modules: []string{"golang.org/x/**"},
extraDeps: extendedDeps,
symbols: slices.Concat(populatedSymbols, extendedSymbols),
wantMainSyms: map[string][]string{"main": {"main"}},
wantDepSyms: map[string][]string{
"github.com/foo/bar": {"Parse"},
"github.com/foo/bar/baz": {"Helper"},
},
wantExtraSyms: []map[string][]string{
{"golang.org/x/net/http2": {"NewClientConn"}},
{"github.com/klauspost/compress/zstd": {"NewReader"}},
},
wantStdlibSyms: map[string][]string{"net/http": {"(*Client).Do"}},
},
{
// a module pattern that restates what the preset already covers must not duplicate or drop anything:
// selection is a per-module boolean, so overlap is idempotent
name: "a module pattern overlapping the preset changes nothing",
scope: cataloging.SymbolScopeExtendedStdlib,
modules: []string{"golang.org/x/**"},
extraDeps: extendedDeps,
symbols: slices.Concat(populatedSymbols, extendedSymbols),
wantExtraSyms: []map[string][]string{
{"golang.org/x/net/http2": {"NewClientConn"}},
nil,
},
wantStdlibSyms: map[string][]string{"net/http": {"(*Client).Do"}},
},
}
for _, tt := range tests {
@@ -1476,27 +1559,52 @@ func Test_buildGoPkgInfo_symbolScope(t *testing.T) {
BuildInfo: &debug.BuildInfo{
GoVersion: "go1.22.0",
Main: debug.Module{Path: "github.com/anchore/syft", Version: "v1.0.0"},
Deps: []*debug.Module{{Path: "github.com/foo/bar", Version: "v1.2.3"}},
Deps: append([]*debug.Module{{Path: "github.com/foo/bar", Version: "v1.2.3"}}, tt.extraDeps...),
},
arch: "amd64",
symbols: tt.symbols,
}
c := newGoBinaryCataloger(CatalogerConfig{CaptureSymbols: tt.scope})
c := newGoBinaryCataloger(CatalogerConfig{CaptureSymbols: tt.scope, CaptureSymbolsModules: tt.modules})
reader, err := unionreader.GetUnionReader(io.NopCloser(strings.NewReader("")))
require.NoError(t, err)
mainPkg, pkgs := c.buildGoPkgInfo(context.Background(), fileresolver.Empty{}, location, mod, mod.arch, reader)
require.NotNil(t, mainPkg)
require.Len(t, pkgs, 1)
require.Len(t, pkgs, 1+len(tt.extraDeps))
assert.Equal(t, tt.wantMainSyms, mainPkg.Metadata.(pkg.GolangBinaryBuildinfoEntry).Symbols, "main module symbols")
assert.Equal(t, tt.wantDepSyms, pkgs[0].Metadata.(pkg.GolangBinaryBuildinfoEntry).Symbols, "dependency symbols")
for i, want := range tt.wantExtraSyms {
assert.Equal(t, want, pkgs[1+i].Metadata.(pkg.GolangBinaryBuildinfoEntry).Symbols, "symbols for %s", pkgs[1+i].Name)
}
assert.Equal(t, tt.wantStdlibSyms, c.stdlibSymbolsFor(location.Coordinates), "recorded stdlib symbols")
// a module selected by nothing must carry no symbols key at all: assert on the serialized form,
// since an empty (non-nil) map would still emit "symbols":{} despite the omitempty tag
for _, p := range append([]pkg.Package{*mainPkg}, pkgs...) {
encoded, err := json.Marshal(p.Metadata)
require.NoError(t, err)
if p.Metadata.(pkg.GolangBinaryBuildinfoEntry).Symbols == nil {
assert.NotContains(t, string(encoded), `"symbols"`, "expected no symbols key for %s", p.Name)
}
}
})
}
}
var (
extendedDeps = []*debug.Module{
{Path: "golang.org/x/net", Version: "v0.30.0"},
{Path: "github.com/klauspost/compress", Version: "v1.17.0"},
}
extendedSymbols = []binarySymbol{
{packagePath: "vendor/golang.org/x/net/http2", name: "vendor/golang.org/x/net/http2.NewClientConn"},
{packagePath: "github.com/klauspost/compress/zstd", name: "github.com/klauspost/compress/zstd.NewReader"},
}
)
// Test_recordStdlibSymbols_merge covers the merge path where the same binary location records stdlib
// symbols more than once. This happens in production for universal/fat Mach-O binaries: scanFile yields
// one build info per architecture and each is recorded under the same location coordinates.
@@ -0,0 +1,113 @@
package golang
import (
"slices"
"github.com/bmatcuk/doublestar/v4"
"golang.org/x/mod/module"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/cataloging"
)
// scopePatterns maps a capture-symbols scope onto the go module path globs it selects. The "none" and
// "all" scopes are intentionally absent: both are answered without matching. The "stdlib" scope selects
// no modules at all (the synthetic stdlib package is not a module and is handled separately).
var scopePatterns = map[cataloging.SymbolScope][]string{
cataloging.SymbolScopeExtendedStdlib: {"golang.org/x/**"},
}
// symbolSelector decides which go module paths get function symbols attached to their metadata. The scope
// preset and any user-supplied module patterns are compiled into a single glob list so there is exactly
// one matcher answering "does this module path get symbols" (two matchers over the same subject drift).
type symbolSelector struct {
scope cataloging.SymbolScope
patterns []string
}
func newSymbolSelector(scope cataloging.SymbolScope, modules []string) symbolSelector {
// normalize here rather than trusting the caller: the CLI runs Parse in PostLoad, but a library
// consumer setting CaptureSymbols directly (or via WithCaptureSymbols) does not, and an unnormalized
// value falls through both switches below into stdlib-only capture rather than the intended scope.
scope = scope.Parse()
var patterns []string
for _, pattern := range slices.Concat(scopePatterns[scope], modules) {
if !doublestar.ValidatePattern(pattern) {
// a typo in a filter that decides what gets vulnerability-scanned must not pass quietly:
// someone would believe they captured symbols they did not. Drop just this pattern and keep
// the rest of the selection in force.
log.WithFields("pattern", pattern).Warn("ignoring malformed golang capture-symbols-modules pattern")
continue
}
patterns = append(patterns, pattern)
}
return symbolSelector{scope: scope, patterns: patterns}
}
// enabled reports whether symbols should be extracted from the binary at all.
func (s symbolSelector) enabled() bool {
return s.scope != cataloging.SymbolScopeNone
}
// selects reports whether the given go module path gets symbols. The binary's own main module is treated
// exactly like a dependency.
func (s symbolSelector) selects(modulePath string) bool {
switch s.scope {
case cataloging.SymbolScopeNone:
return false
case cataloging.SymbolScopeAll:
return true
}
// a major version suffix is part of a module's path but not part of its identity, so patterns are matched
// against the path both with and without it: `github.com/foo/bar` and `github.com/foo/*` each select
// `github.com/foo/bar/v2`, which is what whoever wrote either one meant, and a config does not quietly
// stop covering a module the day it bumps a major version. Spelling a suffix out in the pattern still
// selects that major version alone, since the unsuffixed path cannot match a pattern carrying one.
// Only a trailing suffix is a version: in `github.com/foo/v2/bar` the `v2` is an ordinary path element,
// and SplitPathVersion leaves it there.
unversioned, major, ok := module.SplitPathVersion(modulePath)
versioned := ok && major != ""
for _, pattern := range s.patterns {
if globMatches(pattern, modulePath) {
return true
}
if versioned && globMatches(pattern, unversioned) {
return true
}
}
return false
}
// globMatches treats a pattern that cannot compile as no match, which newSymbolSelector has already warned about.
func globMatches(pattern, modulePath string) bool {
matched, err := doublestar.Match(pattern, modulePath)
if err != nil {
// unreachable: newSymbolSelector rejects (and warns about) patterns that cannot compile
return false
}
return matched
}
// filter drops the entries of a module-path-keyed symbol map that the selector does not select. It returns
// nil when nothing is selected so callers attach no symbols map at all (rather than an empty one, which
// would defeat the omitempty JSON tag).
func (s symbolSelector) filter(byModule map[string]map[string][]string) map[string]map[string][]string {
switch s.scope {
case cataloging.SymbolScopeNone:
return nil
case cataloging.SymbolScopeAll:
return byModule
}
for modulePath := range byModule {
if !s.selects(modulePath) {
delete(byModule, modulePath)
}
}
if len(byModule) == 0 {
return nil
}
return byModule
}
@@ -0,0 +1,280 @@
package golang
import (
"maps"
"slices"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/cataloging"
)
func Test_symbolSelector_selects(t *testing.T) {
tests := []struct {
name string
scope cataloging.SymbolScope
modules []string
modulePath string
want bool
}{
{
name: "none selects nothing",
scope: cataloging.SymbolScopeNone,
modulePath: "github.com/foo/bar",
},
{
name: "none is inert even with module patterns",
scope: cataloging.SymbolScopeNone,
modules: []string{"github.com/foo/**"},
modulePath: "github.com/foo/bar",
},
{
name: "all selects everything",
scope: cataloging.SymbolScopeAll,
modulePath: "github.com/foo/bar",
want: true,
},
{
name: "stdlib selects no modules",
scope: cataloging.SymbolScopeStdlib,
modulePath: "golang.org/x/crypto",
},
{
name: "extended-stdlib matches a golang.org/x module",
scope: cataloging.SymbolScopeExtendedStdlib,
modulePath: "golang.org/x/crypto",
want: true,
},
{
// ** crosses path separators, which is what lets one pattern cover the whole subtree
name: "extended-stdlib matches a nested golang.org/x module",
scope: cataloging.SymbolScopeExtendedStdlib,
modulePath: "golang.org/x/tools/gopls",
want: true,
},
{
// the pattern must not match on a bare prefix, only at a path boundary
name: "extended-stdlib does not match golang.org/xtra",
scope: cataloging.SymbolScopeExtendedStdlib,
modulePath: "golang.org/xtra",
},
{
name: "extended-stdlib does not match an unrelated module",
scope: cataloging.SymbolScopeExtendedStdlib,
modulePath: "github.com/klauspost/compress",
},
{
name: "single star matches one path segment",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/klauspost/*"},
modulePath: "github.com/klauspost/compress",
want: true,
},
{
// a single star does not cross "/", but a major version suffix is not a path segment for this
// purpose: the module is matched with the suffix stripped as well, so a config written before a
// major bump keeps covering the module after it
name: "single star reaches across a major version suffix",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/klauspost/*"},
modulePath: "github.com/klauspost/compress/v2",
want: true,
},
{
name: "single star does not cross a separator that is not a version suffix",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/klauspost/*"},
modulePath: "github.com/klauspost/compress/internal/thing",
},
{
// only a trailing suffix is a version. here "v2" is an ordinary path element naming the major
// subdirectory a nested module lives in, so it is matched literally and ** is the way to reach it
name: "a mid-path version-like element is an ordinary segment",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/anchore/*/thing"},
modulePath: "github.com/anchore/syft/v2/thing",
},
{
name: "doublestar reaches a mid-path version-like element",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/anchore/**/thing"},
modulePath: "github.com/anchore/syft/v2/thing",
want: true,
},
{
name: "doublestar crosses a separator",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/klauspost/**"},
modulePath: "github.com/klauspost/compress/v2",
want: true,
},
{
name: "an exact path covers every major version of that module",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/klauspost/compress"},
modulePath: "github.com/klauspost/compress/v2",
want: true,
},
{
// spelling the suffix out is how a single major version is targeted: v1's path carries no suffix,
// so there is nothing for a suffixed pattern to match
name: "a pattern naming a major version selects only that one",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/klauspost/compress/v2"},
modulePath: "github.com/klauspost/compress",
},
{
name: "a pattern naming a major version selects it",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/klauspost/compress/v2"},
modulePath: "github.com/klauspost/compress/v2",
want: true,
},
{
// gopkg.in spells the major version as a .vN suffix on the last element, which SplitPathVersion
// understands, so it strips the same way
name: "gopkg.in style suffixes strip too",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"gopkg.in/yaml"},
modulePath: "gopkg.in/yaml.v2",
want: true,
},
{
// /v1 and /v0 are not valid major version suffixes, so this is not a versioned path at all and
// nothing is stripped from it
name: "a v1 element is not a version suffix",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"github.com/klauspost/compress"},
modulePath: "github.com/klauspost/compress/v1",
},
{
name: "module patterns widen a preset",
scope: cataloging.SymbolScopeExtendedStdlib,
modules: []string{"github.com/klauspost/**"},
modulePath: "github.com/klauspost/compress",
want: true,
},
{
name: "module patterns cannot narrow a preset",
scope: cataloging.SymbolScopeExtendedStdlib,
modules: []string{"github.com/klauspost/**"},
modulePath: "golang.org/x/net",
want: true,
},
{
name: "exact module path",
scope: cataloging.SymbolScopeStdlib,
modules: []string{"google.golang.org/grpc"},
modulePath: "google.golang.org/grpc",
want: true,
},
{
name: "an empty module pattern list is a no-op",
scope: cataloging.SymbolScopeStdlib,
modules: []string{},
modulePath: "github.com/foo/bar",
},
{
// all short-circuits before any matching, so a module pattern list cannot subtract from it
name: "module patterns cannot narrow all",
scope: cataloging.SymbolScopeAll,
modules: []string{"github.com/klauspost/**"},
modulePath: "github.com/foo/bar",
want: true,
},
{
// selection is a per-module boolean, so a pattern overlapping the preset is idempotent
name: "a module pattern overlapping the preset is idempotent",
scope: cataloging.SymbolScopeExtendedStdlib,
modules: []string{"golang.org/x/**"},
modulePath: "golang.org/x/net",
want: true,
},
{
// library consumers set CaptureSymbols directly without going through PostLoad, so the
// selector normalizes rather than falling through to stdlib-only capture
name: "unnormalized scope is parsed",
scope: "All",
modulePath: "github.com/foo/bar",
want: true,
},
{
name: "unrecognized scope resolves to none",
scope: "bogus",
modules: []string{"github.com/foo/**"},
modulePath: "github.com/foo/bar",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, newSymbolSelector(tt.scope, tt.modules).selects(tt.modulePath))
})
}
}
// a malformed pattern must be dropped at construction, leaving the remaining patterns in force rather
// than aborting the selection or silently matching nothing
func Test_symbolSelector_malformedPattern(t *testing.T) {
s := newSymbolSelector(cataloging.SymbolScopeExtendedStdlib, []string{"[", "github.com/klauspost/**"})
require.Equal(t, []string{"golang.org/x/**", "github.com/klauspost/**"}, s.patterns)
assert.True(t, s.selects("golang.org/x/net"), "preset still applies")
assert.True(t, s.selects("github.com/klauspost/compress"), "valid sibling pattern still applies")
assert.False(t, s.selects("github.com/foo"), "malformed pattern selects nothing")
}
func Test_symbolSelector_filter(t *testing.T) {
symbols := func() map[string]map[string][]string {
return map[string]map[string][]string{
"golang.org/x/net": {"golang.org/x/net/http2": {"NewClientConn"}},
"github.com/klauspost/compress": {"github.com/klauspost/compress/zstd": {"NewReader"}},
}
}
tests := []struct {
name string
scope cataloging.SymbolScope
modules []string
want []string // remaining module paths
}{
{
name: "none drops everything",
scope: cataloging.SymbolScopeNone,
},
{
name: "stdlib drops every module",
scope: cataloging.SymbolScopeStdlib,
},
{
name: "extended-stdlib keeps only golang.org/x",
scope: cataloging.SymbolScopeExtendedStdlib,
want: []string{"golang.org/x/net"},
},
{
name: "module patterns widen the selection",
scope: cataloging.SymbolScopeExtendedStdlib,
modules: []string{"github.com/klauspost/**"},
want: []string{"golang.org/x/net", "github.com/klauspost/compress"},
},
{
name: "all keeps everything",
scope: cataloging.SymbolScopeAll,
want: []string{"golang.org/x/net", "github.com/klauspost/compress"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := newSymbolSelector(tt.scope, tt.modules).filter(symbols())
if len(tt.want) == 0 {
// nil, not an empty map, so the omitempty JSON tag keeps the field out of output
assert.Nil(t, got)
return
}
assert.ElementsMatch(t, tt.want, slices.Collect(maps.Keys(got)))
})
}
}
+4 -2
View File
@@ -30,8 +30,10 @@ type GolangBinaryBuildinfoEntry struct {
// name is the import path, a ".", and the local name. One exception: the binary's main package appears
// under the key "main" (the name the linker assigns), not its original source import path, which is not
// recoverable from the binary. Populated only when the golang cataloger's capture-symbols scope covers
// this package: the "all" scope populates every module package plus the synthetic stdlib package, while
// the "stdlib" scope populates only the stdlib package.
// this package: the "all" scope populates every module package plus the synthetic stdlib package, the
// "extended-stdlib" scope populates the stdlib package plus every module under golang.org/x/, and the
// "stdlib" scope populates only the stdlib package. The capture-symbols-modules glob patterns populate
// any additional modules they match.
Symbols map[string][]string `json:"symbols,omitempty"`
}