feat(golang): add extended-stdlib scope and module patterns for symbol capture (#5154)

* feat(golang): add extended-stdlib scope and include patterns for symbol capture

`golang.capture-symbols` decides how much symbol data lands in the SBOM for grype's reachability analysis. It's `none`, `stdlib`, or `all` today, and the useful middle is missing: `stdlib` stops at the standard library, `all` multiplies SBOM size.

A new `extended-stdlib` configurable covers stdlib plus everything under `golang.org/x/`:

```yaml
golang:
  capture-symbols: extended-stdlib
```

Also, a new `capture-symbols-include` configurable for modules that are noisy in your binaries but not everyone's. It's unioned with whatever the scope selects, so it only ever widens:

```yaml
golang:
  capture-symbols: extended-stdlib
  capture-symbols-include:
    - github.com/klauspost/**
```

Patterns are standard doublestar globs, which matters because module paths carry `/v2`-style suffixes:

```yaml
golang:
  capture-symbols-include:
    - github.com/klauspost/*     # compress, but not compress/v2
    - github.com/klauspost/**    # both
    - k8s.io/client-go           # exact match only
```

Ordering is `none` < `stdlib` < `extended-stdlib` < `all`. The existing three values
and the `none` default are unchanged, and the include list is inert under `none`.
Presets compile into glob lists internally, so a single matcher answers "does this
module get symbols" instead of a preset branch sitting next to a separate glob branch.

An unrecognized `capture-symbols` value still falls back to `none`, but warns now
instead of doing it silently. A malformed include pattern warns and gets skipped.

One thing worth a look beyond the feature: the `Symbols` field description in the JSON
schema was wrong after this (it claimed only `all` and `stdlib` populate anything), and
that description lives in the already-published `16.1.10`. Rather than bump a version for
a sentence, `16.1.10` is amended in place and `schema/json/README.md` grows an explicit
exception for description-only changes: descriptions only, no shape change of any kind,
`$id` unchanged. Anything else still needs a bump. Happy to split that into its own PR if
you'd rather review the policy separately.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* refactor(golang): rename capture-symbols-include to capture-symbols-modules

The key's entries are go module paths, and `-include` sitting next to `capture-symbols` reads as plausibly taking symbol or package names instead. Those spellings parse and match nothing, which is quieter than the confusion `-include` was picked to avoid, so the name now says what the list holds.

`golang.CatalogerConfig.CaptureSymbolsModules` and `WithCaptureSymbolsModules` rename with it. Nothing behavioral changes; the key is new in this PR so there is no compatibility surface.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* feat(golang): match capture-symbols-modules across major version suffixes

`github.com/anchore/*` covered `github.com/anchore/syft` and silently stopped covering it the day it became `github.com/anchore/syft/v2`. The config keeps parsing, nothing warns, and symbols quietly go missing from the SBOM. Exact paths had the same hole: `github.com/klauspost/compress` did not cover `compress/v2` either, so no spelling short of `**` survived a major bump.

A major version suffix is part of a module's path but not part of its identity, so patterns are now matched against the module path both with and without it, using `module.SplitPathVersion` from `golang.org/x/mod` (already a direct dep, already used in this package for `PseudoVersion`).

```yaml
golang:
  capture-symbols-modules:
    - github.com/klauspost/*          # compress and compress/v2
    - github.com/klauspost/compress   # same module at every major version
    - github.com/klauspost/compress/v2  # v2 alone
```

Only a trailing suffix is a version, which is Go's own rule. In `github.com/anchore/syft/v2/thing` the `v2` is an ordinary path element naming a major subdirectory a nested module lives in, so it stays literal and `github.com/anchore/**/thing` is how you reach it. `/v0` and `/v1` are not valid suffixes and are left alone.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

cmd/syft/internal/options/catalog.go

@@ -199,7 +199,8 @@ func (cfg Catalog) ToPackagesConfig() pkgcataloging.Config {
 					WithFromLDFlags(cfg.Golang.MainModuleVersion.FromLDFlags),
 			).
 			WithUsePackagesLib(*multiLevelOption(true, enrichmentEnabled(cfg.Enrich, task.Go, task.Golang), cfg.Golang.UsePackagesLib)).
-			WithCaptureSymbols(cfg.Golang.CaptureSymbols),
+			WithCaptureSymbols(cfg.Golang.CaptureSymbols).
+			WithCaptureSymbolsModules(cfg.Golang.CaptureSymbolsModules),
 		JavaScript: javascript.DefaultCatalogerConfig().
 			WithIncludeDevDependencies(*multiLevelOption(false, cfg.JavaScript.IncludeDevDependencies)).
 			WithSearchRemoteLicenses(*multiLevelOption(false, enrichmentEnabled(cfg.Enrich, task.JavaScript, task.Node, task.NPM), cfg.JavaScript.SearchRemoteLicenses)).

cmd/syft/internal/options/golang.go

@@ -4,6 +4,7 @@ import (
 	"strings"
 
 	"github.com/anchore/clio"
+	"github.com/anchore/syft/internal/log"
 	"github.com/anchore/syft/syft/cataloging"
 	"github.com/anchore/syft/syft/pkg/cataloger/golang"
 )
@@ -19,6 +20,7 @@ type golangConfig struct {
 	MainModuleVersion           golangMainModuleVersionConfig `json:"main-module-version" yaml:"main-module-version" mapstructure:"main-module-version"`
 	UsePackagesLib              *bool                         `json:"use-packages-lib" yaml:"use-packages-lib" mapstructure:"use-packages-lib"`
 	CaptureSymbols              cataloging.SymbolScope        `json:"capture-symbols" yaml:"capture-symbols" mapstructure:"capture-symbols"`
+	CaptureSymbolsModules       []string                      `json:"capture-symbols-modules" yaml:"capture-symbols-modules" mapstructure:"capture-symbols-modules"`
 }
 
 var _ interface {
@@ -42,8 +44,16 @@ if unset this defaults to $GONOPROXY`)
 always show (devel) as the version. Use these options to control heuristics to guess
 a more accurate version from the binary.`)
 	descriptions.Add(&o.UsePackagesLib, `use the golang.org/x/tools/go/packages library, which executes golang tooling found on the path in addition to potential network access to get the most accurate results`)
+	// note: descriptions must be static string literals; the app config discovery that generates the
+	// capability docs reads them straight out of the AST
 	descriptions.Add(&o.CaptureSymbols, `capture function symbols from the binary symbol table (pclntab). valid values are:
-"none" (disabled), "stdlib" (only the synthetic stdlib package), and "all" (all module packages plus stdlib)`)
+"none" (disabled), "stdlib" (only the synthetic stdlib package), "extended-stdlib" (stdlib plus every
+module under golang.org/x/), and "all" (all module packages plus stdlib)`)
+	descriptions.Add(&o.CaptureSymbolsModules, `glob patterns matched against go module paths (e.g. github.com/klauspost/**) that should have symbols
+captured in addition to whatever capture-symbols selects. ** crosses path separators, * does not.
+a trailing major version suffix is ignored when matching, so github.com/foo/* covers github.com/foo/bar/v2;
+spelling a suffix out in the pattern selects only that major version.
+this can only widen the selection, never narrow it, and is inert when capture-symbols is none`)
 	descriptions.Add(&o.MainModuleVersion.FromLDFlags, `look for LD flags that appear to be setting a version (e.g. -X main.version=1.0.0)`)
 	descriptions.Add(&o.MainModuleVersion.FromBuildSettings, `use the build settings (e.g. vcs.version & vcs.time) to craft a v0 pseudo version
 (e.g. v0.0.0-20220308212642-53e6d0aaf6fb) when a more accurate version cannot be found otherwise`)
@@ -51,7 +61,24 @@ a more accurate version from the binary.`)
 }
 
 func (o *golangConfig) PostLoad() error {
+	raw := strings.TrimSpace(string(o.CaptureSymbols))
 	o.CaptureSymbols = o.CaptureSymbols.Parse()
+
+	// an unrecognized value still resolves to "none", but say so rather than silently capturing nothing.
+	// stay quiet for unset and an explicit "none", which is the default and would otherwise warn on nearly
+	// every scan.
+	if o.CaptureSymbols == cataloging.SymbolScopeNone && raw != "" && !strings.EqualFold(raw, string(cataloging.SymbolScopeNone)) {
+		log.Warnf("unknown golang.capture-symbols value %q, defaulting to %q (valid values: none, stdlib, extended-stdlib, all)", raw, cataloging.SymbolScopeNone)
+	}
+
+	// trim only, deliberately not Flatten: viper already splits a comma-separated scalar (env var or a bare
+	// yaml string) into a slice before this point, and there is no CLI flag feeding this key. The one thing
+	// Flatten would add is splitting commas *inside* a list entry, which silently breaks doublestar brace
+	// alternation like github.com/{foo,bar}/**. Viper's split does not trim, so that part is still needed.
+	for i, pattern := range o.CaptureSymbolsModules {
+		o.CaptureSymbolsModules[i] = strings.TrimSpace(pattern)
+	}
+
 	return nil
 }
 
@@ -76,7 +103,8 @@ func defaultGolangConfig() golangConfig {
 			FromContents:      def.MainModuleVersion.FromContents,
 			FromBuildSettings: def.MainModuleVersion.FromBuildSettings,
 		},
-		UsePackagesLib: nil, // this defaults to true, which is the API default
-		CaptureSymbols: def.CaptureSymbols,
+		UsePackagesLib:        nil, // this defaults to true, which is the API default
+		CaptureSymbols:        def.CaptureSymbols,
+		CaptureSymbolsModules: def.CaptureSymbolsModules,
 	}
 }

cmd/syft/internal/options/golang_test.go

@@ -10,10 +10,11 @@ import (
 
 func Test_golangConfig_PostLoad(t *testing.T) {
 	tests := []struct {
-		name     string
-		cfg      golangConfig
-		expected cataloging.SymbolScope
-		wantErr  assert.ErrorAssertionFunc
+		name            string
+		cfg             golangConfig
+		expected        cataloging.SymbolScope
+		expectedModules []string
+		wantErr         assert.ErrorAssertionFunc
 	}{
 		{
 			name:     "normalize all",
@@ -25,6 +26,32 @@ func Test_golangConfig_PostLoad(t *testing.T) {
 			cfg:      golangConfig{CaptureSymbols: "stdlib"},
 			expected: cataloging.SymbolScopeStdlib,
 		},
+		{
+			name:     "normalize extended-stdlib",
+			cfg:      golangConfig{CaptureSymbols: " Extended-Stdlib "},
+			expected: cataloging.SymbolScopeExtendedStdlib,
+		},
+		{
+			name: "module patterns keep embedded commas",
+			cfg: golangConfig{
+				CaptureSymbols: "stdlib",
+				// brace alternation contains a comma; splitting on it would corrupt the pattern
+				CaptureSymbolsModules: []string{"github.com/{foo,bar}/**", "golang.org/x/**"},
+			},
+			expected:        cataloging.SymbolScopeStdlib,
+			expectedModules: []string{"github.com/{foo,bar}/**", "golang.org/x/**"},
+		},
+		{
+			// viper splits a comma-separated scalar (env var or bare yaml string) but does not trim,
+			// so a leading space would otherwise survive into a pattern that silently matches nothing
+			name: "module patterns are trimmed",
+			cfg: golangConfig{
+				CaptureSymbols:        "stdlib",
+				CaptureSymbolsModules: []string{"golang.org/x/**", " github.com/foo/** "},
+			},
+			expected:        cataloging.SymbolScopeStdlib,
+			expectedModules: []string{"golang.org/x/**", "github.com/foo/**"},
+		},
 		{
 			name:     "empty defaults to none",
 			cfg:      golangConfig{CaptureSymbols: ""},
@@ -32,7 +59,7 @@ func Test_golangConfig_PostLoad(t *testing.T) {
 		},
 		{
 			name:     "invalid value defaults to none",
-			cfg:      golangConfig{CaptureSymbols: "bogus"},
+			cfg:      golangConfig{CaptureSymbols: "stdlbi"},
 			expected: cataloging.SymbolScopeNone,
 		},
 		{
@@ -40,6 +67,16 @@ func Test_golangConfig_PostLoad(t *testing.T) {
 			cfg:      golangConfig{CaptureSymbols: "true"},
 			expected: cataloging.SymbolScopeNone,
 		},
+		{
+			name:     "explicit none resolves to none",
+			cfg:      golangConfig{CaptureSymbols: "none"},
+			expected: cataloging.SymbolScopeNone,
+		},
+		{
+			name:     "explicit none is not case sensitive",
+			cfg:      golangConfig{CaptureSymbols: " NONE "},
+			expected: cataloging.SymbolScopeNone,
+		},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
@@ -52,6 +89,7 @@ func Test_golangConfig_PostLoad(t *testing.T) {
 				return
 			}
 			assert.Equal(t, tt.expected, tt.cfg.CaptureSymbols)
+			assert.Equal(t, tt.expectedModules, tt.cfg.CaptureSymbolsModules)
 		})
 	}
 }