Harden cataloger parsers against truncated input (do not panic) (#5355)

* fix(javascript): don't panic on a yarn v1 lockfile line missing its value

A v1 `yarn.lock` dependency line without a version constraint, or a
`version`/`resolved`/`integrity` line without a value, indexed past the end of
the split line. Such dependencies are now skipped and empty fields left empty.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(alpine): don't panic on an apk db ACL or checksum line before its entry

An `M:`, `a:` or `Z:` line that comes before any directory or file entry in
`lib/apk/db/installed` indexed `files[-1]`. Such a line is now skipped.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(erlang): don't panic on a truncated rebar.lock or .app file

The term parser read past the end of the input when a file ended mid-term:
whitespace only, an unclosed list, a lone `<`, or a comment ending in `\r`.
These now return an "unexpected end of input" error.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(lua): don't panic on a truncated .rockspec

The rockspec parser read past the end of the input when a file ended mid-statement:
a lone `[`, an unclosed `[[...]]` value or `[...]` index, or a `local` assignment cut off
before its `=`, value, or line end. These now return an "unexpected end of input" error.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(swift): don't panic on a Podfile.lock pod without a version

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(gentoo): don't panic on a short obj line in portage CONTENTS

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix(spdx): don't panic on an opam checksum without an algorithm

The opam cataloger stores checksums verbatim, so a malformed one reached the SPDX encoder and panicked there, outside the cataloger's recover. Checksums without an `=` are now skipped.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* refactor: use strings.Cut where a split was only indexed after a Contains check

Also guards `windows.FromPosix` against a path with no separator.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: guard untyped package metadata fields

- new `any`-typed metadata fields must be allowlisted with a reason
- allowlisted untyped fields are pushed through `SetID` and every encoder with
  non-finite, deeply nested and large values

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: fuzz every generic cataloger parser

One fuzz target over every glob-selected parser in the capabilities data, run
through its cataloger task. Each input must not panic, must finish in time and
must keep allocations proportional to its size. Seeds come from each parser's
testdata plus the string literals in its package.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: scan malformed inputs under memory and time limits

A directory of small malformed files (deep nesting, oversized declared counts,
truncation, type mismatches, non-finite floats) is scanned by the CLI with a
memory limit and an RSS ceiling. The scan must exit 0 and write an SBOM.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: stop a fuzz input once its live heap passes the allocation limit

The allocation check only ran after the parser returned, so an input on its way to exhausting memory ran the machine into swap first. The harness now polls the live heap while the parser runs and panics past the same limit, which the fuzz engine saves as a crasher.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: bound nesting depth and error context in the erlang and lua parsers

- a deeply nested rebar.lock or .rockspec overflowed the goroutine stack, which is fatal rather than a recoverable panic, so one file could take down the whole scan. Both parsers now stop at 1000 levels with an error.
- parse errors echoed whole input lines into the SBOM as unknowns, and lua appended another copy at every nesting level. `PrintError` now clips to a window around the error column, and lua decorates once at the top.
- the Podfile.lock error no longer echoes the pod entry.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: close ways the malformed-input checks could pass quietly

- a ctx-aware parser hitting the fuzz timeout now fails the input instead of racing the hard deadline
- the heap watchdog measures growth from a post-GC baseline rather than an absolute process-wide number
- a saved fuzz input naming a target that no longer exists fails instead of being skipped
- the CLI malformed-input scan fails on a recovered parser panic, not only on a non-zero exit

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
This commit is contained in:
Alex Goodman
2026-09-29 16:26:42 -04:00
committed by GitHub
parent 57bd6a5f9c
commit d0e1e7be05
40 changed files with 1060 additions and 99 deletions
@@ -0,0 +1,74 @@
package packagemetadata
import (
"reflect"
"strings"
"testing"
)
// untypedFieldAllowlist holds package metadata fields that are allowed to carry untyped (interface) values.
// Values in these fields come straight from whatever a parser decoded, so they are not shaped by syft
// and can hold anything an encoder or hasher chokes on. Every entry is part of existing SBOM output and
// cannot be retyped without breaking it; new entries need a reason and a review.
var untypedFieldAllowlist = map[string]string{
"GGUFFileHeader.RemainingKeyValues": "raw GGUF header key-values; retyping changes the header output and metadataHash",
}
// TestNoUntypedMetadataFields fails when a package metadata type gains a field typed any, []any,
// map[string]any (or any other interface) that is not on the allowlist.
func TestNoUntypedMetadataFields(t *testing.T) {
var hits []string
for _, ty := range AllTypes() {
rt := reflect.TypeOf(ty)
collectUntyped(rt, rt.Name(), map[reflect.Type]bool{}, &hits)
}
seen := map[string]bool{}
for _, h := range hits {
seen[h] = true
if _, ok := untypedFieldAllowlist[h]; !ok {
t.Errorf("untyped field %s: use a concrete type, or add it to untypedFieldAllowlist with a reason", h)
}
}
for k := range untypedFieldAllowlist {
if !seen[k] {
t.Errorf("stale untypedFieldAllowlist entry %s: field no longer exists or is now typed", k)
}
}
}
// collectUntyped records "Struct.Field" for every field whose type reaches an interface, descending
// through pointers, slices, arrays, maps and nested syft-owned structs.
func collectUntyped(rt reflect.Type, path string, visited map[reflect.Type]bool, hits *[]string) {
switch rt.Kind() {
case reflect.Interface:
*hits = append(*hits, path)
case reflect.Pointer, reflect.Slice, reflect.Array:
collectUntyped(rt.Elem(), path, visited, hits)
case reflect.Map:
collectUntyped(rt.Key(), path, visited, hits)
collectUntyped(rt.Elem(), path, visited, hits)
case reflect.Struct:
if visited[rt] {
return
}
if rt.Name() != "" && !strings.HasPrefix(rt.PkgPath(), "github.com/anchore/syft/") {
// a third-party type in the model carries whatever shape the library gives it (the same leak as an
// interface field), so it counts as untyped. The standard library is left alone.
if strings.Contains(strings.Split(rt.PkgPath(), "/")[0], ".") {
*hits = append(*hits, path)
}
return
}
visited[rt] = true
for i := 0; i < rt.NumField(); i++ {
f := rt.Field(i)
// unexported embedded structs still promote their exported fields to the encoders and the hasher. A
// field is only out of reach when it is neither encoded nor hashed by SetID.
if (!f.IsExported() && !f.Anonymous) || (f.Tag.Get("json") == "-" && f.Tag.Get("hash") == "ignore") {
continue
}
collectUntyped(f.Type, rt.Name()+"."+f.Name, visited, hits)
}
}
}
@@ -581,10 +581,13 @@ func toPackageChecksums(p pkg.Package) ([]spdx.Checksum, bool) {
})
case pkg.OpamPackage:
for _, checksum := range meta.Checksums {
parts := strings.Split(checksum, "=")
algorithm, value, ok := strings.Cut(checksum, "=")
if !ok {
continue
}
checksums = append(checksums, spdx.Checksum{
Algorithm: spdx.ChecksumAlgorithm(strings.ToUpper(parts[0])),
Value: parts[1],
Algorithm: spdx.ChecksumAlgorithm(strings.ToUpper(algorithm)),
Value: value,
})
}
}
@@ -516,6 +516,20 @@ func Test_toPackageChecksums(t *testing.T) {
},
filesAnalyzed: false,
},
{
name: "Opam Package with a checksum missing its algorithm separator",
pkg: pkg.Package{
Name: "test",
Version: "1.0.0",
Metadata: pkg.OpamPackage{
Checksums: []string{"f5f1c0b4", "md5=abc"},
},
},
expected: []spdx.Checksum{
{Algorithm: "MD5", Value: "abc"},
},
filesAnalyzed: false,
},
{
name: "Package with no metadata type",
pkg: pkg.Package{
+94
View File
@@ -0,0 +1,94 @@
package format
import (
"bytes"
"fmt"
"math"
"testing"
"github.com/anchore/syft/syft/format/syftjson"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
)
// TestEncodersHandleUntypedMetadataValues pushes awkward values through the untyped metadata fields
// allowlisted in internal/packagemetadata (see untypedFieldAllowlist) and asserts that SetID and every
// encoder neither panic nor fail. A new allowlist entry should get a case here too.
func TestEncodersHandleUntypedMetadataValues(t *testing.T) {
cases := []struct {
name string
value any
// encoding/json rejects non-finite floats, so syft-json returns an error for them. Producers
// have to replace these before they reach the model (the GGUF cataloger does); the case stays
// here so the expectation flips visibly if syft-json ever learns to encode them.
syftJSONRejects bool
}{
{name: "NaN", value: math.NaN(), syftJSONRejects: true},
{name: "+Inf float32", value: float32(math.Inf(1)), syftJSONRejects: true},
{name: "-Inf", value: math.Inf(-1), syftJSONRejects: true},
{name: "NaN in array", value: []any{1.0, math.NaN()}, syftJSONRejects: true},
{name: "invalid utf-8", value: "\xff\xfe"},
{name: "NUL and control characters", value: "a\x00b\x1b"},
{name: "max uint64", value: uint64(math.MaxUint64)},
{name: "deep array", value: nested(2000, func(v any) any { return []any{v} })},
{name: "deep map", value: nested(2000, func(v any) any { return map[string]any{"k": v} })},
{name: "large array", value: make([]any, 100_000)},
}
for _, tt := range cases {
t.Run(tt.name, func(t *testing.T) {
p := pkg.Package{
Name: "model",
Version: "1.0",
Type: pkg.ModelPkg,
Metadata: pkg.GGUFFileHeader{
RemainingKeyValues: map[string]any{"general.value": tt.value},
},
}
if !noPanic(t, "SetID", p.SetID) {
return
}
s := sbom.SBOM{
Artifacts: sbom.Artifacts{Packages: pkg.NewCollection(p)},
Source: source.Description{Metadata: source.DirectoryMetadata{Path: "/models"}},
}
for _, enc := range Encoders() {
id := fmt.Sprintf("%s@%s", enc.ID(), enc.Version())
var buf bytes.Buffer
noPanic(t, id, func() {
err := enc.Encode(&buf, s)
switch {
case tt.syftJSONRejects && enc.ID() == syftjson.ID:
if err == nil {
t.Errorf("%s: expected an encode error for a non-finite float", id)
}
case err != nil:
t.Errorf("%s: encode failed: %v", id, err)
}
})
}
})
}
}
func nested(depth int, wrap func(any) any) any {
var v any = "leaf"
for range depth {
v = wrap(v)
}
return v
}
func noPanic(t *testing.T, what string, fn func()) (ok bool) {
t.Helper()
defer func() {
if r := recover(); r != nil {
t.Errorf("%s: panic: %v", what, r)
ok = false
}
}()
fn()
return true
}
+24 -7
View File
@@ -22,6 +22,15 @@ func SkipWhitespace(data []byte, i *int) {
}
}
// MaxDepth bounds nesting in the recursive hand-written parsers. Real files nest a handful of levels; without a
// bound, deep input overflows the goroutine stack, which is fatal rather than a recoverable panic.
const MaxDepth = 1000
// errorContext is how many bytes of each echoed line PrintError keeps around the error column, so an error in a
// huge single-line input stays small (and does not copy the file into the SBOM as an unknown).
const errorContext = 80
// PrintError describes where in data offset i is, echoing up to two lines of context clipped around the column.
func PrintError(data []byte, i int) string {
line := 1
char := 1
@@ -57,15 +66,23 @@ func PrintError(data []byte, i int) string {
sep := ": "
lines := ""
if len(prev) > 1 {
lines += fmt.Sprintf("%s%s%s\n", l1, sep, prev[len(prev)-2])
}
if len(prev) > 0 {
lines += fmt.Sprintf("%s%s%s\n", l2, sep, prev[len(prev)-1])
start := max(0, char-1-errorContext/2)
clip := func(s string) string {
if start >= len(s) {
return ""
}
return s[start:min(len(s), start+errorContext)]
}
pointer := strings.Repeat(" ", len(l2)+len(sep)+char-1) + "^"
lines := ""
if len(prev) > 1 {
lines += fmt.Sprintf("%s%s%s\n", l1, sep, clip(prev[len(prev)-2]))
}
if len(prev) > 0 {
lines += fmt.Sprintf("%s%s%s\n", l2, sep, clip(prev[len(prev)-1]))
}
pointer := strings.Repeat(" ", len(l2)+len(sep)+char-1-start) + "^"
return fmt.Sprintf("line: %v, char: %v\n%s%s", line, char, lines, pointer)
}
+4
View File
@@ -36,6 +36,10 @@ func FromPosix(posixPath string) (windowsPath string) {
// decode the volume (e.g. /c/<path> --> C:\\) - There should always be a volume name.
// The volume may be a UNC path (e.g. /\\localhost\C$\ --> \\localhost\C$\)
pathFields := strings.Split(posixPath, "/")
if len(pathFields) < 2 {
// no separator means no volume to decode
return filepath.Clean(posixPath)
}
rootPath := strings.ToUpper(pathFields[1])
volumeName := AppendRootTerminator(rootPath)
+2 -2
View File
@@ -156,8 +156,8 @@ var busyboxVersionMatcher = regexp.MustCompile(`BusyBox v[\d.]+`)
func parseBusyBox(contents string) (*Release, error) {
matches := busyboxVersionMatcher.FindAllString(contents, -1)
for _, match := range matches {
parts := strings.Split(match, " ")
version := strings.ReplaceAll(parts[1], "v", "")
_, version, _ := strings.Cut(match, " ")
version = strings.ReplaceAll(version, "v", "")
return simpleRelease(match, "busybox", version, ""), nil
}
+12
View File
@@ -270,6 +270,10 @@ func (f apkField) apply(p *parsedData, ctx *apkFileParsingContext) {
ctx.indexOfLatestDirectory = len(ctx.files) - 1
case "M":
i := ctx.indexOfLatestDirectory
if i < 0 {
log.Debugf("APK field %q appears before any directory entry", f.name)
return
}
latest := ctx.files[i]
var ok bool
@@ -296,6 +300,10 @@ func (f apkField) apply(p *parsedData, ctx *apkFileParsingContext) {
ctx.indexOfLatestRegularFile = len(ctx.files) - 1
case "a":
i := ctx.indexOfLatestRegularFile
if i < 0 {
log.Debugf("APK field %q appears before any file entry", f.name)
return
}
latest := ctx.files[i]
var ok bool
@@ -309,6 +317,10 @@ func (f apkField) apply(p *parsedData, ctx *apkFileParsingContext) {
ctx.files[i] = latest
case "Z":
i := ctx.indexOfLatestRegularFile
if i < 0 {
log.Debugf("APK field %q appears before any file entry", f.name)
return
}
latest := ctx.files[i]
latest.Digest = processChecksum(f.value)
@@ -869,3 +869,29 @@ https://foo.them.org/alpine/v3.14/community`,
})
}
}
func TestParseApkDB_attributeBeforeItsEntry(t *testing.T) {
// ACL (M:, a:) and checksum (Z:) lines describe the most recent directory or file entry, and used to index
// files[-1] when no entry had been seen yet
tests := []struct {
name string
line string
}{
{name: "directory ACL before any directory", line: "M:0:0:755"},
{name: "file ACL before any file", line: "a:0:0:755"},
{name: "checksum before any file", line: "Z:Q1Kz6TXnPVJeDQUfO0IJVxD3HnYmU="},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
contents := strings.Join([]string{"P:foo", "V:1.0-r0", tt.line, ""}, "\n")
reader := file.NewLocationReadCloser(file.NewLocation("installed"), io.NopCloser(strings.NewReader(contents)))
pkgs, _, err := parseApkDB(context.Background(), nil, new(generic.Environment), reader)
require.NoError(t, err)
require.Len(t, pkgs, 1)
assert.Equal(t, "foo", pkgs[0].Name)
assert.Empty(t, pkgs[0].Metadata.(pkg.ApkDBEntry).Files, "an orphaned attribute line should not invent a file")
})
}
}
@@ -102,10 +102,10 @@ func (c Application) Validate() error {
}
func (c Application) GetBinaryFromImage(name, version string) *BinaryFromImage {
if strings.Contains(name, "@") && version == "" {
parts := strings.Split(name, "@")
name = parts[0]
version = parts[1]
if version == "" {
if n, v, ok := strings.Cut(name, "@"); ok {
name, version = n, v
}
}
for _, entry := range c.FromImages {
if entry.Name() == name && entry.Version == version {
+25 -6
View File
@@ -15,6 +15,10 @@ type erlangNode struct {
var errSkipComments = errors.New("")
var errUnexpectedEnd = errors.New("unexpected end of input")
var errTooDeep = fmt.Errorf("nesting deeper than %d levels", parsing.MaxDepth)
func (e erlangNode) Slice() []erlangNode {
out, ok := e.value.([]erlangNode)
if ok {
@@ -79,7 +83,7 @@ func parseErlang(reader io.Reader) (erlangNode, error) {
}
func parseErlangBlock(data []byte, i *int) (erlangNode, error) {
block, err := parseErlangNode(data, i)
block, err := parseErlangNode(data, i, 0)
if err != nil {
return node(nil), err
}
@@ -89,13 +93,19 @@ func parseErlangBlock(data []byte, i *int) (erlangNode, error) {
return block, nil
}
func parseErlangNode(data []byte, i *int) (erlangNode, error) {
func parseErlangNode(data []byte, i *int, depth int) (erlangNode, error) {
parsing.SkipWhitespace(data, i)
if *i >= len(data) {
return node(nil), errUnexpectedEnd
}
c := data[*i]
switch c {
case '[', '{':
offset := *i + 1
parsing.SkipWhitespace(data, &offset)
if offset >= len(data) {
return node(nil), errUnexpectedEnd
}
c2 := data[offset]
// Add support for empty lists
@@ -104,7 +114,7 @@ func parseErlangNode(data []byte, i *int) (erlangNode, error) {
return node(nil), nil
}
return parseErlangList(data, i)
return parseErlangList(data, i, depth+1)
case '"':
fallthrough
case '\'':
@@ -145,6 +155,9 @@ func parseErlangAngleString(data []byte, i *int) (erlangNode, error) {
}
func parseErlangString(data []byte, i *int) (erlangNode, error) {
if *i >= len(data) {
return node(nil), errUnexpectedEnd
}
delim := data[*i]
*i++
var buf bytes.Buffer
@@ -167,13 +180,16 @@ func parseErlangString(data []byte, i *int) (erlangNode, error) {
return node(nil), fmt.Errorf("unterminated string at %d", *i)
}
func parseErlangList(data []byte, i *int) (erlangNode, error) {
func parseErlangList(data []byte, i *int, depth int) (erlangNode, error) {
if depth > parsing.MaxDepth {
return node(nil), errTooDeep
}
*i++
out := erlangNode{
value: []erlangNode{},
}
for *i < len(data) {
item, err := parseErlangNode(data, i)
item, err := parseErlangNode(data, i, depth)
if err != nil {
if err == errSkipComments {
parsing.SkipWhitespace(data, i)
@@ -183,6 +199,9 @@ func parseErlangList(data []byte, i *int) (erlangNode, error) {
}
out.value = append(out.value.([]erlangNode), item)
parsing.SkipWhitespace(data, i)
if *i >= len(data) {
return node(nil), errUnexpectedEnd
}
c := data[*i]
switch c {
case ',':
@@ -210,7 +229,7 @@ func parseErlangComment(data []byte, i *int) {
// Rest of a line is a comment. Deals with CR, LF and CR/LF
if c == '\n' {
break
} else if c == '\r' && data[*i] == '\n' {
} else if c == '\r' && *i < len(data) && data[*i] == '\n' {
*i++
break
}
@@ -2,6 +2,7 @@ package erlang
import (
"bytes"
"strings"
"testing"
"github.com/stretchr/testify/assert"
@@ -128,3 +129,35 @@ func Test_parseErlangString_escapedByte(t *testing.T) {
require.NoError(t, err)
assert.Equal(t, `a"b`, got.String())
}
func TestParseErlang_deepNesting(t *testing.T) {
// 4MB of '[' overflowed the goroutine stack (fatal, not recoverable) before nesting was bounded
_, err := parseErlang(strings.NewReader(strings.Repeat("[a,", 4<<20)))
require.ErrorIs(t, err, errTooDeep)
// the error echoes a clipped window of the single-line input, not the whole thing
assert.Less(t, len(err.Error()), 1024)
}
func Test_parseErlang_truncated(t *testing.T) {
// each of these read past the end of the input before end-of-input was checked
tests := []struct {
name string
content string
wantErr require.ErrorAssertionFunc
}{
{name: "whitespace only", content: " ", wantErr: require.Error},
{name: "unterminated list", content: "[a", wantErr: require.Error},
{name: "unterminated empty list", content: "[ ", wantErr: require.Error},
{name: "lone binary string opener", content: "<", wantErr: require.Error},
{name: "comment ending in CR", content: "%\r", wantErr: require.NoError},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var err error
require.NotPanics(t, func() {
_, err = parseErlang(strings.NewReader(tt.content))
})
tt.wantErr(t, err)
})
}
}
@@ -3,6 +3,8 @@ package gentoo
import (
"testing"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
@@ -170,3 +172,16 @@ func TestCataloger_Globs(t *testing.T) {
})
}
}
func TestPortageCataloger_ShortObjLine(t *testing.T) {
// obj lines missing their path or digest are skipped rather than indexed past their end
pkgtest.NewCatalogTester().
FromDirectory(t, "testdata/short-obj-line").
ExpectsAssertion(func(t *testing.T, pkgs []pkg.Package, _ []artifact.Relationship) {
require.Len(t, pkgs, 1)
m := pkgs[0].Metadata.(pkg.PortageEntry)
require.Len(t, m.Files, 1)
require.Equal(t, "/usr/bin/bar", m.Files[0].Path)
}).
TestCataloger(t, NewPortageCataloger())
}
@@ -75,7 +75,8 @@ func addFiles(resolver file.Resolver, dbLocation file.Location, entry *pkg.Porta
line := strings.Trim(scanner.Text(), "\n")
fields := strings.Split(line, " ")
if fields[0] == "obj" {
// an obj line is "obj <path> <md5> <mtime>"
if len(fields) >= 3 && fields[0] == "obj" {
record := pkg.PortageFileRecord{
Path: fields[1],
}
@@ -0,0 +1,4 @@
dir /usr
obj
obj /usr/bin/foo
obj /usr/bin/bar abc123 1600000000
@@ -206,28 +206,16 @@ func parseYarnV1LockFile(reader io.ReadCloser) ([]yarnPackage, error) {
continue
}
if strings.HasPrefix(line, " ") && !strings.HasPrefix(line, " ") {
line = strings.Trim(line, " ")
array := strings.Split(line, " ")
switch array[0] {
case "version":
pkg.Version = strings.Trim(array[1], "\"")
case "resolved":
name, version, resolved := findResolvedPackageAndVersion(line)
if name != "" && version != "" && resolved != "" {
pkg.Name = name
pkg.Version = version
pkg.Resolved = resolved
} else {
pkg.Resolved = strings.Trim(array[1], "\"")
}
case "integrity":
pkg.Integrity = strings.Trim(array[1], "\"")
}
applyYarnV1Field(&pkg, strings.Trim(line, " "))
continue
}
if strings.HasPrefix(line, " ") {
line = strings.Trim(line, " ")
array := strings.Split(line, " ")
if len(array) < 2 {
// a dependency without a version constraint is malformed; skip it rather than guess
continue
}
dependencyName := strings.Trim(array[0], "\"")
dependencyVersion := strings.Trim(array[1], "\"")
dependencies[dependencyName] = dependencyVersion
@@ -244,6 +232,31 @@ func parseYarnV1LockFile(reader io.ReadCloser) ([]yarnPackage, error) {
return pkgs, nil
}
// applyYarnV1Field sets the package field described by a single (already trimmed) yarn v1 field line.
func applyYarnV1Field(pkg *yarnPackage, line string) {
array := strings.Split(line, " ")
// a field line may be missing its value in a malformed lockfile
var value string
if len(array) > 1 {
value = strings.Trim(array[1], "\"")
}
switch array[0] {
case "version":
pkg.Version = value
case "resolved":
name, version, resolved := findResolvedPackageAndVersion(line)
if name != "" && version != "" && resolved != "" {
pkg.Name = name
pkg.Version = version
pkg.Resolved = resolved
} else {
pkg.Resolved = value
}
case "integrity":
pkg.Integrity = value
}
}
func parseYarnLockYaml(reader io.ReadCloser) ([]yarnPackage, error) {
var lockfile = map[string]yarnV2PackageEntry{}
if err := yaml.NewDecoder(reader, yaml.AllowDuplicateMapKey()).Decode(&lockfile); err != nil {
@@ -6,9 +6,11 @@ import (
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
@@ -1091,3 +1093,35 @@ func setupYarnRegistry() (mux *http.ServeMux, serverURL string, teardown func())
return mux, server.URL, server.Close
}
func TestParseYarnV1LockFile_missingValues(t *testing.T) {
tests := []struct {
name string
content string
want []yarnPackage
}{
{
// a dependency line without a constraint is dropped rather than indexed past its end
name: "dependency without version",
content: "# yarn lockfile v1\n\na@1:\n version \"1\"\n dependencies:\n b\n\n",
want: []yarnPackage{{Name: "a", Version: "1", Dependencies: map[string]string{}}},
},
{
// a field line without a value leaves the field empty
name: "field without value",
content: "# yarn lockfile v1\n\na@1:\n version\n\n",
want: []yarnPackage{{Name: "a", Dependencies: map[string]string{}}},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var got []yarnPackage
var err error
require.NotPanics(t, func() {
got, err = parseYarnV1LockFile(io.NopCloser(strings.NewReader(tt.content)))
})
require.NoError(t, err)
assert.Equal(t, tt.want, got)
})
}
}
+76 -42
View File
@@ -2,6 +2,7 @@ package lua
import (
"bytes"
"errors"
"fmt"
"io"
@@ -33,6 +34,10 @@ func (r rockspecNode) String() string {
return ""
}
var errUnexpectedEnd = errors.New("unexpected end of input")
var errTooDeep = fmt.Errorf("nesting deeper than %d levels", parsing.MaxDepth)
var noReturn = rockspec{
value: nil,
}
@@ -46,10 +51,11 @@ func parseRockspecData(reader io.Reader) (rockspec, error) {
i := 0
locals := make(map[string]string)
blocks, err := parseRockspecBlock(data, &i, locals)
blocks, err := parseRockspecBlock(data, &i, locals, 0)
if err != nil {
return noReturn, err
// decorate once here rather than at every nesting level, which copied the input once per level
return noReturn, fmt.Errorf("%w\n%s", err, parsing.PrintError(data, i))
}
return rockspec{
@@ -57,9 +63,12 @@ func parseRockspecData(reader io.Reader) (rockspec, error) {
}, nil
}
func parseRockspecBlock(data []byte, i *int, locals map[string]string) ([]rockspecNode, error) {
func parseRockspecBlock(data []byte, i *int, locals map[string]string, depth int) ([]rockspecNode, error) {
if depth > parsing.MaxDepth {
return nil, errTooDeep
}
var out []rockspecNode
var iterator func(data []byte, i *int, locals map[string]string) (*rockspecNode, error)
var iterator func(data []byte, i *int, locals map[string]string, depth int) (*rockspecNode, error)
parsing.SkipWhitespace(data, i)
@@ -92,9 +101,9 @@ func parseRockspecBlock(data []byte, i *int, locals map[string]string) ([]rocksp
}
for *i < len(data) {
item, err := iterator(data, i, locals)
item, err := iterator(data, i, locals, depth)
if err != nil {
return nil, fmt.Errorf("%w\n%s", err, parsing.PrintError(data, *i))
return nil, err
}
parsing.SkipWhitespace(data, i)
@@ -114,7 +123,7 @@ func parseRockspecBlock(data []byte, i *int, locals map[string]string) ([]rocksp
}
//nolint:funlen, gocognit
func parseRockspecNode(data []byte, i *int, locals map[string]string) (*rockspecNode, error) {
func parseRockspecNode(data []byte, i *int, locals map[string]string, depth int) (*rockspecNode, error) {
parsing.SkipWhitespace(data, i)
if *i >= len(data) {
@@ -212,7 +221,7 @@ func parseRockspecNode(data []byte, i *int, locals map[string]string) (*rockspec
*i = offset
parsing.SkipWhitespace(data, i)
obj, err := parseRockspecBlock(data, i, locals)
obj, err := parseRockspecBlock(data, i, locals, depth+1)
if err != nil {
return nil, err
@@ -247,6 +256,9 @@ func parseRockspecNode(data []byte, i *int, locals map[string]string) (*rockspec
}
value := str.String()
if *i >= len(data) {
return nil, errUnexpectedEnd
}
c = data[*i]
if c != ']' {
@@ -271,7 +283,7 @@ func parseRockspecNode(data []byte, i *int, locals map[string]string) (*rockspec
}, nil
}
func parseRockspecListItem(data []byte, i *int, locals map[string]string) (*rockspecNode, error) {
func parseRockspecListItem(data []byte, i *int, locals map[string]string, _ int) (*rockspecNode, error) {
parsing.SkipWhitespace(data, i)
if *i >= len(data) {
@@ -378,6 +390,9 @@ out:
if err != nil {
return "", err
}
if *i >= len(data) {
return "", errUnexpectedEnd
}
c = data[*i]
if c != ']' {
return "", fmt.Errorf("unterminated literal at %d", *i)
@@ -394,6 +409,9 @@ out:
}
func parseRockspecString(data []byte, i *int, _ map[string]string) (*rockspecNode, error) {
if *i >= len(data) {
return nil, errUnexpectedEnd
}
delim := data[*i]
var endDelim byte
switch delim {
@@ -434,6 +452,39 @@ func parseComment(data []byte, i *int) {
}
}
// parseLocalValue parses a single value on the right-hand side of a local assignment. Expressions
// resolve to an empty string, and bare references resolve through locals. The caller must ensure *i < len(data).
func parseLocalValue(data []byte, i *int, locals map[string]string) (string, error) {
c := data[*i]
switch c {
case '"', '\'':
value, err := parseRockspecString(data, i, locals)
if err != nil {
return "", err
}
return value.value.(string), nil
default:
ref, err := parseRockspecLiteral(data, i, locals)
if err != nil {
return "", err
}
// skip if it's an expression
skipWhitespaceNoNewLine(data, i)
if *i >= len(data) {
return "", errUnexpectedEnd
}
c := data[*i]
if c != '\n' && c != '\r' {
skipExpression(data, i)
return "", nil
}
return locals[ref], nil
}
}
//nolint:funlen
func parseLocal(data []byte, i *int, locals map[string]string) error {
keys := []string{}
@@ -459,6 +510,9 @@ keys:
keys = append(keys, key)
parsing.SkipWhitespace(data, i)
if *i >= len(data) {
return errUnexpectedEnd
}
c := data[*i]
@@ -477,42 +531,22 @@ keys:
values:
for {
skipWhitespaceNoNewLine(data, i)
c := data[*i]
switch c {
case '"', '\'':
value, err := parseRockspecString(data, i, locals)
if err != nil {
return err
}
values = append(values, value.value.(string))
default:
ref, err := parseRockspecLiteral(data, i, locals)
if err != nil {
return err
}
// Skip if it's an expression
skipWhitespaceNoNewLine(data, i)
c := data[*i]
var value string
if c != '\n' && c != '\r' {
skipExpression(data, i)
value = ""
} else {
value = locals[ref]
}
values = append(values, value)
if *i >= len(data) {
return errUnexpectedEnd
}
skipWhitespaceNoNewLine(data, i)
value, err := parseLocalValue(data, i, locals)
if err != nil {
return err
}
values = append(values, value)
c = data[*i]
skipWhitespaceNoNewLine(data, i)
if *i >= len(data) {
return errUnexpectedEnd
}
c := data[*i]
switch c {
case ',':
@@ -2,6 +2,7 @@ package lua
import (
"bytes"
"strings"
"testing"
"github.com/stretchr/testify/assert"
@@ -282,3 +283,35 @@ hello = foo.. `,
})
}
}
func TestParseRockspec_deepNesting(t *testing.T) {
// deep input overflowed the goroutine stack (fatal, not recoverable) before nesting was bounded, and each level
// used to append its own copy of the input to the error
_, err := parseRockspecData(strings.NewReader(strings.Repeat("a={", 3<<20)))
require.ErrorIs(t, err, errTooDeep)
assert.Less(t, len(err.Error()), 1024)
}
func Test_parseRockspecData_truncated(t *testing.T) {
// each of these ends mid-statement, and used to read past the end of the input
tests := []struct {
name string
content string
}{
{name: "unterminated long string value", content: "a=[[x]"},
{name: "unterminated index literal", content: `a["x"`},
{name: "local missing equals", content: "local a"},
{name: "local missing value", content: "local a="},
{name: "local reference at end of input", content: "local a=b"},
{name: "local string at end of input", content: `local a="x"`},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var err error
require.NotPanics(t, func() {
_, err = parseRockspecData(strings.NewReader(tt.content))
})
require.Error(t, err)
})
}
}
+400
View File
@@ -0,0 +1,400 @@
package cataloger_test
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"go/ast"
"go/parser"
"go/token"
"io/fs"
"os"
"path/filepath"
"regexp"
"runtime"
"runtime/debug"
"runtime/metrics"
"runtime/pprof"
"sort"
"strconv"
"strings"
"testing"
"time"
"github.com/bmatcuk/doublestar/v4"
gosync "github.com/anchore/go-sync"
"github.com/anchore/syft/internal/capabilities"
"github.com/anchore/syft/internal/sbomsync"
"github.com/anchore/syft/internal/task"
"github.com/anchore/syft/internal/unknown"
"github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/internal/pkgtest"
"github.com/anchore/syft/syft/sbom"
"github.com/anchore/syft/syft/source"
"github.com/anchore/syft/syft/source/directorysource"
)
const (
// per-input limits. These are generous on purpose: the net is for inputs that take seconds or
// hundreds of MB, not for small inefficiencies.
fuzzTimeout = 10 * time.Second
// fuzzHangSlack is how long past fuzzTimeout a parser that ignores ctx gets before the process is taken down,
// so a ctx-aware parser always reports its timeout through the normal path first
fuzzHangSlack = 5 * time.Second
fuzzAllocBase = 256 << 20
fuzzAllocPerByte = 1024
fuzzMaxSeedBytes = 256 << 10
fuzzMaxSeedsPerFn = 8
)
type fuzzTarget struct {
cataloger string
glob string
path string // a concrete relative path that matches glob
dir string // the cataloger package directory, relative to this package, for seeds
task task.Task
}
func (ft fuzzTarget) String() string { return ft.cataloger + " " + ft.glob }
// FuzzGenericParsers feeds one input at a time to each glob-selected parser of every generic cataloger, by writing
// it to a path the parser's glob matches and running the cataloger task over that directory. Catalogers that are not
// generic, and processor panics (which the generic cataloger only logs), are not covered.
//
// # What fuzzing is
//
// A fuzz test is a test whose input is generated. Go's fuzzer starts from a set of example inputs (the seed corpus),
// mutates them (flipping bytes, splicing pieces together, inserting values) and keeps any mutation that reaches code
// it has not reached before, so over time it works its way into parser branches that hand-written tests never try.
// Each input is one parser and one file's worth of bytes. An input fails when the parser:
//
// - panics (including a panic the generic cataloger recovers and reports as an unknown)
// - takes longer than fuzzTimeout
// - allocates much more than the input size warrants (see fuzzAllocBase and fuzzAllocPerByte)
//
// A parser returning an error is fine: rejecting bad input is the correct behavior.
//
// # Plain go test (no fuzzing)
//
// Without -fuzz, go test does not generate anything. It runs every seed once, like a table test, and reports them
// as subtests named seed#0, seed#1, ... (go names seeds by position; the failure message says which parser it was).
// The seeds are each parser's own testdata files plus a few inputs built from the string literals in its package,
// which give the parser its own key names and prefixes to trip over. This is what runs in CI, and takes seconds.
// It also replays every saved failure in testdata/fuzz/FuzzGenericParsers/ (see below).
//
// # Running the fuzzer
//
// Fuzzing is only done on demand, locally. Pick one parser with SYFT_FUZZ_TARGET, a regexp over
// "<cataloger> <glob>" (cataloger names are listed by `syft cataloger list`), and give it a time budget:
//
// SYFT_FUZZ_TARGET='^apk-db-cataloger ' go test ./syft/pkg/cataloger -run '^$' -fuzz FuzzGenericParsers -fuzztime 60s
//
// -run '^$' skips the ordinary tests, and -fuzztime bounds the run (without it the fuzzer runs until stopped).
// Without SYFT_FUZZ_TARGET every parser shares one run, so each gets a small slice of the time. The fuzzer's
// "new interesting" inputs are kept in the go build cache, not the repo, so later runs pick up where earlier ones
// left off.
//
// # When the fuzzer finds a failure
//
// It shrinks the input to a small version that still fails, prints the failure, and writes it to
// testdata/fuzz/FuzzGenericParsers/<hash>. That file is two lines of text, the parser name and the input bytes, so
// it can be read in review. From then on plain go test replays it as the subtest FuzzGenericParsers/<hash>, and it
// can be run alone:
//
// go test ./syft/pkg/cataloger -run 'FuzzGenericParsers/<hash>'
//
// The lifecycle of a failure:
//
// 1. fix the parser so the input is rejected with an error (or parsed correctly) rather than panicking or hanging
// 2. commit the saved file in the same change as the fix, as the regression test for it; do not commit one without
// its fix, since it fails every test run until the fix lands
// 3. fuzz that parser again, since one fix often uncovers the next failure behind it
//
// Saved files are kept indefinitely, like any other regression test. Delete one only when it no longer means
// anything (for example, the parser it names was removed). A test case in the parser's own _test.go is an equally
// good home for the input, and reads better there.
func FuzzGenericParsers(f *testing.F) {
targets := fuzzTargets(f)
if len(targets) == 0 {
f.Skip("no targets selected")
}
// inputs name their target rather than index it, so a saved crasher keeps pointing at the same parser when
// the target list changes or is filtered
byName := map[string]fuzzTarget{}
for _, ft := range targets {
byName[ft.String()] = ft
for _, seed := range fuzzSeeds(ft) {
f.Add(ft.String(), seed)
}
}
f.Fuzz(func(t *testing.T, name string, data []byte) {
ft, ok := byName[name]
if !ok {
// while fuzzing, the mutator rewrites names too, and a filter deliberately drops targets. Otherwise this
// is a saved input whose target was renamed or removed, and skipping it would silently drop a regression.
if os.Getenv("SYFT_FUZZ_TARGET") == "" && flag.Lookup("test.fuzz").Value.String() == "" {
t.Fatalf("saved fuzz input names unknown target %q: update or remove it", name)
}
t.Skip("not a selected target")
}
runFuzzTarget(t, ft, data)
})
}
func runFuzzTarget(t *testing.T, ft fuzzTarget, data []byte) {
root := t.TempDir()
p := filepath.Join(root, filepath.FromSlash(ft.path))
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, data, 0o600); err != nil {
t.Fatal(err)
}
src, err := directorysource.NewFromPath(root)
if err != nil {
t.Fatal(err)
}
resolver, err := src.FileResolver(source.AllLayersScope)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(pkgtest.Context(t), fuzzTimeout)
defer cancel()
var before, after runtime.MemStats
runtime.ReadMemStats(&before)
// the heap metric is process-wide and counts unswept garbage, so collect first and measure growth from there
heap := []metrics.Sample{{Name: "/memory/classes/heap/objects:bytes"}}
runtime.GC()
metrics.Read(heap)
heapBase := heap[0].Value.Uint64()
done := make(chan error, 1)
go func() {
defer func() {
if r := recover(); r != nil {
done <- fmt.Errorf("panic: %v\n%s", r, debug.Stack())
}
}()
done <- ft.task.Execute(ctx, resolver, sbomsync.NewBuilder(&sbom.SBOM{Artifacts: sbom.Artifacts{Packages: pkg.NewCollection()}}))
}()
limit := uint64(fuzzAllocBase + fuzzAllocPerByte*len(data))
tick := time.NewTicker(50 * time.Millisecond)
defer tick.Stop()
deadline := time.After(fuzzTimeout + fuzzHangSlack)
wait:
for {
select {
case err = <-done:
break wait
case <-tick.C:
// the after-the-fact TotalAlloc check below can't stop a parser that is on its way to exhausting the
// machine, so watch heap growth while it runs and take the process down once it passes the limit
metrics.Read(heap)
if grown := heap[0].Value.Uint64() - min(heapBase, heap[0].Value.Uint64()); grown > limit {
_ = pprof.Lookup("goroutine").WriteTo(os.Stderr, 2)
panic(fmt.Sprintf("%s: input of %d bytes grew the heap by %d MB (limit %d MB)", ft, len(data), grown>>20, limit>>20))
}
case <-deadline:
// not every parser watches ctx, so a hung one keeps running and would skew every later input in this
// process. Dump where it is stuck and take the process down; the fuzz engine records that as a crasher.
_ = pprof.Lookup("goroutine").WriteTo(os.Stderr, 2)
panic(fmt.Sprintf("%s: input of %d bytes did not finish within %s", ft, len(data), fuzzTimeout+fuzzHangSlack))
}
}
if ctx.Err() != nil {
t.Fatalf("%s: input of %d bytes did not finish within %s", ft, len(data), fuzzTimeout)
}
// a panic recovered further down (a go-sync PanicError, or the generic per-file recover) still surfaces as an error
if errors.As(err, &gosync.PanicError{}) || hasRecoveredPanic(err) {
t.Fatalf("%s: %v", ft, err)
}
// TotalAlloc is cumulative, so this bounds churn rather than peak, and it is read after the fact: a single huge
// allocation kills the process before this check, which the fuzz engine still records as a crasher
runtime.ReadMemStats(&after)
if alloc := after.TotalAlloc - before.TotalAlloc; alloc > limit {
t.Fatalf("%s: input of %d bytes allocated %d MB (limit %d MB)", ft, len(data), alloc>>20, limit>>20)
}
}
// hasRecoveredPanic finds an error anywhere in the tree that starts with "recovered from panic", the wording of the
// generic per-file recover (and of the golang and license recovers). Matching on the start of a node keeps parse
// errors that echo the input from counting.
func hasRecoveredPanic(err error) bool {
if err == nil {
return false
}
if strings.HasPrefix(err.Error(), "recovered from panic") {
return true
}
switch u := err.(type) {
case *unknown.CoordinateError:
// per-file errors arrive wrapped with their location, and CoordinateError does not unwrap
return hasRecoveredPanic(u.Reason)
case interface{ Unwrap() []error }:
for _, e := range u.Unwrap() {
if hasRecoveredPanic(e) {
return true
}
}
case interface{ Unwrap() error }:
return hasRecoveredPanic(u.Unwrap())
}
return false
}
// fuzzTargets pairs every glob-selected parser in the capabilities data with the task that runs its cataloger.
func fuzzTargets(tb testing.TB) []fuzzTarget {
tasks, err := task.DefaultPackageTaskFactories().Tasks(task.DefaultCatalogingFactoryConfig())
if err != nil {
tb.Fatal(err)
}
byName := map[string]task.Task{}
for _, tsk := range tasks {
byName[tsk.Name()] = tsk
}
entries, err := capabilities.Packages()
if err != nil {
tb.Fatal(err)
}
var filter *regexp.Regexp
if v := os.Getenv("SYFT_FUZZ_TARGET"); v != "" {
if filter, err = regexp.Compile(v); err != nil {
tb.Fatalf("SYFT_FUZZ_TARGET: %v", err)
}
}
var targets []fuzzTarget
for _, e := range entries {
tsk, ok := byName[e.Name]
if e.Type != "generic" || !ok {
continue
}
for _, p := range e.Parsers {
// glob parsers only. The mimetype parsers (binary classifiers) need a real executable
// header to be selected at all; add them if a mutating seed turns out to be worth it.
if p.Detector.Method != capabilities.GlobDetection {
continue
}
for _, g := range p.Detector.Criteria {
ft := fuzzTarget{
cataloger: e.Name,
glob: g,
path: concretePath(g),
dir: strings.TrimPrefix(filepath.Dir(e.Source.File), "syft/pkg/cataloger/"),
task: tsk,
}
if ok, _ := doublestar.Match(g, "/"+ft.path); !ok || !filepath.IsLocal(ft.path) {
tb.Logf("skipping %s: no concrete path for glob", ft)
continue
}
if filter != nil && !filter.MatchString(ft.String()) {
continue
}
targets = append(targets, ft)
}
}
}
if filter != nil && len(targets) == 0 {
tb.Fatalf("SYFT_FUZZ_TARGET %q matches no target", filter)
}
return targets
}
var braceAlt = regexp.MustCompile(`\{([^,}]*)[^}]*\}`)
// concretePath turns a glob into one relative path it matches, e.g. `**/*.gguf` -> `x.gguf`.
func concretePath(glob string) string {
s := strings.TrimPrefix(strings.TrimPrefix(glob, "**/"), "/")
s = braceAlt.ReplaceAllString(s, "$1")
s = strings.ReplaceAll(s, "**", "x")
return strings.ReplaceAll(s, "*", "x")
}
// fuzzSeeds is the parser's own testdata that matches its glob, plus a few inputs built from the string literals in
// its package. Random bytes rarely produce the exact key names a parser branches on, the literals give the mutator
// something to splice.
func fuzzSeeds(ft fuzzTarget) [][]byte {
var seeds [][]byte
// the CLI's malformed fixtures are deliberately not seeds here: before their fixes land some of them allocate
// gigabytes, which in-process means swapping the machine rather than failing a check. The CLI test runs them in
// a subprocess under a memory limit instead.
_ = filepath.WalkDir(ft.dir, func(p string, d fs.DirEntry, err error) error {
if err != nil || len(seeds) >= fuzzMaxSeedsPerFn {
return fs.SkipAll
}
if d.IsDir() || !d.Type().IsRegular() {
return nil
}
if ok, _ := doublestar.Match(ft.glob, "/"+filepath.ToSlash(p)); !ok {
return nil
}
if info, err := d.Info(); err != nil || info.Size() > fuzzMaxSeedBytes {
return nil
}
if b, err := os.ReadFile(p); err == nil {
seeds = append(seeds, b)
}
return nil
})
lits := stringLiterals(ft.dir)
if len(lits) == 0 {
return append(seeds, []byte{})
}
obj := map[string]string{}
var lines, kv strings.Builder
for _, l := range lits {
obj[l] = l
lines.WriteString(l + "\n")
kv.WriteString(strconv.Quote(l) + ": " + strconv.Quote(l) + "\n")
}
js, _ := json.Marshal(obj)
return append(seeds, []byte(lines.String()), []byte(kv.String()), js)
}
// stringLiterals returns the distinct short string literals in the non-test go files of a package directory.
func stringLiterals(dir string) []string {
entries, err := os.ReadDir(dir)
if err != nil {
return nil
}
set := map[string]struct{}{}
fset := token.NewFileSet()
for _, e := range entries {
name := e.Name()
if e.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") {
continue
}
f, err := parser.ParseFile(fset, filepath.Join(dir, name), nil, 0)
if err != nil {
continue
}
ast.Inspect(f, func(n ast.Node) bool {
if bl, ok := n.(*ast.BasicLit); ok && bl.Kind == token.STRING {
if s, err := strconv.Unquote(bl.Value); err == nil && len(s) > 1 && len(s) <= 64 {
set[s] = struct{}{}
}
}
return true
})
}
out := make([]string, 0, len(set))
for s := range set {
out = append(out, s)
}
sort.Strings(out)
return out
}
@@ -214,9 +214,8 @@ func guessVersion(constraint string) string {
constraints := strings.Split(constraint, ",")
filteredVersions := map[string]struct{}{}
for _, part := range constraints {
if strings.Contains(part, "!=") {
parts := strings.Split(part, "!=")
filteredVersions[strings.TrimSpace(parts[1])] = struct{}{}
if _, excluded, ok := strings.Cut(part, "!="); ok {
filteredVersions[strings.TrimSpace(excluded)] = struct{}{}
}
}
@@ -236,9 +235,8 @@ func guessVersion(constraint string) string {
continue
}
if strings.Contains(part, "==") {
parts := strings.Split(part, "==")
return strings.TrimSpace(parts[1])
if _, pinned, ok := strings.Cut(part, "=="); ok {
return strings.TrimSpace(pinned)
}
if closestVersion == nil || version.GreaterThan(*closestVersion) {
+3 -4
View File
@@ -60,10 +60,9 @@ func parseBaseDpkgYaml(_ context.Context, _ file.Resolver, _ *generic.Environmen
}
// Handle architecture suffixes (e.g., "libssl1.1:amd64")
if strings.Contains(name, ":") {
archParts := strings.SplitN(name, ":", 2)
name = archParts[0]
snapMetadata.Architecture = archParts[1]
if n, arch, ok := strings.Cut(name, ":"); ok {
name = n
snapMetadata.Architecture = arch
}
debPkg := newDebianPackageFromSnap(
@@ -82,10 +82,9 @@ func parseSystemManifest(_ context.Context, _ file.Resolver, _ *generic.Environm
// Handle architecture suffixes if present
currentMetadata := snapMetadata
if strings.Contains(name, ":") {
archParts := strings.SplitN(name, ":", 2)
name = archParts[0]
currentMetadata.Architecture = archParts[1]
if n, arch, ok := strings.Cut(name, ":"); ok {
name = n
currentMetadata.Architecture = arch
}
debPkg := newDebianPackageFromSnap(
@@ -2,6 +2,7 @@ package swift
import (
"context"
"errors"
"fmt"
"strings"
@@ -45,9 +46,11 @@ func parsePodfileLock(_ context.Context, _ file.Resolver, _ *generic.Environment
default:
return nil, nil, fmt.Errorf("malformed podfile.lock")
}
splits := strings.Split(podBlob, " ")
podName := splits[0]
podVersion := strings.TrimSuffix(strings.TrimPrefix(splits[1], "("), ")")
podName, podVersion, ok := strings.Cut(podBlob, " ")
if !ok {
return nil, nil, errors.New("malformed podfile.lock: pod has no version")
}
podVersion = strings.TrimSuffix(strings.TrimPrefix(podVersion, "("), ")")
podRootPkg := strings.Split(podName, "/")[0]
var pkgHash string
@@ -1,8 +1,13 @@
package swift
import (
"context"
"io"
"strings"
"testing"
"github.com/stretchr/testify/require"
"github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/file"
"github.com/anchore/syft/syft/pkg"
@@ -280,3 +285,15 @@ func Test_corruptPodfile(t *testing.T) {
WithError().
TestParser(t, parsePodfileLock)
}
func TestParsePodfileLock_podWithoutVersion(t *testing.T) {
// a pod entry is "Name (version)"; one without the version used to index past the end of the split
contents := "PODS:\n - Foo\nSPEC CHECKSUMS:\n Foo: abc\n"
reader := file.NewLocationReadCloser(file.NewLocation("Podfile.lock"), io.NopCloser(strings.NewReader(contents)))
var err error
require.NotPanics(t, func() {
_, _, err = parsePodfileLock(context.Background(), nil, nil, reader)
})
require.ErrorContains(t, err, "has no version")
}
@@ -0,0 +1,3 @@
go test fuzz v1
string("lua-rock-cataloger **/*.rockspec")
[]byte("[")
+78
View File
@@ -0,0 +1,78 @@
package cli
import (
"encoding/json"
"os"
"path/filepath"
"runtime"
"syscall"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// maxMalformedScanRSS is far above what scanning a handful of tiny files needs. RSS only sees memory that was actually
// touched, so a large allocation that is never filled can slip under it; the fuzz harness in syft/pkg/cataloger
// measures allocations directly.
const maxMalformedScanRSS = 1 << 30
// each directory under testdata/malformed holds a few tiny files shaped to trip parsers up (deep nesting, huge declared
// counts, truncation, type mismatches, non-finite floats). None of them should cost the rest of the SBOM, so each is
// scanned next to a well-formed requirements.txt whose package must still be reported.
func TestMalformedInputsStillProduceSBOM(t *testing.T) {
dirs, err := filepath.Glob("testdata/malformed/*")
require.NoError(t, err)
require.NotEmpty(t, dirs)
for _, dir := range dirs {
t.Run(filepath.Base(dir), func(t *testing.T) {
root := t.TempDir()
require.NoError(t, os.CopyFS(root, os.DirFS(dir)))
require.NoError(t, os.WriteFile(filepath.Join(root, "requirements.txt"), []byte("well-formed==1.0.0\n"), 0o600))
out := filepath.Join(t.TempDir(), "sbom.json")
env := map[string]string{"GOMEMLIMIT": "512MiB"}
// runSyft aborts the process after 60s
cmd, stdout, stderr := runSyft(t, env, "scan", "dir:"+root, "--override-default-catalogers", "all", "-o", "syft-json="+out)
defer logOutputOnFailure(t, cmd, stdout, stderr)
assertSuccessfulReturnCode(t, stdout, stderr, cmd.ProcessState.ExitCode())
if ru, ok := cmd.ProcessState.SysUsage().(*syscall.Rusage); ok {
rss := int64(ru.Maxrss) // int32 on 32-bit linux
if runtime.GOOS == "linux" {
rss *= 1024 // kilobytes on linux, bytes on darwin
}
assert.Less(t, rss, int64(maxMalformedScanRSS), "max RSS %d MB", rss>>20)
}
b, err := os.ReadFile(out)
require.NoError(t, err, "no SBOM written")
var doc struct {
Artifacts []struct {
Name string `json:"name"`
} `json:"artifacts"`
Files []struct {
Location struct {
Path string `json:"path"`
} `json:"location"`
Unknowns []string `json:"unknowns"`
} `json:"files"`
}
require.NoError(t, json.Unmarshal(b, &doc), "SBOM is not valid JSON")
var names []string
for _, a := range doc.Artifacts {
names = append(names, a.Name)
}
assert.Contains(t, names, "well-formed", "package from the well-formed file is missing")
// a per-file panic is recovered into an unknown and the scan still exits 0, so look for it explicitly
for _, f := range doc.Files {
for _, u := range f.Unknowns {
assert.NotContains(t, u, "recovered from panic", "parser panicked on %s", f.Location.Path)
}
}
})
}
}
+1
View File
@@ -1,3 +1,4 @@
# negate jar and zip exclusions for test fixtures that need these file types
!image-unknowns/*.jar
!image-unknowns/*.zip
!malformed/**/*.jar
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
File diff suppressed because one or more lines are too long
+6
View File
@@ -0,0 +1,6 @@
version = 3
[[package]]
name = ["a"]
version = 1
dependencies = "b"
@@ -0,0 +1 @@
{"name": ["a"], "version": {"v": 1}, "license": [1, 2], "author": 7, "dependencies": []}
Binary file not shown.
+14
View File
@@ -0,0 +1,14 @@
{
"name": "t",
"version": "1.0.0",
"lockfileVersion": 3,
"packages": {
"": {
"name": "t",
"version": "1.0.0"
},
"node_modules/a": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/a/-/a-1.0.0.tgz"
}
}
File diff suppressed because one or more lines are too long
+8
View File
@@ -0,0 +1,8 @@
lockfileVersion: '9.0'
packages:
/a@1.0.0: &p
resolution: {integrity: sha512-x}
/b@1.0.0:
<<: *p
/c@1.0.0:
<<: [*p, {<<: *p}]