mirror of
https://github.com/anchore/syft.git
synced 2026-11-02 23:46:35 +01:00
b254e6d92f28c3868a755f62fb3ca8f26e9fee76
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Alex Goodman
|
6c9bb732dd |
ci: run the full unit suite on windows (#5370)
* ci: run the full unit suite on windows via make unit Drops the curated `unit:windows` subset; the windows job now bootstraps like linux and runs `make unit`. Exceptions get added as failures show up. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * ci: skip fixture refresh for windows unit tests binny has no windows release, so the bootstrap action can't install the tools the fixture refresh needs. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * test: skip linux container fixtures when docker can't run them Windows CI runners only run windows containers, so image fixtures go through `testutils.SkipWithoutLinuxContainers`. No behavior change on linux/mac. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix: close leaked file handles and posix squashfs paths on windows - output writers are now closed on early returns in `scan`/`convert` (open handles block cleanup on windows) - `newSBOMMultiWriter` used `path.Dir` on host paths, so output subdirectories were never created on windows - squashfs walk paths were built with `filepath.Join` - skip a few posix-only test cases (dir perms, xdg fallback, go-rpmdb ndb handle leak upstream) Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(golang): resolve go source module dir natively on windows - `parseGoModFile` passed the resolver's posix (`/c/...`) path to the go toolchain, so go source resolution silently found nothing on windows - skip container-built fixtures (go archs, java builds) and golden image tarballs on windows; stereoscope's layer cache filenames contain `:`, which windows rejects Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix: keep posix paths posix on windows - `ChrootContext.ToChrootPath` nested volumes (`D:\cwd\C:\root`) when the scan root was on another drive, so the prefix was never trimmed - bitnami, python wheel/egg, and alpm parsers used `filepath` on posix location paths - alpm backup-file lookups silently missed on windows Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * ci: pull the test fixture cache for windows unit tests Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix: don't map virtual file tree paths through the host on windows `FiletreeResolver` without a chroot (snaps) ran request paths through `filepath.Abs`, rooting them on the current volume (`/payload.txt` -> `/d/payload.txt`), so snap scans on windows found nothing. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix: match tar entry globs as posix on windows - `matchesAnyGlob` used `doublestar.PathMatch` (host separator) on tar entry names, so tar-wrapped java archives silently produced nothing on windows - test helpers now treat binary snippet keys and `FromFile` locations as posix Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * incorporate windows path fix Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * incorporate go-rpmdb close fix Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * close squashfs fixture writers before finalizing - windows dir listings can report a zero size for files still open, so the manifest was packed empty (also adds a temporary windows diagnostic for the binary cataloger) Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * temp: surface windows binary cataloger diagnostics Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix: keep directory symlink targets posix on windows - the directory indexer joined symlink targets with native separators, so links in the posix file tree pointed nowhere and globs on link names found nothing - binary fixture logical keys used native separators, so nested snippets were picked as the fixture dir on windows Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * keep addSymlinkToIndex under the funlen limit Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * drop windows skips covered by the stereoscope and go-rpmdb bumps Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> --------- Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> |
||
|
Alex Goodman
|
1dc16fbac4 |
fix: report scanner errors and raise the line cap in metadata parsers (#5353)
* fix: report scanner errors instead of returning a partial parse as complete A line longer than `bufio.Scanner`'s 64KB buffer ends the scan with `ErrTooLong`, which only `Err()` reports. Parsers now return what they found along with the error (recorded as an unknown for the file), or log it where there is no error to return. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix: allow lines up to 1MB in line-oriented package metadata parsers Adds `internal.NewLineScanner`, a `bufio.Scanner` capped at 1MB per line rather than the 64KB default, the same cap the alpm parser already uses. Long single-line fields (descriptions, license text, dependency lists) now parse instead of ending the scan early. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix: use io.ReadFull where a short read was silently accepted - dotnet bundle 7-bit int decoding could read a zero-byte result as a zero byte - wordpress plugin header parsing included the unread tail of its buffer Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix: finish moving line scanners to the 1MB cap and checking scanner errors - java manifest, pom.properties, Gemfile.lock, requirements.txt, homebrew formula, alpm and the purl/cpe decoders now use `internal.NewLineScanner` - snap kernel changelog reports a scan error instead of silently dropping the base kernel package - the attestation output reader keeps draining the pipe after a scan error so cosign can't block on a full pipe Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix: stop logging the full setup.py line once per dependency match The debug log ran once per regex match and included the whole line, so log volume grew with the square of line length. With 1MB lines that is hundreds of GB of debug output from a single crafted file. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(gentoo): don't dereference a nil location when a LICENSE scan fails `extractLicenses` accepts a nil location, but the scan error log added earlier on this branch dereferenced it. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(gentoo): bound license group expansion on hostile license_groups - dedup within a group uses a set instead of `slices.Contains`, which was quadratic in tokens per line and got much worse under the 1MB line cap - each group is expanded once and memoized, so a chain of groups that reference each other repeatedly no longer expands exponentially - repeated references to the same group within one line are skipped Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> --------- Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> |
||
|
Alex Goodman
|
30e9e65608 |
Fix: alpm mtree gzip unbounded processing (#5292)
* fix(arch): bound the decompressed mtree listing The ALPM cataloger handed a gzip stream straight to `mtree.ParseSpec`, which materializes every entry before it returns any of them. Nothing capped the decompressed size, so a crafted mtree in a scanned image expanded until the process died. 64KB of input is enough to produce 64MB, and the ratio scales. Reads through a limit now and rejects a listing that exceeds it. The read goes one byte past the cap so that tripping it is distinguishable from a listing that simply ends there, since silently truncating would produce a package missing most of its files. The existing error path already surfaces this as an unknown. The cap is well above any real package: an mtree names every file with its digests, and even the largest packages land far under it. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(arch): bound mtree entry count, not just decompressed bytes The byte cap alone does not bound what the parse retains. go-mtree keeps an entry per line and appends it unconditionally, blank lines included, so short lines cost about 1KB of heap each. Measured on the previous commit: 15KB of gzipped newlines expanding to 16MB, comfortably inside the 64MB byte cap, drove 8.1GB of peak heap over 74 seconds and returned no records and a nil error. Same input now rejects in 0.05s at 51MB peak. Counting lines before parsing is what makes the entry allocation bounded, since the parser materializes all of them before returning. Lines the parser skips only make the count high, never low. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(arch): apply the mtree bounds while streaming Both caps were enforced against a fully buffered listing, so the peak cost scaled with cataloger parallelism, which runs a goroutine per package. They now apply as the stream flows: a limited reader for the byte cap and a counting reader for the entry cap, which has to trip while the parser is still reading since it materializes every entry before returning any of them. Both limits are injectable so the boundary tests can be exact on both sides without allocating their way up to the shipped 64MB. Each cap gets a listing exactly at it, asserting the records parse whole rather than just that no error came back, since quiet truncation is the failure a size check invites. The byte limiter carries one byte of headroom, which is what makes an overrunning listing distinguishable from one that simply ends at the cap. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(arch): refuse mtree line continuations go-mtree collapses a line ending in a backslash into the next one with a quadratic string concat, so the cost is lines times bytes and neither the byte cap nor the line cap can see it. 199k continued lines sitting inside both caps measured 4m34s and 6.1TB of allocation churn, and returned no error. Also re-derived both constants against a real package rather than a guess. `texlive-fontsextra` is 105,847 files and 1,978 directories, so roughly 108k lines at 150-250 bytes each. 16MB would have falsely rejected it, so the byte cap stays at 64MB and the line cap goes 200k -> 300k, which is ~2.8x a real package instead of 1.85x. Rejections are now sentinels so tests can name which bound tripped, and the constants carry the measured ceiling they imply: a crafted listing of minimal fields still reaches ~1.1GB at the byte cap, since go-mtree allocates per whitespace-delimited field. Bounding that further means bounding fields rather than bytes, and legitimate input is already within ~1.3x of the crafted worst case, so there is little to win. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(arch): refuse a continuation spelled with a carriage return The refusal matched the literal bytes backslash-newline, but go-mtree decides "this is a continuation" from the scanner token, and `bufio.ScanLines` drops a trailing carriage return before testing for the backslash suffix. So a listing using CRLF ran the quadratic collapse with nothing watching. Measured: 775KB of gzip expanding to 17MB across 150k lines, inside both other bounds, took 1m33s and returned a nil error. Same input now rejects in 50ms. The check is a single byte pass that counts newlines and looks for a backslash before either line ending, which replaces the two scans and the straddle special case it took to catch only one spelling. A tripped bound is also latched now: `bufio.Scanner` stops at the first error, but nothing about the type said a caller had to, and the refusal was recomputed per chunk. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * test(arch): make the bombpkg fixture able to fail The fixture was 310k bare newlines, which go-mtree turns into blank entries with no name, and the parse loop already skips those. So the assertion that a rejected listing leaves an empty file list held whether or not the bound existed. It is now a repeated file line: 0 records with the bound, 310,000 without. Dropped the two tests that asserted nothing as a result. The cataloger-level one was a weaker restatement of what the golden comparison already pins exactly, and the shipped-size-cap one drove a real 64MB listing through the parser to prove a constant had not moved. Asserting the constants directly catches any change rather than only a loosening. Race-mode package runtime goes 15.7s -> 4.4s. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * test(arch): make three mtree assertions able to fail The truncated-gzip case asserted the failure was not misreported as the size bound, but ran against the shipped 64MB cap on a 3.5KB listing, so the check it guarded could never fire. It now runs against a cap just above the truncation point, where the two genuinely race. The bomb case asserted an allocation delta from `runtime.MemStats`, which is a process-wide cumulative counter and measures the property indirectly. It now counts bytes pulled through the reader, which is what "gave up while reading" actually means and does not drift with GC or parallelism. Added the one bound that had no test at all. A single line over 64KB is refused by go-mtree's scanner, not by anything here, so a dependency bump that called `scanner.Buffer` would remove it silently. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * docs(arch): correct the mtree bound comments and close the gzip reader The comment said both bounds apply while the listing streams. Only the line cap does. `io.LimitedReader` signals a drained budget with `io.EOF`, which the parser reads as the end of a complete listing, so the byte cap is necessarily a check after the fact. Peak cost is the same either way, since accepting a listing at the cap costs what rejecting one just past it does, but the comment was claiming a mechanism that isn't there. Also trimmed the benchmark writeups down to the reasoning that outlives them. The exact transient-heap and wall-clock figures are tied to a go-mtree version with nothing to detect them going stale, and one of them described the CRLF case as prevented when it wasn't. And `TestMtreeParse` was labelled the regression guard for the continuation refusal, but its fixture has no backslashes in it. `gzReader` now gets a `defer Close` to match the other gzip site in the tree. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(arch): stop the alpm database parser panicking on a malformed backup line A `%BACKUP%` entry is `path\tmd5`, and the parser indexed the second field without checking the split produced one. A line with no tab panics. The database is attacker-controlled, and while the task executor recovers, it recovers at task granularity, so one crafted line drops every Arch package from the SBOM. The same loop also never checked the scanner's error. A field over the 1MB token cap ends the scan, and the entry came back populated up to that field and empty after it, with a nil error saying it was fine. That is the failure the mtree bounds in this branch exist to avoid, two functions up in the same file. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * docs(arch): trim the comments added by this branch the comments were carrying reasoning that belongs in the PR description rather than the code. Kept the parts a future reader cannot recover from the code (the cap sizing, why continuations are refused, why the byte check runs after the parse) and cut the rest. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * test(arch): generate the mtree bomb fixture instead of committing it the fixture is a synthetic listing fully described by a line count and a repeated line, but it landed as a committed gzip member, which shows neither and leaves no way to rebuild it when maxMtreeLines moves. the two tests that walk testdata/installed write it first now, and git ignores it. the real pacman listings (gmp, tree-sitter) stay committed since those are captured output rather than something a rule can reproduce. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * correct and simplify comments Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> --------- Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> |
||
  anchore-oss-update-botandanchore-oss-update-bot
|
5b58ec96b7 |
chore(deps): update Go version (#4773)
Signed-off-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com> Co-authored-by: anchore-oss-update-bot <anchore-oss-update-bot@users.noreply.github.com> |
||
 Christopher Angelo PhillipsandAlex Goodman
|
f77d503892 |
detect license ID from full text when incidentally provided as a value (#3876)
--------- Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com> Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com> |
||
 anchore-actions-token-generator[bot]andspiffcs
|
f11377fe30 |
chore(deps): update tools to latest versions (#3775)
--------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com> Co-authored-by: spiffcs <32073428+spiffcs@users.noreply.github.com> |
||
|
Alex Goodman
|
e9b24a29d7 |
Remove mitchellh dependencies (#3748)
* remove mitchellh dependencies Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix failing unit tests Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> --------- Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> |
||
 Keith ZantowandAlex Goodman
|
ccbee94b87 |
feat: report unknowns in sbom (#2998)
Signed-off-by: Keith Zantow <kzantow@gmail.com> Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com> |
||
|
Alex Goodman
|
ada8f009d2 |
Add relationships for ALPM packages (arch linux) (#2851)
* add alpm relationships Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * tweak reader linter rule to check for reader impl Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * update JSON schema with alpm dependency information Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> --------- Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> |
||
 William Murphy
|
3713d97b7b |
chore: use ruleguard to test for missing defer statements (#2837)
* chore: ruleguard to enforce defer use Signed-off-by: Will Murphy <will.murphy@anchore.com> * fix go.mod location Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: defer close in linux release identifier Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: better lint suggestion Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: refactor binary classifier to defer close Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: defer close readers in gentoo cataloger Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: make go license parsing defer close readers Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: defer closing readers in alpine apm parser Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: defer close readers in graalvm parser Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: defer close readers in debian package parser Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: defer close readers in alpm parser Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: defer close readers in executable file cataloger Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: defer close readers in javascript license parser Signed-off-by: Will Murphy <will.murphy@anchore.com> * chore: defer close readers in go mod parser Signed-off-by: Will Murphy <will.murphy@anchore.com> --------- Signed-off-by: Will Murphy <will.murphy@anchore.com> |
||
|
Alex Goodman
|
cdad5e767a |
plumb context through catalogers (#2528)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> |
||
|
Alex Goodman
|
b2f4d7eda2 |
Follow convention for naming catalogers (#2277)
* follow convention for naming catalogers Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix cataloger name example Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> --------- Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> |