mirror of
https://github.com/anchore/syft.git
synced 2026-10-11 21:57:22 +02:00
Alex Goodman
30e9e65608
* fix(arch): bound the decompressed mtree listing The ALPM cataloger handed a gzip stream straight to `mtree.ParseSpec`, which materializes every entry before it returns any of them. Nothing capped the decompressed size, so a crafted mtree in a scanned image expanded until the process died. 64KB of input is enough to produce 64MB, and the ratio scales. Reads through a limit now and rejects a listing that exceeds it. The read goes one byte past the cap so that tripping it is distinguishable from a listing that simply ends there, since silently truncating would produce a package missing most of its files. The existing error path already surfaces this as an unknown. The cap is well above any real package: an mtree names every file with its digests, and even the largest packages land far under it. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(arch): bound mtree entry count, not just decompressed bytes The byte cap alone does not bound what the parse retains. go-mtree keeps an entry per line and appends it unconditionally, blank lines included, so short lines cost about 1KB of heap each. Measured on the previous commit: 15KB of gzipped newlines expanding to 16MB, comfortably inside the 64MB byte cap, drove 8.1GB of peak heap over 74 seconds and returned no records and a nil error. Same input now rejects in 0.05s at 51MB peak. Counting lines before parsing is what makes the entry allocation bounded, since the parser materializes all of them before returning. Lines the parser skips only make the count high, never low. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(arch): apply the mtree bounds while streaming Both caps were enforced against a fully buffered listing, so the peak cost scaled with cataloger parallelism, which runs a goroutine per package. They now apply as the stream flows: a limited reader for the byte cap and a counting reader for the entry cap, which has to trip while the parser is still reading since it materializes every entry before returning any of them. Both limits are injectable so the boundary tests can be exact on both sides without allocating their way up to the shipped 64MB. Each cap gets a listing exactly at it, asserting the records parse whole rather than just that no error came back, since quiet truncation is the failure a size check invites. The byte limiter carries one byte of headroom, which is what makes an overrunning listing distinguishable from one that simply ends at the cap. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(arch): refuse mtree line continuations go-mtree collapses a line ending in a backslash into the next one with a quadratic string concat, so the cost is lines times bytes and neither the byte cap nor the line cap can see it. 199k continued lines sitting inside both caps measured 4m34s and 6.1TB of allocation churn, and returned no error. Also re-derived both constants against a real package rather than a guess. `texlive-fontsextra` is 105,847 files and 1,978 directories, so roughly 108k lines at 150-250 bytes each. 16MB would have falsely rejected it, so the byte cap stays at 64MB and the line cap goes 200k -> 300k, which is ~2.8x a real package instead of 1.85x. Rejections are now sentinels so tests can name which bound tripped, and the constants carry the measured ceiling they imply: a crafted listing of minimal fields still reaches ~1.1GB at the byte cap, since go-mtree allocates per whitespace-delimited field. Bounding that further means bounding fields rather than bytes, and legitimate input is already within ~1.3x of the crafted worst case, so there is little to win. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(arch): refuse a continuation spelled with a carriage return The refusal matched the literal bytes backslash-newline, but go-mtree decides "this is a continuation" from the scanner token, and `bufio.ScanLines` drops a trailing carriage return before testing for the backslash suffix. So a listing using CRLF ran the quadratic collapse with nothing watching. Measured: 775KB of gzip expanding to 17MB across 150k lines, inside both other bounds, took 1m33s and returned a nil error. Same input now rejects in 50ms. The check is a single byte pass that counts newlines and looks for a backslash before either line ending, which replaces the two scans and the straddle special case it took to catch only one spelling. A tripped bound is also latched now: `bufio.Scanner` stops at the first error, but nothing about the type said a caller had to, and the refusal was recomputed per chunk. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * test(arch): make the bombpkg fixture able to fail The fixture was 310k bare newlines, which go-mtree turns into blank entries with no name, and the parse loop already skips those. So the assertion that a rejected listing leaves an empty file list held whether or not the bound existed. It is now a repeated file line: 0 records with the bound, 310,000 without. Dropped the two tests that asserted nothing as a result. The cataloger-level one was a weaker restatement of what the golden comparison already pins exactly, and the shipped-size-cap one drove a real 64MB listing through the parser to prove a constant had not moved. Asserting the constants directly catches any change rather than only a loosening. Race-mode package runtime goes 15.7s -> 4.4s. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * test(arch): make three mtree assertions able to fail The truncated-gzip case asserted the failure was not misreported as the size bound, but ran against the shipped 64MB cap on a 3.5KB listing, so the check it guarded could never fire. It now runs against a cap just above the truncation point, where the two genuinely race. The bomb case asserted an allocation delta from `runtime.MemStats`, which is a process-wide cumulative counter and measures the property indirectly. It now counts bytes pulled through the reader, which is what "gave up while reading" actually means and does not drift with GC or parallelism. Added the one bound that had no test at all. A single line over 64KB is refused by go-mtree's scanner, not by anything here, so a dependency bump that called `scanner.Buffer` would remove it silently. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * docs(arch): correct the mtree bound comments and close the gzip reader The comment said both bounds apply while the listing streams. Only the line cap does. `io.LimitedReader` signals a drained budget with `io.EOF`, which the parser reads as the end of a complete listing, so the byte cap is necessarily a check after the fact. Peak cost is the same either way, since accepting a listing at the cap costs what rejecting one just past it does, but the comment was claiming a mechanism that isn't there. Also trimmed the benchmark writeups down to the reasoning that outlives them. The exact transient-heap and wall-clock figures are tied to a go-mtree version with nothing to detect them going stale, and one of them described the CRLF case as prevented when it wasn't. And `TestMtreeParse` was labelled the regression guard for the continuation refusal, but its fixture has no backslashes in it. `gzReader` now gets a `defer Close` to match the other gzip site in the tree. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * fix(arch): stop the alpm database parser panicking on a malformed backup line A `%BACKUP%` entry is `path\tmd5`, and the parser indexed the second field without checking the split produced one. A line with no tab panics. The database is attacker-controlled, and while the task executor recovers, it recovers at task granularity, so one crafted line drops every Arch package from the SBOM. The same loop also never checked the scanner's error. A field over the 1MB token cap ends the scan, and the entry came back populated up to that field and empty after it, with a nil error saying it was fine. That is the failure the mtree bounds in this branch exist to avoid, two functions up in the same file. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * docs(arch): trim the comments added by this branch the comments were carrying reasoning that belongs in the PR description rather than the code. Kept the parts a future reader cannot recover from the code (the cap sizing, why continuations are refused, why the byte check runs after the parse) and cut the rest. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * test(arch): generate the mtree bomb fixture instead of committing it the fixture is a synthetic listing fully described by a line count and a repeated line, but it landed as a committed gzip member, which shows neither and leaves no way to rebuild it when maxMtreeLines moves. the two tests that walk testdata/installed write it first now, and git ignores it. the real pacman listings (gmp, tree-sitter) stay committed since those are captured output rather than something a rule can reproduce. Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> * correct and simplify comments Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com> --------- Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Syft
A CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Exceptional for vulnerability detection when used with a scanner like Grype.
Features
- Generates SBOMs for container images, filesystems, archives (see the docs for a full list of supported scan targets)
- Supports dozens of packaging ecosystems (e.g. Alpine (apk), Debian (dpkg), RPM, Go, Python, Java, JavaScript, Ruby, Rust, PHP, .NET, and many more)
- Supports OCI, Docker, Singularity, and more image formats
- Works seamlessly with Grype for vulnerability scanning
- Multiple output formats (CycloneDX, SPDX, Syft JSON, and more) including the ability to convert between SBOM formats
- Create signed SBOM attestations using the in-toto specification
Tip
New to Syft? Check out the Getting Started guide for a walkthrough!
Installation
The quickest way to get up and going:
curl -sSfL https://get.anchore.io/syft | sudo sh -s -- -b /usr/local/bin
Tip
See Installation docs for more ways to get Syft, including Homebrew, Docker, Scoop, Chocolatey, Nix, and more!
The basics
See the packages within a container image or directory:
# container image
syft alpine:latest
# directory
syft ./my-project
To get an SBOM, specify one or more output formats:
# SBOM to stdout
syft <image> -o cyclonedx-json
# Multiple SBOMs to files
syft <image> -o spdx-json=./spdx.json -o cyclonedx-json=./cdx.json
Tip
Check out the Getting Started guide to explore all of the capabilities and features.
Want to know all of the ins-and-outs of Syft? Check out the CLI docs, configuration docs, and JSON schema.
Contributing
We encourage users to help make these tools better by submitting issues when you find a bug or want a new feature. Check out our contributing overview and developer-specific documentation if you are interested in providing code contributions.
Syft development is sponsored by Anchore, and is released under the Apache-2.0 License.
The Syft logo by Anchore is licensed under CC BY 4.0
For commercial support options with Syft or Grype, please contact Anchore.
Come talk to us!
The Syft Team holds regular community meetings online. All are welcome to join to bring topics for discussion.
- Check the calendar for the next meeting date.
- Add items to the agenda (join this group for write access to the agenda)
- See you there!