`put` had no per-entry cap, unlike the legacy java path which capped every file at 2 GB through `intFile.SafeCopy`. At depth 1, one entry of an overlapping-entry zip bomb could decompress all the way to the 100 GB disk limit before anything stopped it.
- entries are now read through the same 2 GB cap (`intFile.PerFileReadLimit`, now exported). An entry over it is dropped after cap+1 bytes and the rest of the archive is still extracted
- `Resolver` gains `TruncatedReason`, so the truncation unknown says what happened ("an entry larger than 2.0 GiB was skipped", or the existing disk-limit text) instead of always blaming the limits
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
`identifyFormat` accepted any extractor mholt identified, which put the 7z and rar decoders (`bodgit/sevenzip`, `rardecode`) on untrusted input with no test coverage. Only zip, tar and compressed tar are opened now, until other formats are deliberately supported.
This also fixes java resource adapters: a `.rar` that is really a zip used to be identified by name as RAR and fail to extract. An unsupported identification now falls through to the zip end-of-central-directory check, so it's read as the zip it is.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
sub-pipeline catalogers were already run under `runTaskSafely`, but extraction wasn't: the zip, tar and compressor decoders run on untrusted bytes, and a panic in one unwound the whole walk and failed `CreateSBOM` with "failed to run tasks".
- `processArchive` recovers and returns the panic (with its stack) as an error at the archive's coordinates, so it becomes an unknown and the sibling archives are still cataloged
- `Extract` now defers `Cleanup` on failure instead of calling it on each error path, so a panic no longer leaks the limiter charge or the spill file
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
each `ArchivePath` segment came from `loc.Path()`, so it was the link path when an archive was reached through a symlink and the real path otherwise. That depended on how discovery happened to find the archive. The package location said `store/3f9a1c` while the files inside the same jar were recorded under `...:app/lib/json-simple-1.1.1.jar`, so a consumer couldn't join a package to its files by prefix.
The chain for extracted files is now built from `RealPath`. `Traversal.Location` still carries the access path, so java's name check and `VirtualPath` metadata behave the same as before. The field comment and schema now say every segment has links resolved.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
`:` was escaped as `%3A`, but `%` itself wasn't, so an entry named `a:b.jar` and one named `a%3Ab.jar` got the same `ArchivePath`. Their contents then collide in `FileMetadata` / `FileDigests` (last writer wins) and share artifact IDs.
`%` is now escaped as `%25` before `:`, which makes the encoding injective. The encoding is documented on `Coordinates.ArchivePath` and in the schema.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Archives may contain directory entries with read-only permissions
(e.g., META-INF/ with mode 0555 in JARs). When extracted to temp
directories, these restricted permissions prevented writing files
into those directories, causing "permission denied" errors for
subsequent file entries like META-INF/MANIFEST.MF.
Always use 0755 for extracted directories since they are temporary
working directories owned by the extraction process.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add integration tests covering the full archive extraction pipeline:
- tar.gz containing Python package metadata (single-level)
- zip containing a JAR with MANIFEST.MF (two-level, same format)
- Mixed archive types (zip + tar.gz at same level)
- Depth limit enforcement (3-level zip nesting, limited to 1)
- Extension exclusion filtering
- 3-level mixed format nesting: tar.gz -> zip -> tar.gz -> files
verifying cross-format extraction and full access path chains
- Reverse format crossing: tar.gz -> zip -> tar.gz verifying
content readability at all levels and correct relationship count
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add tests for important orchestrator edge cases:
- Nested archives (zip-inside-zip): verifies multi-round extraction
discovers archives within previously-extracted archives
- Context cancellation: graceful handling without hanging
- Multiple archives at same level: both extracted in one round
- Corrupt archive: skipped without error, doesn't block other work
- Empty archive: not counted as extracted
- Per-archive file limit: partial extraction handled correctly
- Relationship structure: contains relationships emit the archive's
coordinates as the source side
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Implement the Orchestrator that manages the iterative
extract-catalog loop for recursive archive discovery:
- Scans the composite resolver for candidate archive locations,
detecting format by peeking 512 bytes via FileContentsByLocation
rather than copying every non-archive file to a probe-temp file
- Extracts identified archives to per-archive temp directories
- Builds child resolvers via an injectable ResolverFactory (kept
pluggable so tests can substitute a mock resolver without
importing syft/internal/fileresolver)
- Tracks extraction state to avoid reprocessing
- Enforces safety limits (depth, per-archive size, file count,
total extraction budget)
- Generates ContainsRelationship entries linking each archive to
its extracted-contents filesystem
- Manages temp directory lifecycle with Cleanup()
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add tests for important edge cases:
- Nested children (child-in-child): verifies that archives within
archives produce correct multi-level AccessPath chains like
"/archive1.tar.gz:/inner-archive.zip:/deep-file.txt"
- Empty child resolver: no locations leak from empty archives
- Same file path in parent and child: both versions accessible
with distinct FileSystemID routing
- Context cancellation in AllLocations: no hanging
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Implement a file.Resolver decorator that composites a parent resolver
with child resolvers created from extracted archive contents. Files from
child resolvers appear transparently alongside parent files.
Key behaviors:
- FilesByPath/FilesByGlob/FilesByMIMEType delegate to parent AND children
- FileContentsByLocation routes to the correct resolver via FileSystemID
- Locations from children get archive-prefixed AccessPath (using ":"
separator matching Java cataloger convention) and unique FileSystemID
- Thread-safe via RWMutex for concurrent access
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Introduce the internal/archive package with a generic Extractor
interface for extracting archive contents to a destination directory.
Includes two implementations:
- ZipExtractor: handles zip-based archives (.zip, .jar, .war, etc.)
- TarExtractor: handles tar-based archives (.tar, .tar.gz, .tar.bz2, etc.)
Both extractors enforce configurable safety limits and protect against
common malicious-archive vectors:
- Per-archive extraction size limit
- Per-archive file count limit
- Zip-slip path traversal prevention via existing SafeJoin
- Symlink entries are written as real symlinks only when their target
resolves inside destDir; symlinks pointing outside are dropped to
avoid escape via symlink-then-write
- Non-regular non-symlink entries (devices, fifos, sockets) are skipped
because their contents can't safely round-trip to disk
Also provides FindExtractor for auto-detecting archive format, and
IsExcludedExtension for filtering by extension.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Alex Goodman
Keith Zantow
Thomas Bechtold