Add integration tests covering the full archive extraction pipeline:
- tar.gz containing Python package metadata (single-level)
- zip containing a JAR with MANIFEST.MF (two-level, same format)
- Mixed archive types (zip + tar.gz at same level)
- Depth limit enforcement (3-level zip nesting, limited to 1)
- Extension exclusion filtering
- 3-level mixed format nesting: tar.gz -> zip -> tar.gz -> files
verifying cross-format extraction and full access path chains
- Reverse format crossing: tar.gz -> zip -> tar.gz verifying
content readability at all levels and correct relationship count
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Integrate the archive extraction orchestrator into the SBOM creation
pipeline, enabling recursive archive cataloging when configured:
- Add Archive field to CreateSBOMConfig with WithArchiveConfig builder
- Add Exclusions field to CreateSBOMConfig with WithExclusions builder
so user --exclude patterns also filter content inside extracted
archives, not only the source filesystem
- Add setupArchiveOrchestration() that creates the orchestrator when
MaxDepth > 0, runs the iterative extract loop, and returns the
composite resolver with archive contents visible to all catalogers
- Include archive config in configuration audit trail
- Add CLI archive config section (max-depth, size/file limits,
exclude-extensions) wired through Catalog.ToArchiveConfig()
- CLI Catalog.ToSBOMConfig threads cfg.Exclusions through
WithExclusions so child resolvers respect them
When MaxDepth is 0 (default), behavior is unchanged. When > 0, archives
discovered during scanning are extracted and their contents made visible
to all catalogers via the CompositeResolver.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add tests for important orchestrator edge cases:
- Nested archives (zip-inside-zip): verifies multi-round extraction
discovers archives within previously-extracted archives
- Context cancellation: graceful handling without hanging
- Multiple archives at same level: both extracted in one round
- Corrupt archive: skipped without error, doesn't block other work
- Empty archive: not counted as extracted
- Per-archive file limit: partial extraction handled correctly
- Relationship structure: contains relationships emit the archive's
coordinates as the source side
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Implement the Orchestrator that manages the iterative
extract-catalog loop for recursive archive discovery:
- Scans the composite resolver for candidate archive locations,
detecting format by peeking 512 bytes via FileContentsByLocation
rather than copying every non-archive file to a probe-temp file
- Extracts identified archives to per-archive temp directories
- Builds child resolvers via an injectable ResolverFactory (kept
pluggable so tests can substitute a mock resolver without
importing syft/internal/fileresolver)
- Tracks extraction state to avoid reprocessing
- Enforces safety limits (depth, per-archive size, file count,
total extraction budget)
- Generates ContainsRelationship entries linking each archive to
its extracted-contents filesystem
- Manages temp directory lifecycle with Cleanup()
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add tests for important edge cases:
- Nested children (child-in-child): verifies that archives within
archives produce correct multi-level AccessPath chains like
"/archive1.tar.gz:/inner-archive.zip:/deep-file.txt"
- Empty child resolver: no locations leak from empty archives
- Same file path in parent and child: both versions accessible
with distinct FileSystemID routing
- Context cancellation in AllLocations: no hanging
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Implement a file.Resolver decorator that composites a parent resolver
with child resolvers created from extracted archive contents. Files from
child resolvers appear transparently alongside parent files.
Key behaviors:
- FilesByPath/FilesByGlob/FilesByMIMEType delegate to parent AND children
- FileContentsByLocation routes to the correct resolver via FileSystemID
- Locations from children get archive-prefixed AccessPath (using ":"
separator matching Java cataloger convention) and unique FileSystemID
- Thread-safe via RWMutex for concurrent access
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Introduce the internal/archive package with a generic Extractor
interface for extracting archive contents to a destination directory.
Includes two implementations:
- ZipExtractor: handles zip-based archives (.zip, .jar, .war, etc.)
- TarExtractor: handles tar-based archives (.tar, .tar.gz, .tar.bz2, etc.)
Both extractors enforce configurable safety limits and protect against
common malicious-archive vectors:
- Per-archive extraction size limit
- Per-archive file count limit
- Zip-slip path traversal prevention via existing SafeJoin
- Symlink entries are written as real symlinks only when their target
resolves inside destDir; symlinks pointing outside are dropped to
avoid escape via symlink-then-write
- Non-regular non-symlink entries (devices, fifos, sockets) are skipped
because their contents can't safely round-trip to disk
Also provides FindExtractor for auto-detecting archive format, and
IsExcludedExtension for filtering by extension.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add configuration fields for generic recursive archive extraction:
- MaxDepth: maximum nesting depth for archive extraction (default 0 = disabled)
- MaxExtractionSizeBytes: per-archive extraction size limit (default 500MB)
- MaxFileCount: per-archive file count limit (default 10,000)
- MaxTotalExtractionBytes: total extraction size limit (default 2GB)
- ExcludeExtensions: archive extensions to skip during extraction
These fields lay the groundwork for cross-cataloger recursive archive
cataloging, where archives discovered during scanning can be extracted
and their contents made visible to all catalogers.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Empty or whitespace-only .rockspec files cause parseRockspecBlock to
panic with "index out of range" because the existing end-of-data guard
requires len(out) > 0 before returning the "unexpected end of block"
error, letting the bare data[*i] access on the next line crash.
Split the guard so that:
- partial content at end of data still returns the existing error
- empty data (or whitespace-only) returns an empty block cleanly
Closes#4824.
Signed-off-by: Akihiko Komada <aki1770@gmail.com>
Ensures the correct distro id for AlmaLinux and Rocky Linux when falling
back to parsing distro information from the redhat-release file. Also
sets the idlike to `rhel` for these instances as that is necessary to
ensure correct vulnerability data matching.
Signed-off-by: Weston Steimel <author@code.w.steimel.me.uk>
The JRuby project migrated their downloads from S3 to GitHub Releases,
causing the old S3 URLs to return HTTP 403 Forbidden and breaking test
fixture image builds.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Passing '%q' to format strings for integer types is a go vet error in
recent go versions, and likely a bug.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
The pattern 'type Alias *Document' does not strip methods under
encoding/json/v2 (GOEXPERIMENT=jsonv2), causing UnmarshalJSON to call
itself infinitely until the goroutine stack overflows (1GB limit).
Change to 'type Alias Document' with (*Alias)(d) cast — the standard
Go pattern that works correctly with both encoding/json v1 and v2.
Adds a regression test that uses debug.SetMaxStack to shrink the
goroutine stack limit to 8MB, making the overflow happen in milliseconds
rather than minutes if the recursion is reintroduced.
Ref: https://github.com/golang/go/issues/75361
Signed-off-by: Benjamin Grandfond <benjamin.grandfond@docker.com>
Thomas Bechtold
Akihiko Komada
Sebastiaan van Stijn
Rez Moss
Sai Asish Y
anchore-oss-update-botandanchore-oss-update-bot
Will Murphyandanchore-oss-update-bot
Weston Steimel
Alex Goodman
witchcraze
dependabot[bot]
David Dashti
anchore-actions-token-generator[bot]andspiffcs
Yoav Alon
nadimz
Benjamin Grandfond