Add integration tests covering the full archive extraction pipeline:
- tar.gz containing Python package metadata (single-level)
- zip containing a JAR with MANIFEST.MF (two-level, same format)
- Mixed archive types (zip + tar.gz at same level)
- Depth limit enforcement (3-level zip nesting, limited to 1)
- Extension exclusion filtering
- 3-level mixed format nesting: tar.gz -> zip -> tar.gz -> files
verifying cross-format extraction and full access path chains
- Reverse format crossing: tar.gz -> zip -> tar.gz verifying
content readability at all levels and correct relationship count
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Integrate the archive extraction orchestrator into the SBOM creation
pipeline, enabling recursive archive cataloging when configured:
- Add Archive field to CreateSBOMConfig with WithArchiveConfig builder
- Add Exclusions field to CreateSBOMConfig with WithExclusions builder
so user --exclude patterns also filter content inside extracted
archives, not only the source filesystem
- Add setupArchiveOrchestration() that creates the orchestrator when
MaxDepth > 0, runs the iterative extract loop, and returns the
composite resolver with archive contents visible to all catalogers
- Include archive config in configuration audit trail
- Add CLI archive config section (max-depth, size/file limits,
exclude-extensions) wired through Catalog.ToArchiveConfig()
- CLI Catalog.ToSBOMConfig threads cfg.Exclusions through
WithExclusions so child resolvers respect them
When MaxDepth is 0 (default), behavior is unchanged. When > 0, archives
discovered during scanning are extracted and their contents made visible
to all catalogers via the CompositeResolver.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add tests for important orchestrator edge cases:
- Nested archives (zip-inside-zip): verifies multi-round extraction
discovers archives within previously-extracted archives
- Context cancellation: graceful handling without hanging
- Multiple archives at same level: both extracted in one round
- Corrupt archive: skipped without error, doesn't block other work
- Empty archive: not counted as extracted
- Per-archive file limit: partial extraction handled correctly
- Relationship structure: contains relationships emit the archive's
coordinates as the source side
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Implement the Orchestrator that manages the iterative
extract-catalog loop for recursive archive discovery:
- Scans the composite resolver for candidate archive locations,
detecting format by peeking 512 bytes via FileContentsByLocation
rather than copying every non-archive file to a probe-temp file
- Extracts identified archives to per-archive temp directories
- Builds child resolvers via an injectable ResolverFactory (kept
pluggable so tests can substitute a mock resolver without
importing syft/internal/fileresolver)
- Tracks extraction state to avoid reprocessing
- Enforces safety limits (depth, per-archive size, file count,
total extraction budget)
- Generates ContainsRelationship entries linking each archive to
its extracted-contents filesystem
- Manages temp directory lifecycle with Cleanup()
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add tests for important edge cases:
- Nested children (child-in-child): verifies that archives within
archives produce correct multi-level AccessPath chains like
"/archive1.tar.gz:/inner-archive.zip:/deep-file.txt"
- Empty child resolver: no locations leak from empty archives
- Same file path in parent and child: both versions accessible
with distinct FileSystemID routing
- Context cancellation in AllLocations: no hanging
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Implement a file.Resolver decorator that composites a parent resolver
with child resolvers created from extracted archive contents. Files from
child resolvers appear transparently alongside parent files.
Key behaviors:
- FilesByPath/FilesByGlob/FilesByMIMEType delegate to parent AND children
- FileContentsByLocation routes to the correct resolver via FileSystemID
- Locations from children get archive-prefixed AccessPath (using ":"
separator matching Java cataloger convention) and unique FileSystemID
- Thread-safe via RWMutex for concurrent access
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Introduce the internal/archive package with a generic Extractor
interface for extracting archive contents to a destination directory.
Includes two implementations:
- ZipExtractor: handles zip-based archives (.zip, .jar, .war, etc.)
- TarExtractor: handles tar-based archives (.tar, .tar.gz, .tar.bz2, etc.)
Both extractors enforce configurable safety limits and protect against
common malicious-archive vectors:
- Per-archive extraction size limit
- Per-archive file count limit
- Zip-slip path traversal prevention via existing SafeJoin
- Symlink entries are written as real symlinks only when their target
resolves inside destDir; symlinks pointing outside are dropped to
avoid escape via symlink-then-write
- Non-regular non-symlink entries (devices, fifos, sockets) are skipped
because their contents can't safely round-trip to disk
Also provides FindExtractor for auto-detecting archive format, and
IsExcludedExtension for filtering by extension.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add configuration fields for generic recursive archive extraction:
- MaxDepth: maximum nesting depth for archive extraction (default 0 = disabled)
- MaxExtractionSizeBytes: per-archive extraction size limit (default 500MB)
- MaxFileCount: per-archive file count limit (default 10,000)
- MaxTotalExtractionBytes: total extraction size limit (default 2GB)
- ExcludeExtensions: archive extensions to skip during extraction
These fields lay the groundwork for cross-cataloger recursive archive
cataloging, where archives discovered during scanning can be extracted
and their contents made visible to all catalogers.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Thomas Bechtold