Archives may contain directory entries with read-only permissions
(e.g., META-INF/ with mode 0555 in JARs). When extracted to temp
directories, these restricted permissions prevented writing files
into those directories, causing "permission denied" errors for
subsequent file entries like META-INF/MANIFEST.MF.
Always use 0755 for extracted directories since they are temporary
working directories owned by the extraction process.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
The java-pom-cataloger was processing pom.xml files found inside
META-INF/maven/ directories, which are archive metadata embedded by the
Maven build process. These are verbatim copies of the original build POM
containing unresolved ${property} references and dependency versions
inherited from parent POMs that are unavailable inside the archive. This
caused phantom packages with UNKNOWN versions to appear when recursive
archive cataloging opened JARs and exposed their internal pom.xml files
to the pom cataloger.
The java-archive-cataloger already uses pom.properties (which always
contains resolved values) to identify these archives, so the pom
cataloger should skip META-INF/maven/ pom.xml files entirely.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add integration tests covering the full archive extraction pipeline:
- tar.gz containing Python package metadata (single-level)
- zip containing a JAR with MANIFEST.MF (two-level, same format)
- Mixed archive types (zip + tar.gz at same level)
- Depth limit enforcement (3-level zip nesting, limited to 1)
- Extension exclusion filtering
- 3-level mixed format nesting: tar.gz -> zip -> tar.gz -> files
verifying cross-format extraction and full access path chains
- Reverse format crossing: tar.gz -> zip -> tar.gz verifying
content readability at all levels and correct relationship count
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Integrate the archive extraction orchestrator into the SBOM creation
pipeline, enabling recursive archive cataloging when configured:
- Add Archive field to CreateSBOMConfig with WithArchiveConfig builder
- Add Exclusions field to CreateSBOMConfig with WithExclusions builder
so user --exclude patterns also filter content inside extracted
archives, not only the source filesystem
- Add setupArchiveOrchestration() that creates the orchestrator when
MaxDepth > 0, runs the iterative extract loop, and returns the
composite resolver with archive contents visible to all catalogers
- Include archive config in configuration audit trail
- Add CLI archive config section (max-depth, size/file limits,
exclude-extensions) wired through Catalog.ToArchiveConfig()
- CLI Catalog.ToSBOMConfig threads cfg.Exclusions through
WithExclusions so child resolvers respect them
When MaxDepth is 0 (default), behavior is unchanged. When > 0, archives
discovered during scanning are extracted and their contents made visible
to all catalogers via the CompositeResolver.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add tests for important orchestrator edge cases:
- Nested archives (zip-inside-zip): verifies multi-round extraction
discovers archives within previously-extracted archives
- Context cancellation: graceful handling without hanging
- Multiple archives at same level: both extracted in one round
- Corrupt archive: skipped without error, doesn't block other work
- Empty archive: not counted as extracted
- Per-archive file limit: partial extraction handled correctly
- Relationship structure: contains relationships emit the archive's
coordinates as the source side
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Implement the Orchestrator that manages the iterative
extract-catalog loop for recursive archive discovery:
- Scans the composite resolver for candidate archive locations,
detecting format by peeking 512 bytes via FileContentsByLocation
rather than copying every non-archive file to a probe-temp file
- Extracts identified archives to per-archive temp directories
- Builds child resolvers via an injectable ResolverFactory (kept
pluggable so tests can substitute a mock resolver without
importing syft/internal/fileresolver)
- Tracks extraction state to avoid reprocessing
- Enforces safety limits (depth, per-archive size, file count,
total extraction budget)
- Generates ContainsRelationship entries linking each archive to
its extracted-contents filesystem
- Manages temp directory lifecycle with Cleanup()
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add tests for important edge cases:
- Nested children (child-in-child): verifies that archives within
archives produce correct multi-level AccessPath chains like
"/archive1.tar.gz:/inner-archive.zip:/deep-file.txt"
- Empty child resolver: no locations leak from empty archives
- Same file path in parent and child: both versions accessible
with distinct FileSystemID routing
- Context cancellation in AllLocations: no hanging
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Implement a file.Resolver decorator that composites a parent resolver
with child resolvers created from extracted archive contents. Files from
child resolvers appear transparently alongside parent files.
Key behaviors:
- FilesByPath/FilesByGlob/FilesByMIMEType delegate to parent AND children
- FileContentsByLocation routes to the correct resolver via FileSystemID
- Locations from children get archive-prefixed AccessPath (using ":"
separator matching Java cataloger convention) and unique FileSystemID
- Thread-safe via RWMutex for concurrent access
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Introduce the internal/archive package with a generic Extractor
interface for extracting archive contents to a destination directory.
Includes two implementations:
- ZipExtractor: handles zip-based archives (.zip, .jar, .war, etc.)
- TarExtractor: handles tar-based archives (.tar, .tar.gz, .tar.bz2, etc.)
Both extractors enforce configurable safety limits and protect against
common malicious-archive vectors:
- Per-archive extraction size limit
- Per-archive file count limit
- Zip-slip path traversal prevention via existing SafeJoin
- Symlink entries are written as real symlinks only when their target
resolves inside destDir; symlinks pointing outside are dropped to
avoid escape via symlink-then-write
- Non-regular non-symlink entries (devices, fifos, sockets) are skipped
because their contents can't safely round-trip to disk
Also provides FindExtractor for auto-detecting archive format, and
IsExcludedExtension for filtering by extension.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Add configuration fields for generic recursive archive extraction:
- MaxDepth: maximum nesting depth for archive extraction (default 0 = disabled)
- MaxExtractionSizeBytes: per-archive extraction size limit (default 500MB)
- MaxFileCount: per-archive file count limit (default 10,000)
- MaxTotalExtractionBytes: total extraction size limit (default 2GB)
- ExcludeExtensions: archive extensions to skip during extraction
These fields lay the groundwork for cross-cataloger recursive archive
cataloging, where archives discovered during scanning can be extracted
and their contents made visible to all catalogers.
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Thomas Bechtold