The package supplier is optional in SPDX, but fileSource dereferenced it
without the nil check containerSource already has, so a valid SPDX 2.3
document whose described package has primaryPackagePurpose FILE and no
supplier crashed syft convert and grype sbom: input.
A JSON null in files, a package's files or externalRefs (or an SPDX 2.1
package the version converter cannot map) decodes to a nil pointer that
the conversion then dereferences. Drop nil entries once at the start of
ToSyftModel instead of guarding each loop.
Signed-off-by: Tony <tjagodka@gmail.com>
the index was ported from a prototype with per-node `RWMutex` lock upgrading and JSON marshalling. Its only consumer (`archive.Resolver`) builds it sequentially during extraction and then only reads it, so neither was used. `lock.go` also compared method-value pointers to tell the read lock from the write lock, which can't distinguish lock instances.
removed the locks, the upgrade and retry paths that existed only for concurrent writers, and the JSON methods and their test. The radix logic is unchanged, and the README now points at `archive.Resolver` rather than a nonexistent `archive.Index`.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
the SPDX writer only kept relationships whose `From` is a package, so the `file CONTAINS package` edges the archive task records were dropped. For anything that isn't java (npm in a tgz, python in a zip) that edge is the only thing saying which archive a package came out of.
SPDX now also writes `CONTAINS` from a file to a package. The file end is always in the document, since `toFiles` covers every coordinate a relationship names. Other file-side relationships (e.g. binary's file `DEPENDENCY_OF` package) are still dropped as before.
CycloneDX is unchanged: it has no file components to hang the edge on, and war-to-jar structure reaches it through the java `DEPENDENCY_OF` parity fix.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
SPDX wrote `FileName` from `RealPath` only, so two archives' `META-INF/MANIFEST.MF` became two SPDX files with the same name and nothing saying which archive each came from. Worse, a consumer reading just the name would take a nested file for one at that path in the scan root.
- a file inside an archive is now named by its full chain, using the same `:` convention and escaping as `ArchivePath` / java's `VirtualPath`: `app.war:WEB-INF/lib/a.jar:META-INF/MANIFEST.MF`
- `FileComment` gains an `archivePath: ...` line (next to the existing `layerID: ...`, one per line), so decoding recovers `ArchivePath` and the entry's own `RealPath` without guessing where the chain ends
- files outside archives are unchanged
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
`isArchiveMetaPom` was an unconditional `strings.Contains(path, "META-INF/maven/")`, so the pom cataloger dropped real poms everywhere, even with the feature off: exploded wars (`WEB-INF/classes/META-INF/maven/...`), test fixtures under `src/test/resources`, source tarballs. That's a regression on the default path.
the skip exists because inside a jar those poms are raw build copies that java already covers from `pom.properties`. So it now applies only when the traversal on the context is a java archive. Outside one, or inside a tarball java doesn't describe, the poms are cataloged as before this PR.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
the legacy recursion links a nested jar to the jar that holds it: a `DEPENDENCY_OF` edge from inner to outer, and `JavaArchive.Parent` on the inner package. Inside an extracted archive java called `parse(ctx, nil)`, so both went away as soon as the feature was on. What replaced them was `file CONTAINS package`, which SPDX and CycloneDX drop, so those formats lost all war-to-jar structure.
- `archive.Traversal` gains `Parent` and `Packages` (the packages found describing that archive, filled in by the task after its sub-pipeline runs)
- java picks the enclosing jar's main package from the parent traversal (the one whose virtual path is the archive's own; pom-derived aux packages carry a suffixed one) and passes it to `parse`, then sets `Parent` the same way the legacy path does
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
with nested cataloging on, java leaves root-level archives to the archive task, so a root `app.jar` package only exists once that task has run. With the default `FilesOwnedByPackageSelection` the root file catalogers ran in an earlier group, never saw the package, and `app.jar` got no file metadata or digests. It did get them with the feature off.
- in owned-by-package mode the groups are now packages, then the archive task, then the root file tasks. Other modes are unchanged
- the file tasks merge their results into the SBOM instead of replacing the maps, since the archive task has already written nested files into them
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
with `FilesOwnedByPackageSelection` the file catalogers take every package location in the SBOM and look it up by `RealPath` in the resolver they were given. Inside an archive, the java package's location is the archive itself *in its parent's frame* (`WEB-INF/lib/dep.jar`), so the sub-pipeline looked that path up inside `dep.jar`'s own resolver: a miss, or an unrelated same-named entry by coincidence. At the root, coordinates from inside archives were looked up against the scan root the same way.
`coordinatesForSelection` now keeps only coordinates whose `ArchivePath` matches the current frame (the traversal's `ContentsArchivePath()`, or blank at the root).
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
a read error while copying one entry (a zip CRC mismatch, a corrupt deflate stream) was returned from `add`. That aborted the walk, and `Extract` threw away every entry already stored. So a large war with one damaged, irrelevant entry produced zero packages with the feature on, while java's legacy path (which reads only the entries it needs) still cataloged it.
- a read error on one entry now drops that entry and marks the resolver truncated (`entry "x" could not be read: ...`), and the walk continues
- if the stream itself breaks partway (a truncated `.tar.gz`), whatever was read before the break is kept and the archive is truncated. With nothing read it is still an error, as before
- disk limit and decompression budget errors still stop the walk
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
the archive's `CONTAINS` edges were built from every coordinate in the sub-pipeline's scratch SBOM, which gets one of two things wrong:
- with the default `FilesOwnedByPackageSelection`, nothing records the nested archives as files, so the archive-to-archive chain was missing and grandchild packages hung off coordinates with no incoming edge
- the scratch SBOM also holds coordinates from the parent's frame, most notably unknowns recorded at the archive's own coordinates (e.g. a jar with no manifest). Those turned into `x.jar CONTAINS x.jar`
edges now go to the nested archives found in it (always, whatever the file selection), plus coordinates whose `ArchivePath` is this archive's own chain, i.e. files that are actually inside it. The walk reuses that one discovery pass for the recursion.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
with nested cataloging on, java stops recursing into jars itself (the task owns recursion) and its own "nested archives not cataloged" unknown is suppressed. The task then just returned at the depth bound. So `MaxDepth: 1` on a war silently found fewer jars than leaving the feature off, and nothing in the SBOM said so.
at the bound, every archive that sits there (by content sniff) is now recorded as a "nested archive not cataloged: the nested archive depth limit was reached" unknown. The CLI description for `nested-archive-max-depth` notes that it replaces java's unbounded jar-in-jar recursion.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
the limiter bounds what's held *at once*, and each archive gives its bytes back when it's released. So a recursive bomb (42.zip shape) never trips a limit: every level fits on its own, and the scan just runs for effectively forever. Capping each archive at N× its own size wouldn't help either, since every level of such a bomb is high-ratio, so the work still grows about N-fold per level.
- each top-level archive gets one decompression budget of `max(100 × its size, 1 GiB)`, spent by every archive nested under it at any depth
- the budget is attached to the context when the walk enters a top-level archive, so `Extract` picks it up without a signature change
- once it's spent, the archive being extracted is truncated with a "possible decompression bomb" unknown and the walk moves on. The next top-level archive starts fresh
real trees (a war of jars, a layer of tarballs) expand a few times over, well under the ratio.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
`put` had no per-entry cap, unlike the legacy java path which capped every file at 2 GB through `intFile.SafeCopy`. At depth 1, one entry of an overlapping-entry zip bomb could decompress all the way to the 100 GB disk limit before anything stopped it.
- entries are now read through the same 2 GB cap (`intFile.PerFileReadLimit`, now exported). An entry over it is dropped after cap+1 bytes and the rest of the archive is still extracted
- `Resolver` gains `TruncatedReason`, so the truncation unknown says what happened ("an entry larger than 2.0 GiB was skipped", or the existing disk-limit text) instead of always blaming the limits
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
a negative `MaxDepth` really was unbounded, and nothing noticed an archive nested inside a copy of itself. A zip quine at `-1` recursed until memory, disk or file descriptors ran out, running the full sub-pipeline at every level.
- the walk keeps a stack of the digests of the archives it's inside, and an archive identical to one of them is recorded as an unknown instead of being entered
- negative depth now stops at 16 levels (docs and CLI description updated)
- cancellation is checked between candidates, so a cancelled scan stops walking instead of finishing the archive tree
the three unknown-recording blocks in `processArchive` became a one-line helper to stay under `funlen`.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
`identifyFormat` accepted any extractor mholt identified, which put the 7z and rar decoders (`bodgit/sevenzip`, `rardecode`) on untrusted input with no test coverage. Only zip, tar and compressed tar are opened now, until other formats are deliberately supported.
This also fixes java resource adapters: a `.rar` that is really a zip used to be identified by name as RAR and fail to extract. An unsupported identification now falls through to the zip end-of-central-directory check, so it's read as the zip it is.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
sub-pipeline catalogers were already run under `runTaskSafely`, but extraction wasn't: the zip, tar and compressor decoders run on untrusted bytes, and a panic in one unwound the whole walk and failed `CreateSBOM` with "failed to run tasks".
- `processArchive` recovers and returns the panic (with its stack) as an error at the archive's coordinates, so it becomes an unknown and the sibling archives are still cataloged
- `Extract` now defers `Cleanup` on failure instead of calling it on each error path, so a panic no longer leaks the limiter charge or the spill file
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
each `ArchivePath` segment came from `loc.Path()`, so it was the link path when an archive was reached through a symlink and the real path otherwise. That depended on how discovery happened to find the archive. The package location said `store/3f9a1c` while the files inside the same jar were recorded under `...:app/lib/json-simple-1.1.1.jar`, so a consumer couldn't join a package to its files by prefix.
The chain for extracted files is now built from `RealPath`. `Traversal.Location` still carries the access path, so java's name check and `VirtualPath` metadata behave the same as before. The field comment and schema now say every segment has links resolved.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
`discoverArchives` collects MIME hits first, and those are bare real paths. When the glob pass then found the same file through a symlink, it was dropped as a duplicate along with its access path. Java decides whether to describe an archive by its name, so a content-addressed jar (`store/3f9a1c`, linked from `lib/json-simple-1.1.1.jar`) was extracted but produced no package. This is the pnpm / nix / bazel layout.
When deduping, the candidate now takes the first location that carries an access path.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
`:` was escaped as `%3A`, but `%` itself wasn't, so an entry named `a:b.jar` and one named `a%3Ab.jar` got the same `ArchivePath`. Their contents then collide in `FileMetadata` / `FileDigests` (last writer wins) and share artifact IDs.
`%` is now escaped as `%25` before `:`, which makes the encoding injective. The encoding is documented on `Coordinates.ArchivePath` and in the schema.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
syft-json output wasn't reproducible for nested archives. `AllCoordinates()` comes from map iteration, and the coordinate and location sorters ignored `ArchivePath`, so the same path in two archives (e.g. `META-INF/MANIFEST.MF` in two jars in a war) compared equal and came out in random order. Package locations had the same problem.
- `ArchivePath` is now the final tiebreak in `LocationSorter`, `CoordinatesSorter` and the `CoordinateSet.ToSlice` fallback
- `Location.Equals` and `Location.String` include `ArchivePath`, which matches `Coordinates`, since the same path in different archives is a different file
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
nothing in the output said nested archive cataloging ran, or how deep it went: `Archive` wasn't in the configuration audit trail, and `archive-cataloger` never showed up in `catalogers.used`.
- the audit trail gains an `archive` section with the top-level archive config
- `catalogers.used` includes `archive-cataloger` when the task is scheduled
- the `packages` section now reports java's config after the archive search booleans are overlaid, i.e. what java actually used
`descriptor.configuration` is untyped in the schema, so no schema change.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
`ArchiveSearchConfig` lived in two places and each half was dead in one of them: the top-level booleans were never read (only java's copy was), and java's depth and limits did nothing. A library caller setting `Packages.JavaArchive.MaxDepth = 3` got no nested cataloging, and no error.
- java's embedded `ArchiveSearchConfig` and `WithArchiveTraversal` are deprecated; `WithMaxParentRecursiveDepth` replaces the latter's other half
- `CreateSBOM` now rejects `MaxDepth`, `MaxMemoryBytes` or `MaxDiskBytes` set on java's copy with an `invalid configuration` error
- the top-level `IncludeIndexedArchives` / `IncludeUnindexedArchives` are overlaid onto a copy of java's config, and win when they differ from the default. Callers who only set java's copy see no change
- `java.DefaultArchiveCatalogerConfig()` now carries only the booleans, so its descriptor no longer reports limits that java never used
- the CLI fills only the top-level config, and the option descriptions now say the search booleans are ignored when nested cataloging is on
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
`ArchiveSearchConfig{MaxDepth: 2}` is the natural literal, but zero memory and zero disk meant "forbid both", so every archive was truncated to nothing and only unknowns came out.
- `MaxMemoryBytes` / `MaxDiskBytes` of `0` now mean the default, negative is still unbounded
- the limiter keeps its own zero-forbids semantics internally; it's just no longer reachable from config
- tests that relied on zero to force spilling now pass `archive.Limits` directly, and the two "both limits zero" tests are gone since that config no longer exists
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
matches how the per-cataloger progress rows are titled (`Default` only), so the archive row reads like its siblings.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
- log lines shown under the TUI were glued onto the last UI row, and the last row could get erased (and overwritten by the report) when the TUI exits. Fixed in `bubbly` v0.2.2
- log colors were lost because the logger latched its TTY check on the first entry, which went to the TUI footer buffer. `go-logger` v0.2.0 checks per entry and supports `TerminalWriter`, which the footer writer now implements (`NO_COLOR` is still honored)
- log lines under the TUI get a dim `Logs (non-fatal, see -v for more):` title with a branch pointing at the first entry, so it's clear what the trailing lines are
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Docker Hub rebuilt and re-pushed `busybox:1.38.0`, which changed the per-platform config digests and broke `TestMultiPlatformOCIImageSelection` on uncached runs.
- pin the fixture to the busybox 1.38.0 index digest so future re-pushes can't shift it
- update the expected per-platform digests to match
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(java): resolve Groovy 4+ group IDs to org.apache.groovy
Groovy 4.0 moved from the Codehaus coordinates (org.codehaus.groovy) to the
Apache Software Foundation (org.apache.groovy). JARs that ship no pom
metadata fall through to the static artifact-to-group map, which carries the
3.x-era org.codehaus.groovy entries for the whole groovy family, so 4.x
artifacts were attributed to an obsolete group even though their manifest
identifies the Apache vendor.
Where the static map attributes an artifact to the historical group, a
major-version migration resolves groovy 4.x and newer to org.apache.groovy,
while 3.x and earlier keep their historical group. The Eclipse-hosted
groovy-eclipse-* artifacts, which share the prefix but kept their Codehaus
coordinates, are excluded.
Fixes#5311
Signed-off-by: texasich <101962694+texasich@users.noreply.github.com>
* refactor(java): share version-aware groovy group ID lookup with CPE generation
- move the artifact-to-group lookup behind `cpegenerate.ArtifactIDToGroupID(artifactID, version)` so the PURL and CPE paths agree; the exported mutable map is now unexported
- version-dependent entries are expressed as a `groupIDMigration{before, after, atMajor}` (groovy moved to `org.apache.groovy` at 4.0); unconditional entries, including `groovy-eclipse-*` and apache-only 4.0+ modules, stay in the plain map
- add apache-only modules (`groovy-ginq`, `groovy-contracts`, `groovy-callsite`, ...) that otherwise took the group from their per-module `Automatic-Module-Name`
This changes CPEs for pom-less groovy 4+ jars (all real ones): vendors are now `apache` (e.g. `cpe:2.3:a:apache:groovy:4.0.33`) instead of `codehaus`, matching the PURL, jars with pom metadata, and NVD.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: texasich <101962694+texasich@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
closes#5299: docker:// references are OCI images, not GitHub repos.
Returning empty string produces a package with no PURL, matching the
treatment for local composite actions.
Signed-off-by: Yunare Maia <yunare@gmail.com>
An extra-dep can be pinned as "@sha256:<hash>,<size>" or as "@rev:<n>". The hash
slice assumed the first form, so a rev pin left the end index at -1 and the
parser panicked with slice bounds out of range. A rev pin carries no package
hash, so only the sha256 form now yields one.
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
Antoni (Tony) Jagodka
Sverrir Sigurdsson
anchore-oss-update-botandanchore-oss-update-bot
dependabot[bot]
Alex Goodman
texasichandAlex Goodman
Yunare Maia
Arpit Jain
somaz
Keith Zantow