`LocalPath` searched `FullNestedPath()` for the last `!/`, which is just `RealPath` for a file not in
an archive. A host file like `/data/Hello!/world.txt` came back as `/world.txt`, so lookups and
exclusions ran against the wrong path even with nested cataloging off. `ContainerPath` had the same
problem when `FullPath` was set to `RealPath`.
Both now cut `RealPath` off first and only look for a separator in the entry chain after it, which
can't contain `!/` since entry names with it are refused.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Adding an entry checked `HasPath` on every prefix of its path, so a name with thousands of
components did O(depth^2) hashing per entry before anything was charged.
- entry names over 256 path components are refused and reported like other unsafe names
- implied parent directories are found walking from the deepest parent up, stopping at the first
one that already exists
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
`entryName` only rejected NUL and over-long names, so whatever an archive called an entry flowed
straight into `FullPath`, the file tree and every encoder.
- a name containing `!/` could pose as a nested archive boundary and collide with a real nested entry
- control characters and unicode line separators could add lines to SPDX tag-value output
Both are now refused, and refused names (over-long ones included) mark the resolver truncated with
the quoted name instead of being dropped silently.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: catalog GGUF models larger than the header
gguf-parser-go v0.26.3+ rejects a header whose tensors declare more data than follows it. Syft parses only a 50 MB copy of each model, so any larger model is dropped.
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix: fix parser with rollback and verify with test
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix: bound GGUF tensor info before the parser reads it
The header pre-scan stopped after the metadata KVs and left tensor infos to gguf-parser-go. Older releases of the library allocate whatever length a tensor name declares, so a tiny file could force a multi-GB allocation, or a fatal out of memory that panic recovery can't catch.
- tensor names are bounded like metadata keys (non-empty, length limit, shared string budget)
- dimension count is capped at `GGML_MAX_DIMS`
- a tensor count larger than the header now fails at EOF in the pre-scan
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: give the GGUF header copy the model's real size
The cataloger copies at most 50 MB of a model and parses that copy. Newer gguf-parser-go releases check the tensor data the header declares against the file size, so the truncated copy of any larger model was rejected.
- after the header is validated, the copy is extended to the size the resolver reports for the model, leaving the tail as a hole the parser never reads
- sources that report no size (OCI model artifacts today) keep the old behavior
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore(deps): restore gguf-parser-go v0.26.4
The header copy now carries the model's real size, so the pin to v0.26.0 and its dependabot ignore are no longer needed. This also brings back upstream's string and array length bounds, and the library's own `IsNumeric`.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: keep the extended GGUF header copy sparse on Windows
NTFS reserves disk for the full length of a file extended with truncate unless the file is marked sparse, so cataloging a large model on Windows reserved the model's size in the temp dir until cleanup.
- the header copy is marked sparse (`FSCTL_SET_SPARSE`) before it is extended
- if the filesystem can't make it sparse (FAT32, exFAT), the copy is not extended, and models larger than it fail to parse as before
- `golang.org/x/sys` moves from an indirect to a direct dependency
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
The go-make module and its action need to move in lockstep, but dependabot bumps them in separate PRs that drift apart. Automation now bumps both together, so dependabot ignores go-make in both ecosystems.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* Add support for scanning IPK packages using the deb-archive-cataloger
Signed-off-by: Silas Parker <silas@srp.me.uk>
* fix(debian): share License field parsing between opkg status and archive control files
- the control-file `License:` fallback now goes through the same helper as the opkg status DB path, so a package reports the same licenses whether found installed or as an archive
- bitbake `&` / `|` are rewritten to SPDX `AND` / `OR` and the whole value is validated as one expression instead of split by hand, which kept grouping like `(A & B) | C` intact
- non-SPDX values fall back to a deduped list of names with operators and parens dropped (previously the status DB path emitted `&`, `(BSD`, etc as licenses)
- fields over 4KB are ignored, bounding work per archive and keeping deep paren nesting away from the recursive expression parser
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(debian): split Yocto recipe filenames in the Source field
Yocto writes the bitbake recipe filename (`Source: zlib_1.3.2.bb`) where dpkg expects a source package name. This is now split into `Source: zlib` / `SourceVersion: 1.3.2` for both ipk archives and opkg status DB entries, instead of surfacing the `.bb` filename as the upstream package.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(debian): do not label ipk archives as Debian packages
- `.ipk` archives no longer get a `pkg:deb` PURL from the assumed debian distro; opkg-based distros (Yocto, OpenWrt) have no purl type, so these now match packages found in an opkg status DB, which already get no PURL
- docs now cover `.ipk` and note that only ar-framed ipks are supported (OpenWrt's older gzipped-tar framing is rejected as an invalid archive)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test(debian): assemble the ipk fixture at build time instead of committing it
The fixture Dockerfile now builds a Yocto-shaped `.ipk` from a checked-in `control/` directory (copied from the original poky-tiny build) using `tar`, `zstd` and binutils `ar`, the same way bitbake's `package_ipk` packs one. This drops the prebuilt binary and the 20-30 minute Yocto `Dockerfile.build` while keeping the same archive layout and control metadata.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Silas Parker <silas@srp.me.uk>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(redact): reset redaction store between command runs
cli.Command() may be invoked more than once in the same process (e.g.
when syft is embedded as a library). The clio initializer calls
internal/redact.Set on every command execution, but the redact store is
process-global and Set panics when a store already exists, so the second
invocation dies with "replace existing redaction store (probably
unintentional)".
Add redact.Reset() to clear the previous run's store and call it in the
initializer before Set. The double-Set guard in Set is kept: an
unexpected second Set within a single run still panics.
Add TestAppClioSetupConfigInitializerCanRunMultipleTimes which runs the
initializer twice; it panics with the exact reported message on the old
code and passes with the fix.
Fixes#2285
Signed-off-by: JasonMetal <935216773@qq.com>
* fix(redact): release the redact store at the end of each run
Clear the global redact store in the post-run hook instead of resetting it
before every `Set`. The double-`Set` panic keeps its meaning: a store is only
present while a run is in flight, so a second run starting mid-run still
panics rather than splitting secrets across two stores and leaking the ones
in the dropped store. Sequential runs in the same process work since the
previous run releases its store on the way out.
A run only releases the store it set, so it never clears a store owned by
someone else.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: JasonMetal <935216773@qq.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* feat: add containers-storage support for Podman and Buildah, update tests and documentation
Signed-off-by: Bruce Clark <bruce.clark@mandg.com>
* test: update containers-storage tests to validate package.json and handle missing images
Signed-off-by: Bruce Clark <bruce.clark@mandg.com>
* test: fix rootless containers-storage test setup
Install uidmap for Podman and Buildah CI jobs.
Align temporary rootful and rootless storage paths.
Clean up rootless stores within the builder's user namespace.
Signed-off-by: Bruce Clark <bruce.clark@mandg.com>
* fix: only use containers-storage when explicitly requested
- containers-storage is no longer part of automatic image resolution; it's used only for `--from containers-storage`, the `containers-storage:` scheme, or `default-pull-source: containers-storage` (now an accepted value). Opening a store runs full graph driver init (mounts, locks, store writes), which is too invasive as a probe on every plain image reference.
- rootless stores must be read from inside the builder's user namespace (`podman unshare syft ...`), documented in the README.
- the `containers_image_openpgp` tag is now applied to unit/integration tests and lint, so dev tooling compiles the same provider that ships.
- containers-storage CI runs through `make containers-storage-test` and covers both vfs and overlay stores.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: exclude cgo btrfs driver and unmount overlay store in cleanup
- cgo-enabled linux builds (`-race` tests, lint) compile go.podman.io's btrfs graph driver, which needs libbtrfs headers. Add `exclude_graphdriver_btrfs` alongside `containers_image_openpgp` everywhere the tag is set; release builds are `CGO_ENABLED=0` and never included it.
- the overlay driver bind-mounts its graphroot inside the rootless user namespace, so the containers-storage test unmounts it before removing the store, and cleanup no longer fails the run.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Bruce Clark <bruce.clark@mandg.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Bruce Clark <bruce.clark@mandg.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ci: run the full unit suite on windows via make unit
Drops the curated `unit:windows` subset; the windows job now bootstraps like linux and runs `make unit`. Exceptions get added as failures show up.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ci: skip fixture refresh for windows unit tests
binny has no windows release, so the bootstrap action can't install the tools the fixture refresh needs.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: skip linux container fixtures when docker can't run them
Windows CI runners only run windows containers, so image fixtures go through `testutils.SkipWithoutLinuxContainers`. No behavior change on linux/mac.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: close leaked file handles and posix squashfs paths on windows
- output writers are now closed on early returns in `scan`/`convert` (open handles block cleanup on windows)
- `newSBOMMultiWriter` used `path.Dir` on host paths, so output subdirectories were never created on windows
- squashfs walk paths were built with `filepath.Join`
- skip a few posix-only test cases (dir perms, xdg fallback, go-rpmdb ndb handle leak upstream)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(golang): resolve go source module dir natively on windows
- `parseGoModFile` passed the resolver's posix (`/c/...`) path to the go toolchain, so go source resolution silently found nothing on windows
- skip container-built fixtures (go archs, java builds) and golden image tarballs on windows; stereoscope's layer cache filenames contain `:`, which windows rejects
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: keep posix paths posix on windows
- `ChrootContext.ToChrootPath` nested volumes (`D:\cwd\C:\root`) when the scan root was on another drive, so the prefix was never trimmed
- bitnami, python wheel/egg, and alpm parsers used `filepath` on posix location paths
- alpm backup-file lookups silently missed on windows
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ci: pull the test fixture cache for windows unit tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: don't map virtual file tree paths through the host on windows
`FiletreeResolver` without a chroot (snaps) ran request paths through `filepath.Abs`, rooting them on the current volume (`/payload.txt` -> `/d/payload.txt`), so snap scans on windows found nothing.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: match tar entry globs as posix on windows
- `matchesAnyGlob` used `doublestar.PathMatch` (host separator) on tar entry names, so tar-wrapped java archives silently produced nothing on windows
- test helpers now treat binary snippet keys and `FromFile` locations as posix
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* incorporate windows path fix
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* incorporate go-rpmdb close fix
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* close squashfs fixture writers before finalizing
- windows dir listings can report a zero size for files still open, so the manifest was packed empty
(also adds a temporary windows diagnostic for the binary cataloger)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* temp: surface windows binary cataloger diagnostics
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: keep directory symlink targets posix on windows
- the directory indexer joined symlink targets with native separators, so links in the posix file tree pointed nowhere and globs on link names found nothing
- binary fixture logical keys used native separators, so nested snippets were picked as the fixture dir on windows
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* keep addSymlinkToIndex under the funlen limit
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* drop windows skips covered by the stereoscope and go-rpmdb bumps
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(dotnet): resolve packages.lock dependencies within their target framework
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix(dotnet): make packages.lock package metadata deterministic across target frameworks
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix: parse packages lock tests
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix: update lockfile parse to read into stable structure
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix: remove stale comments
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* test: new test for project entry case
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
* fix(dotnet): key packages.lock packages case-insensitively
NuGet IDs are case-insensitive, and edge lookup already treated them that way, but package identity used the exact-case name. Two spellings of one ID and version across target frameworks became two packages, and type precedence never ran between them.
- packages merge on the lowercased name + version, display name comes from the winning entry
- a package listing itself (under any casing) no longer produces a self edge
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
The installer word-split the original arguments when piping into a tagged release's `install.sh`, so `-b '/tmp/syft install path'` arrived as three words and the trailing word was treated as a release tag.
- argument parsing moved into `parse_install_args` so `main` keeps its original args
- tagged script is invoked with `"$@"` instead of the unquoted `PROGRAM_ARGS`
Port of anchore/grype#3738
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: sign release checksums with a sigstore bundle
The cosign update rejects the legacy `--output-certificate` / `--output-signature` flags alongside the new bundle format, which broke signing during the v1.53.0 release.
- checksums are now signed into a single `checksums.txt.sigstore.json` bundle (signature, certificate, and tlog proof)
- `install.sh -v` verifies with `--bundle` for releases >= v1.53.0 and keeps the `.sig` / `.pem` flow for older releases
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore: smooth the cutover to sigstore bundle verification
- `install.sh -v` now hints that bundle verification needs cosign v2.5.0 or newer when verification of a bundle-signed release fails
- install tests keep exercising the legacy `.sig` / `.pem` path against v1.52.0 once the latest release is bundle-signed
- touch the install test environment Dockerfiles so the cached images rebuild with a current cosign
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: only hint at the cosign minimum when cosign is actually too old
- `install.sh -v` used to print the cosign v2.5.0 note on any failed bundle verification, including a missing bundle or a bad signature. It now only prints it when cosign reports it couldn't read the bundle (`bundle does not contain cert`), which is how v2.2.4 through v2.4.1 fail on a sigstore bundle.
- `verify_sign` is now covered with a stub cosign, so the bundle path is exercised before any bundle-signed release is latest
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: stop the install when a release asset is missing or fails its checksum
- `install.sh` now stops when the checksums file, the signature bundle, or the asset can't be downloaded, instead of handing an empty or error-page file to the next step
- a sha256 mismatch between the asset and the checksums file now stops the install. Before this the mismatch was logged and the asset was installed anyway, so `-v` only ever vouched for `checksums.txt`, not the binary
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* chore: pin cosign in the install test images
The install test images pulled whatever cosign was latest, unverified. They now install cosign v3.1.3 checked against its published sha256, so the install tests always run with a cosign that can verify sigstore bundles (releases >= v1.53.0 need v2.5.0 or newer).
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
* fix: revert hashstructure to v0.6.0 to keep artifact IDs stable
hashstructure v1 changed `SlicesAsSets` hashing (every slice now gets a finalizing hash pass, and duplicate elements are deduplicated), which changed nearly every artifact ID syft produces.
- pin `github.com/gohugoio/hashstructure` back to v0.6.0 and have dependabot ignore major bumps
- make the aggregate `Unit tests` job fail when a dependency fails; previously it was skipped, which satisfies the required check and let #5277 auto-merge with failing unit tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ci: fail unit test gate on any failed, cancelled, or skipped dependency
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* ci: require every unit test dependency to succeed
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Alex Goodman
Keith Zantow
Christopher Angelo PhillipsandAlex Goodman
dependabot[bot]
Weston Steimel
anchore-oss-update-botandanchore-oss-update-bot
Keith Zantow
Peter McConnell
Silas ParkerandAlex Goodman
MetalandAlex Goodman
