* fix(binary): detect multi-arch ingress-nginx
Add v1.9.6 ARM64, ARMv7, and s390x fixtures from the published controller image and match their version marker against the nearby Go build version.
Assisted-by: OpenAI Codex
Signed-off-by: Shurong Cao <170531907+CAOShurong@users.noreply.github.com>
* fix(binary): match ingress-nginx release marker on all arches
The release is injected with `-ldflags -X`, so it lands in its own aligned,
NUL-padded data symbol. Only that padding is portable -- the surrounding bytes
are an arch-specific float constant pool, which is why the existing matchers all
anchor on incidental junk like `$a` and `S=v<y5` and only ever worked on amd64.
Matching `v<version>` followed by two NULs finds exactly one hit in every v1.9.6
binary (amd64, arm, arm64, s390x) and in all nine amd64 releases already under
test. Two NULs matter -- with one, s390x matches a vendored `v1.19.0` earlier in
the file and reports the wrong version.
This also drops the ~8KB wildcard window the marker previously needed, so the new
fixtures are ordinary 369 byte snippets instead of 8.4KB ones.
Two things in the fixture tooling had to move for that to work:
- `config.yaml` pinned the multi-arch *index* digest for all four platforms, but
extraction runs `docker create <ref>` with no `--platform`, so every entry
resolved to whichever image was pulled last. `make download` failed outright on
`linux/arm/v7`, since the registry labels that platform `armv7` and docker
normalizes it to `arm`. Now pinned per-platform, matching the redis entry.
- `write-snippet` only recognized EM_X86_64 and EM_AARCH64, so anything else
landed in a `linux-unknown-<hex>` directory and had to be placed by hand.
Verified against the real binaries with `-must-use-original-binaries`.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(binary): put non-amd64 snippets where the tests look for them
`write-snippet` used to fall back to `unknown-<hex>` for any ELF machine it
didn't recognize, so three snippets were sitting in directories nothing reads:
helm/3.12.0/linux-unknown-454d5f53333930 (hex of "EM_S390")
helm/4.1.4/linux-unknown-454d5f53333930
redis-server/7.2.5/linux-unknown-454d5f333836 (hex of "EM_386")
The test table asks for `linux-s390x` and `linux-386`, so those three cases were
quietly falling through to downloading the full binaries instead. Renamed to
match, and taught the tool about EM_386 and EM_RISCV so it stops producing dead
directories (there is an existing `linux-riscv64` snippet it also couldn't have
written).
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Shurong Cao <170531907+CAOShurong@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Shurong Cao <170531907+CAOShurong@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(elf): bound compressed ELF section reads
`debug/elf` takes a section's decompressed size from that section's own
compression header, and a highly compressible stream really does deliver the
bytes that header promises, so `internal/saferio` does not help: it faithfully
allocates every one of them. A 2MB input file drives `elf.NewFile` to allocate
over 10GB and return no error, which is a fatal OOM rather than a recoverable
panic.
Which sections get read is not up to the caller. `elf.NewFile` always reads the
section-name string table, and `File.Symbols` reads `.symtab` plus whatever
section its `Link` field points at, so being selective about sections is not
enough to avoid it.
New `elfutil.NewFile` is a drop-in for `elf.NewFile` that rejects a declared
decompressed size over 128MB. Every production call site goes through it, and a
ruleguard rule keeps the next one from going direct.
The check runs in two parts, since `debug/elf` expands sections at two different
times. The section-name string table is the only one `elf.NewFile` expands
itself, so it is checked against the raw bytes before the call; everything else
is expanded lazily by `(*Section).Open` and is checked after the parse, where
names, types and decompressed sizes are already resolved.
Only the sections syft can actually reach are bounded, which keeps the guard
from costing real binaries. DWARF is excluded since nothing calls `File.DWARF`,
so a large compressed `.debug_info` no longer skips the whole file, and sections
`debug/elf` will not decompress anyway (`SHF_ALLOC`, `SHT_NOBITS`) are left
alone. The legacy `.zdebug` form is matched on the section name the way
`debug/elf` gates it rather than on the `ZLIB` magic, so an ordinary section
starting with those four bytes is not mistaken for a compressed one.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix(elf): gate debug/buildinfo behind the compressed-section check
`debug/buildinfo.Read` opens ELF files with `debug/elf` itself, and `elf.NewFile`
expands the section-name string table as it parses, so the golang cataloger was
still reachable by the same bomb `elfutil` exists to stop. A 261KB fixture drove
1.4GB of allocation through `buildinfo.Read` and returned no error.
`elfutil.CheckSectionNameTable` is now exported for that case: callers that cannot
use `NewFile` because the `debug/elf` call is made for them inside another package.
Both `buildinfo.Read` call sites go through it, including the UPX-decompressed one.
Also corrects claims that did not hold up:
- the package doc's 2MB-to-10GB figure is not reachable with zlib (~1000:1), so it
now carries the measured 510KB-to-2.6GB, and names zstd's 32767:1 since that is
what makes the small inputs possible
- `.go.buildinfo` was listed as a hot-path section elfutil covers, but it is read
through `debug/buildinfo` and never touches `Section.Data`
- the graalvm comment claimed routing size rejections away from `*elf.FormatError`
improved reporting; both branches are skipped by the caller and only the
FormatError branch logs, so it did the opposite
- `sharedLibraries` logged short and truncated files as real ELF failures, since
`debug/elf` returns a bare `io.EOF` rather than an `*elf.FormatError` for those
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* added test comments around the negative cases
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* additional tests
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* better decomposition and comments
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* use a less brittle constant for error detection
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* report each hardlink as its own file when scanning images
image scans previously collapsed a set of hardlinks onto a single file, so only
one path per inode showed up in results. dir scans report every hardlink path,
which made image vs dir SBOMs of the same filesystem diverge (and produce
different SPDX `packageVerificationCode` values for packages that own hardlinked
files).
now both image resolvers (squash and all-layers) surface each hardlink at its
own path as a regular file bound to the target's content, matching dir scans.
user-facing impact:
- SBOMs for images containing hardlinks will list more `file` entries
- SPDX `packageVerificationCode` values change for affected packages, now
matching the equivalent `dir:` scan
- adds `file.NewVirtualLocationFromImage` to the public API
fixes#5019
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix busybox test assertion
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Update the binary classifier cataloger developer README to match the
current workflow:
- Testing section used the non-existent flag -must-use-full-binaries
against .../testdata/...; the real flag is -must-use-original-binaries
and Test_Cataloger_PositiveCases lives in the binary package. Fix the
command and add a single-fixture variant.
- Point add-snippet references at the make target and note the (Y/n/q)
prompt requires y to write the snippet.
- Fix the test file name (classifier_cataloger_test.go) and document the
go generate ./internal/capabilities step for new classifiers.
Fixes#4510
Signed-off-by: alliasgher <alliasgher123@gmail.com>
BPL (Borland Package Library) files are standard PE/DLL format used by
Delphi and C++Builder. Adding the extension to the glob list so syft
picks them up during directory scans without users needing to rename
to .dll first.
---------
Signed-off-by: jfjrh2014 <jfjrh2014@gmail.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Co-authored-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
The elixir-binary and elixir-library classifiers' regexes only matched
the bare semver triplet (and a single sub-segment for the library), so
release-candidate elixir images were either missed entirely or had
their version truncated:
$ syft -q elixir:1.12.0-rc | grep elixir # nothing
$ syft -q elixir:1.13.0-rc.0 | grep elixir
elixir 1.13.0 binary # truncated, "-rc.0" lost
Extend the version capture group to optionally include
"-<a-z0-9>+(\\.<digits>)?" so "1.12.0-rc.1", "1.13.0-rc.0", etc. match
exactly as the elixir.app and the binary's ELIXIR_VERSION line have
them.
Add a logical fixture under testdata/classifiers/snippets/elixir/
1.12.0-rc.1/linux-amd64 (cloned from the existing 1.19.1 fixture with
just the version strings changed) and register it in
Test_Cataloger_PositiveCases.
Closes#4819
Signed-off-by: Chris (ChrisJr404) <11917633+ChrisJr404@users.noreply.github.com>
Co-authored-by: Chris (ChrisJr404) <11917633+ChrisJr404@users.noreply.github.com>
The JRuby project migrated their downloads from S3 to GitHub Releases,
causing the old S3 URLs to return HTTP 403 Forbidden and breaking test
fixture image builds.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
BUILDPLATFORM is automatically set to the host's platform in new Docker,
so having it defined as an arg results in it being overridden by this
automatic value. Since it was always assigned to a literal string in the
test files, just use that string.
Additionally, image platform is better pulled from the manifest, not the
image config, in containerd store, so try that first.
Additionally, python3 is on PATH on new macs by default, but not python.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* migrate fixtures to testdata
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: correct broken symlinks after testdata migration
The migration from test-fixtures to testdata broke several symlinks:
- elf-test-fixtures symlinks pointed to old test-fixtures paths
- elf-test-fixtures needed to be renamed to elf-testdata
- image-pkg-coverage symlink pointed to test-fixtures instead of testdata
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: handle missing classifiers/bin directory in Makefile
The clean-fingerprint target was failing when classifiers/bin doesn't
exist (e.g., on fresh clone without downloaded binaries).
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add gitignore negation for jar/zip fixtures in test/cli
The jar and zip files in test/cli/testdata/image-unknowns were being
gitignored by the root .gitignore patterns. This caused them to be
untracked and not included when building docker images in CI, resulting
in Test_Unknowns failures since the test expects errors from corrupt
archive files that weren't present.
Add a .gitignore in test/cli/testdata to negate the exclusions for
these specific test fixture files.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* switch fixture cache to v2
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: update expected versions for rebuilt fixtures
Update test expectations for packages that have been updated in
upstream repositories when docker images are rebuilt:
- glibc: 2.42-r4 → 2.43-r1 (wolfi)
- php: 8.2.29 → 8.2.30 (ubuntu/apache)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* upgrade go
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add go-shlex dependency for testdata manager tool
The manager tool in syft/pkg/cataloger/binary/testdata/ imports
go-shlex, but since it's in a testdata directory, Go doesn't track
its dependencies. This caused CI failures when go.mod didn't
explicitly list the dependency.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* refactor: move binary classifier manager to internal/
Move the manager tool from testdata/manager to internal/manager so
that Go properly tracks its dependencies. Code in testdata directories
is ignored by Go for dependency tracking, which caused CI failures
when go.mod didn't explicitly list transitive dependencies.
This is a cleaner solution than manually adding dependencies to go.mod
for code that happens to live in testdata.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* fix: add gitignore negations for test fixtures blocked by root patterns
Multiple test fixtures were being blocked by root-level gitignore patterns
like bin/, *.jar, *.tar, and *.exe. This adds targeted .gitignore files with
negation patterns to allow these specific test fixtures to be tracked:
- syft/linux/testdata/os/busybox/bin/busybox (blocked by bin/)
- syft/pkg/cataloger/java/testdata/corrupt/example.{jar,tar} (blocked by *.jar, *.tar)
- syft/pkg/cataloger/binary/testdata/classifiers/snippets/go-version-hint/**/bin/go (blocked by bin/)
- syft/pkg/cataloger/bitnami/testdata/no-rel/.../bin/redis-server (blocked by bin/)
Also updates the bitnami test expectation to include the newly required
.gitignore files in the test fixture.
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* test: update glibc version expectation (2.43-r1 -> 2.43-r2)
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* add capability drift check as unit step
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* dont clear test observations before drift detection
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
* bump stereoscope commit to main
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
---------
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Pujitha PaladuguandPujitha Paladugu
Christopher Angelo Phillips
Yashas Gunderiaandychampion
Rez Moss
Mike ZolotarovandAlex Goodman
Ali Asghar
MarcusandChristopher Phillips
Arpit Jain
witchcraze
ChrisJr404andChris
Keith Zantow
Will Murphy