UPX pads to a 4 byte boundary before it writes the loader stub. The
image-small-upx build happened to need no padding, so the cataloger
tests over it passed with or without the fix in #5347.
- build the fixture with -X main.Version=1.0.11, which needs 2 bytes of
padding, so the existing cataloger tests fail without the fix
- add TestImageSmallUPXNeedsLoaderPadding, which fails if a change to the
fixture's inputs means it no longer needs padding
- run the crafted loader test on both sides of the boundary, so a skip
that always rounds up is caught too
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Passing '%q' to format strings for integer types is a go vet error in
recent go versions, and likely a bug.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: centralize temp files and prefer streaming IO
Catalogers that create temp files ad-hoc can easily forget cleanup,
leaking files on disk. Similarly, io.ReadAll is convenient but risks
OOM on large or malicious inputs.
Introduce internal/tmpdir to manage all cataloger temp storage under
a single root directory with automatic cleanup. Prefer streaming
parsers (bufio.Scanner, json/yaml.NewDecoder, io.LimitReader) over
buffering entire inputs into memory. Add ruleguard rules to enforce
both practices going forward.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: go back to old release parsing
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* simplify to limit reader in version check
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: regex change postponed
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* simplify supplement release to limitreader
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
---------
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
BUILDPLATFORM is automatically set to the host's platform in new Docker,
so having it defined as an arg results in it being overridden by this
automatic value. Since it was always assigned to a literal string in the
test files, just use that string.
Additionally, image platform is better pulled from the manifest, not the
image config, in containerd store, so try that first.
Additionally, python3 is on PATH on new macs by default, but not python.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* fix(deb and snaps): prevent excess reads
Previously, Syft could allocate excess memory or tempfile space if there
were highly compressed objects in deb archives, or at paths where the
kernel changelog was expected by the snap cataloger. Use io.LimitReaders
for extracting parts of deb archives, and refactor the snap cataloger's
reading of the kernel changelog to use a streaming parsing, eliminating
the possibility of excess allocation.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* fix: always cleanup temp file from file source
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* use streaming strategy for deb archives
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
---------
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Otherwise sometimes the test that reads will run during the test that
writes and fail because the yaml file is in a partially written state.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* fix(cpe-generation): set start and end date
Previously, the update job was silently failing because the NVD API
returns a 404 with no body if a start date is specified but not an end
date. Further, the API returns an error if more than 120 days are in
range of the start and end date.
Update the API client to:
1. Return a non-nil error on http 404
2. Chunk the date range into 120 day chunks
3. Pass start and end date to avoid errors.
Also add more tolerant timestamp parsing since the previous update job
would fail with timestamp format errors.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* refactor(cpe-generator): remove callbacks
Previously, this job had callbacks that were there to make sure that
incremental progress could be written to disk. However, incremental
progress was not being written to disk, and there were issues related to
the callbacks like double logging. Therefore, just remove the callbacks
and do simple imperative code to page through the API results.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
---------
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
A CI failure was observed where a generated file was only partly written
when the CI job immediately tried to read it. Put in an fs.Sync call to
eliminate this flakiness.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Otherwise, we get test failures on macOS if macOS has decided to put
.DS_Store entries in the test fixtures.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Because package names in METADATA files may have upper case like
Werkzeug or Jinja2, but Syft artifacts have normalized names and are
lower case, like werkzeug or jinja2, Syft would miss emitting dependency
relationships. Therefore, normalize dependency names before comparing
with existing artifacts.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
We have had a few releases fail because the Apple credentials needed
some sort of fix. These release were operationally more interesting
because they failed after pushing a git tag (which effectively releases
the golagn package). Therefore, try to use these creds early, before
there's a tag pushed.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Red Hat purls the RPM modularity info in a query param in the PURLs in
their vulnerability data. It would be nice if Syft respected this
qualifier so that Grype can use it when a Red Hat purl is passed.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* add binary classifier for hashicorp vault
The Go Binary Cataloger isn't able to parse the version out of the
binary shipped in the DockerHub images of hashicorp/vault because the
version of the main module isn't set in the binary. Therefore, add a
binary classifier cataloger for this binary.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* chore: add test fixtures, update vault
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: set binary classifier package type based on PURL
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: use github.com/hashicorp/vault as package name
Signed-off-by: Keith Zantow <kzantow@gmail.com>
* chore: update tests
Signed-off-by: Keith Zantow <kzantow@gmail.com>
---------
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>
Co-authored-by: Keith Zantow <kzantow@gmail.com>
This prevents the cpe:a:redis:redis... from being emitted for the PHP
Pear / PECL package called redis.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
The bitnami cataloger assigns files under /opt/bitnami/PACKAGE to be
owned by PACKAGE unless they are otherwise owned. Previously, this main
package was identified only by relationships, leading to an edge case
where if there was a bitnami SBOM with a single package in it, there
were no relationships, and so there would be no main package to assign
the files to, leading to deduplication failures.
Instead, when encountering a bitnami SBOM with exactly one package in
it, assume that package is the main package of that SBOM.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Previously, if filling out this template, someone pasted a PR link after
"Fixes #", the issue wouldn't automatically close, probably because the
extra "#" confused the auto-close regex.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Previously, a bug in the formatter would cause SPDX expressions that
were surrounded in redundant parentheses to be dropped instead of
normalized.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Previously, if SYFT_LOG_FILE was not set, and no TTY was present,
log.Warn messages were discarded instead of being sent to stderr.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
* feat: emit dependency relationships found in Cargo.lock
Include updating test Cargo.lock to have dependencies on multiple
versions of the same crate.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
It's not clear to users that they shoudl use --output FORMAT=PATH
instead of --file. Directly suggest the FORMAT=PATH syntax.
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
Will Murphy
