Christopher Angelo PhillipsandAlex Goodman f15c3343e2 Pin Gguff lib to prevent dropping packages (#5447) ...

* test: catalog GGUF models larger than the header

gguf-parser-go v0.26.3+ rejects a header whose tensors declare more data than follows it. Syft parses only a 50 MB copy of each model, so any larger model is dropped.

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>

* fix: fix parser with rollback and verify with test

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>

* fix: bound GGUF tensor info before the parser reads it

The header pre-scan stopped after the metadata KVs and left tensor infos to gguf-parser-go. Older releases of the library allocate whatever length a tensor name declares, so a tiny file could force a multi-GB allocation, or a fatal out of memory that panic recovery can't catch.

- tensor names are bounded like metadata keys (non-empty, length limit, shared string budget)
- dimension count is capped at `GGML_MAX_DIMS`
- a tensor count larger than the header now fails at EOF in the pre-scan

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: give the GGUF header copy the model's real size

The cataloger copies at most 50 MB of a model and parses that copy. Newer gguf-parser-go releases check the tensor data the header declares against the file size, so the truncated copy of any larger model was rejected.

- after the header is validated, the copy is extended to the size the resolver reports for the model, leaving the tail as a hole the parser never reads
- sources that report no size (OCI model artifacts today) keep the old behavior

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore(deps): restore gguf-parser-go v0.26.4

The header copy now carries the model's real size, so the pin to v0.26.0 and its dependabot ignore are no longer needed. This also brings back upstream's string and array length bounds, and the library's own `IsNumeric`.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* fix: keep the extended GGUF header copy sparse on Windows

NTFS reserves disk for the full length of a file extended with truncate unless the file is marked sparse, so cataloging a large model on Windows reserved the model's size in the temp dir until cleanup.

- the header copy is marked sparse (`FSCTL_SET_SPARSE`) before it is extended
- if the filesystem can't make it sparse (FAT32, exFAT), the copy is not extended, and models larger than it fail to parse as before
- `golang.org/x/sys` moves from an indirect to a direct dependency

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Co-authored-by: Alex Goodman <wagoodman@users.noreply.github.com>

2026-10-19 14:40:43 -04:00

16 KiB

Raw History

View Raw