mirror of
https://github.com/anchore/syft.git
synced 2026-10-11 21:57:22 +02:00
Alex Goodman
50dbb33e66
Companion files next to AI models (`config.json`, `README.md`, and the OCI model.file, license and model config layers) are now read with named size limits, and are only used when they look like what they claim to be. - JSON companions are read whole up to a limit and skipped when larger, rather than being fully buffered or parsed truncated. OCI companion layers are fetched one byte past the cap so oversize is detectable. - README and license layers read a prefix. The frontmatter block is capped at 256 KB and list values at 32. - dir and OCI scans share one 8 MB safetensors header cap. Directory scans used to allow 100 MB. - `config.json` is only used when it has `architectures` or `_name_or_path`, and is looked up beside the model or in its parent directory, no longer all the way up to the scan root - OCI model.file layers are processed in a stable order, so the chosen name no longer changes between runs - README frontmatter: - a list-form `license:` is accepted and no longer drops `base_model` - `null` values are ignored - delimiters must be exact `---` lines - model names from `_name_or_path` / `base_model`: - Windows paths are handled - `.`, `..` and `/` fall back to the next name source - names are capped at 256 characters Behavior changes worth noting: - a `config.json` two or more directories above the model no longer names it - JSON `config.json` files without `architectures` or `_name_or_path` (e.g. `generation_config.json`, app configs) are ignored - safetensors headers over 8 MB in directory scans are now rejected, matching OCI scans (real shard headers are well under 1 MB) Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
8 lines
319 B
Go
8 lines
319 B
Go
package internal
|
|
|
|
// MaxSafeTensorsHeaderSize caps a safetensors header's JSON (excluding its
|
|
// 8-byte length prefix). Shared by the ai cataloger and the OCI model source so
|
|
// dir and OCI scans accept the same headers. Real-world shard headers are well
|
|
// under 1 MB.
|
|
const MaxSafeTensorsHeaderSize = 8 * 1024 * 1024
|