docker model package
| Description | Package a GGUF file into a Docker model OCI artifact, with optional licenses. |
|---|---|
| Usage | docker model package --gguf <path> [--license <path>...] [--context-size <tokens>] [--push] MODEL |
Description
Package a GGUF file into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified. When packaging a sharded model --gguf should point to the first shard. All shard files should be siblings and should include the index in the file name (e.g. model-00001-of-00015.gguf).
Options
| Option | Default | Description |
|---|---|---|
--context-size | context size in tokens | |
--gguf | absolute path to gguf file (required) | |
-l, --license | absolute path to a license file | |
--push | push to registry (if not set, the model is loaded into the Model Runner content store) |