From 89cca90898c1e5c959edebc3b2f198c95a23e1ab Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Fri, 10 Jul 2026 21:38:07 +0000 Subject: [PATCH 1/7] Add runpodctl model command reference Document the previously-undocumented runpodctl model command family (add, list, remove), including the new --wait-for-hash/-v flags on model add and the --hash/--version version-removal flags on model remove from runpodctl PR #302. Add model to the command groups table and register the new reference page in navigation. --- docs.json | 1 + runpodctl/overview.mdx | 1 + runpodctl/reference/runpodctl-model.mdx | 107 ++++++++++++++++++++++++ 3 files changed, 109 insertions(+) create mode 100644 runpodctl/reference/runpodctl-model.mdx diff --git a/docs.json b/docs.json index 74606b2a3..6f4058f57 100644 --- a/docs.json +++ b/docs.json @@ -423,6 +423,7 @@ "runpodctl/reference/runpodctl-pod", "runpodctl/reference/runpodctl-serverless", "runpodctl/reference/runpodctl-template", + "runpodctl/reference/runpodctl-model", "runpodctl/reference/runpodctl-hub", "runpodctl/reference/runpodctl-network-volume", "runpodctl/reference/runpodctl-registry", diff --git a/runpodctl/overview.mdx b/runpodctl/overview.mdx index 8fa4d6512..48fa20934 100644 --- a/runpodctl/overview.mdx +++ b/runpodctl/overview.mdx @@ -181,6 +181,7 @@ Runpod CLI organizes commands into groups based on the resource type: | `runpodctl pod` | | Manage Pods (create, list, start, stop, delete) | | `runpodctl serverless` | `sls` | Manage Serverless endpoints | | `runpodctl template` | `tpl` | List, search, and manage templates | +| `runpodctl model` | | Manage models in the model repository (add, list, remove) | | `runpodctl hub` | | Browse and deploy from the Runpod Hub | | `runpodctl network-volume` | `nv` | Manage network volumes | | `runpodctl registry` | `reg` | Manage container registry authentications | diff --git a/runpodctl/reference/runpodctl-model.mdx b/runpodctl/reference/runpodctl-model.mdx new file mode 100644 index 000000000..4f6a9cf58 --- /dev/null +++ b/runpodctl/reference/runpodctl-model.mdx @@ -0,0 +1,107 @@ +--- +title: "model" +sidebarTitle: "model" +--- + +Manage models in the Runpod model repository. Use these subcommands to upload model files, list your models, and remove models or individual model versions. + + +```bash Command +runpodctl model [flags] +``` + + +## Subcommands + +### Add a model + +Add a model to the model repository. To upload model files from your local machine, point `--model-path` at the directory that contains them: + +```bash +runpodctl model add --name "my-model" --owner "my-team" --model-path ./model +``` + +When you upload from a directory in a terminal, `runpodctl` shows a progress bar with upload progress and estimated time remaining, then prints the total model size once the upload finishes. + +#### Add flags + + +Model name. + + + +Model owner namespace (a user or team owner ID). + + + +Directory containing the model files to upload. + + + +Wait for uploaded model files to finish hashing before the command returns. Requires `--model-path`. When hashing completes, `runpodctl` prints a message confirming the model is ready to deploy along with its model URL. + + + +Include the full upload details in the `--wait-for-hash` output. Without this flag, a compact summary is printed instead. + + + + +`runpodctl model add` also accepts lower-level flags for configuring uploads directly, such as `--content-type`, `--file-name`, `--file-size`, `--part-size`, `--metadata`, and `--model-status`. Run `runpodctl model add --help` to see the complete list. + + + +### List models + +List the models in your model repository: + +```bash +runpodctl model list +``` + +#### List flags + + +Filter the results by model name. + + + +Filter the results by provider. + + +### Remove a model + +Remove a model from the model repository: + +```bash +runpodctl model remove --name "my-model" --owner "my-team" +``` + +To remove a single model version instead of the entire model, pass either `--hash` or `--version`. This marks that version as removed and leaves the rest of the model in place. + +```bash +runpodctl model remove --name "my-model" --owner "my-team" --version "" +``` + +#### Remove flags + + +Model name. + + + +Model owner. + + + +Hash of the model version to remove. Cannot be combined with `--version`. + + + +UUID of the model version to remove. Cannot be combined with `--hash`. + + +## Related commands + +- [`runpodctl serverless`](/runpodctl/reference/runpodctl-serverless) +- [`runpodctl config`](/runpodctl/reference/runpodctl-config) From 88e40ad2c9d5c787bc306ad6c1adea45920129c1 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Mon, 13 Jul 2026 22:22:13 +0000 Subject: [PATCH 2/7] Document --hash-timeout flag and runpodctl config env vars Add the --hash-timeout flag to the model add reference and document the RUNPOD_API_KEY, RUNPOD_GRAPHQL_URL, and RUNPOD_API_URL environment variables (which override config file values) introduced in runpodctl PR #302. --- runpodctl/reference/runpodctl-config.mdx | 16 ++++++++++++++++ runpodctl/reference/runpodctl-model.mdx | 4 ++++ 2 files changed, 20 insertions(+) diff --git a/runpodctl/reference/runpodctl-config.mdx b/runpodctl/reference/runpodctl-config.mdx index c4644628b..70b636b54 100644 --- a/runpodctl/reference/runpodctl-config.mdx +++ b/runpodctl/reference/runpodctl-config.mdx @@ -35,6 +35,22 @@ Your Runpod API key, which authenticates the CLI to access your account. You can The Runpod API endpoint URL. The default value should work for most users. +## Environment variables + +You can also configure the CLI using environment variables. When set, these take precedence over the corresponding values in your config file. + + +Your Runpod API key. Overrides the `apiKey` value in your config file. + + + +The Runpod GraphQL API endpoint URL. Overrides the `apiUrl` value in your config file. + + + +The Runpod REST API endpoint URL. Overrides the `restApiUrl` value in your config file. + + ## Related commands - [`runpodctl doctor`](/runpodctl/reference/runpodctl-doctor) diff --git a/runpodctl/reference/runpodctl-model.mdx b/runpodctl/reference/runpodctl-model.mdx index 4f6a9cf58..d39f09aff 100644 --- a/runpodctl/reference/runpodctl-model.mdx +++ b/runpodctl/reference/runpodctl-model.mdx @@ -41,6 +41,10 @@ Directory containing the model files to upload. Wait for uploaded model files to finish hashing before the command returns. Requires `--model-path`. When hashing completes, `runpodctl` prints a message confirming the model is ready to deploy along with its model URL. + +Maximum time to wait when `--wait-for-hash` is set. Set to `0` to wait indefinitely. + + Include the full upload details in the `--wait-for-hash` output. Without this flag, a compact summary is printed instead. From 47ed281b77e765635c1bf6ee4bdf140dfae7cfd3 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Thu, 30 Jul 2026 20:22:03 +0000 Subject: [PATCH 3/7] Document --huggingface-model mirroring on runpodctl model add Add coverage for the --huggingface-model flag (runpodctl PR #320, STO-330) that mirrors a Hugging Face model into the Runpod model repository server-side, including source-vs-destination semantics and mutual exclusivity with the local-upload flags. --- runpodctl/reference/runpodctl-model.mdx | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/runpodctl/reference/runpodctl-model.mdx b/runpodctl/reference/runpodctl-model.mdx index d39f09aff..0915679d3 100644 --- a/runpodctl/reference/runpodctl-model.mdx +++ b/runpodctl/reference/runpodctl-model.mdx @@ -15,7 +15,9 @@ runpodctl model [flags] ### Add a model -Add a model to the model repository. To upload model files from your local machine, point `--model-path` at the directory that contains them: +Add a model to the model repository. You can add a model in two ways: by uploading model files from your local machine, or by mirroring an existing Hugging Face model server-side. + +To upload model files from your local machine, point `--model-path` at the directory that contains them: ```bash runpodctl model add --name "my-model" --owner "my-team" --model-path ./model @@ -23,6 +25,16 @@ runpodctl model add --name "my-model" --owner "my-team" --model-path ./model When you upload from a directory in a terminal, `runpodctl` shows a progress bar with upload progress and estimated time remaining, then prints the total model size once the upload finishes. +To mirror an existing Hugging Face model into the Runpod model repository, use `--huggingface-model` with the source model in `owner/repo` form. `--name` sets the destination model name in the Runpod repository, and `--owner` is optional when mirroring. + +The transfer runs server-side, so there is no local upload session, file upload, or progress bar. Runpod creates the new model version right away and transfers the files from Hugging Face in the background. + +Mirroring cannot be combined with the local-upload flags (`--model-path`, `--create-upload`, `--file-name`, `--file-size`, `--part-size`, `--content-type`, and `--wait-for-hash`). Uploading and mirroring are two separate ways to add a model. + +```bash +runpodctl model add --name tiny-llm --huggingface-model arnir0/Tiny-LLM +``` + #### Add flags @@ -37,6 +49,10 @@ Model owner namespace (a user or team owner ID). Directory containing the model files to upload. + +Hugging Face model to mirror into the Runpod model repository, in `owner/repo` form. Cannot be combined with the local-upload flags (`--model-path`, `--create-upload`, `--file-name`, `--file-size`, `--part-size`, `--content-type`, and `--wait-for-hash`). + + Wait for uploaded model files to finish hashing before the command returns. Requires `--model-path`. When hashing completes, `runpodctl` prints a message confirming the model is ready to deploy along with its model URL. From 53d35d4d8039196e8441e3c94d679b7c3c6234a6 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Wed, 2 Sep 2026 19:18:07 +0000 Subject: [PATCH 4/7] Document model remove dependency block and model-file cleanup flag STO-360 (runpodctl PR #334): runpodctl model remove now refuses to remove a model or version still referenced by a Serverless endpoint, with no override flag. Document the refusal behavior, the detach or replace remediation via serverless update, and the current GPU-endpoint limitation on --model-reference/--clear-models. Add both flags to the serverless update reference. STO-446 (runpodctl PR #333): document the new --delete-my-model-files-after-upload flag on runpodctl model add. --- runpodctl/reference/runpodctl-model.mdx | 24 ++++++++++++++++++++ runpodctl/reference/runpodctl-serverless.mdx | 24 ++++++++++++++++++++ 2 files changed, 48 insertions(+) diff --git a/runpodctl/reference/runpodctl-model.mdx b/runpodctl/reference/runpodctl-model.mdx index 0915679d3..2b98760f9 100644 --- a/runpodctl/reference/runpodctl-model.mdx +++ b/runpodctl/reference/runpodctl-model.mdx @@ -25,6 +25,12 @@ runpodctl model add --name "my-model" --owner "my-team" --model-path ./model When you upload from a directory in a terminal, `runpodctl` shows a progress bar with upload progress and estimated time remaining, then prints the total model size once the upload finishes. +If you reuse one local directory across model versions without this flag, a later upload can include a mix of old and new files. Add `--delete-my-model-files-after-upload` to remove exactly the files this run uploaded from `--model-path` once the model version hash is confirmed server-side. It never touches unrelated files, and it runs only on confirmed success, so a failed or timed-out upload leaves your files in place. This flag requires both `--model-path` and `--wait-for-hash`. + +```bash +runpodctl model add --name "my-model" --owner "my-team" --model-path ./model --wait-for-hash --delete-my-model-files-after-upload +``` + To mirror an existing Hugging Face model into the Runpod model repository, use `--huggingface-model` with the source model in `owner/repo` form. `--name` sets the destination model name in the Runpod repository, and `--owner` is optional when mirroring. The transfer runs server-side, so there is no local upload session, file upload, or progress bar. Runpod creates the new model version right away and transfers the files from Hugging Face in the background. @@ -61,6 +67,14 @@ Wait for uploaded model files to finish hashing before the command returns. Requ Maximum time to wait when `--wait-for-hash` is set. Set to `0` to wait indefinitely. + +Delete the uploaded `--model-path` files after the model version hash is confirmed server-side. Only the files this run uploaded are deleted, including nested subdirectories, and only after the hash is confirmed. If hash confirmation times out or fails, no files are deleted. + +Requires both `--model-path` and `--wait-for-hash`. Both are validated before the upload begins, so the command fails immediately if either is missing. + +If a specific file cannot be deleted, the command exits non-zero and names that file, so there is no silent partial success. A stderr confirmation line prints once at least one file has been deleted. The `deletedModelFiles` (count) and `deletedModelFilesBytes` (bytes) fields are added to the JSON output only when `--verbose`, `-v` is set. + + Include the full upload details in the `--wait-for-hash` output. Without this flag, a compact summary is printed instead. @@ -103,6 +117,16 @@ To remove a single model version instead of the entire model, pass either `--has runpodctl model remove --name "my-model" --owner "my-team" --version "" ``` + + +`runpodctl model remove` refuses to remove a model or version that a Serverless endpoint still references. Removing the whole model is refused if any version is referenced; removing a specific version with `--hash` or `--version` is refused only when that exact version is referenced. The command reports which endpoints block the removal, and nothing is deleted when a removal is refused. + +There is no override or force flag. A model or version in active use cannot be removed until you detach or replace it on the referencing endpoint. If the dependency check cannot complete, for example when the API is unreachable, the removal is also refused, because Runpod treats an unverifiable dependency check as unsafe. + + + +To clear a blocked removal, detach or replace the model on the referencing endpoint, then retry the removal. Detach the model with `runpodctl serverless update --clear-models`, or replace it with `runpodctl serverless update --model-reference `. See the [Update an endpoint](/runpodctl/reference/runpodctl-serverless#update-an-endpoint) section for details. If the referencing endpoint is a GPU endpoint, this update currently fails until the pending fix ships; see the [limitation on `--model-reference` and `--clear-models`](/runpodctl/reference/runpodctl-serverless#update-flags). + #### Remove flags diff --git a/runpodctl/reference/runpodctl-serverless.mdx b/runpodctl/reference/runpodctl-serverless.mdx index 7daaeaff6..e3e28469e 100644 --- a/runpodctl/reference/runpodctl-serverless.mdx +++ b/runpodctl/reference/runpodctl-serverless.mdx @@ -165,6 +165,12 @@ Update endpoint configuration: ```bash runpodctl serverless update --workers-max 5 + +# Cache a model reference on the endpoint +runpodctl serverless update --model-reference https://huggingface.co//: + +# Clear all model references from the endpoint +runpodctl serverless update --clear-models ``` #### Update flags @@ -205,6 +211,24 @@ Enable or disable flash boot for faster worker startup. Execution timeout in seconds. Jobs that exceed this duration are terminated. +Attaching model references caches the referenced models on the endpoint so workers start faster and cost less. See [Model caching](/serverless/endpoints/model-caching) for when and why to cache models. + + +Model reference URL to cache on the endpoint, in the same form used by `serverless create` (a Hugging Face reference like `https://huggingface.co//:`). Repeatable: use multiple `--model-reference` flags to cache several models. Replaces the endpoint's existing model references. Cannot be combined with `--clear-models`. + + + +Remove all model references from the endpoint. Cannot be combined with `--model-reference`. + + + + +Updating an existing endpoint's model references with `--model-reference` or `--clear-models` currently works only on CPU (non-GPU) endpoints. On a GPU endpoint, this update cannot preserve the endpoint's GPU configuration, so it is rejected and the endpoint is left unchanged. A fix is pending. + +This limitation applies only to the update path shown here. GPU endpoints can still attach model references when you create them with `serverless create --model-reference`, or through the console. See [Model caching](/serverless/endpoints/model-caching) for the caching flow that still works on GPU endpoints. + + + ### Delete an endpoint Delete an endpoint: From ccfe15c26a62d7af474988a02643f34d21a8a02e Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Wed, 9 Sep 2026 19:02:36 +0000 Subject: [PATCH 5/7] Correct wait-for-hash and delete-after-upload to describe deployable confirmation runpodctl PR #337 (STO-441) changed --wait-for-hash to wait until a model version is genuinely deployable, not merely hashed, and --delete-my-model-files-after-upload to delete local files only after that deployable confirmation. --- runpodctl/reference/runpodctl-model.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/runpodctl/reference/runpodctl-model.mdx b/runpodctl/reference/runpodctl-model.mdx index 2b98760f9..2a657c18a 100644 --- a/runpodctl/reference/runpodctl-model.mdx +++ b/runpodctl/reference/runpodctl-model.mdx @@ -25,7 +25,7 @@ runpodctl model add --name "my-model" --owner "my-team" --model-path ./model When you upload from a directory in a terminal, `runpodctl` shows a progress bar with upload progress and estimated time remaining, then prints the total model size once the upload finishes. -If you reuse one local directory across model versions without this flag, a later upload can include a mix of old and new files. Add `--delete-my-model-files-after-upload` to remove exactly the files this run uploaded from `--model-path` once the model version hash is confirmed server-side. It never touches unrelated files, and it runs only on confirmed success, so a failed or timed-out upload leaves your files in place. This flag requires both `--model-path` and `--wait-for-hash`. +If you reuse one local directory across model versions without this flag, a later upload can include a mix of old and new files. Add `--delete-my-model-files-after-upload` to remove exactly the files this run uploaded from `--model-path` once the model version is confirmed deployable server-side. It never touches unrelated files, and it runs only after the version is confirmed deployable, so an upload that fails or whose deployable confirmation times out leaves your files in place. This flag requires both `--model-path` and `--wait-for-hash`. ```bash runpodctl model add --name "my-model" --owner "my-team" --model-path ./model --wait-for-hash --delete-my-model-files-after-upload @@ -60,7 +60,7 @@ Hugging Face model to mirror into the Runpod model repository, in `owner/repo` f -Wait for uploaded model files to finish hashing before the command returns. Requires `--model-path`. When hashing completes, `runpodctl` prints a message confirming the model is ready to deploy along with its model URL. +Wait until the uploaded model version is deployable before the command returns; despite the flag name, the wait is for deployability, not just for the file hash. Requires `--model-path`. Once the version is confirmed deployable, `runpodctl` prints a confirmation message, along with the model's URL. @@ -68,7 +68,7 @@ Maximum time to wait when `--wait-for-hash` is set. Set to `0` to wait indefinit -Delete the uploaded `--model-path` files after the model version hash is confirmed server-side. Only the files this run uploaded are deleted, including nested subdirectories, and only after the hash is confirmed. If hash confirmation times out or fails, no files are deleted. +Delete the uploaded `--model-path` files after the model version is confirmed deployable server-side. Only the files this run uploaded are deleted, including nested subdirectories, and only after the version is confirmed deployable. If that confirmation times out or fails, no files are deleted. Requires both `--model-path` and `--wait-for-hash`. Both are validated before the upload begins, so the command fails immediately if either is missing. From 861b2547b000b146cf161d6625daa4f1e8f6f5b0 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Thu, 10 Sep 2026 19:34:50 +0000 Subject: [PATCH 6/7] docs: reflect runpodctl serverless update model-ref/clear-models GPU fix (STO-691) PR runpod/runpodctl#340 fixed the bug where serverless update --model-reference/--clear-models failed on GPU endpoints. Replace the now-false CPU-only limitation warning with the shipped behavior (works on GPU and CPU endpoints, preserves GPU selection including excluded GPU types, output includes modelReferences), and drop the GPU-failure caveat from the model remove remediation prose. --- runpodctl/reference/runpodctl-model.mdx | 2 +- runpodctl/reference/runpodctl-serverless.mdx | 8 +------- 2 files changed, 2 insertions(+), 8 deletions(-) diff --git a/runpodctl/reference/runpodctl-model.mdx b/runpodctl/reference/runpodctl-model.mdx index 2a657c18a..58a25e415 100644 --- a/runpodctl/reference/runpodctl-model.mdx +++ b/runpodctl/reference/runpodctl-model.mdx @@ -125,7 +125,7 @@ There is no override or force flag. A model or version in active use cannot be r -To clear a blocked removal, detach or replace the model on the referencing endpoint, then retry the removal. Detach the model with `runpodctl serverless update --clear-models`, or replace it with `runpodctl serverless update --model-reference `. See the [Update an endpoint](/runpodctl/reference/runpodctl-serverless#update-an-endpoint) section for details. If the referencing endpoint is a GPU endpoint, this update currently fails until the pending fix ships; see the [limitation on `--model-reference` and `--clear-models`](/runpodctl/reference/runpodctl-serverless#update-flags). +To clear a blocked removal, detach or replace the model on the referencing endpoint, then retry the removal. Detach the model with `runpodctl serverless update --clear-models`, or replace it with `runpodctl serverless update --model-reference `. See the [Update an endpoint](/runpodctl/reference/runpodctl-serverless#update-an-endpoint) section for details. #### Remove flags diff --git a/runpodctl/reference/runpodctl-serverless.mdx b/runpodctl/reference/runpodctl-serverless.mdx index e3e28469e..7ef8bd4d7 100644 --- a/runpodctl/reference/runpodctl-serverless.mdx +++ b/runpodctl/reference/runpodctl-serverless.mdx @@ -221,13 +221,7 @@ Model reference URL to cache on the endpoint, in the same form used by `serverle Remove all model references from the endpoint. Cannot be combined with `--model-reference`. - - -Updating an existing endpoint's model references with `--model-reference` or `--clear-models` currently works only on CPU (non-GPU) endpoints. On a GPU endpoint, this update cannot preserve the endpoint's GPU configuration, so it is rejected and the endpoint is left unchanged. A fix is pending. - -This limitation applies only to the update path shown here. GPU endpoints can still attach model references when you create them with `serverless create --model-reference`, or through the console. See [Model caching](/serverless/endpoints/model-caching) for the caching flow that still works on GPU endpoints. - - +Updating an endpoint's model references with `--model-reference` or `--clear-models` works on both GPU and CPU endpoints. On a GPU endpoint, the update preserves the endpoint's GPU selection, including any excluded GPU types, rather than widening it. The command output includes the endpoint's current model references in the `modelReferences` field, which is an empty array after `--clear-models`. ### Delete an endpoint From 923de90959ca6ddf42227bdd281e0eb03e1b1387 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Thu, 17 Sep 2026 19:12:22 +0000 Subject: [PATCH 7/7] Document Model Repo storage-quota pre-check on runpodctl model add (STO-67) --- runpodctl/reference/runpodctl-model.mdx | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/runpodctl/reference/runpodctl-model.mdx b/runpodctl/reference/runpodctl-model.mdx index 58a25e415..4f41975c5 100644 --- a/runpodctl/reference/runpodctl-model.mdx +++ b/runpodctl/reference/runpodctl-model.mdx @@ -31,6 +31,14 @@ If you reuse one local directory across model versions without this flag, a late runpodctl model add --name "my-model" --owner "my-team" --model-path ./model --wait-for-hash --delete-my-model-files-after-upload ``` + + +If your account has a model repository storage quota, `runpodctl model add` checks the total upload size against your remaining quota before it creates the model or any upload session. An upload that would exceed the quota fails immediately, and the error reports how much space is available. To proceed, free up space or contact support to request a higher quota. + +This check applies only to local uploads. Mirroring a Hugging Face model with `--huggingface-model` has no local upload session, so the check does not apply. + + + To mirror an existing Hugging Face model into the Runpod model repository, use `--huggingface-model` with the source model in `owner/repo` form. `--name` sets the destination model name in the Runpod repository, and `--owner` is optional when mirroring. The transfer runs server-side, so there is no local upload session, file upload, or progress bar. Runpod creates the new model version right away and transfers the files from Hugging Face in the background.