rink runs Inkling 2 Mini locally on Apple silicon.
The current version supports text and image input with Inkling 2 Mini on Metal. Support for CUDA and other models is planned.
- macOS on Apple silicon.
- A Rust toolchain with support for edition 2024.
- The Hugging Face CLI (
hf) to download the model files. - About 20 GB of disk space for the model files.
Use a Mac with 48 GB or more unified memory for the default chat settings. For a 32 GB Mac, use the smaller context below.
Download the original model files:
hf download thinkingmachines/Inkling-2-Mini \
--revision 6e7c8248f75265bdd4fbab5273304243b5d8c2cb \
--local-dir checkpoints/inkling-2-mini \
--include config.json tokenizer.json model.safetensors.index.json model-00001-of-00001.safetensorsStart a chat:
cargo run --release -- chatFor a 32 GB Mac, start with a smaller context and reply limit:
cargo run --release -- chat --context 4096 --max-tokens 1024Enter one message per line. Replies appear as the model generates them.
| Command | Action |
|---|---|
/help |
Show the commands. |
/image PATH |
Attach a PNG or JPEG to the next message. |
/image clear |
Remove the pending image. |
/continue |
Continue a reply that reached the token limit. |
/reset |
Clear the conversation. |
/quit |
Exit. |
Chat uses a 262,144-token context, a 16,384-token reply limit, and effort 0.9 by default.
The reply limit includes reasoning tokens. The context limit covers the full conversation.
The KV cache uses FP16. Conversation history stays in memory until reset or exit.
Prefill uses batches of up to 1,024 tokens. This targets about 30 GiB of Metal
memory at 256K context. Use --prefill-batch 128 to reduce memory use. See the
performance measurements.
To ask about a picture, enter /image /path/to/picture.png, then enter your question.
Each message can have one image. Images must use 8-bit channels and can be up to 16,384 pixels on each side.
rink keeps the original image size. Large images use more memory and take longer to process.
Chat shows progress and separate times for image encoder loading, image encoding,
prompt prefill, and generation. It also shows when the model is thinking.
Progress goes to stderr. Use --show-thinking to read the reasoning text.
Use cargo run --release -- chat --help to see all options.
The model configuration allows up to 1,048,576 tokens. This context does not fit in 48 GB of unified memory.
The vast majority of this code was written by GPT 6 Astra with human guidance.