go-llm

go-llm is an application that puts language and speech models from many providers behind one interface, with a web UI, a command-line tool and a Go SDK. Add providers such as Ollama, LM Studio, OpenAI, Anthropic, Google Gemini, Mistral, go-whisper and ElevenLabs. Then ask any of their models to reply, call tools, search the web, set reminders, run agents, embed, read documents, speak, and transcribe or translate speech. Connect remote MCP servers, such as GitHub, Linear or Notion, so your assistant can use their tools too. You can do all of this through the REST API, the web UI, the llm command (which includes a terminal chat), or from Go.

Read the manual to get started.

Quick Start

Warning

Don't run go-llm on the internet. It has no authentication yet, so anyone who can reach it can use your API keys, read your chats, memory and files, and run its tools. Publish its port on localhost only, as below, or put it behind a proxy that signs people in, or a VPN.

docker volume create llm

docker run -d --name llm -p 127.0.0.1:8084:8084 -v llm:/data \
  -e OLLAMA_ADDR=http://host.docker.internal:11434 \
  -e OPENAI_API_KEY \
  ghcr.io/mutablelogic/llm

Then open http://localhost:8084/, and add a conversation:

For the command line:

go install -tags client github.com/mutablelogic/go-llm/cmd/llm@latest
llm models
llm ask --model gpt-4o-mini "Explain a KV cache in two sentences"

See Installation for Docker Compose, and running without Docker, and Getting Started for adding providers.

Documentation

The manual has:

• Features: what go-llm does.
• Installation: running it, and installing the command line.
• Getting Started: providers, a first conversation, and the commands.
• Conversations and Tasks: chat, memory, and scheduled tasks.
• Workspaces: files, and the planned Workspace.
• Models and Providers: providers, capabilities and voices.
• Tools and Agents: tools, connectors and agents.
• Messaging: Telegram, and voice notes.
• API: the REST API, and the Go API reference.
• Development: building, testing, the architecture, and the Go packages.
• Feedback & Issues: reporting bugs, and asking for features.

Its pages are in docs.

Roadmap

What's planned is in the issue list, by milestone:

• 1.6: Make it work.
• 1.7: Add authentication and a workspace, with sign-in, API keys and webhooks, and a Workspace of resources, files and connectors' resources.
• 1.8: Assistant capabilities and permissions, such as channels, scheduled agents, more speech and OCR, and approval before the assistant acts.

License

go-llm is licensed under the Apache License, Version 2.0.