Your GPU, usable by any tool.
The AI Chat module ships an OpenAI-compatible API — serve your local compute to third-party applications: terminal coding agents, editor plugins, automation scripts. Zero API fees, no queues, and your code and prompts never leave the machine.
Connected in three steps
Once a model is running, the chat module doubles as a local inference service — any tool that supports a custom endpoint can connect directly.
- 1
Start a model in the app
Open AI Chat, pick and launch a downloaded model — it becomes the inference engine behind the API.
- 2
Get the endpoint details
The "API Configuration" page in the sidebar lists the Base URL and available model names — copy them.
- 3
Paste into your tool
Enter the Base URL and model name in your tool's custom-provider settings — done. Inference now runs on your own GPU.
Example: coding in the terminal with opencode
opencode is an open-source terminal AI coding agent. Point it at the local endpoint and your coding assistant runs entirely on your own GPU — reading code, editing code, running commands, fully offline.
In opencode's custom-provider settings, you only need three pieces of information:
For the exact configuration format, follow the custom-provider section of the opencode documentation; the endpoint and model names shown in the app's "API Configuration" page are authoritative. Any tool that supports a custom OpenAI-compatible provider — editor plugins, chat clients, scripts — needs the same three pieces.
Tools get a full inference service
OpenAI-compatible
Standard chat-completions endpoint (/v1) with SSE streaming — tools written for the OpenAI API mostly just work.
Anthropic-compatible
An Anthropic-compatible endpoint (/anthropic/v1) is also provided, so tools built for the Claude API connect too.
Local endpoint
The service runs on port 2918 of your computer; by default it binds to localhost only (127.0.0.1) — data stays on your machine.
Streaming responses
SSE streaming works as usual — your coding agent's token-by-token output feels identical to a cloud API.
Context & parameters
Context length and generation parameters depend on the selected model — the same engine as in-app chat.
No quota anxiety
No token billing, no rate-limit tiers — the ceiling is your hardware.
Before you connect
By default the API binds to localhost only. To let other computers on the same network use it, enable LAN mode in Settings — the interface has no authentication, so enable it only on trusted networks and never expose it to the public internet.
Response speed and usable context length are determined by your GPU and the selected model.
Use via the API remains subject to the EULA and the Acceptable Use Policy.