OpenAI-compatible
Standard chat-completions endpoint (/v1) with SSE streaming — tools written for the OpenAI API mostly just work.
API
The AI Chat module ships an OpenAI-compatible API — serve your local compute to third-party applications: terminal coding agents, editor plugins, automation scripts. Zero API fees, no queues, and your code and prompts never leave the machine.
01 — How It Works
Once a model is running, the chat module doubles as a local inference service — any tool that supports a custom endpoint can connect directly.
Standard chat-completions endpoint (/v1) with SSE streaming — tools written for the OpenAI API mostly just work.
An Anthropic-compatible endpoint (/anthropic/v1) is also provided, so tools built for the Claude API connect too.
The service runs on port 2918 of your computer; by default it binds to localhost only (127.0.0.1) — data stays on your machine.
02 — Setup
Open AI Chat, pick and launch a downloaded model — it becomes the inference engine behind the API.
The "API Configuration" page in the sidebar lists the Base URL and available model names — copy them.
Enter the Base URL and model name in your tool's custom-provider settings — done. Inference now runs on your own GPU.
03 — Example
opencode is an open-source terminal AI coding agent. Point it at the local endpoint and your coding assistant runs entirely on your own GPU — reading code, editing code, running commands, fully offline.
In opencode's custom-provider settings, you only need three pieces of information:
For the exact configuration format, follow the custom-provider section of the opencode documentation; the endpoint and model names shown in the app's "API Configuration" page are authoritative. Any tool that supports a custom OpenAI-compatible provider — editor plugins, chat clients, scripts — needs the same three pieces.
03+ — More Ways To Connect
Phones and tablets connect with a quick scan, and AI agents can use this computer's text recognition and speech transcription directly.
04 — Capabilities
By default the API binds to localhost only. To let other computers on the same network use it, turn on local network access in Settings; other devices must send an API key you create on the "API Configuration" page to make calls. The connection itself is unencrypted HTTP, so enable it only on trusted networks and never expose it to the public internet.
Besides the chat endpoints, there is a separate connection for AI agents: it lets tools you trust call text recognition and speech transcription. It is off by default; turn it on under "Settings → AI Agent Connections" and create a connection for each tool.
Response speed and usable context length are determined by your GPU and the selected model.
Use via the API remains subject to the EULA and the Acceptable Use Policy.
05 — Use Cases
Full write-ups from setup to finished result — real machine, real tool, with the shortcomings written down alongside the wins.
Read the case →Next