Using AI Chat and building assistants
Use the built-in chat, and connect your own data to a model sensibly.
The built-in AI Chat
/panel/ai/chat is a chat interface to the models configured on your account: a quick way to test prompts and compare models before wiring anything into an app.
Self-hosted alternatives
If you would rather run the UI and models yourself:
Open WebUI — a polished, self-hosted chat front-end (BSD).
LibreChat — multi-model with conversation management (MIT).
Ollama — run open-weight models locally on your own hardware (MIT).
Both UIs can point at the AI Router as an OpenAI-compatible backend, so you keep one gateway and one set of keys.
Giving an assistant your own data
The useful pattern is retrieval: store your documents, embed them, retrieve the relevant chunks, and put them in the prompt as context.
Chunk your content (a few hundred tokens per chunk, with overlap).
Embed each chunk and store the vectors (pgvector on PostgreSQL, or a vector store).
At query time, embed the question, find the nearest chunks, and add them to the prompt.
Instruct the model to answer only from the provided context, and to say "I do not know" otherwise.
This is how you get an assistant that answers about your product without fine-tuning anything.
Guardrails that matter
Ground it. Without retrieval, a general model will confidently invent details about your business.
Say when it does not know. A hallucinated price or policy is worse than "let me get a human".
Do not feed it secrets you would not paste into a third-party service.
Log and review. Keep prompts and responses for debugging quality and misuse.
Human in the loop for anything consequential — refunds, deletions, legal statements.
Cost control
Token usage scales with prompt + completion length. Retrieval keeps prompts focused; caching identical requests and setting max tokens avoid waste. Track spend per key — see AI Router quickstart.
Related reading