Deploying an AI app or self-hosted model
From API-consuming app to running your own model — sizing, serving and cost.
There are two shapes: an app that calls a model, and a server that runs one. They have very different resource profiles.
Shape 1: an app that calls a model
This is the common case, and it is light. Your app makes HTTP calls to the AI Router (or another provider), so the only costs on your VPS are the app's own CPU/RAM and the model API spend.
Deploy it like any Node or PHP app.
Keep the API key server-side only.
Set per-key spend limits and watch usage.
Consider caching responses for repeated identical queries.
Shape 2: running your own model
Running open-weight models locally removes per-token costs but needs real hardware.
CPU-only: only small models are usable, and slowly. Fine for experiments, not production chat.
GPU: the practical route for anything responsive. VRAM is the constraint — model size must fit, with headroom for context.
RAM/VRAM rule of thumb: a model needs roughly its parameter count in memory at its quantisation. A 7B model at 4-bit quantisation needs on the order of 4–6 GB; larger models scale up sharply.
Tools:
Ollama — easiest way to run a model and expose an OpenAI-compatible API (MIT).
vLLM — high-throughput serving for serious workloads (Apache-2.0).
Serving safely
Never expose a model server to the public internet without auth and rate limits. Put it on the private mesh and front it with your app.
Rate-limit — a single unbounded prompt can consume all your capacity.
Queue long jobs rather than blocking a request thread.
Monitor GPU/VRAM and latency — see Monitoring your resources.
Choosing
| Need | Approach |
|---|---|
| A chatbot or feature in your app | Call the AI Router — cheapest and simplest |
| Data must never leave your hardware | Self-host a model (GPU VPS) |
| High volume, predictable | Self-host if GPU cost beats per-token spend |
| Experimenting | Ollama on a modest box, or the hosted API |
Related reading