A chat interface is the "hello world" of AI features, and there are three or four specific things that make the difference between a demo and something usable.

1. The key stays on your server

Non-negotiable. An API key in browser JavaScript is public, and a public key on a paid API is your credit card in a text file. The browser calls your endpoint; your endpoint calls the model.

chat-proxy.txt
Build a chat endpoint. Stack: <stack>.

- Holds the API key in an environment variable, server-side only.
- Takes the conversation history from the client, adds the system
  prompt server-side, calls the model.
- Streams the response back to the browser rather than waiting for
  the whole thing.
- Rate limits per IP or per session — this costs me money per call.
- Caps the history length sent, and says what it does when the
  conversation gets too long.
- Returns a clear error the UI can show if the model call fails.

2. Stream it

Waiting eight seconds for a paragraph feels broken. Watching it appear feels fast, even though it takes exactly as long. This is the single biggest perceived-quality difference in the whole build, and it's mostly a matter of not buffering.

3. The system prompt is the product

Everything distinctive about your bot lives here. Be specific: who it is, what it will and won't discuss, how long its answers should be, what tone, what to do when it doesn't know.

Vague system prompts produce generic assistants. "You are a helpful assistant" is the prompt that makes every chatbot feel the same.

4. Handle the boring cases

The model call failing. The user sending an empty message. A conversation growing past the context limit — decide whether you truncate the middle, summarise, or start fresh, and tell the user which. Long responses that need to scroll. Copying a reply.

And put a cost cap on it. An unmetered chat endpoint behind a public form is a way to fund a stranger's project with your card.

Show the conversation cost or message count somewhere while developing. Knowing what a session costs changes what you build.