Adding "ask a question about this site" is one of the more genuinely useful AI features, because the content is bounded, you control it, and every answer can be cited.

1. Index at build time, not at query time

Your content doesn't change per request. Chunk and embed it when you deploy, store the vectors, and queries just search. This is much cheaper and much faster than doing anything at request time.

site-search.txt
Build search over my site's content. Content: <format and rough size>.

Index step, run at deploy:
- Chunk on document structure — headings and sections, not character
  counts.
- Keep with each chunk: page URL, page title, heading path.
- Embed and store. Say where, given my size — a file, a database
  column, or a vector store.

Query step:
- Combine keyword and vector search, so exact terms still rank first.
- Return the top chunks to a model with the question.
- Answer only from the retrieved chunks. Cite the page for each claim.
  Say "I don't know" when the chunks don't cover it.

Show me the citations in the interface, linked.

2. Check retrieval before you check answers

Type twenty real questions and look at what comes back, before connecting a model. If the right page isn't in the top few results, no prompt will save the answer.

Most disappointing results in this kind of feature are retrieval problems being blamed on the model.

3. Citations make it trustworthy

Every claim links to the page it came from. This is what turns "an AI said something" into "here's the answer, and here's where it's written". It also lets users catch you being wrong, which is a feature.

4. Handle "not in the content"

People will ask things your site doesn't cover. The answer must be "I don't have anything about that", ideally with the nearest related pages — not a confident answer assembled from unrelated chunks.

5. Cost and abuse

It's a public endpoint that costs money per call. Rate limit it, cap the daily spend, and cache identical questions — on a site with a fixed corpus, the same questions recur constantly and caching is a large saving.

Log the questions people ask, especially the ones that return nothing. It's the best content roadmap you'll ever get.