AI Chat
Stream OpenAI responses through the Vercel AI SDK and AI Elements.
AI chat is a working Example built with the Vercel AI SDK, OpenAI, and AI Elements. It streams UI messages, renders markdown/reasoning/sources, accepts attachments, and lets the user choose an allowed model. The endpoint is public and needs product access controls before launch.
Request flow
ChatBot → useChat + DefaultChatTransport → POST /api/chat
→ Zod request parsing → allowed OpenAI model → streamText
→ UI message stream → AI Elements rendering| File | Role |
|---|---|
src/components/ai/ChatBot.tsx | Chat state, attachments, model/search controls, streaming UI, copy/regenerate/stop |
src/components/ai-elements/* | Conversation, message, prompt, reasoning, source, and attachment primitives |
src/routes/api/chat.ts | Request schema, model-message conversion, system prompt, stream response |
src/lib/ai/chat-config.ts | Model allowlist, default, search-preview mapping, system prompt |
src/lib/ai/index.ts | Resolves the runtime model and creates the OpenAI model handle |
{-$locale}/_marketing/ai/chat.tsx | Public example page and SEO |
Setup
Add the key locally, then store it as a Worker secret in production:
# .env (local only)
OPENAI_API_KEY=sk-your-key
pnpm wrangler secret put OPENAI_API_KEYRestart pnpm dev after changing local secrets. Never expose this as a
VITE_* value.
Models and search mode
The allowlist currently contains gpt-4o-mini (default) and gpt-4o.
resolveAiChatModelId() falls back to the default for an unknown client value,
so the browser cannot select an arbitrary model merely by changing the request.
To add a model, update aiChatModels with its display label and, if supported,
its search-preview ID. Review capability, attachment support, pricing, and
stream behavior before exposing it.
The “Web search” toggle does not register an AI SDK tool. It switches the
selected model to gpt-4o-mini-search-preview or
gpt-4o-search-preview, and sendSources exposes source parts from that
model. Do not describe this as a custom search tool or assume tool policy is
being enforced.
The route enables sendReasoning: true and conditionally enables
sendSources. The UI only renders those parts when the provider returns them;
their presence depends on the chosen model and response.
Customize behavior
- Edit
aiChatSystemPromptfor product scope and refusal guidance. - Change accepted attachments in
PromptInput, then enforce matching server size, media-type, and count limits. - Add persistent conversations in D1 if users need history; the current chat exists in component state only.
- Add server-side retrieval/tools deliberately through
streamTextand define authorization, timeout, and output policy for each tool. - Localize the current English-only chat labels if the product is bilingual.
Production hardening
POST /api/chat currently has no session check, rate limit, quota, message-size
limit, moderation layer, or per-user cost tracking. Before production:
- require a session or issue intentionally scoped anonymous access;
- enforce request/body/attachment limits and a server-side model allowlist;
- add per-user/IP rate limits, quotas, budget caps, and abuse monitoring;
- avoid returning raw provider errors or logging prompts/files containing sensitive data;
- define retention and consent before storing conversations;
- test disconnect/cancellation behavior and timeouts.
Cloudflare Workers can stream responses, but CPU time, subrequest limits, client disconnects, and provider latency still matter. Keep synchronous preprocessing small, stream promptly, and observe long requests in production.
Verify
- Missing/invalid
OPENAI_API_KEYproduces a visible server error without leaking the key. - Default and alternate allowed models stream text and can be stopped/regenerated.
- Unknown model IDs fall back to the default.
- Search mode returns sources only when the search-preview model supplies them.
- Attachments work only within the product's server-enforced policy.
- Logged-out, over-quota, and rate-limited requests are rejected after hardening.