Vercel AI SDK streamed responses tutorial

The Vercel AI SDK gives you a provider-agnostic way to stream text, structured data and tool calls from a server route to a React client. Stream the answer, but keep validation, authorisation and persistence on the server.

What matters in practice

• Tool calls and structured outputs need schemas and error handling.
• Abort the stream when the user navigates away and cap the output length.

How to implement it

1. Install the SDK and a provider. Pin versions and keep the provider key on the server.
2. Create a route handler. Validate the request, build the prompt and call streamText.
3. Wire the client. Use useChat with the route and render a stable loading state.
4. Add tools or structured output. Define schemas and validate the model's arguments.
5. Handle errors and aborts. Return a useful message and cancel the request on unmount.
6. Measure. Track time to first token, total latency, token usage and error rate.

A working example

// app/api/chat/route.ts
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";
export async function POST(req: Request) {
const { messages } = await req.json();
const result = streamText({
model: openai("gpt-4.1-mini"),
messages,
maxOutputTokens: 800,
abortSignal: req.signal,
});
return result.toTextStreamResponse();
}

Cost, latency and reliability

Concern	What drives it	What to do
Time to first token	Model latency and prompt size	Trim context, use a faster model and stream immediately
Cost	Input + output tokens	Cap output, cache context and route simple requests to a smaller model
Aborted requests	Client navigates away	Pass abortSignal and handle cancellation
Provider lock-in	Direct SDK calls everywhere	Use the provider registry and keep model IDs in config
Unsafe output	No validation	Schema-validate tool arguments and structured output

Security and privacy

• Keep provider keys on the server and scope them to the minimum permissions.
• Treat retrieved documents and user input as untrusted; separate instructions from data.
• Redact sensitive fields before sending them to a third-party model.
• Log prompts and outputs only as long as you need them, with access controls.
• Provide a human escalation path for high-impact or low-confidence answers.

Common mistakes

• Putting the provider key in a NEXT_PUBLIC_* variable.
• Streaming an unbounded answer with no output cap.
• Not handling the user closing the tab or navigating away.
• Trusting model-generated JSON without schema validation.

How to evaluate it

1. Build a fixed set. Real questions, ambiguous cases, no-answer cases and adversarial prompts.
2. Score the output. Correctness, citation quality, refusal behaviour, latency and cost.
3. Review failures. Group them by retrieval, prompt, model or product gap.
4. Version changes. Record the prompt, model and retrieval settings with each release.

Evaluation rubric

Dimension	Pass condition	How to test
Correctness	Matches the source or is clearly marked as uncertain	Human review against a reference answer
Grounding	Every factual claim has a source	Citation coverage and spot checks
Refusal	Declines when the context is insufficient	No-answer and adversarial cases
Safety	Does not follow injected instructions	Prompt-injection test set
Latency	Within the product's interaction budget	p95 on a production-like request
Cost	Within the per-task budget	Tokens, retrieval calls and retries

Rollout plan

1. Internal only. Test with the team and a fixed evaluation set; log every failure.
2. Small pilot. Expose it to a narrow user group with a visible feedback control.
3. Human in the loop. Require review for high-impact answers or actions.
4. Measure. Compare task completion, correction rate, latency and cost with the baseline.
5. Expand or stop. Expand only if the rubric passes; keep a kill switch and a fallback.

Data governance

Question	What to decide	Why
What data is sent?	Minimum fields, redaction and retention	Third-party processing and privacy
Where is it stored?	Logs, embeddings and evaluation sets	Access and deletion
Who can see it?	Roles, support and vendor access	Least privilege
How long is it kept?	Prompts, outputs and feedback	Retention policy
How is it deleted?	User, tenant and backup paths	Privacy requests
What is the fallback?	Search, form or human handoff	The model will fail sometimes

Sources and further reading

• Vercel AI SDK documentation (ai-sdk.dev)
• OpenAI API documentation (platform.openai.com)
• Next.js Route Handlers (nextjs.org)

Related reading

• AI chatbot with Next.js
• RAG with LangChain and Supabase
• Process automation