Chapter 3: Implementing the Typewriter Effect: A Deep Dive into Frontend Processing of AI Streaming APIs
If you want to develop a commercial-grade SaaS with AI capabilities, a typewriter effect (Streaming UI) that "spits out text one character at a time" is absolutely essential. If users stare at a blank screen for 10 seconds only to see the full response all at once, they'll just think your website is slow or broken, assuming the server has crashed.
This chapter will guide you step-by-step through parsing and rendering streaming data returned by OpenAI or Anthropic APIs on the frontend.
What is a ReadableStream?
In traditional API requests, we're used to using await fetch(...) to get a response, then directly calling await res.json() to convert the entire data package into a JavaScript object.
But in streaming mode, this approach won't work. The server won't give you the entire JSON package at once—it will feed you text in small chunks, like squeezing toothpaste.
In modern browser JavaScript, the interface for handling this type of data is called ReadableStream.
Hands-on: Building a Frontend Typewriter Hook
To keep our logic clean, we shouldn't cram these complex decoding operations directly into the UI. Let's create a custom Hook called useAiChat.
This Hook's task is: send the user's message to the backend API, then handle the continuous stream of text and concatenate it.
import { useState } from "react"
export function useAiChat() {
const [currentMessage, setCurrentMessage] = useState("")
const [isLoading, setIsLoading] = useState(false)
const sendMessage = async (userText: string) => {
setIsLoading(true)
setCurrentMessage("") // Clear current message to prepare for new response
try {
// Call our own Next.js Route Handler (backend)
const response = await fetch("/api/chat", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ message: userText })
})
if (!response.ok) throw new Error("API request failed")
if (!response.body) throw new Error("Invalid data stream")
// Get the reader from the ReadableStream
const reader = response.body.getReader()
// Decode incoming bytes into human-readable text (UTF-8)
const decoder = new TextDecoder("utf-8")
let done = false
// Core loop: keep reading new text chunks until done
while (!done) {
const { value, done: readerDone } = await reader.read()
done = readerDone
if (value) {
// stream: true tells the decoder these bytes might be incomplete—don't error yet, wait for more
const chunkString = decoder.decode(value, { stream: true })
// Append new text fragments to the current string
setCurrentMessage((prev) => prev + chunkString)
}
}
} catch (error) {
console.error("Streaming error:", error)
setCurrentMessage("Sorry, a connection error occurred.")
} finally {
setIsLoading(false)
}
}
return { currentMessage, sendMessage, isLoading }
}
This seemingly simple while loop actually solves a profound frontend problem. It enables our webpage to receive characters continuously like a water pipe.
Using the Hook in UI
With this powerful Hook handling the complex byte decoding logic, our UI component becomes incredibly clean and simple:
"use client"
import { useState } from "react"
import { useAiChat } from "./useAiChat"
export default function AiChatBox() {
const [input, setInput] = useState("")
const { currentMessage, sendMessage, isLoading } = useAiChat()
const handleSubmit = () => {
if (!input.trim()) return
sendMessage(input)
setInput("") // Clear input field
}
return (
<div className="max-w-md mx-auto p-6 bg-white rounded-xl shadow-lg border">
{/* AI response area */}
<div className="min-h-[150px] mb-4 p-4 bg-gray-50 rounded-lg text-gray-800 whitespace-pre-wrap leading-relaxed shadow-inner">
{currentMessage || (isLoading ? "Thinking..." : "Enter a question below to start chatting.")}
{/* Easter egg: blinking cursor for realistic typewriter effect */}
{isLoading && <span className="inline-block w-2 h-4 ml-1 bg-blue-500 animate-pulse" />}
</div>
{/* Input area */}
<div className="flex gap-2">
<input
value={input}
onChange={(e) => setInput(e.target.value)}
onKeyDown={(e) => e.key === 'Enter' && handleSubmit()}
className="flex-1 p-3 border rounded-xl focus:outline-blue-500 focus:ring-2 focus:ring-blue-200 transition-all"
placeholder="Ask me anything..."
/>
<button
onClick={handleSubmit}
disabled={isLoading}
className="px-6 py-3 bg-blue-600 text-white font-bold rounded-xl hover:bg-blue-700 disabled:opacity-50 transition-all"
>
Send
</button>
</div>
</div>
)
}
Why Use whitespace-pre-wrap?
In the <div> rendering the AI response, we added a Tailwind class called whitespace-pre-wrap. This is an easily overlooked UI detail!
AI responses often contain line breaks (\n). In HTML's default behavior, \n in plain text is treated as regular whitespace, causing all formatting to collapse into a single block.
Adding whitespace-pre-wrap forces the browser to "respect line breaks in the text string," perfectly preserving the paragraph formatting and code indentation generated by the AI.
The AI Ecosystem's Super Cheat Code: Vercel AI SDK
If you think manually implementing ReadableStream and while loops is still too cumbersome, there's a powerful industry solution.
Next.js's parent company Vercel has released an open-source library specifically for handling AI conversations: Vercel AI SDK (npm i ai).
It packages our useAiChat into an ultra-simplified useChat Hook, complete with built-in streaming, conversation history, regeneration, and stop generation features. In real commercial projects, we mostly rely on this powerful SDK to accelerate development.
But understanding the underlying ReadableStream principles gives you unparalleled debugging skills when encountering all sorts of bizarre bugs!
ReadableStream Deep Dive
The Web Streams API provides a standard way to handle streaming data in JavaScript.
Creating a ReadableStream
function createMockAIStream(): ReadableStream {
const words = ['Hello', 'I', 'am', 'an', 'AI', 'assistant', '.', 'How', 'can', 'I', 'help', 'you', '?'];
let index = 0;
return new ReadableStream({
async start(controller) {
for (const word of words) {
await new Promise(r => setTimeout(r, 100)); // Simulate AI delay
controller.enqueue(new TextEncoder().encode(word + ' '));
}
controller.close();
}
});
}
// Usage
const stream = createMockAIStream();
const reader = stream.getReader();
async function readStream() {
while (true) {
const { done, value } = await reader.read();
if (done) break;
console.log('Chunk:', new TextDecoder().decode(value));
}
console.log('Stream complete');
}
readStream();
Vercel AI SDK Streaming
Server-Side (API Route)
// app/api/chat/route.ts
import OpenAI from 'openai';
import { StreamingTextResponse, OpenAIStream } from 'ai';
const openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
});
export async function POST(req: Request) {
const { messages } = await req.json();
const response = await openai.chat.completions.create({
model: 'gpt-4-turbo',
stream: true,
messages,
});
// Convert OpenAI response to a ReadableStream
const stream = OpenAIStream(response);
// Return streaming response
return new StreamingTextResponse(stream);
}
Client-Side (React)
import { useChat } from 'ai/react';
export default function Chat() {
const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat({
api: '/api/chat',
});
return (
<div className="chat-container">
<div className="messages">
{messages.map((m) => (
<div key={m.id} className={`message ${m.role}`}>
<div className="role">{m.role === 'user' ? 'You' : 'AI'}:</div>
<div className="content">{m.content}</div>
</div>
))}
</div>
<form onSubmit={handleSubmit}>
<input
value={input}
onChange={handleInputChange}
placeholder="Type your message..."
disabled={isLoading}
/>
<button type="submit" disabled={isLoading}>
{isLoading ? 'AI is thinking...' : 'Send'}
</button>
</form>
</div>
);
}
Building Your Own useAiChat Hook
import { useState, useCallback, useRef } from 'react';
interface Message {
id: string;
role: 'user' | 'assistant';
content: string;
}
export function useAiChat(apiEndpoint: string) {
const [messages, setMessages] = useState<Message[]>([]);
const [isLoading, setIsLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
const abortRef = useRef<AbortController | null>(null);
const sendMessage = useCallback(async (text: string) => {
setIsLoading(true);
setError(null);
// Add user message
const userMessage: Message = {
id: Date.now().toString(),
role: 'user',
content: text,
};
setMessages(prev => [...prev, userMessage]);
// Create AI message placeholder
const aiMessage: Message = {
id: (Date.now() + 1).toString(),
role: 'assistant',
content: '',
};
setMessages(prev => [...prev, aiMessage]);
try {
abortRef.current = new AbortController();
const response = await fetch(apiEndpoint, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ messages: [...messages, userMessage] }),
signal: abortRef.current.signal,
});
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const reader = response.body?.getReader();
if (!reader) throw new Error('No response body');
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value);
setMessages(prev => {
const updated = [...prev];
const lastMsg = updated[updated.length - 1];
if (lastMsg.role === 'assistant') {
lastMsg.content += chunk;
}
return updated;
});
}
} catch (err: any) {
if (err.name !== 'AbortError') {
setError(err.message);
}
} finally {
setIsLoading(false);
}
}, [messages, apiEndpoint]);
const stop = useCallback(() => {
abortRef.current?.abort();
}, []);
const clearMessages = useCallback(() => {
setMessages([]);
setError(null);
}, []);
return { messages, isLoading, error, sendMessage, stop, clearMessages };
}
Summary
AI streaming delivers responses token by token using ReadableStream. The Vercel AI SDK simplifies this with useChat and StreamingTextResponse, but understanding the underlying stream API is valuable for debugging and customization.
Key takeaways:
- ReadableStream: standard API for streaming data in JavaScript |
- AI streaming: tokens arrive one by one, displayed in real-time |
- Vercel AI SDK:
useChathook +StreamingTextResponse| - Custom hook: fetch → getReader → decode chunks → update state |
- AbortController cancels streaming mid-response |
- Typewriter effect: append chunks to assistant message content |
- Server: OpenAIStream() converts AI response to ReadableStream |
- Client: Web Streams API reads chunks as they arrive |
What's Next: Supabase Realtime
The next chapter covers Supabase Realtime for database-driven live updates.