DEV Community: CopilotKit

AIMock: One Mock Server For Your Entire AI Stack

Anmol Baranwal — Wed, 08 Apr 2026 17:14:01 +0000

TL;DR Our CI was flaky, our tests hit live APIs, and every run burned tokens unnecessarily. So the CopilotKit team built AIMock - a mock server that covers your entire agentic stack so your tests are fast, free, and actually reliable. Let's dive in!

Why we built AIMock

It started with LLM mocking. We built LLMock, open-sourced it, and thought we were done.

But a single agent request in 2026 can touch six or seven services before it returns a response: the LLM, an MCP tool server, a vector database, a reranker, a web search API, a moderation layer, and a sub-agent over A2A.

Most teams mock one of them. The other six are live, non-deterministic, and quietly making your test suite a lie. That is why LLMock became AIMock.

npm install @copilotkit/aimock

AI apps in 2026 call a lot more than just an LLM
Introducing AIMock: mock your entire agentic stack
LLMock: full streaming, tool calls, reasoning across every major provider
MCPMock: a mock MCP server for testing tool integrations
A2AMock: test multi-agent workflows without live agents
VectorMock: deterministic retrieval for RAG pipelines
Services: search, rerank, and moderation
Drift Detection: catch provider changes before your users do
Record and Replay: stop hand-writing fixtures
Chaos Testing: prove your app handles failure
What else is included
AG-UI uses AIMock in production
Migrating to AIMock

AIMock is open source and free.

AI apps in 2026 call a lot more than just an LLM

A realistic agent request looks like this:

User message
→ LLM decides to use a tool
→ Tool call via MCP (file system, database, calendar)
→ RAG retrieval from Pinecone or Qdrant
→ Web search via Tavily
→ Cohere reranker to sort results
→ Back to LLM with full context

Each of these is a live network call in your test environment. Everyone can fail, return something slightly different, or cost you tokens.

We looked at every tool out there. Some handled LLM mocking. Some handled one protocol. None covered the full picture. You would have to stitch together three or four libraries, each with its own config format, and still have gaps.

Here is how AIMock compares to other options out there.

Introducing AIMock: mock your entire agentic stack

It mocks everything your AI app talks to. Here is what is inside:

LLMock : 11 providers, full streaming, tool calls, reasoning models
MCPMock : local MCP server with full JSON-RPC 2.0, session management, tools, resources, prompts
A2AMock : agent card discovery, message routing, SSE streaming between agents
VectorMock : mock vector database for deterministic RAG retrieval
Services : search, rerank, and moderation. The APIs everyone forgets to mock.

On top of that, AIMock does three things no other mocking tool does:

Drift Detection : runs daily against real provider APIs and catches response format changes within 24 hours, before your users do
Record and Replay : proxies real API calls, saves them as fixtures, replays them forever in CI without touching live APIs again
Chaos Testing : inject 500s, malformed JSON, and mid-stream disconnects to prove your app handles failure

Run all of them on one port with a single config file:

{
  "llm": { "fixtures": "./fixtures/llm", "providers": ["openai", "claude", "gemini"] },
  "mcp": { "tools": "./fixtures/mcp/tools.json" },
  "a2a": { "agents": "./fixtures/a2a/agents.json" },
  "vector": { "path": "/vector", "collections": [] }
}

Get started with the AIMock CLI:

npx aimock --config aimock.json --port 4010

Let's go through each one in brief.

LLMock: full streaming, tool calls, reasoning across every major provider

LLMock runs a real HTTP server on a real port, not in-process patching. Any process on the machine can reach it: your Next.js app, agent workers, LangGraph processes, anything that speaks HTTP.

It has native support for 10 providers: OpenAI, Claude, Gemini, Bedrock, Azure, Vertex AI, Ollama, Cohere, OpenRouter, and Anthropic Azure. Reasoning models are supported across all of them.

Any OpenAI-compatible endpoint, such as Mistral, Groq, Together AI, or vLLM, works out of the box, too. Full streaming, tool calls, structured outputs, extended thinking, multi-turn conversations, and WebSocket APIs are all built in.

AIMock handles the translation internally, so one fixture format works across all providers.

Using the programmatic API with vitest, register a fixture and assert on the response. beforeAll starts the mock server once for the suite, afterAll tears it down, and mock.on() registers a fixture that maps a user message to a deterministic response.

import { LLMock } from "@copilotkit/aimock";
import { describe, it, expect, beforeAll, afterAll } from "vitest";

let mock: LLMock;

beforeAll(async () => {
  mock = new LLMock();
  await mock.start();
});

afterAll(async () => {
  await mock.stop();
});

it("non-streaming text response", async () => {
  mock.on({ userMessage: "hello" }, { content: "Hello! How can I help?" });

  const res = await fetch(`${mock.url}/v1/chat/completions`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      model: "gpt-4",
      messages: [{ role: "user", content: "hello" }],
      stream: false,
    }),
  });

  const body = await res.json();
  expect(body.choices[0].message.content).toBe("Hello! How can I help?");
  expect(body.object).toBe("chat.completion");
  expect(body.id).toMatch(/^chatcmpl-/);
});

Point your existing OpenAI client at the mock URL. Nothing else in your code changes. No API keys, no network calls, the same response every time. Full provider docs here.

MCPMock: a mock MCP server for testing tool integrations

MCP is how agents call tools. If your agent uses MCP, every tool call in your test suite hits a live server with real latency and no control over the response.

MCPMock gives you a local server that speaks the full MCP protocol over JSON-RPC 2.0. Your agent connects to it exactly like a real MCP server. You control what comes back.

import { MCPMock } from "@copilotkit/aimock/mcp";

const mcp = new MCPMock();

mcp.addTool({
  name: "search",
  description: "Search the web",
  inputSchema: { type: "object", properties: { query: { type: "string" } } },
});

mcp.onToolCall("search", (args) => {
  const { query } = args as { query: string };
  return `Found 3 results for "${query}"`;
});

const url = await mcp.start();
// point your MCP client at `url`

In a real agent test, your LLM and MCP tool server run together. Mount MCPMock onto LLMock so they share one port:

import { LLMock, MCPMock } from "@copilotkit/aimock";

const llm = new LLMock({ port: 5555 });
const mcp = new MCPMock();

mcp.addTool({ name: "calc", description: "Calculator" });
mcp.onToolCall("calc", (args) => "42");

llm.mount("/mcp", mcp);
await llm.start();
// MCP available at http://127.0.0.1:5555/mcp

It supports:

Tools - register handlers that return strings or rich content
Resources - static files and data your agent can read via URI
Prompts - reusable prompt templates with arguments
Session management - full Mcp-Session-Id lifecycle per the Streamable HTTP spec

Full docs here.

A2AMock: test multi-agent workflows without live agents

A2A (Agent2Agent) is how agents discover and talk to each other. It handles agent cards, message routing, and streaming responses between agents.

Testing a multi-agent system is hard when every agent is live. One agent goes down, your whole test suite breaks.

A2AMock gives you a local A2A server with full agent card discovery, message routing, task management, and SSE streaming. Register your agents, define how they respond, and test your multi-agent workflows end-to-end without anything actually running.

import { A2AMock } from "@copilotkit/aimock/a2a";

const a2a = new A2AMock();

a2a.registerAgent({
  name: "translator",
  description: "Translates text between languages",
  skills: [{ id: "translate", name: "Translate" }],
});

a2a.onMessage("translator", "translate", [{ text: "Translated text" }]);

const url = await a2a.start();
// Agent card at: ${url}/.well-known/agent-card.json
// JSON-RPC at: ${url}/

Here is the pattern for long-running tasks with incremental updates:

a2a.onStreamingTask("agent", "long-task", [
  { type: "status", state: "TASK_STATE_WORKING" },
  { type: "artifact", parts: [{ text: "partial result" }], name: "output" },
  { type: "artifact", parts: [{ text: "final result" }], lastChunk: true, name: "output" },
], 50); // 50ms delay between events

For task management, agent cards, and JSON-RPC methods, see the full docs here.

VectorMock: deterministic retrieval for RAG pipelines

If your app does retrieval-augmented generation, your tests depend on what is in your vector database right now. Your dev index is messy, your staging index does not match prod, and results shift every time someone upserts new vectors.

VectorMock is a mock vector database server that supports Pinecone, Qdrant, and ChromaDB API formats with collection management, upsert, query, and delete operations.

import { VectorMock } from "@copilotkit/aimock/vector";

const vector = new VectorMock();

vector.addCollection("docs", { dimension: 1536 });
vector.onQuery("docs", [
  { id: "doc-1", score: 0.95, metadata: { title: "Getting Started" } },
  { id: "doc-2", score: 0.87, metadata: { title: "API Reference" } },
]);

const url = await vector.start();
// point your vector DB client at `url`

If you need results to vary based on the query, use a dynamic handler instead:

vector.onQuery("docs", (query) => {
  const topK = query.topK ?? 10;
  return Array.from({ length: topK }, (_, i) => ({
    id: `result-${i}`,
    score: 1 - i * 0.1,
  }));
});

Here are compatible endpoints for Pinecone, Qdrant, and ChromaDB APIs.

Consistent retrieval results. Every test run. Full docs here.

Services: search, rerank, and moderation

These are the APIs most people forget to mock, and the ones that quietly make your test suite non-deterministic.

Services within AIMock provides built-in mocks for web search, reranking, and content moderation. Register fixture patterns on your LLMock instance and requests are matched by query/input text. No separate server needed.

Tavily search - mock web search results by query pattern at POST /search
Cohere reranking - mock reranked document lists at POST /v2/rerank
OpenAI moderation - mock moderation decisions at POST /v1/moderations. Unmatched requests return unflagged by default.

import { LLMock } from "@copilotkit/aimock";

const mock = new LLMock();

// String pattern — case-insensitive substring match
mock.onSearch("weather", [
  { title: "Weather Report", url: "https://example.com/weather", content: "Sunny today" },
]);

// RegExp pattern
mock.onSearch(/stock\s+price/i, [
  { title: "ACME Stock", url: "https://example.com/stocks", content: "$42.00", score: 0.95 },
]);

// Catch-all — empty results for unmatched queries
mock.onSearch(/.*/, []);

mock.onRerank("machine learning", [
  { index: 0, relevance_score: 0.99 },
  { index: 2, relevance_score: 0.85 },
]);

mock.onModerate("violent", {
  flagged: true,
  categories: { violence: true, hate: false },
  category_scores: { violence: 0.95, hate: 0.01 },
});

If you just need catch-all responses to stop live requests, enable all three in config:

{
  "services": {
    "search": true,
    "rerank": true,
    "moderate": true
  }
}

String patterns use case-insensitive substring matching. RegExp patterns do full regex testing. First match wins. All service requests are recorded in the journal, so you can inspect exactly what got called. Full docs here.

Drift Detection: catch provider changes before your users do

This is one of the things no other mocking tool does. A mock that does not match reality is worse than no mock.

Here is the problem: mocks are a snapshot of how an API behaved when you wrote the fixture. OpenAI adds a field. Claude changes a default. Gemini tweaks its streaming format. Your mocks still pass. CI is green. Then your app breaks in production.

Drift Detection runs a three-way comparison in CI every day:

SDK types - what TypeScript type definitions say the shape should be
Real API responses - actual live requests to OpenAI, Anthropic, Gemini
AIMock output - what the mock returns for the same request

If any of those three disagree, you know within 24 hours. Not when a user files a bug.

Here is what the output looks like when drift is detected:

$ pnpm test:drift

[critical] AIMOCK DRIFT — field in SDK + real API but missing from mock
Path:    choices[].message.refusal
SDK:     null    Real: null    Mock: <absent>

[critical] TYPE MISMATCH — real API and mock disagree on type
Path:    content[].input
SDK:     object    Real: object    Mock: string

[warning] PROVIDER ADDED FIELD — in real API but not in SDK or mock
Path:    choices[].message.annotations
SDK:     <absent>    Real: array    Mock: <absent>

✓ 2 critical (test fails) · 1 warning (logged) · detected before any user reported it

Severity levels:

Critical : mock mismatches the real API, test fails immediately
Warning : provider added a new field that neither the SDK nor the mock knows about yet, logged as early warning
Ok : all three sources agree, nothing to do

Here is how the three-way comparison works under the hood:

import { extractShape, triangulate, formatDriftReport, shouldFail } from "./schema";

// 1. Get the SDK shape (what TypeScript says)
const sdkShape = openaiChatCompletionShape();

// 2. Call the real API and the mock in parallel
const [realRes, mockRes] = await Promise.all([
  openaiChatNonStreaming(config, [{ role: "user", content: "Say hello" }]),
  httpPost(`${instance.url}/v1/chat/completions`, { /* ... */ }),
]);

// 3. Extract response shapes
const realShape = extractShape(realRes.body);
const mockShape = extractShape(JSON.parse(mockRes.body));

// 4. Three-way comparison
const diffs = triangulate(sdkShape, realShape, mockShape);
const report = formatDriftReport("OpenAI Chat (non-streaming text)", diffs);

// 5. Critical diffs fail the test
if (shouldFail(diffs)) {
  expect.soft([], report).toEqual(
    diffs.filter(d => d.severity === "critical")
  );
}

Run it yourself against live endpoints:

# Run drift checks against live endpoints
pnpm vitest --config vitest.config.drift.ts

MSW, VidaiMock, Mokksy - none of them do this. Your mocks should never silently go stale. Full docs here.

Record and Replay: stop hand-writing fixtures

Writing fixtures by hand is fine for simple cases. It gets painful fast when you are dealing with multi-turn agent conversations with tool calls, streaming responses, and branching logic. MSW, VidaiMock, mock-llm, piyook - none of the LLM mocking tools solve this.

Record and Replay works like a VCR. Here is exactly what happens:

Client sends a request to AIMock
AIMock attempts fixture matching as usual
On miss: the request is forwarded to the configured upstream provider
The upstream response is relayed back to the client immediately
The response is collapsed if streaming and saved as a fixture to disk and memory
Subsequent identical requests match the newly recorded fixture

Quick start using CLI:

$ npx aimock --fixtures ./fixtures \
  --record \
  --provider-openai https://api.openai.com \
  --provider-anthropic https://api.anthropic.com

Recorded fixtures are saved automatically and look like this:

{
  "fixtures": [
    {
      "match": { "userMessage": "What is the weather?" },
      "response": { "content": "I don't have real-time weather data..." }
    }
  ]
}

AIMock handles stream collapsing automatically across six formats: OpenAI SSE, Anthropic SSE, Gemini SSE, Cohere SSE, Ollama NDJSON, and Bedrock EventStream. Auth headers are forwarded to the upstream provider but never saved in fixtures.

It also supports a programmatic API with enableRecording() and requestTransform for normalizing dynamic data like timestamps.

In CI, add --strict so that any unmatched request returns a 503 and fails immediately, rather than slipping through silently. Without it, missing fixtures return 404 responses, and your test suite may never tell you that a live API call was attempted.

Full docs here.

Chaos Testing: prove your app handles failure

Your app will eventually get a 500 from OpenAI. A malformed JSON response from a tool. A mid-stream disconnect from a vector database. The question is whether you find out in testing or in production.

Chaos Testing lets you inject failures at configurable probabilities at three levels: server-wide, per fixture, or per individual request.

Three failure modes:

drop - returns HTTP 500 with {"error":{"message":"Chaos: request dropped","code":"chaos_drop"}}
malformed - returns HTTP 200 with invalid JSON
disconnect - destroys the TCP connection immediately with no response

Server-level chaos applies to all requests:

import { LLMock } from "@copilotkit/aimock";

const mock = new LLMock();
mock.setChaos({
  dropRate: 0.1,        // 10% of requests return 500
  malformedRate: 0.05,  // 5% return broken JSON
  disconnectRate: 0.02, // 2% drop the connection
});

// remove all chaos later
mock.clearChaos();

Fixture-level chaos targets specific responses only:

{
  "fixtures": [
    {
      "match": { "userMessage": "unstable" },
      "response": { "content": "This might fail!" },
      "chaos": {
        "dropRate": 0.3,
        "malformedRate": 0.2,
        "disconnectRate": 0.1
      }
    }
  ]
}

Per-request headers override everything else - useful for forcing a specific failure in a single test:

// Force 100% disconnect on this specific request
await fetch(`${mock.url}/v1/chat/completions`, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-aimock-chaos-disconnect": "1.0",
  },
  body: JSON.stringify({ model: "gpt-4", messages: [{ role: "user", content: "hello" }] }),
});

All chaos events are tracked in the journal with a chaosAction field and counted in Prometheus metrics. Full docs here.

What else is included

Beyond the core modules, AIMock offers a lot more worth knowing.

Embeddings - POST /v1/embeddings is fully supported. Return explicit vectors via fixtures or let AIMock generate them deterministically from a hash of the input text. Same input always produces the same vector, default dimensions are 1536.

WebSocket APIs - OpenAI Realtime, OpenAI Responses over WebSocket, and Gemini Live are all supported using raw RFC 6455 framing. If your app uses voice or real-time streaming agents, these are covered.

Sequential Responses - return different responses for the same prompt on each successive call. Useful for testing retry logic and multi-turn workflows where the same message should behave differently over time.

[
  { "match": { "userMessage": "retry", "sequenceIndex": 0 }, "response": { "content": "First attempt" } },
  { "match": { "userMessage": "retry", "sequenceIndex": 1 }, "response": { "content": "Second attempt" } },
  { "match": { "userMessage": "retry" }, "response": { "content": "Fallback" } }
]

Streaming Physics - configure ttft (time to first token), tps (tokens per second), and jitter to simulate realistic timing. Pre-built profiles cover fast models, reasoning models, and overloaded systems.

Prometheus Metrics - request counts, latency histograms, and current fixture count at /metrics. Enable with --metrics flag.

Docker and Helm - the official Docker image is available at ghcr.io/copilotkit/aimock (GitHub Container Registry) for CI/CD. It runs as a plain HTTP server, so any language can use it. No Node.js needed on the test runner side.

AG-UI uses AIMock in production

AG-UI is the open protocol that connects AI agents to frontend applications - adopted by LangGraph, CrewAI, Mastra, Google ADK, AWS Bedrock AgentCore, and more.

It uses AIMock for its end-to-end test suite, verifying AI agent behavior across LLM providers with fixture-driven responses. AG-UI is a protocol used in production. If you want to see what a real AIMock setup looks like at scale, that is a good place to start.

Migrating to AIMock

The docs include step-by-step migration guides for MSW, VidaiMock, mock-llm, Python mocks and Mokksy. You will find a side-by-side comparison and a breakdown of what you gain and keep.

If you are on MSW, you do not have to replace everything: you can keep MSW for general REST and GraphQL mocking and use AIMock only for AI endpoints.

If you are already using @copilotkit/llmock, the upgrade is a find-and-replace:

pnpm remove @copilotkit/llmock
pnpm add @copilotkit/aimock

The LLMock class, all fixture formats, and the programmatic API are unchanged. Your existing tests will work as-is.

Get started

npm install @copilotkit/aimock

Docs: aimock.copilotkit.dev
GitHub: github.com/CopilotKit/aimock
npm: @copilotkit/aimock
CLI: AIMock CLI

Your test suite should be as complete as your stack. That is what AIMock is for.

You can connect me on GitHub, Twitter and LinkedIn.

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

Google Maps for Codebases: Paste a GitHub URL, Ask Anything

Anmol Baranwal — Mon, 06 Apr 2026 13:09:38 +0000

Navigating a large codebase for the first time is painful. You clone the repo, realize there are 300 files, and have no idea where anything lives.

You can ask an AI assistant, but it burns through context fast and you never know if it hallucinated a file path that does not even exist.

Codebase Navigator solves this. Paste a URL, ask anything in plain English, and watch a real dependency graph built from actual import statements appear in real time.

It is built using CopilotKit, Zenflow, GitHub API and React Flow. You can run it completely free using Ollama locally.

In this blog, we will go through the architecture, the key patterns and how everything works end-to-end.

What are we building?

Codebase Navigator lets you paste any public GitHub repo URL and ask questions about it in plain English. Instead of getting a wall of text back, you get four panels that all update at once.

Panel	What it does
Graph Canvas	Live dependency graph built from real import statements
Code Viewer	File content fetched live from GitHub, relevant lines highlighted
Repo Explorer	Full file tree, click any file to open it
Chat	Follow-up questions and plain English explanations

The graph morphs to show every relevant file connected by real imports, the code viewer opens the file, and the chat explains what each piece does. No context switching.

You can run it completely free using Ollama locally without any API key.

Here is the full request → response flow of what happens when you ask a question:

User types "how does auth work?"
     ↓
CopilotChat → POST /api/copilotkit
     ↓
LLM receives repo context (file paths, current selection, system rules)
     ↓
LLM calls analyzeRepository tool
     ↓
Tool fetches relevant files via /api/github/file
     ↓
Extracts import/require statements → resolves paths → builds dependency graph
     ↓
Zustand store updates
     ↓
All four panels re-render live: graph, file tree, code viewer, chat response

Tech Stack & Tools

At a high level, this project is built by combining:

Next.js 16 - frontend and API routes
CopilotKit - agent-UI state sync and chat interface. provides built-in hooks & components like useAgentContext, useFrontendTool, CopilotChat
Zenflow - workflow tool that planned, tested and orchestrated the build
React Flow & dagre - interactive dependency graphs with automatic layout
Octokit - GitHub API proxy for fetching repo trees and file contents
Zustand - shared state across all four panels
Tailwind CSS - styling
Ollama / OpenAI - local or cloud LLM backend switchable from the UI

The CopilotKit runtime is self-hosted inside a Next.js API route, which lets you plug into any OpenAI-compatible backend, including Ollama, for completely free local inference.

It connects your UI, agents, and tools into a single interaction loop.

What is Zenflow?

Zenflow is an AI development tool that treats building software as a structured engineering process rather than just autocompleting code.

This project was planned, built, tested, reviewed and deployed using Zenflow (Zencoder's workflow engine) in a single session.

It ran a six-phase process from scratch:

Architecture and spec
Scaffolding and foundation
Data layer (GitHub API integration)
AI layer (CopilotKit actions and context)
Visual layer (React Flow graphs)
Final assembly and wiring

Each phase was verified with lint, type checks and tests before moving to the next. After the build, it performed a code review, found 14 issues and fixed 11 systematically:

API keys exposed in headers → moved to httpOnly cookies
Fake star-shaped graphs → replaced with real import-based dependency resolution
Duplicate file-fetching logic → consolidated into a single cached utility

The .zenflow/ folder in the repo contains the task plan, spec, and report files it generated along the way.

That's how we built it with Zenflow. Here's the architecture that came out of it.

Architecture & Project Structure

The app is split into three layers: the browser handles all UI and AI tool logic, Next.js API routes act as a secure proxy to external services, and GitHub API, plus the LLM backend sits at the bottom. Nothing in the browser talks to GitHub or the LLM directly.

Below is a high-level overview of all the layers and how they are organized.

┌─────────────────────────────────────────────────────────┐
│                        BROWSER                          │
│                                                         │
│  ┌─────────┐  ┌──────────┐  ┌──────────┐  ┌────────┐    │
│  │  Chat   │  │  Graph   │  │  Code    │  │  File  │    │
│  │  Panel  │  │  Canvas  │  │  Viewer  │  │  Tree  │    │
│  └────┬────┘  └────┬─────┘  └────┬─────┘  └───┬────┘    │
│       └────────────┴─────────────┴────────────┘         │
│                          │                              │
│                   ┌──────▼──────┐                       │
│                   │   Zustand   │  ← shared state       │
│                   └──────┬──────┘                       │
│                          │                              │
│              ┌───────────▼────────────┐                 │
│              │      CopilotKit        │                 │
│              │   frontend tools       │                 │
│              │  analyzeRepository     │                 │
│              │  fetchFileContent      │                 │
│              │  highlightCode         │                 │
│              └───────────┬────────────┘                 │
└──────────────────────────┼──────────────────────────────┘
                           │
┌──────────────────────────▼──────────────────────────────┐
│                    NEXT.JS API ROUTES                   │
│                                                         │
│   /api/copilotkit     /api/github/*     /api/settings   │
│   (LLM runtime)       (proxy)          (httpOnly cookie)│
└──────────┬────────────────┬─────────────────────────────┘
           │                │
     ┌─────▼─────┐   ┌──────▼──────┐
     │  Ollama   │   │  GitHub API │
     │  / OpenAI │   │  (Octokit)  │
     └───────────┘   └─────────────┘

Here is a simplified view of how the project is structured.

src/
├── app/api/
│   ├── copilotkit/route.ts    → CopilotKit runtime endpoint
│   ├── github/
│   │   ├── tree/route.ts      → fetch repo tree
│   │   ├── file/route.ts      → fetch + base64-decode file
│   │   └── search/route.ts    → code search
│   └── settings/route.ts      → LLM config (httpOnly cookie)
├── components/
│   ├── panels/     → 5 UI panels (chat, graph, code, analysis, repo)
│   └── flow/       → custom React Flow node types
├── hooks/          → AI tool registration, agent context sync, repo loading
├── lib/            → import extraction, dagre layout, Octokit client
├── store/          → Zustand (AppState + SettingsState)
└── .zenflow/       → Zenflow task plan, spec and report

Let's get into how everything works behind the scenes. This will help you understand the key patterns.

How everything works under the hood

Now that you have the big picture, let's go through each piece in detail.

A lot is happening under the surface, so we will break it down into ten parts, starting from when you first load a repo all the way to how state flows across the panels.

1. Loading a repository

When you paste a GitHub URL and click Explore, useRepository.loadRepository() fires (from useRepository.ts hook).

It calls /api/github/tree on the server, which uses Octokit to:

Parse the repo URL into { owner, repo }
Fetch the default branch
Get the full recursive file tree from the GitHub Trees API
Convert the flat array into a nested TreeNode structure

Here is what the src/api/github/tree/route.ts route looks like:

export async function GET(request: NextRequest) {
  const { owner, repo } = parseRepoUrl(repoUrl);
  const resolvedBranch = branch || (await getDefaultBranch(owner, repo));
  const tree = await getRepoTree(owner, repo, resolvedBranch);

  return NextResponse.json({ owner, repo, branch: resolvedBranch, tree });
}

That nested tree powers the sidebar file explorer and becomes the starting point for everything else.

All GitHub calls are proxied through Next.js API routes. The browser never talks to GitHub directly to keep tokens secure.

2. From architecture view to dependency graph

This is the visual moment that makes the app feel alive. The graph you see on load (the architecture view) and the graph you see after asking a question (the dependency graph) are two completely different things, built two different ways.

Both live in src/lib/analyzer.ts - buildOverviewGraph for the architecture view and buildDependencyNodes for the dependency graph.

When the repo loads, useRepository.ts calls buildOverviewGraph right after the tree response comes back and pushes the result straight to the graph:

const overview = buildOverviewGraph(data.tree);
setVisualization(overview.nodes, overview.edges, "architecture");

buildOverviewGraph never fetches any files. It just walks the tree structure and builds a folder map: root node at the top, top-level directories as children, one level of sub-folders below that:

export function buildOverviewGraph(tree: TreeNode) {
  const rootId = `node-${nodeId++}`;
  nodes.push({ id: rootId, type: "module", label: tree.path || "root" });

  const topDirs = (tree.children || []).filter((c) => c.type === "directory");

  for (const dir of topDirs) {
    const fileCount = flattenTree(dir).length;
    nodes.push({ id, label: `${dir.name} (${fileCount})`, type: categorizeDirType(dir.path) });
    edges.push({ source: rootId, target: id, type: "flow" });
  }
}

The edges here mean nothing beyond "this folder is inside that folder."

Once you ask a question, useCopilotActions.ts fetches the actual file content and calls buildDependencyNodes instead:

const graph = buildDependencyNodes(fileData);
setVisualization(graph.nodes, graph.edges, "dependency");

buildDependencyNodes creates one node per file and one edge per real import statement:

export function buildDependencyNodes(files: { path: string; imports: string[] }[]) {
  const nodes = files.map((file) => ({
    id: file.path,
    type: "file",
    label: file.path.split("/").pop() || file.path,
    metadata: { fullPath: file.path },
  }));

  for (const file of files) {
    for (const imp of file.imports) {
      const resolved = resolveImportPath(imp, file.path, filePathSet);
      if (resolved) {
        edges.push({ source: file.path, target: resolved, type: "import" });
      }
    }
  }

  return { nodes, edges };
}

Same setVisualization call, different data. React Flow re-renders and the graph morphs from a folder map into a real dependency graph focused on exactly the files relevant to your question.

3. The CopilotKit runtime

The /api/copilotkit route is where LLM requests actually land. The API key and provider config are stored in an httpOnly cookie, meaning they live on the server and never get sent to the browser.

The route reads that config, creates an OpenAI-compatible client, and hands the request to the CopilotKit runtime:

export const POST = async (req: NextRequest) => {
  const { baseURL, apiKey, model } = await getLLMConfig();

  const openai = new OpenAI({ baseURL, apiKey });
  const serviceAdapter = new OpenAIAdapter({ openai, model });
  const runtime = new CopilotRuntime();

  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

The defaults point to Ollama on localhost:11434/v1 with qwen2.5.

Because the OpenAI SDK accepts a custom baseURL, swapping to OpenAI, Groq or any other provider is just a config change with no code changes needed.

4. Feeding the LLM context

Before the LLM can answer anything useful, it needs to know what repo is loaded and what files exist. That is the job of the useCopilotContext.ts hook.

It calls useAgentContext, a CopilotKit hook that attaches structured data to every message you send. You can call it multiple times to attach different pieces of context.

Here it is called twice: once for the file list and once for the system instructions that tell the LLM how to behave:

useAgentContext({
  description: "File paths in the repository (max 500), one per line",
  value: fileList,  // flattened from the TreeNode structure
});

useAgentContext({
  description: "System instructions",
  value: `You are a Codebase Navigator assistant. You MUST use tool calls to answer questions.
CRITICAL RULES:
1. For ANY question about the repository call the "analyzeRepository" tool.
2. To show a file, call "fetchFileContent" with the exact file path.
3. NEVER respond with only text. ALWAYS call a tool first.
4. Use ONLY file paths from the file list above.`
});

The file list is capped at 500 paths to stay within token limits.

Because this hook re-runs every time the Zustand store changes, the LLM always sees the current repo, selected file and analysis state rather than a stale snapshot from when the page loaded.

Note: If you are running a local model with a large context window, you can raise this limit in useCopilotContext.ts.

5. The four frontend tools

This is the core of how the app works. Instead of just replying with text, the LLM can call tools. Each tool has a name, a set of parameters it accepts and a handler, which is just a function that runs when the LLM calls it.

CopilotKit lets you register these tools directly in the browser using the useFrontendTool hook, which is defined in useCopilotActions.ts.

When a tool's handler runs, it updates Zustand state directly, which is why all four panels, the graph, code viewer, analysis panel, and chat, react at the same time without any extra wiring.

analyzeRepository is the main tool the LLM calls for almost every question. Here is what it does step by step:

useFrontendTool({
  name: "analyzeRepository",
  parameters: z.object({
    query: z.string(),
    explanation: z.string(),
  }),
  handler: async ({ query, explanation }) => {
    // 1. Find relevant files using keyword matching
    const matchedPaths = findFilesByQuery(repo.tree, query);

    // 2. Cap at 15 files to keep things fast
    const capped = matchedPaths.slice(0, 15);

    // 3. Fetch each file content (cached)
    const fileData = await Promise.all(
      capped.map(async (p) => ({
        path: p,
        imports: extractImports(await fetchFile(owner, repo, p, branch)),
      }))
    );

    // 4. Build real dependency graph from actual import statements
    const graph = buildDependencyNodes(fileData);

    // 5. Categorize each node by file type
    graph.nodes.forEach(node => {
      node.type = categorizeFileType(node.metadata.fullPath);
    });

    // 6. Push to Zustand - all four panels react
    setAnalysisResult({ explanation, relevantFiles, flowDiagram: graph });
    setVisualization(graph.nodes, graph.edges, "dependency");
  },
});

fetchFileContent opens a specific file in the code viewer. The LLM calls this when you ask it to show you a file, it just fetches the content and calls setCodeViewer:

useFrontendTool({
  name: "fetchFileContent",
  parameters: z.object({
    filePath: z.string(),
  }),
  handler: async ({ filePath }) => {
    const content = await fetchFile(repo.repoInfo.owner, repo.repoInfo.repo, filePath, repo.repoInfo.branch);
    setCodeViewer(filePath, content);
  },
});

generateFlowDiagram is similar to analyzeRepository but more focused. Instead of searching the whole repo, you give it a specific list of file paths and it builds a graph for just those files. Useful when you want to visualize a slice of the codebase like just the API layer:

useFrontendTool({
  name: "generateFlowDiagram",
  parameters: z.object({
    files: z.array(z.string()),
    diagramType: z.enum(["dependency", "flow", "architecture"]),
  }),
  handler: async ({ files, diagramType }) => {
    const fileData = await Promise.all(files.slice(0, 20).map(async (f) => ({
      path: f,
      imports: extractImports(await fetchFile(repo.repoInfo.owner, repo.repoInfo.repo, f, repo.repoInfo.branch)),
    })));
    const graph = buildDependencyNodes(fileData);
    setVisualization(graph.nodes, graph.edges, diagramType);
  },
});

highlightCode fetches a file and highlights specific line numbers with an explanation. The LLM uses this when it wants to point at exactly where something happens in the code:

useFrontendTool({
  name: "highlightCode",
  parameters: z.object({
    filePath: z.string(),
    lines: z.array(z.number()),
    explanation: z.string(),
  }),
  handler: async ({ filePath, lines, explanation }) => {
    const content = await fetchFile(repo.repoInfo.owner, repo.repoInfo.repo, filePath, repo.repoInfo.branch);
    setCodeViewer(filePath, content, lines, explanation);
  },
});

Notice that none of these tools returns text to the chat. They all just update the state. The LLM is not describing what it found, it is directly controlling what you see on screen.

For instance, I asked about the architecture of v2 packages.

6. Finding relevant files

When analyzeRepository gets a query like "how does authentication work?", it needs to figure out which files to actually look at. That is handled by findFilesByQuery in src/lib/analyzer.ts.

It checks if the query matches one of six predefined categories, each with a set of keywords and file path patterns:

const ANALYSIS_CATEGORIES = [
  {
    name: "authentication",
    keywords: ["auth", "login", "logout", "token", "jwt", "session", "oauth"],
    filePatterns: [/auth/i, /login/i, /session/i, /oauth/i, /jwt/i],
  },
  {
    name: "database",
    keywords: ["database", "db", "model", "schema", "prisma", "mongoose"],
    filePatterns: [/model/i, /schema/i, /migration/i, /prisma/i, /db/i],
  },
  // ... api, configuration, testing, styling
]

If the query matches a category, it filters all file paths using that category's regex patterns.

If nothing matches, it falls back to splitting the query into individual terms and checking if any file path contains them. So even a vague query like "where is the config" will still find files with "config" in the name.

This is intentionally simple. No embeddings, no vector search, just regex on file names. It works well because file names in well-structured projects are usually descriptive enough that matching on them gets you the right files.

7. File caching

Every time a tool fetches a file, it goes through src/lib/fetch-file.ts which maintains an in-memory cache with a 5-minute TTL:

const cache = new Map<string, { content: string; timestamp: number }>();
const CACHE_TTL = 5 * 60 * 1000; // 5 minutes

export async function fetchFile(owner, repo, path, ref) {
  const key = `${owner}/${repo}/${ref}/${path}`;
  const cached = cache.get(key);

  if (cached && Date.now() - cached.timestamp < CACHE_TTL) {
    return cached.content; // serve from cache
  }

  const res = await fetch(`/api/github/file?repo=...&path=...&ref=...`);
  const data = await res.json();
  cache.set(key, { content: data.content, timestamp: Date.now() });

  // LRU eviction: if over 200 items, drop the oldest 50
  if (cache.size > 200) {
    const oldest = [...cache.entries()].sort((a, b) => a[1].timestamp - b[1].timestamp);
    for (let i = 0; i < 50; i++) cache.delete(oldest[i][0]);
  }

  return data.content;
}

This matters because analyzeRepository fetches up to 15 files at once. A follow-up question about the same files hits the cache instead of making 15 more API requests.

Without a GitHub token, you are limited to 60 requests per hour, so caching is what makes repeated questions fast and free.

The cache also evicts the oldest entries once it crosses 200 items, so it never grows unbounded in long sessions.

8. Parsing imports and resolving paths

Once analyzeRepository has the relevant file paths, it fetches each file and parses it for import statements. Both functions that handle this live in src/lib/analyzer.ts.

extractImports runs on the raw file content and pulls out every import, handling both ES6 and CommonJS syntax:

export function extractImports(content: string): string[] {
  const imports: string[] = [];

  // ES6: import X from 'y' and import 'y'
  const esImports = content.matchAll(
    /(?:import\s+.*?\s+from\s+['"](.+?)['"]|import\s+['"](.+?)['"])/g
  );
  for (const match of esImports) imports.push(match[1] || match[2]);

  // CommonJS: require('y')
  const requires = content.matchAll(/require\s*\(\s*['"](.+?)['"]\s*\)/g);
  for (const match of requires) imports.push(match[1]);

  return imports;
}

But raw import strings like "./utils" or "@/lib/github" are not file paths yet.

buildDependencyNodes calls resolveImportPath on each one to turn them into actual paths that exist in the repo:

function resolveImportPath(importPath, fromFile, filePathSet) {
  // Skip npm packages (no ./ or @/)
  if (!importPath.startsWith(".") && !importPath.startsWith("@/")) return null;

  // Resolve @/ alias to src/
  if (importPath.startsWith("@/")) {
    resolved = "src/" + importPath.slice(2);
  }

  // Try with common extensions: .ts, .tsx, .js, /index.ts, etc.
  const extensions = ["", ".ts", ".tsx", ".js", ".jsx", "/index.ts", "/index.tsx"];
  for (const ext of extensions) {
    if (filePathSet.has(resolved + ext)) return resolved + ext;
  }
  return null;
}

npm packages like react or zod are skipped immediately since they do not start with ./ or @/. For local imports, it tries common extensions like .ts, .tsx, and /index.ts, so it handles both direct file imports and folders that export through an index file.

If the resolved path does not exist in the repo, it returns null and no edge is created. That is what keeps the graph honest.

9. Graph layout with Dagre

Once the nodes and edges are built, they need actual positions on screen before React Flow can render them. That is what applyDagreLayout does in src/lib/graph-layout.ts.

It uses Dagre, a JavaScript library that automatically calculates x and y coordinates for every node in a directed graph so nothing overlaps, all on the client side with no server needed:

export function applyDagreLayout(nodes, edges, options = {}) {
  const g = new dagre.graphlib.Graph();
  g.setGraph({ rankdir: "TB", ranksep: 80, nodesep: 40 });

  // Tell dagre the size of each node
  for (const node of nodes) {
    g.setNode(node.id, { width: 200, height: 60 });
  }
  for (const edge of edges) {
    g.setEdge(edge.source, edge.target);
  }

  dagre.layout(g); // dagre calculates x,y for every node

  return nodes.map((node) => {
    const pos = g.node(node.id);
    return { ...node, position: { x: pos.x - 100, y: pos.y - 30 } };
  });
}

The direction can be toggled between top-to-bottom (TB) and left-to-right (LR) from the UI.

Each node type maps to a React Flow component based on the file type categorizeFileType assigned earlier:

module / service → ModuleNode (indigo)
function → FunctionNode (teal)
file → FileNode (gray)

10. State management with Zustand

All four panels stay in sync because they all read from the same Zustand store in src/store/index.ts. The store has four slices, one for each major concern:

interface AppState {
  repo: {
    repoInfo: RepoInfo | null;   // owner, repo, branch
    tree: TreeNode | null;        // full file tree
    selectedFile: string | null;  // active file path
    loading: boolean;
    error: string | null;
  };
  analysis: {
    result: AnalysisResult | null; // explanation + relevant files + graph
    loading: boolean;
    error: string | null;
  };
  visualization: {
    nodes: FlowNode[];
    edges: FlowEdge[];
    graphType: "dependency" | "flow" | "architecture" | null;
  };
  codeViewer: {
    filePath: string | null;
    content: string | null;
    highlightedLines: number[];
    explanation: string | null;
  };
}

Each panel subscribes to only the slice it needs using a selector like useAppStore((s) => s.visualization).

So when a tool calls setVisualization, only the graph panel re-renders. When setCodeViewer fires, only the code viewer updates. Nothing else moves.

Settings like LLM provider, API key and model live in a separate SettingsState store. It syncs to two places:

localStorage so the browser remembers your config across reloads
an httpOnly cookie so the server-side /api/copilotkit route can read them without the API key ever appearing in request headers

This is how everything works behind the scenes. It's time to run it locally.

Running it locally

Clone the GitHub repository and install the dependencies.

git clone <repo-url>
cd codebase-navigator
npm install
npm run dev

Open http://localhost:3000 to view it live in the browser. Here is how to set up each provider.

Using OpenAI

You will need an OpenAI API Key. Then just click the Settings icon in the top right, switch the provider to OpenAI, paste your API key and pick a model like gpt-5.2. That's it.

Using Ollama (free, runs locally)

You can download it from ollama.com or install it via these commands based on the operating system you use.

brew install ollama #macOS
curl -fsSL https://ollama.com/install.sh | sh #linux

Then pull the model and start the server:

ollama serve
ollama pull qwen2.5 # in another terminal

The app points to Ollama by default so nothing else needs to change. Just open http://localhost:3000 and it works.

For GitHub, public repos work without a token (the only catch is that it's limited to 60 requests per hour). Add a GITHUB_TOKEN to a .env.local file for higher limits:

GITHUB_TOKEN=your_token_here

I tried it with the CopilotKit GitHub repo and it immediately maps out the entire repo structure.

I asked it, "tell me about architecture" and it gave me:

a graph of every relevant file connected by real imports
a plain english breakdown of what each file does
raw code fetched live from GitHub
a detailed chat response I could keep following up on

I tried difficult and vague questions and it handled all of them.

The bigger picture

This project started with wiring three ideas together and seeing what happened.

GitHub is the data source, not just a host. Every file the AI references gets fetched live, parsed, and reasoned about on the spot. No hallucination, no guessing from memory.

CopilotKit changes what the AI can actually do in the browser. Instead of replying with text, it calls tools that update the state directly. The graph changes. The code viewer opens. The AI is not describing what it found, it is showing you.

The dependency edges are real. A lot of tools draw graphs that look impressive but mean nothing. Every edge here exists because one file has an actual import statement pointing to another. That is it.

Wire those three together and you get something that actually understands the codebase instead of just searching through it.

There is a lot you could extend from here: PR diff exploration, multi-repo comparison, security audit mode. But as a starting point for understanding any codebase, it already does exactly what it needs to.

Let me know what you think in the comments!

You can connect me on GitHub, Twitter and LinkedIn.

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

Deploying AG-UI Agents to Production with Amazon Bedrock AgentCore

Anmol Baranwal — Tue, 24 Mar 2026 17:00:06 +0000

The Agent-User Interaction Protocol (AG-UI) standardizes how AI agents communicate with user applications.

But deploying those agents means the same manual work: configuring SSE endpoints, writing auth middleware, managing session isolation, and setting up auto-scaling. None of that is your agent logic. All of it had to be built before you could ship.

Amazon recently announced native AG-UI support in Amazon Bedrock AgentCore Runtime across 14 AWS regions. You bring your agent code in a container, set one protocol flag, and AgentCore handles the rest.

So I deployed Strands Agents to learn how it actually works end-to-end. This post covers everything I picked up, including what AgentCore is and the full deployment flow from a local server to a live UI.

Amazon Bedrock AgentCore Runtime

Amazon Bedrock AgentCore is an agentic platform for building, deploying, and operating agents securely at scale. AgentCore Runtime is a fully managed hosting environment.

You bring your agent code in a container, and AgentCore checks who's allowed in (auth), keeps each user's session separate (Session isolation), adds capacity when traffic spikes (auto-scaling) and logs everything to CloudWatch (observability). You don't write any of this code.

It's also framework-agnostic: agents built with Strands, LangGraph, CrewAI, or any other framework can deploy to AgentCore. Think of it as the production layer between your agent and your users.

It supports four protocols:

HTTP: default agent invocation
MCP: agent ↔ tools
A2A: agent ↔ agent
AG-UI: agent ↔ user (new)

You can read more about the differences in message format, discovery and authentication in their docs.

You configure with --protocol AGUI and the protocol layer is taken care of. Your job is just the agent logic.

AG-UI: the interaction protocol

AG-UI (Agent-User Interaction Protocol) is an open, event-based protocol that standardises how AI agents communicate with user-facing applications.

Instead of a custom WebSocket or polling layer, AG-UI defines a shared event stream between your agent backend and your frontend UI. It covers:

Streaming text responses as they generate
Tool call lifecycle (start to result)
State updates and progress
User interactions sent back into the agent mid-execution

In practice, when your agent responds, the frontend receives a sequence of typed events:

data: {"type": "RUN_STARTED", "threadId": "test-123", "runId": "run-456"}
data: {"type": "TEXT_MESSAGE_START", "messageId": "msg-abc", "role": "assistant"}
data: {"type": "TEXT_MESSAGE_CONTENT", "messageId": "msg-abc", "delta": "Hello!"}
data: {"type": "TEXT_MESSAGE_END", "messageId": "msg-abc"}
data: {"type": "RUN_FINISHED", "threadId": "test-123", "runId": "run-456"}

Any AG-UI-compatible frontend, including CopilotKit, can consume that stream directly. In the next section, we will see how it all connects through an architecture diagram.

How It All Connects

CopilotKit is an AG-UI-compatible React client. It connects to an AG-UI endpoint, subscribes to the event stream and renders streaming responses, tool progress, and state updates inside your app.

Once your agent is deployed to AgentCore, HttpAgent points to the AgentCore invocation URL with a Cognito Bearer token in the headers, and user messages flow back over AG-UI into the running agent.

Here is what the full stack looks like when deployed:

Your agent framework  [Strands · LangGraph · CrewAI]
│
│  deployed to
▼
Amazon Bedrock AgentCore Runtime  [Bearer token via Cognito OAuth 2.0]
│  auth · session isolation · auto-scaling · observability
│
│  invokes container
▼
Agent container  [ARM64 · port 8080]
│  FastAPI · /invocations · /ping
│
│  calls model
▼
Amazon Bedrock model
│
│  streams back via
▼
AG-UI protocol  [SSE event stream]
│
│  consumed by
▼
CopilotKit  [HttpAgent · @ag-ui/client]
│
├─  renders streaming responses + tool progress
├─  syncs state updates in real time
└─  forwards user messages → agent via AG-UI
▼
Your application UI

If you want a complete full-stack starting point, AWS also released the Fullstack AgentCore Solution Template - a ready-made stack with AgentCore, LangGraph, CopilotKit and AG-UI all wired together, deployed via CDK. It's the fastest way to go from zero to a production-ready agentic app on AWS.

Deploying your AG-UI agent with Strands

This example uses AWS Strands as the agent framework but the same pattern works with LangGraph, CrewAI and any other AG-UI-compatible framework.

Before deploying, make sure you have:

Python 3.12+
AWS account with credentials configured (aws configure)
IAM permissions for AgentCore Runtime
Docker, Finch, or Podman for building container images
Amazon Cognito user pool configured (Setup guide here)

Start by installing the required packages:

pip install fastapi uvicorn ag_ui_strands strands-agents boto3

ag_ui_strands wraps your Strands agent with AG-UI protocol support, handling event encoding automatically.
strands-agents is the AWS agent framework used in this example.
boto3 is the AWS SDK, used internally by Strands to call Bedrock models.

Step 1: Build your AG-UI server

Create my_agui_server.py. This defines the model, wraps the agent with AG-UI support, and exposes the endpoints AgentCore needs.

import uvicorn
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse, JSONResponse
from ag_ui_strands import StrandsAgent
from ag_ui.core import RunAgentInput
from ag_ui.encoder import EventEncoder
from strands import Agent
from strands.models.bedrock import BedrockModel

model = BedrockModel(
    model_id="us.anthropic.claude-3-5-sonnet-20241022-v2:0",
    region_name="us-west-2",
)

strands_agent = Agent(
    model=model,
    system_prompt="You are a helpful assistant.",
)

agui_agent = StrandsAgent(
    agent=strands_agent,
    name="my_agent",
    description="A helpful assistant",
)

app = FastAPI()

@app.post("/invocations")
async def invocations(input_data: dict, request: Request):
    accept_header = request.headers.get("accept")
    encoder = EventEncoder(accept=accept_header)

    async def event_generator():
        run_input = RunAgentInput(**input_data)
        async for event in agui_agent.run(run_input):
            yield encoder.encode(event)

    return StreamingResponse(
        event_generator(),
        media_type=encoder.get_content_type()
    )

@app.get("/ping")
async def ping():
    return JSONResponse({"status": "Healthy"})

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8080)

Here, StrandsAgent wraps your Strands agent and handles all AG-UI event encoding so you don't write any protocol logic yourself.

AgentCore requires three things from your container:

POST /invocations - handles AG-UI requests, returns the SSE event stream.
GET /ping - health check, must return HTTP 200.
Port 8080 - required, do not change.

For LangGraph, CrewAI, and other frameworks, see the AG-UI framework integrations.

Step 2: Test locally

Start your server.

python my_agui_server.py

The server starts on http://localhost:8080. Send a test request to confirm the SSE stream is working before deploying.

curl -N -X POST http://localhost:8080/invocations \
  -H "Content-Type: application/json" \
  -d '{
    "thread_id": "test-123",
    "run_id": "run-456",
    "state": {},
    "messages": [{"role": "user", "content": "Hello!", "id": "msg-1"}],
    "tools": [], "context": [], "forwardedProps": {}
  }'

You should see the SSE events stream back:

data: {"type": "RUN_STARTED", "threadId": "test-123", "runId": "run-456"}
data: {"type": "TEXT_MESSAGE_START", "messageId": "msg-abc", "role": "assistant"}
data: {"type": "TEXT_MESSAGE_CONTENT", "messageId": "msg-abc", "delta": "Hello!"}
data: {"type": "TEXT_MESSAGE_END", "messageId": "msg-abc"}
data: {"type": "RUN_FINISHED", "threadId": "test-123", "runId": "run-456"}

Step 3: Deploy to AgentCore

The AgentCore starter toolkit handles the entire deployment process for you.

It builds an ARM64 container image, pushes it to Amazon ECR, and creates the AgentCore runtime with protocol flag serverProtocol: "AGUI". You don't write a Dockerfile or configure any infrastructure manually.

Install it first:

pip install bedrock-agentcore-starter-toolkit

Create a requirements.txt so the toolkit knows what to install in the container:

fastapi 
uvicorn 
ag_ui_strands

Now configure and deploy. The configure command sets the protocol to AG-UI and links your Cognito auth setup. The deploy command does the rest:

# Configure with AG-UI protocol + OAuth authentication
agentcore configure -e my_agui_server.py --protocol AGUI

# Deploy to AWS
agentcore deploy

After a successful deploy, you receive an agent runtime ARN. You will need this to invoke the agent and connect your frontend.

arn:aws:bedrock-agentcore:us-west-2:ACCOUNT_ID:runtime/my_agui_server-xyz123

Step 4: Invoke your deployed agent

With the agent running on AgentCore, you can now invoke it and stream AG-UI events over the wire. Set your environment variables first:

export BEARER_TOKEN="<your-cognito-bearer-token>"
export AGENT_ARN="arn:aws:bedrock-agentcore:us-west-2:ACCOUNT_ID:runtime/my_agui_server-xyz123"

Install the HTTP client libraries for making SSE requests:

pip install httpx httpx-sse

Then invoke the agent:

import asyncio
import json
import os
from urllib.parse import quote
from uuid import uuid4

import httpx
from httpx_sse import aconnect_sse

async def invoke_agui_agent(message: str):
    agent_arn = os.environ.get('AGENT_ARN')
    bearer_token = os.environ.get('BEARER_TOKEN')
    escaped_arn = quote(agent_arn, safe='')

    url = f"https://bedrock-agentcore.us-west-2.amazonaws.com/runtimes/{escaped_arn}/invocations?qualifier=DEFAULT"
    headers = {
        "Authorization": f"Bearer {bearer_token}",
        "X-Amzn-Bedrock-AgentCore-Runtime-Session-Id": str(uuid4()),
    }
    payload = {
        "threadId": str(uuid4()),
        "runId": str(uuid4()),
        "messages": [{"id": str(uuid4()), "role": "user", "content": message}],
        "state": {},
        "tools": [],
        "context": [],
        "forwardedProps": {},
    }

    async with httpx.AsyncClient(timeout=300) as client:
        async with aconnect_sse(client, "POST", url, headers=headers, json=payload) as sse:
            async for event in sse.aiter_sse():
                data = json.loads(event.data)
                event_type = data.get("type")
                if event_type == "TEXT_MESSAGE_CONTENT":
                    print(data.get("delta", ""), end="", flush=True)
                elif event_type == "RUN_ERROR":
                    print(f"Error: {data.get('code')} - {data.get('message')}")

asyncio.run(invoke_agui_agent("Hello from production!"))

You should see the agent's response stream token by token.

The same AG-UI events as local testing, now served from AgentCore with managed authentication, session isolation, and auto-scaling.

Step 5: Connect your frontend

With the agent deployed, point HttpAgent from @ag-ui/client at the AgentCore invocation URL instead of localhost. The rest of the client code stays the same.

This is the same AG-UI client used locally - you are just pointing it at the AgentCore URL instead of localhost.

import { HttpAgent } from "@ag-ui/client";

const agent = new HttpAgent({
  url: `https://bedrock-agentcore.us-west-2.amazonaws.com/runtimes/${encodedArn}/invocations?qualifier=DEFAULT`,
  headers: {
    "Authorization": `Bearer ${token}`,
    "X-Amzn-Bedrock-AgentCore-Runtime-Session-Id": sessionId,
  }
});

For SigV4 authentication, sign requests using the AWS SDK. For OAuth 2.0, obtain a token from your configured Cognito user pool. Authentication configuration is the same across all AgentCore protocols. See the AgentCore auth docs for setup.

AG-UI support in AgentCore is available today across 14 AWS regions.

Resources

You can start building right away.

Explore the AgentCore AG-UI docs
Explore the AG-UI Protocol Docs
Check out the AWS Announcement

If you want a complete full-stack starting point, AWS also released the Fullstack AgentCore Solution Template - AgentCore, LangGraph, CopilotKit, and AG-UI all wired together and deployed via CDK.

What This Means

AG-UI works with LangGraph, CrewAI, AWS Strands, Google ADK, Mastra, Microsoft Agent Framework, LlamaIndex, Agno, and more. The protocol itself isn't tied to any of them - you pick the framework, AG-UI handles the interaction layer.

What was missing was a managed way to run these agents in production. That's what this support adds.

Beyond AG-UI hosting, AgentCore also covers persistent memory across sessions, tool connectivity via MCP Gateway, and production observability (though I didn't explore all of these in-depth).

thanks for reading!

You can connect me on GitHub, Twitter and LinkedIn.

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

How to add AI to your App in 5 minutes

Anmol Baranwal — Wed, 11 Mar 2026 15:10:31 +0000

Most AI integrations are overengineered.

Even basic tutorials on adding an AI assistant get mixed up with LangGraph workflows and FastAPI servers. You don't need any of that.

Today, we will build one without any external agent framework or orchestration layers. Just an AI that knows what's in your app and can actually do things.

You will also learn how to implement the Generative UI pattern, which lets you render actual components rather than just text responses.

Let's jump in.

Why AI Integrations Get Complicated

Adding AI to an app sounds simple until you actually try it.

The moment you go beyond a basic chat interface, you run into a fundamental problem: LLMs on their own can't do anything. They can only generate text.

To make an AI that actually takes actions: reads your app's state, updates data, calls APIs - you need to build the plumbing yourself.

That plumbing is what an AI agent is. At its core, every agent runs the same loop:

Observe - take in context (user input, tool results, memory)
Reason - decide what to do next
Act - call a tool, write to a file, hit an API, whatever's needed

Credit: Anthropic

Simple enough in theory. But in practice, you end up writing the orchestration layer that manages this loop, tool registries that expose your app's functions to the LLM, state management to track what the agent knows and error handling for when the model goes sideways.

context = [initial_event]
while True:
  next_step = await llm.determine_next_step(context)
  context.append(next_step)
  if next_step.intent == "done":
    return next_step.final_answer
  result = await execute_step(next_step)
  context.append(result)

In a full agent setup, the LLM sits at the center - orchestrating across data sources, tools, models, and external services.

Credit: Medium blog

That's why tutorials teach for LangGraph and FastAPI servers. And for complex automation pipelines, that complexity is justified.

But if your goal is just to add an AI assistant inside your product - one that understands your UI, reads your data, and can take actions on behalf of the user, you are building a lot of infrastructure to solve a much smaller problem.

CopilotKit handles the loop, the context, the streaming and the frontend integration so you don't have to wire any of it yourself.

For this tutorial, we will use CopilotKit without any external agent framework, just hooks and your LLM of choice.

What CopilotKit Actually Does

CopilotKit is an open source framework to add AI agents to your app that abstracts away everything you just read about.

Instead of building your own agent loop, tool registry and streaming layer, you get a set of hooks and pre-built components that plug directly into your existing frontend.

The mental model is simple, you provide the AI with two things:

Your app's state - so it understands what's on screen and what the user is working with
A set of tools - so it can actually do something useful, not just respond with text

In practice, this maps to two hooks:

useAgentContext - share app state with the AI. Whatever you pass in becomes part of the LLM's context: current user, selected items, loaded data - the AI knows what your app knows.
useFrontendTool - define actions the AI can trigger. You give it a name, a description, and a handler. The LLM decides when to call it based on what the user asks.

The result is that adding an AI assistant to your app becomes a frontend problem, not an infrastructure problem.

Your UI, agent, tools are in one single interaction loop.

How to wire everything (1 minute)

You can follow along with the official docs or just stick with this. I will walk through exactly what each piece does and why it's there.

I'm using Next.js with TypeScript, but this works with any React/Angular framework out there!

// creates a nextjs app  
npx create-next-app@latest .

Install the three CopilotKit packages you need:

npm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

@copilotkit/react-core/v2 : hooks and built-in chat components
@copilotkit/react-ui/v2 : styles
@copilotkit/runtime : backend runtime and LLM adapters

Create the API route at app/api/copilotkit/route.ts.

It's the single endpoint that receives messages from your UI, runs them through the agent loop and streams responses back.

BuiltInAgent is what runs the observe → reason → act loop
CopilotRuntime manages the session, streaming, and tool execution

That's your entire backend.

import { CopilotRuntime, copilotRuntimeNextJSAppRouterEndpoint } from "@copilotkit/runtime";
import { BuiltInAgent } from "@copilotkit/runtime/v2";
import { NextRequest } from "next/server";

const builtInAgent = new BuiltInAgent({
  model: "openai:gpt-5",
  // apiKey: process.env.OPENAI_API_KEY,
});

const runtime = new CopilotRuntime({
  agents: { default: builtInAgent },
});

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    endpoint: "/api/copilotkit",
  });
  return handleRequest(req);
};

Create .env.local in the root and add your OpenAI API key.

OPENAI_API_KEY=sk-proj-...

If you want to switch to any other LLM Provider, you just need to pass the modified string to BuiltInAgent in your API route. Everything else remains the same.

// Anthropic
const builtInAgent = new BuiltInAgent({ model: "anthropic:claude-sonnet-4-5" });

// Google
const builtInAgent = new BuiltInAgent({ model: "google:gemini-2.0-flash" });

Add the matching key to .env.local and you are done. For custom models like Azure OpenAI, AWS Bedrock or Ollama, check the model selection docs.

Now, connect the frontend to that backend by wrapping your app in app/layout.tsx.

import { CopilotKit } from "@copilotkit/react-core";
import "@copilotkit/react-ui/v2/styles.css";

export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>
        <CopilotKit runtimeUrl="/api/copilotkit">
          {children}
        </CopilotKit>
      </body>
    </html>
  );
}

CopilotKit is the context provider that every hook below it depends on. runtimeUrl points to the route you just created.

There are many in-built components like CopilotSidebar, CopilotChat and CopilotPopup. For this, I will be adding the chat sidebar.

import { CopilotSidebar } from "@copilotkit/react-core/v2";

export default function Page() {
  return (
    <main>
      <h1>Your App</h1>
      <CopilotSidebar />
    </main>
  );
}

Run npm run dev and you now have a working AI sidebar assistant! 🎉

It can respond to anything but it has no idea what's actually in your app. It doesn't know your data, your UI state or what the user is looking at. The next two steps fix that.

Give the AI Context

By default, the AI knows nothing about what's in your app. You can ask it, "how much did I spend on food?" and it has no idea - it only sees the conversation, not your app's state.

useAgentContext solves that. It pushes your React state into the agent's context window on every turn, so the AI always has a current snapshot of what's in your UI.

Let's build a simple expense tracker to see how this works.

"use client";
import { useState } from "react";
import { useAgentContext, CopilotSidebar } from "@copilotkit/react-core/v2";

type Expense = {
  id: number;
  description: string;
  amount: number;
  category: string;
};

const initialExpenses: Expense[] = [
  { id: 1, description: "Groceries", amount: 85, category: "Food" },
  { id: 2, description: "Netflix", amount: 15, category: "Entertainment" },
  { id: 3, description: "Uber", amount: 22, category: "Transport" },
];

export default function Page() {
  const [expenses, setExpenses] = useState<Expense[]>(initialExpenses);

  useAgentContext({
    description: "The user's current expense list. Each item has an id, description, amount in dollars, and category.",
    value: expenses,
  });

  return (
    <main className="p-8">
      <h1 className="text-2xl font-bold mb-4">My Expenses</h1>
      <ul className="space-y-2">
        {expenses.map((e) => (
          <li key={e.id} className="flex justify-between border-b py-2">
            <span>
              {e.description}{" "}
              <span className="text-gray-400 text-sm">({e.category})</span>
            </span>
            <span>${e.amount}</span>
          </li>
        ))}
      </ul>
      <CopilotSidebar />
    </main>
  );
}

expenses is your normal React state. useAgentContext takes it and injects it into the LLM's context on every turn.

Now, if the user asks "what's my biggest expense?", the AI is looking at the same list your UI renders.

Let the AI Do Things

Awareness is one thing. Being able to act is what makes an agent actually useful.

useFrontendTool lets you hand the AI a set of actions it can trigger. You define a tool with a name, a description, and a handler - the LLM decides when to call it based on what the user asks.

The LLM reads the description fields to understand what each tool does and what each parameter expects, so write them clearly and the model will fill them accurately even from casual conversational input.

Add useFrontendTool inside the same page.

parameters takes a Zod schema (a TypeScript-first validation library). If you haven't used it before, the idea is simple: you define the shape of your data with z.object({...}) and each field gets a .describe() call that tells the LLM what it means.

Install it with npm install zod.

"use client";
import { useState } from "react";
import { useAgentContext, useFrontendTool, CopilotSidebar } from "@copilotkit/react-core/v2";
import { z } from "zod";

// ... type and initialExpenses stay the same

export default function Page() {
  const [expenses, setExpenses] = useState<Expense[]>(initialExpenses);

  // ... useAgentContext

  useFrontendTool({
  name: "addExpense",
  description:
    "Add a new expense when the user mentions spending money on something.",
  parameters: z.object({
    description: z
      .string()
      .describe("What the expense was for, e.g. Lunch, Taxi, Coffee"),
    amount: z.number().describe("How much was spent in dollars"),
    category: z
      .string()
      .describe("Category: Food, Transport, Entertainment, Health, or Other"),
  }),
  handler: async ({ description, amount, category }) => {
    setExpenses((prev) => [
      ...prev,
      { id: Date.now(), description, amount, category },
    ]);
  },
});

  return (
    // ... remains the same
  );
}

You can send a sample query like "I spent $40 on dinner last night." The agent calls addExpense with { description: "Dinner", amount: 40, category: "Food" }, your handler updates expenses and the new item appears in the list instantly.

This is the observe → reason → act loop running end to end.

The agent observed your message and your current expenses via useAgentContext, reasoned that addExpense is the right call, and acted by invoking your handler with the right parameters. CopilotKit handled everything in between.

Bonus: Generative UI

The AI can read your data and update it. But so far, it only responds in text.

Generative UI is a new idea: instead of the agent describing results, it renders actual UI. For example, if someone asks about their spending, the app would display a breakdown card.

CopilotKit supports this through the render property on useFrontendTool. Instead of a text reply, the tool returns a React component - rendered inline in the chat, using your own design system.

Let's add a summary tool inside the same component.

import { ToolCallStatus, useFrontendTool } from "@copilotkit/react-core/v2";

useFrontendTool({
  name: "showSpendingSummary",
  description:
    "Call this when the user asks for a summary or overview of their expenses.",
  parameters: z.object({}),
  handler: async () => {
    const summary = expenses.reduce(
      (acc, e) => {
        acc[e.category] = (acc[e.category] ?? 0) + e.amount;
        return acc;
      },
      {} as Record<string, number>,
    );
    const total = expenses.reduce((sum, e) => sum + e.amount, 0);
    return JSON.stringify({ summary, total });
  },
  render: ({ result, status }) => {
    return (
      <div className="rounded-lg border p-4 mt-2 space-y-3">
        <p className="font-semibold text-sm">
          {status === ToolCallStatus.InProgress ? "Calculating..." : "Spending Breakdown"}
        </p>
        {status === ToolCallStatus.Complete && result && (
          <>
            {Object.entries(
              (JSON.parse(result) as { summary: Record<string, number>; total: number }).summary
            ).map(([category, amount]) => (
              <div key={category} className="flex justify-between text-sm">
                <span className="text-gray-600">{category}</span>
                <span className="font-medium">${amount}</span>
              </div>
            ))}
            <div className="flex justify-between text-sm font-semibold border-t pt-2">
              <span>Total</span>
              <span>${(JSON.parse(result) as { summary: Record<string, number>; total: number }).total}</span>
            </div>
          </>
        )}
      </div>
    );
  },
});

What's happening here:

parameters: z.object({}) is empty. The LLM doesn't need to pass anything, it just needs to decide when to call the tool
handler does the actual work, calculates category totals from the expenses state and returns them
ToolCallStatus gives you the exact lifecycle states of the tool call
result is the JSON string returned by your handler, parsed back into an object inside render

When you ask, "summarize my spending." LLM reads the query, looks at the context, recognizes that showSpendingSummary is the right tool to call and triggers it.

That's the pattern: the LLM decides when to act, your code does the work, your components display the result.

The working implementation of this minimal expense tracker is available in this GitHub repo in the example/basic branch.

The main branch takes the same pattern further with a full kanban board:

Move tasks between columns by asking the AI
Add, delete and reassign tasks from natural language
Get a visual board summary with generative UI

Here is the demo!

Most AI features that feel magical aren't. They are just an app that knows what's on screen and has a few actions wired up. That's it.

The hard part was always the infrastructure. Turns out it doesn't have to be.

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

Give Your AG2 Agents a UI with AG‑UI and CopilotKit

Anmol Baranwal — Thu, 26 Feb 2026 15:01:48 +0000

Agent frameworks have become increasingly capable at reasoning, orchestration and tool execution. But most of them still operate as isolated systems.

AG2 (formerly AutoGen) changes that. It provides a shared runtime where specialized agents with defined roles, tools and conversation patterns can collaborate as one coordinated team across models and even frameworks.

But orchestration alone isn’t the same as shipping an interactive experience. Every application ends up building custom protocols and UI wiring to observe runs, show tool progress and capture user actions during execution.

AG-UI (Agent-User Interaction Protocol) defines that interaction contract. It standardizes how agent activity is streamed, so any frontend can stay synchronized in real time.

Today, AG2 integrates with AG‑UI and CopilotKit to bring agent orchestration into real applications.

You can refer to the docs and try the interactive AG-UI Dojo if you want to explore it yourself.

Let’s break down how these layers fit together, the end-to-end integration flow and what this unlocks for agent builders.

AG2: Runtime for Agent Teams

AG2 (formerly AutoGen) is an open-source programming framework built around multi-agent collaboration. You define specialized agents, wire tools and run structured conversations to solve tasks.

Instead of a single monolithic loop, AG2 treats agents as composable actors that collaborate through shared context.

At its core is ConversableAgent, the primitive that defines an agent’s role, reasoning model and tool access while handling message exchange.

from autogen import ConversableAgent, LLMConfig
from dotenv import load_dotenv
import os
load_dotenv()

# Create LLM configuration first
llm_config = LLMConfig({
  "api_type": "openai",
  "model": "gpt-5-mini",
  "api_key": os.getenv("OPENAI_API_KEY"),
})

# Create the agent
lesson_planner = ConversableAgent(
    name="planner_agent",
    system_message="You are a lesson planner for a fourth grade class.",
    description="Creates lesson plans.",
    llm_config=llm_config,
)

Agents can be organized into coordinated workflows using built-in patterns such as sequential flows, group conversations, nested calls and human-in-the-loop checkpoints.

In practice, you might model a planner, executor and reviewer as distinct agents: each with its own role and tools and let AG2 manage how they interact over multiple turns.

auto_selection = AutoPattern(
    agents=[teacher, lesson_planner, lesson_reviewer],
    initial_agent=lesson_planner,
    group_manager_args={"llm_config": llm_config},
)

response = run_group_chat(
    pattern=auto_selection,
    messages="Introduce students to the solar system.",
    max_rounds=20,
)

response.process()

This allows backend behavior to evolve by assembling agents differently, rather than rewriting control logic for every new workflow.

AG-UI integration currently wraps a single ConversableAgent. Multi-agent patterns (group chat, delegation, nested flows) still run inside AG2, but are exposed through a unified interaction surface.

Staged or multi-agent UX can be modeled using context variables. The pattern is covered in the integration section.

Read all the basic concepts involved in the docs and check the example repo, which includes many examples.

The Gap: Orchestration Without an Application Contract

Even if multi-agent logic is portable, it does not automatically become app-portable.

In a production application, the UI needs a stable runtime contract for:

Streaming messages and partial results.
Tool lifecycle (start → progress → finish/fail).
State snapshots and updates while the run is still executing.
Human-in-the-loop actions that can steer or interrupt a run

AG2 manages delegation, shared context and orchestration internally. But it does not define a standard event/state interface for how that execution is exposed to a frontend.

Without that contract, every AG2 app ends up inventing its own WebSocket/SSE payload shapes and UI mapping rules. Integration becomes tightly coupled glue code instead of a reusable interaction layer.

This is where AG-UI Protocol fits.

AG-UI: The Execution & Interaction Layer

AG-UI (Agent–User Interaction Protocol) is an event-based protocol that standardizes the live interaction stream between an agent runtime and an application.

During a run, the backend emits a stream of structured event types. The frontend consumes that stream and can send compatible interaction events back.

This matters because AG2 already manages multi-agent coordination internally. What AG-UI defines is how that coordination becomes observable and interactive at the application boundary.

Instead of every app inventing custom payload formats, AG-UI defines a small set of self-describing JSON event types.

Each event includes a clear type and structured payload. For example:

TEXT_MESSAGE_CONTENT : event streams LLM tokens
TOOL_CALL_START/END : convey function call progress
STATE_DELTA : carries JSON Patch deltas to sync state

In practice, AG‑UI standardizes:

Message streaming
Tool lifecycle events (start → progress → finish / fail)
State snapshots and incremental updates
User interaction events (clicks, form submits, confirmations)

Because the stream is standard and ordered, the frontend can reliably interpret what the backend is doing and render progress in real time.

The result is decoupling. The backend doesn’t need to know how the UI renders progress and the UI doesn’t need to understand internal orchestration logic.

Here, AG-UI uses Server-Sent Events (SSE) over HTTP as its primary transport. The AG2 integration serves a text/event-stream response per run.

In AG2, this protocol is implemented via AGUIStream, which wraps a ConversableAgent and exposes it as an ASGI endpoint that streams AG-UI events.

Explore the full event specification in the AG-UI documentation and implementation details in the AG2 user guide, including best practices for when to adopt this integration.

How CopilotKit wires AG2 to AG-UI

On the backend, AG2 exposes agent runs through an AG-UI-compatible endpoint using AGUIStream. That endpoint streams standardized AG-UI events for the lifecycle of a run.

CopilotKit acts as that client and UI runtime. It connects to the AG‑UI endpoint (via @ag-ui/client), subscribes to the event stream, renders agent activity inside your application and forwards user actions back into the same execution loop.

The wiring is straightforward:

Your backend runs an AG2 agent workflow
AGUIStream wraps the agent and translates execution into AG-UI events
Those events are exposed through an AG-UI endpoint over SSE
CopilotKit connects and subscribes to the live event stream
Agent messages, tool progress and state updates are rendered in the UI
User interactions are sent back over AG-UI into the same AG2 run

The same endpoint can be consumed by any other AG-UI-compatible client. At a high level, the full interaction loop looks like this:

AG2 Runtime (multi-agent orchestration)
│
│ executes ConversableAgents
│ manages delegation + shared state
│ invokes tools
▼
AG2 ↔ AG-UI Integration Layer (AGUIStream)
│
│ wraps ConversableAgent
│ maps AG2 activity → standardized AG-UI events
│ handles SSE transport
▼
AG-UI Endpoint (event + state stream)
│
│ agent ↔ application synchronization
│ TEXT_MESSAGE_CONTENT
│ TOOL_CALL_START / END
│ STATE_DELTA / STATE_SNAPSHOT
│ USER_INTERACTION
▼
CopilotKit (AG-UI client runtime)
│
├─ subscribes to live stream
├─ renders messages + tool lifecycle
├─ synchronizes state updates
└─ forwards user actions → AG2 via AG-UI
▼
Application UI

The backend remains purely AG2. CopilotKit simply consumes the AG-UI stream and renders it as product UI.

Integration Flow: AG2 → AG-UI → CopilotKit

We have seen the conceptual flow. Now let’s look at the actual implementation.

You can check the AG-UI integration examples in the build-with-ag2 repository, which consists of two patterns: weather/ directory shows a single-agent chat with tools & factory/ directory demonstrates a multi-stage pipeline using ContextVariables.

For complete CopilotKit + AG2 integration with full UI implementations, explore the ag2-samples repository.

The integration centers on AGUIStream, which wraps a ConversableAgent and exposes it as an AG‑UI endpoint over SSE.

Define your agent exactly as you normally would, then mount the stream on FastAPI. The only important detail is enabling streaming in the LLM configuration.

from autogen import ConversableAgent, LLMConfig
from autogen.ag_ui import AGUIStream
from fastapi import FastAPI

agent = ConversableAgent(
    name="weather_agent",
    system_message="You are a helpful weather assistant. Use the get_weather tool to look up current conditions for any city.",
    llm_config=LLMConfig({
        "model": "gpt-5-mini",
        "stream": True
    }),
    functions=[get_weather],
)

# wrap the agent and mount it as an ASGI endpoint
stream = AGUIStream(agent)  

app = FastAPI()  
app.mount("/chat", stream.build_asgi())

That’s the entire backend bridge.

AGUIStream automatically maps:

Token streaming → TEXT_MESSAGE_*
Tool execution → TOOL_CALL_START, TOOL_CALL_RESULT
Context updates → STATE_SNAPSHOT / STATE_DELTA
Run lifecycle → RUN_STARTED, RUN_FINISHED, RUN_ERROR

The endpoint serves Server-Sent Events (SSE). When a client sends a RunAgentInput payload with Accept: text/event-stream, the response becomes a live event stream.

Here's a simplified example of what flows back:

data: {"type":"RUN_STARTED", ...}
data: {"type":"TEXT_MESSAGE_CONTENT","delta":"The"}
data: {"type":"TEXT_MESSAGE_CONTENT","delta":" weather"}
data: {"type":"TOOL_CALL_START", ...}
data: {"type":"TOOL_CALL_RESULT", ...}
data: {"type":"RUN_FINISHED", ...}

The frontend does not need to understand AG2 internals. It reacts to event types. At the lowest level, consuming AG-UI is just opening an SSE stream:

const response = await fetch("/chat", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Accept": "text/event-stream"
  },
  body: JSON.stringify(runAgentInput)
});

But in practice, you don’t parse SSE manually.

CopilotKit understands AG-UI natively. You keep the same backend (AGUIStream), and simply point CopilotKit to that endpoint. Proxy AG‑UI traffic through a Next.js API route.

import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { HttpAgent } from "@ag-ui/client";
import { NextRequest } from "next/server";

// You can use any service adapter here for multi-agent support
// Use ExperimentalEmptyAdapter since AG2 handles the LLM directly
const serviceAdapter = new ExperimentalEmptyAdapter();

const runtime = new CopilotRuntime({
  agents: {
    weather_agent: new HttpAgent({
      url: "http://localhost:8000/chat"
    })
  }
});

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

Finally, wrap your app with CopilotKit and render the chat. CopilotKit subscribes to the AG‑UI stream and renders messages, tool progress and state updates in real time.

import { CopilotKit } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui";

<CopilotKit runtimeUrl="/api/copilotkit" agent="weather_agent">
  <CopilotChat />
</CopilotKit>

That's the complete loop. The weather example in the build-with-ag2 repository shows this minimal streaming setup.

Multi-Agent Orchestration

The current AG-UI integration wraps a single ConversableAgent. Multi-agent workflows (group chats, delegation, nested conversations) still run inside AG2, but they are exposed through one interaction surface.

To model staged or multi-agent UX, use an orchestrator agent with ContextVariables. It is a shared state that agents use to coordinate, track progress, and make decisions across a conversation.

When a tool updates context_variables, AGUIStream automatically emits a STATE_SNAPSHOT event.

The frontend can consume that snapshot to:

Highlight which agent is active
Show pipeline progress
Render stage-specific UI

Here's the pattern:

from autogen import ConversableAgent
from autogen.ag_ui import AGUIStream

def submit_plan(plan: str, context): # context injected automatically
    """Submit the plan and move to draft stage."""
    context["active_agent"] = "drafter"
    context["stage"] = "drafting"
    context["plan"] = plan
    return {"status": "plan submitted"}

def submit_draft(draft: str, context):
    """Submit the draft and move to review stage."""
    context["active_agent"] = "reviewer"
    context["stage"] = "reviewing"
    context["draft"] = draft
    return {"status": "draft submitted"}

def submit_review(approved: bool, context):
    """Submit review and finalize."""
    context["stage"] = "complete" if approved else "revising"
    return {"status": "review complete"}

orchestrator = ConversableAgent(
    name="orchestrator",
    system_message="You coordinate a multi-stage workflow.",
    llm_config=llm_config,
    functions=[submit_plan, submit_draft, submit_review],
)

stream = AGUIStream(orchestrator)

Each tool call that updates context triggers a STATE_SNAPSHOT. The frontend reads active_agent and stage from the snapshot and updates the UI accordingly.

The factory example shows this pattern with a 3-panel UI: agent pipeline, document preview and conversation log.

Native multi-agent support is actively being developed. This orchestrator pattern is the current workaround.

What This Enables for Agent Builders

With this integration, you can keep AG2 where it’s strong and still ship real UIs without protocol glue. You can:

✅ Build AG2 agent teams and expose them as AG‑UI endpoints with one wrapper (AGUIStream).

✅ Connect those endpoints to CopilotKit (or any AG‑UI client) without custom WebSocket or polling infrastructure.

✅ Reuse AG2 agent logic across products while keeping a single frontend contract (AG-UI) instead of inventing new payloads for every application

✅ Stream agent runs, tool progress and state in real time using CopilotKit components, so users see what the system is doing instead of waiting on a single final response.

Getting Started

You can start building with this integration today.

👉 Explore the AG2 documentation

👉 Check out the CopilotKit / AG2 docs to get started

👉 Check out our example repo for a minimal integration setup

👉 Explore the build-with-ag2 AG‑UI examples: weather/ directory shows a single-agent chat with tools, factory/ directory demonstrates a multi-stage pipeline using ContextVariables

👉 Try the AG2 demo in the AG-UI Dojo playground

What's Next

This launch makes AG2 a first‑class backend for AG‑UI‑compatible frontends.

On the CopilotKit side, the focus is on richer AG‑UI rendering components, better devtools for inspecting AG‑UI streams and more starter projects that bundle AG2 + CopilotKit out of the box.

Native multi-agent support in AG‑UI is an active focus. The AG2 and CopilotKit teams are working together to bring first-class multi-agent patterns to the protocol so you won't need the orchestrator workaround.

Over the next few weeks, expect more reference implementations, clearer integration patterns and production-grade examples that show how these layers work together in real applications.

How to Build a Research Assistant using Deep Agents

Anmol Baranwal — Fri, 20 Feb 2026 14:22:05 +0000

LangChain's Deep Agents provide a new way to build structured, multi-agent systems that can plan, delegate and reason across multiple steps.

It comes with planning, a filesystem for context and subagent spawning built in. But connecting that agent to a live frontend and actually showing what’s happening behind the scenes in real time is still surprisingly hard.

Today, we will build a Deep Agents powered research assistant using Tavily and connect it to a live Next.js UI with CopilotKit, so every step the agent takes streams to the frontend in real time.

You will find architecture, the key patterns, how state flows between the UI ↔ agent and the step-by-step guide to building this from scratch.

Let's build it.

What is covered?

In summary, we are covering these topics in detail.

What are Deep Agents?
Core Components
What are we building?
Building Frontend
Building Backend (FastAPI + Deep Agents + AG-UI)
Running Application
Data flow (frontend ↔ Agent)

Here is the GitHub Repository, deployed link and official docs if you want to explore yourself.

1. What are Deep Agents?

Most agents today are just “LLM in a loop + tools”. That works but it tends to be shallow: no explicit plan, weak long-horizon execution and messy state as runs get longer.

Popular agents like Claude Code, Deep Research and Manus get around this by following a common pattern: they plan first, externalize working context (often via files or a shell) and delegate isolated pieces of work to sub-agents.

Deep Agents package those primitives into a reusable agent runtime.

Instead of designing your own agent loop from scratch, you call create_deep_agent(...) and get a pre-wired execution graph that already knows how to plan, delegate and manage state across many steps.

Credit: LangChain

At a practical level, a Deep Agent created via create_deep_agent is just a LangGraph graph. There’s no separate runtime or hidden orchestration layer.

The "context management" in deep agents is also very practical -- they offload large tool payloads to the filesystem and only fall back to summarization when token usage approaches the model’s context window. You can read more on Context Management for Deep Agents blog by LangChain.

The mental model (how it runs)

Conceptually, the execution flow looks like this:

User goal
  ↓
Deep Agent (LangGraph StateGraph)
  ├─ Plan: write_todos → updates "todos" in state
  ├─ Delegate: task(...) → runs a subagent with its own tool loop
  ├─ Context: ls/read_file/write_file/edit_file → persists working notes/artifacts
  ↓
Final answer

That gives you a usable structure for “plan → do work → store intermediate artifacts → continue” without inventing your own plan format, memory layer or delegation protocol.

You can check official docs.

Where CopilotKit Fits

Deep Agents push key parts into explicit state (e.g. todos + files + messages), which makes runs easier to inspect. That explicit state is also what makes Copilotkit integration possible.

CopilotKit is a frontend runtime that keeps UI state in sync with agent execution by streaming agent events and state updates in real time (using AG-UI under the hood).

This middleware (CopilotKitMiddleware) is what allows the frontend to stay in lock-step with the agent as it runs. You can read the docs at docs.copilotkit.ai/langgraph/deep-agents.

agent = create_deep_agent(
    model="openai:gpt-4o",
    tools=[get_weather],
    middleware=[CopilotKitMiddleware()], # for frontend tools and context
    system_prompt="You are a helpful research assistant."
)

2. Core Components

Here are the core components that we will be using later on:

1) Planning Tools (built-in via Deep Agents) - built-in planning/to‑do behavior so the agent can break the workflow into steps without you writing a separate planning tool.

# Conceptual example (not required in codebase)
@tool
def todo_write(tasks: List[str]) -> str:
    formatted = "\n".join([f"- {task}" for task in tasks])
    return f"Todo list created:\n{formatted}"

2) Subagents - let the main agent delegate focused tasks into isolated execution loops. Each sub-agent has its own prompt, tools and context.

subagents = [
    {
        "name": "job-search-agent",
        "description": "Finds relevant jobs and outputs structured job candidates.",
        "system_prompt": JOB_SEARCH_PROMPT,
        "tools": [internet_search],
    }
]

3) Tools - this is how the agent actually does things. Here, finalize() signals completion.

@tool
def finalize() -> dict:
    """Signal that the agent is done."""
    return {"status": "done"}

How Deep Agents are implemented (Middleware)

If you are wondering how create_deep_agent() actually injects planning, files and subagents into a normal LangGraph agent, the answer is middleware.

Each feature is implemented as a separate middleware. By default, three are attached:

To-do list middleware - adds the write_todos tool and instructions that push the agent to explicitly plan and update a live todo list during multi-step tasks.
Filesystem middleware - adds file tools (ls, read_file, write_file, edit_file) so the agent can externalize notes and artifacts instead of stuffing everything into chat history.
Subagent middleware - adds the task tool, allowing the main agent to delegate work to subagents with isolated context and their own prompts/tools.

This is what makes Deep Agents feel “pre-wired” without introducing a new runtime. If you want to go deeper, the middleware docs show the exact implementation details.

3. What are we building?

Let's create an agent that:

Accepts a research question from the user
Uses Deep Agents to plan multi-step and orchestrate sub-agents
Searches the web using Tavily
Writes intermediate research artifacts using the filesystem middleware
Streams tool results back to the UI via CopilotKit (AG-UI)

The interface is a two-panel app where the left side is a CopilotKit chat UI and the right side is a live workspace that shows the agent’s plan, generated files and sources as the agent works.

Here's a simplified call request → response flow of what will happen:

[User asks research question]
        ↓
Next.js Frontend (CopilotChat + Workspace)
        ↓
CopilotKit Runtime → LangGraphHttpAgent
        ↓
Python Backend (FastAPI + AG-UI)
        ↓
Deep Agent (research_assistant)
    ├── write_todos        (planning, built-in)
    ├── write_file         (filesystem, built-in)
    ├── read_file          (filesystem, built-in)
    └── research(query)
            └── internal Deep Agent [thread-isolated]
                    └── internet_search (Tavily)

We will see the concepts in action as we build the agent.

4. Frontend: wiring the agent to the UI

Let's first build the frontend part. This is how our directory will look.

The src directory hosts the Next.js frontend, including the UI, shared components and the CopilotKit API route (/api/copilotkit) used for agent communication.

.
├── src/                             ← Next.js frontend
│   ├── app/
│   │   ├── page.tsx          
│   │   ├── layout.tsx               ← CopilotKit provider
│   │   └── api/
│   │       └── copilotkit/route.ts  ← CopilotKit AG-UI runtime
│   ├── components/
│   │   ├── FileViewerModal.tsx      ← Markdown file viewer
│   │   ├── WorkSpace.tsx            ← Research progress display
│   │   └── ToolCard.tsx             ← Tool call visualizer
├── lib/
│   └── types.ts
├── package.json                     
├── next.config.ts                   
└── README.md

If you don’t have a frontend, you can create a new Next.js app with TypeScript.

// creates a nextjs app  
npx create-next-app@latest .

Step 1: CopilotKit Provider & Layout

Install the necessary CopilotKit packages.

npm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

@copilotkit/react-core provides the core React hooks and context that connect your UI to an AG-UI compatible agent backend.
@copilotkit/react-ui offers ready-made UI components like <CopilotChat /> to build AI chat or assistant interfaces quickly.
@copilotkit/runtime is the server-side runtime that exposes an API endpoint and bridges the frontend with an AG-UI compatible backend (e.g., a LangGraph HTTP agent).

The <CopilotKit> component must wrap the Copilot-aware parts of your application. In most cases, it's best to place it around the entire app, like in layout.tsx.

import type { Metadata } from "next";

import { CopilotKit } from "@copilotkit/react-core";
import "./globals.css";
import "@copilotkit/react-ui/styles.css";

export const metadata: Metadata = {
  title: "Deep Research Assistant | CopilotKit Deep Agents Demo",
  description: "A research assistant powered by Deep Agents and CopilotKit - demonstrating planning, memory, subagents, and generative UI",
};

export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  return (
    <html lang="en">
      <body className="antialiased">
        <CopilotKit runtimeUrl="/api/copilotkit" agent="research_assistant">
          {children}
        </CopilotKit>
      </body>
    </html>
  );
}

Here, runtimeUrl="/api/copilotkit" points to the Next.js API route CopilotKit uses to talk to the agent backend.

Each page is wrapped in this context so UI components know which agent to invoke and where to send requests.

Step 2: Next.js API Route: Proxy to FastAPI

This Next.js API route acts as a thin proxy between the browser and the Deep Agents. It:

Accepts CopilotKit requests from the UI
Forwards them to the agent over AG-UI
Streams agent state and events back to the frontend

Instead of letting the frontend talk to the FastAPI agent directly, all requests go through a single endpoint /api/copilotkit.

import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { LangGraphHttpAgent } from "@copilotkit/runtime/langgraph";
import { NextRequest } from "next/server";

// Empty adapter since the LLM is handled by the remote agent
const serviceAdapter = new ExperimentalEmptyAdapter();

// Configure CopilotKit runtime with the Deep Agents backend
const runtime = new CopilotRuntime({
  agents: {
    research_assistant: new LangGraphHttpAgent({
      url: process.env.LANGGRAPH_DEPLOYMENT_URL || "http://localhost:8123",
    }),
  },
});

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

Here's a simple explanation of the above code:

The code above registers the research_assistant agent.
LangGraphHttpAgent : defines a remote LangGraph agent endpoint. It points to the Deep Agents backend running on FastAPI.
ExperimentalEmptyAdapter : simple no-op adapter used when the agent backend handles its own LLM calls and orchestration
copilotRuntimeNextJSAppRouterEndpoint : small helper that adapts the Copilot runtime to a Next.js App Router API route and returns a handleRequest function

Step 3: Types (Research State)

Before building components, let's define shared state for todos, files, and sources in lib/types/research.ts. These are the contracts between the tool results from the agent and the local React state.

// Uses local state + useDefaultTool instead of CoAgent (avoids type issues with Python FilesystemMiddleware)

export interface Todo {
  id: string;
  content: string;
  status: "pending" | "in_progress" | "completed";
}

export interface ResearchFile {
  path: string;
  content: string;
  createdAt: string;
}

// Sources found via internet_search (includes content)
export interface Source {
  url: string;
  title: string;
  content?: string;
  status: "found" | "scraped" | "failed";
}

export interface ResearchState {
  todos: Todo[];
  files: ResearchFile[];
  sources: Source[];
}

export const INITIAL_STATE: ResearchState = {
  todos: [],
  files: [],
  sources: [],
};

Instead of dumping raw tool JSON into chat, each tool result routes into a dedicated state slot - write_todos updates todos, write_file appends to files and research appends to sources. This becomes the foundation of the Workspace panel.

Step 4: Building Key Components

I'm only covering the core logic behind each component since the overall code is huge. You can find all the components in the repository at src/components.

✅ ToolCard Component

This is the client component that renders every tool call inline inside chat. It has two modes:

SpecializedToolCard for known tools (write_todos, research, write_file, read_file) with icons, status indicators and result previews
DefaultToolCard for unknown tools that fall back to expandable JSON.

"use client";

import { useState } from "react";
import { Pencil, ClipboardList, Search, Save, BookOpen, Check, ChevronDown } from "lucide-react";

const TOOL_CONFIG = {
  write_todos: {
    icon: Pencil,
    getDisplayText: () => "Updating research plan...",
    getResultSummary: (result, args) => {
      const todos = (args as { todos?: unknown[] })?.todos;
      if (Array.isArray(todos)) {
        return `${todos.length} todo${todos.length !== 1 ? "s" : ""} updated`;
      }
      return null;
    },
  },
  research: {
    icon: Search,
    getDisplayText: (args) =>
      `Researching: ${((args.query as string) || "...").slice(0, 50)}${(args.query as string)?.length > 50 ? "..." : ""}`,
    getResultSummary: (result) => {
      if (result && typeof result === "object" && "sources" in result) {
        const { sources } = result as { summary: string; sources: unknown[] };
        return `Found ${sources.length} source${sources.length !== 1 ? "s" : ""}`;
      }
      return "Research complete";
    },
  },
  write_file: {
    icon: Save,
    getDisplayText: (args) => {
      const path = args.path as string | undefined;
      const filename =
        path?.split("/").pop() || (args.filename as string | undefined);
      return `Writing: ${filename || "file"}`;
    },
    getResultSummary: (_result, args) => {
      const content = args.content as string | undefined;
      if (content) {
        const firstLine = content.split("\n")[0].slice(0, 50);
        return firstLine + (content.length > 50 ? "..." : "");
      }
      return "File written";
    },
  },
  // read_todos, read_file follow the same pattern
};

export function ToolCard({ name, status, args, result }: ToolCardProps) {
  const config = TOOL_CONFIG[name];
  if (config) {
    return (
      <SpecializedToolCard
        name={name}
        status={status}
        args={args}
        result={result}
        config={config}
      />
    );
  }
  return (
    <DefaultToolCard name={name} status={status} args={args} result={result} />
  );
}

Here's a brief explanation:

research and write_todos are expandable, clicking reveals the full query + findings or the live todo checklist with pending / inprogress / completed states.
resultSummary appears as a small green line below the display text so you can glance at the output without expanding.
DefaultToolCard handles any tool not in TOOL_CONFIG, showing collapsible args and result JSON.

In this component, ExpandedDetails handles the per-tool expanded view. research and write_todos get structured layouts; everything else falls back to a JSON pre block.

function ExpandedDetails({ name, result, args }) {
  if (name === "research") {
    const summary = typeof result === "object" && result && "summary" in result
      ? (result as any).summary
      : "";
    return (
      <div>
        <p>Query: {args.query as string}</p>
        <p>{summary}</p>
      </div>
    );
  }

  if (name === "write_todos") {
    const todos = (args as any)?.todos;
    return (
      <div>
        {todos?.map((todo: any, i: number) => (
          <div key={todo.id || i}>
            <span>{todo.status === "completed" ? "✓" : todo.status === "inprogress" ? "●" : "○"}</span>
            <span>{todo.content}</span>
          </div>
        ))}
      </div>
    );
  }

  // Fallback
  return (
    <pre>
      {typeof result === "string" ? result : JSON.stringify(result, null, 2)}
    </pre>
  );
}

Check out the complete code at src/components/ToolCard.tsx.

✅ Workspace Component

This is the right-side panel that displays research progress as it happens. It has three collapsible sections: Research Plan, Files and Sources - each with a live badge count and an empty state when nothing has arrived yet.

export function Workspace({ state }: { state: ResearchState }) {
  const { todos, files, sources } = state;
  const fileCount = files.length;
  const todoCount = todos.length;
  const sourceCount = sources.length;
  
  // State for file viewer modal
  const [selectedFile, setSelectedFile] = useState<ResearchFile | null>(null);

  return (
    <div className="workspace-panel p-6">
      <div className="mb-6">
        <h2 className="text-xl font-bold">Workspace</h2>
        <p className="text-sm">Research progress and artifacts</p>
      </div>
      <Section title="Research Plan" icon={ListTodo} badge={todos.length}>
        <TodoList todos={todos} />
      </Section>
      <Section title="Files" icon={FileText} badge={files.length}>
        <FileList files={files} onFileClick={setSelectedFile} />
      </Section>
      <Section title="Sources" icon={Globe} badge={sources.length}>
        <SourceList sources={sources} />
      </Section>
      <FileViewerModal
        file={selectedFile}
        onClose={() => setSelectedFile(null)}
      />
    </div>
  );
}

function TodoList({ todos }: { todos: Todo[] }) {
  if (todos.length === 0) return <div>...</div>; // empty state

  return (
    <div className="space-y-1">
      {todos.map((todo) => (
        <div
          key={todo.id}
          className={`todo-item ${
            todo.status === "completed"
              ? "todo-item-completed"
              : todo.status === "in_progress"
                ? "todo-item-inprogress"
                : "todo-item-pending"
          }`}
        >
          <span>{/* Check / CircleDot / Circle icon based on status */}</span>
          <span className="text-sm">{todo.content}</span>
        </div>
      ))}
    </div>
  );
}

// FileList — same pattern, items are clickable (onFileClick)
// each row has a download button with e.stopPropagation()
function FileList({
  files,
  onFileClick,
}: {
  files: ResearchFile[];
  onFileClick: (file: ResearchFile) => void;
}) {
  if (files.length === 0) return <div>...</div>; // empty state

  return (
    <div className="space-y-2">
      {files.map((file, i) => (
        <div
          key={`${file.path}-${i}`}
          className="file-item"
          onClick={() => onFileClick(file)}
        >
          <div className="flex items-center gap-3">
            {/* FileText icon */}
            <div>
              <p className="text-sm font-medium">
                {file.path.split("/").pop()}
              </p>
              <p className="text-xs">{file.path}</p>
            </div>
          </div>
          <button
            onClick={(e) => {
              e.stopPropagation();
              downloadFile(file);
            }}
          >
            {/* Download icon */}
          </button>
        </div>
      ))}
    </div>
  );
}

// SourceList — same pattern, colour-codes by source.status 
// (scraped → green check, failed → red X, found → grey circle) 
// each source links to source.url
function SourceList({ sources }: { sources: Source[] }) {
  if (sources.length === 0) return <div>...</div>; // empty state

  return (
    <div className="space-y-2">
      {sources.map((source, i) => (
        <div
          key={`${source.url}-${i}`}
          className={`file-item ${source.status === "failed" ? "source-failed" : ""}`}
        >
          <span>{/* Check / X / Circle based on source.status */}</span>
          <div className="flex-1 min-w-0">
            <p className="text-sm font-medium truncate">
              {source.title || new URL(source.url).hostname}
            </p>
            <a
              href={source.url}
              target="_blank"
              rel="noopener noreferrer"
              className="text-xs truncate block"
            >
              {source.url}
            </a>
          </div>
        </div>
      ))}
    </div>
  );
}

A brief explanation:

TodoList renders each todo with a pending / inprogress / completed icon. Completed items get a strikethrough.
FileList items are clickable: they open FileViewerModal with full Markdown rendering and a download button.
SourceList colour-codes sources: green check for scraped, red X for failed, grey circle for found-but-not-yet-scraped.

Check out the complete code at src/components/WorkSpace.tsx.

✅ FileViewerModal Component

This component renders a modal that displays the contents of a research file written by the Deep Agent. It renders file content as formatted Markdown using the react-markdown package.

Install it using the following command.

npm install react-markdown

Here is the full core implementation:

export function FileViewerModal({ file, onClose }: FileViewerModalProps) {
  const handleKeyDown = useCallback(
    (e: KeyboardEvent) => { if (e.key === "Escape") onClose(); },
    [onClose]
  );

  useEffect(() => {
    if (file) {
      document.addEventListener("keydown", handleKeyDown);
      document.body.style.overflow = "hidden";
    }
    return () => {
      document.removeEventListener("keydown", handleKeyDown);
      document.body.style.overflow = "";
    };
  }, [file, handleKeyDown]);

  if (!file) return null;

  const filename = file.path.split("/").pop() || file.path;

  const handleDownload = () => {
    const blob = new Blob([file.content], { type: "text/markdown" });
    const url = URL.createObjectURL(blob);
    const a = document.createElement("a");
    a.href = url;
    a.download = filename;
    document.body.appendChild(a);
    a.click();
    document.body.removeChild(a);
    URL.revokeObjectURL(url);
  };

  return (
    <div className="fixed inset-0 z-50 flex items-center justify-center p-4">
      <div className="absolute inset-0 bg-black/30 backdrop-blur-sm" onClick={onClose} aria-hidden="true" />
      <div className="relative max-w-3xl w-full max-h-[85vh] flex flex-col" role="dialog" aria-modal="true">
        {/* Header -- filename + download + close */}
        <div className="flex items-center justify-between border-b ...">
          <div className="flex items-center gap-3">
            {/* FileText icon */}
            <h2 className="truncate max-w-md">{filename}</h2>
          </div>
          <div className="flex items-center gap-2">
            <button onClick={handleDownload}>...</button>
            <button onClick={onClose}>...</button>
          </div>
        </div>

        {/* Scrollable Markdown body */}
        <div className="flex-1 overflow-y-auto p-8">
          <div className="prose prose-sm prose-slate max-w-none">
            <ReactMarkdown>{file.content}</ReactMarkdown>
          </div>
        </div>

        {/* Footer -- full file path */}
        <div className="border-t ...">
          <code>{file.path}</code>
        </div>
      </div>
    </div>
  );
}

Here's a brief explanation:

useEffect registers the Escape listener and locks body scroll only when a file is open, and cleans both up on close.
handleDownload creates a temporary anchor element to trigger a browser download without any server round-trip.

Check out the complete code at src/components/FileViewerModal.tsx.

Step 5: Connecting the Chat UI to the Agent

At this point, all the pieces are in place. This page owns ResearchState, passes it to Workspace, uses CopilotChat component for the conversational UI, runs useDefaultTool to intercept every tool call and routes results into state.

research → appends to sources
write_todos → replaces todos
write_file → appends to files

Here's the code.

"use client";

import { useState, useRef } from "react";
import { CopilotChat } from "@copilotkit/react-ui";
import { useDefaultTool } from "@copilotkit/react-core";
import { Workspace } from "@/components/Workspace";
import { ResearchState, INITIAL_STATE, Todo } from "@/types/research";
import { ToolCard } from "@/components/ToolCard";

export default function Page() {
  const [state, setState] = useState<ResearchState>(INITIAL_STATE);
  const processedKeysRef = useRef<Set<string>>(new Set());

  useDefaultTool({
    render: (props) => {
      const { name, status, args, result } = props;

      // Deduplicate on re-renders
      if (status === "complete") {
        const resultStr = result ? JSON.stringify(result) : "";
        const resultHash = resultStr
          ? `${resultStr.length}-${resultStr.slice(0, 100)}`
          : "";
        const key = `${name}-${JSON.stringify(args)}-${resultHash}`;
        if (processedKeysRef.current.has(key)) return <ToolCard {...props} />;
        processedKeysRef.current.add(key);
      }

      if (name === "research" && status === "complete" && result) {
        const researchResult = result as {
          summary: string;
          sources: Array<{
            url: string;
            title: string;
            content?: string;
            status: "found" | "scraped" | "failed";
          }>;
        };
        if (researchResult.sources && researchResult.sources.length > 0) {
          queueMicrotask(() =>
            setState((prev) => ({
              ...prev,
              sources: [...prev.sources, ...researchResult.sources],
            })),
          );
        }
      }

      if (name === "write_todos" && status === "complete" && args?.todos) {
        const todosWithIds = (
          args.todos as Array<{ id?: string; content: string; status: string }>
        ).map((todo, index) => ({
          ...todo,
          id: todo.id || `todo-${Date.now()}-${index}`,
        }));
        queueMicrotask(() =>
          setState((prev) => ({ ...prev, todos: todosWithIds as Todo[] })),
        );
      }

      if (name === "write_file" && status === "complete" && args?.file_path) {
        queueMicrotask(() =>
          setState((prev) => ({
            ...prev,
            files: [
              ...prev.files,
              {
                path: args.file_path as string,
                content: args.content as string,
                createdAt: new Date().toISOString(),
              },
            ],
          })),
        );
      }

      return <ToolCard {...props} />;
    },
  });

  return (
    <div className="relative min-h-screen">
      <main className="relative z-10 h-screen flex overflow-hidden">
        {/* Chat panel -- left side */}
        <div className="w-[38%] h-full border-r flex flex-col">
          {/* Header */}
          <div style={{ flex: 1, minHeight: 0, overflow: "hidden" }}>
            <CopilotChat
              className="h-full"
              labels={{
                title: "Deep Research Assistant",
                initial: "What topic would you like me to research?",
                placeholder: "Ask me to research any topic...",
              }}
            />
          </div>
        </div>

        {/* Workspace panel -- right side */}
        <div className="w-[62%] h-full overflow-hidden">
          <Workspace state={state} />
        </div>
      </main>
    </div>
  );
}

A few things worth noting:

useDefaultTool intercepts every tool call and renders a ToolCard inline inside CopilotChat.
queueMicrotask defers setState so it never fires mid-render.
processedKeysRef deduplicates results since the render callback fires multiple times as status updates stream in.

4. Backend: Building the Agent Service (FastAPI + Deep Agents + Tavily)

We will now build the FastAPI backend that hosts our Deep Agent.

Under the /agent directory lives a FastAPI server that runs the research agent. Here's the project structure of the backend.

agent/
├── main.py           ← FastAPI server + AG-UI
├── agent.py          ← Deep Agents graph
├── tools.py          ← Tavily search tools
├── pyproject.toml    ← Python deps (uv)
├── uv.lock
└── .env

At a high level, the backend:

Builds a Deep Agent graph
Uses Tavily for real-time web search
Wraps research in an isolated sub-agent
Exposes a CopilotKit-compatible AG-UI endpoint
Streams tool calls to the frontend

The backend uses uv for dependency management. Install it if you don't have it:

pip install uv

Initialize a new uv project. This will generate a fresh pyproject.toml.

cd agent
uv init
uv python pin 3.12

Then install the dependencies:

uv add copilotkit deepagents fastapi langchain langchain-openai python-dotenv tavily-python "uvicorn[standard]" ag-ui-langgraph

copilotkit : connects agents to a frontend with streaming, tools and shared state.
deepagents : planning-first agent framework for multi-step execution
fastapi : web framework that exposes the agent API
langchain : agent and tool orchestration layer.
langchain-openai : OpenAI model integration for LangChain.
tavily-python : web search for real-time research
ag-ui-langgraph : AG-UI protocol adapter for LangGraph
uvicorn[standard] : ASGI server to run FastAPI

Now run the following command to generate a uv.lock file pinned with exact versions.

uv sync

Add necessary API Keys

Create a .env file under the agent directory and add your OpenAI API Key and Tavily API Key to the file. I have attached the docs link so it's easy to follow.

OPENAI_API_KEY=sk-proj-...
TAVILY_API_KEY=tvly-dev-...
OPENAI_MODEL=gpt-5.2

OpenAI API Key

Tavily API Key

Step 1: Implement Research Tools

Let's define two tools (tools.py):

internet_search is the low-level Tavily wrapper. It runs a search and returns up to max_results formatted results.
research wraps an internal Deep Agent that runs in a separate thread to prevent its streaming events from leaking to the frontend via LangChain callback propagation.

Here is the code.

import os
from typing import Any
from concurrent.futures import ThreadPoolExecutor
from langchain_core.tools import tool
from langchain_core.messages import HumanMessage
from tavily import TavilyClient


def _do_internet_search(query: str, max_results: int = 5) -> list[dict[str, Any]]:
    tavily_key = os.environ.get("TAVILY_API_KEY")
    if not tavily_key:
        raise RuntimeError("TAVILY_API_KEY not set")

    try:
        client = TavilyClient(api_key=tavily_key)
        results = client.search(
            query=query,
            max_results=max_results,
            include_raw_content=False,
            topic="general",
        )
        return [
            {
                "url": r.get("url", ""),
                "title": r.get("title", ""),
                "content": (r.get("content") or "")[:3000],
            }
            for r in results.get("results", [])
        ]
    except Exception as e:
        return [{"error": str(e)}]


@tool
def internet_search(query: str, max_results: int = 5) -> list[dict[str, Any]]:
    """Search the web and return results with content."""
    return _do_internet_search(query, max_results)


@tool
def research(query: str) -> dict:
    """Research a topic using web search. Returns structured data with sources."""

    from deepagents import create_deep_agent
    from langchain_openai import ChatOpenAI

    def _run_research_isolated():
        search_results = []

        def internet_search_tracked(query: str, max_results: int = 5):
            """Search the web and return results with content."""
            results = _do_internet_search(query, max_results)
            search_results.extend(results)
            return results

        llm = ChatOpenAI(
            model=os.environ.get("OPENAI_MODEL", "gpt-5.2"),
            temperature=0.7,
            api_key=os.environ.get("OPENAI_API_KEY"),
        )

        research_agent = create_deep_agent(
            model=llm,
            system_prompt="""You are a Research Specialist.
Use internet_search to find information. Return a prose summary of findings.
Rules:
- Call internet_search ONCE with a focused query
- Return a brief summary (2-3 sentences) of key findings
- No JSON, no code blocks, just prose""",
            tools=[internet_search_tracked],
        )

        result = research_agent.invoke({"messages": [HumanMessage(content=query)]})

        summary = result["messages"][-1].content
        sources = [
            {
                "url": r["url"],
                "title": r.get("title", ""),
                "content": r.get("content", "")[:3000],
                "status": "found",
            }
            for r in search_results
            if "url" in r and not r.get("error")
        ]

        return {"summary": summary, "sources": sources}

    with ThreadPoolExecutor(max_workers=1) as executor:
        future = executor.submit(_run_research_isolated)
        return future.result()

Here's what's happening:

_do_internet_search caps each result at 3000 chars to stay within token limits and returns an error dict instead of crashing.
internet_search_tracked extends search_results on every call, building a clean source list to return alongside the final summary.
research_agent is constrained to call internet_search once -- prevents looping and keeps latency predictable.
ThreadPoolExecutor isolates the subagent in a separate thread so its stream events don't leak into the frontend chat.

Step 2: Define the Agent Behavior

This is the brain of the system. The agent is a Deep Agents graph that:

Creates a research plan using write_todos
Calls research() for each research question
Writes a final report to /reports/final_report.md

Here is the main system prompt that enforces this workflow:

MAIN_SYSTEM_PROMPT = """You are a Deep Research Assistant, an expert at planning and
executing comprehensive research on any topic.

Hard rules (ALWAYS follow):
- NEVER output raw JSON, data structures, or code blocks in your messages
- Communicate with the user only in natural, readable prose
- When you receive data from research, synthesize it into insights

Your workflow:
1. PLAN: Create a research plan using write_todos with clear, actionable steps
2. RESEARCH: Use research(query) tool to investigate each topic
3. SYNTHESIZE: Write a final report to /reports/final_report.md using write_file

Important guidelines:
- Always start by creating a research plan with write_todos
- Call research() for each distinct research question
- The research tool returns prose summaries of findings
- You write all files - compile findings into a comprehensive report
- Update todos as you complete each step

Example workflow:
1. write_todos(["Research topic A", "Research topic B", "Synthesize findings"])
2. research("Find information about topic A") -> receives prose summary
3. research("Find information about topic B") -> receives prose summary
4. write_file("/reports/final_report.md", "# Research Report\n\n...")

Always maintain a professional, comprehensive research style."""

Here is the core agent graph (build_agent):

import os
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from deepagents import create_deep_agent
from langgraph.checkpoint.memory import MemorySaver
from copilotkit import CopilotKitMiddleware
from tools import research

load_dotenv()

def build_agent():
    api_key = os.environ.get("OPENAI_API_KEY")
    if not api_key:
        raise RuntimeError("Missing OPENAI_API_KEY environment variable")

    tavily_key = os.environ.get("TAVILY_API_KEY")
    if not tavily_key:
        raise RuntimeError("Missing TAVILY_API_KEY environment variable")

    model_name = os.environ.get("OPENAI_MODEL", "gpt-5.2")
    llm = ChatOpenAI(
        model=model_name,
        temperature=0.7,
        api_key=api_key,
    )

    agent_graph = create_deep_agent(
        model=llm,
        system_prompt=MAIN_SYSTEM_PROMPT,
        tools=[research],
        middleware=[CopilotKitMiddleware()],
        checkpointer=MemorySaver(),
    )

    return agent_graph.with_config({"recursion_limit": 100})

Here's what's happening:

CopilotKitMiddleware() enables AG-UI streaming to the frontend
MemorySaver() enables stateful execution
Recursion limit increased to support multi-step research
research is the only explicit tool since write_todos, read_file, write_file are built into Deep Agents automatically

Step 3: Expose the Agent via FastAPI + AG-UI

The last step is to wire everything together and expose it as a FastAPI app. The main.py builds the agent graph, configures which tool calls to emit to the frontend and registers the AG-UI endpoint.

import os
import uvicorn
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from dotenv import load_dotenv
from ag_ui_langgraph import add_langgraph_fastapi_endpoint
from copilotkit import LangGraphAGUIAgent
from copilotkit.langgraph import copilotkit_customize_config

from agent import build_agent

load_dotenv()

app = FastAPI(title="Deep Research Assistant", version="1.0.0")

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

@app.get("/health")
def health():
    return {"status": "ok", "service": "deep-research-agent"}

agent_graph = build_agent()

# Only emit main agent tools — suppress internal subagent tool calls
agui_config = copilotkit_customize_config(
    emit_tool_calls=["research", "write_todos", "write_file", "read_file", "edit_file"]
)
agui_config["recursion_limit"] = 100

add_langgraph_fastapi_endpoint(
    app=app,
    agent=LangGraphAGUIAgent(
        name="research_assistant",
        description="A deep research assistant that plans, searches, and synthesizes research reports",
        graph=agent_graph,
        config=agui_config,
    ),
    path="/",
)

def main():
    uvicorn.run(
        "main:app",
        host=os.getenv("SERVER_HOST", "0.0.0.0"),
        port=int(os.getenv("SERVER_PORT", "8123")),
        reload=True,
    )

if __name__ == "__main__":
    main()

The emit_tool_calls config controls exactly which tool calls CopilotKit streams to the frontend. Without it, the subagent's internal internet_search calls leak through as raw JSON noise in the chat UI.

5. Running the Application

After completing all the parts of the code, it's time to run it locally. Please make sure you have added the credentials to the agent/.env.

From the project root, navigate to the agent directory and start the FastAPI server:

cd agent
uv run python main.py

The backend will start on http://localhost:8123.

In a new terminal window, start the frontend development server using:

npm run dev

Once both servers are running, open the frontend in your browser at http://localhost:3000/ to view it locally.

CopilotKit also provides the Agent Inspector, which is a live AG-UI runtime view that lets you inspect agent runs, state snapshots, messages and tool calls as they stream from the backend. It's accessible from a copilotkit button overlaid on your app.

Here is the complete demo!

6. Data flow

Now that we have built both the frontend and the agent service, this is how data actually flows between them. This should be easy to follow if you have been building along so far.

[User sends research question]
        ↓
Next.js Frontend (CopilotChat + Workspace)
        ↓
POST /api/copilotkit (AG-UI protocol)
        ↓
CopilotKit Runtime → LangGraphHttpAgent (localhost:8123)
        ↓
FastAPI Backend (AG-UI endpoint "/")
        ↓
Deep Agent (research_assistant)
        ↓
Deep Agents orchestration
    ├── write_todos  → todos state   → Workspace (Research Plan)
    ├── research()   → sources state → Workspace (Sources)
    │       └── internal Deep Agent [thread-isolated]
    │               └── internet_search (Tavily API)
    └── write_file   → files state   → Workspace (Files)
        ↓
AG-UI streaming (SSE)
        ↓
useDefaultTool intercepts tool calls
        ↓
ToolCard renders in chat + Workspace updates live

That's it! 🎉

You now have a fully working Deep Agents research assistant that plans, searches, and writes reports, with every step visible in the frontend as it happens.

The real win here is visibility. CopilotKit's AG-UI layer turns what would otherwise be a black-box agent into something users can actually follow and trust.

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

Build a Multi-Agent Telecom Support System with CopilotKit & LangGraph JS

Arindam Majumder — Mon, 09 Feb 2026 14:38:07 +0000

Introduction

Customer support is chaotic, and thinking about ways to automate it can be tricky. You never know what a customer will ask for, or when. To handle this, people usually rely on patterned automated systems that work based on a few conditions. If a request fits those conditions, it works fine. If not, it gets passed to a human.

That’s the problem. Customers don’t follow patterns. A simple question might need escalation, and an urgent issue might have a quick fix.

In this tutorial, we’ll build a multi-agent support system using CopilotKit and LangGraph. In this system, multiple agents will coordinate resolutions based on the customers' questions or issues.

Before we start building, let’s first understand what CopilotKit is and why it fits this problem.

What is CopilotKit?

CopilotKit is an open-source framework for building AI copilots that integrate directly into your applications. It provides UI components and infrastructure to connect backend agents with frontend UIs, enabling real-time context sharing and state synchronization.

Check out CopilotKit's GitHub ⭐️

In this project, it will also use AG-UI (Agentic UI) under the hood, an event-based protocol that lets agents do more than just respond in chat. Agents can trigger actions, update UI state, and interact directly with the application. For example, an agent can modify a service plan, update a customer record, or adjust billing settings through frontend tools.

CopilotKit works with multiple agent frameworks, including LangGraph, and uses AG-UI as a common event layer. If you’re new to CopilotKit, you can explore the documentation to get a better overview before continuing.

What We're Building

We’ll build a telecom support application with a chat interface that lets customers manage their services, update plans, check billing, and get instant help.

Here's what happens when a customer asks something like, "Add international calling to my plan":

The system features four specialized agents:

Intent Agent: Classifies customer messages and determines urgency level
Customer Lookup Agent: Retrieves customer profiles and service details from the database
Reply Agent: Generates personalized responses and processes service changes
Escalation Agent: Routes complex issues to appropriate human support teams with full context

The Tech Stack

At the core, we're using this stack for building the multi-agent support system:

Next.js: Frontend framework with TypeScript
CopilotKit SDK: Embed agents into UI (@copilotkit/react-core, @copilotkit/runtime, @copilotkit/react-ui)
LangGraph (StateGraph): Stateful agent workflows with coordination
OpenAI: LLM for reasoning and response generation
LangChain's OpenAI adapter: Connects OpenAI to LangChain workflows
Turborepo: Monorepo management for web + agent apps
pnpm workspaces: Package management across the monorepo

Architecture Overview

Here's the high-level architecture showing how the frontend, CopilotKit runtime, and LangGraph agents work together:

The Next.js frontend communicates with LangGraph agents through the CopilotKit Runtime API route. When a customer sends a message, it flows through the runtime to the multi-agent system, where specialized agents coordinate to handle the request. State updates stream back to the UI in real-time, so customers see changes as they happen.

Project Structure

This project is organized as a monorepo. It consists of two main applications: the Next.js frontend and the LangGraph multi-agent backend, along with shared configuration files.

support-help/
├── apps/
│   ├── web/                          <- Next.js frontend application
│   │   ├── src/
│   │   │   ├── app/
│   │   │   │   ├── page.tsx          <- Main dashboard UI
│   │   │   │   ├── layout.tsx        <- Root layout with CopilotKit provider
│   │   │   │   ├── globals.css       <- Global styles
│   │   │   │   └── api/
│   │   │   │       └── copilotkit/   <- CopilotKit runtime endpoint
│   │   │   │           └── route.ts  <- API route handler
│   │   │   ├── components/
│   │   │   │   └── WelcomeScreen.tsx  
│   │   │   ├── hooks/
│   │   │   │   └── CustomerContext.tsx <- Customer state & frontend tools
│   │   │   ├── data/
│   │   │   │   └── ticketsData.ts    <- Sample ticket data
│   │   │   └── utils/
│   │   │       └── servicePricing.ts <- Service pricing calculations
│   │   ├── package.json
│   │   └── next.config.ts
│   │
│   └── agent/                        <- LangGraph multi-agent backend
│       ├── src/
│       │   ├── agent.ts              <- Main agent workflow & graph definition
│       │   ├── index.ts              <- Entry point
│       │   ├── agents/               <- Agents
│       │   │   ├── intentAgent.ts   
│       │   │   ├── ...
│       │   ├── tools/                <- Tools
│       │   │   ├── intentClassifier.ts
│       │   │   ├── ...
│       │   ├── types/
│       │   │   └── state.ts 
│       │   └── utils/
│       │       └── dataLoader.ts
│       ├── package.json
│       └── langgraph.json            # LangGraph configuration
│

Here's the GitHub repository if you want to see the project and explore the implementation in more detail.

Add API Key

Create a .env file under the agent directory and add your OpenAI API Key to the file.

OPENAI_API_KEY=<<your-openai-key-here>>

Let's build the frontend first.

Frontend

The frontend is in apps/web and handles the customer dashboard, chat interface, and real-time state management. We’ll connect it to the LangGraph agents using CopilotKit.

If you're starting fresh, create a new Next.js app with TypeScript:

npx create-next-app@latest web

In our cloned repository, everything's already set up, so just install dependencies:

cd apps/web
pnpm install

Step 1: Install CopilotKit Packages

Install the necessary CopilotKit packages:

pnpm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

Here's what each package does:

@copilotkit/react-core - Core hooks and context to connect your React app with the agent backend
@copilotkit/react-ui - Ready-made UI components like <CopilotSidebar /> for chat interfaces
@copilotkit/runtime - Server-side runtime that connects your Next.js API route to LangGraph agents

Step 2: Set Up the Root Layout

The <CopilotKit> provider needs to wrap your app. Open src/app/layout.tsx:

import type { Metadata } from "next";
import { Geist } from "next/font/google";
import "./globals.css";
import { CopilotKit } from "@copilotkit/react-core";
import "@copilotkit/react-ui/styles.css";

// ...
export default function RootLayout({
  children,
}: {
  children: React.ReactNode;
}) {
  return (
    <html lang="en">
      <body className={geist.className}>
        <CopilotKit runtimeUrl="/api/copilotkit">
          {children}
        </CopilotKit>
      </body>
    </html>
  );
}

The runtimeUrl specifies where CopilotKit can find your agent endpoint. It is also the way to securely communicate on the backend. We'll create that next.

Step 3: Create the CopilotKit Runtime Endpoint

Create src/app/api/copilotkit/route.ts. This is the bridge between your frontend and the LangGraph agent:

import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { LangGraphAgent } from "@copilotkit/runtime/langgraph";
import { NextRequest } from "next/server";

// 1. You can use any service adapter here for multi-agent support.
const serviceAdapter = new ExperimentalEmptyAdapter();

// 2. Create the CopilotRuntime instance and utilize the LangGraph AG-UI
//    integration to set up the connection.
const runtime = new CopilotRuntime({
  agents: {
    starterAgent: new LangGraphAgent({
      deploymentUrl: process.env.LANGGRAPH_DEPLOYMENT_URL || "http://localhost:8123",
      graphId: "starterAgent", // <- Graph id will be the agent name that we'll use next in our context file when we'll define the Agent using useAgent hook
      langsmithApiKey: process.env.LANGSMITH_API_KEY || "",
    })
  }
});

// 3. Build a Next.js API route to handle CopilotKit runtime requests.
export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

This API route proxies requests between the frontend and your LangGraph agent, handling authentication and keeping API keys secure on the server.

CopilotRuntime manages agent communication, OpenAIAdapter connects to GPT model, and copilotRuntimeNextJSAppRouterEndpoint processes and routes incoming requests.

Step 4: Create Customer Context with Frontend Tools

Here's the thing about building with CopilotKit: if you want agents to actually modify your UI in real-time, your state management needs to be solid. Scattering state across multiple files or skipping the global state entirely makes it nearly impossible for agents to properly update the UI.

In this system, we're using React Context to centralize customer data and expose it to agents through useAgent and useFrontendTool. This pattern ensures that agents and UI components use the same source of truth and supports bidirectional state synchronization.

Create src/hooks/CustomerContext.tsx. This is where we centralize customer state and expose tools that agents can call to modify the UI.

First, set up the basic structure:

"use client";

import React, { createContext, useContext, useState, useCallback, useRef, useEffect } from "react";
import { useAgent, useFrontendTool } from "@copilotkit/react-core/v2";
import { initialCustomers } from "@/data/ticketsData";

interface Customer {
  id: number;
  customerID: string;
  gender: string;
  SeniorCitizen: string;
  Partner: "Yes" | "No";
  MonthlyCharges: string;
  InternetService: "DSL" | "Fiber optic";
  PaperlessBilling: "Yes" | "No";
  StreamingTV: "Yes" | "No";
  // ... other service fields
}

type AgentState = {
  customers: Customer[];
};

const CustomerContext = createContext<CustomerContextType | undefined>(undefined);

export function CustomerProvider({ children }: { children: React.ReactNode }) {
  // Use useAgent for bidirectional state synchronization with the agent
  const { agent } = useAgent({ agentId: "starterAgent" }) // <- Agent name same as graphId

  // Initialize agent state if not already set
  useEffect(() => {
    if (!agent.state.customers) {
      agent.setState({
        ...agent.state,
        customers: initialCustomers,
      });
    }
  }, [agent]);

  // Optimistic local state with agent sync
  const [localCustomers, setLocalCustomers] = useState<Customer[] | null>(null);

  // Derive customers: local override takes precedence over agent state
  const customers = localCustomers ?? agent.state.customers ?? [];

  // Keep a ref to always have the latest customers in closures
  const customersRef = useRef<Customer[]>([]);
  useEffect(() => {
    customersRef.current = customers;
  }, [customers]);

Next, define helper functions for state updates:

These functions handle the actual logic for modifying customer data, adding or removing services, or updating settings.

const recalculateCharges = useCallback((customer: Customer) => {
    const calculation = calculateMonthlyCharges(customer);
    const monthlyCharges = calculation.total;
    const tenure = parseInt(customer.tenure) || 0;
    const totalCharges = monthlyCharges * tenure;
    return { monthlyCharges, totalCharges };
  }, []);

  // Helper: Add a service addon to a customer
  const addAddon = useCallback((customerId: string, addon: AddonService) => {
    let updatedCustomer: Customer | null = null;

    const updatedCustomers = customersRef.current.map((customer) => {
      if (customer.customerID === customerId) {
        const updates: Partial<Customer> = {};

        // Handle service dependencies (e.g., PhoneService required for MultipleLines)
        if (addon === "MultipleLines" && customer.PhoneService === "No") {
          return customer; // Can't add MultipleLines without PhoneService
        }

        updates[addon] = "Yes";
        const updated = { ...customer, ...updates };

        // Recalculate charges automatically
        const { monthlyCharges, totalCharges } = recalculateCharges(updated);
        updated.MonthlyCharges = monthlyCharges.toFixed(2);
        updated.TotalCharges = totalCharges.toFixed(2);

        updatedCustomer = updated;
        return updated;
      }
      return customer;
    });

    if (updatedCustomer) {
      // Update BOTH local and agent state
      setLocalCustomers(updatedCustomers);
      agent.setState({ ...agent.state, customers: updatedCustomers });
    }

    return updatedCustomer;
  }, [recalculateCharges, agent]);

  // Helper: Update customer settings
  const updateCustomer = useCallback((customerId: number, updates: Partial<Customer>) => {
    let updatedCustomer: Customer | null = null;

    const updatedCustomers = customers.map((customer) => {
      if (customer.id === customerId) {
        const updated = { ...customer, ...updates };

        // Handle service dependencies
        if (updates.PhoneService === "No") {
          updated.MultipleLines = "No phone service";
        }

        // Recalculate charges if any service changed
        const { monthlyCharges, totalCharges } = recalculateCharges(updated);
        updated.MonthlyCharges = monthlyCharges.toFixed(2);
        updated.TotalCharges = totalCharges.toFixed(2);

        updatedCustomer = updated;
        return updated;
      }
      return customer;
    });

    setLocalCustomers(updatedCustomers);
    agent.setState({ ...agent.state, customers: updatedCustomers });
    return updatedCustomer;
  }, [customers, recalculateCharges, agent]);

Define the first frontend tool: Add Addon

Use useFrontendTool to create a tool that agents can call. The handler executes our addAddon function when the agent needs to add a service.

// Frontend Tool: Add service addon
  useFrontendTool({
    name: "addAddonToCustomer",
    description: "Add a service addon to a customer. This will enable a specific service for the customer and recalculate their monthly charges automatically.",
    parameters: [
      {
        name: "customerID",
        type: "string",
        description: "The unique customer ID (e.g., '5575-GNVDE', '7590-VHVEG')",
        required: true,
      },
      {
        name: "addonName",
        type: "string",
        description: "The name of the addon service to add. Valid options: PhoneService, MultipleLines, OnlineSecurity, OnlineBackup, DeviceProtection, TechSupport, StreamingTV, StreamingMovies",
        required: true,
      },
    ],
    handler: async ({ customerID, addonName }) => {
      // Access current state directly from ref to avoid stale closure
      const currentCustomers = customersRef.current;
      const customer = currentCustomers.find((c) => c.customerID === customerID);

      if (!customer) {
        return {
          success: false,
          message: `Customer with ID ${customerID} not found`,
        };
      }

      const result = addAddon(customer.customerID, addonName as AddonService);

      if (!result) {
        return {
          success: false,
          message: `Failed to add ${addonName}. Check if prerequisites are met (e.g., PhoneService required for MultipleLines)`,
        };
      }

      return {
        success: true,
        message: `Successfully added ${addonName} to customer ${customer.customerID}`,
        newMonthlyCharges: result.MonthlyCharges,
        customer: {
          id: result.id,
          customerID: result.customerID,
          monthlyCharges: result.MonthlyCharges,
        },
      };
    },
  });

When an agent calls addAddonToCustomer, the handler finds the customer, calls our addAddon function, and returns confirmation with updated charges.

Define the second frontend tool: Remove Addon

// Frontend Tool: Remove service addon
  useFrontendTool({
    name: "removeAddonFromCustomer",
    description: "Remove a service addon from a customer. This will disable a specific service for the customer and recalculate their monthly charges automatically.",
    parameters: [
      {
        name: "customerID",
        type: "string",
        description: "The unique customer ID (e.g., '5575-GNVDE', '7590-VHVEG')",
        required: true,
      },
      {
        name: "addonName",
        type: "string",
        description: "The name of the addon service to remove. Valid options: PhoneService, MultipleLines, OnlineSecurity, OnlineBackup, DeviceProtection, TechSupport, StreamingTV, StreamingMovies",
        required: true,
      },
    ],
    handler: async ({ customerID, addonName }) => {
      const currentCustomers = customersRef.current;
      const customer = currentCustomers.find((c) => c.customerID === customerID);

      if (!customer) {
        return {
          success: false,
          message: `Customer with ID ${customerID} not found`,
        };
      }

      const result = removeAddon(customer.customerID, addonName as AddonService);

      if (!result) {
        return {
          success: false,
          message: `Failed to remove ${addonName}`,
        };
      }

      return {
        success: true,
        message: `Successfully removed ${addonName} from customer ${customer.customerID}`,
        newMonthlyCharges: result.MonthlyCharges,
      };
    },
  });

The handler removes the addon.

Wrap it up with the provider:

const value: CustomerContextType = {
    customers,
    addCustomer,
    deleteCustomer,
    updateCustomer,
    addAddon,
    removeAddon,
    getCustomerById,
    getCustomerByCustomerId,
    recalculateCharges,
  };

  return (
    <CustomerContext.Provider value={value}>
      {children}
    </CustomerContext.Provider>
  );
}

export function useCustomers() {
  const context = useContext(CustomerContext);
  if (!context) throw new Error("useCustomers must be used within CustomerProvider");
  return context;
}

This pattern keeps the state centralized, helper functions manage the logic, useAgent syncs data bidirectionally with agents through agent.state and agent.setState(), and useFrontendTool creates callable tools that trigger UI updates automatically. The agent can now read customer data and execute actions that are reflected in the UI immediately.

Step 5: Build the Main Dashboard UI

Create src/app/page.tsx:

"use client";

import React, { useState } from "react";
import { CustomerProvider, useCustomers } from "@/hooks/CustomerContext";
import { CopilotSidebar } from "@copilotkit/react-ui";

export default function CustomerSupportPage() {
  return (
    <CustomerProvider>
      <div className="flex h-screen overflow-hidden">
        <MainApp />
        <CopilotSidebar
          defaultOpen={true}
          clickOutsideToClose={false}
          labels={{
            title: "Telecom Support Assistant",
            initial: "Hi! 👋 I'm here to assist you with your telecom support needs.",
          }}
          suggestions={[
            {
              title: "🔍 Check my services",
              message: "Show me services for customer ID: 5575-GNVDE",
            },
            // ... more you can add...
          ]}
        />
      </div>
    </CustomerProvider>
  );
}

function MainApp() {
  const { customers, getCustomerByCustomerId } = useCustomers();
  const [selectedCustomerId, setSelectedCustomerId] = useState<string>("");

  const selectedCustomer = selectedCustomerId
    ? getCustomerByCustomerId(selectedCustomerId)
    : null;

  return (
    <div className="flex-1 flex flex-col bg-gray-50 overflow-hidden">
      {/* Top Header */}
      <header className="bg-white border-b px-6 py-4">
        <div className="flex items-center justify-between">
          <h1 className="text-2xl font-bold">Telecom Support</h1>

          <select
            value={selectedCustomerId}
            onChange={(e) => setSelectedCustomerId(e.target.value)}
            className="px-4 py-2 border rounded-lg"
          >
            <option value="">Select Customer Account</option>
            {customers.map((customer) => (
              <option key={customer.id} value={customer.customerID}>
                {customer.customerID} - {customer.gender}
              </option>
            ))}
          </select>
        </div>
      </header>

      {/* Main Content */}
      <div className="flex-1 overflow-auto p-6">
        {selectedCustomer ? (
          <CustomerCard customer={selectedCustomer} />
        ) : (
          <WelcomeScreen />
        )}
      </div>
    </div>
  );
}

We're using <CopilotSidebar> for the chat interface with starter suggestions to guide users. The MainApp component displays a customer selector and shows account details when a customer is selected.

Step 6: Create the Welcome Screen

Create src/components/WelcomeScreen.tsx:

export function WelcomeScreen() {
  return (
    <div className="max-w-4xl mx-auto">
      <div className="bg-white rounded-2xl shadow-lg p-8 border">
        <h2 className="text-3xl font-bold text-gray-900 mb-4">
          Multi-Agent Telecom Support System
        </h2>
        <p className="text-gray-600 mb-6">
          This system uses 4 specialized AI agents working together to handle customer support:
        </p>

        <div className="grid grid-cols-2 gap-4 mb-8">
          <AgentCard
            icon="🎯"
            title="Intent Agent"
            description="Classifies customer issues and urgency"
          />
          <AgentCard
            icon="🔍"
            title="Lookup Agent"
            description="Finds customer profiles instantly"
          />
          <AgentCard
            icon="💬"
            title="Reply Agent"
            description="Generates personalized responses"
          />
          <AgentCard
            icon="🚨"
            title="Escalation Agent"
            description="Routes complex issues to humans"
          />
        </div>

        <div className="bg-blue-50 border border-blue-200 rounded-lg p-4">
          <p className="text-sm text-blue-900">
            <strong>Get Started:</strong> Select a customer from the dropdown above and ask questions in the chat!
          </p>
        </div>
      </div>
    </div>
  );
}

This screen appears when no customer is selected, providing context on how agents work together.

Step 7: Start the Frontend Server

Now run the frontend:

# cd app/web
pnpm dev

This starts:

Next.js frontend on http://localhost:3000 Open your browser to see the dashboard!

The frontend is ready. Now we need to build the backend agents that actually process customer requests and make decisions.

Backend Agent Service (LangGraph + CopilotKit SDK)

The /apps/agent directory contains the multi-agent backend that handles customer requests and coordinates responses.

The backend uses Node.js and TypeScript with LangGraph. If you're starting fresh, install the dependencies:

cd apps/agent
pnpm install

The key packages you need:

{
  "dependencies": {
    "@copilotkit/sdk-js": "1.10.6",
    "@langchain/core": "^1.0.1",
    "@langchain/langgraph": "1.0.2",
    "@langchain/openai": "^1.1.3",
    "zod": "^3.23.8"
  }
}

Here's what each package does:

@copilotkit/sdk-js - Integrates LangGraph workflows with CopilotKit's state streaming
@langchain/langgraph - State machine framework for building agent workflows
@langchain/core - Core abstractions (messages, runnables, tools)
@langchain/openai - LangChain wrapper for OpenAI models (I used: gpt-5-nano But, any model you can use.)
zod - Schema validation for tool parameters

Define the Agent State

First, create src/types/state.ts. This defines the shared state that flows through all agents:

import { Annotation } from "@langchain/langgraph";
import { BaseMessage } from "@langchain/core/messages";

export const CustomerSupportStateAnnotation = Annotation.Root({
  messages: Annotation<BaseMessage[]>({
    reducer: (x, y) => x.concat(y),
    default: () => [],
  }),

  customers: Annotation<any[]>({
    reducer: (x, y) => y ?? x,
    default: () => [],
  }),

  currentCustomer: Annotation<{
    found: boolean;
    id?: string;
    data?: any;
  }>({
    reducer: (x, y) => ({ ...x, ...y }),
    default: () => ({ found: false }),
  }),

  intent: Annotation<{
    category: string;
    urgency: "low" | "medium" | "high";
    confidence: number;
    keywords: string[];
  } | null>({
    reducer: (x, y) => y ?? x,
    default: () => null,
  }),

  escalation: Annotation<{
    required: boolean;
    reason?: string;
    ticketId?: string;
    assignedTo?: string;
    priority?: number;
    suggestAiFirst?: boolean;
  } | null>({
    reducer: (x, y) => y ?? x,
    default: () => null,
  }),
});

export type CustomerSupportState = typeof CustomerSupportStateAnnotation.State;

The reducer functions control how the state merges when agents update it, messages concatenates new entries to history, while intent and escalation replace their previous values with the latest results.

Build the Agent Tools

Agents need tools to perform specific actions. These are functions that agents can call to classify intent, look up customer data, or make escalation decisions. Let's build them.

Tool 1. Intent Classifier Tool
Create src/tools/intentClassifier.ts:

import { tool } from "@langchain/core/tools";
import { z } from "zod";
import { ChatOpenAI } from "@langchain/openai";

export const intentClassifierTool = tool(
  async ({ message }) => {
    const model = new ChatOpenAI({
      model: "gpt-5-nano",
    });

    const prompt = `You are an intent classifier for telecom customer support.

Analyze this message and classify the intent:
"${message}"

Return JSON with:
{
  "category": "billing_issue" | "service_outage" | "tech_support" | "cancellation" | "general_inquiry" | "upgrade_service",
  "urgency": "low" | "medium" | "high",
  "confidence": 0.0 to 1.0,
  "keywords": ["keyword1", "keyword2"]
}`;

    const response = await model.invoke([{ role: "user", content: prompt }]);
    return response.content;
  },
  {
    name: "classifyIntent",
    description: "Classifies customer message intent and urgency level",
    schema: z.object({
      message: z.string().describe("The customer's message to classify"),
    }),
  }
);

This tool takes a customer message and returns the intent category, urgency level, and confidence score.

Tool 2: Customer Lookup Tool
Create src/tools/customerLookup.ts:

import { tool } from "@langchain/core/tools";
import { z } from "zod";
import { loadCustomerData } from "../utils/dataLoader";

export const customerLookupTool = tool(
  async ({ customerId }) => {
    const customers = await loadCustomerData();
    const customer = customers.find(
      (c) => c.customerID === customerId
    );

    if (!customer) {
      return JSON.stringify({
        found: false,
        message: `Customer ${customerId} not found`,
      });
    }

    return JSON.stringify({
      found: true,
      id: customer.customerID,
      data: {
        customerID: customer.customerID,
        gender: customer.gender,
        tenure: customer.tenure,
        MonthlyCharges: customer.MonthlyCharges,
        InternetService: customer.InternetService,
        PhoneService: customer.PhoneService,
        Churn: customer.Churn,
        SeniorCitizen: customer.SeniorCitizen,
      },
    });
  },
  {
    name: "lookupCustomer",
    description: "Retrieve customer profile and service details by customer ID",
    schema: z.object({
      customerId: z.string().describe("Customer ID (e.g., '5575-GNVDE')"),
    }),
  }
);

This tool retrieves a customer's profile and service details by ID, returning account information such as tenure, charges, and active services.

Tool 3: Escalation Decision Tool
Create src/tools/escalationDecision.ts:

import { tool } from "@langchain/core/tools";
import { z } from "zod";
import { ChatOpenAI } from "@langchain/openai";
import { findCustomerById } from "../utils/dataLoader";

export const escalationDecisionTool = tool(
  async ({ customerId, intent, urgency }) => {
    const customer = customerId ? findCustomerById(customerId) : null;

    const model = new ChatOpenAI({
      model: "gpt-5-nano"
    });

    const customerContext = customer
      ? `Customer Profile:
- Churn Risk: ${customer.Churn}
- Tenure: ${customer.tenure} months
- Monthly Charges: $${customer.MonthlyCharges}
- Senior Citizen: ${customer.SeniorCitizen}`
      : "No customer profile available";

    const systemPrompt = `You are an escalation decision expert for telecom support.

${customerContext}

CURRENT ISSUE:
- Intent: ${intent}
- Urgency: ${urgency}

ESCALATION RULES:
1. ALWAYS ESCALATE if:
   - Churn Risk = "Yes" (customer might cancel)
   - Intent = "cancellation" (need retention specialist)
   - Urgency = "high" (critical issues)
   - Senior Citizen = "Yes" (need extra care)
   - High-value customer (tenure > 50 months OR charges > $80)

2. SUGGEST AI FIRST if:
   - Urgency = "high" BUT Churn Risk = "No"
   - Intent = "service_outage", "tech_support", "internet_issue"
   - Return "suggestAiFirst": true to offer AI suggestions before escalating

3. DEPARTMENT ASSIGNMENT:
   - billing → "billing" team
   - cancellation → "retention" team (PRIORITY 1)
   - service_outage, tech_support → "tech" team

Return JSON:
{
  "required": true/false,
  "reason": "detailed reason",
  "assignedTo": "billing" | "tech" | "retention",
  "priority": 1 | 2 | 3,
  "suggestAiFirst": true/false
}`;

    const response = await model.invoke([
      { role: "system", content: systemPrompt },
      { role: "user", content: "Analyze if escalation is needed." },
    ]);

    return response.content;
  },
  {
    name: "checkEscalation",
    description: "Determine if issue should be escalated to human agent",
    schema: z.object({
      customerId: z.string().optional(),
      intent: z.string().describe("The classified intent category"),
      urgency: z.enum(["low", "medium", "high"]),
    }),
  }
);

This tool determines whether an issue requires human escalation based on factors such as churn risk, urgency, and customer profile characteristics, including senior citizen status or high monthly charges.

Tool 4: Reply Generator Tool
Create src/tools/replyGenerator.ts:

import { tool } from "@langchain/core/tools";
import { z } from "zod";
import { ChatOpenAI } from "@langchain/openai";

export const replyGeneratorTool = tool(
  async ({ customerData, intent, message }) => {
    const model = new ChatOpenAI({
      model: "gpt-5-nano",
    });

    const prompt = `You are a helpful telecom support assistant.

Customer: ${customerData}
Intent: ${intent}
Message: "${message}"

Generate a personalized, friendly response addressing their concern.`;

    const response = await model.invoke([{ role: "user", content: prompt }]);
    return response.content;
  },
  {
    name: "generateReply",
    description: "Generate personalized response based on customer context",
    schema: z.object({
      customerData: z.string().describe("Customer profile information"),
      intent: z.string().describe("Classified intent category"),
      message: z.string().describe("Customer's original message"),
    }),
  }
);

This tool generates personalized responses to customer queries based on their profile, intent, and conversation history.

With all tools ready, we can now connect the agents into a workflow that processes customer requests step by step.

Build the Multi-Agent Workflow

With tools and state defined, we need to orchestrate how agents work together. LangGraph uses a workflow model where you define the execution sequence, which agent runs first, what happens next, and how data flows between them.

Now, let's wire everything together in src/agent.ts:

import { StateGraph, START, END, MemorySaver } from "@langchain/langgraph";
import { CustomerSupportStateAnnotation } from "./types/state";
// We'll create these agent nodes in the next section
import { intentAgentNode } from "./agents/intentAgent";
import { customerLookupAgentNode } from "./agents/customerLookupAgent";
import { escalationAgentNode } from "./agents/escalationAgent";
import { replyAgentNode } from "./agents/replyAgent";

// Create the workflow
const workflow = new StateGraph(CustomerSupportStateAnnotation);

// Add agent nodes(we'll build these next)
workflow.addNode("intent", intentAgentNode);
workflow.addNode("lookup", customerLookupAgentNode);
workflow.addNode("escalation", escalationAgentNode);
workflow.addNode("reply", replyAgentNode);

// Define the flow
workflow.addEdge(START, "intent");
workflow.addEdge("intent", "lookup");
workflow.addEdge("lookup", "escalation");
workflow.addEdge("escalation", "reply");
workflow.addEdge("reply", END);

// Compile with memory
const memory = new MemorySaver();
export const graph = workflow.compile({
  checkpointer: memory,
});

StateGraph creates a workflow where nodes (your agents) are connected by edges that define execution order. Each edge routes one agent's output to the next, forming a pipeline: Intent → Lookup → Escalation → Reply.

The MemorySaver acts as a checkpointer, saving conversation state after each step so the system remembers previous messages across multiple turns.

Build the Agent Nodes

We've defined the workflow structure, but each node needs actual logic to process requests. Nodes are async functions that receive the current state, execute their specific task (like classifying intent or looking up customers), and return state updates. These updates then flow to the next node in the pipeline. Let's build them one by one.

Intent Agent Node

Create src/agents/intentAgent.ts:

import { RunnableConfig } from "@langchain/core/runnables";
import { CustomerSupportState } from "../types/state";
import { intentClassifierTool } from "../tools";

export async function intentAgentNode(
  state: CustomerSupportState,
  config: RunnableConfig
): Promise<Partial<CustomerSupportState>> {
  console.log("Intent Agent: Classifying message...");

  const messages = state.messages || [];
  const lastMessage = messages[messages.length - 1];

  if (!lastMessage || lastMessage._getType() === "ai") {
    return { ...state };
  }

  const messageContent =
    typeof lastMessage.content === "string"
      ? lastMessage.content
      : JSON.stringify(lastMessage.content);

  try {
    const intentResult = await intentClassifierTool.invoke({
      message: messageContent,
    });

    const intent = JSON.parse(intentResult);
    console.log(`Intent classified: ${intent.category} (${intent.urgency})`);

    return {
      ...state,
      intent: intent,
    };
  } catch (error) {
    console.error("Intent classification error:", error);
    return {
      ...state,
      intent: {
        category: "general_inquiry",
        urgency: "low",
        confidence: 0.5,
        keywords: [],
      },
    };
  }
}

This node takes the last user message, classifies its intent and urgency, and stores it in state.intent.

Customer Lookup Agent Node

Create src/agents/customerLookupAgent.ts:

import { RunnableConfig } from "@langchain/core/runnables";
import { CustomerSupportState } from "../types/state";
import { customerLookupTool } from "../tools";

export async function customerLookupAgentNode(
  state: CustomerSupportState,
  config: RunnableConfig
): Promise<Partial<CustomerSupportState>> {
  console.log("Lookup Agent: Retrieving customer data...");

  const messages = state.messages || [];
  const lastMessage = messages[messages.length - 1];

  if (!lastMessage) {
    return { ...state };
  }

  const messageContent =
    typeof lastMessage.content === "string"
      ? lastMessage.content
      : JSON.stringify(lastMessage.content);

  // Extract customer ID from message (e.g., "5575-GNVDE")
  const customerIdMatch = messageContent.match(/\\b\\d{4}-[A-Z]{5}\\b/);

  if (!customerIdMatch) {
    console.log("No customer ID found in message");
    return {
      ...state,
      currentCustomer: { found: false },
    };
  }

  const customerId = customerIdMatch[0];

  try {
    const lookupResult = await customerLookupTool.invoke({ customerId });
    const customer = JSON.parse(lookupResult);

    console.log(`Customer found: ${customer.found}`);

    return {
      ...state,
      currentCustomer: customer,
    };
  } catch (error) {
    console.error("Customer lookup error:", error);
    return {
      ...state,
      currentCustomer: { found: false },
    };
  }
}

This node extracts the customer ID from the message, looks it up, and stores the result in state.currentCustomer.

Escalation Agent Node

Create src/agents/escalationAgent.ts:

import { RunnableConfig } from "@langchain/core/runnables";
import { CustomerSupportState } from "../types/state";
import { escalationDecisionTool } from "../tools";
import { AIMessage } from "@langchain/core/messages";
import { Command } from "@langchain/langgraph";

export async function escalationAgentNode(
  state: CustomerSupportState,
  config: RunnableConfig
): Promise<Partial<CustomerSupportState>> {
  console.log("Escalation Agent: Checking escalation criteria...");

  const intent = state.intent?.category || "general_inquiry";
  const urgency = state.intent?.urgency || "low";
  const customerId = state.currentCustomer?.found
    ? state.currentCustomer.id
    : undefined;

  try {
    const escalationResult = await escalationDecisionTool.invoke({
      customerId,
      intent,
      urgency,
    });

    const escalation = JSON.parse(escalationResult);

    if (escalation.required) {
      console.log(`ESCALATION REQUIRED: ${escalation.reason}`);

      const ticketId = `TKT-${Date.now()}-${Math.random()
        .toString(36)
        .substr(2, 5)
        .toUpperCase()}`;

      const messages = state.messages || [];
      const escalationMessage = new AIMessage({
        content: `I've created a priority ticket (${ticketId}) for you. Our ${escalation.assignedTo} team will contact you shortly. ${escalation.reason}`,
      });

      return {
        ...state,
        escalation: { ...escalation, ticketId },
        messages: [...messages, escalationMessage],
      };
    } else if (escalation.suggestAiFirst) {
      console.log("High urgency - offering AI suggestions first (HITL)");

      const messages = state.messages || [];
      const suggestionPrompt = new AIMessage({
        content: `I understand this is urgent for you. I have some suggestions that might help resolve this quickly. Would you like me to share them with you before escalating to a human agent?`,
      });

      return Command.interrupt({
        ...state,
        escalation: escalation,
        messages: [...messages, suggestionPrompt],
      });
    } else {
      console.log("No escalation needed - AI can handle this");
      return {
        ...state,
        escalation: escalation,
      };
    }
  } catch (error) {
    console.error("Escalation check error:", error);
    return {
      ...state,
      escalation: { required: false },
    };
  }
}

In this node:

If escalation is required, → Creates a ticket and assigns it to the appropriate team (billing, tech, or retention). There's no ticket management UI in this implementation, but you could expand the system to include one.
If urgency is high but churn is low, → AI assistance suggests some solutions before escalating
If no escalation → Proceeds directly to the Reply Agent

Reply Agent Node

Create src/agents/replyAgent.ts:

import { RunnableConfig } from "@langchain/core/runnables";
import { CustomerSupportState } from "../types/state";
import { replyGeneratorTool } from "../tools";
import { AIMessage } from "@langchain/core/messages";

export async function replyAgentNode(
  state: CustomerSupportState,
  config: RunnableConfig
): Promise<Partial<CustomerSupportState>> {
  console.log("Reply Agent: Generating response...");

  const messages = state.messages || [];
  const lastUserMessage = messages
    .filter((m) => m._getType() === "human")
    .pop();

  if (!lastUserMessage) {
    return { ...state };
  }

  const messageContent =
    typeof lastUserMessage.content === "string"
      ? lastUserMessage.content
      : "";

  const customerData = state.currentCustomer?.found
    ? JSON.stringify(state.currentCustomer.data)
    : "No customer data available";

  const intent = state.intent?.category || "general_inquiry";

  try {
    const replyContent = await replyGeneratorTool.invoke({
      customerData,
      intent,
      message: messageContent,
    });

    const replyMessage = new AIMessage({
      content: replyContent,
    });

    return {
      ...state,
      messages: [...messages, replyMessage],
    };
  } catch (error) {
    console.error("Reply generation error:", error);

    const fallbackMessage = new AIMessage({
      content: "I apologize, but I'm having trouble generating a response right now. Please try again.",
    });

    return {
      ...state,
      messages: [...messages, fallbackMessage],
    };
  }
}

This final node generates a personalized response based on customer data, intent, and the original message.

Run the Agent Server

Start the LangGraph server:

pnpm dev

This starts the agent on http://localhost:8123 using the LangGraph CLI.

All four agents work together as a single workflow, passing state between each step.

What You’ve Built

You now have a working multi-agent customer support system. The frontend handles the chat experience and shared state, while four focused agents handle classification, decision-making, and responses. CopilotKit ties the UI and agents together so actions taken by agents appear immediately in the interface, including escalations when a case needs human attention.

The full implementation, including error handling and logging, is available in the GitHub repository.

Wrapping Up

That’s the core idea behind this setup. Instead of hard-coding every step, you let agents coordinate through shared state and keep the UI in sync as work happens. When automation isn’t enough, the system naturally hands things off to a human.

This approach isn’t limited to customer support. The same structure works anywhere different agents need to act on the same data and surface their progress in a UI, whether that’s internal tools, operations dashboards, or workflow-heavy apps.

If you want to go further, you could layer in features such as ticket-tracking for escalations, integrations with external APIs, or lightweight reporting on agent activity.

Be sure to checkout the CopilotKit documentation and the example gallery.

Don't forget to follow CopilotKit on Twitter and join the Discord community of agent builders.

Happy building!

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

How to Build a LangGraph Research Agent that Embeds Dynamic Charts via MCP Apps (CopilotKit & Tako)

Anmol Baranwal — Tue, 03 Feb 2026 15:49:30 +0000

AI research assistants are getting smarter, but turning that reasoning into something usable is still hard.

Real research involves more than text: searching the web, exploring structured data, comparing sources, and gradually assembling a report. Doing that well requires state, tools, and UI -- not just a chat box.

Today, we will build a chat + canvas research assistant that searches the web, queries structured data via MCP, and embeds interactive charts directly into the report, while streaming progress to the frontend in real time.

We will use CopilotKit (Next.js) for the frontend, LangGraph to orchestrate the agent workflow, and MCP (Model Context Protocol) to connect external tools like Tako, which returns embeddable chart UI.

Check out CopilotKit's GitHub ⭐️

You will find architecture, key concepts, how state flows between the UI and the agent, how MCP Apps fit into the system, and a step-by-step guide to building this from scratch.

Let’s build it.

What is covered?

In summary, we will cover these topics in detail.

What are we building?
Tech Stack and Project Structure
Building the Frontend (CopilotKit)
Agent Orchestration (LangGraph)
Backend: MCP Integration & Tools
Running the Application
Complete Data Flow between UI ↔ agent

Here’s the GitHub Repo if you are interested in exploring it yourself.

1. What are we building?

We are building an AI research assistant with a live chat + canvas UI that searches, collects sources, and assembles a report while streaming progress to the frontend in real time.

It will:

Turn a user question into a research plan (research question + data sub-questions)
Run parallel retrieval: Tavily for web search + Tako via MCP apps for charts
Deduplicate results into a clean resources list
Generate a report with [CHART:title] placeholders and then embed the real charts inline
Stream intermediate state (logs/resources/report/charts) so the UI updates as the agent works.

We will see all of these concepts in action as we build the agent.

Core Components

A few days ago, the Anthropic team shipped MCP Apps as an official MCP extension, meaning tools can return interactive UI that the host can render, not just JSON or text.

That’s the pattern we will use here: our agent will call Tako (over MCP) to fetch charts as embeddable UI, so interactive visualizations can drop straight into the report/canvas instead of us building chart components.

At a high level, this project is built by combining:

CopilotKit for the agent chat + canvas UI
AG‑UI for streaming agent ↔ UI events (protocol)
LangGraph for stateful agent orchestration
MCP Apps for external tools
Tako for charts (search engine)
Tavily for web search

For context: AG-UI (Agent–User Interaction Protocol) is an event-based protocol that standardizes this “agent ↔ frontend” real-time communication.

Because it’s transport-agnostic (SSE, WebSockets, etc.), the same frontend can work across different agent backends without custom wiring. CopilotKit uses AG-UI as its synchronization layer.

Here's a simplified call request → response flow of what will happen:

User question
   ↓
CopilotKit UI (chat + canvas)
   ↓
LangGraph agent workflow
   ├─ chat: interpret intent + plan
   ├─ search: parallel retrieval (Tavily + Tako via MCP Apps)
   └─ report: narrative + [CHART:*] markers
   ↓
Streaming state updates (AG‑UI events)
   ↓
Canvas renders report + embedded charts

2. Tech Stack and Architecture

At the core, we are going to use this stack for building the agent:

Next.js 15 : frontend framework with TypeScript
CopilotKit 1.50 : agent UI components + streaming runtime (@copilotkit/react-core, @copilotkit/react-ui, @copilotkit/runtime)
LangGraph: stateful agent orchestration (StateGraph nodes, conditional routing)
FastAPI & Uvicorn : HTTP server for the agent
Tavily: web search API (tavily-python)
Model Context Protocol (MCP): connects the agent to Tako (mcp, httpx)
CopilotKit LangGraph SDK (Python): streams LangGraph state to CopilotKit (copilotkit)
OpenAI: LLM provider (openai)

There are also other libraries used, like react-markdown & remark-gfm for report rendering, langgraph-checkpoint-sqlite for state persistence, and Shadcn UI for components.

See package.json and agents/python/pyproject.toml in the repo for the complete dependency list.

Project structure

This is how our directory will look.

The agents/python/ directory hosts the Python LangGraph agent, exposed via FastAPI, which orchestrates the workflow (chat, search, report), streams state updates, and calls external tools (Tavily, Tako via MCP).

The src/ directory hosts the Next.js frontend, including the UI components, shared types, and the CopilotKit API route (/api/copilotkit/route.ts) that bridges the frontend to the agent backend.

.
├── src/                           ← Next.js frontend (TypeScript)
│   ├── app/
│   │   ├── page.tsx               ← CopilotKit provider & model selector
│   │   ├── Main.tsx               ← Chat + ResearchCanvas split layout
│   │   └── api/
│   │       └── copilotkit/
│   │           └── route.ts      ← CopilotRuntime bridge to agent
│   ├── components/
│   │   ├── ResearchCanvas.tsx    ← Main canvas (orchestrates report + resources)
│   │   ├── Resources.tsx         ← Displays Tavily + Tako resources list
│   │   ├── MarkdownRenderer.tsx  ← Renders report + embeds charts
│   │   └── ui/                   ← Reusable Shadcn UI components
│   └── lib/
│       ├── types.ts                     ← AgentState type
│       ├── utils.ts                     ← Utility functions
│       └── model-selector-provider.tsx  ← Model selection context
│
├── agents/
│   ├── python/                        ← Python LangGraph agent (primary)
│   │   ├── src/
│   │   │   ├── agent.py               ← StateGraph definition & compile
│   │   │   └── lib/
│   │   │       ├── state.py           ← AgentState (Pydantic)
│   │   │       ├── model.py       ← LLM factory (OpenAI / Anthropic / Gemini)
│   │   │       ├── chat.py            ← Chat node & tool definitions
│   │   │       ├── search.py          ← Parallel Tavily + Tako search
│   │   │       ├── mcp_integration.py ← MCP client & iframe helpers
│   │   │       ├── download.py        ← Download node
│   │   │       └── delete.py          ← Delete node
│   │   ├── main.py                    ← FastAPI/Uvicorn entrypoint
│   │   ├── requirements.txt
│   │   └── pyproject.toml
│   │
│   └── src/                       ← TypeScript agent (optional, local dev)
│       └── server.ts              ← Express + CopilotRuntime
│
├── package.json                   ← Frontend deps & scripts
├── .env.example
├── .env.local
└── README.md

Here's the GitHub repository and deployed live at tako-copilotkit.vercel.app if you want to explore yourself. I will cover the implementation and all key concepts in the following sections.

The easiest way to follow along is to clone the repo.

git clone https://github.com/TakoData/tako-copilotkit.git
cd tako-copilotkit

Add necessary API Keys

You can copy the environment template (.env.example) by using this command to create a .env.local in the root directory.

cp .env.example .env.local

Add your OpenAI API Key, Tavily API Key and Tako API Key to the file. I have attached the docs link so it's easy to follow.

OPENAI_API_KEY=sk-proj-...
TAVILY_API_KEY=tvly-dev-...
TAKO_API_TOKEN=your_api_token_here
TAKO_MCP_URL=https://mcp.tako.com  # URL of Tako's MCP server
TAKO_URL=https://tako.com  # URL of Tako's main API

The Tako/MCP integration is an optional data source for the research agent. If you want to query charts or structured datasets via a Tako MCP server, provide TAKO_API_TOKEN and the related URLs.

Otherwise, you can leave these unset & the agent will continue to work normally. For this tutorial, we will be using it.

OpenAI API Key

Tavily API Key

Tako API Key

3. Frontend: wiring the agent to the UI

Let's first build the frontend part.

The src/ directory hosts the Next.js frontend, including the UI components, shared types, and the CopilotKit API route (/api/copilotkit/route.ts) that bridges the frontend to the Python agent backend.

At a high level, the frontend is responsible for:

Sending user queries to the agent backend via CopilotChat
Receiving and rendering real-time state updates from the agent
Embedding Tako chart iframes in the rendered report
Managing the CopilotKit runtime bridge between UI and Python agent

If you are building this from scratch, the easiest approach is to copy the existing package.json from the repo.

It already includes all the required dependencies for CopilotKit, LangGraph integration, UI components, and local dev tooling, so you don’t have to assemble them manually.

I'm only covering the core frontend dependencies you actually need to understand.

Step 1: CopilotKit Provider & Layout

Install the necessary packages:

npm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

@copilotkit/react-core: Core React hooks and context that connect your UI to the agent backend via AG-UI protocol
@copilotkit/react-ui: Ready-made UI components like <CopilotChat /> for building AI chat interfaces
@copilotkit/runtime: Server-side runtime that exposes an API endpoint and bridges the frontend with the LangGraph agent using HTTP and SSE

Everything else in the repo (radix, tailwind, react-split, etc.) is there to support layout, styling, and developer experience -- not the core agent wiring.

The <CopilotKit> component must wrap the agent-aware parts of your application and point to the runtime endpoint so it can communicate with the agent backend.

This is the main entry point (page.tsx):

// page.tsx
"use client";

import { CopilotKit } from "@copilotkit/react-core";
import Main from "./Main";
import {
  ModelSelectorProvider,
  useModelSelectorContext,
} from "@/lib/model-selector-provider";

export default function ModelSelectorWrapper() {
  return (
    <ModelSelectorProvider>
      <Home />
    </ModelSelectorProvider>
  );
}

function Home() {
  const { agent, lgcDeploymentUrl } = useModelSelectorContext();

  // This logic is implemented to demonstrate multi-agent frameworks in this demo project.
  // There are cleaner ways to handle this in a production environment.
  const runtimeUrl = lgcDeploymentUrl
    ? `/api/copilotkit?lgcDeploymentUrl=${lgcDeploymentUrl}`
    : `/api/copilotkit${
        agent.includes("crewai") ? "?coAgentsModel=crewai" : ""
      }`;

  return (
    <div style={{ height: "100vh", overflow: "hidden" }}>
      <CopilotKit runtimeUrl={runtimeUrl} showDevConsole={false} agent={agent}>
        <Main />
      </CopilotKit>
    </div>
  );
}

Here, ModelSelectorProvider is a convenience for switching agents/models in development.

Step 2: Chat + Canvas Layout

The Main.tsx component sets up the core UI. It sets up:

the chat interface (CopilotChat)
the research canvas (where results & charts appear)
state binding to the agent

// app/Main.tsx
import { useCoAgent } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui";
import Split from "react-split";
import { ResearchCanvas } from "@/components/ResearchCanvas";
import { ChatInputWithModelSelector } from "@/components/ChatInputWithModelSelector";
import { AgentState } from "@/lib/types";
import { useModelSelectorContext } from "@/lib/model-selector-provider";

export default function Main() {
  const { model, agent } = useModelSelectorContext();

  const { state, setState } = useCoAgent<AgentState>({
    name: agent,
    initialState: {
      model,
      research_question: "",
      resources: [],
      report: "",
      logs: [],
    },
  });

  return (
    <div style={{ height: "100%", display: "flex", flexDirection: "column" }}>
      <h1>Research Helper</h1>

      <Split sizes={[30, 70]} style={{ flex: 1, display: "flex" }}>
        {/* Chat Panel */}
        <CopilotChat
          Input={ChatInputWithModelSelector}
          onSubmitMessage={async () => {
            setState({ ...state, logs: [] });
            await new Promise((r) => setTimeout(r, 30));
          }}
        />

        {/* Canvas Panel */}
        <ResearchCanvas />
      </Split>
    </div>
  );
}

Here's what's happening:

useCoAgent<AgentState>: bi-directional state synchronization between UI and agent. When the agent emits state updates via copilotkit_emit_state(), this hook automatically picks them up.
CopilotChat: drop-in chat UI that sends messages to the agent and renders tool calls inline
ResearchCanvas: custom component that renders the streaming report, resources list, and embedded charts
Split: provides a resizable split-pane layout

Step 3: Building Key Components

I'm only covering the core logic behind the main components since the overall code is huge. You can find all the components in the repository at src\components.

These components use CopilotKit hooks (like useCoAgentStateRender) to tie everything together.

✅ Research Canvas Component

ResearchCanvas.tsx is where the agent’s current state is rendered:

the accumulating report text
linked resources
embedded charts (iframes from MCP Apps like Tako)

This component listens to the agent state and displays elements as they arrive. It translates [CHART:title] markers into real embedded charts. This pattern is part of CopilotKit's support for generative UI.

// core logic
import { useCoAgent, useCoAgentStateRender, useCopilotAction } from "@copilotkit/react-core";
import { MarkdownRenderer } from "./MarkdownRenderer";
import { AgentState } from "@/lib/types";
import { useRef, useEffect } from "react";

export function ResearchCanvas() {
  const { state, setState } = useCoAgent<AgentState>({
    name: "research_agent",
  });

  // Use refs to prevent flicker during streaming updates
  const lastReportRef = useRef<string>("");
  const lastResourcesRef = useRef<Resource[]>([]);

  if (state.report) lastReportRef.current = state.report;
  if (state.resources?.length) lastResourcesRef.current = state.resources;

  const report = state.report || lastReportRef.current;
  const resources = state.resources || lastResourcesRef.current;

  // Render progress logs during execution
  useCoAgentStateRender({
    name: "research_agent",
    render: ({ state, status }) => {
      if (state.logs?.length) return <Progress logs={state.logs} />;
      return null;
    },
  });

  // Generative UI: Agent requests deletion confirmation from user
  useCopilotAction({
    name: "DeleteResources",
    description: "Prompt user for resource delete confirmation",
    available: "remote",
    parameters: [{ name: "urls", type: "string[]" }],
    renderAndWait: ({ args, handler }) => (
      <div>
        <h3>Delete these resources?</h3>
        <Resources resources={resources.filter(r => args.urls.includes(r.url))} />
        <button onClick={() => handler("YES")}>Delete</button>
        <button onClick={() => handler("NO")}>Cancel</button>
      </div>
    ),
  });

  return (
    <div>
      <h2>Research Question</h2>
      <div>{state.research_question || "Agent will identify your question..."}</div>

      {/* Resources Panel */}
      <Resources resources={resources} />

      {/* Report Panel with Embedded Charts */}
      <MarkdownRenderer content={report} />
    </div>
  );
}

The full component also includes collapsible sections for Tako charts vs web resources, and resource management dialogs. Check src/components/ResearchCanvas.tsx.

✅ Chart Embedding Pattern

The MCP Apps-style embedding happens in MarkdownRenderer.tsx. It embeds Tako charts without flickering during streaming updates. Instead of simple regex replacement, it uses a stable iframe pattern.

// core logic
import React, { useEffect, useRef, useMemo } from "react";
import ReactMarkdown from "react-markdown";

// Global registry: iframe src never changes for a given ID
const iframeRegistry = new Map<string, string>();

export function MarkdownRenderer({ content }: { content: string }) {
  // Split content into text and iframe segments
  const segments = useMemo(() => {
    const embedPattern = /<!doctype html>[\s\S]*?<\/html>/gi;
    const parts = [];
    let lastIndex = 0;

    content.replace(embedPattern, (match, offset) => {
      // Add text before iframe
      if (offset > lastIndex) {
        parts.push({ type: 'text', content: content.slice(lastIndex, offset) });
      }

      // Extract iframe src and create stable ID
      const src = match.match(/src=["']([^"']+)["']/)?.;
      if (src) {
        const id = `embed-${src.replace(/[^a-zA-Z0-9]/g, '-').slice(0, 50)}`;
        iframeRegistry.set(id, src);
        parts.push({ type: 'iframe', id, src });
      }

      lastIndex = offset + match.length;
      return match;
    });

    // Add remaining text
    if (lastIndex < content.length) {
      parts.push({ type: 'text', content: content.slice(lastIndex) });
    }

    return parts;
  }, [content]);

  return (
    <div>
      {segments.map((segment, i) => 
        segment.type === 'text' 
          ? <ReactMarkdown key={i}>{segment.content}</ReactMarkdown>
          : <StableIframe key={segment.id} src={segment.src} />
      )}
    </div>
  );
}

// Memoized iframe - never re-renders once mounted
const StableIframe = React.memo(({ src }: { src: string }) => {
  const iframeRef = useRef<HTMLIFrameElement>(null);

  // Handle Tako resize messages
  useEffect(() => {
    const handleResize = (event: MessageEvent) => {
      if (event.data?.type === "tako::resize" && 
          iframeRef.current?.contentWindow === event.source) {
        iframeRef.current.style.height = `${event.data.height}px`;
      }
    };
    window.addEventListener("message", handleResize);
    return () => window.removeEventListener("message", handleResize);
  }, []);

  return <iframe ref={iframeRef} src={src} style={{ height: "400px" }} />;
});

Tako returns fully-formed chart UI via MCP Apps so the frontend only needs to embed the iframe. Here's what's happening:

Iframes are rendered as stable siblings (outside ReactMarkdown), preventing remounts during streaming updates
Each embed uses a deterministic ID derived from its src, ensuring consistent DOM identity
Resize events from Tako (tako::resize) are handled via postMessage to keep charts responsive
rehypeRaw: Allows ReactMarkdown to render raw HTML (the iframe elements)

Check out the complete code at src/components/MarkdownRenderer.tsx.

✅ Model Selector Provider

The ModelSelectorProvider (src\lib\model-selector-provider.tsx) is a convenience wrapper for switching between different agent configurations during development. It reads URL query parameters to determine which agent/model to use.

"use client";

import React from "react";
import { createContext, useContext, useState, ReactNode } from "react";

type ModelSelectorContextType = {
  model: string;
  setModel: (model: string) => void;
  agent: string;
  lgcDeploymentUrl?: string | null;
  hidden: boolean;
  setHidden: (hidden: boolean) => void;
};

const ModelSelectorContext = createContext<ModelSelectorContextType | undefined>(undefined);

export const ModelSelectorProvider = ({ children }: { children: ReactNode }) => {
  // Read model from URL query params (e.g., ?coAgentsModel=google_genai)
  const model =
    globalThis.window === undefined
      ? "openai"
      : new URL(window.location.href).searchParams.get("coAgentsModel") ?? "openai";

  const [hidden, setHidden] = useState<boolean>(false);

  const setModel = (model: string) => {
    const url = new URL(window.location.href);
    url.searchParams.set("coAgentsModel", model);
    window.location.href = url.toString();
  };

  // Optional: LangGraph Cloud deployment URL for production
  const lgcDeploymentUrl =
    globalThis.window === undefined
      ? null
      : new URL(window.location.href).searchParams.get("lgcDeploymentUrl");

  // Map model to agent name
  let agent = "research_agent";
  if (model === "google_genai") {
    agent = "research_agent_google_genai";
  } else if (model === "crewai") {
    agent = "research_agent_crewai";
  }

  return (
    <ModelSelectorContext.Provider
      value={{ model, agent, lgcDeploymentUrl, hidden, setModel, setHidden }}
    >
      {children}
    </ModelSelectorContext.Provider>
  );
};

export const useModelSelectorContext = () => {
  const context = useContext(ModelSelectorContext);
  if (context === undefined) {
    throw new Error("useModelSelectorContext must be used within a ModelSelectorProvider");
  }
  return context;
};

The repo also includes UI components like Resources.tsx (resource cards), Progress.tsx (logs display), EditResourceDialog.tsx and AddResourceDialog.tsx for resource management, plus ChatInputWithModelSelector for model switching during development.

Step 4: Runtime Bridge (API Route)

This Next.js API route acts as a thin proxy between the browser and the Python LangGraph agent. It:

Accepts CopilotKit requests from the UI
Forwards them to the LangGraph agent via HTTP
Streams agent state and events back to the frontend via SSE

Instead of letting the frontend talk to the FastAPI agent directly, all requests go through a single endpoint /api/copilotkit.

import {
  CopilotRuntime,
  copilotRuntimeNextJSAppRouterEndpoint,
  EmptyAdapter,
} from "@copilotkit/runtime";
import {
  LangGraphHttpAgent,
  LangGraphAgent,
} from "@copilotkit/runtime/langgraph";
import { NextRequest } from "next/server";

// const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
// const llmAdapter = new OpenAIAdapter({ openai } as any);
const llmAdapter = new EmptyAdapter();
const langsmithApiKey = process.env.LANGSMITH_API_KEY as string;

export const POST = async (req: NextRequest) => {
  const searchParams = req.nextUrl.searchParams;
  const deploymentUrl =
    searchParams.get("lgcDeploymentUrl") || process.env.LGC_DEPLOYMENT_URL;

  const baseUrl =
    process.env.REMOTE_ACTION_URL || "http://localhost:2024/copilotkit";

  let runtime = new CopilotRuntime({
    agents: {
      research_agent: new LangGraphHttpAgent({
        url: `${baseUrl}/agents/research_agent`,
      }),
      research_agent_google_genai: new LangGraphHttpAgent({
        url: `${baseUrl}/agents/research_agent_google_genai`,
      }),
    },
  });

  if (deploymentUrl) {
    runtime = new CopilotRuntime({
      agents: {
        research_agent: new LangGraphAgent({
          deploymentUrl,
          langsmithApiKey,
          graphId: "research_agent",
        }),
        research_agent_google_genai: new LangGraphAgent({
          deploymentUrl,
          langsmithApiKey,
          graphId: "research_agent_google_genai",
        }),
      },
    });
  }

  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter: llmAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

The frontend is fully reactive: as the agent works, the UI updates in real time.

4. Agent Orchestration (LangGraph)

Now that the frontend can communicate with an agent, we need to define how the agent thinks and operates. The backend agent lives in agents/python/src/.

It uses LangGraph to orchestrate a multi-step research workflow (chat → search → download → report...) through well-defined steps (nodes), each responsible for a specific task.

Each step:

receives the current AgentState
performs one responsibility
returns partial state updates
decides what node runs next

Step 1: Define the StateGraph (agent.py)

The workflow is defined using LangGraph's StateGraph in agent.py. It models the research assistant as a stateful workflow, where each step (node) operates on shared state and decides which step to run next.

At a high level:

AgentState is the shared memory across the entire agent (messages, report, resources, logs, ...)
Each node is an async function that:
- reads from the current state
- updates part of that state
- returns control-flow instructions
Edges control execution flow between nodes (download → chat → search → download loop)

One important detail is the interrupt_after hook, which allows the agent to pause execution for user confirmation before deleting a resource (delete_node).

# agents/python/src/agent.py
import os

from langgraph.graph import StateGraph

from src.lib.chat import chat_node
from src.lib.delete import delete_node, perform_delete_node
from src.lib.download import download_node
from src.lib.search import search_node
from src.lib.state import AgentState

workflow = StateGraph(AgentState)
workflow.add_node("download", download_node)
workflow.add_node("chat_node", chat_node)
workflow.add_node("search_node", search_node)
workflow.add_node("delete_node", delete_node)
workflow.add_node("perform_delete_node", perform_delete_node)


workflow.set_entry_point("download")
workflow.add_edge("download", "chat_node")
workflow.add_edge("delete_node", "perform_delete_node")
workflow.add_edge("perform_delete_node", "chat_node")
workflow.add_edge("search_node", "download")

# Compile with interrupt support for deletion confirmation
compile_kwargs = {"interrupt_after": ["delete_node"]}

# Check if we're running in LangGraph API mode
if os.environ.get("LANGGRAPH_FASTAPI", "false").lower() == "false":
    # When running in LangGraph API, don't use a custom checkpointer
    graph = workflow.compile(**compile_kwargs)
else:
    # For CopilotKit and other contexts, use MemorySaver
    from langgraph.checkpoint.memory import MemorySaver

    memory = MemorySaver()
    compile_kwargs["checkpointer"] = memory
    graph = workflow.compile(**compile_kwargs)

Why this structure works well:

The download → chat → search → download loop naturally supports iterative research
Optional checkpointing (MemorySaver) allows the same graph to run locally or on LangGraph Cloud (state persistence)

Step 2: Chat Node & Tools (chat.py)

The chat node (chat_node) is the brain of the agent. It orchestrates the LLM, defines tools, manages state streaming, and handles chart injection. It:

Define LLM-callable tools (Search, WriteReport, WriteResearchQuestion, GenerateDataQuestions)
Configure intermediate state emission (stream research question and data questions as they are generated)
Build prompts with available resources and Tako chart context
Inject Tako chart iframes into the report after generation
Route to next node based on tool calls

Here are the core tools (agents/python/src/lib/chat.py).

@tool
def Search(queries: List[str]):
    """A list of one or more search queries to find good resources."""

@tool
def WriteReport(report: str):
    """Write the research report."""

@tool
def WriteResearchQuestion(research_question: str):
    """Write the research question."""

@tool
def GenerateDataQuestions(questions: List[DataQuestion]):
    """
    Generate 3-6 ATOMIC data-focused questions for Tako search.

    CRITICAL: Each query should ask for ONE metric/dimension only.
    BAD: "Compare US GDP and inflation" → compound query
    GOOD: "US GDP 2020-2024", "US inflation rate" → separate atomic queries
    """

Here is the chat node logic.

async def chat_node(state: AgentState, config: RunnableConfig):
    # Configure intermediate state emission
    config = copilotkit_customize_config(
        config,
        emit_intermediate_state=[
            {"state_key": "research_question", "tool": "WriteResearchQuestion", "tool_argument": "research_question"},
            {"state_key": "data_questions", "tool": "GenerateDataQuestions", "tool_argument": "questions"},
        ],
    )

    # Build Tako charts context for LLM
    tako_charts_map = {}
    available_tako_charts = []
    for resource in state["resources"]:
        if resource.get("resource_type") == "tako_chart":
            title = resource.get("title", "")
            card_id = resource.get("card_id")
            description = resource.get("description", "")
            tako_charts_map[title] = {"card_id": card_id, "embed_url": resource.get("embed_url")}
            available_tako_charts.append(f"  - **{title}**\n    Description: {description}")

    # Build prompt with workflow instructions
    response = await model.bind_tools([Search, WriteReport, WriteResearchQuestion, GenerateDataQuestions]).ainvoke([
        SystemMessage(content=f"""
        You are a research assistant. 

        WORKFLOW:
        1. Use WriteResearchQuestion to extract the core research question
        2. Use GenerateDataQuestions to create 3-6 ATOMIC data questions (one metric per query)
        3. Use Search for web resources
        4. Write a comprehensive report using WriteReport

        AVAILABLE CHARTS ({len(tako_charts_map)}):
        {available_tako_charts_str}

        WRITING GUIDELINES:
        - Write a detailed analysis with substantial narrative text
        - For EACH chart, write 1-2 paragraphs discussing insights and trends
        - DO NOT include chart markers or image syntax - charts will be inserted automatically
        - Reference specific data points from chart descriptions
        """),
        *state["messages"],
    ], config)

    # Handle tool calls
    if response.tool_calls:
        if response.tool_calls[0]["name"] == "WriteReport":
            report = response.tool_calls[0]["args"]["report"]

            # Clean up any markdown images LLM incorrectly added
            report = re.sub(r'!\[[^\]]*\]\([^)]+\)', '', report)

            # Inject charts using second LLM pass
            if tako_charts_map:
                chart_list = "\n".join([f"- {title}" for title in tako_charts_map.keys()])

                inject_response = await model.ainvoke([
                    SystemMessage(content=f"""Insert [CHART:exact_title] markers at appropriate positions.

                    AVAILABLE CHARTS:
                    {chart_list}

                    RULES:
                    - Place markers AFTER relevant paragraphs
                    - NEVER place more than two charts consecutively
                    - Each chart is used exactly once
                    """),
                    HumanMessage(content=f"Insert chart markers:\n\n{report}")
                ])

                report_with_markers = inject_response.content

                # Replace markers with actual iframe HTML
                async def replace_marker(match):
                    chart_title = match.group(1).strip()
                    chart_info = tako_charts_map.get(chart_title)
                    if chart_info:
                        iframe_html = await get_visualization_iframe(
                            item_id=chart_info.get("card_id"),
                            embed_url=chart_info.get("embed_url")
                        )
                        return "\n" + iframe_html + "\n" if iframe_html else ""
                    return ""

                # Apply replacements
                markers = list(re.finditer(r'\[CHART:([^\]]+)\]', report_with_markers))
                processed_report = report_with_markers
                for match in reversed(markers):
                    replacement = await replace_marker(match)
                    processed_report = processed_report[:match.start()] + replacement + processed_report[match.end():]

            return Command(goto="chat_node", update={"report": processed_report})

        elif response.tool_calls[0]["name"] == "GenerateDataQuestions":
            data_questions = response.tool_calls[0]["args"]["questions"]
            return Command(goto="search_node", update={"data_questions": data_questions})

    # Route based on tool call
    goto = "__end__"
    if response.tool_calls:
        tool_name = response.tool_calls[0]["name"]
        if tool_name == "Search":
            goto = "search_node"
        elif tool_name == "DeleteResources":
            goto = "delete_node"

    return Command(goto=goto, update={"messages": response})

How this node works in practice:

The LLM never writes free-form text directly. Every meaningful action goes through an explicit tool (WriteResearchQuestion, GenerateDataQuestions, Search, WriteReport). This keeps state changes deterministic and easy to reason about.
Intermediate state (research question and data questions) is streamed immediately using copilotkit_customize_config. The report itself is intentionally not streamed to avoid UI flicker while charts are being injected later.
Chart handling is deliberately two-phase: the model first writes a clean narrative report, then a second pass inserts [CHART:title] markers, which are finally replaced with real Tako iframes. To keep the generation stable and layout-safe.
Control flow is explicit. Each tool call results in a Command that routes execution to the next node (search_node, delete_node, or back to chat_node), making the agent’s behavior predictable and easier to debug.

Step 3: Search Node (search.py)

The search node (search_node) runs web search (Tavily) and structured chart data search (Tako via MCP) in parallel, streams progress to the UI, deduplicates results, and extracts the most relevant resources.

Tavily and Tako searches run concurrently using asyncio.gather, so web pages and structured datasets are fetched at the same time
TavilyClient is synchronous (uses requests library), so we wrap it with loop.run_in_executor() to avoid blocking the async event loop
As each search completes, copilotkit_emit_state is called to update state.logs, allowing the frontend to show real-time progress
Tako chart results are deduplicated using the title to avoid embedding the same visualization multiple times
Web results and chart results are passed through an LLM tool (ExtractResources) to select only the most relevant resources
Each resource is tagged as web or tako_chart, which the frontend uses to decide whether to render a link or an embedded chart

Here is the code for Tavily async wrapper (bridging sync client with async workflow):

# agents/python/src/lib/search.py

from tavily import TavilyClient
import asyncio

_tavily_client = None

def get_tavily_client():
    """Lazy initialization to avoid repeated client creation"""
    global _tavily_client
    if _tavily_client is None:
        _tavily_client = TavilyClient(api_key=os.getenv("TAVILY_API_KEY"))
    return _tavily_client

async def async_tavily_search(query: str) -> Dict[str, Any]:
    """
    Async wrapper for synchronous Tavily client.
    TavilyClient.search() is blocking, so we run it in a thread pool.
    """
    loop = asyncio.get_event_loop()
    return await loop.run_in_executor(
        None,
        lambda: get_tavily_client().search(
            query=query,
            search_depth="advanced",
            include_answer=True,
            max_results=5,
        ),
    )

Search node implementation (parallel execution and resource extraction):

# agents/python/src/lib/search.py (core search pattern)
async def search_node(state: AgentState, config: RunnableConfig):
    """
    Perform Tavily web search and Tako MCP chart search in parallel.
    Stream progress, dedupe results, extract top resources.
    """
    # Extract queries from tool_calls or fall back to research_question
    # (Actual code handles dual routing: Search tool OR GenerateDataQuestions)
    queries = state.get("search_queries", [])
    data_questions = state.get("data_questions", [])

    # PHASE 1: Build parallel tasks
    tavily_tasks = [async_tavily_search(query) for query in queries]
    tako_tasks = [
        search_knowledge_base(q["question"], search_effort="fast")
        for q in data_questions
    ]
    all_tasks = tavily_tasks + tako_tasks

    # Initialize logs for progress tracking
    for query in queries:
        state["logs"].append({"message": f"Web search: {query}", "done": False})
    for q in data_questions:
        state["logs"].append({"message": f"Tako search: {q['question']}", "done": False})

    if all_tasks:
        await copilotkit_emit_state(config, state)

    # PHASE 2: Run all searches concurrently
    all_results = await asyncio.gather(*all_tasks, return_exceptions=True)

    # Split results back into Tavily and Tako
    num_tavily = len(tavily_tasks)
    tavily_results = all_results[:num_tavily]
    tako_results_raw = all_results[num_tavily:]

    # PHASE 3: Stream completion and process results
    search_results = []
    tako_results = []

    # Process Tavily results
    for i, result in enumerate(tavily_results):
        if isinstance(result, Exception):
            search_results.append({"error": str(result)})
        else:
            search_results.append(result)
        state["logs"][i]["done"] = True
        await copilotkit_emit_state(config, state)

    # Process Tako results
    tako_log_offset = num_tavily
    for i, result in enumerate(tako_results_raw):
        if isinstance(result, Exception):
            tako_results.append({"error": str(result)})
        elif result:
            tako_results.extend(result)
        state["logs"][tako_log_offset + i]["done"] = True
        await copilotkit_emit_state(config, state)

    # PHASE 4: Deduplicate Tako charts by title
    # (Same chart may appear in multiple searches)
    seen_titles = set()
    unique_tako = []
    for chart in tako_results:
        title = chart.get("title", "")
        if title and title not in seen_titles:
            seen_titles.add(title)
            unique_tako.append(chart)
        elif not title:
            unique_tako.append(chart)

    # PHASE 5: Use LLM to extract the most relevant resources
    model = get_model(state)

    # Prepare search message with both web and Tako results
    search_message = f"Web search results: {search_results}"
    if unique_tako:
        search_message += f"\n\nTako chart results: {unique_tako}"

    # Build extraction prompt
    extract_messages = [
        SystemMessage(content="""
        Extract the 3-5 most relevant resources from the search results.
        Prioritize Tako charts when they match the research topic.
        """),
        *state["messages"],
        SystemMessage(content=f"Search results:\n{search_message}")
    ]

    # Call LLM with ExtractResources tool
    response = await model.bind_tools(
        [ExtractResources], 
        tool_choice="ExtractResources"
    ).ainvoke(extract_messages, config)

    # PHASE 6: Tag resources with type and attach content
    resources = response.tool_calls[0]["args"]["resources"]

    for resource in resources:
        # Check if this is a Tako chart by matching URL or title
        is_tako = False
        for tako_result in unique_tako:
            if (resource.get("url") == tako_result.get("url") or 
                resource.get("title") == tako_result.get("title")):
                is_tako = True
                resource["resource_type"] = "tako_chart"
                resource["source"] = "Tako"
                resource["card_id"] = tako_result.get("id")
                resource["embed_url"] = tako_result.get("embed_url")
                resource["content"] = tako_result.get("description", "")
                break

        if not is_tako:
            resource["resource_type"] = "web"
            resource["source"] = "Tavily Web Search"
            # Find matching Tavily result for content summary
            for search_result in search_results:
                if isinstance(search_result, dict) and "results" in search_result:
                    for tavily_item in search_result["results"]:
                        if tavily_item.get("url") == resource.get("url"):
                            resource["content"] = tavily_item.get("content", "")
                            break

    # Deduplicate and add to state resources
    # (Production code enforces MAX_TOTAL_RESOURCES limit and generates iframe HTML)
    state["resources"].extend(resources)

    return state

This simplified version shows the core search pattern.

The production code adds fallback searches when Tako returns no results, generates iframe HTML for chart embedding via get_visualization_iframe(), enforces resource limits, and handles dual routing for different tool paths.

Check out the complete implementation at agents/python/src/lib/search.py.

5. Backend: MCP Integration & Tools

At this point, the UI is wired, the agent graph is defined, and the execution flow is clear. What’s left is the backend work that actually fetches data, talks to external systems, and turns results into something the agent can reason about.

The most interesting part here is the MCP integration for Tako charts. The rest of the backend nodes are intentionally simple.

MCP Apps Pattern for Tako Charts

Tako is a data visualization platform with searchable charts on economics, markets, demographics, and more.

What Tako provides:

Searchable chart catalog via knowledge_search tool
Chart search with configurable effort: "fast" (quick lookups), "medium", or "deep" (thorough searches)
Ready-to-embed iframe HTML via open_chart_ui tool
Auto-resizing charts via postMessage protocol

This is the core idea behind MCP Apps: tools return UI components, not just data.

This module mcp_integration.py is where the agent connects to Tako via the Model Context Protocol (MCP). I have broken the core parts of this integration, so it's easier to follow along.

The MCP Client: Session Management

The SimpleMCPClient class handles the Model Context Protocol lifecycle:

SSE connection establishes and maintains the session ID
HTTP messages send JSON-RPC tool calls to the server
Automatic reconnection on session expiry (404/410 errors)

# core client logic -- agents/python/src/lib/mcp_integration.py

class SimpleMCPClient:
    """
    Minimal MCP client following the Model Context Protocol spec.
    Handles connection lifecycle, session management, and message passing.
    """

    async def connect(self):
        """Connect to MCP server via SSE to get session ID."""
        self._sse_task = asyncio.create_task(self._sse_reader())

        # Wait for session_id establishment
        for _ in range(50):
            if self.session_id:
                return True
            await asyncio.sleep(0.1)
        return False

    async def reconnect(self):
        """Reconnect with new session on expiry."""
        if self._sse_task:
            self._sse_task.cancel()

        # Clear state and reconnect
        self.session_id = None
        self._responses.clear()
        await self.connect()
        await self.initialize()

    async def _send(self, method: str, params: dict = None, _retry: bool = True):
        """
        Send JSON-RPC message to server via HTTP.
        Automatically reconnects and retries once on session expiration (410/404).
        """
        msg = {"jsonrpc": "2.0", "id": self.message_id, "method": method}
        if params:
            msg["params"] = params

        future = asyncio.get_event_loop().create_future()
        self._responses[self.message_id] = future

        try:
            resp = await self._client.post(
                f"{self.base_url}/messages/?session_id={self.session_id}",
                json=msg,
            )

            # Handle session expiration
            if resp.status_code in (404, 410):
                if _retry:
                    await self.reconnect()
                    return await self._send(method, params, _retry=False)
                raise SessionExpiredException("Session expired")

        except httpx.HTTPStatusError as e:
            if e.response.status_code in (404, 410) and _retry:
                await self.reconnect()
                return await self._send(method, params, _retry=False)
            raise

        return await asyncio.wait_for(future, timeout=120.0)

# Global client instance (reused across all calls)
_mcp_client: Optional[SimpleMCPClient] = None

A single global client instance maintains the session across all tool calls. If a session expires mid-conversation, the client automatically reconnects and retries once.

Searching Tako's Knowledge Base

The search_knowledge_base function wraps the MCP knowledge_search tool to find relevant charts.

Each result becomes a Resource with resource_type = "tako_chart". The search_node runs these queries in parallel with Tavily web searches using asyncio.gather() for speed (as we discussed in the previous section).

async def search_knowledge_base(
    query: str,
    count: int = 5,
    search_effort: str = "fast"
) -> List[Dict[str, Any]]:
    """
    Search Tako knowledge base via MCP 'knowledge_search' tool.

    Returns a list of charts with metadata (id, title, description, embed_url).
    """
    result = await _call_mcp_tool("knowledge_search", {
        "query": query,
        "api_token": TAKO_API_TOKEN,
        "count": count,
        "search_effort": search_effort
    })

    if result and "results" in result:
        formatted_results = []
        for item in result["results"]:
            item_id = item.get("card_id") or item.get("id")
            title = item.get("title", "")
            description = item.get("description", "")
            embed_url = f"{DATA_SOURCE_URL}/embed/{item_id}/?theme=dark" if item_id else None

            formatted_results.append({
                "type": "data_visualization",
                "id": item_id,
                "title": title,
                "description": description,
                "embed_url": embed_url,
                "url": f"{DATA_SOURCE_URL}/card/{item_id}"
            })

        return formatted_results

    return []

Fetching Embeddable Chart UI

The get_visualization_iframe function is where MCP Apps shines. Instead of returning chart data, it returns complete iframe HTML.

There are two paths involved here:

MCP UI (open_chart_ui): Returns complete iframe HTML with interactive controls and resize logic
Fallback embed URL: Generates basic iframe HTML manually if the MCP call fails

The returned HTML is injected directly into the markdown report. The frontend's MarkdownRenderer extracts these iframe blocks and renders them as siblings to the markdown content (preventing flickering during streaming updates).

async def get_visualization_iframe(item_id: str = None, embed_url: str = None) -> Optional[str]:
    """
    Get embeddable iframe HTML for a Tako chart.
    Uses the MCP 'open_chart_ui' tool with automatic session reconnection.

    Returns ready-to-embed iframe HTML with resize script.
    """
    if item_id:
        result = await _call_mcp_tool("open_chart_ui", {
            "pub_id": item_id,
            "dark_mode": True,
            "width": 900,
            "height": 600
        })

        # Extract HTML from MCP response
        if result and result.get("type") == "resource":
            resource = result.get("resource", {})
            html_content = resource.get("htmlString") or resource.get("text")
            if html_content:
                return html_content

    # Fallback: Generate iframe HTML with embed_url
    if embed_url:
        return f'''<iframe
  width="100%"
  height="600"
  src="{embed_url}"
  scrolling="no"
  frameborder="0"
></iframe>

<script type="text/javascript">
window.addEventListener("message", function(e) {{
  if (e.data.type !== "tako::resize") return;

  for (let iframe of document.querySelectorAll("iframe")) {{
    if (iframe.contentWindow === e.source) {{
      iframe.style.height = e.data.height + "px";
    }}
  }}
}});
</script>'''

    return None

This is how MCP Apps actually works. The agent writes [CHART:title] markers in the report, and the backend replaces them with full iframe HTML from Tako.

The frontend never needs to know how to render Tako charts; it just embeds the iframe. Check out the complete code at agents/python/src/lib/mcp_integration.py.

Other backend nodes (briefly)

The repository includes a few more helper nodes, but they follow standard patterns and don’t introduce new concepts, so I have left them out to avoid unnecessary length.

download_node: Downloads web resource content and caches it (truncated to 3000 chars to reduce context bloat)
delete_node: handles resource deletion with user confirmation (interrupt pattern for generative UI)
state.py: defines TypedDict schemas for AgentState, Resource, DataQuestion, and Log
model.py: function to get LLM instance (OpenAI/Anthropic/Google) based on env config

6. Running the Application

After completing all the parts of the code, it's time to run it locally. Please make sure you have added the credentials to the .env in the root directory.

Install all dependencies (frontend + Python agent) by running this command from the root. This installs the Next.js frontend packages and also runs uv sync in agents/python to install Python dependencies.

npm install

This repository runs the frontend and backend together in development mode. Just run the following command from the root:

npm run dev

Behind the scenes, this is wired up using concurrently in package.json:

"dev": "concurrently -k -n ui,agent -c blue,red \"PORT=3000 npm run dev:ui\" \"PORT=2024 npm run dev:agent\"",
"dev:ui": "next dev",
"dev:agent": "cd agents/python && uv run main.py"

dev:ui → runs Next.js frontend on port 3000
dev:agent → runs Python FastAPI agent backend on port 2024 (using uv run)
concurrently runs both in parallel with colored output (blue for UI, red for agent)

You can navigate to http://localhost:3000 in your browser to view it locally.

Here is the demo!

6. Data flow

Now that we've built the frontend (Next.js + CopilotKit), the agent orchestration (LangGraph) and the backend integrations (MCP + Tavily), this is how data actually flows between them.

This should be easy to follow if you have been building along so far.

[User enters research question]
      ↓
Next.js UI (CopilotKit Provider + CopilotChat + ResearchCanvas)
        ↓
POST /api/copilotkit (CopilotKit runtime endpoint)
        ↓
CopilotRuntime → Python Agent (LangGraph)
        ↓
LangGraph StateGraph orchestration
   ├─ chat_node: interpret intent, decide tools 
   │   └─ (generates report with [CHART:title] markers, then replaces with iframe HTML)
   ├─ search_node: parallel gather + dedupe/normalize resources
   │      ├─ Tavily search (web results)
   │      └─ Tako MCP search (returns chart metadata)
   └─ download_node: cache web resource content
        ↓
copilotkit_emit_state(): stream intermediate AgentState with report containing iframe HTML (SSE)
        ↓
Frontend receives state updates (resources/report with embedded iframes)
        ↓
MarkdownRenderer extracts <!doctype html>...</html> iframe blocks and renders as siblings to markdown
        ↓
User sees: streaming report + resources list + interactive embedded charts

That’s it! 🎉

You now have a research assistant that streams agent state into a chat + canvas UI, blends web search with MCP-powered charts, and produces an interactive report.

What makes this work is the structure: clear agent state, explicit tool boundaries, and a UI that renders progress instead of hiding it. Once those pieces are in place, adding new MCP tools or constraints becomes an implementation detail.

I hope you learned something valuable. Have a great day!

The Developer's Guide to Generative UI in 2026

Anmol Baranwal — Thu, 29 Jan 2026 14:41:12 +0000

AI agents have become much better at reasoning and planning. The UI layer has mostly stayed the same, and it is holding back the experience.

Most agent experiences still rely on chat, even when the task clearly needs forms, previews, controls, or step-by-step feedback.

Generative UI is the idea that allows agents to influence the interface at runtime, so the UI can change as context changes. This is usually done through UI specs like A2UI, Open-JSON-UI, or MCP Apps.

Check out CopilotKit's GitHub ⭐️

Today, we will break down Generative UI, the three practical patterns, and how CopilotKit supports them (using AG-UI protocol under the hood).

Let's jump in.

What is covered?

In summary, we will cover these topics in detail.

What is Generative UI, and why does it matter?
Three Patterns of Generative UI
- Static Generative UI (AG-UI)
- Declarative Generative UI (A2UI and Open-JSON-UI)
- Open-ended Generative UI (MCP Apps)
How AG-UI enables Generative UI at runtime

Here’s the GitHub Repo for the live playground that implements all three approaches.

1. What is Generative UI?

Generative UI (sometimes called GenUI) is a pattern in which parts of the user interface are generated, selected, or controlled by an AI agent at runtime rather than being fully predefined by developers.

Instead of hardcoding every button, for,m or panel upfront, the UI is created dynamically based on the user's needs and context. The agent can decide what UI to show, what inputs it needs from the user, and how the state should update as the task progresses.

Instead of only generating text, the agent can:

request structured inputs
render task-specific UI
update the interface as tools run and state changes

This turns the UI into an active part of the agent’s execution rather than a static wrapper around a chat box. The frontend no longer has to infer intent from text.

Generative UI = LLM output → live, interactive UI

Examples:

A travel query → itinerary cards + map + expandable sections
A comparison query → sortable tables

In the CopilotKit ecosystem, Generative UI is not a single abstraction. It is a set of patterns built on top of agentic UI specifications (such as A2UI and MCP Apps) that define how agents communicate UI updates to applications. You can read more on the website and refer to the docs.

Google describes Generative UI as a way to give models a rich, custom, and interactive user experience that adapts to any prompt. If you are interested in reading more, you can check out their research blog.

A high-level system overview of the generative UI implementation (from Google Research).

Source: Google Research

Why does it matter?

Most agent experiences today still look like chat apps. When agents move beyond chatting and start doing things, text alone becomes a bottleneck.

Common problems with text-only agents:

tool execution and progress are hidden behind chat messages
user inputs are vague, easy to misinterpret, and hard to validate
multi-step flows feel opaque, so users do not trust the outcome

With Generative UI:

render task-specific UI exactly when it is needed
collect structured, validated input
show progress and intermediate results as real UI
adapt the interface as plans evolve

This makes agent systems easier to understand and guide. Instead of interacting with a black box, users interact with a system that exposes its state through UI.

2. Three Patterns of Generative UI

Most Generative UI implementations fit into three patterns. What separates them is how much control stays in the frontend versus how much freedom is given to the agent.

Here are the three types (along with example specs):

1) Static Generative UI (high control, low freedom) - streamed via AG-UI

The frontend owns the UI. The agent only selects which predefined component to show and fills it with data.

2) Declarative Generative UI (shared control) - using A2UI, Open-JSON-UI

The agent returns a structured UI spec (cards, lists, forms). The frontend renders it with its own constraints and styling.

3) Open-ended Generative UI (low control, high freedom) - enabled by MCP Apps

The agent returns a full UI surface (often embedded or free-form). The frontend mainly hosts it, so you trade consistency and safety for flexibility.

If you want to see these patterns side by side, there’s a live playground showing all three, along with a code implementation:

Live playground: go.copilotkit.ai/gen-ui-demo
Code: github.com/CopilotKit/generative-ui-playground

Here is a short demo!

You can check the docs and pick the integration you are using to see detailed implementation guides and examples.

Now, let's cover each pattern in depth and see how to implement them.

✅ Static Generative UI (AG-UI)

Static Generative UI is the most constrained pattern.

Developers pre-build UI components and the agent’s role is limited to deciding when a component appears and what data it receives. The layout and interactions are fully owned by the application.

In CopilotKit, this pattern is implemented using the useFrontendTool hook, which binds predefined UI components to the lifecycle of an action.

// Weather tool - a callable tool that displays weather data in a styled card
useFrontendTool({
  name: "get_weather",
  description: "Get current weather information for a location",
  parameters: z.object({ location: z.string().describe("The city or location to get weather for") }),
  handler: async ({ location }) => {
    await new Promise((r) => setTimeout(r, 500));
    return getMockWeather(location);
  },
  render: ({ status, args, result }) => {
    if (status === "inProgress" || status === "executing") {
      return <WeatherLoadingState location={args?.location} />;
    }
    if (status === "complete" && result) {
      const data = JSON.parse(result) as WeatherData;
      return (
        <WeatherCard
          location={data.location}
          temperature={data.temperature}
          conditions={data.conditions}
          humidity={data.humidity}
          windSpeed={data.windSpeed}
        />
      );
    }
    return <></>;
  },
});

Here, the useFrontendTool hook lets the application register the get_weather tool and attach a predefined React render to each phase of its lifecycle.

Output:

The agent can invoke the tool and stream arguments, but it never controls layout or invents UI.

You can check the complete implementation at github.com/CopilotKit/generative-ui-playground.

✅ Declarative Generative UI (A2UI and Open-JSON-UI)

Declarative Generative UI sits between static and fully open-ended approaches.

Instead of selecting predefined components, the agent returns a structured UI description. This description defines things like cards, lists, forms, or widgets, and the frontend renders them.

Two common declarative specifications used here are:

1) A2UI → declarative Generative UI spec from Google, described as JSONL-based, and agents can use it to return UI widgets as part of their responses

2) Open‑JSON‑UI → open standardization of OpenAI’s internal declarative Generative UI schema

Both define structured, platform-agnostic ways for agents to describe UI in JSON.

Let's first understand the basic flow for implementing A2UI.

Instead of writing A2UI JSON by hand, you can use the A2UI Composer to generate the spec for you. Copy the output and paste it into your agent’s prompt as a reference template.

In your prompt_builder.py, pick one representative A2UI template (e.g., a simple form or list). This is what the agent learns to emit:

UI_EXAMPLES = """
---BEGIN FORM_EXAMPLE---
[
  { "beginRendering": { "surfaceId": "form-surface", "root": "form-column",
                        "styles": { "primaryColor": "#9B8AFF", "font": "Plus Jakarta Sans" } } },
  { "surfaceUpdate": {
      "surfaceId": "form-surface",
      "components": [
        { "id": "form-column", "component": { "Column": {
            "children": { "explicitList": ["form-title", "name-field", "submit-button"] }
        } } },
        { "id": "form-title", "component": { "Text": {
            "usageHint": "h2", "text": { "literalString": "Contact Us" }
        } } },
        { "id": "name-field", "component": { "TextField": {
            "label": { "literalString": "Your Name" },
            "text": { "path": "name" },
            "textFieldType": "shortText"
        } } },
        { "id": "submit-button", "component": { "Button": {
            "child": "submit-text", "primary": true,
            "action": { "name": "submit_form", "context": [
              { "key": "name", "value": { "path": "name" } }
            ] }
        } } },
        { "id": "submit-text", "component": { "Text": {
            "text": { "literalString": "Send Message" }
        } } }
      ]
    }
  },
  { "dataModelUpdate": {
      "surfaceId": "form-surface", "path": "/", "contents": [
        { "key": "name", "valueString": "" }
      ]
    }
  }
]
---END FORM_EXAMPLE---
"""

This single JSONL‑formatted block defines a “Contact Us” form in A2UI. You feed it to the agent so it learns the three message envelopes A2UI expects: surfaceUpdate (components), dataModelUpdate (state), then beginRendering (render signal).

Then add UI_EXAMPLES into the system prompt that the agent uses:

if use_ui:
    instruction = AGENT_INSTRUCTION + get_ui_prompt(self.base_url, UI_EXAMPLES)
else:
    instruction = get_text_prompt()

return LlmAgent(
    model=LiteLlm(model=LITELLM_MODEL),
    name="ui_generator_agent",
    description="Generates dynamic UI via A2UI declarative JSON.",
    instruction=instruction,
    tools=[],
)

By appending UI_EXAMPLES (and the A2UI schema) to your agent’s instruction, you ensure it streams back valid JSON based on your templates whenever the user asks for a UI.

The final step is to use createA2UIMessageRenderer, which takes the agent’s A2UI JSON and your theme, and then renderActivityMessages tells CopilotKit to use it when rendering chat messages.

import { CopilotKitProvider, CopilotSidebar } from "@copilotkitnext/react";
import { createA2UIMessageRenderer } from "@copilotkit/a2ui-renderer";
import { a2uiTheme } from "../theme";

// Instantiate the A2UI renderer once with your theme
const A2UIRenderer = createA2UIMessageRenderer({ theme: a2uiTheme });

export function A2UIPage({ children }: { children: React.ReactNode }) {
  return (
    <CopilotKitProvider
      runtimeUrl="/api/copilotkit-a2ui"
      renderActivityMessages={[A2UIRenderer]}   // ← hook in the A2UI renderer
      showDevConsole={false}
    >
      {children}
      <CopilotSidebar defaultOpen labels={{ modalHeaderTitle: "A2UI Assistant" }} />
    </CopilotKitProvider>
  );
}

Output:

The playground repo is wired for A2UI, but the pattern is the same for Open‑JSON‑UI.

An agent can respond with a JSON payload describing a UI “card,” and the frontend renders it.

// Example: Agent returns Open-JSON-UI specification
{
  type: "open-json-ui",
  spec: {
    components: [
      {
        type: "card",
        properties: {
          title: "Data Visualization",
          content: {...}
        }
      }
    ]
  }
}

Output:

You can check the specs overview at docs.copilotkit.ai/learn/generative-ui/specs/open-json-ui to understand how CopilotKit's AG-UI protocol natively supports Open-JSON-UI.

This pattern is a good fit if you need more flexibility than static UI, while still keeping clear boundaries and predictable rendering.

✅ Open-ended Generative UI (MCP Apps)

Open-ended Generative UI is the most flexible and powerful.

In this pattern, the agent returns an entire UI surface. This can be HTML, an iframe, or other free-form content. The frontend mainly acts as a container that displays whatever the agent provides.

This approach is commonly used with MCP Apps, where external applications expose their own UI that can be embedded into an agent experience.

Traditional MCP tools return text, images, resources or structured data that the host displays as part of the conversation. MCP Apps extend this pattern by allowing tools to declare a reference to an interactive UI in their tool description that the host renders in place.

The trade-offs are real. Rendering arbitrary UI introduces security and performance concerns, inconsistent styling, and harder portability outside the web.

But for complex tools and rich applications, this pattern unlocks capabilities that are impossible with static or declarative UI alone.

In CopilotKit, MCP Apps support is enabled by attaching MCPAppsMiddleware to your agent, which allows the runtime to connect to one or more MCP Apps servers.

// app/api/copilotkit/route.ts
import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { BuiltInAgent } from "@copilotkit/runtime/v2";
import { NextRequest } from "next/server";
import { MCPAppsMiddleware } from "@ag-ui/mcp-apps-middleware";

// 1. Create your agent and add the MCP Apps middleware
const agent = new BuiltInAgent({
  model: "openai/gpt-4o",
  prompt: "You are a helpful assistant.",
}).use(
   new MCPAppsMiddleware({
    mcpServers: [
      {
        type: "http",
        url: "http://localhost:3108/mcp",
        serverId: "my-server" // Recommended: stable identifier
      },
    ],
  }),
)

// 2. Create a service adapter, empty if not relevant
const serviceAdapter = new ExperimentalEmptyAdapter();

// 3. Create the runtime and add the agent
const runtime = new CopilotRuntime({
  agents: {
    default: agent,
  },
});

// 4. Create the API route
export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

Here is the GitHub Repo for the complete code implementation, and you can check the complete integration flow in the blog.

3. How AG-UI enables Generative UI at runtime

Across all three patterns, AG-UI (Agent-User Interaction Protocol) serves as the underlying runtime interaction layer in CopilotKit. It provides a bidirectional connection between the agent and the application that makes Generative UI work.

It is not a UI specification. It is an event/state protocol that defines how UI updates, agent state, and user interactions flow while the agent is working.

At a high level, AG-UI handles:

Tool lifecycles (started → streaming → finished / failed)
User interactions (clicks, form submissions, selections)
Agent state updates (progress, partial results, next steps)
Real-time coordination between agent ↔ UI ↔ application

For instance, here is the role of AG-UI protocol across Generative UI patterns:

Static → signals tool lifecycle → frontend renders predefined components
Declarative → carries UI specs + state → frontend renders structured UI
Open-ended → coordinates tools + UI references → app embeds external UI

Because AG-UI sits below specs like A2UI, Open-JSON-UI, and MCP Apps, CopilotKit can support multiple Generative UI approaches without locking you into a single approach.

In many cases, the real bottleneck is the interface that sits between the agent and the user.

Generative UI provides a practical way forward, and once you build one of these patterns yourself, it becomes clear which approach fits which problem.

I hope you learned something valuable. Have a great day!

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

Don't forget to follow CopilotKit on Twitter and join the Discord community of agent builders.

Happy building!

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

Bring MCP Apps into your OWN app with CopilotKit & AG-UI

Anmol Baranwal — Thu, 22 Jan 2026 15:06:34 +0000

Agent frameworks have become very good at calling tools and returning structured data.

The gap appears when an agent has to present information (charts, dashboards) or collect user input (forms, approvals). If the tool only returns JSON, every host ends up rebuilding the same UI logic for the same tools.

MCP Apps was proposed to solve this exact problem: a standardized way for MCP servers to ship interactive HTML UIs alongside tool outputs.

Today, we are excited to announce CopilotKit’s support for MCP Apps. Now, MCP servers can finally ship an interactive UI that works out of the box in real agent applications.

But showing UI is only step one. The hard part is keeping everything coordinated while the user is interacting:

Tool lifecycle (started → streaming → finished / failed)
User interactions (clicks, form submits, selections)
Agent state updates (progress, partial results, next steps)
Real-time back-and-forth between agent ↔ UI ↔ app

That’s why this integration is powered by AG‑UI: an event/state protocol that ships structured events back and forth (tool progress, UI actions, state updates) so the agent, the UI and your app stay synchronized while MCP Apps run.

The easiest way to get started is using the CopilotKit CLI:

npx copilotkit create -f mcp-apps

You can refer to the official docs or try the live demo if you want to explore it yourself. Now, let's understand the technical details of what is happening under the hood.

1. MCP-UI: where the idea first worked

This proposed standard is inspired by MCP-UI and OpenAI's Apps SDK.

MCP-UI was the community’s first proof that UI can be a first-class MCP resource (not just text/JSON). Tools could ship HTML rendered in sandboxed iframes and hosts could display rich, interactive content instead of rebuilding the UI for every tool.

MCP Apps take the core idea behind MCP-UI and formalize it into a shared, interoperable spec.

SEP-1865 was authored by MCP core maintainers at OpenAI and Anthropic, together with MCP‑UI creators and lead maintainers of the MCP-UI Community Working Group.

You can review the full SEP at modelcontextprotocol/ext-apps.

2. MCP Apps: standardizing UI within ecosystem

MCP Apps doesn’t just say “tools can return UI.” It standardizes how a server exposes UI in a way that hosts can reliably discover, render and interact with.

Credit: MCP Apps post

At a high level, it introduces a few core pieces:

MCP server declares UI resources (using the ui://... URI scheme).
Tools reference UI resources through the _meta field so the host knows what to render.
Hosts can prefetch and review templates before running tools
UI runs in sandboxed iframes for security
All UI ↔ host communication happens using the standard MCP JSON-RPC over postMessage.

A simplified example from a tool response might look like this:

{
  "result": {
    "data": {
      "revenue": [280, 400, 560]
    }
  },
  "_meta": {
    "ui/resourceUri": "ui://charts/revenue"
  }
}

On the server side, that UI is declared explicitly:

{
  "ui://charts/revenue": {
    "type": "html",
    "entrypoint": "/ui/revenue-chart.html"
  }
}

The point is: a server can ship a UI once and multiple hosts can render it consistently because the “resource + link + message channel” pattern is standardized.

What's still missing?

MCP Apps defines the UI surface and a standard way for the UI to talk to the host, but it doesn’t tell you how to keep agent state, tool lifecycle and UI events synchronized in real-time.

That orchestration layer is where AG‑UI protocol fits in.

3. AG‑UI: the sync layer that makes it usable

You still need a consistent way to keep agent ↔ UI ↔ app aligned while the user interacts.

AG-UI (Agent–User Interaction Protocol) is an event-based protocol that standardizes this “agent ↔ frontend” real-time communication.

Because it’s transport-agnostic (SSE, WebSockets, etc.), the same frontend can work across different agent backends without custom wiring.

CopilotKit uses AG-UI as its synchronization layer, which makes it a natural fit for running interactive MCP Apps.

What it looks like (events on the wire)

When an MCP App UI is on screen, AG‑UI is the stream CopilotKit (or any compatible agent framework) uses to keep tool progress and shared state synced while the user interacts.

When the agent starts a tool call, you will see lifecycle events like:

{ "type": "TOOL_CALL_START", "toolCallId": "tool_1", "toolCallName": "get_revenue_data" }

As the agent/frontend state changes, you will see state snapshots like:

{ "type": "STATE_SNAPSHOT", "snapshot": { "status": "loading" } }

Followed by incremental updates (such as JSON Patch deltas) as the workflow progresses:

{
  "type": "STATE_DELTA",
  "delta": [{ "op": "replace", "path": "/status", "value": "ready" }]
}

STATE_DELTA uses JSON Patch so the UI can update without re-sending the entire state each time.

This is how it works in practice.

4. How CopilotKit wires MCP Apps ↔ AG‑UI

CopilotKit acts as the runtime that binds MCP Apps and AG-UI together inside a real application. Here is the simple flow:

The agent calls an MCP tool and receives structured output
The tool response includes a ui://... reference, indicating which UI to render
CopilotKit renders that UI inside your app using a sandboxed iframe
UI sends interactions back and CopilotKit forwards them into the agent loop
AG-UI streams state/progress updates back to the UI while the agent runs.

Agent  
│  
│ calls tool  
▼  
MCP Tool ──────► returns data + ui:// reference  
│  
▼  
CopilotKit (runtime)  
│  
├─ renders MCP App UI (iframe)  
│  
├─ forwards UI messages → agent  
│  
└─ participates in AG-UI event stream  
│ (state / progress updates)  
▼  
MCP App UI

MCP Apps provide the UI surface, AG-UI provides the sync contract and CopilotKit runs the loop.

5. Integration flow: MCP Apps with CopilotKit

The fastest way to get started is to use the CopilotKit CLI:

npx copilotkit create -f mcp-apps

It creates a starter repo, which shows an end-to-end MCP Apps integration with CopilotKit and uses the official Three.js MCP Server.

If you are integrating into an existing Next.js app, here's how you can set things up manually.

To run MCP Apps inside a real agent, you need a runtime layer that can fetch UI resources, sandbox them and keep UI state in sync with the agent.

That’s what the MCPAppsMiddleware provides. Install it along with the required CopilotKit packages:

npm install @copilotkit/react-ui @copilotkit/react-core @copilotkit/runtime @ag-ui/mcp-apps-middleware

Next, configure the CopilotKit API route. This is where the agent is created, the MCP Apps middleware is attached and the CopilotKit runtime is exposed.

BuiltInAgent(...) : defines the agent’s LLM (model) and behavior (prompt)
mcpServers : the list of MCP servers to connect to (you can add multiple)
serverId : stable name for that server so CopilotKit can reload the same MCP App UI later, even if the server URL changes (local, staging, prod)

// app/api/copilotkit/route.ts
import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { BuiltInAgent } from "@copilotkit/runtime/v2";
import { NextRequest } from "next/server";
import { MCPAppsMiddleware } from "@ag-ui/mcp-apps-middleware";

// 1. Create your agent and add the MCP Apps middleware
const agent = new BuiltInAgent({
  model: "openai/gpt-4o",
  prompt: "You are a helpful assistant.",
}).use(
   new MCPAppsMiddleware({
    mcpServers: [
      {
        type: "http",
        url: "http://localhost:3108/mcp",
        serverId: "my-server" // Recommended: stable identifier
      },
    ],
  }),
)

// 2. Create a service adapter, empty if not relevant
const serviceAdapter = new ExperimentalEmptyAdapter();

// 3. Create the runtime and add the agent
const runtime = new CopilotRuntime({
  agents: {
    default: agent,
  },
});

// 4. Create the API route
export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

This will allow the agent to understand ui:// references returned by MCP tools and resolve them into rendered UI.

Create a .env.local file in your frontend directory and add your API key. The example uses OpenAI’s GPT-4o by default but you can also configure other language models.

OPENAI_API_KEY=your_openai_api_key

Wrap your application with the CopilotKit provider so the frontend can connect to the runtime (layout.tsx).

import { CopilotKit } from "@copilotkit/react-core";
import "@copilotkit/react-ui/styles.css";

// ...

export default function RootLayout({ children }: {children: React.ReactNode}) {
    return (
        <html lang="en">
            <body>
                <CopilotKit runtimeUrl="/api/copilotkit">
                    {children}
                </CopilotKit>
            </body>
        </html>
    );
}

Now add the CopilotSidebar chat component to your page.

// app/page.tsx
"use client";

import { CopilotSidebar } from "@copilotkit/react-ui";

export default function Page() {
    return (
        <main>
            <h1>Your App</h1>
            <CopilotSidebar />
        </main>
    );
}

That's it! MCP Apps will now render automatically whenever the agent uses tools that reference UI resources.

The middleware supports both HTTP and SSE transports. You can find the full config options in the official docs.

You can try some of the open source MCP Apps servers from the official repo at github.com/modelcontextprotocol/ext-apps.

What this enables for Agent Builders

This brings interactive UIs into MCP in a way that works across hosts. You can:

Ship interactive tool UIs without rebuilding them per host
Keep UI + agent state in sync without custom wiring
Go from prototype to production with fewer rewrites

You can even collect structured input and include human-in-the-loop steps inside your app.

5. Getting started

To start building:

👉 Start with CopilotKit & MCP Apps: npx copilotkit create -f mcp-apps

👉 Check out the CopilotKit / MCP Apps docs to get started

👉 Check out our example repo for a minimal integration setup

👉 Try the live CopilotKit demo and interact with an MCP App in real time

What's Next

AG-UI is shaping up to be the bridge. Over the next few weeks, expect more examples, tighter primitives and clearer patterns for this integration.

If you want to follow along (or help shape what ships next), join the CopilotKit or AG-UI communities.

There’s a lot more coming soon.

How to build a Frontend for LangChain Deep Agents with CopilotKit!

Anmol Baranwal — Tue, 20 Jan 2026 14:22:27 +0000

LangChain recently introduced Deep Agents: a new way to build structured, multi-agent systems that can plan, delegate, and reason across multiple steps.

It comes with built-in planning, a filesystem for context, and subagent spawning. But connecting that agent to a real frontend is still surprisingly hard.

Today, we will build a Deep Agents powered job search assistant and connect it to a live Next.js UI with CopilotKit, so the frontend stays in sync with the agent in real time.

You will find architecture, the key patterns, how state flows between the UI ↔ agent and the step-by-step guide to building this from scratch.

Let's build it.

Check out CopilotKit's GitHub ⭐️

1. What are Deep Agents?

Most agents today are just “LLM in a loop + tools”. That works, but it tends to be shallow: no explicit plan, weak long-horizon execution, and messy state as runs get longer.

Popular agents like Claude Code, Deep Research, and Manus get around this by following a common pattern: they plan first, externalize working context (often via files or a shell), and delegate isolated pieces of work to sub-agents.

Deep Agents package those primitives into a reusable agent runtime.

Credit: LangChain

At a practical level, a Deep Agent created via create_deep_agent is just a LangGraph graph. There’s no separate runtime or hidden orchestration layer.

That means standard LangGraph features work as-is:

streaming
checkpoints and interrupts
human-in-the-loop controls

The mental model (how it runs)

Conceptually, the execution flow looks like this:

User goal
  ↓
Deep Agent (LangGraph StateGraph)
  ├─ Plan: write_todos → updates "todos" in state
  ├─ Delegate: task(...) → runs a subagent with its own tool loop
  ├─ Context: ls/read_file/write_file/edit_file → persists working notes/artifacts
  ↓
Final answer

That gives you a usable structure for “plan → do work → store intermediate artifacts → continue” without inventing your own plan format, memory layer or delegation protocol.

You can read more at blog.langchain.com/deep-agents and check official docs.

Where CopilotKit Fits

Deep Agents push key parts into explicit state (e.g. todos + files + messages), which makes runs easier to inspect. That explicit state is also what makes Copilotkit integration possible.

CopilotKit is a frontend runtime that keeps UI state in sync with agent execution by streaming agent events and state updates in real time (using AG-UI under the hood).

This middleware (CopilotKitMiddleware) is what allows the frontend to stay in lock-step with the agent as it runs. You can read the docs at docs.copilotkit.ai/langgraph/deep-agents.

agent = create_deep_agent(
    model="openai:gpt-4o",
    tools=[get_weather],
    middleware=[CopilotKitMiddleware()], # for frontend tools and context
    system_prompt="You are a helpful research assistant."
)

The diagram below shows how a user action in the UI is sent via AG-UI to any agent backend and responses flow back as standardized events.

2. Core Components

Here are the core components that we will be using later on:

1) Planning Tools (built-in via Deep Agents) - built-in planning/to‑do behavior so the agent can break the workflow into steps without you writing a separate planning tool.

# Conceptual example (not required in codebase)
@tool
def todo_write(tasks: List[str]) -> str:
    formatted = "\n".join([f"- {task}" for task in tasks])
    return f"Todo list created:\n{formatted}"

2) Subagents - let the main agent delegate focused tasks into isolated execution loops. Each sub-agent has its own prompt, tools and context.

subagents = [
    {
        "name": "job-search-agent",
        "description": "Finds relevant jobs and outputs structured job candidates.",
        "system_prompt": JOB_SEARCH_PROMPT,
        "tools": [internet_search],
    }
]

3) Tools - this is how the agent actually does things. Here, finalize() signals completion.

@tool
def finalize() -> dict:
    """Signal that the agent is done."""
    return {"status": "done"}

How Deep Agents are implemented (Middleware)

If you are wondering how create_deep_agent() actually injects planning, files and subagents into a normal LangGraph agent, the answer is middleware.

Each feature is implemented as a separate middleware. By default, three are attached:

To-do list middleware - adds the write_todos tool and instructions that push the agent to explicitly plan and update a live todo list during multi-step tasks.
Filesystem middleware - adds file tools (ls, read_file, write_file, edit_file) so the agent can externalize notes and artifacts instead of stuffing everything into chat history.
Subagent middleware - adds the task tool, allowing the main agent to delegate work to subagents with isolated context and their own prompts/tools.

This is what makes Deep Agents feel “pre-wired” without introducing a new runtime. If you want to go deeper, the linked middleware docs show the exact implementation details.

What are we building?

Let's create an agent that:

Accepts a resume (PDF) and extracts skills + context
Uses Deep Agents to plan and orchestrate sub-agents
Searches the web for relevant jobs using tools (Tavily)
Streams tool results back to the UI via CopilotKit (AG-UI)

We will see some of these concepts in action as we build the agent.

3. Frontend: wiring the agent to the UI

Let's first build the frontend part. This is how our directory will look.

The src directory hosts the Next.js frontend, including the UI, shared components and the CopilotKit API route (/api/copilotkit) used for agent communication.

.
├── src/                               ← Next.js frontend
│   ├── app/
│   │   ├── page.tsx                      
│   │   ├── layout.tsx                 ← CopilotKit provider
│   │   └── api/
│   │       ├── upload-resume/route.ts ← upload endpoint
│   │       └── copilotkit/route.ts    ← CopilotKit AG-UI runtime
│   ├── components/
│   │   ├── ChatPanel.tsx              ← Chat + tool capture
│   │   ├── ResumeUpload.tsx           ← PDF upload UI
│   │   ├── JobsResults.tsx            ← Jobs table renderer
│   │   └── LivePreviewPanel.tsx          
│   └── lib/
│       └── types.ts   
├── package.json                     
├── next.config.ts                   
└── README.md

Step 1: CopilotKit Provider & Layout

Install the necessary CopilotKit packages.

npm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

@copilotkit/react-core provides the core React hooks and context that connect your UI to an AG-UI compatible agent backend.
@copilotkit/react-ui offers ready-made UI components like <CopilotChat /> to build AI chat or assistant interfaces quickly.
@copilotkit/runtime is the server-side runtime that exposes an API endpoint and bridges the frontend with an external AG-UI compatible agent backend using HTTP and SSE.

The <CopilotKit> component must wrap the Copilot-aware parts of your application. In most cases, it's best to place it around the entire app, like in layout.tsx.

import type { Metadata } from "next";

import { CopilotKit } from "@copilotkit/react-core";
import "./globals.css";
import "@copilotkit/react-ui/styles.css";

export const metadata: Metadata = {
  title: "Job Finder | Deep Agents with CopilotKit",
  description: "A job search assistant powered by Deep Agents and CopilotKit",
};

export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  return (
    <html lang="en">
      <body className={"antialiased"}>
        <CopilotKit runtimeUrl="/api/copilotkit" agent="job_application_assistant">
          {children}
        </CopilotKit>
      </body>
    </html>
  );
}

Here, runtimeUrl="/api/copilotkit" points to the Next.js API route CopilotKit uses to talk to the agent backend.

Each page is wrapped in this context so UI components know which agent to invoke and where to send requests.

Step 2: Next.js API Route: Proxy to FastAPI

This Next.js API route acts as a thin proxy between the browser and the Deep Agents. It:

Accepts CopilotKit requests from the UI
Forwards them to the agent over AG-UI
Streams agent state and events back to the frontend

Instead of letting the frontend talk to the FastAPI agent directly, all requests go through a single endpoint /api/copilotkit.

import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { LangGraphHttpAgent } from "@copilotkit/runtime/langgraph";
import { NextRequest } from "next/server";

const serviceAdapter = new ExperimentalEmptyAdapter();

const runtime = new CopilotRuntime({
  agents: {
    job_application_assistant: new LangGraphHttpAgent({
      url: process.env.LANGGRAPH_DEPLOYMENT_URL || "http://localhost:8123",
    }),
  },
});

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

Here's a simple explanation of the above code:

The code above registers the job_application_assistant agent.
LangGraphHttpAgent : defines a remote LangGraph agent endpoint. It points to the Deep Agents backend running on FastAPI.
ExperimentalEmptyAdapter : simple no-op adapter used when the agent backend handles its own LLM calls and orchestration
copilotRuntimeNextJSAppRouterEndpoint : small helper that adapts the Copilot runtime to a Next.js App Router API route and returns a handleRequest function

Step 3: Resume upload API endpoint

This API route (src\app\api\upload-resume\route.ts) handles resume uploads from the frontend and forwards them to the FastAPI backend. It:

Accepts multipart file uploads from the browser
Proxies the file to the backend resume parser
Returns extracted text and skills to the UI

Keeping resume parsing in the backend lets the agent reuse the same logic and keeps the frontend lightweight.

import { NextRequest, NextResponse } from "next/server";

export async function POST(req: NextRequest) {
  try {
    const formData = await req.formData();
    const file = formData.get("file") as File;

    if (!file) {
      return NextResponse.json({ error: "No file provided" }, { status: 400 });
    }

    const backendFormData = new FormData();
    backendFormData.append("file", file);

    const backendUrl = process.env.BACKEND_URL || "http://localhost:8123";
    const response = await fetch(`${backendUrl}/api/upload-resume`, {
      method: "POST",
      body: backendFormData,
    });

    if (!response.ok) {
      throw new Error("Backend upload failed");
    }

    const data = await response.json();
    return NextResponse.json(data);
  } catch (error) {
    return NextResponse.json(
      { error: error instanceof Error ? error.message : "Upload failed" },
      { status: 500 }
    );
  }
}

Step 4: Building Key Components

I'm only covering the core logic behind each component since the overall code is huge. You can find all the components in the repository at src\components.

These components use CopilotKit hooks (like useCopilotReadable) to tie everything together.

✅ Resume Upload Component

This client component handles resume selection and forwards the file to the backend for parsing.

It accepts a PDF/TXT file, POSTs it to /api/upload-resume and lifts the extracted text and skills back up to the parent component.

"use client";
import { useRef, useState } from "react";

type ResumeUploadResponse = { success: boolean; text: string; skills: string[]; filename: string };

export function ResumeUpload({ onUploadSuccess }: { onUploadSuccess(d: ResumeUploadResponse): void }) {
  const [selectedFile, setSelectedFile] = useState<File | null>(null);
  const [isLoading, setIsLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const inputRef = useRef<HTMLInputElement>(null);

  const onSelect = (e: React.ChangeEvent<HTMLInputElement>) => {
    setError(null);
    const f = e.target.files?.[0] ?? null;
    if (f && !["application/pdf", "text/plain"].includes(f.type)) {
      setSelectedFile(null);
      setError("Please upload a PDF or TXT file");
      e.target.value = ""; // allow re-selecting same file
      return;
    }
    setSelectedFile(f);
  };

  const onSubmit = async (e: React.FormEvent) => {
    e.preventDefault();
    if (!selectedFile) return;

    setIsLoading(true);
    setError(null);

    try {
      const fd = new FormData();
      fd.append("file", selectedFile);

      const res = await fetch("/api/upload-resume", { method: "POST", body: fd });
      if (!res.ok) throw new Error("Upload failed");

      onUploadSuccess((await res.json()) as ResumeUploadResponse);

      setSelectedFile(null);
      if (inputRef.current) inputRef.current.value = "";
    } catch (err) {
      setError(err instanceof Error ? err.message : "Failed to upload resume");
    } finally {
      setIsLoading(false);
    }
  };

  return (
    <form onSubmit={onSubmit}>
      <input ref={inputRef} type="file" accept=".pdf,.txt" onChange={onSelect} />
      <button disabled={!selectedFile || isLoading}>{isLoading ? "Uploading..." : "Upload Resume"}</button>
      {error && <p>{error}</p>}
      {/* ... UI/styling omitted ... */}
    </form>
  );
}

Here's a brief explanation:

Accepts a PDF/TXT file from the user
Sends the file to /api/upload-resume using FormData
Receives extracted text + skills from the backend
Lifts that data via onUploadSuccess so it can be injected into the agent later

Check out the complete code at src/components/ResumeUpload.tsx.

✅ Chat Panel Component

This is the core UI that connects the user, the agent, and the tool outputs. The Chat Panel:

Embeds CopilotChat to handle conversational input and streaming agent responses
Uses useCopilotReadable to continuously sync resume text, detected skills and user preferences into the agent’s context
Intercepts tool calls (like update_jobs_list) to update local UI state without dumping job JSON into chat

We also build the conversational UI using <CopilotChat /> in this component.

"use client";

import { useState, useRef } from "react";
import { useDefaultTool, useCopilotReadable } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui";
import { ResumeUpload } from "./ResumeUpload";
import { JobsResults } from "./JobsResults";

export function ChatPanel() {
  // form + resume state (title, location, skills, resume text…)
  const [jobs, setJobs] = useState<JobPosting[]>([]);
  const processedKeyRef = useRef<string | null>(null); // dedupe repeated tool calls

  // Capture tool output
  useDefaultTool({
    render: ({ name, status, args, result }) => {
      if (name === "update_jobs_list" && status === "complete" && result?.jobs_list) {
        const key = JSON.stringify({
          len: result.jobs_list.length,
          first: result.jobs_list[0]?.url,
        });

        if (processedKeyRef.current !== key) {
          processedKeyRef.current = key;

          // Avoid setState during render
          queueMicrotask(() => {
            setJobs(result.jobs_list);
          });
        }
      }

      // Render tool calls inline
      return (
        <details>
          ...
        </details>
      );
    },
  });

  // Send UI state + resume data into agent context
  useCopilotReadable({
    description: "Job search preferences",
    value: {
      targetTitle,
      targetLocation,
      skillsHint,
      resumeText,
      detectedSkills,
    },
  });

  return (
    <div>
      {/* Resume upload + extracted skills UI */}
      {!resumeUploaded && <ResumeUpload onUploadSuccess={handleUploadSuccess} />}

      {/* Job search inputs (title / location / skills) */}

      {/* CopilotKit chat UI */}
      <CopilotChat />

      {/* Structured output rendered outside chat */}
      <JobsResults jobs={jobs} />
    </div>
  );
}

Check out the complete code at src/components/ChatPanel.tsx.

✅ Jobs Results Component

This is a pure presentational component. It receives the jobs array (populated when update_jobs_list completes) and renders it as a table, keeping the chat output clean.

"use client";
import { JobPosting } from "@/lib/types";

export function JobsResults({ jobs }: { jobs: JobPosting[] }) {
  if (!jobs.length) return null;

  return (
    <div className="mt-4 bg-white border border-slate-200 rounded-lg shadow-sm overflow-hidden">
      <div className="px-4 py-3 border-b border-slate-200">
        <h3 className="font-semibold text-slate-900">Jobs</h3>
      </div>

      <div className="overflow-x-auto">
        <table className="w-full text-sm">
          <thead>{/* Company | Title | Location | Link | Good match */}</thead>
          <tbody>
            {jobs.map((j, idx) => (
              <tr key={idx} className="border-t border-slate-100 text-black">
                <td className="px-4 py-2">{j.company}</td>
                <td className="px-4 py-2">{j.title}</td>
                <td className="px-4 py-2">{j.location}</td>
                <td className="px-4 py-2">
                  <a className="text-blue-600 hover:underline" href={j.url} target="_blank" rel="noreferrer">
                    Open
                  </a>
                </td>
                <td className="px-4 py-2">{j.goodMatch || "Yes"}</td>
              </tr>
            ))}
          </tbody>
        </table>
      </div>
    </div>
  );
}

Check out the complete code at src/components/JobsResults.tsx.

Step 5: Connecting the Chat UI to the Agent

At this point, all the pieces are already in place. This page simply renders the ChatPanel, which is fully wired to the Deep Agents backend via CopilotKit.

A secondary LivePreviewPanel is mounted alongside it. Since tool calls are already rendered inline inside CopilotChat, this panel is optional for now and acts as a work-in-progress space for richer debugging and visualization.

"use client";

import { ChatPanel } from "@/components/ChatPanel";
import { LivePreviewPanel } from "@/components/LivePreviewPanel";

export default function Page() {
  return (
    <main className="min-h-screen flex flex-col">
      {/* App header (branding + description) */}
      <header>
        <h1>Job Application Assistant</h1>
        <p>Find personalized jobs with AI.</p>
        {/* ... badges / styling omitted ... */}
      </header>

      <div className="grid lg:grid-cols-3 gap-6">
        <section className="lg:col-span-2">
          <ChatPanel />
        </section>

        <aside className="lg:col-span-1">
          <LivePreviewPanel />
          {/* Tool calls are already rendered inside CopilotChat */}
          {/* This panel is optional and currently used for experimentation */}
        </aside>
      </div>

      {/* Footer */}
      {/* ... footer content omitted ... */}
    </main>
  );
}

4. Backend: Building the Agent Service (FastAPI + Deep Agents + AG-UI)

We will now build the FastAPI backend that hosts our Deep Agent.

Under the /agent directory lives a FastAPI server that runs the Job Application agent. Here's the project structure of the backend.

.
├── agent/                             ← Deep Agents backend
│   ├── main.py                        ← FastAPI + AG-UI endpoint
│   ├── agent.py                       ← Deep Agents graph & tools
│   ├── pyproject.toml                 ← Python deps (uv)
│   └── uv.lock
...

At a high level, the backend is responsible for:

Exposes a CopilotKit-compatible agent endpoint (for streaming agent state and tool calls)
Provides a /api/upload-resume endpoint for parsing resumes
Constructs a Deep Agents graph that plans, delegates to sub-agents, and searches the web for matching jobs

The backend uses uv for dependency management. Install it if you don't have it in your system.

pip install uv

Initialize a new uv project using the following command. This will generate a fresh pyproject.toml.

cd agent
uv init

Most of the AI tooling used in this backend (especially AG-UI Strands) currently expects Python 3.12+, so make sure to tell uv to use a compatible Python version by using this command:

uv python pin 3.12

Then install the dependencies. This will also create the project’s virtual environment.

uv add copilotkit deepagents fastapi langchain langchain-openai pypdf python-dotenv python-multipart tavily-python "uvicorn[standard]"

copilotkit : connects agents to a frontend with streaming, tools, and shared state.
deepagents : planning-first agent framework for multi-step execution.
fastapi : web framework that exposes the agent API.
langchain : agent and tool orchestration layer.
langchain-openai : OpenAI model integration for LangChain.
pypdf : extracts text from PDF files.
python-dotenv : loads environment variables from .env
python-multipart : enables file uploads in FastAPI.
tavily-python : web search for real-time agent research.
uvicorn[standard] : ASGI server to run FastAPI.

Now run the following command to generate a uv.lock file pinned with exact versions.

uv sync

Add necessary API Keys

Create a .env file under both the agent directory and add your OpenAI API Key and Tavily API Key to the file. I have attached the docs link so it's easy to follow.

OPENAI_API_KEY=sk-proj-...
TAVILY_API_KEY=tvly-dev-...
OPENAI_MODEL=gpt-4-turbo

OpenAI API Key

Tavily API Key

Step 1: Define the agent’s behavior

We start by defining the agent’s behavior using a single, strict system prompt in agent.py.

In Deep Agents, the system prompt acts as the control layer for the workflow, combining planning and delegation to decompose complex tasks into ordered steps.

The MAIN_SYSTEM_PROMPT coordinates tools and sub-agents by enforcing a fixed execution sequence. This prompt ensures:

external actions always happen via tools
UI state is updated in a controlled way
execution ends deterministically with finalize()

MAIN_SYSTEM_PROMPT = """
You are a tool-using agent.

Hard rules:
- Never include job details, URLs, or JSON in assistant messages.
- Only output jobs via update_jobs_list(jobs_json).
- A valid job must be a single job detail page on an ATS or company careers page.
- Do NOT use job boards or listing/search pages.
- company MUST be the hiring company (never Lever/Greenhouse/Ashby/Workday/Talent.com/etc).

Schema (exact keys):
- company, title, location, url, goodMatch

Steps:
1) Call internet_search(query) exactly once.
2) From the returned results, select up to 5 valid individual job postings.
3) Call update_jobs_list(jobs_json) once.
4) Call finalize().
5) Output: Found N jobs.

If you cannot find 5 valid jobs, return as many valid ones as possible.
"""

JOB_SEARCH_PROMPT defines the behavior of a specialized sub-agent. Its responsibility is limited to finding relevant jobs and returning structured results in a controlled format.

JOB_SEARCH_PROMPT = (
    "Search and select 5 real postings that match the user's title, locations, and skills. "
    "Output ONLY this block format (no extra text before/after the wrapper):\n"
    "<JOBS>\n"
    '[{"company":"...","title":"...","location":"...","link":"https://...","Good Match":"one sentence"},'
    ' {"company":"...","title":"...","location":"...","link":"https://...","Good Match":"one sentence"},'
    ' {"company":"...","title":"...","location":"...","link":"https://...","Good Match":"one sentence"},'
    ' {"company":"...","title":"...","location":"...","link":"https://...","Good Match":"one sentence"},'
    ' {"company":"...","title":"...","location":"...","link":"https://...","Good Match":"one sentence"}]'
    "\n</JOBS>"
    "Each job MUST:"
    "- Be a single opening (not a job board, filter page or company jobs index)"
    "- Belong to a specific company with a dedicated job description page"
    "You must:"
    "- Use internet_search to find relevant jobs."
    "- Do NOT output job listings, JSON, or URLs in messages."
    "- Return everything ONLY by calling the parent tool `update_jobs_list` with a JSON string."
)

Step 2: Add resume parsing and skill extraction utilities

We extract raw text from uploaded PDFs using pypdf. This function is used by the FastAPI upload endpoint to turn resumes into plain text.

def parse_pdf_resume(file_path: str) -> str:
    with open(file_path, "rb") as file:
        reader = PdfReader(file)
        return "".join(page.extract_text() for page in reader.pages)

Next, we extract lightweight structured signals (languages, frameworks, tools) from the resume. This influences job search queries and match quality.

def extract_skills_from_resume(resume_text: str) -> List[str]:
    skills_db = {
        "languages": ["Python", "JavaScript", "Go"],
        "frameworks": ["React", "FastAPI", "Django"],
        "cloud": ["AWS", "Docker", "Kubernetes"],
    }

    found = set()
    text = resume_text.lower()

    for skills in skills_db.values():
        for skill in skills:
            if skill.lower() in text:
                found.add(skill)

    return list(found)

Step 3: Define tools for search, UI updates, and termination

Tools are the integration surface between the agent and the external world / UI.

The internet_search tool is responsible for discovering real job postings. It intentionally fetches extra search results, filters out any URL containing a “bad” substring (job boards/search pages) via BAD_URL_SUBSTRINGS and returns only the first max_results clean hits.

BAD_URL_SUBSTRINGS = [
    "linkedin.com/jobs/search",
    "linkedin.com/jobs/",
    "builtin.com/jobs",
    "naukri.com",
    "glassdoor.",
    "/jobs/search",
    "/search?",
]


def _is_bad(url: str) -> bool:
    u = (url or "").lower()
    return any(p in u for p in BAD_URL_SUBSTRINGS)


@tool
def internet_search(query: str, max_results: int = 10) -> List[Dict[str, Any]]:
    """
    Search for jobs using Tavily API. Always returns up to 5 results.
    """
    tavily_key = os.environ.get("TAVILY_API_KEY")
    if not tavily_key:
        raise RuntimeError("TAVILY_API_KEY not set")

    client = TavilyClient(api_key=tavily_key)
    res = client.search(
        query=query,
        max_results=max_results * 3,  # get more, then filter
        include_raw_content=False,
        topic="general",
    )

    trimmed = []
    for r in res.get("results", []):
        url = r.get("url") or ""
        if _is_bad(url):
            continue
        trimmed.append(
            {
                "title": r.get("title"),
                "url": url,
                "content": (r.get("content") or "")[:400],
            }
        )
        if len(trimmed) == max_results:
            break

    print(f"[SEARCH] Returning {len(trimmed)} filtered results")
    print(trimmed)
    return trimmed

The update_jobs_list tool is the only way structured job data reaches the frontend, keeping UI updates explicit and JSON out of chat messages.

@tool
def update_jobs_list(jobs_json: str) -> Dict[str, Any]:
    """Send jobs list to UI state."""
    jobs = json.loads(jobs_json)
    print(f"[TOOL] update_jobs_list: {len(jobs)} jobs")
    return {"jobs_list": jobs}

finalize tool signals that the agent has completed its workflow.

@tool
def finalize() -> dict:
    """Signal completion."""
    print("[TOOL] finalize: Job search complete")
    return {"status": "done"}

Step 4: Assemble the Deep Agents graph with sub-agents

Now we connect everything and build the Deep Agents graph with build_agent().

def build_agent():
    """Build Deep Agents graph with proper recursion limit"""
    api_key = os.environ.get("OPENAI_API_KEY")
    if not api_key:
        raise RuntimeError("Missing OPENAI_API_KEY")

    llm = ChatOpenAI(
        model=os.environ.get("OPENAI_MODEL", "gpt-4-turbo"),
        temperature=0.7,
        api_key=api_key,
    )

    tools = [
        internet_search,
        update_jobs_list,
        finalize,
    ]

    subagents = [
        {
            "name": "job-search-agent",
            "description": "Finds relevant jobs and outputs <JOBS> JSON.",
            "system_prompt": JOB_SEARCH_PROMPT,
            "tools": [internet_search],
        },
    ]

    agent_graph = create_deep_agent(
        model=llm,
        system_prompt=MAIN_SYSTEM_PROMPT,
        tools=tools,
        subagents=subagents,
        middleware=[CopilotKitMiddleware()],
        checkpointer=MemorySaver(),
    )

    print("[AGENT] Deep Agents graph created")
    print(agent_graph)

    return agent_graph

Step 5: FastAPI setup

The last step is to initialize the backend and expose it as a FastAPI app. It also handles resume uploads and PDF parsing, turning raw files into clean text and skills before they are ever sent to the agent.

import os
from fastapi import FastAPI, HTTPException, File, UploadFile
import uvicorn
from dotenv import load_dotenv
from ag_ui_langgraph import add_langgraph_fastapi_endpoint
from copilotkit import LangGraphAGUIAgent
from agent import build_agent, parse_pdf_resume, extract_skills_from_resume
import tempfile

load_dotenv()

app = FastAPI(
    title="Job Application Assistant",
    description="Find personalized job openings based on skills and preferences",
    version="1.0.0",
)

try:
    agent_graph = build_agent()
    print(agent_graph)
    add_langgraph_fastapi_endpoint(
        app=app,
        agent=LangGraphAGUIAgent(
            name="job_application_assistant",
            description="Job finder",
            graph=agent_graph,
        ),
        path="/",
    )
    print("[MAIN] Agent registered")
except Exception as e:
    print(f"[ERROR] Failed to build agent: {str(e)}")
    raise


@app.get("/healthz")
async def health_check():
    """Health check"""
    return {
        "status": "healthy",
        "service": "job-application-assistant",
        "version": "1.0.0",
    }


@app.post("/api/upload-resume")
async def upload_resume(file: UploadFile = File(...)):
    """
    Upload and parse resume (PDF, DOCX, TXT).
    Returns extracted text and skills.
    """
    if not file:
        raise HTTPException(status_code=400, detail="No file provided")

    try:
        with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp:
            content = await file.read()
            tmp.write(content)
            tmp_path = tmp.name

        if file.filename.endswith(".pdf"):
            resume_text = parse_pdf_resume(tmp_path)
        else:
            # for other formats, just read as text
            resume_text = content.decode("utf-8", errors="ignore")

        skills = extract_skills_from_resume(resume_text)

        os.unlink(tmp_path)

        return {
            "success": True,
            "text": resume_text[:1000],
            "skills": skills,
            "filename": file.filename,
        }

    except Exception as e:
        print(f"[ERROR] Resume upload failed: {str(e)}")
        raise HTTPException(status_code=500, detail=str(e))


def main():
    """Run server"""
    host = os.getenv("SERVER_HOST", "0.0.0.0")
    port = int(os.getenv("SERVER_PORT", 8123))

    uvicorn.run(
        "main:app",
        host=host,
        port=port,
        reload=True,
        log_level="info",
    )


if __name__ == "__main__":
    main()

5. Running the Application

After completing all the parts of the code, it's time to run it locally. Please make sure you have added the credentials to the agent/.env.

From the project root, navigate to the agent directory and start the FastAPI server:

cd agent
uv run python main.py

The backend will start on http://localhost:8123.

In a new terminal window, start the frontend development server using:

npm run dev

Once both servers are running, open the frontend in your browser at http://localhost:3000/ to view it locally.

You then upload your resume and search for a job query.

Based on the job query, it can fetch a different number of results. Here is another output!

6. Data flow

Now that we have built both the frontend and the agent service, this is how data actually flows between them. This should be easy to follow if you have been building along so far.

[User uploads resume & submits job query]
        ↓
Next.js UI (ResumeUpload + CopilotChat)
        ↓
useCopilotReadable syncs resume + preferences
        ↓
POST /api/copilotkit (AG-UI protocol)
        ↓
FastAPI + Deep Agents (/copilotkit endpoint)
        ↓
Resume context + skills injected into the agent
        ↓
Deep Agents orchestration
   ├─ internet_search (Tavily)
   ├─ job filtering & normalization
   └─ update_jobs_list (tool call)
        ↓
AG-UI streaming (SSE)
        ↓
CopilotKit runtime receives the tool result
        ↓
Frontend captures the tool output
        ↓
Jobs rendered in table + chat stay clean

That’s it! 🎉

You now have a Deep Agents powered job application assistant with CopilotKit as the frontend layer.

I hope you learned something valuable. Have a great day!

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

Build with Google's new A2UI Spec: Agent User Interfaces with A2UI + AG-UI

Bonnie — Tue, 16 Dec 2025 15:24:45 +0000

TL;DR

In this guide, you will learn how to build full-stack agent-to-user interface (A2UI) agents using the agent-to-agent (A2A) protocol, the AG-UI protocol, and CopilotKit.

Before we jump in, I would also like to cover how A2UI & AG-UI fit together. You can read about it here

For this build, we'll cover:

What is A2UI?
Building A2UI + A2A agent backend
Building A2UI + A2A agent frontend using AG-UI and CopilotKit

Here is a preview of what we will be building:

What is A2UI?

A2UI is a new open-source UI Toolkit to facilitate LLM-generated UIs that allows AI agents to create and present dynamic, interactive user interfaces (UIs) on the fly, rather than relying on a fixed, pre-built interface.

A2UI is built with the A2A protocol and allows an A2A agent to send interactive components instead of just text, using a high-level framework-agnostic format that can be rendered natively on any surface

In simpler terms, instead of the AI agent only responding with text in a chat window, it can respond by generating a complete UI component like:

An interactive chart or graph.
A fully populated form or table.
Specific buttons (like "Approve" or "Reject") relevant only to the task at hand.

You can learn more about A2UI here.

Now that you have learned what the A2UI protocol is, let us see how to use it together with A2A, AG-UI, and CopilotKit to build full-stack A2UI AI agents.

Prerequisites

To fully understand this tutorial, you need to have a basic understanding of React or Next.js

We'll also make use of the following:

Python - a popular programming language for building AI agents with AI agent frameworks; make sure it is installed on your computer.
AG-UI Protocol - The Agent User Interaction Protocol (AG-UI), developed by CopilotKit, is an open-source, lightweight, event-based protocol that facilitates rich, real-time interactions between the frontend and your AI agent backend.
Google ADK - an open-source framework designed by Google to simplify the process of building complex and production-ready AI agents.
Gemini API Key - an API key to enable you to perform various tasks using the Gemini models for ADK agents.
CopilotKit - an open-source copilot framework for building custom AI chatbots, in-app AI agents, and text areas.

Building A2UI + A2A agent backend

In this section, you will learn how to configure your agent to use A2UI using the A2A protocol in the backend.

Let’s get started.

To get started, clone the A2A starter template using the command below:

git clone https://github.com/copilotkit/with-a2a-a2ui.git

After that, create an environment file with your API key:

echo "GEMINI_API_KEY=your_api_key_here" > .env

Then run the command below to install the dependencies:

pnpm install

Finally, run and connect your agent using the following command:

pnpm dev

Let us now see how to configure your agent to use A2UI using the A2A protocol.

Step 1: Define your agent A2UI Response Components

To define your agent A2UI response components, first, define a schema that contains the complete JSON Schema that defines a valid A2UI message, as shown in the agent/restaurant_finder/prompt_builder.py file.

A2UI_SCHEMA = r'''
{
  "title": "A2UI Message Schema",
  "description": "Describes a JSON payload for an A2UI (Agent to UI) message, which is used to construct and update user interfaces dynamically. A message MUST contain exactly ONE of the action properties: 'beginRendering', 'surfaceUpdate', 'dataModelUpdate', or 'deleteSurface'.",
  "type": "object",
  "properties": {
    "beginRendering": {
      "type": "object",
      "description": "Signals the client to begin rendering a surface with a root component and specific styles.",
      "properties": {
        "surfaceId": {
          "type": "string",
          "description": "The unique identifier for the UI surface to be rendered."
        },
        "root": {
          "type": "string",
          "description": "The ID of the root component to render."
        },
        "styles": {
          "type": "object",
          "description": "Styling information for the UI.",
          "properties": {
            "font": {
              "type": "string",
              "description": "The primary font for the UI."
            },
            "primaryColor": {
              "type": "string",
              "description": "The primary UI color as a hexadecimal code (e.g., '#00BFFF').",
              "pattern": "^#[0-9a-fA-F]{6}$"
            }
          }
        }
      },
      "required": ["root", "surfaceId"]
    },

   // ...

  }

Then define A2UI response components that your agent uses to format its responses, as shown in the agent/restaurant_finder/prompt_builder.py file.

RESTAURANT_UI_EXAMPLES = """
---BEGIN SINGLE_COLUMN_LIST_EXAMPLE---
[
  {{ "beginRendering": {{ "surfaceId": "default", "root": "root-column", "styles": {{ "primaryColor": "#FF0000", "font": "Roboto" }} }} }},
  {{ "surfaceUpdate": {{
    "surfaceId": "default",
    "components": [
      {{ "id": "root-column", "component": {{ "Column": {{ "children": {{ "explicitList": ["title-heading", "item-list"] }} }} }} }},
      {{ "id": "title-heading", "component": {{ "Text": {{ "usageHint": "h1", "text": {{ "literalString": "Top Restaurants" }} }} }} }},
      {{ "id": "item-list", "component": {{ "List": {{ "direction": "vertical", "children": {{ "template": {{ "componentId": "item-card-template", "dataBinding": "/items" }} }} }} }} }},
      {{ "id": "item-card-template", "component": {{ "Card": {{ "child": "card-layout" }} }} }},
      {{ "id": "card-layout", "component": {{ "Row": {{ "children": {{ "explicitList": ["template-image", "card-details"] }} }} }} }},
      {{ "id": "template-image", weight: 1, "component": {{ "Image": {{ "url": {{ "path": "imageUrl" }} }} }} }},
      {{ "id": "card-details", weight: 2, "component": {{ "Column": {{ "children": {{ "explicitList": ["template-name", "template-rating", "template-detail", "template-link", "template-book-button"] }} }} }} }},
      {{ "id": "template-name", "component": {{ "Text": {{ "usageHint": "h3", "text": {{ "path": "name" }} }} }} }},
      {{ "id": "template-rating", "component": {{ "Text": {{ "text": {{ "path": "rating" }} }} }} }},
      {{ "id": "template-detail", "component": {{ "Text": {{ "text": {{ "path": "detail" }} }} }} }},
      {{ "id": "template-link", "component": {{ "Text": {{ "text": {{ "path": "infoLink" }} }} }} }},
      {{ "id": "template-book-button", "component": {{ "Button": {{ "child": "book-now-text", "primary": true, "action": {{ "name": "book_restaurant", "context": [ {{ "key": "restaurantName", "value": {{ "path": "name" }} }}, {{ "key": "imageUrl", "value": {{ "path": "imageUrl" }} }}, {{ "key": "address", "value": {{ "path": "address" }} }} ] }} }} }} }},
      {{ "id": "book-now-text", "component": {{ "Text": {{ "text": {{ "literalString": "Book Now" }} }} }} }}
    ]
  }} }},
  {{ "dataModelUpdate": {{
    "surfaceId": "default",
    "path": "/",
    "contents": [
      {{ "key": "items", "valueMap": [
        {{ "key": "item1", "valueMap": [
          {{ "key": "name", "valueString": "The Fancy Place" }},
          {{ "key": "rating", "valueNumber": 4.8 }},
          {{ "key": "detail", "valueString": "Fine dining experience" }},
          {{ "key": "infoLink", "valueString": "https://example.com/fancy" }},
          {{ "key": "imageUrl", "valueString": "https://example.com/fancy.jpg" }},
          {{ "key": "address", "valueString": "123 Main St" }}
        ] }},
        {{ "key": "item2", "valueMap": [
          {{ "key": "name", "valueString": "Quick Bites" }},
          {{ "key": "rating", "valueNumber": 4.2 }},
          {{ "key": "detail", "valueString": "Casual and fast" }},
          {{ "key": "infoLink", "valueString": "https://example.com/quick" }},
          {{ "key": "imageUrl", "valueString": "https://example.com/quick.jpg" }},
          {{ "key": "address", "valueString": "456 Oak Ave" }}
        ] }}
      ] }} // Populate this with restaurant data
    ]
  }} }}
]
---END SINGLE_COLUMN_LIST_EXAMPLE---

// ...

After that, define a function that constructs the full prompt with instructions, A2UI response components and A2UI message JSON Schema, as shown in the agent/restaurant_finder/prompt_builder.py file.

def get_ui_prompt(base_url: str, examples: str) -> str:
    """
    Constructs the full prompt with UI instructions, rules, examples, and schema.

    Args:
        base_url: The base URL for resolving static assets like logos.
        examples: A string containing the specific UI examples for the agent's task.

    Returns:
        A formatted string to be used as the system prompt for the LLM.
    """
    # The f-string substitution for base_url happens here, at runtime.
    formatted_examples = examples.format(base_url=base_url)

    return f"""
    You are a helpful restaurant finding assistant. Your final output MUST be a A2UI UI JSON response.

    To generate the response, you MUST follow these rules:
    1.  Your response MUST be in two parts, separated by the delimiter: `---a2ui_JSON---`.
    2.  The first part is your conversational text response.
    3.  The second part is a single, raw JSON object that is a list of A2UI messages.
    4.  The JSON part MUST validate against the A2UI JSON SCHEMA provided below.

    --- UI TEMPLATE RULES ---
    -   If the query is for a list of restaurants, use the restaurant data you have already received from the `get_restaurants` tool to populate the `dataModelUpdate.contents` array (e.g., as a `valueMap` for the "items" key).
    -   If the number of restaurants is 5 or fewer, you MUST use the `SINGLE_COLUMN_LIST_EXAMPLE` template.
    -   If the number of restaurants is more than 5, you MUST use the `TWO_COLUMN_LIST_EXAMPLE` template.
    -   If the query is to book a restaurant (e.g., "USER_WANTS_TO_BOOK..."), you MUST use the `BOOKING_FORM_EXAMPLE` template.
    -   If the query is a booking submission (e.g., "User submitted a booking..."), you MUST use the `CONFIRMATION_EXAMPLE` template.

    {formatted_examples}

    ---BEGIN A2UI JSON SCHEMA---
    {A2UI_SCHEMA}
    ---END A2UI JSON SCHEMA---
    """

Step 2: Generating A2UI components with A2UI Composer

If you want an easy way to generate components, you can use the A2UI Composer, which enables you to generate A2UI components.

To create a component using the A2UI composer, go to https://a2ui-editor.ag-ui.com/ and describe your A2UI widget, as shown below.

Then click the create button, and the composer will generate your A2UI widget, as shown below.

After that, copy the generated JSON spec and paste it into your agent's prompt as we discussed in step one above.

Step 3: Configure your A2UI agent

Once you have set up your A2A components and assembled your agent prompt, configure your agent, as shown in the agent/restaurant_finder/agent.py file.

// ...

# Local imports from sibling modules
from prompt_builder import (
    A2UI_SCHEMA,  # JSON Schema for A2UI validation
    RESTAURANT_UI_EXAMPLES,  # Example A2UI responses for few-shot learning
    get_text_prompt,  # Prompt for text-only mode
    get_ui_prompt,  # Prompt for UI mode (with schema and examples)
)
from tools import get_restaurants  # Tool function for fetching restaurant data

logger = logging.getLogger(__name__)

AGENT_INSTRUCTION = """
    You are a helpful restaurant finding assistant. Your goal is....
"""

class RestaurantAgent:
    """
    An agent that finds restaurants based on user criteria.

    """

    # Supported content types for A2A protocol
    SUPPORTED_CONTENT_TYPES = ["text", "text/plain"]

    def __init__(self, base_url: str, use_ui: bool = False):
        """
        Initialize the RestaurantAgent.

        Args:
            base_url: The server's base URL (used for resolving image URLs)
            use_ui: If True, generate A2UI JSON responses; if False, generate text
        """
        self.base_url = base_url
        self.use_ui = use_ui

        # Build the underlying LlmAgent with appropriate instructions
        self._agent = self._build_agent(use_ui)

        # User ID for session management (constant for this simple example)
        self._user_id = "remote_agent"

        // ...

        # =====================================================================
        # LOAD AND WRAP A2UI SCHEMA
        # =====================================================================
        # We need to validate LLM responses against the A2UI schema.
        # The schema defines a SINGLE message, but the LLM returns a LIST of messages.
        # So we wrap the schema in an array validator.
        try:
            # Parse the schema for a single A2UI message
            single_message_schema = json.loads(A2UI_SCHEMA)

            # Wrap it: the LLM must return an ARRAY of valid messages
            self.a2ui_schema_object = {"type": "array", "items": single_message_schema}
            logger.info(
                "A2UI_SCHEMA successfully loaded and wrapped in an array validator."
            )
        except json.JSONDecodeError as e:
            logger.error(f"CRITICAL: Failed to parse A2UI_SCHEMA: {e}")
            self.a2ui_schema_object = None  # Validation will be skipped

   // ...

    # =========================================================================
    # STEP 3c: BUILD THE LLM AGENT
    # =========================================================================
    def _build_agent(self, use_ui: bool) -> LlmAgent:
        """
        Builds the LLM agent with appropriate instructions and tools.

        Args:
            use_ui: Whether to include UI instructions and schema in the prompt

        Returns:
            A configured LlmAgent instance
        """
        # Get the model name from the environment variable, with a sensible default
        LITELLM_MODEL = os.getenv("LITELLM_MODEL", "gemini/gemini-2.5-flash")

        if use_ui:
            # UI mode: Include instructions for generating A2UI JSON
            # This adds:
            # - Detailed formatting rules (delimiter, JSON structure)
            # - Template examples for different scenarios
            # - The complete A2UI JSON schema
            instruction = AGENT_INSTRUCTION + get_ui_prompt(
                self.base_url, RESTAURANT_UI_EXAMPLES
            )
        else:
            # Text mode: Simple instructions for text-only responses
            instruction = get_text_prompt()

        # Create the LlmAgent with:
        # - A LiteLLM wrapper (allows using different LLM providers)
        # - Name and description for identification
        # - The instruction (system prompt)
        # - Available tools (get_restaurants function)
        return LlmAgent(
            model=LiteLlm(model=LITELLM_MODEL),
            name="restaurant_agent",
            description="An agent that finds restaurants and helps book tables.",
            instruction=instruction,
            tools=[get_restaurants],  # The agent can call this function
        )

      // ...

Step 4: Configure your A2UI agent executor using A2A protocol

After configuring your A2UI agent, configure the agent with an A2A agent executor that handles A2A requests and responses, as shown in the agent/restaurant_finder/agent_executer.py file.

// ...

# =============================================================================
# STEP 2: AGENT EXECUTOR CLASS
# =============================================================================
class RestaurantAgentExecutor(AgentExecutor):
    """
    Restaurant AgentExecutor - Bridges A2A Protocol and RestaurantAgent.


    """

    # =========================================================================
    # STEP 2a: INITIALIZATION
    # =========================================================================
    def __init__(self, base_url: str):
        """
        Initialize the executor with two agent instances.


        """
        # UI agent: Generates rich A2UI responses with components and data models
        self.ui_agent = RestaurantAgent(base_url=base_url, use_ui=True)

        # Text agent: Generates plain text responses (for clients without A2UI support)
        self.text_agent = RestaurantAgent(base_url=base_url, use_ui=False)

    # =========================================================================
    # STEP 2b: MAIN EXECUTION METHOD
    # =========================================================================
    async def execute(
        self,
        context: RequestContext,
        event_queue: EventQueue,
    ) -> None:
        """
        Main entry point for handling A2A requests.

        This method orchestrates the entire request-response cycle:
        1. Detects whether the client supports A2UI
        2. Parses incoming message parts (text or UI events)
        3. Transforms UI events into natural language queries
        4. Invokes the appropriate agent
        5. Formats and sends the response

        Args:
            context: Contains the incoming message, task state, and client info
            event_queue: Queue for sending events (status updates, responses) to client
        """
        # Initialize variables for tracking the query and UI events
        query = ""  # The final query string to send to the LLM
        ui_event_part = None  # Will hold A2UI event data if present
        action = None  # The action name from UI events (e.g., "book_restaurant")

        # =====================================================================
        # STEP 2b-i: DETECT A2UI SUPPORT
        # =====================================================================
        # Check which protocol extensions the client requested
        logger.info(
            f"--- Client requested extensions: {context.requested_extensions} ---"
        )

        # try_activate_a2ui_extension checks if A2UI is in the requested extensions
        # and marks it as active in the context if so
        use_ui = try_activate_a2ui_extension(context)

        # =====================================================================
        # STEP 2b-ii: SELECT APPROPRIATE AGENT
        # =====================================================================
        # Based on A2UI support, choose which agent to use
        if use_ui:
            agent = self.ui_agent
            logger.info(
                "--- AGENT_EXECUTOR: A2UI extension is active. Using UI agent. ---"
            )
        else:
            agent = self.text_agent
            logger.info(
                "--- AGENT_EXECUTOR: A2UI extension is not active. Using text agent. ---"
            )

        # =====================================================================
        # STEP 2b-iii: PARSE INCOMING MESSAGE PARTS
        # =====================================================================
        # A2A messages can contain multiple "parts" - these can be:
        # - TextPart: Plain text from the user
        # - DataPart: Structured data (like A2UI events from button clicks)
        if context.message and context.message.parts:
            logger.info(
                f"--- AGENT_EXECUTOR: Processing {len(context.message.parts)} message parts ---"
            )

            # Iterate through all parts to find relevant data
            for i, part in enumerate(context.message.parts):
                if isinstance(part.root, DataPart):
                    # DataPart contains structured JSON data
                    if "userAction" in part.root.data:
                        # This is an A2UI ClientEvent - a button was clicked!
                        logger.info(f"  Part {i}: Found a2ui UI ClientEvent payload.")
                        ui_event_part = part.root.data["userAction"]
                    else:
                        # Some other structured data
                        logger.info(f"  Part {i}: DataPart (data: {part.root.data})")
                elif isinstance(part.root, TextPart):
                    # Plain text from the user
                    logger.info(f"  Part {i}: TextPart (text: {part.root.text})")
                else:
                    logger.info(f"  Part {i}: Unknown part type ({type(part.root)})")

        # =====================================================================
        # STEP 2b-iv: TRANSFORM UI EVENTS INTO LLM QUERIES
        # =====================================================================
        # If we received a UI event (button click), we need to transform it
        # into a natural language query that the LLM can understand.
        # This is a key part of the A2UI pattern - UI events become LLM inputs.
        if ui_event_part:
            logger.info(f"Received a2ui ClientEvent: {ui_event_part}")

            # Extract the action name and context data from the event
            action = ui_event_part.get("actionName")
            ctx = ui_event_part.get("context", {})

            # Handle "Book Now" button click
            if action == "book_restaurant":
                # User clicked "Book Now" on a restaurant card
                # Extract restaurant details from the button's context
                restaurant_name = ctx.get("restaurantName", "Unknown Restaurant")
                address = ctx.get("address", "Address not provided")
                image_url = ctx.get("imageUrl", "")

                # Create a structured query that tells the LLM what the user wants
                # The LLM is trained to recognize this pattern and show a booking form
                query = f"USER_WANTS_TO_BOOK: {restaurant_name}, Address: {address}, ImageURL: {image_url}"

            # Handle "Submit Reservation" button click
            elif action == "submit_booking":
                # User filled out the booking form and submitted it
                # Extract all the form data from the button's context
                restaurant_name = ctx.get("restaurantName", "Unknown Restaurant")
                party_size = ctx.get("partySize", "Unknown Size")
                reservation_time = ctx.get("reservationTime", "Unknown Time")
                dietary_reqs = ctx.get("dietary", "None")
                image_url = ctx.get("imageUrl", "")

                # Create a query that tells the LLM to confirm the booking
                query = f"User submitted a booking for {restaurant_name} for {party_size} people at {reservation_time} with dietary requirements: {dietary_reqs}. The image URL is {image_url}"

            else:
                # Unknown action - pass it through as-is
                query = f"User submitted an event: {action} with data: {ctx}"
        else:
            # No UI event - this is a regular text message
            logger.info("No a2ui UI event part found. Falling back to text input.")
            query = context.get_user_input()  # Extract text from the message parts

        logger.info(f"--- AGENT_EXECUTOR: Final query for LLM: '{query}' ---")

        // ...

    # =========================================================================
    # STEP 2c: CANCEL METHOD (NOT IMPLEMENTED)
    # =========================================================================
    async def cancel(
        self, request: RequestContext, event_queue: EventQueue
    ) -> Task | None:
        """
        Handle task cancellation requests.

        This agent does not support cancellation, so we raise an error.
        A more sophisticated agent might gracefully stop ongoing work.

        Raises:
            ServerError: With UnsupportedOperationError
        """
        raise ServerError(error=UnsupportedOperationError())

Step 5: Configure your A2UI agent server using A2A protocol

Once you have configured the agent executor, set up a main entry for your agent that sets up and starts an A2A-compliant server that can:

Respond to agent discovery requests (via AgentCard)
Handle user queries about restaurants
Serve static files (restaurant images)
Support both text-only and A2UI (rich UI) responses

You can configure the server, as shown in the agent/restaurant_finder/__main__.py file.

// ...

@click.command()  # Decorates this function as a CLI command
@click.option("--host", default="localhost")  # Optional --host argument
@click.option("--port", default=10002)  # Optional --port argument
def main(host, port):
    """
    Main entry point for the Restaurant Finder Agent server.


    """
    try:
        # =====================================================================
        # VALIDATE API CREDENTIALS
        # =====================================================================
        # The agent needs either:
        # - GEMINI_API_KEY for direct Gemini API access, OR
        # - GOOGLE_GENAI_USE_VERTEXAI=TRUE for Vertex AI authentication
        if not os.getenv("GOOGLE_GENAI_USE_VERTEXAI") == "TRUE":
            if not os.getenv("GEMINI_API_KEY"):
                raise MissingAPIKeyError(
                    "GEMINI_API_KEY environment variable not set and GOOGLE_GENAI_USE_VERTEXAI is not TRUE."
                )

        # =====================================================================
        # DEFINE AGENT CAPABILITIES
        # =====================================================================
        # AgentCapabilities tells clients what this agent supports:
        # - streaming=True: Agent can stream responses progressively
        # - extensions: List of protocol extensions (here, A2UI for rich UIs)
        capabilities = AgentCapabilities(
            streaming=True,
            extensions=[get_a2ui_agent_extension()],  # Enable A2UI rich UI support
        )

        # =====================================================================
        # DEFINE AGENT SKILLS
        # =====================================================================
        # AgentSkill describes what this agent can do. This metadata is used
        # for agent discovery and helps clients understand the agent's purpose.
        skill = AgentSkill(
            id="find_restaurants",  # Unique identifier for this skill
            name="Find Restaurants Tool",  # Human-readable name
            description="Helps find restaurants based on user criteria (e.g., cuisine, location).",
            tags=["restaurant", "finder"],  # Searchable tags
            examples=["Find me the top 10 chinese restaurants in the US"],  # Example queries
        )

        # =====================================================================
        # CONSTRUCT BASE URL
        # =====================================================================
        # The base URL is used for:
        # - Agent discovery (clients connect to this URL)
        # - Resolving static asset URLs (e.g., restaurant images)
        base_url = f"http://{host}:{port}"

        # =====================================================================
        # CREATE AGENT CARD
        # =====================================================================
        # The AgentCard is the "identity card" for this agent in the A2A protocol.
        # Clients use this to discover and understand the agent.
        agent_card = AgentCard(
            name="Restaurant Agent",  # Display name
            description="This agent helps find restaurants based on user criteria.",
            url=base_url,  # Where to reach this agent
            version="1.0.0",  # Semantic version
            default_input_modes=RestaurantAgent.SUPPORTED_CONTENT_TYPES,  # ["text", "text/plain"]
            default_output_modes=RestaurantAgent.SUPPORTED_CONTENT_TYPES,
            capabilities=capabilities,  # What the agent supports
            skills=[skill],  # What the agent can do
        )

        # =====================================================================
        # CREATE AGENT EXECUTOR
        # =====================================================================
        # The AgentExecutor is the bridge between the A2A protocol and our
        # RestaurantAgent. It handles:
        # - Parsing A2A requests
        # - Invoking the appropriate agent (UI or text)
        # - Formatting A2A responses
        agent_executor = RestaurantAgentExecutor(base_url=base_url)

        # =====================================================================
        # CREATE REQUEST HANDLER
        # =====================================================================
        # DefaultRequestHandler processes incoming A2A requests and delegates
        # to our agent_executor. The InMemoryTaskStore keeps track of ongoing
        # tasks (useful for multi-turn conversations).
        request_handler = DefaultRequestHandler(
            agent_executor=agent_executor,
            task_store=InMemoryTaskStore(),
        )

        # =====================================================================
        # BUILD THE A2A APPLICATION
        # =====================================================================
        # A2AStarletteApplication creates an ASGI web application that:
        # - Exposes the /.well-known/agent.json endpoint (agent discovery)
        # - Handles A2A protocol messages
        server = A2AStarletteApplication(
            agent_card=agent_card, http_handler=request_handler
        )
        import uvicorn

        # Build the Starlette app from our A2A configuration
        app = server.build()

          // ...

        # =====================================================================
        # MOUNT STATIC FILES
        # =====================================================================
        # Serve restaurant images from the ./images directory at /static
        # Example: http://localhost:10002/static/restaurant1.jpg
        app.mount("/static", StaticFiles(directory="images"), name="static")

        # =====================================================================
        # START THE SERVER
        # =====================================================================
        # uvicorn is an ASGI server that will run our application.
        # It handles HTTP connections and routes them to our app.
        uvicorn.run(app, host=host, port=port)

      // ...

Congrats! You've successfully built your A2UI backend using the A2A protocol. Let’s see how to add a frontend to it using AG-UI and CopilotKit.

Building A2UI + A2A agent frontend using AG-UI and CopilotKit

In this section, you will learn how to configure your A2UI + A2A agent with the frontend using AG-UI protocol that handles communicating with your A2UI + A2A agent, and passes the A2UI messages back and forth as ActivityMessage objects.

Let’s get started.

Step 1: Configure the CopilotKit API Route with AG-UI and A2A Client

To connect your A2UI + A2A agent to the frontend, configure a CopilotKit API route handler that connects the frontend to your A2UI + A2A agent backend using AG-UI and A2A client, as shown below.

// CopilotKit Runtime - Core server-side components
import {
  CopilotRuntime,         // Main runtime that orchestrates agents and handles requests
  createCopilotEndpoint,  // Factory function to create an HTTP endpoint
  InMemoryAgentRunner,    // Runs agents with in-memory state (no persistence)
} from "@copilotkitnext/runtime";

// Hono adapter for Vercel/Next.js - Converts Hono app to Next.js route handlers
import { handle } from "hono/vercel";

// A2A (Agent-to-Agent) Integration
import { A2AAgent } from "@ag-ui/a2a";     // CopilotKit wrapper for A2A protocol
import { A2AClient } from "@a2a-js/sdk/client"; // Client to communicate with A2A servers

// =============================================================================
// STEP 2: CREATE A2A, CLIENT
// =============================================================================
/**
 * A2AClient connects to an A2A-compliant agent server.
 */
const a2aClient = new A2AClient("http://localhost:10002");

// =============================================================================
// STEP 3: CREATE A2A AGENT WRAPPER
// =============================================================================
/**
 * A2AAgent wraps the A2AClient to make it compatible with CopilotKit.
 */
const agent = new A2AAgent({ a2aClient, debug: true });

// =============================================================================
// STEP 4: CREATE COPILOTKIT RUNTIME
// =============================================================================
/**
 * CopilotRuntime is the central orchestrator that:
 * - Manages registered agents
 * - Routes requests to the appropriate agent
 * - Handles streaming responses
 * - Manages conversation state
 */
const runtime = new CopilotRuntime({
  agents: {
    default: agent, // Our A2A Restaurant Agent is the default agent
  },
  runner: new InMemoryAgentRunner(),
});

// =============================================================================
// STEP 5: CREATE THE HTTP ENDPOINT
// =============================================================================
/**
 * createCopilotEndpoint creates a Hono web application that handles
 * CopilotKit API requests.
 *
 * The endpoint handles:
 * - POST /api/copilotkit - Main chat endpoint for sending messages
 * - GET /api/copilotkit - Health check and capability discovery
 * - Various sub-routes for streaming, actions, etc.
 */
const app = createCopilotEndpoint({
  runtime,
  basePath: "/api/copilotkit",
});

export const GET = handle(app as any);
export const POST = handle(app as any);

Step 2: Create A2UI Message Renderer

After configuring the API route, create an A2UI message renderer provided by CopilotKit to render A2UI messages and instantiate it with a theme, as shown below.

"use client";

// A2UI Renderer Factory - Creates a message renderer for A2UI protocol responses
import { createA2UIMessageRenderer } from "@copilotkitnext/a2ui-renderer";

// Custom theme configuration for styling A2UI components
import { theme } from "./theme";

// =============================================================================
// CREATE A2UI MESSAGE RENDERER
// =============================================================================
/**
 * A2UIMessageRenderer is a custom message renderer that knows how to
 * display A2UI protocol messages as rich, interactive UI components.
 *
 * When the A2A agent sends back A2UI JSON (with beginRendering, surfaceUpdate,
 * dataModelUpdate messages), this renderer:
 * 1. Parses the A2UI JSON
 * 2. Builds a component tree from the surfaceUpdate
 * 3. Binds data from dataModelUpdate to components
 * 4. Renders native React components (cards, buttons, forms, etc.)
 * 5. Handles user interactions and sends events back to the agent
 *
 * The `theme` option allows customizing colors, fonts, and spacing of
 * The rendered A2UI components to match your app's design.
 */
const A2UIMessageRenderer = createA2UIMessageRenderer({ theme });

// ....

Step 3: Set up the CopilotKit provider and Chat Component

Once you have created the A2UI message renderer, set up the CopilotKit provider together with the chat component.

Then pass the A2UI message renderer to the CopilotKit provider, as shown below.

"use client";

import { CopilotChat, CopilotKitProvider } from "@copilotkitnext/react";

// ...

export default function Home() {
  return (
    // =========================================================================
    // CopilotKitProvider - The Context Provider
    // =========================================================================
    /**
     * CopilotKitProvider wraps the app and provides:
     * - Connection to the backend API
     * - Shared state for all CopilotKit components
     * - Message history management
     * - Action/tool registration
     *
     * Props:
     * - runtimeUrl: The backend API endpoint (our Next.js API route)
     * - showDevConsole: "auto" shows dev tools in development mode only
     * - renderActivityMessages: Custom renderers for agent messages
     *   - A2UIMessageRenderer handles A2UI protocol messages
     *   - Multiple renderers can be provided for different message types
     */
    <CopilotKitProvider
      runtimeUrl="/api/copilotkit"
      showDevConsole="auto"
      renderActivityMessages={[A2UIMessageRenderer]}
    >
      {/* Main container - full viewport height with flex layout */}
      <main
        className="flex min-h-screen flex-1 flex-col overflow-hidden"
        style={{ minHeight: "100dvh" }} // Use dynamic viewport height for mobile
      >
        <Chat />
      </main>
    </CopilotKitProvider>
  );
}

// =============================================================================
// CHAT COMPONENT
// =============================================================================
/**
 * Chat is a wrapper component that contains the CopilotChat interface.
 */
function Chat() {
  return (
    // Flex container that fills available space
    <div className="flex flex-1 flex-col overflow-hidden">
      <CopilotChat style={{ flex: 1, minHeight: "100%" }} />
    </div>
  );
}

Step 4: Running your A2UI + A2A + AG-UI agent

After setting up the CopilotKit provider alongside the chat component, run your agent backend server alongside the frontend server. Then navigate to the frontend localhost URL, and you should see the A2UI agent running with the chat component, as shown below.

After that, ask the agent to find you restaurants in New York with Chinese cuisine. Then you should see the A2UI components and messages rendering with the top restaurants in New York that you can book and reserve, as shown.

Conclusion

In this guide, we have walked through the steps of building a full-stack A2UI agent using A2A + AG-UI protocols and CopilotKit.

While we’ve explored a couple of features, we have barely scratched the surface of the countless use cases for CopilotKit, ranging from building interactive AI chatbots to building agentic solutions—in essence, CopilotKit lets you add a ton of functional AI capabilities to your products in minutes.

Hopefully, this guide makes it easier for you to integrate AI-powered Copilots into your existing application.

Follow CopilotKit on Twitter and say hi. If you'd like to build something cool, join the Discord community.

DEV Community: CopilotKit

AIMock: One Mock Server For Your Entire AI Stack

Why we built AIMock

Table of Contents

AI apps in 2026 call a lot more than just an LLM

Introducing AIMock: mock your entire agentic stack

LLMock: full streaming, tool calls, reasoning across every major provider

MCPMock: a mock MCP server for testing tool integrations

A2AMock: test multi-agent workflows without live agents

VectorMock: deterministic retrieval for RAG pipelines

Services: search, rerank, and moderation

Drift Detection: catch provider changes before your users do

Record and Replay: stop hand-writing fixtures

Chaos Testing: prove your app handles failure

What else is included

AG-UI uses AIMock in production

Migrating to AIMock

Get started

CopilotKit Follow

Google Maps for Codebases: Paste a GitHub URL, Ask Anything

What are we building?

Tech Stack & Tools

What is Zenflow?

Architecture & Project Structure

How everything works under the hood

1. Loading a repository

2. From architecture view to dependency graph

3. The CopilotKit runtime

4. Feeding the LLM context

5. The four frontend tools

6. Finding relevant files

7. File caching

8. Parsing imports and resolving paths

9. Graph layout with Dagre

10. State management with Zustand

Running it locally

Using OpenAI

The bigger picture

CopilotKit Follow

Deploying AG-UI Agents to Production with Amazon Bedrock AgentCore

Amazon Bedrock AgentCore Runtime

AG-UI: the interaction protocol

How It All Connects

Deploying your AG-UI agent with Strands

Step 1: Build your AG-UI server

Step 2: Test locally

Step 3: Deploy to AgentCore

Step 4: Invoke your deployed agent

Step 5: Connect your frontend

Resources

What This Means

CopilotKit Follow

How to add AI to your App in 5 minutes

Why AI Integrations Get Complicated

What CopilotKit Actually Does

How to wire everything (1 minute)

Give the AI Context

Let the AI Do Things

Bonus: Generative UI

CopilotKit Follow

Give Your AG2 Agents a UI with AG‑UI and CopilotKit

AG2: Runtime for Agent Teams

The Gap: Orchestration Without an Application Contract

AG-UI: The Execution & Interaction Layer

How CopilotKit wires AG2 to AG-UI

Integration Flow: AG2 → AG-UI → CopilotKit

Multi-Agent Orchestration

What This Enables for Agent Builders

Getting Started

What's Next

How to Build a Research Assistant using Deep Agents

What is covered?

1. What are Deep Agents?

The mental model (how it runs)

Where CopilotKit Fits

2. Core Components

How Deep Agents are implemented (Middleware)

3. What are we building?

4. Frontend: wiring the agent to the UI

Step 1: CopilotKit Provider & Layout