DEV Community: Agentwright

Claude Agent SDK vs rolling your own tool loop: which one for your product?

Agentwright — Thu, 05 Nov 2026 21:18:12 +0000

You have decided your product needs an agent: Claude should call tools, look at the results and decide what to do next. Before you write any code there is a choice that shapes everything after it. Do you write the tool loop yourself, let the Anthropic SDK run it for you, or embed the Claude Agent SDK and get Claude Code's whole harness as a library?

This post is a decision guide, not a review. Each option is the right answer for some products. Below is runnable TypeScript for all three, what each one leaves you in control of, the token usage we measured when running them, and a table at the end that maps "what you are building" to an option.

Facts and code were checked against the official docs and run against the live API on 2026-11-03 with @anthropic-ai/sdk 1.4.0 and @anthropic-ai/claude-agent-sdk 0.3.317.

The three options

Your own loop on the Messages API. You call client.messages.create(), check stop_reason, run the tools Claude asked for, send the results back, repeat. About 50 lines. You own every request.
The Tool Runner in the Anthropic SDK. Same API, same tools you define, but client.beta.messages.toolRunner() runs the loop. You write tool functions, it calls them and feeds the results back until Claude is done. It is a beta helper.
The Claude Agent SDK. Claude Code packaged as a library (@anthropic-ai/claude-agent-sdk). It ships built-in tools for reading, writing and editing files, running shell commands and searching the web, plus permissions, hooks, subagents and sessions. You call query() with a prompt and options.

The common confusion: the Tool Runner and the Agent SDK sound alike but are different products. The Tool Runner lives in the regular API SDK and only loops over tools you define. The Agent SDK runs the Claude Code binary (the npm package bundles it) with its own tools and its own system prompt. Neither hosts anything for you; both run wherever your code runs.

Option 1: your own loop

The example is a support agent with one tool that looks up an order. In a real product the tool would query your database.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();
const MODEL = process.env.MODEL ?? "claude-sonnet-5-5";

const tools: Anthropic.Tool[] = [
  {
    name: "get_order",
    description: "Look up an order by id. Returns status and total in USD.",
    input_schema: {
      type: "object",
      properties: { order_id: { type: "string" } },
      required: ["order_id"],
      additionalProperties: false,
    },
    strict: true,
  },
];

async function runTool(name: string, input: unknown): Promise<string> {
  if (name === "get_order") {
    const { order_id } = input as { order_id: string };
    return JSON.stringify({ order_id, status: "shipped", total_usd: 99 });
  }
  throw new Error(`Unknown tool: ${name}`);
}

export async function agent(prompt: string, maxTurns = 10) {
  const messages: Anthropic.MessageParam[] = [{ role: "user", content: prompt }];

  for (let turn = 0; turn < maxTurns; turn++) {
    const res = await client.messages.create({
      model: MODEL,
      max_tokens: 4096,
      system: [{ type: "text", text: "You are a support agent for an online shop.", cache_control: { type: "ephemeral" } }],
      tools,
      messages,
    });
    console.log(`turn ${turn}`, res.stop_reason, res.usage); // meter here: one row per request

    messages.push({ role: "assistant", content: res.content });
    if (res.stop_reason !== "tool_use") return res;

    // Run every tool call from this turn, then send ALL results back in ONE user message.
    const calls = res.content.filter((b): b is Anthropic.ToolUseBlock => b.type === "tool_use");
    const results: Anthropic.ToolResultBlockParam[] = await Promise.all(
      calls.map(async (c) => {
        try {
          return { type: "tool_result", tool_use_id: c.id, content: await runTool(c.name, c.input) };
        } catch (err) {
          return { type: "tool_result", tool_use_id: c.id, content: String(err), is_error: true };
        }
      }),
    );
    messages.push({ role: "user", content: results });
  }
  throw new Error("Agent hit maxTurns without finishing");
}

const final = await agent("Where is order A-1042 and how much was it?");
console.log(final.content.filter((b) => b.type === "text").map((b) => b.text).join(""));

Four details in those lines are easy to get wrong:

Return all tool results in one user message. Claude can ask for several tools in one turn. If you split their results across several messages, it learns to stop making parallel calls.
Report failures, do not drop them. A failed tool still gets a tool_result, with is_error: true, so Claude can recover or explain.
Guard the turn count. Without maxTurns, a confused model and a tool that keeps failing can loop until your bill notices.
strict: true makes the API guarantee that the tool input matches your schema, so the cast in runTool is safe.

The loop treats any stop_reason other than tool_use as the end. In production you also want explicit branches for max_tokens (the answer was cut off) and refusal, and for pause_turn if you add server tools such as web search.

Option 2: the Tool Runner

Same tool, same model, no loop code. Tools are defined with Zod and carry their own run function.

import Anthropic from "@anthropic-ai/sdk";
import { betaZodTool } from "@anthropic-ai/sdk/helpers/beta/zod";
import { z } from "zod";

const client = new Anthropic();
const MODEL = process.env.MODEL ?? "claude-sonnet-5-5";

const getOrder = betaZodTool({
  name: "get_order",
  description: "Look up an order by id. Returns status and total in USD.",
  inputSchema: z.object({ order_id: z.string() }),
  run: async ({ order_id }) => JSON.stringify({ order_id, status: "shipped", total_usd: 99 }),
});

const runner = client.beta.messages.toolRunner({
  model: MODEL,
  max_tokens: 4096,
  max_iterations: 10,
  system: "You are a support agent for an online shop.",
  tools: [getOrder],
  messages: [{ role: "user", content: "Where is order A-1042 and how much was it?" }],
});

// Each iteration yields one API response, so you can still meter per request.
for await (const message of runner) {
  console.log(message.stop_reason, message.usage);
}
const final = await runner.done();
console.log(final.content.filter((b) => b.type === "text").map((b) => b.text).join(""));

Iterating over the runner gives you each API response as it arrives, which is where per-request metering, logging and approval checks go. If you only want the answer, await runner.runUntilDone() does it in one line. Approval gates do not need a manual loop either: check inside the tool's run and return "the user declined" as the result.

The trade-off is that this is a beta surface. Pin the SDK version and read the changelog before upgrading.

Option 3: the Claude Agent SDK

Here the task is different on purpose, because this is what the Agent SDK is built for: work on files. No tool code at all; the agent uses its built-in Glob, Grep and Read.

import { query } from "@anthropic-ai/claude-agent-sdk";

for await (const message of query({
  prompt: "List every TODO comment under src/ with file and line number.",
  options: {
    model: process.env.MODEL ?? "claude-sonnet-5-5",
    cwd: process.env.REPO_DIR ?? process.cwd(),
    allowedTools: ["Read", "Glob", "Grep"], // read-only: no Edit, no Bash
    maxTurns: 10,
  },
})) {
  if (message.type === "result") {
    console.log(message.subtype === "success" ? message.result : message.subtype);
    console.log({ turns: message.num_turns, cost_usd: message.total_cost_usd, usage: message.usage });
  }
}

It reads ANTHROPIC_API_KEY from the environment (it does not load .env files) and needs Node.js 18.14 or later. The final result message reports the turn count, the token usage and a cost estimate in dollars, so you do not have to compute it yourself.

One option name is a trap. allowedTools only pre-approves tools so they run without a permission prompt; it does not remove the others from Claude's context. To limit which tools exist at all, use the tools option, or disallowedTools to remove specific ones.

What we measured

We ran each of the three examples once on Claude Haiku 5.5 (MODEL=claude-haiku-5-5), the cheapest current model, and read usage from the responses.

	Requests	Input tokens	Output tokens	Cost
Own loop	2	under 1,000	134	under $0.0002
Tool Runner	2	about 1,000	114	under $0.0002
Agent SDK	2	86,630 (43,487 cache write, 43,139 cache read)	483	$0.0095

Haiku 5.5 list prices on 2026-11-03, per million tokens: $0.10 input, $0.20 for a 1-hour cache write, $0.01 for a cache read, $0.50 output (official pricing).

The two tasks are different, so read the table for its shape, not its ratio. The shape is clear: the Agent SDK sends about 43,000 tokens of its own system prompt and tool definitions with every request. The SDK caches that prefix with a 1-hour cache, so after the first request it is billed at the cache-read price. On Haiku it barely matters. On Claude Sonnet 5.5 ($2 input, $4 one-hour cache write, $0.10 cache read) the same prefix costs about $0.17 to write once per hour and about $0.004 per request after that, before any of your own tokens.

The own loop and the Tool Runner cost the same, because they send the same request. The runner saves you code, not tokens. Also note that their small system prompt was not cached at all (cache_read_input_tokens: 0): prompts shorter than the model's minimum cacheable length are silently not cached. With a real system prompt and a handful of tools, you cross that threshold.

Per-token prices are identical across the three options. What differs is how many tokens each one spends per request, and who decides.

What you control in each

	Own loop	Tool Runner	Agent SDK
System prompt	Yours, every byte	Yours	Claude Code's, which you can customize
Tools	Only yours	Only yours	Built-in file, shell and web tools, plus yours via MCP
Where tools run	Your code	Your code	The machine running the SDK (files, shell)
Per-request hooks	Anything	Per-iteration access	Lifecycle hooks (before and after tool calls, and more)
Fixed overhead per request	Your prompt and tools	Your prompt and tools	About 43,000 tokens, cached
Dependency risk	Stable Messages API	Beta helper	A separate SDK that tracks Claude Code releases
Runs on edge or serverless	Yes	Yes	Needs a Node or Python process that can run a binary

Where the Agent SDK wins

If your agent works on files and code, the Agent SDK saves you weeks. Reading, searching and editing files, running commands, asking for permission before anything destructive, splitting work into subagents, resuming a session tomorrow: all of that already exists and has been hardened by everyone who uses Claude Code. A bot that opens a pull request to fix lint errors, an internal tool that triages failing CI runs, an agent that migrates a config format across a repo: these are a weekend with the Agent SDK and a month by hand.

It also suits internal tools, where the person running the agent is a developer and a 43,000-token prefix is noise next to the value of the result.

Where your own loop wins

Most customer-facing SaaS features are not coding agents. They search your database, call your API, draft an email. For those you want a system prompt of a few hundred tokens, five tools you wrote, a cache hit on every turn, and one row in a usage_events table per request. That is the case for options 1 and 2.

Your own loop also wins when:

You bill users per request. Metering, plan caps and rate limits sit naturally around each messages.create() call. We wrote up the whole metering layer in How to meter Claude API costs per user in a Next.js SaaS.
You route models by plan. Haiku 5.5 for free users and Sonnet 5.5 for paid ones is one variable, with nothing else in the harness to retune.
You deploy to the edge or to serverless functions. No shell, no binary, short time limits. A plain HTTP loop fits; a bundled CLI does not.
Your users' input should never reach a shell. If the agent has no Bash tool, nobody has to reason about what a prompt injection could make it run.

The middle path: the Tool Runner

Pick the Tool Runner when you want everything from option 1 except the loop code. Zod schemas double as input validation, the loop handles parallel tool calls and errors for you, and iterating over the runner still gives you each response for metering. Accept that it is beta: pin the version, and keep the manual loop above in mind as the escape hatch, since both send identical requests.

Decision table

You are building	Start with	Why
A coding agent or developer tool	Agent SDK	Files, shell, permissions and subagents already exist
An internal automation that edits a repo	Agent SDK	The prefix cost is noise next to the time saved
A customer-facing feature with your own tools	Own loop or Tool Runner	Small prompt, full caching, per-request metering
A prototype this afternoon	Tool Runner	Least code that still uses your tools
Something that must run on the edge	Own loop	No binary, no beta dependency

If you are unsure, start with the Tool Runner. Moving from it to a manual loop is a small refactor, because the requests are the same.

Questions or corrections? Please leave them in the comments on the original post. We check those every day and answer there, while replies here on DEV can take a while.

Disclosure: this article was written with the help of AI (Claude). Facts, prices and code were checked against the official documentation and reviewed before publishing.

Originally published on the Agentwright blog.

How to meter Claude API costs per user in a Next.js SaaS (with code)

Agentwright — Tue, 03 Nov 2026 02:30:14 +0000

Originally published on the Agentwright blog.

If you charge users for an AI feature, you need to know what each of them costs you. Not in aggregate, on the Anthropic invoice at the end of the month, but per user, per request, in real time, so you can cap free accounts, show a usage page, and find out which 2% of users are 60% of your bill.

This post walks through a small metering layer for a Next.js 15 app with Drizzle + Postgres and the official @anthropic-ai/sdk. Everything below is plain TypeScript you can drop into an existing project. The design is small on purpose: one table, one pure pricing module, two functions.

What the API gives you

Every response from the Messages API carries a usage object. For a streaming request, the SDK's finalMessage() returns the assembled message with the same fields:

const stream = client.messages.stream({ model, max_tokens, system, tools, messages });
const final = await stream.finalMessage();

final.usage.input_tokens;                 // uncached input, billed at the base input price
final.usage.output_tokens;                // billed at the output price
final.usage.cache_read_input_tokens;      // served from the prompt cache (cheap)
final.usage.cache_creation_input_tokens;  // written to the prompt cache (slightly more than base)

Two things trip people up here.

First, the four numbers are billed at four different rates. If you only store input_tokens + output_tokens, your cost estimate will be wrong as soon as you enable prompt caching, which you should, because a cached system prompt plus tool definitions is the single biggest saving in an agent loop.

Second, an agent turn is not one request. A loop that calls a tool three times makes four API requests, and each one has its own usage. Meter per request, not per user message.

The official prices (checked 2026-11-01)

These are the Claude API list prices from the official pricing page, in USD per million tokens. Verify them yourself before copying; pricing pages change, and this table will not update itself.

Model	Input	Output	Cache write (5 min)	Cache read
Claude Opus 5.5	$4.00	$20.00	$5.00	$0.20
Claude Sonnet 5.5	$2.00	$10.00	$2.50	$0.10
Claude Haiku 5.5 (prompt ≤ 100K tokens)	$0.10	$0.50	$0.125	$0.01
Claude Haiku 5.5 (prompt > 100K tokens)	$0.50	$2.50	$0.625	$0.05

Notes that matter for metering:

Cache reads on Opus 5.5 and Sonnet 5.5 are 5% of the base input price, not the 10% that older models use. Cache reads on Fable 5.1 are 2.5%. Do not hard-code "cache read = input / 10".
The 5-minute cache write is 1.25x the base input price; the 1-hour write is 2x. The code below assumes the 5-minute TTL. If you use ttl: "1h", add a separate rate.
Haiku 5.5 is the only current model with a long-context tier. The whole request is priced at the higher tier when its prompt (including cache reads and writes) exceeds 100,000 tokens.
Opus 5.5, Sonnet 5.5 and Haiku 5.5 all have a 1M-token context window, and apart from the Haiku tier above, there is no long-context surcharge.
The Batch API is 50% off input and output. If you move background jobs to batches, meter them at half price.

Step 1: a pure pricing module

Keep prices in a module with no database import, so your eval runner, a cost calculator in the browser, and the server can all use the same numbers. Store cost as an integer number of micro-dollars (1e-6 USD). Floating-point dollars in a SUM() look fine for a week and then drift.

// lib/billing/pricing.ts
export type ModelPrice = { in: number; out: number; cacheRead: number; cacheWrite: number };

/** USD per million tokens. Source: platform.claude.com/docs/en/about-claude/pricing */
export const PRICE_PER_MTOK: Record<string, ModelPrice> = {
  "claude-opus-5-5":   { in: 4,   out: 20,  cacheRead: 0.2,  cacheWrite: 5 },
  "claude-sonnet-5-5": { in: 2,   out: 10,  cacheRead: 0.1,  cacheWrite: 2.5 },
  // Prompts over 100K tokens are billed at the higher Haiku tier (see haikuPrice below).
  "claude-haiku-5-5":  { in: 0.1, out: 0.5, cacheRead: 0.01, cacheWrite: 0.125 },
};

const HAIKU_LONG: ModelPrice = { in: 0.5, out: 2.5, cacheRead: 0.05, cacheWrite: 0.625 };

export type Usage = { input: number; output: number; cacheRead?: number; cacheWrite?: number };

function priceFor(model: string, u: Usage): ModelPrice {
  const p = PRICE_PER_MTOK[model];
  if (!p) throw new Error(`Unknown model pricing: ${model}`);
  if (model === "claude-haiku-5-5") {
    const promptTokens = u.input + (u.cacheRead ?? 0) + (u.cacheWrite ?? 0);
    if (promptTokens > 100_000) return HAIKU_LONG;
  }
  return p;
}

/** Cost in micro-dollars (1e-6 USD) so it can be stored as a bigint. */
export function costMicroUsd(model: string, u: Usage): number {
  const p = priceFor(model, u);
  const usd =
    (u.input * p.in +
      u.output * p.out +
      (u.cacheRead ?? 0) * p.cacheRead +
      (u.cacheWrite ?? 0) * p.cacheWrite) / 1e6;
  return Math.round(usd * 1e6);
}

export function formatUsd(microUsd: number, digits = 4): string {
  return `$${(microUsd / 1e6).toFixed(digits)}`;
}

A quick sanity check: one Sonnet 5.5 request with 3,000 uncached input tokens, 8,000 cache-read tokens and 600 output tokens costs 3,000 × $2 + 8,000 × $0.10 + 600 × $10 = $6,000 + $800 + $6,000 per million, which is $0.0128. With costMicroUsd that is 12800. Without caching the same prompt would have been 11,000 × $2 + 600 × $10 = $0.028, so the cache cut this request's cost by more than half. That is why you must meter cache reads separately: if you bill them as full input you will overcharge users and under-sell your own margins.

Throwing on an unknown model is deliberate. A silent ?? 0 means a new model ships, someone sets AGENT_MODEL, and every request is recorded as free.

Step 2: one table

// lib/db/schema.ts (excerpt)
export const usageEvents = pgTable(
  "usage_events",
  {
    id: uuid("id").primaryKey().defaultRandom(),
    userId: uuid("user_id").notNull().references(() => users.id, { onDelete: "cascade" }),
    model: text("model").notNull(),
    inputTokens: integer("input_tokens").notNull(),
    outputTokens: integer("output_tokens").notNull(),
    cacheReadTokens: integer("cache_read_tokens").notNull().default(0),
    cacheWriteTokens: integer("cache_write_tokens").notNull().default(0),
    costMicroUsd: bigint("cost_micro_usd", { mode: "number" }).notNull(),
    createdAt: timestamp("created_at").defaultNow().notNull(),
  },
  (t) => [index("usage_events_user_created_idx").on(t.userId, t.createdAt)],
);

export const plans = pgTable("plans", {
  id: text("id").primaryKey(), // free | pro | team
  monthlyTokenCap: bigint("monthly_token_cap", { mode: "number" }).notNull(),
  overageAllowed: integer("overage_allowed").notNull().default(0),
});

One row per API request. Store the raw token counts and the computed cost. The raw counts let you re-price history if you discover a bug or Anthropic changes a price; the stored cost makes the usage page a single SUM().

The composite index on (user_id, created_at) is the only one you need. Every query below is "this user, this month".

Step 3: record and enforce

// lib/billing/metering.ts
import { db } from "@/lib/db";
import { usageEvents, users, plans } from "@/lib/db/schema";
import { and, eq, gte, sql } from "drizzle-orm";
import { costMicroUsd, type Usage } from "./pricing";

export async function recordUsage(userId: string, model: string, u: Usage) {
  await db.insert(usageEvents).values({
    userId,
    model,
    inputTokens: u.input,
    outputTokens: u.output,
    cacheReadTokens: u.cacheRead ?? 0,
    cacheWriteTokens: u.cacheWrite ?? 0,
    costMicroUsd: costMicroUsd(model, u),
  });
}

export async function assertWithinCap(userId: string): Promise<void> {
  const monthStart = new Date();
  monthStart.setUTCDate(1);
  monthStart.setUTCHours(0, 0, 0, 0);

  const [user] = await db.select().from(users).where(eq(users.id, userId));
  const [plan] = await db.select().from(plans).where(eq(plans.id, user.plan));
  const [{ used }] = await db
    .select({
      used: sql<number>`coalesce(sum(${usageEvents.inputTokens} + ${usageEvents.outputTokens}), 0)`,
    })
    .from(usageEvents)
    .where(and(eq(usageEvents.userId, userId), gte(usageEvents.createdAt, monthStart)));

  if (Number(used) >= plan.monthlyTokenCap && !plan.overageAllowed) {
    const err = new Error("Monthly token cap reached. Upgrade your plan to continue.");
    (err as any).code = "CAP_REACHED";
    throw err;
  }
}

The cap here is in tokens, not dollars, because that is what users can reason about on a pricing page ("2 million tokens a month"). If you prefer dollar caps, sum cost_micro_usd instead; the table supports both.

One honest limitation: this check reads before the request and writes after it, so a user can exceed the cap by one request. For a chat product that is fine. If you need a hard ceiling, reserve an estimate before the call (count input tokens with client.messages.countTokens and assume max_tokens output), then reconcile after.

Step 4: wire it into the agent loop

Metering goes around each request in the loop, not around the whole conversation:

// lib/agent/loop.ts (excerpt)
for (let turn = 0; turn < MAX_TURNS; turn++) {
  if (opts.userId) await assertWithinCap(opts.userId);

  const stream = client.messages.stream({
    model,
    max_tokens: opts.maxTokens ?? 8192,
    system: [{ type: "text", text: opts.system, cache_control: { type: "ephemeral" } }],
    tools,
    messages,
  });
  stream.on("text", (t) => opts.onText?.(t));
  const final = await stream.finalMessage();

  if (opts.userId) {
    await recordUsage(opts.userId, model, {
      input: final.usage.input_tokens,
      output: final.usage.output_tokens,
      cacheRead: final.usage.cache_read_input_tokens ?? 0,
      cacheWrite: final.usage.cache_creation_input_tokens ?? 0,
    });
  }

  messages.push({ role: "assistant", content: final.content });
  if (final.stop_reason !== "tool_use") return messages;
  // ...execute tool calls, push results, continue
}

Making userId optional is a small thing that pays off: your eval runner and smoke tests can call the same loop without a database, and production always passes a user.

In the API route, catch CAP_REACHED and return something the client can act on. We stream server-sent events, so the route emits { event: "error", data: { code: "CAP_REACHED", message } } and the chat UI shows an upgrade link instead of a generic failure.

Step 5: the usage page

With cost already in the table, the per-user page is three aggregate queries. The one users look at most is cost by model this month:

const byModel = await db
  .select({
    model: usageEvents.model,
    requests: sql<number>`count(*)`,
    tokens: sql<number>`coalesce(sum(${usageEvents.inputTokens} + ${usageEvents.outputTokens}), 0)`,
    costMicroUsd: sql<number>`coalesce(sum(${usageEvents.costMicroUsd}), 0)`,
  })
  .from(usageEvents)
  .where(and(eq(usageEvents.userId, userId), gte(usageEvents.createdAt, monthStart)))
  .groupBy(usageEvents.model)
  .orderBy(desc(sql`sum(${usageEvents.costMicroUsd})`));

Postgres returns sum() as a string for bigint columns, so wrap results in Number() before sending them to the client. Add a daily series for the last 30 days (date_trunc('day', created_at)) and the 20 most recent requests, and you have everything a user needs to understand their bill. The same queries, without the userId filter, are your admin view.

The mistakes to avoid

Metering only the final turn. A tool-heavy conversation can make five requests for one user message. If you record one row per message, the bill looks several times cheaper than it is.
Treating cache reads as input. On a well-cached agent prompt, most input tokens are cache reads at 5% to 10% of the base price. Billing them as full input overstates cost by a wide margin, and the multiplier differs per model.
Floats for money. SUM(cost_usd) across a month happily produces 0.30000000000000004. Integer micro-dollars avoid the problem entirely.
Treating prices as constants. Keep the source URL and the check date next to the table, and re-check it when a model ships. Pricing is a dependency, not a fact.

What this does not cover

Rate limiting (requests per minute per plan) is a separate concern and lives in its own table. Neither does this post cover overage billing: overageAllowed lets a plan continue past the cap, and a monthly job can sum cost_micro_usd and create a usage-based charge with your payment provider.

Questions or corrections? Please leave them in the comments on the original post. We check those every day and answer there, while replies here on DEV can take a while.

Disclosure: this article was written with the help of AI (Claude). The prices were checked against Anthropic's official pricing page on 2026-11-01, and the code was reviewed before publishing.