DEV Community: Dimitris Kyrkos

Your AI-generated tests aren't testing your code. They're testing the AI's blind spots.

Dimitris Kyrkos — Fri, 04 Sep 2026 06:54:42 +0000

Intro

There's a pitch behind every "AI writes your tests too" workflow: more coverage, less manual toil, a safety net that used to take a sprint now takes minutes.

The pitch skips over what that safety net is actually made of. When the same model writes the implementation and the test suite, you haven't added a second, independent check. You've asked one reviewer to grade its own homework and handed you the green checkmark as if someone else had signed off.

The blind spot loop

A model reasons about a function once, forms an implicit set of assumptions (input shapes, timezone handling, what counts as "empty"), and writes the implementation against those assumptions. Ask the same model to write tests for that function, and it doesn't re-derive correct behavior from scratch. It writes tests against the same assumptions it just used to write the code. If it assumed dates always arrive as ISO strings in UTC, the implementation assumes that, and the tests assume it too. The suite goes green. The assumption is still wrong.

Tests that pass for the wrong reason

(Illustrative, not a specific case, but recognizable to anyone who's shipped an AI-generated suite.) Picture a discount-calculation function where the model assumes quantities are always positive integers. The implementation skips a negative-quantity check. The generated tests exercise 1, 5, and 100, because those are the "normal" values a model reaching for plausible test data will reach for. Nothing ever asks what happens at -1 or 0, because neither pass, the code or the tests, ever considered them worth asking about. Coverage tooling reports 100% on this function. The bug ships anyway.

Coverage becomes a false signal

High line or branch coverage from an AI-authored suite tells you the code paths were exercised, not that the right inputs exercised them. A suite can hit every line of a function and still never send it a null, an empty array, a duplicate key, or a value at a type boundary, if the author, human or model, never imagined those as possibilities in the first place. Coverage percentage was never built to detect a shared blind spot. It just counts what got tried.

What to keep human, and what to hand to AI

The fix isn't "stop using AI for tests." It's separating the two jobs testing actually does:

Defining correct behavior (the assertions) should come from a human who understands the spec or the business rule the function is supposed to honor, decided independently of however the implementation happens to work.
Generating volume and variety (mock data, randomized inputs, edge-case permutations) is exactly what AI is good at, and doing it well doesn't require the model to have written the implementation.
Human review time is best spent on boundary conditions specifically: zero, negative, empty, duplicate, malformed, concurrent, because that's statistically where both human and AI implementations tend to fail first.

This isn't really a testing problem. It's a correlated-error problem wearing a green checkmark. Two independent reviewers catch different mistakes because they're independent. One reviewer checking its own work twice catches the same mistakes it already missed, twice.

Where's your line? Do you let AI touch your test assertions at all, or only the scaffolding, mocks, fixtures, input generation, around them?

Semantic caching isn't a cost-saving hack. It's an admission that most "AI features" are FAQ bots in disguise.

Dimitris Kyrkos — Tue, 01 Sep 2026 09:10:45 +0000

Intro

There's a pitch behind every new AI-powered feature: it understands anything a user throws at it. Open-ended, flexible, genuinely intelligent.

The pitch is half true. What it leaves out is that in most production systems, "anything a user throws at it" collapses into the same twenty questions asked twenty different ways. A support bot doesn't get asked infinite variations of physics, it gets asked "what's your return policy," "how do I get a refund," and "can I send this back" on a loop, forever.

That gap, between the imagined variety of user queries and the actual repetition underneath it, is where the token bill lives.

The naive approach: every request hits the model

Say a team ships a support assistant. Every user message goes straight to a frontier model: embed the question, retrieve context, generate an answer, return it. It works. It's also expensive in a very specific way, identical intent, paid for from scratch, every single time.

(Illustrative, not a specific case, but recognizable to anyone who's watched an LLM API bill by day one versus day thirty.)

Exact-match caching almost works, then doesn't

The obvious fix is a cache: hash the incoming query, check for a hit, skip the model call if found. This works exactly once, for the exact same string. "How do I reset my password" and "forgot my password, help" are the same question and two different cache keys. Exact-match caching is a solution for a problem language doesn't actually have, users asking things identically.

Embeddings turn paraphrase into a solvable problem

Semantic caching replaces the hash with a vector. Embed the incoming query, run a nearest-neighbor search against previously answered queries, and if the closest match clears a similarity threshold, serve the cached response instead of calling the model at all. Below the threshold, call the model as normal and add the new query/response pair to the cache for next time.

This is the actual mechanism at work: not "the AI understood the question was similar," but a distance calculation in vector space with a cutoff you chose.

A stale cache is worse than no cache

Here's the failure mode nobody budgets for. A support answer gets cached in March. The return policy changes in April. The cache doesn't know that, it just has a vector close enough to keep matching, and it will confidently serve outdated information for as long as the entry lives. A cache without an expiration policy isn't a cost optimization, it's a bug that pays for itself to keep running.

Threshold tuning is the whole game

Set the similarity threshold too loose and you'll serve March's return policy answer to an April refund question, because the two "look" adjacent in vector space even though the real answer changed. Set it too tight and you get single-digit hit rates, because no two users phrase anything identically. There's no universal number here, only a number you have to measure against your own traffic.

Build it yourself, or don't

If you're at the scale where cache hit rate is a line item on your infra bill, and your domain has stable, well-defined intents (support, FAQ, internal tooling), rolling your own semantic cache with something like pgvector or Redis is a weekend project, not a research problem. If your queries are genuinely open-ended and low-repeat (creative generation, one-off analysis), you're optimizing for a hit rate that doesn't exist, and the caching layer is wasted engineering effort.

Semantic caching isn't really an AI technique. It's ordinary distributed-systems caching (TTLs, invalidation, cache keys) with an embedding standing in for the key. The AI part was never the hard part.

We ran into a version of this question building Cyclopt Companion's analyzers: a one-line diff shouldn't necessarily trigger a full re-analysis from scratch, and figuring out what counts as "close enough to skip" turned out to be the actual engineering problem, not the analysis itself.

What's your current cache hit rate on user-facing LLM calls, and have you actually measured it, or is it a guess?

AI autocomplete isn't a productivity tool. It's a judgment test you take every few seconds.

Dimitris Kyrkos — Fri, 28 Aug 2026 06:52:03 +0000

Intro

There's a pitch behind every AI coding assistant: it makes you faster. Fewer keystrokes, less boilerplate, more shipped features per sprint.

The pitch is half true. What it leaves out is the gap between a tutorial demo and a real codebase under real pressure. In a demo, every suggestion is correct because the demo was built to make the suggestion look correct. In production, the assistant doesn't know your architecture, your team's conventions, or the ticket you're actually trying to close. It just knows what tends to come next in code that looks like yours.

That gap is where the noise lives.

The instant-accept trap

Say a developer is mid-flow, wiring up a new endpoint. The assistant suggests a validation helper that looks reasonable, so they hit tab. It compiles, tests pass, they move on.

Three weeks later a teammate finds two nearly identical validation helpers in the codebase: one written by a human eight months ago, one autocompleted last sprint. Nobody meant to duplicate logic. The suggestion was locally correct and globally redundant, and nothing about "correct code that compiles" caught that.

(This is an illustrative scenario, not a specific incident, but most teams running Copilot or similar tools for more than a few months will recognize the shape of it.)

Architecture creep, one suggestion at a time

No single autocompleted line breaks your architecture. That's exactly the problem. An assistant trained on generic patterns will happily suggest a new abstraction, a new dependency, a new way of doing something you already do three other ways elsewhere in the codebase, because it has no visibility into "elsewhere." Accept enough of these one at a time and the codebase drifts into a dozen small dialects of the same idea, none of them wrong in isolation.

The review tax

The real cost isn't the code that's obviously bad, that gets caught. It's the code that's plausible enough to pass a quick glance and wrong enough to need real review time later. If you accept every suggestion without evaluating it against the code you already have, you're not saving time, you're deferring the thinking to code review, or worse, to whoever debugs it in production. Teams that measure this honestly often find they're spending more time reviewing and pruning generated code than they would have spent writing the smaller, more deliberate version themselves.

It's not an AI problem, it's a taste problem

Strip away the tooling and this isn't new. Junior engineers have always generated more code than senior engineers, because judgment about what not to write is a skill that takes time to build. AI assistants didn't invent that gap, they just made it faster to fall into, because the suggestion arrives before you've had time to ask whether you need it.

Autocomplete on, or on-demand only?

There's no universal right answer here, but there is a decision worth making deliberately instead of by default:

Always-on autocomplete works if you already have strong instincts for when to reject a suggestion, and you treat every accepted line as your own code, not the tool's.
On-demand AI for specific tasks (boilerplate, tests, migrations) forces a moment of intent before you invoke it, which some teams find keeps architecture decisions in human hands.
Research and planning only keeps AI out of the diff entirely and uses it to think faster, not to type faster. (We lean on this mode a lot while building Cyclopt Companion, mostly to keep the assistant out of the actual diff until a human has decided the shape of the change.)

None of these are wrong. They're trade-offs between speed and friction, and the right one depends on how much you trust your own review discipline in the moment.

So: do you keep autocomplete on at all times, use AI only on-demand for specific tasks, or limit it to research and planning? And has your answer changed since you started using these tools?

AI-Generated Code Doesn't Have a Readability Problem. It Has a Style-Enforcement Problem Wearing an AI Hat.

Dimitris Kyrkos — Tue, 25 Aug 2026 13:18:22 +0000

Intro

Ask an AI assistant to "handle the checkout flow" and it will hand you a working function in about four seconds. Validate the cart, call three different APIs, format a receipt, log the analytics event, catch whatever errors show up, all in one block, three levels of nesting deep. It runs. It passes the one test anyone bothered to write. And it is, structurally, a small readability crisis waiting for whoever opens that file next.

That gap, between code that works and code a human can actually follow six months later, is where most of the real cost of AI-assisted development quietly accumulates. Here's where it actually shows up. (Illustrative composites drawn from common patterns, not specific incidents.)

The function that does six things because nobody told it not to

A generated function rarely sets out to do too much. It just keeps absorbing responsibility one prompt at a time: first it validates input, then someone asks it to also handle a retry, then to also log the outcome, then to also format the response for the frontend. Each addition looks reasonable in isolation. Read top to bottom six weeks later, it's a single function holding four unrelated jobs, and splitting it back apart means first reverse-engineering which lines belong to which job.

Names that describe the prompt, not the domain

Generated code tends to name things after the immediate task rather than the concept it represents: data2, tempResult, processedItems, handleStuff. None of these are wrong in the sense of breaking anything. They're wrong in the sense that a teammate reading the code six months from now has to run it mentally just to figure out what tempResult actually holds, instead of the name just telling them.

A slightly different helper function, invented every time

Ask a model to format a date in one file and it writes formatDate. Ask it again in another file, in the same session even, and it might write toDateString, doing almost the same thing with a slightly different edge case handled. Nothing is technically duplicated, so no linter flags it, but the codebase slowly fills with near-identical helpers that all do roughly the same job slightly differently, because each generation has no memory of what already exists two files over.

Style that resets at every file boundary

File A uses early-return guard clauses. File B nests every conditional three deep because that's what was statistically nearby in training for that particular pattern. Neither is "wrong" on its own, a reviewer skimming one file at a time won't necessarily flag it, but navigating the codebase starts to mean re-learning the local dialect every time you open a new file, because there isn't one.

None of this is a code-quality problem in the traditional sense, where someone wrote something sloppy and a linter catches it. It's a missing style contract problem: nothing external is constraining the model toward the conventions your team already agreed on, so by default it reaches for whatever's statistically common across its training data, which is rarely what's locally correct for your codebase.

So what do you actually do about it? One option is writing an exhaustive style guide and hoping every prompt includes enough of it, which doesn't scale past a few files and quietly rots the moment someone forgets to paste it. The other is treating readability as something enforced at the commit gate, the same way you'd enforce tests passing, independent of whether a human or a model wrote the line. Prompts don't have memory. Commits do. That's really the whole argument for moving readability enforcement from "hope the prompt was good" to "verify the commit is."

How is your team actually enforcing readability on AI-generated code right now, a style guide baked into the prompt, a linter, code review, or honestly, nothing yet?

Using AI to Code Isn't the Risk. Not Understanding What It Shipped Is

Dimitris Kyrkos — Mon, 17 Aug 2026 07:42:25 +0000

Intro

There's a gap between the way AI-assisted coding gets demoed and the way it actually gets used under a deadline. In the demo, the model writes a function, the developer reads it, understands it, and merges it. Under a deadline, the model writes a function, the developer skims it, it looks plausible, and it merges. Same tool, two completely different outcomes, and the difference isn't visible in the diff.

Call the first one cognitive assistance: the AI removes typing, not thinking. Call the second one cognitive offloading: the AI removes thinking, and the developer just does the typing that's left, mostly pressing "accept." Both look identical in a pull request. They only diverge later, when something breaks and someone has to explain why the system does what it does.

Here's where that gap actually shows up in production. (Illustrative composites drawn from common patterns, not specific incidents.)

The schema nobody actually reasoned about

A developer asks an AI assistant to generate a migration adding a new table with two foreign keys. It looks right, it runs locally, tests pass, it ships. Nobody on the team, including the person who wrote the migration, actually worked through the access pattern the new table would see in production. Three months later, a dashboard query against that table starts timing out under real traffic, because there's no index on the column it's actually filtered by, a detail that would have been obvious to anyone who'd designed the schema by hand instead of approving one.

The stack trace that got forwarded, not read

Someone hits a NullPointerException, pastes the trace into an AI chat, gets back a suggested null check, adds it, ships it. The exception stops. Two sprints later, a related bug shows up somewhere downstream, because the null check treated a symptom, a race condition further up the call stack, not the cause. Nobody connects the two, because nobody who touched the first fix ever understood what was actually racing.

The architecture nobody in the room can defend

In a retro, a tech lead asks why a service makes three synchronous calls to downstream services instead of publishing to a queue. Nobody has an answer, because the pattern was generated during a crunch week, skimmed, and merged. It's not that the pattern is necessarily wrong, it's that no one currently on the team could tell you whether it's wrong, which means no one can safely change it either.

Cognitive offloading isn't really an AI problem. It's a code review discipline problem wearing an AI hat. Teams that had weak review habits before AI tools now get weak review at a much higher volume and velocity, which turns a slow leak into a flood.

A quick gut check, if you want one: before you merge AI-generated code, can you explain it to a teammate without opening the chat history again? Could you have arrived at this approach yourself given more time? Do you know why this pattern and not an obvious alternative? If the honest answer to any of those is no, that's not a blocker, it's just useful information about which side of the line you're currently standing on.

Where do you draw the line between assistance and offloading on your team, is it an explicit rule, part of your review process, or just a gut feeling that kicks in after something's already gone wrong?

Your model doesn't need to pass the bar exam. It needs to parse a log file.

Dimitris Kyrkos — Tue, 04 Aug 2026 11:16:44 +0000

Intro

Every few weeks, another frontier model claims a new benchmark, better reasoning, longer context, higher scores on tests built to measure how humans think. None of that is irrelevant, but it's solving a problem a huge share of enterprise workloads don't actually have. Parsing a structured log line, validating a field against a schema, classifying a support ticket into one of six categories - none of that was ever going to need a model that can debate philosophy or pass the bar exam. It needs a model that's fast, cheap, and right, every time, on a narrow task. That's a different design goal than the one the frontier race is optimizing for, and it points toward small, specialized, locally hosted models instead of the next big release. Below are four places where the gap between "benchmark-optimal" and "production-optimal" actually shows up. (Illustrative composites, not case studies from a specific client.)

Latency budgets don't care how smart the model is

Picture a fraud-detection pipeline calling a hosted frontier model on every transaction. It works fine in testing. Under peak load, p95 latency starts spiking, not because the model reasons badly, but because every call is a network round trip through someone else's queue, competing with every other tenant's traffic on that provider that day. A distilled model of a fraction of the size, running on the same box as the service that calls it, doesn't have a network hop to blame. Latency stays predictable in the low tens of milliseconds because there's no shared queue to wait behind.

Your data boundary is only as strong as your last API call

A team building on patient or financial records sends that data to a third-party endpoint for every inference call. It works, until a compliance review asks a simple question: where does this data physically go, who retains it, and for how long. If the answer is "a provider's infrastructure, under their retention policy," that's a boundary the team doesn't fully control and can't fully audit. A small model running inside their own perimeter turns that into a non-question. There's no external boundary to explain, because the data never left.

A pricing or deprecation decision made by someone else is not a risk you control

A workflow built on top of a hosted model API works well for a year, until the provider raises prices, changes rate limits, or deprecates the exact model version the workflow was tuned against. None of that is a bug in the team's code. It's a business decision made by a company they don't work for, and it forces an unplanned re-engineering effort on someone else's timeline. A locally hosted, version-pinned model doesn't have a roadmap owned by another company deciding when it stops being supported.

A model fine-tuned on your schema beats a generalist prompted around it

Ask a general-purpose frontier model to validate rows against a company's actual database schema, and it will usually get it right, and occasionally hallucinate a plausible-sounding field name that doesn't exist, because it's reasoning from general knowledge about how schemas tend to look, not from the ground truth of this one. A small model fine-tuned on the company's real schema isn't guessing at a pattern, it's seen the exact structure it's being asked to validate against.

Every one of these is really the same story with a different failure mode

A task with a narrow, well-defined shape got handed to a general-purpose tool built to be good at everything, at the cost of being cheap, fast, and predictable at any one specific thing.

Build vs. integrate: when a small local model actually wins

If the task is narrow and repetitive with a stable shape, log parsing, schema validation, ticket classification, a small specialized or fine-tuned model tends to win on latency, cost, and data boundaries. If the task involves genuine ambiguity, novel reasoning, or synthesizing loosely related domains on the fly, a frontier model is still the better tool. A useful test in between: could a domain expert write down the rules for what "correct" looks like on this task? If yes, that's a strong signal a small local model can be trained or fine-tuned to do it more cheaply and predictably than a general model can be prompted to.

Where's the line for you? At what point does reaching for the biggest available model stop being ambition and start being the wrong tool for the job?

The all-purpose agent isn't an architecture. It's a single point of failure with a system prompt.

Dimitris Kyrkos — Fri, 31 Jul 2026 09:06:04 +0000

Intro

In a demo, one agent that "does everything" looks like impressive economy: one prompt, one API call, one mental model. Nobody's testing it against thousands of edge cases a day, and nobody's paying for the tokens burned parsing forty unrelated instructions to answer a two-line question. In production, that same design is where teams spend their first real outage. Below are four failure modes that show up once a monolithic agent hits real traffic. (Illustrative composites, not case studies from a specific client.)

The system prompt that nobody can safely edit

Picture a support agent that started as "answer product questions" and grew, feature request by feature request, into a 4,000-word system prompt covering billing, refunds, technical troubleshooting, and tone guidelines for three different customer segments. Adding a new instruction for edge case #41 quietly changes how the model handles edge case #12, because both are competing for the same attention budget in the same call. Nobody can point to which line caused the regression, because there's no isolation between concerns, only proximity.

Every request pays for capabilities it doesn't use

A single do-everything agent charges the same token bill for a request that just needs a database lookup as it does for one that needs careful reasoning about a refund policy exception. Say a team's simplest, highest-volume intent, "what's my order status," routes through the full monolithic prompt anyway. That's paying premium reasoning costs, on every call, for a task a five-line function could handle for a fraction of the price.

One bad instruction degrades everything downstream

Because a monolithic agent handles routing, task execution, and formatting in a single inference pass, a subtle drift in one part of its behavior, say it starts hedging more on refund questions after an unrelated prompt tweak, has no boundary to stay contained inside. There's no seam where you could catch it before it reaches the user, because the "component" that's misbehaving is the entire agent.

Testing becomes a probability exercise, not an engineering one

With a single-purpose function, you write a test, you know the input, you assert the output. With a 4,000-word do-everything prompt, "testing" often means running the same conversation a dozen times and hoping the failure rate stays under some tolerable threshold. That's not a test suite. That's a weather forecast.

A multi-purpose agent isn't one decision. It's twenty, made by an LLM, at inference time, with nothing watching any of them.

Every one of these failure modes comes from the same root cause: collapsing routing, task execution, and state management into a single non-deterministic call and hoping the model sorts it out consistently, every time, forever. It won't, not because the model is bad, but because that's not what a single inference pass was ever built to guarantee.

Route it, don't merge it

The fix isn't a smarter prompt. It's separating what genuinely benefits from a language model's judgment from what doesn't. A small classifier decides intent. Specialized, single-purpose functions handle each isolated task, several of which may not need an LLM at all. Deterministic code owns routing, state, and final formatting, the parts where you actually want predictability, not creativity. Each piece is small enough to unit test on its own, which is the property the monolith could never give you.

Where's the line for you, at what point does "just add another instruction to the prompt" stop being the pragmatic choice and start being the thing you'll be debugging at 2am?

"Unlimited context" is not a feature. It's technical debt with better marketing.

Dimitris Kyrkos — Mon, 27 Jul 2026 12:08:59 +0000

Intro

Model providers keep shipping bigger context windows: 100k tokens, 200k, over a million in some cases. The marketing pitch is simple: paste in everything, the model will figure out what matters. It's a seductive idea, and it's the wrong default for anything that has to run in production.

Here's the gap. In a tutorial or a demo, "long context" means pasting a document and asking a question once. Nobody times it, nobody pays for the tenth call, and nobody notices what got ignored. In production, the same request runs thousands of times a day, against a system that has to be fast, cheap, and correct every single time. The capacity to accept 100k tokens says nothing about whether stuffing that much into every request is a good idea. Below are four failure modes that show up once the naive approach hits real traffic. (The scenarios are illustrative composites, not case studies from a specific client.)

The latency tax nobody notices until it's live

Say a support tool feeds the model the full history of a customer's last 40 tickets on every single message, "just in case it's relevant." In the demo, that's one call, and it feels instant. In production, every reply now takes 10-12 seconds instead of 2, because the model has to process tens of thousands of tokens before it writes a word. Users don't experience "more context." They experience a slow bot, and slow bots get abandoned mid-conversation.

The margin killer

Token costs compound in a way that's easy to miss in a pilot and brutal at scale. A team billing customers per seat, but paying per token behind the scenes, can end up with COGS that scale with context size, not with usage. If every request carries the same bloated payload regardless of whether that history is relevant, the unit economics quietly break as the customer base grows, well after the pricing model has already been sold.

Lost in the middle

Even when the tokens are technically "in the window," attention isn't uniform across it. Models are demonstrably better at using information near the start and end of a context than information buried in the middle. Picture a compliance assistant given a 100-page policy document, where the one answer relevant to a question sits on page 47. The model can have "read" every token and still miss it, because being in context and being attended to are not the same thing.

Noise that turns into hallucination

Retrieval systems that grab too much, "let's include the top 15 chunks just to be safe," aren't playing it safe. Excess context isn't neutral filler. Loosely related documents give the model more material to blend, misattribute, and confidently combine into an answer that sounds right and isn't. The system doesn't fail loudly. It fails convincingly.

Context management is data engineering wearing an LLM's clothes

Every one of these failure modes is a symptom of the same root cause: treating context as an unlimited bucket instead of a resource with a cost and a decay curve. The fix isn't a bigger window. It's a pipeline: chunking, filtering, ranking, summarizing, retrieving only what the current task actually needs. That's not prompt engineering. It's the same discipline that's underpinned good search and recommendation systems for a decade, applied to a new kind of consumer.

Build it or buy it

If context management is core to your product's differentiation, and it usually is once you're past the demo stage, it's worth owning the retrieval and filtering layer directly rather than treating it as an afterthought bolted onto whichever framework shipped a quickstart. Off-the-shelf vector stores and orchestration libraries are a reasonable starting point for early prototypes, but the ranking logic, the summarization strategy, and the decision about what's relevant to a given task are where the actual product value lives. That's not something you want to be entirely dependent on someone else's defaults for, once you're optimizing for cost and correctness at scale.

What's the worst context-bloat bug you've shipped or debugged? Curious what patterns other people have run into.

Banning ChatGPT Won't Fix Your Shadow AI Problem. It'll Just Hide It.

Dimitris Kyrkos — Thu, 23 Jul 2026 09:32:49 +0000

Intro

Every engineering org has a policy about which AI tools developers are allowed to use. Every engineering org also has a Shadow AI problem, whether leadership has noticed it yet or not.

The gap between the two is predictable. Security approves one internal tool, or none at all. Developers still have deadlines, still get stuck, still need to paste a stack trace somewhere and ask "why is this failing." If the sanctioned option doesn't exist, is slow, or is worse than what's publicly available, developers go find their own. Quietly.

What actually gets pasted into public chat windows

This is where it gets concrete. A developer debugging a failing integration test copies the full stack trace into a public AI chat window to ask what's wrong. The stack trace includes a database connection string. Nobody flagged it because nobody was watching, the tool has no audit log and no data retention policy the company controls.

A different developer is stuck on a tricky business rule and pastes the actual proprietary algorithm, the one the company patented, into a public model to get a "cleaner implementation." That algorithm may now sit inside a public training corpus, permanently, with no way to un-paste it.

A third developer, working through a customer support bug, pastes real customer records, including names, emails, and account details, into a chatbot to understand a data corruption issue. That's a compliance violation the moment it happens, whether or not anyone downstream ever notices.

None of these developers think of themselves as a security risk. They think of themselves as someone trying to ship on time with the tools that actually work.

Why banning the tools doesn't work

The instinct from leadership is usually to block the domains at the network level and call it solved. This doesn't remove the underlying need, it just removes the visibility. Developers switch to personal devices, personal accounts, or a browser extension nobody in IT has heard of. The paste-your-code-into-a-public-model behavior doesn't stop. It just becomes invisible to the company that's supposed to be accountable for it.

That's arguably worse than the original problem. An org with visible Shadow AI usage at least knows it has a problem. An org that's driven the behavior underground has the same exposure with none of the ability to see or measure it.

The actual failure is a tooling gap, not a discipline gap

Developers aren't circumventing policy because they're careless. They're circumventing it because the approved path is slower, worse, or nonexistent, and the unapproved path solves their actual problem in thirty seconds. Every time an organization treats this as a training or discipline problem instead of a tooling problem, it gets the same result: policy on paper, Shadow AI in practice.

The fix isn't a stricter acceptable-use policy. It's giving developers an approved tool that's actually good enough that going around it stops being worth the trouble, one that keeps proprietary code, schemas, and customer data inside a boundary the company actually controls, with the visibility to know it's being used instead of guessing.

Anyone else seeing this play out on their team? Is it a policy problem, a tooling problem, or both?

RAG isn't an AI problem. It's a data engineering problem wearing an AI hat.

Dimitris Kyrkos — Tue, 21 Jul 2026 11:22:54 +0000

The tutorial-to-production gap

Every RAG tutorial follows the same arc. Load some documents. Chunk them. Embed them into a vector database. Wire up a retrieval step before the LLM call. Ask a question, get a grounded answer. Twenty minutes, looks great, feels like the future.

Then you try to run it on real data, with real users, at real scale, and you discover that the AI part of RAG was the easy part. The hard part is everything the tutorial skipped.

The problems that surface after the demo

Document lifecycle management

The tutorial loaded documents once. Production data changes constantly. Documents get updated, versioned, deprecated, deleted. Your vector database now has embeddings for three versions of the same policy document. The model retrieves chunks from the version that was superseded last quarter and confidently presents outdated information as current.

Nobody in the tutorial mentioned that you need a sync pipeline that tracks document state, invalidates stale embeddings, handles partial updates without re-embedding the entire corpus, and deals with the fact that deleting a document from the source doesn't automatically delete its chunks from the vector store.

This isn't an AI problem. This is the same data staleness problem that search engines have solved for decades. If you've never built a search index that handles updates and deletions, your RAG system will serve stale data and nobody will know until a user notices the answer references a policy that changed six months ago.

Chunking that loses context

The default chunking strategy in most tutorials is "split by token count with some overlap." This works on clean, well-structured documents. It fails on real documents in predictable ways.

A table gets split across two chunks. The header is in chunk one, the data is in chunk two. Neither chunk is useful alone. A paragraph references "the conditions listed above" but "above" is in the previous chunk. A legal clause has an exception three paragraphs later that changes the meaning entirely, but the retriever only surfaces the clause, not the exception.

The chunking strategy determines what the model can see. Bad chunking means the model gets fragments that look complete but are missing the context that changes the answer. The model doesn't know it's missing context. It generates a confident response based on incomplete information. The user doesn't know either. Everyone trusts the system until someone manually checks and discovers the answer was based on half a table.

Getting chunking right requires understanding the document structure: section boundaries, table integrity, cross-references, hierarchical relationships between headings and content. This is document engineering, not prompt engineering.

Semantic search isn't enough

Vector similarity search is good at finding conceptually related content. It's bad at finding exact matches. A user asks "what is the maximum liability under contract 2024-0847?" The embedding search returns chunks about liability in general, about similar contracts, about maximum limits in different contexts. The exact contract number might not even surface because embedding similarity doesn't prioritize exact string matches.

Production RAG systems need hybrid retrieval: semantic search for conceptual relevance, keyword/BM25 search for exact matching, and a ranking layer that combines both. Most tutorials only show the semantic path because it's the one that uses embeddings, which is the novel part. The keyword path is "boring traditional search" and gets skipped.

The irony is that the boring traditional search part is what determines whether the system can actually answer specific questions about specific things. Semantic search finds the neighborhood. Keyword search finds the address.

Access control is where it gets really hard

The tutorial assumed everyone can see everything. Production data has access controls. Different users have different permissions. A document that's visible to legal shouldn't surface in retrieval for marketing. Patient records should only be retrievable by authorized clinicians. Financial data has regulatory constraints on who can access it.

This means your retrieval layer needs to enforce the same ACLs as your source systems. At query time. For every request. Without being so slow that the response takes 30 seconds.

Most vector databases weren't designed with fine-grained access control in mind. Bolting ACLs onto a system that doesn't natively support them creates either security gaps (filtering after retrieval, which means unauthorized chunks were still read) or performance problems (filtering before retrieval, which limits the candidate pool and degrades relevance).

Getting this right is a security and access management problem, not an AI problem.

RAG is search infrastructure with an LLM on top

The pattern that keeps emerging: every hard problem in production RAG is a problem that existed before LLMs.

Document lifecycle management is a search indexing problem. Chunking is a document processing problem. Hybrid retrieval is a search relevance problem. Access control is an authorization problem. Monitoring for quality degradation is an observability problem.

The LLM is the last mile. It takes the retrieved chunks and generates a natural language response. That part works remarkably well. Everything upstream of it, the part that determines what chunks the model actually sees, is traditional data engineering and search infrastructure.

If your team doesn't have experience with search indexing, document processing pipelines, and access control at the data layer, the RAG system will struggle. Not because the AI is bad, but because the retrieval is bad, and retrieval is a solved problem in the search world that most AI teams are rediscovering from scratch.

The build vs integrate decision

Before building RAG from scratch, check whether your problem is actually "I need semantic search over my documents" or "I need a search engine." If it's the second one, mature search infrastructure (Elasticsearch, Solr, or even a well-configured database with full-text search) might get you 80% of the way there without the operational complexity of a vector database, embedding pipeline, and chunking strategy.

RAG makes sense when you genuinely need the LLM to synthesize information from multiple sources into a coherent answer. If your users are asking specific questions that have specific answers in specific documents, traditional search with good ranking might be the simpler tool.

Same principle as the last post: start with the friction, not the technology.

What's been the hardest production RAG problem your team has hit? Was it the AI part or the data engineering part?

The fallacy of "AI-first." Start with the friction, not the technology.

Dimitris Kyrkos — Fri, 17 Jul 2026 13:01:56 +0000

The label that gets the sequence backwards

"AI-first" has become a branding exercise. Startups put it in their pitch decks. Enterprise teams put it in their strategy documents. Innovation labs put it on their slide templates.

The problem isn't ambition. The problem is sequence. "AI-first" starts with the technology and goes looking for problems to apply it to. That's backwards, and it leads to a specific, predictable pattern of failure.

What happens when you start with the technology

When the starting question is "how can we use AI here?" the team works backwards from the solution to find a problem worth solving. This sounds productive. In practice, it creates three failure modes that show up consistently.

Over-engineered systems. A team builds an LLM-powered workflow to classify support tickets. The model requires prompt engineering, embedding infrastructure, a vector database, and ongoing fine-tuning. The accuracy settles around 91%. A senior support agent looks at the same problem and says "we could do this with five keyword rules and a regex." The keyword approach would have been 95% accurate, cost nothing to run, and taken a day to build. But nobody asked whether AI was the right tool because the mandate was AI-first.

Expensive fragility. AI systems have operational overhead that traditional software doesn't. They hallucinate. They drift. They require monitoring for quality degradation. They cost real money per inference. When AI is the right tool, that overhead is justified by the value it creates. When AI was forced into a workflow where a simpler solution would have worked, you're paying the operational tax of an AI system for a problem that didn't require one.

Maintenance burden nobody scoped. The prototype works in the demo. The team ships it. Six months later, the model's performance has degraded because the data distribution shifted. The prompt that worked in March produces different results in September. The embedding index needs rebuilding. The vector database needs scaling. None of this was budgeted because the initial estimate was based on getting it working, not on keeping it working.

Start with the friction instead

The teams I've seen build the most effective AI systems don't start with AI. They start with a specific operational friction and then evaluate whether AI is the simplest tool to resolve it.

Where is the operational bottleneck? Not "where could AI add value" but "what is actually slow, expensive, or error-prone right now." The bottleneck is the starting point because it's concrete, measurable, and has a clear before/after.

What is causing the delay in decision-making? Sometimes the delay is that a human needs to process information that arrives faster than they can read it. AI is excellent for that. Sometimes the delay is that two departments use different ticket systems and nobody can see the full picture. AI won't fix that. A shared dashboard will.

Is AI actually the simplest tool for this? This is the question that gets skipped most often. A database query that runs in milliseconds. A rule engine that handles the known cases. A spreadsheet formula that does the calculation. A cron job that sends the reminder. These aren't exciting. They're also not fragile, don't hallucinate, don't require monitoring for drift, and don't cost per inference.

The most impressive engineering is the code you didn't write

There's a cultural problem underneath the "AI-first" label. It implies that using AI is inherently more sophisticated than not using it. That's wrong. Using AI where it's not needed is over-engineering, which is the opposite of sophistication.

The most effective engineering decisions are often the ones where someone looked at a problem and said "we don't need AI for this." Not because they're against AI, but because they evaluated the problem honestly and chose the tool that matched the complexity of the problem, not the complexity of the tool.

AI is powerful when applied to problems that actually require it: ambiguous inputs, unstructured data, pattern recognition across large datasets, natural language understanding, tasks where the decision space is too large for rules. For everything else, the simpler tool is the better tool.

Have you seen "AI-first" mandates lead to over-engineered solutions on your team? What's the best example of a problem where AI was proposed but a simpler solution turned out to be better?

AI frameworks make the first 10% feel like magic. The other 90% is where they break you.

Dimitris Kyrkos — Tue, 14 Jul 2026 12:53:10 +0000

The magic window

Every AI framework ships with a demo that looks incredible. Spin up an agent in five lines. Chain three tools together in ten. Build a chatbot with memory in fifteen minutes. The getting-started experience is deliberately frictionless. And it works. You feel productive immediately.

Then you try to do something real.

Where the abstractions become walls

The problem isn't that these frameworks are badly built. Many of them are impressively engineered. The problem is that they're designed to optimize for the first 10% of development at the cost of the remaining 90%.

That remaining 90% is where you need to manage raw API payloads when the abstraction doesn't expose the field you need. Where you need custom retry logic because the default exponential backoff doesn't account for the rate limit patterns your specific provider uses. Where you need to track token costs per user, per session, per feature, and the framework only gives you aggregate totals. Where an agent chain fails silently at step four and you're debugging through three layers of abstraction to figure out what state was passed between steps.

Heavy abstractions help when everything works. They become walls when something doesn't. And in production, something always doesn't.

The debugging tax

This is the part that hits hardest in practice. When your lightweight wrapper breaks, you read the error, look at your code, and fix it. The call stack is yours. The state is visible. The flow is explicit.

When a framework breaks, you're reading through someone else's abstractions trying to figure out what happened between your input and the error. The state got transformed somewhere inside a chain you didn't write. The retry logic fired but you can't tell how many times because it's handled internally. The error message references an internal class name that doesn't appear in the documentation.

The time you saved in the first 10% gets paid back with interest during debugging. And debugging is where production teams spend most of their time.

The cost tracking problem nobody mentions

This one is specific to AI infrastructure but it matters a lot at scale. Most frameworks treat API calls as an implementation detail that happens behind the scenes. That's fine for a prototype. In production, you need to know exactly how many tokens each feature consumes, per user, per session, broken down by model. You need to set cost ceilings per workflow. You need to alert when a chain enters a retry loop that's burning through your budget.

Frameworks that abstract away the API layer also abstract away the cost visibility. You find out what something costs when the invoice arrives, not when the code runs. By then the budget damage is done and the forensics to figure out which workflow caused it requires instrumenting the framework from the outside, which is harder than building the instrumentation into your own code from the start.

When to build your own

Not always. But more often than the ecosystem wants you to believe.

If your use case is straightforward and matches the framework's happy path, use the framework. If your use case involves custom retry logic, granular cost tracking, complex state management across agent chains, or any production concern that the framework treats as an edge case, you're going to fight the abstractions more than they help you.

A lightweight wrapper around the raw API that you wrote, you understand, and you can debug in five minutes is often faster and safer than inheriting someone else's opinionated framework and spending days working around the opinions that don't fit your use case.

The architecture principle

Keep your architecture simple, explicit, and transparent. Every layer of abstraction you add is a layer you have to debug through when something breaks. Every opinion a framework encodes is an opinion you either agree with or work around. The magic of the first ten minutes is not worth the friction of the next ten months if the abstractions don't match your production reality.

The best AI infrastructure I've seen in production is usually the simplest. Thin wrappers, explicit state, visible costs, debuggable chains. Not impressive to demo. Extremely effective to operate.

How much of your AI stack is framework vs custom? And which part causes more production incidents?