What 98 API endpoints taught me about contract drift

Ali Duale — Fri, 09 Oct 2026 08:15:57 +0000

Seven months building a tool that catches API contract drift. Not writing specs. Checking whether the API you ship still matches the contract you published.

I ran it against 98 endpoints. 12 contracts. My own workspace.

Thought drift would be obvious. A field disappearing. An endpoint changing shape completely.

It isn't. Barely any of it is a break. It's small stuff piling up. Nobody decides to do it.

8 of the 98 were drifting. Same four things every time.

memberCount declared as a number. API returning a string. Nothing crashed. Tests passed. A strict client would break.

pathname promised in the response. Not there. Sometimes there, sometimes not. Spec never updated when it became conditional.

A field showing up in 40% of responses. Not required. Spec says it is. Hardest one to catch by reading.

And a field nobody documented that consumers started using. That's a contract now. Nobody wrote it down.

None of these are breaks. All of them are drift.

Why nobody notices: tests are written from the code, not the contract.

Test says "200, body has memberCount: 12". Passes whether it's a number or a string. Test matches the code. Code drifts, test drifts.

The contract is separate. Nothing in CI checks they stay in sync.

So I built DocSemantic. Learns a baseline from real traffic. Compares it to the declared contract on every CI run.

Action is a thin client. One POST, pass or fail:


yaml
name: API drift check
on: [pull_request]

jobs:
  drift:
    runs-on: ubuntu-latest
    steps:
      - uses: LingodocApi/docsemantic-action-check-spec@v1
        with:
          api-key: ${{ secrets.DOCSEMANTIC_API_KEY }}
Contract and traffic disagree, build fails.

Three bugs I shipped before launch. All mine.

First version flagged everything. Buried the real ones. Type change and a flaky field looked the same. Now they're separate.

First auto-fix opened a PR that changed nothing. Two signals hit the same field, cancelled out. Empty diff. Body said two fixes. Now I catch conflicts and send them to review.

Traffic from one spec tried to heal another spec's file. Demo API I imported started opening PRs on my real repo. Fixed — scopes by spec identity now. Fails closed if it doesn't know.

Can't do yet: one workspace per spec. No multi-spec. Shape drift only, not semantic. If a field keeps its type but changes meaning, I miss it. Endpoints with no traffic never get checked.

If you have an OpenAPI spec, check it against real traffic. It's probably already wrong. Most are.

Action is MIT. Free tier on the hosted side.

https://github.com/LingodocApi/docsemantic-action-check-spec
https://docsemantic.com

Anyone else seeing the same four categories? Or does real drift look different at bigger scale.

DocSemantic: catching API spec drift in CI before your customers do

Ali Duale — Tue, 29 Sep 2026 15:31:09 +0000

I come from a social services background—not a developer by training. I just read a lot and get curious.

The first time I heard the word "drift" was when my insurance company sent me an invoice with wrong items and wrong total. Support admitted it was a drift error—their systems had just been updated. I thought: why did it reach me at all? A drift check before sending the invoice would have caught it.

Later, a phone repair guy used the same word for a different problem. Two industries, same concept: two things meant to stay in sync slowly drifting apart. Nobody decides to break it. It just happens.

In software, the clearest example is your API spec vs. your live API. The spec says a field is a number. Somewhere along the way, the API starts returning a string. Nothing crashes. Tests pass. But the contract is now a lie—and nobody knows until something downstream breaks.

I read everything I could find about this, saw the gaps in static spec linters, and spent the last seven months building DocSemantic.

What it does

It compares your OpenAPI or Postman spec against what your API actually does. We learn a baseline from real traffic. When the spec and the live API disagree, you find out in CI—not from a customer email.

The GitHub Action is a thin client: one authenticated POST, pass or fail.


yaml
name: API Contract Check
on: [push, pull_request]

jobs:
  spec-check:
    runs-on: ubuntu-latest
    steps:
      - name: DocSemantic Spec Check
        uses: LingodocApi/docsemantic-action-check-spec@v1
        with:
          api-key: ${{ secrets.DOCSEMANTIC_API_KEY }}

DEV Community: Ali Duale

What 98 API endpoints taught me about contract drift

DocSemantic: catching API spec drift in CI before your customers do

What it does