DEV Community: Sergei Strebulaev

Building a Terraform Provider for Paddle Billing: Why Real-Sandbox Testing Matters

Sergei Strebulaev — Sat, 05 Sep 2026 06:08:59 +0000

We manage our whole infrastructure as code, and wanted our Paddle Billing catalog — products, prices, discounts, webhook config — to work the same way instead of being the one thing still managed by hand through a dashboard. Existing tooling didn't cover enough of what we actually needed, lifecycle actions for subscriptions and refunds especially, so we built our own.

terraform-provider-paddle manages Paddle Billing's catalog — products, prices, discounts, discount groups, notification settings — plus six lifecycle actions (refunds/credits, subscription cancel/pause/resume/charge, notification replay), lookup data sources for subscriptions/transactions/customers/events, and a couple of newer Terraform capabilities: an ephemeral resource for a webhook secret that never touches state, and resource identity + list-block support for bulk-discovering existing infrastructure via terraform query.

This post isn't a feature tour, though — it's about one specific decision that shaped how the whole thing got built: every resource and data source is verified end-to-end against Paddle's real sandbox API before each release, not just tested against mocks. Here's a real bug from that process that explains why.

The problem with mocking someone else's API

Every API client eventually encodes assumptions about the wire format — what fields exist, what shape they're nested in, what a status value actually means. When those assumptions are right, mocks are great: fast, free, deterministic, no network flakiness. When they're wrong, a hand-written mock just encodes the same wrong assumption and "confirms" it forever. A test suite built entirely on mocks can be green for months while quietly testing the wrong thing.

Here's a concrete example. A transaction's line items don't carry a flat price_id field — Paddle nests a full price object under each line item, and the ID you actually want is price.id. A mock built around the flat-field assumption would have exercised the same code path, returned a plausible-looking response, and passed — while the real code silently compared against an empty string every time it ran against the real API.

The fix wasn't just correcting that one field. Paddle publishes a proper machine-readable OpenAPI spec for their whole API, auto-generated from their own backend whenever it changes — so this provider now vendors a copy of it and runs a forward-only contract check on every test run: for every field a client struct actually reads or writes, does that field still exist in Paddle's real API, at the same nested path, with a structurally compatible shape? It's deliberately one-directional — the provider intentionally leaves plenty of Paddle's API surface unmodeled, so checking "does the provider cover everything Paddle has" would just flag every deliberate scope decision as false-positive drift. Checking "does what the provider does model still match the spec" is the direction that actually matches the bug history.

Newer Terraform features, and why they showed up here

Ephemeral resources. A webhook signing secret is real, sensitive data that's genuinely needed at apply time — but there's no reason it should sit in a state file forever. Sensitive: true on a schema attribute only redacts CLI/log output; it does nothing to state itself, which still stores the real value in plaintext. This provider added an ephemeral resource specifically so that secret can be fetched fresh on every apply and never written to state at all, while keeping the old (now-deprecated, still fully working) attribute in place for anyone already depending on it.

Resource identity + list resources. terraform query (Terraform 1.14+) turns out to hard-depend on resource identity being implemented on the target resource first — a list result's identity is required, and it's built from that schema. Once that's wired up for a resource, bulk-discovering everything of that type already sitting in a Paddle account — and generating import config for it — becomes a first-class Terraform workflow instead of something you'd have to script by hand against the raw API.

Try it

Terraform Registry: registry.terraform.io/providers/vivantel/paddle
Repo: github.com/vivantel/terraform-provider-paddle

Feedback and issues welcome — especially anywhere the schema doesn't match real Paddle behavior. That's exactly the kind of thing this whole approach is built to catch, and it's more reliable with more real accounts exercising it.

My AI Keeps Forgetting What We Already Decided

Sergei Strebulaev — Thu, 03 Sep 2026 22:41:57 +0000

Every week I re-explain the same architecture choices to my AI coding
agent. New session, zero memory — like we never talked. That's not a
prompting problem, it's a memory problem, and I fixed it with a
git-based knowledge system that lives right next to the code. Here's
the actual walkthrough, the schema it writes, and where it falls
short — not just the pitch.

The problem in one sentence

I plan a feature with Claude Code on Tuesday. Wednesday, fresh
session: "Where did we leave off?" No idea. I re-explain everything.
By Thursday I've explained it a third time to a different agent. Each
one asks good questions, gets a good answer, and forgets it the moment
the session ends. The plan was never the problem — nothing durable
ever got written down, so there was nothing for the next session to
read.

What I built: kms

kms is a Claude Code plugin that captures decisions, facts, and
guardrails as plain markdown files, version-controlled right alongside
your code. Any agent that reads the repo reads the knowledge base
first, before it asks you anything.

Install:

/plugin marketplace add vivantel/kms
/plugin install kms

Four artifact types, each with one job:

Type	Answers	Lives in
Fact	What's true right now	`docs/facts/`
Decision	What you're committing to, and why	`docs/decisions/`
Guardrail	What must (or must not) happen, derived from a decision	`docs/guardrails/`
Skill	How to act on all of the above	`docs/skills/`

Walkthrough: capturing your first decision

One command:

/kms:quickstart

If docs/{facts,decisions,guardrails,skills}/ doesn't exist yet,
quickstart sets it up first. Then it asks one direct question:
"What decision or plan is currently live for you right now?"

Mine was the multi-platform content strategy I'd re-explained three
times that week — which platforms, what cadence, what CTA on each.
Quickstart interviews you — what's changing, why, what stays fixed —
then writes the result to
docs/decisions/0002-content-multi-platform-strategy.md:

---
id: 0002-content-multi-platform-strategy
title: Multi-platform content strategy for kms outreach
status: accepted
date: 2026-08-31
tags: [kms, content, outreach, platforms, multi-channel]
track: product
scope: All public articles produced in kms_content
---

## Decision

kms public articles expand beyond Medium to Dev.to and Hashnode.
Medium stays canonical; the others get manually adapted copies. Weekly
cadence. Platform-specific primary CTAs — Medium leads with install,
Dev.to with GitHub stars, Hashnode with newsletter signup — rather
than one CTA everywhere.

## Rationale

...

Nothing gets summarized or paraphrased away — that's your call,
captured once, in your own words. The session closes by naming what to
run next, not leaving you to guess: query to pull this decision back
up later with a citation, capture after the next session that
touches it.

From that point on, Claude Code
reads that file the next time it opens the repo and knows exactly
what was decided. Codex reads the same skill set through its own
plugin manifest. Kilo Code CLI reads it too, once kilo.jsonc points
at the published skills manifest.

Beyond decisions

Facts and guardrails link back to why they exist, not just what they
say. Say you decide Medium posts close with an install CTA, Dev.to
posts close with a GitHub-stars ask. That's a decision. The guardrail
that enforces it on every article cites that decision by id. A skill
that tells you how to adapt a draft per platform references the
guardrail. Change the decision later, and capture/lint catch the
guardrail and skill silently drifting out of sync with it — instead
of you finding out three articles later.

Commands worth knowing once you've got a knowledge base going:

query — retrieve a past decision with a citation, instead of re-explaining it.
capture — log what a work session changed, after the fact; flags contradictions it finds along the way.
lint — validate the whole knowledge base on demand: dangling references, missing fields, stale derived artifacts.
brainstorm — generate fresh approaches with no anchor to past decisions, for exploring before anything's locked in.
onboard — a role-tailored, 5-day ramp-up plan for a new teammate, built from the existing knowledge base.
conform — check whether a pending change respects the guardrails before it lands.

What this doesn't solve

It's not magic memory — you still write the interview answers
yourself; quickstart just makes sure they get written down instead of
staying in your head. It doesn't replace tests, code review, or
actual documentation for end users. And it only helps if the habit
sticks — a knowledge base nobody updates after month one is just a
docs/ folder with extra steps. The parts that make that less likely
are capture (so updating it is a five-minute pass after a session,
not a separate chore) and lint (so drift gets caught instead of
silently rotting).

Why this beats a context file you paste in

I've tried system prompts, paste-in context files, separate
note-taking apps I'd open in another window. They all fail the same
way: they live outside the repo, so they don't survive a fresh
session, and they rely on you remembering to open them, paste them in,
and keep them updated. This is different because it's part of the
repository — every decision is version-controlled, every fact is
verified, and a guardrail is derived from a real commitment instead of
vibes. When the project changes, the knowledge base changes with it,
in the same commit history as the code.

Try it

If this is useful to you, a star helps other developers find it:

⭐ Star vivantel/kms on GitHub

Then install the plugin and run /kms:quickstart on one real decision
that's been living in your head. You'll never explain it twice.

Contributions welcome — see
CONTRIBUTING.md
for how to propose new skills or improve existing ones.