Jean-Francois Moy

· 5 min read

Spending judgement where it matters: how we structure AI-native delivery

A few weeks ago I wrote about cognitive debt: the slow loss of judgement that comes from letting a model think in your place. The natural question that follows: how do you use these tools seriously, at full speed, without paying that price?

Here is my answer, after a year of building a production fintech this way. It isn't using the tools less. It isn't trying harder to stay alert, because attention fades and deadlines win. It's structure: a delivery pipeline where human judgement goes to the few decisions that deserve it, and everything else is enforced by gates that don't rely on anyone remembering to care.

The scarce resource has changed

For my whole career, engineering capacity was the constraint. Roadmaps were shaped around what the teams could carry. With agentic tooling, that constraint has visibly moved: a small team can now produce what used to take a department.

What hasn't moved is the supply of judgement. Someone still has to say: this ticket solves the wrong problem. This design will hurt us in a year. This edge case is the one that matters.

I think that is now the real job of an engineering leader. Not adopting AI; everyone has. Deciding where human attention goes, and building the system that protects it.

The lag that fools you

The dangerous thing about uncontrolled usage is that it feels great. The tools are impressive, output pours out, velocity charts point up. Nothing seems wrong.

The bill comes later, and it compounds quietly: vicious bugs in code nobody truly read; confident implementations of acceptance criteria that were invalid, or misunderstood, because nobody challenged them before building; a team that no longer knows its own codebase, because everything was delegated and stamped. By the time the symptoms show, you don't have a bug. You have an archaeology dig, run by people who have lost the fluency to dig.

Cognitive debt and technical debt turn out to be the same debt. The cost is deferred, invisible while it grows, and due all at once. "It's working fine so far" is not evidence. The lag is the trap.

What the structure looks like

The pipeline our team runs mirrors the roles of a delivery team, chained so work flows from ticket to merged, released change with a quality gate at each handoff. Every engineer runs it the same way, each owning the judgement calls on their own tickets.

It starts before any code. Tickets are written as user stories, then challenged: the system's first job is to argue with the scope, surface the gaps, and push the contentious questions to the front, while the engineer shapes the ticket and the technical design. Anything diagnostic follows a data-first rule: logs and database rows before hypotheses.

Implementation is test-first against the acceptance criteria, so behaviour is pinned before code exists. Then the reviews, run as parallel, independent lenses: one checks the change against the ticket, one against the project's conventions and architecture, one for code quality, one for API-contract stability, plus security and supply-chain passes, and a simplification pass whose only job is to remove complexity that doesn't pay rent. None of this depends on someone remembering to run it.

Then live validation: the change runs against a real environment, driving the actual UI, checking database side effects and backend logs, walking the unhappy paths as well as the happy ones. Acceptance criteria are proven on a running system, not asserted on paper. Finally, landing: a rich PR, and a merge gated on clean re-review, green CI and explicit human sign-off, with project state updating itself truthfully.

One command carries a ticket from scope-challenge to merged PR. Human attention is spent at exactly two points: shaping the work before it starts, and the contentious calls the gates escalate. The tiny implementation details that used to eat whole review cycles never reach a person.

Why this keeps judgement sharp

Cognitive debt grows when the model does your thinking and you stamp its output. The structure reverses that twice.

First, it concentrates the thinking that stays human. Nobody reviews boilerplate or formatting, so the judgement each engineer does exercise, on scope, design and the contested calls, gets full attention, not what's left of it at the end of the day. The skill you keep is the skill you use.

Second, the gates are structural, not optional. Speed and safety stop competing the moment quality checks are steps the next stage depends on, rather than virtues someone must remember to practise. A pipeline that cannot merge without independent review, passing tests and live validation doesn't get lazier as the people get faster.

There's a compounding layer too. The system remembers the team's conventions, so lessons are taught once. Preferences become guardrails instead of tribal knowledge. Every mistake becomes a rule, exactly once.

And no, the gates don't slow us down: they spend machine time, not human attention, running in parallel while you shape the next piece of work. Working this way, three engineers took a new product from concept to public launch in six weeks, on a platform that processes millions of financial events a month, in software where a wrong number in front of a customer is unacceptable. I don't believe that pace, at that bar, is reachable for a team this size any other way.

What it costs

None of this is free. The harness took real time to build and tune; the gates encode our conventions and, more importantly, our architecture, and encoding them was work. That last point matters: the codebase has a purposeful shape, decided by people, and the pipeline defends it on every change. A model without that constraint will happily grow you a system with no shape at all. The whole system lives or dies on well-formed acceptance criteria: feed it vague scope and it will confidently build the wrong thing. That's why the scope-challenge stage exists, and why it's the one stage I would never automate away. The live-validation environment is infrastructure that needs an owner, like any other.

And one problem I won't pretend to have solved: this model assumes seasoned judgement at the gates. How a junior grows that judgement inside a system that shields them from the grind that used to build it is, I think, the hardest open problem in AI-era engineering leadership. Anyone who says they've solved it is describing a plan, not a result.

What I'd tell engineering leaders, and engineers

Adoption is not the question any more; the tools are everywhere. The divide that matters is between teams that use them under control and teams quietly accumulating the lag-hidden debt above while their dashboards look great. The second group doesn't feel wrong yet. That's the point.

For leaders, the path that works takes more design effort than either enthusiasm or caution. Decide which decisions deserve human judgement in your context. Build the structure that routes those decisions to people. Make every other quality property a gate, not a hope. What you get, in my experience, is the combination everyone assumes is a trade-off: faster delivery, less debt, and engineers whose judgement sharpens with use, because it's the only thing they spend all day exercising.

And if you're an engineer: don't wait to be handed this. The gates are just skills, plain files that encode what your team knows: the domain, the conventions, the architecture, the strategy behind the codebase. Anyone can write one. The best additions to ours came from whoever last hit a problem worth never hitting again. Champion the framework, improve it, add the rule your yesterday self needed. It isn't your manager's system; it's the team's shared memory, and everyone can write to it.

One boundary worth stating plainly: everything here is about building things right. Whether you're building the right things is a different discipline, and no pipeline answers it for you.

The tools will keep improving. The structure decides whether they make your team more capable, or just more prolific.

Got any questions or comments? Drop me a message on X @moystard