Building with AI: the evidence, the method, and the guardrails

I have been shipping software professionally since 2003. The last three years are the first time the shape of the job changed rather than the tooling. This page is the receipts.

1. The measured part

Four projects, one five-month window in 2026, built solo or lead. These numbers come from git log --all --no-merges in each repository: commits carrying an AI co-author trailer or authored directly by a coding agent, over total commits.

Three quarters of three thousand commits. And within that, 322 commits were authored by an agent outright by copilot-swe-agent[bot], Copilot, or anthropic-code-agent[bot]. An issue was filed, an agent was assigned, and a reviewable pull request came back.

What the number is not. It is not "AI wrote three quarters of the code." A co-author trailer marks a commit an agent participated in, not a percentage of lines. The honest claim is narrower and, I think, more interesting: roughly three quarters of the commits in a five-month, four-product run had an agent in the loop, and the products are real: deployed, tested, observable, and documented.

Three caveats I'd raise if I were reading this skeptically.

• Why all branches? Numerator and denominator come from the same --all --no-merges set, so the ratio is internally consistent. Counting only main would flatter the number, because squash-merges drop agent trailers.
• 3,000 commits in five months is ~20 a day. That is real, and it is a property of the method rather than heroics: agent lanes commit at much finer granularity than a human does, and much of the volume lives on short-lived worktree branches. Share is the meaningful figure here; volume is not.
• INTERLOCK's 85% sits on only 123 commits. Small sample. Cloud Topology Toolkit's 85% across 630 commits is the sturdier number, and Helix's 69% across 1,736 is the sturdiest.
• helix-bot (200 commits) and semantic-release-bot (139 commits) are excluded. They are CI automation, not AI. Including them would have added ~11% for free.

2. The track record before the commit log

The measured window above is recent. The pattern is not.

2019–2023 · Coyote Logistics, Copilot at enterprise scale

Founded and led the Cloud Enablement Team. Led the migration of 900+ repositories from Azure DevOps to GitHub with Actions and GitHub Advanced Security, then drove GitHub Copilot adoption across the engineering org. Co-founded the Coyote DevOps Curriculum and mentored 30+ engineers through modernization and Microsoft certification.

The outcome was public:

• Wall Street Journal: Generative AI Helping Boost Productivity of Some Software Developers. "We can code as fast as we think."
• GitHub Universe / Galaxy 2023 keynote: featured by Inbal Shani and Thomas Dohmke (watch).
• GitHub customer story: How Coyote Logistics uses GitHub.
• GitHub's own Copilot marketing: "Coyote Logistics' developers reduced the time to write Terraform config files by 50% with GitHub Copilot", carrying my pull-quote; and GitHub's CEO publicly asking to borrow the phrase.

2024–2025 · Next Generation Logistics: multi-agent, under a real constraint

Brought in to replace the sole developer of 20 years who had unexpectedly passed away: multiple .NET TMS versions across 9 customers, zero documentation. The recovery was AI-shaped by necessity:

• 250% productivity increase in 6 weeks with 3.5 developers, using GitHub Copilot Enterprise plus a multi-agent framework I built (YAMAUL). Sprint velocity went from 12 → 42 work items, at a 95% PR success rate, with 200+ zero-downtime deployments through GitHub Actions to Azure Container Apps.
• YAMAUL ran on Azure AI Foundry Agent Service with declarative YAML agent configs, Connected Agents, and Multi-Agent Workflows orchestrating the development lifecycle, with vector stores for semantic search across Azure SQL.
• Model Context Protocol between Azure API Management and AI Foundry cut AI operational cost 60% through caching, token controls, and observability across Azure OpenAI, Anthropic, and local models, with Entra ID auth and conditional access in front. Cost control is the part most AI programs skip until the invoice arrives.
• Reverse-engineered the inherited systems well enough to stabilize operations and stand up a sustainable team, the AI's most valuable job there was decoding tribal knowledge nobody was left to explain.

2025–present · Metafora: agents in production, not just in the IDE

Primary engineer on Socket Merlin, an AI-powered support orchestration and workflow automation platform (.NET / ASP.NET Core, React + TypeScript + Vite, PostgreSQL + pgvector, Redis, SSE + SignalR, LangFuse, GKE via Helm, Terraform).

• Built the agent layer on Microsoft Agent Framework with Claude Sonnet, and moved LLM access off a third-party proxy onto a first-party SDK path. That removed a hop from every model call and put authentication, cost attribution, and tracing under our own control instead of a vendor's.
• Made the AI provable rather than plausible. LangFuse traces every agent run, so a bad answer is a thing you can open and read rather than a thing somebody reports. A batch-mode rubric evaluator went to production, which means output is graded systematically instead of spot-checked by whoever happened to notice. And knowledge-graph curation recovers from truncated model responses, because the interesting production failure is not a wrong answer, it is a half-finished one that looks complete.
• Designed the migration architecture for a large online shipping marketplace: a live production system split across AWS Lambda and Windows Servers moving to container-based AWS deployment behind API Gateway, with LaunchDarkly (including session replay) carrying progressive exposure and OpenTelemetry holding trace continuity across both stacks during cutover. Same conviction as INTERLOCK: the flag decides who sees it, not the deploy.
• Built an AI-agent developer toolchain: code index, hooks, devcontainer, so the agents on the team start from the same context a new hire would get.

3. The method

What the commit log doesn't show is the operating discipline. Six rules, all learned the expensive way.

1 · Specs before agents. Every project above starts from a written spec, not a chat. INTERLOCK used GitHub Spec Kit to turn design prototypes into 8 user stories and 54 numbered functional requirements before the first line of code. Cloud Topology Toolkit drives numbered features from numbered issues. Agents are extraordinary at execution and poor at deciding what "done" means; the spec is where the human judgment goes.

2 · Contract-first, then behavior. Define every data shape in OpenAPI, generate the .NET and TypeScript DTOs from it, scaffold a conformant shell, pressure-test the shell, then iterate behavior with the shapes frozen. Agents drift on shapes and hold steady on logic; freezing the shapes removes the entire class of drift.

3 · The guardrail is a gate, not a review. Helix re-validates every agent-proposed code span against HEAD, stamps provenance: agent, and fails CI if anything is orphaned, and it refuses unreviewed agent-mined design points outright. Cloud Topology Toolkit content-hash golden-tests the snapshot → IR mapping. INTERLOCK keeps its readiness algebra hand-written under 34 domain tests. If the only thing standing between an agent and main is a human reading a diff, the human loses by volume.

4 · Make the system say when it doesn't know. This one is a design principle, not a process one, and it's the through-line of everything I build now. In Helix, null means unmeasured and 0 means measured, and the answer is zero: they never render the same way, and a forecast that fails its own back-test is withheld rather than dressed up. In Cloud Topology Toolkit, when two planes don't correlate, both uncorrelated sides get named, never quietly merged. Generative systems are fluent by default; usable ones are calibrated on purpose.

5 · Plan the fan-out instead of guessing it. Helix's plan-paths models vertical (one agent per story), horizontal (one agent per layer), and hybrid decompositions, sweeps the lane count, and reports the capacity the dependency graph can actually keep busy. Then it emits a self-contained prompt per lane and picks the executor: clean single-story work to the Copilot coding agent, harder work to an isolated Claude lane. "How many agents can this work absorb?" is a measurable question.

6 · Watch the bill. MCP in front of the model, through API Management, with caching, token controls, and observability. That's the 60% at NGL. An AI program without FinOps is a pilot, not a platform.

4. What I actually believe about this

The keynote line was "we can code as fast as we think." Three years of living inside that has sharpened it into something less quotable and more useful:

The bottleneck moved. It used to be typing. Now it's knowing what you want precisely enough that a machine can't misinterpret it, and being able to prove afterwards that it didn't.

The clearest evidence that this is a conviction and not a trend: in 2010 I published a CodePlex tool that tried to tie requirements to exact spans of code. It was called Helix, it used the terms design point and code implementation item, and it broke the moment anyone refactored. Helix (2026) is the same idea with the missing sixteen years filled in, including the part where agents, not people, keep the links alive. The 2010 write-up →

That is why my last five projects are, suspiciously, all about proof: a control plane that refuses to call deployment a release, a toolset that refuses to average four different definitions of "done," a visualizer that refuses to draw a line it can't verify. AI made generating software nearly free. Everything valuable now lives in the verification.

Which is, annoyingly, what I already thought at eighteen:

Intelligence was never the scarce half. Experience, the part that tells you which answer is actually right, still isn't automatable. AI just made the first half free.

Related Projects · Full experience · How this page was reviewed