Can we actually ship safe AI code?

Every AI coding tool in 2026 sells you the same promise in a different typeface. Describe what you want. Get production code. We verified it. Cursor, Copilot, v0, Bolt, Devin, Replit Agent — each ships a feature called “safe mode” or “verified” or “intent-aware,” and not one of them tells you what the verification actually proves.

ShipGate (@shipgate.dev/cli, v3.0.0) caught my eye because it made a bet most of the field ducked. Instead of begging an LLM to “be careful,” it asks you — or the LLM — to write down what the code is supposed to do in a small formal language called ISL (Intent Specification Language), then checks the generated code against that contract. SHIP or NO_SHIP. Real exit codes. Real evidence files. Real schemas pinned to a URL. I spent an evening pushing on it. Here is what I found, what is missing, and what would have to be true for the dream to actually arrive.

Two products living in one CLI.

• The Tier-1 merge gate (shipgate next .) — a Next.js App Router scanner that runs your tests, scores 0–100, and refuses to ship below threshold or with a required signal missing. This is the polished half.
• The spec-driven pipeline (isl-generate, gen, verify, gate, vibe) — the bigger bet. You write an .isl file by hand or have ShipGate auto-extract one, and the verifier grades implementations against that contract, emitting a trust score with category breakdowns.

The first works today. The second is the dream, partially built. The distance between them is the whole story.

I cloned vercel/nextgram, a small-but-real Next.js 15 App Router demo. Out of the box it had no tests.

$ shipgate next . --strict
Verdict: NO_SHIP
Exit code: 10
- Score 39 is below minimum threshold 68
- 1 test(s) failed (no test runner found)

I added a single vitest test asserting 1 + 1 === 2 and a "test": "vitest run" script.

$ shipgate next . --strict
Verdict: SHIP
Exit code: 0
Score: 73/100, 1/1 tests passed

The gate flipped. Not a stub that always fails, not one that always passes — a real check that responded to a real change. An evidence file landed at .shipgate/evidence.json, schema-pinned to a CDN URL, with PR-ready markdown sitting next to it. This part works. Wire shipgate next . --strict into GitHub Actions as a required check and you have a real merge gate today.

I escalated to ixartz/Next-js-Boilerplate — 12.9k stars, Clerk auth, Drizzle ORM, Sentry, next-intl, Playwright, Vitest, Storybook, Checkly. Roughly 2,353 lines scanned. Three escalating runs:

Run 3 is the one worth dwelling on. The specless gate said SHIP (73/100, 1 test passed). But the overall verdict came back NO_SHIP anyway, on a route-level finding:

x src/app/api/counter/route.ts -> /counter
  No exported GET/POST/PUT/PATCH/DELETE/OPTIONS/HEAD handler found

Except the file does export a handler:

export const PUT = async (request: Request) => { ... };

ShipGate’s parser misses the const-arrow form. It hunts for export function PUT / export async function PUT. So it flagged a real file for the wrong reason — a false positive about the cause, not about the file’s existence.

This is where the dream is supposed to live: files become specs, specs become the gate, the gate signs off only if the implementation matches the intent. I walked the full pipeline on the boilerplate.

$ shipgate spec bootstrap
-> .shipgate/next-bootstrap/manifest.json    (route + page + env inventory)
-> .shipgate/next-bootstrap/routes-draft.isl (placeholder scaffold)

The bootstrap is refreshingly honest. The generated ISL prints “inferred (not audited)” at the top and tells you outright that the baseline Tier-1 SHIP path does not require hand-written ISL. The product already knows the spec language isn’t pulling its weight. The deeper extractor, isl-generate, produced this for a real route file:

domain Route {
  version: "1.0.0"
}

Twelve lines of nothing. Confidence 0.30. The status comment owns it: “# STATUS: INCOMPLETE — auto-generated typed contract scaffold. This spec captures exact signatures but has no business rules.”

So I wrote the spec myself: a tiny add(a, b) ISL contract and a one-function TypeScript implementation. After a short fight with the grammar:

$ shipgate verify --spec add.isl --impl add.ts
Verification passed (6762ms)

Evidence Score: 65/100   Confidence: 80%
Checks: 5 passed / 3 failed
Trust Score: 83/100

Postconditions  2/3
Invariants      1/2
Scenarios       2/3

Recommendation: Staging Recommended

This is the spec-driven verdict your mental model expects — categories, a confidence number, a recommendation. The infrastructure works. The bug: shipgate gate rejects the very specs that verify accepts. The plumbing is laid; the front door is locked.

It isn’t LLM quality. It isn’t the heal loop. It isn’t the CLI architecture.

The bottleneck is what ISL can say.

Today, an OAuth spec written in ISL can say things like this:

• Session.exists(result.id)
• result.status == ACTIVE
• result.expiresAt > result.issuedAt

It cannot say any of the things that actually keep an OAuth flow safe:

• the state parameter must be validated against a CSRF token
• the redirect URI must sit in a compile-time allowlist
• the PKCE verifier must match the challenge using a constant-time comparison
• refresh tokens must rotate
• cookies must carry SameSite=Lax; HttpOnly; Secure
• the login route must rate-limit at 5 attempts/min/IP
• session-token entropy must come from crypto.randomBytes
• HMAC comparisons must be timing-safe

Until ISL has primitives for those, the gate can happily ship a verified OAuth implementation that is also full of holes — and the gate isn’t lying, because the spec never asked for those properties. The proof is honest. It just answers a smaller question than the one you needed answered.

Three walls. None impossible, none cheap.

Wall 1 — Richer spec language. Borrow from TLA+, Dafny, F*, and SPARK. Add primitives for cookies, redirect allowlists, rate limits, secrets-from-vault, timing-safe comparisons, and token rotation. Call it ~6 months for a meaningful first pass, two years for anything near completeness.

Wall 2 — Real verification. Wire in Z3 for pre/postcondition proof, taint tracking, type-state machines, static analyzers as blocking gate signals, and property-based fuzzing inside the heal loop. One to two years of compiler-engineer time.

Wall 3 — Shrink the LLM blast radius. Pin generated auth code to vetted libraries (lucia-auth, better-auth, auth.js), add an adversarial reviewer agent, run semgrep and CodeQL as blocking gates. Six to twelve months — and the biggest security-per-dev-month return of the three.

The default in AI coding tools right now is “LLM wrote it, looks fine, you decide.” That isn’t engineering — it’s a confidence game. Every team I know shipping AI code in production has quietly built some private review ritual to compensate. ShipGate is one of the few public attempts to fold that ritual into the tool itself, with exit codes and evidence files where everyone else leaves vibes.

It is incomplete: the spec language is the wrong size today, the verifier is shallow today, the auto-extraction is thin today, the gate parser is broken today. But the architecture is correct. The hard parts — sequencing the stages, gating on score, bounding the heal loop, refusing stubs, emitting machine-readable evidence — are already built. What’s missing is mostly the vocabulary the system is allowed to reason about.

The dream isn’t dead. It’s just earlier than the marketing implies.

You can follow the longer arc of this argument in The bet behind ISL.

Tested @shipgate.dev/cli v3.0.0 against vercel/nextgram and ixartz/Next-js-Boilerplate. Walked spec bootstrap -> isl-generate -> check -> verify -> gate -> gen. Hand-wrote an OAuth ISL spec. Read the 50MB bundled cli.cjs. Pro features not run end-to-end. ~12 min read. May 14, 2026.