Writing
What I built, what broke, and the numbers. New pieces also go out by RSS.
- 3,380 consecutive provider failures were a policy failure, not a transient one
My model router kept sending traffic to a provider with no credit for 3,380 failures in a row, because a bandit treats failure as a score and not as a reason to stop.
- 80 Access apps and one wildcard
A Cloudflare Worker that provisions Zero Trust Access apps from a JSON config, never touches group membership, and audits the whole account for broad policies.
- A commit gate you can watch fail
Three commit-time gates (secrets, private terms, tokens in git URLs) published with the evidence that each one blocks bad input, including the false positive that read SVG coordinates as phone numbers.
- A Durable Object per GPU node, for a home Mac mini
How a Cloudflare Worker queues, preempts and recovers jobs for a home GPU box that sleeps, drops its tunnel and restarts whenever it likes.
- A merge queue that only fast-forwards trunk to a commit it verified
I rebuilt a small land queue so the commit trunk advances to is, by construction, the commit the checks ran on, and gave it a shadow mode to run before you trust it.
- A red team that has to quote its sources
A layered red team over policy documents where every quote is checked against the source text, and one unsourced quote makes the run exit 1.
- A test that refuses to ship an LLM stage nobody measured
I wired a stage evaluation into cargo test, so a registered LLM stage with no measured verdict, or a prompt edited since its measurement, fails the build.
- Admission by reservation, not instantaneous load
A scheduler that checked current load let four jobs onto a Mac that then hit load 44, so I made every job reserve its peak before it starts.
- An on-device LLM as autocorrect, with a typed did-you-mean
I pointed Apple's on-device 3B model at autocorrect, forced its answer into a typed schema, and found out how little of that I've actually measured.
- Caching made my rewrite pipeline cost more, and the cheap-model verdict only half held
A three-judge panel scored 810 rewrite candidates; caching raised cost at no quality gain, and a fallback backend I hadn't noticed was skewing the model comparison.
- Cost in the reward, once
armwright learns which model to send each kind of task to, from the outcomes of real calls. What a seeded 20,000-call simulation shows about where the cost knob really is.
- How goldenwatch refuses a release
Most eval scripts fail open. goldenwatch names six ways a golden-set check goes quietly wrong, and refuses each one.
- Mutation testing scoped to a diff, and why survivors need a written reason
A mutation-testing tool that mutates only the functions a diff touched, runs every mutant in a throwaway sandbox, and refuses to ignore a survivor without a written reason.
- My AbortSignal was correct and the call still hung for six hours
A model call hung twice for about six hours behind a correct AbortController, so I moved the deadline to a timer that does not depend on the signal.
- Patching a ReDoS in picomatch without forking the world
A glob library under most JavaScript build tools had an unfixed regex advisory, so I refused one construct, kept every other byte identical, and made the copy fail the build if it drifts.
- Porting a Python gate to Go with golden files, and why append-only ledgers merge cleanly
I replaced a Python checker for a decisions graph with a stdlib-only Go tool, and checked it against the old output byte for byte using recorded files.
- Redact before the model, restore after
A reversible PII boundary around every model call in a chat app, tested by planting identifiers in the input and failing on any that show up in what leaves.
- Six guards in one day that had never been shown to fail
A log of six checks in my own repos that looked present and had never been tested against bad input, and the one habit that exposed them.
- Temperature silently does nothing on reasoning models, and the API will not tell you
A per-provider matrix of what temperature does while a model is thinking, and a small Rust library that sends, drops or refuses it and records which.
- Testing firmware logic on a laptop with a port
I put the timing, retry and sequence logic of a two-board hobby project behind a three-method interface, so most of it runs and breaks on my laptop instead of on a board.
- Three P0s that looked wired
An alert sender with no credentials, an auth check that returned true, and a secret rotator that returned constants, all found in one audit.