Writing
Thoughts on software engineering, privacy, tooling, and building for the long term.
19 posts matching filters
Before you fine-tune a small local model, you build the measuring stick. How I turned months of my own review artifacts into an evidence corpus, picked a baseline with a bake-off, and learned to tell three sources of improvement apart all before training anything.
A local model is only a real fallback when it can do measured, bounded, real work. The standard I used for calling a local model dependable, and the unit economics from my own runs: what local inference actually costs once utilization, maintenance, and optimized cloud pricing enter the ledger.
The finale of this series is a negative result: the QLoRA adapter performed worse than its unmodified base model on the task it was trained for. What the controls caught, what the adapter actually learned, and why the expensive part was producing a conclusion I could trust.
Why I set out to adapt a small open-weights model that runs on my own hardware into a genuinely useful reviewer | the reasoning, the hypothesis I am testing, and a plain-language primer on what LoRA and QLoRA actually are (and, just as importantly, what they are not).
Four local models, eight hosted models, two modest coding tasks, and the same uncomfortable result: useful drafts, but no autonomous worker.
Subscription tiers are a business decision, not a technical guarantee. When the math changes, pipelines built on token discipline adapt. Marathon pipelines break.
Anthropic markets the 1M token context window as the greatest thing since sliced bread. They are not wrong. But the hidden costs will eat you alive if you are not paying attention.
After nine posts about making agents work better, the last lesson is the simplest: sometimes you should just do it yourself.
Model updates will break your pipeline. The question isn't if but when. Build to absorb change, not to resist it.
Your pipeline is decaying 5% per sprint and you can't feel it. Four metrics that catch the slow killer before it catches you.
Before you spin up a fleet of AI workers, ask whether you should. Most teams are bottlenecked on specification, not execution.
Every token in the context window is either signal or noise. Noise doesn't just waste space — it degrades everything around it.
Every failure mode in this series traces back to the same root cause: the spec wasn't good enough. The spec is the product.
LLMs have no ego and no allegiance. Use adversarial framing to get review output that collegial prompts would gloss over.
Your agent's output will route around your gates. Not by intent — by optimization pressure. Here's how to close the gaps.
AI agents don't reliably follow instructions. Programmatic gates catch the drift that human review misses.
Context degradation is gradual, not sudden. By the time auto-compaction fires, your agent has been impaired for a while.
Privacy controls, pricing traps, multi-provider strategy, and token discipline — the packing list for getting started with AI coding.
A maturity model for AI-assisted coding — from copy-paste to semi-autonomous pipelines. Map your level and find your next step.