Owned Compute as Insurance: What a Local Model Actually Has to Prove

Why I wanted a local fallback

In the preamble, I explained why I started this project. Frontier models sit outside consumer control, and access to them can be restricted or changed without input from the people who depend on them. The preamble covers my position on that policy question.

The engineering result is simpler. Hardware I own, running a model I control, gives me a fallback when a cloud service is unavailable, restricted, or no longer suitable for a particular workload.

That fallback only has value when it can perform useful work. Owning a GPU and running a model locally does not mean that the model could replace even one part of a cloud workflow. That capability has to be proven.

Defining a useful fallback

For local compute to function as insurance, the model must meet a measured standard on a well-defined, real task.

• Measured: Performance must be reproducible through an evaluation harness rather than based on informal use.
• Bounded: The task needs a defined scope and explicit success criteria. "Coding" and "being helpful" are too broad to evaluate.
• Real: The task should come from an actual workload rather than a benchmark designed for another use case.

This is why Part 1 — Building the evaluation spine focused on the evidence corpus and evaluation harness before any training took place. Without a corpus, baseline, fixture set, and scoring method, I could not determine whether the local model was useful enough to serve as a fallback.

The same discipline applied when choosing the baseline model. I evaluated candidates using criteria selected in advance, including task fit, hardware requirements, licensing, and fixture performance. Qwen3-8B remained the baseline because it performed better on my review fixtures than the alternatives I tested. Its general reputation was not part of that decision.

Unit economics from the experiment

Local AI cost comparisons are often reduced to API prices versus electricity. That leaves out most of the cost of owning and operating the hardware.

For my bounded review-artifact task, the local baseline generated a batch of 30 artifacts in under four minutes while consuming approximately 7.6 watt-hours of GPU energy.

Using an electricity price of $0.30 per kilowatt-hour and a nominal GPU cost of $0.15 per hour, the estimated cost was approximately $0.39 per 1,000 artifacts.

Applying the same token volumes to a frontier model's published API prices produced estimates of approximately:

• $2.99 per 1,000 artifacts at introductory pricing
• $4.49 per 1,000 artifacts at standard pricing

The local workload therefore had a lower marginal cost under this model, but the comparison needs several qualifications.

Optimized API pricing changes the comparison

The cloud estimate cannot reasonably assume synchronous list pricing when the workload is suitable for batch execution.

A 50 percent batch discount reduces the estimated API cost to approximately:

• $1.50 per 1,000 artifacts at introductory pricing
• $2.24 per 1,000 artifacts at standard pricing

Prompt caching or other provider discounts could reduce it further. Because this workload is not latency-sensitive, optimized API pricing is the correct comparison.

GPU cost includes more than electricity

The $0.15 per GPU-hour figure is a simplified allowance for depreciation, financing, maintenance, component failures, idle capacity, and the host system supporting the GPU. Especially in the current hardware market, wear and tear on the card itself must be considered.

Under the measured workload, the local lane remained cheaper while the all-in GPU cost stayed below approximately:

• $1.38 per hour compared with introductory API pricing
• $2.08 per hour compared with standard API pricing

Those thresholds are less generous when utilization is low. A GPU used for one productive hour per day still incurs ownership costs during the rest of the day. Electricity consumption during inference is therefore only a small part of the economic decision.

Fixed costs require enough volume

Maintenance, replacement risk, administration, and the operator's time must be distributed across the work the system performs.

Even a few hundred dollars in annual fixed costs can require tens or hundreds of thousands of processed artifacts before the local system reaches parity with an optimized API workflow. The exact threshold depends on utilization, hardware cost, workload size, and how much operational time is counted.

I did not have enough evidence to produce a reliable future demand estimate, so I treated these figures as break-even thresholds rather than forecasts.

The direct energy cost was low. Evaluation, hardware ownership, maintenance, and utilization had a much larger effect on the result.

What owned compute does not provide

A local model does not eliminate the need for frontier models.

My working setup remains dual-lane. Frontier models handle tasks requiring deeper reasoning, broad capability, or orchestration. The local model provides a minimum capability that I can run independently.

The experiment also did not show that buying hardware is generally cheaper than using an API. For many individual workloads, optimized cloud pricing will remain less expensive once hardware depreciation and low utilization are included.

The more defensible reason to maintain a local lane is availability and control. That may justify paying more than the cloud alternative, but the difference should be treated as an explicit availability premium rather than hidden inside an optimistic cost model. In other words, the argument that "open weight" means free, is just simply not true.

The standard I used

Before treating a local model as a dependable part of the system, I required it to meet three conditions:

1. It must clear a defined quality threshold on a bounded, real task using a reproducible harness.
2. Its all-in operating cost must beat the optimized cloud alternative at the expected volume, unless I consciously accept a higher cost in exchange for availability and control.
3. Its failures must be detectable and contained by the surrounding process.

By the end of Part 1, the local baseline had met the first condition for the review task. The cost analysis established the utilization and operating-cost limits for the second. The evaluation harness also provided part of the containment required by the third.

The remaining question was whether fine-tuning could make the model meaningfully better than that baseline.

I tested that question using frozen data and evaluation gates defined before the training runs. Part 3 — The results covers the final scorecard, including the places where the original hypothesis failed.