Running LLM models on consumer hardware: Strix Halo, and the unexpected value of owning your AI stack.

As a computer enthusiast I’ve been intrigued by the Framework Desktop since the moment it launched. The value prop is exactly what I want from hardware: Linux-friendly, long term support, repairability, upgradability.

The Desktop is perhaps the first product where Framework breaks one of their design principles (low repairability, little upgrade path). But in exchange, it offers an intriguing trade-off: a massive unified memory system capable of running large LLM models locally, paired with strong compute that’s useful for a wide range of cases.

I postponed the purchase out of pragmatism. Online tools did the job. My use cases were covered. There was no urgency. But then the AI revolution kept getting louder. I could no longer ignore the juicy possibilities of having that kind of capability on my desk. So I started the research.

Although I love the Framework ethos, Corsair had a competing rig at an irresistible price point and immediate stock availability where I live: £2,300 for the AI 300: AMD 395+ with Radeon 8060S, 128GB unified memory, dual 2TB NVMe. That was a clear go vs the £2,700+ Framework with a longer wait (sorry Framework, we’ll meet next time).

When it arrived, I couldn’t be happier. So much to explore.

But this post isn’t really about the hardware.

What I actually want to talk about is what owning the stack gets you that I genuinely did not anticipate. The hardware was the entry ticket. The value that came out of it has been disproportionate to the £2,300 spent (by an order of magnitude or more), and most of that value is invisible to people standing outside the hobby looking in.

So here’s the hierarchy. Ranked by what actually matters.

1. Sovereignty over your stack #

This is the meta-value that frames everything else. When you own the hardware and configure the stack, you stop being a user of someone else’s product and start being the operator of your own infrastructure.

Concretely:

Pay once, yours forever No monthly fee, no rate limits, no dark patterns to endure from products that get enshittified. no “you’ve reached your usage cap, please upgrade.”
Owning your LLM and set up: Once you like a model, it remains as-is for as long as you want. No forced model changes that raise anxiety in users. No broken workflows due to changes in models or tooling.
Fully private: you can be as candid and open as you want, without worrying if your data will be leaked somehow somewhere. No prompts being logged in some lab’s training corpus. No privacy ambiguity about where your most personal questions go
Work offline: You can talk to your model on a long-haul flight or in a remote park with no internet. That is, for any tasks that don’t require the freshest, real-time knowledge.
You set your safety standards: Thanks to uncensored models, you can get responses for any kind of use cases. This is key as AI companies increase their refusal rate purely to mitigate legal risk (which often leads to refusals that should never have happened.)

This matters more than I expected before I had it. There’s a felt difference between renting capability and owning capability, and once you’ve experienced the latter for a few months you don’t really want to go back to the former. Even when the rented capability is technically better.

For anyone running a business or thinking about one, this is no longer optional thinking. Owning your AI toolset and removing dependencies on third parties is becoming a strategic question, not a hobbyist one. The same way owning your domain, your email, and your data are a strategic question.

2. A team in a box #

This was the surprise. I didn’t buy the rig expecting it to function as a startup team. It turns out that with the right setup, it kind of does.

After resigning from corporate, this single piece of hardware has been a goldmine of practical value:

Strategic thinking partner. I’ve used local models to pressure-test business decisions, work through trade-offs, and stress-test plans before talking to humans about them. The quality of the thinking is genuinely high. Not “magical AI gives you the answer”, but “competent senior collaborator who will engage seriously with your reasoning and find holes in it.”
Operations and execution support. Drafting documents, structuring plans, project breakdowns, decision frameworks. The kind of work that previously required either my own time at full attention or paying a consultant.
Coding partner. I launched my portfolio site with extensive AI assistance running locally. Vibe-coding capability is real, and it scales further when you’re not paying per token and not waiting in queue.
A reflection partner. Sometimes you don’t want to think alone. Sometimes you don’t want to think with another human either. The local model fills a third space that didn’t exist before. Neither solo nor social.
The ultimate admin dread-killer. All the bureaucratic nonsense, all the admin stuff that you dread even thinking about. That’s gone through the window. The LLM will be there to get you started, answer your questions, move forward paperwork, clarify what the hell they mean by model 392040-339B on the business critical annual reporting.

Each of these used to require either a team, an agency, or a consultant. None of them are fully replaced by a local rig. Humans still do things models can’t, and high-end frontier models still beat local on the hardest tasks. But for the day-to-day operating tempo of a one-person business, the rig replaces a meaningful slice of what would otherwise be hired help.

For anyone launching a business, transitioning out of a corporate role, or running a small operation: this is the case for owning the hardware that I’d lead with.

3. Learning AI from the inside out #

This is the value I most underestimated when I bought the rig, and the one I’d now argue is the most durable.

When you run models locally, you stop thinking of them as magic. You start thinking of them as systems you can tune. Over a few months of working with the rig, I’ve developed real intuition about:

How context window size affects speed and quality. Most importantly how ‘more context’ can actually harm results and sometimes a sharp short prompt beats eons of user data provided as context through memory, RAG documents or prompting.
The difference between a well-prompted small model and a badly-prompted big model, and how the former regularly wins.
How parameters like temperature, top-K, top-P, and repetition penalty actually impact quality, observable across repeated prompts.
The relationship between model size, architecture (Mixture of Experts vs Dense), and quantization, and how each affects quality and speed differently.
The relationship between quantization in weights vs quantization in cache: these are different decisions with different effects.
The quality differences between different uploaders’ models, censored and uncensored, even of the same base model.

This isn’t theoretical knowledge from reading papers. It’s grounded, tested, observed. The kind of intuition you only get by running the experiments yourself, watching the results, and developing a subconscious sense of cause and effect.

This intuition compounds. It makes me a better prompter on hosted services too. It makes me better at evaluating AI products as a design leader or product manager. It makes me more useful in any conversation about AI tooling.

If you work in tech, design, product, or anything adjacent to AI strategy, this alone justifies the hardware cost.

4. The voice and multimodal stack you don’t have to wait for #

Specific story worth telling. I set up Open WebUI on my rig with local llama.cpp serving the model, plus Whisper for speech-to-text and Piper for text-to-speech, both running locally. Cross-device access from my laptop and phone configured in a day thanks to Tailscale.

The result: a voice conversational interface that I genuinely prefer to Claude’s hosted voice mode for most use cases. Snappier, more configurable, runs offline and it doesn’t break like Claude does (issues with voice interrupting itself, screen lock, etc).

The point is that for a wide range of real use cases, a thoughtfully configured local stack is competitive with or superior to the hosted equivalent, and you don’t have to wait for any vendor to decide what features are worth shipping. You build it yourself, the components are already open source, and you control the whole loop.

5. The tinkerer’s playground #

Last in the hierarchy because it’s least universal.

Running local models is a deeply rewarding tinkering domain. Every month brings new model releases, new incremental or exponential improvements across models, tooling, and user experience. The rabbit holes are infinite and most of them produce real upgrades to the working stack.

I impulsively bought a Strix Halo to see what it could offer. The return on investment has been insane, through direct and practical use cases (thinking partner, coder, etc) as well as indirect ones (learning LLM inside out, learning adjacent technologies, infinite ways of doing research, etc).

Given the challenges with hosted solutions, owning your own AI stack and tooling looks like a strategic option for a range of businesses and people. Open weights and tooling made this possible. Let’s see what the trend is for the coming years.