x/nxpc · 
The three philosophies of the ultimate local AI rig — raw bandwidth, raw capacity, and raw value — plus the brand-new NVIDIA model that's quietly changing the math.
There's a specific kind of person who, at 11pm on a Friday, has a terminal window open, a ollama run llama3.3:70b command queued, and a fan curve that sounds like a small jet preparing for takeoff. If that's you — welcome home. We're talking about the thing every local-AI hobbyist eventually confronts: building the box that runs the big models without paying for someone else's data center.
And here's the thing nobody tells you up front: there is no single "best" local LLM rig. There are three, and they're optimized for three completely different jobs. Let's tear into all of them.
Local LLM inference boils down to one number you can't escape: memory bandwidth. When you generate tokens, the model's weights stream through memory constantly — so tokens-per-second is, to a first approximation, "how fast can you shove gigabytes of weights past the compute." That's why two cards with similar CUDA counts can post wildly different tok/s.
But there's a second number that matters just as much: capacity. If the model doesn't fit in VRAM, speed stops mattering entirely. Offload a 70B model to system RAM and your 45 tok/s card turns into a 1-2 tok/s slideshow.
And the third number? Dollars per gigabyte. Because a $4,800 halo card is only "the best" if it's actually the best for your budget.
Here's how the three camps stack up:
| Rig Philosophy | Config | VRAM / Memory | Bandwidth | Sweet Spot | Cost per GB |
|---|---|---|---|---|---|
| Raw bandwidth | RTX 5090 | 32GB GDDR7 | 1,792 GB/s | 32B models at blazing speed | ~$130-150/GB |
| Raw capacity | AMD Strix Halo 128GB | 128GB unified | ~210-256 GB/s | 120B MoE models that fit whole | ~$27-40/GB |
| Raw value | Used RTX 3090 (x1-2) | 24-48GB GDDR6X | ~936 GB/s each | 70B at Q4 on a budget | ~$25-35/GB |
Three machines, three answers to the same question. Let's geek out on each.
The RTX 5090 is the undisputed throughput king. With 32GB of GDDR7 on a 512-bit bus pushing 1,792 GB/s, it simply outruns everything else in a consumer chassis. Independent benchmarks from Hardware Corner measured 102.7 tokens/sec on Qwen3 14B at Q4_K with 16K context — the kind of speed where responses feel instant, not "streamed."
Drop a 70B model in there and Quantize Lab's numbers put the 5090 at 45+ tokens/sec on Llama 3.3 70B fully in VRAM. That's genuinely usable for real chat. But here's the catch that keeps me up at night: 32GB is the ceiling. Dense 70B models need aggressive Q3/Q4 quantization to squeeze in, and a 120B dense model simply doesn't fit. For those, you're offloading to RAM — and the moment you do, that 45 tok/s becomes 1-2 tok/s.
Price check (Aug 14, 2026):
The 5090 is the card you buy when you want a 32B-class daily driver to feel alive — and you accept that the biggest models stay out of reach without a second card.
This is the plot twist of 2025-2026. AMD's Ryzen AI Max+ 395 (Strix Halo) takes a totally different swing: instead of a small pool of screaming-fast discrete VRAM, it gives you 128GB of unified LPDDR5X memory and a genuinely competent Radeon 8060S iGPU (up to 96GB configurable as VRAM). Bandwidth is the tradeoff — real-world testing pegs it around 210-256 GB/s, roughly a quarter of the 5090.
But here's the magic: MoE (Mixture-of-Experts) models. These models have huge total parameter counts but only activate a fraction per token. So a 120B MoE like GPT-OSS streams at a reported ~55 tok/s on Strix Halo, while a 30B-A3B MoE hits ~100 tok/s — because only the active experts need to move through memory. Dense 70B models are slower (~5-8 tok/s), but they fit.
The kicker: a whole Strix Halo mini PC costs about what a bare RTX 5090 costs.
Price check (Aug 14, 2026):
If your dream is running a 120B-class model fully locally on a single silent box, this is the only consumer path that gets you there for under four grand.
Six years after launch, the used RTX 3090 refuses to die. 24GB of GDDR6X at ~936 GB/s, for $600-850 on the secondary market. That's the entire local-AI community's favorite hack: two identical 3090s give you 48GB of fast VRAM for roughly the price of one mid-tier new card — enough to run 70B models at Q4 with headroom to spare.
The 3090 Ti, by the way, dropped NVLink — so if you're going multi-GPU, stick to plain 3090s. On Amazon, "Renewed" listings are heavily marked up ($1,399-1,649), so the real value lives on the used market (eBay, r/hardwareswap, local classifieds). Budget ~$600-800 each and you're cooking.
Right as this VRAM arms race was heating up, NVIDIA dropped a curveball on August 11, 2026: Nemotron 3.5 Lightning, its first big open-source model push in ages. It's a Mixture-of-Experts model so efficient that creators are running a 30B-class model on just 6GB of VRAM at 10+ tokens/sec — on a gaming laptop, no less.
That matters because it quietly undermines the "buy the biggest GPU you can afford" playbook. If MoE models keep trending smaller-per-active-parameter, the capacity crowd (Strix Halo, dual 3090s) wins more ground every month — and the 5090's 32GB ceiling hurts less because you're not trying to cram a giant dense model in there anyway.
Here's my honest take, budget-first:
And if you're like most of us? Optimize the software first. Alex Ziskind's latest video is a wake-up call: one config tweak took his rig from ~120 tok/s to 1,200+ tok/s — a 10x gain that cost zero dollars. Flash-attention, KV cache quantization, and getting off the default context window settings do more than any GPU upgrade. Before you spend a dime, make sure you're not leaving 10x on the table.
There's never been a better time to build a local LLM box — and never a more confusing one. The market has split into three tribes: bandwidth (5090), capacity (Strix Halo), and value (used 3090). Your answer depends on whether you want a 32B daily driver that feels instant, a 120B model that fits whole, or 70B on a shoestring budget.
Me? I'm a capacity guy with value-brain. If I were building tomorrow, I'd grab a Strix Halo 128GB box for the big MoE models and keep a used 3090 around for fast dense-model work. Best of both worlds, still under what a single 5090 costs at retail.
Now if you'll excuse me, I've got a ollama pull running and a fan curve to tune. 🫡
Prices verified via Amazon US/CA on August 14, 2026. Used-market prices are community-reported ranges and will vary.