x/nxpc · 
By Ryan | nxpc Hardware Reviews | August 21, 2026
Let me paint you a picture. It's 11 PM. You've just installed Ollama. You type ollama run llama3.1:8b and watch tokens fly by at 100+ per second. You feel like a wizard. Then you get greedy — "let me just try the 70B." And you hit it. The wall.
"Error: model requires more VRAM than available."
That wall is VRAM, and it's the single biggest constraint in local AI. But here's the thing: August 2026 is the most exciting moment in local LLM hardware history. We have Intel crashing the party with a 32GB card at $949, dual RTX 5090s beating the H100 in raw tokens-per-second for 70B models, and used RTX 3090s with NVLink still holding the budget crown.
I've spent the week digging through every benchmark I can find — from DatabaseMart's dual 5090 Ollama runs to Hardware Corner's meticulous context scaling tests to VRLA Tech's vLLM production numbers. Let's build the dream local AI rig, tier by tier.
Before we spec anything, let's get oriented. Here's what actually matters for local LLM performance:
Memory bandwidth is king. LLM inference is memory-bandwidth-bound, not compute-bound. During token generation, the GPU reads model weights from VRAM on every single forward pass. A 70B model at Q4 quantization means reading roughly 40GB of weights per token. The faster your GPU can read from VRAM, the more tokens you get. This is why a 1,792 GB/s RTX 5090 smokes a 1,008 GB/s RTX 4090 even if you're not using the extra CUDA cores.
VRAM capacity is the gatekeeper. If your model doesn't fit entirely in VRAM, you're offloading to system RAM, and your tokens-per-second drop from 45 to about 1.5. That's not an exaggeration — it's a cliff.
Multi-GPU works, but it's not linear. Adding a second GPU over PCIe gives you roughly 1.4–1.6× the single-GPU throughput (not 2×), because inter-GPU communication has overhead. NVLink gets you closer to 1.8×, but NVIDIA killed consumer NVLink after the RTX 3090.
In March 2026, Intel dropped a bomb: the Arc Pro B70 with 32GB of GDDR6 on a 256-bit bus, 32 Xe Cores at 2,800 MHz, and a rated 608 GB/s of memory bandwidth — for $949 USD.
Let that sink in. Before this card, the cheapest path to 32GB VRAM was either a $4,500 RTX 5090 or a pair of used RTX 3090s at $1,400+. The Arc Pro B70 is the first card under $1,000 that lets you run a 32B model entirely in VRAM.
The benchmarks are surprisingly solid. Early community testing on llama.cpp shows roughly 45–55 tok/s on 14B-class models at Q4_K_M quantization and 18–22 tok/s on 32B models. That's roughly RTX 3090 territory for generation speed, but with 8GB more VRAM and half the power draw.
The catch? Software maturity. Intel's oneAPI and SYCL stack has come a long way, but if you're deep in the CUDA ecosystem (flash-attention, custom kernels, vLLM), you'll hit rough edges. For Ollama and llama.cpp users, though? It just works. The Arc Pro B70 supports Flash Attention 2 and the latest llama.cpp builds detect it automatically.
| Spec | Intel Arc Pro B70 |
|---|---|
| VRAM | 32GB GDDR6 |
| Bandwidth | 608 GB/s |
| FP32 | 22.9 TFLOPS |
| TDP | ~200W |
| MSRP | $949 USD |
| Best for | 7B–32B models, budget builds |
If you want to run Llama 3.3 70B at Q4 locally without selling a kidney, dual RTX 3090s with NVLink is still the answer in 2026. The math is beautiful: 2 × 24GB = 48GB unified VRAM pool via NVLink at 112.5 GB/s. That's enough for any 70B at Q4_K_M with room for a 16K+ context window.
The RTX 3090 is the last consumer NVIDIA GPU with NVLink support. NVIDIA removed it from the 40-series and 50-series. This makes the 3090 uniquely valuable — it's the only consumer card where you get true hardware VRAM pooling rather than software-level layer splitting.
Real-world performance: 14–16 tok/s on Llama 3 70B Q4 via Ollama or llama.cpp. That's not blazing fast, but it's perfectly usable for chat — roughly reading speed. And at $1,400–$2,000 total (two renewed 3090s at ~$700–$900 each plus a $50 NVLink bridge), it's the cheapest path to 70B inference that doesn't involve offloading to RAM.
Hardware notes: Both cards must be identical RTX 3090s (not 3090 Ti — Ti dropped NVLink). You need a motherboard with two properly spaced PCIe x16 slots and at minimum a 1,000W PSU. Blower-style cards handle thermals better than open-air coolers when stacked. These things run hot.
No NVLink here (RIP), but the RTX 4090's raw compute — 16,384 CUDA cores and 1,008 GB/s of bandwidth per card — means each GPU chews through its assigned layers much faster. Over PCIe 4.0 at 32 GB/s, dual 4090s running Llama 3 70B Q4 deliver 20–24 tok/s. That's a 40–50% uplift over dual 3090s and fast enough that you're waiting on your reading speed, not the GPU.
The cost is steep — two RTX 4090s at $2,700–$3,500 each (yes, they're still expensive even discontinued) plus a 1,200W PSU and a case that can physically fit two 3.5-slot behemoths. But if you want the fastest 70B experience on consumer hardware, this is it.
This is the build that made my jaw drop. DatabaseMart benchmarked dual RTX 5090s running Llama 3.3 70B on Ollama 0.6.5 and the results are staggering:
| Metric | Dual RTX 5090 | NVIDIA H100 | Dual A100 40GB |
|---|---|---|---|
| Model | Llama 3.3 70B | Llama 3.3 70B | Llama 3.3 70B |
| Eval Rate (tok/s) | 27.03 | 24.34 | 18.91 |
| Combined VRAM | 64GB GDDR7 | 80GB HBM2e | 80GB HBM2e |
| Approx. Cost | ~$9,000 (buy) | ~$30,000+ | ~$24,000+ |
Read that again: dual consumer RTX 5090s outperforming an H100 in raw tokens-per-second for 70B inference. The H100 and A100 pull ahead at multi-user serving and training with NVLink scaling, but for a single user running Ollama? Two 5090s is the new meta.
Single RTX 5090 numbers are equally impressive. Hardware Corner's meticulous benchmarks show 102.7 tok/s on Qwen3 14B Q4 at 16K context and 43.8 tok/s on Qwen3 32B Q4 at 32K context. The 1,792 GB/s GDDR7 bandwidth is doing heavy lifting.
The 32GB limit means a single 5090 can't fit a 70B model — but that's where the second card comes in. Dual 5090s give you 64GB of combined VRAM with PCIe 5.0 bandwidth, comfortably handling 70B at Q4 with headroom for KV cache.
If money is genuinely no object: the RTX PRO 6000 Blackwell packs 96GB of ECC GDDR7 on a single card with 1,800 GB/s of bandwidth. That means you can run Llama 3 70B at FP8 on a single GPU with room to spare. No multi-GPU complexity, no layer splitting, no PCIe bottlenecks.
VRLA Tech's published vLLM benchmarks put the RTX PRO 6000 at ~8,425 tok/s on Qwen3-Coder-30B — 1.8× faster than a single RTX 5090. For 70B models, you're looking at 40+ tok/s full FP8 on one card.
The catch? It's a workstation card priced like one. But for production inference serving or if you just want the cleanest possible setup, nothing else comes close in a single slot.
Here's the master comparison — generation speed for the most popular model sizes across every configuration we've discussed:
| Configuration | VRAM | 8B Q4 tok/s | 14B Q4 tok/s | 32B Q4 tok/s | 70B Q4 tok/s | Cost (USD) |
|---|---|---|---|---|---|---|
| Arc Pro B70 | 32GB | ~110 | ~50 | ~20 | N/A | $949 |
| RTX 3090 (single) | 24GB | ~115 | ~52 | ~22 | N/A | ~$1,600 |
| RTX 4090 (single) | 24GB | ~155 | ~79 | ~33 | N/A | ~$2,800 |
| RTX 5090 (single) | 32GB | 200 | 103 | 44 | N/A | ~$4,500 |
| RTX PRO 6000 (single) | 96GB | ~165 | ~130 | ~8,425* | ~40+ | ~$8,000 |
| Dual RTX 3090 (NVLink) | 48GB | — | — | ~30 | 15 | ~$1,700 |
| Dual RTX 4090 (PCIe) | 48GB | — | — | ~45 | 22 | ~$3,600 |
| Dual RTX 5090 (PCIe) | 64GB | — | — | ~65 | 27 | ~$9,000 |
*vLLM production serving with batching — single-user interactive is lower. All Ollama/llama.cpp numbers from community benchmarks, ±10% depending on quantization, context length, and model architecture.
Because Steve's Canadian and dual-market pricing is what we do here at nxpc:
| GPU | Amazon US (USD) | Amazon CA (CAD) | Notes |
|---|---|---|---|
| Intel Arc Pro B70 | $949 (MSRP) | ~$1,350 est. | New, limited availability |
| RTX 3090 (Renewed) | $1,470–$1,730 | $1,700–$2,715 | NVLink bridge: $40–$80 USD |
| RTX 4090 | $2,700–$3,490 | $5,299–$6,499 | Discontinued, limited stock |
| RTX 5080 (new, 16GB) | $1,478–$1,992 | $2,229–$2,799 | Not recommended for LLM — 16GB too tight |
| RTX 5090 | $4,299–$4,899 | $5,947–$8,200 | Wide variance between AIB models |
| RTX PRO 6000 Blackwell | ~$8,000+ | ~$11,500+ | Workstation channel pricing |
Exchange rate context: These CAD prices imply an effective rate of roughly 1.38–1.42 CAD per USD, which is consistent with current mid-2026 forex. Canadians are paying a 5–15% premium on top of the exchange rate for most GPUs.
You've got the hardware — now what do you run?
Ollama is the easiest on-ramp. ollama run llama3.3:70b — that's it. It auto-detects multi-GPU setups and handles layer distribution. For dual GPU configs, it just works. The DatabaseMart benchmarks above were all Ollama 0.6.5.
llama.cpp gives you more control. The --tensor-split flag lets you manually assign layer proportions across mixed GPUs (e.g., --tensor-split 18,14 to split a 32-layer model across a 5090 and 4090). Use -ngl 99 to force all layers to GPU. For the Arc Pro B70, use the latest llama.cpp with SYCL backend.
vLLM is for production serving. It supports tensor parallelism across GPUs, continuous batching, and PagedAttention for efficient KV cache management. This is what the 8,425 tok/s RTX PRO 6000 number comes from — it's a different beast than interactive chat.
If you have $1,000: Get the Intel Arc Pro B70. 32GB of VRAM at $949 is unprecedented. You'll run any 32B model comfortably and even squeeze in a 70B with aggressive quantization plus some RAM offload. It's the new budget king, and frankly, it makes the RTX 5060 Ti 16GB ($729 USD) look silly for AI work.
If you have $1,700: Dual used RTX 3090s with NVLink. This is the sweet spot for 70B model inference. 48GB unified VRAM, 14–16 tok/s, and a proven CUDA ecosystem. The NVLink advantage is real — you're getting true hardware pooling that no 40-series or 50-series card can offer.
If you have $4,500: A single RTX 5090. It's the fastest single consumer GPU for local LLMs, period. 200 tok/s on 8B models, 103 tok/s on 14B, and 44 tok/s on 32B — all at usable context lengths. Add a second one later when you're ready to tackle 70B models.
If you have $9,000: Dual RTX 5090s. This setup literally beats an H100 for 70B Ollama inference. Let that sink in. You're getting H100-tier performance (actually better) for roughly one-third the cost. Plus you can game at 8K when you're not chatting with Llama.
If you have FU money: RTX PRO 6000 Blackwell. 96GB ECC GDDR7. Run 70B at FP8 on one card. Run 120B MoE models. Fine-tune QLoRA without breaking a sweat. It's a workstation card but it's the cleanest possible local AI experience.
Here's something I haven't seen discussed enough: the cost-per-token break-even between local and cloud. Let's do the math.
At $0.06 per million tokens for cloud inference (roughly the going rate for 70B via OpenRouter in mid-2026), a dual RTX 3090 build at $1,700 would need to generate about 28 billion tokens to break even. If you chat with AI for 2 hours a day at 25 tok/s, that's roughly 180,000 tokens per day. Break-even: about 430 days.
But here's the thing: you're not just buying tokens. You're buying privacy, zero rate limits, offline capability, and the freedom to run uncensored models. For a lot of us, that's worth more than the break-even math.
And honestly? There's something magical about watching 70 billion parameters run entirely on GPUs sitting in a box under your desk. No cloud, no API key, no one else's computer. Just you and the silicon.
Questions? Build ideas? Hit me up in the comments. What's your local LLM rig looking like in 2026?