TL;DR
Thorsten Meyer AI published an analysis arguing that the real cost of a 2026 local-inference rig is set by VRAM capacity, not raw GPU speed. The report says used 24GB RTX 3090 cards can offer stronger VRAM value than newer cards, while warning that prices and benchmarks are point-in-time figures.
Thorsten Meyer AI published a new analysis of local-inference rig costs in 2026, arguing that buyers should size systems around VRAM capacity rather than the newest GPU if they want to run AI models locally without falling into cloud-style spending.
The report says the central constraint is the VRAM cliff: if a model’s weights fit inside fast GPU memory, inference can be usable and quick; if the model spills into system RAM, speed can collapse. Citing community benchmarks, the article says an RTX 5090 running a 70B model fully in VRAM may reach about 40 to 50 tokens per second, while the same model partly offloaded to system RAM can fall to about 1 to 2 tokens per second.
Thorsten Meyer AI attributes that gap to the fact that LLM inference is memory-bandwidth bound. In the report’s sizing guide, 7B to 8B models need about 6GB to 8GB at Q4 quantization, 26B to 32B models fit on a single 24GB card, and 70B models require roughly 43GB, pushing buyers toward a 32GB RTX 5090, dual GPUs, a 64GB-class Mac, or heavier quantization.
The analysis identifies the used RTX 3090 as the main value play. It says a 24GB RTX 3090 selling for about $600 to $850 can deliver roughly five times the VRAM-per-dollar of an RTX 5090, while four used 3090s can provide 96GB for under about $3,200. Those figures are attributed to the site’s late-June 2026 price snapshot, not a standing price guarantee.
The real cost of a local-inference rig
Owning beats renting for steady AI work — so what does a local rig cost in 2026? The unintuitive, good news: the most expensive build is almost never the smartest one. It all comes down to one rule.
The difference is only whether the weights fit. LLM inference is memory-bandwidth-bound — VRAM capacity is the hard limit you build around. Compute specs are mostly noise.
The squeeze reframes the rig like everything else in this series: discipline beats maximalism. VRAM is exactly the memory under most pressure, so over-buying it is the 128GB-“to-be-safe” trap, only worse per gigabyte. Take the cheap, high-value step to 24GB (the gateway to the 30B class), reach for used 3090s and MoE models, and use quantization to climb a tier without buying silicon. Sized right, the rig pays for itself against the cloud’s ever-rising hidden bill. Next: Apple Silicon’s quiet memory advantage.
VRAM Sets the Rig Budget
The analysis matters because it reframes the buy-or-rent question for people running steady AI workloads. For users with high utilization, Thorsten Meyer AI says owning hardware can beat renting cloud GPUs, but only if the system is matched to the actual model class being used.
The report does not say the largest build is the best choice. It argues that many buyers can avoid overpaying by targeting 24GB as the gateway to the 30B model class, using quantization to reduce memory needs, and looking at Mixture-of-Experts models that activate fewer parameters per token. This is cost analysis, not financial, tax, or legal advice.
used RTX 3090 24GB graphics card
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
How the Memory Math Works
The article is Part 7 of Thorsten Meyer AI’s Memory Squeeze series, following a prior installment that argued cloud rentals can hide the full bill for sustained AI use. This installment shifts the question to what it costs to run models on local hardware.
The report’s model map uses Q4 quantization as the practical baseline. It says a model needs about 2GB per billion parameters at FP16 precision, while Q4 can cut the weight footprint to about one-quarter of that level, with quality tradeoffs the source describes as modest for many local-inference uses.
“The most expensive local-inference rig is almost never the smartest one.”
— Thorsten Meyer AI
high VRAM GPU for AI inference
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Prices and Speeds May Shift
Several details remain dependent on the market and individual builds. The source says its GPU prices are from late June 2026, and the cited tokens-per-second figures reflect community benchmarks that can vary by model, quantization level, software stack, cooling, power limits, and prompt workload.
It is also not clear from the source how broadly its cloud-versus-owning conclusion applies outside steady, high-utilization work. Buyers with sporadic use, strict warranty needs, limited power capacity, or low tolerance for used hardware risk may see a different cost profile.
NVIDIA RTX 5090 32GB graphics card
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Apple Silicon Moves Into Focus
The next installment in the series is expected to examine Apple Silicon’s memory advantage, according to the source. That topic matters because large unified memory can change the tradeoff for users who want to run larger local models without building a multi-GPU desktop system.
GPU for local AI model inference
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is the main cost driver for a local AI rig in 2026?
The report says the main driver is VRAM capacity, because model weights must fit in fast memory for usable inference speed.
Is a newer GPU always better for local inference?
No. Thorsten Meyer AI argues that VRAM-per-dollar matters more than raw compute for many inference workloads, which is why a used RTX 3090 24GB can be attractive.
What hardware does the report associate with 70B models?
At Q4, the report puts a 70B model near 43GB, pointing to options such as an RTX 5090 32GB, dual RTX 3090 cards, a 64GB-class Mac, or more aggressive quantization.
Does owning hardware beat cloud rental for everyone?
No. The source says owning can beat renting for steady, high-utilization AI work. It does not claim that local hardware is cheaper for occasional use.
Are the price figures guaranteed?
No. The report labels prices as late-June 2026 point-in-time figures. GPU resale prices, availability, and performance results can change quickly.
Source: Thorsten Meyer AI
