Nemotron 3.5 Bolt’s Compact Design Has A Cost: Tokens

🔍 Read the full analysis: Nemotron 3.5 Bolt’s Compact Design Has A Cost: Tokens on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on office and shipping supplies

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

NVIDIA released Nemotron 3.5 Bolt 74B-A9B on October 15, an open-weight model that scored 21.4 on Artificial Analysis’ Intelligence Index v4.3.2, 1.5 points below NVIDIA’s older 550-billion-parameter Nemotron 3 Ultra. The smaller model uses 74 billion total parameters, with 9 billion active per token, but its evaluation required about 82,000 output tokens per task—a trade-off that may mean longer waits for people running it on their own hardware.

Artificial Analysis’ score puts Bolt close to Ultra’s 22.9, despite Bolt having roughly one-seventh as many total parameters and one-sixth as many active parameters. It is the first model in NVIDIA’s new Bolt family. The model accepts text and images, produces text, supports reasoning, and has a 262,144-token context window. The benchmark evaluated its reasoning version.

The score does not make Bolt the leader in its size class. Artificial Analysis ranks it fourth among models below 100 billion total parameters, behind Qwen3.8 27B at 33.7, K2 Horizon MoVA 36B-A4B at 25.3, and G9v3-39A5B at 21.8. It is the highest-scoring US model in that class on the listed comparison. The source says Bolt also accepts images, unlike K2 Horizon MoVA and G9v3.

Bolt’s evaluation profile varies by task. It scored 15.5% on AutomationBench-AA, compared with Ultra’s 3.0%, a result relevant to multi-step workflows across software tools. But Ultra scored higher on long-context reasoning, 79% to Bolt’s 70%, and on Humanity’s Last Exam, 28% to 25%. Both models scored near zero on Terminal-Bench 4.0, so the reported results do not support treating Bolt as a capable shell agent.

Bolt’s Token Use Changes the Trade-Off

The headline score comes with a substantial output-volume caveat. Artificial Analysis reports that Bolt generated about 230 million output tokens across its Intelligence Index evaluation, describing it as very verbose compared with a 100-million-token median for open-weight models of its size. On a per-task basis, the report gives Bolt roughly 82,000 output tokens, against 28,000 for Nemotron 3 Ultra and 67,000 for Qwen3.8 27B.

That difference matters differently depending on how the model is run. For a hosted service, more generated tokens can raise usage charges, though Artificial Analysis lists no hosted API provider, price, speed measurement, or cost per task for Bolt. For self-hosting, the report frames the trade-off in time: at a hypothetical 100 tokens per second, 82,000 tokens would take about 14 minutes to generate; at 50 tokens per second, about 27 minutes. Those are arithmetic estimates, not measured Bolt speeds.

For organizations selecting a model, the results point to a task-specific decision rather than a simple ranking. Bolt’s automation score and image input may suit some uses, while its lower long-document score, high token output, and unmeasured latency may matter more for other workloads. Its status as the top-scoring US model in the cited sub-100-billion-parameter comparison may also be relevant to buyers documenting model provenance, but that does not establish performance or suitability beyond the reported tests.

How Bolt Compares With NVIDIA Models

Bolt follows Nemotron 3 Super, a 120-billion-parameter model with 12 billion active parameters released in March, which scored 12.8 on the same index, according to the source. Bolt’s score is 8.6 points higher, while using 39% fewer total parameters. The comparison suggests a sharp improvement in benchmark score within NVIDIA’s lineup, but it does not by itself explain how the models perform across every practical deployment.

The company’s previous flagship, Nemotron 3 Ultra 550B-A55B, scored 22.9. Bolt’s result is close on the overall index but does not mean the models have the same strengths: Bolt leads on AutomationBench-AA, while Ultra performs better on several knowledge and long-context measures listed in the report. Artificial Analysis also places Bolt behind three other models in the under-100-billion-parameter group, including Qwen3.8 27B, which scores 12 points higher.

NVIDIA describes Bolt as open-weight for self-hosting, and Artificial Analysis classifies its license as permissive. However, the model page does not name the specific license, according to the source. The comparison report also notes that Bolt’s 9 billion active parameters could reduce compute required for each generated token relative to a larger dense model, but says this is an architectural expectation rather than a measured speed result.

“Ultra-class score per task, not per minute.”

— Thorsten Meyer, source report

Speed, Pricing and License Details

Several practical measures remain unavailable in the cited material. No hosted API provider or price is listed, and Artificial Analysis has not published a speed measurement or cost per task for Bolt. The token-generation examples are hypothetical calculations based on assumed speeds, not observed performance on particular hardware.

The report also says the model page classifies the license as permissive but does not identify its name. That leaves the precise license terms to be checked before deployment. The benchmark results describe performance on specified tests; they do not establish how Bolt will perform on a particular organization’s data, tools, hardware, or latency requirements.

The source material ends during its discussion of Bolt’s AA-Omniscience result, reporting a score of −19 but not providing the rest of that analysis. It therefore does not support further conclusions here about the score’s interpretation or the model’s rate of confident errors.

What Deployers Need to Measure

The next useful evidence would be independent or provider-published measurements of tokens per second, end-to-end task time, and cost on specified hardware and workloads. Those figures would show whether Bolt’s lower active-parameter count translates into a practical speed advantage despite its high output volume. A named license and its full terms would also clarify the conditions for commercial and other deployments.

Until those details are available, readers can compare Bolt using the existing benchmark results while treating its speed and deployment economics as unresolved. Teams testing it locally can measure their own generation rate and task completion time, and compare those results with alternatives on the same prompts and hardware. No additional release or benchmark milestone is specified in the source material.

Key Questions

What did NVIDIA release?

NVIDIA released Nemotron 3.5 Bolt 74B-A9B, an open-weight model for self-hosting, on October 15, 2026. It takes text and images as input and produces text.

How close is Bolt to Nemotron 3 Ultra?

Bolt scored 21.4 on Artificial Analysis’ Intelligence Index v4.3.2, compared with 22.9 for Nemotron 3 Ultra. The overall scores are close, but their results differ across individual tests.

Why is Bolt’s token use a concern?

The source reports about 82,000 output tokens per index task. More tokens can mean longer generation times on self-hosted hardware and could affect charges if the model is offered through a paid API; no API price or measured Bolt speed is listed.

Is Bolt the best model under 100 billion parameters?

No. Artificial Analysis ranks it fourth in the under-100-billion-parameter class in the comparison provided. It is listed as the highest-scoring US model in that group.

What is still unknown about using Bolt?

The cited material does not provide a measured generation speed, hosted price, or cost per task. It also says the model page does not name the specific permissive license, so deployers would need to verify its terms.

Source: ThorstenMeyerAI.com