What Is Nemotron 3.5 Lightning? NVIDIA's Fast Open Agent Model

The Execution Engine Beneath Your Agent

Nemotron 3.5 Lightning is NVIDIA's open-weight AI model for the high-volume, low-latency execution layer of long-running AI agents. It's a 30-billion-parameter mixture-of-experts (MoE) model that activates roughly 3 billion parameters per token — the same lightweight A3B design as its predecessor, Nemotron 3 Nano — but with a substantial intelligence jump on top: NVIDIA reports a 9-point gain on the Artificial Analysis Intelligence Index over Nano.

It shipped alongside NeMo Switchyard, an open-source routing library that sends each step of an agent workflow to the most cost-appropriate model: routine tool calls, output validation, and subagent delegation go to Lightning, while genuinely hard planning and reasoning steps can still route to a larger frontier model like Nemotron 3 Ultra.

Built for Speed, Not Peak Reasoning

NVIDIA reports Lightning completing 10,000 tasks roughly 30% faster than the larger Qwen3.6-35B at comparable accuracy, though independent testers have generally found it trailing Qwen-class models on raw intelligence benchmarks — the trade Lightning is explicitly designed to make in exchange for speed and cost.

Lightning Is a Specialist, Not a Generalist

NeMo Switchyard routes each step of an agent's workflow to the model best suited for that specific task.

The honest framing, echoed across independent reviews, is that Lightning isn't meant to be the one model handling everything. It's best used as the execution layer beneath a smarter orchestrator: formatting results, calling tools, validating routine outputs, and processing long streams of repetitive work, while a stronger reasoning model handles planning and the genuinely hard steps.

The Bigger Shift: Routing Instead of One Big Model

Lightning is part of a broader change in how serious AI agent systems are being architected in 2026. For the past few years, the default approach was to route every request — trivial or complex — through the single most capable model available, because switching between models mid-workflow was awkward to engineer and the cost difference didn't matter much at small scale. That calculus breaks down once an agent is making hundreds or thousands of tool calls per task, since paying frontier-model prices for a routine formatting step or a simple validation check is wasteful in both latency and cost.

NeMo Switchyard formalizes this idea instead of leaving it to ad hoc engineering. Rather than a developer manually deciding which model handles which step, the routing library inspects each step of an agent's plan and sends it to whichever model in the stack is appropriately sized for that specific task — a pattern sometimes called model cascading or tiered inference. The practical effect for teams building agents is a meaningful cost reduction on long-running workflows, since the expensive, powerful model is reserved for the small fraction of steps that actually need it.

This mirrors a similar shift happening across the industry more broadly — both OpenAI and Anthropic have discussed internal routing between model sizes for cost efficiency, though NVIDIA's approach is notable for shipping the routing logic itself as an open-source tool rather than keeping it as an internal implementation detail. That openness is likely to matter for the growing number of teams building their own multi-model agent stacks rather than relying entirely on a single vendor's managed infrastructure.

For a solo developer or small team without the resources to run a full multi-model stack, the practical takeaway is less about adopting Lightning specifically and more about the underlying principle: matching model size to task complexity, rather than defaulting to the biggest available model for every step, is quickly becoming standard practice for anyone running agents at meaningful scale. Even a simple two-tier setup — a cheap, fast model for routine steps and a stronger one reserved for genuinely hard decisions — captures most of the cost benefit without needing the full routing infrastructure NVIDIA has open-sourced.

Whether Lightning specifically becomes a popular choice for that smaller tier will likely depend less on its benchmark scores and more on how well NeMo Switchyard integrates with whatever orchestration framework a given team already uses — tooling friction tends to matter more than a few points of intelligence-index difference when teams are deciding what to actually deploy.

Where It Makes Sense

Frequently Asked Questions