NVIDIA Nemotron 3.5 Bolt E4B Review: Open Weights for Edge Agents and Local Deployment

NVIDIA released the Nemotron 3.5 Bolt E4B 4GB model on October 23, 2026. The open-weight small language model targets local conversational and multi-turn agent workloads on devices with limited memory. It runs on GeForce RTX, Jetson, and similar platforms while supporting tool calling and configurable reasoning.

Core Capabilities and Technical Specifications

The model contains roughly 8 billion total parameters with 4 billion effective in its dense architecture, derived from Gemma 4 E4B. It handles a 64K context window and processes text input and output only. Post-training emphasized tool use, instruction following, and output length control through Nemotron RL methods without a final supervised fine-tuning stage in some blends.

Key modes include conversational output tuned for concise or speech-ready responses, plus structured multi-turn tool calling. Users can toggle reasoning on or off and set a compute budget. The release also includes open datasets and training recipes under the OpenMDW-1.1 license, which permits commercial use.

Deployment works through llama.cpp, Transformers, Ollama, LM Studio, and Unsloth. NVIDIA provides a free API endpoint on build.nvidia.com and NIM containers. The BF16 checkpoint requires about 15.88 GB, while the Q4_K_M GGUF version targets the 4 GB footprint in practice.

Performance Metrics, Hardware Results, and Comparisons

NVIDIA reports that the model leads 15 of 21 selected agent and conversational benchmarks in BF16 tests. Average quality improved roughly 40 percent over the prior Nemotron 3 Nano 4B and about 10 percent over peers such as Gemma 4 E4B and Qwen 3.5 4B. Notable scores include 84.8 on IHEval, 67.0 on RoleBench, 53.1 on MultiChallenge, and 63.2 on RelyToolBench. It trails on Tau2-Bench at 55.1 versus Qwen’s 75.1 and on some long-context RULER 64K results at 87.0 versus 95.1.

Measured footprints in Q4_K_M GGUF with batch size 1 show 3.1–4.3 GB across tested devices. On an RTX 5070 the model uses 3.5 GB, delivers 548 ms time-to-first-token, and reaches 133.4 tokens per second on short context. Jetson Orin Nano reports 4.2 GB, 5.8 s TTFT, and 15.5 tokens per second. An optional DFlash drafter can add up to 21 percent throughput.

Early third-party notes from Scale Labs rank the model first among small models on MultiChallenge. Broader independent validation on diverse production setups remains limited as of late October 2026. No widespread security incidents or major bugs have surfaced in the first days after release.

Limitations and Practical Deployment Notes

The model lacks native multimodal or speech input, restricting it to text pipelines. Long-context performance trails some competitors on certain benchmarks, and real-world agent reliability on varied hardware needs further community testing. Customization remains possible through post-training, yet full vLLM integration and production harness examples are still emerging.

Users running multiple models simultaneously benefit from the low memory footprint, which leaves headroom on 8 GB and 12 GB cards. Typical workloads fit within 4.3 GB, though BF16 inference demands significantly more. The permissive license supports both research and commercial redistribution of weights and fine-tunes.