Nvidia targets local AI assistants with a model built around a 4GB memory budget
Nvidia has released a small AI model designed to power local voice assistants, game characters and robots while leaving memory available for the rest of the application. Nemotron 3.5 Bolt E4B 4GB arrived on October 23 with downloadable weights and support for tools including Ollama and LM Studio.
Nvidia’s benchmark comparison uses its selected tests and hardware configurations. Image: Nvidia
The model builds on Google's Gemma 4 E4B and adds Nvidia's training for conversation and actions across multiple turns. It can call application tools, maintain a character's persona and use a conversational mode intended to produce shorter, speech-friendly answers.
The 4GB name depends on what the model is doing
Nvidia's published measurements show why the name needs context. On an RTX 5070, the compressed model used 3.5GB of graphics memory in a short test with about 3,000 input tokens and 50 output tokens. A test retaining 32,000 tokens of context used 4.3GB.
Its Jetson Orin Nano measurements ranged from 4.2GB to 4.9GB in those workloads. These are Nvidia's results for specific settings, rather than a promise that every conversation fits below 4GB. The model supports a context window of up to 64,000 tokens.
Nvidia compares output length and benchmark accuracy across the models it tested. Image: Nvidia
You can download it rather than send every request to a cloud service
The official model page provides the compressed GGUF release and instructions for local inference tools. Nvidia also publishes training recipes and a separate drafter model intended to speed up generation.
Reasoning can be switched on for more complex requests or off for faster replies. The release targets GeForce RTX PCs, Jetson systems, RTX Spark and DGX Spark, extending the local-AI work behind Nvidia’s larger DGX Spark setups to smaller conversational workloads.
Nvidia reports higher average benchmark scores than its previous Nano 4B model, but its own tables also show competitors ahead on individual tests. Application developers still need to check whether the model can reliably perform their particular tasks.
