NVIDIA Releases Nemotron 3.5 ASR: A 600M-Parameter Cache-Conscious Streaming Mannequin Transcribing 40 Language-Locales in Actual Time

NVIDIA’s Nemotron Speech group has launched Nemotron 3.5 ASR. It’s a 600M-parameter streaming Computerized Speech Recognition (ASR) mannequin. A single checkpoint transcribes 40 language-locales in actual time. Punctuation and capitalization are inbuilt natively. The mannequin ships as open weights on Hugging Face. The license is OpenMDW-1.1. The structure is a Cache-Conscious FastConformer-RNNT.

What’s Nemotron 3.5 ASR

Nemotron 3.5 ASR extends nvidia/nemotron-speech-streaming-en-0.6b to many languages. It provides prompt-based language-ID conditioning to the bottom mannequin. That lets one 600M-parameter checkpoint cowl 40 language-locales. No per-language mannequin or model-swapping is required.

The mannequin targets two workloads. The primary is low-latency streaming for dwell audio. The second is high-throughput batch transcription. Output is production-ready textual content with correct casing and punctuation. No separate punctuation-restoration step is required.

How Cache-Conscious FastConformer-RNNT Works

The mannequin has two foremost items. The primary is a Cache-Conscious FastConformer encoder with 24 layers. FastConformer is an environment friendly evolution of the Conformer structure. It makes use of linearly scalable consideration. The second piece is an RNNT (Recurrent Neural Community Transducer) decoder. RNNT emits textual content body by body as audio streams in.

The “cache-aware” design is the effectivity lever. Buffered streaming re-processes overlapping audio home windows at each step. That repeats the identical work and provides delay. This mannequin caches encoder self-attention and convolution activations as a substitute. It reuses these cached states as new audio arrives. So every audio body is processed precisely as soon as, with no overlap. Compute and end-to-end latency each drop, with out an accuracy penalty.

The Latency Knob: att_context_size

One inference setting controls the latency-accuracy tradeoff. It’s the consideration context dimension, att_context_size. Smaller context emits textual content sooner however sees much less future audio. Bigger context raises accuracy at greater latency.

The identical checkpoint covers the total vary. Settings map to chunk sizes of 80ms, 160ms, 320ms, 560ms, and 1.12s. For instance, [56,0] offers an 80ms ultra-low-latency mode. The [56,13] setting offers 1.12s for highest accuracy. Groups choose the working level at inference time, with no retraining.

Language Detection and Protection

The 40 language-locales embrace English, Spanish, German, and French variants. Additionally they cowl Arabic, Japanese, Korean, Mandarin, Hindi, and Thai. A number of different European and Nordic languages are included too.

Language conditioning works two methods. Setting target_lang to a identified locale normally offers the very best accuracy. Setting target_lang=auto lets the mannequin detect the language itself. In auto mode, it emits a language tag after terminal punctuation. One deployment can then transcribe mixed-language site visitors. No separate language-ID element is required.

Comparability

Superb-Tuning Outcomes

As a result of the weights are open, groups can fine-tune for a language, area, or accent. NVIDIA printed a labored instance on Greek and Bulgarian. It fine-tuned the bottom checkpoint with the identical Cache-Conscious FastConformer-RNNT recipe. Every clip carried a target_lang tag for language conditioning. Coaching knowledge got here from public corpora, together with Granary, Widespread Voice, and FLEURS.

Outcomes had been measured as WER on held-out FLEURS, on the 80ms setting. Greek WER fell from 35 to 24, a 32% relative enchancment. Bulgarian fell from 22 to fifteen, a 31% relative enchancment. These are uncooked WER percentages on the lowest-latency streaming mode. NVIDIA notes that evaluating at deployment latency, on held-out knowledge, offers sincere numbers.

Strengths and Concerns

Strengths:

• One 600M-parameter checkpoint covers 40 language-locales, chopping deployment sprawl.
• Cache-aware streaming processes every body as soon as, reported at 17x buffered concurrency on an H100.
• att_context_size tunes latency from 80ms to 1.12s at inference, with no retraining.
• Punctuation, capitalization, and auto language tagging are inbuilt.
• Open weights enabled a 31–32% relative WER drop on Greek and Bulgarian after fine-tuning.

Concerns:

• The mannequin handles English, however NVIDIA recommends its devoted English mannequin for English-only use.
• The 80ms mode trades some accuracy for the bottom latency.
• Japanese and Korean use CER, so cross-language error comparisons want care.
• Throughput figures are measured on H100, so outcomes on different GPUs will differ.
• The manufacturing NIM with gRPC streaming is introduced, however not but launched.

Key Takeaways

• NVIDIA’s Nemotron 3.5 ASR is an open-weights (OpenMDW-1.1), 600M-parameter streaming mannequin transcribing 40 language-locales from one checkpoint.
• Its Cache-Conscious FastConformer-RNNT design processes every audio body as soon as, reported at 17x the concurrent streams of buffered approaches on an H100.
• Latency is configurable from 80ms to 1.12s at inference through att_context_size, with no retraining.
• A brief fine-tune reduce FLEURS WER 32% on Greek (35→24) and 31% on Bulgarian (22→15), on the 80ms setting.
• It’s self-hostable and streaming-native, in contrast to closed APIs (Deepgram, AssemblyAI, ElevenLabs) or offline Whisper.

Marktechpost’s Visible Explainer

Take a look at the Mannequin weights. Additionally, be at liberty to comply with us on Twitter and don’t neglect to affix our 150k+ ML SubReddit and Subscribe to our Publication. Wait! are you on telegram? now you’ll be able to be a part of us on telegram as properly.

Must associate with us for selling your GitHub Repo OR Hugging Face Web page OR Product Launch OR Webinar and many others.? Join with us