---
license: cc-by-nc-4.0
base_model: mistralai/Voxtral-4B-TTS-2603
pipeline_tag: text-to-speech
tags:
  - audio.cpp
  - gguf
  - text-to-speech
language:
  - en
  - fr
  - es
  - de
  - it
  - pt
  - nl
  - ar
  - hi
---

# Voxtral 4B TTS 2603 GGUF

Self-contained models for [audio.cpp](https://github.com/0xShug0/audio.cpp).
Each file includes the Ministral AR backbone, acoustic flow transformer,
Voxtral Codec decoder, tokenizer, model spec, and all **20 preset voices**.
No separate tokenizer, codec, or voice files are needed.

Upstream: [mistralai/Voxtral-4B-TTS-2603](https://huggingface.co/mistralai/Voxtral-4B-TTS-2603),
pinned to [`b81be46c3777f88621676791b512bb01dc1cb970`](https://huggingface.co/mistralai/Voxtral-4B-TTS-2603/tree/b81be46c3777f88621676791b512bb01dc1cb970).
The model and preset embeddings are licensed under **CC BY-NC 4.0**.

The released checkpoint does not include the audio encoder required for
arbitrary-reference voice cloning. These packages support preset-voice TTS.

## Files

| File | Size | Notes |
| --- | --- | --- |
| `voxtral-4b-tts-2603-bf16.gguf` | 8.03 GB | Original BF16 weights, with folded codec weight normalization. |
| `voxtral-4b-tts-2603-q8_0.gguf` | 4.72 GB | Q8_0 matrices; lower measured VRAM and faster CUDA inference. |

Both packages preserve preset embeddings in their original BF16 precision.
The BF16 conversion folds codec weight normalization and materializes the
normalized semantic codebook. Metadata-only updates are checked against all
400 tensor payloads.

## Usage

```bash
audiocpp_cli --task tts --family voxtral_tts \
  --model /path/to/voxtral-4b-tts-2603-q8_0.gguf --backend cuda \
  --text "The next train leaves in ten minutes. Please wait by platform five." \
  --request-option voice_id=casual_male --seed 42 --out speech.wav --log
```

Use `--mode streaming` for incremental audio. Long input uses audio.cpp's
framework text chunker, controlled by `text_chunk_size` and `text_chunk_mode`.
Codec decoding uses bounded initial/continuation graphs rather than retaining
a graph for every recording length.

Options and HTTP examples: [Voxtral TTS documentation](https://github.com/0xShug0/audio.cpp/blob/main/docs/models/voxtral_tts.md).

## Presets

| Group | IDs |
| --- | --- |
| General | `casual_female`, `casual_male`, `cheerful_female`, `neutral_female`, `neutral_male` |
| Portuguese | `pt_male`, `pt_female` |
| Dutch | `nl_male`, `nl_female` |
| Italian | `it_male`, `it_female` |
| French | `fr_male`, `fr_female` |
| Spanish | `es_male`, `es_female` |
| German | `de_male`, `de_female` |
| Arabic | `ar_male` |
| Hindi | `hi_male`, `hi_female` |

## Controlled Parity

These are **separate from performance measurements**. C++ and Python used
F32 weights/computation and F32 AR caches with TF32 disabled through the
environment. Conditioning history and CUDA noise were controlled for the
component comparisons. The source weights are originally BF16.

| Comparison | Result |
| --- | --- |
| Ministral AR, 40 frames with identical input history | 40/40 semantic predictions match; minimum sampled hidden-state cosine 0.99999969. |
| Acoustic flow, identical conditioning and noise | 1,435/1,440 acoustic codes match; five one-bin rounding differences. |
| Codec, full waveform from identical codes | Cosine 0.999997; RMSE 0.0000829. |

The hidden-state check samples 40 values per frame, not the complete vectors.
The codec check compares complete waveforms. Free-running generation is not
waveform-identical: a quantization difference can change later AR conditioning.
The first such difference in the controlled free-running check occurred at
frame 14. No model or backend precision hacks are included in the port.
The installed vLLM-Omni 0.26.0 reference generates noise on CPU, whereas
audio.cpp uses CUDA-style Philox noise. Matching seeds alone therefore do not
align the two random trajectories; the parity checks explicitly align noise.

Normal-run audio.cpp ASR checks recovered the submitted English text from
BF16, Q8_0, and Vulkan output, and the submitted French and Hindi text from
the multilingual BF16 cases. Repeating the same request after changing voice,
text length, and seed produced byte-identical WAVs for both packages.
CUDA and Vulkan were tested with BF16 and Q8_0; CPU was tested with Q8_0.
Metal has not been tested.

## CUDA Performance

RTX 5090, audio.cpp Debug build, 8 CPU threads. Same text, `casual_male`, seed 42,
7 Euler steps, guidance 1.2, maximum 2,048 frames. Performance uses normal
defaults: **no TF32 environment override and no forced computation precision**.

Python uses the official vLLM-Omni 0.26.0 `Omni.generate` pipeline with its
default BF16 precision. C++ uses native package storage and its default F16
AR cache. Warm timings exclude model loading and use an already-loaded model.

| Runtime | First inference | Warm inference | Generated duration (warm) | Warm RTF | Observed peak VRAM |
| --- | ---: | ---: | ---: | ---: | ---: |
| Python vLLM-Omni | 945 ms | 778 ms | 5.36 s | 0.145 | 26,244 MiB |
| C++ BF16 | 1,103 ms | 816 ms | 5.12 s | 0.159 | 8,780 MiB |
| C++ Q8_0 | 952 ms | 621 ms | 5.04 s | 0.123 | 5,626 MiB |

Output durations differ because generation trajectories differ. RTF is
inference wall time divided by each output's actual duration. Peak memory
includes startup and the sequential requests, sampled per process every
20 ms for C++ and 50 ms for Python. Python's number includes vLLM's reserved
serving cache; it is **not a minimum hardware requirement**. Its deployment
used stage memory utilization 0.7/0.1, maximum 32 sequences, and codec chunks
of 5 initial frames, 25 continuation frames, and 25 left-context frames.

## Long-Form

The shared audio.cpp harbor-text case produced **343.44 seconds** of speech
in **46.63 seconds** after a short warmup: **RTF 0.136**, observed peak
**8,768 MiB**, using BF16 and default text chunking. The framework split it
into 28 text chunks. This is a C++ long-form result, not a Python comparison.
An audio.cpp Qwen3-ASR round-trip recovered 1,025 words against 1,028 submitted
words, with 1.75% normalized WER, including the beginning and final sentence.
That check includes ASR recognition errors and is not a Python parity score.

## Memory Accounting

An allocation trace of the repeated BF16 request case identifies:

| Device allocation | Size at the traced peak |
| --- | ---: |
| AR weights | 6,642.62 MiB |
| Acoustic flow weights | 702.50 MiB |
| Codec weights | 318.17 MiB |
| AR decode buffer, including the F16 KV cache | 239.80 MiB |
| Current AR prefill buffer | 92.07 MiB |
| Two bounded codec workspaces | 98.16 MiB |
| Remaining traced constants, embeddings, CUDA workspaces and graph state | 61.24 MiB |
| Total traced CUDA allocations | 8,154.56 MiB |

NVML reports 8,780 MiB for the process. The approximately 625 MiB difference
is outside the traced tensor allocations, including driver/context/module
overhead and allocation granularity. There is one copy of each weight store,
one acoustic graph, and two codec graphs. Changed prompt lengths replace the
prefill graph; they do not accumulate graphs across requests.

## Conversion

```bash
python tests/voxtral_tts/convert_gguf.py \
  --model-dir /path/to/Voxtral-4B-TTS-2603 \
  --output-dir /path/to/Voxtral-4B-TTS-2603-GGUF --type q8_0
```

The conversion script is part of audio.cpp. Preset embeddings and tokenizer
resources are embedded in the output; inference does not read the original
checkpoint directory.