--- license: cc-by-4.0 tags: - audio.cpp - automatic-speech-recognition - speech-translation - gguf --- # OWSM GGUF Self-contained OWSM v4 models for [audio.cpp](https://github.com/0xShug0/audio.cpp). The tokenizer, feature normalization, runtime configuration, and model spec are embedded in each GGUF. The model family is `owsm`. Filenames identify the version, variant, parameter count, and storage type, for example `owsm-v4-base-102m-f32.gguf`, `owsm-v4-small-370m-q8_0.gguf`, and `owsm-v4-medium-1b-q4_k.gguf`. Only v4 checkpoints are currently supported. Upstream checkpoints: - [Base 102M](https://huggingface.co/espnet/owsm_v4_base_102M), revision `96dbb52dba9e173a0a433b9adda6b52da71a53e4`. - [Small 370M](https://huggingface.co/espnet/owsm_v4_small_370M), revision `adf541c613f1a2c90fdf972875c82d76888b099a`. - [Medium 1B](https://huggingface.co/espnet/owsm_v4_medium_1B), revision `2e27c99d9be596ce188997d80d0e50408cae54a7`. ## Warmed Long-Form Performance - Hardware: AMD Ryzen 9 7900X and NVIDIA GeForce RTX 5090; eight CPU threads. - Software: audio.cpp debug build; ESPnet 202610.post2 and PyTorch 2.11.0+cu128. - C++ CPU measurements use a full-model CPU-only debug build with CUDA and Vulkan disabled, native CPU instructions, OpenMP, and eight threads. GPU measurements use the separate GPU-enabled build. Removing GPU backends reduced Base F32 peak RSS by about 283 MiB in a paired check; the tables below use the measured CPU-only results for every C++ CPU row. - Both implementations use English transcription, utterance timestamps, `beam_size=1`, and no previous-text conditioning. This compares matched greedy decoding, not ESPnet's default beam size of five. - Each measurement starts a fresh process, loads one model, runs a 3.505-second warmup, then transcribes the same 327.6-second recording once. Runs are sequential. - Inference time excludes model loading, file reading, and warmup. C++ reports `session.wall_ms`; Python times `Speech2Text.decode_long`, with CUDA synchronized before and after. RTF is inference seconds divided by 327.6; lower is better. - Peak RAM is process RSS; peak VRAM is process GPU memory reported by NVML, not PyTorch's allocated-tensor counter or total desktop GPU usage. Both are sampled with the same monitor at a requested 20 ms interval across loading, warmup, and inference. Sampling can miss brief peaks. - Python uses its default F32 inference, without TF32, precision, or randomness overrides. The F32 comparisons below use F32 GGUFs; quantized C++ packages are reported separately. These performance runs are not controlled parity tests. ### CPU: F32 | Model | C++ inference | Python inference | C++ RTF | Python RTF | C++ peak RSS | Python peak RSS | | --- | ---: | ---: | ---: | ---: | ---: | ---: | | Base | 10.124 s | 22.198 s | 0.03090 | 0.06776 | 1,099 MiB | 1,878 MiB | | Small | 39.620 s | 63.378 s | 0.12094 | 0.19346 | 2,709 MiB | 3,850 MiB | | Medium | 91.704 s | 167.318 s | 0.27993 | 0.51074 | 5,644 MiB | 8,840 MiB | ### CUDA: F32 | Model | C++ inference | Python inference | C++ RTF | Python RTF | C++ peak VRAM | Python peak VRAM | | --- | ---: | ---: | ---: | ---: | ---: | ---: | | Base | 1.193 s | 5.083 s | 0.00364 | 0.01552 | 1,578 MiB | 1,882 MiB | | Small | 1.986 s | 7.232 s | 0.00606 | 0.02208 | 3,142 MiB | 4,544 MiB | | Medium | 3.986 s | 13.244 s | 0.01217 | 0.04043 | 6,052 MiB | 9,628 MiB | CUDA process peak host RAM was 773 / 836 / 1,148 MiB for C++ and 2,381 / 2,648 / 5,015 MiB for Python, in Base / Small / Medium order. These are single warmed long-form measurements, not averages. Autoregressive outputs can differ without numerical controls: Medium CUDA differed near the end of this recording, including a repeated word and subsequent segment boundaries. Do not interpret these timing tables as exact-parity guarantees. ## Parity Checks All long-form segment texts and start/end timestamps matched Python exactly: 128 segments for Base, 116 for Small, and 112 for Medium. The variants can segment the same recording differently. All three also matched Python on a separate short recording. These are focused parity checks, not a multilingual accuracy benchmark. This paragraph describes the separate CPU parity checks; the performance methodology and memory measurements are listed above. Base F32 also matched Python text and timestamps on five separate FLEURS recordings: one each in English, Russian, Kazakh, Kyrgyz, and Uzbek. All three F32 variants matched Python on the short English-to-German speech-translation case. With previous-text conditioning enabled, Base also matched all 41 Python segments and timestamps on the same long recording. With `beam_size=5`, Base matched all 133 Python segments and timestamps on that recording, plus two short transcription cases and English-to-German translation. The performance tables above use greedy decoding, not beam search. With `language=auto`, Base matched official Python language identification and the subsequent transcript and timestamps on all five FLEURS recordings. Both implementations classified the Kyrgyz recording as Kazakh, so an explicit language remains useful when it is known. ## Quantized Performance The same warmed long-form protocol gives the following results for the distributed quantized packages. Compare each variant with its Python F32 row in the corresponding CPU or CUDA table above; Python was not quantized. ### CPU The word-disagreement column comes from the separate CPU output comparison against Python F32, using lowercase words with punctuation removed. It is not error against a human transcript. The new performance runs retained exactly the same quantized CPU outputs as that comparison. | Model / storage | Inference | RTF | Peak RSS | Word disagreement with Python | | --- | ---: | ---: | ---: | ---: | | Base Q8_0 | 6.899 s | 0.02106 | 913 MiB | 1.21% | | Small Q8_0 | 28.648 s | 0.08745 | 1,834 MiB | 0.22% | | Small Q4_K | 26.340 s | 0.08040 | 1,704 MiB | 1.32% | | Medium Q8_0 | 60.514 s | 0.18472 | 3,079 MiB | 0% | | Medium Q4_K | 58.544 s | 0.17870 | 2,651 MiB | 1.87% | ### CUDA | Model / storage | Inference | RTF | Peak VRAM | Peak host RAM | | --- | ---: | ---: | ---: | ---: | | Base Q8_0 | 0.984 s | 0.00300 | 1,384 MiB | 823 MiB | | Small Q8_0 | 1.587 s | 0.00484 | 2,300 MiB | 850 MiB | | Small Q4_K | 1.557 s | 0.00475 | 2,170 MiB | 862 MiB | | Medium Q8_0 | 2.989 s | 0.00912 | 3,466 MiB | 964 MiB | | Medium Q4_K | 2.874 s | 0.00877 | 3,044 MiB | 950 MiB | Quantization changes timestamp segmentation as well as some words; it is not exact-parity. Base Q4_K is larger than Q8_0 because many 384-wide matrices cannot use its 256-element blocks and remain F32. Q8_0 is the more promising Base CPU trade-off, but these checks alone do not establish multilingual accuracy. Medium Q8_0 CPU has identical normalized words on this recording, but 14 of 112 segments differ in timestamps or exact text formatting. ### Backend Checks Base F32 returned identical transcripts, German translation, and timestamps on CPU, CUDA, and Vulkan for the short repeated-request sequence. This is targeted backend coverage, not a full GPU accuracy or performance benchmark. Small and Medium Q8_0 also matched their respective CPU results exactly on CUDA and Vulkan for that sequence. ## Autoregressive Usage ```bash audiocpp_cli --task asr --family owsm \ --model /path/to/OWSM-GGUF/owsm-v4-base-102m-f32.gguf --backend cpu --threads 8 \ --audio input.wav --request-option language=eng \ --request-option return_timestamps=true --segments-out segments.json \ --text-out transcript.txt --log ``` For speech translation, add `--request-option target_language=deu`, for example, to translate into German. Long recordings use timestamp-driven windows. Use `--request-option language=auto` to identify the source language from the first window without a second encoder pass. Greedy decoding is the CPU-first default. Set `--request-option beam_size=5` for the official Python default beam width; this trades additional computation and host memory for a wider search. Add `--mode streaming` for incremental text output from a complete audio file. Greedy decoding emits token deltas; beam search and automatic long-form continuation emit committed window text. This is not live microphone streaming. For HTTP, configure a streaming model entry and use `stream=true` on `/v1/audio/transcriptions`. See the [model guide](https://github.com/0xShug0/audio.cpp/blob/main/docs/models/owsm.md) for options and conversion instructions. The published model weights are licensed under CC BY 4.0. Attribution belongs to the upstream ESPnet OWSM authors. The unused CTC head is omitted from these autoregressive inference packages. Packages include F32 and Q8_0 for Base, Small, and Medium, plus Q4_K for Small and Medium. Base Q4_K was evaluated but is not distributed for the size and speed reasons above.