---
license: cc-by-4.0
tags:
- audio.cpp
- automatic-speech-recognition
- speech-translation
- gguf
---

# OWSM-CTC GGUF

For [audio.cpp](https://github.com/0xShug0/audio.cpp).

Upstream: [espnet/owsm_ctc_v4_1B](https://huggingface.co/espnet/owsm_ctc_v4_1B), revision `5f526a73d919afd1bf51ab425909c8ab18801670`.

`owsm-ctc-v4-1b-f32.gguf` and `owsm-ctc-v4-1b-q8_0.gguf` use a
self-conditioned E-Branchformer encoder with a CTC head instead of the
autoregressive decoder. Both transcribe and translate with the separate `owsm_ctc`
family. CTC uses best-path decoding, without beam search or predicted timestamps.
For long recordings it uses 30-second windows, discards two seconds of context
on each side, then collapses repeated frame tokens across window boundaries.

### CTC Parity

F32 matched official Python exactly on the five-request short sequence on CPU,
CUDA, and Vulkan: two English recordings, English-to-German translation,
unknown-language conditioning, and a repeated first request. The 327.6-second
recording's full 910-word transcript also matched exactly on CPU and CUDA.
The short request following the long recording remained unchanged.
These are transcript comparisons, not a guarantee of identical internal tensors
or a multilingual accuracy benchmark.

Q8_0 is not exact-parity. On the long recording it had one normalized-word
edit on CPU and CUDA (0.11%) relative to Python F32. These measure
disagreement with Python, not error against a human transcript. The short German
translation also changed, and Q8_0 outputs differed slightly between backends.
Use F32 when reference fidelity matters; quantization does not improve the
upstream model's translation quality.

### Warmed Performance

Hardware: AMD Ryzen 9 7900X, NVIDIA GeForce RTX 5090, eight CPU threads.
Software: audio.cpp debug build, ESPnet 202610.post2, PyTorch 2.11.0+cu128.
C++ CPU measurements use a full-model CPU-only build with CUDA and Vulkan
disabled, native CPU instructions, and OpenMP. A paired F32 check reduced peak
RSS by about 282 MiB versus the GPU-enabled binary. GPU measurements still use
the separate GPU-enabled build.

One 3.505-second warmup precedes
one 327.6-second request, with English best-path decoding, 30-second windows,
two-second left/right context, and one window per batch. No TF32 or precision
overrides are used. Python uses `best_path` for the padded warmup and
`decode_long` for the long request. Timing excludes loading, warmup, and file
reading; C++ uses `session.wall_ms`, and Python uses an external timer with CUDA
synchronization. Memory is sampled across loading and inference using process
RSS and NVML process GPU memory: every 20 ms for CPU-only C++, and every 100 ms
for Python and the GPU runs. The numbers below are performance
measurements, separate from the focused output checks above.

| Backend / implementation | Warmed inference | RTF | Peak RAM or VRAM |
| --- | ---: | ---: | ---: |
| CPU Python F32 | 29.586 s | 0.09031 | 8,729 MiB RAM |
| CPU C++ F32 | 33.527 s | 0.10234 | 5,630 MiB RAM |
| CPU C++ Q8_0 | 30.262 s | 0.09238 | 3,050 MiB RAM |
| CUDA Python F32 | 1.082 s | 0.00330 | 10,728 MiB VRAM |
| CUDA C++ F32 | 0.627 s | 0.00191 | 6,080 MiB VRAM |
| CUDA C++ Q8_0 | 0.650 s | 0.00198 | 3,508 MiB VRAM |

**C++ CTC is still slower than Python F32 on CPU in this test:** about 13% for
F32 and 2% for Q8_0. 

```bash
audiocpp_cli --task asr --family owsm_ctc \
  --model /path/to/OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf \
  --backend cuda --threads 8 --audio input.wav \
  --request-option language=eng --text-out transcript.txt --log
```

`language=auto` uses unknown-language
conditioning in each window. `target_language=deu`, for example, selects German
translation. Add `--mode streaming` to emit committed text after each window
from a complete audio file. This produces the same transcript as offline mode;
it is not live microphone streaming. In server mode, use a streaming model
entry and `stream=true` on `/v1/audio/transcriptions`.