---
language:
- en
license: cc-by-nc-4.0
library_name: k2
pipeline_tag: automatic-speech-recognition
datasets:
- esb/datasets
metrics:
- wer
tags:
- automatic-speech-recognition
- speech
- audio
- zipformer
- transducer
- cr-ctc
- k2-fsa
- offline-asr
---

# Zipformer CR-CTC Transducer XL 290M

Offline English ASR model based on the Icefall/K2 pruned transducer Zipformer-CR-CTC
recipe. The model was trained with a transducer objective and CR-CTC
(Consistency-Regularized CTC) auxiliary loss, and supports Icefall transducer
and CTC decoding.

## Files

- `model.pt`: Zipformer CR-CTC transducer checkpoint
- `bpe.model`: SentencePiece BPE model
- `tokens.txt`: Icefall token table exported from `bpe.model`
- `config.yaml`: model architecture, feature extraction, tokenizer, Zipformer
  model, decoding settings, and Hugging Face Hub download-metrics query file

## Evaluation

Open ASR Leaderboard English short-form result, decoded with
`modified_beam_search` and beam size 6:

| Metric | Value |
| --- | ---: |
| Average WER | 5.56 |
| Parameters | 288M |

Dataset WERs:

| Dataset | WER |
| --- | ---: |
| AMI | 11.30 |
| Earnings22 | 7.64 |
| GigaSpeech | 8.41 |
| LibriSpeech Clean | 1.31 |
| LibriSpeech Other | 3.01 |
| SPGISpeech | 1.64 |
| VoxPopuli | 5.60 |
| **Average** | **5.56** |

## Training Data

The model was trained on a combined English training mixture built from the
training portions of the datasets below.

| Dataset | Train Hours | Source |
| --- | ---: | --- |
| LibriSpeech | 960.0 | ESB datasets |
| Earnings-22 | 105.0 | ESB datasets |
| AMI Meeting Corpus | 78.0 | ESB datasets |
| Common Voice Scripted Speech 25.0 - English | ~1,679.0 | Mozilla Data Collective |
| Common Voice Spontaneous Speech 3.0 - English | ~3.6 | Mozilla Data Collective |
| GigaSpeech XL | 10,000.0 | ESB datasets |
| SPGISpeech | 4,900.0 | ESB datasets |
| TED-LIUM Release 3 | 454.0 | ESB datasets |
| VoxPopuli | 523.0 | ESB datasets |
| Total | ~18,700 | ESB datasets and Mozilla Data Collective |

Common Voice Scripted Speech 25.0 - English and Common Voice Spontaneous Speech
3.0 - English are the English scripted and spontaneous Common Voice releases
checked on May 12, 2026.

For data normalization, the training transcripts were processed with direct LLM
normalization using a self-hosted GLM-4.7 setup, plus custom agentic workflows
built in collaboration with Claude Code 4.7 and Codex 5.4.

## Training

The model was trained with the k2-fsa/Icefall framework, PyTorch 2.11, and
CUDA 13.0 on 8 NVIDIA B300 GPUs using bf16 automatic mixed precision.

## Usage With Icefall

From an Icefall checkout, download this model repo locally:

```bash
huggingface-cli download soundsgoodai/Zipformer-cr-ctc-transducer-XL-290M \
  --local-dir Zipformer-cr-ctc-transducer-XL-290M
```

Run offline decoding through Icefall. Input audio must already be 16 kHz:

```bash
cd icefall/egs/librispeech/ASR

PYTHONPATH=../../.. python zipformer/pretrained.py \
  --checkpoint /path/to/Zipformer-cr-ctc-transducer-XL-290M/model.pt \
  --tokens /path/to/Zipformer-cr-ctc-transducer-XL-290M/tokens.txt \
  --method modified_beam_search \
  --beam-size 6 \
  --num-encoder-layers "2,2,4,5,4,2" \
  --downsampling-factor "1,2,4,8,4,2" \
  --feedforward-dim "512,1024,2048,3072,2048,1024" \
  --num-heads "4,4,4,8,4,4" \
  --encoder-dim "192,384,768,1024,768,384" \
  --encoder-unmasked-dim "192,192,320,384,320,192" \
  --query-head-dim "32" \
  --value-head-dim "12" \
  --pos-head-dim "4" \
  --pos-dim 48 \
  --cnn-module-kernel "31,31,15,15,15,31" \
  --decoder-dim 512 \
  --joiner-dim 512 \
  --context-size 2 \
  --causal false \
  --chunk-size "16,32,64,-1" \
  --left-context-frames "64,128,256,-1" \
  --use-transducer true \
  --use-ctc true \
  --use-attention-decoder false \
  --use-cr-ctc true \
  /path/to/audio_1.wav \
  /path/to/audio_2.wav
```

For CTC decoding, use Icefall's CTC entrypoint with the same architecture
settings:

```bash
cd icefall/egs/librispeech/ASR

PYTHONPATH=../../.. python zipformer/pretrained_ctc.py \
  --checkpoint /path/to/Zipformer-cr-ctc-transducer-XL-290M/model.pt \
  --tokens /path/to/Zipformer-cr-ctc-transducer-XL-290M/tokens.txt \
  --method ctc-decoding \
  --sample-rate 16000 \
  --num-encoder-layers "2,2,4,5,4,2" \
  --downsampling-factor "1,2,4,8,4,2" \
  --feedforward-dim "512,1024,2048,3072,2048,1024" \
  --num-heads "4,4,4,8,4,4" \
  --encoder-dim "192,384,768,1024,768,384" \
  --encoder-unmasked-dim "192,192,320,384,320,192" \
  --query-head-dim "32" \
  --value-head-dim "12" \
  --pos-head-dim "4" \
  --pos-dim 48 \
  --cnn-module-kernel "31,31,15,15,15,31" \
  --decoder-dim 512 \
  --joiner-dim 512 \
  --context-size 2 \
  --causal false \
  --chunk-size "16,32,64,-1" \
  --left-context-frames "64,128,256,-1" \
  --use-transducer true \
  --use-ctc true \
  --use-attention-decoder false \
  --use-cr-ctc true \
  /path/to/audio_1.wav \
  /path/to/audio_2.wav
```

The architecture and decoding values above are also recorded in `config.yaml`.

## Decoding Methods

The model supports the following Icefall decoding methods:

Transducer:

- `greedy_search`
- `modified_beam_search`
- `fast_beam_search`

CTC:

- `ctc-decoding`

The reported Open ASR Leaderboard result uses `modified_beam_search` with beam
size 6. CTC decoding is supported, but the table above does not report CTC WERs.

## Feature Extraction And Resampling

For this model, use Kaldi-style fbank features. `kaldifeat` and
`kaldi-native-fbank` are the recommended feature extraction backends.

Audio should be mono 16 kHz before feature extraction. For sample-rate
conversion, `audioop.ratecv` is recommended because it matches the resampling
path used for evaluation. On Python 3.13 and newer, use an `audioop`-compatible
package such as `audioop-lts`.

## Output Formatting

The model emits normalized English text with punctuation and capitalization. It
does not automatically capitalize the first word of every sentence unless that
word is normally capitalized, such as a proper noun, honorific, acronym, or
similar named expression.

The model also normalizes common written forms, including numbers, dates, and
currency, to digit-based forms.

## Open ASR Leaderboard Evaluation

The `open_asr_leaderboard/soundsgoodai` runner downloads this model repo,
resamples audio to 16 kHz with `audioop`, computes fbank features with
`kaldi-native-fbank`, and decodes with Icefall `modified_beam_search`.

```bash
cd open_asr_leaderboard/soundsgoodai
bash run_zipformer.sh
```

## Notes

- The model expects 16 kHz mono audio.
- The packaged leaderboard decoding configuration uses transducer
  `modified_beam_search` with beam size 6.
- The checkpoint is an offline ASR model and is not intended for streaming
  decoding.
- The checkpoint includes both transducer and CR-CTC heads, so Icefall loading
  should use `--use-ctc true` and `--use-cr-ctc true`.

## References

- Zipformer paper: Zengwei Yao, Liyong Guo, Xiaoyu Yang, Wei Kang, Fangjun
  Kuang, Yifan Yang, Zengrui Jin, Long Lin, and Daniel Povey. "Zipformer: A
  faster and better encoder for automatic speech recognition."
  https://arxiv.org/abs/2310.11230
- CR-CTC paper: Zengwei Yao, Wei Kang, Xiaoyu Yang, Fangjun Kuang, Liyong Guo,
  Han Zhu, Zengrui Jin, Zhaoqing Li, Long Lin, and Daniel Povey.
  "CR-CTC: Consistency regularization on CTC for improved speech recognition."
  https://arxiv.org/abs/2410.05101
- Icefall: https://github.com/k2-fsa/icefall
- k2: https://github.com/k2-fsa/k2
- kaldifeat: https://csukuangfj.github.io/kaldifeat/intro.html
- kaldi-native-fbank: https://github.com/csukuangfj/kaldi-native-fbank
- Python `audioop.ratecv`: https://docs.python.org/3.11/library/audioop.html#audioop.ratecv
- audioop-lts for Python 3.13+: https://pypi.org/project/audioop-lts/
- ESB datasets: https://huggingface.co/datasets/esb/datasets
- LibriSpeech: https://www.openslr.org/12/
- Earnings22: https://github.com/revdotcom/speech-datasets/tree/main/earnings22
- AMI Meeting Corpus: https://www.idiap.ch/webarchives/sites/www.amiproject.org/ami-scientific-portal/meeting-corpus/
- Common Voice Scripted Speech 25.0 - English: https://mozilladatacollective.com/datasets/cmndapwry02jnmh07dyo46mot
- Common Voice Spontaneous Speech 3.0 - English: https://datacollective.mozillafoundation.org/datasets/cmn1pv5hi00uto1072y1074y7
- GigaSpeech: https://github.com/SpeechColab/GigaSpeech
- SPGISpeech: https://huggingface.co/datasets/kensho/spgispeech
- TED-LIUM Release 3: https://lium.univ-lemans.fr/en/ted-lium3/
- VoxPopuli: https://github.com/facebookresearch/voxpopuli