--- license: apache-2.0 pipeline_tag: automatic-speech-recognition language: - en tags: - audio - asr - ensemble - consensus - orze --- # Orze-ASR-3Way Orze-ASR-3Way is a **three-checkpoint composite (ensemble)** for English ASR. It is not a standalone checkpoint. It uses [`bosonai/Qwen3-ASR-1.7B-hf-orze`](https://huggingface.co/bosonai/Qwen3-ASR-1.7B-hf-orze) as the anchor and two separately published, frozen voters: [`HojoAI/Hojo-ASR-V1`](https://huggingface.co/HojoAI/Hojo-ASR-V1) and [`OpenMOSS-Team/MOSS-Transcribe-preview-2B`](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-preview-2B). The anchor transcript is retained unless both voters agree on the same aligned word edit. A substitution or deletion is accepted only when both voters agree; an insertion is accepted only when both voters produce the same insertion at the same anchor boundary. The official English normalizer is applied before word alignment. The same rule and decoding parameters are used on every dataset. ## Leaderboard eligibility This repository is a **research and reproduction artifact**, not an eligible Open ASR Leaderboard submission. Orze-ASR-3Way combines three checkpoints, and the leaderboard does not allow composite/ensemble systems. Submission [PR #206](https://github.com/huggingface/open_asr_leaderboard/pull/206) was withdrawn and closed on 2026-08-23. The system must not be assigned or presented with an official leaderboard placement or rank. The results below are self-evaluated, benchmark-fitted development measurements only. They were not verified by the leaderboard maintainers and do not establish performance on the default private scripted or private conversational datasets. Neither the Appen nor DataOcean private leaderboard datasets was accessed during this work. ### Benchmark-fitting disclosure The Open ASR public test-set scores, the public AppTek and MInDS-14 proxy scores, and other public proxy evaluations were used during candidate, decoding, and consensus-rule selection. Consequently, the numbers on this card are benchmark-fitted development results; they are not an independent estimate of generalization. The Orze anchor's fine-tuning data excluded the Open ASR public test audio and the selected AppTek and MInDS-14 proxy audio. See the anchor card for its exact training-data and checkpoint-construction disclosure. The two voters are third-party published checkpoints; their training disclosures are controlled by their respective authors. ## Immutable components | Role | Checkpoint | Revision | |---|---|---| | Anchor | `bosonai/Qwen3-ASR-1.7B-hf-orze` | `bce834cee3d106a9f0ec1ba98e96d995f13f59de` | | Voter 1 | `HojoAI/Hojo-ASR-V1` | `a22c381896ce5f4b70038982bfb888fe9969ed99` | | Voter 2 | `OpenMOSS-Team/MOSS-Transcribe-preview-2B` | `c4b3988677df13c14e79d9db59f356ed761db366` | The reproducible Docker evaluator is hosted at [`erik-at-boson/open-asr-leaderboard-orze-ensemble`](https://huggingface.co/spaces/erik-at-boson/open-asr-leaderboard-orze-ensemble). It runs the checkpoints sequentially and includes the sum of all three inference times in RTFx. ## Self-evaluated public development results These results use the evaluator and normalizer from the Open ASR Leaderboard repository at commit `d1e99b25524814332d6868a5645e568670834cfb`. | Dataset | WER (%) | |---|---:| | AMI Cleaned | 6.93 | | Earnings22 Cleaned AA (chunked) | 5.24 | | GigaSpeech Cleaned | 6.68 | | LibriSpeech test-clean | 1.02 | | LibriSpeech test-other | 2.41 | | SPGISpeech | 1.58 | | VoxPopuli Cleaned AA | 2.78 | | **Mean** | **3.81** | Aggregate component throughput is **29.66 RTFx**, calculated from the sum of all three component inference times. The evaluator also records consensus alignment overhead when it creates fresh manifests. Integrity recheck on 2026-08-23: the pinned scorer reproduced every displayed WER and the 3.81 macro mean; regenerating all seven manifests from the recorded anchor/voter manifests reproduced all **72,341** rows exactly (zero differing rows). This is a reproducibility statement, not an official verification. ## Public proxy checks used during development AppTek Call-Center Dialogues is a public CC-BY-SA-4.0 dataset and is unrelated to the leaderboard's private Appen datasets. Three deterministic 200-segment selections (seeds `8262026`, `8262027`, and `8262028`) used different calls and had zero pairwise segment overlap. They were excluded from the Orze anchor's fine-tuning data. Lower WER is better. | Selection | Anchor overall | Consensus overall | Anchor accent macro | Consensus accent macro | |---|---:|---:|---:|---:| | 1 | 10.23 | 9.88 | 11.274 | 10.786 | | 2 | 7.24 | 6.99 | 7.024 | 6.762 | | 3 | 9.46 | 9.15 | 9.216 | 8.954 | Both individual voters were worse than the anchor on all three selections. This supports the behavior of the unanimity gate on these selected public proxies; it does not establish performance on the private leaderboard families. A deterministic, balanced 300-utterance selection from a stable 20% MInDS-14 partition spanning `en-AU`, `en-GB`, and `en-US` was also used during development. This partition was excluded from anchor fine-tuning. Overall WER changed from **31.24** to **31.04**, and locale-macro WER from **31.32** to **31.13**. The consensus changed 87 normalized transcripts: 38 improved, 34 worsened, and 15 were error-neutral, for 10 fewer word errors. Hojo and MOSS individually scored 52.37 and 39.18 WER on this selection. ## Scope and limitations This is a **9.64B-parameter composite**, using evaluator-reported component counts rounded to two decimals (2.04B Orze Qwen, 5.18B Hojo, and 2.42B MOSS). It trades throughput and total storage for accuracy. Components run sequentially, so peak inference memory is the maximum of the components rather than their sum. The public results were actively used in system selection and should not be interpreted as a blind-test comparison. ## License The consensus code and all three checkpoint repositories at the pinned revisions declare Apache-2.0. The public proxy datasets retain their own licenses; no proxy audio is redistributed in this repository.