# TF-GridNet GGUF

Use with [audio.cpp](https://github.com/0xShug0/audio.cpp) for offline speech
separation. This package contains the ESPnet TF-GridNet WSJ0-2mix checkpoint,
its configuration, STFT window, and audio.cpp model spec in one GGUF.

## Source

- [Original checkpoint archive](https://drive.google.com/file/d/1TasZxZSnbSPsk_Wf7ZDhBAigS6zN8G9G/view): `98epoch.pth` and its matching `config.yaml`.
- [Official ESPnet example](https://kan-bayashi.github.io/espnet/notebook/SpeechEnhancement_CMU_11492_692_Spring2023%28Assignment7%29.html#model-selection).
- [TF-GridNet paper](https://arxiv.org/abs/2211.12433).

The checkpoint has 8,175,874 parameters stored as F32. The F32 conversion verifies
all 541 packaged tensors byte-for-byte, including the generated STFT window.
Only F32 is provided. The 16-bit candidates did not improve speed or peak VRAM
enough to justify their long-form speaker-track assignment differences.
It produces two speaker tracks at 8 kHz. The runtime supports other configured
speaker counts, but that requires a checkpoint trained for that count.

**License:** the downloaded checkpoint archive does not state a weight license.
ESPnet's source-code license does not establish the checkpoint's license.
Confirm permission with the checkpoint authors before redistribution or use.

## Usage

### Continuous Dialogue

A CUDA comparison used a continuous 167.333-second two-speaker conversation
generated by VibeVoice 1.5B. RTX 5090, eight threads, Debug build, logging enabled,
default separation settings, no precision overrides. Each session ran a short
warmup, the conversation, then the same short request. Peak VRAM was sampled
per process every 20 ms. Python was not rerun for this comparison.

| C++ package | Warm time | RTF | Peak VRAM | File size |
| --- | ---: | ---: | ---: | ---: |
| F32 | 24.614 s | 0.14710 | 850 MiB | 31.27 MiB |

The short request repeated after long-form inference was byte-identical,
so this test found no request-state pollution.

### Command

```bash
audiocpp_cli --task sep --family tf_gridnet \
  --model /path/to/tf-gridnet-wsj0-2mix-f32.gguf \
  --backend cuda --audio mixture.wav --out-dir separated --log
```

The outputs are `speaker_1.wav` and `speaker_2.wav`. These labels are output
lanes, not identified people. The default uses 2.4-second windows with 1.6-second
overlap, matches speakers across overlaps, and normalizes each final track to
a peak of 0.9. Long silent gaps can make speaker matching ambiguous.

Use `--request-option audio_chunk_duration_sec=0` for whole-file inference.
Memory then grows with recording length. See the
[model documentation](https://github.com/0xShug0/audio.cpp/blob/main/docs/models/tf_gridnet.md)
for options and conversion instructions.

## Validation

On the official 11.49-second example mixture, C++ CUDA versus ESPnet's chunked
pipeline produced waveform cosine similarities of **0.99999991 / 0.99999998**.
Whole-file inference was also checked on CPU and Vulkan against Python, with
both speaker tracks above **0.9999999** cosine similarity.

The 327.6-second chunked CUDA comparison produced **0.99999841 / 0.99999778**.
Both tracks had exactly the same length as Python and only finite samples.
The short C++ request before and after the long request was byte-identical for
both speakers. These are numerical output comparisons, not claims about speech
separation quality on every recording.