New: curated reference voices and duration comparisons. Compare shorter timing and free timing with the same words, voices and seeds.
Four questions to listen for
Does it clone both voices?
Compare S1 and S2 input recordings with their spoken turns in the output. Reference identity cosine measures the intended speaker match, without swapping the labels to improve the score.
Can it invent distinct hosts?
No-reference dialogues ask for two or three voices. Listen for distinct people and correct switches. Three voices are exploratory: podcast training focused on two.
Do voices remain consistent?
Longer dialogues request 40 or 56 seconds. Early–late cosine compares aligned turns from the same intended speaker. Also compare within-speaker and between-speaker similarity.
Does single-speaker acting survive?
Forty emotions use exactly the same neutral sentence within each language, with two seeds and no audio reference. Only the acting instruction changes.
Evaluation is complete; inspect the measurements and matched examples below.
What the current measurements show
Reference cloning: 1200 takes scored; 1200 have at least one aligned turn for identity comparison. Mean intended-speaker identity cosine is 0.837 and timbre cosine is 0.824. Mean turn-alignment coverage is 0.998. Missing turns remain visible; one correctly cloned voice alone is not evidence that both voices worked.
2 invented voices: 17 / 400 scored outputs were diarized as exactly the requested number of speakers. Mean within-minus-between-speaker identity cosine is 0.014. A larger positive separation supports distinct, recurring voices. Three-speaker results are exploratory.
3 invented voices: 1 / 400 scored outputs were diarized as exactly the requested number of speakers. Mean within-minus-between-speaker identity cosine is -0.019. A larger positive separation supports distinct, recurring voices. Three-speaker results are exploratory.
Long-dialogue consistency: 999 / 1000 scored long takes have repeated, aligned speaker turns. Mean early-to-late identity cosine is 0.758; mean alignment coverage is 0.997. Listen for identity changes alongside this diagnostic.
Single-speaker acting: 400 takes scored across the fixed neutral-sentence emotion test. Mean WER is 1.8%, Content Enjoyment is 4.70 and Genuineness is 4.11. Per-take target-emotion scores and their training-normalized z values are preserved in the catalogue.
Results at a glance
CE and Genuineness are automatic Humaneness EARS Medium predictions. Cosine similarities are diagnostic embedding comparisons, not percentages of correct identity. WER is based on Parakeet Redux. Each table reports its actual scored count; missing alignments are retained in the downloadable records.
Reference Dialogue
| Language | Scored takes | CE ↑ | Genuine ↑ | Reference cosine ↑ | Early–late cosine ↑ | WER ↓ | Duration error ↓ |
|---|---|---|---|---|---|---|---|
| English | 240 | 5.295 | 3.547 | 0.823 | 0.770 | 5.9% | 0.13 s |
| German | 240 | 5.221 | 3.793 | 0.856 | 0.818 | 4.6% | 0.20 s |
| French | 240 | 5.108 | 4.011 | 0.821 | 0.761 | 7.2% | 0.31 s |
| Spanish | 240 | 5.178 | 4.233 | 0.827 | 0.774 | 4.8% | 0.08 s |
| Dutch | 240 | 5.266 | 3.521 | 0.858 | 0.812 | 7.1% | 0.18 s |
Invented Dialogue
| Language | Scored takes | CE ↑ | Genuine ↑ | Reference cosine ↑ | Early–late cosine ↑ | WER ↓ | Duration error ↓ |
|---|---|---|---|---|---|---|---|
| English | 160 | 5.239 | 3.716 | pending | 0.723 | 2.5% | 0.07 s |
| German | 160 | 5.223 | 3.919 | pending | 0.791 | 4.4% | 0.08 s |
| French | 160 | 5.150 | 3.951 | pending | 0.715 | 3.4% | 0.05 s |
| Spanish | 160 | 5.225 | 3.948 | pending | 0.740 | 3.2% | 0.08 s |
| Dutch | 160 | 5.205 | 3.752 | pending | 0.737 | 5.0% | 0.05 s |
Single Emotion
| Language | Scored takes | CE ↑ | Genuine ↑ | Reference cosine ↑ | Early–late cosine ↑ | WER ↓ | Duration error ↓ |
|---|---|---|---|---|---|---|---|
| English | 80 | 4.710 | 4.048 | pending | pending | 3.1% | 0.06 s |
| German | 80 | 4.691 | 4.205 | pending | pending | 2.0% | 0.05 s |
| French | 80 | 4.606 | 3.998 | pending | pending | 1.4% | 0.08 s |
| Spanish | 80 | 4.764 | 4.147 | pending | pending | 0.6% | 0.06 s |
| Dutch | 80 | 4.734 | 4.152 | pending | pending | 1.9% | 0.05 s |
Does “genuine, realistic, authentic” in GENERAL help?
1000 fully matched pairs. Every pair shares spoken words, timing, seed, references, speaker routing and CFG. Mean genuine-minus-blank changes: CE -0.014, Genuineness 0.044, WER 0.001. Extra words in GENERAL are the intervention, rather than a different audio selection.
Machine-readable aggregate results · Audio, exact prompts and reproducibility data
Listen and compare
INPUT players are supplied reference recordings; MODEL OUTPUT is one jointly generated audio sequence. Turns were not generated separately and stitched together. Starting any player immediately pauses other players, including those in another tab of this demo.
What was trained?
The starting model is Humaneness Voice Large 1.7, a supervised acting model in the MOSS semantic-backbone / local audio-transformer family. This new full fine-tune trains the entire model for two epochs, with 80% real-podcast and 20% original-format acting replay by presentation count. It finished 9,874 updates on 32 GPUs. Semantic peak learning rate: 2e-5; local audio transformer: 4e-5; 5% warm-up followed by cosine decay. The final checkpoint had the lowest measured teacher-forced development loss, 4.4653 versus 4.5745 before training. That alone is not a listening-quality result.
Training conditions two episode-local voices, S1 and S2, using separate clean identity anchors and up to 60 seconds total of anchors plus earlier conversation. The five-language source corpus contains approximately 1,789 hours after quality selection; it is not a claim of 500 hours in every language. GENERAL supplies global delivery directions and SCRIPT contains speaker labels, acting directions and requested durations for each turn.
References in this demo
The newly supplied English and German NotebookLM episodes are transcribed with Parakeet Redux and diarized with Nemotron-3-Diarization. Clean word-aligned reference excerpts are selected, and a short edited chronological history includes two or three snippets per speaker when available. The montage is encoded once after waveform assembly. Other identity sets come from high-quality, held-out real-podcast examples with distinct source shows. Core reference players are codec reconstructions, not untouched source recordings. Reference conditioning alone does not guarantee that expression, identity and timing transfer correctly.
Timing and CFG
Requested durations are controls, not millisecond guarantees: the codec advances in 80-ms frames. We report actual duration, frame-cap stops and aligned-turn timing error. CFG 2.5 compares conditional and neutral predictions at every acoustic channel: neutral logits + 2.5 × (conditional − neutral). Both branches keep the same words, speaker identities, timing and audio history. The conditional stop decision is unchanged. Temperature is 1, top-p .95, top-k 50, with two fixed seeds per condition.
How to interpret the scores
Generated turns are located by matching transcribed words to the script, with at least half the words matched before speaker embeddings are compared. Low-coverage turns are recorded as missing, not silently assigned a speaker. Diarization independently estimates the number of active voices. Without reference recordings, within-speaker similarity and between-speaker separation test consistency, but they cannot establish a true human identity. These are automatic diagnostics; use the listening examples to judge the artistic result.