trưởng công an xã chạy sang can ngăn cũng bị tuyến đánh tiếp
Clone a voice from a few seconds of audio, then speak seven languages in it — at 18× realtime on a single GPU. Everything below is the model's own output. Judge it with your ears.
Six held-out speakers from the VIVOS test set — voices no model saw in training. Every system got the same reference clip and the same sentence, on the same L4 GPU. The dashed cards are the anchors: what the model was given, and what the person actually sounds like. Playing one stops the others, so the comparison stays honest.
trưởng công an xã chạy sang can ngăn cũng bị tuyến đánh tiếp
cũng thuộc loại dày đặc nhất trong hệ mặt trời
bác vẫn chưa trả tiền cho tôi đây nhé
hoặc chỉ biểu hiện bằng các rối loạn co bóp
làm hỏng các thiết bị của tên lửa
lương ba cọc ba đồng thì nuôi được ai
The speaker embedding is language-agnostic: a single Vietnamese reference clip carries into English, Chinese, French, German, Japanese and Korean — though the measured cloning fidelity outside Vietnamese is weak (see the English table below), so judge these by ear. Japanese input must be kana — the phonemizer reads kana correctly but drops kanji, so convert with a kanji→kana step first.
Xin chào, đây là giọng nói được nhân bản từ một đoạn ghi âm ngắn.
Hello, this is a cloned voice speaking English.
你好，这是一个克隆的声音。
Bonjour, ceci est une voix clonée.
Hallo, dies ist eine geklonte Stimme.
こんにちは、これはクローンされたこえです。
안녕하세요, 이것은 복제된 목소리입니다.
One NVIDIA L4, all 65 VIVOS speakers (130 utterances), identical
references and sentences for every system. RTF is full-utterance compute ÷ audio duration —
not time-to-first-audio, which is what most speed claims quote. clone% rescales SECS
between the cross-speaker floor (0.725, two different speakers) and the human ceiling (0.936).
| System | RTF ↓ | ×realtime | SECS ↑ | clone% ↑ | UTMOSv2 ↑ | WER ↓ |
|---|---|---|---|---|---|---|
| Kokoro Ninja | 0.055 | 18.2× | 0.909 | 87% | 2.623 | 3.6% |
| omnivoice-vietnamese | 0.399 | 2.5× | 0.933 | 99% | 2.714 | 1.4% |
| VieNeu v3 Turbo | 0.736 | 1.4× | 0.934 | 99% | 2.516 | 3.9% |
| Human recording | — | — | 0.936 | 100% | 2.860 | 1.6% |
| System | RTF ↓ | ×realtime | SECS ↑ | clone% ↑ | UTMOSv2 ↑ | WER ↓ |
|---|---|---|---|---|---|---|
| Kokoro Ninja | 0.047 | 21.1× | 0.706 | 9% | 2.832 | 9.1% |
| Human recording | — | — | 0.960 | 100% | 3.249 | 4.2% |
SECS is scored with WavLM-base-plus-sv rather than CAMPPlus — scoring with the encoder this model is conditioned on would flatter it. WER is PhoWhisper back-transcription, which catches wrong tones, because a wrong tone is a different Vietnamese word. UTMOSv2 is English/Japanese-trained, so on Vietnamese it's a proxy; the human row is the calibration. It is also nondeterministic — rescoring identical audio moves it by up to 0.08, so treat small MOS gaps as noise. SECS and WER are exactly reproducible.
espeak-ng's Vietnamese voice collapses phonemically distinct onsets into a single IPA token. When two different sounds arrive as the same symbol, the model cannot learn the difference — it is not a tuning problem, the information is already gone.
| Written | espeak gives | Should be |
|---|---|---|
| tr | tʃ | ɴ |
| ch | tʃ | tʃ |
| s | s | ʂ |
| x | s | s |
| r | z | ʐ |
| d / gi | z | z |
We phonemize per syllable, look at the written onset and remap the colliding phoneme to the correct retroflex — which already exists in the vocabulary, so nothing about the model has to change. The remap is guarded: it only fires when espeak actually emitted the colliding token. c/k/q → k is left alone, because that one is a genuine merger in Vietnamese.
One practical quirk: the model is trained on sentence-final punctuation and is sensitive to it — text without a terminal mark tends to clip or run on at the end. synthesize() adds one when your text lacks it. On this very benchmark, adding the missing period moved WER from 5.1% to 3.9% and predicted MOS from 2.47 to 2.66.