Tencent has released Hy-SimulMT, a 30-billion-parameter mixture-of-experts model that translates live speech between Chinese and English in either direction.[1][5] The downloadable weights and inference code are licensed under Apache 2.0, and the repository also includes separate training tools.[1][2][3] A meeting or interview can therefore be translated without sending its audio to a commercial translation API, provided the entire inference stack runs locally.[2]
There is an unusually home-friendly option: Tencent published a Q4_K_M GGUF measuring 19.75GB, or 18.39GiB.[4] Subtracting that file size from a nominal 24GiB of VRAM leaves about 5.61GiB before the runtime, context cache and working buffers take their share. The file fits. Running a dependable live translator on an RTX 3090 or 4090 is another question.
The immediate catch is software. Tencent’s deployment guide says Hy-SimulMT support has not yet been merged into official llama.cpp and recommends waiting for a compatible release.[2] Hugging Face’s generic “use this model” panel already displays llama.cpp and Ollama commands, but the model’s own documentation is the one to trust here.[1][2]
What is actually available
Hy-SimulMT listens to an incoming audio stream and begins translating before the speaker finishes.[1] Tencent offers translation-only weights and an “Interleave” version that returns both the source text and translation.[1] Its SimulInfer front end accepts microphone audio, uploaded files or a WebSocket stream.[2]
The GGUF repository contains two complete model files.[4] The 4-bit Q4_K_M file is 19,746,809,728 bytes.[4] The Q8_0 file is 33,312,171,808 bytes, or 31.02GiB; subtracting that from 48GiB leaves nearly 17GiB for overhead, but the file itself is already too large for a 24GB card.[4] HighVRAM fetched the first 8MiB of the Q4 download and confirmed that it is a GGUF v3 file with audio-encoder metadata and 1,170 tensor descriptors. We did not download the full model or run a translation test.
Hugging Face reports 30,396,330,496 total parameters, a 262,144-token context limit and Tencent’s `hy_v3` architecture.[5] Multiplying that parameter count by two bytes for 16-bit weights gives 56.62GiB before runtime overhead; one byte per parameter gives 28.31GiB at 8-bit, while half a byte gives 14.15GiB at 4-bit.[5] The actual Q4 file is about 30% larger than the simple four-bit calculation, so parameter count alone is not a safe download or VRAM estimate.[4][5]
Treat a 24GB Nvidia card as an experimental target for Q4, not a promise. A 32GB card has a healthier margin. Q8 belongs on a 48GB GPU or a large unified-memory machine.[4] Tencent has not published independent, reproducible throughput or VRAM measurements for those GGUF configurations, so a tokens-per-second estimate would be theater.
The working route is still an enthusiast setup
Tencent’s supported launch path uses Linux, an Nvidia GPU, CUDA, Python and vLLM 0.31 or newer with `HYV3ForCausalLM` support.[2] SimulInfer can live on a separate CPU machine, while the model server does the heavy work.[2] The reference command serves the unquantized model in bfloat16, whose raw weights alone are about 56.62GiB.[2][5] Plan on an 80GB-class accelerator or multiple GPUs for that route unless a tested lower-precision vLLM setup says otherwise.
The working route includes a local web page, audio uploads, microphone streaming and an API.[2] If both the front end and model server stay on your network, the source audio does too. Remote microphone access requires HTTPS, which is a welcome bit of friction rather than a defect.[2]
Speech output and voice cloning are separate.[1] Hy-SimulMT generates the translation, but Tencent’s recipe adds a Qwen3-TTS 0.6B service through vLLM-Omni to speak it in a cloned version of the input voice.[2] That needs another model environment and more memory. A user who only wants translated text can skip it.
Tencent’s numbers need an outside test
Tencent says Hy-SimulMT reached about three seconds of StreamLAAL lag on its RealSI long-form evaluation, the lowest among the systems in its chart, while its GEMBA quality score beat the compared open models and four of five commercial APIs.[1] StreamLAAL is a streaming-lag measure, not time to first spoken word.[1]
Those are company results. HighVRAM found no independent review during this reporting pass. Tencent’s own human comparison was small: 18 recordings of roughly six minutes each and six professional interpreters.[1] The company compared Hy-SimulMT with the best score among three interpreters at each certification level, then warned that the test was not a comprehensive assessment of speech quality or voice similarity.[1]
The model is also narrow by design. It handles Chinese and English, not a broad language menu, and an early translation can require self-correction as more of the sentence arrives.[1] That trade is normal for simultaneous interpreting; whether Tencent has managed it gracefully is exactly what a real-world test must establish.
Hy-SimulMT matters because it turns a cloud-style live translation task into something a home workstation may soon run privately. But “may soon” is doing work. If you already maintain a compatible vLLM server with roughly 80GB of accelerator memory, or a tested multi-GPU equivalent, the released weights and Apache license make this worth trying on low-risk audio.[2][3][5] Everyone else should wait for the official llama.cpp merge, then look for measured latency, quality and VRAM use on the Q4 file before committing a 20GB download.