Yu Zhang, Wenxiang Guo, Changhao Pan, Dongyu Yao, Zhiyuan Zhu, Ziyue Jiang, Yuhan Wang, Tao Jin, Zhou Zhao
TCSinger 2 models multilingual singing with style transfer and control from audio and text prompts. This repository provides training and inference code; prepare your trained checkpoints and the auxiliary models before generation.
Start with the model assets and input format, then follow Quick start or the training guide. To cite this work, use the BibTeX entry.
- 2025.07 · Released TCSinger 2 code.
- 2025.07 · Released STARS code.
- 2025.05 · Accepted to ACL 2025 Findings.
| Implementation | Requirements |
|---|---|
| Included code | Training, preprocessing, and singing inference |
| Model checkpoint | Your trained TCSinger 2 model, VAE, and matching encoders |
| Output | 48 kHz singing waveform |
| Runtime | Python 3.10, PyTorch 2.1.0 + CUDA 12.1 |
git clone https://github.com/AaronZ345/TCSinger2.git
cd TCSinger2
conda create -n tcsinger2 python=3.10 -y
conda activate tcsinger2
python -m pip install -r requirements.txt \
--find-links https://download.pytorch.org/whl/torch_stable.htmlThe training guide covers GTSinger metadata, phoneme sets, F0 extraction, mel preprocessing, VAE training, the Custom Audio Encoder, and main-model training. Update the paths in your configuration and preprocessing scripts for your data.
For inference, prepare the trained checkpoint, matching model configuration, and processed manifest. The example script reads per-item mel, reference-prompt mel, F0, phoneme, score, and style fields; the manifest must point to existing files. For speech inputs, use the corresponding VAE setup.
PYTHONPATH=. CUDA_VISIBLE_DEVICES=0 python scripts/test_sing.py \
--config configs/tcsinger2.yaml --ckpt /path/to/your/model.ckpt \
--manifest_path data/sing/new.tsv --save_dir infer_outReplace the checkpoint path with your trained model. Use --scales to configure classifier-free guidance. Outputs and their clap.csv index are written under --save_dir.
| Asset | Preparation |
|---|---|
| VAE and duration predictor | Train with configs/ae_singing.yaml |
| Main singing model | Train with configs/tcsinger2.yaml |
| HIFI-GAN | Download to useful_ckpts/hifigan |
| FLAN-T5 | Hugging Face to useful_ckpts/flan-t5-large |
| Custom Audio Encoder | Follow the contrastive-data and VAE instructions in the training guide |
The training metadata includes ph, word, item_name, ph_durs, wav_fn, singer, ep_pitches, ep_notedurs, ep_types, emotion, singing_method, and technique. Keep the phoneme vocabulary, alignment, F0 files, and checkpoint configuration consistent.
Describe singer by gender and vocal range. Represent technique as a list of skills or text describing their sequence. The training guide explains conversion to the model's manifest format.
Inference handles every eligible row (duration up to 20 seconds), including single-row manifests and repeated names. Use --sample-limit 10 --sample-seed 0 for a reproducible subset; this seed controls row selection. Multiple GPUs receive disjoint rows without duplication. The combined clap.csv is ordered by manifest_index after all ranks complete.
TCSinger 2 uses a Blurred Boundary Content Encoder, a contrastive Custom Audio Encoder, and a Flow-based Custom Transformer with Cus-MOE and F0 conditioning. See the paper for comparisons across style transfer, style control, cross-lingual transfer, and speech-to-singing transfer.
Figure 1. TCSinger 2 architecture, from score and prompt encoding to audio generation.
This implementation uses parts of the code from the following Github repos: Make-An-Audio-3, TCSinger Lumina-T2X as described in our code.
If you find this code useful in your research, please cite our work:
@article{zhang2025tcsinger,
title={TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis},
author={Zhang, Yu and Guo, Wenxiang and Pan, Changhao and Yao, Dongyu and Zhu, Zhiyuan and Jiang, Ziyue and Wang, Yuhan and Jin, Tao and Zhao, Zhou},
journal={arXiv preprint arXiv:2505.14910},
year={2025}
}Any organization or individual is prohibited from using any technology mentioned in this paper to generate someone's singing without his/her consent, including but not limited to government leaders, political figures, and celebrities. If you do not comply with this item, you could be in violation of copyright laws.