PyVideoTrans: Open-Source Video Translation & Dubbing Tool

PyVideoTrans is a comprehensive tool designed for automated video translation and dubbing. It streamlines the process of taking a video in one language and producing a version in another, complete with synchronized subtitles and localized voiceovers.

Speech Recognition

The software integrates several high-performance speech recognition models, including faster-whisper, openai-whisper, Google Speech, and Alibaba’s Chinese-specific model (zh_recogn).

Text Translation

Users can choose from a wide array of translation services to suit their needs: Microsoft Translate, Google Translate, Baidu Translate, Tencent Translate, ChatGPT, Azure AI, Gemini, DeepL, DeepLX, and offline translation via OTT.

Text-to-Speech (TTS)

To provide natural-sounding dubbing, PyVideoTrans supports numerous TTS engines: Microsoft Edge TTS, Google TTS, Azure AI TTS, OpenAI TTS, Elevenlabs TTS, custom TTS API servers, GPT-SoVITS, clone-voice, and ChatTTS-ui.

Other Features

• Background Music Preservation: Uses UVR5 integration to keep original background audio while replacing the speech.
• Extensive Language Support: Supports Simplified/Traditional Chinese, English, Korean, Japanese, Russian, French, German, Italian, Spanish, Portuguese, Vietnamese, Thai, Arabic, Turkish, Hungarian, Hindi, Ukrainian, Kazakh, Indonesian, Malay, and Czech.
• Full Video Translation and Dubbing: Automatically translates speech and embeds hardcoded subtitles.
• Transcription Services: Converts audio or video into text to export as SRT subtitle files.
• Batch Subtitle Dubbing: Generates voiceovers from existing SRT files, supporting both single and batch processing.
• Batch Subtitle Translation: Translates one or multiple SRT files into a target language simultaneously.
• Media Merging: Combines separate audio, video, and subtitle tracks into a single video file.
• Audio Extraction: Splits video files into standalone audio and silent video components.
• YouTube Downloader: Built-in functionality to download videos directly from YouTube.

How to Use

Pre-packaged Version (Windows 10/11 only; Mac/Linux users must use source code)

1. The package is built using PyInstaller. As it does not have a digital signature or code signing certificate, your antivirus software may flag it as a threat. You should add the application to your trusted list or run the software from the source code instead.
2. Extract the package to a directory path that contains no spaces and uses only English characters. Double-click sp.exe to launch. If you encounter permission errors, right-click the file and select "Run as administrator."
3. Do not attempt to run the application from within the ZIP file. Additionally, do not move sp.exe to a different folder after extraction.

MacOS Source Code Setup

1. Open the Terminal and execute the following commands.
2. If you do not have Homebrew installed, install it first: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
3. Once Homebrew is installed, run: eval $(brew --config) brew install libsndfile ffmpeg git [email protected]
4. Configure your environment variables: export PATH="/usr/local/opt/[email protected]/bin:$PATH" source ~/.bash_profile source ~/.zshrc
5. Create a folder (ensuring the path has no spaces or Chinese characters) and navigate to it in the Terminal.
6. Clone the repository and set up the virtual environment: git clone https://github.com/jianchang512/pyvideotrans cd pyvideotrans python -m venv venv source ./venv/bin/activate (Your command prompt should now display (venv). All subsequent commands must be executed within this environment.)
7. Install the necessary dependencies: pip install -r requirements.txt --no-deps If the installation fails, switch to the Alibaba mirror: pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ pip config set install.trusted-host mirrors.aliyun.com Then retry. If issues persist, try: pip install -r requirements.txt --ignore-installed --no-deps
8. Launch the application UI: python sp.py

Linux Source Code Setup

CentOS / RHEL

1. Install Python 3.10 and necessary tools: sudo yum update sudo yum groupinstall "Development Tools" sudo yum install openssl-devel bzip2-devel libffi-devel cd /tmp wget https://www.python.org/ftp/python/3.10.4/Python-3.10.4.tgz tar xzf Python-3.10.4.tgz cd Python-3.10.4 ./configure --enable-optimizations sudo make && sudo make install sudo alternatives --install /usr/bin/python3 python3 /usr/local/bin/python3.10 sudo yum install -y ffmpeg

Ubuntu / Debian

1. Install Python 3.10: apt update && apt upgrade -y apt install software-properties-common -y add-apt-repository ppa:deadsnakes/ppa apt update sudo apt-get install libxcb-cursor0 apt install python3.10 curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10 (Verify the pip version; it should indicate pip 23.2.1 or similar from the Python 3.10 site-packages.) sudo update-alternatives --install /usr/bin/python python /usr/local/bin/python3.10 sudo update-alternatives --config python apt-get install ffmpeg

2. Verify the installation: python3 -V should return 3.10.4.

3. Create a folder (no spaces or Chinese characters in the path) and open the Terminal in that location.

4. Run the following: git clone https://github.com/jianchang512/pyvideotrans cd pyvideotrans python -m venv venv source ./venv/bin/activate (Ensure the prompt shows (venv).)

5. Install dependencies: pip install -r requirements.txt --no-deps If this fails, use the Alibaba mirror (refer to the MacOS instructions) and retry.

6. For CUDA acceleration (NVIDIA GPU and CUDA 11.8+ required): pip uninstall -y torch torchaudio pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu118 pip install nvidia-cublas-cu11 nvidia-cudnn-cu11

7. Extract the ffmpeg.zip file into the source folder, overwriting existing files when prompted. Once finished, the ffmpeg directory must contain ffmpeg.exe, ffprobe.exe, and ytwin32.exe.

8. Launch the tool: python sp.py

Windows 10/11 Source Code Setup

1. Download Python 3.10 from the official website. During installation, ensure "Add Python to PATH" is checked.
2. Install Git for Windows from the official repository.
3. Create a folder (no spaces or Chinese characters). Type cmd in the File Explorer address bar to open a command prompt in that folder.
4. Execute the following: git clone https://github.com/jianchang512/pyvideotrans cd pyvideotrans python -m venv venv .\venv\scripts\activate (The prompt should now display (venv).)
5. Install dependencies: pip install -r requirements.txt --no-deps If the process fails, switch to the Alibaba mirror as described previously.
6. For CUDA acceleration (NVIDIA GPU + CUDA 11.8+ required): pip uninstall -y torch torchaudio pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu118
7. Extract ffmpeg.zip into the source folder. Overwrite files if prompted. Ensure the ffmpeg folder contains ffmpeg.exe, ffprobe.exe, and ytwin32.exe.
8. Launch the application: python sp.py

Source Code Notes

The default ctranslate2 version is 4.x, which requires CUDA 12.x. If your hardware only supports an older version of CUDA and you cannot upgrade, run the following: pip uninstall -y ctranslate2 pip install ctranslate2==3.24.0

If you encounter a "module not found" error, open requirements.txt, locate the missing module, and remove the == and the version number. Save the file and reinstall the dependencies.