-
🍖 Fully automated video translation, generating Netflix-quality subtitles!
-
🎤 Clone your own voice for dubbing! (🚧 Still in development)
-
✨ Click-and-done in Streamlit!
Check out the results! 💪
en_demo.mp4 |
ru_demo.mp4 |
-
Subtitle segmentation using NLP and LLM
-
Intelligent terminology knowledge base for context-aware translation
-
Three-step translation process: Direct Translation - Reflection - Improvement
-
Precise word-level subtitle alignment
-
Create 5 minutes of Netflix-quality bilingual subtitles for just 1 yuan
-
Developer-friendly: Step-by-step structured files for easy customization: Chinese Technical Documentation | English Technical Documentation
You can even run each
step__.pyfile undercoreindividually!
We are continuously improving VideoLingo, but there are still some limitations:
-
Audio Length: Currently only supports videos up to 30 minutes, we plan to extend this limit soon.
-
Input Language Support (WhisperX's output of timestamps and punctuation is unstable for some languages):
| Input Language | Support Level | Example Video |
|---|---|---|
| English | 🤩 | English to Chinese demo |
| Russian | 😊 | Russian to Chinese demo |
| French | 🤩 | French to Japanese demo |
| German | ❓ (Not tested yet) | |
| Spanish | ❓ (Not tested yet) | |
| Japanese | 😖 | ❌ |
| Chinese | 😖 | ❌ |
- Output Language Support: VideoLingo supports translation into all languages that Claude can handle
Thanks to the following open-source projects for their contributions:
- whisper: OpenAI's open-source automatic speech recognition system
- whisper-timestamped: Extension adding timestamp functionality to Whisper
- whisperX: Extension adding timestamp functionality to Whisper
- yt-dlp: Command-line tool for downloading YouTube videos and content from other websites
- GPT-SoVITS & GPT-SoVITS-Inference: Speech synthesis system and inference library based on GPT and SoVITS
- FFmpeg: Complete, cross-platform solution for handling multimedia content
- Ultimate Vocal Remover GUI v5 (UVR5): Tool for separating vocals and instrumentals in music
- json_repair: Super powerful library for repairing and parsing GPT's JSON output, seamlessly replacing json.loads
We welcome all forms of contributions. If you have any ideas or suggestions, please feel free to raise an issue or submit a pull request.
For further communication or assistance, welcome to join our QQ group.