-
Notifications
You must be signed in to change notification settings - Fork 734
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avatar embedding #441
Avatar embedding #441
Conversation
* Fix the avatar size in home page * Update home page style to support mobile device * Add mobile support for most of the page * Remove 'add character' when small screen * Finish hamburger menu and update page layout * Fix minor layout issues
client/web/src/utils/audioUtils.js
Outdated
bs.connect(audioSourceNodeRef.current); | ||
setupAvatarLipSync(audioContextRef.current, bs); | ||
handleFirstInteractionAudio(); // For the first interaction, we need to play a sound to unlock blend shapes | ||
// bs.connect(audioSourceNodeRef.current); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do u still need this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wait, need it. We thought it would affect the audio to be double audio. However, I just realised that it won't work on other no Avatar Characters.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will take a quick fix on it!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All right, fixed it
* Add local Llama2 support from llama2-wrapper backend (#400) * add llama2-wrapper as local backend * update local llm backend * update local llm backend * update * Update __init__.py * Update __init__.py --------- Co-authored-by: Piaoyang Cui <bcstyle@gmail.com> * Fix style issues and refine code (#425) * Minor fix * fix build * fix style issues and refine code * Use consistent name style * Add API_HOST to react-web (#426) * Update style to fit tablet screens (#427) * Update README.md (minor typo) 😅 (#429) * Add a Render deployment guide (#431) * Add a Render deployment guide * Update render_deploy.md * Lint * Format * Lei/use zustand (#428) * update page logic * Apply zustand, fix minor bugs * Solve the scroll issue * Upload zustand files * minor fix * update cli to support next-web (#432) * Update .gitignore (#433) * Update .gitignore (#436) * Reduce VAD latency. (#430) * Lei/mobile next web (#437) * Fix the avatar size in home page * Update home page style to support mobile device * Add mobile support for most of the page * Remove 'add character' when small screen * Lei/mobile next web (#439) * Fix the avatar size in home page * Update home page style to support mobile device * Add mobile support for most of the page * Remove 'add character' when small screen * Finish hamburger menu and update page layout * Fix minor layout issues * Add ion (#442) * Avatar embedding (#441) * fix: update audio * feat: avatar generation embedding * chore: move embedding to top * Lei/mobile next web (#439) * Fix the avatar size in home page * Update home page style to support mobile device * Add mobile support for most of the page * Remove 'add character' when small screen * Finish hamburger menu and update page layout * Fix minor layout issues * fix: no audio in other character --------- Co-authored-by: Lei Qiu <amethystlei@gmail.com> * Add info loggers showing latencies of STT, LLM, TTS processes (#445) * deployment working except for voice cloning * update README: new issue about tts doesn't speak due to bad llm response * deployment successful; essential features all function * update README * prepare to merge with main * Add info loggers showing latencies of STT, LLM, TTS processes * update .gitignore * untrack reset_databash.sh * update README * Add info loggers showing latencies of STT, LLM, TTS processes * Add more latency monitors specific for the APIs * Refactor the timers into decorators; Report latencies together * Add terms of service page (#453) --------- Co-authored-by: Tom <plain1994@gmail.com> Co-authored-by: Piaoyang Cui <bcstyle@gmail.com> Co-authored-by: Lei Qiu <amethystlei@gmail.com> Co-authored-by: Devansh <mdevansh28@gmail.com> Co-authored-by: Fangbai Chai <139947087+hksfang@users.noreply.github.com> Co-authored-by: Edwin Wong <73209427+HongSiu@users.noreply.github.com> Co-authored-by: Yi Guo <guoyi0328@gmail.com>
* Add local Llama2 support from llama2-wrapper backend (#400) * add llama2-wrapper as local backend * update local llm backend * update local llm backend * update * Update __init__.py * Update __init__.py --------- Co-authored-by: Piaoyang Cui <bcstyle@gmail.com> * Fix style issues and refine code (#425) * Minor fix * fix build * fix style issues and refine code * Use consistent name style * Add API_HOST to react-web (#426) * Update style to fit tablet screens (#427) * Update README.md (minor typo) 😅 (#429) * Add a Render deployment guide (#431) * Add a Render deployment guide * Update render_deploy.md * Lint * Format * Lei/use zustand (#428) * update page logic * Apply zustand, fix minor bugs * Solve the scroll issue * Upload zustand files * minor fix * update cli to support next-web (#432) * Update .gitignore (#433) * Update .gitignore (#436) * Reduce VAD latency. (#430) * Lei/mobile next web (#437) * Fix the avatar size in home page * Update home page style to support mobile device * Add mobile support for most of the page * Remove 'add character' when small screen * Lei/mobile next web (#439) * Fix the avatar size in home page * Update home page style to support mobile device * Add mobile support for most of the page * Remove 'add character' when small screen * Finish hamburger menu and update page layout * Fix minor layout issues * Add ion (#442) * Avatar embedding (#441) * fix: update audio * feat: avatar generation embedding * chore: move embedding to top * Lei/mobile next web (#439) * Fix the avatar size in home page * Update home page style to support mobile device * Add mobile support for most of the page * Remove 'add character' when small screen * Finish hamburger menu and update page layout * Fix minor layout issues * fix: no audio in other character --------- Co-authored-by: Lei Qiu <amethystlei@gmail.com> * Add info loggers showing latencies of STT, LLM, TTS processes (#445) * deployment working except for voice cloning * update README: new issue about tts doesn't speak due to bad llm response * deployment successful; essential features all function * update README * prepare to merge with main * Add info loggers showing latencies of STT, LLM, TTS processes * update .gitignore * untrack reset_databash.sh * update README * Add info loggers showing latencies of STT, LLM, TTS processes * Add more latency monitors specific for the APIs * Refactor the timers into decorators; Report latencies together * Add terms of service page (#453) * Implement next-web functionalities. * fix small issue recorderSlice.js (#455) --------- Co-authored-by: Tom <plain1994@gmail.com> Co-authored-by: Piaoyang Cui <bcstyle@gmail.com> Co-authored-by: Lei Qiu <amethystlei@gmail.com> Co-authored-by: Devansh <mdevansh28@gmail.com> Co-authored-by: Fangbai Chai <139947087+hksfang@users.noreply.github.com> Co-authored-by: Edwin Wong <73209427+HongSiu@users.noreply.github.com> Co-authored-by: Yi Guo <guoyi0328@gmail.com> Co-authored-by: Fangbai Chai <fangbaichai@gmail.com>
Latest Update
Showcase
Generation2.mp4