NVIDIA’s artificial intelligence supported Audio2Face technology is a game changer in localization with facial animation and lip sync. Audio2Face is a big step forward in reaching a game’s dialogues and presentation to a global audience. While localizing games is a complex process that requires accurately translating dialogue and text, preserving dramatic or comedic tones, and then re-recording dialogue with voice actors from different countries, NVIDIA technology will make this job easier.
For example, Ghost of Tsushima, which will come to the PC platform and has English and Japanese languages, limits full lip synchronization to two language options due to the animation it contains. This is where generative AI, specifically NVIDIA’s Audio2Face technology, will come into play. Audio2Face will allow the creation of more lip syncs and facial Animations in more languages with artificial intelligence.
The first game to feature this technology will be World of Jade Dynasty, a martial arts MMO from Chinese studio Perfect World Games. With NVIDIA Audio2Face, the studio can produce accurate lip sync and animation in Chinese and English. As an online title, NVIDIA Audio2Face will allow developers to add new audio content with the same realistic animation.
The second game to showcase the technology is RealityArts Studio and Toplitz Productions’ Unreal Engine 5-powered Unawake, a new action-adventure game that will be released with DLSS 3 support later this year. This video offers a dynamic look at the operation of NVIDIA Audio2Face, showcasing a variety of sliders covering a variety of emotions.
