Nvidia introduces AI to generate video conference call heads from 2D images
Nvidia AI researchers introduced AI to generate speaking heads for video conferencing from a single 2D image that can be used to achieve a wide range of manipulations, from rotating and moving a person's head to motion transmission and video reconstruction. The AI takes the first image in a video as a 2D photo and then uses an unsupervised learning method to capture 3D key points in a video. Not only does the AI outperform other approaches when testing benchmark data sets, but it also achieves H.264 quality video with a tenth of the bandwidth previously required.
Nvidia scientists Ting-Chun Wang, Arun Mallya and Ming-Yu Liu published one Paper on the Monday model in the preprint repository arXiv. The results show that the latest AI model does better vid2vid, a couple of GAN recordings described in an article published last year on NeurIPS, lead authored by Wang and Liu.
“By just changing the keypoint transformation, we can generate free view video. By transferring only the key point transformations, we can achieve much better compression ratios than existing methods, ”the article says. "We believe this is an important step towards the future of video conferencing by drastically reducing bandwidth and ensuring a richer experience."
The release of the model Maxine's debut follows in October, an Nvidia video conferencing service. Maxine not only offers virtual backgrounds like zoom, but also subtle features with AI support like face alignment and noise reduction with less noticeable features like an AI avatar for conversations or live translation.
Video calls for Microsoft Teams and Zoom also use AI shapes to do things like blurry backgrounds and power augmented reality animations and effects. An article about the release of Nvidia AI was published the day before Salesforce acquired Slack for $ 27 billion, News that could shake up the corporate communications landscape and fuel the feud between Microsoft Teams and Slack. Microsoft also introduced one Update on the teams' calling experience today.
Nvidia is one of the most famous companies in the world that is being worked on generative adversarial models (GANs) like StyleGan who have the ability to distort reality and blur the lines between real and false. Such AI models have potential uses for entertainment and gaming, but also for disinformation or creating fake accounts. True, there was a lot of worry – thank God not fulfilled – GANs have stepped into the picture about the possibility of deepfakes, which accelerate misinformation in the run-up to the US presidential election in November. In one case, Russian state actors used fake profile pictures this fall generated with GANs as part of an effort to create a fake news agency manned by actual Russian writers to promote propaganda. In another incident in 2019, AI-generated images were used to create a profile for Katie Jones, a fake person with an AI-generated photo targeting political influencers and think tank researchers in Washington DC.