University of Washington researchers have created a tool that will turn audio clips into realistic video with mouth movement.
The new algorithm announced Tuesday creates a lip-synced version of a person speaking the words recorded as an audio file.
Researchers created a sample video using former President Barack Obama that shows a strikingly similar mouth movement to the way he actually presented information.
Video example from UW:
“These type of results have never been shown before,” Ira Kemelmacher-Shlizerman, an assistant professor at the UW’s Paul G. Allen School of Computer Science & Engineering, said in a news release. “Realistic audio-to-video conversion has practical applications like improving video conferencing for meetings, as well as futuristic ones such as being able to hold a conversation with a historical figure in virtual reality by creating visuals just from audio.”
UW said the new system could possibly end video chats that time out from poor connections, as audio streaming over the Internet uses less bandwidth than video.
The research team trained a "neural network to watch videos of an individual and translate different audio sounds into basic mouth shapes."
This process was combined with old research from the UW Graphics and Image Laboratory team to create the new mouth synthesis technology.
Technology companies like Samsung, Google, Facebook and Intel helped fund the research.