AI Converts Text to Video in Seconds

Text-to-image AIs are born and "make waves"

On March 20, Runway, a New York-based AI company, introduced its Gen-2 tool, which can generate videos from still images or text input. For example, if you type in “a cat walking in the rain,” the AI ​​will generate a three-second video with the requested content.

Runway, founded in 2018, is a well-known AI startup like OpenAI and has 45 full-time employees. The company is providing a website that helps users edit videos quickly, such as removing backgrounds or adding effects. They were part of the team that developed the first version of Stable Diffusion - an AI that converts text to images, which was later expanded by Stability AI.

In February 2019, Runway announced a video editing model called Gen-1. The tool focused on converting existing video footage, importing raw 3D animations, or adding AI-generated filters. Gen-2 is still based on the Gen-1 architecture, but focuses more on creating videos using text, input images, or a combination of both.

AI-generated video depicts “a person walking through a jungle.” Photo: Runway

A demo posted on Twitter by Runway co-founder and CEO Cris Valenzuela showed that Gen-2 could produce “drone footage over the desert sky” or “close-ups of human eyes” in seconds. However, the resolution and realism were limited, and there was no sound.

“You can make videos of thrilling car chases, but sometimes the cars can fly away,” Valenzuela said of the Gen-2.

Valenzuela claims that the more detailed the input, the more vivid the content Gen-2 generates, similar to AI that creates images from text like Dall-E or Stable Diffusion. In the future, the tool will be improved to help artists, designers, and filmmakers turn ideas into videos faster and at lower cost.

Professor Hany Farid at the University of California, Berkeley, called the Gen-2 video "super awesome." He predicted that higher quality video from this technology is just a matter of time.

However, he worries that Gen-2 could become a technology that is easily abused and could be more dangerous than deepfake. For example, bad guys could create fake videos to spread fake news, scams, etc.

According to The Verge, Runway is opening a new race for general AI models, where creativity is limitless, regardless of whether the content is text, images or videos. However, this technology site is also concerned that they will be exploited in sophisticated content that makes users immediately trust, regardless of whether it is true or not.

Gen-2 is currently limited access, requiring users to sign up and wait. According to Runway spokesperson Kelsey Rondenet, users will be given broader access in the coming weeks.

According to VnExpress