OpenAI SORA: Creating Video from Text

Announced today. Examples: https://openai.com/sora#research

The press has been ramping up coverage of this over the last few hours. Marques Brownlee posted a video about the examples a short while ago:

Sora:
https://openai.com/sora

Sora:
https://openai.com/sora

Just to avoid confusion, that's the same page as the link in the first post, except this link starts at the top of the page.

It's lovely but it's not here.

For once, this would actually seem to be a real new frontier -- not for image making, or for Hollywood, but for no-budget filmmaking (assuming, of course, the technology is freely available). The "uncanny valley", falling somewhere between live action and animation, would become another dramatic convention, the material sustaining a dreamy remove from both animation and live action movies. Until that effect is worked to death, it may actually serve as an enhancement.

These movies would still require "performance", as in dialogue delivery, but there are ways around that as well, for filmmakers (if they still are "filmmakers") without access to professional actors.

In a word, this is filmmaking for writers. Which would be a revolution in American independent film, where literary sensibilities are (trying to be kind) somewhat rudimentary.

These movies would still require "performance", as in dialogue delivery, but there are ways around that as well, for filmmakers (if they still are "filmmakers") without access to professional actors.

Harvesting of human voice samples and their use to "create" dialogue has already become an issue.

No, I'm thinking of other more basic strategies..... No harvesting or AI science required.

Of course, the interval of promise and freedom will last about 3 hours. Festivals, overwhelmed by submissions, will have to ban these productions outright, since they can't get through the thousands of live action submissions they already have to deal with. And big budget action pictures are already 98% fake, so nothing new there.

Eventually one application will write the script and "produce" the movie, at which point....

Great post, it would be interesting to see if BMD has something up the sleeves in AI coming up soon.

Been carefully observing all this, Chat GPT is very helpful, some of the image generating tools are also interesting.

As far as video, have not seen anything that is ready yet, perhaps, animation and cartoonish stuff for the younger audiences.

Directing acting and actors and creating story that touches on the human level, it is IMO personal touch that is unique to every director and relationship between the two.

Not a great fan nor I believe that true artistic value can be created with AI, generic, probably yes.

Humans are much more complex creatures then 3d models, yes, they can be Metahuman-ized and look stunning, my take on this, bit like Tarantino on film:

Why bother going thru all this trouble to make it like a human, just work with human.

Tarantino says similar thing about the film, why shoot on digital to look like film, just shoot film.

Argument goes both ways and there is no right or wrong, just a personal take.

I was tempted to wait and ad few scenes with AI SORA in my feature film, but after looking deeper into results, decided not to.

Completely agree Darko.

I feel people who say AI's are going to replace everything are gravely underestimating the intelligence, intellect, complexity and creativity of human beings. After all without human input an AI is literally nothing. Everything an AI creates is in one way, shape or form a derivative of what humans already created.

Why bother going thru all this trouble to make it like a human, just work with human.

It's really just a matter of money. Among their samples, what do you think it would cost to build that Japanese set, and execute the crane shot? Or, the skill, investment and equipment needed even to do an animated version of it?

I don't think anyone is arguing for eliminating "human" content. But filmmaking is already 99.98% illusion. What's the difference between building a set on a sound stage and creating a credible one through AI?

Or consider classic Japanese animation, which doesn't aspire to realism. That remove from literal physical reality doesn't at all limit the scope or appeal of the material.

Why bother going thru all this trouble to make it like a human, just work with human.

It's really just a matter of money. Among their samples, what do you think it would cost to build that Japanese set, and execute the crane shot? Or, the skill, investment and equipment needed even to do an animated version of it?

I don't think anyone is arguing for eliminating "human" content. But filmmaking is already 99.98% illusion. What's the difference between building a set on a sound stage and creating a credible one through AI?

Or consider classic Japanese animation, which doesn't aspire to realism. That remove from literal physical reality doesn't at all limit the scope or appeal of the material.

I was referring to 3d models of actors, not sets, directing actors vs asking AI to direct it for you.

AI is more hype IMO. Not sure i would go to cinema and want to pay ticket to watch AI film.

Cinema cameras are safe, not going anywhere.

I am usually excited about technology, and have no issue with AI, it has its use.

Just cant see the acting performance being done to the level of the human.

I expect this will initially be useful for b-roll and stock footage, probably some pretty useful game applications as well. In the near future AI, will become more relevant upstream.

AI basically breaks down the language barrier between humans and machines. It's like Star Trek, "Computer, generate an 1850's gold mining town". Once AI becomes available in apps like Blender, and After Effects things could get a little hairy. When AI is generating meshes then it can handle the broad strokes and the smaller elements can be finesses either through further AI manipulation or by hand.

...and yes, it's kind of a bummer than computers are once again the answer. However, folks are plugged in on average of nearly 7 hours a day. That's a massive demand for content. Ultimately AI will empower better content with higher production value. Sure, some folks will suffer as part of the development, that's how the industry, or any organism evolves.

I don't see a ton of difference between using AI and an LED volume, at a certain point it's all smoke and mirrors. Is it meaningful to the story if the town was generated in 30 seconds or it took 30 guys 30 days to build?

Good Luck

I agree with Howard; this will become the "CGI" of narrative fiction. No production is going to ignore a cheap and easy way to make an otherwise complex and expensive shot. Closeups will probably use a real camera and a real actor, though.

Guys, I don't disagree, but such uses are assumed -- the industry will gladly adopt any cheaper and potentially better way of doing what they've already been doing for years with digital image making and green screen.

The new frontier, if there is one, would be outside the industry. "Real actors" is a different question..... And not as straightforward as it may seem.

I mean, sure; if we're speaking long term, the day will come when consumers will be able to generate our own movie right here with our computers, without paying for a ticket at the movie theater or a subscription to a streaming service. Enter a prompt, modify the results as you like, and you'll have something that's literally perfect, because you made it to your own tastes.

There are many declaring that they'll never appreciate AI-generated content like that, but for the rest of the world, I suspect it won't be a hard pill to swallow. Those illustrators and pixiv artists who made a living off commissions that are now displaced by AI will be happy to chime in.

But for now, I'm speaking in the short term, as I'm sure nobody really knows exactly how we'll be affected, and we can only make predictions.

A student today already asking us for access to Sora. Sorry we are not Tesla

Enter a prompt, modify the results as you like, and you'll have something that's literally perfect, because you made it to your own tastes.

OMG, tons of cat videos and 'adult' stuff…

I'm of the personal belief that at some point people will reject the AI driven content and embrace the artistry of stories created by real people. AI generated content will be worth less that the work created by real humans. In the end, the studio execs who embrace AI as a replacement for people will suffer more financial loses.

Now, AI can be useful for helping filmmakers make their content. But in the end we will still strive to prove our worth by working with a team of people to create stories on the level of our predecessors. They didn't need AI to create such fantastic works. So, in the end artists will strive to prove they don't need it either.

More so we'll use it as a tool to eliminate the monotonous tasks no one likes to do. Rotoscoping, and such is a good example.

Granted, I'm also a person who believes a moron like David Zaslav can be replaced by AI but a filmmaker like Denis Villeneuve cannot be replaced.

As long as there are audiences who notice that difference, I second you.
Unfortunately, if I look at todays politics, and the influence of 'social' media on social interactions, I'm afraid we are a minority.

I don't see why it won't be possible to generate an actor who looks and sounds just like, say, Humphrey Bogart. No shortage of samples to work from. Bogart made 75 films, and there may be plenty of him and his voice that isn't in the films.

Not quite good enough for current cinema? Change cinema to suit. Or massage games to suit. It's said that games are already bigger than cinema.

Add a talented producer, director and writer and you may have a hit on your hands, maybe several.

…, and there may be plenty of him and his voice that isn't in the films.

If somebody took care of the archives.

They could easily replace the poor CGI in Titanic with AI today.

I'm of the personal belief that at some point people will reject the AI driven content and embrace the artistry of stories created by real people. AI generated content will be worth less that the work created by real humans. In the end, the studio execs who embrace AI as a replacement for people will suffer more financial loses.

I think it's a whole lot simpler than that, for now. We're not talking, at least in the case of SORA, of producing anything other than visuals. Much of that is *already* digitally created, and even the less successful attempts (which is most of it) are generally accepted by audiences.

Whether actors could be synthesized, whether AI can write passable scripts, etc. is a whole other question, beyond SORA and what SORA could do for movie creation out of the home, as it were.

What this engine could do is make the writer predominate in movies, or movies of a certain kind, in the same way writers are (for example) predominate in the "golden age" of cable TV, which is recent. SORA potentially rids the writer of two great annoyances: the producer and the financiers. If there were more frustrated screenwriters in this thread, there would likely be more enthusiasm for this tool. Even if the result stands at a remove from live action, so what? The same is true of animation which, in other countries, is regarded as an art form, not children's entertainment or a marketing vehicle for other products.

This does of course raise uncomfortable questions about the nature of cinema -- and cinematography. But since "cinema", in that sense, is generally unattainable without money, it's less urgent at the low budget end. The choice would be between a movie which exists and a movie which never will. So there's nothing to compare.

FYI: Tyler Perry, a guy who writes checks to pay people in the cinema business, has thoughts on the topic:

https://www.hollywoodreporter.com/busin ... 235833276/

We're talking about the same AI that we all tell to "F off and that we want to talk to a person" right? I mean, I can't be the only frustrated by support bots and AI bots for voice navigation in phone services? Right? Those things are the worst. They are worthless and never help solve any issues. You ALWAYS need to "speak to a representative" because the AI is just the worst.

So, I'm not afraid of that AI replacing me and my work. However, replacing someone like a Zaslav I can see because people like that are worthless.

Well, those crappy support bots have been around for years and I'm pretty sure they have not (yet?) been replaced by any serious AI.
Wait until you have to deal with your bank or insurance company, after they chased all their employees and installed a strong AI.

FYI: Tyler Perry, a guy who writes checks to pay people in the cinema business, has thoughts on the topic:

https://www.hollywoodreporter.com/busin ... 235833276/

Thanks Peter. I don't think that the producers and the unions have a common interest in this, and nor do I think that legislatures will intervene to protect the film industry. That aside, it's an interesting article.