vfx Using AI to bypass the Uncanny Valley

So I had talked about this technique at times in the past, but since we haven't gotten to this phase yet, there was no example video. Anyway, a few people had the same idea I had, and made a few test clips, so now I can show at least a crude example of what I've been talking about. All those times I talked about using AI to make the cg characters more human looking, this is the exact process I was talking about.

So this is adding an AI stylization compute to the output of a metahuman in UE5. It's how we can take generic CG characters and make them feel much more human in the very near future. We'll run another stylization AI over the finished character and background to unify elements within the composite.

An interesting spinoff concept here is that I can use the AI layer to hybrid existing movie stars, so if I want a character that's half Brad Pitt, and half Gerard Butler, I can just dial that in.

About 8 seconds in you can see the metahuman base model, for comparison to the final result of the hybrid output.

 
The technology is amazing although meta Tom Cruise looks a little like he suffering from the onset of Parkinson's.

You know, I too am fascinated and amazed by simulated humans. I truly appreciate the challenges that have to be overcome to make something artificial appear to be as real as any person walking down the street.. This is no small feat considering that we all have a lifetime of data to compare the artificial person to, as apposed to, let's say, dinosaurs. With non-human characters there is leeway. Our brains don't know exactly how a dinosaur looks or acts so when we are presented with one on the screen, we really have nothing to compare it to, thus we are more likely to buy into the reality of it. simulated humans don't get off that easy. They are completely believable or they are not. Period.

The Tom Cruise metahuman in the video using AI is pretty good and could certainly pass if not shown on screen too long, but it raises issues outside of the artistic. copyright issues. Tom Cruise owns the likeness rights to himself just as I own these rights to myself. Where is the line that separated infringement and coincidence? "No, your honor. I did not copy Tom Cruise. The digital actor that I modeled just happens to look like him." .. or forget about coincidence, what about intentional comparison? You make an action movie and purposely hire a lead actor that looks like Tom Cruise. The actor has every right to look like Tom Cruise. He was born that way. Could Tom Cruise sue the production company because he believes you purposely used that actor to draw upon his celebrity? I don't now. I've never heard of such legal issues, but let's say it doesn't happen. Actors don't sue production companies who use other actors who look like them. An actor can not be penalized for looking like someone else. To not hire an actor because he looks like Tom Cruise would probably draw it's own law suit... So complicated, but the point is this; if Tom Cruise probably won't sue a production company that has a lead actor that looks like him since the actor is just a person who happens to look like him, then would he sue a production company that employs a digital actor that looks amazingly like him?

Yes, I appreciate the ability to make digital humans that can fool an audience. It sends a chill up my spine when I see an example of a truly exceptional metahuman performance, but each time I look at these creations or look into the progress that artists and programmers are making in their quest to make the perfect metahuman, a question always comes to my mind. It's inescapable. I see these characters and the people making them and I can not help but ask "why?"

There are over 7.7 billion people on the planet to choose from, so why do we want to simulate them? Is it simply the challenge? Could it be that this is just another example of society imploding? We spend our days conversing with people on the internet, people we will probably never know or meet. We text instead of dialing the phone and talking. We are told to be afraid of everyone we don't know. Little by little it seems like physical interaction between people is losing ground to the matrix, oh, I mean the Internet... Will up coming film makers find the idea of using digital actors and a library of mocap data more appealing than dealing with real humans?.... and this will blow your mind; since metahumans will be able to look better and move in cool and appealing ways that are not typical of natural humans, will people start to modify their own behavior and looks to more closely resemble the behavior and appearance of metapeople?


Maybe I'm just showing my age but I don't think metahumans is a good thing. I don't think it's a bad thing either. Instead, I think it's a sign of how society is changing. Whether or not that is good or bad is irreverent. Change will always happen. That is the one certainty in life.
 
Indietalk! I never heard of Deep Fake before. This is so cool. You can use this technology to do digital make-up too.

I've got a lot to catch up on but first, I think this is probably different than digital people. From what I was able to figure out in the past few minutes is that Deep Fake is a way of mapping other faces over an actor. You still use actors. In my post (above) I was commenting on the way mocap and the technology that Nate North wrote about is essentially doing away with on-camera actors in favor of mocap data, and sophisticated CGI puppets.
 
I'd say the tech is getting close. But that example is a little blurry, I suppose the softness helps it look real but it looks more like a transfer job from an old mini-DV tape. Something also I don't like about the neck, and lines of neck. But yeah, it's getting close.
 
Deep fake has been around forever though, what is the difference here?
There is no difference in the technology, other than it's evolving over time to be more and more accurate. What's different is the speed, cost, and implementation friction of the metahuman actors. Hybriding this technology with deepfake AI can have a major effect on how much budget, both money and time, it takes to achieve result x. Deepfake by itself was good for making memes and joke videos, but it wasn't very practical for serious work. There are multiple levels of reasons why this will work better, in example, the source and target of the AI can be aligned much more perfectly, generating a superior result image. Obviously once there is a way to make a CG character pass for human, or at least look good enough that it no longer degrades suspension of disbelief in a story, the opportunities are somewhat limitless. Keep in mind that this demo video I showed was made by some 20 year old kid in his spare time. These results can be improved with effort.

What I'm trying to achieve is really more about budget and time than it is about pure quality. This is more an effort to democratize the art of filmmaking, so it's accesible to the 99.99% of the population that simply can't afford to make a film people will watch. I read an indienews article the other day where they referred to a "sliver of indie filmmakers working without a multimillion dollar budget" If the guy had been standing next to me, I would have punched him in the face. There is a film school in the next town over, where hundreds of excited, would be filmmakers take thousands of hours of classes, make thousands of practice films, take out loans they can't afford to buy equipment they cant afford to actually use. In the town where that film college is, only two people have ever been able to make a film. Two sons of the richest lawyers in town, who had never studied film, inherited millions of dollars as a reward for drinking at bars, and all the people that worked in film their whole lives sat in the audience as rich people who never worked a day in film paraded their accomplishment in front of them. To my knowledge, none of the people who lived and breathed film their whole lives ever got a single opportunity.

All these steps I'm taking, including this research branch with the AI layer metahumans, are ultimately aimed at empowering actual creative people to bring their vision to the screen. Who is the best filmmaker, or the only filmmaker, should not be determined by a rich old man picking out his nephew. It should be determined by the audience, and with traditional film, almost none of us get even one chance in our lives to bring a vision to an audience without being hobbled out of the gate by underfunding.

I'd like to see a world where a person with a creative vision to share could share it, without being preselected by someone from a higher financial caste. These hacks, like I'm demonstrating here, are a path forward for people who don't want to spend their entire creative lives begging to be noticed by someone who could care less about creativity. Those are the people with money. As long as millions of dollars is a prerequisite for turning one's vision into something marketable, that's where we're at.

I did see the movie made by the lawyers two sons. It starred Jessie Ventura, and it was basically a political drama about how all the filthy poor people were leeches that were stealing money from the brave inheritance class. There were a lot of tearful pride moments where god and patriotism mentioned, and the enemies were people that worked 60 hours a week trying to destroy America by asking for a dollar over minimum wage.

I doubt that any of this tech will ever equal "Casino" or "Goodfellas" on screen, but what it could turn into, in the near future, is a path where hardworking creative people could tell their stories, share their visions with an audience, without having to be pre cleared by someone with the "right" last name.

Sorry for the rant, I'm just permanently pissed off about a system where every filmmaker needs "permission" from a non filmmaker to even attempt to create art. We need to work on making good film affordable to produce. Maybe people think I enjoy telling young filmmakers that they are never going to make it, but I don't. In fact I hate it, and I want the reality to change so I can tell them "you're free to take an idea you're excited about, and share it with the world" So that's all about removing one financial obstacle at a time. Actors, obviously, are a great place to start.
 
Last edited:
The technology is amazing although meta Tom Cruise looks a little like he suffering from the onset of Parkinson's.

You know, I too am fascinated and amazed by simulated humans. I truly appreciate the challenges that have to be overcome to make something artificial appear to be as real as any person walking down the street.. This is no small feat considering that we all have a lifetime of data to compare the artificial person to, as apposed to, let's say, dinosaurs. With non-human characters there is leeway. Our brains don't know exactly how a dinosaur looks or acts so when we are presented with one on the screen, we really have nothing to compare it to, thus we are more likely to buy into the reality of it. simulated humans don't get off that easy. They are completely believable or they are not. Period.

The Tom Cruise metahuman in the video using AI is pretty good and could certainly pass if not shown on screen too long, but it raises issues outside of the artistic. copyright issues. Tom Cruise owns the likeness rights to himself just as I own these rights to myself. Where is the line that separated infringement and coincidence? "No, your honor. I did not copy Tom Cruise. The digital actor that I modeled just happens to look like him." .. or forget about coincidence, what about intentional comparison? You make an action movie and purposely hire a lead actor that looks like Tom Cruise. The actor has every right to look like Tom Cruise. He was born that way. Could Tom Cruise sue the production company because he believes you purposely used that actor to draw upon his celebrity? I don't now. I've never heard of such legal issues, but let's say it doesn't happen. Actors don't sue production companies who use other actors who look like them. An actor can not be penalized for looking like someone else. To not hire an actor because he looks like Tom Cruise would probably draw it's own law suit... So complicated, but the point is this; if Tom Cruise probably won't sue a production company that has a lead actor that looks like him since the actor is just a person who happens to look like him, then would he sue a production company that employs a digital actor that looks amazingly like him?

Yes, I appreciate the ability to make digital humans that can fool an audience. It sends a chill up my spine when I see an example of a truly exceptional metahuman performance, but each time I look at these creations or look into the progress that artists and programmers are making in their quest to make the perfect metahuman, a question always comes to my mind. It's inescapable. I see these characters and the people making them and I can not help but ask "why?"

There are over 7.7 billion people on the planet to choose from, so why do we want to simulate them? Is it simply the challenge? Could it be that this is just another example of society imploding? We spend our days conversing with people on the internet, people we will probably never know or meet. We text instead of dialing the phone and talking. We are told to be afraid of everyone we don't know. Little by little it seems like physical interaction between people is losing ground to the matrix, oh, I mean the Internet... Will up coming film makers find the idea of using digital actors and a library of mocap data more appealing than dealing with real humans?.... and this will blow your mind; since metahumans will be able to look better and move in cool and appealing ways that are not typical of natural humans, will people start to modify their own behavior and looks to more closely resemble the behavior and appearance of metapeople?


Maybe I'm just showing my age but I don't think metahumans is a good thing. I don't think it's a bad thing either. Instead, I think it's a sign of how society is changing. Whether or not that is good or bad is irreverent. Change will always happen. That is the one certainty in life.
Ok, it felt like that degenerated into a more general rant about society, but coming from me, that's really the pot calling the kettle black. lol.

About celebrity, and "name brand actors" that's a significant topic by itself, and deeply misunderstood by most indie filmmakers. In very short form, the issue is that celebrities bring credibility, expectation, default familiarity, and more to a film, and the idea that one can be replaced by an unknown with no consequence to the final product is way off. People get optimistic and think that since the story is so important, that they can just blow past this blockade, but in practice, it doesn't work, outside of children's movies, religious fare, and teen horror.

That being said, there are likeness rights issues legally, and I won't be using any celebrity lookalikes for that reason and others. Still, when introducing a new actor, there are some important factors that we can accomplish with this tech. Creating a face with strong features, with character, is possible. It's much more possible with this than it is scouting your local community theater for the next Deniro. I can take multiple faces, and hybrid them, so likeness infringement is not a big issue. I watched that video about 10 times, really letting it sink in, gauging my own response to it. And it's very clear to me that a LOT of the dissonance I feel when looking at the metahumans went away when the AI layer was applied. Not all of it, but I was maybe 80% more comfortable looking at the fake Tom Cruise than I was looking at the raw metahuman.

Does the output above look as good as a real human actor? No. But it does look much more believable than CG actors of the past. People's minds kind of calcify over time, and while people our age will likely never accept these characters fully, people who grow up in a world where they are prevalent will likely see them differently.

You say there are 7.7 million people to choose from, but only 1 in a thousand can even do the job right, and then they have their hand out, and can't work on weekends, and the guy you found lives in another state, and every time 2 people need to be filmed talking about a vending machine it's 2 plane tickets and 2 hotel rooms and 2 ubers, etc. If you get paid pariah wages, like under 20 an hour, you are basically completely screwed from day one, trying to do even the simplest thing. Have you watched a lot of indie films, they are all terrible, including mine. It's not that we are bad at filmmaking, it's just expensive, and we are all forced to make sacrifices that end up degrading and corrupting our vision. Then investors see that bad film, think it's what we intended to make, take it as a representation of our level of talent, and then underfund us into the grave. People don't understand vicious cycles. I say that because of the stupid blank looks I get when I ask "instead of giving him a million dollars and me nothing, and then declaring your guy the winner when his film is better, why don't you give us both half a million, let us compete fairly, and let the audience decide the winner?" It's like they are too dumb to even grasp that they are fixing every fight. I'm not even 100% convinced that George Lucas could beat me in a fair fight, but we'll never know, because it's literally never a fair fight. Always their guy in a F 15 fighting me on a bicycle.

In an interesting corollary, if you ever see a European soccer game, they set it up so every team has equal advantages, and most of the matches end up in a tie after 3 hours, or leading by one point. That's the true picture of humans competing, when you don't try to rig the game every time.

As far as humans becoming more disconnected from each other, that's a tsunami I'm not capable of stopping. I don't really like it, but I am powerless to stop it, so I'm focusing on areas where I think I can make a difference.
 
I like the way you think, Nate and I applaud your efforts. You have a great vision and I think it is completely doable, but how are you going to do it? Will you provide the tools to the film maker along with some sort of a manual or training video. Will they have to buy the Unreal suite of software. Will they have to audition for you before they are accepted into the network of artists producing segments for your ever evolving storyline? I'm very curious.

I agree that freedom of speech and expression should not be limited to only the ones who can afford to speak and express. My own son was a child prodigy on the piano. We discovered this when he was 8 years old but we could only afford to buy him a piece of crap Samick baby grand. So frustrating. If it wasn't for his professor pulling some strings with a local music shop to sell us a Boston baby grand at cost, he never would have been able to reach his full potential... Yes, the quality of the piano makes a tremendous difference.

Film making is not a god given right. It is unfair that only people with money get invited into the club. There are a few exceptions, like Sam Rami but, over all, it's who you know and not what you know. Some middle class people try to make movies using only their own resources and I admire them for having the courage to try.. I know one man who didn't want to make feature films but he did have a great desire to make short films. A series of short films. This man had no money and almost no resources, so what did he do? He used himself as all the actors. There was no explanation as to why all the characters look like him. That's just the way it was. For a green screen he used a green blanket hanging on the wall of his bedroom The CGI he used was the equivalent of animatics or "flying logo" animation with the most basic rendering possible, but guess what? He did it and his series isn't that bad. It has a quality to it that could not have been predicted. That Man is Steve Sutton and his series is The Portal.


Sorry, Nate. I wasn't trying to hijack your thread..
 
Last edited:
Last thing first, first thing last. I like this Steve guy. While the production here is bad, I can clearly see someone who really wants to make a film, trying as hard as they can against overwhelming adversity, and just blowing past every stop sign on the road with a brick on the gas pedal. I feel like I'd get along with this guy, lol.

People don't really understand how Save Point works, and it's not because their stupid, it's just genuinely complicated, and takes some time and experience to fully appreciate.

I'll answer your questions as simply as I can, one at a time.

We do provide all the neccesary software tools and resources, along with training and instruction. The issue so far has been finding people that can sit down and study, work, focus. The Bottleneck is on the human end. Save Point has collected almost 200k worth of resources to date, via a variety of methods, and anyone who joins has full access to all of it, similar to a library system, and legal under the corporate umbrella. It's the same concept as a studio building an old west town set. If filmmakers are going to be coming through making westerns for the next 30 years, there is really no need to keep buying and building that same town over and over. The parent company buys it, and any employee or intern that comes through has free use of any corporate asset. Joining is free, and there is never any charge for anything. It's a 0 money system on the creator end, and on the audience end.

Unreal engine is free for all, and in fact, this video was what originally inspired me to make this system. The first time I watched it, I thought, this feels right, this feels like finally I'm seeing a company act like a human being.


Lastly, yes there is kind of an audition process. It's not so fixed as that sounds though. I would take a person with a strong desire to learn over a skilled person that was lazy, distracted, or had a bad attitude any day. Even the best of us need to be open to learning and improving every day, so I look for those people. Anyone who can match quality with our norms is welcome to come in and work on as much or as little as they like, and is given full credit publicly for anything they contribute.

Also I don't feel like you're hijacking my thread. I don't feel like any thread "belongs" to me, and more to the point, I'm on here to have conversations with people, so any responses are always welcome.

We have hit a roadblock in one area though, where for some reason, basically everyone that tries to join is without a graphics card, which is unfortunately a prerequisite of the new pipeline. These cards are shipping in record numbers, backordered for months, priced at double list even at wall mart, because so many people have ordered them. Yet somehow, 45 people have joined, and not one of them had a graphics card that could even handle one scene. I was shocked when 43 people in a row showed up with instruments. Almost all are gone now, because they just wanted me to put their song in a film, but 50 new tracks were being produced for every one minute of film I could put out with no help. I can't afford to send people 1200 dollar graphics cards, so this has been an issue. So far, I have produced every bit of film you've seen and far more solo. I keep telling people that I'll give them access to hundreds of thousands of dollars worth of 3d models, environments, courses, and even spend hours a week tutoring them free of charge, but so far they just shrug and start playing "smoke on the water" and talking about how that's a more productive use of time.

In case the many posts I wrote about it were before your time here, just FYI, you are free to join, and just like anyone else, I'll supply you with resources, help you personally with the new software, etc. I don't mean some webpage, I mean I'll get on the phone and walk you through it step by step until you are getting the results I'm getting. When I started doing this, I thought people would appreciate the rarity of that, and a few have, but to be honest, it's been a bit underwhelming so far. It was help I really needed when I was trying to lean animation, and literally no one could be bothered to spend even an hour a week teaching me, working along side me, much less supply me with assets.

As far as who gets cut from the program, I only cut people who aren't serious about becoming a filmmaker. People show up and want to chat for a couple of days and get their name on a big product, but those are not my people. I work hard, I care a lot about this artform, and I'm looking to help others that feel the same, and are struggling like me to find a way forward.
 
Last edited:
I hear ya, brother...

I've been on my own journey for long time. I love the journey and I hope to some day get there. Have you ever read the Alchemist by Paulo Coelho? In the story, there is a shop keeper who happens to be a Muslim. Every year he plans on making a pilgrimage to Meca, something most Muslims aspire to do sometime in their life. Well, the shop keeper makes plans every single year to make this journey but he ends up never going. He doesn't consider it a waste of time; planning, because it gives him something to look forward to. A reason to get up every morning...
 
Last edited:
Back
Top