Death March preview

[Nate North]

Still working out a few minor keying issues. Lipsync has been solved, but is not included in this demo yet.

This is a small collection of clips from the in-production film "Death March", which will be the development test project for Save Point pipeline V 7.5.

Getting close to marketable quality now, for animation at least. Speed is over 100x what it would have been without the pipeline, but there's still thousands of hours of work involved to make a film, it just no longer takes 80 people.

 

[Nate North]

Here's a quick test of the moving head lip sync tech in action. I'm polishing and improving details in the process but essentially everything is working now, finally.

Some of these day one test lips aren't perfect, But the nice thing about digital is that you get as many takes as you need.

Note the movement of the actors during the lip sync process, That was the big hang up for almost a year.

 

[sfoster]

Here's a quick test of the moving head lip sync tech in action. I'm polishing and improving details in the process but essentially everything is working now, finally.

Some of these day one test lips aren't perfect, But the nice thing about digital is that you get as many takes as you need.

Note the movement of the actors during the lip sync process, That was the big hang up for almost a year.

Ahhh shoot, unfortunately, this is problematic for me.
The lips are well done, but observe this chatGPT list of priorities for directing human attention

A general prioritization based on common principles in UI/UX design, photography, and visual design might look like this:

1. Movement (Most attention-grabbing due to the human eye's sensitivity to motion)
2. Brightness and Contrast (Brightest part and high contrast elements draw immediate attention)
3. Color (Vivid colors and color contrast are powerful attractors)
4. Size (Larger elements stand out more)
5. Position (Top-left and center positions are typically more noticed in Western cultures)
6. Faces and Eyes (Human faces and the direction of gaze are naturally compelling)
7. Shapes and Lines (Unique shapes and leading lines guide the viewer's eye)
8. Text and Typography (Bold, large, or distinctive text captures attention)
9. Whitespace (Negative space helps highlight and isolate elements)
10. Repetition and Patterns (Consistent elements create visual rhythm)
11. Visual Weight (Heavier elements draw attention, but to a lesser degree compared to the above factors)

Movement really is #1, and the only thing moving in this video is the lips, my eyes are going straight to the lips.
It's almost like a fixation, I have to fight against it and consciously look at the actors eyes instead, but... their eyes are all kinda dead.

There is no nuance in their eyes, there's no life, there's no change at all basically, their eyes aren't interesting, and my focus goes right back to the moving mouth. it's like im just staring at their mouth instead of watching the film, I've never experienced this in any other medium.

is anyone else experiencing this phenomenon or is it just me?

 

[Nate North]

Ahhh shoot, unfortunately, this is problematic for me.
The lips are well done, but observe this chatGPT list of priorities for directing human attention



Movement really is #1, and the only thing moving in this video is the lips, my eyes are going straight to the lips.
It's almost like a fixation, I have to fight against it and consciously look at the actors eyes instead, but... their eyes are all kinda dead.

There is no nuance in their eyes, there's no life, there's no change at all basically, their eyes aren't interesting, and my focus goes right back to the moving mouth. it's like im just staring at their mouth instead of watching the film, I've never experienced this in any other medium.

is anyone else experiencing this phenomenon or is it just me?

I partially agree with you, but I think your perception is more valid for some of the clips than others. I agree that the fixed shots, such as the younger indian soldier, are kind of dead, less compelling, and draw attention to the mouth. In contrast, that first clip isn't bad. I'm just getting started with this new branch, and I'm already learning a lot about how to "lean into" what's working with the lipsync, which will make a big difference. Once I've got a bit of experience with what works better and worse, I can create scenes specifically meant to work well with it, unlike these scenes which were fabricated before I installed the new tech.

Dead eyes are a problem in a lot of animation, and the dark haired girl and indian guy near the end are prime examples. I do actually have some tech that helps enhance eyes specifically, but I did not use it in this reel.

While nothing is perfect yet, what I do have now is the ability to tell stories, to render scripts into animated films. The quality and finesse can be increased iteratively, now that the system is working. Anyway, time to make this movie!

As far as my eyes going straight to the mouth, I'm not having that issue, but what I do stare at constantly is every tiny rotoscope error, every little flicker of an edge. I've already spent weeks getting that more solid, but it really needs to be 100% perfect at all times.

 

[sfoster]

I partially agree with you, but I think your perception is more valid for some of the clips than others. I agree that the fixed shots, such as the younger indian soldier, are kind of dead, less compelling, and draw attention to the mouth. In contrast, that first clip isn't bad. I'm just getting started with this new branch, and I'm already learning a lot about how to "lean into" what's working with the lipsync, which will make a big difference. Once I've got a bit of experience with what works better and worse, I can create scenes specifically meant to work well with it, unlike these scenes which were fabricated before I installed the new tech.

Dead eyes are a problem in a lot of animation, and the dark haired girl and indian guy near the end are prime examples. I do actually have some tech that helps enhance eyes specifically, but I did not use it in this reel.

While nothing is perfect yet, what I do have now is the ability to tell stories, to render scripts into animated films. The quality and finesse can be increased iteratively, now that the system is working. Anyway, time to make this movie!

As far as my eyes going straight to the mouth, I'm not having that issue, but what I do stare at constantly is every tiny rotoscope error, every little flicker of an edge. I've already spent weeks getting that more solid, but it really needs to be 100% perfect at all times.

I'm rooting for you, and look forward to seeing it all put together.

I took a second look and I think once there is some movement in the upper half of the face (blinking, squinting, raising eyebrows + forehead wrinkles, crows feet emotes, etc) it will look a lot more natural to focus on. Restricting dynamic movement to only the bottom half of the face is probably whats creating this phenomenon.

 

[CelticRambler]

the only thing moving in this video is the lips, my eyes are going straight to the lips.
It's almost like a fixation, I have to fight against it and consciously look at the actors eyes instead, but... their eyes are all kinda dead.

Is this as much due to the fact that we're told the video is a "lip synch" test, anyone watching it will be primed to give more attention to the lips?

While I didn't have that same problem (and the lip synch is impressive, mostly - some of the micromovements associated with certain sounds are missing, but that's a small detail). However more of a problem for me than the dead eyes are the (somewhat) dead voices - it feels the same as a (bad) actor mechanically reading a script with very little emotion ... which I suppose is exactly what it is. Hopefully adding some imperfect dicton and irregular breathing noises is one of the future iterations we can look forward to.

 

[Nate North]

Ok, I tried to implement some of that feedback, added more voice expression and timing, added blinking, adams apple movement, some dynamic shadows, here's a look at a few days of progress on the voice+lipsync tech.

I'll continue improving it of course, but I think the added animation and improved delivery is helping. The text is just off the cuff riffing, so nothing to critique. I can do this while the character is in motion, as seen above, but didn't this time so I could focus on the facial movements and line expression.

 

[sfoster]

Definitely better!

 

[Nate North]

So this clip is just showing how I've fine tuned the art style for this project over the last few days. I feel like it's improved.

I did forget to turn on the blinking for a few of these clips, And there's not a ton of motion in this one, It's mainly just about the visual improvements. Some clips are better than others, but the final product should look similar to the best of these clips.

The lip sync animation reduces the quality a little bit but I can restore it later. Here's a few examples of actual frames before that process. In addition to the electron microscope trick I've added a small amount of detail to the eyes, and a larger amount to the shadowing that you see on faces.

Hands now have five fingers and skin has pores and wrinkles dependent on age.

 

[Rusty Bolt]

I particularly like the look you've achieved on these - more of a "graphic novel" look than hyper-realistic, which keeps you firmly away from the Uncanny Valley. It's amazing how far you've brought this!

 

[Nate North]

I particularly like the look you've achieved on these - more of a "graphic novel" look than hyper-realistic, which keeps you firmly away from the Uncanny Valley. It's amazing how far you've brought this!

Thanks! Your comment is spot on. Creating a system where I can have all the advantages of computer animation, without the huge disadvantage that the uncanny valley typically brings alongside it was the goal from day one.

I've never really been able to put into words why the "Secret of Nihm" or Bambi can provoke an emotional response while the vastly more sophisticated and expensive "final fantasy spirits within" could not. Even though I can't articulate it other than by referencing the uncanny valley, It's very clear from the amount of dust aggregating on the CGI Blu Rays on my bookshelf compared to the animated films.

Here's the movie I remembered:

And here's the movie I forgot: