Clothworld Trailer 1

Looks great.

My only $0.02 - I'd like to see a bit more visual distinction between the bright sunny days and the overcast days (e.g. 0:50 -0:55 seconds)
I'm thinking it's color matching all the shots, regardless of weather, and the homogenization is draining away the vibrancy of the sunny days?

Either way it's very cool! Some of the water looks so realistic compared to what you'd get with CGI

Thanks! Good eye on the color matching. I didn't intentionally homogenize it exactly, but I've been kind of leaning towards the Ron Fricke aesthetic through this one. Sometimes when you're trying to dial everything in too much it shows up in the long run as "too even". Most of his stuff is just textbook photography, color, lighting. It always looks amazing, but sometimes a bit unnatural in context as you pointed out. I looked carefully at the section you mentioned, and though I see what you mean, I don't think it looks too bad.

The water and cloth are starting to work a lot better now than they used too. I meant to finish this one before the month was out, and almost made it, but I'm going to build on one more tech section before I move on.

After this, is where it all gets interesting. The technical framework for scalable production has been completed, and it's time to tackle the final boss. If I can succeed at that this year, the results will be........ IDK, it's hard to understand what the impact will be. It's like working for a paycheck your whole life, then suddenly looking up at the sky and seeing 250 million tons of gold coins raining down and burying your entire city. I'm not 100% sure what's going to happen, even if I fully succeed. People will definitely want to just add an extra 3 seasons to their favorite old show, or play an infinite CYOA type movie, but sometimes I wonder how people will have time to watch all the great television that's coming. Netflix has gone a long way towards bringing the audience to a saturation point. Whether it's me or someone else, the clock is ticking down to the day there is suddenly 800x as much content within a few years.

I get a weird feeling sometimes when I see something like a new Scorsese movie coming out on Netflix, and there's so much out that it almost goes unnoticed. Like the greatest director in the world is still just a flash in the pan at this point, through no fault of their own, but just in the context of the meta saturation.

As someone that owns a sewing machine I have one other suggestion/inquiry.


Observe the fabric. It's all patterened and colorful.
Hardly anyone ever buys fabric that is plain solid color, part of the fun of sewing is that you can use these fun patterns and colors.

e.g. I sewed an orange and black pillowcase with ghost and halloween decor that I use in october.

Is it possible to make your fabrics more interesting rather than just a solid color?
Would not some of these cities be flying their flag?

I tried it, back when I was actually making the footage for this. It's no problem to make whatever texture or pattern I need, so I tried everything at first, every type of fabric, every type of pattern.

Just artistically, the patterns stole focus from the scenes, and made it a lot harder to get an artistic image. I actually cut out all the footage in the film with patterns because the look was stronger overall with solid colors. I tried canvas and silk and paisley and checkers, flags and murals and cloth with images on it, chiffon and wool, etc.

Overall, across this very long film, it's just way more coherent and less noisy with solid colors. I guess all art is subjective, but the short answer is it looked better to me this way in terms of frame composition and maintaining style across a long film.

I guess the other side is that the cloth is supposed to be an "alienating element" meaning that the core design of the film is to show an otherworldly alternate reality that doesn't look like anything you've seen before. I'm intentionally avoiding scenes that look "right" like an average sign or flag. It's supposed to be an "offset universe" where you're seeing things you've never seen before. If some scene looks like it could be real, I threw it out during the edit. Realistic portrayals of an offset universe. Like a dream. Images of another reality.

I think you'll see it more if you ever watch the film. It really works long term and produces a coherent flowing film. Anyway, it's already finished and I'm moving on to the next level for engine dev.

sfoster said:

Hardly anyone ever buys fabric that is plain solid color

Click to expand...

As a proud member of the HardlyAnyone Society, my stash of fabrics is well stocked with plain solid colours.

I'm not opposed to patterned fabric (and yeah, of course I pulled them off the rack for the purposes of the photo above), but they're better suited to individual/small scale pieces - items that can be inspected up close. For anything that's being viewed at a distance of more than a few metres, intricate or "busy" patterns either become fuzzy or, as Nate says, distract from the bigger picture.

This is pretty much how fabrics are used in stage performances and other Grand Scale presentations. The image that came to my mind when Nate posted the first clips of Clothworld was a top-down view of dancers at a performance of Carmen I saw a few years ago:



It was much more impressive in real life, but the point I'm trying to make is that the variations in texture and tone come naturally from the play of light and movement. Nate has captured that very well, and I would think the Clothworld decision-makers would want to prioritise the clothiness of their quirky tradition, rather than promoting specific symbols.

Composition of this strange piece was driven by a few goals. esthetic beauty is a default for me, Having trained for many years in color and composition. I won't always hit the nail on the head like this because that would be boring, But I didn't see any harm in making it look nice this time since it was a test film.

I have no plans on telling people why I made it the way it was when I release it publicly, Because I think that information would be distracting to viewers and cause them to dissect the film. Since it's just film makers here, The core reason for the cloth in the first place is very simple. The functionality test for the system is can I creatively generate entirely new worlds by adding subtracting or modifying X. In this case I just took Earth and added a bunch of cloth. It looks kind of awesome. But the core principle would be the same for anything, If I can do this I can make a planet of pyramids or glass or volcanoes or topiary's or zero G, etc. (Dictation software is great at commas) And the process of successfully adding one element to a scene while maintaining realism is essentially the same as adding any number of things.

Can I combine an unknown element with a known element = yes. The second reason was color coding. At this stage in development I'm teaching the cinematography robot to understand the film that it's looking at and I needed an easy way to understand if it's getting it right on the most basic level before I can proceed to more advanced testing. In the final film which I've already made maybe ten 90 minute versions of, You'll see how the robot groups various location shoots together flawlessly, And executes creative transitions such as large red cloth on city flagpole cuts to large red cloth on mountain flagpole. I needed to know that it could visually comprehend basic elements and use them to properly group scene footage. With this simple easy element I could stack other things throughout the film, Cloth and motorcycle, Cloth and space station, And see clearly how the intelligence was responding to my training and controls in terms of building footage segments that looked intentional and coherent to the audience. The final job calls for an understanding of all elements at a deep level, But an early step towards that goal was simply making sure it could be very effective at juggling two or three balls at once. The cloth is one ball the color is another ball and the subject is the third. So far I'm hitting about 97% I would say. It's so good right now that I could chop up something like Samsara and refabricate it and no one would really be the wiser. Some people might even like the new version better. The last reason I made this one the way I did was to test refabrication through verbal command. I can say transportation to the engine, Then tell it to go formulate the film again, And the film will be significantly more themed around transportation.

This is all pretty much been accomplished already, And tonight I'm beginning work on "Cold Harbor", For those of you who know what I mean by that. Me and the robots are up late dissecting an old episode of law and order, and discussing why it "Is the way it is". I don't think I have to say "the robots and I" when the other person is a robot.

Anyway I hope you some of you enjoy the film when I release it very shortly. I'll put up maybe three versions of it here so anyone that's interested can see what I mean by "The film directs and edits itself on the fly". Honestly it's not a bad watch if you're in the right frame of mind for this type of thing. Personally I like art films, Though I know they're not for everyone. For people that don't enjoy art films, You'll be pleased to know that moving forward content will finally be shifting permanently towards conventional narrative work.

Celtic makes some great points.
I declare takesie backsies

sfoster said:

I declare takesie backsies

Click to expand...

Ah, don't worry about it! It's not like I didn't just spend two months trekking around four countries in search of the perfect button - the perfect patterned button - for a waistcoat I'm making to wear to the same opera venue in July.



The venue and production in question, for anyone interested :


Anyhoo, back on topic ...

@Nate North : your post above answered some questions before I'd asked them, but one remains - for long-format movie, do you see a market for a work that changes every time it's viewed? Narrative or non-narrative, I would think that anyone offering a 90-minute visual experience would want a degree of consistency around which they could plan sales or discussion or other after-the-fact added value. If I watch one version and you watch another, wouldn't it be ... challenging ... to share thoughts about what we'd just seen?

And coming at it from a different angle, let's say I wanted to create a piece to project onto the walls of (for example) an exhibition about textiles and modern life, is the output inevitably different every time? Could I run it a dozen times, decide which version I liked best and then tell the system to stick to that version until further notice?

One observation: there seems to be some incoherence with the focus/depth-of-field/size-and-perspective regarding flying objects. The aircraft at 0m28s, 0m58s and 1m13s all seem to be unrealistically small given their apparent proximity to the ground/trees/buildings. The one at 1m20s is better integrated into the image. I have an idea as to why this might be ... but I'll wait for your comment.

CelticRambler said:

@Nate North : your post above answered some questions before I'd asked them, but one remains - for long-format movie, do you see a market for a work that changes every time it's viewed? Narrative or non-narrative, I would think that anyone offering a 90-minute visual experience would want a degree of consistency around which they could plan sales or discussion or other after-the-fact added value. If I watch one version and you watch another, wouldn't it be ... challenging ... to share thoughts about what we'd just seen?

And coming at it from a different angle, let's say I wanted to create a piece to project onto the walls of (for example) an exhibition about textiles and modern life, is the output inevitably different every time? Could I run it a dozen times, decide which version I liked best and then tell the system to stick to that version until further notice?

One observation: there seems to be some incoherence with the focus/depth-of-field/size-and-perspective regarding flying objects. The aircraft at 0m28s, 0m58s and 1m13s all seem to be unrealistically small given their apparent proximity to the ground/trees/buildings. The one at 1m20s is better integrated into the image. I have an idea as to why this might be ... but I'll wait for your comment.

Click to expand...

Those are legitimate questions. To answer the first question first I think right now there is not a market for an ever shifting experience. I was developing a technology that's intended for a lot of uses that are far more practical and I produced this piece, And frankly it has no direct use case. What I do think it is is a unique one of a kind piece of Art that you don't see too often. As far as whether that has value, I think that's subjective. Somebody paid $6 million for a banana peel last month. In my opinion this is a far more significant piece of art, With a lot more to offer the viewer.

The technology that can produce an ever evolving painting like this, Is simultaneously the technology that can prototype a single episode of law and order and make infinite episodes of law and order. I think the methodology the logic the code that I'm building is incredibly valuable, The clothworld film itself is a bit of a head scratcher. When you watch a version of the entire film, It's fast moving, visually impactful, and unique. In the world of often pretentious and undercooked independent films, Some of which are just a camera pointed at a brick wall for an hour and a half (That's literally an actual film that they teach in film schools so that the graduates can financially underperform communications majors when they graduate), I see some merit in a product that has a unique design and a lot to offer for the rare individual who would enjoy or appreciate it.

It's for the very reasons you bring up that I don't actually have much in the way of plans to produce a lot of these perpetually shifting and morphing films. I would call them more of a curiosity and a byproduct of a significantly more useful process. How do you recommend a show to someone when you know what they see won't be what you saw? how do you return to you see your favorite part again? I can kind of answer those questions, But long term I don't know if they're great answers. When the engine builds a film that film is fixed, So if you see a version and you love it you can go back and watch that version or share it indefinitely. I've thought about this a bit and at least right now, I don't actually see a direct purpose for Infinite shifting films outside of the production of save point and similar interactive products. Perhaps time will prove me wrong, This whole concept is in its infancy. I think you could make some incredibly cool art installations with this technology right now. The thing is I don't have enough money to hire lawyers and protect myself so I can't give anyone the code or let it out into the public for fear of what might happen in that circumstance. So that's a problem.

In answer to your second question, yes. I've built the system to be quite versatile and it can produce a variety of types of Outputs dependent upon control panel configuration. These control panels are pretty extensive too if you've seen pictures of the cockpit of the space shuttle you have the idea. It can make the same film ten times in a row or ten different films, I can change things like the attention span of the editing, Make it consistent or inconsistent to any degree, Create rhythmic patterns, And have it react to multiple types of stimulus from several different sources such as the audio, A master director's template, The footage itself as its being edited, And more. It's now a pretty robust system for engineering footage. With the cloth world test pool for example, I could type in the words green cloth and then have it build the movie, And you would be seeing a very different movie than you would if I typed in motorcycle instead. Outputs at this point are all just standard video files, And the main difference between this program I'm building and da Vinci resolve is that everything is built from the ground up to be fully automated, Intelligent, And aware of its content on multiple levels. If you did install this at an art exhibition, Which is currently impossible because of the code security stuff I mentioned. I think a fascinating opportunity would be simply attaching a microphone to the system and setting a volume input threshold, So people in the crowd could yell things at the painting and the painting would start shifting to redirect the video around the loudest voice it heard. This all looks so long winded on the screen.

An answer to your last question is easy. The system still isn't one hundred percent. Those things you're looking at that look like mistakes, Are mistakes. I'm always working on that aspect from multiple angles and I think anyone that's been following this over the last few years can see that that work is genuinely effective. At this point in time, Many aspects are perhaps 98% pure which is up from something like 40% a year ago. Today I made a quick film to see how high I could score in kind of an ideal situation. I think it looks very good. I always tell people to look at the trajectory, And despite diminishing returns, I feel like through a combination of methods that I'm already implementing in the refinement of those methods we will be looking at something indifferentiable from modern television within a year. This video below represents the current state of the finished save .9 .1 pipeline. (91% complete) I think it's well over 60,000 lines of python code now, And also connects externally to multiple other projects and rented supercomputing power, Primarily for processing of nuanced animation.

Anyway thanks for at least noticing that this film directs itself. It was really a huge amount of work to achieve this and I've been a bit shocked by a public reception that seems tired and bored of a brand new technical and artistic achievement before it's even released.

Also I would note that those are very cool buttons you picked out, And you seem to have a real eye for color design and detail.

You are doing beautiful works, Nate.

Thanks I appreciate it.

Wow, some super realistic stuff here. Especially the birds for some reason.

This was the first test video of a new upgrade of the graphics part of the pipeline. I think in terms of visuals, Just one or two more steps and we're there. It's actually quite usable already, But I had to take it down to 720P to get this level of fidelity, So I want to do at least 1080p native upscale to 4 K and increase baseline fidelity at least one more step.

This year the main focus is all the super important things that no one sees on screen. Character development plot pacing camera angles act structure, All the things that give meaning to television that none of these demos had. I've been working on it for a while now. It's very complicated but in another way it's not as complicated as I thought it would be. I'm already seeing pretty solid evidence that this is possible with current technology. In example, I used it to produce a completely new episode of Gilligan's Island last night. Marianne finds a cave on the island full of carrier pigeons and a professor hatches a plot to get a message to shore so they'll be rescued. Obviously Gilligan ends up screwing it all up. Then I made an episode of Hogan's heroes based on an episode of Futurama. you'd be shocked at how well that actually worked.

Just to be clear I'm not talking about finished episodes you can watch, I'm probably still nine months away from that. But you know full scripts, Editing maps, And so on looking pretty good for when I actually finish wiring all this stuff up.

Nate North said:

This was the first test video of a new upgrade of the graphics part of the pipeline.

Click to expand...

So you took my advice!

How does it look when not in slow-mo? I'm guessing a little less realistic. It looks great tho.