Impossible To Watch

[Nate North]

I've wanted to do this for about a year now, but I thought it was probably impossible. Anyway, today I got it working. I'll improve the quality over time but starting today this is a permanent upgrade which will be available in addition to the standard 2D version.

This is a show reel of the 3D Alpha implementation, Unpolished but working. You can watch it on 3D television sets the Meta Quest / 2 or 3 headsets, Or any other stereoscopic capable headset such as the Google cardboard.

Select mode full SBS, and play.

 

[sfoster]

Congrats!

 

[Nate North]

I wanted to do this forever, and today is the day. But before I go nuts converting the Labyrinth. I think I'll just take one night off and convert "The Last Starfighter" into a 3d movie.

 

[sfoster]

I wanted to do this forever, and today is the day. But before I go nuts converting the Labyrinth. I think I'll just take one night off and convert "The Last Starfighter" into a 3d movie.

Oh interesting, I hadn't even considered converting existing movies

 

[Nate North]

Yep, footage is footage, so this advance means all 3 products will have VR or 3D as an option moving forward. Also, anything else I want to watch in 3D. I think I'll do GOT eventually, though that's a 500 hour compute job, and I can't publish it. I'll get this thing more refined first anyway. It's got some z crawl on the edges of the characters still, but I just got it running 5 hours ago, so it's early days on this process.

 

[sfoster]

Okay so here is the big question...

Instead of watching it in VR with stereoscopic vision, can you use generative fill to get rid of the theater environment, and put us actually inside the movie? That's what I've been waiting for and where it gets really exciting to me.

 

[Nate North]

Yes and no but for the present mostly no. I could do it but it would result in a much worse product I think. To start with it would take a lot more work so you have ROI Issues. Then you have cinematography which is really based around directing the attention of the viewer to whatever thing on screen is telling your story, usually a character's face. When you push out to 360 degrees you're looking at a medium that would really excel for stuff like documentary or imax photography, nature shows, et cetera, but would actually be highly detrimental to conventional storytelling.

I would say that that's an issue that could be worked around after the technology had become cost effective for.... I'm guessing a decade. Some people would be able to develop skills in terms of properly directing a spherical film, but right now there just isn't anyone on Earth who has much experience.

You saw this in the first decade of consumer vr product development, Almost 100 percent of the market was kind of clumsy gimmick demos. About a decade in, we still only have a handful of actual aaa games in VR, But the curve is taking off fast with an almost 50% annual growth in consumer adopters now. I estimate it will be quite some time before you see a fully mature feature film in 360 stereo. In my case I could do it now if I wanted to, but it doesn't make sense on many levels. What I'm showing here is a conversion process which means that the 25 hours of film that I've already created for the labyrinth can be retroactively refitted to 3D but definitely not to 360 3d, Generative fill which isn't even actually a thing, it's a marketing term by a company that tried to Co opt an existing technology that we were all using years before they got ahold of it. It's called outpainting, and adobe absolutely did not invent it. Anyway both 3D and VR are niche markets but I think in the near future there will be a lot of cheap 3D devices that don't really support motion tracking VR. An example I think we're five years away from a $250 phone with a 3D screen that doesn't require glasses.

I definitely know what you're talking about and that does sound awesome, you know, Watching a helicopter on fire crash as you turn the head to track its movement through the sky inside the movie. It's just not super practical right now.

 

[Indiecuttingroom]

I've wanted to do this for about a year now, but I thought it was probably impossible. Anyway, today I got it working. I'll improve the quality over time but starting today this is a permanent upgrade which will be available in addition to the standard 2D version.

This is a show reel of the 3D Alpha implementation, Unpolished but working. You can watch it on 3D television sets the Meta Quest / 2 or 3 headsets, Or any other stereoscopic capable headset such as the Google cardboard.

Select mode full SBS, and play.

Love the visuals!

 

[Nate North]

Thanks!

Were any of you able to watch it in 3D? They actually used to sell a lot of 3d Televisions not so long ago.

 

[sfoster]

Yes and no but for the present mostly no. I could do it but it would result in a much worse product I think. To start with it would take a lot more work so you have ROI Issues. Then you have cinematography which is really based around directing the attention of the viewer to whatever thing on screen is telling your story, usually a character's face. When you push out to 360 degrees you're looking at a medium that would really excel for stuff like documentary or imax photography, nature shows, et cetera, but would actually be highly detrimental to conventional storytelling.

I would say that that's an issue that could be worked around after the technology had become cost effective for.... I'm guessing a decade. Some people would be able to develop skills in terms of properly directing a spherical film, but right now there just isn't anyone on Earth who has much experience.

You saw this in the first decade of consumer vr product development, Almost 100 percent of the market was kind of clumsy gimmick demos. About a decade in, we still only have a handful of actual aaa games in VR, But the curve is taking off fast with an almost 50% annual growth in consumer adopters now. I estimate it will be quite some time before you see a fully mature feature film in 360 stereo. In my case I could do it now if I wanted to, but it doesn't make sense on many levels. What I'm showing here is a conversion process which means that the 25 hours of film that I've already created for the labyrinth can be retroactively refitted to 3D but definitely not to 360 3d, Generative fill which isn't even actually a thing, it's a marketing term by a company that tried to Co opt an existing technology that we were all using years before they got ahold of it. It's called outpainting, and adobe absolutely did not invent it. Anyway both 3D and VR are niche markets but I think in the near future there will be a lot of cheap 3D devices that don't really support motion tracking VR. An example I think we're five years away from a $250 phone with a 3D screen that doesn't require glasses.

I definitely know what you're talking about and that does sound awesome, you know, Watching a helicopter on fire crash as you turn the head to track its movement through the sky inside the movie. It's just not super practical right now.

360 is a gimmic, no one wants to watch something and constantly spin around in their chair..
180 degree is perfect, lets you feel like youre inside and you can watch by looking straight ahead

 

[Nate North]

I've seen some really cool stuff in 180 3D. Mostly walking tours of interesting places or events. Shibuya district at sunset, stuff like that. It would definitely be much easier to direct than full surround.

As to outpainting, it doesn't work in motion, at all. What you'd need is just a much higher resolution frame, and a custom model trained on barrel distortion. As of about a month ago, you could kind of do it with Flux Pro 1.1 and some heavy custom training, but no one has done it yet. I'd do it, because I think it would look amazing, but I think I'd have a hard time with returns.

I really feel bad for some of the great 3d content developers. Have you ever done a database search on those oculus films? They were giving people 100k grants to make a 3 minute film, putting it on the front page of the Oculus store, and then getting under 1000 views. When they try to charge 7 bucks for one, lifetime sales are in the low hundreds.

I guess the big issue with 180 stuff is that it's not cross platform. With what I'm doing here, I can make one video that works for both the big and small audience, 2d and 3d. But if I gear for 180 3d native, that's exclusive to headsets with motion tracking.

Believe me, if there were 50 million headsets out in the wild right now, I'd be all over it. It's really an uphill battle for people trying to pioneer new formats.

Just for the record, I have actually seen a few good spherical movies (short films), though they are quite the rarity. it's an interesting and unique experience, but also like you said, kind of feels like a gimmick.

 

[sfoster]

Believe me, if there were 50 million headsets out in the wild right now, I'd be all over it. It's really an uphill battle for people trying to pioneer new formats.

I'm starting to think we are 2-3 generations of headset away from mainstream.
It needs to be way more comfortable and have way better resolution

Could be 10-12 years before VR actually takes off

This was my favorite VR film

 

[Nate North]

I think VR is taking off right now, But I think mainstream vr content is probably 10 12 years away.

They won't get a lot cheaper than the quest three in terms of bang per buck, because those things pack so much hardware at such a low price I'm pretty much convinced it's a loss leader for their subscription services. It's not gonna get much lighter than the big screen beyond which looks like a pair of swimming goggles. Resolution right now in the better headsets and even just the quest 3 is about half of the capacity of the human eye. Because the pixel density is spread out across such a large FOV and comparison to a conventional television, they've all looked pretty low res up until this year. Of the half dozen headsets I've had I would say the quest 3 is the first headset that is really comfortable to use as a television. I'm typing this response right now inside a quest 3, not to get any amazing 3D effect out of the forum page but simply because it's more convenient and works just as well. My three heavy tower computers are now portable throughout the house and yard via the headset and I could flip through them like tv channels with a remote assigning tasks checking status bars etc. The monitor I'm typing this on is actually a bit more legible than my neo qled from 8 feet away.

I actually wouldn't be surprised if the next generation of VR headsets was basically everything you needed it to be, Though I think it might take some time for that news to get out to people because vr has diminished virality in comparison to other tech. You can walk down the sidewalk and see a big screen tv through a store window and be oppressed at how cool it is. When you see someone wearing a great vr headset it's really not the same, I think most people just say that guy looks like an idiot, And walk away without ever actually experiencing the product.

 

[sfoster]

I think VR is taking off right now

Why do you believe that? I haven't heard any killer apps since beat saber came out in 2018... lol
What is this new app you're implying that has everyone excited?

The Meta Quest 3 has a per-eye resolution of 2,064 x 2,208 pixels, which is a 30% increase from the Quest 2's resolution of 1,832 x 1,920 pixels

Yeeeeeeah I'm gonna have to go ahead and call you out on some bullshit.
2k panel.... being blown up to my entire field of view... and then taking a SMALL RECTANGLE OF THAT to display a monitor.. no, just no.

That is low as hell resolution man!!
That's an apparent resolution LOWER THAN SD RESOLUTION, and you're trying to tell me pixelation worse than SD isn't low res? come on..

 

[Nate North]

Why do you believe that? I haven't heard any killer apps since beat saber came out in 2018... lol
What is this new app you're implying that has everyone excited?



Yeeeeeeah I'm gonna have to go ahead and call you out on some bullshit.
2k panel.... being blown up to my entire field of view... and then taking a SMALL RECTANGLE OF THAT to display a monitor.. no, just no.

That is low as hell resolution man!!
That's an apparent resolution LOWER THAN SD RESOLUTION, and you're trying to tell me pixelation worse than SD isn't low res? come on..

Have you actually tried out a quest 3? It's dramatically better and not just because of the resolution. The onboard processor is nothing special, But it's much stronger than the previous ones and also I don't actually use it. The killer app for vr this year is UEVR, Followed by reshade. They are each a type of unlocked software that reconfigures most of the major game engines to run as native vr games or stereoscopic 3D games. I have recently played through black myth WuKong, Silent Hill 2, Horizon Forbidden West, Jedi Survivor, and Balder's Gate 3 on the quest 3. There are unbelievable options for vr this year if you know what you're doing. Hundreds of them.

I had a friend that I talked to 10 years ago about vr and he said at that time oh I'm waiting for VR to become mainstream and get better. Ten years later I talked to him and he said 0h I'm waiting for VR to go mainstream and get better, And then went on to describe all the flaws in the headset he had tried on once 11 years ago. the last three years have been really big for vr. I'll probably never play a game on a flat screen again at this point.

I would note that right now it costs about five grand to get into the good stuff, So that's a pretty big barrier for entry for people that are getting nauseous the first time they try it. You really need a 4090 card to drive a quest three to its maximum capacity, Sir GPU power and networking is actually more of a bottleneck than screen resolution right now. I'd say I'm inside vr about 12 hours a day at this point.

 

[sfoster]

Have you actually tried out a quest 3? It's dramatically better and not just because of the resolution. The onboard processor is nothing special, But it's much stronger than the previous ones and also I don't actually use it. The killer app for vr this year is UEVR, Followed by reshade. They are each a type of unlocked software that reconfigures most of the major game engines to run as native vr games or stereoscopic 3D games. I have recently played through black myth WuKong, Silent Hill 2, Horizon Forbidden West, Jedi Survivor, and Balder's Gate 3 on the quest 3. There are unbelievable options for vr this year if you know what you're doing. Hundreds of them.

I had a friend that I talked to 10 years ago about vr and he said at that time oh I'm waiting for VR to become mainstream and get better. Ten years later I talked to him and he said 0h I'm waiting for VR to go mainstream and get better, And then went on to describe all the flaws in the headset he had tried on once 11 years ago. the last three years have been really big for vr. I'll probably never play a game on a flat screen again at this point.

I would note that right now it costs about five grand to get into the good stuff, So that's a pretty big barrier for entry for people that are getting nauseous the first time they try it. You really need a 4090 card to drive a quest three to its maximum capacity, Sir GPU power and networking is actually more of a bottleneck than screen resolution right now. I'd say I'm inside vr about 12 hours a day at this point.

No, I haven't tried the quest 3, I'm not interested until the resolution gets dramatically higher.
When its a 4k panel per eye I'll give VR another try.

I had the cv1, the quest 1, and the quest 2, and every upgrade the VR community says the same thing youre saying, kidding themselves about how this time it finally looks good enough. it doesn't. it's not even SD quality in appearance.

 

[sfoster]

I'd say I'm inside vr about 12 hours a day at this point.

Holy shit! What?? I missed this part. That's outrageous.
You've gotta be like a 1/5000 outlier.

For context I have an 86" 4k tv in my bedroom, i'd have to be crazy to put on a hot, uncomfortable headset to watch movies in sub-sd resolution instead

 

[Nate North]

I meant to come back and answer this earlier but just didn't get around to it. I got a kidney stone this month and it just kicks up at random times and is....... Distracting.

It's not really that any of your arguments are so far off as much as that this is kind of a multi dimensional thing no pun intended. If I'm watching a tv show in the vr environment, no, the screen does not look as good as my lg oled. Here's those other dimensions I was talking about though. One is the literal z axis dimension. Let's say I want to see every single pixel of a 4K scene that I'm working on. In vr I can grab that lower quality tv with my hand from wherever I've hung it in the air and Just drag it right up to my face until I can literally see every single pixel. Basically in vr the resolution of your screen is relative to how far away it is from you on your Z axis. That's not important when you're watching a movie or a TV show, And if you're just exclusively doing those things the tv is a better option right now for some situations.

The television sets that I spawn in vr by pushing a button are free, require no maintenance, are indestructible, weigh zero lbs, and can simply be deleted when I don't need them around.

Sometimes I'm cooking dinner in the kitchen and want to look up some recipe on youtube to make some food taste better. And I also want to watch the news at the same time. Sometimes I'm also waiting for an email or something like that or a status bar to finish. It's not uncommon for me to be waiting for three status bars to finish on 3 different computers. So I'm standing there in front of the stove waiting for this pot to boil, And I rapidly click five times and spawn 5 television sets around me, A 10 inch window showing the cooking video and instructions, A couple of 5 inch windows showing each of my work computers and their status bars that I just quickly reach up and affix to the wall behind the stove, And a 40 inch tv hanging in the air somewhere over my kitchen table showing the day's national news.

Maybe while I'm cooking dinner one of those status bars finishes. I typically reach over and quickly pause the news television with a click, grab the recipe window which is kind of half blocking the status bar tvs behind it, and move it out of the way with a fast gesture, reassign the computer that finished its job to another job so that no time is wasted. Then I just reach over and grab the cooking instructions or whatever I'm looking at in that window and put it back in front of me, and unpause the news with a click.

An operation like that can be accomplished with one hand in 5 seconds. I'm done with the recipe and the status bars both have a good while to go now that I've reset one so I quickly hit the delete television button that I've left hanging in the air out of my way 4 times leaving only the news hovering above my kitchen table in a 40 inch window. All told that window is probably somewhere around 720p real resolution, But then so is the source video for that so no big deal. If I want to see more detail I can do something else with that window that I can't do with my lg television, which is to simply reach up with my hand and make it double the size. This actually does increase the resolution of that particular screen because it uses more of the native resolution of the VR headset.

This one sub feature of a $500 device definitely does not provide as good of a television experience as a $3000 TV, But it's highly flexible and has a lot of other advantages that have absolutely nothing to do with resolution. I'd also note that my headset does not get hot. That's mainly because I really don't use much internal processing on it, It has fans that blow the very small amount of heat it produces out the top.

The biggest reason that I prefer it so much to my television though, Is that it unifies every device in my entire house and makes them portable to any space on my property. There's a cable box at one end of the first floor, An eighty pound computer at the other end of another floor, Another one in the basement, Another one in my room, etc. With the headset I can flip through all the devices in my house and work with any of them the way you flip through tv channels with a remote.

I would say that whether you're right about the tv being superior or I'm right about the headset being superior really comes down to what your use case is. If you just want to sit down and watch a movie in the best possible quality in your room, the tv is way better and you're absolutely correct. If you wanted to do the kind of things that I'm describing, Or maybe just had a really comfortable chair in a room where your tv isn't, or wanted to play a game Like Max Payne 3 or dragon age in stereoscopic 3D, Or manage multiple work computers while you brush your teeth, The headset is a much better option for that. Point being that there really isn't one of us that's right and one of us that's wrong in this conversation.

The quest 3 is about 30% lighter than the quest 2 and that really makes a difference for comfort. It's not as comfortable as for example a sleep mask that weighs a fraction of an ounce, But it's noticeably lighter and slimmer now. I do wish it was a little more comfortable simply because I use it so much, But bottom line is that it's really not that bad considering I use it 12 plus hours a day and don't have any issues after a year of that. Games I play are indeed lower resolution than their 2D counterparts, But I enjoy them a lot more in full virtual reality presentation than I do on the flat 2D screen, To the extent that I haven't played a game on my television sets in a year now. To me that's the ultimate evidence of being able to call something an upgrade, when you move over to it, spend some time with the new method, and then find you can't go back. I kind of can't imagine walking up two flights of stairs to access one of my computers these days, Everything I need to use is just wherever I am, and that's something no tv had ever been able to give me before.

 

[sfoster]

The quest 3 is about 30% lighter than the quest 2 and that really makes a difference for comfort. It's not as comfortable as for example a sleep mask that weighs a fraction of an ounce, But it's noticeably lighter and slimmer now.

It's not lighter for me though.

My oculus cv1 weighs 470g and your quest 3 weighs 515g.

It's heavier. I find all of your arguments to be pretty strange honestly.
You claim the 'VR is taking off' and I ask why? VR is taking off because you can spend 5-6k and then you can play the same games that are already available, but be less comfortable on a worse screen, but you can do it while you're cooking with a hot stove.

I fail to see how that is taking off.

In the end though, 'good enough' is such a dumb waste of time to debate on the internet.
Clearly it's good enough for you because you're using it!

I have zero desire to use a VR controller in one hand while I'm brushing my teeth with another.
That actually sounds dystopian as fuck to me not every minute of the day has to be squeezed for productivity.

 

[Nate North]

It's not lighter for me though.

My oculus cv1 weighs 470g and your quest 3 weighs 515g.

It's heavier. I find all of your arguments to be pretty strange honestly.
You claim the 'VR is taking off' and I ask why? VR is taking off because you can spend 5-6k and then you can play the same games that are already available, but be less comfortable on a worse screen, but you can do it while you're cooking with a hot stove.

I fail to see how that is taking off.

In the end though, 'good enough' is such a dumb waste of time to debate on the internet.
Clearly it's good enough for you because you're using it!

I have zero desire to use a VR controller in one hand while I'm brushing my teeth with another.
That actually sounds dystopian as fuck to me not every minute of the day has to be squeezed for productivity.

Wellllll......

It is kind of dystopian, But remember that chart you posted of the average life expectancy broken down into weeks? As soon as I realized how many things I wanted to accomplish and how much time there actually was left, I started trying to wring every last drop of life out of every single day. Look at how long it has taken me to get to the point where I'm at with save point right now. If I wasn't pushing as hard as I can to be as efficient as possible this would have taken 10 years for one person with no help and no funding.

To me what's truly dystopian is looking back at people doing the same job 30 years ago, Realizing I've done 200 times as much work as they ever did at eight times the quality and in some cases it's not worth what they got paid over a long weekend. For example I have around 4000 pro quality stock footage clips up on every major marketplace in the world basically. Aerial footage, Advanced Cgi, Anything you could think of. I got paid around $115 over the last 18 months. To put that into context, That exact same library I mentioned used to net me thousands of dollars per year not so long ago. A stock videographer in the 90s could sometimes put up two or 300 clips in 480p, And end up with a half million dollar house, medical insurance for life, a happy family, and three college tuitions for the kids. Today if I do the same amount of work at the same skill level and produce a much better product, I would receive Enough compensation to order three Pizza Hut pizzas on separate occasions. And don't forget that they put up 300 videos on one site, and I put up 4000 videos on 6 sites. It's definitely not just me either, every person I've met in the last decade that was capable of writing a symphony orchestra piece has ended up fighting to stay alive in an efficiency apartment. Anyway that's why I don't feel like I have time to spare.