software-related Something I notice about frame rate conversion

[James Rogers]

Sometimes I leave my t.v. on all day with a 10 hour Youtube video playing. I discovered some wonderful videos that tell no story, but instead, show birds in trees, or mountains, or beautiful landscapes from around the world. I will put 1 of these videos on, in effect, turning my t.v. into an animated painting that is ever changing and pleasing to see as I pass through the room in the course of doing whatever it is I'm doing.

I noticed that some of the videos that feature quad-copter footage become slightly jerky when panning. Upon closer examination, I noticed that all of the video is jerky but only pronounced enough to be noticed during sweeping moves, or accelerated moves that show fast movement in any direction. I assumed it was because of the compression used by Youtube.

A month ago, I was shooting some quad-copter footage for a video I made. The footage was not jerky at all, but when I added it to the timeline in Premiere it took on the jerky look I described. I output the finished video then uploaded it to Youtube. Sure enough, my quad-copter footage was slightly jerky. The footage I shot with my camera was fine though. So, what was the difference between the quad-copter footage and the footage I shot with my camera?

Frame rate. My camera was shooting ~24fps while my quad-copter was shooting ~30fps. Premiere made the adjustment to the quad-copter footage to match the frame rate of the sequence, ~24fps. It apparently did this in a very unsophisticated way. I'm not sure how it works, but the end result is a jerk or noticeable pulse that occurs once every second. Once I saw what was going on, I had to ask; how do I fix this?

The answer seems to be that you have to adjust the time instead of adjusting the frame rate. I'll be honest, I haven't had time to run tests, but it makes sense. Using a program like Fusion or Nuke, 2 programs that I know of that can perform vector based time changes, you would have to slow down your ~30fps video by ~20%, render it, then import it into your editing program, telling it to interpret the footage as ~24fps. This should work, but again, I haven't tried it.

 

[Nate North]

I don't know what drone you have exactly, but I encountered the same problem. There are various ways to fix it, but the best and most effective way is to simply normalize your framerate from base if possible. So I was shooting on a red and a drone, and they were at 23.96 and 24 fps respectively, and I got this same jitter you're describing in the timeline. I went into the menu of the camera (could have been the drone, doesn't matter) and changed the framerate to match the other one. That's the best solution.

Here are all the possible solutions, with their ups and downs.

1. Use Frame Interpolation:
- Many modern editing software packages offer frame interpolation or optical flow features. This technology generates new frames in between existing ones, smoothing out motion and reducing jitter.
- Adobe Premiere Pro, for instance, offers an 'Optical Flow' time interpolation option, while DaVinci Resolve has similar features.
- You may need to lengthen your clip a bit to give the interpolation room to smooth, and it still won't be exactly perfect, just better.

2. Reinterpret Frame Rate:
- Instead of conforming the footage by dropping or duplicating frames, you can reinterpret the frame rate of the footage. This changes the playback speed of the clip without affecting the individual frames.
- This method is particularly useful if the difference in frame rates is not extreme.
- This is the first thing I would try in your situation, for drone footage, which does not have to be synced with on mic audio, time change should be minimal and smoothness should be restored.

3. Use Motion-Blur Effects:
- Adding motion blur can sometimes mask the jerkiness resulting from frame rate conversion. It can make the motion seem more fluid.
- Be cautious with this approach as overdoing it can lead to an unnatural look.
- It's a quick fix, and hides the issue a bit, but not the best.

4. Edit in a Higher Frame Rate Timeline:
- Consider editing in a timeline with a higher frame rate than any of your footage. For example, if you have clips at 24fps and 30fps, edit in a 60fps timeline. This can sometimes provide a smoother result when conforming frame rates.

5. Adjust Shutter Speed When Shooting:
- If possible, adjust your shutter speed when shooting to match the final output frame rate. This can help ensure smoother motion if you know in advance that you'll be conforming footage.

6. Use Specialized Software or Plugins:
- There are tools specifically designed for frame rate conversion, like Twixtor, which can provide more advanced control over the process and often yield better results than the standard options in NLEs (Non-Linear Editing systems). I'd specifically single out Topaz AI as being one of the best at AI footage conforming tasks, and I use it all the time as a general purpose tool for restoring, conforming, and upgrading footage of any type.

 

[James Rogers]

I have a Yuneec Typhoon Q500 4K. It's around 5 years old but I'm happy with the video quality and the performance of the copter itself. It shoots 29.97fps, but come to think of it, I never checked the menu to see if I can change that...

Topaz AI is awesome.