Thursday, October 30, 2008

Dropping Frames considered Harmful

Allowing a video compression codec to drop frames when it is having trouble keeping the bitrate under control seems to be a good idea. Here is why it is usually worse than other solutions that are available to you.

Where a video codec will need to drop a frame is when there is more action in the frame. If the codec drops the frame, it gets back a little bit of bandwidth. However, since there is a lot of action, the next frame will be more different from the first frame, and will require more bandwidth than usual. This will often require the second frame to be dropped. This causes the frame after to be even more different, possibly causing a third frame to be dropped.

By the time the codec can actually put a frame in, it might actually have to put in a keyframe, which will again eat up the bandwidth saved by dropping frames, leading to dropping the next frame. You could potentially end up with a video that plays back as a 5 frame/second slideshow.

If you tell the codec not to drop frames, the video quality will suffer during the period that it would have dropped frames, but dropping video quality during an action sequence isn't as bad as showing the action as a slideshow.
You may also have these options, depending on the situation:
  • Manually set the frame rate to 1/2 the source frame rate. This won't help as much as you think - each frame will take up slightly more space than at full frame rate, due to the larger difference from the previous frame.
  • Reduce the frame size. Again, this won't help as much as you think - video compression seems to scale proportional to the linear dimension.
  • Increase the bitrate.
  • Switch to Variable bitrate - this allows the codec to save some space in the easy to compress areas and use that in the action sequences. If you are already using variable bitrate, you might be able to increase the variability to help when there is a lot of action.
  • There are other codec settings that can affect quality - sometimes counter-intuitively. Sometimes altering another setting will improve the quality enough that the codec no longer has trouble with the action.

No comments: