Current streaming video compression programs seem to be aimed at the past - they seem to be designed for either low bandwidth dialup streaming (where video has to be massively compressed and won't look good, no matter what you do), or download or disc use, where you can, to a certain extent, throw bandwidth at the video until it looks good enough.
Even today's codecs don't compress well for typical modern broadband, where you have a good amount of bandwidth available, don't want to use up all the bandwidth all the time, but can tolerate short spikes in bandwidth.
The main problem I have seen is that you often get video that looks really good until there is a lot of action, at which point the quality suddenly drops in one way or another until some time after the action slows down. This causes the perceived quality of the video to be the lower quality right after the action, rather than the high quality of 95% of the video. Video that is actually lower quality overall that doesn't have perceptible drops in quality will seem to be higher quality.
What I would like to see is the ability to require (where X, Y, A, and N are settings that I could alter):
1. Never drop frames (See my post 'Dropping Frames considered Harmful' for why dropping frames is a bad idea).
2. Under no circumstances can the video ever go above X kbps bitrate.
3. Keep the per-frame quality above A unless you hit the bitrate specified in #2.
4. don't average more than Y kbps over N seconds unless that is needed to meet the quality specified in #3.
5. give me the smallest possible file given requirements 1-4.
I would also want the highest possible quality codec.
No compression program I have seen allows this type of settings, although some have met as many as 3 of the 5. The programs that allow more of these settings have, in my experience, tended to produce better perceived quality overall for a given file size and codec.
Requirement 2 would be prevent the bitrate from going above the estimated audience maximum bandwidth.
Requirement 3 would be ensure that the video quality doesn't collapse
entirely during extreme action.
Requirement 4 would be used to set the overall bitrate of the file and to help keep the bitrate even over the entire duration.
Requirement 5 helps to keep the entire file size down, which saves money.
Friday, October 31, 2008
Thursday, October 30, 2008
Dropping Frames considered Harmful
Allowing a video compression codec to drop frames when it is having trouble keeping the bitrate under control seems to be a good idea. Here is why it is usually worse than other solutions that are available to you.
Where a video codec will need to drop a frame is when there is more action in the frame. If the codec drops the frame, it gets back a little bit of bandwidth. However, since there is a lot of action, the next frame will be more different from the first frame, and will require more bandwidth than usual. This will often require the second frame to be dropped. This causes the frame after to be even more different, possibly causing a third frame to be dropped.
By the time the codec can actually put a frame in, it might actually have to put in a keyframe, which will again eat up the bandwidth saved by dropping frames, leading to dropping the next frame. You could potentially end up with a video that plays back as a 5 frame/second slideshow.
If you tell the codec not to drop frames, the video quality will suffer during the period that it would have dropped frames, but dropping video quality during an action sequence isn't as bad as showing the action as a slideshow.
You may also have these options, depending on the situation:
Where a video codec will need to drop a frame is when there is more action in the frame. If the codec drops the frame, it gets back a little bit of bandwidth. However, since there is a lot of action, the next frame will be more different from the first frame, and will require more bandwidth than usual. This will often require the second frame to be dropped. This causes the frame after to be even more different, possibly causing a third frame to be dropped.
By the time the codec can actually put a frame in, it might actually have to put in a keyframe, which will again eat up the bandwidth saved by dropping frames, leading to dropping the next frame. You could potentially end up with a video that plays back as a 5 frame/second slideshow.
If you tell the codec not to drop frames, the video quality will suffer during the period that it would have dropped frames, but dropping video quality during an action sequence isn't as bad as showing the action as a slideshow.
You may also have these options, depending on the situation:
- Manually set the frame rate to 1/2 the source frame rate. This won't help as much as you think - each frame will take up slightly more space than at full frame rate, due to the larger difference from the previous frame.
- Reduce the frame size. Again, this won't help as much as you think - video compression seems to scale proportional to the linear dimension.
- Increase the bitrate.
- Switch to Variable bitrate - this allows the codec to save some space in the easy to compress areas and use that in the action sequences. If you are already using variable bitrate, you might be able to increase the variability to help when there is a lot of action.
- There are other codec settings that can affect quality - sometimes counter-intuitively. Sometimes altering another setting will improve the quality enough that the codec no longer has trouble with the action.
Subscribe to:
Posts (Atom)