Personal #multimedia#video 2 min

Understanding I, P and B frames

Translation of the original in Catalan. Read it in: CatalàCastellano

When hearing about video formats, codecs, DivX, MPEG4, etc. terms such as the I-frame, P-frame and B-frame often come up in the conversation. Tired of it sounding like Greek to me I have decided to make myself a small summary of what all this is about, always from the point of view of someone who knows very little about these subjects.

We can divide video 'frames' into 3 types:

  • I-Frame: also called keyframe. It has no reference frame and can be decoded on its own. We can think of it as something similar to a JPEG.

  • P-Frame: Its content is deduced from a previous frame of type I or P, it is impossible to decode it without looking at that reference.

  • B-Frame: to decode them you have to look at the previous and following frames of type I and P.

This last type of frames are interesting for two reasons:

  • They are easy to predict

  • They do not impact on the quality of the following frames

Because the B-frames depend on the past and the future, then the decoder needs I-P frames from the past and the future to be able to decode the B frame. That leads us to the PTS/DTS concepts.

  • PTS Presentation Time Stamp: we can also understand it as a way of knowing the frame number, the order in which the decoded frames can be seen.

  • DTS Decoder Time Stamp: order in which the frames are processed to be decoded.

An example, to understand it better:

  • Let's suppose that this is a small video: I-0 B-1 B-2 P-3

  • The DTS order would be: I-0 P-3 B-1 B-2

In order to simplify things the video is stored in DTS order.

The problem

At this point you can imagine the problem that showing a video in order involves, since the decoder has to be composing the frames to show them in addition to inserting frames where there are none in order to show the frame that corresponds.

DIVX (and Xvid)

These famous codecs, in order to attack this problem, use a small trick. The idea would be to use a variant of the PB frames and they pack several frames into one. That makes the applications think that there is only one frame and the codec hides the rest.

For example, if we have a file that has the following packed:

In (0 3 1 2) - - - ...

the codec will have to output this:

Out 0 1 2 3 ...

The codec puts frames of type 'null' in the place where the frames it does not have should be, because of having them packed into a frame. Codecs already know that when they receive a null frame they have to discard them.

From the point of view of the encoder, it is important to note that a delay corresponding to the null frames is not introduced. In an AVI file there is no relation between the PTS/DTS stored anywhere in order to inform the decoder when it has to play.