[Mp4-tech] [Video][H.264] Decoded Reference Picture Marking Process

[Karsten Suehring]

Dear Reshma,
please see my comments inline.
Reshma Prasanna wrote:
> Dear H.264 Experts,
> 
> I would really appreciate it if anyone could answer my questions.
> 
> Qn 1) Decoded reference picture marking.
> Clause 8 of the H.264 std specifies that the only restriction on invoking
> the decoded reference picture marking process is that the decoded picture
> must be a reference picture, there are no constraints on the type of frame
> i.e. I, P, IDR etc.
> 
> Can the adaptive memory control decoded reference picture marking process
> be invoked if the decoded picture is an I frame but not IDR?

Yes, that's possible.
> If yes, then consider the case when there are decoded reference pictures
> stored in the DPB, the current decoded picture is an I frame(not IDR) and
> MMCO = 1 is decoded. This means that some short term ref picture with
> picNum = CurrPicNum - (difference_of_pic_nums_minus_1 + 1) is to be marked
> as "unused for reference". Since the decoded picture has only I slices,
> the reference lists have not been constructed and hence picNum =
> FrameNumWrap has not been calculated for any short tm reference frames
> w.r.t the current picture's frame_num. Then how will the mmco = 1
> operation be properly executed? The same question holds with any other
> mmco operation that requires comparison to reference lists, if the current
> decoded picture is an I frame.

This problem has already been identified. The text was fixed in the
first corrigendum to invoke the FrameNumWrap calculation also for I slices.
This change should already be included in the recommendation/standard
version currently published by ITU and ISO.
> Qn 2) Consider this case in a H.264 encoder:
> The DPB is full and a reconstructed(decoded) picture (not IDR) is to be
> inserted into the DPB. If adaptive_ref_pic_marking_mode_flag = 0, then
> sliding window process will mark the short tm reference frm with smallest
> value of FrameNumWrap as "unused". But if say, the encoder wishes to mark
> the current picture as "used for long term reference", since the picture
> is not IDR, then adaptive_ref_pic_marking_mode_flag must be set equal to 1
> and mmco = 6 must be used. In this case, sliding window process is not
> invoked and hence none of the frames in the DPB will be marked as "unused
> for reference". There will not be any space in the DPB for the current
> frame.

The unmarking must be signaled explicitly in that case.
> My understanding is that the encoder must then find which short term frame
> has smallest frameNumWrap and send another mmco = 1 to set this short tm
> reference frame as "unused" so that the current reconstructed frame can be
> inserted into the DPB. Is this correct?

The encoder is free to decide which picture is marked "unused for
reference" (short or long-term) as long as the unmarking is done before
marking the current picture. That doesn't need to be the short-term
frame with the smallest value of frameNumWrap.
The corrigendum text also contains some clarifications which commands
are allowed in which order and at which points the maximum number of
reference frames constraint is checked.
I would suggest reading the updated text on MMCO.
Best regards,
Karsten