[Mp4-tech] about decoding incomplete h264 video with frame lost

[Wesley De Neve]

Dear Hong Liu,
How many reference frames did you use? The reconstruction of P slice coded frames may rely on multiple reference frames in H.264/AVC.
Also, did you make use of an open GOP structure during encoding? This for instance allows P5 to be dependent on P4 for its reconstruction.
Regarding how frame loss is handled by FFmpeg: a decoder is allowed to do a lot of things to conceal frame loss, like outputting a previously and correctly decoded frame in order to compensate for the currently lost frame. When reference frames are lost, the behavior of an H.264/AVC decoder is also dependent on the value of the gaps_in_frame_num_value_allowed_flag.
Best regards,
Wesley De Neve
  ----- Original Message ----- 
  From: hong liu 
  To: mp4-tech lists.mpegif.org 
  Sent: Wednesday, July 11, 2023 6:15 PM
  Subject: [Mp4-tech] about decoding incomplete h264 video with frame lost
  I have been working on h264 streaming video quality accessment. I use mplayer to receive h.264 videos
  streamed from a Darwin streaming server. MPlayer use ffmpeg to decode h.264 videos. So I have some 
  questions with regard to the ffmpeg in decoding h264 videos. 
  Now I use JM12.2 of the h264 test model to encode 10 frames of a 4:2:0 raw video sequence named test.yuv to an h.264 video named test.264. The Intra interval is 5 frames and no B frame is enabled for
  this testing. So the 10 frame types are listed: I0 P1 P2 P3 P4 I1 P5 P6 P7 P8, where I is denoted I frame
  and P is P frame. 
  Now I manually deleted P3 frame from the H264 encoded video file, test.264 to save it as
  test_damage.264. After ffmpeg was used to decode test_damage.264, 9 frames of a raw video sequence
  were generated, named as test_damage.yuv. Then I analyzed these two raw video sequences: test.yuv
  and test_damage.yuv. I found I0 P1 P2 are the same in both sequences. I1 is the same also. all the left
  frames are different. 
  My questions  are arised from this result. 
  1. Because the P3 frame was deleted manually, so ffmpeg decoder may not decode the  p2 frame
      properly. I am not sure about how the ffmpeg respond to this lost frame. Will it do any error
     concealment on this lost frame. From the analysis result, it may not do anything on it. Under this
     assumption, the p4 frame was decoded, but it was corrupted due to the loss of the P2 frame. Is that true
     for ffmpeg decoder? 
  2. The second question confused me a lot for a while. Because  I1 P5 P6 P7 P8 are intact, these decoded
      uncompressed video frames should be the same as the original ones. Why is there only I1 are found
      exactly the same as the original one? Is encoding P5 depends on not only I1, but also P4? To my 
      understanding, decoding the frames of I1 P5 P6 P7 P8 would not be affected by any frame lost from
     I0 to P4, because their encoding is not dependent on the former video frames. I need an video expert to
     correct me!
  Any help is very appreciated and thank you in advance!
  Hong Liu
  ------------------------------
------------------------------------------------------------------------------
  Yahoo! oneSearch: Finally, mobile search that gives answers, not web links. 
------------------------------------------------------------------------------
  _______________________________________________
  NOTE: Please use clear subject lines for your posts. Include [audio, [video], [systems], [general] or another apppropriate identifier to indicate the type of question you have.
  Note: Conduct on the mailing list is subject to the Antitrust guidelines found at http://www.mpegif.org/public/documents/vault/mp-out-30042-Antitrust.php
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20070711/f59ac1b7/attachment.html