[Mp4-tech] Dropping NAL units on bit errors: Best approach?

Gary Sullivan garysull windows.microsoft.com
Fri Oct 20 14:37:38 ESTEDT 2006 

Yes, it's a byte-alignment recovery trick.  As John points out, extra
zero-valued bytes can be present between any pair of NAL units, but at
least one such byte is required for the most important NAL units to
enable decoder alignment recovery.  Alignment recovery is important in
some scenarios but not others, but the quantity of data necessary to
enable it is so low (about 1-2 bytes per picture) that we decided to
require it to be there all the time.
Best Regards,
Gary Sullivan 
+> -----Original Message-----
+> From: mp4-tech-bounces lists.mpegif.org 
+> [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of John Cox
+> Sent: Friday, October 20, 2023 9:06 AM
+> To: Shankar Manuel Aghito
+> Cc: mp4-tech lists.mpegif.org
+> Subject: Re: [Mp4-tech] Dropping NAL units on bit errors: 
+> Best approach?
+> 
+> Hi
+> 
+> My guess as to why those have NALUs have extra zeros is that 
+> it allows
+> the decoder to find the start of the unit even if the bitstream is
+> presented without knowledge of byte-alignment.
+> 
+> It is always legal to pack the end of a nal_unit with extra zeros
+> (trailing_zero_8bits) so a non-start-of-AU NALU may appear 
+> to have three
+> leading seros but in fact that is one zero on the end of the previous
+> unit and two leading zeros on this unit.  This detail is important if
+> you are trying to determine the exact byte on which a unit 
+> starts (e.g.
+> when encapsulated in PES the PTS applies to the first unit 
+> which starts
+> in that PES packet).
+> 
+> Regards
+> 
+> John Cox
+> SJ Consulting
+> 
+> 
+> >Hi Luqman,
+> >
+> >I've look at the standard and I can see what you say, it 
+> seems that that
+> >zero_byte before start_code_prefix_one_3bytes should be 
+> present only in
+> >SPS, PPS and in the first NALU of each access unit. I don't 
+> know why (I
+> >guess that it is because we are missing something in 
+> understanding the
+> >standard), but I am quite sure that the reference software 
+> put 4 bytes
+> >0x00000001 in the beginning of each NALU, also for multiple 
+> NALUs per
+> >picture, i.e. when there are NALUs that are not the first 
+> in the access
+> >unit. Try my matlab script, posted in "RE: [Mp4-tech] H264 Corrupted
+> >Bitstreams" a couple of days ago.
+> >
+> >Could someone please explain us this "mystery"?
+> >
+> >Also you could try with JM11. I've read of problems with 
+> JM10.2 related
+> >to use of multiple slices/picture.
+> >
+> >If you still have problems, send encoder/decoder output and also
+> >configuration files
+> >
+> >---
+> >
+> >About the error concealment parameter in the decoder 
+> configuration file.
+> >2 (motion copy) often is the best in my experience. But you 
+> will only
+> >see differences when you remove a whole picture, i.e. all 
+> the NALUs of
+> >one picture. If you loose only some parts of a picture, different
+> >concealment algorithms are used, but you cannot control that in the
+> >config file.
+> >
+> >---
+> >
+> >You don't need to have Log2MaxFNumMinus4 soo high! With
+> >Log2MaxFNumMinus4=12 then you can encode 2^(12+4) pictures 
+> before the
+> >counter restarts!!!
+> >
+> >
+> >
+> >Regards,
+> >Manuel
+> >
+> >
+> >-----Original Message-----
+> >From: mp4-tech-bounces lists.mpegif.org
+> >[mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of Luqman
+> >Sent: 18. oktober 2006 17:59
+> >To: mp4-tech lists.mpegif.org
+> >Subject: Re: [Mp4-tech] Dropping NAL units on bit errors: 
+> Best approach?
+> >
+> >
+> >hi Manuel Shankar,
+> >
+> >thanks for your reply. 
+> >
+> >> Shankar Manuel Aghito wanted us to know:
+> >
+> >>The first approach is quite simple and works fine for me.
+> >>You should set the encoder and the decoder to use Annex B 
+> output format
+> >(there are parameters in the configuration files).
+> >
+> >Now, I have implemented NALU dropping code directly from 
+> annexb file.
+> >>
+> >>
+> >>What I am doing is searching for 0x00000001 in the 
+> bitstream. That will
+> >
+> >>indicate the start of each NAL unit.
+> >>I actually did not studied carefully all the details in 
+> Annex B, but in
+> >my experience this always works (the number of NALUs always 
+> correspond
+> >to the expected number).
+> >
+> >Not every NALU has 0x00 0x00 0x00 0x01 as start code 
+> prefix. Some NALUs
+> >can also have 0x00 0x00 0x01 as start code prefix. Only 
+> NALUs with SPS,
+> >PPS and first NALU of an access unit contaings a zero byte 
+> leading the 3
+> >byte start code prefix.
+> >
+> >(page 270: TU-T Rec. H.264 (03/2005) - Prepublished version)
+> >
+> >
+> >>
+> >>If you then remove entire NALUs from the bitstream the 
+> decoder will 
+> >>conceal for the lost NALUs.
+> >
+> >That is something I am having problem with.  Decoder gives different
+> >kinds of warnings and error messages when different NALUs 
+> are dropped. 
+> >
+> >I am not sure whether I should ignore them or try to 
+> improve my code to
+> >remove them.
+> >
+> >>
+> >>Note that if you loose entire pictures, you should enable 
+> the error 
+> >>concealment in the decoder configuration file.
+> >
+> >I have tried 0,1 and 2 in decoder configuration file for error
+> >concealment but without any visible differencec.
+> >>
+> >>You should never remove the first two NALUs which contain 
+> Sequence and 
+> >>Picture parameter set.
+> >>
+> >>Normally I also set Log2MaxFNumMinus4 such that MaxFNum >= 
+> >>number_of_frames_to_encode, otherwise sometimes the 
+> decoder crashes (I 
+> >>think it happens when you remove NALUs in two consecutive frames, 
+> >>corresponding to the positions where the counter restarts, 
+> but if you 
+> >>set this parameter high enough, this will never happen)
+> >>
+> >
+> >In encoder.cfg I set Log2MaxFNumMinus4 = 12 which is the 
+> maximum allowed
+> >value.
+> >
+> >>I am not normally using B frames but I think that if you 
+> use B frames 
+> >>you might also need to take care of Log2MaxPOCLsbMinus4
+> >
+> >I am using B frames either.
+> >
+> >
+> >I will attach output of encoder and decoder for different 
+> dropped NALUs.
+> >
+> >Hope you can give me some hints on where to look for 
+> possible solutions.
+> >
+> >Regards,
+> >
+> >Luqman 
+> >
+> >
+> >
+> >>	-----Original Message----- 
+> >>	From: mp4-tech-bounces lists.mpegif.org on behalf of Luqman 
+> >>	Sent: Fri 13/10/2023 12:52 
+> >>	To: mp4-tech lists.mpegif.org 
+> >>	Cc: 
+> >>	Subject: [Mp4-tech] Dropping NAL units on bit errors: Best
+> >approach?
+> >>	
+> >>	
+> >>
+> >>	hello,
+> >>	
+> >>	I like to simulate video over UMTS link and then drop all NAL
+> >units
+> >>	containg bit error after the transport.
+> >>	
+> >>	One possibility would be to read the bitstream file directly,
+> >look for
+> >>	start_code_prefixes, i.e. 0x000000, to find NAL unit boundaries,
+> >and
+> >>	copy everything except for errored NAL units in a new file.
+> >>	
+> >>	Second approach would involve editing code in decoder directly.
+> >I am
+> >>	working with reference decoder software JM10.2.
+> >>	
+> >>	Since the standard is very complex and I am just a novice on
+> >this topic,
+> >>	I was wondering if the first approach wouldn't be more
+> >appropriate for my
+> >>	purpose. With my limited experience with h264 working on encoded
+> >stream
+> >>	file seems to be more straightforward and universal solution.
+> >>	
+> >>	I would appreciate any comments on this.
+> >>	
+> >>	Regards,
+> >>	
+> >>	--
+> >>	Luqman
+> >>	
+> >>
+> 
+> _______________________________________________
+> NOTE: Please use clear subject lines for your posts. Include 
+> [audio, [video], [systems], [general] or another 
+> apppropriate identifier to indicate the type of question you have.
+> 
+> Note: Conduct on the mailing list is subject to the 
+> Antitrust guidelines found at 
+> http://www.mpegif.org/public/documents/vault/mp-out-30042-Ant
+> itrust.php
+>

More information about the Mp4-tech mailing list