[Mp4-tech] Regarding padding in MPEG-4 part 2

Gary Sullivan garysull windows.microsoft.com
Thu Nov 23 04:05:22 ESTEDT 2006 

Oops.  It sounds like your quotes from a 2004 edition (N5546?) are different than my quotes from the 2001 edition!
So maybe something else happened after what I remember happening.
Hopefully you're right.  I don't like what it said in the 2001 edition.
(Don't blame this topic on H.263+.  I think there was never any confusion about the definition of this aspect of that standard.  I think.)
Best Regards,
Gary Sullivan
+> -----Original Message-----
+> From: Herbert Thoma [mailto:tma iis.fhg.de] 
+> Sent: Thursday, November 23, 2023 3:51 AM
+> To: Gary Sullivan
+> Cc: "Rickard Sjöberg (KI/EAB)"; mp4-tech lists.mpegif.org
+> Subject: Re: [Mp4-tech] Regarding padding in MPEG-4 part 2
+> 
+> Gary Sullivan wrote:
+> > Have you looked at N6362?
+> 
+> I just did. The only item about padding I see is
+> "In subclause 7.6.4, clarify the padding process in case of 
+> interlaced sequences"
+> 
+> We keep the sentence: "... A bounding rectangle is defined 
+> by vop_width and
+> vop_height extended to multiple of 16 ..."
+> 
+> And in ISO/IEC 14496-2:2004 subclause 7.6.4 states later:
+> "... Note that for a rectangular VOP, a reference VOP is defined by
+> video_object_layer_width and video_object_layer_height, extended to
+> a multiple of 16 ..."
+> 
+> So I don't think that we changed anything regarding padding 
+> with N6362.
+> 
+> Things were so easy with H.263: you got SQCIF, QCIF, CIF, 
+> 4CIF and 16CIF.
+> That's it, and all are multiples of 16. With H.263+ we got 
+> custom picture
+> formats ...
+> 
+> Kind Regards,
+>   Herbert.
+> 
+> > Best Regards,
+> > 
+> > -Gary Sullivan
+> > 
+> > +> -----Original Message-----
+> > +> From: Herbert Thoma [mailto:herbert.thoma iis.fraunhofer.de] 
+> > +> Sent: Thursday, November 23, 2023 2:11 AM
+> > +> To: Gary Sullivan
+> > +> Cc: "Rickard Sjöberg (KI/EAB)"; mp4-tech lists.mpegif.org
+> > +> Subject: Re: [Mp4-tech] Regarding padding in MPEG-4 part 2
+> > +> 
+> > +> Gary, Rickard,
+> > +> 
+> > +> I am pretty sure that the padding from 128x176 is the 
+> > +> correct interpretation
+> > +> (meaning that the pixels outside of 120x170 but inside of 
+> > +> 128x176 shall be left
+> > +> as they were decoded).
+> > +> 
+> > +> I remember the discussion in MPEG very well, because I 
+> > +> originally implemented
+> > +> it the other way in my encoder and decoder and changed that 
+> > +> after the corrigendum.
+> > +> 
+> > +> I don't konw if there are any comformance bitstreams 
+> > +> available, but I attached
+> > +> a few frames of forman cropped to 170x120 and encoded with 
+> > +> my encoder. (I can
+> > +> not guarantee that there are actually motion vectors that 
+> > +> test the problem
+> > +> in there, though.)
+> > +> 
+> > +> Kind regards,
+> > +>   Herbert.
+> > +> 
+> > +> Gary Sullivan wrote:
+> > +> > Rickard et al,
+> > +> > 
+> > +> > For a long time, I was pretty sure that I knew what the 
+> > +> answer was, and my interpretation agreed
+> > +> > with yours.  Certainly that is the way the similar feature 
+> > +> works in H.263 (although that fact is
+> > +> > not directly relevant to the question at hand, since 
+> > +> "motion vectors over picture boundaries" is
+> > +> > a non-Baseline feature of H.263 Annex D, while MPEG-4 part 
+> > +> 2 only tries to be compatible with the
+> > +> > Baseline).  I vaguely recall that at one time one 
+> > +> implementation of the reference software was
+> > +> > doing it one way and the other was doing it the other way, 
+> > +> and I believe MPEG eventually approved
+> > +> > a corrigendum to Part 2 and a bug fix to the software to 
+> > +> clarify it.  Unfortunately, I believe the
+> > +> > clarification was according to the other interpretation.
+> > +> > 
+> > +> > There was a corrigendum finalized in 2004, which I think 
+> > +> corresponded to MPEG output document
+> > +> > N6362.  I believe this subject was addressed in that 
+> corrigendum.
+> > +> > 
+> > +> > If I was making an encoder, I might design it to avoid 
+> > +> motion vectors reaching beyond the bottom
+> > +> > and right edges of the reference pictures to be sure that 
+> > +> I would work with decoders that used
+> > +> > either interpretation.  
+> > +> > 
+> > +> > I guess another decent encoder approach would be to use 
+> > +> padding in the source for those areas before
+> > +> > encoding too, so that the only difference between the two 
+> > +> interpretations would be the quantization
+> > +> > error.  With that approach, a little drift might not 
+> > +> produce very bad artifacts.
+> > +> > 
+> > +> > Best Regards,
+> > +> > 
+> > +> > Gary Sullivan
+> > +> > 
+> > +> > +> -----Original Message-----
+> > +> > +> From: mp4-tech-bounces lists.mpegif.org 
+> > +> > +> [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of 
+> > +> > +> Rickard Sjöberg (KI/EAB)
+> > +> > +> Sent: Wednesday, November 22, 2023 5:36 AM
+> > +> > +> To: mp4-tech lists.mpegif.org
+> > +> > +> Subject: [Mp4-tech] Regarding padding in MPEG-4 part 2
+> > +> > +> 
+> > +> > +> 
+> > +> > +> Dear experts,
+> > +> > +> 
+> > +> > +> assume that a simple profile video stream with height and 
+> > +> > +> width of 120 and 170 pixels respectively shall be decoded. 
+> > +> > +> The bounding rectangle of the reference VOP is 
+> 128x176. Now 
+> > +> > +> my question is whether you should pad outside of 
+> 120x170 or 
+> > +> > +> 128x176 when referencing pixels for motion compensation?
+> > +> > +> 
+> > +> > +> The relevant part of the standard is section 7.6.4 
+> i believe 
+> > +> > +> (I looking at the 3rd edition, that's ISO/IEC 
+> > +> 14496-2:2004, N5515):
+> > +> > +> 
+> > +> > +> The coordinates of a reference sample in the 
+> reference VOP, 
+> > +> > +> (yref, xref) is determined as follows :
+> > +> > +> xref = MIN ( MAX (xcurr+dx, vhmcsr), xdim+vhmcsr-1 )
+> > +> > +> yref = MIN ( MAX (ycurr+dy, vvmcsr), ydim+vvmcsr-1)
+> > +> > +> 
+> > +> > +> My interpretation of this is that padding should be done 
+> > +> > +> outside of 128x176 (this means that the pixels outside of 
+> > +> > +> 120x170 but inside of 128x176 shall be left as they were 
+> > +> > +> decoded), is this correct?
+> > +> > +> 
+> > +> > +> Is there any conformance bitstream that tests this 
+> behaviour?
+> > +> > +> 
+> > +> > +> /
+> > +> > +> Rickard Sjoberg
+> > +> > +> Ericsson
+> > +> > +> 
+> > +> > +> _______________________________________________
+> > +> > +> NOTE: Please use clear subject lines for your 
+> posts. Include 
+> > +> > +> [audio, [video], [systems], [general] or another 
+> > +> > +> apppropriate identifier to indicate the type of 
+> > +> question you have.
+> > +> > +> 
+> > +> > +> Note: Conduct on the mailing list is subject to the 
+> > +> > +> Antitrust guidelines found at 
+> > +> > +> 
+> http://www.mpegif.org/public/documents/vault/mp-out-30042-Ant
+> > +> > +> itrust.php
+> > +> > +> 
+> > +> > 
+> > +> > _______________________________________________
+> > +> > NOTE: Please use clear subject lines for your posts. 
+> > +> Include [audio, [video], [systems], [general] or another 
+> > +> apppropriate identifier to indicate the type of 
+> question you have.
+> > +> > 
+> > +> > Note: Conduct on the mailing list is subject to the 
+> > +> Antitrust guidelines found at 
+> > +> http://www.mpegif.org/public/documents/vault/mp-out-30042-Ant
+> > +> itrust.php
+> > +> > 
+> > +> 
+> > +> -- 
+> > +> Herbert Thoma
+> > +> Head of Video Group
+> > +> Multimedia Realtime Systems Department
+> > +> Fraunhofer IIS
+> > +> Am Wolfsmantel 33, 91058 Erlangen, Germany
+> > +> Phone: +49-9131-776-323
+> > +> Fax:   +49-9131-776-399
+> > +> email: tma iis.fhg.de
+> > +> www: http://www.iis.fhg.de/
+> > +> 
+> > 
+> 
+> -- 
+> Herbert Thoma
+> Head of Video Group
+> Multimedia Realtime Systems Department
+> Fraunhofer IIS
+> Am Wolfsmantel 33, 91058 Erlangen, Germany
+> Phone: +49-9131-776-323
+> Fax:   +49-9131-776-399
+> email: tma iis.fhg.de
+> www: http://www.iis.fhg.de/
+> 
+>

More information about the Mp4-tech mailing list