[Mp4-tech] Regarding padding in MPEG-4 part 2
Rickard Sjöberg (KI/EAB)
rickard.sjoberg ericsson.com
Fri Nov 24 15:50:27 ESTEDT 2006
Thank you for resolving this issue. I am happy to see that
MPEG-4 part 2 and H.263 Annex D treats this in the same way.
FYI, I have tested the Microsoft reference software and I
can confirm that there has been a change in the source code
to pad outside the bounding rectangle. Apparently a fix was
done to the Amendment 5 source code.
(ISO/IEC 14496-5:2001/Amd.5:2004).
I have not looked at the Momusys code so I can't say anything
about padding changes there.
/
Regards
Rickard
> -----Original Message-----
> From: Yi-Shin Tung [mailto:tung setabox.com.tw]
> Sent: den 24 november 2023 03:19
> To: Herbert Thoma; Gary Sullivan
> Cc: Rickard Sjöberg (KI/EAB); mp4-tech lists.mpegif.org; Tung
> yi-Shin; ohm ient.rwth-aachen.de
> Subject: Re: [Mp4-tech] Regarding padding in MPEG-4 part 2
>
> Hi Rickard, Gary, Herbert and Jens,
>
> I think both versions of RefSW had been changed to consistent
> with what specified in the 3rd version.
>
> I do not remember there exists some bitstreams available for
> this test in the current conformance set. If you can provide
> some, I could put them in the extended list in the future.
>
> Regards,
> Yi-Shin
>
> ----- Original Message -----
> From: "Herbert Thoma" <tma iis.fhg.de>
> To: "Gary Sullivan" <garysull windows.microsoft.com>
> Cc: ""Rickard Sjöberg (KI/EAB)""
> <rickard.sjoberg ericsson.com>; <mp4-tech lists.mpegif.org>;
> "Tung yi-Shin" <tung cmlab.csie.ntu.edu.tw>; "Yi-Shin Tung"
> <tung setabox.com.tw>; <ohm ient.rwth-aachen.de>
> Sent: Thursday, November 23, 2023 7:59 PM
> Subject: Re: [Mp4-tech] Regarding padding in MPEG-4 part 2
>
>
> > Gary, Yi-Shin, Jens,
> >
> > Please look at the 3rd (2004) edition:
> >
> > The sentence
> > "Note that for rectangular VOP, a reference VOP is defined by
> > video_object_layer_width and video_object_layer_height."
> > in the 2nd edition was changed to
> > "Note that for a rectangular VOP, a reference VOP is defined by
> > video_object_layer_width and video_object_layer_height,
> extended to a
> > multiple of 16"
> > in the 3rd edition.
> >
> > I guess this schould make you happy, Gary :-)
> >
> > Regards,
> > Herbert.
> >
> > Gary Sullivan wrote:
> >> Copying Yi-Shin and Jens, who perhaps remember this topic,
> >>
> >> Actually the action I discussed may have happened longer
> ago than that.
> >> I looked in N6362
> >> and although it touched on some relevant sections (I think
> including
> >> 7.6.3) I did not quite
> >> find an answer to the question there. I also looked in N3664 and
> >> didn't find it there either.
> >> I then looked in the text of the 2nd (2001) edition and although I
> >> got a bit confused by what it says in various plances (although I
> >> haven't really studied the topic fully yet), I think I found it.
> >>
> >> What I think I see supports what I said on this thread.
> At the end
> >> of
> >> 7.6.4 in the 2nd edition
> >> it says the following:
> >>
> >> xref = MIN ( MAX (xcurr+dx, vhmcsr), xdim+vhmcsr-1 ) yref = MIN (
> >> MAX (ycurr+dy, vvmcsr), ydim+vvmcsr-1 )
> >>
> >> and
> >>
> >> "(ydim, xdim) are the dimensions of the bounding rectangle of the
> >> reference VOP"
> >>
> >> and
> >>
> >> "Note that for rectangular VOP, a reference VOP is defined by
> >> video_object_layer_width and video_object_layer_height."
> >>
> >>
> >> Now we know that video_object_layer_width and
> >> video_object_layer_height might not be multiples of 16.
> >>
> >> Based on those equations and quoted sentences, it sounds
> to me like
> >> padding is applied for any location beyond the rectangle having
> >> (width,
> >> height) = (video_object_layer_width, video_object_layer_height).
> >>
> >> This is not something that I am happy about, but I think
> it is what
> >> is currently written and I think there was some historical
> >> corrigendum action that changed it to say that.
> >>
> >> There has been a long history of confusion over this
> issue. I am not
> >> entirely sure that I have read all relevant parts of the
> spec, but I
> >> don't see how the above quoted statements can be interpreted any
> >> other way.
> >>
> >> Best Regards,
> >>
> >> Gary Sullivan
> >>
> >>
> >> +> -----Original Message-----
> >> +> From: Gary Sullivan +> Sent: Thursday, November 23, 2023 2:20 AM
> >> +> To: 'Herbert Thoma'
> >> +> Cc: "Rickard Sjöberg (KI/EAB)"; mp4-tech lists.mpegif.org
> >> +> Subject: RE: [Mp4-tech] Regarding padding in MPEG-4 part 2
> >> +> +> Have you looked at N6362?
> >> +> +> Best Regards,
> >> +> +> -Gary Sullivan
> >> +> +> +> -----Original Message-----
> >> +> +> From: Herbert Thoma
> [mailto:herbert.thoma iis.fraunhofer.de] +>
> >> +> +> +>
> >> Sent: Thursday, November 23, 2023 2:11 AM
> >> +> +> To: Gary Sullivan
> >> +> +> Cc: "Rickard Sjöberg (KI/EAB)"; mp4-tech lists.mpegif.org
> >> +> +> Subject: Re: [Mp4-tech] Regarding padding in MPEG-4 part 2
> >> +> +> +> +> Gary, Rickard,
> >> +> +> +> +> I am pretty sure that the padding from 128x176
> is the +>
> >> +> +> +> +> +>
> >> correct interpretation
> >> +> +> (meaning that the pixels outside of 120x170 but
> inside of +> +>
> >> 128x176 shall be left
> >> +> +> as they were decoded).
> >> +> +> +> +> I remember the discussion in MPEG very well,
> because I +>
> >> +> +> +> +> +>
> >> originally implemented
> >> +> +> it the other way in my encoder and decoder and
> changed that +>
> >> +> +> +>
> >> after the corrigendum.
> >> +> +> +> +> I don't konw if there are any comformance
> bitstreams +>
> >> +> +> +> +> +>
> >> available, but I attached
> >> +> +> a few frames of forman cropped to 170x120 and
> encoded with +>
> >> +> +> +> my
> >> encoder. (I can
> >> +> +> not guarantee that there are actually motion vectors
> that +> +>
> >> test the problem
> >> +> +> in there, though.)
> >> +> +> +> +> Kind regards,
> >> +> +> Herbert.
> >> +> +> +> +> Gary Sullivan wrote:
> >> +> +> > Rickard et al,
> >> +> +> > +> +> > For a long time, I was pretty sure that I
> knew what
> >> +> +> > +> +> > the
> >> +> +> answer was, and my interpretation agreed
> >> +> +> > with yours. Certainly that is the way the similar
> feature +>
> >> +> +> > +>
> >> works in H.263 (although that fact is
> >> +> +> > not directly relevant to the question at hand, since +> +>
> >> "motion vectors over picture boundaries" is
> >> +> +> > a non-Baseline feature of H.263 Annex D, while
> MPEG-4 part +>
> >> +> +> > +>
> >> 2 only tries to be compatible with the
> >> +> +> > Baseline). I vaguely recall that at one time one +> +>
> >> implementation of the reference software was
> >> +> +> > doing it one way and the other was doing it the
> other way, +>
> >> +> +> > +>
> >> and I believe MPEG eventually approved
> >> +> +> > a corrigendum to Part 2 and a bug fix to the
> software to +>
> >> +> +> > +>
> >> clarify it. Unfortunately, I believe the
> >> +> +> > clarification was according to the other interpretation.
> >> +> +> > +> +> > There was a corrigendum finalized in 2004, which I
> >> +> +> > +> +> > think
> >> +> +> corresponded to MPEG output document
> >> +> +> > N6362. I believe this subject was addressed in that +>
> >> corrigendum.
> >> +> +> > +> +> > If I was making an encoder, I might design it to
> >> +> +> > +> +> > avoid +>
> >> +> motion vectors reaching beyond the bottom
> >> +> +> > and right edges of the reference pictures to be
> sure that +>
> >> +> +> > +> I
> >> would work with decoders that used
> >> +> +> > either interpretation. +> +> > +> +> > I guess another
> >> +> +> > decent
> >> encoder approach would be to use +> +> padding in the source for
> >> those areas before
> >> +> +> > encoding too, so that the only difference between
> the two +>
> >> +> +> > +>
> >> interpretations would be the quantization
> >> +> +> > error. With that approach, a little drift might not +> +>
> >> produce very bad artifacts.
> >> +> +> > +> +> > Best Regards,
> >> +> +> > +> +> > Gary Sullivan
> >> +> +> > +> +> > +> -----Original Message-----
> >> +> +> > +> From: mp4-tech-bounces lists.mpegif.org +> +> > +>
> >> [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of +> +> > +>
> >> Rickard Sjöberg (KI/EAB)
> >> +> +> > +> Sent: Wednesday, November 22, 2023 5:36 AM
> >> +> +> > +> To: mp4-tech lists.mpegif.org
> >> +> +> > +> Subject: [Mp4-tech] Regarding padding in MPEG-4 part 2
> >> +> +> > +> +> +> > +> +> +> > +> Dear experts,
> >> +> +> > +> +> +> > +> assume that a simple profile video
> stream with
> >> height and +> +> > +> width of 120 and 170 pixels
> respectively shall
> >> be decoded. +> +> > +> The bounding rectangle of the
> reference VOP is
> >> 128x176. Now +> +> > +> my question is whether you should
> pad outside
> >> of 120x170 or +> +> > +> 128x176 when referencing pixels
> for motion
> >> compensation?
> >> +> +> > +> +> +> > +> The relevant part of the standard is section
> >> +> +> > +> +> +> > +> 7.6.4
> >> i believe +> +> > +> (I looking at the 3rd edition, that's
> ISO/IEC +>
> >> +> 14496-2:2004, N5515):
> >> +> +> > +> +> +> > +> The coordinates of a reference sample in the
> >> reference VOP, +> +> > +> (yref, xref) is determined as follows :
> >> +> +> > +> xref = MIN ( MAX (xcurr+dx, vhmcsr),
> xdim+vhmcsr-1 ) yref
> >> +> +> > +> = MIN ( MAX (ycurr+dy, vvmcsr), ydim+vvmcsr-1)
> >> +> +> > +> +> +> > +> My interpretation of this is that padding
> >> +> +> > +> +> +> > +> should be
> >> done +> +> > +> outside of 128x176 (this means that the pixels
> >> outside of
> >> +> +> > +> 120x170 but inside of 128x176 shall be left as
> they were
> >> +> +> > +> +> +>
> >> > +> decoded), is this correct?
> >> +> +> > +> +> +> > +> Is there any conformance bitstream
> that tests
> >> +> +> > +> +> +> > +> this
> >> behaviour?
> >> +> +> > +> +> +> > +> /
> >> +> +> > +> Rickard Sjoberg
> >> +> +> > +> Ericsson
> >> +> +> > +> +> +> > +>
> _______________________________________________
> >> +> +> > +> NOTE: Please use clear subject lines for your posts.
> >> +> +> > +> Include [audio, [video], [systems], [general]
> or another
> >> +> +> > +> +> +> > +>
> >> apppropriate identifier to indicate the type of +> +>
> question you have.
> >> +> +> > +> +> +> > +> Note: Conduct on the mailing list is
> subject to
> >> +> +> > +> +> +> > +> the
> >> +> +> > +> Antitrust guidelines found at +> +> > +>
> >> http://www.mpegif.org/public/documents/vault/mp-out-30042-Ant
> >> +> +> > +> itrust.php
> >> +> +> > +> +> +> > +> +> >
> >> _______________________________________________
> >> +> +> > NOTE: Please use clear subject lines for your posts. +> +>
> >> Include [audio, [video], [systems], [general] or another +> +>
> >> apppropriate identifier to indicate the type of question you have.
> >> +> +> > +> +> > Note: Conduct on the mailing list is
> subject to the
> >> +> +> > +> +> > +> +>
> >> Antitrust guidelines found at +> +>
> >> http://www.mpegif.org/public/documents/vault/mp-out-30042-Ant
> >> +> +> itrust.php
> >> +> +> > +> +> +> +> --
> >> +> +> Herbert Thoma
> >> +> +> Head of Video Group
> >> +> +> Multimedia Realtime Systems Department Fraunhofer IIS Am
> >> +> +> Wolfsmantel 33, 91058 Erlangen, Germany
> >> +> +> Phone: +49-9131-776-323
> >> +> +> Fax: +49-9131-776-399
> >> +> +> email: tma iis.fhg.de
> >> +> +> www: http://www.iis.fhg.de/
> >> +> +>
> >
> > --
> > Herbert Thoma
> > Head of Video Group
> > Multimedia Realtime Systems Department Fraunhofer IIS Am
> Wolfsmantel
> > 33, 91058 Erlangen, Germany
> > Phone: +49-9131-776-323
> > Fax: +49-9131-776-399
> > email: tma iis.fhg.de
> > www: http://www.iis.fhg.de/
> >
>
>
>
More information about the Mp4-tech
mailing list