[mp4-tech] inverse transforms data range

[Robin Zoo]

Dear Gary,
Thank you for your reply.
What I meant was, for BitDepthy=8,
the bounds of input data(dij), intermediate data(eij ~
mij) and final result(rij) of the 8x8 inverse
transform is -2^15 ~ 2^15-1, inclusive, which means no
data overflow by using 16-bit 2's complementary
arithmetic.
Since I can't image how the encoders guarantee such an
overflow won't happen by using 16-bit input(don't tell
me that they limit input data within 10 bits :), I
still don't know 
1) whether the hardware can be reduced further;
2) and, whether a simulation mis-match is caused by
hardware implementation error or incorrect bit stream.
thanks in advance!
Robin
--- Gary Sullivan <garysull windows.microsoft.com>
wrote:
> 
> Robin et al,
> 
> I'm not sure I understand your question.
> 
> There is no such statement in 8.5.10 or 8.5.11
> exactly saying that the
> "8x8 or 4x4 inverse transform is performed on 16-bit
> integer data",
> although I think I agree with the spirit of that
> statement.
> 
> Regarding how to guarantee that the constraints
> expressed in 8.5.10 or
> 8.5.11 are fulfilled, I call your attention to 3.132
> which says that
> when a "mandatory constraint [is expressed] on the
> values of syntax
> elements or on the results obtained by operation of
> the specified
> decoding process, it is the responsibility of the
> encoder to ensure that
> the constraint is fulfilled."
> 
> Thus if the bounds expressed in 8.5.10 or 8.5.11 are
> exceeded when
> decoding the data, the bitstream is not considered a
> proper, conforming,
> bitstream.  A decoder can do whatever it wants in
> response to an
> incorrect bitstream.
> 
> Best Regards,
> 
> Gary Sullivan
> 
> +> -----Original Message-----
> +> From: mp4-tech-bounces lists.mpegif.org 
> +> [mailto:mp4-tech-bounces lists.mpegif.org] On
> Behalf Of Robin Zoo
> +> Sent: Thursday, September 15, 2023 10:53 AM
> +> To: mp4-tech lists.mpegif.org
> +> Subject: [mp4-tech] inverse transforms data range
> +> 
> +> As specified in subclause 8.5.10/11 of H.264
> spec, the
> +> 8x8 or 4x4 inverse transform is performed on
> 16-bit
> +> integer data.
> +> 
> +> Could someone briefly explain to me how the
> encoded
> +> bit stream and dequantization guarantee that
> there is
> +> no data overflow since the overflow can't be
> avoided
> +> if the only constrain is that the input
> +> data(dequantized transform coefficient) is
> bounded
> +> within -2^16 ~ 2^16-1(for 8-bit video)?
> +> 
> +> By the way, how to simply generate test stream to
> +> maximize test coverage of the inverse transform
> though
> +> the transform operation is quite simple?
> +> 
> +> 
> +> 
> +> 
> +> 		
> +> __________________________________ 
> +> Yahoo! Mail - PC Magazine Editors' Choice 2005 
> +> http://mail.yahoo.com
> +> _______________________________________________
> +> NOTE: Please use clear subject lines for your
> posts. Include 
> +> [audio, [video], [systems], [general] or another 
> +> apppropriate identifier to indicate the type of
> question you have.
> +> 
> +> Note: Conduct on the mailing list is subject to
> the 
> +> Antitrust guidelines found at 
> +>
>
http://www.mpegif.org/public/documents/vault/mp-out-30042-Ant
> +> itrust.php
> +> 
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com