[mp4-tech] inverse transforms data range

[Gary Sullivan]

Robin et al,
I'm not sure I understand your question.
There is no such statement in 8.5.10 or 8.5.11 exactly saying that the
"8x8 or 4x4 inverse transform is performed on 16-bit integer data",
although I think I agree with the spirit of that statement.
Regarding how to guarantee that the constraints expressed in 8.5.10 or
8.5.11 are fulfilled, I call your attention to 3.132 which says that
when a "mandatory constraint [is expressed] on the values of syntax
elements or on the results obtained by operation of the specified
decoding process, it is the responsibility of the encoder to ensure that
the constraint is fulfilled."
Thus if the bounds expressed in 8.5.10 or 8.5.11 are exceeded when
decoding the data, the bitstream is not considered a proper, conforming,
bitstream.  A decoder can do whatever it wants in response to an
incorrect bitstream.
Best Regards,
Gary Sullivan
+> -----Original Message-----
+> From: mp4-tech-bounces lists.mpegif.org 
+> [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of Robin Zoo
+> Sent: Thursday, September 15, 2023 10:53 AM
+> To: mp4-tech lists.mpegif.org
+> Subject: [mp4-tech] inverse transforms data range
+> 
+> As specified in subclause 8.5.10/11 of H.264 spec, the
+> 8x8 or 4x4 inverse transform is performed on 16-bit
+> integer data.
+> 
+> Could someone briefly explain to me how the encoded
+> bit stream and dequantization guarantee that there is
+> no data overflow since the overflow can't be avoided
+> if the only constrain is that the input
+> data(dequantized transform coefficient) is bounded
+> within -2^16 ~ 2^16-1(for 8-bit video)?
+> 
+> By the way, how to simply generate test stream to
+> maximize test coverage of the inverse transform though
+> the transform operation is quite simple?
+> 
+> 
+> 
+> 
+> 		
+> __________________________________ 
+> Yahoo! Mail - PC Magazine Editors' Choice 2005 
+> http://mail.yahoo.com
+> _______________________________________________
+> NOTE: Please use clear subject lines for your posts. Include 
+> [audio, [video], [systems], [general] or another 
+> apppropriate identifier to indicate the type of question you have.
+> 
+> Note: Conduct on the mailing list is subject to the 
+> Antitrust guidelines found at 
+> http://www.mpegif.org/public/documents/vault/mp-out-30042-Ant
+> itrust.php
+>