[M4IF Technotes] AVC: scaling and transform

[Gary Sullivan]

The key to understanding this aspect is dynamic range analysis.  By
swapping the order of the inverse transform and the scaling operations
for the DC coefficients in Intra16x16 mode, we keep the necessary
wordlength smaller for performing the operations.  The design keeps all
processing requirements within 16 bits in the decoder, and keeps all
memory access requirements within 16 bits in the example encoder.
The DC inverse transform is a linear operator, and for values of QP
greater than or equal to 12 (which is a very small step size in this
codec) the scaling has the same a linear multiplier effect on every DC
coefficient value.  In those cases the result would be the same
regardless of which order was specified (although the wordlength
requirement would be greater if the scaling was first in the decoder).
For very tiny QP, I also believe it is better to keep the rounding
nonlinearity at the end of the process, which is where it is specified
to be.
For reference information, I suggest consulting JVT-B038, JVT-B039, and
the vcodex white paper on the transform design
(http://www.vcodex.fsnet.co.uk/h264.html).  There was also a paper at
last year's ICIP conference on this part of the design, and there will
be one soon in the IEEE Transactions on Circuits and Systems for Video
Technology.
Best Regards,
Gary Sullivan
+> -----Original Message-----
+> From: technotes-admin   lists.m4if.org 
+> [mailto:technotes-admin   lists.m4if.org] On Behalf Of Alf Rieckmann
+> Sent: Tuesday, June 10, 2023 1:58 PM
+> To: technotes   lists.m4if.org
+> Subject: [M4IF Technotes] AVC: scaling and transform
+> 
+> 
+> Hi,
+> 
+> I have a question about AVC ('looking currently into JVT-G050r1.doc)
+> decoding.
+> 
+> I was wondering why the order for scaling and transform for the DC
+> coefficients in Intra16x16 mode is reverse in comparison to 
+> the residual
+> mode (which is the order I would have expected).
+> 
+> Is there a document which explains the transform and scaling 
+> algorithm
+> partitioning for Intra16x16 DC values?.
+> 
+> Thanks for help
+> 
+> Alf
+> 
+> 
+> _______________________________________________
+> Technotes mailing list
+> Technotes   lists.m4if.org
+> http://lists.m4if.org/mailman/listinfo/technotes
+>