[Mp4-tech] H264 Transform Precision

Ravi Bulusu Ravi.Bulusu portalplayer.com
Fri Feb 4 17:18:18 ESTEDT 2005


To Gary & all the H264 experts, 
I was reading the H.264 standard on the transformation process for
residual 4x4 blocks and the topic that discusses data precision seems to
be unclear. Any help in clarifying my question is greatly appreciated. 
Section 8.5.8 discusses the precision of data stream that is given to
the transform engine. There are four stages stages in the transformation
process; two stages for horizontal transform and two for vertical. In
each stage, the input and output precisions have been mentioned to be in
the range of -2e15 to 2e15 -1. Following the reasoning the output
matrix, denoted by rij, seems to have a range of -2e9 to 2e9 -1 which
implies that the data is 10-bit precise. 
Here are my questions on this data. 
1. How can the error data at the output of transform engine be 10-bit
precise? Is this not limited to 9-bit precision given the two inputs in
generating error data are only 8-bit data? 
2. What is the exact precision of data which is input to the residual
transform engine? If I trace backward from the final output precision,
it looks like the data cannot be more than 12-bit precise. However, the
specification seems to mention that the data could be 16-bit precise. 
I have a feeling that either the spec is trying to be "safe" - or - I'm
probably wrong in my conclusions. 
Someone please help me resolve this problem. 
Thx, 
Ravi


More information about the Mp4-tech mailing list