[M4IF Technotes] Round-off error in IDCT

[Kevan Ahmadi]

The MPEG 4 and MPEG 2 standards refer to an IEEE document which lists a
series of tests that can be performed on an IDCT implementation to verify
it's accuracy.  The implementation being tested is compared to a full
precision floating point implementation and is allowed to deviate by a
maximum of 2 in any case.
The IEEE document is Std 1180-1990 and is titled "IEEE Standard
Specification for the Implementation of 8x8 Inverse Discrete Cosine
Transform".
If you do get hold of that document be aware that both MPEG 2 and 4 specify
changes to the wording of the tests which have the effect of relaxing the
requirements.
I believe the maximum input coefficient size can be represented in 12 bits
(-2048 to +2047) and the output can be represented in 9 bits (-256 to +255).
Good luck with your project.
Kevan Ahmadi
Leading Design Engineer
PowerVR Technologies
kevan.ahmadi   powervr.com
www.powervr.com
-----Original Message-----
From: Morten Boysen [mailto:mortenb   mips.com]
Sent: 25 September 2023 13:30
To: technotes   lists.m4if.org
Subject: [M4IF Technotes] Round-off error in IDCT
Hi,
The reference decoder uses floating point in computing the IDCT. As a 
part of a schoolproject, i have to rewrite the IDCT, so it only uses 
integers. I know this will cause round-off errors, but the problem is: 
How large are these round-off errors allowed to be according to the 
standard?
Furthermore, i need the maximum allowable value of an IDCT coefficient, 
so i can save as many bits as possible.
Thanks
-- 
--
_    _ ____  ___    Morten Boysen      Mailto:mortenb   mips.com
|\  /|||___)(___    MIPS Denmark       Direct: +45 44 86 55 45
| \/ |||    ____)   Lautrupvang 4 B    Switch: +45 44 86 55 55
  TECHNOLOGIES      DK-2750 Ballerup   Fax...: +45 44 86 55 56
                    Denmark            http://www.mips.com/
_______________________________________________
Technotes mailing list
Technotes   lists.m4if.org
http://lists.m4if.org/mailman/listinfo/technotes