[Mp4-tech] RE: the range of level_prefix in CAVLC

[Nancy Johnson]

Gary,
  Thank you very much for pointing out this important thing for specification. Now I think 19 is correct. For level_prefix=19 (BitDepth=8), it's still possible for everything that is specified in the standard to be within the ranges that are specified in the standard. For level_prefix=20 (BitDepth=8), it seems to be impossible for everything that is specified in the standard to be within the ranges that are specified in the standard due to the effect of (1 << (level_prefix -3)).
  Rgs,
  Nancy
Gary Sullivan <garysull windows.microsoft.com> wrote:
      This is a derived limit. It is not in the spec and we provide no assurances that it is correct.  The question we are trying to answer is not whether it is possible for a coefficient value to be out of range when level_prefix has a particular value, but rather whether it is possible for everything that is specified in the standard to be within the ranges that are specified in the standard when level_prefix has a particular value.  In the High and High 4:2:2 profiles, level_prefix is allowed to have any value that does not force some other constraint expressed in the standard to be violated.
  Best Regards,
  Gary Sullivan
---------------------------------
  From: Nancy Johnson [mailto:nancyjohnson111 yahoo.com] 
Sent: Monday, October 16, 2023 12:42 PM
To: Winger, Lowell; Gary Sullivan; mp4-tech lists.mpegif.org
Subject: Re: the range of level_prefix in CAVLC
  Winger and Gary,
  Thanks a lot. Will this limit on level_prefix for High Profile be added to the SPEC or it is derived from the request (ex., section 8.5.10 on Cij) that the AC/DC levels should be within -2^(7+BitDepth) and 2^(7+BitDepth)-1 (for 8-bit video, BitDepth=8)?
  If it is the latter, I analyzed the mechanism for CAVLC and found that when level_prefix=19, it is still possible for level[i] out of range of [-2^15,2^15-1]. See the following for detail:
  According to section 7.3.5.3.1, the max possible value for abs(level[i]) when level_prefix=19 is:
  (((min(15,level_prefix) << suffixLength) + level_suffix + ((1<<(level_prefix-3))-4096) + 4)/2
  Since the max possible value of level_suffix is 2^16-1 for level_prefix=19 and suffixLength <= 6, so the max possible value for abs(leve[i]) is (15<<6 + 2^16-1 + 2^16 - 4096 + 4)/2 = 63969 = 0xF9E1, which is out of range of [-2^15,2^15-1]. If level_prefix=18, the abs(level[i]) will <= (15<<6 + 2^15-1 + 2^15-4096+4)/2=0x79E1, which is in range of [-2^15,2^15-1].
  Could you please point where I missed for the above analysis? Thanks.
  Rgs,
  Nancy
"Winger, Lowell" <Lowell.Winger lsi.com> wrote:
Nancy, for High Profile the maximum level_prefix is 19 for 8-bit video.
Best Regards,
Lowell Winger, LSI Logic
---------------------------------
  Stay in the know. Pulse on the new Yahoo.com. Check it out. 
---------------------------------
Do you Yahoo!?
 Everyone is raving about the  all-new Yahoo! Mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20061016/953bd471/attachment-0001.html