[Mp4-tech] RE: the range of level_prefix in CAVLC

[Gary Sullivan]

This is a derived limit. It is not in the spec and we provide no
assurances that it is correct.  The question we are trying to answer is
not whether it is possible for a coefficient value to be out of range
when level_prefix has a particular value, but rather whether it is
possible for everything that is specified in the standard to be within
the ranges that are specified in the standard when level_prefix has a
particular value.  In the High and High 4:2:2 profiles, level_prefix is
allowed to have any value that does not force some other constraint
expressed in the standard to be violated.
Best Regards,
Gary Sullivan
________________________________
	From: Nancy Johnson [mailto:nancyjohnson111 yahoo.com] 
	Sent: Monday, October 16, 2023 12:42 PM
	To: Winger, Lowell; Gary Sullivan; mp4-tech lists.mpegif.org
	Subject: Re: the range of level_prefix in CAVLC
	Winger and Gary,
	Thanks a lot. Will this limit on level_prefix for High Profile
be added to the SPEC or it is derived from the request (ex., section
8.5.10 on Cij) that the AC/DC levels should be within -2^(7+BitDepth)
and 2^(7+BitDepth)-1 (for 8-bit video, BitDepth=8)?
	If it is the latter, I analyzed the mechanism for CAVLC and
found that when level_prefix=19, it is still possible for level[i] out
of range of [-2^15,2^15-1]. See the following for detail:
	According to section 7.3.5.3.1, the max possible value for
abs(level[i]) when level_prefix=19 is:
	(((min(15,level_prefix) << suffixLength) + level_suffix +
((1<<(level_prefix-3))-4096) + 4)/2
	Since the max possible value of level_suffix is 2^16-1 for
level_prefix=19 and suffixLength <= 6, so the max possible value for
abs(leve[i]) is (15<<6 + 2^16-1 + 2^16 - 4096 + 4)/2 = 63969 = 0xF9E1,
which is out of range of [-2^15,2^15-1]. If level_prefix=18, the
abs(level[i]) will <= (15<<6 + 2^15-1 + 2^15-4096+4)/2=0x79E1, which is
in range of [-2^15,2^15-1].
	Could you please point where I missed for the above analysis?
Thanks.
	Rgs,
	Nancy
	"Winger, Lowell" <Lowell.Winger lsi.com> wrote:
		Nancy, for High Profile the maximum level_prefix is 19
for 8-bit video.
		Best Regards,
		Lowell Winger, LSI Logic
________________________________
	Stay in the know. Pulse on the new Yahoo.com. Check it out.
<http://us.rd.yahoo.com/evt=42974/*http://www.yahoo.com/preview>  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20061016/0b3e5020/attachment.html