[Mp4-tech] Non zero Coefficients in deblocking filter

[Gary Sullivan]

It is true that the CBP is an 8x8-level indicator while the transform block size is sometimes smaller.  However, we have some significant constraints on the relationship between coded block pattern and the presence of non-zero transform coefficients.  My recollection is that the reasoning behind those restrictions is to allow the checking of the coded block pattern as a substitute for some of those conditions and that this is what the reference software decoder does (or at least did at one time).  I think that was for deblocking filter purposes.
Referring to the JVT-X201 draft, see subclause 7.4.5 below equation 7-34, up to and including Table 7-15.  There are some constraint statements using the word "shall" there that would not be necessary if we weren't trying to use the CBP for some non-obvious purpose.  Those are there for that reason. There is a long history on that topic.
Best Regards,
Gary Sullivan
________________________________
From: Jain, Anurag [mailto:anurag ti.com]
Sent: Tuesday, April 22, 2024 6:41 AM
To: Gary Sullivan; Srikar Dasari; mp4-tech lists.mpegif.org
Subject: RE: [Mp4-tech] Non zero Coefficients in deblocking filter
Gary,
Doesn’t CBP come at 8x8 block level and Total Coefficients come at 4x4 sub-block level. Boundary strength is calculated for sub-block edges so Total Coefficients is a more appropriate measure to check. Hence CBP bit can be non-zero and still some of the sub-blocks may have all-zero coefficients.
I think we need to check Total Coefficients and derive CSBP (Coded Sub-Block Pattern) for loop filtering.
Regards,
Anurag
________________________________
From: mp4-tech-bounces lists.mpegif.org [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of Gary Sullivan
Sent: Tuesday, April 22, 2024 9:28 AM
To: Srikar Dasari; mp4-tech lists.mpegif.org
Subject: RE: [Mp4-tech] Non zero Coefficients in deblocking filter
Srikar,
I suggest to check what the reference software is doing.  If I remember correctly, the text is phrased so that certain things are prohibited (by disallowing the values of all transform coefficients being decoded as zero when some coded block pattern bits are set), so that the complexity of the decoding process is minimized.  The text and software do the same thing, but that is not obvious at first glance, because their equivalence depends on the exact phrasing of the text.  We had one or two correction efforts devoted to this topic in the past, so if you look in some old drafts you will find incorrect text.
Again, my basic advice is to check what you find in the JM reference software.
Best Regards,
Gary Sullivan
________________________________
From: mp4-tech-bounces lists.mpegif.org [mp4-tech-bounces lists.mpegif.org] On Behalf Of Srikar Dasari [srikar pelican-networks.com]
Sent: Monday, April 21, 2024 3:43 PM
To: mp4-tech lists.mpegif.org
Subject: [Mp4-tech] Non zero Coefficients in deblocking filter
Experts,
I have a question in the boundary calculation in deblocking filter. I kinda of asked this question earlier. I need some more clarification.
In the filtering process the boundary strength is set based on one of the conditions being, the luma block p0 or luma block q0 contains non-zero transform coefficient levels.
My question is,
Isn't this non-zero transform coefficient level condition, come from both CBP value and also from the TC (total coefficients) during the CAVLC decoding?
In other words, Do we have to consider the CBP pattern alone or Do we have to take into account TC being zero, in addition to CBP.
Please let me know, if I am not clear anywhere. Your answers are greatly appreciated.
Thanks,
Srikar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20080422/dce3d0a0/attachment-0001.html