[Mp4-tech] [H.264] Use of SPS/PPS in H.264

[Steinberg Erez-RM96322]

Thanks Gary,
So, I understand that an SPS can override a previous SPS with the same
id. I'm guessing this is not allowed in the middle of a sequence, rather
it requires and IDR picture to follow.
The problem I have with supporting multiple PPS/SPS is how to handle
memory allocation. I'm working on a real-time system, and dynamic memory
allocation has to be deterministic. I understand that in order to be
compliant I would have to allocate the worst-case buffer for reference
frames, SPS records, PPS records, etc. This result in a cumbersome
amount of memory, and for an infrastructure system that needs to support
many channels this has a high cost.
Is there a way, perhaps out-of-band, to negotiate between clients on the
maximum number of SPS, PPS, etc. that will be used during a session?
Regards,
Erez
-----Original Message-----
From: Gary Sullivan [mailto:garysull windows.microsoft.com] 
Sent: Wednesday, June 21, 2023 21:58
To: Steinberg Erez-RM96322; mp4-tech lists.mpegif.org
Subject: RE: [Mp4-tech] [H.264] Use of SPS/PPS in H.264
Response in-line below.
Best Regards,
Gary Sullivan 
+> -----Original Message-----
+> From: mp4-tech-bounces lists.mpegif.org 
+> [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of 
+> Steinberg Erez-RM96322
+> Sent: Wednesday, June 21, 2023 2:23 AM
+> To: mp4-tech lists.mpegif.org
+> Subject: [Mp4-tech] [H.264] Use of SPS/PPS in H.264
+> 
+> Hi,
+> 
+> H.264 allows up to 32 sequence parameter sets (SPS) and 256 picture
+> parameter sets (PPS) in a stream.
This is not exactly true.  Those are not limits on the number of SPSs
and PPSs that are allowed in a bitstream.  Those are limits on the
number of SPSs and PPSs that a decoder must be capable of storing at the
same time.
There are no limits on the number of SPSs and PPSs allowed in a
bitstream.
+> 
+> An SPS includes information on the picture resolution, number of
+> reference frame, profile and level. These are cardinal for decoder
+> initialization and setup.
+> 
+> My questions:
+> 
+> (1) Is a decoder expected to handle spontaneous changes in these
+> parameters? I.e., if an IDR picture is received with a different
+> PPS/SPS, is it supposed to reallocate all its memory 
+> buffers? (This is
+> effectively resetting the decoder)
Yes.  There are some special exceptions made about how a decoder may
behave when that happens, but it can happen.
+> 
+> (2) For practical implementations -- are there any known applications
+> that use different SPSs in the same stream? Or, can a decoder assume
+> only one SPS?
Yes there are practical applications.  A decoder cannot assume only one
SPS will be in a bitstream.  Also, even if only one SPS index is used in
a bitstream, the SPS content associated with that index can change.
+> 
+> 
+> Thanks for your help!
+> 
+> Regards,
+> Erez
+> 
+> _______________________________________________
+> NOTE: Please use clear subject lines for your posts. Include 
+> [audio, [video], [systems], [general] or another 
+> apppropriate identifier to indicate the type of question you have.
+> 
+> Note: Conduct on the mailing list is subject to the 
+> Antitrust guidelines found at 
+> http://www.mpegif.org/public/documents/vault/mp-out-30042-Ant
+> itrust.php
+>