[Mp4-tech] [H.264] Use of SPS/PPS in H.264

[Gary Sullivan]

Reply intserpersed:
Best Regards,
Gary Sullivan
+> -----Original Message-----
+> From: Steinberg Erez-RM96322 [mailto:Erez.Steinberg freescale.com] 
+> Sent: Wednesday, June 21, 2023 1:47 PM
+> To: Gary Sullivan; mp4-tech lists.mpegif.org
+> Subject: RE: [Mp4-tech] [H.264] Use of SPS/PPS in H.264
+> 
+> Thanks Gary,
+> 
+> So, I understand that an SPS can override a previous SPS 
+> with the same
+> id. I'm guessing this is not allowed in the middle of a 
+> sequence, rather
+> it requires and IDR picture to follow.
Yes, such overrides are not allowed for the SPS that is "active" for the
decoding of the current coded video sequence.  But new content for the
non-active SPS id's can arrive in the middle of the coded video
sequence.  And repetitions with the same content are allowed for the
active SPS.
+> 
+> The problem I have with supporting multiple PPS/SPS is how to handle
+> memory allocation. I'm working on a real-time system, and 
+> dynamic memory
+> allocation has to be deterministic. I understand that in order to be
+> compliant I would have to allocate the worst-case buffer for 
+> reference
+> frames, SPS records, PPS records, etc. This result in a cumbersome
+> amount of memory, and for an infrastructure system that 
+> needs to support
+> many channels this has a high cost.
My guess is that the quantity of memory reserved for SPS and PPS storage
will ordinarily be fairly small compared to the total memory needed by a
decoder.
+> 
+> Is there a way, perhaps out-of-band, to negotiate between 
+> clients on the
+> maximum number of SPS, PPS, etc. that will be used during a session?
I am not aware of such a protocol.  I think perhaps such a thing exists
in SIP/SDP usage, but I don't know for sure.
+> 
+> 
+> Regards,
+> Erez
+> 
+> -----Original Message-----
+> From: Gary Sullivan [mailto:garysull windows.microsoft.com] 
+> Sent: Wednesday, June 21, 2023 21:58
+> To: Steinberg Erez-RM96322; mp4-tech lists.mpegif.org
+> Subject: RE: [Mp4-tech] [H.264] Use of SPS/PPS in H.264
+> 
+> 
+> Response in-line below.
+> 
+> Best Regards,
+> 
+> Gary Sullivan 
+> 
+> +> -----Original Message-----
+> +> From: mp4-tech-bounces lists.mpegif.org 
+> +> [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of 
+> +> Steinberg Erez-RM96322
+> +> Sent: Wednesday, June 21, 2023 2:23 AM
+> +> To: mp4-tech lists.mpegif.org
+> +> Subject: [Mp4-tech] [H.264] Use of SPS/PPS in H.264
+> +> 
+> +> Hi,
+> +> 
+> +> H.264 allows up to 32 sequence parameter sets (SPS) and 
+> 256 picture
+> +> parameter sets (PPS) in a stream.
+> 
+> This is not exactly true.  Those are not limits on the number of SPSs
+> and PPSs that are allowed in a bitstream.  Those are limits on the
+> number of SPSs and PPSs that a decoder must be capable of 
+> storing at the
+> same time.
+> 
+> There are no limits on the number of SPSs and PPSs allowed in a
+> bitstream.
+> 
+> 
+> +> 
+> +> An SPS includes information on the picture resolution, number of
+> +> reference frame, profile and level. These are cardinal for decoder
+> +> initialization and setup.
+> +> 
+> +> My questions:
+> +> 
+> +> (1) Is a decoder expected to handle spontaneous changes in these
+> +> parameters? I.e., if an IDR picture is received with a different
+> +> PPS/SPS, is it supposed to reallocate all its memory 
+> +> buffers? (This is
+> +> effectively resetting the decoder)
+> 
+> 
+> Yes.  There are some special exceptions made about how a decoder may
+> behave when that happens, but it can happen.
+> 
+> +> 
+> +> (2) For practical implementations -- are there any known 
+> applications
+> +> that use different SPSs in the same stream? Or, can a 
+> decoder assume
+> +> only one SPS?
+> 
+> 
+> Yes there are practical applications.  A decoder cannot 
+> assume only one
+> SPS will be in a bitstream.  Also, even if only one SPS 
+> index is used in
+> a bitstream, the SPS content associated with that index can change.
+> 
+> +> 
+> +> 
+> +> Thanks for your help!
+> +> 
+> +> Regards,
+> +> Erez
+> +> 
+> +> _______________________________________________
+> +> NOTE: Please use clear subject lines for your posts. Include 
+> +> [audio, [video], [systems], [general] or another 
+> +> apppropriate identifier to indicate the type of question you have.
+> +> 
+> +> Note: Conduct on the mailing list is subject to the 
+> +> Antitrust guidelines found at 
+> +> http://www.mpegif.org/public/documents/vault/mp-out-30042-Ant
+> +> itrust.php
+> +> 
+> 
+>