[Mp4-tech] H264: is access unit delimiter required?

[Gary Sullivan]

As far as I know, if MPEG2-TS is packed inside of something else, then whatever is inside the MPEG2-TS would follow ordinary MPEG2-TS formatting rules.
Best Regards,
Gary Sullivan
________________________________
From: Victor lyamtsev [mailto:vlyamtsev gmail.com]
Sent: Saturday, October 27, 2023 2:25 PM
To: Gary Sullivan
Cc: mp4-tech lists.mpegif.org
Subject: Re: [Mp4-tech] H264: is access unit delimiter required?
Different system environments inevitably have some differences in how they "package" their payloads.  RTP headers aren't needed inside of video data streams within MPEG-2 systems, and MPEG-2-style start codes aren't carried in the payloads of RTP packets.
-- OK, what's about the case when mpeg-ts is packed inside of RTP, like in case of VideoLAN streamer? In that case NALU delimiters would be used, wouldn't they? Is that why that "klugy" transport is used - to take advantage of existing parsing libs?
Regards,
-V.
On 10/27/07, Gary Sullivan <garysull windows.microsoft.com<mailto:garysull windows.microsoft.com>> wrote:
Victor et al,
It is really quite simple to understand the differences between carrying data in an RTP environment versus carrying them in other environments.
Systems multiplex environments such as MPEG-2 Part 1 tend to treat the media that they convey as a stream of bits (or bytes), so in such environments it is necessary for the data stream itself to contain special patterns -- which we call "start codes".  A decoder needs to search for the start codes within the data stream in order to find the places in the stream where the data it is looking for can be found.
In contrast, RTP is strictly a packet-based data transmission environment. RTP packets have a variable size that is explicitly indicated in header information. It is clear where to find the data payload in the packet. So there is no need for a decoder to search inside of the payload data for start codes in such an environment.
The video standard specifies its syntax structures in terms of chunks called "NAL units", which are essentially packets.  A NAL unit is specific number of bytes that is preceded by one byte called the "NAL unit type" that indicates what kind of data is in the NAL unit.
Carrying the NAL units in RTP basically consists of mapping NAL units to RTP packets.
Carrying the NAL units in MPEG-2 sytems basically consists of prefixing each NAL unit with a start code and then concatenating them to form a stream of bytes.  This formatting is specified in Annex B of the video coding standard.
Start codes (next_bits(24) = 0x000001) are not added to the NAL units in the RTP case, since they are not needed there.  They are only needed for carrying the NAL units in environments such as MPEG-2 that do not adequately support explicit indicators of where to find NAL units in payload data.
As RFC 3984 states, "Annex B of H.264 defines an encapsulation process to transmit such NAL units over byte-stream oriented networks.  In the scope of this memo, Annex B is not relevant."
Different system environments inevitably have some differences in how they "package" their payloads.  RTP headers aren't needed inside of video data streams within MPEG-2 systems, and MPEG-2-style start codes aren't carried in the payloads of RTP packets.
Best Regards,
Gary Sullivan
________________________________
From: Victor lyamtsev [mailto:vlyamtsev gmail.com<mailto:vlyamtsev gmail.com>]
Sent: Friday, October 26, 2023 2:19 PM
To: Gary Sullivan
Cc: mp4-tech lists.mpegif.org<mailto:mp4-tech lists.mpegif.org>
Subject: Re: [Mp4-tech] H264: is access unit delimiter required?
I'll read it it but that just doesn't make much sense to me... It's been my understanding that byte stream is either "raw", or it's contained within RTP packets as RTP payload, therefore must come with delimiters... Why different "packaging" for RTP transport?
How's about other transports, like mpeg-ts? Should I expect to find NALU delimiters there?
Thank you,
Victor
On 10/26/07, Gary Sullivan < garysull windows.microsoft.com<mailto:garysull windows.microsoft.com>> wrote:
Now you are talking about RTP.  That is a particular specialized environment.  I think perhaps those things are not true in the RTP environment.  My understanding is that RTP is unlikely to include access unit delimiters and will not include start codes (i.e., you'll never find that next_bits(24) == 0x000001).  Start codes are part of the "byte stream" format specified in Annex B of the video standard.  My understanding is that the byte stream format is not used in Annex B.  Instead, you should look up RFC 3984.  And study the video spec, of course.
Best Regards,
Gary Sullivan
________________________________
From: Victor lyamtsev [mailto:vlyamtsev gmail.com<mailto:vlyamtsev gmail.com>]
Sent: Friday, October 26, 2023 10:26 AM
To: Gary Sullivan
Cc: mp4-tech lists.mpegif.org<mailto:mp4-tech lists.mpegif.org>
Subject: Re: [Mp4-tech] H264: is access unit delimiter required?
OK, so if I have to parse h264 stream, should I just start looking for the NALU delimiters in RTP payload ( if I am using RTP as transport layer),  as following?
next_bits (24) == 0x000001 && next_bites(32) == 0x000001 - for NALU start
next_bits(24) == 0x0 && next_bits(24) == 0x000001 - for NALU end
Thanks again,
-vl
On 10/24/07, Gary Sullivan < garysull windows.microsoft.com<mailto:garysull windows.microsoft.com>> wrote:
I am almost certain that the answer would be that the answer is No.  Part of the motivation behind some key aspects of the design of the video standard was to consider environments like RTP where some packets will be lost and it is therefore beneficial to ensure that slices can be decoded without dependency on a single copy of some picture header information that appears only once at the beginning of each picture in the transmission sequence.  If we assume that decoders include the functionality to decode individual slices when some information has been lost, the access unit delimiter would appear to serve no purpose in such an environment.
Best Regards,
Gary Sullivan
________________________________
From: Victor lyamtsev [mailto:vlyamtsev gmail.com<mailto:vlyamtsev gmail.com>]
Sent: Wednesday, October 24, 2023 7:32 AM
To: mp4-tech lists.mpegif.org<mailto:mp4-tech lists.mpegif.org>
Cc: Gary Sullivan
Subject: Re: [Mp4-tech] H264: is access unit delimiter required?
Does RTP transport require it?
Thank you,
-vl
On 10/23/07, Gary Sullivan < garysull windows.microsoft.com<mailto:garysull windows.microsoft.com>> wrote:
Its presence is not required by the video standard.  However, its presence may be required in some application-level specifications.
Best Regards,
Gary Sullivan
________________________________
From: mp4-tech-bounces lists.mpegif.org<mailto:mp4-tech-bounces lists.mpegif.org> [mailto: mp4-tech-bounces lists.mpegif.org<mailto:mp4-tech-bounces lists.mpegif.org>] On Behalf Of Victor lyamtsev
Sent: Tuesday, October 23, 2023 7:31 AM
To: mp4-tech lists.mpegif.org<mailto:mp4-tech lists.mpegif.org>
Subject: [Mp4-tech] H264: is access unit delimiter required?
Hello,
Regarding parsing of H264 stream...
Do I have to start from searching for Access Unit delimiter to read NAL unit? More to the point: is the Access Unit delimiter required by standard and is it common practice to use it?
Thanks,
-vl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20071027/0dfae4df/attachment.html