[Mp4-tech] H264: is access unit delimiter required?

[Victor lyamtsev]

Different system environments inevitably have some differences in how they
"package" their payloads.  RTP headers aren't needed inside of video data
streams within MPEG-2 systems, and MPEG-2-style start codes aren't carried
in the payloads of RTP packets.
-- OK, what's about the case when mpeg-ts is packed inside of RTP, like in
case of VideoLAN streamer? In that case NALU delimiters would be used,
wouldn't they? Is that why that "klugy" transport is used - to take
advantage of existing parsing libs?
Regards,
-V.
On 10/27/07, Gary Sullivan <garysull windows.microsoft.com> wrote:
>
>
>
> Victor et al,
>
> It is really quite simple to understand the differences between carrying
> data in an RTP environment versus carrying them in other environments.
>
> Systems multiplex environments such as MPEG-2 Part 1 tend to treat the
> media that they convey as a stream of bits (or bytes), so in such
> environments it is necessary for the data stream itself to contain special
> patterns -- which we call "start codes".  A decoder needs to search for the
> start codes within the data stream in order to find the places in the stream
> where the data it is looking for can be found.
>
> In contrast, RTP is strictly a packet-based data transmission environment.
> RTP packets have a variable size that is explicitly indicated in header
> information. It is clear where to find the data payload in the packet. So
> there is no need for a decoder to search inside of the payload data for
> start codes in such an environment.
>
> The video standard specifies its syntax structures in terms of chunks
> called "NAL units", which are essentially packets.  A NAL unit is specific
> number of bytes that is preceded by one byte called the "NAL unit type" that
> indicates what kind of data is in the NAL unit.
>
> Carrying the NAL units in RTP basically consists of mapping NAL units to
> RTP packets.
>
> Carrying the NAL units in MPEG-2 sytems basically consists of prefixing
> each NAL unit with a start code and then concatenating them to form a stream
> of bytes.  This formatting is specified in Annex B of the video coding
> standard.
>
> Start codes (next_bits(24) = 0x000001) are not added to the NAL units in
> the RTP case, since they are not needed there.  They are only needed for
> carrying the NAL units in environments such as MPEG-2 that do not adequately
> support explicit indicators of where to find NAL units in payload data.
>
> As RFC 3984 states, "Annex B of H.264 defines an encapsulation process to
> transmit such NAL units over byte-stream oriented networks.  In the scope of
> this memo, Annex B is not relevant."
>
> Different system environments inevitably have some differences in how they
> "package" their payloads.  RTP headers aren't needed inside of video data
> streams within MPEG-2 systems, and MPEG-2-style start codes aren't carried
> in the payloads of RTP packets.
>
> Best Regards,
>
> Gary Sullivan
>
>  ------------------------------
> *From:* Victor lyamtsev [mailto:vlyamtsev gmail.com]
> *Sent:* Friday, October 26, 2023 2:19 PM
> *To:* Gary Sullivan
> *Cc:* mp4-tech lists.mpegif.org
> *Subject:* Re: [Mp4-tech] H264: is access unit delimiter required?
>
>
> I'll read it it but that just doesn't make much sense to me... It's been
> my understanding that byte stream is either "raw", or it's contained within
> RTP packets as RTP payload, therefore must come with delimiters... Why
> different "packaging" for RTP transport?
> How's about other transports, like mpeg-ts? Should I expect to find NALU
> delimiters there?
>
> Thank you,
> Victor
> On 10/26/07, Gary Sullivan < garysull windows.microsoft.com> wrote:
> >
> >  Now you are talking about RTP.  That is a particular specialized
> > environment.  I think perhaps those things are not true in the RTP
> > environment.  My understanding is that RTP is unlikely to include access
> > unit delimiters and will not include start codes (i.e., you'll never
> > find that next_bits(24) == 0x000001).  Start codes are part of the "byte
> > stream" format specified in Annex B of the video standard.  My understanding
> > is that the byte stream format is not used in Annex B.  Instead, you should
> > look up RFC 3984.  And study the video spec, of course.
> >
> > Best Regards,
> >
> > Gary Sullivan
> >
> >  ------------------------------
> > *From:* Victor lyamtsev [mailto:vlyamtsev gmail.com]
> > *Sent:* Friday, October 26, 2023 10:26 AM
> > *To:* Gary Sullivan
> > *Cc:* mp4-tech lists.mpegif.org
> > *Subject:* Re: [Mp4-tech] H264: is access unit delimiter required?
> >
> >  OK, so if I have to parse h264 stream, should I just start looking for
> > the NALU delimiters in RTP payload ( if I am using RTP as transport layer),
> > as following?
> > next_bits (24) == 0x000001 && next_bites(32) == 0x000001 - for NALU
> > start
> > next_bits(24) == 0x0 && next_bits(24) == 0x000001 - for NALU end
> > Thanks again,
> > -vl
> >
> > On 10/24/07, Gary Sullivan < garysull windows.microsoft.com> wrote:
> > >
> > >  I am almost certain that the answer would be that the answer is No.
> > > Part of the motivation behind some key aspects of the design of the video
> > > standard was to consider environments like RTP where some packets will be
> > > lost and it is therefore beneficial to ensure that slices can be decoded
> > > without dependency on a single copy of some picture header information that
> > > appears only once at the beginning of each picture in the transmission
> > > sequence.  If we assume that decoders include the functionality to decode
> > > individual slices when some information has been lost, the access unit
> > > delimiter would appear to serve no purpose in such an environment.
> > >
> > > Best Regards,
> > >
> > > Gary Sullivan
> > >
> > >  ------------------------------
> > > *From:* Victor lyamtsev [mailto:vlyamtsev gmail.com]
> > > *Sent:* Wednesday, October 24, 2023 7:32 AM
> > > *To:* mp4-tech lists.mpegif.org
> > > *Cc:* Gary Sullivan
> > > *Subject:* Re: [Mp4-tech] H264: is access unit delimiter required?
> > >
> > >  Does RTP transport require it?
> > > Thank you,
> > > -vl
> > >
> > >
> > > On 10/23/07, Gary Sullivan <garysull windows.microsoft.com> wrote:
> > > >
> > > >  Its presence is not required by the video standard.  However, its
> > > > presence may be required in some application-level specifications.
> > > >
> > > > Best Regards,
> > > >
> > > > Gary Sullivan
> > > >
> > > >  ------------------------------
> > > > *From:* mp4-tech-bounces lists.mpegif.org [mailto:
> > > > mp4-tech-bounces lists.mpegif.org] *On Behalf Of *Victor lyamtsev
> > > > *Sent:* Tuesday, October 23, 2023 7:31 AM
> > > > *To:* mp4-tech lists.mpegif.org
> > > > *Subject:* [Mp4-tech] H264: is access unit delimiter required?
> > > >
> > > >
> > > >  Hello,
> > > > Regarding parsing of H264 stream...
> > > > Do I have to start from searching for Access Unit delimiter to read
> > > > NAL unit? More to the point: is the Access Unit delimiter required by
> > > > standard and is it common practice to use it?
> > > > Thanks,
> > > > -vl
> > > >
> > > >
> > > >
> > > >
> > >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20071027/85ed7124/attachment-0001.html