[Mp4-tech] Re: Leading frames in AAC streams

[Ralph Sperschneider]

Ralph Sperschneider wrote:
> Dear Ralph,
> 
> thanks for pointing this issue out in detail. MPEG is aware of this 
> ambiguity and is heading for a proper solution. For the moment, the 
> issue is investigated in more detail between the Systems and Audio 
> subgroup. The Audio report from the 65th MPEG meeting says the following:
> 
> "
> Since the AAC decoder is not adequately specified in terms of processing 
> of stored state (i.e. the first half of the overlapped MDCT window), 
> there is uncertainty as to the correct value of the time stamp 
> associated with decoded AAC blocks.
> The solution suggested by the group is that the encoder shall operate 
> such that a timestamp assigned to a compressed Audio block is associated 
> with first sample of the output block produced by decoder when 
> processing that block.
> Additionally, in the interest of facilitating deterministic (and highest 
> possible quality) editing of AAC bitstreams, MPEG should consider how to 
> specify the following:
> §    Normative signaling of pre-roll within an MPEG-4 coded stream
> §    Normative table of pre-roll times or blocks (e.g. for each 
> audioObjectType)
> "
> 
> Best regards,
> 
> Ralph
> 
> 
> Ralph Neff wrote:
> 
>> Hi all,
>>
>> I'd like to ask about a problem we've been seeing which affects
>> synchronization between video and AAC audio.  Some content
>> authors add 1 or 2 empty frames at the beginning of AAC
>> bitstreams, but do not compensate for these frames in the
>> A/V timeline.  At higher sampling rates, this doesn't cause any
>> problems (since each frame has such a short duration, the extra
>> frames don't noticeably affect the sync).  However, when you go
>> down to the lowest sampling rates, the effect is noticeable
>> (e.g. at 8khz, two frames create an unwanted 256 mS audio delay).
>>
>> Some decoders seem to compensate for this -- i.e. they introduce
>> a time offset (effectively removing these empty 'leading frames' from
>> the timeline).  Some decoders do not.  The result is that files
>> authored with 8khz AAC audio seem to play in-sync on some decoders,
>> and are a bit off on others.
>>
>> What is the correct behavior?  Our audio experts couldn't find any
>> special treatment of these leading frames in the MPEG-4 audio
>> or systems/file format specs.  So it seems the right thing to do
>> is to respect the time line in the file format (i.e. render the audio
>> frames at their proper sample times, not adding any special
>> offsets).   This means that if an offset is required for proper sync,
>> then it must be explicitly indicated in the file format (e.g. via an
>> editlist atom).
>>
>> Has anyone run into this problem before?  Is the above interpretation
>> correct, or is there something in the standard that specifies the
>> treatment of these leading frames?  It seems that quite a few 
>> implementations out there are doing this special treatment (i.e. 
>> always adding the frames at authoring time and always compensating
>> for them at decode/render time, even though there is no editlist atom to
>> indicate the need for such adjustment).
>>
>> -Ralph
>>
>> Ralph Neff   *   Packetvideo   *   www.pv.com
>> neff pv.com   *  phone: 858-731-5408   *   fax: 858-731-5311
>>
>>
>> _______________________________________________
>> Technotes mailing list
>> Technotes lists.m4if.org
>> http://lists.m4if.org/mailman/listinfo/technotes
> 
> 

Dear all,
this informs you that an MPEG ad-hoc group was instantiated to deal with this issue:
AHG on Timestamps and Audio Codec Behavior
Mandate:    1. Identify issues concerning deterministic Audio behavior for
                    file-to-file and streaming processing, taking into
                    consideration N6140, Audio Codec Behavior.
Chairman:    Schuyler Quackenbush
Duration:    Until next meeting
Meetings:    No
Reflector:    mp4_timestamps uow.edu.au
Subscribe:    To subscribe, go to:
                 http://mailinglists.uow.edu.au/mailman/listinfo/mp4_timestamps
Interested parties should take the chance to subscribe to this list and to 
contribute to the work of this ad-hoc group.
Best regards,
Ralph
-- 
Dipl.-Ing. Ralph Sperschneider  | Phone: +49 9131 776 344
FhG IIS                         | Fax:   +49 9131 776 398
Am Wolfsmantel 33               | mailto:ralph.sperschneider iis.fraunhofer.de
D 91058 Erlangen                | http://www.iis.fhg.de/amm/