[Mp4-tech] [System]Audio video sync

[Vinayak Pore]

There two possible methods.
1. Keep a timer in the thread with High Priority with what ever best 
resolution / precision that OS can give. This thread just updates the 
system time. Other threads (Audio and Video Renderer) can decide when to 
submit the respective buffers to devices depending on the time stamp of 
the buffer which you get from file. Just a caution to keep the 
processing in the timer thread minimum in order not to load processor.
In this method it is synchronized to the master clock as implemented by 
timer.
2. Second method is to synchronize to the audio buffer playout. So fill 
the audio buffer for corresponding time, and when this is played out the 
Message / Interrupt can be received. The video time can be decided based 
on this. This eliminates the precision timer and processor load for time 
keeping.
hope this helps.
-- 
Best regards,
Vinayak.
--------------
Jayant Chauhan wrote:
>Dear Dipankar and Bharat
>1) Thanks fellas, but had thought of that. The problem I faced with
>that approach was in implementing the clock, as in, if I use a TIMER,
>it becomes process specific. So it would be nice if you could tell as
>to how to implement a timer which runs on system time (if I have a
>thread which keeps calling GETSYSTEMTIME, it will slow down the whole
>system), so could you please suggest me the implementation of the
>clock itself :D
>
>2) Another problem I am facing (due to unavailability of the ISO
>specs) is that in some of the standard ISO mpeg-4 streams provided,
>like v10-transition.mp4, there are alot of visual tracks and they open
>up in different windows when played in VLC player. Now how do I find
>out as to which audio track is associated with which video ( as in,
>how do I use the "tref" atom , assuming that is where I can get the
>info from) and also as to one basic doubt, which follows in the next
>query numbered 3 :)
>
>3) In one moov, is it possible that that there be 2 video tracks !?? I
>have some files which show 2 video tracks , with bothhaving the same
>durations but one has only 1 sample and the other one is the main
>video track. What is this video track with just 1 sample number ?!! Is
>it just a JPEG (though  not shown as a JPEG).
>
>thanking you in advance
>
>with regards
>Jayant
>
>On Mon, 18 Oct 2023 15:55:55 +0530, Dipankar Mitra
><dipankar.mitra lgsoftindia.com> wrote:
>  
>
>>Thanks Bharat,
>>Posting it to the mail list too...
>>
>>Regards,
>>Dipankar
>>=============================================
>>
>>Embassy Icon , 7th Floor
>>Infantry Road
>>Shivajinagar
>>Bangalore - 560001
>>Ph No. - 56938700 Ext : 177
>>Fax      -  56938800
>>
>>-----Original Message-----
>>From: Bharat P. SONI [mailto:bharat.soni st.com]
>>Sent: Monday, October 18, 2023 3:44 PM
>>To: Dipankar Mitra
>>Subject: RE: [Mp4-tech] [System]Audio video sync
>>
>>Hi,
>>
>>I did use the szstem time to synchronize the audio-video.
>>Following are the steps I used to sznchroniye.
>>1. Make sure that all your timeing calculations, like clock
>>time, CTS, DTS are relative to common reference start time.
>>
>>2. while decoding, check if the decoding time of the current
>>access unit is less than current clock time. If yes then
>>decode, otherwise discard that access unit.
>>2. store the decoded audio-video access units in queue along
>>with their CTS.
>>
>>For video,
>>1. get the next pending picture with lowest CTS, but greater
>>than the current system time.
>>2. Schedule the timer for that CTS.
>>3. When timer hits, send the picture to the renderer.
>>
>>For Audio
>>1. I worked on a system in which call back was being called
>>whenever audio renderer finishes rendering the data in its
>>buffer. So whenever the call back hits, check the CTS of
>>audio access units in the queue and select the access unit
>>having CTS closest to the current system time.
>>2. Send this access unit to the renderer.
>>3. remove all the access units having CTS less than current
>>system time.
>>
>>I hope this helps.
>>
>>Regards,
>>Bharat
>>
>>
>>
>>
>>_______________________________________________
>>NOTE: Please use clear subject lines for your posts. Include [audio, [video], [systems], [general] or another apppropriate identifier to indicate the type of question you have.
>>
>>Note: Conduct on the mailing list is subject to the Antitrust guidelines found at http://www.mpegif.org/public/documents/vault/mp-out-30042-Antitrust.php
>>
>>    
>>
>_______________________________________________
>NOTE: Please use clear subject lines for your posts. Include [audio, [video], [systems], [general] or another apppropriate identifier to indicate the type of question you have.
>
>Note: Conduct on the mailing list is subject to the Antitrust guidelines found at http://www.mpegif.org/public/documents/vault/mp-out-30042-Antitrust.php
>
>  
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20041018/d52c28e8/attachment.html