[M4IF Technotes] MPEG 4 Timing Questions
Mariano Padilla
mpadilla sdi-media-usa.com
Tue Oct 23 16:47:54 EDT 2001
Ladies, Gentlemen,
We are currently working along with SMPTE, Society of Motion Pictures and
Television Engineers, on the D-Cinema Subtitle Specs where MPEG-4 is being
considered for the video format. Our concerns listed below are crucial,
however now I have a couple more specific questions that I hope some of you
may know the answer to.
1) Is the timing model of MPEG-4 specific to the normal timing model
of other video formats? For example, if in an AVI we have 30 FPS
we expect the timing model to be 30 frames per second and so on.
I can accurately synchronize on frame 10 of the 25th second of the
23rd minute of the movie.
2) If I have an Mpeg-4 video file and a separate PNG or TIFF format
graphic file with yet another separate text navigation file that gives
me time cues down to the frame, i.e., '00:23:25.10', will I be
able to synchronize the pictures to the video with frame accuracy?
This is crucial for the display of subtitles at the right time.
3) If the MPEG-4 time model is not accurate to the frame, is there
any alternative to synchronize to the frame, or will I have to
synchronize to the nearest second?
Lastly, the jerky playback of MPEG-4 files is for example, I want to play a
movie at 1/4 the speed with the sound mimicking the sound of a tape at 1/4
the speed, where the sound actually sounds a couple of octaves lower, but it
is understandable. This method is commonly known as "scrubbing audio", which
works good on forward playback of the MPEG4 movie. But during the process of
reversing playback, we have problems of "jerkiness", where the frames need
to be "de-coded" or "re-render" to actually just display the previous frame.
For example, I stop at frame 10, and want to go back to frame 9, if frame 9
is not a reference frame, the video playback visibly has to go back to the
key reference frame and "re-render" all the other frames in between the key
reference frame and frame 9. This is my main concern for synchronization. On
a test we've performed, we've noticed that on the long run, subtitles that
we attempted to synchronize 23 minutes into the movie, seem to be off by a
second or so.
Thanks you all for your feedback, and I apologize for the length of the
e-mail.
Mariano J. Padilla
SDI Media USA
Director of IT/R&D
Tel: 323-602-5406
Fax: 323-602-5450
mpadilla sdi-media-usa.com
www.sdi-media-usa.com
-----Original Message-----
From: Mariano Padilla
Sent: Fri, October 19, 2023 9:40 AM
To: 'technotes lists.m4if.org'
Cc: Scott Rose
Subject: RE: MPEG 4
Again, I thank you all for your comments. Hence, if someone on the
list can answer my below questions.
Thank you much. If you do have the time to elaborate on the strict
Timestamps. Our concern is that when we provide for an MPEG-4 video time
specific "Subtitle Stream" which is controlled by a navigation file
containing time stamps for cue in and cue out, the temporal difference from
an "outside" graphic, in this case the subtitle superimposed on the Mpeg-4
video, can it create the same effect as the DVD MPEG-2 standard? With DVD
there is a variance of approximately 4 to 10 frames or more in some
instances. In NTSC, being 30 fps, it's a bit difficult to notice, but on PAL
at 25 fps it is a bit more noticeable. We've been experimenting also with
MPEG-4, and since it is not as accurate as MPEG-1 I-Frame, we notice that a)
when going back it does not stop on the exact frame, usually re-renders the
frames and you can visually see the re-rendering of frames. b) slow
playback, with sound, is extremely jerky and looses track where it is, hence
it needs to re-render the frames. Are these behaviors part of the internal
time correction as well as just the "de-compression" of the frames? Would
these behaviors affect time cueing of non-internal MPEG-4 objects? For
example, I have a graphic that is to be superimposed on a video at exactly
00:23:20.10, would the 10th frame be a reference frame or would it have to
"de-compress" and generate from reference frames and how would this affect
the external temporal synch? Your e-mail below says the MPEG-4 timestamps
can be as accurate as I wish, in this case I would need it to be down to the
frame, is this possible? Would I need to create the subtitle stream of
external tiffs as MPEG-4 BIFS in order to achieve the frame accurate sync?
Mariano J. Padilla
Director of IT/R&D
SDI Media USA
T 323-602-5406
F 323-602-5450
www.sdi-media-usa.com
-----Original Message-----
From: Rob Koenen [mailto:rkoenen intertrust.com]
Sent: Fri, October 19, 2023 9:34 AM
To: 'Mariano Padilla'; 'Michelle Y Kim'
Cc: Rob Koenen; Scott Rose
Subject: RE: MPEG 4
I second Michelle's proposal to send to the list.
However, oit looks to me as if your issue is with the
implementation you are using, not with the standard.
Jerky playback is not something that is built into MPEG-4.
Rob
-----Original Message-----
From: Mariano Padilla
[mailto:mpadilla sdi-media-usa.com]
Sent: Friday, October 19, 2023 9:25 AM
To: 'Michelle Y Kim'
Cc: Rob Koenen; Scott Rose
Subject: RE: MPEG 4
Thank you much. If you do have the time to elaborate
on the strict Timestamps. Our concern is that when we provide for an MPEG-4
video time specific "Subtitle Stream" which is controlled by a navigation
file containing time stamps for cue in and cue out, the temporal difference
from an "outside" graphic, in this case the subtitle superimposed on the
Mpeg-4 video, can it create the same effect as the DVD MPEG-2 standard? With
DVD there is a variance of approximately 4 to 10 frames or more in some
instances. In NTSC, being 30 fps, it's a bit difficult to notice, but on PAL
at 25 fps it is a bit more noticeable. We've been experimenting also with
MPEG-4, and since it is not as accurate as MPEG-1 I-Frame, we notice that a)
when going back it does not stop on the exact frame, usually re-renders the
frames and you can visually see the re-rendering of frames. b) slow
playback, with sound, is extremely jerky and looses track where it is, hence
it needs to re-render the frames. Are these behaviors part of the internal
time correction as well as just the "de-compression" of the frames? Would
these behaviors affect time cueing of non-internal MPEG-4 objects? For
example, I have a graphic that is to be superimposed on a video at exactly
00:23:20.10, would the 10th frame be a reference frame or would it have to
"de-compress" and generate from reference frames and how would this affect
the external temporal synch? Your e-mail below says the MPEG-4 timestamps
can be as accurate as I wish, in this case I would need it to be down to the
frame, is this possible? Would I need to create the subtitle stream of
external tiffs as MPEG-4 BIFS in order to achieve the frame accurate sync?
I thank you all for your comments.
Mariano J. Padilla
Director of IT/R&D
SDI Media USA
T 323-602-5406
F 323-602-5450
www.sdi-media-usa.com
-----Original Message-----
From: Michelle Y Kim [<mailto:mykim us.ibm.com>]
Sent: Fri, October 19, 2023 9:21 AM
To: 'Mariano Padilla'
Cc: Rob Koenen; Scott Rose
Subject: RE: MPEG 4
Mariano,
Since I don't know what kind of problems you're
trying to solve, I can only
offer you the following very general response, and
will have to echo Rob
and ask you to bring this to the reflector.
If you want to achieve frame-level synchronization
of subtitles and video,
and if the video delivery is reliable, then you can
achieve synchronization
without FlexTime. You can simply align video
timestamps to those of BIFS
that contain subtitles. Note that presently MPEG-4
does not support text
streams. Text in MPEG-4 is delivered in BIFS as
timed insertions and
deletions, where the timestamps can be as accurate
as you wish. The level
of synchornization you can from MPEG-4 is at a finer
granularity than SMIL
if you're coming from the SMIL experience.
But if the video delivery is non-deterministic, then
FlexTime is a technic
you can use, which I can help you with, as I
understand the nature of your
question better.
Regards,
Michelle
Michelle Y. Kim, Ph. D.
Manager, Composite Media Technologies
IBM T. J. Watson Research Center
30 Saw Mill River Road, Hawthorne NY, 10532
mykim us.ibm.com
phone: 914-784-7709, fax: 914-784-7455 (tie:
863-7709)
Rob Koenen <rkoenen intertrust.com> on 10/18/2001
12:43:01 PM
To: "'Mariano Padilla'"
<mpadilla sdi-media-usa.com>, Rob Koenen
<rkoenen intertrust.com>
cc: Scott Rose <srose sdi-media-usa.com>, Michelle Y
Kim/Watson/IBM IBMUS
Subject: RE: MPEG 4
Again I suggest that you ask these questions on the
list.
But I copy Michelle Kim of IBM - there is no better
expert
on flextime to be found.
Rob
-----Original Message-----
From: Mariano Padilla
[<mailto:mpadilla sdi-media-usa.com>]
Sent: Thursday, October 18, 2023 9:38 AM
To: 'Rob Koenen'
Cc: Scott Rose
Subject: RE: MPEG 4
Rob,
Thank you. I will subscribe. Specifically we are
wondering the
synchronization of subtitles on MPEG 4 stream. In
the overview, written
by you, in section 8.3.2, 'The FlexTime model allows
the content author to
express synchronization among MPEG-4 objects with
streams or stream
segments, by assigning temporal relationships among
them.' then the
following questions:
1) Are subtitle streams to be synchronized
considered MPEG objects?
2) If the subtitle streams to be synchronized are
MPEG objects, do
they have to behave according to the SMIL text
guidelines in section 4.2.2
of your overview?
3) If 2 above is true, then does SMIL used in MPEG4
allow milisecond
or frame-accurate synchronization with the video?
4) If there is no text/SMIL synchronization of
multinational
subtitles, how can synchronization of external
images be achieved? Is it
similar to the MPEG2/DVD model?
Thanks again,
Mariano
-----Original Message-----
From: Rob Koenen [<mailto:rkoenen intertrust.com>]
Sent: Wed, October 17, 2023 5:17 PM
To: 'Mariano Padilla'; Rob Koenen
Cc: Scott Rose
Subject: RE: MPEG 4
Mariano,
you can ask me but I am not the best source for
technical
details. I could forward your questions to experts.
I would suggest,
however, that you subscribe yourself to the
technotes mailing list
of the MPEG-4 Industry Forum (this is a public list)
and ask your
question there. Then I will monitor that you indeed
do get an answer.
See <http://www.m4if.org> and in particular
<http://www.m4if.org/public/publiclistreg.html>
Hope this helps,
Rob
-----Original Message-----
From: Mariano Padilla
[<mailto:mpadilla sdi-media-usa.com>]
Sent: Wednesday, October 17, 2023 5:11 PM
To: 'rob intertrust.com'
Cc: Scott Rose
Subject: MPEG 4
Rob,
I got your e-mail address form the MPEG-4 standards
article and I
have a couple of questions, specifically regarding
section 8.3 'FlexTime'
and synchronization of subtitles. Can I address
these questions to you,
or if you have someone else that can answer them for
me? Please advise.
We at SDI Media USA specialize in Theatrical, DVD,
Home Video and
broadcast subtitles and captions for over 40
languages. Naturally we are
looking at how subtitles are synchronized in MPEG-4.
I thank you in advanced for the time spent.
Mariano J. Padilla
Director of IT/R&D
SDI Media USA
T 323-602-5406
F 323-602-5450
www.sdi-media-usa.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20011023/f5c824fe/attachment.html
More information about the Mp4-tech
mailing list