[Mp4-tech] [video] performance comparison between open loop andclosed loop motion estimation

[Tourapis, Alexis]

Dear Tae Meon Bae,
At very high bitrates the performance may be close enough but that also depends on the content, and what do you consider as high. Performance behavior would also be affected by how and where you consider original pictures instead of reconstructed ones. Furthermore, performance may be affected considerably from the presence (or not) of Rate Distortion Optimization within your codec (mode decision and motion estimation) and the motion estimation scheme you are using. The presence of RDO (even with approximations in terms of the bit cost) or the use of a "true motion" estimation scheme (i.e. several hierarchical schemes, EPZS, many phase correlation schemes etc) in contrast to a dumb algorithm with no RD considerations would yield much better performance. The coding structure you are using would also affect your results (i.e. presence of B coded pictures and distances between references). Unfortunately I cannot provide you with specific results on this and I am not aware of any published results by anyone comparing performance using AVC (there might be for MPEG-4 or MPEG-2). Still here are some tradeoffs or considerations you could perform or evaluate for your application which may end up improving performance rather considerably :
a) In some schemes instead of using original pictures one can apply a basic "filtering/quantization" process which basically tries to emulate the behavior of the full encoder but at a lower cost. Assuming that the reason why you care about this is to maybe parallelize your encoding process especially if it is hardware, this may somehow account for the mismatches between original and reconstructed images.
b) in a hierarchical ME scheme, one could consider original pictures as references for higher levels, and then only consider the reconstructed at the last refinement. 
c) in an extension of hierarchical ME, some schemes consider original pictures for integer ME, and consider reconstructed images for the subpixel refinement. The idea here is that one could compute all integer mvs at once from one core engine, and when coding a MB the subpixel refinements are computed at that stage.
d) a further extension of the above would be to push the reconstructed consideration to the mode decision level. This is especially important for not only inter but also intra decisions. Different tradeoffs could also be considered in this context (i.e. inter decisions are performed using originals but intra vs inter are based on reconstructed).
I hope this helps.
Best regards,
Alexis
________________________________
From: mp4-tech-bounces lists.mpegif.org [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of Gary Sullivan
Sent: Friday, January 25, 2024 9:42 AM
To: 배ﾅﾂ면; mp4-tech lists.mpegif.org
Subject: RE: [Mp4-tech] [video] performance comparison between open loop andclosed loop motion estimation
Tae Meon Bae et al,
I see.  I thought that might be what you meant, but I wasn't sure.  I don't recall any publications or standardization documents on that topic.  One important thing to keep in mind is that there may be a significant difference between "objective" (e.g., PSNR) and subjective quality relationships in this case.  In some cases the "open loop" scheme may actually improve subjective quality, as it may result in motion vectors that more closely and coherently model the true motion of the objects in the scene.
Best Regards,
Gary Sullivan
________________________________
 From: 배ﾅﾂ면 [mailto:heartles icu.ac.kr] 
 Sent: Thursday, January 24, 2024 8:32 PM
 To: Gary Sullivan; mp4-tech lists.mpegif.org
 Subject: RE: [Mp4-tech] [video] performance comparison between open loop and closed loop motion estimation
 Dear Gary,
 I used the “open loop motion estimation” as motion estimation using original picture as a reference frame.
 And I used the “closed loop motion estimation” to represent the motion estimation using reconstructed picture as reference frame.
 I though that it is common concept. If it is not so, I’m sorry for the misunderstanding. 
 Sincerely,
 Tae Meon Bae
________________________________
 From: Gary Sullivan [mailto:garysull windows.microsoft.com] 
 Sent: Friday, January 25, 2024 3:48 AM
 To: 배ﾅﾂ면; mp4-tech lists.mpegif.org
 Subject: RE: [Mp4-tech] performance comparison between open loop and closed loop motion estimation
 Tae Meon Bae,
 What do you mean by open loop motion estimation?
 Best Regards,
 Gary Sullivan
________________________________
  From: mp4-tech-bounces lists.mpegif.org [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of ???
  Sent: Thursday, January 24, 2024 3:16 AM
  To: mp4-tech lists.mpegif.org
  Subject: [Mp4-tech] performance comparison between open loop and closed loop motion estimation
   <http://mail.icu.ac.kr:80/nara/servlet/webmail.WebMailReConfServ/614641> 
  Dear experts,
  I'm optimizing h.264 encoder, and I have a question about motion estimation.
  Generally open loop motion estimation shows lower performance than closed motion estimation. Is there any JVT document or paper related with this issue? I guess that the performance gap may be small if the target bit rate is high enough. Is this assumption right? 
  Thanks in advance.
  Sincerely,
  Tae Meon Bae
-----------------------------------------
This message (including any attachments) may contain confidential
information intended for a specific individual and purpose.  If you
are not the intended recipient, delete this message.  If you are
not the intended recipient, disclosing, copying, distributing, or
taking any action based on this message is strictly prohibited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20080125/156d8dc3/attachment-0001.html