[Mp4-tech] [video] performance comparison between open loopandclosed loop motion estimation

[배태면]

Dear Alexis,
Your opinion is very helpful. (Gery, B. G. Kim, and Andrew’s opinions are also very helpful. I appreciate it.)       
What I’m planning is similar to a) case.
I tried several experiments. And the results show that the performance gap between “open loop ME” and “closed loop ME” depends on the contents. 
Before deciding the adoption of “open loop ME”, I want to know any reasonable analysis related with this issue. If “open loop ME” is used, motion estimation could be performed independent of the other module in the encoder.  
Sincerely,
Tae Meon Bae
  _____  
From: mp4-tech-bounces lists.mpegif.org [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of Tourapis, Alexis
Sent: Saturday, January 26, 2024 4:06 AM
To: Gary Sullivan; 배태면; mp4-tech lists.mpegif.org
Subject: RE: [Mp4-tech] [video] performance comparison between open loopandclosed loop motion estimation
Dear Tae Meon Bae,
At very high bitrates the performance may be close enough but that also depends on the content, and what do you consider as high. Performance behavior would also be affected by how and where you consider original pictures instead of reconstructed ones. Furthermore, performance may be affected considerably from the presence (or not) of Rate Distortion Optimization within your codec (mode decision and motion estimation) and the motion estimation scheme you are using. The presence of RDO (even with approximations in terms of the bit cost) or the use of a "true motion" estimation scheme (i.e. several hierarchical schemes, EPZS, many phase correlation schemes etc) in contrast to a dumb algorithm with no RD considerations would yield much better performance. The coding structure you are using would also affect your results (i.e. presence of B coded pictures and distances between references). Unfortunately I cannot provide you with specific results on this and I am not aware of any published results by anyone comparing performance using AVC (there might be for MPEG-4 or MPEG-2). Still here are some tradeoffs or considerations you could perform or evaluate for your application which may end up improving performance rather considerably :
a) In some schemes instead of using original pictures one can apply a basic "filtering/quantization" process which basically tries to emulate the behavior of the full encoder but at a lower cost. Assuming that the reason why you care about this is to maybe parallelize your encoding process especially if it is hardware, this may somehow account for the mismatches between original and reconstructed images.
b) in a hierarchical ME scheme, one could consider original pictures as references for higher levels, and then only consider the reconstructed at the last refinement. 
c) in an extension of hierarchical ME, some schemes consider original pictures for integer ME, and consider reconstructed images for the subpixel refinement. The idea here is that one could compute all integer mvs at once from one core engine, and when coding a MB the subpixel refinements are computed at that stage.
d) a further extension of the above would be to push the reconstructed consideration to the mode decision level. This is especially important for not only inter but also intra decisions. Different tradeoffs could also be considered in this context (i.e. inter decisions are performed using originals but intra vs inter are based on reconstructed).
I hope this helps.
Best regards,
Alexis
  _____  
From: mp4-tech-bounces lists.mpegif.org [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of Gary Sullivan
Sent: Friday, January 25, 2024 9:42 AM
To: 腂黠 mp4-tech lists.mpegif.org
Subject: RE: [Mp4-tech] [video] performance comparison between open loop andclosed loop motion estimation
Tae Meon Bae et al,
I see.  I thought that might be what you meant, but I wasn't sure.  I don't recall any publications or standardization documents on that topic.  One important thing to keep in mind is that there may be a significant difference between "objective" (e.g., PSNR) and subjective quality relationships in this case.  In some cases the "open loop" scheme may actually improve subjective quality, as it may result in motion vectors that more closely and coherently model the true motion of the objects in the scene.
Best Regards,
Gary Sullivan
  _____  
From: 腂頛mailto:heartles icu.ac.kr] 
Sent: Thursday, January 24, 2024 8:32 PM
To: Gary Sullivan; mp4-tech lists.mpegif.org
Subject: RE: [Mp4-tech] [video] performance comparison between open loop and closed loop motion estimation
Dear Gary,
I used the open loop motion estimation as motion estimation using original picture as a reference frame.
And I used the closed loop motion estimation to represent the motion estimation using reconstructed picture as reference frame.
I though that it is common concept. If it is not so, Im sorry for the misunderstanding. 
Sincerely,
Tae Meon Bae
  _____  
From: Gary Sullivan [mailto:garysull windows.microsoft.com] 
Sent: Friday, January 25, 2024 3:48 AM
To: 腂鼯SPAN>; mp4-tech lists.mpegif.org
Subject: RE: [Mp4-tech] performance comparison between open loop and closed loop motion estimation
Tae Meon Bae,
What do you mean by open loop motion estimation?
Best Regards,
Gary Sullivan
  _____  
From: mp4-tech-bounces lists.mpegif.org [mailto:mp4-tech-bounces lists.mpegif.org] On Behalf Of ???
Sent: Thursday, January 24, 2024 3:16 AM
To: mp4-tech lists.mpegif.org
Subject: [Mp4-tech] performance comparison between open loop and closed loop motion estimation
  <http://mail.icu.ac.kr:80/nara/servlet/webmail.WebMailReConfServ/614641> 
Dear experts,
I'm optimizing h.264 encoder, and I have a question about motion estimation.
Generally open loop motion estimation shows lower performance than closed motion estimation. Is there any JVT document or paper related with this issue? I guess that the performance gap may be small if the target bit rate is high enough. Is this assumption right? 
Thanks in advance.
Sincerely,
Tae Meon Bae
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20080128/105e1e98/attachment-0001.html