[M4IF Technotes] Common ways to evaluate encoded video quality?

[Christophe Vermeulen]

Regarding that question, and at the risk of beating a dead horse,
I would dare to elaborate a bit.
Sure, you are 100% right, but the fact itself that people keep asking
shows there is a need for some help in this field. Maybe some specific
FAQ or LFAQ (less frequently asked questions) could help.
Could you all consider the following as a very humble, incomplete and 
maybe (hopefully partially) erroneous start, and add/complete/correct it ?
- Is there an way to evaluate objectively the quality of a bitstream ?
  Although many algorithms exist (insert link list), and several commercial
  applications of these algorithms have been provided for years, no single 
  method has ever proven better than subjective testiing from experts.
- what is exactly PSNR ? 
  PSNR is the most widely accepted objective criteria to quantify the
  imperfection of a coding algorithm (experts talk of HRC) by comparing
  the coding result with the original material. Note that this assumes 
  that the original material is "perfect".
  The formula for PSNR is PSNR=10*log10(255^2/MSE) with MSE being the
  mean square difference of all the luminance values (is this correct?
  I don't know why this is a peak) 
  One of the interests of PSNR is that it "speaks" to humans, since
  it gives a result in dB, like analog SNR ratios, with more being 
  better, and an easy to understand/remember range interpretation :
  10-20 is poor quality, while 30-40 is acceptable and 50-60 is
  already very good (for broadcast type of applications at least)
  The use of an inverted logarithmic scale also allows to give 
  more invariance with respect to different sequences as compared
  to using MSE. (If my calculations are correct)
  roughly PSNR = 25,30,35,40 correspond to MSE=200, 65, 20 and 6,5 ?
- What is a HRC ? A Hypothetical Reference Circuit is a reference
  video material that can be used to evaluate coding systems. It is
  the result of coding (and decoding) some source material (SRC?)  
  For example, one can create an HRC by adding noise, coding using
  an algorithm (e.g. MPEG-4 ASP) and its parameters (256 kbps CBR,
  CIF, ...), creating bit errors, etc.
- It seem that ANSI recently adopted the VQM software as the way to 
  test objective quality. Does somebody have some experience with that ?
  (To be answered)
- Where could I find more information ? See list below, add 
  http://live.ece.utexas.edu/research/Quality/frqa.htm
Comments/corrections welcome. I hope I did not derive too much from the reality 
and provide the proverbial 2 cents of contribution ...
CVE.
"Rob Koenen (M4IF)" wrote:
> 
> This questions has been asked several times before. Please see
> http://lists.m4if.org/pipermail/technotes/2002-August/001081.html
> and follow-up, and
> http://lists.m4if.org/pipermail/technotes/2002-November/001471.html
> and follow-up in addition t the links below.
> 
> Nothing really beats exper viewing, even with few experts.
> 
> More in general, we now have the great feature on our website that
> allows you to search the site, including the archives, for questions
> that may have been asked before!
> 
> http://www.m4if.org/index_search.php
> 
> Best,
> Rob
> > -----Original Message-----
> > From: Olivier Amato [mailto:oamato   wanadoo.fr]
> > Sent: Friday, June 20, 2023 01:33
> > To: Allen H.Y. Chin; M4IF Technotes
> > Subject: Re: [M4IF Technotes] Common ways to evaluate encoded
> > video quality?
> >
> >
> > Hi Allen,
> >
> > > So, I'm looking for other metrics to have a more objective
> > view. Any
> > > suggestions?
> >
> > You could maybe take a look at these links :
> > -
> > http://www.sarnoff.com/products_services/video_vision/jndmetri
> x/index.asp
> - http://www.genista.com/Products/VideoPQoS.htm
> - http://www.opticom.de/
> - http://www.vqual.biz/
> - http://www.its.bldrdoc.gov/n3/video/vqmsoftware.htm
> - http://www.vqeg.org/
> 
> Regards,
> 
> Olivier
>