[Mp4-tech] [video] Error concealment - unreferenced memory error

[Bharat Soni]

Hi Steve,
I really didn't get your problem completely. May be some portion of
your explaination is missing here.
For inserting the errors at known places in the stream, I know a
company (www.veneratechnologies.com) that is working on a tool (may be
they have it ready) that allows you to insert errors in the stream at
known places. So it helps in testing the decoder behaviour under
erroneous conditions.
regards,
Bharat
On Sat, 12 Feb 2024 18:13:59 -0800, Adarsh Golikeri <golikeri gmail.com> wrote:
> hello steve,
> 
> the piece of code i gave you operates only on i-frames. so if the
> decoder crashes on frame 18, which was not an i-frame, then that's a
> clue that the problem lies somehere else.
> 
> here's another suggestion: have only i-frames in your stream. then
> check the decoder output. you should see lost macroblocks in the
> output (they appear blurred because of spatial error concealment). if
> this works without crashing, then you can turn your attention to
> p-frames. hope that helps,
> 
> adarsh
> 
> On Sat, 12 Feb 2024 14:28:23 +0000, Steve <S.T.C.Beesley lboro.ac.uk> wrote:
> > Hi Adarsh,
> >
> > Thank you for your quick and very helpful reply, to know that I was heading
> > in the correct direction is great news!
> >
> > I moved my code to decode_one_slice as suggested (I inserted it after
> > read_flag = read_one_macroblock(img,inp); and before
> > decode_one_macroblock(img,inp);), unfortunately I was still getting the
> > same error on the same macroblock of the same frame.
> >
> > I had IntraPeriod=0 set in encoder.cfg, frame 18 was a P frame. As your code
> > only worked on I-Frames I incremented the number of I-Frames in the encoded
> > file and even with just one additional I-Frame at frame 10 the error
> > disappeared.
> >
> > Personally I would not have thought it unreasonable to have 18+ frames
> > between I frames, incrementing the number is acceptable for my work for the
> > time being however does not really solve why the error was occurring in the
> > first place.
> >
> > I will keep looking into this problem, however if your good self or any
> > other experts can shed any further light on the problem then it would be
> > very much appreciated.
> >
> > Thanks again for your help,
> >
> > Steve Beesley
> >
> >
> > Quoting Adarsh Golikeri <golikeri gmail.com>:
> >
> > > hi steve,
> > >
> > > i've tried this same method before. the concept you are using is
> > > correct (setting the ei flag to 1) but the place you are doing it is
> > > wrong. instead of in function exit_picture, try doing it in the
> > > function decode_one_slice. this was the code i used :
> > >
> > >     // lose some MBs in i-frames
> > >     loss = rand()%100;
> > >       if (img->number>0 && img->type == 2 && img->current_mb_nr >= 11 &&
> > > img->current_mb_nr <=22)
> > >        {
> > >          img->mb_data[img->current_mb_nr].ei_flag = 1;
> > >        fprintf(stdout,"\nLosing MB %d of Picture
> > > %d",img->current_mb_nr,img->number);
> > >          }
> > >     // lose some MBs in i-frames
> > >
> > > hope that helps,
> > >
> > > adarsh
> > >
> > >
> > > On Mon,  7 Feb 2024 16:09:30 +0000, Steve <S.T.C.Beesley lboro.ac.uk>
> > > wrote:
> > > > Hi experts,
> > > >
> > > > I have been working with JM93 to look at error concealment,
> > > specifically the
> > > > way that the decoder tries to hide any missing data. In order to
> > > achieve
> > > > this I have been adding something similar to:
> > > >   img->mb_data[207].ei_flag = 1;
> > > > in image.c after line 1206:
> > > >   ...
> > > >   //! mark the start of the first segment
> > > >   if (!dec_picture->MbaffFrameFlag)
> > > >   {
> > > >     ercStartSegment(0, ercSegment, 0 , erc_errorVar);  // <-- 1206
> > > >   ...
> > > >
> > > > For the most part this method seems to work extremely well however
> > > under
> > > > some circumstances the decoder is throwing an unreferenced memory error
> > > due
> > > > to the Motion Vector reference slice being set to -1. One such example
> > > of
> > > > this is in a cif version of mobile.yuv when ProfileIDC = 100 (High) is
> > > used
> > > > at macroblock 207 (w=9, h=9), frame 18 (all frames up to 15 in the
> > > decode
> > > > sequence work ok).
> > > >
> > > > >From what I can tell there are only two lines that ever set the mv[2]
> > > > variable to anything other than 0 and these are again in image.c:
> > > >   pRegion->mv[2] = (dec_picture->ref_idx[idx][ii][jj]);
> > > > or
> > > >   pRegion->mv[2] = dec_picture->ref_idx[LIST_0][ii][jj];
> > > > depending on the type of slice (non-B or B).
> > > >
> > > > I'm wondering if anybody can shed any light onto why this variable
> > > would
> > > > ever be set to -1, judging by the rest of the EC code it looks like it
> > > > assumes that this variable is always going to contain a valid number as
> > > no
> > > > checking appears to be done. One working theory I currently have is
> > > that it
> > > > could be referencing a slice that has either not yet been decoded or is
> > > no
> > > > longer in the decode buffer. Also can anybody suggest where to find the
> > > > information to detect if the returned value is valid or not and what
> > > action
> > > > should be taken in the cases where it is invalid?
> > > >
> > > > If it is the case that I'm barking up the wrong tree here (which is
> > > more
> > > > than possible) then can anyone suggest a better method of forcefully
> > > adding
> > > > errors at the decoder?
> > > >
> > > > Many thanks in advance,
> > > >
> > > > Steve Beesley
> > > > _______________________________________________
> > > > NOTE: Please use clear subject lines for your posts. Include [audio,
> > > [video], [systems], [general] or another apppropriate identifier to
> > > indicate the type of question you have.
> > > >
> > > > Note: Conduct on the mailing list is subject to the Antitrust
> > > guidelines found at
> > > http://www.mpegif.org/public/documents/vault/mp-out-30042-Antitrust.php
> > > >
> > >
> >
> _______________________________________________
> NOTE: Please use clear subject lines for your posts. Include [audio, [video], [systems], [general] or another apppropriate identifier to indicate the type of question you have.
> 
> Note: Conduct on the mailing list is subject to the Antitrust guidelines found at http://www.mpegif.org/public/documents/vault/mp-out-30042-Antitrust.php
>