avfilter/vf_morpho: Fix invalid frees on error

The current code used a pointer to an array (of arrays) that
is offset relative to the start of the actually allocated buffer.
Yet offsetting the pointer is only done on success, whereas the
freeing code believes it to have happened even on error.
So if any of the subarrays (or the subarrays' subarrays) can't
be successfully allocated, one gets a bad free in free_lut().

Furthermore, said offsetting is only permissible in case the
offsetted pointer points in the allocated buffer (here: in case
the LUT's min_r is <= 0), as pointer arithmetic is undefined
in case it exceeds the allocated object.

Moreover, in case one of the subarrays couldn't be allocated,
the code nevertheless tried to free the subarray's subarrays;
and in case one of the subarray's subarrays could not be allocated
successfully, there will be an invalid free, too, because the
pointers for the subarrays' subarrays are also offset compared
to the base pointer.

This commit fixes all of this, by using the actually allocated
pointer for freeing and by adding appropriate checks before
freeing the subarrays. The former also allows to distinguish
the cases in which the lut is currently only half-allocated due to
an error in an earlier allocation attempt from the success case.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>

0 files changed, 0 insertions, 0 deletions