avformat/matroskadec: Fix buffer overflow when demuxing RealAudio 28.8

RealAudio 28.8 (like other RealAudio codecs) uses a special demuxing
mode in which the data of the existing Matroska Blocks is not simply
forwarded as-is. Instead data from several Blocks is recombined
together to output several packets. The parameters governing this
process are parsed from the CodecPrivate: Coded framesize (cfs), frame
size (w) and sub_packet_h (h).

During demuxing, h/2 pieces of data of size cfs each are read from every
Matroska (Simple)Block and put at offset m * 2 * w + n * cfs of a buffer
of size h * w, where m ranges from 0 to h/2 - 1 for each Block while n
is initially zero and incremented after a Block has been parsed until it
is h, at which poin the assembled packets are output and n reset.

The highest offset is given by (h/2 - 1) * 2 * w + (h - 1) * cfs + cfs
while the destination buffer's size is given by h * w. For even h, this
leads to a buffer overflow (and potential segfault) if h * cfs > 2 * w;
for odd h, the condition is h * cfs > 3 * w.

This commit adds a check to rule this out.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

0 files changed, 0 insertions, 0 deletions