diff options
| author | Michael Niedermayer <michael@niedermayer.cc> | 2017-11-13 20:47:48 +0100 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2017-12-07 23:38:06 +0100 |
| commit | 6ccf19198b360cfc3fe5cd274948cfde2fe305e0 (patch) | |
| tree | cb0c3425add889dbb554c4be29e3f74d203c14c7 /libavcodec | |
| parent | 44fb1201123e554ee07a58bca49d7250c5561fa5 (diff) | |
| download | ffmpeg-6ccf19198b360cfc3fe5cd274948cfde2fe305e0.tar.gz | |
avcodec/x86/mpegvideodsp: Fix signedness bug in need_emu
Fixes: out of array read
Fixes: 3516/attachment-311488.dat
Found-by: Insu Yun, Georgia Tech.
Tested-by: wuninsu@gmail.com
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 58cf31cee7a456057f337b3102a03206d833d5e8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Diffstat (limited to 'libavcodec')
| -rw-r--r-- | libavcodec/x86/mpegvideodsp.c | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/libavcodec/x86/mpegvideodsp.c b/libavcodec/x86/mpegvideodsp.c index e0498f3849..6009b64e07 100644 --- a/libavcodec/x86/mpegvideodsp.c +++ b/libavcodec/x86/mpegvideodsp.c @@ -52,8 +52,9 @@ static void gmc_mmx(uint8_t *dst, uint8_t *src, const int dyh = (dyy - (1 << (16 + shift))) * (h - 1); const int dxh = dxy * (h - 1); const int dyw = dyx * (w - 1); - int need_emu = (unsigned) ix >= width - w || - (unsigned) iy >= height - h; + int need_emu = (unsigned) ix >= width - w || width < w || + (unsigned) iy >= height - h || height< h + ; if ( // non-constant fullpel offset (3% of blocks) ((ox ^ (ox + dxw)) | (ox ^ (ox + dxh)) | (ox ^ (ox + dxw + dxh)) | |