aboutsummaryrefslogtreecommitdiffstats
path: root/libavcodec/x86/rv40dsp_init.c
diff options
context:
space:
mode:
authorYang Wang <yang.y.wang@intel.com>2012-07-24 00:51:10 +0200
committerDerek Buitenhuis <derek.buitenhuis@gmail.com>2012-07-25 14:22:18 -0400
commit845e92fd6abf749a960354becdc5a9bc498f1f96 (patch)
tree401c7d0bce5593d46b2f9434fb506c10384be490 /libavcodec/x86/rv40dsp_init.c
parent6d1c9945dd0a539aa9017697ac7572fde3dc5b83 (diff)
downloadffmpeg-845e92fd6abf749a960354becdc5a9bc498f1f96.tar.gz
dsputil_mmx: fix incorrect assembly code
In ff_put_pixels_clamped_mmx(), there are two assembly code blocks. In the first block (in the unrolled loop), the instructions "movq 8%3, %%mm1 \n\t", and so forth, have problems. From above instruction, it is clear what the programmer wants: a load from p + 8. But this assembly code doesn’t guarantee that. It only works if the compiler puts p in a register to produce an instruction like this: "movq 8(%edi), %mm1". During compiler optimization, it is possible that the compiler will be able to constant propagate into p. Suppose p = &x[10000]. Then operand 3 can become 10000(%edi), where %edi holds &x. And the instruction becomes "movq 810000(%edx)". That is, it will stride by 810000 instead of 8. This will cause a segmentation fault. This error was fixed in the second block of the assembly code, but not in the unrolled loop. How to reproduce: This error is exposed when we build using Intel C++ Compiler, with IPO+PGO optimization enabled. Crashed when decoding an MJPEG video. Signed-off-by: Michael Niedermayer <michaelni@gmx.at> Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Diffstat (limited to 'libavcodec/x86/rv40dsp_init.c')
0 files changed, 0 insertions, 0 deletions