aboutsummaryrefslogtreecommitdiffstats
path: root/libpostproc
diff options
context:
space:
mode:
authorLynne <dev@lynne.ee>2021-01-09 20:41:25 +0100
committerLynne <dev@lynne.ee>2021-01-09 20:54:56 +0100
commit91e1625db15fe8853ceedca9eed14307aaa514c7 (patch)
tree4b81ac9c4098b35eea16df71a852aedd56e73197 /libpostproc
parent29993b2947a99806cf41dd58853af510b0ce152c (diff)
downloadffmpeg-91e1625db15fe8853ceedca9eed14307aaa514c7.tar.gz
lavu/tx: clip when converting table values to fixed-point
INT32_MAX (2147483647) isn't exactly representable by a floating point value, with the closest being 2147483648.0. So when rescaling a value of 1.0, this could overflow when casting the 64-bit value returned from lrintf() into 32 bits. Unfortunately the properties of integer overflows don't match up well with how a Fourier Transform operates. So clip the value before casting to a 32-bit int. Should be noted we don't have overflows with the table values we're currently using. However, converting a Kaiser-Bessel window function with a length of 256 and a parameter of 5.0 to fixed point did create overflows. So this is more of insurance to save debugging time in case something changes in the future. The macro is only used during init, so it being a little slower is not a problem.
Diffstat (limited to 'libpostproc')
0 files changed, 0 insertions, 0 deletions