diff options
| author | Justin Ruggles <justin.ruggles@gmail.com> | 2012-11-07 14:48:28 -0500 |
|---|---|---|
| committer | Reinhard Tartler <siretart@tauware.de> | 2013-01-02 20:44:31 +0100 |
| commit | 0ba0e3195517e423c35b232384e380b975b88b4e (patch) | |
| tree | 594773c324ea0e0c2706266cd330f4ac99c1fede | |
| parent | abe345251a1c7d56a8afc140d8002f4fe2d2a42b (diff) | |
| download | ffmpeg-0ba0e3195517e423c35b232384e380b975b88b4e.tar.gz | |
flacenc: ensure the order is within the min/max range in LPC order search
This fixes use of uninitialized values when the FLAC encoder uses the
2-level, 4-level, and 8-level search methods. Fixes failure of the
fate-flac-24-comp-8 test when run using valgrind.
(cherry picked from commit 3a2731cbd31d0c5681ddbc7c78edd5c53c4d0032)
Conflicts:
libavcodec/flacenc.c
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
| -rw-r--r-- | libavcodec/flacenc.c | 8 |
1 files changed, 5 insertions, 3 deletions
diff --git a/libavcodec/flacenc.c b/libavcodec/flacenc.c index 94e381d17e..2bd47883e0 100644 --- a/libavcodec/flacenc.c +++ b/libavcodec/flacenc.c @@ -915,14 +915,16 @@ static int encode_residual_ch(FlacEncodeContext *s, int ch) omethod == ORDER_METHOD_8LEVEL) { int levels = 1 << omethod; uint32_t bits[1 << ORDER_METHOD_8LEVEL]; - int order; + int order = -1; int opt_index = levels-1; opt_order = max_order-1; bits[opt_index] = UINT32_MAX; for (i = levels-1; i >= 0; i--) { + int last_order = order; order = min_order + (((max_order-min_order+1) * (i+1)) / levels)-1; - if (order < 0) - order = 0; + order = av_clip(order, min_order - 1, max_order - 1); + if (order == last_order) + continue; encode_residual_lpc(res, smp, n, order+1, coefs[order], shift[order]); bits[i] = find_subframe_rice_params(s, sub, order+1); if (bits[i] < bits[opt_index]) { |