| ??? 10/05/07 05:01 Modified: 10/05/07 05:31 Read: times Msg Score: +1 +1 Good Answer/Helpful |
#145441 - Correct precision Responding to: ???'s previous message |
Aha, sorry,
The actual precision was a little better than the last post. I mistakenly posted the precision before I added the rounding to the last log10 calculation. The correct precision is, (-1.88293/256, +0.12735/256) = about (-0.007, +0.0005) When 20 log(x) is applied for dB calculation, the precision is, (-0.15, +0.01) dB The algorithm is easily extended to 24bit or 32bit input number. Even in these cases, the same 8bit decimal table will provide similar precision, because of the nature of the algorithm. Tsuneo PS. Please add this line at the entry of subroutine log10 in the implementation.
if ( x == 0 ) return 0; // avoid infinite loop
|



