Discussion about this post

User's avatar
Jeremiah's avatar

> When less than 10 fractional bits are stored, they will be stored in the low bits (in other words, packed to the right.)

Hmm, I think they're actually supposed to be left packed? Take the following example from the article:

S:0000000000000000:00000000000001A -> S: (0000000001A * 2^-14) / 1024

Let's consider 0:0000000000000000:000000000000011. If I understood fixed-point correctly, this represents the value 3 / 2^15. Okay, now applying the conversion pattern we get

(00000000011 * 2^-14) / 1024 = (3 * 2^-14) / 2^10 = 3 / (2^14 * 2^10) = 3 / 2^24

But if I left pack the mantissa bits, I get the expected result. The conversion pattern would be

S:0000000000000000:00000000000001A -> S: (1A000000000 * 2^-14) / 1024

Applying this to the example, we get

3 / 2^15

= 0:0000000000000000:000000000000011

= (11000000000 * 2^-14) / 1024

= ((2^10 + 2^9) * 2^-14) / 2^10

= (2^9 * (2^1 + 1)) / (2^14 + 2^10)

= (2^9 * 3) / 2^24

= 3 / 2^15

Expand full comment
Bas Zavyalov's avatar

After the sentence - "measuring runtime performance where values are more likely to be in the range of microseconds...", then we've got a table:

FROM | TO | APPROX

| 0x00000005 | 0x0801 | 0.000122 |

How did we get it?

0x00000005 => 0b 000 0000 0000 0101 -- that is supposed to be a fraction part?

Any hints?

Expand full comment
1 more comment...

No posts