Discussion about this post

User's avatar
Omid's avatar

I might have misunderstood the lossy compression function, but shouldn’t it be smth like this?

uint32_t compress(uint32_t d)

{

return ((d & 0x80000000) >> 11) | ((d & 0xFFFE0) >> 5);

}

I mean for the 1:5:12 format, the sign bit is shifted by 11 to account for the unused bits so I'd expect same for 1:5:10 and also 0xFFFD0 doesn’t seem to discard last 5 bits, so maybe it should be 0xFFFE0? or am I missing something?

Expand full comment

No posts