Texture filtering and addressing question

[Zeross]

I was under the assumption that FP16 was enough for addressing 65 536 texels i.e a 256x256 texture or a 65 536x1 texture but I was told that this is only true when no filtering is enabled.

So I was wondering how does texture addressing work when bilinear filtering is enabled ? How many bits of precision are "lost" to get the extra samples needed for bilinear ?

 

[Mintmaster]

First, you need to remember that in FP each possible value is not equally spaced. The mantissa is 11 bits (and i believe there's one implied bit), so if you address 4096 texels (in any direction, not total texels like you thought) without filtering, each texel should come out pretty much fine, although small errors could make each texel come out differently sized. Remember that this includes repeating, e.g. a 256x256 texture repeated 16 times, or a 4096x4096 texture with no repetition

For filtering, it would make sense to consider the worst case scenario. Lets say one texel is black, and one next to it is white. Given that 32-bit color is generally enough to make a gradient without banding, you need 8 bits more precision in your texture coordinate data. This is only really true, however, if you are close enough to a texture that two texels are 256 pixels apart on the screen, which is some pretty serious magnification.

In the end there is no definitive answer, just slowly degrading picture quality with lower precision. To further complicate matters, most hardware probably does filtering calculations at higher precision, even if there is lower precision data at the vertices. For this reason you'd probably only need a few (maybe 4) extra bits so that you don't have any local shifting of the texels. However, this wouldn't work for dependent texture lookups (e.g. reflective water).

 

[OpenGL guy]

First, you need to remember that in FP each possible value is not equally spaced. The mantissa is 11 bits (and i believe there's one implied bit), so if you address 4096 texels (in any direction, not total texels like you thought) without filtering, each texel should come out pretty much fine, although small errors could make each texel come out differently sized. Remember that this includes repeating, e.g. a 256x256 texture repeated 16 times, or a 4096x4096 texture with no repetition

I'm not sure repeating impacts things the way you say. Since the mantissa is always between 0 and 1, then you'll need to increase the exponent, if you want to repeat the texture, and this doesn't impact your precision.

For filtering, it would make sense to consider the worst case scenario. Lets say one texel is black, and one next to it is white. Given that 32-bit color is generally enough to make a gradient without banding, you need 8 bits more precision in your texture coordinate data. This is only really true, however, if you are close enough to a texture that two texels are 256 pixels apart on the screen, which is some pretty serious magnification.

I believe this is the main reason why people say that FP16 is barely enough for 256x256 textures.

 

[sireric]

FP16 is an M10E5 format, giving you 10b of precision. Generally, you require subtexel precision, and you want at least 4b (1/16).

So you end up with these maximum texture sizes:
No subtexel: 1024x1024
Correct subtexel: 64x64

At 64x64 and below, it's fine. As you go above 64x64, you will start getting errors and visual anomalies. Above 1024x1024, it will get impossible to select individual texels -- You can still address "regions", but will have significant visual errors (i.e. you really don't want to do that).

So, a 256x256 texture will have some visual degradation.

 

[Simon F]

FP16 is an M10E5 format, giving you 10b of precision.

Most float specs I've seen have an implied bit in the mantissa (except for denormalised modes). Surely you would it include in the precision.

 

[sireric]

Good point. The leading 1 is implied, which causes only half the range to be of the specified precision (i.e. from 512 to 1024, you have 10b, but you have more below 512). That translates to doubling the precision. So you should be able to address up to 2048 texels with 10b mantissa, with more precision in the lower half than upper half. With 4b texel subprecision, you'd be limited to 128x128.

Thanks for the correction.