This is more a theoretical question. I'm wondering what happens when I write float values into an empty RGBA texture of type gl.UNSIGNED_BYTE in WebGL 1.0. Obviously, every channel then has 8 bits. But that wouldn't be enough to represent floating point values, would it? Nevertheless it somehow works, more or less. See part of a fragment shader code here:

precision highp float;

uniform vec2 u_Resolution;

void main(void){
   gl_FragColor = vec4(gl_FragCoord.xy / u_Resolution, 0.0, 1.0);

When I specify the texture to be of type gl.FLOAT (using the OES_texture_float extension), the results of course look better. But I want to know exactly why they do.

In addition: does the specified precision have an effect on the texture (to be written)?


Unsigned byte textures in a graphics API are usually "normalized"—the byte values are interpreted as ranging from 0.0 to 1.0, with a byte of 255 mapping to 1.0. So no, they aren't truly floating-point values, but the hardware will automatically convert them to/from float when you read/write the texture. If you try to write a value outside the [0, 1] range, it will be clamped.

As for the precision highp part, it doesn't directly affect the texture, but it can certainly affect any calculations of values you write to the texture. For example, to calculate an 8-bit texture you certainly want a minimum of 8 bits of mantissa precision (i.e. lowp)—and you probably want more than that, depending on what calculations you're doing. I'd advise you to check what the actual precision levels are on your device, since they vary widely depending on what GPU you have. See this question for more.

  • $\begingroup$ So, when writing to the texture, WebGL internally does something like floor(x*255) to map the [0,1] float value x to [0,255]? And then later, when reading from that texture again, x will probably deviate from the original value. Is that correct? And in case of a gl.FLOAT texture, no mapping to [0,255] occurs? $\endgroup$ – Muad Jul 28 '17 at 20:14
  • 2
    $\begingroup$ @Muad It would be round(x*255) instead of floor, but yes. And yeah, when you read it back, you'll likely get a slightly different value due to the rounding. In a float texture, it just stores and retrieves the exact value you write, no remapping (and no clamping to [0, 1]). $\endgroup$ – Nathan Reed Jul 28 '17 at 22:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.