As of May 31, 2023, we have updated our Code of Conduct.

Questions tagged [float]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
59 views

Splitting Floats To Texture Channels

I know, this is a silly question, but since I need this so often, I just want to double check that I made no mistake. Working with most shader languages, a texture can store $8$ bits per channels, ...
fweth's user avatar
  • 127
0 votes
1 answer
81 views

Plot inverse of emulated double-precision floats

I need to accurately plot a line chart using WebGL. The numbers have a precision of around 33 bits - that's too many to fit into a single-precision float's mantissa. WebGL does not support the double-...
purefanatic's user avatar
1 vote
1 answer
59 views

How do (Direct3D) precision conversions for floats work?

There is this article about conversions in Direct3D: https://docs.microsoft.com/en-us/windows/win32/direct3d10/d3d10-graphics-programming-guide-resources-data-conversion I do not understand the part ...
Tare's user avatar
  • 1,477
0 votes
0 answers
102 views

Loop subdivision, certain areas converging towards degenerate geometry

I coded loop subdivision and ran it on a cube with triangle faces. I noticed this: I am using doubles to represent my vertex positions, I am not 100% sure these are caused by floating point ...
Makogan's user avatar
  • 1,624
2 votes
1 answer
895 views

Floating Point Operations in Line Drawing Algorithms

When it comes to line drawing, DDA is the simplest and most intuitive algorithm, the core idea being: compute the slope of the line for every increment in x, increment y by the slope However, DDA is ...
wsaleem's user avatar
  • 121
5 votes
1 answer
452 views

Precision problem on AMD

I have a problem with precision on AMD in shaders (hlsl). In vertex shader I calculate UVs for particle using modulus operator ...
Derag's user avatar
  • 596
2 votes
1 answer
507 views

How to work with half and snorm

I have just come across this article about half, snorm and unorm, but I don't get some of the points (or why they work). According to the article, a half is ...
Tare's user avatar
  • 1,477