Questions tagged [float]
The float tag has no usage guidance.
8
questions
0
votes
0
answers
49
views
Floating point precision in 3d modeling software
I'm working on a simple 3d modeling application. I'm using 1 meter as unit of measurement. When working with small triangles where an edge length is around 1 centimeter (0.01 units) or less, I started ...
- 516
0
votes
1
answer
102
views
Splitting Floats To Texture Channels
I know, this is a silly question, but since I need this so often, I just want to double check that I made no mistake.
Working with most shader languages, a texture can store $8$ bits per channels, ...
- 127
0
votes
1
answer
109
views
Plot inverse of emulated double-precision floats
I need to accurately plot a line chart using WebGL. The numbers have a precision of around 33 bits - that's too many to fit into a single-precision float's mantissa. WebGL does not support the double-...
- 101
1
vote
1
answer
72
views
How do (Direct3D) precision conversions for floats work?
There is this article about conversions in Direct3D: https://docs.microsoft.com/en-us/windows/win32/direct3d10/d3d10-graphics-programming-guide-resources-data-conversion
I do not understand the part ...
- 1,541
0
votes
0
answers
110
views
Loop subdivision, certain areas converging towards degenerate geometry
I coded loop subdivision and ran it on a cube with triangle faces. I noticed this:
I am using doubles to represent my vertex positions, I am not 100% sure these are caused by floating point ...
- 1,696
2
votes
1
answer
955
views
Floating Point Operations in Line Drawing Algorithms
When it comes to line drawing, DDA is the simplest and most intuitive algorithm, the core idea being:
compute the slope of the line
for every increment in x, increment y by the slope
However, DDA is ...
- 121
5
votes
1
answer
492
views
Precision problem on AMD
I have a problem with precision on AMD in shaders (hlsl). In vertex shader I calculate UVs for particle using modulus operator ...
- 596
2
votes
1
answer
576
views
How to work with half and snorm
I have just come across this article about half, snorm and unorm, but I don't get some of the points (or why they work).
According to the article, a half is ...
- 1,541