Questions tagged [float]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
129 views

Plot inverse of emulated double-precision floats

I need to accurately plot a line chart using WebGL. The numbers have a precision of around 33 bits - that's too many to fit into a single-precision float's mantissa. WebGL does not support the double-...
0 votes
0 answers
58 views

Floating point precision in 3d modeling software

I'm working on a simple 3d modeling application. I'm using 1 meter as unit of measurement. When working with small triangles where an edge length is around 1 centimeter (0.01 units) or less, I started ...
0 votes
1 answer
131 views

Splitting Floats To Texture Channels

I know, this is a silly question, but since I need this so often, I just want to double check that I made no mistake. Working with most shader languages, a texture can store $8$ bits per channels, ...
1 vote
1 answer
82 views

How do (Direct3D) precision conversions for floats work?

There is this article about conversions in Direct3D: https://docs.microsoft.com/en-us/windows/win32/direct3d10/d3d10-graphics-programming-guide-resources-data-conversion I do not understand the part ...
0 votes
0 answers
121 views

Loop subdivision, certain areas converging towards degenerate geometry

I coded loop subdivision and ran it on a cube with triangle faces. I noticed this: I am using doubles to represent my vertex positions, I am not 100% sure these are caused by floating point ...
2 votes
1 answer
999 views

Floating Point Operations in Line Drawing Algorithms

When it comes to line drawing, DDA is the simplest and most intuitive algorithm, the core idea being: compute the slope of the line for every increment in x, increment y by the slope However, DDA is ...
5 votes
1 answer
522 views

Precision problem on AMD

I have a problem with precision on AMD in shaders (hlsl). In vertex shader I calculate UVs for particle using modulus operator ...
2 votes
1 answer
623 views

How to work with half and snorm

I have just come across this article about half, snorm and unorm, but I don't get some of the points (or why they work). According to the article, a half is ...