# Questions tagged [float]

The tag has no usage guidance.

8 questions
Filter by
Sorted by
Tagged with
129 views

### Plot inverse of emulated double-precision floats

I need to accurately plot a line chart using WebGL. The numbers have a precision of around 33 bits - that's too many to fit into a single-precision float's mantissa. WebGL does not support the double-...
58 views

### Floating point precision in 3d modeling software

I'm working on a simple 3d modeling application. I'm using 1 meter as unit of measurement. When working with small triangles where an edge length is around 1 centimeter (0.01 units) or less, I started ...
131 views

### Splitting Floats To Texture Channels

I know, this is a silly question, but since I need this so often, I just want to double check that I made no mistake. Working with most shader languages, a texture can store $8$ bits per channels, ...
1 vote
82 views

121 views

### Loop subdivision, certain areas converging towards degenerate geometry

I coded loop subdivision and ran it on a cube with triangle faces. I noticed this: I am using doubles to represent my vertex positions, I am not 100% sure these are caused by floating point ...
999 views

### Floating Point Operations in Line Drawing Algorithms

When it comes to line drawing, DDA is the simplest and most intuitive algorithm, the core idea being: compute the slope of the line for every increment in x, increment y by the slope However, DDA is ...
522 views

### Precision problem on AMD

I have a problem with precision on AMD in shaders (hlsl). In vertex shader I calculate UVs for particle using modulus operator ...
623 views

### How to work with half and snorm

I have just come across this article about half, snorm and unorm, but I don't get some of the points (or why they work). According to the article, a half is ...