5
$\begingroup$

I have a problem with precision on AMD in shaders (hlsl). In vertex shader I calculate UVs for particle using modulus operator on float. With simple % there is a problem on Nvidia and on AMD (e.g. 7 % 7 = 7) so I have written modulus as:

float Mod( float x, float y )    
{       
     return x - floor(x / y) * y;    
}

This have fixed the problem on Nvidia but AMD still returns wrong values.

Any idea what is going on? And how to fixed it?

I know I can use:

float( int( x ) % int( y ) )

but I think it is slower and I wonder if there is any better way.

$\endgroup$
2
  • $\begingroup$ Are you using DX or are you using vulkan? Both support HLSL... $\endgroup$
    – Krupip
    May 30, 2018 at 19:58
  • $\begingroup$ I'm using DirectX11 $\endgroup$
    – Derag
    Jun 13, 2018 at 21:16

1 Answer 1

1
$\begingroup$

In HLSL, you can use the built-in fmod function for this.

I'm not sure what's going wrong with your implementation, though. It looks mathematically correct. It's conceivable you're encountering a precision issue with the subtraction, depending on the inputs.

BTW, an alternative implementation which avoids the subtraction is:

float Mod(float x, float y)
{
    return frac(x / y) * y;
}
$\endgroup$
2
  • $\begingroup$ But frac(x / y) * y is exactly what normal modulus operator compiles to (on GCN at least) and it gives incorrect results on both Nvidia and AMD. I'm not sure about fmod, though. It seems to give slightly different assembly code. I will check it, thanks. $\endgroup$
    – Derag
    Apr 19, 2018 at 19:45
  • $\begingroup$ So I have chacked it and frac(x / y) * y, operator % and fmod give incorrect results on Nvidia and AMD. x - floor(x / y) * y works correct on Nvidia but AMD not so much. $\endgroup$
    – Derag
    Apr 20, 2018 at 7:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.