Getting rid of branching to check if a variable is within a range in GLSL

Question

I have this GLSL function that I am trying to optimize because it is going to be ran on many pixels of an older devices GPU. There is no room for branching inefficiency. Essentially this function returns a 0 or a 1 based on the variables actualY shouldY and state

float val = 0.0;
if (actualY < shouldY) {
    val = 1.0;
}

if (state >= 20.0 && state < 40.0) { //The equal/not equals is VERY important
  return 1.0 - val;
}
return val;

Usually I am fairly good at removing branching from GLSL code but I really don't know what to do with the range check for the state variable.

Answer 1

This could be done that way:

First condition is float val = 1.0-step(shouldY, ActualY);

Then, the condition: state >= 20.0 && state < 40.0 can be changed to

step(20.0, state) - step(40.0, state). This equals 0.0 if outside of range and 1.0 if inside.

Let a = step(20.0, state) - step(40.0, state)

Then we know that mix function:

genType mix(genType x,
    genType y,
    genType a);

Is $$x * (1 - a) + y * a = val * (1 - a) + (1 - val) * a = val - 2*a*val + a$$

So final code is:

float val = 1.0-step(shouldY, ActualY);
float a = step(20.0, state) - step(40.0, state);
return val - 2.0*a*val + a;

Answer 2

The question asks about branch avoidance, so doing that while retaining comparison is still fair play(*).

Checking if some scalar or vector equals its clamped version is a terse, intuitive option:

Combine float(clampN(d, dMin, dMax) == d), which yields 0.0 if false or 1.0 if true, with whatever way you use this resulting value, for example, mix.

d can be scalar or any vector, while dMin/dMax are either scalars, or whatever d is.

If < needs to be distinguished from <=, it gets a bit more tedious as additional comparisons are needed for the min/max values or subtracting the value from an extremum and getting a sign, unless one can safely use some other approach such as introducing an epsilon value.

For either case, performance may differ (or match) other solutions; in part, because...

(*) I'm not aware of anything that'd prevent the shader compiler from introducing its own branching if it feels like so; conversely, nothing prevents the shader compiler from converting branching (if) into single-track code, if it can tackle the data flow to the extent to be certain that the results are the same. IIRC the latter is actually done for at least simple cases.