What I am trying to achieve
So I'm a fractal enthusiast and decided to build a 2D/3D fractal generator in WebGL using raymarching, with Typescript as scripting language. I've been a C#/Typescript dev for several years but having zero experience with 3d programming, I used Michael Walczyk's blog as a starting point. Some of my code I use here is derived from his tutorial.
I added the functionality that you can move through the object in my 3d raymarched world using WASDQEZC keys. WS = strafe forward-back, AD = strafe left-right, QE = strafe up-down, ZC = roll left-right. I combine this with a mouse look function which moves in the direction the mouse pointer is located on the rendering canvas. So what I want is total freedom of movement like in a spacesim. For this I am using a separate camera rotation matrix together with translation values and send them to the shader like this. So far I've got WS, AD and QE working okayish (please see 'my problem' to find out why).
setCameraMatrix(): void {
let cameraRotationMatrixLocation: WebGLUniformLocation = this.currentContext.getUniformLocation(this.currentProgram, "u_cameraRotation");
let cameraTranslationLocation: WebGLUniformLocation = this.currentContext.getUniformLocation(this.currentProgram, "u_cameraTranslation");
let foVLocation: WebGLUniformLocation = this.currentContext.getUniformLocation(this.currentProgram, "u_foV");
//add point of camera rotation at beginning
let cameraRotationMatrix: Array<number> = Matrix3D.identity();
//set camera rotation and translation, Z-axis (heading) first, then X-axis (pitch), then Y-axis (roll)
cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.rotateZ(this.cameraRotateZ * Math.PI / 180));
cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.rotateX(this.cameraRotateX * Math.PI / 180));
cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.rotateY(this.cameraRotateY * Math.PI / 180));
//cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.translate(this.cameraTranslateX, this.cameraTranslateY, this.cameraTranslateZ));
cameraRotationMatrix = Matrix3D.inverse(cameraRotationMatrix);
let cameraPosition: Array<number> = [
this.cameraTranslateX,
this.cameraTranslateY,
-this.cameraTranslateZ,
];
this.currentContext.uniformMatrix4fv(cameraRotationMatrixLocation, false, cameraRotationMatrix);
this.currentContext.uniform3fv(cameraTranslationLocation, cameraPosition);
this.currentContext.uniform1f(foVLocation, this.foV);
}
I tried adding the camera translation values to the camera matrix but that didn't work. I got weird distortion effects and couldn't get it right so I commented that line out and left it there for now for clarity. The reason I did it this way is because of the way my GLSL code is constructed:
The main function from the fragment shader with the call to the ray_march function. v_position is a vec2 with x,y coordinates coming from the vertex shader.:
void main() {
outColor = vec4(ray_march(u_cameraTranslation, u_cameraRotation * vec4(rayDirection(u_foV,v_position),1), u_world, vec3(u_light * vec4(0,0,0,1)).xyz ).xyz,1);
}
The ray_march function I am using. This derives from the sample code in Michael Walczyk's blog.
vec3 ray_march(in vec3 ro, in vec4 rd, in mat4 wm, in vec3 lightPosition) //ro = ray origin, rd = ray direction gt = geometry position after matrix multiplication
{
float total_distance_traveled = 0.0;
const int NUMBER_OF_STEPS = 1024;
float MINIMUM_HIT_DISTANCE = 0.001 * min_hit_distance_correction;
const float MAXIMUM_TRACE_DISTANCE = 1000.0;
for (int i = 0; i < NUMBER_OF_STEPS; i++)
{
vec3 current_position = ro + total_distance_traveled * vec3(rd);
float distance_to_closest = map(current_position, wm);
if (distance_to_closest < MINIMUM_HIT_DISTANCE)
{
vec3 normal = calculate_normal(current_position, wm);
vec3 outColor = vec3(1.0,0,0);
vec3 v_surfaceToLight = lightPosition - current_position;
vec3 v_surfaceToView = ro - current_position;
//insert lighting code below this line
return outColor;
}
if (total_distance_traveled > MAXIMUM_TRACE_DISTANCE)
{
break;
}
total_distance_traveled += distance_to_closest;
}
return vec3(0.25);//gray background
}
The rayDirection function I am using.
vec3 rayDirection(float fieldOfView, vec2 p) {
float z = 1.0 / (tan(radians(fieldOfView) / 2.0));
return normalize(vec3(p.xy, -z));
}
My problem
I have issues moving and rotating my camera correctly in 3d world. I'm doing this by applying some trigonometry to get the movement right. E.g., when I move forward, that is the Z-axis. But when I make a 90 degree turn to the right the X-axis now becomes the Z-axis. I am using trigonometry to correct this and actually got something working but now I am stuck in a quagmire of trigonometry trying to get 'roll', rotation around the Z-axis working when being combined with rotations around the X and Z axis, with no end in sight and I have a feeling there must be a better and less complicated way. To see what I'm talking about, here is the code of the 'move' function:
move(event: KeyboardEvent): void {
//strafe forward-back
let tXForwardBack: number = (Math.sin(this.cameraRotateY * Math.PI / 180) * Math.cos(this.cameraRotateX * Math.PI / 180)) * this.clipSpaceFactor * this.speed;
let tYForwardBack: number = Math.sin(this.cameraRotateX * Math.PI / 180) * this.speed;
let tZForwardBack: number = (Math.cos(this.cameraRotateY * Math.PI / 180) * Math.cos(this.cameraRotateX * Math.PI / 180)) * this.clipSpaceFactor * this.speed;
//strafe up-down
let tXUpDown: number = ((Math.sin(this.cameraRotateX * Math.PI / 180) * Math.sin(this.cameraRotateY * Math.PI / 180)) * this.clipSpaceFactor * this.speed);
let tYUpDown: number = Math.cos(this.cameraRotateX * Math.PI / 180) * this.speed;
let tZUpDown: number = Math.sin(this.cameraRotateX * Math.PI / 180) * Math.cos(this.cameraRotateY * Math.PI / 180) * this.clipSpaceFactor * this.speed;
//strafe left-right without roll. TODO: implement roll
let tXLeftRight: number = Math.cos(this.cameraRotateY * Math.PI / 180) * this.clipSpaceFactor * this.speed;
let tYLeftRight: number = 0;
let tZLeftRight: number = Math.sin(this.cameraRotateY * Math.PI / 180) * this.clipSpaceFactor * this.speed;
switch (event.key) {
case "w": { //strafe forward
this.cameraTranslateX = this.cameraTranslateX + tXForwardBack;
this.cameraTranslateY = this.cameraTranslateY - tYForwardBack;
this.cameraTranslateZ = this.cameraTranslateZ + tZForwardBack;
//this.cameraTranslateZ = this.cameraTranslateZ + (this.clipSpaceFactor * this.speed);
break;
}
case "s": { //strafe back
this.cameraTranslateX = this.cameraTranslateX - tXForwardBack;
this.cameraTranslateY = this.cameraTranslateY + tYForwardBack;
this.cameraTranslateZ = this.cameraTranslateZ - tZForwardBack;
break;
}
case "a": {//strafe left
this.cameraTranslateX = this.cameraTranslateX - tXLeftRight;
this.cameraTranslateY = this.cameraTranslateY + tYLeftRight;
this.cameraTranslateZ = this.cameraTranslateZ + tZLeftRight;
break;
}
case "d": { //strafe right
this.cameraTranslateX = this.cameraTranslateX + tXLeftRight;
this.cameraTranslateY = this.cameraTranslateY - tYLeftRight;
this.cameraTranslateZ = this.cameraTranslateZ - tZLeftRight;
break;
}
case "q": { //strafe up
this.cameraTranslateX = this.cameraTranslateX + tXUpDown;
this.cameraTranslateY = this.cameraTranslateY + tYUpDown;
this.cameraTranslateZ = this.cameraTranslateZ + tZUpDown;
break;
}
case "e": { //strafe down
this.cameraTranslateX = this.cameraTranslateX - tXUpDown;
this.cameraTranslateY = this.cameraTranslateY - tYUpDown;
this.cameraTranslateZ = this.cameraTranslateZ - tZUpDown;
break;
}
case "z": { //roll left
this.cameraRotateZ = (this.cameraRotateZ + (this.sensitivity * this.speed)) % 360;
break;
}
case "c": { //roll right
this.cameraRotateZ = (this.cameraRotateZ - (this.sensitivity * this.speed)) % 360;
break;
}
}
It actually works to some degree, but you can see where this is going :( Also, I get a 'dead' zone in the center when I look up and down along the Y-axis. I found This thread which seems to describe my problem and says 'The trick is to apply the translation to the z-axis but in the local coordinate system of the camera.'
But how do I do that with my existing code? I tried multiplying the world matrix u_world by the rotationmatrix u_rotationMatrix but then the lighting changes as well and it's just an object rotation instead of a separate camera rotation. In the thread I posted there is no lighting so multiplying the camera matrix with the world matrix works for them. But it doesn't for me because of the lighting I implemented. Also, I can't seem to apply the normals separately this way so that I only apply the normals to the world matrix and not to the camera rotation matrix, so that the lighting stays in place when I rotate/translate the camera.
The only way I can get correct normals to the world matrix and a separate cameramatrix is by multiplying the rotationMatrix with the rayDirection like so u_cameraRotation * vec4(rayDirection(u_foV,v_position),1). But when I do this I have to apply all this horrible, partially working trigonometry mess to get something decent. What I want is getting it to work like 'The trick is to apply the translation to the z-axis but in the local coordinate system of the camera.'
But I don't know how. I tried all kinds of things but I'm currently stuck. Any help would be greatly appreciated. I think I've outlined my problem sufficiently enough, if you miss anything please let me know. I posted my question on StackOverflow earlier but I was told this was a more appropriate place. Thanks in advance.
