implementing right handed coord system APIs on top of a left-handed rendering engine

Question

I am trying to plug in a 3D game engine as the rendering engine for an existing graphic API system that uses a right-handed coordinate system.

Game engines often uses a left handed coordinate system internally.

So how can I convert the API parameters, 3d vectors, transformation matrices, and quaternions?

The right handed versions at the API boundary have to be converted to left handed equivalents before passing to the engine's APIs at a lower layer, and then the output parameters from the engine have to be converted back to the right hand representations for the higher level APIs.

Both coordinate systems I am referring to here are, x pointing to the right, y pointing to the top, and z point away from the eye (left hand) and towards or behinds the eye (right hand), with the eye at the origin or some distance behind the xy plane to allow the origin to be in the camera view.

Example: For simple 3d-points, the conversion step would be just to flip the sign of the z value.

Answer 1

What you probably need is this 4 x 4 matrix:

$$\mathtt{T} = \begin{bmatrix}1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0 \\ 0 & 0 & -1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{bmatrix}$$ which is flipping the z-axis. The inverse of this particular matrix is the matrix itself: $\mathtt{T}^{-1} = \mathtt{T}$

For 3D points $\mathbf{x}$ multiply it from the left: $$\mathbf{x}' = \mathtt{T}\,\, \mathbf{x}$$ and for 3D transformations multiply it from the right: $$\mathtt{P}' = \mathtt{P} \,\,\mathtt{T}$$

Thus, you have not changed anything in the projection because: $$\mathtt{P}'\,\,\mathbf{x}' = \mathtt{P} \,\,\mathtt{T} \, \,\mathtt{T}\,\, \mathbf{x} = \mathtt{P} \,\,(\mathtt{T}^{-1} \, \,\mathtt{T})\,\, \mathbf{x} = \mathtt{P}\,\,\mathbf{x}$$

Quaternions $q=(s,x,y,z)^\top$ could be convert into a 3D transformation matrix with:

$$\mathtt{P} = \begin{bmatrix} 1-2y^2-2z^2 & 2xy-2sz& 2xz+2ys & 0\\ 2xy+2sz & 1-2x^2-2z^2& 2yz-2sx & 0\\ 2xz-2sy & 2yz+2sx& 1-2x^2-2y^2 & 0\\ 0 & 0 & 0 & 1\end{bmatrix}$$ and could then be treated like other 3D transformations.

However, 3D engines typically apply transformation in a transformation hierarchy. If this is the case, make sure you are only modifying the 3D transformation "closest" to the 3D point in the hierarchy.

Sources: Graphics Programming I - 3D Transformations | Thorsten Thormählen