Rotation of meshes in PyTorch

I am doing some project on object reconstruction and I need to rotate an object in PyTorch during the training to make sure I am augmenting the data properly. My data consists of points and it's normals. My question is, how should I (or should I train them at all) rotate the normals of my mesh. Note that this rotation also needs to keep the object inside the unit cube, where it was before the transformation.

My question is, how should I (...) rotate normals of my mesh.

Normals are rotated the same way as the rest of your mesh/point cloud. But make sure that you use a "pure rotation matrix" for this (no scaling, no translation). Since rotations preserve lengths, you do not need to renormalize your normals (usually - floating-point errors might accumulate).

This sentence of you

Note that this rotation also needs to keep object in unit cube, where it was before running this code.

lets me guess, that you are actually asking how to rotate an object and its normals and not just the normals. In python, you can use the packages SciPy and NumPy for that.

The Rotation package (documentation) of SciPy helps you to generate an arbitrary rotation matrix. Then you can use NumPys matmul function to rotate your points and normals.

Here is a short example script that rotates a single point 20 degrees around the z axis:

from scipy.spatial.transform import Rotation as Rot
import numpy as np

p = np.array([1, 1, 0]).transpose()
rot_mat = Rot.from_euler(seq="z", angles=20, degrees=True).as_matrix()

p_r = np.matmul(rot_mat, p)

print(p)
print(p_r)
print(np.linalg.norm(p))
print(np.linalg.norm(p_r))


output:

[1 1 0]
[0.59767248 1.28171276 0.        ]
1.4142135623730951
1.414213562373095


Info: Depending on your SciPy version you might need to replace as_matrix with as_dcm

As you can see, apart from some minor floating-point error, the length (last 2 outputs) did not change. Furthermore, I chose the example point, so that it gets outside the unit cube. To address this problem, you need to find the maximum absolute value inside the transformed data. You can use amax and amin to get the maximum and minimum value (see also here) if you put all your coordinates into one large 2d array (a matrix). Then compare the absolute values of the amax and amin results and pick the larger one. Divide all your coordinates (not the normals!) by this value, to scale your model back into the unit cube. Alternatively, you can create a uniform scaling matrix

$$\begin{bmatrix} s&0&0\\ 0&s&0\\ 0&0&s \end{bmatrix}$$

with $$s$$ being the reciprocal of the maximum absolute value. Multiply this matrix with all points.

To sum it up:

• calculate the rotation matrix with SciPy
• Multiply the rotation matrix with all points and normals
• find the maximum absolute value in the rotated data
• scale your coordinates with the found maximum value

Remark: If you put your points and normals into 2 matrices, you can apply the rotations and scaling matrices without a loop, which makes it much faster.

• But if the model is centered around the center (0,0,0) in a unit cube, is there really a need for rescaling points back to the unit cube? Moreover, can I use the same rotation matrix for normals as I used for points? – GreatDuke May 4 at 1:26
• also, I don't like your approach by scaling by maximum value across all axises (x,y,z), since it degrades pointcloud objects significantly. Say you have a pencil like object, if you scale it by it's maximum value, which is surely going to be along the pencil length, and divide both it's width and height by the same value, you will get something that looks more like a small radius stick than the pencil. – GreatDuke May 4 at 11:32
• Question 1: Depends on your model. But take the unit cube itself as an example. If you rotate it by any angle that isn't a multiple of 90 degrees, The corners will end up outside the unit cube. To visualize it, take two dice and put them on top of each other so that they are aligned. Now rotate the upper one by 45 degrees. See the problem? Only if all your points are initially inside the unit sphere, you can be sure that no point will end up outside the unit cube after an arbitrary rotation. – wychmaster May 4 at 12:09
• Also, why scaling by some value, it's rotation and should be shape and length preserving, wouldn't shifting/translation be correct way of bringing object back to the unit cube? – GreatDuke May 4 at 12:11
• Question 2: Yes, actually you HAVE to, otherwise you get wrong normals. A quote from my answer: "Normals are rotated the same way as the rest of your mesh/point cloud." – wychmaster May 4 at 12:13