# How can I offset/shrink a triangular polygon in GLSL?

I need to offset all (blue) triangles, each independently of the others, using the vertex-shader. In order to manipulate the triangle as a whole, I've created custom (vec3) attributes for each vertex (red) representing the leftward (purple) and rightward (green) neighboring vertices. From this, I need to derive the orange point, equidistant (in screen space) from both adjoining edges. With three such orange points derived from each triangle, the processed (orange) triangle is passed on to the fragment shader.

Ideally, the triangle will be culled (as in backfacing/ not rendered) if the offsets negate any available space within the triangle, such as in the second triangle in the second picture.

I'm using THREE.BufferGeometry() as my data structure.

Here is a screen shot of the effect I'm aiming for:

• Could you add a bit more about the wider context? Are the offset triangles to remain attached as in the original mesh? Does "culled" mean the original triangle is discarded, or just that the offsetting is abandoned, leaving the triangle at its original size? – trichoplax May 9 '17 at 22:18
• So... how does this work with meshes? Because in a mesh, a vertex has more than 2 neighbors. Or is this just for individual triangles? – Nicol Bolas May 9 '17 at 22:32
• My implementation is such that all triangles are laid out in a continuous buffer: [P1.x, P1.y, P1.z, P2.x, P2.y, P2.z ... Pn.x, Pn.y, Pn.z] with neighboring points also laid out explicitly at attributes. This way, each vertex of each face can be calculated and manipulated without affecting neighboring faces. Nicol Bolas, yes, dealing with each triangle separately. – Jackalope May 9 '17 at 23:02
• trichoplax - "Culled" means thrown out, not rendered, as in a back-facing, single-sided primitive. – Jackalope May 11 '17 at 17:38
• @Jackalope: "Both of you seem to suggest that the GPU sees faces as "tethered" to other faces." That's because, generally speaking, this is true. Most meshes don't merely have neighboring triangles use "identical attributes"; they reuse the same vertices. This could be through triangle lists that use the same index multiple times, or through triangle strips, or whatever. But generally speaking, meshes reuse neighboring vertices. Your meshes do not, but your specific case doesn't change the general case. That's why I asked for clarification. – Nicol Bolas May 12 '17 at 16:44

Given triangle ▲ABC, we bisect angle ∠BAC with line AD, derived with Angle Bisector Theorem:

BA / BD = CA / CD Point E represents our objective refined position on the desired resulting inset triangle. As it lies upon angle bisector AD, it is equidistant from sides BA & CA, forming identical right triangles ▲AFE & ▲AGE. We can now use Sine for Right Triangles to find the length of AE:

AE = EG / Sin(∠EAG)

That's all the math we need, so let's cook up some GLSL!

We start off with all the typical attributes: position, normal, and transformation matrices, but since the vertex shader only works on a single vertex, we need to add the neighboring vertices as additional attributes. This way, each vertex will find its own "point E", creating the resulting inset triangle. (Note: I don't call them "B" & "C" here, because they are not yet in screen space.)

    attribute vec3 left; //vertex to the left of this vertex
attribute vec3 right; //vertex to the right of this vertex


Speaking of screen space, I'm also including the aspect ratio of the display, (and making it a uniform, in case the window is resized.)

After preparing varying normals for the fragment shader, and transforming the face into clipping-space, we can get down to the business of applying the above math:

        attribute vec3 left; //vertex to the left of this vertex
attribute vec3 right; //vertex to the right of this vertex
uniform float aspect;
varying vec3 vNormal;
varying vec2 vUv;

void main() {
vNormal = normal;
vUv = uv;

mat4 xform= projectionMatrix * modelViewMatrix;
vec4 A = xform * vec4( position, 1.0 );
vec4 B = xform * vec4( left, 1.0 );
vec4 C = xform * vec4( right, 1.0 );

vec3 CB = C.xyz - B.xyz;
vec2 BA = B.xy - A.xy;
vec2 CA = C.xy - A.xy;
float lengthBA = length(BA);
float lengthCA = length(CA);
float ratio = lengthBA / ( lengthBA + lengthCA );
vec3 D = B.xyz + ratio * CB.xyz;
vec3 AD = D - A.xyz;

float theta = acos( dot(BA, CA) / (lengthBA * lengthCA) ) / 2.0;
float AE = 1.0/(sin(theta)*aspect);
newPos.z += AE/length(AD) * (D.z - A.z);
newPos.x += bisect.x*AE;
newPos.y += bisect.y*AE;

gl_Position = newPos;
}


This code gives us the results below.

Note, there are a few edge-cases having to do with almost backface-culled triangles being flipped by this process, and I began to address this in code, yet decided to simply avoid these cases for now. Perhaps I'll revisit it when I get this project done.

• Nice work figuring this out! Really like the mathematical description at the beginning. – user1118321 May 13 '17 at 3:36

This can be achieved without trigonometric functions by scaling down the incircle of the triangle.

incircle() computes the incircle of the triangle formed by the vertices A,B,C, it returns center and radius as vec4. The vertices X=A,B,C are then moved inwards by the fraction of their distance to the incircle center (Q-X) which is equal to the ratio of desired margin to incircle radius (m/Q.w).

vec4 incircle(vec3 A, vec3 B, vec3 C) {
float a = length(B - C), b = length(C - A), c = length(A - B);
float abc = a + b + c;
// http://mathworld.wolfram.com/Incenter.html
vec3 I = (a * A + b * B + c * C) / abc;