What is this technique called and how can I reproduce it algorithmically?

Question

I don't know the name of this technique, but the idea is to adjust the density and location of identical dots or small objects in order to produce a gray tone image or even a suggestion of three dimensionality. It's better to use these two images below as examples.

I'd like to do something like this on my own, but by writing relatively simple algorithm rather than a program or package. I'll be starting with a 3D representation of an object's surface inside a python script running in Blender. My question isn't how to implement, but how to understand what's happening.

Could someone explain what this is called and share some amount of insight into how this works? I don't need a full-blown derivation, just some idea to get started on something simple.

In the screenshot, the objects (in software) follow the face contour in 3D, but their position (and brightness) is fluid and their separation varies continuously. While they appear to be approximately on a grid, I'm not sure if they are a fixed projected grid (I think I've read somewhere it was done using a Kinect) and move because of the movement of the model, or if they move algorithmically as well.

edit: More info on the Kinect in the Below the Line article Level 256 VFX Produces The Final Moments of Karl Brant for Nerdist.com.

“Leveraging the power of open source applications like Processing, Python, and bash scripting, we were able to take the point cloud data from the Kinect, manipulate it, and get it into Nuke for integration with the live-action plates for texture re-projection,” Kleinberg explained. “The process evolved and grew over time, and when we finally saw the fruits of our labor on the screen, we were excited to see Karl Brant fully realized, taking on an expressive life of his own.”

I'm familliar with something simple like using a random number to decide if the pixel should be on or off depending on the gray value; if it's 0.3, then if a random number (chosen for each pixel, but not necessarily randomized every frame) is below 0.3, the pixel should be ON (seems to be called 1-bit dithering). But I think there is a lot more going on here than that.

below: Cropped screen shot at 11:23 in the short film The Final Moments of Karl Brandt:

Answer 1

I've done something similar. The way I did it was by using a height map image to change the positions of vertices in a grid in a vertex shader. I generated a 2D grid of vertices that were evenly spaced on the CPU. I uploaded the vertices and a texture and in the vertex shader, I sampled the texture and used the resulting brightness as the height. So I'd keep the x and y positions of each vertex and change the z based on the image.

Now there are other ways you could utilize this to get something more like in the video. You could either have a 2D grid of single vertices, or even a grid of small 3D models. Change the z position based on the color of a height map. But additionally, you could upload a small noise texture and vary the x and y positions by a small amount based on the noise, which might vary per frame. I would see if you could vary the x and y by a large enough amount to see, but small enough that no point ever crossed over a neighbor point. That would get you some more movement and randomness.