4
$\begingroup$

I don't know the name of this technique, but the idea is to adjust the density and location of identical dots or small objects in order to produce a gray tone image or even a suggestion of three dimensionality. It's better to use these two images below as examples.

I'd like to do something like this on my own, but by writing relatively simple algorithm rather than a program or package. I'll be starting with a 3D representation of an object's surface inside a python script running in Blender. My question isn't how to implement, but how to understand what's happening.

Could someone explain what this is called and share some amount of insight into how this works? I don't need a full-blown derivation, just some idea to get started on something simple.

In the screenshot, the objects (in software) follow the face contour in 3D, but their position (and brightness) is fluid and their separation varies continuously. While they appear to be approximately on a grid, I'm not sure if they are a fixed projected grid (I think I've read somewhere it was done using a Kinect) and move because of the movement of the model, or if they move algorithmically as well.

edit: More info on the Kinect in the Below the Line article Level 256 VFX Produces The Final Moments of Karl Brant for Nerdist.com.

“Leveraging the power of open source applications like Processing, Python, and bash scripting, we were able to take the point cloud data from the Kinect, manipulate it, and get it into Nuke for integration with the live-action plates for texture re-projection,” Kleinberg explained. “The process evolved and grew over time, and when we finally saw the fruits of our labor on the screen, we were excited to see Karl Brant fully realized, taking on an expressive life of his own.”

I'm familliar with something simple like using a random number to decide if the pixel should be on or off depending on the gray value; if it's 0.3, then if a random number (chosen for each pixel, but not necessarily randomized every frame) is below 0.3, the pixel should be ON (seems to be called 1-bit dithering). But I think there is a lot more going on here than that.

below: Cropped screen shot at 11:23 in the short film The Final Moments of Karl Brandt:

enter image description here

$\endgroup$
  • $\begingroup$ Need some help with the tags. $\endgroup$ – uhoh Nov 16 '17 at 13:52
  • 1
    $\begingroup$ The two images you have show completely unrelated techniques. The first is placing 3D objects on a surface. The second just downsamples a 2D image to 1-bit using dithering. Which are you interested in? $\endgroup$ – Dan Hulme Nov 16 '17 at 14:25
  • $\begingroup$ @DanHulme ya OK I see what you mean - there is no effort to show 3D in the 2nd one, it's just coming from shading (original lighting) of the model. I've removed it. $\endgroup$ – uhoh Nov 16 '17 at 14:27
  • 1
    $\begingroup$ The kinect can produce depth images for you. The depth image essentially gives you a set of points in 3d space. A point cloud much like what we see above. You'll have to downsample, the resolution will be much too high. Then to get the shade of them (steeper areas should have darker colors) you can take the gradient of the image for your normals. Not sure how his lights are positioned or what kind of lights he's using exactly though. There does seem to be some sort of blur going on or maybe that's the fog. $\endgroup$ – Andrew Wilson Nov 17 '17 at 0:24
2
$\begingroup$

I've done something similar. The way I did it was by using a height map image to change the positions of vertices in a grid in a vertex shader. I generated a 2D grid of vertices that were evenly spaced on the CPU. I uploaded the vertices and a texture and in the vertex shader, I sampled the texture and used the resulting brightness as the height. So I'd keep the x and y positions of each vertex and change the z based on the image.

Now there are other ways you could utilize this to get something more like in the video. You could either have a 2D grid of single vertices, or even a grid of small 3D models. Change the z position based on the color of a height map. But additionally, you could upload a small noise texture and vary the x and y positions by a small amount based on the noise, which might vary per frame. I would see if you could vary the x and y by a large enough amount to see, but small enough that no point ever crossed over a neighbor point. That would get you some more movement and randomness.

$\endgroup$
  • $\begingroup$ OK I'll give your suggestions a try, but the BTLNews article I've linked to in the question explains that they've started with a 3D representation or possibly a point cloud from a hacked Kinect box. So they start with an actual heigh of the surface, rather than generate one from a color or brightness. Possibly they re-interpolate it to a regular x-y grid of points with continuous z (z being "out of the page"). $\endgroup$ – uhoh Nov 30 '17 at 5:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.