3
$\begingroup$

I writing the this video process app which requires the each video frame to be processed such that parts of the video are enlarged. As a reference, one of the snapchat lens filter enlarges the eyes of a person in the video. This is the exact effect I want to achieve.

How can I achieve this effect?

Update:

I have attached an example of eye enlargement from the web. enter image description here

$\endgroup$
  • 1
    $\begingroup$ Before/after screenshots would be helpful, otherwise we're all trying to imagine what you mean. Also, is the question about how to enlarge a known region (i.e. you have already identified where the eyes are), or about how to find/track the objects to enlarge? Because those are very different questions. :) $\endgroup$ – Nathan Reed Jan 22 '16 at 19:13
  • $\begingroup$ Welcome to Computer Graphics Stack Exchange. Asking for off-site resources is off-topic for this site, that's why I removed that part of your question. $\endgroup$ – Nero Jan 22 '16 at 19:18
  • $\begingroup$ @Nero no worries. $\endgroup$ – crab oz Jan 23 '16 at 4:56
  • $\begingroup$ @NathanReed I have already tracked the region of the video for enlargement. I am only looking at enlarging the regions are blend well with the rest of the video content. $\endgroup$ – crab oz Jan 23 '16 at 4:58
2
$\begingroup$

The answer is surprisingly simple: You move the vertices of the face-geometry.

To elaborate, the white wireframe mesh on the left of your image is placed so the geometry aligns with the photo. It is then texture-mapped with the photo, meaning that for each vertex in the mesh, a texture-coordinate is assigned (the point on the photo that should map to the vertex).

With every vertex assigned a texture-coordinate you can deform the mesh, doing ordinary texture-sampling at the interpolated texture-coordinates to have the image stretch - as if it was painted onto the head.

If you need to do this in realtime, you can use your graphics-card to do this; texture mapping is one of the things GPUs are really good at.

Now, if you want to do this on a video, you need to track the facial features to continuously align the geometry to the video. This is not an easy task, but you can probably find a wealth of libraries to help you out.

I am a bit uncertain which of the above would be difficult, so please feel free to ask any questions.

$\endgroup$
1
$\begingroup$

In a post processing step instead of sampling the origin pixel directly from where the target pixel is you instead sample the origin pixel slightly offset.

Where you get the offset can differ. For example you could provide an extra texture with the offsets encoded in the red and green channels.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.