0
$\begingroup$

I'm writing a simple text editor using Vulkan for rendering, and I'm seeing some strange outline around the rendered letters if I take a screenshot of the text and zoom in, which bothers me because I can't explain why it's there.

Here's what RenderDoc shows in the texture view (note that in the pixel context view at the bottom right there's no outline around the text) RenderDoc textyre view

However, if I take a screenshot and zoom in, I'll see this border around the letters: text border

I use a pre-rendered font atlas in a texture stored as R8_UNORM. Then I use a very basic shader which takes the texture and produces a pixel with premultiplied alpha, ready to be blended by just adding the colors of src and dest:

#version 450 core

layout(set = 1, binding = 0) uniform sampler2D texSampler;

layout(location = 0) in vec2 v_tex_coord;
layout(location = 1) in vec3 v_color;

layout(location = 0) out vec4 f_color;

void main() {
    f_color = vec4(v_color, 1.0) * texture(texSampler, v_tex_coord).a;
}

I'm puzzled and out of ideas on why this outline is there - I don't see why the pixels next to text border would suddenly become darker when I'm only adding positive color components and none of them are negative (just to make sure there are no negative components I tried modifying the shader and took absolute value of every component with exactly the same result).

In case it matters, my swapchain surface format is B8G8R8A8_UNORM and the colorspace is SRGB_NONLINEAR_KHR. The colors are exactly what I expect them to be so I assumed that gamma-wise it's OK but maybe I'm wrong.

Would appreciate any help with this.

UPD. I just made it work on my office Macbook and there's no outline there! So it must be something to do with either my window surface on windows or even with screenshot-taking apps (I've tried snipping tool, printscr and snip & sketch) enter image description here

$\endgroup$
9
  • 1
    $\begingroup$ Looks like you're "zooming in" with a cubic filter. $\endgroup$
    – Wyck
    Mar 24 at 1:49
  • $\begingroup$ @Wyck I'm zooming in in GIMP, I've tried capturing both my text and text from VSCode on the same screenshot - my text has an outline, the other doesn't. So it must be something else $\endgroup$ Mar 24 at 3:46
  • 1
    $\begingroup$ Premultiplied alpha doesn't mean you "just add src and dest" you still have to multiply the dest by 1-src.alpha. Non-premultiplied src: dest' = dest * (1-src.alpha) + src * (src.alpha) Premultiplied src: dest' = dest * (1-src.alpha) + src $\endgroup$
    – Wyck
    Mar 24 at 3:54
  • $\begingroup$ @Wyck that's a good point - however, it still wouldn't explain why the pixels got darker. Imagine that the src is a white pixel with a 10% opacity and the dest is some dest_color. Then the resulting color will be dest_color * 0.9 + 0.1 * 1.0, which is always brighter than dest_color. I've just made all the text white and there's still an outline :(. Thank you very much for looking into it, it's hard to debug without having the app running. $\endgroup$ Mar 24 at 5:13
  • $\begingroup$ I'm talking about your blend function not your shader. You've prepared a pre-multiplied color value but you assigned an alpha value of 1.0 (why not assign an alpha value of texture(texSampler, v_tex_coord).a?). Presumably your DST blend factor should be set to VK_BLEND_FACTOR_ONE_MINUS_SRC_ALPHA. And the SRC blend factor should be set to VK_BLEND_FACTOR_ONE. How did you set up your blending? Is blending enabled? (I assume so) Pedantically: alpha = texture(texSampler, v_tex_coord).a; f_color = vec4(v_color, 1.0) * alpha or vec4(v_color * alpha, alpha) $\endgroup$
    – Wyck
    Mar 24 at 5:31

1 Answer 1

1
$\begingroup$

Almost a month after I posted this question I finally found the answer.

The steps I took:

  1. I decided to disable blending altogether and play with it: no-blending

  2. But when inspecting the produced image more closely I noticed that the outline was still there, and it was sticking outside of the rendered quad, which suggested that it's a post-processing effect: more closely

  3. I decided to browse through the settings of the graphics driver and found this: oh dear, radeon

Disabling it removed the outline.

[hard-facepalm-sound.ogg]

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.