First, adressing the concerns of speed and performance, you will always have to make some trade-offs between quality and performance, as you already do. If the first version looks fine enough for your use-case and all other versions are much slower, you have to make the decision yourself. You will, however, not get around actually measuring any time difference. But since various ways to do this are not too hard to integrate into an existing wide lines renderer, just testing the various approaches in action and getting some time measurements shouldn't be too difficult, especially with OpenGL timer queries (or whatever the Direct3D equivalent).
So let's get to some possible improvements to your situation. It seems from your link and your explanations that you're using a classic geometry shader based approach for wide line rendereing, converting each line into a screen-aligned quad (or two triangles), computing the vertex positions based on the line width you want to achieve. I recently built something similar, too and there's a few ways you can improve the quality.
As you said yourself, since you're working on rather arbitrary rounded shapes it's probably best to let the fragment shader figure out the exact pixel-shape rather than trying to approximate the circular middle region in the geometry shader. Though, that idea is not completely off-the table and might give for a nice adjustable quality-speed trade-off by adjusting the number of vertices of that circle-part. But it also, together with your current approach, has the problem that it actually needs to know the line's neighbouring lines in order to compute the middle section, which makes it difficult to just put into an existing shader-agnostic rendering system that just fraws all the lines as vertex pairs (not to speak of line strips/loops), since you need to add the adjacency information for each line.
If you just want to let the fragmet shader do the job using simple alpha testing, you generate a screen-aligned rectangle that's a little larger than the line, i.e. it spans half the width across the endpoints and thus covers the circular middle section, too. In the fragment shader you just check each fragment's screen-space distance to the actual line center (whose vertices you transfer from the geometry shader to the fragment shader as varying). And whenever that distance is larger than the line width (or half the line width, to be precise), you discard the fragment. This way you will basically render a rounded box for each line.
To be precise you render your line as three quad-sections (i.e. 6 triangles in a single strip). The middle section between the ednpoints as one rectangle and the two end-caps as rectangles spanning from the endpoint half the width in the direction of the line. You do this separation to give the rightmost 4 vertices their respective endpoint as varying value and the leftmost one their respective endpoint. This way you get smooth interpolation of the line point across the middle section and clamping in the cap sections.
This distance computation might also be used to antialias the lines, i.e. by using alpha blending (or multisampling with alpha to coverage) and letting alpha go from 1 to 0 along the outermost ~1-pixel thick border of the line.
But this approach has the disadvantage of drawing the uneccessary grey area from your above picture, twice even. This not only causes possibly slowing overdraw (and the explicit discarding inhibits early z-testing), it will also interfere with any possible alpha-based antialiasing as described above.
- A solution to mitigate this could be to actually just draw the completely uncapped lines as rectangles from one ednpoint to the other (i.e. like your very first picture. And then drawing the connection points as additional circular large points (i.e. using a similar technique of transforming points into screen-aligned quads with the geometry shader and testing for the distance to the actual point center in the fragment shader to get pixel-perfect circles).
This has the advantage that you can actually use stencil testing to only draw the middle sections in the regions where you haven't draw other parts of the line set before and thus only the violet area, which will tremendously reduce overdraw (at the cost of more complicated rendering and per-fragment operations). This gives very nice results with an alpha-based antialiasing approach as describes above (there are however still some minor areas where there's some tiny artefacts).
This stencil testing approach might as well be used on the earlier approach where you draw the line section together with its end-caps when you don't perform any kind of antialiasing or blending, but I think to remember that it caused more artefacts than the separate rendering if you do. I haven't spend too many thoughts on this right now, though, and it might be worth some more investigation.
Those are just a few thoughts that I happened to ponder on just recently and deemed worth sharing. As said, if they are appropriate for your use-case has to be evaluated, possible with some timing and quality measurements.
