I often hear that tessellation shaders aren't great for performance reasons. Something along the lines that if you try to use a tessellation shader, you'll find that the data you get out isn't sufficient and you'll have to re-do calculations anyways. Or it might have been that this path was less efficient than going another route and wasn't a good use of bandwidth?
Can anyone explain the details of why tessellation shaders are not advised, and if there are any exception cases where tessellation is a good thing?
It's bad on mobile: ARM doesn't have a tessellation unit so is emulated in software. Tiled based architectures will write vertex data to main memory before pixel shading which is a bandwidth issue. qualcom switches to direct rendering mode if you enable tessellation which isn't cheap.
It's bad on AMD / Console
There is complexity in setting the tessellation factors to give good results
it sounds like it's difficult not to get too much resolution in some ways (aliasing), and not enough in others (low quality results). You essentially need to know screen space derivatives of vertices to calculate good tessellation factors, and this is not easy to get even decent approximations of.
A bummer about tessellation shaders is that they basically work by sampling discrete sample points (at vertices) so for a better result, you need a denser mesh to start with, which sort of defeats the purpose of dynamic tessellation. Can be addressed a bit with mipmapping type strategies.
Tessellation is still used by major games in specific cases though, and terrain does seem to be one usage case