We are implementing a "lazy container" that caches renders of an object at various cutoffs and scales up/down the render-textures from those caches instead of drawing the actual object directly. This is done primarily is reduce the computational overhead of applying (multiple) shader filters.
Our renderer also implements a grid, which kicks in a high levels of zoom. If we render a downloaded image, then on zooming in we would find that each grid box contains only one color, e.g. it is one pixel in the image. This also applies if we apply any shader filter.
However, if we enable our "lazy" container, then we find that the grid box contain more than one color. Each texture pixel is not aligned with the grid's lines, sometimes even being smaller than one grid box.
An example of a zoomed in layer blur is given below (applied on a 320x180 image that is cached at 1024x576 on a 1024x1024 render-texture):
Is there an explanation for why this is happening?