Simply put, what does the transparency data of an EXR file represent?
In an EXR, the convention is to store associated alpha, also known as premultiplied. The latter term however doesn't do justice to the nuances of the alpha format as compared to its evil distant cousin known as unassociated alpha, aka straight or key alpha.
In an associated alpha image, two facets are represented via the RGB triplet and its associated alpha channel:
In an associated alpha image, the alpha channel could be considered to represent a distillation of the geometry of occlusion covering a given pixel coordinate. 0.5 would represent a baked down 50% occlusion, while the RGB values would represent the degree of emission.
This also means that an RGBA triplet such as 0.0,12.4,10282.9,0.0 is entirely valid, and represents a pixel that is emitting and unoccluding. Naturally occuring things such as candles represent such a combination. Consider the following image:
The above image could not be created using unassociated alpha, and is symptomatic as to why ray tracing renderers cannot generate any alpha model other than associated alpha.
The fundamental difference between the two alpha formats is defined by the two different over operations, as described by Porter Duff.
Associated alpha over:
FG.RGB + ((1.0-FG.Alpha) * BG.RGB
Unassociated alpha over:
(FG.Alpha * FG.RGB) + ((1.0-FG.Alpha) * BG.RGB
The nuances of the difference mean that care should be taken when dealing with alpha and the RGB channels.
Generally, the "a" channel is referred to as the alpha channel. It represents the amount of a pixel that an object covers. For computer-generated imagery this is pretty straightforward.
If I generate a circle, parts of the circle on the interior will cover a full pixel, but parts on the edges of the circle may only cover a portion of the pixel. Internal pixels would have an alpha channel of 100%, whereas those edge pixels that only cover a portion of the pixel would have a value less than 100%. Outside the circle all pixels would have an alpha value of 0%.
For captured imagery, generally all pixels have an alpha value of 100%. However, if you use keying or matting tools to remove a green screen or blue screen, for example, there will be some pixels that are partially green (or blue), and they may end up with an alpha value between 0% and 100%. pixels that are solid green (or blue) will have an alpha value of 0%, and pixels that contain no green (or blue) will have an alpha value of 100%.
This is used when you want to composite one image over another. For normal compositing, you will mix the top and bottom pixels by multiplying the color value of the top pixel by its coverage or alpha, multiplying the color value of the bottom pixel by the inverse of the alpha (1.0 - alpha), and adding the two together. So it would look something like this:
composite = top * alpha + bottom * (1.0 - alpha)