For each texel of the generated diffuse cube map (which has $6$ faces times $32 \times 32$ texels), the diffuse equation combines all texels of the environment map (which has $6$ faces times $64 \times 64$ texels). Thus $32 \times 32 \times 6 \times 64 \times 64 \times 6$ as stated.
Since the article considers SH with $9$ coefficients, and each coefficient needs to combine all texels of the environment map, that's $9 \times 64 \times 64 \times 6$.
"Bruteforce" doesn't necessarily refer to graph theory. In this article it refers to the naive approach of going through every single texel of the diffuse map and each time combine every single texel of the environment map.
Let's describe the problem at hand.
As an input, we have an environment map: it's a set of six textures forming a cube, that represents the light coming from all directions. The light is assumed to be coming from sources far enough that, at our scale, it's as if they were infinitely far. An environment map typically looks like the left figure of this illustration from the GPU Gems article:
As an output, for a any point of a given object that we're rendering, we want to know the diffuse irradiance affecting it. What does that mean?
Models describing how the perceived color of a surface varies depending the light reaching it, typically consist in integrating all incoming light over an hemisphere, with a weight that depends both on the direction of the light and of the observer.
For opaque materials, it is common to consider two components contributing to that color: the diffuse and the specular. The diffuse is due to light slightly penetrating the surface, getting partly absorbed, then getting out in random directions. The specular is due to light bouncing off the surface without entering it.
Back to the problem
So we want to know the color of the points of an object, we know that to compute it we need to integrate light in all directions, and we have a cube map that represents light coming from all directions. So far so good.
A problem though is integrating basically means we need to go over every single texel of the hemisphere (so half the environment map: $(64 \times 64 \times 6) / 2$ texels), do some math, and add that to the result. That's a lot of computation for each point, which we'd like to avoid if possible.
We know that the contribution of one light depends on the light direction. If we consider the environment to be static (the lighting doesn't change), then we can isolate the part of the computation that only depends on the light and surface normal (and not on the material or the observer), pre-compute it and store it to use later. For the diffuse term, that's the diffuse irradiance mentioned earlier, and it typically looks like the center figure of the illustration. Each pixel represents the irradiance term for a given surface normal. The right figure is the specular environment map, computed in a similar way but with a different integral.
With that texture, instead of painfully integrating, we just fetch the precomputed result, combine it with the rest of the equation (the material diffuse color, etc.) and voilà.
Finding the 32x32x6x64x64x6
Since the result looks very blurry, it probably doesn't need to be $64 \times 64$ for storing, and the article assumes $32 \times 32$. But still, each of those $32 \times 32 \times 6$ texels' color is computed separately, by integrating over the entire corresponding hemisphere. That part never disappeared: we just moved it from the rendering stage to a pre-computation stage.
So that's still $(32 \times 32 \times 6) \times (64 \times 64 \times 6) / 2$. That's half the figure stated in the article. I suppose the author was either assuming a model that integrates of the entire sphere, or just dropped the $0.5$ factor for the sake of clarity.
Finding the 9x64x64x6
We can't help but notice this irradiance diffuse environment map is really blurry: surely there must be a way to store its information using a lot less than $32 \times 32 \times 6$ terms, and hopefully reduce the amount of work as well. That's what spherical harmonics do, by compressing the irradiance in just a few coefficients.
I am not familiar enough with the math to engage in an explanation, but suffice to say it is generally considered that for diffuse, order three SH are sufficient. Order three SH involve nine coefficient, and again, for each of these we need to integrate over the entire hemisphere.
So that's $(9 \times 64 \times 64 \times 6) / 2$ texels to fetch and process. Here too a $2$ factor is missing from the article.