Quoting from this web page:

For normalized spherical coordinates the gradient is:

The irradiance contribution from a direction on the hemisphere about the surface normal, specified using spherical coordinates φ [0, 2π) and θ [0, π / 2) is:

Where L is the incident radiance in the supplied direction. So the gradient vector at any point on the hemisphere:

The vector v is a unit vector on the plane of the hemisphere pointing in the perpendicular direction to the angle φ.

The page author further concludes the gradient vector of irradiance on local hemisphere(after dividing by cosine pdf) is as below

The z coordinate is 0 so I assume the gradient vector here lies on the plain of the hemisphere.

However I don't understand specifically why should lie on "the plane of the hemisphere". The two spherical unit vectors(phi and theta) are tangent to the sphere surface. My understanding is that this vector is a spherical unit vector(theta) pointing opposite the z axis.

Is it not correct to say that the gradient of the cosine is:

and the vector theta: with spherical coordinate as(explained here):

If this is correct I can't get my head around why the grad vector should lie on the plane of the hemisphere.