One point: the Lambert BRDF is not $N\cdot L$, it's just the albedo divided by pi. The $N \cdot L$ factor comes from the $\cos \theta$ in the rendering equation. So, when sampling with a cosine-weighted distribution the $N \cdot L$s and pis will cancel out and you should just be accumulating $\frac{1}{N} \sum L_i * \text{albedo}$.

It looks like in your code you're doing this correctly for the cosine-weighted distribution. But then in the case of the uniform hemispherical distribution you're not putting in the needed factor of $2(N \cdot L)$. (The factor of 2 comes from dividing by the uniform hemispherical pdf of 1/2π, and canceling the 1/π in the BRDF.) So I think that your uniform sampling method does not give quite correct results here.

I'm not quite sure what you meant by "image doesn't converge to ground truth". Do you mean that it's still noisy even after quite a lot of samples? The problem could be in your accumulation strategy. Depending on the bit depth of your framebuffer, after some number of frames the contribution of one more frame will be so small as to be smaller than the least-significant bit of the accumulated value. If you're using 16-bit float format for instance this will happen after about 1000 frames; if using R11G11B10_FLOAT then it will take only 30-60 frames before new accumulated values will no longer have any effect. The accumulation framebuffer should be 32-bit float at a minimum.

With multiple bounces, it doesn't look like you're accounting for throughput along a path correctly. You have payload.color += bounceColor * albedo.rgb, but note that this only takes account of the albedo at the current surface. If this is the Nth bounce, the color should be multiplied by the albedos of all previous surfaces in the path—since that's how this light is getting to the camera, by bouncing through all of them. The path payload structure needs to include not just an accumulated color, but a value typically called "throughput", that contains the product of all the (BRDF * cos(theta) / pdf) factors along the path so far. On each bounce you update the throughput by multiplying in the factor for the latest bounce, then update the color by multiplying the sampled radiance with the accumulated throughput.

One point: the Lambert BRDF is not $N\cdot L$, it's just the albedo divided by pi. The $N \cdot L$ factor comes from the $\cos \theta$ in the rendering equation. So, when sampling with a cosine-weighted distribution the $N \cdot L$s and pis will cancel out and you should just be accumulating $\frac{1}{N} \sum L_i * \text{albedo}$.

It looks like in your code you're doing this correctly for the cosine-weighted distribution. But then in the case of the uniform hemispherical distribution you're not putting in the needed factor of $2(N \cdot L)$. (The factor of 2 comes from dividing by the uniform hemispherical pdf of 1/2π, and canceling the 1/π in the BRDF.) So I think that your uniform sampling method does not give quite correct results here.

I'm not quite sure what you meant by "image doesn't converge to ground truth". Do you mean that it's still noisy even after quite a lot of samples? The problem could be in your accumulation strategy. Depending on the bit depth of your framebuffer, after some number of frames the contribution of one more frame will be so small as to be smaller than the least-significant bit of the accumulated value. If you're using 16-bit float format for instance this will happen after about 1000 frames; if using R11G11B10_FLOAT then it will take only 30-60 frames before new accumulated values will no longer have any effect. The accumulation framebuffer should be 32-bit float at a minimum.

With multiple bounces, it doesn't look like you're accounting for throughput along a path correctly. You have payload.color += bounceColor * albedo.rgb, but note that this only takes account of the albedo at the current surface. If this is the Nth bounce, the color should be multiplied by the albedos of all previous surfaces in the path—since that's how this light is getting to the camera, by bouncing through all of them. The path payload structure needs to include not just an accumulated color, but a value typically called "throughput", that contains the product of all the (BRDF * cos(theta) / pdf) factors along the path so far. On each bounce you update the throughput by multiplying in the factor for the latest bounce, then update the color by multiplying the sampled radiance with the accumulated throughput.

Also, do you have any exposure control / tonemapping on the final image? An image with multiple bounces is expected to be brighter overall than the same scene with 1 bounce, as more light is being accumulated. If you don't adjust the exposure and apply a tone curve of some sort (and gamma correction), you can end up with things looking bad/wrong on screen, even if the internal HDR framebuffer is correct.