No, because the underlying physics is not the same, nor the lobe shape - not to speak of their parameters such as color and Fresnel term.
Specular is really true surface interaction with the interface material/air, so it has Fresnel modulation and the internal medium has no influence on colors. But the surface condition strongly influence the reflectance, of course. Think of it as the reflections on ocean surface.
Diffuse is due to the subsurface scattering, light entering the medium, and thus gains the color characteristic and loose directionality. Think of it as the green color of the water turbidity. If depth of penetration is less than pixel size CG people don't call it "subsurface" but the physics is the same.
Of course diffuse input is what specular let pass, which is angle (and polarization) dependant.
Beside, there exist transparent medium with internal interfaces (e.g. thin shells), so case exists were light goes inside but still acts specularely (or even wavely) and not diffusely. Oppositely, most plastics and paints/varnishs and biological medium are intrinsically transparent, but contain pigments (often based on a metal atom) that cause the opacity (diffusion or specular on these "objects in the object"). Think of them as the fishes in clear water :-) .
