Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
@NathanReed Also the same problem is addressed in the comment section and they say it's because the HDR image I'm using has abnormally high value by, for example, directly catching the sun, which is indeed my case.
I was following Learn OpenGL's PBR-Diffuse Irradiance chapter and turns out they were converting HDR into LDR before the sampling. I must have missed that step.