TL;DR
Yes, you can do it like that, you just have to divide the result by the probability of choosing the direction.
Full Answer
The topic of sampling in path tracers allowing materials with both reflection and refraction is actually a little bit more complex.
Let's start with some background first. If you allow BSDFs - not just BRDFs - in your path tracer, you have to integrate over the whole sphere instead of just the positive hemisphere. Monte Carlo samples can be generated by various strategies: for the direct illumination you can use BSDF and light sampling, for the indirect illumination the only meaningful strategy usually is the BSDF sampling. The sampling strategies themselves usually contain the decision about which hemisphere to sample (e.g. whether reflection or refraction is computed).
In the simplest version, the light sampling usually doesn't take care much about reflection or refraction. It samples the light sources or the environment map (if present) with respect to the light properties. You can improve sampling of environment maps by picking just the hemisphere in which the material has non-zero contribution, but the rest of the material properties is usually ignored. Note that for and ideally smooth Fresnel material the light sampling doesn't work.
For BSDF sampling, the situation is much more interesting. The case you described deals with an ideal Fresnel surface, where there are only two contributing directions (since Fresnel BSDF is in fact just a sum of two delta functions). You can easily split the integral into a sum of two parts - one reflection and one for refraction. Since, as you mentioned, we don’t want to go in both directions in a path tracer, we have to pick one. This means that we want to estimate the sum of numbers by picking just one of them. This can be done by discrete Monte Carlo estimation: pick one of the addends randomly and divide it by the probability of it being picked. In an ideal case you want to have the sampling probability proportional the the addends, but since we don't know their values (we wouldn't have to estimate the sum if we knew them), we just estimate them by neglecting some of the factors. In this case, we ignore the incoming light amount and use just the Fresnel reflectance/transmittance as our estimates.
The BSDF sampling routine for the case of smooth Fresnel surface is, therefore, to pick one of the directions randomly with probability proportional to the the Fresnel reflectance and, at some point, divide the result for that direction by probability of picking the direction. The estimator will look like:
$$
\frac
{L_{i}\left(\omega_{i}\right)F\left(\theta_{i}\right)}
{P\left(\omega_{i}\right)} =
\frac
{L_{i}\left(\omega_{i}\right)F\left(\theta_{i}\right)}
{F\left(\theta_{i}\right)} =
L_{i}\left(\omega_{i}\right)
$$
Where $\omega_{i}=\left( \phi_{i}, \theta_{i} \right)$ is the chosen incident light direction, $L_{i}\left(\omega_{i}\right)$ is the amount of incident radiance, $F\left(\theta_{i}\right)$ is either the Fresnel reflectance for the reflection case or 1 - Fresnel reflectance for the refraction case, $P\left(\omega_{i}\right)$ is the discrete probability of picking the direction and is equal to $F\left(\theta_{i}\right)$.
In case of more sophisticated BSDF models like those based on microfacet theory, the sampling is slightly more complex, but the idea of splitting the whole integral into a finite sum of sub-integrals and using discrete Monte Carlo afterwards can usually be applied too.