And then I found this "motion detection: fast and robust algorithms" paper where they propose a hierarchical and conditional motion detection algorithm... I didn't read it throughly, but they seem to have isolated well all the objects in motion, since noise was almost completely removed (last image of the paper). However they say:
As its complexity remains low, this algorithm is well suited for very light embedded systems. Future work will consider other difficult sequences with the presence of clutter like snow, rain or moving trees.
What if my intention was actually use their "morphological post-processing" technique to remove the detection of small objects instead of "stand-alone pixels" (their size would be defined according to the frame resolution)? This paper was published in 2009 and I found this related paper, also from 2009... Are there any newer updates on this or different techniques?