1
$\begingroup$

I would like to programatically identify pitch and roll numbers that will level the horizon on each frame in a series of equirectangular 360 panorama images.

Here's why. I recently ended up with a repaired 360 video file from an insta360 camera that had lost its horizon data (due to a camera bug preventing a smooth end to the recording). The video consists of equirectangular images (30fps).

To complete the repair of the file, I ended up using Premiere Pro, manually identifying the horizon at time intervals through the video, and applying a spherical rotation to make it (about) right using keyframes, and then exporting a somewhat higher resolution rotated equirectangular video. I was then able to import the "fixed" video into the usual insta360 toolset.

I noticed while doing this edit that you can spot by eye whether a 360 panorama image is correctly levelled even while looking at the rather weird-looking equirectangular image. The distortions if the horizon is "off" are very recognisable, and verticals should be vertical if the video is level. I speculate that a machine could easily identify these distortions in such an image without resorting to AI, via some sort of image decomposition to infer an orientation for each frame.

The reason I was able to reorientate the images was - I guess - because (perhaps focusing on higher spatial frequency components in a scene) there are always a lot of objects with verticals and horizontals (trees, poles, signs, buildings etc).

I am interested in hacking together some code that can identify an orientation for an agreed subset of frames in a video and spit out a text file with those orientations (presumably roll, pitch, yaw). Afterwards there could be a second pass on those numbers to ensure that the yaw is not all over the place.

Can anyone suggest an approach to doing this? I have coded in various languages over the years (though not really Python or Java but I guess I could learn); it would be good to do this using readily available languages and tools (unix/web or PC).

Update

I did some further investigation and found the following two papers, free download links provided:

These two papers discuss the Hough Transform, which is a method of identifying straight lines in an image, and adapt it in (I think) different ways to equirectangular images. However the focus is on highly rectilinear environments like buildings. I was more interested in action camera footage myself (for example ski footage). I suspect the straight lines won't be of such high quality in this footage.

This work was presumably state of the art in 2012/2016 which makes me less hopeful that a solution to my problem is in the public domain.

I think what I am looking for is an image transformation that allows me to identify the dominant great circles in the equirectangular image.

Here is what those great circles look like (based on one of the above papers): Equirectangular great circles

And here are a couple of examples of the images where I'd like to detect these great circles: Example frame 1 Example frame 2

I feel there should be fairly simple approach to spotting these curves.

Thanks in advance...

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.