360 photo sphere

Discussion in 'Image Processing' started by Nathanael, Sep 12, 2016.

  1. Nathanael

    Nathanael Mu-43 Veteran

    436
    Oct 12, 2015
    Anyone have experience with stitching these things? Kind of weird to describe, but I'm sure most of you have seen it before. Basically you take enough photos to cover a full 360 degrees in all directions (or use a special 360 camera), stitch them together and then project it in a virtual reality sphere. So you have to pan around the image to see the full 360, like Google Street View.

    Anyways. I made a pretty sloppy first attempt with a total of 5 photos with my 7.5mm. I wanted to share using the new Facebook feature which presents your image in the interactive panning setup. I got it to work, but i used a mixture of lightroom defishing profiles, photoshop photomerge, extensive warping with adaptive wide angle, blending, cloning, etc. Messy but eventually I filled up the 2:1 canvas with something. But not projected correctly so the top of the sphere is super distorted. (of course) Definitely need more than 5 frames next time. Feels like the fisheye effect messes things up as you tilt up or down though..

    Ok SHORT VERSION: What is the best way to orient a fisheye when taking the photos. What's the best way to put those into a 2:1 equirectangular projection. It needs to be exactly 2:1 to be recognized as 360 photo.

    @wjiang@wjiang I feel like you might be the man with my answers.


    Here's what I came up with for a first attempt

    https://www.facebook.com/natemyton/posts/10208896183955965
     
    Last edited: Sep 12, 2016
    • Like Like x 3
  2. christofp

    christofp Mu-43 Regular

    143
    Jul 21, 2012
    I use the 7.5 fisheye too. But defishing is the wrong way, the world is a sphere, not flat. Instead, you should stay with a fisheye or sphere projection until the final projection. If your tool does not support that, it's the wrong tool.

    I take 6 shots in portrait orientation, one additional sky and one floor. I set exposure and white balance to a fix value. All shots from tripod and with a L-bracket and a short macro rail for correct axis.

    I import them to opensource stitcher "Hugin", select equirectangular projection and stitch. In a second workthrough, I have to mask out tripod leg relicts (or my fingers if I haven't been careful enough), correct horizon if it was not detected in first attempt, but thats it. Perfect result.

    Christof
     
    • Agree Agree x 2
    • Appreciate Appreciate x 1
  3. wjiang

    wjiang Mu-43 Hall of Famer Subscribing Member

    Exactly as christofp said - you need a tool that stays in a 3D spherical working space until final output, I too use Hugin. It's the same technique for making full 'little planets' (just a different output projection).

    I'll add that if you prefer PS's edge merging algorithm or need to do touch ups (Hugin isn't always perfect), you can still output the individual reprojected frames from Hugin in addition to the blended result.
     
    • Appreciate Appreciate x 1
  4. Nathanael

    Nathanael Mu-43 Veteran

    436
    Oct 12, 2015
    THANK YOU! @christofp@christofp and @wjiang@wjiang

    I made it difficult by taking too few pictures which made the merging in Photoshop a hard time. I knew that Hugin was good for re-projecting single images, but didn't know that it would do the stitching and everything as well.

    I'll give it another shot. Thanks guys.
     
  5. christofp

    christofp Mu-43 Regular

    143
    Jul 21, 2012
    Ha, for me the experience with Hugin was the other way round: I did not know how to defish a single photo in Hugin. The wizard always asked for more than one photo. 3 months later I discovered that there is a expert-mode UI too ... :dash2:

    I watched your 360° photo now, very cool. I recently started taking astro photos again but your location is really wonderful!
     
  6. inkista

    inkista Mu-43 Veteran

    345
    Jan 13, 2012
    San Diego, CA
    I resurrect a dead thread. :)

    Yes. Sadly. Too much, probably.

    Portrait orientation gets you more vertical coverage, so having the camera in portrait always works best. You can always get more horizontal coverage by rotating in yaw and shooting more images. As was stated up-thread, as a beginner, you'll want at least 6 images around at 60º intervals, with a zenith (straight up) and nadir (straight down) shot. Also, as stated, you'd probably prefer a tripod and panohead calibrated to rotate around the NPP [no-parallax point], particularly if you're shooting smaller scenes indoors. I like my Nodal Ninja. But if you have to go handheld, a plumbline is another way to orient your camera.

    You could probably cover the scene "baseball"-style with two tilted up and two tilted down, but you'll have minimal overlap, and it's very very VERY easy to end up with gaps in the coverage. Best I could get away with was four tilted down and a zenith and it was pretty hit'n'miss.

    You need a stitcher that does equirectangulars directly from fisheye images. As the other folks in the thread have mentioned, Hugin is a good open source package for this, but if you decide to do this as a regular thing, then a PTGui license might be worth paying for. PTGui has some better algorithms, and well as much better layer/mask tools for erasing ghosts and clones, and the viewpoint correction tool for nadir patching is outstanding.

    Another useful software package to have in the toolbox is Pano2VR, which can create interactive output as HTML5 or Flash files, as well as remap the equirectangular to cube faces and vice versa, if you need to do some additional patch/mask work for tripod or ghost/clone erasure.

    You may also want to play with remappings. Obviously, the stereographic little planet is a very popular way to go and it's easily done with Hugin.


    One fun Photoshop plugin to have if you have a collection of equirectangulars is Flexify, which has a plethora of remappings you can play with. My favorite is probably the origami balloon mapping where, if you can fold an origami balloon, you can turn your pano into a paper cube:

    2499734561_a529ebc996_z.

    And if you want to stick with open source, then Gimp's Mathmap filter and the Quincuncial script can lead to hours of headbanging. :)


    4529295991_7348f6435b_o.
     
    • Informative Informative x 2
    • Appreciate Appreciate x 2
  7. Nathanael

    Nathanael Mu-43 Veteran

    436
    Oct 12, 2015
    Thank you for a thorough run-down. Trying these again has been on the back burner but I should give it another stab sometime. Anyways always appreciate when people take the time to break down a complicated process for a beginner, so thanks :)
     
  8. inkista

    inkista Mu-43 Veteran

    345
    Jan 13, 2012
    San Diego, CA
    You're welcome.

    I've also written up two stackexchange answers on the subject of 360x180 panos that might be helpful:

    What are the best techniques to take 360° panoramas?

    How are virtual tour photos taken?
     
    • Informative Informative x 2
    • Appreciate Appreciate x 1
  9. 3dpan

    3dpan Mu-43 Regular

    80
    Mar 11, 2017
    Far North, New Zealand
    Alec
    I've used QTVR via Pano2VR. With a 180 full frame fisheye you need about 5 frames for full 360 X 360.
    You can zoom in and out and of course do the full rotation.
     
  10. inkista

    inkista Mu-43 Veteran

    345
    Jan 13, 2012
    San Diego, CA
    IIRC, the Samyang/Rokinon/etc. 7.5mm f/3.5 yields 135º HFoV. So, only 180º diagonally. If shaved and put on an APS-C body, it can work for four around at 90º intervals to cover the sphere, but for MFT, won't happen. If you do have a 180º HFoV lens, you don't actually need the zenith. I used to be able to cover the sphere with a Sigma 8mm f/3.5 circular fisheye for full frame on my APS-C/Full-Frame Canon bodies with only 4 shots around at 90º intervals, no zenith, no nadir required for coverage (just patching).
     
  11. 3dpan

    3dpan Mu-43 Regular

    80
    Mar 11, 2017
    Far North, New Zealand
    Alec
    I used the ZD 8mm fisheye on an E-3, I don't actually recall if I used 5 or 6 frames.
    The Pano2VR software is/was very powerful stitching, I suspect it actually used PTGui.
    Cheers,
     
  12. inkista

    inkista Mu-43 Veteran

    345
    Jan 13, 2012
    San Diego, CA
    I think you may be confused. Pano2VR doesn't do any stitching, it needs an already-stitched panorama as input. They say so right here on their messageboard. Pano2VR is for turning an equirectangular into in interactive VR-presentation format, such as QTVR cubic, Flash, or HTML5. But it can also be used for remapping to and from other 360ºx180º formats (cube faces, cross, equirectangular, etc.)

    While you can accomplish some of these tasks with Hugin, it's not necessarily as easy as using a dedicated tool like Pano2VR. Hugin was built by geeks for geeks, and a bit of command line/scripting know-how can often be necessary.
     
    • Informative Informative x 1
  13. runner girl

    runner girl Mu-43 Veteran Subscribing Member

    213
    Nov 26, 2011
    I have made many using PTGui and a fisheye lens. However, I always shoot a minimum 30 photos to stitch, sometimes 60+. That software allows you to go back and match or change control points to tweak it. You can download a trial to see how it works. Also be sure to note whether you're using a fisheye before you merge.
     
  14. 3dpan

    3dpan Mu-43 Regular

    80
    Mar 11, 2017
    Far North, New Zealand
    Alec

    You're absolutely correct. I had forgotten. (called Oldtimers Disease) :confused-53:
    Use Autopano Pro to create the initial panorama. That's the very powerful stitching software. I didn't even use a tripod for my fisheye images.
    Cheers,