Re-projection of fish-eye images (aka 'defishing') using Hugin

Discussion in 'Image Processing' started by wjiang, Mar 1, 2016.

  1. wjiang

    wjiang Mu-43 Hall of Famer

    I've had a number of people ask about how to 'de-fish' an image taken with a fish-eye lens (e.g. the Samyang 7.5mm f/3.5). I thought I would explain the process that I use to do this using the cross-platform, open source software called Hugin.

    'De-fishing' is the act of taking a fish-eye planar projection of a spherical view (i.e. the real world) and re-mapping it into a different planar projection, for the purposes of changing the distortion characteristics of the image. Changing is the key term here - there will always be distortion when trying to map a spherical view of the real world down to a flat plane, but we can change what the nature of the distortion is to suit the subject matter.

    There are a wide range of planar projections. Rectilinear is the one we are most familiar with in the photography world, but it is no more 'ideal' than any other. In fact, with a 180 degree angle of view fish-eye, a rectilinear projection is one of the most hideous options. This is why most wide panoramas use some form of cylindrical projection that preserves vertical straight lines at the expense of horizontals. You can even turn a rectilinear ultra wide into a more natural looking projection to reduce the stretching at the edges.

    From the Hugin Manual (which gives an overview of some more common projections):
    Example Use Case
    Here we have an example image taken with the Samyang 7.5mm fish-eye lens.


    On the left is the original image after basic RAW processing and export to 16-bit TIFF, on the right is the re-projected output from Hugin (resized and converted to JPEG for web). There is obviously less of the bulbous fish-eye effect, but also notice that the verticals are straight, but the horizontals are... not quite. This is an example of the highly flexible Panini General projection supported by Hugin.


    Open up Hugin using the default simple interface.


    Load your image (TIFF or JPEG typically). It will ask you what type of lens it came from. If it's an automatic lens it will contain some of the lens data, but you probably have to fill some of it yourself anyway. In our case we have a Full frame fisheye, 7.5 mm focal length, and being m4/3 have an approximate focal length multiplier of 2. Don't worry about HFOV, that gets calculated automatically.


    Check that your image loaded with the correct parameters. Hugin is actually also an HDR panorama tool, so it will complain about you only having one image for a panorama. This isn't actually a problem - we can ignore it. By default the preview shows an equirectangular projection, not your original - don't worry at this stage.


    The most exciting part is in the projection tab.


    Select the projection type in the middle (here Panini General). Some projections have adjustable parameters on the right. In the case of Panini General, the Cmpr parameter allows you to go from fully cylindrical projection at 100 to fully rectilinear at 0. I picked 60 here, the best thing to do is to play around with the projections and their parameters, and look at the preview to determine what you like.

    Note the fit button on the left, you can click that a few times to get the preview to show you the whole image - you generally have to do this after changing any settings.


    That previous one was good, already much less fishy, but seems I didn't shoot it straight. Never fear, the next tab over allows you to fine tune the yaw, pitch and roll. You can also drag using the mouse, but I find this hard to fine tune and use the numbers up top. Once again, you can use the fit button to adjust the preview.


    Okay, so the image got levelled and straightened. We still seem to have some barrel distortion in the verticals though, even though the Panini General is supposed to have straight verticals. What's wrong? 2x crop factor isn't exactly 2x! This will vary from camera to camera, I had to go back to the first tab and in this case settled on a 1.85x crop factor to produce straight verticals.


    We're pretty much done. In the final tab you can use autocrop to maximise the rectangular output that you get. You can crop in Hugin but I prefer to fine tune the cropping in a proper editor...


    To export, launch the panorama editor view...


    Go to the final stitcher tab.


    I tend to just click all the buttons on the right to auto-optimise cropping and size. For most projections this works but for some Hugin cannot do it automatically. The main thing to pay attention to is the resulting area left after the crop - if this is too small (and it will be for some projections), you will need to manually increase the canvas size and redo the fit/autocrop in the preview window so you actually get a decent number of pixels at the end of it.

    For our use case we want LDR, TIFF/JPEG, then stitch away!


    At this point you can still go back and retune settings in the preview window and restitch. Once you're happy, you can save the settings in a file for latter use (e.g. in case you re-process the RAW).


    Hugin is a challenging tool, but very powerful. Remember that different images will suit different projections. Some of the important factors are:
    • horizontal versus vertical FoV
    • horizontal or vertical straight lines
    • important subjects in the middle of frame versus edge of frame
    • natural free-form elements (especially people) versus man-made structural elements (e.g. buildings).
    Once you start playing around a bit more, you will hopefully start to get the hang of visualising what is possible after re-projection and frame the original fish-eye image accordingly.

    Have fun!
    Last edited by a moderator: Aug 23, 2016
    • Like Like x 3
    • Appreciate Appreciate x 2
    • Informative Informative x 1
    • Useful Useful x 1
  2. TassieFig

    TassieFig Mu-43 Top Veteran

    Oct 28, 2013
    Tasmania, Australia
    Thanks for this!!! Great stuff. I did most of it right but I just couldn't get the look I wanted. I will try with a few more pics and see what i can get out of it. :thumbup:
  3. wjiang

    wjiang Mu-43 Hall of Famer

    It really varies based on what the starting point looks like. Some images work, some don't. Some I just leave it as fisheye but do a small straightening out, some I reproject to a more pleasing stereographic (but still fishy) projection. Like I said, once you have encountered enough different scenarios, you start to get an idea of what re-projections might be possible and can frame accordingly.
    • Like Like x 1
  4. Amin Sabet

    Amin Sabet Administrator

    Apr 10, 2009
    Boston, MA (USA)
    I just converted this resource to a thread where it can get more attention. These were the reviews I had to delete:

    • Appreciate Appreciate x 2