Native lenses without LensFun profiles.

[0000]

View attachment 707905 @junkyardsparkle Would you mind lenses for other systems turning up in this thread? I have a couple I'd like to see covered - and if I could be somewhat certain of them being added, I could provide the data (i.e. the images).

Welllll... it's not so much that I would mind... it's... complicated. There's a fairly large backlog of uploads already, and due to the way that backend was set up, it's not easy to see if there's already a waiting upload for a given lens unless the uploader happened to add that information as a comment (I probably would have made this much a mandatory part of the upload process, myself). Of course, it's not the end of the world if there's some reduplication of photo submission effort, but it still seems better to avoid it if possible.

The bigger issue for me personally is that wanting to focus my own efforts on native m4/3 lenses isn't purely a matter of "playing favorites"; it's partly a pragmatic concern for giving priority to lenses that pretty much need software correction to be usable, by design. So... given that, you can evaluate how much the lenses you're thinkng of fit into that category and triage accordingly.

btw. How about camera profiles - is that the same pipeline?

Camera profiles in Lensfun don't involve much effort beyond somebody pointing out that they're missing... they essentially consist of a model name (as given by exiv2, preferably), a crop factor (preferably precise to a couple of decimal places), and the lens mount... if you're talking about camera support in darktable, that's a separate thing.

 

[MoonMind]

Welllll... it's not so much that I would mind... it's... complicated. There's a fairly large backlog of uploads already, and due to the way that backend was set up, it's not easy to see if there's already a waiting upload for a given lens unless the uploader happened to add that information as a comment (I probably would have made this much a mandatory part of the upload process, myself). Of course, it's not the end of the world if there's some reduplication of photo submission effort, but it still seems better to avoid it if possible.

The bigger issue for me personally is that wanting to focus my own efforts on native m4/3 lenses isn't purely a matter of "playing favorites"; it's partly a pragmatic concern for giving priority to lenses that pretty much need software correction to be usable, by design. So... given that, you can evaluate how much the lenses you're thinkng of fit into that category and triage accordingly.

Camera profiles in Lensfun don't involve much effort beyond somebody pointing out that they're missing... they essentially consist of a model name (as given by exiv2, preferably), a crop factor (preferably precise to a couple of decimal places), and the lens mount... if you're talking about camera support in darktable, that's a separate thing.

Thanks for the very thoughtful and precise reply; I'll look into what results in the best benefits all around.

M.

 

[RyanM]

Fixed here, I can't close the issue on the sourceforge tracker, so maybe you can. In general, stuff related to the lens database is probably better reported at the github repository, while the SF one is still the official place for bugs related to the software itself... yeah, this is messy. I suspect if the software development ever becomes very active again, it will get moved from sourceforge, but who knows... anyway, as workarounds go, you might be better off using the exiv2 configuration file, as described here: Exiv2 - Image metadata library and tools

In exiv2, the lens is listed as "Lumix G 25mm F1.7 Asph." So does "f/1.7" need to be changed to "F1.7" as well? I guess this description suggests it doesn't: Adding new lenses to the database. Edit: I also can't seem to close the sourceforge bug. Maybe just comment that it's been fixed?

I don't think there's been any "judging" involved, there just hasn't been much attacking of the backlog there in general lately... there was mostly only one other person besides myself doing the profiling there, and I'm assuming they probably got as burned out as I did at some point... anyway, I'll make that one next on my list...

Yeah, I figured things must not be too active. How difficult is it to do the profiling? At some point I looked into it and couldn't find detailed instructions (or maybe the instructions were for linux, not windows? I don't remember). Anyways, let me know if there are any issues with the files, if/when you get around to it.

Cheers, and thank you for your work to benefit the open source photography community!

 

[archaeopteryx]

At some point I looked into it and couldn't find detailed instructions (or maybe the instructions were for linux, not windows? I don't remember).

There's four (!) sets of instructions here, all different but broadly similar. It might not be the most welcome observation but it's my inclination to cluster some of the difficulties @junkyardsparkle's described as symptomatic of poor communication. It appears

• expectations for adding lens models haven't been defined, leading to a staffing gap where submitters are frustrated by lack of status and lack of results due to volunteer burnout
• a distribution and update model hasn't been wholly articulated to end users and partners, resulting in some amount of release gaps, backlog, differences between platforms, and at least one positioning problem (exposing alphas as apparently stable releases)
• there's some divergence between SourceForge and github and I, at least, am no longer sure what issues should be opened where or managed how

Other than @junkyardsparkle, I don't have a sense of who's involved with lensfun so it's hard to suggest changes which might help. The pattern suggests relatively small investments in organization are getting dropped, which does imply low staffing. Given the number of funded projects relying on lensfun that's maybe a little odd, but is consistent with limited communication.

From a technical standpoint it seems like

• lens profile creation is underinvested and therefore too costly for the available resource base; whilst more people help in the short term in the long term they'll burn out or move on, so technology investment to reduce the effort required may help with making it sustainable
• increased availability of IBIS creates technology debt in profiling; wholly accurate correction now requires lens models covering more of the image circle and the ability to identify where within the circle an image falls
• improvements in understanding of copy variation suggest a lens profile might be better viewed as more a Bayesian prior than an always optimal solution
• within the current correction model there's some amount of "hidden" backlog as profiles aren't necessarily complete with regards to deviations or operating parameters (for the lenses I've looked at it's mainly vignetting and focus distances which are missing, but that's an anecdotal sample)

Case in point, I've been meaning to submit comprehensive image sets or maybe develop profiles for the copies of the m43 lenses I have for a while but have failed to make time to do so.

 

[RyanM]

There's four (!) sets of instructions here, all different but broadly similar.

[...]

Case in point, I've been meaning to submit comprehensive image sets or maybe develop profiles for the copies of the m43 lenses I have for a while but have failed to make time to do so.

Thanks for the link; I will have to look at these in more detail, and see if I can comprehend any of them! I'm quite a noob when it comes to coding and so forth, and hugin confuses me every time I try to use it, so finding instructions detailed enough for me to use them might be tough. Which just makes me appreciate all the more the efforts of those like @junkyardsparkle.

 

[0000]

In exiv2, the lens is listed as "Lumix G 25mm F1.7 Asph." So does "f/1.7" need to be changed to "F1.7" as well?

Nope, stuff like that gets ignored for matching purposes... just need to make sure all the "words" are there.

Yeah, I figured things must not be too active. How difficult is it to do the profiling?

For distortion and TCA correction, I would say not too difficult for anybody already comfortable with using Hugin and editing text/XML files. There can occasionally be "gotchas" that turn into... errr... "extensive opportunities for learning"... definitely start with prime lenses, and maybe hold off on vignetting correction until you're eager for more problems.

There's four (!) sets of instructions here, all different but broadly similar.

Of those, the Torsten Bronger write-up could currently be considered the closest thing to "canonical" - he's the one who set up the server for uploading calibration images, then became overwhelmed at some point, and put a call out for reinforcements... that's how I got involved...

The Andreas Schneider stuff is quite recent, and I haven't been following closely... he created a new version of Bronger's calibration script with a different set of dependencies, but still python and probably even more Linux-specific... good to see some enthusiasm for the project, though! My assessment of the situation would broadly agree with yours, with some nuances that I don't have the brain power to explicate right now... but yeah, it's a mess.

 

[archaeopteryx]

Well, from my perception mess might be a bit strong. Maybe more at an inflection or growth point where changes in approach might be good future investments. Windows offers around 75% of the near-term potential for crowdsourcing profiles so, from a capability building standpoint, investing specifically in Linux is probably not the optimal direction.

Historically, it's been more realistic to discuss installing python on Windows than PowerShell on Linux but that changed last year with PowerShell Core 6, which runs on MacOS as well and would therefore access the other main source for profiles. Might be something to consider. I'm still sorting out what that means for cross platform delivery of my projects (got UX I can't afford to port but could move to .NET Core 3) but it's the first time I've felt there's been something at least semi-user friendly that was potentially affordable to small scale, volunteer development efforts.

 

[barry]

Hi, the 12-40 is working in my RawTherapee 5.5 (Windows), and it and the Oly 45mm's listed as covered at Lensfun's coverage
The 300mm isn't listed, but I don't have one.

Thanks

I've a spreadsheet with a lensfun listing of support as of RawTherapee 5.5. @junkyardsparkle, PM me your email if interested and I can send it over. Short version:

• 46 of the 92 native mount lenses I'm tracking have some level of support.
• 25 of the lenses without support are likely low priority due to limited use: Lensbaby, Kenko, Mitakon, Rokinon/Samyang, SLR Magic, Voigtlander.
• 12 Panasonic and 8 Olympus lenses lack support, including teleconverters in the most common pairings. Of these 20 cases 11 are probably low priority.

The more significant unsupported lenses appear to me to be:

• Olympus 12-40 PRO
• Olympus 45 1.2 and 1.8
• Olympus 300
• Panasonic 100-300 II (the I is supported but either the versions vignette differently or the model for the I has substantial error relative to my copy of the II)
• Panasonic-Leica 8-18
  
• Panasonic-Leica 50-200
  
• Panasonic-Leica 100-400
• Venus Laowa 7.5

There is a model for the Olympus 14-150 I which @Baenwort might be able to assess for use with the II.

 

[0000]

Issue 2: I've uploaded calibration files for my SLR Magic 8mm f/4, but nothing's happened for >6 months (presumably because this lens has been judged "low priority"). Calibration upload b14284 · Issue #406 · lensfun/lensfun. I'd do the calibration myself, but I don't really know how. Also, it may be a moot point as I just bought a Laowa 7.5mm f/2

It's been added to the database, and since you seem to be ok with editing the XML files yourself, I'll just post the contents here, in case it doesn't get emailed to you:

    <lens>
        <maker>SLR Magic</maker>
        <model>SLR Magic 8mm f/4</model>
        <mount>Micro 4/3 System</mount>
        <cropfactor>2</cropfactor>
        <aspect-ratio>4:3</aspect-ratio>
        <calibration>
            <!-- taken with Olympus E-M5 -->
            <distortion model="ptlens" focal="8" a="0" b="-0.00737723" c="0"/>
            <tca model="poly3" focal="8" vr="1.0006528" vb="0.9994821"/>
            <vignetting model="pa" focal="8" aperture="4.0" distance="10" k1="-0.9506" k2="0.1688" k3="0.1096"/>
            <vignetting model="pa" focal="8" aperture="4.0" distance="1000" k1="-0.9506" k2="0.1688" k3="0.1096"/>
            <vignetting model="pa" focal="8" aperture="5.6" distance="10" k1="-1.0826" k2="0.8051" k3="-0.3432"/>
            <vignetting model="pa" focal="8" aperture="5.6" distance="1000" k1="-1.0826" k2="0.8051" k3="-0.3432"/>
            <vignetting model="pa" focal="8" aperture="8.0" distance="10" k1="-1.0924" k2="0.7963" k3="-0.2940"/>
            <vignetting model="pa" focal="8" aperture="8.0" distance="1000" k1="-1.0924" k2="0.7963" k3="-0.2940"/>
            <vignetting model="pa" focal="8" aperture="16.0" distance="10" k1="-1.1580" k2="0.9371" k3="-0.3760"/>
            <vignetting model="pa" focal="8" aperture="16.0" distance="1000" k1="-1.1580" k2="0.9371" k3="-0.3760"/>
        </calibration>
    </lens>

It would be good if you could test the vignetting correction with some blue sky photos or similar... in my experience there's a tendency with wide lenses to create diffuser-based calibration images that actually over-correct with real world scenes (looked ok on the window shots, as far as I could tell). Thanks for including the links to the github issue, btw... gold star calibration request.

 

[twigboy]

Quick question: these calibrations are done with the lens without any filters, protection or otherwise, correct?

 

[0000]

Well, from my perception mess might be a bit strong. Maybe more at an inflection or growth point where changes in approach might be good future investments.

Yeah, that's what I said... a mess. I'll tone it down to "messy", that's probably better. If you would like to create a channel for Windows users, be my guest. Just contact Torsten Bronger for write access to the repository. At this juncture I'm going to try to focus my own efforts on the backlog at hand. As far as the improvements you suggest, I don't disagree at all, I just have the impression that the current Lensfun devs aren't interested in activity beyond minimal maintenance, based on the lack of response to my own proposals for built-in updating, etc... YMMV.

 

[0000]

Quick question: these calibrations are done with the lens without any filters, protection or otherwise, correct?

Yep. Hopefully something like a UV filter wouldn't make much difference, but better to use just the lens... for vignetting correction in particular, nothing else should be on the lens, probably not even hoods.

 

[RyanM]

It's been added to the database, and since you seem to be ok with editing the XML files yourself, I'll just post the contents here, in case it doesn't get emailed to you:

    <lens>
        <maker>SLR Magic</maker>
        <model>SLR Magic 8mm f/4</model>
        <mount>Micro 4/3 System</mount>
        <cropfactor>2</cropfactor>
        <aspect-ratio>4:3</aspect-ratio>
        <calibration>
            <!-- taken with Olympus E-M5 -->
            <distortion model="ptlens" focal="8" a="0" b="-0.00737723" c="0"/>
            <tca model="poly3" focal="8" vr="1.0006528" vb="0.9994821"/>
            <vignetting model="pa" focal="8" aperture="4.0" distance="10" k1="-0.9506" k2="0.1688" k3="0.1096"/>
            <vignetting model="pa" focal="8" aperture="4.0" distance="1000" k1="-0.9506" k2="0.1688" k3="0.1096"/>
            <vignetting model="pa" focal="8" aperture="5.6" distance="10" k1="-1.0826" k2="0.8051" k3="-0.3432"/>
            <vignetting model="pa" focal="8" aperture="5.6" distance="1000" k1="-1.0826" k2="0.8051" k3="-0.3432"/>
            <vignetting model="pa" focal="8" aperture="8.0" distance="10" k1="-1.0924" k2="0.7963" k3="-0.2940"/>
            <vignetting model="pa" focal="8" aperture="8.0" distance="1000" k1="-1.0924" k2="0.7963" k3="-0.2940"/>
            <vignetting model="pa" focal="8" aperture="16.0" distance="10" k1="-1.1580" k2="0.9371" k3="-0.3760"/>
            <vignetting model="pa" focal="8" aperture="16.0" distance="1000" k1="-1.1580" k2="0.9371" k3="-0.3760"/>
        </calibration>
    </lens>

It would be good if you could test the vignetting correction with some blue sky photos or similar... in my experience there's a tendency with wide lenses to create diffuser-based calibration images that actually over-correct with real world scenes (looked ok on the window shots, as far as I could tell). Thanks for including the links to the github issue, btw... gold star calibration request.

Wow, you certainly made quick work of that; thanks again! I will see if I can try out the vignetting correction on some test shots.

 

[archaeopteryx]

for vignetting correction in particular, nothing else should be on the lens, probably not even hoods

Agree. A vignette diffuser which doesn't fit within a scalloped hood is potentially problematic as light from behind the camera can come through the gaps. Reduced potential for the same exists with gaps between the diffuser and lens filter ring, rings of attached filters, or flat profile hoods. Never having measured I'm unsure how flat the ends of hoods really are and filter ring knurling allows many small light leaks.

So far as intended transmission goes, I'm not aware of any cases where clear protection filters would be significant. UV and haze filters reduce purples and blues to various extents. I can't think of a mechanism where this would be significant to vignetting but there's potential for minor influence on CA corrections. My guess would be it's probably less significant than copy variation, though.

Just contact Torsten Bronger for write access to the repository.

Nice try. The two OSS projects I already volunteer for already want more time than I have. I'm not in a position to move off of either anytime soon.

 

[barry]

I had noticed that RawTherapee didn't have the Oly Pro primes before RT 5.5; now it has them, but my 17mm 1.2 Pro isn't being automatically detected (my other lenses are).
The EXIF contains 'ED' and 'Pro' in the lens description but the name in the lens table doesn't.
Anything I can do to help get that fixed?

Subscribe to see EXIF info for this image (if available)

 

[0000]

Wow, you certainly made quick work of that

Maybe a little too quick - I should have simplified the distortion line to this:

<distortion model="poly3" focal="8" k1="-0.00737723"/>

since the extra ptlens complexity isn't needed in this case. It doesn't change the correction, it's just a more concise way of crunching the numbers.

The EXIF contains 'ED' and 'Pro' in the lens description but the name in the lens table doesn't.
Anything I can do to help get that fixed?

Well, mentioning it here should have been enough, but... I was doubting my sanity because I didn't remember those being added to the github repository... and sure enough, when I checked, they weren't there. I found the commit at sourceforge where they were added a few months ago... apparently things are out of sync, so I retract my retraction of the word "mess".

Anyway, the fix is in the pipeline, thanks for the heads up.

 

[RyanM]

Since I recently picked up a Laowa 7.5mm f/2, I noticed that the current lensfun support has distortion/TCA correction, but not vignetting. I had a few minutes today, so I took some calibration images and uploaded them. If/when I get emailed a github link I will add that here for easy reference. Let me know if there are any issues with the images etc.

 

[0000]

Since I recently picked up a Laowa 7.5mm f/2, I noticed that the current lensfun support has distortion/TCA correction, but not vignetting. I had a few minutes today, so I took some calibration images and uploaded them. If/when I get emailed a github link I will add that here for easy reference. Let me know if there are any issues with the images etc.

Thanks. There's been some talk on the mailing list related to somebody trying to profile that lens recently, but nothing submitted yet as far as I know. I suspect the upload server is no longer generating github issues since October 1st, so it might be in hiberhation. All of that is run on a private server, by somebody who doesn't seem to have much time for the project these days (which is understandable). If need be, we'll figure something else out.

The project is due for some reorganization, but the hope is to hold off until after the next release. The current 0.3.95 alpha release needs more testing by software developers first, so anybody using software that uses lensfun can encourage the developers to do this (without being annoying about it, of course ).

Meanwhile, I'm hoping to spend most of the next week away from "civilization", so probably won't be responsive during that time. Anybody comfortable with running python3 programs might check out the new calibration script from Andreas Schneider here: Andreas Schneider / lens_calibrate · GitLab - it's been patched for proper vignetting correction recently, but depends on darkdable and some other packages, so check the deps list.

 

[0000]

Since I recently picked up a Laowa 7.5mm f/2, I noticed that the current lensfun support has distortion/TCA correction, but not vignetting. I had a few minutes today, so I took some calibration images and uploaded them. If/when I get emailed a github link I will add that here for easy reference. Let me know if there are any issues with the images etc.

As an FYI follow-up, I had a chance to check the upload server, and found yours, along with quite a few new ones, none of which are generating issues on github at the moment... so, while I'm not going to say that nobody should use the upload form right now, be aware that the pipeline for processing uploads is currently broken, and there's a backlog even larger than the open issues on github would suggest. You might want to wait and see if somebody else has already done the work for you.

I'll try to get to the Laowa calibration tonight or tomorrow.

 

[0000]

The Laowa 7.5mm vignetting shots contributed by @RyanM have been crunched, and the results added to the database. An updated individual lens profile can be found here. Please check the results on some real-world blue-sky shots or similar. Profiling wide lenses using this method can sometimes result in over-correction, and I have no way to check that without the lens in hand. That said, the results look great on the nicely shot test images at least, and it always makes me a little happier to see that people are capable of following a fairly complicated set of instructions to produce those.