1. Welcome to Mu-43.com—a friendly Micro 4/3 camera & photography discussion forum!

    If you are thinking of buying a camera or need help with your photos, you will find our forum members full of advice! Click here to join for free!

Native lenses without LensFun profiles.

Discussion in 'Image Processing' started by junkyardsparkle, Dec 29, 2018.

  1. junkyardsparkle

    junkyardsparkle haunted scrap heap Subscribing Member

    Nov 17, 2016
    Like, The Valley
    Starting a new thread for this, since OP of other thread seems to be looking for discussion about other stuff, but... what lens or lenses specifically? The profile-creation-from-uploads side-project for LensFun has pretty much bogged down for the moment, but I could probably find the time to fill gaps in the support for the system that I actually use...

    Let's start a list here of currently unsupported native lenses, with priority for ones being actively used by people who want to correct them in darktable or other software implementing LensFun. I'll check the backlog of existing uploads, and if there's nothing there to work with, we can discuss what's needed in that regard...

    Check this list for any lenses you think are unsupported, it might be more recent than your database: Lensfun's coverage
     
    Last edited: Dec 29, 2018
    • Like Like x 1
  2. Baenwort

    Baenwort Mu-43 Regular

    76
    Aug 22, 2017
    Wisconsin, USA
    Olympus 14-150 MK II
    Sigma 16mm F1.4

    I've submitted to them but I never seem to get it right as they are rejected after ~4 months without any good feedback on what to do better.

    I'm going to try again after the new year as there is a new highrise that just completed building that may yield better lines.
     
  3. archaeopteryx

    archaeopteryx Gambian sidling bush Subscribing Member

    771
    Feb 25, 2017
    I've a spreadsheet with a lensfun listing of support as of RawTherapee 5.5. @junkyardsparkle@junkyardsparkle, PM me your email if interested and I can send it over. Short version:
    • 46 of the 92 native mount lenses I'm tracking have some level of support.
    • 25 of the lenses without support are likely low priority due to limited use: Lensbaby, Kenko, Mitakon, Rokinon/Samyang, SLR Magic, Voigtlander.
    • 12 Panasonic and 8 Olympus lenses lack support, including teleconverters in the most common pairings. Of these 20 cases 11 are probably low priority.
    The more significant unsupported lenses appear to me to be:
    • Olympus 300
    • Panasonic 100-300 II (the I is supported but either the versions vignette differently or the model for the I has substantial error relative to my copy of the II)
    • Panasonic-Leica 8-18
    • Panasonic-Leica 50-200
    • Panasonic-Leica 100-400
    • Venus Laowa 7.5
    There is a model for the Olympus 14-150 I which @Baenwort@Baenwort might be able to assess for use with the II.
     
    Last edited: Jan 4, 2019
    • Agree Agree x 1
    • Informative Informative x 1
  4. junkyardsparkle

    junkyardsparkle haunted scrap heap Subscribing Member

    Nov 17, 2016
    Like, The Valley
    Communication is very clumsy through that channel... the person doing the calibration doesn't really have any way of directly contacting the uploader unless the uploader announces themselves on the github issue. We can discuss the criteria for images here, and hopefully it will be useful to more people.

    One of the most common things that make an image unusable for calibration is not having a single, reliably straight line running all the way across the image as close as possible to a long edge (top/bottom doesn't matter). For some wide lenses with extreme in-camera correction, it can be difficult to tell how close to the actual RAW uncorrected image edge something is until viewing the RAW file after the fact, so some trial-and-error can be needed in that case.

    It's also worth mentioning some things that don't matter: as long as the line is reliably straight and reasonably in-focus across the frame, it can be pretty much anything... the vertical side of a structure works as well as a roofline. When taking multiple shots for a zoom, they don't need to be taken from the same place, or of the same thing. It can be really difficult to find one scene that works well for both the long and wide ends. For TCA correction, it's important to have lots of good contrasty edges extending over as much of the frame as possible - the ideal would be concentric rings, but a busy cluster of bare tree branches against the sky works pretty well, too... it's important not to overexpose, better to err on the side of under-exposure.

    Re the 14-150 II: do you happen to know if the optical formula is actually different from the Mk I model? If the Mk I profile seems to work correctly, I'll just create a duplicate entry in the database...
     
  5. twigboy

    twigboy Mu-43 Top Veteran

    679
    Sep 10, 2016
    Virginia
    I know, 4/3 lenses aren't native.... the Zuiko 70-300mm is missing in the support. The 9mm BCL is listed!
     
  6. junkyardsparkle

    junkyardsparkle haunted scrap heap Subscribing Member

    Nov 17, 2016
    Like, The Valley
    I know for a fact some of those are in the database... there's a listing here: Lensfun's coverage
    One problem is that it hasn't really been made easy enough for people to update their local database... the one bundled with the last LensFun release is WAY behind by now. There's a python script included in the project called lensfun-update-data, but packagers for many distros have a perverse tendency to not include it in the base package, and I have no idea if it even runs on Windows...
    I think we were having that thought at the same time... it's fairly common for these versions to differ only in ways that don't require new profiles, from what I've seen.
    We can probably consider any 4/3 lenses that are good enough to still be in common use as "native" for our purposes... although from what I gather, many of those weren't designed with the assumption of software correction, so might not need it as badly.
     
    • Informative Informative x 2
  7. archaeopteryx

    archaeopteryx Gambian sidling bush Subscribing Member

    771
    Feb 25, 2017
    It's absent from RawTherapee's Windows install, not that it really matters as the proportion of Windows machines with python is about 0% and the process of figuring out which variant of 2.7 or 3 to install using which method is more cryptic than most users will bother with. It's also been my experience anything not part of the regular distro through PyPl is quickly broken by some update in some required wheel. So, even if the script was included and the machine did have python, it seems likely it would either fail to run or create some versioning hell where the required configuration breaks other python functionality the user needs.

    If distro inclusion was feasible a C++ or .NET based updater would be much more likely to work but there's still the problem of both darktable and RawTherapee lacking any place in their UIs from which it'd be logical to kick it off as that doesn't compose with their update models. The most realistic approach might be to flow lensfun updates as part those tools' regular releases, accepting that support levels might differ between the two since each has its own copy and follows a 6-12 month release cycle.
     
    • Like Like x 1
  8. junkyardsparkle

    junkyardsparkle haunted scrap heap Subscribing Member

    Nov 17, 2016
    Like, The Valley
    Given the state of darktable's config window, I don't think adding a tab for it there would be a crime against humanity. :hiding: ...but yeah, a standalone utility would be a great little project for somebody with cross-platform skills... meanwhile, there are tarballs available: http://wilson.bronger.org/db/version_1.tar.bz2 for 3.x versions of lensfun, and http://wilson.bronger.org/db/version_0.tar.bz2 if you're still stuck with 2.8 for some reason. The XML files can be placed in ~/.local/share/lensfun on Linux, not sure what the paths are for Mac/Windows offhand. As of this post, those are updated to 2018-09-09.

    Further thoughts on the "nativeness" of lenses: for the Sigma DN lenses that are sold in both Sony and m4/3 mounts, the profiling effort would really be better put into image samples from the larger sensor camera, since those profiles can be used for m4/3, but the reverse isn't true. Profiling primes isn't a huge amount of work (compared to zooms, anyway), so I might go ahead with the 30 and 16 that have been uploaded already, if the images turn out to be suitable...
     
  9. archaeopteryx

    archaeopteryx Gambian sidling bush Subscribing Member

    771
    Feb 25, 2017
    For Windows, darktable and RawTherapee default to
    • %SystemRoot%\Program Files\darktable\share\lensfun\version_1 (timestamp.txt has 1536512875 for darktable 2.6; that looks like the *nix epoch for 2018-09-09)
    • %SystemRoot%\Program Files\RawTherapee\<version>\share\lensfun (timestamp.txt 1536512901 for RawTherapee 5.5, a bit later on 2018-09-09)
    where %SystemRoot% is C: by default. ~ maps to %UserProfile% (%SystemRoot%\Users\<username>) in Windows. .local would be %LocalAppData% (%SystemRoot%\Users\<username>\AppData\Local) if lensfun honors this, which isn't necessarily a good choice as local means the local machine rather than the user's roaming profile. As I recall from our previous discussions, it was unclear if lensfun supported such locations on Windows.

    The Windows convention is .zip. tar is a *nix thing and, whilst some users will have some utility installed which supports whatever the latest *nix compression fad is, most won't. A more realistic update flow given the current state of lensfun distribution is therefore downloading the latest .zip from Lensfun - Browse Files at SourceForge.net and then copying the .xmls out of lensfun-<version>\data\db within the .zip (last updated to 0.3.95 on 2018-06-29, so presumably a downgrade from current darktable or RawTherapee bits). In general, it's wisest to assume the user doesn't have administrator permissions and can't make updates under %SystemRoot%\Program Files. Most users have no awareness of their AppData folder (among other things, it's a hidden directory) so a more Windows aligned approach would be to distribute a lensfun .msi or .exe which does a user specific install (recent builds of G'MIC have moved to this model, for example). I'm not familiar enough with lensfun to know if it has a way of recognizing the case where a darktable or RawTherapee update obsoletes a per user install, so that might be something to consider.

    Don't have access to a Mac with either raw developer installed. But maybe someone else could take a look.
    From what I can tell, the Windows UI for darktable may lack such a config window. The closest thing seems to be the options panel for the lens correction module.
    darktable lens correction.png
    Subscribe to see EXIF info for this image (if available)
     
    • Informative Informative x 2
  10. junkyardsparkle

    junkyardsparkle haunted scrap heap Subscribing Member

    Nov 17, 2016
    Like, The Valley
    NOOOOOOOOOOOO! :eek:  Those are release packages, so the ones with compatible versions of the XML files will be no fresher than what came with the lensfun you have... not only that, but the release that SF defaults to in all its wisdom is actually an ALPHA release, with a db version that isn't even compatible with the current stable release. It would probably be better if you nuke that link, it will only confuse people. But yeah, somebody should make zip files available, agreed. I'll dig around for the userspace path for the db files is on Windows, I'm fairly sure there is one...
     
    • Useful Useful x 1
  11. junkyardsparkle

    junkyardsparkle haunted scrap heap Subscribing Member

    Nov 17, 2016
    Like, The Valley
    Apparently it's the expected "/Users/[user]/.local/share/lensfun/" but I can't test it either.
    I meant the "global preferences" window that pops up when you click on the little gear icon at the top of the main window... but I'm not sure darktable wants that gig anyway.

    So, asking on the official darktable-for-Windows thread at pixls.us, the response was "darktable/share/lensfun"... so apparently no software on windows uses any standard, shared database... so much for all that. To avoid further head-hurt, I'm just going to stick to thinking about profile creation for now... :rolleyes: 
     
    • Funny Funny x 1
  12. junkyardsparkle

    junkyardsparkle haunted scrap heap Subscribing Member

    Nov 17, 2016
    Like, The Valley
    I found the uploaded files for this one, they were pretty much ideal for calibration, so I'm not sure what the "rejected" stuff was... I don't run the upload server. Anyway, I'm sorry you had to wait so long, but it's in the official repository now, and meanwhile I also uploaded a database file with only that profile here. You can put it in ~/.local/share/lensfun and fire up darktable to test it. It should be automatically selected, but sometimes there's name matching weirdness; let me know if that seems to be the case.
     
    • Winner Winner x 1
  13. Baenwort

    Baenwort Mu-43 Regular

    76
    Aug 22, 2017
    Wisconsin, USA
    Which name does it use for matching? I can supply the EXIF tag name if that helps get it right.

    The rejects, I'm not sure of the cause. The one you closed was the second set for that lens. I try to follow the directions each time and I change the building I use.
     
  14. Baenwort

    Baenwort Mu-43 Regular

    76
    Aug 22, 2017
    Wisconsin, USA
    I did and tried to talk but they just kept linking the same instruction page I was following already.

    As to Mark I vs II on the 14-150 I think from the DPreview announcement
    There is only a coating and seal changes. Looking at the listing the number of elements and groups are the same. However the newer one mentions "HR" elements and the Mark I does not. DXo does rate the Mark II as sharper than the Mark I and that it has less vigilantism.

    I'm happy to shoot some more images after the new year. There is a new building that I have a long approach to that would be great across the entire zoom range. It is an architectural nightmare but it has lots of crossing straight lines.
     
    Last edited: Dec 30, 2018
  15. junkyardsparkle

    junkyardsparkle haunted scrap heap Subscribing Member

    Nov 17, 2016
    Like, The Valley
    Proper lensfun, as used by darktable at least, matches the supplied model name against the database entries in a fuzzy manner described here: lensfun: Declaring lenses: \<lens\>

    What name is supplied is where things can get a little wonky sometimes, due to how manufacturers use the LensID Exif tag... it's up to the software to map this numeric identifier to a string, and sometimes there are collisions or confusion, especially now that people are using "smart" adapters with lenses manufactured for one brand/namespace on cameras from a different one... that said, the model name I used for the profile works for me in darktable built against exiv2 2.5... older versions may not have the information. Check the "image information" panel in darkroom mode if it's not working automagically.
    Profiles for super-zooms are an order of magnitude more work than for primes, so try manually selecting the Mk I profile first, and if it doesn't seem to be giving good results, we'll talk. :D 
     
  16. archaeopteryx

    archaeopteryx Gambian sidling bush Subscribing Member

    771
    Feb 25, 2017
    Hmm, the most definitive statement I can find is Robin Wong's that the I and II are "optically similar" with the increments being weather sealing, ZERO coating, and revised body. Olympus' legacy specifications on the I suggest the exotic elements are the same as II but are disclaimed:
    DxOMark also mentions the I and the II are "the same" internally. Imaging Resource says 2 AS and 2 ED for the I which is potentially consistent with the II depending on how they counted the elements (the II has one DSA, EDA, and AS for aspherics and an EDA, ED, and EHR for ED). Photography Blog mentions a DSA and an EDA on the I.

    So, best guess, the I and II might want somewhat different corrections due to differences at the HR and EHR elements. Be interesting if any such difference proves distinguishable from copy variation and Olympus' enthusiasm for having lots of different names for element types.
     
  17. junkyardsparkle

    junkyardsparkle haunted scrap heap Subscribing Member

    Nov 17, 2016
    Like, The Valley
    Yeah, correction for zooms, and especially "superzooms", is never going to be perfect anyway - some of the calibration shots people have uploaded for fixed-lens cameras had image circles that were way off center - so even if the existing profile doesn't produce perfect correction on a given copy, it still doesn't absolutely mean it's the profile that's "wrong". On the other had, if anybody can confirm that it produces good correction on their copy, we can pretty much call it good. Slight differences in TCA would probably be more visible than slight changes in distortion, so that would be good thing for somebody who owns this lens to check for. Look at high-contrast (but not blown out) areas in corners, especially.
     
  18. RyanM

    RyanM Mu-43 Veteran

    334
    Jun 16, 2017
    Not sure if this is necessarily the place for it, but if the topic at hand is "issues with Lensfun for m43 use", I have a couple topics to mention.

    First, I'll note that I'm using darktable 2.4.4 on windows (I've downloaded the new version 2.6, but haven't gotten to the install yet).

    Issue 1: The Pan 25mm f/1.7 has a correction profile in lensfun, but the name doesn't match the one that darktable recognizes from the EXIF info read by Exiv2 (described in my bug report here: Lensfun / Bugs / #96 Panasonic 25mm f/1.7 name doesn't match Exiv2). As a result, automatic correction doesn't work properly. I've found a workaround my modifying my local copy of mil-panasonic.xml, but it's a bit of a pain every time I update.

    Issue 2: I've uploaded calibration files for my SLR Magic 8mm f/4, but nothing's happened for >6 months (presumably because this lens has been judged "low priority"). Calibration upload b14284 · Issue #406 · lensfun/lensfun. I'd do the calibration myself, but I don't really know how. Also, it may be a moot point as I just bought a Laowa 7.5mm f/2

    Cheers
     
  19. junkyardsparkle

    junkyardsparkle haunted scrap heap Subscribing Member

    Nov 17, 2016
    Like, The Valley
    Fixed here, I can't close the issue on the sourceforge tracker, so maybe you can. In general, stuff related to the lens database is probably better reported at the github repository, while the SF one is still the official place for bugs related to the software itself... yeah, this is messy. I suspect if the software development ever becomes very active again, it will get moved from sourceforge, but who knows... anyway, as workarounds go, you might be better off using the exiv2 configuration file, as described here: Exiv2 - Image metadata library and tools
    I don't think there's been any "judging" involved, there just hasn't been much attacking of the backlog there in general lately... there was mostly only one other person besides myself doing the profiling there, and I'm assuming they probably got as burned out as I did at some point... anyway, I'll make that one next on my list...
     
    • Appreciate Appreciate x 2
    • Winner Winner x 1
  20. MoonMind

    MoonMind Mu-43 Top Veteran Subscribing Member

    975
    Oct 25, 2014
    Switzerland
    Matt
    @junkyardsparkle@junkyardsparkle Would you mind lenses for other systems turning up in this thread? I have a couple I'd like to see covered - and if I could be somewhat certain of them being added, I could provide the data (i.e. the images).

    btw. How about camera profiles - is that the same pipeline?

    M.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.