1. Reminder: Please use our affiliate links for holiday shopping!

Rawdigging with ISO 100 & 200

Discussion in 'Olympus Cameras' started by pdk42, Feb 14, 2014.

  1. pdk42

    pdk42 One of the "Eh?" team

    Jan 11, 2013
    Leamington Spa, UK
    Hi all,

    This is a long post, please accept my apologies... If you find it boring, move on :wink:


    Having recently picked up an E-P5 and upgraded my E-M5 to the latest firmware, I'm now the proud owner of two cameras with an ISO 100 ("LOW") capability. I was curious to see what is going on so did a bit of experimentation. Here's a shot I took of my E-P5 with the E-M5:

    [​IMG]
    General view - ISO 200, 1/10s f4, EV+0.5. LR5 processed from raw. Corrected for WB, but no additional sharpening or EV correction.

    It's at EV +0.5 to counteract the large amount of white in the image. The camera's over/under "blinkies" showed only a little overexposure on the edges of the EP-5 top plate. In fact, when I opened the file in Lightroom and turned on its clipping viewer, the area of overexposure almost exactly matched what was on the screen.

    I then took two more shots:

    - ISO 100 EV+0.5 - 1/4s f4
    - ISO 200 EV+1.5 - also 1/4s f4 (i.e. overexposed by an additional 1 stop).

    I then pulled the ISO 200 EV+1.5 back by 1-stop in LR. The images all look pretty much similar:

    [​IMG]
    Left to right - ISO 100 EV+0.5, ISO 200 EV+0.5, ISO 200 EV+1.5 & pulled back -1EV in LR

    And here are the LR clipping regions for all three:


    [​IMG]
    Screengrab from LR showing clipping on the top-plate. ISO 200, 1/10s f4, EV+0.5

    [​IMG]
    ISO 100 EV+0.5

    [​IMG]
    ISO 200 EV+1.5 & pulled back -1EV in LR

    The fact that the clipping is nearly identical on both the ISO 100 and overexposed ISO 200 shots is pretty incriminating evidence as to how the ISO 100 is being implemented.

    I then decided to let Rawdigger take a look. For those who don't know Rawdigger, it's a program that gives a statistical view of the pixel values in raw images. The results were a little surprising (to me at least!):

    [​IMG]
    Stats from image. ISO 200, EV+0.5

    [​IMG]
    Stats from image. ISO 100, EV+0.5

    [​IMG]
    Stats from image. ISO 200, EV+1.5

    You can see that the ISO 100 and ISO 200 +0.5EV have over 50% of the pixels apparently past the max value (i.e. completely blown). However, neither the camera's on-screen blinkies, nor the LR clipping view reported this at all. The max value was auto-determined by Rawdigger as 3792 and mousing over the image reveals that large areas are indeed all showing levels of 3792 - i.e. blown. This didn't make sense...

    So, I then went back to LR and tried to recover some detail in the "blown" areas. The camera was resting on some mounting board and looking at the ISO 200 non-overexposed image, I found that by cranking down the highlights and pulling the exposure back by -1.5 stops, I could find some detail:

    [​IMG]
    Recovering highlights - ISO 200, EV+0.5. Highlights -100, Exp -1.5

    I then did the same thing on the ISO 100 and over-exposed ISO 200 shots (the ones that Rawdigger says are completely blown!):

    [​IMG]
    Recovering highlights - ISO 100, EV+0.5. Highlights -100, Exp -1.5

    [​IMG]
    Recovering highlights - ISO 200, EV+1.5. Highlights -100, Exp -2.5

    So, I've successfully recovered details in these two shots too - which doesn't tally at all with what Rawdigger is saying, but does with what LR says. Pixel peeking on all three didn't reveal any significant difference in the level of detail either.

    And just to underline what Rawdigger is saying, here's its clipping display:

    [​IMG]
    RRawdigger clipping display

    Now, I know this is only a single test and since it's under artificial light maybe the DR isn't wide enough to uncover problems, but here are my conclusions anyhow:

    1 - Rawdigger is getting it wrong!
    2 - Shots at ISO 100 and ISO 200 overexposed by 1 stop then pulled back in LR give the same results
    3 - Highlights can be recovered from both ISO 100 and overexposed ISO 200 shots to a level pretty much the same as a normally-exposed ISO 200 shot
    4 - I'll have no hesitation using ISO 100 in the future

    You still there (and awake!).

    Hope it makes sense and at least one other person finds it useful!!
     
    • Like Like x 8
  2. klee

    klee Mu-43 Veteran

    367
    Mar 20, 2013
    Houston, TX
    Kevin
    I'm appalled! following this thread.

    this kinda stuff genuinely interests me though. I wonder what's going on.
     
  3. MAubrey

    MAubrey Photographer

    Jul 9, 2012
    Bellingham, WA
    Mike Aubrey
    Well...this is basically exactly what DxO found. Not too surprising.
     
  4. b_rubenstein

    b_rubenstein Mu-43 All-Pro

    Mar 20, 2012
    Melbourne, FL
    Now that we are well into the second decade of digital photography, it seems that almost no one does formal metering. They chimp, look at histograms, look at blinkies, create new methods and languages, and basically make stuff up. Incident light metering and ISO sensitivity ratings adhere to standards, procedures and definitions. Standards and definitions exist to avoid the chaos and gibberish that gets posted on internet forums by people who either unaware or don't understand the standards. I have a Sekonic L-358 incident light meter. If I take a picture using the meter's exposures settings, and then look at it on a calibrated monitor (brightness set to 120 Cd/m^3) it will look just like the scene I just shot. If I print it out on a printer that has uses the paper's profile, I get a print that looks like what is on the monitor and what the scene looked like. On to the Olympus Low setting.

    The Low setting is called Low and not ISO 100 because it does not conform to the ISO standard for 100. The following 3 pictures were done with a E-M5 on a tripod and the exposure set for 100, 200 & 400 ISO. The camera was set to Large Normal JPG. ISO was set to Low, 200 & 400, f2.8 and the shutter speed was varied along with the ISO. (Full res files can be downloaded.)

    LOW:

    ISO Low by b_rubenstein, on Flickr

    ISO 200:

    ISO 200 by b_rubenstein, on Flickr

    ISO 400:

    ISO 400 by b_rubenstein, on Flickr

    If one uses a photo editor that can read out the RGB values of any point in a frame, they would see that the RGB values are almost identical in the ISO 200 & 400 images, and higher in the LOW image. Even with the camera's JPG processor, the Low image is about 1/3 stop hotter.
     
  5. MAubrey

    MAubrey Photographer

    Jul 9, 2012
    Bellingham, WA
    Mike Aubrey
    Try not to sound too condescending...
     
  6. bikerhiker

    bikerhiker Mu-43 All-Pro

    Dec 24, 2013
    Canada
    David
    A RAW file is basically a binary file of 1s and 0s (not a picture) and the RAW program interprets this file and forms an image. So basically the final RAW image is only flawed if the metering was set wrong by the photographer or the photographer was unaware of the sensor's performing dynamic range or in the film world era, it's known as its exposure latitude.

    Here's a number of things to make aware of. All metering system on a digital camera meters at 18% grey, so the meter tends to want to underexpose white and turn it grey. Vice versa, the meter tends to want to overexpose black to grey. What the modern metering system does (ESP by Olympus or Matrix from Nikon or Multi-segment by Pentax) is to use patterning zones in the image area metered at 18% and then the metering computer will use the zones to recall from its built-in metering computer database to apply the appropriate compensation from the 18% metered. If you want to apply compensation on a subject matter metered using ESP (matrix metering), the best approach would be using Olympus global metering off-set. This, I found, tell the Olympus computer metering system to over ride its own metering which actually tends to protect the highlights on the pixel level. This behaviour is also happening on the E-PL1, which is why I found it ODD for some reviews to comment that the E-M10 is trying to protect highlights, when in fact, it's been doing this for the past 4 years on my old camera.

    So what is happening I think is that, the poster tries to compensate for the white background by slightly overexposing. When you do that, the metering system will revert back to what it's supposed to do -- not compensate by trying to protect the highlights, but because the poster thought it is underexposing, it overexposes which basically burned the highlight areas. So the ESP metering computer can actually be giving the operator the wrong information in regards to exposure latitude to fit the sensor's dynamic range at its performing ISO. At least, this has been my impression thus far shooting with my E-PL1.

    Which is why OIympus provides us with 2 extra metering options from auto and center-weighted. It's called Spot Hi-Key and Spot Low-Key on the older models and Spot Highlight Control and Spot Shadow Control on the newer models for this specific reasons.

    These spot metering modes are enough to determine the high and low key ranges so you know how much of a latitude you can squeeze out of your scene. You don't really need a Sekonic meter. It's ok in daylight or with a soft diffused light like the sun or a big soft box or a big light. I see here that harsh lighting was used, so that can introduce various tonality in the dark or bright areas that can be easily clipped. Spot metering is a better way. By the way, my Sekonic L308 is gathering dust as we speak, though I do use it occasionally.

    Since I push my E-PL1 up to ISO 3200, I need to be absolutely sure of my exposure latitude between my highlights, midtones and shadows, so I meter accordingly in Manual mode and use Spot to determine my latitude. I always shoot in Manual btw.

    Try the E-M5's Spot metering options and understand how to use it. If you meter with Spot, you don't have to fight with the metering computer as your compensation is the only exposure compensation will be applied. If you use ESP metering, then you are fighting with the computer meter when it's applying its own and contradicting with what you are applying.

    When I shoot RAW ORF, I will use either OV 3 or DXO Optics Pro extensively and it's never always wrong in the exposure department as long as you manually metered the scene correctly.

    Hope this helps.
     
    • Like Like x 3
  7. pdk42

    pdk42 One of the "Eh?" team

    Jan 11, 2013
    Leamington Spa, UK
    Thanks for the posts. Setting aside rubenstein's smug attitude, there's some interesting stuff here ( even from him! :smile:). However, the discussion so far has been on metering but that wasn't what I set out to examine. I wanted to know how the ISO 100 setting behaved and whether I'd be losing highlight headroom to any significant degree by using it. Frankly, I wanted to have harsh lighting and I wanted to overexpose!

    As I said in my conclusions, I'm fairly happy that I can get detail back from 'overexposed' areas in ISO 100 shots, but the thing that's really perplexing me is what raw digger is telling me. It's saying that 50% of the image is completely blown (pixels at max value), yet I can recover details from these areas. Either it's plain wrong, or I'm misunderstanding something very basic.
     
  8. Iliah Borg

    Iliah Borg Mu-43 Rookie

    15
    Feb 10, 2014
    > If one uses a photo editor that can read out the RGB values of any point in a frame, they would see that the RGB values are almost identical in the ISO 200 & 400 images, and higher in the LOW image. Even with the camera's JPG processor, the Low image is about 1/3 stop hotter.

    Right. And that is because the difference between ISO 100 and ISO 200 is not 1 stop here.
     
  9. ThomD

    ThomD Mu-43 Veteran

    426
    Jun 1, 2013
    SF Bay Area
    I haven't used Rawdigger, so I may be misreading the results. I read them to meant that the image is blown in one or two color channels only. The recovered detail is in the other channel(s). You should get detail, but not the right colors.
     
  10. Iliah Borg

    Iliah Borg Mu-43 Rookie

    15
    Feb 10, 2014
    > 1 - Rawdigger is getting it wrong!

    Rawdigger statistics shows that one channel is not blown out, and that is the blue channel. One can still restore some information for fairly neutral surfaces from just 1 channel. From the point of view of image processing what is happening is data extrapolation. Effectively 3/4 of resolution is lost as the result. It may be OK for neutral specular highlights and shiny surfaces. But even on pearls it does not work leaving ugly halos.
     
    • Like Like x 1
  11. pdk42

    pdk42 One of the "Eh?" team

    Jan 11, 2013
    Leamington Spa, UK
    Ah, so that's what's happening! Thank you for explaining it. I understand now. Of course in my test shot, the areas I pulled back are white card, so there's no issue. I guess I should try it with a blown area that's not neutral.
     
  12. Iliah Borg

    Iliah Borg Mu-43 Rookie

    15
    Feb 10, 2014
    Right. One more thing - how are your overexposure warning parameters set in Preferences?
     
  13. b_rubenstein

    b_rubenstein Mu-43 All-Pro

    Mar 20, 2012
    Melbourne, FL
    The only parameter that can be set in the warnings are the thresholds for the sum of the color channels. A warning will not appear if only a single channel is clipping. For Caucasian skin tones it's very easy for the red channel to clip and that can result in a dreadful image.

    I was trying out an Olympus XZ-2, which has the same high/low light warnings as the µ4/3 cameras. No highlight warning when I took this picture, but the red channel is clipped:
    12520488704_035d8c3c99_c.
    XZ240159 by b_rubenstein, on Flickr

    The metering methods and tools used by commercial photographers for shooting transparency films (get it right in the camera), work equally well for digital. Of course, if an incident light meter can't be used, and a reflected meter needs to be used, things aren't as simple.
     
  14. Iliah Borg

    Iliah Borg Mu-43 Rookie

    15
    Feb 10, 2014
    > the red channel is clipped

    Probably clipped in raw conversion due to white balance, not in raw data.

    > The metering methods and tools used by commercial photographers for shooting transparency films (get it right in the camera), work equally well for digital.

    Not so if ISO is uncalibrated as you discovered in one of your previous posts here in the thread.
     
  15. bikerhiker

    bikerhiker Mu-43 All-Pro

    Dec 24, 2013
    Canada
    David
    There seemed to be an overly focused attention in regards to RGB channel clipping data and not much on the LIGHT used.

    Photography is about understanding the relationship between light and its effect on the subject matter. It has nothing to do with RGB channels, ISO calibration blah blah blah. That's important only in understanding the aspect of light values measured in EV (Exposure Value) and is useful only for a CAMERA REPAIR TECHNICIAN or an engineer, but has absolutely no relevancy in regards to why you are getting clipping in your channels or as a photographer.

    Basic ISO whether it is calibrated or not need to put the exposure value in the midtones. It's because everything is calibrated to 18% grey, so why are we talking nonsense about RGB and make it overcomplicated.

    Basically, his harsh lighting creates tonality in the shadow and highlight areas that EXCEED the dynamic range that the sensor is capable of recording. Some clipping of the channels will occur. It's not the fault of Raw Digger or any RAW software. The poster made a conscious choice of using this harsh lighting to expose the subject matter. Basically like any measurebator would do, he or she would measure the exposure latitude of the low setting of the E-P5 or E-M5 or even my old E-PL1 to determine the ranges. This is how a commercial photographer (even today) would do for shooting transparency films (slides) work equally well for digital. Slide films have a much narrower latitude than digital and EVEN in the good old film days the ISO rating stamped on the film itself isn't even right. We would all do testing and grade the actual film speed with a similar test like what the original poster had done, but this is only because we want to make sure what actual film speed is so that our exposure is bang on, so we can place the midtones where it should be and then understand where the shadow and highlights would lie. Then we frame the picture to emphasize the subject matter in relationship with the light!

    Most beginner photographers always make the same mistakes as a photographer who just started using a wide angle lens. They do not understand the relationship between subject matter and FOV of the lens and what usually happens is that, most newbies tend to try and cram in as much stuff into the wide angle lens to capture it only to yield a rather BORING photo. This is the same with exposure. They try to cram in all the shadow and highlight areas and then use Photoshop or RAW Digger to fix it. If you would focus on the relationship of light with the subject matter and get it right in the first place in the camera, you will have minimal issues in the first place. You would add more lights and reflectors to balance the light so the shadow and highlight tonality would fall within the exposure latitude (DR) of the sensor and you won't have as much channel clippings.

    Cheers.
     
  16. Iliah Borg

    Iliah Borg Mu-43 Rookie

    15
    Feb 10, 2014
    > everything is calibrated to 18% grey

    Raw is not calibrated to 18% grey, easy to check.

    There are some technical aspects to photography, film or digital. If you look at film data sheets you will see they used to bother presenting characteristic and spectral curves, exactly because not blowing out a layer is important. Any standard cameramen' education course spent quite some time on sensitometry and densitometry.
     
  17. bikerhiker

    bikerhiker Mu-43 All-Pro

    Dec 24, 2013
    Canada
    David
    RAW is 1s and 0s, so how can it be calibrated to anything other than it is calibrated to show the midtones with the software used? It's up to the programmers as well as how the photographer captured that light. If you look at the histogram, it's always showing predominantly the midtones correct if you expose the subject properly? Do you have a RAW converter that shows only the predominantly the highlights or the shadows more than 4 stops over?!? You would if you screwed up the exposure.

    Most camera shops own a Kyoritsu EF-1, EF-5 and EF-5000 and uses the Kyoritsu BM-3000 to calibrate them to. Guess what they are based on sir? 18% grey. If you place a camera and meter using the Sunny 16 RULE, it will yield exactly the same exposure values in Tv and Av based on this rule. You can play games with your RGB thoughts but this is the industry standards. Unless of course you sir like to claim that all Kyoritsu devices are calibrated incorrectly or you are implying the Sunny 16 RULE is just a lie?

    All camera metering systems when they are repaired go through these machines for light calibration.

    Cheers..
     
  18. Ulfric M Douglas

    Ulfric M Douglas Mu-43 Hall of Famer

    Mar 6, 2010
    Northumberland
    I use FastStone 47 to delve into the Olympus ISO 100 200 mysteries.
    Its free and does not apply camera-specified exposure correction to the ORFs, which is nice.
     
  19. Iliah Borg

    Iliah Borg Mu-43 Rookie

    15
    Feb 10, 2014
    > RAW is 1s and 0s

    Think of sensor as of a measurement device which it is in fact. Raw is an array of numbers proportional to the light that hits the sensel. In other words, an array of light measurements. Any measurement device needs to be calibrated. A camera can be calibrated to 18% in raw for low ISO settings and to 9% (one full stop lower) for regular ISO settings. Check and you will see.

    Calibration can depend on white balance too, especially if they are set to matrix metering.

    Cameras are not user-calibratable, except for exposure compensation. Repair shops calibrate as per manufacturers' instructions that look like this:

    When RAW recorded:
    Within the area of 425 pixels x 425 pixels at the center of screen,
    calculate the average of G-14 bit data.
    When TIFF/JPEG recorded:
    Within the area of 425 pixels x 425 pixels at the center of screen,
    calculate the average of Y-8 bit data
    Standard
    RAW (14bit)
    Lo 1 3428±200 (±0.1EV)
    Lo 0.7 2717±160 (±0.1EV)
    Lo 0.5 2421±145 (±0.1EV)
    Lo 0.3 2158±130 (±0.1EV)
    ISO 200-ISO6400 1714±100 (±0.1EV)
    [TIFF,JPEG]
    ・Standard Lo 1, Lo 0.7, Lo 0.5, Lo 0.3, ISO200-6400 134-144
    ・Neutral Lo 1, Lo 0.7, Lo 0.5, Lo 0.3, ISO200-6400 131-140
    ・Vivid Lo 1, Lo 0.7, Lo 0.5, Lo 0.3, ISO200-6400 132-144

    You can see that different calibrations in raw must result in the same calibrations of TIFF/JPEGs.
     
  20. bikerhiker

    bikerhiker Mu-43 All-Pro

    Dec 24, 2013
    Canada
    David
    Absolutely what you are saying is correct. Yet, you seemed to be continually missing the point. That is, the light source and its quality and placement dictates the exposure and its final DR and NOT the camera. You can set up the same scene with reflectors and additional lights and balance the tonality of the shadow and highlight areas to yield a photo with minimal channel clippings than this. There's no denying the performance of the LOW setting of the expanded ISO from the new firmware, but you need to understand this.

    There is always a limitation in camera equipment and knowing is good only if it will help you achieve a better well balanced photograph. A successful photographer becomes successful because he or she understands completely the limitation of his gear and work around. Which was why Galen Rowell never shied away from using a cheap 35-80 AF zoom lens and a consumer camera model where all his workshop ******s and fangirls seemed to be fixated on gear this and that --- the measurebator syndrome. Sort of like Ernst Haas once scolded the person who asked him -- what's the best wide angle lens. He replied with something like -- just take 2 steps back and you're done!

    Take care. Btw, I really enjoyed your analysis as always and had learned a lot from you at DPreview.