E-M1 and noise removal

[barry]

Hi,

I took a bunch of shots at a high school game this week, at night; I had the ISO set to auto, and it varied between 800 and 4000.

Most of the out-of-camera JPEGs look pretty good as far as noise goes (iirc, the camera is set to the 'low' de-noise setting), but I wanted to make some other corrections (color temp, sharpness, etc.) using the RAW files.

Unfortunately, the RAW files showed a lot of noise, even at ISO 1000.

I tried the de-noise function in RawTherapee, but it seemed to not be very effective (very poor compared to the Oly jpegs), even with sharpening disabled.

Should I try the Olympus Studio (or whatever it's called now)? Does it have the same de-noise code as the camera?

Or does anyone have a recommendation for a good denoise setting in RawTherapee or GIMP?

I'm also willing to try DarkTable or any other OpenSource software.

Thanks!
Barry

 

[bobpur]

Hi

I find darktable's profiled NR for my EM-5 works quite well, there is also Lightzone that I am starting to use is good as well.
But what I find is that Olympus is better at the Jpg than Raw, drop the sharpening to -2 and there is only a little grain noise to
clean up and none of the terrible chroma color confetti . In moderate light even 3200 is very clean.
Must be the jpg conversion that does the work.

 

[tomO2013]

Some brands bake in some NR into their RAW files. Olympus don't. An ORF file is a blank canvas for you to develop as you see fit.

I tried DarkTable originally with Em5 files but didn't have much look - it tended to smear details but that was a long time ago. Things have likely taken leaps and bounds forward since. As for GIMP, it's been a while since I've tried it, but the last time I tried, I never managed to get anywhere near the types of results that I would get from Lightroom and Photoshop.
Another very good option is PhotoNinja (both from a demosaiccing, highlight recovery and noise perspectives). The best of the lot that I've found is DXO Optics pro. Their PRIME technology is on another level. It's slow and you can only see the effect in output files, not in the main preview window (which can be quite confusing). But realistically it looks to give the equivalent of 1-2 stops difference compared to RAW processors. If you have a beefy graphics card it can speed things up.
However these are all paid for options so probably won't suit you :S
Olympus Viewer 3 is good but make sure you have the latest and greatest version - they improved the IQ in more recent builds and improved the Jpeg output engine. I've found with older builds that the OV3 (while it should be the same as OOC) didn't do nearly as good a job with sharpening and noise reduction as the OOC jpegs. Newer builds as vastly superior.
OV3 also manages to pull a lot more detail and dynamic range from it's demosaicing engine than many other RAW processors :
https://www.youtube.com/watch?v=2wfZ69LqPW0
One option is to use OV3 (without NR) and export a Tiff that you then then process in GIMP etc...

In general terms if you want some tips for improving your noise reduction...
I would desaturate the image temporarily to B&W which makes it easier to see luminence noise and adjust this to a level that you are comfortable with. Then swap back to colour and reduce any coloured speckling that you may see. Finally add some grain back in post to remove any water colour effect and give some texture - or you could leave the grain there to begin with!
Another thing to consider - how you shoot.
Take a look at peka pokka's article on maximizing your dynamic range and noise control :
http://www.pekkapotka.com/journal/2012/6/14/olympus-e-m5-exposing-to-the-right-and-lightroom-41.html

Finally, there is a lot of hoopla about noise. This day and age most cameras will give you excellent noise control up to 1600. I will happily take the EM1 up to 6400 and get great prints from it. I'll still choose to add grain a lot of the time to avoid the watercolour effect. Generally speaking the prints and photos that I put up that people comment on most, are the ones that I have had the least amount of noise reduction irrespective of whether they came from my EM1, 5d mk iii etc.. That tells me that what DXO 'noise' charts see and what people subjectively interpret as attractive are two entirely different things. I'd hate to see how they rate TMax 100 film...

...

 

[barry]

Hi

I find darktable's profiled NR for my EM-5 works quite well, there is also Lightzone that I am starting to use is good as well.
But what I find is that Olympus is better at the Jpg than Raw, drop the sharpening to -2 and there is only a little grain noise to
clean up and none of the terrible chroma color confetti . In moderate light even 3200 is very clean.
Must be the jpg conversion that does the work.

Thanks, I'll try out Darktable, Olympus, & LightZone (hadn't heard of it before).

OV3 also manages to pull a lot more detail and dynamic range from it's demosaicing engine than many other RAW processors :
https://www.youtube.com/watch?v=2wfZ69LqPW0
One option is to use OV3 (without NR) and export a Tiff that you then then process in GIMP etc...

Curious... is there any advantage of TIFF over PNG?
Otherwise, I prefer PNGs as they're smaller and work in more apps.

In general terms if you want some tips for improving your noise reduction...
I would desaturate the image temporarily to B&W which makes it easier to see luminence noise and adjust this to a level that you are comfortable with. Then swap back to colour and reduce any coloured speckling that you may see. Finally add some grain back in post to remove any water colour effect and give some texture - or you could leave the grain there to begin with!

Thanks, the desaturation and luminance denoise trick helps a lot!.
With that, I am able to get the noise level similar to the Olympus OOC JPEG (it's hard to tell exactly as the JPEG was badly overexposed; I'll probably post another thread about that).

Another thing to consider - how you shoot.
Take a look at peka pokka's article on maximizing your dynamic range and noise control :
http://www.pekkapotka.com/journal/2012/6/14/olympus-e-m5-exposing-to-the-right-and-lightroom-41.html

I normally 'shoot to the right' but I was having too much trouble with slow shutter speeds and high ISOs that I gave up on it for this game.

Finally, there is a lot of hoopla about noise. This day and age most cameras will give you excellent noise control up to 1600.

Unfortunately, I was way over 1600 ISO much of the time; the lights at the high school stadium weren't very bright. I started with a 4/3 70-300mm, but was getting a lot of blurry pictures even at 70mm due to slow shutter speeds, even with the ISO auto at 3200, so I raised it to 4000, and still had too much motion blur (it was a Lacrosse game and the ball was a blur most of the time).
So, I switched to my Takumar 50mm f1.4, shooting around f2.4 or so, and that helped a bit, but the fov was rather wide.

Anyways, I'll post some sample pictures soon.

Thanks!
Barry

 

[barry]

I just tried Oly Viewer 3, version 1.21...

It does seem to be able to match the E-M1 camera's built-in noise removal - good.
However its auto white balance seems poor compared to RawTherapee.

Barry

 

[barry]

OK, here's samples from 2 different images; both were ISO 4000 (the camera was being rather dumb; the first was badly over-exposed at 1/50 sec, and the second was at 1/100 sec but it should have changed the ISO to 2000 instead).

Looks like the forum didn't keep them in order, but I think it's:
RawTherapee, no NR or sharpen
RawTherapee with noise reduction and sharpen
OOC (over-exposed (tried to fix 2nd image's brightness/contrast in GIMP) and bad white balance)

Barry

Subscribe to see EXIF info for this image (if available)

Subscribe to see EXIF info for this image (if available)

Subscribe to see EXIF info for this image (if available)

Subscribe to see EXIF info for this image (if available)

Subscribe to see EXIF info for this image (if available)

Subscribe to see EXIF info for this image (if available)

 

[tomO2013]

Hi again!

Ok of the three sample images, the second one came out the best and looks like you did a good job

Raw therapee has a pretty good demosaiccing engine - it looks to pull lots of details. But with a quick play there, I don't think it is as good as DXO or PhotoNinja or Olympus Viewer 3 IMHO. Is there a reason besides costs that you are not using Lightroom / Capture One/ PhotoNinja/DXO etc.. ? Even on Ubuntu workstation I dual boot into Windows for that, or even use windows virtual machine in Ubuntu if I don't mind waiting.

Curious... is there any advantage of TIFF over PNG?
Otherwise, I prefer PNGs as they're smaller and work in more apps.

Hmmm..... this can get contentious but I feel that TIFF every time. PNG was never originally conceived for photography, it was conceived to replace GIF. Being open and open-source does not necessarily mean that it is the better solution. TIFF while originally an Adobe file format is pretty well understood by the majority of professional grade applications available. While both support lossless, TIFF was designed for photography with support for embedded EXIF data, layer support - this is a pretty big deal when you export into and out of plugins and want to retain information. Another option is to use DNG format, but most of the time TIFF with its full support of CMYK, 16bit information is fine. Please note that how you convert from Raw to TIFF is very important; it's important that your raw processor does not truncate information from the highlights/blacks during the conversion process. I.E. try and do any major exposure adjustments on the RAW file itself before exporting to TIFF - you can use your histogram as a guideline.
Yes file sizes can grow pretty big, but you can always add them to a compressed folder. No biggie

I normally 'shoot to the right' but I was having too much trouble with slow shutter speeds and high ISOs that I gave up on it for this game.

Different approaches and all that, but shooting hockey games here ISO2500 and primarily at 6400 I got pretty good prints - ok self praise is no praise and all that but genuinely most printed really well. I have my camera setup for ETTR technique and with blinkies enabled so taht I know that I am capturing as much information as possible in the highlight region. The majority of information in a file is stored in the highlights area so important to keep as much as possible there and then reducing exposure to keep as many tones as possible in the shadows. It's not always possible and likewise my hockey pics didn't always come out the best requiring some work. Again it's a super fast moving game, lots of flashing lights and my copy of the Pan100-300 gets very soft from 285mm upwards. I discovered this after the fact!
The other thing regarding noise (and i know this may sound hard to believe) but I've genuinely found with the EM1 that it depends on the lens used. My Pan100-300 at the very long end tends to produce quite a bit of noise, especially since I need to stop it down some to get it sharp.
The Oly70-300 also can be soft at the long end and also needs to be stopped down to get sharpness back into the frame. Usually for both I believe that it is recommended to use an apperture of F7 or so. It's a tradeoff then - do you take sharper image with more chroma noise and luminence noise or do you take a softer image etc... I suspect that the sensor in the EM1 is out resolving the Pan100-300 and Oly 70-300 and this probably shows most at higher ISO's. Just a personal opinion - I could be very wrong.
You may be a good candidate for Olympus' new 300 F4 lens that is due... or the older four thirds 300mm F2.8 (but thats mega money!).
Looking at your samples - my gut feeling (and I hope I am not insulting you saying this) is that, like myself, it is a matter of both exposure capture technique and processing technique. I've found that I really have a LOT to learn from capturing on the long end!

In terms of your samples, keep in mind the samples you showed are crops from a much larger image (care to share the larger image?). You are ultimately nitpicking on a 100% view, that is then shared and uploaded to a website that itself further compresses the samples It isn't going to look good ever. It's also not how you would ever look at photos anyway when they hang on a wall! Also when it comes to print, ink and paper tend to work a LOT different to your screen.

For your shots I would raise the color noise/chrominence noise reduction slider a good bit but maybe make a small adjustment to the luminence noise reduction slider. You should also do this from a fit to screen perspective as well as 100%. Many times people zoom into 100% and reduce 'noise' only to be left with a smeared waterclour mess. zooming in and out to see the effect in it's entirety is really handy to avoid this.
Ensure that you apply some level of masking when applying sharpening otherwise you will end up sharpening the grain and making it worse. The Olympus Viewer Sharpening tool can over sharpen sometimes and tends to work best with it's own NR algorithms. It doesn't often leave you much room for further post processing.

These were mostly shot at ISO6400. I would even have gone higher had I not set the auto limit to 6400, for the sake of being able to ETTR and more importantly balance keeping the shutter speed up. With longer lenses, even with IBIS, it's important to have a relatively quick shutter speeds.

Subscribe to see EXIF info for this image (if available)

Man of the match - Ben Scribens,- coming back out on the ice after the game to throw his hat to the crowd! by tom.ohle, on Flickr

ISO6400
View attachment 358653 "I skate to where the puck is going to be, not where it's been" ~ Wayne Gretzky by tom.ohle, on Flickr

ISO6400

Subscribe to see EXIF info for this image (if available)

Hockey Player for the sharks by tom.ohle, on Flickr

ISO6400
View attachment 358655 Face Off - NHL Revealed! by tom.ohle, on Flickr

Finally one last thing I just thought of - do you have the latest Olympus firmware with the anti shock feature enabled? I did a few tests and the only lens where I have noticed any type of 'shutter shock' is with my Pan100-300, it probably saw the biggest benefit from the new 0-shock feature. I never saw or experience the problem before that.

Hope this post is of some help to you Barry.

best of luck and happy shooting.

Tom.

 

[David A]

Were your crop areas in darker or shadowed parts of the frame? Noise is worst in the darker areas.

If they were in the darker areas of the frame, were they the parts of the frame that you were most interested in or is the focus of the image somewhere else? If they were in the area you were most interested in, then you've underexposed your subject and need to correct your exposure technique. From your other query re overexposure where you said you were using the default metering mode, I'd suggest using a different metering technique for scenes where there is a lot of variation in brightness across the frame and basing exposure more on the area of interest rather than the average of the whole frame. It the cropped areas weren't in the area you were really interested in when you took the image then you need to decide how important a bit of noise in an unimportant area of the frame is. If it is important and if removing it with a whole frame noise reduction adjustment is detrimental to the image as a whole, then learn how to do spot adjustments with an adjustment brush or some other localised correction technique.

If you've got a wide variation in illumination over the whole frame then getting your exposure correct is trickier than getting it right on a more uniformly and well lit scene. In scenes including a wide brightness range you can usually expect to find some noise in the less well lit/more shadowed areas if you try to bring the brightness of those areas up with an exposure adjustment of some kind, especially as your ISO setting increases.

 

[barry]

Subscribe to see EXIF info for this image (if available)

Were your crop areas in darker or shadowed parts of the frame? Noise is worst in the darker areas.

In terms of your samples, keep in mind the samples you showed are crops from a much larger image (care to share the larger image?).

Hi,
Small versions of the uncropped photos attached. (for purpose of showing lack of heavy shadows, etc.; I know you can't see the noise in these small versions).

Both have noise in the faces (more than I am comfortable with), and a lot of noise on the green sweatshirt. The properly exposed one is slightly worse, and also shoes noise in the bleachers (the background; see crops above), which are not dark at all.

Thanks for all the advice!

Barry

 

[barry]

Finally one last thing I just thought of - do you have the latest Olympus firmware with the anti shock feature enabled?

Latest firmware - yes, and I did enable anti-shock in my 'mysets', but I was shooting in continuous drive mode (where you hold the shutter and it takes 5 or 10 shots per second) at the game, and afaics you can't have anti-shock on in those modes.

Nice hockey pics!

Barry

 

[tomO2013]

So I'm guessing the left one is the raw therapee NR + Sharpen and the right is the SOOC ?

It is very hard to judge these as the samples are not full size / full resolution images but down-sampled for screen shot purposes. A lot of other things could be coming into play - for example, there looks to be some sharpening artifacts in the SOOC. This may simply be a result of the down-sampling for screen use or may actually be sharpening artifacts introduced by OV3, or sharpening of grain as I suspect is the issue. I'm guessing the first one though

On the long telephoto end, without tripod, shutter speed is king. I would even push higher ISO for the sake of keeping your shutter speed up. It is fairly easy to clean up most images if they are tack sharp. The only caveat is that I would setup your white balance in camera before taking the shot. Regardless of manufacturer it is easier to get the WB correct in camera before taking a shot than fixing it up after in post for high ISO shots, or at least I've found this to be the case.

Getting back to your sample, I can see what you mean - the Raw Therapee does not look to be pulling as much information as the OOC. Looking at your shot on the right, the OOC, it is definitely pulling in a lot more detail but slight sharpening artifacts (possibly because of resizing for web).

If the original exposure was deliberately overexposed for ETTR, then you obviously had to pull back the Raw Therapee shots. I would have expected these to look considerably cleaner and to hold more detail. Can you share the ORF file itself?

BTW I'm very much an amateur when it comes to telephoto work. I also shoot a Canon 5d mk iii with a 300 F4 that I rent from time to time. I've gotten my share of turkey shots that came bad from time to time too! Invariably the problem (whether the shot was taken with EM1 or 5d mkiii ) is always me. Telephoto work is tough and takes a lot of practice!
However, every time I doubt my equipment I check out the 70-300 and 100-300 sample image threads where some spectacular shots have been taken with far lesser equipment in worse situations.

If you want to share your ORF files, I'd be happy to run them through DXO, photoninja etc.. and see if perhaps we can remove the raw processor from the equation...

Edit: Check out this thread :
Share: High ISO MFT Photography

Hi,
Small versions of the uncropped photos attached. (for purpose of showing lack of heavy shadows, etc.; I know you can't see the noise in these small versions).

Both have noise in the faces (more than I am comfortable with), and a lot of noise on the green sweatshirt. The properly exposed one is slightly worse, and also shoes noise in the bleachers (the background; see crops above), which are not dark at all.

Thanks for all the advice!

Barry

 

[barry]

Hi Tom,

Both of the last images were SOOC.

One was over-exposed by the camera, the other was not (maybe under-exposed).

The 100% crops posted earlier were from these images.

I wasn't doing ETTR on these shots as I was seeing a lot of over-exposed highlights (white uniforms) during the game so I set EC back to '0'.

I'll check out that other thread.

Thanks,
Barry