Olympus iAuto & RAW files - clarification sought

[pdh]

I may have made some naive assumptions here, so please forgive my ignorance if so ...

I thought RAW was RAW was RAW (realising that different manufacturers use different RAW encodings), so that the RAW (in Olypmus' case, ORF) that I get out of the camera is basically an unprocessed data file, that I can use later to process ("develop"); the film analogy is that the RAW file is the negative (like all analogies, it is rather imperfect)

I know if I use "Art Filters" in the camera, the RAW file is untouched.

So, I expected that if I shot with RAW+JPEG enabled (as I always do), then the camera's own firmware will process the RAW file to produce a JPEG, but that the original RAW file will remain untouched.

I've been shooting for fun with iAuto a couple of days, but when I unload the camera, I find that if I load the RAW (ORF) into OV2 (or my demo LR3), the RAW displays exactly the same as the JPEG.

(I can undo it setting by setting, and get back to what looks like an unprocessed RAW)

So, which bit am I not understanding properly?

Or is it a shortcoming of the iAuto mode?

I use an E-P2, by the way

 

[Streetshooter]

What your seeing if I'm not mistaken and I could be is....
a preview. LR uses the Jpeg as a preview. Turn that feature off in Import and then you'll only see the raw files....

 

[pictor]

A raw file does not contain only the sensor data, but also metadata like the white balance and picture controls (contrast, sharpening and so on). It is common practice to save settings like these in the raw file, every manufacturer does this. The manufacturer's raw converter can read these settings, assumes that the photographer wants the photograph to look according to the saved settings and applies them after loading the raw. Other raw converters cannot read all of these settings and interpret the raw data in their own way.

One cannot display the naked raw data of the sensor, it has to be interpreted first. The best way to do this is to apply the settings which the user has set in his camera, since it is the user who knows best what the pictures should look like. Using iAuto means, that the camera decides the result. However, shooting raw means, that the photographer can change any setting besides ISO, aperture and shutter speed without loss. Shooting JPEG does not provide this possibility.

 

[dulaney22]

I don't know about your specific question, but I wonder how you are able to shoot in iAuto? I can't get the AF worth a crap and, from what I can tell, can't turn it to detect in the center while in auto mode. Even when I'm basically shooting auto on the E-P1 I have to set it to A or S mode and center Af. If I don't it focuses all around what I actually want at least half the time.

 

[pdh]

@Streetshooter -- Ok that sounds reasonable -- unfortunately I can't find a setting that corresponds to that !
(or is it the "treat jpg next to raw as separate"?)

@Pictor -- I didn't realise that RAW files would contain metadata other than ISO, shutter and aperture info (if available)

However I also note that LR and OV2 do display un-modified RAW files when using an "Art Filter" whereas in iAuto mode they appear the same as the JPEG (I'll obviously test this more carefully after I've checked out Streetshooter's suggestion)

@dulaney22 -- perhaps others can help with your difficulties if you start a new thread? If we try and treat two people's different queries in the same thread it'll probably get confusing ...

 

[DDG]

...from what I can tell, can't turn it to detect in the center while in auto mode.

You can reposition the AF box in iAuto. Push the left directional button (the one marked "[...]". When the grid comes up, it'll be completely green, but if you then push any of the directional buttons, only one square will remain green, and you can move it around on the grid.

 

[dulaney22]

...from what I can tell, can't turn it to detect in the center while in auto mode.

You can reposition the AF box in iAuto. Push the left directional button (the one marked "[...]". When the grid comes up, it'll be completely green, but if you then push any of the directional buttons, only one square will remain green, and you can move it around on the grid.

Wonderful! I figured there was a setting for this but I had fits at the Memphis Zoo a couple weekends ago and swore off that more.:smile: I'll fix my setting when I get home. Thanks!

 

[DDG]

Sorry, I didn't read closely enough, and thought you had the E-PL1. However, I Googled up the E-P1 instruction manual. You probably have the camera in "all target AF mode" rather than "single-target AF mode". There are three ways to change the mode, but the one I understand best is doing it through the main menu, because it's identical to how it's done on the E-PL1. It's on page 58 of the manual.

 

[PeteMarshall]

@Streetshooter

@Pictor -- I didn't realise that RAW files would contain metadata other than ISO, shutter and aperture info (if available)

.

If you want to see all the various things that can be in a RAW file then take a look at the DNG specification here.
This is the only published format and is built in such a way that it has places for everything the manufacturers put into their various formats. A large amount of this is ignored by the more sophisticated RAW processors such as Lightroom, which ignores camera processing settings, but some special data in manufacturers RAW files is read by Lightroom, of interest to micro 4:3's users the lens correction data is read and used, rather than separate lens profiles as with dSLR's although the lens profile tool is still there if you want to build your own (very useful for legacy lenses)

 

[pictor]

If you want to see all the various things that can be in a RAW file then take a look at the DNG specification here.

There is no possibility to get all data of a raw after converting it to DNG. The specification might be mighty enough (although I doubt even that), but since at least some manufacturer encrypt their camera processing settings, these settings get lost by converting the native raw to DNG. These settings can only be read using the manufacturer's software.

 

[squeegee]

...
So, I expected that if I shot with RAW+JPEG enabled (as I always do), then the camera's own firmware will process the RAW file to produce a JPEG, but that the original RAW file will remain untouched.

I've been shooting for fun with iAuto a couple of days, but when I unload the camera, I find that if I load the RAW (ORF) into OV2 (or my demo LR3), the RAW displays exactly the same as the JPEG.
...

I don't see the problem here. It's working pretty much as should be expected isn't it?

1) you take a photo in raw (or raw+jpeg)
2) you attempt to load the raw image on a computer
2.1) the computer can not display raw data - as an example your computer does not have the colour depth of the raw file, so the computer must convert the raw file to a computer renderable format, i.e. jpeg or png style colour depth etc
2.2) the computer will choose what it thinks is best for it's initial settings while converting from raw to say jpg for display, what it thinks is best, is what your camera though was best too.
3) the resulting JPG is displayed on the screen which looks like the jpg that came out of the camera

That sounds pretty much like what you're seeing and is pretty much what you should expect.

What were you expecting that was different?

Now the process doesn't exactly stop there, that's just the "initial" display of the raw image, the difference may be seen when you, as an example, turn up the brightness of the raw image and turn up the brightness of the original JPG. If you were to attempt that, on the RAW file you may see details emerge in what use to be black areas of the photo. On the JPG, you would see the black area of the photo turn into a grey area of the photo (no more details).

 

[mauve]

What a so-called raw picture really is...

There's no such thing as a RAW image. The closest you can get to sensor data is by using a raw developer like dcraw and telling it to turn off every options. You end up with a greyscale matrix of sort, each grey pixel showing the intensity of light recorded at that sensor place ; the catch here is every sensor pixel is behind its own coloured filter (blue, green or red), so the 'raw' greyscale image looks 'molted', and absolutely not like a clean black & white picture.

Then, the software goes over that matrix and multiply each pixel by a value representing each colour opacity (it's a known or reverse engineered physical constant), and turns the pixel to a pure blue, red or green point according to the Bayer's matrix layout. At this stage, the picture looks like a Seurat painting ; from a certain distance, you can make out the final image, but at close range, it looks like a mosaic of pure colour points of various intensity.

Next step is to de-matrix the image via an algorithm ; the maths involved here are quite complex. The idea is to add to each single coloured point (say, a green one) the 2 missing hues (red and blue channels) to end up with a complete RGB pixel at each location. The 2 missing values are deduced from the neighbourhood of the considered pixel. Some algorithms are quick but a bit harsh, some others do multiple pass over the data set, take longer, but produce a much smoother image. Trouble is, once this step is done, it's much more difficult to curb chromatic aberrations and remove noise (think of unwanted colours as spots of jam on your shirt : you'd rather remove them while they are still small, and this step sort of 'spread' those colours to adjacent places). That's why many cameras today do silently record noise-processed data instead of pure raw images. It makes them look better in tests at the expense of some unsharpness that can be blamed on the anti aliasing filter.

But wait, we're not done yet ! We now have a RGB picture, but it's uncalibrated. The software has to map this picture into a colourspace that can be shared with screens and printers. One part of this is to apply a gamma curve (a modern sensor outputs mostly linear values [when real life intensity grows by 10, the recorded value grows by 10], but screens and printers expect a curved response map [recorded low intensity and high intensity do not linearly follow the scene real intensity]) ; the other part is to tweak the colour temperature to fit the image into a colourspace.

Once this is done, your software finally may display what is commonly called a 'raw' image...

 

[pdh]

some very detailed replies for which I thank you all ... I did understand quite a lot of this before, but I've got some new information too ...

as to what I originally posted, it's no big deal, and I haven't described (or think of it) as a "problem" ... I've simply noticed that if I use an "art filter" in the camera, the resultant RAW is displayed like I'd taken it using P, M, S or A mode and the default camera processing whereas the JPEG displays it with all the extra processing that the "art filter" adds; whereas if I use iAuto, the resultant RAW file and the JPEG displays exactly the same ...

I just wondered about the inconsistency really ... perhaps someone else with an E-P2 and LR3 or OV2 would care to check it out?

It's perfectly possible that I've made a mistake about this and am talking out of my arse of course ... in which case I'll be sorry I wasted everyone's time

 

[mauve]

The two cases you describe are pretty different.

On the one hand, iAuto supplies photographic informations as metadata in the RAW file the raw developer can use (speed, colour temperature, colourspace, NR settings, contrast, etc.).

On the other hand, art filters are a supplementary step on top of raw development, it's not a photographic data that can be included in the RAW file. There's a binary byte somewhere in the ORF file format to tell Olympus software 'this picture was taken with art filter number z', but no other than olympus knows how to apply this effect. It's an algorithmic 'secret sauce' like a photoshop plugin. If you don't have the plugin, you can't make sense of the data. Olympus simply duplicated the filter from the camera in their software to be able to post-process the picture on a computer with the same program as the one in the camera.

No wonder the on screen result is different when you use another 3rd party developer.

 

[Streetshooter]

I'd like to thank everyone on this thread. It is informing and by no means, a waste of time.
Shooter

 

[squeegee]

I just wondered about the inconsistency really ... perhaps someone else with an E-P2 and LR3 or OV2 would care to check it out?

So, I expected that if I shot with RAW+JPEG enabled (as I always do), then the camera's own firmware will process the RAW file to produce a JPEG, but that the original RAW file will remain untouched.

I kind of don't understand the problem still, in both cases your RAW file is as you expect it, it's "untouched".

I think you're thinking it's been touched because it looks the same on your display as the jpg "by default". The default display or rendering of your raw files does not mean it's been touched or altered. It's just that the default display is the same processing algorithm as iAuto.

 

[pdh]

Well, to repeat, I'm not describing it as a problem ... all I am saying si that I notice a difference between how OV2 and LR3 display RAW files depending on whether the image has been captured using iAuto or not ...and I wondered what would account for that ... and to be honest I am not sure I am able to describe what I'm seeing any more clearly than I have already without us sitting in front of a screen together so I can point at things! Which possibility (given that you are at least an ocean away by the looks of things) seems unlikely !

(perhaps I have not yet been clear enough that I do not mean OV2 and LR3 display the same RAW file differently from each other, so I am not looking to understand the difference between the two bits of software)