Main weakness of m43 (according do me)

[archaeopteryx]

I'm pretty sure that Photons to Photos DR-ISO chart should not be used between brands/systems. I'm not sure if even DxoMark should be used like that, but it gives more sensible results.

Photons to Photos's methods are documented. Since they're neither brand or sensor specific it doesn't appear this conclusion is correct. It might also be useful to review translation between DxOMark DNR and PDR. It's my impression cross-brand/cross-system comparison difficulties are more with DxOMark's scores for different sensor sizes. However, DxOMark does also change measurement scenes between versions of their protocols and that might affect the dynamic range values reported. It would be appropriate for DxOMark to more transparent about this and do basic things like indicating the protocol version of a set of measurements.

Photon shot noise is a function of pixel area so, if it was the only form of noise present on the IMX272 and IMX410 (the A7 III sensor) then we would naively expect a 1.7 stop difference due to the imaging areas and pixel counts of the sensors. That the measured difference between the G9 and A7 III is 1.2 stops suggests about half a stop of other stuff is going on (the E-M1 III shows some additional difference as its DNR is less than the G9's below ISO 3200). That half stop is presumably some combination of factors such as microlens efficiency, actual versus nominal photosite size, dark current, kTC noise, pattern noise, read noise, and probably other stuff I'm not immediately thinking to mention. Unless one has details of the image sensor design and knowledge of surrounding factors, such as the PSRR (power supply rejection ratio) of the ADCs and the quality of the camera body's circuit board design, it is difficult to reason about the relative importance of each mechanism's contribution. There is a tendency for newer bodies using a given sensor to have slightly higher DNR but it's similarly difficult to say how much that might be due to fab process tweaks on the sensor versus incremental improvements in the camera bodies.

So far as I know there's also been little effort expended on attempting to quantify body to body variation. It's nonzero due to fab process variations and tolerances on components such as power supply capacitors and it appears to be less of a thing than lens copy variation. My guess from looking through Photos to Photos is ±0.1 stop in DNR might not be unreasonable.

 

[RS86]

Photons to Photos's methods are documented. Since they're neither brand or sensor specific it doesn't appear this conclusion is correct. It might also be useful to review translation between DxOMark DNR and PDR. It's my impression cross-brand/cross-system comparison difficulties are more with DxOMark's scores for different sensor sizes. However, DxOMark does also change measurement scenes between versions of their protocols and that might affect the dynamic range values reported. It would be appropriate for DxOMark to more transparent about this and do basic things like indicating the protocol version of a set of measurements.

Photon shot noise is a function of pixel area so, if it was the only form of noise present on the IMX272 and IMX410 (the A7 III sensor) then we would naively expect a 1.7 stop difference due to the imaging areas and pixel counts of the sensors. That the measured difference between the G9 and A7 III is 1.2 stops suggests about half a stop of other stuff is going on (the E-M1 III shows some additional difference as its DNR is less than the G9's below ISO 3200). That half stop is presumably some combination of factors such as microlens efficiency, actual versus nominal photosite size, dark current, kTC noise, pattern noise, read noise, and probably other stuff I'm not immediately thinking to mention. Unless one has details of the image sensor design and knowledge of surrounding factors, such as the PSRR (power supply rejection ratio) of the ADCs and the quality of the camera body's circuit board design, it is difficult to reason about the relative importance of each mechanism's contribution. There is a tendency for newer bodies using a given sensor to have slightly higher DNR but it's similarly difficult to say how much that might be due to fab process tweaks on the sensor versus incremental improvements in the camera bodies.

So far as I know there's also been little effort expended on attempting to quantify body to body variation. It's nonzero due to fab process variations and tolerances on components such as power supply capacitors and it appears to be less of a thing than lens copy variation. My guess from looking through Photos to Photos is ±0.1 stop in DNR might not be unreasonable.

I don't know much about this stuff, but I have read that at least results before E-M1iii were skewed when comparing to other systems, because how Olympus did their ISO ratings. You can see the drop in performance for E-M1iii vs the earlier E-M1ii in the chart I put, and it is the same sensor.

Like I said, more knowledgeable members have to chime in to this debate, but to me it seems obvious that there should be about 2 Ev difference between FF & M43 in same generation sensors, and the Sony A7iii is 1,5 years newer with a known big jump in performance, so if it is not seen in some chart, I have high doubts it is a correct comparison.

 

[archaeopteryx]

there should be about 2 Ev difference between FF & M43 in same generation sensor

This is incorrect for the reasons just mentioned. CMOS imaging and ADC noise budgets are well characterized and not subject to debate.

But since the preference is reject objective measurements for failing to conform to a poorly informed and apparently a priori conception of what they should be I'll leave you to it.

 

[mfturner]

One question I've always pondered regarding DxO processing is that I believe this is performed on jpeg with default camera settings. If that's true, how much will sw (high iso noise reduction) affect the result between brands, or individual settings preferences (even picture modes)?

And if I'm wrong, that would be good to know too.

My impression is that the relative plots on Photon to Photos is a very good guide to what you might expect with my old 60D, my m10.3, my m5.3 and even using the e500 data as a proxy for the PM1. Even the old 60d, where it really lacked was DR at low iso, high iso wasn't terrible, and I learned that if I'm careful I can make usable photos with around 6 EV of DR. I'd rather not, but if my kids are in a poorly lit stage and that's all I've got, I'm going to use it.

 

[RS86]

This is incorrect for the reasons just mentioned. CMOS imaging and ADC noise budgets are well characterized and not subject to debate.

But since the preference is reject objective measurements for failing to conform to a poorly informed and apparently a priori conception of what they should be I'll leave you to it.

Umm.. I'm one of the fiercest M43 defenders imo, and you'd know it if you have read my posts.

I have no problem M43 being more competitive against FF, in fact what you say is a good surprise to me.

I'm still waiting for more knowledgeable (than me) members input on this matter, as I have read this stuff here and other places.

What is your view on that Photons to Photos shows that E-M1iii is worse than E-M1ii, while it is a newer camera with the same sensor?

 

[mfturner]

I'm not speaking for @archaeopteryx , but could this be simply because of the jpeg and default settings usage I mentioned? Maybe they changed sharpening, black/ white point, or something else in the default jpeg rendering?

 

[RS86]

I'm not speaking for @archaeopteryx , but could this be simply because of the jpeg and default settings usage I mentioned? Maybe they changed sharpening, black/ white point, or something else in the default jpeg rendering?

I don't understand why "sensor testing" would be done with jpeg's.

 

[PakkyT]

I don't understand why "sensor testing" would be done with jpeg's.

Good question and I suppose the answer is probably something along the lines of the manufacturer of the camera knows the sensor best and how to manipulate the data to get what the manufacturer perceives as the best possible image under normal circumstances, so the testing is done with what the manufacturer has deemed "best image".

I recall Popular Photography magazine used to publish lens reviews and tests and their angle was that they didn't use a camera but instead put the lens on a light table to measure everything about only the lens to determine things like resolution, distortion, etc. I haven't look at that mag in years but I suspect this method won't work very well anymore since modern cameras with modern processing use a lot of image correction now and therefore lens designs factor that it where the lens itself may rate terrible but the corrected image would rate highly and it would be unfair to judge the lens with the uncorrected optics. This might be the same reasoning behind using the manufacturer's JPG engine for sensor data output as your subject rather than a raw file (which some systems I believe even the raw data is correctly or altered a bit).

 

[RS86]

Good question and I suppose the answer is probably something along the lines of the manufacturer of the camera knows the sensor best and how to manipulate the data to get what the manufacturer perceives as the best possible image under normal circumstances, so the testing is done with what the manufacturer has deemed "best image".

I recall Popular Photography magazine used to publish lens reviews and tests and their angle was that they didn't use a camera but instead put the lens on a light table to measure everything about only the lens to determine things like resolution, distortion, etc. I haven't look at that mag in years but I suspect this method won't work very well anymore since modern cameras with modern processing use a lot of image correction now and therefore lens designs factor that it where the lens itself may rate terrible but the corrected image would rate highly and it would be unfair to judge the lens with the uncorrected optics. This might be the same reasoning behind using the manufacturer's JPG engine for sensor data output as your subject rather than a raw file (which some systems I believe even the raw data is correctly or altered a bit).

I still don't understand why would someone for example take Dynamic Range capabilities from a jpeg.

 

[mfturner]

I was about to agree, then I started thinking about how I would do it on raw, and the Bayer conversion stumped me. Not sure how to get around that, maybe someone else has an idea?