Dx0 "disaster" for the GH2

Discussion in 'Panasonic Cameras' started by dko22, Nov 5, 2010.

  1. dko22

    dko22 Mu-43 Regular

    163
    Jul 26, 2010
    Stuttgart, Germany
    come on everyone --after seeing the GH2 Dx0 sensor test giving only 60 as opposed to 64 for the GH1 (link through 43 rumors)I thought that by now everyone would be either 1. cancelling their preorders 2. resignedly saying, "oh, well --it's a sort of correction to all the other raves but not too bad 3. saying that Dx0 tests are unreliable/worthless/limited etc as they're all doing over on the other forum. Interested in the reactions here!

    David
     
  2. grebeman

    grebeman Mu-43 All-Pro

    Mar 13, 2010
    South Brent, south Devon (UK)
    Barrie
    Oh no, not more dumbing down, can't take much more :smile:
    I'll be blowing the dust off my Olympus E 10 soon, now what would that score, minus 10 perhaps :smile: , still it was a nice camera to use and I've recently got a lithium external battery pack for it, that might be my best purchase of the last few months :biggrin:.

    Barrie
     
  3. pictor

    pictor Mu-43 Top Veteran

    636
    Jul 17, 2010
    Small sensor and many pixels - where is the surprise?
     
  4. soundimageplus

    soundimageplus Mu-43 Top Veteran

    782
    Feb 2, 2010
    Worcestershire
    I've never understood Dx0 sensor tests, as they have never matched what I've seen before me. I've used cameras that they rate highly and been very disappointed with the files they produced and vice-versa.

    Theres a explanation of how it works at luminous-landscape.
    DxO Analyzer Explained

    and they include these caveats.

    — DxO Analyzer can only be used to test lenses in combination with a digital sensor. A lens by itself can not be tested, and a lens in combination with a film camera can not be tested.

    — In the case of DSLRs and interchangeable lenses, measurements are only meaningful for a specific type of combination. For example, a 135mm lens on a full-frame Canon 1Ds will measure differently than on a reduced frame Canon 10D. This of course is because much more of the lens is being used by the full-frame camera and therefore most measurements will differ due to the difference in coverage.

    — Identical lenses, aren't. No two cameras or lenses are the same. Even ones off the same assembly line. It isn't unusual to find significant variance, and indeed some pros typically test a number of lenses of the same brand and type before choosing one.

    — Only test results of lenses of similar focal lengths should be compared with each other. Long lenses are always going to perform as well as measure better than wide-angle lenses, regardless of brand or price.

    — Don't compare zooms and prime lenses at the same focal length. With very few exceptions prime lenses will always be superior to zooms. The trade-off is convenience.

    — Measurements don't tell the whole story. One camera / lens combination may perform better optically than another, yet because of design, handling and other non-measurable performance differences be less desirable for actually producing photographs than one that measures better.

    — Small differences may not be important. Learn to read the results properly. Just because there is a small measurable difference between one system and another doesn't mean that this difference is necessarily visible on a print — regardless of size.


    So like anything else, it should be treated with caution. As far as I understand the tests are carried out with a two-dimensional chart. Since for the most part we photograph a three dimensional world, and lens/camera combinations can render one well and the other not so well, their tests may not be an indicator of how the combination works in the real world. Their GH2 results have a comparison between a GH1 and a 7D. I have both cameras and though their results indicate that the 7D comes out with better figures, that is not my experience. I find my GH1 produces consistently sharper files from raw file conversion and jpgs than my 7D. It may be of course that I have a not so good 7D, which of course is what Luminous Landscape warned about. To make the tests truly objective, you would have to use the same lens. It may be in the data but I couldn't find out what lens they used on the GH2. Say you used a 14-42 kit zoom on a Panasonic & a 24-70mm f2.8 zoom on a Canon 7D. How would this compare with an 18-55 Kit zoom on the Canon and the 20mm f/1.7 on the GH2? I suspect the results would be radically different.

    So what we can determine from the figures DxO produced is that the GH2 they tested performed worse at the two dimensional test they carried out using their parameters, under their testing conditions, than the GH1 they tested under the same circumstances. Whether this means anything I cannot say.

    In answer to your question as to how people would/should respond all I can say on my own behalf, is no I haven't cancelled my GH2 order, nor am I going to. I prefer the opinion and judgement of a photographer who has used the camera over any lab tests. Michael Reichmann, at the aforementioned Luminous Landscape gave it a very positive review and that means more to me than anything else.
     
  5. ~tc~

    ~tc~ Mu-43 Hall of Famer

    Oct 22, 2010
    Houston, TX
    I was going to ask if anyone had ever correlated the objective Dxo score to something as TOTALLY subjective as image quality.

    I only got introduced to Dxo testing when the whole "fast lenses are a waste with digital sensors" thing came out a few weeks ago, but what I read there about how they test led me to believe the Dxo measurements are dubious at best. Congrats on trying to find a way to put a number on image quality, but I don't think it's quite there yet.
     
  6. pictor

    pictor Mu-43 Top Veteran

    636
    Jul 17, 2010
    And the fact that you post and quote an article about DxO lens tests confirms this...
     
  7. soundimageplus

    soundimageplus Mu-43 Top Veteran

    782
    Feb 2, 2010
    Worcestershire
    your point being......?
     
  8. Amin Sabet

    Amin Sabet Administrator

    Apr 10, 2009
    Boston, MA (USA)
    Let's be nice, please!
     
    • Like Like x 1
  9. Amin Sabet

    Amin Sabet Administrator

    Apr 10, 2009
    Boston, MA (USA)
    While I realize that DxO meaurements are not the same as real world results, I think they are better than any review site when it comes to predicting real world signal to noise, dynamic range, and tonal range. What they don't give you:

    • Color fidelity
    • Presence/absence of banding
    • Absolute resolution
    • Aliasing tendency
    • JPEG processing
    • etc
    They are also potentially susceptible to noise reduction which has been applied directly to the RAW file.

    I did carefully controlled RAW file conversion signal/noise testing of all Micro 4/3 bodies against Samsung NX and Sony NEX. My results were basically identical to what DxO produced for those cameras and in marked contrast to what DPReview published, so I trust the DxO numbers for what they are worth.

    With regards to the GH2 results, I don't find them to be very disappointing. A little disappointing because some of the JPEG samples looked so good. However, the GH2 results were barely different than GH1, and we should bear in mind that the GH1 DxOmark results are very good.

    Based on sensor size alone, a Micro 4/3 sensor at native aspect ratio is predicted to have a 0.54 stop S/N deficit compared to 1.6X (Canon) APS-C (at native aspect ratio) and a 0.72 stop S/N deficit compared to a 1.5X (Sony/Pentax/Nikon) APS-C sensor. If you compare the GH1 or GH2 performance to most APS-C sensors on DxOmark, you'll find it very competitive on a per-sensor area basis. That said, the new K5/D7000/A650 is very impressive, particularly with regards to dynamic range at base ISO.

    If Panasonic was able to very nearly match the signal to noise and dynamic range of the GH1 and at the same time eliminate or significantly address the GH1 banding issue, then I think they have done well.
     
    • Like Like x 1
  10. pictor

    pictor Mu-43 Top Veteran

    636
    Jul 17, 2010
    You criticize tests of sensors by critical remarks about lens tests. You talk about sharpness, for example, which is not even part of any sensor test of DxO. You just do not present any argument.
     
  11. goldenlight

    goldenlight Mu-43 All-Pro

    Jan 30, 2010
    Essex
    John
    I sometimes wonder if we are close to the point when digital sensors in general are so good that comparison tests become irelevant. I would now put handling, features, size, available lenses, cost, etc before IQ when chosing a camera because in practical terms it is no longer an issue. If you can produce beautiful prints at A3 or larger will a marginal increase in IQ make any worthwhile difference?

    Of course there will always be exceptions for photographers with specific needs. If you regularly shoot billboards for advertizing then you might need something with a larger sensor than :43:. Same if most of your work is principally hand held low light stuff. Let's be honest, how many of us fall into categories like that?

    Specialist cameras will always be needed by specialist photographers but for the vast majority of us the camera that we most enjoy using will be the one which gives us the best results, regardless of sensor type, size or number of pixels.
     
    • Like Like x 4
  12. feppe

    feppe Mu-43 Regular

    All metrics DxO measures are most certainly NOT subjective by any means.
     
  13. soundimageplus

    soundimageplus Mu-43 Top Veteran

    782
    Feb 2, 2010
    Worcestershire
    Your point

    Lets be polite here and assume you misunderstood what I said. I made a brief comment at the start about DxO sensor tests. I then went on to discuss Dx0 tests in general. I assume its impossible to test a sensor without using a lens. I can see how you can produce a set of figures for a camera/lens combination, but how you can produce valid comparisons on colour sensitivity, tonal range etc. without reference to the lens escapes me.

    Was the test between the GH1 and GH2 done "fresh" with the same lens or did they just do a new test on the GH2 and compare the results with a old GH1 test?

    DxO state on their website:-
    All Sensor Scores reflect only the RAW sensor performance of a camera body. All measurements are performed on the RAW image file BEFORE demosaicing or other processing prior to final image delivery. DxOMark does not address such other important criteria as image signal processing, mechanical robustness, ease of use, flexibility, optics quality, value for money, etc. While RAW sensor performance is critically important, it is not the only factor that should be taken into consideration when choosing a digital camera.

    To take a sensor in isolation achieves exactly what? How a sensor performs in a picture-taking, rather than a testing situation, depends on many things. I often use a dc-raw based programme to get the best idea I can as to what the sensor on any given camera produces before the manufacturers software adds its corrections. Under most circumstances I get a file that is usually unusable. Raw file software "correction" parameters in the manufacturers software and Photoshop gives a more usable file. DXO claim that To help photographers rank and compare photographic equipment, DxOMark provides Sensor Scores showing the camera sensor image quality performance for a number of photographic Use Cases. But does what they produce help us to compare photographic equipment? To my mind they seem to miss out several pieces of information that could help us come to an informed conclusion.

    Its fairly obvious that I don't believe DxO tests to be of any use to me in any decision I make concerning buying cameras. I also object to the way that people trot out their test scores as if it were some kind of absolute truth, sent down from on high to inform us ignorant peasants. People quote their results and draw conclusions from them without understanding, as I freely admit I don't either, their methodology and testing practices.

    People seem far too willing to accept stuff like this without actually questioning it. The number of forum posts where people seem unable to make a decision without the "seal of approval" from this site or that site still astounds me. Within a short time of a camera being announced there are usually all sorts of files available to view. Usually well in advance of a camera becoming available to buy. I usually find using my eyes works quite well.

    Finally DxO aren't exactly some independent scientific study group. They sell software. So they aren't exactly doing this out of the kindness of their heart.

    Does that constitute presenting an argument?
     
    • Like Like x 1
  14. Narnian

    Narnian Nobody in particular ...

    Aug 6, 2010
    Midlothian, VA
    Richard Elliott
    Interestingly the GH2 is not coming up on Amazon - I wonder if they pulled it? Maybe there is a sensor issue.
     
  15. Amin Sabet

    Amin Sabet Administrator

    Apr 10, 2009
    Boston, MA (USA)
    Maybe after they get a certain number of pre-orders with unknown supply, they pull the item? I've seen the same thing happen with other high demand pre-order cameras.
     
  16. Narnian

    Narnian Nobody in particular ...

    Aug 6, 2010
    Midlothian, VA
    Richard Elliott
    You speculation is far more likely - but mine is more fun. Karma for messing with the GF1.
     
  17. Rudi

    Rudi Mu-43 Top Veteran

    574
    Aug 16, 2010
    Australia
    Exactly! I never saw the need to squeeze 18MP onto that small sensor, but for some reason Panasonic did. And now it's biting them on the ... well, you know. :biggrin:

    It does make you wonder - is this really the same company that has decided to keep the LX3 and LX5 at 10MP?
     
  18. pictor

    pictor Mu-43 Top Veteran

    636
    Jul 17, 2010
    I am very sorry for this post. There was no reason to get impolite.

    However, I still don't think, that you made a valid argument against the sensor tests of DxO in your first post. I have read your post three times now. You brought valid arguments regarding their lens tests. These arguments are valid for other lens tests, too, and should be kept in mind when reading review sites. However, lens tests and sensor tests are two different tests. As long as you don't bring any argument, why the arguments against lens tests are also valid for sensor tests, you don't argue against sensor tests by arguing against lens tests.

    Regarding all those tests I am a skeptic, too, but I still think, that a well done test can indeed say something about the tested lens or camera. Provided, that the methodology of a test is correct, the tests results still need to be interpreted correctly. This is the case with every well done test, not only in photography.

    Most of the time, tests are not interpreted thoroughly, which one can see regarding this test, too. In the special case we are talking about in this thread, the differences are only minor, even according to DxO's description of their tests. We are talking about a difference of less than 1/3 of a stop (according to DxO). Obviously, that should not prevent any sane person from buying this camera, even if these tests are as valid as they can be. This test just shows, that the GH1 and the GH2 are about equal regarding to the tested parameters.

    You are not alone.

    However, regarding the tested parameters, it is no surprise that the result of the GH2 is not better than that of the GH1, since Panasonic chose to raise the megapixel count. On the other hand this might be an indication of a fine achievement of Panasonic, that the increase of pixels has not influenced the tested parameters significantly, which is a surprise. But we still miss the parameters which have not been tested in any labor yet, that is, how the camera performs in real life.

    Of course, but again, this is no valid argument, as long as you cannot show, how that influences their test results. Research can be found in other places than universities, too, and there is no reason to question the integrity of scientists and technicians just because they are working for money.

    BTW, Panasonic sells cameras. So they aren't exactly doing this out of kindness of their heart, too. Olympus sells cameras. So they aren't exactly doing this out of kindness of their heart, too. Nikon sells cameras. So they aren't exactly doing this out of kindness of their heart, too. ... Where is the valid argument?
     
    • Like Like x 2
  19. soundimageplus

    soundimageplus Mu-43 Top Veteran

    782
    Feb 2, 2010
    Worcestershire
    Thank you for your post. I'm going to leave it there, as I believe we've stated our respective positions.
     
    • Like Like x 1
  20. dko22

    dko22 Mu-43 Regular

    163
    Jul 26, 2010
    Stuttgart, Germany
    I'm glad that most have treated this in a sanguine way and even the slightly less sanguine exchange was for me at least, very interesting. Thanks to all who have posted. Like most, I prefer to see real images and reviews taken from RAW images and we're not quite there yet. I have to admit that I will never really understand sensor tests, not having any sort of engineering background. In particular, I'm not clear what role data processing and circuitry play in these tests. If the GH2 has better overall electronics and thus produces visibly better images in real life than the GH1, then will the DxO tests show this? If not, are they not rather meaningless? There is such a gap between early reviews and tests with, to me, a visible improvement in image quality, compared to the negative DxO test. Is it really possibly that everything has been done with jpg fakery? My experience is that well converted raw files will nearly always give much better results than any camera jpg engine which would suggests that the GH2 is indeed a significantly better camera from the point of IQ

    David