Lens Rating System

Discussion in 'Native Lenses' started by boatman37210, Jul 10, 2012.

  1. boatman37210

    boatman37210 Mu-43 Regular

    Jun 28, 2012
    I bought a new EPL-1 and am in the process of making a list of lenses I would like to purchase. Based on responses to a previous post, I've ordered the Zuiko 14-42 Mk2 as my first lens. In order to keep this post as short as possible, I will not go into how I arrived at the 14-42.

    Here is a post from another forum concerning the 14-42 M4/3and comparing it with the 12-60 4/3. I post this as merely a reference and not as a fact:

    "I had the zuiko 12-60 which is considered one of the best lenses in it's class.
    I have many photos taken with that lens; and many taken with the 14-42 with so called horrible resolution. You would not tell the difference between the two at any print size. I guarantee it.
    Unless, of course, you are shooting test charts and looking at the results on a computer screen at 200%. But surely you are not doing that sort of photographer are you?"

    I think most people do not care for the 14-42. Those that think it is a decent lens and is underrated are in the minority. But as the reference says above, if you are not doing test charts and 200% computer screens, what does it matter. If, and I stress if, the printed pictures are equal, then why does the 14-42 gets such a bad reputation. I understand that maybe at some picture size the qualities will start to diverge, but is that point so large that 99.9% of the users do not go there and do not care.

    As I go through the process of trying to choose lenses, I've been thinking there has got to be an easier process. You read and read and read and still come up with conflicting information.

    I pose the following question. What if lens were rated based on acceptable print size? For example, a lens could be rated as acceptable for printing pictures up to 8 X 10 before seeing any noticeable distortion. Another lens could be rated as acceptable to print pictures up to 11 X 17. The 11 X 17 lens is $200 more, but all I need is a lens that will print nothing bigger than 8 X 10, so I will choose that lens and save myself $200. Not sure if rating by print size would serve those who only use the web and PC, but hopefully you understand what I am getting at. I have seen reference to print size on occasion concerning certain cameras, but do not remember seeing on any lens.

    The best lenses rise to the top of everyone's conversation and the review sites, so you can feel comfortable in buying those lens. But if other lenses other than the best will serve your purpose and save you money, to me there is not an easy way to determine what those lenses are. Too many differing opinions and to me the review sites are too vague.

    I just think there should be a rating system that would make it much easier to pick a lens. Even after reading all the available information, I still have doubts if my choice of lens was the right choice. To me there should be a rating system where you could determine if a lens will serve your purpose and end result. Maybe that is not possible and what we now have in place is the best we can do.

    Would appreciate other people's thoughts. Does my rating system proposal have merit or being a newbie am I showing my ignorance?

  2. Humiliated_grape

    Humiliated_grape Mu-43 Rookie

    Jul 8, 2012
    Sydney, Australia
    Real Name:
    I think your rating system is an interesting idea (but also being a newbie, maybe I'm showing my ignorance as well). Having gone through the process over the last few months of trying to choose my first few lenses, and knowing little to nothing about photography when I started the process, I completely understand your frustration with people's differing views on quality of lenses. It nearly drives you insane - just as you've settled on a lens being "the one" for you, you find someone who says its crap.

    However, I think your system would still be very subjective - after all, people will still have different standards as to what they find acceptable quality for different print sizes. For example, I have some of my photos on my desktop that I find perfect quality for that size, but someone with a keener eye would probably find it completely unacceptable. I'm not sure there's ever any one ratings systems you could apply to any piece of technology.
  3. mattia

    mattia Mu-43 Hall of Famer

    May 3, 2012
    The Netherlands
    The problem is that resolving capacity of a lens really isn't the only issue - in fact, for most types of photography it's ultimately fairly irrelevant. Most decent to good lenses will resolve more than enough detail for even large prints. Absurdly large prints (which I am quite fond of, to be fair) will place greater demands on the glass, but again, only at close viewing distances.

    What arguments such as the above ignore are other features such as AF speed, handling, size, weight and slightly less quantifiable variables such as rendering. My Contax/Zeiss 50/1.4, for example, has beautiful rendition of color and contrast, but is nowhere near the sharpest 50mm lens I own. Still a favorite. The Canon 35/1.4L has fantastic rendering, but the 24-105 @ 35mm, and definitely the Zeiss 35/2.8 are better landscape lenses when stopped down - but less useful or attractive for environmental portraits.
  4. boatman37210

    boatman37210 Mu-43 Regular

    Jun 28, 2012
    I know other factors enter into the decision making process such as AF, speed, color and contrast, etc., but you need to start somewhere. Since the purpose of a camera and lens is to take a picture, I thought the printed picture would be a good place to start. Other rating categories could be included and users could filter through to pick the lens that best suits their needs. Maybe there are too many variables and subjective items for any standardized system to be created.
  5. jyc860923

    jyc860923 Mu-43 Hall of Famer

    Feb 28, 2012
    Shenyang, China
    Real Name:
    That's true, I agree.

    Sent from my HTC Incredible S using Tapatalk 2
  6. boatman37210

    boatman37210 Mu-43 Regular

    Jun 28, 2012
    "Most decent to good lenses will resolve more than enough detail for even large prints."

    I am proposing a rating system because trying to figure out what is "good or decent" is not an easy process. For instance the Sigma 19mm and 30mm. Many people do not like them and they have gotten some bad reviews. But if they will produce a printed picture as good as any other lens up to say 10 X 12 for example, then they would be on my list of possible choices. But because so many people do not like them and bad reviews, they are not on my list. But what was stated earlier about it being subjective what is acceptable and what is not comes into play.
  7. D@ne

    D@ne Mu-43 Top Veteran

    Feb 23, 2012
    In order for me to print a picture, it has to impress me on my computer's display. If the lens used cannot capture the detail, match my expectations for sharpness, etc., I probably won't be interested in it.

    My (personal) theory - if one is on a budget, I completely understand that getting good quality for the money is important. One would have to factor in quality-for-price into the equation in order to rate lenses, but this could be a moot point if one's budget allowed for a more expensive lens. In general, all of us have different priorities when choosing a lens, so a rating system would be too subjective to be accurate and effective.

    Just my 2 cents.
    • Like Like x 1
  8. mattia

    mattia Mu-43 Hall of Famer

    May 3, 2012
    The Netherlands
    I haven't seen many people complain about the image quality delivered by Sigma. Gripes, if any, seem to be size (not designed for MFT natively, so unnecessarily large), speed (relatively slow compared to the Oly and Panny offerings) and autofocus ability.

    Honestly, I don't think there are any really bad lenses available for MFT at the moment.
  9. boatman37210

    boatman37210 Mu-43 Regular

    Jun 28, 2012
    "In general, all of us have different priorities when choosing a lens, so a rating system would be too subjective to be accurate and effective."

    I agree everyone has their priorities, but that would be the purpose of a rating system. A tool to help determine the lenses that meet your priorities. A $500 lens might meet my priorities where another person might need a $1,000 lens. Maybe the only difference between the two lens is that the $1,000 lens prints larger pictures and I do not need large, so the $500 lens would be fine. Maybe the $500 lens has more distortion or CA, but does not show up until you get to a certain size picture. That might work for me since I do not need pictures that large.
  10. Fmrvette

    Fmrvette This Space For Rent

    May 26, 2012
    Detroit, Michigan
    Real Name:
    Boatman -

    Interesting proposal.

    I would first suggest that only a small minority of photographs are actually printed these days; most end up being shared as .jpg files on FlickR, being emailed for review, placed in one of those loathsome 'electronic picture frames', etc.

    The 'acceptable print size' might not have much correlation to how photographs taken with a rated lens look when displayed on a laptop/desktop/netbook/tablet monitor.

    'Acceptable print size' is also printer dependent. Not that some printers can handle larger paper stock than others, rather that some printers by their very design generate better looking print than those created by lesser machines on identical paper sizes.

    For example, a Kodak Hero 7 printer cannot easily measure up to a Pixma Pro 1, and prints from the two machines will show a palpable difference in quality even if the same digital "negative" is used.

    So if a Pixma Pro 1 is used to create an "acceptable" 8x10 and the lens is rated using that measure, those who print using the Kodak may find their results less acceptable. (I would also point out that not all paper is created equal; an image that is 'acceptable' on one print paper may not look as sharp when printed on different stock).

    Another factor that affects a lens rating is subject matter; a Nikon 35-135 lens can create an image that easily prints well at 8x10 if the subject is, for instance, a portrait. However take the same lens and photograph a building and the resulting print may show an unacceptable amount of barrel distortion - the vertical lines of the building do not follow a true path.

    So - is the 35-135 "acceptable" at 8x10? Or is it to be rated "unacceptable" because it cannot render an 8x10 of all subjects?

    Post processing, of course, adds an entire new set of considerations. If post processing is allowed then one has to factor in the software being used, the skill of the operator, etc. If no post processing is allowed then one may find oneself lowering the rating of a given lens even though the issues that keep a print from the lens from being 'acceptable' are easily correctable in post processing. (The Nikon lens above is a good example - barrel distortion can quite easily be corrected in, say, PhotoShop Elements thus allowing an 'acceptable' 8x10 to be made).

    I absolutely understand your frustration in the lack of a common denominator in rating lenses. However I'm not certain that having a 'maximum acceptable print size' is doable because there are simply too many variables in what constitutes an 'acceptable' print, not the least of which is the eye of the beholder.

    Just my 2 cents, fully worth half of that.


    • Like Like x 1
  11. DeeJayK

    DeeJayK Mu-43 Hall of Famer

    Feb 8, 2011
    Pacific Northwest, USA
    Real Name:
    While I understand and can appreciate the desire to have some sort of lens ranking, trying to base your proposed system on print size seems simplistic. Frankly, just about any :43: lens in capable hands and with good shooting conditions (lighting, etc.) can produce quality output of nearly limitless size (given that as print sizes increase, viewing distance also increases).

    Additionally, print size is most directly related to resolution, so the camera's sensor (the number of megapixels) has a much greater impact on the maximum print size than does the lens.

    The Olympus 14-42 kit zoom is perhaps the least well regarded lens in the system. Bear in mind, I'm not saying it's a bad lens, in fact, it is a terrific value. A good photographer, given that lens and any :43: body you choose and a day could certainly create an image that I would happily hang in my home at any print size.

    Frankly, what you're generally paying for with more expensive lenses is to increase the range of shooting conditions in which you can capture a usable image as well as the ability to compose the shot you are envisioning. Generally this has to do with the lens' maximum aperture (the size of the diaphragm through which light reaches the lens). Aperture is expressed as the f/ number, which is a ratio of the lens' focal length to the diameter of the aperture. Lenses with a larger aperture (smaller f/ number) will be capable of performing in less light and/or with fast shutter speeds and/or with a shallower depth of field. All of these attributes can allow you to capture shots that you couldn't get with a lens that does not capture as much light (i.e. one with a smaller maximum aperture).

    Lenses with a larger maximum aperture are said to be "faster." Designing, engineering and manufacturing fast lenses is more difficult, so these lenses are more expensive than "slow" lenses.

    Fast lenses are not necessarily better than slow lenses. Generally they are heavier, for example. If your shooting is restricted to well lit, static subjects then you may find that there is no need to invest in faster lenses.

    So, in short, I think that it's much more useful to think in terms of conditions in which a given lens can provide adequate results than in print size. Once you determine the conditions in which you need or want to shoot, then you can choose the lens that best fits those situations.
  12. boatman37210

    boatman37210 Mu-43 Regular

    Jun 28, 2012
    Thanks for your response Jim. Great points.

    Maybe you could rate based on how pictures are viewed on a PC eliminating the printer and paper problem. No post processing allowed. Let the user determine if he has the software and ability to fix any problems. Give CA, distortion, shading, and sharpness ratings at different viewing percentages. The user would have to be able to view the same picture or pictures at the same monitor resolution that are used to do the ratings. Resolution and monitor pixel density would need to me standardized.

    I know there are things I have left out, but I can't help but think that smarter brains that mine can find a better way.

    It is what it is and not going to change. I was just interested in getting other peoples thoughts.

  13. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Real Name:
    Overall, I think lens sharpness is severely overrated. In most cases, decisions between 1 lens and another should be made based upon budget, focal length (range), AF speed, size and max. aperture (if shooting in dim conditions). And indeed in the vast majority of cases, differences in optical quality are somewhere between negligible and invisible on prints and websites.

    I agree that a simple rating system might be nice, but in the end, even that is a bit suspect to me, since it's the non-optical factors that should typically be decisive. The only case where that is relevant is if there are several lenses meeting one's criteria. Then sharpness makes some sense as a tiebreaker.

    • Like Like x 1
  14. strang

    strang Mu-43 Veteran

    May 7, 2012
    Anything subjective should be avoided.

    Wait... That is photography in general.

    It's tough to put a score to a lens. What I'd like to see is as much information about a lens as possible in a clear concise and readable fashion. Then the informed can make their own decision.

    We already have a few sources for information. Four-Thirds' official site, SLRGear, the list on this forum, etc. Perhaps what should happen is to centralize all of that even further and crowd-source the information.
  15. phrenic

    phrenic Mu-43 All-Pro

    Sep 13, 2010
    Neat idea..the hard part is determining what's acceptable, purely a judgement call. That said..I'm sure almost none of us really print that much (compared to time spent on photo sites/forums) taking pictures etc. I print mostly 8x10's and I can't always tell the difference between my nicer lenses like the 20mm and the 14-42mm oly mk1 at typical viewing distances.

    I would be reluctant to upgrade for better sharpness..but I think the big difference for the more expensive lenses is the speed, where you will either get the shot or you won't.

    But maybe I should make some more big prints. :) Haven't gone past 11x14 (which looked great with the 14-45mm, 9-18mm and OM 50 1.8..all perfectly acceptable).
  16. fin azvandi

    fin azvandi Mu-43 All-Pro

    Mar 12, 2011
    South Bend, IN
    Quoted for truth. In good light any m4/3 body and native lens should be able to produce wonderfully sharp images for 8x10 prints. The way to decide if you need a new lens is to decide what other factors DH listed above are most important to *you* for the images you are trying to make.
  17. usayit

    usayit Mu-43 Hall of Famer

    Yup... I think rating and pixel peeping reviews of lenses sucks the enjoyment out of photography. This may sound silly but I prefer more subjective reviews.. some even write pretty well too.
  18. Fmrvette

    Fmrvette This Space For Rent

    May 26, 2012
    Detroit, Michigan
    Real Name:
    Actually I have the perfect methodology for selecting a lens.

    After doing due diligence in reading tech specs, getting reviews from Amazon, checking out DPReview, studying The Online Photographer site, (and avoiding Ken Rockwell like I would the three day measles) I narrow the choices down to the top two choices.

    I print out the specs and the purchase price.

    I ask my wife.

    She says NO!

    I am off the hook for buying new gear and can happily go outside and play with my existing toys.

    Works every time. :rofl:

    • Like Like x 1
  19. boatman37210

    boatman37210 Mu-43 Regular

    Jun 28, 2012
    Sounds like to me no rating system needed. You have given me a better understanding and made the lens choice process a lot easier.
  20. cookme

    cookme Mu-43 Regular

    May 25, 2012
    From all my m43 lens researches, I could only come up with 2 categories: the exceptional lenses that you can't really go wrong, and the usable lenses as long as the cost/performance ratio is acceptable. No lens-to-avoid types, yet.

    There are just too many variables in all the "scientific" and user reviews/ratings. Shooting condition, cameras used for tests, human bias (physical & mental), and quality control issues from the manufacture are a few big ones that make you wonder if the reviews/ratings you read will match your own experience. Trusting the common 5-star user review systems are an ok way to judge the overall performance of a lens but it doesn't work for new or expensive lenses since the sample size is none or too small.

    Unfortunately many of the local camera stores don't have enough/any m43 lenses to test with, so that rules out of test-before-you-buy for most people, especially with new lenses. Until the day when a reviewer has enough dough to get lens from multiple sources and test them on multiple cameras using a wide range of test methods, I think this is always going to be an issue for all the lenses that are not top-of-the-line super-performance types.

    If people can buy test lenses and cameras from reviewers to offset the sample variety cost, it might be a workable model. However the resources and time needed to test the lens + camera + test condition permutation is still way too heavy.