4 cores vs. 6+ cores for Lightroom

Discussion in 'Hardware' started by dhazeghi, Mar 18, 2014.

  1. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    So I'm still wrestling with unsatisfactory Lightroom performance (everything, but mainly rendering). Current system is a Lynnfield Core i7 860 (4 x 2.93GHZ), 12GB RAM, 240GB SSD.

    My question is whether anybody here has experience using a 6+ core Intel-based system for Lightroom, and whether the additional cores make a real difference. Going off of Lloyd Chamber's benchmarks, my guess would be no, but it would be good to have confirmation either way. Going to 6 cores increases the cost of my upgrade by a good $500 (X79 + 4930K vs. Z87 + 4670K), but if it really speeds things up by 35 or 40%, I think it would be worth it!
     
  2. John M Flores

    John M Flores Mu-43 All-Pro

    Jan 7, 2011
    Somerville, NJ
    I've got a Core i7 2.2GHZ, 16GB RAM, 512SSD Macbook Pro. Lightroom was crawling. I've since broken down my 200k photo catalog into smaller catalogs and performance is much better.

    My main catalog is now 116k photos and it is much much faster. It probably has something to do with the fact that my *Previews.lrdata has shrunk from over 100GB to 45GB.
     
  3. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    Yeah, the previews collection can be an issue, but that's not what I'm seeing. Indeed, I started a thread a few months back and the performance I'm seeing on my system is pretty normal. I do my testing with a fresh catalog and whether I use and SSD, HDD or RAM disk makes no meaningful difference on the rendering times I see.
     
  4. dav1dz

    dav1dz Mu-43 Top Veteran

    926
    Nov 6, 2012
    Canada
    Though I don't like reading Lloyd Chambers, this is quite good: http://macperformanceguide.com/Optimizing-Lightroom.html

    Though it's written for OS X, the theory is the same and Adobe has largely used the same core for different builds of their products. Makes sense given that they need to roll out updates quickly and efficiently.
     
  5. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    Thanks - yes I've been reading Lloyd's site for a bit now. The crux of the matter is that in his benchmarks, the 6 core 2013 Mac Pro is a mere 11% faster than the 4 core variant, and the 8 core is 11% faster than the 6. Despite being Macs, they're basically using the CPUs I'd be in my own build (Xeon versions of the 4820K and the 4930K).

    If Lightroom were a properly optimized program, you'd expect to see a 35-40% increase by going from 4 to 6 cores. So either his test is wrong, or Lightroom is leaving a lot of performance on the table. Question is whether other people have similar (or different) experiences?
     
  6. nstelemark

    nstelemark Mu-43 Hall of Famer

    May 28, 2013
    Halifax, Nova Scotia Canada
    Larry
    Multiple cores are not magic and as b_rubenstein noted writing software for multiple cores benefits certain algorithms a lot and others not so much. I also would not assume that multiple core benchmarks across platforms are valid. Software is written on top of an OS API which then talks (after some number of layers) to the hardware. That hardware is somewhat the same for Mac and PCs but the multi-threading APIs in windows and mac OS are guaranteed to be different. I have not been keeping tabs on Grand Central (the MacOSX multi-core API) as much recently but I know that it certainly has had issues. Windows I really don't keep tabs on, so your YMMV.

    Your best bet is actual benchmarks with the software you want on the platform you want.
     
  7. nstelemark

    nstelemark Mu-43 Hall of Famer

    May 28, 2013
    Halifax, Nova Scotia Canada
    Larry
    I looked at his benchmarks and I don't think his comparisons are particularly valid. He is assuming that "cores" are his limiting factor.

    My bet is system throughput is really what he is measuring. Note how well the MBP did. The combination of PCIe SSD and the newer chipsets seems to be a bigger factor than "cores".
     
  8. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    Except that we already know that Lightroom does split the rendering across different cores - just re-render the preview of a single image and watch the activity spike on several cores. Indeed the scaling from 2-4 cores does give a 1.5x-1.6x increase in speed, but the appearance is that from 4 to 8, it gives far less.

    Even if what you said were true in this case, Lightroom is still bottlenecked where it shouldn't be. Lloyd's doing an import of 100 images, which can be split into up to 100 threads, or more realistically, one thread for each of n cores, with a queue of ceil(100/n) images to render. The only serial part of the operation is updating the Lightroom database, which in practice is nearly instantaneous with a fast SSD. The 12-core machine has a base clock of 2.7GHZ. The 6 core machine has a max turbo clock of 3.9GHZ, or 44.4% faster. The 6-core machine has to process 17 images/core for a balanced load vs. 9/core on the 12-core machine.

    In practice, the 12-core machine runs this test in 647 seconds, vs. 635 seconds. Yet even the wildly conservative assumptions above suggest we should be doing much better. If we assume an excessive 35 seconds for DB updates and other serial operations, that leaves, 600 seconds for the 6-core to render 17 images on each core, or roughly 35 seconds/image. The 12-core, if it required 44% longer for each image, would need 51 seconds/core. At 9 images core, that makes for 458 seconds. Add back the 35 seconds of serial operations and you get 493 seconds, more than 20% faster than the actual numbers.
     
  9. Ricoh

    Ricoh Mu-43 Top Veteran

    906
    Nov 2, 2013
    UK
    Steve
    Had a scan of the posts above, mention of a GPU and it's effects on performance doesn't leap out at me. I'm sure everyone realises the benefits c.f shared graphics processing.
     
  10. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    I doubt the threading API is especially significant here - else we would have heard complaints about Lightroom being substantially slower/faster in Windows/MacOS X. Notably, Lightroom import and rendering performance hasn't changed significantly since at least Lightroom 3 (I tested on Windows and MacOS X), and LR 3 certainly didn't use GCD.

    Well, the platform I'd be using is essentially identical to the one he is - MacOS X on an X79 or C60x-based system using Ivy Bridge-EP. Or perhaps Z87 Haswell (which is essentially what their iMacs run off). Not that I wouldn't mind doing my own test, but as I'm building my own system, it'd be kind of hard to send it back if I wasn't satisfied.

    I'd be very very surprised if SSD performance were more than incidental to his results. Even on much slower systems that I've tested on, the difference in import performance between a slowish SSD and a RAM disk was basically within the margin of error for the test.

    I also stumbled upon a comparison on Anandtech of export performance which suggests even worse results - LR on the 12-core Mac Pro was slower than on the current 4-core iMac.
     
  11. nstelemark

    nstelemark Mu-43 Hall of Famer

    May 28, 2013
    Halifax, Nova Scotia Canada
    Larry
    This was exactly my point. I don't think the application is CPU bound at all. If it was the 12 core MacPro and the new MacPro with the dual Fire GPUs would be substantially faster than the MacBookPro. The current MPBs do have upgraded storage (SATA vs PCIe SSD) and based on the numbers this looks like it has more impact than CPU/GPU performance. Could LR be faster? Maybe, but right now it looks system bound not CPU bound.
     
  12. pdk42

    pdk42 One of the "Eh?" team

    Jan 11, 2013
    Leamington Spa, UK
    I'm not wild about LR's performance either and I find it especially slow when rendering multiple images in the X/Y view of Library. My machine isn't too powerful by modern standards and I find it's getting 100% maxed out on CPU. OTOH, LR is way faster than Viewer 3, so I'm coping...

    As regards using all processor threads, I suspect that LR's basic design pre-dated serious use of parallel processing so it's not surprising that it's not using all cores to max effect - although I agree with others posters about the difficulty in writing software that does efficiently run parallel threads.
     
  13. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    I'm not sure what the GPUs have to do with the whole thing - for good or ill, LR makes no use of them to accelerate its rendering.

    As to the difference in storage, I'm just not seeing either theoretically or practically how it could matter. All of the recent systems Anand tested are using PCIe flash, which are good for 800MB/s real-world transfer rates. How that could be a significant bottleneck when the rendering is topping out at 1 image per second, I don't know.

    At any rate, I just ran a quick sanity check - exporting 100 images from a catalog - and the difference between running the catalog off of a RAM disk and a 7200RPM HDD amounts to less than 3 seconds in a 450 second operation. That's well within the sampling error, and considering that I sped up the I/O bound part of the process by literally >1000x, doesn't jive at all with the idea that the operations are I/O bound.

    Olympus doesn't really do software well. Then again, neither do any of the manufacturers except maybe Canon. It's a shame we can't go back to the good old days where cameras shipped with the equivalent of PSE, and all the manufacturers did was provide a plugin.

    I quite agree that efficiently threaded software is a challenge, but we are after all paying for a professional piece of software. If this were a hobbyist project or a bundled application a la Viewer, it would be different. IIRC development began in 2005, at which point all high-end machines were dual core. By the time version 1 left beta, the high end was already quad core. So they've certainly had plenty of time to design appropriately. Granted, Camera RAW has a longer history, but it has certainly seen major overhauls over the same period (sharpening, noise reduction, layered edits).
     
  14. ckrueger

    ckrueger Mu-43 Veteran

    304
    Jul 16, 2011
    I've been using a 12 cores for a while and while it does help a lot with preview rendering (which splits across all 12 cores nicely for LONG periods of time on large imports), the rest of the app doesn't scale well.

    If you spend a lot of time ranking photos, zooming in to 100% to check focus, and letting LR re-process your files as you drag sliders around, you're not going to use all those cores. I haven't looked at Lloyd Chambers' suggestions, but watching CPU use while LR picks its nose for the past couple years, I'm pretty confident recommending you put your CPU budget towards fast cores instead of more cores, because the poorly parallelized parts of the UI are where you're going to be spending most of your time (assuming you walk away from the computer while batch rendering previews like I do).

    If you're talking Xeons, I'd aim more at a CPU like the E5-2643v2 (6-core 3.5GHz) than the E5-2670v2 (10-core 2.5GHz). Not specific recommendations (my Xeon experience is only with my one LR computer and at work for server use), but to illustrate what I mean.

    You might consider instead a high-end i7 like the (last-gen) 4930K (6-core 3.4GHz) or one of the new Haswell quad cores and putting the large savings in motherboard and CPU into storage. I saw a 1TB Samsung SATA SSD for $500 on Slickdeals the other day, and with such a big savings you could even venture into PCIe flash.

    As an aside, don't bother with a bunch of RAM. I had a DIMM go bad on my LR computer and went down from 16GB to 8GB to no noticeable effect.
     
  15. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    Another interesting datapoint is Tony Hart's Lightroom test mule benchmark. I'm still digesting the results, but it looks like the jump from 6 to 8 cores also doesn't produce too much of an improvement (~8% overall, 12% for rendering), although from a 4-core iMac 2013 to a 6-core Mac Pro 2013 does actually move things a bit more (~20% for rendering).
     
  16. nstelemark

    nstelemark Mu-43 Hall of Famer

    May 28, 2013
    Halifax, Nova Scotia Canada
    Larry
    This is what I was driving at, not IO performance but system performance. Faster overall system performance and likely the best single threaded performance is what you need. My comment on the PCIe SSD came from the fact that the current MBP is pretty much the same as the old one 4 core i7 etc and the only real boost was the SSD. But really any app that performs roughly the same on the MPB as the new MacPro is not optimised for the new hardware. If LR really is primarily single threaded then GPU support would be a big benefit for any image handling.
     
  17. jnewell

    jnewell Mu-43 All-Pro

    Jun 23, 2011
    Boston, MA
  18. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    Thanks very much for the report. It's great to have some real-world feedback.

    The RAM part makes sense - I went from 6 to 12GB recently with no notable gain either. Regarding CPUs, I'm aiming primarily for 'bang-for-the-buck' so Xeons are probably out. The main question is whether it's worth it go with a platform that will scale to 6 cores (X79 + 4930K) or stick with one that's limited to 4 (Z87 + 4670K). I'm getting the feeling that 6 cores isn't really worthwhile for Lightroom, so the premium (+$100 MB, +$350 CPU) may not make sense. On the flip side, a $1k 6 core box would be kinda cool just for the heck of it. I guess the other option would be to wait for the Haswell-E refresh, but my impression is that it'll be a long wait (3 months for Intel, then 4-5 months for Apple/the Hackintosh community) for minimal improvements other than a possible price drop for the 6 core CPUs.
     
  19. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    Okay, I think I see your point. It's just there aren't that many relevant variables that I can think of that differ significantly among those systems other than CPU clock and core count. Memory bus width and speed is pretty comparable, PCIe implementation is comparable, etc etc.

    I should be clear that I don't think LR is entirely single-threaded for any significant operation - there's clearly LR thread activity on all 4 cores of my machine whenever I'm importing, exporting or developing images. But it doesn't seem to be scaling particularly efficiently past a certain point. I did get to run it briefly on an 8 core system a while back with different numbers of cores enabled, and it didn't get very far above 4 cores. Unfortunately, that system had a bunch of other issues, so it wasn't enough to draw any definite conclusions.

    Thanks for the link. Interesting to hear that you didn't find the 4770 to be such a huge improvement as the hardware would suggest.
     
  20. dornblaser

    dornblaser Mu-43 Hall of Famer

    Aug 13, 2012
    Chicago-area
    David Dornblaser
    I can't comment about LR as I use Aperture but I do have some observations about Adobe. Adobe is pretty good about evolving their software to take advantage of hardware evolution. When it comes to hardware decisions today, I would want error on the side of future proofing. While the number of cores as important for PS (& presumably LR), I think that the hex really hits the sweet spot for number of cores and clock speed. I also think that SSD and plenty of RAM are two options that should not be overlooked.

    FWIW - I am currently iMac based and soon to be hex Mac Pro based.