Benchmarking Lightroom on different hardware

Discussion in 'Open Discussion' started by dhazeghi, Jan 10, 2014.

  1. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    I've been finding the performance of Lightroom frustratingly slow at rendering ORF files for quite a while, and I'm finally at the point of considering a serious hardware upgrade for my computer (which I realized is now more than 4 years old). Before I do that though, I'd like to have a better sense of how much of a difference newer hardware would make. I was thinking that there are a substantial number of folks on this forum with pretty good machines, and if a few could run a sample workload as a benchmark, it would help immeasurably in the task.

    Basically, the idea would be to time (with a stopwatch) Lightroom's import and rendering of a pre-selected set of ORFs using some common settings (Adobe default image settings, 1:1 previews, no other operations), and report back here. Preferably using the same Lightroom version (5.x), but I doubt that will make a whole lot of difference.

    Would anybody be willing to do this? If there is interest, I'd be more than happy to provide a set of images. Thanks!
     
  2. Amin Sabet

    Amin Sabet Administrator

    Apr 10, 2009
    Boston, MA (USA)
    I'm willing!

    Sent from my SCH-I535 using Mu-43 mobile app
     
  3. ThomD

    ThomD Mu-43 Veteran

    428
    Jun 1, 2013
    SF Bay Area
    Sign me up
     
  4. jloden

    jloden Mu-43 Hall of Famer

    May 15, 2012
    Hunterdon County, NJ
    Jay
    Sure, I'd be curious too.

    I'm on a Macbook Pro 2.4GHz Core 2 Duo with 8GB RAM, and I've found LR to be painfully sluggish at times... be curious to see where I sit on the performance continuum and how much I might expect to gain from a hardware upgrade.
     
  5. brettmaxwell

    brettmaxwell Mu-43 Veteran

    350
    Dec 8, 2012
    I'd be willing. I've got two machines right now, one is an entry level current model Mac Mini, the other is a PC laptop that's very close to the fastest laptop you could purchase in 2012. Latest LR on both.
     
  6. pdk42

    pdk42 One of the "Eh?" team

    Jan 11, 2013
    Leamington Spa, UK
    Id be happy to enrol too. My PCs aren't über-latest spec, but that might act as a good calibration point. One's a desktop E5200 dual-core Pentium overclocked to 3.2GHz which benchmarks at about 1/8th power of a 3.2GHz i7. The other's a laptop with a Core a Duo at 2.8GHz which is about 25-50% faster than the E5200. Som- neither great performers by modern standards.

    LR is acceptable on both though - but frustratingly slow on occasion.
     
  7. pdk42

    pdk42 One of the "Eh?" team

    Jan 11, 2013
    Leamington Spa, UK
    I'd be happy to enrol too. My PCs aren't über-latest spec, but that might act as a good calibration point. One's a desktop E5200 dual-core Pentium overclocked to 3.2GHz which benchmarks at about 1/8th power of a 3.2GHz i7. The other's a laptop with a Core a Duo at 2.8GHz which is about 25-50% faster than the E5200. So, neither great performers by modern standards.

    LR is acceptable on both though - but frustratingly slow on occasion - which in itself is an interesting observation; and it's not disk activity, the CPU maxes out sometimes when rendering a shot.
     
  8. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    Okay, great! Looks like more than enough folks for a good test. I really appreciate the offers. I'll upload some ORFs when I've got access to a fast connection this afternoon, and post the link here. I'm thinking 20-25 files would be a sufficient test.
     
  9. mattia

    mattia Mu-43 Hall of Famer

    May 3, 2012
    The Netherlands
    If it's for LR4 I can chip in with MacBook Air (i7, previous generation, 4gb RAM) and a hexacore Mac Pro.
     
  10. danieru

    danieru Mu-43 Regular

    61
    Nov 6, 2011
    Netherlands
    Daniel
    Sign me up :biggrin: I'd be really interested as well, coincidentally I just upgraded my pc this week, one of the reasons being lightroom performance, I benchmarked Lightroom before and after, using Topaz Denoise plugin and timed it, performance is at least twice as fast if not even faster compared to older system.

    I swapped out motherboard and cpu, basically went from AMD (been using them for more then a decade) to Intel system.

    Old:
    AMD Phenom II X4 945 3.0 Ghz
    16 GB memory
    Ati HD 7570

    New:
    Intel i7 3770 3.4 GHz
    16 GB
    Ati HD 7570
     
  11. b_rubenstein

    b_rubenstein Mu-43 All-Pro

    Mar 20, 2012
    Melbourne, FL
    There are a number of LR settings that will effect import/rendering speeds such as the size of rendered image. Whether or not you have an SSD for writing the files to also effects speeds. If you go to the Adobe site, there are guides for optimizing LR performance under Windows and on Macs.

    FWIW, I have two PCs that I run LR 5.3 on. One is based on a i5-3570K and the other on a i5-4670K, both OC'd to around 4Ghz, with 16gb RAM and libraries stored on SSD's. I generally do batch imports and don't pay much attention to how long it takes, but it's on the order seconds per file. While working with a file, I don't notice any difference in responsiveness between RAW and JPG files: it's all instantaneous.

    I don't recall if LR uses the graphics card for accelerating rendering, but I'm just using graphics built into the CPU (no video card).
     
  12. ThomD

    ThomD Mu-43 Veteran

    428
    Jun 1, 2013
    SF Bay Area
    We need some consistent methodology to be able to compare results. We need processor, RAM, OS, hard drive type (and model), LR config options and what exactly we are timing. Ideally, each test should be run at least 3 times reporting the average. I wonder if there is some other 3rd party benchmark we could also include.

    On my machine I have 3 drives, so I'll run the tests on catalogs on each and report those results. Come to think about it, I'll probably run this at least 4 times - once importing from my fastest SSD into a new catalog on each drive. That means importing from the SSD to the SSD. So, I'll do an extra import from my second fastest drive into the SSD.

    I don't know what LR settings play into this, so somebody else will need to spec those. Should these imports go into a new clean catalog? I don't' know how catalog size plays into performance.

    I'm going to guess that SSD vs HD makes a bigger difference than any other moderate config change. We'll see.
     
  13. b_rubenstein

    b_rubenstein Mu-43 All-Pro

    Mar 20, 2012
    Melbourne, FL
    Running a series of test is time consuming. The test results aren't very useful unless the variables are controlled, and there is no way to have consistent OS environments on different personal computers. The difference between a clean install/old install, having all the same software installed the same background and the same background programs running, it's not possible to clearly determine speed differences.

    It's probably more productive to do a web search for LR or PS benchmark results.
     
    • Like Like x 1
  14. lombardispot

    lombardispot Mu-43 Regular

    100
    Sep 19, 2013
    Maybe so. But when someone reports a fast speed, you've got something good that others can imitate.





    Sent from my iPhone using Mu-43 mobile app
     
  15. pdk42

    pdk42 One of the "Eh?" team

    Jan 11, 2013
    Leamington Spa, UK
    On reflection, I think I agree - unless the sample size is big enough to draw some statistical conclusions; but I guess it'll have to be quite large and probably bigger than we can muster here...
     
  16. silver92b

    silver92b Mu-43 All-Pro

    Feb 7, 2013
    Atlanta, GA
    I just upgraded to LR5 and I'd be quite willing to test it in my Macbook pro and my Windows 7 64 bit desktop. BTW, the LR5 seems to be very fast for rendering raw files from the EM5 as well as the GX7 in my Macbook pro....
     
  17. HappyFish

    HappyFish Mu-43 Top Veteran

    983
    Sep 8, 2012
    Chad
    Few thoughts
    I have a post production company we deal with pro togs only so we do a lot of LR work PS work album design etc...

    To me develop module is where I want things to be quick as you can't do other things going from one image to the next

    Import export etc.. I can start and check email eat lunch :) do many other things
    Not saying one does not want those quick also but as a workflow person automating as much as one can and doing other things when one can during that import export is key

    Before my post company I would shoot about 200 jobs a year and had a pretty good workflow :)

    For develop speed LR5 is a lot quicker and building smart previews then put a x in the folder name so it forces smart previews to be used for those who want quicker develop module speed :)

    Testing some import export amongst Mac pros 1,1 and 3,1 and 5,1 models LR gets in the way more than hardware ?
    How full our ones drives ?
    SSD or regular ? SSD are quicker :)
    RAID arrays like Areca with large buffers etc...
    Even multiple SSD in raid arrays don't make huge differences
    Yes they get quicker but not huge gains sadly

    Which is a pain about LR the import the building previews etc..


    Will be interesting to read info though :)
     
  18. ThomD

    ThomD Mu-43 Veteran

    428
    Jun 1, 2013
    SF Bay Area
    So, for anyone who thinks this is not time well spent or interesting, think of this as a photo challenge that you decline to participate in. The limitations of this exercise have been pointed out.

    Don't be a wet blanket.
     
  19. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    All right, I've uploaded a test data set of 25 ORFs. It's about a 325MB download. No copyright, so feel free to do whatever you please with them.

    For testing procedure, I'd recommend the following:

    1) Create a new testing catalog. This will avoid messing with your real catalog, not to mention the overhead of different folks having different sized databases.

    2) When importing the test images, make sure that under 'File-Handling', 'Build Previews' is set to '1:1' and 'Develop Settings' is set to 'None'. If you've changed the default preset applied to the E-M5, please set it back a preset consisting of Adobe Standard and the LR original defaults before running the test. I have one called 'Simple' with those settings.

    3) Start timing when you click 'Import' and stop when the progress bar for 'Building 1:1 Previews' has completed.


    When reporting results, please include basic configuration information (processor type and speed, memory, OS, LR version and disk type - SSD/SATA HDD etc.). I'm looking forward to seeing some results. If some sort of pattern emerges (hopefully it will!), I'll put together a plot and post it up here as well.

    For a first data point I have:

    82 seconds - HP Z800 - 2 x 3.46GHZ Xeon 5677 - 16GB RAM - 480GB SSD - Windows 7 - LR 5.2
    109 seconds - iMac (2010) - 2.8GHZ Core i5 860 - 8GB RAM - 240GB SSD - MacOS X 10.9 - LR 5.3
    167 seconds - MacBook Air (2011) - 1.7GHZ Core i5 2557M - 4GB RAM - 120GB SSD - MacOS X 10.9 - LR 5.3
     
  20. dhazeghi

    dhazeghi Mu-43 Hall of Famer

    Aug 6, 2010
    San Jose, CA
    Dara
    Which I have done, repeatedly. Unfortunately, Lightroom results are scarce and none that I've found cover a decent range of different CPUs. Photoshop results are plentiful, but bear only the roughest correlation to the Lightroom results since Photoshop and it's filters are designed and written quite differently.

    As to the difficulties of a controlled test, I am well aware that it is almost impossible to account for all variables. That said, in my experience, Lightroom importing and development are highly CPU bound, with other variables not having a very significant role. The impact of disk speed is quite minimal (I did a test between a laptop HDD and a RAM disk and saw only a negligible performance difference). Lightroom makes no use whatsoever of the GPU (unlike Photoshop). Lightroom does not use extensive memory-caching (unlike Photoshop again), so beyond not hitting swap/VM, RAM doesn't really play much of a role in performance. Lightroom is not even particularly well-threaded, so there's usually plenty of idle cycles for the OS and background processes to run on even when LR is importing, assuming a machine with 2+ cores.

    I agree with all of this. Batch operations can be done in the background and ignored (in most cases). Interactive operations are where performance or lack thereof is a stumbling block. However, it's very difficult to measure the speed of development operations, and very easy to measure the speed of the import process. Since it is largely the same software data path that's being exercise in both processes (modulo the catalog manipulation, which in a small catalog tends to be very quick), I'm trying to use import performance as a proxy to gauge development performance.

    I guess to simplify, if there are machines that imports 50-100% faster than my current one, that suggests to me that it's worth upgrading, as development operations will be noticeably faster.