1. Reminder: Please use our affiliate links for holiday shopping!

Photo storage and workflow configuration question

Discussion in 'Image Processing' started by Speedliner, Feb 14, 2016.

Tags:
  1. Speedliner

    Speedliner Mu-43 All-Pro

    Mar 2, 2015
    Southern NJ, USA
    Rob
    thought I'd ask you experts about how you store your photos, including backups, and work areas for new images, etc...

    Currently I have all of my images stored on a raid 1 array on a file server running 24x7 in the basement. They are backed up each night to a local HD and also to an off-site service - CrashPlan - which is very inexpensive for the peace of mind that it provides. Total content is about 350gb so far.

    The majority of these images are jpegs from my pre-DSLR days. An advantage of having them on a server is that everyone can view them including our Tivos which enable us to view pictures and home videos on any TV in the house. That's nice. If I didn't have TiVo I'd have Apple TV or something similar to enable on-TV viewing because it so easy to view your recent pics in room-filling HD...someday soon in 4K. Don't want to give that central storage up. One challenge with this though is that you can't view RAW files, certainly not LR modified views on these devices. That's a drawback.

    That's the background, here comes the what to do next part...
    I found LR slow. No surprise right? So I upgraded my PC. More horsepower, memory and a SSD. Better, but still sluggish. Storing images on the file server is part of the speed problem. My catalog is local on the SSD of course, but I'm still moving large raw files over the net to a raid 1 array. Also, I'm not able to view recent photos outside of LR.

    So the workflow change that I'm starting is to load all new images on the local SSD. I'll work with them there. When I'm finished with the keepers I'll export them as jpegs to the file server where we store and view all of our images. I'll then move the raw files to a new directory structure on the same file server just for raw images. That way the family and the Tivos and devices can access good old jpegs where they have always been and can easily be found, while I still preserve the keeper raw files elsewhere. I'll backup the local files while I'm working on them just in case. Once all is moved to the server, they'll benefit from the dual local and off-site backups.

    Does that sound like a practical process for both archiving raw images, making viewable jpegs accessible and protecting the images? Does it sound like it would work for physical workflow?

    The other question is regarding hardware. With all of these backups I'm wondering if a raid array is really necessary. I'm also,wondering if it's time to move to all SSD. I don't need a lot of storage yet. 1tb would give me room to grow for a couple more years. I cull my pictures aggressively.

    What do you experienced folks think about physical storage? Raid still beneficial? SSD speed still worthwhile over a network? SSDs reliable enough to make raid unnecessary?

    Thanks.
     
  2. Drdul

    Drdul Mu-43 Regular

    104
    May 16, 2015
    Vancouver, BC
    Richard
    I use LR, too. My images are stored on an external RAID drive, which is backed up in multiple on-site and off-site locations (+1 for Crashplan). For my purposes, RAID is desirable for my primary storage, as the backups don't necessarily happen immediately, which means a non-RAID drive failure would risk losing some amount of images/changes. Given the price and capacity of hard drives these days, the extra cost of a RAID setup is minimal. BTW, my setup is a software RAID array in a 2-drive WD Thunderbolt enclosure, which is plenty fast enough that I have not considered SSDs.

    No matter where my MacBook Pro is, all my images are immediately accessible because I have created Smart Previews of each image, and the Smart Previews are stored on the MBP. I can still edit/adjust images if I want, and even export them from the Smart Previews.

    Images I want to be available for showing to others or whatever are synced from LR to Lightroom Mobile, which means they can be viewed on my iPad or iPhone (which I can always mirror to my TV via AirPlay, but never have), as well as via the web.

    My 2012 MBP only has 512 GB of storage, so I have to store my images offline. In a few years when I upgrade to a new MacBook Pro, I'll likely be able to get one with a few terabytes of storage, and may move all my images back onto the computer. But for now, external storage with local Smart Previes works a treat.
     
    • Like Like x 1
  3. Clint

    Clint Mu-43 All-Pro

    Apr 22, 2013
    San Diego area, CA
    Clint
    Programs on a SSD, photos get loaded via Lightroom to a docked 3TB hard disk and copied to another docked 3TB hard disk. Both disk have the same file structure. I cull and edit from my primary docked disk and a sync program syncs the files between the two HD. Depending on my workload I swap out the secondary disk (weekly, monthly, or at least at the end of each quarter) with another backup that I keep offsite and then the older disk is synced to bring it up to date.

    BTW, The Lightroom catalog and previews are on the docked disk. I have no speed issues and still process D800 files routinely.

    Final JPEG files are output to my internal HD for use, anything unique to customer needs, along with full sized TIFF files of my best photos . If I want to share these with family then they are copied to a media server. The file structure for these is also the same as the primary docked disk. The photo files are then backed up to BlueRay disk once and a documents backup HD that is rotated.
     
    • Like Like x 1
  4. MarkRyan

    MarkRyan Instagram: @MRSallee

    772
    May 3, 2013
    California
    Like in other things, I'm a minimalist when it comes to storage/workflow.

    I use Lightroom and Photoshop, depending on the photo or my mood. I don't bother backing up catalogs or PSDs. Lightroom catalogs are too clunky to bother (especially since a large catalog starts impacting the program's performance during editing), and PSD files are huge. I don't make any edits that can't be redone in a few minutes anyway. I don't like clutter, and that's exactly what catalogs/PSDs feel like.

    Hanging onto raw files and JPEG outputs is enough for me. I have a MacBook with 256 GB SSD, so enough room to keep the JPEGs handy, but not enough for backing up raw files.

    When I import raw files, I import to an SSD folder shared with Google Drive ("Photos bridge"). Same thing with my JPEG exports -- just export to a folder shared with Drive. Google Drive automatically syncs with cloud storage. Periodically, once every few months, I'll jump into the cloud interface to pull the raw files onto an archive folder on Drive ("Photos archive") which is not shared with my SSD. The process is immediate on Google Drive, and then my MacBook is relieved of the storage. JPEGs also get uploaded to Flickr, but mostly for sharing -- I don't think of Flickr as backup storage.

    I have just a single point of failure -- Google Drive -- for my backups, but in my experience I'm infinitely more likely to kill a local hard drive than lose a file to the cloud. Google's got plenty of backups, and for $10 a month I'm happy to let them handle the hassle. I've kept local backups before, but it's tedious, slow, and the technology gets outdated.
     
    • Like Like x 1
  5. mcasan

    mcasan Mu-43 All-Pro

    Feb 26, 2014
    Atlanta
    Import from SD cards to the 2016 folder on the Macbook SSD. Cull and do keywords plus basic edits via Lr Presets. Keepers are then moved via Lr to an external library drive (a RAID set) into folders by year and date (my default in Lr import). Separate Time Machine drives backup the entire file system. If my photos drove my income, they would also be backed up online and/or a copy wold go into a bank box at least once a week.

    If I have a huge amount of files on the SD card to cull, I will use the Browser Module in OnOne Photo 10 to quickly cull. Only the keepers are imported into Lr. IMHO, Lr is too slow to import a huge set of files and then cull inside Lr.
     
  6. barry13

    barry13 Super Moderator; Photon Wrangler

    Mar 7, 2014
    Southern California
    Barry
    Hi, another user here did some tests and found that an SSD for photo storage barely made a difference in speeding up opening images in Lightroom.

    And RAID-1 should be more reliable than an SSD.

    Make sure your RAID software or hardware does SMART tests and scrubs and will notify you if a disk starts going bad.

    Barry