Large and growing library, need advice

Discussion in 'Hardware' started by Neftun, Nov 26, 2013.

  1. Neftun

    Neftun Mu-43 Veteran

    408
    Jul 15, 2012
    Norway
    Patrick Kristiansen
    Hi all!

    I have a potential problem. I currently have around 17000 files in my aperture library, and it will grow larger. I do weed out excessive files, but I will also take more photos.

    My current setup:

    2010 model 27" iMac with 1tb hdd
    Timecapsule 1tb running periodic backup.

    My worry is that I'm going to run out if hdd-space, and I gather the imacs are not very upgradeable in this department. I am thinking of an external hdd, but how does that work with an image library? And of course, timemachine? Is a network solution better? Is making an archive of sorts any good idea, if so, is it smarter to archive jpegs, or raw?

    Any advice highly appreciated.


    Patrick K
     
  2. RoadTraveler

    RoadTraveler Mu-43 All-Pro

    Nov 23, 2012
    I don't have a solution for you, however I have a pretty large image database myself, which grows constantly, and I was slow to accept digital over film (less than 10 yrs shooting digital). I could/should go back and delete hundreds of image files I don't need or want, but that takes some time. Most prolific shooters will someday face a storage problem.

    Related, I've started to shoot more RAW images lately, though when I can I still prefer to shoot mostly jpeg to save on the size of files as the years tick past.
     
  3. yakky

    yakky Mu-43 Top Veteran

    662
    Jul 1, 2013
    17k isn't really that bad. I have just under 30, and I don't really consider it a "problem" yet. :biggrin: Some thoughts:

    -Offsite backup - so many people back stuff up, but what happens if your house is destroyed or computer is stolen?
    -Aggressively clean right after a photo shoot. If you are on the fence, toss it. Don't e-hoard
    -Additional disk space is cheap, and external drive will do fine for your immediate problem
     
    • Like Like x 2
  4. Narnian

    Narnian Nobody in particular ...

    Aug 6, 2010
    Midlothian, VA
    Richard Elliott
    I try to do more culling upfront when I shoot and transfer them to Lightroom. I also tag them using the "5 star" rating system so as I need room I will be able to go back and quickly identify the "junk".
     
  5. Pecos

    Pecos Mu-43 Top Veteran

    776
    Jan 20, 2013
    The Natural State
    Space is eventually a problem for most shooters, no matter how much storage you start with. I shot all day in RAW+JPEG once recently and was surprised how much space the raw files took up. And considering that I didn't feel the majority weren't worth keeping, and the vast majority would never be printed, I stopped shooting RAW for all but the most demanding or important shots.
    If I had a terrabyte of photos, nobody - and I mean nobody will ever go through them all; I won't, and I'm sure my heirs won't.
    So I shoot in JPEG, delete the obvious bad shots after shooting, and back everything up to a Synology Disk Station (NAS), which in turn backs up to another one off-site.
     
    • Like Like x 2
  6. flash

    flash Mu-43 Hall of Famer

    Apr 29, 2010
    1 hour from Sydney Australia.
    Gordon
    I use Lightroom but Aperture is essentially able to do the same thing. My current image holding is about 1/4 million images spread over three catalogues.

    All images can be stored on external drives and referenced by the catalogue, just fine. Obviously you won't be able to work on images if the drive isn't connected (I archive work images after 6 months) in Aperture (you can with Lightroom) but a disconnected drive won't mess up your catalogue. A decent USB3 drive should be fine for most purposes but can be a bit slower for large collections.

    So all you need to do is buy a decent drive and dedicate it to images. You should also be able to copy off your current images to the new drive. You *may* even be able to keep the catalogue on that drive as well. I think that's an option with the later versions of Aperture.

    Gordon

    p.s. Buy two identical drives so you can have a proper non connected backup of your files.
     
    • Like Like x 1
  7. kevinparis

    kevinparis Cantankerous Scotsman

    Feb 12, 2010
    Gent, Belgium
    One thing to look at is to store your images as referenced files as opposed to the default for aperture which is to store them in the Aperture library. You can then keep the image files on a separate hard disk or hard disks.

    The is a command to Relocate Masters that will do the work for you

    K
     
    • Like Like x 1
  8. Cruzan80

    Cruzan80 Mu-43 All-Pro

    Aug 23, 2012
    Denver, Co
    Sean Rastsmith
    Best thing I figured out about culling is in LR (and I am sure Aperture has the same kind of command), you can reject pics with one keypress, then batch delete all the rejects, instead of having to continually confirm deletion, or accidentally delete from LR library but not HD.
     
  9. Neftun

    Neftun Mu-43 Veteran

    408
    Jul 15, 2012
    Norway
    Patrick Kristiansen
    Lots of good advice here, thanks! Kevin, gordon, the functions you mention I was completely unaware of. I guess they will solve my problem:)


    Patrick K
     
  10. alex66

    alex66 Mu-43 Top Veteran

    715
    Jul 23, 2010
    On a more practical level if you get a Firewire drive 800 speed they work a lot faster than a USB Aperture will manage a library on an out board drive as well. It would get a same sized usb to do backups and remove to another location as well, a backup that is permanantly connected to the machine only covers disk failure.
     
    • Like Like x 1
  11. LeeOsenton

    LeeOsenton Mu-43 Button Clicker

    297
    Jun 25, 2010
    Hayes, Virginia, U.S.A.
    Lee Osenton
    +1 took me forever to figure this out. If it isn't good or interesting enough to post here or it isn't a good family snapshot, then it is toast.

    Lee


    Sent from my iPhone using Mu-43
     
  12. dav1dz

    dav1dz Mu-43 Top Veteran

    926
    Nov 6, 2012
    Canada
    I do this, except my time frame is 1 year. I keep my Lightroom catalog file on my system drive though.

    I have seen some tutorials that shows how to exports Aperture libraries. If you let Aperture take care of your library, this is a great way to move a single file onto an external drive. Since all the catalog metadata, along with RAW files are stored within one pseudo-folder.

    If those externals are formatted as Mac OS X Journaled (HFS+ Journaled) then your Time Capsule, or any external drive set as a Time Machine will pick it up during backup when they're connected.
     
    • Like Like x 1
  13. stargate

    stargate Mu-43 Regular

    131
    Aug 14, 2013
    Greece
    The answer is an external NAS. A 2 bay RAID system stores the files in two drives for backup in case of drive failure. Adding an off site backup for security will be enough safety. Then you can have network access when in the house and FTP access to your files from the internet and your computer will be free from lots of files and snappy.
     
    • Like Like x 1
  14. flash

    flash Mu-43 Hall of Famer

    Apr 29, 2010
    1 hour from Sydney Australia.
    Gordon
    Newer Macs support USB3 which is faster than firewire 800. Plus you buy the cheaper "PC" version and reformat it for Mac instead of the "Mac" version of the same thing.

    gordon
     
  15. yakky

    yakky Mu-43 Top Veteran

    662
    Jul 1, 2013
    Raid arrays make sense for speed these days. For safety a backup solution (offsite preferable) is much better. Drives that stay put (non laptop/portable) have pretty good reliability these days. Most OSs can monitor drive performance and alert you when drive issues start cropping up. The biggest threat to files these days is the user and malware. An on site array does nothing to prevent that.
     
  16. SpecFoto

    SpecFoto Mu-43 Veteran

    315
    Aug 28, 2012
    So Cal
    Jim
    Your IMac has firewire 800 and USB 2. Any external drives should take advantage of the FW800 port, it is twice at fast as USB. Here is the dual raid drive that I use, my 2 Aperture libraries have over 45,000 files. Suggest at minimum you get a 4TB model, it is not that much at $325 complete with software and cables.

    http://eshop.macsales.com/shop/firewire/1394/USB/EliteAL/PerformanceRAID

    If you decide to later upgrade your iMac, there is a $30 adapter for thunderbolt to FW that will keep this drive current, and it has a e-sata port too. With this case is it is pretty easy to upgrade the hard drives yourself, if you are so inclined.
     
    • Like Like x 1
  17. Gary Ramey

    Gary Ramey Mu-43 Veteran

    240
    Dec 27, 2012
    Aurora Colorado
    ...and don't forget to UPS your backup drives. I lost 2 at once, along with the internal drive due to a power hit. It was nearly catastophic. While I'm sure I lost about 2 years worth of really old family photos...1999-2001... I had the rest burned to DVD. So if you're going to leave them connected, protect them!
     
    • Like Like x 1