1. Reminder: Please use our affiliate links for holiday shopping!

How do you organise your digital library?

Discussion in 'Image Processing' started by k4t, Apr 18, 2011.

  1. k4t

    k4t Mu-43 Regular

    44
    Apr 15, 2011
    Melbourne, Australia
    While I twiddle my thumbs waiting for my first :43: to arrive I've decided it's time to try downloading some demo post-processing software and take it for a spin on my existing photos. In case I decide to keep one of these programs in the long run (likely Aperture I'd guess) I want to get off to a nice organised start when I import my images.

    Searching for advice on how to organise photo libraries on other sites has opened up a huge can of worms eg:
    Chronological vs name/event vs project based
    The old concept of 'film rolls'
    Whether to use a hierarchy of stacked folders
    To spend time assigning keywords and use smart folders or not
    To use or manually assign GPS data or not
    Professional vs amateur needs

    I thought I'd canvas for opinions on how people do it here?
    What's your approach to organising your digital library?

    One particular question of mine is do you delete your real failures or hide them away somehow? There are bound to be lots of shockers on my learning curve which I don't want cluttering my working screen but I'm reluctant to permanently discard much with storage as cheap as it is these days. Sometimes that little crop of a single person from an otherwise terrible photo is just what you're looking for months later.
     
    • Like Like x 1
  2. flash

    flash Mu-43 Hall of Famer

    Apr 29, 2010
    1 hour from Sydney Australia.
    Gordon
    I'm not the typical case. I take several thousand images each week and I get paid, so data security is high on my agenda. I often have to find images. I think your organization needs to work for you and so any one else's workflow should be taken as general advice only. My most important tips are.
    1. ALWAYS make a second copy of you images that is off line.
    2. Store your actual images on external drives, not on your OS drives. They take a lot of space and when hard drives fill up, they slow down.
    3. You're on a Mac. DON'T use iPhoto to copy your files from your camera. It uses a proprietary system of storage. You can't easily get to your images in emergencies.
    4. I love library style systems (Lightroom, Aperture) I use keywords and collections extensively. It's not time consuming except for the initial setup. And it's almost essential for larger library collections.
    5. I use a regular project style naming system for folders with a date prefix. I also have a master folder by year. Keywords negate the need for anything more complex.
    6. I keep most files except for the total failures. More so with personal images. It's not as important that these are perfect to me, so I keep them.

    Hope this helps

    Gordon
     
    • Like Like x 3
  3. Iconindustries

    Iconindustries Mu-43 Hall of Famer

    I can tell you what I do. I guess there's no proper way but you need something you're comfortable with.

    I use Aperture to do the big crunching. Which I run off my external HDD Everything from adding my Raw preset, adding adjustments right through to borders get done there.

    To show off my photo's to family or friends I use the massively polished iphoto. It is fast and made especially for quick and neat image display.

    I adjust the images in Aperture to how I like them and Export them as jpeg. The jpeg is imported into iphoto. I have Aperture running off my 500GB HDD so I don't really care about the mess (duplicates, original files ect) because when I export the images into iphoto onto my computer it is just what I want without extra baggage.

    In other words, Aperture on my external HDD is messy as, but iphoto on my internal HDD is neat as a snow flake. I name each event and that's all.
    Each picture I import into iphoto is printed and put in a book and that's when I write a small description to each photo.

    As I put them into a book and write a description there I don't worry about doing it on my computer.

    That's my workflow. Seems to work ok for now. haha
     
    • Like Like x 1
  4. k4t

    k4t Mu-43 Regular

    44
    Apr 15, 2011
    Melbourne, Australia
    Thanks Gordon. They sound like excellent organisational principles to live by, especially the ones about backups.

    I've found out first hand what you mean about iPhoto. I remember one of the earlier versions having a much more transparent file system, but I nearly had a heart attack the first time I tried to extract original files from my more recent version. It took me at least 20min of fiddling to find them - not intuitive at all.
     
  5. Iconindustries

    Iconindustries Mu-43 Hall of Famer

    By The Way Welcome k4t to this forum! We're all a bunch of good friends here so make yourself at home.

    icon, from over Queensland way.
     
    • Like Like x 1
  6. GingerCat

    GingerCat Mu-43 Regular

    26
    Apr 16, 2011
    I use dated folders ('yyyymmdd occasion') on two external drives, one at home and one in work (I'm not a pro, I just like to have an offsite backup of all my stuff. I've heard a couple of horror stories about house fires lately and my photos mean more to me than most stuff in my house). I use Picasa to look at them and Photoshop to edit.

    It's basic but it works for me - I like simplicity.

    I'm also waiting for my new kit to arrive. I might take a leaf out of your book and reorganise a bit beforehand. I could probably do with deleting a few thousand really rubbish shots!
     
    • Like Like x 1
  7. Narnian

    Narnian Nobody in particular ...

    Aug 6, 2010
    Midlothian, VA
    Richard Elliott
    Gordon has an excellent point - keywords are the key to organizing photos.

    Any tree folder structure always has limitations. I do create some top level folders (e.g. Family, Travel, etc.) and sub-level folders but too many pictures fit into multiple categories so keywords are the way to add that "3rd dimension" and cross reference them.

    It also means you do not have to worry about renaming files.

    It does require discipline but it pays off over time.
     
    • Like Like x 2
  8. mushmorok

    mushmorok Mu-43 Regular

    27
    Jan 5, 2011
    I am a complete amateur when it comes to photography and all images that I take a just for personal use. Still I have a separate file server running freenas for storing this data. This is how it is organized:
    1. RAID1 disk mirror (2 identical disks that are mirrored) - this protects the data from physical disk coruption
    2. Third separate disk on which the data from the mirrored disks is copied every day using rsync, syncing only the new files - that way if someone deletes anything by mistake it won't get synced onto the third disk and I'll be able to recover. This protects against user i.e. wife errors (she had done this twice already - practically deleting everything) :2thumbs:
    3. The folder structure is based on dates with some additional description for specific events and years as master folders (like "Photos/2010/2010-10 BD Peter" which would be for my sons birthday in October)
    4. As library I'm trying to use Digikam, but I still haven't found the least error prone, future proof way of organizing my photos inside it.

    This of course won't prevent me from loosing everything in the event of fire ... I have never thought of it from that angle. Most probably I should think of adding a second rsync folder that is remote and located on the same computer/location.
     
    • Like Like x 1
  9. sherlock

    sherlock Mu-43 Regular

    83
    Mar 31, 2011
    Everything in Lightroom — all my Photoshop TIFFs included. I take photos, so I don't want to manage a filesystem by hand. Smart Collections, tags & metadata make it easy to find what I want, and everything is backed up to external storage.

    Why anyone creates folders by hand is beyond me.
     
    • Like Like x 1
  10. mushmorok

    mushmorok Mu-43 Regular

    27
    Jan 5, 2011
    What do you mean by "by hand"? Does Lightroom create the folder structure for you based on some tags/metadata?
     
  11. sherlock

    sherlock Mu-43 Regular

    83
    Mar 31, 2011
    Lightroom does it by date. I never actually browse the folders themselves though (ie. in Finder or Windows Explorer) because I browse them through Lightroom itself. I can then do searches based on any metadata and get immediate results — no need to dig through folders.

    My workflow is to import as DNG's, tag/collectivize the shots & then edit away. Any TIFF's I bring back from Photoshop automatically get stacked & tagged with the original DNG's. No need to move files or create folders, ever.
     
  12. k4t

    k4t Mu-43 Regular

    44
    Apr 15, 2011
    Melbourne, Australia
    Thanks for the warm welcome Iconindustries. I've already found this one of the most generous and helpful online communities around. And there seems to be a great mix of beginners and experienced professionals willing to share their knowledge.

    I like your idea of having one program like iPhoto with an uncluttered collection of finished works.

    I don't think you can have too many reminders about the importance of backups, even though people should know better by now. I'm relying on an apple time capsule for now but I think setting up a mirrored RAID drive might be my next step as my collection is bound to become large enough for a hard drive of its own if I start shooting RAW files. (In fact I used to shoot RAW images on my first compact camera - a Canon Powershot s45 - but I hardly did any post-processing so it was pointless at the time. They must be on DVDs in the attic somewhere waiting to be revisited.)

    Offsite backup has been on my to-do list for a while. I've wasted time looking at the cost of internet based hosting services. It never occurred to me to just backup to a portable hard drive occasionally then leave it at work. Great approach GingerCat. A much simpler and more cost-effective solution!

    Thanks for mentioning Digikam mushmorok. I hadn't come across it before although I'm fond of gimp. I'll definitely give it a try at some stage.
     
    • Like Like x 1
  13. mushmorok

    mushmorok Mu-43 Regular

    27
    Jan 5, 2011
    And where are these tags stored? What bugs me the most is that if these library organizing software someday becomes obsolete - i.e. is not developed, is not supported on my new shiny OS etc. ... and if these tags/metadata are somehow "embedded" or readble only in that particular software, suddenly my collection will loose all that flexibility that you've mentioned.

    Of course this question (becoming obsolete) stands for file formats and storage types like hard drives/CD/DVD/Blurays - but this is another story...
     
  14. Alanroseman

    Alanroseman Super Moderator Emeritus

    Dec 21, 2010
    New England
    What a great question! I'm looking forward to seeing the memberships methodology on this topic...

    Cheers, Alan
     
  15. GingerCat

    GingerCat Mu-43 Regular

    26
    Apr 16, 2011
    I've wondered about this too. My system of dated folders may be old fashioned and clunky but it's probably fairly future proof.

    Storage media will change but that's not really a worry for me, I'll just copy the whole lot onto two of whatever the next thing is. Sorted : )

    About the fire thing - do make it a priority if you haven't. I know it seems unlikely but in the last two years I've heard of both a relative and a friend-of-a-friend who had devastating housefires and lost every photo they'd ever taken. They weren't pros or even hobbyists, so it was just snapshots that were lost, but it was decades of memories. I can't imagine the sense of loss I'd feel if it happened to me.
     
  16. sprinke

    sprinke Mu-43 All-Pro

    Apr 5, 2011
    Pasadena, CA
    Debi
    I use Lightroom and catalog everything by date. I don't mind going through the folders to find what I'm looking for (I can usually remember when a particular event happened) so I don't do that much keywording. Every year I usually put together a family DVD/slideshow and the dated filing system helps me do that.

    I store only the "good" images on my computer's hard drive. But I keep a backup copy of EVERYTHING I've shot on a huge external drive, and the same is also backed up externally to the cloud.
     
  17. sherlock

    sherlock Mu-43 Regular

    83
    Mar 31, 2011
    The metadata is in XMPP sidecars, which is an open format. The best thing is that I get all the benefits now, and if the doomsayers are right, then I go back to a year-date folder hierarchy, which is what Lightroom creates under the hood anyway.
     
  18. walt_tbay

    walt_tbay Mu-43 Veteran

    322
    Aug 24, 2010
    Ontario, Canada
    Well, here's what I do:

    Adobe Lightroom 3 is my main tool for organizing my images.

    When I first import photos from my camera, I convert them from the camera's Raw to the DNG format. The files are renamed to the "Date - Filename" format and are stored in a file folder on my iMac named for the year (i.e., currently, everything is being copied to a sub-folder named after the date within a folder named "2011"). I generally find this to be the easiest way for me to track down specific images.

    I then copy the original Raw images to my iMac for a second time. Again, I rename the files using the "Date - Filename" format, but save the images to folders named after the specific camera I used to take the pictures. I save the original Raw files because I've heard that using the DNG conversion can cause you to lose certain file info. I have no clue whether this is a real concern, but I don't see a downside to doing it.

    I use Apple stuff and have a Time Capsule wireless router with 2Tb of storage. This gadget automatically backs up all my image files and Lightroom catalogues, so if my iMac ever crashes, I have a current version that can be restored. I've actually had to do this and it works great.

    As an extra backup, I also have a hard drive attached to my iMac. I use Intego Backup Manager Pro to copy and synchronize my files every Sunday night.

    Finally, every couple of months I back up everything to a portable hard drive and take it to work so I can have off-site storage.

    The above might be overkill, but it makes me feel quite comfortable that my images are safe. :smile:
     
    • Like Like x 1
  19. acercanto

    acercanto Mu-43 Regular

    107
    Apr 15, 2011
    SW VA, USA
    I'm pretty simple. I use YYYY-MM-DD--HH.MM.SS for my folders, and just leave the images named what they are off the camera.
    Being a P&S user up until a few days ago, I use Picasa and Gimp occasionally, and upload the presentable ones to Flickr, where I have them organized by event (Spring 2010 West Virginia Backpack, for example). I just recently got a DVD burner, and have backed up all my photos to DVD with the same folder format as my hard drive. No offsite storage yet, beyond Flickr.

    Acer