Input on image management

Discussion in 'Open Discussion' started by jon595, Jan 3, 2013.

  1. jon595

    jon595 Mu-43 Veteran

    May 2, 2011
    I'm currently debating what to do with my photo collection. As of this posting, my Lightroom 4 catalog has 14,711 photo's going back to 2007. The majority of these are RAW files that have been fully processed, tagged and rated.

    Side note, I switched to Lightroom back in Aug. 2011 from Bibble 5 Pro. I had Bibble write metadata to standard xmp files, which Lightroom has mostly read. Going back through earlier photo's however, it appears that some of the tags are gone or not tagged correctly. Since this is only on photo's that were originally tagged in Bibble, I'm guessing that's the issue. I've begun to go back and fix everything in Lightroom.

    This made me question why I'm keeping thousands of pictures in RAW format that for 98% of them, I will never go and reprocess. Being that my laptop is due for replacing and I'm seriously considering going back to a full time linux distro (MacBook Pro now, but I don't love Apple's new 'disposable' laptop policy and not being able to upgrade anything).

    Switching will require a new RAW editor, which I'm fine with, but it again points to my dependance on Lightroom. If I don't process those pictures as JPEG, then they will be unprocessed and potentially have more tagging issues if/when I do switch to something besides Lightroom.

    So after a long winded explanation, I guess my question to you is... Do you keep your RAW files after you process them? I'm seriously considering keeping only my favorite RAW images (a few hundred or so) so I can mess with them later, but converting all of the rest to high quality jpeg's and dumping those RAW's. Anyone have any thoughts on this? I'm thinking long term down the road, where I don't know what OS/software combination I'll be using, but want the pictures easily accessible and searchable.
  2. stingx

    stingx Mu-43 Veteran

    Aug 10, 2012
    Weird, NJ
    Real Name:
    Hugh Douche
    I just keep moving everything to more storage. Too much work to sort and toss. Up to 8TB NAS now :(
  3. LeoS

    LeoS Mu-43 Top Veteran

    Aug 6, 2012
    How big of a data are we talking about?
    How about archiving them on bluray discs or hdd (use hdd docks to make them easy to swap) and to shelf them offline? They won't take up any space\electricity, but it's not as final as deleting them permanently...
  4. jon595

    jon595 Mu-43 Veteran

    May 2, 2011
    Well, my RAW folder is almost 200GB right now. The data size isn't the issue as much as it is the long term solution of dealing with RAW files.

    I'm a software engineer, so by nature, I like to switch OS's, try new software and play around with things. This isn't a good thing when dealing with things like RAW images where each program makes proprietary changes that other programs can't read.

    My thoughts are to process all of my RAW's into high quality jpeg's (10MB in size or so each) and find a new way to organize the physically files. That combined with my religious like zeal in tagging should make those images usable in any program or OS.

    Eventually, I want to take an older machine and set it up as a server on my home network to store and serve the images, so I can get them from there and onto my other devices (phones, laptops, tablets) without much issue.
  5. LeoS

    LeoS Mu-43 Top Veteran

    Aug 6, 2012
    Ah ok... Whatabout DNG? Do they work as advertised?
    I'm new to raw photography, but have managed file servers in the past. I've messed with debian, ubuntu, openfiler, freenas and opensolaris (zfs rocks!), but in the end i just throw drives to synology units and let it handle my network storage. Slightly more expensive than barebone servers but worth the simplicity, guaranteed performance and ease of maintenance so i can waste my time on other useless matters :biggrin:
  6. elavon

    elavon Mu-43 All-Pro

    Sep 1, 2012
    Tel Aviv Israel
    Real Name:
    If you convert your RAW to JPG and delete the RAW file you loss future manipulation of the files.
    DNG is probably the way to go. It is on its way to become and ISO standard to replace TIFF. When this will happen we will see broader support for it.
    The big advantage of DNG is that metadata and XMP data is embedded in the file for easier transfer. The embedded data and the open standard blessed by ISO will give in my view, a very robust standard with great future.
  7. Ned

    Ned Mu-43 Legend

    Jul 18, 2010
    Alberta, Canada
    The only purpose of a RAW file is a working file for editing.

    Does that answer your question? ;)
  8. LeoS

    LeoS Mu-43 Top Veteran

    Aug 6, 2012
    So I think you should just process the raws to jpg as you want, and put off the unneeded raws to an offline hdd and shelf it somewhere...
  9. Promit

    Promit Mu-43 All-Pro

    Jun 6, 2011
    Baltimore, MD
    Real Name:
    Promit Roy
    A quick reminder: when LR 4 came out, a lot of people went back to very old RAW files from many years ago and reprocessed them with the modern ACR to get much better results than they previously had. That would not have been possible if they said "well it's processed, I'm done" and deleted the RAW.
  10. addieleman

    addieleman Mu-43 All-Pro

    Aug 5, 2010
    The Netherlands
    Real Name:
    I'm one of those people that reprocess old raw files. I have more than 20,000 raw files and not for a moment I will consider throwing them away. I too feel locked in by Lightroom because once a raw file is processed in LR there is no other program (I think) than LR in which you can see how your processed raw file looks like. So I regularly process all raw files to full-size JPEGs in AdobeRGB to have at least a program-independent processed version, but I don't see the advantage of throwing away raw files after that.

    I switched from Nikon CaptureNX to Adobe Camera Raw a few years ago (when I bought my Panny G1) and now I'd have to buy a new version of CaptureNX when I want to reprocess my raw files with the changes I made at the time. Fortunately I also have the full-size JPEGs, so if I want to reprocess I do that in LR, in fact redoing the work that I did in CaptureNX long time ago.
  11. dwig

    dwig Mu-43 Top Veteran

    Jun 26, 2010
    Key West FL
    • If I keep and image that was orginally a RAW, a TIFF, or a film image I keep that orignal image in its original form.
    • If the image is not worth the effort/space for keeping the RAW file, or original film image, then it's not worth keeping in any form.

    I do occasionally delete "alternate" images. Ones that were not the final chosen framing or such become waste after a period of time long enough for me to be secure in my choice of the proper shot.

    I generally keep the RAW in LR along with the final PSD versions (I frequently build specific PSDs sized for various print sizes) for any of my "art" images. Family snaps and web-only images are generally archives as final JPEGs along with the original RAW, TIFF (my wife's P&S saves JPEG or TIFF but not RAW), or "in camera" JPEG.
  12. jon595

    jon595 Mu-43 Veteran

    May 2, 2011
    Thanks for all the input. I guess I should have mentioned that a good 90% of the images are simply snapshot style things of family. A good portion portion of the pictures have no artistic value, but since they have family members in them, I'm keeping them for way down the road. I don't think I'd ever care about re-processing those 90% ever, and would rather have them as jpeg's so other programs/OS's can view them as they are.

    For those shots that I do consider artistic and worth keeping, I'll archive the RAW's somewhere. I guess my next step is to pick a good directory structure to put all of those jpeg's into and find a good image management program on linux.

    Once again, I'm doing this to essentially future proof my photo's. Since I'm looking to make the switch back to a linux OS, I figure now is a good time to get everything in order. The jpeg's will be good for this purpose since they will be processed and contain all of the tags I've added, so any program I choose can read them. So 10 years down the road, if I want to run Windows 18, I wont' have to re-process 30,000 or so images and I'll be assured my tagging is maintained in the files themselves. A new program can just scan all of the files and get the information it needs.

    So... anyone know a good linux distro besides Ubuntu (not a fan of what they are doing with the windowing system)?
  13. kinlau

    kinlau Mu-43 Top Veteran

    Feb 29, 2012
    200GB is relatively little with the size of today's drives, there's no harm in just backing it up on an external HD. And being lazy anyhow, I find it way easier to just backup everything rather than only doing it selectively.

    Again, back to being lazy, I just shoot family snaps in jpeg :)
  14. jon595

    jon595 Mu-43 Veteran

    May 2, 2011
    Thanks for all the suggestions. After thinking more about it after a few days, I can't give up Lightroom. It's just such a good processing and management tool.

    I still need a faster machine, so I just bought a 13" MacBook Pro (non-retina) with a 128GB SSD installed. Since I almost never burn CD's, I will remove the SuperDrive and replace it with a 750GB 7200 rpm HDD for data (CD drive will go into an external enclosure). To finish the machine off, I ordered a 16GB RAM kit to put in. So when it's done, I'll have..

    13" MacBook Pro
    - core i7 @ 2.9GHz
    - 16GB DDR3 RAM
    - 128GB SSD for the OS and apps
    - 750GB HDD for the data

    So I'll have the speed of an SSD, the mass of a HDD and Lightroom in a beautiful chassis. Now I just need to wait a week before all the stuff gets here and I rip apart a brand new laptop...