DNG v RAW and a few LR3 questions

Discussion in 'Image Processing' started by BillN, Oct 8, 2010.

  1. BillN

    BillN Mu-43 All-Pro Charter Member

    Jan 19, 2010
    SW France
    I am still struggling with how to sync my (lightroom) image files between my iMac and MacBookPro

    I have bought the ChronSync software

    A few questions:

    1). DNG or RAW - what's the latest opinion - should I convert all to DNG - does that mean that there will no longer be any "sidecar" files meaning that all the (valuable) metadata will now be stored in the file.

    2). Which, (names and extensions), are the Lightroom files which store all the adjustments made in LR - if I identify all the LR files that are generated in the process(es) that happen in LR - can I sync these files between two computers or is there something stopping this happening, (within the files)

    3). A MAC Question - ChronoSync will not access some of the files on my MacBookPro - it is basically telling me that "I do not have (all the) permissions to access" - I have marked both Hard Disks, (iMac and MBP), as "sharing" and specified the names of the users that can share these files - but still no joy - is there anything else that I need to do to allow sharing between two Macs?

    4). Can anyone identify ALL the files that I need to locate to copy a "Lightroom" "adjusted" image from one LR application to another LR applications

    Thanks


    (PS - I did start another thread with similar LR (backup) question - but it never got into SPECIFIC file detail)
     
  2. antithetic

    antithetic Mu-43 Regular

    150
    Jun 7, 2010
    City of Angels
    I can try to answer a few of your questions:

    1) I shoot RAW with my E-PL1 but keep it in the Olympus RAW format. Since I also use LR3, I let the application manage the RAW + metadata. DNG probably would make migrating from LR3 to Aperture or another photo processing application.

    2) LR3 has a "catalog" file and "preview cache". The catalog file has the extension of .lrcat. This file is what you would need to sync across machines with ChronSync.

    The LR catalog file is most likely your User directory under Pictures.
     
    • Like Like x 4
  3. tomrock

    tomrock Mu-43 Regular

    132
    Jun 21, 2010
    Indianapolis, IN
    What you're trying to do isn't trivial. Here's what I do.

    I have a bus-powered 2.5" firewire drive where I store all my pictures and my LR files. That way, whatever Mac I plug the drive into, everything is there. I would never try to sync raw files and LR catalogs (and previews) between two Macs. Because the drive is bus-powered, I don't need a power cable. Just the one firewire cable.

    If you switch to DNG rather than raw files, all the changes you make to a file are stored inside the DNG. Therefore, if you just change the brightness a little on a file, the DNG is changed and whatever you use to sync (or backup for that matter) sees a changed file and has to copy it. DNG are usually bigger than raw files.

    Using native raw files, if you change the brightness a little, the change is stored in the LR database and only the LR database needs to be backed up.

    Good luck with whatever you choose to do.
     
    • Like Like x 4
  4. BillN

    BillN Mu-43 All-Pro Charter Member

    Jan 19, 2010
    SW France
    OK

    progress, (if you can call it that), to date

    1). It is reasonably straightforward to sync images, i.e. you just sync the folders on each computer, (drive), which contains the images, (Folder = Pictures on both my Macs).

    2). AFAIK, and, as has been indicated, and is the default setting in LR3 - the Lightroom catalogue is in the Lightroom folder, file = Lightroom 3 Catalog.Ircat, plus the LR Backup folder and preview file
    I would think that if I try to sync the .Ircat files between the two Macs - all I will get is the latest file and I will loose all the Lightroom adjustment settings on the set of images on one of the computers, (if you see what I mean).

    The reason I want to sync my two computers is that most of my images are on my Imac, (desktop), when I travel around I take my MacBookPro, download images and work on them in LR3 - clearly the only way to get these images, (the ones on my MBP), onto my IMac desktop is to copy the "raw" images from the MBP to the Imac and then make the same adjustments (again) on the Imac - effectively duplicating all the work I have done on my (MacBookPro)

    Question: Will DNG files help to solve this, i.e. will the LR adjustments etc., be recorded in the DNG file(s) or will some/all still be contained in the LR catalogue - I would image that anything done in LR is recorded in the Lightroom .Ircat file and not a .dng file

    Hope this helps and if anyone has any comments I would be very appreciative to receive them

    Cheers
     
  5. Pelao

    Pelao Mu-43 Top Veteran

    959
    Feb 3, 2010
    Ontario, Canada
    • Like Like x 2
  6. BillN

    BillN Mu-43 All-Pro Charter Member

    Jan 19, 2010
    SW France
    Thanks - but it is not really a "true" sync - it is achieved by manually moving, (copying from one computer to another), then importing and exporting Lightroom catalogues and images.

    What I am after is to set up an automatic sync, (time delayed), that runs in the background to take in LR adjustments etc., as I make them on either computer - i.e. by using Chronosync or something similar

    If that is the best there is - that's fine but it is time consuming and as I said it has to be done as a manual exercise rather than setting the sync up once and just "clicking" on it every hour or so - or setting it up to run at specified time periods - That's my reading of it anyway

    A couple of other points to consider that I have read about - if you are using "time machine" to back up your Mac, apparently you should allow the LR backups to run, (using Time Machine), when the LR application is not running

    Also if you convert a RAW image to DNG in Lightroom after LR adjustments etc., it appears to loose all those adjustments and the "history" starts with the "import" - maybe I'm wrong on this one as i need to check it out a little more.

    Cheers
     
  7. khollister

    khollister Mu-43 Veteran Subscribing Member

    259
    Sep 16, 2010
    Orlando, FL
    Real Name:
    Keith
    I convert my ORF raw files to DNG because of 2 things:
    1- no sidecar files
    2- the edits are reflected in the preview jpg in the DNG

    When I run the Adobe DNG converter the edits in the sidecar file are pulled into the DNG
     
  8. pictor

    pictor Mu-43 Top Veteran

    635
    Jul 17, 2010
    This would be fine, but I cannot see that this is the case. I convert the raw files while importing them and I am quite sure that the changes I apply in Lightroom afterwards cannot be seen in the preview jpg. Maybe this would be the case, if I converted the files to dng after developing the raws in Lightroom.

    Or is there some trick I don't know?
     
  9. Streetshooter

    Streetshooter Administrator Emeritus Subscribing Member Charter Member

    Dec 15, 2009
    Phila, Pa USA
    Pictor,
    LR works in a non destructive way. You don't see the changes because they aren't really in the DNG file. Your DNG file always stays as original.
     
    • Like Like x 4
  10. khollister

    khollister Mu-43 Veteran Subscribing Member

    259
    Sep 16, 2010
    Orlando, FL
    Real Name:
    Keith
    Tha trick might be that I'm not using LR. DNG files have embedded jpg previews that normally reflect all of the edits. Camera Raw in Photoshop allows you to set a preference as to the size of the preview - I set mine to full size. The browser that I use (Photo Mechanic) allows you to either display the thumbnails using the embedded jpg previews or to be created from decoding the raw file with the OS raw decoder (which will not show the ACR edits).

    And the edit steps/instructions are in the DNG as opposed to sidecar files. LR may not have an option to display thumbnails based on the embedded jpg, which would be the cause of your observation.
     
    • Like Like x 2
  11. PeteMarshall

    PeteMarshall Mu-43 Regular

    34
    Aug 30, 2010
    The DNG file stores the metadata from LR in the header of the file as xmp data rather than sidecar files. LR does this only if you actually tell it too. No changes are made to the file. Storing a copy of the metadata either in the header or as a sidecar does not replace the storing of the data in the LR catalog, it supplements it. Some data (such as virtual copies) cannot be stored in the file or as xmp and is only stored in the catalog.

    Storing the data as xmp either as sidecars or in the headers allows other applications such as Bridge and separate LR catalogs to read and apply changes made in the LR catalog to the file.

    DNG compressions is more effective than most propitiatory compression so DNG files will be smaller than the propitiatory formats, The usual reason that you have a larger file will usually be because you have embedded the RAW file in it propitiatory format, which will mean have two copies of the same data.

    For archival purposes the DNG format is a much better bet than propitiatory formats as it is future proof in that it is published data.
     
    • Like Like x 2
  12. Jimboh

    Jimboh Mu-43 Regular

    75
    Sep 17, 2010
    Florida
    ALR vs Mac

    I dont have mac, so can't comment on that.
    -------------
    I had to work on a project on the road, so I exported part of the catalog with only the photos to be processed into it's own catalog and let ALR copy the files and previews to the laptop via the server. While in the hotel I made my edits, then merged the catalog w/ edits back once I got home. Seemed to work ok.
    I won't detail steps, I used Scott Kelby's book, its in there.
    --------------
    I am standardizing on DNG. It incorporates more metadata than RAW and PSD and Adobe guarantees it'll be around in 10-20 years. Can't say that about a RAW file.

    If your camera maker changes it's RAW format, (I think Nikon has) it'll be one less headache to deal with too.

    Caveats:
    --Be aware there's no (free) way to view DNG in Windows Explorer yet.
    --Also, some apps cannot read the DNG embedded metadata. They still require XMP sidecar. Be sure to test.

    SPeaking of embedded metadata, short video on importance.
    Getting Started Right: Adobe Photoshop Lightroom 3′s Critical Preference Menus

    Finally, I originally bought ALR because I got the student discount. I was reasonably happy with Photoshop Elements (PSE). I have to tell you ALR is easily worth the full price, in fact, I'd suggest buying ALR first, then PS or PSE if you find you can't do things in ALR. ALR does sharpening a whole lot better than anything else I've seen, it allows for fast cataloging, spot removal and other changes en-mass. Once you learn the templates you can really speed your work up too. Long list of things to really like. I was really fustrated with the app until I bought a good book to go with it, so be sure to budget for that too.
     
  13. BBW

    BBW Super Moderator Emeritus Charter Member

    Another handy Lightroom thread.:bump:
     
  14. flash

    flash Mu-43 Hall of Famer

    Apr 29, 2010
    1 hour from Sydney Australia.
    Real Name:
    Gordon
    Ummm...

    If all you are trying to do is to move images one way ( from the laptop to the desktop) it's super easy to keep everything. Just export the desired images as a catalogue and then import that catalogue on the main machine. This has the advantage that ALL your data is saved and transferred. XMP files do not store history and virtual copy settings, for example.

    If you are trying to use LR back and forwards on two machines, the only practical solution is to use an external drive (this is what I do with my catalogue). You will need to have the catalogue, previews and presets on the external drive, with your working images. Your archived images can be elsewhere.

    Gordon
     
  15. DHart

    DHart Mu-43 Hall of Famer Charter Member

    Jan 7, 2010
    Scottsdale, Arizona
    Real Name:
    Don
    Bill.. here's the method I use.

    I have four Macs that are used for image processing (MacPro & iMac 27" i7), client presentation (MacMini w/61" 1080p hi-def screen), and 15" MacBook Pro i7 for working wherever I wish to work.

    Right after a shoot I go to the MacPro (or MBP if I'm away from the studio) and on it's hard drive I create a folder with name of the client or event. In that folder I put the RAWs from the shoot. I also create a Lightroom catalog just for the images of the client or event that are in that folder.

    Then, I copy the folder I just made from that shoot (RAWS and the catalog file) onto my "mobile" hard drive which is a 500GB 7200RPM laptop drive housed in a Mercury On The Go, bus-powered FW800 enclosure (Otherworldcomputing.com).

    Now there are two complete copies of the original RAWS and the LR catalog: one on the MacPro and one on the mobile HDD.

    When I want to prepare and edit the images for client presentation, using the iMac or my MacBook Pro, I simply take the mobile drive to that machine, copy the folder with the files for the client I want to work on from the mobile drive onto the hard drive of the that Mac - using FW800. Then I work on the images in Lightroom on that Mac.

    When I'm done working on the images, all I have to do is copy the LR catalog file from the Mac I'm using back to the mobile drive. As you can see, this way the mobile drive is always up to date - as is the last Mac I used.

    The RAWs haven't changed a bit, just the LR catalog file, so all I need to update back to the mobile drive is the LR catalog file.

    This way, the mobile drive and the last Mac I used each have the latest versions of the catalog and a complete set of the RAW image files.

    Before the client comes to view their images, I take the mobile drive to the presentation room and copy the client folder with the RAWs and edited LR catalog onto the MacMini hard drive (actually an external drive that lives with the MacMini).

    Then the client presentation happens, with more changes made to the images in the LR catalog during the order appointment. At the end of the order appointment, the changed LR catalog is copied back to the mobile drive.

    Sometime later, comes time to do final edits to the ordered images. The mobile hard drive is taken to the iMac or MacPro or MacBook Pro and the LR catalog (and RAWs if they don't already reside on the machine to be used for editing) is copied onto the machine for final image editing and preparation for print or FTP upload, and sending completed images to the lab via the internet. At the end of the final editing session, the catalog is copied back onto the mobile drive.

    At this point there are numerous discrete hard drives each containing the RAWs and the LR catalog. I don't worry much anymore about losing client images or the catalog that contains all the edits and enhancements that were made before the client saw their pics and after they completed their order appointment. All hard drives can and do go down unexpectedly, sooner or later. This way, at least THREE separate drives would have to go down, at one time, before I lost ANY client images. That's fairly unlikely.

    And by keeping a separate LR catalog for each client and event, my LR catalogs are small and very snappy. This way, generally I don't have any more than 200-300 images in a catalog.

    Hope this helps in some way.
     
    • Like Like x 2
  16. Streetshooter

    Streetshooter Administrator Emeritus Subscribing Member Charter Member

    Dec 15, 2009
    Phila, Pa USA
    Don,
    Thanks for the very detailed response.
    Perhaps you could explain a little more about backing up the catalog.

    Where does one find the current catalog on a Mac?

    How do you copy it to another drive. Do you over write the existing catalog on the hard drive you are copying to?

    Thanks...don
     
    • Like Like x 2
  17. DHart

    DHart Mu-43 Hall of Famer Charter Member

    Jan 7, 2010
    Scottsdale, Arizona
    Real Name:
    Don
    Don... When you open Lightroom it presents a dialog box asking if you want to open an existing catalog (and it lists the previous catalogs that have been opened) or you can choose to open a new catalog. This is where I start a new catalog for each client or event. You can always click on a pre existing catalog as listed as well. A search of your hard drive will tell you where those pre existing catalogs reside. For a new client, I start by creating a folder with that client's last name, first name. Inside that folder I will have another folder containing the RAW files from the shoot. And I'll create a second folder in that client's folder to hold the two Lightroom Catalog files that are created when you create a new catalog.

    I always keep a complete set of the RAW files together with the LR catalog in a folder with the clients name whenever I move the catalog. But once the RAWs exist on another drive, you only have to copy the catalog back and forth when changes are made to the catalog, you don't need to copy the RAWs back and forth, because they are not changed. You copy the client/event folder (with RAWs and catalog inside) to another hard drive by simply ckicking and draging to the other hard drive. Yes, I write over previous catalogs so that the newest catalog is what is present. If you use two computers, like an iMac and a MacBookPro, you will have three complete copies of your original images and the latest catalog version for them, one on each machine and one on the mobile drive. Excellent redundancy to safeguard your images. And it's smart to dump another set of this onto a 1TB or 2TB external drive to be kept OFF premises (although refreshed to be up to date from time to time) to safeguard against fire, theft, etc.
     
  18. soundimageplus

    soundimageplus Mu-43 Top Veteran

    782
    Feb 2, 2010
    Worcestershire
    Slightly off-topic.

    One of the reasons I don't use Lightroom is because of this need to "import" files. Though it is possible to track down the files, I like to have them where I want them and have the ability to access them easily when I want. The Lightroom system seems over complicated to me and I use Photoshop and File Browser for all image sorting and editing. That way I can organise in a way thats convenient and makes sense to me rather than something thats convenient for the software. Its also a lot quicker doing everything in one package.

    I convert everything to DNG and save everything from a shoot on three separate backup drives before I even start editing, which I do on a fourth separate drive, which I keep solely for editing. After creating edited versions I then save these on another three separate drives. When a drive fills up I store it away and buy a new bigger one. I've had drives break on me and they do wear out. I started this system before Lightroom ever appeared.

    I also use Time Machine on all my Macs to back up the internal disk with all my applications etc. I never store anything on the disk with the operating system.
     
  19. DHart

    DHart Mu-43 Hall of Famer Charter Member

    Jan 7, 2010
    Scottsdale, Arizona
    Real Name:
    Don
    I can certainly see how Lightroom, Aperture, etc. may not be for everyone. Photographers all do different things with different needs, so different strokes for different folks rules the day.

    But for my studio applications Lightroom has made a HUGE difference in the processing and presentation of my images and has significantly helped me boost my client sales, very handsomely. All of my images are immediately at hand, readily available, where I want and need them. I find it an amazing and invaluable tool for not only my business but for my personal work as well. And I feel no crimp in my style at all. To the contrary, it helps my organization as well. I wouldn't ever want to be without Lightroom... it's an amazingly powerful tool for image management, image processing, slide show presentation, and website gallery building.
     
  20. Pelao

    Pelao Mu-43 Top Veteran

    959
    Feb 3, 2010
    Ontario, Canada
    Interesting, because one of the reasons I like LR so much is that it allows me to keep the images wherever I want, in folders and sub-folders within my system, just like Bridge etc. All the import does is copy a ref to the image for the LR catalogue. At any time you can move images around very easily.