1. Reminder: Please user our affiliate links to get to your favorite stores for holiday shopping!

E-M1 burst jpg/raw file write order bug?

Discussion in 'Olympus Cameras' started by Erich Linder, Sep 22, 2015.

  1. Erich Linder

    Erich Linder New to Mu-43

    7
    Sep 22, 2015
    Cleveland, OH
    Hi all, first post here. A few weeks ago I bought an E-M1, and so far I'm very impressed. However, I've noticed what I think is a firmware bug. I asked over on Fredmiranda.com and had one other person say they've also seen it.

    If I set the camera to shoot both super fine JPG and RAW files on continuous low or high bursts, the files are not always written in sequential order. What I mean is that when I load the images into Lightroom with the files sorted by capture time, the file names are not sequential. The time stamp must be slightly off between the corresponding jpg and raw file. Sometimes they are in the correct order, but at times they will randomly show up as a few jpgs, then a few raw files. There doesn't seem to be any consistancy.

    I submitted a report to Olympus and they promptly wrote back and said it is not a know issue and want more information on when it happens.

    Anyone else seen this?

    Erich
     
  2. drd1135

    drd1135 Zen Snapshooter

    Mar 17, 2011
    Southwest Virginia
    Steve
    I see this sometimes in Adobe Bridge and I never use burst mode. They are usually closely timed shots, however. They are usually in the same neighborhood, however, so I've never worried about it.
     
  3. bassman

    bassman Mu-43 Top Veteran

    678
    Apr 22, 2013
    New Jersey
    Scott
    I've seen it on my E-M1 as well. I suspect that the time stamps on the images are actually the same, so LR can't tell what order they were actually taken. Remember, you're asking it to display by the timestamp, not the order ingested.
     
  4. Clint

    Clint Mu-43 All-Pro

    Apr 22, 2013
    San Diego area, CA
    Clint
    How do you know for fact that the time stamps are off? How do you know that Lightroom is sorting your files exactly as you expect?

    Olympus records the <xmp:CreateDate> to the hundredth of a second. Every time stamp for jpeg/raw combinations are exactly the same time, to the hundredth of a second.

    Sometimes I think Lightroom has a mind of it's own and changes my sort order (I'm probably doing something that causes Lightroom to change it) so I have to occasionally reset the sort order.
     
  5. Erich Linder

    Erich Linder New to Mu-43

    7
    Sep 22, 2015
    Cleveland, OH
    You're right, I don't know for a fact, but I'm assuming it's something to do with the time stamp of the jpg and its corresponding raw file not being exactly the same, otherwise they would be next to each other when sorting by created time in Lightroom. Perhaps it's writing into the xmp the time of file writing and not frame capture, so when the buffer gets behind in writing it's enough to change the order of when the files get written.

    Over the past 2 years I've been shooting lots of sports, primarily equestrian competitions, so I do just about everything in high speed bursts. All 3 Nikon cameras I've used sort properly in Lightroom in that way, and consistently. The fact that the E-M1 is not sorting the files in a consistant manner tells me something isn't right. Sometimes I'll get jpg, raw, jpg, raw, then suddenly jpg, jpg, jpg, raw, raw, jpg, raw, raw. I've never seen that with the Nikon files. Since I'm not the only one who has seen it also tells me something is amiss.

    Erich
     
  6. wjiang

    wjiang Mu-43 Hall of Famer

    Somewhat unrelated to the problem at hand, but why shoot in RAW+JPEG for continuous bursts? You end up filling the buffer faster and end up with double the number of files to cull.
     
  7. Clint

    Clint Mu-43 All-Pro

    Apr 22, 2013
    San Diego area, CA
    Clint
    Eric, I gave some incorrect info. The time stamp visible in <xmp:CreateDate> only goes to the second - and now I can't find where in the XMP data the hundredth of a second is. And what I mentioned about sorting was in Lightroom 3 or 4, so is now outdated.

    I just went into a folder of 600+ photos that I shot a lot of jpeg/raw burst. I did have to resort a couple of times to get what I wanted, with most of the photos came out on .orf, jpg order - however there are several sets of orf, orf, orf, jpg, jpg, jpg - and I can't 100% of all of the photos to sort orf, jpg.

    But when I go to the Windows OS file listing, they sort as they should.

    I do remember having a hell of time with Lightroom 3 or 4 to get them sorted correctly. Just can't remember exactly what I did at the time.
     
  8. eteless

    eteless Mu-43 All-Pro

    Jun 20, 2014
    On a sample size of a few dozen places randomly selected out of a folder with thousands of pictures in it all of mine were in order. Sometimes the order of jpeg/raw reverses in bridge (jpeg comes first, then raw, then raw, then jpeg. Rather than always jpeg, raw) however this appears to just be the adobe implementation for Bridge as Lightroom is always correct.

    Could you link to the FM post about it? I would be interested to have a look out of curiosity if nothing else.
     
  9. Erich Linder

    Erich Linder New to Mu-43

    7
    Sep 22, 2015
    Cleveland, OH
    Wjiang, I'm shooting both jpg+raw partly out of habit, as my Nikons have dual cards and I write one file to each card as backup. I'm doing it now since I've only had the camera a few weeks, so I'm comparing the in camera jpg to what I get processing to raw files until I get a feel for them. The raws also have much greater processing flexibility.

    The thread at Fredmiranda doesn't have many replies yet. http://www.fredmiranda.com/forum/topic/1388192

    I did a few more tests. Sorting the files on my Mac in Finder by date/time displays them in the correct order. I also loaded them into the last (now abandoned) version of Aperture, also sorts them correctly by date/time. Now I was even more perplexed, so I created a new Lightroom catalog and reimported the files. They sort by capture time in the exact same non-correct order as the other Lightroom catalog. So at least it's consistently random. :)

    Erich
     
  10. Drdul

    Drdul Mu-43 Regular

    101
    May 16, 2015
    Vancouver, BC
    Richard
    FWIW, this happens on my E-PL5 shooting in Raw only (no jpeg). On my camera at least, it appears the camera fills the buffer and then writes the images in reverse order after writing the first correctly ordered image in the sequence (e.g. for 4 images, the camera writes 1, 4, 3, 2).
     
  11. Growltiger

    Growltiger Mu-43 Top Veteran

    640
    Mar 26, 2014
    UK
    There is nothing wrong with the camera, it writes them in order.
    The problem is computer software that only sorts by whole numbers of seconds. So they muddle up the files taken in the same second.
    You can easily prove this by sorting in Windows in File Explorer, or in Finder on the Mac. They work properly.
     
  12. Drdul

    Drdul Mu-43 Regular

    101
    May 16, 2015
    Vancouver, BC
    Richard
    On my camera, at least, it numbers the images in the wrong order, so even if they are taken in the same second, they are numbered (and I assume written to the memory card) in the incorrect order.
     
  13. Erich Linder

    Erich Linder New to Mu-43

    7
    Sep 22, 2015
    Cleveland, OH
    No, it's definitely something about how Olympus writes the files. All the jpg+raw bursts of 11 frames per second from my Nikon D4s are ALWAYS in the correct order in Lightroom.

    Erich
     
  14. Clint

    Clint Mu-43 All-Pro

    Apr 22, 2013
    San Diego area, CA
    Clint
    and yet

    ??? or it must be fuzzy logic.
     
  15. Growltiger

    Growltiger Mu-43 Top Veteran

    640
    Mar 26, 2014
    UK
    It could be that both are true. The format may be slightly different between Nikon and Olympus but both could be to standard and both could be correctly handled by Explorer and Finder. Lightroom could be failing to handle the Olympus version correctly.
     
  16. eteless

    eteless Mu-43 All-Pro

    Jun 20, 2014
    Another question is which version of Lightroom? Mine is working just fine with no sorting errors (whatever comes with CC).
     
  17. Erich Linder

    Erich Linder New to Mu-43

    7
    Sep 22, 2015
    Cleveland, OH
    I'm using the newest stand alone Lightroom Mac version 6.1.1 with Camera RAW version 9.1.1 along with Photoshop CS6.

    After a lot of admittedly anecdotal reading online, it appears that when sorting by time Aperture reads the exif to the hundredth of a second, and if those values are the same resorts to a secondary sorting order of numerical/alphabetical order. If I look at the exif file properties in Photoshop the Nikon files show a time stamp to the hundredth of a sec at the beginning, and then buried further down is another time stamp that goes to the thousandth of a sec. Looking at both those fields on the Olympus files show both go to the hundredth.

    Does anyone have a program called ExifTool by Phil Harvey? It seems to be the goto program for EXIF, but for some reason I can't get it to run on my Mac. Nothing happens when I execute the command line in Terminal.

    This weekend I'm shooting a horse show, so I'm going to set up the E-M1 as a remote camera if my shutter release plug adapter for my Pocketwizards arrives in time. I should have lots of shots to experiment with, as I plan on trying just jpg, just raw, and jpg+raw modes all at high speed.

    Erich
     
  18. bassman

    bassman Mu-43 Top Veteran

    678
    Apr 22, 2013
    New Jersey
    Scott
    I believe that the cameras record the timestamp to a suitable precision for sorting under most or all circumstances. However, LR (CC 2015) displays only to the nearest second, and may not use the full precision for sorting. And since it appears it only has the bad sorting behavior when shooting continuous sequences, it stands to reason that the problem is the timestamps it uses to sort are not precise enough.
     
  19. Erich Linder

    Erich Linder New to Mu-43

    7
    Sep 22, 2015
    Cleveland, OH
    Olympus tech support wrote back again requesting more information. They're also downloading a bunch of image files I posted to Dropbox so they can look at the files directly to see if they can replicate the issue.

    Erich
     
  20. franklyadam

    franklyadam Mu-43 Rookie

    24
    Mar 3, 2014
    My e-m1/LR CC does this too, shooting raw mode only. I just sort all my folders by file name vs capture time, solves the problem :)

    I think it's a Lightroom problem not using enough precision on the time sorting.