Fwiw, I use (for about 2+ years) the Pen-F with several M43 Zuiko lenses and have had no issues requiring back-focus adjustments. In other experiences, Pentax K digital bodies + Pentax lenses=no BF issues given 10+ years.
I've never had that luck, but I'm a picky ****** too. My old Canon 5D Mark II had a serious left-to-right AF focus shift. Got rid of it and bought a Nikon D800E and it needed a -19 AF tune (It only went to a -20). I put up with it until it needed something greater due to weather or age and I took a hex wrench to the secondary AF mirror cam and brought the AF tuning back into a workable range.
With Olympus, and all their AF tuning points (Overkill perhaps!), I managed to work through them all but it took days and a lot of re-thinking the tuning methods. Sort of a learning experience given Canon allows for a Close-and-Tele AF tune, and Nikon does too now but with older models you only had one. The newest Nikons have some Auto-Tuning for their AF (Something Olympus should have done and they could have done it with all 25 points too!).
Below are some screenshots of my E-M1 Mark II AF tuning numbers of several of my lenses. Sometimes I think the bayonet has a tilt to it given the shift in lower right, but at least it is tunable. The 45 f/1.2 is a wild one and might be field curvature given the AF tuning is wide open seeing the outer numbers. The E-M1X has a whole different set of numbers.
Subscribe to see EXIF info for this image (if available)
Don't know if IR shooting would affect this AF stuff and why I wondered about the E-M5 and if it allowed for some focus deviation adjustments should IR mess with its focus a bit. From my experiments with the Pen-F and IR, whatever Olympus is using for a sensor filter does seem to affect the IR a lot. A 720nm filter isn't doing much, and a 760nm seems to help, but overall it's internal UV/IR filter on the sensor is fighting me against using an external IR filter in terms of speed and not getting that much of an IR effect in the output. They may have tuned the Pen-F's spectral sensitivity to a very tight visible light spectrum and are thereby blocking more UV and IR making it difficult to use for IR. Fwiw, that Lifepixel website above is also doing a Pen-F to a Full Spectrum conversion for UV or IR if you drill down to Olympus models that they convert, $350 I think it was.