Improving the flat field performance of the Mk. II scanner

Last time on Deep Space Nine we managed some impressive performance improvements, today we claw those slightly back by improving image quality.

The problem

The problem is patchy images, as usual.

Typical patchy image, there is a vertical split right of the gull where the grey values switch from magenta to green.
This image still isn't perfectly corrected, but is better.

The problem is often ignorable, but really shows up when a flat grey background is part of the image.

What's happening is kind of obvious, the sensor or imaging system as a whole has some green/magenta gain issues. Could it be stray light?¿

The effect is most noticeable with colour images since the green/magenta difference is mostly smoothed out in greyscale mode (where no CCM's are used, for example).

The fix

The fix, as usual, is to spend all of a Sunday working the problem. I spent a long time trying flat field corrections that had the R/B channels swapped, which obviously causes some issues with corrections.

The final fix I ended up on is to remove the fancy object-plane lens hood I spent hours making, and re-enabling flat field correction. It seems the hood causes various weird interactions with the object being imaged that makes it nearly impossible to correct the effect with a standard flat field. This was visible in the flat field image when contrast enhanced, typically as a higher response in the upper left.

As seen above perfection hasn't been achieved for all images but the effect has been mitigated without major performance cost.

A slight magenta/green line is visible vertically along the center blue column, and behind the boat horizontally.
With adaptive FFC the issue is more or less resolved again.

With the lens hood removed I was able to get a good flat field image that actually looked symmetrical.

Contrast-maximised flat-field image including some magenta/green shading

Here's another comparison. Excepting the first gull picture the exact same processing was used for all these comparions. The gull picture was originally processed in Display P3 while the rescan is in Rec.2020 so some differences are visible.

In this original scan the right side of the image has a green cast
With the adaptive FFC  the right side of the image is less green. The purple dots I can't really explain.

I ended up implementing adaptive FFC where I use a fixed flat-field image (showing a light grey), and for each subframe I turn off the backlight and use that capture as the black reference for each frame. This extra data capture does lead to a ~50% speed reduction, in that each 35 mm frame takes around 3 minutes instead of 2.

With the object plane hood removed the system becomes more sensitive to ambient light so doing this seems to generally improve image quality.

On the processing side I made some changes, embedding the dark-frame image in the same file (switching from .npy to .npz format) for convenience when post processing.

I switched to debayering on the scanner computer as the final step before writing files to disk, offloaded to a separate thread. This way the debayering (which takes about as long on the scanner as the server) is done as the data comes in, and the final file output can happen after the scanner has started work on the next frame.

The total time saved is minor but it makes reprocessing afterwards much faster (something I have to do a lot when iterating over e.g. different flat field corrections). The downside is that a 35 mm raw scan now takes up around 4.5 GB instead of around 400 MB.

Addendum: Flickering backlight issues

After publishing this I did some additional testing to try to work out what was causing the residual patchiness I observed.

Since I do frame averaging, I decided to test performing the averaging by doing max()/min() operations, as well as geometric averaging, all of which are more sensitive to any individual image being off vs. the average.

This revealed that the root cause of the residual error seems to be the WS2812B backlight flickering vs. the readout rate of the rolling shutter. This is probably worse with a rolling shutter camera vs. most of the previous testing done with global shutter cameras.

I did some testing with an honest to god light bulb and immediately had perfect stitching. Future work will have to be to make a non-flickering backlight now that this issue is known.

Some colour work

I made a change to my colour workflow from Display P3 to Rec.2020 format, starting from this article I expect most images will be in this format.

The main reason for doing this is simply that the R/G/B primary wavelengths of the scanner backlight match the Rec.2020 primaries perfectly, so I figure this should be more accurate? The images seem to render fine but I've noticed some software (mainly Photos.app and Preview.app on macOS) displays them darker than expected, Chromium based browsers, quick look preview, and iOS Photos.app seem to display them properly.

Curiously Chromium doesn't seem to know about Rec.2020 so it calls it "Adobe RGB/ProPhoto RGB", the latter having similar primaries.

This makes things different though it doesn't do anything about the hyper-saturated reds, that's a separate issue.

Still don't exactly know what I'm doing here, as might be obvious.