Intro
We found that the colour response by different camera models under the same lighting conditions can vary considerably. This meant that large volumes of training data that we had collected and labelled previously was not much use if using a different camera model.
We initially tried to get the colour spaces to match by adjusting the cameras’ white balances, but were unable to get an accurate enough match.
Sensors
We use a range of machine vision and DSLR cameras. For example:
- PixeLINK PL-D755-CU
- PixeLINK PL-D753-CU
- Allied Vision Alvium 1500 C-500
- Allied Vision Alvium 1800 C-507
- Nikon D5600
Workflow and implementation
Generating ICC Profiles
For each camera we capture images of a ColorChecker inside a light box with a standard illuminant.
We then generate the ICC profiles using the Argyll CMS
colprof and scanin CLI applications.
Converting Images
Now that we have ICC profiles for each camera, we are able to convert their images to the colour spaces of other cameras or to the standard srbg
colour space. We do this with the ImageCMS module within Pillow.
Training and Inferring
The easiest method would now be to transform each image to a standardised colour space (e.g. srgb
) and use these to train the model. One would then convert the colour space of the captured image to srgb
before inferring on it.
Our use case requires very low latency real-time inferences and converting the colour space before inference is just another pre-processing step increasing latency. We thus train models for specific camera models by converting the training data to that camera’s color space to before training the model. We are then able infer on the image with minimal pre-processing.
Camera deltas
The attached image contains a table with some raw camera examples (top row) and their conversion to a standard srgb
colour space (bottom row). Notice that the Camera 1 raw images are a much more intense green than those of Camera 2. The resulting srgb
colour spaces are much closer to each other.
With regards to your question:
Also, to confirm it sounds like you’re interested in both (A) tactical mitigations for challenges with the current approach and more so (B) strategic suggestions for overcoming this category of probably for a scalable solution that might remove the need for the homegrown mitigation.
This is correct, a strategic solution that removes the need for our homegrown solution would be ideal, but any tactical ideas would be greatly appreciated.