Image artifacts in raw TIFF from Pi Camera disappear when I first save as DNG and then convert. What gives?

Question

I'm trying to use the Raspberry Pi plus the NoIR Picamera module V2.1 to collect raw (bayer-filtered) IR images for NDVI analysis of plants (programs written in Python). So far, I haven't found anyone else on the Internet run into the particular problem I have with handling raw image formats on the Pi.

My process is roughly as follows:

• Capture filtered IR-image (with option 'bayer=True')
• Save image in appropriate raw format (in this case, DNG or TIFF)
• Open the image in software and apply an NDVI Lookup Table to see how much IR light the plants are reflecting/absorbing

The process for the saving the images is a little different between the two formats. The relevant code for each is below:

• TIFF:

  with picamera.PiCamera() as camera:
      with picamera.array.PiBayerArray(camera) as stream:
          #take a picture with raw bayered data attached to jpeg
          camera.capture(stream, 'jpeg', bayer=True)
  
          #use picamera.array built-in demosaic function
          output = (stream.demosaic() >> 2).astype(np.uint8)
  
          #use OpenCV library to save our demosaiced array to a tiff file
          cv2.imwrite(file_name.tiff,output)
  

• DNG:

  with picamera.PiCamera() as camera:
      camera.capture(file_name.jpg,bayer=True)
  
      #we skip the demosaicing step when saving to a DNG because imwrite from
      #OpenCV does not support DNG
  
      #instead we use the pydng tool to save the JPEG+RAW directly to a DNG
      pydng.createDNG(file_name.jpg)
  

(Full disclosure: I do not know what kind of demosaicing method picamera.array uses, nor am I deeply familiar with pydng and how it works).

I was initially using the first procedure (saving debayered raw image as TIFF) and obtained images where the center is clearly much brighter than the edges. Since this does not appear in the JPEG, I assumed this was due to lens artifacts that are revealed in raw images and are otherwise processed out in compressed formats. This bullseye distortion becomes even more evident when I use Fiji (aka Image-J) to apply an NDVI lookup table (LUT).

The following three images are of: 1. basic JPEG from Picamera; 2. TIFF without LUT; 3. TIFF with NDVI LUT applied.

However, in the second process (saving bayer-filtered raw image as DNG), I open and demosaic the file in a raw-image processing software called Rawtherapee, export to TIFF-format, apply the same lookup table in Fiji, and get a very different result. The shadows are gone (see first image below), and the NDVI image looks way nicer (second image below). This is what I wanted the whole time!

I'm curious to figure out how/why these two TIFF files look so different, magenta-tinting aside (this is a documented product of using pydng). Where does this bullseye distortion come from? Based on the differences between the two procedures, I figure the problem must lie in one of these steps:

• pydng's conversion from RAW to DNG: is it doing extra processing?
• picamera.array's demosaicing algorithm: maybe it sucks?
• OpenCV's imsave function, and how it saves TIFF files: maybe I'm misusing the function?
• Rawtherapee software: is it doing some additional image processing that I missed?

Can anyone point me in the right direction, or has anyone else seen something like this? I'd be really grateful for any thoughts or ideas you might have!

Answer 1

That "bulls-eye" in the top image looks exactly like lens-shading artefacts to me (it's exactly what I'd expect from the raw bayer output which is prior to lens shading correction in the pipeline).

I can confirm that picamera's demosaic definitely sucks - it's an extremely basic demo I threw together (code is here for the terminally curious - it's quite well commented but uses some dirty numpy hacks for a bit more performance - and it's still dog slow which gives you an idea of how much faster the GPU is at this stuff).

I'm not familiar with pydng either, but its README includes the line:

shade - Contains 16-bit bayer lens shading profiles in numpy format.

That suggests to me that it's very likely it's doing lens-shading correction for you. It's notable that library is tuned for the v2 Raspberry Pi camera module, which is exactly what you're using (otherwise I'd expect any lens shading to be slightly wrong - would be interesting to see its output with a v1 module!).

Answer 2

So based on Dave's observation, I zeroed in on the RAW->DNG->(demosaic)->TIFF process. I commented out the portions of pydng that had anything to do with lens shading and compared the resulting images with and without that correction.

Even without the LUT applied, once we demosaic and export the DNG to a TIFF, we can see the familiar shadows from the RAW->TIFF process. The LUT makes the distortion even more evident:

In other words, Dave's hunch was right! The distortion in my original TIFF files were coming from lens shading, and pydng was correcting this. Unfortunately, I don't have a Picamera V1 to play with, otherwise it would indeed be really interesting to see how pydng can handle correcting a different camera's lens shading issues.

Thanks Dave! I know this is probably not a very common issue people run into, but I hope someone else will find it useful for their raw image processing applications(: