1

I'm trying to use the Raspberry Pi plus the NoIR Picamera module V2.1 to collect raw (bayer-filtered) IR images for NDVI analysis of plants (programs written in Python). So far, I haven't found anyone else on the Internet run into the particular problem I have with handling raw image formats on the Pi.

My process is roughly as follows:

  • Capture filtered IR-image (with option 'bayer=True')
  • Save image in appropriate raw format (in this case, DNG or TIFF)
  • Open the image in software and apply an NDVI Lookup Table to see how much IR light the plants are reflecting/absorbing

The process for the saving the images is a little different between the two formats. The relevant code for each is below:

  • TIFF:

    with picamera.PiCamera() as camera:
        with picamera.array.PiBayerArray(camera) as stream:
            #take a picture with raw bayered data attached to jpeg
            camera.capture(stream, 'jpeg', bayer=True)
    
            #use picamera.array built-in demosaic function
            output = (stream.demosaic() >> 2).astype(np.uint8)
    
            #use OpenCV library to save our demosaiced array to a tiff file
            cv2.imwrite(file_name.tiff,output)
    
  • DNG:

    with picamera.PiCamera() as camera:
        camera.capture(file_name.jpg,bayer=True)
    
        #we skip the demosaicing step when saving to a DNG because imwrite from
        #OpenCV does not support DNG
    
        #instead we use the pydng tool to save the JPEG+RAW directly to a DNG
        pydng.createDNG(file_name.jpg)
    

(Full disclosure: I do not know what kind of demosaicing method picamera.array uses, nor am I deeply familiar with pydng and how it works).

I was initially using the first procedure (saving debayered raw image as TIFF) and obtained images where the center is clearly much brighter than the edges. Since this does not appear in the JPEG, I assumed this was due to lens artifacts that are revealed in raw images and are otherwise processed out in compressed formats. This bullseye distortion becomes even more evident when I use Fiji (aka Image-J) to apply an NDVI lookup table (LUT).

The following three images are of: 1. basic JPEG from Picamera; 2. TIFF without LUT; 3. TIFF with NDVI LUT applied.

Basic JPEG of the plants

TIFF produced using first method; we expect a darker image, but not that the edges are mysteriously darker than the center

The LUT shows a bullseye pattern in the lighting

However, in the second process (saving bayer-filtered raw image as DNG), I open and demosaic the file in a raw-image processing software called Rawtherapee, export to TIFF-format, apply the same lookup table in Fiji, and get a very different result. The shadows are gone (see first image below), and the NDVI image looks way nicer (second image below). This is what I wanted the whole time!

Magenta tinting is apparently a known artifact of using pydng. But notice that the lighting in this photo is even, more like the processed JPEG

NDVI LUT also shows more even reflectance from the plants, and no strange bulls-eye distortion

I'm curious to figure out how/why these two TIFF files look so different, magenta-tinting aside (this is a documented product of using pydng). Where does this bullseye distortion come from? Based on the differences between the two procedures, I figure the problem must lie in one of these steps:

  • pydng's conversion from RAW to DNG: is it doing extra processing?
  • picamera.array's demosaicing algorithm: maybe it sucks?
  • OpenCV's imsave function, and how it saves TIFF files: maybe I'm misusing the function?
  • Rawtherapee software: is it doing some additional image processing that I missed?

Can anyone point me in the right direction, or has anyone else seen something like this? I'd be really grateful for any thoughts or ideas you might have!

0

That "bulls-eye" in the top image looks exactly like lens-shading artefacts to me (it's exactly what I'd expect from the raw bayer output which is prior to lens shading correction in the pipeline).

I can confirm that picamera's demosaic definitely sucks - it's an extremely basic demo I threw together (code is here for the terminally curious - it's quite well commented but uses some dirty numpy hacks for a bit more performance - and it's still dog slow which gives you an idea of how much faster the GPU is at this stuff).

I'm not familiar with pydng either, but its README includes the line:

shade - Contains 16-bit bayer lens shading profiles in numpy format.

That suggests to me that it's very likely it's doing lens-shading correction for you. It's notable that library is tuned for the v2 Raspberry Pi camera module, which is exactly what you're using (otherwise I'd expect any lens shading to be slightly wrong - would be interesting to see its output with a v1 module!).

  • Thanks for your response, and sorry for my delay in getting back! I'm in the process of taking another round of pictures and will update here as soon as I have some new info(: – user81979 Mar 13 '18 at 15:44
0

So based on Dave's observation, I zeroed in on the RAW->DNG->(demosaic)->TIFF process. I commented out the portions of pydng that had anything to do with lens shading and compared the resulting images with and without that correction.

Image without LUT applied

Even without the LUT applied, once we demosaic and export the DNG to a TIFF, we can see the familiar shadows from the RAW->TIFF process. The LUT makes the distortion even more evident:

The bullseye has returned!

In other words, Dave's hunch was right! The distortion in my original TIFF files were coming from lens shading, and pydng was correcting this. Unfortunately, I don't have a Picamera V1 to play with, otherwise it would indeed be really interesting to see how pydng can handle correcting a different camera's lens shading issues.

Thanks Dave! I know this is probably not a very common issue people run into, but I hope someone else will find it useful for their raw image processing applications(:

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.