My understanding of the how a Raspberry Pi camera attached to the CSI port of the Pi works is that the raw data from it ends up directly in the GPU where it is slightly processed and compressed before being either sent to the CPU or displayed with the exception of the Pi 5 where some of that is handled on the RP1 chip first. The big benefit of this, as I understand it, is that we can theoretically access the raw data for computer vision purposes without it ever needing to pass through the CPU bottleneck, however I can't seem to find any explanation about how I would set up such a pipeline to directly use the camera data with any of the common libraries such as OpenCV or OpenGL? Specifically I am intending this project to be in C/C++. Am I missing something where it was implied that the pi automatically understands that the camera data is on the GPU and avoids copying it to the CPU and back or do they just not support it? How would I even check whether or not the camera frame passes through the CPU vs gets processed directly on the GPU?

I saw the suggestion in the answer to this question: How to access raw video data from CSI camera? which says to examine raspivid but that seems to be targeted at the data eventually being compressed and sent to the CPU if I understood that correctly, which I hope to avoid and also doesn't show how to use a more portable library for the processing I would have to do to the raw frame.

I also saw this post: https://robotblogging.blogspot.com/2013/10/gpu-accelerated-camera-processing-on.html but while it has access to what he calls raw YUV camera data, I'm fairly sure that's not the raw format that the camera outputs and I'm not entirely convinced that its pipeline isn't going "raw on GPU--> YUV encoding on GPU --> CPU --> image processing on GPU" from the description of it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.