1

I would like to be imbed a filter in the Pi’s HDMI output such that pixels are slightly rearranged. Specifically, I’m after a retro pincushion effect ([ ] -> ( )). I need it to be active even in the console/boot sequence, thus X11 mods are out.

I realise this is not an easy task, and that I may end up having to modify the DRM or V4L2 driver in order to achieve this. That’s something I would be willing to do if it came to that.

Does anyone have an idea as to how I might be able to go about this? It’s for a Pi Zero and it doesn’t necessarily need to work with a GUI environment.

0
+100

You should start with the "Hello Triangle" sample code that comes with Raspbian and works on all models of Raspberry Pi to get started with OpenGL. Just get rid of the spinning and 5 of the 6 cube faces.

To capture the linux framebuffer you can take fb2png apart. Just focus on the parts that acquire the framebuffer /dev/fb0 and convert it into a 32-bit color pixel array and skip the PNG part.

In OpenGL use it to create a texture. If you are confident with double buffering you can use a non-blocking approach. Each frame you will have to update the content of the texture using glTexSubImage2D (I don't recommend releasing and creating a new texture). If you are trying to increase performance, you can try to find the differences in the previously captured and the newly captured linux framebuffer and update only a rectangle instead of the whole texture.

To test your program you should launch it detached by calling it like ./myprog &. This way the program creates the OpenGL context and overlays the screen but the console still is in control of the keyboard. To stop your program type fg on the console to pass keyboard control back to the OpenGL program and then hit [ctrl]+[c] to terminate it.

If you have it working the way you like it you can launch it on boot by creating a systemd service. I'm not sure about which dependencies it might have on boot time so this will be some trail and error to load it as soon as possible during startup.

1

It always depends on what you put in and expect to get out. So what is your video source? Assuming a USB-Webcam as your V4L2 suggests makes it very CPU intensive. Also doing the deformation on the CPU will limit you in the resolution you can use as the amount of work scales with the pixel count.

The green field for something like that would be to stay on VC4 and do as much hardware accelerated as possible. So the only inputs would be MPEG-4/AVC video streams or the official Raspberry Pi camera. Then you can use the video and pass it over to OpenGL where you can do your cushion deformation in geometry or in a fragement shader.

Leaving that narrow green flield will result in higher CPU usage and lower than realtime frame rates.

  • It's for the console output, which is essentially ASCII. That means I don't need a high framerate at all. 1 FPS would be fine, and not much CPU is used for other tasks (that need real-time output). But because I need it for the console, I can't use GPU acceleration like OpenGL without something particularly fancy – Warpspace Apr 10 at 12:34
  • So your expectation is: you boot up your Raspbian (Lite) into the regular console and want the image to be distorted while still being interactive. – kwasmich Apr 10 at 12:41
  • That might work like this: capture the framebuffer of the console and use OpenGL to overlay the real console with the distorted screenshot. Perhaps using EGL without OpenGL will also be a viable solution. – kwasmich Apr 10 at 12:49
  • How would you do such a thing during bootup? – Warpspace Apr 10 at 12:50
  • You launch your application as a daemon with systemd. – kwasmich Apr 10 at 12:52
0

I think if it was me, given the constraints, I'd define a new pixel-mangling encoder in vc4 drm that used the writeback engine ("txp") to capture HVS scenes to memory, and make sure that your console and X11 were displaying to that and not HDMI.

Once each write back completes, I'd do whatever algorithm on the captured scene into a new buffer, and then emit pageflips to your HDMI (or whatever) encoder you want to actually see.

  • Ah, that's the kind of thing I was hoping to do. I'd award you the bounty if I could. That Writeback Engine sounds perfect – Warpspace Apr 20 at 7:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.