0

I would like to know the size of my I/O buffer (defined as the queue where the packets arrive, e.g. from internet, and wait to be proccessed). How could I find this size?

And if the buffer is too short for my application (real time app, e.g. video streaming that implicate high amount of data), how could I know the limitation of the I/O buffer?

Thank you

  • 2
    Your question is a bit low on details. What buffer and packets are you talking about? And processed by what? Please update your question with more detailed information. – Dirk Nov 15 '18 at 11:36
  • "if the buffer is too short for my application" -> You have this the wrong way around. The question should be, "if my application is too slow for the buffer" (in which case you will lose data), then you should design your application so that it reads more often, if need be into your another buffer of your own creation, if need be using multiple threads or asynchronous handling. If that buffer then grows endlessly because you cannot process it fast enough, then you will have to throttle your input. – goldilocks Nov 19 '18 at 16:59
1

There is no single IO queue in Linux where all data arrive.

For disk IO, there is the buffer cache. That's the one you flush with the sync command. It's usually managed pretty well so there's no reason to touch it, but in case you're interested, there are a few relevant kernel interface files available in /proc/sys/vm/. On top of that, there are various buffers specific to open files, which can be managed by setbuf and similar tools/calls.

Network buffering configuration can be seen in /proc/sys/net/core/, or managed via sysctl. For incoming packets, look into net.core.rmem_max and net.core.rmem_default (replacing rmem with wmem will give the respective buffer sizes for outgoing packets). The default value is applied to all sockets and can be increased (up to the maximum) by the application which opens the socket.

For video devices, the buffering is typically managed by the application, using IOCTL calls on the video device file (e.g. /dev/video1). Applications are limited by what the drivers implement: typically, the driver can either stream to a mapped memory, copy data between buffers using DMA or swap buffers with application via pointers. As a user, take a look at v4l2-ctl to see what options are available to you.

  • 1
    The 'r' in 'rmem' stands for read -- i.e., those are input buffers which is what the OP is concerned about, but there is also wmem values for output buffers. In addition to sysctl they can be found in /proc/sys/net/core/. The value per socket can be set programmatically up to the maximum (which is usually what the default is), see SO_RCVBUF (and SO_SNDBUF) in man 7 socket. – goldilocks Nov 19 '18 at 14:36
  • @goldilocks Thanks for your addition. This is not meant to be an exhaustive (or even orderly) description, I just wanted to give some pointers from which one could find relevant docs and man pages. – Dmitry Grigoryev Nov 19 '18 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.