1

Cross posting here, since the question suits at both sites - https://stackoverflow.com/questions/39524234/bug-with-writing-to-file-in-linux-sys-class-gpio


I'm having the strangest bug I've ever seen with a linux system right now and there seem to be only two possible explanations for it -

  • Either appending sudo makes file writes instant
  • Or appending sudo produces a short delay in executing statements
  • Or I've got no clue what's happening with my program

Well let me give you some background. I'm currently writing a c++ program for raspberry pi gpio manipulation. There are no visible error in the program as far as I know & since it works with sudo successfully and with delays successfully too. So here's how rpi's gpio work -

  • First you've to export one, to reserve it for manipulation, it will create a new directory as gpio+number with several files in it.

    echo 17 > /sys/class/gpio/export

  • Then set it's direction(in means read and out means write)

    echo "write" > /sys/class/gpio/gpio17/direction

  • Then write the value (0 or 1 for off and on)

    echo 1 > /sys/class/gpio/gpio17/value

  • At the end, unexport it back, the directory will get deleted.

    echo 17 > /sys/class/gpio/unexport

It doesn't matter whether you do this through bash commands or through c/c++ or any other language IO, since in unix these are just files and you just need to read/write to them. Everything works fine till now. I've tested this manually and it works, so my manual test passes.


Now I've a simple test written for my program which looks like this -

TEST(LEDWrites, LedDevice)
{
    Led led1(17, "MyLED");
    // auto b = sleep(1);
    EXPECT_EQ(true, led1.on());
}

The Led class constructor does the export part - echo 17 > /sys/class/gpio/export, while the .on() call sets the direction - echo "write" > /sys/class/gpio/gpio17/direction and outputs the value as well - echo 1 > /sys/class/gpio/gpio17/value. Forget about unexport here since it is handled by destructor and plays no role here.

If you're curious, these functions handle I/O like this -

{
    const std::string direction = _dir ? "out" : "in";

    const std::string path = GPIO_PATH + "/gpio" + std::to_string(powerPin) + "/direction";

    std::ofstream dirStream(path.c_str(), std::ofstream::trunc);
    if (dirStream) {
        dirStream << direction;
    } else {
        // LOG error here
        return false;
    }
    return true;
}

means basic c++ file/io. Now let me explain the bug.

First, here are 3 runs of same test -

Normal run FAILS

[isaac@alarmpi build]$ ./test/testexe
Running main() from gtest_main.cc
[==========] Running 2 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 1 test from LEDConstruction
[ RUN      ] LEDConstruction.LedDevice
[       OK ] LEDConstruction.LedDevice (1 ms)
[----------] 1 test from LEDConstruction (1 ms total)

[----------] 1 test from LEDWrites
[ RUN      ] LEDWrites.LedDevice
../test/test.cpp:20: Failure
Value of: led1.on()
  Actual: false
Expected: true
[  FAILED  ] LEDWrites.LedDevice (2 ms)
[----------] 1 test from LEDWrites (3 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 2 test cases ran. (6 ms total)
[  PASSED  ] 1 test.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LEDWrites.LedDevice

 1 FAILED TEST

run with sudo PASSES

[isaac@alarmpi build]$ sudo ./test/testexe
[sudo] password for isaac: 
Running main() from gtest_main.cc
[==========] Running 2 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 1 test from LEDConstruction
[ RUN      ] LEDConstruction.LedDevice
[       OK ] LEDConstruction.LedDevice (1 ms)
[----------] 1 test from LEDConstruction (2 ms total)

[----------] 1 test from LEDWrites
[ RUN      ] LEDWrites.LedDevice
[       OK ] LEDWrites.LedDevice (2 ms)
[----------] 1 test from LEDWrites (2 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 2 test cases ran. (5 ms total)
[  PASSED  ] 2 tests.

wtf delay run PASSES has uncommented // auto b = sleep(1);

[isaac@alarmpi build]$ ./test/testexe
Running main() from gtest_main.cc
[==========] Running 2 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 1 test from LEDConstruction
[ RUN      ] LEDConstruction.LedDevice
[       OK ] LEDConstruction.LedDevice (1 ms)
[----------] 1 test from LEDConstruction (2 ms total)

[----------] 1 test from LEDWrites
[ RUN      ] LEDWrites.LedDevice
[       OK ] LEDWrites.LedDevice (1001 ms)
[----------] 1 test from LEDWrites (1003 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 2 test cases ran. (1005 ms total)
[  PASSED  ] 2 tests.

The only difference b/w delay and normal run is of single uncommented line - // auto b = sleep(1); Everything is same including device, directory structure, build conf and everything. The only things that explains this is linux might be creating that file and its friends sometimes later or it takes some time? and I call .on() before that. Well that could explain it...

But then why does sudo invocation with no delay passes? Does it makes those writes faster/instant or does it puts the delay statement by itself? Is this the cause of some kind of buffering? Please say no :/


If it matters, I'm using following dev rule for getting non sudo access to gpio directory -

SUBSYSTEM=="gpio*", PROGRAM="/bin/sh -c 'chown -R root:gpio /sys/class/gpio && chmod -R 770 /sys/class/gpio; chown -R root:gpio /sys/devices/virtual/gpio && chmod -R 770 /sys/devices/virtual/gpio'"

Strace to the rescue


Let's see the execution of the failing build command -

open("/sys/class/gpio/export", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/unexport", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/export", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/gpio17/value", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = -1 EACCES (Permission denied)
open("/sys/class/gpio/gpio17/direction", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = -1 EACCES (Permission denied)
open("/sys/class/gpio/unexport", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3

..., 0666) = -1 EACCES (Permission denied)

Okaaay, here's something, that explains why it is passing with sudo. But why is it passing with delay? Let's check that too,

open("/sys/class/gpio/export", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/unexport", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/export", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/gpio17/value", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/gpio17/direction", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 4
open("/sys/class/gpio/unexport", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3

No wait, wtf? This means the permission denied must be for if files aren't created at that time. But how does using sudo solves that?

Here's relevant output for sudo -

open("/sys/class/gpio/export", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/unexport", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/export", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/gpio17/value", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
open("/sys/class/gpio/gpio17/direction", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 4
open("/sys/class/gpio/unexport", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
  • "Or I've got no clue what's happening with my program" neither do we, because you haven't listed it. – Milliways Sep 16 '16 at 6:38
  • @Milliways I've provided enough code, I've added strace output too. I cannot provide a minimal example because if I'm trying to change its execution, things aren't reproducible. However you're free to check the source - github.com/agauniyal/isaac-core/tree/update/readme – Abhinav Gauniyal Sep 16 '16 at 6:44
  • 1
    A one paragraph summary of the problem and question would have been helpful. There are race conditions to do with setting up the GPIO via the sysfs. The solution is to add delays. – joan Sep 16 '16 at 8:36
  • try * force flush() on the ostream used to write to the files * using stat in a loop with short sleeps after attempting to create the gpio folder/files to wait until they are actually created. – Shreyas Murali Sep 18 '16 at 22:44
  • Hmm stat looks good. What do you mean by force flush()? The stream is flushed as soon as function exists, because RAII in c++ causes to invoke destructor, which will explicitly flush the streams when fd is closed. – Abhinav Gauniyal Sep 18 '16 at 22:52
1

It's a race condition between your code and some lower-level device code such as the udev rules. After you create gpio17 with echo 17 > export, some udev code fires up. As you noted in your question, on the Raspberry Pi the rule calls chown -R and chmod -R. Recursively walking through entire directory trees is gonna take some time, but the udev rule does NOT block your code while it works! If your code immediately starts to tickle anything in that tree, it's touching things that are potentially still being modified by another process. User "Arnot" gives some detail on the issue in his answer to: https://stackoverflow.com/questions/39524234/bug-with-writing-to-file-in-linux-sys-class-gpio

You can do a lot with pauses and checking permissions to see if things are ready, but it's not guaranteed. Arnot wrote that the system might not even start the udev script until your code is finished, in which case semaphores or any other changes in the udev script can't fix the core problem. It's understandable that udev doesn't want to block everything whenever a device gets plugged in, but at least in the case of GPIO's it would be nice to have the default behavior err on the side of safety over performance. I don't know much about udev (yet), so maybe there's an existing flag or technique already.

In the meantime, I've moved all my GPIO exports to a separate step. After all exports are done, I resort to the very ugly tactic of "sleep", followed by some less-ugly checks for permissions and validity, and finally another sleep out of guilt that I wrote the first one. Only then do I start setting directions and writing out values. GPIO has been 100% rock solid ever since, even when I deliberately load the system to try to break it. Still, the device we're building might be used for life-critical analysis someday (it detects certain DNA patterns), so despite the track record I hope to replace the sleeps someday with a better understanding of udev and proper locking.

(EDIT) By request, here is some example code showing the sleep method. Note that we probably don't need to sleep for 3 seconds, yet there is no guarantee that 3 seconds is long enough.

#!/usr/bin/env bash
xx=17
yy=18
trap "\
echo $xx > /sys/class/gpio/unexport ;
echo $yy > /sys/class/gpio/unexport" EXIT
echo $xx > /sys/class/gpio/export
echo $yy > /sys/class/gpio/export
sleep 3 # Add perm checks and more sleeps here as paranoia demands.
echo out > /sys/class/gpio/gpio${xx}/direction
echo out > /sys/class/gpio/gpio${yy}/direction
while sleep 1
do for i in $xx $yy
   do echo $(( SECONDS % 2 )) |
           tee /sys/class/gpio/gpio${i}/value |
           sed "s/^/Setting GPIO $i value to: /"
   done
done

As long as I'm at it, here is "blinkys_fun_club.sh". It's some old code I wrote that demonstrates an alternative method to the GPIO/udev problem which eliminates the risk of using "sleep". Unfortunately it doesn't work in the general case (see the notes section for details), but at least it proves that the use of "sleep" can be avoided in certain situations.

#!/usr/bin/env bash
# Randomly blink on/off a set of GPIO's. --willdye, 2016, public domain

# If this "eval" trips a coding-standard warning, see the notes section.
peat(){ eval $(echo "${@//^^^^/"{17,18,27}"};");} # Define GPIO set here.

trap 'peat "echo ^^^^ > /sys/class/gpio/unexport"' EXIT   # Remove GPIOs.
peat       "echo ^^^^ > /sys/class/gpio/export"           # Create GPIOs.

while sleep 0.$((RANDOM%9)) # The delay between blinks is defined here.
do peat 'echo out           > /sys/class/gpio/gpio^^^^/direction'
   peat 'echo $((RANDOM%2)) > /sys/class/gpio/gpio^^^^/value'
done

#########################################################################
# Notes section:                                                        #
#########################################################################
# It's an infinite loop, so exit with control-C or some equivalent. Any #
# stderr warnings can be ignored if they stop quickly and the GPIO pin  #
# works. The culprit can be races with udev rules or pre-existing pins. #
#########################################################################
# If you use this script as a template for something else, please read  #
# this note carefully. When you echo to "export", lower-level code such #
# as udev rules trigger and run in parallel to your code. This race     #
# condition can be harmless, but in some situations it can also cause   #
# sporadic, hard-to-recreate failures. This script avoids the problem   #
# by echoing to direction before every echo to value, but that trick    #
# only works because any failures would soon be followed by another     #
# attempt. If you adapt this code to a situation where direction or     #
# value are only set once, then your new code will *not* be protected.  #
#########################################################################
# When testing changes to this script, swap all ">" with "\>", or try   #
# running: cd /sys/class/gpio && watch -n .1 'grep . gpio*/value || ls' #
#########################################################################
# The name "blinkys_fun_club.sh" refers to a 1960's Colorado TV show.   #
#########################################################################
# Tested in 2016 on a Raspberry Pi 3 and a Zero, both using Raspbian.   #
#########################################################################
# Yes, yes, "eval" is evil. You're right, I'm a bad coder for using it. #
# Self-modifying code is an anachronism, like two spaces after periods. #
# If encumbered by a modicum of sanity, replace it with a loop.  Wimps. #
#########################################################################
  • This is a good answer but could be better with addition of the code you talk about in your bottom paragraph. – sir_ian Feb 16 '17 at 19:10
  • Yep, sleep was the preferred solution in my case too but watch out for scenarios when system becomes unresponsive and things don't go smoothly as planned. – Abhinav Gauniyal Feb 17 '17 at 7:18
0

As already mentioned, the problem is that the export happens in parallel and the files might even be created with the correct permission but they may not respond even to successful file operations.

Add to this that you can't change pullups using gpio sysfs, why even bother using it? There's only one really good reason, if you want want input pin change interrupts. If you are just doing output, you might as well just use gpiomem (memory mapping the gpio controller) and bopping the bits yourself. Ok, that's not portable, but it is efficient.

#define GPIOBASE 0x20200000
#define GPIOSIZE 4096

fd=open("/dev/gpiomem", O_RDWR|O_SYNC);
gpio=mmap(NULL, GPIOSIZE, PROT_READ|PROT_WRITE, MAP_SHARED, fd, GPIOBASE);
close(fd);
// Sorry, read BCM2835 Peripherals manual to see which bits to bop

For a biphase encoder knob, I set direction, pullup using gpiomem, then export, sleep some, set edge to "both" and get an fd for value to use in a poll().

  • Could you please add some example code to your solution? It is currently too vague. – David Apr 5 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.