1

Dear community, I'm almost at my wit's end and would be so happy to receive help. It would be a great Christmas present to me if some of you take the time to read and provide inquiry, knowledge or even a solution. I tried to structure it as neatly as possible. Happy about any help!

I'll update the information to my problem if I missed something or upon your request.

My Setup:
I'm using a Simulink model created with MatLab 2017b.
It's being transferred to and built on a Raspberry Pi 3B running Raspbian Buster
via SSH wirelessly over local network.
The second to fourth and otherwise identical setups are running Raspbian Jessie.
Connected to the RasPi are two circuit boards using TWI / I2C protocol. One controls two motors and the voltage (from a battery pack) for the RasPi and the other board. This second one provides data from two light sensors.
(The boards are controlled by an STM32 microcontroller, the firmware for which was written by a colleague.)

My Problem:
The whole setup (using Jessie) was working just fine. Some random day it stopped working.
At first it just hung-up presumably when there was more data transfer than possible with the set bus frequency.
Then it stopped seconds after begin of successful transmission.
Now it doesn't respond whatsoever. The Simulink model still builds but there's no action happening.

Edit: using another fresh OS-build I got it to work for now.

Debug attempts:
Using MatLab's Log signal data and reading from the log tells me: EXT_I2C_write/write: Remote I/O error

Trying to debug using some python code with the smbus package gives the folloving error message:

Traceback (most recent call last):
  File "Downloads/read_sensor.py", line 14, in <module>
    value1 = read_line_value(i2cbus,i2caddress,0x01)
  File "Downloads/read_sensor.py", line 6, in read_line_value
    return bus.read_word_data(dev, addr)
IOError: [Errno 121] Remote I/O error

(As the script doesn't contain a lot more than what's seen in the error message, I didn't include it here. dev, addr, i2cbus and i2caddress are simply storing address values.)

Falling back to linux command line:
i2cdetect is working just fine, showing devices (on 0x11 and 0x19)
i2cset seemed to work (or at least didn't output an error message).
i2cget returns Error: Read failed

Steps I tried:

  • reinstalling i2c-tools
  • setting up the whole system from zero
    • reinstalling RaspbianOS (updated to Buster) + packages necessary for MatLab = MathWorks' Raspbian version 1
    • configuring network and the like, enabling i2c
  • taking a first look at it with an oscilloscope. Not being used to using that I can, as of now, only tell:
    • i2cdetect shows a signal - this behaviour matches with the successful result
    • i2cset and i2cget don't seem to cause a signal, but it appears to cause a short lag / delay in the signal

Edit: Checking up on the firmware implementation...

Further...
I found a couple somewhat similar questions/threads but nothing that helped me. Some of these include:

  • problems with the wiring
    • as I have four setups all showing similar problems I don't think that's the reason. Also it doesn't explain why some commands should work while others don't
    • though I don't quite understand / nowhere read how that's supposed to cause the problem
    • Edit: using another OS-build I got it to work for now and checked the cables and sensor-boards one by one. All are working. So there's no general problem with the hardware.
  • in another seemingly similar case [2] it's recommended to activate the repeated start condition
    • summary: create file /etc/modprobe.d/i2c.conf and write options i2c_bcm2708 combined=1 to it
    • I tried that, rebooted, does not work in my case, i2cdump still shows XX everywhere, so I removed it again.
    • indicating to use this option is a missing STOP signal. I'll check for that with the oscilloscope soon.
    • I'm not too fond of specifying some option I don't understand* and don't know how it interacts with other configurations, especially when "simply creating file XY" in some directory*, without afore mentioned knowledge... So if someone has detailed information or links for further reading regarding how the different config-files in different directories interact, I'd be happy about that, too. (* see links [2a-c])

Generally I didn't come across a lot of posts explaining exactly why and how the proposed method causes the failure or interacts with other configurations etc.

Sources:
1 https://github.com/mathworks/Raspbian_OS_Setup
[2a] https://stackoverflow.com/questions/44550692/i2c-tools-i2cdetect-and-i2cset-works-perfectly-but-i2cget-returns-read-failed
[2b] https://rabbit-note.com/2015/02/15/raspberry-pi-i2c-repeated-start/
[2c] https://rabbit--note-com.translate.goog/2015/02/15/raspberry-pi-i2c-repeated-start/?_x_tr_sl=ja&_x_tr_tl=de&_x_tr_hl=en&_x_tr_pto=sc

Images:
RasPi and connected boards with wires I hope this picture makes something clearer. On specified request I can take more detailed pictures.

5
  • The error suggests a connection/wring problem. Please provide clear photos showing the I2C wiring at the Pi end and the device end.
    – joan
    Dec 15 '21 at 22:10
  • some option I don't understand - what option? "simply creating file XY" in some directory, without afore mentioned knowledge what does this even refer to?
    – Bravo
    Dec 16 '21 at 0:50
  • @Bravo The former is meant generally. I like to understand what I'm doing and not blindly following some steps. - The latter refers, as stated in the post, to Link [2]
    – Nep
    Dec 16 '21 at 10:21
  • I added a picture and further clarification. Thanks in advance for advice
    – Nep
    Dec 16 '21 at 18:52
  • If i2cdetect shows the two devices then the comms seem ok because the devices have sent back ack responses. I would check with the person writing the microcontroller code that it actually implements the form of the read function you are calling. It looks as though the problem is at the microcontroller end to me. Dec 17 '21 at 10:21
1

You've asked a good and detailed question. Unfortunately, it seems unlikely that anyone here is going to divine a silver bullet solution that will resolve your problems. That should be no cause for despondence, but rather to take a different tack to resolve your issue(s) - and please believe me when I say that there is a solution.

The different tack is to adopt a logical and systematic troubleshooting effort; i.e. you must consider all potential sources of failure, and go about eliminating them one-by-one through a sequence of tests and measurements.

Reading your question, it sounds like you have devoted most of your effort to software-related causes. Without trying to be rude, I feel obligated to ask, "Why do you suspect software?" Your question states, "Some random day it stopped working" - which sounds like there were no software changes that would have triggered the malfunction. Am I missing something?

Based on the information provided in your question, I would lean very much toward an initial diagnosis of a hardware or wiring issue. I say this even if the hardware had never changed either. Why do I say this? Because a change in external environment (e.g. a new source of EMI) can create interference; because wiring and connections (crimp joints, solder, etc) can be moved in subtle ways that will create issues, and of course hardware can fail.

Raspberry Pi was designed to serve hobbyists and students with an inexpensive platform for learning and experimentation. In my experience, low cost and interconnect integrity are not generally both residing in the same hardware. Ultimately, one pays the price for this through fussy hardware - especially in a case such as yours with what sounds to be a fairly elaborate set of interconnects.

As far as a suggestion on where to start, I wonder if reducing your I2C clock frequency might be worthwhile? If so, I'd also suggest reducing the clock to the lowest frequency supported by your hardware. I say this because in troubleshooting, it can be productive to make large changes for maximum effect as they may expose issues that smaller changes won't reveal.

I think you also should make an effort to overcome your inexperience with basic instrumentation - specifically the oscilloscope. They are invaluable for situations like the one you've described.

6
  • thank you a lot for taking the time for such a large reply. I'll have the time for a detailed answer on monday
    – Nep
    Dec 18 '21 at 15:05
  • Thanks again for your time and reply. I'm happy I did well on the question. That silver bullet - to my werewolf I suppose ;-) - would be great but unlikely, considering the unique composition of such projects. What I'm searching for realistically is a road sign to where to look. Gaining experience is partly the reason for my assistant job and working on this side project. Reading your answer, drafting my reply and thinking about it I did find some steps I've overlooked or to do much more systematically. So thanks already!
    – Nep
    Dec 21 '21 at 9:17
  • For why I don't suspect hardware: I didn't move anything, the connectors appear quite robust; it's not apparent to me what would have caused it. To my understanding an error would mainly occur when cable breaks or contacts don't touch right anymore. Also I found this speculation in other posts, but no explanation reasoning by principle. You mentioned they could have moved slightly. Would this cause it to work/stop sometimes and sometimes not? This happened even when stationary. I would understand if it stopped transmitting completely...
    – Nep
    Dec 21 '21 at 9:20
  • The newest progress seems to support it not being hardware related: Testing with a different and fresh OS I got it to work for now and tested cables and sensor boards one by one and found it to be working so far. Thanks for your suggestions of considering EMI and lower I2C frequency. I'll use the latter for debugging in the future and try to find out about the former.
    – Nep
    Dec 21 '21 at 9:21
  • @Nep: After designing, developing, building and testing electronic hardware for many years, I have seen the sort of hardware faults I described occur rather frequently. Only trying to give the benefit of that experience here, but of course you must find your clues & follow them.
    – Seamus
    Dec 21 '21 at 9:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.