-1

I want to run a python program that receives the data through serial communication and saves it to a file, but after receiving the data this error shows up. Any idea? (code runs ok in Windows but in Raspbian OS I have this error)(the data is in JSON format)

error:

Traceback (most recent call last): File "/home/pi/Desktop/ReadFromPort.py", line 11, in line = ser.readline().decode("utf-8") UnicodeDecoderError: 'utf-8' codec can't decode byte 0xbd in position 2: invalid start byte

Python code:

from serial import *
from serial.tools import list_ports
import os

ser = Serial('/dev/ttyS0',115200, timeout = 1)


with open("output1.txt","w",encoding="utf-8") as file:
    while True:
        line = ser.readline().decode("utf-8")
        if(line != ""):
            #print(line)
            file.write(line)
            file.flush()
            os.fsync(file)
5
  • what happens if you remove the .decode("utf-8")?
    – jsotola
    Commented Dec 12, 2021 at 18:53
  • @jsotola file.write(line) TypeError: write() argument must be str, not bytes
    – Pouya
    Commented Dec 12, 2021 at 19:44
  • Not Pi specific. This is a general coding question.
    – Milliways
    Commented Dec 12, 2021 at 21:11
  • @Milliways My problem is on Pi. no problem in windows :)
    – Pouya
    Commented Dec 12, 2021 at 21:14
  • UTF-8 is the native encoding in RpiOS. The problem here is your input is either not UTF-8 or the data is getting corrupted.
    – goldilocks
    Commented Dec 12, 2021 at 22:24

2 Answers 2

3

File is not in utf8 format, so the decoder is seeing an invalid (i.e., non-utf8) byte sequence. 0xbd is the ½ character in Latin-1, utf8 represents it as the two-byte sequence \xc2\xbd

2
  • so what is the solution to this problem do you think?
    – Pouya
    Commented Dec 12, 2021 at 19:47
  • You need to debug the serial communication, maybe there's a baud rate mismatch or some interference causing invalid characters to be received. Alternatively the sending device is sending stuff you're not expecting.
    – Bra1n
    Commented Dec 12, 2021 at 20:04
0

Type of ser.readline() is bytes.

bytes.decode(encoding='utf-8') return a str if and only if there is not a UnicodeError risen. It is this in fact that appens in your line

line=ser.readline().decode('utf-8')

So, you must give a good value to the encoding parameter.

You'll find the different encoding here. If it is a latin-1 (for instance), you must have the following code :

with open("output1.txt","w",encoding="utf-8") as file:
while True:
    line = ser.readline().decode("latin-1")
    if(line != ""):
        #print(line)
        file.write(line)
        file.flush()
        os.fsync(file)

To improve code quality, you can take avantage of catching exceptions ;)

4
  • thanks, but the data have some Persian character, so i should use utf-8 i guess.
    – Pouya
    Commented Dec 12, 2021 at 20:53
  • Might help if you indicated where the data is from because chances are your problem is it isn't in UTF-8. How do you know it contains Persian characters?
    – goldilocks
    Commented Dec 12, 2021 at 22:26
  • @goldilocks My Raspberry Pi zero (Raspian OS) which has a python program receives the JSON data (indicates Persian character) through serial communication (GPIO-UART) from Raspberry Pi 4 (8 gig ram model) (Windows 10 IoT Enterprise - C# app)
    – Pouya
    Commented Dec 14, 2021 at 8:07
  • Well, UTF-8 isn't the native format used by Windows, it's UTF-16, although I am sure it is not that either (since they are 100% incompatible, ie., it would be wrong from the first byte or two). So you might try and confirm whether it is supposed to be utf-8, or something else.
    – goldilocks
    Commented Dec 14, 2021 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.