How do I get Python 3 to work again on my Raspberry Pi?

Question

I am running Raspbian on a Pi 3 that has one job: it runs a clock on a Sense Hat, from Python 3 source code. Recently I found I was unable to run my clock program, which generated an error stack that ended this way:

  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1467, in exec_module
  File "<frozen importlib._bootstrap>", line 1570, in get_code
  File "<frozen importlib._bootstrap>", line 656, in _compile_bytecode
EOFError: marshal data too short

I found a number of possible solutions, none of which worked. Any attempt to uninstall/reinstall modules with pip3 generated the same error stack. I read instructions to delete all the .pyc files in the Python3 tree and recompile them, but after that any attemtp to load a module just gave me

"defining __gutstate__ cannot be picKled")

and I was unable to recreate the .pyc files.

Essentially everything I did that involved touching Python files, whether through apt-get or pip3, failed on some Python error, either the "marshal data too short" or the "cannot be picKled". I can't even remove or purge Python3 itself - I get Python errors that won't let me reinstall Python.

Is there a solution for this? My clock actually runs under Python 2.7, but I'd like to be able to run a supported version of Python if possible. I could reinstall Raspbian but there's no guarantee it wouldn't fail the same way and put me right back where I am.

Answer 1

I found the suggestion to delete pycache in another thread and deleted all the pycache directories in the /usr/lib/python3 and /usr/lib/python3.4 directories. All that happened was that python3 stopped working completely and I had to restore the system from backup. I also tried apt-get purge python3, and numerous components failed to uninstall with exactly the same "marshal data too short" error that happened when I tried to load a library. In fact, the whole stacktrace was identical:

Traceback (most recent call last):
  File "/usr/bin/py3clean", line 24, in <module>
    import logging
  File "/usr/lib/python3.4/logging/__init__.py", line 26, in <module>
    import sys, os, time, io, traceback, warnings, weakref, collections
  File "/usr/lib/python3.4/traceback.py", line 3, in <module>
    import linecache
  File "/usr/lib/python3.4/linecache.py", line 10, in <module>
    import tokenize
  File "/usr/lib/python3.4/tokenize.py", line 29, in <module>
    import collections
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1467, in exec_module
  File "<frozen importlib._bootstrap>", line 1570, in get_code
  File "<frozen importlib._bootstrap>", line 656, in _compile_bytecode
EOFError: marshal data too short

Answer 2

You should delete all the __pycache__ directories in any python library directories that are part of your program, including your program directory. So if you import requests then delete the __pycache__ for in the main requests lib folder (should be something like /usr/lib/pythonX.X/site-packages/requests)

It would seem the .pyc compiled files have been automatically updated with some corrupt data.