5

One of my raspberry pi's is hanging whenever I try to install or update anything using apt-get.

Running Raspbian 7 (wheezy).

Symptoms (using watchdog as an example as it has no dependencies):

# apt-get install watchdog
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  watchdog
0 upgraded, 1 newly installed, 0 to remove and 129 not upgraded.
Need to get 0 B/81.7 kB of archives.
After this operation, 209 kB of additional disk space will be used.
Preconfiguring packages ...
Selecting previously unselected package watchdog.
(Reading database ...

... at which point it hangs. Ctrl-C doesn't even work to kill it; I've been using Ctrl-Z and then kill %1 to get rid of it.

By hanging, I mean nothing happens for over 24 hours wall clock time.

Running with strace ends with the last 2 lines endlessly repeating:

wait4(6254, 0xbee07570, WNOHANG, NULL)  = 0                                   
pselect6(18, [0 15 17], NULL, NULL, {1, 0}, {[], 8}) = 1 (in [17], left {0, 448307553})                                                                     
read(17, "Selecting previously unselected "..., 1024) = 49                    
write(1, "Selecting previously unselected "..., 49Selecting previously unselected package watchdog.) = 49                                                   
write(4, "Selecting previously unselected "..., 49) = 49                      
wait4(6254, 0xbee07570, WNOHANG, NULL)  = 0                                   
pselect6(18, [0 15 17], NULL, NULL, {1, 0}, {[], 8}) = 1 (in [17], left {0, 999814003})                                                                     
read(17, "\r\n", 1024)                  = 2                                   
write(1, "\r\n", 2^M                                                          )                     = 2                                                     
write(4, "\r\n", 2)                     = 2                                   
wait4(6254, 0xbee07570, WNOHANG, NULL)  = 0                                   
pselect6(18, [0 15 17], NULL, NULL, {1, 0}, {[], 8}) = 1 (in [17], left {0, 999947001})                                                                     
read(17, "(Reading database ... \r", 1024) = 23                               
write(1, "(Reading database ... \r", 23(Reading database ... ^M) = 23         
write(4, "(Reading database ... \r", 23) = 23                                 
wait4(6254, 0xbee07570, WNOHANG, NULL)  = 0                                   
pselect6(18, [0 15 17], NULL, NULL, {1, 0}, {[], 8}) = 0 (Timeout)            
wait4(6254, 0xbee07570, WNOHANG, NULL)  = 0                                   
pselect6(18, [0 15 17], NULL, NULL, {1, 0}, {[], 8}) = 0 (Timeout)            
wait4(6254, 0xbee07570, WNOHANG, NULL)  = 0                                   
pselect6(18, [0 15 17], NULL, NULL, {1, 0}, {[], 8}) = 0 (Timeout)            
wait4(6254, 0xbee07570, WNOHANG, NULL)  = 0                                   

Top indicates the pi is 98% idle (the 2% active is basically the top command itself). This is a headless pi not running X so has plenty of free memory. Status lines from top (during a "hang") are:

Tasks: 104 total,   1 running, 103 sleeping,   0 stopped,   0 zombie
%Cpu(s):  1.0 us,  1.3 sy,  0.0 ni, 97.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0
KiB Mem:    184900 total,   158188 used,    26712 free,    28336 buffers
KiB Swap:   102396 total,        0 used,   102396 free,    67240 cached            

There is sufficient free disk space. df -h gives:

Filesystem                    Size  Used Avail Use% Mounted on
/dev/root                     3.4G  2.5G  757M  77% /                         
devtmpfs                       87M     0   87M   0% /dev                      
tmpfs                          19M  264K   18M   2% /run                      
tmpfs                         5.0M     0  5.0M   0% /run/lock                 
tmpfs                          37M     0   37M   0% /run/shm                 
/dev/mmcblk0p5                 60M   21M   39M  36% /boot           

Things I have tried which don't make any difference:

  • rpi-update (Suggested by a similar issue here)

  • apt-get update (which works) followed by apt-get upgrade (which hangs)

  • Copying the status-old file over the existing status file in /var/lib/dpkg

  • apt-get install -f (hangs)

  • dpkg --configure -a

  • dpkg --clear-avail

  • 1
    I've just tried running with strace; results added into question. – nickcrabtree Mar 25 '17 at 7:38
  • You may need to go manually delete the cache files and databases and pull fresh. Probably bit flipped on the DB... ouch. Can we get ZFS soon? – Piotr Kula Mar 27 '17 at 11:18
2
+50

This is what I would do:

  1. Exclude filesystem issues by running fsck and fixing errors if any

  2. Run apt-get clean and apt-get autoremove followed by an update/upgrade

  3. Backup&remove /var/cache/apt and /var/lib/apt/lists, and run update/upgrade to recreate them. You may need to manually recreate the lock file and some directories.

  4. Look though your strace log to find out which file it tries to access and backup&remove that file to see if it can be recreated. Depending on which file it is, manually fixing the file may be an option.

Having said that, broken package databases can be a real mess. It already happened to me to give up, back up my files and reinstall. It will be an opportunity to upgrade to Jessie, too.

  • 1
    Thanks for the pointer in point 4. The trick was using strace -f to follow the forked subprocesses - it turned out dpkg was hanging trying to read /var/lib/dpkg/info/python3.list. Deleting that file allowed the installation to complete. There are now warnings that files lists are incomplete but I expect re-installing the offending 8 packages will get everything working again. – nickcrabtree Mar 27 '17 at 20:22
0

some of this are CPU / IO intense processes. Open a second terminal and check if CPU is on 100%. Try to shut down some of the processes that consume high CPU. There can be different reasons why the system becomes unresponsive. Maybe you can find an hint in the logs.

tail /var/log/apt/

Sometimes people recommend nice

sudo nice apt-get install watchdog

Sometimes this hangs are traced back to memory issues. An indication can be if the CPU is not at 100%. The system seems to be unresponsive while swapping memory to SD card. Also check if there is sufficient space on your SD Card.

free -m

can be an option to get some more memory during the apt-get process. If you run your raspberry without GUI try to reduce the memory allocated for graphics.

  • just take a look to this question. – Joe Platano Mar 21 '17 at 2:20
  • I don't think that is my problem. This system has plenty of free memory and disk space, and is showing as 98% idle. I've updated the question with more info. – nickcrabtree Mar 21 '17 at 6:41
  • there is no information in your log files? – Joe Platano Mar 24 '17 at 23:28
  • No; there are 2 log files in the /var/log/apt/ directory, history.log and term.log, but neither contains any further information to that output on the terminal – nickcrabtree Mar 25 '17 at 6:55
  • are you running your pi on IPv4 or IPv6? might be a approach to uncomment precedence ::ffff:0:0/96 100 in /etc/gai.conf. take care to uncomment the correct line... some looks almost identical – Joe Platano Mar 25 '17 at 7:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.