systemd user service shows status "dead" on boot, but runs when manually restarted

Question

I'm writing some code to run a sound/light show from a raspberry pi 3 A+, using the latest Bullseye OS. I've been testing it manually, running via ssh, and all is well. However, it needs to run headless and unattended, so now I'm trying to turn it into a systemd service, and can't get it to start properly on reboot.

At first, I tried it as a system service. But because it uses pulseaudio, which runs in userland, it was failing to connect to pulse - even with the User= and Group= options set. After much googling, it seems that common wisdom is to not run pulse system-wide, and instead run the script as a user service. That's fine by me, as it makes the runtime environment more like my testing environment. (Raspi-config is set to auto-login my user.)

I installed it in .config/systemd/user, with the following unit file, and enabled it with systemctl --user enable lpr

[Unit]
Description=LPR Service
After=network-online.target

[Service]
Type=idle
ExecStart=/home/meeotch/test/mqlighting.py
WorkingDirectory=/home/meeotch/test
StandardInput=null
StandardOutput=file:/home/meeotch/logs/mqlighting_log.txt
StandardError=file:/home/meeotch/logs/mqlighting_log.txt
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

For testing, I stripped down the actual script to a minimal:

#! /usr/bin/env python

import sys, os
import csv
import subprocess

print("FOO", flush=True)
audio_dir = "/home/meeotch/audio"
silence = os.path.join(audio_dir, "750-milliseconds-of-silence.mp3")

while 1:
    pass

But when I reboot the pi, give it some time, then log in via ssh, I see the log file doesn't exist. And systemctl --user status lpr gives:

● lpr.service - LPR Service
     Loaded: loaded (/home/meeotch/.config/systemd/user/lpr.service; enabled; vendor preset: enabled)
     Active: inactive (dead)

grep -i lpr syslog shows nothing interesting. It's just dead, and never wrote anything to the log file. But systemctl --user start lpr works fine if I start it manually - the log file gets written, status shows as "running".

I've also tried with After=multi-user.target, but no love.

Any suggestions as to how I can figure out why it's dead, and not restarting on boot?

UPDATE: well, my several hours of googling seem to have not turned up a basic fact. From this thread: https://bbs.archlinux.org/viewtopic.php?id=250813 I discovered that neither multi-user.target nor network-online.target are user targets.

Swapped out the After= and WantedBy= to "default.target", and it seems to be working. However, it failed for another startup script that also uses networking (mqtt), with "OSError: [Errno 101] Network is unreachable".

So I guess now I have a follow-up question: the systemd docs say, "When a systemd user instance starts, it brings up the per user target default.target." It seems that this doesn't guarantee network services.

So what's the appropriate "late" target to use, when the service in question needs network, dbus, and other stuff? Ultimately, I'd like to turn off the GUI altogether, so graphical-session.target doesn't seem like a good choice.

UPDATE #2: So I discovered how to get a graphical representation with systemd-analyze plot > boot_analysis.svg, which was pretty helpful. The last few targets to run are: bluetooth, multi-user, and graphical. Since multi-user isn't available to a user service, and graphical doesn't run when I switch the boot to "Console with Login" (to disable the window manager and stuff I don't need), I chose bluetooth.target for the second service that was failing with "Network is unreachable". And now that service seems to work, too.

One weird thing about systemd-analyze plot is that it doesn't seem to show either of my services. I guess maybe it only shows system services, but also user-managed targets?

Answer 1

You can get a list of possible targets with

systemctl list-units --type target --user

However, systemd probably isn't the best way to start things in a user session. It might be better to add it as a startup item in your desktop environment.

Answer 2

As indicated in the updates to my original post, I've got a scheme that works. As described by user10489 in his comment below, there are two choices:

1. auto-login user, use user services (possibly using enable-linger to prevent them from dying if the user gets logged out for some reason).
2. use system services with User= set appropriately, including a service that starts a pulseaudio session for the user.

I'm going with #1 until something else breaks. Further info on #2 is available here: https://forums.raspberrypi.com/viewtopic.php?p=2080466#p2080466 https://forums.raspberrypi.com/viewtopic.php?p=2078308&hilit=pulseaudio#p2078290