How to make a service to run a python script which includes browser automation

Question

I have python script which is using selenium web driver to do some web automation task. This python code is running perfectly fine when I am running it from the terminal. It launches firefox and does what is required.

I have made a service which looks like below:

[Unit]
Description=Start python web browser code

[Service]
Type=simple
ExecStart=/usr/bin/python3.5 /home/pi/web_automation.py
Restart=on-failure
RestartSec=30s 
StandardOutput=null

[Install]
WantedBy=multi-user.target

When starting this service, it always remains in the failed state. I have also tried editing the /home/pi/.config/lxsession/LXDE/autostart/ and include below lines

@/usr/bin/python3.5 /home/pi/web_automation.py

But it's still not working. How can I make it work. Thanks.

Answer 1

Haven't just a RasPi with Monitor by hand to test it but here are some pointers as far as I can see to "fix" your unit.

First you should delete the line StandardOutput=null. Default Standard Output in a service is the journal. Redirecting it to /dev/null will suppress any (error)messages from your script. Otherwise you will see what's going wrong. That may help:

rpi ~$ systemctl status web_automation.service
rpi ~$ systemctl journalctl --boot --pager-end --unit=web_automation.service

The next is that firefox needs a graphical user interface so it must be WantedBy=graphical.target. It also needs a DISPLAY environment variable to find its output.

Because your script runs in a console from a logged in user, I guess it is pi, you should for a first try run the script also with this user. If it runs then you can optimize it for another user. Default is root. Does your script run as root?

Putting it together I would see this unit (Type=simple is default):

[Unit]
Description=Start python web browser code

[Service]
User=pi
WorkingDirectory=/home/pi
Environment=DISPLAY=:0
ExecStart=/usr/bin/python3.5 /home/pi/web_automation.py
Restart=on-failure
RestartSec=30s

[Install]
WantedBy=graphical.target

Answer 2

Your selenium job requires launching Firefox, which I don't believe will be possible when you run it as a service.

You might be able to save this by changing browsers from Firefox to the HtmlUnit "browser" supported by selenium. That has no graphical interface so it seems like it might work. I'd do further research on it (or a small test) before you try to convert any big script to use it. I have not used selenium recently, but my recollection is that there were some issues with HtmlUnit where it didn't support a full range of real-browser capabilities. That may or may not still be true.

The other alternative would be to leave selenium and do something in pure scripting. If the webpage that you want to hit is not complicated, then you can do a lot with the command-line tool curl called from a scripting language, sometimes even bash but more likely Ruby, Perl, or (if you must) Python. I also used a Ruby module to interact with a more complicated page once, but I can no longer remember the module name. You could probably search and find it (or the equivalent in a different language). These differ from selenium in that they are not web drivers, they emulate the key interactions with the webpage internally. This means they don't need to launch something external to themselves, which might make them more suitable for running as a service.