How to make my raspberry pi download my files at specific time?

Question

So I want to save some power and let the raspberry pi download my files instead of my PC which is a power monster

I want to start my downloads at 3am and pause them at 7am. I tried to use crontab but it never works

How can I do this :?

I've written a python script to do this for me but I'm not sure how to always execute it at startup or leave it running because I only use SSH. I'm not sure if it's ok to always have this script running in terms of cpu usage or other safety stuff! (it just executes wget and sends terminate signal if its 3am or 7am, I have a file on desktop which i add my links too thought ssh)

I have archlinux installed on it!

here's my crontab which doesn't work:

0 3 * * * wget -c -i /home/mename/Desktop/downloads.txt
0 7 * * * killall wget

edit:

still doesn't work, i made this job * * * * * echo "WAIT WHAT?" > /home/alarm/Desktop/Iscromwell.list adding this to SU works (running crontab -ewith sudo) isn't this dangerous :? why doesn't it work as a normal user!?

Edit:

I moved this to Linux section as this is more Linux related now. Things got serious! I have no cron service running and i get an error trying to run it

Answer 1

You have to write a cron job to do that task.

1. Write crontab -e in your terminal to access crontab
2. Then follow below image to write a cronjob.

In the you have to write source\to\python file.py to execute at a given time.

For more information refer this link.

Answer 2

Try a python (.py) file. Running it through Python should work if you can get the correct modules to be able to download them. The language is very easy and shouldn't be much of a task to learn.

Update:

Should have probably read all of your post first. Once you get connected to the Pi via SSH, you should be able to do python pythonscript.py. If you don't have it in the default directory (/home/pi), you'll need to do cd /home/pi/... and so on. I would also suggest implementing SMTP which allows you to email the files to yourself, which you can do by doing a check to see if they have completed or not. You could also add a crontab where on startup or reboot, the python file executes automatically.

Answer 3

I suggest invoking wget this way:

wget -b -v -c -i /home/mename/Desktop/downloads.txt -o /tmp/wget.log

instead of the way you are doing it. The extra switches, from man wget, are:

  -b
  --background
       Go to background immediately after startup.

That is probably the critical one, if the problem is solved.

   -o logfile
   --output-file=logfile
       Log all messages to logfile.  The messages are normally reported to standard error.

And finally:

   -v
   --verbose
       Turn on verbose output, with all the available data.  The default output is verbose.

Which seems redundant since that's the default, but sometimes I am dubious with wget (and why would they have a switch to turn on a default feature?).

There is also -d if you still aren't getting anything in /tmp/wget.log. Beware that /tmp is a volatile directory in RAM on most systems, so if you turn the pi on and off you will lose that information.

Answer 4

"I've written a python script to do this for me but I'm not sure how to always execute it at startup"

If this is your only problem, then you can always write a line in your crontab to specify that it should run a specific program at boot time like this:

@reboot /home/user/download.py

This runs the program as the user, who owns the crontab.

Answer 5

I'm no bash ninja but I just wrote this and it seems to work. Edit the cron entry line and the wget command as needed. Add everything to a shell script and run it.

if [ -f /home/$USER/cron_downloads/crontab_$USER.bak ]
then
    echo Already added cron entry. Exit to avoid duplicate cron entries
else
    echo The file does not exist. Continue...

    # make a new directory called cron_downloads to avoid 
    # overwriting existing scripts in home dir
    mkdir /home/$USER/cron_downloads

    # enter the new dir
    cd /home/$USER/cron_downloads

    # write shebang to a new script
    echo '#!/bin/bash' > download_script.sh

    # append wget command to the new script
    echo "wget -c -i /home/$USER/cron_downloads/urls_to_fetch.txt -o /home/$USER/cron_downloads/download_script.log" >> download_script.sh

    # set executable bit
    chmod +x download_script.sh

    # add a URL to the file that's read by wget
    echo "http://example.com" >> urls_to_fetch.txt

    # backup existing user crontab
    crontab -l > crontab_$USER.bak

    # copy backup so we can safely add entries and leave the backup untouched
    cp crontab_$USER.bak crontab_$USER.tmp

    # add a new entry to the temporary crontab. Set schedule as needed
    echo "* * * * * /home/$USER/cron_downloads/download_script.sh" >> crontab_$USER.tmp

    # replace existing crontab with the contents of crontab_$USER.tmp
    crontab crontab_$USER.tmp

    # remove the temporary crontab. Leave the backup
    rm crontab_$USER.tmp

fi

To undo all changes to cron (leaving the newly created directory intact):

# undo changes
crontab /home/$USER/cron_downloads/crontab_$USER.bak

• Add download urls (one per line) to /home/$USER/cron_downloads/urls_to_fetch.txt
• Wget activity is logged to /home/$USER/cron_downloads/download_script.log

Output:

I placed the shell commands in a script called add_cron_download_entry.sh

pi@rpi ~ $ sudo chmod +x add_cron_download_entry.sh 

pi@rpi ~ $ sh add_cron_download_entry.sh 
The file does not exist. Continue...

pi@rpi ~ $ while :; do ls /home/pi/cron_downloads; sleep 2; done
crontab_pi.bak  download_script.sh  urls_to_fetch.txt
crontab_pi.bak  download_script.sh  urls_to_fetch.txt
crontab_pi.bak  download_script.sh  urls_to_fetch.txt
crontab_pi.bak  download_script.sh  urls_to_fetch.txt
# BELOW WE SEE THAT WGET CREATED A LOG
crontab_pi.bak  download_script.log  download_script.sh  urls_to_fetch.txt

pi@rpi ~ $ cat cron_downloads/download_script.log 
--2016-03-16 17:57:02--  http://example.com/
Resolving example.com (example.com)... 93.184.216.34, 2606:2800:220:1:248:1893:25c8:1946
Connecting to example.com (example.com)|93.184.216.34|:80... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

    The file is already fully retrieved; nothing to do.

Answer 6

My crontab service wasn't even running, I used systemctl and fixed it :) systemctl start cronie.service