0

I am running Jesse lite with MATE gui on a pi zero w. I am currently running a python program that scrapes data from 6 different web sites one after the other starting at the same time every day on a pi3. I would like to switch this task over to the pi zero w and haven't installed a browser yet.

  • Is the pi zero capable of reliably performing such a task daily?
  • Are there any browsers that work better than others on the pi zero w?
Improve this question
5
  • Why don't you just try it? Why do you think it won't work? Why do you need a browser to web scrape?
    – Steve Robillard
    Feb 19, 2018 at 2:17
  • 1
    Instead of installing/uninstalling multiple different browsers I thought this may be a good forum to learn from others knowledge and experiences. I think it will work but could time out and kill my running script if it's too slow. I was not aware that a browser was not required and will have to look into the curl command as mentioned in the answer. I have no formal programming training and am appreciative of any help in the learning process.
    – wanderweeer
    Feb 19, 2018 at 12:58
  • 1
    I think the question is reasonable insofar as you seem locked into using a browser. I agree with @SteveRobillard and the current answer, however, that you should typically need a browser to scrape pages. From the command line, curl will work in many cases. If you need to emulate a browser (like you need to login to the page and keep track of cookies), several popular scripting languages have libraries for that. You don't seem locked into a specific browser, so maybe consider making the switch to no browser when you change hardware.
    – Brick
    Feb 19, 2018 at 14:21
  • 3
    The answer to your question is going to depend on the details of your script. If you're using something like beautifulsoup in your script, you should not need any desktop browsers. If you've somehow tied python to a browser, the best answer might be "there's a better way". You can definitely write web scraping scripts in python, and you can definitely run python scripts on the RPi Zero. The Zero can certainly handle the task, although it may not be particularly fast at it.
    – bobstro
    Feb 19, 2018 at 15:54
  • @bobstro: I'm using beautifulsoup and requests. I see now that I don't need to open a browser with requests and that solved my problem.
    – wanderweeer
    Feb 19, 2018 at 16:50

1 Answer 1

Reset to default
4

As you can see from the comparison below, the main differences you need to worry about is the ram, and the number of cores. Depending on the data you are trying to scrape, you don't need a browser. The curl command can return text data from a web site.

Time is really going to be the big issue. If the process normally takes lets say one minute on your pi3 it may take a lot longer on your zero. But if its only running once a day and it now takes even 10 minutes, this wont be an issue.

The zero is also only a single core(vs 4 on the pi3) This makes running programs that are multi threaded (like browsers) not work very well or are very slow.

Unless you add an ethernet port to the zero you will be using wireless which will also make your data collection slower.

enter image description here

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.