Python requests sometimes fails with Errno -2

Question

I'm writing a scraper which runs perfectly on my Ubunutu 13.04 based laptop, however when running it on my RaspBMC based pi it throws the error below. Sometimes straight away, sometimes after running/scraping/downloading for an hour or two. I'm using requests to handle opening urls etc. Any help would be massively appreciated!I've included the entire traceback.

Traceback (most recent call last):
  File "trailer-dl.py", line 290, in <module>
    main()
  File "trailer-dl.py", line 282, in main
    if coming_soon_bool       : makeSoup(base_url + '/coming-soon/')
  File "trailer-dl.py", line 267, in makeSoup
    makeNewSoup(current_link)                                 # Start again with a new link to see if there is anything to grab
  File "trailer-dl.py", line 247, in makeNewSoup
    checkLink(new_current_link)
  File "trailer-dl.py", line 147, in checkLink
    downloadLink(active_link)   
  File "trailer-dl.py", line 165, in downloadLink
    r = requests.get(url, headers={"User-agent":"Quicktime"}, stream = True)  # Identify ourselves as a quicktime player and open url as a stream
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 378, in send
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='movietrailers.apple.com', port=80): Max retries exceeded with url: /movies/focus_features/badwords/badwords-tlr1_h720p.mov (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)

Answer 1

Since you are certain that it is a Pi specific problem and that the issue seems to arise from a network issue I'm going to assume a power supply issue.

I have has similar problems where the network IC drops the connection because of a spike in power draw in a (completely unrelated) part of the setup, usually this gives it's self away by the network status LEDs.

Best to ensure that you are using a supply rated for at least 800mA plus the rated load of any peripherals.