0

I have a python file that is running ok if i run it in a terminal in raspberry pi. But if i set it to automatically run after it reboots (using crontab), the file runs ok until it reaches the line where it requires to find the matching Chinese character using regex. I suspect it is related to the encoding. But why the problem only arises in the reboot? Since the output is not shown at startup, I cannot copy the error message. But I really think this is the problem. Part of the code as below: Edited to output to a log file:

# -*- coding: utf-8 -*-
import os
import sys
import requests
import re
import logging 

logging.basicConfig(filename='/home/pi/numbers4d/samlog.log', level=logging.DEBUG)


fetch = requests.get('http://www.check4d.com')   
print fetch
print "============="

match_date = re.search(r'<td class="resultdrawdate">(.+?)</td>' , fetch.text)
print match_date
print "======"

match_M = re.search(r'<td class="resultm4dlable">Magnum 4D 萬能(.+?)</table></div>' , fetch.text)
print("error-check", sys.exc_info()[0])
print match_M
matched = match_M.group()
print matched
logging.debug('error magnum matched obj: %s', matched)

What i get in the terminal output is like this:

python check4d_r9test.py
<Response [200]>
=============
<_sre.SRE_Match object at 0x76cb2c20>
======
('error-check', None)
<_sre.SRE_Match object at 0x76658b60>
<td class="resultm4dlable">Magnum 4D è¬è½</td></tr></table></td></tr><tr><td colspan="5"><table class="resultTable2" cellpadding="0" cellspacing="5"><tr><td class="resultdrawdate">Date: 03-09-2016 (Sat)</td><td class="resultdrawdate">Draw No: 696/16</td></tr></table></td></tr><tr><td colspan="5"><table class="resultTable2" cellpadding="0" cellspacing="0"><tr><td style="width:45%" class="resultprizelable">1st Prize é¦ç</td><td class="resulttop">3617</td></tr><tr><td style="width:45%" class="resultprizelable">2nd Prize äºç</td><td class="resulttop">5039</td></tr><tr><td style="width:45%" class="resultprizelable">3rd Prize ä¸ç</td><td class="resulttop">4693</td></tr></table></td></tr><tr><td colspan="5" class="resultprizelable">Special ç¹å¥ç</td></tr><tr><td class="resultbottom">7408</td><td class="resultbottom">7013</td><td class="resultbottom">7401</td><td class="resultbottom">7331</td><td class="resultbottom">----</td></tr><tr><td class="resultbottom">0595</td><td class="resultbottom">6838</td><td class="resultbottom">----</td><td class="resultbottom">6984</td><td class="resultbottom">0811</td></tr><tr><td style="width:20%"></td><td class="resultbottom">----</td><td class="resultbottom">2605</td><td class="resultbottom">6854</td><td style="width:20%"></td></tr><tr><td colspan="5" class="resultprizelable">Consolation å®æ°ç</td></tr><tr><td class="resultbottom">6259</td><td class="resultbottom">2958</td><td class="resultbottom">4109</td><td class="resultbottom">0947</td><td class="resultbottom">6402</td></tr><tr><td class="resultbottom">7736</td><td class="resultbottom">5663</td><td class="resultbottom">7370</td><td class="resultbottom">7570</td><td class="resultbottom">9137</td></tr></table></div>

But what i get in the output log file (using crontab and output to a file)

@reboot sleep 15; python /home/pi/numbers4d/check4d_r9test.py > /home/pi/samlog.log

is missing the last part, like this:

<Response [200]>
=============
<_sre.SRE_Match object at 0x76caec20>
======
('error-check', None)
<_sre.SRE_Match object at 0x171fc60>
  • Please add the text of the error message to your question. – Steve Robillard Sep 3 '16 at 14:50
  • I think it has nothing to do with Raspberry Pi. I tried fetching the data you mentioned above but the output of Match_M = None. There is no match for your regular expression – Shan-Desai Sep 3 '16 at 15:36
  • @Steve, Actually, there is no way i can see the error message once the python file runs in the background after reboot, unlike if I run it in a terminal after it has booted up. So i can only use logging feature of python and output it to a text file. And the content of the log file become: INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): www.check4d.com DEBUG:requests.packages.urllib3.connectionpool:"GET / HTTP/1.1" 200 None DEBUG:root:find-the-matchtext: <_sre.SRE_Match object at 0xa4b8e0> DEBUG:root:true-or-false: 1 – sam Sep 3 '16 at 16:08
  • Try redirecting the output of the script to a file when running it at boot time to see the error. E.g. python myscript.my > /path/to/logfile.log – awatts Sep 3 '16 at 21:00
  • @awatts, I have output to a log file as you say, seems like the output log cannot print the match_M. Why is that so? – sam Sep 4 '16 at 5:04
0

After some trials and errors and searching, I am able to make it work. The logfile will not display the output of the match because the output contains Chinese characters which looks like these... è¬è½ In order to solve this, i make the output text to convert to ascii.

matched = match_M.group(0).encode('ascii', 'replace')

The whole code now becomes:

# -*- coding: utf-8 -*-
import os
import sys
import requests
import re
import logging


logging.basicConfig(filename='/home/pi/numbers4d/samlog.log', level=logging.DEBUG)


fetch = requests.get('http://www.check4d.com')
print "============="

match_date = re.search(ur'<td class="resultdrawdate">(.+?)</td>' , fetch.text)
print match_date
print "======"
print "avuchfcnrcfnc"


match_M = re.search(ur'<td class="resultm4dlable">Magnum 4D 萬能(.+?)</table></div>'.encode('utf-8') , fetch.text)
#match_M = re.search(ur'<td class="resultm4dlable">Magnum 4D(.+?)</table></div>' , fetch.text)
print("error-check", sys.exc_info()[0])
print match_M
matched = match_M.group(0).encode('ascii', 'replace')
#matched = match_M.group(0)
print "xxxx"
print matched
logging.debug('error magnum matched obj: %s', matched)
print "yyyyy"

and the output becomes:

=============
<_sre.SRE_Match object at 0x76ce8c20>
======
avuchfcnrcfnc
('error-check', None)
<_sre.SRE_Match object at 0x7668cea0>
xxxx
<td class="resultm4dlable">Magnum 4D ??????</td></tr></table></td></tr><tr><td c                                                                                        olspan="5"><table class="resultTable2" cellpadding="0" cellspacing="5"><tr><td c                                                                                        lass="resultdrawdate">Date: 28-09-2016 (Wed)</td><td class="resultdrawdate">Draw                                                                                         No: 709/16</td></tr></table></td></tr><tr><td colspan="5"><table class="resultT                                                                                        able2" cellpadding="0" cellspacing="0"><tr><td style="width:45%" class="resultpr                                                                                        izelable">1st Prize ??????</td><td class="resulttop">7947</td></tr><tr><td style                                                                                        ="width:45%" class="resultprizelable">2nd Prize ??????</td><td class="resulttop"                                                                                        >4237</td></tr><tr><td style="width:45%" class="resultprizelable">3rd Prize ????                                                                                        ??</td><td class="resulttop">7344</td></tr></table></td></tr><tr><td colspan="5"                                                                                         class="resultprizelable">Special ?????????</td></tr><tr><td class="resultbottom                                                                                        ">3820</td><td class="resultbottom">7637</td><td class="resultbottom">----</td><                                                                                        td class="resultbottom">8400</td><td class="resultbottom">8100</td></tr><tr><td                                                                                         class="resultbottom">1358</td><td class="resultbottom">5357</td><td class="resul                                                                                        tbottom">----</td><td class="resultbottom">1948</td><td class="resultbottom">335                                                                                        8</td></tr><tr><td style="width:20%"></td><td class="resultbottom">----</td><td                                                                                         class="resultbottom">6998</td><td class="resultbottom">9648</td><td style="width                                                                                        :20%"></td></tr><tr><td colspan="5" class="resultprizelable">Consolation ???????                                                                                        ??</td></tr><tr><td class="resultbottom">6555</td><td class="resultbottom">5315<                                                                                        /td><td class="resultbottom">2549</td><td class="resultbottom">9428</td><td clas                                                                                        s="resultbottom">4487</td></tr><tr><td class="resultbottom">7778</td><td class="                                                                                        resultbottom">7750</td><td class="resultbottom">3896</td><td class="resultbottom                                                                                        ">5717</td><td class="resultbottom">4682</td></tr></table></div>
yyyyy

But i still do not understand why it cannot output unicode/special characters only during startup using crontab, while there is no problem at all running the file in terminal.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.