python - Searching bing results with a script results in an encoding issue -
i've written following in order number of search results per word in wordlist
:
with open ("c:\wordslist.txt") f: lines = f.readlines() def bingsearch(word): r = requests.get('http://www.bing.com/search', params={'q':'"'+word+'"'} ) soup = beautifulsoup(r.text, "html.parser") return (soup.find('span',{'class':'sb_count'})) matches = [re.search(regex,line).groups() line in lines] match in matches: searchword = match[0] found = bingsearch(searchword) print (found.text)
it works , accurate results, except words containing special characters, example word: "número"
.
if call bingsearch("número")
accurate result. if call bingsearch(match[0])
(where printing match[0]
yields "número"
) inaccurate result.
i've tried stuff str(match[0])
, match[0].encode(encoding="utf-8")
, no success.
any ideas?
try giving encoding directly when open file, can make difference
with open ("c:\wordslist.txt", encoding="utf-8") f:
Comments
Post a Comment