Web Scraping爬虫 for Mac urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>
Solution: if 1) does not work, then try to use 2). Reference is here.
1)
pip install --upgrade certifi
2)
open /Applications/Python\ 3.6/Install\ Certificates.command
if running a python file to use https call, have error like
"SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661"
Solution:
then solve it by adding
configuration.verify_ssl = False
in the test file.
Then it will skip verifying SSL certificate when calling API from https server.
if running a python file to use https call, have the warning of InsecureRequestWarning
Solution:
then solve it by adding
import urllib3
urllib3.disable_warnings()
in the test file.
Then it will skip warning.
- Remove leading whitespace 去除string前面的空格
Solution:
>>> ' hello world!'.lstrip()
'hello world!'
-
Web Scraping爬虫
看网页编码方式( 一般为“utf-8”)
Solution: F12功能键,即可使用开发者工具,在窗口console标签下,键入 "document.charset" 即可查看网页的编码方式