爬虫3 Request 入门
Pycharm安装Request
利用Pycharm的terminal直接命令语句(可以使用清华源)
链接: https://mirrors.tuna.tsinghua.edu.cn/help/pypi/.
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple some-package
pip install requests
requests 应用
requests模块实现网页文本抓取
import requests
query = input("输入一个你喜欢的明星")
url = f"https://www.sogou.com/web?query={query}" # 地址栏均为get请求
dic = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36"
}
resp = requests.get(url, headers=dic) # 处理一个小小的反爬
print(resp.text) # 得到网站文本内容
requests实现爬取百度翻译
import requests
url = "https://fanyi.baidu.com/sug"
s = input("请输入你要查询的单词")
dic = {
"kw": s
}
resp = requests.post(url, data=dic) # 注意为post
print(resp.json()) # 得到网站翻译