1.什么是requests?
requests是一个Python第三方库,处理URL资源特别方便
2.安装requests
pip3 install requests
如果遇到Permission denied安装失败,请加上sudo重试
3.使用requests
3.1get请求方法
3.1.1基本的get请求
import requests response = requests.get('http://httpbin.org/get') print(response.text)
3.1.2带参数的get请求
对于带参数的,传入一个dict作为params参数:
import requests def loginGet(): r=requests.get( url='http://47.95.142.233:8000/login/auth/', params={"name":"wuya","age":18}) print('协议状态码',r.status_code) #获取到的是字典的数据类型 print('返回字典的数据类型',r.json()) #获取请求的地址信息 print('获取请求地址的信息',r.url) #获取到的是字符串 print('获取的是字符串的数据类型',r.text) #获取到的是bytest的数据类型 print('获取到的是bytest的数据类型',r.content) #获取响应头 print('响应头',r.headers)
备注:响应数据一般是r.text来获取,如果响应数据是json'格式,那么使用r.json()
查询电话号码归属地:
import requests r=requests.get( url='http://ws.webxml.com.cn/WebServices/MobileCodeWS.asmx/getMobileCodeInfo', params={"mobileCode":"15191084297","userId":""}) print('协议状态码', r.status_code) print('获取请求地址的信息', r.url) print('响应头', r.headers) print('获取的是字符串的数据类型',r.text)
3.2post请求方法
3.2.1json数据格式:
import json
def loginPost(): r = requests.post( url='http://47.95.142.233:8000/login/auth/', json={"username":"15191084297","password":"123456"}, headers={'content-type': 'application/json'}) print(json.dumps(r.json(),ensure_ascii=False,indent=True))
3.2.2表单的数据格式:
r=requests.post( url='http://ws.webxml.com.cn/WebServices/MobileCodeWS.asmx/getMobileCodeInfo', data={"mobileCode":"15191084297","userId":""}, headers={'content-type':'application/x-www-form-urlencoded'}) print('获取的是字符串的数据类型',r.text)
json和data的区别:
什么时候使用json
当请求参数是JSON的时候使用json的参数当请求参数是json,但是要使用data的参数,那么请求参数要进行序列化的处理
什么时候使用data
当请求参数是表单的时候使用data当请求参数是JSON格式的时候,那么请求参数要进行序列化的处理
拉勾网实战:
r = requests.post( url='https://www.lagou.com/jobs/v2/positionAjax.json', data={ 'data': 'Iphu%2FZ6ba7xjPqUDKO03NaTge36Zu7CANc5xZfNf0Brv4y1pJEeEuKl1PTDANiyGvMDYzLyoZqP7pPX3T9vFbzaVfuiXlpLScGwoDEJ6YlgCeyA38q5TJ%2FeCp2OmMywUQalg1pLh5vpFJIazFh7A6ovCMqZFGFB2DmsZXn22k4LdQJhEnS8Y4rmCv6a15mFX'}, headers={'content-type': 'application/x-www-form-urlencoded', 'Cookie': 'RECOMMEND_TIP=true; PRE_UTM=; PRE_LAND=https%3A%2F%2Fwww.lagou.com%2F; user_trace_token=20220119144853-8c70d9f2-7f4f-4097-ab6f-77a57190e796; LGUID=20220119144853-fa5d8ce2-6aa5-4fd0-9353-a266309a766c; privacyPolicyPopup=false; _ga=GA1.2.673573823.1642574934; _gid=GA1.2.1607293937.1642574935; LGSID=20220119144854-49e4e6e0-cb73-4e52-b471-58cefe6e79dc; PRE_HOST=cn.bing.com; PRE_SITE=https%3A%2F%2Fcn.bing.com%2F; sajssdk_2015_cross_new_user=1; WEBTJ-ID=20220119145841-17e712165122f0-02258ae20ef9a5-5e181655-1327104-17e71216513558; JSESSIONID=ABAAABAABEIABCI41426992A613A6F1F2FAAD0189C97FFD; sensorsdata2015session=%7B%7D; Hm_lvt_4233e74dff0ae5bd0a3d81c6ccf756e6=1642574935,1642575522; index_location_city=%E5%85%A8%E5%9B%BD; X_HTTP_TOKEN=68221908d10068cd4255752461329f451998b7de45; Hm_lpvt_4233e74dff0ae5bd0a3d81c6ccf756e6=1642575525; TG-TRACK-CODE=index_search; LGRID=20220119145857-272d10f0-d02a-451c-b154-f0074a3fb737; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%2217e711873e74c8-0e9dec4a6c3c5a-5e181655-1327104-17e711873e818b%22%2C%22first_id%22%3A%22%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E8%87%AA%E7%84%B6%E6%90%9C%E7%B4%A2%E6%B5%81%E9%87%8F%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC%22%2C%22%24latest_referrer%22%3A%22https%3A%2F%2Fcn.bing.com%2F%22%2C%22%24os%22%3A%22Windows%22%2C%22%24browser%22%3A%22Chrome%22%2C%22%24browser_version%22%3A%2297.0.4692.71%22%7D%2C%22%24device_id%22%3A%2217e711873e74c8-0e9dec4a6c3c5a-5e181655-1327104-17e711873e818b%22%7D', 'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36 Edg/97.0.1072.62', 'referer':'https://www.lagou.com/wn/jobs?labelWords=&fromSearch=true&suginput=&kd=%25E6%25B5%258B%25E8%25AF%2595%25E5%25BC%2580%25E5%258F%2591%25E5%25B7%25A5%25E7%25A8%258B%25E5%25B8%2588'}) print('协议状态码:',r.status_code) print('返回协议数据:',r.text) print(json.dumps(r.json(),indent=True,ensure_ascii=False))
4.requests参数详解
headers
里面填写请求头的信息,请求头的信息都是字典的数据类型
cookies
在请求头里面需要带上服务端的响应头中返回来的set- cookie的值
51cto实战:
def login(): r=requests.post( url='https://home.51cto.com/index?reback=https%3A%2F%2Fhome.51cto.com&iframe=0&is_go_to_user_set_mobile=1', data={"_csrf":"ZXk0VTBJaXENEEE5eCsjN1U9ZGZ0LwI7LBRjAQM7Dh1IDAAeagoZRw", "LoginForm[username]":"15191084297", "LoginForm[password]":"cy980716", "show_qr":"0"}, headers={'content-type':'application/x-www-form-urlencoded', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36', 'Referer': '//home.51cto.com/index', 'Cookie': 'www51cto=02B0203B122F2012E0EC7BAC6020A4C2pHub; _csrf=6a44440f3396ae1244c0004595444c2bc471690ec46ad7c1fa772eb23bb216daa%3A2%3A%7Bi%3A0%3Bs%3A5%3A%22_csrf%22%3Bi%3A1%3Bs%3A32%3A%22hiulHbJF0DP3DfkJImWT3rgl-u4KZCp6%22%3B%7D; _ourplusFirstTime=122-1-13-15-38-6; sajssdk_2015_cross_new_user=1; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%2217e525f5627a34-0113622e3482c8-36657407-1296000-17e525f5628112a%22%2C%22first_id%22%3A%22%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E7%9B%B4%E6%8E%A5%E6%B5%81%E9%87%8F%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC_%E7%9B%B4%E6%8E%A5%E6%89%93%E5%BC%80%22%2C%22%24latest_referrer%22%3A%22%22%7D%2C%22%24device_id%22%3A%2217e525f5627a34-0113622e3482c8-36657407-1296000-17e525f5628112a%22%7D; Hm_lvt_110fc9b2e1cae4d110b7959ee4f27e3b=1642059408,1642084977; pub_cookietime=2592000; lastlogin=on; Hm_lpvt_110fc9b2e1cae4d110b7959ee4f27e3b=1642085620; Hm_lvt_844390da7774b6a92b34d40f8e16f5ac=1642085057,1642085464,1642085663,1642086193; callback_api_url=https%3A%2F%2Fhome.51cto.com%2Findex; pub_wechatopen=DgZDExYRBjVaYERiLmkgAWt0GV0kZWIHGhIBMTwLMWwlN0oVAxAPBVlec2I2aSVMR1ZESR9RfHkzalAMalZTB1ZdDQdTAwdVVz0DB1cLVgwCBQ1X; PHPSESSID=8gfk7kgeg4jjo1a6vkfk2vcc22; once_p=ce2eb9; _ourplusReturnCount=18; _ourplusReturnTime=122-1-13-23-12-4; login_from=home.51cto.com; reg_from=home.51cto.com; Hm_lpvt_844390da7774b6a92b34d40f8e16f5ac=1642086725'}) print(r.status_code) print(r.text) print(r.cookies) # r.cookies简单理解就是拿到响应头里面的set-cookie返回给客户端的指,也就是sessionID return r.cookies def profile(): r=requests.get( url='https://edu.51cto.com/center/user/info/get-user-info', # 通过请求头里面的cookie把返回来的sessionID发送给服务端进行身份验证 cookies=login()) print(r.status_code) print(r.text)
files
文件上传的参数
auth
权限验证
import requests from requests.auth import HTTPBasicAuth api=requests.get( url='http://101.43.158.84:5000/v1/api/books', auth=HTTPBasicAuth(username='admin',password='admin'), timeout=180) print(api.status_code) print(json.dumps(api.json(),ensure_ascii=False,indent=True))
timeout
主要指的是服务端响应数据很慢,但有时候很快,所以这个时候建议加tiaout,相当于隐性等待,还有一种就是当网络请求出现requests.exceptions.ConnectTimeout的错误时候,那么就意味着该参数需要上场了
import requests from requests.auth import HTTPBasicAuth api=requests.get( url='http://101.43.158.84:5000/v1/api/books', auth=HTTPBasicAuth(username='admin',password='admin'), timeout=180) print(api.status_code) print(json.dumps(api.json(),ensure_ascii=False,indent=True))
verify
当你的代码没任何的问题,但是请求就是返回ssl 以及403的问题,那么就意味着需要设置verify的参数如返回错误:requests.exceptions.SSLError。那么就需要带上该参数
import requests from requests.auth import HTTPBasicAuth api=requests.get( url='http://101.43.158.84:5000/v1/api/books', auth=HTTPBasicAuth(username='admin',password='admin'), timeout=180,verify=False) print(api.status_code) print(json.dumps(api.json(),ensure_ascii=False,indent=True))
5.requests中动态参数解决
第一种写法:
imoport requests import json def login(): dict1={"username":"15191084297","password":"123456"} r=requests.post( url='http://47.95.142.233:8000/login/auth/', data=json.dumps(dict1), headers={'content-type':'application/json'}) return r.json()['token'] def index(): r=requests.get( url='http://47.95.142.233:8000/interface/index', headers={"Authorization":"JWT {0}".format(login())}) print(r.text) assert r.json()['count']['api']==4
第二种写法:
imoport requests
import json
def login(): dict1={"username":"15191084297","password":"123456"} r=requests.post( url='http://47.95.142.233:8000/login/auth/', data=json.dumps(dict1), headers={'content-type':'application/json'}) return r.json()['token'] def index(): r=requests.get( url='http://47.95.142.233:8000/interface/index', headers={"Authorization":"JWT {token}".format(token=login())}) print(r.text) assert r.json()['count']['api']==4