使用monkey技术修改python requests模块

例如请求前和请求后各来一条日志,这样就不需要在自己的每个代码都去加日志了。

其实也可以直接记录'urllib3.connectionpool'  logger name的日志。


修改了requests,所有requests的地方将自动打印日志,前提要给custom_request添加handlers
import requests
from app.utils.utils_ydf import LogManager logger = LogManager('custom_request').get_logger_without_handlers() class CustomSession(requests.Session):
def request(self, method, url,
params=None, data=None, headers=None, cookies=None, files=None,
auth=None, timeout=None, allow_redirects=True, proxies=None,
hooks=None, stream=None, verify=None, cert=None, json=None):
"""Constructs a :class:`Request <Request>`, prepares it and sends it.
Returns :class:`Response <Response>` object. :param method: method for the new :class:`Request` object.
:param url: URL for the new :class:`Request` object.
:param params: (optional) Dictionary or bytes to be sent in the query
string for the :class:`Request`.
:param data: (optional) Dictionary, bytes, or file-like object to send
in the body of the :class:`Request`.
:param json: (optional) json to send in the body of the
:class:`Request`.
:param headers: (optional) Dictionary of HTTP Headers to send with the
:class:`Request`.
:param cookies: (optional) Dict or CookieJar object to send with the
:class:`Request`.
:param files: (optional) Dictionary of ``'filename': file-like-objects``
for multipart encoding upload.
:param auth: (optional) Auth tuple or callable to enable
Basic/Digest/Custom HTTP Auth.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple
:param allow_redirects: (optional) Set to True by default.
:type allow_redirects: bool
:param proxies: (optional) Dictionary mapping protocol or protocol and
hostname to the URL of the proxy.
:param stream: (optional) whether to immediately download the response
content. Defaults to ``False``.
:param verify: (optional) Either a boolean, in which case it controls whether we verify
the server's TLS certificate, or a string, in which case it must be a path
to a CA bundle to use. Defaults to ``True``.
:param cert: (optional) if String, path to ssl client cert file (.pem).
If Tuple, ('cert', 'key') pair.
:rtype: requests.Response
"""
# Create the Request.
req = requests.Request(
method=method.upper(),
url=url,
headers=headers,
files=files,
data=data or {},
json=json,
params=params or {},
auth=auth,
cookies=cookies,
hooks=hooks,
)
logger.debug('start {} this url --> {} '.format(method, url))
prep = self.prepare_request(req) proxies = proxies or {} settings = self.merge_environment_settings(
prep.url, proxies, stream, verify, cert
) # Send the request.
send_kwargs = {
'timeout': timeout,
'allow_redirects': allow_redirects,
}
send_kwargs.update(settings)
resp = self.send(prep, **send_kwargs)
logger.debug('{} {} {} {} {} {}'.format(method, resp.url, resp.status_code, resp.elapsed.total_seconds(), resp.is_redirect, resp.text.__len__()))
return resp def custom_request(method, url, **kwargs):
with CustomSession() as session:
return session.request(method=method, url=url, **kwargs) def patch_request():
LogManager('custom_request').get_logger_and_add_handlers()
requests.api.request = custom_request if __name__ == '__main__':
patch_request()
resp = requests.get('http://www.sina.com.cn')
requests.get('https://www.baidu.com')

1、结果就是这样。这样所有地方的requests请求都会生效了。

正常情况下我有自己围绕Session类实例包装的SessionWrapper类,通常情况下我不会直接去使用requests的那些函数请求接口,但同事都是直接requests,然后每个地方打印print一下请求和返回,以便能看看到发了什么请求,那样有些重复。而且项目有几百个文件,突然蹦出来的print语句,根本不知道哪里print的。

使用这种猴子技术后,不用修改库源码,就能生效。

使用monkey技术修改python requests模块

2、使用猴子技术要注意的是,patch的地方并不是随便替换就能生效的。

如果库里面的那个地方是from xx import yy

你直接yy.function = yourfunction

这是无效的,一定要找到正确的地方替换,要搞清除到底怎么才能生效,必须要熟悉python的模块导入机制。可以看我的上上篇的三个试验文件。

python 模块会导入几次?猴子补丁为什么可以实现?

上一篇:Java框架spring Boot学习笔记(三):Controller的使用


下一篇:Windows下安装Python requests模块