2021年中旬就计划着搭建一套压测系统,大约9月份已经搭建完成,使用至今还是比较稳定了,分享一下搭建思路及过程:
为什么选择Locust呢,因为Locust可以仅需要执行命令就可以完成压测任务,并且集群压测也很简单,只需压测机安装locust并把压测脚本推送到服务器即可。
画了一个大致的思路图:
我们说的全链路其实有几层意思:
1.多接口多场景,而非单接口或单url
2.按照用户访问场景及频率,用户访问的路径是有先后的,访问的接口频率也是不一样的。怎么理解这个呢,很简单,比如获取列表的接口(get_list)和获取内容的接口(get_content),用户访问任何页面有可能都会访问
get_list,但用户可能都不会点击详情,所以调用get_list的频率会更多。
怎么真实的获取到用户访问的链路场景呢?
1.通过用户访问的日志,分析用户的行为,然后编写压测场景用例
2.模拟用户场景,导出用户记录
A.浏览器直接导出记录生成.har文件
B.app通过抓包工具获取用户记录导出生成.har文件
当然有的人说har文件解析生成接口后,后续压测能一直有效么,比如token等校验通不过,解决这个问题很简单,和研发商量一下,请求参数里加每个值或对特定设备或标识放开就行,后续一路畅通无阻。
压测脚本来源有了,第二步就是解析har文件,模块库里有解析har的,但发现不满足自己使用,自己写吧,项目结构仅供参考:
解析Har文件:
1 # -*- coding = utf-8 -*- 2 # ------------------------------ 3 # @time: 2021/3/22 14:53 4 # @Author: drew_gg 5 # @File: disassemble_har.py 6 # @Software: cover_app_platform 7 # ------------------------------ 8 9 import json 10 from app.locust.anasiysis_har import judgment_exist as jud 11 from app.locust.anasiysis_har import deal_headers as dh 12 from app.locust.anasiysis_har import deal_request_data as dr 13 from app.config.har_to_api import api_filter as af 14 15 16 key_words = af.key_words 17 18 19 def disassemble_har(har_file, api_only=0): 20 """ 21 提取分解har文件 22 :param har_file: .har文件 23 :param api_only: 1:去重,其他:不去重 24 :return: 25 """ 26 27 req_l = [] 28 rdl = [] 29 rdl_set = [] 30 host = '' 31 count = 1 32 # url过滤非接口请求 33 with open(har_file, "r", encoding='utf-8') as f: 34 f = json.loads(f.read()) 35 for i in f['log']['entries']: 36 if jud.judgment_exist(i['request']['url'], key_words) is False: 37 req_l.append(i) 38 for index, i in enumerate(req_l): 39 rd = {} 40 # 解析host 41 host = i['request']['url'].split('//')[0] + '//' + i['request']['url'].split('//')[1].split('/')[0] 42 # 解析子url 43 # son_url = i['request']['url'].split(host)[1].split('&')[0] 44 son_url = i['request']['url'].split(host)[1] 45 deal_url = son_url.split('?')[0] 46 if deal_url == '/': 47 if len(son_url.split('?'))> 1: 48 deal_url = son_url.split('?')[1] 49 else: 50 deal_url = '/' 51 deal_url = deal_url.replace('/', '_').replace('-', '_').replace('.', '_').strip('_').lstrip('_') 52 if api_only == 1: 53 method_name = 'api_' + deal_url.lower() 54 else: 55 method_name = 'api_' + deal_url.lower() + '_' + str(index) 56 # 解析处理header 57 headers = dh.deal_headers(i['request']['headers']) 58 method = i['request']['method'] 59 # 解析处理请求参数 60 if method.upper() == "POST": 61 request_data = dr.deal_request_data(method, i['request']['postData']) 62 if method.upper() == "GET": 63 request_data = '\'' + i['request']['url'].split(son_url)[1] + '\'' 64 host = '"' + host + '"' 65 son_url = '"' + son_url + '"' 66 rd['host'] = host 67 rd['url'] = son_url 68 rd['headers'] = headers 69 rd['method'] = method 70 rd['method_name'] = method_name 71 rd['request_data'] = request_data 72 if api_only == 1: 73 # 去重并计数判断 74 if index == 0: 75 rd['count'] = count 76 rdl_set.append(rd) 77 else: 78 for x in rdl_set: 79 if son_url == x['url']: 80 x['count'] += 1 81 count = x['count'] 82 else: 83 if count == 1: 84 rd['count'] = count 85 rdl_set.append(rd) 86 count = 1 87 else: 88 rd['count'] = count 89 rdl.append(rd) 90 if api_only != 1: 91 rdl_set = rdl 92 return rdl_set, host 93 94 95 if __name__ == '__main__': 96 har_path = r'D:\thecover_project\cover_app_platform\app\file_upload\首页普通\20210803-113719\syptxq.har' 97 disassemble_har(har_path)
解析har文件,处理header、获取接口必要参数,然后对请求做分析,如果要去重,则统计相同请求的数量,压测时生成压测权重,如果不去重,后续生成压测脚本时则需要对处理方法名称。
解析好har文件后,需要生成调试脚本和压测脚本:
我处理方式实直接生成py文件,事先创建好模板,如:
生成调试脚本比较简单,只需要一个模板就行,生成locust压测脚本则稍微负责点,我是分拆成多个模板,然后整合到一个模板。
生成的脚本都规范放在目录里:
生成脚本目录结构:
生成压测脚本示例:
1 # -*- coding = utf-8 -*- 2 # ------------------------------ 3 # @time: 2021-04-19 13:43:10.380837 4 # @Author: drew_gg 5 # @File: liao_bao.py 6 # @Software: api_locust 7 # ------------------------------ 8 9 10 from locust import SequentialTaskSet, task, constant, tag, TaskSet 11 from locust.contrib.fasthttp import FastHttpUser 12 13 14 class LiaoBao20210419(TaskSet): 15 16 @task(1) 17 @tag('api_getlist') 18 def api_getlist(self): 19 headers = {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'tenantId': '7'} 20 # 请求参数组装 ## r_url:固定参数 21 r_url = "/getList?vno=6.4.0" 22 requests_data = {'account': 'E2247B94-51E2-4952-BC06-24752911C060', 'client': 'iOS', 'data': '{"operation_type":0,"news_id":0,xxxxxxxxxxxxxxxxxxx'} 23 # 发起请求 24 with self.client.post(r_url, data=requests_data, catch_response=True, name=r_url) as r: 25 if r.content == b"": 26 r.failure("No data") 27 if r.status_code != 200: 28 em = "request error --" + str(r.status_code) 29 r.failure(em) 30 31 @task(4) 32 @tag('api_getsysnotice') 33 def api_getsysnotice(self): 34 headers = {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'tenantId': '7'} 35 # 请求参数组装 ## r_url:固定参数 36 r_url = "/getSysnotice?vno=6.4.0" 37 requests_data = {'account': 'E251179A-6309-4326-9827-73C892131605', 'client': 'iOS', 'data': '{"page_size":15,"page":1}', xxxxxxxxxxxxxxxxxxxxxxxx} 38 # 发起请求 39 with self.client.post(r_url, data=requests_data, catch_response=True, name=r_url) as r: 40 if r.content == b"": 41 r.failure("No data") 42 if r.status_code != 200: 43 em = "request error --" + str(r.status_code) 44 r.failure(em) 45 46 @task(4) 47 @tag('api_user_preparecancelaccount') 48 def api_user_preparecancelaccount(self): 49 headers = {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'tenantId': '7'} 50 # 请求参数组装 ## r_url:固定参数 51 r_url = "/user/prepareCancelAccount?vno=6.4.0" 52 requests_data = {'account': '2FF3D47C-995B-4D7E-93CD-58B4F1E94B74', 'client': 'iOS', 'data': '{}', xxxxxxxxxxxxxxxxxxxxxxx} 53 # 发起请求 54 with self.client.post(r_url, data=requests_data, catch_response=True, name=r_url) as r: 55 if r.content == b"": 56 r.failure("No data") 57 if r.status_code != 200: 58 em = "request error --" + str(r.status_code) 59 r.failure(em) 60 61 62 class liao_bao_locust(FastHttpUser): 63 host = "https://xxxxxx.xxxxx.com" 64 wait_time = constant(0) 65 tasks = {LiaoBao20210419: 1}
生成好脚本后,需要生成执行命令:
1 # -*- coding = utf-8 -*- 2 # ------------------------------ 3 # @time: 2021/3/3 11:08 4 # @Author: drew_gg 5 # @File: locust_create_cmd.py 6 # @Software: cover_app_platform 7 # ------------------------------ 8 9 10 def create_master_cmd(locust_pra): 11 """ 12 生成master命令 13 :param locust_pra: 14 :return: 15 """ 16 # locust master 命令样式: 17 """ 18 locust -f /work/locust/api_locust/locust_view/fm_api/locust_api/locust_fm_640.py 19 --master 20 --master-bind-port 9800 21 --headless 22 -u 600 23 -r 200 24 --expect-worker 16 25 -t 10m 26 -s 10 27 --csv /work/locust/locust_report/fm/locust_get_dynamic.py0223145309 28 --html /work/locust/api_locust/resource/html/new_html/locust_get_operation_parm.html 29 """ 30 run_port = '9800' 31 master_cmd = "locust -f %s --master --master-bind-port %s --headless " % (locust_pra['to_file'], run_port) 32 master_pra = "-u %s -r %s --expect-worker %s -t %ss -s 10 --csv %s --html %s > %s" % \ 33 (locust_pra['user'], locust_pra['rate'], locust_pra['thread'], locust_pra['time'], locust_pra['csv'], 34 locust_pra['html'], locust_pra['master_log']) 35 master_cmd = master_cmd + master_pra 36 return master_cmd 37 38 39 def create_slave_cmd(locust_pra): 40 """ 41 生成slave命令 42 :return: 43 """ 44 run_port = '9800' 45 if len(locust_pra['api']) == 1 and locust_pra['api'][0] == '': 46 slave_cmd = "locust -f %s --master-host %s --master-port %s --headless --worker > %s" % \ 47 (locust_pra['to_file'], locust_pra['master'].split('-')[0], run_port, locust_pra['slave_log']) 48 else: 49 tags = '' 50 for i in locust_pra['api']: 51 tags += i.split(".py")[0] + ' ' 52 slave_cmd = "locust -f %s --master-host %s --master-port %s --headless --worker -T %s > %s" % \ 53 (locust_pra['to_file'], locust_pra['master'].split('-')[0], run_port, tags, locust_pra['slave_log']) 54 return slave_cmd
然后把文件推送到服务器上,服务器也需要有规定的目录:
每台压测机上建立三个目录:
master上存储压测生成的报告、csv文件,然后写个定时程序拉去报告到项目服务器,压测完后可直接查询报告。
这部分处理业务稍微复杂(代码贴了flask的一些页面和主要处理逻辑,感兴趣的可以具体讨论):
1 # -*- coding = utf-8 -*- 2 # ------------------------------ 3 # @time: 2021/6/10 11:38 4 # @Author: drew_gg 5 # @File: locust_view.py 6 # @Software: cover_app_platform 7 # ------------------------------ 8 9 import os 10 import time 11 import requests 12 import datetime 13 from app.dao import read_sql as sql 14 from flask import Blueprint, render_template, request, url_for, redirect, send_file, jsonify, session 15 from app.locust.create_locust_run import create_linux_run as cl 16 from app.locust.run_locust import locust_create_cmd as lc 17 from app.dao import flask_mysql as har_sql 18 from app.common import common_run_cmd as cmd 19 from app.common.common_remote_link import remote_linux as ssh_linux 20 from flask_login import login_required 21 from app.dao import zen_tao_read_sql as zt_sql 22 23 24 pl = os.getcwd().split('cover_app_platform') 25 path_root = pl[0] + 'cover_app_platform' 26 path_upload_path = pl[0] + r'cover_app_platform\\app\\file_upload\\' 27 linux_user = "root" 28 linux_key = pl[0] + r'cover_app_platform\app\\common\key\linux_key' 29 tem_master = pl[0] + r'cover_app_platform\app\locust\template_file\locust_template_master.py' 30 tem_slave = pl[0] + r'cover_app_platform\app\locust\template_file\locust_template_slave.py' 31 save_report = pl[0] + r'cover_app_platform\\app\\views\\report\\' 32 33 34 locust_v = Blueprint('locust_v', '__main__') 35 36 37 @locust_v.route('/ywzx/har_upload/') 38 @login_required 39 def upload_index(): 40 """ 41 上传页面 42 :return: 43 """ 44 return render_template('/post/upload/upload_har.html') 45 46 47 @locust_v.route('/locust', methods=['POST', 'GET']) 48 @login_required 49 def get_har_file(): 50 """ 51 性能测试主页 52 :return: 53 """ 54 un = [session.get('username').split('-')[0]] 55 u = zt_sql.deal_mysql("get_zen_tao_group.sql", un) 56 new_mysql = [] 57 # 判断权限,测试角色和管理员角色才能看到所有的记录 58 if u[0][1] not in ['qa', 'qd', 'td', 'admin'] and un[0].lower() not in ['virgil', 'frankie', 'steven']: 59 mysql_l = sql.deal_mysql('get_har_file_cs.sql') 60 for i in mysql_l: 61 new_mysql.append(list(i)) 62 else: 63 # alter table app_pkg.har_info auto_increment=1; 64 mysql_l = sql.deal_mysql('get_har_file.sql') 65 for i in mysql_l: 66 new_mysql.append(list(i)) 67 return render_template('/post/locust/ywzx.html', data=new_mysql) 68 69 70 @locust_v.route('/har/<har_name>') 71 @login_required 72 def get_locust_debug(har_name): 73 """ 74 API接口调试主页 75 :param har_name: 76 :return: 77 """ 78 par_har = [] 79 har_dic = {'upload_id': har_name.split('_')[0], 'har_name': har_name.split('_')[1]} 80 par_har.append(har_dic) 81 r = sql.deal_more_mysql("get_locust_debug.sql", par_har) 82 new_mysql = [] 83 for i in r: 84 new_mysql.append(list(i)) 85 return render_template('/post/locust/locust_debug.html', data=new_mysql) 86 87 88 @locust_v.route('/py_detail/<py_name>') 89 @login_required 90 def get_py_detail(py_name): 91 """ 92 获取脚本代码详情 93 :param py_name: 94 :return: 95 """ 96 f_name = py_name.split(',')[-1] 97 file_path = path_root + py_name.replace(',', '\\') 98 with open(file_path, 'r', encoding='utf-8') as f: 99 f = f.read() 100 return render_template('/post/locust/py_detail.html', data=f, pn=f_name, yn=py_name, path=file_path.split('\\')) 101 102 103 @locust_v.route('/run_detail/<py_name>') 104 @login_required 105 def get_run_detail(py_name): 106 """ 107 执行代码并格式化显示代码 108 :param py_name: 109 :return: 110 """ 111 f_name = py_name.split(',')[-1] 112 file_path = path_root + py_name.replace(',', '\\') 113 run_cmd = "python " + file_path 114 run_result = cmd.run_cmd(run_cmd) 115 tempters = run_result._stream 116 # 这个地方坑死啊,换成utf-8中文会乱码,换成gbk就ok,无语无语!! 117 result = tempters.buffer.read().decode(encoding='gbk', errors='ignore') 118 replace_result = result 119 if "<" in result or ">" in result: 120 replace_result = result.replace('"', '\\"') 121 replace_result = replace_result.replace('"', '') 122 replace_result = replace_result.replace('\'', '"') 123 replace_result = replace_result.replace('\r', '') 124 replace_result = replace_result.replace('\n', '') 125 replace_result = replace_result.replace('False', 'false') 126 replace_result = replace_result.replace('True', 'true') 127 replace_result = replace_result.replace('"[', '[') 128 replace_result = replace_result.replace(']"', ']') 129 if replace_result == '': 130 replace_result = {"data": '******接口无数据返回********'} 131 return render_template('/post/locust/run_detail.html', data=replace_result, pn=f_name) 132 133 134 @locust_v.route('/locust_run/<py_name>', methods=['POST', 'GET']) 135 @login_required 136 def locust_run(py_name): 137 """ 138 压测配置页面 139 :param py_name: 140 :return: 141 """ 142 par_har = [] 143 har_dic = {'upload_id': py_name.split('$$')[0], 'har_name': py_name.split('$$')[2]} 144 master_path = path_root.replace('\\', ',') + py_name.split('$$')[3] 145 par_har.append(har_dic) 146 result_debug = sql.deal_more_mysql("get_locust_debug.sql", par_har) 147 new_mysql = [] 148 project_name = py_name.split('$$')[0] + '_' + py_name.split('$$')[1] + '_' + py_name.split('$$')[2] 149 for i in range(len(result_debug)): 150 data_dic = {'id': i, 'title': result_debug[i][4].split('\\')[-1]} 151 new_mysql.append(data_dic) 152 153 result_server = sql.deal_more_mysql("get_server_info.sql", par_har) 154 server_all = [] 155 for s in range(len(result_server)): 156 ip = result_server[s][1] + '-' + result_server[s][2] 157 server = {'id': s, 'title': ip} 158 server_all.append(server) 159 return render_template('/post/locust/run_locust.html', data=new_mysql, master_path=master_path, project_name=project_name, server=server_all) 160 161 162 @locust_v.route('/report/locust/<lp>', methods=['POST', 'GET']) 163 @login_required 164 def locust_perform(lp): 165 """ 166 执行压测 167 :param lp: 168 :return: 169 """ 170 run_status = sql.deal_mysql("get_run_status.sql") 171 if run_status: 172 run_project = run_status[0][0] 173 run_time = run_status[0][1] 174 return "已有【%s】项目正在压测,请【%s】后再申请压测" % (run_project, run_time) 175 elif request.method == 'POST' and request.view_args: 176 locust_pra = { 177 "api": lp.split('&')[0].split(','), 178 "master": lp.split('&')[1], 179 "slave": lp.split('&')[2].split(','), 180 "thread": lp.split('&')[3], 181 "user": lp.split('&')[4], 182 "rate": lp.split('&')[5], 183 "time": lp.split('&')[6], 184 "master_path": lp.split('&')[7].replace(',', '\\'), 185 "har_id": lp.split('&')[8].split('_')[0], 186 "project_name": lp.split('&')[8].split('_')[1], 187 "har_name": lp.split('&')[8].split('_')[2] 188 } 189 # 需要上传的主locust文件 190 from_file = locust_pra['master_path'] 191 # linux服务器上存放script的目录:项目名称/时间 192 script_to_path = '/work/locust/locust_run/locust_script/' + locust_pra['project_name'] + '/' + locust_pra['master_path'].split('\\')[-2] 193 # linux服务器上存放slave的目录:项目名称/时间 194 slave_to_path = '/work/locust/locust_run/locust_slave/' + locust_pra['project_name'] + '/' + locust_pra['master_path'].split('\\')[-2] 195 # linux服务器上存放master的目录:项目名称/时间 196 master_to_path = '/work/locust/locust_run/locust_master/' + locust_pra['project_name'] + '/' + locust_pra['master_path'].split('\\')[-2] 197 # linux服务器上存放script的文件目录 198 to_file = script_to_path + '/' + locust_pra['master_path'].split('\\')[-1] 199 # linux服务器master端存放html的目录 200 to_file_report = master_to_path + '/' + locust_pra['master_path'].split('\\')[-1].split('.py')[0] + '.html' 201 # linux服务器master端存放执行日志的目录 202 to_file_log_master = master_to_path + '/' + locust_pra['master_path'].split('\\')[-1].split('.py')[0] + '.log' 203 # linux服务器slave端存放html目录 204 to_file_log_slave = slave_to_path + '/' + locust_pra['master_path'].split('\\')[-1].split('.py')[0] + '.log' 205 # linux服务器master端存放csv的目录 206 to_file_csv = master_to_path + '/' + locust_pra['master_path'].split('\\')[-1].split('.py')[0] 207 # 相关字段存入字典 208 locust_pra['to_file'] = to_file 209 locust_pra['html'] = to_file_report 210 locust_pra['csv'] = to_file_csv 211 locust_pra['master_log'] = to_file_log_master 212 locust_pra['slave_log'] = to_file_log_slave 213 # 把项目路径的主路径存入字典,用于存放run_master.py和run_slave.py 214 locust_pra['project_path'] = locust_pra['master_path'].split(locust_pra['master_path'].split('\\')[-1])[0] 215 # 生成master脚本 216 master_cmd = lc.create_master_cmd(locust_pra) 217 run_master = locust_pra['project_path'] + 'run_master.py' 218 cl.create_linux_run(tem_master, run_master, master_cmd) 219 # 生成slave脚本 220 slave_cmd = lc.create_slave_cmd(locust_pra) 221 run_slave = locust_pra['project_path'] + 'run_slave.py' 222 thread = round(int(locust_pra['thread'])/len(locust_pra['slave'])) 223 cl.create_linux_run(tem_slave, run_slave, slave_cmd, thread) 224 # 创建脚本文件推送目录 225 cmd_script_create_script = "cd /work/locust/locust_run/locust_script/; mkdir -p " + locust_pra['project_name'] \ 226 + '/' + locust_pra['master_path'].split('\\')[-2] + "; chmod - R777 " + locust_pra['project_name'] + ";" 227 # 创建slave执行文件目录 228 cmd_slave_create_script = "cd /work/locust/locust_run/locust_slave/; mkdir -p " + locust_pra['project_name'] \ 229 + '/' + locust_pra['master_path'].split('\\')[-2] + "; chmod - R777 " + locust_pra['project_name'] + ";" 230 # 创建master执行文件目录 231 cmd_master_create_script = "cd /work/locust/locust_run/locust_master/; mkdir -p " + locust_pra['project_name'] \ 232 + '/' + locust_pra['master_path'].split('\\')[-2] + "; chmod - R777 " + locust_pra['project_name'] + ";" 233 # linux服务器上存放run_master.py的目录 234 to_run_master_file = master_to_path + '/' + 'run_master.py' 235 to_run_master_cmd = 'python ' + master_to_path + '/' + 'run_master.py' 236 # linux服务器上存放run_slave.py的目录 237 to_run_slave_file = slave_to_path + '/' + 'run_slave.py' 238 to_run_slave_cmd = 'python ' + slave_to_path + '/' + 'run_slave.py' 239 # 执行master脚本的服务器ip 240 host_master = locust_pra['master'].split('-')[1] 241 # 生成host集合 242 host_slave = [] 243 for i in locust_pra['slave']: 244 host_slave.append(i.split('-')[1]) 245 host_all = host_slave + [host_master] 246 host_l = list(set(host_all)) 247 # 完成linux远程连接上传文件和执行命令 248 start_time = datetime.datetime.now() 249 for con in host_l: 250 # 创建连接对象 251 master_ssh = ssh_linux.SSHConnection(con, linux_user, linux_key) 252 # 开始连接 253 master_ssh.connect() 254 # 执行创建locust_script下的项目路径 255 master_ssh.run_cmd(cmd_script_create_script) 256 time.sleep(2) 257 # 把主压测脚本文件上传到服务器上 258 master_ssh.upload(from_file, to_file) 259 # 如果是master服务器 260 if con == host_master: 261 # locust_master下创建项目路径 262 master_ssh.run_cmd(cmd_master_create_script) 263 time.sleep(2) 264 # 上传run_master.py到服务器上 265 master_ssh.upload(run_master, to_run_master_file) 266 time.sleep(2) 267 start_time = datetime.datetime.now() 268 master_ssh.run_cmd(to_run_master_cmd) 269 if con in host_slave: 270 # locust_slave下创建项目路径 271 master_ssh.run_cmd(cmd_slave_create_script) 272 time.sleep(2) 273 # 上传run_slave.py到服务器上 274 master_ssh.upload(run_slave, to_run_slave_file) 275 time.sleep(2) 276 master_ssh.run_cmd(to_run_slave_cmd) 277 master_ssh.close() 278 in_sql = "XXXXXX"314 d = har_sql.Database() 315 d.exec_no_query(in_sql) 316 history_url = locust_pra['har_id'] + '_' + locust_pra['har_name'] 317 return redirect(url_for("locust_v.locust_report", py_name=history_url)) 318 319 320 @locust_v.route('/locust_report/<py_name>', methods=['POST', 'GET']) 321 @login_required 322 def locust_report(py_name): 323 """ 324 压测报告处理 325 :param py_name: 326 :return: 327 """ 328 par_har = [] 329 har_dic = {'har_id': py_name.split('_')[0], 'har_name': py_name.split('_')[1]} 330 par_har.append(har_dic) 331 sql_result = sql.deal_more_mysql("get_locust_history.sql", par_har) 332 new_mysql = [] 333 for i in sql_result: 334 new_mysql.append(list(i)) 335 return render_template('/post/locust/locust_history_report.html', data=new_mysql, py_name=py_name) 336 337 338 @locust_v.route('/report/<report_id>', methods=['POST', 'GET']) 339 @login_required 340 def locust_report_detail(report_id): 341 """ 342 压测报告处理 343 :param report_id: 344 :return: 345 """ 346 html_file = report_id.split('$')[0] + '\\' + report_id.split('$')[1].replace('-', '').replace(':', '').replace(' ', '-') + '\\' + report_id.split('$')[2] 347 html = save_report + html_file 348 return send_file(html) 349 350 351 @locust_v.route('/server/<report_id>', methods=['POST', 'GET']) 352 @login_required 353 def locust_server(report_id): 354 server = {} 355 if len(report_id.split('$$')) > 5 and report_id.split('$$')[4] != '' and len(report_id.split('$$')[1].split('.')) > 2 and len(report_id.split('$$')[2].split('.')) > 2: 356 server['type'] = report_id.split('$$')[0] 357 server['intranet_ip'] = report_id.split('$$')[1] 358 server['external_ip'] = report_id.split('$$')[2] 359 server['ram'] = report_id.split('$$')[3] 360 server['cpu_threads'] = report_id.split('$$')[4] 361 server['bandwidth'] = report_id.split('$$')[5] 362 server['supplier'] = report_id.split('$$')[6] 363 sql_server = "XXXXXXX"397 d = har_sql.Database() 398 try: 399 d.exec_no_query(sql_server) 400 except Exception as e: 401 print(e) 402 return "IP已经存在!" 403 sql_result = sql.deal_mysql("get_server_info.sql") 404 new_mysql = [] 405 for i in sql_result: 406 new_mysql.append(list(i)) 407 return render_template('/post/locust/locust_server.html', data=new_mysql) 408 409 410 @locust_v.route('/locust_stop/<report_id>', methods=['POST', 'GET']) 411 @login_required 412 def locust_stop(report_id): 413 """ 414 压测报告处理 415 :param report_id: 416 :return: 417 """ 418 locust_run_id = report_id.split('$')[0] 419 py_name = report_id.split('$')[1] 420 sql_ip = sql.deal_mysql("get_run_ip.sql", list([locust_run_id])) 421 422 run_ip = [sql_ip[0][0].split('-')[1]] 423 for ip in sql_ip[0][1].split(','): 424 run_ip.append(ip.split("'")[1].split('-')[1]) 425 run_ip = list(set(run_ip)) 426 try: 427 for s in run_ip: 428 master_ssh = ssh_linux.SSHConnection(s, linux_user, linux_key) 429 # 开始连接 430 master_ssh.connect() 431 # 执行创建locust_script下的项目路径 432 stop_locust_cmd = "killall -9 locust" 433 master_ssh.run_cmd(stop_locust_cmd) 434 master_ssh.close() 435 update_status = """UPDATE locust_run set `status`=2 WHERE id = '%s' and `status`not in('1', '2') and is_delete = 0 limit 1 ;""" % locust_run_id 436 d = har_sql.Database() 437 d.exec_no_query(update_status) 438 except Exception as e: 439 print(e) 440 return redirect(url_for("locust_v.locust_report", py_name=py_name)) 441 442 443 @locust_v.route('/save/', methods=['POST', 'GET'], strict_slashes=False) 444 @login_required 445 def save_content(): 446 """448 :return: 449 """ 450 # 这个地方得用ajax发起post请求 451 # get请求:当发送数据时,GET方法向URL添加数据;URL的长度是受限制的(URL的最大长度是2048个字符),post请求:没有限制。 452 content = request.get_json() 453 path = content.split('$$')[0].replace(',', '\\') 454 content = content.split('$$')[1] 455 # 解析16进制为字符串 456 py_content = '' 457 try: 458 for b in content.split(','): 459 for i in [int(b, 16)]: 460 py_content += chr(i) 461 except Exception as e: 462 print(e) 463 move_cmd = "move " + path + ' ' + path.split('.py')[0] + '_copy.py' 464 os.system(move_cmd) 465 # 忽略特殊字符导致的转码错误 466 """ 467 s.encode('utf-8', 'replace').decode('utf-8') 468 如果设置为ignore,则会忽略非法字符; 469 如果设置为replace,则会用?取代非法字符; 470 如果设置为xmlcharrefreplace,则使用XML的字符引用。 471 """ 472 py_content = py_content.encode('utf-8', 'ignore').decode('utf-8') 473 try: 474 with open(path, 'w', encoding='utf-8') as f: 475 f.write(py_content) 476 except Exception as e: 477 print(e) 478 py_name = ',app' + path.split('app')[2].replace('\\', ',') 479 return redirect(url_for("locust_v.get_py_detail", py_name=py_name)) 480 481 482 @locust_v.route('/api_create', methods=['POST', 'GET'], strict_slashes=False) 483 @login_required 484 def create_api(): 485 """ 486 首页 487 :return: 488 """ 489 if request.json: 490 req_data = request.json 491 url = req_data["http"] + '://' + req_data['host'] + req_data['url'] 492 data = req_data['params'] 493 headers = req_data['headers'] 494 try: 495 if req_data['method'] == 'post': 496 r = requests.post(url, data=data, headers=headers) 497 if req_data['method'] == 'get': 498 r = requests.get(url, params=data, headers=headers) 499 except Exception as e: 500 return jsonify({"error": '请求异常', "msg": "请求失败", "data": e.__context__.args[0]}) 501 try: 502 data = r.json() 503 except Exception as e: 504 data = r.text 505 print(e) 506 return jsonify({"success": 200, "msg": "请求成功", "data": data}) 507 else: 508 return render_template('/post/locust/create_api.html')
平台主要界面:
1.首页
2.上传并解析har文件页面
3.压测脚本在线编辑执行页面
4.接口调试页面
5.调试结果页
6.压测配置页面
7.压测执行及记录页面
8.压测报告页面
9.服务器管理页面
大致包含这些功能,当然,项目搭建过程中遇到各种坑,要尝试才知道,后续打算优化一下代码,再升级几个版本,也算彻底搞定。
欢迎感兴趣的一起研究讨论。