基于locust全链路压测系统

2021年中旬就计划着搭建一套压测系统,大约9月份已经搭建完成,使用至今还是比较稳定了,分享一下搭建思路及过程:

为什么选择Locust呢,因为Locust可以仅需要执行命令就可以完成压测任务,并且集群压测也很简单,只需压测机安装locust并把压测脚本推送到服务器即可。

画了一个大致的思路图:

基于locust全链路压测系统

 

 

 我们说的全链路其实有几层意思:

1.多接口多场景,而非单接口或单url

2.按照用户访问场景及频率,用户访问的路径是有先后的,访问的接口频率也是不一样的。怎么理解这个呢,很简单,比如获取列表的接口(get_list)和获取内容的接口(get_content),用户访问任何页面有可能都会访问

get_list,但用户可能都不会点击详情,所以调用get_list的频率会更多。

怎么真实的获取到用户访问的链路场景呢?

1.通过用户访问的日志,分析用户的行为,然后编写压测场景用例

2.模拟用户场景,导出用户记录
  A.浏览器直接导出记录生成.har文件

  B.app通过抓包工具获取用户记录导出生成.har文件

当然有的人说har文件解析生成接口后,后续压测能一直有效么,比如token等校验通不过,解决这个问题很简单,和研发商量一下,请求参数里加每个值或对特定设备或标识放开就行,后续一路畅通无阻。

压测脚本来源有了,第二步就是解析har文件,模块库里有解析har的,但发现不满足自己使用,自己写吧,项目结构仅供参考:

基于locust全链路压测系统

 

解析Har文件:

 1 # -*- coding = utf-8 -*-
 2 # ------------------------------
 3 # @time: 2021/3/22 14:53
 4 # @Author: drew_gg
 5 # @File: disassemble_har.py
 6 # @Software: cover_app_platform
 7 # ------------------------------
 8 
 9 import json
10 from app.locust.anasiysis_har import judgment_exist as jud
11 from app.locust.anasiysis_har import deal_headers as dh
12 from app.locust.anasiysis_har import deal_request_data as dr
13 from app.config.har_to_api import api_filter as af
14 
15 
16 key_words = af.key_words
17 
18 
19 def disassemble_har(har_file, api_only=0):
20     """
21     提取分解har文件
22     :param har_file: .har文件
23     :param api_only: 1:去重,其他:不去重
24     :return:
25     """
26 
27     req_l = []
28     rdl = []
29     rdl_set = []
30     host = ''
31     count = 1
32     # url过滤非接口请求
33     with open(har_file, "r", encoding='utf-8') as f:
34         f = json.loads(f.read())
35         for i in f['log']['entries']:
36             if jud.judgment_exist(i['request']['url'], key_words) is False:
37                 req_l.append(i)
38     for index, i in enumerate(req_l):
39         rd = {}
40         # 解析host
41         host = i['request']['url'].split('//')[0] + '//' + i['request']['url'].split('//')[1].split('/')[0]
42         # 解析子url
43         # son_url = i['request']['url'].split(host)[1].split('&')[0]
44         son_url = i['request']['url'].split(host)[1]
45         deal_url = son_url.split('?')[0]
46         if deal_url == '/':
47             if len(son_url.split('?'))> 1:
48                 deal_url = son_url.split('?')[1]
49             else:
50                 deal_url = '/'
51         deal_url = deal_url.replace('/', '_').replace('-', '_').replace('.', '_').strip('_').lstrip('_')
52         if api_only == 1:
53             method_name = 'api_' + deal_url.lower()
54         else:
55             method_name = 'api_' + deal_url.lower() + '_' + str(index)
56         # 解析处理header
57         headers = dh.deal_headers(i['request']['headers'])
58         method = i['request']['method']
59         # 解析处理请求参数
60         if method.upper() == "POST":
61             request_data = dr.deal_request_data(method, i['request']['postData'])
62         if method.upper() == "GET":
63             request_data = '\'' + i['request']['url'].split(son_url)[1] + '\''
64         host = '"' + host + '"'
65         son_url = '"' + son_url + '"'
66         rd['host'] = host
67         rd['url'] = son_url
68         rd['headers'] = headers
69         rd['method'] = method
70         rd['method_name'] = method_name
71         rd['request_data'] = request_data
72         if api_only == 1:
73             # 去重并计数判断
74             if index == 0:
75                 rd['count'] = count
76                 rdl_set.append(rd)
77             else:
78                 for x in rdl_set:
79                     if son_url == x['url']:
80                         x['count'] += 1
81                         count = x['count']
82                 else:
83                     if count == 1:
84                         rd['count'] = count
85                         rdl_set.append(rd)
86                     count = 1
87         else:
88             rd['count'] = count
89         rdl.append(rd)
90     if api_only != 1:
91         rdl_set = rdl
92     return rdl_set, host
93 
94 
95 if __name__ == '__main__':
96     har_path = r'D:\thecover_project\cover_app_platform\app\file_upload\首页普通\20210803-113719\syptxq.har'
97     disassemble_har(har_path)

解析har文件,处理header、获取接口必要参数,然后对请求做分析,如果要去重,则统计相同请求的数量,压测时生成压测权重,如果不去重,后续生成压测脚本时则需要对处理方法名称。

解析好har文件后,需要生成调试脚本和压测脚本:

我处理方式实直接生成py文件,事先创建好模板,如:

基于locust全链路压测系统

 

 生成调试脚本比较简单,只需要一个模板就行,生成locust压测脚本则稍微负责点,我是分拆成多个模板,然后整合到一个模板。

生成的脚本都规范放在目录里:

基于locust全链路压测系统

 

生成脚本目录结构:

 基于locust全链路压测系统

 

 生成压测脚本示例:

 1 # -*- coding = utf-8 -*-
 2 # ------------------------------
 3 # @time: 2021-04-19 13:43:10.380837
 4 # @Author: drew_gg
 5 # @File: liao_bao.py
 6 # @Software: api_locust
 7 # ------------------------------
 8 
 9 
10 from locust import SequentialTaskSet, task, constant, tag, TaskSet
11 from locust.contrib.fasthttp import FastHttpUser
12 
13 
14 class LiaoBao20210419(TaskSet):
15 
16     @task(1)
17     @tag('api_getlist')
18     def api_getlist(self):
19         headers = {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'tenantId': '7'}
20         # 请求参数组装 ## r_url:固定参数
21         r_url = "/getList?vno=6.4.0"
22         requests_data = {'account': 'E2247B94-51E2-4952-BC06-24752911C060', 'client': 'iOS', 'data': '{"operation_type":0,"news_id":0,xxxxxxxxxxxxxxxxxxx'}
23         # 发起请求
24         with self.client.post(r_url, data=requests_data, catch_response=True, name=r_url) as r:
25             if r.content == b"":
26                 r.failure("No data")
27             if r.status_code != 200:
28                 em = "request error --" + str(r.status_code)
29                 r.failure(em)
30 
31     @task(4)
32     @tag('api_getsysnotice')
33     def api_getsysnotice(self):
34         headers = {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'tenantId': '7'}
35         # 请求参数组装 ## r_url:固定参数
36         r_url = "/getSysnotice?vno=6.4.0"
37         requests_data = {'account': 'E251179A-6309-4326-9827-73C892131605', 'client': 'iOS', 'data': '{"page_size":15,"page":1}', xxxxxxxxxxxxxxxxxxxxxxxx}
38         # 发起请求
39         with self.client.post(r_url, data=requests_data, catch_response=True, name=r_url) as r:
40             if r.content == b"":
41                 r.failure("No data")
42             if r.status_code != 200:
43                 em = "request error --" + str(r.status_code)
44                 r.failure(em)
45 
46     @task(4)
47     @tag('api_user_preparecancelaccount')
48     def api_user_preparecancelaccount(self):
49         headers = {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'tenantId': '7'}
50         # 请求参数组装 ## r_url:固定参数
51         r_url = "/user/prepareCancelAccount?vno=6.4.0"
52         requests_data = {'account': '2FF3D47C-995B-4D7E-93CD-58B4F1E94B74', 'client': 'iOS', 'data': '{}', xxxxxxxxxxxxxxxxxxxxxxx}
53         # 发起请求
54         with self.client.post(r_url, data=requests_data, catch_response=True, name=r_url) as r:
55             if r.content == b"":
56                 r.failure("No data")
57             if r.status_code != 200:
58                 em = "request error --" + str(r.status_code)
59                 r.failure(em)
60 
61 
62 class liao_bao_locust(FastHttpUser):
63     host = "https://xxxxxx.xxxxx.com"
64     wait_time = constant(0)
65     tasks = {LiaoBao20210419: 1}

生成好脚本后,需要生成执行命令:

 1 # -*- coding = utf-8 -*-
 2 # ------------------------------
 3 # @time: 2021/3/3 11:08
 4 # @Author: drew_gg
 5 # @File: locust_create_cmd.py
 6 # @Software: cover_app_platform
 7 # ------------------------------
 8 
 9 
10 def create_master_cmd(locust_pra):
11     """
12     生成master命令
13     :param locust_pra:
14     :return:
15     """
16     # locust master 命令样式:
17     """    
18         locust -f /work/locust/api_locust/locust_view/fm_api/locust_api/locust_fm_640.py  
19         --master  
20         --master-bind-port 9800 
21         --headless 
22         -u 600 
23         -r 200 
24         --expect-worker 16 
25         -t 10m 
26         -s 10  
27         --csv /work/locust/locust_report/fm/locust_get_dynamic.py0223145309 
28         --html /work/locust/api_locust/resource/html/new_html/locust_get_operation_parm.html
29     """
30     run_port = '9800'
31     master_cmd = "locust -f %s  --master  --master-bind-port %s --headless " % (locust_pra['to_file'], run_port)
32     master_pra = "-u %s -r %s --expect-worker %s -t %ss -s 10  --csv %s --html %s > %s" % \
33                  (locust_pra['user'], locust_pra['rate'], locust_pra['thread'], locust_pra['time'], locust_pra['csv'],
34                   locust_pra['html'], locust_pra['master_log'])
35     master_cmd = master_cmd + master_pra
36     return master_cmd
37 
38 
39 def create_slave_cmd(locust_pra):
40     """
41     生成slave命令
42     :return:
43     """
44     run_port = '9800'
45     if len(locust_pra['api']) == 1 and locust_pra['api'][0] == '':
46         slave_cmd = "locust -f %s --master-host  %s --master-port %s --headless --worker > %s" % \
47                     (locust_pra['to_file'], locust_pra['master'].split('-')[0], run_port, locust_pra['slave_log'])
48     else:
49         tags = ''
50         for i in locust_pra['api']:
51             tags += i.split(".py")[0] + ' '
52         slave_cmd = "locust -f %s --master-host  %s --master-port %s --headless --worker -T %s > %s" % \
53                     (locust_pra['to_file'], locust_pra['master'].split('-')[0], run_port, tags, locust_pra['slave_log'])
54     return slave_cmd

然后把文件推送到服务器上,服务器也需要有规定的目录:

每台压测机上建立三个目录:

基于locust全链路压测系统

 

 master上存储压测生成的报告、csv文件,然后写个定时程序拉去报告到项目服务器,压测完后可直接查询报告。

基于locust全链路压测系统

 

 这部分处理业务稍微复杂(代码贴了flask的一些页面和主要处理逻辑,感兴趣的可以具体讨论):

  1 # -*- coding = utf-8 -*-
  2 # ------------------------------
  3 # @time: 2021/6/10 11:38
  4 # @Author: drew_gg
  5 # @File: locust_view.py
  6 # @Software: cover_app_platform
  7 # ------------------------------
  8 
  9 import os
 10 import time
 11 import requests
 12 import datetime
 13 from app.dao import read_sql as sql
 14 from flask import Blueprint, render_template, request, url_for, redirect, send_file, jsonify, session
 15 from app.locust.create_locust_run import create_linux_run as cl
 16 from app.locust.run_locust import locust_create_cmd as lc
 17 from app.dao import flask_mysql as har_sql
 18 from app.common import common_run_cmd as cmd
 19 from app.common.common_remote_link import remote_linux as ssh_linux
 20 from flask_login import login_required
 21 from app.dao import zen_tao_read_sql as zt_sql
 22 
 23 
 24 pl = os.getcwd().split('cover_app_platform')
 25 path_root = pl[0] + 'cover_app_platform'
 26 path_upload_path = pl[0] + r'cover_app_platform\\app\\file_upload\\'
 27 linux_user = "root"
 28 linux_key = pl[0] + r'cover_app_platform\app\\common\key\linux_key'
 29 tem_master = pl[0] + r'cover_app_platform\app\locust\template_file\locust_template_master.py'
 30 tem_slave = pl[0] + r'cover_app_platform\app\locust\template_file\locust_template_slave.py'
 31 save_report = pl[0] + r'cover_app_platform\\app\\views\\report\\'
 32 
 33 
 34 locust_v = Blueprint('locust_v', '__main__')
 35 
 36 
 37 @locust_v.route('/ywzx/har_upload/')
 38 @login_required
 39 def upload_index():
 40     """
 41     上传页面
 42     :return:
 43     """
 44     return render_template('/post/upload/upload_har.html')
 45 
 46 
 47 @locust_v.route('/locust', methods=['POST', 'GET'])
 48 @login_required
 49 def get_har_file():
 50     """
 51     性能测试主页
 52     :return:
 53     """
 54     un = [session.get('username').split('-')[0]]
 55     u = zt_sql.deal_mysql("get_zen_tao_group.sql", un)
 56     new_mysql = []
 57     # 判断权限,测试角色和管理员角色才能看到所有的记录
 58     if u[0][1] not in ['qa', 'qd', 'td', 'admin'] and un[0].lower() not in ['virgil', 'frankie', 'steven']:
 59         mysql_l = sql.deal_mysql('get_har_file_cs.sql')
 60         for i in mysql_l:
 61             new_mysql.append(list(i))
 62     else:
 63         # alter table app_pkg.har_info auto_increment=1;
 64         mysql_l = sql.deal_mysql('get_har_file.sql')
 65         for i in mysql_l:
 66             new_mysql.append(list(i))
 67     return render_template('/post/locust/ywzx.html', data=new_mysql)
 68 
 69 
 70 @locust_v.route('/har/<har_name>')
 71 @login_required
 72 def get_locust_debug(har_name):
 73     """
 74     API接口调试主页
 75     :param har_name:
 76     :return:
 77     """
 78     par_har = []
 79     har_dic = {'upload_id': har_name.split('_')[0], 'har_name': har_name.split('_')[1]}
 80     par_har.append(har_dic)
 81     r = sql.deal_more_mysql("get_locust_debug.sql", par_har)
 82     new_mysql = []
 83     for i in r:
 84         new_mysql.append(list(i))
 85     return render_template('/post/locust/locust_debug.html', data=new_mysql)
 86 
 87 
 88 @locust_v.route('/py_detail/<py_name>')
 89 @login_required
 90 def get_py_detail(py_name):
 91     """
 92     获取脚本代码详情
 93     :param py_name:
 94     :return:
 95     """
 96     f_name = py_name.split(',')[-1]
 97     file_path = path_root + py_name.replace(',', '\\')
 98     with open(file_path, 'r', encoding='utf-8') as f:
 99         f = f.read()
100     return render_template('/post/locust/py_detail.html', data=f, pn=f_name, yn=py_name, path=file_path.split('\\'))
101 
102 
103 @locust_v.route('/run_detail/<py_name>')
104 @login_required
105 def get_run_detail(py_name):
106     """
107     执行代码并格式化显示代码
108     :param py_name:
109     :return:
110     """
111     f_name = py_name.split(',')[-1]
112     file_path = path_root + py_name.replace(',', '\\')
113     run_cmd = "python " + file_path
114     run_result = cmd.run_cmd(run_cmd)
115     tempters = run_result._stream
116     # 这个地方坑死啊,换成utf-8中文会乱码,换成gbk就ok,无语无语!!
117     result = tempters.buffer.read().decode(encoding='gbk', errors='ignore')
118     replace_result = result
119     if "<" in result or ">" in result:
120         replace_result = result.replace('"', '\\"')
121     replace_result = replace_result.replace('"', '')
122     replace_result = replace_result.replace('\'', '"')
123     replace_result = replace_result.replace('\r', '')
124     replace_result = replace_result.replace('\n', '')
125     replace_result = replace_result.replace('False', 'false')
126     replace_result = replace_result.replace('True', 'true')
127     replace_result = replace_result.replace('"[', '[')
128     replace_result = replace_result.replace(']"', ']')
129     if replace_result == '':
130         replace_result = {"data": '******接口无数据返回********'}
131     return render_template('/post/locust/run_detail.html', data=replace_result, pn=f_name)
132 
133 
134 @locust_v.route('/locust_run/<py_name>', methods=['POST', 'GET'])
135 @login_required
136 def locust_run(py_name):
137     """
138     压测配置页面
139     :param py_name:
140     :return:
141     """
142     par_har = []
143     har_dic = {'upload_id': py_name.split('$$')[0], 'har_name': py_name.split('$$')[2]}
144     master_path = path_root.replace('\\', ',') + py_name.split('$$')[3]
145     par_har.append(har_dic)
146     result_debug = sql.deal_more_mysql("get_locust_debug.sql", par_har)
147     new_mysql = []
148     project_name = py_name.split('$$')[0] + '_' + py_name.split('$$')[1] + '_' + py_name.split('$$')[2]
149     for i in range(len(result_debug)):
150         data_dic = {'id': i, 'title': result_debug[i][4].split('\\')[-1]}
151         new_mysql.append(data_dic)
152 
153     result_server = sql.deal_more_mysql("get_server_info.sql", par_har)
154     server_all = []
155     for s in range(len(result_server)):
156         ip = result_server[s][1] + '-' + result_server[s][2]
157         server = {'id': s, 'title': ip}
158         server_all.append(server)
159     return render_template('/post/locust/run_locust.html', data=new_mysql, master_path=master_path, project_name=project_name, server=server_all)
160 
161 
162 @locust_v.route('/report/locust/<lp>', methods=['POST', 'GET'])
163 @login_required
164 def locust_perform(lp):
165     """
166     执行压测
167     :param lp:
168     :return:
169     """
170     run_status = sql.deal_mysql("get_run_status.sql")
171     if run_status:
172         run_project = run_status[0][0]
173         run_time = run_status[0][1]
174         return "已有【%s】项目正在压测,请【%s】后再申请压测" % (run_project, run_time)
175     elif request.method == 'POST' and request.view_args:
176         locust_pra = {
177             "api":  lp.split('&')[0].split(','),
178             "master": lp.split('&')[1],
179             "slave": lp.split('&')[2].split(','),
180             "thread": lp.split('&')[3],
181             "user": lp.split('&')[4],
182             "rate": lp.split('&')[5],
183             "time": lp.split('&')[6],
184             "master_path": lp.split('&')[7].replace(',', '\\'),
185             "har_id": lp.split('&')[8].split('_')[0],
186             "project_name": lp.split('&')[8].split('_')[1],
187             "har_name": lp.split('&')[8].split('_')[2]
188         }
189         # 需要上传的主locust文件
190         from_file = locust_pra['master_path']
191         # linux服务器上存放script的目录:项目名称/时间
192         script_to_path = '/work/locust/locust_run/locust_script/' + locust_pra['project_name'] + '/' + locust_pra['master_path'].split('\\')[-2]
193         # linux服务器上存放slave的目录:项目名称/时间
194         slave_to_path = '/work/locust/locust_run/locust_slave/' + locust_pra['project_name'] + '/' + locust_pra['master_path'].split('\\')[-2]
195         # linux服务器上存放master的目录:项目名称/时间
196         master_to_path = '/work/locust/locust_run/locust_master/' + locust_pra['project_name'] + '/' + locust_pra['master_path'].split('\\')[-2]
197         # linux服务器上存放script的文件目录
198         to_file = script_to_path + '/' + locust_pra['master_path'].split('\\')[-1]
199         # linux服务器master端存放html的目录
200         to_file_report = master_to_path + '/' + locust_pra['master_path'].split('\\')[-1].split('.py')[0] + '.html'
201         # linux服务器master端存放执行日志的目录
202         to_file_log_master = master_to_path + '/' + locust_pra['master_path'].split('\\')[-1].split('.py')[0] + '.log'
203         # linux服务器slave端存放html目录
204         to_file_log_slave = slave_to_path + '/' + locust_pra['master_path'].split('\\')[-1].split('.py')[0] + '.log'
205         # linux服务器master端存放csv的目录
206         to_file_csv = master_to_path + '/' + locust_pra['master_path'].split('\\')[-1].split('.py')[0]
207         # 相关字段存入字典
208         locust_pra['to_file'] = to_file
209         locust_pra['html'] = to_file_report
210         locust_pra['csv'] = to_file_csv
211         locust_pra['master_log'] = to_file_log_master
212         locust_pra['slave_log'] = to_file_log_slave
213         # 把项目路径的主路径存入字典,用于存放run_master.py和run_slave.py
214         locust_pra['project_path'] = locust_pra['master_path'].split(locust_pra['master_path'].split('\\')[-1])[0]
215         # 生成master脚本
216         master_cmd = lc.create_master_cmd(locust_pra)
217         run_master = locust_pra['project_path'] + 'run_master.py'
218         cl.create_linux_run(tem_master, run_master, master_cmd)
219         # 生成slave脚本
220         slave_cmd = lc.create_slave_cmd(locust_pra)
221         run_slave = locust_pra['project_path'] + 'run_slave.py'
222         thread = round(int(locust_pra['thread'])/len(locust_pra['slave']))
223         cl.create_linux_run(tem_slave, run_slave, slave_cmd, thread)
224         # 创建脚本文件推送目录
225         cmd_script_create_script = "cd /work/locust/locust_run/locust_script/; mkdir -p " + locust_pra['project_name'] \
226                                    + '/' + locust_pra['master_path'].split('\\')[-2] + "; chmod - R777 " + locust_pra['project_name'] + ";"
227         # 创建slave执行文件目录
228         cmd_slave_create_script = "cd /work/locust/locust_run/locust_slave/; mkdir -p " + locust_pra['project_name'] \
229                                   + '/' + locust_pra['master_path'].split('\\')[-2] + "; chmod - R777 " + locust_pra['project_name'] + ";"
230         # 创建master执行文件目录
231         cmd_master_create_script = "cd /work/locust/locust_run/locust_master/; mkdir -p " + locust_pra['project_name'] \
232                                    + '/' + locust_pra['master_path'].split('\\')[-2] + "; chmod - R777 " + locust_pra['project_name'] + ";"
233         # linux服务器上存放run_master.py的目录
234         to_run_master_file = master_to_path + '/' + 'run_master.py'
235         to_run_master_cmd = 'python ' + master_to_path + '/' + 'run_master.py'
236         # linux服务器上存放run_slave.py的目录
237         to_run_slave_file = slave_to_path + '/' + 'run_slave.py'
238         to_run_slave_cmd = 'python ' + slave_to_path + '/' + 'run_slave.py'
239         # 执行master脚本的服务器ip
240         host_master = locust_pra['master'].split('-')[1]
241         # 生成host集合
242         host_slave = []
243         for i in locust_pra['slave']:
244             host_slave.append(i.split('-')[1])
245         host_all = host_slave + [host_master]
246         host_l = list(set(host_all))
247         # 完成linux远程连接上传文件和执行命令
248         start_time = datetime.datetime.now()
249         for con in host_l:
250             # 创建连接对象
251             master_ssh = ssh_linux.SSHConnection(con, linux_user, linux_key)
252             # 开始连接
253             master_ssh.connect()
254             # 执行创建locust_script下的项目路径
255             master_ssh.run_cmd(cmd_script_create_script)
256             time.sleep(2)
257             # 把主压测脚本文件上传到服务器上
258             master_ssh.upload(from_file, to_file)
259             # 如果是master服务器
260             if con == host_master:
261                 # locust_master下创建项目路径
262                 master_ssh.run_cmd(cmd_master_create_script)
263                 time.sleep(2)
264                 # 上传run_master.py到服务器上
265                 master_ssh.upload(run_master, to_run_master_file)
266                 time.sleep(2)
267                 start_time = datetime.datetime.now()
268                 master_ssh.run_cmd(to_run_master_cmd)
269             if con in host_slave:
270                 # locust_slave下创建项目路径
271                 master_ssh.run_cmd(cmd_slave_create_script)
272                 time.sleep(2)
273                 # 上传run_slave.py到服务器上
274                 master_ssh.upload(run_slave, to_run_slave_file)
275                 time.sleep(2)
276                 master_ssh.run_cmd(to_run_slave_cmd)
277         master_ssh.close()
278         in_sql = "XXXXXX"314         d = har_sql.Database()
315         d.exec_no_query(in_sql)
316         history_url = locust_pra['har_id'] + '_' + locust_pra['har_name']
317     return redirect(url_for("locust_v.locust_report", py_name=history_url))
318 
319 
320 @locust_v.route('/locust_report/<py_name>', methods=['POST', 'GET'])
321 @login_required
322 def locust_report(py_name):
323     """
324     压测报告处理
325     :param py_name:
326     :return:
327     """
328     par_har = []
329     har_dic = {'har_id': py_name.split('_')[0], 'har_name': py_name.split('_')[1]}
330     par_har.append(har_dic)
331     sql_result = sql.deal_more_mysql("get_locust_history.sql", par_har)
332     new_mysql = []
333     for i in sql_result:
334         new_mysql.append(list(i))
335     return render_template('/post/locust/locust_history_report.html', data=new_mysql, py_name=py_name)
336 
337 
338 @locust_v.route('/report/<report_id>', methods=['POST', 'GET'])
339 @login_required
340 def locust_report_detail(report_id):
341     """
342     压测报告处理
343     :param report_id:
344     :return:
345     """
346     html_file = report_id.split('$')[0] + '\\' + report_id.split('$')[1].replace('-', '').replace(':', '').replace(' ', '-') + '\\' + report_id.split('$')[2]
347     html = save_report + html_file
348     return send_file(html)
349 
350 
351 @locust_v.route('/server/<report_id>', methods=['POST', 'GET'])
352 @login_required
353 def locust_server(report_id):
354     server = {}
355     if len(report_id.split('$$')) > 5 and report_id.split('$$')[4] != '' and len(report_id.split('$$')[1].split('.')) > 2 and len(report_id.split('$$')[2].split('.')) > 2:
356         server['type'] = report_id.split('$$')[0]
357         server['intranet_ip'] = report_id.split('$$')[1]
358         server['external_ip'] = report_id.split('$$')[2]
359         server['ram'] = report_id.split('$$')[3]
360         server['cpu_threads'] = report_id.split('$$')[4]
361         server['bandwidth'] = report_id.split('$$')[5]
362         server['supplier'] = report_id.split('$$')[6]
363         sql_server = "XXXXXXX"397         d = har_sql.Database()
398         try:
399             d.exec_no_query(sql_server)
400         except Exception as e:
401             print(e)
402             return "IP已经存在!"
403     sql_result = sql.deal_mysql("get_server_info.sql")
404     new_mysql = []
405     for i in sql_result:
406         new_mysql.append(list(i))
407     return render_template('/post/locust/locust_server.html', data=new_mysql)
408 
409 
410 @locust_v.route('/locust_stop/<report_id>', methods=['POST', 'GET'])
411 @login_required
412 def locust_stop(report_id):
413     """
414     压测报告处理
415     :param report_id:
416     :return:
417     """
418     locust_run_id = report_id.split('$')[0]
419     py_name = report_id.split('$')[1]
420     sql_ip = sql.deal_mysql("get_run_ip.sql", list([locust_run_id]))
421 
422     run_ip = [sql_ip[0][0].split('-')[1]]
423     for ip in sql_ip[0][1].split(','):
424         run_ip.append(ip.split("'")[1].split('-')[1])
425     run_ip = list(set(run_ip))
426     try:
427         for s in run_ip:
428             master_ssh = ssh_linux.SSHConnection(s, linux_user, linux_key)
429             # 开始连接
430             master_ssh.connect()
431             # 执行创建locust_script下的项目路径
432             stop_locust_cmd = "killall -9 locust"
433             master_ssh.run_cmd(stop_locust_cmd)
434             master_ssh.close()
435         update_status = """UPDATE locust_run set `status`=2 WHERE id = '%s' and `status`not in('1', '2') and is_delete = 0 limit 1 ;""" % locust_run_id
436         d = har_sql.Database()
437         d.exec_no_query(update_status)
438     except Exception as e:
439         print(e)
440     return redirect(url_for("locust_v.locust_report", py_name=py_name))
441 
442 
443 @locust_v.route('/save/', methods=['POST', 'GET'], strict_slashes=False)
444 @login_required
445 def save_content():
446     """448     :return:
449     """
450     # 这个地方得用ajax发起post请求
451     # get请求:当发送数据时,GET方法向URL添加数据;URL的长度是受限制的(URL的最大长度是2048个字符),post请求:没有限制。
452     content = request.get_json()
453     path = content.split('$$')[0].replace(',', '\\')
454     content = content.split('$$')[1]
455     # 解析16进制为字符串
456     py_content = ''
457     try:
458         for b in content.split(','):
459             for i in [int(b, 16)]:
460                 py_content += chr(i)
461     except Exception as e:
462         print(e)
463     move_cmd = "move " + path + ' ' + path.split('.py')[0] + '_copy.py'
464     os.system(move_cmd)
465     # 忽略特殊字符导致的转码错误
466     """
467         s.encode('utf-8', 'replace').decode('utf-8')
468         如果设置为ignore,则会忽略非法字符;
469         如果设置为replace,则会用?取代非法字符;
470         如果设置为xmlcharrefreplace,则使用XML的字符引用。
471     """
472     py_content = py_content.encode('utf-8', 'ignore').decode('utf-8')
473     try:
474         with open(path, 'w', encoding='utf-8') as f:
475             f.write(py_content)
476     except Exception as e:
477         print(e)
478     py_name = ',app' + path.split('app')[2].replace('\\', ',')
479     return redirect(url_for("locust_v.get_py_detail", py_name=py_name))
480 
481 
482 @locust_v.route('/api_create', methods=['POST', 'GET'], strict_slashes=False)
483 @login_required
484 def create_api():
485     """
486     首页
487     :return:
488     """
489     if request.json:
490         req_data = request.json
491         url = req_data["http"] + '://' + req_data['host'] + req_data['url']
492         data = req_data['params']
493         headers = req_data['headers']
494         try:
495             if req_data['method'] == 'post':
496                 r = requests.post(url, data=data, headers=headers)
497             if req_data['method'] == 'get':
498                 r = requests.get(url, params=data, headers=headers)
499         except Exception as e:
500             return jsonify({"error": '请求异常', "msg": "请求失败", "data": e.__context__.args[0]})
501         try:
502             data = r.json()
503         except Exception as e:
504             data = r.text
505             print(e)
506         return jsonify({"success": 200, "msg": "请求成功", "data": data})
507     else:
508         return render_template('/post/locust/create_api.html')

 

平台主要界面:

1.首页

基于locust全链路压测系统

2.上传并解析har文件页面

基于locust全链路压测系统

 

 3.压测脚本在线编辑执行页面

基于locust全链路压测系统

 

 

4.接口调试页面

基于locust全链路压测系统

 

 5.调试结果页

基于locust全链路压测系统

 

 6.压测配置页面

基于locust全链路压测系统

 

 7.压测执行及记录页面

基于locust全链路压测系统

 

 8.压测报告页面

基于locust全链路压测系统

 

 

9.服务器管理页面

基于locust全链路压测系统

 

 大致包含这些功能,当然,项目搭建过程中遇到各种坑,要尝试才知道,后续打算优化一下代码,再升级几个版本,也算彻底搞定。

欢迎感兴趣的一起研究讨论。

 

上一篇:7-1 求解买股票问题


下一篇:5.Mock参数化