通过手工执行kubectl scale命令,可以实现pod扩容或缩容。但是不符合对kubernetes的定位目标——自动化、智能化。分布式系统要能够根据当前负载的变化情况自动触发水平扩展或者缩容行为,因为这一过程可能是频繁发生、不可预料的,所以手动控制的方式不现实。
Horizontal Pod Autoscaler(pod横向自动扩容,简称HPA)。HPA与RC、Deployment一样,也属于一种kubernetes资源对象。通过跟踪分析RC控制的所有目标pod的负载变化情况,来确定是否需要针对性的调整目标Pod的副本数,这是HPA的实现原理。HPA用于实现基于CPU使用率进行自动pod扩容和缩容的功能。HPA控制器基于Master的kube-controller-manager服务启动参数--horizontal-pod-autoscaler-sync-period定义的时长(默认值为30s),周期性的监测目标pod的cpu使用率,并在满足条件时对ReplicationController或者Deployment中的pod的副本数量进行调整,以符合用户定义的平均pod cpu使用率。pod cpu使用率来自于heapster组件。
创建HPA时可以使用kubectl autoscale命令进行快速创建或者使用yaml配置文件进行创建。在创建HPA之前,需要已经存在一个deployment/RC对象,并且该Deployment/RC中的pod必须定义resources.requests.cpu的资源请求值,如果不设置该值,则heapster将无法采集到该pod的cpu使用情况,会导致HPA无法正常工作。
为一个RC设置HPA,然后使用一个客户端对其进行压力测试。
[root@kub_master ~]# cd k8s/ [root@kub_master k8s]# mkdir hpa [root@kub_master k8s]# cd hpa [root@kub_master hpa]# vim nginx-rc.yaml [root@kub_master hpa]# cat nginx-rc.yaml apiVersion: v1 kind: ReplicationController metadata: name: myweb spec: replicas: 3 selector: app: myweb template: metadata: labels: app: myweb spec: containers: - name: myweb image: 192.168.0.212:5000/nginx:1.15 imagePullPolicy: IfNotPresent ports: - containerPort: 80 resources: limits: cpu: 100m memory: 50Mi requests: cpu: 100m memory: 50Mi [root@kub_master hpa]# kubectl create -f nginx-rc.yaml replicationcontroller "myweb" created [root@kub_master hpa]# kubectl get all -o wide NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR rc/myweb 3 3 3 59s myweb 192.168.0.212:5000/nginx:1.15 app=myweb rc/nginx 2 2 2 2d nginx 192.168.0.212:5000/nginx:1.15 app=nginx,version=1.15 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR svc/kubernetes 192.168.0.1 <none> 443/TCP 8d <none> NAME READY STATUS RESTARTS AGE IP NODE po/myweb-2jmrf 1/1 Running 0 59s 172.16.81.6 192.168.0.212 po/myweb-lq299 1/1 Running 0 59s 172.16.46.3 192.168.0.184 po/myweb-r3446 1/1 Running 0 59s 172.16.66.3 192.168.0.208 po/nginx-bp61w 1/1 Running 0 1d 172.16.81.3 192.168.0.212 po/nginx-sg498 1/1 Running 0 1d 172.16.66.2 192.168.0.208
接下来为RC“myweb”创建一个HPA控制器,在1和8之间调整pod的副本数量,以使得平均pod cpu使用率维持在10%
使用kubectl autoscale命令进行创建
[root@kub_master hpa]# kubectl autoscale replicationcontroller myweb --max=8 --min=1 --cpu-percent=10 replicationcontroller "myweb" autoscaled [root@kub_master hpa]# kubectl get all -o wide NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE hpa/myweb ReplicationController/myweb 10% 0% 1 8 13s NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR rc/myweb 1 1 1 10m myweb 192.168.0.212:5000/nginx:1.15 app=myweb rc/nginx 2 2 2 2d nginx 192.168.0.212:5000/nginx:1.15 app=nginx,version=1.15 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR svc/kubernetes 192.168.0.1 <none> 443/TCP 8d <none> NAME READY STATUS RESTARTS AGE IP NODE po/myweb-lq299 1/1 Running 0 10m 172.16.46.3 192.168.0.184 po/nginx-bp61w 1/1 Running 0 1d 172.16.81.3 192.168.0.212 po/nginx-sg498 1/1 Running 0 1d 172.16.66.2 192.168.0.208
或者通过yaml配置文件创建HPA,需要在scaleTargetRef字段指定需要管理的Deployment/RC的名字,然后设置minReplicas、maxReplicas和targetCPUUtilizationPercentage参数。
[root@kub_master hpa]# vim hpa-myweb.yaml [root@kub_master hpa]# cat hpa-myweb.yaml apiVersion: autoscaling/v1 kind: HorizontalPodAutoscaler metadata: name: myweb spec: scaleTargetRef: apiVersion: apps/v1beta1 kind: ReplicationController name: myweb minReplicas: 1 maxReplicas: 8 targetCPUUtilizationPercentage: 10
[root@kub_master hpa]# kubectl create -f hpa-myweb.yaml
[root@kub_master hpa]# kubectl get hpa -o wide NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE myweb ReplicationController/myweb 10% 0% 1 8 9m
压力测试:
#压力测试工具命令
[root@kub_master hpa]# yum install httpd-tools -y
[root@kub_master hpa]# kubectl get all -o wide NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE hpa/myweb ReplicationController/myweb 10% 0% 1 8 41m NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR rc/myweb 1 1 1 51m myweb 192.168.0.212:5000/nginx:1.15 app=myweb rc/nginx 2 2 2 2d nginx 192.168.0.212:5000/nginx:1.15 app=nginx,version=1.15 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR svc/kubernetes 192.168.0.1 <none> 443/TCP 9d <none> NAME READY STATUS RESTARTS AGE IP NODE po/myweb-lq299 1/1 Running 0 51m 172.16.46.3 192.168.0.184 po/nginx-bp61w 1/1 Running 0 1d 172.16.81.3 192.168.0.212 po/nginx-sg498 1/1 Running 0 1d 172.16.66.2 192.168.0.208 #压测命令
[root@kub_master hpa]# ab -n 500000 -c 100 http://172.16.46.3/index.html This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 172.16.46.3 (be patient) Completed 50000 requests Completed 100000 requests
#扩容的
[root@kub_master hpa]# kubectl get all -o wide NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE hpa/myweb ReplicationController/myweb 10% 74% 1 8 42m NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR rc/myweb 4 4 4 53m myweb 192.168.0.212:5000/nginx:1.15 app=myweb rc/nginx 2 2 2 2d nginx 192.168.0.212:5000/nginx:1.15 app=nginx,version=1.15 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR svc/kubernetes 192.168.0.1 <none> 443/TCP 9d <none> NAME READY STATUS RESTARTS AGE IP NODE po/myweb-4292v 1/1 Running 0 11s 172.16.81.7 192.168.0.212 po/myweb-bw6sj 1/1 Running 0 11s 172.16.66.3 192.168.0.208 po/myweb-lq299 1/1 Running 0 53m 172.16.46.3 192.168.0.184 po/myweb-x6x5g 1/1 Running 0 11s 172.16.81.6 192.168.0.212 po/nginx-bp61w 1/1 Running 0 1d 172.16.81.3 192.168.0.212 po/nginx-sg498 1/1 Running 0 1d 172.16.66.2 192.168.0.208
可以看到HPA已经根据Pod的CPU使用率的提高对RC进行了自动扩容。
#缩容
[root@kub_master hpa]# kubectl get all -o wide NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE hpa/myweb ReplicationController/myweb 10% 0% 1 8 47m NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR rc/myweb 1 1 1 58m myweb 192.168.0.212:5000/nginx:1.15 app=myweb rc/nginx 2 2 2 2d nginx 192.168.0.212:5000/nginx:1.15 app=nginx,version=1.15 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR svc/kubernetes 192.168.0.1 <none> 443/TCP 9d <none> NAME READY STATUS RESTARTS AGE IP NODE po/myweb-lq299 1/1 Running 0 58m 172.16.46.3 192.168.0.184 po/nginx-bp61w 1/1 Running 0 1d 172.16.81.3 192.168.0.212 po/nginx-sg498 1/1 Running 0 1d 172.16.66.2 192.168.0.208
#压测的整个过程
[root@kub_master hpa]# ab -n 500000 -c 100 http://172.16.46.3/index.html This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 172.16.46.3 (be patient) Completed 50000 requests Completed 100000 requests Completed 150000 requests Completed 200000 requests Completed 250000 requests Completed 300000 requests Completed 350000 requests Completed 400000 requests Completed 450000 requests Completed 500000 requests Finished 500000 requests Server Software: nginx/1.15.5 Server Hostname: 172.16.46.3 Server Port: 80 Document Path: /index.html Document Length: 612 bytes Concurrency Level: 100 Time taken for tests: 177.998 seconds Complete requests: 500000 Failed requests: 0 Write errors: 0 Total transferred: 422500000 bytes HTML transferred: 306000000 bytes Requests per second: 2809.02 [#/sec] (mean) Time per request: 35.600 [ms] (mean) Time per request: 0.356 [ms] (mean, across all concurrent requests) Transfer rate: 2317.99 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 9 95.8 1 7016 Processing: 0 26 65.5 3 5097 Waiting: 0 25 63.8 3 5097 Total: 1 35 119.1 5 7086 Percentage of the requests served within a certain time (ms) 50% 5 66% 6 75% 13 80% 73 90% 82 95% 88 98% 210 99% 409 100% 7086 (longest request)
可以看到HPA根据Pod CPU使用率的降低对副本数量进行了缩容操作,pod副本数量变成了1个。
当前的HPA还只支持将CPU使用率作为pod副本扩容和缩容的触发条件,在将来的版本中,将会支持应用自定义指标作为触发条件。