Horizontal Pod Autoscaling,简称HPA,是Kubernetes中实现Pod水平自动伸缩功能的资源对象。Kubernetes 1.9 或更高版本支持HPA V2版本,允许我们是用自定义监控指标来控制Pod数目。
本文用阿里云容器服务来介绍HPA使用自定义指标进行弹性伸缩的功能。
安装配置Prometheus和Prometheus adaptor
我们使用Prometheus收集指标,通过Prometheus adaptor来扩展Kubernetes 自定义监控指标。
使用阿里云容器服务控制台创建好集群,并创建monitoring命名空间。然后进行如下步骤:
安装Prometheus
在应用目录-App Hub中搜索prometheus可以看到如下组件
点击prometheus进入安装页面,点击参数进行查看并配置(本文persistentVolume enabled设置为false,即使用emptyDir),选择集群命名空间后点击安装。
安装Prometheus adaptor
安装过程同上,注意配置参数中prometheus url要使用之前部署好的prometheus地址
验证安装是否成功
使用控制台的cloudshell或kubectl连接集群,执行如下命令:
kubectl api-versions
结果中可以看到如下内容,表明集群支持hpa v2并可以使用自定义监控指标
autoscaling/v2beta1
custom.metrics.k8s.io/v1beta1
我们可以进一步查看监控项,如pod支持的监控指标
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1" |jq . | grep "pods/"
"name": "pods/start_time_seconds",
"name": "pods/cpu_user",
"name": "pods/fs_limit_bytes",
"name": "pods/cpu_cfs_throttled",
"name": "pods/memory_failcnt",
"name": "pods/fs_writes_bytes",
"name": "pods/kube_pod_created",
"name": "pods/kube_pod_status_phase",
"name": "pods/tasks_state",
"name": "pods/last_seen",
"name": "pods/kube_pod_container_status_running",
"name": "pods/fs_reads_merged",
"name": "pods/memory_swap",
"name": "pods/spec_cpu_quota",
"name": "pods/fs_io_current",
"name": "pods/kube_pod_container_resource_requests_memory_bytes",
"name": "pods/kube_pod_container_status_waiting",
"name": "pods/fs_sector_reads",
"name": "pods/kube_pod_status_scheduled",
"name": "pods/kube_pod_container_status_waiting_reason",
"name": "pods/fs_reads_bytes",
"name": "pods/kube_pod_container_resource_requests_cpu_cores",
"name": "pods/kube_pod_container_status_terminated_reason",
"name": "pods/cpu_system",
"name": "pods/fs_read",
"name": "pods/fs_writes",
"name": "pods/kube_pod_owner",
"name": "pods/memory_mapped_file",
"name": "pods/spec_cpu_shares",
"name": "pods/kube_pod_container_resource_limits_cpu_cores",
"name": "pods/cpu_usage",
"name": "pods/kube_pod_container_resource_limits",
"name": "pods/kube_pod_info",
"name": "pods/memory_working_set_bytes",
"name": "pods/fs_writes_merged",
"name": "pods/memory_usage_bytes",
"name": "pods/kube_pod_container_resource_requests",
"name": "pods/fs_io_time_weighted",
"name": "pods/spec_cpu_period",
"name": "pods/spec_memory_reservation_limit_bytes",
"name": "pods/cpu_cfs_throttled_periods",
"name": "pods/kube_pod_container_resource_limits_memory_bytes",
"name": "pods/cpu_cfs_periods",
"name": "pods/kube_pod_status_ready",
"name": "pods/kube_pod_container_status_terminated",
"name": "pods/memory_failures",
"name": "pods/memory_rss",
"name": "pods/spec_memory_swap_limit_bytes",
"name": "pods/memory_cache",
"name": "pods/kube_pod_start_time",
"name": "pods/fs_io_time",
"name": "pods/fs_reads",
"name": "pods/fs_inodes_free",
"name": "pods/kube_pod_container_status_ready",
"name": "pods/fs_sector_writes",
"name": "pods/cpu_load_average_10s",
"name": "pods/spec_memory_limit_bytes",
"name": "pods/kube_pod_labels",
"name": "pods/kube_pod_container_status_restarts",
"name": "pods/kube_pod_container_info",
"name": "pods/fs_usage_bytes",
"name": "pods/kube_pod_completion_time",
"name": "pods/kube_pod_container_status_last_terminated_reason",
"name": "pods/kube_pod_status_scheduled_time",
"name": "pods/memory_max_usage_bytes",
"name": "pods/fs_inodes",
"name": "pods/fs_write",
部署Deployment
在容器服务控制台中点击无状态->使用镜像创建,创建并部署一个nginx deployment,配置如下
在访问设置中配置服务
创建成功后可以在无状态列表中找到名为nginx的deployment,点击详情可以查看到相关信息
创建HPA
在上述详情页中点击容器组水平伸缩器,点击创建,配置相关内容(指标使用pods,参考上述自定义监控指标内容填写一项,这里使用memory_usage_bytes,阈值10485760=10M)
创建成功后可以在列表中查看到nginx-hpa
测试并触发自动伸缩
使用压测工具ab进行压测
for a in `seq 1 50`; do ab -rSqd -c 100 -n 2000 http://ng.cfc7d1315bdd14d529a7f102c2163736d.cn-beijing.alicontainer.com/;done
等待一段时间,可以看到pod数量的变化
总结
通过 Prometheus 来监控应用程序并暴露出用于弹性伸缩的指标,使用HPA利用这些监控指标对应用进行弹性伸缩,可以应对一些突发状况,提高应用的可用性。