自建K8s上,如果部署了Nginx-Ingress,通常一些默认的参数有些可能需要优化下以便提升它的性能(阿里云之类的云厂商提供的Ingress是优化过的)。
我下面是自建的测试K8s上部署的ingress,这里贴下优化的地方:
kubectl get cm -n ingress-nginx
NAME DATA AGE
ingress-controller-leader-nginx 0 6d19h
ingress-nginx-controller 0 6d19h
nginx-ingress-controller 22 5d20h
1、修改些nginx的常用参数
kubectl get cm -n ingress-nginx nginx-ingress-controller -oyaml 下面是data的内容,部分可以再根据实际情况修改(例如日志文件的路径)
apiVersion: v1
data:
access-log-path: /var/log/nginx/access.log
disable-access-log: "false"
disable-ipv6: "true"
disable-ipv6-dns: "true"
enable-modsecurity: "false"
enable-multi-accept: "true"
enable-opentracing: "true"
error-log-level: notice
error-log-path: /var/log/nginx/error.log
generate-request-id: "true"
keep-alive: "60"
keep-alive-requests: "10000"
load-balance: round_robin
log-format-escape-json: "true"
max-worker-connections: "65535"
max-worker-open-files: "10240"
nginx-status-ipv4-whitelist: 0.0.0.0
reuse-port: "true"
upstream-keepalive-connections: "200"
upstream-keepalive-requests: "100"
upstream-keepalive-timeout: "60"
worker-processes: "4"
kind: ConfigMap
.... 其余部分忽略....
2、修改下nginx容器的内核参数
kubectl get deployments -n ingress-nginx -oyaml 注意这里加了个initContainers的配置,主要是修改一些内核参数,更适配Nginx的使用场景
dnsPolicy: ClusterFirst
initContainers:
- command:
- /bin/sh
- -c
- |
mount -o remount rw /proc/sys
sysctl -w net.core.somaxconn=65535
sysctl -w net.ipv4.tcp_tw_reuse=1
sysctl -w net.ipv4.ip_local_port_range="1024 65535"
sysctl -w fs.file-max=1048576
sysctl -w fs.inotify.max_user_instances=16384
sysctl -w fs.inotify.max_user_watches=524288
sysctl -w fs.inotify.max_queued_events=16384
image: busybox
imagePullPolicy: IfNotPresent
name: init-sysctl
resources: {}
securityContext:
capabilities:
add:
- SYS_ADMIN
drop:
- ALL
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
nodeSelector:
kubernetes.io/os: linux
restartPolicy: Always
schedulerName: default-scheduler
参考:
https://zhuanlan.zhihu.com/p/212620792
https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/
https://www.nginx.com/blog/tuning-nginx/
https://mp.weixin.qq.com/s/NAwz4dlsPuJnqfWYBHkfGg