Traefik2.2从坑出发

2023-10-03 22:55:16

Traefik2.2从坑出发

Traefik2.2安装

Traefik 2.2新增的功能如下：

  1. 支持了udp
  2. traefik2.2 支持使用K/V存储做为动态配置的源，分别是 consul, etcd, Redis, zookeeper
  3. 能够使用kubernetes CRD自定义资源定义UDP负载平衡 Ingre***outeUDP。
  4. 能够使用 rancher， consul catalog， docker和 marathon中的标签定义UDP的负载平衡
  5. 增加了对ingress注解的主持
  6. 将TLS存储功能 TLSStores添加到Kubernetes CRD中，使kubernetes用户无需使用配置文件和安装证书即可提供默认证书。
  7. 在日志中增加了http的请求方式,是http还是https
  8. 因为TLS的配置可能会影响CPU的使用率，因此增加了 TLS version和 TLS cipher使用的指标信息
  9. 当前的WRR算法对于权重不平衡端点存在严重的偏差问题，将EDF调度算法用于WeightedRoundRobin， Envoy也是使用了 EOF调度算法
  10. 支持请求主体用于流量镜像
  11. 增加了 ElasticAPM作为traefik的tracing系统。
  12. Traefik的Dashboard增加了UDP的页面
  13. Traefik也增加了黑暗主题

下面进行安装过程。

注：我们这里是将traefik部署在ingress-traefik命名空间，如果你需要部署在其他命名空间，需要更改资源清单，如果你是部署在和我同样的命令空间中，你需要创建该命名空间。

创建命名空间：

# kubectl create ns ingress-traefik

1、创建CRD资源

Traefik 2.0版本后开始使用CRD来对资源进行管理配置，所以我们需要先创建CRD资源。
traefik-crd.yaml

## Ingre***oute
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingre***outes.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Ingre***oute
    plural: ingre***outes
    singular: ingre***oute
---
## Ingre***outeTCP
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingre***outetcps.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Ingre***outeTCP
    plural: ingre***outetcps
    singular: ingre***outetcp
---
## Middleware
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
---
## TraefikService
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: traefikservices.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TraefikService
    plural: traefikservices
    singular: traefikservice

---
## TraefikTLSStore
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsstores.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSStore
    plural: tlsstores
    singular: tlsstore

---
## Ingre***outeUDP
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingre***outeudps.traefik.containo.us 
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Ingre***outeUDP
    plural: ingre***outeudps
    singular: ingre***outeudp

创建资源清单：

# kubectl apply -f traefik-crd.yaml
# kubectl get crd
NAME                                    CREATED AT
ingre***outes.traefik.containo.us       2019-12-13T05:40:30Z
ingre***outetcps.traefik.containo.us    2019-12-13T05:40:30Z
middlewares.traefik.containo.us         2019-12-13T05:40:30Z

2、创建RBAC权限

traefik-rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: ingress-traefik 
  name: traefik-ingress-controller
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups: [""]
    resources: ["services","endpoints","secrets"]
    verbs: ["get","list","watch"]
  - apiGroups: ["extensions"]
    resources: ["ingresses"]
    verbs: ["get","list","watch"]
  - apiGroups: ["extensions"]
    resources: ["ingresses/status"]
    verbs: ["update"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["middlewares"]
    verbs: ["get","list","watch"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["ingre***outes","traefikservices"]
    verbs: ["get","list","watch"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["ingre***outetcps","ingre***outeudps"]
    verbs: ["get","list","watch"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["tlsoptions","tlsstores"]
    verbs: ["get","list","watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: ingress-traefik

创建RBAC资源清单：

# kubectl apply -f traefik-rbac.yaml

3、创建traefik配置文件

traefik-config.yaml

kind: ConfigMap
apiVersion: v1
metadata:
  name: traefik-config
  namespace: ingress-traefik
data:
  traefik.yaml: |-
    serversTransport:
      insecureSkipVerify: true
    api:
      insecure: true
      dashboard: true
      debug: true
    metrics:
      prometheus: ""
    entryPoints:
      web:
        address: ":80"
      websecure:
        address: ":443"
    providers:
      kubernetesCRD: ""
    log:
      filePath: ""
      level: error
      format: json
    accessLog:
      filePath: ""
      format: json
      bufferingSize: 0
      filters:
        retryAttempts: true
        minDuration: 20
      fields:
        defaultMode: keep
        names:
          ClientUsername: drop
        headers:
          defaultMode: keep
          names:
            User-Agent: redact
            Authorization: drop
            Content-Type: keep

创建ConfigMap:

# kubectl apply -f traefik-config.yaml

4、给节点打标签

由于是 Kubernetes DeamonSet 这种方式部署 Traefik，所以需要提前给节点设置 Label，这样当程序部署时 Pod 会自动调度到设置 Label 的点上。

节点设置 Label 标签

  • 格式：kubectl label nodes [节点名] [key=value] 
    kubectl label nodes 172.16.0.33 IngressProxy=true

查看节点标签：

# kubectl get nodes --show-labels 
NAME          STATUS                     ROLES    AGE   VERSION   LABELS
172.16.0.33   Ready,SchedulingDisabled   master   20d   v1.15.0   IngressProxy=true,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/fluentd-ds-ready=true,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=172.16.0.33,kubernetes.io/os=linux,kubernetes.io/role=master
172.16.0.52   Ready                      node     20d   v1.15.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/fluentd-ds-ready=true,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=172.16.0.52,kubernetes.io/os=linux,kubernetes.io/role=node

5、部署traefik

很多时候我们会采用DS方式部署，并且设置网络为hostNetwork=True，这样方便流量进入。但是在这里我采用的是service的nodeport进行暴露。

（1）、Service的配置清单
traefik-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: traefik
  namespace: ingress-traefik
spec:
  type: NodePort
  ports:
    - name: web
      port: 80
    - name: websecure
      port: 443
    - name: admin
      port: 8080
  selector:
    app: traefik

（2）、DS的配置清单
traefik-ds.yaml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: traefik-ingress-controller
  namespace: ingress-traefik
  labels:
    app: traefik
spec:
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      name: traefik
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 1
      containers:
        - image: registry.cn-hangzhou.aliyuncs.com/rookieops/traefik:v2.2.0
          name: traefik-ingress-lb
          ports:
            - name: web
              containerPort: 80
            - name: websecure
              containerPort: 443
            - name: admin
              containerPort: 8080
          resources:
            limits:
              cpu: 2000m
              memory: 1024Mi
            requests:
              cpu: 1000m
              memory: 1024Mi
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
          args:
            - --configfile=/config/traefik.yaml
          volumeMounts:
            - mountPath: "/config"
              name: "config"
      volumes:
        - name: config
          configMap:
            name: traefik-config 
      tolerations:              #设置容忍所有污点，防止节点被设置污点
        - operator: "Exists"
      nodeSelector:             #设置node筛选器，在特定label的节点上启动
        IngressProxy: "true"

3) 配置traefik路由规则

apiVersion: traefik.containo.us/v1alpha1
kind: Ingre***oute
metadata:
  name: traefik-dashboard-route
  namespace: ingress-traefik
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`traefik.coolops.cn`)
      kind: Rule
      services:
        - name: traefik
          port: 8080

然后创建资源：

# kubectl apply -f traefik-svc.yaml
# kubectl apply -f traefik-ds.yaml
# kubectl get svc -n ingress-traefik 
NAME      TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                                     AGE
traefik   NodePort   10.102.225.66   <none>        80:30161/TCP,443:30629/TCP,8080:32359/TCP   31m
# kubectl get pod -n ingress-traefik 
NAME                               READY   STATUS    RESTARTS   AGE
traefik-ingress-controller-whbjm   1/1     Running   0          22m

然后本地配置hosts

10.1.10.128 traefik.coolops.cn

由于我们ingress是通过svc的nodeport暴露的，所以输入以下访问
Traefik2.2从坑出发

并且可以看到我配置的HTTP
Traefik2.2从坑出发

6、配置SSL

这里使用Let's Encrypt 来进行自动化 HTTPS。
修改配置文件，新增如下内容：

    certificatesresolvers:
      default:
        acme:
          tlsChallenge: {}
          email: "coolops@163.com"
          storage: "acme.json"

更新配置文件。

新增一个https的ingre***oute，如下：

apiVersion: traefik.containo.us/v1alpha1
kind: Ingre***oute
metadata:
  name: traefik-webui-tls
  namespace: ingress-traefik 
spec:
  entryPoints:
  - websecure  # 注意这里是websecure这个entryPoint，监控443端口
  routes:
  - match: Host(`traefik.coolops.cn`)
    kind: Rule
    services:
    - name: traefik
      port: 8080
  tls:
    certResolver: default

查看pod的日志信息，报错如下：

{"level":"error","msg":"Unable to obtain ACME certificate for domains \"traefik.coolops.cn\": cannot get ACME client get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp 172.65.32.248:443: connect: connection refused","providerName":"default.acme","routerName":"ingress-traefik-traefik-webui-tls-fcde00a088d29eefb3a6@kubernetescrd","rule":"Host(`traefik.coolops.cn`)","time":"2020-05-26T02:49:49Z"}

这是因为pod里的网络和https://acme-v02.api.letsencrypt.org/directory 不通造成的。

部署成功后可以使用HTTPS访问了。
Traefik2.2从坑出发
Traefik2.2从坑出发

上面是自动生成证书，如果有自己的域名证书，那么一切都简单了，你只需要配置一个secret，然后在ingre***oute中引用即可，比如下面来自官方的例子：

apiVersion: v1
kind: Secret
metadata:
  name: supersecret
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0=
  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCi0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0=

---

apiVersion: traefik.containo.us/v1alpha1
kind: Ingre***oute
metadata:
  name: ingre***outetls
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`foo.com`) && PathPrefix(`/bar`)
    kind: Rule
    services:
    - name: whoami
      port: 443
  tls:
    secretName: supersecret

用Let's Encrypt的话，虽然免费，用起来还是有坑，这就需要自己去踩了~~！

完

上一篇：容器技术-01-Docker的本质


下一篇：oracle 12c 数据库上编写SQL语句时卡顿的问题解决