通过ansible-playbook，以Kubeadm方式部署K8S集群（一主多从）。
kubernetes安装目录： /etc/kubernetes/ KubeConfig： ~/.kube/config Version： v1.18.3
主机说明：
| 系统 | ip | 角色 | cpu | 内存 | hostname |
|---|---|---|---|---|---|
| CentOS 7.8 | 192.168.30.128 | master | >=2 | >=2G | master |
| CentOS 7.8 | 192.168.30.129 | node | >=2 | >=2G | node1 |
| CentOS 7.8 | 192.168.30.130 | node | >=2 | >=2G | node2 |
| CentOS 7.8 | 192.168.30.131 | node | >=2 | >=2G | node3 |
准备
- 将所有部署k8s集群的主机分组：
# vim /etc/ansible/hosts[master]192.168.30.128 hostname=master[node]192.168.30.129 hostname=node1 192.168.30.130 hostname=node2 192.168.30.131 hostname=node3
- 创建管理目录：
mkdir -p k8s/roles/{docker_install,master_install,node_install,addons_install}/{files,handlers,meta,tasks,templates,vars}cd k8s/
说明：
files：存放需要同步到异地服务器的源码文件及配置文件； handlers：当资源发生变化时需要进行的操作，若没有此目录可以不建或为空； meta：存放说明信息、说明角色依赖等信息，可留空； tasks：K8S 安装过程中需要进行执行的任务； templates：用于执行 K8S 安装的模板文件，一般为脚本； vars：本次安装定义的变量
tree ..├── k8s.yml └── roles ├── addons_install │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ ├── calico.yml │ │ ├── ingress.yml │ │ └── main.yml │ ├── templates │ │ ├── calico-rbac.yaml │ │ ├── calico.yaml │ │ └── ingress-nginx.yaml │ └── vars │ └── main.yml ├── docker_install │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ ├── install.yml │ │ ├── main.yml │ │ └── prepare.yml │ ├── templates │ │ ├── daemon.json │ │ ├── install.sh │ │ ├── kubernetes.conf │ │ └── kubernetes.repo │ └── vars │ └── main.yml ├── master_install │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ ├── install.yml │ │ └── main.yml │ ├── templates │ │ └── kubeadm-config.yaml │ └── vars │ └── main.yml └── node_install ├── files ├── handlers ├── meta ├── tasks │ ├── install.yml │ └── main.yml ├── templates └── vars └── main.yml 29 directories, 23 files
- 创建安装入口文件，用来调用roles：
vim k8s.yml
---- hosts: all remote_user: root gather_facts: True roles: - docker_install- hosts: master remote_user: root gather_facts: True roles: - master_install- hosts: node remote_user: root gather_facts: True roles: - node_install - hosts: master remote_user: root gather_facts: True roles: - addons_install
docker部分
- 创建docker入口文件，用来调用docker_install：
vim docker.yml
- hosts: all remote_user: root gather_facts: True roles: - docker_install
- 创建变量：
vim roles/docker_install/vars/main.yml
SOURCE_DIR: /softwareVERSION: 1.18.3
- 创建模板文件：
docker配置daemon.json
vim roles/docker_install/templates/daemon.json
{
"registry-mirrors": ["http://f1361db2.m.daocloud.io"],
"exec-opts":["native.cgroupdriver=systemd"]}
系统环境kubernetes.conf
vim roles/docker_install/templates/kubernetes.conf
net.bridge.bridge-nf-call-iptables=1 net.bridge.bridge-nf-call-ip6tables=1 net.ipv4.ip_forward=1 vm.swappiness=0 vm.overcommit_memory=1 vm.panic_on_oom=0 fs.inotify.max_user_watches=89100
repo文件kubernetes.repo
vim roles/docker_install/templates/kubernetes.repo
[kubernetes] name=Kubernetes baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
docker-py安装脚本install.sh
vim roles/docker_install/templates/install.sh
#!/bin/bashloop_exec() {
CMD=$1
while :; do
${CMD}
if [ $? -eq 0 ] ; then
break;
fi
done}main() {
loop_exec "yum install -y python python-pip"
loop_exec "pip install --upgrade pip"
loop_exec "pip install docker-py"}main
- 环境准备prepare.yml：
vim roles/docker_install/tasks/prepare.yml
- name: 关闭firewalld service: name=firewalld state=stopped enabled=no - name: 临时关闭 selinux shell: "setenforce 0" failed_when: false- name: 永久关闭 selinux lineinfile: dest: /etc/selinux/config regexp: "^SELINUX=" line: "SELINUX=disabled"- name: 添加EPEL仓库 yum: name=epel-release state=latest- name: 安装常用软件包 yum: name: - vim - lrzsz - net-tools - wget - curl - bash-completion - rsync - gcc - unzip - git - iptables - conntrack - ipvsadm - ipset - jq - sysstat - libseccomp state: latest- name: 更新系统 shell: "yum update -y --exclude kubeadm,kubelet,kubectl" ignore_errors: yes args: warn: False - name: 配置iptables shell: "iptables -F && iptables -X && iptables -F -t nat && iptables -X -t nat && iptables -P FORWARD ACCEPT"- name: 关闭swap shell: "swapoff -a && sed -i '/swap/s/^\\(.*\\)$/#\\1/g' /etc/fstab" - name: 系统配置 template: src=kubernetes.conf dest=/etc/sysctl.d/kubernetes.conf- name: 加载br_netfilter shell: "modprobe br_netfilter"- name: 生效配置 shell: "sysctl -p /etc/sysctl.d/kubernetes.conf"
- docker安装install.yml：
vim roles/docker_install/tasks/install.yml
- name: 创建software目录 file: name={{ SOURCE_DIR }} state=directory- name: 更改hostname raw: "echo {{ hostname }} > /etc/hostname"- name: 更改生效 shell: "hostname {{ hostname }}"- name: 设置本地dns shell: "if [ `grep '{{ ansible_ssh_host }} {{ hostname }}' /etc/hosts |wc -l` -eq 0 ]; then echo {{ ansible_ssh_host }} {{ hostname }} >> /etc/hosts; fi"- name: 下载repo文件 shell: "if [ ! -f /etc/yum.repos.d/docker.repo ]; then curl http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -o /etc/yum.repos.d/docker.repo; fi"- name: 生成缓存 shell: "yum makecache fast"
args:
warn: False- name: 安装docker-ce yum:
name: docker-ce state: latest- name: 启动docker并开机启动 service:
name: docker state: started enabled: yes
- name: 配置docker template: src=daemon.json dest=/etc/docker/daemon.json- name: 重启docker service:
name: docker state: restarted
- name: 配置kubernetes源 template: src=kubernetes.repo dest=/etc/yum.repos.d/kubernetes.repo- name: 安装kubernetes-cni yum:
name: kubernetes-cni state: latest
- name: 安装kubeadm、kubelet、kubectl shell: "yum install -y kubeadm-{{ VERSION }} kubelet-{{ VERSION }} kubectl-{{ VERSION }} --disableexcludes=kubernetes"
args:
warn: False- name: 启动kubelet并开机启动 service:
name: kubelet state: started enabled: yes
- name: 拷贝脚本 template: src=install.sh dest={{ SOURCE_DIR }} mode=0755
- name: 安装docker-py script: "{{ SOURCE_DIR }}/install.sh"
- 引用文件main.yml：
vim roles/docker_install/tasks/main.yml
- include: prepare.yml- include: install.yml
master部分
- 创建master入口文件，用来调用master_install：
vim master.yml
- hosts: master remote_user: root gather_facts: True roles: - master_install
- 创建变量：
vim roles/master_install/vars/main.yml
SOURCE_DIR: /softwareVERSION: v1.18.3POD_CIDR: 172.10.0.0/16MASTER_IP: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}"
- 创建模板文件：
kubeadm配置文件 kubeadm-config.yaml
vim roles/master_install/templates/kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta2kind: ClusterConfigurationkubernetesVersion: "{{ VERSION }}"controlPlaneEndpoint: "{{ MASTER_IP }}:6443"networking:
podSubnet: "{{ POD_CIDR }}"imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
- 集群初始化install.yml：
vim roles/master_install/tasks/install.yml
- name: 拷贝kubeadm配置文件 template: src=kubeadm-config.yaml dest={{ SOURCE_DIR }}- name: 集群初始化准备1 shell: "swapoff -a && kubeadm reset -f"- name: 集群初始化准备2 shell: "systemctl daemon-reload && systemctl restart kubelet"
- name: 集群初始化准备3 shell: "iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X"- name: 拉取镜像 shell: "kubeadm config images pull --kubernetes-version={{ VERSION }} --image-repository=registry.aliyuncs.com/google_containers"- name: 集群初始化 shell: "kubeadm init --config={{ SOURCE_DIR }}/kubeadm-config.yaml --upload-certs &>{{ SOURCE_DIR }}/token"- name: 获取master的token shell: "grep -B2 'control-plane --certificate-key' {{ SOURCE_DIR }}/token > {{ SOURCE_DIR }}/master.sh"- name: 获取node的token shell: "grep -A1 'kubeadm join' {{ SOURCE_DIR }}/token |tail -2 > {{ SOURCE_DIR }}/node.sh"- name: 分发master.sh shell: "ansible master -m copy -a 'src={{ SOURCE_DIR }}/master.sh dest={{ SOURCE_DIR }} mode=0755'"
args:
warn: False
- name: 分发node.sh shell: "ansible node -m copy -a 'src={{ SOURCE_DIR }}/node.sh dest={{ SOURCE_DIR }} mode=0755'"
args:
warn: False- name: 创建 $HOME/.kube 目录 file: name=$HOME/.kube state=directory
- name: 拷贝KubeConfig copy: src=/etc/kubernetes/admin.conf dest=$HOME/.kube/config owner=root group=root- name: kubectl命令补全1 shell: "kubectl completion bash > $HOME/.kube/completion.bash.inc"
- name: kubectl命令补全2 shell: "if [ `grep 'source $HOME/.kube/completion.bash.inc' $HOME/.bash_profile |wc -l` -eq 0 ]; then echo 'source $HOME/.kube/completion.bash.inc' >> $HOME/.bash_profile; fi"
- name: 生效配置 shell: "source $HOME/.bash_profile"
ignore_errors: yes
- 引用文件main.yml：
vim roles/master_install/tasks/main.yml
- include: install.yml
node部分
- 创建node入口文件，用来调用node_install：
vim node.yml
- hosts: node remote_user: root gather_facts: True roles: - node_install
- 创建变量：
vim roles/node_install/vars/main.yml
SOURCE_DIR: /software
- 添加node到集群install.yml：
vim roles/node_install/tasks/install.yml
- name: 集群初始化准备1 shell: "swapoff -a && kubeadm reset -f"- name: 集群初始化准备2 shell: "systemctl daemon-reload && systemctl restart kubelet"
- name: 集群初始化准备3 shell: "iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X"
- name: 集群增加node script: "{{ SOURCE_DIR }}/node.sh"
- name: 删除node的token file: name={{ SOURCE_DIR }}/node.sh state=absent
- 引用文件main.yml：
vim roles/node_install/tasks/main.yml
- include: install.yml
addons部分
- 创建addons入口文件，用来调用addons_install：
vim addons.yml
- hosts: master remote_user: root gather_facts: True roles: - addons_install
- 创建变量：
vim roles/addons_install/vars/main.yml
SOURCE_DIR: /softwarePOD_CIDR: 172.10.0.0/16CALICO_VER: v3.15.1BACKEND_VER: 1.5INGRESS_VER: 0.19.0
- 创建模板文件：
calico rbac配置文件 calico-rbac.yaml
vim roles/addons_install/templates/calico-rbac.yaml
apiVersion: v1kind: ServiceAccountmetadata: name: calico-kube-controllers namespace: kube-system ---kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1metadata: name: calico-kube-controllersrules: - apiGroups: [""] resources: - nodes verbs: - watch - list - get - apiGroups: [""] resources: - pods verbs: - get - apiGroups: ["crd.projectcalico.org"] resources: - ippools verbs: - list - apiGroups: ["crd.projectcalico.org"] resources: - blockaffinities - ipamblocks - ipamhandles verbs: - get - list - create - update - delete - apiGroups: ["crd.projectcalico.org"] resources: - hostendpoints verbs: - get - list - create - update - delete - apiGroups: ["crd.projectcalico.org"] resources: - clusterinformations verbs: - get - create - update - apiGroups: ["crd.projectcalico.org"] resources: - kubecontrollersconfigurations verbs: - get - create - update - watch ---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1metadata: name: calico-kube-controllersroleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: calico-kube-controllerssubjects:- kind: ServiceAccount name: calico-kube-controllers namespace: kube-system---apiVersion: v1kind: ServiceAccountmetadata: name: calico-node namespace: kube-system---kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1metadata: name: calico-noderules: - apiGroups: [""] resources: - pods - nodes - namespaces verbs: - get - apiGroups: [""] resources: - endpoints - services verbs: - watch - list - get - apiGroups: [""] resources: - configmaps verbs: - get - apiGroups: [""] resources: - nodes/status verbs: - patch - update - apiGroups: ["networking.k8s.io"] resources: - networkpolicies verbs: - watch - list - apiGroups: [""] resources: - pods - namespaces - serviceaccounts verbs: - list - watch - apiGroups: [""] resources: - pods/status verbs: - patch - apiGroups: ["crd.projectcalico.org"] resources: - globalfelixconfigs - felixconfigurations - bgppeers - globalbgpconfigs - bgpconfigurations - ippools - ipamblocks - globalnetworkpolicies - globalnetworksets - networkpolicies - networksets - clusterinformations - hostendpoints - blockaffinities verbs: - get - list - watch - apiGroups: ["crd.projectcalico.org"] resources: - ippools - felixconfigurations - clusterinformations verbs: - create - update - apiGroups: [""] resources: - nodes verbs: - get - list - watch - apiGroups: ["crd.projectcalico.org"] resources: - bgpconfigurations - bgppeers verbs: - create - update - apiGroups: ["crd.projectcalico.org"] resources: - blockaffinities - ipamblocks - ipamhandles verbs: - get - list - create - update - delete - apiGroups: ["crd.projectcalico.org"] resources: - ipamconfigs verbs: - get - apiGroups: ["crd.projectcalico.org"] resources: - blockaffinities verbs: - watch - apiGroups: ["apps"] resources: - daemonsets verbs: - get---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata: name: calico-noderoleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: calico-nodesubjects:- kind: ServiceAccount name: calico-node namespace: kube-system
calico配置文件 calico.yaml
vim roles/addons_install/templates/calico.yaml
apiVersion: v1kind: ConfigMapmetadata:
name: calico-config namespace: kube-systemdata:
typha_service_name: "none"
calico_backend: "bird"
veth_mtu: "1440"
cni_network_config: |-
{
"name": "k8s-pod-network",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "calico",
"log_level": "info",
"datastore_type": "kubernetes",
"nodename": "__KUBERNETES_NODE_NAME__",
"mtu": __CNI_MTU__,
"ipam": {
"type": "calico-ipam"
},
"policy": {
"type": "k8s"
},
"kubernetes": {
"kubeconfig": "__KUBECONFIG_FILEPATH__"
}
},
{
"type": "portmap",
"snat": true,
"capabilities": {"portMappings": true}
},
{
"type": "bandwidth",
"capabilities": {"bandwidth": true}
}
]
}
---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: bgpconfigurations.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: BGPConfiguration listKind: BGPConfigurationList plural: bgpconfigurations singular: bgpconfiguration scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
asNumber:
format: int32 type: integer logSeverityScreen:
type: string nodeToNodeMeshEnabled:
type: boolean serviceClusterIPs:
items:
properties:
cidr:
type: string type: object type: array serviceExternalIPs:
items:
properties:
cidr:
type: string type: object type: array type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: bgppeers.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: BGPPeer listKind: BGPPeerList plural: bgppeers singular: bgppeer scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
asNumber:
format: int32 type: integer node:
type: string nodeSelector:
type: string peerIP:
type: string peerSelector:
type: string required:
- asNumber - peerIP type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: blockaffinities.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: BlockAffinity listKind: BlockAffinityList plural: blockaffinities singular: blockaffinity scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
cidr:
type: string deleted:
type: string node:
type: string state:
type: string required:
- cidr - deleted - node - state type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: clusterinformations.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: ClusterInformation listKind: ClusterInformationList plural: clusterinformations singular: clusterinformation scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
calicoVersion:
type: string clusterGUID:
type: string clusterType:
type: string datastoreReady:
type: boolean variant:
type: string type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: felixconfigurations.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: FelixConfiguration listKind: FelixConfigurationList plural: felixconfigurations singular: felixconfiguration scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
bpfConnectTimeLoadBalancingEnabled:
type: boolean bpfDataIfacePattern:
type: string bpfDisableUnprivileged:
type: boolean bpfEnabled:
type: boolean bpfExternalServiceMode:
type: string bpfKubeProxyEndpointSlicesEnabled:
type: boolean bpfKubeProxyIptablesCleanupEnabled:
type: boolean bpfKubeProxyMinSyncPeriod:
type: string bpfLogLevel:
type: string chainInsertMode:
type: string dataplaneDriver:
type: string debugDisableLogDropping:
type: boolean debugMemoryProfilePath:
type: string debugSimulateCalcGraphHangAfter:
type: string debugSimulateDataplaneHangAfter:
type: string defaultEndpointToHostAction:
type: string deviceRouteProtocol:
type: integer deviceRouteSourceAddress:
type: string disableConntrackInvalidCheck:
type: boolean endpointReportingDelay:
type: string endpointReportingEnabled:
type: boolean externalNodesList:
items:
type: string type: array failsafeInboundHostPorts:
items:
properties:
port:
type: integer protocol:
type: string required:
- port - protocol type: object type: array failsafeOutboundHostPorts:
items:
properties:
port:
type: integer protocol:
type: string required:
- port - protocol type: object type: array genericXDPEnabled:
type: boolean healthEnabled:
type: boolean healthHost:
type: string healthPort:
type: integer interfaceExclude:
type: string interfacePrefix:
type: string ipipEnabled:
type: boolean ipipMTU:
type: integer ipsetsRefreshInterval:
type: string iptablesBackend:
type: string iptablesFilterAllowAction:
type: string iptablesLockFilePath:
type: string iptablesLockProbeInterval:
type: string iptablesLockTimeout:
type: string iptablesMangleAllowAction:
type: string iptablesMarkMask:
format: int32 type: integer iptablesNATOutgoingInterfaceFilter:
type: string iptablesPostWriteCheckInterval:
type: string iptablesRefreshInterval:
type: string ipv6Support:
type: boolean kubeNodePortRanges:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array logFilePath:
type: string logPrefix:
type: string logSeverityFile:
type: string logSeverityScreen:
type: string logSeveritySys:
type: string maxIpsetSize:
type: integer metadataAddr:
type: string metadataPort:
type: integer natOutgoingAddress:
type: string natPortRange:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
netlinkTimeout:
type: string openstackRegion:
type: string policySyncPathPrefix:
type: string prometheusGoMetricsEnabled:
type: boolean prometheusMetricsEnabled:
type: boolean prometheusMetricsHost:
type: string prometheusMetricsPort:
type: integer prometheusProcessMetricsEnabled:
type: boolean removeExternalRoutes:
type: boolean reportingInterval:
type: string reportingTTL:
type: string routeRefreshInterval:
type: string routeSource:
type: string routeTableRange:
properties:
max:
type: integer min:
type: integer required:
- max - min type: object sidecarAccelerationEnabled:
type: boolean usageReportingEnabled:
type: boolean usageReportingInitialDelay:
type: string usageReportingInterval:
type: string useInternalDataplaneDriver:
type: boolean vxlanEnabled:
type: boolean vxlanMTU:
type: integer vxlanPort:
type: integer vxlanVNI:
type: integer wireguardEnabled:
type: boolean wireguardInterfaceName:
type: string wireguardListeningPort:
type: integer wireguardMTU:
type: integer wireguardRoutingRulePriority:
type: integer xdpEnabled:
type: boolean xdpRefreshInterval:
type: string required:
- bpfLogLevel type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: globalnetworkpolicies.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: GlobalNetworkPolicy listKind: GlobalNetworkPolicyList plural: globalnetworkpolicies singular: globalnetworkpolicy scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
applyOnForward:
type: boolean doNotTrack:
type: boolean egress:
items:
properties:
action:
type: string destination:
properties:
namespaceSelector:
type: string nets:
items:
type: string type: array notNets:
items:
type: string type: array notPorts:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array notSelector:
type: string ports:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array selector:
type: string serviceAccounts:
properties:
names:
items:
type: string type: array selector:
type: string type: object type: object http:
properties:
methods:
items:
type: string type: array paths:
items:
properties:
exact:
type: string prefix:
type: string type: object type: array type: object icmp:
properties:
code:
type: integer type:
type: integer type: object ipVersion:
type: integer metadata:
properties:
annotations:
additionalProperties:
type: string type: object type: object notICMP:
properties:
code:
type: integer type:
type: integer type: object notProtocol:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
source:
properties:
namespaceSelector:
type: string nets:
items:
type: string type: array notNets:
items:
type: string type: array notPorts:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array notSelector:
type: string ports:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array selector:
type: string serviceAccounts:
properties:
names:
items:
type: string type: array selector:
type: string type: object type: object required:
- action type: object type: array ingress:
items:
properties:
action:
type: string destination:
properties:
namespaceSelector:
type: string nets:
items:
type: string type: array notNets:
items:
type: string type: array notPorts:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array notSelector:
type: string ports:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array selector:
type: string serviceAccounts:
properties:
names:
items:
type: string type: array selector:
type: string type: object type: object http:
properties:
methods:
items:
type: string type: array paths:
items:
properties:
exact:
type: string prefix:
type: string type: object type: array type: object icmp:
properties:
code:
type: integer type:
type: integer type: object ipVersion:
type: integer metadata:
properties:
annotations:
additionalProperties:
type: string type: object type: object notICMP:
properties:
code:
type: integer type:
type: integer type: object notProtocol:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
source:
properties:
namespaceSelector:
type: string nets:
items:
type: string type: array notNets:
items:
type: string type: array notPorts:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array notSelector:
type: string ports:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array selector:
type: string serviceAccounts:
properties:
names:
items:
type: string type: array selector:
type: string type: object type: object required:
- action type: object type: array namespaceSelector:
type: string order:
type: number preDNAT:
type: boolean selector:
type: string serviceAccountSelector:
type: string types:
items:
type: string type: array type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: globalnetworksets.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: GlobalNetworkSet listKind: GlobalNetworkSetList plural: globalnetworksets singular: globalnetworkset scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
nets:
items:
type: string type: array type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: hostendpoints.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: HostEndpoint listKind: HostEndpointList plural: hostendpoints singular: hostendpoint scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
expectedIPs:
items:
type: string type: array interfaceName:
type: string node:
type: string ports:
items:
properties:
name:
type: string port:
type: integer protocol:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
required:
- name - port - protocol type: object type: array profiles:
items:
type: string type: array type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: ipamblocks.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: IPAMBlock listKind: IPAMBlockList plural: ipamblocks singular: ipamblock scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
affinity:
type: string allocations:
items:
type: integer nullable: true
type: array attributes:
items:
properties:
handle_id:
type: string secondary:
additionalProperties:
type: string type: object type: object type: array cidr:
type: string deleted:
type: boolean strictAffinity:
type: boolean unallocated:
items:
type: integer type: array required:
- allocations - attributes - cidr - deleted - strictAffinity - unallocated type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: ipamconfigs.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: IPAMConfig listKind: IPAMConfigList plural: ipamconfigs singular: ipamconfig scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
autoAllocateBlocks:
type: boolean strictAffinity:
type: boolean required:
- autoAllocateBlocks - strictAffinity type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: ipamhandles.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: IPAMHandle listKind: IPAMHandleList plural: ipamhandles singular: ipamhandle scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
block:
additionalProperties:
type: integer type: object handleID:
type: string required:
- block - handleID type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: ippools.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: IPPool listKind: IPPoolList plural: ippools singular: ippool scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
blockSize:
type: integer cidr:
type: string disabled:
type: boolean ipip:
properties:
enabled:
type: boolean mode:
type: string type: object ipipMode:
type: string nat-outgoing:
type: boolean natOutgoing:
type: boolean nodeSelector:
type: string vxlanMode:
type: string required:
- cidr type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: kubecontrollersconfigurations.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: KubeControllersConfiguration listKind: KubeControllersConfigurationList plural: kubecontrollersconfigurations singular: kubecontrollersconfiguration scope: Cluster versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
controllers:
properties:
namespace:
properties:
reconcilerPeriod:
type: string type: object node:
properties:
hostEndpoint:
properties:
autoCreate:
type: string type: object reconcilerPeriod:
type: string syncLabels:
type: string type: object policy:
properties:
reconcilerPeriod:
type: string type: object serviceAccount:
properties:
reconcilerPeriod:
type: string type: object workloadEndpoint:
properties:
reconcilerPeriod:
type: string type: object type: object etcdV3CompactionPeriod:
type: string healthChecks:
type: string logSeverityScreen:
type: string required:
- controllers type: object status:
properties:
environmentVars:
additionalProperties:
type: string type: object runningConfig:
properties:
controllers:
properties:
namespace:
properties:
reconcilerPeriod:
type: string type: object node:
properties:
hostEndpoint:
properties:
autoCreate:
type: string type: object reconcilerPeriod:
type: string syncLabels:
type: string type: object policy:
properties:
reconcilerPeriod:
type: string type: object serviceAccount:
properties:
reconcilerPeriod:
type: string type: object workloadEndpoint:
properties:
reconcilerPeriod:
type: string type: object type: object etcdV3CompactionPeriod:
type: string healthChecks:
type: string logSeverityScreen:
type: string required:
- controllers type: object type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: networkpolicies.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: NetworkPolicy listKind: NetworkPolicyList plural: networkpolicies singular: networkpolicy scope: Namespaced versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
egress:
items:
properties:
action:
type: string destination:
properties:
namespaceSelector:
type: string nets:
items:
type: string type: array notNets:
items:
type: string type: array notPorts:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array notSelector:
type: string ports:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array selector:
type: string serviceAccounts:
properties:
names:
items:
type: string type: array selector:
type: string type: object type: object http:
properties:
methods:
items:
type: string type: array paths:
items:
properties:
exact:
type: string prefix:
type: string type: object type: array type: object icmp:
properties:
code:
type: integer type:
type: integer type: object ipVersion:
type: integer metadata:
properties:
annotations:
additionalProperties:
type: string type: object type: object notICMP:
properties:
code:
type: integer type:
type: integer type: object notProtocol:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
source:
properties:
namespaceSelector:
type: string nets:
items:
type: string type: array notNets:
items:
type: string type: array notPorts:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array notSelector:
type: string ports:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array selector:
type: string serviceAccounts:
properties:
names:
items:
type: string type: array selector:
type: string type: object type: object required:
- action type: object type: array ingress:
items:
properties:
action:
type: string destination:
properties:
namespaceSelector:
type: string nets:
items:
type: string type: array notNets:
items:
type: string type: array notPorts:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array notSelector:
type: string ports:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array selector:
type: string serviceAccounts:
properties:
names:
items:
type: string type: array selector:
type: string type: object type: object http:
properties:
methods:
items:
type: string type: array paths:
items:
properties:
exact:
type: string prefix:
type: string type: object type: array type: object icmp:
properties:
code:
type: integer type:
type: integer type: object ipVersion:
type: integer metadata:
properties:
annotations:
additionalProperties:
type: string type: object type: object notICMP:
properties:
code:
type: integer type:
type: integer type: object notProtocol:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
source:
properties:
namespaceSelector:
type: string nets:
items:
type: string type: array notNets:
items:
type: string type: array notPorts:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array notSelector:
type: string ports:
items:
anyOf:
- type: integer - type: string pattern: ^.* x-kubernetes-int-or-string: true
type: array selector:
type: string serviceAccounts:
properties:
names:
items:
type: string type: array selector:
type: string type: object type: object required:
- action type: object type: array order:
type: number selector:
type: string serviceAccountSelector:
type: string types:
items:
type: string type: array type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []---apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:
name: networksets.crd.projectcalico.orgspec:
group: crd.projectcalico.org names:
kind: NetworkSet listKind: NetworkSetList plural: networksets singular: networkset scope: Namespaced versions:
- name: v1 schema:
openAPIV3Schema:
properties:
apiVersion:
type: string kind:
type: string metadata:
type: object spec:
properties:
nets:
items:
type: string type: array type: object type: object served: true
storage: truestatus:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---apiVersion: apps/v1kind: DaemonSetmetadata:
name: calico-node namespace: kube-system labels:
k8s-app: calico-nodespec:
selector:
matchLabels:
k8s-app: calico-node updateStrategy:
type: RollingUpdate rollingUpdate:
maxUnavailable: 1
template:
metadata:
labels:
k8s-app: calico-node spec:
nodeSelector:
kubernetes.io/os: linux hostNetwork: true
tolerations:
- effect: NoSchedule operator: Exists - key: CriticalAddonsOnly operator: Exists - effect: NoExecute operator: Exists serviceAccountName: calico-node terminationGracePeriodSeconds: 0
priorityClassName: system-node-critical initContainers:
- name: upgrade-ipam image: calico/cni:{{ CALICO_VER }}
command: ["/opt/cni/bin/calico-ipam", "-upgrade"]
env:
- name: KUBERNETES_NODE_NAME valueFrom:
fieldRef:
fieldPath: spec.nodeName - name: CALICO_NETWORKING_BACKEND valueFrom:
configMapKeyRef:
name: calico-config key: calico_backend volumeMounts:
- mountPath: /var/lib/cni/networks name: host-local-net-dir - mountPath: /host/opt/cni/bin name: cni-bin-dir securityContext:
privileged: true
- name: install-cni image: calico/cni:{{ CALICO_VER }}
command: ["/install-cni.sh"]
env:
- name: CNI_CONF_NAME value: "10-calico.conflist"
- name: CNI_NETWORK_CONFIG valueFrom:
configMapKeyRef:
name: calico-config key: cni_network_config - name: KUBERNETES_NODE_NAME valueFrom:
fieldRef:
fieldPath: spec.nodeName - name: CNI_MTU valueFrom:
configMapKeyRef:
name: calico-config key: veth_mtu - name: SLEEP value: "false"
volumeMounts:
- mountPath: /host/opt/cni/bin name: cni-bin-dir - mountPath: /host/etc/cni/net.d name: cni-net-dir securityContext:
privileged: true
- name: flexvol-driver image: calico/pod2daemon-flexvol:{{ CALICO_VER }}
volumeMounts:
- name: flexvol-driver-host mountPath: /host/driver securityContext:
privileged: true
containers:
- name: calico-node image: calico/node:{{ CALICO_VER }}
env:
- name: DATASTORE_TYPE value: "kubernetes"
- name: WAIT_FOR_DATASTORE value: "true"
- name: NODENAME valueFrom:
fieldRef:
fieldPath: spec.nodeName - name: CALICO_NETWORKING_BACKEND valueFrom:
configMapKeyRef:
name: calico-config key: calico_backend - name: CLUSTER_TYPE value: "k8s,bgp"
- name: IP value: "autodetect"
- name: IP_AUTODETECTION_METHOD value: "interface=eth.*" #匹配本地有效网卡
- name: CALICO_IPV4POOL_IPIP value: "Always"
- name: CALICO_IPV4POOL_VXLAN value: "Never"
- name: FELIX_IPINIPMTU valueFrom:
configMapKeyRef:
name: calico-config key: veth_mtu - name: FELIX_VXLANMTU valueFrom:
configMapKeyRef:
name: calico-config key: veth_mtu - name: FELIX_WIREGUARDMTU valueFrom:
configMapKeyRef:
name: calico-config key: veth_mtu - name: CALICO_IPV4POOL_CIDR value: "{{ POD_CIDR }}" #与前面定义的pod的CIDR保持一致
- name: CALICO_DISABLE_FILE_LOGGING value: "true"
- name: FELIX_DEFAULTENDPOINTTOHOSTACTION value: "ACCEPT"
- name: FELIX_IPV6SUPPORT value: "false"
- name: FELIX_LOGSEVERITYSCREEN value: "info"
- name: FELIX_HEALTHENABLED value: "true"
securityContext:
privileged: true
resources:
requests:
cpu: 250m livenessProbe:
exec:
command:
- /bin/calico-node - -felix-live - -bird-live periodSeconds: 10
initialDelaySeconds: 10
failureThreshold: 6
readinessProbe:
exec:
command:
- /bin/calico-node - -felix-ready - -bird-ready periodSeconds: 10
volumeMounts:
- mountPath: /lib/modules name: lib-modules readOnly: true
- mountPath: /run/xtables.lock name: xtables-lock readOnly: false
- mountPath: /var/run/calico name: var-run-calico readOnly: false
- mountPath: /var/lib/calico name: var-lib-calico readOnly: false
- name: policysync mountPath: /var/run/nodeagent volumes:
- name: lib-modules hostPath:
path: /lib/modules - name: var-run-calico hostPath:
path: /var/run/calico - name: var-lib-calico hostPath:
path: /var/lib/calico - name: xtables-lock hostPath:
path: /run/xtables.lock type: FileOrCreate - name: cni-bin-dir hostPath:
path: /opt/cni/bin - name: cni-net-dir hostPath:
path: /etc/cni/net.d - name: host-local-net-dir hostPath:
path: /var/lib/cni/networks - name: policysync hostPath:
type: DirectoryOrCreate path: /var/run/nodeagent - name: flexvol-driver-host hostPath:
type: DirectoryOrCreate path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds
---apiVersion: apps/v1kind: Deploymentmetadata:
name: calico-kube-controllers namespace: kube-system labels:
k8s-app: calico-kube-controllersspec:
replicas: 1
selector:
matchLabels:
k8s-app: calico-kube-controllers strategy:
type: Recreate template:
metadata:
name: calico-kube-controllers namespace: kube-system labels:
k8s-app: calico-kube-controllers spec:
nodeSelector:
kubernetes.io/os: linux tolerations:
- key: CriticalAddonsOnly operator: Exists - key: node-role.kubernetes.io/master effect: NoSchedule serviceAccountName: calico-kube-controllers priorityClassName: system-cluster-critical containers:
- name: calico-kube-controllers image: calico/kube-controllers:{{ CALICO_VER }}
env:
- name: ENABLED_CONTROLLERS value: node - name: DATASTORE_TYPE value: kubernetes readinessProbe:
exec:
command:
- /usr/bin/check-status - -r
ingress配置文件 ingress-nginx.yaml
vim roles/addons_install/templates/ingress-nginx.yaml
apiVersion: v1kind: Namespacemetadata:
name: ingress-nginx---apiVersion: v1kind: Servicemetadata:
name: default-http-backend namespace: ingress-nginx labels:
app.kubernetes.io/name: default-http-backend app.kubernetes.io/part-of: ingress-nginxspec:
ports:
- port: 80
targetPort: 8080
selector:
app.kubernetes.io/name: default-http-backend app.kubernetes.io/part-of: ingress-nginx
---apiVersion: apps/v1kind: Deploymentmetadata:
name: default-http-backend namespace: ingress-nginx labels:
app.kubernetes.io/name: default-http-backend app.kubernetes.io/part-of: ingress-nginxspec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: default-http-backend app.kubernetes.io/part-of: ingress-nginx template:
metadata:
labels:
app.kubernetes.io/name: default-http-backend app.kubernetes.io/part-of: ingress-nginx spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-http-backend image: k8s.gcr.io/defaultbackend-amd64:{{ BACKEND_VER }}
imagePullPolicy: IfNotPresent ports:
- containerPort: 8080
resources:
limits:
cpu: 10m memory: 20Mi requests:
cpu: 10m memory: 20Mi livenessProbe:
httpGet:
path: /healthz port: 8080
scheme: HTTP initialDelaySeconds: 30
timeoutSeconds: 5---apiVersion: v1kind: ConfigMapmetadata:
name: nginx-configuration namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx---apiVersion: v1kind: ConfigMapmetadata:
name: tcp-services namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx---apiVersion: v1kind: ConfigMapmetadata:
name: udp-services namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx---apiVersion: v1kind: ServiceAccountmetadata:
name: nginx-ingress-serviceaccount namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx---apiVersion: rbac.authorization.k8s.io/v1beta1kind: ClusterRolemetadata:
name: nginx-ingress-clusterrole labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginxrules:
- apiGroups:
- ""
resources:
- configmaps - endpoints - nodes - pods - secrets verbs:
- list - watch - apiGroups:
- ""
resources:
- nodes verbs:
- get - apiGroups:
- ""
resources:
- services verbs:
- get - list - watch - apiGroups:
- "extensions"
resources:
- ingresses verbs:
- get - list - watch - apiGroups:
- ""
resources:
- events verbs:
- create - patch - apiGroups:
- "extensions"
resources:
- ingresses/status verbs:
- update---apiVersion: rbac.authorization.k8s.io/v1beta1kind: Rolemetadata:
name: nginx-ingress-role namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginxrules:
- apiGroups:
- ""
resources:
- configmaps - pods - secrets - namespaces verbs:
- get - apiGroups:
- ""
resources:
- configmaps resourceNames:
- "ingress-controller-leader-nginx"
verbs:
- get - update - apiGroups:
- ""
resources:
- configmaps verbs:
- create - apiGroups:
- ""
resources:
- endpoints verbs:
- get---apiVersion: rbac.authorization.k8s.io/v1beta1kind: ClusterRoleBindingmetadata:
name: nginx-ingress-clusterrole-binding labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginxroleRef:
apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrolesubjects:
- kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx
---apiVersion: rbac.authorization.k8s.io/v1beta1kind: RoleBindingmetadata:
name: nginx-ingress-role-binding namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginxroleRef:
apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-rolesubjects:
- kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx---apiVersion: v1kind: Servicemetadata:
name: ingress-nginx namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginxspec:
selector:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx ports:
- name: http port: 80
targetPort: http - name: https port: 443
targetPort: https
---apiVersion: apps/v1kind: DaemonSetmetadata:
name: ingress-nginx namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginxspec:
revisionHistoryLimit: 2147483647
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx updateStrategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
containers:
- name: nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:{{ INGRESS_VER }}
imagePullPolicy: IfNotPresent args:
- /nginx-ingress-controller - --default-backend-service=$(POD_NAMESPACE)/default-http-backend - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix=nginx.ingress.kubernetes.io env:
- name: POD_NAME valueFrom:
fieldRef:
apiVersion: v1 fieldPath: metadata.name - name: POD_NAMESPACE valueFrom:
fieldRef:
apiVersion: v1 fieldPath: metadata.namespace ports:
- containerPort: 80
hostPort: 80
name: http protocol: TCP - containerPort: 443
hostPort: 443
name: https protocol: TCP livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz port: 10254
scheme: HTTP initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz port: 10254
scheme: HTTP periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
securityContext:
capabilities:
add:
- NET_BIND_SERVICE drop:
- ALL procMount: Default runAsUser: 33
terminationMessagePath: /dev/termination-log terminationMessagePolicy: File dnsPolicy: ClusterFirst hostNetwork: true
restartPolicy: Always schedulerName: default-scheduler securityContext: {}
serviceAccount: nginx-ingress-serviceaccount serviceAccountName: nginx-ingress-serviceaccount terminationGracePeriodSeconds: 30
- coredns安装calico.yml：
vim roles/addons_install/tasks/calico.yml
- name: 创建addons目录 file: name=/etc/kubernetes/addons state=directory- name: 拷贝calico-rbac.yaml template: src=calico-rbac.yaml dest=/etc/kubernetes/addons- name: 拷贝calico.yaml template: src=calico.yaml dest=/etc/kubernetes/addons- name: 拉取kube-controllers镜像 shell: "ansible all -m docker_image -a 'name=calico/kube-controllers tag={{ CALICO_VER }} source=pull timeout=3600'"- name: 拉取cni镜像 shell: "ansible all -m docker_image -a 'name=calico/cni tag={{ CALICO_VER }} source=pull timeout=3600'"
- name: 拉取pod2daemon-flexvol镜像 shell: "ansible all -m docker_image -a 'name=calico/pod2daemon-flexvol tag={{ CALICO_VER }} source=pull timeout=3600'"- name: 拉取node镜像 shell: "ansible all -m docker_image -a 'name=calico/node tag={{ CALICO_VER }} source=pull timeout=3600'"
- name: 部署calico-rbac shell: "kubectl apply -f /etc/kubernetes/addons/calico-rbac.yaml"
- name: 部署calico shell: "kubectl apply -f /etc/kubernetes/addons/calico.yaml"
- ingress安装ingress.yml：
vim roles/addons_install/tasks/ingress.yml
- name: 拷贝ingress-nginx.yaml template: src=ingress-nginx.yaml dest=/etc/kubernetes/addons- name: 拉取defaultbackend-amd64镜像 shell: "ansible node -m docker_image -a 'name=huqian123/nginx-ingress-default-backend tag={{ BACKEND_VER }} source=pull timeout=3600'"- name: tag defaultbackend-amd64镜像 shell: "ansible node -m shell -a 'docker tag huqian123/nginx-ingress-default-backend:{{ BACKEND_VER }} k8s.gcr.io/defaultbackend-amd64:{{ BACKEND_VER }}'"
args:
warn: False
- name: 拉取nginx-ingress-controller镜像 shell: "ansible node -m docker_image -a 'name=quay.io/kubernetes-ingress-controller/nginx-ingress-controller tag={{ INGRESS_VER }} source=pull timeout=3600'"- name: 部署ingress-nginx shell: "kubectl apply -f /etc/kubernetes/addons/ingress-nginx.yaml"
- 引用文件main.yml：
vim roles/addons_install/tasks/main.yml
- include: calico.yml- include: ingress.yml
安装测试
- 执行安装：
ansible-playbook k8s.yml
kubectl get nodes NAME STATUS ROLES AGE VERSION master Ready master 5m30s v1.18.6 node1 Ready <none> 4m27s v1.18.6 node2 Ready <none> 4m29s v1.18.6 node3 Ready <none> 4m27s v1.18.6 kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-578894d4cd-m47sg 1/1 Running 0 4m31s calico-node-89vkf 1/1 Running 0 4m31s calico-node-g2lsr 1/1 Running 0 4m31s calico-node-vdfgq 1/1 Running 0 4m31s calico-node-x8jmd 1/1 Running 0 4m31s coredns-546565776c-5gbmm 1/1 Running 0 5m58s coredns-546565776c-kvb6c 1/1 Running 0 5m58s etcd-master 1/1 Running 0 6m13s kube-apiserver-master 1/1 Running 0 6m13s kube-controller-manager-master 1/1 Running 0 6m13s kube-proxy-j8pc2 1/1 Running 0 5m15s kube-proxy-jn9wg 1/1 Running 0 5m15s kube-proxy-m5hx4 1/1 Running 0 5m58s kube-proxy-rhnbh 1/1 Running 0 5m17s kube-scheduler-master 1/1 Running 0 6m13s kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE default-http-backend-6bf4c44778-fk7rp 1/1 Running 0 4m41s ingress-nginx-ng7mh 1/1 Running 0 4m1s ingress-nginx-r4hxw 1/1 Running 0 4m38s ingress-nginx-xw7f6 1/1 Running 0 4m35s
source ~/.bash_profile #kubectl命令补全生效kubectl edit cm kube-proxy -n kube-system #修改mode为ipvskubectl delete pod -n kube-system `kubectl get pod -n kube-system |grep 'kube-proxy' | awk '{print $1}'` #重启kube-proxy
ps：建议提前准备好镜像；不建议部署至生产环境。
测试安装没有问题，注意kubernetes组件版本尽量一致。已存放至个人gitgub：ansible-playbook