非容器应用与K8s工作负载的服务网格化实践-6 基于ASM的VM应用动态落迁实践

2022-04-10 00:32:33

在完成了POD和VM之间互访验证后，本篇将进入VM中，重点关注两个常用的流量管理能力：

应用通过标签进行分组
每个分组的多个副本可以动态落组和迁出

本篇示例的拓扑如下图所示。ack中部署上游服务hello1，请求下游服务hello2。在4个ecs节点上，各部署了一个hello2应用，其中两个为en版本，与hello1之间的通信使用蓝线表示；另外两个为fr版本，与hello1之间的通信使用绿线表示。

1 搭建实验环境

部署hello1 POD

alias k="kubectl --kubeconfig $USER_CONFIG"
k apply -f yaml/hello1-deploy.yaml

部署hello2 app

在 vm1/vm2两个ecs节点上启动如下docker container，作为group1

sh sh/ssh1.sh

docker run \
--rm \
--network host \
--name http_v1 \
registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v1:1.0.1

在 vm3/vm4两个ecs节点上启动如下docker container，作为group2

sh sh/ssh3.sh

docker run \
--rm \
--network host \
--name http_v2 \
registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v2:1.0.1

部署hello2 WorkloadEntry

MESH_ID=$(head -n 1 "$MESHID_CONFIG")
aliyun servicemesh AddVmAppToMesh \
  --ServiceMeshId "$MESH_ID" \
  --Namespace vm-blue-green \
  --ServiceName hello2-svc \
  --Ips "$VM_PRI_1","$VM_PRI_2","$VM_PRI_3","$VM_PRI_4" \
  --Ports http:8001 \
  --Labels app=http-workload
echo "done"

为4个WorkloadEntry增加version标签，v1/v2的设置为v1，v3/v4的设置为v2

spec:
  address: 192.168.0.170
  labels:
    app: http-workload
    version: v1

2 蓝绿部署验证

hello2 VirtualService

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  namespace: hello-grouping
  name: hello2-vs
spec:
  hosts:
    - hello2-svc
  http:
    - name: http-route
      match:
        - uri:
            prefix: /hello
      route:
        - destination:
            host: hello2-svc
            subset: v1
          weight: 50
        - destination:
            host: hello2-svc
            subset: v2
          weight: 50

hello2 DestinationRule

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  namespace: hello-grouping
  name: hello2-dr
spec:
  host: hello2-svc
  subsets:
    - name: v1
      labels:
        version: v1
      trafficPolicy:
        loadBalancer:
          simple: ROUND_ROBIN
    - name: v2
      labels:
        version: v2
      trafficPolicy:
        loadBalancer:
          simple: ROUND_ROBIN

轮询验证

hello1_pod=$(k get pod -l app=hello1-deploy -n hello-grouping -o jsonpath={.items..metadata.name})

verify_in_loop() {
  for i in {1..8}; do
    echo ">$i test hello2-svc.hello-grouping.svc.cluster.local"
    resp=$(k exec "$hello1_pod" -c hello-v1-deploy -n hello-grouping -- \
      curl -s hello2-svc.hello-grouping.svc.cluster.local:8001/hello/eric)
    if [[ "no healthy upstream" == $resp ]]; then
      echo "stop, no healthy upstream."
      exit
    fi
    echo "$resp"
  done
}

m get workloadentry -n hello-grouping -o wide
verify_in_loop

预期的结果如下所示。流量转移首先会按照group间(v1和v2)的比例配置进行，进入group后会按负载均衡策略(ROUND_ROBIN)进行路由。

...
>5 test hello2-svc.hello-grouping.svc.cluster.local
Hello eric(192.168.0.171)
>6 test hello2-svc.hello-grouping.svc.cluster.local
Hello eric(192.168.0.170)
>7 test hello2-svc.hello-grouping.svc.cluster.local
Bonjour eric(192.168.0.172)
>8 test hello2-svc.hello-grouping.svc.cluster.local
Bonjour eric(192.168.0.198)

3 应用落迁验证

当前group1和group2各有2个实例，我们按如下顺序动态删除和增加workloadentry并验证流量：

将vm4从group2中迁出，使group1和group2节点比例为2:1
将vm2从group1中迁出，使group1和group2节点比例为1:1
将vm4落入group2，使group1和group2节点比例为1:2
将vm2落入group1，使group1和group2节点比例为2:2

hello1_pod=$(k get pod -l app=hello1-deploy -n hello-grouping -o jsonpath={.items..metadata.name})
echo "1 Test blue-green 2:1"
m delete workloadentry mesh-expansion-hello2-svc-4 -n hello-grouping
m get workloadentry -n hello-grouping -o wide
verify_in_loop

echo "2 Test blue-green 1:1"
m delete workloadentry mesh-expansion-hello2-svc-2 -n hello-grouping
m get workloadentry -n hello-grouping -o wide
verify_in_loop

echo "3 Test blue-green 1:2"
m apply -f yaml/wl4.yaml
m get workloadentry -n hello-grouping -o wide
verify_in_loop

echo "4 Test blue-green 2:2"
m apply -f yaml/wl2.yaml
m get workloadentry -n hello-grouping -o wide
verify_in_loop

verify_in_loop() {
  echo >test_traffic_result
  for i in {1..100}; do
    resp=$(k exec "$hello1_pod" -c hello-v1-deploy -n hello-grouping -- curl -s hello2-svc.hello-grouping.svc.cluster.local:8001/hello/eric)
    if [[ "no healthy upstream" == $resp ]]; then
      echo "stop, no healthy upstream."
      rm -f test_traffic_result
      exit
    fi
    echo "$resp" >>test_traffic_result
  done
  echo "result:"
  sort test_traffic_result | grep -v "^[[:space:]]*$" | uniq -c | sort -nrk1
  rm -f test_traffic_result
}

期待的结果如下。

1 Test blue-green 2:1
workloadentry.networking.istio.io "mesh-expansion-hello2-svc-4" deleted
NAME                          AGE
mesh-expansion-hello2-svc-1   28m
mesh-expansion-hello2-svc-2   64s
mesh-expansion-hello2-svc-3   28m
result:
  56 Bonjour eric(192.168.0.172)
  22 Hello eric(192.168.0.171)
  22 Hello eric(192.168.0.170)

2 Test blue-green 1:1
workloadentry.networking.istio.io "mesh-expansion-hello2-svc-2" deleted
NAME                          AGE
mesh-expansion-hello2-svc-1   28m
mesh-expansion-hello2-svc-3   28m
result:
  51 Bonjour eric(192.168.0.172)
  49 Hello eric(192.168.0.170)

3 Test blue-green 1:2
workloadentry.networking.istio.io/mesh-expansion-hello2-svc-4 created
NAME                          AGE
mesh-expansion-hello2-svc-1   29m
mesh-expansion-hello2-svc-3   29m
mesh-expansion-hello2-svc-4   0s
result:
  53 Hello eric(192.168.0.170)
  24 Bonjour eric(192.168.0.198)
  23 Bonjour eric(192.168.0.172)

4 Test blue-green 2:2
workloadentry.networking.istio.io/mesh-expansion-hello2-svc-2 created
NAME                          AGE
mesh-expansion-hello2-svc-1   29m
mesh-expansion-hello2-svc-2   1s
mesh-expansion-hello2-svc-3   29m
mesh-expansion-hello2-svc-4   37s
result:
  26 Hello eric(192.168.0.171)
  26 Hello eric(192.168.0.170)
  24 Bonjour eric(192.168.0.198)
  24 Bonjour eric(192.168.0.172)

到此，VM应用动态落迁实践验证完毕。通过本篇实验，我们可以掌握如何将VM应用进行分组，并根据实际情况，通过workload entry进行动态落组和迁出。

码农公寓