[关闭]
@zhangsiming65965 2019-03-24T05:46:48.000000Z 字数 33656 阅读 136

Kubernetes深入学习(上)

K8S

---Author：张思明 ZhangSiming

---Mail：1151004164@cnu.edu.cn

---QQ：1030728296

如果有梦想，就放开的去追；
因为只有奋斗，才能改变命运。

一、kubectl命令行管理工具

1.1kubectl管理命令概要

命令 描述
create 通过文件名或者标准输入创建资源，例如-f+.yaml文件创建或者创建集群角色等...
expose 将一个资源公开为一个新的service
run 在集群中运行一个特定的镜像
set 在对象上设置特定的功能
get 查看一个或多个资源，常见的有get pods或者get svc，get node等...还支持-o wide查看详细信息或者-n kube-system指定不同的命名空间等
explain 文档参考资料
edit 使用默认的编辑器编辑一个资源
delete 通过文件名、标准输入、资源名称或者标签选择器来删除资源
命令 描述
rollout 管理资源的发布
rolling-update 对给定的复制控制器滚动更新
scale 扩容或缩容Pod数量，Deployment、ReplicaSet、RC或Job
autoscale 创建一个自动选择扩容或缩容并设置Pod数量
命令 描述
certificate 修改证书资源
cluster-info 显示集群信息
top 显示资源（CPU/Memory/Storage）使用。需要先部署Heapster
cordon 标记节点不可调度，标记了就不会有pod调度过来
uncordon 标记节点可调度
drain 驱逐节点上的应用，准备下线维护
taint 修改节点taint标记
命令 描述
describe 查看特定资源或资源组的详细信息
logs 在一个Pod中打印一个容器日志。如果Pod只有一个容器，容器名称是可选的
attach 附加到一个运行的容器
exec 执行命令到容器
port-forward 转发一个或多个本地端口到一个pod
proxy 运行一个proxy到KubernetesAPIserver
cp 拷贝文件或目录到容器中
auth 检查授权
命令 描述
apply 通过文件名或标准输入对资源应用配置
patch 使用补丁修改、更新资源的字段
replace 通过文件名或标准输入替换一个资源
convert 不同的API版本之间转换配置文件
命令 描述
label 更新资源上的标签
annotate 更新资源上的注释
completion 用于实现kubectl工具自动补全
命令 描述
api-versions 打印受支持的API版本
config 生成或者修改kubeconfig文件（用于访问API，比如配置认证信息）
help 所有命令帮助
plugin 运行一个命令行插件
version 打印客户端和服务版本信息

1.2kubectl管理应用程序生命周期

流程：

创建-->发布-->更新-->回滚-->删除

  1. #创建一个label为nginx的项目，3个副本(高可用)，使用nginx1.14版本镜像，端口为80
  2. [root@ZhangSiming ~]# kubectl run nginx --replicas=3 --image=nginx:1.10 --port=80
  3. kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
  4. deployment.apps/nginx created
  5. #K8S项目的发布是由镜像发布的，不是jar包或者war包，发布之后由scheduler自动调度到合适的Node
  6. [root@ZhangSiming ~]# kubectl get pods
  7. NAME                     READY   STATUS    RESTARTS   AGE
  8. nginx-7b67cfbf9f-6t7z9   1/1     Running   0          18s
  9. nginx-7b67cfbf9f-96v9d   1/1     Running   0          18s
  10. nginx-7b67cfbf9f-q5hgl   1/1     Running   0          18s
  11. #事实上，除了pod，操作的run命令是生产了deployment，是用来管理更新项目的
  12. [root@ZhangSiming ~]# kubectl get deployments,pods
  13. NAME                          DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
  14. deployment.extensions/nginx   3         3         3            3           3m29s
  16. NAME                         READY   STATUS    RESTARTS   AGE
  17. pod/nginx-7b67cfbf9f-6t7z9   1/1     Running   0          3m28s
  18. pod/nginx-7b67cfbf9f-96v9d   1/1     Running   0          3m28s
  19. pod/nginx-7b67cfbf9f-q5hgl   1/1     Running   0          3m28s
  1. [root@ZhangSiming ~]# kubectl expose deployment nginx --port=100 --type=NodePort --target-port=80 --name=nginx-service
  2. #指定deployment更新项目，开放服务供外界连接
  3. #--port表示供Node集群访问端口
  4. #--type=NodePort这个类型是供外界访问的类型
  5. #--target-port是容器内的服务端口
  6. service/nginx-service exposed
  7. [root@ZhangSiming ~]# kubectl get service
  8. NAME            TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
  9. kubernetes      ClusterIP   10.0.0.1     <none>        443/TCP         27h
  10. nginx           NodePort    10.0.0.211   <none>        80:30792/TCP    28h
  11. nginx-service   NodePort    10.0.0.55    <none>        100:35373/TCP   3s
  13. #测试访问：
  14. [root@ZhangSiming ~]# curl -I 10.0.0.55:100
  15. HTTP/1.1 200 OK
  16. Server: nginx/1.14.2
  17. Date: Mon, 25 Feb 2019 07:08:54 GMT
  18. Content-Type: text/html
  19. Content-Length: 612
  20. Last-Modified: Tue, 04 Dec 2018 14:44:49 GMT
  21. Connection: keep-alive
  22. ETag: "5c0692e1-264"
  23. Accept-Ranges: bytes

image_1d4ho880c1joe60g1rum1nnv4vr9.png-30.8kB

  1. [root@ZhangSiming ~]# kubectl get pods
  2. NAME                     READY   STATUS    RESTARTS   AGE
  3. nginx-7b67cfbf9f-6t7z9   1/1     Running   0          20m
  4. nginx-7b67cfbf9f-96v9d   1/1     Running   0          20m
  5. nginx-7b67cfbf9f-q5hgl   1/1     Running   0          20m
  6. [root@ZhangSiming ~]# kubectl describe pods nginx-7b67cfbf9f-q5hgl | grep 1.10
  7.     Image:          nginx:1.10
  8.   Normal  Pulling    20m   kubelet, 192.168.17.132  pulling image "nginx:1.10"
  9.   Normal  Pulled     20m   kubelet, 192.168.17.132  Successfully pulled image "nginx:1.10"
  10. #查看好现在是nginx1.14的镜像版本，实现CI好了docker镜像，想要把服务更新为nginx1.10版本
  11. [root@ZhangSiming ~]# kubectl set image deployment/nginx nginx=nginx1.14
  12. deployment.extensions/nginx image updated
  13. #根据deployment更新nginx镜像为1.14
  14. [root@ZhangSiming ~]# kubectl set image deployment/nginx nginx=nginx:1.14
  15. deployment.extensions/nginx image updated
  16. [root@ZhangSiming ~]# kubectl get pods
  17. NAME                     READY   STATUS              RESTARTS   AGE
  18. nginx-787b58fd95-psw9g   1/1     Running             0          2m7s
  19. nginx-787b58fd95-vxtr4   1/1     Running             0          2m7s
  20. nginx-7b67cfbf9f-m6mp2   0/1     ContainerCreating   0          2s
  21. nginx-d87b556b7-tmbj6    0/1     Terminating         0          64s
  22. #更新是平滑更新，停止一个pod副本更新一个pod，不影响正常服务
  23. [root@ZhangSiming ~]# kubectl describe pod nginx-7b67cfbf9f-cksvf | grep 1.14
  24.     Image:          nginx:1.14
  25.   Normal  Pulled     2m17s  kubelet, 192.168.17.132  Container image "nginx:1.1" already present on machine
  26. #成功更新nginx为1.14版本
  1. #更新的如果有问题，要及时回滚
  2. [root@ZhangSiming ~]# kubectl rollout history deployment.apps/nginx
  3. deployment.apps/nginx 
  4. REVISION  CHANGE-CAUSE
  5. 1         <none>
  6. 4         <none>
  7. 5         <none>
  9. [root@ZhangSiming ~]# kubectl rollout undo deployment.apps/nginx --to-revision=1deployment.apps/nginx
  10. [root@ZhangSiming ~]# kubectl get pods
  11. NAME                     READY   STATUS              RESTARTS   AGE
  12. nginx-787b58fd95-7pmnd   1/1     Running             0          2s
  13. nginx-787b58fd95-rcg9d   0/1     ContainerCreating   0          0s
  14. nginx-7b67cfbf9f-m6mp2   1/1     Running             0          8m31s
  15. nginx-7b67cfbf9f-scs5h   1/1     Terminating         0          8m28s
  16. [root@ZhangSiming ~]# kubectl describe pod nginx-787b58fd95-7pmnd | grep 1.10
  17.     Image:          nginx:1.10
  18.   Normal  Pulled     60s   kubelet, 192.168.17.132  Container image "nginx:1.10" already present on machine
  19. #成功回滚到了nginx1.10版本
  1. #我们主要就是部署了deployment管理和service服务，只要删除这两个，pod自然就没了
  2. [root@ZhangSiming ~]# kubectl delete deployment/nginx
  3. deployment.extensions "nginx" deleted
  4. [root@ZhangSiming ~]# kubectl delete svc/nginx-service
  5. service "nginx-service" deleted
  6. [root@ZhangSiming ~]# kubectl get pods
  7. No resources found.
  8. #注意如果不删除deployment直接删除pod，是没有用的，因为deployment会自动生成pod

1.3kubectl远程连接K8S集群进行管理

我们上面管理K8S集群都是通过Master节点的本地kubuctl管理的，这是因为kubectl默认连接的本地的apiserver，可以进行管理。但是如果工作中我们需要远程管理怎么办？
需要生成一个kubectl配置文件，在远程节点上才能管理K8S集群。

  1. #在Node节点上
  2. [root@ZhangSiming ~]# cd /opt/kubernetes/ssl/
  3. #利用kubectl生成kubectl连接K8S集群的配置文件
  4. [root@ZhangSiming ssl]# cat kubeconfig.sh 
  5. #!/bin/bash
  6. kubectl config set-cluster kubernetes \
  7.   --certificate-authority=/opt/kubernetes/ssl/ca.pem \
  8.   --embed-certs=true \
  9.   --server=https://192.168.17.135:6443 \
  10.   --kubeconfig=config
  11. #连接的是MasterVIP
  12. kubectl config set-credentials cluster-admin \
  13.   --certificate-authority=ca.pem \
  14.   --embed-certs=true \
  15.   --client-key=admin-key.pem \
  16.   --client-certificate=admin.pem \
  17.   --kubeconfig=config
  19. kubectl config set-context default \
  20.   --cluster=kubernetes \
  21.   --user=cluster-admin \
  22.   --kubeconfig=config
  23. #上下文连接
  24. kubectl config use-context default --kubeconfig=config
  26. [root@ZhangSiming ssl]# sh kubeconfig.sh 
  27. Cluster "kubernetes" set.
  28. User "cluster-admin" set.
  29. Context "default" created.
  30. Switched to context "default".
  31. [root@ZhangSiming ssl]# mv config /opt/kubernetes/cfg
  32. [root@ZhangSiming ssl]# kubectl --kubeconfig=/opt/kubernetes/cfg/config get all
  33. NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
  34. service/kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP   28h
  35. #访问成功

二、YAML文件(资源编排)

使用kubecrl管理配置K8S集群的时候，一条一条输入的话太慢，而且也不好修改也不好调试。可以把所有的部署配置都写到一个yaml文件中，直接kubectl create -f 执行这个yaml文件，达到一键部署的效果，更适合后期的修改调试。

2.1yaml文件格式说明

YAML是一种简介的非标记语言。

语法格式

2.2YAML文件创建资源对象

  1. #yaml文件编写
  2. [root@ZhangSiming ssl]# cat nginx-deployment.yaml 
  3. apiVersion: apps/v1
  4. kind: Deployment
  5. metadata:
  6.   name: nginx-deployment
  7.   labels:
  8.     app: nginx
  9. spec:
  10.   replicas: 3
  11.   selector:
  12.     matchLabels:
  13.       app: nginx
  14. #以上是控制器定义
  15.   template:
  16.     metadata:
  17.       labels:
  18.         app: nginx
  19.     spec:
  20.       containers:
  21.       - name: nginx
  22.         image: nginx:1.15.4
  23.         ports:
  24.         - containerPort: 80
  25. #这些是被管理对象定义
  27. [root@ZhangSiming ssl]# cat nginx-service.yaml 
  28. apiVersion: v1
  29. kind: Service
  30. metadata:
  31.   name: nginx-service
  32.   labels:
  33.     app: nginx
  34. spec:
  35.   type: NodePort
  36.   ports:
  37.   - port: 80
  38.     targetPort: 80
  39.   selector:
  40.     app: nginx
  41. #执行YAML文件生成项目
  42. [root@ZhangSiming ssl]# kubectl create -f nginx-deployment.yaml
  43. deployment.apps/nginx-deployment created
  44. [root@ZhangSiming ssl]# kubectl get pods
  45. NAME                              READY   STATUS    RESTARTS   AGE
  46. nginx-deployment-d55b94fd-qd8ds   1/1     Running   0          30s
  47. nginx-deployment-d55b94fd-swmm5   1/1     Running   0          30s
  48. nginx-deployment-d55b94fd-z4npc   1/1     Running   0          30s
  49. [root@ZhangSiming ssl]# kubectl create -f nginx-service.yaml 
  50. service/nginx-service created
  51. [root@ZhangSiming ssl]# kubectl get svc
  52. NAME            TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
  53. kubernetes      ClusterIP   10.0.0.1     <none>        443/TCP        28h
  54. nginx-service   NodePort    10.0.0.189   <none>        80:30177/TCP   5s

创建什么资源，创建什么对象是要在一个文件中写全的。相比于kubectl，YAML文件更容易留存，修改调试，不再像kubectl一条一条输入繁琐。
在部署一些微服务或者复杂的服务的时候，YAML文件方式的优点就大大体现出来了。

2.3导出YAML文件留存

实际工作中，YAML字段太多记不住，我们可以从网上下载YAML模板进行修改，也可以进行YAML文件导出。

  1. #把不是真实存在的资源对象试运行之后导出yaml文件
  2. [root@ZhangSiming ssl]# kubectl run nginx --replicas=3 --image=nginx:1.14 --port=80 --dry-run -o yaml  > ~/nginx-deployment.yaml
  3. #-o指定输出格式，可选的有YAML和json等
  4. #--dry-run表示测试运行，不会真的运行
  5. kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
  6. [root@ZhangSiming ssl]# cat nginx-deployment.yaml 
  7. apiVersion: apps/v1
  8. kind: Deployment
  9. metadata:
  10.   name: nginx-deployment
  11.   labels:
  12.     app: nginx
  13. spec:
  14.   replicas: 3
  15.   selector:
  16.     matchLabels:
  17.       app: nginx
  18.   template:
  19.     metadata:
  20.       labels:
  21.         app: nginx
  22.     spec:
  23.       containers:
  24.       - name: nginx
  25.         image: nginx:1.15.4
  26.         ports:
  27.         - containerPort: 80
  29. #把已经存在的资源对象的YAML文件导出来
  30. [root@ZhangSiming ssl]# kubectl get pods
  31. NAME                              READY   STATUS    RESTARTS   AGE
  32. nginx-deployment-d55b94fd-qd8ds   1/1     Running   0          23m
  33. nginx-deployment-d55b94fd-swmm5   1/1     Running   0          23m
  34. nginx-deployment-d55b94fd-z4npc   1/1     Running   0          23m
  35. [root@ZhangSiming ssl]# kubectl get deployment.apps/nginx-deployment --export -o yaml > ~/nginx-deployment.yaml 
  36. [root@ZhangSiming ssl]# cat ~/nginx-deployment.yaml
  37. apiVersion: apps/v1
  38. kind: Deployment
  39. metadata:
  40.   labels:
  41.     app: nginx
  42.   name: nginx-deployment
  43.   replicas: 3
  44.   selector:
  45.     matchLabels:
  46.       app: nginx
  47.     type: RollingUpdate
  48.   template:
  49.     metadata:
  50.       labels:
  51.         app: nginx
  52.     spec:
  53.       containers:
  54.       - image: nginx:1.15.4
  55.         imagePullPolicy: IfNotPresent
  56.         name: nginx
  57.         ports:
  58.         - containerPort: 80
  59.           protocol: TCP
  60. #可能比较多，不认识的，或者为{}的空段可以删除
  62. #如果哪个字段不会写了，可以explain进行查看
  63. [root@ZhangSiming ssl]# kubectl explain deployment

三、深入理解Pod对象

官方网站：https://kubernetes.io/docs/concepts/

5.1Pod的介绍与容器的分类

5.1.1Pod介绍

  • 最小部署单元；
  • 一组容器的集合，将一些紧密协作的容器放到一个pod中；
  • 一个pod中的容器共享网络命名空间；
  • pod是短暂的，刚开始设置的时候是无状态的pod，调度到哪个Node节点都不用考虑。

5.1.2Pod中容器的分类

Infrastructure Container:基础容器，创建Pod的时候自动拉取kubelet配置文件中指定的镜像仓库进行构建，作用是维护整个Pod网络空间；
InitContainers：初始化容器，先于业务容器开始执行，完成一些初始化操作，哪个业务容器先运行之类的；
Containers：业务容器，所有业务容器并行启动，真正执行业务。

5.2Pod镜像拉取策略

当你创建一个资源对象的时候，镜像的构建是有一个镜像拉取策略的，由部署K8S项目的yaml文件的imagePullPolicy字段管控。
imagePullPolicy取值有三种：
1.IfNotPresent：默认值，镜像在Node节点上不存在时才拉取；
2.Always：每次创建Pod 都会重新拉取一次镜像；
3.Never：Pod 永远不会主动拉取这个镜像。

  1. [root@ZhangSiming ~]# kubectl get deploy/nginx-deployment -o yaml | grep imagePullPolicy
  2.         imagePullPolicy: IfNotPresent
  3. #查看已经运行的资源的镜像拉取策略

如果是公开的镜像仓库，可以直接根据Pod镜像拉取策略拉取镜像部署Pod；但是如果是私有云镜像仓库Harbor，还需要认证，就要在yaml文件添加一条凭证进行认证。

  1. #在已经登录Harbor的服务器获取认证信息
  2. [root@ZhangSiming ~]# docker login -uadmin -paptx65965697 www.yunjisuan2.com
  3. WARNING! Using --password via the CLI is insecure. Use --password-stdin.
  4. WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
  5. Configure a credential helper to remove this warning. See
  6. https://docs.docker.com/engine/reference/commandline/login/#credentials-store
  8. Login Succeeded
  9. [root@ZhangSiming kubernetes]# cat /root/.docker/config.json | base64 -w 0
  10. ewoJImF1dGhzIjogewoJCSJ3d3cueXVuamlzdWFuMi5jb20iOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2WVhCMGVEWTFPVFkxTmprMyIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTguMDkuMSAobGludXgpIgoJfQp9
  11. #不换行的复制下来备用
  13. #切换到Master节点
  14. [root@ZhangSiming ~]# cat centos.yaml 
  15. apiVersion: apps/v1
  16. kind: Deployment 
  17. metadata: 
  18.   name: deploy-centos 
  19. spec: 
  20.   replicas: 1 
  21.   selector: 
  22.     matchLabels: 
  23.       app: centos 
  24.   template: 
  25.     metadata: 
  26.       labels: 
  27.         app: centos 
  28.     spec: 
  29.       containers: 
  30.       - name: app-centos 
  31.         image: www.yunjisuan2.com/library/mongo
  32. #从我们自己的Harbor上拉取镜像
  33.         imagePullPolicy: Always
  34. #拉取策略是每次构建都必须重新拉取
  35.       imagePullSecrets: 
  36.       - name: registry-pull-secret
  37. #这是验证信息，在下面生成
  39. [root@ZhangSiming ~]# cat centos-secret.yaml
  40. apiVersion: v1 
  41. kind: Secret
  42. metadata: 
  43.   name: registry-pull-secret 
  44. data:
  45.   .dockerconfigjson: ewoJImF1dGhzIjogewoJCSJ3d3cueXVuamlzdWFuMi5jb20iOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2WVhCMGVEWTFPVFkxTmprMyIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTguMDkuMSAobGludXgpIgoJfQp9
  46. type: kubetnetes.io/dockerconfigjson
  47. #这是生成验证Harbor的凭据
  49. [root@ZhangSiming ~]# kubectl create -f centos-secret.yaml 
  50. secret/registry-pull-secret created
  51. [root@ZhangSiming ~]# kubectl get secrets
  52. NAME                   TYPE                                  DATA   AGE
  53. default-token-f2kfs    kubernetes.io/service-account-token   3      7d16h
  54. registry-pull-secret   kubetnetes.io/dockerconfigjson        1      7s
  55. [root@ZhangSiming ~]# kubectl create -f centos.yaml 
  56. deployment.apps/deploy-centos created
  58. #查看拉取镜像结果
  59. [root@ZhangSiming ~]# kubectl get pods
  60. NAME                             READY   STATUS    RESTARTS   AGE
  61. deploy-centos-6fc88f94c9-xbqtr   1/1     Running   0          55s
  62. #拉取成功，一定要拉取已经有的镜像哦~

5.3Pod进行资源限制

  1. [root@ZhangSiming ~]# cat limit.yaml
  2. apiVersion: v1
  3. kind: Pod
  4. metadata:
  5.   name: frontend
  6. spec:
  7.   containers:
  8.   - name: db
  9.     image: mysql
  10.     env:
  11.     - name: MYSQL_ROOT_PASSWORD
  12.       value: "password"
  13.     resources:
  14.       requests:
  15.         memory: "64Mi"
  16.         cpu: "250m"
  17.       limits:
  18.         memory: "128Mi"
  19.         cpu: "500m"
  20.   - name: wp
  21.     image: wordpress
  22.     resources:
  23.       requests:
  24.         memory: "64Mi"
  25.         cpu: "250m"
  26.       limits:
  27.         memory: "128Mi"
  28.         cpu: "500m"
  29. #字段是resources，requests是调度的时候有这么多资源才会调度到指定node，limits是运行最高占用资源不超过多少
  30. [root@ZhangSiming ~]# kubectl create -f limit.yaml 
  31. pod/frontend created
  32. [root@ZhangSiming ~]# kubectl get pods -o wide
  33. NAME       READY   STATUS    RESTARTS   AGE     IP           NODE             NOMINATED NODE
  34. frontend   2/2     Running   0          5m29s   172.17.3.3   192.168.17.132   <none>
  35. [root@ZhangSiming ~]# kubectl describe nodes 192.168.17.132
  36. #可以查看到限制资源的信息

5.4Pod重启策略

  • Always：当容器终止退出后，总是重启容器，默认策略;
  • OnFailure：当容器异常退出（退出状态码非0）时，才重启容器;
  • Never:：当容器终止退出，从不重启容器.
    一直运行的容器，重启策略设置为Always就可以了，有特殊需求再改。

5.5Pod健康检查(Probe探针)

Probe有以下两种类型：

  • livenessProbe
    如果检查失败，将杀死容器，根据Pod的restartPolicy(三种重启策略)来操作；
  • readinessProbe
    如果检查失败，Kubernetes会把Pod从service endpoints中剔除，用户就访问不了了。

Probe支持以下三种检查方法：

  • httpGet
    发送HTTP请求，返回200-400范围状态码为成功;
  • exec
    执行Shell命令返回状态码是0为成功;
  • tcpSocket
    发起TCP Socket建立成功,看socket进程有没有响应。
  1. [root@ZhangSiming ~]# cat healthy.yaml 
  2. apiVersion: v1
  3. kind: Pod
  4. metadata:
  5.   labels:
  6.     test: liveness
  7.   name: liveness-exec
  8. spec:
  9.   containers:
  10.   - name: liveness
  11.     image: busybox
  12.     args:
  13.     - /bin/sh
  14.     - -c
  15.     - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy
  16.     livenessProbe:
  17.       exec:
  18.         command:
  19.         - cat
  20.         - /tmp/healthy
  21.       initialDelaySeconds: 5
  22.       periodSeconds: 5
  23. #在30秒之后的健康检查会触发重启pod
  24. [root@ZhangSiming ~]# kubectl create -f healthy.yaml
  25. pod/liveness-exec created
  26. [root@ZhangSiming ~]# kubectl get pods
  27. NAME            READY   STATUS    RESTARTS   AGE
  28. liveness-exec   1/1     Running   0          7s
  29. [root@ZhangSiming ~]# kubectl get pods
  30. NAME            READY   STATUS    RESTARTS   AGE
  31. liveness-exec   1/1     Running   0          19s
  32. [root@ZhangSiming ~]# kubectl get pods
  33. NAME            READY   STATUS    RESTARTS   AGE
  34. liveness-exec   1/1     Running   1          33s

5.6Pod调度约束

默认的调度是根据每个节点的资源利用率进行调度，已经很合理了。实际工作中我们还可以根据需求进行控制调度。

image_1d4plagj21e951faj1186hoog789.png-55.3kB

Pod发布流程

1.用户在通过kubectl创建Pod；
2.apiserver写入Pod信息到Etcd-cluster中；
3.Scheduler通过watch获取到Pod信息，根据设置的调度算法返回合理的调度节点信息到Etcd集群，之后返回apiserver；
4.调度节点的kubelet通过watch检测到自己是被选举出来的调度节点，之后在自身部署Pod；
5.启动docker容器引擎运行服务；
6.kubelet更新信息回到Etcd-cluster集群之后，返回给apiserver，用户就可以看到Pod部署状况了。

调度约束

  • nodeName用于将Pod调度到指定的Node名称上
  • nodeSelector用于将Pod调度到匹配Label的Node上
  1. [root@ZhangSiming ~]# cat sche.yaml
  2. apiVersion: v1
  3. kind: Pod
  4. metadata:
  5.   name: pod-example
  6.   labels:
  7.     app: nginx
  8. spec:
  9.   nodeName: 192.168.17.132
  10. #nodeName是跳过调度器，强制调度
  11.   containers:
  12.   - name: nginx
  13.     image: nginx:1.15
  14. [root@ZhangSiming ~]# kubectl create -f sche.yaml 
  15. pod/pod-example created
  16. [root@ZhangSiming ~]# kubectl get pods -o wide
  17. NAME          READY   STATUS    RESTARTS   AGE   IP           NODE             NOMINATED NODE
  18. pod-example   1/1     Running   0          16s   172.17.3.3   192.168.17.132   <none>
  1. [root@ZhangSiming ~]# kubectl label nodes 192.168.17.132 team=a
  2. node/192.168.17.132 labeled
  3. #给Node一个label
  4. [root@ZhangSiming ~]# kubectl get nodes --show-labels
  5. NAME             STATUS     ROLES    AGE     VERSION   LABELS
  6. 192.168.17.132   Ready      <none>   7d16h   v1.12.1   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=192.168.17.132,team=a
  7. 192.168.17.133   NotReady   <none>   7d15h   v1.12.1   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=192.168.17.133
  8. [root@ZhangSiming ~]# vim sche.yaml 
  9. [root@ZhangSiming ~]# cat sche.yaml
  10. apiVersion: v1
  11. kind: Pod
  12. metadata:
  13.   name: pod-example
  14.   labels:
  15.     app: nginx
  16. spec:
  17.   nodeSelector: 
  18.     team: a
  19. #通过调度器调度到team：a的node节点
  20.   containers:
  21.   - name: nginx
  22.     image: nginx:1.15
  23. [root@ZhangSiming ~]# kubectl create -f sche.yaml 
  24. pod/pod-example created
  25. [root@ZhangSiming ~]# kubectl get pods -o wide
  26. NAME          READY   STATUS    RESTARTS   AGE   IP           NODE             NOMINATED NODE
  27. pod-example   1/1     Running   0          25s   172.17.3.3   192.168.17.132   <none>

5.7Pod故障排查

5.7.1Pod的五种状态

状态 描述
Pending Pod创建已经提交到Kubernetes。但是，因为某种原因而不能顺利创建。例如下载镜像慢，调度不成功
Running Pod已经绑定到一个节点，并且已经创建了所有容器。至少有一个容器正在运行中，或正在启动或重新启动
Succeeded Pod中的所有容器都已成功终止，不会重新启动
Failed Pod的所有容器均已终止，且至少有一个容器已在故障中终止。也就是说，容器要么以非零状态退出，要么被系统终止
Unknown 由于某种原因apiserver无法获得Pod的状态，通常是由于Master与Pod所在主机kubelet通信时出错 
  1. [root@ZhangSiming ~]# kubectl describe TYPE/NAME
  2. #查看事件，可以看事件位置，是卡在拉取镜像，还是卡在调度
  3. [root@ZhangSiming ~]# kubectl logs TYPE/NAME [-c CONTAINER]
  4. #查看容器里的日志
  5. [root@ZhangSiming ~]# kubectl exec POD [-c CONTAINER] --COMMAND [args...]
  6. #可以进入到容器中查看应用的状态

四、深入理解Service(与外界连通)

1.Service是为了防止Pod失联，Service可以动态感知Pod的ip变化；
2.定义一组Pod的访问策略，把一些Pod关联起来，同时起到负载均衡的效果，把请求平均分到Pod的副本上；
3.Service的三种类型：ClusterIP、NodePort、LoadBalance；
4.Service的底层实现主要有Iptables和IPVS梁总网络模式，这两种网络模式决定了如何去转发请求的流量，和怎么起到负载均衡的作用。

4.1Pod与Service的关系

image_1d6fdfbpv14qibkup3c1epr1c7p9.png-40.6kB

通过Pod指定标签，Service上使用Selector匹配标签关联Service与Pod。
Service实现了Pod的4层负载均衡(TCP/UDP)，默认是根据endpoints控制台检测到的Podip地址进行轮询算法调度服务。

4.2Service定义

  1. [root@ZhangSiming ~]# vim service.yaml
  2. [root@ZhangSiming ~]# cat service.yaml
  3. kind: Service
  4. apiVersion: v1
  5. metadata:
  6.   name: my-service
  7.   namespace: default
  8. spec:
  9.   clusterIP: 10.0.0.123
  10. #Service默认是ClusterIP类型，如果不指定默认分配一个随机的ClusterIP
  11.   selector:
  12.     app: nginx
  13. #这个selector是为了匹配用的，好让K8S知道这个是哪个Pod的Service
  14.   ports:
  15.   - name: http
  16.     protocol: TCP
  17.     port: 80
  18. #Service的80端口
  19.     targetPort: 80
  20. #
  22. #创建Service
  23. [root@ZhangSiming ~]# kubectl create -f service.yaml 
  24. service/my-service created
  25. [root@ZhangSiming ~]# kubectl get svc
  26. NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
  27. kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP   21d
  28. my-service   ClusterIP   10.0.0.123   <none>        80/TCP    6s
  29. [root@ZhangSiming ~]# kubectl get endpoints
  30. NAME         ENDPOINTS             AGE
  31. kubernetes   192.168.17.130:6443   21d
  32. my-service   <none>                66s
  33. #endpoints是可以查看Service关联到哪些Pod的，这里我们没有关联

解释：Service是为了防止Pod失联，Service可以动态感知Pod的ip变化
实际是由endpoints来感知的，当Pod重启有了新的IP，endpoints控制器感知到新的IP，之后将新的IP关联到Service，才可以访问。

  1. #可以describe查看Service详细信息
  2. [root@ZhangSiming ~]# kubectl describe svc my-service
  3. Name:              my-service
  4. Namespace:         default
  5. Labels:            <none>
  6. Annotations:       <none>
  7. Selector:          app=nginx
  8. Type:              ClusterIP
  9. IP:                10.0.0.123
  10. Port:              http  80/TCP
  11. TargetPort:        80/TCP
  12. Endpoints:         <none>
  13. Session Affinity:  None
  14. Events:            <none>

4.3Service类型

image_1d6fdqrf3smp51b130jptlult19.png-55.6kB

这种方式是用于集群内部组件之间访问的，是默认的Service类型。Service只是逻辑上将多个Pod关联起来，底层是Iptables或者IPVS实现的负载均衡，只需要知道Service的clusterip即可。

YAML格式：

  1. [root@ZhangSiming ~]# cat service.yaml
  2. apiVersion: v1
  3. kind: Service
  4. metadata:
  5.   name: my-service
  6. spec:
  7.   selector:
  8.     app: A
  9.   ports:
  10.     - protocol: TCP
  11.       port: 80
  12.       targetPort: 8080

image_1d6fe2uq4nna35d1mjikn51fnc1m.png-56.2kB

NodePort类型的Service不仅仅会生成一个CLusterIP类型的集群内部访问端口，还会在每个关联的Pod上都开放监听一个端口，可以通过这个端口在集群外部访问到Pod。
NodeIP一般是不固定的，开放的Port一般是需要固定的。

YAML格式：

  1. [root@ZhangSiming ~]# cat service.yaml
  2. apiVersion: v1
  3. kind: Service
  4. metadata:
  5.   name: my-service
  6. spec:
  7.   selector:
  8.     app: A
  9.   ports:
  10.     - protocol: TCP
  11.       port: 80
  12.       targetPort: 8080
  13.       nodePort: 30001
  14. #固定nodePort端口
  15.   type: NodePort
  16. [root@ZhangSiming ~]# kubectl apply -f service.yaml 
  17. #更新服务
  18. Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
  19. service/my-service configured
  20. [root@ZhangSiming ~]# kubectl get svc
  21. NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
  22. kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP        21d
  23. my-service   NodePort    10.0.0.123   <none>        80:30001/TCP   29m
  24. #80是ClusterIP访问端口；30001是NodePort访问端口
  26. #Node节点
  27. [root@ZhangSiming ~]# netstat -antup | grep 30001
  28. tcp6       0      0 :::30001                :::*                    LISTEN      884/kube-proxy

image_1d6fesa9n4frnfb18fcd9u9g23.png-54kB

LoadBalance类型一般是工作在公有云上的，不适用于自身创建的K8S集群。用户访问会先连接公有云上的LB负载均衡，LB负载均衡自动关联到下面Service的IP进行访问。

LoadBalance提供特定云提供商底层LB接口。利于AWS、Google、Openstack，自动调用负载均衡关联底层IP。

4.4Service代理模式

实际上Service的底层流量转发与负载均衡实现是由K8SNode节点的kube-proxy进行的。

  1. [root@ZhangSiming kubernetes]# cat cfg/kube-proxy 
  2. KUBE_PROXY_OPTS=\"--logtostderr=true \
  3. --v=4 \
  4. --hostname-override=192.168.17.132 \
  5. --cluster-cidr=10.0.0.0/24 \
  6. --proxy-mode=ipvs \
  7. #我们现在使用的是IPVS作为Service底层代理
  8. --kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig"
  1. #切换IPVS代理模式到iptables代理模式
  2. [root@ZhangSiming kubernetes]# vim cfg/kube-proxy
  3. [root@ZhangSiming kubernetes]# cat cfg/kube-proxy
  5. KUBE_PROXY_OPTS="--logtostderr=true \
  6. --v=4 \
  7. --hostname-override=192.168.17.132 \
  8. --cluster-cidr=10.0.0.0/24 \
  9. --kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig"
  11. [root@ZhangSiming kubernetes]# systemctl restart kube-proxy.service 
  12. [root@ZhangSiming kubernetes]# ps -elf | grep kube-proxy
  13. 4 S root      21842      1  1  80   0 - 10661 futex_ 14:40 ?        00:00:00 /opt/kubernetes/bin/kube-proxy --logtostderr=true --v=4 --hostname-override=192.168.17.132 --cluster-cidr=10.0.0.0/24 --kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig
  14. 0 R root      21969   1315  0  80   0 - 28176 -      14:40 pts/0    00:00:00 grep --color=auto kube-proxy
  1. [root@ZhangSiming ~]# kubectl create -f nginx.yaml
  2. deployment.apps/nginx created
  3. [root@ZhangSiming ~]# kubectl create -f service.yaml 
  4. service/my-service created
  5. [root@ZhangSiming ~]# kubectl get svc
  6. NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
  7. kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP        21d
  8. my-service   NodePort    10.0.0.133   <none>        80:30001/TCP   27s
  9. #找出Service关联的集群PodIP
  1. [root@ZhangSiming kubernetes]# iptables-save | grep 10.0.0.133
  2. -A KUBE-SERVICES ! -s 10.0.0.0/24 -d 10.0.0.133/32 -p tcp -m comment --comment "default/my-service: cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
  3. -A KUBE-SERVICES -d 10.0.0.133/32 -p tcp -m comment --comment "default/my-service: cluster IP" -m tcp --dport 80 -j KUBE-SVC-KEAUNL7HVWWSEZA6
  4. [root@ZhangSiming kubernetes]# iptables-save | grep KUBE-SVC-KEAUNL7HVWWSEZA6
  5. :KUBE-SVC-KEAUNL7HVWWSEZA6 - [0:0]
  6. -A KUBE-NODEPORTS -p tcp -m comment --comment "default/my-service:" -m tcp --dport 30001 -j KUBE-SVC-KEAUNL7HVWWSEZA6
  7. -A KUBE-SERVICES -d 10.0.0.133/32 -p tcp -m comment --comment "default/my-service: cluster IP" -m tcp --dport 80 -j KUBE-SVC-KEAUNL7HVWWSEZA6
  8. -A KUBE-SVC-KEAUNL7HVWWSEZA6 -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-JQ4W2NCACGVYJ6IM
  9. -A KUBE-SVC-KEAUNL7HVWWSEZA6 -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-YVQHBEOICAVG6NCJ
  10. -A KUBE-SVC-KEAUNL7HVWWSEZA6 -j KUBE-SEP-H32GYTWB4BTUZQJM
  11. #可以看出是轮询算法，从上往下匹配，第一个0.3概率，之后1:1概率实现Service的轮询转发

iptables方式的弊端：

1.会创建规则(更新的话，是非增量式更新，比较耗费资源)；
2.iptables规则是从上到下进行逐条匹配的，如果规模较大，可能会有延迟。

引入IPVS：

基于此，引入了IPVS方式代理。
好处是：任何处理都是基于内核进行处理的，效率高，可以解决iptables遇到的问题。

  1. [root@ZhangSiming ~]# kubectl get endpoints
  2. NAME         ENDPOINTS                                      AGE
  3. kubernetes   192.168.17.130:6443                            21d
  4. my-service   172.17.83.4:80,172.17.83.5:80,172.17.83.6:80   38m
  5. [root@ZhangSiming ~]# kubectl get svc
  6. NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
  7. kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP        21d
  8. my-service   NodePort    10.0.0.133   <none>        80:30001/TCP   39m
  1. [root@ZhangSiming kubernetes]# ipvsadm -ln | grep -A 3 10.0.0.133
  2. TCP  10.0.0.133:80 rr
  3.   -> 172.17.83.4:80               Masq    1      0          0         
  4.   -> 172.17.83.5:80               Masq    1      0          0         
  5.   -> 172.17.83.6:80               Masq    1      0          0     
  6. #基于内核态的IPVS负载均衡要比基于用户态的iptables负载均衡要好的多的多，效率大得多，所以一般kube-proxy我们选择IPVS代理模式。

综上所述，Service代理模式iptables与IPVS对比

iptables IPVS
灵活，功能强大（可以在数据包不同阶段对包进行操作） 工作在内核态，有更好的性能
规则遍历匹配和更新，呈线性时延 调度算法丰富：rr，wrr，lc，wlc，ip hash...

轮询算法等配置都在kube-proxy配置文件中配置。

4.5DNS(coredns)

DNS服务监视Kubernetes API，为每一个Service创建DNS记录用于域名解析。

  1. [root@ZhangSiming ~]# cat /opt/kubernetes/cfg/kubelet.config 
  3. kind: KubeletConfiguration
  4. apiVersion: kubelet.config.k8s.io/v1beta1
  5. address: 192.168.17.132
  6. port: 10250
  7. cgroupDriver: cgroupfs
  8. clusterDNS:
  9. - 10.0.0.2 
  10. clusterDomain: cluster.local.
  11. #这里的clusterDomain和clusterDNS要在后面创建coreDNS进行对应修改
  12. failSwapOn: false
  13. authentication:
  14.   anonymous:
  15.     enabled: true
  16. [root@ZhangSiming ~]# vim coredns.yaml.sed 
  17. [root@ZhangSiming ~]# cat coredns.yaml.sed
  18. # Warning: This is a file generated from the base underscore template file: coredns.yaml.base
  20. apiVersion: v1
  21. kind: ServiceAccount
  22. metadata:
  23.   name: coredns
  24.   namespace: kube-system
  25.   labels:
  26.       kubernetes.io/cluster-service: "true"
  27.       addonmanager.kubernetes.io/mode: Reconcile
  28. ---
  29. apiVersion: rbac.authorization.k8s.io/v1
  30. kind: ClusterRole
  31. metadata:
  32.   labels:
  33.     kubernetes.io/bootstrapping: rbac-defaults
  34.     addonmanager.kubernetes.io/mode: Reconcile
  35.   name: system:coredns
  36. rules:
  37. - apiGroups:
  38.   - ""
  39.   resources:
  40.   - endpoints
  41.   - services
  42.   - pods
  43.   - namespaces
  44.   verbs:
  45.   - list
  46.   - watch
  47. - apiGroups:
  48.   - ""
  49.   resources:
  50.   - nodes
  51.   verbs:
  52.   - get
  53. ---
  54. apiVersion: rbac.authorization.k8s.io/v1
  55. kind: ClusterRoleBinding
  56. metadata:
  57.   annotations:
  58.     rbac.authorization.kubernetes.io/autoupdate: "true"
  59.   labels:
  60.     kubernetes.io/bootstrapping: rbac-defaults
  61.     addonmanager.kubernetes.io/mode: EnsureExists
  62.   name: system:coredns
  63. roleRef:
  64.   apiGroup: rbac.authorization.k8s.io
  65.   kind: ClusterRole
  66.   name: system:coredns
  67. subjects:
  68. - kind: ServiceAccount
  69.   name: coredns
  70.   namespace: kube-system
  71. ---
  72. apiVersion: v1
  73. kind: ConfigMap
  74. metadata:
  75.   name: coredns
  76.   namespace: kube-system
  77.   labels:
  78.       addonmanager.kubernetes.io/mode: EnsureExists
  79. data:
  80.   Corefile: |
  81.     .:53 {
  82.         errors
  83.         health
  84.         kubernetes cluster.local in-addr.arpa ip6.arpa {
  85. #修改coreDNS域
  86.             pods insecure
  87.             upstream
  88.             fallthrough in-addr.arpa ip6.arpa
  89.         }
  90.         prometheus :9153
  91.         forward . /etc/resolv.conf
  92.         cache 30
  93.         loop
  94.         reload
  95.         loadbalance
  96.     }
  97. ---
  98. apiVersion: apps/v1
  99. kind: Deployment
  100. metadata:
  101.   name: coredns
  102.   namespace: kube-system
  103.   labels:
  104.     k8s-app: kube-dns
  105.     kubernetes.io/cluster-service: "true"
  106.     addonmanager.kubernetes.io/mode: Reconcile
  107.     kubernetes.io/name: "CoreDNS"
  108. spec:
  109.   # replicas: not specified here:
  110.   # 1. In order to make Addon Manager do not reconcile this replicas parameter.
  111.   # 2. Default is 1.
  112.   # 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on.
  113.   strategy:
  114.     type: RollingUpdate
  115.     rollingUpdate:
  116.       maxUnavailable: 1
  117.   selector:
  118.     matchLabels:
  119.       k8s-app: kube-dns
  120.   template:
  121.     metadata:
  122.       labels:
  123.         k8s-app: kube-dns
  124.       annotations:
  125.         seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
  126.     spec:
  127.       priorityClassName: system-cluster-critical
  128.       serviceAccountName: coredns
  129.       tolerations:
  130.         - key: "CriticalAddonsOnly"
  131.           operator: "Exists"
  132.       nodeSelector:
  133.         beta.kubernetes.io/os: linux
  134.       containers:
  135.       - name: coredns
  136.         image: coredns/coredns:1.3.1
  137. #修改拉取的镜像源
  138.         imagePullPolicy: IfNotPresent
  139.         resources:
  140.           limits:
  141.             memory: 170Mi
  142.           requests:
  143.             cpu: 100m
  144.             memory: 70Mi
  145.         args: [ "-conf", "/etc/coredns/Corefile" ]
  146.         volumeMounts:
  147.         - name: config-volume
  148.           mountPath: /etc/coredns
  149.           readOnly: true
  150.         ports:
  151.         - containerPort: 53
  152.           name: dns
  153.           protocol: UDP
  154.         - containerPort: 53
  155.           name: dns-tcp
  156.           protocol: TCP
  157.         - containerPort: 9153
  158.           name: metrics
  159.           protocol: TCP
  160.         livenessProbe:
  161.           httpGet:
  162.             path: /health
  163.             port: 8080
  164.             scheme: HTTP
  165.           initialDelaySeconds: 60
  166.           timeoutSeconds: 5
  167.           successThreshold: 1
  168.           failureThreshold: 5
  169.         readinessProbe:
  170.           httpGet:
  171.             path: /health
  172.             port: 8080
  173.             scheme: HTTP
  174.         securityContext:
  175.           allowPrivilegeEscalation: false
  176.           capabilities:
  177.             add:
  178.             - NET_BIND_SERVICE
  179.             drop:
  180.             - all
  181.           readOnlyRootFilesystem: true
  182.       dnsPolicy: Default
  183.       volumes:
  184.         - name: config-volume
  185.           configMap:
  186.             name: coredns
  187.             items:
  188.             - key: Corefile
  189.               path: Corefile
  190. ---
  191. apiVersion: v1
  192. kind: Service
  193. metadata:
  194.   name: kube-dns
  195.   namespace: kube-system
  196.   annotations:
  197.     prometheus.io/port: "9153"
  198.     prometheus.io/scrape: "true"
  199.   labels:
  200.     k8s-app: kube-dns
  201.     kubernetes.io/cluster-service: "true"
  202.     addonmanager.kubernetes.io/mode: Reconcile
  203.     kubernetes.io/name: "CoreDNS"
  204. spec:
  205.   selector:
  206.     k8s-app: kube-dns
  207.   clusterIP: 10.0.0.2
  208. #修改clusterIP
  209.   ports:
  210.   - name: dns
  211.     port: 53
  212.     protocol: UDP
  213.   - name: dns-tcp
  214.     port: 53
  215.     protocol: TCP
  216.   - name: metrics
  217.     port: 9153
  218.     protocol: TCP
  221. #启动coredns
  222. [root@ZhangSiming ~]# kubectl create -f coredns.yaml.sed
  223. serviceaccount/coredns created
  224. clusterrole.rbac.authorization.k8s.io/system:coredns created
  225. clusterrolebinding.rbac.authorization.k8s.io/system:coredns created
  226. configmap/coredns created
  227. deployment.apps/coredns created
  228. service/kube-dns created
  229. [root@ZhangSiming ~]# kubectl get pods -n kube-system
  230. NAME                                    READY   STATUS    RESTARTS   AGE
  231. coredns-64479cf49b-6mgpv                0/1     Running   0          17s
  232. kubernetes-dashboard-5f5bfdc89f-zkc9n   1/1     Running   6          24d
  234. #测试DNS解析
  235. [root@ZhangSiming ~]# kubectl get svc
  236. NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
  237. kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP        21d
  238. my-service   NodePort    10.0.0.133   <none>        80:30001/TCP   126m
  239. #coredns解析的是Service的NAME和CLUSTER-IP对应关系
  240. [root@ZhangSiming ~]# kubectl run -it --image=busybox:1.28.4 --rm --restart=Never sh
  241. If you don\'t see a command prompt, try pressing enter.
  242. / # nslookup kubernetes
  243. Server:    10.0.0.2
  244. Address 1: 10.0.0.2 kube-dns.kube-system.svc.cluster.local
  246. Name:      kubernetes
  247. Address 1: 10.0.0.1 kubernetes.default.svc.cluster.local
  248. / # nslookup my-service
  249. Server:    10.0.0.2
  250. Address 1: 10.0.0.2 kube-dns.kube-system.svc.cluster.local
  252. Name:      my-service
  253. Address 1: 10.0.0.133 my-service.default.svc.cluster.local
  254. #成功解析
  1. [root@ZhangSiming ~]# kubectl get svc -n default
  2. NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
  3. kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP        21d
  4. my-service   NodePort    10.0.0.133   <none>        80:30001/TCP   131m
  5. [root@ZhangSiming ~]# kubectl run -it --image=busybox:1.28.4 --rm --restart=Never sh -n kube-system
  6. If you don't see a command prompt, try pressing enter.
  7. / # nslookup my-service
  8. Server:    10.0.0.2
  9. Address 1: 10.0.0.2 kube-dns.kube-system.svc.cluster.local
  11. nslookup: can't resolve 'my-service'
  12. / # nslookup my-service.default
  13. Server:    10.0.0.2
  14. Address 1: 10.0.0.2 kube-dns.kube-system.svc.cluster.local
  16. Name:      my-service.default
  17. Address 1: 10.0.0.133 my-service.default.svc.cluster.local
  18. #在Service的NAME后面加.namespace可以跨命名空间进行Service的DNS解析

五、Ingress学习

5.1K8S的服务类别

5.2Ingress Controller部署

image_1d6ft8324hshu6q1ou9fgb1qb42g.png-25.6kB

通过Ingress Controller控制器关联域名和服务，进行暴露服务对外访问。

Ingress Controller控制器YAML下载地址:

https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml

  1. [root@ZhangSiming ~]# mkdir ingress
  2. [root@ZhangSiming ~]# cd ingress/
  3. [root@ZhangSiming ingress]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
  4. --2019-03-21 18:44:09--  https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
  5. Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.228.133
  6. Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.228.133|:443... connected.
  7. HTTP request sent, awaiting response... 200 OK
  8. Length: 5976 (5.8K) [text/plain]
  9. Saving to: ‘mandatory.yaml’
  11. 100%[======================================>] 5,976       --.-K/s   in 0s      
  13. 2019-03-21 18:44:10 (38.5 MB/s) - ‘mandatory.yaml’ saved [5976/5976]
  14. [root@ZhangSiming ingress]# ls
  15. mandatory.yaml
  16. #查看Ingress ControllerYAML文件
  17. [root@ZhangSiming ingress]# cat mandatory.yaml 
  18. apiVersion: v1
  19. kind: Namespace
  20. metadata:
  21.   name: ingress-nginx
  22.   labels:
  23.     app.kubernetes.io/name: ingress-nginx
  24.     app.kubernetes.io/part-of: ingress-nginx
  26. ---
  28. kind: ConfigMap
  29. apiVersion: v1
  30. metadata:
  31.   name: nginx-configuration
  32.   namespace: ingress-nginx
  33.   labels:
  34.     app.kubernetes.io/name: ingress-nginx
  35.     app.kubernetes.io/part-of: ingress-nginx
  37. ---
  38. kind: ConfigMap
  39. apiVersion: v1
  40. metadata:
  41.   name: tcp-services
  42.   namespace: ingress-nginx
  43.   labels:
  44.     app.kubernetes.io/name: ingress-nginx
  45.     app.kubernetes.io/part-of: ingress-nginx
  47. ---
  48. kind: ConfigMap
  49. apiVersion: v1
  50. metadata:
  51.   name: udp-services
  52.   namespace: ingress-nginx
  53.   labels:
  54.     app.kubernetes.io/name: ingress-nginx
  55.     app.kubernetes.io/part-of: ingress-nginx
  57. ---
  58. apiVersion: v1
  59. kind: ServiceAccount
  60. metadata:
  61.   name: nginx-ingress-serviceaccount
  62.   namespace: ingress-nginx
  63.   labels:
  64.     app.kubernetes.io/name: ingress-nginx
  65.     app.kubernetes.io/part-of: ingress-nginx
  67. ---
  68. apiVersion: rbac.authorization.k8s.io/v1beta1
  69. kind: ClusterRole
  70. metadata:
  71.   name: nginx-ingress-clusterrole
  72.   labels:
  73.     app.kubernetes.io/name: ingress-nginx
  74.     app.kubernetes.io/part-of: ingress-nginx
  75. rules:
  76.   - apiGroups:
  77.       - ""
  78.     resources:
  79.       - configmaps
  80.       - endpoints
  81.       - nodes
  82.       - pods
  83.       - secrets
  84.     verbs:
  85.       - list
  86.       - watch
  87.   - apiGroups:
  88.       - ""
  89.     resources:
  90.       - nodes
  91.     verbs:
  92.       - get
  93.   - apiGroups:
  94.       - ""
  95.     resources:
  96.       - services
  97.     verbs:
  98.       - get
  99.       - list
  100.       - watch
  101.   - apiGroups:
  102.       - "extensions"
  103.     resources:
  104.       - ingresses
  105.     verbs:
  106.       - get
  107.       - list
  108.       - watch
  109.   - apiGroups:
  110.       - ""
  111.     resources:
  112.       - events
  113.     verbs:
  114.       - create
  115.       - patch
  116.   - apiGroups:
  117.       - "extensions"
  118.     resources:
  119.       - ingresses/status
  120.     verbs:
  121.       - update
  123. ---
  124. apiVersion: rbac.authorization.k8s.io/v1beta1
  125. kind: Role
  126. metadata:
  127.   name: nginx-ingress-role
  128.   namespace: ingress-nginx
  129.   labels:
  130.     app.kubernetes.io/name: ingress-nginx
  131.     app.kubernetes.io/part-of: ingress-nginx
  132. rules:
  133.   - apiGroups:
  134.       - ""
  135.     resources:
  136.       - configmaps
  137.       - pods
  138.       - secrets
  139.       - namespaces
  140.     verbs:
  141.       - get
  142.   - apiGroups:
  143.       - ""
  144.     resources:
  145.       - configmaps
  146.     resourceNames:
  147.       # Defaults to "<election-id>-<ingress-class>"
  148.       # Here: "<ingress-controller-leader>-<nginx>"
  149.       # This has to be adapted if you change either parameter
  150.       # when launching the nginx-ingress-controller.
  151.       - "ingress-controller-leader-nginx"
  152.     verbs:
  153.       - get
  154.       - update
  155.   - apiGroups:
  156.       - ""
  157.     resources:
  158.       - configmaps
  159.     verbs:
  160.       - create
  161.   - apiGroups:
  162.       - ""
  163.     resources:
  164.       - endpoints
  165.     verbs:
  166.       - get
  168. ---
  169. apiVersion: rbac.authorization.k8s.io/v1beta1
  170. kind: RoleBinding
  171. metadata:
  172.   name: nginx-ingress-role-nisa-binding
  173.   namespace: ingress-nginx
  174.   labels:
  175.     app.kubernetes.io/name: ingress-nginx
  176.     app.kubernetes.io/part-of: ingress-nginx
  177. roleRef:
  178.   apiGroup: rbac.authorization.k8s.io
  179.   kind: Role
  180.   name: nginx-ingress-role
  181. subjects:
  182.   - kind: ServiceAccount
  183.     name: nginx-ingress-serviceaccount
  184.     namespace: ingress-nginx
  186. ---
  187. apiVersion: rbac.authorization.k8s.io/v1beta1
  188. kind: ClusterRoleBinding
  189. metadata:
  190.   name: nginx-ingress-clusterrole-nisa-binding
  191.   labels:
  192.     app.kubernetes.io/name: ingress-nginx
  193.     app.kubernetes.io/part-of: ingress-nginx
  194. roleRef:
  195.   apiGroup: rbac.authorization.k8s.io
  196.   kind: ClusterRole
  197.   name: nginx-ingress-clusterrole
  198. subjects:
  199.   - kind: ServiceAccount
  200.     name: nginx-ingress-serviceaccount
  201.     namespace: ingress-nginx
  203. ---
  205. apiVersion: apps/v1
  206. kind: Deployment
  207. metadata:
  208.   name: nginx-ingress-controller
  209.   namespace: ingress-nginx
  210.   labels:
  211.     app.kubernetes.io/name: ingress-nginx
  212.     app.kubernetes.io/part-of: ingress-nginx
  213. spec:
  214.   replicas: 1
  215.   selector:
  216.     matchLabels:
  217.       app.kubernetes.io/name: ingress-nginx
  218.       app.kubernetes.io/part-of: ingress-nginx
  219.   template:
  220.     metadata:
  221.       labels:
  222.         app.kubernetes.io/name: ingress-nginx
  223.         app.kubernetes.io/part-of: ingress-nginx
  224.       annotations:
  225.         prometheus.io/port: "10254"
  226.         prometheus.io/scrape: "true"
  227.     spec:
  228.       hostNetwork: true
  229. #使用宿主机网络，意思是调度到Node节点使用本地的网络，80,443端口都会在本地体现。也就是说，本地的80、443端口不能被占用，否则启动不起来
  230.       serviceAccountName: nginx-ingress-serviceaccount
  231.       containers:
  232.         - name: nginx-ingress-controller
  233.           image: lizhenliang/nginx-ingress-controller:0.20.0
  234.           args:
  235.             - /nginx-ingress-controller
  236.             - --configmap=$(POD_NAMESPACE)/nginx-configuration
  237.             - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
  238.             - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
  239.             - --publish-service=$(POD_NAMESPACE)/ingress-nginx
  240.             - --annotations-prefix=nginx.ingress.kubernetes.io
  241.           securityContext:
  242.             allowPrivilegeEscalation: true
  243.             capabilities:
  244.               drop:
  245.                 - ALL
  246.               add:
  247.                 - NET_BIND_SERVICE
  248.             # www-data -> 33
  249.             runAsUser: 33
  250.           env:
  251.             - name: POD_NAME
  252.               valueFrom:
  253.                 fieldRef:
  254.                   fieldPath: metadata.name
  255.             - name: POD_NAMESPACE
  256.               valueFrom:
  257.                 fieldRef:
  258.                   fieldPath: metadata.namespace
  259.             - name: KUBERNETES_MASTER
  260.               value: http://192.168.17.130:8080
  261. #需要加入这一行，否则会报错
  262.           ports:
  263.             - name: http
  264.               containerPort: 80
  265.             - name: https
  266.               containerPort: 443
  267.           livenessProbe:
  268.             failureThreshold: 3
  269.             httpGet:
  270.               path: /healthz
  271.               port: 10254
  272.               scheme: HTTP
  273.             initialDelaySeconds: 10
  274.             periodSeconds: 10
  275.             successThreshold: 1
  276.             timeoutSeconds: 10
  277.           readinessProbe:
  278.             failureThreshold: 3
  279.             httpGet:
  280.               path: /healthz
  281.               port: 10254
  282.               scheme: HTTP
  283.             periodSeconds: 10
  284.             successThreshold: 1
  285.             timeoutSeconds: 10
  287. ---
  288. #部署Ingress Controller控制器
  289. [root@ZhangSiming ingress]# kubectl create -f mandatory.yaml 
  291. namespace/ingress-nginx created
  292. configmap/nginx-configuration created
  293. configmap/tcp-services created
  294. configmap/udp-services created
  295. serviceaccount/nginx-ingress-serviceaccount created
  296. clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
  297. role.rbac.authorization.k8s.io/nginx-ingress-role created
  298. rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
  299. clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
  300. deployment.apps/nginx-ingress-controller created
  301. [root@ZhangSiming ingress]# kubectl get pods -n ingress-nginx
  302. NAME                                        READY   STATUS    RESTARTS   AGE
  303. nginx-ingress-controller-7d8dc989d6-qzcll   1/1     Running   0          4m52s
  304. #启动成功

5.2Ingress规则编写

我们已经部署好了Ingress Controller控制器，也就是说可以提供Ingress对外访问服务了，但是具体怎么提供对外服务，我们需要自己定义Ingress规则。

  1. [root@ZhangSiming ingress]# kubectl get svc
  2. NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
  3. kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP        21d
  4. my-service   NodePort    10.0.0.21    <none>        80:30001/TCP   9s
  5. [root@ZhangSiming ingress]# vim ingress.yaml
  6. [root@ZhangSiming ingress]# cat ingress.yaml
  7. apiVersion: extensions/v1beta1
  8. kind: Ingress
  9. metadata:
  10.   name: example-ingress
  11. spec:
  12.   rules:
  13.   - host: example.foo.com
  14. #域名
  15.     http:
  16.       paths:
  17.       - backend:
  18.           serviceName: my-service
  19. #绑定NodePort的Service
  20.           servicePort: 80
  21. #端口写K8S集群内部端口
  22. [root@ZhangSiming ingress]# kubectl create -f ingress.yaml 
  23. ingress.extensions/example-ingress created
  24. [root@ZhangSiming ingress]# kubectl get ingress
  25. NAME              HOSTS             ADDRESS   PORTS   AGE
  26. example-ingress   example.foo.com             80      12s

image_1d6fuu2gd18q74s3o9m13khibq3a.png-66.8kB

image_1d6futnre154gsruf83a8g5d92t.png-46.9kB

5.4Ingress（HTTP与HTTPS）

添加新批注
在作者公开此批注前，只有你和作者可见。
回复批注