大橙子网站建设,新征程启航

为企业提供网站建设、域名注册、服务器等服务

KubernetesClusterHA如何配置

本篇内容介绍了“Kubernetes Cluster HA如何配置”的有关知识,在实际案例的操作过程中,不少人都会遇到这样的困境,接下来就让小编带领大家学习一下如何处理这些情况吧!希望大家仔细阅读,能够学有所成!

创新互联建站坚持“要么做到,要么别承诺”的工作理念,服务领域包括:成都网站建设、成都网站制作、企业官网、英文网站、手机端网站、网站推广等服务,满足客户于互联网时代的东川网站设计、移动媒体设计的需求,帮助企业找到有效的互联网解决方案。努力成为您成熟可靠的网络建设合作伙伴!

Kubernetes Master HA架构图

Kubernetes Cluster HA如何配置

配置与说明

  1. 所有组件可以通过kubelet static pod的方式启动和管理,由kubelet static pod机制保证宿主机上各个组件的高可用, 注意kubelet要添加配置--allow-privileged=true;

  2. 管理static pod的kubelet的高可用通过systemd来负责;

  3. 当然,你也可以直接通过进程来部署这些组件,systemd来直接管理这些进程;(我们选择的是这种方式,降低复杂度。)

  4. 上图中,etcd和Master部署在一起,三个Master节点分别部署了三个etcd,这三个etcd组成一个集群;(当然,如果条件允许,建议将etcd集群和Master节点分开部署。)

  5. 每个Master中的apiserver、controller-manager、scheduler都使用hostNetwork, controller-manager和scheduler通过localhost连接到本节点的apiserver,而不会和其他两个Master节点的apiserver连接;

  6. 外部的rest-client、kubectl、kubelet、kube-proxy等都通过TLS证书,在LB节点做TLS Termination,LB出来就是http请求发到经过LB策略(RR)到对应的apiserver instance;

  7. apiserver到kubelet server和kube-proxy server的访问也类似,Https到LB这里做TLS Termination,然后http请求出来到对应node的kubelet/kube-proxy server;

  8. apiserver的HA通过经典的haproxy + keepalived来保证,集群对外暴露VIP;

  9. controller-manager和scheduler的HA通过自身提供的leader选举功能(--leader-elect=true),使得3个controller-manager和scheduler都分别只有一个是leader,leader处于正常工作状态,当leader失败,会重新选举新leader来顶替继续工作;

  10. 因此,该HA方案中,通过haproxy+keepalived来做apiserver的LB和HA,controller-manager和scheduler通过自身的leader选举来达到HA,etcd通过raft协议保证etcd cluster数据的一致性,达到HA;

  11. keepalived的配置可参考如下:

    vrrp_script check_script {
     script  "/etc/keepalived/check_haproxy.py  http://caicloud:caicloud@127.0.0.1/haproxy?stats"
     interval 5 # check every 5 seconds
     weight 5
     fall 2 # require 2 fail for KO
     rise 1 # require 1 successes for OK
    }
    
    vrrp_instance VI_01 {
    	state MASTER (BACKUP)
    	interface eth2
    	track_interface {
    		eth2
    	}
    
    
    	vrrp_garp_master_repeat 5
    	vrrp_garp_master_refresh 10
    
    	virtual_router_id 51
    	priority 100 (97)
    
    	advert_int 1
    
    	authentication {
    		auth_type PASS
    		auth_pass username
    	}
    
    	virtual_ipaddress {
    		192.168.205.254 dev eth2 label eth2:vip
    	}
    
    	track_script {
    		check_script
    	}
    
    	notify "etc/keepalived/notify_state.sh"
    }

  12. haproxy的配置可参考如下:

    global
    	log 127.0.0.1 local0
    	maxconn 32768
    	pidfile /run/haproxy.pid
    	# turn on stats unix socket
    	stats socket /run/haproxy.stats
    	tune.ssl.default-dh-param  2048
    
    default 
    	log global
    	mode http
    	option httplog
    	option dontlognull
    	retries 3
    	timeout connect 5000ms
    	timeout client 50000ms
    	timeout server 50000ms
    	timeout check 50000ms
    	timeout queue 50000ms
    
    frontend frontend-apisver-http
    	bind *:8080
    	option forwardfor
    
    	acl local_net src 192.168.205.0/24
    
    	http-request allow if local_net
    	http-request deny
    
    	default_backend backend-apiserver-http
    
    frontedn frontend-apiserver-https
    	# haproxy enable ssl
    	bind *:443 ssl crt /etc/kubernetes/master-lb.pem
    	option forwardfor
    	default_backend backend-apiserver-http
    
    backend backend-apiserver-http
    	balance roundrobin
    	option forward-for
    
    	server master-1 192.168.205.11:8080  check
    	server master-2 192.168.205.12:8080  check
    	server master-3 192.168.205.13:8080  check
    
    listen  admin_stats
           bind  0.0.0.0:80
           log  global
           mode  http
           maxconn  10
           stats  enable
           #Hide  HAPRoxy version, a necessity for any public-facing site
           stats  hide-version
           stats  refresh 30s
           stats  show-node
           stats  realm Haproxy\ Statistics
           stats  auth caicloud:caicloud
           stats  uri /haproxy?stats

  13. LB所在的节点,注意确保ip_vs model已加载、ip_forward和ip_nonlocal_bind已开启;

    # make sure ip_vs kernel model is loaded
    modprobe ip_vs
    modprobe ip_vs_rr
    modprobe ip_vs_wrr
    
    # enable ip_forward and ip_nonlocal_bind
    echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
    echo "net.ipv4.ip_nonlocal_bind = 1" >> /etc/sysctl.conf

  14. 如果你通过pod来部署K8S的组件,可参考官方给出的Yaml:

  • apiserver

    apiVersion: v1
    kind: Pod
    metadata:
      name: kube-apiserver
    spec:
      hostNetwork: true
      containers:
      - name: kube-apiserver
        image: gcr.io/google_containers/kube-apiserver:9680e782e08a1a1c94c656190011bd02
        command:
        - /bin/sh
        - -c
        - /usr/local/bin/kube-apiserver --address=127.0.0.1 --etcd-servers=http://127.0.0.1:4001
          --cloud-provider=gce   --admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota
          --service-cluster-ip-range=10.0.0.0/16 --client-ca-file=/srv/kubernetes/ca.crt
          --basic-auth-file=/srv/kubernetes/basic_auth.csv --cluster-name=e2e-test-bburns
          --tls-cert-file=/srv/kubernetes/server.cert --tls-private-key-file=/srv/kubernetes/server.key
          --secure-port=443 --token-auth-file=/srv/kubernetes/known_tokens.csv  --v=2
          --allow-privileged=False 1>>/var/log/kube-apiserver.log 2>&1
        ports:
        - containerPort: 443
          hostPort: 443
          name: https
        - containerPort: 7080
          hostPort: 7080
          name: http
        - containerPort: 8080
          hostPort: 8080
          name: local
        volumeMounts:
        - mountPath: /srv/kubernetes
          name: srvkube
          readOnly: true
        - mountPath: /var/log/kube-apiserver.log
          name: logfile
        - mountPath: /etc/ssl
          name: etcssl
          readOnly: true
        - mountPath: /usr/share/ssl
          name: usrsharessl
          readOnly: true
        - mountPath: /var/ssl
          name: varssl
          readOnly: true
        - mountPath: /usr/ssl
          name: usrssl
          readOnly: true
        - mountPath: /usr/lib/ssl
          name: usrlibssl
          readOnly: true
        - mountPath: /usr/local/openssl
          name: usrlocalopenssl
          readOnly: true
        - mountPath: /etc/openssl
          name: etcopenssl
          readOnly: true
        - mountPath: /etc/pki/tls
          name: etcpkitls
          readOnly: true
      volumes:
      - hostPath:
          path: /srv/kubernetes
        name: srvkube
      - hostPath:
          path: /var/log/kube-apiserver.log
        name: logfile
      - hostPath:
          path: /etc/ssl
        name: etcssl
      - hostPath:
          path: /usr/share/ssl
        name: usrsharessl
      - hostPath:
          path: /var/ssl
        name: varssl
      - hostPath:
          path: /usr/ssl
        name: usrssl
      - hostPath:
          path: /usr/lib/ssl
        name: usrlibssl
      - hostPath:
          path: /usr/local/openssl
        name: usrlocalopenssl
      - hostPath:
          path: /etc/openssl
        name: etcopenssl
      - hostPath:
          path: /etc/pki/tls
        name: etcpkitls

     

    apiVersion: v1
    kind: Pod
    metadata:
      name: kube-controller-manager
    spec:
      containers:
      - command:
        - /bin/sh
        - -c
        - /usr/local/bin/kube-controller-manager --master=127.0.0.1:8080 --cluster-name=e2e-test-bburns
          --cluster-cidr=10.245.0.0/16 --allocate-node-cidrs=true --cloud-provider=gce  --service-account-private-key-file=/srv/kubernetes/server.key
          --v=2 --leader-elect=true 1>>/var/log/kube-controller-manager.log 2>&1
        image: gcr.io/google_containers/kube-controller-manager:fda24638d51a48baa13c35337fcd4793
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10252
          initialDelaySeconds: 15
          timeoutSeconds: 1
        name: kube-controller-manager
        volumeMounts:
        - mountPath: /srv/kubernetes
          name: srvkube
          readOnly: true
        - mountPath: /var/log/kube-controller-manager.log
          name: logfile
        - mountPath: /etc/ssl
          name: etcssl
          readOnly: true
        - mountPath: /usr/share/ssl
          name: usrsharessl
          readOnly: true
        - mountPath: /var/ssl
          name: varssl
          readOnly: true
        - mountPath: /usr/ssl
          name: usrssl
          readOnly: true
        - mountPath: /usr/lib/ssl
          name: usrlibssl
          readOnly: true
        - mountPath: /usr/local/openssl
          name: usrlocalopenssl
          readOnly: true
        - mountPath: /etc/openssl
          name: etcopenssl
          readOnly: true
        - mountPath: /etc/pki/tls
          name: etcpkitls
          readOnly: true
      hostNetwork: true
      volumes:
      - hostPath:
          path: /srv/kubernetes
        name: srvkube
      - hostPath:
          path: /var/log/kube-controller-manager.log
        name: logfile
      - hostPath:
          path: /etc/ssl
        name: etcssl
      - hostPath:
          path: /usr/share/ssl
        name: usrsharessl
      - hostPath:
          path: /var/ssl
        name: varssl
      - hostPath:
          path: /usr/ssl
        name: usrssl
      - hostPath:
          path: /usr/lib/ssl
        name: usrlibssl
      - hostPath:
          path: /usr/local/openssl
        name: usrlocalopenssl
      - hostPath:
          path: /etc/openssl
        name: etcopenssl
      - hostPath:
          path: /etc/pki/tls
        name: etcpkitls

     

    apiVersion: v1
    kind: Pod
    metadata:
      name: kube-scheduler
    spec:
      hostNetwork: true
      containers:
      - name: kube-scheduler
        image: gcr.io/google_containers/kube-scheduler:34d0b8f8b31e27937327961528739bc9
        command:
        - /bin/sh
        - -c
        - /usr/local/bin/kube-scheduler --master=127.0.0.1:8080 --v=2 --leader-elect=true 1>>/var/log/kube-scheduler.log
          2>&1
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10251
          initialDelaySeconds: 15
          timeoutSeconds: 1
        volumeMounts:
        - mountPath: /var/log/kube-scheduler.log
          name: logfile
        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          name: default-token-s8ejd
          readOnly: true
      volumes:
      - hostPath:
          path: /var/log/kube-scheduler.log
        name: logfile

     

    apiVersion: v1
    kind: Pod
    metadata:
      name: etcd-server
    spec:
      hostNetwork: true
      containers:
      - image: gcr.io/google_containers/etcd:2.0.9
        name: etcd-container
        command:
        - /usr/local/bin/etcd
        - --name
        - ${NODE_NAME}
        - --initial-advertise-peer-urls
        - http://${NODE_IP}:2380
        - --listen-peer-urls
        - http://${NODE_IP}:2380
        - --advertise-client-urls
        - http://${NODE_IP}:4001
        - --listen-client-urls
        - http://127.0.0.1:4001
        - --data-dir
        - /var/etcd/data
        - --discovery
        - ${DISCOVERY_TOKEN}
        ports:
        - containerPort: 2380
          hostPort: 2380
          name: serverport
        - containerPort: 4001
          hostPort: 4001
          name: clientport
        volumeMounts:
        - mountPath: /var/etcd
          name: varetcd
        - mountPath: /etc/ssl
          name: etcssl
          readOnly: true
        - mountPath: /usr/share/ssl
          name: usrsharessl
          readOnly: true
        - mountPath: /var/ssl
          name: varssl
          readOnly: true
        - mountPath: /usr/ssl
          name: usrssl
          readOnly: true
        - mountPath: /usr/lib/ssl
          name: usrlibssl
          readOnly: true
        - mountPath: /usr/local/openssl
          name: usrlocalopenssl
          readOnly: true
        - mountPath: /etc/openssl
          name: etcopenssl
          readOnly: true
        - mountPath: /etc/pki/tls
          name: etcpkitls
          readOnly: true
      volumes:
      - hostPath:
          path: /var/etcd/data
        name: varetcd
      - hostPath:
          path: /etc/ssl
        name: etcssl
      - hostPath:
          path: /usr/share/ssl
        name: usrsharessl
      - hostPath:
          path: /var/ssl
        name: varssl
      - hostPath:
          path: /usr/ssl
        name: usrssl
      - hostPath:
          path: /usr/lib/ssl
        name: usrlibssl
      - hostPath:
          path: /usr/local/openssl
        name: usrlocalopenssl
      - hostPath:
          path: /etc/openssl
        name: etcopenssl
      - hostPath:
          path: /etc/pki/tls
        name: etcpkitls

“Kubernetes Cluster HA如何配置”的内容就介绍到这里了,感谢大家的阅读。如果想了解更多行业相关的知识可以关注创新互联网站,小编将为大家输出更多高质量的实用文章!


本文名称:KubernetesClusterHA如何配置
文章分享:http://dzwzjz.com/article/giieop.html
在线咨询
服务热线
服务热线:028-86922220
TOP