Centos8 部署 kubernetes

开启网络设置

1
2
3
4
5
6
7
cd /etc/sysconfig/network-scripts/
vi ifcfg-ens18
把 ONBOOT=no 设置为 yes

## 更新库
dnf update
dnf upgrade

添加阿里源

1
2
rm -rfv /etc/yum.repos.d/*
curl -o /etc/yum.repos.d/CentOS-Base.repo <http://mirrors.aliyun.com/repo/Centos-8.repo>

配置主机名

1
2
3
4
5
6
[root@master ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.30 k8s-m1
10.0.0.31 k8s-s1
10.0.0.32 k8s-s2

设置永久主机名称

1
2
3
sudo hostnamectl set-hostname k8s-m1
sudo hostnamectl set-hostname k8s-s1
sudo hostnamectl set-hostname k8s-s2

关闭swap,注释swap分区

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[root@master ~]# swapoff -a
删除 swap 挂载,这样系统下次启动不会再挂载 swap
[root@master ~]# cat /etc/fstab
# 注释 swap 行

#
# /etc/fstab
# Created by anaconda on Tue Mar 31 22:44:34 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/cl-root / xfs defaults 0 0
UUID=5fecb240-379b-4331-ba04-f41338e81a6e /boot ext4 defaults 1 2
/dev/mapper/cl-home /home xfs defaults 0 0
#/dev/mapper/cl-swap swap swap defaults 0 0

配置内核参数,将桥接的IPv4流量传递到iptables的链

1
2
3
4
5
6
[root@master ~]# cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

安装常用包

1
2
yum install vim bash-completion net-tools gcc -y

通用安装Docker

1
2
3
4
5
6
7
8
9
10
11
12
13
14
## 安装docker所需的依赖包
sudo dnf install -y yum-utils \\
device-mapper-persistent-data \\
lvm2

## 安装Docker稳定储存库
sudo yum-config-manager \\
--add-repo \\
<https://download.docker.com/linux/centos/docker-ce.repo>

## 安装Docker
dnf install <https://download.docker.com/linux/centos/8/x86_64/stable/Packages/docker-ce-19.03.13-3.el8.x86_64.rpm>
dnf install docker-ce docker-ce-cli

安装kubectl、kubelet、kubeadm

1
2
3
4
5
6
7
8
9
10
[root@master ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg <https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg>
EOF

1
2
3
[root@master ~]# yum install kubectl kubelet kubeadm
[root@master ~]# systemctl enable kubelet

初始化k8s集群

POD的网段为: 10.11.0.0/16, api server地址就是master本机IP
这一步很关键,由于kubeadm 默认从官网k8s.grc.io下载所需镜像,国内无法访问,因此需要通过–image-repository指定阿里云镜像仓库地址

1
2
3
4
5
 kubeadm init --kubernetes-version=v1.20.1  \\
--apiserver-advertise-address=10.0.0.30 \\
--image-repository registry.aliyuncs.com/google_containers \\
--service-cidr=10.10.0.0/16 --pod-network-cidr=10.11.0.0/16

master 节点输入

1
2
3
4
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

node 节点加入输入

1
2
3
kubeadm join 10.0.0.30:6443 --token kxyqzb.kvpwpv3ho7ucb73s \\
--discovery-token-ca-cert-hash sha256:361860a76516baae22e850e7258a532d2adbe5e661299cd42f35649ba944db39

执行下面命令,使kubectl可以自动补充

1
2
source <(kubectl completion bash)

查看节点,pod

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[root@master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
master.paas.com NotReady master 2m29s v1.18.0
[root@master ~]# kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-7ff77c879f-fsj9l 0/1 Pending 0 2m12s
kube-system coredns-7ff77c879f-q5ll2 0/1 Pending 0 2m12s
kube-system etcd-master.paas.com 1/1 Running 0 2m22s
kube-system kube-apiserver-master.paas.com 1/1 Running 0 2m22s
kube-system kube-controller-manager-master.paas.com 1/1 Running 0 2m22s
kube-system kube-proxy-th472 1/1 Running 0 2m12s
kube-system kube-scheduler-master.paas.com 1/1 Running 0 2m22s
[root@master ~]#

安装calico网络

1
2
kubectl apply -f <https://docs.projectcalico.org/manifests/calico.yaml>

查看pod和node

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[root@master ~]# kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-555fc8cc5c-k8rbk 1/1 Running 0 36s
kube-system calico-node-5km27 1/1 Running 0 36s
kube-system coredns-7ff77c879f-fsj9l 1/1 Running 0 5m22s
kube-system coredns-7ff77c879f-q5ll2 1/1 Running 0 5m22s
kube-system etcd-master.paas.com 1/1 Running 0 5m32s
kube-system kube-apiserver-master.paas.com 1/1 Running 0 5m32s
kube-system kube-controller-manager-master.paas.com 1/1 Running 0 5m32s
kube-system kube-proxy-th472 1/1 Running 0 5m22s
kube-system kube-scheduler-master.paas.com 1/1 Running 0 5m32s
[root@master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
master.paas.com Ready master 5m47s v1.18.0
[root@master ~]#

安装kubernetes-dashboard

官方部署dashboard的服务没使用nodeport,将yaml文件下载到本地,在service里添加nodeport

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
[root@master ~]# wget  <https://raw.githubusercontent.com/kubernetes/dashboard/v2.1.0/aio/deploy/recommended.yaml>
[root@master ~]# vim recommended.yaml
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
type: NodePort
ports:
- port: 443
targetPort: 8443
nodePort: 30000
selector:
k8s-app: kubernetes-dashboard

[root@master ~]# kubectl create -f recommended.yaml
namespace/kubernetes-dashboard created
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
deployment.apps/dashboard-metrics-scraper created

1
2
3
4
5
6
7
## 官方设置方法
kubectl apply -f <https://raw.githubusercontent.com/kubernetes/dashboard/v2.1.0/aio/deploy/recommended.yaml>
// 删除原来的配置
kubectl delete -f recommended.yaml
// 创建新的配置
kubectl create -f recommended.yaml

查看pod,service

1
2
3
4
5
6
7
8
9
NAME                                        READY   STATUS    RESTARTS   AGE
dashboard-metrics-scraper-dc6947fbf-869kf 1/1 Running 0 37s
kubernetes-dashboard-5d4dc8b976-sdxxt 1/1 Running 0 37s
[root@master ~]# kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.10.58.93 <none> 8000/TCP 44s
kubernetes-dashboard NodePort 10.10.132.66 <none> 443:30000/TCP 44s
[root@master ~]#

使用token进行登录,执行下面命令获取token

1
2
3
4
kubectl describe secrets -n kubernetes-dashboard kubernetes-dashboard-token  | grep token | awk 'NR==3{print $2}'

eyJhbGciOiJSUzI1NiIsImtpZCI6IlJBMmF5WXYxVnVTeEJId2gtVE5ka25LcjZZcmdIWXZZZTdtTEwtUjUybmMifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi10eHQ5ZyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImFmNTVlZWVhLWQwNjMtNDNlNi04Y2I5LWUxY2UxY2U1MTIxNiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.q214u1ao8hVbzgzYv3kVsp40OPZvkVqD5iAmV9y4hGJdXRMTjCxafjiAq7AKdBG19fdJQOLZLqMVba1fX465MyAUbKGE9Z6UlQpU0XGZVNSx67r_QYDu_Pby1b3HzFnu0xW8Rhk6q9dP2al-3PaNaUrwShPg11L-NAP6E78coRDd3tmyZN_GvPPQXVg3nOJbmPlXLMwv88GalhtzpEJIzvICTH2-Fq_ltoMwWRu_LwhmP_w1S7giRMhte1-N7TAXE-ZPnGuBhrdVXcdrxtiPu3-W5233gOr-mdGXdDKaTB-O46VVF9KlYCf5dEjJ64NqjlgyvixZVF-WzwuytgTotg

Q&A

无法找见命名空间问题

1
2
3
4
5
## 删除旧的权限
kubectl delete clusterrolebinding serviceaccount-cluster-admin
## 添加权限
kubectl create clusterrolebinding serviceaccount-cluster-admin --clusterrole=cluster-admin --user=system:serviceaccount:kubernetes-dashboard:kubernetes-dashboard

查询端口占用情况

1
netstat -ltnp | grep -w ":10250"

更新Node 节点

1
2
3
systemctl stop kubelet
rm -f /etc/kubernetes/kubelet.conf
rm -f /etc/kubernetes/pki/ca.crt

Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of “crypto/rsa: verification error” while trying to verify candidate authority certi

1
2
3
4
5
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
sudo systemctl status kubelet
sudo systemctl restart kubelet

Kubernetes报错:Error from server: error dialing backend: dial tcp x.x.x.x:10250: connect: no route to host

1
2
3
4
5
6
7
8
9
10
11
12
13
14
执行kubectl命令时报错
#查看日志
kubectl logs xxxxx
#进入pod容器
kubectl exec -it xxxx

检查Pod所在节点的防火墙是否开启

#查看Pod所在的Node
kubectl get pods -o wide
#在Node节点查看防火墙状态
systemctl status firewalld
#如果出现绿色running状态,可考虑关闭防火墙,或通过iptables放行入方向10250端口,这里仅关闭防火墙
systemctl disable --now firewalld

其他资料