关注 k8s 的小伙伴们都知道在 k8s1.24版 Dockershim 代码正式的从 k8s 移除了。这也就意味着k8s 与 docker 的对接再也不像以前那么丝滑了。这篇文档记录一下我在使用 docker 作为 CRI 插件时部署 k8s 的步骤。我依然借助kubeadm来部署 k8s,关于 kubeadm 的使用可以参考我前面的文档,和这篇文档重复的步骤我就不在这儿写了,我们只关注和docker 相关的部分。
配置 dnf 源
OpenEuler 的包管理工具使用的是 rpm,可以配置 centos 的 docker yum 源,这里我们使用阿里云的 yum 源。
创建文件 /etc/yum.repos.d/docker-ce.repo 并填入以下内容:
[docker]name=Docker CE Stablebaseurl=https://download.docker.com/linux/centos/7/x86_64/stableenabled=1gpgcheck=0[extra]name=Extrabaseurl=http://mirrors.aliyun.com/centos/7/extras/x86_64/enabled=1gpgcheck=0
配置好 yum 源,执行 dnf 命令安装:
dnf makecachednf install docker-ce -y
注意:这里有两个源组成,在 centos 下使用只需要配置第一个源就可以了,但是因为我使用的系统是openEuler,只用第一个源的话会缺很多依赖包。安装时会报下面这些错误:
Last metadata expiration check: 0:00:22 ago on Wed 31 Aug 2022 04:37:29 PM CST.Error:Problem: package docker-ce-3:20.10.17-3.el7.x86_64 requires docker-ce-rootless-extras, but none of the providers can be installed- cannot install the best candidate for the job- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.0-3.el7.x86_64- nothing provides slirp.NETns >= 0.4 needed by docker-ce-rootless-extras-20.10.0-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.1-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.1-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.10-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.10-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.11-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.11-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.12-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.12-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.13-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.13-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.14-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.14-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.15-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.15-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.16-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.16-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.17-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.17-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.2-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.2-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.3-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.3-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.4-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.4-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.5-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.5-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.6-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.6-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.7-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.7-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.8-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.8-3.el7.x86_64- nothing provides fuse-overlayfs >= 0.7 needed by docker-ce-rootless-extras-20.10.9-3.el7.x86_64- nothing provides slirp4netns >= 0.4 needed by docker-ce-rootless-extras-20.10.9-3.el7.x86_64(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
配置 docker 加速cat >/etc/docker/daemon.json << EOF"exec-opts": ["native.cgroupdriver=systemd"],"registry-mirrors" : ["https://8xpk5wnt.mirror.aliyuncs.com"EOF
systemctl restart docker
下载并配置 cri-dockered
下载地址:https://github.com/Mirantis/cri-dockerd/releases
这是比 1.24 之前多的关键步骤,k8s 把 dockershim 的代码移除后,需要额外安装 cri-dockered 充当之前 dockershim 的角色。
wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.2.5/cri-dockerd-0.2.5.amd64.tgz
注意:v0.2.5 是我写这篇文档是的最新的 cri-dockerd,大家在安装的时候要注意调整这个版本号到最新。
下载完成后解压并 copy 的 /usr/bin 目录
tar -xf cri-dockerd-0.2.5.amd64.tgzcp cri-dockerd/cri-dockerd /usr/bin/chmod +x /usr/bin/cri-dockerd
配置 service 文件[Unit]Description=CRI Interface for Docker Application Container EngineDocumentation=https://docs.mirantis.comAfter=network-online.target firewalld.service docker.serviceWants=network-online.targetRequires=cri-docker.socket[Service]Type=notifyExecStart=/usr/bin/cri-dockerd --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.7ExecReload=/bin/kill -s HUP $MAINPIDTimeoutSec=0RestartSec=2Restart=alwaysStartLimitBurst=3StartLimitInterval=60sLimitNOFILE=infinityLimitNPROC=infinityLimitCORE=infinityTasksMax=infinityDelegate=yesKillMode=process[Install]WantedBy=multi-user.target
注意:这里的参数 --pod-infra-container-image,这个值要根据实际情况调整,如果你的安装环境不能访问互联网,那你就需要配置为内部镜像仓库。
创建 socket 文件
创建一个 socker 文件以便 kubelet 与 cri-dockerd 通信
cat <<"EOF" > /usr/lib/systemd/system/cri-docker.Socket[Unit]Description=CRI Docker Socket for the APIPartOf=cri-docker.service[Socket]ListenStream=%t/cri-dockerd.sockSocketMode=0660SocketUser=rootSocketGroup=docker[Install]WantedBy=sockets.target
启动 cri-docker
systemctl daemon-reloadsystemctl start cri-docker
使用 kubeadm 部署 k8s
这部分可以参考我之前的文档 的部分章节初始化环境和安装 flannel 网络插件,这里就不过多赘述了。
总结:在 k8是1.24 版以后我们需要额外安装 cri-dockerd 来弥补 k8s 中原来 dockershim 完成的功能。
k8s 为什么移除 dockershim
在之前的文章多维度聊一聊 k8s 和 openstack,我吐槽过 k8s 对 docker 的渣男行为。
当然这只是纯粹的吐槽哈,那么我们接下来就执行探讨一下 docker 移除 dockershim 的原因。
我们先来看一看 kubelet 使用 docker 作为 cri 插件创建容器的步骤:
我们可以看到第 2 步和第 3 步只是起到了翻译的工作,没有做任何实质操作,而且还多了一个 Docker Daemon 服务在运行。把第 2 步和第 3 步去掉,kubelet 之间和 containerd 通信是 k8s 一直想做的事,只不过之前 containerd 还不是很成熟,使用的人也不少很多在加上 docker 本身的知名度所以 k8s 一直保留着 dockershim,k8s 做出遗弃 dockershim 的决定也是经过深思熟虑的,同时也表明了 containerd已经足够成熟了。