Docker Swarm 集群创建详解

167次阅读

共计 12791 个字符，预计需要花费 32 分钟才能阅读完成。

Docker swarm 是 docker 官方的三大项目之一，提供 docker 容器集群服务，是 docker 官方对容器云生态支持的核心方案，Docker Swarm 项目开始于 2014 年，是 Docker 公司推出的第一个容器集群项目。使用 docker swarm 可以将多个 docker 主机封装为一个大型的虚拟 docker 主机，swarm 集群提供给用户管理集群内所有容器的操作接口与使用一台 docker 主机几乎相同，从而可以快速打造一套容器云服务。

Docker Swarmkit 项目开始于 2016 年，是 Docker 公司推出的第二个容器集群项目，于 Docker1.12 版本正式发布。虽然也叫 Swarm，但是与第一个项目完全不同。该项目直接在 Docker Engine 上内嵌了集群管理功能，并新增了集群管理的用户接口。
两个容器集群项目可能实现了相同的功能，但其上层接口还是有很大的不同，Docker 公司推荐用户使用更适合自己的项目，如果都没有使用过，推荐使用后者。另外，Docker Swarm 项目并没有被 Docker 公司列为不推荐的项目，仍然会继续支持新的 Docker Engine 的功能。

swarm v1 是典型的 mater-slave 结构，需要通过发现服务来选举出 manager，而 manager 是中心管理节点，而各个节点通过运行 agent 接受 manager 的统一管理

swarm v2 中，swarm 集群会自动通过 Raft 协议分布式选举出 manager 节点，因此无需配置额外的发现服务，从而避免了单点瓶颈；且 swarm v2 内置了 DNS 负载均衡和对外部负载均衡机制的支持

本实验基于 docker swarm v1 版本，实验环境如下：
docker-1 192.168.1.193
docker-2 192.168.1.194
docker-3 192.168.1.195

在各个 docker 主机上安装 swarm，启动 docker daemon
root@docker-1:~# docker pull swarm
root@docker-1:~# docker run –rm swarm -v
swarm version 1.2.8 (48d86b1)
root@docker-1:~# service docker stop
docker stop/waiting
root@docker-1:~# dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock
root@docker-1:~# netstat -tunlp|grep docker
tcp6 0 0 :::2375 :::* LISTEN 1698/dockerd

root@docker-2:~# docker pull swarm
root@docker-2:~# service docker stop
docker stop/waiting
root@docker-2:~# dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

root@docker-3:~# docker pull swarm
root@docker-3:~# service docker stop
docker stop/waiting
root@docker-3:~# dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

在任意一个节点上使用 docker swarm 命令来在 docker hub 上进行注册（采用 docker hub 的服务发现后端，但必须要求所有 docker 主机可以访问外网），且会产生一个 token，用来标识和管理集群
root@docker-1:~# docker run –rm swarm create
1081dc3745b8dfd45ff863c13d74d96c
在所有要加入 swarm 集群的 docker 主机上执行 swarm join 命令，将各个节点加入 swarm 集群
root@docker-1:~# docker run -d swarm join –addr=192.168.1.193:2375 token://1081dc3745b8dfd45ff863c13d74d96c
fbc0c47268e1e631c4839b41107cac08bd33c9b27380283768efda84ab3ae373
root@docker-2:~# docker run -d swarm join –addr=192.168.1.194:2375 token://1081dc3745b8dfd45ff863c13d74d96c
b88ab2e5eb4e787ad56df06fd6f7e879aacbd35b36743d2943002b882cd8865f

root@docker-3:~# docker run -d swarm join –addr=192.168.1.195:2375 token://1081dc3745b8dfd45ff863c13d74d96c
1b62af7b1ecf7d428f7dd25df71a0523742cecdb1341eb76560225a77336b12f
因为没有配置 manager 节点，所以此命令无法使用
root@docker-1:~# docker node ls
Error response from daemon: This node is not a swarm manager. Use “docker swarm init” or “docker swarm join” to connect this node to swarm and try again.

查看各个节点的容器情况，可以与后续实验做对比
root@docker-1:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fbc0c47268e1 swarm “/swarm join –add…” 6 minutes ago Up 6 minutes 2375/tcp competent_benz
root@docker-1:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fbc0c47268e1 swarm “/swarm join –add…” 6 minutes ago Up 6 minutes 2375/tcp competent_benz

root@docker-2:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b88ab2e5eb4e swarm “/swarm join –add…” 4 minutes ago Up 4 minutes 2375/tcp romantic_poincare
root@docker-2:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b88ab2e5eb4e swarm “/swarm join –add…” 4 minutes ago Up 4 minutes 2375/tcp romantic_poincare

root@docker-3:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b88ab2e5eb4e swarm “/swarm join –add…” 4 minutes ago Up 4 minutes 2375/tcp romantic_poincare
root@docker-3:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b88ab2e5eb4e swarm “/swarm join –add…” 4 minutes ago Up 4 minutes 2375/tcp romantic_poincare

用 swarm manage 命令将 docker- 1 节点配置成为管理节点该命令将启用 manager 服务, 且默认监听到 2375 端口，所有集群的管理都可以通过该服务的接口进行操作，但是 manager 服务和 docker 服务
的监听端口相同，而这样做是为了兼容其他基于 docker 的服务，可以无缝的切换到 swarm 平台；但有与本次实验采用的是 docker 容器的方式启用 manager 服务，本地的 2375 端口已被 docker daemon 占用，
所以将 manager 服务的端口映射到一个空闲的端口 12375
root@docker-1:~# docker run -d -p 12375:2375 swarm manage token://1081dc3745b8dfd45ff863c13d74d96c
ad81e842302de642ba7bf5514b12ce6ea46070abc68c6e15a4eefc0b8e5dfc57
root@docker-1:~# netstat -tunlp |grep docker
tcp6 0 0 :::12375 :::* LISTEN 2191/docker-proxy
tcp6 0 0 :::2375 :::* LISTEN 1698/dockerd
产看 docker- 1 节点运行的容器和所有容器
root@docker-1:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ad81e842302d swarm “/swarm manage tok…” 4 seconds ago Up 3 seconds 0.0.0.0:12375->2375/tcp hungry_benz
fbc0c47268e1 swarm “/swarm join –add…” 19 minutes ago Up 19 minutes 2375/tcp competent_benz
root@docker-1:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ad81e842302d swarm “/swarm manage tok…” 13 seconds ago Up 12 seconds 0.0.0.0:12375->2375/tcp hungry_benz
fbc0c47268e1 swarm “/swarm join –add…” 19 minutes ago Up 19 minutes 2375/tcp competent_benz

可以再任意节点执行 swarm list 命令查看集群的节点列表
root@docker-1:~# docker run –rm swarm list token://1081dc3745b8dfd45ff863c13d74d96c
192.168.1.193:2375
192.168.1.195:2375
192.168.1.194:2375
root@docker-2:~# docker run –rm swarm list token://1081dc3745b8dfd45ff863c13d74d96c
192.168.1.193:2375
192.168.1.195:2375
192.168.1.194:2375

可以再任意节点可以创建集群容器，如 docker-3
root@docker-3:~# docker -H 192.168.1.193:12375 run -d CentOS ping www.baidu.com
41ca94b3da41767ecd87de7d37a3611141dc8b36aad52211b75374b153628312
查看集群所有节点运行的容器
root@docker-3:~# docker -H 192.168.1.193:12375 ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
41ca94b3da41 centos “ping www.baidu.com” 3 minutes ago Up 3 minutes docker-2/vigilant_bose
查看集群所有节点的容器
root@docker-3:~# docker -H 192.168.1.193:12375 ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
41ca94b3da41 centos “ping www.baidu.com” 3 minutes ago Up 3 minutes docker-2/vigilant_bose
ad81e842302d swarm “/swarm manage tok…” 13 minutes ago Up 13 minutes 192.168.1.193:12375->2375/tcp docker-1/hungry_benz
1b62af7b1ecf swarm “/swarm join –add…” 30 minutes ago Up 30 minutes 2375/tcp docker-3/ecstatic_elion
b88ab2e5eb4e swarm “/swarm join –add…” 30 minutes ago Up 30 minutes 2375/tcp docker-2/romantic_poincare
fbc0c47268e1 swarm “/swarm join –add…” 32 minutes ago Up 32 minutes 2375/tcp docker-1/competent_benz
查看本节点 docker-3 正在运行的容器
root@docker-3:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1b62af7b1ecf swarm “/swarm join –add…” 31 minutes ago Up 31 minutes 2375/tcp ecstatic_elion
查看本节点 docker-3 的所有容器
root@docker-3:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1b62af7b1ecf swarm “/swarm join –add…” 31 minutes ago Up 31 minutes 2375/tcp ecstatic_elion
root@docker-3:~#

在 docker- 1 主机上分别执行节点和集群的 docker info 看看，可以对比一下

root@docker-1:~# docker info
Containers: 2
Running: 2
Paused: 0
Stopped: 0
Images: 1
Server Version: 17.05.0-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 7
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
apparmor
Kernel Version: 4.4.0-31-generic
Operating System: Ubuntu 14.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 726.7MiB
Name: docker-1
ID: A3OC:Z223:S7IY:6KF6:U2TE:N26E:CEMT:2VEH:5AR6:LVY5:U2KI:52P5
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

root@docker-1:~# docker -H 192.168.1.193:12375 info
Containers: 5
Running: 5
Paused: 0
Stopped: 0
Images: 5
Server Version: swarm/1.2.8
Role: primary
Strategy: spread
Filters: health, port, containerslots, dependency, affinity, constraint, whitelist
Nodes: 3
docker-1: 192.168.1.193:2375
└ ID: A3OC:Z223:S7IY:6KF6:U2TE:N26E:CEMT:2VEH:5AR6:LVY5:U2KI:52P5|192.168.1.193:2375
└ Status: Healthy
└ Containers: 2 (2 Running, 0 Paused, 0 Stopped)
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 763 MiB
└ Labels: kernelversion=4.4.0-31-generic, operatingsystem=Ubuntu 14.04.5 LTS, ostype=linux, storagedriver=aufs
└ UpdatedAt: 2017-10-15T13:18:50Z
└ ServerVersion: 17.05.0-ce
docker-2: 192.168.1.194:2375
└ ID: LTSZ:NUCW:U7PJ:E2MB:UD4K:QNS4:EOYG:YC5B:XEYU:BNM5:MZGE:X4R4|192.168.1.194:2375
└ Status: Healthy
└ Containers: 2 (2 Running, 0 Paused, 0 Stopped)
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 776.6 MiB
└ Labels: kernelversion=4.4.0-31-generic, operatingsystem=Ubuntu 14.04.5 LTS, ostype=linux, storagedriver=aufs
└ UpdatedAt: 2017-10-15T13:19:04Z
└ ServerVersion: 17.05.0-ce
docker-3: 192.168.1.195:2375
└ ID: A42N:WYIC:KJRG:SPED:IJIS:IYQP:23T4:EWDP:6CT7:IFMB:ZIDF:37WN|192.168.1.195:2375
└ Status: Healthy
└ Containers: 1 (1 Running, 0 Paused, 0 Stopped)
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 513.1 MiB
└ Labels: kernelversion=4.4.0-31-generic, operatingsystem=Ubuntu 14.04.5 LTS, ostype=linux, storagedriver=aufs
└ UpdatedAt: 2017-10-15T13:19:03Z
└ ServerVersion: 17.05.0-ce
Plugins:
Volume:
Network:
Swarm:
NodeID:
Is Manager: false
Node Address:
Kernel Version: 4.4.0-31-generic
Operating System: linux
Architecture: amd64
CPUs: 3
Total Memory: 2.005GiB
Name: ad81e842302d
Docker Root Dir:
Debug Mode (client): false
Debug Mode (server): false
Experimental: false
Live Restore Enabled: false
可以发现 docker- 1 上运行有创建 manager 服务和 swarm join 时的两个容器，docker- 2 上运行有 swarm join 和 ping www.baidu.com
(这个容器是在 docker- 3 上通过 manager 服务创建的，但却运行在 docker- 2 上，可见是 swarm 集群调度策略分配的）的两个容器，docker- 3 上只有 swarm join

创建一个 nginx 服务
root@docker-1:~# docker -H 192.168.1.193:12375 run -d –name my-web –publish 8080:80 nginx
2a888979b129767ff73562587e5cf6672aac13c47d69ea4673887d87f6be552a
通过下面命令可以看到 nginx 运行在 docker- 3 上（swarm 调度策略分配的）
root@docker-1:~# docker -H 192.168.1.193:12375 ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2a888979b129 nginx “nginx -g ‘daemon …” 15 seconds ago Up 14 seconds 192.168.1.195:8080->80/tcp docker-3/my-web
41ca94b3da41 centos “ping www.baidu.com” 2 hours ago Up 2 hours docker-2/vigilant_bose

root@docker-3:~# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 910/sshd
tcp6 0 0 :::22 :::* LISTEN 910/sshd
tcp6 0 0 :::2375 :::* LISTEN 1549/dockerd
tcp6 0 0 :::8080 :::* LISTEN 3761/docker-proxy

root@docker-1:~# curl http://192.168.1.195:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href=”http://nginx.org/”>nginx.org</a>.<br/>
Commercial support is available at
<a href=”http://nginx.com/”>nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

进入 nginx 容器进行服务修改
root@docker-1:~# docker -H 192.168.1.193:12375 exec -it 2a888979b129 /bin/bash
root@2a888979b129:/# apt-get update &&apt-get install -y vim net-tools
root@2a888979b129:/# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1/nginx: master pro

修改 nginx 服务的配置文件
root@2a888979b129:/# vim /etc/nginx/conf.d/default.conf
location / {
root /var/www/html/;
index index.html index.htm;
}

root@2a888979b129:/# mkdir /var/www/html/ -p
root@2a888979b129:/# vim /var/www/html/index.html
<h1> Hello,Docker ^_^</h1>

root@2a888979b129:/# service nginx restart
[….] Restarting nginx: nginx
重启 nginx 后，nginx 容器会终止，需要重新启动
root@docker-1:~# docker -H 192.168.1.193:12375 start 2a888979b129
2a888979b129

修改后，访问成功
root@docker-2:~# curl http://192.168.1.195:8080
<h1> Hello,Docker ^_^</h1>

Docker Swarm 集群创建详解