Elasticsearch单机多实例环境部署

共计 14652 个字符，预计需要花费 37 分钟才能阅读完成。

Elasticsearch 的功能，主要用在搜索领域，这里，我来研究这个，也是项目需要，为公司开发了一款 CMS 系统，网站上的搜索栏功能，我打算采用 Elasticsearch 来实现。Elasticsearch 的高性能，低延时是最大的吸引力。

系统环境：

CentOS6.8，X86_64。

Elasticsearch 的版本：2.3.5，下载地址来自官网，选择的是 RPM 包。

es 安装完毕后，默认安装的路径是 /usr/share/elasticsearch, 配置文件，默认是在 /etc/elasticsearch 目录。这里要重点说说配置文件的信息：

 1 [root@CloudGame bin]# cd /etc/elasticsearch/
 2 [root@CloudGame elasticsearch]# ll
 3 total 12
 4 -rwxr-x---. 1 root elasticsearch 3189 Jul 27 18:34 elasticsearch.yml
 5 -rwxr-x---. 1 root elasticsearch 2571 Jul 27 18:34 logging.yml
 6 drwxr-x---. 2 root elasticsearch 4096 Jul 27 18:44 scripts
 7 [root@CloudGame elasticsearch]# tree
 8 .
 9 ├── elasticsearch.yml     #es 的系统配置文件
10 ├── logging.yml        #es 的日志配置文件
11 └── scripts
12 
13 1 directory, 2 files

单实例的启动很简单，安装完成后，直接 service elasticsearch start 即可启动，日志在 /var/log/elasticsearch 目录，数据在 /var/lib/elasticsearch 目录。

这里重点放在如何配置单机多实例上面。为了简单起见，多实例就选择启动 2 个实例，更多的实例，和 2 个实例是一样的配置，只是简单的相应改改参数即可。

1. 在 /etc 目录下创建 esconf 目录，然后在 esconf 下面创建 esins1 以及 esins2 目录。将原本 /etc/elasticsearch 目录的内容 copy 到 esins1 以及 esins2 目录下。

 1 [root@CloudGame esconf]# pwd
 2 /etc/esconf
 3 [root@CloudGame esconf]# tree
 4 .
 5 ├── esins1
 6 │   ├── elasticsearch.yml
 7 │   ├── logging.yml
 8 │   └── scripts
 9 ├── esins2
10     ├── elasticsearch.yml
11     ├── logging.yml
12     └── scripts
13 
14 
15 4 directories, 6 files

2. 修改配置文件 elasticsearch.yml, 对 esins1 以及 esins2 下面的这个文件都做修改。主要是修改其节点信息，网络链接信息，以及日志数据路径信息。下面直接上修改好的配置文件内容。

实例 1：esins1 目录下的 elasticsearch.yml 内容

 1 [root@CloudGame esins1]# cat elasticsearch.yml 
 2 # ======================== Elasticsearch Configuration =========================
 3 #
 4 # NOTE: Elasticsearch comes with reasonable defaults for most settings.
 5 #       Before you set out to tweak and tune the configuration, make sure you
 6 #       understand what are you trying to accomplish and the consequences.
 7 #
 8 # The primary way of configuring a node is via this file. This template lists
 9 # the most important settings you may want to configure for a production cluster.
10 #
11 # Please see the documentation for further information on configuration options:
12 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
13 #
14 # ---------------------------------- Cluster -----------------------------------
15 #
16 # Use a descriptive name for your cluster:
17 #
18  cluster.name: tksearch        #es 默认的集群名称是 elasticsearch，注意，集群中是以 name 来区分节点属于那个集群的。19 #
20 # ------------------------------------ Node ------------------------------------
21 #
22 # Use a descriptive name for the node:
23 #
24  node.name: node1             # 节点的名称
25 #
26 # Add custom attributes to the node:
27 #
28 # node.rack: r1
29 #
30  node.master: true            # 是否让这个节点作为默认的 master，若不是，默认会选择集群里面的第一个作为 master，es 有一套选择那个节点作为 master 的机制 
31 # ----------------------------------- Paths ------------------------------------
32 #
33 # Path to directory where to store the data (separate multiple locations by comma):
34 #
35  path.data: /home/AppData/es/esins1/data # 配置节点数据存放的目录
36 #
37 # Path to log files:
38 #
39  path.logs: /home/AppData/es/esins1/logs # 配置节点日志存放的目录
40 #
41 # ----------------------------------- Memory -----------------------------------
42 #
43 # Lock the memory on startup:
44 #
45 # bootstrap.mlockall: true
46 #
47 # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
48 # available on the system and that the owner of the process is allowed to use this limit.
49 #
50 # Elasticsearch performs poorly when the system is swapping the memory.
51 #
52 # ---------------------------------- Network -----------------------------------
53 #
54 # Set the bind address to a specific IP (IPv4 or IPv6):
55 #
56  network.host: 0.0.0.0   # 配置节点绑定的地址，全 0 表示可以绑定任何地址，当然这里，本机可以是 127.0.0.1 回还地址，也可以是 ifconfig 看到的 eth1 的地址。
57 #
58 # Set a custom port for HTTP:
59 #
60  http.port: 9200   # 配置当前节点对外 http 访问的端口号，默认是 9200，不配的话，es 会从 9200-9299 当中找一个未用过的。
61 #
62 # For more information, see the documentation at:
63 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
64 #
65  transport.tcp.port: 9300  #es 集群节点之间的通信端口号。默认 9300.
66 # --------------------------------- Discovery ----------------------------------
67 #
68 # Pass an initial list of hosts to perform discovery when new node is started:
69 # The default list of hosts is ["127.0.0.1", "[::1]"]
70 #
71  discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"]  # 集群多播时发现其他节点的主机列表，真实多机集群环境下，这里会是多个主机的 IP 列表，默认格式“host：port”的数组
72 #
73 # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
74 #
75 # discovery.zen.minimum_master_nodes: 3
76 #
77 # For more information, see the documentation at:
78 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
79 #
80 # ---------------------------------- Gateway -----------------------------------
81 #
82 # Block initial recovery after a full cluster restart until N nodes are started:
83 #
84 # gateway.recover_after_nodes: 3
85 #
86 # For more information, see the documentation at:
87 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
88 #
89 # ---------------------------------- Various -----------------------------------
90 #
91 # Disable starting multiple nodes on a single system:
92 #
93  node.max_local_storage_nodes: 2  # 默认情况下，是不建议单机启动多个 node 的，这里这个参数，就是告知 es 单机上启动了几个实例，这里我们配置 2 个，若是要配置 3 个或者更多实例，修改这个数字即可 
94 #
95 # Require explicit names when deleting indices:
96 #
97 # action.destructive_requires_name: true

实例 2：esins2 目录下的 elasticsearch.yml 内容，配置和 esins1 几乎一样。

 1 [root@CloudGame esins2]# cat elasticsearch.yml 
 2 # ======================== Elasticsearch Configuration =========================
 3 #
 4 # NOTE: Elasticsearch comes with reasonable defaults for most settings.
 5 #       Before you set out to tweak and tune the configuration, make sure you
 6 #       understand what are you trying to accomplish and the consequences.
 7 #
 8 # The primary way of configuring a node is via this file. This template lists
 9 # the most important settings you may want to configure for a production cluster.
10 #
11 # Please see the documentation for further information on configuration options:
12 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
13 #
14 # ---------------------------------- Cluster -----------------------------------
15 #
16 # Use a descriptive name for your cluster:
17 #
18  cluster.name: tksearch
19 #
20 # ------------------------------------ Node ------------------------------------
21 #
22 # Use a descriptive name for the node:
23 #
24  node.name: node2
25 #
26 # Add custom attributes to the node:
27 #
28 # node.rack: r1
29 #
30  node.master: false
31 # ----------------------------------- Paths ------------------------------------
32 #
33 # Path to directory where to store the data (separate multiple locations by comma):
34 #
35  path.data: /home/AppData/es/esins2/data
36 #
37 # Path to log files:
38 #
39  path.logs: /home/AppData/es/esins2/logs
40 #
41 # ----------------------------------- Memory -----------------------------------
42 #
43 # Lock the memory on startup:
44 #
45 # bootstrap.mlockall: true
46 #
47 # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
48 # available on the system and that the owner of the process is allowed to use this limit.
49 #
50 # Elasticsearch performs poorly when the system is swapping the memory.
51 #
52 # ---------------------------------- Network -----------------------------------
53 #
54 # Set the bind address to a specific IP (IPv4 or IPv6):
55 #
56  network.host: 0.0.0.0
57 #
58 # Set a custom port for HTTP:
59 #
60  http.port: 9201
61 #
62 # For more information, see the documentation at:
63 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
64 #
65  transport.tcp.port: 9301
66 # --------------------------------- Discovery ----------------------------------
67 #
68 # Pass an initial list of hosts to perform discovery when new node is started:
69 # The default list of hosts is ["127.0.0.1", "[::1]"]
70 #
71  discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"]
72 #
73 # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
74 #
75 # discovery.zen.minimum_master_nodes: 3
76 #
77 # For more information, see the documentation at:
78 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
79 #
80 # ---------------------------------- Gateway -----------------------------------
81 #
82 # Block initial recovery after a full cluster restart until N nodes are started:
83 #
84 # gateway.recover_after_nodes: 3
85 #
86 # For more information, see the documentation at:
87 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
88 #
89 # ---------------------------------- Various -----------------------------------
90 #
91 # Disable starting multiple nodes on a single system:
92 #
93  node.max_local_storage_nodes: 2
94 #
95 # Require explicit names when deleting indices:
96 #
97 # action.destructive_requires_name: true

View Code

3. 由于配置中指定了配置文件和数据的路径了，所以，要在相应的路径下创建所需的目录。

1 /home/AppData/es/esins1/data
2 /home/AppData/es/esins1/logs
3 /home/AppData/es/esins2/data
4 /home/AppData/es/esins2/logs

4. 启动 es，先启动 es 实例 1. 注意，es 启动时指定的配置文件时，要指定配置文件所在的路径，这个路径包含 elasticsearch.yml 以及 logging.yml

1 [root@CloudGame bin]# ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
2 [root@CloudGame bin]# Exception in thread "main" Java.lang.RuntimeException: don't run elasticsearch as root.
3     at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:93)
4     at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:144)
5     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
6     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
7 Refer to the log for complete error details.

哟，错了啊， 不能用 root 启动 es，见鬼，再来一次吧。

 1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
 2 [water@CloudGame bin]$ Exception in thread "main" SettingsException[Failed to open stream for url [/etc/esconf/esins1/elasticsearch.yml]]; nested: AccessDeniedException[/etc/esconf/esins1/elasticsearch.yml];
 3 Likely root cause: java.nio.file.AccessDeniedException: /etc/esconf/esins1/elasticsearch.yml
 4     at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
 5     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
 6     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
 7     at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
 8     at java.nio.file.Files.newByteChannel(Files.java:317)
 9     at java.nio.file.Files.newByteChannel(Files.java:363)
10     at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:380)
11     at java.nio.file.Files.newInputStream(Files.java:108)
12     at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1067)
13     at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88)
14     at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
15     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
16     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
17 Refer to the log for complete error details.

哟嘿，还是错？？？访问 etc/esconf/esins1/elasticsearch.yml 被阻止了？？？想起来了，这个文件是在 root 用户下创建配置的。修改一下用户组吧： chown -R water:water /etc/esconf/esins1。再来一次。。。

 1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
 2 [water@CloudGame bin]$ Exception in thread "main" SettingsException[Failed to open stream for url [/etc/esconf/esins1/elasticsearch.yml]]; nested: AccessDeniedException[/etc/esconf/esins1/elasticsearch.yml];
 3 Likely root cause: java.nio.file.AccessDeniedException: /etc/esconf/esins1/elasticsearch.yml
 4     at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
 5     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
 6     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
 7     at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
 8     at java.nio.file.Files.newByteChannel(Files.java:317)
 9     at java.nio.file.Files.newByteChannel(Files.java:363)
10     at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:380)
11     at java.nio.file.Files.newInputStream(Files.java:108)
12     at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1067)
13     at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88)
14     at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
15     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
16     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
17 Refer to the log for complete error details.

怎么还是一样的错啊，去看看日志吧，靠，/home/AppData/es/esins1/logs 下面什么也没有啊？？？？？？？倒，这个目录也是在 root 权限下创建的，water 没有写的权限。。。。。。再给这数据和日志目录改属组。。。。。

 1 [root@CloudGame es]# ll
 2 total 8
 3 drwxr-xr-x. 4 root root 4096 Oct 18 11:20 esins1
 4 drwxr-xr-x. 4 root root 4096 Oct 18 11:21 esins2
 5 [root@CloudGame es]# chown -R water:water *
 6 [root@CloudGame es]# ll
 7 total 8
 8 drwxr-xr-x. 4 water water 4096 Oct 18 11:20 esins1
 9 drwxr-xr-x. 4 water water 4096 Oct 18 11:21 esins2
10 [root@CloudGame es]# cd esins1/
11 [root@CloudGame esins1]# ll
12 total 8
13 drwxr-xr-x. 2 water water 4096 Oct 18 11:20 data
14 drwxr-xr-x. 2 water water 4096 Oct 18 11:49 logs

再来启动一次，应该可以了吧。。。。

 1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
 2 [water@CloudGame bin]$ Exception in thread "main" java.nio.file.AccessDeniedException: /etc/esconf/esins1.pid
 3     at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
 4     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
 5     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
 6     at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
 7     at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:430)
 8     at java.nio.file.Files.newOutputStream(Files.java:172)
 9     at org.elasticsearch.common.PidFile.create(PidFile.java:76)
10     at org.elasticsearch.common.PidFile.create(PidFile.java:55)
11     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:247)
12     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
13 Refer to the log for complete error details.

去你的，怎么还是权限的问题，来 pid 文件没有办法写入。。。。。再搞一次权限配置。。。。

1 [root@CloudGame etc]# chown -R water:water esconf

再启动，再不好，是不是就不玩了。。。。

1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
2 [water@CloudGame bin]$

哈哈哈，很好，这次启动不错了，说明八九不离十了。

同样的，参照启动 esins1 的过程，启动 esins2. 一切都 ok。esconf 下面有两个 pid 文件

1 [root@CloudGame esconf]# ll -al
2 total 32
3 drwxr-xr-x.   4 water water  4096 Oct 18 12:00 .
4 drwxr-xr-x. 126 root  root  12288 Oct 18 13:25 ..
5 drwxr-xr-x.   3 water water  4096 Oct 18 11:38 esins1
6 -rw-rw-r--.   1 water water     5 Oct 18 11:55 esins1.pid
7 drwxr-xr-x.   3 water water  4096 Oct 18 11:39 esins2
8 -rw-rw-r--.   1 water water     5 Oct 18 12:00 esins2.pid