阿里云-云小站(无限量代金券发放中)
【腾讯云】云服务器、云数据库、COS、CDN、短信等热卖云产品特惠抢购

Elasticsearch单机多实例环境部署

186次阅读
没有评论

共计 14652 个字符,预计需要花费 37 分钟才能阅读完成。

Elasticsearch 的功能,主要用在搜索领域,这里,我来研究这个,也是项目需要,为公司开发了一款 CMS 系统,网站上的搜索栏功能,我打算采用 Elasticsearch 来实现。Elasticsearch 的高性能,低延时是最大的吸引力。

系统环境:

CentOS6.8,X86_64。

Elasticsearch 的版本:2.3.5,下载地址来自官网,选择的是 RPM 包。

es 安装完毕后,默认安装的路径是 /usr/share/elasticsearch, 配置文件,默认是在 /etc/elasticsearch 目录。这里要重点说说配置文件的信息:

 1 [root@CloudGame bin]# cd /etc/elasticsearch/
 2 [root@CloudGame elasticsearch]# ll
 3 total 12
 4 -rwxr-x---. 1 root elasticsearch 3189 Jul 27 18:34 elasticsearch.yml
 5 -rwxr-x---. 1 root elasticsearch 2571 Jul 27 18:34 logging.yml
 6 drwxr-x---. 2 root elasticsearch 4096 Jul 27 18:44 scripts
 7 [root@CloudGame elasticsearch]# tree
 8 .
 9 ├── elasticsearch.yml     #es 的系统配置文件
10 ├── logging.yml        #es 的日志配置文件
11 └── scripts
12 
13 1 directory, 2 files

单实例的启动很简单,安装完成后,直接 service elasticsearch start 即可启动,日志在 /var/log/elasticsearch 目录,数据在 /var/lib/elasticsearch 目录。

这里重点放在如何配置单机多实例上面。为了简单起见,多实例就选择启动 2 个实例,更多的实例,和 2 个实例是一样的配置,只是简单的相应改改参数即可。

1. 在 /etc 目录下创建 esconf 目录,然后在 esconf 下面创建 esins1 以及 esins2 目录。将原本 /etc/elasticsearch 目录的内容 copy 到 esins1 以及 esins2 目录下。

 1 [root@CloudGame esconf]# pwd
 2 /etc/esconf
 3 [root@CloudGame esconf]# tree
 4 .
 5 ├── esins1
 6 │   ├── elasticsearch.yml
 7 │   ├── logging.yml
 8 │   └── scripts
 9 ├── esins2
10     ├── elasticsearch.yml
11     ├── logging.yml
12     └── scripts
13 
14 
15 4 directories, 6 files 

2. 修改配置文件 elasticsearch.yml, 对 esins1 以及 esins2 下面的这个文件都做修改。主要是修改其节点信息,网络链接信息,以及日志数据路径信息。下面直接上修改好的配置文件内容。

实例 1:esins1 目录下的 elasticsearch.yml 内容

 1 [root@CloudGame esins1]# cat elasticsearch.yml 
 2 # ======================== Elasticsearch Configuration =========================
 3 #
 4 # NOTE: Elasticsearch comes with reasonable defaults for most settings.
 5 #       Before you set out to tweak and tune the configuration, make sure you
 6 #       understand what are you trying to accomplish and the consequences.
 7 #
 8 # The primary way of configuring a node is via this file. This template lists
 9 # the most important settings you may want to configure for a production cluster.
10 #
11 # Please see the documentation for further information on configuration options:
12 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
13 #
14 # ---------------------------------- Cluster -----------------------------------
15 #
16 # Use a descriptive name for your cluster:
17 #
18  cluster.name: tksearch        #es 默认的集群名称是 elasticsearch,注意,集群中是以 name 来区分节点属于那个集群的。19 #
20 # ------------------------------------ Node ------------------------------------
21 #
22 # Use a descriptive name for the node:
23 #
24  node.name: node1             # 节点的名称
25 #
26 # Add custom attributes to the node:
27 #
28 # node.rack: r1
29 #
30  node.master: true            # 是否让这个节点作为默认的 master,若不是,默认会选择集群里面的第一个作为 master,es 有一套选择那个节点作为 master 的机制 
31 # ----------------------------------- Paths ------------------------------------
32 #
33 # Path to directory where to store the data (separate multiple locations by comma):
34 #
35  path.data: /home/AppData/es/esins1/data # 配置节点数据存放的目录
36 #
37 # Path to log files:
38 #
39  path.logs: /home/AppData/es/esins1/logs # 配置节点日志存放的目录
40 #
41 # ----------------------------------- Memory -----------------------------------
42 #
43 # Lock the memory on startup:
44 #
45 # bootstrap.mlockall: true
46 #
47 # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
48 # available on the system and that the owner of the process is allowed to use this limit.
49 #
50 # Elasticsearch performs poorly when the system is swapping the memory.
51 #
52 # ---------------------------------- Network -----------------------------------
53 #
54 # Set the bind address to a specific IP (IPv4 or IPv6):
55 #
56  network.host: 0.0.0.0   # 配置节点绑定的地址,全 0 表示可以绑定任何地址,当然这里,本机可以是 127.0.0.1 回还地址,也可以是 ifconfig 看到的 eth1 的地址。
57 #
58 # Set a custom port for HTTP:
59 #
60  http.port: 9200   # 配置当前节点对外 http 访问的端口号,默认是 9200,不配的话,es 会从 9200-9299 当中找一个未用过的。
61 #
62 # For more information, see the documentation at:
63 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
64 #
65  transport.tcp.port: 9300  #es 集群节点之间的通信端口号。默认 9300.
66 # --------------------------------- Discovery ----------------------------------
67 #
68 # Pass an initial list of hosts to perform discovery when new node is started:
69 # The default list of hosts is ["127.0.0.1", "[::1]"]
70 #
71  discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"]  # 集群多播时发现其他节点的主机列表,真实多机集群环境下,这里会是多个主机的 IP 列表,默认格式“host:port”的数组
72 #
73 # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
74 #
75 # discovery.zen.minimum_master_nodes: 3
76 #
77 # For more information, see the documentation at:
78 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
79 #
80 # ---------------------------------- Gateway -----------------------------------
81 #
82 # Block initial recovery after a full cluster restart until N nodes are started:
83 #
84 # gateway.recover_after_nodes: 3
85 #
86 # For more information, see the documentation at:
87 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
88 #
89 # ---------------------------------- Various -----------------------------------
90 #
91 # Disable starting multiple nodes on a single system:
92 #
93  node.max_local_storage_nodes: 2  # 默认情况下,是不建议单机启动多个 node 的,这里这个参数,就是告知 es 单机上启动了几个实例,这里我们配置 2 个,若是要配置 3 个或者更多实例,修改这个数字即可 
94 #
95 # Require explicit names when deleting indices:
96 #
97 # action.destructive_requires_name: true

实例 2:esins2 目录下的 elasticsearch.yml 内容,配置和 esins1 几乎一样。

 
 1 [root@CloudGame esins2]# cat elasticsearch.yml 
 2 # ======================== Elasticsearch Configuration =========================
 3 #
 4 # NOTE: Elasticsearch comes with reasonable defaults for most settings.
 5 #       Before you set out to tweak and tune the configuration, make sure you
 6 #       understand what are you trying to accomplish and the consequences.
 7 #
 8 # The primary way of configuring a node is via this file. This template lists
 9 # the most important settings you may want to configure for a production cluster.
10 #
11 # Please see the documentation for further information on configuration options:
12 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
13 #
14 # ---------------------------------- Cluster -----------------------------------
15 #
16 # Use a descriptive name for your cluster:
17 #
18  cluster.name: tksearch
19 #
20 # ------------------------------------ Node ------------------------------------
21 #
22 # Use a descriptive name for the node:
23 #
24  node.name: node2
25 #
26 # Add custom attributes to the node:
27 #
28 # node.rack: r1
29 #
30  node.master: false
31 # ----------------------------------- Paths ------------------------------------
32 #
33 # Path to directory where to store the data (separate multiple locations by comma):
34 #
35  path.data: /home/AppData/es/esins2/data
36 #
37 # Path to log files:
38 #
39  path.logs: /home/AppData/es/esins2/logs
40 #
41 # ----------------------------------- Memory -----------------------------------
42 #
43 # Lock the memory on startup:
44 #
45 # bootstrap.mlockall: true
46 #
47 # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
48 # available on the system and that the owner of the process is allowed to use this limit.
49 #
50 # Elasticsearch performs poorly when the system is swapping the memory.
51 #
52 # ---------------------------------- Network -----------------------------------
53 #
54 # Set the bind address to a specific IP (IPv4 or IPv6):
55 #
56  network.host: 0.0.0.0
57 #
58 # Set a custom port for HTTP:
59 #
60  http.port: 9201
61 #
62 # For more information, see the documentation at:
63 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
64 #
65  transport.tcp.port: 9301
66 # --------------------------------- Discovery ----------------------------------
67 #
68 # Pass an initial list of hosts to perform discovery when new node is started:
69 # The default list of hosts is ["127.0.0.1", "[::1]"]
70 #
71  discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"]
72 #
73 # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
74 #
75 # discovery.zen.minimum_master_nodes: 3
76 #
77 # For more information, see the documentation at:
78 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
79 #
80 # ---------------------------------- Gateway -----------------------------------
81 #
82 # Block initial recovery after a full cluster restart until N nodes are started:
83 #
84 # gateway.recover_after_nodes: 3
85 #
86 # For more information, see the documentation at:
87 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
88 #
89 # ---------------------------------- Various -----------------------------------
90 #
91 # Disable starting multiple nodes on a single system:
92 #
93  node.max_local_storage_nodes: 2
94 #
95 # Require explicit names when deleting indices:
96 #
97 # action.destructive_requires_name: true

3. 由于配置中指定了配置文件和数据的路径了,所以,要在相应的路径下创建所需的目录。

1 /home/AppData/es/esins1/data
2 /home/AppData/es/esins1/logs
3 /home/AppData/es/esins2/data
4 /home/AppData/es/esins2/logs

4. 启动 es,先启动 es 实例 1. 注意,es 启动时指定的配置文件时,要指定配置文件所在的路径,这个路径包含 elasticsearch.yml 以及 logging.yml

1 [root@CloudGame bin]# ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
2 [root@CloudGame bin]# Exception in thread "main" Java.lang.RuntimeException: don't run elasticsearch as root.
3     at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:93)
4     at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:144)
5     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
6     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
7 Refer to the log for complete error details.

哟,错了啊, 不能用 root 启动 es,见鬼,再来一次吧。

 1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
 2 [water@CloudGame bin]$ Exception in thread "main" SettingsException[Failed to open stream for url [/etc/esconf/esins1/elasticsearch.yml]]; nested: AccessDeniedException[/etc/esconf/esins1/elasticsearch.yml];
 3 Likely root cause: java.nio.file.AccessDeniedException: /etc/esconf/esins1/elasticsearch.yml
 4     at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
 5     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
 6     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
 7     at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
 8     at java.nio.file.Files.newByteChannel(Files.java:317)
 9     at java.nio.file.Files.newByteChannel(Files.java:363)
10     at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:380)
11     at java.nio.file.Files.newInputStream(Files.java:108)
12     at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1067)
13     at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88)
14     at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
15     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
16     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
17 Refer to the log for complete error details.

哟嘿,还是错???访问 etc/esconf/esins1/elasticsearch.yml 被阻止了???想起来了,这个文件是在 root 用户下创建配置的。修改一下用户组吧: chown -R water:water /etc/esconf/esins1。再来一次。。。

 1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
 2 [water@CloudGame bin]$ Exception in thread "main" SettingsException[Failed to open stream for url [/etc/esconf/esins1/elasticsearch.yml]]; nested: AccessDeniedException[/etc/esconf/esins1/elasticsearch.yml];
 3 Likely root cause: java.nio.file.AccessDeniedException: /etc/esconf/esins1/elasticsearch.yml
 4     at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
 5     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
 6     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
 7     at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
 8     at java.nio.file.Files.newByteChannel(Files.java:317)
 9     at java.nio.file.Files.newByteChannel(Files.java:363)
10     at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:380)
11     at java.nio.file.Files.newInputStream(Files.java:108)
12     at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1067)
13     at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88)
14     at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
15     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
16     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
17 Refer to the log for complete error details.

怎么还是一样的错啊,去看看日志吧,靠,/home/AppData/es/esins1/logs 下面什么也没有啊???????倒,这个目录也是在 root 权限下创建的,water 没有写的权限。。。。。。再给这数据和日志目录改属组。。。。。

 1 [root@CloudGame es]# ll
 2 total 8
 3 drwxr-xr-x. 4 root root 4096 Oct 18 11:20 esins1
 4 drwxr-xr-x. 4 root root 4096 Oct 18 11:21 esins2
 5 [root@CloudGame es]# chown -R water:water *
 6 [root@CloudGame es]# ll
 7 total 8
 8 drwxr-xr-x. 4 water water 4096 Oct 18 11:20 esins1
 9 drwxr-xr-x. 4 water water 4096 Oct 18 11:21 esins2
10 [root@CloudGame es]# cd esins1/
11 [root@CloudGame esins1]# ll
12 total 8
13 drwxr-xr-x. 2 water water 4096 Oct 18 11:20 data
14 drwxr-xr-x. 2 water water 4096 Oct 18 11:49 logs

再来启动一次,应该可以了吧。。。。

 1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
 2 [water@CloudGame bin]$ Exception in thread "main" java.nio.file.AccessDeniedException: /etc/esconf/esins1.pid
 3     at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
 4     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
 5     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
 6     at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
 7     at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:430)
 8     at java.nio.file.Files.newOutputStream(Files.java:172)
 9     at org.elasticsearch.common.PidFile.create(PidFile.java:76)
10     at org.elasticsearch.common.PidFile.create(PidFile.java:55)
11     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:247)
12     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
13 Refer to the log for complete error details.

去你的,怎么还是权限的问题,来 pid 文件没有办法写入。。。。。再搞一次权限配置。。。。

1 [root@CloudGame etc]# chown -R water:water esconf

再启动,再不好,是不是就不玩了。。。。

1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
2 [water@CloudGame bin]$

哈哈哈,很好,这次启动不错了,说明八九不离十了。

同样的,参照启动 esins1 的过程,启动 esins2. 一切都 ok。esconf 下面有两个 pid 文件

1 [root@CloudGame esconf]# ll -al
2 total 32
3 drwxr-xr-x.   4 water water  4096 Oct 18 12:00 .
4 drwxr-xr-x. 126 root  root  12288 Oct 18 13:25 ..
5 drwxr-xr-x.   3 water water  4096 Oct 18 11:38 esins1
6 -rw-rw-r--.   1 water water     5 Oct 18 11:55 esins1.pid
7 drwxr-xr-x.   3 water water  4096 Oct 18 11:39 esins2
8 -rw-rw-r--.   1 water water     5 Oct 18 12:00 esins2.pid

到此,单机双实例的启动完成。

真的完成了么?要不访问看看,那就验证一下吧,下面来几张截图 demo 一下最后的结果!

esins1 的基本信息:

Elasticsearch 单机多实例环境部署

esins2 的基本信息:

Elasticsearch 单机多实例环境部署

esins1 的健康状态:

Elasticsearch 单机多实例环境部署

esins2 的健康状态:

Elasticsearch 单机多实例环境部署

ElasticSearch 最新版本 2.20 发布下载了   http://www.linuxidc.com/Linux/2016-02/128166.htm

Linux 上安装部署 ElasticSearch 全程记录  http://www.linuxidc.com/Linux/2015-09/123241.htm

Elasticsearch 安装使用教程 http://www.linuxidc.com/Linux/2015-02/113615.htm

ElasticSearch 配置文件译文解析 http://www.linuxidc.com/Linux/2015-02/114244.htm

ElasticSearch 集群搭建实例  http://www.linuxidc.com/Linux/2015-02/114243.htm

分布式搜索 ElasticSearch 单机与服务器环境搭建  http://www.linuxidc.com/Linux/2012-05/60787.htm

ElasticSearch 的工作机制  http://www.linuxidc.com/Linux/2014-11/109922.htm 

Elasticsearch 的安装,运行和基本配置 http://www.linuxidc.com/Linux/2016-07/133057.htm

使用 Elasticsearch + Logstash + Kibana 搭建日志集中分析平台实践  http://www.linuxidc.com/Linux/2015-12/126587.htm

Ubuntu 14.04 搭建 ELK 日志分析系统 (Elasticsearch+Logstash+Kibana) http://www.linuxidc.com/Linux/2016-06/132618.htm

Elasticsearch1.7 升级到 2.3 实践总结  http://www.linuxidc.com/Linux/2016-11/137282.htm

ElasticSearch 的详细介绍 :请点这里
ElasticSearch 的下载地址 :请点这里 

本文永久更新链接地址 :http://www.linuxidc.com/Linux/2016-11/137359.htm

正文完
星哥说事-微信公众号
post-qrcode
 0
星锅
版权声明:本站原创文章,由 星锅 于2022-01-21发表,共计14652字。
转载说明:除特殊说明外本站文章皆由CC-4.0协议发布,转载请注明出处。
【腾讯云】推广者专属福利,新客户无门槛领取总价值高达2860元代金券,每种代金券限量500张,先到先得。
阿里云-最新活动爆款每日限量供应
评论(没有评论)
验证码
【腾讯云】云服务器、云数据库、COS、CDN、短信等云产品特惠热卖中