License: Attribution-NonCommercial-ShareAlike 4.0 International
本文出自 Suzf Blog。 如未注明，均为 SUZF.NET 原创。
前言
ES 群集的部署可谓是“傻瓜式”的，需要自定义的地方就是 `/etc/elasticsearch/elasticsearch.yml` 里的 cluster.name。然后，在 `Disicovery` 可达的范围内，所有的 elasticsearch node 会自动寻找和自己相同的cluster.name 的兄弟， 然后按照最朴素的先来后到的规则确定master。至此集群创建完成。
节点类型
当我们启动Elasticsearch的实例，就会启动至少一个节点。相同集群名的多个节点的连接就组成了一个集群，在默认情况下，集群中的每个节点都可以处理http请求和集群节点间的数据传输，集群中所有的节点都知道集群中其他所有的节点，可以将客户端请求转发到适当的节点。节点有以下类型：
- 主(master)节点：在一个节点上当node.master设置为True（默认）的时候，它有资格被选作为主节点，控制整个集群。
- 数据(data)节点：在一个节点上node.data设置为True（默认）的时候。该节点保存数据和执行数据相关的操作，如增删改查，搜索，和聚合。
- 客户端节点：当一个节点的node.master和node.data都设置为false的时候，它既不能保持数据也不能成为主节点，该节点可以作为客户端节点，可以响应用户的情况，并把相关操作发送到其他节点。
- 部落节点： 当一个节点配置tribe.*的时候，它是一个特殊的客户端，它可以连接多个集群，在所有连接的集群上执行搜索和其他操作。
默认情况下，节点配置是一个主节点和一个数据节点。这是非常方便的小集群，但随着集群的发展，分离主节点和数据节点将变得很重要。
部署
安装 Java JDK
# yum yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel -y OR wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u65-b17/jdk-8u65-linux-x64.rpm" rpm -Uvh jdk-8u65-linux-x64.rpm cat > /etc/profile.d/java.sh << EOF JAVA_HOME=/usr/java/latest PATH=\$JAVA_HOME/bin:\$PATH export JAVA_HOME PATH EOF source /etc/profile.d/java.sh # 查看 JDK java -version
集群节点
cat >> /etc/hosts << EOF 172.16.9.11 el1.suzf.net 172.16.9.50 el2.suzf.net 172.16.9.60 el3.suzf.net EOF
集群配置
注： 其它节点配置类似， 只需修改 node & `network` 部分
#cat /etc/elasticsearch/elasticsearch.yml # ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es.suzf.net # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: es1.suzf.net # # Add custom attributes to the node: # node.attr.rack: es1.suzf.net # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: /var/lib/elasticsearch # # Path to log files: # path.logs: /var/log/elasticsearch # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # bootstrap.memory_lock: true bootstrap.seccomp: false # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 172.16.9.11 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.zen.ping.unicast.hosts: ["es1.suzf.net", "es2.suzf.net", "es3.suzf.net"] # # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1): # discovery.zen.minimum_master_nodes: 2 # # For more information, consult the zen discovery module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # gateway.recover_after_nodes: 2 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # action.destructive_requires_name: true
查看群集状态信息
# curl -XGET 'es2.suzf.net:9200/_cluster/health?pretty'
{
"cluster_name" : "es.suzf.net",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 3,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
FAQ
官方提供了生产环境下 Elasticsearch 所在操作系统的相关建议，详见 这里
1. 跨域支持 根据官网建议添加了如下配置到elasticsearc.yml主配置文件 http.cors.enabled: true http.cors.allow-origin: "*" 问题二： max number of threads [1024] for user [elasticsearch] is too low, increase to at least [2048] 解决方法：进入limits.d下的配置文件：vim /etc/security/limits.d/90-nproc.conf ，修改配置如下： vim /etc/security/limits.d/90-nproc.conf * soft nproc 1024 修改为： * soft nproc 2048 问题三： max file descriptors [64000] for elasticsearch process is too low, increase to at least [65536] 解决办法: 1、虚拟内存设置，编辑 /etc/sysctl.conf ，追加： vm.max_map_count=262144 2、修改文件句柄限制，编辑 /etc/security/limits.conf ，追加： root># vim /etc/security/limits.conf * soft nofile 65536 * hard nofile 65536 问题四： elasticsearch5.0启动失败，出现如下提示： 1、Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed; error='Cannot allocate memory' (errno=12) 由于elasticsearch5.0默认分配jvm空间大小为2g，修改jvm空间分配 # vim config/jvm.options -Xms2g -Xmx2g 修改为 -Xms512m -Xmx512m # Ps 虚拟机内存分配为 1G 问题五： Unable to lock JVM Memory: error=12, reason=Cannot allocate memory cat >> /etc/security/limits.conf << EOF # elasticsearch setting elasticsearch soft nofile 65536 elasticsearch hard nofile 65536 elasticsearch soft nproc 2048 elasticsearch hard nproc 2048 elasticsearch - memlock unlimited EOF # grep LimitMEMLOCK /usr/lib/systemd/system/elasticsearch.service LimitMEMLOCK=infinity # reload systemd systemctl daemon-reload # Link -- https://www.elastic.co/guide/en/elasticsearch/reference/5.2/setting-system-settings.html#limits.conf -- https://discuss.elastic.co/t/jvm-memory-in-5-2-2/77663/3
Reference
[0] https://www.elastic.co/guide/index.html
[1] https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html
[2] https://www.elastic.co/guide/en/elasticsearch/reference/current/secure-settings.html
[3] http://rickywag.com/archives/287
