首页  

elasticsearch5.0重要系统配置     所属分类 elasticsearch 浏览量 1531
根据原文翻译整理
https://www.elastic.co/guide/en/elasticsearch/reference/5.0/system-config.html


Ideally, Elasticsearch should run alone on a server and use all of the resources available to it.
理想情况下,ES应该单独在服务器上运行,使用所有可用的资源

生产环境必须做以下配置

Set JVM heap size  JVM堆大小设置
Disable swapping  禁用内存交换
Increase file descriptors 增加打开文件数限制
Ensure sufficient virtual memory  足够的虚拟内存,地址空间 
Ensure sufficient threads    足够的线程数


development mode vs production mode

开发模式和生产模式

开发模式  配置不正确时会写警告日志,可以正常启动

network.host 配置之后,进入生产模式 ,配置错误将无法启动
 
系统配置 
压缩包安装方式
临时配置 使用  ulimit
永久配置 使用  /etc/security/limits.conf

rpm包安装 大部分配置都在 配置文件中
/etc/sysconfig/elasticsearch

sudo ulimit -n 65536
Change the max number of open files.

/etc/security/limits.conf  

elasticsearch  -  nofile  65536

set the maximum number of open files for the elasticsearch user to 65,536


When using the RPM or Debian packages on systems that use systemd, system limits must be specified via systemd.
The systemd service file (/usr/lib/systemd/system/elasticsearch.service) contains the limits that are applied by default.
To override these, add a file called /etc/systemd/system/elasticsearch.service.d/elasticsearch.conf

[Service]
LimitMEMLOCK=infinity


JVM参数设置 

config/jvm.options 

RPM包安装
/etc/elasticsearch/jvm.options

export ES_JAVA_OPTS="$ES_JAVA_OPTS -Djava.io.tmpdir=/path/to/temp/dir"

./bin/elasticsearch


-Xms2g -Xmx2g
-Xms4000m -Xmx4000m

Good rules of thumb 经验法则
最小堆大小(Xms)和最大堆大小(Xmx)设成一样
The more heap available to Elasticsearch, the more memory it can use for caching. 
But note that too much heap can subject you to long garbage collection pauses.
堆越大,缓存越大,但是会导致长时间的GC停顿

Set Xmx to no more than 50% of your physical RAM, to ensure that there is enough physical RAM left for kernel file system caches.
Xmx设置不超过物理RAM的50%,确保有足够的物理RAM留给内核文件系统缓存。

ordinary object pointer 普通对象指针
64 bit JVM正式支持 -XX:+UseCompressedOops  压缩指针 节约内存占用
零基压缩优化(Zero Based Compressd Oops) 
进一步优化 通过改变正常指针的随机地址分配特性,强制从零开始做分配(需要OS支持),进一步提高了压解压效率。 

要启用零基压缩,你分配给JVM的内存大小必须控制在4G以上,32G以下。  最好不要超过26G 


 try to stay below the threshold for zero-based compressed oops
 -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode
 heap address: 0x000000011be00000, size: 27648 MB, zero based Compressed Oops
 
 非  zero based Compressed Oops 的情况 
 heap address: 0x0000000118400000, size: 28672 MB, Compressed Oops with base: 0x00000001183ff000
 
 
Disable swapping 禁用内存交换
Swapping is very bad for performance and for node stability and should be avoided at all costs. 
交换对性能和节点稳定性非常不利,应该不惜一切代价避免。

It can cause garbage collections to last for minutes instead of milliseconds 
and can cause nodes to respond slowly or even to disconnect from the cluster.

它会导致垃圾收集持续几分钟而不是几毫秒,还会导致节点响应缓慢,甚至与集群断开连接。

禁用内存交换的方法

bootstrap.memory_lock: true
http://127.0.0.1:9200/_nodes?filter_path=**.mlockall
如果mlockall为false 说明 锁定内存操作失败  显示错误日志  Unable to lock JVM Memory
最有可能的原因是没有权限 
Set ulimit -l unlimited as root before starting Elasticsearch, 
or set memlock to unlimited in /etc/security/limits.conf

临时 禁用交换 sudo swapoff -a
edit the /etc/fstab file and comment out any lines that contain the word swap.
注释掉包含swap的行

Linux系统上可用的第二个选项是确保 sysctl值 vm.swappiness设置为1。
这减少了内核交换的倾向,在正常情况下不应该导致交换,同时仍然允许整个系统在紧急情况下交换。


mlockall might cause the JVM or shell session to exit if it tries to allocate more memory than is available!


Make sure to increase the limit on the number of open files descriptors for the user running Elasticsearch to 65,536 or higher.

set ulimit -n 65536 as root before starting Elasticsearch
or set nofile to 65536 in /etc/security/limits.conf.

http://127.0.0.1:9200/_nodes/stats/process?filter_path=**.max_file_descriptors

Virtual memory
Elasticsearch uses a hybrid mmapfs / niofs directory by default to store its indices. 
The default operating system limits on mmap counts is likely to be too low, which may result in out of memory exceptions.
sysctl -w vm.max_map_count=262144

update the vm.max_map_count setting in /etc/sysctl.conf


Elasticsearch uses a number of thread pools for different types of operations. 
Make sure that the number of threads that the Elasticsearch user can create is at least 2048.

setting ulimit -u 2048 as root before starting Elasticsearch, 
or by setting nproc to 2048 in /etc/security/limits.conf

上一篇     下一篇
elasticsearch5.0配置

elasticsearch5.0的重要配置

elasticsearch5.0启动检查

elasticsearch5.0API约定

elasticsearch5.0文档索引API

elasticsearch5.0文档读取API