Elastic Stack学习--elasticsearch部署常见问题-低调大师

Elastic Stack学习--elasticsearch部署常见问题

2018-03-11 729

linux内核版本低于3.5，不支持seccomp；

[2018-03-12T11:56:52,328][WARN ][o.e.b.JNANatives         ] unable to install syscall filter:
java.lang.UnsupportedOperationException: seccomp unavailable: requires kernel 3.5+ with CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER compiled in
        at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:328) ~[elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:616) ~[elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:258) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:113) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:110) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.2.2.jar:6.2.2]
        at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.2.2.jar:6.2.2]

解决办法：
修改elasticsearch.yaml文件，设置禁用seccomp；

bootstrap.system_call_filter: false

官方文档参考

用户没有权限使用mlockall;

[2018-03-12T11:56:52,328][WARN ][o.e.b.JNANatives         ] unable to install syscall filter:
java.lang.UnsupportedOperationException: seccomp unavailable: requires kernel 3.5+ with CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER compiled in
        at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:328) ~[elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:616) ~[elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:258) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:113) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:110) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.2.2.jar:6.2.2]
        at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.2.2.jar:6.2.2]
[2018-03-12T11:56:52,359][WARN ][o.e.b.JNANatives         ] Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
[2018-03-12T11:56:52,359][WARN ][o.e.b.JNANatives         ] This can result in part of the JVM being swapped out.
[2018-03-12T11:56:52,359][WARN ][o.e.b.JNANatives         ] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
[2018-03-12T11:56:52,359][WARN ][o.e.b.JNANatives         ] These can be adjusted by modifying /etc/security/limits.conf, for example:
        # allow user 'work' mlockall
        work soft memlock unlimited
        work hard memlock unlimited
[2018-03-12T11:56:52,359][WARN ][o.e.b.JNANatives         ] If you are logged in interactively, you will have to re-login for the new limits to take effect.

解决办法：
使用root用户修改/etc/security/limits.conf文件：

work soft memlock unlimited
work hard memlock unlimited

官方文档参考

最大文件句柄数设置过低；

解决办法：
使用root用户修改/etc/security/limits.conf文件：

work soft nofile 65536
work hard nofile 65536

官方文档参考

最大线程数设置过低；

解决办法：
使用root用户修改/etc/security/limits.conf文件：

work soft nproc 4096
work hard nproc 4096

官方文档参考

vm.max_map_count设置过低

[2018-03-12T14:25:44,413][WARN ][o.e.b.BootstrapChecks    ] [yf-beidou-dmp00.yf01.baidu.com] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

解决办法：
1）执行如下命令，实时生效：

sysctl -w vm.max_map_count=262144

2）修改/etc/sysctl.conf，重启后生效：

vm.max_map_count = 262144

官方文档参考

注：要使 limits.conf 文件配置生效，必须要确保 pam_limits.so 文件被加入到启动文件中。查看 /etc/pam.d/login 文件中有：session required /lib/security/pam_limits.so这一行配置；

安装x-pack重启后报错：Failed to create native process factories for Machine Learning

org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning
        at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:422) ~[?:?]
        at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:373) ~[?:?]
        at org.elasticsearch.node.Node.lambda$new$7(Node.java:397) ~[elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.node.Node$$Lambda$1374/1560391896.apply(Unknown Source) ~[?:?]
        at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) ~[?:1.8.0_45]
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374) ~[?:1.8.0_45]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512) ~[?:1.8.0_45]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502) ~[?:1.8.0_45]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_45]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_45]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_45]
        at org.elasticsearch.node.Node.<init>(Node.java:400) ~[elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.2.2.jar:6.2.2]
        at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.2.2.jar:6.2.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.2.2.jar:6.2.2]

这是由于x-pack中的machine learning功能依赖于2.9以上版本GLIBC，环境中缺乏glibc库所致。可通过如下命令定位问题：

${ES_HOME}/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller --version

输出缺少依赖的日志如下：

./controller: /lib64/tls/libc.so.6: version `GLIBC_2.4' not found (required by ./controller)
./controller: /lib64/tls/libc.so.6: version `GLIBC_2.4' not found (required by /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/libMlCore.so)
./controller: /lib64/tls/libc.so.6: version `GLIBC_2.7' not found (required by /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/libMlCore.so)
./controller: /lib64/tls/libpthread.so.0: version `GLIBC_2.4' not found (required by /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libapr-1.so.0)
./controller: /lib64/tls/libc.so.6: version `GLIBC_2.9' not found (required by /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libapr-1.so.0)
./controller: /lib64/tls/libc.so.6: version `GLIBC_2.7' not found (required by /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libxml2.so.2)

或者通过ldd命令查看controller的依赖：

ldd ${ES_HOME}/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller

输出日志如下：

./controller: /lib64/tls/libc.so.6: version `GLIBC_2.4' not found (required by ./controller)
./controller: /lib64/tls/libc.so.6: version `GLIBC_2.4' not found (required by /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/libMlCore.so)
./controller: /lib64/tls/libc.so.6: version `GLIBC_2.7' not found (required by /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/libMlCore.so)
./controller: /lib64/tls/libpthread.so.0: version `GLIBC_2.4' not found (required by /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libapr-1.so.0)
./controller: /lib64/tls/libc.so.6: version `GLIBC_2.9' not found (required by /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libapr-1.so.0)
./controller: /lib64/tls/libc.so.6: version `GLIBC_2.7' not found (required by /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libxml2.so.2)
    libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x00007fb285c82000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007fb285b7f000)
    librt.so.1 => /lib64/tls/librt.so.1 (0x00007fb285a65000)
    liblog4cxx.so.10 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/liblog4cxx.so.10 (0x00007fb28567e000)
    libboost_program_options-gcc62-mt-1_65_1.so.1.65.1 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/libboost_program_options-gcc62-mt-1_65_1.so.1.65.1 (0x00007fb2853fd000)
    libMlCore.so => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/libMlCore.so (0x00007fb2850d1000)
    libstdc++.so.6 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/libstdc++.so.6 (0x00007fb284d27000)
    libm.so.6 => /lib64/tls/libm.so.6 (0x00007fb284ba1000)
    libgcc_s.so.1 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/libgcc_s.so.1 (0x00007fb28498a000)
    libc.so.6 => /lib64/tls/libc.so.6 (0x00007fb284756000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fb285d97000)
    libaprutil-1.so.0 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libaprutil-1.so.0 (0x00007fb28452f000)
    libexpat.so.0 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libexpat.so.0 (0x00007fb284304000)
    libapr-1.so.0 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libapr-1.so.0 (0x00007fb2840ce000)
    libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fb283f9a000)
    libxml2.so.2 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libxml2.so.2 (0x00007fb283c20000)
    libz.so.1 => /usr/lib64/libz.so.1 (0x00007fb283b0d000)
    libboost_regex-gcc62-mt-1_65_1.so.1.65.1 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libboost_regex-gcc62-mt-1_65_1.so.1.65.1 (0x00007fb283815000)
    libboost_iostreams-gcc62-mt-1_65_1.so.1.65.1 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libboost_iostreams-gcc62-mt-1_65_1.so.1.65.1 (0x00007fb283600000)
    libboost_filesystem-gcc62-mt-1_65_1.so.1.65.1 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libboost_filesystem-gcc62-mt-1_65_1.so.1.65.1 (0x00007fb2833e5000)
    libboost_system-gcc62-mt-1_65_1.so.1.65.1 => /home/work/elasticsearch-6.2.2/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/../lib/./libboost_system-gcc62-mt-1_65_1.so.1.65.1 (0x00007fb2831e1000)

解决办法1：如果不需要使用machine learning功能，则可以在elasticsearch.yml中设置禁用：

xpack.ml.enabled: false

解决办法2：安装或者升级glibc；如果使用centos4.3版本，则果断升级操作系统；

参考官方论坛

如何修改临时文件目录

elasticsearch以及x-pack插件的运行，依赖于ES_TMPDIR环境变量的值作为临时文件目录；如果未设置，则默认为/tmp/elasticsearch;可通过在.bashrc或者.bash_profile文件中添加环境变量显式设置临时文件目录：

export ES_HOME=/home/work/elasticsearch-6.2.2
export ES_TMPDIR="${ES_HOME}/tmp"

微信关注我们

原文链接：https://yq.aliyun.com/articles/531617

转载内容版权归作者及来源网站所有！

低调大师中文资讯倾力打造互联网数据资讯、行业资源、电子商务、移动互联网、网络营销平台。持续更新报道IT业界、互联网、市场资讯、驱动更新,是最及时权威的产业资讯及硬件资讯报道平台。

3月12日云栖精选夜读：操作阿里云Kibana

Elastic公司的“ELK”是目前最火的日志分析三剑客，其中ElasticSearch负责日志的索引，Logstash负责日志的收集，Kibana负责日志的展示和分析。事实上并不局限于日志分析，ELK技术栈同样适用于通用的数据采集、索引、分析业务。不可谓不强大。 Elastic与阿里云达成了合作伙伴关系，提供“阿里云 Elasticsearch”的新服务，提供了开箱即用的Elastic 热点热议操作阿里云Kibana 作者：lxiange 阿里云Redis开发规范作者：carlosfu 阿里云MVP共创空间正式发布，参与技术共创，成为阿里云MVP 作者：阿里云服务知识整理使用ASP.NET实现Windows Service定时执行任务作者：优惠券发放 Java多线程实例作者：anmypeng1 机器学习是如何借鉴物理学思想的？从伊辛模型谈起（万字长文）作者：技术小能手 python之文件的复制作者：科技小能手通过apache创建一个yum仓库作者：麦老师美文回顾视频直播常见问题与解决办法汇总【系列一】作者：樰篱设计模式--代理模式作者：积淀 MySQL...

2018-03-11

799

云HBASE上Phoenix支持的DML select upsert values upsert select delete 1. SELECT 从一个或者多个表中查询数据。LIMIT(或者FETCH FIRST) 在ORDER BY子句后将转换为top-N查询。OFFSET子句指定返回查询结果前跳过的行数。示例 SELECT * FROM TEST LIMIT 1000; SELECT * FROM TEST LIMIT 1000 OFFSET 100; SELECT full_name FROM SALES_PERSON WHERE ranking >= 5.0 UNION ALL SELECT reviewer_name FROM CUSTOMER_REVIEW WHERE score >= 8.0 2. UP

2018-03-11

804

资源下载

更多资源

Mario

马里奥是站在游戏界顶峰的超人气多面角色。马里奥靠吃蘑菇成长，特征是大鼻子、头戴帽子、身穿背带裤，还留着胡子。与他的双胞胎兄弟路易基一起，长年担任任天堂的招牌角色。

Rocky Linux

Rocky Linux（中文名：洛基）是由Gregory Kurtzer于2020年12月发起的企业级Linux发行版，作为CentOS稳定版停止维护后与RHEL（Red Hat Enterprise Linux）完全兼容的开源替代方案，由社区拥有并管理，支持x86_64、aarch64等架构。其通过重新编译RHEL源代码提供长期稳定性，采用模块化包装和SELinux安全架构，默认包含GNOME桌面环境及XFS文件系统，支持十年生命周期更新。

Sublime Text

Sublime Text具有漂亮的用户界面和强大的功能，例如代码缩略图，Python的插件，代码段等。还可自定义键绑定，菜单和工具栏。Sublime Text 的主要功能包括：拼写检查，书签，完整的 Python API ， Goto 功能，即时项目切换，多选择，多窗口等等。Sublime Text 是一个跨平台的编辑器，同时支持Windows、Linux、Mac OS X等操作系统。

WebStorm

WebStorm 是jetbrains公司旗下一款JavaScript 开发工具。目前已经被广大中国JS开发者誉为“Web前端开发神器”、“最强大的HTML5编辑器”、“最智能的JavaScript IDE”等。与IntelliJ IDEA同源，继承了IntelliJ IDEA强大的JS部分的功能。