使用elasticdump迁移数据到新es集群
参考文档
https://blog.csdn.net/u012731379/article/details/79856113 https://blog.csdn.net/u010466329/article/details/79209236 https://blog.csdn.net/laoyang360/article/details/65449407
迁移方法
- 通过logstash的input和output配置迁移(配置灵活适用于长期数据同步等)
- 通过迁移工具如elasticdump等(适用于备份一次性小量数据操作)
- 通过elasticsarch自带快照功能(适用于一次性迁移大量数据)
操作步骤
1、安装elasticdump
[root@VM_8_24_centos ~]# yum install nodejs npm [root@VM_8_24_centos ~]# npm install elasticdump /root └─┬ elasticdump@4.4.0 ├── async@2.6.1 ├─┬ aws-sdk@2.400.0 │ ├─┬ buffer@4.9.1 │ │ ├── base64-js@1.3.0 │ │ └── isarray@1.0.0 │ ├── events@1.1.1 │ ├── ieee754@1.1.8 │ ├── jmespath@0.15.0 │ ├── querystring@0.2.0 │ ├── sax@1.2.1 │ ├─┬ url@0.10.3 │ │ └── punycode@1.3.2 │ ├── uuid@3.3.2 │ └─┬ xml2js@0.4.19 │ └── xmlbuilder@9.0.7 ├── aws4@1.8.0 ├── bytes@3.1.0 ├── decimal.js@10.0.2 ├── ini@1.3.5 ├─┬ JSONStream@1.3.5 │ ├── jsonparse@1.3.1 │ └── through@2.3.8 ├── lodash@4.17.11 ├── lossless-json@1.0.3 ├─┬ optimist@0.6.1 │ ├── minimist@0.0.10 │ └── wordwrap@0.0.3 ├─┬ request@2.88.0 │ ├── aws-sign2@0.7.0 │ ├── caseless@0.12.0 │ ├─┬ combined-stream@1.0.7 │ │ └── delayed-stream@1.0.0 │ ├── extend@3.0.2 │ ├── forever-agent@0.6.1 │ ├─┬ form-data@2.3.3 │ │ └── asynckit@0.4.0 │ ├─┬ har-validator@5.1.3 │ │ ├─┬ ajv@6.9.1 │ │ │ ├── fast-deep-equal@2.0.1 │ │ │ ├── fast-json-stable-stringify@2.0.0 │ │ │ ├── json-schema-traverse@0.4.1 │ │ │ └─┬ uri-js@4.2.2 │ │ │ └── punycode@2.1.1 │ │ └── har-schema@2.0.0 │ ├─┬ http-signature@1.2.0 │ │ ├── assert-plus@1.0.0 │ │ ├─┬ jsprim@1.4.1 │ │ │ ├── extsprintf@1.3.0 │ │ │ ├── json-schema@0.2.3 │ │ │ └── verror@1.10.0 │ │ └─┬ sshpk@1.16.1 │ │ ├── asn1@0.2.4 │ │ ├── bcrypt-pbkdf@1.0.2 │ │ ├── dashdash@1.14.1 │ │ ├── ecc-jsbn@0.1.2 │ │ ├── getpass@0.1.7 │ │ ├── jsbn@0.1.1 │ │ ├── safer-buffer@2.1.2 │ │ └── tweetnacl@0.14.5 │ ├── is-typedarray@1.0.0 │ ├── isstream@0.1.2 │ ├── json-stringify-safe@5.0.1 │ ├─┬ mime-types@2.1.21 │ │ └── mime-db@1.37.0 │ ├── oauth-sign@0.9.0 │ ├── performance-now@2.1.0 │ ├── qs@6.5.2 │ ├── safe-buffer@5.1.2 │ ├─┬ tough-cookie@2.4.3 │ │ ├── psl@1.1.31 │ │ └── punycode@1.4.1 │ └── tunnel-agent@0.6.0 ├─┬ requestretry@3.1.0 │ └── when@3.7.8 └─┬ s3-stream-upload@2.0.2 ├── buffer-queue@1.0.0 └─┬ readable-stream@2.3.6 ├── core-util-is@1.0.2 ├── inherits@2.0.3 ├── process-nextick-args@2.0.0 ├── string_decoder@1.1.1 └── util-deprecate@1.0.2 npm WARN enoent ENOENT: no such file or directory, open '/root/package.json' npm WARN root No description npm WARN root No repository field. npm WARN root No README data npm WARN root No license field. [root@VM_8_24_centos ~]# cd node_modules/elasticdump/bin
2、导出mapping
[root@VM_8_24_centos bin]# ./elasticdump --input=http://10.2.3.159:9200/reconciliation --output=http://10.2.100.24:9200/reconciliationonline --type=mapping Mon, 11 Feb 2019 09:28:10 GMT | starting dump Mon, 11 Feb 2019 09:28:10 GMT | got 1 objects from source elasticsearch (offset: 0) Mon, 11 Feb 2019 09:28:16 GMT | sent 1 objects to destination elasticsearch, wrote 1 Mon, 11 Feb 2019 09:28:16 GMT | got 0 objects from source elasticsearch (offset: 1) Mon, 11 Feb 2019 09:28:16 GMT | Total Writes: 1 Mon, 11 Feb 2019 09:28:16 GMT | dump complete [root@VM_8_24_centos bin]# curl 10.2.100.24:9200/_cat/indices|grep reconciliation % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 43 32578 43 14037 0 0 5694 0 0:00:05 0:00:02 0:00:03 5694yellow open reconciliationonline KnZrsU-7TmWWBs-1-dBRrw 5 1 0 0 1.1kb 1.1kb
3、导出data
[root@VM_8_24_centos bin]# ./elasticdump --input=http://10.2.3.159:9200/reconciliation --output=http://10.2.100.24:9200/reconciliationonline --type=data Mon, 11 Feb 2019 09:28:58 GMT | starting dump Mon, 11 Feb 2019 09:28:58 GMT | got 100 objects from source elasticsearch (offset: 0) Mon, 11 Feb 2019 09:29:19 GMT | sent 100 objects to destination elasticsearch, wrote 100 Mon, 11 Feb 2019 09:29:19 GMT | got 100 objects from source elasticsearch (offset: 100) Mon, 11 Feb 2019 09:29:51 GMT | sent 100 objects to destination elasticsearch, wrote 100 Mon, 11 Feb 2019 09:29:51 GMT | got 100 objects from source elasticsearch (offset: 200) Mon, 11 Feb 2019 09:30:13 GMT | sent 100 objects to destination elasticsearch, wrote 100 Mon, 11 Feb 2019 09:30:13 GMT | got 100 objects from source elasticsearch (offset: 300) Mon, 11 Feb 2019 09:30:39 GMT | sent 100 objects to destination elasticsearch, wrote 100 Mon, 11 Feb 2019 09:30:39 GMT | got 100 objects from source elasticsearch (offset: 400) Mon, 11 Feb 2019 09:31:53 GMT | sent 100 objects to destination elasticsearch, wrote 100 Mon, 11 Feb 2019 09:31:53 GMT | got 100 objects from source elasticsearch (offset: 500) Mon, 11 Feb 2019 09:32:28 GMT | sent 100 objects to destination elasticsearch, wrote 100 Mon, 11 Feb 2019 09:32:28 GMT | got 100 objects from source elasticsearch (offset: 600) Mon, 11 Feb 2019 09:33:24 GMT | sent 100 objects to destination elasticsearch, wrote 100 Mon, 11 Feb 2019 09:33:24 GMT | got 100 objects from source elasticsearch (offset: 700) Mon, 11 Feb 2019 09:34:19 GMT | sent 100 objects to destination elasticsearch, wrote 100 Mon, 11 Feb 2019 09:34:19 GMT | got 7 objects from source elasticsearch (offset: 800) Mon, 11 Feb 2019 09:34:47 GMT | sent 7 objects to destination elasticsearch, wrote 7 Mon, 11 Feb 2019 09:34:47 GMT | got 0 objects from source elasticsearch (offset: 807) Mon, 11 Feb 2019 09:34:47 GMT | Total Writes: 807 Mon, 11 Feb 2019 09:34:47 GMT | dump complete [root@VM_8_24_centos bin]# curl 10.2.100.24:9200/_cat/indices|grep reconciliationonline % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- 0:00:16 --:--:-- 0yellow open reconciliationonline KnZrsU-7TmWWBs-1-dBRrw 5 1 807 0 4.3mb 4.3mb 100 32578 100 32578 0 0 2010 0 0:00:16 0:00:16 --:--:-- 7769
低调大师中文资讯倾力打造互联网数据资讯、行业资源、电子商务、移动互联网、网络营销平台。
持续更新报道IT业界、互联网、市场资讯、驱动更新,是最及时权威的产业资讯及硬件资讯报道平台。
转载内容版权归作者及来源网站所有,本站原创内容转载请注明来源。
- 上一篇
web站点迁移Azure Web Site实战攻略
之前有分享过关于Azure web site的一些基本情况以及使用的方法,除此之外,还介绍了如何将Azure web site从一个区域迁移到另一个区域,没看过的童鞋可以去下边三个链接看看 https://blog.51cto.com/mxyit/2336399 https://blog.51cto.com/mxyit/2336828 https://blog.51cto.com/mxyit/2336938 对于IT人员来说,可能会有这样的一些情况,IT人员最关心的并不只是Azure web site有多好,而是如何将传统的一些web site迁移到Azure web site中,这可能是IT人员更关心的问题。因此,特地写了一片文章,为各位介绍一下如何将传统的网站迁移到Azure web site中,本文中主要举例如何将传统的on premise IIS迁移到Azure web site中。 首先这里准备的是一台VM,VM中部署了一个IIS网站,可以看到下图中的网站访问截图 迁移IIS的方法,这里用到的是微软官方的迁移工具Azure app service migration assi...
- 下一篇
新专栏 | 6位大牛匠心之作,欢迎进群!
新专栏早鸟优惠,与<学专栏,送图书>活动同享→ 新专栏No.1:Ubuntu Server 新专栏No.2:Java EE 新专栏No.3:以太坊开发 新专栏No.4:Azure公有云 新专栏5:Python数据分析 新专栏6:Linux命令行 进群就找我
相关文章
文章评论
共有0条评论来说两句吧...
文章二维码
点击排行
推荐阅读
最新文章
- CentOS8编译安装MySQL8.0.19
- MySQL8.0.19开启GTID主从同步CentOS8
- SpringBoot2全家桶,快速入门学习开发网站教程
- CentOS8,CentOS7,CentOS6编译安装Redis5.0.7
- CentOS7,CentOS8安装Elasticsearch6.8.6
- Red5直播服务器,属于Java语言的直播服务器
- CentOS8安装MyCat,轻松搞定数据库的读写分离、垂直分库、水平分库
- SpringBoot2整合MyBatis,连接MySql数据库做增删改查操作
- CentOS6,CentOS7官方镜像安装Oracle11G
- Jdk安装(Linux,MacOS,Windows),包含三大操作系统的最全安装