centos7部署MongoDB数据库复制集（超详细）-低调大师

centos7部署MongoDB数据库复制集（超详细）

2018-06-13 703

centos7部署MongoDB数据库复制集（超详细）
重点：复制集概述；复制集实现原理；复制集的应用案例；
一、概述：
组成：
Mongodb复制集（副本集replica set）由一组Mongod实例（进程）组成，包含一个Primary节点和多个Secondary节点，Mongodb Driver（客户端）的所有数据都写入Primary，Secondary通过oplog来同步Primary的数据，保证主节点和从节点数据的一致性，复制集在完成主从复制的基础上，通过心跳机制，一旦primary节点出现宕机，则触发选举一个新的主节点，剩下的secondary节点指向新的primary，时间应该在10-30s内完成感知primary节点故障，实现高可用数据库集群；
特点：
主是唯一的，但不是固定的；
由大多数据原则保证数据的一致性；
从库无法写入（默认情况下，不使用驱动连接时，也是不能查询的）；
相对于传统的主从结构，复制集可以自动容灾；

二、原理：
角色（按是否存储数据划分）：
Primary：主节点，由选举产生，负责客户端的写操作，产生oplog日志文件；
Secondary：从节点，负责客户端的读操作，提供数据的备份和故障的切换；
Arbiter：仲裁节点，只参与选举的投票，不会成为primary，也不向Primary同步数据，若部署了一个2个节点的复制集，1个Primary，1个Secondary，任意节点宕机，复制集将不能提供服务了（无法选出Primary），这时可以给复制集添加一个Arbiter节点，即使有节点宕机，仍能选出Primary；
角色（按类型区分）：
Standard（标准）：这种是常规节点，它存储一份完整的数据副本，参与投票选举，有可能成为主节点；
Passive（被动）：存储完整的数据副本，参与投票，不能成为活跃节点。
Arbiter（投票）：仲裁节点只参与投票，不接收复制的数据，也不能成为活跃节点。
注：每个参与节点（非仲裁者）有个优先权（0-1000），优先权（priority）为0则是被动的，不能成为活跃节点，优先权不为0的，按照由大到小选出活跃节点，优先值一样的则看谁的数据比较新；
注：Mongodb 3.0里，复制集成员最多50个，参与Primary选举投票的成员最多7个；

选举：
每个节点通过优先级定义出节点的类型（标准、被动、投票）；
标准节点通过对比自身数据进行选举出peimary节点或者secondary节点；
影响选举的因素：
1.心跳检测：复制集内成员每隔两秒向其他成员发送心跳检测信息，若10秒内无响应，则标记其为不可用；
2.连接：在多个节点中，最少保证两个节点为活跃状态，如果集群中共三个节点，挂掉两个节点，那么剩余的节点无论状态是primary还是处于选举过程中，都会直接被降权为secondary；
触发选举的情况：
1.初始化状态 2.从节点们无法与主节点进行通信 3.主节点辞职
主节点辞职的情况：
1.在接收到replSetStepDown命令后；
2.在现有的环境中，其他secondary节点的数据落后于本身10s内，且拥有更高优先级；
3.当主节点无法与群集中多数节点通信；
注：当主节点辞职后，主节点将关闭自身所有的连接，避免出现客户端在从节点进行写入操作；

三、应用案例：

异常处理：
当Primary宕机时，如果有数据未同步到Secondary，当Primary重新加入时，如果新的Primary上已经发生了写操作，则旧Primary需要回滚部分操作，以保证数据集与新的Primary一致。旧Primary将回滚的数据写到单独的rollback目录下，数据库管理员可根据需要使用mongorestore进行恢复。

环境：
系统主机名 IP地址软件
Centos 7.4 mongodb.benet.com 192.168.100.101 mongodb-linux-x86_64-rhel70-3.6.3.tgz

 实验步骤：
1.安装mongodb；
2.创建并启动四个实例；
3.配置实例；
4.创建复制集并添加节点；
5.模拟Primary节点出现故障，查看角色切换情况；
6.手动切换Primary角色；
7.指定节点的优先级，验证角色重选情况；
8.将标准节点统统停掉，被动节点也不会成为主节点；
9.查询复制集状态以及查看oplog日志文件的大小；

部署用户认证登录（密钥对）的复制集；

 安装mongodb；
下载mongodb软件包；
[root@mongodb ~]# tar zxvf mongodb-linux-x86_64-rhel70-3.6.3.tgz
[root@mongodb ~]# mv mongodb-linux-x86_64-rhel70-3.6.3 /usr/local/mongodb
[root@mongodb ~]# echo "export PATH=/usr/local/mongodb/bin:\$PATH" >>/etc/profile
[root@mongodb ~]# source /etc/profile
[root@mongodb ~]# ulimit -n 25000
[root@mongodb ~]# ulimit -u 25000
[root@mongodb ~]# echo 0 >/proc/sys/vm/zone_reclaim_mode
[root@mongodb ~]# sysctl -w vm.zone_reclaim_mode=0
[root@mongodb ~]# echo never >/sys/kernel/mm/transparent_hugepage/enabled
[root@mongodb ~]# echo never >/sys/kernel/mm/transparent_hugepage/defrag

 创建并启动四个实例；
[root@mongodb ~]# cd /usr/local/mongodb/bin/
[root@mongodb bin]# mkdir {../mongodb1,../mongodb2,../mongodb3,../mongodb4}
[root@mongodb bin]# mkdir ../logs
[root@mongodb bin]# touch ../logs/mongodb{1..4}.log
[root@mongodb bin]# chmod 777 ../logs/mongodb
[root@mongodb bin]# cat <<END >>/usr/local/mongodb/bin/mongodb1.conf
bind_ip=192.168.100.101
port=27017
dbpath=/usr/local/mongodb/mongodb1/
logpath=/usr/local/mongodb/logs/mongodb1.log
logappend=true
fork=true
maxConns=5000
replSet=haha
#replication name
END
[root@mongodb bin]# cat <<END >>/usr/local/mongodb/bin/mongodb2.conf
bind_ip=192.168.100.101
port=27018
dbpath=/usr/local/mongodb/mongodb2/
logpath=/usr/local/mongodb/logs/mongodb2.log
logappend=true
fork=true
maxConns=5000
replSet=haha
END
[root@mongodb bin]# cat <<END >>/usr/local/mongodb/bin/mongodb3.conf
bind_ip=192.168.100.101
port=27019
dbpath=/usr/local/mongodb/mongodb3/
logpath=/usr/local/mongodb/logs/mongodb3.log
logappend=true
fork=true
maxConns=5000
replSet=haha
END
[root@mongodb bin]# cat <<END >>/usr/local/mongodb/bin/mongodb4.conf
bind_ip=192.168.100.101
port=27020
dbpath=/usr/local/mongodb/mongodb4/
logpath=/usr/local/mongodb/logs/mongodb4.log
logappend=true
fork=true
maxConns=5000
replSet=haha
END
[root@mongodb bin]# cd
[root@mongodb ~]# mongod -f /usr/local/mongodb/bin/mongodb1.conf
[root@mongodb ~]# mongod -f /usr/local/mongodb/bin/mongodb2.conf
[root@mongodb ~]# mongod -f /usr/local/mongodb/bin/mongodb3.conf
[root@mongodb ~]# mongod -f /usr/local/mongodb/bin/mongodb4.conf
[root@mongodb ~]# netstat -utpln |grep mongod
tcp 0 0 192.168.100.101:27019 0.0.0.0: LISTEN 2271/mongod
tcp 0 0 192.168.100.101:27020 0.0.0.0: LISTEN 15260/mongod
tcp 0 0 192.168.100.101:27017 0.0.0.0: LISTEN 2440/mongod
tcp 0 0 192.168.100.101:27018 0.0.0.0:* LISTEN 1412/mongod
[root@mongodb ~]# echo -e "/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/mongodb1.conf \n/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/mongodb2.conf\n/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/mongodb3.conf\n/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/mongodb4.conf">>/etc/rc.local
[root@mongodb ~]# chmod +x /etc/rc.local
[root@mongodb ~]# cat <<END >>/etc/init.d/mongodb
#!/bin/bash
INSTANCE=\$1
ACTION=\$2
case "\$ACTION" in
'start')
/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/"\$INSTANCE".conf;;
'stop')
/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/"\$INSTANCE".conf --shutdown;;
'restart')
/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/"\$INSTANCE".conf --shutdown
/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/"\$INSTANCE".conf;;
esac
END
[root@mongodb ~]# chmod +x /etc/init.d/mongodb
[root@mongodb ~]# /etc/init.d/mongodb mongodb1 stop
killing process with pid: 1301
[root@mongodb ~]# /etc/init.d/mongodb mongodb1 start
about to fork child process, waiting until server is ready for connections.
forked process: 1457
child process started successfully, parent exiting
[root@mongodb ~]# mongo --port 27017 --host 192.168.100.101

 配置实例；
[root@mongodb ~]# mongo --port 27017 --host 192.168.100.101

show dbs
2018-04-19T04:34:35.871+0800 E QUERY [thread1] Error: listDatabases failed:{
"ok" : 0,
"errmsg" : "not master and slaveOk=false",
"code" : 13435,
"codeName" : "NotMasterNoSlaveOk"
} :br/>_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:65:1br/>shellHelper.show@src/mongo/shell/utils.js:816:19
shellHelper@src/mongo/shell/utils.js:706:15br/>@(shellhelp2):1:1
rs.status()
{
"info" : "run rs.initiate(...) if not yet done for the set",
"ok" : 0,
"errmsg" : "no replset config has been received",
"code" : 94,
"codeName" : "NotYetInitialized"
}
rs.isMaster() ##查看复制集节点
{
"ismaster" : false,
"secondary" : false,
"info" : "Does not have a valid replica set config",
"isreplicaset" : true,
"maxBsonObjectSize" : 16777216,
"maxMessageSizeBytes" : 48000000,
"maxWriteBatchSize" : 100000,
"localTime" : ISODate("2018-04-18T20:36:31.698Z"),
"minWireVersion" : 0,
"maxWireVersion" : 6,
"readOnly" : false,
"ok" : 1
}
exit
[root@mongodb ~]# mongo --port 27018 --host 192.168.100.101
show dbs
2018-04-19T04:34:56.884+0800 E QUERY [thread1] Error: listDatabases failed:{
"ok" : 0,
"errmsg" : "not master and slaveOk=false",
"code" : 13435,
"codeName" : "NotMasterNoSlaveOk"
} :br/>_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:65:1br/>shellHelper.show@src/mongo/shell/utils.js:816:19
shellHelper@src/mongo/shell/utils.js:706:15br/>@(shellhelp2):1:1
exit

 创建复制集并添加节点；
[root@mongodb ~]# mongo --port 27017 --host 192.168.100.101

cfg={"_id":"haha","members":[{"_id":0,"host":"192.168.100.101:27017"},{"_id":1,"host":"192.168.100.101:27018"},{"_id":2,"host":"192.168.100.101:27019"}]} ##添加节点
{
"_id" : "haha",
"members" : [
{
"_id" : 0,
"host" : "192.168.100.101:27017"
},
{
"_id" : 1,
"host" : "192.168.100.101:27018"
},
{
"_id" : 2,
"host" : "192.168.100.101:27019"
}
]
}
rs.initiate(cfg) ##初始化节点
{
"ok" : 1,
"operationTime" : Timestamp(1524083843, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1524083843, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
haha:OTHER>
haha:PRIMARY>
haha:PRIMARY> rs.status()
{
"set" : "haha",
"date" : ISODate("2018-04-18T20:37:54.095Z"),
"myState" : 1,
"term" : NumberLong(1),
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1524083855, 5),
"t" : NumberLong(1)
},
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1524083855, 5),
"t" : NumberLong(1)
},
"appliedOpTime" : {
"ts" : Timestamp(1524083855, 5),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1524083855, 5),
"t" : NumberLong(1)
}
},
"members" : [
{
"_id" : 0,
"name" : "192.168.100.101:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 329,
"optime" : {
"ts" : Timestamp(1524083855, 5),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-18T20:37:35Z"),
"infoMessage" : "could not find member to sync from",
"electionTime" : Timestamp(1524083854, 1),
"electionDate" : ISODate("2018-04-18T20:37:34Z"),
"configVersion" : 1,
"self" : true
},
{
"_id" : 1,
"name" : "192.168.100.101:27018",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 30,
"optime" : {
"ts" : Timestamp(1524083855, 5),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1524083855, 5),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-18T20:37:35Z"),
"optimeDurableDate" : ISODate("2018-04-18T20:37:35Z"),
"lastHeartbeat" : ISODate("2018-04-18T20:37:54.043Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T20:37:52.499Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "192.168.100.101:27017",
"configVersion" : 1
},
{
"_id" : 2,
"name" : "192.168.100.101:27019",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 30,
"optime" : {
"ts" : Timestamp(1524083855, 5),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1524083855, 5),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-18T20:37:35Z"),
"optimeDurableDate" : ISODate("2018-04-18T20:37:35Z"),
"lastHeartbeat" : ISODate("2018-04-18T20:37:54.043Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T20:37:52.500Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "192.168.100.101:27017",
"configVersion" : 1
}
],
"ok" : 1,
"operationTime" : Timestamp(1524083855, 5),
"$clusterTime" : {
"clusterTime" : Timestamp(1524083855, 5),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
haha:PRIMARY> rs.addArb("192.168.100.101:27020") ##添加仲裁节点
{
"ok" : 1,
"operationTime" : Timestamp(1524083905, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1524083905, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
haha:PRIMARY> rs.status()
{
"set" : "haha",
"date" : ISODate("2018-04-18T20:38:41.468Z"),
"myState" : 1,
"term" : NumberLong(1),
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1524083915, 1),
"t" : NumberLong(1)
},
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1524083915, 1),
"t" : NumberLong(1)
},
"appliedOpTime" : {
"ts" : Timestamp(1524083915, 1),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1524083915, 1),
"t" : NumberLong(1)
}
},
"members" : [
{
"_id" : 0,
"name" : "192.168.100.101:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 376,
"optime" : {
"ts" : Timestamp(1524083915, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-18T20:38:35Z"),
"infoMessage" : "could not find member to sync from",
"electionTime" : Timestamp(1524083854, 1),
"electionDate" : ISODate("2018-04-18T20:37:34Z"),
"configVersion" : 2,
"self" : true
},
{
"_id" : 1,
"name" : "192.168.100.101:27018",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 78,
"optime" : {
"ts" : Timestamp(1524083905, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1524083905, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-18T20:38:25Z"),
"optimeDurableDate" : ISODate("2018-04-18T20:38:25Z"),
"lastHeartbeat" : ISODate("2018-04-18T20:38:41.134Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T20:38:40.134Z"),
"pingMs" : NumberLong(0),
"configVersion" : 2
},
{
"_id" : 2,
"name" : "192.168.100.101:27019",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 78,
"optime" : {
"ts" : Timestamp(1524083905, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1524083905, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-18T20:38:25Z"),
"optimeDurableDate" : ISODate("2018-04-18T20:38:25Z"),
"lastHeartbeat" : ISODate("2018-04-18T20:38:41.134Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T20:38:40.136Z"),
"pingMs" : NumberLong(0),
"configVersion" : 2
},
{
"_id" : 3,
"name" : "192.168.100.101:27020",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 16,
"lastHeartbeat" : ISODate("2018-04-18T20:38:41.137Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T20:38:40.140Z"),
"pingMs" : NumberLong(0),
"configVersion" : 2
}
],
"ok" : 1,
"operationTime" : Timestamp(1524083915, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1524083915, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
注：rs.add()和rs.remove()命令分别用于添加和删除标准节点
haha:PRIMARY> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
haha:PRIMARY> use cloud
switched to db cloud
haha:PRIMARY> db.users.insert({"id":"1","name":"xiaoming"})
WriteResult({ "nInserted" : 1 })
haha:PRIMARY> db.users.find()
{ "_id" : ObjectId("5ad7b245f6308759d4605b5c"), "id" : "1", "name" : "xiaoming" }
haha:PRIMARY> show dbs
admin 0.000GB
cloud 0.000GB
config 0.000GB
local 0.000GB
haha:PRIMARY> exit
[root@mongodb ~]# mongo --port 27018 --host 192.168.100.101
haha:SECONDARY> rs.status()
{
"set" : "haha",
"date" : ISODate("2018-04-18T20:50:50.975Z"),
"myState" : 2,
"term" : NumberLong(1),
"syncingTo" : "192.168.100.101:27017",
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1524084645, 1),
"t" : NumberLong(1)
},
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1524084645, 1),
"t" : NumberLong(1)
},
"appliedOpTime" : {
"ts" : Timestamp(1524084645, 1),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1524084645, 1),
"t" : NumberLong(1)
}
},
"members" : [
{
"_id" : 0,
"name" : "192.168.100.101:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 805,
"optime" : {
"ts" : Timestamp(1524084645, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1524084645, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-18T20:50:45Z"),
"optimeDurableDate" : ISODate("2018-04-18T20:50:45Z"),
"lastHeartbeat" : ISODate("2018-04-18T20:50:49.966Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T20:50:49.997Z"),
"pingMs" : NumberLong(0),
"electionTime" : Timestamp(1524083854, 1),
"electionDate" : ISODate("2018-04-18T20:37:34Z"),
"configVersion" : 2
},
{
"_id" : 1,
"name" : "192.168.100.101:27018",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 1497,
"optime" : {
"ts" : Timestamp(1524084645, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-18T20:50:45Z"),
"syncingTo" : "192.168.100.101:27017",
"configVersion" : 2,
"self" : true
},
{
"_id" : 2,
"name" : "192.168.100.101:27019",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 805,
"optime" : {
"ts" : Timestamp(1524084645, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1524084645, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-18T20:50:45Z"),
"optimeDurableDate" : ISODate("2018-04-18T20:50:45Z"),
"lastHeartbeat" : ISODate("2018-04-18T20:50:49.811Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T20:50:49.965Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "192.168.100.101:27017",
"configVersion" : 2
},
{
"_id" : 3,
"name" : "192.168.100.101:27020",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 745,
"lastHeartbeat" : ISODate("2018-04-18T20:50:49.966Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T20:50:50.378Z"),
"pingMs" : NumberLong(0),
"configVersion" : 2
}
],
"ok" : 1,
"operationTime" : Timestamp(1524084645, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1524084645, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
haha:SECONDARY> show dbs ##secondary节点默认无法读取，可以通过以下方式或者驱动方式实现
2018-04-19T04:52:42.813+0800 E QUERY [thread1] Error: listDatabases failed:{
"operationTime" : Timestamp(1524084755, 1),
"ok" : 0,
"errmsg" : "not master and slaveOk=false",
"code" : 13435,
"codeName" : "NotMasterNoSlaveOk",
"$clusterTime" : {
"clusterTime" : Timestamp(1524084755, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
} :br/>_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:65:1br/>shellHelper.show@src/mongo/shell/utils.js:816:19
shellHelper@src/mongo/shell/utils.js:706:15br/>@(shellhelp2):1:1
haha:SECONDARY> db.getMongo().setSlaveOk();
haha:SECONDARY> show dbs
admin 0.000GB
cloud 0.000GB
config 0.000GB
local 0.000GB
haha:SECONDARY> use cloud
switched to db cloud
haha:SECONDARY> db.users.insert({"id":"2","name":"xiaohong"}) ##secondary节点无法写入
WriteResult({ "writeError" : { "code" : 10107, "errmsg" : "not master" } })
haha:SECONDARY> db.users.find()
{ "_id" : ObjectId("5ad7b245f6308759d4605b5c"), "id" : "1", "name" : "xiaoming" }
haha:SECONDARY> exit

 模拟Primary节点出现故障，查看角色切换情况；
[root@mongodb ~]# mongod -f /usr/local/mongodb/bin/mongodb1.conf --shutdown
killing process with pid: 1457
[root@mongodb ~]# mongo --port 27018 --host 192.168.100.101
haha:SECONDARY> rs.status()
{
"set" : "haha",
"date" : ISODate("2018-04-18T21:16:22.623Z"),
"myState" : 2,
"term" : NumberLong(2),
"syncingTo" : "192.168.100.101:27019",
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1524086155, 1),
"t" : NumberLong(1)
},
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1524086155, 1),
"t" : NumberLong(1)
},
"appliedOpTime" : {
"ts" : Timestamp(1524086174, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1524086174, 1),
"t" : NumberLong(2)
}
},
"members" : [
{
"_id" : 0,
"name" : "192.168.100.101:27017",
"health" : 0,
"state" : 8,
"stateStr" : "(not reachable/healthy)",
"uptime" : 0,
"optime" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDurable" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2018-04-18T21:16:19.030Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T21:16:03.344Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "Connection refused",
"configVersion" : -1
},
{
"_id" : 1,
"name" : "192.168.100.101:27018",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 3029,
"optime" : {
"ts" : Timestamp(1524086174, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2018-04-18T21:16:14Z"),
"syncingTo" : "192.168.100.101:27019",
"infoMessage" : "syncing from: 192.168.100.101:27019",
"configVersion" : 2,
"self" : true
},
{
"_id" : 2,
"name" : "192.168.100.101:27019",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 2337,
"optime" : {
"ts" : Timestamp(1524086174, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1524086174, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2018-04-18T21:16:14Z"),
"optimeDurableDate" : ISODate("2018-04-18T21:16:14Z"),
"lastHeartbeat" : ISODate("2018-04-18T21:16:19.030Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T21:16:21.575Z"),
"pingMs" : NumberLong(0),
"electionTime" : Timestamp(1524086173, 1),
"electionDate" : ISODate("2018-04-18T21:16:13Z"),
"configVersion" : 2
},
{
"_id" : 3,
"name" : "192.168.100.101:27020",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 2277,
"lastHeartbeat" : ISODate("2018-04-18T21:16:19.030Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T21:16:20.894Z"),
"pingMs" : NumberLong(0),
"configVersion" : 2
}
],
"ok" : 1,
"operationTime" : Timestamp(1524086174, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1524086174, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
haha:SECONDARY> exit
[root@mongodb ~]# mongo --port 27019 --host 192.168.100.101
haha:PRIMARY> exit
[root@mongodb ~]# mongod -f /usr/local/mongodb/bin/mongodb1.conf

 手动切换Primary角色；
[root@mongodb ~]# mongo --port 27019 --host 192.168.100.101
haha:PRIMARY> rs.stepDown(60,30)
haha:SECONDARY> exit
[root@mongodb ~]# mongo --port 27018 --host 192.168.100.101
haha:PRIMARY> exit

 指定节点的优先级，验证角色重选情况；
[root@mongodb ~]# mongo --port 27018 --host 192.168.100.101
haha:PRIMARY> cfg={"_id":"haha","members":[{"_id":0,"host":"192.168.100.101:27017","priority":100},{"_id":1,"host":"192.168.100.101:27018","priority":100},{"_id":2,"host":"192.168.100.101:27019","priority":10}]}
{
"_id" : "haha",
"members" : [
{
"_id" : 0,
"host" : "192.168.100.101:27017",
"priority" : 100
},
{
"_id" : 1,
"host" : "192.168.100.101:27018",
"priority" : 100
},
{
"_id" : 2,
"host" : "192.168.100.101:27019",
"priority" : 10
}
]
}
haha:PRIMARY> rs.reconfig(cfg)
{
"ok" : 1,
"operationTime" : Timestamp(1524086716, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1524086716, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
haha:PRIMARY> rs.isMaster()
{
"hosts" : [
"192.168.100.101:27017",
"192.168.100.101:27018",
"192.168.100.101:27019"
],
"setName" : "haha",
"setVersion" : 3,
"ismaster" : true,
"secondary" : false,
"primary" : "192.168.100.101:27018",
"me" : "192.168.100.101:27018",
"electionId" : ObjectId("7fffffff0000000000000003"),
"lastWrite" : {
"opTime" : {
"ts" : Timestamp(1524086733, 1),
"t" : NumberLong(3)
},
"lastWriteDate" : ISODate("2018-04-18T21:25:33Z"),
"majorityOpTime" : {
"ts" : Timestamp(1524086733, 1),
"t" : NumberLong(3)
},
"majorityWriteDate" : ISODate("2018-04-18T21:25:33Z")
},
"maxBsonObjectSize" : 16777216,
"maxMessageSizeBytes" : 48000000,
"maxWriteBatchSize" : 100000,
"localTime" : ISODate("2018-04-18T21:25:42.122Z"),
"logicalSessionTimeoutMinutes" : 30,
"minWireVersion" : 0,
"maxWireVersion" : 6,
"readOnly" : false,
"ok" : 1,
"operationTime" : Timestamp(1524086733, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1524086733, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
haha:PRIMARY> exit
[root@mongodb ~]# mongod -f /usr/local/mongodb/bin/mongodb2.conf --shutdown
[root@mongodb ~]# mongo --port 27017 --host 192.168.100.101
haha:PRIMARY> exit

 将标准节点统统停掉，被动节点也不会成为主节点；
[root@mongodb ~]# mongod -f /usr/local/mongodb/bin/mongodb1.conf --shutdown
[root@mongodb ~]# mongo --port 27019 --host 192.168.100.101
haha:SECONDARY> rs.status()
{
"set" : "haha",
"date" : ISODate("2018-04-18T21:29:42.973Z"),
"myState" : 2,
"term" : NumberLong(5),
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1524086961, 1),
"t" : NumberLong(5)
},
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1524086961, 1),
"t" : NumberLong(5)
},
"appliedOpTime" : {
"ts" : Timestamp(1524086961, 1),
"t" : NumberLong(5)
},
"durableOpTime" : {
"ts" : Timestamp(1524086961, 1),
"t" : NumberLong(5)
}
},
"members" : [
{
"_id" : 0,
"name" : "192.168.100.101:27017",
"health" : 0,
"state" : 8,
"stateStr" : "(not reachable/healthy)",
"uptime" : 0,
"optime" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDurable" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2018-04-18T21:29:38.566Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T21:29:26.484Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "Connection refused",
"configVersion" : -1
},
{
"_id" : 1,
"name" : "192.168.100.101:27018",
"health" : 0,
"state" : 8,
"stateStr" : "(not reachable/healthy)",
"uptime" : 0,
"optime" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDurable" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2018-04-18T21:29:38.566Z"),
"lastHeartbeatRecv" : ISODate("2018-04-18T21:26:42.249Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "Connection refused",
"configVersion" : -1
},
{
"_id" : 2,
"name" : "192.168.100.101:27019",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 3826,
"optime" : {
"ts" : Timestamp(1524086961, 1),
"t" : NumberLong(5)
},
"optimeDate" : ISODate("2018-04-18T21:29:21Z"),
"infoMessage" : "could not find member to sync from",
"configVersion" : 3,
"self" : true
}
],
"ok" : 1,
"operationTime" : Timestamp(1524086961, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1524086961, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
haha:SECONDARY> exit
[root@mongodb ~]# mongod -f /usr/local/mongodb/bin/mongodb1.conf
[root@mongodb ~]# mongod -f /usr/local/mongodb/bin/mongodb2.conf
[root@mongodb ~]# netstat -utpln |grep mongod
tcp 0 0 192.168.100.101:27019 0.0.0.0: LISTEN 1357/mongod
tcp 0 0 192.168.100.101:27020 0.0.0.0: LISTEN 1385/mongod
tcp 0 0 192.168.100.101:27017 0.0.0.0: LISTEN 14363/mongod
tcp 0 0 192.168.100.101:27018 0.0.0.0: LISTEN 14442/mongod

 查询复制集状态以及查看oplog日志文件的大小；
[root@mongodb ~]# mongo --port 27018 --host 192.168.100.101
haha:PRIMARY> use local
haha:PRIMARY> rs.printSlaveReplicationInfo() ##查看节点信息
source: 192.168.100.101:27017
syncedTo: Thu Apr 19 2018 05:34:37 GMT+0800 (CST)
0 secs (0 hrs) behind the primary
source: 192.168.100.101:27019
syncedTo: Thu Apr 19 2018 05:34:37 GMT+0800 (CST)
0 secs (0 hrs) behind the primary
haha:PRIMARY> rs.printReplicationInfo()
configured oplog size: 990MB
log length start to end: 3464secs (0.96hrs)
oplog first event time: Thu Apr 19 2018 04:37:23 GMT+0800 (CST)
oplog last event time: Thu Apr 19 2018 05:35:07 GMT+0800 (CST)
now: Thu Apr 19 2018 05:35:16 GMT+0800 (CST)
haha:PRIMARY> db.oplog.rs.stats()
{
"ns" : "local.oplog.rs",
"size" : 50356,
"count" : 450,
"avgObjSize" : 111,
"storageSize" : 61440,
"capped" : true,
"max" : -1,
"maxSize" : 1038090240, ##单位是KB
...

 部署用户认证登录（密钥对）的复制集；
[root@mongodb ~]# mongo --port 27018 --host 192.168.100.101
haha:PRIMARY> use admin
switched to db admin
haha:PRIMARY> db.createUser({"user":"root","pwd":"123","roles":["root"]})
Successfully added user: { "user" : "root", "roles" : [ "root" ] }
haha:PRIMARY> exit
[root@mongodb ~]# echo -e "clusterAuthMode=keyFile\nkeyFile=/usr/local/mongodb/bin/cloudkey1" >>/usr/local/mongodb/bin/mongodb1.conf
[root@mongodb ~]# echo -e "clusterAuthMode=keyFile\nkeyFile=/usr/local/mongodb/bin/cloudkey2" >>/usr/local/mongodb/bin/mongodb2.conf
[root@mongodb ~]# echo -e "clusterAuthMode=keyFile\nkeyFile=/usr/local/mongodb/bin/cloudkey3" >>/usr/local/mongodb/bin/mongodb3.conf
[root@mongodb ~]# echo "haha key" >/usr/local/mongodb/bin/cloudkey1
[root@mongodb ~]# echo "haha key" >/usr/local/mongodb/bin/cloudkey2
[root@mongodb ~]# echo "haha key" >/usr/local/mongodb/bin/cloudkey3
[root@mongodb ~]# chmod 600 /usr/local/mongodb/bin/cloudkey*
[root@mongodb ~]# /etc/init.d/mongodb mongodb1 restart
[root@mongodb ~]# /etc/init.d/mongodb mongodb2 restart
[root@mongodb ~]# /etc/init.d/mongodb mongodb3 restart
[root@mongodb ~]# mongo --port 27018 --host 192.168.100.101
haha:PRIMARY> show dbs
2018-04-19T12:02:57.720+0800 E QUERY [thread1] Error: listDatabases failed:{
"operationTime" : Timestamp(1524110569, 1),
"ok" : 0,
"errmsg" : "not authorized on admin to execute command { listDatabases: 1.0, $clusterTime: { clusterTime: Timestamp(1524110569, 1), signature: { hash: BinData(0, 85A5B587BFFE4B2FB534789EA87032CB263D8121), keyId: 6545890313586606081 } }, $db: \"admin\" }",
...
haha:PRIMARY> use admin
switched to db admin
haha:PRIMARY> db.auth("root","123")
1
haha:PRIMARY> show dbs
admin 0.000GB
cloud 0.000GB
config 0.000GB
local 0.000GB
haha:PRIMARY> exit

微信关注我们

原文链接：https://blog.51cto.com/11638832/2128933

转载内容版权归作者及来源网站所有！

低调大师中文资讯倾力打造互联网数据资讯、行业资源、电子商务、移动互联网、网络营销平台。持续更新报道IT业界、互联网、市场资讯、驱动更新,是最及时权威的产业资讯及硬件资讯报道平台。

Mariadb半同步复制，主从复制与双主复制

知识背景一.mariadb复制相关知识 1.mariadb的来源 mysql数据库目前在数据库市场中所占份额仅仅次于oracle数据库，同时也是开源数据库的龙头老大。由于mysql数据库的开源性和高性能,同时还是免费的(不过被oracle收购后不容乐观)，因此，mysql之父又开启了一个mysql的新分支mariadb，mariadb相对mysql而言版更新快，新特性实装速度迅速，mysql官方版本的优点mariadb也会很快的加以吸收。 2.mariadb复制的流程 mariadb数据库支持同步复制，半同步复制和异步复制，在复制的过程中一个服务器充当主服务器，而一个或者多个服务器做为从服务器。主服务器在进行修改时，会将更新写入二进制日志文件，并且维护文件的一个索引来跟踪日志循环。日志的记录会发送到从服务器进行数据修改。当从服务器到主服务器上获取二进制日志时，它会通知主服务器从日志中读取最后一次成功的更新的位置，从服务器则会接收从该位置之后发生的全部操作。 3.同步复制，异步复制，半同步复制同步复制：Master提交事务，直到事务在所有的Slave都...

2018-06-12

778

分布式之数据库和缓存双写一致性方案解析

引言为什么写这篇文章？首先，缓存由于其高并发和高性能的特性，已经在项目中被广泛使用。在读取缓存方面，大家没啥疑问，都是按照下图的流程来进行业务操作。但是在更新缓存方面，对于更新完数据库，是更新缓存呢，还是删除缓存。又或者是先删除缓存，再更新数据库，其实大家存在很大的争议。目前没有一篇全面的博客，对这几种方案进行解析。于是博主战战兢兢，顶着被大家喷的风险，写了这篇文章。文章结构本文由以下三个部分组成1、讲解缓存更新策略2、对每种策略进行缺点分析3、针对缺点给出改进方案正文先做一个说明，从理论上来说，给缓存设置过期时间，是保证最终一致性的解决方案。这种方案下，我们可以对存入缓存的数据设置过期时间，所有的写操作以数据库为准，对缓存操作只是尽最大努力即可。也就是说如果数据库写成功，缓存更新失败，那么只要到达过期时间，则后面的读请求自然会从数据库中读取新值然后回填缓存。因此，接下来讨论的思路不依赖于给缓存设置过期时间这个方案。在这里，我们讨论三种更新策略：先更新数据库，再更新缓存先删除缓存，再更新数据库先更新数据库，再删除缓存应该没人问我，为什么没有先更新缓存，再更新数据库这...

2018-06-15

685

发表评论

资源下载

更多资源

Mario

马里奥是站在游戏界顶峰的超人气多面角色。马里奥靠吃蘑菇成长，特征是大鼻子、头戴帽子、身穿背带裤，还留着胡子。与他的双胞胎兄弟路易基一起，长年担任任天堂的招牌角色。

Spring

Spring框架（Spring Framework）是由Rod Johnson于2002年提出的开源Java企业级应用框架，旨在通过使用JavaBean替代传统EJB实现方式降低企业级编程开发的复杂性。该框架基于简单性、可测试性和松耦合性设计理念，提供核心容器、应用上下文、数据访问集成等模块，支持整合Hibernate、Struts等第三方框架，其适用范围不仅限于服务器端开发，绝大多数Java应用均可从中受益。

Rocky Linux

Rocky Linux（中文名：洛基）是由Gregory Kurtzer于2020年12月发起的企业级Linux发行版，作为CentOS稳定版停止维护后与RHEL（Red Hat Enterprise Linux）完全兼容的开源替代方案，由社区拥有并管理，支持x86_64、aarch64等架构。其通过重新编译RHEL源代码提供长期稳定性，采用模块化包装和SELinux安全架构，默认包含GNOME桌面环境及XFS文件系统，支持十年生命周期更新。

Sublime Text

Sublime Text具有漂亮的用户界面和强大的功能，例如代码缩略图，Python的插件，代码段等。还可自定义键绑定，菜单和工具栏。Sublime Text 的主要功能包括：拼写检查，书签，完整的 Python API ， Goto 功能，即时项目切换，多选择，多窗口等等。Sublime Text 是一个跨平台的编辑器，同时支持Windows、Linux、Mac OS X等操作系统。