该方法应用面比较窄,
适用于 : 一主 一备 一投票节点,数据库较大,oplog 比较小,备库需要修复而且主库不能停机的情况.
该方法仅限于学习测试,线上环境慎用.
集群结构：
opsdba-vbj01-1:27018 ARBITE
opsdba-vbj01-1:27019 PRIMARY
opsdba-vbj01-1:27016 SECONDARY
模拟opsdba-vbj01-1:27016 crash ,使用一致性备份搭建备库.
1.在主库,自建role,赋予restore oplog权限.
use admin
db.runCommand({ createRole: "restoreoplog",
privileges:
[
{ resource: { anyResource: true }, actions: [ "anyAction" ] }
],
roles:
[]
});
db.grantRolesToUser( "root", ["restoreoplog"] );
2.主库一致性备份.
[root@opsdba-vbj01-1 dump]# mongodump -uroot -proot123 --port=27019 --oplog --authenticationDatabase=admin -o all_backup
2016-10-11T15:45:58.019+0800 writing admin.system.users to
2016-10-11T15:45:58.019+0800 done dumping admin.system.users (1 document)
2016-10-11T15:45:58.019+0800 writing admin.system.roles to
2016-10-11T15:45:58.020+0800 done dumping admin.system.roles (1 document)
2016-10-11T15:45:58.020+0800 writing admin.system.version to
2016-10-11T15:45:58.020+0800 done dumping admin.system.version (1 document)
2016-10-11T15:45:58.021+0800 writing test.testData to
2016-10-11T15:45:58.021+0800 writing test.tab to
2016-10-11T15:45:58.190+0800 done dumping test.tab (34056 documents)
2016-10-11T15:46:01.022+0800 [###########.............] test.testData 451185/909000 (49.6%)
2016-10-11T15:46:04.022+0800 [####################....] test.testData 774771/909000 (85.2%)
2016-10-11T15:46:05.024+0800 [########################] test.testData 913877/909000 (100.5%)
2016-10-11T15:46:05.024+0800 done dumping test.testData (913877 documents)
2016-10-11T15:46:05.025+0800 writing captured oplog to
2016-10-11T15:46:05.781+0800 dumped 6470 oplog entries
3.把opsdba-vbj01-1:27016 从SECONDARY 转为单节点空库.
/data/mongodb/mongodb/bin/mongod -f /data/mongodb/mongodb1/conf/mongod.cnf --shutdown
清空data目录,单节点启动:
修改port = 27015,
注释掉replSet
/data/mongodb/mongodb/bin/mongod -f /data/mongodb/mongodb1/conf/mongod.cnf
创建管理员用户:
[root@opsdba-vbj01-1 mongodb1]# mongo --port=27015 admin
use admin
db.createUser(
{
user: "root",
pwd: "root123",
roles:
[
{
role: "root",
db: "admin"
}
]
}
);
自建role,赋予restore oplog权限:
[root@opsdba-vbj01-1 mongodb1]# mongo -uroot -proot123 --port=27015 admin
use admin
db.runCommand({ createRole: "restoreoplog",
privileges:
[
{ resource: { anyResource: true }, actions: [ "anyAction" ] }
],
roles:
[]
});
db.grantRolesToUser( "root", ["restoreoplog"] );
4.导入
[root@opsdba-vbj01-1 dump]# mongorestore -uroot -proot123 --port=27015 --authenticationDatabase=admin --oplogReplay --dir=all_backup
2016-10-11T15:59:48.505+0800 building a list of dbs and collections to restore from all_backup dir
2016-10-11T15:59:48.507+0800 reading metadata for test.testData from all_backup/test/testData.metadata.json
2016-10-11T15:59:48.508+0800 reading metadata for test.tab from all_backup/test/tab.metadata.json
2016-10-11T15:59:48.556+0800 restoring test.tab from all_backup/test/tab.bson
2016-10-11T15:59:48.604+0800 restoring test.testData from all_backup/test/testData.bson
2016-10-11T15:59:49.147+0800 restoring indexes for collection test.tab from metadata
2016-10-11T15:59:49.147+0800 finished restoring test.tab (34056 documents)
2016-10-11T15:59:51.507+0800 [####....................] test.testData 18.5 MB/94.1 MB (19.7%)
2016-10-11T15:59:54.507+0800 [##########..............] test.testData 39.8 MB/94.1 MB (42.3%)
2016-10-11T15:59:57.507+0800 [###############.........] test.testData 59.7 MB/94.1 MB (63.5%)
2016-10-11T16:00:00.507+0800 [####################....] test.testData 80.1 MB/94.1 MB (85.1%)
2016-10-11T16:00:03.288+0800 [########################] test.testData 94.1 MB/94.1 MB (100.0%)
2016-10-11T16:00:03.288+0800 restoring indexes for collection test.testData from metadata
2016-10-11T16:00:03.289+0800 finished restoring test.testData (913877 documents)
2016-10-11T16:00:03.289+0800 restoring users from all_backup/admin/system.users.bson
2016-10-11T16:00:03.416+0800 restoring roles from all_backup/admin/system.roles.bson
2016-10-11T16:00:03.466+0800 replaying oplog
2016-10-11T16:00:03.808+0800 done
5.获取最后一个oplog的时间戳
[root@opsdba-vbj01-1 dump]# cd all_backup/
[root@opsdba-vbj01-1 all_backup]# bsondump oplog.bson >oplog.txt
[root@opsdba-vbj01-1 all_backup]# tail -1 oplog.txt
{"ts":{"$timestamp":{"t":1476171965,"i":805}},"t":{"$numberLong":"2"},"h":{"$numberLong":"6906152948185446623"},"v":2,"op":"i","ns":"test.testData","o":{"_id":{"$oid":"57fc98bddfa99af76706f721"},"x":6470.0,"name":"MACLEAN","name1":"MACLEAN","name2":"MACLEAN","name3":"MACLEAN"}}
6.初始化local库的相关表
[root@opsdba-vbj01-1 mongodb1]# mongo -uroot -proot123 --port=27015 admin
use local
db.runCommand( { create: "oplog.rs", capped: true, size: (1* 1024 * 1024 * 1024) } );
#数据来自oplog.txt
db.oplog.rs.save({"ts" : Timestamp(1476171965, 805),"h" : NumberLong("6906152948185446623")});
db.db.replset.minvalid.save({"ts" : Timestamp(1476171965,805), "t" : NumberLong(2)});
#数据来自主库的数据查询
db.replset.election.save({ "_id" : ObjectId("57fc5ea0cfa6486e03e975d0"), "term" : NumberLong(2), "candidateIndex" : NumberLong(2) });
db.system.replset.save({ "_id" : "myrelset", "version" : 5, "protocolVersion" : NumberLong(1), "members" : [ { "_id" : 1, "host" : "opsdba-vbj01-1:27018", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : { }, "slaveDelay" : NumberLong(0), "votes" : 1 }, { "_id" : 2, "host" : "opsdba-vbj01-1:27019", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : { }, "slaveDelay" : NumberLong(0), "votes" : 1 }, { "_id" : 3, "host" : "opsdba-vbj01-1:27016", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : { }, "slaveDelay" : NumberLong(0), "votes" : 1 } ], "settings" : { "chainingAllowed" : true, "heartbeatIntervalMillis" : 2000, "heartbeatTimeoutSecs" : 10, "electionTimeoutMillis" : 10000, "getLastErrorModes" : { }, "getLastErrorDefaults" : { "w" : 1, "wtimeout" : 0 }, "replicaSetId" : ObjectId("57bfdcdcd40cbe4bf173396a") } });
7.重启
use admin
db.shutdownServer();
修改为原始值.
port=27016
replSet 取消注释
启动
/data/mongodb/mongodb/bin/mongod -f /data/mongodb/mongodb1/conf/mongod.cnf
8.检测:
比对行数.
db.collection.count()
9.启动日志
2016-10-11T16:42:51.520+0800 I CONTROL [initandlisten] MongoDB starting : pid=633 port=27016 dbpath=/data/mongodb/mongodb1/data 64-bit host=opsdba-vbj01-1
2016-10-11T16:42:51.520+0800 I CONTROL [initandlisten] db version v3.2.8
2016-10-11T16:42:51.520+0800 I CONTROL [initandlisten] git version: ed70e33130c977bda0024c125b56d159573dbaf0
2016-10-11T16:42:51.520+0800 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] allocator: tcmalloc
2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] modules: none
2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] build environment:
2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] distmod: rhel62
2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] distarch: x86_64
2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] target_arch: x86_64
2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] options: { config: "/data/mongodb/mongodb1/conf/mongod.cnf", net: { http: { enabled: false }, maxIncomingConnections: 3000, port: 27016, unixDomainSocket: { pathPrefix: "/data/mongodb/mongodb1/data" } }, operationProfiling: { mode: "slowOp", slowOpThresholdMs: 800 }, processManagement: { fork: true, pidFilePath: "/data/mongodb/mongodb1/data/mongod.pid" }, replication: { replSet: "myrelset" }, security: { clusterAuthMode: "keyFile", keyFile: "/data/mongodb/mongodb1/conf/myrelset.keyfile" }, storage: { dbPath: "/data/mongodb/mongodb1/data", directoryPerDB: true, engine: "wiredTiger", journal: { commitIntervalMs: 300, enabled: true }, mmapv1: { nsSize: 32 }, repairPath: "/data/mongodb/mongodb1/data", syncPeriodSecs: 60.0, wiredTiger: { engineConfig: { cacheSizeGB: 1 } } }, systemLog: { destination: "file", path: "/data/mongodb/mongodb1/log/mongod.log", quiet: true, timeStampFormat: "iso8601-local" } }
2016-10-11T16:42:51.521+0800 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-10-11T16:42:51.944+0800 I STORAGE [initandlisten] Starting WiredTigerRecordStoreThread local.oplog.rs
2016-10-11T16:42:51.944+0800 I STORAGE [initandlisten] The size storer reports that the oplog contains 1 records totaling to 45 bytes
2016-10-11T16:42:51.944+0800 I STORAGE [initandlisten] Scanning the oplog to determine where to place markers for truncation
2016-10-11T16:42:51.982+0800 W STORAGE [initandlisten] Detected configuration for non-active storage engine mmapv1 when current storage engine is wiredTiger
2016-10-11T16:42:51.982+0800 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2016-10-11T16:42:51.982+0800 I CONTROL [initandlisten]
2016-10-11T16:42:51.982+0800 I CONTROL [initandlisten]
2016-10-11T16:42:51.982+0800 I CONTROL [initandlisten] ** WARNING: soft rlimits too low. rlimits set to 1024 processes, 65535 files. Number of processes should be at least 32767.5 : 0.5 times number of files.
2016-10-11T16:42:52.030+0800 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/data/mongodb/mongodb1/data/diagnostic.data'
2016-10-11T16:42:52.030+0800 I NETWORK [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
2016-10-11T16:42:52.031+0800 I NETWORK [initandlisten] waiting for connections on port 27016
2016-10-11T16:42:52.066+0800 I REPL [ReplicationExecutor] New replica set config in use: { _id: "myrelset", version: 5, protocolVersion: 1, members: [ { _id: 1, host: "opsdba-vbj01-1:27018", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 2, host: "opsdba-vbj01-1:27019", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 3, host: "opsdba-vbj01-1:27016", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('57bfdcdcd40cbe4bf173396a') } }
2016-10-11T16:42:52.067+0800 I REPL [ReplicationExecutor] This node is opsdba-vbj01-1:27016 in the config
2016-10-11T16:42:52.067+0800 I REPL [ReplicationExecutor] transition to STARTUP2
2016-10-11T16:42:52.067+0800 I REPL [ReplicationExecutor] Starting replication applier threads
2016-10-11T16:42:52.067+0800 I REPL [ReplicationExecutor] transition to RECOVERING
2016-10-11T16:42:52.072+0800 I REPL [ReplicationExecutor] transition to SECONDARY
2016-10-11T16:42:52.102+0800 I ASIO [NetworkInterfaceASIO-Replication-0] Successfully connected to opsdba-vbj01-1:27018
2016-10-11T16:42:52.102+0800 I ASIO [NetworkInterfaceASIO-Replication-0] Successfully connected to opsdba-vbj01-1:27019
2016-10-11T16:42:52.102+0800 I REPL [ReplicationExecutor] Member opsdba-vbj01-1:27018 is now in state SECONDARY
2016-10-11T16:42:52.103+0800 I REPL [ReplicationExecutor] Member opsdba-vbj01-1:27019 is now in state PRIMARY
2016-10-11T16:42:58.069+0800 I REPL [ReplicationExecutor] syncing from: opsdba-vbj01-1:27018
2016-10-11T16:42:58.086+0800 I REPL [SyncSourceFeedback] setting syncSourceFeedback to opsdba-vbj01-1:27018
2016-10-11T16:42:58.104+0800 I ASIO [NetworkInterfaceASIO-BGSync-0] Successfully connected to opsdba-vbj01-1:27018