一哥们要导出一个数据表的数据,结果导出一半,硬盘不够用,卡死了, 然后重启主机,导致mongo启动后进程自动死掉, 报错如下。
Mon Oct 28 10:39:02.270 [initandlisten] connection accepted from 192.168.85.144:41102 #25 (10 connections now open)
Mon Oct 28 10:39:02.270 [conn25] authenticate db: EmailSMSPlatform { authenticate: 1, user: "writeuser", nonce: "d3d91dba32b5f7be", key: "c8226274e6e4e6258a18b1eb4d4d90db" }
Mon Oct 28 10:39:02.271 [conn25] EmailSMSPlatform.session Deleted record list corrupted in bucket 2, link number 3, invalid link is 26780528:1a6bd24, throwing Fatal Assertion
Mon Oct 28 10:39:02.271 [conn25] EmailSMSPlatform.session Fatal Assertion 16469
0xdcf361 0xd8f0d3 0xa59842 0xa5a0b2 0xabff1f 0xac130f 0xa8b72a 0xa8d877 0x9f09d3 0x9f5f6e 0x6e747a 0xdbbb7e 0x33c3c0683d 0x33c2cd4fad
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xdcf361]
/usr/bin/mongod(_ZN5mongo13fassertFailedEi+0xa3) [0xd8f0d3]
/usr/bin/mongod(_ZN5mongo16NamespaceDetails10__stdAllocEib+0x462) [0xa59842]
/usr/bin/mongod(_ZN5mongo16NamespaceDetails13allocWillBeAtEPKci+0x32) [0xa5a0b2]
/usr/bin/mongod(_ZN5mongo11DataFileMgr6insertEPKcPKvibbbPb+0x114f) [0xabff1f]
/usr/bin/mongod(_ZN5mongo11DataFileMgr16insertWithObjModEPKcRNS_7BSONObjEbb+0x4f) [0xac130f]
/usr/bin/mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEPNS_11RemoveSaverEbRKNS_24QueryPlanSelectionPolicyEb+0x2efa) [0xa8b72a]
/usr/bin/mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEbRKNS_24QueryPlanSelectionPolicyE+0xb7) [0xa8d877]
/usr/bin/mongod(_ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE+0x4d3) [0x9f09d3]
/usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xb1e) [0x9f5f6e]
/usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x9a) [0x6e747a]
/usr/bin/mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x42e) [0xdbbb7e]
/lib64/libpthread.so.0 [0x33c3c0683d]
/lib64/libc.so.6(clone+0x6d) [0x33c2cd4fad]
Mon Oct 28 10:39:02.278 [conn25]
***aborting after fassert() failure
Mon Oct 28 10:39:02.279 Got signal: 6 (Aborted).
Mon Oct 28 10:39:02.286 Backtrace:
0xdcf361 0x6cf729 0x33c2c302f0 0x33c2c30285 0x33c2c31d30 0xd8f10e 0xa59842 0xa5a0b2 0xabff1f 0xac130f 0xa8b72a 0xa8d877 0x9f09d3 0x9f5f6e 0x6e747a 0xdbbb7e 0x33c3c0683d 0x33c2cd4fad
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xdcf361]
/usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0x6cf729]
/lib64/libc.so.6 [0x33c2c302f0]
/lib64/libc.so.6(gsignal+0x35) [0x33c2c30285]
/lib64/libc.so.6(abort+0x110) [0x33c2c31d30]
/usr/bin/mongod(_ZN5mongo13fassertFailedEi+0xde) [0xd8f10e]
/usr/bin/mongod(_ZN5mongo16NamespaceDetails10__stdAllocEib+0x462) [0xa59842]
/usr/bin/mongod(_ZN5mongo16NamespaceDetails13allocWillBeAtEPKci+0x32) [0xa5a0b2]
/usr/bin/mongod(_ZN5mongo11DataFileMgr6insertEPKcPKvibbbPb+0x114f) [0xabff1f]
/usr/bin/mongod(_ZN5mongo11DataFileMgr16insertWithObjModEPKcRNS_7BSONObjEbb+0x4f) [0xac130f]
/usr/bin/mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEPNS_11RemoveSaverEbRKNS_24QueryPlanSelectionPolicyEb+0x2efa) [0xa8b72a]
/usr/bin/mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEbRKNS_24QueryPlanSelectionPolicyE+0xb7) [0xa8d877]
/usr/bin/mongod(_ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE+0x4d3) [0x9f09d3]
/usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xb1e) [0x9f5f6e]
/usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x9a) [0x6e747a]
/usr/bin/mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x42e) [0xdbbb7e]
/lib64/libpthread.so.0 [0x33c3c0683d]
/lib64/libc.so.6(clone+0x6d) [0x33c2cd4fad]
尝试的未果的解决方法
1. export LC_ALL=C
2. export LC_CTYPE="en_US.UTF-8"
都木有任何作用, 官方bugd地址 https://jira.mongodb.org/browse/SERVER-10563
最终解决方法
!!!使用 --repair参数 即启动命令 /usr/bin/mongod -f /etc/mongod.conf --repair 之后经过一段时间的修复,终于ok,希望不要再出问题。