graylog日志平台搭建 (一) 部署mongo集群

此次搭建graylog日志平台,使用到的组件有mongo、elasticsearch、kafka、zookeeper、graylog、filebeat,下面总结一下搭建过程:

一、部署mongo集群(单机版也可以)

1、时间同步

安装ntpdate,添加定时任务同步时间

最好所有机器都配置时间同步

yum install ntpdate -y
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

# crontab -e
5 * * * * ntpdate -u ntp.ntsc.ac.cn

2、yum安装mongo

配置仓库源并安装
vim /etc/yum.repos.d/mongodb-org.repo
[mongodb-org-4.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.2/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.2.asc


yum -y install mongodb-org

启动mongo并设置开机自启

systemctl daemon-reload
systemctl enable mongod.service
systemctl start mongod.service
3、集群模式专有配置(单机版可以忽略此步) 修改配置文件设置副本集
# vim /etc/mongod.conf
# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# Where and how to store data.
storage:
  dbPath: /var/lib/mongo
  journal:
    enabled: true
#  engine:
#  wiredTiger:

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod.pid  # location of pidfile
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27017
  bindIp: 0.0.0.0  # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.


#security:

#operationProfiling:

replication:
  replSetName: graylog-rs  #设置副本集名称

#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:

初始化副本集

> use admin;
switched to db admin
> rs.initiate( {
...      _id : "graylog-rs",
...      members: [
...          { _id: 0, host: "192.168.1.10:27017"},
...          { _id: 1, host: "192.168.1.11:27017"},
...          { _id: 2, host: "192.168.1.12:27017"}
...      ]
...  })

确认副本集状态

rs.status()

        "members" : [
                {
                        "_id" : 0,
                        "name" : "192.168.1.10:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 623,
                        "optime" : {
                                "ts" : Timestamp(1621748878, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2021-05-23T13:47:58Z"),
                        "syncingTo" : "",
                        "syncSourceHost" : "",
                        "electionTime" : Timestamp(1621748728, 1),
                        "electionDate" : ISODate("2021-05-23T13:45:28Z"),
                        "configVersion" : 1,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "192.168.1.11:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 162,
                        "optime" : {
                                "ts" : Timestamp(1621748728, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1621748728, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2021-05-23T13:45:28Z"),
                        "optimeDurableDate" : ISODate("2021-05-23T13:45:28Z"),
                        "lastHeartbeat" : ISODate("2021-05-23T13:45:28.690Z"),
                        "lastHeartbeatRecv" : ISODate("2021-05-23T13:45:28.288Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncingTo" : "192.168.1.10:27017",
                        "syncSourceHost" : "192.168.1.10:27017",
                        "syncSourceId" : 0,
                        "infoMessage" : "",
                        "configVersion" : 1
                },
                {
                        "_id" : 2,
                        "name" : "192.168.1.12:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 162,
                        "optime" : {
                                "ts" : Timestamp(1621748728, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1621748728, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2021-05-23T13:45:28Z"),
                        "optimeDurableDate" : ISODate("2021-05-23T13:45:28Z"),
                        "lastHeartbeat" : ISODate("2021-05-23T13:45:28.690Z"),
                        "lastHeartbeatRecv" : ISODate("2021-05-23T13:45:28.286Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncingTo" : "192.168.1.10:27017",
                        "syncSourceHost" : "192.168.1.10:27017",
                        "syncSourceId" : 0,
                        "infoMessage" : "",
                        "configVersion" : 1
                }
        ]
生成 keyFile 文件,拷贝到其它节点,在第一台PRIMARY节点执行

openssl rand -base64 756 > /var/lib/mongo/access.key
chown -R mongod.mongod /var/lib/mongo/access.key
chmod 600 /var/lib/mongo/access.key
scp -r /var/lib/mongo/access.key 192.168.1.11:/var/lib/mongo/
scp -r /var/lib/mongo/access.key 192.168.1.12:/var/lib/mongo/
拷贝完成后,需要修改配置文件

集群搭建到此结束。

4、创建用户

创建管理员用户 随便找一台机器执行即可
use admin
db.createUser({user: "admin", pwd: "Admin_123456", roles: ["root"]})
db.auth("admin","Admin_123456")

创建用于graplog连接的用户

use graylog
db.createUser({
   user: "graylog", 
   pwd: "Graylog_123456",
  "roles" : [{
      "role" : "dbOwner",
      "db" : "graylog"
    }, {
      "role" : "readWrite",
      "db" : "graylog"
    }]
})

开启安全认证配置,三台机器都需要如此设置,然后重启服务

# vim /etc/mongod.conf
#添加如下配置
security:
  keyFile: /var/lib/mongo/access.key    #单机mongo不用加这行
  authorization: enabled

  systemctl restart mongod 5、验证 1、登陆是否能认证成功 2、副本集状态是否正常   到此mongo集群副本集部署好了。

 

上一篇:新制度会计学研究学习笔记3:新技术、大数据与数字时代的会计研究


下一篇:451 Research:71%受访者使用K8S管理容器