具体实际的取舍可以参考官方文档,我使用的是4 node 4 driver 模式
环境机器说明
192.168.31.2 192.168.31.3 192.168.31.4 192.168.31.5
每台机器单独挂在4块盘 data1 data2 data3 data4
minio 安装包参考github 网站
参考图
启动&&运行
- 配置key(实际可以直接配置到profile中)
export MINIO_ACCESS_KEY=<ACCESS_KEY>
export MINIO_SECRET_KEY=<SECRET_KEY>
- 启动(每台机器执行)
/usr/local/bin/minio server http://192.168.31.2/data1 http://192.168.31.2/data2 \
http://192.168.31.2/data3 http://192.168.31.2/data4 \
http://192.168.31.3/data1 http://192.168.31.3/data2 \
http://192.168.31.3/data3 http://192.168.31.3/data4 \
http://192.168.31.4/data1 http://192.168.31.4/data2 \
http://192.168.31.4/data3 http://192.168.31.4/data4 \
http://192.168.31.5/data1 http://192.168.31.5/data2 \
http://192.168.31.5/data3 http://192.168.31.5/data4
等待片刻,节点加入成功,并会格式化磁盘
配置nginx 支持lb
我使用了lvs nginx 是在realserver 上,并使用了dr 模式
upstream minio {
server 192.168.31.2:9000 weight=10 max_fails=2 fail_timeout=30s;
server 192.168.31.3:9000 weight=10 max_fails=2 fail_timeout=30s;
server 192.168.31.4:9000 weight=10 max_fails=2 fail_timeout=30s;
server 192.168.31.5:9000 weight=10 max_fails=2 fail_timeout=30s;
}
server {
listen 9000;
server_name localhost;
charset utf-8;
default_type text/html;
location /{
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $remote_addr;
client_body_buffer_size 10M;
client_max_body_size 10G;
proxy_buffers 1024 4k;
proxy_read_timeout 300;
proxy_next_upstream error timeout http_404;
proxy_pass http://minio;
}
}
访问测试
说明
实际使用可能需要使用systemd 进行管理,官方提供了脚本
- 参考脚本
[Unit]
Description=Minio
Documentation=https://docs.minio.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio
[Service]
WorkingDirectory=/usr/local
User=minio-user
Group=minio-user
PermissionsStartOnly=true
EnvironmentFile=-/etc/default/minio
ExecStartPre=/bin/bash -c "[ -n \"${MINIO_VOLUMES}\" ] || echo \"Variable MINIO_VOLUMES not set in /etc/defaults/minio\""
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES
# Let systemd restart this service only if it has ended with the clean exit code or signal.
Restart=on-success
StandardOutput=journal
StandardError=inherit
# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536
# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0
# SIGTERM signal is used to stop Minio
KillSignal=SIGTERM
SendSIGKILL=no
SuccessExitStatus=0
[Install]
WantedBy=multi-user.target
# Built for ${project.name}-${project.version} (${project.name})
参考资料
https://docs.minio.io/docs/distributed-minio-quickstart-guide
https://github.com/minio/minio-service