本文仅介绍在linux系统中部署minio
软件下载
!!!强烈建议同一个业务系统使用同一固定版本,防止版本变动带来的不必要的问题
下载最新版文件
wget https://dl.min.io/server/minio/release/linux-amd64/minio wget https://dl.min.io/client/mc/release/linux-amd64/mc
注意:
部署运行 安装minio 和 mc chmod +x minio mv minio /usr/local/bin/ chmod +x mc mv mc /usr/local/bin/
创建minio用户 # 创建用户组:-r创建一个系统账户 groupadd -r minio # 创建用户 -M不创建用户的主目录 -r创建一个系统账户 -g新账户主组的名称或 ID useradd -M -r -g minio minio
脚本方式部署
首先我们建议将脚本统一放在/opt/minio目录下,并授权给minio用户,包括后面自行编写的导入导出脚本
# 授权用户目录权限 chown minio:minio /opt/minio # 赋予脚本执行权限 chmod 777 /opt/minio/*.sh
单节点单驱动器部署
不具有纠删码校验修复能力 简单、快速部署
1.创建和授权minio用户数据目录
mkdir /mnt/data chown minio:minio /mnt/data
2.启动脚本 /opt/minio/minio-simple-start.sh
#!/bin/bash RUNNING_USER=minio MINIO_root_USER=minioadmin MINIO_ROOT_PASSword=miniopwd MINIO_DATA=/mnt/data START_CMD="MINIO_ROOT_USER=${MINIO_ROOT_USER} MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD} nohup minio server --console-address ":19000" ${MINIO_DATA} > ./minio-simple-run.log 2>&1 &" su ${RUNNING_USER} -c "${START_CMD}" echo "${START_CMD}"
3.管理操作
# 启动实例 ./minio-simple-start.sh # 访问控制台 (注意:访问http://ip:9000也会直接重定向到19000端口) http://ip:19000 # 查看日志 tAIl -100f minio-simple-run.log # 查看所有minio进程 ps -ef |grep minio # 杀死所有minio进程 ps -aux | grep minio | awk '{print $2}' | xargs kill
单节点多驱动器部署
具有纠删码校验修复能力 下例中/mnt/disk{1...4},用的是目录模拟的驱动器,实际生产环境中,应挂载为4个磁盘
1.创建和授权minio用户数据模拟的驱动器目录
mkdir /mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4 chown minio:minio /mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4
2.启动脚本 /opt/minio/minio-start.sh
#!/bin/bash RUNNING_USER=minio MINIO_ROOT_USER=minioadmin MINIO_ROOT_PASSWORD=miniopwd MINIO_DATA=/mnt/disk{1...4} # CI=true 允许根目录磁盘作为驱动器(模拟时使用,生产环境可去除) START_CMD="MINIO_ROOT_USER=${MINIO_ROOT_USER} MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD} CI=true nohup minio server --console-address ":19000" ${MINIO_DATA} > ./minio-run.log 2>&1 &" su ${RUNNING_USER} -c "${START_CMD}" echo "${START_CMD}"
3.管理操作
# 启动实例 ./minio-start.sh # 访问控制台 (注意:访问http://ip:9000也会直接重定向到19000端口) http://ip:19000 # 查看日志 tail -100f minio-run.log # 查看所有minio进程 ps -ef |grep minio # 杀死所有minio进程 ps -aux | grep minio | awk '{print $2}' | xargs kill
多节点多驱动器部署
具有纠删码校验修复能力 模拟准备四个节点,每个节点四个驱动器 本例中用的是目录模拟的节点和驱动器,实际生产环境中,应以4个服务器作为节点,每个节点挂载为4个磁盘
1.创建和授权minio用户数据模拟的节点以及驱动器目录
mkdir /mnt/mdata && mkdir /mnt/mdata/node1 /mnt/mdata/node2 /mnt/mdata/node3 /mnt/mdata/node4 mkdir /mnt/mdata/node1/disk1 /mnt/mdata/node1/disk2 /mnt/mdata/node1/disk3 /mnt/mdata/node1/disk4 mkdir /mnt/mdata/node2/disk1 /mnt/mdata/node2/disk2 /mnt/mdata/node2/disk3 /mnt/mdata/node2/disk4 mkdir /mnt/mdata/node3/disk1 /mnt/mdata/node3/disk2 /mnt/mdata/node3/disk3 /mnt/mdata/node3/disk4 mkdir /mnt/mdata/node4/disk1 /mnt/mdata/node4/disk2 /mnt/mdata/node4/disk3 /mnt/mdata/node4/disk4 # 赋予用户和文件夹读写权限,否则会无权限错误 chown minio:minio /mnt/mdata # 赋予所有节点可读写权限,否则建立驱动器时会报错 chmod -R 777 /mnt/mdata
2.启动脚本 /opt/minio/minio-multi-start.sh
#!/bin/bash RUNNING_USER=minio MINIO_HOST=192.168.1.110 MINIO_ROOT_USER="minioadmin" MINIO_ROOT_PASSWORD="miniopwd" for i in {1..4}; do START_CMD="MINIO_ROOT_USER=${MINIO_ROOT_USER} MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD} nohup minio server --address "${MINIO_HOST}:900${i}" --console-address "${MINIO_HOST}:1900${i}" http://${MINIO_HOST}:9001/mnt/mdata/node1/disk{1...4} http://${MINIO_HOST}:9002/mnt/mdata/node2/disk{1...4} http://${MINIO_HOST}:9003/mnt/mdata/node3/disk{1...4} http://${MINIO_HOST}:9004/mnt/mdata/node4/disk{1...4} > ./minio-multi-run${i}.log 2>&1 &" su ${RUNNING_USER} -c "${START_CMD}" echo "${START_CMD}" done
3.管理操作
# 启动实例 ./minio-multi-start.sh # 访问控制台 (注意:访问http://ip:9001也会直接重定向到19001端口) http://ip:19001 http://ip:19002 ... # 查看日志 tail -100f minio-multi-run1.log tail -100f minio-multi-run2.log ... # 查看所有minio进程 ps -ef |grep minio # 杀死所有minio进程 ps -aux | grep minio | awk '{print $2}' | xargs kill
4.多节点Nginx负载均衡
upstream minio { server 192.168.1.110:9001; server 192.168.1.110:9002; server 192.168.1.110:9003; server 192.168.1.110:9004; } server{ listen 9000; server_name 192.168.1.110; ignore_invalid_headers off; proxy_buffering off; location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Host $host:$server_port; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto; proxy_set_header Host $http_host; proxy_connect_timeout 300; #秒 proxy_http_version 1.1; client_body_buffer_size 100m; client_max_body_size 1000m; chunked_transfer_encoding off; proxy_ignore_client_abort on; proxy_pass http://minio; } }
然后就可以通过:192.168.1.110:9000 直接访问API了,当然访问控制台还是会重定向到1900{1...4}端口。
服务方式部署 官方文档
官方文档
https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-single-node-single-drive.html
使用多驱动器的建议
mkfs.xfs /dev/sdb -L DISK1 mkfs.xfs /dev/sdc -L DISK2 mkfs.xfs /dev/sdd -L DISK3 mkfs.xfs /dev/sde -L DISK4 nano /etc/fstab # LABEL=DISK1 /mnt/disk1 xfs defaults,noatime 0 2 LABEL=DISK2 /mnt/disk2 xfs defaults,noatime 0 2 LABEL=DISK3 /mnt/disk3 xfs defaults,noatime 0 2 LABEL=DISK4 /mnt/disk4 xfs defaults,noatime 0 2
单节点单/多驱动器部署
1.创建 /etc/systemd/system/minio.service 服务
cd /etc/systemd/system/ touch minio.service
/etc/systemd/system/minio.service
[Unit] Description=MinIO Documentation=https://min.io/docs/minio/linux/index.html Wants.NETwork-online.target After=network-online.target AssertFileIsExecutable=/usr/local/bin/minio [Service] WorkingDirectory=/usr/local # Run Os User User=minio Group=minio ProtectProc=invisible EnvironmentFile=/etc/default/minio ExecStartPre=/bin/bash -c "if [ -z "${MINIO_VOLUMES}" ]; then echo "Variable MINIO_VOLUMES not set in /etc/default/minio"; exit 1; fi" ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES # Let systemd Restart this service always Restart=always # Specifies the maximum file descriptor number that can be opened by this process LimitNOFILE=65536 # Specifies the maximum number of threads this process can create TasksMax=infinity # Disable timeout logic and wait until process is stopped TimeoutStopSec=infinity SendSIGKILL=no [Install] WantedBy=multi-user.target # Built for ${project.name}-${project.version} (${project.name})
2.创建和授权minio用户数据模拟的驱动器目录
# 用于模拟多驱动器的目录 mkdir /mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4 chown minio:minio /mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4 # 用于单驱动器的目录 mkdir /mnt/data chown minio:minio /mnt/data
3.创建环境变量(注意更改用户名/密码)
# windows的在C:minioconfig touch /etc/default/minio vi /etc/default/minio
/etc/default/minio 文件内容如下
# MINIO_ROOT_USER and MINIO_ROOT_PASSWORD sets the root account for the MinIO server. # This user has unrestricted permissions to perform S3 and administrative API operations on any resource in the deployment. # Omit to use the default values 'minioadmin:minioadmin'. # MinIO recommends setting non-default values as a best practice, regardless of environment MINIO_ROOT_USER=minioadmin MINIO_ROOT_PASSWORD=miniopwd # MINIO_VOLUMES sets the storage volume or path to use for the MinIO server. # 单驱动器 # MINIO_VOLUMES="/mnt/data" # 多驱动器(至少4个以上) MINIO_VOLUMES="/mnt/disk{1...4}" # 控制台访问端口 MINIO_OPTS="--console-address :19000" # 该参数设置为true支持挂载根目录的多驱动器部署(不建议) CI=true # MINIO_SERVER_URL sets the hostname of the local machine for use with the MinIO Server # MinIO assumes your network control plane can correctly resolve this hostname to the local machine # Uncomment the following line and replace the value with the correct hostname for the local machine. #MINIO_SERVER_URL="http://minio.example.net"
4.管理操作
# 启用服务 systemctl enable minio.service # 启动 sudo systemctl start minio.service systemctl start minio.service # 重启 systemctl restart minio.service # 停止 systemctl stop minio.service # 查看状态 systemctl status minio.service journalctl -f -u minio.service
多节点多驱动器
对于多节点驱动器部署,至少需要4个以上的节点才能满足最低的纠删码校验修复功能
多节点多驱动器服务部署是官方的方式目前没有实验过,所以本文档可能参考的意义不多。
建议可直接翻阅官方文档。
多节点多驱动器官方文档
https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html
MinIO 强烈建议为部署中的所有节点选择基本相似的硬件配置。确保硬件(CPU、内存、主板、存储适配器)和软件(操作系统、内核设置、系统服务)在所有节点上保持一致。
1.按照单节点的步骤部署4台存储服务
部署具有单个服务器池,该池由四个具有顺序主机名的 MinIO 服务器主机组成。
# IP 192.168.10.101 192.168.10.102 192.168.10.103 192.168.10.104 # 或主机名 minio1.example.com minio3.example.com minio2.example.com minio4.example.com
每个服务器上的 /etc/default/minio 文件内容调整如下
# Set the hosts and volumes MinIO uses at startup # The command uses MinIO expansion notation {x...y} to denote a # sequential series. # # The following example covers four MinIO hosts # with 4 drives each at the specified hostname and drive locations. # The command includes the port that each MinIO server listens on # (default 9000) # MINIO_VOLUMES="http://192.168.10.10{1...4}:9000/mnt/disk{1...4}" MINIO_VOLUMES="https://minio{1...4}.example.net:9000/mnt/disk{1...4}" # Set all MinIO server options # # The following explicitly sets the MinIO Console listen address to # port 9001 on all network interfaces. The default behavior is dynamic # port selection. MINIO_OPTS="--console-address :19000" # Set the root username. This user has unrestricted permissions to # perform S3 and administrative API operations on any resource in the # deployment. # # Defer to your organizations requirements for superadmin user name. MINIO_ROOT_USER=minioadmin # Set the root password # # Use a long, random, unique string that meets your organizations # requirements for passwords. MINIO_ROOT_PASSWORD=miniopwd # 负载均衡URL设置 # Set to the URL of the load balancer for the MinIO deployment # This value *must* match across all MinIO servers. If you do # not have a load balancer, set this value to to any *one* of the # MinIO hosts in the deployment as a temporary measure. # MINIO_SERVER_URL="http://192.168.10.100:9000" MINIO_SERVER_URL="https://minio.example.net:9000"
可能的问题 1. 遇到类似 Unknown lvalue 'ProtectProc' in section 'Service' # 升级systemd yum install systemd-* -y
2. Error: Disk /mnt/disk1 is part of root disk # 将 /etc/default/minio 文件下 设置CI=true # 这是因为minio默认不允许多驱动器模式直接使用根目录的磁盘 CI=true
存储迁移
主要是利用mc进行迁移操作
# 从机器1 迁移到 机器2 # 设置机器1别名 mc alias set mdata http://127.0.0.1:9000 minioadmin miniopwd # 查看已设置的别名列表 mc alias list # 导出备份 mdata:别名,base:存储桶 mc cp --recursive mdata/base/ /mnt/minio_data_bak/ # 将minio_data_bak打包迁移到机器2 # 设置机器2别名 mc alias set mdata http://127.0.0.1:9000 minioadmin miniopwd # 导入备份文件 mdata:别名,base:存储桶 mc cp --recursive /mnt/minio_data_bak/ mdata/base/
更多MC命令可参考官方文档:mc命令操作说明
https://min.io/docs/minio/linux/reference/minio-mc.html#minio-client
结语
如有错误请指正,非常感谢!