GrayLog多节点集群架构如下
(图片可点击放大查看)
下面介绍Graylog4.2集群部署过程
三台服务器：centos7.9的虚拟机
/data分区（LVM) 用于Elasticsearch日志数据存储
内存大小为6GB
IP地址和主机名如下：
均已关闭SElinux
(图片可点击放大查看)
(图片可点击放大查看)
cat > /etc/yum.repos.d/mongodb-org.repo << EOF
[mongodb-org]
name=MongoDB Repository
baseurl=https://mirrors.aliyun.com/mongodb/yum/redhat/$releasever/mongodb-org/4.4/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.4.asc
EOF
yum install -y mongodb-org
(图片可点击放大查看)
systemctl daemon-reload
systemctl enable mongod.service
systemctl start mongod.service
systemctl --type=service --state=active | grep mongod
firewall-cmd --add-port=27017/tcp --permanent --zone=public
firewall-cmd --reload
(图片可点击放大查看)
openssl rand -base64 756 > /var/lib/mongo/access.keyfile
chown mongod:mongod /var/lib/mongo/access.keyfile
chmod 600 /var/lib/mongo/access.keyfile
scp -rp /var/lib/mongo/access.keyfile root@graylog02:/var/lib/mongo/
scp -rp /var/lib/mongo/access.keyfile root@graylog03:/var/lib/mongo/
(图片可点击放大查看)
(图片可点击放大查看)
以主节点为例，修改配置文件
vi /etc/mongod.conf
net:
port: 27017
bindIp: 192.168.31.211
security:
keyFile: /var/lib/mongo/access.keyfile
replication:
replSetName: graylog-rs
(图片可点击放大查看)
graylog02修改成如下
(图片可点击放大查看)
graylog03修改成如下
(图片可点击放大查看)
这里为了本地能登录mongo
先将bindIp: 192.168.31.211修改为bindIp: 0.0.0.0
并重启服务
systemctl restart mongod.service
输入mongo进数据库
use admin
rs.initiate( {
_id : "graylog-rs",
members: [
{ _id: 0, host: "192.168.31.211:27017" },
{ _id: 1, host: "192.168.31.212:27017" },
{ _id: 2, host: "192.168.31.213:27017" }
]
})
rs.status()查看集群状态
(图片可点击放大查看)
#修改admin用户密码
use admin
db.createUser({user: "admin", pwd: "Admin@2021", roles: ["root"]})
db.auth("admin","Admin@2021")
#创建graylog数据库并设置密码
use graylog
db.createUser({
user: "graylog",
pwd: "Graylog2021",
"roles" : [{
"role" : "dbOwner",
"db" : "graylog"
}, {
"role" : "readWrite",
"db" : "graylog"
}]
})
(图片可点击放大查看)
将graylog01的bindIp修改成192.168.31.211并重启服务
这时使用账号和密码登录mongo
(图片可点击放大查看)
在graylog02 重启mongod服务，使graylog01重新变为Primary角色
cd /opt
wget https://mirrors.cloud.tencent.com/elasticstack/yum/elastic-7.x/7.16.2/elasticsearch-7.16.2-x86_64.rpm
rpm -ivh elasticsearch-7.16.2-x86_64.rpm
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service
firewall-cmd --add-port=9200/tcp --permanent --zone=public
firewall-cmd --reload
(图片可点击放大查看)
mkdir -p /data/elasticsearch/data
mkdir -p /data/elasticsearch/logs
chown -R elasticsearch:elasticsearch /data/elasticsearch
cp /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml_default
(图片可点击放大查看)
vi/etc/elasticsearch/elasticsearch.yml
1)graylog01节点
#添加并修改成如下行
cluster.name: graylog-cluster
action.auto_create_index: false
node.name: graylog01
node.master: true
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
network.host: 192.168.31.211
http.port: 9200
transport.port: 9300
discovery.seed_hosts: ["192.168.31.211:9300", "192.168.31.212:9300", "192.168.31.213:9300"]
cluster.initial_master_nodes: ["graylog01"]
(图片可点击放大查看)
2)graylog02节点
cat /etc/elasticsearch/elasticsearch.yml | grep -v "^#" | grep -v "^$"
cluster.name: graylog-cluster
action.auto_create_index: false
node.name: graylog02
node.master: false
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
network.host: 192.168.31.212
http.port: 9200
transport.port: 9300
discovery.seed_hosts: ["192.168.31.211:9300", "192.168.31.212:9300", "192.168.31.213:9300"]
3)graylog03节点
cat /etc/elasticsearch/elasticsearch.yml | grep -v "^#" | grep -v "^$"
cluster.name: graylog-cluster
action.auto_create_index: false
node.name: graylog03
node.master: false
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
network.host: 192.168.31.213
http.port: 9200
transport.port: 9300
discovery.seed_hosts: ["192.168.31.211:9300", "192.168.31.212:9300", "192.168.31.213:9300"]
firewall-cmd --add-port=9300/tcp --permanent --zone=public
firewall-cmd --reload
vim /etc/elasticsearch/jvm.options
设置jvm内存大小为物理内存的一半
并重启elasticsearch.service
systemctl restart elasticsearch.service
(图片可点击放大查看)
(图片可点击放大查看)
curl -s -XGET 'http://192.168.31.211:9200/_cluster/health?pretty=true'
curl -s -XGET 'http://192.168.31.211:9200/_cat/nodes?v'
(图片可点击放大查看)
yum install -y JAVA-1.8.0-openjdk-headless.x86_64
yum install -y pwgen
(图片可点击放大查看)
(图片可点击放大查看)
rpm -ivh https://packages.graylog2.org/repo/packages/graylog-4.2-repository_latest.rpm
yum install graylog-server -y
(图片可点击放大查看)
systemctl enable graylog-server
systemctl start graylog-server
cp /etc/graylog/server/server.conf /etc/graylog/server/server.conf_default
pwgen -N 1 -s 96
XC0Epiv5SnWFdm82nsUWAJN3t2MHaFEaSFHd6RLPf1nzxwnmubT0n7NQdrK8jCDEOS05DtrkGHDDE61490OUJKBOOXIAT4LI
echo -n "Enter Password: " && head -1 </dev/stdin | tr -d 'n' | sha256sum | cut -d" " -f1
Enter Password: Graylog@2021
10dfabb9595634675701865aa1c6e774d89d59f4a104ab128fbffcdaa3cf8f7b
(图片可点击放大查看)
主节点graylog01的配置如下
/etc/graylog/server/server.conf
cat /etc/graylog/server/server.conf | grep -v "^#" | grep -v "^$"
is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = XC0Epiv5SnWFdm82nsUWAJN3t2MHaFEaSFHd6RLPf1nzxwnmubT0n7NQdrK8jCDEOS05DtrkGHDDE61490OUJKBOOXIAT4LI
root_password_sha2 = 10dfabb9595634675701865aa1c6e774d89d59f4a104ab128fbffcdaa3cf8f7b
root_timezone = Asia/Shanghai
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 192.168.31.211:9000
http_publish_uri = http://192.168.31.211:9000/
elasticsearch_hosts = http://graylog01:9200,http://graylog02:9200,http://graylog03:9200
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = true
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 8
outputbuffer_processors = 16
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://graylog:Graylog2021@graylog01:27017,graylog02:27017,graylog03:27017/graylog?replicaSet=graylog-rs
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
proxied_requests_thread_pool_size = 32
(图片可点击放大查看)
建议将配置文件scp到graylog02和graylog03上
(图片可点击放大查看)
firewall-cmd --add-port=9000/tcp --permanent --zone=public
firewall-cmd --reload
systemctl restart graylog-server
(图片可点击放大查看)
graylog02,graylog03节点修改如下三处即可
is_master = true
http_bind_address = 192.168.31.213:9000
http_publish_uri = http://192.168.31.213:9000/
(图片可点击放大查看)
(图片可点击放大查看)
(图片可点击放大查看)
记得三台几点防火墙开放Input设置的端口，例如syslog1514
firewall-cmd --add-port=1514/udp --permanent --zone=public
firewall-cmd --reload
(图片可点击放大查看)
(图片可点击放大查看)
(图片可点击放大查看)
Tips：修改graylog的jvm内存大小
(图片可点击放大查看)
(图片可点击放大查看)
https://docs.graylog.org/v1/docs/multinode-setup
https://docs.mongodb.com/manual/tutorial/deploy-replica-set/
https://docs.mongodb.com/manual/tutorial/deploy-replica-set-with-keyfile-access-control/#std-label-deploy-repl-set-with-auth
https://cloud.tencent.com/developer/article/1615815
https://zhuanlan.zhihu.com/p/120698020
https://www.cnblogs.com/opsdemo/p/15035379.html
京公网安备 11010802035086号