docker-compose的方式对接ELK
(1)一键部署docker和docker-compose环境
docker容器简介 (一键部署脚本)
#!/bin/bash
install docker shell
#step 1: 安装必要的一些系统工具
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
2: 添加软件源信息
sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
3: 更新并安装 Docker-CE
sudo yum makecache fast
sudo yum -y install docker-ce
#4: 开启Docker服务
sudo service docker start
4.添加镜像加速器
echo “4.添加镜像加速器…”
sudo tee /etc/docker/daemon.json <<-‘EOF’
{
“registry-mirrors”: [“https://xcweb0za.mirror.aliyuncs.com”]
}
EOF
5.重新启动服务
echo “5.重新启动服务…”
sudo systemctl daemon-reload
sudo systemctl restart docker
6.检测
echo “6.检测…”
docker info
7.安装docker-compose
echo “7.安装docker-compose”
curl -L https://get.daocloud.io/docker/compose/releases/download/1.22.0/docker-compose-uname -s
-uname -m
> /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
8.验证是否安装成功
echo “8.验证是否安装成功…”
docker-compose -v
(2)编写docker-compose文件
version: ‘2’
services:
filebeat-logcenter:
image: harbor.qilu-dev.com/elk_public/filebeat:6.8.2
container_name: filebeat-logcenter
network_mode: host
restart: always
volumes:
- /root/log/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
- /usr/java/logs/:/usr/java/logs/:ro
- /usr/java/apache-tomcat-8.0.46/logs/:/usr/java/apache-tomcat-8.0.46/logs/:ro
environment: - TZ=Asia/Shanghai
- KAFKA_HOSTS=10.29.131.62:9092,10.29.131.63:9092,10.29.131.64:9092,10.29.131.65:9092
(3)filebeat模板文件
filebeat.inputs:
-
type: log
enabled: true
paths: -
/usr/java/logs/ztapi.log
fields:
headers.appName: ztapi
headers.key: ztapi
fields_under_root: true
multiline.pattern: ‘^[’
multiline.negate: ‘true’
multiline.match: ‘after’ -
type: log
enabled: true
paths: -
/usr/java/apache-tomcat-8.0.46/logs/catalina.out
fields:
headers.appName: ztapitomcat
headers.key: ztapitomcat
fields_under_root: true
multiline.pattern: ‘^\d{2}-\w{3}-\d{4}’
multiline.negate: ‘true’
multiline.match: ‘after’ -
type: log
enabled: true
paths: -
/usr/java/apache-tomcat-8.0.46/logs/localhost_access_log*
fields:
headers.appName: ztapilocalhostaccess
headers.key: ztapilocalhostaccess
fields_under_root: true
multiline.pattern: ‘^\d+.\d+.\d+.\d+’
multiline.negate: ‘true’
multiline.match: ‘after’
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
output.kafka:
enabled: true
hosts: “${KAFKA_HOSTS}”
topic: logCenter-other
key: appname
partition.hash:
reachable_only: false
keep_alive: 60
required_acks: 1
processors:
- add_host_metadata:
cache.ttl: 5m - rename:
fields: - from: “host.name”
to: “headers.host” - drop_fields:
fields: [“metadata”, “log”, “input”, “ecs”, “agent”, “host”, “prospector”, “source”, “beat”, “offset”]
(3)一键启动对接
设置好相应目录文件
一键启动
docker-compose up -d