引用博文:https://blog.csdn.net/qq_22648091/article/details/114140816
侵权联删
Ansible 服务部署-部署 ELK 集群服务
创建三台 云主机分别命名为 elk01、elk02 和 elk03,Ansible 主机可以使用上一题的环境。要求 Ansible 节点编写剧本,执 行 Ansible 剧本可以在这三个节点部署 ELK 集群服务(在/root 目录下创建 install_elk 目录作 为 ansible 工作目录,部署的入口文件命名为 install_elk.yaml)。具体要求为三个节点均安装 Elasticserach 服务并配置为 Elasticserach 集群;kibana 安装在第一个节点;Logstash 安装在 第二个节点。
ELK需要的二进制包,如下是下载链接
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.0-linux-x86_64.tar.gz
https://artifacts.elastic.co/downloads/kibana/kibana-7.17.0-linux-x86_64.tar.gz
https://artifacts.elastic.co/downloads/logstash/logstash-7.17.0-linux-x86_64.tar.gz
https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.17.0-linux-x86_64.tar.gz
地址规划:
角色 | 主机名 | IP地址 |
---|---|---|
ansible母机 | ansible | 10.18.4.46 |
ela01 && kibana | elk-1 | 10.18.4.201 |
ela02 && logstash | elk-2 | 10.18.4.202 |
ela03 | elk-3 | 10.18.4.203 |
基础准备
#域名解析(all node)
cat >> /etc/hosts <<EOF
10.18.4.46 ansible
10.18.4.201 elk-1
10.18.4.202 elk-2
10.18.4.203 elk-3
EOF
#####centos7.9 node#####
hostnamectl set-hostname elk-1
hostnamectl set-hostname elk-2
hostnamectl set-hostname elk-3
####Ansible node#####
mkdir /root/install_elk
touch /root/install_elk/install_elk.yaml
mkdir -p /root/install_elk/roles/{ela,kib,log}/{files,handlers,tasks,templates,vars}
#在ansible节点与三台节点建立免密钥通信
ssh-keygen
ssh-copy-id elk-1
ssh-copy-id elk-2
ssh-copy-id elk-3
#ansible主机清单
cat >> /etc/ansible/hosts <<EOF
[ela]
elk-1 node_name=elk-1
elk-2 node_name=elk-2
elk-3 node_name=elk-3
[kib]
elk-1
[log]
elk-2
EOF
项目目录结构
主入口文件
/root/install_elk/install_elk.yaml
cat > /root/install_elk/install_elk.yaml <<EOF
---
- hosts: ela
gather_facts: no
remote_user: root
roles:
- ela
- hosts: kib
gather_facts: no
remote_user: root
roles:
- kib
- hosts: log
gather_facts: no
remote_user: root
roles:
- log
EOF
部署ela集群
/root/install_elk/roles/ela/tasks/main.yaml
入口文件
cat > /root/install_elk/roles/ela/tasks/main.yaml <<EOF
- name: 安装Java环境
unarchive:
src: jdk-8u144-linux-x64.tar.gz
dest: /opt
list_files: yes
- name: 添加Java环境变量
shell: echo 'export JAVA_HOME=/opt/jdk1.8.0_144' >> /etc/profile && echo 'export PATH=$PATH:/opt/jdk1.8.0_144/bin' >> /etc/profile
- name: 生效环境变量
shell: source /etc/profile
- name: 创建用户
user:
name: ela
- name: 传输本地软件包到远程主机并且解压到指定目录
unarchive:
src: elasticsearch-7.17.0-linux-x86_64.tar.gz
dest: /opt
owner: ela
group: ela
list_files: yes
register: ret
- name: 创建软链接
file:
src: /opt/{{ ret.files.0 | regex_replace('/.*') }}
dest: /opt/elasticsearch
state: link
- name: 传输配置文件
template:
src: elasticsearch.j2
dest: /opt/elasticsearch/config/elasticsearch.yml
owner: ela
group: ela
- name: 传输系统配置文件
copy:
src: limits.conf
dest: /etc/security/limits.conf
- name: 传输系统配置文件
copy:
src: sysctl.conf
dest: /etc/sysctl.conf
- name: 加载 /etc/sysctl.conf文件,使内核参数生效
shell: sysctl -p
- name: 启动服务
# 使用ela用户执行此命令
become: yes
become_user: ela
command:
# argv 是一个列表,存放了需要执行的命令及其参数
# 一行一个
argv:
- nohup
- /opt/elasticsearch/bin/elasticsearch
- -d
EOF
/root/install_elk/roles/ela/templates/elasticsearch.j2
,elasticsearch的主配置文件模板
cat > /root/install_elk/roles/ela/templates/elasticsearch.j2 <<EOF
cluster.name: elk
node.name: {{ node_name }}
node.data: true
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts:
- 10.18.4.201
- 10.18.4.202
- 10.18.4.203
cluster.initial_master_nodes: ["elk-1","elk-2","elk-3"]
EOF
/root/install_elk/roles/ela/files/limits.conf
,系统文件句柄数配置文件
cat > /root/install_elk/roles/ela/files/limits.conf <<EOF
* soft nofile 65535
* hard nofile 65535
* soft nproc 4096
* hard nproc 4096
EOF
/root/install_elk/roles/ela/files/sysctl.conf
,系统的内核配置文件
只有
# elasticsearch
下面的配置内容是必须的,其他可选
cat > /root/install_elk/roles/ela/files/sysctl.conf <<EOF
vm.max_map_count=655360
EOF
验证:
部署kibana
/root/install_elk/roles/kib/tasks/main.yaml
任务入口文件
cat > /root/install_elk/roles/kib/tasks/main.yaml <<EOF
- name: 传输本地软件包到远程主机并且解压到指定目录
unarchive:
src: kibana-7.17.0-linux-x86_64.tar.gz
dest: /opt
owner: ela
group: ela
list_files: yes
register: ret
- name: 创建软链接
file:
src: /opt/{{ ret.files.0 | regex_replace('/.*') }}
dest: /opt/kibana
state: link
- name: 创建日志与PID存放目录
shell: mkdir -p /var/log/kibana /run/kibana
- name: 给如上目录设置权限
shell: chown -R ela:ela /var/log/kibana /run/kibana
- name: 传输配置文件
copy:
src: kibana.yml
dest: /opt/kibana/config/kibana.yml
- name: 传输服务管理文件
template:
src: kibana.service.j2
dest: /etc/systemd/system/kibana.service
- name: 启动服务
systemd:
name: kibana
state: started
daemon_reload: yes
EOF
/root/install_elk/roles/kib/files/kibana.yml
Kibana 程序的配置文件
cat > /root/install_elk/roles/kib/files/kibana.yml <<EOF
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://10.18.4.201:9200"]
i18n.locale: "zh-CN"
pid.file: /run/kibana/kibana.pid
logging.dest: /var/log/kibana/kibana.log
EOF
/root/install_elk/roles/kib/templates/kibana.service.j2
,Kibana 程序管理模板文件
cat > /root/install_elk/roles/kib/templates/kibana.service.j2 <<EOF
[Unit]
Description=Kibana
Documentation=https://www.elastic.co
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=ela
Group=ela
ExecStart=/opt/kibana/bin/kibana
ExecStop=/bin/pkill -F "/run/kibana/kibana.pid"
Restart=on-failure
RestartSec=3
StartLimitBurst=3
StartLimitInterval=60
WorkingDirectory=/opt/kibana
StandardOutput=journal
StandardError=inherit
[Install]
WantedBy=multi-user.target
EOF
验证:
部署Logstash
/root/install_elk/roles/log/tasks/main.yaml
任务入口文件
cat > /root/install_elk/roles/log/tasks/main.yaml <<EOF
- name: 创建日志目录
file:
path: /var/log/logstash
state: directory
- name: 服务日志目录权限
shell: chown -R ela:ela /var/log/logstash
- name: 传输本地软件包到远程主机并且解压到指定目录
unarchive:
src: logstash-7.17.0-linux-x86_64.tar.gz
dest: /opt
list_files: yes
register: ret
- name: 创建软链接
file:
src: /opt/{{ ret.files.0 | regex_replace('/.*') }}
dest: /opt/logstash
state: link
- name: 传输配置文件
template:
src: logstash.yml
dest: /opt/logstash/config/logstash.yml
- name: 传输管道配置文件
copy:
src: logstash.conf
dest: /opt/logstash/config/logstash-sample.conf
- name: 传输系统服务文件
template:
src: logstash.service.j2
dest: /etc/systemd/system/logstash.service
- name: 启动 logstash
systemd:
name: logstash
state: started
daemon_reload: yes
- name: restart logstash
systemd:
name: logstash
state: restarted
daemon_reload: yes
#- name: 启动服务
# become: yes
# become_user: ela
# shell: sh /opt/logstash/bin/logstash -f /opt/logstash/config/logstash-sample.conf
EOF
/root/install_elk/roles/log/templates/logstash.yml
,Logstash 程序主配置文件
cat > /root/install_elk/roles/log/templates/logstash.yml <<EOF
http.host: "0.0.0.0"
path.logs: /var/log/logstash/
EOF
/root/install_elk/roles/log/files/logstash.conf
,Logstash 程序管道配置文件
cat > /root/install_elk/roles/log/files/logstash.conf <<EOF
#将本地的/var/log/yum.log内日志标准输入
input {
file {
path => "/var/log/yum.log"
type => "yum_log"
start_position => "beginning"
}
}
#标准输出到elasticsearch中
output {
elasticsearch {
hosts => ["10.18.4.201:9200","10.18.4.202:9200","10.18.4.203:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
#user => "elastic"
#password => "changeme"
}
}
EOF
/root/install_elk/roles/log/templates/logstash.service.j2
,Logstash 程序进程管理模板文件
cat > /root/install_elk/roles/log/templates/logstash.service.j2 <<EOF
Unit]
Description=logstash
[Service]
Type=simple
ExecStart=/opt/logstash/bin/logstash "-f" "/opt/logstash/config/*.conf"
Restart=always
WorkingDirectory=/
LimitNOFILE=65535
TimeoutStopSec=infinity
[Install]
WantedBy=multi-user.target
EOF
验证:
在任务栏中再到管理 -》堆栈管理。
创建一个索引模式,选择所有的索引。
测试安装一个软件来生成数据。
[root@elk-2 ~]#yum -y install httpd
来到discover中检索数据。文章来源:https://www.toymoban.com/news/detail-403717.html
文章来源地址https://www.toymoban.com/news/detail-403717.html
到了这里,关于Ansible 服务部署-部署 ELK 集群服务的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!