ES:用来日志存储
Logstash:用来日志的搜集,进行日志格式转换并且传送给别人(转发)
Kibana:主要用于日志的展示和分析
kafka
Filebeat:搜集文件数据
es-1
本地解析
vi /etc/hosts
scp /etc/hosts es-2:/etc/hosts
scp /etc/hosts es-3:/etc/hosts
yum -y install wget
安装配置jdk
wget 8u191
scp -3
tar xf jdk-8u191-linux-x64.tar.gz -C /usr/local/
[root@es-1 ~]# vim /etc/profile
JAVA_HOME=/usr/local/java
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME PATH
[root@es-1~]# source /etc/profile
安装配置ES
useradd elsearch
ssh es-2 useradd elsearch
ssh es-3 useradd elsearch
echo "123456" | passwd --stdin "elsearch"
wget els包
vim /usr/local/elasticsearch/config/elasticsearch.yml
(都删了)
cluster.name: xingdiancloud-elk
node.name: es-1
node.master: true
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 0.0.0.0
http.port: 9200
discovery.zen.ping.unicast.hosts: ["es-1", "es-2","es-3"]
discovery.zen.ping_timeout: 150s
discovery.zen.fd.ping_retries: 10
client.transport.ping_timeout: 60s
http.cors.enabled: true
http.cors.allow-origin: "*"
创建ES数据及日志存储目录
mkdir -p /data/elasticsearch/{logs,data}
scp /usr/local/elasticsearch/config/elasticsearch.yml es-3:/usr/local/elasticsearch/config/elasticsearch.yml
修改安装目录及存储目录权限
系统优化
vi /etc/security/limits.conf
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sysctl -p
su - elsearch -c "cd /usr/local/elasticsearch && nohup bin/elasticsearch &"
es-2
echo "123456" | passwd --stdin "elsearch"
mkdir -p /data/elasticsearch/{logs,data}
echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sysctl -p
su - elsearch -c "cd /usr/local/elasticsearch && nohup bin/elasticsearch &"
ss -antpl //看9200
es-3
echo "123456" | passwd --stdin "elsearch"
mkdir -p /data/elasticsearch/{logs,data}
vim /usr/local/elasticsearch-6.5.4/config/elasticsearch.yml (改名字)
echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sysctl -p
su - elsearch -c "cd /usr/local/elasticsearch && nohup bin/elasticsearch &"
ss -antpl //看9200
浏览器访问:IP:9300
kibana
本地解析
vi /etc/hosts
IP es-1、2、3
上传kibana包
vi /usr/local/kibana/config/kibana.yml
server.port: 5601
server.host: "172.16.244.28"
elasticsearch.url: "http://172.16.244.25:9200"
kibana.index: ".kibana"
jobs
ss -antpl 看端口5600
浏览器访问 IP :5601
Nginx安装反向代理
安装
yum -y install nginx httpd-tools
验证节点与kibana时间
三:logstash部署(新机器)
安装jdk8
vi /etc/hosts
ip es-1
vim /etc/profile
JAVA_HOME=/usr/local/java
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME PATH
source /etc/profile
java -version
2.安装logstash
上传logstash包
vi /opt/messages.conf
执行logstash
logstash -f /opt/messages.conf
数据的测试
1)基本的输入输出
logstash :
终端输入
es查看
Kibana上
管理 -------------------下一步-------
监控nginx logs
选RPM
es-1、2、3
新机子
上传filebeat.rpm包
vi /etc/hosts
ip +es-1
vi nginx.yml
yum -y install nginx
systemctl start nginx
curl localhost
vu nginx.yml
生效
date(时间统一)
排错中(不出错不做)
干掉es
yum -y install psmisc
文章来源:https://www.toymoban.com/news/detail-727019.html
systemctl restart filebeat文章来源地址https://www.toymoban.com/news/detail-727019.html
到了这里,关于ELK集群 日志中心集群、kafka、logstash的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!