一、概述
Elasticsearch是一个基于Lucene的搜索引擎。它提供了具有HTTP Web界面和无架构JSON文档的分布式,多租户能力的全文搜索引擎。Elasticsearch是用Java开发的,根据Apache许可条款作为开源发布。
官方文档:https://www.elastic.co/guide/en/elasticsearch/reference/master/getting-started.html
GitHub: https://github.com/elastic/elasticsearch
也可以参考我这篇文章:面试宝典之——Elasticsearch
二、Elasticsearch 编排部署
地址:https://artifacthub.io/packages/helm/elastic/elasticsearch
1)添加源并下载编排部署包
helm repo add elastic https://helm.elastic.co
helm pull elastic/elasticsearch --version 7.17.3
tar -xf elasticsearch-7.17.3.tgz
2)构建镜像
Elasticsearch 各版本下载地址:https://www.elastic.co/cn/downloads/past-releases#elasticsearch
这里就不重新构建镜像了,有不知道怎么构建镜像的小伙伴可以给我留言或私信,这里是将远程的镜像推送到我们本地harbor,加速拉取镜像。
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.17.3
docker tag docker.elastic.co/elasticsearch/elasticsearch:7.17.3 myharbor.com/bigdata/elasticsearch:7.17.3
# 上传镜像
docker push myharbor.com/bigdata/elasticsearch:7.17.3
# 删除镜像
docker rmi myharbor.com/bigdata/elasticsearch:7.17.3
crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3
3)修改yaml编排
elasticsearch/values.yaml
image: "myharbor.com/bigdata/elasticsearch"
...
...
### 去掉这几行
volumeClaimTemplate:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 30Gi
....
persistence:
enabled: true
labels:
# Add default labels for the volumeClaimTemplate of the StatefulSet
enabled: false
annotations: {}
accessModes:
- ReadWriteOnce
size: 1Gi
storageClass: "elasticsearch-local-storage"
local:
- name: elasticsearch-0
host: "local-168-182-110"
path: "/opt/bigdata/servers/elasticsearch/data/data1"
- name: elasticsearch-1
host: "local-168-182-111"
path: "/opt/bigdata/servers/elasticsearch/data/data1"
- name: elasticsearch-2
host: "local-168-182-112"
path: "/opt/bigdata/servers/elasticsearch/data/data1"
...
protocol: http
httpPort: 9200
transportPort: 9300
service:
enabled: true
type: NodePort
nodePort: 30920
httpPortName: http
elasticsearch/templates/storage-class.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: {{ .Values.persistence.storageClass }}
provisioner: kubernetes.io/no-provisioner
elasticsearch/templates/pv.yaml
{{- range .Values.persistence.local }}
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .name }}
labels:
name: {{ .name }}
spec:
storageClassName: {{ $.Values.persistence.storageClass }}
capacity:
storage: {{ $.Values.persistence.size }}
accessModes:
{{- range $.Values.persistence.accessModes }}
- {{ . | quote }}
{{- end }}
local:
path: {{ .path }}
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- {{ .host }}
---
{{- end }}
elasticsearch/templates/statefulset.yaml
spec:
volumeClaimTemplates:
spec:
# 去掉这行
{{ toYaml .Values.volumeClaimTemplate | indent 6 }}
# 新增以下内容:
accessModes:
{{- range .Values.persistence.accessModes }}
- {{ . | quote }}
{{- end }}
resources:
requests:
storage: {{ .Values.persistence.size | quote }}
{{- if .Values.persistence.storageClass }}
{{- if (eq "-" .Values.persistence.storageClass) }}
storageClassName: ""
{{- else }}
storageClassName: "{{ .Values.persistence.storageClass }}"
{{- end }}
{{- end }}
4)开始部署
# 先创建本地存储目录
mkdir -p /opt/bigdata/servers/elasticsearch/data/data1
chmod -R 777 /opt/bigdata/servers/elasticsearch/data/data1
helm install my-elasticsearch ./elasticsearch -n elasticsearch --create-namespace
# 查看
helm get notes my-elasticsearch -n elasticsearch
kubectl get pods,svc -n elasticsearch -owide
NOTES
NAME: my-elasticsearch
LAST DEPLOYED: Wed Oct 12 23:47:17 2022
NAMESPACE: elasticsearch
STATUS: deployed
REVISION: 1
NOTES:
1. Watch all cluster members come up.
$ kubectl get pods --namespace=elasticsearch -l app=elasticsearch-master -w2. Test cluster health using Helm test.
$ helm --namespace=elasticsearch test my-elasticsearch
5)测试验证
curl http://192.168.182.110:30920/
curl http://192.168.182.110:30920/_cat/nodes
curl http://192.168.182.110:30920/_cat/health?pretty
6)elasticsearch-head
elasticsearch-head GitHub下载地址:https://github.com/mobz/elasticsearch-head
Google 浏览器 elasticsearch-head插件:
链接:https://pan.baidu.com/s/1kYcTjBDPmSWVzsku2hEW7w?pwd=67v4
提取码:67v4
7)卸载
helm uninstall my-elasticsearch -n elasticsearch
kubectl delete ns elasticsearch --force
rm -fr /opt/bigdata/servers/elasticsearch/data/data1/*
ssh local-168-182-111 "rm -fr /opt/bigdata/servers/elasticsearch/data/data1/*"
ssh local-168-182-112 "rm -fr /opt/bigdata/servers/elasticsearch/data/data1/*"
docker rmi myharbor.com/bigdata/elasticsearch:7.17.3
crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3
ssh local-168-182-111 "crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3"
ssh local-168-182-112 "crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3"
三、Kibana 编排部署
地址:https://artifacthub.io/packages/helm/bitnami/kibana?modal=install
1)添加源并下载编排部署包
helm repo add bitnami https://charts.bitnami.com/bitnami
helm pull bitnami/kibana --version 10.2.6
tar -xf kibana-10.2.6.tgz
2)构建镜像
这里也不重新构建镜像了,只是将镜像推送到本地harbor加速,对构建镜像不清楚的可以留言或私信。【注意】版本号需要跟ES相对应,版本不一样,可能会存储不兼容的情况。
docker pull docker.io/bitnami/kibana:7.17.3
docker tag docker.io/bitnami/kibana:7.17.3 myharbor.com/bigdata/kibana:7.17.3
# 上传镜像
docker push myharbor.com/bigdata/kibana:7.17.3
# 删除镜像
docker rmi myharbor.com/bigdata/kibana:7.17.3
crictl rmi myharbor.com/bigdata/kibana:7.17.3
3)修改yaml编排
kibana/values.yaml
image:
registry: myharbor.com
repository: bigdata/kibana
tag: 8.4.3-debian-11-r1
...
replicaCount: 1
...
persistence:
enabled: true
accessModes:
- ReadWriteOnce
size: 1Gi
storageClass: "kibana-local-storage"
local:
- name: kibana-0
host: "local-168-182-111"
path: "/opt/bigdata/servers/kibana/data/data1"
...
service:
ports:
http: 5601
type: NodePort
nodePorts:
http: "30601"
...
elasticsearch:
hosts:
- elasticsearch-master.elasticsearch
port: "9200"
kibana/templates/values.yaml
{{- range .Values.persistence.local }}
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .name }}
labels:
name: {{ .name }}
spec:
storageClassName: {{ $.Values.persistence.storageClass }}
capacity:
storage: {{ $.Values.persistence.size }}
accessModes:
{{- range $.Values.persistence.accessModes }}
- {{ . | quote }}
{{- end }}
local:
path: {{ .path }}
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- {{ .host }}
---
{{- end }}
kibana/templates/storage-class.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: {{ .Values.persistence.storageClass }}
provisioner: kubernetes.io/no-provisioner
kibana/templates/pv.yaml
{{- range .Values.persistence.local }}
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .name }}
labels:
name: {{ .name }}
spec:
storageClassName: {{ $.Values.persistence.storageClass }}
capacity:
storage: {{ $.Values.persistence.size }}
accessModes:
{{- range $.Values.persistence.accessModes }}
- {{ . | quote }}
{{- end }}
local:
path: {{ .path }}
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- {{ .host }}
---
{{- end }}
4)开始部署
# 先创建本地存储目录
mkdir -p /opt/bigdata/servers/kibana/data/data1
chmod -R 777 /opt/bigdata/servers/kibana/data/data1
helm install my-kibana ./kibana -n kibana --create-namespace
# 查看
helm get notes my-kibana -n kibana
kubectl get pods,svc -n kibana -owide
NOTES
NAME: my-kibana
LAST DEPLOYED: Thu Oct 13 22:43:30 2022
NAMESPACE: kibana
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: kibana
CHART VERSION: 10.2.6
APP VERSION: 8.4.3
** Please be patient while the chart is being deployed **######################################################################################################
### ERROR: You did not provide the Elasticsearch external host or port in your 'helm install' call ###
######################################################################################################
Complete your Kibana deployment by running:
helm upgrade --namespace kibana my-kibana my-repo/kibana \
--set elasticsearch.hosts[0]=YOUR_ES_HOST,elasticsearch.port=YOUR_ES_PORT
Replacing "YOUR_ES_HOST" and "YOUR_ES_PORT" placeholders by the proper values of your Elasticsearch deployment.
5)测试验证
web:http://192.168.182.111:30601
6)卸载
helm uninstall my-kibana -n kibana
kubectl delete ns kibana --force
ssh local-168-182-111 rm -fr /opt/bigdata/servers/kibana/data/data1/*
docker rmi myharbor.com/bigdata/bigdata:8.4.3-debian-11-r1
crictl rmi myharbor.com/bigdata/bigdata:8.4.3-debian-11-r1
elasticsearch-on-k8s 下载地址:https://gitee.com/hadoop-bigdata/elasticsearch-on-k8s
kibana-on-k8s 下载地址:https://gitee.com/hadoop-bigdata/kibana-on-k8s文章来源:https://www.toymoban.com/news/detail-402065.html
Elasticsearch + kibana on k8s 编排部署讲解与实战操作就先到这里了,这里只是简单的演示了查询,更多的操作,可以参考官方文档,有疑问的小伙欢迎给我留言,后续会持续更新【云原生+大数据】相关的文章~文章来源地址https://www.toymoban.com/news/detail-402065.html
到了这里,关于【云原生】Elasticsearch + kibana on k8s 讲解与实战操作的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!