【Kubernetes资源篇】ingress-nginx最佳实践详解

这篇具有很好参考价值的文章主要介绍了【Kubernetes资源篇】ingress-nginx最佳实践详解。希望对大家有所帮助。如果存在错误或未考虑完全的地方,请大家不吝赐教,您也可以点击"举报违法"按钮提交疑问。

一、Ingress Controller理论知识

Ingress官方中文参考文档:

1、Ingress Controller、Ingress简介

Ingress Controller是一个七层负载调度器,常见的七层负载均衡器有nginx、traefik,以我们熟悉的nginx为例,客户端的请求首先会到Ingress Controller七层负载调度器,由七层负载调度器将请求代理到后端的Pod。

以Nginx举例,客户端请求首先会到Nginx中,由Nginx中的upstream模块将请求代理到后端的服务上,但是K8s场景下,后端Pod的IP地址不是固定的,因此在Pod前面需要添加一个service资源,请求到达Service,由Service代理到后端的Pod。

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

Ingress是K8S中的资源,简单理解就是Ingress Controller的配置文件,创建ingress规则在管理Ingress Controller。

2、四层代理与七层代理的区别

四层代理:

  • 工作在传输层,可以解析传输层协议,TCP、UDP等。
  • 四层代理 基于传IP+端口方式进行转发。

七层代理:

  • 工作在应有层,可以解析应用层协议,如HTTP、FTP等。
  • 七层负载工作在四层的基础之上,基于虚拟主机的URL或主机的IP进行转发。

总体而言,四层代理更关注于网络层面的流量控制和安全,主要基于传输层的信息进行处理;而七层代理更加智能,能够理解和处理应用层协议的内容,提供更加精细的控制和调度。选择使用哪种类型的代理取决于具体需求和使用场景。

OSI七层模型:

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器
722190254323.png)]

3、Ingress Controller中封装Nginx,为什么不直接用Nginx呢?

在宿主机安装Nginx,只要配置文件有改动,就必须手动reload加载才可以生效,但是如果使用Ingress Controller封装的Nginx,你ingress维护配置,ingress创建好了之后,会自动把配置文件传到Ingress Controller这个Pod中,自动进行reload加载。

4、Ingress Controller代理K8S内部Pod流程

第一步:部署Ingress Controller

第二步:创建Pod,可以使用控制器进行创建

第三步:创建Service,管理Pod

第四步:创建Ingress http或https规则

第五步:测试,客户端通过七层访问

二、实践:部署Ingress Controller高可用架构

高可用架构请求转发图:

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

ingress-nginx GitHub地址:

ingress-nginx YAML GitHub地址:

1、部署Ingress Controller

1、编写YAML文件,基于官方下载,根基自己需求进行对应修改。

cat ingress-controller-nginx.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx

---
# Source: ingress-nginx/templates/controller-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx
  namespace: ingress-nginx
automountServiceAccountToken: true
---
# Source: ingress-nginx/templates/controller-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
data:
  allow-snippet-annotations: "true"
---
# Source: ingress-nginx/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
  name: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
      - namespaces
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingressclasses
    verbs:
      - get
      - list
      - watch
---
# Source: ingress-nginx/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
  - kind: ServiceAccount
    name: ingress-nginx
    namespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - endpoints
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingressclasses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      - ingress-controller-leader
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
---
# Source: ingress-nginx/templates/controller-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
  - kind: ServiceAccount
    name: ingress-nginx
    namespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-service-webhook.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  type: ClusterIP
  ports:
    - name: https-webhook
      port: 443
      targetPort: webhook
      appProtocol: https
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  type: NodePort
  ipFamilyPolicy: SingleStack
  ipFamilies:
    - IPv4
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
      appProtocol: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https
      appProtocol: https
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/component: controller
  revisionHistoryLimit: 10
  minReadySeconds: 0
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/component: controller
    spec:
      hostNetwork: true
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/name: ingress-nginx
                topologyKey: kubernetes.io/hostname
      dnsPolicy: ClusterFirstWithHostNet
      containers:
        - name: controller
          image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.1.0
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          args:
            - /nginx-ingress-controller
            - --election-id=ingress-controller-leader
            - --controller-class=k8s.io/ingress-nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            runAsUser: 101
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          livenessProbe:
            failureThreshold: 5
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
            - name: webhook
              containerPort: 8443
              protocol: TCP
          volumeMounts:
            - name: webhook-cert
              mountPath: /usr/local/certificates/
              readOnly: true
          resources:
            requests:
              cpu: 100m
              memory: 90Mi
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
        - name: webhook-cert
          secret:
            secretName: ingress-nginx-admission
---
# Source: ingress-nginx/templates/controller-ingressclass.yaml
# We don't support namespaced ingressClass yet
# So a ClusterRole and a ClusterRoleBinding is required
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: nginx
  namespace: ingress-nginx
spec:
  controller: k8s.io/ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
# before changing this value, check the required kubernetes version
# https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  name: ingress-nginx-admission
webhooks:
  - name: validate.nginx.ingress.kubernetes.io
    matchPolicy: Equivalent
    rules:
      - apiGroups:
          - networking.k8s.io
        apiVersions:
          - v1
        operations:
          - CREATE
          - UPDATE
        resources:
          - ingresses
    failurePolicy: Fail
    sideEffects: None
    admissionReviewVersions:
      - v1
    clientConfig:
      service:
        namespace: ingress-nginx
        name: ingress-nginx-controller-admission
        path: /networking/v1/ingresses
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ingress-nginx-admission
  namespace: ingress-nginx
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
rules:
  - apiGroups:
      - admissionregistration.k8s.io
    resources:
      - validatingwebhookconfigurations
    verbs:
      - get
      - update
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
  - kind: ServiceAccount
    name: ingress-nginx-admission
    namespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: ingress-nginx-admission
  namespace: ingress-nginx
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
rules:
  - apiGroups:
      - ""
    resources:
      - secrets
    verbs:
      - get
      - create
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ingress-nginx-admission
  namespace: ingress-nginx
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
  - kind: ServiceAccount
    name: ingress-nginx-admission
    namespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: ingress-nginx-admission-create
  namespace: ingress-nginx
  annotations:
    helm.sh/hook: pre-install,pre-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
spec:
  template:
    metadata:
      name: ingress-nginx-admission-create
      labels:
        helm.sh/chart: ingress-nginx-4.0.10
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 1.1.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
    spec:
      containers:
        - name: create
          image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1
          imagePullPolicy: IfNotPresent
          args:
            - create
            - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
            - --namespace=$(POD_NAMESPACE)
            - --secret-name=ingress-nginx-admission
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          securityContext:
            allowPrivilegeEscalation: false
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx-admission
      nodeSelector:
        kubernetes.io/os: linux
      securityContext:
        runAsNonRoot: true
        runAsUser: 2000
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: ingress-nginx-admission-patch
  namespace: ingress-nginx
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-4.0.10
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
spec:
  template:
    metadata:
      name: ingress-nginx-admission-patch
      labels:
        helm.sh/chart: ingress-nginx-4.0.10
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 1.1.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
    spec:
      containers:
        - name: patch
          image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1
          imagePullPolicy: IfNotPresent
          args:
            - patch
            - --webhook-name=ingress-nginx-admission
            - --namespace=$(POD_NAMESPACE)
            - --patch-mutating=false
            - --secret-name=ingress-nginx-admission
            - --patch-failure-policy=Fail
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          securityContext:
            allowPrivilegeEscalation: false
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx-admission
      nodeSelector:
        kubernetes.io/os: linux
      securityContext:
        runAsNonRoot: true
        runAsUser: 2000

2、执行YAML文件

kubectl apply -f ingress-controller-nginx.yaml

如果执行YAML文件有报错,如下:

报错内容:Error from server (InternalError): error when creating “ingress.yaml“: Internal error occurred: fail

报错解决方法:

kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission

3、查看创建的Pod资源状态是否已运行

kubectl get pod -n ingress-nginx

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

2、在Node节点上安装并配置Nginx、keepalived

1、上面部署ingress controller分配在不同的两台Node节点(两台Node节点同步操作)

yum install  epel-release  nginx keepalived nginx-mod-stream nc -y

2、修改 nginx.conf 配置文件(两台Node节点同步操作)

mv /etc/nginx/nginx.conf{,.$(date +%F)}
vim /etc/nginx/nginx.conf

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

# 四层负载
stream {

    log_format  main  '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';

    access_log  /var/log/nginx/k8s-access.log  main;
	# 定义后端负载节点
    upstream k8s-ingress-controller {
       server 16.32.15.201:80 weight=5 max_fails=3 fail_timeout=30s;
       server 16.32.15.202:80 weight=5 max_fails=3 fail_timeout=30s;
    }
    
    # 访问30080代理到后端节点
    server {
       listen 30080; 
       proxy_pass k8s-ingress-controller;
    }
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

}

检查配置 && 启动并加入开机自启动

nginx -t 
systemctl enable nginx --now
systemctl status nginx

3、修改Keepalived Master节点配置文件(Keepalived Master操作,我这里将16.32.15.201定义为主)

mv /etc/keepalived/keepalived.conf{,.$(date +%F)}
vim /etc/keepalived/keepalived.conf
vrrp_script check_nginx {
    script "/etc/keepalived/check_nginx.sh"
}

vrrp_instance VI_1 { 
    state MASTER 
    interface ens33  # 网卡名称
    virtual_router_id 51 
    priority 100 
    advert_int 1   
    authentication { 
        auth_type PASS      
        auth_pass 1111 
    }  
    # 虚拟IP
    virtual_ipaddress { 
        16.32.15.100/24
    } 
    track_script {
        check_nginx
    } 
}

添加判断Nginx是否运行脚本

vim /etc/keepalived/check_nginx.sh
#!/bin/bash
nc -z localhost 30080
if [[ $? -ne 0 ]];then
    systemctl stop keepalived.service
fi

chmod +x /etc/keepalived/check_nginx.sh

启动主节点keepalived

systemctl enable keepalived --now

4、修改Keepalived Backup节点配置文件(Keepalived Master操作,我这里将16.32.15.202定义为备)

mv /etc/keepalived/keepalived.conf{,.$(date +%F)}
vim /etc/keepalived/keepalived.conf
vrrp_script check_nginx {
    script "/etc/keepalived/check_nginx.sh"
}

vrrp_instance VI_1 { 
    state BACKUP 
    interface ens33  # 网卡名称
    virtual_router_id 51 
    priority 90
    advert_int 1   
    authentication { 
        auth_type PASS      
        auth_pass 1111 
    }  
    # 虚拟IP
    virtual_ipaddress { 
        16.32.15.100/24
    } 
    track_script {
        check_nginx
    } 
}

添加判断Nginx是否运行脚本

vim /etc/keepalived/check_nginx.sh
#!/bin/bash
nc -z localhost 30080
if [[ $? -ne 0 ]];then
    systemctl stop keepalived.service
fi

chmod +x /etc/keepalived/check_nginx.sh

启动备节点keepalived

systemctl enable keepalived --now

3、测试主备切换

1、在主机停止nginx服务

systemctl stop nginx

2、在备机,查看VIP是否漂移过去

ip a|grep 100

如果漂移过去表示无问题,如下图:

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

3、在主机启动,VIP会自动表漂移到主机

systemctl start nginx keepalived
ip a|grep 100

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

三、实践:创建Ingress规则进行七层转发

Ingress规则官方参考文档

1、基于HTTP七层代理转发后端Pod

1、创建后端Pod、Server资源

cat ingress-demo.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: ingress-tomcat-service
  namespace: default
spec:
  selector:
    app: tomcat
  ports:
  - name: http
    targetPort: 8080
    port: 8080
  - name: ajp
    targetPort: 8009
    port: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-tomcat-deployment
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: tomcat
  template:
    metadata:
      labels:
        app: tomcat
    spec:
      containers:
      - name: tomcat
        image: tomcat:8.5.34-jre8-alpine 
        imagePullPolicy: IfNotPresent  
        ports:
        - name: http
          containerPort: 8080
          name: ajp
          containerPort: 8009

执行YAML文件:

kubectl apply -f ingress-demo.yaml

查看创建的Pod、Service

kubectl get pods,svc

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

2、创建Ingress转发规则

cat ingress-tomcat.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-tomcat
  namespace: default
spec: 
  ingressClassName: nginx     # 指定ingress类名称,这里是Nginx
  rules:
  - host: tomcat.ingress.com  # 客户端访问的域名
    http:
      paths:
      - backend:
          service:
            name: ingress-tomcat-service  # 转发到SVC名称
            port:
              number: 8080                # 转发到SVC端口
        path: /                           # 转发到/
        pathType: Prefix

执行YAML

kubectl apply -f ingress-tomcat.yaml

3、添加域名解析:

打开 C:\Windows\System32\drivers\etc\hosts 文件,添加解析,如下图:

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

浏览器访问tomcat.ingress.com:30080进行测试

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

2、基于HTTPS七层代理转发后端

基于上面 HTTP七层代理转发的 Pod、Service做实验,不在创建新的资源。

1、创建证书

生成一个私钥

openssl genrsa -out tls.key 2048

基于私钥生成根证书,并签发qinzt.ingress.com 域名

openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=qinzt.ingress.com

2、创建secret,对证书进行加密

kubectl create secret tls ingress-tomcat-secret --cert=tls.crt --key=tls.key

查看secret

kubectl describe secret ingress-tomcat-secret

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

3、创建ingress规则

cat ingress-tomcat-tls.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-tomcat-tls
  namespace: default
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    -  qinzt.ingress.com
    secretName: ingress-tomcat-secret   # secret名称
  rules:
  - host: qinzt.ingress.com
    http:
      paths:
      - path: /
        pathType:  Prefix
        backend:
         service:
           name: tomcat
           port:
            number: 8080

执行YAML文件:

kubectl apply -f ingress-tomcat-tls.yaml

4、添加域名解析:

打开 C:\Windows\System32\drivers\etc\hosts 文件,添加解析,如下图:

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

5、浏览器访问域名测试

由于证书是自签名,所有浏览器会提示不安全,点击确认继续访问即可

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器

【Kubernetes资源篇】ingress-nginx最佳实践详解,# 3-Kubernetes容器编排,kubernetes,nginx,容器文章来源地址https://www.toymoban.com/news/detail-608003.html

到了这里,关于【Kubernetes资源篇】ingress-nginx最佳实践详解的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处: 如若内容造成侵权/违法违规/事实不符,请点击违法举报进行投诉反馈,一经查实,立即删除!

领支付宝红包 赞助服务器费用

相关文章

  • 记安装ingress-nginx遇到的一些坑

    环境: Linux:CentOS Linux release 7.9.2009 (Core) Kubernetes:v1.23.5 Dokcer:20.10.14 一、安装ingress-nginx 二、修改deploy.yaml文件 1、注释掉external Traffic Policy:Local 2、因默认镜像源国内无法访问 ,修改为dockerhub上的镜像源        修改controller/deploy的image:

    2023年04月09日
    浏览(33)
  • k8s部署ingress-nginx

    k8s服务对外暴露有三种方式 NodePort,LoadBalancer,Ingress Nodeport : 服务暴露需要在集群每个节点都开放一个同样的端口,通过 nodtIp:nodePort 来访问,如果服务数量多了,开放的端口就难以管理 LoadBalancer : 大部分情况下只适用于支持外部负载均衡器的云提供商(AWS,阿里云,华为云等)

    2024年02月04日
    浏览(35)
  • k8s helm ingress-nginx Error: template: ingress-nginx/templates/controller-role “update-status“

    k8s,使用helm包管理器安装ingress-nginx时,安装文件出错 helm 版本:Version:“v3.2.3” k8s 版本:Kubernetes v1.23.6 ingress-nginx chart version:4.9.0 app version:1.9.5 前面都按网上的教程做,没有什么大问题 最后一步执行安装命令 命令执行失败,报错 问题可能出现在对.Values.controller.extraArg

    2024年03月20日
    浏览(123)
  • K8s的ingress-nginx配置https

    在另一台机器上配置hosts解析www.yaoyao.com,然后访问 curl --cacert tls.crt https://www.yaoyao.com:10443 这里的10443端口是ingress-nginx-controller服务暴露的nodeport端口

    2024年02月07日
    浏览(36)
  • K8S(1.28)--部署ingress-nginx(1.9.1)

    原文网址:K8S(1.28)--部署ingress-nginx(1.9.1)-CSDN博客 本文介绍K8S部署ingress-nginx的方法。 本文使用的K8S和ingress-nginx都是最新的版本。 官网地址 https://kubernetes.github.io/ingress-nginx/deploy/ Ingress里Nginx的代理流程: 1.确定版本 首先确定版本:https://github.com/kubernetes/ingress-nginx 我K8S是1.2

    2024年02月20日
    浏览(40)
  • K8s实战4-使用Helm在Azure上部署Ingress-Nginx和Tokengateway

    az login az account set --subscription ${sub ID} az aks get-credentials --resource-group ${groupname} --name ${aks name} curl -LO https://github.com/kubernetes/ingress-nginx/releases/download/helm-chart-4.7.1/ingress-nginx-4.7.1.tgz sudo tar -xvf ingress-nginx-4.2.5.tgz sudo cd ingress-nginx #创建命名空间 kubectl create ns ingress-nginx-public # 使用

    2024年02月12日
    浏览(39)
  • 第12关 精通K8s下的Ingress-Nginx控制器:生产环境实战配置指南

    ------ 课程视频同步分享在今日头条和B站 大家好,我是博哥爱运维,这节课带来k8s的流量入口ingress,作为业务对外服务的公网入口,它的重要性不言而喻,大家一定要仔细阅读,跟着博哥的教程一步步实操去理解。 Ingress基本概念 在Kubernetes集群中,Ingress作为集群内服务对外

    2024年02月03日
    浏览(46)
  • grafana呈现loki从k8s/ingress-nginx中收集的日志-地区经纬度部分

    1,在用grafana展示loki日志的时候,想到用可视化的方式来呈现更直观,于是网上查找,找到一篇《如何用Loki来绘制Ingress Nginx监控大屏》被复用多次,按照其过程实践引用了12559的面板,但效果与原文描述的不一致,可能是gf版本,或者插件版本变更等原因造成。尝试了几次终

    2024年02月21日
    浏览(51)
  • Kubernetes(K8s)与虚拟GPU(vGPU):实现高效管理和利用GPU资源的最佳实践

    目录 第一节:Kubernetes简介 第二节:虚拟GPU(vGPU)简介 第三节:Kubernetes中的GPU资源管理 第四节:虚拟GPU(vGPU)的部署和配置 第五节:GPU资源调度和负载均衡 第六节:GPU资源监控和调优 结论: 可先阅读一下参考: kubernetes如何将异构GPU(如NVIDIA、海光、寒武纪)统一协同

    2024年04月13日
    浏览(46)
  • kubernetes Ingress资源管理

    k8s 对外服务之 Ingress //Ingress 简介 service的作用体现在两个方面,对集群内部,它不断跟踪pod的变化,更新endpoint中对应pod的对象,提供了ip不断变化的pod的服务发现机制;对集群外部,他类似负载均衡器,可以在集群内外部对pod进行访问。 在Kubernetes中,Pod的IP地址和service的

    2024年02月16日
    浏览(64)

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

博客赞助

微信扫一扫打赏

请作者喝杯咖啡吧~博客赞助

支付宝扫一扫领取红包,优惠每天领

二维码1

领取红包

二维码2

领红包