kubernetes集群部署springcloud项目【AL】

kubernetes集群部署springcloud项目【AL】 （先手工做，非自动化）

#环境：
192.168.73.138 master 
192.168.73.139 node1 
192.168.73.140 node2 
192.168.73.137 harbor、mysql

git clone https://github.com/lizhenliang/simple-microservice.git 

地址：https://github.com/lizhenliang/simple-microservice/   
代码分支说明：
dev1交付代码
dev2 编写Dockerfile构建镜像
dev3 K8S资源编排
dev4 微服务链路监控
master 最终上线

整体逻辑架构图

1、代码文件解释

1、服务编排
2、在K8S平台部署Erueka集群
3、导入数据库文件到MySQL
4、部署网关(gateway)
5、部署业务程序(product、stock.order)
6、部署前端(portal)

部署须知
1、导入db目录下数据库文件到自己的MySQL服务器
2、修改配置环境（xxx-service/src/main/resources/application.yml，active值决定启用环境配置文件）
3、修改连接数据库配置（xxx-service/src/main/resources/application-fat.yml）
4、修改前端页面连接网关地址（portal-service/src/main/resources/static/js/productList.js和orderList.js）
5、服务启动顺序：eureka -> mysql -> product,stock,order -> gateway -> portal

2、代码编译构建，打包镜像

#上图的-P prod ，可指定对应的配置文件
mvn clean package -Dmaven.test.skip=true

#通过dockerfile ,打包镜像，上传至harbor镜像仓库

2.1 dev1分支的文件 （交付代码）：

2.2 gateway-service 网关服务

tips：

不同环境配置文件怎么区分?
#1
java -jar -Dspring.profiles.active=dev xxx.jar

#2 
使用配置中心，如Apollo、disconf

2.3 product-service 商品服务

看biz的目录即可， API目录里面是一些接口工具

2.4 eureka-service 注册中心服务

其他的文件夹要看的关键文件，以此类推。

3 、 dev2 编写Dockerfile构建镜像。 以及在最外层的目录pom.xml执行maven打包

#整体打包。 此时每个微服务的文件夹，会生成target。 target文件夹下会有对应的jar包。
mvn clean package -Dmaven.test.skip=true

target下的jar包

3.1 每个微服务都有个dockerfile

docker build -t product .

4 、启动harbor镜像仓库

docker-compose up -d

dev3 批量打包docker镜像脚本

docker_build.sh
tips:执行前需docker login 192.168.31.70

#执行操作
./docker_build.sh

#!/bin/bash

docker_registry=192.168.31.70
# 存储登录Harbor认证信息
kubectl create secret docker-registry registry-pull-secret --docker-server=$docker_registry --docker-username=admin --docker-password=Harbor12345 --docker-email=admin@ctnrs.com -n ms

service_list="eureka-service gateway-service order-service product-service stock-service portal-service"
#参数1的值如果不存在，则使用-:后面的值，$1有值那就用$1的值 。 具体参考shell expansion
service_list=${1:-${service_list}}
work_dir=$(dirname $PWD)
current_dir=$PWD

cd $work_dir
mvn clean package -Dmaven.test.skip=true

for service in $service_list; do
   cd $work_dir/$service
   # 业务程序需进入biz目录里构建。 
   if ls |grep biz &>/dev/null; then
      cd ${service}-biz
   fi
   #取service的名字。eureka-service会变成eureka   ；      具体参考shell expansion
   service=${service%-*}
   image_name=$docker_registry/microservice/${service}:$(date +%F-%H-%M-%S)
   docker build -t ${image_name} .
   docker push ${image_name} 
   # 修改yaml中镜像地址为新推送的，并apply。 如果将下面两行注释，则不会部署到k8s中。 效果就是纯打包docker镜像，推送docker镜像。
   sed -i -r "s#(image: )(.*)#\1$image_name#" ${current_dir}/${service}.yaml
   kubectl apply -f ${current_dir}/${service}.yaml
done

脚本执行完后，仓库结果，6个镜像：

5、 准备工作

1、服务编排
2、在K8S平台部署Erueka集群
3、导入数据库文件到mysql
4、部署网关(api-gateway)
5、部署业务程序(product、stock、order)
6、部署前端(portal)

5.1 mysql数据库的准备（略写）

导入DB文件夹里面的sql文件即可 （order.sql 、product.sql、 stock.sql）

项目架构图.jpg

配置文件（写在最后）:

1、eureka-service   8888
2、gateway-service  9999
3、portal-service   8080
4、order-service    8020
5、product-service  8010
6、stock-service    8030

这里是没对接自动化的软件的配置。

1、全部微服务的dockerfile

1、eureka-service

FROM java:8-jdk-alpine
RUN  sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories && \
     apk add -U tzdata && \
     ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
COPY ./target/eureka-service.jar ./
EXPOSE 8888
# #注意这里注入的变量MY_POD_NAME。  对应的dockerfile启动命令有所关联。 pod启动时，会有固定的名称: eureka-0/1/2
#可以启动本pod后执行 hostname -f ，看具体的FQDN：(Fully Qualified Domain Name)全限定域名。 启动后能相互注册。
CMD java -jar -Deureka.instance.hostname=${MY_POD_NAME}.eureka.ms /eureka-service.jar

2、gateway-service

FROM java:8-jdk-alpine
RUN  sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories && \
     apk add -U tzdata && \
     ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
COPY ./target/gateway-service.jar ./
EXPOSE 9999
CMD java -jar /gateway-service.jar

3、portal-service

FROM java:8-jdk-alpine
RUN  sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories && \
     apk add -U tzdata && \
     ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
COPY ./target/portal-service.jar ./
EXPOSE 8080
CMD java -jar /portal-service.jar

4、order-service

FROM java:8-jdk-alpine
RUN  sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories && \
     apk add -U tzdata && \
     ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
COPY ./target/order-service-biz.jar ./
EXPOSE 8020
CMD java -jar /order-service-biz.jar

5、product-service

FROM java:8-jdk-alpine
RUN  sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories && \
     apk add -U tzdata && \
     ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
COPY ./target/product-service-biz.jar ./
EXPOSE 8010
CMD java -jar /product-service-biz.jar

6、stock-service

FROM java:8-jdk-alpine
RUN  sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories && \
     apk add -U tzdata && \
     ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
COPY ./target/stock-service-biz.jar ./
EXPOSE 8030
CMD java -jar /stock-service-biz.jar

2、全部微服务的yaml

1、eureka-service   Ingress、Service、StatefulSet
2、gateway-service  Ingress、Service、Deployment （对外暴露）
3、portal-service   Ingress、Service、Deployment （对外暴露）
4、order-service    Deployment
5、product-service  Deployment
6、stock-service    Deployment
#根据 “项目架构图.jpg” 内部的order 、product、stock不需要写service。 因为已注册到eureka中。 APIgateway会调用内部的微服务，反馈给用户。

部署顺序

#手动操作。非自动化
kubectl create secret docker-registry registry-pull-secret --docker-server=$docker_registry --docker-username=admin --docker-password=Harbor12345 --docker-email=admin@ctnrs.com -n ms

kubectl apply -f eureka.yaml
kubectl apply -f gateway.yaml

kubectl apply -f portal.yaml
kubectl apply -f stock.yaml
kubectl apply -f product.yaml
kubectl apply -f order.yaml

一些执行后的图：

kubectl exec -it eureka-0 sh -n ms
nslookup eureka
nslookup eureka-0.eureka.ms

eureka界面：

其他微服务注册进来的信息：

ms命名空间下的微服务启动情况

kubectl get pods -n ms

修改hosts文件，映射IP与域名（略）

浏览器访问项目效果：

portal.ctnrs.com

查询商品：

查询订单服务：

小结

第一步:熟悉Spring Cloud微服务项目
第二步:源代码编译构建
第三步:构建项目镜像并推送到镜像仓库
第四步:在K8S中部署Spring Cloud微服务项目的逻辑架构
第五步:K8S服务编排

第六步:在k8S中部署Eureka集群（注册中心)
第七步:部署微服务网关服务
第八步:部署微服务业务程序
第九步:部署微服务前端
第十步:微服务扩容与发布

在K8S中部署Eureka集群

deployment的解释。 具体可参考 kubectl explain

1、eureka-service （Ingress、Service、StatefulSet）

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: eureka 
  namespace: ms 
spec:
  rules:
    - host: eureka.aliangedu.cn 
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service: 
              name: eureka
              port: 
                number: 8888
---
apiVersion: v1
kind: Service
metadata:
  name: eureka
  namespace: ms
spec:
  clusterIP: None
  ports:
  - port: 8888
    name: eureka 
  selector:
    project: ms
    app: eureka

---

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: eureka
  namespace: ms 
spec:
  replicas: 3
  selector:
    matchLabels:
      project: ms
      app: eureka
  serviceName: "eureka"
  template:
    metadata:
      labels:
        project: ms 
        app: eureka
    spec:
      imagePullSecrets:
      - name: registry-pull-secret
      containers:
      - name: eureka
        image: 192.168.31.70/microservice/eureka:2019-07-10-21-33-14
        ports:
          - protocol: TCP
            containerPort: 8888
        #注意这里注入的变量MY_POD_NAME。  对应的dockerfile启动命令有所关联。 pod启动时，会有固定的名称: eureka-0/1/2
        #可以启动本pod后执行 hostname -f ，看具体的FQDN：(Fully Qualified Domain Name)全限定域名。 启动后能相互注册。
        env:
          - name: MY_POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
        resources:
          requests:
            cpu: 0.5
            memory: 256Mi
          limits:
            cpu: 1 
            memory: 1Gi
        readinessProbe:
          tcpSocket:
            port: 8888
          initialDelaySeconds: 60
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 8888
          initialDelaySeconds: 60
          periodSeconds: 10

2、gateway-service （Ingress、Service、Deployment）

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gateway 
  namespace: ms 
spec:
  rules:
    - host: gateway.aliangedu.cn 
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service: 
              name: gateway
              port: 
                number: 9999
---
apiVersion: v1
kind: Service
metadata:
  name: gateway
  namespace: ms
spec:
  ports:
  - port: 9999 
    name: gateway
  selector:
    project: ms
    app: gateway
---
apiVersion: apps/v1
kind: Deployment 
metadata:
  name: gateway
  namespace: ms 
spec:
  replicas: 2
  selector:
    matchLabels:
      project: ms
      app: gateway
  template:
    metadata:
      labels:
        project: ms 
        app: gateway
    spec:
      imagePullSecrets:
      - name: registry-pull-secret
      containers:
      - name: gateway
        image: 192.168.31.70/microservice/gateway:2019-07-10-21-26-23
        imagePullPolicy: Always
        ports:
          - protocol: TCP
            containerPort: 9999 
        resources:
          requests:
            cpu: 0.5
            memory: 256Mi
          limits:
            cpu: 1
            memory: 1Gi
        readinessProbe:
          tcpSocket:
            port: 9999
          initialDelaySeconds: 60
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 9999
          initialDelaySeconds: 60
          periodSeconds: 10

3、portal-service （Ingress、Service、Deployment）

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: portal 
  namespace: ms 
spec:
  rules:
    - host: portal.aliangedu.cn
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service: 
              name: portal
              port: 
                number: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: portal
  namespace: ms
spec:
  ports:
  - port: 8080
    name: portal 
  selector:
    project: ms
    app: portal
---
apiVersion: apps/v1
kind: Deployment 
metadata:
  name: portal
  namespace: ms 
spec:
  replicas: 2
  selector:
    matchLabels:
      project: ms
      app: portal
  template:
    metadata:
      labels:
        project: ms 
        app: portal
    spec:
      imagePullSecrets:
      - name: registry-pull-secret
      containers:
      - name: portal
        image: 192.168.31.70/microservice/portal:2019-07-10-21-35-13
        imagePullPolicy: Always
        ports:
          - protocol: TCP
            containerPort: 8080 
        resources:
          requests:
            cpu: 0.5
            memory: 256Mi
          limits:
            cpu: 1
            memory: 1Gi
        readinessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 60
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 60
          periodSeconds: 10

4、order-service （Deployment）

apiVersion: apps/v1
kind: Deployment 
metadata:
  name: order
  namespace: ms 
spec:
  replicas: 2
  selector:
    matchLabels:
      project: ms
      app: order
  template:
    metadata:
      labels:
        project: ms 
        app: order
    spec:
      imagePullSecrets:
      - name: registry-pull-secret
      containers:
      - name: order
        image: 192.168.31.70/microservice/order:2019-07-10-21-34-08
        imagePullPolicy: Always
        ports:
          - protocol: TCP
            containerPort: 8020 
        resources:
          requests:
            cpu: 0.5
            memory: 256Mi
          limits:
            cpu: 1
            memory: 1Gi
        readinessProbe:
          tcpSocket:
            port: 8020
          initialDelaySeconds: 60
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 8020
          initialDelaySeconds: 60
          periodSeconds: 10

5、product-service （Deployment）

apiVersion: apps/v1
kind: Deployment 
metadata:
  name: product
  namespace: ms 
spec:
  replicas: 2
  selector:
    matchLabels:
      project: ms
      app: product
  template:
    metadata:
      labels:
        project: ms 
        app: product
    spec:
      imagePullSecrets:
      - name: registry-pull-secret
      containers:
      - name: product
        image: 192.168.31.70/microservice/product:2019-07-10-21-34-34
        imagePullPolicy: Always
        ports:
          - protocol: TCP
            containerPort: 8010 
        resources:
          requests:
            cpu: 0.5
            memory: 256Mi
          limits:
            cpu: 1
            memory: 1Gi
        readinessProbe:
          tcpSocket:
            port: 8010
          initialDelaySeconds: 60
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 8010
          initialDelaySeconds: 60
          periodSeconds: 10

6、stock-service

apiVersion: apps/v1
kind: Deployment 
metadata:
  name: stock
  namespace: ms 
spec:
  replicas: 2
  selector:
    matchLabels:
      project: ms
      app: stock
  template:
    metadata:
      labels:
        project: ms 
        app: stock
    spec:
      imagePullSecrets:
      - name: registry-pull-secret
      containers:
      - name: stock
        image: 192.168.31.70/microservice/stock:2019-07-10-20-41-44
        imagePullPolicy: Always
        ports:
          - protocol: TCP
            containerPort: 8030
        resources:
          requests:
            cpu: 0.5
            memory: 256Mi
          limits:
            cpu: 1
            memory: 1Gi
        readinessProbe:
          tcpSocket:
            port: 8030
          initialDelaySeconds: 60
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 8030
          initialDelaySeconds: 60
          periodSeconds: 10

3、全部微服务的配置文件 application.yml

1、eureka-service

application.yml

server:
  port: 8888
spring:
  application:
    name: eureka-server
  profiles:
    active: fat

application-fat.yml

eureka:
  server:
    renewal-percent-threshold: 0.9
    enable-self-preservation: false
    eviction-interval-timer-in-ms: 40000
  instance:
    hostname: 127.0.0.1
    prefer-ip-address: false
  client:
    register-with-eureka: true
    serviceUrl:
      defaultZone: http://eureka-0.eureka.ms:${server.port}/eureka/,http://eureka-1.eureka.ms:${server.port}/eureka/,http://eureka-2.eureka.ms:${server.port}/eureka/
    fetch-registry: true

2、gateway-service

application.yml

server:
  port: 9999
spring:
  profiles:
    active: fat
  application:
    name: @artifactId@

application-fat.yml

spring:
  cloud:
    gateway:
      discovery:
        locator:
          #开启以服务id去注册中心上获取转发地址
          enabled: true
          ##小写serviceId
          lower-case-service-id: true
      routes:
        - id: product-service
          uri: lb://product-service
          filters:
            - StripPrefix=1
          predicates:
            - Path=/product/**

        - id: order-service
          uri: lb://order-service
          filters:
            - StripPrefix=1
          predicates:
            - Path=/order/**

        - id: stock-service
          uri: lb://stock-service
          filters:
            - StripPrefix=1
          predicates:
            - Path=/stock/**

eureka:
  instance:
    prefer-ip-address: true
  client:
    register-with-eureka: true
    fetch-registry: true
    service-url:
      defaultZone: http://eureka-0.eureka.ms:8888/eureka,http://eureka-1.eureka.ms:8888/eureka,http://eureka-2.eureka.ms:8888/eureka

3、portal-service

application.yml

server:
  port: 8080
  undertow:
    io-threads: 16
    worker-threads: 256
    buffer-size: 1024
    direct-buffers: true
spring:
  application:
    name: portal-service
  profiles:
    active: fat

application-fat.yml

eureka:
  instance:
    prefer-ip-address: true
  client:
    service-url:
      defaultZone: http://eureka-0.eureka.ms:8888/eureka,http://eureka-1.eureka.ms:8888/eureka,http://eureka-2.eureka.ms:8888/eureka
    register-with-eureka: true
    fetch-registry: true

spring:
  freemarker:
    allow-request-override: false
    allow-session-override: false
    cache: true
    charset: UTF-8
    check-template-location: true
    content-type: text/html
    enabled: true
    expose-request-attributes: false
    expose-session-attributes: false
    expose-spring-macro-helpers: true
    prefer-file-system-access: true
    suffix: .ftl
    template-loader-path: classpath:/templates/

4、order-service （有数据库的配置）

application.yml

server:
  port: 8020
spring:
  profiles:
    active: fat
  application:
    name: order-service

application-fat.yml

spring:
  datasource:
    url: jdbc:mysql://192.168.31.70:3306/tb_order?characterEncoding=utf-8
    username: root
    password: 123456
    driver-class-name: com.mysql.jdbc.Driver

eureka:
  instance:
    prefer-ip-address: true
  client:
    register-with-eureka: true
    fetch-registry: true
    service-url:
      defaultZone: http://eureka-0.eureka.ms:8888/eureka,http://eureka-1.eureka.ms:8888/eureka,http://eureka-2.eureka.ms:8888/eureka

5、product-service

application.yml

server:
  port: 8010
spring:
  profiles:
    active: fat
  application:
    name: product-service

application-fat.yml

spring:
  datasource:
    url: jdbc:mysql://192.168.31.70:3306/tb_product?characterEncoding=utf-8
    username: root
    password: 123456
    driver-class-name: com.mysql.jdbc.Driver

eureka:
  instance:
    prefer-ip-address: true
  client:
    register-with-eureka: true
    fetch-registry: true
    service-url:
      defaultZone: http://eureka-0.eureka.ms:8888/eureka,http://eureka-1.eureka.ms:8888/eureka,http://eureka-2.eureka.ms:8888/eureka

6、stock-service

application.yml

server:
  port: 8030
spring:
  profiles:
    active: fat
  application:
    name: stock-service

application-fat.yml

spring:
  datasource:
    url: jdbc:mysql://192.168.31.70:3306/tb_stock?characterEncoding=utf-8
    username: root
    password: 123456
    driver-class-name: com.mysql.jdbc.Driver

eureka:
  instance:
    prefer-ip-address: true
  client:
    register-with-eureka: true
    fetch-registry: true
    service-url:
      defaultZone: http://eureka-0.eureka.ms:8888/eureka,http://eureka-1.eureka.ms:8888/eureka,http://eureka-2.eureka.ms:8888/eureka

参考：
https://cloud.tencent.com/developer/article/1702154
https://blog.csdn.net/weixin_45623111/article/details/105215255 《k8s部署springcloud》文章来源地址https://www.toymoban.com/news/detail-628751.html

到了这里，关于kubernetes集群部署springcloud项目【AL】的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！