IaC基础设施即代码：Terraform 创建ACK集群 与部署应用

目录

 一、实验

1.环境

2.Terraform 创建网络资源

3. 阿里云给RAM添加权限

4.Terraform 创建 ACK集群

5.在ACK集群中部署应用

6.销毁资源

二、问题

1.Terraform 验证失败

2.Terraform申请资源失败

 文章来源地址https://www.toymoban.com/news/detail-821889.html

 

 一、实验

1.环境

（1）主机

表1-1 主机

2.Terraform 创建网络资源

（1）查看项目

（2）网络配置文件

network.tf 

//VPC 专有网络
resource "alicloud_vpc" "vpc" {
  vpc_name   = "k8s_vpc"
  cidr_block = "172.16.0.0/12"
}

//switch 交换机
resource "alicloud_vswitch" "vsw" {
  vpc_id     = alicloud_vpc.vpc.id
  cidr_block = "172.16.0.0/16"
  zone_id    = "cn-hangzhou-j"
}

(3) 版本配置文件

versions.tf

terraform {
  required_providers {
    alicloud = {
      source  = "aliyun/alicloud"
      version = "1.214.1"
    }
  }
}

# Configure the Alicloud Provider 默认供应商
provider "alicloud" {
  access_key = var.access_key
  secret_key = var.secret_key
  region     = "cn-hangzhou"
}

（4）变量配置文件

variables.tf

variable "access_key" {
  description = "access_key"

}

variable "secret_key" {
  description = "secret_key"
}

(5) 密钥配置文件

terraform.tfvars

（6）初始化

terraform init

（7）格式化代码

terraform fmt

（8）验证代码

terraform validate

（9）计划与预览

 terraform plan

（10）申请资源

terraform apply

（11）登录阿里云系统查看VPC

VPC已新增1个 （cn-hangzhou）

交换机已新增1个 （cn-hangzhou）

3. 阿里云给RAM添加权限

（1）AliyunCSFullAcess

（2）AliyunApiGatewayFullAcess

（3）NATGatewayFullAcess

4.Terraform 创建 ACK集群

（1）查看alicloud provider 示例

Terraform Registry

托管版K8S 示例

……

resource "alicloud_cs_managed_kubernetes" "k8s" {
  name         = var.name
  cluster_spec = "ack.pro.small"
  # version can not be defined in variables.tf.
  version            = "1.26.3-aliyun.1"
  worker_vswitch_ids = length(var.vswitch_ids) > 0 ? split(",", join(",", var.vswitch_ids)) : length(var.vswitch_cidrs) < 1 ? [] : split(",", join(",", alicloud_vswitch.vswitches.*.id))
  pod_vswitch_ids    = length(var.terway_vswitch_ids) > 0 ? split(",", join(",", var.terway_vswitch_ids)) : length(var.terway_vswitch_cidrs) < 1 ? [] : split(",", join(",", alicloud_vswitch.terway_vswitches.*.id))
  new_nat_gateway    = true
  node_cidr_mask     = var.node_cidr_mask
  proxy_mode         = var.proxy_mode
  service_cidr       = var.service_cidr

  dynamic "addons" {
    for_each = var.cluster_addons
    content {
      name   = lookup(addons.value, "name", var.cluster_addons)
      config = lookup(addons.value, "config", var.cluster_addons)
    }
  }
}

(2) 修改主配置文件

main.tf

locals {
  cluster_version = "1.26.3-aliyun.1"
  service_cidr    = "192.168.0.0/16"
  pod_cidr        = "10.212.0.0/16"
}

resource "alicloud_cs_managed_kubernetes" "k8s" {
  name                 = var.cluster_name
  version              = local.cluster_version  
  cluster_spec         = "ack.standard"
  worker_vswitch_ids   = [alicloud_vswitch.vsw.id] 
  new_nat_gateway      = true
  pod_cidr             = local.service_cidr
  service_cidr         = local.pod_cidr
  load_balancer_spec   = "slb.s1.small"
  slb_internet_enabled = true

  dynamic "addons" {
    for_each = var.cluster_addons
    content {
      name   = lookup(addons.value, "name", var.cluster_addons)
      config = lookup(addons.value, "config", var.cluster_addons)
    }
  }
}

resource "alicloud_cs_kubernetes_node_pool" "default" {
  name                 = var.nodepool_name
  cluster_id           = alicloud_cs_managed_kubernetes.k8s.id
  vswitch_ids          = [alicloud_vswitch.vsw.id]
  instance_types       = ["ecs.g6.xlarge"]
  system_disk_category = "cloud_efficiency"
  system_disk_size     = 40
  desired_size         = 1
  password             = "Admin@123"
  runtime_name         = "containerd"
  runtime_version      = "1.6.20"
}

(3) 修改变量配置文件

variables.tf

variable "access_key" {
  description = "access_key"

}

variable "secret_key" {
  description = "secret_key"
}

variable "cluster_name" {
  default = "k8s_cluster_01"
}

variable "nodepool_name" {
  default = "k8s-nodepool"
}

variable "cluster_addons" {
  type = list(object({
    name   = string
    config = string
  }))

  default = [
    {
      "name"   = "flannel",
      "config" = "",
    },
    {
      "name"   = "csi-plugin",
      "config" = "",
    },
    {
      "name"   = "csi-provisioner",
      "config" = "",
    },
    {
      "name"   = "logtail-ds",
      "config" = "{'IngressDashboardEnabled':'true'}",
    },
    {
      "name"   = "nginx-ingress-controller",
      "config" = "{'IngressSlbNetworkType':'internet'}",
    },
    {
      "name"   = "arms-prometheus",
      "config" = "",
    },
    {
      "name"   = "ack-node-problem-detector",
      "config" = "{'sls_project_name':''}",
    }
  ]
}

(4)  验证代码

terraform validate

(5) 计划与预览

 terraform plan

(6)申请资源

terraform apply

yes ,用时大约6分钟

(7) 登录阿里云系统查看ACK集群

初始化中

运行中

（8）查看节点池

节点池

伸缩活动

（9）查看命名空间

（10）查看网络

服务 service

5.在ACK集群中部署应用

（1）查看目录

（2）Terraform模板(docker)

Terraform Registry

USE PROVIDER

terraform {
  required_providers {
    kubernetes = {
      source = "hashicorp/kubernetes"
      version = "2.25.2"
    }
  }
}

provider "kubernetes" {
  # Configuration options
}

(3)下载软件包

https://github.com/hashicorp/terraform-provider-kubernetes/releases

（3）修改K8S集群配置文件

阿里云系统查看连接集群信息

复制上面的连接集群信息到clustera.config

（5）修改主配置文件

provider "kubernetes" {
  # Configuration options
  config_path    = "../config/clustera.config"
  config_context = "kubernetes-admin-c718a5ce282f94d539ee5ce1986370194"
  alias          = "clustera"
  insecure       = true
}

resource "kubernetes_namespace" "jenkins" {
  provider = kubernetes.clustera
  metadata {
    name = "devops"
  }
}

（6）修改版本配置文件

terraform {
  required_providers {
    kubernetes = {
      source = "hashicorp/kubernetes"
      version = "2.25.2"
    }
  }
}

provider "kubernetes" {
  # Configuration options
}

（7）修改输出配置文件

output "service_name" {
  value = kubernetes_service_v1.jenkins.metadata[0].name

}

（8）修改服务配置文件

jenkins.tf

resource "kubernetes_deployment_v1" "jenkins" {
  provider = kubernetes.clustera
  metadata {
    name = "jenkins"
    labels = {
      app = "jenkins"
    }
    namespace = kubernetes_namespace.jenkins.id
  }

  spec {
    replicas = 1

    selector {
      match_labels = {
        app = "jenkins"
      }
    }

    template {
      metadata {
        labels = {
          app = "jenkins"
        }
      }

      spec {
        container {
          image             = "jenkins/jenkins:latest"
          name              = "jenkins"
          image_pull_policy = "IfNotPresent"

          port {
            container_port = 8080
          }

          resources {
            limits = {
              cpu    = "1000m"
              memory = "4096Mi"
            }
            requests = {
              cpu    = "250m"
              memory = "1024Mi"
            }
          }
          #   liveness_probe {
          #     http_get {
          #       path = "/"
          #       port = 8080
          #     }
          #     initial_delay_seconds = 30
          #     period_seconds        = 3
          #   }
        }
      }
    }
  }
}


resource "kubernetes_service_v1" "jenkins" {
  provider = kubernetes.clustera
  metadata {
    name      = "jenkins-service"
    namespace = kubernetes_namespace.jenkins.id
  }
  spec {
    selector = {
      app = kubernetes_deployment_v1.jenkins.metadata[0].labels.app
    }
    port {
      port        = 8080
      target_port = 8080
    }

    type = "ClusterIP"
  }
}


resource "kubernetes_ingress_v1" "jenkins_ingress" {
  provider = kubernetes.clustera
  metadata {
    name      = "jenkins-ingress"
    namespace = kubernetes_namespace.jenkins.id
  }

  spec {
    rule {
      host = "jenkins.maojing.site"
      http {
        path {
          backend {
            service {
              name = kubernetes_service_v1.jenkins.metadata[0].name
              port {
                number = 8080
              }
            }
          }
          path_type = "Prefix"
          path = "/"
        }
      }
    }
  }
}

(9)初始化

terraform init

(10)格式化代码

terraform fmt

(11)验证代码

terraform validate

(12)计划与预览

 terraform plan

(13) 申请资源

terraform apply

yes , 4个资源将被添加

（14）登录阿里云系统查看

命名空间新增1个 devops

工作负载（无状态deployment）新增1个jenkins

进入jenkins，状态为running

服务service

service关联路由

（15）修改输出配置文件

outputs.tf,添加如下代码

output "ingress_ip" {
  value = kubernetes_ingress_v1.jenkins_ingress.status[0].load_balancer[0].ingress[0].ip
}

（16）计划与预览

 terraform plan

成功拿到ingress的ip

（17）添加DNS配置文件

dns.tf

#  DNS
resource "alicloud_dns_record" "record" {
  name        = "maojing.site"
  host_record = "jenkins"
  type        = "A"
  value       = kubernetes_ingress_v1.jenkins_ingress.status[0].load_balancer[0].ingress[0].ip
}

(18) 添加变量配置文件

variables.tf

variable "access_key" {
  description = "access_key"

}

variable "secret_key" {
  description = "secret_key"
}

(19) 修改版本配置文件

terraform {
  required_providers {
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "2.25.2"
    }
    alicloud = {
      source  = "aliyun/alicloud"
      version = "1.214.1"
    }
  }
}

provider "kubernetes" {
  # Configuration options
}

# Configure the Alicloud Provider 默认供应商
provider "alicloud" {
  access_key = var.access_key
  secret_key = var.secret_key
  region     = "cn-hangzhou"
}

（20）初始化

terraform init

（21）格式化代码

terraform fmt

（22）验证代码

terraform validate

（23）计划与预览

 terraform plan

（24）申请资源

terraform apply

yes

（25）阿里云系统查看

域名解析已新增

（26）dig测试DNS

dig jenkins.maojing.site

(27) 浏览器测试

显示Jenkins安装界面

(28) 查看集群监控

6.销毁资源

（1）销毁服务资源

terraform destroy

yes

（2）登录阿里云系统

DNS解析已删除

devops命名空间已删除

（3）销毁集群资源

terraform destroy

yes ，用时大约5分钟

(4)登录阿里云系统查看集群

删除中

已删除

二、问题

1.Terraform 验证失败

（1）报错

╷
│ Error: "availability_zone": [REMOVED] Field 'availability_zone' has been removed from provider version 1.212.0.
│
│   with alicloud_cs_managed_kubernetes.k8s,
│   on main.tf line 7, in resource "alicloud_cs_managed_kubernetes" "k8s":
│    7: resource "alicloud_cs_managed_kubernetes" "k8s" {

│ Error: "availability_zone": [REMOVED] Field 'availability_zone' has been removed from provider version 1.212.0.
│
│   with alicloud_cs_managed_kubernetes.k8s,
│   on main.tf line 7, in resource "alicloud_cs_managed_kubernetes" "k8s":
│    7: resource "alicloud_cs_managed_kubernetes" "k8s" {

│ Error: "runtime": [REMOVED] Field 'runtime' has been removed from provider version 1.212.0. Please use resource 'alicloud_cs_kubernetes_node_pool' to manage cluster nodes, by using field 'runtime_name' and 'runtime_version' to replace it.
│
│   with alicloud_cs_managed_kubernetes.k8s,
│   on main.tf line 7, in resource "alicloud_cs_managed_kubernetes" "k8s":
│    7: resource "alicloud_cs_managed_kubernetes" "k8s" {

（2）原因分析

Terraform Registry

从1.212版本开始，部分关键地段被移除，推荐使用alicloud_cs_kubernetes_node_pool 管理工作节点。

From version 1.212.0, runtime,enable_ssh,rds_instances,exclude_autoscaler_nodes,worker_number,worker_instance_types,password,key_name,kms_encrypted_password,kms_encryption_context,worker_instance_charge_type,worker_period,worker_period_unit,worker_auto_renew,worker_auto_renew_period,worker_disk_category,worker_disk_size,worker_data_disks,node_name_mode,node_port_range,os_type,platform,image_id,cpu_policy,user_data,taints,worker_disk_performance_level,worker_disk_snapshot_policy_id,install_cloud_monitor,kube_config,availability_zone are removed. Please use resource alicloud_cs_kubernetes_node_pool to manage your cluster worker nodes.

（3）解决方法

修改配置文件。

2.Terraform申请资源失败

（1）报错

Error: [ERROR] terraform-provider-alicloud/alicloud/resource_alicloud_cs_kubernetes.go:1230: Resource c28e6d5ac0cf64922a476e6963f1239b8 DescribeNatGateways Failed!!! [SDK alibaba-cloud-sdk-go ERROR]:
│ SDK.ServerError
│ ErrorCode: Forbidden.RAM
│ Recommend: https://api.aliyun.com/troubleshoot?q=Forbidden.RAM&product=Vpc&requestId=0254494A-FE5F-51C9-96DA-394123C37E13
│ RequestId: 0254494A-FE5F-51C9-96DA-394123C37E13
│ Message: User not authorized to operate on the specified resource, or this API doesn't support RAM.
│ RespHeaders: map[Access-Control-Allow-Origin:[*] Access-Control-Expose-Headers:[*] Connection:[keep-alive] Content-Length:[568] Content-Type:[application/json;charset=utf-8] Date:[Tue, 23 Jan 2024 05:11:28 GMT] Keep-Alive:[timeout=25] X-Acs-Request-Id:[0254494A-FE5F-51C9-96DA-394123C37E13] X-Acs-Trace-Id:[740d51a284c42eb37e67556a9d62faa6]]
│ AccessDeniedDetail: map[AuthPrincipalDisplayName:205814005146961779 AuthPrincipalOwnerId:1889388625243280 AuthPrincipalType:SubUser EncodedDiagnosticMessage:AQEAAAAAZa9KgzAyNTQ0OTRBLUZFNUYtNTFDOS05NkRBLTM5NDEyM0MzN0UxMw==]
│
│   with alicloud_cs_managed_kubernetes.k8s,
│   on main.tf line 7, in resource "alicloud_cs_managed_kubernetes" "k8s":
│    7: resource "alicloud_cs_managed_kubernetes" "k8s" {

（2）原因分析

RAM缺少NATGatewayFullAcess权限

（3）解决方法

RAM添加NATGatewayFullAcess权限。

重新申请资源

yes,先删除旧的实例

开始创建新实例

到了这里，关于IaC基础设施即代码：Terraform 创建ACK集群 与部署应用的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！