terraform

Wait Conditions in the Kubernetes Provider for HashiCorp Terraform

Discover how the Kubernetes and Kubernetes-alpha providers for HashiCorp Terraform allow users to specify whether they must meet certain conditions before they successfully apply a resource, giving users more control over their Kubernetes infrastructure.

Feb 08 2021Phil Sautter

The HashiCorp Terraform Strategic Integrations team is working hard to bring you a Kubernetes provider for HashiCorp Terraform with a better-than-native experience. We recently improved the wait_for configurations on several resources, as well as introduced an entirely new generic waiter. The new and improved configuration options allow you to specify whether Terraform should wait for a specific condition, or not, before continuing to apply your configuration or complete successfully.

»Wait For Rollout

The wait_for_rollout attribute is available on both the kubernetes_deployment and the kubernetes_stateful_set resources. The default value for wait_for_rollout is true, so if that’s the behavior you want, you don’t need to do anything. However, there are cases where you may not expect a rollout to complete before you’re finished applying your Kubernetes configuration. In these cases, you can set wait_for_rollout to false and the Kubernetes provider will move on after the Deployment or StatefulSet have been successfully submitted to Kubernetes. Below is an example configuration that demonstrates the usage of wait_for in a deployment resource.

terraform {
 required_providers {
   kubernetes = {
     source = "hashicorp/kubernetes"
   }
 }
}
 
provider "kubernetes" {
   config_path = "~/.kube/config"
}
 
resource "kubernetes_deployment" "nginx" {
 metadata {
   name = "example-nginx"
   labels = {
     App = "ExampleNGINX"
   }
 }
 
 spec {
   replicas = 2
   selector {
     match_labels = {
       App = "ExampleNGINX"
     }
   }
   template {
     metadata {
       labels = {
         App = "ExampleNGINX"
       }
     }
     spec {
       container {
         image = "nginx:1.7.8"
         name  = "example"
 
         port {
           container_port = 80
         }
       }
     }
   }
 }
   wait_for_rollout = false
}

»Wait For Load Balancer

The wait_for_load_balancer attribute is an option on the kubernetes_ingress resource. An Ingress controls access to services in the Kubernetes cluster. By default, wait_for_loadbalancer is set to false, which means the Kubernetes provider will move on after Kubernetes has accepted the Ingress object API request. When creating or modifying an Ingress with a load balancer configuration, some users want to make sure at least one endpoint is available through the load balancer before the Terraform apply succeeds, or moves on. When wait_for_load_balancer is set to true, that’s exactly what the Kubernetes provider will do.

terraform {
 required_providers {
   kubernetes = {
     source = "hashicorp/kubernetes"
   }
 }
}
 
provider "kubernetes" {
 config_path = "~/.kube/config"
}
 
resource "kubernetes_ingress" "example" {
 metadata {
   name = "example"
 
   annotations = {
     "ingress.kubernetes.io/rewrite-target" = "/"
   }
 }
 
 spec {
   backend {
     service_name = "example"
     service_port = 8080
   }
 
   rule {
     host = "myminikube.info"
 
     http {
       path {
         path = "/"
 
         backend {
           service_name = "exampleserver"
           service_port = 8080
         }
       }
     }
   }
 }
 wait_for_load_balancer = true
}

»Wait For Completion

The last wait for attribute on the Kubernetes provider is wait_for_completion, which is available on the kubernetes_job resource. Jobs create Pods that run until a specific task either completes successfully or fails. This attribute ensures Terraform waits until the job has completed or failed before continuing.

terraform {
 required_providers {
   kubernetes = {
     source = "hashicorp/kubernetes"
   }
 }
}
 
provider "kubernetes" {
 config_path = "~/.kube/config"
}
 
resource "kubernetes_job" "test-pr" {
 metadata {
   name = "job-with-wait"
   namespace = "default"
 }
 spec {
   completions = 1
   template {
     metadata {}
     spec {
       container {
         name = "sleep"
         image = "busybox:latest"
         command = ["sleep", "30"]
       }
       restart_policy = "Never"
     }
   }
 }
 wait_for_completion = true
 timeouts {
   create = "40s"
 }
}

»The New Generic Wait

The Kubernetes-alpha provider introduces a new, generic kubernetes_manifest resource and with it, a new generic wait_for attribute. Unlike the previous wait_for_x attributes, the new generic wait_for configuration gives users more control over wait conditions. This new attribute instructs the Terraform provider to wait on a condition based on any attribute of a Kubernetes object.

For example, you can specify that Terraform should consider a deployment with 10 replicas complete when two of the pods enter the ready state, and the ingress has an IP address assigned. Keep in mind that the wait_for attribute waits for all conditions to be met simultaneously. So, in this example, if the deployment’s readyReplicas value is greater than two before the ingress has an IP address assigned, the conditions won’t be met and the provider will eventually timeout. Currently, the Kubernetes-alpha provider doesn’t support a configurable timeout. To address this, you can also use regular expressions in your conditions.

terraform {
 required_providers {
   kubernetes = {
     source = "hashicorp/kubernetes-alpha"
   }
 }
}
 
provider "kubernetes-alpha" {
  config_path = "~/.kube/config"
}
 
resource "kubernetes_manifest" "example" {
  provider = kubernetes-alpha
 
  manifest = {
    apiVersion = "v1"
    kind       = "Pod"
 
    metadata = {
      name      = "example-pod"
      namespace = "default"
 
      annotations = {
        "test.terraform.io" = "test"      
      }
 
      labels = {
        app = "nginx"
      }
    }
 
    spec = {
      containers = [
        {
          name  = "nginx"
          image = "nginx:1.19"
 
          readinessProbe = {
            initialDelaySeconds = 10
 
            httpGet = {
              path = "/"
              port = 80
            }
          }
        }
      ]
    }
  }
 
  wait_for = {
    fields = {
				"metadata.annotations[\"test.terraform.io\"]" = "test",
 
				"status.containerStatuses[0].restartCount" = "0",
				"status.containerStatuses[0].ready"        = "true",
 
				"status.podIP" = "^(\\d+(\\.|$)){4}",
				"status.phase" = "Running",
    }
  }
}

For more information on the new Kubernetes Provider, visit the Terraform Registry. To discover more about managing Kubernetes with Terraform, review the guides on HashiCorp Learn.

We would love to hear your feedback and expand on these projects! You can post bugs and feature requests for the Kubernetes-alpha provider by opening an issue at hashicorp/terraform-provider-kubernetes-alpha, or the Kubernetes provider at hashicorp/terraform-provider-kubernetes.

You can also engage with us and the community on HashiCorp Discuss and in #terraform-providers on the Kubernetes Slack (Sign up here).

Kubernetes

Sign up for the latest HashiCorp news

By submitting this form, you acknowledge and agree that HashiCorp will process your personal information in accordance with the Privacy Policy.

More blog posts like this one

Terraform delivers launch-day support for Amazon S3 Express
November 28 2023 | Products & Technology
Terraform delivers launch-day support for Amazon S3 Express

HashiCorp’s Terraform provider for AWS now enables users to manage their S3 Express buckets.

Terraform extension for VS Code adds Terraform Cloud view
November 28 2023 | Products & Technology
Terraform extension for VS Code adds Terraform Cloud view

A new view in the HashiCorp Terraform extension for Visual Studio Code shows your Terraform Cloud workspaces and runs, reducing context-switching.

HashiCorp at re:Invent 2023: A year of collaboration with AWS
November 28 2023 | Company
HashiCorp at re:Invent 2023: A year of collaboration with AWS

A recap of HashiCorp infrastructure and security news and developments on AWS from the past year, from self-service provisioning to fighting secrets sprawl and more.