Wait Conditions in the Kubernetes Provider for HashiCorp Terraform
Discover how the Kubernetes and Kubernetes-alpha providers for HashiCorp Terraform allow users to specify whether they must meet certain conditions before they successfully apply a resource, giving users more control over their Kubernetes infrastructure.
The HashiCorp Terraform Strategic Integrations team is working hard to bring you a Kubernetes provider for HashiCorp Terraform with a better-than-native experience. We recently improved the wait_for
configurations on several resources, as well as introduced an entirely new generic waiter. The new and improved configuration options allow you to specify whether Terraform should wait for a specific condition, or not, before continuing to apply
your configuration or complete successfully.
» Wait For Rollout
The wait_for_rollout
attribute is available on both the kubernetes_deployment
and the kubernetes_stateful_set
resources. The default value for wait_for_rollout
is true
, so if that’s the behavior you want, you don’t need to do anything. However, there are cases where you may not expect a rollout to complete before you’re finished applying your Kubernetes configuration. In these cases, you can set wait_for_rollout
to false
and the Kubernetes provider will move on after the Deployment or StatefulSet have been successfully submitted to Kubernetes. Below is an example configuration that demonstrates the usage of wait_for
in a deployment resource.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
}
}
provider "kubernetes" {
config_path = "~/.kube/config"
}
resource "kubernetes_deployment" "nginx" {
metadata {
name = "example-nginx"
labels = {
App = "ExampleNGINX"
}
}
spec {
replicas = 2
selector {
match_labels = {
App = "ExampleNGINX"
}
}
template {
metadata {
labels = {
App = "ExampleNGINX"
}
}
spec {
container {
image = "nginx:1.7.8"
name = "example"
port {
container_port = 80
}
}
}
}
}
wait_for_rollout = false
}
» Wait For Load Balancer
The wait_for_load_balancer
attribute is an option on the kubernetes_ingress
resource. An Ingress controls access to services in the Kubernetes cluster. By default, wait_for_loadbalancer
is set to false
, which means the Kubernetes provider will move on after Kubernetes has accepted the Ingress object API request. When creating or modifying an Ingress with a load balancer configuration, some users want to make sure at least one endpoint is available through the load balancer before the Terraform apply
succeeds, or moves on. When wait_for_load_balancer
is set to true
, that’s exactly what the Kubernetes provider will do.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
}
}
provider "kubernetes" {
config_path = "~/.kube/config"
}
resource "kubernetes_ingress" "example" {
metadata {
name = "example"
annotations = {
"ingress.kubernetes.io/rewrite-target" = "/"
}
}
spec {
backend {
service_name = "example"
service_port = 8080
}
rule {
host = "myminikube.info"
http {
path {
path = "/"
backend {
service_name = "exampleserver"
service_port = 8080
}
}
}
}
}
wait_for_load_balancer = true
}
» Wait For Completion
The last wait for attribute on the Kubernetes provider is wait_for_completion
, which is available on the kubernetes_job
resource. Jobs create Pods that run until a specific task either completes successfully or fails. This attribute ensures Terraform waits until the job has completed or failed before continuing.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
}
}
provider "kubernetes" {
config_path = "~/.kube/config"
}
resource "kubernetes_job" "test-pr" {
metadata {
name = "job-with-wait"
namespace = "default"
}
spec {
completions = 1
template {
metadata {}
spec {
container {
name = "sleep"
image = "busybox:latest"
command = ["sleep", "30"]
}
restart_policy = "Never"
}
}
}
wait_for_completion = true
timeouts {
create = "40s"
}
}
» The New Generic Wait
The Kubernetes-alpha provider introduces a new, generic kubernetes_manifest
resource and with it, a new generic wait_for
attribute. Unlike the previous wait_for_x
attributes, the new generic wait_for
configuration gives users more control over wait conditions. This new attribute instructs the Terraform provider to wait on a condition based on any attribute of a Kubernetes object.
For example, you can specify that Terraform should consider a deployment with 10 replicas complete when two of the pods enter the ready state, and the ingress has an IP address assigned. Keep in mind that the wait_for
attribute waits for all conditions to be met simultaneously. So, in this example, if the deployment’s readyReplicas
value is greater than two before the ingress has an IP address assigned, the conditions won’t be met and the provider will eventually timeout. Currently, the Kubernetes-alpha provider doesn’t support a configurable timeout. To address this, you can also use regular expressions in your conditions.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes-alpha"
}
}
}
provider "kubernetes-alpha" {
config_path = "~/.kube/config"
}
resource "kubernetes_manifest" "example" {
provider = kubernetes-alpha
manifest = {
apiVersion = "v1"
kind = "Pod"
metadata = {
name = "example-pod"
namespace = "default"
annotations = {
"test.terraform.io" = "test"
}
labels = {
app = "nginx"
}
}
spec = {
containers = [
{
name = "nginx"
image = "nginx:1.19"
readinessProbe = {
initialDelaySeconds = 10
httpGet = {
path = "/"
port = 80
}
}
}
]
}
}
wait_for = {
fields = {
"metadata.annotations[\"test.terraform.io\"]" = "test",
"status.containerStatuses[0].restartCount" = "0",
"status.containerStatuses[0].ready" = "true",
"status.podIP" = "^(\\d+(\\.|$)){4}",
"status.phase" = "Running",
}
}
}
For more information on the new Kubernetes Provider, visit the Terraform Registry. To discover more about managing Kubernetes with Terraform, review the guides on HashiCorp Learn.
We would love to hear your feedback and expand on these projects! You can post bugs and feature requests for the Kubernetes-alpha provider by opening an issue at hashicorp/terraform-provider-kubernetes-alpha, or the Kubernetes provider at hashicorp/terraform-provider-kubernetes.
You can also engage with us and the community on HashiCorp Discuss and in #terraform-providers on the Kubernetes Slack (Sign up here).
Sign up for the latest HashiCorp news
More blog posts like this one
Enhancing Azure Deployments with AzureRM and AzAPI Terraform Providers
This blog compares the AzureRM and AzAPI Terraform providers, offering insights on when to use each for optimal Azure infrastructure management.
Terraform Stacks, explained
Terraform Stacks simplify provisioning and managing resources at scale, reducing the time and overhead of managing infrastructure.
Don’t leave cloud security to chance: 7 mistakes and how to avoid them
Learn how to avoid 7 common cloud security mistakes and reduce risk through Infrastructure Lifecycle Management best practices.