consul

Announcing Consul-Terraform-Sync 0.4 for Terraform Cloud

Compliance-driven network infrastructure automation with Consul-Terraform-Sync 0.4 is now generally available for HashiCorp Terraform Cloud.

Oct 14 2021Devarshi Shah, Jairo Camacho

We are pleased to announce the general availability of Consul-Terraform-Sync (CTS) 0.4, which supports HashiCorp Terraform Cloud (binaries here). This release marks another step in the maturity of our larger Network Infrastructure Automation (NIA) solution. The focus for CTS is on automating Day 2 networking tasks and reducing the burden on operators caused by manual ticketing systems.

This post will highlight the new features in CTS 0.4, including changes to trigger tasks in Consul KV and the extension of policy and governance functionality from the previous release into the Terraform Cloud integration. We’ll also discuss the benefits of Terraform Cloud integration and share an exciting new integration with Oracle Cloud Infrastructure. CTS 0.4 is available for download on our release page. But, before we dive into what’s new in CTS 0.4, let’s take a look at how CTS has progressed over the past few releases.

»CTS 0.1 Through 0.3

The 0.1 and 0.2 CTS releases enabled a publisher-subscriber (pub-sub) paradigm for updating your network infrastructure based on changes in the HashiCorp Consul catalog, leveraging the Terraform CLI on the local node. These releases focused on establishing core networking use cases for practitioners, including applying firewall policies, updating load-balancer member pools, and more.

The 0.3 release of Consul-Terraform-Sync featured regex support for service triggers and Terraform Enterprise integration. The integration allowed organizations to dynamically manage their Day 2 applications’ networking delivery lifecycle and oversee infrastructure with governance and security functionality.

»CTS Integration with Terraform Cloud

CTS integration with Terraform Cloud is a Consul Enterprise-only feature available through the CTS Cloud binary. To get started, operators require a valid Consul Enterprise license that supports a new CTS network driver named Terraform Cloud driver:

driver "terraform-cloud" {
  hostname     = "app.terraform.io"
  organization = "my-org"
  token        = "<TEAM_TOKEN>"
  // Optionally set the token to be securely queried from Vault instead of
  // written directly to the configuration file.
  // token = "{{ with secret \"secret/my/path\" }}{{ .Data.data.foo }}{{ end }}"
 
  required_providers {
    myprovider = {
      source = "namespace/myprovider"
      version = "1.3.0"
    }
  }
}

Tasks are the heart of Consul-Terraform-Sync’s automation capabilities. The Terraform Cloud driver enables the creation and management of Terraform Cloud workspaces for all tasks defined in CTS configuration using the API-driven run workflow.

A Terraform Cloud / Enterprise workspace created and managed by CTS

A Terraform Cloud workspace created and managed by CTS.

Users of the CTS integration with Terraform Cloud can do a number of useful things, including:

  • Scale resources: Reduce dependency on ticketing systems traditionally handled by networking and security teams, allowing them to optimize their workflows to match the velocity of development teams.
  • Reduce risk: Provide teams with distinct workspaces that clearly define users’ privileges to access and manipulate resources. Restricting access to sensitive information can reduce human-driven misconfiguration errors and potential data leakage.
  • Improve automation controls: Choose the desired level of automation with approval workflows. While end-to-end automation may be the end goal, many organizations opt for an incremental approach. Set notification triggers to inform a team member when a predetermined event occurs. They can review a plan quickly to verify that the configuration is as expected and apply the changes with just a few clicks.
  • Simplify auditing: Automatically catalog every change made to the environment. Provide a comprehensive forensic trail to keep compliance teams in the loop and analyze issues as they arise.
  • Apply compliance policies: Policy as code with Sentinel allows organizations to define, manage, and enforce key policies to safeguard against potentially dangerous actions and encourage efficient use of resources.

»Trigger Tasks on Consul KV Changes

In addition to securing network traffic and providing service discovery and integrated health checking, Consul includes a key-value (KV) store, which you can use to dynamically configure applications and coordinate services, along with a myriad of other uses.

With the 0.4 release, CTS now allows users to subscribe to a Consul KV prefix and automatically trigger a task whenever the prefix is created, deleted, or updated. Potential workflows that can be enabled through this CTS capability include blue/green deployments, which shift traffic from one load balancer to another by populating the appropriate backend pool members to avoid downtime.

Available for both CTS OSS and Enterprise binaries, this capability can be configured for a single Consul KV entry or for any Consul KV entries that are prefixed with a given path:

task {
  name = "consul_kv_condition_task"
  description = "execute on changes to Consul KV entry"
  source = "path/to/consul-kv-module"
  providers = ["my-provider"]
  services = ["web-api"]
 
  condition "consul-kv" {
	path            	= "my-key"
	recurse         	= false
	datacenter      	= "dc1"
	namespace       	= "default"
	source_includes_var = false
  }
}

»CTS for Terraform Cloud Partnership with Oracle Cloud Infrastructure

The partner ecosystem is at the core of CTS for Terraform Cloud. That’s why we are pleased to announce a new integration with Oracle, allowing Oracle Cloud Infrastructure (OCI) customers to automate Day 2 networking operations across hybrid and multi-cloud environments. Our friends at Oracle shared a few words about the value of this integration in helping workloads scale to alleviate network outages caused by sudden load spikes:

Oracle Cloud Infrastructure: Consul-Terraform-Sync Sample Solution

“Oracle is pleased to announce a sample Consul-Terraform-Sync (CTS) solution that updates Oracle Cloud Infrastructure (OCI) Load Balancer (LB) Backend servers dynamically, based upon changes to a Consul service. While this is a very simple solution, it highlights how easy it is for customers to build their own solutions leveraging CTS to rapidly scale their OCI infrastructure. Oracle and HashiCorp are working together to help ensure that users have the automated workflows required to achieve a repeatable application deployment lifecycle in a compliance centric manner. Our mutual customers are able to create their own CTS-based solutions that allow them to push out OCI infrastructure changes in a matter of minutes based on dynamic changes to application services, reducing operational complexity, reducing overhead, and enhancing performance.” — Tim Clegg, DevSecOps, Developer Relations, Oracle

»Learn More and Get Started

CTS supports an integration with Terraform Cloud to let organizations manage their application networking delivery lifecycle in a more dynamic way, while providing governance and security oversight. This capability is delivered as an enterprise feature through the CTS for Terraform Cloud binary.

To learn more about Consul-Terraform-Sync for Terraform Cloud please check out our documentation and our tutoials on HashiCorp Learn.

Related Content
Consul-Terraform-Sync for Terraform Cloud
Oct 13 2021 | HashiCorp Learn

Feel free to try out CTS 0.4 and give us feedback in the issue tracker. You can also stay up to date on CTS by following our public roadmap or checking the changelog. For more information about Consul, please visit our product page.

Sign up for the latest HashiCorp news

By submitting this form, you acknowledge and agree that HashiCorp will process your personal information in accordance with the Privacy Policy.

More blog posts like this one

Gain operator confidence with HCP Consul Central observability features
November 27 2023 | Products & Technology
Gain operator confidence with HCP Consul Central observability features

HCP Consul Central’s new observability features were recently used by a customer to help troubleshoot an issue with assistance from HashiCorp engineers.

HashiCorp at AWS re:Invent: Your blueprint for cloud success
November 21 2023 | Company
HashiCorp at AWS re:Invent: Your blueprint for cloud success

If you’re attending AWS re:Invent in Las Vegas, Nov. 27 - Dec. 1, visit us for breakout sessions, expert talks, and product demos to learn how to accelerate your adoption of a cloud operating model.

Consul 1.17 GA adds locality-aware routing and multi-port support
November 08 2023 | Products & Technology
Consul 1.17 GA adds locality-aware routing and multi-port support

Consul 1.17 enhances unified workflow management, reliability, scale, and security for service networking.