consulterraform

Consul-Terraform-Sync 0.5 Adds New APIs and UX Improvements

HashiCorp Consul-Terraform-Sync (CTS) 0.5 adds improvements to task lifecycle management and new ecosystem integrations.

Feb 25 2022Devarshi Shah, Jairo Camacho, Lorna Song

We are pleased to announce the general availability of Consul-Terraform-Sync (CTS) 0.5. This release marks another step in the maturity of our larger Network Infrastructure Automation (NIA) solution.

CTS combines Terraform and Consul functionality to eliminate manual ticket-based systems across on-premises and cloud environments. It can be broken down into two parts. For Day 0 and 1, teams use Terraform to quickly deploy network devices and infrastructure in a consistent and reproducible manner. Once established, teams manage Day 2 networking tasks by integrating Consul’s catalog to register services into the system via CTS. Whenever a change is recorded to the service catalog, CTS triggers a Terraform run that uses partner ecosystem integrations to automate updates and deployments for load balancers, firewall policies, and other service-defined networking components.

This post highlights the new features in CTS 0.5, including new APIs to manage the lifecycle of tasks and user experience enhancements. We’ll also discuss the new integrations with Cisco Secure Firewall Management Center (FMC) and Fortinet FortiManager. CTS 0.5 is available for download on the CTS release page, but before we dive into what’s new in CTS 0.5, let’s look at how CTS has progressed over the past few releases.

»CTS 0.1 Through 0.4

  • The 0.1 and 0.2 CTS releases enabled a publisher-subscriber (pub-sub) paradigm for updating your network infrastructure based on changes in the HashiCorp Consul catalog, leveraging the Terraform CLI on the local node. These releases focused on establishing core networking use cases for practitioners, including applying firewall policies, updating load balancer member pools, and more.
  • The 0.3 release marked a major milestone by adding support for Consul-Terraform-Sync Enterprise. This release featured regex support for service triggers and Terraform Enterprise integration.
  • The 0.4 release of Consul-Terraform-Sync enabled integration with Terraform Cloud and support to trigger a Terraform workflow based on Consul key-value (KV) changes.
  • The integration with Terraform Enterprise and Terraform Cloud allowed organizations to dynamically manage their Day 2 applications’ networking delivery lifecycle and oversee infrastructure with governance and security functionality.

»Task Lifecycle APIs and CLIs

Tasks are at the core of CTS’ automation capabilities. It ensures CTS can monitor Consul for updates and trigger a Terraform workflow. But we learned from our users that managing the lifecycle of a task post-CTS initialization required them to stop the CTS process, make changes to the config file, and restart the CTS process. To address these challenges, we are adding support for four new secure API endpoints and a CLI, which allow users to create, update, delete, and read tasks.

Payload:

{
  "task": {
    "description": "Writes the service name, id, and IP address to a file",
    "enabled": true,
    "name": "task_a",
    "providers": ["my-provider"],
    "condition": {
      "services": {
        "names": ["web", "api"]
      }
    },
    "module": "path/to/module"
  }
}

Request:

curl --header "Content-Type: application/json" \
  --request POST \
  --data @payload.json \
  localhost:8558/v1/tasks

»Streamlined Configuration

Our goal has been to provide our users with a smooth and consistent experience for configuring and operating CTS. As we add more capabilities into CTS to trigger workflows — not just based on service changes, but other updates from Consul as well, like key-value changes — we wanted to streamline the configuration for monitoring these changes into the core task functionality.

In CTS 0.4, we introduced module_input blocks to the task configuration, which allowed users to monitor services by matching regexes in a task definition to services within the Consul catalog. At that time, a module_input block could be configured only with scheduled tasks. With this release, the module_input block can now be used with all task types. Users can also configure more than one module_input block per task so long as module input types are unique.

With this capability, users can leverage CTS for application config change management workflows that rely on Consul KV. As a part of this effort, we are deprecating the services field. The full list of deprecations in CTS 0.5 can be found in the release notes.

Previously, a configuration with a catalog-services condition that monitors some services might have looked like this:

task {
 name = "service_registration_task"
 module = "path/to/module"
 
 condition "catalog-services" {
   regexp = ".*"
 }
 services = ["api-1", "api-2", "api-3"]
}

With the expanded usage of module_input blocks, the services field can be replaced with module_input "services". Users also have the option to configure monitoring for consul-kv.

task {
 name = "service_registration_task"
 module = "path/to/module"
 
 condition "catalog-services" {
   regexp = ".*"
 }
 module_input "services" {
   // option to list the services by name or regexp
   regexp = "api-"
 }
 module_input "consul-kv" {
   // new option to add other types of module inputs
   path = "data/api"
 }
}

»CTS Ecosystem Integrations

Our partner ecosystem plays a critical role in helping users automate manual networking tasks. That’s why we are pleased to announce new integrations with Cisco Secure Firewall Management Center (FMC) and Fortinet FortiManager.

»Cisco Secure Firewall Management Center (FMC) Integration

Cisco FMC provides complete and unified management over firewalls, application control, intrusion prevention, URL filtering, and advanced malware protection. The integration with Cisco FMC is achieved through a module that updates dynamic objects in the Cisco Secure Firewall Management Center (FMC) based on service changes within Consul and applies the dynamic objects as access rules to the firewall directly.

»Fortinet FortiManager Integration

FortiManager supports network operations use cases for centralized management, best-practices compliance, and workflow automation to provide better protection against breaches. FortiManager integration with CTS enables management of firewall address groups based on service changes detected within Consul. This integration is delivered through a Terraform module.

»Learn More

There are several HashiCorp Learn tutorials available to help you get started with CTS:

You can find more CTS tutorials by visiting the CTS tutorial collection at learn.hashicorp.com.

Feel free to try out CTS 0.5 and give us feedback in the issue tracker. You can also stay up to date on CTS by following our public roadmap or checking the changelog. For more information about Consul, please visit our Consul product page.

Sign up for the latest HashiCorp news

By submitting this form, you acknowledge and agree that HashiCorp will process your personal information in accordance with the Privacy Policy

More blog posts like this one

Terraform Enterprise adds new flexible deployment options
September 20 2023 | Products & Technology
Terraform Enterprise adds new flexible deployment options

Terraform Enterprise now supports more flexible deployment options for self-hosted environments, including cloud-managed Kubernetes services.

Terraform Cloud now supports assigning agent pools at the organization level
September 19 2023 | Products & Technology
Terraform Cloud now supports assigning agent pools at the organization level

Assigning agents at the organization level provides a faster, more consistent, and scalable approach to agent pool configuration.

Creating a multi-cloud golden image pipeline with Terraform Cloud and HCP Packer
September 18 2023 | Products & Technology
Creating a multi-cloud golden image pipeline with Terraform Cloud and HCP Packer

Learn how creating a golden image pipeline can help unify and streamline your imaging and provisioning workflows throughout your infrastructure estate.