Automatic Installation of Third-Party Providers with Terraform 0.13

In June at HashiConf digital we announced the beta version of HashiCorp Terraform 0.13. Many of the improvements in Terraform 0.13 focus on the diverse, rapidly-growing collection of official, partner, and community providers. With Terraform 0.13, terraform init will automatically download and install partner and community providers in the HashiCorp Terraform Registry, following the same clear workflow as HashiCorp-supported official providers. These improvements to the ecosystem will benefit Terraform users and provider developers alike.

This blog will discuss the introduction of a new provider source attribute that allows you to declare the registry source of a Terraform provider. Its primary design goal was to create a clear workflow for using providers from the Terraform Registry. This attribute is part of the required_providers setting inside a terraform configuration block.

»Background

Historically, the Terraform Registry has been a one-stop shop for Terraform modules of all kinds. You could search for modules by name, author, or keyword, and filter your results by major cloud provider type. And you could look over basic information about how to provision the module, see where the module lives on GitHub, and check out the readme and reference info.

Today, you can find both modules and providers in the Terraform Registry. The module pages work much the same as before, and you'll notice some similarities with the provider pages. Providers include their full documentation for you to look through and reference.

The required provider syntax is the scaffolding that empowers users to include partner and community providers into their configuration. It will allow Terraform to download those providers automatically. It also solves the issue of namespace collisions which can arise from forks and generic provider names (ex. DNS).

»Provider Source

terraform {
    required_providers {
        # HashiCorp's dns provider
        hdns = {
            source = "hashicorp/dns"
        }
        # A hypothetical alternative dns provider
        mydns = {
            source = "mycorp/dns"
        }
    }
}

A provider source string is made up of the following parts:

[hostname]/[namespace]/type

Since there is potential for multiple providers with the same name, we are introducing a new concept alongside source: a provider source address. Inside Terraform, a provider source address is created by parsing the source string and extracting the hostname, namespace, and type.

If hostname is omitted, Terraform will use the Terraform Registry hostname as the default hostname. Both hostname and namespace are optional for providers in the hashicorp namespace in the Terraform Registry. namespace is always required when hostname is set.

If "source" is omitted completely, Terraform assumes that the provider is referring to a provider in the hashicorp namespace in the public registry. Terraform therefore determines the following providers all have the same source address, "https://registry.terraform.io/providers/hashicorp/random":

# A fully-qualified source address string includes the host, namespace, and type
random = {
    source = "registry.terraform.io/hashicorp/random"
}

# If the host is omitted, terraform assumes that the host is
# "registry.terraform.io".
random = {
    source = "hashicorp/random"   
}

# This is the same as the example above; source is case-insensitive.
random = {
    source = "HashiCorp/random"
}

# If the source string only includes the type, terraform assumes that the host
# is "registry.terraform.io" and the namespace is "hashicorp".
random = {
    source =  "random"
}

# If there is no source, terraform assumes that the host is "registry.terraform.io",
# the namespace is "hashicorp", and the type is the map key (random).
random = {}

»Local Name

If you have multiple providers with the same type in a single configuration, you can declare a module-specific local name for each provider to easily identify them in your configuration.

In the following example, two providers with the same type ("dns") are declared. One is given the local name "hdns" and the other is called "mydns":

terraform {
    required_providers {
        # HashiCorp's dns provider
        hdns = {
            source = "hashicorp/dns"
        }
        # A hypothetical alternative dns provider
        mydns = {
            source = "mycorp/dns"
        }
    }
}

Use the local name as the label when configuring the provider in a provider block:

provider "hdns" {
    # "hashicorp/dns" provider configuration
}

provider "mydns" {
    # "mycorp/dns" provider configuration
}

If the local name is not the same as the provider type, you must specify the provider for each resource:

resource "dns_record" {
    provider = "hdns"
    # resource configuration
}

resource "dns_record_set" {
    provider = "mydns" 
    # resource configuration
}

»Third-party Providers

Terraform needs a way to tell providers on disk apart— binary name is no longer sufficient—so we've created a new directory hierarchy that Terraform can use to precisely determine the source address of each provider it finds on disk:

$PLUGIN_DIRECTORY/$SOURCEHOSTNAME/$SOURCENAMESPACE/$NAME/$VERSION/$OS_$ARCH/

Third-party provider plugins — locally installed providers, not on the registry — need to be assigned an (arbitrary) source and placed in the appropriate subdirectory for Terraform to find and use them.

When installing custom plugins, you may choose any arbitrary identifier (comprised of letters and hyphens) for the $SOURCEHOSTNAME and $SOURCENAMESPACE subdirectories.

For example, if you wanted to use the community-created dominoes provider

providers {
  customplugin = {
    versions = ["0.1"]
    source = "example.com/myorg/customplugin"
  }
}

The binary must be placed in the following directory (provided you use a linux_amd64-based platform):

./plugins/example.com/myorg/customplugin/0.1/linux_amd64/

For an end-to-end example using a third-party provider with Terraform 0.13, and to learn more about provider usage, try the tutorial on our Learn platform.

»Changes to the provider installer

While most users won’t need these additional details, those using local providers should pay attention to the new directory structure driven by the source address of those providers.

»Upgrade Steps

You will need to upgrade your configuration by adding a required_providers entry for any providers in your configuration except HashiCorp-owned providers.

A new 0.13upgrade command analyzes your configuration and writes a required_providers entry for each provider in your configuration. Locally installed providers require additional steps as outlined in our upgrade guide.

We think most users and providers developers alike will delight in the inclusion of community and partner providers as part of the core terraform init workflow. Download the Terraform 0.13 beta today to take a test drive. Join us on our community forums for discussions, questions, and more!