Keynote: The State of Terraform (plus Vagrant and Packer)

Transcript

Hi, my name is Robbie Th'ng from the product team at HashiCorp. I'm super excited to speak to you today about what we've been working on in the infrastructure product line.

Let's kick it off with Vagrant and Packer. Vagrant and Packer continue to be some of the most popular OSS tools in the DevOps space, which is probably not a surprise to a lot of people.

Vagrant taking off was my introduction to the HashiCorp tool set, as I'm sure it was for a quite a lot of people, and today a huge number of people still use Vagrant to work with a variety of different kinds of environments.

Vagrant Cloud, as you can see, has only bolstered that. With Vagrant 2.3, the general availability launch for improved triggers and full config desk support, as well as some enhanced security features and Windows support, are coming down the line.

On Packer 1.6.4, we'll kick off a number of releases, which are going to focus more heavily on builders and the way they interact with the core Packer experience.

We want to ensure a solid flow of the way the most popular builders are used while setting up future functionality for the way people work with non-core builders as well for their unique environments.

These products are important to a lot of people, with the way they work today, and that's why we've spent a bit of time talking about them, but it's also true for Terraform; it's also an important tool that a lot of people use.

Terraform is quite a big product at this point. There are 3 pieces that we're going to talk about in detail:

• The open-source project

• Terraform Enterprise and Terraform Cloud, our platform products

• The Terraform provider ecosystem

Terraform Open Source

We have had a huge couple of months. With the release of 0.12, the team was working hard to stabilize all the functionality and all of the features necessary to introduce something like HCL2. They were also working on a lot of important features, and have been ever since then as we move toward 1.0.

With 0.13 being generally available, the team has now fully focused on the final steps to take us to 1.0. One of our core engineers, Kristin Laemmert, gave a talk on what makes a 1.0 release for us on the Terraform team. She covered how seriously we take that.

With 0.14, we're rounding out a lot of the functionality and changes that we consider core to the infrastructure as code experience, and we'll be looking to get the community's feedback. As always, I'm excited to hear what you think about some of those features.

After that, we're going to take the necessary stability steps toward 1.0. Does that mean that there will be another major release after 0.14? It's hard to say right now, but I'd recommend checking out Kristin's talk for more details about what we think about that.

That being said, I want to talk a bit about some of my favorite features from the 0.13 GA. Module for_each and module count functions were at the top of the list of things that we wanted to work on after 0.12. Both, unsurprisingly, were at the center of the 0.13 release.

The applications for for_each are extensive, especially in lots of places where iterating over a set of things is going to be useful.

Inside module development, they often become even more powerful.

This speaks to the way that we wanted to finish our functionality and code experiences that we started with the introduction of HCL2.

It was such a big step forward, and there were so many improvements that went into that language change, that we were able to finish up these experiences because of the hard work that went into 0.12.

A funny thing about for_each is that Mitchell was a strong advocate of us closing out for_each and count in modules quite soon after 0.12. It turns out he has written a few modules in his time at HashiCorp, and he definitely wanted to see that functionality in the OSS project.

Moving on to my favorite feature, variable validation. I often make mistakes when I'm writing config, so it's really nice having one or more validation blocks, to be able to go into as much detail as I want as to the kind of validation I want to kick back when I'm trying to find a mistake that I've made.

Terraform Enterprise and Terraform Cloud

The last year has been quite important for Terraform Cloud. In May of last year, we announced remote state management for everybody on Terraform Cloud.

We spent a lot of time building an enterprise collaboration project around Terraform, and we'd seen a lot of teams get started and have issues when getting running, collaborating on Terraform at scale.

The first issue that customers and people usually run into is state management. It's often the first hurdle. To test if other people felt the same way that we felt about this, we launched a free beta, where anybody could sign up for Terraform Cloud and get free state management from the SaaS platform.

This went really well, and we started to look at a base set of features and functions that would allow teams and sole developers to get started with the best practices for collaborating with Terraform. These included the version control workflow and the module registry.

These collaboration tools were taken from the things that we'd learned from building the enterprise product and shipping that to customers.

Moving on, we then introduced paid functionality with the paid tiers. The features that went into this were Sentinel, our policy as code language, and cost estimation, as well as the ability to do more concurrent runs on the platform.

At this point early customers were starting to push the platform further, and cost estimation was a functionality that we wanted to offer at that stage.

Again, we relied on things that we'd learned from the enterprise platform. At this point, we'd been serving some of the largest businesses in the world with Terraform Enterprise, and so moving into paid functionality was something that we could easily do.

That takes us to today, where we've just launched the business tier for the Terraform Cloud product. Like with everything, we designed it based on the feedback from our users.

Many customers at this point were starting to use Terraform Cloud as a mission-critical piece of their infrastructure-provisioning story, and that's an amazing thing for us to hear, but it's also a responsibility that we take incredibly seriously.

The business tier was built with this in mind. So SSO with Okta and Azure Active Directory, audit logging with Splunk and also with the API endpoints, as well as cloud agents were all rolled into this tier.

I'm really excited about the cloud agents, because the ability to run Terraform locally and have it work with our platform opens up a lot of possibilities for customers who are running against multiple environments in different places, or just have something really custom in their environment that they want to have Terraform be aware of when it's running.

This was a nice switch, because we'd started by taking things from the Enterprise product and pushing them into Cloud to try and iterate, but were no developing features in Cloud iterating over them quickly, and we're going to be bringing some of them back into the Enterprise product. So it's a nice reversal of the way things have worked, and it rounds out where we are today.

Today, Terraform Cloud has seen more activity than we could have possibly dreamed of. We knew there was definitely going to be demand for collaborating with Terraform, and so we're really excited with all of the usage that we've seen.

We're working on the next set of features and functionality to make the experience even better, and so a huge thanks to all of our users on the platform who are helping us make the product better.

On that note, straight after this is a talk from Kyle Ruddy and Petros Kolyvas of the Terraform team called "From CLI to Cloud and Back." They'll be covering some of the things I've discussed in the open-source world, as well as how those work with Terraform Cloud, possibly even delving into some of the new things from 0.14. It's going to be a really good way to get more detailed, technical examples of how these features work.

The Terraform Ecosystem

The Terraform ecosystem incorporates our providers and also the modules that people create to create deployable infrastructure patterns with those providers. It's an integral part of the infrastructure as code experience, and it has been growing consistently.

One of the things that's so great about it is the community. There are a lot of developers out there writing Terraform modules for things that they want to see instrumented with infrastructure as code, and there are even more developers than that writing modules to create best practice patterns that they can deploy.

The cool thing about this is everybody is doing it on the registry, and so new developers coming into the space can benefit from all of the activity and all the work that's going on there. The numbers grow so much that it's hard to do a slide with the right scale of the numbers, because they change so quickly, but I guess that's a good problem to have.

We're also fortunate in this space to have working relationships with the largest cloud providers. These partnerships are ensuring a level of collaboration and a quality of integration that is hard to replicate outside of Terraform.

We're working closely with them to ensure that fixes happen for the next releases and to make sure that the bar is always there for anybody who's going to be using either one of the modules or one of the providers associated with them.

We also get to work on larger projects with these partners that will actively improve the quality of the ecosystem as a whole, and hopefully we're going to be able to talk more about them throughout the year.

However, the biggest increase in providers comes from our community. The ones I was mentioning earlier, who are constantly adding new things every day that they want to instrument, and with that amount of interaction and attention, we want to make sure that we're setting up for scale with our community. So we have been doing that in the last few months.

In 0.13, we added provider source, and this enables a better story for declaring the source of your providers explicitly. You can declare whether you want to fetch them from the registry or whether you want to fetch them from a custom location that makes sense to you.

That, combined with the version constraints, gives you a Terraform config that can be fully explicit about its dependencies. It makes the behavior of a config much more predictable in general. And perhaps much more importantly, this functionality sets the stage for a much larger expansion of the community providers.

With the combination of that plus the new registry and the work that went into all of this, we were able to launch providers in the registry earlier this year, which showcases not only obviously the top cloud integrations that we have, but the continually growing number of community providers.

Like I said before, the community effort is the most important thing about this. It really is exciting that it's all coming together with the registry, that people can benefit from that. And with the recently redesigned registry navigation, we've tried to make it even clearer for new users to understand what they can expect when they're interacting with the provider.

We have 3 tiers: official, verified, and community.

• Official stating that our engineers will be backing it and they'll be working with that provider partner.

• Verified meaning that they can expect support from one of our many partner teams. It'll be a partner-led effort.

• Community means that the community is leading the effort on that provider.

We're looking to make the registry experience even closer to Terraform Cloud and Terraform Enterprise this year, so please stay tuned for more to come.

Finally, I want to thank everybody who works on Terraform and everybody who makes this possible, from the engineers to the product people to the designers to the support people, everybody who works on Terraform at HashiCorp.

An enormous thank you for everything you've done to make this project great.

And also the community. The community is such an important part of what we do. The product wouldn't be where it is today without all of the things that the community does to make it great, to push it further, and to continually give us feedback on ways that we can move that make sense.

So thank you to everybody involved. It wouldn't be what it is without you, and I'm looking forward to seeing you all again next year.