Building a PaaS with Waypoint

Hear about how Citrix used HashiCorp Waypoint as a foundation to build a PaaS-like experience for developers.

Waypoint was released in 2020, to streamline the build, deploy, and release process for developers. In this session, see how Waypoint is leveraged to build a PaaS-like experience for developers and create a higher level abstraction platform called "WayPaaS," automating the whole lifecycle of Waypoint and integrating GitOps practices.

»Transcript

George Hantzaras:

Hi, everyone. Thank you for joining me in this presentation about building a Platform-as-a-Service with HashiCorp Waypoint. In this presentation, we'll explore a tool we built around Waypoint called WayPaaS. You understand, we're not here about creative naming. 

A bit about me. I've been a HashiCorp tool user for many, many years. I think it’s been something like 10 years since I first used HashiCorp Vagrant, and then maybe 6 or 7 years with HashiCorp Terraform. So, I'm usually one of the first people to jump on the Terraform tooling. 

At Citrix, I lead the team that does cloud platform engineering and sovereign cloud engineering. In terms of tooling, our goal is to build tools and infrastructure that are going to enable teams to move faster and safer in the cloud. Essentially, it's about developer productivity, developer experience, faster time to market for new features, and easier-to-adopt security standards. This is a motivation behind Waypoint; so, improving developer experience and improving developer productivity are two of the goals for this.

We're going to start by looking at what HashiCorp Waypoint is, how it works, and where it would fit into your lifecycle. We'll see what it's still kind of missing — and that's just an opinion — and we're going to see what WayPaaS is and why it was necessary to build that, to build around Waypoint in order to enable this Platform-as-a-Service experience.

»Defining Waypoint and WayPaas

I said we're going to start by seeing what Waypoint is, but a little bit of a spoiler: since WayPaaS is the main reason we’re here, let's just define it and then we can go a bit deeper. WayPaaS is a platform built on top of Waypoint to provide a PaaS workflow, aimed at improving developer experience and productivity. If you've seen the Waypoint website, you might think, "Okay, but isn't that what Waypoint does? Isn't Waypoint essentially there to replace Heroku or provide this PaaS experience?" And it kind of is, but it also kind of isn't if you're talking about more complex use cases and more complex scenarios. This is what we're going to be focusing on today.

In order to see why it isn't, let's try to define Waypoint first. Let's see how it works. One of the biggest questions the community has at the moment is, where exactly does Waypoint fit? So I also want to go fast over that, and then we can talk about WayPaaS. 

So, a bit of a story. I remember when I first went into the Terraform website. And if you see Terraform, it says, "Automate infrastructure in any cloud." I think we kind of know exactly what it does, right? Now, when I first went into the Waypoint website, what it says is, "Waypoint allows developers to deploy, manage, and observe their applications through a consistent abstraction of underlying infrastructure." That's a bit more words, right? It makes sense, because Waypoint is a lot more complicated — the use case is a lot more complicated — but it focuses more on what you get.

You start thinking about it: "Okay, but Waypoint, is that the PaaS? Am I going to replace Heroku with it? Or is it something like Helm for all infrastructure?" And then you might start panicking because you think that it might replace Jenkins, which you love and you've written all those scripts for. But it's, I would say, none of the above. And — a bit of a spoiler — it sits along with your CI/CD, so you're going to be able to keep all this nice Jenkins stuff you've built.

My definition of Waypoint is: Waypoint enables operators to build PaaS experiences for development teams with full access to underlying, abstracted infrastructure. Essentially, that means that operations teams are able to have full access to the infrastructure, but this infrastructure is abstracted for the development teams. Essentially, it's a PaaS for simple enough use cases, but if your use case is complicated enough, then it's a tool to enable you to build a PaaS on top of it. That's how, at least, we understood the tool.

»The Why and How of Waypoint

The business case for Waypoint, the "why" behind it, is

• First of all, hide some of the infrastructure complexity for the development team

• Improve developer experience

• Reduce time to market for new features

• Finally, streamline the application lifecycle management.

When I was putting together this presentation, I wanted to create a slide where we can talk for 30 seconds about how Waypoint works. Sorry for this — this is the best I could come up with. A project is a repository, and then you have the pipeline. Essentially, the lifecycle of a pipeline has three stages; build, deploy, and release. And on the right, you see roughly what a Waypoint HCL file would look like.

So, you have the build stage. That could be buildpacks. We might know buildpacks from Heroku, a great tool. You might have Docker, where you build based on a Dockerfile. You have the deploy stage, where it can be Amazon ECS, it can be Kubernetes, it can be a lot of different things. And then you have the release stage. 

This is probably my favorite part of Waypoint, that it's one of the first tools that is really opinionated about the release being different than the deployment stage. So you can have the CI, you can have the CD, and then you can have the release-on-demand with Waypoint, which is really nice. You kind of see this one-to-one relationship between the project and the repository. You push your code, and then you just run one command and do nothing with infrastructure and this is deployed automatically. This kind of feels like a PaaS, right? Well, as I said before, if the use case is simple enough, yes, that's the intention, and it can be a PaaS where you just run waypoint up and everything happens magically, and you essentially have a Heroku. 

But remember, the initial goal was to abstract the underlying infrastructure. That means to enable people who are not aware of Kubernetes to deploy in Kubernetes without them having to read or to understand how to write Helm charts or even HashiCorp configuration language (HCL) for that matter.

»Waypoint is a Starting Point

Let's just use an example. Going back to the deploy stage and starting from the image on the far right, if your application is simple enough, if you just have a Flask server or a Node.js server, you're going to just say “Deploy,” use Kubernetes, and magically, everything is going to be up in Kubernetes. But that's not going to work in the real world. You're going to need some replication. Easy enough, you're going to add the number about replicas, and you're going to be done. But wouldn't you need some autoscaling? So you add a bit more. And then you want to add some health probes and you want to change the container port. And as you see, you slowly end up writing essentially a health chart, just in HCL. So, the development team is going to have two reasons to not like you.

This is one of the most important things that was missing for us. And I'm going to start with something that Mitchell (Hashimoto) tweeted when Waypoint was first released. If you focus on what he's writing, it’s essentially that you want something to replace Heroku without being as opinionated as Heroku. That's a good thing on the one side, but what that does is, when you have more complex needs around Kubernetes deployment, you end up writing a whole Helm chart, or you expect developers to understand and write HCL. In a time where we shift everything left — we shift observability left, we shift security left, we shift infrastructure left — we are actually not helping developer productivity that much. So, that's the one issue: We wanted to find a way to use Waypoint without having to write all the HCL. The second point is — as I said, it has a really good pipeline structure, so you have to build, deploy, and release, but we never talked about testing. If we are talking about lifecycle management, CI is automatically out of this. 

The third thing is that although authentication exists — anything that we do in the Waypoint server is authenticated — it's just authenticated, so we don't have role-based access control, we don't have authorization and those kinds of things. And of course, there is some use case–specific functionality that we wanted to restrict. We don't want someone to be able to run waypoint exec and execute commands in a production environment. That's also something we wanted to be able to cut.

If we go back to the business case, yes, it definitely improves developer productivity. 100% reduces time to market. It does abstract infrastructure components; but potentially, in our use case and in complex use cases, we need a bit more abstraction. And of course, it's really easy to manage the lifecycle, but the lifecycle needs some more steps in it.

»How WayPaas Works

Finally, we are here to see what you came for. WayPaaS is what we built in order to solve these problems. The initial thoughts we had on Waypoint was to: 

• Add an extra layer

• Make it a bit more opinionated and tie this to enterprise identity

• Use enterprise identity, group memberships and configure user profiles in that sense

• Have defaults, have templates per team. That way, development teams don't have to do essentially anything. They don't have to write a configuration file anymore, they don't have to learn HCL. For them, it's essentially the same if they deploy on ECS or if they deploy in Kubernetes, it's just a different flag in the command line. 

• Narrow the scope of the abilities the development team has in production.

So, let's see how this works. We start with the wps init command. That's kind of the setup command. What happens here is, you authenticate with your enterprise identity — AD in my case — and you get team membership, you get group membership. And based on that, you configure the user profile. Each team has specific ECS/ECR regions for dev, stage, and production; different kubeconfigs, again for different environments; replication standards; health probe standards; auto-scaling; and so on.

Now the second part, and we're still setting up the project, is wps create. Essentially, that's where a new project is created. What happens is, based on the profile we had before, we preconfigure and auto-generate the waypoint.hcl file based on the template. And the user has a few parameters, a few command-line arguments, if you want to override the default. Your team might have defaults or templates for both ECS and Kubernetes, so you use the infra flag to say if you're deploying on ECS or Kubernetes. Resources — you might be able to override default resource allocation: if your team standard is 1GB and you need 2, you can override it that way. And ports, if you want to change the default container port. But again, those are optional. Your team has predefined those, and you're just using the flags if you want to override those.

What happens in the background is, when you run the ‘create’ command — let's take the example that you write it without any parameters — what happens is you first read locally. You get the local git config in order to find the remote. So you tie the Waypoint project with the Git repository you're working on. You read the team's configuration, and then you create the Waypoint HCL based on that configuration. 

Going back to the Kubernetes example, the only thing that you've done is wps init, added your credentials, wps create, and this is what you have. For example, in the registry part, you're using ECR as your registry. Region is something that your team is using, us-east-1. Same — all your team has agreed that our /health is going to be our health check endpoint. All these are pre-agreed and preconfigured. Of course, you can override them, but just running two commands, you can end up with this.

»Preparing for Release

What we talked about up to now essentially is not lifecycle management, it's just setting up the process. What happens in the rest of the lifecycle — and essentially, this is pure Waypoint, almost — we have three stages, which are build, deploy, release, and it's based on a really, really simple Git flow. So we have master, dev, and feature branches. When you want to work on the feature branch, you push that, then CI is triggered, then you run just the build command. You run waypoint build, and your artifact is built. 

What's going to happen after, when you're done with working on the feature, you're going to open a PR. This is going to be reviewed. You're going to merge the PR. When this is merged in dev, that's the point where we're going to run waypoint deploy in the dev environment.

Still, we're not going to be releasing yet. The difference between deploy and release is that release also gives you public IP, routes traffic, so we are able to deploy something. We have internal access to it, we have private access, but we are not opening it up to the public. So when we merge to dev, we deploy in dev private access. Then when we merge this to master, we again deploy the same Waypoint file, different kubeconfig, so we deploy in the staging cluster, but the same artifact. There is some plumbing under the hood, because if you're using ECR and you have some images, you have to move images between the regions or the sub accounts, but that's not important. 

Finally, when you reach a point that you've tested it in staging and you want to go to production, you just run a wps promote command, and what that does is, again, same Waypoint configuration, different kubeconfig; deploys in production, but that also runs the release command that gives it the public IP. We have the Waypoint URL service that gives the public IP and opens it up to public traffic.

»Future Directions for WayPaas

This is what we've built up to now. And this is really young, so we have a long way ahead of us. This is still in the POC stage. We need a more complex, sophisticated Git flow. We need support for more use cases, and we need some extensibility in it. 

As we said before, all the cases we see here are about a simple web server or simple services, but in the real world you're probably going to deploy 5 or 10 different microservices, a queue, a database, and a few other good things. So, we need to have the ability to deploy a whole stack. One of the things we want is to further explore plugins. Some of the functionality that we've written might better go into a plugin. Like most HashiCorp tools, Waypoint supports plugins, and it's pretty powerful. 

Waypoint 0.9 is about to be released, so there is going to be a lot of nice stuff there and probably take away some pain from us. And finally, HCP Waypoint — earlier today, there was an announcement about HCP Waypoint. That's also something to look forward to.

So, today we learned about what Waypoint is; where exactly does it fit in your lifecycle; what we think is missing from Waypoint; and how WayPaaS, our Platform-as-a-Service tool, comes to solve that and help improve developer experience. Thank you. We have some time, so we can take a couple of questions. Anyone?Audience member:

A simple question: what really stands out from WayPaaS to what Waypoint is actually serving? So what does Waypoint lack that you achieved?

George:

If I had to say one thing, it's basing templates and team standards based on enterprise identity. If Waypoint had this, if this comes out, probably WayPaaS is something we can throw away.