Network Middleware Automation with HashiCorp Consul

Transcript

Hi, I'm Jake Lundberg with the solutions engineering group at HashiCorp. I'm here to talk to you about network middleware automation with HashiCorp Consul.

Common challenges when updating network middleware

One of the problems that we have with traditional network devices is that they're often cumbersome to configure, often involving human interaction either through command line interfaces and sometimes GUI interfaces.

This slows our ability to update and dynamically configure these devices in highly dynamic environments. This is where Consul can come in and help your organization.

Network middleware automation

Consul allows you to automatically register services both as they come alive and take them out of the service registry when they go away. Through this process, we can automatically update any kind of network middleware devices that can integrate with Consul.

In a typical load balancer workflow, a vip (virtual IP) is created and generally associated with a pool of servers that it's going to service. In order for new servers to be made available to the pool, they have to be configured in some manner. Typically speaking, these are manually added to the pool via command line interfaces or possibly through a web interface.

Once registered, health checks on the load balancers allow for connectivity to any of the available nodes. In order to add new nodes, we have to go through the same process over and over again–either some kind of an API or CLI interface—often driven by human interaction.

With Consul, we can change this to be a lot more dynamic. We can take the service registry concepts of Consul where any new nodes that come alive can automatically register themselves and then—through plugin mechanisms—update any of the load balancer endpoints.

Network automation in a cloud-based environment

What happens when we move any of our devices out of our data centers and move them into the cloud? If we want to keep the same type of disciplines of using our existing networking systems like an F5 BIG-IP or some similar kind of a load balancer it's very hard to do any kind of manual updates with these things using the existing interfaces. Furthermore, we find that dynamic nodes and things like Auto Scaling groups can make that process even harder because it's hard to predict the type of IP addresses that are going to be allocated for those particular systems.

And Consul can fill the gap here. Much like a native service discovery and connection process with Consul, new nodes can automatically be registered with Consul. Then any kind of plugin mechanisms with your software networking devices can be updated on the fly by using the service registry within Consul.

A typical workflow would look like provision an autoscaling group—allow those Auto Scaling group nodes to automatically themselves with Consul. Then your network devices can query Consul to automatically determine which nodes are alive and which nodes are dead to be added into your load balancer pool set.

In the event that an Auto Scaling group scales up, those nodes will automatically be added to Consul. And the opposite is true. In the event that any Auto Scaling groups are scaled down, those nodes will automatically be removed from the service registry, automatically updating any load balancer pools or vips that traffic is only sent to healthy and available nodes.