We're releasing a new feature from the HashiCorp Consul + Kubernetes announcement every week.
This week we're releasing the service catalog sync functionality to sync Kubernetes services to the Consul catalog and vice versa. This enables cross-cluster or platform service discovery using the native service discovery tooling expected. Additionally, a central catalog allows organizations to gracefully migrate workloads into or out of Kubernetes.
» Kubernetes to Consul
Kubernetes services are actively synced to the Consul catalog without any changes to existing Kubernetes service resources. Consul users can then discover and connect to these services using Consul DNS or the HTTP API.
The most common use case is to enable non-Kubernetes workloads to discover and connect to workloads running in Kubernetes. The syncing process ensures that the Consul catalog is always up to date with the latest Kubernetes state. Kubernetes users just use standard Kubernetes tooling to register services. If pods are rescheduled onto different nodes, scaled up or down, etc. the service remains in sync in Consul.
The syncing process only syncs Kubernetes services that are likely to be routable outside of the cluster. In this initial release these are
LoadBalancer services, and services with an external IP set. The following is a standard
LoadBalancer service definition.
apiVersion: v1 kind: Service metadata: name: consul-ui spec: ports: - name: http port: 80 targetPort: 8500 selector: app: consul component: server release: consul type: LoadBalancer
Shortly after registering the service, the service becomes available in Consul:
# From outside of Kubernetes $ dig consul-ui.service.consul. ;; QUESTION SECTION: ;consul-ui.service.consul. IN A ;; ANSWER SECTION: consul-ui.service.consul. 0 IN A 18.104.22.168 ;; ADDITIONAL SECTION: consul-ui.service.consul. 0 IN TXT "external-source=kubernetes"
Externally registered services from Kubernetes are clearly marked in the UI with the Kubernetes icon (requires Consul 1.2.3+):
Optional annotations can be specified on the Kubernetes service to control the syncing behavior. For example, you can change the service name in Consul by setting the
consul.hashicorp.com/service-name annotations, specify additional metadata using
consul.hashicorp.com/service-meta-KEY, etc. The full list of available options can be found in the service sync documentation.
» Consul to Kubernetes
Consul services can also be actively synced into first-class Kubernetes services. Kubernetes users can then discover and connect to these services using native Kubernetes DNS, environment variables, and more.
This enables Kubernetes workloads to discover and connect to non-Kubernetes workloads. This includes applications running on non-Kubernetes nodes, but also hosted services such as SaaS.
# From a Kubernetes Pod using Kubernetes DNS $ dig consul.default.svc.cluster.local ;; QUESTION SECTION: ;consul.default.svc.cluster.local. IN A ;; ANSWER SECTION: consul.default.svc.cluster.local. 0 IN CNAME consul.service.consul. consul.service.consul. 0 IN A 10.0.2.8 consul.service.consul. 0 IN A 10.0.4.6 consul.service.consul. 0 IN A 10.0.1.7
» Running the Catalog Syncer
syncCatalog: enabled: true
The sync process can also be manually run outside of a Kubernetes cluster. The Consul server cluster can also be either in or out of the Kubernetes cluster.
The sync functionality is part of the consul-k8s project. This project encapsulates multiple official integrations between Consul and Kubernetes. By building this project separate from Consul core, we can iterate on Kubernetes integrations more quickly and release new versions without forcing users to do a full Consul upgrade.
The Kubernetes and Consul service catalog sync feature is available immediately. To learn more, see the service sync documentation. The source code to this functionality is available in the consul-k8s project. Installing the service catalog sync process with the official Helm chart requires version 0.2.0 of the Consul Helm chart.
We have more exciting Consul + Kubernetes integrations being released in the coming weeks. Please read the Consul and Kubernetes announcement blog post to learn more.