Jonathan Lange, Weaveworks' VP of engineering, talks in brief to Jeff Barr, the AWS director of evangelism, about how Weaveworks is able to monitor highly available clusters of multi-tenant containers—and do it at scale, using HashiCorp Consul.
Highlights
- Metrics are received from a consistent hash ring, implemented using Consul
- They're routed to three ingesters randomly chosen from a dynamically scaled pool of ingesters
- The ingester buffers the data via an EC2 instance, and stores them in a DynamoDB table, including several indexes
- The buffering serves to de-amplifify writes, which reduces costs and increases scalability