Jul 25 2017 Rob Genova
Running Apache Spark on HashiCorp Nomad
Apache Spark is a popular data processing engine/framework that has been architected to use third-party schedulers. The schedulers that are available, however, involve a level of complexity that can be undesirable for many potential Spark users. To help fill this gap, we are pleased to announce that the HashiCorp Nomad ecosystem now includes a version of Apache Spark that natively integrates Nomad as a Spark cluster manager and scheduler.
Jul 20 2017 James Phillips
HashiCorp Consul 0.9
Today we are releasing Consul 0.9 which includes a number of improvements, bug fixes, and features. There are some important backward incompatibilities in this release that are easy to adjust for as you deploy, so please be sure to read the 0.9 upgrade guide for details before you upgrade.
Jul 11 2017 Nic Jackson
Auto-bootstrapping a Nomad Cluster
In a previous post, we explored how HashiCorp Consul discovers other agents using cloud metadata to bootstrap a cluster. This post looks at HashiCorp Nomad's auto-joining functionality and how we can use Terraform to create an autoscaled cluster.
Jul 10 2017 George Kontridze
Bugsnag Uses HashiCorp Terraform to Quickly Provision and Safely Maintain Their Infrastructure
This guest blog is by George Kontridze, Production Engineer at Bugsnag. Bugsnag is an automated production error monitoring tool, supporting over 50 different platforms. Bugsnag provides open-source libraries for most popular programming languages which make it very easy for customers to integrate Bugsnag into their workflow. Once integrated, Bugsnag automatically detects application exceptions and provides the data and tooling to prioritize and fix errors with the greatest user impact.
At Bugsnag part of the challenge we face is the fast pace of iteration; be it external, connecting with an API to make a new integration available to our customers, or internally to regularly provisioning and scale a cluster of machines to run our services as our system’s performance characteristics evolve.
As our product evolves, it becomes incredibly important to put the tools in place to help us evolve the infrastructure that runs our services. The time and effort we invest in these tools are also quite valuable to us, so we need to choose wisely.
On the infrastructure side of things, we need to be able to ship configuration changes in production. We do configuration changes for existing resources or to add new resources. Regardless, we need to be able to do this with ease and high visibility. This is where the HashiCorp toolset comes into play.
Jul 07 2017 Chris Kent
Recap and Videos from HashiDays London
HashiCorp hosted HashiDays in London on June 12, the second event from our new series created as one-day, single-track, deeply technical, and community-focused events. HashiDays brings together customers, practitioners, as well as HashiCorp employees to explore and discuss different areas around infrastructure and cloud adoption, with topics ranging from customer stories and use cases, to technical product exploration.
Jul 06 2017 Jon Currey
Making Gossip More Robust with Lifeguard
Today we are proud to announce our first publication by HashiCorp Research, titled "Lifeguard: SWIM-ing with Situational Awareness". The paper details a number of novel improvements we have introduced to Serf, Consul, and Nomad to make their underlying gossip protocol more robust. Collectively called Lifeguard, these extensions reduce by 50x the false positives produced by the failure detector and allow us to detect true failures faster.
Distributed systems such as BitTorrent, Apache Cassandra, Microsoft Orleans, and HashiCorp Consul commonly use Gossip protocols. They are typically embedded to provide features such as cluster membership (who is in the cluster), failure detection (which members are alive), and event broadcast. Their peer to peer nature often makes them much more scalable and reliable than centralized approaches to solving the same problem. However, the reduced amount of communication makes them sensitive to slow processing of their messages.
Many of our tools leverage work from the academic community, and with HashiCorp Research we hope to contribute back. Our focus is on novel work and whitepapers about the algorithms and system designs we are using in practice. Lifeguard is our first published work, and our users operating the tools in production environments drive the focus of these improvements.
Read on to learn more about Lifeguard.
Jun 28 2017 Paddy Foran
Introducing the HashiBot GitHub Bot
At HashiCorp, we take pride in our community and community contributions. As the quantity and adoption of our open source projects increases, it introduces more complexity in daily maintenance and triage. To help scale the community management of our GitHub repos, we are introducing “HashiBot” on GitHub.
Jun 22 2017 Chris Kent
Recap and Videos From Our First HashiDays in New York City
HashiCorp hosted the very first HashiDays on May 15 in New York. HashiDays is a new series created as one-day, single-track, deeply technical, and community-focused events. HashiDays brings together customers, practitioners, as well as HashiCorp employees to explore and discuss different areas and topics ranging from customer stories and use cases, to technical product exploration.
Jun 21 2017 Matthew Lapworth
HashiCorp Vault helps New Relic manage secrets for their digital intelligence platform
This is a guest post by Matthew Lapworth, Senior Application Security Engineer at New Relic. New Relic is a leading digital intelligence company, delivering full-stack visibility and analytics with more than 14,000 paid business accounts. The New Relic Digital Intelligence Platform provides actionable insights to drive digital business results. Companies of all sizes trust New Relic to monitor application and infrastructure performance so they can quickly resolve issues, and improve digital customer experiences.
At New Relic, our systems and infrastructure had grown, and we were facing challenges with securely storing and managing credentials. HashiCorp Vault has provided us with a consistent approach to manage secrets and credentials.
Jun 17 2017 Rani Osnat
Aqua Security helps enterprises to securely manage secrets in containers with Vault
This is a guest post by Rani Osnat, VP Marketing at Aqua Security. Aqua is a HashiCorp technology partner and focuses on securing container-based applications from development to production, on any platform.
Aqua Security, founded in 2015, focuses on securing applications that are developed and run using virtual containers, e.g. using Docker. We provide automated security controls for the entire lifecycle of containers, starting from development and all the way to protecting container workloads in production. We work with large enterprises that already use containers or are migrating to containers, and have security and regulatory requirements to ensure their applications are protected and monitored.
We chose to integrate with HashiCorp Vault after learning of a customer need for secrets management in containers. Vault is the leading product for secrets management in the enterprise, is widely used by large enterprises, and is easy to integrate with.
Jun 14 2017 Chris Kent
How Vault Encrypts Application Data During Transit and at Rest
Companies today are adopting the cloud and looking for ways to accelerate application delivery. Migrations can often times create challenges around data privacy and secrets management, since distributed applications and infrastructure need to share and transmit data between different components and layers. HashiCorp Vault Encryption as a Service focuses on keeping application data secure across distributed infrastructure.
Jun 09 2017 Seth Vargo
Upcoming Provider Changes in Terraform 0.10
Since 2014, Terraform provider growth has been explosive. At Terraform's initial launch, there were less than ten providers. Today, there are nearly 70 builtin providers that ship with Terraform and countless more are distributed as plugins by the community. Community is and will continue to be the core of Terraform's adoption and success. We are excited to share our future plans for the Terraform provider ecosystem, starting with Terraform 0.10.
May 31 2017 Justin Campbell
Vagrant Cloud Migration Announcement
We are excited to announce that HashiCorp Vagrant features will be extracted from Atlas on June 27th into its own product, HashiCorp Vagrant Cloud.
Features include:Vagrant Box Creation to publish public or private Vagrant boxes for others to consume Vagrant Box Versioning to update boxes and communicate these changes to consumers Vagrant Box Catalog to search and discover public Vagrant boxes
Going forward, Vagrant Cloud will be developed independently, which allows us to improve current functionality and eventually provide new services around Vagrant.
If you only use Vagrant to download and run public boxes, then nothing is changing. All box names, versions, and URLs will stay the same (or redirect) with no changes to your workflow or Vagrantfiles.
Read more to learn about the migration of existing hosted Vagrant boxes and accounts.
May 17 2017 Clément Contini
cloud.ca Develops Custom DevOps-Friendly Provider For Terraform
This is a guest post by Clément Contini, Cloud System Administrator for cloud.ca. cloud.ca is a HashiCorp technology partner and provides Canadian regional cloud infrastructure for companies who have data sovereignty requirements. cloud.ca works as a standalone IaaS platform, or can be part of a hybrid or multi-cloud solution.
The cloud.ca team has developed a purpose-built provider for HashiCorp Terraform, making it possible to automate infrastructure deployments on the cloud.ca platform. Terraform is one of the numerous open-source tools for infrastructure management available from HashiCorp. It provides a DevOps-friendly approach to deploying complex IaaS environments, enabling increased agility and flexibility, particularly when it involves re-use of existing deployment architectures. You can download the cloud.ca provider here: https://github.com/cloud-ca/terraform-provider-cloudca.