The vision with tools like Terraform and Vault isn't unique to cloud," McCaul says. "The technology is so flexible and applicable to other areas of our business that we're eager to share our learnings with colleagues across the organization and roll out common tools that will position us all for sustained success for years to come.
Ryan McCaul, Lead Cloud Architecture and Automation at TMX Group
Like all modern businesses, TMX Group needed to digitize its operations across its various holdings to remain competitive within the market and profitable. The company wanted to streamline its operations, improve data analytics capabilities, and reduce operating costs by moving more of its core operations to the cloud. But diverse business objectives and a mix of legacy and cloud infrastructure made it difficult for the company's cloud technology team to align the talent and resources needed to put all those plans in action.
"We weren't fully realizing the efficiencies and subsequent benefits we expected from our cloud operations. Different groups across our organization brought diverse skills, experiences, and philosophies with respect to cloud technology," says Mark Salam, TMX Group's Director of Cloud Technology. "We realized that we needed a way to standardize our deployment methodology and infrastructure management by automating provisioning across all our products and services."
Misaligned philosophies lead to inefficiencies and increased costs
TMX Group operates a number of small, dedicated teams focused on a range of data analytics subscription services, venture exchanges, and other web properties aimed at monetizing the firm's expansive, proprietary data stores.
"Every one of our line-of-business teams has unique objectives and respective strategies for how to achieve them," Salam says. "but often these approaches didn't fully align, which spread our team thin and impacted efficiencies due to the additional time and effort dedicated to making everything work to deliver on time."
In response, Salam's team sought to simplify the firm's cloud infrastructure provisioning using native provisioning tools. "At first glance, the native toolset seemed like a great option because it is designed specifically for our preferred cloud environment," says Ryan McCaul, the cloud architecture and automation lead at TMX Group. "Over time we discovered that we needed something more module friendly that would allow us to build reusable modules that were pre-approved for our security and infrastructure guidelines and that we could share with developers who could use it without a lot of training or ramp up time."
Accelerating infrastructure provisioning across multiple business units and functions
Standardizing deployment methodologies for greater efficiency, security, and auditability
Eliminating burdensome manual access key management practices
Over time we discovered that we needed something more module friendly that would allow us to build reusable modules that were pre-approved for our security and infrastructure guidelines and that we could share with developers who could use it without a lot of training or ramp up time.
Ryan McCaul, Lead Cloud Architecture and Automation at TMX Group
Automatic for the people
Rather than spend more valuable time testing out a range of automation tools, TMX Group opted to use one from HashiCorp, which was already assisting the company by providing data governance services. "We were already in the middle of implementing HashiCorp Vault for our secrets and data management," McCaul says. "Terraform seemed like a great way to tackle both our infrastructure and secrets management provisioning from a single ecosystem."
With Terraform, TMX Group developers easily package infrastructure as code using a simple, human-readable language into modules that are reused across a variety of functions and accessed by any authorized user to support greater collaboration and efficiency.
Users preview an execution plan to see exactly what the solution will do when code templates are applied to the infrastructure — and in what order — to optimize time-to-value while avoiding unintentionally destroying critical infrastructure resources. At the same time, the pre-approved templates give company leaders the peace of mind that everything the team provisions meets the compliance standards for transparency and security.
"Terraform makes it quick and easy to stand up essential infrastructure to support both our DevOps pipeline and containerized environment by automating everything via a code base," McCaul explains. "High-priority activities like establishing an elastic containers pipeline that used to take as long as a month to setup and fine tune manually now take just a day to set up in a development environment and be in full production in just a week if necessary — without any of the security or compliance concerns we had in the past due to the use of pre-approved modules."
Unified disparate deployment philosophies into a standardized, automated methodology
Enabled reusable code templates for use across business units and use cases
Built automated policy enforcement for security and compliance into the provisioning workflow
Enhanced data security and auditability
Reduced access key renewal time from 180 days to 7 days
Accelerated infrastructure provisioning by up to 75%
Decreased the time to deploy containers from one month down to one day
Moved from development to production in one week
TMX Group uses Terraform and Vault to automate and streamline infrastructure deployment and secrets management across its various lines of business and the rest of its extensive portfolio of markets and exchanges.
Safe, secure, and transparent automation now and in the future
Salam says that in addition to automating complex infrastructure deployments, the use of Terraform and Vault has also dramatically improved the company's governance practices by automating access key rotation within its existing environment.
"We used to manage access keys manually by rotating and recycling them periodically, which was a tedious exercise " he says. "With Terraform, we can automatically configure and populate Vault so end users can manage their own access keys without our help, which has shortened the entire process considerably."
Both Salam and McCaul anticipate HashiCorp to play an increasingly central role in TMX Group's operations in the future. Eventually, TMX Group intends to use Sentinel's embedded policy-as-code framework to help with security, compliance and management of operational consistency across the business.
In addition to provisioning with Sentinel, building out an image pipeline with HashiCorp Packer and simplifying service discovery with HashiCorp Consul, the team hopes to extend the HashiCorp solutions to other less cloud-heavy areas of the business.
"The vision with tools like Terraform and Vault isn't unique to cloud," McCaul says. "The technology is so flexible and applicable to other areas of our business that we're eager to share our learnings with colleagues across the organization and roll out common tools that will position us all for sustained success for years to come."
TMX Group Partners
Mark Salam Director of Cloud Technology TMX Group
Mark Salam, Director of Cloud Technology at TMX Group oversees the adoption and use of public cloud. More than 15 years of experience in Information Technology have culminated in a focus on leveraging cloud technology to achieve ever increasing levels of organizational innovation, efficiency and security.
Ryan McCaul Lead Cloud Architecture and Automation TMX Group
Ryan McCaul, Lead Cloud Architecture and Automation at TMX Group, leads the Cloud Architecture and Automation Programs. Over 12 years of experience in Information Technology with a current focus on building practices around automation and DevOps.
- AWS, Google Cloud, On-premises bare metal
- VMs, Containers, Serverless