HashiCorp has improved and expanded its product suite with a slew of new features. The additions aim to move more users toward paid versions, make more features available first to paying customers, and fully commit to the managed service offerings of the HashiCorp Cloud Platform.
Of course, the existing (free and open-source) community versions of tools like Terraform, Packer, Nomad and Vault will remain available, but mainly for initiates to the HashiCorp magic. Companies that want to take their cloud-native application development to the next level and have access to advanced features such as policy-as-code, remote state management, and multi-tenant capabilities should really move to the enterprise versions of the mentioned tools.
Or perhaps even better, embrace the entire HashiCorp Cloud Platform (HCP). The latter is the company’s managed service that relieves its customers of all sorts of operational tasks in provisioning, securing, connecting, and decommissioning cloud workloads.
‘A lot of customers initially think they can do infrastructure better than we can,’ CEO Dave McJannet divulged during a briefing with the press at HashiConf, the company’s annual event in Boston this year. ‘At the end of the day, many end up saying: you know what, you guys do it.’ That’s how McJannet likes to see it, as his company is increasingly shifting toward large enterprises with the aim of ‘establishing long-term relationships.’ Read: providing managed services to those companies.
Customers from the Fortune 500
At the same time, many new features are still ‘only’ in public beta. HashiCorp, which boasts that 205 of its more than 4,700 paying customers are Fortune 500 companies, will eventually have to make all the fancy new additions generally available if it wants such companies to adopt these features with confidence.
The company acknowledges as much in response to our questions, but emphasizes that those same customers want nothing less than the best product HashiCorp can deliver. That requires extensive testing and a long beta phase if necessary. Most customers are looking for a way to run their cloud-native workloads with as little risk as possible and as scalable as possible. We should view all HashiCorp developments in that light, the company said.
Tip: With Infrastructure Cloud, HashiCorp reinvents its entire product portfolio
To dispel any doubt about HashiCorp’s direction, the new features will only be available in the HCP or enterprise versions of the HashiCorp tools. This article focuses on the capabilities that HashiCorp markets as part of its ‘infrastructure’ portfolio—in other words, everything related to building, deploying, and managing cloud-native infrastructures. Security is the other pillar of HashiCorp’s product offerings, but to keep things clear, those will be covered in another article.
Terraform stacks
One of the most important new additions is the ability to create stacks within infrastructure-as-code tool Terraform. This allows different modules to be lumped together, simplifying their maintenance and interdependency. This functionality is not really new; after all, it has been available in private preview for about a year, but is now coming out in public beta. The feature ensures that the complexity of configurations no longer spills into the work process, (e.g. ‘Do all instances of a given app in their various workspaces have the most up-to-date configuration?’) but limits this complexity to the code used at the beginning of the process, or Day 0, as HashiCorp itself would say.
All components within a stack get their marching orders from the same configuration, eliminating the need for manual tweaks, inputs, tracking, and setting up dependencies across configurations. That is simply all done through the Terraform-config.
What really gives this feature some meat on its bones is the addition of deferred changes, where deployments that encounter too many unknown variables can still proceed (partially) without crippling the entire process. This should especially help accelerate Kubernetes or VM workloads, where the stack functionality actually adds the most value. It is worth remembering that HashiCorp’s products are primarily intended for (often small) platform teams setting up cloud environments that are used by as many as thousands of engineers per customer.
Orchestration rules via code
In addition, admins can now set up orchestration rules via code. This is particularly useful for repetitive tasks with the same acceptance criteria every time. Previously, these criteria had to be redefined for each action, but by programming them directly into the Terraform workflow, this is done fully automatically —once set. As a result, interdependencies of different configurations can be synched automatically (via code), and no manual adjustments are needed afterward.
The example that CTO Armon Dadgar (pictured above) gave during his keynote was that of an app that needs to run on a Kubernetes workspace that does not yet exist. Since deploying a particular app depends on the new workspace to be created, it would give an error message when creating it because the workspace is missing. This requires manual action from the user. Using the orchestration rules within the stacks functionality, this can be automated. This means that Terraform recognizes what is needed to initiate such workloads, limiting mistakes and saving time and effort. Customers can provision 500 resources with stacks in the beta phase.
Users who already have workloads running in the (free) Terraform Community edition and want to scale up to the enterprise or HCP version can now do so automatically. This feature is also in public beta. Users can still review all the details before the migration actually takes place, but otherwise, the new feature should greatly simplify the labor-intensive and error-prone migration process. The fact that free users become paying customers while doing so is a welcome bonus for HashiCorp.
Module lifecycle management
HashiCorp has even more in store for Terraform, which is now increasingly drifting away from its younger sibling OpenTofu, which was forked last year. Yet another feature available in public beta now concerns module lifecycle management for apps and environments already fully running (a Day 2 scenario, in HashiCorp’s parlance). It often proves to be challenging to dismantle these multitudes of modules and apps in a timely manner. This leaves all sorts of unused or obsolete modules wasting space and computing power, as well as posing a security risk.
The new feature allows the team responsible for managing these to set rules for regular depreciation at the very beginning of the process —again, in code— and communicate to app builders, for example, that the version of a module they are using is due for an update.
That message can arrive via email or in Terraform’s workspace UI itself. Change requests, module depreciation, and team notifications are available only in the more expensive Plus version of HCP Terraform. According to HashiCorp, if companies are serious about their workflows, they might as well pay to professionalise these processes.
Short-term workspaces
This series of innovations would not be complete without mentioning one that has actually been generally available since September 18: the introduction of so-called ephemeral workspaces, meaning workspaces only in use for a short time.
As is often the case with HashiCorp, this is a conceptual term. As far as we know, there is no actual feature in Terraform that says ‘create an ephemeral workspace here’. You can create one with new, expanded auto-destroy features that are manageable on a per-project basis. An app, website, tool, or other application gets automatically destroyed —again, via code— after a predetermined period or upon meeting certain criteria. This prevents such apps from passing their expiration date, or worse, hanging about orphaned and vulnerable in users’ systems.
New features for Packer and Waypoint
In addition to flagship Terraform, HashiCorp also announced additions for Packer, the tool for automatically building preconfigured images for provisioning virtual machines or containers. It now offers role-based access control (RBAC) at the bucket level, which allows admins at that top level to very precisely set up entry for users, devices and applications based on the principle of least privileged access. This keeps the most important images out of reach of those with no business snooping around unwanted. This feature was already present in security tool Vault Secrets, but is now coming to other tools, of which Packer is the first.
It is also good to point out that Packer has for some time now offered CI/CD pipeline metadata that can be used to log things like pipeline ID, commits, details about the operating system and other metrics. This increases Packer’s value for compliance purposes.
Waypoint: making app deployment easier
Waypoint is a relatively new addition to the HCP portfolio and facilitates application deployment (an internal developer platform, as it’s called). This way, developers who are not part of the platform team and, as such, are not responsible for the underlying infrastructure can still turn on certain workflows. Of course, these are pre-approved by the infrastructure administrators, who can now offer them as templates or add-ons.
The Waypoint templates are for provisioning the underlying infrastructure, and the add-ons are for managing application dependencies. Both Waypoint and the new add-ons are now generally available, but only to users of HCP Terraform Plus. Incidentally, the functionality to define and execute the actions required to run applications properly using Waypoint is in public beta.
Meaning that, when a user of the cloud environment executes a template for creating a (temporary) application, actions are immediately attached to it. To make that a little less abstract, an action could be to take a web page offline for maintenance, roll back a software version to an earlier version, make a new deployment, or take a snapshot of a database.
Setting up Nvidia GPU quotas with Nomad
To wrap up this series of announcements for the infrastructure portion of HCP, let’s take a quick look at Nomad. This is an orchestration tool for managing containerized and non-containerized applications across different multi-cloud and on-prem environments. The tool can now also manage resources for Nvidia GPUs running AI workloads.
This allows users to use their precious GPUs as efficiently as possible by determining exactly which GPUs are running a given workload at a given time for various users. Furthermore, by assigning quotas, an upper limit can be placed on the use of GPUs per namespace or region. With this announcement, AI has managed to enter the conversation, which seemed conspicuously absent from the list of enhancements and improved functionalities.
Where do updates land first?
It’s good to note that this update for Nomad concerns the enterprise version 1.9 and not the HCP version, at least for now. HashiCorp’s method of updating can sometimes seem a bit confusing. To clarify: unless there is a version number after a product name, it refers to an update for the HCP version. Only enterprise products get numbered versions.
HCP doesn’t have these because HashiCorp manages this suite itself and updates continuously. Communicating version numbers to the outside world wouldn’t make much sense. Enterprise tools are hosted by customers themselves, who are also responsible for setting up the servers and databases as well as upgrading, patching, and bug fixing. Typically, updates and new additions to enterprise tools become available to their counterparts in HCP within a few weeks. The company usually does not publicize that; ‘it just gets done,’ as a company spokesperson explained.
The exception to this rule is Terraform. New features come first to the HCP version and only then to the enterprise version. Other tools, such as HCP Packer, have their own particular update schedule working somewhat differently from the ‘enterprise-first, HCP later’ ritual. ‘It can be a bit confusing’, the company spokesperson acknowledged.
Read also: This ‘Maturity Model’ underlies HashiCorp’s standard for cloud infrastructures