Kubernetes Secrets Management with HashiCorp Vault and Rafay

January 22, 2021

Naren Narendra

Kubernetes and containers enable applications to be instantaneously deployed across public clouds, data centers and the edge. While this enables an organization to be nimble and fast, it presents unique challenges for DevOps and DevSecOps personnel needing to secure their applications and data across their infrastructure.

Ensuring security principles such as authentication, privacy, and authorization are properly in place across a complex infrastructure is a daunting task. In order to secure access to various applications and their dependent components, companies typically manually manage hundreds of application usernames and passwords, additional credentials such as those for databases, digital certificates and API tokens, just to name a few.

These credentials are often stored (many times in plain text) in numerous places such as within source code, configuration files, version control systems and even CI/CD tools -- over time leading to secrets sprawl. This typical scenario makes it impossible to keep secrets secret, to reduce the risk of breaches, audit who did what, and effectively rotate secrets over time (a security best practice).

Kubernetes deployments face the exact same challenges and are often exacerbated due to the dynamic and distributed nature of microservices.

HashiCorp Vault offers an elegant and scalable solution to centrally manage and distribute secrets for Kubernetes deployments. In this post, we’ll detail the key features and use cases of HashiCorp Vault and how Rafay’s integration with HashiCorp Vault simplifies the installation and operational overhead associated with secrets management in your Kubernetes infrastructure.

What is HashiCorp Vault?

HashiCorp Vault secures applications running on Kubernetes by centrally managing and controlling access to secrets and encryption keys. Vault can store and dynamically distribute secrets such as tokens, passwords, certificates and encryption keys for secure access to infrastructure and applications. Vault also provides encryption-as-a-service with centralized key management and enables encryption of data in transit and at rest across the cloud, data center and the edge. Here are some of the key features of Vault:

• Secret Storage: Vault supports a plethora of application and service choices as a storage backend for secret storage. Examples include MySQL, Cassandra, Consul, MSSQL, Azure and Google Cloud Storage. Vault offers a set of plugins to integrate with these storage backends.
• Audit Logs: Vault offers detailed audit logs to track client interactions for authentication, token creation, secret access and revocation. This provides a detailed audit trail and can be used to detect security anomalies and breaches.
• Dynamic Secrets: Secrets can be programmatically generated on-demand for users and applications with an expiry timer (TTL) and an option to renew. Hence, secrets do not lurk around and they are generated as and when needed.
• API-Driven Encryption: Vault offers Encryption-as-a-Service by providing HTTPS based APIs to encrypt and decrypt application data.
• Policy-Driven Key Rotation: Encryption keys can be rotated periodically or on-demand with Vault.

Vault offers a comprehensive set of capabilities to manage and distribute secrets in a Kubernetes deployment. However, the creation and deployment of an application securely as a set of microservices using Kubernetes touches application developers and DevOps personnel equally.

Integrating Vault with the Rafay Kubernetes Management Cloud

The Rafay Kubernetes Management Cloud (KMC) is a SaaS-first Kubernetes management solution with enterprise-class scalability, zero-trust security and interoperability for managing apps across public clouds, data centers & edge. To enable security rapidly, Rafay has a seamless and robust integration with Vault.

Here are the key highlights of Rafay’s integration with Vault:

• Establishing Two Way Trust Between Vault and Kubernetes clusters: is a manual task. Rafay makes it very easy and intuitive for DevOps to complete this task.
• System of Record for Vault Configuration: Rafay provides centralized access and visibility of Vault configuration parameters spanning across multiple clusters.
• Simple Annotations to Dynamically Retrieve Secrets from Vault Server: Rafay enables an application developer to secure their application secrets by simply adding a few annotations into the Kubernetes YAML file or helm chart to integrate with Vault.
• Consistent Secrets Management Across Development Streams: With Rafay’s integration of Vault, application developers don’t need to go through the process of setting up Vault on their own.

How to Integrate Vault with the Rafay Kubernetes Management Cloud

Once Hashicorp Vault is installed on your infrastructure, you can follow these steps to easily establish trust between your new, centralized HashiCorp Vault server and your Rafay Managed clusters.

Step 1: Create a Secret Store by logging into the Rafay KMC, clicking on Integrations > Secret Stores, then “New Secret Store”. Provide a name for the secret store, select “Vault” from the Provider drop-down and click “CREATE”.

Step 2: Edit the Secret Store by providing the Vault Host, select the cluster name from the drop-down list and provide an “Auth Path” for the selected cluster. Then select save changes.

Once the Vault integration configuration is deployed to the clusters, copy the related Vault settings for each of the clusters and complete the configuration in Vault.

Step 3: Create an Auth Path for Clusters. In order for Vault to grant access to the clusters to retrieve secrets, the Kubernetes Auth Method for each of the clusters will need to be created in Vault from the information retrieved at the end of Step 2 above.

Application workloads can use Rafay's Secret Store annotations to dynamically retrieve secrets from the Vault server. The secrets are made available to the containers as part of their "Startup Configuration" and can be in the form of environment variables or files.

Kubernetes Secrets Management Made Easy with Vault and Rafay

HashiCorp Vault is a scalable and powerful tool for centrally managing secrets, encryption keys, tokens, passwords and certificates. It can help DevOps and DevSecOps personnel to implement security best practices while significantly reducing the attack surface. Rafay’s Kubernetes Management Cloud provides an easy and seamless integration with Vault, to ensure cross-functional teams are working hand in hand operationally.

The combination of Hashicorp Vault and Rafay enables organizations to rapidly secure their Kubernetes infrastructure and the applications running within it.