Kubers - Kubernetes Remote Secrets
Kubers is a simple implementation of a SecOps pattern, where a sidecar or init container is responsible for retrieving secrets from services like Azure KeyVault and making them available to the user’s application via shared memory volume.
Supported providers and services
- Azure (KeyVault)
- AWS (Secrets Manager)
kubectl apply -k firstname.lastname@example.org:jacops/kubers.git
Please go to
./chart folder for instructions and available configuration.
Enable agent injection by adding the following annotation:
Set agent provider by adding the following annotation (for example
This annotation can be optional if the provider is set globally in the
To configure secret injection, please add the following annotation:
unique-name- the filename of the rendered secret and must be unique if multiple secrets are defined by the user.
service-type- Type of a service used for retrieving the secret. For example:
service-name(optional for some services) - Name of a remote service where the secret is stored.
key-name- Key name under which the secret is stored in the remote service.
spec: template: metadata: annotations: kubers.jacops.pl/agent-provider: "azure" kubers.jacops.pl/agent-inject: "true" kubers.jacops.pl/agent-inject-secret-htpasswd: keyvault://examplekv/nginx-htpasswd
The secret should be available from the main container in
Below is a table containing other kubers annotations.
||Secret with Azure credentials. See
||Secret with AWS credentials. See
||AWS Region. Optional if region is set globally.|
||Overrides a default docker image for an agent.|
||Specifies where the secrets are to be mounted after fetching.|
||Specifies where the
||If enabled will preserve the case of secret name. By default the name is converted to lower case.|
||If enabled will preserve the case of
||Overrides log level for agent. Possible values:
||Overrides log format for agent. Possible values:
Every secrets manager service requires a user to authenticate against. While this responsibility is offloaded from your application, the sidecar container still needs to do this.
Below is the guide on how to authenticate against different service providers.
There are two ways to authenticate against Azure KeyVault. Service Principal and Managed Identity.
Managed Identity (recommended)
Using a system/user assigned identity has many benefits and in my opinion the main one is that you don’t have to use any credentials and worry about secrets rotation.
To use Azure managed identities with your Kubernetes cluster, you need to install
AAD Pod Identity: https://github.com/Azure/aad-pod-identity and assign some identities to pods as per instructions.
Obviously this will only work in AKS and AKS-engine based clusters.
This method requires you to create a service principal and a Kubernetes secret with the credentials, from which the sidecar container can authenticate itself against a KeyVault.
Using service principal is not recommended and should be only used in development or if you are in a non Azure environment.
To configure kubers agent to use service principal authentication, please add the following annotation to your pod:
secret-name-of-the-sp-credentials secret should be structured as below:
data: clientId: xxx clientSecret: xxx subscriptionId: xxx tenantId: xxx
The agent injector supplies the sp credentials to the sidecar via environmental variables. While this is not the greatest method, it works well with
azure-sdk-for-go. This however, can be refactored in the future.
There are two ways to authenticate against AWS Secrets Manager. API Access Keys and IAM role.
IAM role (recommended)
This authentication method is recommended as it doesn’t require a user to distribute API access keys and worry about rotation and access to them.
kubers with IAM roles, a user needs to install an application like: https://github.com/jtblin/kube2iam.
No extra configuration is needed on
API Access Keys
This method requires you to create a pair of API access keys and a Kubernetes secret with the credentials, using which the sidecar container can authenticate itself against a secrets Manager.
Using API Access Keys is not recommended and should be only used in development or if you are in a non AWS environment.
kubers agent to use API Access Keys for authentication, please add the following annotation to your pod:
<secret-name> secret should be structured as below:
data: accessKeyId: xxx secretAccessKey: xxx
Please go to
This repository is heavily inspired by https://github.com/hashicorp/vault-k8s. Huge kudos to the Hashicorp folks for the work they have done around the Vault integration with Kubernetes.