Kubernetes Docker Registries Cookbook

Showing simple recipes on how to use various Docker registries with Kubernetes.

Docker registries details are configured in Kubernetes as Secrets. A specific docker-registry type of Secret.

Also examples of encrypting the secrets with Sealed Secrets for a GitOps Flux based cluster.

Kubernetes cluster

• The hard way
• EKS
• GKE
• AKR
• DOKS

Kubectl

• kubernetes.io/docs/tasks/tools/install-kubectl/
  • brew install kubectl

Provider CLI

Optional.

• eksctl
  • brew tap weaveworks/tap;
brew install weaveworks/tap/eksctl
• gcloud
  • brew install google-cloud-sdk
• az
  • brew install azure-cli
• doctl
  • brew install doctl

Sealed Secrets

Optional.

• github.com/bitnami-labs/sealed-secrets
• github.com/flurdy/lemmings/#sealed-secrets
  • helm install sealed-secrets
  • brew install kubeseal

jq

Optional.

• stedolan.github.io/jq/
  • brew install jq

• cloud.docker.com

Docker Hub now finally offer access tokens.

1. Go into your Docker Hub account security
   • hub.docker.com/settings/security
2. Click on New Access Token to create an access key
   • Give it a name, e.g. cluster name.
   • Copy the access token
3. Create a registry file
   • kubectl create secret docker-registry dockerhub-registry \
  --dry-run -o yaml \
  --docker-server=https://index.docker.io/v1/ \
  --docker-username=YOUR_USERNAME \
  --docker-email=YOUR_EMAIL \
  --docker-password=YOUR_ACCESS_TOKEN \
  > dockerhub-registry.yml
   • Replace your username, email and the recently created access token.

If you do not use GitOps and sealed secrets, dockerhub-registry.yml can now be applied to the cluster.

• kubectl apply -f dockerhub-registry.yml

• quay.io

Quay uses robot accounts to access their registry.

1. Log in to Quay.
2. Go to account settings.
3. Click robot accounts tab
   • quay.io/user/YOURUSERNAME?tab=robots
4. Click on Create Robot Account
   • Give it a name, e.g. cluster name. Note lowercase alphanumerical only.
   • Describe intended use.
   • Click Create robot account
5. Specify permission required for each repository.
   • Click Add permissions
6. Click on the settings cog on your new YOURUSERNAME+ROBOTNAME account row.
   • Select View Credentials
7. Either download the secret directly from the Kubernetes Secret tab or create the secret manually by:
   • Select the Robot Token tab.
   • Note username and token.
   • kubectl create secret docker-registry quay-registry \
  --dry-run -o yaml \
  --docker-server=https://quay.io \
  --docker-username=USERNAME+ROBOTNAME \
  --docker-password=ROBOT_TOKEN \
  > quay-registry.yml

If you do not use GitOps and sealed secrets, quay-registry.yml can now be applied to the cluster.

• kubectl apply -f quay-registry.yml

• cloud.google.com/container-registry/

For GCR you need a JSON token from GCP's IAM.

1. Log into your GCP project's IAM section, specifically Service Accounts.
   • console.cloud.google.com/iam-admin/serviceaccounts
2. Click on Create Service Account.
   • Give it a decent name.
   • And describe its intentions, i.e. GCR read only. Or write if you are pushing builds as well.
3. Click on Create
4. Select the Select a role dropdown
   • Choose Cloud Storage
   • And Storage Admin on the submenu
5. Click on Continue
6. No need to grant any users, so click done
7. Find new service account on the list and click on it to edit (the email address is clickable
8. Click on the Add key button, select "Create new key" on the popup.
   • Choose JSON
9. Click on Create
   Save the JSON file e.g. as gcr-registry.json.

Note GCP sometimes changes this flow.

Then we need to create a Kubernetes Secret using the JSON as a one-line data token.

1. kubectl create secret docker-registry gcr-registry \
  --dry-run -o yaml \
  --docker-server=https://gcr.io \
  --docker-username=_json_key \
  --docker-email=[email protected] \
  --docker-password=(jq -c . gcp-service-account.json) \
  > gcr-registry.yml
   • I use Fish as my shell, so the docker-password argument was generated using
     • --docker-password=(jq -c . gcp-service-account.json)
   • With Bash, you may use
     • --docker-password="$(jq -c . gcp-service-account.json)"

If you do not use GitOps and sealed secrets, gcr-registry.yml can now be applied to the cluster.

• kubectl apply -f gcr-registry.yml

If you use GitOps, you would want to encrypt the registry secrets before adding only the encrypted files to version control.

1. First fetch the Sealed Secret public key.
   • kubeseal --fetch-cert \
  --controller-namespace=kube-system \
  --controller-name=sealed-secrets \
  > sealed-secrets-cert.pem
2. Then for every registry secret
   • Seal the secret
     • kubeseal --format=yaml \
  --cert=sealed-secrets-cert.pem \
  < gcr-registry.yml \
  > sealed-gcr-registry.yml
   • Then only commit the sealed secret to git. And never the plain text secret.
     • git add sealed-gcr-registry.yml;
rm gcr-registry.yml

Please contribute recipes for other registries:

• AWS' ECR
• Azure's AKR
• Harbor
• Gitlab Container Registry
• Github Packages
• DigitalOcean Container Registry
• JFrog Container Registry
• Sonatype Nexus
• Docker Registry

I may eventually add further registry examples myself.

Please fork and send a pull request with recipes. Please also submit PRs to correct any typos, or useful additions.

Hire me for short term advice or long term consultancy. Sponsor me to create more howtos.

Otherwise contact me. Especially for things factually incorrect. Apologies for procrastinated replies.