cc by-sa flurdy

Kubernetes Docker Registries Cookbook

Recipes for using various Docker registries with Kubernetes.

Started: December 2019. Last updated: 7th January 2020.

( This document has recently been published. There may still be many typos and code errors. Please let me know. )

Aim

Showing simple recipes on how to use various Docker registries with Kubernetes.

Docker registries details are configured in Kubernetes as Secrets. A specific docker-registry type of Secret.

Also examples of encrypting the secrets with Sealed Secrets for a GitOps Flux based cluster.

Prerequisite

Kubernetes cluster
Kubectl
Provider CLI

Optional.

  • eksctl
    • brew tap weaveworks/tap;
      brew install weaveworks/tap/eksctl
  • gcloud
    • brew install google-cloud-sdk
  • az
    • brew install azure-cli
  • doctl
    • brew install doctl
Sealed Secrets

Optional.

jq

Optional.

Docker Hub Recipe

Docker Hub now finally offer access tokens.

  1. Go into your Docker Hub account security
  2. Click on New Access Token to create an access key
    • Give it a name, e.g. cluster name.
    • Copy the access token
  3. Create a registry file
    • kubectl create secret docker-registry dockerhub-registry \
        --dry-run -o yaml \
        --docker-server=https://index.docker.io/v1/ \
        --docker-username=YOUR_USERNAME \
        --docker-email=YOUR_EMAIL \
        --docker-password=YOUR_ACCESS_TOKEN \
        > dockerhub-registry.yml
    • Replace your username, email and the recently created access token.

If you do not use GitOps and sealed secrets, dockerhub-registry.yml can now be applied to the cluster.

  • kubectl apply -f dockerhub-registry.yml

Quay Recipe

Quay uses robot accounts to access their registry.

  1. Log in to Quay.
  2. Go to account settings.
  3. Click robot accounts tab
    • quay.io/user/YOURUSERNAME?tab=robots
  4. Click on Create Robot Account
    • Give it a name, e.g. cluster name. Note lowercase alphanumerical only.
    • Describe intended use.
    • Click Create robot account
  5. Specify permission required for each repository.
    • Click Add permissions
  6. Click on the settings cog on your new YOURUSERNAME+ROBOTNAME account row.
    • Select View Credentials
  7. Either download the secret directly from the Kubernetes Secret tab or create the secret manually by:
    • Select the Robot Token tab.
    • Note username and token.
    • kubectl create secret docker-registry quay-registry \
        --dry-run -o yaml \
        --docker-server=https://quay.io \
        --docker-username=USERNAME+ROBOTNAME \
        --docker-password=ROBOT_TOKEN \
        > quay-registry.yml

If you do not use GitOps and sealed secrets, quay-registry.yml can now be applied to the cluster.

  • kubectl apply -f quay-registry.yml

GCR - Google Container Registry Recipe

For GCR you need a JSON token from GCP's IAM.

  1. Log into your GCP project's IAM section, specifically Service Accounts.
  2. Click on Create Service Account.
    • Give it a decent name.
    • And describe its intentions, i.e. GCR read only. Or write if you are pushing builds as well.
  3. Click on Create
  4. Select the Select a role dropdown
    • Choose Cloud Storage
    • And Storage Admin on the submenu
  5. Click on Continue
  6. No need to grant any users, so click done
  7. Find new service account on the list and click on it to edit (the email address is clickable
  8. Click on the Add key button, select "Create new key" on the popup.
    • Choose JSON
  9. Click on Create
      Save the JSON file e.g. as gcr-registry.json.

Note GCP sometimes changes this flow.

Then we need to create a Kubernetes Secret using the JSON as a one-line data token.

  1. kubectl create secret docker-registry gcr-registry \
      --dry-run -o yaml \
      --docker-server=https://gcr.io \
      --docker-username=_json_key \
      --docker-email=[email protected] \
      --docker-password=(jq -c . gcp-service-account.json) \
      > gcr-registry.yml
    • I use Fish as my shell, so the docker-password argument was generated using
      • --docker-password=(jq -c . gcp-service-account.json)
    • With Bash, you may use
      • --docker-password="$(jq -c . gcp-service-account.json)"

If you do not use GitOps and sealed secrets, gcr-registry.yml can now be applied to the cluster.

  • kubectl apply -f gcr-registry.yml

Sealing registry secrets

If you use GitOps, you would want to encrypt the registry secrets before adding only the encrypted files to version control.

  1. First fetch the Sealed Secret public key.
    • kubeseal --fetch-cert \
        --controller-namespace=kube-system \
        --controller-name=sealed-secrets \
        > sealed-secrets-cert.pem
  2. Then for every registry secret
    • Seal the secret
      • kubeseal --format=yaml \
          --cert=sealed-secrets-cert.pem \
          < gcr-registry.yml \
          > sealed-gcr-registry.yml
    • Then only commit the sealed secret to git. And never the plain text secret.
      • git add sealed-gcr-registry.yml;
        rm gcr-registry.yml

Contribute recipes

Please contribute recipes for other registries:

I may eventually add further registry examples myself.

Please fork and send a pull request with recipes. Please also submit PRs to correct any typos, or useful additions.

Feedback

Hire me for short term advice or long term consultancy. Sponsor me to create more howtos.

Otherwise contact me. Especially for things factually incorrect. Apologies for procrastinated replies.