cc by-sa flurdy

Migrate blog to Jekyll

(with CircleCI & Kubernetes)

Migrating my own blog from Blogger to a Jekyll generated blog.

Started: February 2019. Last updated: 14th December 2019.

Background

I posted two blog posts recently: The first post was about that I am moving my blog away from Blogger, and the reasons why. And the second post was that I was moving it to a Jekyll based blog, and specifying briefly why and how.

Aim

This howto is to show how it was done in detail. And not why as that is covered in the blog posts.

This howto will show you how to create a Jekyll site using a Docker container. How to migrate blog posts from Blogger to Jekyll. What theme I chose. How I build my blog with CircleCI, and how to host it with Kubernetes.

My blog is at blog.flurdy.com and the repository is at github.com/flurdy/blog.

Jekyll

Jekyll is a web site generator. It is written in Ruby and was created by GitHub to power their GitHub pages sites.

It outputs static html pages that you can host with any webserver.

Create Jekyll site

Jekyll is normally installed as a Ruby Gem. I do not want Ruby installed locally, so thankfully there is a good Jekyll Docker image that lets you do all the development inside the container.

Lets launch the container with a bash prompt, that mounts our local folder as /srv/jekyll within the container:

docker run --rm --volume="$PWD:/srv/jekyll" -it jekyll/jekyll:3.8 /bin/bash

And inside the container create a basic site and then exit, which destroys the container. However all the files are retained in the mounted volume folder.

jekyll new myblog
exit

Initialise a Git repository inside the folder and launch a container now exposing a port mapping.

cd myblog
git init
docker run --rm --volume="$PWD:/srv/jekyll" -it -p 4000:4000 jekyll/jekyll:3.8 /bin/bash

Lets start the built-in Jekyll webserver from bash within the container.

bundle exec jekyll bundler

Your plain website is now available at localhost:4000. (Assuming your docker host is local)

Edit _config.yml and other files to see how the changes are reflected.

Docker Compose

For ease of development I created a Docker Compose file to quickly launch the Jekyll server. And relaunch it quickly if tweaking the Gemfile or _config.yml which does not auto reload as opposed to most other files.

vi docker-compose.yml version: '3'

services:
  jekyll:
    image: jekyll/jekyll:3.8
    container_name: blog
    environment:
      - JEKYLL_ENV=docker
    command: jekyll serve --force_polling --livereload --drafts
    ports:
      - 4000:4000
      - 35729:35729
    volumes:
      - ./:/srv/jekyll
 docker-compose up

Migrate old Blogger posts

Jekyll does come with a lot of plugins to migrate from other Blog and websites to Jekyll. The Blogger plugin works well.

First export your existing Blogger posts. Move the downloaded xml file into a _import folder for example.

Then run the import script from within a Ruby container, a Jekyll one will do:

ruby -r rubygems -e 'require "jekyll-import";
   JekyllImport::Importers::Blogger.run({
      "source" => "_include/blog-MM-DD-YYYY.xml",
      "no-blogger-info" => false,
      "replace-internal-link" => false,
})'

This will create all your posts as files within _posts.

In Blogger each post of mine had a year and month prefix folder path, but by default Jekyll also each post in day of the month subfolder as well. To maintain the same path/file names I needed to tweak Jekyll's permalink configuration.

vi _config.yml permalink: '/:year/:month/:title:output_ext'

Theme

There are a lot of Jekyll themes out there. Lost of free ones, quite an extensive amount of themes for sale between $25-$50. Some are Ruby Gem based which makes installation a breeze.

There was a few paid ones that looked good, but I find it hard to pay for a theme when I am not sure I would end up using it. However they still got to eat and pay rent so I don't know a better monitisation for them. I would guess a post-pay has a lot lower return rate than pre-pay.

Not all are suitable for a blog based websites. And I had further requirements such as a good sidebar, built in support for tags. In the end I settled on the excellent Galada theme by Artem Sheludko. It is MIT licensed.

I modified the Galada theme a little bit, such as adding a white background to blog posts for easier reading. I added an avatar vanity part to the sidebar. For easier browsing to old posts I added archive links in the sidebar, and archive templates. And I added a common License template for the blog posts content.

To support these changes I added these plugins to the _config.yml and Gemfile files:

vi _config.yml plugins:
  - jekyll-feed
  - jekyll-paginate
  - jekyll-sitemap
  - jekyll-archives vi Gemfile group :jekyll_plugins do
  gem "jekyll-paginate"
  gem "jekyll-sitemap"
  gem "jekyll-archives"
  gem "jekyll-feed", "~> 0.6"
end

CI Build

The beauty of Jekyll is that its output is static html files and the assets are in the _site folder. And that folder can be copied to any webserver and that is all you need.

This set up uses CircleCI as the CI build tool. It will generate the _site folder, create a Docker image, and publish that image to a Docker registry. For this howto the registry used is Quay.io.

For this project I create a GitHub repository that I push each change to. On CircleCI I then added a new build based on this Git repository.

Dockerfile

To generate a Docker image to deploy the blog site anywhere, a very simple Dockerfile based on Nginx image will do.

vi Dockerfile FROM nginx:1.15-alpine

COPY _site /usr/share/nginx/html

Generate _site

To generate the _site these steps are required. (We also unfortunately need to hack a bundler reinstallation step.)

mkdir .circleci
vi .circleci/config.yml version: 2.1
jobs:
  build:
    docker:
      - image: circleci/ruby:2.6.0-node
    environment:
      BUNDLE_PATH: ~/repo/vendor/bundle
      BUNDLE_VERSION: 2.0.1
    working_directory: ~/repo
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "Gemfile.lock" }}
            - v1-dependencies-
      - run:
          name: install bundler
          command: |
            sudo gem update --system
            sudo gem uninstall bundler
            sudo rm /usr/local/bin/bundle
            sudo rm /usr/local/bin/bundler
            sudo gem install bundler
      - run:
          name: install dependencies
          command: |
            bundle check || bundle install
      - save_cache:
          paths:
            - ./vendor/bundle
          key: v1-dependencies-{{ checksum "Gemfile.lock" }}
      - run:
          name: Jekyll build
          command: bundle exec jekyll build
      - persist_to_workspace:
          root: ./
          paths:
            - _site

Build docker image

Building the Docker image is relatively straight forward. We also save the image as a tar file to be reused in the later publish step.

build-image:
  docker:
    - image: circleci/buildpack-deps:stretch
  environment:
    IMAGE_NAME: quay.io/yourusername/myblog
  steps:
    - checkout
    - attach_workspace:
        at: ./
    - setup_remote_docker
    - run:
        name: Build Docker image
        command: docker build -t $IMAGE_NAME:latest .
    - run:
        name: Archive Docker image
        command: docker save -o image.tar $IMAGE_NAME
    - persist_to_workspace:
        root: .
        paths:
          - ./image.tar

Docker registry

Before you upload any Docker images anywhere you need to create a Docker registry for it to upload it.

For this howto we create a Quay.io repository. And a Quay robot user with write permissions to authenticate with the repository.

Publish Docker image

To upload the Docker image to the registry a few _run_ commands are needed. Also you need to add into CircleCI's project settings two environment variables: DOCKER_LOGIN and DOCKER_PASSWORD.

From Quay.io these are from the robot user authentication details.

publish-image:
  docker:
    - image: circleci/buildpack-deps:stretch
  environment:
    IMAGE_NAME: quay.io/yourusername/myblog
  steps:
    - setup_remote_docker
    - attach_workspace:
        at: ./
    - run:
        name: Load archived Docker image
        command: docker load -i image.tar
    - run:
        name: Upload to registry
        command: |
          echo "$DOCKER_PASSWORD" | docker login quay.io -u "$DOCKER_LOGIN" --password-stdin
          docker push $IMAGE_NAME:latest
          IMAGE_TAG="1.0.${CIRCLE_BUILD_NUM}"
          docker tag $IMAGE_NAME:latest $IMAGE_NAME:$IMAGE_TAG
          docker push $IMAGE_NAME:$IMAGE_TAG

We tie all these up with a CircleCI workflows section:

workflows:
  version: 2
  build-master:
    jobs:
      - build:
          filters:
            branches:
              only: master
      - build-image:
          requires:
            - build
          filters:
            branches:
              only: master
      - publish-image:
          requires:
            - build-image
          filters:
            branches:
              only: master

For each Git push that triggers a CircleCI build you should now also have Docker images in Quay.io.

Hosting

You can host a Jekyll site in so many ways. You can use AWS S3, GitHub Pages and may others.

I prefer to use my own Kubernetes solution. That way it can use any plugins it wants and is not tied to a service provider.

If interested in Kubernetes you can choose between so many providers these days: Your own Kubernetes the-hard-way set up, using Google's GKE, using Amazon's EKS, using Microsoft Azure's AKS, etc.

For my blog I chose Digital Ocean's recent Kubernetes solution.

Digital Ocean

In Digital Ocean's console I create a new Kubernetes cluster. I add one node to it.

I then download the kubectl config to authenticate with the cluster. I make sure this works by getting the cluster information:

kubectl cluster-info

Note: Digital Oceans' _doctl_ tool makes updating your Kubernetes credentials a lot easier.

Kubernetes secret

To enable my K8s setup to download images from Quay I need to create a secret.

First I go into Quay.io's console for my docker registry to add another robot user, this time with only read permissions.

Then I create the secret with:

kubectl create secret docker-registry myblog-registry --docker-server=quay.io \
--docker-username:MYQUAYROBOT --docker-password=MYQUAYROBOTPASSWORD

Kubernetes deployment

A simple deployment that refers to the Quay.io stored image and secret.

mkdir kube
vi kube/deployment.yml apiVersion: apps/v1
kind: Deployment
metadata:
  name: myblog-deployment
spec:
  selector:
    matchLabels:
      app: myblog
  replicas: 2
  template:
    metadata:
      labels:
        app: myblog
    spec:
      containers:
        - name: myblog-container
          image: quay.io/yourusername/myblog:1.0.2
          ports:
            - containerPort: 80
      imagePullSecrets:
        - name: myblog-registry

Note the tag 1.0.2. That must be which ever tag the latest Docker image tag in the registry. An alternative is to use latest, but it is a bit arbitrarily to which latest is currently deployed...

You then apply it to K8s:

kubectl apply -f kube/deployment.yml

And take a look at it with either:

kubectl get deployments kubectl describe deployments myblog-deployment

Kubernetes service

We also need a service to expose the deployment. You can go direct with a NodePort type of service, but I use Load balancer that uses Digital Ocean's cloud controller.

It is relatively simple:

vi kube/service.yml apiVersion: v1
kind: Service
metadata:
  name: myblog-service
spec:
  type: LoadBalancer
  selector:
    app: myblog
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

You then apply the service to K8s:

kubectl apply -f kube/service.yml

And then keep an eye on it until it receives an external IP:

kubectl get service --watch kubectl describe service myblog-service

When it has an external IP, then that is it. The blog is live. Just add the IP to a DNS entry somewhere.

Blog posts procedure

I draft blog posts using markdown in the _drafts folder. I do not prefix with a date. Locally the Docker Compose set up will pick the drafts up as blog posts as it has the --drafts argument.

mkdir _drafts
vi _drafts/a-new-blog.md # A new blog!

Blah blah

When the draft post is ready to be published, I move it to the _posts folder and prefix with a date.

mv _drafts/a-new-blog.md _posts/2019-02-20-a-new-blog.md

I commit this to git and push to the origin repository. CirceCI starts a build, I get notified of the new docker image tag by Quay.io. I then update the tag number in the K8s deployment.yml file and reapply it to active the blog post on the live site:

vi kube/deployment.yml ...
    image: quay.io/yourusername/myblog:1.0.43
... kubectl apply -f kube/deployment.yml

Ps. I also temporarily move most of the posts out of _posts whilst developing for a faster site generation feedback loop. Do not commit these removals to git.

Continuous deployment

A later addition to my blog set up is to automatically deploy my blog images to Kubernetes. I.e. a type of continuous deployment.

For this I use Keel. Keel lets you set labels that decide how a deployment container is updated.

I installed Keel via Helm. Please read my Kubernetes Ingress & TLS with Helm doc on how to install Helm.

I then modified my blog depoyment.yml with:

vi kube/deployment.yml ...
metadata:
  name: myblog-deployment
  labels:
    keel.sh/policy: major
    keel.sh/trigger: poll
  annotations:
    keel.sh/pollschedule: "@every 10m"
... kubectl apply -f kube/deployment.yml

So whenever a new Semver tag of my blog image is uploaded to my registry (by CircleCI), Keel updates my Kubernetes deployment.

Feedback

Please fork and send a pull request for to correct any typos, or useful additions.

Buy a t-shirt if you found this guide useful. Hire me for short term advice or long term consultancy.

Otherwise contact me. Especially for things factually incorrect. Apologies for procrastinated replies.