/ ghost

Ghost, Digital Ocean and Terraform

Level - Advanced. Read Time ~ <20 minutes.

tl;dr

Repo https://github.com/blairg/ghost-setup.

I did a talk at Leeds JS too on most of the topics I'm covering, you can find that here.

Will cover the following topics

This article is littered with hyperlinks. Hopefully this will help with areas which I have glossed over or where I have made assumptions of prior knowledge. If any of this is unclear, wrong or nonsensical please leave me a comment and I'll try my best to improve the quality and readability of the article.

learning

Why did I want to blog?

Around 3 years ago I started http://hackerlite.xyz. The reason for setting up my own blog was to share back with the tech community primarily. By sharing my experiences and knowledge this felt like a positive outlet and it has been. Wish I'd done more, but it what it is.

I've enjoyed reading technical articles for many years prior to attempting to write my own. I never paid that much attention to the technology they written in or the platform they were hosted on. I got the impression there were a lot of WordPress blogs out there. Due it's very wide exposure and the ease you can get up and running with it. At that point in my career I was doing a lot of JavaScript in my spare time and also bits at work. So, I thought what is there in the JavaScript ecosystem to blog with. I'll be honest I didn't explore the alternatives, I quickly discovered Ghost Blog and proceeded to learn and love it.

ghost_logo

Docker

Docker was very much on my radar at this point of time. We had been using it at work for our local development environment with great enjoyment and success. This is not a post about Docker, but if you have never used Docker I strongly urge you to have a play. Vast swathes of online documentation, books, podcasts, YouTube videos etc to help your educational endeavours. Prior to using Docker, I was using vagrant, there is nothing wrong with Vagrant but it is significantly different and can have a much slower boot time. Consult this post for a better explanation than I can articulate. Oh, and Hashicorp are a great company, I have nothing but admiration for them. Check out their other projects. Right, so I've blown the Docker trumpet. But low and behold there is an adequately placed official image on Docker Hub. The story continues from here...

docker_facebook_share

The next step was to start playing with Ghost using the official Docker image. So, in my trusty zsh terminal, I ran the below Docker command: -

$ docker run -d --name some-ghost ghost

After the above command had executed, I headed towards http://localhost:2368/ in Chrome. After having a look at the pre-populated articles and the general look and feel of the blog, I rapidly delved into the blog writing part by heading to http://localhost:2368/ghost. After establishing user credentials, I was presented with a page very similar to the below: -
ghost1editor

Digital Ocean

Once I'd had a play with Ghost running in Docker locally, the Ghost blogging platform felt very suitable to cater for my blogging needs. Didn't have a huge amount of content I desired to share initially but Ghost appeared very functional and feature rich. So, to follow on from this running locally, I wanted to progress to having it running somewhere in cloud. I was already familiar with Azure and AWS. From having commercial experience with them, either was an obvious choice. I didn't end up opting for either as I wasn't sure how much a VM would actually cost per month. Their pricing schemes both looked equally confusing. Bringing me to the conclusion I'd like to shop around. A colleague at work mentioned Digital Ocean offering small VM's for $5 a month. Which sounded very affordable indeed! Hosting my Ghost blog was the next piece of the puzzle in my journey to running my own blog.

DigitalOcean_logo

So, in Digital Ocean land if you want to spin up your own VM, these are called Droplets. By looking on their website, you can see what they offer. They offer the ability to create a Linux VM running one of the following distributions: Ubuntu, CentOS, Debian, FreeBSD, Fedora. In my case I choose CentOS, mainly because I've heard positive remarks about it and I wanted learn it too. The next option you are presented with is the size of droplet you desire. Bigger droplet equals costlier. Because I wanted to run it cheaply and I wasn't expecting many visitors, I opted for the entry level price of $5 per month.

Docker Compose

After I decided upon running the Docker Ghost image on Digital Ocean, my next decision was how I was going to run it on the Droplet. Just use a docker run command, Docker Swarm or Docker Compose. I initially thought running a docker run command to be a bit verbose and native. Especially when you start to specify ports, volumes etc. As I'd configured Docker Compose for a local development environment, I thought why not try it on the Droplet. May just satisfy my use case. So, I ended up with a docker-compose.yml file similar to below: -

version: '2'
services:
  ghostblog:
    image: ghost
    restart: always
    volumes:
      - /home/ghostblog/data:/var/lib/ghost/content
    expose:
      - "2368"

docker-compose-drupal

NGINX

Another technology I had been learning and getting stronger with at work was NGINX. So, in my wisdom I thought why not run NGINX in front of the Ghost blog. This can be easily wired together and proxy requests from NGINX directly to the Ghost blog. My main reasons for doing this was performance related, also I could lean on the caching functionality of NGINX. The Docker Compose file I settled on can be seen below.

version: '2'
services:
  nginx:
    build: .
    ports:
      - "80:80"    
    links:
      - ghostblog:ghostblog
  ghostblog:
    image: ghost
    restart: always
    volumes:
      - /home/ghostblog/data:/var/lib/ghost/content
    expose:
      - "2368"

I also did some Googling into the best NGINX config solutions. The final NGINX config I engineered can be found here (I'm sure people out there can do much better than this :)). It included things like SSL support with certs, caching and HTTP/2 enabled. I was also hopeful in adding additional security proxying request through NGINX. By denying X-Frame-Options and X-Content-Type-Options. I'm by no means a security expert/consultant, but I can't see settings like this making my blog less secure or less performant. I did some benchmarks at the time and the results were promising. Unfortunately, I no longer have these results to hand.

Domain Registration

The blogging platform, running of the Docker image and hosting environment had been chosen. My next step was to acquire a domain for my Blog. This would enable my blog to be search engine friendly and give it a brand and identity. The most recognisable company to me was Go Daddy. Lots of options out there, but Go Daddy felt easy and quick to use. After searching on their website for the .com, .co.uk, .info domains etc. I actually found the .xyz was super cheap and a bit quirky. The name hackerlite didn't have much thought put into it, just a name I quickly came up with. And decided it will do :)

To get http://hackerlite.xyz pointing at my Digital Ocean droplet. I first had to change the DNS in Go Daddy to point to the Digital Ocean Name Servers. A tutorial which I followed can be found here. After doing this is I set up the A and AAAA records. I didn't set an MX record as I wasn't exploring email possibilities at this juncture. This was done in the networking section on the Digital Ocean UI.

godaddy

Let's Encrypt

I ran for quite a while over port 80 and HTTP. For two reasons really, one I'd not purchased a certificate before and the other reason I had to learn how to use a certificate in NGINX. Over time I gained knowledge and understood HTTPS is a superior way of communication. Which led me on a path to make my blog secure end-to-end.

My first port of call was to enquire on Go Daddy to see what they had to offer and their pricing scheme. It didn't look too expensive, but there were a few verification steps and I thought what other options are out there. Then Let's Encrypt popped into my head, I had originally gained sight of Let's Encrypt at Thought Works Tech Radar event in Manchester. Let's Encrypt is also free, and being from Yorkshire, this was highly desirable.

Let's Encrypt makes life very easy for you by offering a neat utility called Cert Bot. You tell it your webserver (NGINX) and flavour of Linux (CentOS 7), it then provides detailed clear instructions of how to request a fresh certificate for your domain. A caveat to Let's Encrypt certificates is the validity length. They only last for 90 days as opposed to conventional certificates which last 1 year+. But they offer a neat way you can run a cron to auto-renew your certificate. These guys have you covered :)

I won't lie and say Let's Encrypt worked first time for me. But this was my own fault and not their offering or implementation. With my setup of spinning up a NGINX container and proxying requests to Ghost Blog with Docker Compose, I was preventing Cert Bot from working as natured intended. What I had to do was stop my Docker Compose setup and run NGINX natively on Cent OS. I was missing having the path /\.well-known/acme-challenge/ open in my NGINX config. To see how the NGINX config had to be configured see the below example and for a complete solution head here: -

location ~ /\.well-known/acme-challenge/ {
    allow all;
    root /var/www/letsencrypt;
    try_files $uri =404;
    break;
}

Once I had figured out the above solution. I was able to successfully request a certificate from Let's Encrypt using Cert Bot. So, I had acquired a certificate, the obvious next step was to use it. I had to make amendments to my Dockerfile as seen here. I manually moved the certificates into a relative folder named sslcerts for convenience when building my Docker image. To make use of the certificate in NGINX I had to add references, see here. After the Dockerfile and NGINX changes had been made, I then ran docker-compose up, headed to Chrome and I got that lovely padlock for https://hackerlite.xyz.

le-logo-

My Customisations

If you scan my docker-compose.yml configuration found here, you'll notice I mount volumes on the host droplet. The reason for doing this is that wanted to improve the user experience by adding the following features: -

Service Worker

By adding a service worker, I was able to make the blog perform perfectly well offline, providing the user had already hit the page prior and their browser had cached it. Other benefits of having a service worker are the following: -

  • JS and CSS are cached locally when hitting other articles
  • Blog can be installed as a PWA(progressive web application) application on Android and iOS.

For more insight and which browsers support service workers head here.

Disqus

To allow fellow developers and consumers of my blog to comment on my articles, I needed a commenting solution. After some Googling I found enabling Disqus for Ghost is not the most taxing endeavour. There was already a section commented out in the post.hbs Handlebars template. After creating an account and amending the URL to be Hackerlite, I had enabled comments on my blog. You can find the section here.

Cloudflare

I now had my own blog up and running and I had achieved the following: -

  • Created a VM (Droplet) on Digital Ocean
  • Configured a Docker Compose setup using the official Ghost Docker
  • Sat NGINX in front of my Ghost blog and proxying the requests to it
  • Purchased the hackerlite.xyz domain
  • Installed a Let's Encrypt certificate, making my blog secure
  • Enabled HTTP/2 in NGINX

Over the last couple of years, I had become more knowledgeable about how websites utilise a CDN. I had not worked with Cloudflare before but a colleague mentioned they had used it before and they offer a free tier. This sounded very attractive. So, the upshot is: a free CDN setup with SSL, CDN experience and Cloudflare experience. It was a win win situation. To enable this all I had to do was point Go Daddy DNS to the Cloudflare name servers, then in Cloudflare point it to the IP Address of my Digital Ocean droplet. The process didn't take very long and when I hit my website again, I could see the Cloudflare certificate in place of the Let's Encrypt certificate. As the below image suggests: -

cloudflarecert

I'm not going to do a sales pitch for Cloudflare or showcase their offering to its full extent. I don't understand all the features on offer and I believe they can do a better job than I can. So, for a detailed overview of their offering I implore you to browse their website and read what they have offer and lap it up. And also, to get practical experience and have a play yourself.

cloudflare

Terraform

Where does Terraform fit in this puzzle? My blog had been sitting comfortably for almost 3 years. In that time Digital Ocean had upgraded their basic offering for a $5 droplet from 512MB to 1GB RAM. Also, in this period I had not patched my Droplet either. It was time for me to migrate to a new Droplet. Because I had SSH'd onto Droplet and installed all the relevant software by hand such as Docker, NGINX, Cert Bot etc. I had no way to way to automate the Droplet migration. So, to do this properly I was going to need to ideally lean on an IAC tool such as Puppet, Ansible, Chef, Saltstack or Terraform. If there are strong contenders, I'd be happy to hear about them.

iac

I had very briefly used Ansible at work. This was a stronger contender to reach for. But I had read and heard a lot about Terraform and it sounded impressive. As I didn't have a strong grounding in any of the IAC tools, any of them was fair game. Terraform especially jumped out at as it is very modern compared to say Chef/Puppet and also that it is being used widely in the Kubernetes community. We are using it at work also, so it was a desirable skill to attain.

The obvious next step was to head to Google in my browser and type "Terraform Digital Ocean Droplet". To my joy I was presented with a link to the Terraform documentation (which is very good) and specifically the Digital Ocean provider and the associated resources. As the Terraform language was unfamiliar to me, I had to first acquaint myself with it and grasp the following terms: Providers, Provisioners, Modules and Backends. Also, how to use the CLI.

After doing tutorials locally and learning how to create a Droplet with Terraform, I started in earnest to re-create my Droplet configuration using Terraform. To my existing Cent OS droplet, I added a floating IP address. This was to allow me to point my domain to the floating IP address when I spun up the new Droplet. I also pointed Cloudflare to this IP address too. So, with Terraform I wanted to achieve the following tasks (and I did): -

  • Create a Droplet
  • Install Digital Ocean Monitoring tools
  • Install NGINX
  • Install Docker
  • Install Cert Bot (Let's Encrypt)
  • Copy custom NGINX config
  • Clone my custom Ghost setup
  • Install Google Cloud SDK (for Cloud Storage)
  • Use Google Cloud to manage the Droplet state
  • Assign a floating IP address to the new Droplet

All the magic happens below: -


provider "digitalocean" {
  token = "${var.do_token}"
}
resource "digitalocean_droplet" "hackerlite-droplet" {
  image = "${var.digitalocean_image}"
  name = "${var.digitalocean_droplet_name}"
  region = "${var.digitalocean_droplet_region}"
  size = "${var.digitalocean_droplet_size}"
  private_networking = true
  tags = ["hackerlite", "ghost", "ubuntu"]
  ssh_keys = [
    "${var.ssh_fingerprint}"
  ]

  connection {
    user = "root"
    type = "ssh"
    private_key = "${file(var.pvt_key)}"
    timeout = "2m"
  }

  # Enable DO Monitoring
  provisioner "remote-exec" {
    inline = [
      "curl -sSL https://agent.digitalocean.com/install.sh | sh"
    ]
  }

	# Install NGINX, Docker and Certbot
  provisioner "remote-exec" {
    inline = [
      "export PATH=$PATH:/usr/bin",
      "sudo apt-get update",
    # Install NGINX
      "sudo apt install -y nginx unzip zip",
      "sudo ufw allow 'Nginx Full'",
      "sudo ufw enable -y",
    # install Docker
      "snap install docker",
      "sudo curl -L \"https://github.com/docker/compose/releases/download/1.23.2/docker-compose-$(uname -s)-$(uname -m)\" -o /usr/local/bin/docker-compose",
      "sudo chmod +x /usr/local/bin/docker-compose",
      "sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose",
      "docker-compose version",
    # Install Certbot - Let's Encrypt
      "sudo apt-get update",
      "sudo apt-get install -y -q software-properties-common",
      "sudo add-apt-repository -y universe",
      "sudo add-apt-repository -y ppa:certbot/certbot",
      "sudo apt-get update",
      "sudo apt-get install -y python-certbot-nginx"
     ]
  }

  # Copies the nginx.conf file to /etc/myapp.conf
  provisioner "file" {
    source      = "resources/nginx.conf"
    destination = "/etc/nginx/sites-available/default" // copy to different location then overwrite
  }

  # Restart NGINX
  provisioner "remote-exec" {
    inline = [
      "sudo /etc/init.d/nginx restart"
    ]
  }

  # Clone my Repo from Github
  provisioner "remote-exec" {
    inline = [
      "cd /home", 
      "git clone https://github.com/blairg/ghost-setup.git",
      "cd ghost-setup",
      "git checkout hackerlite-blog",
      "cp -r /home/ghost-setup/hackerlite /home/hackerlite"
    ]
  }

  # Gcloud Auth File
  provisioner "file" {
    source      = "gcsauth.json"
    destination = "/home/gcsauth.json" 
  }

  # Install Google Cloud SDK and copying blog data from GCS
  provisioner "remote-exec" {
    inline = [
      "cd /home",
      "mkdir mygcloud",
      "export CLOUD_SDK_REPO=\"cloud-sdk-$(lsb_release -c -s)\"",
      "echo \"deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main\" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list",
      "curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -",
      "sudo apt-get update && sudo apt-get install -y google-cloud-sdk",
      "gcloud auth activate-service-account --key-file gcsauth.json",
      "mkdir -p /home/hackerlite/data",
      "mkdir -p /home/hackerlite/sslcerts",
      "gsutil -m cp -r gs://hackerlite/v2/data /home/hackerlite/",
      "gsutil -m cp -r gs://hackerlite/v2/sslcerts /home/hackerlite/",
      "gsutil -m cp -r gs://hackerlite/v2/data/images /home/hackerlite/data/images/",
      "sudo openssl dhparam -out /home/hackerlite/sslcerts/dhparam.pem 2048",
      "sudo systemctl stop nginx",
      "cd /home/hackerlite",
      "docker-compose up -d"
    ]
  }

  # Cronjob to hourly rsync the blog data
  provisioner "remote-exec" {
    inline = [
      "cd /home",
      "gsutil cp gs://hackerlite/cronjobs.txt cronjobs.txt",
      "(crontab -l ; cat cronjobs.txt 2>&1) | grep -v \"no crontab\" | sort | uniq | crontab -"
    ]
  }
}

resource "digitalocean_floating_ip_assignment" "hackerlite-droplet" {
  ip_address = "${var.digitalocean_floating_ip}"
  droplet_id = "${digitalocean_droplet.hackerlite-droplet.id}"
}

To actually execute the above Terraform script, I had to set the following variables: -

Once I had acquired the values for the above variables, the next course action was to plan and execute my plan. It's worth mentioning at this point I used brew to install the Terraform CLI locally. If you wish to keep your Mac clean, you could opt for the Hashicorp Docker image. Windows users you are on your own :)

# Create plan
terraform plan -out=tfplan -input=false 
-var "do_token=${DO_PAT}" 
-var "pub_key=$HOME/.ssh/id_rsa.pub" 
-var "pvt_key=$HOME/.ssh/id_rsa" 
-var "ssh_fingerprint=${SSH_FINGERPRINT}" 
-var "digitalocean_floating_ip=$FLOATING_IP"

# Execute plan
terraform apply tfplan

Google Cloud

I mentioned above that I had installed the Google Cloud SDK on my droplet. What was the reason for this? Well, this was twofold, one for the Terraform Backend state and the other for managing the state of the droplet. To make use of the Google Cloud SDK, I had to have the authentication json file generated from the Google Cloud Console, to allow access to the Storage Buckets.

To set up the GCS backend, see below: -

terraform {
  backend "gcs" {
    bucket  = "hackerlite"
    prefix  = "terraform/state"
    credentials = "gcsauth.json"
  }
}

The second reason for managing state of the Droplet. On provisioning of the new Droplet, the data from the Storage Bucket is copied to the Droplet. Then there is an hourly cron job with runs a rsync with the bucket, to perform essentially a backup: -

# Copy content
gsutil -m cp -r gs://hackerlite/v2/data /home/hackerlite/
gsutil -m cp -r gs://hackerlite/v2/sslcerts /home/hackerlite/
gsutil -m cp -r gs://hackerlite/v2/data/images /home/hackerlite/data/images/


# rsync with Bucket
0 0-23 1 1-12 0-6 gsutil -m rsync -e -r /home/hackerlite/data/ gs://hackerlite/v2/data/ > rsync-data-data
0 0-23 1 1-12 0-6 gsutil -m rsync -e -r /home/hackerlite/sslcerts/ gs://hackerlite/v2/sslcerts/ > rsync-data-sslcerts
0 0-23 1 1-12 0-6 gsutil -m rsync -e -r /home/hackerlite/data/images/ gs://hackerlite/v2/data/images/ > rsync-data-images

Webp.net-compress-image--1-

Future Work

Quite a lot has been accomplished and I have a repeatable pattern to migrate my Droplet and also includes backing up of the state. But there are a few things I'd still like to do: -

  • Automate the renewal of SSL Certificates
  • Have a load balancer with more than 1 Droplet
  • Perform blue/green zero downtime deployments
  • Utilise some paid features of Cloudflare
  • Post droplet notification such as a Slack hook
  • CI to test the Terraform plan
  • CD to execute the Terraform plan

If this is of use to anyone and you've made it this far, well done! If I have made wild assumptions or any of this is not clear or difficult to follow let me know and I'll tidy it up. I welcome any comments.

Repo can be found here https://github.com/blairg/ghost-setup

phew_arnie