Using GitLab Pipelines to deploy Hugo sites to AWS

Written by Sam McGeown

Published on 9/1/2020 - Read in about 5 min (963 words)

Published under Community #Automation #blogging #hugo #gitlab #devops

I’ve posted previously about moving to Hugo as a publishing platform for this blog, this post is a bit more about how I’m managing the publishing using GitLab’s CI/CD Pipelines.

Firstly, I need to mention that I’m using three different repositories for my code base, and why. The three repositories are:

• definit-hugo - this contains the hugo site configuration
• definit-content - this contains the site content - markdown files, images etc
• definit-theme - this contains the VMware Clarity-based theme I use for my site

definit-content and definit-theme are git submodules in the definit-hugo project, mapped into the /content and /themes folders respectively. This allows me to keep the configuration, content and theme separate, and to manage them as separate entities. The aim is that the theme will eventually be in a position to be released, and I don’t want to have to extract it from my hugo code base later on.

Using submodules in this way throws up a couple of challenges when I’m triggering a build pipeline, which I’ll get into later on.

At a high level, I want my publishing process to be something like this:

1. Write a post in markdown
2. Commit and push to the definit-content repo
3. Build the staging site with draft and posts to be published in the future
4. Push the staging site to AWS S3
5. Build the production site
6. Push the production site to AWS S3
7. Invalidate the AWS CloudFront cache

The reason for pushing drafts and future posts to staging is to provide a bit of a preview without needing to have hugo running locally. I tend to use hugo server while I’m writing to preview changes, but others (cough Simon cough) don’t necessarily have the same setup or knowledge as me - and that’s fair enough, he knows plenty of stuff that I don’t, that’s why he’s a great co-author on this blog.

Configuring the Hugo Build Pipeline in GitLab

GitLab CI/CD pipelines are configured using a .gitlab-ci.yml file in the root of the repository. This file is used to describe the various stages of the build, and is automatically triggered when changes are committed to the repository. Although it’s just a YAML file, I found the syntax to be a little daunting at first, but fortunately there’s an extensive Pipeline Configuration Reference and plenty of templates to get you started. I started with the Hugo template provided.

In the definit-hugo repository, I have created some variables (Settings > CI/CD > Variables) to hold the more sensitive data:

These variables are used in the .gitlab-ci.yml file below, which I’ve commented heavily to show what’s happening.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

---
# Set the default container image to GitLab's Hugo image
image: registry.gitlab.com/pages/hugo:latest

variables:
  # Ensure the git submodules are updated when the pipeline runs
  GIT_SUBMODULE_STRATEGY: recursive
  
stages:
  - build
  - deploy

# Build the Staging site
buildStaging:
  stage: build
  script:
    # Run hugo with a different baseURL to ensure links work in the staging site, and flag to include drafts and future posts
    - hugo --minify --baseURL http://$AWS_S3_Staging_Bucket.s3-website.$AWS_DEFAULT_REGION.amazonaws.com/ --buildDrafts --buildFuture
  artifacts:
    paths:
      # Create an artifact folder of the hugo "public" output
      - public

# Deploy the Staging site
deployStaging:
  # In the deploy stage
  stage: deploy
  # Use a docker image with AWS CLI pre-installed
  image: garland/aws-cli-docker
  script:
  # Synchronise the "public" folder passed from buildStaging to the Staging AWS S3 bucket name from the variable
  - aws s3 sync ./public s3://$AWS_S3_Staging_Bucket --delete --only-show-errors
  dependencies:
  # Depend on the buildStaging job to ensure the "public" artifact is available
  - buildStaging

# Build the Production site
buildProduction:
  # In the build stage
  stage: build
  script:
    # Run hugo with just the minify command (no drafts or future posts)
    - hugo --minify
  artifacts:
    paths:
      # Create an artifact folder of the hugo "public" output
      - public

# Deploy the Production site
deployProduction:
  # In the deploy stage
  stage: deploy
  # Use a docker image with AWS CLI pre-installed
  image: garland/aws-cli-docker
  script:
  # Synchronise the "public" folder passed from buildProduction to the Production AWS S3 bucket name from the variable
  - aws s3 sync ./public s3://$AWS_S3_Production_Bucket --delete --only-show-errors
  # Invalidate the cloudfront cache to make the changes live
  - aws cloudfront create-invalidation --distribution-id E37FU6DHWYT5T2 --paths "/*"

  dependencies:
  # Depend on the buildProduction job to ensure the "public" artifact is available
  - buildProduction

Commiting the .gitlab-ci.yml file to the definit-hugo repository actually triggers the pipeline.

When the draft: true property is set in the hugo content’s front matter, the draft content is built and released to the staging environment, but not the “live” blog thanks to the --buildDrafts flag:

All good so far, but there’s a problem here. I want to commit new posts to the definit-content repository, not the definit-hugo repository, so I need to create a pipeline on the definit-content repository to trigger the pipeline in the definit-hugo project. Fortunately GitLab makes this easy using the trigger syntax, simply creating a job with the trigger action and the path to my definit-hugo project.

1
2
3

trigger_hugo_build:
  stage: deploy
  trigger: sammcgeown/definit-hugo

Now when there’s a commit to the definit-content projcet the pipeline triggers the downstream build, and all the downstream steps are visible in the pipeline:

Share this post