Written by Sam McGeown on 9/1/2020 · Read in about 4 min (828 words)
Published under Automation, Community and Development

I’ve posted previously about moving to Hugo as a publishing platform for this blog, this post is a bit more about how I’m managing the publishing using GitLab’s CI/CD Pipelines.

Firstly, I need to mention that I’m using three different repositories for my code base, and why. The three repositories are:

  • definit-hugo - this contains the hugo site configuration
  • definit-content - this contains the site content - markdown files, images etc
  • definit-theme - this contains the VMware Clarity-based theme I use for my site

definit-content and definit-theme are git submodules in the definit-hugo project, mapped into the /content and /themes folders respectively. This allows me to keep the configuration, content and theme separate, and to manage them as separate entities. The aim is that the theme will eventually be in a position to be released, and I don’t want to have to extract it from my hugo code base later on.

Using submodules in this way throws up a couple of challenges when I’m triggering a build pipeline, which I’ll get into later on.

At a high level, I want my publishing process to be something like this:

  1. Write a post in markdown
  2. Commit and push to the definit-content repo
  3. Build the staging site with draft and posts to be published in the future
  4. Push the staging site to AWS S3
  5. Build the production site
  6. Push the production site to AWS S3
  7. Invalidate the AWS CloudFront cache

The reason for pushing drafts and future posts to staging is to provide a bit of a preview without needing to have hugo running locally. I tend to use hugo server while I’m writing to preview changes, but others (cough Simon cough) don’t necessarily have the same setup or knowledge as me - and that’s fair enough, he knows plenty of stuff that I don’t, that’s why he’s a great co-author on this blog.

Configuring the Hugo Build Pipeline in GitLab

GitLab CI/CD pipelines are configured using a .gitlab-ci.yml file in the root of the repository. This file is used to describe the various stages of the build, and is automatically triggered when changes are committed to the repository. Although it’s just a YAML file, I found the syntax to be a little daunting at first, but fortunately there’s an extensive Pipeline Configuration Reference and plenty of templates to get you started. I started with the Hugo template provided.

Setting up the GitLab CI/CD Pipeline

In the definit-hugo repository, I have created some variables (Settings > CI/CD > Variables) to hold the more sensitive data:

Key Value
AWS_DEFAULT_REGION The AWS region my S3 bucket resides
AWS_S3_Staging_Bucket The name of the Staging site S3 bucket
AWS_S3_Production_Bucket The name of the Production site S3 bucket

These variables are used in the .gitlab-ci.yml file below, which I’ve commented heavily to show what’s happening.

 2# Set the default container image to GitLab's Hugo image
 6  # Ensure the git submodules are updated when the pipeline runs
10  - build
11  - deploy
13# Build the Staging site
15  stage: build
16  script:
17    # Run hugo with a different baseURL to ensure links work in the staging site, and flag to include drafts and future posts
18    - hugo --minify --baseURL http://$AWS_S3_Staging_Bucket.s3-website.$ --buildDrafts --buildFuture
19  artifacts:
20    paths:
21      # Create an artifact folder of the hugo "public" output
22      - public
24# Deploy the Staging site
26  # In the deploy stage
27  stage: deploy
28  # Use a docker image with AWS CLI pre-installed
29  image: garland/aws-cli-docker
30  script:
31  # Synchronise the "public" folder passed from buildStaging to the Staging AWS S3 bucket name from the variable
32  - aws s3 sync ./public s3://$AWS_S3_Staging_Bucket --delete --only-show-errors
33  dependencies:
34  # Depend on the buildStaging job to ensure the "public" artifact is available
35  - buildStaging
37# Build the Production site
39  # In the build stage
40  stage: build
41  script:
42    # Run hugo with just the minify command (no drafts or future posts)
43    - hugo --minify
44  artifacts:
45    paths:
46      # Create an artifact folder of the hugo "public" output
47      - public
49# Deploy the Production site
51  # In the deploy stage
52  stage: deploy
53  # Use a docker image with AWS CLI pre-installed
54  image: garland/aws-cli-docker
55  script:
56  # Synchronise the "public" folder passed from buildProduction to the Production AWS S3 bucket name from the variable
57  - aws s3 sync ./public s3://$AWS_S3_Production_Bucket --delete --only-show-errors
58  # Invalidate the cloudfront cache to make the changes live
59  - aws cloudfront create-invalidation --distribution-id E37FU6DHWYT5T2 --paths "/*"
61  dependencies:
62  # Depend on the buildProduction job to ensure the "public" artifact is available
63  - buildProduction

Commiting the .gitlab-ci.yml file to the definit-hugo repository actually triggers the pipeline.

Pipeline build and deploy jobs

When the draft: true property is set in the hugo content’s front matter, the draft content is built and released to the staging environment, but not the “live” blog thanks to the --buildDrafts flag:

Draft content released to staging

All good so far, but there’s a problem here. I want to commit new posts to the definit-content repository, not the definit-hugo repository, so I need to create a pipeline on the definit-content repository to trigger the pipeline in the definit-hugo project. Fortunately GitLab makes this easy using the trigger syntax, simply creating a job with the trigger action and the path to my definit-hugo project.

2  stage: deploy
3  trigger: sammcgeown/definit-hugo

Now when there’s a commit to the definit-content projcet the pipeline triggers the downstream build, and all the downstream steps are visible in the pipeline:

Trigger downstream pipeline build and deploy jobs

Share this post