Basic GitLab CI pipeline for deploying to Google Cloud Run

Tired of GCP ClickOps? ​

Are you tired of ClickOps to deploy your app to Google Cloud Run?
Or even worse: already ran into issues managing the deployment of your service by clicking through graphical user interfaces? Yet not convinced a heavy-weight infrastructure-as-code product like Terraform 😱 is worth it?

Let's explore the middle-ground in between where you get repeatable, solid automation, while not over-complicating things. I'll show you how to deploy your app to Google Cloud Run Services using a simple CI/CD workflow, cutting down on risk and manual, repetitive steps.

What we'll do ​

We'll set up GitLab and Google Cloud and configure a CI/CD pipeline on GitLab CI to build and deploy your web app to Google Cloud Run Services. We will sue GitLab's container registry to store Docker images, and make Cloud Run pull images from it for deployment.
Without ClickOps and without any manual steps needed to get your app deployed every time changes are pushed to your repository.

As an added benefit you'll get review apps for your merge requests, which allow you, developers and quality engineers to explore and test new features even before a merge request gets merged and deployed to production. Sweet!

What you'll need ​

• A web app you want to deploy—Google Cloud Run Services are suitable for any app that takes HTTP requests. I'll use a demo app built with Ruby on Rails.
• A Dockerfile for building the image containing your app for deployment. Rails comes with a production-ready Dockerfile out-of-the-box.
• A GitLab repository with your app's source code—let me know if you are not using GitLab and would like to get this working with another CI/CD provider like GitHub Actions.
• A Google Cloud account and project. If you don't have one set up already, go to https://cloud.google.com and set up your account and project. They offer free 300$ credit, allowing you to explore without spending money right away.
  Don't hesitate to reach out if you find yourself struggling, or if you don't know if Google Cloud will be a good fit for your project.

Initial Cloud Setup ​

Before we dive into configuring the CI/CD pipeline, we need to make sure your Google Cloud Project and your GitLab repository are hooked up. Luckily GitLab comes with native Google Cloud support, which takes care of most of the heavy lifting and allows you to get started quickly. It involves some ClickOps, but I promise it's just a one-time process. Once that's done, we'll dive into automation.

Feel free to skip over the steps you already set up and have working.

Authorize GitLab, create Service Accounts ​

GitLab will need a Service Account with Google Cloud for deploying the app to Google Cloud Run. Let's have GitLab create what it needs:

1. Open your Repository on GitLab.
2. In the Project's menu, go to "Operate" / "Google Cloud".
3. Click "Create service account".
   This will prompt you to log into your Google Cloud account and to grant GitLab access. Follow the instructions, but make sure you understand what's going on. This will allow GitLab to access your Google Cloud account on your behalf.
4. Back in the GitLab flow, select the Google Cloud project you want to use, and as "Refs" select "All". Tick the checkbox that you understand what managing service account keys entails. Feel free to hit me up if you don't.
   
5. Finally click "Create service account".
6. You should see the message "Service account generated successfully".

What just happened? ​

• GitLab is now allowed to talk to your Google Cloud Project.
• GitLab created a Service Account in your Google Cloud Project that will be used to deploy your app to Google Cloud Run.
• GitLab fetched a Service Account Key for this Service Account and stored it in your Repo's CI/CD variables, which are available to CI/CD jobs. We'll need those in a minute.
  • You can find the variables in your GitLab Repo under "Settings" / "CI/CD" / "Variables".
  • It added GCP_PROJECT_ID, GCP_SERVICE_ACCOUNTand GCP_SERVICE_ACCOUNT_KEY.

With this all set up, we are (almost) able to automate your app's deployment to Google Cloud Run. We are this close to no more clicking, bear with me!

Enable Google Cloud to pull from GitLab registry ​

We still need to run through a few one-off steps to set things up with Google Cloud.

Authorize GCP to use GitLab registry ​

GitLab is hosting the source code of the application, and it will deal with building my app's Docker image, so I also want to use GitLab's container registry to store the built images, and have Google Cloud pull the images from there for deployment.

In order to enable Google Cloud to access GitLab container registry, we need to, well grant it access. This is done by configuring the GitLab container registry with Google Cloud Artifacts registry, using a GitLab Deploy Token for authentication.

Create a GitLab Deploy Token ​

1. Open your Repository on GitLab.
2. In the Project's menu, go to "Settings" / "Repository" and open "Deploy tokens".
3. Click "Add token".
4. Give it a telling name. I'll use GitLab GCP Deploy Token so I remember what this is used for.
5. Check the read_registry scope.
6. Hit "Create deploy token".
7. Copy the generated username and token value.

Configure Google Cloud Artifacts Registry ​

For the rest of the setup we'll need the gcloud CLI tool, authenticated to your Google Cloud account with sufficient permissions to create secrets and configure Artifacts Registry.

The quickest, easiest way to run the gcloud CLI tool is in a "Cloud Shell", in your browser, right on console.cloud.google.com.

Store the deploy token in a Google Cloud Secret Manager secret

printf "gldt-TOKEN-VALUE-YOU-COPIED-BEFORE" | \
  gcloud secrets create gitlab-gcp-deploy-token --data-file=-

You can double check that the secret was stored properly by retrieving its value with the following command:

gcloud secrets versions access latest --secret=gitlab-gcp-deploy-token

Configure Google Artifacts Registry for GitLab

gcloud artifacts repositories create gitlab \ 
  --location=europe-west1 \ 
  --repository-format=docker --mode=remote-repository \ 
  --remote-docker-repo=https://registry.gitlab.com \ 
  --remote-username=gitlab+deploy-token-USERNAME-YOU-COPIED-BEFORE \ 
  --remote-password-secret-version=projects/$(gcloud config get project)/secrets/gitlab-gcp-deploy-token/versions/latest

Finally, allow Artifacts Registry to access the secret
By default, the built-in Service Account for Artifacts Registry is not allowed to access secrets. If that's the case with your project, the above command creating the GitLab repository will mention that alongside the command necessary to grant Artifacts Registry access to secrets. Make sure to run that to wrap up the setup. It will look something like this:

gcloud projects add-iam-policy-binding [YOUR-PROJECT-ID] --member='serviceAccount:service-1234567890@gcp-sa-artifactregistry.iam.gserviceaccount.com' --role='roles/secretmanager.secretAccessor'

Create a GitLab CI pipeline ​

Now to the fun part of automating deployments. GitLab CI is configured through a .gitlab-ci.yml file you put into the root of your GitLab repository. Read more about how GitLab CI works here, and feel free to drop me a message if you need help.

CI built Docker image ​

GitLab and also independent contributors offer templates and components you can use in your GitLab CI pipeline. Those are a great starting point to get your pipeline up and running quickly. We'll use a template to build the app's Docker image.
If you haven't already, create a .gitlab-ci.yml file:

stages: # List of stages for jobs, and their order of execution
  - build
  - test
  - deploy
    
# GitLab-provided template for building Docker images
include:
  - remote: https://gitlab.com/gitlab-org/gitlab/-/raw/master/lib/gitlab/ci/templates/Docker.gitlab-ci.yml

Once you commit this file and push it to your GitLab repository, you should have a CI pipeline up and running, building the app's Docker image 🚀.
Find your pipeline runs in GitLab's Project menu under "Build" / "Pipelines".

Deploy review apps to Cloud Run ​

Great, we've got a Docker image that is built every time we have a new feature branch, now let's deploy it!

GitLab comes with this great feature called Review Apps. It's based on dynamic environments, which just means that you don't have one single, fixed staging or sandbox environment, but rather a dynamic, new environment created for each Merge Request.
In combination with a cloud platform like Google Cloud Run, this allows us to deploy each Merge Request individually, accessible through its own unique, one-off URL.

Deploying a Review App for Merge Requests is done by an additional step added to the gitlab-ci.yml file we created earlier. This step is going to grab the Service Account key needed to authenticate to the Google Cloud project from the CI/CD variables GitLab added to the project during setup.
The step uses the Google provided cloud-sdk Docker image that comes with the gcloud CLI tool. After the app was deployed, we get the newly created Cloud Run service's URL, and configure that as the Review App environment's url.

deploy-review-app:
  stage: deploy
  image: google/cloud-sdk:latest
  before_script:
    # Grab the Google Cloud Service Account key from GitLab provided variable,
    # and authenticate to Google Cloud:
    - apt update
    - apt install --yes --quiet jq
    - echo $GCP_SERVICE_ACCOUNT_KEY | jq --raw-output '.private_key_data' > gcp-key.json
    - gcloud auth activate-service-account --key-file gcp-key.json
  variables:
    SERVICE_NAME: $CI_PROJECT_NAME-$CI_COMMIT_REF_SLUG
    GCP_GITLAB_DOCKER_IMAGE: "$GCP_REGION-docker.pkg.dev/$GCP_PROJECT_ID/gitlab/$CI_PROJECT_PATH:$CI_COMMIT_REF_SLUG"
    # This step doesn't need the app sources, skip git fetch:
    GIT_STRATEGY: empty
  script:
    - |
      gcloud run deploy "$SERVICE_NAME" \
        --project="$GCP_PROJECT_ID" \
        --region="$GCP_REGION" \
        --allow-unauthenticated \
        --port 80 \
        --startup-probe=httpGet.path=/up \
        --liveness-probe=httpGet.path=/up \
        --image="$GCP_GITLAB_DOCKER_IMAGE"
    - echo -n "REVIEW_APP_URL=" > deploy.env
    - |
      gcloud run services describe "$SERVICE_NAME" \
        --region="$GCP_REGION" \
        --project="$GCP_PROJECT_ID" \
        --format='value(status.url)' >> deploy.env
  artifacts:
    reports:
      dotenv: deploy.env
  environment:
    name: review/$CI_COMMIT_REF_NAME
    url: $REVIEW_APP_URL
    on_stop: stop-review-app
    auto_stop_in: 1 hour
  rules:
    - if: $CI_COMMIT_BRANCH && $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH

With this we are set for deploying an instance of our app for each Merge Request. It will create a new Cloud Run service for each Merge Request, and GitLab will surface the link to the Review App instance on the Merge Request page. Just click View app to open up a new browser window with the instance that got deployed:

The only thing left to do is to add a step for stopping and removing the Cloud Run service instance once the MR was closed. I also want to have this ephemeral instances be shut down after a given period time, so they do not stick around and hog resources for longer than needed.
You might have noticed the auto_stop_in config key above. That's telling GitLab to stop the Review App after the given amount of time passed. If you need the Review App once it got shut down, GitLab let's you spin it up again later from the Merge Request page.

stop-review-app:
  stage: deploy
  image: google/cloud-sdk:latest
  before_script:
    - apt update
    - apt install --yes --quiet jq
    - echo $GCP_SERVICE_ACCOUNT_KEY | jq --raw-output '.private_key_data' > gcp-key.json
    - gcloud auth activate-service-account --key-file gcp-key.json
  variables:
    SERVICE_NAME: $CI_PROJECT_NAME-$CI_COMMIT_REF_SLUG
    # This step doesn't need the app sources, skip git fetch:
    GIT_STRATEGY: empty
  script:
    - gcloud run services delete "$SERVICE_NAME" --region="$GCP_REGION" --project="$GCP_PROJECT_ID" --quiet
  environment:
    name: review/$CI_COMMIT_REF_NAME
    action: stop
  rules:
    - if: $CI_COMMIT_BRANCH && $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
      when: manual
      allow_failure: true

At this point, when creating a new Merge Request, you should have GitLab CI:

• Build a Docker image from your feature branch.
• Deploy the app to an ephemeral Cloud Run Service instance.
• Put a link to the Cloud Run Service for your feature branch on the Merge Request page.
• Remove the Cloud Run Service instance after 1 hour or when the MR is merged.

Let me know what you think about Review Apps? Are they worth it? Do they help with QA and visibility into development of new features?

Deploy to Production ​

Finally, let's deploy to Production each time changes are pushed to the main (default) branch. This is very similar to how Review Apps are deployed. The main differences are the trigger for the pipeline step and the Cloud Run Service's name.
We want this step to be triggered when commits are pushed to the main branch. And instead of a random name, we need a stable name and URL for our production Cloud Run Service. Let's add the config needed for the deployment step to gitlab-ci.yml to wrap things up:

deploy-production:
  stage: deploy
  image: google/cloud-sdk:latest
  before_script:
    # Grab the Google Cloud Service Account key from GitLab provided variable,
    # and authenticate to Google Cloud:
    - apt update
    - apt install --yes --quiet jq
    - echo $GCP_SERVICE_ACCOUNT_KEY | jq --raw-output '.private_key_data' > gcp-key.json
    - gcloud auth activate-service-account --key-file gcp-key.json
  variables:
    # Stable service name
    SERVICE_NAME: $CI_PROJECT_NAME-production
    # Using the `latest` image
    GCP_GITLAB_DOCKER_IMAGE: "$GCP_REGION-docker.pkg.dev/$GCP_PROJECT_ID/gitlab/$CI_PROJECT_PATH:latest"
    # This step doesn't need the app sources, skip git fetch:
    GIT_STRATEGY: empty
  script:
    - |
      gcloud run deploy "$SERVICE_NAME" \
        --project="$GCP_PROJECT_ID" \
        --region="$GCP_REGION" \
        --allow-unauthenticated \
        --port 80 \
        --startup-probe=httpGet.path=/up \
        --liveness-probe=httpGet.path=/up \
        --image="$GCP_GITLAB_DOCKER_IMAGE"
    # Optional: we can still grap the Service's URL
    - echo -n "PRODUCTION_APP_URL=" > deploy.env
    - |
      gcloud run services describe "$SERVICE_NAME" \
        --region="$GCP_REGION" \
        --project="$GCP_PROJECT_ID" \
        --format='value(status.url)' >> deploy.env
  artifacts:
    reports:
      dotenv: deploy.env
  environment:
    # Optional: tell GitLab about production environment
    name: production
    url: $PRODUCTION_APP_URL
  rules:
    # only on main branch
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Once this is committed, pushed and merge to main, you should now have production deploys automatically going every time changes are pushed to your main branch. 🥳

Final thoughts ​

Whew, that was quite a long one, but we accomplished a lot and got our feet wet with:

• GitLab CI/CD and
  • its native Google Cloud integration
  • using a template for building Docker images
  • Review Apps for every Merge Request
  • automated deploys to Google Cloud Run
• Google Cloud and
  • setting up GitLab with Artifacts Registry
  • Secret Manager and creating secrets
  • Cloud Run Service deployments

What's next?
There's a ton of things we could add to our new pipeline, like running linters and tests. We could even run tests on the actual Docker image we built for our app already. Or we could look into using the Review App we deploy for each MR for browser testing (think Cypress) or even performance testing to catch regressions early on. What do you think, what's something you want to explore or fix with your CI/CD pipelines? Let me know!