CI / CD Pipeline with Github Actions

Overview

info

This guide uses the github-actions-deployment-pipeline example   from our public repo.

The setup assumes you have a Github monorepo that holds all CDK connectors built by your organization.

Even if you don't use Github Actions in your CI / CD setup, this guide explains basic ideas you need to configure your CI / CD tool (e.g. Jenkins / CircleCI etc.)

You can configure a CI / CD pipeline using Github actions to manage deployments of connectors built through CDK.

Once you configure the workflow, it will automatically detect changes to connectors and deploy the changed connector to multiple regions, ensuring a streamlined and automated release process.

Copy the .github folder from the examples repo to your CDK connectors Github repo.

The folder contains:

  • actions - Contains two actions:
    • To deploy
    • To filter out deleted connectors from the repo and fail the pipeline if more than 1 connector has changed
  • scripts - A shell script to poll deployment status
  • workflows - Contains the Deployment workflow which uses the actions and scripts to get the job done

Workflow explanation

Here’s a breakdown of the Workflow file:

Workflow Overview

  • Triggers:
    • Push event: Runs when there’s a push to the main branch.
    • Manual dispatch: Allows manually triggering the workflow using the workflow_dispatch event, which requires input for the git_sha (commit SHA) of the code to publish.
  • Environment Variables:
    • FILES_IGNORE : Specifies directories that should be ignored when detecting changes (i.e., .github/ and terraform/ ).

Jobs

  1. detect-changes - This job checks for any changes in the repository that affect connectors:
    • Runs on: ubuntu-latest
    • Outputs:
      • CONNECTOR_DIR_NAME : The directory name of the changed connector.
      • CONTAINS_CONNECTOR_CHANGES : A flag to indicate whether connector-related changes were detected.
    • Steps:
      1. Checkout the repository: Checks out the code from the repository using the commit SHA provided as input ( git_sha ) or the latest commit on main.
      2. Get all changed files: Uses the tj-actions/changed-files action to get a list of all changed files, including directory names, while ignoring the directories specified in FILES_IGNORE.
      3. Filter out deleted connectors: This custom action processes the list of changed files to identify connector-related changes and filters out deleted connectors.
  2. deploy - This job deploys the connector if connector changes are detected:
    • Conditional Execution: It only runs if the CONTAINS_CONNECTOR_CHANGES output from the detect-changes job is true .
    • Matrix Strategy:
      • Defines a deployment strategy across multiple regions: US , EU , and APAC .
      • Each region has specific API configurations (e.g., API_BASE_URL and TRAY_API_KEY ).
      • The matrix allows parallel execution of the deploy process for each region, with a maximum of 3 parallel jobs.
    info

    You can edit the region array to target deployments in the regions you use.

    • Steps:
      1. Checkout the repository: Checks out the repository again to ensure the code is available for deployment.
      2. Deploy connector: Uses a custom action ( .github/actions/deploy ) to deploy the connector. It passes:
        • CONNECTOR_DIR_NAME : The directory name of the connector that changed.
        • API_BASE_URL : The region-specific API URL.
        • TRAY_API_KEY : The region-specific API key, stored as a secret in the repository.

Key Concepts

  • Matrix Deployment: Deploys the connector in parallel to different regions ( US , EU , APAC ) with specific environment variables for each.
  • Conditional Job Execution: The deploy job only runs if the repository contains connector-related changes, as determined by the detect-changes job.
  • Custom Actions: The workflow uses two custom actions ( filter-out-deleted-connectors and deploy ). These handle filtering changes and the actual deployment, respectively.