Running Integration Tests like Cypress in AWS with Coherence

Browser based integration testing with a framework like Cypress.io or Selenium is an important part of many teams’ testing pipeline.  These tests are rarely run by engineers on their own laptops - they can require quite a bit of machine resources, take a long time to run, and require a lot of setup and maintenance, so they are more often run in dedicated environments outside of the typical developer’s workflow.  Configuring these tests to run in dedicated cloud pipelines can also take a lot of trial and error.

Fortunately, Coherence provides an easy interface for incorporating your integration tests into your build pipeline that can be run on any commit to any branch.  Your build pipeline is run in your cloud so you can watch execution and inspect test results.  Adding integration tests in your Coherence flow is as easy as adding two things: a Docker image with your test container and a command to start that container.  With that, we will run your tests for you against a deployed version of your app, and here’s how we do it.

All we require from a Coherence user is a Docker image for your test runner and the command to be passed to that image.  That can be a custom image that is built and stored in an accessible registry, or just a link to an image like the ones provided by Cypress on AWS Elastic Container Registry.  To run Cypress tests in your build pipeline, all we require is this in your coherence.yml file:

‍integration_test:
  type: integration_test
  command: [“npx”, "cypress", "run"]
  image: "cypress/included:10.9.1"

AWS

‍

On AWS, we run your application’s build using AWS CodePipeline.  Integration tests are run as a CodeBuild Pipeline stage, with your source code as a CodePipeline Input Artifact.  In Terraform, here’s how we set up a new CodeBuild Project:

resource "aws_codebuild_project" "integration-tests-codebuild" {
    name = "integration-tests-codebuild"
    service_role = your:codebuild:arn
    artifacts {
        type = "CODEPIPELINE"
    }

    environment {
        compute_type = "BUILD_GENERAL1_SMALL"
        image = “public.ecr.aws/cypress-io/cypress/included:10.6.0”
        type = "LINUX_CONTAINER"

        environment_variable {
            name = "COHERENCE_BASE_URL"
            value = // we inject the url of your preview environment here for testing
        }
    }

    build_batch_config {
        service_role = "${aws_iam_role.codebuild_role.arn}"
    }
    
    source {
        type = "CODEPIPELINE"
    }
}

‍

Then we add that CodeBuild Project as a stage in our terraform creating the CodePipeline for the project:

resource "aws_codepipeline" "your_pipeline" {
    name = "your-app-name-pipeline"
    role_arn = "your:codepipeline:arn"

    artifact_store {
        location = "some S3 bucket"
        type = "S3"
    }

    stage {
        name = “run_integration_tests"
        action {
            name = "run_integration_tests"
            category = "Test"
            owner = "AWS"
            provider = "CodeBuild"
            input_artifacts = ["source"]
            version = "1"
            configuration = {
                ProjectName = "integration-tests-codebuild"
            }
        }
    }
}

‍

And finally we add a Buildspec .yml file for the integration test CodeBuild project:

version: 0.2

phases:
  build:
    commands:
      - echo in build commands for integration tests..
      - (your command here e.g. cypress run –record)

‍

After running terraform to create the pipeline, add the buildspec to your project using the update_project method on the boto3 CodeBuild client:

with open(
          f"{self.generated_config_dir}/{rendered_template_path}.yml",
          "r"
      ) as f:
         buildspec_contents = yaml.safe_load(f)

codebuild_client.update_project(
            name=integration-tests-codebuild,
            source={
                "type": "CODEPIPELINE",
                "buildspec": buildspec_contents
            }
        )

‍

We populate your image into the terraform and the command you provide into the buildspec.yml, and submit it to AWS as part of the larger CodePipeline run.  In this case, we assume that your cypress configuration file (e.g. cypress.config.js) is at the top of level of your project files, although the location of that file can be configured on the command line.  

Because Coherence comes with built in preview environments, there is no need to start an instance of your application just for the integration test run.  Your tests can run against a fully resourced version of your application that has already been deployed after the last commit on your branch.  All we require is a docker image and the command you want us to run on it.

‍