Running TestCafe tests in Docker, locally and in Jenkins

While you can run TestCafe tests on any machine that has Node.js installed, it is more convenient to run them in a Docker container. This has the advantage when running the tests you don’t need to think about any dependencies, Node.js version etc. It also makes tests less flaky, because the environment the tests run in is always the same, either locally or in the CI environment. This is even more important when using TestCafe for visual regression testing, because then the screenshots will look the same wherever you run the tests.

Setting up

  • Recent version of Docker.
  • Node.js on your local development machine so we can test our code before creating the Docker image.
  • Checkout my example git repository. It contains a TestCafe framework and tests. You can find it here:

Creating the Docker image

We will use volume mounts when running the tests locally in Docker. In CI environment we will use git to pull our testcode in our Docker container.
This keeps our image lean and evergreen.

Most other guides use the testcafe/testcafe Docker image as a base image. This image is created by the TestCafe team. While I was experimenting with Dockerizing my tests I found this Docker image to be limiting. It has only the testcafe npm package pre-installed. My test frameworks usually depend on more packages than only the testcafe package, so the image is too barebones for me. Also it expects you run your test with the testcafe command, so if you like me use custom runners written in Typescript it won’t work out of the box.

After some searching I found the circleci/node:browsers image. This image contains Node.js, Java, all the browsers we need and is regularly updated. The only thing that isn’t available out of the box is TestCafe itself. But we can fix this easily by installing it and all other npm dependencies using the yarn install command:

What this Dockerfile does:

  • It copies the package.json from our project into the Docker image in the runner directory. There it installs the dependencies using yarn. This results that all our needed dependencies (including testcafe) are installed inside the Docker image.
  • Finally it sets the WORKDIR to a subdirectory called framework. When we want to run our tests we will mount our testcode inside that directory. Why we do this is that we don’t want to mount our local node_modules directory inside the Docker container, because of OS differences this will lead to issues. And to prevent overriding the already installed node_modules, we mount the test code in a subdirectory. When Node.js cannot find the required dependencies in the node_modules folder in the current directory it will look into the node_modules folder in the parent directory etc.

Build the Docker image usingyarn docker:build
Or run docker build . -t testcafe-runner .

Running the tests locally

  • The testframework and tests.
  • The report directory so when the tests are finished the reports are saved on our local machine and not in the Docker container that gets deleted.
  • We don’t want to mount our local node_modules folder inside the Docker container, so we create an empty volume for this directory.

We can achieve this by running:

docker run -it --rm \
-v ${PWD}:/runner/framework \
-v ${PWD}/reports/allure:/runner/allure \
-v /runner/framework/node_modules \
testcafe-runner yarn test:ci

This command will spin up a container of type testcafe-runner, remove it after it is done running, mounts the needed volumes inside the container and will execute the yarn test:ci command inside the container.

This can also be executed using the yarn docker:run:test command.

After the tests have run you can check the Allure test report using the
yarn test:report:open command.

Running the tests in Jenkins (or any other CI env)

To try this out locally you need a Jenkins instance with Docker support. To set this up easily I created a Docker compose file that sets up a Jenkins container with Docker in Docker support:

Run with docker compose up -d .

The Jenkins container it creates has no security features whatsoever, so only use for local testing!

Now navigate to http://localhost:8080/ and click on Create a job .

Create a Pipeline job:

And use the following pipeline script:

Run the pipeline. It will now:

  • Retrieve the Docker image and start the container
  • Checkout the code using git inside the container
  • Install the npm dependencies using Yarn
  • Run the TestCafe tests
  • Create and archive the Allure report

Afterwards you can view the report using the Test results link in the menu:

This will open the Allure HTML report, with screenshots for failed tests:

Conclusion

Test Automation Enthusiast

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store