PHPUnit

Series: Drupal Docker

tl;dr: Drupal offers a few types of phpunit regression testing suites, and those tests can also be run in CI/CD and in local development.

Series Description: This series details my development environment and CI/CD deployment pipeline. The first post in the series provides an overview and the code for the entire demonstration can be found in my GitLab. You can navigate other posts in the series using the list of links in the sidebar.

As with the previous couple posts in this series, this one is about testing. When you're maintaining a site over an extended period of time, something extremely useful is to have automated regression testing that will catch if you break one thing in the process of fixing another thing. This is regression testing, and with Drupal, that means phpunit.

I've also heard the suggestion that it is best to write the tests before you even write the live code. That way, as you work on the code, you can have the automated tests confirming what you have solved and what you haven't. I have not gotten very close to that point yet, and there are probably some scenarios where it isn't practical because I don't know the exact outcome I want until I start seeing it in practice. It does help demonstrate the real ideal, though, with constant verification that everything is still doing what it should before it gets merged and eventually sent to production.

Types of Tests in Drupal

In Drupal, there are four built-in test suites, in escalating complexity where the faster ones are also the least flexible:

1. Unit tests: these are bare bones PHP. If the function I'm testing is very simple, this might be adequate and would be very fast.
2. Kernel tests: this adds some core Drupal functionality, making it much easier to invoke services or install other modules, and still run almost as fast as plain unit tests.
3. Functional tests: this adds the ability to walk through a workflow as a mock user and confirm certain behaviours. This is great for checking the end result, like testing that permissions are having the effect I expect, or that a view is displaying content the way I thought it would. They run much slower, but are appropriate in a lot of scenarios.
4. Functional with JavaScript tests: this is similar to the above, but with JavaScript, so I want these when I need to confirm interactivity is still functioning as expected.

I have also installed Drupal Test Traits which is not included in core, but haven't gotten deep into understanding it yet.

Running Tests Locally

I already had an excellent general PHP extension installed called DEVSENSE.phptools-vscode. In the documentation for that, it says that it will automatically list and allow running the tests from the VS Code Tests Explorer panel. I could never get that to work. Maybe it's a weird bit of configuration I missed. Maybe it's a conflict with some other extension. Maybe it just doesn't work as well as the developers thought it would.

I then added another extension to help, PHPUnit Test Explorer. This is only for the purpose of integrating tests to Tests Explorer. It works well. I can see all tests and run all tests. I can even run the tests while generating a coverage report, and the coverage information will then be displayed elsewhere in VS Code to help see what has coverage and what doesn't, although I have found that it isn't entirely consistent in how often those results show up and on which folders.

Supporting the Tests

Running the unit tests and the kernel tests came together pretty easily to run locally, not needing any extra configuration.

Running the functional tests got more complicated, because my local developer environment is running its command line as root instead of the www-data that is used for the Apache hosting the site. To run functional tests on local requires forcing the tests to run as www-data. I already had a postCreateCommand.sh script in my project that would run when starting up the VS Code developer environment, so it made the most sense to put it in there:

# phpunit functional tests need to run as www-data
echo -e '\nphpunit() {\n    su -s /bin/bash -c "vendor/bin/phpunit $@" www-data\n}' >> /root/.bashrc
source /root/.bashrc
mkdir -p web/sites/simpletest && chmod 777 web/sites/simpletest

That allowed me to run the tests from the command line. Unfortunately that didn't solve the Tests Explorer integration for functional tests. For that, I also needed to add a setting to the devcontainer.json:

"settings": {
  "phpunit.php": "/opt/drupal/scripts/phpunit-wrapper.sh",
}

This tells the Tests Explorer to use that script instead of the default phpunit. That script looks like this, telling it to run the tests as www-data:

#!/bin/bash
# Needed for running phpunit functional tests as www-data
# within Tests Explorer in VS Code with help of an extension.
if [ "$1" = "/opt/drupal/vendor/bin/phpunit" ]; then
  shift
fi
su www-data -s /bin/bash -c "exec /opt/drupal/vendor/bin/phpunit $(printf "'%s' " "$@")"

That's it. The tests now work from the command line and from Tests Explorer.

Running Tests in CI/CD

What if I forget to run the tests before making a commit, though? Running all the tests in certain conditions upon committing or merging to my repository is the best way to be sure that I don't push through a regression. To do that, I need an image that can run the tests, and the GitLab CI file to define in what circumstances to run that job.

phpunit_test:
  stage: test
  image: $CI_REGISTRY_IMAGE/web:$CI_COMMIT_REF_SLUG
  script:
    - cd /opt/drupal
    - mkdir -p /opt/drupal/web/sites/simpletest/browser_output
    - /opt/drupal/vendor/bin/phpunit --coverage-html $CI_PROJECT_DIR/coverage  > phpunit_errors.txt
  artifacts:
    when: always
    paths:
			- coverage/*
      - phpunit_errors.txt
    expire_in: 5 months
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: never
    - if: "$CI_COMMIT_BRANCH =~ /^202/ || $CI_COMMIT_BRANCH =~ /^(dev|staging)$/"

This uses the fully built image, which is necessary to have the phpunit executable added from composer. It keeps an artifact of the reports of errors and coverage for 5 months (because typically I'm doing major updates every 4 months).

Most of note is the rules section. In theory, I would love to run these tests on every commit. That isn't feasible, though, as the functional tests can take an extremely long time to run. Even when I took out the part providing a coverage report, it still took over an hour to run tests on a project that doesn't even have close to full coverage yet. Requiring them to pass before every merge to a release branch (the ones starting with 202) would mean that I would basically only be able to do one or two merges per day with long waits before getting started on the next issue branch. That's a lot of time lost when the changes are a lot smaller. Instead, I decided to strike a middle ground: I would run them after merges into the release branch as well as dev and staging, to make sure that if there are any issues I will have time to come back and fix them well before it gets to production, but I'm not being held up on starting my next issue while waiting for the tests to pass.