Simpletest, Drush 8, test-run, and Exit Codes.

If you're using Drush to run your tests, swapping Drush version 6.x out for 8 isn't a simple replacement. First drush test-run has been removed so running Drupal simpletests now requires using the This would be workable, however under Drupal 7, scripts/ does not return correct exit codes so if you're relying on failed tests returning a 1 you are out of luck. While this has been fixed for Drupal 8, if you're running Drupal 7 you're going to have to go an alternate route.

To avoid maintaining a patched core and keeping test running scripts out of the web root I copied the script to bin and applied the exit code patch. This isn't sufficient however because has assumptions about the Drupal directory structure hard coded. Thus a second change is needed to ensure that it knows where the webroot is and how to call itself again to run the next test. I should have made the fix be an argument that you pass in. Sorry.

I combined the exit code patch for D7 from and the directory fixes into this patch:

I'm able to run my tests from outside the Drupal root with: php -C bin/run-tests.php --url http://mysite.local MyTests
If you're using CircleCi you will likely need to specify php directly: php -C bin/run-tests.php --php /home/ubuntu/.phpenv/shims/php --url http://mysite.local MyTests

If you're building your site with composer, it's straight forward to apply the exit code patch with your composer file in the Drupal root with something like this:

  "require": {
    "cweagans/composer-patches": "~1.0",
    "drupal/drupal": "7.0.*@dev"
  "config": {
    "preferred-install": "source"
  "extra": {
    "patches": {
      "drupal/drupal": {
        "Patch to support correct exit codes.": ""