DrupalWebTest and settings.php: these are not the variables you are looking for.

Short version: if you are testing and have configuration in settings.php you probably want to override the DrupalWebTestCase::refreshVariables() method. 

I've run into a number of potholes navigating Drupal's web test setup. I need to be able to test on an existing database rather than having Drupal build me a test instance every time I want to test. Because I'm mostly doing integration testing I want to be able to easily test against "good" data without having to wait for a data migration. I find that the easier and faster it is to test the more likely that tests will be useful and developers will write more of them. Drupal's built in testing doesn't really work this way- thankfully it's fairly easy to set Drupal up to do this and is documented in lots of places, but for clarity:

class MyTest extends DrupalWebTestCase {

  /** 
   * Force the use of the current database and environment. 
   */
  function setUp() {
    $this->setup = TRUE; 
  }

  /**
   * End testing.
   *
   * Must implement to prevent the deletion of the database on test completion. 
   */
  function tearDown() {
  }

Provide the setUp() method so you can set $this->setup. This is all you need. Well and the tearDown() method- override that too because you don't want to loose your database. So this works well and gives me lots of freedom to test data workflows on "real" content. Nonetheless, there are of course some dark corners. I found that if you want to use the handy tools for logging in users, creating content, etc. that Drupal provides ($this->drupalCreateUser(), $this->drupalGet(), etc.) you're going to get some surprising results:

var_dump(variable_get('my_variable')); 
$user = $this->drupalCreateUser(); 
$this->drupalLogin($user);
var_dump(variable_get('my_variable'));

Yields:

string(14) "/mnt/directory" 
NULL

Wait. What? After calling the testing user/login functions the variable is not set?

To figure out what's going on you need to do a bit of a code dive and you'll follow the thread to calls to refreshVariables(). Here's you're going to see that the all of Drupal's variables are being reloaded with one caveat- variables defined in settings.php aren't going to be available. variable_initialize() gets called but it's with the cached configuration data in the database, not in settings.php.

So if you manage aspects of environmental configuration via settings.php suddenly you've lost your context. This is a problem.

To be fair, I am already sneaking around Drupal's intended setup which tries to manage some aspects of your configuration. One could argue that a site build process should store things via Drupal's variables table which would avoid this problem. I don't personally find that a compelling argument- I think that core site configuration should be explicit and should not be overridable with changes to the database. Of course this depends on what kind of configuration we're actually talking about, but I'll err on the side of "if I put it in my settings.php file, it is always defined that way".

To get around this you can just define your own version of the refreshVariables() method in your test class to prevent the defined data in $conf from being lost:

protected function refreshVariables() {
}

This isn't a great way to do this since the intent of calling this function was to get configuration changes that might have been triggered by a login or other kind of activity. You could reload $conf from settings.php and apply that over what variable_initialize() does but then you're introducing more code to test things that may have unintended concequences. In my case the things I needed to test do not depend on any assumed changes to configuration. I hope.

So moral of the story? It's probably at least two fold- if you're not doing things the way that you're supposed to expect some potholes. If you want to do data integration testing Behat is probably a much safer bet as you're not relying on Drupal to test Drupal... as much.