Running Selenium with Docker During VirtualBox Test Runs

In this video we are continuing on with our GitLab CI Multi Runner VirtualBox setup. We will remove the external dependency on Selenium, by making use of Docker to run a headless version of Selenium on the same Virtual Machine that runs our tests.

There are a few key steps to this process:

• We need some acceptance tests to run
• We need a running instance of Selenium to run against
• We don't want to be downloading the Docker container during our build process
• Docker must be able to communicate back to our Virtual Machine / NGiNX host (via port 80)

The idea here then is to boot up our 'master' image - the Virtual Machine that GitLab CI Multi Runner will clone from during the Build process, and then install and run the Docker container.

From here we can firstly: validate that everything works as expected, and secondly: 'save' the downloaded container so it is 'immediately' available on our clone.

There's a good reason for doing this - the Docker container is about 700mb to download, so I only want to do this once, if I can get away with it. Lord knows, the build already takes long enough as it is (3+ minutes).

Firstly, let's very quickly look at the test suite:

<?php

// /tests/acceptance/GenericCest.php

class GenericCest
{
    public function _before(AcceptanceTester $I)
    {
    }

    public function _after(AcceptanceTester $I)
    {
    }

    // tests
    public function visitLocal(AcceptanceTester $I)
    {
        $I->wantTo('ensure that frontpage works');
        $I->amOnPage('/codereviewvideos');
        $I->see('My Githut');
    }

    public function visitWikipedia(AcceptanceTester $I)
    {
        $I->wantTo('ensure that wikipedia is up');
        $I->amOnUrl('https://en.wikipedia.org/wiki/Main_Pages');
        $I->see('Main pages');
    }
}

There's really not a lot going on here. All I want to check here is that I can access my Symfony project, and that I can also access external websites. These are Codeception acceptance tests by the way, in case you were wondering.

And here is the output of acceptance.suite.yml:

# /tests/acceptance.suite.yml

class_name: AcceptanceTester
modules:
    enabled:
        - WebDriver
        - \Helper\Acceptance
    config:
        WebDriver:
            url: 'http://symfony-3.dev/app_codecept_local.php'
            host: '192.168.1.127'
            port: '4444'
            browser: firefox
            capabilities:
                unexpectedAlertBehaviour: 'accept'

The port of 4444 is the default selenium port, which we will need shortly when setting up our Docker container.

The host is the IP address of my local machine - the machine that until I switch to Docker will be running selenium locally.

If you'd like to try Selenium on your local machine before going into the Docker version then this video should be able to help you further.

Going Docker... Sort Of

GitLab CI is heavily into Docker. My personal qualm with this is that I am not heavily into Docker. I recognise the potential, but up to now I have not had the time to invest in migrating my nicely working Ansible / VM setup over to Docker containers.

However, for the longest time I have wanted to run my acceptance tests as part of my GitLab CI build process. This had completely stumped me until quite recently. Then I hit upon a plan of running Selenium on the same machine I was using as my GitLab CI Multi Runner host.

The problem was though, that selenium would sometimes die. I would infrequently need to log on to the box and restart Selenium. Also, this just felt horrible, and was not a practical solution.

Then, I stumbled upon this blog post where davert of Codeception fame wrote about "Acceptance Testing With No Selenium or PhantomJS Installed".

Amazing.

He kindly provides a docker container that we can simply spin up, and run our acceptance suite against without any real effort at all. The hardest part is figuring out the port combination. But not to worry as I've done that bit for you.

The Plan

As we know from the previous video, GitLab CI Multi Runner with the VirtualBox executor can take a clone of your existing Virtual Machine and use that as a base to run your entire test suite against. This is fantastic.

With the Docker container running a headless version of Selenium (think: no need for a GUI), there is now a possibility of running a self contained Virtual Box that can also run your browser-based acceptance tests. Amazing. Thank you so much davert.

Firstly this means we need to install Docker onto our Virtual Box. This might go wrong, so be sure to take a backup / snapshot of your Virtual Machine before proceeding.

Then, simply follow the instructions here.

Be sure to add the user that your tests will run as to the docker group! This is very important. In my case this command would be:

sudo usermod -aG docker deploy

Once Docker is installed we can then use Docker to pull down the container that davert has created that includes Selenium and all the other requirements to make this work:

docker run -i -t -p 4444:4444 -e APP_PORT=80 davert/selenium-env

Once you run this command for the first time, Docker will pull down the container and run through all the various stages that are required to bring that container up to the point we need it to be at. Now, this takes a while - it's about 700mb, and this is why we want to do this on our master / base image only once, so it is available every time the machine gets cloned during the test / GitLab CI build phase.

Once this command finishes this will run the selenium instance inside your terminal - thanks to the -i flag. I would advise doing it this way as it is infinitely easier to work this way when first working with Docker, in my opinion. The alternative is to run the container in a detached state, but this is more confusing initially. We will do this, however, in our real test script / .gitlab-ci.yml file.

With the inclusion of the -e APP_PORT=80 environment variable, this will allow your Docker container to communicate with your local webserver running on your Virtual Machine - assuming you are using port 80 of course. Change this up if / as needed.

To run the acceptance tests now will require little tweaking of our acceptance.suite.yml file:

# /tests/acceptance.suite.yml

class_name: AcceptanceTester
modules:
    enabled:
        - WebDriver
        - \Helper\Acceptance
    config:
        WebDriver:
            url: 'http://symfony-3.dev/app_codecept_local.php'
            # host: '192.168.1.127'
            port: '4444'
            browser: firefox
            capabilities:
                unexpectedAlertBehaviour: 'accept'

By commenting out the host line, the Codeception suite will now run connect to the local Docker Selenium instance, meaning we no longer need to run an external Selenium instance. Wow.

Getting This To Work With GitLab CI Multi Runner

A change is required in the .gitlab-ci.yml file to make all this work:

before_script:
  - docker run -d -p 4444:4444 -e APP_PORT=80 davert/selenium-env
  - composer install

stages:
  - test

test:
  script:
  - php vendor/bin/codecept run unit
  - php vendor/bin/codecept run acceptance

Here we can see that the docker command will run before anything else. Combined with how long the composer install command will usually take, we can rest assured that our Selenium instance will be up and running by the time we get to the test stage of our build.

This docker command must run in detached mode (-d) or you will get an error, as Docker would effectively take over the entire terminal break your test / build run.

You won't see any output during this process, other than what is logged to the command line.