Part 1: Automated Drupal testing with Behat, Selenium, and Headless Firefox

I had a bit of a hard time understanding where one component ended and another started in the context of Behat, browser controlling, and Seleniuem. My hopes are to share the things I've learned and use this for documentation in the future.

Below are the technologies I'm currently using for our automated testing. This documentation is meant to help explain what each of their roles are and how they interact together.

Technologies Involved

• Behat
  • Mink
    • Goutte
    • Selenium2Driver
  • Behat Drupal Extension
• Composer
• Selenium Standalone Server (v.: 2.x) (aka Selenium2)
  • Hub
  • Node
• Xvfb
  • Browser (Firefox)

Behat

Behat's official docs open with: "Behat is an open source Behavior Driven Development framework for PHP 5.3+". My summary is this: it's a testing framework written in PHP that allows a neat syntax for expected functionality scenarios.
Below is an example of Behat's test case syntax:

Scenario: Listing two files in a directory
  Given I am in a directory "test"
  And I have a file named "foo"
  And I have a file named "bar"
  When I run "ls"
  Then I should get:
  """
  bar
  foo
  """

Behat and other contributed extensions provide definitions for parts of the scenario, so it knows how to handle things like I or in a director.

Mink

Because Behat is just a framework, Mink is the part that offers web browser-related functionality. Mink is an abstraction layer for various browser emulators and/or controllers. A browser emulator such as Goutte is capable of handling simple browser emulation (think curl calls). Emulators cannot process Javascript. Therefore, AJAX related testing is not possible. To solve this, Mink can also accommodate browser controller extensions to hand this kind of testing over to an application that can do this, such as Selenium.

I highly recommend reading Mink's At a Glance page. It does a great job of explaining the differences between a browser emulator and browser controller.

Goutte

Goutte is a screen scraping and web crawling library for PHP. Goutte provides a nice API to crawl websites and extract data from the HTML/XML responses.

Behat uses Mink as a wrapper for browser emulators such as Goutte to test page responses. It's much faster than using something that requires a browser instance to startup such as Selenium. It can not process client-side scripting, though, so you'd still need Selenium for Javascript related functionality.

Selenium2Driver

Mink's Selenium2 (webdriver) Driver handles interactions between Mink and Selenium. Selenium is required if we want to test Javascript related functionality. Mink will tell Selenium to spawn a new browser and control the browser to execute the various test case scenarios. In our situation, the Ubuntu machine doesn't have a UX. We'll leverage Xvfb to run Firefox in a dummy display (headless'ish).

Behat Drupal Extension

The Behat Drupal Extension describes Drupal related test scenario language to Behat and Mink. An example would be I am logged in.

Composer

Composer is a tool for dependency management in PHP. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.

Selenium Standalone Server (aka Selenium2)

Because Selenium has gone through several evolving processes, the names of things changed and got a little confusing. Selenium Standalone Server is also called Selenium2 depending on what you're reading.

It can run as a "Hub" or a "Node"; this is important!

Selenium as a Hub

Selenium Standalone Server listens for incoming requests from "drivers". Think of drivers as a communication protocol specific for Selenium. When a request via a driver comes into Selenium Hub, it will pass the process off to available nodes meeting the browser requirements that was specified in the request.

Selenium as a Node

Nodes register themselves to a Selenium Hub. When they register, they advertise their capabilities such as available browsers, their version, and OS. When a request comes into the hub, it gets delegated to the available nodes which will perform the various actions in the browser, emulating a person. Because of the browser instance startup, Selenium driven tests can be kind of slow.

Because nodes register themselves to a hub, they can be across many machines consisting of different operating systems. This is one of the many things that makes Selenium pretty awesome.

Xvfb

Xvfb is an X server that can run on machines with no display hardware and no physical input devices. It emulates a dumb framebuffer using virtual memory. We will leverage Xvfb to have Selenium run Firefox in the background.

Firefox (headless)

We will use the latest stable version of Firefox but have it run in a dummy display via Xvfb to make it "headless".