Behat Autotesting. BDD Behaviour Driven Development
Behat is a php framework for autotesting your business expectations.
Behaviour-driven development is an “outside-in” methodology. It starts at the outside by identifying business outcomes, and then drills down into the feature set that will achieve those outcomes. What’s in a story by Dan North
A tool to do behavioral testing.
The tests are human-readable text files.
Supports several different drivers (via Mink): Goutte - pulls HTML and analyzes it, Selenium - uses a real web browser, Zombie.js - simulated web browser using NodeJS
Suitable for non coders.
BDD is a development methodology that is based on the Agile workflow. It starts by describing a user story in a structured way using the Gherkin language, and then using the Behat framework to convert the user story in to testable code.
Important to understand is that BDD is more about facilitating efficient communication between the business and development team. BDD is not about testing!
Feature: [title]
In order to [benefit]
As a [role]
I need to [feature]
Scenario: Find add blog post form
Given I am logged in as a user
with the “editor” role
When I follow “Add content”
And I follow “Blog post”
Then I should be on
“/node/add/blog”
TDD vs BDD
In TDD (Test-driven development) you write your unit tests BEFORE writing the code that they are testing. IN BDD Behavior-driven development you write your behavioral tests BEFORE creating the features that they are testing. You are actually testing the features not the code itself. The big difference comes when you realize you are testing what actually business and not the correctness the technical ideas are implemented.
Behat
Behat is the PHP implementation of a BDD framework. It is inspired on Cucumber. It uses the Gherkin language for its test scenarios and is run from the command line.
Feature: Blog posts
In order to offer blog posts to the visitors
As an editor
I need to be able to create blog posts
Scenario: Editor can see blog creation form
Given I am logged in as a user with the "editor" role
When I am at "dashboard"
And I click "Add content"
And I click "Blog post"
Then I should see a "Title" field
And I should see a "Body" field
Scenario: New blog posts are unpublished
Given I am logged in as editor
When I create blog posts
| title | body |
| My title | My body text |
| <script>alert('xss');</script> | <script>alert('xss');</script> |
And I visit "dashboard/unpublished/blog-post"
Then I should see 2 blog posts
And I should not see "publish"
Scenario: Publish a blog post
Given I am logged in as administrator
And there is 1 blog post
And I am on "dashboard/unpublished/blog-post"
Then I should see 1 blog post
When I click on "Publish"
Then I should see "Are you sure you want to publish the blog post?"
When I click on "Publish"
Then I should see "The blog post has been published"
When I am on "dashboard/unpublished/blog-post"
Then I should see 0 blog posts
A Scenario Outline
Feature: < a short description of the feature>
In order to < a business objective>
As a < user role or persona >
I want to < some action that helps me reach the goal >
Scenario: < a short description of a business situation >
Given < a precondition >
And < another precondition >
When < some action >
And < some other action >
Then < a testable outcome >
And < something else we can test >
Setup
The first step is to create a folder in the root of your project that will contain the behat scenarios. Give this a nice descriptive name (eg. “tests”):
$ cd /take/me/to/my/root/folder
$ mkdir tests
In this folder, create two files: behat.yml (containing project configuration for behat), and composer.json (containing the dependencies that need to be installed):
behat.yml
default:
suites:
default:
contexts:
- FeatureContext
- Drupal\DrupalExtension\Context\DrupalContext
- Drupal\DrupalExtension\Context\DrushContext
- Drupal\DrupalExtension\Context\MessageContext
- Drupal\DrupalExtension\Context\MinkContext
extensions:
Behat\MinkExtension:
goutte: ~
selenium2: ~
javascript_session: 'selenium2'
Drupal\DrupalExtension:
blackbox: ~
api_driver: 'drupal'
region_map:
content: "#content"
footer: "#footer"
left header: "#header-left"
right header: "#header-right"
right sidebar: "#column-right"
selectors:
message_selector: '.messages'
error_message_selector: '.messages.messages-error'
success_message_selector: '.messages.messages-status'
composer.json
{
"require": {
"drupal/drupal-extension": "~3.0"
},
"config": {
"bin-dir": "bin/"
}
}
Now install Behat with composer: composer install
Before we can run Behat we need to set up some environment specific configuration such as the base URL and the root path of the Drupal installation. These are stored as a JSON array in the $BEHAT_PARAMS
environment variable. Behat will look for this variable and use that in addition to the settings in the behat.yml
file. Execute the following line, making sure to replace http://localhost
and /var/www/myproject
with your own base URL and root path:
Hard way: manually apply the configuration of the local environment.
$ export BEHAT_PARAMS='{"extensions":{"Behat\\MinkExtension":{"base_url":"http://localhost"},"Drupal\\DrupalExtension":{"drupal":{"drupal_root":"/var/www/myproject"},"subcontexts":{"paths":["/var/www/myproject/sites/all/modules"]}}}}'
You should have listed the full list of available definitions:
behat -dl
Keeping the repository clean
Let’s make sure we never accidentally commit any of these local composer and configuration files. Add the following to your .gitignore (in the root folder of your project):
.gitignore
/tests/bin
/tests/config.local
/tests/vendor
Scaffolding
Execute behat --init
inside your tests/ directory. This will create a skeleton installation:
$ ./bin/behat --init
+d features - place your *.feature files here
+d features/bootstrap - place your context classes here
+f features/bootstrap/FeatureContext.php - place your definitions, transformations and hooks here
If you inspect your folder you will see a bunch of new files. These are placeholders. Behat should be ready and working, even though we have no tests yet. Let’s try it out:
$ ./bin/behat
No scenarios
No steps
0m0.27s (26.79Mb)
If you see the output above, Behat is working! Congratulations!
Now, we’ll test how actually computer needs to see it.
Gherkin
The Gherkin language is a Business Readable, Domain Specific Language that is primarily intended to describe the expected behaviour of an application in a way that is easy to understand for the client, and can also be used to automatically test the described behaviour.
Gherkin is a structured way to describe the expected behaviour of a feature
What does a Gherkin test look like again?
Now let’s add a sample test. These are written in the Gherkin language as explained above. If you need a refresher on how to write test scenarios, Behat can show you a helpful example:
$ ./bin/behat --story-syntax
Testing the homepage
Test scenarios are grouped per "Feature". Every feature is stored in a separate file ending with the *.feature extension. They are saved in the features/ folder. Let's create a test scenario for the "Frontpage" feature:
features/frontpage.feature
Feature: Frontpage
In order to have an overview of the website
As an anonymous user
I want to see relevant information on the frontpage
Scenario: Anonymous user can see the frontpage
Given I am not logged in
When I go to the homepage
Then I should see the welcome text
And I should not see the text "Log out"
You might think: how can Behat understand “I should see the welcome text” ? Excellent reasoning! Let’s run the test and see what Behat thinks of it:
$ ./bin/behat
1 scenario (1 undefined)
4 steps (2 passed, 1 undefined, 1 skipped)
0m1.05s (28.83Mb)
--- FeatureContext has missing steps. Define them with these snippets:
/**
* @Then I should see the welcome text
*/
public function iShouldSeeTheWelcomeText()
{
throw new PendingException();
}
Behat has detected that we need to define the “I should see the welcome text” step. Generating placeholders for missing steps
We can ask Behat to generate placeholder definitions for this missing step by using the --append-snippets
option:
$ ./bin/behat --append-snippets
u features/bootstrap/FeatureContext.php - `I should see the welcome text` definition added
As we can see from the output above the definitions will be placed in the file features/bootstrap/FeatureContext.php
.
Initially it looks like this in features/bootstrap/FeatureContext.php
:
<?php
/**
* @Then I should see the welcome text
*/
public function iShouldSeeTheWelcomeText()
{
throw new PendingException();
}
?>
List all existing steps
In order to easily reuse the step definitions that are already present in the project, you can list the existing step definitions with behat -di
:
$ behat -di
default | Given I am logged in as a(n) :role
| at `FeatureContext::iAmLoggedInAsAn()`
default | When I go to :path
| at `FeatureContext::iGoTo()`
default | When I submit the form
| at `FeatureContext::iSubmitTheForm()`
default | Then I can see the blog post
| at `FeatureContext::iCanSeeTheBlogPost()`
default | Given I am logged in
| at `FeatureContext::iAmLoggedIn()`
default | Then the user :name should have :number followers
| at `FeatureContext::theUserShouldHaveFollowers()`
default | Given the following users:
| at `FeatureContext::theFollowingUsers()`
What does a Gherkin test look like?
Now let’s add a sample test. These are written in the Gherkin language as explained above. If you need a refresher on how to write test scenarios, Behat can show you a helpful example:
$ ./bin/behat --story-syntax
Testing the homepage
Test scenarios are grouped per “Feature”. Every feature is stored in a separate file ending with the *.feature extension. They are saved in the features/ folder. Let’s create a test scenario for the “Frontpage” feature:
features/frontpage.feature
Feature: Frontpage
In order to have an overview of the website
As an anonymous user
I want to see relevant information on the frontpage
Scenario: Anonymous user can see the frontpage
Given I am not logged in
When I go to the homepage
Then I should see the welcome text
And I should not see the text "Log out"
You might think: how can Behat understand "I should see the welcome text" ? Excellent reasoning! Let's run the test and see what Behat thinks of it:
$ ./bin/behat
1 scenario (1 undefined)
4 steps (2 passed, 1 undefined, 1 skipped)
0m1.05s (28.83Mb)
--- FeatureContext has missing steps. Define them with these snippets:
/**
* @Then I should see the welcome text
*/
public function iShouldSeeTheWelcomeText()
{
throw new PendingException();
}
Behat has detected that we need to define the “I should see the welcome text” step. Generating placeholders for missing steps
We can ask Behat to generate placeholder definitions for this missing step by using the –append-snippets option:
$ ./bin/behat --append-snippets
u features/bootstrap/FeatureContext.php - `I should see the welcome text` definition added
As we can see from the output above the definitions will be placed in the file features/bootstrap/FeatureContext.php. Initially it looks like this: features/bootstrap/FeatureContext.php
<?php
/**
* @Then I should see the welcome text
*/
public function iShouldSeeTheWelcomeText()
{
throw new PendingException();
}
?>
When to use a custom context
When working on a larger project it is not feasible to put all step definitions in FeatureContext. This file would become unmanageably long! Luckily we can provide our own contexts. A good guideline is to define a context per feature. Laying the groundwork: PSR-4
We first need to define a namespace where our custom contexts can live. We will follow the Drupal naming standards, and use the namespace \Drupal\myproject\Context. In order for the autoloader to find our files, we need to tell Composer about it. Edit composer.json and add the following section: composer.json
{
...
"autoload": {
"psr-4": {
"Drupal\\myproject\\": "src"
}
}
}
Make sure to replace “myproject” with the name of your project (smile) An example feature for our context
Imagine our client wants to have an overview showing 10 blog posts on their site, along with a pager. The requirements were discussed, and the following scenario was agreed upon and written out in the Jira ticket. First of all we need to copy this scenario into the project, in the features/ folder: features/blog.feature
Feature: Blog
In order to promote the technical knowhow of our department
As a technical lead
I want to blog about our latest technical achievements
@api
Scenario: Blog post overview pager
Given I have 15 blog posts
When I visit the blog overview
Then I should see 10 blog posts
And I should see the next page link
But I should not see the previous page link
When I click the next page link
Then I should see 5 blog posts
And I should see the previous page link
But I should not see the next page link
A home for our custom step definitions
Our test scenario contains several steps that are specific to the blog. Let’s add them to a custom context file which will be called BlogContext. First create the folder structure:
$ mkdir -p src/Context
And in it, create the file containing our custom context. This is a class that implements the \Behat\Behat\Context\Context interface. A number of base classes ship with the Drupal Behat Extension, and it is recommended to extend RawDrupalContext as it contains a number of helpful methods which we can leverage. Most of these steps are quite straightforward to implement, you can find plenty of examples in \Drupal\DrupalExtension\Context\DrupalContext.
src/Context/BlogContext.php
<?php
/**
* @file
* Contains \Drupal\myproject\Context\BlogContext.
*/
namespace Drupal\myproject\Context;
use Drupal\DrupalExtension\Context\RawDrupalContext;
/**
* Provides step definitions for the Blog feature.
*/
class BlogContext extends RawDrupalContext {
/**
* The earliest creation date of a blog post, relative to the current time.
*
* This is defined as -30 days, in seconds.
*/
const MIN_DATE = -2592000;
/**
* Creates n blog posts with random title and content.
*
* @Given (I have ):number blog post(s)
*
* @param int $number
* The number of random blog posts to create.
*/
public function createBlogPosts($number) {
$number = (int) $number;
if ($number < 1) {
throw new \InvalidArgumentException('Invalid number.');
}
for ($i = 0 ; $i < $number ; $i++) {
$blog_post = (object) array(
'type' => 'blog',
'title' => $this->getRandom()->string(),
'body' => $this->getRandom()->string(),
'status' => 1,
// Use a random creation date in the past 7 days.
'created' => date('Y-m-d h:i:s', rand(time() - self::MIN_DATE, time())),
);
$this->nodeCreate($blog_post);
}
}
/**
* @When (I )visit the blog overview
* @When I am on the blog overview
*/
public function iAmOnBlogOverview() {
$this->visitPath('/blog');
}
/**
* @Then I should see :number blog posts
*
* @param int $number
* The number of blog posts that should be seen.
*/
public function assertBlogPostCount($number) {
$this->assertSession()->elementsCount('css', 'article.node--type-blog', intval($number));
}
}
Shared step definitions belong in FeatureContext
Our scenario also has a number of step definitions that are looking for pager elements. These still belong in our default FeatureContext class.
features/bootstrap/FeatureContext.php
<?php
/**
* @file
* Contains \FeatureContext.
*/
use Drupal\DrupalExtension\Context\RawDrupalContext;
use Behat\Behat\Context\SnippetAcceptingContext;
/**
* Defines application features from the specific context.
*/
class FeatureContext extends RawDrupalContext implements SnippetAcceptingContext {
/**
* Initializes context.
*
* Every scenario gets its own context instance.
* You can also pass arbitrary arguments to the
* context constructor through behat.yml.
*/
public function __construct() {
}
/**
* @Then I should see the next page link
*/
public function assertNextPageLinkExists() {
$this->assertSession()->elementExists('css', 'li.pager__item--next');
}
/**
* @Then I should not see the next page link
*/
public function assertNextPageLinkNotExists() {
$this->assertSession()->elementNotExists('css', 'li.pager__item--next');
}
/**
* @Then I should see the previous page link
*/
public function assertPreviousPageLinkExists() {
$this->assertSession()->elementExists('css', 'li.pager__item--previous');
}
/**
* @Then I should not see the previous page link
*/
public function assertPreviousPageLinkNotExists() {
$this->assertSession()->elementNotExists('css', 'li.pager__item--previous');
}
/**
* @When I click the next page link
*/
public function clickNextPageLink() {
$this->getSession()->getPage()->clickLink('Go to next page');
}
}
Before our tests can pass we also need to build the actual functionality: a blog overview with a pager. This is left as an exercise for the reader.
Run it
$ ./bin/behat
Feature: Blog
In order to promote the technical knowhow of our department
As a technical lead
I want to blog about our latest technical achievements
@api @javascript
Scenario: Blog post overview pager # features/blog.feature:7
Given I have 15 blog posts # Drupal\myproject\Context\BlogContext::createBlogPosts()
When I visit the blog overview # Drupal\myproject\Context\BlogContext::iAmOnBlogOverview()
Then I should see 10 blog posts # Drupal\myproject\Context\BlogContext::assertBlogPostCount()
And I should see the next page link # FeatureContext::assertNextPageLinkExists()
And I should not see the previous page link # FeatureContext::assertPreviousPageLinkNotExists()
When I click the next page link # FeatureContext::clickNextPageLink()
Then I should see 5 blog posts # Drupal\myproject\Context\BlogContext::assertBlogPostCount()
And I should see the previous page link # FeatureContext::assertPreviousPageLinkExists()
And I should not see the next page link # FeatureContext::assertNextPageLinkNotExists()
1 scenario (1 passed)
9 steps (9 passed)
0m1.95s (37.89Mb)
All tests passed, great job!
Sharing step definitions across projects
Every project has its own business language, so typically most of its own step definitions are unique to that project. There are however cases where it is interesting to share step definitions between different projects, especially if they are using the same contributed modules.
For example every project that is using the CAS module might benefit from:
@Given I am logged in through CAS
Or projects that are using Organic Groups might need a step such as:
@Given I have the role :role in the group :group
The Drupal Behat Extension provides a mechanism for this subcontexts.
Creating a subcontext
For the full instructions, please refer to the official documentation: Contributed module subcontexts. Here is a short summary:
Create a file in your module folder called MYMODULE.behat.inc.
Inside this file, create a class which extends \Drupal\DrupalExtension\Context\DrupalSubContextBase. In this class you can provide your custom step definitions:
sites/all/modules/contrib/ec_super_search/ec_super_search.behat.inc
<?php
/**
* @file
* Contains \ECSuperSearchSubContext.
*/
use Drupal\DrupalExtension\Context\DrupalSubContextBase;
/**
* Step definitions for the European Commission Super Search module.
*/
class ECSuperSearchSubContext extends DrupalSubContextBase {
/**
* Navigates to the search results page.
*
* @When (I )visit the search results page
* @When (I am )on the search results page
*/
public function iAmOnSearchResults() {
$this->visitPath('/search/results');
}
/**
* Checks that the search results contains a given number of items.
*
* @param int $number
* The number of search results that should be seen.
*
* @Then I should see :number search result(s)
*/
public function assertSearchResultsCount($number) {
$this->assertSession()->elementsCount('css', '.view-ec-super-search-results tbody tr', intval($number));
}
}
In your project, update the behat.yml file to include the paths that you want to scan for files named MODULENAME.behat.inc. All step definitions in these files will become automatically available in all your tests:
behat.yml
extensions:
Drupal\DrupalExtension:
...
subcontexts:
paths:
- "/path/to/drupal/sites/all/modules"
Now go ahead and write tests!
features/search_results.feature
Scenario: Test search results
...
And I visit the search results page
Then I should see 4 search results
Setting up required PHP Storm configuration
Check if your PHP interpreter is correctly configured. You can check that in the preferences under the following path → File | Settings | Languages & Frameworks | PHP
after setting up the PHP interpreter click on the 'i' to check if you have an access to the PHP configuration details. If yes you can proceed to the next step.
the settings that are shown below on the picture are based on a local PHP interpreter. You can also configure a remote PHP interpreter (ex. interpreter installed on the VPS machine)
more information about how to configure interpreters can be found here:
Configuring Local PHP Interpreters
Configuring Remote PHP Interpreters
Setting up PHP
Setting up Behat configuration
point the "path_to_your_project/bin/behat" as the path to Behat directory
if the PHP interpreter is setup correctly and you point to a valid binary you should be able to see the Behat version. You can find an example on the picture below.
you can also specify location of the default .yml Behat configuration file. This is an optional steop as you can also do that later during the set up of the 'run configuration'.
more info can be found here:
Behat
Setting up ‘run configuration’ for the Behat test
To add a custom 'run configuration' you need to click on the dropdown menu in the 'run navigation bar section' and click 'Edit Configurations' (look at the picture below)
more information about setting up 'run/debugging configurations' can be found here:
Creating and Editing Run/Debug Configurations
The next step is clicking the '+' icon and picking 'Behat' option from the list (look at the picture below)
Next thing you need to do is to provide some information regarding the test you want to run
put the name of the configuration
pick the test scope; here the most common option is 'File' or 'Scenario'
'File' - sets run configuration for the given .feature file
'Scenario' - sets run configuration for the given scenario of the selected .feature file
click checkbox 'Use alternative configuration file' and point the path to the Behat .yml configuration on your environment (look at the picture below)
after setting up 'test runner' click the 'Apply' and 'OK' buttons
more information regarding the Behat 'run configuration' can be found here:
Run/Debug Configuration: Behat
Run Behat test using prepared ‘run configuration’
Before running the configuration be sure that selected Behat configuration .yml file is setup for your current local environment (especially selenium port and browser).
To run the prepared configuration select the configuration on the dropdown menu in the 'run navigation bar section' and hit the 'Play' icon (look at the picture below)
After running the configuration you should see a similar output to the one on the picture presented below. If something is not configured you will receive an error message in the console window.
You can click on the given test step which is available in the executed configuration and by doing that you will be redirected to the step in the .feature file.
@api @javascript
Feature: Embedded videos
In order to make my website more attractive
As a contributor
I can embed videos from Youtube, Dailymotion or Vimeo in my content
Background:
Given the module is enabled
| modules |
| embedded_video |
And I am logged in as a user with the 'contributor' role
Scenario Outline: Embed youtube video via media web tab
When I go to "node/add/page"
And I fill in "title" with "<title>"
And I click "Add media"
Then the media browser opens
And I click the "WEB" tab
Then I fill in "File URL or media resource" with "<url>"
And I press "Next"
Then I reach the "Media browser" screen
And the field title is filled with <title>
Then I enter "text" in the "Video Description"
And I press save
Then I press submit
Then the media browser closes
Then I save the node
Then I should see the video with a banner "Please accept <provider> cookies to play this video."
Examples:
|provider | title | url |
|youtube | Interview with Dries Buytaert, founder of Drupal | https://www.youtube.com/watch?v=i8AENFzUTHk |
|dailymotion | drupal-new88 | http://www.dailymotion.com/video/x4gj1bp |
|Vimeo | A successful build in Jenkins |https://vimeo.com/129687265 |
Notes
bin/behat -dl
bin/behat -di #extended info